2024-05-08T08:41:01.269 INFO:root:teuthology version: 0.0.1.dev271+g1ae7ad8 2024-05-08T08:41:01.269 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/teuthology-2024-05-07_21:16:03-rbd-squid-distro-default-smithi/7696006 --description rbd/device/{base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-zlib supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_nbd} --owner scheduled_teuthology@teuthology --name teuthology-2024-05-07_21:16:03-rbd-squid-distro-default-smithi --verbose -- /home/teuthworker/archive/teuthology-2024-05-07_21:16:03-rbd-squid-distro-default-smithi/7696006/orig.config.yaml 2024-05-08T08:41:01.302 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-08T08:41:01.387 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-05-07_21:16:03-rbd-squid-distro-default-smithi/7696006 branch: squid description: rbd/device/{base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-zlib supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_nbd} email: ceph-qa@ceph.com first_in_suite: false job_id: '7696006' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: teuthology-2024-05-07_21:16:03-rbd-squid-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - machine: cpus: 1 disk: 40 ram: 8000 volumes: count: 4 size: 30 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: conf: global: mon client directed command retry: 5 mon warn on pool no app: false ms inject socket failures: 5000 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bluestore block size: 96636764160 bluestore compression algorithm: zlib bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED sha1: 96805dcf909541c4ca9a1e90258de8ae5b6fa1bc ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: extra_packages: - rbd-nbd flavor: default sha1: 96805dcf909541c4ca9a1e90258de8ae5b6fa1bc selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 96805dcf909541c4ca9a1e90258de8ae5b6fa1bc owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mon.c - osd.0 - osd.1 - osd.2 - - mon.b - mgr.x - osd.3 - osd.4 - osd.5 - - client.0 seed: 5498 sha1: 96805dcf909541c4ca9a1e90258de8ae5b6fa1bc sleep_before_teardown: 0 subset: 27/128 suite: rbd suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 96805dcf909541c4ca9a1e90258de8ae5b6fa1bc targets: smithi059.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBg0P9U0pxwjx5v8P3f3SvOuxAdTttS4TvebBYtwUGso7cvHGXe1+hP8GAc1+EfBtvHugZXj2yavJ6Kga6/1I0U= smithi192.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPhSindb4GTLW2TymcdFVVhhzFZ4VL+z8n7VGax3t1HrAxAVrf4J7odZeY6swp7OU7YaYSq+rxneF5yxJ4lM2ws= smithi196.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCxyjPTl3acb93N2AVX4US1bTo3wpLdIdsHd1POcNcRuWWfAh3Tz4pIg0lFHA7gZe4r5KoM2Ynw9ODfCF4KT9qE= tasks: - install: null - ceph: null - thrashosds: timeout: 1200 - rbd_fsx: clients: - client.0 holebdy: 512 nbd: true ops: 6000 punch_holes: true readbdy: 512 truncbdy: 512 writebdy: 512 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 1ae7ad82388e92a475afff437d49054826c019a1 timestamp: 2024-05-07_21:16:03 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-05-08T08:41:01.387 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/qa; will attempt to use it 2024-05-08T08:41:01.388 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/qa/tasks 2024-05-08T08:41:01.388 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-08T08:41:01.390 INFO:teuthology.task.internal:Checking packages... 2024-05-08T08:41:01.413 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc' 2024-05-08T08:41:01.413 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-08T08:41:01.413 INFO:teuthology.packaging:ref: None 2024-05-08T08:41:01.413 INFO:teuthology.packaging:tag: None 2024-05-08T08:41:01.413 INFO:teuthology.packaging:branch: squid 2024-05-08T08:41:01.413 INFO:teuthology.packaging:sha1: 96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-08T08:41:01.414 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-05-08T08:41:01.736 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2515.g96805dcf 2024-05-08T08:41:01.737 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-08T08:41:01.744 INFO:teuthology.task.internal:no buildpackages task found 2024-05-08T08:41:01.745 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-08T08:41:01.858 INFO:teuthology.task.internal:Saving configuration 2024-05-08T08:41:01.871 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-08T08:41:01.877 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-08T08:41:01.902 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi059.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-05-07_21:16:03-rbd-squid-distro-default-smithi/7696006', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-08 08:35:48.896931', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBg0P9U0pxwjx5v8P3f3SvOuxAdTttS4TvebBYtwUGso7cvHGXe1+hP8GAc1+EfBtvHugZXj2yavJ6Kga6/1I0U='} 2024-05-08T08:41:01.925 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi192.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-05-07_21:16:03-rbd-squid-distro-default-smithi/7696006', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-08 08:35:48.898636', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPhSindb4GTLW2TymcdFVVhhzFZ4VL+z8n7VGax3t1HrAxAVrf4J7odZeY6swp7OU7YaYSq+rxneF5yxJ4lM2ws='} 2024-05-08T08:41:01.945 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi196.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-05-07_21:16:03-rbd-squid-distro-default-smithi/7696006', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-08 08:35:48.894963', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCxyjPTl3acb93N2AVX4US1bTo3wpLdIdsHd1POcNcRuWWfAh3Tz4pIg0lFHA7gZe4r5KoM2Ynw9ODfCF4KT9qE='} 2024-05-08T08:41:01.945 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-08T08:41:01.952 INFO:teuthology.task.internal:roles: ubuntu@smithi059.front.sepia.ceph.com - ['mon.a', 'mon.c', 'osd.0', 'osd.1', 'osd.2'] 2024-05-08T08:41:01.952 INFO:teuthology.task.internal:roles: ubuntu@smithi192.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.3', 'osd.4', 'osd.5'] 2024-05-08T08:41:01.952 INFO:teuthology.task.internal:roles: ubuntu@smithi196.front.sepia.ceph.com - ['client.0'] 2024-05-08T08:41:01.952 INFO:teuthology.run_tasks:Running task console_log... 2024-05-08T08:41:02.053 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fcb073a5e50>, signals=[15]) 2024-05-08T08:41:02.054 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-08T08:41:02.060 INFO:teuthology.task.internal:Opening connections... 2024-05-08T08:41:02.060 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi059.front.sepia.ceph.com 2024-05-08T08:41:02.062 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T08:41:02.143 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi192.front.sepia.ceph.com 2024-05-08T08:41:02.144 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T08:41:02.220 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi196.front.sepia.ceph.com 2024-05-08T08:41:02.221 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T08:41:02.302 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-08T08:41:02.309 DEBUG:teuthology.orchestra.run.smithi059:> uname -m 2024-05-08T08:41:02.328 INFO:teuthology.orchestra.run.smithi059.stdout:x86_64 2024-05-08T08:41:02.328 DEBUG:teuthology.orchestra.run.smithi059:> cat /etc/os-release 2024-05-08T08:41:02.386 INFO:teuthology.orchestra.run.smithi059.stdout:NAME="CentOS Stream" 2024-05-08T08:41:02.386 INFO:teuthology.orchestra.run.smithi059.stdout:VERSION="9" 2024-05-08T08:41:02.386 INFO:teuthology.orchestra.run.smithi059.stdout:ID="centos" 2024-05-08T08:41:02.386 INFO:teuthology.orchestra.run.smithi059.stdout:ID_LIKE="rhel fedora" 2024-05-08T08:41:02.386 INFO:teuthology.orchestra.run.smithi059.stdout:VERSION_ID="9" 2024-05-08T08:41:02.386 INFO:teuthology.orchestra.run.smithi059.stdout:PLATFORM_ID="platform:el9" 2024-05-08T08:41:02.386 INFO:teuthology.orchestra.run.smithi059.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-08T08:41:02.386 INFO:teuthology.orchestra.run.smithi059.stdout:ANSI_COLOR="0;31" 2024-05-08T08:41:02.386 INFO:teuthology.orchestra.run.smithi059.stdout:LOGO="fedora-logo-icon" 2024-05-08T08:41:02.386 INFO:teuthology.orchestra.run.smithi059.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-08T08:41:02.386 INFO:teuthology.orchestra.run.smithi059.stdout:HOME_URL="https://centos.org/" 2024-05-08T08:41:02.387 INFO:teuthology.orchestra.run.smithi059.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-08T08:41:02.387 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-08T08:41:02.387 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-08T08:41:02.387 INFO:teuthology.lock.ops:Updating smithi059.front.sepia.ceph.com on lock server 2024-05-08T08:41:02.415 DEBUG:teuthology.orchestra.run.smithi192:> uname -m 2024-05-08T08:41:02.433 INFO:teuthology.orchestra.run.smithi192.stdout:x86_64 2024-05-08T08:41:02.433 DEBUG:teuthology.orchestra.run.smithi192:> cat /etc/os-release 2024-05-08T08:41:02.490 INFO:teuthology.orchestra.run.smithi192.stdout:NAME="CentOS Stream" 2024-05-08T08:41:02.490 INFO:teuthology.orchestra.run.smithi192.stdout:VERSION="9" 2024-05-08T08:41:02.490 INFO:teuthology.orchestra.run.smithi192.stdout:ID="centos" 2024-05-08T08:41:02.490 INFO:teuthology.orchestra.run.smithi192.stdout:ID_LIKE="rhel fedora" 2024-05-08T08:41:02.490 INFO:teuthology.orchestra.run.smithi192.stdout:VERSION_ID="9" 2024-05-08T08:41:02.490 INFO:teuthology.orchestra.run.smithi192.stdout:PLATFORM_ID="platform:el9" 2024-05-08T08:41:02.490 INFO:teuthology.orchestra.run.smithi192.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-08T08:41:02.491 INFO:teuthology.orchestra.run.smithi192.stdout:ANSI_COLOR="0;31" 2024-05-08T08:41:02.491 INFO:teuthology.orchestra.run.smithi192.stdout:LOGO="fedora-logo-icon" 2024-05-08T08:41:02.491 INFO:teuthology.orchestra.run.smithi192.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-08T08:41:02.491 INFO:teuthology.orchestra.run.smithi192.stdout:HOME_URL="https://centos.org/" 2024-05-08T08:41:02.491 INFO:teuthology.orchestra.run.smithi192.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-08T08:41:02.491 INFO:teuthology.orchestra.run.smithi192.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-08T08:41:02.491 INFO:teuthology.orchestra.run.smithi192.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-08T08:41:02.491 INFO:teuthology.lock.ops:Updating smithi192.front.sepia.ceph.com on lock server 2024-05-08T08:41:02.515 DEBUG:teuthology.orchestra.run.smithi196:> uname -m 2024-05-08T08:41:02.533 INFO:teuthology.orchestra.run.smithi196.stdout:x86_64 2024-05-08T08:41:02.533 DEBUG:teuthology.orchestra.run.smithi196:> cat /etc/os-release 2024-05-08T08:41:02.590 INFO:teuthology.orchestra.run.smithi196.stdout:NAME="CentOS Stream" 2024-05-08T08:41:02.590 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION="9" 2024-05-08T08:41:02.590 INFO:teuthology.orchestra.run.smithi196.stdout:ID="centos" 2024-05-08T08:41:02.590 INFO:teuthology.orchestra.run.smithi196.stdout:ID_LIKE="rhel fedora" 2024-05-08T08:41:02.590 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION_ID="9" 2024-05-08T08:41:02.590 INFO:teuthology.orchestra.run.smithi196.stdout:PLATFORM_ID="platform:el9" 2024-05-08T08:41:02.591 INFO:teuthology.orchestra.run.smithi196.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-08T08:41:02.591 INFO:teuthology.orchestra.run.smithi196.stdout:ANSI_COLOR="0;31" 2024-05-08T08:41:02.591 INFO:teuthology.orchestra.run.smithi196.stdout:LOGO="fedora-logo-icon" 2024-05-08T08:41:02.591 INFO:teuthology.orchestra.run.smithi196.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-08T08:41:02.591 INFO:teuthology.orchestra.run.smithi196.stdout:HOME_URL="https://centos.org/" 2024-05-08T08:41:02.591 INFO:teuthology.orchestra.run.smithi196.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-08T08:41:02.591 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-08T08:41:02.592 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-08T08:41:02.592 INFO:teuthology.lock.ops:Updating smithi196.front.sepia.ceph.com on lock server 2024-05-08T08:41:02.615 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-08T08:41:02.624 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-08T08:41:02.630 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-08T08:41:02.630 DEBUG:teuthology.orchestra.run.smithi059:> test '!' -e /home/ubuntu/cephtest 2024-05-08T08:41:02.633 DEBUG:teuthology.orchestra.run.smithi192:> test '!' -e /home/ubuntu/cephtest 2024-05-08T08:41:02.635 DEBUG:teuthology.orchestra.run.smithi196:> test '!' -e /home/ubuntu/cephtest 2024-05-08T08:41:02.651 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-08T08:41:02.657 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-08T08:41:02.658 DEBUG:teuthology.orchestra.run.smithi059:> test -z $(ls -A /var/lib/ceph) 2024-05-08T08:41:02.690 DEBUG:teuthology.orchestra.run.smithi192:> test -z $(ls -A /var/lib/ceph) 2024-05-08T08:41:02.693 DEBUG:teuthology.orchestra.run.smithi196:> test -z $(ls -A /var/lib/ceph) 2024-05-08T08:41:02.720 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-08T08:41:02.796 INFO:teuthology.run_tasks:Running task kernel... 2024-05-08T08:41:02.810 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-05-08T08:41:02.810 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}, 'client.0': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-05-08T08:41:02.810 DEBUG:teuthology.orchestra.run.smithi059:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-08T08:41:02.811 DEBUG:teuthology.orchestra.run.smithi192:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-08T08:41:02.811 DEBUG:teuthology.orchestra.run.smithi196:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-08T08:41:02.827 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T08:41:02.827 DEBUG:teuthology.orchestra.run.smithi196:> uname -r 2024-05-08T08:41:02.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T08:41:02.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T08:41:02.829 DEBUG:teuthology.orchestra.run.smithi059:> uname -r 2024-05-08T08:41:02.830 DEBUG:teuthology.orchestra.run.smithi192:> uname -r 2024-05-08T08:41:02.884 INFO:teuthology.orchestra.run.smithi196.stdout:5.14.0-437.el9.x86_64 2024-05-08T08:41:02.884 INFO:teuthology.task.kernel:Running kernel on smithi196: 5.14.0-437.el9.x86_64 2024-05-08T08:41:02.884 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum install -y kernel 2024-05-08T08:41:02.886 INFO:teuthology.orchestra.run.smithi192.stdout:5.14.0-437.el9.x86_64 2024-05-08T08:41:02.887 INFO:teuthology.task.kernel:Running kernel on smithi192: 5.14.0-437.el9.x86_64 2024-05-08T08:41:02.887 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum install -y kernel 2024-05-08T08:41:02.888 INFO:teuthology.orchestra.run.smithi059.stdout:5.14.0-437.el9.x86_64 2024-05-08T08:41:02.888 INFO:teuthology.task.kernel:Running kernel on smithi059: 5.14.0-437.el9.x86_64 2024-05-08T08:41:02.888 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum install -y kernel 2024-05-08T08:41:05.310 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - BaseOS 9.6 MB/s | 8.1 MB 00:00 2024-05-08T08:41:05.833 INFO:teuthology.orchestra.run.smithi059.stdout:CentOS Stream 9 - BaseOS 6.0 MB/s | 8.1 MB 00:01 2024-05-08T08:41:07.186 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - AppStream 24 MB/s | 19 MB 00:00 2024-05-08T08:41:09.193 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - BaseOS 1.7 MB/s | 8.1 MB 00:04 2024-05-08T08:41:10.517 INFO:teuthology.orchestra.run.smithi059.stdout:CentOS Stream 9 - AppStream 5.4 MB/s | 19 MB 00:03 2024-05-08T08:41:11.670 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - CRB 7.0 MB/s | 6.2 MB 00:00 2024-05-08T08:41:12.643 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - AppStream 8.1 MB/s | 19 MB 00:02 2024-05-08T08:41:13.096 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - Extras packages 62 kB/s | 16 kB 00:00 2024-05-08T08:41:15.158 INFO:teuthology.orchestra.run.smithi192.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-05-08T08:41:15.225 INFO:teuthology.orchestra.run.smithi059.stdout:CentOS Stream 9 - CRB 6.4 MB/s | 6.2 MB 00:00 2024-05-08T08:41:16.670 INFO:teuthology.orchestra.run.smithi059.stdout:CentOS Stream 9 - Extras packages 67 kB/s | 16 kB 00:00 2024-05-08T08:41:17.094 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - CRB 7.1 MB/s | 6.2 MB 00:00 2024-05-08T08:41:18.088 INFO:teuthology.orchestra.run.smithi059.stdout:Extra Packages for Enterprise Linux 17 MB/s | 21 MB 00:01 2024-05-08T08:41:18.661 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - Extras packages 41 kB/s | 16 kB 00:00 2024-05-08T08:41:19.687 INFO:teuthology.orchestra.run.smithi196.stdout:Extra Packages for Enterprise Linux 24 MB/s | 21 MB 00:00 2024-05-08T08:41:20.247 INFO:teuthology.orchestra.run.smithi192.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-05-08T08:41:22.253 INFO:teuthology.orchestra.run.smithi192.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-08T08:41:22.253 INFO:teuthology.orchestra.run.smithi192.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-08T08:41:22.353 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-05-08T08:41:22.361 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-05-08T08:41:22.362 INFO:teuthology.orchestra.run.smithi192.stdout: Package Architecture Version Repository Size 2024-05-08T08:41:22.362 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-05-08T08:41:22.362 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-05-08T08:41:22.362 INFO:teuthology.orchestra.run.smithi192.stdout: kernel x86_64 5.14.0-444.el9 baseos 484 k 2024-05-08T08:41:22.362 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2024-05-08T08:41:22.362 INFO:teuthology.orchestra.run.smithi192.stdout: kernel-core x86_64 5.14.0-444.el9 baseos 15 M 2024-05-08T08:41:22.362 INFO:teuthology.orchestra.run.smithi192.stdout: kernel-modules x86_64 5.14.0-444.el9 baseos 34 M 2024-05-08T08:41:22.362 INFO:teuthology.orchestra.run.smithi192.stdout: kernel-modules-core x86_64 5.14.0-444.el9 baseos 28 M 2024-05-08T08:41:22.362 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:41:22.362 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-05-08T08:41:22.362 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-05-08T08:41:22.362 INFO:teuthology.orchestra.run.smithi192.stdout:Install 4 Packages 2024-05-08T08:41:22.362 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:41:22.363 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 78 M 2024-05-08T08:41:22.363 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 123 M 2024-05-08T08:41:22.363 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-05-08T08:41:22.731 INFO:teuthology.orchestra.run.smithi192.stdout:(1/4): kernel-5.14.0-444.el9.x86_64.rpm 1.4 MB/s | 484 kB 00:00 2024-05-08T08:41:23.386 INFO:teuthology.orchestra.run.smithi059.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-05-08T08:41:24.770 INFO:teuthology.orchestra.run.smithi196.stdout:lab-extras 50 kB/s | 1.7 kB 00:00 2024-05-08T08:41:25.501 INFO:teuthology.orchestra.run.smithi059.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-08T08:41:25.501 INFO:teuthology.orchestra.run.smithi059.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-08T08:41:25.585 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies resolved. 2024-05-08T08:41:25.602 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-05-08T08:41:25.602 INFO:teuthology.orchestra.run.smithi059.stdout: Package Architecture Version Repository Size 2024-05-08T08:41:25.603 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-05-08T08:41:25.603 INFO:teuthology.orchestra.run.smithi059.stdout:Installing: 2024-05-08T08:41:25.603 INFO:teuthology.orchestra.run.smithi059.stdout: kernel x86_64 5.14.0-444.el9 baseos 484 k 2024-05-08T08:41:25.603 INFO:teuthology.orchestra.run.smithi059.stdout:Installing dependencies: 2024-05-08T08:41:25.603 INFO:teuthology.orchestra.run.smithi059.stdout: kernel-core x86_64 5.14.0-444.el9 baseos 15 M 2024-05-08T08:41:25.603 INFO:teuthology.orchestra.run.smithi059.stdout: kernel-modules x86_64 5.14.0-444.el9 baseos 34 M 2024-05-08T08:41:25.603 INFO:teuthology.orchestra.run.smithi059.stdout: kernel-modules-core x86_64 5.14.0-444.el9 baseos 28 M 2024-05-08T08:41:25.603 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:41:25.603 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction Summary 2024-05-08T08:41:25.603 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-05-08T08:41:25.603 INFO:teuthology.orchestra.run.smithi059.stdout:Install 4 Packages 2024-05-08T08:41:25.604 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:41:25.604 INFO:teuthology.orchestra.run.smithi059.stdout:Total download size: 78 M 2024-05-08T08:41:25.604 INFO:teuthology.orchestra.run.smithi059.stdout:Installed size: 123 M 2024-05-08T08:41:25.604 INFO:teuthology.orchestra.run.smithi059.stdout:Downloading Packages: 2024-05-08T08:41:25.606 INFO:teuthology.orchestra.run.smithi192.stdout:(2/4): kernel-core-5.14.0-444.el9.x86_64.rpm 4.8 MB/s | 15 MB 00:03 2024-05-08T08:41:25.946 INFO:teuthology.orchestra.run.smithi059.stdout:(1/4): kernel-5.14.0-444.el9.x86_64.rpm 2.1 MB/s | 484 kB 00:00 2024-05-08T08:41:26.760 INFO:teuthology.orchestra.run.smithi196.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-08T08:41:26.760 INFO:teuthology.orchestra.run.smithi196.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-08T08:41:26.860 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-05-08T08:41:26.870 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-05-08T08:41:26.870 INFO:teuthology.orchestra.run.smithi196.stdout: Package Architecture Version Repository Size 2024-05-08T08:41:26.870 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-05-08T08:41:26.871 INFO:teuthology.orchestra.run.smithi196.stdout:Installing: 2024-05-08T08:41:26.871 INFO:teuthology.orchestra.run.smithi196.stdout: kernel x86_64 5.14.0-444.el9 baseos 484 k 2024-05-08T08:41:26.871 INFO:teuthology.orchestra.run.smithi196.stdout:Installing dependencies: 2024-05-08T08:41:26.871 INFO:teuthology.orchestra.run.smithi196.stdout: kernel-core x86_64 5.14.0-444.el9 baseos 15 M 2024-05-08T08:41:26.871 INFO:teuthology.orchestra.run.smithi196.stdout: kernel-modules x86_64 5.14.0-444.el9 baseos 34 M 2024-05-08T08:41:26.871 INFO:teuthology.orchestra.run.smithi196.stdout: kernel-modules-core x86_64 5.14.0-444.el9 baseos 28 M 2024-05-08T08:41:26.871 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:41:26.871 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-05-08T08:41:26.871 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-05-08T08:41:26.871 INFO:teuthology.orchestra.run.smithi196.stdout:Install 4 Packages 2024-05-08T08:41:26.871 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:41:26.872 INFO:teuthology.orchestra.run.smithi196.stdout:Total download size: 78 M 2024-05-08T08:41:26.872 INFO:teuthology.orchestra.run.smithi196.stdout:Installed size: 123 M 2024-05-08T08:41:26.872 INFO:teuthology.orchestra.run.smithi196.stdout:Downloading Packages: 2024-05-08T08:41:27.388 INFO:teuthology.orchestra.run.smithi196.stdout:(1/4): kernel-5.14.0-444.el9.x86_64.rpm 2.0 MB/s | 484 kB 00:00 2024-05-08T08:41:27.706 INFO:teuthology.orchestra.run.smithi192.stdout:(3/4): kernel-modules-core-5.14.0-444.el9.x86_6 5.6 MB/s | 28 MB 00:04 2024-05-08T08:41:28.322 INFO:teuthology.orchestra.run.smithi059.stdout:(2/4): kernel-core-5.14.0-444.el9.x86_64.rpm 5.9 MB/s | 15 MB 00:02 2024-05-08T08:41:30.031 INFO:teuthology.orchestra.run.smithi192.stdout:(4/4): kernel-modules-5.14.0-444.el9.x86_64.rpm 4.4 MB/s | 34 MB 00:07 2024-05-08T08:41:30.031 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:41:30.032 INFO:teuthology.orchestra.run.smithi192.stdout:Total 10 MB/s | 78 MB 00:07 2024-05-08T08:41:30.155 INFO:teuthology.orchestra.run.smithi196.stdout:(2/4): kernel-core-5.14.0-444.el9.x86_64.rpm 5.1 MB/s | 15 MB 00:03 2024-05-08T08:41:30.709 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-05-08T08:41:30.988 INFO:teuthology.orchestra.run.smithi059.stdout:(3/4): kernel-modules-core-5.14.0-444.el9.x86_6 5.5 MB/s | 28 MB 00:05 2024-05-08T08:41:31.038 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-05-08T08:41:31.038 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-05-08T08:41:31.688 INFO:teuthology.orchestra.run.smithi196.stdout:(3/4): kernel-modules-core-5.14.0-444.el9.x86_6 6.5 MB/s | 28 MB 00:04 2024-05-08T08:41:31.957 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-05-08T08:41:31.957 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-05-08T08:41:32.204 INFO:teuthology.orchestra.run.smithi059.stdout:(4/4): kernel-modules-5.14.0-444.el9.x86_64.rpm 5.2 MB/s | 34 MB 00:06 2024-05-08T08:41:32.205 INFO:teuthology.orchestra.run.smithi059.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:41:32.205 INFO:teuthology.orchestra.run.smithi059.stdout:Total 12 MB/s | 78 MB 00:06 2024-05-08T08:41:32.913 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction check 2024-05-08T08:41:33.055 INFO:teuthology.orchestra.run.smithi196.stdout:(4/4): kernel-modules-5.14.0-444.el9.x86_64.rpm 5.7 MB/s | 34 MB 00:05 2024-05-08T08:41:33.055 INFO:teuthology.orchestra.run.smithi196.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:41:33.055 INFO:teuthology.orchestra.run.smithi196.stdout:Total 13 MB/s | 78 MB 00:06 2024-05-08T08:41:33.352 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction check succeeded. 2024-05-08T08:41:33.352 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction test 2024-05-08T08:41:33.739 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction check 2024-05-08T08:41:34.068 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction check succeeded. 2024-05-08T08:41:34.068 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction test 2024-05-08T08:41:34.291 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction test succeeded. 2024-05-08T08:41:34.292 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction 2024-05-08T08:41:34.380 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-05-08T08:41:35.003 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction test succeeded. 2024-05-08T08:41:35.003 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction 2024-05-08T08:41:35.089 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : kernel-modules-core-5.14.0-444.el9.x86_64 1/4 2024-05-08T08:41:35.173 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-08T08:41:36.845 INFO:teuthology.orchestra.run.smithi059.stdout: Preparing : 1/1 2024-05-08T08:41:37.406 INFO:teuthology.orchestra.run.smithi196.stdout: Preparing : 1/1 2024-05-08T08:41:37.521 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : kernel-modules-core-5.14.0-444.el9.x86_64 1/4 2024-05-08T08:41:37.604 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-08T08:41:37.854 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-08T08:41:37.977 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-08T08:41:38.102 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : kernel-modules-core-5.14.0-444.el9.x86_64 1/4 2024-05-08T08:41:38.188 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-08T08:41:40.418 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-08T08:41:40.548 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-08T08:41:40.833 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-08T08:41:40.963 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-08T08:41:42.707 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-08T08:41:42.838 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : kernel-5.14.0-444.el9.x86_64 4/4 2024-05-08T08:41:45.620 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-08T08:41:45.659 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : kernel-5.14.0-444.el9.x86_64 4/4 2024-05-08T08:41:45.694 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-08T08:41:45.848 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : kernel-5.14.0-444.el9.x86_64 4/4 2024-05-08T08:41:47.571 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-08T08:41:50.589 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-08T08:41:50.753 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-08T08:42:17.284 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 4/4 2024-05-08T08:42:17.322 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 4/4 2024-05-08T08:42:17.703 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: kernel-5.14.0-444.el9.x86_64 4/4 2024-05-08T08:42:17.703 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : kernel-5.14.0-444.el9.x86_64 1/4 2024-05-08T08:42:17.703 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-08T08:42:17.703 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-08T08:42:18.549 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-08T08:42:18.550 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:42:18.550 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-05-08T08:42:18.550 INFO:teuthology.orchestra.run.smithi192.stdout: kernel-5.14.0-444.el9.x86_64 2024-05-08T08:42:18.550 INFO:teuthology.orchestra.run.smithi192.stdout: kernel-core-5.14.0-444.el9.x86_64 2024-05-08T08:42:18.550 INFO:teuthology.orchestra.run.smithi192.stdout: kernel-modules-5.14.0-444.el9.x86_64 2024-05-08T08:42:18.550 INFO:teuthology.orchestra.run.smithi192.stdout: kernel-modules-core-5.14.0-444.el9.x86_64 2024-05-08T08:42:18.550 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:42:18.550 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-05-08T08:42:18.898 DEBUG:teuthology.orchestra.run.smithi192:> rpm -q kernel | sort -rV | head -n 1 2024-05-08T08:42:18.967 INFO:teuthology.orchestra.run.smithi192.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-08T08:42:18.968 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-08T08:42:18.968 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-444.el9.x86_64 2024-05-08T08:42:18.968 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-08T08:42:18.968 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-08T08:42:18.968 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-05-08T08:42:18.968 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi192.front.sepia.ceph.com, path=None, version=distro) 2024-05-08T08:42:18.969 DEBUG:teuthology.orchestra.run.smithi192:> rpm -q kernel | sort -rV | head -n 1 2024-05-08T08:42:19.017 INFO:teuthology.orchestra.run.smithi192.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-08T08:42:19.018 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-08T08:42:19.018 DEBUG:teuthology.orchestra.run.smithi192:> sudo rpm -qi grub2-tools 2024-05-08T08:42:19.048 INFO:teuthology.orchestra.run.smithi192.stdout:Name : grub2-tools 2024-05-08T08:42:19.048 INFO:teuthology.orchestra.run.smithi192.stdout:Epoch : 1 2024-05-08T08:42:19.048 INFO:teuthology.orchestra.run.smithi192.stdout:Version : 2.06 2024-05-08T08:42:19.048 INFO:teuthology.orchestra.run.smithi192.stdout:Release : 68.el9 2024-05-08T08:42:19.049 INFO:teuthology.orchestra.run.smithi192.stdout:Architecture: x86_64 2024-05-08T08:42:19.049 INFO:teuthology.orchestra.run.smithi192.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-08T08:42:19.049 INFO:teuthology.orchestra.run.smithi192.stdout:Group : Unspecified 2024-05-08T08:42:19.049 INFO:teuthology.orchestra.run.smithi192.stdout:Size : 8264254 2024-05-08T08:42:19.049 INFO:teuthology.orchestra.run.smithi192.stdout:License : GPLv3+ 2024-05-08T08:42:19.049 INFO:teuthology.orchestra.run.smithi192.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-08T08:42:19.049 INFO:teuthology.orchestra.run.smithi192.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-08T08:42:19.049 INFO:teuthology.orchestra.run.smithi192.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-08T08:42:19.049 INFO:teuthology.orchestra.run.smithi192.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-08T08:42:19.049 INFO:teuthology.orchestra.run.smithi192.stdout:Packager : builder@centos.org 2024-05-08T08:42:19.049 INFO:teuthology.orchestra.run.smithi192.stdout:Vendor : CentOS 2024-05-08T08:42:19.049 INFO:teuthology.orchestra.run.smithi192.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-08T08:42:19.049 INFO:teuthology.orchestra.run.smithi192.stdout:Summary : Support tools for GRUB. 2024-05-08T08:42:19.049 INFO:teuthology.orchestra.run.smithi192.stdout:Description : 2024-05-08T08:42:19.050 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:42:19.050 INFO:teuthology.orchestra.run.smithi192.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-08T08:42:19.050 INFO:teuthology.orchestra.run.smithi192.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-08T08:42:19.050 INFO:teuthology.orchestra.run.smithi192.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-08T08:42:19.050 INFO:teuthology.orchestra.run.smithi192.stdout:hardware devices. 2024-05-08T08:42:19.050 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:42:19.050 INFO:teuthology.orchestra.run.smithi192.stdout:This subpackage provides tools for support of all platforms. 2024-05-08T08:42:19.051 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-08T08:42:19.051 INFO:teuthology.task.kernel:Updating grub on smithi192 to boot 5.14.0-444.el9.x86_64 2024-05-08T08:42:19.051 DEBUG:teuthology.orchestra.run.smithi192:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-08T08:42:19.603 INFO:teuthology.orchestra.run.smithi192.stderr:Generating grub configuration file ... 2024-05-08T08:42:20.024 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 4/4 2024-05-08T08:42:20.062 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 4/4 2024-05-08T08:42:20.372 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: kernel-5.14.0-444.el9.x86_64 4/4 2024-05-08T08:42:20.372 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : kernel-5.14.0-444.el9.x86_64 1/4 2024-05-08T08:42:20.372 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-08T08:42:20.372 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-08T08:42:20.933 INFO:teuthology.orchestra.run.smithi192.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-08T08:42:20.975 INFO:teuthology.orchestra.run.smithi192.stderr:done 2024-05-08T08:42:20.977 DEBUG:teuthology.orchestra.run.smithi192:> mktemp 2024-05-08T08:42:20.995 INFO:teuthology.orchestra.run.smithi192.stdout:/tmp/tmp.VmvQyEHFpV 2024-05-08T08:42:20.995 DEBUG:teuthology.orchestra.run.smithi192:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.VmvQyEHFpV 2024-05-08T08:42:21.061 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 0666 /tmp/tmp.VmvQyEHFpV 2024-05-08T08:42:21.183 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-08T08:42:21.183 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:42:21.183 INFO:teuthology.orchestra.run.smithi196.stdout:Installed: 2024-05-08T08:42:21.183 INFO:teuthology.orchestra.run.smithi196.stdout: kernel-5.14.0-444.el9.x86_64 2024-05-08T08:42:21.183 INFO:teuthology.orchestra.run.smithi196.stdout: kernel-core-5.14.0-444.el9.x86_64 2024-05-08T08:42:21.183 INFO:teuthology.orchestra.run.smithi196.stdout: kernel-modules-5.14.0-444.el9.x86_64 2024-05-08T08:42:21.183 INFO:teuthology.orchestra.run.smithi196.stdout: kernel-modules-core-5.14.0-444.el9.x86_64 2024-05-08T08:42:21.183 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:42:21.184 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-05-08T08:42:21.196 DEBUG:teuthology.orchestra.remote:smithi192:/tmp/tmp.VmvQyEHFpV is 6KB 2024-05-08T08:42:21.245 DEBUG:teuthology.orchestra.run.smithi192:> rm -fr /tmp/tmp.VmvQyEHFpV 2024-05-08T08:42:21.259 DEBUG:teuthology.orchestra.run.smithi192:> sudo /bin/ls /boot/loader/entries || true 2024-05-08T08:42:21.320 INFO:teuthology.orchestra.run.smithi192.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-08T08:42:21.320 INFO:teuthology.orchestra.run.smithi192.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-08T08:42:21.320 INFO:teuthology.orchestra.run.smithi192.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-08T08:42:21.321 INFO:teuthology.orchestra.run.smithi192.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64.conf 2024-05-08T08:42:21.322 DEBUG:teuthology.orchestra.run.smithi192:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64 2024-05-08T08:42:21.450 DEBUG:teuthology.orchestra.run.smithi192:> sudo shutdown -r now 2024-05-08T08:42:21.510 DEBUG:teuthology.orchestra.run.smithi196:> rpm -q kernel | sort -rV | head -n 1 2024-05-08T08:42:21.565 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 4/4 2024-05-08T08:42:21.604 INFO:teuthology.orchestra.run.smithi196.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-08T08:42:21.604 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-08T08:42:21.605 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-444.el9.x86_64 2024-05-08T08:42:21.605 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-08T08:42:21.605 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-08T08:42:21.605 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-05-08T08:42:21.605 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi196.front.sepia.ceph.com, path=None, version=distro) 2024-05-08T08:42:21.605 DEBUG:teuthology.orchestra.run.smithi196:> rpm -q kernel | sort -rV | head -n 1 2024-05-08T08:42:21.606 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 4/4 2024-05-08T08:42:21.654 INFO:teuthology.orchestra.run.smithi196.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-08T08:42:21.655 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-08T08:42:21.655 DEBUG:teuthology.orchestra.run.smithi196:> sudo rpm -qi grub2-tools 2024-05-08T08:42:21.686 INFO:teuthology.orchestra.run.smithi196.stdout:Name : grub2-tools 2024-05-08T08:42:21.686 INFO:teuthology.orchestra.run.smithi196.stdout:Epoch : 1 2024-05-08T08:42:21.686 INFO:teuthology.orchestra.run.smithi196.stdout:Version : 2.06 2024-05-08T08:42:21.687 INFO:teuthology.orchestra.run.smithi196.stdout:Release : 68.el9 2024-05-08T08:42:21.687 INFO:teuthology.orchestra.run.smithi196.stdout:Architecture: x86_64 2024-05-08T08:42:21.687 INFO:teuthology.orchestra.run.smithi196.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-08T08:42:21.687 INFO:teuthology.orchestra.run.smithi196.stdout:Group : Unspecified 2024-05-08T08:42:21.687 INFO:teuthology.orchestra.run.smithi196.stdout:Size : 8264254 2024-05-08T08:42:21.687 INFO:teuthology.orchestra.run.smithi196.stdout:License : GPLv3+ 2024-05-08T08:42:21.687 INFO:teuthology.orchestra.run.smithi196.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-08T08:42:21.687 INFO:teuthology.orchestra.run.smithi196.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-08T08:42:21.687 INFO:teuthology.orchestra.run.smithi196.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-08T08:42:21.687 INFO:teuthology.orchestra.run.smithi196.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-08T08:42:21.687 INFO:teuthology.orchestra.run.smithi196.stdout:Packager : builder@centos.org 2024-05-08T08:42:21.687 INFO:teuthology.orchestra.run.smithi196.stdout:Vendor : CentOS 2024-05-08T08:42:21.687 INFO:teuthology.orchestra.run.smithi196.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-08T08:42:21.687 INFO:teuthology.orchestra.run.smithi196.stdout:Summary : Support tools for GRUB. 2024-05-08T08:42:21.688 INFO:teuthology.orchestra.run.smithi196.stdout:Description : 2024-05-08T08:42:21.688 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:42:21.688 INFO:teuthology.orchestra.run.smithi196.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-08T08:42:21.688 INFO:teuthology.orchestra.run.smithi196.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-08T08:42:21.688 INFO:teuthology.orchestra.run.smithi196.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-08T08:42:21.688 INFO:teuthology.orchestra.run.smithi196.stdout:hardware devices. 2024-05-08T08:42:21.688 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:42:21.688 INFO:teuthology.orchestra.run.smithi196.stdout:This subpackage provides tools for support of all platforms. 2024-05-08T08:42:21.689 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-08T08:42:21.689 INFO:teuthology.task.kernel:Updating grub on smithi196 to boot 5.14.0-444.el9.x86_64 2024-05-08T08:42:21.689 DEBUG:teuthology.orchestra.run.smithi196:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-08T08:42:21.971 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: kernel-5.14.0-444.el9.x86_64 4/4 2024-05-08T08:42:21.971 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : kernel-5.14.0-444.el9.x86_64 1/4 2024-05-08T08:42:21.971 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-08T08:42:21.971 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-08T08:42:22.314 INFO:teuthology.orchestra.run.smithi196.stderr:Generating grub configuration file ... 2024-05-08T08:42:22.759 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-08T08:42:22.759 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:42:22.759 INFO:teuthology.orchestra.run.smithi059.stdout:Installed: 2024-05-08T08:42:22.760 INFO:teuthology.orchestra.run.smithi059.stdout: kernel-5.14.0-444.el9.x86_64 2024-05-08T08:42:22.760 INFO:teuthology.orchestra.run.smithi059.stdout: kernel-core-5.14.0-444.el9.x86_64 2024-05-08T08:42:22.760 INFO:teuthology.orchestra.run.smithi059.stdout: kernel-modules-5.14.0-444.el9.x86_64 2024-05-08T08:42:22.760 INFO:teuthology.orchestra.run.smithi059.stdout: kernel-modules-core-5.14.0-444.el9.x86_64 2024-05-08T08:42:22.760 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:42:22.760 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2024-05-08T08:42:23.097 DEBUG:teuthology.orchestra.run.smithi059:> rpm -q kernel | sort -rV | head -n 1 2024-05-08T08:42:23.177 INFO:teuthology.orchestra.run.smithi059.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-08T08:42:23.177 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-08T08:42:23.177 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-444.el9.x86_64 2024-05-08T08:42:23.178 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-08T08:42:23.178 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-08T08:42:23.178 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-05-08T08:42:23.178 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi059.front.sepia.ceph.com, path=None, version=distro) 2024-05-08T08:42:23.178 DEBUG:teuthology.orchestra.run.smithi059:> rpm -q kernel | sort -rV | head -n 1 2024-05-08T08:42:23.229 INFO:teuthology.orchestra.run.smithi059.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-08T08:42:23.229 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-08T08:42:23.229 DEBUG:teuthology.orchestra.run.smithi059:> sudo rpm -qi grub2-tools 2024-05-08T08:42:23.260 INFO:teuthology.orchestra.run.smithi059.stdout:Name : grub2-tools 2024-05-08T08:42:23.260 INFO:teuthology.orchestra.run.smithi059.stdout:Epoch : 1 2024-05-08T08:42:23.260 INFO:teuthology.orchestra.run.smithi059.stdout:Version : 2.06 2024-05-08T08:42:23.260 INFO:teuthology.orchestra.run.smithi059.stdout:Release : 68.el9 2024-05-08T08:42:23.260 INFO:teuthology.orchestra.run.smithi059.stdout:Architecture: x86_64 2024-05-08T08:42:23.260 INFO:teuthology.orchestra.run.smithi059.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-08T08:42:23.260 INFO:teuthology.orchestra.run.smithi059.stdout:Group : Unspecified 2024-05-08T08:42:23.260 INFO:teuthology.orchestra.run.smithi059.stdout:Size : 8264254 2024-05-08T08:42:23.260 INFO:teuthology.orchestra.run.smithi059.stdout:License : GPLv3+ 2024-05-08T08:42:23.260 INFO:teuthology.orchestra.run.smithi059.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-08T08:42:23.261 INFO:teuthology.orchestra.run.smithi059.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-08T08:42:23.261 INFO:teuthology.orchestra.run.smithi059.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-08T08:42:23.261 INFO:teuthology.orchestra.run.smithi059.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-08T08:42:23.261 INFO:teuthology.orchestra.run.smithi059.stdout:Packager : builder@centos.org 2024-05-08T08:42:23.261 INFO:teuthology.orchestra.run.smithi059.stdout:Vendor : CentOS 2024-05-08T08:42:23.261 INFO:teuthology.orchestra.run.smithi059.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-08T08:42:23.261 INFO:teuthology.orchestra.run.smithi059.stdout:Summary : Support tools for GRUB. 2024-05-08T08:42:23.261 INFO:teuthology.orchestra.run.smithi059.stdout:Description : 2024-05-08T08:42:23.261 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:42:23.261 INFO:teuthology.orchestra.run.smithi059.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-08T08:42:23.261 INFO:teuthology.orchestra.run.smithi059.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-08T08:42:23.261 INFO:teuthology.orchestra.run.smithi059.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-08T08:42:23.261 INFO:teuthology.orchestra.run.smithi059.stdout:hardware devices. 2024-05-08T08:42:23.261 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:42:23.262 INFO:teuthology.orchestra.run.smithi059.stdout:This subpackage provides tools for support of all platforms. 2024-05-08T08:42:23.262 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-08T08:42:23.262 INFO:teuthology.task.kernel:Updating grub on smithi059 to boot 5.14.0-444.el9.x86_64 2024-05-08T08:42:23.263 DEBUG:teuthology.orchestra.run.smithi059:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-08T08:42:23.808 INFO:teuthology.orchestra.run.smithi196.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-08T08:42:23.839 INFO:teuthology.orchestra.run.smithi196.stderr:done 2024-05-08T08:42:23.841 DEBUG:teuthology.orchestra.run.smithi196:> mktemp 2024-05-08T08:42:23.856 INFO:teuthology.orchestra.run.smithi196.stdout:/tmp/tmp.WmlAUtO7oB 2024-05-08T08:42:23.857 DEBUG:teuthology.orchestra.run.smithi196:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.WmlAUtO7oB 2024-05-08T08:42:23.900 INFO:teuthology.orchestra.run.smithi059.stderr:Generating grub configuration file ... 2024-05-08T08:42:23.923 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 0666 /tmp/tmp.WmlAUtO7oB 2024-05-08T08:42:24.059 DEBUG:teuthology.orchestra.remote:smithi196:/tmp/tmp.WmlAUtO7oB is 6KB 2024-05-08T08:42:24.107 DEBUG:teuthology.orchestra.run.smithi196:> rm -fr /tmp/tmp.WmlAUtO7oB 2024-05-08T08:42:24.121 DEBUG:teuthology.orchestra.run.smithi196:> sudo /bin/ls /boot/loader/entries || true 2024-05-08T08:42:24.183 INFO:teuthology.orchestra.run.smithi196.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-08T08:42:24.183 INFO:teuthology.orchestra.run.smithi196.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-08T08:42:24.183 INFO:teuthology.orchestra.run.smithi196.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-08T08:42:24.183 INFO:teuthology.orchestra.run.smithi196.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64.conf 2024-05-08T08:42:24.184 DEBUG:teuthology.orchestra.run.smithi196:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64 2024-05-08T08:42:24.325 DEBUG:teuthology.orchestra.run.smithi196:> sudo shutdown -r now 2024-05-08T08:42:25.347 INFO:teuthology.orchestra.run.smithi059.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-08T08:42:25.381 INFO:teuthology.orchestra.run.smithi059.stderr:done 2024-05-08T08:42:25.383 DEBUG:teuthology.orchestra.run.smithi059:> mktemp 2024-05-08T08:42:25.400 INFO:teuthology.orchestra.run.smithi059.stdout:/tmp/tmp.8mO7WICe6S 2024-05-08T08:42:25.401 DEBUG:teuthology.orchestra.run.smithi059:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.8mO7WICe6S 2024-05-08T08:42:25.467 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 0666 /tmp/tmp.8mO7WICe6S 2024-05-08T08:42:25.608 DEBUG:teuthology.orchestra.remote:smithi059:/tmp/tmp.8mO7WICe6S is 6KB 2024-05-08T08:42:25.656 DEBUG:teuthology.orchestra.run.smithi059:> rm -fr /tmp/tmp.8mO7WICe6S 2024-05-08T08:42:25.671 DEBUG:teuthology.orchestra.run.smithi059:> sudo /bin/ls /boot/loader/entries || true 2024-05-08T08:42:25.734 INFO:teuthology.orchestra.run.smithi059.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-08T08:42:25.734 INFO:teuthology.orchestra.run.smithi059.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-08T08:42:25.734 INFO:teuthology.orchestra.run.smithi059.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-08T08:42:25.734 INFO:teuthology.orchestra.run.smithi059.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64.conf 2024-05-08T08:42:25.735 DEBUG:teuthology.orchestra.run.smithi059:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64 2024-05-08T08:42:25.880 DEBUG:teuthology.orchestra.run.smithi059:> sudo shutdown -r now 2024-05-08T08:42:51.479 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-05-08T08:42:51.480 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi192.front.sepia.ceph.com' 2024-05-08T08:42:51.481 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T08:42:54.331 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-05-08T08:42:54.331 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-05-08T08:42:54.331 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T08:42:55.884 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-05-08T08:42:55.885 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi059.front.sepia.ceph.com' 2024-05-08T08:42:55.885 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T08:43:09.986 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.192 2024-05-08T08:43:12.798 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.196 2024-05-08T08:43:14.338 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.59 2024-05-08T08:43:18.990 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi192.front.sepia.ceph.com' 2024-05-08T08:43:18.991 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T08:43:21.802 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-05-08T08:43:21.803 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T08:43:22.046 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.192 2024-05-08T08:43:23.340 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi059.front.sepia.ceph.com' 2024-05-08T08:43:23.341 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T08:43:24.865 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.196 2024-05-08T08:43:34.049 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi192.front.sepia.ceph.com' 2024-05-08T08:43:34.050 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T08:43:36.866 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-05-08T08:43:36.867 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T08:43:55.295 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.196 2024-05-08T08:44:05.825 DEBUG:teuthology.orchestra.run.smithi192:> true 2024-05-08T08:44:06.243 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi192.front.sepia.ceph.com' 2024-05-08T08:44:06.243 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-444.el9.x86_64"... 2024-05-08T08:44:06.244 DEBUG:teuthology.orchestra.run.smithi192:> uname -r 2024-05-08T08:44:06.258 INFO:teuthology.orchestra.run.smithi192.stdout:5.14.0-444.el9.x86_64 2024-05-08T08:44:06.259 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-444.el9.x86_64 vs 5.14.0-444.el9.x86_64 2024-05-08T08:44:06.259 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-08T08:44:06.259 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-08T08:44:07.260 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-05-08T08:44:07.260 DEBUG:teuthology.orchestra.run.smithi192:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-08T08:44:07.335 INFO:teuthology.orchestra.run.smithi192.stdout:ttyS1 2024-05-08T08:44:07.374 DEBUG:teuthology.parallel:result is None 2024-05-08T08:44:10.296 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-05-08T08:44:10.298 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T08:44:10.597 DEBUG:teuthology.orchestra.run.smithi196:> true 2024-05-08T08:44:11.102 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-05-08T08:44:11.102 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-444.el9.x86_64"... 2024-05-08T08:44:11.103 DEBUG:teuthology.orchestra.run.smithi196:> uname -r 2024-05-08T08:44:11.117 INFO:teuthology.orchestra.run.smithi196.stdout:5.14.0-444.el9.x86_64 2024-05-08T08:44:11.117 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-444.el9.x86_64 vs 5.14.0-444.el9.x86_64 2024-05-08T08:44:11.117 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-08T08:44:11.117 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-08T08:44:12.118 INFO:teuthology.task.kernel:Enabling kdb on client.0... 2024-05-08T08:44:12.118 DEBUG:teuthology.orchestra.run.smithi196:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-08T08:44:12.240 INFO:teuthology.orchestra.run.smithi196.stdout:ttyS1 2024-05-08T08:44:12.298 DEBUG:teuthology.parallel:result is None 2024-05-08T08:44:23.343 DEBUG:teuthology.orchestra.remote:timed out 2024-05-08T08:44:35.343 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi059.front.sepia.ceph.com' 2024-05-08T08:44:35.344 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T08:44:35.720 DEBUG:teuthology.orchestra.run.smithi059:> true 2024-05-08T08:44:36.076 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi059.front.sepia.ceph.com' 2024-05-08T08:44:36.077 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-444.el9.x86_64"... 2024-05-08T08:44:36.077 DEBUG:teuthology.orchestra.run.smithi059:> uname -r 2024-05-08T08:44:36.093 INFO:teuthology.orchestra.run.smithi059.stdout:5.14.0-444.el9.x86_64 2024-05-08T08:44:36.094 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-444.el9.x86_64 vs 5.14.0-444.el9.x86_64 2024-05-08T08:44:36.094 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-08T08:44:36.094 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-08T08:44:37.094 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-05-08T08:44:37.095 DEBUG:teuthology.orchestra.run.smithi059:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-08T08:44:37.162 INFO:teuthology.orchestra.run.smithi059.stdout:ttyS1 2024-05-08T08:44:37.201 DEBUG:teuthology.parallel:result is None 2024-05-08T08:44:37.202 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-08T08:44:37.209 INFO:teuthology.task.internal:Creating test directory... 2024-05-08T08:44:37.210 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-08T08:44:37.212 DEBUG:teuthology.orchestra.run.smithi192:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-08T08:44:37.215 DEBUG:teuthology.orchestra.run.smithi196:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-08T08:44:37.235 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-08T08:44:37.241 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-08T08:44:37.250 INFO:teuthology.task.internal:Creating archive directory... 2024-05-08T08:44:37.251 DEBUG:teuthology.orchestra.run.smithi059:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-08T08:44:37.271 DEBUG:teuthology.orchestra.run.smithi192:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-08T08:44:37.275 DEBUG:teuthology.orchestra.run.smithi196:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-08T08:44:37.488 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-08T08:44:37.495 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-08T08:44:37.495 DEBUG:teuthology.orchestra.run.smithi059:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-08T08:44:37.498 DEBUG:teuthology.orchestra.run.smithi192:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-08T08:44:37.501 DEBUG:teuthology.orchestra.run.smithi196:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-08T08:44:37.529 INFO:teuthology.orchestra.run.smithi192.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T08:44:37.534 INFO:teuthology.orchestra.run.smithi196.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T08:44:37.534 INFO:teuthology.orchestra.run.smithi059.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T08:44:37.541 INFO:teuthology.orchestra.run.smithi192.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T08:44:37.546 INFO:teuthology.orchestra.run.smithi196.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T08:44:37.546 INFO:teuthology.orchestra.run.smithi059.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T08:44:37.548 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-08T08:44:37.555 INFO:teuthology.task.internal:Configuring sudo... 2024-05-08T08:44:37.556 DEBUG:teuthology.orchestra.run.smithi059:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-08T08:44:37.591 DEBUG:teuthology.orchestra.run.smithi192:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-08T08:44:37.593 DEBUG:teuthology.orchestra.run.smithi196:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-08T08:44:37.622 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-08T08:44:37.631 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-08T08:44:37.631 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-08T08:44:37.661 DEBUG:teuthology.orchestra.run.smithi192:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-08T08:44:37.663 DEBUG:teuthology.orchestra.run.smithi196:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-08T08:44:37.681 DEBUG:teuthology.orchestra.run.smithi059:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T08:44:37.764 DEBUG:teuthology.orchestra.run.smithi059:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T08:44:37.814 DEBUG:teuthology.orchestra.run.smithi059:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T08:44:37.899 DEBUG:teuthology.orchestra.run.smithi059:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T08:44:37.942 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-05-08T08:44:37.942 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-08T08:44:38.010 DEBUG:teuthology.orchestra.run.smithi192:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T08:44:38.056 DEBUG:teuthology.orchestra.run.smithi192:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T08:44:38.107 DEBUG:teuthology.orchestra.run.smithi192:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T08:44:38.191 DEBUG:teuthology.orchestra.run.smithi192:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T08:44:38.233 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-08T08:44:38.233 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-08T08:44:38.299 DEBUG:teuthology.orchestra.run.smithi196:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T08:44:38.345 DEBUG:teuthology.orchestra.run.smithi196:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T08:44:38.394 DEBUG:teuthology.orchestra.run.smithi196:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T08:44:38.477 DEBUG:teuthology.orchestra.run.smithi196:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T08:44:38.523 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-05-08T08:44:38.523 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-08T08:44:38.590 DEBUG:teuthology.orchestra.run.smithi059:> sudo service rsyslog restart 2024-05-08T08:44:38.593 DEBUG:teuthology.orchestra.run.smithi192:> sudo service rsyslog restart 2024-05-08T08:44:38.596 DEBUG:teuthology.orchestra.run.smithi196:> sudo service rsyslog restart 2024-05-08T08:44:38.638 INFO:teuthology.orchestra.run.smithi192.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T08:44:38.641 INFO:teuthology.orchestra.run.smithi059.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T08:44:38.685 INFO:teuthology.orchestra.run.smithi196.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T08:44:39.152 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-08T08:44:39.160 INFO:teuthology.task.internal:Starting timer... 2024-05-08T08:44:39.161 INFO:teuthology.run_tasks:Running task pcp... 2024-05-08T08:44:39.172 INFO:teuthology.run_tasks:Running task selinux... 2024-05-08T08:44:39.180 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-05-08T08:44:39.180 DEBUG:teuthology.orchestra.run.smithi059:> sudo service auditd rotate 2024-05-08T08:44:39.232 INFO:teuthology.orchestra.run.smithi059.stdout:Rotating logs: 2024-05-08T08:44:39.234 DEBUG:teuthology.orchestra.run.smithi192:> sudo service auditd rotate 2024-05-08T08:44:39.308 INFO:teuthology.orchestra.run.smithi192.stdout:Rotating logs: 2024-05-08T08:44:39.310 DEBUG:teuthology.orchestra.run.smithi196:> sudo service auditd rotate 2024-05-08T08:44:39.385 INFO:teuthology.orchestra.run.smithi196.stdout:Rotating logs: 2024-05-08T08:44:39.387 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-08T08:44:39.387 DEBUG:teuthology.orchestra.run.smithi059:> /usr/sbin/getenforce 2024-05-08T08:44:39.426 INFO:teuthology.orchestra.run.smithi059.stdout:Permissive 2024-05-08T08:44:39.427 DEBUG:teuthology.orchestra.run.smithi192:> /usr/sbin/getenforce 2024-05-08T08:44:39.461 INFO:teuthology.orchestra.run.smithi192.stdout:Permissive 2024-05-08T08:44:39.461 DEBUG:teuthology.orchestra.run.smithi196:> /usr/sbin/getenforce 2024-05-08T08:44:39.502 INFO:teuthology.orchestra.run.smithi196.stdout:Permissive 2024-05-08T08:44:39.502 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi059.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi192.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi196.front.sepia.ceph.com': 'permissive'} 2024-05-08T08:44:39.502 DEBUG:teuthology.orchestra.run.smithi059:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-08T08:44:39.532 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T08:44:39.532 DEBUG:teuthology.orchestra.run.smithi192:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-08T08:44:39.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T08:44:39.560 DEBUG:teuthology.orchestra.run.smithi196:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-08T08:44:39.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T08:44:39.591 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-08T08:44:39.591 DEBUG:teuthology.orchestra.run.smithi059:> sudo /usr/sbin/setenforce permissive 2024-05-08T08:44:39.620 DEBUG:teuthology.orchestra.run.smithi192:> sudo /usr/sbin/setenforce permissive 2024-05-08T08:44:39.663 DEBUG:teuthology.orchestra.run.smithi196:> sudo /usr/sbin/setenforce permissive 2024-05-08T08:44:39.692 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-08T08:44:39.702 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-05-08T08:44:39.707 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-05-08T08:44:39.834 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-08T08:44:39.849 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-08T08:44:39.850 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi059.front.sepia.ceph.com,smithi192.front.sepia.ceph.com,smithi196.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-08T08:48:42.873 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi059.front.sepia.ceph.com'), Remote(name='ubuntu@smithi192.front.sepia.ceph.com'), Remote(name='ubuntu@smithi196.front.sepia.ceph.com')] 2024-05-08T08:48:42.875 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi059.front.sepia.ceph.com' 2024-05-08T08:48:42.876 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T08:48:42.953 DEBUG:teuthology.orchestra.run.smithi059:> true 2024-05-08T08:48:43.035 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi059.front.sepia.ceph.com' 2024-05-08T08:48:43.035 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi192.front.sepia.ceph.com' 2024-05-08T08:48:43.036 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T08:48:43.109 DEBUG:teuthology.orchestra.run.smithi192:> true 2024-05-08T08:48:43.191 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi192.front.sepia.ceph.com' 2024-05-08T08:48:43.191 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-05-08T08:48:43.191 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T08:48:43.268 DEBUG:teuthology.orchestra.run.smithi196:> true 2024-05-08T08:48:43.349 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-05-08T08:48:43.349 INFO:teuthology.run_tasks:Running task clock... 2024-05-08T08:48:43.361 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-08T08:48:43.361 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-08T08:48:43.361 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T08:48:43.364 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-08T08:48:43.364 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T08:48:43.366 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-08T08:48:43.366 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T08:48:43.395 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-08T08:48:43.396 INFO:teuthology.orchestra.run.smithi059.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-08T08:48:43.410 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-08T08:48:43.413 INFO:teuthology.orchestra.run.smithi059.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-08T08:48:43.424 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-08T08:48:43.434 INFO:teuthology.orchestra.run.smithi192.stderr:sudo: ntpd: command not found 2024-05-08T08:48:43.440 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-08T08:48:43.441 INFO:teuthology.orchestra.run.smithi059.stderr:sudo: ntpd: command not found 2024-05-08T08:48:43.446 INFO:teuthology.orchestra.run.smithi192.stdout:506 Cannot talk to daemon 2024-05-08T08:48:43.453 INFO:teuthology.orchestra.run.smithi059.stdout:506 Cannot talk to daemon 2024-05-08T08:48:43.460 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-08T08:48:43.466 INFO:teuthology.orchestra.run.smithi196.stderr:sudo: ntpd: command not found 2024-05-08T08:48:43.468 INFO:teuthology.orchestra.run.smithi059.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-08T08:48:43.473 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-08T08:48:43.478 INFO:teuthology.orchestra.run.smithi196.stdout:506 Cannot talk to daemon 2024-05-08T08:48:43.483 INFO:teuthology.orchestra.run.smithi059.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-08T08:48:43.492 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-08T08:48:43.507 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-08T08:48:43.527 INFO:teuthology.orchestra.run.smithi192.stderr:bash: line 1: ntpq: command not found 2024-05-08T08:48:43.531 INFO:teuthology.orchestra.run.smithi192.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T08:48:43.531 INFO:teuthology.orchestra.run.smithi192.stdout:=============================================================================== 2024-05-08T08:48:43.531 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T08:48:43.531 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T08:48:43.531 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T08:48:43.531 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T08:48:43.538 INFO:teuthology.orchestra.run.smithi059.stderr:bash: line 1: ntpq: command not found 2024-05-08T08:48:43.542 INFO:teuthology.orchestra.run.smithi059.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T08:48:43.542 INFO:teuthology.orchestra.run.smithi059.stdout:=============================================================================== 2024-05-08T08:48:43.542 INFO:teuthology.orchestra.run.smithi059.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T08:48:43.542 INFO:teuthology.orchestra.run.smithi059.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T08:48:43.542 INFO:teuthology.orchestra.run.smithi059.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T08:48:43.542 INFO:teuthology.orchestra.run.smithi059.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T08:48:43.566 INFO:teuthology.orchestra.run.smithi196.stderr:bash: line 1: ntpq: command not found 2024-05-08T08:48:43.569 INFO:teuthology.orchestra.run.smithi196.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T08:48:43.569 INFO:teuthology.orchestra.run.smithi196.stdout:=============================================================================== 2024-05-08T08:48:43.569 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T08:48:43.569 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T08:48:43.569 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T08:48:43.569 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T08:48:43.570 INFO:teuthology.run_tasks:Running task install... 2024-05-08T08:48:43.578 DEBUG:teuthology.task.install:project ceph 2024-05-08T08:48:43.578 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'extra_packages': ['rbd-nbd'], 'flavor': 'default', 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc'}} 2024-05-08T08:48:43.578 DEBUG:teuthology.task.install:config {'extra_packages': ['rbd-nbd'], 'flavor': 'default', 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc'} 2024-05-08T08:48:43.578 INFO:teuthology.task.install:Using flavor: default 2024-05-08T08:48:43.584 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-08T08:48:43.584 INFO:teuthology.task.install:extra packages: ['rbd-nbd'] 2024-05-08T08:48:43.585 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': ['rbd-nbd'], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc', 'tag': None, 'wait_for_package': False} 2024-05-08T08:48:43.585 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-08T08:48:43.588 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': ['rbd-nbd'], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc', 'tag': None, 'wait_for_package': False} 2024-05-08T08:48:43.588 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-08T08:48:43.589 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': ['rbd-nbd'], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc', 'tag': None, 'wait_for_package': False} 2024-05-08T08:48:43.589 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-08T08:48:43.730 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/ 2024-05-08T08:48:43.730 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2515.g96805dcf 2024-05-08T08:48:43.736 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/ 2024-05-08T08:48:43.736 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2515.g96805dcf 2024-05-08T08:48:43.800 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/ 2024-05-08T08:48:43.801 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2515.g96805dcf 2024-05-08T08:48:43.837 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-08T08:48:43.838 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-08T08:48:43.838 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-08T08:48:43.842 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-08T08:48:43.842 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-05-08T08:48:43.842 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-08T08:48:43.864 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, rbd-nbd on remote rpm x86_64 2024-05-08T08:48:43.864 DEBUG:teuthology.orchestra.run.smithi192:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-08T08:48:43.869 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, rbd-nbd on remote rpm x86_64 2024-05-08T08:48:43.869 DEBUG:teuthology.orchestra.run.smithi196:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-08T08:48:43.905 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-08T08:48:43.905 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-05-08T08:48:43.905 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-08T08:48:43.931 DEBUG:teuthology.orchestra.run.smithi192:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-08T08:48:43.933 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, rbd-nbd on remote rpm x86_64 2024-05-08T08:48:43.934 DEBUG:teuthology.orchestra.run.smithi059:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-08T08:48:43.936 DEBUG:teuthology.orchestra.run.smithi196:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-08T08:48:44.001 DEBUG:teuthology.orchestra.run.smithi059:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-08T08:48:44.006 DEBUG:teuthology.orchestra.run.smithi192:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-08T08:48:44.012 DEBUG:teuthology.orchestra.run.smithi196:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-08T08:48:44.070 INFO:teuthology.orchestra.run.smithi192.stdout:check_obsoletes = 1 2024-05-08T08:48:44.071 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum clean all 2024-05-08T08:48:44.076 INFO:teuthology.orchestra.run.smithi196.stdout:check_obsoletes = 1 2024-05-08T08:48:44.077 DEBUG:teuthology.orchestra.run.smithi059:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-08T08:48:44.078 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum clean all 2024-05-08T08:48:44.142 INFO:teuthology.orchestra.run.smithi059.stdout:check_obsoletes = 1 2024-05-08T08:48:44.144 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum clean all 2024-05-08T08:48:44.309 INFO:teuthology.orchestra.run.smithi192.stdout:54 files removed 2024-05-08T08:48:44.317 INFO:teuthology.orchestra.run.smithi196.stdout:54 files removed 2024-05-08T08:48:44.337 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-radosgw 2024-05-08T08:48:44.346 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install ceph-radosgw 2024-05-08T08:48:44.390 INFO:teuthology.orchestra.run.smithi059.stdout:54 files removed 2024-05-08T08:48:44.418 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum -y install ceph-radosgw 2024-05-08T08:48:44.951 INFO:teuthology.orchestra.run.smithi192.stdout:ceph packages for x86_64 243 kB/s | 83 kB 00:00 2024-05-08T08:48:45.006 INFO:teuthology.orchestra.run.smithi196.stdout:ceph packages for x86_64 215 kB/s | 83 kB 00:00 2024-05-08T08:48:45.088 INFO:teuthology.orchestra.run.smithi059.stdout:ceph packages for x86_64 217 kB/s | 83 kB 00:00 2024-05-08T08:48:45.369 INFO:teuthology.orchestra.run.smithi192.stdout:ceph noarch packages 31 kB/s | 11 kB 00:00 2024-05-08T08:48:45.457 INFO:teuthology.orchestra.run.smithi196.stdout:ceph noarch packages 29 kB/s | 11 kB 00:00 2024-05-08T08:48:45.539 INFO:teuthology.orchestra.run.smithi059.stdout:ceph noarch packages 29 kB/s | 11 kB 00:00 2024-05-08T08:48:45.728 INFO:teuthology.orchestra.run.smithi192.stdout:ceph source packages 6.0 kB/s | 1.9 kB 00:00 2024-05-08T08:48:45.816 INFO:teuthology.orchestra.run.smithi196.stdout:ceph source packages 6.0 kB/s | 1.9 kB 00:00 2024-05-08T08:48:45.998 INFO:teuthology.orchestra.run.smithi059.stdout:ceph source packages 4.6 kB/s | 1.9 kB 00:00 2024-05-08T08:48:47.066 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - BaseOS 6.7 MB/s | 8.1 MB 00:01 2024-05-08T08:48:48.767 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - AppStream 31 MB/s | 19 MB 00:00 2024-05-08T08:48:49.036 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - BaseOS 2.5 MB/s | 8.1 MB 00:03 2024-05-08T08:48:49.240 INFO:teuthology.orchestra.run.smithi059.stdout:CentOS Stream 9 - BaseOS 2.5 MB/s | 8.1 MB 00:03 2024-05-08T08:48:50.821 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - AppStream 27 MB/s | 19 MB 00:00 2024-05-08T08:48:52.716 INFO:teuthology.orchestra.run.smithi059.stdout:CentOS Stream 9 - AppStream 8.2 MB/s | 19 MB 00:02 2024-05-08T08:48:56.676 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - CRB 1.4 MB/s | 6.2 MB 00:04 2024-05-08T08:48:56.759 INFO:teuthology.orchestra.run.smithi059.stdout:CentOS Stream 9 - CRB 17 MB/s | 6.2 MB 00:00 2024-05-08T08:48:57.439 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - CRB 2.0 MB/s | 6.2 MB 00:03 2024-05-08T08:48:58.093 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - Extras packages 67 kB/s | 16 kB 00:00 2024-05-08T08:48:58.410 INFO:teuthology.orchestra.run.smithi059.stdout:CentOS Stream 9 - Extras packages 37 kB/s | 16 kB 00:00 2024-05-08T08:48:58.856 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - Extras packages 67 kB/s | 16 kB 00:00 2024-05-08T08:48:59.703 INFO:teuthology.orchestra.run.smithi059.stdout:Extra Packages for Enterprise Linux 18 MB/s | 21 MB 00:01 2024-05-08T08:49:00.610 INFO:teuthology.orchestra.run.smithi192.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-05-08T08:49:05.070 INFO:teuthology.orchestra.run.smithi059.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-05-08T08:49:05.691 INFO:teuthology.orchestra.run.smithi192.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-05-08T08:49:06.168 INFO:teuthology.orchestra.run.smithi196.stdout:Extra Packages for Enterprise Linux 2.7 MB/s | 21 MB 00:07 2024-05-08T08:49:07.483 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies resolved. 2024-05-08T08:49:07.486 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-05-08T08:49:07.486 INFO:teuthology.orchestra.run.smithi059.stdout: Package Arch Version Repository Size 2024-05-08T08:49:07.487 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-05-08T08:49:07.487 INFO:teuthology.orchestra.run.smithi059.stdout:Installing: 2024-05-08T08:49:07.487 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-radosgw x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 10 M 2024-05-08T08:49:07.487 INFO:teuthology.orchestra.run.smithi059.stdout:Upgrading: 2024-05-08T08:49:07.487 INFO:teuthology.orchestra.run.smithi059.stdout: librados2 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 3.4 M 2024-05-08T08:49:07.487 INFO:teuthology.orchestra.run.smithi059.stdout: librbd1 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 3.1 M 2024-05-08T08:49:07.487 INFO:teuthology.orchestra.run.smithi059.stdout:Installing dependencies: 2024-05-08T08:49:07.487 INFO:teuthology.orchestra.run.smithi059.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-08T08:49:07.487 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-base x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 5.4 M 2024-05-08T08:49:07.487 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-common x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 21 M 2024-05-08T08:49:07.487 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-selinux x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 25 k 2024-05-08T08:49:07.487 INFO:teuthology.orchestra.run.smithi059.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-08T08:49:07.487 INFO:teuthology.orchestra.run.smithi059.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-08T08:49:07.487 INFO:teuthology.orchestra.run.smithi059.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-08T08:49:07.488 INFO:teuthology.orchestra.run.smithi059.stdout: libcephfs2 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 746 k 2024-05-08T08:49:07.488 INFO:teuthology.orchestra.run.smithi059.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-08T08:49:07.488 INFO:teuthology.orchestra.run.smithi059.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-08T08:49:07.488 INFO:teuthology.orchestra.run.smithi059.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-08T08:49:07.488 INFO:teuthology.orchestra.run.smithi059.stdout: libradosstriper1 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 500 k 2024-05-08T08:49:07.488 INFO:teuthology.orchestra.run.smithi059.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-08T08:49:07.488 INFO:teuthology.orchestra.run.smithi059.stdout: librgw2 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 5.3 M 2024-05-08T08:49:07.488 INFO:teuthology.orchestra.run.smithi059.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-08T08:49:07.488 INFO:teuthology.orchestra.run.smithi059.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-08T08:49:07.488 INFO:teuthology.orchestra.run.smithi059.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-08T08:49:07.488 INFO:teuthology.orchestra.run.smithi059.stdout: python3-ceph-argparse x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 45 k 2024-05-08T08:49:07.488 INFO:teuthology.orchestra.run.smithi059.stdout: python3-ceph-common x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 135 k 2024-05-08T08:49:07.488 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cephfs x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 162 k 2024-05-08T08:49:07.488 INFO:teuthology.orchestra.run.smithi059.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-08T08:49:07.489 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rados x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 321 k 2024-05-08T08:49:07.489 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rbd x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 298 k 2024-05-08T08:49:07.489 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rgw x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 99 k 2024-05-08T08:49:07.489 INFO:teuthology.orchestra.run.smithi059.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-08T08:49:07.489 INFO:teuthology.orchestra.run.smithi059.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-08T08:49:07.489 INFO:teuthology.orchestra.run.smithi059.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-08T08:49:07.489 INFO:teuthology.orchestra.run.smithi059.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-08T08:49:07.489 INFO:teuthology.orchestra.run.smithi059.stdout:Installing weak dependencies: 2024-05-08T08:49:07.489 INFO:teuthology.orchestra.run.smithi059.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-08T08:49:07.489 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:49:07.489 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction Summary 2024-05-08T08:49:07.489 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-05-08T08:49:07.489 INFO:teuthology.orchestra.run.smithi059.stdout:Install 30 Packages 2024-05-08T08:49:07.489 INFO:teuthology.orchestra.run.smithi059.stdout:Upgrade 2 Packages 2024-05-08T08:49:07.490 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:49:07.490 INFO:teuthology.orchestra.run.smithi059.stdout:Total download size: 60 M 2024-05-08T08:49:07.490 INFO:teuthology.orchestra.run.smithi059.stdout:Downloading Packages: 2024-05-08T08:49:08.024 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-05-08T08:49:08.037 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-05-08T08:49:08.037 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2024-05-08T08:49:08.037 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-05-08T08:49:08.037 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-05-08T08:49:08.038 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-radosgw x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 10 M 2024-05-08T08:49:08.038 INFO:teuthology.orchestra.run.smithi192.stdout:Upgrading: 2024-05-08T08:49:08.038 INFO:teuthology.orchestra.run.smithi192.stdout: librados2 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 3.4 M 2024-05-08T08:49:08.038 INFO:teuthology.orchestra.run.smithi192.stdout: librbd1 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 3.1 M 2024-05-08T08:49:08.038 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2024-05-08T08:49:08.038 INFO:teuthology.orchestra.run.smithi192.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-08T08:49:08.038 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-base x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 5.4 M 2024-05-08T08:49:08.038 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-common x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 21 M 2024-05-08T08:49:08.038 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-selinux x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 25 k 2024-05-08T08:49:08.038 INFO:teuthology.orchestra.run.smithi192.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-08T08:49:08.038 INFO:teuthology.orchestra.run.smithi192.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-08T08:49:08.038 INFO:teuthology.orchestra.run.smithi192.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-08T08:49:08.038 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs2 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 746 k 2024-05-08T08:49:08.038 INFO:teuthology.orchestra.run.smithi192.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-08T08:49:08.039 INFO:teuthology.orchestra.run.smithi192.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-08T08:49:08.039 INFO:teuthology.orchestra.run.smithi192.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-08T08:49:08.039 INFO:teuthology.orchestra.run.smithi192.stdout: libradosstriper1 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 500 k 2024-05-08T08:49:08.039 INFO:teuthology.orchestra.run.smithi192.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-08T08:49:08.039 INFO:teuthology.orchestra.run.smithi192.stdout: librgw2 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 5.3 M 2024-05-08T08:49:08.039 INFO:teuthology.orchestra.run.smithi192.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-08T08:49:08.039 INFO:teuthology.orchestra.run.smithi192.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-08T08:49:08.039 INFO:teuthology.orchestra.run.smithi192.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-08T08:49:08.039 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-argparse x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 45 k 2024-05-08T08:49:08.039 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-common x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 135 k 2024-05-08T08:49:08.039 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cephfs x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 162 k 2024-05-08T08:49:08.039 INFO:teuthology.orchestra.run.smithi192.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-08T08:49:08.039 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rados x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 321 k 2024-05-08T08:49:08.039 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rbd x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 298 k 2024-05-08T08:49:08.040 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rgw x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 99 k 2024-05-08T08:49:08.040 INFO:teuthology.orchestra.run.smithi192.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-08T08:49:08.040 INFO:teuthology.orchestra.run.smithi192.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-08T08:49:08.040 INFO:teuthology.orchestra.run.smithi192.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-08T08:49:08.040 INFO:teuthology.orchestra.run.smithi192.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-08T08:49:08.040 INFO:teuthology.orchestra.run.smithi192.stdout:Installing weak dependencies: 2024-05-08T08:49:08.040 INFO:teuthology.orchestra.run.smithi192.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-08T08:49:08.040 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:49:08.040 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-05-08T08:49:08.040 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-05-08T08:49:08.040 INFO:teuthology.orchestra.run.smithi192.stdout:Install 30 Packages 2024-05-08T08:49:08.040 INFO:teuthology.orchestra.run.smithi192.stdout:Upgrade 2 Packages 2024-05-08T08:49:08.040 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:49:08.041 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 60 M 2024-05-08T08:49:08.041 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-05-08T08:49:08.455 INFO:teuthology.orchestra.run.smithi059.stdout:(1/32): ceph-base-19.0.0-2515.g96805dcf.el9.x86 6.6 MB/s | 5.4 MB 00:00 2024-05-08T08:49:08.522 INFO:teuthology.orchestra.run.smithi059.stdout:(2/32): ceph-selinux-19.0.0-2515.g96805dcf.el9. 378 kB/s | 25 kB 00:00 2024-05-08T08:49:08.691 INFO:teuthology.orchestra.run.smithi059.stdout:(3/32): libcephfs2-19.0.0-2515.g96805dcf.el9.x8 4.4 MB/s | 746 kB 00:00 2024-05-08T08:49:08.772 INFO:teuthology.orchestra.run.smithi059.stdout:(4/32): libradosstriper1-19.0.0-2515.g96805dcf. 5.9 MB/s | 500 kB 00:00 2024-05-08T08:49:09.431 INFO:teuthology.orchestra.run.smithi059.stdout:(5/32): librgw2-19.0.0-2515.g96805dcf.el9.x86_6 8.1 MB/s | 5.3 MB 00:00 2024-05-08T08:49:09.498 INFO:teuthology.orchestra.run.smithi059.stdout:(6/32): python3-ceph-argparse-19.0.0-2515.g9680 679 kB/s | 45 kB 00:00 2024-05-08T08:49:09.582 INFO:teuthology.orchestra.run.smithi059.stdout:(7/32): python3-ceph-common-19.0.0-2515.g96805d 1.6 MB/s | 135 kB 00:00 2024-05-08T08:49:09.665 INFO:teuthology.orchestra.run.smithi059.stdout:(8/32): python3-cephfs-19.0.0-2515.g96805dcf.el 1.9 MB/s | 162 kB 00:00 2024-05-08T08:49:09.757 INFO:teuthology.orchestra.run.smithi059.stdout:(9/32): python3-rados-19.0.0-2515.g96805dcf.el9 3.4 MB/s | 321 kB 00:00 2024-05-08T08:49:09.849 INFO:teuthology.orchestra.run.smithi059.stdout:(10/32): python3-rbd-19.0.0-2515.g96805dcf.el9. 3.2 MB/s | 298 kB 00:00 2024-05-08T08:49:09.933 INFO:teuthology.orchestra.run.smithi059.stdout:(11/32): python3-rgw-19.0.0-2515.g96805dcf.el9. 1.2 MB/s | 99 kB 00:00 2024-05-08T08:49:10.108 INFO:teuthology.orchestra.run.smithi059.stdout:(12/32): boost-program-options-1.75.0-8.el9.x86 611 kB/s | 107 kB 00:00 2024-05-08T08:49:10.200 INFO:teuthology.orchestra.run.smithi059.stdout:(13/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.7 MB/s | 160 kB 00:00 2024-05-08T08:49:10.259 INFO:teuthology.orchestra.run.smithi059.stdout:(14/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 775 kB/s | 45 kB 00:00 2024-05-08T08:49:10.434 INFO:teuthology.orchestra.run.smithi059.stdout:(15/32): librdkafka-1.6.1-102.el9.x86_64.rpm 3.7 MB/s | 662 kB 00:00 2024-05-08T08:49:10.534 INFO:teuthology.orchestra.run.smithi059.stdout:(16/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.9 MB/s | 292 kB 00:00 2024-05-08T08:49:10.709 INFO:teuthology.orchestra.run.smithi059.stdout:(17/32): python3-prettytable-0.7.2-27.el9.noarc 242 kB/s | 42 kB 00:00 2024-05-08T08:49:10.843 INFO:teuthology.orchestra.run.smithi059.stdout:(18/32): qatlib-24.02.0-1.el9.x86_64.rpm 1.6 MB/s | 221 kB 00:00 2024-05-08T08:49:10.902 INFO:teuthology.orchestra.run.smithi059.stdout:(19/32): qatlib-service-24.02.0-1.el9.x86_64.rp 617 kB/s | 36 kB 00:00 2024-05-08T08:49:10.952 INFO:teuthology.orchestra.run.smithi059.stdout:(20/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 951 kB/s | 47 kB 00:00 2024-05-08T08:49:11.202 INFO:teuthology.orchestra.run.smithi059.stdout:(21/32): gperftools-libs-2.9.1-2.el9.x86_64.rpm 1.2 MB/s | 309 kB 00:00 2024-05-08T08:49:11.217 INFO:teuthology.orchestra.run.smithi192.stdout:(1/32): ceph-base-19.0.0-2515.g96805dcf.el9.x86 1.9 MB/s | 5.4 MB 00:02 2024-05-08T08:49:11.317 INFO:teuthology.orchestra.run.smithi192.stdout:(2/32): ceph-selinux-19.0.0-2515.g96805dcf.el9. 252 kB/s | 25 kB 00:00 2024-05-08T08:49:11.386 INFO:teuthology.orchestra.run.smithi059.stdout:(22/32): libarrow-9.0.0-11.el9.x86_64.rpm 24 MB/s | 4.4 MB 00:00 2024-05-08T08:49:11.436 INFO:teuthology.orchestra.run.smithi059.stdout:(23/32): libarrow-doc-9.0.0-11.el9.noarch.rpm 522 kB/s | 26 kB 00:00 2024-05-08T08:49:11.460 INFO:teuthology.orchestra.run.smithi196.stdout:lab-extras 18 kB/s | 1.7 kB 00:00 2024-05-08T08:49:11.486 INFO:teuthology.orchestra.run.smithi059.stdout:(24/32): liboath-2.6.7-2.el9.x86_64.rpm 971 kB/s | 49 kB 00:00 2024-05-08T08:49:11.537 INFO:teuthology.orchestra.run.smithi059.stdout:(25/32): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-05-08T08:49:11.604 INFO:teuthology.orchestra.run.smithi059.stdout:(26/32): parquet-libs-9.0.0-11.el9.x86_64.rpm 12 MB/s | 839 kB 00:00 2024-05-08T08:49:11.654 INFO:teuthology.orchestra.run.smithi059.stdout:(27/32): re2-20211101-3.el9.x86_64.rpm 3.8 MB/s | 192 kB 00:00 2024-05-08T08:49:11.684 INFO:teuthology.orchestra.run.smithi192.stdout:(3/32): ceph-radosgw-19.0.0-2515.g96805dcf.el9. 3.2 MB/s | 10 MB 00:03 2024-05-08T08:49:11.710 INFO:teuthology.orchestra.run.smithi192.stdout:(4/32): libcephfs2-19.0.0-2515.g96805dcf.el9.x8 1.9 MB/s | 746 kB 00:00 2024-05-08T08:49:11.729 INFO:teuthology.orchestra.run.smithi059.stdout:(28/32): thrift-0.15.0-2.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-05-08T08:49:11.843 INFO:teuthology.orchestra.run.smithi192.stdout:(5/32): libradosstriper1-19.0.0-2515.g96805dcf. 3.1 MB/s | 500 kB 00:00 2024-05-08T08:49:11.927 INFO:teuthology.orchestra.run.smithi192.stdout:(6/32): python3-ceph-argparse-19.0.0-2515.g9680 542 kB/s | 45 kB 00:00 2024-05-08T08:49:12.019 INFO:teuthology.orchestra.run.smithi192.stdout:(7/32): python3-ceph-common-19.0.0-2515.g96805d 1.4 MB/s | 135 kB 00:00 2024-05-08T08:49:12.128 INFO:teuthology.orchestra.run.smithi192.stdout:(8/32): python3-cephfs-19.0.0-2515.g96805dcf.el 1.5 MB/s | 162 kB 00:00 2024-05-08T08:49:12.529 INFO:teuthology.orchestra.run.smithi059.stdout:(29/32): librados2-19.0.0-2515.g96805dcf.el9.x8 4.2 MB/s | 3.4 MB 00:00 2024-05-08T08:49:12.928 INFO:teuthology.orchestra.run.smithi192.stdout:(9/32): python3-rados-19.0.0-2515.g96805dcf.el9 402 kB/s | 321 kB 00:00 2024-05-08T08:49:13.095 INFO:teuthology.orchestra.run.smithi192.stdout:(10/32): python3-rbd-19.0.0-2515.g96805dcf.el9. 1.8 MB/s | 298 kB 00:00 2024-05-08T08:49:13.104 INFO:teuthology.orchestra.run.smithi059.stdout:(30/32): librbd1-19.0.0-2515.g96805dcf.el9.x86_ 5.5 MB/s | 3.1 MB 00:00 2024-05-08T08:49:13.178 INFO:teuthology.orchestra.run.smithi192.stdout:(11/32): python3-rgw-19.0.0-2515.g96805dcf.el9. 1.2 MB/s | 99 kB 00:00 2024-05-08T08:49:13.312 INFO:teuthology.orchestra.run.smithi192.stdout:(12/32): boost-program-options-1.75.0-8.el9.x86 803 kB/s | 107 kB 00:00 2024-05-08T08:49:13.379 INFO:teuthology.orchestra.run.smithi192.stdout:(13/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.3 MB/s | 160 kB 00:00 2024-05-08T08:49:13.429 INFO:teuthology.orchestra.run.smithi192.stdout:(14/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 909 kB/s | 45 kB 00:00 2024-05-08T08:49:13.496 INFO:teuthology.orchestra.run.smithi192.stdout:(15/32): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 662 kB 00:00 2024-05-08T08:49:13.546 INFO:teuthology.orchestra.run.smithi192.stdout:(16/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-05-08T08:49:13.580 INFO:teuthology.orchestra.run.smithi192.stdout:(17/32): python3-prettytable-0.7.2-27.el9.noarc 1.2 MB/s | 42 kB 00:00 2024-05-08T08:49:13.631 INFO:teuthology.orchestra.run.smithi192.stdout:(18/32): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-05-08T08:49:13.664 INFO:teuthology.orchestra.run.smithi192.stdout:(19/32): qatlib-service-24.02.0-1.el9.x86_64.rp 1.1 MB/s | 36 kB 00:00 2024-05-08T08:49:13.698 INFO:teuthology.orchestra.run.smithi192.stdout:(20/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-05-08T08:49:13.773 INFO:teuthology.orchestra.run.smithi192.stdout:(21/32): librgw2-19.0.0-2515.g96805dcf.el9.x86_ 2.6 MB/s | 5.3 MB 00:02 2024-05-08T08:49:13.810 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-05-08T08:49:13.813 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-05-08T08:49:13.814 INFO:teuthology.orchestra.run.smithi196.stdout: Package Arch Version Repository Size 2024-05-08T08:49:13.814 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-05-08T08:49:13.814 INFO:teuthology.orchestra.run.smithi196.stdout:Installing: 2024-05-08T08:49:13.814 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-radosgw x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 10 M 2024-05-08T08:49:13.814 INFO:teuthology.orchestra.run.smithi196.stdout:Upgrading: 2024-05-08T08:49:13.814 INFO:teuthology.orchestra.run.smithi196.stdout: librados2 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 3.4 M 2024-05-08T08:49:13.814 INFO:teuthology.orchestra.run.smithi196.stdout: librbd1 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 3.1 M 2024-05-08T08:49:13.814 INFO:teuthology.orchestra.run.smithi196.stdout:Installing dependencies: 2024-05-08T08:49:13.815 INFO:teuthology.orchestra.run.smithi196.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-08T08:49:13.815 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-base x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 5.4 M 2024-05-08T08:49:13.815 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-common x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 21 M 2024-05-08T08:49:13.815 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-selinux x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 25 k 2024-05-08T08:49:13.815 INFO:teuthology.orchestra.run.smithi196.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-08T08:49:13.815 INFO:teuthology.orchestra.run.smithi196.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-08T08:49:13.815 INFO:teuthology.orchestra.run.smithi196.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-08T08:49:13.815 INFO:teuthology.orchestra.run.smithi196.stdout: libcephfs2 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 746 k 2024-05-08T08:49:13.815 INFO:teuthology.orchestra.run.smithi196.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-08T08:49:13.815 INFO:teuthology.orchestra.run.smithi196.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-08T08:49:13.815 INFO:teuthology.orchestra.run.smithi196.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-08T08:49:13.816 INFO:teuthology.orchestra.run.smithi196.stdout: libradosstriper1 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 500 k 2024-05-08T08:49:13.816 INFO:teuthology.orchestra.run.smithi196.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-08T08:49:13.816 INFO:teuthology.orchestra.run.smithi196.stdout: librgw2 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 5.3 M 2024-05-08T08:49:13.816 INFO:teuthology.orchestra.run.smithi196.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-08T08:49:13.816 INFO:teuthology.orchestra.run.smithi196.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-08T08:49:13.816 INFO:teuthology.orchestra.run.smithi196.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-08T08:49:13.816 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ceph-argparse x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 45 k 2024-05-08T08:49:13.816 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ceph-common x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 135 k 2024-05-08T08:49:13.816 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cephfs x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 162 k 2024-05-08T08:49:13.816 INFO:teuthology.orchestra.run.smithi196.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-08T08:49:13.816 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rados x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 321 k 2024-05-08T08:49:13.816 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rbd x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 298 k 2024-05-08T08:49:13.816 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rgw x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 99 k 2024-05-08T08:49:13.816 INFO:teuthology.orchestra.run.smithi196.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-08T08:49:13.817 INFO:teuthology.orchestra.run.smithi196.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-08T08:49:13.817 INFO:teuthology.orchestra.run.smithi196.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-08T08:49:13.817 INFO:teuthology.orchestra.run.smithi196.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-08T08:49:13.817 INFO:teuthology.orchestra.run.smithi196.stdout:Installing weak dependencies: 2024-05-08T08:49:13.817 INFO:teuthology.orchestra.run.smithi196.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-08T08:49:13.817 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:49:13.817 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-05-08T08:49:13.817 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-05-08T08:49:13.817 INFO:teuthology.orchestra.run.smithi196.stdout:Install 30 Packages 2024-05-08T08:49:13.817 INFO:teuthology.orchestra.run.smithi196.stdout:Upgrade 2 Packages 2024-05-08T08:49:13.817 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:49:13.818 INFO:teuthology.orchestra.run.smithi196.stdout:Total download size: 60 M 2024-05-08T08:49:13.818 INFO:teuthology.orchestra.run.smithi196.stdout:Downloading Packages: 2024-05-08T08:49:14.424 INFO:teuthology.orchestra.run.smithi192.stdout:(22/32): ceph-common-19.0.0-2515.g96805dcf.el9. 3.5 MB/s | 21 MB 00:05 2024-05-08T08:49:14.730 INFO:teuthology.orchestra.run.smithi059.stdout:(31/32): ceph-common-19.0.0-2515.g96805dcf.el9. 3.0 MB/s | 21 MB 00:07 2024-05-08T08:49:14.774 INFO:teuthology.orchestra.run.smithi192.stdout:(23/32): gperftools-libs-2.9.1-2.el9.x86_64.rpm 287 kB/s | 309 kB 00:01 2024-05-08T08:49:14.874 INFO:teuthology.orchestra.run.smithi192.stdout:(24/32): libarrow-doc-9.0.0-11.el9.noarch.rpm 58 kB/s | 26 kB 00:00 2024-05-08T08:49:14.899 INFO:teuthology.orchestra.run.smithi192.stdout:(25/32): liboath-2.6.7-2.el9.x86_64.rpm 388 kB/s | 49 kB 00:00 2024-05-08T08:49:15.149 INFO:teuthology.orchestra.run.smithi192.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 245 kB/s | 67 kB 00:00 2024-05-08T08:49:15.361 INFO:teuthology.orchestra.run.smithi196.stdout:(1/32): ceph-base-19.0.0-2515.g96805dcf.el9.x86 5.0 MB/s | 5.4 MB 00:01 2024-05-08T08:49:15.436 INFO:teuthology.orchestra.run.smithi196.stdout:(2/32): ceph-selinux-19.0.0-2515.g96805dcf.el9. 336 kB/s | 25 kB 00:00 2024-05-08T08:49:15.491 INFO:teuthology.orchestra.run.smithi192.stdout:(27/32): re2-20211101-3.el9.x86_64.rpm 561 kB/s | 192 kB 00:00 2024-05-08T08:49:15.580 INFO:teuthology.orchestra.run.smithi059.stdout:(32/32): ceph-radosgw-19.0.0-2515.g96805dcf.el9 1.3 MB/s | 10 MB 00:07 2024-05-08T08:49:15.584 INFO:teuthology.orchestra.run.smithi059.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:49:15.584 INFO:teuthology.orchestra.run.smithi059.stdout:Total 7.4 MB/s | 60 MB 00:08 2024-05-08T08:49:15.595 INFO:teuthology.orchestra.run.smithi196.stdout:(3/32): libcephfs2-19.0.0-2515.g96805dcf.el9.x8 4.6 MB/s | 746 kB 00:00 2024-05-08T08:49:15.694 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction check 2024-05-08T08:49:15.695 INFO:teuthology.orchestra.run.smithi196.stdout:(4/32): libradosstriper1-19.0.0-2515.g96805dcf. 4.9 MB/s | 500 kB 00:00 2024-05-08T08:49:15.735 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction check succeeded. 2024-05-08T08:49:15.735 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction test 2024-05-08T08:49:15.975 INFO:teuthology.orchestra.run.smithi192.stdout:(28/32): parquet-libs-9.0.0-11.el9.x86_64.rpm 780 kB/s | 839 kB 00:01 2024-05-08T08:49:16.375 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction test succeeded. 2024-05-08T08:49:16.376 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction 2024-05-08T08:49:16.412 INFO:teuthology.orchestra.run.smithi196.stdout:(5/32): librgw2-19.0.0-2515.g96805dcf.el9.x86_6 7.4 MB/s | 5.3 MB 00:00 2024-05-08T08:49:16.487 INFO:teuthology.orchestra.run.smithi196.stdout:(6/32): python3-ceph-argparse-19.0.0-2515.g9680 602 kB/s | 45 kB 00:00 2024-05-08T08:49:16.613 INFO:teuthology.orchestra.run.smithi196.stdout:(7/32): python3-ceph-common-19.0.0-2515.g96805d 1.1 MB/s | 135 kB 00:00 2024-05-08T08:49:16.688 INFO:teuthology.orchestra.run.smithi196.stdout:(8/32): python3-cephfs-19.0.0-2515.g96805dcf.el 2.1 MB/s | 162 kB 00:00 2024-05-08T08:49:16.771 INFO:teuthology.orchestra.run.smithi196.stdout:(9/32): python3-rados-19.0.0-2515.g96805dcf.el9 3.8 MB/s | 321 kB 00:00 2024-05-08T08:49:16.855 INFO:teuthology.orchestra.run.smithi196.stdout:(10/32): python3-rbd-19.0.0-2515.g96805dcf.el9. 3.5 MB/s | 298 kB 00:00 2024-05-08T08:49:16.930 INFO:teuthology.orchestra.run.smithi196.stdout:(11/32): python3-rgw-19.0.0-2515.g96805dcf.el9. 1.3 MB/s | 99 kB 00:00 2024-05-08T08:49:17.214 INFO:teuthology.orchestra.run.smithi196.stdout:(12/32): ceph-radosgw-19.0.0-2515.g96805dcf.el9 3.5 MB/s | 10 MB 00:02 2024-05-08T08:49:17.450 INFO:teuthology.orchestra.run.smithi192.stdout:(29/32): thrift-0.15.0-2.el9.x86_64.rpm 827 kB/s | 1.6 MB 00:01 2024-05-08T08:49:17.772 INFO:teuthology.orchestra.run.smithi196.stdout:(13/32): boost-program-options-1.75.0-8.el9.x86 127 kB/s | 107 kB 00:00 2024-05-08T08:49:17.782 INFO:teuthology.orchestra.run.smithi059.stdout: Preparing : 1/1 2024-05-08T08:49:17.873 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-05-08T08:49:17.902 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-05-08T08:49:17.927 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-05-08T08:49:17.950 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-05-08T08:49:17.978 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-05-08T08:49:18.089 INFO:teuthology.orchestra.run.smithi196.stdout:(14/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 188 kB/s | 160 kB 00:00 2024-05-08T08:49:18.162 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/34 2024-05-08T08:49:18.222 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/34 2024-05-08T08:49:18.417 INFO:teuthology.orchestra.run.smithi192.stdout:(30/32): librbd1-19.0.0-2515.g96805dcf.el9.x86_ 3.2 MB/s | 3.1 MB 00:00 2024-05-08T08:49:18.457 INFO:teuthology.orchestra.run.smithi196.stdout:(15/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 66 kB/s | 45 kB 00:00 2024-05-08T08:49:18.640 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-05-08T08:49:18.695 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 7/34 2024-05-08T08:49:18.717 INFO:teuthology.orchestra.run.smithi192.stdout:(31/32): librados2-19.0.0-2515.g96805dcf.el9.x8 1.2 MB/s | 3.4 MB 00:02 2024-05-08T08:49:18.788 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 7/34 2024-05-08T08:49:18.878 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-rados-2:19.0.0-2515.g96805dcf.el9.x86_64 8/34 2024-05-08T08:49:18.905 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 9/34 2024-05-08T08:49:18.948 INFO:teuthology.orchestra.run.smithi196.stdout:(16/32): ceph-common-19.0.0-2515.g96805dcf.el9. 4.5 MB/s | 21 MB 00:04 2024-05-08T08:49:18.951 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 9/34 2024-05-08T08:49:18.988 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-05-08T08:49:19.033 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-ceph-argparse-2:19.0.0-2515.g96805dcf.el9. 11/34 2024-05-08T08:49:19.102 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cephfs-2:19.0.0-2515.g96805dcf.el9.x86_64 12/34 2024-05-08T08:49:19.124 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86_6 13/34 2024-05-08T08:49:19.172 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86_6 13/34 2024-05-08T08:49:19.206 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/34 2024-05-08T08:49:19.265 INFO:teuthology.orchestra.run.smithi196.stdout:(17/32): librdkafka-1.6.1-102.el9.x86_64.rpm 564 kB/s | 662 kB 00:01 2024-05-08T08:49:19.319 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : re2-1:20211101-3.el9.x86_64 15/34 2024-05-08T08:49:19.349 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/34 2024-05-08T08:49:19.391 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-05-08T08:49:19.430 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 18/34 2024-05-08T08:49:19.460 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/34 2024-05-08T08:49:19.492 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-05-08T08:49:19.507 INFO:teuthology.orchestra.run.smithi196.stdout:(18/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 279 kB/s | 292 kB 00:01 2024-05-08T08:49:19.592 INFO:teuthology.orchestra.run.smithi192.stdout:(32/32): libarrow-9.0.0-11.el9.x86_64.rpm 777 kB/s | 4.4 MB 00:05 2024-05-08T08:49:19.598 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:49:19.598 INFO:teuthology.orchestra.run.smithi192.stdout:Total 5.2 MB/s | 60 MB 00:11 2024-05-08T08:49:19.624 INFO:teuthology.orchestra.run.smithi196.stdout:(19/32): python3-prettytable-0.7.2-27.el9.noarc 63 kB/s | 42 kB 00:00 2024-05-08T08:49:19.702 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-05-08T08:49:19.738 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-05-08T08:49:19.739 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-05-08T08:49:19.837 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-05-08T08:49:19.880 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 22/34 2024-05-08T08:49:19.972 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 22/34 2024-05-08T08:49:20.002 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-rbd-2:19.0.0-2515.g96805dcf.el9.x86_64 23/34 2024-05-08T08:49:20.054 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-05-08T08:49:20.199 INFO:teuthology.orchestra.run.smithi196.stdout:(20/32): qatlib-24.02.0-1.el9.x86_64.rpm 236 kB/s | 221 kB 00:00 2024-05-08T08:49:20.225 INFO:teuthology.orchestra.run.smithi196.stdout:(21/32): qatlib-service-24.02.0-1.el9.x86_64.rp 50 kB/s | 36 kB 00:00 2024-05-08T08:49:20.317 INFO:teuthology.orchestra.run.smithi196.stdout:(22/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 68 kB/s | 47 kB 00:00 2024-05-08T08:49:20.363 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-05-08T08:49:20.363 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-05-08T08:49:20.459 INFO:teuthology.orchestra.run.smithi196.stdout:(23/32): libarrow-doc-9.0.0-11.el9.noarch.rpm 185 kB/s | 26 kB 00:00 2024-05-08T08:49:20.484 INFO:teuthology.orchestra.run.smithi196.stdout:(24/32): gperftools-libs-2.9.1-2.el9.x86_64.rpm 1.1 MB/s | 309 kB 00:00 2024-05-08T08:49:20.517 INFO:teuthology.orchestra.run.smithi196.stdout:(25/32): liboath-2.6.7-2.el9.x86_64.rpm 824 kB/s | 49 kB 00:00 2024-05-08T08:49:20.534 INFO:teuthology.orchestra.run.smithi196.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-05-08T08:49:20.554 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 25/34 2024-05-08T08:49:20.582 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 26/34 2024-05-08T08:49:20.593 INFO:teuthology.orchestra.run.smithi196.stdout:(27/32): re2-20211101-3.el9.x86_64.rpm 3.2 MB/s | 192 kB 00:00 2024-05-08T08:49:20.655 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 26/34 2024-05-08T08:49:20.677 INFO:teuthology.orchestra.run.smithi196.stdout:(28/32): parquet-libs-9.0.0-11.el9.x86_64.rpm 5.2 MB/s | 839 kB 00:00 2024-05-08T08:49:20.736 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-rgw-2:19.0.0-2515.g96805dcf.el9.x86_64 27/34 2024-05-08T08:49:20.768 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-ceph-common-2:19.0.0-2515.g96805dcf.el9.x8 28/34 2024-05-08T08:49:20.869 INFO:teuthology.orchestra.run.smithi196.stdout:(29/32): thrift-0.15.0-2.el9.x86_64.rpm 5.7 MB/s | 1.6 MB 00:00 2024-05-08T08:49:21.111 INFO:teuthology.orchestra.run.smithi196.stdout:(30/32): libarrow-9.0.0-11.el9.x86_64.rpm 5.0 MB/s | 4.4 MB 00:00 2024-05-08T08:49:21.252 INFO:teuthology.orchestra.run.smithi196.stdout:(31/32): librados2-19.0.0-2515.g96805dcf.el9.x8 5.9 MB/s | 3.4 MB 00:00 2024-05-08T08:49:21.519 INFO:teuthology.orchestra.run.smithi196.stdout:(32/32): librbd1-19.0.0-2515.g96805dcf.el9.x86_ 4.8 MB/s | 3.1 MB 00:00 2024-05-08T08:49:21.525 INFO:teuthology.orchestra.run.smithi196.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:49:21.525 INFO:teuthology.orchestra.run.smithi196.stdout:Total 7.7 MB/s | 60 MB 00:07 2024-05-08T08:49:21.629 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction check 2024-05-08T08:49:21.665 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction check succeeded. 2024-05-08T08:49:21.666 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction test 2024-05-08T08:49:21.876 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-05-08T08:49:21.952 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-05-08T08:49:21.982 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-05-08T08:49:22.015 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-05-08T08:49:22.046 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-05-08T08:49:22.071 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-05-08T08:49:22.214 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/34 2024-05-08T08:49:22.271 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/34 2024-05-08T08:49:22.305 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction test succeeded. 2024-05-08T08:49:22.305 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction 2024-05-08T08:49:22.648 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-05-08T08:49:22.699 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 7/34 2024-05-08T08:49:22.791 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 7/34 2024-05-08T08:49:22.877 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rados-2:19.0.0-2515.g96805dcf.el9.x86_64 8/34 2024-05-08T08:49:22.909 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 9/34 2024-05-08T08:49:22.955 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 9/34 2024-05-08T08:49:22.984 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-05-08T08:49:23.027 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-ceph-argparse-2:19.0.0-2515.g96805dcf.el9. 11/34 2024-05-08T08:49:23.095 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cephfs-2:19.0.0-2515.g96805dcf.el9.x86_64 12/34 2024-05-08T08:49:23.120 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86_6 13/34 2024-05-08T08:49:23.184 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86_6 13/34 2024-05-08T08:49:23.188 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 29/34 2024-05-08T08:49:23.218 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/34 2024-05-08T08:49:23.231 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 29/34 2024-05-08T08:49:23.334 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : re2-1:20211101-3.el9.x86_64 15/34 2024-05-08T08:49:23.361 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/34 2024-05-08T08:49:23.403 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-05-08T08:49:23.442 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 18/34 2024-05-08T08:49:23.472 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/34 2024-05-08T08:49:23.504 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-05-08T08:49:23.738 INFO:teuthology.orchestra.run.smithi196.stdout: Preparing : 1/1 2024-05-08T08:49:23.816 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-05-08T08:49:23.817 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-05-08T08:49:23.827 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 29/34 2024-05-08T08:49:23.849 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-05-08T08:49:23.859 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 22/34 2024-05-08T08:49:23.869 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 30/34 2024-05-08T08:49:23.874 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-05-08T08:49:23.897 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-05-08T08:49:23.924 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-05-08T08:49:23.941 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 22/34 2024-05-08T08:49:23.961 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 30/34 2024-05-08T08:49:23.961 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-08T08:49:23.961 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-08T08:49:23.961 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:49:23.981 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rbd-2:19.0.0-2515.g96805dcf.el9.x86_64 23/34 2024-05-08T08:49:24.003 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 31/34 2024-05-08T08:49:24.016 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-05-08T08:49:24.121 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/34 2024-05-08T08:49:24.177 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/34 2024-05-08T08:49:24.486 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 25/34 2024-05-08T08:49:24.512 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 26/34 2024-05-08T08:49:24.513 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-05-08T08:49:24.608 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 26/34 2024-05-08T08:49:24.617 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 7/34 2024-05-08T08:49:24.685 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rgw-2:19.0.0-2515.g96805dcf.el9.x86_64 27/34 2024-05-08T08:49:24.710 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 7/34 2024-05-08T08:49:24.714 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-ceph-common-2:19.0.0-2515.g96805dcf.el9.x8 28/34 2024-05-08T08:49:24.797 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-rados-2:19.0.0-2515.g96805dcf.el9.x86_64 8/34 2024-05-08T08:49:24.827 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 9/34 2024-05-08T08:49:24.890 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 9/34 2024-05-08T08:49:24.918 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-05-08T08:49:24.962 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-ceph-argparse-2:19.0.0-2515.g96805dcf.el9. 11/34 2024-05-08T08:49:25.030 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cephfs-2:19.0.0-2515.g96805dcf.el9.x86_64 12/34 2024-05-08T08:49:25.054 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86_6 13/34 2024-05-08T08:49:25.118 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86_6 13/34 2024-05-08T08:49:25.144 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/34 2024-05-08T08:49:25.243 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : re2-1:20211101-3.el9.x86_64 15/34 2024-05-08T08:49:25.279 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/34 2024-05-08T08:49:25.320 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-05-08T08:49:25.352 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 18/34 2024-05-08T08:49:25.381 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/34 2024-05-08T08:49:25.414 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-05-08T08:49:25.735 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-05-08T08:49:25.777 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 22/34 2024-05-08T08:49:25.867 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 22/34 2024-05-08T08:49:25.898 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-rbd-2:19.0.0-2515.g96805dcf.el9.x86_64 23/34 2024-05-08T08:49:25.934 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-05-08T08:49:26.403 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 25/34 2024-05-08T08:49:26.429 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 26/34 2024-05-08T08:49:26.500 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 26/34 2024-05-08T08:49:26.578 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-rgw-2:19.0.0-2515.g96805dcf.el9.x86_64 27/34 2024-05-08T08:49:26.606 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-ceph-common-2:19.0.0-2515.g96805dcf.el9.x8 28/34 2024-05-08T08:49:26.999 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 29/34 2024-05-08T08:49:27.036 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 29/34 2024-05-08T08:49:27.607 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 29/34 2024-05-08T08:49:27.648 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 30/34 2024-05-08T08:49:27.743 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 30/34 2024-05-08T08:49:27.743 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-08T08:49:27.743 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-08T08:49:27.743 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:49:27.775 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 31/34 2024-05-08T08:49:28.806 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 29/34 2024-05-08T08:49:28.853 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 29/34 2024-05-08T08:49:29.416 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 29/34 2024-05-08T08:49:29.456 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 30/34 2024-05-08T08:49:29.541 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 30/34 2024-05-08T08:49:29.541 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-08T08:49:29.541 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-08T08:49:29.541 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:49:29.583 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 31/34 2024-05-08T08:49:33.980 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 31/34 2024-05-08T08:49:33.980 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /sys 2024-05-08T08:49:33.980 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /proc 2024-05-08T08:49:33.980 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /mnt 2024-05-08T08:49:33.980 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /var/tmp 2024-05-08T08:49:33.980 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /home 2024-05-08T08:49:33.980 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /root 2024-05-08T08:49:33.980 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /tmp 2024-05-08T08:49:33.981 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:49:34.955 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 32/34 2024-05-08T08:49:34.982 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 32/34 2024-05-08T08:49:34.982 INFO:teuthology.orchestra.run.smithi059.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T08:49:34.983 INFO:teuthology.orchestra.run.smithi059.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-08T08:49:34.983 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-08T08:49:34.983 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-08T08:49:34.983 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:49:34.984 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-05-08T08:49:35.067 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-05-08T08:49:35.067 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-05-08T08:49:36.253 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-05-08T08:49:36.254 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 1/34 2024-05-08T08:49:36.254 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 2/34 2024-05-08T08:49:36.254 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 3/34 2024-05-08T08:49:36.254 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 4/34 2024-05-08T08:49:36.254 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 5/34 2024-05-08T08:49:36.254 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86_6 6/34 2024-05-08T08:49:36.254 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 7/34 2024-05-08T08:49:36.254 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2515.g96805dcf.el9. 8/34 2024-05-08T08:49:36.254 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-ceph-common-2:19.0.0-2515.g96805dcf.el9.x8 9/34 2024-05-08T08:49:36.254 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cephfs-2:19.0.0-2515.g96805dcf.el9.x86_64 10/34 2024-05-08T08:49:36.254 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-rados-2:19.0.0-2515.g96805dcf.el9.x86_64 11/34 2024-05-08T08:49:36.255 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-rbd-2:19.0.0-2515.g96805dcf.el9.x86_64 12/34 2024-05-08T08:49:36.255 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-rgw-2:19.0.0-2515.g96805dcf.el9.x86_64 13/34 2024-05-08T08:49:36.255 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-05-08T08:49:36.255 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-05-08T08:49:36.255 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-05-08T08:49:36.255 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-05-08T08:49:36.255 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-05-08T08:49:36.255 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-05-08T08:49:36.255 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-05-08T08:49:36.255 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-05-08T08:49:36.255 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-05-08T08:49:36.255 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 23/34 2024-05-08T08:49:36.255 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 24/34 2024-05-08T08:49:36.255 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 25/34 2024-05-08T08:49:36.255 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-05-08T08:49:36.256 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-05-08T08:49:36.256 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 28/34 2024-05-08T08:49:36.256 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : re2-1:20211101-3.el9.x86_64 29/34 2024-05-08T08:49:36.256 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-05-08T08:49:36.256 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 31/34 2024-05-08T08:49:36.256 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-05-08T08:49:36.256 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 33/34 2024-05-08T08:49:37.184 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-05-08T08:49:37.184 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:49:37.184 INFO:teuthology.orchestra.run.smithi059.stdout:Upgraded: 2024-05-08T08:49:37.185 INFO:teuthology.orchestra.run.smithi059.stdout: librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:49:37.185 INFO:teuthology.orchestra.run.smithi059.stdout: librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:49:37.185 INFO:teuthology.orchestra.run.smithi059.stdout:Installed: 2024-05-08T08:49:37.185 INFO:teuthology.orchestra.run.smithi059.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-08T08:49:37.185 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:49:37.185 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:49:37.185 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:49:37.185 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:49:37.185 INFO:teuthology.orchestra.run.smithi059.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-08T08:49:37.185 INFO:teuthology.orchestra.run.smithi059.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-08T08:49:37.185 INFO:teuthology.orchestra.run.smithi059.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-08T08:49:37.185 INFO:teuthology.orchestra.run.smithi059.stdout: libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:49:37.185 INFO:teuthology.orchestra.run.smithi059.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-08T08:49:37.185 INFO:teuthology.orchestra.run.smithi059.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-08T08:49:37.186 INFO:teuthology.orchestra.run.smithi059.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-08T08:49:37.186 INFO:teuthology.orchestra.run.smithi059.stdout: libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:49:37.186 INFO:teuthology.orchestra.run.smithi059.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-08T08:49:37.186 INFO:teuthology.orchestra.run.smithi059.stdout: librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:49:37.186 INFO:teuthology.orchestra.run.smithi059.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-08T08:49:37.186 INFO:teuthology.orchestra.run.smithi059.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-08T08:49:37.186 INFO:teuthology.orchestra.run.smithi059.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-08T08:49:37.186 INFO:teuthology.orchestra.run.smithi059.stdout: python3-ceph-argparse-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:49:37.186 INFO:teuthology.orchestra.run.smithi059.stdout: python3-ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:49:37.186 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cephfs-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:49:37.186 INFO:teuthology.orchestra.run.smithi059.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-08T08:49:37.186 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rados-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:49:37.186 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rbd-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:49:37.186 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rgw-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:49:37.187 INFO:teuthology.orchestra.run.smithi059.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-08T08:49:37.187 INFO:teuthology.orchestra.run.smithi059.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-08T08:49:37.187 INFO:teuthology.orchestra.run.smithi059.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-08T08:49:37.187 INFO:teuthology.orchestra.run.smithi059.stdout: re2-1:20211101-3.el9.x86_64 2024-05-08T08:49:37.187 INFO:teuthology.orchestra.run.smithi059.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-08T08:49:37.187 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:49:37.187 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2024-05-08T08:49:37.308 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 31/34 2024-05-08T08:49:37.308 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /sys 2024-05-08T08:49:37.309 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /proc 2024-05-08T08:49:37.309 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /mnt 2024-05-08T08:49:37.309 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /var/tmp 2024-05-08T08:49:37.309 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /home 2024-05-08T08:49:37.309 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /root 2024-05-08T08:49:37.309 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /tmp 2024-05-08T08:49:37.309 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:49:37.522 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum -y install ceph-test 2024-05-08T08:49:38.112 INFO:teuthology.orchestra.run.smithi059.stdout:Last metadata expiration check: 0:00:33 ago on Wed 08 May 2024 08:49:05 AM UTC. 2024-05-08T08:49:38.238 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies resolved. 2024-05-08T08:49:38.239 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-05-08T08:49:38.239 INFO:teuthology.orchestra.run.smithi059.stdout: Package Arch Version Repository Size 2024-05-08T08:49:38.239 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-05-08T08:49:38.239 INFO:teuthology.orchestra.run.smithi059.stdout:Installing: 2024-05-08T08:49:38.239 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-test x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 48 M 2024-05-08T08:49:38.239 INFO:teuthology.orchestra.run.smithi059.stdout:Installing dependencies: 2024-05-08T08:49:38.239 INFO:teuthology.orchestra.run.smithi059.stdout: libcephsqlite x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 163 k 2024-05-08T08:49:38.239 INFO:teuthology.orchestra.run.smithi059.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-08T08:49:38.239 INFO:teuthology.orchestra.run.smithi059.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-08T08:49:38.239 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:49:38.239 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction Summary 2024-05-08T08:49:38.239 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-05-08T08:49:38.239 INFO:teuthology.orchestra.run.smithi059.stdout:Install 4 Packages 2024-05-08T08:49:38.240 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:49:38.240 INFO:teuthology.orchestra.run.smithi059.stdout:Total download size: 49 M 2024-05-08T08:49:38.240 INFO:teuthology.orchestra.run.smithi059.stdout:Installed size: 207 M 2024-05-08T08:49:38.240 INFO:teuthology.orchestra.run.smithi059.stdout:Downloading Packages: 2024-05-08T08:49:38.261 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 32/34 2024-05-08T08:49:38.286 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 32/34 2024-05-08T08:49:38.286 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T08:49:38.287 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-08T08:49:38.287 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-08T08:49:38.287 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-08T08:49:38.287 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:49:38.288 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-05-08T08:49:38.372 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-05-08T08:49:38.372 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-05-08T08:49:38.505 INFO:teuthology.orchestra.run.smithi059.stdout:(1/4): socat-1.7.4.1-5.el9.x86_64.rpm 1.3 MB/s | 305 kB 00:00 2024-05-08T08:49:38.530 INFO:teuthology.orchestra.run.smithi059.stdout:(2/4): libcephsqlite-19.0.0-2515.g96805dcf.el9. 652 kB/s | 163 kB 00:00 2024-05-08T08:49:38.555 INFO:teuthology.orchestra.run.smithi059.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-05-08T08:49:39.086 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 31/34 2024-05-08T08:49:39.086 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /sys 2024-05-08T08:49:39.086 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /proc 2024-05-08T08:49:39.087 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /mnt 2024-05-08T08:49:39.087 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /var/tmp 2024-05-08T08:49:39.087 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /home 2024-05-08T08:49:39.087 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /root 2024-05-08T08:49:39.087 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /tmp 2024-05-08T08:49:39.087 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:49:39.543 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-05-08T08:49:39.543 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 1/34 2024-05-08T08:49:39.543 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 2/34 2024-05-08T08:49:39.543 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 3/34 2024-05-08T08:49:39.543 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 4/34 2024-05-08T08:49:39.543 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 5/34 2024-05-08T08:49:39.544 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86_6 6/34 2024-05-08T08:49:39.544 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 7/34 2024-05-08T08:49:39.544 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2515.g96805dcf.el9. 8/34 2024-05-08T08:49:39.544 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-ceph-common-2:19.0.0-2515.g96805dcf.el9.x8 9/34 2024-05-08T08:49:39.544 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cephfs-2:19.0.0-2515.g96805dcf.el9.x86_64 10/34 2024-05-08T08:49:39.544 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rados-2:19.0.0-2515.g96805dcf.el9.x86_64 11/34 2024-05-08T08:49:39.544 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rbd-2:19.0.0-2515.g96805dcf.el9.x86_64 12/34 2024-05-08T08:49:39.544 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rgw-2:19.0.0-2515.g96805dcf.el9.x86_64 13/34 2024-05-08T08:49:39.544 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-05-08T08:49:39.544 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-05-08T08:49:39.544 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-05-08T08:49:39.544 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-05-08T08:49:39.544 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-05-08T08:49:39.544 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-05-08T08:49:39.544 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-05-08T08:49:39.545 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-05-08T08:49:39.545 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-05-08T08:49:39.545 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 23/34 2024-05-08T08:49:39.545 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 24/34 2024-05-08T08:49:39.545 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 25/34 2024-05-08T08:49:39.545 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-05-08T08:49:39.545 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-05-08T08:49:39.545 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 28/34 2024-05-08T08:49:39.545 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : re2-1:20211101-3.el9.x86_64 29/34 2024-05-08T08:49:39.545 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-05-08T08:49:39.545 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 31/34 2024-05-08T08:49:39.545 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-05-08T08:49:39.545 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 33/34 2024-05-08T08:49:40.059 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 32/34 2024-05-08T08:49:40.084 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 32/34 2024-05-08T08:49:40.084 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T08:49:40.085 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-08T08:49:40.085 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-08T08:49:40.085 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-08T08:49:40.085 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:49:40.087 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-05-08T08:49:40.170 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-05-08T08:49:40.170 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-05-08T08:49:40.372 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-05-08T08:49:40.373 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:49:40.373 INFO:teuthology.orchestra.run.smithi192.stdout:Upgraded: 2024-05-08T08:49:40.373 INFO:teuthology.orchestra.run.smithi192.stdout: librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:49:40.373 INFO:teuthology.orchestra.run.smithi192.stdout: librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:49:40.373 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-05-08T08:49:40.373 INFO:teuthology.orchestra.run.smithi192.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-08T08:49:40.373 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:49:40.373 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:49:40.373 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:49:40.373 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:49:40.373 INFO:teuthology.orchestra.run.smithi192.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-08T08:49:40.373 INFO:teuthology.orchestra.run.smithi192.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-08T08:49:40.374 INFO:teuthology.orchestra.run.smithi192.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-08T08:49:40.374 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:49:40.374 INFO:teuthology.orchestra.run.smithi192.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-08T08:49:40.374 INFO:teuthology.orchestra.run.smithi192.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-08T08:49:40.374 INFO:teuthology.orchestra.run.smithi192.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-08T08:49:40.374 INFO:teuthology.orchestra.run.smithi192.stdout: libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:49:40.374 INFO:teuthology.orchestra.run.smithi192.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-08T08:49:40.374 INFO:teuthology.orchestra.run.smithi192.stdout: librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:49:40.374 INFO:teuthology.orchestra.run.smithi192.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-08T08:49:40.374 INFO:teuthology.orchestra.run.smithi192.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-08T08:49:40.374 INFO:teuthology.orchestra.run.smithi192.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-08T08:49:40.374 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-argparse-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:49:40.374 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:49:40.375 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cephfs-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:49:40.375 INFO:teuthology.orchestra.run.smithi192.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-08T08:49:40.375 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rados-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:49:40.375 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rbd-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:49:40.375 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rgw-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:49:40.375 INFO:teuthology.orchestra.run.smithi192.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-08T08:49:40.375 INFO:teuthology.orchestra.run.smithi192.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-08T08:49:40.375 INFO:teuthology.orchestra.run.smithi192.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-08T08:49:40.375 INFO:teuthology.orchestra.run.smithi192.stdout: re2-1:20211101-3.el9.x86_64 2024-05-08T08:49:40.375 INFO:teuthology.orchestra.run.smithi192.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-08T08:49:40.375 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:49:40.375 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-05-08T08:49:40.724 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-test 2024-05-08T08:49:41.139 INFO:teuthology.orchestra.run.smithi059.stdout:(4/4): ceph-test-19.0.0-2515.g96805dcf.el9.x86_ 17 MB/s | 48 MB 00:02 2024-05-08T08:49:41.141 INFO:teuthology.orchestra.run.smithi059.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:49:41.141 INFO:teuthology.orchestra.run.smithi059.stdout:Total 17 MB/s | 49 MB 00:02 2024-05-08T08:49:41.170 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction check 2024-05-08T08:49:41.193 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction check succeeded. 2024-05-08T08:49:41.193 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction test 2024-05-08T08:49:41.298 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:00:36 ago on Wed 08 May 2024 08:49:05 AM UTC. 2024-05-08T08:49:41.311 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-05-08T08:49:41.311 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 1/34 2024-05-08T08:49:41.311 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 2/34 2024-05-08T08:49:41.311 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 3/34 2024-05-08T08:49:41.311 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 4/34 2024-05-08T08:49:41.311 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 5/34 2024-05-08T08:49:41.311 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86_6 6/34 2024-05-08T08:49:41.312 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 7/34 2024-05-08T08:49:41.312 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2515.g96805dcf.el9. 8/34 2024-05-08T08:49:41.312 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-ceph-common-2:19.0.0-2515.g96805dcf.el9.x8 9/34 2024-05-08T08:49:41.312 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cephfs-2:19.0.0-2515.g96805dcf.el9.x86_64 10/34 2024-05-08T08:49:41.312 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-rados-2:19.0.0-2515.g96805dcf.el9.x86_64 11/34 2024-05-08T08:49:41.312 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-rbd-2:19.0.0-2515.g96805dcf.el9.x86_64 12/34 2024-05-08T08:49:41.312 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-rgw-2:19.0.0-2515.g96805dcf.el9.x86_64 13/34 2024-05-08T08:49:41.312 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-05-08T08:49:41.312 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-05-08T08:49:41.312 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-05-08T08:49:41.312 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-05-08T08:49:41.312 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-05-08T08:49:41.312 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-05-08T08:49:41.312 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-05-08T08:49:41.313 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-05-08T08:49:41.313 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-05-08T08:49:41.313 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 23/34 2024-05-08T08:49:41.313 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 24/34 2024-05-08T08:49:41.313 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 25/34 2024-05-08T08:49:41.313 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-05-08T08:49:41.313 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-05-08T08:49:41.313 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 28/34 2024-05-08T08:49:41.313 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : re2-1:20211101-3.el9.x86_64 29/34 2024-05-08T08:49:41.313 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-05-08T08:49:41.313 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 31/34 2024-05-08T08:49:41.313 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-05-08T08:49:41.313 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 33/34 2024-05-08T08:49:41.419 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-05-08T08:49:41.420 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-05-08T08:49:41.420 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2024-05-08T08:49:41.420 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-05-08T08:49:41.420 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-05-08T08:49:41.420 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-test x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 48 M 2024-05-08T08:49:41.421 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2024-05-08T08:49:41.421 INFO:teuthology.orchestra.run.smithi192.stdout: libcephsqlite x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 163 k 2024-05-08T08:49:41.421 INFO:teuthology.orchestra.run.smithi192.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-08T08:49:41.421 INFO:teuthology.orchestra.run.smithi192.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-08T08:49:41.421 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:49:41.421 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-05-08T08:49:41.421 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-05-08T08:49:41.421 INFO:teuthology.orchestra.run.smithi192.stdout:Install 4 Packages 2024-05-08T08:49:41.421 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:49:41.422 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 49 M 2024-05-08T08:49:41.422 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 207 M 2024-05-08T08:49:41.422 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-05-08T08:49:41.581 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction test succeeded. 2024-05-08T08:49:41.581 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction 2024-05-08T08:49:41.845 INFO:teuthology.orchestra.run.smithi192.stdout:(1/4): socat-1.7.4.1-5.el9.x86_64.rpm 994 kB/s | 305 kB 00:00 2024-05-08T08:49:41.904 INFO:teuthology.orchestra.run.smithi192.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-05-08T08:49:41.929 INFO:teuthology.orchestra.run.smithi192.stdout:(3/4): libcephsqlite-19.0.0-2515.g96805dcf.el9. 418 kB/s | 163 kB 00:00 2024-05-08T08:49:42.122 INFO:teuthology.orchestra.run.smithi059.stdout: Preparing : 1/1 2024-05-08T08:49:42.176 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-05-08T08:49:42.177 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:49:42.177 INFO:teuthology.orchestra.run.smithi196.stdout:Upgraded: 2024-05-08T08:49:42.177 INFO:teuthology.orchestra.run.smithi196.stdout: librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:49:42.177 INFO:teuthology.orchestra.run.smithi196.stdout: librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:49:42.177 INFO:teuthology.orchestra.run.smithi196.stdout:Installed: 2024-05-08T08:49:42.177 INFO:teuthology.orchestra.run.smithi196.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-08T08:49:42.177 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:49:42.177 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:49:42.177 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:49:42.177 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:49:42.177 INFO:teuthology.orchestra.run.smithi196.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-08T08:49:42.177 INFO:teuthology.orchestra.run.smithi196.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-08T08:49:42.177 INFO:teuthology.orchestra.run.smithi196.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-08T08:49:42.178 INFO:teuthology.orchestra.run.smithi196.stdout: libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:49:42.178 INFO:teuthology.orchestra.run.smithi196.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-08T08:49:42.178 INFO:teuthology.orchestra.run.smithi196.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-08T08:49:42.178 INFO:teuthology.orchestra.run.smithi196.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-08T08:49:42.178 INFO:teuthology.orchestra.run.smithi196.stdout: libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:49:42.178 INFO:teuthology.orchestra.run.smithi196.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-08T08:49:42.178 INFO:teuthology.orchestra.run.smithi196.stdout: librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:49:42.178 INFO:teuthology.orchestra.run.smithi196.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-08T08:49:42.178 INFO:teuthology.orchestra.run.smithi196.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-08T08:49:42.178 INFO:teuthology.orchestra.run.smithi196.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-08T08:49:42.178 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ceph-argparse-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:49:42.178 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:49:42.178 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cephfs-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:49:42.178 INFO:teuthology.orchestra.run.smithi196.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-08T08:49:42.178 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rados-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:49:42.179 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rbd-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:49:42.179 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rgw-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:49:42.179 INFO:teuthology.orchestra.run.smithi196.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-08T08:49:42.179 INFO:teuthology.orchestra.run.smithi196.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-08T08:49:42.179 INFO:teuthology.orchestra.run.smithi196.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-08T08:49:42.179 INFO:teuthology.orchestra.run.smithi196.stdout: re2-1:20211101-3.el9.x86_64 2024-05-08T08:49:42.179 INFO:teuthology.orchestra.run.smithi196.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-08T08:49:42.179 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:49:42.179 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-05-08T08:49:42.209 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-05-08T08:49:42.264 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/4 2024-05-08T08:49:42.291 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 3/4 2024-05-08T08:49:42.558 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install ceph-test 2024-05-08T08:49:43.126 INFO:teuthology.orchestra.run.smithi196.stdout:Last metadata expiration check: 0:00:32 ago on Wed 08 May 2024 08:49:11 AM UTC. 2024-05-08T08:49:43.247 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-05-08T08:49:43.247 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-05-08T08:49:43.247 INFO:teuthology.orchestra.run.smithi196.stdout: Package Arch Version Repository Size 2024-05-08T08:49:43.248 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-05-08T08:49:43.248 INFO:teuthology.orchestra.run.smithi196.stdout:Installing: 2024-05-08T08:49:43.248 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-test x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 48 M 2024-05-08T08:49:43.248 INFO:teuthology.orchestra.run.smithi196.stdout:Installing dependencies: 2024-05-08T08:49:43.248 INFO:teuthology.orchestra.run.smithi196.stdout: libcephsqlite x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 163 k 2024-05-08T08:49:43.248 INFO:teuthology.orchestra.run.smithi196.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-08T08:49:43.248 INFO:teuthology.orchestra.run.smithi196.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-08T08:49:43.248 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:49:43.248 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-05-08T08:49:43.248 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-05-08T08:49:43.248 INFO:teuthology.orchestra.run.smithi196.stdout:Install 4 Packages 2024-05-08T08:49:43.248 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:49:43.249 INFO:teuthology.orchestra.run.smithi196.stdout:Total download size: 49 M 2024-05-08T08:49:43.249 INFO:teuthology.orchestra.run.smithi196.stdout:Installed size: 207 M 2024-05-08T08:49:43.249 INFO:teuthology.orchestra.run.smithi196.stdout:Downloading Packages: 2024-05-08T08:49:43.707 INFO:teuthology.orchestra.run.smithi196.stdout:(1/4): socat-1.7.4.1-5.el9.x86_64.rpm 1.6 MB/s | 305 kB 00:00 2024-05-08T08:49:43.749 INFO:teuthology.orchestra.run.smithi196.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-05-08T08:49:43.862 INFO:teuthology.orchestra.run.smithi192.stdout:(4/4): ceph-test-19.0.0-2515.g96805dcf.el9.x86_ 21 MB/s | 48 MB 00:02 2024-05-08T08:49:43.864 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:49:43.864 INFO:teuthology.orchestra.run.smithi192.stdout:Total 20 MB/s | 49 MB 00:02 2024-05-08T08:49:43.894 INFO:teuthology.orchestra.run.smithi196.stdout:(3/4): libcephsqlite-19.0.0-2515.g96805dcf.el9. 435 kB/s | 163 kB 00:00 2024-05-08T08:49:43.894 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-05-08T08:49:43.913 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-05-08T08:49:43.914 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-05-08T08:49:44.279 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-05-08T08:49:44.279 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-05-08T08:49:44.803 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-05-08T08:49:44.890 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-05-08T08:49:44.928 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/4 2024-05-08T08:49:44.947 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 3/4 2024-05-08T08:49:45.841 INFO:teuthology.orchestra.run.smithi196.stdout:(4/4): ceph-test-19.0.0-2515.g96805dcf.el9.x86_ 21 MB/s | 48 MB 00:02 2024-05-08T08:49:45.843 INFO:teuthology.orchestra.run.smithi196.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:49:45.843 INFO:teuthology.orchestra.run.smithi196.stdout:Total 19 MB/s | 49 MB 00:02 2024-05-08T08:49:45.871 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction check 2024-05-08T08:49:45.892 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction check succeeded. 2024-05-08T08:49:45.892 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction test 2024-05-08T08:49:46.258 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction test succeeded. 2024-05-08T08:49:46.259 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction 2024-05-08T08:49:46.773 INFO:teuthology.orchestra.run.smithi196.stdout: Preparing : 1/1 2024-05-08T08:49:46.863 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-05-08T08:49:46.912 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/4 2024-05-08T08:49:46.934 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 3/4 2024-05-08T08:49:47.118 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 3/4 2024-05-08T08:49:47.191 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-test-2:19.0.0-2515.g96805dcf.el9.x86_64 4/4 2024-05-08T08:49:49.443 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-test-2:19.0.0-2515.g96805dcf.el9.x86_64 4/4 2024-05-08T08:49:49.443 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-test-2:19.0.0-2515.g96805dcf.el9.x86_64 1/4 2024-05-08T08:49:49.443 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 2/4 2024-05-08T08:49:49.443 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/4 2024-05-08T08:49:49.502 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 3/4 2024-05-08T08:49:49.579 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-test-2:19.0.0-2515.g96805dcf.el9.x86_64 4/4 2024-05-08T08:49:49.718 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-05-08T08:49:49.718 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:49:49.718 INFO:teuthology.orchestra.run.smithi059.stdout:Installed: 2024-05-08T08:49:49.718 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-test-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:49:49.718 INFO:teuthology.orchestra.run.smithi059.stdout: libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:49:49.718 INFO:teuthology.orchestra.run.smithi059.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-08T08:49:49.718 INFO:teuthology.orchestra.run.smithi059.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-08T08:49:49.719 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:49:49.719 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2024-05-08T08:49:49.999 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum -y install ceph 2024-05-08T08:49:50.592 INFO:teuthology.orchestra.run.smithi059.stdout:Last metadata expiration check: 0:00:45 ago on Wed 08 May 2024 08:49:05 AM UTC. 2024-05-08T08:49:50.724 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies resolved. 2024-05-08T08:49:50.726 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-05-08T08:49:50.726 INFO:teuthology.orchestra.run.smithi059.stdout: Package Arch Version Repository Size 2024-05-08T08:49:50.727 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-05-08T08:49:50.727 INFO:teuthology.orchestra.run.smithi059.stdout:Installing: 2024-05-08T08:49:50.727 INFO:teuthology.orchestra.run.smithi059.stdout: ceph x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 6.6 k 2024-05-08T08:49:50.727 INFO:teuthology.orchestra.run.smithi059.stdout:Installing dependencies: 2024-05-08T08:49:50.727 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mds x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 2.4 M 2024-05-08T08:49:50.727 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 1.5 M 2024-05-08T08:49:50.727 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 249 k 2024-05-08T08:49:50.727 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mon x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 4.7 M 2024-05-08T08:49:50.727 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-osd x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 17 M 2024-05-08T08:49:50.727 INFO:teuthology.orchestra.run.smithi059.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-08T08:49:50.727 INFO:teuthology.orchestra.run.smithi059.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-08T08:49:50.727 INFO:teuthology.orchestra.run.smithi059.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-08T08:49:50.727 INFO:teuthology.orchestra.run.smithi059.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-08T08:49:50.728 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-08T08:49:50.728 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-08T08:49:50.728 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-08T08:49:50.728 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-08T08:49:50.728 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-08T08:49:50.728 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-08T08:49:50.728 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-collections 2024-05-08T08:49:50.728 INFO:teuthology.orchestra.run.smithi059.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-05-08T08:49:50.728 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-08T08:49:50.728 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-08T08:49:50.728 INFO:teuthology.orchestra.run.smithi059.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-08T08:49:50.728 INFO:teuthology.orchestra.run.smithi059.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-08T08:49:50.728 INFO:teuthology.orchestra.run.smithi059.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-08T08:49:50.728 INFO:teuthology.orchestra.run.smithi059.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-08T08:49:50.729 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-08T08:49:50.729 INFO:teuthology.orchestra.run.smithi059.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-08T08:49:50.729 INFO:teuthology.orchestra.run.smithi059.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-08T08:49:50.729 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-08T08:49:50.729 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-08T08:49:50.729 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-08T08:49:50.729 INFO:teuthology.orchestra.run.smithi059.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-08T08:49:50.729 INFO:teuthology.orchestra.run.smithi059.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-08T08:49:50.729 INFO:teuthology.orchestra.run.smithi059.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-08T08:49:50.729 INFO:teuthology.orchestra.run.smithi059.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-08T08:49:50.729 INFO:teuthology.orchestra.run.smithi059.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-08T08:49:50.729 INFO:teuthology.orchestra.run.smithi059.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-08T08:49:50.729 INFO:teuthology.orchestra.run.smithi059.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-08T08:49:50.729 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:49:50.729 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction Summary 2024-05-08T08:49:50.730 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-05-08T08:49:50.730 INFO:teuthology.orchestra.run.smithi059.stdout:Install 36 Packages 2024-05-08T08:49:50.730 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:49:50.731 INFO:teuthology.orchestra.run.smithi059.stdout:Total download size: 30 M 2024-05-08T08:49:50.731 INFO:teuthology.orchestra.run.smithi059.stdout:Installed size: 107 M 2024-05-08T08:49:50.731 INFO:teuthology.orchestra.run.smithi059.stdout:Downloading Packages: 2024-05-08T08:49:51.400 INFO:teuthology.orchestra.run.smithi059.stdout:(1/36): ceph-19.0.0-2515.g96805dcf.el9.x86_64.r 32 kB/s | 6.6 kB 00:00 2024-05-08T08:49:51.524 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 3/4 2024-05-08T08:49:51.566 INFO:teuthology.orchestra.run.smithi059.stdout:(2/36): ceph-mgr-19.0.0-2515.g96805dcf.el9.x86_ 4.0 MB/s | 1.5 MB 00:00 2024-05-08T08:49:51.607 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-test-2:19.0.0-2515.g96805dcf.el9.x86_64 4/4 2024-05-08T08:49:51.942 INFO:teuthology.orchestra.run.smithi059.stdout:(3/36): ceph-mds-19.0.0-2515.g96805dcf.el9.x86_ 3.2 MB/s | 2.4 MB 00:00 2024-05-08T08:49:51.985 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-test-2:19.0.0-2515.g96805dcf.el9.x86_64 4/4 2024-05-08T08:49:51.985 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-test-2:19.0.0-2515.g96805dcf.el9.x86_64 1/4 2024-05-08T08:49:51.985 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 2/4 2024-05-08T08:49:51.985 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/4 2024-05-08T08:49:52.042 INFO:teuthology.orchestra.run.smithi059.stdout:(4/36): ceph-mgr-modules-core-19.0.0-2515.g9680 2.4 MB/s | 249 kB 00:00 2024-05-08T08:49:52.209 INFO:teuthology.orchestra.run.smithi059.stdout:(5/36): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.5 MB/s | 253 kB 00:00 2024-05-08T08:49:52.247 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-05-08T08:49:52.248 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:49:52.248 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-05-08T08:49:52.248 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-test-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:49:52.248 INFO:teuthology.orchestra.run.smithi192.stdout: libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:49:52.248 INFO:teuthology.orchestra.run.smithi192.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-08T08:49:52.248 INFO:teuthology.orchestra.run.smithi192.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-08T08:49:52.248 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:49:52.248 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-05-08T08:49:52.543 INFO:teuthology.orchestra.run.smithi059.stdout:(6/36): ceph-osd-19.0.0-2515.g96805dcf.el9.x86_ 17 MB/s | 17 MB 00:00 2024-05-08T08:49:52.565 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph 2024-05-08T08:49:52.601 INFO:teuthology.orchestra.run.smithi059.stdout:(7/36): ceph-mon-19.0.0-2515.g96805dcf.el9.x86_ 3.9 MB/s | 4.7 MB 00:01 2024-05-08T08:49:52.643 INFO:teuthology.orchestra.run.smithi059.stdout:(8/36): python3-cryptography-36.0.1-4.el9.x86_6 2.8 MB/s | 1.2 MB 00:00 2024-05-08T08:49:52.702 INFO:teuthology.orchestra.run.smithi059.stdout:(9/36): python3-requests-2.25.1-8.el9.noarch.rp 2.1 MB/s | 125 kB 00:00 2024-05-08T08:49:52.735 INFO:teuthology.orchestra.run.smithi059.stdout:(10/36): python3-ply-3.11-14.el9.noarch.rpm 554 kB/s | 106 kB 00:00 2024-05-08T08:49:52.777 INFO:teuthology.orchestra.run.smithi059.stdout:(11/36): python3-pycparser-2.20-6.el9.noarch.rp 768 kB/s | 135 kB 00:00 2024-05-08T08:49:52.802 INFO:teuthology.orchestra.run.smithi059.stdout:(12/36): python3-urllib3-1.26.5-5.el9.noarch.rp 2.1 MB/s | 215 kB 00:00 2024-05-08T08:49:52.919 INFO:teuthology.orchestra.run.smithi059.stdout:(13/36): python3-markupsafe-1.1.1-12.el9.x86_64 298 kB/s | 35 kB 00:00 2024-05-08T08:49:52.945 INFO:teuthology.orchestra.run.smithi059.stdout:(14/36): lua-5.4.4-4.el9.x86_64.rpm 901 kB/s | 188 kB 00:00 2024-05-08T08:49:52.978 INFO:teuthology.orchestra.run.smithi059.stdout:(15/36): python3-mako-1.1.4-6.el9.noarch.rpm 858 kB/s | 172 kB 00:00 2024-05-08T08:49:53.004 INFO:teuthology.orchestra.run.smithi059.stdout:(16/36): python3-toml-0.10.2-6.el9.noarch.rpm 712 kB/s | 42 kB 00:00 2024-05-08T08:49:53.029 INFO:teuthology.orchestra.run.smithi059.stdout:(17/36): python3-pytz-2021.1-5.el9.noarch.rpm 467 kB/s | 51 kB 00:00 2024-05-08T08:49:53.071 INFO:teuthology.orchestra.run.smithi059.stdout:(18/36): lua-devel-5.4.4-4.el9.x86_64.rpm 242 kB/s | 22 kB 00:00 2024-05-08T08:49:53.143 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:00:48 ago on Wed 08 May 2024 08:49:05 AM UTC. 2024-05-08T08:49:53.188 INFO:teuthology.orchestra.run.smithi059.stdout:(19/36): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 274 kB/s | 43 kB 00:00 2024-05-08T08:49:53.238 INFO:teuthology.orchestra.run.smithi059.stdout:(20/36): luarocks-3.9.2-1.el9.noarch.rpm 646 kB/s | 151 kB 00:00 2024-05-08T08:49:53.280 INFO:teuthology.orchestra.run.smithi059.stdout:(21/36): python3-cheroot-8.6.0-4.el9.noarch.rpm 823 kB/s | 172 kB 00:00 2024-05-08T08:49:53.282 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-05-08T08:49:53.285 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-05-08T08:49:53.285 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2024-05-08T08:49:53.285 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-05-08T08:49:53.285 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-05-08T08:49:53.285 INFO:teuthology.orchestra.run.smithi192.stdout: ceph x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 6.6 k 2024-05-08T08:49:53.285 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2024-05-08T08:49:53.285 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mds x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 2.4 M 2024-05-08T08:49:53.285 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 1.5 M 2024-05-08T08:49:53.286 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 249 k 2024-05-08T08:49:53.286 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mon x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 4.7 M 2024-05-08T08:49:53.286 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-osd x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 17 M 2024-05-08T08:49:53.286 INFO:teuthology.orchestra.run.smithi192.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-08T08:49:53.286 INFO:teuthology.orchestra.run.smithi192.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-08T08:49:53.286 INFO:teuthology.orchestra.run.smithi192.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-08T08:49:53.286 INFO:teuthology.orchestra.run.smithi192.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-08T08:49:53.286 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-08T08:49:53.286 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-08T08:49:53.286 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-08T08:49:53.286 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-08T08:49:53.286 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-08T08:49:53.286 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-08T08:49:53.286 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-collections 2024-05-08T08:49:53.287 INFO:teuthology.orchestra.run.smithi192.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-05-08T08:49:53.287 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-08T08:49:53.287 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-08T08:49:53.287 INFO:teuthology.orchestra.run.smithi192.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-08T08:49:53.287 INFO:teuthology.orchestra.run.smithi192.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-08T08:49:53.287 INFO:teuthology.orchestra.run.smithi192.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-08T08:49:53.287 INFO:teuthology.orchestra.run.smithi192.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-08T08:49:53.287 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-08T08:49:53.287 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-08T08:49:53.287 INFO:teuthology.orchestra.run.smithi192.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-08T08:49:53.287 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-08T08:49:53.287 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-08T08:49:53.287 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-08T08:49:53.287 INFO:teuthology.orchestra.run.smithi192.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-08T08:49:53.288 INFO:teuthology.orchestra.run.smithi192.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-08T08:49:53.288 INFO:teuthology.orchestra.run.smithi192.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-08T08:49:53.288 INFO:teuthology.orchestra.run.smithi192.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-08T08:49:53.288 INFO:teuthology.orchestra.run.smithi192.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-08T08:49:53.288 INFO:teuthology.orchestra.run.smithi192.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-08T08:49:53.288 INFO:teuthology.orchestra.run.smithi192.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-08T08:49:53.288 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:49:53.289 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-05-08T08:49:53.289 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-05-08T08:49:53.289 INFO:teuthology.orchestra.run.smithi192.stdout:Install 36 Packages 2024-05-08T08:49:53.289 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:49:53.289 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 30 M 2024-05-08T08:49:53.289 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 107 M 2024-05-08T08:49:53.289 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-05-08T08:49:53.305 INFO:teuthology.orchestra.run.smithi059.stdout:(22/36): python3-jaraco-8.2.1-3.el9.noarch.rpm 160 kB/s | 11 kB 00:00 2024-05-08T08:49:53.339 INFO:teuthology.orchestra.run.smithi059.stdout:(23/36): python3-jaraco-classes-3.2.1-5.el9.noa 302 kB/s | 18 kB 00:00 2024-05-08T08:49:53.364 INFO:teuthology.orchestra.run.smithi059.stdout:(24/36): python3-jaraco-collections-3.0.0-8.el9 392 kB/s | 23 kB 00:00 2024-05-08T08:49:53.390 INFO:teuthology.orchestra.run.smithi059.stdout:(25/36): python3-cherrypy-18.6.1-2.el9.noarch.r 1.7 MB/s | 358 kB 00:00 2024-05-08T08:49:53.415 INFO:teuthology.orchestra.run.smithi059.stdout:(26/36): python3-jaraco-functools-3.5.0-2.el9.n 257 kB/s | 19 kB 00:00 2024-05-08T08:49:53.440 INFO:teuthology.orchestra.run.smithi059.stdout:(27/36): python3-jaraco-text-3.2.0-6.el9.noarch 261 kB/s | 20 kB 00:00 2024-05-08T08:49:53.465 INFO:teuthology.orchestra.run.smithi059.stdout:(28/36): python3-logutils-0.3.5-21.el9.noarch.r 614 kB/s | 46 kB 00:00 2024-05-08T08:49:53.490 INFO:teuthology.orchestra.run.smithi059.stdout:(29/36): python3-more-itertools-8.12.0-2.el9.no 1.0 MB/s | 79 kB 00:00 2024-05-08T08:49:53.516 INFO:teuthology.orchestra.run.smithi059.stdout:(30/36): python3-portend-3.1.0-2.el9.noarch.rpm 326 kB/s | 16 kB 00:00 2024-05-08T08:49:53.541 INFO:teuthology.orchestra.run.smithi059.stdout:(31/36): python3-pecan-1.4.2-3.el9.noarch.rpm 2.6 MB/s | 272 kB 00:00 2024-05-08T08:49:53.558 INFO:teuthology.orchestra.run.smithi059.stdout:(32/36): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.3 MB/s | 90 kB 00:00 2024-05-08T08:49:53.583 INFO:teuthology.orchestra.run.smithi059.stdout:(33/36): python3-tempora-5.0.0-2.el9.noarch.rpm 535 kB/s | 36 kB 00:00 2024-05-08T08:49:53.608 INFO:teuthology.orchestra.run.smithi059.stdout:(34/36): python3-webob-1.8.7-6.el9.noarch.rpm 3.4 MB/s | 230 kB 00:00 2024-05-08T08:49:53.634 INFO:teuthology.orchestra.run.smithi059.stdout:(35/36): python3-zc-lockfile-2.0-10.el9.noarch. 398 kB/s | 20 kB 00:00 2024-05-08T08:49:53.676 INFO:teuthology.orchestra.run.smithi059.stdout:(36/36): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.6 MB/s | 427 kB 00:00 2024-05-08T08:49:53.677 INFO:teuthology.orchestra.run.smithi059.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:49:53.678 INFO:teuthology.orchestra.run.smithi059.stdout:Total 10 MB/s | 30 MB 00:02 2024-05-08T08:49:53.826 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction check 2024-05-08T08:49:53.870 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction check succeeded. 2024-05-08T08:49:53.870 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction test 2024-05-08T08:49:53.989 INFO:teuthology.orchestra.run.smithi192.stdout:(1/36): ceph-19.0.0-2515.g96805dcf.el9.x86_64.r 31 kB/s | 6.6 kB 00:00 2024-05-08T08:49:54.009 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-test-2:19.0.0-2515.g96805dcf.el9.x86_64 4/4 2024-05-08T08:49:54.010 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-test-2:19.0.0-2515.g96805dcf.el9.x86_64 1/4 2024-05-08T08:49:54.010 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 2/4 2024-05-08T08:49:54.010 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/4 2024-05-08T08:49:54.300 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-05-08T08:49:54.301 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:49:54.301 INFO:teuthology.orchestra.run.smithi196.stdout:Installed: 2024-05-08T08:49:54.301 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-test-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:49:54.301 INFO:teuthology.orchestra.run.smithi196.stdout: libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:49:54.301 INFO:teuthology.orchestra.run.smithi196.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-08T08:49:54.301 INFO:teuthology.orchestra.run.smithi196.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-08T08:49:54.301 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:49:54.301 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-05-08T08:49:54.305 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction test succeeded. 2024-05-08T08:49:54.306 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction 2024-05-08T08:49:54.340 INFO:teuthology.orchestra.run.smithi192.stdout:(2/36): ceph-mgr-19.0.0-2515.g96805dcf.el9.x86_ 2.7 MB/s | 1.5 MB 00:00 2024-05-08T08:49:54.381 INFO:teuthology.orchestra.run.smithi192.stdout:(3/36): ceph-mds-19.0.0-2515.g96805dcf.el9.x86_ 3.9 MB/s | 2.4 MB 00:00 2024-05-08T08:49:54.465 INFO:teuthology.orchestra.run.smithi192.stdout:(4/36): ceph-mgr-modules-core-19.0.0-2515.g9680 2.9 MB/s | 249 kB 00:00 2024-05-08T08:49:54.585 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install ceph 2024-05-08T08:49:54.649 INFO:teuthology.orchestra.run.smithi192.stdout:(5/36): ceph-mon-19.0.0-2515.g96805dcf.el9.x86_ 7.1 MB/s | 4.7 MB 00:00 2024-05-08T08:49:55.154 INFO:teuthology.orchestra.run.smithi196.stdout:Last metadata expiration check: 0:00:44 ago on Wed 08 May 2024 08:49:11 AM UTC. 2024-05-08T08:49:55.283 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-05-08T08:49:55.285 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-05-08T08:49:55.285 INFO:teuthology.orchestra.run.smithi196.stdout: Package Arch Version Repository Size 2024-05-08T08:49:55.285 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-05-08T08:49:55.285 INFO:teuthology.orchestra.run.smithi196.stdout:Installing: 2024-05-08T08:49:55.286 INFO:teuthology.orchestra.run.smithi196.stdout: ceph x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 6.6 k 2024-05-08T08:49:55.286 INFO:teuthology.orchestra.run.smithi196.stdout:Installing dependencies: 2024-05-08T08:49:55.286 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mds x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 2.4 M 2024-05-08T08:49:55.286 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 1.5 M 2024-05-08T08:49:55.286 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 249 k 2024-05-08T08:49:55.286 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mon x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 4.7 M 2024-05-08T08:49:55.286 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-osd x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 17 M 2024-05-08T08:49:55.286 INFO:teuthology.orchestra.run.smithi196.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-08T08:49:55.286 INFO:teuthology.orchestra.run.smithi196.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-08T08:49:55.286 INFO:teuthology.orchestra.run.smithi196.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-08T08:49:55.286 INFO:teuthology.orchestra.run.smithi196.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-08T08:49:55.286 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-08T08:49:55.287 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-08T08:49:55.287 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-08T08:49:55.287 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-08T08:49:55.287 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-08T08:49:55.287 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-08T08:49:55.287 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-collections 2024-05-08T08:49:55.287 INFO:teuthology.orchestra.run.smithi196.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-05-08T08:49:55.287 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-08T08:49:55.287 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-08T08:49:55.287 INFO:teuthology.orchestra.run.smithi196.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-08T08:49:55.287 INFO:teuthology.orchestra.run.smithi196.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-08T08:49:55.287 INFO:teuthology.orchestra.run.smithi196.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-08T08:49:55.287 INFO:teuthology.orchestra.run.smithi196.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-08T08:49:55.287 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-08T08:49:55.288 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-08T08:49:55.288 INFO:teuthology.orchestra.run.smithi196.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-08T08:49:55.288 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-08T08:49:55.288 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-08T08:49:55.288 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-08T08:49:55.288 INFO:teuthology.orchestra.run.smithi196.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-08T08:49:55.288 INFO:teuthology.orchestra.run.smithi196.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-08T08:49:55.288 INFO:teuthology.orchestra.run.smithi196.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-08T08:49:55.288 INFO:teuthology.orchestra.run.smithi196.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-08T08:49:55.288 INFO:teuthology.orchestra.run.smithi196.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-08T08:49:55.288 INFO:teuthology.orchestra.run.smithi196.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-08T08:49:55.288 INFO:teuthology.orchestra.run.smithi196.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-08T08:49:55.288 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:49:55.288 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-05-08T08:49:55.289 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-05-08T08:49:55.289 INFO:teuthology.orchestra.run.smithi196.stdout:Install 36 Packages 2024-05-08T08:49:55.289 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:49:55.289 INFO:teuthology.orchestra.run.smithi196.stdout:Total download size: 30 M 2024-05-08T08:49:55.289 INFO:teuthology.orchestra.run.smithi196.stdout:Installed size: 107 M 2024-05-08T08:49:55.289 INFO:teuthology.orchestra.run.smithi196.stdout:Downloading Packages: 2024-05-08T08:49:55.424 INFO:teuthology.orchestra.run.smithi192.stdout:(6/36): ceph-osd-19.0.0-2515.g96805dcf.el9.x86_ 15 MB/s | 17 MB 00:01 2024-05-08T08:49:55.449 INFO:teuthology.orchestra.run.smithi192.stdout:(7/36): python3-cffi-1.14.5-5.el9.x86_64.rpm 257 kB/s | 253 kB 00:00 2024-05-08T08:49:55.628 INFO:teuthology.orchestra.run.smithi059.stdout: Preparing : 1/1 2024-05-08T08:49:55.687 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/36 2024-05-08T08:49:55.735 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/36 2024-05-08T08:49:55.773 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/36 2024-05-08T08:49:55.848 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/36 2024-05-08T08:49:55.892 INFO:teuthology.orchestra.run.smithi196.stdout:(1/36): ceph-19.0.0-2515.g96805dcf.el9.x86_64.r 30 kB/s | 6.6 kB 00:00 2024-05-08T08:49:55.897 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : luarocks-3.9.2-1.el9.noarch 5/36 2024-05-08T08:49:55.916 INFO:teuthology.orchestra.run.smithi192.stdout:(8/36): python3-cryptography-36.0.1-4.el9.x86_6 991 kB/s | 1.2 MB 00:01 2024-05-08T08:49:55.930 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/36 2024-05-08T08:49:56.026 INFO:teuthology.orchestra.run.smithi196.stdout:(2/36): ceph-mgr-19.0.0-2515.g96805dcf.el9.x86_ 4.3 MB/s | 1.5 MB 00:00 2024-05-08T08:49:56.028 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/36 2024-05-08T08:49:56.113 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/36 2024-05-08T08:49:56.158 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 9/36 2024-05-08T08:49:56.193 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/36 2024-05-08T08:49:56.208 INFO:teuthology.orchestra.run.smithi192.stdout:(9/36): python3-ply-3.11-14.el9.noarch.rpm 136 kB/s | 106 kB 00:00 2024-05-08T08:49:56.222 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/36 2024-05-08T08:49:56.257 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/36 2024-05-08T08:49:56.290 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/36 2024-05-08T08:49:56.301 INFO:teuthology.orchestra.run.smithi196.stdout:(3/36): ceph-mds-19.0.0-2515.g96805dcf.el9.x86_ 3.8 MB/s | 2.4 MB 00:00 2024-05-08T08:49:56.309 INFO:teuthology.orchestra.run.smithi192.stdout:(10/36): python3-pycparser-2.20-6.el9.noarch.rp 157 kB/s | 135 kB 00:00 2024-05-08T08:49:56.331 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/36 2024-05-08T08:49:56.374 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/36 2024-05-08T08:49:56.393 INFO:teuthology.orchestra.run.smithi196.stdout:(4/36): ceph-mgr-modules-core-19.0.0-2515.g9680 2.7 MB/s | 249 kB 00:00 2024-05-08T08:49:56.409 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/36 2024-05-08T08:49:56.447 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/36 2024-05-08T08:49:56.476 INFO:teuthology.orchestra.run.smithi192.stdout:(11/36): lua-5.4.4-4.el9.x86_64.rpm 1.1 MB/s | 188 kB 00:00 2024-05-08T08:49:56.525 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 18/36 2024-05-08T08:49:56.560 INFO:teuthology.orchestra.run.smithi196.stdout:(5/36): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.5 MB/s | 253 kB 00:00 2024-05-08T08:49:56.584 INFO:teuthology.orchestra.run.smithi192.stdout:(12/36): python3-mako-1.1.4-6.el9.noarch.rpm 2.0 MB/s | 172 kB 00:00 2024-05-08T08:49:56.679 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 19/36 2024-05-08T08:49:56.726 INFO:teuthology.orchestra.run.smithi192.stdout:(13/36): python3-requests-2.25.1-8.el9.noarch.r 154 kB/s | 125 kB 00:00 2024-05-08T08:49:56.751 INFO:teuthology.orchestra.run.smithi192.stdout:(14/36): python3-markupsafe-1.1.1-12.el9.x86_64 209 kB/s | 35 kB 00:00 2024-05-08T08:49:56.768 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 20/36 2024-05-08T08:49:56.826 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 21/36 2024-05-08T08:49:56.835 INFO:teuthology.orchestra.run.smithi192.stdout:(15/36): python3-pytz-2021.1-5.el9.noarch.rpm 615 kB/s | 51 kB 00:00 2024-05-08T08:49:56.860 INFO:teuthology.orchestra.run.smithi192.stdout:(16/36): python3-toml-0.10.2-6.el9.noarch.rpm 385 kB/s | 42 kB 00:00 2024-05-08T08:49:56.864 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 22/36 2024-05-08T08:49:56.924 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-ply-3.11-14.el9.noarch 23/36 2024-05-08T08:49:56.977 INFO:teuthology.orchestra.run.smithi196.stdout:(6/36): ceph-osd-19.0.0-2515.g96805dcf.el9.x86_ 18 MB/s | 17 MB 00:00 2024-05-08T08:49:57.002 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 24/36 2024-05-08T08:49:57.010 INFO:teuthology.orchestra.run.smithi196.stdout:(7/36): python3-cryptography-36.0.1-4.el9.x86_6 2.7 MB/s | 1.2 MB 00:00 2024-05-08T08:49:57.052 INFO:teuthology.orchestra.run.smithi192.stdout:(17/36): luarocks-3.9.2-1.el9.noarch.rpm 789 kB/s | 151 kB 00:00 2024-05-08T08:49:57.069 INFO:teuthology.orchestra.run.smithi196.stdout:(8/36): python3-pycparser-2.20-6.el9.noarch.rpm 2.3 MB/s | 135 kB 00:00 2024-05-08T08:49:57.094 INFO:teuthology.orchestra.run.smithi192.stdout:(18/36): lua-devel-5.4.4-4.el9.x86_64.rpm 86 kB/s | 22 kB 00:00 2024-05-08T08:49:57.119 INFO:teuthology.orchestra.run.smithi196.stdout:(9/36): python3-requests-2.25.1-8.el9.noarch.rp 2.5 MB/s | 125 kB 00:00 2024-05-08T08:49:57.119 INFO:teuthology.orchestra.run.smithi192.stdout:(19/36): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 648 kB/s | 43 kB 00:00 2024-05-08T08:49:57.144 INFO:teuthology.orchestra.run.smithi192.stdout:(20/36): python3-urllib3-1.26.5-5.el9.noarch.rp 229 kB/s | 215 kB 00:00 2024-05-08T08:49:57.170 INFO:teuthology.orchestra.run.smithi192.stdout:(21/36): python3-cheroot-8.6.0-4.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-05-08T08:49:57.195 INFO:teuthology.orchestra.run.smithi196.stdout:(10/36): ceph-mon-19.0.0-2515.g96805dcf.el9.x86 3.6 MB/s | 4.7 MB 00:01 2024-05-08T08:49:57.195 INFO:teuthology.orchestra.run.smithi192.stdout:(22/36): python3-cherrypy-18.6.1-2.el9.noarch.r 4.7 MB/s | 358 kB 00:00 2024-05-08T08:49:57.218 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 25/36 2024-05-08T08:49:57.219 INFO:teuthology.orchestra.run.smithi196.stdout:(11/36): python3-ply-3.11-14.el9.noarch.rpm 439 kB/s | 106 kB 00:00 2024-05-08T08:49:57.220 INFO:teuthology.orchestra.run.smithi192.stdout:(23/36): python3-jaraco-8.2.1-3.el9.noarch.rpm 142 kB/s | 11 kB 00:00 2024-05-08T08:49:57.246 INFO:teuthology.orchestra.run.smithi196.stdout:(12/36): python3-urllib3-1.26.5-5.el9.noarch.rp 1.7 MB/s | 215 kB 00:00 2024-05-08T08:49:57.246 INFO:teuthology.orchestra.run.smithi192.stdout:(24/36): python3-jaraco-classes-3.2.1-5.el9.noa 235 kB/s | 18 kB 00:00 2024-05-08T08:49:57.268 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 26/36 2024-05-08T08:49:57.271 INFO:teuthology.orchestra.run.smithi192.stdout:(25/36): python3-jaraco-collections-3.0.0-8.el9 464 kB/s | 23 kB 00:00 2024-05-08T08:49:57.296 INFO:teuthology.orchestra.run.smithi192.stdout:(26/36): python3-jaraco-functools-3.5.0-2.el9.n 259 kB/s | 19 kB 00:00 2024-05-08T08:49:57.322 INFO:teuthology.orchestra.run.smithi192.stdout:(27/36): python3-jaraco-text-3.2.0-6.el9.noarch 261 kB/s | 20 kB 00:00 2024-05-08T08:49:57.347 INFO:teuthology.orchestra.run.smithi192.stdout:(28/36): python3-logutils-0.3.5-21.el9.noarch.r 926 kB/s | 46 kB 00:00 2024-05-08T08:49:57.350 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 27/36 2024-05-08T08:49:57.372 INFO:teuthology.orchestra.run.smithi192.stdout:(29/36): python3-more-itertools-8.12.0-2.el9.no 1.0 MB/s | 79 kB 00:00 2024-05-08T08:49:57.406 INFO:teuthology.orchestra.run.smithi192.stdout:(30/36): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-05-08T08:49:57.431 INFO:teuthology.orchestra.run.smithi192.stdout:(31/36): python3-portend-3.1.0-2.el9.noarch.rpm 283 kB/s | 16 kB 00:00 2024-05-08T08:49:57.448 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 28/36 2024-05-08T08:49:57.456 INFO:teuthology.orchestra.run.smithi192.stdout:(32/36): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.1 MB/s | 90 kB 00:00 2024-05-08T08:49:57.481 INFO:teuthology.orchestra.run.smithi192.stdout:(33/36): python3-tempora-5.0.0-2.el9.noarch.rpm 477 kB/s | 36 kB 00:00 2024-05-08T08:49:57.483 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 29/36 2024-05-08T08:49:57.506 INFO:teuthology.orchestra.run.smithi192.stdout:(34/36): python3-webob-1.8.7-6.el9.noarch.rpm 4.5 MB/s | 230 kB 00:00 2024-05-08T08:49:57.532 INFO:teuthology.orchestra.run.smithi192.stdout:(35/36): python3-werkzeug-2.0.3-3.el9.1.noarch. 5.6 MB/s | 427 kB 00:00 2024-05-08T08:49:57.557 INFO:teuthology.orchestra.run.smithi192.stdout:(36/36): python3-zc-lockfile-2.0-10.el9.noarch. 267 kB/s | 20 kB 00:00 2024-05-08T08:49:57.561 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:49:57.561 INFO:teuthology.orchestra.run.smithi192.stdout:Total 7.0 MB/s | 30 MB 00:04 2024-05-08T08:49:57.609 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 30/36 2024-05-08T08:49:57.707 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-05-08T08:49:57.747 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-05-08T08:49:57.748 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-05-08T08:49:57.780 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2515.g96805dcf.el9. 31/36 2024-05-08T08:49:57.809 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 32/36 2024-05-08T08:49:57.838 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 32/36 2024-05-08T08:49:57.838 INFO:teuthology.orchestra.run.smithi059.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T08:49:57.838 INFO:teuthology.orchestra.run.smithi059.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-08T08:49:57.838 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-08T08:49:57.838 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-08T08:49:57.839 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:49:57.912 INFO:teuthology.orchestra.run.smithi196.stdout:(13/36): python3-markupsafe-1.1.1-12.el9.x86_64 52 kB/s | 35 kB 00:00 2024-05-08T08:49:58.070 INFO:teuthology.orchestra.run.smithi196.stdout:(14/36): python3-mako-1.1.4-6.el9.noarch.rpm 202 kB/s | 172 kB 00:00 2024-05-08T08:49:58.130 INFO:teuthology.orchestra.run.smithi196.stdout:(15/36): lua-5.4.4-4.el9.x86_64.rpm 202 kB/s | 188 kB 00:00 2024-05-08T08:49:58.205 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-05-08T08:49:58.205 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-05-08T08:49:58.271 INFO:teuthology.orchestra.run.smithi196.stdout:(16/36): lua-devel-5.4.4-4.el9.x86_64.rpm 158 kB/s | 22 kB 00:00 2024-05-08T08:49:58.621 INFO:teuthology.orchestra.run.smithi196.stdout:(17/36): python3-pytz-2021.1-5.el9.noarch.rpm 72 kB/s | 51 kB 00:00 2024-05-08T08:49:58.746 INFO:teuthology.orchestra.run.smithi196.stdout:(18/36): luarocks-3.9.2-1.el9.noarch.rpm 319 kB/s | 151 kB 00:00 2024-05-08T08:49:58.771 INFO:teuthology.orchestra.run.smithi196.stdout:(19/36): python3-toml-0.10.2-6.el9.noarch.rpm 59 kB/s | 42 kB 00:00 2024-05-08T08:49:58.955 INFO:teuthology.orchestra.run.smithi196.stdout:(20/36): python3-cheroot-8.6.0-4.el9.noarch.rpm 826 kB/s | 172 kB 00:00 2024-05-08T08:49:58.980 INFO:teuthology.orchestra.run.smithi196.stdout:(21/36): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 121 kB/s | 43 kB 00:00 2024-05-08T08:49:59.048 INFO:teuthology.orchestra.run.smithi196.stdout:(22/36): python3-jaraco-8.2.1-3.el9.noarch.rpm 116 kB/s | 11 kB 00:00 2024-05-08T08:49:59.081 INFO:teuthology.orchestra.run.smithi196.stdout:(23/36): python3-jaraco-classes-3.2.1-5.el9.noa 176 kB/s | 18 kB 00:00 2024-05-08T08:49:59.140 INFO:teuthology.orchestra.run.smithi196.stdout:(24/36): python3-jaraco-collections-3.0.0-8.el9 252 kB/s | 23 kB 00:00 2024-05-08T08:49:59.173 INFO:teuthology.orchestra.run.smithi196.stdout:(25/36): python3-jaraco-functools-3.5.0-2.el9.n 211 kB/s | 19 kB 00:00 2024-05-08T08:49:59.232 INFO:teuthology.orchestra.run.smithi196.stdout:(26/36): python3-jaraco-text-3.2.0-6.el9.noarch 214 kB/s | 20 kB 00:00 2024-05-08T08:49:59.257 INFO:teuthology.orchestra.run.smithi196.stdout:(27/36): python3-cherrypy-18.6.1-2.el9.noarch.r 738 kB/s | 358 kB 00:00 2024-05-08T08:49:59.291 INFO:teuthology.orchestra.run.smithi196.stdout:(28/36): python3-logutils-0.3.5-21.el9.noarch.r 395 kB/s | 46 kB 00:00 2024-05-08T08:49:59.333 INFO:teuthology.orchestra.run.smithi196.stdout:(29/36): python3-more-itertools-8.12.0-2.el9.no 785 kB/s | 79 kB 00:00 2024-05-08T08:49:59.369 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 33/36 2024-05-08T08:49:59.383 INFO:teuthology.orchestra.run.smithi196.stdout:(30/36): python3-pecan-1.4.2-3.el9.noarch.rpm 2.1 MB/s | 272 kB 00:00 2024-05-08T08:49:59.398 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 33/36 2024-05-08T08:49:59.398 INFO:teuthology.orchestra.run.smithi059.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T08:49:59.398 INFO:teuthology.orchestra.run.smithi059.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-08T08:49:59.398 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-08T08:49:59.398 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-08T08:49:59.398 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:49:59.408 INFO:teuthology.orchestra.run.smithi196.stdout:(31/36): python3-portend-3.1.0-2.el9.noarch.rpm 140 kB/s | 16 kB 00:00 2024-05-08T08:49:59.433 INFO:teuthology.orchestra.run.smithi196.stdout:(32/36): python3-pyOpenSSL-21.0.0-1.el9.noarch. 895 kB/s | 90 kB 00:00 2024-05-08T08:49:59.461 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-05-08T08:49:59.484 INFO:teuthology.orchestra.run.smithi196.stdout:(33/36): python3-tempora-5.0.0-2.el9.noarch.rpm 356 kB/s | 36 kB 00:00 2024-05-08T08:49:59.527 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/36 2024-05-08T08:49:59.577 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/36 2024-05-08T08:49:59.600 INFO:teuthology.orchestra.run.smithi196.stdout:(34/36): python3-zc-lockfile-2.0-10.el9.noarch. 171 kB/s | 20 kB 00:00 2024-05-08T08:49:59.604 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/36 2024-05-08T08:49:59.626 INFO:teuthology.orchestra.run.smithi196.stdout:(35/36): python3-werkzeug-2.0.3-3.el9.1.noarch. 2.2 MB/s | 427 kB 00:00 2024-05-08T08:49:59.673 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/36 2024-05-08T08:49:59.701 INFO:teuthology.orchestra.run.smithi196.stdout:(36/36): python3-webob-1.8.7-6.el9.noarch.rpm 786 kB/s | 230 kB 00:00 2024-05-08T08:49:59.703 INFO:teuthology.orchestra.run.smithi196.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:49:59.703 INFO:teuthology.orchestra.run.smithi196.stdout:Total 6.8 MB/s | 30 MB 00:04 2024-05-08T08:49:59.711 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : luarocks-3.9.2-1.el9.noarch 5/36 2024-05-08T08:49:59.743 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/36 2024-05-08T08:49:59.834 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/36 2024-05-08T08:49:59.845 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction check 2024-05-08T08:49:59.861 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 34/36 2024-05-08T08:49:59.885 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction check succeeded. 2024-05-08T08:49:59.885 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction test 2024-05-08T08:49:59.886 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 34/36 2024-05-08T08:49:59.886 INFO:teuthology.orchestra.run.smithi059.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T08:49:59.886 INFO:teuthology.orchestra.run.smithi059.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-08T08:49:59.886 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-08T08:49:59.887 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-08T08:49:59.887 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:49:59.922 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/36 2024-05-08T08:49:59.962 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 9/36 2024-05-08T08:49:59.998 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/36 2024-05-08T08:50:00.028 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/36 2024-05-08T08:50:00.063 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/36 2024-05-08T08:50:00.095 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/36 2024-05-08T08:50:00.128 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 35/36 2024-05-08T08:50:00.136 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/36 2024-05-08T08:50:00.165 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 35/36 2024-05-08T08:50:00.166 INFO:teuthology.orchestra.run.smithi059.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T08:50:00.166 INFO:teuthology.orchestra.run.smithi059.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-08T08:50:00.166 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-08T08:50:00.166 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-08T08:50:00.166 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:50:00.171 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/36 2024-05-08T08:50:00.214 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/36 2024-05-08T08:50:00.227 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-2:19.0.0-2515.g96805dcf.el9.x86_64 36/36 2024-05-08T08:50:00.252 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/36 2024-05-08T08:50:00.325 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 18/36 2024-05-08T08:50:00.367 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction test succeeded. 2024-05-08T08:50:00.368 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction 2024-05-08T08:50:00.462 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 19/36 2024-05-08T08:50:00.543 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 20/36 2024-05-08T08:50:00.596 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 21/36 2024-05-08T08:50:00.635 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 22/36 2024-05-08T08:50:00.692 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-ply-3.11-14.el9.noarch 23/36 2024-05-08T08:50:00.748 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 24/36 2024-05-08T08:50:00.940 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 25/36 2024-05-08T08:50:00.981 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 26/36 2024-05-08T08:50:01.059 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 27/36 2024-05-08T08:50:01.155 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 28/36 2024-05-08T08:50:01.197 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 29/36 2024-05-08T08:50:01.314 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 30/36 2024-05-08T08:50:01.479 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2515.g96805dcf.el9. 31/36 2024-05-08T08:50:01.508 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 32/36 2024-05-08T08:50:01.553 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 32/36 2024-05-08T08:50:01.553 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T08:50:01.553 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-08T08:50:01.553 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-08T08:50:01.553 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-08T08:50:01.553 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:50:01.615 INFO:teuthology.orchestra.run.smithi196.stdout: Preparing : 1/1 2024-05-08T08:50:01.695 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/36 2024-05-08T08:50:01.738 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/36 2024-05-08T08:50:01.772 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/36 2024-05-08T08:50:01.843 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/36 2024-05-08T08:50:01.890 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-2:19.0.0-2515.g96805dcf.el9.x86_64 36/36 2024-05-08T08:50:01.890 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-2:19.0.0-2515.g96805dcf.el9.x86_64 1/36 2024-05-08T08:50:01.890 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 2/36 2024-05-08T08:50:01.890 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 3/36 2024-05-08T08:50:01.890 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 4/36 2024-05-08T08:50:01.890 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 5/36 2024-05-08T08:50:01.890 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2515.g96805dcf.el9. 6/36 2024-05-08T08:50:01.890 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/36 2024-05-08T08:50:01.890 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/36 2024-05-08T08:50:01.890 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/36 2024-05-08T08:50:01.891 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/36 2024-05-08T08:50:01.891 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/36 2024-05-08T08:50:01.891 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/36 2024-05-08T08:50:01.891 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/36 2024-05-08T08:50:01.891 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/36 2024-05-08T08:50:01.891 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/36 2024-05-08T08:50:01.891 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/36 2024-05-08T08:50:01.891 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/36 2024-05-08T08:50:01.891 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 18/36 2024-05-08T08:50:01.891 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 19/36 2024-05-08T08:50:01.891 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 20/36 2024-05-08T08:50:01.891 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-05-08T08:50:01.891 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 22/36 2024-05-08T08:50:01.891 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 23/36 2024-05-08T08:50:01.892 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 24/36 2024-05-08T08:50:01.892 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 25/36 2024-05-08T08:50:01.892 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 26/36 2024-05-08T08:50:01.892 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 27/36 2024-05-08T08:50:01.892 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 28/36 2024-05-08T08:50:01.892 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 29/36 2024-05-08T08:50:01.892 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-05-08T08:50:01.892 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 31/36 2024-05-08T08:50:01.892 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 32/36 2024-05-08T08:50:01.892 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 33/36 2024-05-08T08:50:01.892 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 34/36 2024-05-08T08:50:01.892 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 35/36 2024-05-08T08:50:01.896 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : luarocks-3.9.2-1.el9.noarch 5/36 2024-05-08T08:50:01.929 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/36 2024-05-08T08:50:02.021 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/36 2024-05-08T08:50:02.099 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/36 2024-05-08T08:50:02.140 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 9/36 2024-05-08T08:50:02.175 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/36 2024-05-08T08:50:02.205 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/36 2024-05-08T08:50:02.231 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/36 2024-05-08T08:50:02.256 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/36 2024-05-08T08:50:02.297 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/36 2024-05-08T08:50:02.332 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/36 2024-05-08T08:50:02.366 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/36 2024-05-08T08:50:02.404 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/36 2024-05-08T08:50:02.477 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 18/36 2024-05-08T08:50:02.615 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 19/36 2024-05-08T08:50:02.682 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 36/36 2024-05-08T08:50:02.682 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:50:02.682 INFO:teuthology.orchestra.run.smithi059.stdout:Installed: 2024-05-08T08:50:02.682 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:50:02.682 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:50:02.682 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:50:02.683 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-modules-core-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-08T08:50:02.683 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:50:02.683 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:50:02.683 INFO:teuthology.orchestra.run.smithi059.stdout: lua-5.4.4-4.el9.x86_64 2024-05-08T08:50:02.683 INFO:teuthology.orchestra.run.smithi059.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-08T08:50:02.683 INFO:teuthology.orchestra.run.smithi059.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-08T08:50:02.683 INFO:teuthology.orchestra.run.smithi059.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-08T08:50:02.683 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-08T08:50:02.683 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-08T08:50:02.683 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-08T08:50:02.683 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-08T08:50:02.683 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-08T08:50:02.683 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-08T08:50:02.684 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-08T08:50:02.684 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-08T08:50:02.684 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-08T08:50:02.684 INFO:teuthology.orchestra.run.smithi059.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-08T08:50:02.684 INFO:teuthology.orchestra.run.smithi059.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-08T08:50:02.684 INFO:teuthology.orchestra.run.smithi059.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-08T08:50:02.684 INFO:teuthology.orchestra.run.smithi059.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-08T08:50:02.684 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-08T08:50:02.684 INFO:teuthology.orchestra.run.smithi059.stdout: python3-ply-3.11-14.el9.noarch 2024-05-08T08:50:02.684 INFO:teuthology.orchestra.run.smithi059.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-08T08:50:02.684 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-08T08:50:02.684 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-08T08:50:02.684 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-08T08:50:02.684 INFO:teuthology.orchestra.run.smithi059.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-08T08:50:02.684 INFO:teuthology.orchestra.run.smithi059.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-08T08:50:02.685 INFO:teuthology.orchestra.run.smithi059.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-08T08:50:02.685 INFO:teuthology.orchestra.run.smithi059.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-08T08:50:02.685 INFO:teuthology.orchestra.run.smithi059.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-08T08:50:02.685 INFO:teuthology.orchestra.run.smithi059.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-08T08:50:02.685 INFO:teuthology.orchestra.run.smithi059.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-08T08:50:02.685 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:50:02.685 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2024-05-08T08:50:02.695 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 20/36 2024-05-08T08:50:02.757 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 21/36 2024-05-08T08:50:02.796 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 22/36 2024-05-08T08:50:02.852 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-ply-3.11-14.el9.noarch 23/36 2024-05-08T08:50:02.914 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 24/36 2024-05-08T08:50:02.984 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 33/36 2024-05-08T08:50:03.005 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum -y install ceph-base 2024-05-08T08:50:03.011 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 33/36 2024-05-08T08:50:03.011 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T08:50:03.011 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-08T08:50:03.011 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-08T08:50:03.012 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-08T08:50:03.012 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:50:03.119 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 25/36 2024-05-08T08:50:03.175 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 26/36 2024-05-08T08:50:03.253 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 27/36 2024-05-08T08:50:03.341 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 28/36 2024-05-08T08:50:03.382 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 29/36 2024-05-08T08:50:03.451 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 34/36 2024-05-08T08:50:03.476 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 34/36 2024-05-08T08:50:03.476 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T08:50:03.476 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-08T08:50:03.476 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-08T08:50:03.476 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-08T08:50:03.476 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:50:03.499 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 30/36 2024-05-08T08:50:03.597 INFO:teuthology.orchestra.run.smithi059.stdout:Last metadata expiration check: 0:00:58 ago on Wed 08 May 2024 08:49:05 AM UTC. 2024-05-08T08:50:03.655 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2515.g96805dcf.el9. 31/36 2024-05-08T08:50:03.681 INFO:teuthology.orchestra.run.smithi059.stdout:Package ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 is already installed. 2024-05-08T08:50:03.684 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 32/36 2024-05-08T08:50:03.710 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 35/36 2024-05-08T08:50:03.722 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 32/36 2024-05-08T08:50:03.722 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T08:50:03.722 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-08T08:50:03.722 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-08T08:50:03.722 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-08T08:50:03.722 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:50:03.725 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies resolved. 2024-05-08T08:50:03.726 INFO:teuthology.orchestra.run.smithi059.stdout:Nothing to do. 2024-05-08T08:50:03.726 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2024-05-08T08:50:03.734 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 35/36 2024-05-08T08:50:03.734 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T08:50:03.735 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-08T08:50:03.735 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-08T08:50:03.735 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-08T08:50:03.735 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:50:03.789 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum -y install cephadm 2024-05-08T08:50:03.799 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-2:19.0.0-2515.g96805dcf.el9.x86_64 36/36 2024-05-08T08:50:04.375 INFO:teuthology.orchestra.run.smithi059.stdout:Last metadata expiration check: 0:00:59 ago on Wed 08 May 2024 08:49:05 AM UTC. 2024-05-08T08:50:04.502 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies resolved. 2024-05-08T08:50:04.503 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-05-08T08:50:04.503 INFO:teuthology.orchestra.run.smithi059.stdout: Package Arch Version Repository Size 2024-05-08T08:50:04.503 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-05-08T08:50:04.503 INFO:teuthology.orchestra.run.smithi059.stdout:Installing: 2024-05-08T08:50:04.504 INFO:teuthology.orchestra.run.smithi059.stdout: cephadm noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 763 k 2024-05-08T08:50:04.504 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:50:04.504 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction Summary 2024-05-08T08:50:04.504 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-05-08T08:50:04.504 INFO:teuthology.orchestra.run.smithi059.stdout:Install 1 Package 2024-05-08T08:50:04.504 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:50:04.504 INFO:teuthology.orchestra.run.smithi059.stdout:Total download size: 763 k 2024-05-08T08:50:04.504 INFO:teuthology.orchestra.run.smithi059.stdout:Installed size: 769 k 2024-05-08T08:50:04.504 INFO:teuthology.orchestra.run.smithi059.stdout:Downloading Packages: 2024-05-08T08:50:04.778 INFO:teuthology.orchestra.run.smithi059.stdout:cephadm-19.0.0-2515.g96805dcf.el9.noarch.rpm 2.7 MB/s | 763 kB 00:00 2024-05-08T08:50:04.779 INFO:teuthology.orchestra.run.smithi059.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:50:04.779 INFO:teuthology.orchestra.run.smithi059.stdout:Total 2.7 MB/s | 763 kB 00:00 2024-05-08T08:50:04.779 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction check 2024-05-08T08:50:04.785 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction check succeeded. 2024-05-08T08:50:04.785 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction test 2024-05-08T08:50:04.790 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction test succeeded. 2024-05-08T08:50:04.791 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction 2024-05-08T08:50:04.866 INFO:teuthology.orchestra.run.smithi059.stdout: Preparing : 1/1 2024-05-08T08:50:05.160 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 33/36 2024-05-08T08:50:05.188 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 33/36 2024-05-08T08:50:05.188 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T08:50:05.188 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-08T08:50:05.188 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-08T08:50:05.188 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-08T08:50:05.188 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:50:05.288 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 1/1 2024-05-08T08:50:05.377 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 1/1 2024-05-08T08:50:05.627 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 34/36 2024-05-08T08:50:05.651 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 34/36 2024-05-08T08:50:05.652 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T08:50:05.652 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-08T08:50:05.652 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-08T08:50:05.652 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-08T08:50:05.652 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:50:05.662 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-2:19.0.0-2515.g96805dcf.el9.x86_64 36/36 2024-05-08T08:50:05.662 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-2:19.0.0-2515.g96805dcf.el9.x86_64 1/36 2024-05-08T08:50:05.662 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 2/36 2024-05-08T08:50:05.662 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 3/36 2024-05-08T08:50:05.662 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 4/36 2024-05-08T08:50:05.662 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 5/36 2024-05-08T08:50:05.662 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2515.g96805dcf.el9. 6/36 2024-05-08T08:50:05.662 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/36 2024-05-08T08:50:05.662 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/36 2024-05-08T08:50:05.662 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/36 2024-05-08T08:50:05.663 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/36 2024-05-08T08:50:05.663 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/36 2024-05-08T08:50:05.663 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/36 2024-05-08T08:50:05.663 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/36 2024-05-08T08:50:05.663 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/36 2024-05-08T08:50:05.663 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/36 2024-05-08T08:50:05.663 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/36 2024-05-08T08:50:05.663 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/36 2024-05-08T08:50:05.663 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 18/36 2024-05-08T08:50:05.663 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 19/36 2024-05-08T08:50:05.663 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 20/36 2024-05-08T08:50:05.663 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-05-08T08:50:05.663 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 22/36 2024-05-08T08:50:05.663 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 23/36 2024-05-08T08:50:05.664 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 24/36 2024-05-08T08:50:05.664 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 25/36 2024-05-08T08:50:05.664 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 26/36 2024-05-08T08:50:05.664 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 27/36 2024-05-08T08:50:05.664 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 28/36 2024-05-08T08:50:05.664 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 29/36 2024-05-08T08:50:05.664 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-05-08T08:50:05.664 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 31/36 2024-05-08T08:50:05.664 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 32/36 2024-05-08T08:50:05.664 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 33/36 2024-05-08T08:50:05.664 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 34/36 2024-05-08T08:50:05.664 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 35/36 2024-05-08T08:50:05.742 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 1/1 2024-05-08T08:50:05.878 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 35/36 2024-05-08T08:50:05.902 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 35/36 2024-05-08T08:50:05.903 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T08:50:05.903 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-08T08:50:05.903 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-08T08:50:05.903 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-08T08:50:05.903 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:50:05.966 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-2:19.0.0-2515.g96805dcf.el9.x86_64 36/36 2024-05-08T08:50:06.000 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 1/1 2024-05-08T08:50:06.000 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:50:06.000 INFO:teuthology.orchestra.run.smithi059.stdout:Installed: 2024-05-08T08:50:06.000 INFO:teuthology.orchestra.run.smithi059.stdout: cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-08T08:50:06.000 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:50:06.001 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2024-05-08T08:50:06.216 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum -y install ceph-immutable-object-cache 2024-05-08T08:50:06.463 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 36/36 2024-05-08T08:50:06.463 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:50:06.463 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-05-08T08:50:06.463 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:50:06.463 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:50:06.463 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:50:06.464 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-modules-core-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-08T08:50:06.464 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:50:06.464 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:50:06.464 INFO:teuthology.orchestra.run.smithi192.stdout: lua-5.4.4-4.el9.x86_64 2024-05-08T08:50:06.464 INFO:teuthology.orchestra.run.smithi192.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-08T08:50:06.464 INFO:teuthology.orchestra.run.smithi192.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-08T08:50:06.464 INFO:teuthology.orchestra.run.smithi192.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-08T08:50:06.464 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-08T08:50:06.464 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-08T08:50:06.464 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-08T08:50:06.464 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-08T08:50:06.464 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-08T08:50:06.464 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-08T08:50:06.464 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-08T08:50:06.465 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-08T08:50:06.465 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-08T08:50:06.465 INFO:teuthology.orchestra.run.smithi192.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-08T08:50:06.465 INFO:teuthology.orchestra.run.smithi192.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-08T08:50:06.465 INFO:teuthology.orchestra.run.smithi192.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-08T08:50:06.465 INFO:teuthology.orchestra.run.smithi192.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-08T08:50:06.465 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-08T08:50:06.465 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ply-3.11-14.el9.noarch 2024-05-08T08:50:06.465 INFO:teuthology.orchestra.run.smithi192.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-08T08:50:06.465 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-08T08:50:06.465 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-08T08:50:06.465 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-08T08:50:06.465 INFO:teuthology.orchestra.run.smithi192.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-08T08:50:06.465 INFO:teuthology.orchestra.run.smithi192.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-08T08:50:06.465 INFO:teuthology.orchestra.run.smithi192.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-08T08:50:06.466 INFO:teuthology.orchestra.run.smithi192.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-08T08:50:06.466 INFO:teuthology.orchestra.run.smithi192.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-08T08:50:06.466 INFO:teuthology.orchestra.run.smithi192.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-08T08:50:06.466 INFO:teuthology.orchestra.run.smithi192.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-08T08:50:06.466 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:50:06.466 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-05-08T08:50:06.807 INFO:teuthology.orchestra.run.smithi059.stdout:Last metadata expiration check: 0:01:01 ago on Wed 08 May 2024 08:49:05 AM UTC. 2024-05-08T08:50:06.833 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-base 2024-05-08T08:50:06.937 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies resolved. 2024-05-08T08:50:06.937 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-05-08T08:50:06.937 INFO:teuthology.orchestra.run.smithi059.stdout: Package Arch Version Repo Size 2024-05-08T08:50:06.937 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-05-08T08:50:06.937 INFO:teuthology.orchestra.run.smithi059.stdout:Installing: 2024-05-08T08:50:06.937 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 145 k 2024-05-08T08:50:06.937 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:50:06.937 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction Summary 2024-05-08T08:50:06.937 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-05-08T08:50:06.938 INFO:teuthology.orchestra.run.smithi059.stdout:Install 1 Package 2024-05-08T08:50:06.938 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:50:06.938 INFO:teuthology.orchestra.run.smithi059.stdout:Total download size: 145 k 2024-05-08T08:50:06.938 INFO:teuthology.orchestra.run.smithi059.stdout:Installed size: 435 k 2024-05-08T08:50:06.938 INFO:teuthology.orchestra.run.smithi059.stdout:Downloading Packages: 2024-05-08T08:50:07.243 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-immutable-object-cache-19.0.0-2515.g96805d 476 kB/s | 145 kB 00:00 2024-05-08T08:50:07.243 INFO:teuthology.orchestra.run.smithi059.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:50:07.243 INFO:teuthology.orchestra.run.smithi059.stdout:Total 474 kB/s | 145 kB 00:00 2024-05-08T08:50:07.244 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction check 2024-05-08T08:50:07.252 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction check succeeded. 2024-05-08T08:50:07.252 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction test 2024-05-08T08:50:07.312 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction test succeeded. 2024-05-08T08:50:07.312 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction 2024-05-08T08:50:07.398 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:02 ago on Wed 08 May 2024 08:49:05 AM UTC. 2024-05-08T08:50:07.469 INFO:teuthology.orchestra.run.smithi059.stdout: Preparing : 1/1 2024-05-08T08:50:07.479 INFO:teuthology.orchestra.run.smithi192.stdout:Package ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 is already installed. 2024-05-08T08:50:07.521 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-05-08T08:50:07.521 INFO:teuthology.orchestra.run.smithi192.stdout:Nothing to do. 2024-05-08T08:50:07.521 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-05-08T08:50:07.527 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2515.g96805dcf. 1/1 2024-05-08T08:50:07.554 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2515.g96805dcf. 1/1 2024-05-08T08:50:07.554 INFO:teuthology.orchestra.run.smithi059.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T08:50:07.554 INFO:teuthology.orchestra.run.smithi059.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-08T08:50:07.554 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:50:07.584 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install cephadm 2024-05-08T08:50:07.722 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-2:19.0.0-2515.g96805dcf.el9.x86_64 36/36 2024-05-08T08:50:07.722 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-2:19.0.0-2515.g96805dcf.el9.x86_64 1/36 2024-05-08T08:50:07.722 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 2/36 2024-05-08T08:50:07.722 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 3/36 2024-05-08T08:50:07.722 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 4/36 2024-05-08T08:50:07.723 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 5/36 2024-05-08T08:50:07.723 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2515.g96805dcf.el9. 6/36 2024-05-08T08:50:07.723 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/36 2024-05-08T08:50:07.723 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/36 2024-05-08T08:50:07.723 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/36 2024-05-08T08:50:07.723 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/36 2024-05-08T08:50:07.723 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/36 2024-05-08T08:50:07.723 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/36 2024-05-08T08:50:07.724 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/36 2024-05-08T08:50:07.724 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/36 2024-05-08T08:50:07.724 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/36 2024-05-08T08:50:07.724 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/36 2024-05-08T08:50:07.724 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/36 2024-05-08T08:50:07.724 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 18/36 2024-05-08T08:50:07.724 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 19/36 2024-05-08T08:50:07.724 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 20/36 2024-05-08T08:50:07.725 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-05-08T08:50:07.725 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 22/36 2024-05-08T08:50:07.725 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 23/36 2024-05-08T08:50:07.725 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 24/36 2024-05-08T08:50:07.725 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 25/36 2024-05-08T08:50:07.725 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 26/36 2024-05-08T08:50:07.725 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 27/36 2024-05-08T08:50:07.725 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 28/36 2024-05-08T08:50:07.726 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 29/36 2024-05-08T08:50:07.726 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-05-08T08:50:07.726 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 31/36 2024-05-08T08:50:07.726 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 32/36 2024-05-08T08:50:07.726 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 33/36 2024-05-08T08:50:07.726 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 34/36 2024-05-08T08:50:07.726 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 35/36 2024-05-08T08:50:08.147 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:03 ago on Wed 08 May 2024 08:49:05 AM UTC. 2024-05-08T08:50:08.270 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-05-08T08:50:08.271 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-05-08T08:50:08.271 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2024-05-08T08:50:08.271 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-05-08T08:50:08.271 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-05-08T08:50:08.271 INFO:teuthology.orchestra.run.smithi192.stdout: cephadm noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 763 k 2024-05-08T08:50:08.271 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:50:08.271 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-05-08T08:50:08.271 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-05-08T08:50:08.271 INFO:teuthology.orchestra.run.smithi192.stdout:Install 1 Package 2024-05-08T08:50:08.271 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:50:08.271 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 763 k 2024-05-08T08:50:08.272 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 769 k 2024-05-08T08:50:08.272 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-05-08T08:50:08.442 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2515.g96805dcf. 1/1 2024-05-08T08:50:08.443 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:50:08.443 INFO:teuthology.orchestra.run.smithi059.stdout:Installed: 2024-05-08T08:50:08.443 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-immutable-object-cache-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:50:08.443 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:50:08.443 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2024-05-08T08:50:08.521 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 36/36 2024-05-08T08:50:08.521 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:50:08.521 INFO:teuthology.orchestra.run.smithi196.stdout:Installed: 2024-05-08T08:50:08.521 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:50:08.521 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:50:08.521 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:50:08.521 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-modules-core-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-08T08:50:08.521 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:50:08.521 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:50:08.521 INFO:teuthology.orchestra.run.smithi196.stdout: lua-5.4.4-4.el9.x86_64 2024-05-08T08:50:08.522 INFO:teuthology.orchestra.run.smithi196.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-08T08:50:08.522 INFO:teuthology.orchestra.run.smithi196.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-08T08:50:08.522 INFO:teuthology.orchestra.run.smithi196.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-08T08:50:08.522 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-08T08:50:08.522 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-08T08:50:08.522 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-08T08:50:08.522 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-08T08:50:08.522 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-08T08:50:08.522 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-08T08:50:08.522 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-08T08:50:08.522 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-08T08:50:08.522 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-08T08:50:08.522 INFO:teuthology.orchestra.run.smithi196.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-08T08:50:08.523 INFO:teuthology.orchestra.run.smithi196.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-08T08:50:08.523 INFO:teuthology.orchestra.run.smithi196.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-08T08:50:08.523 INFO:teuthology.orchestra.run.smithi196.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-08T08:50:08.523 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-08T08:50:08.523 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ply-3.11-14.el9.noarch 2024-05-08T08:50:08.523 INFO:teuthology.orchestra.run.smithi196.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-08T08:50:08.523 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-08T08:50:08.523 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-08T08:50:08.523 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-08T08:50:08.523 INFO:teuthology.orchestra.run.smithi196.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-08T08:50:08.523 INFO:teuthology.orchestra.run.smithi196.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-08T08:50:08.523 INFO:teuthology.orchestra.run.smithi196.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-08T08:50:08.524 INFO:teuthology.orchestra.run.smithi196.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-08T08:50:08.524 INFO:teuthology.orchestra.run.smithi196.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-08T08:50:08.524 INFO:teuthology.orchestra.run.smithi196.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-08T08:50:08.524 INFO:teuthology.orchestra.run.smithi196.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-08T08:50:08.524 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:50:08.524 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-05-08T08:50:08.645 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum -y install ceph-mgr 2024-05-08T08:50:08.795 INFO:teuthology.orchestra.run.smithi192.stdout:cephadm-19.0.0-2515.g96805dcf.el9.noarch.rpm 1.4 MB/s | 763 kB 00:00 2024-05-08T08:50:08.797 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:50:08.797 INFO:teuthology.orchestra.run.smithi192.stdout:Total 1.4 MB/s | 763 kB 00:00 2024-05-08T08:50:08.797 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-05-08T08:50:08.803 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-05-08T08:50:08.803 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-05-08T08:50:08.808 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-05-08T08:50:08.808 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-05-08T08:50:08.878 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-05-08T08:50:08.908 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install ceph-base 2024-05-08T08:50:09.233 INFO:teuthology.orchestra.run.smithi059.stdout:Last metadata expiration check: 0:01:04 ago on Wed 08 May 2024 08:49:05 AM UTC. 2024-05-08T08:50:09.319 INFO:teuthology.orchestra.run.smithi059.stdout:Package ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 is already installed. 2024-05-08T08:50:09.327 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 1/1 2024-05-08T08:50:09.363 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies resolved. 2024-05-08T08:50:09.364 INFO:teuthology.orchestra.run.smithi059.stdout:Nothing to do. 2024-05-08T08:50:09.364 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2024-05-08T08:50:09.399 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 1/1 2024-05-08T08:50:09.426 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum -y install ceph-mgr-dashboard 2024-05-08T08:50:09.478 INFO:teuthology.orchestra.run.smithi196.stdout:Last metadata expiration check: 0:00:58 ago on Wed 08 May 2024 08:49:11 AM UTC. 2024-05-08T08:50:09.559 INFO:teuthology.orchestra.run.smithi196.stdout:Package ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 is already installed. 2024-05-08T08:50:09.601 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-05-08T08:50:09.601 INFO:teuthology.orchestra.run.smithi196.stdout:Nothing to do. 2024-05-08T08:50:09.601 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-05-08T08:50:09.668 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install cephadm 2024-05-08T08:50:09.764 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 1/1 2024-05-08T08:50:10.009 INFO:teuthology.orchestra.run.smithi059.stdout:Last metadata expiration check: 0:01:05 ago on Wed 08 May 2024 08:49:05 AM UTC. 2024-05-08T08:50:10.029 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 1/1 2024-05-08T08:50:10.030 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:50:10.030 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-05-08T08:50:10.030 INFO:teuthology.orchestra.run.smithi192.stdout: cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-08T08:50:10.030 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:50:10.030 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-05-08T08:50:10.136 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies resolved. 2024-05-08T08:50:10.138 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-05-08T08:50:10.138 INFO:teuthology.orchestra.run.smithi059.stdout: Package Arch Version Repository Size 2024-05-08T08:50:10.138 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-05-08T08:50:10.138 INFO:teuthology.orchestra.run.smithi059.stdout:Installing: 2024-05-08T08:50:10.138 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 3.5 M 2024-05-08T08:50:10.138 INFO:teuthology.orchestra.run.smithi059.stdout:Installing dependencies: 2024-05-08T08:50:10.138 INFO:teuthology.orchestra.run.smithi059.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-08T08:50:10.138 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 32 k 2024-05-08T08:50:10.138 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 16 k 2024-05-08T08:50:10.138 INFO:teuthology.orchestra.run.smithi059.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-08T08:50:10.139 INFO:teuthology.orchestra.run.smithi059.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-08T08:50:10.139 INFO:teuthology.orchestra.run.smithi059.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-08T08:50:10.139 INFO:teuthology.orchestra.run.smithi059.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-08T08:50:10.139 INFO:teuthology.orchestra.run.smithi059.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-08T08:50:10.139 INFO:teuthology.orchestra.run.smithi059.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-08T08:50:10.139 INFO:teuthology.orchestra.run.smithi059.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-08T08:50:10.139 INFO:teuthology.orchestra.run.smithi059.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-08T08:50:10.139 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:50:10.139 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction Summary 2024-05-08T08:50:10.139 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-05-08T08:50:10.139 INFO:teuthology.orchestra.run.smithi059.stdout:Install 12 Packages 2024-05-08T08:50:10.139 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:50:10.140 INFO:teuthology.orchestra.run.smithi059.stdout:Total download size: 8.6 M 2024-05-08T08:50:10.140 INFO:teuthology.orchestra.run.smithi059.stdout:Installed size: 103 M 2024-05-08T08:50:10.140 INFO:teuthology.orchestra.run.smithi059.stdout:Downloading Packages: 2024-05-08T08:50:10.231 INFO:teuthology.orchestra.run.smithi196.stdout:Last metadata expiration check: 0:00:59 ago on Wed 08 May 2024 08:49:11 AM UTC. 2024-05-08T08:50:10.261 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-immutable-object-cache 2024-05-08T08:50:10.354 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-05-08T08:50:10.355 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-05-08T08:50:10.355 INFO:teuthology.orchestra.run.smithi196.stdout: Package Arch Version Repository Size 2024-05-08T08:50:10.355 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-05-08T08:50:10.355 INFO:teuthology.orchestra.run.smithi196.stdout:Installing: 2024-05-08T08:50:10.355 INFO:teuthology.orchestra.run.smithi196.stdout: cephadm noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 763 k 2024-05-08T08:50:10.355 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:50:10.355 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-05-08T08:50:10.355 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-05-08T08:50:10.356 INFO:teuthology.orchestra.run.smithi196.stdout:Install 1 Package 2024-05-08T08:50:10.356 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:50:10.356 INFO:teuthology.orchestra.run.smithi196.stdout:Total download size: 763 k 2024-05-08T08:50:10.356 INFO:teuthology.orchestra.run.smithi196.stdout:Installed size: 769 k 2024-05-08T08:50:10.356 INFO:teuthology.orchestra.run.smithi196.stdout:Downloading Packages: 2024-05-08T08:50:10.572 INFO:teuthology.orchestra.run.smithi059.stdout:(1/12): ceph-prometheus-alerts-19.0.0-2515.g968 85 kB/s | 16 kB 00:00 2024-05-08T08:50:10.622 INFO:teuthology.orchestra.run.smithi059.stdout:(2/12): ceph-grafana-dashboards-19.0.0-2515.g96 130 kB/s | 32 kB 00:00 2024-05-08T08:50:10.736 INFO:teuthology.orchestra.run.smithi196.stdout:cephadm-19.0.0-2515.g96805dcf.el9.noarch.rpm 2.0 MB/s | 763 kB 00:00 2024-05-08T08:50:10.737 INFO:teuthology.orchestra.run.smithi196.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:50:10.737 INFO:teuthology.orchestra.run.smithi196.stdout:Total 2.0 MB/s | 763 kB 00:00 2024-05-08T08:50:10.737 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction check 2024-05-08T08:50:10.742 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction check succeeded. 2024-05-08T08:50:10.742 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction test 2024-05-08T08:50:10.748 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction test succeeded. 2024-05-08T08:50:10.748 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction 2024-05-08T08:50:10.806 INFO:teuthology.orchestra.run.smithi059.stdout:(3/12): protobuf-3.14.0-13.el9.x86_64.rpm 4.3 MB/s | 1.0 MB 00:00 2024-05-08T08:50:10.821 INFO:teuthology.orchestra.run.smithi196.stdout: Preparing : 1/1 2024-05-08T08:50:10.838 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:05 ago on Wed 08 May 2024 08:49:05 AM UTC. 2024-05-08T08:50:10.839 INFO:teuthology.orchestra.run.smithi059.stdout:(4/12): python3-protobuf-3.14.0-13.el9.noarch.r 1.2 MB/s | 269 kB 00:00 2024-05-08T08:50:10.940 INFO:teuthology.orchestra.run.smithi059.stdout:(5/12): protobuf-compiler-3.14.0-13.el9.x86_64. 6.3 MB/s | 863 kB 00:00 2024-05-08T08:50:10.963 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-05-08T08:50:10.964 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-05-08T08:50:10.964 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repo Size 2024-05-08T08:50:10.964 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-05-08T08:50:10.964 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-05-08T08:50:10.964 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 145 k 2024-05-08T08:50:10.964 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:50:10.964 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-05-08T08:50:10.964 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-05-08T08:50:10.964 INFO:teuthology.orchestra.run.smithi192.stdout:Install 1 Package 2024-05-08T08:50:10.964 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:50:10.964 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 145 k 2024-05-08T08:50:10.965 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 435 k 2024-05-08T08:50:10.965 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-05-08T08:50:11.090 INFO:teuthology.orchestra.run.smithi059.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 130 kB/s | 19 kB 00:00 2024-05-08T08:50:11.165 INFO:teuthology.orchestra.run.smithi059.stdout:(7/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 1.6 MB/s | 548 kB 00:00 2024-05-08T08:50:11.232 INFO:teuthology.orchestra.run.smithi059.stdout:(8/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 2.1 MB/s | 144 kB 00:00 2024-05-08T08:50:11.294 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 1/1 2024-05-08T08:50:11.311 INFO:teuthology.orchestra.run.smithi192.stdout:ceph-immutable-object-cache-19.0.0-2515.g96805d 418 kB/s | 145 kB 00:00 2024-05-08T08:50:11.312 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:50:11.313 INFO:teuthology.orchestra.run.smithi192.stdout:Total 416 kB/s | 145 kB 00:00 2024-05-08T08:50:11.313 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-05-08T08:50:11.316 INFO:teuthology.orchestra.run.smithi059.stdout:(9/12): ceph-mgr-dashboard-19.0.0-2515.g96805dc 3.8 MB/s | 3.5 MB 00:00 2024-05-08T08:50:11.320 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-05-08T08:50:11.320 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-05-08T08:50:11.333 INFO:teuthology.orchestra.run.smithi059.stdout:(10/12): python3-repoze-lru-0.7-16.el9.noarch.r 306 kB/s | 31 kB 00:00 2024-05-08T08:50:11.377 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-05-08T08:50:11.377 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-05-08T08:50:11.381 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 1/1 2024-05-08T08:50:11.475 INFO:teuthology.orchestra.run.smithi059.stdout:(11/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 5.3 MB/s | 2.0 MB 00:00 2024-05-08T08:50:11.525 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-05-08T08:50:11.591 INFO:teuthology.orchestra.run.smithi059.stdout:(12/12): python3-routes-2.5.1-5.el9.noarch.rpm 685 kB/s | 188 kB 00:00 2024-05-08T08:50:11.593 INFO:teuthology.orchestra.run.smithi059.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:50:11.593 INFO:teuthology.orchestra.run.smithi059.stdout:Total 5.9 MB/s | 8.6 MB 00:01 2024-05-08T08:50:11.604 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2515.g96805dcf. 1/1 2024-05-08T08:50:11.630 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2515.g96805dcf. 1/1 2024-05-08T08:50:11.630 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T08:50:11.630 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-08T08:50:11.630 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:50:11.643 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction check 2024-05-08T08:50:11.661 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction check succeeded. 2024-05-08T08:50:11.662 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction test 2024-05-08T08:50:11.751 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 1/1 2024-05-08T08:50:11.850 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction test succeeded. 2024-05-08T08:50:11.851 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction 2024-05-08T08:50:11.987 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 1/1 2024-05-08T08:50:11.988 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:50:11.988 INFO:teuthology.orchestra.run.smithi196.stdout:Installed: 2024-05-08T08:50:11.988 INFO:teuthology.orchestra.run.smithi196.stdout: cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-08T08:50:11.988 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:50:11.988 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-05-08T08:50:12.198 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install ceph-immutable-object-cache 2024-05-08T08:50:12.449 INFO:teuthology.orchestra.run.smithi059.stdout: Preparing : 1/1 2024-05-08T08:50:12.518 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-05-08T08:50:12.529 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2515.g96805dcf. 1/1 2024-05-08T08:50:12.529 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:50:12.529 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-05-08T08:50:12.530 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-immutable-object-cache-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:50:12.530 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:50:12.530 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-05-08T08:50:12.561 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-05-08T08:50:12.609 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-05-08T08:50:12.646 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-05-08T08:50:12.743 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-05-08T08:50:12.766 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-mgr 2024-05-08T08:50:12.768 INFO:teuthology.orchestra.run.smithi196.stdout:Last metadata expiration check: 0:01:01 ago on Wed 08 May 2024 08:49:11 AM UTC. 2024-05-08T08:50:12.881 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-05-08T08:50:12.894 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-05-08T08:50:12.895 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-05-08T08:50:12.895 INFO:teuthology.orchestra.run.smithi196.stdout: Package Arch Version Repo Size 2024-05-08T08:50:12.895 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-05-08T08:50:12.895 INFO:teuthology.orchestra.run.smithi196.stdout:Installing: 2024-05-08T08:50:12.895 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 145 k 2024-05-08T08:50:12.895 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:50:12.895 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-05-08T08:50:12.895 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-05-08T08:50:12.895 INFO:teuthology.orchestra.run.smithi196.stdout:Install 1 Package 2024-05-08T08:50:12.896 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:50:12.896 INFO:teuthology.orchestra.run.smithi196.stdout:Total download size: 145 k 2024-05-08T08:50:12.896 INFO:teuthology.orchestra.run.smithi196.stdout:Installed size: 435 k 2024-05-08T08:50:12.896 INFO:teuthology.orchestra.run.smithi196.stdout:Downloading Packages: 2024-05-08T08:50:12.972 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-05-08T08:50:13.021 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-05-08T08:50:13.056 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-05-08T08:50:13.088 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2515.g96805dcf.el9 10/12 2024-05-08T08:50:13.094 INFO:teuthology.orchestra.run.smithi196.stdout:ceph-immutable-object-cache-19.0.0-2515.g96805d 734 kB/s | 145 kB 00:00 2024-05-08T08:50:13.095 INFO:teuthology.orchestra.run.smithi196.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:50:13.095 INFO:teuthology.orchestra.run.smithi196.stdout:Total 729 kB/s | 145 kB 00:00 2024-05-08T08:50:13.095 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction check 2024-05-08T08:50:13.102 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction check succeeded. 2024-05-08T08:50:13.102 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction test 2024-05-08T08:50:13.159 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction test succeeded. 2024-05-08T08:50:13.159 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction 2024-05-08T08:50:13.306 INFO:teuthology.orchestra.run.smithi196.stdout: Preparing : 1/1 2024-05-08T08:50:13.343 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:08 ago on Wed 08 May 2024 08:49:05 AM UTC. 2024-05-08T08:50:13.395 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2515.g96805dcf. 1/1 2024-05-08T08:50:13.420 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2515.g96805dcf. 1/1 2024-05-08T08:50:13.420 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T08:50:13.420 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-08T08:50:13.420 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:50:13.425 INFO:teuthology.orchestra.run.smithi192.stdout:Package ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 is already installed. 2024-05-08T08:50:13.468 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-05-08T08:50:13.469 INFO:teuthology.orchestra.run.smithi192.stdout:Nothing to do. 2024-05-08T08:50:13.469 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-05-08T08:50:13.531 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-mgr-dashboard 2024-05-08T08:50:14.090 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:09 ago on Wed 08 May 2024 08:49:05 AM UTC. 2024-05-08T08:50:14.187 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2515.g96805dcf.el 11/12 2024-05-08T08:50:14.213 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-05-08T08:50:14.213 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-05-08T08:50:14.213 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2024-05-08T08:50:14.213 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-05-08T08:50:14.213 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-05-08T08:50:14.214 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 3.5 M 2024-05-08T08:50:14.214 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2024-05-08T08:50:14.214 INFO:teuthology.orchestra.run.smithi192.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-08T08:50:14.214 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 32 k 2024-05-08T08:50:14.214 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 16 k 2024-05-08T08:50:14.214 INFO:teuthology.orchestra.run.smithi192.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-08T08:50:14.214 INFO:teuthology.orchestra.run.smithi192.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-08T08:50:14.214 INFO:teuthology.orchestra.run.smithi192.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-08T08:50:14.214 INFO:teuthology.orchestra.run.smithi192.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-08T08:50:14.214 INFO:teuthology.orchestra.run.smithi192.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-08T08:50:14.214 INFO:teuthology.orchestra.run.smithi192.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-08T08:50:14.214 INFO:teuthology.orchestra.run.smithi192.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-08T08:50:14.215 INFO:teuthology.orchestra.run.smithi192.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-08T08:50:14.215 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:50:14.215 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-05-08T08:50:14.215 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-05-08T08:50:14.215 INFO:teuthology.orchestra.run.smithi192.stdout:Install 12 Packages 2024-05-08T08:50:14.215 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:50:14.216 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 8.6 M 2024-05-08T08:50:14.216 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 103 M 2024-05-08T08:50:14.216 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-05-08T08:50:14.231 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.noa 12/12 2024-05-08T08:50:14.305 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2515.g96805dcf. 1/1 2024-05-08T08:50:14.305 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:50:14.305 INFO:teuthology.orchestra.run.smithi196.stdout:Installed: 2024-05-08T08:50:14.305 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-immutable-object-cache-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:50:14.305 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:50:14.305 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-05-08T08:50:14.539 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install ceph-mgr 2024-05-08T08:50:14.884 INFO:teuthology.orchestra.run.smithi192.stdout:(1/12): ceph-prometheus-alerts-19.0.0-2515.g968 88 kB/s | 16 kB 00:00 2024-05-08T08:50:14.884 INFO:teuthology.orchestra.run.smithi192.stdout:(2/12): ceph-grafana-dashboards-19.0.0-2515.g96 134 kB/s | 32 kB 00:00 2024-05-08T08:50:14.976 INFO:teuthology.orchestra.run.smithi192.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 2.9 MB/s | 269 kB 00:00 2024-05-08T08:50:15.000 INFO:teuthology.orchestra.run.smithi192.stdout:(4/12): protobuf-3.14.0-13.el9.x86_64.rpm 6.0 MB/s | 1.0 MB 00:00 2024-05-08T08:50:15.110 INFO:teuthology.orchestra.run.smithi196.stdout:Last metadata expiration check: 0:01:04 ago on Wed 08 May 2024 08:49:11 AM UTC. 2024-05-08T08:50:15.168 INFO:teuthology.orchestra.run.smithi192.stdout:(5/12): protobuf-compiler-3.14.0-13.el9.x86_64. 4.4 MB/s | 863 kB 00:00 2024-05-08T08:50:15.194 INFO:teuthology.orchestra.run.smithi196.stdout:Package ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 is already installed. 2024-05-08T08:50:15.237 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-05-08T08:50:15.238 INFO:teuthology.orchestra.run.smithi196.stdout:Nothing to do. 2024-05-08T08:50:15.238 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-05-08T08:50:15.251 INFO:teuthology.orchestra.run.smithi192.stdout:(6/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 2.1 MB/s | 548 kB 00:00 2024-05-08T08:50:15.285 INFO:teuthology.orchestra.run.smithi192.stdout:(7/12): grpc-data-1.46.7-10.el9.noarch.rpm 165 kB/s | 19 kB 00:00 2024-05-08T08:50:15.300 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install ceph-mgr-dashboard 2024-05-08T08:50:15.394 INFO:teuthology.orchestra.run.smithi192.stdout:(8/12): python3-grpcio-1.46.7-10.el9.x86_64.rpm 14 MB/s | 2.0 MB 00:00 2024-05-08T08:50:15.419 INFO:teuthology.orchestra.run.smithi192.stdout:(9/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 1.1 MB/s | 144 kB 00:00 2024-05-08T08:50:15.444 INFO:teuthology.orchestra.run.smithi192.stdout:(10/12): python3-repoze-lru-0.7-16.el9.noarch.r 610 kB/s | 31 kB 00:00 2024-05-08T08:50:15.483 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.noa 12/12 2024-05-08T08:50:15.483 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2515.g96805dcf.el 1/12 2024-05-08T08:50:15.483 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.noa 2/12 2024-05-08T08:50:15.483 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2515.g96805dcf.el9 3/12 2024-05-08T08:50:15.483 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-05-08T08:50:15.483 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-05-08T08:50:15.483 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-05-08T08:50:15.483 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-05-08T08:50:15.483 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-05-08T08:50:15.483 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-05-08T08:50:15.484 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-05-08T08:50:15.484 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-05-08T08:50:15.486 INFO:teuthology.orchestra.run.smithi192.stdout:(11/12): python3-routes-2.5.1-5.el9.noarch.rpm 2.8 MB/s | 188 kB 00:00 2024-05-08T08:50:15.783 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-05-08T08:50:15.783 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:50:15.783 INFO:teuthology.orchestra.run.smithi059.stdout:Installed: 2024-05-08T08:50:15.784 INFO:teuthology.orchestra.run.smithi059.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-08T08:50:15.784 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-grafana-dashboards-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-08T08:50:15.784 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-08T08:50:15.784 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-prometheus-alerts-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-08T08:50:15.784 INFO:teuthology.orchestra.run.smithi059.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-08T08:50:15.784 INFO:teuthology.orchestra.run.smithi059.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-08T08:50:15.784 INFO:teuthology.orchestra.run.smithi059.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-08T08:50:15.784 INFO:teuthology.orchestra.run.smithi059.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-08T08:50:15.784 INFO:teuthology.orchestra.run.smithi059.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-08T08:50:15.784 INFO:teuthology.orchestra.run.smithi059.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-08T08:50:15.784 INFO:teuthology.orchestra.run.smithi059.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-08T08:50:15.785 INFO:teuthology.orchestra.run.smithi059.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-08T08:50:15.785 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:50:15.785 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2024-05-08T08:50:15.811 INFO:teuthology.orchestra.run.smithi192.stdout:(12/12): ceph-mgr-dashboard-19.0.0-2515.g96805d 3.0 MB/s | 3.5 MB 00:01 2024-05-08T08:50:15.813 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:50:15.813 INFO:teuthology.orchestra.run.smithi192.stdout:Total 5.4 MB/s | 8.6 MB 00:01 2024-05-08T08:50:15.861 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-05-08T08:50:15.862 INFO:teuthology.orchestra.run.smithi196.stdout:Last metadata expiration check: 0:01:04 ago on Wed 08 May 2024 08:49:11 AM UTC. 2024-05-08T08:50:15.878 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-05-08T08:50:15.878 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-05-08T08:50:15.985 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-05-08T08:50:15.986 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-05-08T08:50:15.986 INFO:teuthology.orchestra.run.smithi196.stdout: Package Arch Version Repository Size 2024-05-08T08:50:15.986 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-05-08T08:50:15.986 INFO:teuthology.orchestra.run.smithi196.stdout:Installing: 2024-05-08T08:50:15.986 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 3.5 M 2024-05-08T08:50:15.986 INFO:teuthology.orchestra.run.smithi196.stdout:Installing dependencies: 2024-05-08T08:50:15.987 INFO:teuthology.orchestra.run.smithi196.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-08T08:50:15.987 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 32 k 2024-05-08T08:50:15.987 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 16 k 2024-05-08T08:50:15.987 INFO:teuthology.orchestra.run.smithi196.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-08T08:50:15.987 INFO:teuthology.orchestra.run.smithi196.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-08T08:50:15.987 INFO:teuthology.orchestra.run.smithi196.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-08T08:50:15.987 INFO:teuthology.orchestra.run.smithi196.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-08T08:50:15.987 INFO:teuthology.orchestra.run.smithi196.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-08T08:50:15.987 INFO:teuthology.orchestra.run.smithi196.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-08T08:50:15.987 INFO:teuthology.orchestra.run.smithi196.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-08T08:50:15.987 INFO:teuthology.orchestra.run.smithi196.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-08T08:50:15.987 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:50:15.987 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-05-08T08:50:15.988 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-05-08T08:50:15.988 INFO:teuthology.orchestra.run.smithi196.stdout:Install 12 Packages 2024-05-08T08:50:15.988 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:50:15.988 INFO:teuthology.orchestra.run.smithi196.stdout:Total download size: 8.6 M 2024-05-08T08:50:15.988 INFO:teuthology.orchestra.run.smithi196.stdout:Installed size: 103 M 2024-05-08T08:50:15.988 INFO:teuthology.orchestra.run.smithi196.stdout:Downloading Packages: 2024-05-08T08:50:15.989 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum -y install ceph-mgr-diskprediction-local 2024-05-08T08:50:16.057 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-05-08T08:50:16.058 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-05-08T08:50:16.584 INFO:teuthology.orchestra.run.smithi059.stdout:Last metadata expiration check: 0:01:11 ago on Wed 08 May 2024 08:49:05 AM UTC. 2024-05-08T08:50:16.658 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-05-08T08:50:16.681 INFO:teuthology.orchestra.run.smithi196.stdout:(1/12): ceph-grafana-dashboards-19.0.0-2515.g96 124 kB/s | 32 kB 00:00 2024-05-08T08:50:16.707 INFO:teuthology.orchestra.run.smithi196.stdout:(2/12): ceph-prometheus-alerts-19.0.0-2515.g968 59 kB/s | 16 kB 00:00 2024-05-08T08:50:16.713 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies resolved. 2024-05-08T08:50:16.714 INFO:teuthology.orchestra.run.smithi059.stdout:======================================================================================= 2024-05-08T08:50:16.714 INFO:teuthology.orchestra.run.smithi059.stdout: Package Arch Version Repository Size 2024-05-08T08:50:16.714 INFO:teuthology.orchestra.run.smithi059.stdout:======================================================================================= 2024-05-08T08:50:16.714 INFO:teuthology.orchestra.run.smithi059.stdout:Installing: 2024-05-08T08:50:16.714 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 7.4 M 2024-05-08T08:50:16.714 INFO:teuthology.orchestra.run.smithi059.stdout:Installing dependencies: 2024-05-08T08:50:16.714 INFO:teuthology.orchestra.run.smithi059.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-08T08:50:16.715 INFO:teuthology.orchestra.run.smithi059.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-08T08:50:16.715 INFO:teuthology.orchestra.run.smithi059.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-08T08:50:16.715 INFO:teuthology.orchestra.run.smithi059.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-08T08:50:16.715 INFO:teuthology.orchestra.run.smithi059.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-08T08:50:16.715 INFO:teuthology.orchestra.run.smithi059.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-08T08:50:16.715 INFO:teuthology.orchestra.run.smithi059.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-08T08:50:16.715 INFO:teuthology.orchestra.run.smithi059.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-08T08:50:16.715 INFO:teuthology.orchestra.run.smithi059.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-08T08:50:16.715 INFO:teuthology.orchestra.run.smithi059.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-08T08:50:16.715 INFO:teuthology.orchestra.run.smithi059.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-08T08:50:16.715 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:50:16.715 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction Summary 2024-05-08T08:50:16.715 INFO:teuthology.orchestra.run.smithi059.stdout:======================================================================================= 2024-05-08T08:50:16.715 INFO:teuthology.orchestra.run.smithi059.stdout:Install 12 Packages 2024-05-08T08:50:16.716 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:50:16.716 INFO:teuthology.orchestra.run.smithi059.stdout:Total download size: 38 M 2024-05-08T08:50:16.716 INFO:teuthology.orchestra.run.smithi059.stdout:Installed size: 209 M 2024-05-08T08:50:16.716 INFO:teuthology.orchestra.run.smithi059.stdout:Downloading Packages: 2024-05-08T08:50:16.745 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-05-08T08:50:16.780 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-05-08T08:50:16.826 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-05-08T08:50:16.866 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-05-08T08:50:16.943 INFO:teuthology.orchestra.run.smithi059.stdout:(1/12): libquadmath-11.4.1-3.el9.x86_64.rpm 2.0 MB/s | 195 kB 00:00 2024-05-08T08:50:16.973 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-05-08T08:50:16.993 INFO:teuthology.orchestra.run.smithi059.stdout:(2/12): libgfortran-11.4.1-3.el9.x86_64.rpm 5.3 MB/s | 803 kB 00:00 2024-05-08T08:50:17.010 INFO:teuthology.orchestra.run.smithi059.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 488 kB/s | 33 kB 00:00 2024-05-08T08:50:17.052 INFO:teuthology.orchestra.run.smithi059.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 439 kB/s | 18 kB 00:00 2024-05-08T08:50:17.094 INFO:teuthology.orchestra.run.smithi059.stdout:(5/12): openblas-0.3.21-2.el9.x86_64.rpm 847 kB/s | 35 kB 00:00 2024-05-08T08:50:17.099 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-05-08T08:50:17.193 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-05-08T08:50:17.202 INFO:teuthology.orchestra.run.smithi059.stdout:(6/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 14 MB/s | 3.0 MB 00:00 2024-05-08T08:50:17.238 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-05-08T08:50:17.267 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-05-08T08:50:17.278 INFO:teuthology.orchestra.run.smithi059.stdout:(7/12): python3-devel-3.9.18-3.el9.x86_64.rpm 3.2 MB/s | 244 kB 00:00 2024-05-08T08:50:17.290 INFO:teuthology.orchestra.run.smithi196.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 462 kB/s | 269 kB 00:00 2024-05-08T08:50:17.298 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2515.g96805dcf.el9 10/12 2024-05-08T08:50:17.386 INFO:teuthology.orchestra.run.smithi059.stdout:(8/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 16 MB/s | 4.7 MB 00:00 2024-05-08T08:50:17.390 INFO:teuthology.orchestra.run.smithi196.stdout:(4/12): ceph-mgr-dashboard-19.0.0-2515.g96805dc 3.6 MB/s | 3.5 MB 00:00 2024-05-08T08:50:17.462 INFO:teuthology.orchestra.run.smithi059.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 3.3 MB/s | 250 kB 00:00 2024-05-08T08:50:17.499 INFO:teuthology.orchestra.run.smithi196.stdout:(5/12): protobuf-3.14.0-13.el9.x86_64.rpm 1.2 MB/s | 1.0 MB 00:00 2024-05-08T08:50:17.524 INFO:teuthology.orchestra.run.smithi196.stdout:(6/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 4.0 MB/s | 548 kB 00:00 2024-05-08T08:50:17.558 INFO:teuthology.orchestra.run.smithi196.stdout:(7/12): grpc-data-1.46.7-10.el9.noarch.rpm 332 kB/s | 19 kB 00:00 2024-05-08T08:50:17.578 INFO:teuthology.orchestra.run.smithi059.stdout:(10/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 17 MB/s | 5.1 MB 00:00 2024-05-08T08:50:17.608 INFO:teuthology.orchestra.run.smithi196.stdout:(8/12): python3-grpcio-1.46.7-10.el9.x86_64.rpm 24 MB/s | 2.0 MB 00:00 2024-05-08T08:50:17.633 INFO:teuthology.orchestra.run.smithi196.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 1.2 MB/s | 31 kB 00:00 2024-05-08T08:50:17.659 INFO:teuthology.orchestra.run.smithi196.stdout:(10/12): python3-grpcio-tools-1.46.7-10.el9.x86 1.4 MB/s | 144 kB 00:00 2024-05-08T08:50:17.684 INFO:teuthology.orchestra.run.smithi196.stdout:(11/12): python3-routes-2.5.1-5.el9.noarch.rpm 3.7 MB/s | 188 kB 00:00 2024-05-08T08:50:17.751 INFO:teuthology.orchestra.run.smithi196.stdout:(12/12): protobuf-compiler-3.14.0-13.el9.x86_64 1.8 MB/s | 863 kB 00:00 2024-05-08T08:50:17.752 INFO:teuthology.orchestra.run.smithi196.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:50:17.753 INFO:teuthology.orchestra.run.smithi196.stdout:Total 4.9 MB/s | 8.6 MB 00:01 2024-05-08T08:50:17.801 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction check 2024-05-08T08:50:17.817 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction check succeeded. 2024-05-08T08:50:17.818 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction test 2024-05-08T08:50:17.986 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction test succeeded. 2024-05-08T08:50:17.987 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction 2024-05-08T08:50:18.020 INFO:teuthology.orchestra.run.smithi059.stdout:(11/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 29 MB/s | 16 MB 00:00 2024-05-08T08:50:18.204 INFO:teuthology.orchestra.run.smithi059.stdout:(12/12): ceph-mgr-diskprediction-local-19.0.0-2 5.5 MB/s | 7.4 MB 00:01 2024-05-08T08:50:18.205 INFO:teuthology.orchestra.run.smithi059.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:50:18.205 INFO:teuthology.orchestra.run.smithi059.stdout:Total 26 MB/s | 38 MB 00:01 2024-05-08T08:50:18.331 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2515.g96805dcf.el 11/12 2024-05-08T08:50:18.366 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.noa 12/12 2024-05-08T08:50:18.570 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction check 2024-05-08T08:50:18.586 INFO:teuthology.orchestra.run.smithi196.stdout: Preparing : 1/1 2024-05-08T08:50:18.598 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction check succeeded. 2024-05-08T08:50:18.598 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction test 2024-05-08T08:50:18.661 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-05-08T08:50:18.704 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-05-08T08:50:18.750 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-05-08T08:50:18.780 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-05-08T08:50:18.873 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-05-08T08:50:18.999 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-05-08T08:50:19.060 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction test succeeded. 2024-05-08T08:50:19.060 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction 2024-05-08T08:50:19.084 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-05-08T08:50:19.127 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-05-08T08:50:19.164 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-05-08T08:50:19.195 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2515.g96805dcf.el9 10/12 2024-05-08T08:50:19.722 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.noa 12/12 2024-05-08T08:50:19.723 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2515.g96805dcf.el 1/12 2024-05-08T08:50:19.723 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.noa 2/12 2024-05-08T08:50:19.723 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2515.g96805dcf.el9 3/12 2024-05-08T08:50:19.723 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-05-08T08:50:19.723 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-05-08T08:50:19.723 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-05-08T08:50:19.723 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-05-08T08:50:19.723 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-05-08T08:50:19.723 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-05-08T08:50:19.723 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-05-08T08:50:19.723 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-05-08T08:50:19.845 INFO:teuthology.orchestra.run.smithi059.stdout: Preparing : 1/1 2024-05-08T08:50:19.921 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-05-08T08:50:19.961 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-05-08T08:50:20.028 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-05-08T08:50:20.028 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:50:20.028 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-05-08T08:50:20.028 INFO:teuthology.orchestra.run.smithi192.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-08T08:50:20.028 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-grafana-dashboards-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-08T08:50:20.028 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-08T08:50:20.028 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-prometheus-alerts-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-08T08:50:20.028 INFO:teuthology.orchestra.run.smithi192.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-08T08:50:20.028 INFO:teuthology.orchestra.run.smithi192.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-08T08:50:20.028 INFO:teuthology.orchestra.run.smithi192.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-08T08:50:20.028 INFO:teuthology.orchestra.run.smithi192.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-08T08:50:20.028 INFO:teuthology.orchestra.run.smithi192.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-08T08:50:20.029 INFO:teuthology.orchestra.run.smithi192.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-08T08:50:20.029 INFO:teuthology.orchestra.run.smithi192.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-08T08:50:20.029 INFO:teuthology.orchestra.run.smithi192.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-08T08:50:20.029 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:50:20.029 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-05-08T08:50:20.065 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-05-08T08:50:20.104 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-05-08T08:50:20.243 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2515.g96805dcf.el 11/12 2024-05-08T08:50:20.279 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.noa 12/12 2024-05-08T08:50:20.288 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-05-08T08:50:20.294 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-mgr-diskprediction-local 2024-05-08T08:50:20.320 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-05-08T08:50:20.402 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-05-08T08:50:20.860 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:15 ago on Wed 08 May 2024 08:49:05 AM UTC. 2024-05-08T08:50:20.981 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-05-08T08:50:20.983 INFO:teuthology.orchestra.run.smithi192.stdout:======================================================================================= 2024-05-08T08:50:20.983 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2024-05-08T08:50:20.983 INFO:teuthology.orchestra.run.smithi192.stdout:======================================================================================= 2024-05-08T08:50:20.983 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-05-08T08:50:20.983 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 7.4 M 2024-05-08T08:50:20.983 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2024-05-08T08:50:20.983 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-08T08:50:20.983 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-08T08:50:20.983 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-08T08:50:20.983 INFO:teuthology.orchestra.run.smithi192.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-08T08:50:20.983 INFO:teuthology.orchestra.run.smithi192.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-08T08:50:20.983 INFO:teuthology.orchestra.run.smithi192.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-08T08:50:20.984 INFO:teuthology.orchestra.run.smithi192.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-08T08:50:20.984 INFO:teuthology.orchestra.run.smithi192.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-08T08:50:20.984 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-08T08:50:20.984 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-08T08:50:20.984 INFO:teuthology.orchestra.run.smithi192.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-08T08:50:20.984 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:50:20.984 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-05-08T08:50:20.984 INFO:teuthology.orchestra.run.smithi192.stdout:======================================================================================= 2024-05-08T08:50:20.984 INFO:teuthology.orchestra.run.smithi192.stdout:Install 12 Packages 2024-05-08T08:50:20.984 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:50:20.985 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 38 M 2024-05-08T08:50:20.985 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 209 M 2024-05-08T08:50:20.985 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-05-08T08:50:21.082 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-05-08T08:50:21.196 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-05-08T08:50:21.572 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.noa 12/12 2024-05-08T08:50:21.572 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2515.g96805dcf.el 1/12 2024-05-08T08:50:21.573 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.noa 2/12 2024-05-08T08:50:21.573 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2515.g96805dcf.el9 3/12 2024-05-08T08:50:21.573 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-05-08T08:50:21.573 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-05-08T08:50:21.573 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-05-08T08:50:21.573 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-05-08T08:50:21.573 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-05-08T08:50:21.573 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-05-08T08:50:21.573 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-05-08T08:50:21.573 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-05-08T08:50:21.671 INFO:teuthology.orchestra.run.smithi192.stdout:(1/12): libquadmath-11.4.1-3.el9.x86_64.rpm 363 kB/s | 195 kB 00:00 2024-05-08T08:50:21.788 INFO:teuthology.orchestra.run.smithi192.stdout:(2/12): flexiblas-3.0.4-8.el9.x86_64.rpm 281 kB/s | 33 kB 00:00 2024-05-08T08:50:21.887 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-05-08T08:50:21.887 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:50:21.887 INFO:teuthology.orchestra.run.smithi196.stdout:Installed: 2024-05-08T08:50:21.887 INFO:teuthology.orchestra.run.smithi196.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-08T08:50:21.887 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-grafana-dashboards-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-08T08:50:21.887 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-08T08:50:21.887 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-prometheus-alerts-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-08T08:50:21.887 INFO:teuthology.orchestra.run.smithi196.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-08T08:50:21.888 INFO:teuthology.orchestra.run.smithi196.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-08T08:50:21.888 INFO:teuthology.orchestra.run.smithi196.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-08T08:50:21.888 INFO:teuthology.orchestra.run.smithi196.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-08T08:50:21.888 INFO:teuthology.orchestra.run.smithi196.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-08T08:50:21.888 INFO:teuthology.orchestra.run.smithi196.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-08T08:50:21.888 INFO:teuthology.orchestra.run.smithi196.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-08T08:50:21.888 INFO:teuthology.orchestra.run.smithi196.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-08T08:50:21.888 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:50:21.888 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-05-08T08:50:21.914 INFO:teuthology.orchestra.run.smithi192.stdout:(3/12): libgfortran-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 803 kB 00:00 2024-05-08T08:50:21.997 INFO:teuthology.orchestra.run.smithi192.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 218 kB/s | 18 kB 00:00 2024-05-08T08:50:22.081 INFO:teuthology.orchestra.run.smithi192.stdout:(5/12): openblas-0.3.21-2.el9.x86_64.rpm 423 kB/s | 35 kB 00:00 2024-05-08T08:50:22.123 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install ceph-mgr-diskprediction-local 2024-05-08T08:50:22.512 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-05-08T08:50:22.614 INFO:teuthology.orchestra.run.smithi192.stdout:(6/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 3.6 MB/s | 3.0 MB 00:00 2024-05-08T08:50:22.693 INFO:teuthology.orchestra.run.smithi196.stdout:Last metadata expiration check: 0:01:11 ago on Wed 08 May 2024 08:49:11 AM UTC. 2024-05-08T08:50:22.706 INFO:teuthology.orchestra.run.smithi192.stdout:(7/12): python3-devel-3.9.18-3.el9.x86_64.rpm 2.6 MB/s | 244 kB 00:00 2024-05-08T08:50:22.816 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-05-08T08:50:22.817 INFO:teuthology.orchestra.run.smithi196.stdout:======================================================================================= 2024-05-08T08:50:22.817 INFO:teuthology.orchestra.run.smithi196.stdout: Package Arch Version Repository Size 2024-05-08T08:50:22.818 INFO:teuthology.orchestra.run.smithi196.stdout:======================================================================================= 2024-05-08T08:50:22.818 INFO:teuthology.orchestra.run.smithi196.stdout:Installing: 2024-05-08T08:50:22.818 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 7.4 M 2024-05-08T08:50:22.818 INFO:teuthology.orchestra.run.smithi196.stdout:Installing dependencies: 2024-05-08T08:50:22.818 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-08T08:50:22.818 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-08T08:50:22.818 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-08T08:50:22.818 INFO:teuthology.orchestra.run.smithi196.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-08T08:50:22.818 INFO:teuthology.orchestra.run.smithi196.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-08T08:50:22.818 INFO:teuthology.orchestra.run.smithi196.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-08T08:50:22.818 INFO:teuthology.orchestra.run.smithi196.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-08T08:50:22.818 INFO:teuthology.orchestra.run.smithi196.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-08T08:50:22.819 INFO:teuthology.orchestra.run.smithi196.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-08T08:50:22.819 INFO:teuthology.orchestra.run.smithi196.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-08T08:50:22.819 INFO:teuthology.orchestra.run.smithi196.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-08T08:50:22.819 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:50:22.819 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-05-08T08:50:22.819 INFO:teuthology.orchestra.run.smithi196.stdout:======================================================================================= 2024-05-08T08:50:22.819 INFO:teuthology.orchestra.run.smithi196.stdout:Install 12 Packages 2024-05-08T08:50:22.819 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:50:22.819 INFO:teuthology.orchestra.run.smithi196.stdout:Total download size: 38 M 2024-05-08T08:50:22.820 INFO:teuthology.orchestra.run.smithi196.stdout:Installed size: 209 M 2024-05-08T08:50:22.820 INFO:teuthology.orchestra.run.smithi196.stdout:Downloading Packages: 2024-05-08T08:50:23.257 INFO:teuthology.orchestra.run.smithi192.stdout:(8/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 4.0 MB/s | 4.7 MB 00:01 2024-05-08T08:50:23.348 INFO:teuthology.orchestra.run.smithi192.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.7 MB/s | 250 kB 00:00 2024-05-08T08:50:23.383 INFO:teuthology.orchestra.run.smithi196.stdout:(1/12): libquadmath-11.4.1-3.el9.x86_64.rpm 761 kB/s | 195 kB 00:00 2024-05-08T08:50:23.416 INFO:teuthology.orchestra.run.smithi196.stdout:(2/12): libgfortran-11.4.1-3.el9.x86_64.rpm 2.7 MB/s | 803 kB 00:00 2024-05-08T08:50:23.500 INFO:teuthology.orchestra.run.smithi196.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 280 kB/s | 33 kB 00:00 2024-05-08T08:50:23.550 INFO:teuthology.orchestra.run.smithi196.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 364 kB/s | 18 kB 00:00 2024-05-08T08:50:23.597 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-05-08T08:50:23.600 INFO:teuthology.orchestra.run.smithi196.stdout:(5/12): openblas-0.3.21-2.el9.x86_64.rpm 704 kB/s | 35 kB 00:00 2024-05-08T08:50:23.627 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2515.g96805 12/12 2024-05-08T08:50:23.799 INFO:teuthology.orchestra.run.smithi192.stdout:(10/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.7 MB/s | 5.1 MB 00:01 2024-05-08T08:50:24.057 INFO:teuthology.orchestra.run.smithi192.stdout:(11/12): ceph-mgr-diskprediction-local-19.0.0-2 2.5 MB/s | 7.4 MB 00:02 2024-05-08T08:50:24.259 INFO:teuthology.orchestra.run.smithi196.stdout:(6/12): ceph-mgr-diskprediction-local-19.0.0-25 6.5 MB/s | 7.4 MB 00:01 2024-05-08T08:50:24.310 INFO:teuthology.orchestra.run.smithi196.stdout:(7/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 3.3 MB/s | 3.0 MB 00:00 2024-05-08T08:50:24.476 INFO:teuthology.orchestra.run.smithi196.stdout:(8/12): python3-devel-3.9.18-3.el9.x86_64.rpm 1.1 MB/s | 244 kB 00:00 2024-05-08T08:50:24.593 INFO:teuthology.orchestra.run.smithi196.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.1 MB/s | 250 kB 00:00 2024-05-08T08:50:24.910 INFO:teuthology.orchestra.run.smithi196.stdout:(10/12): openblas-openmp-0.3.21-2.el9.x86_64.rp 3.6 MB/s | 4.7 MB 00:01 2024-05-08T08:50:25.377 INFO:teuthology.orchestra.run.smithi196.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.8 MB/s | 5.1 MB 00:01 2024-05-08T08:50:25.985 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2515.g96805 12/12 2024-05-08T08:50:25.986 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2515.g96805 1/12 2024-05-08T08:50:25.986 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-05-08T08:50:25.986 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-05-08T08:50:25.986 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-05-08T08:50:25.986 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-05-08T08:50:25.986 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-05-08T08:50:25.986 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-05-08T08:50:25.986 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-05-08T08:50:25.986 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-05-08T08:50:25.986 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-05-08T08:50:25.986 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-05-08T08:50:26.099 INFO:teuthology.orchestra.run.smithi192.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 5.9 MB/s | 16 MB 00:02 2024-05-08T08:50:26.100 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:50:26.100 INFO:teuthology.orchestra.run.smithi192.stdout:Total 7.4 MB/s | 38 MB 00:05 2024-05-08T08:50:26.399 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-05-08T08:50:26.400 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:50:26.400 INFO:teuthology.orchestra.run.smithi059.stdout:Installed: 2024-05-08T08:50:26.400 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-08T08:50:26.400 INFO:teuthology.orchestra.run.smithi059.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-08T08:50:26.400 INFO:teuthology.orchestra.run.smithi059.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-08T08:50:26.400 INFO:teuthology.orchestra.run.smithi059.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-08T08:50:26.400 INFO:teuthology.orchestra.run.smithi059.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-08T08:50:26.400 INFO:teuthology.orchestra.run.smithi059.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-08T08:50:26.400 INFO:teuthology.orchestra.run.smithi059.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-08T08:50:26.401 INFO:teuthology.orchestra.run.smithi059.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-08T08:50:26.401 INFO:teuthology.orchestra.run.smithi059.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-08T08:50:26.401 INFO:teuthology.orchestra.run.smithi059.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-08T08:50:26.401 INFO:teuthology.orchestra.run.smithi059.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-08T08:50:26.401 INFO:teuthology.orchestra.run.smithi059.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-08T08:50:26.401 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:50:26.401 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2024-05-08T08:50:26.452 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-05-08T08:50:26.478 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-05-08T08:50:26.479 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-05-08T08:50:26.749 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum -y install ceph-mgr-rook 2024-05-08T08:50:26.906 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-05-08T08:50:26.906 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-05-08T08:50:27.347 INFO:teuthology.orchestra.run.smithi059.stdout:Last metadata expiration check: 0:01:22 ago on Wed 08 May 2024 08:49:05 AM UTC. 2024-05-08T08:50:27.475 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies resolved. 2024-05-08T08:50:27.476 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-05-08T08:50:27.477 INFO:teuthology.orchestra.run.smithi059.stdout: Package Arch Version Repository Size 2024-05-08T08:50:27.477 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-05-08T08:50:27.477 INFO:teuthology.orchestra.run.smithi059.stdout:Installing: 2024-05-08T08:50:27.477 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-rook noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 49 k 2024-05-08T08:50:27.477 INFO:teuthology.orchestra.run.smithi059.stdout:Installing dependencies: 2024-05-08T08:50:27.478 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-08T08:50:27.478 INFO:teuthology.orchestra.run.smithi059.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-08T08:50:27.478 INFO:teuthology.orchestra.run.smithi059.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-08T08:50:27.478 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-08T08:50:27.478 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-08T08:50:27.479 INFO:teuthology.orchestra.run.smithi059.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-08T08:50:27.479 INFO:teuthology.orchestra.run.smithi059.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-08T08:50:27.479 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-08T08:50:27.479 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-08T08:50:27.479 INFO:teuthology.orchestra.run.smithi059.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-08T08:50:27.479 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-08T08:50:27.480 INFO:teuthology.orchestra.run.smithi059.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-08T08:50:27.480 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:50:27.480 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction Summary 2024-05-08T08:50:27.480 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-05-08T08:50:27.480 INFO:teuthology.orchestra.run.smithi059.stdout:Install 13 Packages 2024-05-08T08:50:27.480 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:50:27.481 INFO:teuthology.orchestra.run.smithi059.stdout:Total download size: 2.2 M 2024-05-08T08:50:27.481 INFO:teuthology.orchestra.run.smithi059.stdout:Installed size: 26 M 2024-05-08T08:50:27.481 INFO:teuthology.orchestra.run.smithi059.stdout:Downloading Packages: 2024-05-08T08:50:27.493 INFO:teuthology.orchestra.run.smithi196.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 5.6 MB/s | 16 MB 00:02 2024-05-08T08:50:27.495 INFO:teuthology.orchestra.run.smithi196.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:50:27.495 INFO:teuthology.orchestra.run.smithi196.stdout:Total 8.1 MB/s | 38 MB 00:04 2024-05-08T08:50:27.665 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-05-08T08:50:27.740 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-05-08T08:50:27.780 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-05-08T08:50:27.856 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction check 2024-05-08T08:50:27.883 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction check succeeded. 2024-05-08T08:50:27.884 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction test 2024-05-08T08:50:27.889 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-05-08T08:50:27.924 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-05-08T08:50:28.100 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-05-08T08:50:28.131 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-05-08T08:50:28.213 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-05-08T08:50:28.278 INFO:teuthology.orchestra.run.smithi059.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 76 kB/s | 19 kB 00:00 2024-05-08T08:50:28.294 INFO:teuthology.orchestra.run.smithi059.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 95 kB/s | 26 kB 00:00 2024-05-08T08:50:28.326 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction test succeeded. 2024-05-08T08:50:28.326 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction 2024-05-08T08:50:28.436 INFO:teuthology.orchestra.run.smithi059.stdout:(3/13): ceph-mgr-rook-19.0.0-2515.g96805dcf.el9 120 kB/s | 49 kB 00:00 2024-05-08T08:50:28.487 INFO:teuthology.orchestra.run.smithi059.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 827 kB/s | 159 kB 00:00 2024-05-08T08:50:28.537 INFO:teuthology.orchestra.run.smithi059.stdout:(5/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 858 kB/s | 222 kB 00:00 2024-05-08T08:50:28.554 INFO:teuthology.orchestra.run.smithi059.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 801 kB/s | 54 kB 00:00 2024-05-08T08:50:28.771 INFO:teuthology.orchestra.run.smithi059.stdout:(7/13): python3-pyasn1-modules-0.4.8-6.el9.noar 835 kB/s | 279 kB 00:00 2024-05-08T08:50:28.796 INFO:teuthology.orchestra.run.smithi059.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 62 kB/s | 15 kB 00:00 2024-05-08T08:50:28.821 INFO:teuthology.orchestra.run.smithi059.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 113 kB/s | 32 kB 00:00 2024-05-08T08:50:28.867 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-05-08T08:50:28.913 INFO:teuthology.orchestra.run.smithi059.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 646 kB/s | 59 kB 00:00 2024-05-08T08:50:28.987 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-05-08T08:50:29.022 INFO:teuthology.orchestra.run.smithi059.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 827 kB/s | 90 kB 00:00 2024-05-08T08:50:29.055 INFO:teuthology.orchestra.run.smithi059.stdout:(12/13): python3-google-auth-2.29.0-1.el9.noarc 724 kB/s | 206 kB 00:00 2024-05-08T08:50:29.078 INFO:teuthology.orchestra.run.smithi196.stdout: Preparing : 1/1 2024-05-08T08:50:29.122 INFO:teuthology.orchestra.run.smithi059.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 3.1 MB/s | 1.0 MB 00:00 2024-05-08T08:50:29.123 INFO:teuthology.orchestra.run.smithi059.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:50:29.123 INFO:teuthology.orchestra.run.smithi059.stdout:Total 1.3 MB/s | 2.2 MB 00:01 2024-05-08T08:50:29.144 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-05-08T08:50:29.191 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-05-08T08:50:29.198 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction check 2024-05-08T08:50:29.214 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction check succeeded. 2024-05-08T08:50:29.215 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction test 2024-05-08T08:50:29.291 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-05-08T08:50:29.310 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction test succeeded. 2024-05-08T08:50:29.310 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction 2024-05-08T08:50:29.326 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-05-08T08:50:29.503 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-05-08T08:50:29.533 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-05-08T08:50:29.619 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-05-08T08:50:29.967 INFO:teuthology.orchestra.run.smithi059.stdout: Preparing : 1/1 2024-05-08T08:50:30.137 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-05-08T08:50:30.192 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-05-08T08:50:30.242 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-05-08T08:50:30.256 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-05-08T08:50:30.271 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-05-08T08:50:30.292 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-05-08T08:50:30.342 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-05-08T08:50:30.388 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-05-08T08:50:30.439 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-05-08T08:50:30.667 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 7/13 2024-05-08T08:50:30.719 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-05-08T08:50:31.271 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-05-08T08:50:31.307 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2515.g96805 12/12 2024-05-08T08:50:31.483 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-05-08T08:50:31.541 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-05-08T08:50:31.574 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-05-08T08:50:31.623 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-05-08T08:50:31.633 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-05-08T08:50:31.652 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 13/13 2024-05-08T08:50:32.395 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 13/13 2024-05-08T08:50:32.395 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 1/13 2024-05-08T08:50:32.395 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-05-08T08:50:32.395 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-05-08T08:50:32.396 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-05-08T08:50:32.396 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-05-08T08:50:32.396 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-05-08T08:50:32.396 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-05-08T08:50:32.396 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-05-08T08:50:32.396 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-05-08T08:50:32.396 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 10/13 2024-05-08T08:50:32.396 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-05-08T08:50:32.396 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-05-08T08:50:32.668 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-05-08T08:50:32.691 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2515.g96805 12/12 2024-05-08T08:50:32.722 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-05-08T08:50:32.722 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:50:32.722 INFO:teuthology.orchestra.run.smithi059.stdout:Installed: 2024-05-08T08:50:32.722 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-08T08:50:32.722 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-08T08:50:32.722 INFO:teuthology.orchestra.run.smithi059.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-08T08:50:32.722 INFO:teuthology.orchestra.run.smithi059.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-08T08:50:32.723 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-08T08:50:32.723 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-08T08:50:32.723 INFO:teuthology.orchestra.run.smithi059.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-08T08:50:32.723 INFO:teuthology.orchestra.run.smithi059.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-08T08:50:32.723 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-08T08:50:32.723 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-08T08:50:32.723 INFO:teuthology.orchestra.run.smithi059.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-08T08:50:32.723 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-08T08:50:32.723 INFO:teuthology.orchestra.run.smithi059.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-08T08:50:32.723 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:50:32.723 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2024-05-08T08:50:32.947 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum -y install ceph-mgr-cephadm 2024-05-08T08:50:33.544 INFO:teuthology.orchestra.run.smithi059.stdout:Last metadata expiration check: 0:01:28 ago on Wed 08 May 2024 08:49:05 AM UTC. 2024-05-08T08:50:33.672 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies resolved. 2024-05-08T08:50:33.673 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-05-08T08:50:33.673 INFO:teuthology.orchestra.run.smithi059.stdout: Package Arch Version Repository Size 2024-05-08T08:50:33.673 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-05-08T08:50:33.673 INFO:teuthology.orchestra.run.smithi059.stdout:Installing: 2024-05-08T08:50:33.673 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 142 k 2024-05-08T08:50:33.673 INFO:teuthology.orchestra.run.smithi059.stdout:Installing dependencies: 2024-05-08T08:50:33.673 INFO:teuthology.orchestra.run.smithi059.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-08T08:50:33.673 INFO:teuthology.orchestra.run.smithi059.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-08T08:50:33.673 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-08T08:50:33.674 INFO:teuthology.orchestra.run.smithi059.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-08T08:50:33.674 INFO:teuthology.orchestra.run.smithi059.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-08T08:50:33.674 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:50:33.674 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction Summary 2024-05-08T08:50:33.674 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-05-08T08:50:33.674 INFO:teuthology.orchestra.run.smithi059.stdout:Install 6 Packages 2024-05-08T08:50:33.674 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:50:33.675 INFO:teuthology.orchestra.run.smithi059.stdout:Total download size: 7.0 M 2024-05-08T08:50:33.675 INFO:teuthology.orchestra.run.smithi059.stdout:Installed size: 33 M 2024-05-08T08:50:33.675 INFO:teuthology.orchestra.run.smithi059.stdout:Downloading Packages: 2024-05-08T08:50:33.910 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2515.g96805 12/12 2024-05-08T08:50:33.910 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2515.g96805 1/12 2024-05-08T08:50:33.910 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-05-08T08:50:33.910 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-05-08T08:50:33.910 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-05-08T08:50:33.910 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-05-08T08:50:33.910 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-05-08T08:50:33.910 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-05-08T08:50:33.910 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-05-08T08:50:33.910 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-05-08T08:50:33.911 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-05-08T08:50:33.911 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-05-08T08:50:34.273 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-05-08T08:50:34.273 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:50:34.274 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-05-08T08:50:34.274 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-08T08:50:34.274 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-08T08:50:34.274 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-08T08:50:34.274 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-08T08:50:34.274 INFO:teuthology.orchestra.run.smithi192.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-08T08:50:34.274 INFO:teuthology.orchestra.run.smithi192.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-08T08:50:34.274 INFO:teuthology.orchestra.run.smithi192.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-08T08:50:34.274 INFO:teuthology.orchestra.run.smithi192.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-08T08:50:34.274 INFO:teuthology.orchestra.run.smithi192.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-08T08:50:34.274 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-08T08:50:34.274 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-08T08:50:34.274 INFO:teuthology.orchestra.run.smithi192.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-08T08:50:34.275 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:50:34.275 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-05-08T08:50:34.620 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-mgr-rook 2024-05-08T08:50:34.633 INFO:teuthology.orchestra.run.smithi059.stdout:(1/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 794 kB/s | 248 kB 00:00 2024-05-08T08:50:34.658 INFO:teuthology.orchestra.run.smithi059.stdout:(2/6): ceph-mgr-cephadm-19.0.0-2515.g96805dcf.e 420 kB/s | 142 kB 00:00 2024-05-08T08:50:34.883 INFO:teuthology.orchestra.run.smithi059.stdout:(3/6): python3-babel-2.9.1-2.el9.noarch.rpm 11 MB/s | 6.0 MB 00:00 2024-05-08T08:50:35.009 INFO:teuthology.orchestra.run.smithi059.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 165 kB/s | 58 kB 00:00 2024-05-08T08:50:35.205 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:30 ago on Wed 08 May 2024 08:49:05 AM UTC. 2024-05-08T08:50:35.234 INFO:teuthology.orchestra.run.smithi059.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 146 kB/s | 51 kB 00:00 2024-05-08T08:50:35.269 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2515.g96805 12/12 2024-05-08T08:50:35.269 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2515.g96805 1/12 2024-05-08T08:50:35.269 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-05-08T08:50:35.269 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-05-08T08:50:35.269 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-05-08T08:50:35.269 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-05-08T08:50:35.269 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-05-08T08:50:35.269 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-05-08T08:50:35.269 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-05-08T08:50:35.269 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-05-08T08:50:35.269 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-05-08T08:50:35.270 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-05-08T08:50:35.284 INFO:teuthology.orchestra.run.smithi059.stdout:(6/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 842 kB/s | 548 kB 00:00 2024-05-08T08:50:35.285 INFO:teuthology.orchestra.run.smithi059.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:50:35.285 INFO:teuthology.orchestra.run.smithi059.stdout:Total 4.3 MB/s | 7.0 MB 00:01 2024-05-08T08:50:35.329 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-05-08T08:50:35.330 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-05-08T08:50:35.330 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2024-05-08T08:50:35.330 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-05-08T08:50:35.330 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-05-08T08:50:35.330 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-rook noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 49 k 2024-05-08T08:50:35.331 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2024-05-08T08:50:35.331 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-08T08:50:35.331 INFO:teuthology.orchestra.run.smithi192.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-08T08:50:35.331 INFO:teuthology.orchestra.run.smithi192.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-08T08:50:35.331 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-08T08:50:35.331 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-08T08:50:35.331 INFO:teuthology.orchestra.run.smithi192.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-08T08:50:35.331 INFO:teuthology.orchestra.run.smithi192.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-08T08:50:35.331 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-08T08:50:35.331 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-08T08:50:35.331 INFO:teuthology.orchestra.run.smithi192.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-08T08:50:35.331 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-08T08:50:35.331 INFO:teuthology.orchestra.run.smithi192.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-08T08:50:35.331 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:50:35.332 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-05-08T08:50:35.332 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-05-08T08:50:35.332 INFO:teuthology.orchestra.run.smithi192.stdout:Install 13 Packages 2024-05-08T08:50:35.332 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:50:35.332 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 2.2 M 2024-05-08T08:50:35.332 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 26 M 2024-05-08T08:50:35.332 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-05-08T08:50:35.363 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction check 2024-05-08T08:50:35.376 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction check succeeded. 2024-05-08T08:50:35.376 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction test 2024-05-08T08:50:35.481 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction test succeeded. 2024-05-08T08:50:35.486 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction 2024-05-08T08:50:35.660 INFO:teuthology.orchestra.run.smithi192.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 432 kB/s | 26 kB 00:00 2024-05-08T08:50:35.670 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-05-08T08:50:35.670 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:50:35.670 INFO:teuthology.orchestra.run.smithi196.stdout:Installed: 2024-05-08T08:50:35.670 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-08T08:50:35.670 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-08T08:50:35.670 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-08T08:50:35.671 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-08T08:50:35.671 INFO:teuthology.orchestra.run.smithi196.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-08T08:50:35.671 INFO:teuthology.orchestra.run.smithi196.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-08T08:50:35.671 INFO:teuthology.orchestra.run.smithi196.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-08T08:50:35.671 INFO:teuthology.orchestra.run.smithi196.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-08T08:50:35.671 INFO:teuthology.orchestra.run.smithi196.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-08T08:50:35.671 INFO:teuthology.orchestra.run.smithi196.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-08T08:50:35.671 INFO:teuthology.orchestra.run.smithi196.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-08T08:50:35.671 INFO:teuthology.orchestra.run.smithi196.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-08T08:50:35.671 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:50:35.671 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-05-08T08:50:35.677 INFO:teuthology.orchestra.run.smithi192.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 252 kB/s | 19 kB 00:00 2024-05-08T08:50:35.778 INFO:teuthology.orchestra.run.smithi192.stdout:(3/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.0 MB/s | 159 kB 00:00 2024-05-08T08:50:35.802 INFO:teuthology.orchestra.run.smithi192.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.1 MB/s | 222 kB 00:00 2024-05-08T08:50:35.809 INFO:teuthology.orchestra.run.smithi059.stdout: Preparing : 1/1 2024-05-08T08:50:35.861 INFO:teuthology.orchestra.run.smithi192.stdout:(5/13): python3-requests-oauthlib-1.3.0-12.el9. 922 kB/s | 54 kB 00:00 2024-05-08T08:50:35.886 INFO:teuthology.orchestra.run.smithi192.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 2.5 MB/s | 279 kB 00:00 2024-05-08T08:50:35.912 INFO:teuthology.orchestra.run.smithi192.stdout:(7/13): ceph-mgr-rook-19.0.0-2515.g96805dcf.el9 158 kB/s | 49 kB 00:00 2024-05-08T08:50:36.000 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-05-08T08:50:36.053 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install ceph-mgr-rook 2024-05-08T08:50:36.054 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-05-08T08:50:36.087 INFO:teuthology.orchestra.run.smithi192.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 75 kB/s | 15 kB 00:00 2024-05-08T08:50:36.112 INFO:teuthology.orchestra.run.smithi192.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 128 kB/s | 32 kB 00:00 2024-05-08T08:50:36.163 INFO:teuthology.orchestra.run.smithi192.stdout:(10/13): python3-google-auth-2.29.0-1.el9.noarc 820 kB/s | 206 kB 00:00 2024-05-08T08:50:36.296 INFO:teuthology.orchestra.run.smithi192.stdout:(11/13): python3-kubernetes-26.1.0-2.el9.noarch 4.9 MB/s | 1.0 MB 00:00 2024-05-08T08:50:36.321 INFO:teuthology.orchestra.run.smithi192.stdout:(12/13): python3-rsa-4.9-2.el9.noarch.rpm 283 kB/s | 59 kB 00:00 2024-05-08T08:50:36.346 INFO:teuthology.orchestra.run.smithi192.stdout:(13/13): python3-websocket-client-1.2.3-2.el9.n 489 kB/s | 90 kB 00:00 2024-05-08T08:50:36.349 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:50:36.350 INFO:teuthology.orchestra.run.smithi192.stdout:Total 2.2 MB/s | 2.2 MB 00:01 2024-05-08T08:50:36.414 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-05-08T08:50:36.429 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-05-08T08:50:36.429 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-05-08T08:50:36.513 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-05-08T08:50:36.514 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-05-08T08:50:36.608 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-05-08T08:50:36.632 INFO:teuthology.orchestra.run.smithi196.stdout:Last metadata expiration check: 0:01:25 ago on Wed 08 May 2024 08:49:11 AM UTC. 2024-05-08T08:50:36.700 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-05-08T08:50:36.757 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-05-08T08:50:36.758 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-05-08T08:50:36.758 INFO:teuthology.orchestra.run.smithi196.stdout: Package Arch Version Repository Size 2024-05-08T08:50:36.758 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-05-08T08:50:36.758 INFO:teuthology.orchestra.run.smithi196.stdout:Installing: 2024-05-08T08:50:36.759 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-rook noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 49 k 2024-05-08T08:50:36.759 INFO:teuthology.orchestra.run.smithi196.stdout:Installing dependencies: 2024-05-08T08:50:36.759 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-08T08:50:36.759 INFO:teuthology.orchestra.run.smithi196.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-08T08:50:36.759 INFO:teuthology.orchestra.run.smithi196.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-08T08:50:36.760 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-08T08:50:36.760 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-08T08:50:36.760 INFO:teuthology.orchestra.run.smithi196.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-08T08:50:36.760 INFO:teuthology.orchestra.run.smithi196.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-08T08:50:36.760 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-08T08:50:36.760 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-08T08:50:36.760 INFO:teuthology.orchestra.run.smithi196.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-08T08:50:36.761 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-08T08:50:36.761 INFO:teuthology.orchestra.run.smithi196.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-08T08:50:36.761 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:50:36.761 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-05-08T08:50:36.761 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-05-08T08:50:36.761 INFO:teuthology.orchestra.run.smithi196.stdout:Install 13 Packages 2024-05-08T08:50:36.762 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:50:36.763 INFO:teuthology.orchestra.run.smithi196.stdout:Total download size: 2.2 M 2024-05-08T08:50:36.763 INFO:teuthology.orchestra.run.smithi196.stdout:Installed size: 26 M 2024-05-08T08:50:36.763 INFO:teuthology.orchestra.run.smithi196.stdout:Downloading Packages: 2024-05-08T08:50:36.779 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-05-08T08:50:36.806 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 6/6 2024-05-08T08:50:37.024 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-05-08T08:50:37.066 INFO:teuthology.orchestra.run.smithi196.stdout:(1/13): ceph-mgr-rook-19.0.0-2515.g96805dcf.el9 266 kB/s | 49 kB 00:00 2024-05-08T08:50:37.197 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-05-08T08:50:37.249 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-05-08T08:50:37.313 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-05-08T08:50:37.357 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-05-08T08:50:37.408 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-05-08T08:50:37.483 INFO:teuthology.orchestra.run.smithi196.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 32 kB/s | 19 kB 00:00 2024-05-08T08:50:37.503 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-05-08T08:50:37.539 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 6/6 2024-05-08T08:50:37.539 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 1/6 2024-05-08T08:50:37.539 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-05-08T08:50:37.539 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-05-08T08:50:37.539 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-05-08T08:50:37.539 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-05-08T08:50:37.608 INFO:teuthology.orchestra.run.smithi196.stdout:(3/13): python3-jsonpatch-1.21-16.el9.noarch.rp 35 kB/s | 26 kB 00:00 2024-05-08T08:50:37.629 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 7/13 2024-05-08T08:50:37.685 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-05-08T08:50:37.806 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-05-08T08:50:37.806 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:50:37.806 INFO:teuthology.orchestra.run.smithi059.stdout:Installed: 2024-05-08T08:50:37.806 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-08T08:50:37.806 INFO:teuthology.orchestra.run.smithi059.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-08T08:50:37.806 INFO:teuthology.orchestra.run.smithi059.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-08T08:50:37.806 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-08T08:50:37.806 INFO:teuthology.orchestra.run.smithi059.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-08T08:50:37.807 INFO:teuthology.orchestra.run.smithi059.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-08T08:50:37.807 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:50:37.807 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2024-05-08T08:50:37.966 INFO:teuthology.orchestra.run.smithi196.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 247 kB/s | 222 kB 00:00 2024-05-08T08:50:38.049 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum -y install ceph-fuse 2024-05-08T08:50:38.367 INFO:teuthology.orchestra.run.smithi196.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 180 kB/s | 159 kB 00:00 2024-05-08T08:50:38.437 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-05-08T08:50:38.482 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-05-08T08:50:38.516 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-05-08T08:50:38.564 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-05-08T08:50:38.567 INFO:teuthology.orchestra.run.smithi196.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 291 kB/s | 279 kB 00:00 2024-05-08T08:50:38.594 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 13/13 2024-05-08T08:50:38.617 INFO:teuthology.orchestra.run.smithi196.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 129 kB/s | 32 kB 00:00 2024-05-08T08:50:38.644 INFO:teuthology.orchestra.run.smithi059.stdout:Last metadata expiration check: 0:01:33 ago on Wed 08 May 2024 08:49:05 AM UTC. 2024-05-08T08:50:38.676 INFO:teuthology.orchestra.run.smithi196.stdout:(8/13): python3-requests-oauthlib-1.3.0-12.el9. 76 kB/s | 54 kB 00:00 2024-05-08T08:50:38.771 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies resolved. 2024-05-08T08:50:38.772 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-05-08T08:50:38.772 INFO:teuthology.orchestra.run.smithi059.stdout: Package Arch Version Repository Size 2024-05-08T08:50:38.772 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-05-08T08:50:38.772 INFO:teuthology.orchestra.run.smithi059.stdout:Installing: 2024-05-08T08:50:38.772 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-fuse x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 889 k 2024-05-08T08:50:38.772 INFO:teuthology.orchestra.run.smithi059.stdout:Installing dependencies: 2024-05-08T08:50:38.772 INFO:teuthology.orchestra.run.smithi059.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-08T08:50:38.772 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:50:38.773 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction Summary 2024-05-08T08:50:38.773 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-05-08T08:50:38.773 INFO:teuthology.orchestra.run.smithi059.stdout:Install 2 Packages 2024-05-08T08:50:38.773 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:50:38.773 INFO:teuthology.orchestra.run.smithi059.stdout:Total download size: 970 k 2024-05-08T08:50:38.773 INFO:teuthology.orchestra.run.smithi059.stdout:Installed size: 2.8 M 2024-05-08T08:50:38.773 INFO:teuthology.orchestra.run.smithi059.stdout:Downloading Packages: 2024-05-08T08:50:38.826 INFO:teuthology.orchestra.run.smithi196.stdout:(9/13): python3-certifi-2021.10.8-2.el9.noarch. 58 kB/s | 15 kB 00:00 2024-05-08T08:50:38.868 INFO:teuthology.orchestra.run.smithi196.stdout:(10/13): python3-google-auth-2.29.0-1.el9.noarc 820 kB/s | 206 kB 00:00 2024-05-08T08:50:38.977 INFO:teuthology.orchestra.run.smithi196.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 828 kB/s | 90 kB 00:00 2024-05-08T08:50:39.010 INFO:teuthology.orchestra.run.smithi196.stdout:(12/13): python3-rsa-4.9-2.el9.noarch.rpm 321 kB/s | 59 kB 00:00 2024-05-08T08:50:39.045 INFO:teuthology.orchestra.run.smithi059.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 620 kB/s | 80 kB 00:00 2024-05-08T08:50:39.203 INFO:teuthology.orchestra.run.smithi059.stdout:(2/2): ceph-fuse-19.0.0-2515.g96805dcf.el9.x86_ 3.0 MB/s | 889 kB 00:00 2024-05-08T08:50:39.203 INFO:teuthology.orchestra.run.smithi059.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:50:39.204 INFO:teuthology.orchestra.run.smithi059.stdout:Total 2.2 MB/s | 970 kB 00:00 2024-05-08T08:50:39.216 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction check 2024-05-08T08:50:39.239 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction check succeeded. 2024-05-08T08:50:39.240 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction test 2024-05-08T08:50:39.317 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 13/13 2024-05-08T08:50:39.317 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 1/13 2024-05-08T08:50:39.318 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-05-08T08:50:39.318 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-05-08T08:50:39.318 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-05-08T08:50:39.318 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-05-08T08:50:39.318 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-05-08T08:50:39.318 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-05-08T08:50:39.318 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-05-08T08:50:39.318 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-05-08T08:50:39.318 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 10/13 2024-05-08T08:50:39.318 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-05-08T08:50:39.318 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-05-08T08:50:39.323 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction test succeeded. 2024-05-08T08:50:39.323 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction 2024-05-08T08:50:39.335 INFO:teuthology.orchestra.run.smithi196.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 1.5 MB/s | 1.0 MB 00:00 2024-05-08T08:50:39.337 INFO:teuthology.orchestra.run.smithi196.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:50:39.337 INFO:teuthology.orchestra.run.smithi196.stdout:Total 874 kB/s | 2.2 MB 00:02 2024-05-08T08:50:39.409 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction check 2024-05-08T08:50:39.425 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction check succeeded. 2024-05-08T08:50:39.425 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction test 2024-05-08T08:50:39.511 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction test succeeded. 2024-05-08T08:50:39.513 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction 2024-05-08T08:50:39.516 INFO:teuthology.orchestra.run.smithi059.stdout: Preparing : 1/1 2024-05-08T08:50:39.629 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-05-08T08:50:39.629 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:50:39.629 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-05-08T08:50:39.629 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-08T08:50:39.629 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-08T08:50:39.629 INFO:teuthology.orchestra.run.smithi192.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-08T08:50:39.629 INFO:teuthology.orchestra.run.smithi192.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-08T08:50:39.630 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-08T08:50:39.630 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-08T08:50:39.630 INFO:teuthology.orchestra.run.smithi192.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-08T08:50:39.630 INFO:teuthology.orchestra.run.smithi192.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-08T08:50:39.630 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-08T08:50:39.630 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-08T08:50:39.630 INFO:teuthology.orchestra.run.smithi192.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-08T08:50:39.630 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-08T08:50:39.630 INFO:teuthology.orchestra.run.smithi192.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-08T08:50:39.630 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:50:39.630 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-05-08T08:50:39.659 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-05-08T08:50:39.730 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 2/2 2024-05-08T08:50:39.867 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-mgr-cephadm 2024-05-08T08:50:40.016 INFO:teuthology.orchestra.run.smithi196.stdout: Preparing : 1/1 2024-05-08T08:50:40.176 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-05-08T08:50:40.228 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-05-08T08:50:40.292 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-05-08T08:50:40.330 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-05-08T08:50:40.362 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 2/2 2024-05-08T08:50:40.363 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 1/2 2024-05-08T08:50:40.373 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-05-08T08:50:40.435 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:35 ago on Wed 08 May 2024 08:49:05 AM UTC. 2024-05-08T08:50:40.464 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-05-08T08:50:40.558 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-05-08T08:50:40.558 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-05-08T08:50:40.559 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2024-05-08T08:50:40.559 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-05-08T08:50:40.559 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-05-08T08:50:40.559 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 142 k 2024-05-08T08:50:40.559 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2024-05-08T08:50:40.559 INFO:teuthology.orchestra.run.smithi192.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-08T08:50:40.559 INFO:teuthology.orchestra.run.smithi192.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-08T08:50:40.559 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-08T08:50:40.559 INFO:teuthology.orchestra.run.smithi192.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-08T08:50:40.559 INFO:teuthology.orchestra.run.smithi192.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-08T08:50:40.559 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:50:40.559 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-05-08T08:50:40.560 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-05-08T08:50:40.560 INFO:teuthology.orchestra.run.smithi192.stdout:Install 6 Packages 2024-05-08T08:50:40.560 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:50:40.560 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 7.0 M 2024-05-08T08:50:40.560 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 33 M 2024-05-08T08:50:40.560 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-05-08T08:50:40.566 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 7/13 2024-05-08T08:50:40.598 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-05-08T08:50:40.599 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:50:40.599 INFO:teuthology.orchestra.run.smithi059.stdout:Installed: 2024-05-08T08:50:40.599 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-05-08T08:50:40.599 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:50:40.599 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2024-05-08T08:50:40.614 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-05-08T08:50:40.860 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum -y install ceph-volume 2024-05-08T08:50:41.333 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-05-08T08:50:41.358 INFO:teuthology.orchestra.run.smithi192.stdout:(1/6): ceph-mgr-cephadm-19.0.0-2515.g96805dcf.e 237 kB/s | 142 kB 00:00 2024-05-08T08:50:41.378 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-05-08T08:50:41.412 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-05-08T08:50:41.458 INFO:teuthology.orchestra.run.smithi059.stdout:Last metadata expiration check: 0:01:36 ago on Wed 08 May 2024 08:49:05 AM UTC. 2024-05-08T08:50:41.459 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-05-08T08:50:41.481 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 13/13 2024-05-08T08:50:41.587 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies resolved. 2024-05-08T08:50:41.588 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-05-08T08:50:41.588 INFO:teuthology.orchestra.run.smithi059.stdout: Package Arch Version Repository Size 2024-05-08T08:50:41.588 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-05-08T08:50:41.588 INFO:teuthology.orchestra.run.smithi059.stdout:Installing: 2024-05-08T08:50:41.588 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-volume noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 264 k 2024-05-08T08:50:41.588 INFO:teuthology.orchestra.run.smithi059.stdout:Installing dependencies: 2024-05-08T08:50:41.588 INFO:teuthology.orchestra.run.smithi059.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-08T08:50:41.589 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:50:41.589 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction Summary 2024-05-08T08:50:41.589 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-05-08T08:50:41.589 INFO:teuthology.orchestra.run.smithi059.stdout:Install 2 Packages 2024-05-08T08:50:41.589 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:50:41.589 INFO:teuthology.orchestra.run.smithi059.stdout:Total download size: 341 k 2024-05-08T08:50:41.589 INFO:teuthology.orchestra.run.smithi059.stdout:Installed size: 1.5 M 2024-05-08T08:50:41.589 INFO:teuthology.orchestra.run.smithi059.stdout:Downloading Packages: 2024-05-08T08:50:41.634 INFO:teuthology.orchestra.run.smithi192.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 1.9 MB/s | 548 kB 00:00 2024-05-08T08:50:41.684 INFO:teuthology.orchestra.run.smithi192.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 1.1 MB/s | 58 kB 00:00 2024-05-08T08:50:41.709 INFO:teuthology.orchestra.run.smithi192.stdout:(4/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 261 kB/s | 248 kB 00:00 2024-05-08T08:50:41.735 INFO:teuthology.orchestra.run.smithi192.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 1.0 MB/s | 51 kB 00:00 2024-05-08T08:50:42.185 INFO:teuthology.orchestra.run.smithi192.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 4.2 MB/s | 6.0 MB 00:01 2024-05-08T08:50:42.187 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:50:42.187 INFO:teuthology.orchestra.run.smithi192.stdout:Total 4.3 MB/s | 7.0 MB 00:01 2024-05-08T08:50:42.258 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-05-08T08:50:42.269 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-05-08T08:50:42.270 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-05-08T08:50:42.281 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 13/13 2024-05-08T08:50:42.281 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 1/13 2024-05-08T08:50:42.281 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-05-08T08:50:42.281 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-05-08T08:50:42.281 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-05-08T08:50:42.281 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-05-08T08:50:42.281 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-05-08T08:50:42.281 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-05-08T08:50:42.281 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-05-08T08:50:42.281 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-05-08T08:50:42.281 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 10/13 2024-05-08T08:50:42.282 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-05-08T08:50:42.282 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-05-08T08:50:42.366 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-05-08T08:50:42.367 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-05-08T08:50:42.416 INFO:teuthology.orchestra.run.smithi059.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 99 kB/s | 77 kB 00:00 2024-05-08T08:50:42.592 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-05-08T08:50:42.592 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:50:42.592 INFO:teuthology.orchestra.run.smithi196.stdout:Installed: 2024-05-08T08:50:42.592 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-08T08:50:42.592 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-08T08:50:42.592 INFO:teuthology.orchestra.run.smithi196.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-08T08:50:42.592 INFO:teuthology.orchestra.run.smithi196.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-08T08:50:42.592 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-08T08:50:42.592 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-08T08:50:42.592 INFO:teuthology.orchestra.run.smithi196.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-08T08:50:42.593 INFO:teuthology.orchestra.run.smithi196.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-08T08:50:42.593 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-08T08:50:42.593 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-08T08:50:42.593 INFO:teuthology.orchestra.run.smithi196.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-08T08:50:42.593 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-08T08:50:42.593 INFO:teuthology.orchestra.run.smithi196.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-08T08:50:42.593 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:50:42.593 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-05-08T08:50:42.668 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-05-08T08:50:42.842 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install ceph-mgr-cephadm 2024-05-08T08:50:42.845 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-05-08T08:50:42.903 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-05-08T08:50:43.225 INFO:teuthology.orchestra.run.smithi059.stdout:(2/2): ceph-volume-19.0.0-2515.g96805dcf.el9.no 166 kB/s | 264 kB 00:01 2024-05-08T08:50:43.225 INFO:teuthology.orchestra.run.smithi059.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:50:43.226 INFO:teuthology.orchestra.run.smithi059.stdout:Total 208 kB/s | 341 kB 00:01 2024-05-08T08:50:43.241 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction check 2024-05-08T08:50:43.254 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction check succeeded. 2024-05-08T08:50:43.254 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction test 2024-05-08T08:50:43.293 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction test succeeded. 2024-05-08T08:50:43.294 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction 2024-05-08T08:50:43.414 INFO:teuthology.orchestra.run.smithi196.stdout:Last metadata expiration check: 0:01:32 ago on Wed 08 May 2024 08:49:11 AM UTC. 2024-05-08T08:50:43.417 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-05-08T08:50:43.447 INFO:teuthology.orchestra.run.smithi059.stdout: Preparing : 1/1 2024-05-08T08:50:43.506 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-05-08T08:50:43.536 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-05-08T08:50:43.537 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-05-08T08:50:43.537 INFO:teuthology.orchestra.run.smithi196.stdout: Package Arch Version Repository Size 2024-05-08T08:50:43.537 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-05-08T08:50:43.537 INFO:teuthology.orchestra.run.smithi196.stdout:Installing: 2024-05-08T08:50:43.537 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 142 k 2024-05-08T08:50:43.537 INFO:teuthology.orchestra.run.smithi196.stdout:Installing dependencies: 2024-05-08T08:50:43.538 INFO:teuthology.orchestra.run.smithi196.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-08T08:50:43.538 INFO:teuthology.orchestra.run.smithi196.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-08T08:50:43.538 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-08T08:50:43.538 INFO:teuthology.orchestra.run.smithi196.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-08T08:50:43.538 INFO:teuthology.orchestra.run.smithi196.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-08T08:50:43.538 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:50:43.538 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-05-08T08:50:43.538 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-05-08T08:50:43.538 INFO:teuthology.orchestra.run.smithi196.stdout:Install 6 Packages 2024-05-08T08:50:43.538 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:50:43.539 INFO:teuthology.orchestra.run.smithi196.stdout:Total download size: 7.0 M 2024-05-08T08:50:43.539 INFO:teuthology.orchestra.run.smithi196.stdout:Installed size: 33 M 2024-05-08T08:50:43.539 INFO:teuthology.orchestra.run.smithi196.stdout:Downloading Packages: 2024-05-08T08:50:43.577 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-05-08T08:50:43.608 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 6/6 2024-05-08T08:50:43.642 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-05-08T08:50:43.685 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 2/2 2024-05-08T08:50:43.702 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 2/2 2024-05-08T08:50:43.703 INFO:teuthology.orchestra.run.smithi059.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T08:50:43.703 INFO:teuthology.orchestra.run.smithi059.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-08T08:50:43.703 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:50:43.971 INFO:teuthology.orchestra.run.smithi196.stdout:(1/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 1.1 MB/s | 248 kB 00:00 2024-05-08T08:50:44.360 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 6/6 2024-05-08T08:50:44.360 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 1/6 2024-05-08T08:50:44.360 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-05-08T08:50:44.360 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-05-08T08:50:44.360 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-05-08T08:50:44.360 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-05-08T08:50:44.423 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 1/2 2024-05-08T08:50:44.596 INFO:teuthology.orchestra.run.smithi196.stdout:(2/6): python3-babel-2.9.1-2.el9.noarch.rpm 7.0 MB/s | 6.0 MB 00:00 2024-05-08T08:50:44.616 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-05-08T08:50:44.617 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:50:44.617 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-05-08T08:50:44.617 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-08T08:50:44.617 INFO:teuthology.orchestra.run.smithi192.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-08T08:50:44.617 INFO:teuthology.orchestra.run.smithi192.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-08T08:50:44.617 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-08T08:50:44.617 INFO:teuthology.orchestra.run.smithi192.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-08T08:50:44.617 INFO:teuthology.orchestra.run.smithi192.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-08T08:50:44.617 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:50:44.617 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-05-08T08:50:44.621 INFO:teuthology.orchestra.run.smithi196.stdout:(3/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 843 kB/s | 548 kB 00:00 2024-05-08T08:50:44.710 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-05-08T08:50:44.710 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:50:44.710 INFO:teuthology.orchestra.run.smithi059.stdout:Installed: 2024-05-08T08:50:44.710 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-08T08:50:44.710 INFO:teuthology.orchestra.run.smithi059.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-08T08:50:44.710 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:50:44.710 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2024-05-08T08:50:44.755 INFO:teuthology.orchestra.run.smithi196.stdout:(4/6): ceph-mgr-cephadm-19.0.0-2515.g96805dcf.e 140 kB/s | 142 kB 00:01 2024-05-08T08:50:44.894 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-fuse 2024-05-08T08:50:44.930 INFO:teuthology.orchestra.run.smithi196.stdout:(5/6): python3-natsort-7.1.1-5.el9.noarch.rpm 173 kB/s | 58 kB 00:00 2024-05-08T08:50:44.956 INFO:teuthology.orchestra.run.smithi196.stdout:(6/6): python3-typing-extensions-4.4.0-2.el9.no 153 kB/s | 51 kB 00:00 2024-05-08T08:50:44.957 INFO:teuthology.orchestra.run.smithi196.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:50:44.957 INFO:teuthology.orchestra.run.smithi196.stdout:Total 4.9 MB/s | 7.0 MB 00:01 2024-05-08T08:50:44.992 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum -y install librados-devel 2024-05-08T08:50:45.028 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction check 2024-05-08T08:50:45.039 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction check succeeded. 2024-05-08T08:50:45.039 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction test 2024-05-08T08:50:45.135 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction test succeeded. 2024-05-08T08:50:45.136 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction 2024-05-08T08:50:45.436 INFO:teuthology.orchestra.run.smithi196.stdout: Preparing : 1/1 2024-05-08T08:50:45.469 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:40 ago on Wed 08 May 2024 08:49:05 AM UTC. 2024-05-08T08:50:45.587 INFO:teuthology.orchestra.run.smithi059.stdout:Last metadata expiration check: 0:01:40 ago on Wed 08 May 2024 08:49:05 AM UTC. 2024-05-08T08:50:45.590 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-05-08T08:50:45.591 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-05-08T08:50:45.591 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2024-05-08T08:50:45.591 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-05-08T08:50:45.591 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-05-08T08:50:45.591 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-fuse x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 889 k 2024-05-08T08:50:45.592 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2024-05-08T08:50:45.592 INFO:teuthology.orchestra.run.smithi192.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-08T08:50:45.592 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:50:45.592 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-05-08T08:50:45.592 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-05-08T08:50:45.592 INFO:teuthology.orchestra.run.smithi192.stdout:Install 2 Packages 2024-05-08T08:50:45.592 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:50:45.593 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 970 k 2024-05-08T08:50:45.593 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 2.8 M 2024-05-08T08:50:45.593 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-05-08T08:50:45.614 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-05-08T08:50:45.674 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-05-08T08:50:45.715 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies resolved. 2024-05-08T08:50:45.715 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-05-08T08:50:45.715 INFO:teuthology.orchestra.run.smithi059.stdout: Package Arch Version Repo Size 2024-05-08T08:50:45.715 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-05-08T08:50:45.715 INFO:teuthology.orchestra.run.smithi059.stdout:Installing: 2024-05-08T08:50:45.715 INFO:teuthology.orchestra.run.smithi059.stdout: librados-devel x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 127 k 2024-05-08T08:50:45.715 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:50:45.715 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction Summary 2024-05-08T08:50:45.715 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-05-08T08:50:45.716 INFO:teuthology.orchestra.run.smithi059.stdout:Install 1 Package 2024-05-08T08:50:45.716 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:50:45.716 INFO:teuthology.orchestra.run.smithi059.stdout:Total download size: 127 k 2024-05-08T08:50:45.716 INFO:teuthology.orchestra.run.smithi059.stdout:Installed size: 456 k 2024-05-08T08:50:45.716 INFO:teuthology.orchestra.run.smithi059.stdout:Downloading Packages: 2024-05-08T08:50:46.046 INFO:teuthology.orchestra.run.smithi059.stdout:librados-devel-19.0.0-2515.g96805dcf.el9.x86_64 384 kB/s | 127 kB 00:00 2024-05-08T08:50:46.047 INFO:teuthology.orchestra.run.smithi059.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:50:46.047 INFO:teuthology.orchestra.run.smithi059.stdout:Total 381 kB/s | 127 kB 00:00 2024-05-08T08:50:46.047 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction check 2024-05-08T08:50:46.053 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction check succeeded. 2024-05-08T08:50:46.053 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction test 2024-05-08T08:50:46.071 INFO:teuthology.orchestra.run.smithi192.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 506 kB/s | 80 kB 00:00 2024-05-08T08:50:46.121 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction test succeeded. 2024-05-08T08:50:46.122 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction 2024-05-08T08:50:46.198 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-05-08T08:50:46.290 INFO:teuthology.orchestra.run.smithi059.stdout: Preparing : 1/1 2024-05-08T08:50:46.293 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-05-08T08:50:46.356 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-05-08T08:50:46.386 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 6/6 2024-05-08T08:50:46.388 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : librados-devel-2:19.0.0-2515.g96805dcf.el9.x86_64 1/1 2024-05-08T08:50:46.630 INFO:teuthology.orchestra.run.smithi192.stdout:(2/2): ceph-fuse-19.0.0-2515.g96805dcf.el9.x86_ 1.2 MB/s | 889 kB 00:00 2024-05-08T08:50:46.631 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:50:46.631 INFO:teuthology.orchestra.run.smithi192.stdout:Total 934 kB/s | 970 kB 00:01 2024-05-08T08:50:46.643 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-05-08T08:50:46.665 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-05-08T08:50:46.665 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-05-08T08:50:46.745 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-05-08T08:50:46.745 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-05-08T08:50:46.807 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: librados-devel-2:19.0.0-2515.g96805dcf.el9.x86_64 1/1 2024-05-08T08:50:46.929 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-05-08T08:50:47.058 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librados-devel-2:19.0.0-2515.g96805dcf.el9.x86_64 1/1 2024-05-08T08:50:47.058 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:50:47.058 INFO:teuthology.orchestra.run.smithi059.stdout:Installed: 2024-05-08T08:50:47.058 INFO:teuthology.orchestra.run.smithi059.stdout: librados-devel-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:50:47.058 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:50:47.058 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2024-05-08T08:50:47.081 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-05-08T08:50:47.115 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 6/6 2024-05-08T08:50:47.115 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 1/6 2024-05-08T08:50:47.115 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-05-08T08:50:47.115 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-05-08T08:50:47.115 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-05-08T08:50:47.115 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-05-08T08:50:47.156 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 2/2 2024-05-08T08:50:47.311 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum -y install libcephfs2 2024-05-08T08:50:47.364 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-05-08T08:50:47.365 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:50:47.365 INFO:teuthology.orchestra.run.smithi196.stdout:Installed: 2024-05-08T08:50:47.365 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-08T08:50:47.365 INFO:teuthology.orchestra.run.smithi196.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-08T08:50:47.365 INFO:teuthology.orchestra.run.smithi196.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-08T08:50:47.365 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-08T08:50:47.365 INFO:teuthology.orchestra.run.smithi196.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-08T08:50:47.365 INFO:teuthology.orchestra.run.smithi196.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-08T08:50:47.365 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:50:47.365 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-05-08T08:50:47.638 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install ceph-fuse 2024-05-08T08:50:47.873 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 2/2 2024-05-08T08:50:47.874 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 1/2 2024-05-08T08:50:47.908 INFO:teuthology.orchestra.run.smithi059.stdout:Last metadata expiration check: 0:01:42 ago on Wed 08 May 2024 08:49:05 AM UTC. 2024-05-08T08:50:47.993 INFO:teuthology.orchestra.run.smithi059.stdout:Package libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 is already installed. 2024-05-08T08:50:48.037 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies resolved. 2024-05-08T08:50:48.038 INFO:teuthology.orchestra.run.smithi059.stdout:Nothing to do. 2024-05-08T08:50:48.039 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2024-05-08T08:50:48.104 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum -y install libcephfs-devel 2024-05-08T08:50:48.143 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-05-08T08:50:48.143 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:50:48.143 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-05-08T08:50:48.143 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-05-08T08:50:48.143 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:50:48.143 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-05-08T08:50:48.209 INFO:teuthology.orchestra.run.smithi196.stdout:Last metadata expiration check: 0:01:37 ago on Wed 08 May 2024 08:49:11 AM UTC. 2024-05-08T08:50:48.333 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-05-08T08:50:48.333 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-05-08T08:50:48.333 INFO:teuthology.orchestra.run.smithi196.stdout: Package Arch Version Repository Size 2024-05-08T08:50:48.333 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-05-08T08:50:48.333 INFO:teuthology.orchestra.run.smithi196.stdout:Installing: 2024-05-08T08:50:48.333 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-fuse x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 889 k 2024-05-08T08:50:48.334 INFO:teuthology.orchestra.run.smithi196.stdout:Installing dependencies: 2024-05-08T08:50:48.334 INFO:teuthology.orchestra.run.smithi196.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-08T08:50:48.334 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:50:48.334 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-05-08T08:50:48.334 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-05-08T08:50:48.334 INFO:teuthology.orchestra.run.smithi196.stdout:Install 2 Packages 2024-05-08T08:50:48.334 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:50:48.335 INFO:teuthology.orchestra.run.smithi196.stdout:Total download size: 970 k 2024-05-08T08:50:48.335 INFO:teuthology.orchestra.run.smithi196.stdout:Installed size: 2.8 M 2024-05-08T08:50:48.335 INFO:teuthology.orchestra.run.smithi196.stdout:Downloading Packages: 2024-05-08T08:50:48.438 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-volume 2024-05-08T08:50:48.699 INFO:teuthology.orchestra.run.smithi059.stdout:Last metadata expiration check: 0:01:43 ago on Wed 08 May 2024 08:49:05 AM UTC. 2024-05-08T08:50:48.774 INFO:teuthology.orchestra.run.smithi196.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 504 kB/s | 80 kB 00:00 2024-05-08T08:50:48.827 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies resolved. 2024-05-08T08:50:48.828 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-05-08T08:50:48.828 INFO:teuthology.orchestra.run.smithi059.stdout: Package Arch Version Repo Size 2024-05-08T08:50:48.828 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-05-08T08:50:48.829 INFO:teuthology.orchestra.run.smithi059.stdout:Installing: 2024-05-08T08:50:48.829 INFO:teuthology.orchestra.run.smithi059.stdout: libcephfs-devel x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 32 k 2024-05-08T08:50:48.829 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:50:48.829 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction Summary 2024-05-08T08:50:48.829 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-05-08T08:50:48.829 INFO:teuthology.orchestra.run.smithi059.stdout:Install 1 Package 2024-05-08T08:50:48.829 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:50:48.829 INFO:teuthology.orchestra.run.smithi059.stdout:Total download size: 32 k 2024-05-08T08:50:48.829 INFO:teuthology.orchestra.run.smithi059.stdout:Installed size: 142 k 2024-05-08T08:50:48.829 INFO:teuthology.orchestra.run.smithi059.stdout:Downloading Packages: 2024-05-08T08:50:49.027 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:44 ago on Wed 08 May 2024 08:49:05 AM UTC. 2024-05-08T08:50:49.149 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-05-08T08:50:49.150 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-05-08T08:50:49.150 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2024-05-08T08:50:49.150 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-05-08T08:50:49.150 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-05-08T08:50:49.150 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-volume noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 264 k 2024-05-08T08:50:49.150 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2024-05-08T08:50:49.150 INFO:teuthology.orchestra.run.smithi192.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-08T08:50:49.151 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:50:49.151 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-05-08T08:50:49.151 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-05-08T08:50:49.151 INFO:teuthology.orchestra.run.smithi192.stdout:Install 2 Packages 2024-05-08T08:50:49.151 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:50:49.151 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 341 k 2024-05-08T08:50:49.151 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 1.5 M 2024-05-08T08:50:49.151 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-05-08T08:50:49.178 INFO:teuthology.orchestra.run.smithi059.stdout:libcephfs-devel-19.0.0-2515.g96805dcf.el9.x86_6 91 kB/s | 32 kB 00:00 2024-05-08T08:50:49.179 INFO:teuthology.orchestra.run.smithi059.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:50:49.180 INFO:teuthology.orchestra.run.smithi059.stdout:Total 90 kB/s | 32 kB 00:00 2024-05-08T08:50:49.180 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction check 2024-05-08T08:50:49.183 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction check succeeded. 2024-05-08T08:50:49.184 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction test 2024-05-08T08:50:49.203 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction test succeeded. 2024-05-08T08:50:49.203 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction 2024-05-08T08:50:49.284 INFO:teuthology.orchestra.run.smithi059.stdout: Preparing : 1/1 2024-05-08T08:50:49.291 INFO:teuthology.orchestra.run.smithi196.stdout:(2/2): ceph-fuse-19.0.0-2515.g96805dcf.el9.x86_ 1.3 MB/s | 889 kB 00:00 2024-05-08T08:50:49.292 INFO:teuthology.orchestra.run.smithi196.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:50:49.292 INFO:teuthology.orchestra.run.smithi196.stdout:Total 1.0 MB/s | 970 kB 00:00 2024-05-08T08:50:49.305 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction check 2024-05-08T08:50:49.328 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction check succeeded. 2024-05-08T08:50:49.328 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction test 2024-05-08T08:50:49.371 INFO:teuthology.orchestra.run.smithi192.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 424 kB/s | 77 kB 00:00 2024-05-08T08:50:49.404 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libcephfs-devel-2:19.0.0-2515.g96805dcf.el9.x86_64 1/1 2024-05-08T08:50:49.406 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction test succeeded. 2024-05-08T08:50:49.406 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction 2024-05-08T08:50:49.597 INFO:teuthology.orchestra.run.smithi196.stdout: Preparing : 1/1 2024-05-08T08:50:49.726 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-05-08T08:50:49.758 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: libcephfs-devel-2:19.0.0-2515.g96805dcf.el9.x86_64 1/1 2024-05-08T08:50:49.790 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 2/2 2024-05-08T08:50:49.878 INFO:teuthology.orchestra.run.smithi192.stdout:(2/2): ceph-volume-19.0.0-2515.g96805dcf.el9.no 382 kB/s | 264 kB 00:00 2024-05-08T08:50:49.879 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:50:49.879 INFO:teuthology.orchestra.run.smithi192.stdout:Total 468 kB/s | 341 kB 00:00 2024-05-08T08:50:49.891 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-05-08T08:50:49.901 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-05-08T08:50:49.901 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-05-08T08:50:49.932 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-05-08T08:50:49.932 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-05-08T08:50:49.984 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libcephfs-devel-2:19.0.0-2515.g96805dcf.el9.x86_64 1/1 2024-05-08T08:50:49.984 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:50:49.984 INFO:teuthology.orchestra.run.smithi059.stdout:Installed: 2024-05-08T08:50:49.984 INFO:teuthology.orchestra.run.smithi059.stdout: libcephfs-devel-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:50:49.985 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:50:49.985 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2024-05-08T08:50:50.074 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-05-08T08:50:50.145 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum -y install librados2 2024-05-08T08:50:50.244 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-05-08T08:50:50.281 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 2/2 2024-05-08T08:50:50.296 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 2/2 2024-05-08T08:50:50.296 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T08:50:50.296 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-08T08:50:50.296 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:50:50.431 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 2/2 2024-05-08T08:50:50.431 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 1/2 2024-05-08T08:50:50.700 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-05-08T08:50:50.700 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:50:50.700 INFO:teuthology.orchestra.run.smithi196.stdout:Installed: 2024-05-08T08:50:50.700 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-05-08T08:50:50.700 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:50:50.701 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-05-08T08:50:50.742 INFO:teuthology.orchestra.run.smithi059.stdout:Last metadata expiration check: 0:01:45 ago on Wed 08 May 2024 08:49:05 AM UTC. 2024-05-08T08:50:50.827 INFO:teuthology.orchestra.run.smithi059.stdout:Package librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 is already installed. 2024-05-08T08:50:50.870 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies resolved. 2024-05-08T08:50:50.871 INFO:teuthology.orchestra.run.smithi059.stdout:Nothing to do. 2024-05-08T08:50:50.871 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2024-05-08T08:50:50.938 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum -y install librbd1 2024-05-08T08:50:50.959 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install ceph-volume 2024-05-08T08:50:50.977 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 1/2 2024-05-08T08:50:51.283 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-05-08T08:50:51.284 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:50:51.284 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-05-08T08:50:51.284 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-08T08:50:51.284 INFO:teuthology.orchestra.run.smithi192.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-08T08:50:51.284 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:50:51.284 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-05-08T08:50:51.532 INFO:teuthology.orchestra.run.smithi059.stdout:Last metadata expiration check: 0:01:46 ago on Wed 08 May 2024 08:49:05 AM UTC. 2024-05-08T08:50:51.540 INFO:teuthology.orchestra.run.smithi196.stdout:Last metadata expiration check: 0:01:40 ago on Wed 08 May 2024 08:49:11 AM UTC. 2024-05-08T08:50:51.592 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install librados-devel 2024-05-08T08:50:51.613 INFO:teuthology.orchestra.run.smithi059.stdout:Package librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 is already installed. 2024-05-08T08:50:51.658 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies resolved. 2024-05-08T08:50:51.659 INFO:teuthology.orchestra.run.smithi059.stdout:Nothing to do. 2024-05-08T08:50:51.659 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2024-05-08T08:50:51.669 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-05-08T08:50:51.670 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-05-08T08:50:51.670 INFO:teuthology.orchestra.run.smithi196.stdout: Package Arch Version Repository Size 2024-05-08T08:50:51.670 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-05-08T08:50:51.670 INFO:teuthology.orchestra.run.smithi196.stdout:Installing: 2024-05-08T08:50:51.670 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-volume noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 264 k 2024-05-08T08:50:51.670 INFO:teuthology.orchestra.run.smithi196.stdout:Installing dependencies: 2024-05-08T08:50:51.670 INFO:teuthology.orchestra.run.smithi196.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-08T08:50:51.670 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:50:51.671 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-05-08T08:50:51.671 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-05-08T08:50:51.671 INFO:teuthology.orchestra.run.smithi196.stdout:Install 2 Packages 2024-05-08T08:50:51.671 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:50:51.672 INFO:teuthology.orchestra.run.smithi196.stdout:Total download size: 341 k 2024-05-08T08:50:51.672 INFO:teuthology.orchestra.run.smithi196.stdout:Installed size: 1.5 M 2024-05-08T08:50:51.672 INFO:teuthology.orchestra.run.smithi196.stdout:Downloading Packages: 2024-05-08T08:50:51.725 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum -y install python3-rados 2024-05-08T08:50:51.854 INFO:teuthology.orchestra.run.smithi196.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 662 kB/s | 77 kB 00:00 2024-05-08T08:50:52.171 INFO:teuthology.orchestra.run.smithi196.stdout:(2/2): ceph-volume-19.0.0-2515.g96805dcf.el9.no 608 kB/s | 264 kB 00:00 2024-05-08T08:50:52.172 INFO:teuthology.orchestra.run.smithi196.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:50:52.172 INFO:teuthology.orchestra.run.smithi196.stdout:Total 680 kB/s | 341 kB 00:00 2024-05-08T08:50:52.176 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:47 ago on Wed 08 May 2024 08:49:05 AM UTC. 2024-05-08T08:50:52.184 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction check 2024-05-08T08:50:52.193 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction check succeeded. 2024-05-08T08:50:52.194 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction test 2024-05-08T08:50:52.231 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction test succeeded. 2024-05-08T08:50:52.231 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction 2024-05-08T08:50:52.300 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-05-08T08:50:52.300 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-05-08T08:50:52.300 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repo Size 2024-05-08T08:50:52.301 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-05-08T08:50:52.301 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-05-08T08:50:52.301 INFO:teuthology.orchestra.run.smithi192.stdout: librados-devel x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 127 k 2024-05-08T08:50:52.301 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:50:52.301 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-05-08T08:50:52.301 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-05-08T08:50:52.301 INFO:teuthology.orchestra.run.smithi192.stdout:Install 1 Package 2024-05-08T08:50:52.301 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:50:52.302 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 127 k 2024-05-08T08:50:52.302 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 456 k 2024-05-08T08:50:52.302 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-05-08T08:50:52.318 INFO:teuthology.orchestra.run.smithi059.stdout:Last metadata expiration check: 0:01:47 ago on Wed 08 May 2024 08:49:05 AM UTC. 2024-05-08T08:50:52.376 INFO:teuthology.orchestra.run.smithi196.stdout: Preparing : 1/1 2024-05-08T08:50:52.402 INFO:teuthology.orchestra.run.smithi059.stdout:Package python3-rados-2:19.0.0-2515.g96805dcf.el9.x86_64 is already installed. 2024-05-08T08:50:52.446 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies resolved. 2024-05-08T08:50:52.447 INFO:teuthology.orchestra.run.smithi059.stdout:Nothing to do. 2024-05-08T08:50:52.447 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2024-05-08T08:50:52.509 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum -y install python3-rgw 2024-05-08T08:50:52.553 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-05-08T08:50:52.598 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 2/2 2024-05-08T08:50:52.614 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 2/2 2024-05-08T08:50:52.614 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T08:50:52.614 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-08T08:50:52.614 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:50:52.652 INFO:teuthology.orchestra.run.smithi192.stdout:librados-devel-19.0.0-2515.g96805dcf.el9.x86_64 362 kB/s | 127 kB 00:00 2024-05-08T08:50:52.653 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:50:52.653 INFO:teuthology.orchestra.run.smithi192.stdout:Total 360 kB/s | 127 kB 00:00 2024-05-08T08:50:52.653 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-05-08T08:50:52.659 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-05-08T08:50:52.659 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-05-08T08:50:52.723 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-05-08T08:50:52.724 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-05-08T08:50:52.881 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-05-08T08:50:52.999 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librados-devel-2:19.0.0-2515.g96805dcf.el9.x86_64 1/1 2024-05-08T08:50:53.103 INFO:teuthology.orchestra.run.smithi059.stdout:Last metadata expiration check: 0:01:48 ago on Wed 08 May 2024 08:49:05 AM UTC. 2024-05-08T08:50:53.186 INFO:teuthology.orchestra.run.smithi059.stdout:Package python3-rgw-2:19.0.0-2515.g96805dcf.el9.x86_64 is already installed. 2024-05-08T08:50:53.231 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies resolved. 2024-05-08T08:50:53.232 INFO:teuthology.orchestra.run.smithi059.stdout:Nothing to do. 2024-05-08T08:50:53.232 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2024-05-08T08:50:53.295 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum -y install python3-cephfs 2024-05-08T08:50:53.310 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 1/2 2024-05-08T08:50:53.495 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librados-devel-2:19.0.0-2515.g96805dcf.el9.x86_64 1/1 2024-05-08T08:50:53.627 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-05-08T08:50:53.628 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:50:53.628 INFO:teuthology.orchestra.run.smithi196.stdout:Installed: 2024-05-08T08:50:53.628 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-08T08:50:53.628 INFO:teuthology.orchestra.run.smithi196.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-08T08:50:53.629 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:50:53.629 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-05-08T08:50:53.731 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librados-devel-2:19.0.0-2515.g96805dcf.el9.x86_64 1/1 2024-05-08T08:50:53.731 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:50:53.731 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-05-08T08:50:53.731 INFO:teuthology.orchestra.run.smithi192.stdout: librados-devel-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:50:53.731 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:50:53.731 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-05-08T08:50:53.903 INFO:teuthology.orchestra.run.smithi059.stdout:Last metadata expiration check: 0:01:48 ago on Wed 08 May 2024 08:49:05 AM UTC. 2024-05-08T08:50:53.926 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install librados-devel 2024-05-08T08:50:53.987 INFO:teuthology.orchestra.run.smithi059.stdout:Package python3-cephfs-2:19.0.0-2515.g96805dcf.el9.x86_64 is already installed. 2024-05-08T08:50:53.995 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install libcephfs2 2024-05-08T08:50:54.031 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies resolved. 2024-05-08T08:50:54.032 INFO:teuthology.orchestra.run.smithi059.stdout:Nothing to do. 2024-05-08T08:50:54.032 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2024-05-08T08:50:54.095 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum -y install python3-rbd 2024-05-08T08:50:54.510 INFO:teuthology.orchestra.run.smithi196.stdout:Last metadata expiration check: 0:01:43 ago on Wed 08 May 2024 08:49:11 AM UTC. 2024-05-08T08:50:54.577 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:49 ago on Wed 08 May 2024 08:49:05 AM UTC. 2024-05-08T08:50:54.635 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-05-08T08:50:54.636 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-05-08T08:50:54.636 INFO:teuthology.orchestra.run.smithi196.stdout: Package Arch Version Repo Size 2024-05-08T08:50:54.636 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-05-08T08:50:54.636 INFO:teuthology.orchestra.run.smithi196.stdout:Installing: 2024-05-08T08:50:54.636 INFO:teuthology.orchestra.run.smithi196.stdout: librados-devel x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 127 k 2024-05-08T08:50:54.637 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:50:54.637 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-05-08T08:50:54.637 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-05-08T08:50:54.637 INFO:teuthology.orchestra.run.smithi196.stdout:Install 1 Package 2024-05-08T08:50:54.637 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:50:54.637 INFO:teuthology.orchestra.run.smithi196.stdout:Total download size: 127 k 2024-05-08T08:50:54.637 INFO:teuthology.orchestra.run.smithi196.stdout:Installed size: 456 k 2024-05-08T08:50:54.637 INFO:teuthology.orchestra.run.smithi196.stdout:Downloading Packages: 2024-05-08T08:50:54.660 INFO:teuthology.orchestra.run.smithi192.stdout:Package libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 is already installed. 2024-05-08T08:50:54.694 INFO:teuthology.orchestra.run.smithi059.stdout:Last metadata expiration check: 0:01:49 ago on Wed 08 May 2024 08:49:05 AM UTC. 2024-05-08T08:50:54.702 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-05-08T08:50:54.703 INFO:teuthology.orchestra.run.smithi192.stdout:Nothing to do. 2024-05-08T08:50:54.703 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-05-08T08:50:54.783 INFO:teuthology.orchestra.run.smithi059.stdout:Package python3-rbd-2:19.0.0-2515.g96805dcf.el9.x86_64 is already installed. 2024-05-08T08:50:54.787 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install libcephfs-devel 2024-05-08T08:50:54.828 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies resolved. 2024-05-08T08:50:54.829 INFO:teuthology.orchestra.run.smithi059.stdout:Nothing to do. 2024-05-08T08:50:54.829 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2024-05-08T08:50:54.868 INFO:teuthology.orchestra.run.smithi196.stdout:librados-devel-19.0.0-2515.g96805dcf.el9.x86_64 548 kB/s | 127 kB 00:00 2024-05-08T08:50:54.869 INFO:teuthology.orchestra.run.smithi196.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:50:54.869 INFO:teuthology.orchestra.run.smithi196.stdout:Total 544 kB/s | 127 kB 00:00 2024-05-08T08:50:54.869 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction check 2024-05-08T08:50:54.874 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction check succeeded. 2024-05-08T08:50:54.875 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction test 2024-05-08T08:50:54.894 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum -y install rbd-fuse 2024-05-08T08:50:54.941 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction test succeeded. 2024-05-08T08:50:54.941 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction 2024-05-08T08:50:55.091 INFO:teuthology.orchestra.run.smithi196.stdout: Preparing : 1/1 2024-05-08T08:50:55.208 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : librados-devel-2:19.0.0-2515.g96805dcf.el9.x86_64 1/1 2024-05-08T08:50:55.360 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:50 ago on Wed 08 May 2024 08:49:05 AM UTC. 2024-05-08T08:50:55.482 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-05-08T08:50:55.483 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-05-08T08:50:55.483 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repo Size 2024-05-08T08:50:55.483 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-05-08T08:50:55.484 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-05-08T08:50:55.484 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs-devel x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 32 k 2024-05-08T08:50:55.484 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:50:55.484 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-05-08T08:50:55.484 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-05-08T08:50:55.484 INFO:teuthology.orchestra.run.smithi192.stdout:Install 1 Package 2024-05-08T08:50:55.484 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:50:55.484 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 32 k 2024-05-08T08:50:55.485 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 142 k 2024-05-08T08:50:55.485 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-05-08T08:50:55.502 INFO:teuthology.orchestra.run.smithi059.stdout:Last metadata expiration check: 0:01:50 ago on Wed 08 May 2024 08:49:05 AM UTC. 2024-05-08T08:50:55.633 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies resolved. 2024-05-08T08:50:55.634 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-05-08T08:50:55.634 INFO:teuthology.orchestra.run.smithi059.stdout: Package Architecture Version Repository Size 2024-05-08T08:50:55.634 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-05-08T08:50:55.635 INFO:teuthology.orchestra.run.smithi059.stdout:Installing: 2024-05-08T08:50:55.635 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-fuse x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 85 k 2024-05-08T08:50:55.635 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:50:55.635 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction Summary 2024-05-08T08:50:55.635 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-05-08T08:50:55.635 INFO:teuthology.orchestra.run.smithi059.stdout:Install 1 Package 2024-05-08T08:50:55.635 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:50:55.635 INFO:teuthology.orchestra.run.smithi059.stdout:Total download size: 85 k 2024-05-08T08:50:55.635 INFO:teuthology.orchestra.run.smithi059.stdout:Installed size: 231 k 2024-05-08T08:50:55.635 INFO:teuthology.orchestra.run.smithi059.stdout:Downloading Packages: 2024-05-08T08:50:55.645 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librados-devel-2:19.0.0-2515.g96805dcf.el9.x86_64 1/1 2024-05-08T08:50:55.730 INFO:teuthology.orchestra.run.smithi192.stdout:libcephfs-devel-19.0.0-2515.g96805dcf.el9.x86_6 129 kB/s | 32 kB 00:00 2024-05-08T08:50:55.731 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:50:55.731 INFO:teuthology.orchestra.run.smithi192.stdout:Total 128 kB/s | 32 kB 00:00 2024-05-08T08:50:55.731 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-05-08T08:50:55.734 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-05-08T08:50:55.735 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-05-08T08:50:55.750 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-05-08T08:50:55.751 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-05-08T08:50:55.827 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-05-08T08:50:55.884 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librados-devel-2:19.0.0-2515.g96805dcf.el9.x86_64 1/1 2024-05-08T08:50:55.884 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:50:55.884 INFO:teuthology.orchestra.run.smithi196.stdout:Installed: 2024-05-08T08:50:55.884 INFO:teuthology.orchestra.run.smithi196.stdout: librados-devel-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:50:55.885 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:50:55.885 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-05-08T08:50:55.943 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libcephfs-devel-2:19.0.0-2515.g96805dcf.el9.x86_64 1/1 2024-05-08T08:50:55.949 INFO:teuthology.orchestra.run.smithi059.stdout:rbd-fuse-19.0.0-2515.g96805dcf.el9.x86_64.rpm 272 kB/s | 85 kB 00:00 2024-05-08T08:50:55.949 INFO:teuthology.orchestra.run.smithi059.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:50:55.950 INFO:teuthology.orchestra.run.smithi059.stdout:Total 270 kB/s | 85 kB 00:00 2024-05-08T08:50:55.950 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction check 2024-05-08T08:50:55.958 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction check succeeded. 2024-05-08T08:50:55.958 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction test 2024-05-08T08:50:56.026 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction test succeeded. 2024-05-08T08:50:56.026 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction 2024-05-08T08:50:56.105 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install libcephfs2 2024-05-08T08:50:56.178 INFO:teuthology.orchestra.run.smithi059.stdout: Preparing : 1/1 2024-05-08T08:50:56.282 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : rbd-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 1/1 2024-05-08T08:50:56.321 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libcephfs-devel-2:19.0.0-2515.g96805dcf.el9.x86_64 1/1 2024-05-08T08:50:56.550 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libcephfs-devel-2:19.0.0-2515.g96805dcf.el9.x86_64 1/1 2024-05-08T08:50:56.550 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:50:56.550 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-05-08T08:50:56.550 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs-devel-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:50:56.550 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:50:56.550 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-05-08T08:50:56.658 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: rbd-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 1/1 2024-05-08T08:50:56.683 INFO:teuthology.orchestra.run.smithi196.stdout:Last metadata expiration check: 0:01:45 ago on Wed 08 May 2024 08:49:11 AM UTC. 2024-05-08T08:50:56.720 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install librados2 2024-05-08T08:50:56.766 INFO:teuthology.orchestra.run.smithi196.stdout:Package libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 is already installed. 2024-05-08T08:50:56.810 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-05-08T08:50:56.811 INFO:teuthology.orchestra.run.smithi196.stdout:Nothing to do. 2024-05-08T08:50:56.811 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-05-08T08:50:56.874 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install libcephfs-devel 2024-05-08T08:50:56.903 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : rbd-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 1/1 2024-05-08T08:50:56.903 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:50:56.903 INFO:teuthology.orchestra.run.smithi059.stdout:Installed: 2024-05-08T08:50:56.903 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:50:56.903 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:50:56.903 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2024-05-08T08:50:57.117 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum -y install rbd-mirror 2024-05-08T08:50:57.283 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:52 ago on Wed 08 May 2024 08:49:05 AM UTC. 2024-05-08T08:50:57.362 INFO:teuthology.orchestra.run.smithi192.stdout:Package librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 is already installed. 2024-05-08T08:50:57.403 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-05-08T08:50:57.404 INFO:teuthology.orchestra.run.smithi192.stdout:Nothing to do. 2024-05-08T08:50:57.404 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-05-08T08:50:57.444 INFO:teuthology.orchestra.run.smithi196.stdout:Last metadata expiration check: 0:01:46 ago on Wed 08 May 2024 08:49:11 AM UTC. 2024-05-08T08:50:57.467 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install librbd1 2024-05-08T08:50:57.566 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-05-08T08:50:57.567 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-05-08T08:50:57.567 INFO:teuthology.orchestra.run.smithi196.stdout: Package Arch Version Repo Size 2024-05-08T08:50:57.567 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-05-08T08:50:57.567 INFO:teuthology.orchestra.run.smithi196.stdout:Installing: 2024-05-08T08:50:57.567 INFO:teuthology.orchestra.run.smithi196.stdout: libcephfs-devel x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 32 k 2024-05-08T08:50:57.567 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:50:57.567 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-05-08T08:50:57.567 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-05-08T08:50:57.567 INFO:teuthology.orchestra.run.smithi196.stdout:Install 1 Package 2024-05-08T08:50:57.567 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:50:57.568 INFO:teuthology.orchestra.run.smithi196.stdout:Total download size: 32 k 2024-05-08T08:50:57.568 INFO:teuthology.orchestra.run.smithi196.stdout:Installed size: 142 k 2024-05-08T08:50:57.568 INFO:teuthology.orchestra.run.smithi196.stdout:Downloading Packages: 2024-05-08T08:50:57.717 INFO:teuthology.orchestra.run.smithi059.stdout:Last metadata expiration check: 0:01:52 ago on Wed 08 May 2024 08:49:05 AM UTC. 2024-05-08T08:50:57.740 INFO:teuthology.orchestra.run.smithi196.stdout:libcephfs-devel-19.0.0-2515.g96805dcf.el9.x86_6 184 kB/s | 32 kB 00:00 2024-05-08T08:50:57.740 INFO:teuthology.orchestra.run.smithi196.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:50:57.741 INFO:teuthology.orchestra.run.smithi196.stdout:Total 183 kB/s | 32 kB 00:00 2024-05-08T08:50:57.741 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction check 2024-05-08T08:50:57.744 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction check succeeded. 2024-05-08T08:50:57.744 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction test 2024-05-08T08:50:57.762 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction test succeeded. 2024-05-08T08:50:57.762 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction 2024-05-08T08:50:57.845 INFO:teuthology.orchestra.run.smithi196.stdout: Preparing : 1/1 2024-05-08T08:50:57.847 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies resolved. 2024-05-08T08:50:57.848 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-05-08T08:50:57.848 INFO:teuthology.orchestra.run.smithi059.stdout: Package Arch Version Repo Size 2024-05-08T08:50:57.848 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-05-08T08:50:57.848 INFO:teuthology.orchestra.run.smithi059.stdout:Installing: 2024-05-08T08:50:57.848 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-mirror x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 3.1 M 2024-05-08T08:50:57.848 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:50:57.848 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction Summary 2024-05-08T08:50:57.848 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-05-08T08:50:57.848 INFO:teuthology.orchestra.run.smithi059.stdout:Install 1 Package 2024-05-08T08:50:57.848 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:50:57.849 INFO:teuthology.orchestra.run.smithi059.stdout:Total download size: 3.1 M 2024-05-08T08:50:57.849 INFO:teuthology.orchestra.run.smithi059.stdout:Installed size: 13 M 2024-05-08T08:50:57.849 INFO:teuthology.orchestra.run.smithi059.stdout:Downloading Packages: 2024-05-08T08:50:57.954 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libcephfs-devel-2:19.0.0-2515.g96805dcf.el9.x86_64 1/1 2024-05-08T08:50:58.036 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:53 ago on Wed 08 May 2024 08:49:05 AM UTC. 2024-05-08T08:50:58.117 INFO:teuthology.orchestra.run.smithi192.stdout:Package librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 is already installed. 2024-05-08T08:50:58.160 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-05-08T08:50:58.161 INFO:teuthology.orchestra.run.smithi192.stdout:Nothing to do. 2024-05-08T08:50:58.161 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-05-08T08:50:58.226 INFO:teuthology.orchestra.run.smithi059.stdout:rbd-mirror-19.0.0-2515.g96805dcf.el9.x86_64.rpm 8.2 MB/s | 3.1 MB 00:00 2024-05-08T08:50:58.227 INFO:teuthology.orchestra.run.smithi059.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:50:58.227 INFO:teuthology.orchestra.run.smithi059.stdout:Total 8.2 MB/s | 3.1 MB 00:00 2024-05-08T08:50:58.227 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction check 2024-05-08T08:50:58.228 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install python3-rados 2024-05-08T08:50:58.237 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction check succeeded. 2024-05-08T08:50:58.237 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction test 2024-05-08T08:50:58.318 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: libcephfs-devel-2:19.0.0-2515.g96805dcf.el9.x86_64 1/1 2024-05-08T08:50:58.322 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction test succeeded. 2024-05-08T08:50:58.323 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction 2024-05-08T08:50:58.546 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libcephfs-devel-2:19.0.0-2515.g96805dcf.el9.x86_64 1/1 2024-05-08T08:50:58.546 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:50:58.546 INFO:teuthology.orchestra.run.smithi196.stdout:Installed: 2024-05-08T08:50:58.546 INFO:teuthology.orchestra.run.smithi196.stdout: libcephfs-devel-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:50:58.547 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:50:58.547 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-05-08T08:50:58.708 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install librados2 2024-05-08T08:50:58.779 INFO:teuthology.orchestra.run.smithi059.stdout: Preparing : 1/1 2024-05-08T08:50:58.798 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:53 ago on Wed 08 May 2024 08:49:05 AM UTC. 2024-05-08T08:50:58.836 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 1/1 2024-05-08T08:50:58.863 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 1/1 2024-05-08T08:50:58.863 INFO:teuthology.orchestra.run.smithi059.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T08:50:58.863 INFO:teuthology.orchestra.run.smithi059.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-08T08:50:58.863 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-08T08:50:58.863 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-08T08:50:58.864 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:50:58.880 INFO:teuthology.orchestra.run.smithi192.stdout:Package python3-rados-2:19.0.0-2515.g96805dcf.el9.x86_64 is already installed. 2024-05-08T08:50:58.921 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-05-08T08:50:58.922 INFO:teuthology.orchestra.run.smithi192.stdout:Nothing to do. 2024-05-08T08:50:58.922 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-05-08T08:50:58.988 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install python3-rgw 2024-05-08T08:50:59.280 INFO:teuthology.orchestra.run.smithi196.stdout:Last metadata expiration check: 0:01:48 ago on Wed 08 May 2024 08:49:11 AM UTC. 2024-05-08T08:50:59.361 INFO:teuthology.orchestra.run.smithi196.stdout:Package librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 is already installed. 2024-05-08T08:50:59.403 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-05-08T08:50:59.404 INFO:teuthology.orchestra.run.smithi196.stdout:Nothing to do. 2024-05-08T08:50:59.404 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-05-08T08:50:59.468 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install librbd1 2024-05-08T08:50:59.558 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:54 ago on Wed 08 May 2024 08:49:05 AM UTC. 2024-05-08T08:50:59.640 INFO:teuthology.orchestra.run.smithi192.stdout:Package python3-rgw-2:19.0.0-2515.g96805dcf.el9.x86_64 is already installed. 2024-05-08T08:50:59.682 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-05-08T08:50:59.683 INFO:teuthology.orchestra.run.smithi192.stdout:Nothing to do. 2024-05-08T08:50:59.683 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-05-08T08:50:59.750 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install python3-cephfs 2024-05-08T08:50:59.919 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 1/1 2024-05-08T08:50:59.919 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:50:59.919 INFO:teuthology.orchestra.run.smithi059.stdout:Installed: 2024-05-08T08:50:59.919 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:50:59.919 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:50:59.920 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2024-05-08T08:51:00.039 INFO:teuthology.orchestra.run.smithi196.stdout:Last metadata expiration check: 0:01:49 ago on Wed 08 May 2024 08:49:11 AM UTC. 2024-05-08T08:51:00.121 INFO:teuthology.orchestra.run.smithi196.stdout:Package librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 is already installed. 2024-05-08T08:51:00.138 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum -y install rbd-nbd 2024-05-08T08:51:00.162 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-05-08T08:51:00.163 INFO:teuthology.orchestra.run.smithi196.stdout:Nothing to do. 2024-05-08T08:51:00.163 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-05-08T08:51:00.223 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install python3-rados 2024-05-08T08:51:00.316 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:55 ago on Wed 08 May 2024 08:49:05 AM UTC. 2024-05-08T08:51:00.397 INFO:teuthology.orchestra.run.smithi192.stdout:Package python3-cephfs-2:19.0.0-2515.g96805dcf.el9.x86_64 is already installed. 2024-05-08T08:51:00.438 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-05-08T08:51:00.439 INFO:teuthology.orchestra.run.smithi192.stdout:Nothing to do. 2024-05-08T08:51:00.439 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-05-08T08:51:00.504 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install python3-rbd 2024-05-08T08:51:00.743 INFO:teuthology.orchestra.run.smithi059.stdout:Last metadata expiration check: 0:01:55 ago on Wed 08 May 2024 08:49:05 AM UTC. 2024-05-08T08:51:00.801 INFO:teuthology.orchestra.run.smithi196.stdout:Last metadata expiration check: 0:01:49 ago on Wed 08 May 2024 08:49:11 AM UTC. 2024-05-08T08:51:00.871 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies resolved. 2024-05-08T08:51:00.872 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-05-08T08:51:00.872 INFO:teuthology.orchestra.run.smithi059.stdout: Package Architecture Version Repository Size 2024-05-08T08:51:00.872 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-05-08T08:51:00.872 INFO:teuthology.orchestra.run.smithi059.stdout:Installing: 2024-05-08T08:51:00.872 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-nbd x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 171 k 2024-05-08T08:51:00.872 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:51:00.872 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction Summary 2024-05-08T08:51:00.873 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-05-08T08:51:00.873 INFO:teuthology.orchestra.run.smithi059.stdout:Install 1 Package 2024-05-08T08:51:00.873 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:51:00.873 INFO:teuthology.orchestra.run.smithi059.stdout:Total download size: 171 k 2024-05-08T08:51:00.873 INFO:teuthology.orchestra.run.smithi059.stdout:Installed size: 494 k 2024-05-08T08:51:00.873 INFO:teuthology.orchestra.run.smithi059.stdout:Downloading Packages: 2024-05-08T08:51:00.881 INFO:teuthology.orchestra.run.smithi196.stdout:Package python3-rados-2:19.0.0-2515.g96805dcf.el9.x86_64 is already installed. 2024-05-08T08:51:00.923 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-05-08T08:51:00.924 INFO:teuthology.orchestra.run.smithi196.stdout:Nothing to do. 2024-05-08T08:51:00.924 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-05-08T08:51:00.978 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install python3-rgw 2024-05-08T08:51:01.072 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:56 ago on Wed 08 May 2024 08:49:05 AM UTC. 2024-05-08T08:51:01.113 INFO:teuthology.orchestra.run.smithi059.stdout:rbd-nbd-19.0.0-2515.g96805dcf.el9.x86_64.rpm 715 kB/s | 171 kB 00:00 2024-05-08T08:51:01.114 INFO:teuthology.orchestra.run.smithi059.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:51:01.114 INFO:teuthology.orchestra.run.smithi059.stdout:Total 710 kB/s | 171 kB 00:00 2024-05-08T08:51:01.114 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction check 2024-05-08T08:51:01.122 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction check succeeded. 2024-05-08T08:51:01.122 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction test 2024-05-08T08:51:01.153 INFO:teuthology.orchestra.run.smithi192.stdout:Package python3-rbd-2:19.0.0-2515.g96805dcf.el9.x86_64 is already installed. 2024-05-08T08:51:01.190 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction test succeeded. 2024-05-08T08:51:01.191 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction 2024-05-08T08:51:01.195 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-05-08T08:51:01.196 INFO:teuthology.orchestra.run.smithi192.stdout:Nothing to do. 2024-05-08T08:51:01.196 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-05-08T08:51:01.261 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install rbd-fuse 2024-05-08T08:51:01.352 INFO:teuthology.orchestra.run.smithi059.stdout: Preparing : 1/1 2024-05-08T08:51:01.464 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : rbd-nbd-2:19.0.0-2515.g96805dcf.el9.x86_64 1/1 2024-05-08T08:51:01.561 INFO:teuthology.orchestra.run.smithi196.stdout:Last metadata expiration check: 0:01:50 ago on Wed 08 May 2024 08:49:11 AM UTC. 2024-05-08T08:51:01.642 INFO:teuthology.orchestra.run.smithi196.stdout:Package python3-rgw-2:19.0.0-2515.g96805dcf.el9.x86_64 is already installed. 2024-05-08T08:51:01.684 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-05-08T08:51:01.685 INFO:teuthology.orchestra.run.smithi196.stdout:Nothing to do. 2024-05-08T08:51:01.685 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-05-08T08:51:01.747 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install python3-cephfs 2024-05-08T08:51:01.845 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:56 ago on Wed 08 May 2024 08:49:05 AM UTC. 2024-05-08T08:51:01.891 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: rbd-nbd-2:19.0.0-2515.g96805dcf.el9.x86_64 1/1 2024-05-08T08:51:01.976 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-05-08T08:51:01.977 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-05-08T08:51:01.977 INFO:teuthology.orchestra.run.smithi192.stdout: Package Architecture Version Repository Size 2024-05-08T08:51:01.977 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-05-08T08:51:01.977 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-05-08T08:51:01.977 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-fuse x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 85 k 2024-05-08T08:51:01.977 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:51:01.977 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-05-08T08:51:01.977 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-05-08T08:51:01.978 INFO:teuthology.orchestra.run.smithi192.stdout:Install 1 Package 2024-05-08T08:51:01.978 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:51:01.978 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 85 k 2024-05-08T08:51:01.978 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 231 k 2024-05-08T08:51:01.978 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-05-08T08:51:02.160 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : rbd-nbd-2:19.0.0-2515.g96805dcf.el9.x86_64 1/1 2024-05-08T08:51:02.160 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:51:02.160 INFO:teuthology.orchestra.run.smithi059.stdout:Installed: 2024-05-08T08:51:02.160 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-nbd-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:51:02.160 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:51:02.161 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2024-05-08T08:51:02.259 INFO:teuthology.orchestra.run.smithi192.stdout:rbd-fuse-19.0.0-2515.g96805dcf.el9.x86_64.rpm 304 kB/s | 85 kB 00:00 2024-05-08T08:51:02.259 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:51:02.260 INFO:teuthology.orchestra.run.smithi192.stdout:Total 301 kB/s | 85 kB 00:00 2024-05-08T08:51:02.260 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-05-08T08:51:02.267 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-05-08T08:51:02.267 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-05-08T08:51:02.317 INFO:teuthology.orchestra.run.smithi196.stdout:Last metadata expiration check: 0:01:51 ago on Wed 08 May 2024 08:49:11 AM UTC. 2024-05-08T08:51:02.332 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-05-08T08:51:02.332 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-05-08T08:51:02.398 INFO:teuthology.orchestra.run.smithi196.stdout:Package python3-cephfs-2:19.0.0-2515.g96805dcf.el9.x86_64 is already installed. 2024-05-08T08:51:02.407 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum -y install rbd-nbd 2024-05-08T08:51:02.439 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-05-08T08:51:02.440 INFO:teuthology.orchestra.run.smithi196.stdout:Nothing to do. 2024-05-08T08:51:02.440 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-05-08T08:51:02.475 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-05-08T08:51:02.505 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install python3-rbd 2024-05-08T08:51:02.582 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : rbd-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 1/1 2024-05-08T08:51:03.005 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: rbd-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 1/1 2024-05-08T08:51:03.013 INFO:teuthology.orchestra.run.smithi059.stdout:Last metadata expiration check: 0:01:58 ago on Wed 08 May 2024 08:49:05 AM UTC. 2024-05-08T08:51:03.079 INFO:teuthology.orchestra.run.smithi196.stdout:Last metadata expiration check: 0:01:52 ago on Wed 08 May 2024 08:49:11 AM UTC. 2024-05-08T08:51:03.099 INFO:teuthology.orchestra.run.smithi059.stdout:Package rbd-nbd-2:19.0.0-2515.g96805dcf.el9.x86_64 is already installed. 2024-05-08T08:51:03.143 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies resolved. 2024-05-08T08:51:03.144 INFO:teuthology.orchestra.run.smithi059.stdout:Nothing to do. 2024-05-08T08:51:03.144 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2024-05-08T08:51:03.161 INFO:teuthology.orchestra.run.smithi196.stdout:Package python3-rbd-2:19.0.0-2515.g96805dcf.el9.x86_64 is already installed. 2024-05-08T08:51:03.204 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-05-08T08:51:03.205 INFO:teuthology.orchestra.run.smithi196.stdout:Nothing to do. 2024-05-08T08:51:03.205 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-05-08T08:51:03.210 DEBUG:teuthology.parallel:result is None 2024-05-08T08:51:03.271 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : rbd-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 1/1 2024-05-08T08:51:03.271 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:51:03.271 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-05-08T08:51:03.271 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:51:03.271 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:51:03.271 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-05-08T08:51:03.272 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install rbd-fuse 2024-05-08T08:51:03.526 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install rbd-mirror 2024-05-08T08:51:03.859 INFO:teuthology.orchestra.run.smithi196.stdout:Last metadata expiration check: 0:01:52 ago on Wed 08 May 2024 08:49:11 AM UTC. 2024-05-08T08:51:03.991 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-05-08T08:51:03.991 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-05-08T08:51:03.991 INFO:teuthology.orchestra.run.smithi196.stdout: Package Architecture Version Repository Size 2024-05-08T08:51:03.992 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-05-08T08:51:03.992 INFO:teuthology.orchestra.run.smithi196.stdout:Installing: 2024-05-08T08:51:03.992 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-fuse x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 85 k 2024-05-08T08:51:03.992 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:51:03.992 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-05-08T08:51:03.992 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-05-08T08:51:03.992 INFO:teuthology.orchestra.run.smithi196.stdout:Install 1 Package 2024-05-08T08:51:03.992 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:51:03.993 INFO:teuthology.orchestra.run.smithi196.stdout:Total download size: 85 k 2024-05-08T08:51:03.993 INFO:teuthology.orchestra.run.smithi196.stdout:Installed size: 231 k 2024-05-08T08:51:03.993 INFO:teuthology.orchestra.run.smithi196.stdout:Downloading Packages: 2024-05-08T08:51:04.107 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:59 ago on Wed 08 May 2024 08:49:05 AM UTC. 2024-05-08T08:51:04.235 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-05-08T08:51:04.236 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-05-08T08:51:04.236 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repo Size 2024-05-08T08:51:04.236 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-05-08T08:51:04.237 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-05-08T08:51:04.237 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-mirror x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 3.1 M 2024-05-08T08:51:04.237 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:51:04.237 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-05-08T08:51:04.237 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-05-08T08:51:04.237 INFO:teuthology.orchestra.run.smithi192.stdout:Install 1 Package 2024-05-08T08:51:04.237 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:51:04.237 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 3.1 M 2024-05-08T08:51:04.237 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 13 M 2024-05-08T08:51:04.237 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-05-08T08:51:04.318 INFO:teuthology.orchestra.run.smithi196.stdout:rbd-fuse-19.0.0-2515.g96805dcf.el9.x86_64.rpm 262 kB/s | 85 kB 00:00 2024-05-08T08:51:04.319 INFO:teuthology.orchestra.run.smithi196.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:51:04.319 INFO:teuthology.orchestra.run.smithi196.stdout:Total 260 kB/s | 85 kB 00:00 2024-05-08T08:51:04.320 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction check 2024-05-08T08:51:04.327 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction check succeeded. 2024-05-08T08:51:04.327 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction test 2024-05-08T08:51:04.393 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction test succeeded. 2024-05-08T08:51:04.393 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction 2024-05-08T08:51:04.536 INFO:teuthology.orchestra.run.smithi196.stdout: Preparing : 1/1 2024-05-08T08:51:04.651 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : rbd-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 1/1 2024-05-08T08:51:04.770 INFO:teuthology.orchestra.run.smithi192.stdout:rbd-mirror-19.0.0-2515.g96805dcf.el9.x86_64.rpm 5.8 MB/s | 3.1 MB 00:00 2024-05-08T08:51:04.772 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:51:04.772 INFO:teuthology.orchestra.run.smithi192.stdout:Total 5.8 MB/s | 3.1 MB 00:00 2024-05-08T08:51:04.772 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-05-08T08:51:04.780 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-05-08T08:51:04.780 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-05-08T08:51:04.860 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-05-08T08:51:04.861 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-05-08T08:51:05.032 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: rbd-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 1/1 2024-05-08T08:51:05.295 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : rbd-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 1/1 2024-05-08T08:51:05.295 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:51:05.295 INFO:teuthology.orchestra.run.smithi196.stdout:Installed: 2024-05-08T08:51:05.295 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:51:05.295 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:51:05.295 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-05-08T08:51:05.301 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-05-08T08:51:05.363 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 1/1 2024-05-08T08:51:05.389 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 1/1 2024-05-08T08:51:05.389 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T08:51:05.389 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-08T08:51:05.389 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-08T08:51:05.389 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-08T08:51:05.389 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:51:05.545 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install rbd-mirror 2024-05-08T08:51:06.121 INFO:teuthology.orchestra.run.smithi196.stdout:Last metadata expiration check: 0:01:55 ago on Wed 08 May 2024 08:49:11 AM UTC. 2024-05-08T08:51:06.246 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-05-08T08:51:06.247 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-05-08T08:51:06.247 INFO:teuthology.orchestra.run.smithi196.stdout: Package Arch Version Repo Size 2024-05-08T08:51:06.247 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-05-08T08:51:06.247 INFO:teuthology.orchestra.run.smithi196.stdout:Installing: 2024-05-08T08:51:06.247 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-mirror x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 3.1 M 2024-05-08T08:51:06.247 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:51:06.247 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-05-08T08:51:06.247 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-05-08T08:51:06.248 INFO:teuthology.orchestra.run.smithi196.stdout:Install 1 Package 2024-05-08T08:51:06.248 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:51:06.248 INFO:teuthology.orchestra.run.smithi196.stdout:Total download size: 3.1 M 2024-05-08T08:51:06.248 INFO:teuthology.orchestra.run.smithi196.stdout:Installed size: 13 M 2024-05-08T08:51:06.248 INFO:teuthology.orchestra.run.smithi196.stdout:Downloading Packages: 2024-05-08T08:51:06.456 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 1/1 2024-05-08T08:51:06.456 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:51:06.456 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-05-08T08:51:06.456 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:51:06.456 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:51:06.456 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-05-08T08:51:06.646 INFO:teuthology.orchestra.run.smithi196.stdout:rbd-mirror-19.0.0-2515.g96805dcf.el9.x86_64.rpm 7.8 MB/s | 3.1 MB 00:00 2024-05-08T08:51:06.647 INFO:teuthology.orchestra.run.smithi196.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:51:06.647 INFO:teuthology.orchestra.run.smithi196.stdout:Total 7.7 MB/s | 3.1 MB 00:00 2024-05-08T08:51:06.647 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction check 2024-05-08T08:51:06.656 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction check succeeded. 2024-05-08T08:51:06.656 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction test 2024-05-08T08:51:06.698 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install rbd-nbd 2024-05-08T08:51:06.736 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction test succeeded. 2024-05-08T08:51:06.736 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction 2024-05-08T08:51:07.179 INFO:teuthology.orchestra.run.smithi196.stdout: Preparing : 1/1 2024-05-08T08:51:07.255 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 1/1 2024-05-08T08:51:07.276 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:02:02 ago on Wed 08 May 2024 08:49:05 AM UTC. 2024-05-08T08:51:07.281 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 1/1 2024-05-08T08:51:07.281 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T08:51:07.281 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-08T08:51:07.281 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-08T08:51:07.281 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-08T08:51:07.281 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:51:07.404 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-05-08T08:51:07.405 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-05-08T08:51:07.405 INFO:teuthology.orchestra.run.smithi192.stdout: Package Architecture Version Repository Size 2024-05-08T08:51:07.405 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-05-08T08:51:07.405 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-05-08T08:51:07.405 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-nbd x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 171 k 2024-05-08T08:51:07.405 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:51:07.405 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-05-08T08:51:07.405 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-05-08T08:51:07.405 INFO:teuthology.orchestra.run.smithi192.stdout:Install 1 Package 2024-05-08T08:51:07.406 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:51:07.406 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 171 k 2024-05-08T08:51:07.406 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 494 k 2024-05-08T08:51:07.406 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-05-08T08:51:07.757 INFO:teuthology.orchestra.run.smithi192.stdout:rbd-nbd-19.0.0-2515.g96805dcf.el9.x86_64.rpm 487 kB/s | 171 kB 00:00 2024-05-08T08:51:07.758 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:51:07.758 INFO:teuthology.orchestra.run.smithi192.stdout:Total 485 kB/s | 171 kB 00:00 2024-05-08T08:51:07.758 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-05-08T08:51:07.766 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-05-08T08:51:07.766 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-05-08T08:51:07.828 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-05-08T08:51:07.829 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-05-08T08:51:07.983 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-05-08T08:51:08.072 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : rbd-nbd-2:19.0.0-2515.g96805dcf.el9.x86_64 1/1 2024-05-08T08:51:08.371 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 1/1 2024-05-08T08:51:08.371 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:51:08.371 INFO:teuthology.orchestra.run.smithi196.stdout:Installed: 2024-05-08T08:51:08.371 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:51:08.371 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:51:08.371 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-05-08T08:51:08.468 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: rbd-nbd-2:19.0.0-2515.g96805dcf.el9.x86_64 1/1 2024-05-08T08:51:08.583 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install rbd-nbd 2024-05-08T08:51:08.720 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : rbd-nbd-2:19.0.0-2515.g96805dcf.el9.x86_64 1/1 2024-05-08T08:51:08.721 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:51:08.721 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-05-08T08:51:08.721 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-nbd-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:51:08.721 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:51:08.721 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-05-08T08:51:08.974 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install rbd-nbd 2024-05-08T08:51:09.170 INFO:teuthology.orchestra.run.smithi196.stdout:Last metadata expiration check: 0:01:58 ago on Wed 08 May 2024 08:49:11 AM UTC. 2024-05-08T08:51:09.294 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-05-08T08:51:09.295 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-05-08T08:51:09.295 INFO:teuthology.orchestra.run.smithi196.stdout: Package Architecture Version Repository Size 2024-05-08T08:51:09.295 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-05-08T08:51:09.295 INFO:teuthology.orchestra.run.smithi196.stdout:Installing: 2024-05-08T08:51:09.295 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-nbd x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 171 k 2024-05-08T08:51:09.295 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:51:09.296 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-05-08T08:51:09.296 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-05-08T08:51:09.296 INFO:teuthology.orchestra.run.smithi196.stdout:Install 1 Package 2024-05-08T08:51:09.296 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:51:09.296 INFO:teuthology.orchestra.run.smithi196.stdout:Total download size: 171 k 2024-05-08T08:51:09.296 INFO:teuthology.orchestra.run.smithi196.stdout:Installed size: 494 k 2024-05-08T08:51:09.296 INFO:teuthology.orchestra.run.smithi196.stdout:Downloading Packages: 2024-05-08T08:51:09.517 INFO:teuthology.orchestra.run.smithi196.stdout:rbd-nbd-19.0.0-2515.g96805dcf.el9.x86_64.rpm 776 kB/s | 171 kB 00:00 2024-05-08T08:51:09.517 INFO:teuthology.orchestra.run.smithi196.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:51:09.518 INFO:teuthology.orchestra.run.smithi196.stdout:Total 772 kB/s | 171 kB 00:00 2024-05-08T08:51:09.518 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction check 2024-05-08T08:51:09.525 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction check succeeded. 2024-05-08T08:51:09.526 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction test 2024-05-08T08:51:09.553 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:02:04 ago on Wed 08 May 2024 08:49:05 AM UTC. 2024-05-08T08:51:09.590 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction test succeeded. 2024-05-08T08:51:09.591 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction 2024-05-08T08:51:09.633 INFO:teuthology.orchestra.run.smithi192.stdout:Package rbd-nbd-2:19.0.0-2515.g96805dcf.el9.x86_64 is already installed. 2024-05-08T08:51:09.676 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-05-08T08:51:09.677 INFO:teuthology.orchestra.run.smithi192.stdout:Nothing to do. 2024-05-08T08:51:09.678 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-05-08T08:51:09.741 INFO:teuthology.orchestra.run.smithi196.stdout: Preparing : 1/1 2024-05-08T08:51:09.773 DEBUG:teuthology.parallel:result is None 2024-05-08T08:51:09.848 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : rbd-nbd-2:19.0.0-2515.g96805dcf.el9.x86_64 1/1 2024-05-08T08:51:10.276 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: rbd-nbd-2:19.0.0-2515.g96805dcf.el9.x86_64 1/1 2024-05-08T08:51:10.510 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : rbd-nbd-2:19.0.0-2515.g96805dcf.el9.x86_64 1/1 2024-05-08T08:51:10.511 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:51:10.511 INFO:teuthology.orchestra.run.smithi196.stdout:Installed: 2024-05-08T08:51:10.511 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-nbd-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:51:10.511 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:51:10.511 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-05-08T08:51:10.767 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install rbd-nbd 2024-05-08T08:51:11.351 INFO:teuthology.orchestra.run.smithi196.stdout:Last metadata expiration check: 0:02:00 ago on Wed 08 May 2024 08:49:11 AM UTC. 2024-05-08T08:51:11.435 INFO:teuthology.orchestra.run.smithi196.stdout:Package rbd-nbd-2:19.0.0-2515.g96805dcf.el9.x86_64 is already installed. 2024-05-08T08:51:11.478 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-05-08T08:51:11.479 INFO:teuthology.orchestra.run.smithi196.stdout:Nothing to do. 2024-05-08T08:51:11.479 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-05-08T08:51:11.577 DEBUG:teuthology.parallel:result is None 2024-05-08T08:51:11.577 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-08T08:51:11.719 DEBUG:teuthology.orchestra.run.smithi059:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-08T08:51:11.746 INFO:teuthology.orchestra.run.smithi059.stdout:19.0.0-2515.g96805dcf.el9 2024-05-08T08:51:11.747 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2515.g96805dcf.el9 2024-05-08T08:51:11.747 INFO:teuthology.task.install:The correct ceph version 19.0.0-2515.g96805dcf is installed. 2024-05-08T08:51:11.749 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-08T08:51:11.890 DEBUG:teuthology.orchestra.run.smithi192:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-08T08:51:11.915 INFO:teuthology.orchestra.run.smithi192.stdout:19.0.0-2515.g96805dcf.el9 2024-05-08T08:51:11.915 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2515.g96805dcf.el9 2024-05-08T08:51:11.915 INFO:teuthology.task.install:The correct ceph version 19.0.0-2515.g96805dcf is installed. 2024-05-08T08:51:11.917 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-08T08:51:12.141 DEBUG:teuthology.orchestra.run.smithi196:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-08T08:51:12.162 INFO:teuthology.orchestra.run.smithi196.stdout:19.0.0-2515.g96805dcf.el9 2024-05-08T08:51:12.163 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2515.g96805dcf.el9 2024-05-08T08:51:12.163 INFO:teuthology.task.install:The correct ceph version 19.0.0-2515.g96805dcf is installed. 2024-05-08T08:51:12.165 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-08T08:51:12.165 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-05-08T08:51:12.165 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-08T08:51:12.196 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-08T08:51:12.196 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-08T08:51:12.226 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-05-08T08:51:12.226 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-08T08:51:12.251 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-08T08:51:12.251 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-05-08T08:51:12.251 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/usr/bin/daemon-helper 2024-05-08T08:51:12.281 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-08T08:51:12.350 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-08T08:51:12.350 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/usr/bin/daemon-helper 2024-05-08T08:51:12.378 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-08T08:51:12.446 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-05-08T08:51:12.446 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/usr/bin/daemon-helper 2024-05-08T08:51:12.471 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-08T08:51:12.537 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-08T08:51:12.538 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-05-08T08:51:12.538 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-08T08:51:12.568 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-08T08:51:12.636 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-08T08:51:12.637 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-08T08:51:12.665 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-08T08:51:12.733 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-05-08T08:51:12.733 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-08T08:51:12.758 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-08T08:51:12.823 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-08T08:51:12.823 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-05-08T08:51:12.823 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/usr/bin/stdin-killer 2024-05-08T08:51:12.854 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-08T08:51:12.922 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-08T08:51:12.923 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/usr/bin/stdin-killer 2024-05-08T08:51:12.948 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-08T08:51:13.013 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-05-08T08:51:13.013 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/usr/bin/stdin-killer 2024-05-08T08:51:13.038 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-08T08:51:13.103 INFO:teuthology.run_tasks:Running task ceph... 2024-05-08T08:51:13.192 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-05-08T08:51:13.192 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 777 /var/log/ceph 2024-05-08T08:51:13.195 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 777 /var/log/ceph 2024-05-08T08:51:13.197 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 777 /var/log/ceph 2024-05-08T08:51:13.223 INFO:tasks.ceph:Disabling ceph logrotate... 2024-05-08T08:51:13.223 DEBUG:teuthology.orchestra.run.smithi059:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-08T08:51:13.266 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-08T08:51:13.269 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-08T08:51:13.294 INFO:tasks.ceph:Creating extra log directories... 2024-05-08T08:51:13.294 DEBUG:teuthology.orchestra.run.smithi059:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-08T08:51:13.337 DEBUG:teuthology.orchestra.run.smithi192:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-08T08:51:13.339 DEBUG:teuthology.orchestra.run.smithi196:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-08T08:51:13.373 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-05-08T08:51:13.374 INFO:tasks.ceph:config {'conf': {'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zlib', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-05-08T08:51:13.374 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-05-07_21:16:03-rbd-squid-distro-default-smithi/7696006', 'branch': 'squid', 'description': 'rbd/device/{base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-zlib supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_nbd}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7696006', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'teuthology-2024-05-07_21:16:03-rbd-squid-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'machine': {'cpus': 1, 'disk': 40, 'ram': 8000}, 'volumes': {'count': 4, 'size': 30}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zlib', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED'], 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'extra_packages': ['rbd-nbd'], 'flavor': 'default', 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mon.c', 'osd.0', 'osd.1', 'osd.2'], ['mon.b', 'mgr.x', 'osd.3', 'osd.4', 'osd.5'], ['client.0']], 'seed': 5498, 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc', 'sleep_before_teardown': 0, 'subset': '27/128', 'suite': 'rbd', 'suite_branch': 'squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc', 'targets': {'smithi059.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBg0P9U0pxwjx5v8P3f3SvOuxAdTttS4TvebBYtwUGso7cvHGXe1+hP8GAc1+EfBtvHugZXj2yavJ6Kga6/1I0U=', 'smithi192.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPhSindb4GTLW2TymcdFVVhhzFZ4VL+z8n7VGax3t1HrAxAVrf4J7odZeY6swp7OU7YaYSq+rxneF5yxJ4lM2ws=', 'smithi196.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCxyjPTl3acb93N2AVX4US1bTo3wpLdIdsHd1POcNcRuWWfAh3Tz4pIg0lFHA7gZe4r5KoM2Ynw9ODfCF4KT9qE='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'thrashosds': {'timeout': 1200}}, {'rbd_fsx': {'clients': ['client.0'], 'holebdy': 512, 'nbd': True, 'ops': 6000, 'punch_holes': True, 'readbdy': 512, 'truncbdy': 512, 'writebdy': 512}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '1ae7ad82388e92a475afff437d49054826c019a1', 'timestamp': '2024-05-07_21:16:03', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-05-08T08:51:13.374 DEBUG:teuthology.orchestra.run.smithi059:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-08T08:51:13.407 DEBUG:teuthology.orchestra.run.smithi192:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-08T08:51:13.409 DEBUG:teuthology.orchestra.run.smithi196:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-08T08:51:13.424 DEBUG:teuthology.orchestra.run.smithi059:> sudo install -d -m0777 -- /var/run/ceph 2024-05-08T08:51:13.465 DEBUG:teuthology.orchestra.run.smithi192:> sudo install -d -m0777 -- /var/run/ceph 2024-05-08T08:51:13.468 DEBUG:teuthology.orchestra.run.smithi196:> sudo install -d -m0777 -- /var/run/ceph 2024-05-08T08:51:13.492 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-05-08T08:51:13.493 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/scratch_devs of=/dev/stdout 2024-05-08T08:51:13.550 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-08T08:51:13.550 DEBUG:teuthology.orchestra.run.smithi059:> stat /dev/vg_nvme/lv_1 2024-05-08T08:51:13.607 INFO:teuthology.orchestra.run.smithi059.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-08T08:51:13.607 INFO:teuthology.orchestra.run.smithi059.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T08:51:13.607 INFO:teuthology.orchestra.run.smithi059.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-05-08T08:51:13.607 INFO:teuthology.orchestra.run.smithi059.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T08:51:13.607 INFO:teuthology.orchestra.run.smithi059.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T08:51:13.607 INFO:teuthology.orchestra.run.smithi059.stdout:Access: 2024-05-08 08:50:59.121145282 +0000 2024-05-08T08:51:13.607 INFO:teuthology.orchestra.run.smithi059.stdout:Modify: 2024-05-08 08:48:23.868606417 +0000 2024-05-08T08:51:13.607 INFO:teuthology.orchestra.run.smithi059.stdout:Change: 2024-05-08 08:48:23.868606417 +0000 2024-05-08T08:51:13.607 INFO:teuthology.orchestra.run.smithi059.stdout: Birth: 2024-05-08 08:48:23.868606417 +0000 2024-05-08T08:51:13.608 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-08T08:51:13.673 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records in 2024-05-08T08:51:13.674 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records out 2024-05-08T08:51:13.674 INFO:teuthology.orchestra.run.smithi059.stderr:512 bytes copied, 0.000266718 s, 1.9 MB/s 2024-05-08T08:51:13.675 DEBUG:teuthology.orchestra.run.smithi059:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-08T08:51:13.733 DEBUG:teuthology.orchestra.run.smithi059:> stat /dev/vg_nvme/lv_2 2024-05-08T08:51:13.790 INFO:teuthology.orchestra.run.smithi059.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-08T08:51:13.791 INFO:teuthology.orchestra.run.smithi059.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T08:51:13.791 INFO:teuthology.orchestra.run.smithi059.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-05-08T08:51:13.791 INFO:teuthology.orchestra.run.smithi059.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T08:51:13.791 INFO:teuthology.orchestra.run.smithi059.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T08:51:13.791 INFO:teuthology.orchestra.run.smithi059.stdout:Access: 2024-05-08 08:50:59.121145282 +0000 2024-05-08T08:51:13.791 INFO:teuthology.orchestra.run.smithi059.stdout:Modify: 2024-05-08 08:48:24.207602108 +0000 2024-05-08T08:51:13.791 INFO:teuthology.orchestra.run.smithi059.stdout:Change: 2024-05-08 08:48:24.207602108 +0000 2024-05-08T08:51:13.791 INFO:teuthology.orchestra.run.smithi059.stdout: Birth: 2024-05-08 08:48:24.207602108 +0000 2024-05-08T08:51:13.791 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-08T08:51:13.856 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records in 2024-05-08T08:51:13.856 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records out 2024-05-08T08:51:13.856 INFO:teuthology.orchestra.run.smithi059.stderr:512 bytes copied, 0.000195916 s, 2.6 MB/s 2024-05-08T08:51:13.859 DEBUG:teuthology.orchestra.run.smithi059:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-08T08:51:13.917 DEBUG:teuthology.orchestra.run.smithi059:> stat /dev/vg_nvme/lv_3 2024-05-08T08:51:13.974 INFO:teuthology.orchestra.run.smithi059.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-08T08:51:13.974 INFO:teuthology.orchestra.run.smithi059.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T08:51:13.974 INFO:teuthology.orchestra.run.smithi059.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-05-08T08:51:13.974 INFO:teuthology.orchestra.run.smithi059.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T08:51:13.975 INFO:teuthology.orchestra.run.smithi059.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T08:51:13.975 INFO:teuthology.orchestra.run.smithi059.stdout:Access: 2024-05-08 08:50:59.122145269 +0000 2024-05-08T08:51:13.975 INFO:teuthology.orchestra.run.smithi059.stdout:Modify: 2024-05-08 08:48:24.561597609 +0000 2024-05-08T08:51:13.975 INFO:teuthology.orchestra.run.smithi059.stdout:Change: 2024-05-08 08:48:24.561597609 +0000 2024-05-08T08:51:13.975 INFO:teuthology.orchestra.run.smithi059.stdout: Birth: 2024-05-08 08:48:24.561597609 +0000 2024-05-08T08:51:13.975 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-08T08:51:14.039 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records in 2024-05-08T08:51:14.040 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records out 2024-05-08T08:51:14.040 INFO:teuthology.orchestra.run.smithi059.stderr:512 bytes copied, 0.000200766 s, 2.6 MB/s 2024-05-08T08:51:14.041 DEBUG:teuthology.orchestra.run.smithi059:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-08T08:51:14.099 DEBUG:teuthology.orchestra.run.smithi059:> stat /dev/vg_nvme/lv_4 2024-05-08T08:51:14.155 INFO:teuthology.orchestra.run.smithi059.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-08T08:51:14.156 INFO:teuthology.orchestra.run.smithi059.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T08:51:14.156 INFO:teuthology.orchestra.run.smithi059.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-05-08T08:51:14.156 INFO:teuthology.orchestra.run.smithi059.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T08:51:14.156 INFO:teuthology.orchestra.run.smithi059.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T08:51:14.156 INFO:teuthology.orchestra.run.smithi059.stdout:Access: 2024-05-08 08:50:59.122145269 +0000 2024-05-08T08:51:14.156 INFO:teuthology.orchestra.run.smithi059.stdout:Modify: 2024-05-08 08:48:24.905593236 +0000 2024-05-08T08:51:14.156 INFO:teuthology.orchestra.run.smithi059.stdout:Change: 2024-05-08 08:48:24.905593236 +0000 2024-05-08T08:51:14.156 INFO:teuthology.orchestra.run.smithi059.stdout: Birth: 2024-05-08 08:48:24.905593236 +0000 2024-05-08T08:51:14.156 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-08T08:51:14.221 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records in 2024-05-08T08:51:14.221 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records out 2024-05-08T08:51:14.221 INFO:teuthology.orchestra.run.smithi059.stderr:512 bytes copied, 0.000223759 s, 2.3 MB/s 2024-05-08T08:51:14.222 DEBUG:teuthology.orchestra.run.smithi059:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-08T08:51:14.279 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-05-08T08:51:14.280 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-08T08:51:14.280 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/scratch_devs of=/dev/stdout 2024-05-08T08:51:14.296 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-08T08:51:14.297 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_1 2024-05-08T08:51:14.353 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-08T08:51:14.353 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T08:51:14.353 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-05-08T08:51:14.353 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T08:51:14.353 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T08:51:14.353 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2024-05-08 08:51:05.643230984 +0000 2024-05-08T08:51:14.353 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2024-05-08 08:48:18.259946326 +0000 2024-05-08T08:51:14.353 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2024-05-08 08:48:18.259946326 +0000 2024-05-08T08:51:14.354 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: 2024-05-08 08:48:18.259946326 +0000 2024-05-08T08:51:14.354 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-08T08:51:14.418 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2024-05-08T08:51:14.418 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2024-05-08T08:51:14.418 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.000210668 s, 2.4 MB/s 2024-05-08T08:51:14.419 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-08T08:51:14.476 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_2 2024-05-08T08:51:14.532 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-08T08:51:14.532 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T08:51:14.532 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-05-08T08:51:14.532 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T08:51:14.532 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T08:51:14.532 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2024-05-08 08:51:05.644230964 +0000 2024-05-08T08:51:14.532 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2024-05-08 08:48:18.600938801 +0000 2024-05-08T08:51:14.532 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2024-05-08 08:48:18.600938801 +0000 2024-05-08T08:51:14.532 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: 2024-05-08 08:48:18.600938801 +0000 2024-05-08T08:51:14.532 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-08T08:51:14.595 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2024-05-08T08:51:14.596 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2024-05-08T08:51:14.596 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.00022589 s, 2.3 MB/s 2024-05-08T08:51:14.597 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-08T08:51:14.653 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_3 2024-05-08T08:51:14.708 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-08T08:51:14.709 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T08:51:14.709 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-05-08T08:51:14.709 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T08:51:14.709 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T08:51:14.709 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2024-05-08 08:51:05.644230964 +0000 2024-05-08T08:51:14.709 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2024-05-08 08:48:18.934931433 +0000 2024-05-08T08:51:14.709 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2024-05-08 08:48:18.934931433 +0000 2024-05-08T08:51:14.709 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: 2024-05-08 08:48:18.934931433 +0000 2024-05-08T08:51:14.709 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-08T08:51:14.771 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2024-05-08T08:51:14.772 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2024-05-08T08:51:14.772 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.000222153 s, 2.3 MB/s 2024-05-08T08:51:14.773 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-08T08:51:14.829 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_4 2024-05-08T08:51:14.885 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-08T08:51:14.885 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T08:51:14.885 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-05-08T08:51:14.886 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T08:51:14.886 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T08:51:14.886 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2024-05-08 08:51:05.644230964 +0000 2024-05-08T08:51:14.886 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2024-05-08 08:48:19.272923975 +0000 2024-05-08T08:51:14.886 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2024-05-08 08:48:19.272923975 +0000 2024-05-08T08:51:14.886 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: 2024-05-08 08:48:19.272923975 +0000 2024-05-08T08:51:14.886 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-08T08:51:14.948 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2024-05-08T08:51:14.949 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2024-05-08T08:51:14.949 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.00022868 s, 2.2 MB/s 2024-05-08T08:51:14.950 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-08T08:51:15.007 INFO:tasks.ceph:osd dev map: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-05-08T08:51:15.007 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi059.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}, Remote(name='ubuntu@smithi192.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'}} 2024-05-08T08:51:15.007 INFO:tasks.ceph:Generating config... 2024-05-08T08:51:15.008 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-05-08T08:51:15.008 INFO:tasks.ceph:[global] mon warn on pool no app = False 2024-05-08T08:51:15.008 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-05-08T08:51:15.008 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-05-08T08:51:15.008 INFO:tasks.ceph:[mgr] debug ms = 1 2024-05-08T08:51:15.008 INFO:tasks.ceph:[mon] debug mon = 20 2024-05-08T08:51:15.008 INFO:tasks.ceph:[mon] debug ms = 1 2024-05-08T08:51:15.009 INFO:tasks.ceph:[mon] debug paxos = 20 2024-05-08T08:51:15.009 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-05-08T08:51:15.009 INFO:tasks.ceph:[osd] bluestore compression algorithm = zlib 2024-05-08T08:51:15.009 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2024-05-08T08:51:15.009 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-05-08T08:51:15.009 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-05-08T08:51:15.009 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-05-08T08:51:15.009 INFO:tasks.ceph:[osd] debug ms = 1 2024-05-08T08:51:15.009 INFO:tasks.ceph:[osd] debug osd = 20 2024-05-08T08:51:15.009 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-05-08T08:51:15.009 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-05-08T08:51:15.009 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-05-08T08:51:15.009 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-05-08T08:51:15.009 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-05-08T08:51:15.010 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-05-08T08:51:15.010 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-05-08T08:51:15.010 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-05-08T08:51:15.010 INFO:tasks.ceph:Setting up mon.a... 2024-05-08T08:51:15.010 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-05-08T08:51:15.094 INFO:teuthology.orchestra.run.smithi059.stdout:creating /etc/ceph/ceph.keyring 2024-05-08T08:51:15.097 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-05-08T08:51:15.146 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-08T08:51:15.173 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.59'), ('mon.c', '[v2:172.21.15.59:3301,v1:172.21.15.59:6790]'), ('mon.b', '172.21.15.192')] 2024-05-08T08:51:15.173 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.59,[v2:172.21.15.59:3301,v1:172.21.15.59:6790],172.21.15.192', 'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zlib', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2024-05-08T08:51:15.174 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-05-08T08:51:15.174 DEBUG:teuthology.orchestra.run.smithi059:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-05-08T08:51:15.231 DEBUG:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.59 --addv c '[v2:172.21.15.59:3301,v1:172.21.15.59:6790]' --add b 172.21.15.192 --print /home/ubuntu/cephtest/ceph.monmap 2024-05-08T08:51:15.320 INFO:teuthology.orchestra.run.smithi059.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-05-08T08:51:15.320 INFO:teuthology.orchestra.run.smithi059.stdout:monmaptool: generated fsid 38dfae8c-5c5e-4b0f-a264-91f22c6c5554 2024-05-08T08:51:15.320 INFO:teuthology.orchestra.run.smithi059.stdout:setting min_mon_release = quincy 2024-05-08T08:51:15.320 INFO:teuthology.orchestra.run.smithi059.stdout:epoch 0 2024-05-08T08:51:15.321 INFO:teuthology.orchestra.run.smithi059.stdout:fsid 38dfae8c-5c5e-4b0f-a264-91f22c6c5554 2024-05-08T08:51:15.321 INFO:teuthology.orchestra.run.smithi059.stdout:last_changed 2024-05-08T08:51:15.324892+0000 2024-05-08T08:51:15.321 INFO:teuthology.orchestra.run.smithi059.stdout:created 2024-05-08T08:51:15.324892+0000 2024-05-08T08:51:15.321 INFO:teuthology.orchestra.run.smithi059.stdout:min_mon_release 17 (quincy) 2024-05-08T08:51:15.321 INFO:teuthology.orchestra.run.smithi059.stdout:election_strategy: 1 2024-05-08T08:51:15.321 INFO:teuthology.orchestra.run.smithi059.stdout:0: [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] mon.a 2024-05-08T08:51:15.321 INFO:teuthology.orchestra.run.smithi059.stdout:1: [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] mon.b 2024-05-08T08:51:15.321 INFO:teuthology.orchestra.run.smithi059.stdout:2: [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] mon.c 2024-05-08T08:51:15.321 INFO:teuthology.orchestra.run.smithi059.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-05-08T08:51:15.322 DEBUG:teuthology.orchestra.run.smithi059:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-05-08T08:51:15.338 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 38dfae8c-5c5e-4b0f-a264-91f22c6c5554... 2024-05-08T08:51:15.338 DEBUG:teuthology.orchestra.run.smithi059:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-08T08:51:15.381 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-08T08:51:15.383 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-08T08:51:15.424 INFO:teuthology.orchestra.run.smithi059.stdout:[global] 2024-05-08T08:51:15.424 INFO:teuthology.orchestra.run.smithi059.stdout: chdir = "" 2024-05-08T08:51:15.424 INFO:teuthology.orchestra.run.smithi059.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-08T08:51:15.424 INFO:teuthology.orchestra.run.smithi059.stdout: auth supported = cephx 2024-05-08T08:51:15.424 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:51:15.424 INFO:teuthology.orchestra.run.smithi059.stdout: filestore xattr use omap = true 2024-05-08T08:51:15.424 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:51:15.424 INFO:teuthology.orchestra.run.smithi059.stdout: mon clock drift allowed = 1.000 2024-05-08T08:51:15.424 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:51:15.424 INFO:teuthology.orchestra.run.smithi059.stdout: osd crush chooseleaf type = 0 2024-05-08T08:51:15.424 INFO:teuthology.orchestra.run.smithi059.stdout: auth debug = true 2024-05-08T08:51:15.425 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:51:15.425 INFO:teuthology.orchestra.run.smithi059.stdout: ms die on old message = true 2024-05-08T08:51:15.425 INFO:teuthology.orchestra.run.smithi059.stdout: ms die on bug = true 2024-05-08T08:51:15.425 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:51:15.425 INFO:teuthology.orchestra.run.smithi059.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-08T08:51:15.425 INFO:teuthology.orchestra.run.smithi059.stdout: mon pg warn max object skew = 0 2024-05-08T08:51:15.425 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:51:15.425 INFO:teuthology.orchestra.run.smithi059.stdout: # disable pg_autoscaler by default for new pools 2024-05-08T08:51:15.425 INFO:teuthology.orchestra.run.smithi059.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-08T08:51:15.425 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:51:15.425 INFO:teuthology.orchestra.run.smithi059.stdout: osd pool default size = 2 2024-05-08T08:51:15.425 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:51:15.425 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd allow primary affinity = true 2024-05-08T08:51:15.425 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd allow pg remap = true 2024-05-08T08:51:15.426 INFO:teuthology.orchestra.run.smithi059.stdout: mon warn on legacy crush tunables = false 2024-05-08T08:51:15.426 INFO:teuthology.orchestra.run.smithi059.stdout: mon warn on crush straw calc version zero = false 2024-05-08T08:51:15.426 INFO:teuthology.orchestra.run.smithi059.stdout: mon warn on no sortbitwise = false 2024-05-08T08:51:15.426 INFO:teuthology.orchestra.run.smithi059.stdout: mon warn on osd down out interval zero = false 2024-05-08T08:51:15.426 INFO:teuthology.orchestra.run.smithi059.stdout: mon warn on too few osds = false 2024-05-08T08:51:15.426 INFO:teuthology.orchestra.run.smithi059.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-08T08:51:15.426 INFO:teuthology.orchestra.run.smithi059.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-08T08:51:15.426 INFO:teuthology.orchestra.run.smithi059.stdout: mon_allow_pool_size_one = true 2024-05-08T08:51:15.426 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:51:15.426 INFO:teuthology.orchestra.run.smithi059.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-08T08:51:15.426 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:51:15.426 INFO:teuthology.orchestra.run.smithi059.stdout: osd default data pool replay window = 5 2024-05-08T08:51:15.426 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:51:15.427 INFO:teuthology.orchestra.run.smithi059.stdout: mon allow pool delete = true 2024-05-08T08:51:15.427 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:51:15.427 INFO:teuthology.orchestra.run.smithi059.stdout: mon cluster log file level = debug 2024-05-08T08:51:15.427 INFO:teuthology.orchestra.run.smithi059.stdout: debug asserts on shutdown = true 2024-05-08T08:51:15.427 INFO:teuthology.orchestra.run.smithi059.stdout: mon health detail to clog = false 2024-05-08T08:51:15.427 INFO:teuthology.orchestra.run.smithi059.stdout: mon host = "172.21.15.59,[v2:172.21.15.59:3301,v1:172.21.15.59:6790],172.21.15.192" 2024-05-08T08:51:15.427 INFO:teuthology.orchestra.run.smithi059.stdout: mon client directed command retry = 5 2024-05-08T08:51:15.427 INFO:teuthology.orchestra.run.smithi059.stdout: mon warn on pool no app = False 2024-05-08T08:51:15.427 INFO:teuthology.orchestra.run.smithi059.stdout: ms inject socket failures = 5000 2024-05-08T08:51:15.427 INFO:teuthology.orchestra.run.smithi059.stdout: fsid = 38dfae8c-5c5e-4b0f-a264-91f22c6c5554 2024-05-08T08:51:15.427 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:51:15.427 INFO:teuthology.orchestra.run.smithi059.stdout:[osd] 2024-05-08T08:51:15.427 INFO:teuthology.orchestra.run.smithi059.stdout: osd journal size = 100 2024-05-08T08:51:15.427 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:51:15.428 INFO:teuthology.orchestra.run.smithi059.stdout: osd scrub load threshold = 5.0 2024-05-08T08:51:15.428 INFO:teuthology.orchestra.run.smithi059.stdout: osd scrub max interval = 600 2024-05-08T08:51:15.428 INFO:teuthology.orchestra.run.smithi059.stdout: osd mclock profile = high_recovery_ops 2024-05-08T08:51:15.428 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:51:15.428 INFO:teuthology.orchestra.run.smithi059.stdout: osd recover clone overlap = true 2024-05-08T08:51:15.428 INFO:teuthology.orchestra.run.smithi059.stdout: osd recovery max chunk = 1048576 2024-05-08T08:51:15.428 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:51:15.428 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug shutdown = true 2024-05-08T08:51:15.428 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug op order = true 2024-05-08T08:51:15.428 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug verify stray on activate = true 2024-05-08T08:51:15.428 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug trim objects = true 2024-05-08T08:51:15.428 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:51:15.428 INFO:teuthology.orchestra.run.smithi059.stdout: osd open classes on start = true 2024-05-08T08:51:15.428 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug pg log writeout = true 2024-05-08T08:51:15.428 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:51:15.429 INFO:teuthology.orchestra.run.smithi059.stdout: osd deep scrub update digest min age = 30 2024-05-08T08:51:15.429 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:51:15.429 INFO:teuthology.orchestra.run.smithi059.stdout: osd map max advance = 10 2024-05-08T08:51:15.429 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:51:15.429 INFO:teuthology.orchestra.run.smithi059.stdout: journal zero on create = true 2024-05-08T08:51:15.429 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:51:15.429 INFO:teuthology.orchestra.run.smithi059.stdout: filestore ondisk finisher threads = 3 2024-05-08T08:51:15.429 INFO:teuthology.orchestra.run.smithi059.stdout: filestore apply finisher threads = 3 2024-05-08T08:51:15.429 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:51:15.429 INFO:teuthology.orchestra.run.smithi059.stdout: bdev debug aio = true 2024-05-08T08:51:15.429 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug misdirected ops = true 2024-05-08T08:51:15.429 INFO:teuthology.orchestra.run.smithi059.stdout: bluestore block size = 96636764160 2024-05-08T08:51:15.429 INFO:teuthology.orchestra.run.smithi059.stdout: bluestore compression algorithm = zlib 2024-05-08T08:51:15.429 INFO:teuthology.orchestra.run.smithi059.stdout: bluestore compression mode = aggressive 2024-05-08T08:51:15.429 INFO:teuthology.orchestra.run.smithi059.stdout: bluestore fsck on mount = True 2024-05-08T08:51:15.429 INFO:teuthology.orchestra.run.smithi059.stdout: debug bluefs = 1/20 2024-05-08T08:51:15.430 INFO:teuthology.orchestra.run.smithi059.stdout: debug bluestore = 1/20 2024-05-08T08:51:15.430 INFO:teuthology.orchestra.run.smithi059.stdout: debug ms = 1 2024-05-08T08:51:15.430 INFO:teuthology.orchestra.run.smithi059.stdout: debug osd = 20 2024-05-08T08:51:15.430 INFO:teuthology.orchestra.run.smithi059.stdout: debug rocksdb = 4/10 2024-05-08T08:51:15.430 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd backfillfull_ratio = 0.85 2024-05-08T08:51:15.430 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd full ratio = 0.9 2024-05-08T08:51:15.430 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd nearfull ratio = 0.8 2024-05-08T08:51:15.430 INFO:teuthology.orchestra.run.smithi059.stdout: osd blocked scrub grace period = 3600 2024-05-08T08:51:15.430 INFO:teuthology.orchestra.run.smithi059.stdout: osd failsafe full ratio = 0.95 2024-05-08T08:51:15.430 INFO:teuthology.orchestra.run.smithi059.stdout: osd max markdown count = 1000 2024-05-08T08:51:15.430 INFO:teuthology.orchestra.run.smithi059.stdout: osd objectstore = bluestore 2024-05-08T08:51:15.430 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:51:15.430 INFO:teuthology.orchestra.run.smithi059.stdout:[mgr] 2024-05-08T08:51:15.431 INFO:teuthology.orchestra.run.smithi059.stdout: debug ms = 1 2024-05-08T08:51:15.431 INFO:teuthology.orchestra.run.smithi059.stdout: debug mgr = 20 2024-05-08T08:51:15.431 INFO:teuthology.orchestra.run.smithi059.stdout: debug mon = 20 2024-05-08T08:51:15.431 INFO:teuthology.orchestra.run.smithi059.stdout: debug auth = 20 2024-05-08T08:51:15.431 INFO:teuthology.orchestra.run.smithi059.stdout: mon reweight min pgs per osd = 4 2024-05-08T08:51:15.431 INFO:teuthology.orchestra.run.smithi059.stdout: mon reweight min bytes per osd = 10 2024-05-08T08:51:15.431 INFO:teuthology.orchestra.run.smithi059.stdout: mgr/telemetry/nag = false 2024-05-08T08:51:15.431 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:51:15.431 INFO:teuthology.orchestra.run.smithi059.stdout:[mon] 2024-05-08T08:51:15.431 INFO:teuthology.orchestra.run.smithi059.stdout: debug ms = 1 2024-05-08T08:51:15.431 INFO:teuthology.orchestra.run.smithi059.stdout: debug mon = 20 2024-05-08T08:51:15.431 INFO:teuthology.orchestra.run.smithi059.stdout: debug paxos = 20 2024-05-08T08:51:15.431 INFO:teuthology.orchestra.run.smithi059.stdout: debug auth = 20 2024-05-08T08:51:15.431 INFO:teuthology.orchestra.run.smithi059.stdout: mon data avail warn = 5 2024-05-08T08:51:15.431 INFO:teuthology.orchestra.run.smithi059.stdout: mon mgr mkfs grace = 240 2024-05-08T08:51:15.432 INFO:teuthology.orchestra.run.smithi059.stdout: mon reweight min pgs per osd = 4 2024-05-08T08:51:15.432 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd reporter subtree level = osd 2024-05-08T08:51:15.432 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd prime pg temp = true 2024-05-08T08:51:15.432 INFO:teuthology.orchestra.run.smithi059.stdout: mon reweight min bytes per osd = 10 2024-05-08T08:51:15.432 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:51:15.432 INFO:teuthology.orchestra.run.smithi059.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-08T08:51:15.432 INFO:teuthology.orchestra.run.smithi059.stdout: auth mon ticket ttl = 660 # 11m 2024-05-08T08:51:15.432 INFO:teuthology.orchestra.run.smithi059.stdout: auth service ticket ttl = 240 # 4m 2024-05-08T08:51:15.432 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:51:15.432 INFO:teuthology.orchestra.run.smithi059.stdout: # don't complain about insecure global_id in the test suite 2024-05-08T08:51:15.432 INFO:teuthology.orchestra.run.smithi059.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-08T08:51:15.432 INFO:teuthology.orchestra.run.smithi059.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-08T08:51:15.432 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:51:15.432 INFO:teuthology.orchestra.run.smithi059.stdout: # 1m isn't quite enough 2024-05-08T08:51:15.432 INFO:teuthology.orchestra.run.smithi059.stdout: mon_down_mkfs_grace = 2m 2024-05-08T08:51:15.433 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:51:15.433 INFO:teuthology.orchestra.run.smithi059.stdout: mon_warn_on_filestore_osds = false 2024-05-08T08:51:15.433 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:51:15.433 INFO:teuthology.orchestra.run.smithi059.stdout:[client] 2024-05-08T08:51:15.433 INFO:teuthology.orchestra.run.smithi059.stdout: rgw cache enabled = true 2024-05-08T08:51:15.433 INFO:teuthology.orchestra.run.smithi059.stdout: rgw enable ops log = true 2024-05-08T08:51:15.433 INFO:teuthology.orchestra.run.smithi059.stdout: rgw enable usage log = true 2024-05-08T08:51:15.433 INFO:teuthology.orchestra.run.smithi059.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-08T08:51:15.433 INFO:teuthology.orchestra.run.smithi059.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-08T08:51:15.433 INFO:teuthology.orchestra.run.smithi059.stdout:[mon.a] 2024-05-08T08:51:15.433 INFO:teuthology.orchestra.run.smithi059.stdout:[mon.c] 2024-05-08T08:51:15.433 INFO:teuthology.orchestra.run.smithi059.stdout:[mon.b] 2024-05-08T08:51:15.434 INFO:teuthology.orchestra.run.smithi192.stdout:[global] 2024-05-08T08:51:15.434 INFO:teuthology.orchestra.run.smithi192.stdout: chdir = "" 2024-05-08T08:51:15.434 INFO:teuthology.orchestra.run.smithi192.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-08T08:51:15.434 INFO:teuthology.orchestra.run.smithi192.stdout: auth supported = cephx 2024-05-08T08:51:15.434 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:51:15.434 INFO:teuthology.orchestra.run.smithi192.stdout: filestore xattr use omap = true 2024-05-08T08:51:15.434 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:51:15.434 INFO:teuthology.orchestra.run.smithi192.stdout: mon clock drift allowed = 1.000 2024-05-08T08:51:15.434 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:51:15.434 INFO:teuthology.orchestra.run.smithi192.stdout: osd crush chooseleaf type = 0 2024-05-08T08:51:15.435 INFO:teuthology.orchestra.run.smithi192.stdout: auth debug = true 2024-05-08T08:51:15.435 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:51:15.435 INFO:teuthology.orchestra.run.smithi192.stdout: ms die on old message = true 2024-05-08T08:51:15.435 INFO:teuthology.orchestra.run.smithi192.stdout: ms die on bug = true 2024-05-08T08:51:15.435 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:51:15.435 INFO:teuthology.orchestra.run.smithi192.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-08T08:51:15.435 INFO:teuthology.orchestra.run.smithi192.stdout: mon pg warn max object skew = 0 2024-05-08T08:51:15.435 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:51:15.435 INFO:teuthology.orchestra.run.smithi192.stdout: # disable pg_autoscaler by default for new pools 2024-05-08T08:51:15.435 INFO:teuthology.orchestra.run.smithi192.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-08T08:51:15.435 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:51:15.435 INFO:teuthology.orchestra.run.smithi192.stdout: osd pool default size = 2 2024-05-08T08:51:15.435 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:51:15.435 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd allow primary affinity = true 2024-05-08T08:51:15.435 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd allow pg remap = true 2024-05-08T08:51:15.436 INFO:teuthology.orchestra.run.smithi192.stdout: mon warn on legacy crush tunables = false 2024-05-08T08:51:15.436 INFO:teuthology.orchestra.run.smithi192.stdout: mon warn on crush straw calc version zero = false 2024-05-08T08:51:15.436 INFO:teuthology.orchestra.run.smithi192.stdout: mon warn on no sortbitwise = false 2024-05-08T08:51:15.436 INFO:teuthology.orchestra.run.smithi192.stdout: mon warn on osd down out interval zero = false 2024-05-08T08:51:15.436 INFO:teuthology.orchestra.run.smithi192.stdout: mon warn on too few osds = false 2024-05-08T08:51:15.436 INFO:teuthology.orchestra.run.smithi192.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-08T08:51:15.436 INFO:teuthology.orchestra.run.smithi192.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-08T08:51:15.436 INFO:teuthology.orchestra.run.smithi192.stdout: mon_allow_pool_size_one = true 2024-05-08T08:51:15.436 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:51:15.436 INFO:teuthology.orchestra.run.smithi192.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-08T08:51:15.436 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:51:15.436 INFO:teuthology.orchestra.run.smithi192.stdout: osd default data pool replay window = 5 2024-05-08T08:51:15.436 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:51:15.436 INFO:teuthology.orchestra.run.smithi192.stdout: mon allow pool delete = true 2024-05-08T08:51:15.436 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:51:15.437 INFO:teuthology.orchestra.run.smithi192.stdout: mon cluster log file level = debug 2024-05-08T08:51:15.437 INFO:teuthology.orchestra.run.smithi192.stdout: debug asserts on shutdown = true 2024-05-08T08:51:15.437 INFO:teuthology.orchestra.run.smithi192.stdout: mon health detail to clog = false 2024-05-08T08:51:15.437 INFO:teuthology.orchestra.run.smithi192.stdout: mon host = "172.21.15.59,[v2:172.21.15.59:3301,v1:172.21.15.59:6790],172.21.15.192" 2024-05-08T08:51:15.437 INFO:teuthology.orchestra.run.smithi192.stdout: mon client directed command retry = 5 2024-05-08T08:51:15.437 INFO:teuthology.orchestra.run.smithi192.stdout: mon warn on pool no app = False 2024-05-08T08:51:15.437 INFO:teuthology.orchestra.run.smithi192.stdout: ms inject socket failures = 5000 2024-05-08T08:51:15.437 INFO:teuthology.orchestra.run.smithi192.stdout: fsid = 38dfae8c-5c5e-4b0f-a264-91f22c6c5554 2024-05-08T08:51:15.437 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:51:15.437 INFO:teuthology.orchestra.run.smithi192.stdout:[osd] 2024-05-08T08:51:15.437 INFO:teuthology.orchestra.run.smithi192.stdout: osd journal size = 100 2024-05-08T08:51:15.437 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:51:15.437 INFO:teuthology.orchestra.run.smithi192.stdout: osd scrub load threshold = 5.0 2024-05-08T08:51:15.437 INFO:teuthology.orchestra.run.smithi192.stdout: osd scrub max interval = 600 2024-05-08T08:51:15.437 INFO:teuthology.orchestra.run.smithi192.stdout: osd mclock profile = high_recovery_ops 2024-05-08T08:51:15.438 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:51:15.438 INFO:teuthology.orchestra.run.smithi192.stdout: osd recover clone overlap = true 2024-05-08T08:51:15.438 INFO:teuthology.orchestra.run.smithi192.stdout: osd recovery max chunk = 1048576 2024-05-08T08:51:15.438 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:51:15.438 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug shutdown = true 2024-05-08T08:51:15.438 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug op order = true 2024-05-08T08:51:15.438 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug verify stray on activate = true 2024-05-08T08:51:15.438 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug trim objects = true 2024-05-08T08:51:15.438 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:51:15.438 INFO:teuthology.orchestra.run.smithi192.stdout: osd open classes on start = true 2024-05-08T08:51:15.438 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug pg log writeout = true 2024-05-08T08:51:15.438 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:51:15.438 INFO:teuthology.orchestra.run.smithi192.stdout: osd deep scrub update digest min age = 30 2024-05-08T08:51:15.438 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:51:15.439 INFO:teuthology.orchestra.run.smithi192.stdout: osd map max advance = 10 2024-05-08T08:51:15.439 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:51:15.439 INFO:teuthology.orchestra.run.smithi192.stdout: journal zero on create = true 2024-05-08T08:51:15.439 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:51:15.439 INFO:teuthology.orchestra.run.smithi192.stdout: filestore ondisk finisher threads = 3 2024-05-08T08:51:15.439 INFO:teuthology.orchestra.run.smithi192.stdout: filestore apply finisher threads = 3 2024-05-08T08:51:15.439 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:51:15.439 INFO:teuthology.orchestra.run.smithi192.stdout: bdev debug aio = true 2024-05-08T08:51:15.439 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug misdirected ops = true 2024-05-08T08:51:15.439 INFO:teuthology.orchestra.run.smithi192.stdout: bluestore block size = 96636764160 2024-05-08T08:51:15.439 INFO:teuthology.orchestra.run.smithi192.stdout: bluestore compression algorithm = zlib 2024-05-08T08:51:15.439 INFO:teuthology.orchestra.run.smithi192.stdout: bluestore compression mode = aggressive 2024-05-08T08:51:15.439 INFO:teuthology.orchestra.run.smithi192.stdout: bluestore fsck on mount = True 2024-05-08T08:51:15.439 INFO:teuthology.orchestra.run.smithi192.stdout: debug bluefs = 1/20 2024-05-08T08:51:15.440 INFO:teuthology.orchestra.run.smithi192.stdout: debug bluestore = 1/20 2024-05-08T08:51:15.440 INFO:teuthology.orchestra.run.smithi192.stdout: debug ms = 1 2024-05-08T08:51:15.440 INFO:teuthology.orchestra.run.smithi192.stdout: debug osd = 20 2024-05-08T08:51:15.440 INFO:teuthology.orchestra.run.smithi192.stdout: debug rocksdb = 4/10 2024-05-08T08:51:15.440 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd backfillfull_ratio = 0.85 2024-05-08T08:51:15.440 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd full ratio = 0.9 2024-05-08T08:51:15.440 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd nearfull ratio = 0.8 2024-05-08T08:51:15.440 INFO:teuthology.orchestra.run.smithi192.stdout: osd blocked scrub grace period = 3600 2024-05-08T08:51:15.440 INFO:teuthology.orchestra.run.smithi192.stdout: osd failsafe full ratio = 0.95 2024-05-08T08:51:15.440 INFO:teuthology.orchestra.run.smithi192.stdout: osd max markdown count = 1000 2024-05-08T08:51:15.440 INFO:teuthology.orchestra.run.smithi192.stdout: osd objectstore = bluestore 2024-05-08T08:51:15.440 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:51:15.440 INFO:teuthology.orchestra.run.smithi192.stdout:[mgr] 2024-05-08T08:51:15.440 INFO:teuthology.orchestra.run.smithi192.stdout: debug ms = 1 2024-05-08T08:51:15.440 INFO:teuthology.orchestra.run.smithi192.stdout: debug mgr = 20 2024-05-08T08:51:15.440 INFO:teuthology.orchestra.run.smithi192.stdout: debug mon = 20 2024-05-08T08:51:15.441 INFO:teuthology.orchestra.run.smithi192.stdout: debug auth = 20 2024-05-08T08:51:15.441 INFO:teuthology.orchestra.run.smithi192.stdout: mon reweight min pgs per osd = 4 2024-05-08T08:51:15.441 INFO:teuthology.orchestra.run.smithi192.stdout: mon reweight min bytes per osd = 10 2024-05-08T08:51:15.441 INFO:teuthology.orchestra.run.smithi192.stdout: mgr/telemetry/nag = false 2024-05-08T08:51:15.441 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:51:15.441 INFO:teuthology.orchestra.run.smithi192.stdout:[mon] 2024-05-08T08:51:15.441 INFO:teuthology.orchestra.run.smithi192.stdout: debug ms = 1 2024-05-08T08:51:15.441 INFO:teuthology.orchestra.run.smithi192.stdout: debug mon = 20 2024-05-08T08:51:15.441 INFO:teuthology.orchestra.run.smithi192.stdout: debug paxos = 20 2024-05-08T08:51:15.441 INFO:teuthology.orchestra.run.smithi192.stdout: debug auth = 20 2024-05-08T08:51:15.441 INFO:teuthology.orchestra.run.smithi192.stdout: mon data avail warn = 5 2024-05-08T08:51:15.441 INFO:teuthology.orchestra.run.smithi192.stdout: mon mgr mkfs grace = 240 2024-05-08T08:51:15.441 INFO:teuthology.orchestra.run.smithi192.stdout: mon reweight min pgs per osd = 4 2024-05-08T08:51:15.441 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd reporter subtree level = osd 2024-05-08T08:51:15.442 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd prime pg temp = true 2024-05-08T08:51:15.442 INFO:teuthology.orchestra.run.smithi192.stdout: mon reweight min bytes per osd = 10 2024-05-08T08:51:15.442 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:51:15.442 INFO:teuthology.orchestra.run.smithi192.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-08T08:51:15.442 INFO:teuthology.orchestra.run.smithi192.stdout: auth mon ticket ttl = 660 # 11m 2024-05-08T08:51:15.442 INFO:teuthology.orchestra.run.smithi192.stdout: auth service ticket ttl = 240 # 4m 2024-05-08T08:51:15.442 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:51:15.442 INFO:teuthology.orchestra.run.smithi192.stdout: # don't complain about insecure global_id in the test suite 2024-05-08T08:51:15.442 INFO:teuthology.orchestra.run.smithi192.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-08T08:51:15.442 INFO:teuthology.orchestra.run.smithi192.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-08T08:51:15.442 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:51:15.442 INFO:teuthology.orchestra.run.smithi192.stdout: # 1m isn't quite enough 2024-05-08T08:51:15.442 INFO:teuthology.orchestra.run.smithi192.stdout: mon_down_mkfs_grace = 2m 2024-05-08T08:51:15.442 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:51:15.442 INFO:teuthology.orchestra.run.smithi192.stdout: mon_warn_on_filestore_osds = false 2024-05-08T08:51:15.443 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T08:51:15.443 INFO:teuthology.orchestra.run.smithi192.stdout:[client] 2024-05-08T08:51:15.443 INFO:teuthology.orchestra.run.smithi192.stdout: rgw cache enabled = true 2024-05-08T08:51:15.443 INFO:teuthology.orchestra.run.smithi192.stdout: rgw enable ops log = true 2024-05-08T08:51:15.443 INFO:teuthology.orchestra.run.smithi192.stdout: rgw enable usage log = true 2024-05-08T08:51:15.443 INFO:teuthology.orchestra.run.smithi192.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-08T08:51:15.443 INFO:teuthology.orchestra.run.smithi192.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-08T08:51:15.443 INFO:teuthology.orchestra.run.smithi192.stdout:[mon.a] 2024-05-08T08:51:15.443 INFO:teuthology.orchestra.run.smithi192.stdout:[mon.c] 2024-05-08T08:51:15.443 INFO:teuthology.orchestra.run.smithi192.stdout:[mon.b] 2024-05-08T08:51:15.444 INFO:teuthology.orchestra.run.smithi196.stdout:[global] 2024-05-08T08:51:15.444 INFO:teuthology.orchestra.run.smithi196.stdout: chdir = "" 2024-05-08T08:51:15.444 INFO:teuthology.orchestra.run.smithi196.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-08T08:51:15.444 INFO:teuthology.orchestra.run.smithi196.stdout: auth supported = cephx 2024-05-08T08:51:15.444 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:51:15.445 INFO:teuthology.orchestra.run.smithi196.stdout: filestore xattr use omap = true 2024-05-08T08:51:15.445 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:51:15.445 INFO:teuthology.orchestra.run.smithi196.stdout: mon clock drift allowed = 1.000 2024-05-08T08:51:15.445 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:51:15.445 INFO:teuthology.orchestra.run.smithi196.stdout: osd crush chooseleaf type = 0 2024-05-08T08:51:15.445 INFO:teuthology.orchestra.run.smithi196.stdout: auth debug = true 2024-05-08T08:51:15.445 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:51:15.445 INFO:teuthology.orchestra.run.smithi196.stdout: ms die on old message = true 2024-05-08T08:51:15.445 INFO:teuthology.orchestra.run.smithi196.stdout: ms die on bug = true 2024-05-08T08:51:15.445 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:51:15.445 INFO:teuthology.orchestra.run.smithi196.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-08T08:51:15.445 INFO:teuthology.orchestra.run.smithi196.stdout: mon pg warn max object skew = 0 2024-05-08T08:51:15.445 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:51:15.445 INFO:teuthology.orchestra.run.smithi196.stdout: # disable pg_autoscaler by default for new pools 2024-05-08T08:51:15.446 INFO:teuthology.orchestra.run.smithi196.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-08T08:51:15.446 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:51:15.446 INFO:teuthology.orchestra.run.smithi196.stdout: osd pool default size = 2 2024-05-08T08:51:15.446 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:51:15.446 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd allow primary affinity = true 2024-05-08T08:51:15.446 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd allow pg remap = true 2024-05-08T08:51:15.446 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on legacy crush tunables = false 2024-05-08T08:51:15.446 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on crush straw calc version zero = false 2024-05-08T08:51:15.446 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on no sortbitwise = false 2024-05-08T08:51:15.446 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on osd down out interval zero = false 2024-05-08T08:51:15.446 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on too few osds = false 2024-05-08T08:51:15.446 INFO:teuthology.orchestra.run.smithi196.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-08T08:51:15.446 INFO:teuthology.orchestra.run.smithi196.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-08T08:51:15.446 INFO:teuthology.orchestra.run.smithi196.stdout: mon_allow_pool_size_one = true 2024-05-08T08:51:15.447 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:51:15.447 INFO:teuthology.orchestra.run.smithi196.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-08T08:51:15.447 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:51:15.447 INFO:teuthology.orchestra.run.smithi196.stdout: osd default data pool replay window = 5 2024-05-08T08:51:15.447 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:51:15.447 INFO:teuthology.orchestra.run.smithi196.stdout: mon allow pool delete = true 2024-05-08T08:51:15.447 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:51:15.447 INFO:teuthology.orchestra.run.smithi196.stdout: mon cluster log file level = debug 2024-05-08T08:51:15.447 INFO:teuthology.orchestra.run.smithi196.stdout: debug asserts on shutdown = true 2024-05-08T08:51:15.447 INFO:teuthology.orchestra.run.smithi196.stdout: mon health detail to clog = false 2024-05-08T08:51:15.447 INFO:teuthology.orchestra.run.smithi196.stdout: mon host = "172.21.15.59,[v2:172.21.15.59:3301,v1:172.21.15.59:6790],172.21.15.192" 2024-05-08T08:51:15.447 INFO:teuthology.orchestra.run.smithi196.stdout: mon client directed command retry = 5 2024-05-08T08:51:15.447 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on pool no app = False 2024-05-08T08:51:15.447 INFO:teuthology.orchestra.run.smithi196.stdout: ms inject socket failures = 5000 2024-05-08T08:51:15.447 INFO:teuthology.orchestra.run.smithi196.stdout: fsid = 38dfae8c-5c5e-4b0f-a264-91f22c6c5554 2024-05-08T08:51:15.448 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:51:15.448 INFO:teuthology.orchestra.run.smithi196.stdout:[osd] 2024-05-08T08:51:15.448 INFO:teuthology.orchestra.run.smithi196.stdout: osd journal size = 100 2024-05-08T08:51:15.448 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:51:15.448 INFO:teuthology.orchestra.run.smithi196.stdout: osd scrub load threshold = 5.0 2024-05-08T08:51:15.448 INFO:teuthology.orchestra.run.smithi196.stdout: osd scrub max interval = 600 2024-05-08T08:51:15.448 INFO:teuthology.orchestra.run.smithi196.stdout: osd mclock profile = high_recovery_ops 2024-05-08T08:51:15.448 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:51:15.448 INFO:teuthology.orchestra.run.smithi196.stdout: osd recover clone overlap = true 2024-05-08T08:51:15.448 INFO:teuthology.orchestra.run.smithi196.stdout: osd recovery max chunk = 1048576 2024-05-08T08:51:15.448 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:51:15.448 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug shutdown = true 2024-05-08T08:51:15.448 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug op order = true 2024-05-08T08:51:15.449 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug verify stray on activate = true 2024-05-08T08:51:15.449 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug trim objects = true 2024-05-08T08:51:15.449 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:51:15.449 INFO:teuthology.orchestra.run.smithi196.stdout: osd open classes on start = true 2024-05-08T08:51:15.449 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug pg log writeout = true 2024-05-08T08:51:15.449 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:51:15.449 INFO:teuthology.orchestra.run.smithi196.stdout: osd deep scrub update digest min age = 30 2024-05-08T08:51:15.449 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:51:15.449 INFO:teuthology.orchestra.run.smithi196.stdout: osd map max advance = 10 2024-05-08T08:51:15.449 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:51:15.449 INFO:teuthology.orchestra.run.smithi196.stdout: journal zero on create = true 2024-05-08T08:51:15.449 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:51:15.449 INFO:teuthology.orchestra.run.smithi196.stdout: filestore ondisk finisher threads = 3 2024-05-08T08:51:15.450 INFO:teuthology.orchestra.run.smithi196.stdout: filestore apply finisher threads = 3 2024-05-08T08:51:15.450 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:51:15.450 INFO:teuthology.orchestra.run.smithi196.stdout: bdev debug aio = true 2024-05-08T08:51:15.450 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug misdirected ops = true 2024-05-08T08:51:15.450 INFO:teuthology.orchestra.run.smithi196.stdout: bluestore block size = 96636764160 2024-05-08T08:51:15.450 INFO:teuthology.orchestra.run.smithi196.stdout: bluestore compression algorithm = zlib 2024-05-08T08:51:15.450 INFO:teuthology.orchestra.run.smithi196.stdout: bluestore compression mode = aggressive 2024-05-08T08:51:15.450 INFO:teuthology.orchestra.run.smithi196.stdout: bluestore fsck on mount = True 2024-05-08T08:51:15.450 INFO:teuthology.orchestra.run.smithi196.stdout: debug bluefs = 1/20 2024-05-08T08:51:15.450 INFO:teuthology.orchestra.run.smithi196.stdout: debug bluestore = 1/20 2024-05-08T08:51:15.450 INFO:teuthology.orchestra.run.smithi196.stdout: debug ms = 1 2024-05-08T08:51:15.450 INFO:teuthology.orchestra.run.smithi196.stdout: debug osd = 20 2024-05-08T08:51:15.450 INFO:teuthology.orchestra.run.smithi196.stdout: debug rocksdb = 4/10 2024-05-08T08:51:15.450 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd backfillfull_ratio = 0.85 2024-05-08T08:51:15.450 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd full ratio = 0.9 2024-05-08T08:51:15.451 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd nearfull ratio = 0.8 2024-05-08T08:51:15.451 INFO:teuthology.orchestra.run.smithi196.stdout: osd blocked scrub grace period = 3600 2024-05-08T08:51:15.451 INFO:teuthology.orchestra.run.smithi196.stdout: osd failsafe full ratio = 0.95 2024-05-08T08:51:15.451 INFO:teuthology.orchestra.run.smithi196.stdout: osd max markdown count = 1000 2024-05-08T08:51:15.451 INFO:teuthology.orchestra.run.smithi196.stdout: osd objectstore = bluestore 2024-05-08T08:51:15.451 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:51:15.451 INFO:teuthology.orchestra.run.smithi196.stdout:[mgr] 2024-05-08T08:51:15.451 INFO:teuthology.orchestra.run.smithi196.stdout: debug ms = 1 2024-05-08T08:51:15.451 INFO:teuthology.orchestra.run.smithi196.stdout: debug mgr = 20 2024-05-08T08:51:15.451 INFO:teuthology.orchestra.run.smithi196.stdout: debug mon = 20 2024-05-08T08:51:15.451 INFO:teuthology.orchestra.run.smithi196.stdout: debug auth = 20 2024-05-08T08:51:15.451 INFO:teuthology.orchestra.run.smithi196.stdout: mon reweight min pgs per osd = 4 2024-05-08T08:51:15.451 INFO:teuthology.orchestra.run.smithi196.stdout: mon reweight min bytes per osd = 10 2024-05-08T08:51:15.451 INFO:teuthology.orchestra.run.smithi196.stdout: mgr/telemetry/nag = false 2024-05-08T08:51:15.452 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:51:15.452 INFO:teuthology.orchestra.run.smithi196.stdout:[mon] 2024-05-08T08:51:15.452 INFO:teuthology.orchestra.run.smithi196.stdout: debug ms = 1 2024-05-08T08:51:15.452 INFO:teuthology.orchestra.run.smithi196.stdout: debug mon = 20 2024-05-08T08:51:15.452 INFO:teuthology.orchestra.run.smithi196.stdout: debug paxos = 20 2024-05-08T08:51:15.452 INFO:teuthology.orchestra.run.smithi196.stdout: debug auth = 20 2024-05-08T08:51:15.452 INFO:teuthology.orchestra.run.smithi196.stdout: mon data avail warn = 5 2024-05-08T08:51:15.452 INFO:teuthology.orchestra.run.smithi196.stdout: mon mgr mkfs grace = 240 2024-05-08T08:51:15.452 INFO:teuthology.orchestra.run.smithi196.stdout: mon reweight min pgs per osd = 4 2024-05-08T08:51:15.452 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd reporter subtree level = osd 2024-05-08T08:51:15.452 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd prime pg temp = true 2024-05-08T08:51:15.452 INFO:teuthology.orchestra.run.smithi196.stdout: mon reweight min bytes per osd = 10 2024-05-08T08:51:15.452 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:51:15.452 INFO:teuthology.orchestra.run.smithi196.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-08T08:51:15.452 INFO:teuthology.orchestra.run.smithi196.stdout: auth mon ticket ttl = 660 # 11m 2024-05-08T08:51:15.453 INFO:teuthology.orchestra.run.smithi196.stdout: auth service ticket ttl = 240 # 4m 2024-05-08T08:51:15.453 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:51:15.453 INFO:teuthology.orchestra.run.smithi196.stdout: # don't complain about insecure global_id in the test suite 2024-05-08T08:51:15.453 INFO:teuthology.orchestra.run.smithi196.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-08T08:51:15.453 INFO:teuthology.orchestra.run.smithi196.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-08T08:51:15.453 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:51:15.453 INFO:teuthology.orchestra.run.smithi196.stdout: # 1m isn't quite enough 2024-05-08T08:51:15.453 INFO:teuthology.orchestra.run.smithi196.stdout: mon_down_mkfs_grace = 2m 2024-05-08T08:51:15.453 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:51:15.453 INFO:teuthology.orchestra.run.smithi196.stdout: mon_warn_on_filestore_osds = false 2024-05-08T08:51:15.453 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-08T08:51:15.453 INFO:teuthology.orchestra.run.smithi196.stdout:[client] 2024-05-08T08:51:15.453 INFO:teuthology.orchestra.run.smithi196.stdout: rgw cache enabled = true 2024-05-08T08:51:15.453 INFO:teuthology.orchestra.run.smithi196.stdout: rgw enable ops log = true 2024-05-08T08:51:15.453 INFO:teuthology.orchestra.run.smithi196.stdout: rgw enable usage log = true 2024-05-08T08:51:15.454 INFO:teuthology.orchestra.run.smithi196.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-08T08:51:15.454 INFO:teuthology.orchestra.run.smithi196.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-08T08:51:15.454 INFO:teuthology.orchestra.run.smithi196.stdout:[mon.a] 2024-05-08T08:51:15.454 INFO:teuthology.orchestra.run.smithi196.stdout:[mon.c] 2024-05-08T08:51:15.454 INFO:teuthology.orchestra.run.smithi196.stdout:[mon.b] 2024-05-08T08:51:15.455 INFO:tasks.ceph:Creating admin key on mon.a... 2024-05-08T08:51:15.455 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-05-08T08:51:15.551 INFO:tasks.ceph:Copying monmap to all nodes... 2024-05-08T08:51:15.551 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-05-08T08:51:15.551 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-05-08T08:51:15.567 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-05-08T08:51:15.567 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-05-08T08:51:15.623 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi059.front.sepia.ceph.com 2024-05-08T08:51:15.623 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-05-08T08:51:15.623 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-08T08:51:15.623 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-08T08:51:15.699 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-05-08T08:51:15.699 DEBUG:teuthology.orchestra.run.smithi059:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-08T08:51:15.755 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi192.front.sepia.ceph.com 2024-05-08T08:51:15.755 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-08T08:51:15.755 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-08T08:51:15.756 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-08T08:51:15.791 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-08T08:51:15.791 DEBUG:teuthology.orchestra.run.smithi192:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-08T08:51:15.847 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi196.front.sepia.ceph.com 2024-05-08T08:51:15.847 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-05-08T08:51:15.847 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-08T08:51:15.847 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-08T08:51:15.888 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-05-08T08:51:15.888 DEBUG:teuthology.orchestra.run.smithi196:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-08T08:51:15.946 INFO:tasks.ceph:Setting up mon nodes... 2024-05-08T08:51:15.946 INFO:tasks.ceph:Setting up mgr nodes... 2024-05-08T08:51:15.946 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-05-08T08:51:16.004 INFO:teuthology.orchestra.run.smithi192.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-05-08T08:51:16.007 INFO:tasks.ceph:Setting up mds nodes... 2024-05-08T08:51:16.007 INFO:tasks.ceph_client:Setting up client nodes... 2024-05-08T08:51:16.007 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-08T08:51:16.060 INFO:teuthology.orchestra.run.smithi196.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-05-08T08:51:16.073 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-05-08T08:51:16.074 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi059.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}, Remote(name='ubuntu@smithi192.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'}} 2024-05-08T08:51:16.074 DEBUG:teuthology.orchestra.run.smithi059:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-05-08T08:51:16.100 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-05-08T08:51:16.100 INFO:tasks.ceph:role: osd.0 2024-05-08T08:51:16.101 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi059.front.sepia.ceph.com 2024-05-08T08:51:16.101 DEBUG:teuthology.orchestra.run.smithi059:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-08T08:51:16.174 INFO:teuthology.orchestra.run.smithi059.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T08:51:16.174 INFO:teuthology.orchestra.run.smithi059.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T08:51:16.174 INFO:teuthology.orchestra.run.smithi059.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T08:51:16.174 INFO:teuthology.orchestra.run.smithi059.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-08T08:51:16.174 INFO:teuthology.orchestra.run.smithi059.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T08:51:16.174 INFO:teuthology.orchestra.run.smithi059.stdout: = sunit=0 swidth=0 blks 2024-05-08T08:51:16.175 INFO:teuthology.orchestra.run.smithi059.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T08:51:16.175 INFO:teuthology.orchestra.run.smithi059.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-08T08:51:16.175 INFO:teuthology.orchestra.run.smithi059.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T08:51:16.175 INFO:teuthology.orchestra.run.smithi059.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T08:51:16.666 INFO:teuthology.orchestra.run.smithi059.stdout:Discarding blocks...Done. 2024-05-08T08:51:16.668 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi059.front.sepia.ceph.com -o noatime 2024-05-08T08:51:16.668 DEBUG:teuthology.orchestra.run.smithi059:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-05-08T08:51:16.814 DEBUG:teuthology.orchestra.run.smithi059:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-05-08T08:51:16.848 DEBUG:teuthology.orchestra.run.smithi059:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-05-08T08:51:16.915 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-05-08T08:51:16.915 INFO:tasks.ceph:role: osd.1 2024-05-08T08:51:16.915 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi059.front.sepia.ceph.com 2024-05-08T08:51:16.915 DEBUG:teuthology.orchestra.run.smithi059:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-08T08:51:16.989 INFO:teuthology.orchestra.run.smithi059.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T08:51:16.989 INFO:teuthology.orchestra.run.smithi059.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T08:51:16.989 INFO:teuthology.orchestra.run.smithi059.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T08:51:16.989 INFO:teuthology.orchestra.run.smithi059.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-08T08:51:16.989 INFO:teuthology.orchestra.run.smithi059.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T08:51:16.990 INFO:teuthology.orchestra.run.smithi059.stdout: = sunit=0 swidth=0 blks 2024-05-08T08:51:16.990 INFO:teuthology.orchestra.run.smithi059.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T08:51:16.990 INFO:teuthology.orchestra.run.smithi059.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-08T08:51:16.990 INFO:teuthology.orchestra.run.smithi059.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T08:51:16.990 INFO:teuthology.orchestra.run.smithi059.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T08:51:17.435 INFO:teuthology.orchestra.run.smithi059.stdout:Discarding blocks...Done. 2024-05-08T08:51:17.437 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi059.front.sepia.ceph.com -o noatime 2024-05-08T08:51:17.437 DEBUG:teuthology.orchestra.run.smithi059:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-05-08T08:51:17.529 DEBUG:teuthology.orchestra.run.smithi059:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-05-08T08:51:17.559 DEBUG:teuthology.orchestra.run.smithi059:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-05-08T08:51:17.625 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-05-08T08:51:17.626 INFO:tasks.ceph:role: osd.2 2024-05-08T08:51:17.626 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi059.front.sepia.ceph.com 2024-05-08T08:51:17.626 DEBUG:teuthology.orchestra.run.smithi059:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-08T08:51:17.699 INFO:teuthology.orchestra.run.smithi059.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T08:51:17.699 INFO:teuthology.orchestra.run.smithi059.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T08:51:17.699 INFO:teuthology.orchestra.run.smithi059.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T08:51:17.699 INFO:teuthology.orchestra.run.smithi059.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-08T08:51:17.699 INFO:teuthology.orchestra.run.smithi059.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T08:51:17.699 INFO:teuthology.orchestra.run.smithi059.stdout: = sunit=0 swidth=0 blks 2024-05-08T08:51:17.699 INFO:teuthology.orchestra.run.smithi059.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T08:51:17.699 INFO:teuthology.orchestra.run.smithi059.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-08T08:51:17.700 INFO:teuthology.orchestra.run.smithi059.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T08:51:17.700 INFO:teuthology.orchestra.run.smithi059.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T08:51:18.201 INFO:teuthology.orchestra.run.smithi059.stdout:Discarding blocks...Done. 2024-05-08T08:51:18.203 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi059.front.sepia.ceph.com -o noatime 2024-05-08T08:51:18.203 DEBUG:teuthology.orchestra.run.smithi059:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-05-08T08:51:18.349 DEBUG:teuthology.orchestra.run.smithi059:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-05-08T08:51:18.381 DEBUG:teuthology.orchestra.run.smithi059:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T08:51:18.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T08:51:18.480+0000 7f9442efd640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-05-08T08:51:18.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T08:51:18.485+0000 7f9442efd640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-05-08T08:51:18.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T08:51:18.485+0000 7f9442efd640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-08T08:51:18.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T08:51:18.485+0000 7f9442efd640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-08T08:51:18.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T08:51:18.485+0000 7f9442efd640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-08T08:51:18.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T08:51:18.492+0000 7f9442efd640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-05-08T08:51:21.341 DEBUG:teuthology.orchestra.run.smithi059:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-05-08T08:51:21.372 DEBUG:teuthology.orchestra.run.smithi059:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T08:51:21.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T08:51:21.468+0000 7f179340e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-05-08T08:51:21.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T08:51:21.468+0000 7f179340e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-05-08T08:51:21.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T08:51:21.469+0000 7f179340e640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-08T08:51:21.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T08:51:21.469+0000 7f179340e640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-08T08:51:21.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T08:51:21.469+0000 7f179340e640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-08T08:51:21.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T08:51:21.469+0000 7f179340e640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-05-08T08:51:24.316 DEBUG:teuthology.orchestra.run.smithi059:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-05-08T08:51:24.347 DEBUG:teuthology.orchestra.run.smithi059:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T08:51:24.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T08:51:24.442+0000 7f8e1b4ae640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-05-08T08:51:24.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T08:51:24.443+0000 7f8e1b4ae640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-05-08T08:51:24.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T08:51:24.443+0000 7f8e1b4ae640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-08T08:51:24.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T08:51:24.443+0000 7f8e1b4ae640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-08T08:51:24.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T08:51:24.443+0000 7f8e1b4ae640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-08T08:51:24.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T08:51:24.443+0000 7f8e1b4ae640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-05-08T08:51:27.292 DEBUG:teuthology.orchestra.run.smithi059:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-05-08T08:51:27.322 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-05-08T08:51:27.352 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-05-08T08:51:27.352 INFO:tasks.ceph:role: osd.3 2024-05-08T08:51:27.352 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi192.front.sepia.ceph.com 2024-05-08T08:51:27.353 DEBUG:teuthology.orchestra.run.smithi192:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-08T08:51:27.428 INFO:teuthology.orchestra.run.smithi192.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T08:51:27.429 INFO:teuthology.orchestra.run.smithi192.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T08:51:27.429 INFO:teuthology.orchestra.run.smithi192.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T08:51:27.429 INFO:teuthology.orchestra.run.smithi192.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-08T08:51:27.429 INFO:teuthology.orchestra.run.smithi192.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T08:51:27.429 INFO:teuthology.orchestra.run.smithi192.stdout: = sunit=0 swidth=0 blks 2024-05-08T08:51:27.429 INFO:teuthology.orchestra.run.smithi192.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T08:51:27.429 INFO:teuthology.orchestra.run.smithi192.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-08T08:51:27.429 INFO:teuthology.orchestra.run.smithi192.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T08:51:27.429 INFO:teuthology.orchestra.run.smithi192.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T08:51:27.938 INFO:teuthology.orchestra.run.smithi192.stdout:Discarding blocks...Done. 2024-05-08T08:51:27.941 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi192.front.sepia.ceph.com -o noatime 2024-05-08T08:51:27.941 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-3 2024-05-08T08:51:28.045 DEBUG:teuthology.orchestra.run.smithi192:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-05-08T08:51:28.082 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-05-08T08:51:28.149 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-05-08T08:51:28.150 INFO:tasks.ceph:role: osd.4 2024-05-08T08:51:28.150 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi192.front.sepia.ceph.com 2024-05-08T08:51:28.150 DEBUG:teuthology.orchestra.run.smithi192:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-08T08:51:28.223 INFO:teuthology.orchestra.run.smithi192.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T08:51:28.224 INFO:teuthology.orchestra.run.smithi192.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T08:51:28.224 INFO:teuthology.orchestra.run.smithi192.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T08:51:28.224 INFO:teuthology.orchestra.run.smithi192.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-08T08:51:28.224 INFO:teuthology.orchestra.run.smithi192.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T08:51:28.224 INFO:teuthology.orchestra.run.smithi192.stdout: = sunit=0 swidth=0 blks 2024-05-08T08:51:28.224 INFO:teuthology.orchestra.run.smithi192.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T08:51:28.224 INFO:teuthology.orchestra.run.smithi192.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-08T08:51:28.225 INFO:teuthology.orchestra.run.smithi192.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T08:51:28.225 INFO:teuthology.orchestra.run.smithi192.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T08:51:28.685 INFO:teuthology.orchestra.run.smithi192.stdout:Discarding blocks...Done. 2024-05-08T08:51:28.688 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi192.front.sepia.ceph.com -o noatime 2024-05-08T08:51:28.688 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-4 2024-05-08T08:51:28.780 DEBUG:teuthology.orchestra.run.smithi192:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-05-08T08:51:28.809 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-05-08T08:51:28.877 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-05-08T08:51:28.878 INFO:tasks.ceph:role: osd.5 2024-05-08T08:51:28.878 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi192.front.sepia.ceph.com 2024-05-08T08:51:28.878 DEBUG:teuthology.orchestra.run.smithi192:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-08T08:51:28.949 INFO:teuthology.orchestra.run.smithi192.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T08:51:28.950 INFO:teuthology.orchestra.run.smithi192.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T08:51:28.950 INFO:teuthology.orchestra.run.smithi192.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T08:51:28.950 INFO:teuthology.orchestra.run.smithi192.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-08T08:51:28.950 INFO:teuthology.orchestra.run.smithi192.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T08:51:28.950 INFO:teuthology.orchestra.run.smithi192.stdout: = sunit=0 swidth=0 blks 2024-05-08T08:51:28.950 INFO:teuthology.orchestra.run.smithi192.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T08:51:28.950 INFO:teuthology.orchestra.run.smithi192.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-08T08:51:28.950 INFO:teuthology.orchestra.run.smithi192.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T08:51:28.950 INFO:teuthology.orchestra.run.smithi192.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T08:51:29.445 INFO:teuthology.orchestra.run.smithi192.stdout:Discarding blocks...Done. 2024-05-08T08:51:29.447 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi192.front.sepia.ceph.com -o noatime 2024-05-08T08:51:29.448 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-5 2024-05-08T08:51:29.580 DEBUG:teuthology.orchestra.run.smithi192:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-05-08T08:51:29.610 DEBUG:teuthology.orchestra.run.smithi192:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T08:51:29.715 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T08:51:29.718+0000 7fb26f661640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-05-08T08:51:29.718 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T08:51:29.722+0000 7fb26f661640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-05-08T08:51:29.718 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T08:51:29.722+0000 7fb26f661640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-08T08:51:29.718 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T08:51:29.722+0000 7fb26f661640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-08T08:51:29.718 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T08:51:29.722+0000 7fb26f661640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-08T08:51:29.722 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T08:51:29.725+0000 7fb26f661640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-05-08T08:51:32.790 DEBUG:teuthology.orchestra.run.smithi192:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-05-08T08:51:32.821 DEBUG:teuthology.orchestra.run.smithi192:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T08:51:32.913 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T08:51:32.916+0000 7fe39c81e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-05-08T08:51:32.914 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T08:51:32.917+0000 7fe39c81e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-05-08T08:51:32.914 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T08:51:32.917+0000 7fe39c81e640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-08T08:51:32.914 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T08:51:32.917+0000 7fe39c81e640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-08T08:51:32.914 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T08:51:32.917+0000 7fe39c81e640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-08T08:51:32.914 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T08:51:32.917+0000 7fe39c81e640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-05-08T08:51:35.761 DEBUG:teuthology.orchestra.run.smithi192:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-05-08T08:51:35.791 DEBUG:teuthology.orchestra.run.smithi192:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T08:51:35.882 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T08:51:35.884+0000 7f4542bb6640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-05-08T08:51:35.882 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T08:51:35.884+0000 7f4542bb6640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-05-08T08:51:35.882 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T08:51:35.884+0000 7f4542bb6640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-08T08:51:35.882 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T08:51:35.885+0000 7f4542bb6640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-08T08:51:35.882 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T08:51:35.885+0000 7f4542bb6640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-08T08:51:35.882 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T08:51:35.885+0000 7f4542bb6640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-05-08T08:51:38.747 DEBUG:teuthology.orchestra.run.smithi192:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-05-08T08:51:38.776 INFO:tasks.ceph:Reading keys from all nodes... 2024-05-08T08:51:38.777 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-05-08T08:51:38.777 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-05-08T08:51:38.808 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-05-08T08:51:38.809 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-05-08T08:51:38.879 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-05-08T08:51:38.879 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-05-08T08:51:38.949 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-08T08:51:38.949 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-05-08T08:51:38.977 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-08T08:51:38.977 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-05-08T08:51:39.044 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-08T08:51:39.044 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-05-08T08:51:39.110 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-08T08:51:39.110 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-05-08T08:51:39.176 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-05-08T08:51:39.176 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-05-08T08:51:39.196 INFO:tasks.ceph:Adding keys to all mons... 2024-05-08T08:51:39.196 DEBUG:teuthology.orchestra.run.smithi059:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-08T08:51:39.199 DEBUG:teuthology.orchestra.run.smithi192:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-08T08:51:39.225 INFO:teuthology.orchestra.run.smithi059.stdout:[osd.0] 2024-05-08T08:51:39.225 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQAGPTtmOkuvHBAAioO8N7SD+6UjCKJiOcvPZw== 2024-05-08T08:51:39.225 INFO:teuthology.orchestra.run.smithi059.stdout:[osd.1] 2024-05-08T08:51:39.225 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQAJPTtmDdP+GxAAeqw9B084vgshqBNiUqtaaA== 2024-05-08T08:51:39.225 INFO:teuthology.orchestra.run.smithi059.stdout:[osd.2] 2024-05-08T08:51:39.226 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQAMPTtmxRN1GhAAY1GgDq+FptTGUTYafFbQtA== 2024-05-08T08:51:39.226 INFO:teuthology.orchestra.run.smithi059.stdout:[mgr.x] 2024-05-08T08:51:39.226 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQAEPTtmlRmIABAA0A9wNjUnI6HZy4UA/MvRrQ== 2024-05-08T08:51:39.226 INFO:teuthology.orchestra.run.smithi059.stdout:[osd.3] 2024-05-08T08:51:39.226 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQARPTtmLNvhKhAAESn5bZTpcm//njvNzfiyvg== 2024-05-08T08:51:39.226 INFO:teuthology.orchestra.run.smithi059.stdout:[osd.4] 2024-05-08T08:51:39.226 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQAUPTtmdmK1NhAAt+iuTYqqK91tBlbUmtuMZQ== 2024-05-08T08:51:39.226 INFO:teuthology.orchestra.run.smithi059.stdout:[osd.5] 2024-05-08T08:51:39.226 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQAXPTtmwjvJNBAA6wjT3GXJ5AqyhVfDOeL/Kw== 2024-05-08T08:51:39.226 INFO:teuthology.orchestra.run.smithi059.stdout:[client.0] 2024-05-08T08:51:39.226 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQAEPTtmQMnbAxAACGcpYRFqo0Va8fwG+IB3OQ== 2024-05-08T08:51:39.240 INFO:teuthology.orchestra.run.smithi192.stdout:[osd.0] 2024-05-08T08:51:39.240 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQAGPTtmOkuvHBAAioO8N7SD+6UjCKJiOcvPZw== 2024-05-08T08:51:39.240 INFO:teuthology.orchestra.run.smithi192.stdout:[osd.1] 2024-05-08T08:51:39.240 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQAJPTtmDdP+GxAAeqw9B084vgshqBNiUqtaaA== 2024-05-08T08:51:39.241 INFO:teuthology.orchestra.run.smithi192.stdout:[osd.2] 2024-05-08T08:51:39.241 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQAMPTtmxRN1GhAAY1GgDq+FptTGUTYafFbQtA== 2024-05-08T08:51:39.241 INFO:teuthology.orchestra.run.smithi192.stdout:[mgr.x] 2024-05-08T08:51:39.241 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQAEPTtmlRmIABAA0A9wNjUnI6HZy4UA/MvRrQ== 2024-05-08T08:51:39.241 INFO:teuthology.orchestra.run.smithi192.stdout:[osd.3] 2024-05-08T08:51:39.241 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQARPTtmLNvhKhAAESn5bZTpcm//njvNzfiyvg== 2024-05-08T08:51:39.241 INFO:teuthology.orchestra.run.smithi192.stdout:[osd.4] 2024-05-08T08:51:39.241 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQAUPTtmdmK1NhAAt+iuTYqqK91tBlbUmtuMZQ== 2024-05-08T08:51:39.241 INFO:teuthology.orchestra.run.smithi192.stdout:[osd.5] 2024-05-08T08:51:39.241 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQAXPTtmwjvJNBAA6wjT3GXJ5AqyhVfDOeL/Kw== 2024-05-08T08:51:39.241 INFO:teuthology.orchestra.run.smithi192.stdout:[client.0] 2024-05-08T08:51:39.241 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQAEPTtmQMnbAxAACGcpYRFqo0Va8fwG+IB3OQ== 2024-05-08T08:51:39.242 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T08:51:39.273 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T08:51:39.333 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T08:51:39.336 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T08:51:39.388 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T08:51:39.390 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T08:51:39.442 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-08T08:51:39.444 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-08T08:51:39.496 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T08:51:39.498 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T08:51:39.549 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T08:51:39.552 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T08:51:39.602 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T08:51:39.605 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T08:51:39.655 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-08T08:51:39.658 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-08T08:51:39.711 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-05-08T08:51:39.712 DEBUG:teuthology.orchestra.run.smithi059:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-05-08T08:51:39.737 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-08T08:51:39.834 DEBUG:teuthology.orchestra.run.smithi059:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-05-08T08:51:39.860 DEBUG:teuthology.orchestra.run.smithi059:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-05-08T08:51:39.926 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-08T08:51:40.023 DEBUG:teuthology.orchestra.run.smithi059:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-05-08T08:51:40.051 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-05-08T08:51:40.076 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-08T08:51:40.172 DEBUG:teuthology.orchestra.run.smithi192:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-05-08T08:51:40.197 DEBUG:teuthology.orchestra.run.smithi059:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-08T08:51:40.199 DEBUG:teuthology.orchestra.run.smithi192:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-08T08:51:40.253 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-05-08T08:51:40.254 INFO:tasks.ceph.mon.a:Restarting daemon 2024-05-08T08:51:40.254 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-05-08T08:51:40.256 INFO:tasks.ceph.mon.a:Started 2024-05-08T08:51:40.256 INFO:tasks.ceph.mon.c:Restarting daemon 2024-05-08T08:51:40.257 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-05-08T08:51:40.258 INFO:tasks.ceph.mon.c:Started 2024-05-08T08:51:40.259 INFO:tasks.ceph.mon.b:Restarting daemon 2024-05-08T08:51:40.259 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-05-08T08:51:40.296 INFO:tasks.ceph.mon.b:Started 2024-05-08T08:51:40.296 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-05-08T08:51:40.296 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-05-08T08:51:40.296 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-05-08T08:51:40.299 INFO:tasks.ceph.mgr.x:Started 2024-05-08T08:51:40.299 INFO:tasks.ceph:Setting crush tunables to default 2024-05-08T08:51:40.299 DEBUG:teuthology.orchestra.run.smithi059:> sudo ceph --cluster ceph osd crush tunables default 2024-05-08T08:51:45.677 INFO:teuthology.orchestra.run.smithi059.stderr:adjusted tunables profile to default 2024-05-08T08:51:45.690 INFO:tasks.ceph:check_enable_crimson: False 2024-05-08T08:51:45.690 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-05-08T08:51:45.690 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-05-08T08:51:45.690 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-05-08T08:51:45.701 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-08T08:51:45.704+0000 7fcd81e42040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T08:51:45.717 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-05-08T08:51:45.717 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-05-08T08:51:45.784 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-05-08T08:51:45.784 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-05-08T08:51:45.849 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-08T08:51:45.849 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-05-08T08:51:45.860 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-08T08:51:45.864+0000 7fcd81e42040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T08:51:45.876 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-08T08:51:45.876 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-05-08T08:51:45.916 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-08T08:51:45.919+0000 7fcd81e42040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T08:51:45.941 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-08T08:51:45.941 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-05-08T08:51:46.006 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --cluster ceph osd new 300801a5-1919-4b23-8c6d-bc7dad61cc4d 0 2024-05-08T08:51:46.192 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-08T08:51:46.194+0000 7fcd81e42040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T08:51:46.233 INFO:teuthology.orchestra.run.smithi192.stdout:0 2024-05-08T08:51:46.239 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-08T08:51:46.242+0000 7fcd81e42040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T08:51:46.245 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --cluster ceph osd new 0fead0de-6a65-4eba-b400-eea5692702e0 1 2024-05-08T08:51:46.433 INFO:teuthology.orchestra.run.smithi192.stdout:1 2024-05-08T08:51:46.445 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --cluster ceph osd new 1a233de6-1b82-4a51-8e96-4cd4abdf16c8 2 2024-05-08T08:51:46.673 INFO:teuthology.orchestra.run.smithi192.stdout:2 2024-05-08T08:51:46.684 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --cluster ceph osd new cc4870b8-dcd9-4e43-978d-8de6c6396abf 3 2024-05-08T08:51:46.871 INFO:teuthology.orchestra.run.smithi192.stdout:3 2024-05-08T08:51:46.883 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --cluster ceph osd new d1d05620-42fb-4107-a2f9-e83542d076ee 4 2024-05-08T08:51:47.031 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-08T08:51:47.034+0000 7fcd81e42040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T08:51:47.069 INFO:teuthology.orchestra.run.smithi192.stdout:4 2024-05-08T08:51:47.080 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --cluster ceph osd new ab26136c-31c3-4090-9fa2-988dba4f731f 5 2024-05-08T08:51:47.124 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-08T08:51:47.127+0000 7fcd81e42040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T08:51:47.168 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-08T08:51:47.172+0000 7fcd81e42040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T08:51:47.266 INFO:teuthology.orchestra.run.smithi192.stdout:5 2024-05-08T08:51:47.278 INFO:tasks.ceph.osd.0:Restarting daemon 2024-05-08T08:51:47.278 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-05-08T08:51:47.281 INFO:tasks.ceph.osd.0:Started 2024-05-08T08:51:47.281 INFO:tasks.ceph.osd.1:Restarting daemon 2024-05-08T08:51:47.281 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-05-08T08:51:47.283 INFO:tasks.ceph.osd.1:Started 2024-05-08T08:51:47.283 INFO:tasks.ceph.osd.2:Restarting daemon 2024-05-08T08:51:47.284 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-05-08T08:51:47.286 INFO:tasks.ceph.osd.2:Started 2024-05-08T08:51:47.286 INFO:tasks.ceph.osd.3:Restarting daemon 2024-05-08T08:51:47.286 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-05-08T08:51:47.321 INFO:tasks.ceph.osd.3:Started 2024-05-08T08:51:47.321 INFO:tasks.ceph.osd.4:Restarting daemon 2024-05-08T08:51:47.321 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-05-08T08:51:47.323 INFO:tasks.ceph.osd.4:Started 2024-05-08T08:51:47.323 INFO:tasks.ceph.osd.5:Restarting daemon 2024-05-08T08:51:47.323 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-05-08T08:51:47.326 INFO:tasks.ceph.osd.5:Started 2024-05-08T08:51:47.326 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T08:51:47.346 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-08T08:51:47.349+0000 7fcd81e42040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T08:51:47.401 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-08T08:51:47.404+0000 7fcd81e42040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T08:51:47.455 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-08T08:51:47.458+0000 7fcd81e42040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T08:51:47.510 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:51:47.510 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":8,"fsid":"38dfae8c-5c5e-4b0f-a264-91f22c6c5554","created":"2024-05-08T08:51:45.571369+0000","modified":"2024-05-08T08:51:47.267430+0000","last_up_change":"0.000000","last_in_change":"2024-05-08T08:51:47.267430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"300801a5-1919-4b23-8c6d-bc7dad61cc4d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"0fead0de-6a65-4eba-b400-eea5692702e0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"1a233de6-1b82-4a51-8e96-4cd4abdf16c8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"cc4870b8-dcd9-4e43-978d-8de6c6396abf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"d1d05620-42fb-4107-a2f9-e83542d076ee","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"ab26136c-31c3-4090-9fa2-988dba4f731f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T08:51:47.521 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-05-08T08:51:47.521 INFO:tasks.ceph:Waiting for OSDs to come up 2024-05-08T08:51:47.565 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-08T08:51:47.568+0000 7fcd81e42040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T08:51:47.888 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:51:47.891+0000 7fd76b226640 -1 Falling back to public interface 2024-05-08T08:51:47.891 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:51:47.894+0000 7f636d043640 -1 Falling back to public interface 2024-05-08T08:51:47.893 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:51:47.896+0000 7fb62976a640 -1 Falling back to public interface 2024-05-08T08:51:47.913 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:51:47.916+0000 7fe4e8225640 -1 Falling back to public interface 2024-05-08T08:51:47.922 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:51:47.925+0000 7fba4660c640 -1 Falling back to public interface 2024-05-08T08:51:47.924 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:51:47.927+0000 7fed609fc640 -1 Falling back to public interface 2024-05-08T08:51:47.941 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-08T08:51:47.944+0000 7fcd81e42040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T08:51:48.127 DEBUG:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-08T08:51:48.195 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-08T08:51:48.198+0000 7fcd81e42040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T08:51:48.248 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-08T08:51:48.251+0000 7fcd81e42040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T08:51:48.296 INFO:teuthology.misc.health.smithi059.stdout: 2024-05-08T08:51:48.296 INFO:teuthology.misc.health.smithi059.stdout:{"epoch":8,"fsid":"38dfae8c-5c5e-4b0f-a264-91f22c6c5554","created":"2024-05-08T08:51:45.571369+0000","modified":"2024-05-08T08:51:47.267430+0000","last_up_change":"0.000000","last_in_change":"2024-05-08T08:51:47.267430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"300801a5-1919-4b23-8c6d-bc7dad61cc4d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"0fead0de-6a65-4eba-b400-eea5692702e0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"1a233de6-1b82-4a51-8e96-4cd4abdf16c8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"cc4870b8-dcd9-4e43-978d-8de6c6396abf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"d1d05620-42fb-4107-a2f9-e83542d076ee","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"ab26136c-31c3-4090-9fa2-988dba4f731f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T08:51:48.306 DEBUG:teuthology.misc:0 of 6 OSDs are up 2024-05-08T08:51:48.362 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-08T08:51:48.365+0000 7fcd81e42040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T08:51:48.647 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-08T08:51:48.650+0000 7fcd81e42040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T08:51:48.693 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-08T08:51:48.696+0000 7fcd81e42040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T08:51:48.848 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-08T08:51:48.852+0000 7fcd81e42040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T08:51:48.982 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-08T08:51:48.985+0000 7fcd81e42040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T08:51:49.033 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-08T08:51:49.036+0000 7fcd81e42040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T08:51:49.101 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-08T08:51:49.104+0000 7fcd81e42040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T08:51:49.146 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-08T08:51:49.149+0000 7fcd81e42040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T08:51:49.615 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-08T08:51:49.618+0000 7fcd81e42040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T08:51:49.728 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-08T08:51:49.731+0000 7fcd81e42040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T08:51:50.745 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-08T08:51:50.748+0000 7fcd22b26640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-05-08T08:51:50.745 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-08T08:51:50.748+0000 7fcd22b26640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-05-08T08:51:51.370 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:51:51.373+0000 7fd76b226640 -1 osd.0 0 log_to_monitors true 2024-05-08T08:51:51.384 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:51:51.387+0000 7fe4e8225640 -1 osd.3 0 log_to_monitors true 2024-05-08T08:51:51.401 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:51:51.404+0000 7f636d043640 -1 osd.2 0 log_to_monitors true 2024-05-08T08:51:51.421 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:51:51.424+0000 7fb62976a640 -1 osd.1 0 log_to_monitors true 2024-05-08T08:51:51.428 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:51:51.431+0000 7fed609fc640 -1 osd.5 0 log_to_monitors true 2024-05-08T08:51:51.433 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:51:51.437+0000 7fba4660c640 -1 osd.4 0 log_to_monitors true 2024-05-08T08:51:54.912 DEBUG:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-08T08:51:55.295 INFO:teuthology.misc.health.smithi059.stdout: 2024-05-08T08:51:55.295 INFO:teuthology.misc.health.smithi059.stdout:{"epoch":10,"fsid":"38dfae8c-5c5e-4b0f-a264-91f22c6c5554","created":"2024-05-08T08:51:45.571369+0000","modified":"2024-05-08T08:51:52.744071+0000","last_up_change":"0.000000","last_in_change":"2024-05-08T08:51:47.267430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"300801a5-1919-4b23-8c6d-bc7dad61cc4d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"0fead0de-6a65-4eba-b400-eea5692702e0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"1a233de6-1b82-4a51-8e96-4cd4abdf16c8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"cc4870b8-dcd9-4e43-978d-8de6c6396abf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"d1d05620-42fb-4107-a2f9-e83542d076ee","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"ab26136c-31c3-4090-9fa2-988dba4f731f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T08:51:55.310 DEBUG:teuthology.misc:0 of 6 OSDs are up 2024-05-08T08:51:55.828 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:51:55.831+0000 7fba41f2a640 -1 osd.4 0 waiting for initial osdmap 2024-05-08T08:51:55.829 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:51:55.832+0000 7fe4e3c8b640 -1 osd.3 0 waiting for initial osdmap 2024-05-08T08:51:55.837 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:51:55.840+0000 7fed5c992640 -1 osd.5 0 waiting for initial osdmap 2024-05-08T08:51:55.841 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:51:55.845+0000 7fba39d4a640 -1 osd.4 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T08:51:55.842 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:51:55.846+0000 7fe4dbaab640 -1 osd.3 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T08:51:55.849 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:51:55.853+0000 7fed547b2640 -1 osd.5 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T08:51:56.044 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:51:56.047+0000 7f6368fd9640 -1 osd.2 0 waiting for initial osdmap 2024-05-08T08:51:56.057 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:51:56.061+0000 7f6360df9640 -1 osd.2 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T08:51:56.115 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:51:56.118+0000 7fb625f14640 -1 osd.1 0 waiting for initial osdmap 2024-05-08T08:51:56.122 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:51:56.126+0000 7fd766c15640 -1 osd.0 0 waiting for initial osdmap 2024-05-08T08:51:56.125 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:51:56.129+0000 7fb61d520640 -1 osd.1 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T08:51:56.140 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:51:56.144+0000 7fd75ea35640 -1 osd.0 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T08:52:01.916 DEBUG:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-08T08:52:02.219 INFO:teuthology.misc.health.smithi059.stdout: 2024-05-08T08:52:02.219 INFO:teuthology.misc.health.smithi059.stdout:{"epoch":14,"fsid":"38dfae8c-5c5e-4b0f-a264-91f22c6c5554","created":"2024-05-08T08:51:45.571369+0000","modified":"2024-05-08T08:51:59.860706+0000","last_up_change":"2024-05-08T08:51:56.847706+0000","last_in_change":"2024-05-08T08:51:47.267430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T08:51:57.787790+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"300801a5-1919-4b23-8c6d-bc7dad61cc4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":802160151},{"type":"v1","addr":"172.21.15.59:6801","nonce":802160151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":802160151},{"type":"v1","addr":"172.21.15.59:6803","nonce":802160151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":802160151},{"type":"v1","addr":"172.21.15.59:6807","nonce":802160151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":802160151},{"type":"v1","addr":"172.21.15.59:6805","nonce":802160151}]},"public_addr":"172.21.15.59:6801/802160151","cluster_addr":"172.21.15.59:6803/802160151","heartbeat_back_addr":"172.21.15.59:6807/802160151","heartbeat_front_addr":"172.21.15.59:6805/802160151","state":["exists","up"]},{"osd":1,"uuid":"0fead0de-6a65-4eba-b400-eea5692702e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":4116801011},{"type":"v1","addr":"172.21.15.59:6817","nonce":4116801011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":4116801011},{"type":"v1","addr":"172.21.15.59:6819","nonce":4116801011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":4116801011},{"type":"v1","addr":"172.21.15.59:6823","nonce":4116801011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":4116801011},{"type":"v1","addr":"172.21.15.59:6821","nonce":4116801011}]},"public_addr":"172.21.15.59:6817/4116801011","cluster_addr":"172.21.15.59:6819/4116801011","heartbeat_back_addr":"172.21.15.59:6823/4116801011","heartbeat_front_addr":"172.21.15.59:6821/4116801011","state":["exists","up"]},{"osd":2,"uuid":"1a233de6-1b82-4a51-8e96-4cd4abdf16c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1547661179},{"type":"v1","addr":"172.21.15.59:6809","nonce":1547661179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1547661179},{"type":"v1","addr":"172.21.15.59:6811","nonce":1547661179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1547661179},{"type":"v1","addr":"172.21.15.59:6815","nonce":1547661179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1547661179},{"type":"v1","addr":"172.21.15.59:6813","nonce":1547661179}]},"public_addr":"172.21.15.59:6809/1547661179","cluster_addr":"172.21.15.59:6811/1547661179","heartbeat_back_addr":"172.21.15.59:6815/1547661179","heartbeat_front_addr":"172.21.15.59:6813/1547661179","state":["exists","up"]},{"osd":3,"uuid":"cc4870b8-dcd9-4e43-978d-8de6c6396abf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":4054400810},{"type":"v1","addr":"172.21.15.192:6801","nonce":4054400810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":4054400810},{"type":"v1","addr":"172.21.15.192:6803","nonce":4054400810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":4054400810},{"type":"v1","addr":"172.21.15.192:6807","nonce":4054400810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":4054400810},{"type":"v1","addr":"172.21.15.192:6805","nonce":4054400810}]},"public_addr":"172.21.15.192:6801/4054400810","cluster_addr":"172.21.15.192:6803/4054400810","heartbeat_back_addr":"172.21.15.192:6807/4054400810","heartbeat_front_addr":"172.21.15.192:6805/4054400810","state":["exists","up"]},{"osd":4,"uuid":"d1d05620-42fb-4107-a2f9-e83542d076ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":58985565},{"type":"v1","addr":"172.21.15.192:6809","nonce":58985565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":58985565},{"type":"v1","addr":"172.21.15.192:6811","nonce":58985565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":58985565},{"type":"v1","addr":"172.21.15.192:6815","nonce":58985565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":58985565},{"type":"v1","addr":"172.21.15.192:6813","nonce":58985565}]},"public_addr":"172.21.15.192:6809/58985565","cluster_addr":"172.21.15.192:6811/58985565","heartbeat_back_addr":"172.21.15.192:6815/58985565","heartbeat_front_addr":"172.21.15.192:6813/58985565","state":["exists","up"]},{"osd":5,"uuid":"ab26136c-31c3-4090-9fa2-988dba4f731f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":1283571573},{"type":"v1","addr":"172.21.15.192:6817","nonce":1283571573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1283571573},{"type":"v1","addr":"172.21.15.192:6819","nonce":1283571573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":1283571573},{"type":"v1","addr":"172.21.15.192:6823","nonce":1283571573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":1283571573},{"type":"v1","addr":"172.21.15.192:6821","nonce":1283571573}]},"public_addr":"172.21.15.192:6817/1283571573","cluster_addr":"172.21.15.192:6819/1283571573","heartbeat_back_addr":"172.21.15.192:6823/1283571573","heartbeat_front_addr":"172.21.15.192:6821/1283571573","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T08:51:52.396946+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T08:51:52.453160+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T08:51:52.387606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T08:51:52.360952+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T08:51:52.468681+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T08:51:52.401080+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T08:52:02.230 DEBUG:teuthology.misc:6 of 6 OSDs are up 2024-05-08T08:52:02.230 INFO:tasks.ceph:Creating RBD pool 2024-05-08T08:52:02.230 DEBUG:teuthology.orchestra.run.smithi059:> sudo ceph --cluster ceph osd pool create rbd 8 2024-05-08T08:52:02.875 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'rbd' created 2024-05-08T08:52:02.891 DEBUG:teuthology.orchestra.run.smithi059:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2024-05-08T08:52:03.884 INFO:teuthology.orchestra.run.smithi059.stderr:enabled application 'rbd' on pool 'rbd' 2024-05-08T08:52:03.902 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-05-08T08:52:03.902 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-08T08:52:03.902 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-05-08T08:52:03.903 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-05-08T08:52:03.903 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-05-08T08:52:04.262 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:52:04.273 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":5,"flags":0,"active_gid":4105,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":4155038905},{"type":"v1","addr":"172.21.15.192:6825","nonce":4155038905}]},"active_addr":"172.21.15.192:6825/4155038905","active_change":"2024-05-08T08:51:49.734333+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.192:0","nonce":3921260605}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.192:0","nonce":574763665}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.192:0","nonce":59179521}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.192:0","nonce":4120971118}]}]} 2024-05-08T08:52:04.280 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-05-08T08:52:04.280 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-05-08T08:52:04.280 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T08:52:04.577 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:52:04.577 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":16,"fsid":"38dfae8c-5c5e-4b0f-a264-91f22c6c5554","created":"2024-05-08T08:51:45.571369+0000","modified":"2024-05-08T08:52:03.880055+0000","last_up_change":"2024-05-08T08:51:56.847706+0000","last_in_change":"2024-05-08T08:51:47.267430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T08:51:57.787790+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-05-08T08:52:02.563677+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"300801a5-1919-4b23-8c6d-bc7dad61cc4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":802160151},{"type":"v1","addr":"172.21.15.59:6801","nonce":802160151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":802160151},{"type":"v1","addr":"172.21.15.59:6803","nonce":802160151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":802160151},{"type":"v1","addr":"172.21.15.59:6807","nonce":802160151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":802160151},{"type":"v1","addr":"172.21.15.59:6805","nonce":802160151}]},"public_addr":"172.21.15.59:6801/802160151","cluster_addr":"172.21.15.59:6803/802160151","heartbeat_back_addr":"172.21.15.59:6807/802160151","heartbeat_front_addr":"172.21.15.59:6805/802160151","state":["exists","up"]},{"osd":1,"uuid":"0fead0de-6a65-4eba-b400-eea5692702e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":4116801011},{"type":"v1","addr":"172.21.15.59:6817","nonce":4116801011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":4116801011},{"type":"v1","addr":"172.21.15.59:6819","nonce":4116801011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":4116801011},{"type":"v1","addr":"172.21.15.59:6823","nonce":4116801011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":4116801011},{"type":"v1","addr":"172.21.15.59:6821","nonce":4116801011}]},"public_addr":"172.21.15.59:6817/4116801011","cluster_addr":"172.21.15.59:6819/4116801011","heartbeat_back_addr":"172.21.15.59:6823/4116801011","heartbeat_front_addr":"172.21.15.59:6821/4116801011","state":["exists","up"]},{"osd":2,"uuid":"1a233de6-1b82-4a51-8e96-4cd4abdf16c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1547661179},{"type":"v1","addr":"172.21.15.59:6809","nonce":1547661179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1547661179},{"type":"v1","addr":"172.21.15.59:6811","nonce":1547661179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1547661179},{"type":"v1","addr":"172.21.15.59:6815","nonce":1547661179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1547661179},{"type":"v1","addr":"172.21.15.59:6813","nonce":1547661179}]},"public_addr":"172.21.15.59:6809/1547661179","cluster_addr":"172.21.15.59:6811/1547661179","heartbeat_back_addr":"172.21.15.59:6815/1547661179","heartbeat_front_addr":"172.21.15.59:6813/1547661179","state":["exists","up"]},{"osd":3,"uuid":"cc4870b8-dcd9-4e43-978d-8de6c6396abf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":4054400810},{"type":"v1","addr":"172.21.15.192:6801","nonce":4054400810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":4054400810},{"type":"v1","addr":"172.21.15.192:6803","nonce":4054400810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":4054400810},{"type":"v1","addr":"172.21.15.192:6807","nonce":4054400810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":4054400810},{"type":"v1","addr":"172.21.15.192:6805","nonce":4054400810}]},"public_addr":"172.21.15.192:6801/4054400810","cluster_addr":"172.21.15.192:6803/4054400810","heartbeat_back_addr":"172.21.15.192:6807/4054400810","heartbeat_front_addr":"172.21.15.192:6805/4054400810","state":["exists","up"]},{"osd":4,"uuid":"d1d05620-42fb-4107-a2f9-e83542d076ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":58985565},{"type":"v1","addr":"172.21.15.192:6809","nonce":58985565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":58985565},{"type":"v1","addr":"172.21.15.192:6811","nonce":58985565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":58985565},{"type":"v1","addr":"172.21.15.192:6815","nonce":58985565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":58985565},{"type":"v1","addr":"172.21.15.192:6813","nonce":58985565}]},"public_addr":"172.21.15.192:6809/58985565","cluster_addr":"172.21.15.192:6811/58985565","heartbeat_back_addr":"172.21.15.192:6815/58985565","heartbeat_front_addr":"172.21.15.192:6813/58985565","state":["exists","up"]},{"osd":5,"uuid":"ab26136c-31c3-4090-9fa2-988dba4f731f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":1283571573},{"type":"v1","addr":"172.21.15.192:6817","nonce":1283571573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1283571573},{"type":"v1","addr":"172.21.15.192:6819","nonce":1283571573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":1283571573},{"type":"v1","addr":"172.21.15.192:6823","nonce":1283571573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":1283571573},{"type":"v1","addr":"172.21.15.192:6821","nonce":1283571573}]},"public_addr":"172.21.15.192:6817/1283571573","cluster_addr":"172.21.15.192:6819/1283571573","heartbeat_back_addr":"172.21.15.192:6823/1283571573","heartbeat_front_addr":"172.21.15.192:6821/1283571573","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T08:51:52.396946+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T08:51:52.453160+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T08:51:52.387606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T08:51:52.360952+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T08:51:52.468681+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T08:51:52.401080+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T08:52:04.590 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-05-08T08:52:04.590 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T08:52:04.892 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:52:04.893 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":17,"fsid":"38dfae8c-5c5e-4b0f-a264-91f22c6c5554","created":"2024-05-08T08:51:45.571369+0000","modified":"2024-05-08T08:52:04.884581+0000","last_up_change":"2024-05-08T08:51:56.847706+0000","last_in_change":"2024-05-08T08:51:47.267430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T08:51:57.787790+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-05-08T08:52:02.563677+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"300801a5-1919-4b23-8c6d-bc7dad61cc4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":802160151},{"type":"v1","addr":"172.21.15.59:6801","nonce":802160151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":802160151},{"type":"v1","addr":"172.21.15.59:6803","nonce":802160151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":802160151},{"type":"v1","addr":"172.21.15.59:6807","nonce":802160151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":802160151},{"type":"v1","addr":"172.21.15.59:6805","nonce":802160151}]},"public_addr":"172.21.15.59:6801/802160151","cluster_addr":"172.21.15.59:6803/802160151","heartbeat_back_addr":"172.21.15.59:6807/802160151","heartbeat_front_addr":"172.21.15.59:6805/802160151","state":["exists","up"]},{"osd":1,"uuid":"0fead0de-6a65-4eba-b400-eea5692702e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":4116801011},{"type":"v1","addr":"172.21.15.59:6817","nonce":4116801011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":4116801011},{"type":"v1","addr":"172.21.15.59:6819","nonce":4116801011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":4116801011},{"type":"v1","addr":"172.21.15.59:6823","nonce":4116801011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":4116801011},{"type":"v1","addr":"172.21.15.59:6821","nonce":4116801011}]},"public_addr":"172.21.15.59:6817/4116801011","cluster_addr":"172.21.15.59:6819/4116801011","heartbeat_back_addr":"172.21.15.59:6823/4116801011","heartbeat_front_addr":"172.21.15.59:6821/4116801011","state":["exists","up"]},{"osd":2,"uuid":"1a233de6-1b82-4a51-8e96-4cd4abdf16c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1547661179},{"type":"v1","addr":"172.21.15.59:6809","nonce":1547661179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1547661179},{"type":"v1","addr":"172.21.15.59:6811","nonce":1547661179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1547661179},{"type":"v1","addr":"172.21.15.59:6815","nonce":1547661179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1547661179},{"type":"v1","addr":"172.21.15.59:6813","nonce":1547661179}]},"public_addr":"172.21.15.59:6809/1547661179","cluster_addr":"172.21.15.59:6811/1547661179","heartbeat_back_addr":"172.21.15.59:6815/1547661179","heartbeat_front_addr":"172.21.15.59:6813/1547661179","state":["exists","up"]},{"osd":3,"uuid":"cc4870b8-dcd9-4e43-978d-8de6c6396abf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":4054400810},{"type":"v1","addr":"172.21.15.192:6801","nonce":4054400810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":4054400810},{"type":"v1","addr":"172.21.15.192:6803","nonce":4054400810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":4054400810},{"type":"v1","addr":"172.21.15.192:6807","nonce":4054400810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":4054400810},{"type":"v1","addr":"172.21.15.192:6805","nonce":4054400810}]},"public_addr":"172.21.15.192:6801/4054400810","cluster_addr":"172.21.15.192:6803/4054400810","heartbeat_back_addr":"172.21.15.192:6807/4054400810","heartbeat_front_addr":"172.21.15.192:6805/4054400810","state":["exists","up"]},{"osd":4,"uuid":"d1d05620-42fb-4107-a2f9-e83542d076ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":58985565},{"type":"v1","addr":"172.21.15.192:6809","nonce":58985565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":58985565},{"type":"v1","addr":"172.21.15.192:6811","nonce":58985565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":58985565},{"type":"v1","addr":"172.21.15.192:6815","nonce":58985565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":58985565},{"type":"v1","addr":"172.21.15.192:6813","nonce":58985565}]},"public_addr":"172.21.15.192:6809/58985565","cluster_addr":"172.21.15.192:6811/58985565","heartbeat_back_addr":"172.21.15.192:6815/58985565","heartbeat_front_addr":"172.21.15.192:6813/58985565","state":["exists","up"]},{"osd":5,"uuid":"ab26136c-31c3-4090-9fa2-988dba4f731f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":1283571573},{"type":"v1","addr":"172.21.15.192:6817","nonce":1283571573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1283571573},{"type":"v1","addr":"172.21.15.192:6819","nonce":1283571573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":1283571573},{"type":"v1","addr":"172.21.15.192:6823","nonce":1283571573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":1283571573},{"type":"v1","addr":"172.21.15.192:6821","nonce":1283571573}]},"public_addr":"172.21.15.192:6817/1283571573","cluster_addr":"172.21.15.192:6819/1283571573","heartbeat_back_addr":"172.21.15.192:6823/1283571573","heartbeat_front_addr":"172.21.15.192:6821/1283571573","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T08:51:52.396946+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T08:51:52.453160+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T08:51:52.387606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T08:51:52.360952+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T08:51:52.468681+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T08:51:52.401080+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T08:52:04.906 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-05-08T08:52:04.906 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-05-08T08:52:04.907 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-05-08T08:52:04.907 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-05-08T08:52:04.908 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-05-08T08:52:04.908 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-05-08T08:52:05.095 INFO:teuthology.orchestra.run.smithi059.stdout:47244640262 2024-05-08T08:52:05.096 INFO:teuthology.orchestra.run.smithi059.stdout:47244640259 2024-05-08T08:52:05.096 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-08T08:52:05.097 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-08T08:52:05.120 INFO:teuthology.orchestra.run.smithi059.stdout:47244640262 2024-05-08T08:52:05.120 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-08T08:52:05.125 INFO:teuthology.orchestra.run.smithi059.stdout:47244640261 2024-05-08T08:52:05.125 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-08T08:52:05.135 INFO:teuthology.orchestra.run.smithi059.stdout:47244640259 2024-05-08T08:52:05.136 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-08T08:52:05.137 INFO:teuthology.orchestra.run.smithi059.stdout:47244640260 2024-05-08T08:52:05.137 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-08T08:52:05.463 INFO:teuthology.orchestra.run.smithi059.stdout:47244640258 2024-05-08T08:52:05.478 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640259 got 47244640258 for osd.1 2024-05-08T08:52:05.518 INFO:teuthology.orchestra.run.smithi059.stdout:47244640260 2024-05-08T08:52:05.532 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640261 got 47244640260 for osd.3 2024-05-08T08:52:05.598 INFO:teuthology.orchestra.run.smithi059.stdout:47244640260 2024-05-08T08:52:05.604 INFO:teuthology.orchestra.run.smithi059.stdout:47244640261 2024-05-08T08:52:05.612 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640262 got 47244640260 for osd.0 2024-05-08T08:52:05.622 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640262 got 47244640261 for osd.4 2024-05-08T08:52:05.640 INFO:teuthology.orchestra.run.smithi059.stdout:47244640258 2024-05-08T08:52:05.653 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640259 got 47244640258 for osd.5 2024-05-08T08:52:05.676 INFO:teuthology.orchestra.run.smithi059.stdout:47244640259 2024-05-08T08:52:05.689 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640260 got 47244640259 for osd.2 2024-05-08T08:52:06.478 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-08T08:52:06.532 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-08T08:52:06.613 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-08T08:52:06.623 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-08T08:52:06.654 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-08T08:52:06.690 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-08T08:52:06.925 INFO:teuthology.orchestra.run.smithi059.stdout:47244640259 2024-05-08T08:52:06.945 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640259 got 47244640259 for osd.1 2024-05-08T08:52:06.945 DEBUG:teuthology.parallel:result is None 2024-05-08T08:52:06.954 INFO:teuthology.orchestra.run.smithi059.stdout:47244640262 2024-05-08T08:52:06.968 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640262 got 47244640262 for osd.0 2024-05-08T08:52:06.968 INFO:teuthology.orchestra.run.smithi059.stdout:47244640261 2024-05-08T08:52:06.970 DEBUG:teuthology.parallel:result is None 2024-05-08T08:52:06.981 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640261 got 47244640261 for osd.3 2024-05-08T08:52:06.981 DEBUG:teuthology.parallel:result is None 2024-05-08T08:52:07.069 INFO:teuthology.orchestra.run.smithi059.stdout:47244640259 2024-05-08T08:52:07.082 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640259 got 47244640259 for osd.5 2024-05-08T08:52:07.082 DEBUG:teuthology.parallel:result is None 2024-05-08T08:52:07.083 INFO:teuthology.orchestra.run.smithi059.stdout:47244640262 2024-05-08T08:52:07.096 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640262 got 47244640262 for osd.4 2024-05-08T08:52:07.096 DEBUG:teuthology.parallel:result is None 2024-05-08T08:52:07.148 INFO:teuthology.orchestra.run.smithi059.stdout:47244640260 2024-05-08T08:52:07.161 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640260 got 47244640260 for osd.2 2024-05-08T08:52:07.161 DEBUG:teuthology.parallel:result is None 2024-05-08T08:52:07.161 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-08T08:52:07.162 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T08:52:07.459 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:52:07.459 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-05-08T08:52:07.471 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":16,"stamp":"2024-05-08T08:52:05.743881+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":12,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":163096,"kb_used_data":976,"kb_used_omap":9,"kb_used_meta":162038,"kb_avail":566067944,"statfs":{"total":579820584960,"available":579653574656,"internally_reserved":0,"allocated":999424,"data_stored":1462358,"data_compressed":11046,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":9568,"internal_metadata":165927584},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.863567"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-08T08:52:04.890534+0000","last_change":"2024-05-08T08:52:03.893505+0000","last_active":"2024-05-08T08:52:04.890534+0000","last_peered":"2024-05-08T08:52:04.890534+0000","last_clean":"2024-05-08T08:52:04.890534+0000","last_became_active":"2024-05-08T08:52:03.893187+0000","last_became_peered":"2024-05-08T08:52:03.893187+0000","last_unstale":"2024-05-08T08:52:04.890534+0000","last_undegraded":"2024-05-08T08:52:04.890534+0000","last_fullsized":"2024-05-08T08:52:04.890534+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:51:04.039242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-08T08:52:03.897593+0000","last_change":"2024-05-08T08:52:03.896159+0000","last_active":"2024-05-08T08:52:03.897593+0000","last_peered":"2024-05-08T08:52:03.897593+0000","last_clean":"2024-05-08T08:52:03.897593+0000","last_became_active":"2024-05-08T08:52:03.895797+0000","last_became_peered":"2024-05-08T08:52:03.895797+0000","last_unstale":"2024-05-08T08:52:03.897593+0000","last_undegraded":"2024-05-08T08:52:03.897593+0000","last_fullsized":"2024-05-08T08:52:03.897593+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:03:19.373388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-08T08:52:04.890521+0000","last_change":"2024-05-08T08:52:04.144764+0000","last_active":"2024-05-08T08:52:04.890521+0000","last_peered":"2024-05-08T08:52:04.890521+0000","last_clean":"2024-05-08T08:52:04.890521+0000","last_became_active":"2024-05-08T08:52:04.144444+0000","last_became_peered":"2024-05-08T08:52:04.144444+0000","last_unstale":"2024-05-08T08:52:04.890521+0000","last_undegraded":"2024-05-08T08:52:04.890521+0000","last_fullsized":"2024-05-08T08:52:04.890521+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:58:31.498922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-08T08:52:04.145306+0000","last_change":"2024-05-08T08:52:04.144543+0000","last_active":"2024-05-08T08:52:04.145306+0000","last_peered":"2024-05-08T08:52:04.145306+0000","last_clean":"2024-05-08T08:52:04.145306+0000","last_became_active":"2024-05-08T08:52:04.144242+0000","last_became_peered":"2024-05-08T08:52:04.144242+0000","last_unstale":"2024-05-08T08:52:04.145306+0000","last_undegraded":"2024-05-08T08:52:04.145306+0000","last_fullsized":"2024-05-08T08:52:04.145306+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:41:40.496612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-08T08:52:03.896817+0000","last_change":"2024-05-08T08:52:03.895937+0000","last_active":"2024-05-08T08:52:03.896817+0000","last_peered":"2024-05-08T08:52:03.896817+0000","last_clean":"2024-05-08T08:52:03.896817+0000","last_became_active":"2024-05-08T08:52:03.895628+0000","last_became_peered":"2024-05-08T08:52:03.895628+0000","last_unstale":"2024-05-08T08:52:03.896817+0000","last_undegraded":"2024-05-08T08:52:03.896817+0000","last_fullsized":"2024-05-08T08:52:03.896817+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:34:27.763188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-08T08:52:03.893183+0000","last_change":"2024-05-08T08:52:03.891768+0000","last_active":"2024-05-08T08:52:03.893183+0000","last_peered":"2024-05-08T08:52:03.893183+0000","last_clean":"2024-05-08T08:52:03.893183+0000","last_became_active":"2024-05-08T08:52:03.891374+0000","last_became_peered":"2024-05-08T08:52:03.891374+0000","last_unstale":"2024-05-08T08:52:03.893183+0000","last_undegraded":"2024-05-08T08:52:03.893183+0000","last_fullsized":"2024-05-08T08:52:03.893183+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:40:47.635551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-08T08:52:04.890523+0000","last_change":"2024-05-08T08:52:03.891551+0000","last_active":"2024-05-08T08:52:04.890523+0000","last_peered":"2024-05-08T08:52:04.890523+0000","last_clean":"2024-05-08T08:52:04.890523+0000","last_became_active":"2024-05-08T08:52:03.891296+0000","last_became_peered":"2024-05-08T08:52:03.891296+0000","last_unstale":"2024-05-08T08:52:04.890523+0000","last_undegraded":"2024-05-08T08:52:04.890523+0000","last_fullsized":"2024-05-08T08:52:04.890523+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:54.806343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-08T08:52:03.896832+0000","last_change":"2024-05-08T08:52:03.895923+0000","last_active":"2024-05-08T08:52:03.896832+0000","last_peered":"2024-05-08T08:52:03.896832+0000","last_clean":"2024-05-08T08:52:03.896832+0000","last_became_active":"2024-05-08T08:52:03.895575+0000","last_became_peered":"2024-05-08T08:52:03.895575+0000","last_unstale":"2024-05-08T08:52:03.896832+0000","last_undegraded":"2024-05-08T08:52:03.896832+0000","last_fullsized":"2024-05-08T08:52:03.896832+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:32:48.278684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":168,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-08T08:52:04.890396+0000","last_change":"2024-05-08T08:51:58.861899+0000","last_active":"2024-05-08T08:52:04.890396+0000","last_peered":"2024-05-08T08:52:04.890396+0000","last_clean":"2024-05-08T08:52:04.890396+0000","last_became_active":"2024-05-08T08:51:58.861596+0000","last_became_peered":"2024-05-08T08:51:58.861596+0000","last_unstale":"2024-05-08T08:52:04.890396+0000","last_undegraded":"2024-05-08T08:52:04.890396+0000","last_fullsized":"2024-05-08T08:52:04.890396+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:51:57.849666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:51:57.849666+0000","last_clean_scrub_stamp":"2024-05-08T08:51:57.849666+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:53:21.739950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":11046,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640259,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":147456,"data_stored":46937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640262,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":147456,"data_stored":46937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640261,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":637305,"data_compressed":5523,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640260,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":147456,"data_stored":46937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640259,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":147456,"data_stored":46937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640262,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":637305,"data_compressed":5523,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5523,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5523,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T08:52:07.472 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T08:52:07.767 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:52:07.767 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-05-08T08:52:07.779 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":17,"stamp":"2024-05-08T08:52:07.744210+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":12,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":163096,"kb_used_data":976,"kb_used_omap":9,"kb_used_meta":162038,"kb_avail":566067944,"statfs":{"total":579820584960,"available":579653574656,"internally_reserved":0,"allocated":999424,"data_stored":1462358,"data_compressed":11046,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":9568,"internal_metadata":165927584},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.863895"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-08T08:52:04.890534+0000","last_change":"2024-05-08T08:52:03.893505+0000","last_active":"2024-05-08T08:52:04.890534+0000","last_peered":"2024-05-08T08:52:04.890534+0000","last_clean":"2024-05-08T08:52:04.890534+0000","last_became_active":"2024-05-08T08:52:03.893187+0000","last_became_peered":"2024-05-08T08:52:03.893187+0000","last_unstale":"2024-05-08T08:52:04.890534+0000","last_undegraded":"2024-05-08T08:52:04.890534+0000","last_fullsized":"2024-05-08T08:52:04.890534+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:51:04.039242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-08T08:52:03.897593+0000","last_change":"2024-05-08T08:52:03.896159+0000","last_active":"2024-05-08T08:52:03.897593+0000","last_peered":"2024-05-08T08:52:03.897593+0000","last_clean":"2024-05-08T08:52:03.897593+0000","last_became_active":"2024-05-08T08:52:03.895797+0000","last_became_peered":"2024-05-08T08:52:03.895797+0000","last_unstale":"2024-05-08T08:52:03.897593+0000","last_undegraded":"2024-05-08T08:52:03.897593+0000","last_fullsized":"2024-05-08T08:52:03.897593+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:03:19.373388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-08T08:52:04.890521+0000","last_change":"2024-05-08T08:52:04.144764+0000","last_active":"2024-05-08T08:52:04.890521+0000","last_peered":"2024-05-08T08:52:04.890521+0000","last_clean":"2024-05-08T08:52:04.890521+0000","last_became_active":"2024-05-08T08:52:04.144444+0000","last_became_peered":"2024-05-08T08:52:04.144444+0000","last_unstale":"2024-05-08T08:52:04.890521+0000","last_undegraded":"2024-05-08T08:52:04.890521+0000","last_fullsized":"2024-05-08T08:52:04.890521+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:58:31.498922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-08T08:52:04.145306+0000","last_change":"2024-05-08T08:52:04.144543+0000","last_active":"2024-05-08T08:52:04.145306+0000","last_peered":"2024-05-08T08:52:04.145306+0000","last_clean":"2024-05-08T08:52:04.145306+0000","last_became_active":"2024-05-08T08:52:04.144242+0000","last_became_peered":"2024-05-08T08:52:04.144242+0000","last_unstale":"2024-05-08T08:52:04.145306+0000","last_undegraded":"2024-05-08T08:52:04.145306+0000","last_fullsized":"2024-05-08T08:52:04.145306+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:41:40.496612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-08T08:52:03.896817+0000","last_change":"2024-05-08T08:52:03.895937+0000","last_active":"2024-05-08T08:52:03.896817+0000","last_peered":"2024-05-08T08:52:03.896817+0000","last_clean":"2024-05-08T08:52:03.896817+0000","last_became_active":"2024-05-08T08:52:03.895628+0000","last_became_peered":"2024-05-08T08:52:03.895628+0000","last_unstale":"2024-05-08T08:52:03.896817+0000","last_undegraded":"2024-05-08T08:52:03.896817+0000","last_fullsized":"2024-05-08T08:52:03.896817+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:34:27.763188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-08T08:52:03.893183+0000","last_change":"2024-05-08T08:52:03.891768+0000","last_active":"2024-05-08T08:52:03.893183+0000","last_peered":"2024-05-08T08:52:03.893183+0000","last_clean":"2024-05-08T08:52:03.893183+0000","last_became_active":"2024-05-08T08:52:03.891374+0000","last_became_peered":"2024-05-08T08:52:03.891374+0000","last_unstale":"2024-05-08T08:52:03.893183+0000","last_undegraded":"2024-05-08T08:52:03.893183+0000","last_fullsized":"2024-05-08T08:52:03.893183+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:40:47.635551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-08T08:52:04.890523+0000","last_change":"2024-05-08T08:52:03.891551+0000","last_active":"2024-05-08T08:52:04.890523+0000","last_peered":"2024-05-08T08:52:04.890523+0000","last_clean":"2024-05-08T08:52:04.890523+0000","last_became_active":"2024-05-08T08:52:03.891296+0000","last_became_peered":"2024-05-08T08:52:03.891296+0000","last_unstale":"2024-05-08T08:52:04.890523+0000","last_undegraded":"2024-05-08T08:52:04.890523+0000","last_fullsized":"2024-05-08T08:52:04.890523+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:54.806343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-08T08:52:03.896832+0000","last_change":"2024-05-08T08:52:03.895923+0000","last_active":"2024-05-08T08:52:03.896832+0000","last_peered":"2024-05-08T08:52:03.896832+0000","last_clean":"2024-05-08T08:52:03.896832+0000","last_became_active":"2024-05-08T08:52:03.895575+0000","last_became_peered":"2024-05-08T08:52:03.895575+0000","last_unstale":"2024-05-08T08:52:03.896832+0000","last_undegraded":"2024-05-08T08:52:03.896832+0000","last_fullsized":"2024-05-08T08:52:03.896832+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:32:48.278684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":168,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-08T08:52:04.890396+0000","last_change":"2024-05-08T08:51:58.861899+0000","last_active":"2024-05-08T08:52:04.890396+0000","last_peered":"2024-05-08T08:52:04.890396+0000","last_clean":"2024-05-08T08:52:04.890396+0000","last_became_active":"2024-05-08T08:51:58.861596+0000","last_became_peered":"2024-05-08T08:51:58.861596+0000","last_unstale":"2024-05-08T08:52:04.890396+0000","last_undegraded":"2024-05-08T08:52:04.890396+0000","last_fullsized":"2024-05-08T08:52:04.890396+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:51:57.849666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:51:57.849666+0000","last_clean_scrub_stamp":"2024-05-08T08:51:57.849666+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:53:21.739950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":11046,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640259,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":147456,"data_stored":46937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640262,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":147456,"data_stored":46937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640261,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":637305,"data_compressed":5523,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640260,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":147456,"data_stored":46937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640259,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":147456,"data_stored":46937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640262,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":637305,"data_compressed":5523,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5523,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5523,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T08:52:07.780 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-08T08:52:07.780 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-08T08:52:07.780 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-05-08T08:52:07.780 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-08T08:52:08.115 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:52:08.115 INFO:teuthology.orchestra.run.smithi059.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-08T08:52:08.128 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-05-08T08:52:08.128 INFO:teuthology.run_tasks:Running task thrashosds... 2024-05-08T08:52:08.139 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-05-08T08:52:08.140 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zlib', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED'], 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'extra_packages': ['rbd-nbd'], 'flavor': 'default', 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc'}} 2024-05-08T08:52:08.140 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-05-08T08:52:08.140 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-05-08T08:52:08.456 INFO:tasks.thrashosds:Beginning thrashosds... 2024-05-08T08:52:08.456 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-08T08:52:08.456 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T08:52:08.750 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:52:08.751 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-05-08T08:52:08.762 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":17,"stamp":"2024-05-08T08:52:07.744210+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":12,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":163096,"kb_used_data":976,"kb_used_omap":9,"kb_used_meta":162038,"kb_avail":566067944,"statfs":{"total":579820584960,"available":579653574656,"internally_reserved":0,"allocated":999424,"data_stored":1462358,"data_compressed":11046,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":9568,"internal_metadata":165927584},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.863895"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-08T08:52:04.890534+0000","last_change":"2024-05-08T08:52:03.893505+0000","last_active":"2024-05-08T08:52:04.890534+0000","last_peered":"2024-05-08T08:52:04.890534+0000","last_clean":"2024-05-08T08:52:04.890534+0000","last_became_active":"2024-05-08T08:52:03.893187+0000","last_became_peered":"2024-05-08T08:52:03.893187+0000","last_unstale":"2024-05-08T08:52:04.890534+0000","last_undegraded":"2024-05-08T08:52:04.890534+0000","last_fullsized":"2024-05-08T08:52:04.890534+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:51:04.039242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-08T08:52:03.897593+0000","last_change":"2024-05-08T08:52:03.896159+0000","last_active":"2024-05-08T08:52:03.897593+0000","last_peered":"2024-05-08T08:52:03.897593+0000","last_clean":"2024-05-08T08:52:03.897593+0000","last_became_active":"2024-05-08T08:52:03.895797+0000","last_became_peered":"2024-05-08T08:52:03.895797+0000","last_unstale":"2024-05-08T08:52:03.897593+0000","last_undegraded":"2024-05-08T08:52:03.897593+0000","last_fullsized":"2024-05-08T08:52:03.897593+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:03:19.373388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-08T08:52:04.890521+0000","last_change":"2024-05-08T08:52:04.144764+0000","last_active":"2024-05-08T08:52:04.890521+0000","last_peered":"2024-05-08T08:52:04.890521+0000","last_clean":"2024-05-08T08:52:04.890521+0000","last_became_active":"2024-05-08T08:52:04.144444+0000","last_became_peered":"2024-05-08T08:52:04.144444+0000","last_unstale":"2024-05-08T08:52:04.890521+0000","last_undegraded":"2024-05-08T08:52:04.890521+0000","last_fullsized":"2024-05-08T08:52:04.890521+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:58:31.498922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-08T08:52:04.145306+0000","last_change":"2024-05-08T08:52:04.144543+0000","last_active":"2024-05-08T08:52:04.145306+0000","last_peered":"2024-05-08T08:52:04.145306+0000","last_clean":"2024-05-08T08:52:04.145306+0000","last_became_active":"2024-05-08T08:52:04.144242+0000","last_became_peered":"2024-05-08T08:52:04.144242+0000","last_unstale":"2024-05-08T08:52:04.145306+0000","last_undegraded":"2024-05-08T08:52:04.145306+0000","last_fullsized":"2024-05-08T08:52:04.145306+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:41:40.496612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-08T08:52:03.896817+0000","last_change":"2024-05-08T08:52:03.895937+0000","last_active":"2024-05-08T08:52:03.896817+0000","last_peered":"2024-05-08T08:52:03.896817+0000","last_clean":"2024-05-08T08:52:03.896817+0000","last_became_active":"2024-05-08T08:52:03.895628+0000","last_became_peered":"2024-05-08T08:52:03.895628+0000","last_unstale":"2024-05-08T08:52:03.896817+0000","last_undegraded":"2024-05-08T08:52:03.896817+0000","last_fullsized":"2024-05-08T08:52:03.896817+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:34:27.763188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-08T08:52:03.893183+0000","last_change":"2024-05-08T08:52:03.891768+0000","last_active":"2024-05-08T08:52:03.893183+0000","last_peered":"2024-05-08T08:52:03.893183+0000","last_clean":"2024-05-08T08:52:03.893183+0000","last_became_active":"2024-05-08T08:52:03.891374+0000","last_became_peered":"2024-05-08T08:52:03.891374+0000","last_unstale":"2024-05-08T08:52:03.893183+0000","last_undegraded":"2024-05-08T08:52:03.893183+0000","last_fullsized":"2024-05-08T08:52:03.893183+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:40:47.635551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-08T08:52:04.890523+0000","last_change":"2024-05-08T08:52:03.891551+0000","last_active":"2024-05-08T08:52:04.890523+0000","last_peered":"2024-05-08T08:52:04.890523+0000","last_clean":"2024-05-08T08:52:04.890523+0000","last_became_active":"2024-05-08T08:52:03.891296+0000","last_became_peered":"2024-05-08T08:52:03.891296+0000","last_unstale":"2024-05-08T08:52:04.890523+0000","last_undegraded":"2024-05-08T08:52:04.890523+0000","last_fullsized":"2024-05-08T08:52:04.890523+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:54.806343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-08T08:52:03.896832+0000","last_change":"2024-05-08T08:52:03.895923+0000","last_active":"2024-05-08T08:52:03.896832+0000","last_peered":"2024-05-08T08:52:03.896832+0000","last_clean":"2024-05-08T08:52:03.896832+0000","last_became_active":"2024-05-08T08:52:03.895575+0000","last_became_peered":"2024-05-08T08:52:03.895575+0000","last_unstale":"2024-05-08T08:52:03.896832+0000","last_undegraded":"2024-05-08T08:52:03.896832+0000","last_fullsized":"2024-05-08T08:52:03.896832+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:32:48.278684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":168,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-08T08:52:04.890396+0000","last_change":"2024-05-08T08:51:58.861899+0000","last_active":"2024-05-08T08:52:04.890396+0000","last_peered":"2024-05-08T08:52:04.890396+0000","last_clean":"2024-05-08T08:52:04.890396+0000","last_became_active":"2024-05-08T08:51:58.861596+0000","last_became_peered":"2024-05-08T08:51:58.861596+0000","last_unstale":"2024-05-08T08:52:04.890396+0000","last_undegraded":"2024-05-08T08:52:04.890396+0000","last_fullsized":"2024-05-08T08:52:04.890396+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:51:57.849666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:51:57.849666+0000","last_clean_scrub_stamp":"2024-05-08T08:51:57.849666+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:53:21.739950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":11046,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640259,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":147456,"data_stored":46937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640262,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":147456,"data_stored":46937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640261,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":637305,"data_compressed":5523,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640260,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":147456,"data_stored":46937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640259,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":147456,"data_stored":46937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640262,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":637305,"data_compressed":5523,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5523,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5523,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T08:52:08.763 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T08:52:09.056 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:52:09.057 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-05-08T08:52:09.069 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":17,"stamp":"2024-05-08T08:52:07.744210+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":12,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":163096,"kb_used_data":976,"kb_used_omap":9,"kb_used_meta":162038,"kb_avail":566067944,"statfs":{"total":579820584960,"available":579653574656,"internally_reserved":0,"allocated":999424,"data_stored":1462358,"data_compressed":11046,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":9568,"internal_metadata":165927584},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.863895"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-08T08:52:04.890534+0000","last_change":"2024-05-08T08:52:03.893505+0000","last_active":"2024-05-08T08:52:04.890534+0000","last_peered":"2024-05-08T08:52:04.890534+0000","last_clean":"2024-05-08T08:52:04.890534+0000","last_became_active":"2024-05-08T08:52:03.893187+0000","last_became_peered":"2024-05-08T08:52:03.893187+0000","last_unstale":"2024-05-08T08:52:04.890534+0000","last_undegraded":"2024-05-08T08:52:04.890534+0000","last_fullsized":"2024-05-08T08:52:04.890534+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:51:04.039242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-08T08:52:03.897593+0000","last_change":"2024-05-08T08:52:03.896159+0000","last_active":"2024-05-08T08:52:03.897593+0000","last_peered":"2024-05-08T08:52:03.897593+0000","last_clean":"2024-05-08T08:52:03.897593+0000","last_became_active":"2024-05-08T08:52:03.895797+0000","last_became_peered":"2024-05-08T08:52:03.895797+0000","last_unstale":"2024-05-08T08:52:03.897593+0000","last_undegraded":"2024-05-08T08:52:03.897593+0000","last_fullsized":"2024-05-08T08:52:03.897593+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:03:19.373388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-08T08:52:04.890521+0000","last_change":"2024-05-08T08:52:04.144764+0000","last_active":"2024-05-08T08:52:04.890521+0000","last_peered":"2024-05-08T08:52:04.890521+0000","last_clean":"2024-05-08T08:52:04.890521+0000","last_became_active":"2024-05-08T08:52:04.144444+0000","last_became_peered":"2024-05-08T08:52:04.144444+0000","last_unstale":"2024-05-08T08:52:04.890521+0000","last_undegraded":"2024-05-08T08:52:04.890521+0000","last_fullsized":"2024-05-08T08:52:04.890521+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:58:31.498922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-08T08:52:04.145306+0000","last_change":"2024-05-08T08:52:04.144543+0000","last_active":"2024-05-08T08:52:04.145306+0000","last_peered":"2024-05-08T08:52:04.145306+0000","last_clean":"2024-05-08T08:52:04.145306+0000","last_became_active":"2024-05-08T08:52:04.144242+0000","last_became_peered":"2024-05-08T08:52:04.144242+0000","last_unstale":"2024-05-08T08:52:04.145306+0000","last_undegraded":"2024-05-08T08:52:04.145306+0000","last_fullsized":"2024-05-08T08:52:04.145306+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:41:40.496612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-08T08:52:03.896817+0000","last_change":"2024-05-08T08:52:03.895937+0000","last_active":"2024-05-08T08:52:03.896817+0000","last_peered":"2024-05-08T08:52:03.896817+0000","last_clean":"2024-05-08T08:52:03.896817+0000","last_became_active":"2024-05-08T08:52:03.895628+0000","last_became_peered":"2024-05-08T08:52:03.895628+0000","last_unstale":"2024-05-08T08:52:03.896817+0000","last_undegraded":"2024-05-08T08:52:03.896817+0000","last_fullsized":"2024-05-08T08:52:03.896817+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:34:27.763188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-08T08:52:03.893183+0000","last_change":"2024-05-08T08:52:03.891768+0000","last_active":"2024-05-08T08:52:03.893183+0000","last_peered":"2024-05-08T08:52:03.893183+0000","last_clean":"2024-05-08T08:52:03.893183+0000","last_became_active":"2024-05-08T08:52:03.891374+0000","last_became_peered":"2024-05-08T08:52:03.891374+0000","last_unstale":"2024-05-08T08:52:03.893183+0000","last_undegraded":"2024-05-08T08:52:03.893183+0000","last_fullsized":"2024-05-08T08:52:03.893183+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:40:47.635551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-08T08:52:04.890523+0000","last_change":"2024-05-08T08:52:03.891551+0000","last_active":"2024-05-08T08:52:04.890523+0000","last_peered":"2024-05-08T08:52:04.890523+0000","last_clean":"2024-05-08T08:52:04.890523+0000","last_became_active":"2024-05-08T08:52:03.891296+0000","last_became_peered":"2024-05-08T08:52:03.891296+0000","last_unstale":"2024-05-08T08:52:04.890523+0000","last_undegraded":"2024-05-08T08:52:04.890523+0000","last_fullsized":"2024-05-08T08:52:04.890523+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:54.806343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-08T08:52:03.896832+0000","last_change":"2024-05-08T08:52:03.895923+0000","last_active":"2024-05-08T08:52:03.896832+0000","last_peered":"2024-05-08T08:52:03.896832+0000","last_clean":"2024-05-08T08:52:03.896832+0000","last_became_active":"2024-05-08T08:52:03.895575+0000","last_became_peered":"2024-05-08T08:52:03.895575+0000","last_unstale":"2024-05-08T08:52:03.896832+0000","last_undegraded":"2024-05-08T08:52:03.896832+0000","last_fullsized":"2024-05-08T08:52:03.896832+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:32:48.278684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":168,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-08T08:52:04.890396+0000","last_change":"2024-05-08T08:51:58.861899+0000","last_active":"2024-05-08T08:52:04.890396+0000","last_peered":"2024-05-08T08:52:04.890396+0000","last_clean":"2024-05-08T08:52:04.890396+0000","last_became_active":"2024-05-08T08:51:58.861596+0000","last_became_peered":"2024-05-08T08:51:58.861596+0000","last_unstale":"2024-05-08T08:52:04.890396+0000","last_undegraded":"2024-05-08T08:52:04.890396+0000","last_fullsized":"2024-05-08T08:52:04.890396+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:51:57.849666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:51:57.849666+0000","last_clean_scrub_stamp":"2024-05-08T08:51:57.849666+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:53:21.739950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":11046,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640259,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":147456,"data_stored":46937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640262,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":147456,"data_stored":46937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640261,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":637305,"data_compressed":5523,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640260,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":147456,"data_stored":46937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640259,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":147456,"data_stored":46937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640262,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":637305,"data_compressed":5523,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5523,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5523,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T08:52:09.070 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-08T08:52:09.070 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-05-08T08:52:09.367 INFO:teuthology.orchestra.run.smithi059.stdout:epoch 17 2024-05-08T08:52:09.367 INFO:teuthology.orchestra.run.smithi059.stdout:fsid 38dfae8c-5c5e-4b0f-a264-91f22c6c5554 2024-05-08T08:52:09.367 INFO:teuthology.orchestra.run.smithi059.stdout:created 2024-05-08T08:51:45.571369+0000 2024-05-08T08:52:09.367 INFO:teuthology.orchestra.run.smithi059.stdout:modified 2024-05-08T08:52:04.884581+0000 2024-05-08T08:52:09.367 INFO:teuthology.orchestra.run.smithi059.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-05-08T08:52:09.367 INFO:teuthology.orchestra.run.smithi059.stdout:crush_version 4 2024-05-08T08:52:09.367 INFO:teuthology.orchestra.run.smithi059.stdout:full_ratio 0.95 2024-05-08T08:52:09.367 INFO:teuthology.orchestra.run.smithi059.stdout:backfillfull_ratio 0.9 2024-05-08T08:52:09.368 INFO:teuthology.orchestra.run.smithi059.stdout:nearfull_ratio 0.85 2024-05-08T08:52:09.368 INFO:teuthology.orchestra.run.smithi059.stdout:require_min_compat_client luminous 2024-05-08T08:52:09.368 INFO:teuthology.orchestra.run.smithi059.stdout:min_compat_client jewel 2024-05-08T08:52:09.368 INFO:teuthology.orchestra.run.smithi059.stdout:require_osd_release squid 2024-05-08T08:52:09.368 INFO:teuthology.orchestra.run.smithi059.stdout:stretch_mode_enabled false 2024-05-08T08:52:09.368 INFO:teuthology.orchestra.run.smithi059.stdout:pool 1 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 14 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 6.06 2024-05-08T08:52:09.368 INFO:teuthology.orchestra.run.smithi059.stdout:pool 2 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 17 flags hashpspool stripe_width 0 application rbd read_balance_score 2.25 2024-05-08T08:52:09.368 INFO:teuthology.orchestra.run.smithi059.stdout:max_osd 6 2024-05-08T08:52:09.368 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0 up in weight 1 up_from 11 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.59:6800/802160151,v1:172.21.15.59:6801/802160151] [v2:172.21.15.59:6802/802160151,v1:172.21.15.59:6803/802160151] exists,up 300801a5-1919-4b23-8c6d-bc7dad61cc4d 2024-05-08T08:52:09.368 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1 up in weight 1 up_from 11 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.59:6816/4116801011,v1:172.21.15.59:6817/4116801011] [v2:172.21.15.59:6818/4116801011,v1:172.21.15.59:6819/4116801011] exists,up 0fead0de-6a65-4eba-b400-eea5692702e0 2024-05-08T08:52:09.368 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2 up in weight 1 up_from 11 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.59:6808/1547661179,v1:172.21.15.59:6809/1547661179] [v2:172.21.15.59:6810/1547661179,v1:172.21.15.59:6811/1547661179] exists,up 1a233de6-1b82-4a51-8e96-4cd4abdf16c8 2024-05-08T08:52:09.368 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3 up in weight 1 up_from 11 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.192:6800/4054400810,v1:172.21.15.192:6801/4054400810] [v2:172.21.15.192:6802/4054400810,v1:172.21.15.192:6803/4054400810] exists,up cc4870b8-dcd9-4e43-978d-8de6c6396abf 2024-05-08T08:52:09.368 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4 up in weight 1 up_from 11 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.192:6808/58985565,v1:172.21.15.192:6809/58985565] [v2:172.21.15.192:6810/58985565,v1:172.21.15.192:6811/58985565] exists,up d1d05620-42fb-4107-a2f9-e83542d076ee 2024-05-08T08:52:09.369 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5 up in weight 1 up_from 11 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.192:6816/1283571573,v1:172.21.15.192:6817/1283571573] [v2:172.21.15.192:6818/1283571573,v1:172.21.15.192:6819/1283571573] exists,up ab26136c-31c3-4090-9fa2-988dba4f731f 2024-05-08T08:52:09.380 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 11 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.59:6800/802160151,v1:172.21.15.59:6801/802160151] [v2:172.21.15.59:6802/802160151,v1:172.21.15.59:6803/802160151] exists,up 300801a5-1919-4b23-8c6d-bc7dad61cc4d', 'osd.1 up in weight 1 up_from 11 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.59:6816/4116801011,v1:172.21.15.59:6817/4116801011] [v2:172.21.15.59:6818/4116801011,v1:172.21.15.59:6819/4116801011] exists,up 0fead0de-6a65-4eba-b400-eea5692702e0', 'osd.2 up in weight 1 up_from 11 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.59:6808/1547661179,v1:172.21.15.59:6809/1547661179] [v2:172.21.15.59:6810/1547661179,v1:172.21.15.59:6811/1547661179] exists,up 1a233de6-1b82-4a51-8e96-4cd4abdf16c8', 'osd.3 up in weight 1 up_from 11 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.192:6800/4054400810,v1:172.21.15.192:6801/4054400810] [v2:172.21.15.192:6802/4054400810,v1:172.21.15.192:6803/4054400810] exists,up cc4870b8-dcd9-4e43-978d-8de6c6396abf', 'osd.4 up in weight 1 up_from 11 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.192:6808/58985565,v1:172.21.15.192:6809/58985565] [v2:172.21.15.192:6810/58985565,v1:172.21.15.192:6811/58985565] exists,up d1d05620-42fb-4107-a2f9-e83542d076ee', 'osd.5 up in weight 1 up_from 11 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.192:6816/1283571573,v1:172.21.15.192:6817/1283571573] [v2:172.21.15.192:6818/1283571573,v1:172.21.15.192:6819/1283571573] exists,up ab26136c-31c3-4090-9fa2-988dba4f731f'] 2024-05-08T08:52:09.380 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-05-08T08:52:09.483 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:52:09.483 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "mon.a", 2024-05-08T08:52:09.484 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster": "ceph", 2024-05-08T08:52:09.484 INFO:teuthology.orchestra.run.smithi059.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-05-08T08:52:09.484 INFO:teuthology.orchestra.run.smithi059.stdout: "admin_socket_mode": "", 2024-05-08T08:52:09.484 INFO:teuthology.orchestra.run.smithi059.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-05-08T08:52:09.484 INFO:teuthology.orchestra.run.smithi059.stdout: "auth_client_required": "cephx, none", 2024-05-08T08:52:09.484 INFO:teuthology.orchestra.run.smithi059.stdout: "auth_cluster_required": "cephx", 2024-05-08T08:52:09.484 INFO:teuthology.orchestra.run.smithi059.stdout: "auth_debug": "true", 2024-05-08T08:52:09.484 INFO:teuthology.orchestra.run.smithi059.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-05-08T08:52:09.484 INFO:teuthology.orchestra.run.smithi059.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-05-08T08:52:09.484 INFO:teuthology.orchestra.run.smithi059.stdout: "auth_service_required": "cephx", 2024-05-08T08:52:09.484 INFO:teuthology.orchestra.run.smithi059.stdout: "auth_service_ticket_ttl": "240.000000", 2024-05-08T08:52:09.484 INFO:teuthology.orchestra.run.smithi059.stdout: "auth_supported": "cephx", 2024-05-08T08:52:09.484 INFO:teuthology.orchestra.run.smithi059.stdout: "bdev_aio": "true", 2024-05-08T08:52:09.484 INFO:teuthology.orchestra.run.smithi059.stdout: "bdev_aio_max_queue_depth": "1024", 2024-05-08T08:52:09.485 INFO:teuthology.orchestra.run.smithi059.stdout: "bdev_aio_poll_ms": "250", 2024-05-08T08:52:09.485 INFO:teuthology.orchestra.run.smithi059.stdout: "bdev_aio_reap_max": "16", 2024-05-08T08:52:09.485 INFO:teuthology.orchestra.run.smithi059.stdout: "bdev_async_discard": "false", 2024-05-08T08:52:09.485 INFO:teuthology.orchestra.run.smithi059.stdout: "bdev_block_size": "4096", 2024-05-08T08:52:09.485 INFO:teuthology.orchestra.run.smithi059.stdout: "bdev_debug_aio": "false", 2024-05-08T08:52:09.485 INFO:teuthology.orchestra.run.smithi059.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-05-08T08:52:09.485 INFO:teuthology.orchestra.run.smithi059.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-05-08T08:52:09.485 INFO:teuthology.orchestra.run.smithi059.stdout: "bdev_debug_inflight_ios": "false", 2024-05-08T08:52:09.485 INFO:teuthology.orchestra.run.smithi059.stdout: "bdev_enable_discard": "false", 2024-05-08T08:52:09.485 INFO:teuthology.orchestra.run.smithi059.stdout: "bdev_flock_retry": "3", 2024-05-08T08:52:09.485 INFO:teuthology.orchestra.run.smithi059.stdout: "bdev_flock_retry_interval": "0.100000", 2024-05-08T08:52:09.485 INFO:teuthology.orchestra.run.smithi059.stdout: "bdev_inject_crash": "0", 2024-05-08T08:52:09.485 INFO:teuthology.orchestra.run.smithi059.stdout: "bdev_inject_crash_flush_delay": "2", 2024-05-08T08:52:09.485 INFO:teuthology.orchestra.run.smithi059.stdout: "bdev_ioring": "false", 2024-05-08T08:52:09.486 INFO:teuthology.orchestra.run.smithi059.stdout: "bdev_ioring_hipri": "false", 2024-05-08T08:52:09.486 INFO:teuthology.orchestra.run.smithi059.stdout: "bdev_ioring_sqthread_poll": "false", 2024-05-08T08:52:09.486 INFO:teuthology.orchestra.run.smithi059.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-05-08T08:52:09.486 INFO:teuthology.orchestra.run.smithi059.stdout: "bdev_read_buffer_alignment": "4096", 2024-05-08T08:52:09.486 INFO:teuthology.orchestra.run.smithi059.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-05-08T08:52:09.486 INFO:teuthology.orchestra.run.smithi059.stdout: "bdev_type": "", 2024-05-08T08:52:09.486 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_alloc_size": "1048576", 2024-05-08T08:52:09.486 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_allocator": "hybrid", 2024-05-08T08:52:09.486 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_buffered_io": "true", 2024-05-08T08:52:09.486 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_check_for_zeros": "false", 2024-05-08T08:52:09.486 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_check_volume_selector_often": "false", 2024-05-08T08:52:09.486 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-05-08T08:52:09.486 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_compact_log_sync": "false", 2024-05-08T08:52:09.487 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-05-08T08:52:09.487 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-05-08T08:52:09.487 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_log_compact_min_size": "16777216", 2024-05-08T08:52:09.487 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_log_replay_check_allocations": "true", 2024-05-08T08:52:09.487 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_max_log_runway": "4194304", 2024-05-08T08:52:09.487 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_max_prefetch": "1048576", 2024-05-08T08:52:09.487 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_min_flush_size": "524288", 2024-05-08T08:52:09.487 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_min_log_runway": "1048576", 2024-05-08T08:52:09.487 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_replay_recovery": "false", 2024-05-08T08:52:09.487 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-05-08T08:52:09.487 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_shared_alloc_size": "65536", 2024-05-08T08:52:09.487 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_sync_write": "false", 2024-05-08T08:52:09.487 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-05-08T08:52:09.487 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-05-08T08:52:09.487 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-05-08T08:52:09.488 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_allocation_from_file": "true", 2024-05-08T08:52:09.488 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_allocator": "hybrid", 2024-05-08T08:52:09.488 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-05-08T08:52:09.488 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-05-08T08:52:09.488 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-05-08T08:52:09.488 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-05-08T08:52:09.488 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-05-08T08:52:09.488 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-05-08T08:52:09.488 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_blobid_prealloc": "10240", 2024-05-08T08:52:09.488 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_block_create": "true", 2024-05-08T08:52:09.488 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_block_db_create": "false", 2024-05-08T08:52:09.488 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_block_db_path": "", 2024-05-08T08:52:09.488 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_block_db_size": "0", 2024-05-08T08:52:09.488 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_block_path": "", 2024-05-08T08:52:09.489 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_block_preallocate_file": "false", 2024-05-08T08:52:09.489 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_block_size": "107374182400", 2024-05-08T08:52:09.489 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_block_wal_create": "false", 2024-05-08T08:52:09.489 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_block_wal_path": "", 2024-05-08T08:52:09.489 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_block_wal_size": "100663296", 2024-05-08T08:52:09.489 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_bluefs": "true", 2024-05-08T08:52:09.489 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-05-08T08:52:09.489 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_bluefs_env_mirror": "false", 2024-05-08T08:52:09.489 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-05-08T08:52:09.489 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-05-08T08:52:09.489 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-05-08T08:52:09.489 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-05-08T08:52:09.489 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-05-08T08:52:09.489 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-05-08T08:52:09.490 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_autotune": "true", 2024-05-08T08:52:09.490 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-05-08T08:52:09.490 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-05-08T08:52:09.490 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-05-08T08:52:09.490 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-05-08T08:52:09.490 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_size": "0", 2024-05-08T08:52:09.490 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-05-08T08:52:09.490 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-05-08T08:52:09.490 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-05-08T08:52:09.490 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-05-08T08:52:09.490 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_type": "2q", 2024-05-08T08:52:09.490 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-05-08T08:52:09.490 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_clone_cow": "true", 2024-05-08T08:52:09.490 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_compression_algorithm": "snappy", 2024-05-08T08:52:09.491 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_compression_max_blob_size": "0", 2024-05-08T08:52:09.491 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-05-08T08:52:09.491 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-05-08T08:52:09.491 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_compression_min_blob_size": "0", 2024-05-08T08:52:09.491 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-05-08T08:52:09.491 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-05-08T08:52:09.491 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_compression_mode": "none", 2024-05-08T08:52:09.491 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-05-08T08:52:09.491 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_csum_type": "crc32c", 2024-05-08T08:52:09.491 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_debug_enforce_settings": "default", 2024-05-08T08:52:09.491 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_debug_freelist": "false", 2024-05-08T08:52:09.491 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_debug_fsck_abort": "false", 2024-05-08T08:52:09.491 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-05-08T08:52:09.491 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-05-08T08:52:09.491 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_debug_inject_read_err": "false", 2024-05-08T08:52:09.492 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_debug_legacy_omap": "false", 2024-05-08T08:52:09.492 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-05-08T08:52:09.492 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-05-08T08:52:09.492 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-05-08T08:52:09.492 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-05-08T08:52:09.492 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_debug_prefill": "0.000000", 2024-05-08T08:52:09.492 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-05-08T08:52:09.492 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-05-08T08:52:09.492 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-05-08T08:52:09.492 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_debug_small_allocations": "0", 2024-05-08T08:52:09.492 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-05-08T08:52:09.492 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_default_buffered_read": "true", 2024-05-08T08:52:09.492 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_default_buffered_write": "false", 2024-05-08T08:52:09.492 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_deferred_batch_ops": "0", 2024-05-08T08:52:09.493 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-05-08T08:52:09.493 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-05-08T08:52:09.493 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_elastic_shared_blobs": "true", 2024-05-08T08:52:09.493 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-05-08T08:52:09.493 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-05-08T08:52:09.493 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-05-08T08:52:09.493 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-05-08T08:52:09.493 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-05-08T08:52:09.493 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_fail_eio": "false", 2024-05-08T08:52:09.493 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-05-08T08:52:09.493 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-05-08T08:52:09.493 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-05-08T08:52:09.493 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-05-08T08:52:09.493 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_fsck_on_mkfs": "true", 2024-05-08T08:52:09.494 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-05-08T08:52:09.494 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_fsck_on_mount": "false", 2024-05-08T08:52:09.494 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-05-08T08:52:09.494 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_fsck_on_umount": "false", 2024-05-08T08:52:09.494 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-05-08T08:52:09.494 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-05-08T08:52:09.494 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-05-08T08:52:09.494 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-05-08T08:52:09.494 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-05-08T08:52:09.494 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-05-08T08:52:09.494 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-05-08T08:52:09.494 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-05-08T08:52:09.494 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_ignore_data_csum": "false", 2024-05-08T08:52:09.494 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-05-08T08:52:09.495 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_kvbackend": "rocksdb", 2024-05-08T08:52:09.495 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-05-08T08:52:09.495 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-05-08T08:52:09.495 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_log_op_age": "5.000000", 2024-05-08T08:52:09.495 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_max_alloc_size": "0", 2024-05-08T08:52:09.495 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_max_blob_size": "0", 2024-05-08T08:52:09.495 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-05-08T08:52:09.495 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-05-08T08:52:09.495 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_max_defer_interval": "3.000000", 2024-05-08T08:52:09.495 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_max_deferred_txc": "32", 2024-05-08T08:52:09.495 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_min_alloc_size": "0", 2024-05-08T08:52:09.495 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-05-08T08:52:09.495 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-05-08T08:52:09.495 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_nid_prealloc": "1024", 2024-05-08T08:52:09.496 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_prefer_deferred_size": "0", 2024-05-08T08:52:09.496 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-05-08T08:52:09.496 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-05-08T08:52:09.496 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_qfsck_on_mount": "true", 2024-05-08T08:52:09.496 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_retry_disk_reads": "3", 2024-05-08T08:52:09.496 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_rocksdb_cf": "true", 2024-05-08T08:52:09.496 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-05-08T08:52:09.496 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-05-08T08:52:09.496 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_rocksdb_options_annex": "", 2024-05-08T08:52:09.496 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_spdk_coremask": "0x1", 2024-05-08T08:52:09.496 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_spdk_io_sleep": "5", 2024-05-08T08:52:09.496 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_spdk_max_io_completion": "0", 2024-05-08T08:52:09.496 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_spdk_mem": "512", 2024-05-08T08:52:09.496 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_sync_submit_transaction": "false", 2024-05-08T08:52:09.496 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_throttle_bytes": "67108864", 2024-05-08T08:52:09.497 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_throttle_cost_per_io": "0", 2024-05-08T08:52:09.497 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-05-08T08:52:09.497 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-05-08T08:52:09.497 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-05-08T08:52:09.497 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-05-08T08:52:09.497 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_tracing": "false", 2024-05-08T08:52:09.497 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-05-08T08:52:09.497 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-05-08T08:52:09.497 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_volume_selection_reserved": "0", 2024-05-08T08:52:09.497 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-05-08T08:52:09.497 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-05-08T08:52:09.497 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-05-08T08:52:09.497 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-05-08T08:52:09.497 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-05-08T08:52:09.497 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-05-08T08:52:09.498 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_zero_block_detection": "false", 2024-05-08T08:52:09.498 INFO:teuthology.orchestra.run.smithi059.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-05-08T08:52:09.498 INFO:teuthology.orchestra.run.smithi059.stdout: "cephfs_mirror_action_update_interval": "2", 2024-05-08T08:52:09.498 INFO:teuthology.orchestra.run.smithi059.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-05-08T08:52:09.498 INFO:teuthology.orchestra.run.smithi059.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-05-08T08:52:09.498 INFO:teuthology.orchestra.run.smithi059.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-05-08T08:52:09.498 INFO:teuthology.orchestra.run.smithi059.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-05-08T08:52:09.498 INFO:teuthology.orchestra.run.smithi059.stdout: "cephfs_mirror_mount_timeout": "10", 2024-05-08T08:52:09.498 INFO:teuthology.orchestra.run.smithi059.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-05-08T08:52:09.498 INFO:teuthology.orchestra.run.smithi059.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-05-08T08:52:09.498 INFO:teuthology.orchestra.run.smithi059.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-05-08T08:52:09.498 INFO:teuthology.orchestra.run.smithi059.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-05-08T08:52:09.498 INFO:teuthology.orchestra.run.smithi059.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-05-08T08:52:09.499 INFO:teuthology.orchestra.run.smithi059.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-05-08T08:52:09.499 INFO:teuthology.orchestra.run.smithi059.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-05-08T08:52:09.499 INFO:teuthology.orchestra.run.smithi059.stdout: "cephx_cluster_require_signatures": "false", 2024-05-08T08:52:09.499 INFO:teuthology.orchestra.run.smithi059.stdout: "cephx_cluster_require_version": "2", 2024-05-08T08:52:09.499 INFO:teuthology.orchestra.run.smithi059.stdout: "cephx_require_signatures": "false", 2024-05-08T08:52:09.499 INFO:teuthology.orchestra.run.smithi059.stdout: "cephx_require_version": "2", 2024-05-08T08:52:09.499 INFO:teuthology.orchestra.run.smithi059.stdout: "cephx_service_require_signatures": "false", 2024-05-08T08:52:09.499 INFO:teuthology.orchestra.run.smithi059.stdout: "cephx_service_require_version": "2", 2024-05-08T08:52:09.499 INFO:teuthology.orchestra.run.smithi059.stdout: "cephx_sign_messages": "true", 2024-05-08T08:52:09.499 INFO:teuthology.orchestra.run.smithi059.stdout: "chdir": "", 2024-05-08T08:52:09.499 INFO:teuthology.orchestra.run.smithi059.stdout: "client_acl_type": "", 2024-05-08T08:52:09.499 INFO:teuthology.orchestra.run.smithi059.stdout: "client_asio_thread_count": "2", 2024-05-08T08:52:09.499 INFO:teuthology.orchestra.run.smithi059.stdout: "client_cache_mid": "0.750000", 2024-05-08T08:52:09.499 INFO:teuthology.orchestra.run.smithi059.stdout: "client_cache_size": "16384", 2024-05-08T08:52:09.500 INFO:teuthology.orchestra.run.smithi059.stdout: "client_caps_release_delay": "5", 2024-05-08T08:52:09.500 INFO:teuthology.orchestra.run.smithi059.stdout: "client_check_pool_perm": "true", 2024-05-08T08:52:09.500 INFO:teuthology.orchestra.run.smithi059.stdout: "client_collect_and_send_global_metrics": "false", 2024-05-08T08:52:09.500 INFO:teuthology.orchestra.run.smithi059.stdout: "client_debug_force_sync_read": "false", 2024-05-08T08:52:09.500 INFO:teuthology.orchestra.run.smithi059.stdout: "client_debug_getattr_caps": "false", 2024-05-08T08:52:09.500 INFO:teuthology.orchestra.run.smithi059.stdout: "client_debug_inject_tick_delay": "0", 2024-05-08T08:52:09.500 INFO:teuthology.orchestra.run.smithi059.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-05-08T08:52:09.500 INFO:teuthology.orchestra.run.smithi059.stdout: "client_die_on_failed_remount": "false", 2024-05-08T08:52:09.500 INFO:teuthology.orchestra.run.smithi059.stdout: "client_dirsize_rbytes": "true", 2024-05-08T08:52:09.500 INFO:teuthology.orchestra.run.smithi059.stdout: "client_force_lazyio": "false", 2024-05-08T08:52:09.500 INFO:teuthology.orchestra.run.smithi059.stdout: "client_fs": "", 2024-05-08T08:52:09.500 INFO:teuthology.orchestra.run.smithi059.stdout: "client_inject_fixed_oldest_tid": "false", 2024-05-08T08:52:09.500 INFO:teuthology.orchestra.run.smithi059.stdout: "client_inject_release_failure": "false", 2024-05-08T08:52:09.500 INFO:teuthology.orchestra.run.smithi059.stdout: "client_max_inline_size": "4096", 2024-05-08T08:52:09.501 INFO:teuthology.orchestra.run.smithi059.stdout: "client_max_retries_on_remount_failure": "5", 2024-05-08T08:52:09.501 INFO:teuthology.orchestra.run.smithi059.stdout: "client_mds_namespace": "", 2024-05-08T08:52:09.501 INFO:teuthology.orchestra.run.smithi059.stdout: "client_metadata": "", 2024-05-08T08:52:09.501 INFO:teuthology.orchestra.run.smithi059.stdout: "client_mount_gid": "-1", 2024-05-08T08:52:09.501 INFO:teuthology.orchestra.run.smithi059.stdout: "client_mount_timeout": "300", 2024-05-08T08:52:09.501 INFO:teuthology.orchestra.run.smithi059.stdout: "client_mount_uid": "-1", 2024-05-08T08:52:09.501 INFO:teuthology.orchestra.run.smithi059.stdout: "client_mountpoint": "/", 2024-05-08T08:52:09.501 INFO:teuthology.orchestra.run.smithi059.stdout: "client_notify_timeout": "10", 2024-05-08T08:52:09.501 INFO:teuthology.orchestra.run.smithi059.stdout: "client_oc": "true", 2024-05-08T08:52:09.501 INFO:teuthology.orchestra.run.smithi059.stdout: "client_oc_max_dirty": "104857600", 2024-05-08T08:52:09.501 INFO:teuthology.orchestra.run.smithi059.stdout: "client_oc_max_dirty_age": "5.000000", 2024-05-08T08:52:09.501 INFO:teuthology.orchestra.run.smithi059.stdout: "client_oc_max_objects": "1000", 2024-05-08T08:52:09.501 INFO:teuthology.orchestra.run.smithi059.stdout: "client_oc_size": "209715200", 2024-05-08T08:52:09.501 INFO:teuthology.orchestra.run.smithi059.stdout: "client_oc_target_dirty": "8388608", 2024-05-08T08:52:09.501 INFO:teuthology.orchestra.run.smithi059.stdout: "client_permissions": "true", 2024-05-08T08:52:09.502 INFO:teuthology.orchestra.run.smithi059.stdout: "client_quota": "true", 2024-05-08T08:52:09.502 INFO:teuthology.orchestra.run.smithi059.stdout: "client_quota_df": "true", 2024-05-08T08:52:09.502 INFO:teuthology.orchestra.run.smithi059.stdout: "client_readahead_max_bytes": "0", 2024-05-08T08:52:09.502 INFO:teuthology.orchestra.run.smithi059.stdout: "client_readahead_max_periods": "4", 2024-05-08T08:52:09.502 INFO:teuthology.orchestra.run.smithi059.stdout: "client_readahead_min": "131072", 2024-05-08T08:52:09.502 INFO:teuthology.orchestra.run.smithi059.stdout: "client_reconnect_stale": "false", 2024-05-08T08:52:09.502 INFO:teuthology.orchestra.run.smithi059.stdout: "client_shutdown_timeout": "30", 2024-05-08T08:52:09.502 INFO:teuthology.orchestra.run.smithi059.stdout: "client_snapdir": ".snap", 2024-05-08T08:52:09.502 INFO:teuthology.orchestra.run.smithi059.stdout: "client_tick_interval": "1", 2024-05-08T08:52:09.502 INFO:teuthology.orchestra.run.smithi059.stdout: "client_trace": "", 2024-05-08T08:52:09.502 INFO:teuthology.orchestra.run.smithi059.stdout: "client_try_dentry_invalidate": "false", 2024-05-08T08:52:09.502 INFO:teuthology.orchestra.run.smithi059.stdout: "client_use_faked_inos": "false", 2024-05-08T08:52:09.502 INFO:teuthology.orchestra.run.smithi059.stdout: "client_use_random_mds": "false", 2024-05-08T08:52:09.503 INFO:teuthology.orchestra.run.smithi059.stdout: "clog_to_graylog": "false", 2024-05-08T08:52:09.503 INFO:teuthology.orchestra.run.smithi059.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-05-08T08:52:09.503 INFO:teuthology.orchestra.run.smithi059.stdout: "clog_to_graylog_port": "12201", 2024-05-08T08:52:09.503 INFO:teuthology.orchestra.run.smithi059.stdout: "clog_to_monitors": "default=true", 2024-05-08T08:52:09.503 INFO:teuthology.orchestra.run.smithi059.stdout: "clog_to_syslog": "false", 2024-05-08T08:52:09.503 INFO:teuthology.orchestra.run.smithi059.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-05-08T08:52:09.503 INFO:teuthology.orchestra.run.smithi059.stdout: "clog_to_syslog_level": "info", 2024-05-08T08:52:09.503 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "-", 2024-05-08T08:52:09.503 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_network": "", 2024-05-08T08:52:09.503 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_network_interface": "", 2024-05-08T08:52:09.503 INFO:teuthology.orchestra.run.smithi059.stdout: "compressor_zlib_isal": "false", 2024-05-08T08:52:09.503 INFO:teuthology.orchestra.run.smithi059.stdout: "compressor_zlib_level": "5", 2024-05-08T08:52:09.503 INFO:teuthology.orchestra.run.smithi059.stdout: "compressor_zlib_winsize": "-15", 2024-05-08T08:52:09.503 INFO:teuthology.orchestra.run.smithi059.stdout: "compressor_zstd_level": "1", 2024-05-08T08:52:09.503 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-05-08T08:52:09.504 INFO:teuthology.orchestra.run.smithi059.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-05-08T08:52:09.504 INFO:teuthology.orchestra.run.smithi059.stdout: "crimson_alien_op_num_threads": "6", 2024-05-08T08:52:09.504 INFO:teuthology.orchestra.run.smithi059.stdout: "crimson_alien_thread_cpu_cores": "", 2024-05-08T08:52:09.504 INFO:teuthology.orchestra.run.smithi059.stdout: "crimson_osd_obc_lru_size": "512", 2024-05-08T08:52:09.504 INFO:teuthology.orchestra.run.smithi059.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-05-08T08:52:09.504 INFO:teuthology.orchestra.run.smithi059.stdout: "crimson_seastar_cpu_cores": "", 2024-05-08T08:52:09.504 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_location": "", 2024-05-08T08:52:09.504 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_location_hook": "", 2024-05-08T08:52:09.504 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_location_hook_timeout": "10", 2024-05-08T08:52:09.504 INFO:teuthology.orchestra.run.smithi059.stdout: "daemonize": "false", 2024-05-08T08:52:09.504 INFO:teuthology.orchestra.run.smithi059.stdout: "daos_pool": "tank", 2024-05-08T08:52:09.504 INFO:teuthology.orchestra.run.smithi059.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-05-08T08:52:09.504 INFO:teuthology.orchestra.run.smithi059.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-05-08T08:52:09.505 INFO:teuthology.orchestra.run.smithi059.stdout: "dbstore_db_name_prefix": "dbstore", 2024-05-08T08:52:09.505 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_alienstore": "0/5", 2024-05-08T08:52:09.505 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_allow_any_pool_priority": "false", 2024-05-08T08:52:09.505 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_asok": "1/5", 2024-05-08T08:52:09.505 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_asok_assert_abort": "false", 2024-05-08T08:52:09.505 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_asserts_on_shutdown": "true", 2024-05-08T08:52:09.505 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_auth": "20/20", 2024-05-08T08:52:09.505 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_bdev": "1/3", 2024-05-08T08:52:09.505 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_bluefs": "1/5", 2024-05-08T08:52:09.505 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_bluestore": "1/5", 2024-05-08T08:52:09.505 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_buffer": "0/1", 2024-05-08T08:52:09.505 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_ceph_exporter": "1/5", 2024-05-08T08:52:09.505 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_cephfs_mirror": "0/5", 2024-05-08T08:52:09.505 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_cephsqlite": "0/5", 2024-05-08T08:52:09.505 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_client": "0/5", 2024-05-08T08:52:09.505 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_compressor": "1/5", 2024-05-08T08:52:09.506 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_context": "0/1", 2024-05-08T08:52:09.506 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_crimson_interrupt": "0/5", 2024-05-08T08:52:09.506 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_crush": "1/1", 2024-05-08T08:52:09.506 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_crypto": "1/5", 2024-05-08T08:52:09.506 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_cyanstore": "0/5", 2024-05-08T08:52:09.506 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_deliberately_leak_memory": "false", 2024-05-08T08:52:09.506 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_disable_randomized_ping": "false", 2024-05-08T08:52:09.506 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_dpdk": "1/5", 2024-05-08T08:52:09.506 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_eventtrace": "1/5", 2024-05-08T08:52:09.506 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_filer": "0/1", 2024-05-08T08:52:09.506 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_filestore": "1/3", 2024-05-08T08:52:09.506 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_finisher": "1/1", 2024-05-08T08:52:09.506 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_fuse": "1/5", 2024-05-08T08:52:09.506 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_heartbeat_testing_span": "0", 2024-05-08T08:52:09.507 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_heartbeatmap": "1/5", 2024-05-08T08:52:09.507 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_immutable_obj_cache": "0/5", 2024-05-08T08:52:09.507 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_javaclient": "1/5", 2024-05-08T08:52:09.507 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_journal": "1/3", 2024-05-08T08:52:09.507 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_journaler": "0/5", 2024-05-08T08:52:09.507 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_kstore": "1/5", 2024-05-08T08:52:09.507 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_lockdep": "0/1", 2024-05-08T08:52:09.507 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_mclock": "1/5", 2024-05-08T08:52:09.507 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_mds": "1/5", 2024-05-08T08:52:09.507 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_mds_balancer": "1/5", 2024-05-08T08:52:09.507 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_mds_locker": "1/5", 2024-05-08T08:52:09.507 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_mds_log": "1/5", 2024-05-08T08:52:09.507 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_mds_log_expire": "1/5", 2024-05-08T08:52:09.507 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_mds_migrator": "1/5", 2024-05-08T08:52:09.507 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_mds_quiesce": "3/5", 2024-05-08T08:52:09.508 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_memstore": "1/5", 2024-05-08T08:52:09.508 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_mgr": "2/5", 2024-05-08T08:52:09.508 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_mgrc": "1/5", 2024-05-08T08:52:09.508 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_mon": "20/20", 2024-05-08T08:52:09.508 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_monc": "0/10", 2024-05-08T08:52:09.508 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_ms": "1/1", 2024-05-08T08:52:09.508 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_none": "0/5", 2024-05-08T08:52:09.508 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_objclass": "0/5", 2024-05-08T08:52:09.508 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_objectcacher": "0/5", 2024-05-08T08:52:09.508 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_objecter": "0/1", 2024-05-08T08:52:09.508 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_optracker": "0/5", 2024-05-08T08:52:09.508 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_osd": "1/5", 2024-05-08T08:52:09.508 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_paxos": "20/20", 2024-05-08T08:52:09.508 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_perfcounter": "1/5", 2024-05-08T08:52:09.508 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_prioritycache": "1/5", 2024-05-08T08:52:09.508 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_rados": "0/5", 2024-05-08T08:52:09.509 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_rbd": "0/5", 2024-05-08T08:52:09.509 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_rbd_mirror": "0/5", 2024-05-08T08:52:09.509 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_rbd_pwl": "0/5", 2024-05-08T08:52:09.509 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_rbd_replay": "0/5", 2024-05-08T08:52:09.509 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_refs": "0/0", 2024-05-08T08:52:09.509 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_reserver": "1/1", 2024-05-08T08:52:09.509 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_rgw": "1/5", 2024-05-08T08:52:09.509 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_rgw_access": "1/5", 2024-05-08T08:52:09.509 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_rgw_datacache": "1/5", 2024-05-08T08:52:09.509 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_rgw_dbstore": "1/5", 2024-05-08T08:52:09.509 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_rgw_flight": "1/5", 2024-05-08T08:52:09.509 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_rgw_lifecycle": "1/5", 2024-05-08T08:52:09.509 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_rgw_sync": "1/5", 2024-05-08T08:52:09.509 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_rocksdb": "4/5", 2024-05-08T08:52:09.509 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_seastore": "0/5", 2024-05-08T08:52:09.510 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_seastore_backref": "0/5", 2024-05-08T08:52:09.510 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_seastore_cache": "0/5", 2024-05-08T08:52:09.510 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_seastore_cleaner": "0/5", 2024-05-08T08:52:09.510 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_seastore_device": "0/5", 2024-05-08T08:52:09.510 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_seastore_epm": "0/5", 2024-05-08T08:52:09.510 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-05-08T08:52:09.510 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_seastore_journal": "0/5", 2024-05-08T08:52:09.510 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_seastore_lba": "0/5", 2024-05-08T08:52:09.510 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_seastore_odata": "0/5", 2024-05-08T08:52:09.510 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_seastore_omap": "0/5", 2024-05-08T08:52:09.510 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_seastore_onode": "0/5", 2024-05-08T08:52:09.510 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_seastore_t": "0/5", 2024-05-08T08:52:09.510 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_seastore_tm": "0/5", 2024-05-08T08:52:09.510 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_striper": "0/1", 2024-05-08T08:52:09.511 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_test": "0/5", 2024-05-08T08:52:09.511 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_throttle": "1/1", 2024-05-08T08:52:09.511 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_timer": "0/1", 2024-05-08T08:52:09.511 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_tp": "0/5", 2024-05-08T08:52:09.511 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_trace": "1/5", 2024-05-08T08:52:09.511 INFO:teuthology.orchestra.run.smithi059.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-05-08T08:52:09.511 INFO:teuthology.orchestra.run.smithi059.stdout: "device_failure_prediction_mode": "none", 2024-05-08T08:52:09.511 INFO:teuthology.orchestra.run.smithi059.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-05-08T08:52:09.511 INFO:teuthology.orchestra.run.smithi059.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2024-05-08T08:52:09.511 INFO:teuthology.orchestra.run.smithi059.stdout: "err_to_graylog": "false", 2024-05-08T08:52:09.511 INFO:teuthology.orchestra.run.smithi059.stdout: "err_to_journald": "false", 2024-05-08T08:52:09.511 INFO:teuthology.orchestra.run.smithi059.stdout: "err_to_stderr": "true", 2024-05-08T08:52:09.511 INFO:teuthology.orchestra.run.smithi059.stdout: "err_to_syslog": "false", 2024-05-08T08:52:09.511 INFO:teuthology.orchestra.run.smithi059.stdout: "event_tracing": "false", 2024-05-08T08:52:09.511 INFO:teuthology.orchestra.run.smithi059.stdout: "exporter_addr": "0.0.0.0", 2024-05-08T08:52:09.512 INFO:teuthology.orchestra.run.smithi059.stdout: "exporter_http_port": "9926", 2024-05-08T08:52:09.512 INFO:teuthology.orchestra.run.smithi059.stdout: "exporter_prio_limit": "5", 2024-05-08T08:52:09.512 INFO:teuthology.orchestra.run.smithi059.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-05-08T08:52:09.512 INFO:teuthology.orchestra.run.smithi059.stdout: "exporter_sort_metrics": "true", 2024-05-08T08:52:09.512 INFO:teuthology.orchestra.run.smithi059.stdout: "exporter_stats_period": "5", 2024-05-08T08:52:09.512 INFO:teuthology.orchestra.run.smithi059.stdout: "fake_statfs_for_testing": "0", 2024-05-08T08:52:09.512 INFO:teuthology.orchestra.run.smithi059.stdout: "fatal_signal_handlers": "true", 2024-05-08T08:52:09.512 INFO:teuthology.orchestra.run.smithi059.stdout: "filer_max_purge_ops": "10", 2024-05-08T08:52:09.512 INFO:teuthology.orchestra.run.smithi059.stdout: "filer_max_truncate_ops": "128", 2024-05-08T08:52:09.512 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_apply_finisher_threads": "1", 2024-05-08T08:52:09.512 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_blackhole": "false", 2024-05-08T08:52:09.512 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_btrfs_clone_range": "true", 2024-05-08T08:52:09.512 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_btrfs_snap": "true", 2024-05-08T08:52:09.512 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_caller_concurrency": "10", 2024-05-08T08:52:09.513 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_collect_device_partition_information": "true", 2024-05-08T08:52:09.513 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_commit_timeout": "600.000000", 2024-05-08T08:52:09.513 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_debug_inject_read_err": "false", 2024-05-08T08:52:09.513 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_debug_omap_check": "false", 2024-05-08T08:52:09.513 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_debug_random_read_err": "0.000000", 2024-05-08T08:52:09.513 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_debug_verify_split": "false", 2024-05-08T08:52:09.513 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_dump_file": "", 2024-05-08T08:52:09.513 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-05-08T08:52:09.513 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-05-08T08:52:09.513 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_fadvise": "true", 2024-05-08T08:52:09.513 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_fail_eio": "true", 2024-05-08T08:52:09.513 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_fd_cache_shards": "16", 2024-05-08T08:52:09.513 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_fd_cache_size": "128", 2024-05-08T08:52:09.513 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_fiemap": "false", 2024-05-08T08:52:09.513 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_fiemap_threshold": "4096", 2024-05-08T08:52:09.514 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-05-08T08:52:09.514 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_index_retry_probability": "0.000000", 2024-05-08T08:52:09.514 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_inject_stall": "0", 2024-05-08T08:52:09.514 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_journal_parallel": "false", 2024-05-08T08:52:09.514 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_journal_trailing": "false", 2024-05-08T08:52:09.514 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_journal_writeahead": "false", 2024-05-08T08:52:09.514 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_kill_at": "0", 2024-05-08T08:52:09.514 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-05-08T08:52:09.514 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_max_inline_xattr_size": "0", 2024-05-08T08:52:09.514 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-05-08T08:52:09.514 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-05-08T08:52:09.514 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-05-08T08:52:09.514 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_max_inline_xattrs": "0", 2024-05-08T08:52:09.514 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-05-08T08:52:09.514 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_max_inline_xattrs_other": "2", 2024-05-08T08:52:09.515 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-05-08T08:52:09.515 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_max_sync_interval": "5.000000", 2024-05-08T08:52:09.515 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_max_xattr_value_size": "0", 2024-05-08T08:52:09.515 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-05-08T08:52:09.515 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-05-08T08:52:09.515 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-05-08T08:52:09.515 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_merge_threshold": "-10", 2024-05-08T08:52:09.515 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_min_sync_interval": "0.010000", 2024-05-08T08:52:09.515 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_odsync_write": "false", 2024-05-08T08:52:09.515 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_omap_backend": "rocksdb", 2024-05-08T08:52:09.515 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_omap_backend_path": "", 2024-05-08T08:52:09.515 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_omap_header_cache_size": "1024", 2024-05-08T08:52:09.515 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_ondisk_finisher_threads": "1", 2024-05-08T08:52:09.515 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-05-08T08:52:09.516 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_op_thread_timeout": "60", 2024-05-08T08:52:09.516 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_op_threads": "2", 2024-05-08T08:52:09.516 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_punch_hole": "false", 2024-05-08T08:52:09.516 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-05-08T08:52:09.516 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-05-08T08:52:09.516 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-05-08T08:52:09.516 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-05-08T08:52:09.516 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-05-08T08:52:09.516 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_queue_max_bytes": "104857600", 2024-05-08T08:52:09.516 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-05-08T08:52:09.516 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-05-08T08:52:09.516 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-05-08T08:52:09.516 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_queue_max_ops": "50", 2024-05-08T08:52:09.516 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-05-08T08:52:09.517 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_seek_data_hole": "false", 2024-05-08T08:52:09.517 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_sloppy_crc": "false", 2024-05-08T08:52:09.517 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-05-08T08:52:09.517 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_splice": "false", 2024-05-08T08:52:09.517 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_split_multiple": "2", 2024-05-08T08:52:09.517 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_split_rand_factor": "20", 2024-05-08T08:52:09.517 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_update_to": "1000", 2024-05-08T08:52:09.517 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-05-08T08:52:09.517 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-05-08T08:52:09.517 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-05-08T08:52:09.517 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-05-08T08:52:09.517 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-05-08T08:52:09.517 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-05-08T08:52:09.517 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_wbthrottle_enable": "true", 2024-05-08T08:52:09.518 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-05-08T08:52:09.518 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-05-08T08:52:09.518 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-05-08T08:52:09.518 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-05-08T08:52:09.518 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-05-08T08:52:09.518 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-05-08T08:52:09.518 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_xfs_extsize": "false", 2024-05-08T08:52:09.518 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_zfs_snap": "false", 2024-05-08T08:52:09.518 INFO:teuthology.orchestra.run.smithi059.stdout: "fio_dir": "/tmp/fio", 2024-05-08T08:52:09.518 INFO:teuthology.orchestra.run.smithi059.stdout: "fsid": "38dfae8c-5c5e-4b0f-a264-91f22c6c5554", 2024-05-08T08:52:09.518 INFO:teuthology.orchestra.run.smithi059.stdout: "fuse_allow_other": "true", 2024-05-08T08:52:09.518 INFO:teuthology.orchestra.run.smithi059.stdout: "fuse_atomic_o_trunc": "true", 2024-05-08T08:52:09.518 INFO:teuthology.orchestra.run.smithi059.stdout: "fuse_big_writes": "true", 2024-05-08T08:52:09.518 INFO:teuthology.orchestra.run.smithi059.stdout: "fuse_debug": "false", 2024-05-08T08:52:09.519 INFO:teuthology.orchestra.run.smithi059.stdout: "fuse_default_permissions": "false", 2024-05-08T08:52:09.519 INFO:teuthology.orchestra.run.smithi059.stdout: "fuse_disable_pagecache": "false", 2024-05-08T08:52:09.519 INFO:teuthology.orchestra.run.smithi059.stdout: "fuse_max_write": "0", 2024-05-08T08:52:09.519 INFO:teuthology.orchestra.run.smithi059.stdout: "fuse_multithreaded": "true", 2024-05-08T08:52:09.519 INFO:teuthology.orchestra.run.smithi059.stdout: "fuse_require_active_mds": "true", 2024-05-08T08:52:09.519 INFO:teuthology.orchestra.run.smithi059.stdout: "fuse_set_user_groups": "true", 2024-05-08T08:52:09.519 INFO:teuthology.orchestra.run.smithi059.stdout: "fuse_splice_move": "true", 2024-05-08T08:52:09.519 INFO:teuthology.orchestra.run.smithi059.stdout: "fuse_splice_read": "true", 2024-05-08T08:52:09.519 INFO:teuthology.orchestra.run.smithi059.stdout: "fuse_splice_write": "true", 2024-05-08T08:52:09.519 INFO:teuthology.orchestra.run.smithi059.stdout: "fuse_syncfs_on_mksnap": "true", 2024-05-08T08:52:09.519 INFO:teuthology.orchestra.run.smithi059.stdout: "fuse_use_invalidate_cb": "true", 2024-05-08T08:52:09.519 INFO:teuthology.orchestra.run.smithi059.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-05-08T08:52:09.519 INFO:teuthology.orchestra.run.smithi059.stdout: "gss_target_name": "ceph", 2024-05-08T08:52:09.519 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_file": "", 2024-05-08T08:52:09.519 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_inject_failure": "0", 2024-05-08T08:52:09.520 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_interval": "5", 2024-05-08T08:52:09.520 INFO:teuthology.orchestra.run.smithi059.stdout: "host": "smithi059", 2024-05-08T08:52:09.520 INFO:teuthology.orchestra.run.smithi059.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-05-08T08:52:09.520 INFO:teuthology.orchestra.run.smithi059.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-05-08T08:52:09.520 INFO:teuthology.orchestra.run.smithi059.stdout: "immutable_object_cache_max_size": "1073741824", 2024-05-08T08:52:09.520 INFO:teuthology.orchestra.run.smithi059.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-05-08T08:52:09.520 INFO:teuthology.orchestra.run.smithi059.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-05-08T08:52:09.520 INFO:teuthology.orchestra.run.smithi059.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-05-08T08:52:09.520 INFO:teuthology.orchestra.run.smithi059.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-05-08T08:52:09.520 INFO:teuthology.orchestra.run.smithi059.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-05-08T08:52:09.520 INFO:teuthology.orchestra.run.smithi059.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-05-08T08:52:09.520 INFO:teuthology.orchestra.run.smithi059.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-05-08T08:52:09.520 INFO:teuthology.orchestra.run.smithi059.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-05-08T08:52:09.520 INFO:teuthology.orchestra.run.smithi059.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-05-08T08:52:09.521 INFO:teuthology.orchestra.run.smithi059.stdout: "immutable_object_cache_watermark": "0.900000", 2024-05-08T08:52:09.521 INFO:teuthology.orchestra.run.smithi059.stdout: "inject_early_sigterm": "false", 2024-05-08T08:52:09.521 INFO:teuthology.orchestra.run.smithi059.stdout: "jaeger_agent_port": "6799", 2024-05-08T08:52:09.521 INFO:teuthology.orchestra.run.smithi059.stdout: "jaeger_tracing_enable": "false", 2024-05-08T08:52:09.521 INFO:teuthology.orchestra.run.smithi059.stdout: "journal_aio": "true", 2024-05-08T08:52:09.521 INFO:teuthology.orchestra.run.smithi059.stdout: "journal_align_min_size": "65536", 2024-05-08T08:52:09.521 INFO:teuthology.orchestra.run.smithi059.stdout: "journal_block_align": "true", 2024-05-08T08:52:09.521 INFO:teuthology.orchestra.run.smithi059.stdout: "journal_block_size": "4096", 2024-05-08T08:52:09.521 INFO:teuthology.orchestra.run.smithi059.stdout: "journal_dio": "true", 2024-05-08T08:52:09.521 INFO:teuthology.orchestra.run.smithi059.stdout: "journal_discard": "false", 2024-05-08T08:52:09.521 INFO:teuthology.orchestra.run.smithi059.stdout: "journal_force_aio": "false", 2024-05-08T08:52:09.521 INFO:teuthology.orchestra.run.smithi059.stdout: "journal_ignore_corruption": "false", 2024-05-08T08:52:09.521 INFO:teuthology.orchestra.run.smithi059.stdout: "journal_max_write_bytes": "10485760", 2024-05-08T08:52:09.521 INFO:teuthology.orchestra.run.smithi059.stdout: "journal_max_write_entries": "100", 2024-05-08T08:52:09.521 INFO:teuthology.orchestra.run.smithi059.stdout: "journal_replay_from": "0", 2024-05-08T08:52:09.522 INFO:teuthology.orchestra.run.smithi059.stdout: "journal_throttle_high_multiple": "0.000000", 2024-05-08T08:52:09.522 INFO:teuthology.orchestra.run.smithi059.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-05-08T08:52:09.522 INFO:teuthology.orchestra.run.smithi059.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-05-08T08:52:09.522 INFO:teuthology.orchestra.run.smithi059.stdout: "journal_throttle_max_multiple": "0.000000", 2024-05-08T08:52:09.522 INFO:teuthology.orchestra.run.smithi059.stdout: "journal_write_header_frequency": "0", 2024-05-08T08:52:09.522 INFO:teuthology.orchestra.run.smithi059.stdout: "journal_zero_on_create": "false", 2024-05-08T08:52:09.522 INFO:teuthology.orchestra.run.smithi059.stdout: "journaler_prefetch_periods": "10", 2024-05-08T08:52:09.522 INFO:teuthology.orchestra.run.smithi059.stdout: "journaler_prezero_periods": "5", 2024-05-08T08:52:09.522 INFO:teuthology.orchestra.run.smithi059.stdout: "journaler_write_head_interval": "15", 2024-05-08T08:52:09.522 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "", 2024-05-08T08:52:09.522 INFO:teuthology.orchestra.run.smithi059.stdout: "keyfile": "", 2024-05-08T08:52:09.522 INFO:teuthology.orchestra.run.smithi059.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-05-08T08:52:09.522 INFO:teuthology.orchestra.run.smithi059.stdout: "kstore_backend": "rocksdb", 2024-05-08T08:52:09.522 INFO:teuthology.orchestra.run.smithi059.stdout: "kstore_default_stripe_size": "65536", 2024-05-08T08:52:09.522 INFO:teuthology.orchestra.run.smithi059.stdout: "kstore_fsck_on_mount": "false", 2024-05-08T08:52:09.523 INFO:teuthology.orchestra.run.smithi059.stdout: "kstore_fsck_on_mount_deep": "true", 2024-05-08T08:52:09.523 INFO:teuthology.orchestra.run.smithi059.stdout: "kstore_max_bytes": "67108864", 2024-05-08T08:52:09.523 INFO:teuthology.orchestra.run.smithi059.stdout: "kstore_max_ops": "512", 2024-05-08T08:52:09.523 INFO:teuthology.orchestra.run.smithi059.stdout: "kstore_nid_prealloc": "1024", 2024-05-08T08:52:09.523 INFO:teuthology.orchestra.run.smithi059.stdout: "kstore_onode_map_size": "1024", 2024-05-08T08:52:09.523 INFO:teuthology.orchestra.run.smithi059.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-05-08T08:52:09.523 INFO:teuthology.orchestra.run.smithi059.stdout: "kstore_sync_submit_transaction": "false", 2024-05-08T08:52:09.523 INFO:teuthology.orchestra.run.smithi059.stdout: "kstore_sync_transaction": "false", 2024-05-08T08:52:09.523 INFO:teuthology.orchestra.run.smithi059.stdout: "librados_thread_count": "2", 2024-05-08T08:52:09.523 INFO:teuthology.orchestra.run.smithi059.stdout: "lockdep": "false", 2024-05-08T08:52:09.523 INFO:teuthology.orchestra.run.smithi059.stdout: "lockdep_force_backtrace": "false", 2024-05-08T08:52:09.523 INFO:teuthology.orchestra.run.smithi059.stdout: "log_coarse_timestamps": "true", 2024-05-08T08:52:09.523 INFO:teuthology.orchestra.run.smithi059.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-05-08T08:52:09.523 INFO:teuthology.orchestra.run.smithi059.stdout: "log_flush_on_exit": "false", 2024-05-08T08:52:09.524 INFO:teuthology.orchestra.run.smithi059.stdout: "log_graylog_host": "127.0.0.1", 2024-05-08T08:52:09.524 INFO:teuthology.orchestra.run.smithi059.stdout: "log_graylog_port": "12201", 2024-05-08T08:52:09.524 INFO:teuthology.orchestra.run.smithi059.stdout: "log_max_new": "1000", 2024-05-08T08:52:09.524 INFO:teuthology.orchestra.run.smithi059.stdout: "log_max_recent": "10000", 2024-05-08T08:52:09.524 INFO:teuthology.orchestra.run.smithi059.stdout: "log_stderr_prefix": "", 2024-05-08T08:52:09.524 INFO:teuthology.orchestra.run.smithi059.stdout: "log_stop_at_utilization": "0.970000", 2024-05-08T08:52:09.524 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": "true", 2024-05-08T08:52:09.524 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_graylog": "false", 2024-05-08T08:52:09.524 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_journald": "false", 2024-05-08T08:52:09.524 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_stderr": "false", 2024-05-08T08:52:09.524 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_syslog": "false", 2024-05-08T08:52:09.524 INFO:teuthology.orchestra.run.smithi059.stdout: "max_rotating_auth_attempts": "10", 2024-05-08T08:52:09.524 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-05-08T08:52:09.524 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_action_on_write_error": "1", 2024-05-08T08:52:09.525 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_alternate_name_max": "8192", 2024-05-08T08:52:09.525 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_asio_thread_count": "2", 2024-05-08T08:52:09.525 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_bal_export_pin": "true", 2024-05-08T08:52:09.525 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_bal_fragment_dirs": "true", 2024-05-08T08:52:09.525 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-05-08T08:52:09.525 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_bal_fragment_interval": "5", 2024-05-08T08:52:09.525 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_bal_fragment_size_max": "100000", 2024-05-08T08:52:09.525 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_bal_idle_threshold": "0.000000", 2024-05-08T08:52:09.525 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_bal_interval": "10", 2024-05-08T08:52:09.525 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_bal_max": "-1", 2024-05-08T08:52:09.525 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_bal_max_until": "-1", 2024-05-08T08:52:09.525 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_bal_merge_size": "50", 2024-05-08T08:52:09.525 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_bal_midchunk": "0.300000", 2024-05-08T08:52:09.525 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_bal_min_rebalance": "0.100000", 2024-05-08T08:52:09.525 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_bal_min_start": "0.200000", 2024-05-08T08:52:09.526 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_bal_minchunk": "0.001000", 2024-05-08T08:52:09.526 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_bal_mode": "0", 2024-05-08T08:52:09.526 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_bal_need_max": "1.200000", 2024-05-08T08:52:09.526 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_bal_need_min": "0.800000", 2024-05-08T08:52:09.526 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_bal_overload_epochs": "2", 2024-05-08T08:52:09.526 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-05-08T08:52:09.526 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_bal_sample_interval": "3.000000", 2024-05-08T08:52:09.526 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_bal_split_bits": "3", 2024-05-08T08:52:09.526 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_bal_split_rd": "25000.000000", 2024-05-08T08:52:09.526 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_bal_split_size": "10000", 2024-05-08T08:52:09.526 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_bal_split_wr": "10000.000000", 2024-05-08T08:52:09.526 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_bal_target_decay": "10.000000", 2024-05-08T08:52:09.526 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-05-08T08:52:09.526 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_beacon_grace": "15.000000", 2024-05-08T08:52:09.527 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_beacon_interval": "4.000000", 2024-05-08T08:52:09.527 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_beacon_mon_down_grace": "60", 2024-05-08T08:52:09.527 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_cache_memory_limit": "4294967296", 2024-05-08T08:52:09.527 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_cache_mid": "0.700000", 2024-05-08T08:52:09.527 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_cache_quiesce_decay_rate": "1.000000", 2024-05-08T08:52:09.527 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_cache_quiesce_delay": "0", 2024-05-08T08:52:09.527 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_cache_quiesce_sleep": "200", 2024-05-08T08:52:09.527 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_cache_quiesce_splitauth": "true", 2024-05-08T08:52:09.527 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_cache_quiesce_threshold": "524288", 2024-05-08T08:52:09.527 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_cache_release_free_interval": "10", 2024-05-08T08:52:09.527 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_cache_reservation": "0.050000", 2024-05-08T08:52:09.527 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-05-08T08:52:09.527 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_cache_trim_interval": "1", 2024-05-08T08:52:09.527 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_cache_trim_threshold": "262144", 2024-05-08T08:52:09.527 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-05-08T08:52:09.528 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-05-08T08:52:09.528 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_client_delegate_inos_pct": "50", 2024-05-08T08:52:09.528 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_client_prealloc_inos": "1000", 2024-05-08T08:52:09.528 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-05-08T08:52:09.528 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_connect_bootstrapping": "false", 2024-05-08T08:52:09.528 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_damage_table_max_entries": "10000", 2024-05-08T08:52:09.528 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-05-08T08:52:09.528 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_debug_auth_pins": "false", 2024-05-08T08:52:09.528 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_debug_frag": "false", 2024-05-08T08:52:09.528 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_debug_scatterstat": "false", 2024-05-08T08:52:09.528 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_debug_subtrees": "false", 2024-05-08T08:52:09.528 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_decay_halflife": "5.000000", 2024-05-08T08:52:09.528 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_default_dir_hash": "2", 2024-05-08T08:52:09.528 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_defer_session_stale": "true", 2024-05-08T08:52:09.528 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_deny_all_reconnect": "false", 2024-05-08T08:52:09.529 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_dir_keys_per_op": "16384", 2024-05-08T08:52:09.529 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_dir_max_commit_size": "10", 2024-05-08T08:52:09.529 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_dir_max_entries": "0", 2024-05-08T08:52:09.529 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_dir_prefetch": "true", 2024-05-08T08:52:09.529 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_dirstat_min_interval": "1.000000", 2024-05-08T08:52:09.529 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_dump_cache_after_rejoin": "false", 2024-05-08T08:52:09.529 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_dump_cache_on_map": "false", 2024-05-08T08:52:09.529 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_dump_cache_threshold_file": "0", 2024-05-08T08:52:09.529 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-05-08T08:52:09.529 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_early_reply": "true", 2024-05-08T08:52:09.529 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_enable_op_tracker": "true", 2024-05-08T08:52:09.529 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_enforce_unique_name": "true", 2024-05-08T08:52:09.529 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_export_ephemeral_distributed": "true", 2024-05-08T08:52:09.529 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-05-08T08:52:09.530 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_export_ephemeral_random": "true", 2024-05-08T08:52:09.530 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-05-08T08:52:09.530 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-05-08T08:52:09.530 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_forward_all_requests_to_auth": "false", 2024-05-08T08:52:09.530 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-05-08T08:52:09.530 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-05-08T08:52:09.530 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-05-08T08:52:09.530 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-05-08T08:52:09.530 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_health_cache_threshold": "1.500000", 2024-05-08T08:52:09.530 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_health_summarize_threshold": "10", 2024-05-08T08:52:09.530 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_heartbeat_grace": "15.000000", 2024-05-08T08:52:09.530 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_heartbeat_reset_grace": "1000", 2024-05-08T08:52:09.530 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_inject_health_dummy": "false", 2024-05-08T08:52:09.530 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-05-08T08:52:09.530 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_inject_migrator_session_race": "false", 2024-05-08T08:52:09.531 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-05-08T08:52:09.531 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-05-08T08:52:09.531 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-05-08T08:52:09.531 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_join_fs": "", 2024-05-08T08:52:09.531 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_journal_format": "1", 2024-05-08T08:52:09.531 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_kill_create_at": "0", 2024-05-08T08:52:09.531 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_kill_export_at": "0", 2024-05-08T08:52:09.531 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_kill_import_at": "0", 2024-05-08T08:52:09.531 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_kill_journal_at": "0", 2024-05-08T08:52:09.531 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_kill_journal_expire_at": "0", 2024-05-08T08:52:09.531 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_kill_journal_replay_at": "0", 2024-05-08T08:52:09.531 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_kill_link_at": "0", 2024-05-08T08:52:09.531 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_kill_mdstable_at": "0", 2024-05-08T08:52:09.531 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_kill_openc_at": "0", 2024-05-08T08:52:09.531 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_kill_rename_at": "0", 2024-05-08T08:52:09.532 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_kill_shutdown_at": "0", 2024-05-08T08:52:09.532 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_kill_skip_replaying_inotable": "false", 2024-05-08T08:52:09.532 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_log_event_large_threshold": "524288", 2024-05-08T08:52:09.532 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_log_events_per_segment": "1024", 2024-05-08T08:52:09.532 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_log_major_segment_event_ratio": "12", 2024-05-08T08:52:09.532 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_log_max_events": "-1", 2024-05-08T08:52:09.532 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_log_max_segments": "128", 2024-05-08T08:52:09.532 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_log_pause": "false", 2024-05-08T08:52:09.532 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_log_segment_size": "0", 2024-05-08T08:52:09.532 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_log_skip_corrupt_events": "false", 2024-05-08T08:52:09.532 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_log_skip_unbounded_events": "false", 2024-05-08T08:52:09.532 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_log_trim_decay_rate": "1.000000", 2024-05-08T08:52:09.532 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_log_trim_threshold": "128", 2024-05-08T08:52:09.532 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_log_trim_upkeep_interval": "1000", 2024-05-08T08:52:09.532 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_log_warn_factor": "2.000000", 2024-05-08T08:52:09.533 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_max_caps_per_client": "1048576", 2024-05-08T08:52:09.533 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_max_completed_flushes": "100000", 2024-05-08T08:52:09.533 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_max_completed_requests": "100000", 2024-05-08T08:52:09.533 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_max_export_size": "20971520", 2024-05-08T08:52:09.533 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_max_file_recover": "32", 2024-05-08T08:52:09.533 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_max_purge_files": "64", 2024-05-08T08:52:09.533 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_max_purge_ops": "8192", 2024-05-08T08:52:09.533 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-05-08T08:52:09.533 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-05-08T08:52:09.533 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_max_snaps_per_dir": "100", 2024-05-08T08:52:09.533 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_metrics_update_interval": "2", 2024-05-08T08:52:09.533 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_min_caps_per_client": "100", 2024-05-08T08:52:09.533 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_min_caps_working_set": "10000", 2024-05-08T08:52:09.533 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-05-08T08:52:09.534 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_numa_node": "-1", 2024-05-08T08:52:09.534 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-05-08T08:52:09.534 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_op_complaint_time": "30.000000", 2024-05-08T08:52:09.534 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_op_history_duration": "600", 2024-05-08T08:52:09.534 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_op_history_size": "20", 2024-05-08T08:52:09.534 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_op_history_slow_op_size": "20", 2024-05-08T08:52:09.534 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_op_history_slow_op_threshold": "10", 2024-05-08T08:52:09.534 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_op_log_threshold": "5", 2024-05-08T08:52:09.534 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_ping_grace": "15", 2024-05-08T08:52:09.534 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_ping_interval": "5", 2024-05-08T08:52:09.534 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-05-08T08:52:09.534 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-05-08T08:52:09.534 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_recall_max_caps": "30000", 2024-05-08T08:52:09.534 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-05-08T08:52:09.535 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_recall_max_decay_threshold": "131072", 2024-05-08T08:52:09.535 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-05-08T08:52:09.537 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_recall_warning_threshold": "262144", 2024-05-08T08:52:09.537 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_reconnect_timeout": "45.000000", 2024-05-08T08:52:09.537 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_replay_interval": "1.000000", 2024-05-08T08:52:09.537 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-05-08T08:52:09.537 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-05-08T08:52:09.537 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_root_ino_gid": "0", 2024-05-08T08:52:09.538 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_root_ino_uid": "0", 2024-05-08T08:52:09.538 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-05-08T08:52:09.538 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_session_blocklist_on_evict": "true", 2024-05-08T08:52:09.538 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_session_blocklist_on_timeout": "true", 2024-05-08T08:52:09.538 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-05-08T08:52:09.538 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-05-08T08:52:09.538 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-05-08T08:52:09.538 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-05-08T08:52:09.538 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-05-08T08:52:09.538 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_session_metadata_threshold": "16777216", 2024-05-08T08:52:09.538 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-05-08T08:52:09.538 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_shutdown_check": "0", 2024-05-08T08:52:09.538 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_skip_ino": "0", 2024-05-08T08:52:09.538 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_sleep_rank_change": "0.000000", 2024-05-08T08:52:09.539 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_snap_max_uid": "4294967294", 2024-05-08T08:52:09.539 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_snap_min_uid": "0", 2024-05-08T08:52:09.539 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_snap_rstat": "false", 2024-05-08T08:52:09.539 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_standby_replay_damaged": "false", 2024-05-08T08:52:09.539 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_symlink_recovery": "true", 2024-05-08T08:52:09.539 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_task_status_update_interval": "2.000000", 2024-05-08T08:52:09.539 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_thrash_exports": "0", 2024-05-08T08:52:09.539 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_thrash_fragments": "0", 2024-05-08T08:52:09.539 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_tick_interval": "5.000000", 2024-05-08T08:52:09.539 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_valgrind_exit": "false", 2024-05-08T08:52:09.539 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_verify_backtrace": "1", 2024-05-08T08:52:09.539 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_verify_scatter": "false", 2024-05-08T08:52:09.539 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_wipe_ino_prealloc": "false", 2024-05-08T08:52:09.539 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_wipe_sessions": "false", 2024-05-08T08:52:09.539 INFO:teuthology.orchestra.run.smithi059.stdout: "mempool_debug": "false", 2024-05-08T08:52:09.540 INFO:teuthology.orchestra.run.smithi059.stdout: "memstore_debug_omit_block_device_write": "false", 2024-05-08T08:52:09.540 INFO:teuthology.orchestra.run.smithi059.stdout: "memstore_device_bytes": "1073741824", 2024-05-08T08:52:09.540 INFO:teuthology.orchestra.run.smithi059.stdout: "memstore_page_set": "false", 2024-05-08T08:52:09.540 INFO:teuthology.orchestra.run.smithi059.stdout: "memstore_page_size": "65536", 2024-05-08T08:52:09.540 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr_client_bytes": "134217728", 2024-05-08T08:52:09.540 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr_client_messages": "512", 2024-05-08T08:52:09.540 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-05-08T08:52:09.540 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr_connect_retry_interval": "1.000000", 2024-05-08T08:52:09.540 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-05-08T08:52:09.540 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-05-08T08:52:09.540 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr_disabled_modules": "", 2024-05-08T08:52:09.540 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-05-08T08:52:09.540 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr_max_pg_num_change": "128", 2024-05-08T08:52:09.540 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr_mds_bytes": "134217728", 2024-05-08T08:52:09.540 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr_mds_messages": "128", 2024-05-08T08:52:09.541 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-05-08T08:52:09.541 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr_mon_bytes": "134217728", 2024-05-08T08:52:09.541 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr_mon_messages": "128", 2024-05-08T08:52:09.541 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr_osd_bytes": "536870912", 2024-05-08T08:52:09.541 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr_osd_messages": "8192", 2024-05-08T08:52:09.541 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr_pool": "true", 2024-05-08T08:52:09.541 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr_service_beacon_grace": "60.000000", 2024-05-08T08:52:09.541 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr_standby_modules": "true", 2024-05-08T08:52:09.541 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr_stats_period": "5", 2024-05-08T08:52:09.541 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr_stats_threshold": "5", 2024-05-08T08:52:09.541 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr_tick_period": "2", 2024-05-08T08:52:09.541 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-05-08T08:52:09.541 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_accept_timeout_factor": "2.000000", 2024-05-08T08:52:09.541 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_allow_pool_delete": "true", 2024-05-08T08:52:09.541 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_allow_pool_size_one": "true", 2024-05-08T08:52:09.542 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_auth_validate_all_caps": "true", 2024-05-08T08:52:09.542 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-05-08T08:52:09.542 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-05-08T08:52:09.542 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_client_bytes": "104857600", 2024-05-08T08:52:09.542 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_client_directed_command_retry": "5", 2024-05-08T08:52:09.542 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_client_hunt_interval": "3.000000", 2024-05-08T08:52:09.542 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-05-08T08:52:09.542 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-05-08T08:52:09.542 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-05-08T08:52:09.542 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_client_hunt_parallel": "3", 2024-05-08T08:52:09.542 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_client_log_interval": "1.000000", 2024-05-08T08:52:09.542 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-05-08T08:52:09.542 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_client_ping_interval": "10.000000", 2024-05-08T08:52:09.542 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_client_ping_timeout": "30.000000", 2024-05-08T08:52:09.543 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_client_target_rank": "-1", 2024-05-08T08:52:09.543 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_clock_drift_allowed": "1.000000", 2024-05-08T08:52:09.543 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-05-08T08:52:09.543 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-05-08T08:52:09.543 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_cluster_log_file_level": "debug", 2024-05-08T08:52:09.543 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_cluster_log_to_file": "true", 2024-05-08T08:52:09.543 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_cluster_log_to_graylog": "false", 2024-05-08T08:52:09.543 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-05-08T08:52:09.544 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-05-08T08:52:09.544 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_cluster_log_to_journald": "false", 2024-05-08T08:52:09.544 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_cluster_log_to_stderr": "false", 2024-05-08T08:52:09.544 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-05-08T08:52:09.544 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-05-08T08:52:09.545 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_cluster_log_to_syslog_level": "info", 2024-05-08T08:52:09.545 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_compact_on_bootstrap": "false", 2024-05-08T08:52:09.545 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_compact_on_start": "false", 2024-05-08T08:52:09.545 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_compact_on_trim": "true", 2024-05-08T08:52:09.545 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_con_tracker_persist_interval": "10", 2024-05-08T08:52:09.545 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_con_tracker_score_halflife": "43200", 2024-05-08T08:52:09.545 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_config_key_max_entry_size": "65536", 2024-05-08T08:52:09.545 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_cpu_threads": "4", 2024-05-08T08:52:09.545 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_crush_min_required_version": "hammer", 2024-05-08T08:52:09.545 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_daemon_bytes": "419430400", 2024-05-08T08:52:09.545 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-05-08T08:52:09.545 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_data_avail_crit": "5", 2024-05-08T08:52:09.545 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_data_avail_warn": "5", 2024-05-08T08:52:09.545 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_data_size_warn": "16106127360", 2024-05-08T08:52:09.545 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_debug_block_osdmap_trim": "false", 2024-05-08T08:52:09.546 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-05-08T08:52:09.546 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_debug_dump_json": "false", 2024-05-08T08:52:09.546 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-05-08T08:52:09.546 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_debug_dump_transactions": "false", 2024-05-08T08:52:09.546 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_debug_extra_checks": "false", 2024-05-08T08:52:09.546 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-05-08T08:52:09.546 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-05-08T08:52:09.546 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_debug_no_require_reef": "false", 2024-05-08T08:52:09.546 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_debug_no_require_squid": "false", 2024-05-08T08:52:09.546 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-05-08T08:52:09.546 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_delta_reset_interval": "10.000000", 2024-05-08T08:52:09.546 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_dns_srv_name": "ceph-mon", 2024-05-08T08:52:09.546 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_down_mkfs_grace": "120", 2024-05-08T08:52:09.546 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_down_uptime_grace": "60", 2024-05-08T08:52:09.547 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_election_default_strategy": "1", 2024-05-08T08:52:09.547 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_election_timeout": "5.000000", 2024-05-08T08:52:09.547 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-05-08T08:52:09.547 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_elector_ping_divisor": "2", 2024-05-08T08:52:09.547 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_elector_ping_timeout": "2.000000", 2024-05-08T08:52:09.547 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_enable_op_tracker": "true", 2024-05-08T08:52:09.547 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_fake_pool_delete": "false", 2024-05-08T08:52:09.547 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_force_quorum_join": "false", 2024-05-08T08:52:09.547 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_globalid_prealloc": "10000", 2024-05-08T08:52:09.547 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_health_detail_to_clog": "false", 2024-05-08T08:52:09.547 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_health_log_update_period": "5", 2024-05-08T08:52:09.547 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_health_max_detail": "50", 2024-05-08T08:52:09.547 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_health_to_clog": "true", 2024-05-08T08:52:09.547 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_health_to_clog_interval": "600", 2024-05-08T08:52:09.548 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-05-08T08:52:09.548 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_host": "172.21.15.59,[v2:172.21.15.59:3301,v1:172.21.15.59:6790],172.21.15.192", 2024-05-08T08:52:09.548 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_host_override": "", 2024-05-08T08:52:09.548 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_initial_members": "", 2024-05-08T08:52:09.548 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-05-08T08:52:09.548 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-05-08T08:52:09.548 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-05-08T08:52:09.548 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-05-08T08:52:09.548 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_keyvaluedb": "rocksdb", 2024-05-08T08:52:09.548 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_lease": "5.000000", 2024-05-08T08:52:09.548 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-05-08T08:52:09.548 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-05-08T08:52:09.548 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_log_full_interval": "50", 2024-05-08T08:52:09.548 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_log_max": "10000", 2024-05-08T08:52:09.548 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_log_max_summary": "50", 2024-05-08T08:52:09.549 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_max_log_entries_per_event": "4096", 2024-05-08T08:52:09.549 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_max_log_epochs": "500", 2024-05-08T08:52:09.549 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_max_mdsmap_epochs": "500", 2024-05-08T08:52:09.549 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_max_mgrmap_epochs": "500", 2024-05-08T08:52:09.549 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_max_osd": "10000", 2024-05-08T08:52:09.549 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_max_pg_per_osd": "10000", 2024-05-08T08:52:09.549 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_max_pool_pg_num": "65536", 2024-05-08T08:52:09.549 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-05-08T08:52:09.549 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-05-08T08:52:09.549 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_mds_force_trim_to": "0", 2024-05-08T08:52:09.549 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_mds_skip_sanity": "false", 2024-05-08T08:52:09.549 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_memory_autotune": "true", 2024-05-08T08:52:09.549 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_memory_target": "2147483648", 2024-05-08T08:52:09.549 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_mgr_beacon_grace": "30", 2024-05-08T08:52:09.549 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-05-08T08:52:09.550 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_mgr_digest_period": "5", 2024-05-08T08:52:09.550 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_mgr_inactive_grace": "60", 2024-05-08T08:52:09.550 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_mgr_mkfs_grace": "240", 2024-05-08T08:52:09.550 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-05-08T08:52:09.550 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_min_osdmap_epochs": "500", 2024-05-08T08:52:09.550 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_op_complaint_time": "30", 2024-05-08T08:52:09.550 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_op_history_duration": "600", 2024-05-08T08:52:09.550 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_op_history_size": "20", 2024-05-08T08:52:09.550 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_op_history_slow_op_size": "20", 2024-05-08T08:52:09.550 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_op_history_slow_op_threshold": "10", 2024-05-08T08:52:09.550 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_op_log_threshold": "5", 2024-05-08T08:52:09.550 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-05-08T08:52:09.550 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-05-08T08:52:09.550 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-05-08T08:52:09.551 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_osd_auto_mark_in": "false", 2024-05-08T08:52:09.551 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_osd_auto_mark_new_in": "true", 2024-05-08T08:52:09.551 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-05-08T08:52:09.551 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-05-08T08:52:09.551 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_osd_cache_size": "500", 2024-05-08T08:52:09.551 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_osd_cache_size_min": "134217728", 2024-05-08T08:52:09.551 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_osd_crush_smoke_test": "true", 2024-05-08T08:52:09.551 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_osd_destroyed_out_interval": "600", 2024-05-08T08:52:09.551 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_osd_down_out_interval": "600", 2024-05-08T08:52:09.551 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-05-08T08:52:09.551 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-05-08T08:52:09.551 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_osd_force_trim_to": "0", 2024-05-08T08:52:09.551 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_osd_full_ratio": "0.950000", 2024-05-08T08:52:09.551 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-05-08T08:52:09.551 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_osd_laggy_halflife": "3600", 2024-05-08T08:52:09.552 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_osd_laggy_max_interval": "300", 2024-05-08T08:52:09.552 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_osd_laggy_weight": "0.300000", 2024-05-08T08:52:09.552 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-05-08T08:52:09.552 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_osd_max_creating_pgs": "1024", 2024-05-08T08:52:09.552 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_osd_max_initial_pgs": "1024", 2024-05-08T08:52:09.552 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_osd_min_down_reporters": "2", 2024-05-08T08:52:09.552 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-05-08T08:52:09.552 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-05-08T08:52:09.552 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-05-08T08:52:09.552 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_osd_prime_pg_temp": "true", 2024-05-08T08:52:09.552 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-05-08T08:52:09.552 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-05-08T08:52:09.552 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_osd_report_timeout": "900", 2024-05-08T08:52:09.552 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-05-08T08:52:09.552 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-05-08T08:52:09.553 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_osd_warn_num_repaired": "10", 2024-05-08T08:52:09.553 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_osd_warn_op_age": "32.000000", 2024-05-08T08:52:09.553 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-05-08T08:52:09.553 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_osdmap_full_prune_interval": "10", 2024-05-08T08:52:09.553 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_osdmap_full_prune_min": "10000", 2024-05-08T08:52:09.553 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_osdmap_full_prune_txsize": "100", 2024-05-08T08:52:09.553 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-05-08T08:52:09.553 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_pg_stuck_threshold": "60", 2024-05-08T08:52:09.553 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-05-08T08:52:09.553 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_pg_warn_min_objects": "10000", 2024-05-08T08:52:09.553 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_pg_warn_min_per_osd": "0", 2024-05-08T08:52:09.553 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-05-08T08:52:09.553 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_pool_quota_crit_threshold": "0", 2024-05-08T08:52:09.553 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_pool_quota_warn_threshold": "0", 2024-05-08T08:52:09.553 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_probe_timeout": "2.000000", 2024-05-08T08:52:09.554 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_reweight_max_change": "0.050000", 2024-05-08T08:52:09.554 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_reweight_max_osds": "4", 2024-05-08T08:52:09.554 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-05-08T08:52:09.554 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-05-08T08:52:09.554 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-05-08T08:52:09.554 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-05-08T08:52:09.554 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-05-08T08:52:09.554 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_scrub_interval": "86400", 2024-05-08T08:52:09.554 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_scrub_max_keys": "100", 2024-05-08T08:52:09.554 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_scrub_timeout": "300", 2024-05-08T08:52:09.555 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_session_timeout": "300", 2024-05-08T08:52:09.555 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_smart_report_timeout": "5", 2024-05-08T08:52:09.555 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_stat_smooth_intervals": "6", 2024-05-08T08:52:09.555 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-05-08T08:52:09.555 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_stretch_pool_min_size": "2", 2024-05-08T08:52:09.555 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_stretch_pool_size": "4", 2024-05-08T08:52:09.555 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-05-08T08:52:09.555 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_subscribe_interval": "86400.000000", 2024-05-08T08:52:09.555 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_sync_debug": "false", 2024-05-08T08:52:09.555 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_sync_max_payload_keys": "2000", 2024-05-08T08:52:09.555 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_sync_max_payload_size": "1048576", 2024-05-08T08:52:09.555 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_sync_provider_kill_at": "0", 2024-05-08T08:52:09.555 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_sync_requester_kill_at": "0", 2024-05-08T08:52:09.555 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_sync_timeout": "60.000000", 2024-05-08T08:52:09.556 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_target_pg_per_osd": "100", 2024-05-08T08:52:09.556 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_tick_interval": "5", 2024-05-08T08:52:09.556 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_timecheck_interval": "300.000000", 2024-05-08T08:52:09.556 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-05-08T08:52:09.556 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_use_min_delay_socket": "false", 2024-05-08T08:52:09.556 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_warn_older_version_delay": "604800", 2024-05-08T08:52:09.556 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-05-08T08:52:09.556 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-05-08T08:52:09.556 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-05-08T08:52:09.556 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_warn_on_filestore_osds": "false", 2024-05-08T08:52:09.556 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-05-08T08:52:09.556 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-05-08T08:52:09.556 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-05-08T08:52:09.556 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_warn_on_misplaced": "false", 2024-05-08T08:52:09.556 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-05-08T08:52:09.557 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_warn_on_older_version": "true", 2024-05-08T08:52:09.557 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-05-08T08:52:09.557 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_warn_on_pool_no_app": "false", 2024-05-08T08:52:09.557 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-05-08T08:52:09.557 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-05-08T08:52:09.557 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-05-08T08:52:09.557 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-05-08T08:52:09.557 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_warn_on_too_few_osds": "false", 2024-05-08T08:52:09.557 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-05-08T08:52:09.557 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-05-08T08:52:09.557 INFO:teuthology.orchestra.run.smithi059.stdout: "monmap": "", 2024-05-08T08:52:09.557 INFO:teuthology.orchestra.run.smithi059.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-05-08T08:52:09.557 INFO:teuthology.orchestra.run.smithi059.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-05-08T08:52:09.557 INFO:teuthology.orchestra.run.smithi059.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-05-08T08:52:09.557 INFO:teuthology.orchestra.run.smithi059.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-05-08T08:52:09.558 INFO:teuthology.orchestra.run.smithi059.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-05-08T08:52:09.558 INFO:teuthology.orchestra.run.smithi059.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-05-08T08:52:09.558 INFO:teuthology.orchestra.run.smithi059.stdout: "motr_tracing_enabled": "false", 2024-05-08T08:52:09.558 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_async_op_threads": "3", 2024-05-08T08:52:09.558 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_async_rdma_buffer_size": "131072", 2024-05-08T08:52:09.558 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_async_rdma_cm": "false", 2024-05-08T08:52:09.558 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_async_rdma_device_name": "", 2024-05-08T08:52:09.558 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_async_rdma_dscp": "96", 2024-05-08T08:52:09.558 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-05-08T08:52:09.558 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_async_rdma_gid_idx": "0", 2024-05-08T08:52:09.558 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_async_rdma_local_gid": "", 2024-05-08T08:52:09.558 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_async_rdma_polling_us": "1000", 2024-05-08T08:52:09.558 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_async_rdma_port_num": "1", 2024-05-08T08:52:09.558 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-05-08T08:52:09.558 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-05-08T08:52:09.559 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_async_rdma_roce_ver": "1", 2024-05-08T08:52:09.559 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_async_rdma_send_buffers": "1024", 2024-05-08T08:52:09.559 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_async_rdma_sl": "3", 2024-05-08T08:52:09.559 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_async_rdma_support_srq": "true", 2024-05-08T08:52:09.559 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_async_rdma_type": "ib", 2024-05-08T08:52:09.559 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_async_reap_threshold": "5", 2024-05-08T08:52:09.559 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_bind_before_connect": "false", 2024-05-08T08:52:09.559 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_bind_ipv4": "true", 2024-05-08T08:52:09.559 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_bind_ipv6": "false", 2024-05-08T08:52:09.559 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_bind_msgr1": "true", 2024-05-08T08:52:09.559 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_bind_msgr2": "true", 2024-05-08T08:52:09.559 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_bind_port_max": "7568", 2024-05-08T08:52:09.559 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_bind_port_min": "6800", 2024-05-08T08:52:09.559 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_bind_prefer_ipv4": "false", 2024-05-08T08:52:09.560 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_bind_retry_count": "3", 2024-05-08T08:52:09.560 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_bind_retry_delay": "5", 2024-05-08T08:52:09.560 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_blackhole_client": "false", 2024-05-08T08:52:09.560 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_blackhole_mds": "false", 2024-05-08T08:52:09.560 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_blackhole_mgr": "false", 2024-05-08T08:52:09.560 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_blackhole_mon": "false", 2024-05-08T08:52:09.560 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_blackhole_osd": "false", 2024-05-08T08:52:09.560 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_client_mode": "crc secure", 2024-05-08T08:52:09.560 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_cluster_mode": "crc secure", 2024-05-08T08:52:09.560 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_cluster_type": "", 2024-05-08T08:52:09.560 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_compress_secure": "false", 2024-05-08T08:52:09.560 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_connection_idle_timeout": "900", 2024-05-08T08:52:09.560 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_connection_ready_timeout": "10", 2024-05-08T08:52:09.560 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_crc_data": "true", 2024-05-08T08:52:09.560 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_crc_header": "true", 2024-05-08T08:52:09.561 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_die_on_bad_msg": "false", 2024-05-08T08:52:09.561 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_die_on_bug": "true", 2024-05-08T08:52:09.561 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_die_on_old_message": "true", 2024-05-08T08:52:09.561 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_die_on_skipped_message": "false", 2024-05-08T08:52:09.561 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_die_on_unhandled_msg": "false", 2024-05-08T08:52:09.561 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-05-08T08:52:09.561 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_dpdk_coremask": "0xF", 2024-05-08T08:52:09.561 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-05-08T08:52:09.561 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_dpdk_devs_allowlist": "", 2024-05-08T08:52:09.561 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_dpdk_enable_tso": "true", 2024-05-08T08:52:09.561 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-05-08T08:52:09.561 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-05-08T08:52:09.561 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_dpdk_hugepages": "", 2024-05-08T08:52:09.561 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_dpdk_hw_flow_control": "true", 2024-05-08T08:52:09.561 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-05-08T08:52:09.562 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_dpdk_lro": "true", 2024-05-08T08:52:09.562 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_dpdk_memory_channel": "4", 2024-05-08T08:52:09.562 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-05-08T08:52:09.562 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_dpdk_pmd": "", 2024-05-08T08:52:09.562 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_dpdk_port_id": "0", 2024-05-08T08:52:09.562 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-05-08T08:52:09.562 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_dump_corrupt_message_level": "1", 2024-05-08T08:52:09.562 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_dump_on_send": "false", 2024-05-08T08:52:09.562 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_initial_backoff": "0.200000", 2024-05-08T08:52:09.562 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_inject_delay_max": "1.000000", 2024-05-08T08:52:09.562 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_inject_delay_probability": "0.000000", 2024-05-08T08:52:09.562 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_inject_delay_type": "", 2024-05-08T08:52:09.562 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_inject_internal_delays": "0.000000", 2024-05-08T08:52:09.562 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_inject_network_congestion": "0", 2024-05-08T08:52:09.563 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_inject_socket_failures": "5000", 2024-05-08T08:52:09.563 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_learn_addr_from_peer": "true", 2024-05-08T08:52:09.563 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_max_accept_failures": "4", 2024-05-08T08:52:09.563 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_max_backoff": "15.000000", 2024-05-08T08:52:09.563 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_mon_client_mode": "secure crc", 2024-05-08T08:52:09.563 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_mon_cluster_mode": "secure crc", 2024-05-08T08:52:09.563 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_mon_service_mode": "secure crc", 2024-05-08T08:52:09.563 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_osd_compress_min_size": "1024", 2024-05-08T08:52:09.563 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_osd_compress_mode": "none", 2024-05-08T08:52:09.563 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_osd_compression_algorithm": "snappy", 2024-05-08T08:52:09.563 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-05-08T08:52:09.563 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_pq_min_cost": "65536", 2024-05-08T08:52:09.563 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_public_type": "", 2024-05-08T08:52:09.563 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_service_mode": "crc secure", 2024-05-08T08:52:09.563 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_tcp_listen_backlog": "512", 2024-05-08T08:52:09.564 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_tcp_nodelay": "true", 2024-05-08T08:52:09.564 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-05-08T08:52:09.564 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_tcp_rcvbuf": "0", 2024-05-08T08:52:09.564 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_type": "async+posix", 2024-05-08T08:52:09.564 INFO:teuthology.orchestra.run.smithi059.stdout: "no_config_file": "false", 2024-05-08T08:52:09.564 INFO:teuthology.orchestra.run.smithi059.stdout: "objecter_completion_locks_per_session": "32", 2024-05-08T08:52:09.564 INFO:teuthology.orchestra.run.smithi059.stdout: "objecter_debug_inject_relock_delay": "false", 2024-05-08T08:52:09.564 INFO:teuthology.orchestra.run.smithi059.stdout: "objecter_inflight_op_bytes": "104857600", 2024-05-08T08:52:09.564 INFO:teuthology.orchestra.run.smithi059.stdout: "objecter_inflight_ops": "1024", 2024-05-08T08:52:09.564 INFO:teuthology.orchestra.run.smithi059.stdout: "objecter_inject_no_watch_ping": "false", 2024-05-08T08:52:09.564 INFO:teuthology.orchestra.run.smithi059.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-05-08T08:52:09.564 INFO:teuthology.orchestra.run.smithi059.stdout: "objecter_tick_interval": "5.000000", 2024-05-08T08:52:09.564 INFO:teuthology.orchestra.run.smithi059.stdout: "objecter_timeout": "10.000000", 2024-05-08T08:52:09.564 INFO:teuthology.orchestra.run.smithi059.stdout: "objectstore_blackhole": "false", 2024-05-08T08:52:09.564 INFO:teuthology.orchestra.run.smithi059.stdout: "objectstore_debug_throw_on_failed_txc": "false", 2024-05-08T08:52:09.565 INFO:teuthology.orchestra.run.smithi059.stdout: "openssl_engine_opts": "", 2024-05-08T08:52:09.565 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_agent_delay_time": "5.000000", 2024-05-08T08:52:09.565 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_agent_hist_halflife": "1000", 2024-05-08T08:52:09.565 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_agent_max_low_ops": "2", 2024-05-08T08:52:09.565 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_agent_max_ops": "4", 2024-05-08T08:52:09.565 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-05-08T08:52:09.565 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_agent_quantize_effort": "0.100000", 2024-05-08T08:52:09.565 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_agent_slop": "0.020000", 2024-05-08T08:52:09.565 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-05-08T08:52:09.565 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_allow_recovery_below_min_size": "true", 2024-05-08T08:52:09.565 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_asio_thread_count": "2", 2024-05-08T08:52:09.565 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_async_recovery_min_cost": "100", 2024-05-08T08:52:09.565 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_auto_mark_unfound_lost": "false", 2024-05-08T08:52:09.565 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_backfill_retry_interval": "30.000000", 2024-05-08T08:52:09.566 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_backfill_scan_max": "512", 2024-05-08T08:52:09.566 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_backfill_scan_min": "64", 2024-05-08T08:52:09.566 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_backoff_on_degraded": "false", 2024-05-08T08:52:09.566 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_backoff_on_peering": "false", 2024-05-08T08:52:09.566 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_backoff_on_unfound": "true", 2024-05-08T08:52:09.566 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_beacon_report_interval": "300", 2024-05-08T08:52:09.566 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_bench_duration": "30", 2024-05-08T08:52:09.566 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-05-08T08:52:09.566 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_bench_max_block_size": "67108864", 2024-05-08T08:52:09.566 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_bench_small_size_max_iops": "100", 2024-05-08T08:52:09.566 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_blkin_trace_all": "false", 2024-05-08T08:52:09.566 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_blocked_scrub_grace_period": "120", 2024-05-08T08:52:09.566 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-05-08T08:52:09.566 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-05-08T08:52:09.566 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-05-08T08:52:09.567 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_check_for_log_corruption": "false", 2024-05-08T08:52:09.567 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-05-08T08:52:09.567 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-05-08T08:52:09.567 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2024-05-08T08:52:09.567 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-05-08T08:52:09.567 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_class_update_on_start": "true", 2024-05-08T08:52:09.567 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_client_message_cap": "256", 2024-05-08T08:52:09.567 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_client_message_size_cap": "524288000", 2024-05-08T08:52:09.567 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_client_op_priority": "63", 2024-05-08T08:52:09.567 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_client_watch_timeout": "30", 2024-05-08T08:52:09.567 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_command_max_records": "256", 2024-05-08T08:52:09.567 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_compact_on_start": "false", 2024-05-08T08:52:09.567 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-05-08T08:52:09.567 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_crush_chooseleaf_type": "0", 2024-05-08T08:52:09.567 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_crush_initial_weight": "-1.000000", 2024-05-08T08:52:09.568 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_crush_update_on_start": "true", 2024-05-08T08:52:09.568 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_crush_update_weight_set": "true", 2024-05-08T08:52:09.568 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-05-08T08:52:09.568 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-05-08T08:52:09.568 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-05-08T08:52:09.568 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_debug_drop_ping_duration": "0", 2024-05-08T08:52:09.568 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-05-08T08:52:09.568 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_debug_feed_pullee": "-1", 2024-05-08T08:52:09.568 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-05-08T08:52:09.568 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-05-08T08:52:09.568 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-05-08T08:52:09.568 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_debug_misdirected_ops": "false", 2024-05-08T08:52:09.568 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_debug_no_acting_change": "false", 2024-05-08T08:52:09.568 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_debug_no_purge_strays": "false", 2024-05-08T08:52:09.568 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_debug_op_order": "false", 2024-05-08T08:52:09.569 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_debug_pg_log_writeout": "false", 2024-05-08T08:52:09.569 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_debug_pretend_recovery_active": "false", 2024-05-08T08:52:09.569 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-05-08T08:52:09.569 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-05-08T08:52:09.569 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_debug_shutdown": "false", 2024-05-08T08:52:09.569 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-05-08T08:52:09.569 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-05-08T08:52:09.569 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_debug_trim_objects": "false", 2024-05-08T08:52:09.569 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_debug_verify_cached_snaps": "false", 2024-05-08T08:52:09.569 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_debug_verify_missing_on_start": "false", 2024-05-08T08:52:09.569 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_debug_verify_snaps": "false", 2024-05-08T08:52:09.569 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-05-08T08:52:09.569 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-05-08T08:52:09.569 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_deep_scrub_keys": "1024", 2024-05-08T08:52:09.570 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-05-08T08:52:09.570 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-05-08T08:52:09.570 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-05-08T08:52:09.570 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_deep_scrub_stride": "524288", 2024-05-08T08:52:09.570 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-05-08T08:52:09.570 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_default_data_pool_replay_window": "5", 2024-05-08T08:52:09.570 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_default_notify_timeout": "30", 2024-05-08T08:52:09.570 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_delete_sleep": "0.000000", 2024-05-08T08:52:09.570 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-05-08T08:52:09.570 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-05-08T08:52:09.570 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-05-08T08:52:09.570 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_discard_disconnected_ops": "true", 2024-05-08T08:52:09.570 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_enable_op_tracker": "true", 2024-05-08T08:52:09.570 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-05-08T08:52:09.570 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_extblkdev_plugins": "vdo", 2024-05-08T08:52:09.571 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-05-08T08:52:09.571 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-05-08T08:52:09.571 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_fast_info": "true", 2024-05-08T08:52:09.571 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_fast_shutdown": "true", 2024-05-08T08:52:09.571 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-05-08T08:52:09.571 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_fast_shutdown_timeout": "15", 2024-05-08T08:52:09.571 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-05-08T08:52:09.571 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-05-08T08:52:09.571 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-05-08T08:52:09.571 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_function_tracing": "false", 2024-05-08T08:52:09.571 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_heartbeat_grace": "20", 2024-05-08T08:52:09.571 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_heartbeat_interval": "6", 2024-05-08T08:52:09.571 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-05-08T08:52:09.571 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_heartbeat_min_peers": "10", 2024-05-08T08:52:09.571 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_heartbeat_min_size": "2000", 2024-05-08T08:52:09.572 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_heartbeat_stale": "600", 2024-05-08T08:52:09.572 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-05-08T08:52:09.572 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_hit_set_max_size": "100000", 2024-05-08T08:52:09.572 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_hit_set_min_size": "1000", 2024-05-08T08:52:09.572 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-05-08T08:52:09.572 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-05-08T08:52:09.572 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-05-08T08:52:09.572 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-05-08T08:52:09.572 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-05-08T08:52:09.572 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_journal_flush_on_shutdown": "true", 2024-05-08T08:52:09.572 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_journal_size": "5120", 2024-05-08T08:52:09.572 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_kill_backfill_at": "0", 2024-05-08T08:52:09.572 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_loop_before_reset_tphandle": "64", 2024-05-08T08:52:09.572 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_map_cache_size": "50", 2024-05-08T08:52:09.573 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_map_dedup": "true", 2024-05-08T08:52:09.573 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_map_message_max": "40", 2024-05-08T08:52:09.573 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_map_message_max_bytes": "10485760", 2024-05-08T08:52:09.573 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_map_share_max_epochs": "40", 2024-05-08T08:52:09.573 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_max_attr_name_len": "100", 2024-05-08T08:52:09.573 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_max_attr_size": "0", 2024-05-08T08:52:09.573 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_max_backfills": "1", 2024-05-08T08:52:09.573 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_max_markdown_count": "5", 2024-05-08T08:52:09.573 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_max_markdown_period": "600", 2024-05-08T08:52:09.573 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_max_object_name_len": "2048", 2024-05-08T08:52:09.573 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_max_object_namespace_len": "256", 2024-05-08T08:52:09.573 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_max_object_size": "134217728", 2024-05-08T08:52:09.573 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-05-08T08:52:09.573 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_max_omap_entries_per_request": "1024", 2024-05-08T08:52:09.573 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_max_pg_blocked_by": "16", 2024-05-08T08:52:09.574 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_max_pg_log_entries": "10000", 2024-05-08T08:52:09.574 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-05-08T08:52:09.574 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_max_pgls": "1024", 2024-05-08T08:52:09.574 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_max_push_cost": "8388608", 2024-05-08T08:52:09.574 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_max_push_objects": "10", 2024-05-08T08:52:09.574 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_max_scrubs": "3", 2024-05-08T08:52:09.574 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-05-08T08:52:09.574 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_max_trimming_pgs": "2", 2024-05-08T08:52:09.574 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_max_write_op_reply_len": "64", 2024-05-08T08:52:09.574 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_max_write_size": "90", 2024-05-08T08:52:09.574 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-05-08T08:52:09.574 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-05-08T08:52:09.574 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-05-08T08:52:09.574 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-05-08T08:52:09.575 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-05-08T08:52:09.575 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-05-08T08:52:09.575 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-05-08T08:52:09.575 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mclock_override_recovery_settings": "false", 2024-05-08T08:52:09.575 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mclock_profile": "balanced", 2024-05-08T08:52:09.575 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-05-08T08:52:09.575 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-05-08T08:52:09.575 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-05-08T08:52:09.575 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-05-08T08:52:09.575 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-05-08T08:52:09.575 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-05-08T08:52:09.575 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-05-08T08:52:09.575 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-05-08T08:52:09.575 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-05-08T08:52:09.575 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-05-08T08:52:09.576 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mclock_skip_benchmark": "false", 2024-05-08T08:52:09.576 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_memory_base": "805306368", 2024-05-08T08:52:09.576 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_memory_cache_min": "134217728", 2024-05-08T08:52:09.576 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-05-08T08:52:09.576 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-05-08T08:52:09.576 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_memory_target": "4294967296", 2024-05-08T08:52:09.576 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_memory_target_autotune": "false", 2024-05-08T08:52:09.576 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-05-08T08:52:09.576 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_min_pg_log_entries": "250", 2024-05-08T08:52:09.576 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_min_recovery_priority": "0", 2024-05-08T08:52:09.576 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mon_heartbeat_interval": "30", 2024-05-08T08:52:09.576 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-05-08T08:52:09.576 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mon_report_interval": "5", 2024-05-08T08:52:09.576 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mon_report_max_in_flight": "2", 2024-05-08T08:52:09.577 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-05-08T08:52:09.577 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_num_cache_shards": "32", 2024-05-08T08:52:09.577 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_num_op_tracker_shard": "32", 2024-05-08T08:52:09.577 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_numa_auto_affinity": "true", 2024-05-08T08:52:09.577 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_numa_node": "-1", 2024-05-08T08:52:09.577 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_numa_prefer_iface": "true", 2024-05-08T08:52:09.577 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-05-08T08:52:09.577 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_objecter_finishers": "1", 2024-05-08T08:52:09.577 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_objectstore": "bluestore", 2024-05-08T08:52:09.577 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_objectstore_fuse": "false", 2024-05-08T08:52:09.577 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_objectstore_tracing": "false", 2024-05-08T08:52:09.577 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_op_complaint_time": "30.000000", 2024-05-08T08:52:09.577 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_op_history_duration": "600", 2024-05-08T08:52:09.577 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_op_history_size": "20", 2024-05-08T08:52:09.577 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_op_history_slow_op_size": "20", 2024-05-08T08:52:09.578 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-05-08T08:52:09.578 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_op_log_threshold": "5", 2024-05-08T08:52:09.578 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_op_num_shards": "0", 2024-05-08T08:52:09.578 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_op_num_shards_hdd": "5", 2024-05-08T08:52:09.578 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_op_num_shards_ssd": "8", 2024-05-08T08:52:09.578 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_op_num_threads_per_shard": "0", 2024-05-08T08:52:09.578 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-05-08T08:52:09.578 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-05-08T08:52:09.578 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-05-08T08:52:09.578 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_op_pq_min_cost": "65536", 2024-05-08T08:52:09.578 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_op_queue": "mclock_scheduler", 2024-05-08T08:52:09.578 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_op_queue_cut_off": "high", 2024-05-08T08:52:09.578 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_op_thread_suicide_timeout": "150", 2024-05-08T08:52:09.578 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_op_thread_timeout": "15", 2024-05-08T08:52:09.578 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_open_classes_on_start": "true", 2024-05-08T08:52:09.579 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_os_flags": "0", 2024-05-08T08:52:09.579 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_peering_op_priority": "255", 2024-05-08T08:52:09.579 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pg_delete_cost": "1048576", 2024-05-08T08:52:09.579 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pg_delete_priority": "5", 2024-05-08T08:52:09.579 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-05-08T08:52:09.579 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-05-08T08:52:09.579 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pg_log_dups_tracked": "3000", 2024-05-08T08:52:09.579 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pg_log_trim_max": "10000", 2024-05-08T08:52:09.579 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pg_log_trim_min": "100", 2024-05-08T08:52:09.579 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-05-08T08:52:09.579 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pg_object_context_cache_count": "64", 2024-05-08T08:52:09.579 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pg_stat_report_interval_max_epochs": "500", 2024-05-08T08:52:09.579 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pg_stat_report_interval_max_seconds": "5", 2024-05-08T08:52:09.579 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-05-08T08:52:09.579 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-05-08T08:52:09.579 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-05-08T08:52:09.580 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-05-08T08:52:09.580 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-05-08T08:52:09.580 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-05-08T08:52:09.580 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pool_default_crimson": "false", 2024-05-08T08:52:09.580 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pool_default_crush_rule": "-1", 2024-05-08T08:52:09.580 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pool_default_ec_fast_read": "false", 2024-05-08T08:52:09.580 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-05-08T08:52:09.580 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pool_default_flag_bulk": "false", 2024-05-08T08:52:09.580 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-05-08T08:52:09.580 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pool_default_flag_nodelete": "false", 2024-05-08T08:52:09.580 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-05-08T08:52:09.580 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-05-08T08:52:09.580 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pool_default_flags": "0", 2024-05-08T08:52:09.580 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-05-08T08:52:09.580 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pool_default_min_size": "0", 2024-05-08T08:52:09.581 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-05-08T08:52:09.581 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pool_default_pg_num": "32", 2024-05-08T08:52:09.581 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pool_default_pgp_num": "0", 2024-05-08T08:52:09.581 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-05-08T08:52:09.581 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pool_default_read_ratio": "70", 2024-05-08T08:52:09.581 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pool_default_size": "2", 2024-05-08T08:52:09.581 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pool_default_type": "replicated", 2024-05-08T08:52:09.581 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-05-08T08:52:09.581 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pool_use_gmt_hitset": "true", 2024-05-08T08:52:09.581 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_push_per_object_cost": "1000", 2024-05-08T08:52:09.581 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_read_ec_check_for_errors": "false", 2024-05-08T08:52:09.581 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_recover_clone_overlap": "true", 2024-05-08T08:52:09.581 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_recover_clone_overlap_limit": "10", 2024-05-08T08:52:09.581 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_recovery_cost": "20971520", 2024-05-08T08:52:09.582 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_recovery_delay_start": "0.000000", 2024-05-08T08:52:09.582 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_recovery_max_active": "0", 2024-05-08T08:52:09.582 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_recovery_max_active_hdd": "3", 2024-05-08T08:52:09.582 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_recovery_max_active_ssd": "10", 2024-05-08T08:52:09.583 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_recovery_max_chunk": "8388608", 2024-05-08T08:52:09.583 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-05-08T08:52:09.583 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_recovery_max_single_start": "1", 2024-05-08T08:52:09.583 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_recovery_op_priority": "3", 2024-05-08T08:52:09.583 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_recovery_op_warn_multiple": "16", 2024-05-08T08:52:09.583 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_recovery_priority": "5", 2024-05-08T08:52:09.583 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_recovery_retry_interval": "30.000000", 2024-05-08T08:52:09.583 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_recovery_sleep": "0.000000", 2024-05-08T08:52:09.583 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-05-08T08:52:09.583 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-05-08T08:52:09.583 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-05-08T08:52:09.583 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_repair_during_recovery": "false", 2024-05-08T08:52:09.583 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_requested_scrub_priority": "120", 2024-05-08T08:52:09.583 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-05-08T08:52:09.584 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_rollback_to_cluster_snap": "", 2024-05-08T08:52:09.584 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_scrub_auto_repair": "false", 2024-05-08T08:52:09.584 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-05-08T08:52:09.584 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-05-08T08:52:09.584 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_scrub_begin_hour": "0", 2024-05-08T08:52:09.584 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_scrub_begin_week_day": "0", 2024-05-08T08:52:09.584 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_scrub_chunk_max": "25", 2024-05-08T08:52:09.584 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_scrub_chunk_min": "5", 2024-05-08T08:52:09.584 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_scrub_cost": "52428800", 2024-05-08T08:52:09.584 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_scrub_disable_reservation_queuing": "true", 2024-05-08T08:52:09.584 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_scrub_during_recovery": "false", 2024-05-08T08:52:09.584 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_scrub_end_hour": "0", 2024-05-08T08:52:09.584 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_scrub_end_week_day": "0", 2024-05-08T08:52:09.584 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_scrub_event_cost": "4096", 2024-05-08T08:52:09.585 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-05-08T08:52:09.585 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-05-08T08:52:09.585 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_scrub_invalid_stats": "true", 2024-05-08T08:52:09.585 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_scrub_load_threshold": "0.500000", 2024-05-08T08:52:09.585 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_scrub_max_interval": "604800.000000", 2024-05-08T08:52:09.585 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_scrub_max_preemptions": "5", 2024-05-08T08:52:09.585 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_scrub_min_interval": "86400.000000", 2024-05-08T08:52:09.585 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_scrub_priority": "5", 2024-05-08T08:52:09.585 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_scrub_sleep": "0.000000", 2024-05-08T08:52:09.585 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-05-08T08:52:09.585 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-05-08T08:52:09.585 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_shutdown_pgref_assert": "false", 2024-05-08T08:52:09.585 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_skip_data_digest": "false", 2024-05-08T08:52:09.585 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_smart_report_timeout": "5", 2024-05-08T08:52:09.585 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_snap_trim_cost": "1048576", 2024-05-08T08:52:09.585 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_snap_trim_priority": "5", 2024-05-08T08:52:09.586 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_snap_trim_sleep": "0.000000", 2024-05-08T08:52:09.586 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-05-08T08:52:09.586 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-05-08T08:52:09.586 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-05-08T08:52:09.586 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-05-08T08:52:09.586 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_stats_update_period_scrubbing": "15", 2024-05-08T08:52:09.586 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-05-08T08:52:09.586 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_target_transaction_size": "30", 2024-05-08T08:52:09.587 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-05-08T08:52:09.587 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-05-08T08:52:09.587 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-05-08T08:52:09.587 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-05-08T08:52:09.587 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-05-08T08:52:09.587 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-05-08T08:52:09.587 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-05-08T08:52:09.587 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_tier_default_cache_mode": "writeback", 2024-05-08T08:52:09.587 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-05-08T08:52:09.587 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-05-08T08:52:09.587 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_tracing": "false", 2024-05-08T08:52:09.588 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_use_stale_snap": "false", 2024-05-08T08:52:09.588 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-05-08T08:52:09.588 INFO:teuthology.orchestra.run.smithi059.stdout: "osdc_blkin_trace_all": "false", 2024-05-08T08:52:09.588 INFO:teuthology.orchestra.run.smithi059.stdout: "paxos_kill_at": "0", 2024-05-08T08:52:09.589 INFO:teuthology.orchestra.run.smithi059.stdout: "paxos_max_join_drift": "10", 2024-05-08T08:52:09.589 INFO:teuthology.orchestra.run.smithi059.stdout: "paxos_min": "500", 2024-05-08T08:52:09.589 INFO:teuthology.orchestra.run.smithi059.stdout: "paxos_min_wait": "0.050000", 2024-05-08T08:52:09.589 INFO:teuthology.orchestra.run.smithi059.stdout: "paxos_propose_interval": "1.000000", 2024-05-08T08:52:09.589 INFO:teuthology.orchestra.run.smithi059.stdout: "paxos_service_trim_max": "500", 2024-05-08T08:52:09.589 INFO:teuthology.orchestra.run.smithi059.stdout: "paxos_service_trim_max_multiplier": "20", 2024-05-08T08:52:09.589 INFO:teuthology.orchestra.run.smithi059.stdout: "paxos_service_trim_min": "250", 2024-05-08T08:52:09.589 INFO:teuthology.orchestra.run.smithi059.stdout: "paxos_stash_full_interval": "25", 2024-05-08T08:52:09.589 INFO:teuthology.orchestra.run.smithi059.stdout: "paxos_trim_max": "500", 2024-05-08T08:52:09.589 INFO:teuthology.orchestra.run.smithi059.stdout: "paxos_trim_min": "250", 2024-05-08T08:52:09.589 INFO:teuthology.orchestra.run.smithi059.stdout: "perf": "true", 2024-05-08T08:52:09.589 INFO:teuthology.orchestra.run.smithi059.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-05-08T08:52:09.589 INFO:teuthology.orchestra.run.smithi059.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-05-08T08:52:09.590 INFO:teuthology.orchestra.run.smithi059.stdout: "plugin_dir": "/usr/lib64/ceph", 2024-05-08T08:52:09.590 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "-", 2024-05-08T08:52:09.590 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrv": "", 2024-05-08T08:52:09.590 INFO:teuthology.orchestra.run.smithi059.stdout: "public_bind_addr": "-", 2024-05-08T08:52:09.590 INFO:teuthology.orchestra.run.smithi059.stdout: "public_network": "", 2024-05-08T08:52:09.590 INFO:teuthology.orchestra.run.smithi059.stdout: "public_network_interface": "", 2024-05-08T08:52:09.590 INFO:teuthology.orchestra.run.smithi059.stdout: "qat_compressor_enabled": "false", 2024-05-08T08:52:09.590 INFO:teuthology.orchestra.run.smithi059.stdout: "qat_compressor_session_max_number": "256", 2024-05-08T08:52:09.590 INFO:teuthology.orchestra.run.smithi059.stdout: "rados_mon_op_timeout": "0", 2024-05-08T08:52:09.590 INFO:teuthology.orchestra.run.smithi059.stdout: "rados_osd_op_timeout": "0", 2024-05-08T08:52:09.590 INFO:teuthology.orchestra.run.smithi059.stdout: "rados_tracing": "false", 2024-05-08T08:52:09.590 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_atime_update_interval": "60", 2024-05-08T08:52:09.591 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-05-08T08:52:09.591 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_balance_parent_reads": "false", 2024-05-08T08:52:09.591 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_balance_snap_reads": "false", 2024-05-08T08:52:09.591 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_blkin_trace_all": "false", 2024-05-08T08:52:09.591 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_blocklist_expire_seconds": "0", 2024-05-08T08:52:09.591 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_blocklist_on_break_lock": "true", 2024-05-08T08:52:09.591 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_cache": "true", 2024-05-08T08:52:09.591 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_cache_block_writes_upfront": "false", 2024-05-08T08:52:09.591 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_cache_max_dirty": "25165824", 2024-05-08T08:52:09.591 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-05-08T08:52:09.591 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_cache_max_dirty_object": "0", 2024-05-08T08:52:09.592 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_cache_policy": "writearound", 2024-05-08T08:52:09.592 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_cache_size": "33554432", 2024-05-08T08:52:09.592 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_cache_target_dirty": "16777216", 2024-05-08T08:52:09.592 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-05-08T08:52:09.592 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_clone_copy_on_read": "false", 2024-05-08T08:52:09.592 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_compression_hint": "none", 2024-05-08T08:52:09.592 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_concurrent_management_ops": "10", 2024-05-08T08:52:09.592 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-05-08T08:52:09.592 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_default_clone_format": "auto", 2024-05-08T08:52:09.592 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_default_data_pool": "", 2024-05-08T08:52:09.592 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_default_features": "61", 2024-05-08T08:52:09.592 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_default_format": "2", 2024-05-08T08:52:09.592 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_default_map_options": "", 2024-05-08T08:52:09.592 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_default_order": "22", 2024-05-08T08:52:09.593 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_default_pool": "rbd", 2024-05-08T08:52:09.593 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-05-08T08:52:09.593 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_default_stripe_count": "0", 2024-05-08T08:52:09.593 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_default_stripe_unit": "0", 2024-05-08T08:52:09.593 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_disable_zero_copy_writes": "true", 2024-05-08T08:52:09.593 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_discard_granularity_bytes": "65536", 2024-05-08T08:52:09.593 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-05-08T08:52:09.593 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_enable_alloc_hint": "true", 2024-05-08T08:52:09.593 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-05-08T08:52:09.593 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_io_scheduler": "simple", 2024-05-08T08:52:09.594 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-05-08T08:52:09.594 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_journal_commit_age": "5.000000", 2024-05-08T08:52:09.594 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-05-08T08:52:09.594 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-05-08T08:52:09.594 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-05-08T08:52:09.594 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-05-08T08:52:09.594 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_journal_object_flush_interval": "0", 2024-05-08T08:52:09.594 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-05-08T08:52:09.594 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-05-08T08:52:09.595 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_journal_order": "24", 2024-05-08T08:52:09.595 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_journal_pool": "", 2024-05-08T08:52:09.595 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_journal_splay_width": "4", 2024-05-08T08:52:09.595 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_localize_parent_reads": "false", 2024-05-08T08:52:09.595 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_localize_snap_reads": "false", 2024-05-08T08:52:09.595 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-05-08T08:52:09.595 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-05-08T08:52:09.595 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-05-08T08:52:09.595 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-05-08T08:52:09.596 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-05-08T08:52:09.596 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-05-08T08:52:09.596 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_mirror_image_policy_type": "simple", 2024-05-08T08:52:09.596 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-05-08T08:52:09.596 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-05-08T08:52:09.596 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-05-08T08:52:09.596 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-05-08T08:52:09.596 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-05-08T08:52:09.596 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-05-08T08:52:09.596 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-05-08T08:52:09.596 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_mirror_memory_autotune": "true", 2024-05-08T08:52:09.596 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_mirror_memory_base": "805306368", 2024-05-08T08:52:09.596 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-05-08T08:52:09.597 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-05-08T08:52:09.597 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-05-08T08:52:09.597 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-05-08T08:52:09.597 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_mirror_memory_target": "4294967296", 2024-05-08T08:52:09.597 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-05-08T08:52:09.597 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-05-08T08:52:09.597 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-05-08T08:52:09.597 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_mirroring_delete_delay": "0", 2024-05-08T08:52:09.597 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-05-08T08:52:09.597 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_mirroring_replay_delay": "0", 2024-05-08T08:52:09.597 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-05-08T08:52:09.597 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-05-08T08:52:09.597 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_move_to_trash_on_remove": "false", 2024-05-08T08:52:09.597 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-05-08T08:52:09.597 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_mtime_update_interval": "60", 2024-05-08T08:52:09.597 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_non_blocking_aio": "true", 2024-05-08T08:52:09.598 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_op_thread_timeout": "60", 2024-05-08T08:52:09.598 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_op_threads": "1", 2024-05-08T08:52:09.598 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_parent_cache_enabled": "false", 2024-05-08T08:52:09.598 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_persistent_cache_mode": "disabled", 2024-05-08T08:52:09.598 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_persistent_cache_path": "/tmp", 2024-05-08T08:52:09.598 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_persistent_cache_size": "1073741824", 2024-05-08T08:52:09.598 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_plugins": "", 2024-05-08T08:52:09.598 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_qos_bps_burst": "0", 2024-05-08T08:52:09.598 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-05-08T08:52:09.598 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_qos_bps_limit": "0", 2024-05-08T08:52:09.598 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_qos_exclude_ops": "0", 2024-05-08T08:52:09.598 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_qos_iops_burst": "0", 2024-05-08T08:52:09.599 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-05-08T08:52:09.599 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_qos_iops_limit": "0", 2024-05-08T08:52:09.599 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_qos_read_bps_burst": "0", 2024-05-08T08:52:09.599 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-05-08T08:52:09.599 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_qos_read_bps_limit": "0", 2024-05-08T08:52:09.599 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_qos_read_iops_burst": "0", 2024-05-08T08:52:09.599 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-05-08T08:52:09.599 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_qos_read_iops_limit": "0", 2024-05-08T08:52:09.599 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_qos_schedule_tick_min": "50", 2024-05-08T08:52:09.599 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_qos_write_bps_burst": "0", 2024-05-08T08:52:09.599 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-05-08T08:52:09.599 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_qos_write_bps_limit": "0", 2024-05-08T08:52:09.600 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_qos_write_iops_burst": "0", 2024-05-08T08:52:09.600 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-05-08T08:52:09.600 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_qos_write_iops_limit": "0", 2024-05-08T08:52:09.600 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_quiesce_notification_attempts": "10", 2024-05-08T08:52:09.600 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_read_from_replica_policy": "default", 2024-05-08T08:52:09.600 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-05-08T08:52:09.600 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_readahead_max_bytes": "524288", 2024-05-08T08:52:09.600 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_readahead_trigger_requests": "10", 2024-05-08T08:52:09.600 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_request_timed_out_seconds": "30", 2024-05-08T08:52:09.600 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_skip_partial_discard": "true", 2024-05-08T08:52:09.600 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-05-08T08:52:09.600 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_tracing": "false", 2024-05-08T08:52:09.601 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_validate_names": "true", 2024-05-08T08:52:09.601 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_validate_pool": "true", 2024-05-08T08:52:09.601 INFO:teuthology.orchestra.run.smithi059.stdout: "restapi_base_url": "", 2024-05-08T08:52:09.601 INFO:teuthology.orchestra.run.smithi059.stdout: "restapi_log_level": "", 2024-05-08T08:52:09.601 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_account_default_quota_max_objects": "-1", 2024-05-08T08:52:09.601 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_account_default_quota_max_size": "-1", 2024-05-08T08:52:09.601 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_acl_grants_max_num": "100", 2024-05-08T08:52:09.601 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_admin_entry": "admin", 2024-05-08T08:52:09.601 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-05-08T08:52:09.601 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_backend_store": "rados", 2024-05-08T08:52:09.601 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_barbican_url": "", 2024-05-08T08:52:09.601 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_beast_enable_async": "true", 2024-05-08T08:52:09.601 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_bucket_counters_cache": "false", 2024-05-08T08:52:09.601 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_bucket_counters_cache_size": "10000", 2024-05-08T08:52:09.602 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-05-08T08:52:09.602 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-05-08T08:52:09.602 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_bucket_index_max_aio": "128", 2024-05-08T08:52:09.602 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-05-08T08:52:09.602 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-05-08T08:52:09.602 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_bucket_quota_ttl": "600", 2024-05-08T08:52:09.602 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-05-08T08:52:09.602 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_cache_enabled": "true", 2024-05-08T08:52:09.602 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_cache_expiry_interval": "900", 2024-05-08T08:52:09.602 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_cache_lru_size": "25000", 2024-05-08T08:52:09.602 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_config_store": "rados", 2024-05-08T08:52:09.602 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_content_length_compat": "false", 2024-05-08T08:52:09.602 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_copy_obj_progress": "true", 2024-05-08T08:52:09.603 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-05-08T08:52:09.603 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_cors_rules_max_num": "100", 2024-05-08T08:52:09.603 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_cross_domain_policy": "", 2024-05-08T08:52:09.603 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_crypt_default_encryption_key": "", 2024-05-08T08:52:09.603 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_crypt_kmip_addr": "", 2024-05-08T08:52:09.603 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_crypt_kmip_ca_path": "", 2024-05-08T08:52:09.603 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_crypt_kmip_client_cert": "", 2024-05-08T08:52:09.603 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_crypt_kmip_client_key": "", 2024-05-08T08:52:09.603 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-05-08T08:52:09.603 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_crypt_kmip_password": "", 2024-05-08T08:52:09.603 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-05-08T08:52:09.603 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_crypt_kmip_username": "", 2024-05-08T08:52:09.604 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_crypt_require_ssl": "true", 2024-05-08T08:52:09.604 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-05-08T08:52:09.604 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-05-08T08:52:09.604 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-05-08T08:52:09.604 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-05-08T08:52:09.604 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-05-08T08:52:09.604 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-05-08T08:52:09.604 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-05-08T08:52:09.604 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-05-08T08:52:09.604 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-05-08T08:52:09.604 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-05-08T08:52:09.604 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-05-08T08:52:09.604 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-05-08T08:52:09.604 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-05-08T08:52:09.605 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-05-08T08:52:09.605 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_crypt_suppress_logs": "true", 2024-05-08T08:52:09.605 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_crypt_vault_addr": "", 2024-05-08T08:52:09.605 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_crypt_vault_auth": "token", 2024-05-08T08:52:09.605 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_crypt_vault_namespace": "", 2024-05-08T08:52:09.605 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_crypt_vault_prefix": "", 2024-05-08T08:52:09.605 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-05-08T08:52:09.605 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-05-08T08:52:09.605 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-05-08T08:52:09.605 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-05-08T08:52:09.605 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_crypt_vault_token_file": "", 2024-05-08T08:52:09.605 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-05-08T08:52:09.605 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_curl_buffersize": "524288", 2024-05-08T08:52:09.605 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_curl_low_speed_limit": "1024", 2024-05-08T08:52:09.605 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_curl_low_speed_time": "300", 2024-05-08T08:52:09.606 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_curl_tcp_keepalive": "0", 2024-05-08T08:52:09.606 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-05-08T08:52:09.606 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-05-08T08:52:09.606 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-05-08T08:52:09.606 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-05-08T08:52:09.606 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-05-08T08:52:09.606 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_d3n_l1_fadvise": "4", 2024-05-08T08:52:09.606 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-05-08T08:52:09.606 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-05-08T08:52:09.606 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-05-08T08:52:09.606 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_d4n_host": "127.0.0.1", 2024-05-08T08:52:09.606 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_d4n_port": "6379", 2024-05-08T08:52:09.606 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-05-08T08:52:09.606 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_data_log_changes_size": "1000", 2024-05-08T08:52:09.607 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_data_log_num_shards": "128", 2024-05-08T08:52:09.607 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-05-08T08:52:09.607 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_data_log_window": "30", 2024-05-08T08:52:09.607 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_data_notify_interval_msec": "0", 2024-05-08T08:52:09.607 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_data_sync_poll_interval": "20", 2024-05-08T08:52:09.607 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_data_sync_spawn_window": "20", 2024-05-08T08:52:09.607 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_debug_inject_latency_bi_unlink": "0", 2024-05-08T08:52:09.607 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-05-08T08:52:09.607 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-05-08T08:52:09.607 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_default_data_log_backing": "fifo", 2024-05-08T08:52:09.607 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-05-08T08:52:09.607 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_default_region_info_oid": "default.region", 2024-05-08T08:52:09.607 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-05-08T08:52:09.608 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-05-08T08:52:09.608 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_defer_to_bucket_acls": "", 2024-05-08T08:52:09.608 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-05-08T08:52:09.608 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-05-08T08:52:09.608 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-05-08T08:52:09.608 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-05-08T08:52:09.608 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-05-08T08:52:09.608 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-05-08T08:52:09.608 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-05-08T08:52:09.608 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-05-08T08:52:09.608 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_dmclock_data_res": "500.000000", 2024-05-08T08:52:09.608 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-05-08T08:52:09.608 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-05-08T08:52:09.609 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-05-08T08:52:09.609 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-05-08T08:52:09.609 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_dns_name": "", 2024-05-08T08:52:09.609 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_dns_s3website_name": "", 2024-05-08T08:52:09.609 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_dynamic_resharding": "true", 2024-05-08T08:52:09.609 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-05-08T08:52:09.609 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_enable_gc_threads": "true", 2024-05-08T08:52:09.609 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_enable_lc_threads": "true", 2024-05-08T08:52:09.609 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_enable_ops_log": "false", 2024-05-08T08:52:09.609 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_enable_quota_threads": "true", 2024-05-08T08:52:09.609 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_enable_static_website": "false", 2024-05-08T08:52:09.609 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_enable_usage_log": "false", 2024-05-08T08:52:09.609 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_enforce_swift_acls": "true", 2024-05-08T08:52:09.609 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_exit_timeout_secs": "120", 2024-05-08T08:52:09.610 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_expose_bucket": "false", 2024-05-08T08:52:09.610 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_extended_http_attrs": "", 2024-05-08T08:52:09.610 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_filter": "none", 2024-05-08T08:52:09.610 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-05-08T08:52:09.610 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_frontends": "beast port=7480", 2024-05-08T08:52:09.610 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_gc_max_concurrent_io": "10", 2024-05-08T08:52:09.610 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_gc_max_deferred": "50", 2024-05-08T08:52:09.610 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-05-08T08:52:09.610 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_gc_max_objs": "32", 2024-05-08T08:52:09.610 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_gc_max_queue_size": "134213632", 2024-05-08T08:52:09.610 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_gc_max_trim_chunk": "16", 2024-05-08T08:52:09.610 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_gc_obj_min_wait": "7200", 2024-05-08T08:52:09.611 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_gc_processor_max_time": "3600", 2024-05-08T08:52:09.611 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_gc_processor_period": "3600", 2024-05-08T08:52:09.611 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-05-08T08:52:09.611 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_get_obj_window_size": "16777216", 2024-05-08T08:52:09.611 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_healthcheck_disabling_path": "", 2024-05-08T08:52:09.611 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_ignore_get_invalid_range": "false", 2024-05-08T08:52:09.611 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_init_timeout": "300", 2024-05-08T08:52:09.611 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-05-08T08:52:09.611 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-05-08T08:52:09.611 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_kafka_connection_idle": "300", 2024-05-08T08:52:09.611 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_kafka_message_timeout": "5000", 2024-05-08T08:52:09.611 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_kafka_sleep_timeout": "10", 2024-05-08T08:52:09.611 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-05-08T08:52:09.612 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_keystone_accepted_reader_roles": "", 2024-05-08T08:52:09.612 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-05-08T08:52:09.612 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_keystone_admin_domain": "", 2024-05-08T08:52:09.612 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_keystone_admin_password": "", 2024-05-08T08:52:09.612 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_keystone_admin_password_path": "", 2024-05-08T08:52:09.612 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_keystone_admin_project": "", 2024-05-08T08:52:09.612 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_keystone_admin_tenant": "", 2024-05-08T08:52:09.612 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_keystone_admin_token": "", 2024-05-08T08:52:09.612 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_keystone_admin_token_path": "", 2024-05-08T08:52:09.612 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_keystone_admin_user": "", 2024-05-08T08:52:09.612 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_keystone_api_version": "2", 2024-05-08T08:52:09.612 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_keystone_barbican_domain": "", 2024-05-08T08:52:09.612 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_keystone_barbican_password": "", 2024-05-08T08:52:09.612 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_keystone_barbican_project": "", 2024-05-08T08:52:09.613 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_keystone_barbican_tenant": "", 2024-05-08T08:52:09.613 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_keystone_barbican_user": "", 2024-05-08T08:52:09.613 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-05-08T08:52:09.613 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_keystone_implicit_tenants": "false", 2024-05-08T08:52:09.613 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-05-08T08:52:09.613 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_keystone_service_token_enabled": "false", 2024-05-08T08:52:09.613 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_keystone_token_cache_size": "10000", 2024-05-08T08:52:09.613 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_keystone_url": "", 2024-05-08T08:52:09.613 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_keystone_verify_ssl": "true", 2024-05-08T08:52:09.613 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_lc_debug_interval": "-1", 2024-05-08T08:52:09.613 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_lc_lock_max_time": "90", 2024-05-08T08:52:09.614 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_lc_max_objs": "32", 2024-05-08T08:52:09.614 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_lc_max_rules": "1000", 2024-05-08T08:52:09.614 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_lc_max_worker": "3", 2024-05-08T08:52:09.614 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_lc_max_wp_worker": "3", 2024-05-08T08:52:09.614 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_lc_thread_delay": "0", 2024-05-08T08:52:09.614 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-05-08T08:52:09.614 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_ldap_dnattr": "uid", 2024-05-08T08:52:09.614 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-05-08T08:52:09.614 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_ldap_searchfilter": "", 2024-05-08T08:52:09.614 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-05-08T08:52:09.614 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_ldap_uri": "", 2024-05-08T08:52:09.614 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-05-08T08:52:09.615 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-05-08T08:52:09.615 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-05-08T08:52:09.615 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_log_http_headers": "", 2024-05-08T08:52:09.615 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_log_nonexistent_bucket": "false", 2024-05-08T08:52:09.615 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-05-08T08:52:09.615 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_log_object_name_utc": "false", 2024-05-08T08:52:09.615 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_lua_max_memory_per_state": "128000", 2024-05-08T08:52:09.615 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_luarocks_location": "/tmp/rgw_luarocks/mon.a", 2024-05-08T08:52:09.615 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_max_attr_name_len": "0", 2024-05-08T08:52:09.615 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_max_attr_size": "0", 2024-05-08T08:52:09.615 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_max_attrs_num_in_req": "0", 2024-05-08T08:52:09.615 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_max_chunk_size": "4194304", 2024-05-08T08:52:09.616 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_max_concurrent_requests": "1024", 2024-05-08T08:52:09.616 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-05-08T08:52:09.616 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_max_dynamic_shards": "1999", 2024-05-08T08:52:09.616 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_max_listing_results": "1000", 2024-05-08T08:52:09.616 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_max_notify_retries": "10", 2024-05-08T08:52:09.616 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_max_objs_per_shard": "100000", 2024-05-08T08:52:09.616 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_max_put_param_size": "1048576", 2024-05-08T08:52:09.616 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_max_put_size": "5368709120", 2024-05-08T08:52:09.616 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_max_slo_entries": "1000", 2024-05-08T08:52:09.616 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_md_log_max_shards": "64", 2024-05-08T08:52:09.616 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_md_notify_interval_msec": "200", 2024-05-08T08:52:09.616 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_meta_sync_poll_interval": "20", 2024-05-08T08:52:09.616 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_meta_sync_spawn_window": "20", 2024-05-08T08:52:09.617 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-05-08T08:52:09.617 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_mp_lock_max_time": "600", 2024-05-08T08:52:09.617 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-05-08T08:52:09.617 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_multipart_min_part_size": "5242880", 2024-05-08T08:52:09.617 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-05-08T08:52:09.617 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-05-08T08:52:09.617 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_nfs_fhcache_size": "2017", 2024-05-08T08:52:09.617 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-05-08T08:52:09.617 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-05-08T08:52:09.617 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_nfs_lru_lanes": "5", 2024-05-08T08:52:09.617 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_nfs_max_gc": "300", 2024-05-08T08:52:09.617 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-05-08T08:52:09.617 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_nfs_run_gc_threads": "false", 2024-05-08T08:52:09.618 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_nfs_run_lc_threads": "false", 2024-05-08T08:52:09.618 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_nfs_run_quota_threads": "false", 2024-05-08T08:52:09.618 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_nfs_run_sync_thread": "false", 2024-05-08T08:52:09.618 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-05-08T08:52:09.618 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-05-08T08:52:09.618 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_num_async_rados_threads": "32", 2024-05-08T08:52:09.618 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_num_control_oids": "8", 2024-05-08T08:52:09.618 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_numa_node": "-1", 2024-05-08T08:52:09.618 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_obj_stripe_size": "4194304", 2024-05-08T08:52:09.618 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-05-08T08:52:09.618 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_objexp_chunk_size": "100", 2024-05-08T08:52:09.618 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_objexp_gc_interval": "600", 2024-05-08T08:52:09.619 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_objexp_hints_num_shards": "127", 2024-05-08T08:52:09.619 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-05-08T08:52:09.619 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-05-08T08:52:09.619 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_op_thread_timeout": "600", 2024-05-08T08:52:09.619 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_op_tracing": "false", 2024-05-08T08:52:09.619 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_opa_token": "", 2024-05-08T08:52:09.619 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_opa_url": "", 2024-05-08T08:52:09.619 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_opa_verify_ssl": "true", 2024-05-08T08:52:09.619 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-05-08T08:52:09.619 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-05-08T08:52:09.619 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_ops_log_rados": "false", 2024-05-08T08:52:09.619 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_ops_log_socket_path": "", 2024-05-08T08:52:09.619 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-05-08T08:52:09.620 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-05-08T08:52:09.620 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-05-08T08:52:09.620 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_period_push_interval": "2.000000", 2024-05-08T08:52:09.620 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_period_push_interval_max": "30.000000", 2024-05-08T08:52:09.620 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_period_root_pool": ".rgw.root", 2024-05-08T08:52:09.620 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-05-08T08:52:09.620 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_posix_base_path": "/tmp/rgw_posix_driver", 2024-05-08T08:52:09.620 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_posix_cache_lanes": "3", 2024-05-08T08:52:09.620 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_posix_cache_lmdb_count": "3", 2024-05-08T08:52:09.620 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_posix_cache_max_buckets": "100", 2024-05-08T08:52:09.620 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_posix_cache_partitions": "3", 2024-05-08T08:52:09.620 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_posix_database_root": "/var/lib/ceph/radosgw", 2024-05-08T08:52:09.620 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_print_continue": "true", 2024-05-08T08:52:09.621 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_print_prohibited_content_length": "false", 2024-05-08T08:52:09.621 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-05-08T08:52:09.621 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-05-08T08:52:09.621 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-05-08T08:52:09.621 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-05-08T08:52:09.621 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_rados_tracing": "false", 2024-05-08T08:52:09.621 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_realm": "", 2024-05-08T08:52:09.621 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_realm_id": "", 2024-05-08T08:52:09.621 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-05-08T08:52:09.621 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_region": "", 2024-05-08T08:52:09.621 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_region_root_pool": ".rgw.root", 2024-05-08T08:52:09.621 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_relaxed_region_enforcement": "false", 2024-05-08T08:52:09.621 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-05-08T08:52:09.622 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-05-08T08:52:09.622 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_request_uri": "", 2024-05-08T08:52:09.622 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_reshard_batch_size": "64", 2024-05-08T08:52:09.622 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-05-08T08:52:09.622 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_reshard_max_aio": "128", 2024-05-08T08:52:09.622 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_reshard_num_logs": "16", 2024-05-08T08:52:09.622 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_reshard_thread_interval": "600", 2024-05-08T08:52:09.622 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_resolve_cname": "false", 2024-05-08T08:52:09.622 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_rest_getusage_op_compat": "false", 2024-05-08T08:52:09.622 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_run_sync_thread": "true", 2024-05-08T08:52:09.622 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_s3_auth_disable_signature_url": "false", 2024-05-08T08:52:09.622 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-05-08T08:52:09.622 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_s3_auth_use_keystone": "false", 2024-05-08T08:52:09.623 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_s3_auth_use_ldap": "false", 2024-05-08T08:52:09.623 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_s3_auth_use_rados": "true", 2024-05-08T08:52:09.623 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_s3_auth_use_sts": "false", 2024-05-08T08:52:09.623 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-05-08T08:52:09.623 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_s3_success_create_obj_status": "0", 2024-05-08T08:52:09.623 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-05-08T08:52:09.623 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_scheduler_type": "throttler", 2024-05-08T08:52:09.623 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_script_uri": "", 2024-05-08T08:52:09.623 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_service_provider_name": "", 2024-05-08T08:52:09.624 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-05-08T08:52:09.624 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_sts_client_id": "", 2024-05-08T08:52:09.624 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_sts_client_secret": "", 2024-05-08T08:52:09.624 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_sts_entry": "sts", 2024-05-08T08:52:09.624 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_sts_key": "sts", 2024-05-08T08:52:09.624 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_sts_max_session_duration": "43200", 2024-05-08T08:52:09.624 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_sts_min_session_duration": "900", 2024-05-08T08:52:09.624 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_sts_token_introspection_url": "", 2024-05-08T08:52:09.624 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_swift_account_in_url": "false", 2024-05-08T08:52:09.624 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_swift_auth_entry": "auth", 2024-05-08T08:52:09.624 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_swift_auth_url": "", 2024-05-08T08:52:09.624 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_swift_custom_header": "", 2024-05-08T08:52:09.624 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_swift_enforce_content_length": "false", 2024-05-08T08:52:09.625 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_swift_need_stats": "true", 2024-05-08T08:52:09.625 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_swift_tenant_name": "", 2024-05-08T08:52:09.625 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_swift_token_expiration": "86400", 2024-05-08T08:52:09.625 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_swift_url": "", 2024-05-08T08:52:09.625 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_swift_url_prefix": "swift", 2024-05-08T08:52:09.625 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_swift_versioning_enabled": "false", 2024-05-08T08:52:09.625 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-05-08T08:52:09.625 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-05-08T08:52:09.625 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_sync_lease_period": "120", 2024-05-08T08:52:09.625 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-05-08T08:52:09.625 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_sync_log_trim_interval": "1200", 2024-05-08T08:52:09.625 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-05-08T08:52:09.625 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-05-08T08:52:09.626 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-05-08T08:52:09.626 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_sync_obj_etag_verify": "false", 2024-05-08T08:52:09.626 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_sync_trace_history_size": "4096", 2024-05-08T08:52:09.626 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-05-08T08:52:09.626 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-05-08T08:52:09.626 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_thread_pool_size": "512", 2024-05-08T08:52:09.626 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_topic_persistency_max_retries": "0", 2024-05-08T08:52:09.626 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_topic_persistency_sleep_duration": "0", 2024-05-08T08:52:09.626 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_topic_persistency_time_to_live": "0", 2024-05-08T08:52:09.626 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_topic_require_publish_policy": "false", 2024-05-08T08:52:09.626 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_torrent_comment": "", 2024-05-08T08:52:09.626 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_torrent_createby": "", 2024-05-08T08:52:09.626 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_torrent_encoding": "", 2024-05-08T08:52:09.627 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_torrent_flag": "false", 2024-05-08T08:52:09.627 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_torrent_max_size": "5368709120", 2024-05-08T08:52:09.627 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_torrent_origin": "", 2024-05-08T08:52:09.627 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_torrent_sha_unit": "524288", 2024-05-08T08:52:09.627 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_torrent_tracker": "", 2024-05-08T08:52:09.627 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_trust_forwarded_https": "false", 2024-05-08T08:52:09.627 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-05-08T08:52:09.627 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_usage_log_tick_interval": "30", 2024-05-08T08:52:09.627 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_usage_max_shards": "32", 2024-05-08T08:52:09.627 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_usage_max_user_shards": "1", 2024-05-08T08:52:09.627 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_use_opa_authz": "false", 2024-05-08T08:52:09.627 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_user_counters_cache": "false", 2024-05-08T08:52:09.627 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_user_counters_cache_size": "10000", 2024-05-08T08:52:09.627 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-05-08T08:52:09.628 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_user_default_quota_max_size": "-1", 2024-05-08T08:52:09.628 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_user_max_buckets": "1000", 2024-05-08T08:52:09.628 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_user_policies_max_num": "100", 2024-05-08T08:52:09.628 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-05-08T08:52:09.628 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-05-08T08:52:09.628 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_user_quota_sync_interval": "86400", 2024-05-08T08:52:09.628 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-05-08T08:52:09.628 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_user_unique_email": "true", 2024-05-08T08:52:09.628 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_verify_ssl": "true", 2024-05-08T08:52:09.628 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_website_routing_rules_max_num": "50", 2024-05-08T08:52:09.628 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_zone": "", 2024-05-08T08:52:09.628 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_zone_id": "", 2024-05-08T08:52:09.628 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-05-08T08:52:09.628 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_zonegroup": "", 2024-05-08T08:52:09.629 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_zonegroup_id": "", 2024-05-08T08:52:09.629 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-05-08T08:52:09.629 INFO:teuthology.orchestra.run.smithi059.stdout: "rgwlc_auto_session_clear": "true", 2024-05-08T08:52:09.629 INFO:teuthology.orchestra.run.smithi059.stdout: "rgwlc_skip_bucket_step": "false", 2024-05-08T08:52:09.629 INFO:teuthology.orchestra.run.smithi059.stdout: "rocksdb_block_size": "4096", 2024-05-08T08:52:09.629 INFO:teuthology.orchestra.run.smithi059.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-05-08T08:52:09.629 INFO:teuthology.orchestra.run.smithi059.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-05-08T08:52:09.629 INFO:teuthology.orchestra.run.smithi059.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-05-08T08:52:09.629 INFO:teuthology.orchestra.run.smithi059.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-05-08T08:52:09.629 INFO:teuthology.orchestra.run.smithi059.stdout: "rocksdb_cache_shard_bits": "4", 2024-05-08T08:52:09.629 INFO:teuthology.orchestra.run.smithi059.stdout: "rocksdb_cache_size": "536870912", 2024-05-08T08:52:09.629 INFO:teuthology.orchestra.run.smithi059.stdout: "rocksdb_cache_type": "binned_lru", 2024-05-08T08:52:09.629 INFO:teuthology.orchestra.run.smithi059.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-05-08T08:52:09.630 INFO:teuthology.orchestra.run.smithi059.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-05-08T08:52:09.630 INFO:teuthology.orchestra.run.smithi059.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-05-08T08:52:09.630 INFO:teuthology.orchestra.run.smithi059.stdout: "rocksdb_collect_compaction_stats": "false", 2024-05-08T08:52:09.630 INFO:teuthology.orchestra.run.smithi059.stdout: "rocksdb_collect_extended_stats": "false", 2024-05-08T08:52:09.630 INFO:teuthology.orchestra.run.smithi059.stdout: "rocksdb_collect_memory_stats": "false", 2024-05-08T08:52:09.630 INFO:teuthology.orchestra.run.smithi059.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-05-08T08:52:09.630 INFO:teuthology.orchestra.run.smithi059.stdout: "rocksdb_index_type": "binary_search", 2024-05-08T08:52:09.630 INFO:teuthology.orchestra.run.smithi059.stdout: "rocksdb_log_to_ceph_log": "true", 2024-05-08T08:52:09.630 INFO:teuthology.orchestra.run.smithi059.stdout: "rocksdb_metadata_block_size": "4096", 2024-05-08T08:52:09.630 INFO:teuthology.orchestra.run.smithi059.stdout: "rocksdb_partition_filters": "false", 2024-05-08T08:52:09.630 INFO:teuthology.orchestra.run.smithi059.stdout: "rocksdb_perf": "false", 2024-05-08T08:52:09.630 INFO:teuthology.orchestra.run.smithi059.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-05-08T08:52:09.630 INFO:teuthology.orchestra.run.smithi059.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-05-08T08:52:09.631 INFO:teuthology.orchestra.run.smithi059.stdout: "rotating_keys_renewal_timeout": "10", 2024-05-08T08:52:09.631 INFO:teuthology.orchestra.run.smithi059.stdout: "run_dir": "/var/run/ceph", 2024-05-08T08:52:09.631 INFO:teuthology.orchestra.run.smithi059.stdout: "seastore_block_create": "true", 2024-05-08T08:52:09.631 INFO:teuthology.orchestra.run.smithi059.stdout: "seastore_cache_lru_size": "67108864", 2024-05-08T08:52:09.631 INFO:teuthology.orchestra.run.smithi059.stdout: "seastore_cbjournal_size": "5368709120", 2024-05-08T08:52:09.631 INFO:teuthology.orchestra.run.smithi059.stdout: "seastore_data_delta_based_overwrite": "0", 2024-05-08T08:52:09.631 INFO:teuthology.orchestra.run.smithi059.stdout: "seastore_default_max_object_size": "16777216", 2024-05-08T08:52:09.631 INFO:teuthology.orchestra.run.smithi059.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-05-08T08:52:09.631 INFO:teuthology.orchestra.run.smithi059.stdout: "seastore_device_size": "53687091200", 2024-05-08T08:52:09.631 INFO:teuthology.orchestra.run.smithi059.stdout: "seastore_journal_batch_capacity": "16", 2024-05-08T08:52:09.631 INFO:teuthology.orchestra.run.smithi059.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-05-08T08:52:09.631 INFO:teuthology.orchestra.run.smithi059.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-05-08T08:52:09.631 INFO:teuthology.orchestra.run.smithi059.stdout: "seastore_journal_iodepth_limit": "5", 2024-05-08T08:52:09.632 INFO:teuthology.orchestra.run.smithi059.stdout: "seastore_main_device_type": "SSD", 2024-05-08T08:52:09.632 INFO:teuthology.orchestra.run.smithi059.stdout: "seastore_max_concurrent_transactions": "8", 2024-05-08T08:52:09.632 INFO:teuthology.orchestra.run.smithi059.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-05-08T08:52:09.632 INFO:teuthology.orchestra.run.smithi059.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-05-08T08:52:09.632 INFO:teuthology.orchestra.run.smithi059.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-05-08T08:52:09.632 INFO:teuthology.orchestra.run.smithi059.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-05-08T08:52:09.632 INFO:teuthology.orchestra.run.smithi059.stdout: "seastore_segment_size": "67108864", 2024-05-08T08:52:09.632 INFO:teuthology.orchestra.run.smithi059.stdout: "set_keepcaps": "false", 2024-05-08T08:52:09.632 INFO:teuthology.orchestra.run.smithi059.stdout: "setgroup": "", 2024-05-08T08:52:09.632 INFO:teuthology.orchestra.run.smithi059.stdout: "setuser": "", 2024-05-08T08:52:09.632 INFO:teuthology.orchestra.run.smithi059.stdout: "setuser_match_path": "", 2024-05-08T08:52:09.632 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_misplaced_ratio": "0.050000", 2024-05-08T08:52:09.632 INFO:teuthology.orchestra.run.smithi059.stdout: "thp": "false", 2024-05-08T08:52:09.632 INFO:teuthology.orchestra.run.smithi059.stdout: "threadpool_default_timeout": "60", 2024-05-08T08:52:09.632 INFO:teuthology.orchestra.run.smithi059.stdout: "threadpool_empty_queue_max_wait": "2", 2024-05-08T08:52:09.633 INFO:teuthology.orchestra.run.smithi059.stdout: "throttler_perf_counter": "true" 2024-05-08T08:52:09.633 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:52:09.635 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-05-08T08:52:09.751 INFO:teuthology.orchestra.run.smithi059.stdout:mon.a: {} 2024-05-08T08:52:09.752 INFO:teuthology.orchestra.run.smithi059.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-05-08T08:52:09.762 INFO:teuthology.orchestra.run.smithi059.stdout:mon.b: {} 2024-05-08T08:52:09.762 INFO:teuthology.orchestra.run.smithi059.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-05-08T08:52:09.771 INFO:teuthology.orchestra.run.smithi059.stdout:mon.c: {} 2024-05-08T08:52:09.772 INFO:teuthology.orchestra.run.smithi059.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-05-08T08:52:09.783 DEBUG:teuthology.orchestra.run.smithi059:> type ceph-objectstore-tool 2024-05-08T08:52:09.798 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-05-08T08:52:09.798 DEBUG:teuthology.orchestra.run.smithi192:> type ceph-objectstore-tool 2024-05-08T08:52:09.817 INFO:teuthology.orchestra.run.smithi192.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-05-08T08:52:09.817 INFO:teuthology.run_tasks:Running task rbd_fsx... 2024-05-08T08:52:09.827 INFO:tasks.rbd_fsx:starting rbd_fsx... 2024-05-08T08:52:09.827 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-05-08T08:52:09.828 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-05-08T08:52:09.828 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T08:52:09.828 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '1000', 'type': 'test-reweight-by-utilization'} 2024-05-08T08:52:09.828 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 1000 0.05 2024-05-08T08:52:09.828 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-05-08T08:52:09.829 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-05-08T08:52:09.829 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T08:52:09.829 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-05-08T08:52:09.829 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:52:09.830 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-05-08T08:52:09.830 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-05-08T08:52:09.830 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 512 -w 512 -t 512 -h 512 -l 250000000 -S 0 -N 6000 -M pool_client.0 image_client.0 2024-05-08T08:52:09.840 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:52:09.843+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:52:09.893 INFO:teuthology.orchestra.run.smithi196.stdout:mapped writes DISABLED 2024-05-08T08:52:09.893 INFO:teuthology.orchestra.run.smithi196.stdout:mapped reads DISABLED 2024-05-08T08:52:09.893 INFO:teuthology.orchestra.run.smithi196.stdout:Seed set to 1807 2024-05-08T08:52:09.893 INFO:teuthology.orchestra.run.smithi196.stdout:rbd-nbd mode enabled 2024-05-08T08:52:09.931 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:09.934+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:09.962 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:52:09.963 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:52:09.963 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:52:09.963 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:52:09.972 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:52:09.978 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:52:09.979 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:52:10.016 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:52:10.016 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:52:10.032 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:52:10.035+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:52:10.052 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:52:10.052 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:52:10.070 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:52:10.071 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:52:10.086 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:10.086 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:10.089 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:52:10.089 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:52:10.095 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:52:10.108 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:52:10.109 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:52:10.165 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:52:10.168+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:52:10.192 INFO:teuthology.orchestra.run.smithi059.stdout:moved 0 / 18 (0%) 2024-05-08T08:52:10.192 INFO:teuthology.orchestra.run.smithi059.stdout:avg 3 2024-05-08T08:52:10.192 INFO:teuthology.orchestra.run.smithi059.stdout:stddev 1.52753 -> 1.52753 (expected baseline 1.58114) 2024-05-08T08:52:10.192 INFO:teuthology.orchestra.run.smithi059.stdout:min osd.4 with 1 -> 1 pgs (0.333333 -> 0.333333 * mean) 2024-05-08T08:52:10.193 INFO:teuthology.orchestra.run.smithi059.stdout:max osd.1 with 5 -> 5 pgs (1.66667 -> 1.66667 * mean) 2024-05-08T08:52:10.193 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:52:10.193 INFO:teuthology.orchestra.run.smithi059.stdout:oload 1000 2024-05-08T08:52:10.193 INFO:teuthology.orchestra.run.smithi059.stdout:max_change 0.05 2024-05-08T08:52:10.193 INFO:teuthology.orchestra.run.smithi059.stdout:max_change_osds 4 2024-05-08T08:52:10.193 INFO:teuthology.orchestra.run.smithi059.stdout:average_utilization 0.0003 2024-05-08T08:52:10.193 INFO:teuthology.orchestra.run.smithi059.stdout:overload_utilization 0.0029 2024-05-08T08:52:10.193 INFO:teuthology.orchestra.run.smithi059.stderr:no change 2024-05-08T08:52:10.205 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-05-08T08:52:10.205 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 1.0 2024-05-08T08:52:10.219 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:10.219 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:10.228 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:52:10.270 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:52:10.273+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:52:10.334 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:52:10.337+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:52:10.382 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:10.382 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:10.391 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:52:10.475 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:52:10.478+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:52:10.524 INFO:teuthology.orchestra.run.smithi059.stdout:moved 0 / 18 (0%) 2024-05-08T08:52:10.524 INFO:teuthology.orchestra.run.smithi059.stdout:avg 3 2024-05-08T08:52:10.524 INFO:teuthology.orchestra.run.smithi059.stdout:stddev 1.52753 -> 1.52753 (expected baseline 1.58114) 2024-05-08T08:52:10.524 INFO:teuthology.orchestra.run.smithi059.stdout:min osd.4 with 1 -> 1 pgs (0.333333 -> 0.333333 * mean) 2024-05-08T08:52:10.525 INFO:teuthology.orchestra.run.smithi059.stdout:max osd.1 with 5 -> 5 pgs (1.66667 -> 1.66667 * mean) 2024-05-08T08:52:10.525 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:52:10.525 INFO:teuthology.orchestra.run.smithi059.stdout:oload 110 2024-05-08T08:52:10.525 INFO:teuthology.orchestra.run.smithi059.stdout:max_change 1 2024-05-08T08:52:10.525 INFO:teuthology.orchestra.run.smithi059.stdout:max_change_osds 4 2024-05-08T08:52:10.525 INFO:teuthology.orchestra.run.smithi059.stdout:average_utilization 0.0003 2024-05-08T08:52:10.525 INFO:teuthology.orchestra.run.smithi059.stdout:overload_utilization 0.0003 2024-05-08T08:52:10.525 INFO:teuthology.orchestra.run.smithi059.stderr:no change 2024-05-08T08:52:10.536 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:10.540+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:10.537 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-05-08T08:52:10.538 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 3.0 2024-05-08T08:52:10.542 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:10.542 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:10.551 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:52:10.637 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:52:10.641+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:52:10.707 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:10.707 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:10.716 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:52:10.761 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:52:10.764+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:52:10.827 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:10.827 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:10.836 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:52:10.849 INFO:teuthology.orchestra.run.smithi059.stdout:moved 0 / 18 (0%) 2024-05-08T08:52:10.849 INFO:teuthology.orchestra.run.smithi059.stdout:avg 3 2024-05-08T08:52:10.849 INFO:teuthology.orchestra.run.smithi059.stdout:stddev 1.52753 -> 1.52753 (expected baseline 1.58114) 2024-05-08T08:52:10.850 INFO:teuthology.orchestra.run.smithi059.stdout:min osd.4 with 1 -> 1 pgs (0.333333 -> 0.333333 * mean) 2024-05-08T08:52:10.850 INFO:teuthology.orchestra.run.smithi059.stdout:max osd.1 with 5 -> 5 pgs (1.66667 -> 1.66667 * mean) 2024-05-08T08:52:10.850 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:52:10.850 INFO:teuthology.orchestra.run.smithi059.stdout:oload 110 2024-05-08T08:52:10.850 INFO:teuthology.orchestra.run.smithi059.stdout:max_change 3 2024-05-08T08:52:10.850 INFO:teuthology.orchestra.run.smithi059.stdout:max_change_osds 4 2024-05-08T08:52:10.850 INFO:teuthology.orchestra.run.smithi059.stdout:average_utilization 0.0003 2024-05-08T08:52:10.850 INFO:teuthology.orchestra.run.smithi059.stdout:overload_utilization 0.0003 2024-05-08T08:52:10.850 INFO:teuthology.orchestra.run.smithi059.stderr:no change 2024-05-08T08:52:10.851 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:52:10.853+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:52:10.863 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-05-08T08:52:10.863 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 0.05 2024-05-08T08:52:10.912 INFO:teuthology.orchestra.run.smithi059.stderr:noscrub is set 2024-05-08T08:52:10.967 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:10.967 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:10.967 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:52:10.970+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:52:10.977 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:52:11.040 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:52:11.043+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:52:11.128 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:11.128 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:11.137 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:52:11.140 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:11.143+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:11.209 INFO:teuthology.orchestra.run.smithi059.stdout:moved 0 / 82 (0%) 2024-05-08T08:52:11.209 INFO:teuthology.orchestra.run.smithi059.stdout:avg 13.6667 2024-05-08T08:52:11.209 INFO:teuthology.orchestra.run.smithi059.stdout:stddev 1.59861 -> 1.59861 (expected baseline 3.37474) 2024-05-08T08:52:11.209 INFO:teuthology.orchestra.run.smithi059.stdout:min osd.2 with 11 -> 11 pgs (0.804878 -> 0.804878 * mean) 2024-05-08T08:52:11.210 INFO:teuthology.orchestra.run.smithi059.stdout:max osd.3 with 16 -> 16 pgs (1.17073 -> 1.17073 * mean) 2024-05-08T08:52:11.210 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:52:11.210 INFO:teuthology.orchestra.run.smithi059.stdout:oload 110 2024-05-08T08:52:11.210 INFO:teuthology.orchestra.run.smithi059.stdout:max_change 0.05 2024-05-08T08:52:11.210 INFO:teuthology.orchestra.run.smithi059.stdout:max_change_osds 4 2024-05-08T08:52:11.210 INFO:teuthology.orchestra.run.smithi059.stdout:average_utilization 0.0003 2024-05-08T08:52:11.210 INFO:teuthology.orchestra.run.smithi059.stdout:overload_utilization 0.0003 2024-05-08T08:52:11.210 INFO:teuthology.orchestra.run.smithi059.stderr:no change 2024-05-08T08:52:11.222 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '1000', 'type': 'reweight-by-utilization'} 2024-05-08T08:52:11.222 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 1000 3.0 2024-05-08T08:52:11.254 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:11.255 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:11.264 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:52:11.267 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:52:11.270+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:52:11.342 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:11.346+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:11.370 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:11.370 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:11.379 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:52:11.463 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:11.466+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:11.519 INFO:teuthology.orchestra.run.smithi059.stdout:moved 0 / 82 (0%) 2024-05-08T08:52:11.519 INFO:teuthology.orchestra.run.smithi059.stdout:avg 13.6667 2024-05-08T08:52:11.519 INFO:teuthology.orchestra.run.smithi059.stdout:stddev 1.59861 -> 1.59861 (expected baseline 3.37474) 2024-05-08T08:52:11.519 INFO:teuthology.orchestra.run.smithi059.stdout:min osd.2 with 11 -> 11 pgs (0.804878 -> 0.804878 * mean) 2024-05-08T08:52:11.520 INFO:teuthology.orchestra.run.smithi059.stdout:max osd.3 with 16 -> 16 pgs (1.17073 -> 1.17073 * mean) 2024-05-08T08:52:11.520 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:52:11.520 INFO:teuthology.orchestra.run.smithi059.stdout:oload 1000 2024-05-08T08:52:11.520 INFO:teuthology.orchestra.run.smithi059.stdout:max_change 3 2024-05-08T08:52:11.520 INFO:teuthology.orchestra.run.smithi059.stdout:max_change_osds 4 2024-05-08T08:52:11.520 INFO:teuthology.orchestra.run.smithi059.stdout:average_utilization 0.0003 2024-05-08T08:52:11.520 INFO:teuthology.orchestra.run.smithi059.stdout:overload_utilization 0.0029 2024-05-08T08:52:11.520 INFO:teuthology.orchestra.run.smithi059.stderr:no change 2024-05-08T08:52:11.524 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:11.524 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:11.533 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:52:11.574 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:52:11.577+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:52:11.645 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:52:11.649+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:52:11.680 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:11.680 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:11.689 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:52:11.746 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:11.749+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:11.796 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:11.796 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:11.805 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:52:11.847 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:52:11.850+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:52:11.947 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:11.950+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:11.957 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:11.957 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:11.967 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:52:12.049 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:52:12.052+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:52:12.112 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:12.112 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:12.121 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:52:12.124 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T08:52:12.169 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:52:12.172+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:52:12.225 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:12.226 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:12.235 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:52:12.254 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:52:12.255 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:52:12.272 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:52:12.272 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:52:12.278 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:12.282+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:12.289 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:52:12.289 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:52:12.307 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:52:12.307 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:52:12.324 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:52:12.324 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:52:12.340 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:12.340 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:12.342 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:52:12.342 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:52:12.349 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:52:12.390 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:12.393+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:12.453 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:52:12.456+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:52:12.458 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:52:12.458 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:52:12.458 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:52:12.458 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:52:12.467 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:52:12.553 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:12.557+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:12.618 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:52:12.618 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:52:12.618 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:52:12.619 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:52:12.619 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:52:12.627 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:52:12.654 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:52:12.658+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:52:12.739 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:52:12.739 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:52:12.739 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:52:12.739 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:52:12.739 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.739 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(client.4189.0:34 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'20, mlcod=13'15)", 2024-05-08T08:52:12.739 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:51:58.898343+0000", 2024-05-08T08:52:12.740 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 13.843014674999999, 2024-05-08T08:52:12.740 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0043629640000000004, 2024-05-08T08:52:12.740 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:12.740 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:12.740 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:12.740 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:12.740 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.740 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:12.740 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:58.898343+0000", 2024-05-08T08:52:12.740 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:12.740 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.740 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.740 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:12.741 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:58.898343+0000", 2024-05-08T08:52:12.741 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:12.741 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.741 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.741 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:12.741 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:58.898344+0000", 2024-05-08T08:52:12.741 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3300000000000001e-07 2024-05-08T08:52:12.741 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.741 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.741 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:12.741 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:58.898349+0000", 2024-05-08T08:52:12.741 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4840000000000003e-06 2024-05-08T08:52:12.742 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.742 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.742 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:12.742 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:58.898361+0000", 2024-05-08T08:52:12.742 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2064e-05 2024-05-08T08:52:12.742 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.742 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.742 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:12.742 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:58.898367+0000", 2024-05-08T08:52:12.742 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3569999999999997e-06 2024-05-08T08:52:12.742 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.743 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.743 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:12.743 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:58.898415+0000", 2024-05-08T08:52:12.743 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8921000000000001e-05 2024-05-08T08:52:12.743 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.743 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.743 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:12.743 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:58.898436+0000", 2024-05-08T08:52:12.743 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0027999999999999e-05 2024-05-08T08:52:12.743 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.743 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.743 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:12.744 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:58.902657+0000", 2024-05-08T08:52:12.744 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0042216780000000004 2024-05-08T08:52:12.744 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.744 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.744 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:12.744 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:58.902706+0000", 2024-05-08T08:52:12.744 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8699000000000002e-05 2024-05-08T08:52:12.744 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:12.744 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:12.744 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:12.744 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.744 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.744 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(client.4189.0:35 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'21, mlcod=13'17)", 2024-05-08T08:52:12.744 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:51:58.899026+0000", 2024-05-08T08:52:12.745 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 13.842332094, 2024-05-08T08:52:12.745 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0037181219999999999, 2024-05-08T08:52:12.745 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:12.745 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:12.745 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:12.745 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:12.745 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.745 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:12.745 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:58.899026+0000", 2024-05-08T08:52:12.745 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:12.745 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.745 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.745 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:12.746 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:58.899026+0000", 2024-05-08T08:52:12.746 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:12.746 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.746 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.746 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:12.746 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:58.899026+0000", 2024-05-08T08:52:12.746 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1299999999999999e-07 2024-05-08T08:52:12.746 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.746 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.747 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:12.747 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:58.899086+0000", 2024-05-08T08:52:12.747 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9919e-05 2024-05-08T08:52:12.747 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.747 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.747 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:12.747 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:58.899100+0000", 2024-05-08T08:52:12.747 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3448999999999999e-05 2024-05-08T08:52:12.747 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.747 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.747 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:12.747 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:58.899105+0000", 2024-05-08T08:52:12.747 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4920000000000001e-06 2024-05-08T08:52:12.748 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.748 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.748 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:12.748 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:58.899156+0000", 2024-05-08T08:52:12.748 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0970999999999997e-05 2024-05-08T08:52:12.748 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.748 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.748 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:12.748 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:58.899177+0000", 2024-05-08T08:52:12.748 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0865000000000001e-05 2024-05-08T08:52:12.748 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.749 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.749 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:12.749 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:58.902709+0000", 2024-05-08T08:52:12.749 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0035318680000000001 2024-05-08T08:52:12.749 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.749 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.749 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:12.749 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:58.902744+0000", 2024-05-08T08:52:12.749 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4745e-05 2024-05-08T08:52:12.749 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:12.749 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:12.749 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:12.749 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.750 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.750 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(client.4189.0:36 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'22, mlcod=13'17)", 2024-05-08T08:52:12.750 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:51:58.899774+0000", 2024-05-08T08:52:12.750 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 13.841584086999999, 2024-05-08T08:52:12.750 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0054675119999999999, 2024-05-08T08:52:12.750 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:12.750 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:12.750 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:12.750 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:12.750 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.750 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:12.750 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:58.899774+0000", 2024-05-08T08:52:12.751 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:12.751 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.751 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.751 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:12.751 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:58.899774+0000", 2024-05-08T08:52:12.751 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:12.751 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.751 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.751 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:12.751 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:58.899774+0000", 2024-05-08T08:52:12.751 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9299999999999997e-07 2024-05-08T08:52:12.751 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.751 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.752 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:12.752 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:58.899780+0000", 2024-05-08T08:52:12.752 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.49e-06 2024-05-08T08:52:12.752 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.752 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.752 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:12.752 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:58.899790+0000", 2024-05-08T08:52:12.752 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0484e-05 2024-05-08T08:52:12.752 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.752 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.752 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:12.752 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:58.899796+0000", 2024-05-08T08:52:12.752 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2179999999999998e-06 2024-05-08T08:52:12.752 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.753 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.753 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:12.753 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:58.899860+0000", 2024-05-08T08:52:12.753 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4420999999999999e-05 2024-05-08T08:52:12.753 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.753 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.753 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:12.753 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:58.899882+0000", 2024-05-08T08:52:12.753 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1919000000000001e-05 2024-05-08T08:52:12.753 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.753 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.753 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:12.753 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:58.905204+0000", 2024-05-08T08:52:12.754 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0053217830000000001 2024-05-08T08:52:12.754 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.754 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.754 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:12.754 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:58.905241+0000", 2024-05-08T08:52:12.754 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7404000000000003e-05 2024-05-08T08:52:12.754 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:12.754 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:12.754 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:12.754 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.754 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.754 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(client.4189.0:37 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'23, mlcod=13'17)", 2024-05-08T08:52:12.754 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:51:58.900239+0000", 2024-05-08T08:52:12.754 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 13.841118667, 2024-05-08T08:52:12.755 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.005034841, 2024-05-08T08:52:12.755 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:12.755 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:12.755 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:12.755 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:12.755 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.755 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:12.755 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:58.900239+0000", 2024-05-08T08:52:12.755 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:12.755 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.755 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.755 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:12.755 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:58.900239+0000", 2024-05-08T08:52:12.755 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:12.755 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.756 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.756 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:12.756 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:58.900239+0000", 2024-05-08T08:52:12.756 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5900000000000002e-07 2024-05-08T08:52:12.756 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.756 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.756 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:12.756 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:58.900243+0000", 2024-05-08T08:52:12.756 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9260000000000002e-06 2024-05-08T08:52:12.756 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.756 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.756 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:12.756 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:58.900252+0000", 2024-05-08T08:52:12.756 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1079999999999999e-06 2024-05-08T08:52:12.757 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.757 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.757 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:12.757 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:58.900257+0000", 2024-05-08T08:52:12.757 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4499999999999997e-06 2024-05-08T08:52:12.757 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.757 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.757 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:12.757 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:58.900294+0000", 2024-05-08T08:52:12.757 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6808999999999999e-05 2024-05-08T08:52:12.757 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.757 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.758 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:12.758 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:58.900308+0000", 2024-05-08T08:52:12.758 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.4596999999999999e-05 2024-05-08T08:52:12.758 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.758 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.758 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:12.758 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:58.905246+0000", 2024-05-08T08:52:12.758 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0049373569999999999 2024-05-08T08:52:12.758 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.758 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.758 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:12.758 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:58.905274+0000", 2024-05-08T08:52:12.758 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8135000000000001e-05 2024-05-08T08:52:12.758 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:12.759 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:12.759 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:12.759 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.759 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.759 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(client.4189.0:54 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'32, mlcod=13'29)", 2024-05-08T08:52:12.759 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:51:58.924771+0000", 2024-05-08T08:52:12.759 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 13.816586559999999, 2024-05-08T08:52:12.759 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0030611430000000001, 2024-05-08T08:52:12.759 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:12.759 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:12.759 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:12.759 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:12.759 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.759 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:12.759 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:58.924771+0000", 2024-05-08T08:52:12.759 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:12.760 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.760 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.760 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:12.760 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:58.924771+0000", 2024-05-08T08:52:12.760 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:12.760 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.760 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.760 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:12.760 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:58.924772+0000", 2024-05-08T08:52:12.760 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7899999999999998e-07 2024-05-08T08:52:12.760 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.760 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.760 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:12.760 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:58.924777+0000", 2024-05-08T08:52:12.760 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8749999999999999e-06 2024-05-08T08:52:12.761 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.761 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.761 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:12.761 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:58.924787+0000", 2024-05-08T08:52:12.761 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0149999999999999e-05 2024-05-08T08:52:12.761 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.761 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.761 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:12.761 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:58.924792+0000", 2024-05-08T08:52:12.761 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0660000000000003e-06 2024-05-08T08:52:12.761 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.761 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.761 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:12.761 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:58.924843+0000", 2024-05-08T08:52:12.761 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0754999999999998e-05 2024-05-08T08:52:12.762 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.762 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.762 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:12.762 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:58.924863+0000", 2024-05-08T08:52:12.762 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0429999999999999e-05 2024-05-08T08:52:12.762 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.762 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.762 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:12.762 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:58.927794+0000", 2024-05-08T08:52:12.762 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0029304969999999998 2024-05-08T08:52:12.762 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.762 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.762 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:12.762 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:58.927832+0000", 2024-05-08T08:52:12.763 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8690999999999999e-05 2024-05-08T08:52:12.763 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:12.763 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:12.763 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:12.763 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.763 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.763 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(client.4189.0:55 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'33, mlcod=13'29)", 2024-05-08T08:52:12.763 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:51:58.925246+0000", 2024-05-08T08:52:12.763 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 13.816111419, 2024-05-08T08:52:12.763 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0029862389999999999, 2024-05-08T08:52:12.763 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:12.763 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:12.763 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:12.763 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:12.763 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.763 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:12.763 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:58.925246+0000", 2024-05-08T08:52:12.764 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:12.764 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.764 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.764 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:12.764 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:58.925246+0000", 2024-05-08T08:52:12.764 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:12.764 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.764 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.764 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:12.764 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:58.925247+0000", 2024-05-08T08:52:12.764 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6400000000000002e-07 2024-05-08T08:52:12.764 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.764 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.764 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:12.765 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:58.925311+0000", 2024-05-08T08:52:12.765 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4265999999999998e-05 2024-05-08T08:52:12.765 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.765 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.765 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:12.765 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:58.925323+0000", 2024-05-08T08:52:12.765 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2183e-05 2024-05-08T08:52:12.765 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.765 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.765 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:12.765 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:58.925329+0000", 2024-05-08T08:52:12.765 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6879999999999996e-06 2024-05-08T08:52:12.765 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.765 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.766 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:12.766 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:58.925373+0000", 2024-05-08T08:52:12.766 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4515999999999999e-05 2024-05-08T08:52:12.766 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.766 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.766 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:12.766 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:58.925388+0000", 2024-05-08T08:52:12.766 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.4972e-05 2024-05-08T08:52:12.766 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.766 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.766 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:12.767 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:58.928194+0000", 2024-05-08T08:52:12.767 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0028053890000000001 2024-05-08T08:52:12.767 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.767 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.767 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:12.767 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:58.928232+0000", 2024-05-08T08:52:12.767 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8661000000000001e-05 2024-05-08T08:52:12.767 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:12.767 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:12.767 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:12.767 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.767 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.768 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(client.4189.0:108 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'70, mlcod=13'66)", 2024-05-08T08:52:12.768 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:51:59.119504+0000", 2024-05-08T08:52:12.768 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 13.621853671, 2024-05-08T08:52:12.768 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0059888720000000001, 2024-05-08T08:52:12.768 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:12.768 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:12.768 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:12.768 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:12.768 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.768 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:12.768 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.119504+0000", 2024-05-08T08:52:12.768 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:12.769 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.769 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.769 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:12.769 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.119504+0000", 2024-05-08T08:52:12.769 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:12.769 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.769 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.769 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:12.769 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.119504+0000", 2024-05-08T08:52:12.769 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5999999999999999e-07 2024-05-08T08:52:12.769 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.769 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.770 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:12.770 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.119510+0000", 2024-05-08T08:52:12.770 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1229999999999999e-06 2024-05-08T08:52:12.770 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.770 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.770 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:12.770 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.119517+0000", 2024-05-08T08:52:12.770 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7170000000000008e-06 2024-05-08T08:52:12.770 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.770 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.770 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:12.770 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.119525+0000", 2024-05-08T08:52:12.771 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8590000000000005e-06 2024-05-08T08:52:12.771 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.771 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.771 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:12.771 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.119565+0000", 2024-05-08T08:52:12.771 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0331999999999999e-05 2024-05-08T08:52:12.771 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.771 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.771 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:12.771 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.119585+0000", 2024-05-08T08:52:12.771 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9695e-05 2024-05-08T08:52:12.772 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.772 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.772 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:12.772 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.125469+0000", 2024-05-08T08:52:12.772 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0058836670000000004 2024-05-08T08:52:12.772 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.772 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.772 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:12.772 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.125493+0000", 2024-05-08T08:52:12.772 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4019000000000002e-05 2024-05-08T08:52:12.772 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:12.772 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:12.773 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:12.773 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.773 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.773 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(client.4189.0:109 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'71, mlcod=13'66)", 2024-05-08T08:52:12.773 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:51:59.119857+0000", 2024-05-08T08:52:12.773 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 13.621500482, 2024-05-08T08:52:12.773 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0056918130000000004, 2024-05-08T08:52:12.773 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:12.773 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:12.773 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:12.773 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:12.773 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.774 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:12.774 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.119857+0000", 2024-05-08T08:52:12.774 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:12.774 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.774 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.774 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:12.774 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.119857+0000", 2024-05-08T08:52:12.774 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:12.774 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.774 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.774 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:12.775 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.119857+0000", 2024-05-08T08:52:12.775 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6800000000000001e-07 2024-05-08T08:52:12.775 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.775 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.775 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:12.775 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.119919+0000", 2024-05-08T08:52:12.775 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1148999999999993e-05 2024-05-08T08:52:12.775 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.775 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.775 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:12.775 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.119925+0000", 2024-05-08T08:52:12.775 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0599999999999996e-06 2024-05-08T08:52:12.776 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.776 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.776 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:12.776 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.119928+0000", 2024-05-08T08:52:12.776 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4960000000000001e-06 2024-05-08T08:52:12.776 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.776 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.776 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:12.776 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.120002+0000", 2024-05-08T08:52:12.776 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3589000000000003e-05 2024-05-08T08:52:12.776 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.776 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.776 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:12.777 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.120015+0000", 2024-05-08T08:52:12.777 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.288e-05 2024-05-08T08:52:12.777 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.777 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.777 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:12.777 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.125530+0000", 2024-05-08T08:52:12.777 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0055153540000000001 2024-05-08T08:52:12.777 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.777 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.777 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:12.777 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.125549+0000", 2024-05-08T08:52:12.778 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.8916999999999999e-05 2024-05-08T08:52:12.778 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:12.778 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:12.778 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:12.778 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.778 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.778 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(client.4189.0:110 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'72, mlcod=13'66)", 2024-05-08T08:52:12.778 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:51:59.120812+0000", 2024-05-08T08:52:12.778 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 13.620545477, 2024-05-08T08:52:12.778 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0047515250000000004, 2024-05-08T08:52:12.778 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:12.779 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:12.779 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:12.779 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:12.779 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.779 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:12.779 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.120812+0000", 2024-05-08T08:52:12.779 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:12.779 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.779 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.779 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:12.779 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.120812+0000", 2024-05-08T08:52:12.779 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:12.780 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.780 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.780 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:12.780 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.120813+0000", 2024-05-08T08:52:12.780 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:52:12.780 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.780 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.780 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:12.780 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.120815+0000", 2024-05-08T08:52:12.780 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8600000000000001e-06 2024-05-08T08:52:12.780 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.780 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.781 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:12.781 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.120822+0000", 2024-05-08T08:52:12.781 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0129999999999996e-06 2024-05-08T08:52:12.781 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.781 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.781 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:12.781 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.120826+0000", 2024-05-08T08:52:12.781 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.337e-06 2024-05-08T08:52:12.781 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.781 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.781 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:12.781 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.120858+0000", 2024-05-08T08:52:12.781 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1924999999999997e-05 2024-05-08T08:52:12.781 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.782 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.782 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:12.782 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.120870+0000", 2024-05-08T08:52:12.782 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2019e-05 2024-05-08T08:52:12.782 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.782 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.782 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:12.782 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.125550+0000", 2024-05-08T08:52:12.782 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0046803820000000003 2024-05-08T08:52:12.782 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.782 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.782 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:12.782 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.125564+0000", 2024-05-08T08:52:12.783 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.362e-05 2024-05-08T08:52:12.783 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:12.783 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:12.783 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:12.783 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.783 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.783 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(client.4189.0:111 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'73, mlcod=13'68)", 2024-05-08T08:52:12.783 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:51:59.121373+0000", 2024-05-08T08:52:12.783 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 13.619984702, 2024-05-08T08:52:12.783 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0042064609999999999, 2024-05-08T08:52:12.783 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:12.783 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:12.783 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:12.783 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:12.784 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.784 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:12.784 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.121373+0000", 2024-05-08T08:52:12.784 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:12.784 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.784 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.784 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:12.784 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.121373+0000", 2024-05-08T08:52:12.784 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:12.784 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.784 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.784 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:12.784 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.121373+0000", 2024-05-08T08:52:12.784 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8800000000000002e-07 2024-05-08T08:52:12.785 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.785 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.785 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:12.785 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.121380+0000", 2024-05-08T08:52:12.785 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1909999999999997e-06 2024-05-08T08:52:12.785 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.785 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.785 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:12.785 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.121390+0000", 2024-05-08T08:52:12.785 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8980000000000008e-06 2024-05-08T08:52:12.785 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.785 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.785 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:12.785 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.121394+0000", 2024-05-08T08:52:12.786 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5109999999999997e-06 2024-05-08T08:52:12.786 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.786 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.786 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:12.786 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.121432+0000", 2024-05-08T08:52:12.786 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7755e-05 2024-05-08T08:52:12.786 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.786 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.786 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:12.786 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.121445+0000", 2024-05-08T08:52:12.786 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3213e-05 2024-05-08T08:52:12.786 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.786 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.786 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:12.787 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.125566+0000", 2024-05-08T08:52:12.787 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0041205870000000002 2024-05-08T08:52:12.787 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.787 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.787 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:12.787 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.125579+0000", 2024-05-08T08:52:12.787 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3718e-05 2024-05-08T08:52:12.787 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:12.787 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:12.787 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:12.787 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.787 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.787 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(client.4189.0:112 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'74, mlcod=13'68)", 2024-05-08T08:52:12.787 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:51:59.121732+0000", 2024-05-08T08:52:12.787 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 13.619625666999999, 2024-05-08T08:52:12.788 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0038871740000000002, 2024-05-08T08:52:12.788 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:12.788 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:12.788 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:12.788 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:12.788 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.788 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:12.788 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.121732+0000", 2024-05-08T08:52:12.788 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:12.788 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.789 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.789 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:12.789 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.121732+0000", 2024-05-08T08:52:12.789 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:12.789 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.789 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.789 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:12.789 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.121733+0000", 2024-05-08T08:52:12.789 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1799999999999995e-07 2024-05-08T08:52:12.789 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.789 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.789 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:12.789 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.121774+0000", 2024-05-08T08:52:12.789 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0998999999999998e-05 2024-05-08T08:52:12.790 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.790 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.790 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:12.790 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.121784+0000", 2024-05-08T08:52:12.790 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0175e-05 2024-05-08T08:52:12.790 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.790 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.790 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:12.790 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.121789+0000", 2024-05-08T08:52:12.790 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8609999999999997e-06 2024-05-08T08:52:12.790 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.790 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.790 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:12.791 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.121822+0000", 2024-05-08T08:52:12.791 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3723999999999998e-05 2024-05-08T08:52:12.791 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.791 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.791 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:12.791 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.121834+0000", 2024-05-08T08:52:12.791 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1365e-05 2024-05-08T08:52:12.791 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.791 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.791 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:12.791 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.125589+0000", 2024-05-08T08:52:12.791 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0037548220000000001 2024-05-08T08:52:12.791 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.792 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.792 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:12.792 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.125619+0000", 2024-05-08T08:52:12.792 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.061e-05 2024-05-08T08:52:12.792 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:12.792 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:12.792 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:12.792 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.792 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.792 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(client.4189.0:113 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'75, mlcod=13'68)", 2024-05-08T08:52:12.792 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:51:59.122382+0000", 2024-05-08T08:52:12.792 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 13.618975764, 2024-05-08T08:52:12.792 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0032539499999999998, 2024-05-08T08:52:12.792 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:12.792 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:12.793 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:12.793 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:12.793 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.793 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:12.793 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.122382+0000", 2024-05-08T08:52:12.793 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:12.793 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.793 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.793 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:12.793 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.122382+0000", 2024-05-08T08:52:12.793 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:12.793 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.794 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.794 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:12.794 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.122382+0000", 2024-05-08T08:52:12.794 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1399999999999997e-07 2024-05-08T08:52:12.794 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.794 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.794 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:12.794 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.122389+0000", 2024-05-08T08:52:12.794 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5370000000000003e-06 2024-05-08T08:52:12.794 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.794 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.794 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:12.795 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.122399+0000", 2024-05-08T08:52:12.795 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0462000000000001e-05 2024-05-08T08:52:12.795 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.795 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.795 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:12.795 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.122406+0000", 2024-05-08T08:52:12.795 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5239999999999997e-06 2024-05-08T08:52:12.795 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.795 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.795 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:12.795 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.122447+0000", 2024-05-08T08:52:12.795 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0640999999999999e-05 2024-05-08T08:52:12.795 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.795 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.796 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:12.796 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.122462+0000", 2024-05-08T08:52:12.796 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5543e-05 2024-05-08T08:52:12.796 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.796 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.796 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:12.796 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.125620+0000", 2024-05-08T08:52:12.796 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0031581859999999999 2024-05-08T08:52:12.796 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.796 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.796 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:12.797 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.125636+0000", 2024-05-08T08:52:12.797 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5443000000000001e-05 2024-05-08T08:52:12.797 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:12.797 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:12.797 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:12.797 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.797 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.797 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(client.4189.0:114 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'76, mlcod=13'68)", 2024-05-08T08:52:12.797 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:51:59.122701+0000", 2024-05-08T08:52:12.797 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 13.618656223, 2024-05-08T08:52:12.797 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00295269, 2024-05-08T08:52:12.797 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:12.797 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:12.798 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:12.798 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:12.798 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.798 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:12.798 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.122701+0000", 2024-05-08T08:52:12.798 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:12.798 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.798 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.798 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:12.798 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.122701+0000", 2024-05-08T08:52:12.798 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:12.798 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.799 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.799 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:12.799 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.122702+0000", 2024-05-08T08:52:12.799 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.63e-07 2024-05-08T08:52:12.799 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.799 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.799 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:12.799 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.122708+0000", 2024-05-08T08:52:12.799 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4829999999999996e-06 2024-05-08T08:52:12.799 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.799 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.799 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:12.799 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.122717+0000", 2024-05-08T08:52:12.800 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.2859999999999999e-06 2024-05-08T08:52:12.800 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.800 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.800 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:12.800 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.122721+0000", 2024-05-08T08:52:12.800 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0500000000000002e-06 2024-05-08T08:52:12.800 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.800 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.800 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:12.800 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.122754+0000", 2024-05-08T08:52:12.800 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.2886999999999999e-05 2024-05-08T08:52:12.800 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.801 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.801 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:12.801 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.122765+0000", 2024-05-08T08:52:12.801 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1409e-05 2024-05-08T08:52:12.801 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.801 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.801 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:12.801 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.125636+0000", 2024-05-08T08:52:12.801 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0028714550000000002 2024-05-08T08:52:12.801 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.801 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.802 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:12.802 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.125654+0000", 2024-05-08T08:52:12.802 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.7657000000000001e-05 2024-05-08T08:52:12.802 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:12.802 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:12.802 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:12.802 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.802 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.802 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(client.4189.0:115 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'77, mlcod=13'68)", 2024-05-08T08:52:12.802 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:51:59.123084+0000", 2024-05-08T08:52:12.802 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 13.618274040999999, 2024-05-08T08:52:12.803 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0050328839999999996, 2024-05-08T08:52:12.803 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:12.803 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:12.803 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:12.803 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:12.803 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.803 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:12.803 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.123084+0000", 2024-05-08T08:52:12.803 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:12.803 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.803 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.803 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:12.803 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.123084+0000", 2024-05-08T08:52:12.804 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:12.804 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.804 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.804 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:12.804 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.123084+0000", 2024-05-08T08:52:12.804 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4799999999999999e-07 2024-05-08T08:52:12.804 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.804 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.804 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:12.804 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.123185+0000", 2024-05-08T08:52:12.804 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010115000000000001 2024-05-08T08:52:12.804 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.804 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.805 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:12.805 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.123192+0000", 2024-05-08T08:52:12.805 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0589999999999997e-06 2024-05-08T08:52:12.805 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.805 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.805 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:12.805 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.123196+0000", 2024-05-08T08:52:12.805 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8319999999999999e-06 2024-05-08T08:52:12.805 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.805 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.805 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:12.806 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.123228+0000", 2024-05-08T08:52:12.806 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1770000000000002e-05 2024-05-08T08:52:12.806 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.806 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.806 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:12.806 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.123240+0000", 2024-05-08T08:52:12.806 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2757e-05 2024-05-08T08:52:12.806 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.806 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.806 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:12.806 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.128085+0000", 2024-05-08T08:52:12.806 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0048445249999999997 2024-05-08T08:52:12.807 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.807 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.807 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:12.807 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.128116+0000", 2024-05-08T08:52:12.807 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1442999999999997e-05 2024-05-08T08:52:12.807 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:12.807 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:12.807 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:12.807 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.807 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.807 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(client.4189.0:116 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'78, mlcod=13'68)", 2024-05-08T08:52:12.807 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:51:59.123722+0000", 2024-05-08T08:52:12.808 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 13.617635156, 2024-05-08T08:52:12.808 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0044126670000000003, 2024-05-08T08:52:12.808 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:12.808 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:12.808 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:12.808 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:12.808 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.808 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:12.808 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.123722+0000", 2024-05-08T08:52:12.808 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:12.808 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.808 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.809 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:12.809 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.123722+0000", 2024-05-08T08:52:12.809 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:12.809 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.809 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.809 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:12.809 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.123723+0000", 2024-05-08T08:52:12.809 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.96e-07 2024-05-08T08:52:12.809 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.809 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.809 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:12.809 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.123728+0000", 2024-05-08T08:52:12.809 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0819999999999998e-06 2024-05-08T08:52:12.810 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.810 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.810 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:12.810 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.123735+0000", 2024-05-08T08:52:12.810 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3089999999999998e-06 2024-05-08T08:52:12.810 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.810 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.810 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:12.810 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.123739+0000", 2024-05-08T08:52:12.810 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5990000000000002e-06 2024-05-08T08:52:12.810 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.811 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.811 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:12.811 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.125162+0000", 2024-05-08T08:52:12.811 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0014232330000000001 2024-05-08T08:52:12.811 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.811 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.811 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:12.811 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.125176+0000", 2024-05-08T08:52:12.811 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3565e-05 2024-05-08T08:52:12.811 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.811 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.811 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:12.811 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.128118+0000", 2024-05-08T08:52:12.811 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0029420340000000001 2024-05-08T08:52:12.811 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.812 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.812 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:12.812 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.128135+0000", 2024-05-08T08:52:12.812 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.7549000000000001e-05 2024-05-08T08:52:12.812 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:12.812 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:12.812 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:12.812 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.812 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.812 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(client.4189.0:117 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'79, mlcod=13'68)", 2024-05-08T08:52:12.812 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:51:59.124080+0000", 2024-05-08T08:52:12.812 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 13.61727788, 2024-05-08T08:52:12.812 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0040724029999999996, 2024-05-08T08:52:12.812 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:12.812 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:12.813 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:12.813 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:12.813 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.813 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:12.813 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.124080+0000", 2024-05-08T08:52:12.813 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:12.813 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.813 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.813 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:12.813 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.124080+0000", 2024-05-08T08:52:12.813 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:12.813 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.813 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.813 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:12.814 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.124080+0000", 2024-05-08T08:52:12.814 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6100000000000002e-07 2024-05-08T08:52:12.814 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.814 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.814 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:12.814 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.124085+0000", 2024-05-08T08:52:12.814 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3990000000000003e-06 2024-05-08T08:52:12.814 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.814 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.814 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:12.814 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.124092+0000", 2024-05-08T08:52:12.814 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1460000000000001e-06 2024-05-08T08:52:12.814 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.814 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.814 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:12.815 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.124100+0000", 2024-05-08T08:52:12.815 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4440000000000008e-06 2024-05-08T08:52:12.815 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.815 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.815 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:12.815 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.125335+0000", 2024-05-08T08:52:12.815 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0012354009999999999 2024-05-08T08:52:12.815 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.815 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.815 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:12.815 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.125348+0000", 2024-05-08T08:52:12.815 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2554999999999999e-05 2024-05-08T08:52:12.815 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.815 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.816 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:12.816 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.128137+0000", 2024-05-08T08:52:12.816 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0027884730000000001 2024-05-08T08:52:12.816 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.816 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.816 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:12.816 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.128152+0000", 2024-05-08T08:52:12.816 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5624000000000002e-05 2024-05-08T08:52:12.816 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:12.816 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:12.816 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:12.816 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.816 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.816 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(client.4189.0:119 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'81, mlcod=13'78)", 2024-05-08T08:52:12.816 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:51:59.129560+0000", 2024-05-08T08:52:12.816 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 13.611797294, 2024-05-08T08:52:12.817 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0028899590000000001, 2024-05-08T08:52:12.817 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:12.817 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:12.817 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:12.817 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:12.817 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.817 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:12.817 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.129560+0000", 2024-05-08T08:52:12.817 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:12.817 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.817 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.817 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:12.817 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.129560+0000", 2024-05-08T08:52:12.817 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:12.818 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.818 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.818 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:12.818 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.129561+0000", 2024-05-08T08:52:12.818 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7100000000000001e-07 2024-05-08T08:52:12.818 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.818 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.818 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:12.818 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.129617+0000", 2024-05-08T08:52:12.818 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6181000000000003e-05 2024-05-08T08:52:12.818 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.818 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.818 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:12.818 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.129627+0000", 2024-05-08T08:52:12.819 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4280000000000002e-06 2024-05-08T08:52:12.819 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.819 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.819 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:12.819 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.129631+0000", 2024-05-08T08:52:12.819 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2200000000000003e-06 2024-05-08T08:52:12.819 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.819 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.819 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:12.819 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.129665+0000", 2024-05-08T08:52:12.819 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4115000000000001e-05 2024-05-08T08:52:12.819 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.819 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.819 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:12.819 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.129678+0000", 2024-05-08T08:52:12.819 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2632e-05 2024-05-08T08:52:12.820 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.820 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.820 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:12.820 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.132395+0000", 2024-05-08T08:52:12.820 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.002717854 2024-05-08T08:52:12.820 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.820 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.820 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:12.820 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.132450+0000", 2024-05-08T08:52:12.820 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4858000000000002e-05 2024-05-08T08:52:12.820 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:12.820 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:12.820 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:12.820 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.820 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.820 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(client.4189.0:120 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'82, mlcod=13'78)", 2024-05-08T08:52:12.821 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:51:59.130173+0000", 2024-05-08T08:52:12.821 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 13.61118433, 2024-05-08T08:52:12.821 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0049977449999999996, 2024-05-08T08:52:12.821 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:12.821 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:12.821 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:12.821 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:12.821 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.821 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:12.821 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.130173+0000", 2024-05-08T08:52:12.821 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:12.821 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.821 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.821 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:12.822 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.130173+0000", 2024-05-08T08:52:12.822 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:12.822 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.822 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.822 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:12.822 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.130174+0000", 2024-05-08T08:52:12.822 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.82e-07 2024-05-08T08:52:12.822 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.822 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.822 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:12.822 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.130275+0000", 2024-05-08T08:52:12.822 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000101258 2024-05-08T08:52:12.823 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.823 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.823 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:12.823 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.130283+0000", 2024-05-08T08:52:12.823 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0460000000000006e-06 2024-05-08T08:52:12.823 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.823 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.823 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:12.823 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.130287+0000", 2024-05-08T08:52:12.823 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1189999999999999e-06 2024-05-08T08:52:12.823 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.824 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.824 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:12.824 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.130323+0000", 2024-05-08T08:52:12.824 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6050999999999998e-05 2024-05-08T08:52:12.824 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.824 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.824 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:12.824 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.130334+0000", 2024-05-08T08:52:12.824 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1252e-05 2024-05-08T08:52:12.824 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.824 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.824 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:12.824 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.135139+0000", 2024-05-08T08:52:12.825 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0048042149999999997 2024-05-08T08:52:12.825 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.825 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.825 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:12.825 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.135171+0000", 2024-05-08T08:52:12.825 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.2322e-05 2024-05-08T08:52:12.825 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:12.825 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:12.825 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:12.825 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.825 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.825 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(client.4189.0:121 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'83, mlcod=13'78)", 2024-05-08T08:52:12.826 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:51:59.130847+0000", 2024-05-08T08:52:12.826 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 13.610510340999999, 2024-05-08T08:52:12.826 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0043406790000000001, 2024-05-08T08:52:12.826 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:12.826 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:12.826 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:12.826 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:12.826 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.826 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:12.826 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.130847+0000", 2024-05-08T08:52:12.826 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:12.826 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.826 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.827 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:12.827 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.130847+0000", 2024-05-08T08:52:12.827 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:12.827 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.827 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.827 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:12.827 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.130848+0000", 2024-05-08T08:52:12.827 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.2099999999999998e-07 2024-05-08T08:52:12.827 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.827 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.827 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:12.827 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.130978+0000", 2024-05-08T08:52:12.827 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013019999999999999 2024-05-08T08:52:12.827 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.827 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.828 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:12.828 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.130985+0000", 2024-05-08T08:52:12.828 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2069999999999998e-06 2024-05-08T08:52:12.828 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.828 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.828 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:12.828 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.130989+0000", 2024-05-08T08:52:12.828 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6550000000000002e-06 2024-05-08T08:52:12.828 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.828 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.828 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:12.828 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.131031+0000", 2024-05-08T08:52:12.828 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2249999999999997e-05 2024-05-08T08:52:12.828 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.828 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.829 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:12.829 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.131044+0000", 2024-05-08T08:52:12.829 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.325e-05 2024-05-08T08:52:12.829 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.829 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.829 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:12.829 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.135173+0000", 2024-05-08T08:52:12.829 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0041290479999999997 2024-05-08T08:52:12.829 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.829 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.829 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:12.829 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.135188+0000", 2024-05-08T08:52:12.829 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.4748e-05 2024-05-08T08:52:12.830 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:12.830 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:12.830 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:12.830 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.830 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.830 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(client.4189.0:137 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'95, mlcod=13'91)", 2024-05-08T08:52:12.830 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:51:59.448715+0000", 2024-05-08T08:52:12.830 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 13.292643025, 2024-05-08T08:52:12.830 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0037325769999999999, 2024-05-08T08:52:12.830 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:12.830 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:12.830 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:12.830 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:12.831 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.831 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:12.831 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.448715+0000", 2024-05-08T08:52:12.831 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:12.831 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.831 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.831 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:12.831 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.448715+0000", 2024-05-08T08:52:12.831 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:12.831 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.831 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.831 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:12.831 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.448715+0000", 2024-05-08T08:52:12.832 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1899999999999998e-07 2024-05-08T08:52:12.832 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.832 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.832 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:12.832 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.448841+0000", 2024-05-08T08:52:12.832 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000126592 2024-05-08T08:52:12.832 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.832 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.832 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:12.832 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.448849+0000", 2024-05-08T08:52:12.832 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5410000000000003e-06 2024-05-08T08:52:12.832 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.832 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.832 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:12.833 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.448853+0000", 2024-05-08T08:52:12.833 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8569999999999997e-06 2024-05-08T08:52:12.833 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.833 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.833 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:12.833 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.448882+0000", 2024-05-08T08:52:12.833 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9518000000000001e-05 2024-05-08T08:52:12.833 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.833 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.833 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:12.833 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.448893+0000", 2024-05-08T08:52:12.833 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0991e-05 2024-05-08T08:52:12.834 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.834 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.834 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:12.834 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.452424+0000", 2024-05-08T08:52:12.834 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0035303029999999998 2024-05-08T08:52:12.834 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:12.834 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:12.834 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:12.834 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:51:59.452447+0000", 2024-05-08T08:52:12.834 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3456e-05 2024-05-08T08:52:12.834 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:12.834 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:12.834 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:12.834 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:12.835 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:12.835 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:52:12.836 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:52:12.837 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:12.839+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:12.933 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-05-08T08:52:12.936 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:12.940+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:12.951 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:52:12.951 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:52:12.951 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:52:12.951 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:52:12.961 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:52:13.037 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:13.040+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:13.116 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:52:13.116 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:52:13.117 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:52:13.117 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:52:13.117 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:52:13.126 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:52:13.138 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:13.141+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:13.235 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:52:13.236 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:52:13.236 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:52:13.236 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:52:13.236 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:13.236 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(client.4416.0:7 3.1a e20/18 3:5cea7035:::rbd_info:head v 20'1, mlcod=0'0)", 2024-05-08T08:52:13.236 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:12.926400+0000", 2024-05-08T08:52:13.236 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.31320762200000002, 2024-05-08T08:52:13.236 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00093956900000000002, 2024-05-08T08:52:13.236 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:13.236 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:13.236 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:13.236 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:13.236 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:13.237 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:13.237 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:12.926400+0000", 2024-05-08T08:52:13.237 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:13.237 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:13.237 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:13.237 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:13.237 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:12.926400+0000", 2024-05-08T08:52:13.237 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:13.237 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:13.237 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:13.237 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:13.238 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:12.926401+0000", 2024-05-08T08:52:13.238 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9999999999999999e-07 2024-05-08T08:52:13.238 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:13.238 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:13.238 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:13.238 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:12.926405+0000", 2024-05-08T08:52:13.238 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.763e-06 2024-05-08T08:52:13.238 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:13.238 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:13.239 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:13.239 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:12.926418+0000", 2024-05-08T08:52:13.239 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3101e-05 2024-05-08T08:52:13.239 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:13.239 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:13.239 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:13.239 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:12.926427+0000", 2024-05-08T08:52:13.239 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.5960000000000004e-06 2024-05-08T08:52:13.239 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:13.239 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:13.239 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:13.239 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:12.926488+0000", 2024-05-08T08:52:13.239 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0931999999999999e-05 2024-05-08T08:52:13.239 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:13.240 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:13.240 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:13.240 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:12.926516+0000", 2024-05-08T08:52:13.240 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7864999999999998e-05 2024-05-08T08:52:13.240 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:13.240 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:13.240 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:13.240 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:12.927297+0000", 2024-05-08T08:52:13.240 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00078093399999999995 2024-05-08T08:52:13.240 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:13.240 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:13.240 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:13.240 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:12.927340+0000", 2024-05-08T08:52:13.240 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3177999999999999e-05 2024-05-08T08:52:13.240 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:13.241 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:13.241 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:13.241 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:13.241 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:13.241 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:52:13.242 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:52:13.245+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:52:13.244 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:52:13.343 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:13.347+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:13.395 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:52:13.395 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:52:13.395 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:52:13.395 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:52:13.404 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:52:13.445 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:52:13.448+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:52:13.513 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:52:13.514 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:52:13.514 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:52:13.514 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:52:13.514 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:52:13.522 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:52:13.545 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:13.549+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:13.632 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:52:13.632 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:52:13.632 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:52:13.632 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [] 2024-05-08T08:52:13.632 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:52:13.640 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:52:13.646 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:52:13.649+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:52:13.747 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:13.750+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:13.758 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:52:13.758 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:52:13.759 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:52:13.759 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:52:13.768 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:52:13.852 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:13.855+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:13.913 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:52:13.913 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:52:13.913 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:52:13.914 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:52:13.914 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:52:13.922 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:52:13.930 INFO:teuthology.orchestra.run.smithi059.stderr:nodeep-scrub is set 2024-05-08T08:52:13.963 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:52:13.966+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:52:14.049 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:52:14.053+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:52:14.070 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:52:14.071 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:52:14.071 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:52:14.071 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:52:14.071 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.071 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4189.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-05-08T08:52:14.071 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:51:58.895514+0000", 2024-05-08T08:52:14.071 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 15.177708745, 2024-05-08T08:52:14.071 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007747159, 2024-05-08T08:52:14.071 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:14.071 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:14.071 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:14.071 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:14.071 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4189", 2024-05-08T08:52:14.072 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/574763665", 2024-05-08T08:52:14.072 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 34 2024-05-08T08:52:14.072 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.072 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:14.072 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.072 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:14.072 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:58.895514+0000", 2024-05-08T08:52:14.072 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:14.072 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.072 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.072 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:14.072 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:58.895514+0000", 2024-05-08T08:52:14.072 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:14.072 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.072 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.073 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:14.073 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:58.895515+0000", 2024-05-08T08:52:14.073 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6700000000000003e-07 2024-05-08T08:52:14.073 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.073 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.073 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:14.073 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:58.895519+0000", 2024-05-08T08:52:14.073 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1239999999999998e-06 2024-05-08T08:52:14.073 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.073 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.073 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:14.073 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:58.895525+0000", 2024-05-08T08:52:14.073 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7559999999999996e-06 2024-05-08T08:52:14.073 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.073 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.074 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:14.074 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:58.895530+0000", 2024-05-08T08:52:14.074 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8849999999999998e-06 2024-05-08T08:52:14.074 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.074 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.074 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:14.074 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:58.897978+0000", 2024-05-08T08:52:14.074 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002448254 2024-05-08T08:52:14.074 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.074 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.074 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:14.074 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:58.898075+0000", 2024-05-08T08:52:14.074 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6502000000000005e-05 2024-05-08T08:52:14.074 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.075 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.075 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-05-08T08:52:14.075 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:58.898210+0000", 2024-05-08T08:52:14.075 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00013510400000000001 2024-05-08T08:52:14.075 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.075 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.075 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:52:14.075 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:58.900378+0000", 2024-05-08T08:52:14.075 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002168564 2024-05-08T08:52:14.075 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.075 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.075 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:14.075 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:58.903076+0000", 2024-05-08T08:52:14.075 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0026974220000000001 2024-05-08T08:52:14.075 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.076 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.076 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:14.076 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:58.903217+0000", 2024-05-08T08:52:14.076 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000141526 2024-05-08T08:52:14.076 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.076 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.076 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:14.076 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:58.903262+0000", 2024-05-08T08:52:14.076 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4455e-05 2024-05-08T08:52:14.076 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:14.076 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:14.076 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:14.076 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.076 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.076 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4189.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-05-08T08:52:14.077 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:51:58.895539+0000", 2024-05-08T08:52:14.077 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 15.177684126000001, 2024-05-08T08:52:14.077 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0078164510000000003, 2024-05-08T08:52:14.077 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:14.077 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:14.077 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:14.077 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:14.077 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4189", 2024-05-08T08:52:14.077 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/574763665", 2024-05-08T08:52:14.077 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 35 2024-05-08T08:52:14.077 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.077 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:14.077 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.078 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:14.078 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:58.895539+0000", 2024-05-08T08:52:14.078 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:14.078 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.078 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.078 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:14.078 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:58.895539+0000", 2024-05-08T08:52:14.078 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:14.078 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.078 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.078 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:14.078 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:58.895540+0000", 2024-05-08T08:52:14.078 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4499999999999997e-07 2024-05-08T08:52:14.078 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.079 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.079 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:14.079 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:58.895591+0000", 2024-05-08T08:52:14.079 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1218000000000003e-05 2024-05-08T08:52:14.079 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.079 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.079 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:14.079 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:58.895600+0000", 2024-05-08T08:52:14.079 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1589999999999999e-06 2024-05-08T08:52:14.079 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.079 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.079 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:14.079 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:58.895606+0000", 2024-05-08T08:52:14.080 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7679999999999997e-06 2024-05-08T08:52:14.080 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.080 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.080 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:14.080 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:58.898702+0000", 2024-05-08T08:52:14.080 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0030958869999999999 2024-05-08T08:52:14.080 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.080 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.080 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:14.080 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:58.898799+0000", 2024-05-08T08:52:14.080 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7738000000000005e-05 2024-05-08T08:52:14.080 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.081 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.081 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-05-08T08:52:14.081 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:58.898909+0000", 2024-05-08T08:52:14.081 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000109503 2024-05-08T08:52:14.081 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.081 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.081 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:52:14.081 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:58.900551+0000", 2024-05-08T08:52:14.081 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001641717 2024-05-08T08:52:14.081 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.081 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.081 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:14.081 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:58.903295+0000", 2024-05-08T08:52:14.082 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0027440799999999999 2024-05-08T08:52:14.082 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.082 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.082 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:14.082 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:58.903328+0000", 2024-05-08T08:52:14.082 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3714000000000003e-05 2024-05-08T08:52:14.082 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.082 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.082 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:14.082 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:58.903356+0000", 2024-05-08T08:52:14.082 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7222000000000001e-05 2024-05-08T08:52:14.082 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:14.082 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:14.082 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:14.082 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.083 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.083 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4189.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-05-08T08:52:14.083 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:51:58.895617+0000", 2024-05-08T08:52:14.083 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 15.177605929, 2024-05-08T08:52:14.083 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0098298229999999997, 2024-05-08T08:52:14.083 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:14.083 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:14.083 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:14.083 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:14.083 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4189", 2024-05-08T08:52:14.083 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/574763665", 2024-05-08T08:52:14.083 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 36 2024-05-08T08:52:14.083 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.083 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:14.083 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.084 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:14.084 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:58.895617+0000", 2024-05-08T08:52:14.084 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:14.084 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.084 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.084 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:14.084 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:58.895617+0000", 2024-05-08T08:52:14.084 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:14.084 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.084 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.085 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:14.085 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:58.895618+0000", 2024-05-08T08:52:14.085 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3499999999999996e-07 2024-05-08T08:52:14.085 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.085 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.085 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:14.085 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:58.895623+0000", 2024-05-08T08:52:14.085 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9400000000000001e-06 2024-05-08T08:52:14.085 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.085 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.086 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:14.086 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:58.895629+0000", 2024-05-08T08:52:14.086 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4849999999999997e-06 2024-05-08T08:52:14.086 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.086 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.086 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:14.086 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:58.895635+0000", 2024-05-08T08:52:14.086 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6300000000000003e-06 2024-05-08T08:52:14.086 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.086 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.086 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:14.086 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:58.899506+0000", 2024-05-08T08:52:14.086 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003871425 2024-05-08T08:52:14.087 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.087 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.087 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:14.087 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:58.899561+0000", 2024-05-08T08:52:14.087 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4214e-05 2024-05-08T08:52:14.087 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.087 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.087 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-05-08T08:52:14.087 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:58.899641+0000", 2024-05-08T08:52:14.087 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0527999999999999e-05 2024-05-08T08:52:14.087 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.087 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.087 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:52:14.088 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:58.900556+0000", 2024-05-08T08:52:14.088 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00091496000000000004 2024-05-08T08:52:14.088 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.088 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.088 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:14.088 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:58.905397+0000", 2024-05-08T08:52:14.088 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0048413700000000002 2024-05-08T08:52:14.088 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.088 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.088 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:14.088 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:58.905426+0000", 2024-05-08T08:52:14.088 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8158e-05 2024-05-08T08:52:14.088 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.089 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.089 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:14.089 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:58.905447+0000", 2024-05-08T08:52:14.089 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1478e-05 2024-05-08T08:52:14.089 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:14.089 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:14.089 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:14.089 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.089 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.089 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4189.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-05-08T08:52:14.089 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:51:58.895645+0000", 2024-05-08T08:52:14.089 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 15.177577897999999, 2024-05-08T08:52:14.089 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0099493129999999996, 2024-05-08T08:52:14.090 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:14.090 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:14.090 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:14.090 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:14.090 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4189", 2024-05-08T08:52:14.090 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/574763665", 2024-05-08T08:52:14.090 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 37 2024-05-08T08:52:14.090 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.090 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:14.090 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.090 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:14.090 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:58.895645+0000", 2024-05-08T08:52:14.091 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:14.091 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.091 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.091 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:14.091 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:58.895645+0000", 2024-05-08T08:52:14.091 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:14.091 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.091 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.091 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:14.091 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:58.895646+0000", 2024-05-08T08:52:14.091 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0999999999999999e-07 2024-05-08T08:52:14.091 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.091 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.092 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:14.092 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:58.895650+0000", 2024-05-08T08:52:14.092 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4279999999999998e-06 2024-05-08T08:52:14.092 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.092 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.092 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:14.092 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:58.895657+0000", 2024-05-08T08:52:14.092 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9029999999999998e-06 2024-05-08T08:52:14.092 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.092 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.092 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:14.092 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:58.895666+0000", 2024-05-08T08:52:14.092 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2779999999999999e-06 2024-05-08T08:52:14.092 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.093 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.093 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:14.093 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:58.899901+0000", 2024-05-08T08:52:14.093 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0042346689999999999 2024-05-08T08:52:14.093 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.093 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.093 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:14.093 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:58.899998+0000", 2024-05-08T08:52:14.093 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6903000000000004e-05 2024-05-08T08:52:14.093 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.093 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.093 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-05-08T08:52:14.093 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:58.900134+0000", 2024-05-08T08:52:14.094 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00013635599999999999 2024-05-08T08:52:14.094 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.094 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.094 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:52:14.094 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:58.900632+0000", 2024-05-08T08:52:14.094 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00049720500000000002 2024-05-08T08:52:14.094 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.094 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.094 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:14.094 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:58.905492+0000", 2024-05-08T08:52:14.095 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0048606200000000004 2024-05-08T08:52:14.095 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.095 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.095 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:14.095 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:58.905550+0000", 2024-05-08T08:52:14.095 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8171999999999998e-05 2024-05-08T08:52:14.095 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.095 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.095 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:14.095 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:58.905595+0000", 2024-05-08T08:52:14.095 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4268999999999999e-05 2024-05-08T08:52:14.095 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:14.096 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:14.096 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:14.096 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.096 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.096 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4189.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-05-08T08:52:14.096 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:51:59.117209+0000", 2024-05-08T08:52:14.096 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 14.956014274999999, 2024-05-08T08:52:14.096 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0084809269999999992, 2024-05-08T08:52:14.096 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:14.096 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:14.096 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:14.096 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:14.096 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4189", 2024-05-08T08:52:14.097 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/574763665", 2024-05-08T08:52:14.097 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 108 2024-05-08T08:52:14.097 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.097 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:14.097 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.097 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:14.097 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117209+0000", 2024-05-08T08:52:14.097 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:14.097 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.097 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.097 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:14.097 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117209+0000", 2024-05-08T08:52:14.097 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:14.097 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.098 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.098 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:14.098 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117210+0000", 2024-05-08T08:52:14.098 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5000000000000002e-07 2024-05-08T08:52:14.098 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.098 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.098 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:14.098 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117214+0000", 2024-05-08T08:52:14.098 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7620000000000003e-06 2024-05-08T08:52:14.098 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.098 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.098 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:14.098 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117224+0000", 2024-05-08T08:52:14.099 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6150000000000003e-06 2024-05-08T08:52:14.099 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.099 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.099 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:14.099 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117230+0000", 2024-05-08T08:52:14.099 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6219999999999997e-06 2024-05-08T08:52:14.099 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.099 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.099 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:14.099 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.119295+0000", 2024-05-08T08:52:14.099 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020656440000000002 2024-05-08T08:52:14.099 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.100 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.100 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:14.100 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.119338+0000", 2024-05-08T08:52:14.100 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2567000000000001e-05 2024-05-08T08:52:14.100 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.100 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.101 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-05-08T08:52:14.101 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.119406+0000", 2024-05-08T08:52:14.101 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8559999999999994e-05 2024-05-08T08:52:14.101 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.101 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.101 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:52:14.101 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.121130+0000", 2024-05-08T08:52:14.101 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001723895 2024-05-08T08:52:14.101 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.101 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.101 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:14.101 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.125627+0000", 2024-05-08T08:52:14.101 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0044962860000000004 2024-05-08T08:52:14.101 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.102 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.102 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:14.102 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.125660+0000", 2024-05-08T08:52:14.102 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3855e-05 2024-05-08T08:52:14.102 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.102 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.102 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:14.102 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.125690+0000", 2024-05-08T08:52:14.102 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9471e-05 2024-05-08T08:52:14.102 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:14.102 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:14.102 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:14.102 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.103 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.103 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4189.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-05-08T08:52:14.103 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:51:59.117252+0000", 2024-05-08T08:52:14.103 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 14.955971678000001, 2024-05-08T08:52:14.103 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0085290609999999992, 2024-05-08T08:52:14.103 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:14.103 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:14.103 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:14.103 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:14.103 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4189", 2024-05-08T08:52:14.103 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/574763665", 2024-05-08T08:52:14.103 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 109 2024-05-08T08:52:14.103 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.103 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:14.104 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.104 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:14.104 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117252+0000", 2024-05-08T08:52:14.104 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:14.104 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.104 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.104 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:14.104 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117252+0000", 2024-05-08T08:52:14.104 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:14.104 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.104 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.104 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:14.104 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117252+0000", 2024-05-08T08:52:14.104 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6300000000000005e-07 2024-05-08T08:52:14.105 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.105 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.105 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:14.105 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117299+0000", 2024-05-08T08:52:14.105 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6727000000000002e-05 2024-05-08T08:52:14.105 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.105 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.105 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:14.105 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117307+0000", 2024-05-08T08:52:14.105 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3669999999999997e-06 2024-05-08T08:52:14.105 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.105 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.105 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:14.106 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117313+0000", 2024-05-08T08:52:14.106 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8699999999999997e-06 2024-05-08T08:52:14.106 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.106 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.106 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:14.106 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.119657+0000", 2024-05-08T08:52:14.106 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0023440840000000002 2024-05-08T08:52:14.106 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.106 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.106 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:14.106 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.119701+0000", 2024-05-08T08:52:14.106 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3748000000000002e-05 2024-05-08T08:52:14.106 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.106 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.107 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-05-08T08:52:14.107 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.119768+0000", 2024-05-08T08:52:14.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6828000000000005e-05 2024-05-08T08:52:14.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.107 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:52:14.107 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.122138+0000", 2024-05-08T08:52:14.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0023701389999999998 2024-05-08T08:52:14.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.107 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:14.107 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.125729+0000", 2024-05-08T08:52:14.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003590766 2024-05-08T08:52:14.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.108 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:14.108 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.125758+0000", 2024-05-08T08:52:14.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9023999999999999e-05 2024-05-08T08:52:14.108 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.108 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.108 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:14.108 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.125781+0000", 2024-05-08T08:52:14.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2945000000000001e-05 2024-05-08T08:52:14.108 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:14.108 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:14.108 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:14.108 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.108 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.108 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4189.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-05-08T08:52:14.109 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:51:59.117325+0000", 2024-05-08T08:52:14.109 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 14.955898619999999, 2024-05-08T08:52:14.109 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0085358469999999992, 2024-05-08T08:52:14.109 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:14.109 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:14.109 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:14.109 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:14.109 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4189", 2024-05-08T08:52:14.109 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/574763665", 2024-05-08T08:52:14.109 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 110 2024-05-08T08:52:14.109 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.109 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:14.109 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.109 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:14.109 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117325+0000", 2024-05-08T08:52:14.110 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:14.110 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.110 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.110 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:14.110 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117325+0000", 2024-05-08T08:52:14.110 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:14.110 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.110 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.110 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:14.110 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117325+0000", 2024-05-08T08:52:14.110 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6800000000000001e-07 2024-05-08T08:52:14.110 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.110 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.111 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:14.111 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117329+0000", 2024-05-08T08:52:14.111 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3150000000000002e-06 2024-05-08T08:52:14.111 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.111 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.111 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:14.111 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117336+0000", 2024-05-08T08:52:14.111 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3459999999999999e-06 2024-05-08T08:52:14.111 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.111 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.111 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:14.111 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117342+0000", 2024-05-08T08:52:14.111 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8529999999999997e-06 2024-05-08T08:52:14.111 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.111 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.112 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:14.112 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.120613+0000", 2024-05-08T08:52:14.112 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0032715370000000001 2024-05-08T08:52:14.112 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.112 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.112 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:14.112 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.120657+0000", 2024-05-08T08:52:14.112 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4292000000000002e-05 2024-05-08T08:52:14.112 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.112 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.112 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-05-08T08:52:14.112 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.120725+0000", 2024-05-08T08:52:14.112 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7880999999999996e-05 2024-05-08T08:52:14.112 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.113 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.113 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:52:14.113 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.122844+0000", 2024-05-08T08:52:14.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0021191840000000001 2024-05-08T08:52:14.113 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.113 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.113 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:14.113 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.125814+0000", 2024-05-08T08:52:14.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0029697529999999999 2024-05-08T08:52:14.113 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.113 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.113 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:14.113 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.125839+0000", 2024-05-08T08:52:14.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5221000000000001e-05 2024-05-08T08:52:14.114 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.114 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.114 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:14.114 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.125860+0000", 2024-05-08T08:52:14.114 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0996999999999999e-05 2024-05-08T08:52:14.114 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:14.114 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:14.114 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:14.114 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.114 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.114 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4189.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-05-08T08:52:14.114 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:51:59.117351+0000", 2024-05-08T08:52:14.115 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 14.955872261, 2024-05-08T08:52:14.115 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0085922610000000003, 2024-05-08T08:52:14.115 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:14.115 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:14.115 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:14.115 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:14.115 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4189", 2024-05-08T08:52:14.115 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/574763665", 2024-05-08T08:52:14.115 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 111 2024-05-08T08:52:14.115 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.115 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:14.115 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.115 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:14.115 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117351+0000", 2024-05-08T08:52:14.116 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:14.116 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.116 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.116 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:14.116 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117351+0000", 2024-05-08T08:52:14.116 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:14.116 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.116 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.116 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:14.116 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117351+0000", 2024-05-08T08:52:14.116 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5300000000000004e-07 2024-05-08T08:52:14.116 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.116 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.117 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:14.117 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117355+0000", 2024-05-08T08:52:14.117 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9979999999999997e-06 2024-05-08T08:52:14.117 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.117 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.117 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:14.117 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117363+0000", 2024-05-08T08:52:14.117 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7889999999999995e-06 2024-05-08T08:52:14.117 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.117 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.117 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:14.117 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117371+0000", 2024-05-08T08:52:14.117 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2849999999999997e-06 2024-05-08T08:52:14.117 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.118 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.118 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:14.118 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.121156+0000", 2024-05-08T08:52:14.118 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0037849720000000002 2024-05-08T08:52:14.118 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.118 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.118 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:14.118 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.121197+0000", 2024-05-08T08:52:14.118 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1743000000000003e-05 2024-05-08T08:52:14.118 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.118 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.118 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-05-08T08:52:14.118 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.121267+0000", 2024-05-08T08:52:14.118 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9754999999999999e-05 2024-05-08T08:52:14.118 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.119 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.119 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:52:14.119 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.122851+0000", 2024-05-08T08:52:14.119 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015838689999999999 2024-05-08T08:52:14.119 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.119 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.119 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:14.119 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.125892+0000", 2024-05-08T08:52:14.119 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0030415569999999999 2024-05-08T08:52:14.119 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.119 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.119 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:14.119 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.125917+0000", 2024-05-08T08:52:14.119 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4828e-05 2024-05-08T08:52:14.120 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.120 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.120 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:14.120 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.125943+0000", 2024-05-08T08:52:14.120 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5911999999999999e-05 2024-05-08T08:52:14.120 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:14.120 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:14.120 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:14.120 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.120 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.120 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4189.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-05-08T08:52:14.120 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:51:59.117383+0000", 2024-05-08T08:52:14.120 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 14.955840307000001, 2024-05-08T08:52:14.121 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0086381340000000004, 2024-05-08T08:52:14.121 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:14.121 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:14.121 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:14.121 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:14.121 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4189", 2024-05-08T08:52:14.121 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/574763665", 2024-05-08T08:52:14.121 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 112 2024-05-08T08:52:14.121 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.121 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:14.121 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.121 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:14.122 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117383+0000", 2024-05-08T08:52:14.122 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:14.122 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.122 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.122 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:14.122 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117383+0000", 2024-05-08T08:52:14.122 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:14.122 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.122 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.122 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:14.122 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117383+0000", 2024-05-08T08:52:14.122 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6499999999999999e-07 2024-05-08T08:52:14.122 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.123 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.123 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:14.123 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117438+0000", 2024-05-08T08:52:14.123 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4775000000000003e-05 2024-05-08T08:52:14.123 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.123 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.123 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:14.123 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117446+0000", 2024-05-08T08:52:14.123 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5950000000000001e-06 2024-05-08T08:52:14.123 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.123 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.123 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:14.123 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117452+0000", 2024-05-08T08:52:14.123 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2299999999999996e-06 2024-05-08T08:52:14.124 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.124 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.124 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:14.124 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.121500+0000", 2024-05-08T08:52:14.124 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0040479069999999999 2024-05-08T08:52:14.124 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.124 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.124 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:14.124 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.121547+0000", 2024-05-08T08:52:14.124 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7274000000000002e-05 2024-05-08T08:52:14.124 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.124 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.124 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-05-08T08:52:14.125 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.121618+0000", 2024-05-08T08:52:14.125 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0733000000000002e-05 2024-05-08T08:52:14.125 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.125 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.125 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:52:14.125 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.123852+0000", 2024-05-08T08:52:14.125 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0022343649999999999 2024-05-08T08:52:14.125 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.125 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.125 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:14.125 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.125975+0000", 2024-05-08T08:52:14.125 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0021230609999999999 2024-05-08T08:52:14.125 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.125 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.126 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:14.126 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.126000+0000", 2024-05-08T08:52:14.126 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4875000000000002e-05 2024-05-08T08:52:14.126 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.126 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.126 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:14.126 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.126021+0000", 2024-05-08T08:52:14.126 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0854000000000001e-05 2024-05-08T08:52:14.126 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:14.126 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:14.126 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:14.126 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.126 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.127 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4189.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-05-08T08:52:14.127 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:51:59.117464+0000", 2024-05-08T08:52:14.127 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 14.955759679, 2024-05-08T08:52:14.127 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0086338920000000006, 2024-05-08T08:52:14.127 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:14.127 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:14.127 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:14.127 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:14.127 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4189", 2024-05-08T08:52:14.127 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/574763665", 2024-05-08T08:52:14.127 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 113 2024-05-08T08:52:14.127 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.127 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:14.127 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.128 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:14.128 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117464+0000", 2024-05-08T08:52:14.128 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:14.129 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.129 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.129 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:14.129 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117464+0000", 2024-05-08T08:52:14.129 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:14.129 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.129 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.130 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:14.130 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117464+0000", 2024-05-08T08:52:14.130 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2e-07 2024-05-08T08:52:14.130 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.130 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.130 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:14.130 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117468+0000", 2024-05-08T08:52:14.130 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2610000000000004e-06 2024-05-08T08:52:14.130 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.130 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.130 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:14.130 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117475+0000", 2024-05-08T08:52:14.130 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3400000000000003e-06 2024-05-08T08:52:14.131 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.131 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.131 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:14.131 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117480+0000", 2024-05-08T08:52:14.131 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8740000000000003e-06 2024-05-08T08:52:14.131 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.131 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.131 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:14.131 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.122167+0000", 2024-05-08T08:52:14.131 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0046871889999999996 2024-05-08T08:52:14.131 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.131 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.131 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:14.131 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.122212+0000", 2024-05-08T08:52:14.131 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5488000000000002e-05 2024-05-08T08:52:14.132 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.132 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.132 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-05-08T08:52:14.132 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.122278+0000", 2024-05-08T08:52:14.132 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5690000000000003e-05 2024-05-08T08:52:14.132 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.132 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.132 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:52:14.132 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.123858+0000", 2024-05-08T08:52:14.132 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015798100000000001 2024-05-08T08:52:14.132 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.132 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.132 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:14.132 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.126052+0000", 2024-05-08T08:52:14.132 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002194714 2024-05-08T08:52:14.133 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.133 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.133 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:14.133 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.126078+0000", 2024-05-08T08:52:14.133 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5669000000000001e-05 2024-05-08T08:52:14.133 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.133 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.133 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:14.133 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.126097+0000", 2024-05-08T08:52:14.133 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9337000000000001e-05 2024-05-08T08:52:14.133 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:14.133 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:14.133 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:14.133 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.133 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.134 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4189.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-05-08T08:52:14.134 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:51:59.117493+0000", 2024-05-08T08:52:14.134 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 14.955730364000001, 2024-05-08T08:52:14.134 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0086786899999999993, 2024-05-08T08:52:14.134 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:14.134 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:14.134 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:14.134 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:14.134 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4189", 2024-05-08T08:52:14.134 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/574763665", 2024-05-08T08:52:14.134 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 114 2024-05-08T08:52:14.134 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.134 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:14.135 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.135 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:14.135 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117493+0000", 2024-05-08T08:52:14.135 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:14.135 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.135 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.135 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:14.135 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117493+0000", 2024-05-08T08:52:14.135 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:14.135 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.135 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.135 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:14.135 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117493+0000", 2024-05-08T08:52:14.135 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.06e-07 2024-05-08T08:52:14.135 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.136 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.136 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:14.136 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117498+0000", 2024-05-08T08:52:14.136 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2599999999999999e-06 2024-05-08T08:52:14.136 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.136 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.136 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:14.136 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117505+0000", 2024-05-08T08:52:14.136 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9870000000000002e-06 2024-05-08T08:52:14.136 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.136 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.136 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:14.136 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117510+0000", 2024-05-08T08:52:14.136 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9830000000000004e-06 2024-05-08T08:52:14.136 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.137 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.137 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:14.137 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.122513+0000", 2024-05-08T08:52:14.137 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0050029499999999999 2024-05-08T08:52:14.137 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.137 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.137 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:14.137 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.122555+0000", 2024-05-08T08:52:14.137 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2715e-05 2024-05-08T08:52:14.137 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.137 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.137 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-05-08T08:52:14.137 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.122623+0000", 2024-05-08T08:52:14.137 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7261999999999994e-05 2024-05-08T08:52:14.138 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.138 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.138 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:52:14.138 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.123865+0000", 2024-05-08T08:52:14.138 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0012428960000000001 2024-05-08T08:52:14.138 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.138 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.138 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:14.138 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.126126+0000", 2024-05-08T08:52:14.138 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0022607389999999999 2024-05-08T08:52:14.138 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.138 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.138 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:14.138 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.126151+0000", 2024-05-08T08:52:14.138 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5258000000000001e-05 2024-05-08T08:52:14.139 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.139 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.139 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:14.139 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.126172+0000", 2024-05-08T08:52:14.139 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0134000000000001e-05 2024-05-08T08:52:14.139 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:14.139 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:14.139 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:14.139 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.139 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.139 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4189.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-05-08T08:52:14.139 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:51:59.117577+0000", 2024-05-08T08:52:14.139 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 14.955646202, 2024-05-08T08:52:14.139 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010674029, 2024-05-08T08:52:14.140 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:14.140 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:14.140 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:14.140 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:14.140 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4189", 2024-05-08T08:52:14.140 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/574763665", 2024-05-08T08:52:14.140 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 115 2024-05-08T08:52:14.140 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.140 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:14.140 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.140 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:14.140 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117577+0000", 2024-05-08T08:52:14.140 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:14.140 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.141 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.141 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:14.141 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117577+0000", 2024-05-08T08:52:14.141 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:14.141 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.141 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.141 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:14.141 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117578+0000", 2024-05-08T08:52:14.141 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0299999999999999e-07 2024-05-08T08:52:14.141 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.141 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.141 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:14.141 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117622+0000", 2024-05-08T08:52:14.141 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4935999999999998e-05 2024-05-08T08:52:14.142 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.142 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.142 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:14.142 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117630+0000", 2024-05-08T08:52:14.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6899999999999992e-06 2024-05-08T08:52:14.142 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.142 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.142 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:14.142 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117636+0000", 2024-05-08T08:52:14.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3119999999999999e-06 2024-05-08T08:52:14.142 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.142 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.142 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:14.143 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.122880+0000", 2024-05-08T08:52:14.143 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0052432930000000004 2024-05-08T08:52:14.143 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.143 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.143 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:14.143 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.122923+0000", 2024-05-08T08:52:14.143 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3022999999999998e-05 2024-05-08T08:52:14.143 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.143 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.143 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-05-08T08:52:14.143 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.122997+0000", 2024-05-08T08:52:14.143 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4574999999999997e-05 2024-05-08T08:52:14.143 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.143 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.144 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:52:14.144 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.124183+0000", 2024-05-08T08:52:14.144 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011852099999999999 2024-05-08T08:52:14.144 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.144 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.144 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:14.144 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.128204+0000", 2024-05-08T08:52:14.144 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0040213649999999998 2024-05-08T08:52:14.144 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.144 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.144 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:14.144 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.128230+0000", 2024-05-08T08:52:14.145 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6210000000000001e-05 2024-05-08T08:52:14.145 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.145 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.145 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:14.145 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.128251+0000", 2024-05-08T08:52:14.145 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0911999999999999e-05 2024-05-08T08:52:14.145 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:14.145 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:14.145 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:14.145 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.145 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.145 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4189.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-05-08T08:52:14.145 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:51:59.117662+0000", 2024-05-08T08:52:14.145 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 14.955561696, 2024-05-08T08:52:14.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010668166999999999, 2024-05-08T08:52:14.146 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:14.146 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:14.146 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:14.146 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:14.146 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4189", 2024-05-08T08:52:14.146 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/574763665", 2024-05-08T08:52:14.146 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 116 2024-05-08T08:52:14.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.146 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:14.146 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.146 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:14.146 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117662+0000", 2024-05-08T08:52:14.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:14.147 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.147 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.147 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:14.147 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117662+0000", 2024-05-08T08:52:14.147 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:14.147 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.147 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.147 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:14.147 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117662+0000", 2024-05-08T08:52:14.147 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.82e-07 2024-05-08T08:52:14.147 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.147 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.147 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:14.148 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117668+0000", 2024-05-08T08:52:14.148 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4099999999999999e-06 2024-05-08T08:52:14.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.148 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.148 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:14.148 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117675+0000", 2024-05-08T08:52:14.148 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7230000000000004e-06 2024-05-08T08:52:14.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.148 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.148 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:14.148 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117680+0000", 2024-05-08T08:52:14.148 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8029999999999996e-06 2024-05-08T08:52:14.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.148 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.148 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:14.149 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.123525+0000", 2024-05-08T08:52:14.149 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0058446749999999997 2024-05-08T08:52:14.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.149 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:14.149 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.123569+0000", 2024-05-08T08:52:14.149 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3853999999999997e-05 2024-05-08T08:52:14.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.149 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-05-08T08:52:14.149 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.123638+0000", 2024-05-08T08:52:14.149 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9666999999999996e-05 2024-05-08T08:52:14.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.150 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:52:14.150 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.124242+0000", 2024-05-08T08:52:14.150 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00060348599999999997 2024-05-08T08:52:14.150 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.150 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.150 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:14.150 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.128282+0000", 2024-05-08T08:52:14.150 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0040407739999999996 2024-05-08T08:52:14.150 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.150 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.150 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:14.150 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.128309+0000", 2024-05-08T08:52:14.150 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6519000000000001e-05 2024-05-08T08:52:14.150 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.151 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.151 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:14.151 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.128330+0000", 2024-05-08T08:52:14.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0673999999999999e-05 2024-05-08T08:52:14.151 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:14.151 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:14.151 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:14.151 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.151 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.151 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4189.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-05-08T08:52:14.151 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:51:59.117691+0000", 2024-05-08T08:52:14.151 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 14.955532266000001, 2024-05-08T08:52:14.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010707118999999999, 2024-05-08T08:52:14.151 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:14.151 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:14.151 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:14.152 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:14.152 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4189", 2024-05-08T08:52:14.152 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/574763665", 2024-05-08T08:52:14.152 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 117 2024-05-08T08:52:14.152 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.152 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:14.152 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.152 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:14.152 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117691+0000", 2024-05-08T08:52:14.152 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:14.152 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.152 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.152 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:14.152 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117691+0000", 2024-05-08T08:52:14.153 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:14.153 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.153 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.153 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:14.153 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117691+0000", 2024-05-08T08:52:14.153 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4900000000000001e-07 2024-05-08T08:52:14.153 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.153 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.153 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:14.153 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117696+0000", 2024-05-08T08:52:14.153 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0509999999999996e-06 2024-05-08T08:52:14.153 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.153 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.153 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:14.154 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117703+0000", 2024-05-08T08:52:14.154 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1229999999999997e-06 2024-05-08T08:52:14.154 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.154 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.154 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:14.154 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117708+0000", 2024-05-08T08:52:14.154 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5310000000000001e-06 2024-05-08T08:52:14.154 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.154 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.154 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:14.154 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.123886+0000", 2024-05-08T08:52:14.154 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0061775750000000003 2024-05-08T08:52:14.154 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.155 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.155 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:14.155 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.123937+0000", 2024-05-08T08:52:14.155 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1669999999999998e-05 2024-05-08T08:52:14.155 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.155 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.155 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-05-08T08:52:14.155 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.124009+0000", 2024-05-08T08:52:14.155 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1815000000000003e-05 2024-05-08T08:52:14.155 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.155 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.155 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:52:14.155 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.124474+0000", 2024-05-08T08:52:14.156 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00046435299999999998 2024-05-08T08:52:14.156 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.156 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.156 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:14.156 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.128354+0000", 2024-05-08T08:52:14.156 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003880175 2024-05-08T08:52:14.156 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.156 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.156 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:14.156 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.128379+0000", 2024-05-08T08:52:14.156 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5143999999999998e-05 2024-05-08T08:52:14.156 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.156 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.156 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:14.156 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.128398+0000", 2024-05-08T08:52:14.157 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9233e-05 2024-05-08T08:52:14.157 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:14.157 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:14.157 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:14.157 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.157 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.157 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4189.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e13)", 2024-05-08T08:52:14.157 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:51:59.444265+0000", 2024-05-08T08:52:14.157 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 14.62895808, 2024-05-08T08:52:14.157 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0083803479999999993, 2024-05-08T08:52:14.157 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:14.157 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:14.157 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:14.157 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:14.158 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4189", 2024-05-08T08:52:14.158 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/574763665", 2024-05-08T08:52:14.158 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 137 2024-05-08T08:52:14.158 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.158 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:14.158 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.158 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:14.158 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.444265+0000", 2024-05-08T08:52:14.158 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:14.158 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.158 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.158 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:14.158 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.444265+0000", 2024-05-08T08:52:14.159 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:14.159 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.159 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.159 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:14.159 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.444265+0000", 2024-05-08T08:52:14.159 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3300000000000001e-07 2024-05-08T08:52:14.159 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.159 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.159 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:14.159 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.444290+0000", 2024-05-08T08:52:14.159 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4549000000000002e-05 2024-05-08T08:52:14.159 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.159 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.159 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:14.160 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.444294+0000", 2024-05-08T08:52:14.160 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8410000000000002e-06 2024-05-08T08:52:14.160 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.160 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.160 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:14.160 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.444297+0000", 2024-05-08T08:52:14.160 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.977e-06 2024-05-08T08:52:14.160 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.160 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.160 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:14.160 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.448384+0000", 2024-05-08T08:52:14.160 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0040870589999999997 2024-05-08T08:52:14.160 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.160 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.161 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:14.161 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.448462+0000", 2024-05-08T08:52:14.161 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8636999999999999e-05 2024-05-08T08:52:14.161 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.161 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.161 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-05-08T08:52:14.161 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.448596+0000", 2024-05-08T08:52:14.161 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00013311400000000001 2024-05-08T08:52:14.161 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.161 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.161 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:52:14.161 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.450551+0000", 2024-05-08T08:52:14.161 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0019551759999999999 2024-05-08T08:52:14.161 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.161 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.162 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:14.162 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.452554+0000", 2024-05-08T08:52:14.162 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020033429999999999 2024-05-08T08:52:14.162 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.162 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.162 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:14.162 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.452607+0000", 2024-05-08T08:52:14.162 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.287e-05 2024-05-08T08:52:14.162 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.162 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.162 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:14.162 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.452645+0000", 2024-05-08T08:52:14.162 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8549e-05 2024-05-08T08:52:14.162 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:14.163 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:14.163 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:14.163 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.163 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.163 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4189.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e13)", 2024-05-08T08:52:14.163 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:51:59.444302+0000", 2024-05-08T08:52:14.163 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 14.628921590999999, 2024-05-08T08:52:14.163 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0084568960000000002, 2024-05-08T08:52:14.163 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:14.163 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:14.163 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:14.163 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:14.163 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4189", 2024-05-08T08:52:14.163 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/574763665", 2024-05-08T08:52:14.163 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 138 2024-05-08T08:52:14.164 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.164 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:14.164 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.164 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:14.164 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.444302+0000", 2024-05-08T08:52:14.164 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:14.164 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.164 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.164 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:14.164 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.444302+0000", 2024-05-08T08:52:14.164 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:14.164 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.164 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.164 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:14.164 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.444302+0000", 2024-05-08T08:52:14.165 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3999999999999997e-07 2024-05-08T08:52:14.165 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.165 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.165 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:14.165 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.444306+0000", 2024-05-08T08:52:14.165 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.703e-06 2024-05-08T08:52:14.165 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.165 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.165 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:14.165 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.444309+0000", 2024-05-08T08:52:14.165 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.247e-06 2024-05-08T08:52:14.165 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.165 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.165 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:14.165 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.444312+0000", 2024-05-08T08:52:14.166 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.489e-06 2024-05-08T08:52:14.166 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.166 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.166 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:14.166 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.449442+0000", 2024-05-08T08:52:14.166 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0051297809999999999 2024-05-08T08:52:14.166 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.166 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.166 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:14.166 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.449491+0000", 2024-05-08T08:52:14.166 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8884000000000002e-05 2024-05-08T08:52:14.166 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.166 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.166 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-05-08T08:52:14.166 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.449561+0000", 2024-05-08T08:52:14.167 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0291999999999997e-05 2024-05-08T08:52:14.167 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.167 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.167 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:52:14.167 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.452239+0000", 2024-05-08T08:52:14.167 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0026780279999999998 2024-05-08T08:52:14.167 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.167 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.167 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:14.167 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.452694+0000", 2024-05-08T08:52:14.167 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00045447199999999998 2024-05-08T08:52:14.167 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.167 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.167 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:14.167 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.452724+0000", 2024-05-08T08:52:14.168 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0032e-05 2024-05-08T08:52:14.168 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.168 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.168 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:14.168 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.452759+0000", 2024-05-08T08:52:14.168 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4628000000000001e-05 2024-05-08T08:52:14.168 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:14.168 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:14.168 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:14.168 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.168 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.168 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4189.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e13)", 2024-05-08T08:52:14.168 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:51:59.444340+0000", 2024-05-08T08:52:14.168 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 14.628883231, 2024-05-08T08:52:14.168 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0085447240000000001, 2024-05-08T08:52:14.169 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:14.169 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:14.169 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:14.169 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:14.169 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4189", 2024-05-08T08:52:14.169 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/574763665", 2024-05-08T08:52:14.169 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 139 2024-05-08T08:52:14.169 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.169 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:14.169 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.169 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:14.169 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.444340+0000", 2024-05-08T08:52:14.169 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:14.169 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.170 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.170 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:14.170 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.444340+0000", 2024-05-08T08:52:14.170 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:14.170 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.170 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.170 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:14.170 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.444340+0000", 2024-05-08T08:52:14.170 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2300000000000001e-07 2024-05-08T08:52:14.170 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.170 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.170 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:14.170 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.444345+0000", 2024-05-08T08:52:14.170 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7770000000000002e-06 2024-05-08T08:52:14.170 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.171 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.171 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:14.171 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.444353+0000", 2024-05-08T08:52:14.171 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1610000000000003e-06 2024-05-08T08:52:14.171 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.171 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.171 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:14.171 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.444359+0000", 2024-05-08T08:52:14.171 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2789999999999998e-06 2024-05-08T08:52:14.171 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.171 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.171 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:14.171 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.450067+0000", 2024-05-08T08:52:14.171 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.005707869 2024-05-08T08:52:14.171 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.172 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.172 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:14.172 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.450141+0000", 2024-05-08T08:52:14.172 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4035000000000005e-05 2024-05-08T08:52:14.172 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.172 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.172 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-05-08T08:52:14.172 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.450261+0000", 2024-05-08T08:52:14.172 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000120219 2024-05-08T08:52:14.172 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.172 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.172 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:52:14.172 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.452245+0000", 2024-05-08T08:52:14.173 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0019839329999999998 2024-05-08T08:52:14.173 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.173 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.173 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:14.173 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.452800+0000", 2024-05-08T08:52:14.173 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00055537999999999996 2024-05-08T08:52:14.173 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.173 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.173 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:14.173 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.452849+0000", 2024-05-08T08:52:14.173 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8922999999999999e-05 2024-05-08T08:52:14.173 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.173 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.173 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:14.173 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.452885+0000", 2024-05-08T08:52:14.174 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5624999999999999e-05 2024-05-08T08:52:14.174 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:14.174 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:14.174 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:14.174 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.174 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.174 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4189.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e13)", 2024-05-08T08:52:14.174 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:51:59.444369+0000", 2024-05-08T08:52:14.174 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 14.628853935, 2024-05-08T08:52:14.174 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0086469149999999998, 2024-05-08T08:52:14.174 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:14.174 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:14.174 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:14.174 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:14.175 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4189", 2024-05-08T08:52:14.175 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/574763665", 2024-05-08T08:52:14.175 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 140 2024-05-08T08:52:14.175 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.175 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:14.175 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.175 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:14.175 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.444369+0000", 2024-05-08T08:52:14.175 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:14.175 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.175 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.175 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:14.175 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.444369+0000", 2024-05-08T08:52:14.176 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:14.176 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.176 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.176 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:14.176 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.444370+0000", 2024-05-08T08:52:14.176 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6600000000000002e-07 2024-05-08T08:52:14.176 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.176 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.176 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:14.176 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.444433+0000", 2024-05-08T08:52:14.176 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3433000000000001e-05 2024-05-08T08:52:14.176 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.176 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.176 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:14.177 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.444440+0000", 2024-05-08T08:52:14.177 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.985e-06 2024-05-08T08:52:14.177 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.177 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.177 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:14.177 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.444444+0000", 2024-05-08T08:52:14.177 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8670000000000004e-06 2024-05-08T08:52:14.177 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.177 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.177 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:14.177 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.450579+0000", 2024-05-08T08:52:14.177 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0061353969999999999 2024-05-08T08:52:14.177 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.177 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.178 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:14.178 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.450629+0000", 2024-05-08T08:52:14.178 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0062000000000003e-05 2024-05-08T08:52:14.178 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.178 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.178 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-05-08T08:52:14.178 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.450708+0000", 2024-05-08T08:52:14.178 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8576000000000001e-05 2024-05-08T08:52:14.178 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.178 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.178 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:52:14.178 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.452440+0000", 2024-05-08T08:52:14.178 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0017315620000000001 2024-05-08T08:52:14.178 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.179 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.179 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:14.179 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.452930+0000", 2024-05-08T08:52:14.179 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00049034199999999995 2024-05-08T08:52:14.179 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.179 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.179 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:14.179 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.452980+0000", 2024-05-08T08:52:14.179 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0436999999999998e-05 2024-05-08T08:52:14.179 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.179 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.179 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:14.179 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.453016+0000", 2024-05-08T08:52:14.179 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5787999999999997e-05 2024-05-08T08:52:14.179 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:14.180 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:14.180 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:14.180 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.180 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.180 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4189.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e13)", 2024-05-08T08:52:14.180 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:51:59.444452+0000", 2024-05-08T08:52:14.180 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 14.628771215, 2024-05-08T08:52:14.180 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0088408830000000008, 2024-05-08T08:52:14.180 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:14.180 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:14.180 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:14.181 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:14.181 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4189", 2024-05-08T08:52:14.181 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/574763665", 2024-05-08T08:52:14.181 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 141 2024-05-08T08:52:14.181 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.181 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:14.181 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.181 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:14.181 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.444452+0000", 2024-05-08T08:52:14.181 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:14.181 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.181 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.181 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:14.181 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.444452+0000", 2024-05-08T08:52:14.181 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:14.182 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.182 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.182 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:14.182 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.444452+0000", 2024-05-08T08:52:14.182 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.39e-07 2024-05-08T08:52:14.182 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.182 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.182 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:14.182 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.444455+0000", 2024-05-08T08:52:14.182 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4439999999999998e-06 2024-05-08T08:52:14.182 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.182 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.183 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:14.183 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.444458+0000", 2024-05-08T08:52:14.183 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5769999999999998e-06 2024-05-08T08:52:14.183 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.183 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.183 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:14.183 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.444461+0000", 2024-05-08T08:52:14.183 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6309999999999999e-06 2024-05-08T08:52:14.183 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.183 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.183 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:14.183 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.451765+0000", 2024-05-08T08:52:14.183 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007303749 2024-05-08T08:52:14.184 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.184 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.184 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:14.184 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.451841+0000", 2024-05-08T08:52:14.184 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6202e-05 2024-05-08T08:52:14.184 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.184 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.184 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-05-08T08:52:14.184 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.451965+0000", 2024-05-08T08:52:14.184 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012429699999999999 2024-05-08T08:52:14.184 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.184 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.184 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:14.185 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.453203+0000", 2024-05-08T08:52:14.185 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001237995 2024-05-08T08:52:14.185 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.185 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.185 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:52:14.185 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.453214+0000", 2024-05-08T08:52:14.185 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0841e-05 2024-05-08T08:52:14.185 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.185 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.185 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:14.185 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.453256+0000", 2024-05-08T08:52:14.185 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1597000000000002e-05 2024-05-08T08:52:14.185 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.186 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.186 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:14.186 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.453293+0000", 2024-05-08T08:52:14.186 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7110999999999998e-05 2024-05-08T08:52:14.186 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:14.186 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:14.186 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:14.186 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.186 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.186 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4189.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-05-08T08:52:14.186 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:51:59.875347+0000", 2024-05-08T08:52:14.186 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 14.197876054, 2024-05-08T08:52:14.186 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0076771210000000003, 2024-05-08T08:52:14.186 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:14.186 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:14.187 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:14.187 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:14.187 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4189", 2024-05-08T08:52:14.187 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/574763665", 2024-05-08T08:52:14.187 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 159 2024-05-08T08:52:14.187 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.187 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:14.187 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.187 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:14.187 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.875347+0000", 2024-05-08T08:52:14.187 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:14.187 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.187 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.187 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:14.188 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.875347+0000", 2024-05-08T08:52:14.188 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:14.188 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.188 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.188 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:14.188 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.875347+0000", 2024-05-08T08:52:14.188 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3799999999999999e-07 2024-05-08T08:52:14.188 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.188 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.188 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:14.188 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.875350+0000", 2024-05-08T08:52:14.188 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3499999999999999e-06 2024-05-08T08:52:14.188 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.188 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.189 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:14.189 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.875353+0000", 2024-05-08T08:52:14.189 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1540000000000002e-06 2024-05-08T08:52:14.189 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.189 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.189 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:14.189 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.875355+0000", 2024-05-08T08:52:14.189 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4119999999999999e-06 2024-05-08T08:52:14.189 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.189 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.189 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:14.189 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.879510+0000", 2024-05-08T08:52:14.189 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0041541959999999998 2024-05-08T08:52:14.189 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.190 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.190 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:14.190 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.879556+0000", 2024-05-08T08:52:14.190 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6008999999999999e-05 2024-05-08T08:52:14.190 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.190 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.190 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-05-08T08:52:14.190 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.879623+0000", 2024-05-08T08:52:14.190 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7297000000000003e-05 2024-05-08T08:52:14.190 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.190 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.190 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:14.190 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.880428+0000", 2024-05-08T08:52:14.191 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00080470100000000001 2024-05-08T08:52:14.191 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.191 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.191 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:52:14.191 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.882950+0000", 2024-05-08T08:52:14.191 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0025222500000000002 2024-05-08T08:52:14.191 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.191 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.191 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:14.191 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.882991+0000", 2024-05-08T08:52:14.191 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1609000000000001e-05 2024-05-08T08:52:14.191 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:14.191 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:14.191 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:14.191 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.883024+0000", 2024-05-08T08:52:14.192 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2904999999999998e-05 2024-05-08T08:52:14.192 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:14.192 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:14.192 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:14.192 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:14.192 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:14.192 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:52:14.192 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:52:14.194 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:14.197+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:14.294 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:14.298+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:14.298 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:52:14.298 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:52:14.298 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:52:14.298 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:52:14.307 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:52:14.353 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T08:52:14.395 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:52:14.399+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:52:14.453 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:52:14.453 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:52:14.453 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:52:14.453 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:52:14.453 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:52:14.462 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:52:14.483 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:52:14.483 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:52:14.501 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:52:14.501 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:52:14.503 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:14.506+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:14.517 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:52:14.518 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:52:14.536 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:52:14.536 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:52:14.554 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:52:14.573 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:52:14.574 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:52:14.597 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:14.601+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:14.610 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:14.610 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:14.619 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:52:14.702 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:14.706+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:14.764 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:14.764 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:14.773 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:52:14.799 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:52:14.803+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:52:14.901 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:14.904+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:14.919 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:14.919 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:14.928 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:52:15.002 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:52:15.005+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:52:15.073 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:15.074 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:15.083 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:52:15.103 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:52:15.106+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:52:15.199 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:15.199 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:15.204 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:15.206+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:15.208 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:52:15.304 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:15.308+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:15.317 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:15.317 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:15.327 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:52:15.405 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:15.408+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:15.436 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:15.436 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:15.445 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:52:15.506 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:52:15.509+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:52:15.554 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:15.554 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:15.564 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:52:15.607 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:15.610+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:15.707 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:15.711+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:15.714 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:15.714 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:15.723 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:52:15.766 INFO:teuthology.orchestra.run.smithi196.stdout:1 write 0xb1ccc00 thru 0xb1d2516 (0x5917 bytes) 2024-05-08T08:52:15.766 INFO:teuthology.orchestra.run.smithi196.stdout:2 write 0x3b8d200 thru 0x3b99853 (0xc654 bytes) 2024-05-08T08:52:15.787 INFO:teuthology.orchestra.run.smithi196.stdout:3 write 0xce01600 thru 0xce08244 (0x6c45 bytes) 2024-05-08T08:52:15.788 INFO:teuthology.orchestra.run.smithi196.stdout:4 punch from 0x724dc00 to 0x725be00, (0xe200 bytes) 2024-05-08T08:52:15.788 INFO:teuthology.orchestra.run.smithi196.stdout:6 read 0xbf4ea00 thru 0xbf5d2f6 (0xe8f7 bytes) 2024-05-08T08:52:15.788 INFO:teuthology.orchestra.run.smithi196.stdout:8 read 0x1a64400 thru 0x1a73b31 (0xf732 bytes) 2024-05-08T08:52:15.788 INFO:teuthology.orchestra.run.smithi196.stdout:10 write 0x7bcb400 thru 0x7bccf29 (0x1b2a bytes) 2024-05-08T08:52:15.788 INFO:teuthology.orchestra.run.smithi196.stdout:12 read 0xb2ef400 thru 0xb2fe164 (0xed65 bytes) 2024-05-08T08:52:15.789 INFO:teuthology.orchestra.run.smithi196.stdout:14 read 0x6f4c00 thru 0x6ff191 (0xa592 bytes) 2024-05-08T08:52:15.789 INFO:teuthology.orchestra.run.smithi196.stdout:truncating to largest ever: 0x167ea00 2024-05-08T08:52:15.802 INFO:teuthology.orchestra.run.smithi196.stdout:16 trunc from 0xce08400 to 0x167ea00 2024-05-08T08:52:15.808 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:52:15.811+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:52:15.824 INFO:teuthology.orchestra.run.smithi196.stdout:18 read 0x785200 thru 0x791823 (0xc624 bytes) 2024-05-08T08:52:15.834 INFO:teuthology.orchestra.run.smithi196.stdout:19 write 0x19a2a00 thru 0x19ab1ec (0x87ed bytes) 2024-05-08T08:52:15.835 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:15.835 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:15.844 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:52:15.853 INFO:teuthology.orchestra.run.smithi196.stdout:20 write 0x5b13200 thru 0x5b18933 (0x5734 bytes) 2024-05-08T08:52:15.870 INFO:teuthology.orchestra.run.smithi196.stdout:24 write 0x8345e00 thru 0x834e2a3 (0x84a4 bytes) 2024-05-08T08:52:15.871 INFO:teuthology.orchestra.run.smithi196.stdout:25 read 0xcc2800 thru 0xcd0777 (0xdf78 bytes) 2024-05-08T08:52:15.871 INFO:teuthology.orchestra.run.smithi196.stdout:26 punch from 0x482ae00 to 0x482fa00, (0x4c00 bytes) 2024-05-08T08:52:15.889 INFO:teuthology.orchestra.run.smithi196.stdout:32 write 0xde23800 thru 0xde23cc6 (0x4c7 bytes) 2024-05-08T08:52:15.890 INFO:teuthology.orchestra.run.smithi196.stdout:34 punch from 0xab2b600 to 0xab34400, (0x8e00 bytes) 2024-05-08T08:52:15.890 INFO:teuthology.orchestra.run.smithi196.stdout:35 read 0x8db4c00 thru 0x8db9d9c (0x519d bytes) 2024-05-08T08:52:15.890 INFO:teuthology.orchestra.run.smithi196.stdout:36 write 0xd068200 thru 0xd076730 (0xe531 bytes) 2024-05-08T08:52:15.890 INFO:teuthology.orchestra.run.smithi196.stdout:37 punch from 0xd3e400 to 0xd4de00, (0xfa00 bytes) 2024-05-08T08:52:15.891 INFO:teuthology.orchestra.run.smithi196.stdout:40 punch from 0x9750200 to 0x9758a00, (0x8800 bytes) 2024-05-08T08:52:15.891 INFO:teuthology.orchestra.run.smithi196.stdout:41 write 0x5781400 thru 0x57909cf (0xf5d0 bytes) 2024-05-08T08:52:15.891 INFO:teuthology.orchestra.run.smithi196.stdout:42 write 0x4eeb800 thru 0x4ef0fc2 (0x57c3 bytes) 2024-05-08T08:52:15.891 INFO:teuthology.orchestra.run.smithi196.stdout:43 write 0x9501c00 thru 0x95115fb (0xf9fc bytes) 2024-05-08T08:52:15.891 INFO:teuthology.orchestra.run.smithi196.stdout:truncating to largest ever: 0x3100a00 2024-05-08T08:52:15.904 INFO:teuthology.orchestra.run.smithi196.stdout:46 trunc from 0xde23e00 to 0x3100a00 2024-05-08T08:52:15.928 INFO:teuthology.orchestra.run.smithi196.stdout:skipping zero length punch hole 2024-05-08T08:52:15.928 INFO:teuthology.orchestra.run.smithi196.stdout:49 read 0x45c800 thru 0x4617f1 (0x4ff2 bytes) 2024-05-08T08:52:15.929 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:52:15.932+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:52:15.929 INFO:teuthology.orchestra.run.smithi196.stdout:50 read 0x265f800 thru 0x266a0f2 (0xa8f3 bytes) 2024-05-08T08:52:15.930 INFO:teuthology.orchestra.run.smithi196.stdout:54 read 0x779400 thru 0x7840a9 (0xacaa bytes) 2024-05-08T08:52:15.930 INFO:teuthology.orchestra.run.smithi196.stdout:55 read 0xf48600 thru 0xf4ccf4 (0x46f5 bytes) 2024-05-08T08:52:15.946 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-05-08T08:52:15.952 INFO:teuthology.orchestra.run.smithi196.stdout:57 write 0xeb58200 thru 0xeb659f2 (0xd7f3 bytes) 2024-05-08T08:52:15.953 INFO:teuthology.orchestra.run.smithi196.stdout:60 read 0x6fe1800 thru 0x6feff2f (0xe730 bytes) 2024-05-08T08:52:15.953 INFO:teuthology.orchestra.run.smithi196.stdout:63 write 0x6f4000 thru 0x703f27 (0xff28 bytes) 2024-05-08T08:52:15.968 INFO:teuthology.orchestra.run.smithi196.stdout:64 trunc from 0xeb65a00 to 0x2a60400 2024-05-08T08:52:15.989 INFO:teuthology.orchestra.run.smithi196.stdout:65 read 0xdd6400 thru 0xde3453 (0xd054 bytes) 2024-05-08T08:52:15.995 INFO:teuthology.orchestra.run.smithi196.stdout:69 read 0x1de0200 thru 0x1dee2de (0xe0df bytes) 2024-05-08T08:52:15.995 INFO:teuthology.orchestra.run.smithi196.stdout:70 write 0xbe7800 thru 0xbed384 (0x5b85 bytes) 2024-05-08T08:52:15.995 INFO:teuthology.orchestra.run.smithi196.stdout:truncating to largest ever: 0x6e09c00 2024-05-08T08:52:16.000 INFO:teuthology.orchestra.run.smithi196.stdout:71 trunc from 0x2a60400 to 0x6e09c00 2024-05-08T08:52:16.004 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:16.004 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:16.010 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:16.014+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:16.014 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:52:16.029 INFO:teuthology.orchestra.run.smithi196.stdout:74 trunc from 0x6e09c00 to 0x13cf200 2024-05-08T08:52:16.060 INFO:teuthology.orchestra.run.smithi196.stdout:76 write 0x5e04200 thru 0x5e04a08 (0x809 bytes) 2024-05-08T08:52:16.069 INFO:teuthology.orchestra.run.smithi196.stdout:79 trunc from 0x5e04c00 to 0x32c9200 2024-05-08T08:52:16.085 INFO:teuthology.orchestra.run.smithi196.stdout:80 read 0x5d3600 thru 0x5d40b1 (0xab2 bytes) 2024-05-08T08:52:16.092 INFO:teuthology.orchestra.run.smithi196.stdout:82 read 0xbcd200 thru 0xbcf9f9 (0x27fa bytes) 2024-05-08T08:52:16.092 INFO:teuthology.orchestra.run.smithi196.stdout:84 read 0x27cbc00 thru 0x27d544f (0x9850 bytes) 2024-05-08T08:52:16.112 INFO:teuthology.orchestra.run.smithi196.stdout:85 write 0xb575400 thru 0xb57871f (0x3320 bytes) 2024-05-08T08:52:16.112 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:52:16.115+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:52:16.119 INFO:teuthology.orchestra.run.smithi196.stdout:88 read 0x347a600 thru 0x3485da5 (0xb7a6 bytes) 2024-05-08T08:52:16.119 INFO:teuthology.orchestra.run.smithi196.stdout:89 write 0x40d3c00 thru 0x40e0ee0 (0xd2e1 bytes) 2024-05-08T08:52:16.119 INFO:teuthology.orchestra.run.smithi196.stdout:90 punch from 0x9a08200 to 0x9a13000, (0xae00 bytes) 2024-05-08T08:52:16.131 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:16.131 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:16.137 INFO:teuthology.orchestra.run.smithi196.stdout:91 write 0xc4b6200 thru 0xc4c4c17 (0xea18 bytes) 2024-05-08T08:52:16.141 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:52:16.144 INFO:teuthology.orchestra.run.smithi196.stdout:92 read 0x3c87c00 thru 0x3c9755e (0xf95f bytes) 2024-05-08T08:52:16.144 INFO:teuthology.orchestra.run.smithi196.stdout:95 read 0x5280a00 thru 0x52891a9 (0x87aa bytes) 2024-05-08T08:52:16.144 INFO:teuthology.orchestra.run.smithi196.stdout:96 punch from 0x3052200 to 0x305b400, (0x9200 bytes) 2024-05-08T08:52:16.144 INFO:teuthology.orchestra.run.smithi196.stdout:97 read 0x118b400 thru 0x118b81e (0x41f bytes) 2024-05-08T08:52:16.144 INFO:teuthology.orchestra.run.smithi196.stdout:98 punch from 0x2284e00 to 0x228d800, (0x8a00 bytes) 2024-05-08T08:52:16.144 INFO:teuthology.orchestra.run.smithi196.stdout:99 punch from 0x622800 to 0x62ea00, (0xc200 bytes) 2024-05-08T08:52:16.145 INFO:teuthology.orchestra.run.smithi196.stdout:102 read 0x7dd0400 thru 0x7dd5b3c (0x573d bytes) 2024-05-08T08:52:16.145 INFO:teuthology.orchestra.run.smithi196.stdout:truncating to largest ever: 0xe262a00 2024-05-08T08:52:16.146 INFO:teuthology.orchestra.run.smithi196.stdout:103 trunc from 0xc4c4e00 to 0xe262a00 2024-05-08T08:52:16.165 INFO:teuthology.orchestra.run.smithi196.stdout:106 trunc from 0xe262a00 to 0x6a81c00 2024-05-08T08:52:16.192 INFO:teuthology.orchestra.run.smithi196.stdout:108 trunc from 0x6a81c00 to 0xaaeca00 2024-05-08T08:52:16.207 INFO:teuthology.orchestra.run.smithi196.stdout:109 punch from 0x8dfd800 to 0x8e00000, (0x2800 bytes) 2024-05-08T08:52:16.212 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:16.216+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:16.219 INFO:teuthology.orchestra.run.smithi196.stdout:110 trunc from 0xaaeca00 to 0x91de00 2024-05-08T08:52:16.253 INFO:teuthology.orchestra.run.smithi196.stdout:111 trunc from 0x91de00 to 0x7892400 2024-05-08T08:52:16.289 INFO:teuthology.orchestra.run.smithi196.stdout:113 write 0xbfa6000 thru 0xbfaf273 (0x9274 bytes) 2024-05-08T08:52:16.289 INFO:teuthology.orchestra.run.smithi196.stdout:114 punch from 0x7fa7200 to 0x7fb1a00, (0xa800 bytes) 2024-05-08T08:52:16.289 INFO:teuthology.orchestra.run.smithi196.stdout:117 write 0x9dac00 thru 0x9e526c (0xa66d bytes) 2024-05-08T08:52:16.289 INFO:teuthology.orchestra.run.smithi196.stdout:118 read 0x35a3c00 thru 0x35b2b94 (0xef95 bytes) 2024-05-08T08:52:16.295 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:16.295 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:16.302 INFO:teuthology.orchestra.run.smithi196.stdout:119 trunc from 0xbfaf400 to 0x16d5600 2024-05-08T08:52:16.306 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:52:16.313 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:52:16.317+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:52:16.336 INFO:teuthology.orchestra.run.smithi196.stdout:120 write 0x6c5a600 thru 0x6c62166 (0x7b67 bytes) 2024-05-08T08:52:16.341 INFO:teuthology.orchestra.run.smithi196.stdout:121 read 0x2e25800 thru 0x2e2ead3 (0x92d4 bytes) 2024-05-08T08:52:16.341 INFO:teuthology.orchestra.run.smithi196.stdout:122 read 0x1867000 thru 0x1868be2 (0x1be3 bytes) 2024-05-08T08:52:16.341 INFO:teuthology.orchestra.run.smithi196.stdout:123 write 0xb6ca00 thru 0xb72065 (0x5666 bytes) 2024-05-08T08:52:16.342 INFO:teuthology.orchestra.run.smithi196.stdout:125 read 0x6049800 thru 0x604a5d9 (0xdda bytes) 2024-05-08T08:52:16.342 INFO:teuthology.orchestra.run.smithi196.stdout:127 punch from 0x3e17200 to 0x3e1f800, (0x8600 bytes) 2024-05-08T08:52:16.343 INFO:teuthology.orchestra.run.smithi196.stdout:128 read 0x27be800 thru 0x27c0d10 (0x2511 bytes) 2024-05-08T08:52:16.343 INFO:teuthology.orchestra.run.smithi196.stdout:130 read 0x57f1400 thru 0x57f47bc (0x33bd bytes) 2024-05-08T08:52:16.343 INFO:teuthology.orchestra.run.smithi196.stdout:135 write 0x747a00 thru 0x74ed39 (0x733a bytes) 2024-05-08T08:52:16.360 INFO:teuthology.orchestra.run.smithi196.stdout:136 write 0x9a9c000 thru 0x9a9f1c1 (0x31c2 bytes) 2024-05-08T08:52:16.365 INFO:teuthology.orchestra.run.smithi196.stdout:137 read 0x166e00 thru 0x173698 (0xc899 bytes) 2024-05-08T08:52:16.366 INFO:teuthology.orchestra.run.smithi196.stdout:138 punch from 0x5b4bc00 to 0x5b57800, (0xbc00 bytes) 2024-05-08T08:52:16.366 INFO:teuthology.orchestra.run.smithi196.stdout:139 write 0x224c800 thru 0x2257a14 (0xb215 bytes) 2024-05-08T08:52:16.368 INFO:teuthology.orchestra.run.smithi196.stdout:143 trunc from 0x9a9f200 to 0x7b33800 2024-05-08T08:52:16.408 INFO:teuthology.orchestra.run.smithi196.stdout:145 write 0xb4fc200 thru 0xb50bdfb (0xfbfc bytes) 2024-05-08T08:52:16.414 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:52:16.417+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:52:16.421 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:16.422 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:16.422 INFO:teuthology.orchestra.run.smithi196.stdout:147 trunc from 0xb50be00 to 0x4939200 2024-05-08T08:52:16.431 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:52:16.441 INFO:teuthology.orchestra.run.smithi196.stdout:153 write 0x17e2600 thru 0x17e4c91 (0x2692 bytes) 2024-05-08T08:52:16.458 INFO:teuthology.orchestra.run.smithi196.stdout:159 write 0x4a43c00 thru 0x4a4eab7 (0xaeb8 bytes) 2024-05-08T08:52:16.465 INFO:teuthology.orchestra.run.smithi196.stdout:160 punch from 0x1fc2400 to 0x1fd1a00, (0xf600 bytes) 2024-05-08T08:52:16.465 INFO:teuthology.orchestra.run.smithi196.stdout:161 read 0x4a07c00 thru 0x4a167ba (0xebbb bytes) 2024-05-08T08:52:16.465 INFO:teuthology.orchestra.run.smithi196.stdout:163 read 0xa97c00 thru 0xaa2d28 (0xb129 bytes) 2024-05-08T08:52:16.465 INFO:teuthology.orchestra.run.smithi196.stdout:164 read 0x3bde000 thru 0x3bebe89 (0xde8a bytes) 2024-05-08T08:52:16.465 INFO:teuthology.orchestra.run.smithi196.stdout:166 read 0xa95600 thru 0xa9eed2 (0x98d3 bytes) 2024-05-08T08:52:16.487 INFO:teuthology.orchestra.run.smithi196.stdout:167 write 0xe1ab200 thru 0xe1b9646 (0xe447 bytes) 2024-05-08T08:52:16.493 INFO:teuthology.orchestra.run.smithi196.stdout:169 punch from 0xaa63600 to 0xaa6e800, (0xb200 bytes) 2024-05-08T08:52:16.493 INFO:teuthology.orchestra.run.smithi196.stdout:170 punch from 0x50c4e00 to 0x50c7200, (0x2400 bytes) 2024-05-08T08:52:16.507 INFO:teuthology.orchestra.run.smithi196.stdout:171 trunc from 0xe1b9800 to 0x25dba00 2024-05-08T08:52:16.517 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:16.520+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:16.532 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-05-08T08:52:16.532 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T08:52:16.532 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-05-08T08:52:16.532 INFO:tasks.thrashosds.thrasher:Setting osd 4 primary_affinity to 0.989263 2024-05-08T08:52:16.533 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 0.9892633933831223 2024-05-08T08:52:16.534 INFO:teuthology.orchestra.run.smithi196.stdout:172 write 0x5cd7800 thru 0x5ce0f20 (0x9721 bytes) 2024-05-08T08:52:16.536 INFO:teuthology.orchestra.run.smithi196.stdout:173 read 0x4669800 thru 0x467039c (0x6b9d bytes) 2024-05-08T08:52:16.538 INFO:teuthology.orchestra.run.smithi196.stdout:174 trunc from 0x5ce1000 to 0x84bb000 2024-05-08T08:52:16.550 INFO:teuthology.orchestra.run.smithi196.stdout:175 read 0x636400 thru 0x63e0e0 (0x7ce1 bytes) 2024-05-08T08:52:16.565 INFO:teuthology.orchestra.run.smithi196.stdout:176 write 0xd276c00 thru 0xd27877c (0x1b7d bytes) 2024-05-08T08:52:16.572 INFO:teuthology.orchestra.run.smithi196.stdout:177 write 0x5981c00 thru 0x599122d (0xf62e bytes) 2024-05-08T08:52:16.573 INFO:teuthology.orchestra.run.smithi196.stdout:181 punch from 0x706de00 to 0x7079800, (0xba00 bytes) 2024-05-08T08:52:16.573 INFO:teuthology.orchestra.run.smithi196.stdout:182 read 0xa45e00 thru 0xa4c59e (0x679f bytes) 2024-05-08T08:52:16.585 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T08:52:16.589 INFO:teuthology.orchestra.run.smithi196.stdout:183 trunc from 0xd278800 to 0x1a6200 2024-05-08T08:52:16.592 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:16.593 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:16.604 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:52:16.624 INFO:teuthology.orchestra.run.smithi196.stdout:187 write 0x6a67000 thru 0x6a683fd (0x13fe bytes) 2024-05-08T08:52:16.629 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:52:16.633+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:52:16.652 INFO:teuthology.orchestra.run.smithi196.stdout:189 write 0xc8a0a00 thru 0xc8ae827 (0xde28 bytes) 2024-05-08T08:52:16.660 INFO:teuthology.orchestra.run.smithi196.stdout:190 read 0x1f63800 thru 0x1f6e874 (0xb075 bytes) 2024-05-08T08:52:16.660 INFO:teuthology.orchestra.run.smithi196.stdout:191 read 0xa5f8000 thru 0xa5fcbb7 (0x4bb8 bytes) 2024-05-08T08:52:16.682 INFO:teuthology.orchestra.run.smithi196.stdout:193 write 0xec71200 thru 0xec71226 (0x27 bytes) 2024-05-08T08:52:16.683 INFO:teuthology.orchestra.run.smithi196.stdout:194 write 0x7ebdc00 thru 0x7ecacfd (0xd0fe bytes) 2024-05-08T08:52:16.686 INFO:teuthology.orchestra.run.smithi196.stdout:195 trunc from 0xec71400 to 0xc505c00 2024-05-08T08:52:16.718 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:52:16.721+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:52:16.719 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:52:16.719 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:52:16.720 INFO:teuthology.orchestra.run.smithi196.stdout:197 write 0x90bac00 thru 0x90c7c37 (0xd038 bytes) 2024-05-08T08:52:16.721 INFO:teuthology.orchestra.run.smithi196.stdout:truncating to largest ever: 0xe316600 2024-05-08T08:52:16.723 INFO:teuthology.orchestra.run.smithi196.stdout:198 trunc from 0xc505c00 to 0xe316600 2024-05-08T08:52:16.741 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:52:16.742 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:52:16.747 INFO:teuthology.orchestra.run.smithi196.stdout:199 write 0x3475600 thru 0x3481019 (0xba1a bytes) 2024-05-08T08:52:16.747 INFO:teuthology.orchestra.run.smithi196.stdout:201 read 0x3584600 thru 0x35889bb (0x43bc bytes) 2024-05-08T08:52:16.747 INFO:teuthology.orchestra.run.smithi196.stdout:202 punch from 0x5c61200 to 0x5c63400, (0x2200 bytes) 2024-05-08T08:52:16.747 INFO:teuthology.orchestra.run.smithi196.stdout:203 write 0x6a7cc00 thru 0x6a8aeea (0xe2eb bytes) 2024-05-08T08:52:16.748 INFO:teuthology.orchestra.run.smithi196.stdout:207 write 0xc9d2200 thru 0xc9d3b5b (0x195c bytes) 2024-05-08T08:52:16.748 INFO:teuthology.orchestra.run.smithi196.stdout:209 punch from 0x1d6e600 to 0x1d7ca00, (0xe400 bytes) 2024-05-08T08:52:16.748 INFO:teuthology.orchestra.run.smithi196.stdout:210 punch from 0xdb30800 to 0xdb31000, (0x800 bytes) 2024-05-08T08:52:16.748 INFO:teuthology.orchestra.run.smithi196.stdout:213 read 0x5aada00 thru 0x5ab6926 (0x8f27 bytes) 2024-05-08T08:52:16.748 INFO:teuthology.orchestra.run.smithi196.stdout:217 write 0x2140c00 thru 0x214b879 (0xac7a bytes) 2024-05-08T08:52:16.748 INFO:teuthology.orchestra.run.smithi196.stdout:220 punch from 0x9305e00 to 0x9314000, (0xe200 bytes) 2024-05-08T08:52:16.749 INFO:teuthology.orchestra.run.smithi196.stdout:221 write 0x2fea00 thru 0x3096d1 (0xacd2 bytes) 2024-05-08T08:52:16.749 INFO:teuthology.orchestra.run.smithi196.stdout:222 write 0x4193a00 thru 0x4194864 (0xe65 bytes) 2024-05-08T08:52:16.750 INFO:teuthology.orchestra.run.smithi196.stdout:224 trunc from 0xe316600 to 0xd403a00 2024-05-08T08:52:16.760 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:16.760 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:16.764 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:52:16.764 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:52:16.771 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:52:16.779 INFO:teuthology.orchestra.run.smithi196.stdout:226 read 0x6682600 thru 0x66890ee (0x6aef bytes) 2024-05-08T08:52:16.779 INFO:teuthology.orchestra.run.smithi196.stdout:228 read 0x918800 thru 0x92321c (0xaa1d bytes) 2024-05-08T08:52:16.780 INFO:teuthology.orchestra.run.smithi196.stdout:229 read 0x3ed5a00 thru 0x3edd1ac (0x77ad bytes) 2024-05-08T08:52:16.783 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:52:16.783 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:52:16.783 INFO:teuthology.orchestra.run.smithi196.stdout:231 trunc from 0xd403a00 to 0xa941400 2024-05-08T08:52:16.797 INFO:teuthology.orchestra.run.smithi196.stdout:233 read 0x8616a00 thru 0x861fa78 (0x9079 bytes) 2024-05-08T08:52:16.798 INFO:teuthology.orchestra.run.smithi196.stdout:234 read 0x8709800 thru 0x8711716 (0x7f17 bytes) 2024-05-08T08:52:16.798 INFO:teuthology.orchestra.run.smithi196.stdout:236 write 0xc42400 thru 0xc4d3be (0xafbf bytes) 2024-05-08T08:52:16.798 INFO:teuthology.orchestra.run.smithi196.stdout:237 write 0x62ae000 thru 0x62b3613 (0x5614 bytes) 2024-05-08T08:52:16.798 INFO:teuthology.orchestra.run.smithi196.stdout:238 punch from 0x4157000 to 0x4161e00, (0xae00 bytes) 2024-05-08T08:52:16.799 INFO:teuthology.orchestra.run.smithi196.stdout:240 write 0x43ac800 thru 0x43b9feb (0xd7ec bytes) 2024-05-08T08:52:16.801 INFO:teuthology.orchestra.run.smithi196.stdout:241 punch from 0xa80600 to 0xa83000, (0x2a00 bytes) 2024-05-08T08:52:16.801 INFO:teuthology.orchestra.run.smithi196.stdout:244 write 0x2f2e400 thru 0x2f3427b (0x5e7c bytes) 2024-05-08T08:52:16.801 INFO:teuthology.orchestra.run.smithi196.stdout:247 punch from 0x259000 to 0x25d200, (0x4200 bytes) 2024-05-08T08:52:16.801 INFO:teuthology.orchestra.run.smithi196.stdout:248 write 0x4577c00 thru 0x457db67 (0x5f68 bytes) 2024-05-08T08:52:16.801 INFO:teuthology.orchestra.run.smithi196.stdout:249 read 0x9362c00 thru 0x9372608 (0xfa09 bytes) 2024-05-08T08:52:16.803 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:52:16.804 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:52:16.819 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:52:16.822+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:52:16.822 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:52:16.823 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:52:16.826 INFO:teuthology.orchestra.run.smithi196.stdout:250 write 0xeb36000 thru 0xeb43a20 (0xda21 bytes) 2024-05-08T08:52:16.826 INFO:teuthology.orchestra.run.smithi196.stdout:252 read 0x90f1800 thru 0x90f41c2 (0x29c3 bytes) 2024-05-08T08:52:16.832 INFO:teuthology.orchestra.run.smithi196.stdout:254 trunc from 0xeb43c00 to 0x948a800 2024-05-08T08:52:16.851 INFO:teuthology.orchestra.run.smithi196.stdout:255 read 0x8b24800 thru 0x8b2fb62 (0xb363 bytes) 2024-05-08T08:52:16.868 INFO:teuthology.orchestra.run.smithi196.stdout:256 write 0xe6a0400 thru 0xe6addcf (0xd9d0 bytes) 2024-05-08T08:52:16.868 INFO:teuthology.orchestra.run.smithi196.stdout:260 write 0x1d97c00 thru 0x1d9a8b8 (0x2cb9 bytes) 2024-05-08T08:52:16.869 INFO:teuthology.orchestra.run.smithi196.stdout:261 write 0x4862000 thru 0x4864c5c (0x2c5d bytes) 2024-05-08T08:52:16.869 INFO:teuthology.orchestra.run.smithi196.stdout:263 write 0x8a43c00 thru 0x8a4cac8 (0x8ec9 bytes) 2024-05-08T08:52:16.869 INFO:teuthology.orchestra.run.smithi196.stdout:266 punch from 0xb12f800 to 0xb13be00, (0xc600 bytes) 2024-05-08T08:52:16.869 INFO:teuthology.orchestra.run.smithi196.stdout:271 read 0xd583a00 thru 0xd591a66 (0xe067 bytes) 2024-05-08T08:52:16.870 INFO:teuthology.orchestra.run.smithi196.stdout:274 write 0x9eb600 thru 0x9f689d (0xb29e bytes) 2024-05-08T08:52:16.870 INFO:teuthology.orchestra.run.smithi196.stdout:275 punch from 0x381bc00 to 0x381c200, (0x600 bytes) 2024-05-08T08:52:16.871 INFO:teuthology.orchestra.run.smithi196.stdout:276 punch from 0xc39f400 to 0xc3a6c00, (0x7800 bytes) 2024-05-08T08:52:16.871 INFO:teuthology.orchestra.run.smithi196.stdout:278 read 0xc01ba00 thru 0xc027d7f (0xc380 bytes) 2024-05-08T08:52:16.876 INFO:teuthology.orchestra.run.smithi196.stdout:280 trunc from 0xe6ade00 to 0x96e4600 2024-05-08T08:52:16.897 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:52:16.897 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:52:16.897 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:52:16.897 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:52:16.901 INFO:teuthology.orchestra.run.smithi196.stdout:281 read 0x16bc00 thru 0x173531 (0x7932 bytes) 2024-05-08T08:52:16.902 INFO:teuthology.orchestra.run.smithi196.stdout:282 write 0x963d200 thru 0x9644d5c (0x7b5d bytes) 2024-05-08T08:52:16.902 INFO:teuthology.orchestra.run.smithi196.stdout:284 write 0x2af0e00 thru 0x2af121c (0x41d bytes) 2024-05-08T08:52:16.902 INFO:teuthology.orchestra.run.smithi196.stdout:285 read 0x1f4e800 thru 0x1f5d3b4 (0xebb5 bytes) 2024-05-08T08:52:16.903 INFO:teuthology.orchestra.run.smithi196.stdout:288 write 0x1b77200 thru 0x1b7b885 (0x4686 bytes) 2024-05-08T08:52:16.903 INFO:teuthology.orchestra.run.smithi196.stdout:289 punch from 0x6d3aa00 to 0x6d43400, (0x8a00 bytes) 2024-05-08T08:52:16.903 INFO:teuthology.orchestra.run.smithi196.stdout:290 read 0x2d06200 thru 0x2d12cb8 (0xcab9 bytes) 2024-05-08T08:52:16.903 INFO:teuthology.orchestra.run.smithi196.stdout:293 write 0x38b3800 thru 0x38b5475 (0x1c76 bytes) 2024-05-08T08:52:16.903 INFO:teuthology.orchestra.run.smithi196.stdout:294 read 0x62c9600 thru 0x62d05ae (0x6faf bytes) 2024-05-08T08:52:16.904 INFO:teuthology.orchestra.run.smithi196.stdout:295 read 0x8c33e00 thru 0x8c3a179 (0x637a bytes) 2024-05-08T08:52:16.904 INFO:teuthology.orchestra.run.smithi196.stdout:296 punch from 0x201b400 to 0x2029000, (0xdc00 bytes) 2024-05-08T08:52:16.904 INFO:teuthology.orchestra.run.smithi196.stdout:301 write 0x7259600 thru 0x7265be3 (0xc5e4 bytes) 2024-05-08T08:52:16.908 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:52:16.910 INFO:teuthology.orchestra.run.smithi196.stdout:302 trunc from 0x96e4600 to 0x508b600 2024-05-08T08:52:16.932 INFO:teuthology.orchestra.run.smithi196.stdout:307 write 0x2b8a800 thru 0x2b96f27 (0xc728 bytes) 2024-05-08T08:52:16.933 INFO:teuthology.orchestra.run.smithi196.stdout:308 write 0x37f5400 thru 0x3803cdd (0xe8de bytes) 2024-05-08T08:52:16.935 INFO:teuthology.orchestra.run.smithi059.stderr:set osd.4 primary-affinity to 0.989263 (8648322) 2024-05-08T08:52:16.942 INFO:teuthology.orchestra.run.smithi059.stderr:noscrub is unset 2024-05-08T08:52:16.951 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:16.955+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:16.953 INFO:teuthology.orchestra.run.smithi196.stdout:310 write 0x90bf400 thru 0x90c48d8 (0x54d9 bytes) 2024-05-08T08:52:16.958 INFO:teuthology.orchestra.run.smithi196.stdout:311 trunc from 0x90c4a00 to 0x605b200 2024-05-08T08:52:16.974 INFO:teuthology.orchestra.run.smithi196.stdout:316 write 0x3ce4a00 thru 0x3cee8dc (0x9edd bytes) 2024-05-08T08:52:16.980 INFO:teuthology.orchestra.run.smithi196.stdout:317 trunc from 0x605b200 to 0xab3b400 2024-05-08T08:52:16.993 INFO:teuthology.orchestra.run.smithi196.stdout:319 read 0x61f7e00 thru 0x61ff180 (0x7381 bytes) 2024-05-08T08:52:16.994 INFO:teuthology.orchestra.run.smithi196.stdout:320 punch from 0x627de00 to 0x6283800, (0x5a00 bytes) 2024-05-08T08:52:17.008 INFO:teuthology.orchestra.run.smithi196.stdout:321 write 0xbed3c00 thru 0xbee2802 (0xec03 bytes) 2024-05-08T08:52:17.016 INFO:teuthology.orchestra.run.smithi196.stdout:324 write 0x279f000 thru 0x27a93cb (0xa3cc bytes) 2024-05-08T08:52:17.021 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:52:17.024+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:52:17.024 INFO:teuthology.orchestra.run.smithi196.stdout:325 trunc from 0xbee2a00 to 0x4855a00 2024-05-08T08:52:17.029 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:52:17.029 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:52:17.029 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:52:17.029 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:52:17.029 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:52:17.039 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:52:17.052 INFO:teuthology.orchestra.run.smithi196.stdout:326 read 0x2f63200 thru 0x2f65cfe (0x2aff bytes) 2024-05-08T08:52:17.060 INFO:teuthology.orchestra.run.smithi196.stdout:330 read 0x468f600 thru 0x46983f9 (0x8dfa bytes) 2024-05-08T08:52:17.064 INFO:teuthology.orchestra.run.smithi196.stdout:333 trunc from 0x4855a00 to 0xc87e00 2024-05-08T08:52:17.087 INFO:teuthology.orchestra.run.smithi196.stdout:336 read 0x2db800 thru 0x2dcb6b (0x136c bytes) 2024-05-08T08:52:17.093 INFO:teuthology.orchestra.run.smithi196.stdout:337 read 0x423c00 thru 0x424f20 (0x1321 bytes) 2024-05-08T08:52:17.093 INFO:teuthology.orchestra.run.smithi196.stdout:338 read 0x3c4400 thru 0x3c6243 (0x1e44 bytes) 2024-05-08T08:52:17.108 INFO:teuthology.orchestra.run.smithi196.stdout:340 write 0x42a8600 thru 0x42a9bf9 (0x15fa bytes) 2024-05-08T08:52:17.114 INFO:teuthology.orchestra.run.smithi196.stdout:341 read 0x174d800 thru 0x175cb33 (0xf334 bytes) 2024-05-08T08:52:17.117 INFO:teuthology.orchestra.run.smithi196.stdout:342 trunc from 0x42a9c00 to 0x1b6a800 2024-05-08T08:52:17.121 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:52:17.124+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:52:17.136 INFO:teuthology.orchestra.run.smithi196.stdout:343 punch from 0x91a600 to 0x927200, (0xcc00 bytes) 2024-05-08T08:52:17.158 INFO:teuthology.orchestra.run.smithi196.stdout:344 write 0x3e72c00 thru 0x3e7ac1b (0x801c bytes) 2024-05-08T08:52:17.164 INFO:teuthology.orchestra.run.smithi196.stdout:345 read 0x2c63e00 thru 0x2c6aa40 (0x6c41 bytes) 2024-05-08T08:52:17.172 INFO:teuthology.orchestra.run.smithi196.stdout:346 trunc from 0x3e7ae00 to 0xb3fc400 2024-05-08T08:52:17.189 INFO:teuthology.orchestra.run.smithi196.stdout:347 write 0x7467600 thru 0x746a854 (0x3255 bytes) 2024-05-08T08:52:17.195 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:52:17.195 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:52:17.195 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:52:17.196 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:52:17.196 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.196 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4416.0:7 3.1a 3:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-05-08T08:52:17.196 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:12.925684+0000", 2024-05-08T08:52:17.196 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.2722389960000005, 2024-05-08T08:52:17.196 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001890126, 2024-05-08T08:52:17.196 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:17.196 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:17.196 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:17.196 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:17.196 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4416", 2024-05-08T08:52:17.196 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:52:17.196 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7 2024-05-08T08:52:17.197 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.197 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:17.197 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.197 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:17.197 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:12.925684+0000", 2024-05-08T08:52:17.197 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:17.197 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.197 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.197 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:17.197 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:12.925684+0000", 2024-05-08T08:52:17.197 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:17.197 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.197 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.197 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:17.197 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:12.925686+0000", 2024-05-08T08:52:17.198 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.485e-06 2024-05-08T08:52:17.198 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.198 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.198 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:17.198 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:12.925708+0000", 2024-05-08T08:52:17.198 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1906e-05 2024-05-08T08:52:17.198 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.198 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.198 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:17.198 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:12.925722+0000", 2024-05-08T08:52:17.198 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4708e-05 2024-05-08T08:52:17.198 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.198 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.198 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:17.199 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:12.925734+0000", 2024-05-08T08:52:17.199 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1826e-05 2024-05-08T08:52:17.199 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.199 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.199 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:17.199 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:12.926024+0000", 2024-05-08T08:52:17.199 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00028986399999999997 2024-05-08T08:52:17.199 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.199 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.199 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:17.199 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:12.926105+0000", 2024-05-08T08:52:17.199 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0875000000000001e-05 2024-05-08T08:52:17.199 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.200 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.200 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-05-08T08:52:17.200 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:12.926210+0000", 2024-05-08T08:52:17.200 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104489 2024-05-08T08:52:17.200 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.200 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.200 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:52:17.200 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:12.927352+0000", 2024-05-08T08:52:17.200 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011428160000000001 2024-05-08T08:52:17.200 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.200 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.200 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:17.200 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:12.927514+0000", 2024-05-08T08:52:17.200 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00016137200000000001 2024-05-08T08:52:17.201 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.201 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.201 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:17.201 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:12.927546+0000", 2024-05-08T08:52:17.201 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2404999999999999e-05 2024-05-08T08:52:17.201 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.201 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.201 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:17.201 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:12.927575+0000", 2024-05-08T08:52:17.201 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8379999999999999e-05 2024-05-08T08:52:17.201 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.201 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:17.201 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.201 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.202 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.202 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4416.0:9 3.1a 3:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-05-08T08:52:17.202 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:13.935120+0000", 2024-05-08T08:52:17.202 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.2628038679999998, 2024-05-08T08:52:17.202 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.008168804, 2024-05-08T08:52:17.202 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:17.202 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:17.202 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:17.202 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:17.202 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4416", 2024-05-08T08:52:17.202 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:52:17.202 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 9 2024-05-08T08:52:17.203 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.203 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:17.203 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.203 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:17.203 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:13.935120+0000", 2024-05-08T08:52:17.203 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:17.203 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.203 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.203 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:17.203 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:13.935120+0000", 2024-05-08T08:52:17.203 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:17.203 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.203 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.203 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:17.203 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:13.935121+0000", 2024-05-08T08:52:17.204 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.015e-06 2024-05-08T08:52:17.204 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.204 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.204 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:17.204 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:13.935125+0000", 2024-05-08T08:52:17.204 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9559999999999996e-06 2024-05-08T08:52:17.204 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.204 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.204 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:17.204 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:13.935135+0000", 2024-05-08T08:52:17.204 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5089999999999999e-06 2024-05-08T08:52:17.204 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.204 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.204 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:17.205 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:13.935143+0000", 2024-05-08T08:52:17.205 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4800000000000001e-06 2024-05-08T08:52:17.205 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.205 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.205 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:17.205 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:13.935184+0000", 2024-05-08T08:52:17.205 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0308000000000001e-05 2024-05-08T08:52:17.205 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.205 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.205 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op must wait for map", 2024-05-08T08:52:17.205 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:13.935188+0000", 2024-05-08T08:52:17.205 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3499999999999999e-06 2024-05-08T08:52:17.205 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.205 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.205 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:17.206 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:13.938333+0000", 2024-05-08T08:52:17.206 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031447799999999998 2024-05-08T08:52:17.206 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.206 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.206 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:17.206 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:13.938396+0000", 2024-05-08T08:52:17.206 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2738999999999997e-05 2024-05-08T08:52:17.206 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.206 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.206 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-05-08T08:52:17.206 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:13.938480+0000", 2024-05-08T08:52:17.206 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4808999999999994e-05 2024-05-08T08:52:17.206 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.207 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.207 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:17.207 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:13.939565+0000", 2024-05-08T08:52:17.207 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0010842740000000001 2024-05-08T08:52:17.207 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.207 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.207 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:52:17.207 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:13.943198+0000", 2024-05-08T08:52:17.207 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0036330500000000001 2024-05-08T08:52:17.207 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.207 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.207 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:17.207 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:13.943247+0000", 2024-05-08T08:52:17.207 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9224999999999997e-05 2024-05-08T08:52:17.207 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.208 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.208 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:17.208 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:13.943288+0000", 2024-05-08T08:52:17.208 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1309e-05 2024-05-08T08:52:17.208 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.208 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:17.208 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.208 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.208 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.208 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4416.0:10 3.1c e21/18 3:3831950c:::rbd_directory:head v 21'1, mlcod=0'0)", 2024-05-08T08:52:17.208 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:13.944986+0000", 2024-05-08T08:52:17.208 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.2529369410000002, 2024-05-08T08:52:17.208 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003849182, 2024-05-08T08:52:17.209 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:17.209 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:17.209 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:17.209 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:17.209 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.209 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:17.209 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:13.944986+0000", 2024-05-08T08:52:17.209 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:17.209 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.209 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.209 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:17.209 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:13.944986+0000", 2024-05-08T08:52:17.209 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:17.209 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.210 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.210 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:17.210 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:13.944987+0000", 2024-05-08T08:52:17.210 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0299999999999998e-07 2024-05-08T08:52:17.210 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.210 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.210 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:17.210 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:13.944994+0000", 2024-05-08T08:52:17.210 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5679999999999996e-06 2024-05-08T08:52:17.210 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.210 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.210 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:17.210 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:13.945004+0000", 2024-05-08T08:52:17.211 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.026e-05 2024-05-08T08:52:17.211 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.211 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.211 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:17.211 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:13.945011+0000", 2024-05-08T08:52:17.211 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7129999999999999e-06 2024-05-08T08:52:17.211 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.211 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.211 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:17.211 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:13.945173+0000", 2024-05-08T08:52:17.211 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00016231700000000001 2024-05-08T08:52:17.211 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.211 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.211 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:17.211 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:13.945198+0000", 2024-05-08T08:52:17.212 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4766e-05 2024-05-08T08:52:17.212 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.212 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.212 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:17.212 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:13.948806+0000", 2024-05-08T08:52:17.212 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0036079049999999998 2024-05-08T08:52:17.212 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.212 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.212 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:17.212 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:13.948836+0000", 2024-05-08T08:52:17.212 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9850000000000001e-05 2024-05-08T08:52:17.212 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.212 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:17.212 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.212 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.213 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.213 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4465.0:1423 3.1c e21/18 3:3e22f3f9:::rbd_data.1140cf79ec85.000000000000000b:head v 21'6, mlcod=21'4)", 2024-05-08T08:52:17.213 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:16.811116+0000", 2024-05-08T08:52:17.213 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.38680750000000003, 2024-05-08T08:52:17.213 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0018725860000000001, 2024-05-08T08:52:17.213 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:17.213 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:17.213 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:17.213 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:17.213 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.213 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:17.213 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.811116+0000", 2024-05-08T08:52:17.213 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:17.213 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.213 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.214 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:17.214 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.811116+0000", 2024-05-08T08:52:17.214 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:17.214 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.214 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.214 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:17.214 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.811117+0000", 2024-05-08T08:52:17.214 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.02e-06 2024-05-08T08:52:17.214 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.214 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.214 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:17.214 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.811138+0000", 2024-05-08T08:52:17.214 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1195999999999998e-05 2024-05-08T08:52:17.215 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.215 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.215 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:17.215 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.811151+0000", 2024-05-08T08:52:17.215 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2707999999999999e-05 2024-05-08T08:52:17.215 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.215 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.215 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:17.215 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.811158+0000", 2024-05-08T08:52:17.215 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9979999999999997e-06 2024-05-08T08:52:17.215 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.215 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.215 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:17.215 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.811207+0000", 2024-05-08T08:52:17.215 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9215999999999998e-05 2024-05-08T08:52:17.215 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.216 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.216 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:17.216 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.811221+0000", 2024-05-08T08:52:17.216 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3906e-05 2024-05-08T08:52:17.216 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.216 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.216 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:17.216 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.812958+0000", 2024-05-08T08:52:17.216 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001737328 2024-05-08T08:52:17.216 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.216 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.216 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:17.216 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.812988+0000", 2024-05-08T08:52:17.216 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0213999999999999e-05 2024-05-08T08:52:17.216 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.217 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:17.217 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.217 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.217 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.217 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4465.0:1560 3.e e21/18 3:72ee2809:::rbd_data.1140cf79ec85.0000000000000012:head v 21'7, mlcod=21'5)", 2024-05-08T08:52:17.217 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:16.878811+0000", 2024-05-08T08:52:17.217 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.31911288199999999, 2024-05-08T08:52:17.217 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001551736, 2024-05-08T08:52:17.217 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:17.217 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:17.217 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:17.217 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:17.217 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.217 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:17.217 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.878811+0000", 2024-05-08T08:52:17.217 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:17.218 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.218 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.218 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:17.218 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.878811+0000", 2024-05-08T08:52:17.218 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:17.218 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.218 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.218 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:17.218 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.878811+0000", 2024-05-08T08:52:17.218 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2799999999999995e-07 2024-05-08T08:52:17.218 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.218 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.218 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:17.218 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.878824+0000", 2024-05-08T08:52:17.219 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3091000000000001e-05 2024-05-08T08:52:17.219 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.219 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.219 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:17.219 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.878837+0000", 2024-05-08T08:52:17.219 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.276e-05 2024-05-08T08:52:17.219 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.219 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.219 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:17.219 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.878844+0000", 2024-05-08T08:52:17.219 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1380000000000001e-06 2024-05-08T08:52:17.219 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.219 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.219 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:17.219 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.878941+0000", 2024-05-08T08:52:17.219 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6490000000000006e-05 2024-05-08T08:52:17.219 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.220 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.220 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:17.220 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.878962+0000", 2024-05-08T08:52:17.220 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0971e-05 2024-05-08T08:52:17.220 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.220 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.220 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:17.220 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.880318+0000", 2024-05-08T08:52:17.220 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001356534 2024-05-08T08:52:17.220 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.220 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.220 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:17.220 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.880362+0000", 2024-05-08T08:52:17.220 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4023999999999997e-05 2024-05-08T08:52:17.221 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.221 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:17.221 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.221 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.221 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.221 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4465.0:1563 3.1c e21/18 3:3d4a1ed3:::rbd_data.1140cf79ec85.000000000000002c:head v 21'7, mlcod=21'5)", 2024-05-08T08:52:17.221 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:16.881256+0000", 2024-05-08T08:52:17.221 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.31666707500000002, 2024-05-08T08:52:17.221 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00164033, 2024-05-08T08:52:17.221 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:17.221 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:17.221 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:17.221 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:17.221 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.221 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:17.221 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.881256+0000", 2024-05-08T08:52:17.222 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:17.222 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.222 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.222 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:17.222 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.881256+0000", 2024-05-08T08:52:17.222 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:17.222 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.222 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.222 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:17.222 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.881257+0000", 2024-05-08T08:52:17.222 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3799999999999996e-07 2024-05-08T08:52:17.222 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.222 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.222 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:17.223 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.881279+0000", 2024-05-08T08:52:17.223 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1579000000000001e-05 2024-05-08T08:52:17.223 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.223 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.223 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:17.223 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.881291+0000", 2024-05-08T08:52:17.223 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2249000000000001e-05 2024-05-08T08:52:17.223 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.223 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.223 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:17.223 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.881298+0000", 2024-05-08T08:52:17.223 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0639999999999996e-06 2024-05-08T08:52:17.223 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.224 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.224 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:17.224 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.881380+0000", 2024-05-08T08:52:17.224 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2150000000000005e-05 2024-05-08T08:52:17.224 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.224 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.224 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:17.224 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.881399+0000", 2024-05-08T08:52:17.224 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9182999999999999e-05 2024-05-08T08:52:17.224 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.224 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.224 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:17.224 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.882847+0000", 2024-05-08T08:52:17.224 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001447257 2024-05-08T08:52:17.225 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.225 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.225 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:17.225 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.882897+0000", 2024-05-08T08:52:17.225 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.011e-05 2024-05-08T08:52:17.225 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.225 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:17.225 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.225 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.225 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.225 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4465.0:1565 3.11 e21/18 3:8f98052b:::rbd_data.1140cf79ec85.0000000000000030:head v 21'7, mlcod=21'5)", 2024-05-08T08:52:17.225 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:16.883835+0000", 2024-05-08T08:52:17.225 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.31408834400000002, 2024-05-08T08:52:17.225 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00069063600000000001, 2024-05-08T08:52:17.225 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:17.226 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:17.226 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:17.226 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:17.226 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.226 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:17.226 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.883835+0000", 2024-05-08T08:52:17.226 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:17.226 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.226 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.226 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:17.226 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.883835+0000", 2024-05-08T08:52:17.226 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:17.226 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.226 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.227 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:17.227 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.883836+0000", 2024-05-08T08:52:17.227 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0500000000000002e-07 2024-05-08T08:52:17.227 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.227 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.227 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:17.227 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.883862+0000", 2024-05-08T08:52:17.227 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6227000000000001e-05 2024-05-08T08:52:17.227 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.227 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.227 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:17.227 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.883873+0000", 2024-05-08T08:52:17.227 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.131e-05 2024-05-08T08:52:17.227 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.228 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.228 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:17.228 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.883881+0000", 2024-05-08T08:52:17.228 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8480000000000001e-06 2024-05-08T08:52:17.228 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.228 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.228 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:17.228 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.883936+0000", 2024-05-08T08:52:17.228 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5206e-05 2024-05-08T08:52:17.228 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.228 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.228 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:17.228 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.883955+0000", 2024-05-08T08:52:17.228 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8284999999999999e-05 2024-05-08T08:52:17.229 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.229 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.229 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:17.229 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.884494+0000", 2024-05-08T08:52:17.229 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00053929499999999997 2024-05-08T08:52:17.229 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.229 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.229 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:17.229 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.884526+0000", 2024-05-08T08:52:17.229 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1659999999999998e-05 2024-05-08T08:52:17.229 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.229 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:17.229 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.229 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.229 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.230 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4465.0:1583 3.1c e21/18 3:3d4a1ed3:::rbd_data.1140cf79ec85.000000000000002c:head v 21'8, mlcod=21'6)", 2024-05-08T08:52:17.230 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:16.891038+0000", 2024-05-08T08:52:17.230 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.306885256, 2024-05-08T08:52:17.230 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00070491800000000002, 2024-05-08T08:52:17.230 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:17.230 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:17.230 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:17.230 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:17.230 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.230 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:17.230 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.891038+0000", 2024-05-08T08:52:17.230 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:17.230 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.230 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.231 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:17.231 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.891038+0000", 2024-05-08T08:52:17.231 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:17.231 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.231 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.231 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:17.231 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.891039+0000", 2024-05-08T08:52:17.231 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1099999999999995e-07 2024-05-08T08:52:17.231 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.231 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.231 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:17.231 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.891046+0000", 2024-05-08T08:52:17.231 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6649999999999997e-06 2024-05-08T08:52:17.231 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.231 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.232 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:17.232 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.891060+0000", 2024-05-08T08:52:17.232 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4129e-05 2024-05-08T08:52:17.232 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.232 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.232 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:17.232 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.891067+0000", 2024-05-08T08:52:17.232 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5190000000000003e-06 2024-05-08T08:52:17.232 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.232 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.232 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:17.232 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.891120+0000", 2024-05-08T08:52:17.232 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2556000000000003e-05 2024-05-08T08:52:17.232 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.232 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.233 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:17.233 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.891140+0000", 2024-05-08T08:52:17.233 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0534999999999999e-05 2024-05-08T08:52:17.233 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.233 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.233 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:17.233 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.891704+0000", 2024-05-08T08:52:17.233 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00056399399999999995 2024-05-08T08:52:17.233 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.233 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.233 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:17.233 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.891743+0000", 2024-05-08T08:52:17.233 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8809000000000001e-05 2024-05-08T08:52:17.233 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.234 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:17.234 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.234 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.234 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.234 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4465.0:1584 3.11 e21/18 3:8f98052b:::rbd_data.1140cf79ec85.0000000000000030:head v 21'8, mlcod=21'6)", 2024-05-08T08:52:17.234 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:16.891392+0000", 2024-05-08T08:52:17.234 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.30653101799999999, 2024-05-08T08:52:17.234 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001063542, 2024-05-08T08:52:17.234 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:17.234 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:17.234 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:17.234 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:17.234 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.234 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:17.235 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.891392+0000", 2024-05-08T08:52:17.235 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:17.235 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.235 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.235 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:17.235 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.891392+0000", 2024-05-08T08:52:17.235 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:17.235 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.235 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.235 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:17.235 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.891393+0000", 2024-05-08T08:52:17.235 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2900000000000003e-07 2024-05-08T08:52:17.236 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.236 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.236 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:17.236 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.891399+0000", 2024-05-08T08:52:17.236 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1380000000000004e-06 2024-05-08T08:52:17.236 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.236 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.236 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:17.236 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.891411+0000", 2024-05-08T08:52:17.236 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1369000000000001e-05 2024-05-08T08:52:17.236 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.236 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.237 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:17.237 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.891418+0000", 2024-05-08T08:52:17.237 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0060000000000003e-06 2024-05-08T08:52:17.237 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.237 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.237 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:17.237 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.891471+0000", 2024-05-08T08:52:17.237 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3446000000000003e-05 2024-05-08T08:52:17.237 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.237 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.237 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:17.237 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.891490+0000", 2024-05-08T08:52:17.237 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8569000000000001e-05 2024-05-08T08:52:17.237 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.238 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.238 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:17.238 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.892414+0000", 2024-05-08T08:52:17.238 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00092496400000000002 2024-05-08T08:52:17.238 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.238 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.238 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:17.238 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.892456+0000", 2024-05-08T08:52:17.238 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1421000000000002e-05 2024-05-08T08:52:17.238 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.238 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:17.238 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.238 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.238 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.239 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4465.0:1641 3.11 e21/18 3:895c0c70:::rbd_data.1140cf79ec85.0000000000000006:head v 21'9, mlcod=21'7)", 2024-05-08T08:52:17.239 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:16.911661+0000", 2024-05-08T08:52:17.239 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.28626278100000002, 2024-05-08T08:52:17.239 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016336409999999999, 2024-05-08T08:52:17.239 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:17.239 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:17.239 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:17.239 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:17.239 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.239 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:17.239 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.911661+0000", 2024-05-08T08:52:17.239 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:17.239 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.240 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.240 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:17.240 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.911661+0000", 2024-05-08T08:52:17.240 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:17.240 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.240 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.240 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:17.240 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.911662+0000", 2024-05-08T08:52:17.240 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.175e-06 2024-05-08T08:52:17.240 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.240 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.240 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:17.240 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.911690+0000", 2024-05-08T08:52:17.240 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7787e-05 2024-05-08T08:52:17.241 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.241 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.241 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:17.241 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.911707+0000", 2024-05-08T08:52:17.241 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7343e-05 2024-05-08T08:52:17.241 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.241 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.241 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:17.241 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.911716+0000", 2024-05-08T08:52:17.241 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4709999999999996e-06 2024-05-08T08:52:17.241 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.241 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.241 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:17.241 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.911790+0000", 2024-05-08T08:52:17.241 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3345999999999999e-05 2024-05-08T08:52:17.242 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.242 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.242 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:17.242 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.911815+0000", 2024-05-08T08:52:17.242 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5378999999999999e-05 2024-05-08T08:52:17.242 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.242 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.242 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:17.242 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.913249+0000", 2024-05-08T08:52:17.242 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0014342669999999999 2024-05-08T08:52:17.242 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.242 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.242 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:17.242 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.913294+0000", 2024-05-08T08:52:17.243 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4873000000000002e-05 2024-05-08T08:52:17.243 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.243 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:17.243 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.243 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.243 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.243 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4465.0:1666 3.a e21/18 3:51fd3db2:::rbd_data.1140cf79ec85.0000000000000017:head v 21'117, mlcod=21'115)", 2024-05-08T08:52:17.243 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:16.925332+0000", 2024-05-08T08:52:17.243 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.27259179300000003, 2024-05-08T08:52:17.243 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00088647499999999996, 2024-05-08T08:52:17.243 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:17.243 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:17.243 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:17.243 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:17.243 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.244 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:17.244 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.925332+0000", 2024-05-08T08:52:17.244 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:17.244 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.244 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.244 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:17.244 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.925332+0000", 2024-05-08T08:52:17.244 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:17.244 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.244 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.244 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:17.244 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.925332+0000", 2024-05-08T08:52:17.245 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3099999999999996e-07 2024-05-08T08:52:17.245 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.245 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.245 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:17.245 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.925339+0000", 2024-05-08T08:52:17.245 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9269999999999999e-06 2024-05-08T08:52:17.245 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.245 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.245 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:17.245 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.925353+0000", 2024-05-08T08:52:17.245 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.341e-05 2024-05-08T08:52:17.245 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.245 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.245 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:17.246 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.925361+0000", 2024-05-08T08:52:17.246 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7489999999999998e-06 2024-05-08T08:52:17.246 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.246 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.246 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:17.246 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.925410+0000", 2024-05-08T08:52:17.246 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9978000000000002e-05 2024-05-08T08:52:17.246 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.246 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.246 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:17.246 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.925433+0000", 2024-05-08T08:52:17.246 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2555e-05 2024-05-08T08:52:17.247 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.247 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.247 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:17.247 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.926180+0000", 2024-05-08T08:52:17.247 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00074676600000000005 2024-05-08T08:52:17.247 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.247 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.247 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:17.247 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.926218+0000", 2024-05-08T08:52:17.247 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8259000000000001e-05 2024-05-08T08:52:17.247 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.247 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:17.247 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.247 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.248 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.248 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4465.0:1667 3.1 e21/18 3:82c79e2a:::rbd_data.1140cf79ec85.0000000000000018:head v 21'7, mlcod=21'5)", 2024-05-08T08:52:17.248 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:16.925403+0000", 2024-05-08T08:52:17.248 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.27252032900000001, 2024-05-08T08:52:17.248 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00078039899999999998, 2024-05-08T08:52:17.248 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:17.248 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:17.248 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:17.248 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:17.248 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.248 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:17.248 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.925403+0000", 2024-05-08T08:52:17.248 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:17.248 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.249 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.249 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:17.249 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.925403+0000", 2024-05-08T08:52:17.249 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:17.249 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.249 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.249 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:17.249 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.925404+0000", 2024-05-08T08:52:17.249 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9100000000000004e-07 2024-05-08T08:52:17.249 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.249 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.249 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:17.249 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.925408+0000", 2024-05-08T08:52:17.249 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2289999999999997e-06 2024-05-08T08:52:17.249 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.249 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.249 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:17.250 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.925417+0000", 2024-05-08T08:52:17.250 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9889999999999998e-06 2024-05-08T08:52:17.250 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.250 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.250 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:17.250 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.925432+0000", 2024-05-08T08:52:17.250 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5485000000000002e-05 2024-05-08T08:52:17.250 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.250 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.250 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:17.250 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.925487+0000", 2024-05-08T08:52:17.250 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4409e-05 2024-05-08T08:52:17.250 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.250 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.251 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:17.251 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.925505+0000", 2024-05-08T08:52:17.251 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8664e-05 2024-05-08T08:52:17.251 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.251 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.251 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:17.251 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.926143+0000", 2024-05-08T08:52:17.251 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00063743399999999998 2024-05-08T08:52:17.251 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.251 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.251 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:17.251 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.926183+0000", 2024-05-08T08:52:17.251 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0698000000000002e-05 2024-05-08T08:52:17.251 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.251 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:17.252 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.252 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.252 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.252 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4465.0:1848 3.1 e22/18 3:82c79e2a:::rbd_data.1140cf79ec85.0000000000000018:head v 22'8, mlcod=21'6)", 2024-05-08T08:52:17.252 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:17.001199+0000", 2024-05-08T08:52:17.252 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.19672456299999999, 2024-05-08T08:52:17.252 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00087624899999999995, 2024-05-08T08:52:17.252 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:17.252 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:17.252 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:17.252 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:17.252 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.252 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:17.252 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.001199+0000", 2024-05-08T08:52:17.252 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:17.253 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.253 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.253 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:17.253 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.001199+0000", 2024-05-08T08:52:17.253 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:17.253 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.253 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.253 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:17.253 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.001199+0000", 2024-05-08T08:52:17.253 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.84e-07 2024-05-08T08:52:17.253 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.253 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.253 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:17.253 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.001231+0000", 2024-05-08T08:52:17.253 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1908e-05 2024-05-08T08:52:17.254 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.254 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.254 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:17.254 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.001239+0000", 2024-05-08T08:52:17.254 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7389999999999999e-06 2024-05-08T08:52:17.254 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.254 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.254 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:17.254 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.001243+0000", 2024-05-08T08:52:17.254 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4610000000000001e-06 2024-05-08T08:52:17.254 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.254 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.254 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:17.255 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.001284+0000", 2024-05-08T08:52:17.255 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1085999999999999e-05 2024-05-08T08:52:17.255 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.255 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.255 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:17.255 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.001298+0000", 2024-05-08T08:52:17.255 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3811999999999999e-05 2024-05-08T08:52:17.255 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.255 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.255 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:17.255 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.002048+0000", 2024-05-08T08:52:17.255 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000750054 2024-05-08T08:52:17.255 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.255 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.256 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:17.256 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.002075+0000", 2024-05-08T08:52:17.256 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6905000000000001e-05 2024-05-08T08:52:17.256 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.256 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:17.256 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.256 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.256 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.256 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4465.0:1906 3.a e22/18 3:53569f60:::rbd_data.1140cf79ec85.0000000000000009:head v 22'129, mlcod=22'127)", 2024-05-08T08:52:17.256 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:17.036537+0000", 2024-05-08T08:52:17.256 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.16138684, 2024-05-08T08:52:17.256 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0023619919999999998, 2024-05-08T08:52:17.256 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:17.257 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:17.257 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:17.257 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:17.257 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.257 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:17.257 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.036537+0000", 2024-05-08T08:52:17.257 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:17.257 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.257 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.257 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:17.257 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.036537+0000", 2024-05-08T08:52:17.257 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:17.257 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.258 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.258 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:17.258 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.036537+0000", 2024-05-08T08:52:17.258 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9500000000000002e-07 2024-05-08T08:52:17.258 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.258 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.258 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:17.258 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.036558+0000", 2024-05-08T08:52:17.258 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0440000000000001e-05 2024-05-08T08:52:17.258 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.258 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.258 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:17.258 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.036570+0000", 2024-05-08T08:52:17.259 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.271e-05 2024-05-08T08:52:17.259 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.259 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.259 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:17.259 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.036579+0000", 2024-05-08T08:52:17.259 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3739999999999998e-06 2024-05-08T08:52:17.259 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.259 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.259 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:17.259 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.036634+0000", 2024-05-08T08:52:17.259 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4976999999999999e-05 2024-05-08T08:52:17.260 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.260 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.260 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:17.260 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.036650+0000", 2024-05-08T08:52:17.260 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6209e-05 2024-05-08T08:52:17.260 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.260 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.260 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:17.260 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.038858+0000", 2024-05-08T08:52:17.260 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00220835 2024-05-08T08:52:17.260 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.260 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.261 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:17.261 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.038899+0000", 2024-05-08T08:52:17.261 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0237000000000001e-05 2024-05-08T08:52:17.261 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.261 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:17.261 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.261 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.261 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.261 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4465.0:1977 3.1b e22/18 3:db5d4586:::rbd_data.1140cf79ec85.0000000000000010:head v 22'8, mlcod=21'6)", 2024-05-08T08:52:17.261 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:17.073810+0000", 2024-05-08T08:52:17.261 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.12411309099999999, 2024-05-08T08:52:17.261 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00068858799999999996, 2024-05-08T08:52:17.261 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:17.262 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:17.262 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:17.262 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:17.262 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.262 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:17.262 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.073810+0000", 2024-05-08T08:52:17.262 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:17.262 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.262 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.262 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:17.262 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.073810+0000", 2024-05-08T08:52:17.262 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:17.263 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.263 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.263 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:17.263 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.073811+0000", 2024-05-08T08:52:17.263 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7400000000000002e-07 2024-05-08T08:52:17.263 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.263 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.263 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:17.263 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.073817+0000", 2024-05-08T08:52:17.263 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7780000000000004e-06 2024-05-08T08:52:17.263 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.263 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.264 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:17.264 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.073830+0000", 2024-05-08T08:52:17.264 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3504e-05 2024-05-08T08:52:17.264 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.264 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.264 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:17.264 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.073837+0000", 2024-05-08T08:52:17.264 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8360000000000003e-06 2024-05-08T08:52:17.264 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.264 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.264 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:17.264 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.073894+0000", 2024-05-08T08:52:17.265 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6629999999999998e-05 2024-05-08T08:52:17.265 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.265 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.265 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:17.265 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.073914+0000", 2024-05-08T08:52:17.265 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0061999999999999e-05 2024-05-08T08:52:17.265 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.265 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.265 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:17.265 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.074456+0000", 2024-05-08T08:52:17.265 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00054217100000000004 2024-05-08T08:52:17.265 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.266 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.266 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:17.266 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.074499+0000", 2024-05-08T08:52:17.266 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2833000000000001e-05 2024-05-08T08:52:17.266 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.266 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:17.266 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.266 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.266 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.266 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4465.0:1963 3.11 e22/18 3:895c0c70:::rbd_data.1140cf79ec85.0000000000000006:head v 22'10, mlcod=21'8)", 2024-05-08T08:52:17.266 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:17.074011+0000", 2024-05-08T08:52:17.266 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.12391277000000001, 2024-05-08T08:52:17.267 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001461894, 2024-05-08T08:52:17.267 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:17.267 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:17.267 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:17.267 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:17.267 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.267 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:17.267 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.074011+0000", 2024-05-08T08:52:17.267 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:17.267 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.267 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.267 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:17.267 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.074011+0000", 2024-05-08T08:52:17.267 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:17.268 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.268 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.268 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:17.268 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.074011+0000", 2024-05-08T08:52:17.268 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8999999999999996e-07 2024-05-08T08:52:17.268 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.268 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.268 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:17.268 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.074017+0000", 2024-05-08T08:52:17.268 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4369999999999998e-06 2024-05-08T08:52:17.268 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.268 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.268 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:17.269 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.074026+0000", 2024-05-08T08:52:17.269 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4709999999999996e-06 2024-05-08T08:52:17.269 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.269 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.269 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:17.269 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.074032+0000", 2024-05-08T08:52:17.269 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.054e-06 2024-05-08T08:52:17.269 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.269 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.269 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:17.269 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.074084+0000", 2024-05-08T08:52:17.270 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2259000000000002e-05 2024-05-08T08:52:17.270 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.270 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.270 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:17.270 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.074102+0000", 2024-05-08T08:52:17.270 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7207e-05 2024-05-08T08:52:17.270 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.270 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.270 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:17.270 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.075433+0000", 2024-05-08T08:52:17.271 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0013315079999999999 2024-05-08T08:52:17.271 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.271 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.271 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:17.271 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.075473+0000", 2024-05-08T08:52:17.271 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9368e-05 2024-05-08T08:52:17.271 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.271 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:17.271 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.271 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.271 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.271 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4465.0:1967 3.a e22/18 3:53569f60:::rbd_data.1140cf79ec85.0000000000000009:head v 22'133, mlcod=22'131)", 2024-05-08T08:52:17.271 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:17.074816+0000", 2024-05-08T08:52:17.272 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.123107207, 2024-05-08T08:52:17.272 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001321865, 2024-05-08T08:52:17.272 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:17.272 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:17.272 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:17.272 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:17.272 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.272 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:17.272 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.074816+0000", 2024-05-08T08:52:17.272 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:17.272 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.272 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.272 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:17.272 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.074816+0000", 2024-05-08T08:52:17.272 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:17.273 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.273 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.273 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:17.273 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.074817+0000", 2024-05-08T08:52:17.273 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2300000000000001e-07 2024-05-08T08:52:17.273 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.273 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.273 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:17.273 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.074822+0000", 2024-05-08T08:52:17.273 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2549999999999997e-06 2024-05-08T08:52:17.273 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.273 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.273 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:17.274 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.074835+0000", 2024-05-08T08:52:17.274 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.277e-05 2024-05-08T08:52:17.274 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.274 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.274 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:17.274 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.074842+0000", 2024-05-08T08:52:17.274 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9589999999999998e-06 2024-05-08T08:52:17.274 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.274 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.274 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:17.274 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.074893+0000", 2024-05-08T08:52:17.274 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0902999999999997e-05 2024-05-08T08:52:17.274 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.274 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.275 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:17.275 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.074914+0000", 2024-05-08T08:52:17.275 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1115999999999999e-05 2024-05-08T08:52:17.275 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.275 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.275 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:17.275 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.076106+0000", 2024-05-08T08:52:17.275 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00119199 2024-05-08T08:52:17.275 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.275 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.275 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:17.275 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.076138+0000", 2024-05-08T08:52:17.275 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2249000000000002e-05 2024-05-08T08:52:17.275 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.276 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:17.276 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.276 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.276 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.276 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4465.0:2076 3.1b e22/18 3:db5d4586:::rbd_data.1140cf79ec85.0000000000000010:head v 22'9, mlcod=21'7)", 2024-05-08T08:52:17.276 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:17.123892+0000", 2024-05-08T08:52:17.276 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.074031713999999998, 2024-05-08T08:52:17.276 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00082955599999999996, 2024-05-08T08:52:17.276 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:17.276 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:17.276 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:17.276 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:17.276 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.276 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:17.276 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.123892+0000", 2024-05-08T08:52:17.276 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:17.277 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.277 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.277 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:17.277 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.123892+0000", 2024-05-08T08:52:17.277 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:17.277 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.277 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.277 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:17.277 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.123893+0000", 2024-05-08T08:52:17.277 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1179999999999999e-06 2024-05-08T08:52:17.277 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.277 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.277 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:17.277 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.123914+0000", 2024-05-08T08:52:17.277 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1509999999999999e-05 2024-05-08T08:52:17.278 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.278 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.278 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:17.278 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.123925+0000", 2024-05-08T08:52:17.278 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1072e-05 2024-05-08T08:52:17.278 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.278 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.278 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:17.278 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.123931+0000", 2024-05-08T08:52:17.278 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.13e-06 2024-05-08T08:52:17.278 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.278 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.278 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:17.278 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.123983+0000", 2024-05-08T08:52:17.279 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2695000000000003e-05 2024-05-08T08:52:17.279 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.279 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.279 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:17.279 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.124001+0000", 2024-05-08T08:52:17.279 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8031000000000001e-05 2024-05-08T08:52:17.279 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.279 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.279 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:17.279 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.124690+0000", 2024-05-08T08:52:17.279 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000688859 2024-05-08T08:52:17.279 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.279 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.279 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:17.279 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.124721+0000", 2024-05-08T08:52:17.280 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1140999999999999e-05 2024-05-08T08:52:17.280 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.280 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:17.280 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.280 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.280 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.280 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "pg_update_log_missing(3.a epoch 22/18 rep_tid 289 entries 22'137 (0'0) error 3:51a998f2:::rbd_header.1140cf79ec85:head by client.4416.0:785 0.000000 -16 [r=-16+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 22'135)", 2024-05-08T08:52:17.280 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:17.125898+0000", 2024-05-08T08:52:17.280 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.072025669, 2024-05-08T08:52:17.280 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0014140190000000001, 2024-05-08T08:52:17.280 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:17.280 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:17.280 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-05-08T08:52:17.280 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:17.281 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.281 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:17.281 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.125898+0000", 2024-05-08T08:52:17.281 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:17.281 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.281 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.281 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:17.281 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.125898+0000", 2024-05-08T08:52:17.281 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:17.281 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.281 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.281 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:17.281 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.125898+0000", 2024-05-08T08:52:17.281 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1900000000000003e-07 2024-05-08T08:52:17.282 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.282 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.282 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:17.282 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.125905+0000", 2024-05-08T08:52:17.282 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8460000000000002e-06 2024-05-08T08:52:17.282 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.282 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.282 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:17.282 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.125924+0000", 2024-05-08T08:52:17.282 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9388000000000001e-05 2024-05-08T08:52:17.282 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.283 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.283 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:17.283 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.125948+0000", 2024-05-08T08:52:17.283 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3331000000000001e-05 2024-05-08T08:52:17.283 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.283 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.283 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:17.283 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.126024+0000", 2024-05-08T08:52:17.283 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6429999999999998e-05 2024-05-08T08:52:17.283 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.283 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.284 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:17.284 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.127312+0000", 2024-05-08T08:52:17.284 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001287505 2024-05-08T08:52:17.284 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.284 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:17.284 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.284 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.284 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.284 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4465.0:2097 3.a e22/18 3:51a998f2:::rbd_header.1140cf79ec85:head v 22'138, mlcod=22'136)", 2024-05-08T08:52:17.284 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:17.133227+0000", 2024-05-08T08:52:17.284 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.064696188000000002, 2024-05-08T08:52:17.285 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00077306100000000002, 2024-05-08T08:52:17.285 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:17.285 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:17.285 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:17.285 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:17.285 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.285 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:17.285 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.133227+0000", 2024-05-08T08:52:17.285 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:17.285 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.285 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.285 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:17.285 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.133227+0000", 2024-05-08T08:52:17.285 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:17.286 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.286 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.286 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:17.286 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.133228+0000", 2024-05-08T08:52:17.286 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8000000000000005e-07 2024-05-08T08:52:17.286 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.286 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.286 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:17.286 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.133234+0000", 2024-05-08T08:52:17.286 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9660000000000001e-06 2024-05-08T08:52:17.286 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.286 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.286 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:17.286 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.133247+0000", 2024-05-08T08:52:17.287 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2707e-05 2024-05-08T08:52:17.287 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.287 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.287 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:17.287 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.133257+0000", 2024-05-08T08:52:17.287 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0553e-05 2024-05-08T08:52:17.287 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.287 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.287 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:17.287 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.133304+0000", 2024-05-08T08:52:17.287 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6325000000000001e-05 2024-05-08T08:52:17.287 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.287 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.287 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:17.287 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.133323+0000", 2024-05-08T08:52:17.288 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9318999999999999e-05 2024-05-08T08:52:17.288 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.288 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.288 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:17.288 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.133969+0000", 2024-05-08T08:52:17.288 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00064627200000000004 2024-05-08T08:52:17.288 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.288 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.288 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:17.288 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.134000+0000", 2024-05-08T08:52:17.288 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1139000000000001e-05 2024-05-08T08:52:17.288 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.288 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:17.288 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.289 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.289 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:17.289 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:52:17.292 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:52:17.292 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:52:17.293+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:52:17.294 INFO:teuthology.orchestra.run.smithi196.stdout:349 write 0x4b50800 thru 0x4b551dd (0x49de bytes) 2024-05-08T08:52:17.294 INFO:teuthology.orchestra.run.smithi196.stdout:350 read 0x9fb1a00 thru 0x9fbc2c4 (0xa8c5 bytes) 2024-05-08T08:52:17.294 INFO:teuthology.orchestra.run.smithi196.stdout:353 trunc from 0xb3fc400 to 0x1c9aa00 2024-05-08T08:52:17.294 INFO:teuthology.orchestra.run.smithi196.stdout:355 write 0x60de400 thru 0x60ea120 (0xbd21 bytes) 2024-05-08T08:52:17.294 INFO:teuthology.orchestra.run.smithi196.stdout:356 write 0xd97c600 thru 0xd98a973 (0xe374 bytes) 2024-05-08T08:52:17.294 INFO:teuthology.orchestra.run.smithi196.stdout:358 write 0x7dea600 thru 0x7dee02b (0x3a2c bytes) 2024-05-08T08:52:17.295 INFO:teuthology.orchestra.run.smithi196.stdout:359 punch from 0x6e2be00 to 0x6e34800, (0x8a00 bytes) 2024-05-08T08:52:17.295 INFO:teuthology.orchestra.run.smithi196.stdout:360 read 0x9c10400 thru 0x9c11db0 (0x19b1 bytes) 2024-05-08T08:52:17.295 INFO:teuthology.orchestra.run.smithi196.stdout:361 write 0x4e28400 thru 0x4e2edfd (0x69fe bytes) 2024-05-08T08:52:17.295 INFO:teuthology.orchestra.run.smithi196.stdout:362 read 0x533dc00 thru 0x534a9cd (0xcdce bytes) 2024-05-08T08:52:17.295 INFO:teuthology.orchestra.run.smithi196.stdout:364 read 0x5b6de00 thru 0x5b7d2d5 (0xf4d6 bytes) 2024-05-08T08:52:17.295 INFO:teuthology.orchestra.run.smithi196.stdout:366 read 0x3aed000 thru 0x3af636e (0x936f bytes) 2024-05-08T08:52:17.295 INFO:teuthology.orchestra.run.smithi196.stdout:367 read 0x45dac00 thru 0x45e13f2 (0x67f3 bytes) 2024-05-08T08:52:17.295 INFO:teuthology.orchestra.run.smithi196.stdout:369 write 0xacc9a00 thru 0xaccf63f (0x5c40 bytes) 2024-05-08T08:52:17.295 INFO:teuthology.orchestra.run.smithi196.stdout:371 read 0x4c0da00 thru 0x4c0f5d6 (0x1bd7 bytes) 2024-05-08T08:52:17.295 INFO:teuthology.orchestra.run.smithi196.stdout:372 write 0x3a40600 thru 0x3a44792 (0x4193 bytes) 2024-05-08T08:52:17.295 INFO:teuthology.orchestra.run.smithi196.stdout:373 read 0x3a4e800 thru 0x3a4f8f5 (0x10f6 bytes) 2024-05-08T08:52:17.296 INFO:teuthology.orchestra.run.smithi196.stdout:375 write 0xc735e00 thru 0xc744d34 (0xef35 bytes) 2024-05-08T08:52:17.296 INFO:teuthology.orchestra.run.smithi196.stdout:377 write 0xd8b6200 thru 0xd8c414e (0xdf4f bytes) 2024-05-08T08:52:17.296 INFO:teuthology.orchestra.run.smithi196.stdout:378 write 0x9757c00 thru 0x975d425 (0x5826 bytes) 2024-05-08T08:52:17.296 INFO:teuthology.orchestra.run.smithi196.stdout:382 write 0x390000 thru 0x39a956 (0xa957 bytes) 2024-05-08T08:52:17.296 INFO:teuthology.orchestra.run.smithi196.stdout:383 punch from 0xbfce400 to 0xbfcfa00, (0x1600 bytes) 2024-05-08T08:52:17.296 INFO:teuthology.orchestra.run.smithi196.stdout:386 read 0xd958400 thru 0xd965a21 (0xd622 bytes) 2024-05-08T08:52:17.296 INFO:teuthology.orchestra.run.smithi196.stdout:387 read 0xa47ba00 thru 0xa47d325 (0x1926 bytes) 2024-05-08T08:52:17.296 INFO:teuthology.orchestra.run.smithi196.stdout:393 read 0xd289e00 thru 0xd293a2d (0x9c2e bytes) 2024-05-08T08:52:17.296 INFO:teuthology.orchestra.run.smithi196.stdout:394 read 0xd986000 thru 0xd9876df (0x16e0 bytes) 2024-05-08T08:52:17.300 INFO:teuthology.orchestra.run.smithi196.stdout:395 read 0x9ab4a00 thru 0x9abc23e (0x783f bytes) 2024-05-08T08:52:17.300 INFO:teuthology.orchestra.run.smithi196.stdout:396 write 0x25b2e00 thru 0x25b7b0f (0x4d10 bytes) 2024-05-08T08:52:17.301 INFO:teuthology.orchestra.run.smithi196.stdout:398 write 0x67ef200 thru 0x67fc4e8 (0xd2e9 bytes) 2024-05-08T08:52:17.301 INFO:teuthology.orchestra.run.smithi196.stdout:400 read 0xd46f000 thru 0xd47d16c (0xe16d bytes) 2024-05-08T08:52:17.305 INFO:teuthology.orchestra.run.smithi196.stdout:401 trunc from 0xd98aa00 to 0xa137000 2024-05-08T08:52:17.332 INFO:teuthology.orchestra.run.smithi196.stdout:403 read 0x3e91e00 thru 0x3e9b246 (0x9447 bytes) 2024-05-08T08:52:17.339 INFO:teuthology.orchestra.run.smithi196.stdout:405 read 0x5604000 thru 0x56055cd (0x15ce bytes) 2024-05-08T08:52:17.339 INFO:teuthology.orchestra.run.smithi196.stdout:406 write 0x3caae00 thru 0x3cb3f92 (0x9193 bytes) 2024-05-08T08:52:17.339 INFO:teuthology.orchestra.run.smithi196.stdout:407 punch from 0x59de800 to 0x59dee00, (0x600 bytes) 2024-05-08T08:52:17.339 INFO:teuthology.orchestra.run.smithi196.stdout:408 write 0x72e0000 thru 0x72e804e (0x804f bytes) 2024-05-08T08:52:17.339 INFO:teuthology.orchestra.run.smithi196.stdout:410 punch from 0x3ea9e00 to 0x3eae600, (0x4800 bytes) 2024-05-08T08:52:17.349 INFO:teuthology.orchestra.run.smithi196.stdout:411 trunc from 0xa137000 to 0x28d7c00 2024-05-08T08:52:17.370 INFO:teuthology.orchestra.run.smithi196.stdout:412 read 0x62dc00 thru 0x63232c (0x472d bytes) 2024-05-08T08:52:17.371 INFO:teuthology.orchestra.run.smithi196.stdout:413 punch from 0x6a7600 to 0x6b1a00, (0xa400 bytes) 2024-05-08T08:52:17.388 INFO:teuthology.orchestra.run.smithi196.stdout:414 write 0x8bf8c00 thru 0x8c064ad (0xd8ae bytes) 2024-05-08T08:52:17.389 INFO:teuthology.orchestra.run.smithi196.stdout:416 read 0x2906a00 thru 0x2911c94 (0xb295 bytes) 2024-05-08T08:52:17.389 INFO:teuthology.orchestra.run.smithi196.stdout:417 read 0x106b200 thru 0x106c41c (0x121d bytes) 2024-05-08T08:52:17.389 INFO:teuthology.orchestra.run.smithi196.stdout:418 read 0x5539800 thru 0x553bdce (0x25cf bytes) 2024-05-08T08:52:17.389 INFO:teuthology.orchestra.run.smithi196.stdout:420 read 0x29b7000 thru 0x29be654 (0x7655 bytes) 2024-05-08T08:52:17.390 INFO:teuthology.orchestra.run.smithi196.stdout:422 read 0x56fb200 thru 0x5700c04 (0x5a05 bytes) 2024-05-08T08:52:17.390 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:17.394+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:17.402 INFO:teuthology.orchestra.run.smithi196.stdout:423 write 0x9163400 thru 0x9173060 (0xfc61 bytes) 2024-05-08T08:52:17.402 INFO:teuthology.orchestra.run.smithi196.stdout:424 write 0x8759000 thru 0x875925f (0x260 bytes) 2024-05-08T08:52:17.403 INFO:teuthology.orchestra.run.smithi196.stdout:426 trunc from 0x9173200 to 0x9d94200 2024-05-08T08:52:17.425 INFO:teuthology.orchestra.run.smithi196.stdout:428 write 0x8ff7e00 thru 0x8ff8a0d (0xc0e bytes) 2024-05-08T08:52:17.425 INFO:teuthology.orchestra.run.smithi196.stdout:430 read 0x9b9c200 thru 0x9b9d1aa (0xfab bytes) 2024-05-08T08:52:17.425 INFO:teuthology.orchestra.run.smithi196.stdout:431 read 0x88b200 thru 0x88e36e (0x316f bytes) 2024-05-08T08:52:17.426 INFO:teuthology.orchestra.run.smithi196.stdout:433 read 0xed7400 thru 0xee07d3 (0x93d4 bytes) 2024-05-08T08:52:17.427 INFO:teuthology.orchestra.run.smithi196.stdout:434 write 0x2e84000 thru 0x2e92c58 (0xec59 bytes) 2024-05-08T08:52:17.427 INFO:teuthology.orchestra.run.smithi196.stdout:436 read 0x3483000 thru 0x34837f6 (0x7f7 bytes) 2024-05-08T08:52:17.427 INFO:teuthology.orchestra.run.smithi196.stdout:437 read 0x9459000 thru 0x946876d (0xf76e bytes) 2024-05-08T08:52:17.453 INFO:teuthology.orchestra.run.smithi196.stdout:438 write 0xd8ebc00 thru 0xd8f0383 (0x4784 bytes) 2024-05-08T08:52:17.453 INFO:teuthology.orchestra.run.smithi196.stdout:441 write 0x2cef000 thru 0x2cefb6f (0xb70 bytes) 2024-05-08T08:52:17.454 INFO:teuthology.orchestra.run.smithi196.stdout:443 write 0xbe0c600 thru 0xbe10176 (0x3b77 bytes) 2024-05-08T08:52:17.454 INFO:teuthology.orchestra.run.smithi196.stdout:444 write 0x8c2d800 thru 0x8c33e36 (0x6637 bytes) 2024-05-08T08:52:17.454 INFO:teuthology.orchestra.run.smithi196.stdout:445 write 0x305fa00 thru 0x30610e7 (0x16e8 bytes) 2024-05-08T08:52:17.454 INFO:teuthology.orchestra.run.smithi196.stdout:447 write 0x3ac6800 thru 0x3ad4101 (0xd902 bytes) 2024-05-08T08:52:17.455 INFO:teuthology.orchestra.run.smithi196.stdout:truncating to largest ever: 0xeca2a00 2024-05-08T08:52:17.455 INFO:teuthology.orchestra.run.smithi196.stdout:450 trunc from 0xd8f0400 to 0xeca2a00 2024-05-08T08:52:17.456 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:52:17.456 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:52:17.456 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:52:17.456 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:52:17.467 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:52:17.473 INFO:teuthology.orchestra.run.smithi196.stdout:451 write 0xc62a800 thru 0xc6395d1 (0xedd2 bytes) 2024-05-08T08:52:17.475 INFO:teuthology.orchestra.run.smithi196.stdout:453 read 0x1cd6c00 thru 0x1cd868b (0x1a8c bytes) 2024-05-08T08:52:17.475 INFO:teuthology.orchestra.run.smithi196.stdout:454 read 0xba6f200 thru 0xba797fe (0xa5ff bytes) 2024-05-08T08:52:17.475 INFO:teuthology.orchestra.run.smithi196.stdout:455 read 0x7347600 thru 0x7348828 (0x1229 bytes) 2024-05-08T08:52:17.475 INFO:teuthology.orchestra.run.smithi196.stdout:457 read 0xac52e00 thru 0xac60a21 (0xdc22 bytes) 2024-05-08T08:52:17.475 INFO:teuthology.orchestra.run.smithi196.stdout:458 read 0x90d6c00 thru 0x90dfc03 (0x9004 bytes) 2024-05-08T08:52:17.489 INFO:teuthology.orchestra.run.smithi196.stdout:460 trunc from 0xeca2a00 to 0x31fcc00 2024-05-08T08:52:17.509 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:17.512+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:17.539 INFO:teuthology.orchestra.run.smithi196.stdout:461 write 0xe153e00 thru 0xe158529 (0x472a bytes) 2024-05-08T08:52:17.541 INFO:teuthology.orchestra.run.smithi196.stdout:462 write 0x1778c00 thru 0x177fffb (0x73fc bytes) 2024-05-08T08:52:17.541 INFO:teuthology.orchestra.run.smithi196.stdout:463 write 0x1c67000 thru 0x1c736c2 (0xc6c3 bytes) 2024-05-08T08:52:17.541 INFO:teuthology.orchestra.run.smithi196.stdout:464 read 0x100c800 thru 0x1016438 (0x9c39 bytes) 2024-05-08T08:52:17.541 INFO:teuthology.orchestra.run.smithi196.stdout:465 read 0x6deea00 thru 0x6df8540 (0x9b41 bytes) 2024-05-08T08:52:17.541 INFO:teuthology.orchestra.run.smithi196.stdout:468 read 0xbf50200 thru 0xbf53ebb (0x3cbc bytes) 2024-05-08T08:52:17.541 INFO:teuthology.orchestra.run.smithi196.stdout:469 read 0x3a76200 thru 0x3a80bcd (0xa9ce bytes) 2024-05-08T08:52:17.541 INFO:teuthology.orchestra.run.smithi196.stdout:472 write 0x461e600 thru 0x461f9c5 (0x13c6 bytes) 2024-05-08T08:52:17.541 INFO:teuthology.orchestra.run.smithi196.stdout:473 write 0x9ce6600 thru 0x9cf08ba (0xa2bb bytes) 2024-05-08T08:52:17.541 INFO:teuthology.orchestra.run.smithi196.stdout:475 read 0xd2ab000 thru 0xd2b12c7 (0x62c8 bytes) 2024-05-08T08:52:17.542 INFO:teuthology.orchestra.run.smithi196.stdout:479 read 0xcccf200 thru 0xccd6a44 (0x7845 bytes) 2024-05-08T08:52:17.542 INFO:teuthology.orchestra.run.smithi196.stdout:482 read 0x8f47200 thru 0x8f562f0 (0xf0f1 bytes) 2024-05-08T08:52:17.542 INFO:teuthology.orchestra.run.smithi196.stdout:483 read 0x8460e00 thru 0x846a99d (0x9b9e bytes) 2024-05-08T08:52:17.544 INFO:teuthology.orchestra.run.smithi196.stdout:484 trunc from 0xe158600 to 0xb7a2e00 2024-05-08T08:52:17.572 INFO:teuthology.orchestra.run.smithi196.stdout:489 write 0x8804000 thru 0x88062cd (0x22ce bytes) 2024-05-08T08:52:17.574 INFO:teuthology.orchestra.run.smithi196.stdout:491 punch from 0x73d3e00 to 0x73e3a00, (0xfc00 bytes) 2024-05-08T08:52:17.577 INFO:teuthology.orchestra.run.smithi196.stdout:493 read 0x338be00 thru 0x338c8a3 (0xaa4 bytes) 2024-05-08T08:52:17.578 INFO:teuthology.orchestra.run.smithi196.stdout:494 read 0x6feb600 thru 0x6ff86e5 (0xd0e6 bytes) 2024-05-08T08:52:17.578 INFO:teuthology.orchestra.run.smithi196.stdout:495 write 0x82dbc00 thru 0x82e9e96 (0xe297 bytes) 2024-05-08T08:52:17.578 INFO:teuthology.orchestra.run.smithi196.stdout:497 read 0x45b1e00 thru 0x45c0c45 (0xee46 bytes) 2024-05-08T08:52:17.578 INFO:teuthology.orchestra.run.smithi196.stdout:498 read 0x6aaf600 thru 0x6ab5e1f (0x6820 bytes) 2024-05-08T08:52:17.593 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:17.596+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:17.597 INFO:teuthology.orchestra.run.smithi196.stdout:499 write 0xebf2800 thru 0xec00d44 (0xe545 bytes) 2024-05-08T08:52:17.598 INFO:teuthology.orchestra.run.smithi196.stdout:503 read 0x855be00 thru 0x85612e0 (0x54e1 bytes) 2024-05-08T08:52:17.598 INFO:teuthology.orchestra.run.smithi196.stdout:504 read 0xcc19400 thru 0xcc1fbad (0x67ae bytes) 2024-05-08T08:52:17.602 INFO:teuthology.orchestra.run.smithi196.stdout:505 trunc from 0xec00e00 to 0xb7ca200 2024-05-08T08:52:17.626 INFO:teuthology.orchestra.run.smithi196.stdout:506 write 0x9608c00 thru 0x9612293 (0x9694 bytes) 2024-05-08T08:52:17.626 INFO:teuthology.orchestra.run.smithi196.stdout:507 read 0x378bc00 thru 0x378f530 (0x3931 bytes) 2024-05-08T08:52:17.626 INFO:teuthology.orchestra.run.smithi196.stdout:514 punch from 0x8e2f800 to 0x8e37a00, (0x8200 bytes) 2024-05-08T08:52:17.626 INFO:teuthology.orchestra.run.smithi196.stdout:518 write 0x7c88600 thru 0x7c96e89 (0xe88a bytes) 2024-05-08T08:52:17.627 INFO:teuthology.orchestra.run.smithi196.stdout:519 read 0x1646e00 thru 0x16488a5 (0x1aa6 bytes) 2024-05-08T08:52:17.627 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:52:17.627 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:52:17.627 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:52:17.627 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:52:17.627 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:52:17.627 INFO:teuthology.orchestra.run.smithi196.stdout:522 read 0x2a3aa00 thru 0x2a47544 (0xcb45 bytes) 2024-05-08T08:52:17.637 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:52:17.652 INFO:teuthology.orchestra.run.smithi196.stdout:523 write 0xd22f200 thru 0xd230350 (0x1151 bytes) 2024-05-08T08:52:17.664 INFO:teuthology.orchestra.run.smithi196.stdout:525 trunc from 0xd230400 to 0x212b600 2024-05-08T08:52:17.686 INFO:teuthology.orchestra.run.smithi196.stdout:528 read 0x18f0400 thru 0x18f65fc (0x61fd bytes) 2024-05-08T08:52:17.708 INFO:teuthology.orchestra.run.smithi196.stdout:530 write 0xa688a00 thru 0xa693ad8 (0xb0d9 bytes) 2024-05-08T08:52:17.708 INFO:teuthology.orchestra.run.smithi196.stdout:533 read 0xe3400 thru 0xe8ffa (0x5bfb bytes) 2024-05-08T08:52:17.709 INFO:teuthology.orchestra.run.smithi196.stdout:534 punch from 0xa53ec00 to 0xa541e00, (0x3200 bytes) 2024-05-08T08:52:17.721 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:17.724+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:17.729 INFO:teuthology.orchestra.run.smithi196.stdout:535 write 0xdd43a00 thru 0xdd4f45a (0xba5b bytes) 2024-05-08T08:52:17.736 INFO:teuthology.orchestra.run.smithi196.stdout:537 punch from 0x77a4a00 to 0x77a7c00, (0x3200 bytes) 2024-05-08T08:52:17.736 INFO:teuthology.orchestra.run.smithi196.stdout:538 read 0xbc2de00 thru 0xbc3d38e (0xf58f bytes) 2024-05-08T08:52:17.736 INFO:teuthology.orchestra.run.smithi196.stdout:540 write 0x15b2400 thru 0x15ba32d (0x7f2e bytes) 2024-05-08T08:52:17.737 INFO:teuthology.orchestra.run.smithi196.stdout:541 clone 1 order 19 su 262144 sc 10 2024-05-08T08:52:17.792 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:52:17.792 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:52:17.792 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:52:17.792 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:52:17.792 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.792 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4465.0:1552 3.17 e21/18 3:e8515fe1:::rbd_object_map.1140cf79ec85:head v 21'149, mlcod=21'146)", 2024-05-08T08:52:17.792 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:16.874806+0000", 2024-05-08T08:52:17.792 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.91925961300000003, 2024-05-08T08:52:17.793 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0054696579999999996, 2024-05-08T08:52:17.793 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:17.793 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:17.793 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:17.793 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:17.793 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.793 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:17.793 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.874806+0000", 2024-05-08T08:52:17.793 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:17.793 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.793 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.793 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:17.793 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.874806+0000", 2024-05-08T08:52:17.793 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:17.794 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.794 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.794 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:17.794 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.874806+0000", 2024-05-08T08:52:17.794 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5499999999999998e-07 2024-05-08T08:52:17.794 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.794 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.794 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:17.794 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.874812+0000", 2024-05-08T08:52:17.794 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5589999999999996e-06 2024-05-08T08:52:17.794 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.794 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.795 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:17.795 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.874819+0000", 2024-05-08T08:52:17.795 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.435e-06 2024-05-08T08:52:17.795 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.795 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.795 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:17.795 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.874825+0000", 2024-05-08T08:52:17.795 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6790000000000002e-06 2024-05-08T08:52:17.795 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.795 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.795 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:17.795 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.874870+0000", 2024-05-08T08:52:17.796 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4774999999999997e-05 2024-05-08T08:52:17.796 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.796 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.796 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:17.796 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.874889+0000", 2024-05-08T08:52:17.796 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9134999999999999e-05 2024-05-08T08:52:17.796 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.796 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.796 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:17.796 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.880237+0000", 2024-05-08T08:52:17.796 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0053480919999999996 2024-05-08T08:52:17.796 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.796 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.797 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:17.797 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.880275+0000", 2024-05-08T08:52:17.797 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8427999999999999e-05 2024-05-08T08:52:17.797 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.797 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:17.797 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.797 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.797 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.797 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4465.0:1553 3.17 e21/18 3:e8515fe1:::rbd_object_map.1140cf79ec85:head v 21'150, mlcod=21'146)", 2024-05-08T08:52:17.797 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:16.875484+0000", 2024-05-08T08:52:17.797 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.91858182899999996, 2024-05-08T08:52:17.797 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0048115149999999997, 2024-05-08T08:52:17.797 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:17.797 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:17.798 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:17.798 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:17.798 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.798 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:17.798 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.875484+0000", 2024-05-08T08:52:17.798 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:17.798 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.798 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.798 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:17.798 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.875484+0000", 2024-05-08T08:52:17.798 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:17.798 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.799 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.799 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:17.799 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.875484+0000", 2024-05-08T08:52:17.799 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4299999999999999e-07 2024-05-08T08:52:17.799 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.799 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.799 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:17.799 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.875487+0000", 2024-05-08T08:52:17.799 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.275e-06 2024-05-08T08:52:17.799 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.799 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.799 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:17.799 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.875493+0000", 2024-05-08T08:52:17.800 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4580000000000001e-06 2024-05-08T08:52:17.800 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.800 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.800 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:17.800 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.875496+0000", 2024-05-08T08:52:17.800 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5030000000000002e-06 2024-05-08T08:52:17.800 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.800 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.800 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:17.800 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.875533+0000", 2024-05-08T08:52:17.800 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.714e-05 2024-05-08T08:52:17.800 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.800 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.800 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:17.801 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.875544+0000", 2024-05-08T08:52:17.801 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0899e-05 2024-05-08T08:52:17.801 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.801 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.801 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:17.801 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.880278+0000", 2024-05-08T08:52:17.801 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0047333820000000004 2024-05-08T08:52:17.801 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.801 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.801 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:17.801 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.880295+0000", 2024-05-08T08:52:17.801 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7515000000000001e-05 2024-05-08T08:52:17.801 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.801 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:17.801 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.802 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.802 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.802 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4465.0:1559 3.17 e21/18 3:e8515fe1:::rbd_object_map.1140cf79ec85:head v 21'153, mlcod=21'146)", 2024-05-08T08:52:17.802 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:16.877984+0000", 2024-05-08T08:52:17.802 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.91608148599999994, 2024-05-08T08:52:17.802 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0050888729999999998, 2024-05-08T08:52:17.802 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:17.802 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:17.802 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:17.802 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:17.802 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.802 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:17.802 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.877984+0000", 2024-05-08T08:52:17.803 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:17.803 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.803 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.803 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:17.803 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.877984+0000", 2024-05-08T08:52:17.803 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:17.803 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.803 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.803 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:17.803 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.877984+0000", 2024-05-08T08:52:17.803 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7300000000000001e-07 2024-05-08T08:52:17.803 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.804 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.804 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:17.804 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.877988+0000", 2024-05-08T08:52:17.804 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2990000000000001e-06 2024-05-08T08:52:17.804 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.804 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.804 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:17.804 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.877995+0000", 2024-05-08T08:52:17.804 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6029999999999999e-06 2024-05-08T08:52:17.804 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.804 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.804 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:17.804 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.877999+0000", 2024-05-08T08:52:17.805 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8879999999999999e-06 2024-05-08T08:52:17.805 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.805 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.805 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:17.805 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.878033+0000", 2024-05-08T08:52:17.805 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3346000000000002e-05 2024-05-08T08:52:17.805 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.805 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.805 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:17.805 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.878045+0000", 2024-05-08T08:52:17.805 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2252000000000001e-05 2024-05-08T08:52:17.805 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.805 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.806 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:17.806 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.883036+0000", 2024-05-08T08:52:17.806 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0049908319999999997 2024-05-08T08:52:17.806 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.806 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.806 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:17.806 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.883073+0000", 2024-05-08T08:52:17.806 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.718e-05 2024-05-08T08:52:17.806 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.806 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:17.806 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.806 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.807 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.807 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4465.0:1586 3.16 3:6ae684b5:::rbd_data.1140cf79ec85.0000000000000039:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-05-08T08:52:17.807 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:16.890498+0000", 2024-05-08T08:52:17.807 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.90356730500000004, 2024-05-08T08:52:17.807 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0050242250000000002, 2024-05-08T08:52:17.807 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:17.807 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:17.807 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:17.807 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:17.807 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4465", 2024-05-08T08:52:17.807 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/390275688", 2024-05-08T08:52:17.807 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1586 2024-05-08T08:52:17.807 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.808 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:17.808 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.808 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:17.808 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.890498+0000", 2024-05-08T08:52:17.808 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:17.808 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.808 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.808 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:17.808 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.890498+0000", 2024-05-08T08:52:17.808 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:17.808 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.808 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.808 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:17.808 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.890499+0000", 2024-05-08T08:52:17.808 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3099999999999998e-07 2024-05-08T08:52:17.809 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.809 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.809 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:17.809 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.890503+0000", 2024-05-08T08:52:17.809 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9859999999999996e-06 2024-05-08T08:52:17.809 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.809 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.809 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:17.809 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.890510+0000", 2024-05-08T08:52:17.809 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6140000000000003e-06 2024-05-08T08:52:17.809 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.809 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.809 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:17.809 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.890516+0000", 2024-05-08T08:52:17.809 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5230000000000003e-06 2024-05-08T08:52:17.810 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.810 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.810 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:17.810 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.890564+0000", 2024-05-08T08:52:17.810 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7998999999999999e-05 2024-05-08T08:52:17.810 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.810 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.810 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:17.810 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.890635+0000", 2024-05-08T08:52:17.810 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1641000000000001e-05 2024-05-08T08:52:17.810 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.810 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.810 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 3", 2024-05-08T08:52:17.810 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.890734+0000", 2024-05-08T08:52:17.810 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8393000000000005e-05 2024-05-08T08:52:17.811 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.811 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.811 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:17.811 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.892579+0000", 2024-05-08T08:52:17.811 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001845621 2024-05-08T08:52:17.811 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.811 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.811 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:52:17.811 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.895466+0000", 2024-05-08T08:52:17.811 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002887106 2024-05-08T08:52:17.811 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.811 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.811 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:17.811 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.895498+0000", 2024-05-08T08:52:17.811 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1562e-05 2024-05-08T08:52:17.812 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.812 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.812 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:17.812 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.895522+0000", 2024-05-08T08:52:17.812 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4349e-05 2024-05-08T08:52:17.812 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.812 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:17.812 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.812 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.812 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.812 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4465.0:1711 3.16 3:6e8ec35b:::rbd_data.1140cf79ec85.000000000000000a:head [write 3710976~53248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-05-08T08:52:17.812 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:16.943469+0000", 2024-05-08T08:52:17.812 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.85059676500000003, 2024-05-08T08:52:17.812 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0051402790000000002, 2024-05-08T08:52:17.812 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:17.813 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:17.813 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:17.813 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:17.813 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4465", 2024-05-08T08:52:17.813 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/390275688", 2024-05-08T08:52:17.813 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1711 2024-05-08T08:52:17.813 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.813 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:17.813 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.813 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:17.813 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.943469+0000", 2024-05-08T08:52:17.813 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:17.813 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.813 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.813 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:17.814 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.943469+0000", 2024-05-08T08:52:17.814 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:17.814 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.814 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.814 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:17.814 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.943470+0000", 2024-05-08T08:52:17.814 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1629999999999999e-06 2024-05-08T08:52:17.814 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.814 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.814 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:17.814 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.943550+0000", 2024-05-08T08:52:17.814 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0261999999999998e-05 2024-05-08T08:52:17.814 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.814 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.815 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:17.815 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.943564+0000", 2024-05-08T08:52:17.815 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4126e-05 2024-05-08T08:52:17.815 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.815 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.815 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:17.815 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.943572+0000", 2024-05-08T08:52:17.815 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2369999999999992e-06 2024-05-08T08:52:17.815 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.815 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.815 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:17.815 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.943628+0000", 2024-05-08T08:52:17.815 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5671000000000003e-05 2024-05-08T08:52:17.815 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.815 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.816 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:17.816 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.943705+0000", 2024-05-08T08:52:17.816 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6941999999999996e-05 2024-05-08T08:52:17.816 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.816 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.816 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 3", 2024-05-08T08:52:17.816 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.943830+0000", 2024-05-08T08:52:17.816 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012502300000000001 2024-05-08T08:52:17.816 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.816 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.816 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:52:17.816 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.947481+0000", 2024-05-08T08:52:17.816 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0036509979999999999 2024-05-08T08:52:17.816 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.816 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.816 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:17.817 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.948543+0000", 2024-05-08T08:52:17.817 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001061761 2024-05-08T08:52:17.817 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.817 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.817 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:17.817 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.948588+0000", 2024-05-08T08:52:17.817 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5228000000000001e-05 2024-05-08T08:52:17.817 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.817 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.817 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:17.817 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.948609+0000", 2024-05-08T08:52:17.817 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0868000000000001e-05 2024-05-08T08:52:17.817 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.817 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:17.817 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.818 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.818 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.818 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4465.0:1713 3.16 3:696eb650:::rbd_data.1140cf79ec85.000000000000000e:head [write 0~16384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-05-08T08:52:17.818 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:16.943593+0000", 2024-05-08T08:52:17.818 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.85047261200000002, 2024-05-08T08:52:17.818 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0052808450000000002, 2024-05-08T08:52:17.818 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:17.818 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:17.818 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:17.818 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:17.818 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4465", 2024-05-08T08:52:17.818 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/390275688", 2024-05-08T08:52:17.818 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1713 2024-05-08T08:52:17.818 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.819 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:17.819 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.819 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:17.819 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.943593+0000", 2024-05-08T08:52:17.819 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:17.819 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.819 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.819 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:17.819 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.943593+0000", 2024-05-08T08:52:17.819 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:17.819 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.819 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.819 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:17.820 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.943593+0000", 2024-05-08T08:52:17.820 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7700000000000004e-07 2024-05-08T08:52:17.820 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.820 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.820 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:17.820 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.943610+0000", 2024-05-08T08:52:17.820 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6750999999999999e-05 2024-05-08T08:52:17.820 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.820 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.820 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:17.820 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.943618+0000", 2024-05-08T08:52:17.820 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9990000000000008e-06 2024-05-08T08:52:17.820 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.820 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.820 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:17.821 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.943624+0000", 2024-05-08T08:52:17.821 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5430000000000001e-06 2024-05-08T08:52:17.821 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.821 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.821 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:17.821 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.946529+0000", 2024-05-08T08:52:17.821 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002905281 2024-05-08T08:52:17.821 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.821 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.821 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:17.821 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.946574+0000", 2024-05-08T08:52:17.821 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5269999999999999e-05 2024-05-08T08:52:17.822 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.822 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.822 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 3", 2024-05-08T08:52:17.822 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.946684+0000", 2024-05-08T08:52:17.822 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010990299999999999 2024-05-08T08:52:17.822 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.822 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.822 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:52:17.822 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.947887+0000", 2024-05-08T08:52:17.822 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001202514 2024-05-08T08:52:17.822 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.822 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.822 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:17.823 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.948825+0000", 2024-05-08T08:52:17.823 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00093866700000000002 2024-05-08T08:52:17.823 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.823 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.823 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:17.823 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.948854+0000", 2024-05-08T08:52:17.823 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8377e-05 2024-05-08T08:52:17.823 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.823 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.823 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:17.823 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:16.948874+0000", 2024-05-08T08:52:17.823 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9962999999999998e-05 2024-05-08T08:52:17.823 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.824 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:17.824 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.824 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.824 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.824 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4465.0:1964 3.16 3:6d9a0c0e:::rbd_data.1140cf79ec85.0000000000000007:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-05-08T08:52:17.824 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:17.073412+0000", 2024-05-08T08:52:17.824 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.72065379500000004, 2024-05-08T08:52:17.824 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0057005270000000004, 2024-05-08T08:52:17.824 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:17.824 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:17.824 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:17.824 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:17.824 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4465", 2024-05-08T08:52:17.824 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/390275688", 2024-05-08T08:52:17.824 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1964 2024-05-08T08:52:17.825 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.825 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:17.825 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.825 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:17.825 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.073412+0000", 2024-05-08T08:52:17.825 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:17.825 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.825 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.825 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:17.825 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.073412+0000", 2024-05-08T08:52:17.825 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:17.825 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.825 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.826 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:17.826 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.073412+0000", 2024-05-08T08:52:17.826 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8899999999999999e-07 2024-05-08T08:52:17.826 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.826 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.826 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:17.826 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.073415+0000", 2024-05-08T08:52:17.826 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6089999999999999e-06 2024-05-08T08:52:17.826 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.826 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.826 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:17.826 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.073423+0000", 2024-05-08T08:52:17.826 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3869999999999995e-06 2024-05-08T08:52:17.827 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.827 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.827 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:17.827 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.073428+0000", 2024-05-08T08:52:17.827 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6140000000000002e-06 2024-05-08T08:52:17.827 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.827 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.827 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:17.827 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.073473+0000", 2024-05-08T08:52:17.827 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5395999999999997e-05 2024-05-08T08:52:17.827 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.827 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.827 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:17.827 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.073544+0000", 2024-05-08T08:52:17.828 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1031000000000005e-05 2024-05-08T08:52:17.828 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.828 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.828 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 3", 2024-05-08T08:52:17.828 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.073648+0000", 2024-05-08T08:52:17.828 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000103488 2024-05-08T08:52:17.828 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.828 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.828 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:52:17.828 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.075789+0000", 2024-05-08T08:52:17.828 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0021411400000000001 2024-05-08T08:52:17.828 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.828 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.828 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:17.829 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.079024+0000", 2024-05-08T08:52:17.829 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0032352140000000001 2024-05-08T08:52:17.829 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.829 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.829 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:17.829 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.079071+0000", 2024-05-08T08:52:17.829 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7284999999999999e-05 2024-05-08T08:52:17.829 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.829 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.829 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:17.829 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.079112+0000", 2024-05-08T08:52:17.829 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0774e-05 2024-05-08T08:52:17.829 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.829 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:17.829 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.829 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.830 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.830 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4465.0:1969 3.16 3:6e8ec35b:::rbd_data.1140cf79ec85.000000000000000a:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-05-08T08:52:17.830 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:17.073467+0000", 2024-05-08T08:52:17.830 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.72059868800000004, 2024-05-08T08:52:17.830 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0058463730000000002, 2024-05-08T08:52:17.830 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:17.830 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:17.830 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:17.830 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:17.830 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4465", 2024-05-08T08:52:17.830 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/390275688", 2024-05-08T08:52:17.830 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1969 2024-05-08T08:52:17.830 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.831 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:17.831 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.831 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:17.831 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.073467+0000", 2024-05-08T08:52:17.831 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:17.831 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.831 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.831 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:17.831 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.073467+0000", 2024-05-08T08:52:17.831 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:17.831 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.831 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.831 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:17.832 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.073467+0000", 2024-05-08T08:52:17.832 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3600000000000004e-07 2024-05-08T08:52:17.832 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.832 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.832 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:17.832 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.073471+0000", 2024-05-08T08:52:17.832 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2550000000000002e-06 2024-05-08T08:52:17.832 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.832 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.832 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:17.832 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.073479+0000", 2024-05-08T08:52:17.832 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0649999999999999e-06 2024-05-08T08:52:17.832 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.833 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.833 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:17.833 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.073485+0000", 2024-05-08T08:52:17.833 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4320000000000004e-06 2024-05-08T08:52:17.833 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.833 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.833 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:17.833 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.073971+0000", 2024-05-08T08:52:17.833 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00048583099999999999 2024-05-08T08:52:17.833 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.833 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.833 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:17.834 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.074131+0000", 2024-05-08T08:52:17.834 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000159944 2024-05-08T08:52:17.834 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.834 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.834 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 3", 2024-05-08T08:52:17.834 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.074238+0000", 2024-05-08T08:52:17.834 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000107413 2024-05-08T08:52:17.834 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.834 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.834 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:52:17.834 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.078675+0000", 2024-05-08T08:52:17.834 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0044368680000000001 2024-05-08T08:52:17.834 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.835 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.835 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:17.835 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.079235+0000", 2024-05-08T08:52:17.835 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00056026700000000003 2024-05-08T08:52:17.835 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.835 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.835 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:17.835 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.079272+0000", 2024-05-08T08:52:17.835 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6712999999999999e-05 2024-05-08T08:52:17.835 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.835 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.835 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:17.836 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.079313+0000", 2024-05-08T08:52:17.836 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1048999999999999e-05 2024-05-08T08:52:17.836 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.836 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:17.836 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.836 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.836 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.836 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4465.0:1970 3.1c 3:3e22f3f9:::rbd_data.1140cf79ec85.000000000000000b:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-05-08T08:52:17.836 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:17.073497+0000", 2024-05-08T08:52:17.836 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.72056887700000005, 2024-05-08T08:52:17.836 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.005236503, 2024-05-08T08:52:17.836 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:17.836 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:17.836 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:17.836 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:17.837 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4465", 2024-05-08T08:52:17.837 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/390275688", 2024-05-08T08:52:17.837 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1970 2024-05-08T08:52:17.837 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.837 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:17.837 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.837 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:17.837 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.073497+0000", 2024-05-08T08:52:17.837 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:17.837 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.837 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.837 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:17.837 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.073497+0000", 2024-05-08T08:52:17.837 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:17.837 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.838 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.838 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:17.838 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.073497+0000", 2024-05-08T08:52:17.838 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2600000000000002e-07 2024-05-08T08:52:17.838 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.838 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.838 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:17.838 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.073500+0000", 2024-05-08T08:52:17.838 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4389999999999999e-06 2024-05-08T08:52:17.838 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.838 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.838 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:17.838 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.073507+0000", 2024-05-08T08:52:17.838 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4939999999999997e-06 2024-05-08T08:52:17.838 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.839 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.839 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:17.839 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.073513+0000", 2024-05-08T08:52:17.839 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5199999999999997e-06 2024-05-08T08:52:17.839 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.839 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.839 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:17.839 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.073563+0000", 2024-05-08T08:52:17.839 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0073e-05 2024-05-08T08:52:17.839 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.839 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.839 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:17.839 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.073634+0000", 2024-05-08T08:52:17.839 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1453999999999997e-05 2024-05-08T08:52:17.839 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.840 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.840 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 4", 2024-05-08T08:52:17.840 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.073738+0000", 2024-05-08T08:52:17.840 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000103544 2024-05-08T08:52:17.840 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.840 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.840 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:17.840 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.074627+0000", 2024-05-08T08:52:17.840 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00088923300000000002 2024-05-08T08:52:17.840 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.840 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.840 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:52:17.840 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.078650+0000", 2024-05-08T08:52:17.840 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0040231140000000004 2024-05-08T08:52:17.840 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.840 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.841 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:17.841 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.078693+0000", 2024-05-08T08:52:17.841 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2886999999999998e-05 2024-05-08T08:52:17.841 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.841 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.841 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:17.841 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.078733+0000", 2024-05-08T08:52:17.841 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0219000000000002e-05 2024-05-08T08:52:17.841 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.841 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:17.841 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.841 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.841 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.841 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4465.0:1973 3.16 3:696eb650:::rbd_data.1140cf79ec85.000000000000000e:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-05-08T08:52:17.842 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:17.073531+0000", 2024-05-08T08:52:17.842 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.720534542, 2024-05-08T08:52:17.842 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.005921862, 2024-05-08T08:52:17.842 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:17.842 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:17.842 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:17.842 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:17.842 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4465", 2024-05-08T08:52:17.842 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/390275688", 2024-05-08T08:52:17.842 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1973 2024-05-08T08:52:17.842 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.842 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:17.842 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.842 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:17.842 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.073531+0000", 2024-05-08T08:52:17.843 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:17.843 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.843 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.843 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:17.843 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.073531+0000", 2024-05-08T08:52:17.843 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:17.843 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.843 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.843 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:17.843 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.073531+0000", 2024-05-08T08:52:17.843 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8299999999999997e-07 2024-05-08T08:52:17.843 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.843 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.843 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:17.843 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.073534+0000", 2024-05-08T08:52:17.843 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.672e-06 2024-05-08T08:52:17.844 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.844 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.844 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:17.844 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.073541+0000", 2024-05-08T08:52:17.844 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4640000000000003e-06 2024-05-08T08:52:17.844 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.844 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.844 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:17.844 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.073546+0000", 2024-05-08T08:52:17.844 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1950000000000002e-06 2024-05-08T08:52:17.844 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.844 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.844 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:17.844 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.074524+0000", 2024-05-08T08:52:17.844 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00097872299999999992 2024-05-08T08:52:17.844 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.845 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.845 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:17.845 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.074611+0000", 2024-05-08T08:52:17.846 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6675999999999995e-05 2024-05-08T08:52:17.846 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.846 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.846 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 3", 2024-05-08T08:52:17.846 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.074719+0000", 2024-05-08T08:52:17.846 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000107948 2024-05-08T08:52:17.846 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.846 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.846 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:52:17.846 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.078682+0000", 2024-05-08T08:52:17.846 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0039632549999999997 2024-05-08T08:52:17.847 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.847 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.847 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:17.847 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.079362+0000", 2024-05-08T08:52:17.847 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00067955200000000004 2024-05-08T08:52:17.847 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.847 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.847 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:17.847 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.079413+0000", 2024-05-08T08:52:17.847 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1048999999999999e-05 2024-05-08T08:52:17.847 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.847 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.847 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:17.847 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.079453+0000", 2024-05-08T08:52:17.847 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9845000000000002e-05 2024-05-08T08:52:17.848 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.848 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:17.848 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.848 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.848 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.848 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4465.0:2331 3.17 e22/18 3:e8515fe1:::rbd_object_map.1140cf79ec85:head v 22'204, mlcod=22'198)", 2024-05-08T08:52:17.848 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:17.299539+0000", 2024-05-08T08:52:17.848 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.49452614299999997, 2024-05-08T08:52:17.848 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007140497, 2024-05-08T08:52:17.848 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:17.848 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:17.848 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:17.849 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:17.849 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.849 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:17.849 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.299539+0000", 2024-05-08T08:52:17.849 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:17.849 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.849 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.849 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:17.849 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.299539+0000", 2024-05-08T08:52:17.849 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:17.849 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.849 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.849 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:17.849 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.299540+0000", 2024-05-08T08:52:17.849 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6400000000000002e-07 2024-05-08T08:52:17.850 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.850 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.850 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:17.850 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.299548+0000", 2024-05-08T08:52:17.850 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0199999999999994e-06 2024-05-08T08:52:17.850 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.850 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.850 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:17.850 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.299563+0000", 2024-05-08T08:52:17.850 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5512999999999999e-05 2024-05-08T08:52:17.850 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.850 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.850 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:17.850 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.299574+0000", 2024-05-08T08:52:17.851 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0936e-05 2024-05-08T08:52:17.851 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.851 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.851 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:17.851 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.299613+0000", 2024-05-08T08:52:17.851 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8850000000000002e-05 2024-05-08T08:52:17.851 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.851 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.851 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:17.851 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.299632+0000", 2024-05-08T08:52:17.851 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8581999999999999e-05 2024-05-08T08:52:17.851 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.851 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.851 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:17.851 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.306624+0000", 2024-05-08T08:52:17.852 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0069927130000000002 2024-05-08T08:52:17.852 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.852 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.852 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:17.852 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.306680+0000", 2024-05-08T08:52:17.852 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5318999999999997e-05 2024-05-08T08:52:17.852 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.852 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:17.852 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.852 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.852 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.852 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4465.0:2339 3.16 3:6fb4970e:::rbd_data.1140cf79ec85.000000000000001b:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2276864~35328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-05-08T08:52:17.852 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:17.300012+0000", 2024-05-08T08:52:17.852 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.49405358500000002, 2024-05-08T08:52:17.853 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0067541759999999998, 2024-05-08T08:52:17.853 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:17.853 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:17.853 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:17.853 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:17.853 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4465", 2024-05-08T08:52:17.853 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/390275688", 2024-05-08T08:52:17.853 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2339 2024-05-08T08:52:17.853 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.853 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:17.853 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.853 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:17.853 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.300012+0000", 2024-05-08T08:52:17.853 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:17.853 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.854 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.854 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:17.854 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.300012+0000", 2024-05-08T08:52:17.854 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:17.854 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.854 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.854 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:17.854 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.300013+0000", 2024-05-08T08:52:17.854 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2500000000000005e-07 2024-05-08T08:52:17.854 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.854 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.854 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:17.854 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.300037+0000", 2024-05-08T08:52:17.855 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4352999999999999e-05 2024-05-08T08:52:17.855 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.855 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.855 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:17.855 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.300049+0000", 2024-05-08T08:52:17.855 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2445e-05 2024-05-08T08:52:17.855 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.855 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.855 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:17.855 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.300056+0000", 2024-05-08T08:52:17.855 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1920000000000002e-06 2024-05-08T08:52:17.855 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.855 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.855 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:17.855 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.300108+0000", 2024-05-08T08:52:17.856 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1990999999999997e-05 2024-05-08T08:52:17.856 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.856 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.856 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:17.856 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.300156+0000", 2024-05-08T08:52:17.856 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8180000000000003e-05 2024-05-08T08:52:17.856 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.856 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.856 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 3", 2024-05-08T08:52:17.856 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.300233+0000", 2024-05-08T08:52:17.856 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7556000000000001e-05 2024-05-08T08:52:17.856 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.856 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.856 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:17.856 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.302201+0000", 2024-05-08T08:52:17.856 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0019680930000000002 2024-05-08T08:52:17.857 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.857 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.857 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:52:17.857 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.306675+0000", 2024-05-08T08:52:17.857 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0044735280000000001 2024-05-08T08:52:17.857 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.857 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.857 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:17.857 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.306726+0000", 2024-05-08T08:52:17.857 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1462999999999998e-05 2024-05-08T08:52:17.857 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.857 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.857 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:17.857 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.306766+0000", 2024-05-08T08:52:17.857 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9650000000000002e-05 2024-05-08T08:52:17.858 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.858 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:17.858 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.858 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.858 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.858 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4465.0:2332 3.17 e22/18 3:e8515fe1:::rbd_object_map.1140cf79ec85:head v 22'205, mlcod=22'201)", 2024-05-08T08:52:17.858 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:17.300470+0000", 2024-05-08T08:52:17.858 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.49359510299999998, 2024-05-08T08:52:17.858 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0062394210000000002, 2024-05-08T08:52:17.858 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:17.858 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:17.858 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:17.858 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:17.858 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.858 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:17.859 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.300470+0000", 2024-05-08T08:52:17.859 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:17.859 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.859 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.859 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:17.859 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.300470+0000", 2024-05-08T08:52:17.859 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:17.859 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.859 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.859 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:17.859 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.300471+0000", 2024-05-08T08:52:17.859 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3399999999999999e-07 2024-05-08T08:52:17.859 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.859 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.859 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:17.860 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.300478+0000", 2024-05-08T08:52:17.860 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8940000000000004e-06 2024-05-08T08:52:17.860 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.860 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.860 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:17.860 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.300488+0000", 2024-05-08T08:52:17.860 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9450000000000005e-06 2024-05-08T08:52:17.860 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.860 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.860 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:17.860 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.300493+0000", 2024-05-08T08:52:17.860 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6699999999999999e-06 2024-05-08T08:52:17.860 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.860 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.860 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:17.861 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.300540+0000", 2024-05-08T08:52:17.861 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6310000000000002e-05 2024-05-08T08:52:17.861 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.861 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.861 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:17.861 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.300565+0000", 2024-05-08T08:52:17.861 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5001999999999999e-05 2024-05-08T08:52:17.861 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.861 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.861 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:17.861 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.306683+0000", 2024-05-08T08:52:17.861 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0061178589999999998 2024-05-08T08:52:17.861 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.861 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.862 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:17.862 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.306710+0000", 2024-05-08T08:52:17.862 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7206999999999999e-05 2024-05-08T08:52:17.862 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.862 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:17.862 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.862 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.862 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.862 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4465.0:2333 3.17 e22/18 3:e8515fe1:::rbd_object_map.1140cf79ec85:head v 22'206, mlcod=22'201)", 2024-05-08T08:52:17.862 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:17.301145+0000", 2024-05-08T08:52:17.862 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.49292061199999998, 2024-05-08T08:52:17.862 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0055926830000000002, 2024-05-08T08:52:17.862 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:17.862 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:17.863 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:17.863 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:17.863 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.863 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:17.863 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.301145+0000", 2024-05-08T08:52:17.863 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:17.863 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.863 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.863 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:17.863 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.301145+0000", 2024-05-08T08:52:17.863 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:17.863 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.863 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.863 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:17.863 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.301145+0000", 2024-05-08T08:52:17.864 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3700000000000001e-07 2024-05-08T08:52:17.864 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.864 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.864 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:17.864 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.301148+0000", 2024-05-08T08:52:17.864 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0580000000000002e-06 2024-05-08T08:52:17.864 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.864 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.864 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:17.864 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.301156+0000", 2024-05-08T08:52:17.864 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9759999999999995e-06 2024-05-08T08:52:17.864 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.864 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.864 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:17.864 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.301161+0000", 2024-05-08T08:52:17.865 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.006e-06 2024-05-08T08:52:17.865 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.865 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.865 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:17.865 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.301191+0000", 2024-05-08T08:52:17.865 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0001999999999998e-05 2024-05-08T08:52:17.865 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.865 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.865 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:17.865 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.301203+0000", 2024-05-08T08:52:17.865 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1443e-05 2024-05-08T08:52:17.865 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.865 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.865 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:17.865 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.306711+0000", 2024-05-08T08:52:17.865 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0055082580000000003 2024-05-08T08:52:17.866 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.866 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.866 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:17.866 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.306738+0000", 2024-05-08T08:52:17.866 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6503e-05 2024-05-08T08:52:17.866 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.866 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:17.866 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.866 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.866 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.866 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4465.0:2340 3.13 e22/18 3:ce9cf5f5:::rbd_data.1140cf79ec85.0000000000000025:head v 22'16, mlcod=22'13)", 2024-05-08T08:52:17.866 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:17.307873+0000", 2024-05-08T08:52:17.866 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.48619251400000002, 2024-05-08T08:52:17.866 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0054063779999999999, 2024-05-08T08:52:17.867 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:17.867 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:17.867 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:17.867 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:17.867 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.867 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:17.867 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.307873+0000", 2024-05-08T08:52:17.867 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:17.867 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.867 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.868 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:17.868 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.307873+0000", 2024-05-08T08:52:17.868 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:17.868 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.868 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.868 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:17.868 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.307874+0000", 2024-05-08T08:52:17.868 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2300000000000001e-07 2024-05-08T08:52:17.868 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.868 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.868 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:17.868 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.307901+0000", 2024-05-08T08:52:17.868 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7239000000000001e-05 2024-05-08T08:52:17.868 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.868 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.869 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:17.869 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.307912+0000", 2024-05-08T08:52:17.869 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0839000000000001e-05 2024-05-08T08:52:17.869 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.869 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.869 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:17.869 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.307918+0000", 2024-05-08T08:52:17.869 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5930000000000002e-06 2024-05-08T08:52:17.869 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.869 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.869 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:17.869 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.308003+0000", 2024-05-08T08:52:17.869 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4485000000000002e-05 2024-05-08T08:52:17.869 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.869 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.870 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:17.870 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.308028+0000", 2024-05-08T08:52:17.870 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5429e-05 2024-05-08T08:52:17.870 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.870 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.870 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:17.870 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.313251+0000", 2024-05-08T08:52:17.870 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0052224200000000002 2024-05-08T08:52:17.870 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.870 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.870 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:17.870 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.313279+0000", 2024-05-08T08:52:17.870 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8750000000000001e-05 2024-05-08T08:52:17.870 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.871 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:17.871 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.871 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.871 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.871 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4465.0:2350 3.17 e22/18 3:e8515fe1:::rbd_object_map.1140cf79ec85:head v 22'208, mlcod=22'205)", 2024-05-08T08:52:17.871 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:17.310572+0000", 2024-05-08T08:52:17.871 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.48349382899999999, 2024-05-08T08:52:17.871 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0053539829999999997, 2024-05-08T08:52:17.871 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:17.871 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:17.871 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:17.871 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:17.871 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.871 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:17.871 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.310572+0000", 2024-05-08T08:52:17.872 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:17.872 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.872 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.872 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:17.872 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.310572+0000", 2024-05-08T08:52:17.872 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:17.872 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.872 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.872 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:17.872 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.310572+0000", 2024-05-08T08:52:17.872 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9200000000000001e-07 2024-05-08T08:52:17.872 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.872 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.872 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:17.872 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.310581+0000", 2024-05-08T08:52:17.873 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4290000000000001e-06 2024-05-08T08:52:17.873 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.873 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.873 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:17.873 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.310590+0000", 2024-05-08T08:52:17.873 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4069999999999999e-06 2024-05-08T08:52:17.873 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.873 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.873 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:17.873 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.310597+0000", 2024-05-08T08:52:17.873 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0280000000000003e-06 2024-05-08T08:52:17.873 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.873 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.873 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:17.873 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.310638+0000", 2024-05-08T08:52:17.873 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1298999999999999e-05 2024-05-08T08:52:17.874 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.874 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.874 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:17.874 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.310661+0000", 2024-05-08T08:52:17.874 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2277e-05 2024-05-08T08:52:17.874 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.874 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.874 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:17.874 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.315896+0000", 2024-05-08T08:52:17.874 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0052348589999999997 2024-05-08T08:52:17.874 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.874 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.874 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:17.874 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.315926+0000", 2024-05-08T08:52:17.875 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0091999999999999e-05 2024-05-08T08:52:17.875 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.875 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:17.875 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.875 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.875 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.875 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4465.0:2894 3.16 3:6d9a0c0e:::rbd_data.1140cf79ec85.0000000000000007:head [set-alloc-hint object_size 4194304 write_size 4194304,write 421888~53248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-05-08T08:52:17.875 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:17.556028+0000", 2024-05-08T08:52:17.875 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.23803695899999999, 2024-05-08T08:52:17.875 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0066881880000000003, 2024-05-08T08:52:17.875 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:17.875 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:17.875 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:17.875 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:17.875 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4465", 2024-05-08T08:52:17.876 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/390275688", 2024-05-08T08:52:17.876 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2894 2024-05-08T08:52:17.876 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.876 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:17.876 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.876 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:17.876 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.556028+0000", 2024-05-08T08:52:17.876 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:17.876 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.876 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.876 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:17.876 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.556028+0000", 2024-05-08T08:52:17.876 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:17.876 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.876 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.877 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:17.877 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.556029+0000", 2024-05-08T08:52:17.877 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5600000000000005e-07 2024-05-08T08:52:17.877 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.877 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.877 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:17.877 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.556086+0000", 2024-05-08T08:52:17.877 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6709000000000002e-05 2024-05-08T08:52:17.877 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.877 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.877 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:17.877 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.556097+0000", 2024-05-08T08:52:17.877 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0672999999999999e-05 2024-05-08T08:52:17.877 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.877 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.878 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:17.878 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.556103+0000", 2024-05-08T08:52:17.878 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.72e-06 2024-05-08T08:52:17.878 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.878 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.878 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:17.878 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.556179+0000", 2024-05-08T08:52:17.878 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5698000000000006e-05 2024-05-08T08:52:17.878 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.878 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.878 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:17.878 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.556260+0000", 2024-05-08T08:52:17.878 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0869000000000001e-05 2024-05-08T08:52:17.878 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.878 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.879 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 3", 2024-05-08T08:52:17.879 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.556401+0000", 2024-05-08T08:52:17.879 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014076900000000001 2024-05-08T08:52:17.879 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.879 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.879 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:52:17.879 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.562614+0000", 2024-05-08T08:52:17.879 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0062128579999999999 2024-05-08T08:52:17.879 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.879 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.879 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:17.879 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.562658+0000", 2024-05-08T08:52:17.879 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4656000000000001e-05 2024-05-08T08:52:17.879 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.879 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.880 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:17.880 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.562691+0000", 2024-05-08T08:52:17.880 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3132000000000001e-05 2024-05-08T08:52:17.880 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.880 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.880 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:17.880 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.562717+0000", 2024-05-08T08:52:17.880 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5347999999999999e-05 2024-05-08T08:52:17.880 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.880 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:17.880 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.880 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.880 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.880 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4465.0:3095 3.17 e22/18 3:ec85a88a:::rbd_data.1140cf79ec85.000000000000000c:head v 22'263, mlcod=22'261)", 2024-05-08T08:52:17.880 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:17.675617+0000", 2024-05-08T08:52:17.881 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.11844811700000001, 2024-05-08T08:52:17.881 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00502558, 2024-05-08T08:52:17.881 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:17.881 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:17.881 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:17.881 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:17.881 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.881 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:17.881 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.675617+0000", 2024-05-08T08:52:17.881 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:17.881 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.881 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.881 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:17.881 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.675617+0000", 2024-05-08T08:52:17.881 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:17.882 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.882 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.882 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:17.882 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.675618+0000", 2024-05-08T08:52:17.882 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0999999999999998e-07 2024-05-08T08:52:17.882 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.882 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.882 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:17.882 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.675624+0000", 2024-05-08T08:52:17.882 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6590000000000004e-06 2024-05-08T08:52:17.882 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.883 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.883 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:17.883 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.675634+0000", 2024-05-08T08:52:17.883 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0148e-05 2024-05-08T08:52:17.883 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.883 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.883 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:17.883 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.675641+0000", 2024-05-08T08:52:17.883 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7200000000000006e-06 2024-05-08T08:52:17.883 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.883 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.883 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:17.883 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.675706+0000", 2024-05-08T08:52:17.883 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4270000000000006e-05 2024-05-08T08:52:17.883 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.883 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.884 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:17.884 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.675725+0000", 2024-05-08T08:52:17.884 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9669000000000001e-05 2024-05-08T08:52:17.884 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.884 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.884 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:17.884 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.680611+0000", 2024-05-08T08:52:17.884 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00488595 2024-05-08T08:52:17.884 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.884 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.884 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:17.884 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.680643+0000", 2024-05-08T08:52:17.884 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1554000000000003e-05 2024-05-08T08:52:17.884 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.884 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:17.885 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.885 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.885 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.885 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4465.0:3105 3.13 e22/18 3:ce9cf5f5:::rbd_data.1140cf79ec85.0000000000000025:head v 22'25, mlcod=22'23)", 2024-05-08T08:52:17.885 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:17.676069+0000", 2024-05-08T08:52:17.885 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.11799683800000001, 2024-05-08T08:52:17.885 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0045878270000000001, 2024-05-08T08:52:17.885 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:17.885 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:17.885 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:17.885 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:17.885 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.885 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:17.885 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.676069+0000", 2024-05-08T08:52:17.885 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:17.886 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.886 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.886 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:17.886 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.676069+0000", 2024-05-08T08:52:17.886 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:17.886 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.886 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.886 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:17.886 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.676069+0000", 2024-05-08T08:52:17.886 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7700000000000004e-07 2024-05-08T08:52:17.886 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.886 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.886 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:17.886 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.676075+0000", 2024-05-08T08:52:17.886 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1800000000000001e-06 2024-05-08T08:52:17.887 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.887 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.887 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:17.887 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.676089+0000", 2024-05-08T08:52:17.887 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3301e-05 2024-05-08T08:52:17.887 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.887 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.887 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:17.887 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.676097+0000", 2024-05-08T08:52:17.887 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3429999999999996e-06 2024-05-08T08:52:17.887 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.887 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.887 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:17.887 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.676154+0000", 2024-05-08T08:52:17.887 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6498e-05 2024-05-08T08:52:17.888 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.888 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.888 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:17.888 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.676174+0000", 2024-05-08T08:52:17.888 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0406000000000001e-05 2024-05-08T08:52:17.888 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.888 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.888 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:17.888 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.680618+0000", 2024-05-08T08:52:17.888 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0044441039999999999 2024-05-08T08:52:17.888 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.888 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.888 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:17.888 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.680656+0000", 2024-05-08T08:52:17.888 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8318000000000001e-05 2024-05-08T08:52:17.889 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.889 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:17.889 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.889 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.889 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.889 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4465.0:3226 3.1c 3:3f66ab46:::rbd_data.1140cf79ec85.0000000000000005:head [write 1777664~36864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-05-08T08:52:17.889 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:17.741844+0000", 2024-05-08T08:52:17.889 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.052221616999999998, 2024-05-08T08:52:17.889 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0052744749999999998, 2024-05-08T08:52:17.889 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:17.889 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:17.889 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:17.889 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:17.889 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4465", 2024-05-08T08:52:17.889 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/390275688", 2024-05-08T08:52:17.890 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 3226 2024-05-08T08:52:17.890 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.890 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:17.890 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.890 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:17.890 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.741844+0000", 2024-05-08T08:52:17.890 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:17.890 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.890 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.890 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:17.890 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.741844+0000", 2024-05-08T08:52:17.890 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:17.891 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.891 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.891 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:17.891 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.741844+0000", 2024-05-08T08:52:17.891 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6499999999999999e-07 2024-05-08T08:52:17.891 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.891 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.891 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:17.891 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.741866+0000", 2024-05-08T08:52:17.891 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1597999999999999e-05 2024-05-08T08:52:17.891 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.891 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.891 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:17.891 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.741874+0000", 2024-05-08T08:52:17.891 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8650000000000001e-06 2024-05-08T08:52:17.892 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.892 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.892 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:17.892 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.741880+0000", 2024-05-08T08:52:17.892 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6779999999999997e-06 2024-05-08T08:52:17.892 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.892 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.892 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:17.892 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.741913+0000", 2024-05-08T08:52:17.892 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3056999999999999e-05 2024-05-08T08:52:17.892 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.892 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.892 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:17.892 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.741966+0000", 2024-05-08T08:52:17.892 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3770000000000002e-05 2024-05-08T08:52:17.892 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.893 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.893 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 4", 2024-05-08T08:52:17.893 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.742045+0000", 2024-05-08T08:52:17.893 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8238999999999994e-05 2024-05-08T08:52:17.893 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.893 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.893 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:52:17.893 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.744304+0000", 2024-05-08T08:52:17.893 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002259474 2024-05-08T08:52:17.893 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.893 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.893 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:17.893 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.747063+0000", 2024-05-08T08:52:17.893 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0027583360000000001 2024-05-08T08:52:17.893 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.894 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.894 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:17.894 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.747094+0000", 2024-05-08T08:52:17.894 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.154e-05 2024-05-08T08:52:17.894 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:17.894 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:17.894 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:17.894 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.747118+0000", 2024-05-08T08:52:17.894 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4253e-05 2024-05-08T08:52:17.894 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.894 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:17.894 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.894 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:17.894 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:17.894 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:52:17.896 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:52:17.899+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:52:17.896 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:52:17.997 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:52:18.000+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:52:18.011 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:52:18.012 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:52:18.012 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:52:18.012 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:52:18.021 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:52:18.104 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:52:18.108+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:52:18.173 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:52:18.173 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:52:18.173 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:52:18.173 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:52:18.173 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:52:18.183 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:52:18.199 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:18.202+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:18.275 INFO:teuthology.orchestra.run.smithi196.stdout:542 clone 2 order 19 su 16384 sc 11 2024-05-08T08:52:18.300 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:52:18.303+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:52:18.339 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:52:18.339 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:52:18.339 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:52:18.339 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:52:18.339 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.339 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4465.0:1320 3.17 3:e8515fe1:::rbd_object_map.1140cf79ec85:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-05-08T08:52:18.339 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:16.752452+0000", 2024-05-08T08:52:18.340 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 1.58842653, 2024-05-08T08:52:18.340 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.007163287, 2024-05-08T08:52:18.340 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:18.340 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:18.340 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:18.340 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:18.340 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4465", 2024-05-08T08:52:18.340 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/390275688", 2024-05-08T08:52:18.340 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1320 2024-05-08T08:52:18.340 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.340 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:18.340 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.340 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:18.340 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.752452+0000", 2024-05-08T08:52:18.341 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:18.341 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.341 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.341 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:18.341 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.752452+0000", 2024-05-08T08:52:18.341 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:18.341 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.341 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.341 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:18.341 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.752452+0000", 2024-05-08T08:52:18.341 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7700000000000004e-07 2024-05-08T08:52:18.341 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.341 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.341 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:18.342 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.752456+0000", 2024-05-08T08:52:18.342 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5949999999999999e-06 2024-05-08T08:52:18.342 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.342 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.342 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:18.342 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.752465+0000", 2024-05-08T08:52:18.342 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.9469999999999992e-06 2024-05-08T08:52:18.342 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.342 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.342 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:18.342 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.752469+0000", 2024-05-08T08:52:18.342 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3270000000000002e-06 2024-05-08T08:52:18.342 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.342 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.343 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:18.343 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.753378+0000", 2024-05-08T08:52:18.343 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000909132 2024-05-08T08:52:18.343 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.343 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.343 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:18.343 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.753474+0000", 2024-05-08T08:52:18.343 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5210000000000005e-05 2024-05-08T08:52:18.343 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.343 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.343 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 5", 2024-05-08T08:52:18.343 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.753983+0000", 2024-05-08T08:52:18.343 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00050976000000000005 2024-05-08T08:52:18.344 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.344 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.344 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:18.344 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.757203+0000", 2024-05-08T08:52:18.344 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0032196799999999999 2024-05-08T08:52:18.344 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.344 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.344 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:18.344 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.759560+0000", 2024-05-08T08:52:18.344 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.002356679 2024-05-08T08:52:18.344 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.344 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.344 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:18.345 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.759591+0000", 2024-05-08T08:52:18.345 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0879e-05 2024-05-08T08:52:18.345 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.345 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.345 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:18.345 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.759615+0000", 2024-05-08T08:52:18.345 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4501000000000001e-05 2024-05-08T08:52:18.345 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:18.345 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:18.345 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:18.345 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.345 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.346 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4465.0:1321 3.17 3:e8515fe1:::rbd_object_map.1140cf79ec85:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-05-08T08:52:18.346 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:16.752563+0000", 2024-05-08T08:52:18.346 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 1.588315785, 2024-05-08T08:52:18.346 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0072053339999999999, 2024-05-08T08:52:18.346 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:18.346 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:18.346 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:18.346 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:18.346 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4465", 2024-05-08T08:52:18.346 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/390275688", 2024-05-08T08:52:18.346 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1321 2024-05-08T08:52:18.346 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.347 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:18.347 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.347 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:18.347 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.752563+0000", 2024-05-08T08:52:18.347 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:18.347 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.347 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.347 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:18.347 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.752563+0000", 2024-05-08T08:52:18.347 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:18.347 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.347 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.347 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:18.348 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.752563+0000", 2024-05-08T08:52:18.348 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.15e-07 2024-05-08T08:52:18.348 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.348 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.348 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:18.348 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.752566+0000", 2024-05-08T08:52:18.348 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4899999999999999e-06 2024-05-08T08:52:18.348 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.348 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.348 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:18.348 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.752572+0000", 2024-05-08T08:52:18.348 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5010000000000001e-06 2024-05-08T08:52:18.348 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.348 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.349 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:18.349 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.752575+0000", 2024-05-08T08:52:18.349 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.2339999999999999e-06 2024-05-08T08:52:18.349 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.349 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.349 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:18.349 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.754655+0000", 2024-05-08T08:52:18.349 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0020801909999999999 2024-05-08T08:52:18.349 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.349 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.349 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:18.349 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.754726+0000", 2024-05-08T08:52:18.349 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0980000000000001e-05 2024-05-08T08:52:18.350 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.350 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.350 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 5", 2024-05-08T08:52:18.350 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.754997+0000", 2024-05-08T08:52:18.350 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000270664 2024-05-08T08:52:18.350 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.350 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.350 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:18.350 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.757211+0000", 2024-05-08T08:52:18.350 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.002213867 2024-05-08T08:52:18.350 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.350 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.350 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:18.351 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.759663+0000", 2024-05-08T08:52:18.351 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0024516870000000001 2024-05-08T08:52:18.351 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.351 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.351 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:18.351 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.759722+0000", 2024-05-08T08:52:18.351 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9524000000000001e-05 2024-05-08T08:52:18.351 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.351 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.351 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:18.351 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.759768+0000", 2024-05-08T08:52:18.351 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5781000000000001e-05 2024-05-08T08:52:18.351 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:18.352 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:18.352 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:18.352 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.352 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.352 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4465.0:1322 3.17 3:e8515fe1:::rbd_object_map.1140cf79ec85:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-05-08T08:52:18.352 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:16.752649+0000", 2024-05-08T08:52:18.352 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 1.5882295119999998, 2024-05-08T08:52:18.352 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0074330530000000002, 2024-05-08T08:52:18.352 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:18.352 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:18.352 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:18.352 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:18.352 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4465", 2024-05-08T08:52:18.352 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/390275688", 2024-05-08T08:52:18.353 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1322 2024-05-08T08:52:18.353 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.353 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:18.353 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.353 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:18.353 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.752649+0000", 2024-05-08T08:52:18.353 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:18.353 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.353 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.353 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:18.353 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.752649+0000", 2024-05-08T08:52:18.353 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:18.354 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.354 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.354 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:18.354 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.752649+0000", 2024-05-08T08:52:18.354 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0800000000000001e-07 2024-05-08T08:52:18.354 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.354 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.354 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:18.354 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.752651+0000", 2024-05-08T08:52:18.354 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0140000000000001e-06 2024-05-08T08:52:18.354 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.354 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.354 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:18.355 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.752656+0000", 2024-05-08T08:52:18.355 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4950000000000002e-06 2024-05-08T08:52:18.355 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.355 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.355 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:18.355 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.752659+0000", 2024-05-08T08:52:18.355 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9299999999999999e-06 2024-05-08T08:52:18.355 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.355 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.355 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:18.355 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.755409+0000", 2024-05-08T08:52:18.355 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0027508189999999998 2024-05-08T08:52:18.355 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.355 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.356 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:18.356 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.755498+0000", 2024-05-08T08:52:18.356 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.9012999999999996e-05 2024-05-08T08:52:18.356 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.356 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.356 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 5", 2024-05-08T08:52:18.356 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.755977+0000", 2024-05-08T08:52:18.356 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00047824800000000002 2024-05-08T08:52:18.356 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.356 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.356 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:18.356 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.759140+0000", 2024-05-08T08:52:18.356 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0031629090000000002 2024-05-08T08:52:18.356 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.357 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.357 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:18.357 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.759982+0000", 2024-05-08T08:52:18.357 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00084187900000000002 2024-05-08T08:52:18.357 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.357 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.357 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:18.357 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.760037+0000", 2024-05-08T08:52:18.357 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5356000000000003e-05 2024-05-08T08:52:18.357 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.357 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.357 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:18.358 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.760082+0000", 2024-05-08T08:52:18.358 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5082e-05 2024-05-08T08:52:18.358 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:18.358 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:18.358 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:18.358 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.358 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.358 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4465.0:1323 3.17 3:e8515fe1:::rbd_object_map.1140cf79ec85:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-05-08T08:52:18.358 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:16.752964+0000", 2024-05-08T08:52:18.358 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 1.5879147360000001, 2024-05-08T08:52:18.358 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0072806950000000002, 2024-05-08T08:52:18.358 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:18.358 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:18.358 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:18.359 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:18.359 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4465", 2024-05-08T08:52:18.359 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/390275688", 2024-05-08T08:52:18.359 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1323 2024-05-08T08:52:18.359 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.359 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:18.359 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.359 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:18.359 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.752964+0000", 2024-05-08T08:52:18.359 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:18.359 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.359 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.359 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:18.360 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.752964+0000", 2024-05-08T08:52:18.360 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:18.360 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.360 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.360 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:18.360 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.752964+0000", 2024-05-08T08:52:18.360 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6000000000000004e-07 2024-05-08T08:52:18.360 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.360 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.360 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:18.360 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.752976+0000", 2024-05-08T08:52:18.360 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1357e-05 2024-05-08T08:52:18.360 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.360 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.361 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:18.361 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.752982+0000", 2024-05-08T08:52:18.361 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3400000000000003e-06 2024-05-08T08:52:18.361 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.361 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.361 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:18.361 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.752985+0000", 2024-05-08T08:52:18.361 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.399e-06 2024-05-08T08:52:18.361 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.361 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.361 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:18.361 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.756450+0000", 2024-05-08T08:52:18.361 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00346482 2024-05-08T08:52:18.361 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.361 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.362 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:18.362 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.756499+0000", 2024-05-08T08:52:18.362 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8674999999999997e-05 2024-05-08T08:52:18.362 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.362 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.362 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 5", 2024-05-08T08:52:18.362 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.756693+0000", 2024-05-08T08:52:18.362 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00019454499999999999 2024-05-08T08:52:18.362 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.362 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.362 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:18.362 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.759147+0000", 2024-05-08T08:52:18.362 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0024532040000000001 2024-05-08T08:52:18.362 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.363 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.363 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:18.363 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.760144+0000", 2024-05-08T08:52:18.363 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00099782899999999999 2024-05-08T08:52:18.363 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.363 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.363 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:18.363 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.760200+0000", 2024-05-08T08:52:18.363 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5928999999999999e-05 2024-05-08T08:52:18.363 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.363 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.363 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:18.363 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.760244+0000", 2024-05-08T08:52:18.363 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4036999999999999e-05 2024-05-08T08:52:18.364 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:18.364 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:18.364 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:18.364 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.364 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.364 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4465.0:1326 3.17 3:e8515fe1:::rbd_object_map.1140cf79ec85:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-05-08T08:52:18.364 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:16.754655+0000", 2024-05-08T08:52:18.364 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 1.586223757, 2024-05-08T08:52:18.364 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0063295690000000002, 2024-05-08T08:52:18.364 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:18.364 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:18.364 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:18.364 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:18.364 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4465", 2024-05-08T08:52:18.364 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/390275688", 2024-05-08T08:52:18.365 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1326 2024-05-08T08:52:18.365 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.365 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:18.365 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.365 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:18.365 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.754655+0000", 2024-05-08T08:52:18.365 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:18.365 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.365 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.365 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:18.365 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.754655+0000", 2024-05-08T08:52:18.365 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:18.365 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.365 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.365 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:18.366 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.754655+0000", 2024-05-08T08:52:18.366 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1900000000000002e-07 2024-05-08T08:52:18.366 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.366 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.366 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:18.366 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.754663+0000", 2024-05-08T08:52:18.366 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1720000000000001e-06 2024-05-08T08:52:18.366 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.366 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.366 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:18.366 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.754672+0000", 2024-05-08T08:52:18.366 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.9180000000000006e-06 2024-05-08T08:52:18.366 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.367 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.367 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:18.367 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.754679+0000", 2024-05-08T08:52:18.367 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3330000000000002e-06 2024-05-08T08:52:18.367 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.367 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.367 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:18.367 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.758148+0000", 2024-05-08T08:52:18.367 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0034696789999999998 2024-05-08T08:52:18.367 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.367 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.367 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:18.367 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.758254+0000", 2024-05-08T08:52:18.367 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010602 2024-05-08T08:52:18.368 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.368 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.368 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 5", 2024-05-08T08:52:18.368 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.758714+0000", 2024-05-08T08:52:18.368 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00045989499999999999 2024-05-08T08:52:18.368 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.368 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.368 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:18.368 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.760462+0000", 2024-05-08T08:52:18.368 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001747136 2024-05-08T08:52:18.368 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.368 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.368 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:18.368 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.760875+0000", 2024-05-08T08:52:18.369 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00041329600000000001 2024-05-08T08:52:18.369 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.369 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.369 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:18.369 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.760932+0000", 2024-05-08T08:52:18.369 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7073000000000001e-05 2024-05-08T08:52:18.369 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.369 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.369 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:18.369 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.760984+0000", 2024-05-08T08:52:18.369 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2327999999999998e-05 2024-05-08T08:52:18.369 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:18.369 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:18.369 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:18.369 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.370 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.370 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4465.0:1552 3.17 3:e8515fe1:::rbd_object_map.1140cf79ec85:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-05-08T08:52:18.370 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:16.873814+0000", 2024-05-08T08:52:18.370 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 1.4670642329999999, 2024-05-08T08:52:18.370 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0067091709999999999, 2024-05-08T08:52:18.370 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:18.370 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:18.370 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:18.370 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:18.370 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4465", 2024-05-08T08:52:18.370 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/390275688", 2024-05-08T08:52:18.370 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1552 2024-05-08T08:52:18.370 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.370 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:18.371 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.371 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:18.371 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.873814+0000", 2024-05-08T08:52:18.371 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:18.371 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.371 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.371 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:18.371 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.873814+0000", 2024-05-08T08:52:18.371 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:18.371 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.371 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.371 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:18.371 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.873815+0000", 2024-05-08T08:52:18.371 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3900000000000005e-07 2024-05-08T08:52:18.372 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.372 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.372 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:18.372 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.873818+0000", 2024-05-08T08:52:18.372 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5360000000000001e-06 2024-05-08T08:52:18.372 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.372 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.372 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:18.372 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.873827+0000", 2024-05-08T08:52:18.372 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.8040000000000007e-06 2024-05-08T08:52:18.372 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.372 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.372 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:18.372 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.873833+0000", 2024-05-08T08:52:18.373 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9619999999999998e-06 2024-05-08T08:52:18.373 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.373 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.373 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:18.373 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.874439+0000", 2024-05-08T08:52:18.373 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00060590000000000004 2024-05-08T08:52:18.373 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.373 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.373 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:18.373 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.874499+0000", 2024-05-08T08:52:18.373 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0275999999999997e-05 2024-05-08T08:52:18.373 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.373 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.373 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 5", 2024-05-08T08:52:18.374 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.874738+0000", 2024-05-08T08:52:18.374 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000238688 2024-05-08T08:52:18.374 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.374 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.374 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:18.374 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.877423+0000", 2024-05-08T08:52:18.374 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0026849320000000001 2024-05-08T08:52:18.374 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.374 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.374 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:18.374 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.880418+0000", 2024-05-08T08:52:18.374 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0029950910000000001 2024-05-08T08:52:18.374 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.374 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.375 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:18.375 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.880476+0000", 2024-05-08T08:52:18.375 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7599999999999997e-05 2024-05-08T08:52:18.375 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.375 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.375 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:18.375 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.880523+0000", 2024-05-08T08:52:18.375 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7843000000000003e-05 2024-05-08T08:52:18.375 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:18.375 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:18.375 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:18.375 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.375 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.375 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4465.0:1553 3.17 3:e8515fe1:::rbd_object_map.1140cf79ec85:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-05-08T08:52:18.375 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:16.873846+0000", 2024-05-08T08:52:18.376 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 1.4670328079999999, 2024-05-08T08:52:18.376 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0067676710000000003, 2024-05-08T08:52:18.376 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:18.376 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:18.376 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:18.376 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:18.376 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4465", 2024-05-08T08:52:18.376 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/390275688", 2024-05-08T08:52:18.376 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1553 2024-05-08T08:52:18.376 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.376 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:18.376 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.376 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:18.376 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.873846+0000", 2024-05-08T08:52:18.376 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:18.377 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.377 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.377 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:18.377 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.873846+0000", 2024-05-08T08:52:18.377 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:18.377 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.377 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.377 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:18.377 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.873846+0000", 2024-05-08T08:52:18.377 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.13e-07 2024-05-08T08:52:18.377 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.377 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.377 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:18.377 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.873850+0000", 2024-05-08T08:52:18.378 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7400000000000002e-06 2024-05-08T08:52:18.378 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.378 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.378 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:18.378 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.873872+0000", 2024-05-08T08:52:18.378 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2147e-05 2024-05-08T08:52:18.378 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.378 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.378 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:18.378 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.873878+0000", 2024-05-08T08:52:18.378 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3029999999999996e-06 2024-05-08T08:52:18.378 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.378 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.378 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:18.379 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.875034+0000", 2024-05-08T08:52:18.379 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001156058 2024-05-08T08:52:18.379 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.379 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.379 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:18.379 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.875083+0000", 2024-05-08T08:52:18.379 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8143999999999998e-05 2024-05-08T08:52:18.379 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.379 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.379 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 5", 2024-05-08T08:52:18.379 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.875413+0000", 2024-05-08T08:52:18.379 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000330396 2024-05-08T08:52:18.379 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.379 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.379 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:18.380 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.877436+0000", 2024-05-08T08:52:18.380 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0020231099999999998 2024-05-08T08:52:18.380 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.380 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.380 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:18.380 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.880558+0000", 2024-05-08T08:52:18.380 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.003122077 2024-05-08T08:52:18.380 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.380 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.380 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:18.380 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.880589+0000", 2024-05-08T08:52:18.380 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1075000000000003e-05 2024-05-08T08:52:18.380 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.380 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.381 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:18.381 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.880613+0000", 2024-05-08T08:52:18.381 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4008000000000001e-05 2024-05-08T08:52:18.381 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:18.381 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:18.381 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:18.381 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.381 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.381 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4465.0:1554 3.17 3:e8515fe1:::rbd_object_map.1140cf79ec85:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-05-08T08:52:18.381 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:16.873892+0000", 2024-05-08T08:52:18.381 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 1.466986634, 2024-05-08T08:52:18.381 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0068799799999999999, 2024-05-08T08:52:18.381 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:18.381 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:18.382 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:18.382 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:18.382 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4465", 2024-05-08T08:52:18.382 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/390275688", 2024-05-08T08:52:18.382 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1554 2024-05-08T08:52:18.382 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.382 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:18.382 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.382 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:18.382 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.873892+0000", 2024-05-08T08:52:18.382 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:18.382 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.382 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.382 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:18.382 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.873892+0000", 2024-05-08T08:52:18.383 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:18.383 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.383 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.383 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:18.383 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.873892+0000", 2024-05-08T08:52:18.383 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9399999999999995e-07 2024-05-08T08:52:18.383 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.383 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.383 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:18.383 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.873897+0000", 2024-05-08T08:52:18.383 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3649999999999997e-06 2024-05-08T08:52:18.383 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.383 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.383 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:18.384 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.873904+0000", 2024-05-08T08:52:18.384 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.148e-06 2024-05-08T08:52:18.384 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.384 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.384 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:18.384 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.873909+0000", 2024-05-08T08:52:18.384 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0490000000000003e-06 2024-05-08T08:52:18.384 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.384 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.384 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:18.384 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.875746+0000", 2024-05-08T08:52:18.384 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001836978 2024-05-08T08:52:18.384 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.384 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.385 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:18.385 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.875846+0000", 2024-05-08T08:52:18.385 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000100104 2024-05-08T08:52:18.385 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.385 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.385 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 5", 2024-05-08T08:52:18.385 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.876251+0000", 2024-05-08T08:52:18.385 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00040462199999999999 2024-05-08T08:52:18.385 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.385 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.385 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:18.385 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.877444+0000", 2024-05-08T08:52:18.385 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001193394 2024-05-08T08:52:18.385 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.386 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.386 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:18.386 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.880671+0000", 2024-05-08T08:52:18.386 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.003226989 2024-05-08T08:52:18.386 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.386 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.386 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:18.386 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.880728+0000", 2024-05-08T08:52:18.386 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6994999999999999e-05 2024-05-08T08:52:18.386 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.386 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.386 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:18.386 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.880772+0000", 2024-05-08T08:52:18.386 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3841999999999998e-05 2024-05-08T08:52:18.386 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:18.387 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:18.387 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:18.387 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.387 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.387 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4465.0:1559 3.17 3:e8515fe1:::rbd_object_map.1140cf79ec85:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-05-08T08:52:18.387 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:16.875204+0000", 2024-05-08T08:52:18.387 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 1.4656744349999999, 2024-05-08T08:52:18.387 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0081227499999999998, 2024-05-08T08:52:18.387 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:18.387 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:18.387 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:18.387 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:18.387 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4465", 2024-05-08T08:52:18.387 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/390275688", 2024-05-08T08:52:18.388 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1559 2024-05-08T08:52:18.388 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.388 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:18.388 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.388 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:18.388 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.875204+0000", 2024-05-08T08:52:18.388 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:18.388 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.388 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.388 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:18.388 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.875204+0000", 2024-05-08T08:52:18.388 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:18.388 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.388 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.388 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:18.389 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.875205+0000", 2024-05-08T08:52:18.389 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4200000000000004e-07 2024-05-08T08:52:18.389 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.389 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.389 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:18.389 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.875214+0000", 2024-05-08T08:52:18.389 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.6300000000000004e-06 2024-05-08T08:52:18.389 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.389 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.389 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:18.389 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.875226+0000", 2024-05-08T08:52:18.389 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2072999999999999e-05 2024-05-08T08:52:18.389 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.389 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.390 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:18.390 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.875234+0000", 2024-05-08T08:52:18.390 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.5250000000000005e-06 2024-05-08T08:52:18.390 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.390 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.390 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:18.390 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.877473+0000", 2024-05-08T08:52:18.390 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0022386910000000001 2024-05-08T08:52:18.390 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.390 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.390 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:18.390 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.877557+0000", 2024-05-08T08:52:18.390 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4305e-05 2024-05-08T08:52:18.390 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.390 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.391 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 5", 2024-05-08T08:52:18.391 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.877913+0000", 2024-05-08T08:52:18.391 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00035600900000000001 2024-05-08T08:52:18.391 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.391 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.391 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:18.391 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.880094+0000", 2024-05-08T08:52:18.391 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0021808029999999998 2024-05-08T08:52:18.391 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.391 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.391 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:18.391 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.883222+0000", 2024-05-08T08:52:18.392 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0031282630000000001 2024-05-08T08:52:18.392 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.392 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.392 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:18.392 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.883279+0000", 2024-05-08T08:52:18.392 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6369999999999997e-05 2024-05-08T08:52:18.392 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.392 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.392 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:18.392 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.883327+0000", 2024-05-08T08:52:18.392 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8139000000000001e-05 2024-05-08T08:52:18.392 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:18.392 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:18.392 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:18.393 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.393 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.393 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4465.0:1904 3.17 3:e8515fe1:::rbd_object_map.1140cf79ec85:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-05-08T08:52:18.393 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:17.029330+0000", 2024-05-08T08:52:18.393 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 1.3115488820000001, 2024-05-08T08:52:18.393 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0063870239999999998, 2024-05-08T08:52:18.393 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:18.393 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:18.393 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:18.393 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:18.393 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4465", 2024-05-08T08:52:18.393 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/390275688", 2024-05-08T08:52:18.393 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1904 2024-05-08T08:52:18.393 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.393 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:18.394 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.394 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:18.394 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.029330+0000", 2024-05-08T08:52:18.394 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:18.394 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.394 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.394 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:18.394 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.029330+0000", 2024-05-08T08:52:18.394 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:18.394 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.394 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.394 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:18.394 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.029330+0000", 2024-05-08T08:52:18.395 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4700000000000002e-07 2024-05-08T08:52:18.395 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.395 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.395 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:18.395 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.029332+0000", 2024-05-08T08:52:18.395 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9750000000000001e-06 2024-05-08T08:52:18.395 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.395 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.395 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:18.395 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.029336+0000", 2024-05-08T08:52:18.395 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8789999999999996e-06 2024-05-08T08:52:18.395 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.395 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.395 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:18.396 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.029339+0000", 2024-05-08T08:52:18.396 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9610000000000001e-06 2024-05-08T08:52:18.396 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.396 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.396 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:18.396 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.029918+0000", 2024-05-08T08:52:18.396 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00057881800000000004 2024-05-08T08:52:18.396 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.396 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.396 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:18.396 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.029980+0000", 2024-05-08T08:52:18.396 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2744999999999996e-05 2024-05-08T08:52:18.396 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.396 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.397 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 5", 2024-05-08T08:52:18.397 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.030189+0000", 2024-05-08T08:52:18.397 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000208755 2024-05-08T08:52:18.397 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.397 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.397 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:18.397 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.031943+0000", 2024-05-08T08:52:18.397 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001753597 2024-05-08T08:52:18.397 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.397 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.397 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:18.397 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.035610+0000", 2024-05-08T08:52:18.397 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0036676220000000002 2024-05-08T08:52:18.397 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.397 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.398 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:18.398 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.035661+0000", 2024-05-08T08:52:18.398 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0939999999999997e-05 2024-05-08T08:52:18.398 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.398 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.398 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:18.398 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.035717+0000", 2024-05-08T08:52:18.398 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5384999999999999e-05 2024-05-08T08:52:18.398 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:18.398 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:18.398 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:18.398 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.399 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.399 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4465.0:2329 3.17 3:e8515fe1:::rbd_object_map.1140cf79ec85:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-05-08T08:52:18.399 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:17.295805+0000", 2024-05-08T08:52:18.399 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 1.0450737640000001, 2024-05-08T08:52:18.399 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0069030209999999996, 2024-05-08T08:52:18.399 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:18.399 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:18.399 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:18.399 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:18.399 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4465", 2024-05-08T08:52:18.399 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/390275688", 2024-05-08T08:52:18.400 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2329 2024-05-08T08:52:18.400 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.400 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:18.400 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.400 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:18.400 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.295805+0000", 2024-05-08T08:52:18.400 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:18.400 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.400 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.400 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:18.400 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.295805+0000", 2024-05-08T08:52:18.400 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:18.400 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.400 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.401 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:18.401 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.295805+0000", 2024-05-08T08:52:18.401 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3900000000000001e-07 2024-05-08T08:52:18.401 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.401 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.401 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:18.401 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.295807+0000", 2024-05-08T08:52:18.401 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.759e-06 2024-05-08T08:52:18.401 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.401 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.401 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:18.401 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.295812+0000", 2024-05-08T08:52:18.401 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2410000000000003e-06 2024-05-08T08:52:18.401 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.401 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.402 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:18.402 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.295815+0000", 2024-05-08T08:52:18.402 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7300000000000001e-06 2024-05-08T08:52:18.402 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.402 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.402 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:18.402 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.298606+0000", 2024-05-08T08:52:18.402 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0027917570000000002 2024-05-08T08:52:18.402 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.402 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.402 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:18.402 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.298654+0000", 2024-05-08T08:52:18.402 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7682999999999997e-05 2024-05-08T08:52:18.403 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.403 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.403 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 5", 2024-05-08T08:52:18.403 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.298868+0000", 2024-05-08T08:52:18.403 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000213854 2024-05-08T08:52:18.403 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.403 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.403 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:18.403 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.300086+0000", 2024-05-08T08:52:18.403 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0012178250000000001 2024-05-08T08:52:18.403 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.403 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.403 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:18.403 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.302591+0000", 2024-05-08T08:52:18.403 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0025054650000000001 2024-05-08T08:52:18.404 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.404 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.404 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:18.404 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.302654+0000", 2024-05-08T08:52:18.404 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3260000000000001e-05 2024-05-08T08:52:18.404 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.404 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.404 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:18.404 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.302708+0000", 2024-05-08T08:52:18.404 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3208000000000003e-05 2024-05-08T08:52:18.404 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:18.404 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:18.404 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:18.404 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.404 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.405 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4465.0:2331 3.17 3:e8515fe1:::rbd_object_map.1140cf79ec85:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-05-08T08:52:18.405 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:17.295871+0000", 2024-05-08T08:52:18.405 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 1.0450069829999999, 2024-05-08T08:52:18.405 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.010985981000000001, 2024-05-08T08:52:18.405 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:18.405 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:18.405 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:18.405 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:18.405 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4465", 2024-05-08T08:52:18.405 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/390275688", 2024-05-08T08:52:18.405 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2331 2024-05-08T08:52:18.405 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.405 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:18.405 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.406 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:18.406 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.295871+0000", 2024-05-08T08:52:18.406 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:18.406 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.406 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.406 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:18.406 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.295871+0000", 2024-05-08T08:52:18.406 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:18.406 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.406 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.406 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:18.406 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.295872+0000", 2024-05-08T08:52:18.406 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8799999999999998e-07 2024-05-08T08:52:18.406 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.407 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.407 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:18.407 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.295874+0000", 2024-05-08T08:52:18.407 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2239999999999998e-06 2024-05-08T08:52:18.407 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.407 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.407 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:18.407 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.295880+0000", 2024-05-08T08:52:18.407 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6130000000000003e-06 2024-05-08T08:52:18.407 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.407 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.407 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:18.407 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.295884+0000", 2024-05-08T08:52:18.407 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1829999999999996e-06 2024-05-08T08:52:18.407 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.408 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.408 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:18.408 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.299175+0000", 2024-05-08T08:52:18.408 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.003291165 2024-05-08T08:52:18.408 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.408 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.408 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:18.408 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.299228+0000", 2024-05-08T08:52:18.408 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2830000000000001e-05 2024-05-08T08:52:18.408 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.408 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.408 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 5", 2024-05-08T08:52:18.408 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.299482+0000", 2024-05-08T08:52:18.408 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00025414300000000002 2024-05-08T08:52:18.409 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.409 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.409 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:18.409 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.301380+0000", 2024-05-08T08:52:18.409 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001898046 2024-05-08T08:52:18.409 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.409 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.409 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:18.409 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.306802+0000", 2024-05-08T08:52:18.409 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0054220730000000003 2024-05-08T08:52:18.409 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.409 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.409 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:18.409 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.306833+0000", 2024-05-08T08:52:18.410 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0491e-05 2024-05-08T08:52:18.410 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.410 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.410 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:18.410 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.306857+0000", 2024-05-08T08:52:18.410 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4825e-05 2024-05-08T08:52:18.410 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:18.410 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:18.410 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:18.410 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.410 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.410 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4465.0:2332 3.17 3:e8515fe1:::rbd_object_map.1140cf79ec85:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-05-08T08:52:18.410 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:17.295889+0000", 2024-05-08T08:52:18.410 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 1.044989207, 2024-05-08T08:52:18.411 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.011142335999999999, 2024-05-08T08:52:18.411 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:18.411 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:18.411 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:18.411 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:18.411 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4465", 2024-05-08T08:52:18.411 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/390275688", 2024-05-08T08:52:18.411 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2332 2024-05-08T08:52:18.411 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.411 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:18.411 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.411 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:18.411 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.295889+0000", 2024-05-08T08:52:18.411 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:18.411 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.412 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.412 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:18.412 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.295889+0000", 2024-05-08T08:52:18.412 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:18.412 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.412 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.412 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:18.412 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.295889+0000", 2024-05-08T08:52:18.412 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8099999999999999e-07 2024-05-08T08:52:18.412 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.412 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.412 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:18.412 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.295891+0000", 2024-05-08T08:52:18.412 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.511e-06 2024-05-08T08:52:18.413 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.413 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.413 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:18.413 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.295894+0000", 2024-05-08T08:52:18.413 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.247e-06 2024-05-08T08:52:18.413 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.413 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.413 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:18.413 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.295897+0000", 2024-05-08T08:52:18.413 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4609999999999998e-06 2024-05-08T08:52:18.413 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.413 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.413 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:18.413 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.300103+0000", 2024-05-08T08:52:18.413 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0042062569999999997 2024-05-08T08:52:18.414 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.414 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.414 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:18.414 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.300154+0000", 2024-05-08T08:52:18.414 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0788999999999998e-05 2024-05-08T08:52:18.414 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.414 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.414 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 5", 2024-05-08T08:52:18.414 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.300402+0000", 2024-05-08T08:52:18.414 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00024846299999999998 2024-05-08T08:52:18.414 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.414 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.414 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:18.414 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.301383+0000", 2024-05-08T08:52:18.415 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00098072799999999994 2024-05-08T08:52:18.415 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.415 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.415 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:18.415 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.306919+0000", 2024-05-08T08:52:18.415 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0055362520000000002 2024-05-08T08:52:18.415 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.415 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.415 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:18.415 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.306986+0000", 2024-05-08T08:52:18.415 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7292999999999994e-05 2024-05-08T08:52:18.415 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.415 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.415 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:18.415 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.307032+0000", 2024-05-08T08:52:18.416 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5053999999999999e-05 2024-05-08T08:52:18.416 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:18.416 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:18.416 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:18.416 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.416 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.416 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4465.0:2333 3.17 3:e8515fe1:::rbd_object_map.1140cf79ec85:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-05-08T08:52:18.416 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:17.295975+0000", 2024-05-08T08:52:18.416 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 1.044903404, 2024-05-08T08:52:18.416 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.01114335, 2024-05-08T08:52:18.416 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:18.416 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:18.416 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:18.416 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:18.417 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4465", 2024-05-08T08:52:18.417 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/390275688", 2024-05-08T08:52:18.417 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2333 2024-05-08T08:52:18.417 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.417 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:18.417 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.417 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:18.417 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.295975+0000", 2024-05-08T08:52:18.417 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:18.417 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.417 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.417 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:18.417 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.295975+0000", 2024-05-08T08:52:18.417 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:18.417 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.418 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.418 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:18.418 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.295975+0000", 2024-05-08T08:52:18.418 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1199999999999999e-07 2024-05-08T08:52:18.418 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.418 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.418 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:18.418 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.295977+0000", 2024-05-08T08:52:18.418 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.103e-06 2024-05-08T08:52:18.418 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.418 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.418 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:18.418 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.295982+0000", 2024-05-08T08:52:18.418 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1309999999999999e-06 2024-05-08T08:52:18.418 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.419 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.419 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:18.419 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.295984+0000", 2024-05-08T08:52:18.419 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8909999999999999e-06 2024-05-08T08:52:18.419 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.419 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.419 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:18.419 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.300741+0000", 2024-05-08T08:52:18.419 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0047561069999999999 2024-05-08T08:52:18.419 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.419 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.419 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:18.419 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.300804+0000", 2024-05-08T08:52:18.419 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2976000000000001e-05 2024-05-08T08:52:18.420 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.420 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.420 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 5", 2024-05-08T08:52:18.420 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.301079+0000", 2024-05-08T08:52:18.420 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00027533000000000001 2024-05-08T08:52:18.420 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.420 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.420 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:18.420 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.301647+0000", 2024-05-08T08:52:18.420 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00056773499999999998 2024-05-08T08:52:18.420 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.420 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.420 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:18.420 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.307067+0000", 2024-05-08T08:52:18.420 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0054205089999999996 2024-05-08T08:52:18.421 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.421 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.421 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:18.421 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.307096+0000", 2024-05-08T08:52:18.421 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9037e-05 2024-05-08T08:52:18.421 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.421 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.421 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:18.421 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.307118+0000", 2024-05-08T08:52:18.421 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2218999999999999e-05 2024-05-08T08:52:18.421 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:18.421 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:18.421 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:18.421 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.422 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.422 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4465.0:2350 3.17 3:e8515fe1:::rbd_object_map.1140cf79ec85:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-05-08T08:52:18.422 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:17.309624+0000", 2024-05-08T08:52:18.422 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 1.031254696, 2024-05-08T08:52:18.422 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0067184030000000004, 2024-05-08T08:52:18.422 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:18.422 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:18.422 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:18.422 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:18.422 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4465", 2024-05-08T08:52:18.422 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/390275688", 2024-05-08T08:52:18.422 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2350 2024-05-08T08:52:18.422 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.422 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:18.423 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.423 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:18.423 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.309624+0000", 2024-05-08T08:52:18.423 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:18.423 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.423 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.423 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:18.423 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.309624+0000", 2024-05-08T08:52:18.423 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:18.423 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.423 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.423 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:18.423 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.309624+0000", 2024-05-08T08:52:18.423 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9400000000000001e-07 2024-05-08T08:52:18.423 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.424 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.424 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:18.424 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.309627+0000", 2024-05-08T08:52:18.424 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9799999999999998e-06 2024-05-08T08:52:18.424 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.424 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.424 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:18.424 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.309636+0000", 2024-05-08T08:52:18.424 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.9260000000000006e-06 2024-05-08T08:52:18.424 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.424 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.424 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:18.424 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.309639+0000", 2024-05-08T08:52:18.424 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4400000000000001e-06 2024-05-08T08:52:18.424 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.425 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.425 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:18.425 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.310232+0000", 2024-05-08T08:52:18.425 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00059308400000000002 2024-05-08T08:52:18.425 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.425 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.425 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:18.425 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.310288+0000", 2024-05-08T08:52:18.425 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5962999999999999e-05 2024-05-08T08:52:18.425 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.425 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.425 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 5", 2024-05-08T08:52:18.425 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.310501+0000", 2024-05-08T08:52:18.425 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000212754 2024-05-08T08:52:18.425 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.426 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.426 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:18.426 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.311893+0000", 2024-05-08T08:52:18.426 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001391588 2024-05-08T08:52:18.426 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.426 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.426 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:18.426 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.316217+0000", 2024-05-08T08:52:18.426 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0043244219999999996 2024-05-08T08:52:18.426 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.426 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.426 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:18.426 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.316319+0000", 2024-05-08T08:52:18.426 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000101606 2024-05-08T08:52:18.427 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.427 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.427 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:18.427 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.316342+0000", 2024-05-08T08:52:18.427 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3346e-05 2024-05-08T08:52:18.427 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:18.427 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:18.427 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:18.427 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.427 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.427 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4465.0:2873 3.17 3:e8515fe1:::rbd_object_map.1140cf79ec85:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-05-08T08:52:18.427 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:17.549314+0000", 2024-05-08T08:52:18.427 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.79156429399999995, 2024-05-08T08:52:18.427 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0062881350000000003, 2024-05-08T08:52:18.428 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:18.428 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:18.428 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:18.428 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:18.428 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4465", 2024-05-08T08:52:18.428 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/390275688", 2024-05-08T08:52:18.428 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2873 2024-05-08T08:52:18.428 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.428 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:18.428 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.428 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:18.428 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.549314+0000", 2024-05-08T08:52:18.428 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:18.428 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.428 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.429 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:18.429 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.549314+0000", 2024-05-08T08:52:18.429 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:18.429 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.429 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.429 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:18.429 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.549314+0000", 2024-05-08T08:52:18.429 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7599999999999998e-07 2024-05-08T08:52:18.429 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.429 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.429 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:18.429 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.549316+0000", 2024-05-08T08:52:18.429 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9589999999999998e-06 2024-05-08T08:52:18.429 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.429 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.430 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:18.430 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.549320+0000", 2024-05-08T08:52:18.430 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9829999999999998e-06 2024-05-08T08:52:18.430 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.430 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.430 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:18.430 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.549323+0000", 2024-05-08T08:52:18.430 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.853e-06 2024-05-08T08:52:18.430 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.430 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.430 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:18.430 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.549967+0000", 2024-05-08T08:52:18.430 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00064347700000000005 2024-05-08T08:52:18.430 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.430 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.431 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:18.431 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.550069+0000", 2024-05-08T08:52:18.431 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000101869 2024-05-08T08:52:18.431 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.431 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.431 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 5", 2024-05-08T08:52:18.431 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.550413+0000", 2024-05-08T08:52:18.431 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000344006 2024-05-08T08:52:18.431 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.431 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.431 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:18.431 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.551387+0000", 2024-05-08T08:52:18.431 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00097466199999999995 2024-05-08T08:52:18.431 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.431 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.432 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:18.432 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.555543+0000", 2024-05-08T08:52:18.432 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0041560980000000004 2024-05-08T08:52:18.432 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.432 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.432 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:18.432 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.555579+0000", 2024-05-08T08:52:18.432 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5343999999999999e-05 2024-05-08T08:52:18.432 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.432 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.432 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:18.432 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.555602+0000", 2024-05-08T08:52:18.432 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3507999999999999e-05 2024-05-08T08:52:18.432 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:18.433 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:18.433 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:18.433 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.433 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.433 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4465.0:2874 3.17 3:e8515fe1:::rbd_object_map.1140cf79ec85:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-05-08T08:52:18.433 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:17.549328+0000", 2024-05-08T08:52:18.433 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.79155062600000003, 2024-05-08T08:52:18.433 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0066336010000000003, 2024-05-08T08:52:18.433 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:18.433 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:18.433 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:18.433 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:18.433 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4465", 2024-05-08T08:52:18.433 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/390275688", 2024-05-08T08:52:18.434 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2874 2024-05-08T08:52:18.434 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.434 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:18.434 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.434 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:18.434 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.549328+0000", 2024-05-08T08:52:18.434 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:18.434 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.434 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.434 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:18.434 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.549328+0000", 2024-05-08T08:52:18.434 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:18.434 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.434 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.435 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:18.435 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.549328+0000", 2024-05-08T08:52:18.435 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.28e-07 2024-05-08T08:52:18.435 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.435 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.435 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:18.435 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.549329+0000", 2024-05-08T08:52:18.435 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3790000000000001e-06 2024-05-08T08:52:18.435 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.435 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.435 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:18.435 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.549333+0000", 2024-05-08T08:52:18.435 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.2650000000000001e-06 2024-05-08T08:52:18.435 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.436 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.436 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:18.436 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.549335+0000", 2024-05-08T08:52:18.436 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6539999999999999e-06 2024-05-08T08:52:18.436 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.436 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.436 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:18.436 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.550726+0000", 2024-05-08T08:52:18.436 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001390898 2024-05-08T08:52:18.436 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.436 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.436 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:18.436 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.550780+0000", 2024-05-08T08:52:18.437 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3418000000000003e-05 2024-05-08T08:52:18.437 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.437 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.437 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 5", 2024-05-08T08:52:18.437 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.551004+0000", 2024-05-08T08:52:18.437 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00022409599999999999 2024-05-08T08:52:18.437 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.437 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.437 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:18.437 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.552935+0000", 2024-05-08T08:52:18.437 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0019310779999999999 2024-05-08T08:52:18.437 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.437 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.437 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:18.437 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.555898+0000", 2024-05-08T08:52:18.438 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0029634380000000001 2024-05-08T08:52:18.438 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.438 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.438 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:18.438 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.555930+0000", 2024-05-08T08:52:18.438 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1647999999999999e-05 2024-05-08T08:52:18.438 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.438 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.438 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:18.438 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.555961+0000", 2024-05-08T08:52:18.438 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1498999999999998e-05 2024-05-08T08:52:18.438 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:18.438 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:18.438 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:18.439 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.439 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.439 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4465.0:2875 3.17 3:e8515fe1:::rbd_object_map.1140cf79ec85:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-05-08T08:52:18.439 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:17.549361+0000", 2024-05-08T08:52:18.439 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.79151738500000002, 2024-05-08T08:52:18.439 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0066527289999999996, 2024-05-08T08:52:18.439 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:18.439 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:18.439 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:18.439 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:18.439 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4465", 2024-05-08T08:52:18.439 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/390275688", 2024-05-08T08:52:18.439 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2875 2024-05-08T08:52:18.439 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.440 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:18.440 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.440 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:18.440 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.549361+0000", 2024-05-08T08:52:18.440 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:18.440 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.440 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.440 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:18.440 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.549361+0000", 2024-05-08T08:52:18.440 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:18.440 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.440 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.440 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:18.440 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.549361+0000", 2024-05-08T08:52:18.441 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9999999999999999e-07 2024-05-08T08:52:18.441 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.441 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.441 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:18.441 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.549364+0000", 2024-05-08T08:52:18.441 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8059999999999999e-06 2024-05-08T08:52:18.441 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.441 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.441 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:18.441 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.549368+0000", 2024-05-08T08:52:18.441 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3540000000000002e-06 2024-05-08T08:52:18.441 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.441 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.441 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:18.441 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.549371+0000", 2024-05-08T08:52:18.442 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.858e-06 2024-05-08T08:52:18.442 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.442 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.442 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:18.442 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.551480+0000", 2024-05-08T08:52:18.442 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.002108646 2024-05-08T08:52:18.442 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.442 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.442 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:18.442 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.551534+0000", 2024-05-08T08:52:18.442 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4202e-05 2024-05-08T08:52:18.442 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.442 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.442 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 5", 2024-05-08T08:52:18.443 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.551738+0000", 2024-05-08T08:52:18.443 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000203928 2024-05-08T08:52:18.443 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.443 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.443 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:18.443 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.552970+0000", 2024-05-08T08:52:18.443 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0012320829999999999 2024-05-08T08:52:18.443 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.443 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.443 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:18.443 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.555968+0000", 2024-05-08T08:52:18.443 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0029977839999999999 2024-05-08T08:52:18.443 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.443 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.443 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:18.444 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.555993+0000", 2024-05-08T08:52:18.444 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.5304e-05 2024-05-08T08:52:18.444 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.444 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.444 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:18.444 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.556014+0000", 2024-05-08T08:52:18.444 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0463999999999999e-05 2024-05-08T08:52:18.444 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:18.444 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:18.444 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:18.444 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.444 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.444 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4465.0:2876 3.17 3:e8515fe1:::rbd_object_map.1140cf79ec85:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-05-08T08:52:18.444 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:17.549379+0000", 2024-05-08T08:52:18.444 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.79149972400000002, 2024-05-08T08:52:18.445 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0066828520000000004, 2024-05-08T08:52:18.445 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:18.445 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:18.445 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:18.445 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:18.445 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4465", 2024-05-08T08:52:18.445 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/390275688", 2024-05-08T08:52:18.445 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2876 2024-05-08T08:52:18.445 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.445 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:18.445 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.445 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:18.445 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.549379+0000", 2024-05-08T08:52:18.445 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:18.445 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.446 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.446 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:18.446 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.549379+0000", 2024-05-08T08:52:18.446 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:18.446 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.446 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.446 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:18.446 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.549379+0000", 2024-05-08T08:52:18.446 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.2899999999999999e-07 2024-05-08T08:52:18.446 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.446 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.446 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:18.446 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.549381+0000", 2024-05-08T08:52:18.446 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5939999999999999e-06 2024-05-08T08:52:18.447 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.447 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.447 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:18.447 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.549388+0000", 2024-05-08T08:52:18.447 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3119999999999996e-06 2024-05-08T08:52:18.447 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.447 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.447 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:18.447 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.549391+0000", 2024-05-08T08:52:18.447 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4939999999999999e-06 2024-05-08T08:52:18.447 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.447 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.447 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:18.447 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.552078+0000", 2024-05-08T08:52:18.447 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0026861340000000002 2024-05-08T08:52:18.448 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.448 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.448 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:18.448 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.552149+0000", 2024-05-08T08:52:18.448 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1465999999999996e-05 2024-05-08T08:52:18.448 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.448 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.448 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 5", 2024-05-08T08:52:18.448 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.552491+0000", 2024-05-08T08:52:18.448 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00034228299999999998 2024-05-08T08:52:18.448 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.448 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.448 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:18.448 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.553418+0000", 2024-05-08T08:52:18.449 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00092652900000000005 2024-05-08T08:52:18.449 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.449 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.449 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:18.449 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.556020+0000", 2024-05-08T08:52:18.449 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0026016910000000002 2024-05-08T08:52:18.449 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.449 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.449 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:18.449 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.556043+0000", 2024-05-08T08:52:18.449 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3156999999999999e-05 2024-05-08T08:52:18.449 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.449 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.449 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:18.449 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.556062+0000", 2024-05-08T08:52:18.450 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.8862999999999999e-05 2024-05-08T08:52:18.450 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:18.450 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:18.450 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:18.450 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.450 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.450 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4465.0:3098 3.1f 3:f90107d2:::rbd_data.1140cf79ec85.000000000000001c:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-05-08T08:52:18.450 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:17.675370+0000", 2024-05-08T08:52:18.450 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.665508196, 2024-05-08T08:52:18.450 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0063966439999999999, 2024-05-08T08:52:18.450 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:18.450 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:18.450 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:18.450 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:18.451 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4465", 2024-05-08T08:52:18.451 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/390275688", 2024-05-08T08:52:18.451 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 3098 2024-05-08T08:52:18.451 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.451 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:18.451 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.451 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:18.451 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.675370+0000", 2024-05-08T08:52:18.451 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:18.451 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.451 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.451 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:18.451 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.675370+0000", 2024-05-08T08:52:18.451 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:18.451 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.452 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.452 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:18.452 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.675371+0000", 2024-05-08T08:52:18.452 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7e-07 2024-05-08T08:52:18.452 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.452 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.452 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:18.452 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.675373+0000", 2024-05-08T08:52:18.452 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7300000000000001e-06 2024-05-08T08:52:18.452 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.452 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.452 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:18.452 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.675380+0000", 2024-05-08T08:52:18.452 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0430000000000002e-06 2024-05-08T08:52:18.453 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.453 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.453 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:18.453 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.675387+0000", 2024-05-08T08:52:18.453 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4910000000000002e-06 2024-05-08T08:52:18.453 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.453 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.453 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:18.453 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.675432+0000", 2024-05-08T08:52:18.453 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5192999999999999e-05 2024-05-08T08:52:18.453 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.453 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.453 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:18.453 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.675939+0000", 2024-05-08T08:52:18.453 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00050675000000000004 2024-05-08T08:52:18.454 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.454 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.454 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 3", 2024-05-08T08:52:18.454 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.676046+0000", 2024-05-08T08:52:18.454 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010745499999999999 2024-05-08T08:52:18.454 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.454 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.454 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:18.454 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.677718+0000", 2024-05-08T08:52:18.454 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001672116 2024-05-08T08:52:18.454 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.454 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.454 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:18.454 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.681693+0000", 2024-05-08T08:52:18.454 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0039740599999999997 2024-05-08T08:52:18.455 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.455 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.455 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:18.455 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.681724+0000", 2024-05-08T08:52:18.455 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1816999999999997e-05 2024-05-08T08:52:18.455 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.455 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.455 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:18.455 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.681767+0000", 2024-05-08T08:52:18.455 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2518999999999997e-05 2024-05-08T08:52:18.455 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:18.455 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:18.455 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:18.455 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:18.455 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:18.455 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:52:18.457 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:52:18.458 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:18.460+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:18.558 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:52:18.561+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:52:18.609 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:52:18.609 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:52:18.609 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:52:18.609 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:52:18.617 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:52:18.659 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:52:18.662+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:52:18.760 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:52:18.763+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:52:18.768 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:52:18.768 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:52:18.768 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:52:18.768 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:52:18.768 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:52:18.777 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:52:18.835 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T08:52:18.861 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:18.864+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:18.931 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:52:18.932 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:52:18.932 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:52:18.932 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:52:18.932 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.932 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4465.0:1324 3.0 3:074a62aa:::rbd_data.1140cf79ec85.0000000000000024:head [write 3169792~57856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-05-08T08:52:18.932 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:16.753104+0000", 2024-05-08T08:52:18.933 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 2.179630178, 2024-05-08T08:52:18.933 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0019559899999999999, 2024-05-08T08:52:18.933 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:18.933 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:18.933 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:18.933 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:18.933 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4465", 2024-05-08T08:52:18.934 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/390275688", 2024-05-08T08:52:18.934 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1324 2024-05-08T08:52:18.934 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.934 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:18.934 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.934 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:18.935 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.753104+0000", 2024-05-08T08:52:18.935 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:18.935 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.935 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.935 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:18.935 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.753104+0000", 2024-05-08T08:52:18.935 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:18.935 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.935 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.935 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:18.936 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.753105+0000", 2024-05-08T08:52:18.936 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.22e-06 2024-05-08T08:52:18.936 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.936 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.936 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:18.936 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.753162+0000", 2024-05-08T08:52:18.936 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6441999999999999e-05 2024-05-08T08:52:18.937 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.937 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.937 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:18.937 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.753181+0000", 2024-05-08T08:52:18.937 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.8967e-05 2024-05-08T08:52:18.937 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.937 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.937 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:18.937 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.753190+0000", 2024-05-08T08:52:18.937 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5130000000000002e-06 2024-05-08T08:52:18.938 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.938 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.938 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:18.938 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.753242+0000", 2024-05-08T08:52:18.938 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1795999999999997e-05 2024-05-08T08:52:18.939 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.939 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.939 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:18.939 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.753292+0000", 2024-05-08T08:52:18.939 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9722000000000003e-05 2024-05-08T08:52:18.939 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.940 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.940 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 2", 2024-05-08T08:52:18.940 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.753430+0000", 2024-05-08T08:52:18.940 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013834900000000001 2024-05-08T08:52:18.940 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.940 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.941 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:18.941 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.754559+0000", 2024-05-08T08:52:18.941 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001128684 2024-05-08T08:52:18.941 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.941 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.942 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:18.942 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.754947+0000", 2024-05-08T08:52:18.942 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00038794900000000001 2024-05-08T08:52:18.942 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.942 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.942 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:18.942 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.755013+0000", 2024-05-08T08:52:18.942 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6414000000000005e-05 2024-05-08T08:52:18.943 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.943 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.943 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:18.943 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.755060+0000", 2024-05-08T08:52:18.943 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6934000000000002e-05 2024-05-08T08:52:18.943 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:18.943 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:18.943 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:18.943 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.943 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.943 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(client.4465.0:1557 3.8 e21/18 3:1443f49a:::rbd_data.1140cf79ec85.0000000000000002:head v 21'7, mlcod=21'5)", 2024-05-08T08:52:18.944 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:16.875412+0000", 2024-05-08T08:52:18.944 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 2.0573225939999999, 2024-05-08T08:52:18.944 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.004428909, 2024-05-08T08:52:18.944 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:18.944 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:18.944 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:18.945 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:18.945 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.945 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:18.945 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.875412+0000", 2024-05-08T08:52:18.945 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:18.945 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.946 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.946 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:18.946 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.875412+0000", 2024-05-08T08:52:18.946 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:18.947 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.947 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.947 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:18.947 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.875413+0000", 2024-05-08T08:52:18.947 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.7499999999999998e-07 2024-05-08T08:52:18.948 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.948 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.948 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:18.948 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.875435+0000", 2024-05-08T08:52:18.948 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2427999999999999e-05 2024-05-08T08:52:18.948 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.948 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.949 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:18.949 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.875451+0000", 2024-05-08T08:52:18.949 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5998000000000002e-05 2024-05-08T08:52:18.949 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.949 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.949 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:18.949 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.875459+0000", 2024-05-08T08:52:18.949 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4919999999999993e-06 2024-05-08T08:52:18.949 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.949 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.949 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:18.950 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.875538+0000", 2024-05-08T08:52:18.950 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9009000000000002e-05 2024-05-08T08:52:18.951 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.951 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.951 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:18.951 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.875555+0000", 2024-05-08T08:52:18.951 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6656000000000001e-05 2024-05-08T08:52:18.951 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.952 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.952 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:18.952 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.879799+0000", 2024-05-08T08:52:18.952 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0042440560000000004 2024-05-08T08:52:18.952 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.952 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.953 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:18.953 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.879840+0000", 2024-05-08T08:52:18.953 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1294999999999997e-05 2024-05-08T08:52:18.953 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:18.953 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:18.953 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:18.954 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.954 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.954 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4465.0:1561 3.4 3:239ea5ea:::rbd_data.1140cf79ec85.0000000000000022:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2371584~40960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-05-08T08:52:18.954 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:16.880847+0000", 2024-05-08T08:52:18.954 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 2.0518872639999999, 2024-05-08T08:52:18.955 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0026051, 2024-05-08T08:52:18.955 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:18.955 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:18.955 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:18.955 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:18.955 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4465", 2024-05-08T08:52:18.956 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/390275688", 2024-05-08T08:52:18.956 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1561 2024-05-08T08:52:18.956 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.956 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:18.956 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.956 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:18.957 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.880847+0000", 2024-05-08T08:52:18.957 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:18.957 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.957 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.957 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:18.957 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.880847+0000", 2024-05-08T08:52:18.958 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:18.958 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.958 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.958 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:18.958 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.880848+0000", 2024-05-08T08:52:18.958 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.04e-06 2024-05-08T08:52:18.959 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.959 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.959 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:18.959 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.880879+0000", 2024-05-08T08:52:18.959 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0988000000000002e-05 2024-05-08T08:52:18.959 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.960 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.960 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:18.960 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.880891+0000", 2024-05-08T08:52:18.960 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1600000000000001e-05 2024-05-08T08:52:18.960 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.960 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.961 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:18.961 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.880896+0000", 2024-05-08T08:52:18.961 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3720000000000004e-06 2024-05-08T08:52:18.961 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.961 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.961 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:18.962 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.880947+0000", 2024-05-08T08:52:18.962 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0657e-05 2024-05-08T08:52:18.962 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.962 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.962 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:18.962 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.881043+0000", 2024-05-08T08:52:18.963 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6020000000000006e-05 2024-05-08T08:52:18.963 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.963 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.963 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 2", 2024-05-08T08:52:18.963 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.881123+0000", 2024-05-08T08:52:18.964 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0461000000000001e-05 2024-05-08T08:52:18.964 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.964 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.964 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:18.964 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.883189+0000", 2024-05-08T08:52:18.964 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.002065903 2024-05-08T08:52:18.965 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.965 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.965 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:18.965 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.883396+0000", 2024-05-08T08:52:18.966 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000207126 2024-05-08T08:52:18.966 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.966 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.966 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:18.966 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.883425+0000", 2024-05-08T08:52:18.966 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9159999999999999e-05 2024-05-08T08:52:18.967 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.967 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.967 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:18.967 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.883452+0000", 2024-05-08T08:52:18.967 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6772999999999999e-05 2024-05-08T08:52:18.967 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:18.968 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:18.968 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:18.968 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.968 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.968 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4465.0:1905 3.b 3:d2f86791:::rbd_data.1140cf79ec85.000000000000002f:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2961408~64000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-05-08T08:52:18.969 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:17.032951+0000", 2024-05-08T08:52:18.969 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 1.8997827620000001, 2024-05-08T08:52:18.969 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0031612749999999998, 2024-05-08T08:52:18.969 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:18.969 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:18.970 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:18.970 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:18.970 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4465", 2024-05-08T08:52:18.970 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/390275688", 2024-05-08T08:52:18.970 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1905 2024-05-08T08:52:18.970 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.971 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:18.971 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.971 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:18.971 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.032951+0000", 2024-05-08T08:52:18.971 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:18.971 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.972 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.972 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:18.972 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.032951+0000", 2024-05-08T08:52:18.972 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:18.972 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.972 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.973 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:18.973 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.032952+0000", 2024-05-08T08:52:18.973 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9699999999999995e-07 2024-05-08T08:52:18.973 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.973 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.973 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:18.974 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.032995+0000", 2024-05-08T08:52:18.974 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2412e-05 2024-05-08T08:52:18.974 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.974 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.974 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:18.975 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.033023+0000", 2024-05-08T08:52:18.975 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8174000000000002e-05 2024-05-08T08:52:18.975 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.975 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.975 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:18.975 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.033028+0000", 2024-05-08T08:52:18.976 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4530000000000001e-06 2024-05-08T08:52:18.976 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.976 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.976 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:18.976 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.033072+0000", 2024-05-08T08:52:18.976 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3609999999999998e-05 2024-05-08T08:52:18.977 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.977 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.977 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:18.977 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.033123+0000", 2024-05-08T08:52:18.977 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1311999999999999e-05 2024-05-08T08:52:18.977 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.978 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.978 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 0", 2024-05-08T08:52:18.978 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.033203+0000", 2024-05-08T08:52:18.978 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9711999999999998e-05 2024-05-08T08:52:18.978 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.978 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.979 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:18.979 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.035910+0000", 2024-05-08T08:52:18.979 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0027069429999999998 2024-05-08T08:52:18.979 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.979 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.980 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:18.980 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.036054+0000", 2024-05-08T08:52:18.980 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000143912 2024-05-08T08:52:18.980 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.980 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.981 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:18.981 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.036087+0000", 2024-05-08T08:52:18.981 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3210000000000002e-05 2024-05-08T08:52:18.981 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.981 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.981 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:18.981 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.036113+0000", 2024-05-08T08:52:18.981 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.5740000000000001e-05 2024-05-08T08:52:18.982 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:18.982 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:18.982 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:18.982 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.982 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.982 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4465.0:1910 3.b 3:d2f86791:::rbd_data.1140cf79ec85.000000000000002f:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-05-08T08:52:18.982 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:17.044143+0000", 2024-05-08T08:52:18.982 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 1.8885908730000001, 2024-05-08T08:52:18.982 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0021849299999999999, 2024-05-08T08:52:18.982 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:18.982 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:18.982 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:18.982 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:18.982 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4465", 2024-05-08T08:52:18.983 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/390275688", 2024-05-08T08:52:18.983 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1910 2024-05-08T08:52:18.983 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.983 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:18.983 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.983 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:18.983 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.044143+0000", 2024-05-08T08:52:18.983 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:18.983 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.983 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.983 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:18.983 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.044143+0000", 2024-05-08T08:52:18.983 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:18.983 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.983 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.984 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:18.984 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.044144+0000", 2024-05-08T08:52:18.984 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0100000000000004e-07 2024-05-08T08:52:18.984 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.984 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.984 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:18.984 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.044148+0000", 2024-05-08T08:52:18.984 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.481e-06 2024-05-08T08:52:18.984 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.985 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.985 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:18.985 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.044159+0000", 2024-05-08T08:52:18.985 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0383999999999999e-05 2024-05-08T08:52:18.985 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.985 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.985 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:18.985 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.044165+0000", 2024-05-08T08:52:18.985 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.677e-06 2024-05-08T08:52:18.986 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.986 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.986 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:18.986 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.044217+0000", 2024-05-08T08:52:18.986 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2298e-05 2024-05-08T08:52:18.986 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.986 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.986 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:18.987 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.044303+0000", 2024-05-08T08:52:18.987 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.5712999999999997e-05 2024-05-08T08:52:18.987 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.987 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.987 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 0", 2024-05-08T08:52:18.987 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.044418+0000", 2024-05-08T08:52:18.987 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000115186 2024-05-08T08:52:18.987 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.988 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.988 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:18.988 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.044919+0000", 2024-05-08T08:52:18.988 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00050089000000000004 2024-05-08T08:52:18.988 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.988 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.988 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:18.988 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.046274+0000", 2024-05-08T08:52:18.988 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0013558470000000001 2024-05-08T08:52:18.989 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.989 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.989 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:18.989 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.046304+0000", 2024-05-08T08:52:18.989 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9391999999999999e-05 2024-05-08T08:52:18.989 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.989 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.989 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:18.990 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.046328+0000", 2024-05-08T08:52:18.990 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4360999999999999e-05 2024-05-08T08:52:18.990 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:18.990 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:18.990 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:18.990 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.990 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.990 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(client.4465.0:2341 3.8 e22/18 3:12cd7241:::rbd_data.1140cf79ec85.000000000000002b:head v 22'9, mlcod=21'7)", 2024-05-08T08:52:18.990 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:17.300738+0000", 2024-05-08T08:52:18.990 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 1.631996386, 2024-05-08T08:52:18.990 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0016794399999999999, 2024-05-08T08:52:18.990 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:18.990 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:18.991 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:18.991 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:18.991 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.991 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:18.991 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.300738+0000", 2024-05-08T08:52:18.991 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:18.991 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.991 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.991 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:18.991 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.300738+0000", 2024-05-08T08:52:18.991 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:18.992 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.992 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.992 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:18.992 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.300738+0000", 2024-05-08T08:52:18.992 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2600000000000002e-07 2024-05-08T08:52:18.992 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.992 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.992 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:18.992 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.300755+0000", 2024-05-08T08:52:18.992 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6509999999999999e-05 2024-05-08T08:52:18.992 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.992 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.992 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:18.992 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.300769+0000", 2024-05-08T08:52:18.993 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3593e-05 2024-05-08T08:52:18.993 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.993 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.993 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:18.993 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.300776+0000", 2024-05-08T08:52:18.993 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1049999999999997e-06 2024-05-08T08:52:18.993 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.993 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.993 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:18.993 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.300833+0000", 2024-05-08T08:52:18.993 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7503000000000002e-05 2024-05-08T08:52:18.993 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.993 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.993 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:18.994 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.300856+0000", 2024-05-08T08:52:18.994 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2518999999999999e-05 2024-05-08T08:52:18.994 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.994 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.994 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:18.994 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.302389+0000", 2024-05-08T08:52:18.994 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0015333040000000001 2024-05-08T08:52:18.994 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.994 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.994 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:18.994 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.302417+0000", 2024-05-08T08:52:18.994 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.828e-05 2024-05-08T08:52:18.994 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:18.994 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:18.995 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:18.995 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.995 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.995 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4465.0:2342 3.19 3:9b6e72ee:::rbd_data.1140cf79ec85.0000000000000031:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3362816~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-05-08T08:52:18.995 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:17.302887+0000", 2024-05-08T08:52:18.995 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 1.6298476630000001, 2024-05-08T08:52:18.995 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0041277589999999999, 2024-05-08T08:52:18.995 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:18.995 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:18.995 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:18.995 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:18.995 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4465", 2024-05-08T08:52:18.995 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/390275688", 2024-05-08T08:52:18.995 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2342 2024-05-08T08:52:18.996 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.996 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:18.996 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.996 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:18.996 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.302887+0000", 2024-05-08T08:52:18.996 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:18.996 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.996 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.996 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:18.996 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.302887+0000", 2024-05-08T08:52:18.996 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:18.996 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.996 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.996 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:18.997 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.302888+0000", 2024-05-08T08:52:18.997 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2470000000000001e-06 2024-05-08T08:52:18.997 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.997 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.997 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:18.997 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.302963+0000", 2024-05-08T08:52:18.997 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5199999999999998e-05 2024-05-08T08:52:18.997 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.997 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.997 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:18.997 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.302977+0000", 2024-05-08T08:52:18.997 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3709e-05 2024-05-08T08:52:18.997 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.997 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.998 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:18.998 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.302984+0000", 2024-05-08T08:52:18.998 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0960000000000003e-06 2024-05-08T08:52:18.998 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.998 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.998 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:18.998 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.303031+0000", 2024-05-08T08:52:18.998 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6907000000000003e-05 2024-05-08T08:52:18.998 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.998 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.998 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:18.998 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.303082+0000", 2024-05-08T08:52:18.998 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0871000000000001e-05 2024-05-08T08:52:18.998 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.998 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.999 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 4", 2024-05-08T08:52:18.999 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.303162+0000", 2024-05-08T08:52:18.999 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0925000000000002e-05 2024-05-08T08:52:18.999 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.999 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.999 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:18.999 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.306552+0000", 2024-05-08T08:52:18.999 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0033892850000000001 2024-05-08T08:52:18.999 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:18.999 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:18.999 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:18.999 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.306946+0000", 2024-05-08T08:52:18.999 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00039394199999999999 2024-05-08T08:52:18.999 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.000 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.000 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:19.000 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.306984+0000", 2024-05-08T08:52:19.000 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8278999999999997e-05 2024-05-08T08:52:19.000 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.000 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.000 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:19.000 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.307014+0000", 2024-05-08T08:52:19.000 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0298e-05 2024-05-08T08:52:19.000 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:19.000 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:19.000 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:19.000 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.001 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.001 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4465.0:2347 3.19 3:9a36fc17:::rbd_data.1140cf79ec85.0000000000000013:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2260992~28672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-05-08T08:52:19.001 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:17.307197+0000", 2024-05-08T08:52:19.001 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 1.62553751, 2024-05-08T08:52:19.001 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0043368809999999999, 2024-05-08T08:52:19.001 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:19.001 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:19.001 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:19.001 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:19.001 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4465", 2024-05-08T08:52:19.001 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/390275688", 2024-05-08T08:52:19.001 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2347 2024-05-08T08:52:19.001 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.001 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:19.001 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.002 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:19.002 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.307197+0000", 2024-05-08T08:52:19.002 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:19.002 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.002 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.002 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:19.002 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.307197+0000", 2024-05-08T08:52:19.002 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:19.002 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.002 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.002 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:19.002 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.307197+0000", 2024-05-08T08:52:19.002 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0800000000000004e-07 2024-05-08T08:52:19.002 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.003 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.003 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:19.003 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.307217+0000", 2024-05-08T08:52:19.003 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9627999999999999e-05 2024-05-08T08:52:19.003 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.003 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.003 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:19.003 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.307227+0000", 2024-05-08T08:52:19.003 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.7750000000000004e-06 2024-05-08T08:52:19.003 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.003 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.003 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:19.003 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.307232+0000", 2024-05-08T08:52:19.003 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0000000000000004e-06 2024-05-08T08:52:19.004 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.004 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.004 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:19.004 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.307265+0000", 2024-05-08T08:52:19.004 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3074000000000002e-05 2024-05-08T08:52:19.004 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.004 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.004 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:19.004 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.307305+0000", 2024-05-08T08:52:19.004 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0581999999999999e-05 2024-05-08T08:52:19.004 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.004 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.004 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 4", 2024-05-08T08:52:19.004 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.307382+0000", 2024-05-08T08:52:19.005 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.6136000000000004e-05 2024-05-08T08:52:19.005 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.005 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.005 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:19.005 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.310844+0000", 2024-05-08T08:52:19.005 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0034625250000000002 2024-05-08T08:52:19.005 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.005 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.005 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:19.005 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.311482+0000", 2024-05-08T08:52:19.005 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00063792899999999997 2024-05-08T08:52:19.005 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.005 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.005 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:19.006 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.311511+0000", 2024-05-08T08:52:19.006 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9023999999999999e-05 2024-05-08T08:52:19.006 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.006 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.006 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:19.006 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.311534+0000", 2024-05-08T08:52:19.006 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2500000000000001e-05 2024-05-08T08:52:19.006 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:19.006 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:19.006 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:19.006 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.006 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.006 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4465.0:2348 3.b 3:d2f86791:::rbd_data.1140cf79ec85.000000000000002f:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3990528~5632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-05-08T08:52:19.007 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:17.307246+0000", 2024-05-08T08:52:19.007 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 1.625488496, 2024-05-08T08:52:19.007 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001911451, 2024-05-08T08:52:19.007 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:19.007 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:19.007 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:19.007 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:19.007 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4465", 2024-05-08T08:52:19.007 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/390275688", 2024-05-08T08:52:19.007 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2348 2024-05-08T08:52:19.007 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.007 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:19.007 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.007 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:19.007 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.307246+0000", 2024-05-08T08:52:19.008 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:19.008 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.008 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.008 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:19.008 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.307246+0000", 2024-05-08T08:52:19.008 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:19.008 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.008 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.008 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:19.008 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.307246+0000", 2024-05-08T08:52:19.008 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6600000000000002e-07 2024-05-08T08:52:19.008 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.008 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.008 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:19.009 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.307251+0000", 2024-05-08T08:52:19.009 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6630000000000001e-06 2024-05-08T08:52:19.009 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.009 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.009 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:19.009 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.307255+0000", 2024-05-08T08:52:19.009 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2660000000000003e-06 2024-05-08T08:52:19.009 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.009 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.009 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:19.009 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.307260+0000", 2024-05-08T08:52:19.009 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0690000000000001e-06 2024-05-08T08:52:19.009 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.010 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.010 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:19.010 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.307298+0000", 2024-05-08T08:52:19.010 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7897e-05 2024-05-08T08:52:19.010 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.010 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.010 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:19.010 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.307343+0000", 2024-05-08T08:52:19.010 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4762000000000003e-05 2024-05-08T08:52:19.010 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.010 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.010 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 0", 2024-05-08T08:52:19.010 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.307422+0000", 2024-05-08T08:52:19.010 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9251000000000004e-05 2024-05-08T08:52:19.011 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.011 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.011 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:19.011 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.308885+0000", 2024-05-08T08:52:19.011 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0014628429999999999 2024-05-08T08:52:19.011 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.011 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.011 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:19.011 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.309098+0000", 2024-05-08T08:52:19.011 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000213697 2024-05-08T08:52:19.011 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.011 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.011 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:19.011 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.309133+0000", 2024-05-08T08:52:19.012 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4539999999999998e-05 2024-05-08T08:52:19.012 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.012 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.012 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:19.012 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.309157+0000", 2024-05-08T08:52:19.012 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4097e-05 2024-05-08T08:52:19.012 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:19.012 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:19.012 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:19.012 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.012 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.012 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4465.0:2458 3.19 3:9a36fc17:::rbd_data.1140cf79ec85.0000000000000013:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-05-08T08:52:19.012 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:17.357589+0000", 2024-05-08T08:52:19.012 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 1.5751451350000001, 2024-05-08T08:52:19.013 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0040882360000000003, 2024-05-08T08:52:19.013 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:19.013 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:19.013 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:19.013 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:19.013 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4465", 2024-05-08T08:52:19.013 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/390275688", 2024-05-08T08:52:19.013 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2458 2024-05-08T08:52:19.013 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.013 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:19.013 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.013 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:19.013 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.357589+0000", 2024-05-08T08:52:19.013 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:19.013 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.014 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.014 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:19.014 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.357589+0000", 2024-05-08T08:52:19.014 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:19.014 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.014 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.014 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:19.014 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.357590+0000", 2024-05-08T08:52:19.014 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9999999999999996e-07 2024-05-08T08:52:19.014 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.014 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.014 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:19.014 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.357594+0000", 2024-05-08T08:52:19.014 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7419999999999999e-06 2024-05-08T08:52:19.015 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.015 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.015 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:19.015 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.357604+0000", 2024-05-08T08:52:19.015 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.059e-05 2024-05-08T08:52:19.015 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.015 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.015 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:19.015 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.357611+0000", 2024-05-08T08:52:19.015 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3150000000000003e-06 2024-05-08T08:52:19.015 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.015 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.015 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:19.015 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.357665+0000", 2024-05-08T08:52:19.016 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3075000000000003e-05 2024-05-08T08:52:19.016 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.016 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.016 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:19.016 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.357736+0000", 2024-05-08T08:52:19.016 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0938000000000004e-05 2024-05-08T08:52:19.016 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.016 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.016 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 4", 2024-05-08T08:52:19.016 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.357860+0000", 2024-05-08T08:52:19.016 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012432400000000001 2024-05-08T08:52:19.016 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.016 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.016 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:19.017 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.358334+0000", 2024-05-08T08:52:19.017 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00047395 2024-05-08T08:52:19.017 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.017 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.017 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:19.017 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.361581+0000", 2024-05-08T08:52:19.017 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0032476810000000001 2024-05-08T08:52:19.017 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.017 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.017 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:19.017 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.361634+0000", 2024-05-08T08:52:19.017 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2123999999999998e-05 2024-05-08T08:52:19.017 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.018 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.018 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:19.018 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.361677+0000", 2024-05-08T08:52:19.018 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3696999999999999e-05 2024-05-08T08:52:19.018 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:19.018 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:19.018 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:19.018 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.018 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.018 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4465.0:2580 3.0 3:074a62aa:::rbd_data.1140cf79ec85.0000000000000024:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1454080~66048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-05-08T08:52:19.018 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:17.409757+0000", 2024-05-08T08:52:19.018 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 1.522977241, 2024-05-08T08:52:19.018 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0039510279999999997, 2024-05-08T08:52:19.019 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:19.019 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:19.019 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:19.019 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:19.019 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4465", 2024-05-08T08:52:19.019 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/390275688", 2024-05-08T08:52:19.019 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2580 2024-05-08T08:52:19.019 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.019 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:19.019 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.019 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:19.019 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.409757+0000", 2024-05-08T08:52:19.019 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:19.019 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.019 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.020 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:19.020 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.409757+0000", 2024-05-08T08:52:19.020 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:19.020 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.020 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.020 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:19.020 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.409758+0000", 2024-05-08T08:52:19.020 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.8700000000000004e-07 2024-05-08T08:52:19.020 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.020 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.020 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:19.020 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.412377+0000", 2024-05-08T08:52:19.020 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0026194349999999998 2024-05-08T08:52:19.020 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.021 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.021 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:19.021 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.412388+0000", 2024-05-08T08:52:19.021 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0479000000000001e-05 2024-05-08T08:52:19.021 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.021 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.021 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:19.021 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.412394+0000", 2024-05-08T08:52:19.021 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6499999999999999e-06 2024-05-08T08:52:19.021 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.021 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.021 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:19.021 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.412441+0000", 2024-05-08T08:52:19.021 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7105999999999999e-05 2024-05-08T08:52:19.021 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.022 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.022 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:19.022 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.412490+0000", 2024-05-08T08:52:19.022 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8355e-05 2024-05-08T08:52:19.022 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.022 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.022 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 2", 2024-05-08T08:52:19.022 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.412574+0000", 2024-05-08T08:52:19.022 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3796999999999997e-05 2024-05-08T08:52:19.022 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.022 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.022 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:19.022 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.413477+0000", 2024-05-08T08:52:19.023 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00090339999999999995 2024-05-08T08:52:19.023 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.023 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.023 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:19.023 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.413652+0000", 2024-05-08T08:52:19.023 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00017475499999999999 2024-05-08T08:52:19.023 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.023 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.023 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:19.023 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.413683+0000", 2024-05-08T08:52:19.023 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1333e-05 2024-05-08T08:52:19.023 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.023 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.023 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:19.024 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.413708+0000", 2024-05-08T08:52:19.024 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4831e-05 2024-05-08T08:52:19.024 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:19.024 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:19.024 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:19.024 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.024 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.024 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4465.0:2768 3.19 3:9b6e72ee:::rbd_data.1140cf79ec85.0000000000000031:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2269184~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-05-08T08:52:19.024 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:17.496090+0000", 2024-05-08T08:52:19.024 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 1.436644579, 2024-05-08T08:52:19.024 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0037026239999999998, 2024-05-08T08:52:19.024 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:19.024 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:19.024 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:19.024 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:19.025 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4465", 2024-05-08T08:52:19.025 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/390275688", 2024-05-08T08:52:19.025 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2768 2024-05-08T08:52:19.025 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.025 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:19.025 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.025 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:19.025 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.496090+0000", 2024-05-08T08:52:19.025 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:19.025 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.025 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.025 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:19.025 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.496090+0000", 2024-05-08T08:52:19.025 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:19.025 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.026 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.026 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:19.026 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.496091+0000", 2024-05-08T08:52:19.026 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0780000000000001e-06 2024-05-08T08:52:19.026 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.026 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.026 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:19.026 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.496143+0000", 2024-05-08T08:52:19.026 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.24e-05 2024-05-08T08:52:19.026 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.026 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.026 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:19.026 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.496154+0000", 2024-05-08T08:52:19.026 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.059e-05 2024-05-08T08:52:19.027 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.027 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.027 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:19.027 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.496159+0000", 2024-05-08T08:52:19.027 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.237e-06 2024-05-08T08:52:19.027 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.027 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.027 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:19.027 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.496206+0000", 2024-05-08T08:52:19.027 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7188000000000003e-05 2024-05-08T08:52:19.027 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.027 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.027 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:19.027 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.496255+0000", 2024-05-08T08:52:19.028 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9072e-05 2024-05-08T08:52:19.028 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.028 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.028 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 4", 2024-05-08T08:52:19.028 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.496336+0000", 2024-05-08T08:52:19.028 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1128e-05 2024-05-08T08:52:19.028 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.028 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.028 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:19.028 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.499473+0000", 2024-05-08T08:52:19.028 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.003136841 2024-05-08T08:52:19.028 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.028 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.028 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:19.028 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.499735+0000", 2024-05-08T08:52:19.029 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000261382 2024-05-08T08:52:19.029 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.029 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.029 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:19.029 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.499768+0000", 2024-05-08T08:52:19.029 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3586e-05 2024-05-08T08:52:19.029 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.029 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.029 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:19.029 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.499792+0000", 2024-05-08T08:52:19.029 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4122e-05 2024-05-08T08:52:19.029 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:19.029 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:19.029 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:19.030 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.030 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.030 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4465.0:2774 3.4 3:239ea5ea:::rbd_data.1140cf79ec85.0000000000000022:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-05-08T08:52:19.030 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:17.504631+0000", 2024-05-08T08:52:19.030 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 1.42810342, 2024-05-08T08:52:19.030 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0023409099999999999, 2024-05-08T08:52:19.030 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:19.030 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:19.030 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:19.030 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:19.030 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4465", 2024-05-08T08:52:19.030 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/390275688", 2024-05-08T08:52:19.030 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2774 2024-05-08T08:52:19.031 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.031 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:19.031 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.031 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:19.031 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.504631+0000", 2024-05-08T08:52:19.031 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:19.031 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.031 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.031 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:19.031 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.504631+0000", 2024-05-08T08:52:19.031 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:19.031 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.031 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.031 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:19.031 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.504631+0000", 2024-05-08T08:52:19.032 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7899999999999998e-07 2024-05-08T08:52:19.032 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.032 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.032 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:19.032 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.504635+0000", 2024-05-08T08:52:19.032 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.698e-06 2024-05-08T08:52:19.032 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.032 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.032 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:19.032 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.504645+0000", 2024-05-08T08:52:19.032 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.01e-05 2024-05-08T08:52:19.032 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.032 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.032 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:19.033 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.504651+0000", 2024-05-08T08:52:19.033 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.322e-06 2024-05-08T08:52:19.033 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.033 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.033 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:19.033 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.504702+0000", 2024-05-08T08:52:19.033 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1232e-05 2024-05-08T08:52:19.033 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.033 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.033 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:19.033 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.504768+0000", 2024-05-08T08:52:19.033 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6427999999999995e-05 2024-05-08T08:52:19.033 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.033 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.034 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 2", 2024-05-08T08:52:19.034 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.504830+0000", 2024-05-08T08:52:19.034 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2131000000000005e-05 2024-05-08T08:52:19.034 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.034 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.034 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:19.034 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.506584+0000", 2024-05-08T08:52:19.034 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0017540450000000001 2024-05-08T08:52:19.034 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.034 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.034 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:19.034 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.506912+0000", 2024-05-08T08:52:19.034 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00032734700000000002 2024-05-08T08:52:19.034 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.035 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.035 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:19.035 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.506941+0000", 2024-05-08T08:52:19.035 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9453000000000001e-05 2024-05-08T08:52:19.035 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.035 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.035 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:19.035 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.506972+0000", 2024-05-08T08:52:19.035 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0474999999999999e-05 2024-05-08T08:52:19.035 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:19.035 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:19.035 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:19.035 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.035 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.036 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4465.0:2777 3.0 3:074a62aa:::rbd_data.1140cf79ec85.0000000000000024:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-05-08T08:52:19.036 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:17.504663+0000", 2024-05-08T08:52:19.036 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 1.4280710329999999, 2024-05-08T08:52:19.036 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0047115480000000003, 2024-05-08T08:52:19.036 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:19.036 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:19.036 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:19.036 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:19.036 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4465", 2024-05-08T08:52:19.036 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/390275688", 2024-05-08T08:52:19.036 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2777 2024-05-08T08:52:19.036 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.036 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:19.036 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.037 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:19.037 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.504663+0000", 2024-05-08T08:52:19.037 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:19.037 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.037 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.037 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:19.037 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.504663+0000", 2024-05-08T08:52:19.037 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:19.037 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.037 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.037 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:19.037 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.504664+0000", 2024-05-08T08:52:19.037 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9400000000000001e-07 2024-05-08T08:52:19.037 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.038 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.038 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:19.038 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.504666+0000", 2024-05-08T08:52:19.038 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4380000000000002e-06 2024-05-08T08:52:19.038 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.038 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.038 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:19.038 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.504674+0000", 2024-05-08T08:52:19.038 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3569999999999998e-06 2024-05-08T08:52:19.038 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.038 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.038 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:19.039 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.504680+0000", 2024-05-08T08:52:19.039 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4859999999999997e-06 2024-05-08T08:52:19.039 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.039 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.039 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:19.039 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.504722+0000", 2024-05-08T08:52:19.039 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2004e-05 2024-05-08T08:52:19.039 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.039 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.039 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:19.039 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.504795+0000", 2024-05-08T08:52:19.039 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3042999999999998e-05 2024-05-08T08:52:19.039 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.039 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.039 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 2", 2024-05-08T08:52:19.040 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.504915+0000", 2024-05-08T08:52:19.040 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011968699999999999 2024-05-08T08:52:19.040 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.040 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.040 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:19.040 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.507133+0000", 2024-05-08T08:52:19.040 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0022181660000000001 2024-05-08T08:52:19.040 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.040 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.040 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:19.040 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.509322+0000", 2024-05-08T08:52:19.040 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0021888340000000002 2024-05-08T08:52:19.040 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.040 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.040 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:19.041 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.509348+0000", 2024-05-08T08:52:19.041 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6332000000000001e-05 2024-05-08T08:52:19.041 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.041 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.041 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:19.041 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.509375+0000", 2024-05-08T08:52:19.041 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6806999999999999e-05 2024-05-08T08:52:19.041 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:19.041 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:19.041 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:19.041 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.041 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.041 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4465.0:2779 3.b 3:d2f86791:::rbd_data.1140cf79ec85.000000000000002f:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-05-08T08:52:19.041 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:17.504692+0000", 2024-05-08T08:52:19.041 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 1.428042228, 2024-05-08T08:52:19.042 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0049124169999999997, 2024-05-08T08:52:19.042 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:19.042 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:19.042 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:19.042 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:19.042 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4465", 2024-05-08T08:52:19.042 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/390275688", 2024-05-08T08:52:19.042 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2779 2024-05-08T08:52:19.042 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.042 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:19.042 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.042 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:19.042 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.504692+0000", 2024-05-08T08:52:19.042 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:19.043 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.043 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.043 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:19.043 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.504692+0000", 2024-05-08T08:52:19.043 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:19.043 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.043 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.043 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:19.043 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.504692+0000", 2024-05-08T08:52:19.043 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8500000000000002e-07 2024-05-08T08:52:19.043 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.043 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.043 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:19.043 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.504695+0000", 2024-05-08T08:52:19.044 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.322e-06 2024-05-08T08:52:19.044 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.044 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.044 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:19.044 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.504703+0000", 2024-05-08T08:52:19.044 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9100000000000005e-06 2024-05-08T08:52:19.044 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.044 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.044 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:19.044 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.504708+0000", 2024-05-08T08:52:19.044 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3000000000000001e-06 2024-05-08T08:52:19.044 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.044 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.044 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:19.045 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.504751+0000", 2024-05-08T08:52:19.045 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3371999999999997e-05 2024-05-08T08:52:19.045 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.045 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.045 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:19.045 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.504820+0000", 2024-05-08T08:52:19.045 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8287999999999994e-05 2024-05-08T08:52:19.045 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.045 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.045 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 0", 2024-05-08T08:52:19.045 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.504915+0000", 2024-05-08T08:52:19.045 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5087999999999995e-05 2024-05-08T08:52:19.045 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.045 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.046 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:19.046 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.509305+0000", 2024-05-08T08:52:19.046 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0043902680000000001 2024-05-08T08:52:19.046 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.046 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.046 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:19.046 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.509537+0000", 2024-05-08T08:52:19.046 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00023209700000000001 2024-05-08T08:52:19.046 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.046 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.046 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:19.046 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.509566+0000", 2024-05-08T08:52:19.046 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8872000000000001e-05 2024-05-08T08:52:19.046 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.047 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.047 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:19.047 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.509604+0000", 2024-05-08T08:52:19.047 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8515e-05 2024-05-08T08:52:19.047 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:19.047 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:19.047 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:19.047 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.047 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.047 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4465.0:2781 3.19 3:9b6e72ee:::rbd_data.1140cf79ec85.0000000000000031:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-05-08T08:52:19.047 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:17.504719+0000", 2024-05-08T08:52:19.047 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 1.428015555, 2024-05-08T08:52:19.047 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0046410219999999999, 2024-05-08T08:52:19.047 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:19.048 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:19.048 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:19.048 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:19.048 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4465", 2024-05-08T08:52:19.048 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/390275688", 2024-05-08T08:52:19.048 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2781 2024-05-08T08:52:19.048 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.048 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:19.048 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.048 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:19.048 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.504719+0000", 2024-05-08T08:52:19.048 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:19.048 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.048 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.049 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:19.049 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.504719+0000", 2024-05-08T08:52:19.049 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:19.049 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.049 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.049 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:19.049 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.504719+0000", 2024-05-08T08:52:19.049 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.96e-07 2024-05-08T08:52:19.049 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.049 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.049 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:19.049 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.504721+0000", 2024-05-08T08:52:19.049 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1900000000000002e-06 2024-05-08T08:52:19.049 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.050 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.050 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:19.050 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.504728+0000", 2024-05-08T08:52:19.050 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3559999999999998e-06 2024-05-08T08:52:19.050 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.050 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.050 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:19.050 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.504733+0000", 2024-05-08T08:52:19.050 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1279999999999999e-06 2024-05-08T08:52:19.050 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.050 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.050 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:19.050 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.504770+0000", 2024-05-08T08:52:19.050 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7185999999999999e-05 2024-05-08T08:52:19.051 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.051 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.051 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:19.051 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.504827+0000", 2024-05-08T08:52:19.051 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6736e-05 2024-05-08T08:52:19.051 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.051 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.051 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 4", 2024-05-08T08:52:19.051 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.504883+0000", 2024-05-08T08:52:19.051 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6637e-05 2024-05-08T08:52:19.051 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.051 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.051 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:19.051 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.506524+0000", 2024-05-08T08:52:19.052 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0016410800000000001 2024-05-08T08:52:19.052 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.052 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.052 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:19.052 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.509293+0000", 2024-05-08T08:52:19.052 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0027684099999999998 2024-05-08T08:52:19.052 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.052 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.052 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:19.052 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.509338+0000", 2024-05-08T08:52:19.052 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5571999999999997e-05 2024-05-08T08:52:19.052 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.052 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.052 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:19.052 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.509360+0000", 2024-05-08T08:52:19.053 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1331e-05 2024-05-08T08:52:19.053 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:19.053 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:19.053 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:19.053 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.053 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.053 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4465.0:2924 3.4 3:239ea5ea:::rbd_data.1140cf79ec85.0000000000000022:head [set-alloc-hint object_size 4194304 write_size 4194304,write 16384~12288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-05-08T08:52:19.053 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:17.579985+0000", 2024-05-08T08:52:19.053 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 1.3527487060000001, 2024-05-08T08:52:19.053 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0015500550000000001, 2024-05-08T08:52:19.053 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:19.053 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:19.053 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:19.053 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:19.054 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4465", 2024-05-08T08:52:19.054 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/390275688", 2024-05-08T08:52:19.054 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2924 2024-05-08T08:52:19.054 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.054 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:19.054 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.054 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:19.054 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.579985+0000", 2024-05-08T08:52:19.054 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:19.054 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.054 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.054 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:19.054 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.579985+0000", 2024-05-08T08:52:19.054 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:19.055 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.055 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.055 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:19.055 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.579987+0000", 2024-05-08T08:52:19.055 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1000000000000001e-06 2024-05-08T08:52:19.055 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.055 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.055 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:19.055 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.580003+0000", 2024-05-08T08:52:19.055 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6081000000000001e-05 2024-05-08T08:52:19.055 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.055 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.055 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:19.055 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.580014+0000", 2024-05-08T08:52:19.055 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.154e-05 2024-05-08T08:52:19.056 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.056 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.056 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:19.056 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.580020+0000", 2024-05-08T08:52:19.056 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5860000000000004e-06 2024-05-08T08:52:19.056 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.056 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.056 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:19.056 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.580072+0000", 2024-05-08T08:52:19.056 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2367999999999997e-05 2024-05-08T08:52:19.056 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.056 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.056 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:19.056 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.580122+0000", 2024-05-08T08:52:19.057 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9988999999999999e-05 2024-05-08T08:52:19.057 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.057 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.057 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 2", 2024-05-08T08:52:19.057 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.580205+0000", 2024-05-08T08:52:19.057 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.2729e-05 2024-05-08T08:52:19.057 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.057 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.057 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:19.057 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.581160+0000", 2024-05-08T08:52:19.057 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00095505000000000004 2024-05-08T08:52:19.057 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.057 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.057 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:19.058 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.581452+0000", 2024-05-08T08:52:19.058 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00029158899999999998 2024-05-08T08:52:19.058 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.058 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.058 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:19.058 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.581503+0000", 2024-05-08T08:52:19.058 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1672999999999998e-05 2024-05-08T08:52:19.058 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.058 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.058 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:19.058 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.581536+0000", 2024-05-08T08:52:19.058 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.235e-05 2024-05-08T08:52:19.058 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:19.058 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:19.059 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:19.059 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.059 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.059 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(client.4465.0:3047 3.6 e22/18 3:607c4559:::rbd_data.1140cf79ec85.000000000000001f:head v 22'12, mlcod=22'10)", 2024-05-08T08:52:19.059 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:17.637523+0000", 2024-05-08T08:52:19.059 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 1.2952112869999999, 2024-05-08T08:52:19.059 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0033293229999999999, 2024-05-08T08:52:19.059 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:19.059 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:19.059 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:19.059 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:19.059 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.059 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:19.059 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.637523+0000", 2024-05-08T08:52:19.060 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:19.060 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.060 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.060 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:19.060 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.637523+0000", 2024-05-08T08:52:19.060 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:19.060 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.060 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.060 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:19.060 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.637524+0000", 2024-05-08T08:52:19.060 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5000000000000002e-07 2024-05-08T08:52:19.060 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.060 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.060 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:19.060 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.637548+0000", 2024-05-08T08:52:19.061 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4150000000000001e-05 2024-05-08T08:52:19.061 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.061 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.061 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:19.061 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.637560+0000", 2024-05-08T08:52:19.061 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1838000000000001e-05 2024-05-08T08:52:19.061 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.061 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.061 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:19.061 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.637568+0000", 2024-05-08T08:52:19.061 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.2479999999999996e-06 2024-05-08T08:52:19.061 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.061 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.061 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:19.062 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.637628+0000", 2024-05-08T08:52:19.062 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9623000000000002e-05 2024-05-08T08:52:19.062 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.062 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.062 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:19.062 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.637651+0000", 2024-05-08T08:52:19.062 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3779000000000001e-05 2024-05-08T08:52:19.062 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.062 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.062 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:19.062 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.640819+0000", 2024-05-08T08:52:19.062 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0031681220000000002 2024-05-08T08:52:19.062 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.062 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.063 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:19.063 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.640852+0000", 2024-05-08T08:52:19.063 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.2812999999999999e-05 2024-05-08T08:52:19.063 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:19.063 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:19.063 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:19.063 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.063 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.063 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4465.0:3102 3.4 3:239ea5ea:::rbd_data.1140cf79ec85.0000000000000022:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-05-08T08:52:19.063 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:17.675394+0000", 2024-05-08T08:52:19.063 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 1.257340205, 2024-05-08T08:52:19.063 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00167337, 2024-05-08T08:52:19.063 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:19.063 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:19.064 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:19.064 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:19.064 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4465", 2024-05-08T08:52:19.064 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/390275688", 2024-05-08T08:52:19.064 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 3102 2024-05-08T08:52:19.064 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.064 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:19.064 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.064 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:19.064 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.675394+0000", 2024-05-08T08:52:19.064 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:19.064 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.064 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.064 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:19.065 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.675394+0000", 2024-05-08T08:52:19.065 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:19.065 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.065 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.065 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:19.065 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.675395+0000", 2024-05-08T08:52:19.065 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.271e-06 2024-05-08T08:52:19.065 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.065 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.065 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:19.065 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.675400+0000", 2024-05-08T08:52:19.065 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6999999999999999e-06 2024-05-08T08:52:19.065 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.065 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.065 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:19.066 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.675410+0000", 2024-05-08T08:52:19.066 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0547000000000001e-05 2024-05-08T08:52:19.066 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.066 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.066 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:19.066 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.675420+0000", 2024-05-08T08:52:19.066 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1239999999999994e-06 2024-05-08T08:52:19.066 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.066 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.066 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:19.066 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.675482+0000", 2024-05-08T08:52:19.066 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2377000000000003e-05 2024-05-08T08:52:19.066 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.066 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.067 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:19.067 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.675568+0000", 2024-05-08T08:52:19.067 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.6113999999999996e-05 2024-05-08T08:52:19.067 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.067 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.067 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 2", 2024-05-08T08:52:19.067 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.675679+0000", 2024-05-08T08:52:19.067 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000110963 2024-05-08T08:52:19.067 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.067 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.067 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:19.067 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.676334+0000", 2024-05-08T08:52:19.067 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000655228 2024-05-08T08:52:19.067 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.067 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.068 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:19.068 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.676987+0000", 2024-05-08T08:52:19.068 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00065267999999999999 2024-05-08T08:52:19.068 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.068 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.068 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:19.068 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.677041+0000", 2024-05-08T08:52:19.068 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3826999999999998e-05 2024-05-08T08:52:19.068 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.068 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.068 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:19.068 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.677067+0000", 2024-05-08T08:52:19.068 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6539000000000001e-05 2024-05-08T08:52:19.068 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:19.068 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:19.069 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:19.069 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.069 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.069 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:1123 3.2 3:4550e1e5:::rbd_id.image_client.0-clone1:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:52:19.069 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:17.982305+0000", 2024-05-08T08:52:19.069 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.95042889399999997, 2024-05-08T08:52:19.069 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0035903520000000002, 2024-05-08T08:52:19.069 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:19.069 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:19.069 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:19.069 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:19.069 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:52:19.069 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:52:19.069 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1123 2024-05-08T08:52:19.070 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.070 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:19.070 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.070 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:19.070 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.982305+0000", 2024-05-08T08:52:19.070 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:19.070 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.070 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.070 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:19.070 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.982305+0000", 2024-05-08T08:52:19.070 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:19.070 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.070 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.070 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:19.071 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.982306+0000", 2024-05-08T08:52:19.071 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1739999999999999e-06 2024-05-08T08:52:19.071 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.071 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.071 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:19.071 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.982310+0000", 2024-05-08T08:52:19.071 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8859999999999997e-06 2024-05-08T08:52:19.071 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.071 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.071 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:19.071 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.982318+0000", 2024-05-08T08:52:19.071 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4039999999999998e-06 2024-05-08T08:52:19.071 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.071 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.071 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:19.072 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.982328+0000", 2024-05-08T08:52:19.072 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8060000000000006e-06 2024-05-08T08:52:19.072 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.072 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.072 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:19.072 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.982392+0000", 2024-05-08T08:52:19.072 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4939999999999998e-05 2024-05-08T08:52:19.072 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.072 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.072 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op must wait for map", 2024-05-08T08:52:19.072 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.982397+0000", 2024-05-08T08:52:19.072 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4050000000000002e-06 2024-05-08T08:52:19.072 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.072 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.073 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:19.073 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.984416+0000", 2024-05-08T08:52:19.073 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.002019004 2024-05-08T08:52:19.073 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.073 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.073 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:19.073 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.984545+0000", 2024-05-08T08:52:19.073 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000129567 2024-05-08T08:52:19.073 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.073 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.073 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 5", 2024-05-08T08:52:19.073 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.984774+0000", 2024-05-08T08:52:19.073 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000228562 2024-05-08T08:52:19.073 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.073 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.074 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:19.074 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.985542+0000", 2024-05-08T08:52:19.074 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000768226 2024-05-08T08:52:19.074 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.074 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.074 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:19.074 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.985785+0000", 2024-05-08T08:52:19.074 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000242871 2024-05-08T08:52:19.074 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.074 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.074 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:19.074 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.985851+0000", 2024-05-08T08:52:19.074 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6235000000000005e-05 2024-05-08T08:52:19.074 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:19.075 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:19.075 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:19.075 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.985896+0000", 2024-05-08T08:52:19.075 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4271999999999999e-05 2024-05-08T08:52:19.075 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:19.075 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:19.075 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:19.075 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:19.075 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:19.075 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:52:19.075 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-05-08T08:52:19.076 INFO:teuthology.orchestra.run.smithi196.stdout:truncating image image_client.0-clone1 from 0xdd4f600 (overlap 0xdd4f600) to 0x0 2024-05-08T08:52:19.077 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:52:19.078 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:52:19.078 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:52:19.078 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:52:19.079 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:52:19.079 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:52:19.080 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:52:19.080 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:52:19.080+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:52:19.085 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:52:19.086 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:52:19.123 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:52:19.123 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:52:19.166 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:52:19.167 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:52:19.177 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:52:19.181+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:52:19.276 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:19.276 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:19.278 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:19.281+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:19.289 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:52:19.379 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:19.383+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:19.412 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:19.412 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:19.422 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:52:19.480 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:52:19.484+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:52:19.533 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:19.533 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:19.542 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:52:19.581 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:52:19.585+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:52:19.653 INFO:teuthology.orchestra.run.smithi196.stdout:checking clone #0, image image_client.0 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent1 2024-05-08T08:52:19.654 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:19.654 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:19.663 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:52:19.706 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:19.710+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:19.774 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:19.774 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:19.783 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:52:19.784 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:52:19.786+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:52:19.885 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:52:19.888+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:52:19.938 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:19.939 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:19.948 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:52:19.950 INFO:teuthology.orchestra.run.smithi059.stderr:nodeep-scrub is unset 2024-05-08T08:52:19.991 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:19.995+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:20.057 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:20.057 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:20.066 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:52:20.086 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:52:20.090+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:52:20.188 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:52:20.192+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:52:20.212 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:20.212 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:20.218 INFO:teuthology.orchestra.run.smithi196.stdout:543 read 0xbf60a00 thru 0xbf67cae (0x72af bytes) 2024-05-08T08:52:20.219 INFO:teuthology.orchestra.run.smithi196.stdout:544 write 0xcff2600 thru 0xcff7af3 (0x54f4 bytes) 2024-05-08T08:52:20.219 INFO:teuthology.orchestra.run.smithi196.stdout:548 read 0xd4c0200 thru 0xd4c0f44 (0xd45 bytes) 2024-05-08T08:52:20.221 INFO:teuthology.orchestra.run.smithi196.stdout:549 read 0x3b96200 thru 0x3ba1688 (0xb489 bytes) 2024-05-08T08:52:20.221 INFO:teuthology.orchestra.run.smithi196.stdout:550 write 0x43ae200 thru 0x43bcbe5 (0xe9e6 bytes) 2024-05-08T08:52:20.222 INFO:teuthology.orchestra.run.smithi196.stdout:552 read 0x1905a00 thru 0x190dc1c (0x821d bytes) 2024-05-08T08:52:20.222 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:52:20.231 INFO:teuthology.orchestra.run.smithi196.stdout:553 trunc from 0xdd4f600 to 0x52e3400 2024-05-08T08:52:20.273 INFO:teuthology.orchestra.run.smithi196.stdout:554 read 0x25c5600 thru 0x25cda7e (0x847f bytes) 2024-05-08T08:52:20.281 INFO:teuthology.orchestra.run.smithi196.stdout:560 trunc from 0x52e3400 to 0xbdf6200 2024-05-08T08:52:20.288 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:20.292+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:20.291 INFO:teuthology.orchestra.run.smithi196.stdout:561 punch from 0x7196200 to 0x719b400, (0x5200 bytes) 2024-05-08T08:52:20.297 INFO:teuthology.orchestra.run.smithi196.stdout:562 write 0x301de00 thru 0x3024bf2 (0x6df3 bytes) 2024-05-08T08:52:20.305 INFO:teuthology.orchestra.run.smithi196.stdout:563 trunc from 0xbdf6200 to 0x5368000 2024-05-08T08:52:20.335 INFO:teuthology.orchestra.run.smithi196.stdout:566 read 0x60b800 thru 0x61528c (0x9a8d bytes) 2024-05-08T08:52:20.337 INFO:teuthology.orchestra.run.smithi196.stdout:567 read 0x1e32c00 thru 0x1e3b269 (0x866a bytes) 2024-05-08T08:52:20.337 INFO:teuthology.orchestra.run.smithi196.stdout:569 write 0x3ed5a00 thru 0x3ed7b5d (0x215e bytes) 2024-05-08T08:52:20.342 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:20.342 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:20.354 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:52:20.361 INFO:teuthology.orchestra.run.smithi196.stdout:570 write 0x71a6a00 thru 0x71b11b4 (0xa7b5 bytes) 2024-05-08T08:52:20.362 INFO:teuthology.orchestra.run.smithi196.stdout:572 trunc from 0x71b1200 to 0x753c200 2024-05-08T08:52:20.378 INFO:teuthology.orchestra.run.smithi196.stdout:573 punch from 0x3f7ba00 to 0x3f85000, (0x9600 bytes) 2024-05-08T08:52:20.389 INFO:teuthology.orchestra.run.smithi196.stdout:574 trunc from 0x753c200 to 0x481800 2024-05-08T08:52:20.390 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:52:20.393+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:52:20.446 INFO:teuthology.orchestra.run.smithi196.stdout:575 punch from 0x10bc00 to 0x114a00, (0x8e00 bytes) 2024-05-08T08:52:20.452 INFO:teuthology.orchestra.run.smithi196.stdout:576 read 0x36b000 thru 0x37943c (0xe43d bytes) 2024-05-08T08:52:20.483 INFO:teuthology.orchestra.run.smithi196.stdout:577 write 0xebe0000 thru 0xebed6ff (0xd700 bytes) 2024-05-08T08:52:20.484 INFO:teuthology.orchestra.run.smithi196.stdout:578 read 0x2127600 thru 0x21280a1 (0xaa2 bytes) 2024-05-08T08:52:20.484 INFO:teuthology.orchestra.run.smithi196.stdout:582 read 0x4cd1400 thru 0x4cd9d02 (0x8903 bytes) 2024-05-08T08:52:20.484 INFO:teuthology.orchestra.run.smithi196.stdout:583 punch from 0x88d7e00 to 0x88e5000, (0xd200 bytes) 2024-05-08T08:52:20.487 INFO:teuthology.orchestra.run.smithi196.stdout:584 write 0xafd8200 thru 0xafe4396 (0xc197 bytes) 2024-05-08T08:52:20.487 INFO:teuthology.orchestra.run.smithi196.stdout:585 write 0x4837c00 thru 0x4843fcb (0xc3cc bytes) 2024-05-08T08:52:20.490 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:20.494+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:20.494 INFO:teuthology.orchestra.run.smithi196.stdout:586 trunc from 0xebed800 to 0x9443c00 2024-05-08T08:52:20.521 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:20.521 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:20.524 INFO:teuthology.orchestra.run.smithi196.stdout:587 read 0x2daa600 thru 0x2dacc55 (0x2656 bytes) 2024-05-08T08:52:20.525 INFO:teuthology.orchestra.run.smithi196.stdout:589 read 0x4c6e400 thru 0x4c6fff1 (0x1bf2 bytes) 2024-05-08T08:52:20.530 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:52:20.535 INFO:teuthology.orchestra.run.smithi196.stdout:591 write 0xb158a00 thru 0xb16336d (0xa96e bytes) 2024-05-08T08:52:20.536 INFO:teuthology.orchestra.run.smithi196.stdout:592 read 0x51f9e00 thru 0x520815e (0xe35f bytes) 2024-05-08T08:52:20.537 INFO:teuthology.orchestra.run.smithi196.stdout:594 trunc from 0xb163400 to 0xbfc9800 2024-05-08T08:52:20.550 INFO:teuthology.orchestra.run.smithi196.stdout:598 read 0x98cee00 thru 0x98d808c (0x928d bytes) 2024-05-08T08:52:20.551 INFO:teuthology.orchestra.run.smithi196.stdout:600 read 0x4c85000 thru 0x4c8fe68 (0xae69 bytes) 2024-05-08T08:52:20.552 INFO:teuthology.orchestra.run.smithi196.stdout:602 write 0x5b2d400 thru 0x5b34745 (0x7346 bytes) 2024-05-08T08:52:20.552 INFO:teuthology.orchestra.run.smithi196.stdout:603 read 0x34f2c00 thru 0x34fdb48 (0xaf49 bytes) 2024-05-08T08:52:20.555 INFO:teuthology.orchestra.run.smithi196.stdout:604 trunc from 0xbfc9800 to 0x91ab000 2024-05-08T08:52:20.578 INFO:teuthology.orchestra.run.smithi196.stdout:605 read 0x453ae00 thru 0x4546ebf (0xc0c0 bytes) 2024-05-08T08:52:20.583 INFO:teuthology.orchestra.run.smithi196.stdout:606 write 0x6e5600 thru 0x6eb733 (0x6134 bytes) 2024-05-08T08:52:20.583 INFO:teuthology.orchestra.run.smithi196.stdout:611 write 0x2ca2400 thru 0x2cacd5e (0xa95f bytes) 2024-05-08T08:52:20.591 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:52:20.594+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:52:20.605 INFO:teuthology.orchestra.run.smithi196.stdout:612 write 0xdc50200 thru 0xdc5b12e (0xaf2f bytes) 2024-05-08T08:52:20.612 INFO:teuthology.orchestra.run.smithi196.stdout:618 write 0xc2b0200 thru 0xc2be59b (0xe39c bytes) 2024-05-08T08:52:20.612 INFO:teuthology.orchestra.run.smithi196.stdout:619 punch from 0x7573e00 to 0x7579c00, (0x5e00 bytes) 2024-05-08T08:52:20.613 INFO:teuthology.orchestra.run.smithi196.stdout:626 write 0x25fb200 thru 0x2602774 (0x7575 bytes) 2024-05-08T08:52:20.613 INFO:teuthology.orchestra.run.smithi196.stdout:628 write 0x9b9da00 thru 0x9ba5872 (0x7e73 bytes) 2024-05-08T08:52:20.614 INFO:teuthology.orchestra.run.smithi196.stdout:629 write 0x96dfc00 thru 0x96e8d67 (0x9168 bytes) 2024-05-08T08:52:20.629 INFO:teuthology.orchestra.run.smithi196.stdout:633 trunc from 0xdc5b200 to 0x5b9800 2024-05-08T08:52:20.687 INFO:teuthology.orchestra.run.smithi196.stdout:635 read 0x2ffa00 thru 0x301bb2 (0x21b3 bytes) 2024-05-08T08:52:20.691 INFO:teuthology.orchestra.run.smithi196.stdout:636 trunc from 0x5b9800 to 0x3399000 2024-05-08T08:52:20.692 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:20.692 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:20.692 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:20.695+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:20.701 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:52:20.705 INFO:teuthology.orchestra.run.smithi196.stdout:637 write 0x59da00 thru 0x59f670 (0x1c71 bytes) 2024-05-08T08:52:20.706 INFO:teuthology.orchestra.run.smithi196.stdout:641 read 0x2ff7800 thru 0x2ffaa59 (0x325a bytes) 2024-05-08T08:52:20.728 INFO:teuthology.orchestra.run.smithi196.stdout:642 write 0x9952600 thru 0x99585a0 (0x5fa1 bytes) 2024-05-08T08:52:20.735 INFO:teuthology.orchestra.run.smithi196.stdout:644 read 0x77c4600 thru 0x77cb3ac (0x6dad bytes) 2024-05-08T08:52:20.735 INFO:teuthology.orchestra.run.smithi196.stdout:646 punch from 0x814aa00 to 0x8157a00, (0xd000 bytes) 2024-05-08T08:52:20.736 INFO:teuthology.orchestra.run.smithi196.stdout:647 write 0x81efa00 thru 0x81f1045 (0x1646 bytes) 2024-05-08T08:52:20.736 INFO:teuthology.orchestra.run.smithi196.stdout:649 read 0x636a800 thru 0x636bfd0 (0x17d1 bytes) 2024-05-08T08:52:20.739 INFO:teuthology.orchestra.run.smithi196.stdout:651 trunc from 0x9958600 to 0x754d600 2024-05-08T08:52:20.758 INFO:teuthology.orchestra.run.smithi196.stdout:653 read 0x4a6c800 thru 0x4a70d9a (0x459b bytes) 2024-05-08T08:52:20.759 INFO:teuthology.orchestra.run.smithi196.stdout:654 read 0x3a18a00 thru 0x3a24c36 (0xc237 bytes) 2024-05-08T08:52:20.778 INFO:teuthology.orchestra.run.smithi196.stdout:655 write 0x871c600 thru 0x871c697 (0x98 bytes) 2024-05-08T08:52:20.779 INFO:teuthology.orchestra.run.smithi196.stdout:657 read 0x44b6200 thru 0x44bdaa6 (0x78a7 bytes) 2024-05-08T08:52:20.793 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:20.796+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:20.803 INFO:teuthology.orchestra.run.smithi196.stdout:658 write 0xda85200 thru 0xda8caf1 (0x78f2 bytes) 2024-05-08T08:52:20.803 INFO:teuthology.orchestra.run.smithi196.stdout:660 read 0x2341400 thru 0x234b54a (0xa14b bytes) 2024-05-08T08:52:20.803 INFO:teuthology.orchestra.run.smithi196.stdout:662 read 0x8159200 thru 0x815b1bd (0x1fbe bytes) 2024-05-08T08:52:20.803 INFO:teuthology.orchestra.run.smithi196.stdout:664 read 0xa368600 thru 0xa371f42 (0x9943 bytes) 2024-05-08T08:52:20.803 INFO:teuthology.orchestra.run.smithi196.stdout:667 read 0xf01a00 thru 0xf0b122 (0x9723 bytes) 2024-05-08T08:52:20.815 INFO:teuthology.orchestra.run.smithi196.stdout:668 write 0xda8d600 thru 0xda95392 (0x7d93 bytes) 2024-05-08T08:52:20.817 INFO:teuthology.orchestra.run.smithi196.stdout:670 read 0x8788200 thru 0x8790353 (0x8154 bytes) 2024-05-08T08:52:20.817 INFO:teuthology.orchestra.run.smithi196.stdout:671 punch from 0x36d5200 to 0x36e4000, (0xee00 bytes) 2024-05-08T08:52:20.819 INFO:teuthology.orchestra.run.smithi196.stdout:672 read 0x377d200 thru 0x377da90 (0x891 bytes) 2024-05-08T08:52:20.819 INFO:teuthology.orchestra.run.smithi196.stdout:674 write 0xb9a3600 thru 0xb9aa9b1 (0x73b2 bytes) 2024-05-08T08:52:20.819 INFO:teuthology.orchestra.run.smithi196.stdout:675 write 0x26d9a00 thru 0x26e89cf (0xefd0 bytes) 2024-05-08T08:52:20.819 INFO:teuthology.orchestra.run.smithi196.stdout:677 punch from 0x2a6e200 to 0x2a75c00, (0x7a00 bytes) 2024-05-08T08:52:20.820 INFO:teuthology.orchestra.run.smithi196.stdout:679 trunc from 0xda95400 to 0xc7b1c00 2024-05-08T08:52:20.855 INFO:teuthology.orchestra.run.smithi196.stdout:681 read 0x7099400 thru 0x70a549b (0xc09c bytes) 2024-05-08T08:52:20.857 INFO:teuthology.orchestra.run.smithi196.stdout:682 trunc from 0xc7b1c00 to 0xb6c8800 2024-05-08T08:52:20.865 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:20.865 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:20.867 INFO:teuthology.orchestra.run.smithi196.stdout:683 read 0x45bf600 thru 0x45cc4ed (0xceee bytes) 2024-05-08T08:52:20.868 INFO:teuthology.orchestra.run.smithi196.stdout:684 write 0xa8e9400 thru 0xa8f8f99 (0xfb9a bytes) 2024-05-08T08:52:20.868 INFO:teuthology.orchestra.run.smithi196.stdout:685 read 0x4ecfa00 thru 0x4edaaca (0xb0cb bytes) 2024-05-08T08:52:20.869 INFO:teuthology.orchestra.run.smithi196.stdout:688 read 0x17c5600 thru 0x17c8ffc (0x39fd bytes) 2024-05-08T08:52:20.869 INFO:teuthology.orchestra.run.smithi196.stdout:690 read 0xa06a200 thru 0xa06ed4d (0x4b4e bytes) 2024-05-08T08:52:20.869 INFO:teuthology.orchestra.run.smithi196.stdout:691 punch from 0x63d5400 to 0x63e3000, (0xdc00 bytes) 2024-05-08T08:52:20.869 INFO:teuthology.orchestra.run.smithi196.stdout:693 read 0x44b5200 thru 0x44bfee4 (0xace5 bytes) 2024-05-08T08:52:20.870 INFO:teuthology.orchestra.run.smithi196.stdout:695 read 0xdc8a00 thru 0xdcacad (0x22ae bytes) 2024-05-08T08:52:20.870 INFO:teuthology.orchestra.run.smithi196.stdout:698 read 0x87600 thru 0x8ad68 (0x3769 bytes) 2024-05-08T08:52:20.877 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:52:20.878 INFO:teuthology.orchestra.run.smithi196.stdout:699 read 0xcd8600 thru 0xcd8623 (0x24 bytes) 2024-05-08T08:52:20.878 INFO:teuthology.orchestra.run.smithi196.stdout:700 write 0x9bdbe00 thru 0x9be713e (0xb33f bytes) 2024-05-08T08:52:20.879 INFO:teuthology.orchestra.run.smithi196.stdout:702 write 0x1c7f400 thru 0x1c8d84d (0xe44e bytes) 2024-05-08T08:52:20.879 INFO:teuthology.orchestra.run.smithi196.stdout:705 read 0x2fafa00 thru 0x2fb2d77 (0x3378 bytes) 2024-05-08T08:52:20.879 INFO:teuthology.orchestra.run.smithi196.stdout:707 write 0x90e5e00 thru 0x90e8ace (0x2ccf bytes) 2024-05-08T08:52:20.879 INFO:teuthology.orchestra.run.smithi196.stdout:709 write 0x9287c00 thru 0x92940e3 (0xc4e4 bytes) 2024-05-08T08:52:20.879 INFO:teuthology.orchestra.run.smithi196.stdout:710 read 0x7dc4c00 thru 0x7dcfb1e (0xaf1f bytes) 2024-05-08T08:52:20.884 INFO:teuthology.orchestra.run.smithi196.stdout:712 trunc from 0xb6c8800 to 0x57ca400 2024-05-08T08:52:20.923 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:52:20.926+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:52:20.929 INFO:teuthology.orchestra.run.smithi196.stdout:714 punch from 0x1ea4200 to 0x1eb0a00, (0xc800 bytes) 2024-05-08T08:52:20.932 INFO:teuthology.orchestra.run.smithi196.stdout:718 read 0x4c8e00 thru 0x4d5e96 (0xd097 bytes) 2024-05-08T08:52:20.938 INFO:teuthology.orchestra.run.smithi196.stdout:720 trunc from 0x57ca400 to 0x2ea2c00 2024-05-08T08:52:20.973 INFO:teuthology.orchestra.run.smithi196.stdout:721 write 0x9759600 thru 0x9763dfc (0xa7fd bytes) 2024-05-08T08:52:20.973 INFO:teuthology.orchestra.run.smithi196.stdout:724 write 0x1e7f600 thru 0x1e8f1e1 (0xfbe2 bytes) 2024-05-08T08:52:20.973 INFO:teuthology.orchestra.run.smithi196.stdout:726 read 0x1775600 thru 0x178303c (0xda3d bytes) 2024-05-08T08:52:20.973 INFO:teuthology.orchestra.run.smithi196.stdout:727 punch from 0x6084400 to 0x608aa00, (0x6600 bytes) 2024-05-08T08:52:20.973 INFO:teuthology.orchestra.run.smithi196.stdout:728 punch from 0x8c22000 to 0x8c2ec00, (0xcc00 bytes) 2024-05-08T08:52:20.974 INFO:teuthology.orchestra.run.smithi196.stdout:729 write 0x4587a00 thru 0x458a3e0 (0x29e1 bytes) 2024-05-08T08:52:20.974 INFO:teuthology.orchestra.run.smithi196.stdout:730 read 0x4763c00 thru 0x476774e (0x3b4f bytes) 2024-05-08T08:52:20.994 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:52:20.998+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:52:21.007 INFO:teuthology.orchestra.run.smithi196.stdout:732 write 0xb961000 thru 0xb96d6ef (0xc6f0 bytes) 2024-05-08T08:52:21.007 INFO:teuthology.orchestra.run.smithi196.stdout:734 punch from 0x34dfe00 to 0x34e9800, (0x9a00 bytes) 2024-05-08T08:52:21.014 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:21.014 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:21.017 INFO:teuthology.orchestra.run.smithi196.stdout:739 punch from 0x9140e00 to 0x914a200, (0x9400 bytes) 2024-05-08T08:52:21.017 INFO:teuthology.orchestra.run.smithi196.stdout:740 read 0x2a4c00 thru 0x2ae06c (0x946d bytes) 2024-05-08T08:52:21.018 INFO:teuthology.orchestra.run.smithi196.stdout:741 write 0xa88ac00 thru 0xa893c83 (0x9084 bytes) 2024-05-08T08:52:21.025 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:52:21.030 INFO:teuthology.orchestra.run.smithi196.stdout:742 trunc from 0xb96d800 to 0xd67c00 2024-05-08T08:52:21.092 INFO:teuthology.orchestra.run.smithi196.stdout:744 write 0x2a20e00 thru 0x2a2b2f2 (0xa4f3 bytes) 2024-05-08T08:52:21.095 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:21.099+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:21.097 INFO:teuthology.orchestra.run.smithi196.stdout:745 read 0x28a000 thru 0x293636 (0x9637 bytes) 2024-05-08T08:52:21.114 INFO:teuthology.orchestra.run.smithi196.stdout:746 write 0x3fa8600 thru 0x3fb825a (0xfc5b bytes) 2024-05-08T08:52:21.127 INFO:teuthology.orchestra.run.smithi196.stdout:747 trunc from 0x3fb8400 to 0xe89c600 2024-05-08T08:52:21.145 INFO:teuthology.orchestra.run.smithi196.stdout:749 read 0x39fc600 thru 0x3a02ed9 (0x68da bytes) 2024-05-08T08:52:21.146 INFO:teuthology.orchestra.run.smithi196.stdout:750 write 0x8dbc000 thru 0x8dc72ee (0xb2ef bytes) 2024-05-08T08:52:21.146 INFO:teuthology.orchestra.run.smithi196.stdout:753 write 0x368c800 thru 0x3693ef7 (0x76f8 bytes) 2024-05-08T08:52:21.146 INFO:teuthology.orchestra.run.smithi196.stdout:754 write 0xbf13000 thru 0xbf15e1a (0x2e1b bytes) 2024-05-08T08:52:21.147 INFO:teuthology.orchestra.run.smithi196.stdout:755 flatten 2024-05-08T08:52:21.167 INFO:teuthology.orchestra.run.smithi196.stdout:757 read 0xc460a00 thru 0xc46b8c2 (0xaec3 bytes) 2024-05-08T08:52:21.180 INFO:teuthology.orchestra.run.smithi196.stdout:760 trunc from 0xe89c600 to 0x4923400 2024-05-08T08:52:21.190 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:21.190 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:21.191 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T08:52:21.197 INFO:teuthology.orchestra.run.smithi196.stdout:761 read 0x1f99600 thru 0x1f9d5a4 (0x3fa5 bytes) 2024-05-08T08:52:21.200 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:52:21.200 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:52:21.203+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:52:21.210 INFO:teuthology.orchestra.run.smithi196.stdout:765 write 0x5f54a00 thru 0x5f613e7 (0xc9e8 bytes) 2024-05-08T08:52:21.210 INFO:teuthology.orchestra.run.smithi196.stdout:766 punch from 0x1fe3600 to 0x1ff0800, (0xd200 bytes) 2024-05-08T08:52:21.230 INFO:teuthology.orchestra.run.smithi196.stdout:767 write 0xe5a4c00 thru 0xe5a99a8 (0x4da9 bytes) 2024-05-08T08:52:21.231 INFO:teuthology.orchestra.run.smithi196.stdout:770 read 0x8123e00 thru 0x8129f8b (0x618c bytes) 2024-05-08T08:52:21.231 INFO:teuthology.orchestra.run.smithi196.stdout:771 punch from 0xaf18000 to 0xaf19600, (0x1600 bytes) 2024-05-08T08:52:21.236 INFO:teuthology.orchestra.run.smithi196.stdout:772 read 0x4568400 thru 0x4577cf4 (0xf8f5 bytes) 2024-05-08T08:52:21.244 INFO:teuthology.orchestra.run.smithi196.stdout:774 trunc from 0xe5a9a00 to 0x7772c00 2024-05-08T08:52:21.270 INFO:teuthology.orchestra.run.smithi196.stdout:775 write 0xe0e200 thru 0xe1471f (0x6520 bytes) 2024-05-08T08:52:21.271 INFO:teuthology.orchestra.run.smithi196.stdout:776 trunc from 0x7772c00 to 0x8b7a400 2024-05-08T08:52:21.295 INFO:teuthology.orchestra.run.smithi196.stdout:777 trunc from 0x8b7a400 to 0xd7bfc00 2024-05-08T08:52:21.296 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:21.300+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:21.304 INFO:teuthology.orchestra.run.smithi196.stdout:779 punch from 0x2ff2e00 to 0x2ff8600, (0x5800 bytes) 2024-05-08T08:52:21.321 INFO:teuthology.orchestra.run.smithi196.stdout:782 write 0xeb90000 thru 0xeb97e38 (0x7e39 bytes) 2024-05-08T08:52:21.330 INFO:teuthology.orchestra.run.smithi196.stdout:784 trunc from 0xeb98000 to 0x7918200 2024-05-08T08:52:21.338 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:52:21.338 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:52:21.348 INFO:teuthology.orchestra.run.smithi196.stdout:785 write 0x3f13a00 thru 0x3f1dc68 (0xa269 bytes) 2024-05-08T08:52:21.349 INFO:teuthology.orchestra.run.smithi196.stdout:786 read 0x2a18800 thru 0x2a1d714 (0x4f15 bytes) 2024-05-08T08:52:21.349 INFO:teuthology.orchestra.run.smithi196.stdout:788 read 0x2573000 thru 0x258083b (0xd83c bytes) 2024-05-08T08:52:21.359 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:52:21.359 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:52:21.370 INFO:teuthology.orchestra.run.smithi196.stdout:790 write 0xcbf4200 thru 0xcc02b25 (0xe926 bytes) 2024-05-08T08:52:21.371 INFO:teuthology.orchestra.run.smithi196.stdout:791 write 0xb1d1800 thru 0xb1d542a (0x3c2b bytes) 2024-05-08T08:52:21.371 INFO:teuthology.orchestra.run.smithi196.stdout:793 write 0xbed4200 thru 0xbee1818 (0xd619 bytes) 2024-05-08T08:52:21.371 INFO:teuthology.orchestra.run.smithi196.stdout:794 write 0x47d2a00 thru 0x47d8c4c (0x624d bytes) 2024-05-08T08:52:21.371 INFO:teuthology.orchestra.run.smithi196.stdout:795 read 0x1261a00 thru 0x126a156 (0x8757 bytes) 2024-05-08T08:52:21.371 INFO:teuthology.orchestra.run.smithi196.stdout:797 read 0x446e00 thru 0x45213b (0xb33c bytes) 2024-05-08T08:52:21.373 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:52:21.373 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:52:21.373 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:52:21.373 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:52:21.373 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.373 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4465.0:1320 3.17 3:e8515fe1:::rbd_object_map.1140cf79ec85:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-05-08T08:52:21.373 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:16.752452+0000", 2024-05-08T08:52:21.373 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 4.6221823259999999, 2024-05-08T08:52:21.373 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.007163287, 2024-05-08T08:52:21.373 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:21.373 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:21.374 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:21.374 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:21.374 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4465", 2024-05-08T08:52:21.374 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/390275688", 2024-05-08T08:52:21.374 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1320 2024-05-08T08:52:21.374 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.374 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:21.374 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.374 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:21.374 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.752452+0000", 2024-05-08T08:52:21.374 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:21.374 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.374 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.375 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:21.375 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.752452+0000", 2024-05-08T08:52:21.375 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:21.375 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.375 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.375 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:21.375 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.752452+0000", 2024-05-08T08:52:21.375 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7700000000000004e-07 2024-05-08T08:52:21.375 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.375 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.375 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:21.375 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.752456+0000", 2024-05-08T08:52:21.375 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5949999999999999e-06 2024-05-08T08:52:21.376 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.376 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.376 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:21.376 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.752465+0000", 2024-05-08T08:52:21.376 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.9469999999999992e-06 2024-05-08T08:52:21.376 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.376 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.376 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:21.376 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.752469+0000", 2024-05-08T08:52:21.376 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3270000000000002e-06 2024-05-08T08:52:21.376 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.376 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.376 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:21.377 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.753378+0000", 2024-05-08T08:52:21.377 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000909132 2024-05-08T08:52:21.377 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.377 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.377 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:21.377 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.753474+0000", 2024-05-08T08:52:21.377 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5210000000000005e-05 2024-05-08T08:52:21.377 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.377 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.377 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 5", 2024-05-08T08:52:21.377 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.753983+0000", 2024-05-08T08:52:21.377 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00050976000000000005 2024-05-08T08:52:21.377 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.377 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.378 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:21.378 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.757203+0000", 2024-05-08T08:52:21.378 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0032196799999999999 2024-05-08T08:52:21.378 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.378 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.378 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:21.378 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.759560+0000", 2024-05-08T08:52:21.378 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.002356679 2024-05-08T08:52:21.378 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.378 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.379 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:21.379 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.759591+0000", 2024-05-08T08:52:21.379 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0879e-05 2024-05-08T08:52:21.379 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.379 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.379 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:21.379 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.759615+0000", 2024-05-08T08:52:21.379 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4501000000000001e-05 2024-05-08T08:52:21.379 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:21.379 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:21.379 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:21.380 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.380 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.380 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4465.0:1321 3.17 3:e8515fe1:::rbd_object_map.1140cf79ec85:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-05-08T08:52:21.380 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:16.752563+0000", 2024-05-08T08:52:21.380 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 4.6220715810000002, 2024-05-08T08:52:21.380 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0072053339999999999, 2024-05-08T08:52:21.380 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:21.380 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:21.380 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:21.380 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:21.380 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4465", 2024-05-08T08:52:21.380 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/390275688", 2024-05-08T08:52:21.380 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1321 2024-05-08T08:52:21.380 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.381 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:21.381 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.381 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:21.381 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.752563+0000", 2024-05-08T08:52:21.381 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:21.381 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.381 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.381 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:21.381 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.752563+0000", 2024-05-08T08:52:21.381 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:21.381 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.381 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.381 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:21.381 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.752563+0000", 2024-05-08T08:52:21.382 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.15e-07 2024-05-08T08:52:21.382 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.382 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.382 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:21.382 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.752566+0000", 2024-05-08T08:52:21.382 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4899999999999999e-06 2024-05-08T08:52:21.382 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.382 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.382 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:21.382 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.752572+0000", 2024-05-08T08:52:21.382 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5010000000000001e-06 2024-05-08T08:52:21.382 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.382 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.383 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:21.383 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.752575+0000", 2024-05-08T08:52:21.383 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.2339999999999999e-06 2024-05-08T08:52:21.383 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.383 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.383 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:21.383 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.754655+0000", 2024-05-08T08:52:21.383 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0020801909999999999 2024-05-08T08:52:21.383 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.383 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.383 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:21.383 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.754726+0000", 2024-05-08T08:52:21.383 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0980000000000001e-05 2024-05-08T08:52:21.383 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.384 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.384 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 5", 2024-05-08T08:52:21.384 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.754997+0000", 2024-05-08T08:52:21.384 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000270664 2024-05-08T08:52:21.384 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.384 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.384 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:21.384 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.757211+0000", 2024-05-08T08:52:21.384 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.002213867 2024-05-08T08:52:21.384 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.384 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.384 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:21.384 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.759663+0000", 2024-05-08T08:52:21.384 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0024516870000000001 2024-05-08T08:52:21.385 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.385 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.385 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:21.385 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.759722+0000", 2024-05-08T08:52:21.385 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9524000000000001e-05 2024-05-08T08:52:21.385 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.385 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.385 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:21.385 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.759768+0000", 2024-05-08T08:52:21.385 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5781000000000001e-05 2024-05-08T08:52:21.385 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:21.385 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:21.385 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:21.385 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.385 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.385 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4465.0:1322 3.17 3:e8515fe1:::rbd_object_map.1140cf79ec85:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-05-08T08:52:21.386 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:16.752649+0000", 2024-05-08T08:52:21.386 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 4.6219853080000002, 2024-05-08T08:52:21.386 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0074330530000000002, 2024-05-08T08:52:21.386 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:21.386 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:21.386 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:21.386 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:21.386 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4465", 2024-05-08T08:52:21.386 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/390275688", 2024-05-08T08:52:21.386 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1322 2024-05-08T08:52:21.386 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.386 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:21.386 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.387 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:21.387 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.752649+0000", 2024-05-08T08:52:21.387 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:21.387 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.387 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.387 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:21.387 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.752649+0000", 2024-05-08T08:52:21.387 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:21.387 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.387 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.387 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:21.387 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.752649+0000", 2024-05-08T08:52:21.387 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0800000000000001e-07 2024-05-08T08:52:21.388 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.388 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.388 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:21.388 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.752651+0000", 2024-05-08T08:52:21.388 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0140000000000001e-06 2024-05-08T08:52:21.388 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.388 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.388 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:21.388 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.752656+0000", 2024-05-08T08:52:21.388 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4950000000000002e-06 2024-05-08T08:52:21.388 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.388 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.388 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:21.389 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.752659+0000", 2024-05-08T08:52:21.389 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9299999999999999e-06 2024-05-08T08:52:21.389 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.389 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.389 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:21.389 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.755409+0000", 2024-05-08T08:52:21.389 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0027508189999999998 2024-05-08T08:52:21.389 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.389 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.389 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:21.389 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.755498+0000", 2024-05-08T08:52:21.389 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.9012999999999996e-05 2024-05-08T08:52:21.389 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.389 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.389 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 5", 2024-05-08T08:52:21.390 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.755977+0000", 2024-05-08T08:52:21.390 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00047824800000000002 2024-05-08T08:52:21.390 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.390 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.390 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:21.390 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.759140+0000", 2024-05-08T08:52:21.390 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0031629090000000002 2024-05-08T08:52:21.390 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.391 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.391 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:21.391 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.759982+0000", 2024-05-08T08:52:21.391 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00084187900000000002 2024-05-08T08:52:21.391 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.391 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.391 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:21.391 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.760037+0000", 2024-05-08T08:52:21.391 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5356000000000003e-05 2024-05-08T08:52:21.391 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.391 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.391 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:21.392 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.760082+0000", 2024-05-08T08:52:21.392 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5082e-05 2024-05-08T08:52:21.392 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:21.392 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:21.392 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:21.392 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.392 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.392 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4465.0:1323 3.17 3:e8515fe1:::rbd_object_map.1140cf79ec85:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-05-08T08:52:21.392 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:16.752964+0000", 2024-05-08T08:52:21.392 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 4.6216705319999996, 2024-05-08T08:52:21.392 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0072806950000000002, 2024-05-08T08:52:21.393 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:21.393 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:21.393 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:21.393 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:21.393 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4465", 2024-05-08T08:52:21.393 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/390275688", 2024-05-08T08:52:21.393 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1323 2024-05-08T08:52:21.393 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.393 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:21.393 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.393 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:21.393 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.752964+0000", 2024-05-08T08:52:21.393 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:21.393 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.394 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.394 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:21.394 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.752964+0000", 2024-05-08T08:52:21.394 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:21.394 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.394 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.394 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:21.394 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.752964+0000", 2024-05-08T08:52:21.394 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6000000000000004e-07 2024-05-08T08:52:21.394 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.394 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.394 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:21.394 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.752976+0000", 2024-05-08T08:52:21.395 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1357e-05 2024-05-08T08:52:21.395 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.395 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.395 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:21.395 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.752982+0000", 2024-05-08T08:52:21.395 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3400000000000003e-06 2024-05-08T08:52:21.395 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.395 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.395 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:21.395 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.752985+0000", 2024-05-08T08:52:21.395 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.399e-06 2024-05-08T08:52:21.395 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.395 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.395 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:21.395 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.756450+0000", 2024-05-08T08:52:21.396 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00346482 2024-05-08T08:52:21.396 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.396 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.396 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:21.396 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.756499+0000", 2024-05-08T08:52:21.396 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8674999999999997e-05 2024-05-08T08:52:21.396 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.396 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.396 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 5", 2024-05-08T08:52:21.396 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.756693+0000", 2024-05-08T08:52:21.396 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00019454499999999999 2024-05-08T08:52:21.396 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.396 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.396 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:21.396 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.759147+0000", 2024-05-08T08:52:21.397 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0024532040000000001 2024-05-08T08:52:21.397 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.397 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.397 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:21.397 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.760144+0000", 2024-05-08T08:52:21.397 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00099782899999999999 2024-05-08T08:52:21.397 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.397 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.397 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:21.397 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.760200+0000", 2024-05-08T08:52:21.397 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5928999999999999e-05 2024-05-08T08:52:21.397 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.397 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.397 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:21.397 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.760244+0000", 2024-05-08T08:52:21.398 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4036999999999999e-05 2024-05-08T08:52:21.398 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:21.398 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:21.398 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:21.398 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.398 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.398 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4465.0:1326 3.17 3:e8515fe1:::rbd_object_map.1140cf79ec85:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-05-08T08:52:21.398 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:16.754655+0000", 2024-05-08T08:52:21.398 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 4.6199795530000003, 2024-05-08T08:52:21.398 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0063295690000000002, 2024-05-08T08:52:21.398 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:21.399 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:21.399 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:21.399 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:21.399 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4465", 2024-05-08T08:52:21.399 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/390275688", 2024-05-08T08:52:21.399 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1326 2024-05-08T08:52:21.399 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.399 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:21.399 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.399 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:21.399 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.754655+0000", 2024-05-08T08:52:21.399 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:21.399 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.399 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.399 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:21.400 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.754655+0000", 2024-05-08T08:52:21.400 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:21.400 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.400 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.400 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:21.400 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.754655+0000", 2024-05-08T08:52:21.400 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1900000000000002e-07 2024-05-08T08:52:21.400 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.400 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.400 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:21.400 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.754663+0000", 2024-05-08T08:52:21.400 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1720000000000001e-06 2024-05-08T08:52:21.400 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.400 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.400 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:21.401 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.754672+0000", 2024-05-08T08:52:21.401 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.9180000000000006e-06 2024-05-08T08:52:21.401 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.401 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.401 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:21.401 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.754679+0000", 2024-05-08T08:52:21.401 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3330000000000002e-06 2024-05-08T08:52:21.401 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.401 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.401 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:21.401 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.758148+0000", 2024-05-08T08:52:21.402 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0034696789999999998 2024-05-08T08:52:21.402 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.402 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.402 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:21.402 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.758254+0000", 2024-05-08T08:52:21.402 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010602 2024-05-08T08:52:21.402 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.402 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.402 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 5", 2024-05-08T08:52:21.402 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.758714+0000", 2024-05-08T08:52:21.402 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00045989499999999999 2024-05-08T08:52:21.402 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.402 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.403 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:21.403 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.760462+0000", 2024-05-08T08:52:21.403 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001747136 2024-05-08T08:52:21.403 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.403 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.403 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:21.403 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.760875+0000", 2024-05-08T08:52:21.403 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00041329600000000001 2024-05-08T08:52:21.403 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.403 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.403 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:21.403 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.760932+0000", 2024-05-08T08:52:21.403 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7073000000000001e-05 2024-05-08T08:52:21.403 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.404 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.404 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:21.404 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.760984+0000", 2024-05-08T08:52:21.404 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2327999999999998e-05 2024-05-08T08:52:21.404 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:21.404 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:21.404 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:21.404 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.404 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.404 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4465.0:1552 3.17 3:e8515fe1:::rbd_object_map.1140cf79ec85:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-05-08T08:52:21.404 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:16.873814+0000", 2024-05-08T08:52:21.404 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 4.5008200289999998, 2024-05-08T08:52:21.404 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0067091709999999999, 2024-05-08T08:52:21.404 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:21.405 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:21.405 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:21.405 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:21.405 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4465", 2024-05-08T08:52:21.405 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/390275688", 2024-05-08T08:52:21.405 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1552 2024-05-08T08:52:21.405 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.405 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:21.405 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.405 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:21.405 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.873814+0000", 2024-05-08T08:52:21.405 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:21.405 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.405 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.406 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:21.406 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.873814+0000", 2024-05-08T08:52:21.406 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:21.406 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.406 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.406 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:21.406 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.873815+0000", 2024-05-08T08:52:21.406 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3900000000000005e-07 2024-05-08T08:52:21.406 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.406 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.406 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:21.406 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.873818+0000", 2024-05-08T08:52:21.406 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5360000000000001e-06 2024-05-08T08:52:21.406 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.407 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.407 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:21.407 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.873827+0000", 2024-05-08T08:52:21.407 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.8040000000000007e-06 2024-05-08T08:52:21.407 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.407 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.407 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:21.407 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.873833+0000", 2024-05-08T08:52:21.407 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9619999999999998e-06 2024-05-08T08:52:21.407 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.407 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.407 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:21.408 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.874439+0000", 2024-05-08T08:52:21.408 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00060590000000000004 2024-05-08T08:52:21.408 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.408 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.408 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:21.408 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.874499+0000", 2024-05-08T08:52:21.408 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0275999999999997e-05 2024-05-08T08:52:21.408 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.408 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.408 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 5", 2024-05-08T08:52:21.408 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.874738+0000", 2024-05-08T08:52:21.408 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000238688 2024-05-08T08:52:21.408 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.408 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.408 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:21.409 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.877423+0000", 2024-05-08T08:52:21.409 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0026849320000000001 2024-05-08T08:52:21.409 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.409 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.409 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:21.409 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.880418+0000", 2024-05-08T08:52:21.409 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0029950910000000001 2024-05-08T08:52:21.409 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.409 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.409 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:21.409 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.880476+0000", 2024-05-08T08:52:21.409 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7599999999999997e-05 2024-05-08T08:52:21.409 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.410 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.410 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:21.410 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.880523+0000", 2024-05-08T08:52:21.410 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7843000000000003e-05 2024-05-08T08:52:21.410 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:21.410 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:21.410 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:21.410 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.410 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.410 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4465.0:1553 3.17 3:e8515fe1:::rbd_object_map.1140cf79ec85:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-05-08T08:52:21.410 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:16.873846+0000", 2024-05-08T08:52:21.410 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 4.5007886040000002, 2024-05-08T08:52:21.411 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0067676710000000003, 2024-05-08T08:52:21.411 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:21.411 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:21.411 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:21.411 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:21.411 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4465", 2024-05-08T08:52:21.411 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/390275688", 2024-05-08T08:52:21.411 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1553 2024-05-08T08:52:21.411 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.411 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:21.411 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.411 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:21.411 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.873846+0000", 2024-05-08T08:52:21.411 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:21.411 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.412 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.412 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:21.412 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.873846+0000", 2024-05-08T08:52:21.412 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:21.412 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.412 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.412 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:21.412 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.873846+0000", 2024-05-08T08:52:21.412 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.13e-07 2024-05-08T08:52:21.412 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.412 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.412 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:21.412 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.873850+0000", 2024-05-08T08:52:21.412 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7400000000000002e-06 2024-05-08T08:52:21.413 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.413 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.413 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:21.413 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.873872+0000", 2024-05-08T08:52:21.413 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2147e-05 2024-05-08T08:52:21.413 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.413 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.413 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:21.413 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.873878+0000", 2024-05-08T08:52:21.413 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3029999999999996e-06 2024-05-08T08:52:21.413 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.413 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.413 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:21.413 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.875034+0000", 2024-05-08T08:52:21.414 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001156058 2024-05-08T08:52:21.414 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.414 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.414 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:21.414 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.875083+0000", 2024-05-08T08:52:21.414 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8143999999999998e-05 2024-05-08T08:52:21.414 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.414 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.414 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 5", 2024-05-08T08:52:21.414 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.875413+0000", 2024-05-08T08:52:21.414 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000330396 2024-05-08T08:52:21.414 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.414 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.414 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:21.415 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.877436+0000", 2024-05-08T08:52:21.415 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0020231099999999998 2024-05-08T08:52:21.415 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.415 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.415 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:21.415 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.880558+0000", 2024-05-08T08:52:21.415 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.003122077 2024-05-08T08:52:21.415 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.415 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.415 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:21.415 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.880589+0000", 2024-05-08T08:52:21.415 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1075000000000003e-05 2024-05-08T08:52:21.415 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.416 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.416 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:21.416 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.880613+0000", 2024-05-08T08:52:21.416 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4008000000000001e-05 2024-05-08T08:52:21.416 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:21.416 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:21.416 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:21.416 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.416 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.416 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4465.0:1554 3.17 3:e8515fe1:::rbd_object_map.1140cf79ec85:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-05-08T08:52:21.416 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:16.873892+0000", 2024-05-08T08:52:21.416 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 4.5007424299999998, 2024-05-08T08:52:21.416 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0068799799999999999, 2024-05-08T08:52:21.416 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:21.416 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:21.417 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:21.417 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:21.417 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4465", 2024-05-08T08:52:21.417 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/390275688", 2024-05-08T08:52:21.417 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1554 2024-05-08T08:52:21.417 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.417 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:21.417 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.417 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:21.417 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.873892+0000", 2024-05-08T08:52:21.417 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:21.417 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.417 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.417 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:21.418 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.873892+0000", 2024-05-08T08:52:21.418 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:21.418 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.418 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.418 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:21.418 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.873892+0000", 2024-05-08T08:52:21.418 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9399999999999995e-07 2024-05-08T08:52:21.418 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.418 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.418 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:21.418 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.873897+0000", 2024-05-08T08:52:21.418 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3649999999999997e-06 2024-05-08T08:52:21.418 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.419 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.419 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:21.419 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.873904+0000", 2024-05-08T08:52:21.419 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.148e-06 2024-05-08T08:52:21.419 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.419 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.419 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:21.419 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.873909+0000", 2024-05-08T08:52:21.419 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0490000000000003e-06 2024-05-08T08:52:21.419 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.419 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.419 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:21.419 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.875746+0000", 2024-05-08T08:52:21.419 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001836978 2024-05-08T08:52:21.420 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.420 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.420 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:21.420 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.875846+0000", 2024-05-08T08:52:21.420 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000100104 2024-05-08T08:52:21.420 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.420 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.420 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 5", 2024-05-08T08:52:21.420 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.876251+0000", 2024-05-08T08:52:21.420 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00040462199999999999 2024-05-08T08:52:21.420 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.420 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.420 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:21.420 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.877444+0000", 2024-05-08T08:52:21.421 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001193394 2024-05-08T08:52:21.421 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.421 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.421 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:21.421 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.880671+0000", 2024-05-08T08:52:21.421 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.003226989 2024-05-08T08:52:21.421 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.421 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.421 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:21.421 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.880728+0000", 2024-05-08T08:52:21.421 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6994999999999999e-05 2024-05-08T08:52:21.421 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.421 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.421 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:21.422 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.880772+0000", 2024-05-08T08:52:21.422 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3841999999999998e-05 2024-05-08T08:52:21.422 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:21.422 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:21.422 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:21.422 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.422 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.422 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4465.0:1559 3.17 3:e8515fe1:::rbd_object_map.1140cf79ec85:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-05-08T08:52:21.422 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:16.875204+0000", 2024-05-08T08:52:21.422 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 4.4994302309999998, 2024-05-08T08:52:21.422 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0081227499999999998, 2024-05-08T08:52:21.422 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:21.422 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:21.422 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:21.423 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:21.423 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4465", 2024-05-08T08:52:21.423 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/390275688", 2024-05-08T08:52:21.423 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1559 2024-05-08T08:52:21.423 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.423 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:21.423 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.423 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:21.423 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.875204+0000", 2024-05-08T08:52:21.423 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:21.423 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.424 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.424 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:21.424 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.875204+0000", 2024-05-08T08:52:21.424 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:21.424 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.424 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.424 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:21.424 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.875205+0000", 2024-05-08T08:52:21.424 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4200000000000004e-07 2024-05-08T08:52:21.424 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.424 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.424 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:21.424 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.875214+0000", 2024-05-08T08:52:21.424 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.6300000000000004e-06 2024-05-08T08:52:21.424 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.425 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.425 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:21.425 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.875226+0000", 2024-05-08T08:52:21.425 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2072999999999999e-05 2024-05-08T08:52:21.425 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.425 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.425 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:21.425 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.875234+0000", 2024-05-08T08:52:21.425 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.5250000000000005e-06 2024-05-08T08:52:21.425 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.425 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.425 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:21.425 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.877473+0000", 2024-05-08T08:52:21.425 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0022386910000000001 2024-05-08T08:52:21.425 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.426 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.426 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:21.426 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.877557+0000", 2024-05-08T08:52:21.426 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4305e-05 2024-05-08T08:52:21.426 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.426 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.426 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 5", 2024-05-08T08:52:21.426 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.877913+0000", 2024-05-08T08:52:21.426 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00035600900000000001 2024-05-08T08:52:21.426 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.426 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.426 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:21.426 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.880094+0000", 2024-05-08T08:52:21.426 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0021808029999999998 2024-05-08T08:52:21.427 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.427 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.427 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:21.427 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.883222+0000", 2024-05-08T08:52:21.427 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0031282630000000001 2024-05-08T08:52:21.427 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.427 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.427 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:21.427 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.883279+0000", 2024-05-08T08:52:21.427 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6369999999999997e-05 2024-05-08T08:52:21.427 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.427 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.427 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:21.427 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.883327+0000", 2024-05-08T08:52:21.427 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8139000000000001e-05 2024-05-08T08:52:21.428 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:21.428 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:21.428 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:21.428 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.428 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.428 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4465.0:1904 3.17 3:e8515fe1:::rbd_object_map.1140cf79ec85:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-05-08T08:52:21.428 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:17.029330+0000", 2024-05-08T08:52:21.428 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 4.3453046779999998, 2024-05-08T08:52:21.428 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0063870239999999998, 2024-05-08T08:52:21.428 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:21.428 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:21.428 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:21.428 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:21.429 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4465", 2024-05-08T08:52:21.429 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/390275688", 2024-05-08T08:52:21.429 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1904 2024-05-08T08:52:21.429 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.429 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:21.429 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.429 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:21.429 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.029330+0000", 2024-05-08T08:52:21.429 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:21.429 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.429 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.430 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:21.430 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.029330+0000", 2024-05-08T08:52:21.430 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:21.430 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.430 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.430 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:21.430 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.029330+0000", 2024-05-08T08:52:21.430 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4700000000000002e-07 2024-05-08T08:52:21.430 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.430 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.430 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:21.430 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.029332+0000", 2024-05-08T08:52:21.430 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9750000000000001e-06 2024-05-08T08:52:21.430 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.431 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.431 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:21.431 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.029336+0000", 2024-05-08T08:52:21.431 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8789999999999996e-06 2024-05-08T08:52:21.431 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.431 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.431 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:21.431 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.029339+0000", 2024-05-08T08:52:21.431 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9610000000000001e-06 2024-05-08T08:52:21.431 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.431 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.431 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:21.431 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.029918+0000", 2024-05-08T08:52:21.431 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00057881800000000004 2024-05-08T08:52:21.431 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.432 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.432 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:21.432 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.029980+0000", 2024-05-08T08:52:21.432 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2744999999999996e-05 2024-05-08T08:52:21.432 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.432 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.432 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 5", 2024-05-08T08:52:21.432 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.030189+0000", 2024-05-08T08:52:21.432 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000208755 2024-05-08T08:52:21.432 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.432 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.432 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:21.432 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.031943+0000", 2024-05-08T08:52:21.432 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001753597 2024-05-08T08:52:21.432 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.433 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.433 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:21.433 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.035610+0000", 2024-05-08T08:52:21.433 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0036676220000000002 2024-05-08T08:52:21.433 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.433 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.433 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:21.433 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.035661+0000", 2024-05-08T08:52:21.433 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0939999999999997e-05 2024-05-08T08:52:21.433 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.433 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.433 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:21.433 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.035717+0000", 2024-05-08T08:52:21.433 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5384999999999999e-05 2024-05-08T08:52:21.433 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:21.434 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:21.434 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:21.434 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.434 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.434 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4465.0:2329 3.17 3:e8515fe1:::rbd_object_map.1140cf79ec85:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-05-08T08:52:21.434 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:17.295805+0000", 2024-05-08T08:52:21.434 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 4.07882956, 2024-05-08T08:52:21.434 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0069030209999999996, 2024-05-08T08:52:21.434 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:21.434 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:21.434 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:21.434 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:21.434 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4465", 2024-05-08T08:52:21.435 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/390275688", 2024-05-08T08:52:21.435 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2329 2024-05-08T08:52:21.435 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.435 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:21.435 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.435 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:21.435 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.295805+0000", 2024-05-08T08:52:21.435 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:21.436 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.436 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.436 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:21.436 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.295805+0000", 2024-05-08T08:52:21.436 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:21.436 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.436 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.436 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:21.436 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.295805+0000", 2024-05-08T08:52:21.436 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3900000000000001e-07 2024-05-08T08:52:21.436 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.436 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.437 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:21.437 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.295807+0000", 2024-05-08T08:52:21.437 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.759e-06 2024-05-08T08:52:21.437 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.437 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.437 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:21.437 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.295812+0000", 2024-05-08T08:52:21.437 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2410000000000003e-06 2024-05-08T08:52:21.437 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.437 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.437 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:21.437 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.295815+0000", 2024-05-08T08:52:21.437 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7300000000000001e-06 2024-05-08T08:52:21.437 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.437 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.438 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:21.438 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.298606+0000", 2024-05-08T08:52:21.438 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0027917570000000002 2024-05-08T08:52:21.438 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.438 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.438 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:21.438 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.298654+0000", 2024-05-08T08:52:21.438 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7682999999999997e-05 2024-05-08T08:52:21.439 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.439 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.439 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 5", 2024-05-08T08:52:21.439 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.298868+0000", 2024-05-08T08:52:21.439 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000213854 2024-05-08T08:52:21.439 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.439 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.439 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:21.439 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.300086+0000", 2024-05-08T08:52:21.439 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0012178250000000001 2024-05-08T08:52:21.439 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.439 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.439 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:21.439 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.302591+0000", 2024-05-08T08:52:21.440 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0025054650000000001 2024-05-08T08:52:21.440 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.440 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.440 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:21.440 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.302654+0000", 2024-05-08T08:52:21.440 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3260000000000001e-05 2024-05-08T08:52:21.440 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.440 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.440 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:21.440 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.302708+0000", 2024-05-08T08:52:21.440 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3208000000000003e-05 2024-05-08T08:52:21.440 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:21.440 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:21.440 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:21.441 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.441 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.441 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4465.0:2331 3.17 3:e8515fe1:::rbd_object_map.1140cf79ec85:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-05-08T08:52:21.441 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:17.295871+0000", 2024-05-08T08:52:21.441 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 4.0787627789999998, 2024-05-08T08:52:21.441 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.010985981000000001, 2024-05-08T08:52:21.441 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:21.441 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:21.441 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:21.441 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:21.441 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4465", 2024-05-08T08:52:21.441 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/390275688", 2024-05-08T08:52:21.441 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2331 2024-05-08T08:52:21.441 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.441 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:21.442 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.442 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:21.442 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.295871+0000", 2024-05-08T08:52:21.442 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:21.442 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.442 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.442 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:21.442 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.295871+0000", 2024-05-08T08:52:21.442 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:21.442 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.442 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.442 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:21.442 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.295872+0000", 2024-05-08T08:52:21.442 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8799999999999998e-07 2024-05-08T08:52:21.443 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.443 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.443 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:21.443 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.295874+0000", 2024-05-08T08:52:21.443 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2239999999999998e-06 2024-05-08T08:52:21.443 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.443 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.443 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:21.443 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.295880+0000", 2024-05-08T08:52:21.443 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6130000000000003e-06 2024-05-08T08:52:21.443 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.443 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.443 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:21.443 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.295884+0000", 2024-05-08T08:52:21.444 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1829999999999996e-06 2024-05-08T08:52:21.444 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.444 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.444 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:21.444 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.299175+0000", 2024-05-08T08:52:21.444 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.003291165 2024-05-08T08:52:21.444 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.444 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.444 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:21.444 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.299228+0000", 2024-05-08T08:52:21.444 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2830000000000001e-05 2024-05-08T08:52:21.444 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.445 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.445 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 5", 2024-05-08T08:52:21.445 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.299482+0000", 2024-05-08T08:52:21.445 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00025414300000000002 2024-05-08T08:52:21.445 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.445 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.445 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:21.445 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.301380+0000", 2024-05-08T08:52:21.445 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001898046 2024-05-08T08:52:21.445 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.446 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.446 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:21.446 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.306802+0000", 2024-05-08T08:52:21.446 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0054220730000000003 2024-05-08T08:52:21.446 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.446 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.446 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:21.446 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.306833+0000", 2024-05-08T08:52:21.446 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0491e-05 2024-05-08T08:52:21.446 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.446 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.446 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:21.446 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.306857+0000", 2024-05-08T08:52:21.446 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4825e-05 2024-05-08T08:52:21.447 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:21.447 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:21.447 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:21.447 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.447 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.447 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4465.0:2332 3.17 3:e8515fe1:::rbd_object_map.1140cf79ec85:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-05-08T08:52:21.447 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:17.295889+0000", 2024-05-08T08:52:21.447 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 4.0787450029999999, 2024-05-08T08:52:21.447 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.011142335999999999, 2024-05-08T08:52:21.447 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:21.447 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:21.447 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:21.447 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:21.447 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4465", 2024-05-08T08:52:21.448 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/390275688", 2024-05-08T08:52:21.448 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2332 2024-05-08T08:52:21.448 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.448 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:21.448 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.448 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:21.448 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.295889+0000", 2024-05-08T08:52:21.448 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:21.448 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.448 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.448 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:21.448 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.295889+0000", 2024-05-08T08:52:21.448 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:21.449 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.449 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.449 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:21.449 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.295889+0000", 2024-05-08T08:52:21.449 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8099999999999999e-07 2024-05-08T08:52:21.449 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.449 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.449 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:21.449 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.295891+0000", 2024-05-08T08:52:21.449 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.511e-06 2024-05-08T08:52:21.449 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.449 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.450 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:21.450 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.295894+0000", 2024-05-08T08:52:21.450 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.247e-06 2024-05-08T08:52:21.450 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.450 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.450 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:21.450 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.295897+0000", 2024-05-08T08:52:21.450 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4609999999999998e-06 2024-05-08T08:52:21.450 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.451 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.451 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:21.451 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.300103+0000", 2024-05-08T08:52:21.451 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0042062569999999997 2024-05-08T08:52:21.451 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.451 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.451 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:21.451 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.300154+0000", 2024-05-08T08:52:21.451 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0788999999999998e-05 2024-05-08T08:52:21.452 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.452 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.452 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 5", 2024-05-08T08:52:21.452 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.300402+0000", 2024-05-08T08:52:21.452 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00024846299999999998 2024-05-08T08:52:21.452 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.452 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.452 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:21.452 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.301383+0000", 2024-05-08T08:52:21.452 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00098072799999999994 2024-05-08T08:52:21.452 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.452 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.452 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:21.452 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.306919+0000", 2024-05-08T08:52:21.453 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0055362520000000002 2024-05-08T08:52:21.453 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.453 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.453 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:21.453 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.306986+0000", 2024-05-08T08:52:21.453 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7292999999999994e-05 2024-05-08T08:52:21.453 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.453 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.453 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:21.453 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.307032+0000", 2024-05-08T08:52:21.453 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5053999999999999e-05 2024-05-08T08:52:21.453 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:21.453 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:21.453 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:21.454 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.454 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.454 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4465.0:2333 3.17 3:e8515fe1:::rbd_object_map.1140cf79ec85:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-05-08T08:52:21.454 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:17.295975+0000", 2024-05-08T08:52:21.454 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 4.0786591999999997, 2024-05-08T08:52:21.454 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.01114335, 2024-05-08T08:52:21.454 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:21.454 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:21.454 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:21.454 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:21.454 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4465", 2024-05-08T08:52:21.454 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/390275688", 2024-05-08T08:52:21.454 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2333 2024-05-08T08:52:21.455 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.455 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:21.455 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.455 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:21.455 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.295975+0000", 2024-05-08T08:52:21.455 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:21.455 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.455 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.455 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:21.455 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.295975+0000", 2024-05-08T08:52:21.455 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:21.455 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.456 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.456 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:21.456 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.295975+0000", 2024-05-08T08:52:21.456 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1199999999999999e-07 2024-05-08T08:52:21.456 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.456 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.456 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:21.456 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.295977+0000", 2024-05-08T08:52:21.456 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.103e-06 2024-05-08T08:52:21.456 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.456 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.456 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:21.456 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.295982+0000", 2024-05-08T08:52:21.456 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1309999999999999e-06 2024-05-08T08:52:21.456 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.457 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.457 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:21.457 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.295984+0000", 2024-05-08T08:52:21.457 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8909999999999999e-06 2024-05-08T08:52:21.457 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.457 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.457 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:21.457 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.300741+0000", 2024-05-08T08:52:21.457 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0047561069999999999 2024-05-08T08:52:21.457 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.457 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.457 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:21.457 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.300804+0000", 2024-05-08T08:52:21.457 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2976000000000001e-05 2024-05-08T08:52:21.457 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.457 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.458 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 5", 2024-05-08T08:52:21.458 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.301079+0000", 2024-05-08T08:52:21.458 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00027533000000000001 2024-05-08T08:52:21.458 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.458 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.458 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:21.458 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.301647+0000", 2024-05-08T08:52:21.458 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00056773499999999998 2024-05-08T08:52:21.458 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.458 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.458 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:21.458 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.307067+0000", 2024-05-08T08:52:21.458 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0054205089999999996 2024-05-08T08:52:21.459 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.459 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.459 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:21.459 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.307096+0000", 2024-05-08T08:52:21.459 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9037e-05 2024-05-08T08:52:21.459 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.459 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.459 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:21.459 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.307118+0000", 2024-05-08T08:52:21.459 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2218999999999999e-05 2024-05-08T08:52:21.459 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:21.459 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:21.459 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:21.459 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.459 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.460 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4465.0:2350 3.17 3:e8515fe1:::rbd_object_map.1140cf79ec85:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-05-08T08:52:21.460 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:17.309624+0000", 2024-05-08T08:52:21.460 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 4.0650104919999999, 2024-05-08T08:52:21.460 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0067184030000000004, 2024-05-08T08:52:21.460 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:21.460 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:21.460 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:21.460 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:21.460 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4465", 2024-05-08T08:52:21.460 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/390275688", 2024-05-08T08:52:21.460 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2350 2024-05-08T08:52:21.460 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.460 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:21.460 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.461 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:21.461 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.309624+0000", 2024-05-08T08:52:21.461 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:21.461 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.461 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.461 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:21.461 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.309624+0000", 2024-05-08T08:52:21.461 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:21.461 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.461 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.461 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:21.461 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.309624+0000", 2024-05-08T08:52:21.461 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9400000000000001e-07 2024-05-08T08:52:21.461 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.461 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.462 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:21.462 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.309627+0000", 2024-05-08T08:52:21.462 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9799999999999998e-06 2024-05-08T08:52:21.462 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.462 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.462 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:21.462 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.309636+0000", 2024-05-08T08:52:21.462 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.9260000000000006e-06 2024-05-08T08:52:21.462 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.462 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.462 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:21.463 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.309639+0000", 2024-05-08T08:52:21.463 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4400000000000001e-06 2024-05-08T08:52:21.463 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.463 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.463 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:21.463 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.310232+0000", 2024-05-08T08:52:21.463 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00059308400000000002 2024-05-08T08:52:21.463 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.463 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.463 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:21.463 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.310288+0000", 2024-05-08T08:52:21.464 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5962999999999999e-05 2024-05-08T08:52:21.464 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.464 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.464 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 5", 2024-05-08T08:52:21.464 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.310501+0000", 2024-05-08T08:52:21.464 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000212754 2024-05-08T08:52:21.464 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.464 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.464 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:21.464 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.311893+0000", 2024-05-08T08:52:21.464 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001391588 2024-05-08T08:52:21.465 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.465 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.465 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:21.465 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.316217+0000", 2024-05-08T08:52:21.465 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0043244219999999996 2024-05-08T08:52:21.465 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.465 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.465 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:21.465 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.316319+0000", 2024-05-08T08:52:21.465 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000101606 2024-05-08T08:52:21.465 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.465 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.465 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:21.466 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.316342+0000", 2024-05-08T08:52:21.466 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3346e-05 2024-05-08T08:52:21.466 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:21.466 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:21.466 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:21.466 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.466 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.466 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4465.0:2873 3.17 3:e8515fe1:::rbd_object_map.1140cf79ec85:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-05-08T08:52:21.466 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:17.549314+0000", 2024-05-08T08:52:21.466 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 3.82532009, 2024-05-08T08:52:21.466 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0062881350000000003, 2024-05-08T08:52:21.466 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:21.466 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:21.467 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:21.467 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:21.467 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4465", 2024-05-08T08:52:21.467 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/390275688", 2024-05-08T08:52:21.467 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2873 2024-05-08T08:52:21.467 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.467 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:21.467 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.467 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:21.467 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.549314+0000", 2024-05-08T08:52:21.467 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:21.467 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.468 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.468 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:21.468 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.549314+0000", 2024-05-08T08:52:21.468 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:21.468 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.468 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.468 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:21.468 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.549314+0000", 2024-05-08T08:52:21.468 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7599999999999998e-07 2024-05-08T08:52:21.468 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.468 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.468 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:21.468 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.549316+0000", 2024-05-08T08:52:21.468 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9589999999999998e-06 2024-05-08T08:52:21.469 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.469 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.469 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:21.469 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.549320+0000", 2024-05-08T08:52:21.469 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9829999999999998e-06 2024-05-08T08:52:21.469 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.469 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.469 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:21.469 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.549323+0000", 2024-05-08T08:52:21.469 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.853e-06 2024-05-08T08:52:21.469 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.470 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.470 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:21.470 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.549967+0000", 2024-05-08T08:52:21.470 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00064347700000000005 2024-05-08T08:52:21.470 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.470 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.470 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:21.470 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.550069+0000", 2024-05-08T08:52:21.470 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000101869 2024-05-08T08:52:21.470 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.470 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.470 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 5", 2024-05-08T08:52:21.471 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.550413+0000", 2024-05-08T08:52:21.471 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000344006 2024-05-08T08:52:21.471 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.471 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.471 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:21.471 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.551387+0000", 2024-05-08T08:52:21.471 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00097466199999999995 2024-05-08T08:52:21.471 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.471 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.471 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:21.471 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.555543+0000", 2024-05-08T08:52:21.471 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0041560980000000004 2024-05-08T08:52:21.471 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.472 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.472 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:21.472 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.555579+0000", 2024-05-08T08:52:21.472 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5343999999999999e-05 2024-05-08T08:52:21.472 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.472 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.472 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:21.472 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.555602+0000", 2024-05-08T08:52:21.472 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3507999999999999e-05 2024-05-08T08:52:21.472 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:21.472 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:21.472 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:21.472 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.473 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.473 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4465.0:2874 3.17 3:e8515fe1:::rbd_object_map.1140cf79ec85:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-05-08T08:52:21.473 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:17.549328+0000", 2024-05-08T08:52:21.473 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 3.8253064220000002, 2024-05-08T08:52:21.473 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0066336010000000003, 2024-05-08T08:52:21.473 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:21.473 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:21.473 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:21.473 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:21.473 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4465", 2024-05-08T08:52:21.473 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/390275688", 2024-05-08T08:52:21.473 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2874 2024-05-08T08:52:21.473 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.474 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:21.474 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.474 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:21.474 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.549328+0000", 2024-05-08T08:52:21.474 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:21.474 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.474 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.474 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:21.474 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.549328+0000", 2024-05-08T08:52:21.474 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:21.474 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.474 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.474 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:21.474 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.549328+0000", 2024-05-08T08:52:21.475 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.28e-07 2024-05-08T08:52:21.475 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.475 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.475 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:21.475 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.549329+0000", 2024-05-08T08:52:21.475 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3790000000000001e-06 2024-05-08T08:52:21.475 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.475 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.475 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:21.475 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.549333+0000", 2024-05-08T08:52:21.475 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.2650000000000001e-06 2024-05-08T08:52:21.475 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.475 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.475 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:21.475 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.549335+0000", 2024-05-08T08:52:21.476 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6539999999999999e-06 2024-05-08T08:52:21.476 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.476 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.476 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:21.476 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.550726+0000", 2024-05-08T08:52:21.476 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001390898 2024-05-08T08:52:21.476 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.476 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.476 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:21.476 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.550780+0000", 2024-05-08T08:52:21.476 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3418000000000003e-05 2024-05-08T08:52:21.476 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.476 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.476 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 5", 2024-05-08T08:52:21.477 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.551004+0000", 2024-05-08T08:52:21.477 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00022409599999999999 2024-05-08T08:52:21.477 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.477 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.477 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:21.477 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.552935+0000", 2024-05-08T08:52:21.477 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0019310779999999999 2024-05-08T08:52:21.477 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.477 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.477 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:21.477 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.555898+0000", 2024-05-08T08:52:21.477 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0029634380000000001 2024-05-08T08:52:21.477 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.477 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.477 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:21.478 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.555930+0000", 2024-05-08T08:52:21.478 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1647999999999999e-05 2024-05-08T08:52:21.478 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.478 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.478 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:21.478 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.555961+0000", 2024-05-08T08:52:21.478 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1498999999999998e-05 2024-05-08T08:52:21.478 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:21.478 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:21.478 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:21.478 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.478 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.478 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4465.0:2875 3.17 3:e8515fe1:::rbd_object_map.1140cf79ec85:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-05-08T08:52:21.479 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:17.549361+0000", 2024-05-08T08:52:21.479 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 3.825273181, 2024-05-08T08:52:21.479 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0066527289999999996, 2024-05-08T08:52:21.479 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:21.479 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:21.479 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:21.479 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:21.479 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4465", 2024-05-08T08:52:21.479 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/390275688", 2024-05-08T08:52:21.479 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2875 2024-05-08T08:52:21.479 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.479 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:21.479 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.479 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:21.480 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.549361+0000", 2024-05-08T08:52:21.480 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:21.480 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.480 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.480 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:21.480 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.549361+0000", 2024-05-08T08:52:21.480 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:21.480 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.480 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.480 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:21.480 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.549361+0000", 2024-05-08T08:52:21.480 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9999999999999999e-07 2024-05-08T08:52:21.481 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.481 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.481 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:21.481 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.549364+0000", 2024-05-08T08:52:21.481 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8059999999999999e-06 2024-05-08T08:52:21.481 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.481 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.481 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:21.481 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.549368+0000", 2024-05-08T08:52:21.481 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3540000000000002e-06 2024-05-08T08:52:21.481 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.481 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.481 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:21.481 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.549371+0000", 2024-05-08T08:52:21.481 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.858e-06 2024-05-08T08:52:21.482 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.482 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.482 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:21.482 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.551480+0000", 2024-05-08T08:52:21.482 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.002108646 2024-05-08T08:52:21.482 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.482 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.482 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:21.482 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.551534+0000", 2024-05-08T08:52:21.482 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4202e-05 2024-05-08T08:52:21.483 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.483 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.483 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 5", 2024-05-08T08:52:21.483 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.551738+0000", 2024-05-08T08:52:21.483 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000203928 2024-05-08T08:52:21.483 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.483 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.483 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:21.483 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.552970+0000", 2024-05-08T08:52:21.483 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0012320829999999999 2024-05-08T08:52:21.483 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.483 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.484 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:21.484 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.555968+0000", 2024-05-08T08:52:21.484 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0029977839999999999 2024-05-08T08:52:21.484 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.484 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.484 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:21.484 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.555993+0000", 2024-05-08T08:52:21.484 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.5304e-05 2024-05-08T08:52:21.484 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.484 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.484 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:21.484 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.556014+0000", 2024-05-08T08:52:21.484 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0463999999999999e-05 2024-05-08T08:52:21.484 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:21.484 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:21.485 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:21.485 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.485 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.485 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4465.0:2876 3.17 3:e8515fe1:::rbd_object_map.1140cf79ec85:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-05-08T08:52:21.485 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:17.549379+0000", 2024-05-08T08:52:21.485 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 3.8252555199999998, 2024-05-08T08:52:21.485 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0066828520000000004, 2024-05-08T08:52:21.485 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:21.485 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:21.485 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:21.485 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:21.485 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4465", 2024-05-08T08:52:21.485 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/390275688", 2024-05-08T08:52:21.485 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2876 2024-05-08T08:52:21.485 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.486 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:21.486 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.486 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:21.486 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.549379+0000", 2024-05-08T08:52:21.486 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:21.486 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.486 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.486 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:21.486 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.549379+0000", 2024-05-08T08:52:21.486 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:21.486 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.486 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.486 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:21.487 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.549379+0000", 2024-05-08T08:52:21.487 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.2899999999999999e-07 2024-05-08T08:52:21.487 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.487 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.487 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:21.487 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.549381+0000", 2024-05-08T08:52:21.487 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5939999999999999e-06 2024-05-08T08:52:21.487 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.487 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.487 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:21.487 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.549388+0000", 2024-05-08T08:52:21.487 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3119999999999996e-06 2024-05-08T08:52:21.487 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.487 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.487 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:21.488 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.549391+0000", 2024-05-08T08:52:21.488 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4939999999999999e-06 2024-05-08T08:52:21.488 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.488 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.488 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:21.488 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.552078+0000", 2024-05-08T08:52:21.488 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0026861340000000002 2024-05-08T08:52:21.488 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.488 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.488 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:21.488 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.552149+0000", 2024-05-08T08:52:21.488 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1465999999999996e-05 2024-05-08T08:52:21.488 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.488 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.489 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 5", 2024-05-08T08:52:21.489 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.552491+0000", 2024-05-08T08:52:21.489 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00034228299999999998 2024-05-08T08:52:21.489 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.489 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.489 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:21.489 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.553418+0000", 2024-05-08T08:52:21.489 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00092652900000000005 2024-05-08T08:52:21.489 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.489 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.489 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:21.489 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.556020+0000", 2024-05-08T08:52:21.489 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0026016910000000002 2024-05-08T08:52:21.489 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.489 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.490 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:21.490 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.556043+0000", 2024-05-08T08:52:21.490 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3156999999999999e-05 2024-05-08T08:52:21.490 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.490 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.490 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:21.490 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.556062+0000", 2024-05-08T08:52:21.490 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.8862999999999999e-05 2024-05-08T08:52:21.490 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:21.490 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:21.490 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:21.490 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.490 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.491 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4465.0:3098 3.1f 3:f90107d2:::rbd_data.1140cf79ec85.000000000000001c:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-05-08T08:52:21.491 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:17.675370+0000", 2024-05-08T08:52:21.491 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 3.6992639920000001, 2024-05-08T08:52:21.491 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0063966439999999999, 2024-05-08T08:52:21.491 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:21.491 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:21.491 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:21.491 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:21.491 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4465", 2024-05-08T08:52:21.491 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/390275688", 2024-05-08T08:52:21.491 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 3098 2024-05-08T08:52:21.491 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.492 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:21.492 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.492 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:21.492 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.675370+0000", 2024-05-08T08:52:21.492 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:21.492 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.492 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.492 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:21.492 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.675370+0000", 2024-05-08T08:52:21.492 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:21.492 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.492 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.492 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:21.492 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.675371+0000", 2024-05-08T08:52:21.492 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7e-07 2024-05-08T08:52:21.492 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.493 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.493 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:21.493 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.675373+0000", 2024-05-08T08:52:21.493 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7300000000000001e-06 2024-05-08T08:52:21.493 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.493 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.493 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:21.493 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.675380+0000", 2024-05-08T08:52:21.493 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0430000000000002e-06 2024-05-08T08:52:21.493 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.493 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.493 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:21.494 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.675387+0000", 2024-05-08T08:52:21.494 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4910000000000002e-06 2024-05-08T08:52:21.494 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.494 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.494 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:21.494 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.675432+0000", 2024-05-08T08:52:21.494 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5192999999999999e-05 2024-05-08T08:52:21.494 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.494 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.494 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:21.494 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.675939+0000", 2024-05-08T08:52:21.494 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00050675000000000004 2024-05-08T08:52:21.494 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.495 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.495 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 3", 2024-05-08T08:52:21.495 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.676046+0000", 2024-05-08T08:52:21.495 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010745499999999999 2024-05-08T08:52:21.495 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.495 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.495 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:21.495 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.677718+0000", 2024-05-08T08:52:21.495 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001672116 2024-05-08T08:52:21.495 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.495 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.495 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:21.495 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.681693+0000", 2024-05-08T08:52:21.495 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0039740599999999997 2024-05-08T08:52:21.496 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.496 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.496 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:21.496 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.681724+0000", 2024-05-08T08:52:21.496 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1816999999999997e-05 2024-05-08T08:52:21.496 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.496 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.496 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:21.496 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.681767+0000", 2024-05-08T08:52:21.496 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2518999999999997e-05 2024-05-08T08:52:21.496 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:21.496 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:21.497 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:21.497 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:21.497 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:21.497 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:52:21.498 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:21.501+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:21.500 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:52:21.500 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:52:21.500 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:52:21.501 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:52:21.501 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:52:21.502 INFO:teuthology.orchestra.run.smithi196.stdout:799 punch from 0x6e9b400 to 0x6ea1200, (0x5e00 bytes) 2024-05-08T08:52:21.502 INFO:teuthology.orchestra.run.smithi196.stdout:802 read 0x29b6800 thru 0x29c4db8 (0xe5b9 bytes) 2024-05-08T08:52:21.502 INFO:teuthology.orchestra.run.smithi196.stdout:803 read 0x30cc400 thru 0x30d653c (0xa13d bytes) 2024-05-08T08:52:21.502 INFO:teuthology.orchestra.run.smithi196.stdout:804 write 0x7bb1400 thru 0x7bbd028 (0xbc29 bytes) 2024-05-08T08:52:21.502 INFO:teuthology.orchestra.run.smithi196.stdout:805 write 0x13b8e00 thru 0x13bc6a0 (0x38a1 bytes) 2024-05-08T08:52:21.502 INFO:teuthology.orchestra.run.smithi196.stdout:806 read 0x842d800 thru 0x842fa94 (0x2295 bytes) 2024-05-08T08:52:21.502 INFO:teuthology.orchestra.run.smithi196.stdout:807 read 0xac8cc00 thru 0xac90f97 (0x4398 bytes) 2024-05-08T08:52:21.502 INFO:teuthology.orchestra.run.smithi196.stdout:812 write 0xcf62200 thru 0xcf67903 (0x5704 bytes) 2024-05-08T08:52:21.502 INFO:teuthology.orchestra.run.smithi196.stdout:813 punch from 0x6b87c00 to 0x6b92c00, (0xb000 bytes) 2024-05-08T08:52:21.502 INFO:teuthology.orchestra.run.smithi196.stdout:814 punch from 0xcce2600 to 0xccec200, (0x9c00 bytes) 2024-05-08T08:52:21.502 INFO:teuthology.orchestra.run.smithi196.stdout:815 trunc from 0xcf67a00 to 0x6f1a400 2024-05-08T08:52:21.502 INFO:teuthology.orchestra.run.smithi196.stdout:817 punch from 0x370f800 to 0x371e000, (0xe800 bytes) 2024-05-08T08:52:21.502 INFO:teuthology.orchestra.run.smithi196.stdout:821 trunc from 0x6f1a400 to 0xae4e00 2024-05-08T08:52:21.503 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:52:21.503 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:52:21.503 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:52:21.504 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:52:21.518 INFO:teuthology.orchestra.run.smithi196.stdout:823 write 0x78a6000 thru 0x78ae1b7 (0x81b8 bytes) 2024-05-08T08:52:21.599 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:21.602+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:21.654 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:52:21.654 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:52:21.654 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:52:21.654 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:52:21.662 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:52:21.700 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:52:21.703+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:52:21.772 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:52:21.772 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:52:21.772 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:52:21.772 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:52:21.772 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:52:21.781 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:52:21.783 INFO:teuthology.orchestra.run.smithi196.stdout:824 write 0x898fa00 thru 0x8998d40 (0x9341 bytes) 2024-05-08T08:52:21.783 INFO:teuthology.orchestra.run.smithi196.stdout:825 read 0x6ddd400 thru 0x6ded1fd (0xfdfe bytes) 2024-05-08T08:52:21.784 INFO:teuthology.orchestra.run.smithi196.stdout:830 write 0x60ac200 thru 0x60ba31c (0xe11d bytes) 2024-05-08T08:52:21.784 INFO:teuthology.orchestra.run.smithi196.stdout:831 write 0x4381c00 thru 0x438c7af (0xabb0 bytes) 2024-05-08T08:52:21.784 INFO:teuthology.orchestra.run.smithi196.stdout:833 write 0x221bc00 thru 0x222b4e0 (0xf8e1 bytes) 2024-05-08T08:52:21.784 INFO:teuthology.orchestra.run.smithi196.stdout:834 read 0x1fec200 thru 0x1ff687f (0xa680 bytes) 2024-05-08T08:52:21.784 INFO:teuthology.orchestra.run.smithi196.stdout:835 write 0x2b87600 thru 0x2b8b51c (0x3f1d bytes) 2024-05-08T08:52:21.784 INFO:teuthology.orchestra.run.smithi196.stdout:837 write 0x370000 thru 0x3733a7 (0x33a8 bytes) 2024-05-08T08:52:21.785 INFO:teuthology.orchestra.run.smithi196.stdout:838 read 0x8597800 thru 0x85a2cf9 (0xb4fa bytes) 2024-05-08T08:52:21.801 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:21.804+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:21.825 INFO:teuthology.orchestra.run.smithi196.stdout:839 write 0xec96a00 thru 0xeca42c6 (0xd8c7 bytes) 2024-05-08T08:52:21.826 INFO:teuthology.orchestra.run.smithi196.stdout:841 read 0x60800 thru 0x63a57 (0x3258 bytes) 2024-05-08T08:52:21.826 INFO:teuthology.orchestra.run.smithi196.stdout:843 read 0x2f25c00 thru 0x2f299b7 (0x3db8 bytes) 2024-05-08T08:52:21.826 INFO:teuthology.orchestra.run.smithi196.stdout:845 punch from 0xdf1de00 to 0xdf27a00, (0x9c00 bytes) 2024-05-08T08:52:21.826 INFO:teuthology.orchestra.run.smithi196.stdout:848 write 0x8046000 thru 0x8050d90 (0xad91 bytes) 2024-05-08T08:52:21.827 INFO:teuthology.orchestra.run.smithi196.stdout:849 clone 3 order 22 su 1048576 sc 13 2024-05-08T08:52:21.902 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:21.906+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:21.904 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:52:21.905 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:52:21.905 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:52:21.905 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:52:21.905 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.905 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(client.4465.0:1557 3.8 e21/18 3:1443f49a:::rbd_data.1140cf79ec85.0000000000000002:head v 21'7, mlcod=21'5)", 2024-05-08T08:52:21.905 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:16.875412+0000", 2024-05-08T08:52:21.905 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 5.0314960739999997, 2024-05-08T08:52:21.905 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.004428909, 2024-05-08T08:52:21.906 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:21.906 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:21.906 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:21.906 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:21.906 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.906 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:21.906 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.875412+0000", 2024-05-08T08:52:21.906 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:21.907 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.907 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.907 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:21.907 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.875412+0000", 2024-05-08T08:52:21.907 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:21.907 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.907 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.907 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:21.907 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.875413+0000", 2024-05-08T08:52:21.907 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.7499999999999998e-07 2024-05-08T08:52:21.907 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.908 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.908 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:21.908 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.875435+0000", 2024-05-08T08:52:21.908 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2427999999999999e-05 2024-05-08T08:52:21.908 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.908 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.908 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:21.908 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.875451+0000", 2024-05-08T08:52:21.908 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5998000000000002e-05 2024-05-08T08:52:21.908 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.908 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.909 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:21.909 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.875459+0000", 2024-05-08T08:52:21.909 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4919999999999993e-06 2024-05-08T08:52:21.909 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.909 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.909 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:21.909 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.875538+0000", 2024-05-08T08:52:21.909 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9009000000000002e-05 2024-05-08T08:52:21.909 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.909 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.909 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:21.909 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.875555+0000", 2024-05-08T08:52:21.910 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6656000000000001e-05 2024-05-08T08:52:21.910 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.910 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.910 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:21.910 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.879799+0000", 2024-05-08T08:52:21.910 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0042440560000000004 2024-05-08T08:52:21.910 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.910 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.910 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:21.911 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.879840+0000", 2024-05-08T08:52:21.911 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1294999999999997e-05 2024-05-08T08:52:21.911 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:21.911 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:21.911 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:21.911 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.911 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.911 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4465.0:2342 3.19 3:9b6e72ee:::rbd_data.1140cf79ec85.0000000000000031:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3362816~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-05-08T08:52:21.911 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:17.302887+0000", 2024-05-08T08:52:21.911 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 4.6040211429999998, 2024-05-08T08:52:21.912 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0041277589999999999, 2024-05-08T08:52:21.912 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:21.912 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:21.912 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:21.912 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:21.912 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4465", 2024-05-08T08:52:21.912 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/390275688", 2024-05-08T08:52:21.912 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2342 2024-05-08T08:52:21.912 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.912 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:21.912 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.912 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:21.913 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.302887+0000", 2024-05-08T08:52:21.913 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:21.913 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.913 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.913 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:21.913 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.302887+0000", 2024-05-08T08:52:21.913 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:21.913 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.913 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.913 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:21.913 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.302888+0000", 2024-05-08T08:52:21.913 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2470000000000001e-06 2024-05-08T08:52:21.913 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.914 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.914 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:21.914 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.302963+0000", 2024-05-08T08:52:21.914 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5199999999999998e-05 2024-05-08T08:52:21.914 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.914 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.914 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:21.914 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.302977+0000", 2024-05-08T08:52:21.914 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3709e-05 2024-05-08T08:52:21.914 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.914 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.914 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:21.915 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.302984+0000", 2024-05-08T08:52:21.915 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0960000000000003e-06 2024-05-08T08:52:21.915 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.915 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.915 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:21.915 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.303031+0000", 2024-05-08T08:52:21.915 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6907000000000003e-05 2024-05-08T08:52:21.915 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.915 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.915 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:21.915 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.303082+0000", 2024-05-08T08:52:21.915 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0871000000000001e-05 2024-05-08T08:52:21.916 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.916 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.916 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 4", 2024-05-08T08:52:21.916 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.303162+0000", 2024-05-08T08:52:21.916 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0925000000000002e-05 2024-05-08T08:52:21.916 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.916 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.916 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:21.916 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.306552+0000", 2024-05-08T08:52:21.916 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0033892850000000001 2024-05-08T08:52:21.916 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.917 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.917 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:21.917 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.306946+0000", 2024-05-08T08:52:21.917 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00039394199999999999 2024-05-08T08:52:21.917 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.917 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.917 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:21.917 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.306984+0000", 2024-05-08T08:52:21.917 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8278999999999997e-05 2024-05-08T08:52:21.917 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.917 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.917 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:21.918 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.307014+0000", 2024-05-08T08:52:21.918 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0298e-05 2024-05-08T08:52:21.918 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:21.918 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:21.918 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:21.918 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.918 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.918 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4465.0:2347 3.19 3:9a36fc17:::rbd_data.1140cf79ec85.0000000000000013:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2260992~28672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-05-08T08:52:21.918 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:17.307197+0000", 2024-05-08T08:52:21.918 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 4.5997109900000002, 2024-05-08T08:52:21.918 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0043368809999999999, 2024-05-08T08:52:21.918 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:21.919 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:21.919 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:21.919 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:21.919 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4465", 2024-05-08T08:52:21.919 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/390275688", 2024-05-08T08:52:21.919 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2347 2024-05-08T08:52:21.919 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.919 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:21.919 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.919 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:21.919 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.307197+0000", 2024-05-08T08:52:21.919 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:21.919 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.920 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.920 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:21.920 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.307197+0000", 2024-05-08T08:52:21.920 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:21.920 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.920 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.920 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:21.920 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.307197+0000", 2024-05-08T08:52:21.920 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0800000000000004e-07 2024-05-08T08:52:21.920 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.920 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.920 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:21.920 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.307217+0000", 2024-05-08T08:52:21.921 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9627999999999999e-05 2024-05-08T08:52:21.921 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.921 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.921 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:21.921 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.307227+0000", 2024-05-08T08:52:21.921 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.7750000000000004e-06 2024-05-08T08:52:21.921 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.921 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.921 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:21.921 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.307232+0000", 2024-05-08T08:52:21.921 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0000000000000004e-06 2024-05-08T08:52:21.921 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.922 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.922 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:21.922 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.307265+0000", 2024-05-08T08:52:21.922 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3074000000000002e-05 2024-05-08T08:52:21.922 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.922 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.922 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:21.922 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.307305+0000", 2024-05-08T08:52:21.922 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0581999999999999e-05 2024-05-08T08:52:21.922 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.922 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.922 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 4", 2024-05-08T08:52:21.923 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.307382+0000", 2024-05-08T08:52:21.923 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.6136000000000004e-05 2024-05-08T08:52:21.923 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.923 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.923 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:21.923 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.310844+0000", 2024-05-08T08:52:21.923 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0034625250000000002 2024-05-08T08:52:21.923 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.923 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.923 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:21.923 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.311482+0000", 2024-05-08T08:52:21.923 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00063792899999999997 2024-05-08T08:52:21.923 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.924 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.924 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:21.924 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.311511+0000", 2024-05-08T08:52:21.924 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9023999999999999e-05 2024-05-08T08:52:21.924 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.924 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.924 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:21.924 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.311534+0000", 2024-05-08T08:52:21.924 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2500000000000001e-05 2024-05-08T08:52:21.924 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:21.924 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:21.924 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:21.925 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.925 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.925 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4465.0:2458 3.19 3:9a36fc17:::rbd_data.1140cf79ec85.0000000000000013:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-05-08T08:52:21.925 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:17.357589+0000", 2024-05-08T08:52:21.925 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 4.5493186149999998, 2024-05-08T08:52:21.925 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0040882360000000003, 2024-05-08T08:52:21.925 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:21.925 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:21.925 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:21.925 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:21.925 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4465", 2024-05-08T08:52:21.925 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/390275688", 2024-05-08T08:52:21.926 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2458 2024-05-08T08:52:21.926 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.926 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:21.926 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.926 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:21.926 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.357589+0000", 2024-05-08T08:52:21.926 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:21.926 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.926 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.926 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:21.926 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.357589+0000", 2024-05-08T08:52:21.926 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:21.927 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.927 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.927 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:21.927 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.357590+0000", 2024-05-08T08:52:21.927 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9999999999999996e-07 2024-05-08T08:52:21.927 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.927 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.927 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:21.927 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.357594+0000", 2024-05-08T08:52:21.927 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7419999999999999e-06 2024-05-08T08:52:21.927 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.927 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.928 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:21.928 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.357604+0000", 2024-05-08T08:52:21.928 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.059e-05 2024-05-08T08:52:21.928 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.928 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.928 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:21.928 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.357611+0000", 2024-05-08T08:52:21.928 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3150000000000003e-06 2024-05-08T08:52:21.928 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.928 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.928 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:21.928 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.357665+0000", 2024-05-08T08:52:21.928 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3075000000000003e-05 2024-05-08T08:52:21.929 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.929 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.929 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:21.929 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.357736+0000", 2024-05-08T08:52:21.929 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0938000000000004e-05 2024-05-08T08:52:21.929 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.929 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.929 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 4", 2024-05-08T08:52:21.929 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.357860+0000", 2024-05-08T08:52:21.929 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012432400000000001 2024-05-08T08:52:21.929 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.929 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.929 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:21.930 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.358334+0000", 2024-05-08T08:52:21.930 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00047395 2024-05-08T08:52:21.930 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.930 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.930 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:21.930 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.361581+0000", 2024-05-08T08:52:21.930 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0032476810000000001 2024-05-08T08:52:21.930 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.930 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.930 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:21.930 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.361634+0000", 2024-05-08T08:52:21.931 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2123999999999998e-05 2024-05-08T08:52:21.931 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.931 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.931 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:21.931 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.361677+0000", 2024-05-08T08:52:21.931 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3696999999999999e-05 2024-05-08T08:52:21.931 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:21.931 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:21.931 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:21.931 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.931 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.931 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4465.0:2580 3.0 3:074a62aa:::rbd_data.1140cf79ec85.0000000000000024:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1454080~66048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-05-08T08:52:21.932 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:17.409757+0000", 2024-05-08T08:52:21.932 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 4.4971507209999997, 2024-05-08T08:52:21.932 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0039510279999999997, 2024-05-08T08:52:21.932 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:21.932 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:21.932 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:21.932 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:21.932 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4465", 2024-05-08T08:52:21.932 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/390275688", 2024-05-08T08:52:21.932 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2580 2024-05-08T08:52:21.932 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.932 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:21.933 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.933 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:21.933 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.409757+0000", 2024-05-08T08:52:21.933 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:21.933 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.933 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.933 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:21.933 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.409757+0000", 2024-05-08T08:52:21.933 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:21.933 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.933 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.933 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:21.933 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.409758+0000", 2024-05-08T08:52:21.934 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.8700000000000004e-07 2024-05-08T08:52:21.934 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.934 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.934 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:21.934 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.412377+0000", 2024-05-08T08:52:21.934 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0026194349999999998 2024-05-08T08:52:21.934 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.934 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.934 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:21.934 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.412388+0000", 2024-05-08T08:52:21.934 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0479000000000001e-05 2024-05-08T08:52:21.935 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.935 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.935 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:21.935 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.412394+0000", 2024-05-08T08:52:21.935 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6499999999999999e-06 2024-05-08T08:52:21.935 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.935 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.935 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:21.935 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.412441+0000", 2024-05-08T08:52:21.935 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7105999999999999e-05 2024-05-08T08:52:21.935 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.936 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.936 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:21.936 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.412490+0000", 2024-05-08T08:52:21.936 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8355e-05 2024-05-08T08:52:21.936 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.936 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.936 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 2", 2024-05-08T08:52:21.936 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.412574+0000", 2024-05-08T08:52:21.936 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3796999999999997e-05 2024-05-08T08:52:21.936 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.936 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.937 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:21.937 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.413477+0000", 2024-05-08T08:52:21.937 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00090339999999999995 2024-05-08T08:52:21.937 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.937 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.937 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:21.937 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.413652+0000", 2024-05-08T08:52:21.937 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00017475499999999999 2024-05-08T08:52:21.938 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.938 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.938 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:21.938 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.413683+0000", 2024-05-08T08:52:21.938 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1333e-05 2024-05-08T08:52:21.938 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.938 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.938 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:21.938 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.413708+0000", 2024-05-08T08:52:21.938 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4831e-05 2024-05-08T08:52:21.938 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:21.939 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:21.939 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:21.939 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.939 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.939 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4465.0:2768 3.19 3:9b6e72ee:::rbd_data.1140cf79ec85.0000000000000031:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2269184~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-05-08T08:52:21.939 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:17.496090+0000", 2024-05-08T08:52:21.939 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 4.4108180590000003, 2024-05-08T08:52:21.939 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0037026239999999998, 2024-05-08T08:52:21.939 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:21.939 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:21.939 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:21.939 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:21.940 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4465", 2024-05-08T08:52:21.940 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/390275688", 2024-05-08T08:52:21.940 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2768 2024-05-08T08:52:21.940 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.940 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:21.940 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.940 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:21.940 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.496090+0000", 2024-05-08T08:52:21.940 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:21.940 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.940 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.940 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:21.941 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.496090+0000", 2024-05-08T08:52:21.941 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:21.941 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.941 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.941 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:21.941 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.496091+0000", 2024-05-08T08:52:21.941 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0780000000000001e-06 2024-05-08T08:52:21.941 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.941 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.941 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:21.941 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.496143+0000", 2024-05-08T08:52:21.941 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.24e-05 2024-05-08T08:52:21.941 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.942 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.942 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:21.942 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.496154+0000", 2024-05-08T08:52:21.942 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.059e-05 2024-05-08T08:52:21.942 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.942 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.942 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:21.942 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.496159+0000", 2024-05-08T08:52:21.942 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.237e-06 2024-05-08T08:52:21.942 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.942 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.942 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:21.943 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.496206+0000", 2024-05-08T08:52:21.943 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7188000000000003e-05 2024-05-08T08:52:21.943 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.943 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.943 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:21.943 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.496255+0000", 2024-05-08T08:52:21.943 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9072e-05 2024-05-08T08:52:21.943 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.943 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.943 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 4", 2024-05-08T08:52:21.943 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.496336+0000", 2024-05-08T08:52:21.943 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1128e-05 2024-05-08T08:52:21.943 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.944 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.944 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:21.944 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.499473+0000", 2024-05-08T08:52:21.944 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.003136841 2024-05-08T08:52:21.944 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.944 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.944 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:21.944 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.499735+0000", 2024-05-08T08:52:21.944 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000261382 2024-05-08T08:52:21.944 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.944 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.944 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:21.945 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.499768+0000", 2024-05-08T08:52:21.945 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3586e-05 2024-05-08T08:52:21.945 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.945 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.945 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:21.945 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.499792+0000", 2024-05-08T08:52:21.945 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4122e-05 2024-05-08T08:52:21.945 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:21.945 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:21.945 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:21.945 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.945 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.945 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4465.0:2777 3.0 3:074a62aa:::rbd_data.1140cf79ec85.0000000000000024:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-05-08T08:52:21.946 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:17.504663+0000", 2024-05-08T08:52:21.946 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 4.4022445130000003, 2024-05-08T08:52:21.946 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0047115480000000003, 2024-05-08T08:52:21.946 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:21.946 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:21.946 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:21.946 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:21.946 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4465", 2024-05-08T08:52:21.946 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/390275688", 2024-05-08T08:52:21.946 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2777 2024-05-08T08:52:21.946 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.946 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:21.947 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.947 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:21.947 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.504663+0000", 2024-05-08T08:52:21.947 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:21.947 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.947 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.947 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:21.947 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.504663+0000", 2024-05-08T08:52:21.947 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:21.947 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.947 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.947 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:21.948 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.504664+0000", 2024-05-08T08:52:21.948 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9400000000000001e-07 2024-05-08T08:52:21.948 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.948 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.948 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:21.948 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.504666+0000", 2024-05-08T08:52:21.948 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4380000000000002e-06 2024-05-08T08:52:21.948 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.948 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.948 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:21.948 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.504674+0000", 2024-05-08T08:52:21.948 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3569999999999998e-06 2024-05-08T08:52:21.949 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.949 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.949 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:21.949 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.504680+0000", 2024-05-08T08:52:21.949 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4859999999999997e-06 2024-05-08T08:52:21.949 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.949 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.949 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:21.949 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.504722+0000", 2024-05-08T08:52:21.949 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2004e-05 2024-05-08T08:52:21.949 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.949 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.949 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:21.950 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.504795+0000", 2024-05-08T08:52:21.950 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3042999999999998e-05 2024-05-08T08:52:21.950 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.950 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.950 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 2", 2024-05-08T08:52:21.950 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.504915+0000", 2024-05-08T08:52:21.950 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011968699999999999 2024-05-08T08:52:21.950 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.950 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.950 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:21.950 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.507133+0000", 2024-05-08T08:52:21.950 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0022181660000000001 2024-05-08T08:52:21.951 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.951 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.951 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:21.951 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.509322+0000", 2024-05-08T08:52:21.951 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0021888340000000002 2024-05-08T08:52:21.951 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.951 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.951 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:21.951 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.509348+0000", 2024-05-08T08:52:21.951 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6332000000000001e-05 2024-05-08T08:52:21.951 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.951 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.951 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:21.952 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.509375+0000", 2024-05-08T08:52:21.952 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6806999999999999e-05 2024-05-08T08:52:21.952 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:21.952 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:21.952 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:21.952 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.952 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.952 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4465.0:2779 3.b 3:d2f86791:::rbd_data.1140cf79ec85.000000000000002f:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-05-08T08:52:21.952 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:17.504692+0000", 2024-05-08T08:52:21.952 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 4.4022157079999999, 2024-05-08T08:52:21.952 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0049124169999999997, 2024-05-08T08:52:21.952 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:21.952 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:21.953 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:21.953 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:21.953 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4465", 2024-05-08T08:52:21.953 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/390275688", 2024-05-08T08:52:21.953 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2779 2024-05-08T08:52:21.953 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.953 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:21.953 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.953 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:21.953 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.504692+0000", 2024-05-08T08:52:21.953 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:21.953 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.954 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.954 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:21.954 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.504692+0000", 2024-05-08T08:52:21.954 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:21.954 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.954 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.954 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:21.954 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.504692+0000", 2024-05-08T08:52:21.954 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8500000000000002e-07 2024-05-08T08:52:21.954 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.954 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.955 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:21.955 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.504695+0000", 2024-05-08T08:52:21.955 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.322e-06 2024-05-08T08:52:21.955 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.955 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.955 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:21.955 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.504703+0000", 2024-05-08T08:52:21.955 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9100000000000005e-06 2024-05-08T08:52:21.955 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.955 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.955 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:21.955 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.504708+0000", 2024-05-08T08:52:21.955 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3000000000000001e-06 2024-05-08T08:52:21.956 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.956 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.956 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:21.956 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.504751+0000", 2024-05-08T08:52:21.956 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3371999999999997e-05 2024-05-08T08:52:21.956 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.956 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.956 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:21.956 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.504820+0000", 2024-05-08T08:52:21.956 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8287999999999994e-05 2024-05-08T08:52:21.956 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.956 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.957 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 0", 2024-05-08T08:52:21.957 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.504915+0000", 2024-05-08T08:52:21.957 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5087999999999995e-05 2024-05-08T08:52:21.957 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.957 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.957 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:21.957 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.509305+0000", 2024-05-08T08:52:21.957 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0043902680000000001 2024-05-08T08:52:21.957 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.957 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.957 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:21.957 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.509537+0000", 2024-05-08T08:52:21.957 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00023209700000000001 2024-05-08T08:52:21.958 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.958 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.958 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:21.958 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.509566+0000", 2024-05-08T08:52:21.958 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8872000000000001e-05 2024-05-08T08:52:21.958 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.958 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.958 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:21.958 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.509604+0000", 2024-05-08T08:52:21.958 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8515e-05 2024-05-08T08:52:21.958 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:21.958 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:21.958 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:21.959 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.959 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.959 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4465.0:2781 3.19 3:9b6e72ee:::rbd_data.1140cf79ec85.0000000000000031:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-05-08T08:52:21.959 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:17.504719+0000", 2024-05-08T08:52:21.959 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 4.4021890350000001, 2024-05-08T08:52:21.959 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0046410219999999999, 2024-05-08T08:52:21.959 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:21.959 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:21.959 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:21.959 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:21.959 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4465", 2024-05-08T08:52:21.959 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/390275688", 2024-05-08T08:52:21.960 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2781 2024-05-08T08:52:21.960 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.960 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:21.960 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.960 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:21.960 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.504719+0000", 2024-05-08T08:52:21.960 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:21.960 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.960 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.960 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:21.960 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.504719+0000", 2024-05-08T08:52:21.960 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:21.960 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.961 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.961 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:21.961 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.504719+0000", 2024-05-08T08:52:21.961 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.96e-07 2024-05-08T08:52:21.961 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.961 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.961 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:21.961 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.504721+0000", 2024-05-08T08:52:21.961 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1900000000000002e-06 2024-05-08T08:52:21.961 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.961 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.961 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:21.961 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.504728+0000", 2024-05-08T08:52:21.962 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3559999999999998e-06 2024-05-08T08:52:21.962 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.962 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.962 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:21.962 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.504733+0000", 2024-05-08T08:52:21.962 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1279999999999999e-06 2024-05-08T08:52:21.962 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.962 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.962 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:21.962 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.504770+0000", 2024-05-08T08:52:21.962 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7185999999999999e-05 2024-05-08T08:52:21.962 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.962 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.963 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:21.963 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.504827+0000", 2024-05-08T08:52:21.963 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6736e-05 2024-05-08T08:52:21.963 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.963 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.963 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 4", 2024-05-08T08:52:21.963 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.504883+0000", 2024-05-08T08:52:21.963 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6637e-05 2024-05-08T08:52:21.963 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.963 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.963 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:21.963 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.506524+0000", 2024-05-08T08:52:21.964 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0016410800000000001 2024-05-08T08:52:21.964 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.964 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.964 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:21.964 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.509293+0000", 2024-05-08T08:52:21.964 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0027684099999999998 2024-05-08T08:52:21.964 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.964 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.964 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:21.964 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.509338+0000", 2024-05-08T08:52:21.964 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5571999999999997e-05 2024-05-08T08:52:21.964 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.965 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.965 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:21.965 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.509360+0000", 2024-05-08T08:52:21.965 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1331e-05 2024-05-08T08:52:21.965 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:21.965 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:21.965 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:21.965 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.965 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.965 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(client.4465.0:3047 3.6 e22/18 3:607c4559:::rbd_data.1140cf79ec85.000000000000001f:head v 22'12, mlcod=22'10)", 2024-05-08T08:52:21.965 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:17.637523+0000", 2024-05-08T08:52:21.965 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 4.269384767, 2024-05-08T08:52:21.966 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0033293229999999999, 2024-05-08T08:52:21.966 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:21.966 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:21.966 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:21.966 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:21.966 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.966 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:21.966 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.637523+0000", 2024-05-08T08:52:21.966 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:21.966 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.966 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.967 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:21.967 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.637523+0000", 2024-05-08T08:52:21.967 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:21.967 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.967 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.967 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:21.967 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.637524+0000", 2024-05-08T08:52:21.967 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5000000000000002e-07 2024-05-08T08:52:21.967 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.967 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.967 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:21.967 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.637548+0000", 2024-05-08T08:52:21.968 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4150000000000001e-05 2024-05-08T08:52:21.968 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.968 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.968 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:21.968 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.637560+0000", 2024-05-08T08:52:21.968 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1838000000000001e-05 2024-05-08T08:52:21.968 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.968 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.968 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:21.968 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.637568+0000", 2024-05-08T08:52:21.968 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.2479999999999996e-06 2024-05-08T08:52:21.968 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.968 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.969 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:21.969 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.637628+0000", 2024-05-08T08:52:21.969 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9623000000000002e-05 2024-05-08T08:52:21.969 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.969 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.969 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:21.969 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.637651+0000", 2024-05-08T08:52:21.969 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3779000000000001e-05 2024-05-08T08:52:21.969 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.969 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.969 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:21.969 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.640819+0000", 2024-05-08T08:52:21.970 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0031681220000000002 2024-05-08T08:52:21.970 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.970 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.970 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:21.970 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.640852+0000", 2024-05-08T08:52:21.970 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.2812999999999999e-05 2024-05-08T08:52:21.970 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:21.970 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:21.970 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:21.971 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.971 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.971 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:1123 3.2 3:4550e1e5:::rbd_id.image_client.0-clone1:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:52:21.971 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:17.982305+0000", 2024-05-08T08:52:21.971 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 3.924602374, 2024-05-08T08:52:21.971 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0035903520000000002, 2024-05-08T08:52:21.971 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:21.971 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:21.972 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:21.972 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:21.972 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:52:21.972 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:52:21.972 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1123 2024-05-08T08:52:21.972 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.972 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:21.972 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.972 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:21.972 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.982305+0000", 2024-05-08T08:52:21.972 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:21.972 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.973 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.973 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:21.973 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.982305+0000", 2024-05-08T08:52:21.973 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:21.973 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.973 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.973 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:21.973 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.982306+0000", 2024-05-08T08:52:21.973 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1739999999999999e-06 2024-05-08T08:52:21.973 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.974 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.974 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:21.974 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.982310+0000", 2024-05-08T08:52:21.974 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8859999999999997e-06 2024-05-08T08:52:21.974 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.974 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.974 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:21.974 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.982318+0000", 2024-05-08T08:52:21.974 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4039999999999998e-06 2024-05-08T08:52:21.974 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.974 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.974 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:21.975 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.982328+0000", 2024-05-08T08:52:21.975 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8060000000000006e-06 2024-05-08T08:52:21.975 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.975 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.975 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:21.975 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.982392+0000", 2024-05-08T08:52:21.975 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4939999999999998e-05 2024-05-08T08:52:21.975 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.975 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.975 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op must wait for map", 2024-05-08T08:52:21.975 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.982397+0000", 2024-05-08T08:52:21.975 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4050000000000002e-06 2024-05-08T08:52:21.975 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.976 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.976 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:21.976 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.984416+0000", 2024-05-08T08:52:21.976 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.002019004 2024-05-08T08:52:21.976 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.976 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.976 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:21.976 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.984545+0000", 2024-05-08T08:52:21.976 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000129567 2024-05-08T08:52:21.976 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.976 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.976 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 5", 2024-05-08T08:52:21.976 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.984774+0000", 2024-05-08T08:52:21.976 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000228562 2024-05-08T08:52:21.977 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.977 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.977 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:21.977 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.985542+0000", 2024-05-08T08:52:21.977 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000768226 2024-05-08T08:52:21.977 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.977 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.977 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:21.977 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.985785+0000", 2024-05-08T08:52:21.977 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000242871 2024-05-08T08:52:21.977 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.978 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.978 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:21.978 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.985851+0000", 2024-05-08T08:52:21.978 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6235000000000005e-05 2024-05-08T08:52:21.978 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.978 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.978 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:21.978 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.985896+0000", 2024-05-08T08:52:21.978 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4271999999999999e-05 2024-05-08T08:52:21.978 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:21.979 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:21.979 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:21.979 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.979 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.979 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4516.0:4626 3.b 3:d48b030f:::rbd_data.1140ff5ddfe9.0000000000000198:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-05-08T08:52:21.979 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:21.438099+0000", 2024-05-08T08:52:21.979 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.46880825799999998, 2024-05-08T08:52:21.979 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.003199164, 2024-05-08T08:52:21.979 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:21.979 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:21.979 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:21.979 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:21.979 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4516", 2024-05-08T08:52:21.980 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1777277669", 2024-05-08T08:52:21.980 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 4626 2024-05-08T08:52:21.980 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.980 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:21.980 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.980 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:21.980 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.438099+0000", 2024-05-08T08:52:21.980 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:21.980 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.980 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.980 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:21.980 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.438099+0000", 2024-05-08T08:52:21.980 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:21.980 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.981 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.981 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:21.981 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.438100+0000", 2024-05-08T08:52:21.981 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9699999999999995e-07 2024-05-08T08:52:21.981 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.981 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.981 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:21.981 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.438105+0000", 2024-05-08T08:52:21.981 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5240000000000002e-06 2024-05-08T08:52:21.981 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.981 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.981 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:21.981 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.438130+0000", 2024-05-08T08:52:21.982 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.5154999999999999e-05 2024-05-08T08:52:21.982 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.982 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.982 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:21.982 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.438141+0000", 2024-05-08T08:52:21.982 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1042e-05 2024-05-08T08:52:21.982 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.982 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.982 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:21.983 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.438203+0000", 2024-05-08T08:52:21.983 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1699999999999995e-05 2024-05-08T08:52:21.983 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.983 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.983 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:21.983 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.438292+0000", 2024-05-08T08:52:21.983 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.9316999999999999e-05 2024-05-08T08:52:21.983 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.983 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.983 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 0", 2024-05-08T08:52:21.983 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.438583+0000", 2024-05-08T08:52:21.983 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00029081499999999999 2024-05-08T08:52:21.983 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.983 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.983 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:21.983 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.440100+0000", 2024-05-08T08:52:21.984 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0015175659999999999 2024-05-08T08:52:21.984 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.984 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.984 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:21.984 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.441213+0000", 2024-05-08T08:52:21.984 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001113139 2024-05-08T08:52:21.984 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.984 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.984 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:21.984 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.441263+0000", 2024-05-08T08:52:21.984 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9249000000000002e-05 2024-05-08T08:52:21.984 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.984 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.984 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:21.984 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.441299+0000", 2024-05-08T08:52:21.985 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5859999999999999e-05 2024-05-08T08:52:21.985 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:21.985 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:21.985 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:21.985 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.985 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.985 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4516.0:4755 3.0 3:0036c371:::rbd_data.1140ff5ddfe9.0000000000000020:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-05-08T08:52:21.985 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:21.469209+0000", 2024-05-08T08:52:21.985 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.43769840700000001, 2024-05-08T08:52:21.985 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0037019700000000002, 2024-05-08T08:52:21.985 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:21.985 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:21.985 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:21.985 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:21.986 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4516", 2024-05-08T08:52:21.986 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1777277669", 2024-05-08T08:52:21.986 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 4755 2024-05-08T08:52:21.986 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.986 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:21.986 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.986 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:21.986 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.469209+0000", 2024-05-08T08:52:21.986 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:21.986 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.986 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.986 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:21.986 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.469209+0000", 2024-05-08T08:52:21.986 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:21.987 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.987 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.987 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:21.987 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.469210+0000", 2024-05-08T08:52:21.987 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3600000000000002e-07 2024-05-08T08:52:21.987 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.987 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.987 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:21.987 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.469214+0000", 2024-05-08T08:52:21.987 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0049999999999998e-06 2024-05-08T08:52:21.987 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.987 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.987 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:21.987 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.469225+0000", 2024-05-08T08:52:21.987 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.044e-05 2024-05-08T08:52:21.987 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.988 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.988 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:21.988 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.469230+0000", 2024-05-08T08:52:21.988 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3650000000000003e-06 2024-05-08T08:52:21.988 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.988 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.988 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:21.988 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.469282+0000", 2024-05-08T08:52:21.988 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2231000000000002e-05 2024-05-08T08:52:21.989 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.989 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.989 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:21.989 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.469366+0000", 2024-05-08T08:52:21.989 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3997000000000002e-05 2024-05-08T08:52:21.989 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.989 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.989 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 2", 2024-05-08T08:52:21.989 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.469474+0000", 2024-05-08T08:52:21.989 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000107433 2024-05-08T08:52:21.989 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.989 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.989 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:21.989 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.470354+0000", 2024-05-08T08:52:21.989 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00088061200000000002 2024-05-08T08:52:21.990 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.990 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.990 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:21.990 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.472832+0000", 2024-05-08T08:52:21.990 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0024776580000000002 2024-05-08T08:52:21.990 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.990 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.990 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:21.990 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.472872+0000", 2024-05-08T08:52:21.990 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0170999999999999e-05 2024-05-08T08:52:21.990 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.990 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.990 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:21.990 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.472911+0000", 2024-05-08T08:52:21.991 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9221999999999998e-05 2024-05-08T08:52:21.991 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:21.991 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:21.991 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:21.991 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.991 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.991 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4516.0:4758 3.0 3:053fa6db:::rbd_data.1140ff5ddfe9.000000000000002a:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-05-08T08:52:21.991 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:21.469244+0000", 2024-05-08T08:52:21.991 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.437663199, 2024-05-08T08:52:21.991 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0060817939999999997, 2024-05-08T08:52:21.991 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:21.991 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:21.991 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:21.992 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:21.992 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4516", 2024-05-08T08:52:21.992 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1777277669", 2024-05-08T08:52:21.992 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 4758 2024-05-08T08:52:21.992 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.992 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:21.992 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.992 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:21.992 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.469244+0000", 2024-05-08T08:52:21.992 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:21.993 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.993 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.993 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:21.993 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.469244+0000", 2024-05-08T08:52:21.993 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:21.993 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.993 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.993 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:21.993 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.469245+0000", 2024-05-08T08:52:21.993 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1399999999999997e-07 2024-05-08T08:52:21.993 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.993 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.994 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:21.994 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.469250+0000", 2024-05-08T08:52:21.994 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8160000000000001e-06 2024-05-08T08:52:21.994 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.994 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.994 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:21.994 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.469257+0000", 2024-05-08T08:52:21.994 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1799999999999999e-06 2024-05-08T08:52:21.994 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.994 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.994 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:21.994 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.469264+0000", 2024-05-08T08:52:21.994 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5009999999999998e-06 2024-05-08T08:52:21.995 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.995 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.995 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:21.995 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.469828+0000", 2024-05-08T08:52:21.995 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000563136 2024-05-08T08:52:21.995 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.995 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.995 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:21.995 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.469906+0000", 2024-05-08T08:52:21.995 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8613000000000001e-05 2024-05-08T08:52:21.995 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.995 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.995 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 2", 2024-05-08T08:52:21.995 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.470013+0000", 2024-05-08T08:52:21.996 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000107363 2024-05-08T08:52:21.996 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.996 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.996 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:21.996 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.472932+0000", 2024-05-08T08:52:21.996 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.002918655 2024-05-08T08:52:21.996 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.996 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.996 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:21.996 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.475238+0000", 2024-05-08T08:52:21.996 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0023056660000000001 2024-05-08T08:52:21.996 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.996 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.997 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:21.997 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.475287+0000", 2024-05-08T08:52:21.997 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8789000000000003e-05 2024-05-08T08:52:21.997 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.997 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.997 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:21.997 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.475326+0000", 2024-05-08T08:52:21.997 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9660999999999999e-05 2024-05-08T08:52:21.997 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:21.997 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:21.997 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:21.998 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.998 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.998 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(client.4516.0:4752 3.1a e26/18 3:5b3f15b5:::rbd_data.1140ff5ddfe9.000000000000001e:head v 26'58, mlcod=26'56)", 2024-05-08T08:52:21.998 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:21.469528+0000", 2024-05-08T08:52:21.998 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.43738012799999998, 2024-05-08T08:52:21.998 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.003339758, 2024-05-08T08:52:21.998 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:21.998 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:21.998 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:21.998 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:21.999 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:21.999 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:21.999 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.469528+0000", 2024-05-08T08:52:21.999 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:21.999 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:21.999 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.000 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:22.000 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.469528+0000", 2024-05-08T08:52:22.000 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:22.000 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.000 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.000 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:22.000 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.469528+0000", 2024-05-08T08:52:22.001 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1500000000000003e-07 2024-05-08T08:52:22.001 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.001 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.001 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:22.001 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.469534+0000", 2024-05-08T08:52:22.001 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8289999999999996e-06 2024-05-08T08:52:22.001 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.001 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.001 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:22.001 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.469544+0000", 2024-05-08T08:52:22.001 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0304e-05 2024-05-08T08:52:22.001 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.002 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.002 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:22.002 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.469552+0000", 2024-05-08T08:52:22.002 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0999999999999998e-06 2024-05-08T08:52:22.002 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.002 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.002 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:22.002 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.469607+0000", 2024-05-08T08:52:22.002 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5228e-05 2024-05-08T08:52:22.002 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.002 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.002 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:22.003 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.469636+0000", 2024-05-08T08:52:22.003 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9271000000000001e-05 2024-05-08T08:52:22.003 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.003 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.003 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:22.003 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.472824+0000", 2024-05-08T08:52:22.003 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0031881129999999998 2024-05-08T08:52:22.004 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.004 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.004 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:22.004 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.472867+0000", 2024-05-08T08:52:22.004 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3098e-05 2024-05-08T08:52:22.004 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:22.004 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:22.004 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:22.004 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.004 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.005 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4516.0:4771 3.19 3:9c7cd016:::rbd_data.1140ff5ddfe9.000000000000004e:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-05-08T08:52:22.005 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:21.470235+0000", 2024-05-08T08:52:22.005 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.43667276999999999, 2024-05-08T08:52:22.005 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00499418, 2024-05-08T08:52:22.005 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:22.005 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:22.005 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:22.005 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:22.005 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4516", 2024-05-08T08:52:22.005 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1777277669", 2024-05-08T08:52:22.005 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 4771 2024-05-08T08:52:22.005 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.006 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:22.006 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.006 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:22.006 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.470235+0000", 2024-05-08T08:52:22.006 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:22.006 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.006 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.006 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:22.006 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.470235+0000", 2024-05-08T08:52:22.006 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:22.006 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.006 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.007 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:22.007 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.470236+0000", 2024-05-08T08:52:22.007 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3200000000000004e-07 2024-05-08T08:52:22.007 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.007 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.007 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:22.007 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.470240+0000", 2024-05-08T08:52:22.007 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3370000000000001e-06 2024-05-08T08:52:22.007 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.007 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.007 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:22.007 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.470260+0000", 2024-05-08T08:52:22.007 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0228e-05 2024-05-08T08:52:22.007 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.008 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.008 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:22.008 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.470269+0000", 2024-05-08T08:52:22.008 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.833e-06 2024-05-08T08:52:22.008 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.008 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.008 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:22.008 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.470318+0000", 2024-05-08T08:52:22.008 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8667e-05 2024-05-08T08:52:22.008 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.008 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.008 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:22.008 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.470391+0000", 2024-05-08T08:52:22.009 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2945e-05 2024-05-08T08:52:22.009 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.009 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.009 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 4", 2024-05-08T08:52:22.009 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.470487+0000", 2024-05-08T08:52:22.009 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5798999999999995e-05 2024-05-08T08:52:22.009 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.009 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.009 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:22.009 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.471291+0000", 2024-05-08T08:52:22.009 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00080471899999999996 2024-05-08T08:52:22.009 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.010 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.010 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:22.010 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.475140+0000", 2024-05-08T08:52:22.010 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.003849014 2024-05-08T08:52:22.010 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.010 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.010 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:22.010 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.475186+0000", 2024-05-08T08:52:22.010 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6128999999999998e-05 2024-05-08T08:52:22.011 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.011 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.011 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:22.011 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.475229+0000", 2024-05-08T08:52:22.011 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2676999999999998e-05 2024-05-08T08:52:22.011 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:22.012 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:22.012 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:22.012 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.012 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.012 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4516.0:4809 3.4 3:266a05d3:::rbd_data.1140ff5ddfe9.0000000000000074:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-05-08T08:52:22.012 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:21.475220+0000", 2024-05-08T08:52:22.013 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.43168725800000002, 2024-05-08T08:52:22.013 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.004957655, 2024-05-08T08:52:22.013 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:22.013 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:22.013 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:22.013 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:22.013 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4516", 2024-05-08T08:52:22.014 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1777277669", 2024-05-08T08:52:22.014 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 4809 2024-05-08T08:52:22.014 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.014 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:22.014 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.014 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:22.015 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.475220+0000", 2024-05-08T08:52:22.015 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:22.015 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.015 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.015 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:22.015 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.475220+0000", 2024-05-08T08:52:22.015 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:22.015 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.015 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.015 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:22.015 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.475221+0000", 2024-05-08T08:52:22.015 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5199999999999995e-07 2024-05-08T08:52:22.016 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.016 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.016 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:22.016 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.475225+0000", 2024-05-08T08:52:22.016 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0559999999999998e-06 2024-05-08T08:52:22.016 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.016 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.016 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:22.016 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.475236+0000", 2024-05-08T08:52:22.016 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1027e-05 2024-05-08T08:52:22.016 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.016 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.016 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:22.016 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.475244+0000", 2024-05-08T08:52:22.016 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8779999999999998e-06 2024-05-08T08:52:22.017 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.017 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.017 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:22.017 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.475307+0000", 2024-05-08T08:52:22.017 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2349999999999998e-05 2024-05-08T08:52:22.017 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.017 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.017 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:22.017 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.475396+0000", 2024-05-08T08:52:22.017 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.9797999999999996e-05 2024-05-08T08:52:22.017 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.017 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.017 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 2", 2024-05-08T08:52:22.017 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.475501+0000", 2024-05-08T08:52:22.017 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000104541 2024-05-08T08:52:22.017 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.018 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.018 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:22.018 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.477854+0000", 2024-05-08T08:52:22.018 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0023532890000000002 2024-05-08T08:52:22.018 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.018 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.018 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:22.018 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.480081+0000", 2024-05-08T08:52:22.018 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0022266479999999999 2024-05-08T08:52:22.018 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.018 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.018 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:22.018 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.480134+0000", 2024-05-08T08:52:22.018 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2957000000000001e-05 2024-05-08T08:52:22.019 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.019 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.019 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:22.019 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.480178+0000", 2024-05-08T08:52:22.019 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4159000000000002e-05 2024-05-08T08:52:22.019 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:22.019 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:22.019 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:22.019 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.019 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.019 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(client.4516.0:4820 3.7 e26/18 3:e7640ded:::rbd_data.1140ff5ddfe9.000000000000007d:head v 26'42, mlcod=26'40)", 2024-05-08T08:52:22.019 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:21.476102+0000", 2024-05-08T08:52:22.019 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.430806048, 2024-05-08T08:52:22.019 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0040419439999999996, 2024-05-08T08:52:22.020 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:22.020 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:22.020 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:22.020 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:22.020 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.020 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:22.020 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.476102+0000", 2024-05-08T08:52:22.020 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:22.020 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.020 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.020 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:22.020 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.476102+0000", 2024-05-08T08:52:22.020 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:22.020 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.021 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.021 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:22.021 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.476102+0000", 2024-05-08T08:52:22.021 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3799999999999996e-07 2024-05-08T08:52:22.021 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.021 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.021 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:22.021 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.476108+0000", 2024-05-08T08:52:22.021 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4929999999999998e-06 2024-05-08T08:52:22.021 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.021 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.021 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:22.021 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.476122+0000", 2024-05-08T08:52:22.022 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3859999999999999e-05 2024-05-08T08:52:22.022 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.022 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.022 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:22.022 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.476128+0000", 2024-05-08T08:52:22.022 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.584e-06 2024-05-08T08:52:22.022 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.022 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.022 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:22.022 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.476222+0000", 2024-05-08T08:52:22.022 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3907999999999996e-05 2024-05-08T08:52:22.022 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.023 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.023 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:22.023 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.476241+0000", 2024-05-08T08:52:22.023 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.8413000000000001e-05 2024-05-08T08:52:22.023 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.023 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.023 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:22.023 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.480103+0000", 2024-05-08T08:52:22.023 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0038627990000000001 2024-05-08T08:52:22.023 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.023 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.023 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:22.023 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.480144+0000", 2024-05-08T08:52:22.023 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0148999999999998e-05 2024-05-08T08:52:22.024 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:22.024 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:22.024 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:22.024 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.024 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.024 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(client.4516.0:4826 3.1a e26/18 3:5ae30c8c:::rbd_data.1140ff5ddfe9.000000000000007f:head v 26'60, mlcod=26'58)", 2024-05-08T08:52:22.024 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:21.476802+0000", 2024-05-08T08:52:22.024 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.430105456, 2024-05-08T08:52:22.024 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0033911229999999998, 2024-05-08T08:52:22.024 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:22.024 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:22.024 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:22.024 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:22.024 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.024 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:22.025 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.476802+0000", 2024-05-08T08:52:22.025 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:22.025 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.025 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.025 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:22.025 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.476802+0000", 2024-05-08T08:52:22.025 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:22.025 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.025 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.026 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:22.026 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.476803+0000", 2024-05-08T08:52:22.026 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5000000000000002e-07 2024-05-08T08:52:22.026 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.026 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.026 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:22.026 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.476808+0000", 2024-05-08T08:52:22.026 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2240000000000003e-06 2024-05-08T08:52:22.026 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.026 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.026 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:22.026 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.476821+0000", 2024-05-08T08:52:22.026 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2559e-05 2024-05-08T08:52:22.026 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.027 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.027 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:22.027 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.476827+0000", 2024-05-08T08:52:22.027 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3069999999999999e-06 2024-05-08T08:52:22.027 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.027 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.027 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:22.027 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.476882+0000", 2024-05-08T08:52:22.027 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5223999999999998e-05 2024-05-08T08:52:22.027 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.027 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.027 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:22.027 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.476904+0000", 2024-05-08T08:52:22.027 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1594999999999999e-05 2024-05-08T08:52:22.027 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.028 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.028 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:22.028 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.480121+0000", 2024-05-08T08:52:22.028 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0032169379999999999 2024-05-08T08:52:22.028 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.028 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.028 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:22.028 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.480193+0000", 2024-05-08T08:52:22.028 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2626000000000006e-05 2024-05-08T08:52:22.028 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:22.028 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:22.028 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:22.028 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.028 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.029 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4516.0:4830 3.4 3:235b1dd4:::rbd_data.1140ff5ddfe9.0000000000000080:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-05-08T08:52:22.029 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:21.477007+0000", 2024-05-08T08:52:22.029 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.42990072400000001, 2024-05-08T08:52:22.029 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0035180300000000001, 2024-05-08T08:52:22.029 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:22.029 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:22.029 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:22.029 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:22.029 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4516", 2024-05-08T08:52:22.029 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1777277669", 2024-05-08T08:52:22.029 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 4830 2024-05-08T08:52:22.029 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.029 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:22.030 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.030 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:22.030 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.477007+0000", 2024-05-08T08:52:22.030 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:22.030 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.030 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.030 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:22.030 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.477007+0000", 2024-05-08T08:52:22.030 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:22.030 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.030 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.030 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:22.031 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.477008+0000", 2024-05-08T08:52:22.031 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7800000000000001e-07 2024-05-08T08:52:22.031 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.031 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.031 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:22.031 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.477012+0000", 2024-05-08T08:52:22.031 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2119999999999997e-06 2024-05-08T08:52:22.031 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.031 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.031 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:22.031 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.477033+0000", 2024-05-08T08:52:22.031 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0978000000000001e-05 2024-05-08T08:52:22.032 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.032 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.032 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:22.032 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.477041+0000", 2024-05-08T08:52:22.032 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4149999999999999e-06 2024-05-08T08:52:22.032 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.032 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.032 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:22.032 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.477099+0000", 2024-05-08T08:52:22.032 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7766000000000002e-05 2024-05-08T08:52:22.032 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.032 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.032 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:22.032 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.477177+0000", 2024-05-08T08:52:22.033 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8208999999999996e-05 2024-05-08T08:52:22.033 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.033 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.033 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 2", 2024-05-08T08:52:22.033 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.477287+0000", 2024-05-08T08:52:22.033 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000109632 2024-05-08T08:52:22.033 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.033 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.033 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:22.033 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.480185+0000", 2024-05-08T08:52:22.033 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.002898603 2024-05-08T08:52:22.033 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.034 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.034 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:22.034 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.480450+0000", 2024-05-08T08:52:22.034 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00026431600000000001 2024-05-08T08:52:22.034 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.034 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.034 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:22.034 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.480486+0000", 2024-05-08T08:52:22.034 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6646000000000001e-05 2024-05-08T08:52:22.034 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.034 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.034 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:22.034 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.480525+0000", 2024-05-08T08:52:22.035 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8575000000000003e-05 2024-05-08T08:52:22.035 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:22.035 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:22.035 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:22.035 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:22.035 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:22.035 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:52:22.035 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-05-08T08:52:22.035 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T08:52:22.036 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-05-08T08:52:22.036 INFO:tasks.thrashosds.thrasher:Setting osd 5 primary_affinity to 0.279897 2024-05-08T08:52:22.036 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 0.27989700981805676 2024-05-08T08:52:22.037 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-05-08T08:52:22.039 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:52:22.039 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:52:22.041+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:52:22.139 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:52:22.142+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:52:22.240 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:22.243+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:22.268 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:52:22.268 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:52:22.268 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:52:22.268 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:52:22.282 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:52:22.341 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:22.344+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:22.386 INFO:teuthology.orchestra.run.smithi196.stdout:checking clone #1, image image_client.0-clone1 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent2 2024-05-08T08:52:22.398 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:52:22.398 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:52:22.398 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:52:22.399 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:52:22.399 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:52:22.407 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:52:22.441 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:22.444+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:22.443 INFO:teuthology.orchestra.run.smithi196.stdout:851 read 0xb23c400 thru 0xb23d76a (0x136b bytes) 2024-05-08T08:52:22.443 INFO:teuthology.orchestra.run.smithi196.stdout:852 write 0x3809200 thru 0x3809eac (0xcad bytes) 2024-05-08T08:52:22.443 INFO:teuthology.orchestra.run.smithi196.stdout:853 write 0x8d31000 thru 0x8d3d59e (0xc59f bytes) 2024-05-08T08:52:22.455 INFO:teuthology.orchestra.run.smithi196.stdout:856 trunc from 0xeca4400 to 0x57d6c00 2024-05-08T08:52:22.513 INFO:teuthology.orchestra.run.smithi196.stdout:859 trunc from 0x57d6c00 to 0xd784600 2024-05-08T08:52:22.523 INFO:teuthology.orchestra.run.smithi196.stdout:860 write 0x2000 thru 0x2f06 (0xf07 bytes) 2024-05-08T08:52:22.531 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:52:22.531 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:52:22.531 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:52:22.531 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:52:22.531 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.531 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4465.0:1557 3.8 3:1443f49a:::rbd_data.1140cf79ec85.0000000000000002:head [write 2011136~49152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-05-08T08:52:22.531 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:16.875070+0000", 2024-05-08T08:52:22.532 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 5.6583585149999998, 2024-05-08T08:52:22.532 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0050154939999999997, 2024-05-08T08:52:22.532 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:22.532 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:22.532 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:22.532 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:22.532 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4465", 2024-05-08T08:52:22.532 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/390275688", 2024-05-08T08:52:22.532 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1557 2024-05-08T08:52:22.532 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.532 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:22.532 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.532 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:22.532 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.875070+0000", 2024-05-08T08:52:22.533 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:22.533 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.533 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.533 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:22.533 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.875070+0000", 2024-05-08T08:52:22.533 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:22.533 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.533 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.533 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:22.533 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.875074+0000", 2024-05-08T08:52:22.533 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4599999999999999e-06 2024-05-08T08:52:22.533 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.533 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.533 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:22.533 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.875135+0000", 2024-05-08T08:52:22.534 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1413000000000003e-05 2024-05-08T08:52:22.534 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.534 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.534 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:22.534 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.875148+0000", 2024-05-08T08:52:22.534 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2692e-05 2024-05-08T08:52:22.534 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.534 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.534 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:22.534 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.875156+0000", 2024-05-08T08:52:22.534 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1960000000000008e-06 2024-05-08T08:52:22.534 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.535 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.535 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:22.535 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.875203+0000", 2024-05-08T08:52:22.535 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7139999999999999e-05 2024-05-08T08:52:22.535 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.535 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.535 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:22.535 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.875250+0000", 2024-05-08T08:52:22.535 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7227e-05 2024-05-08T08:52:22.535 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.535 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.535 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 1", 2024-05-08T08:52:22.535 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.875338+0000", 2024-05-08T08:52:22.535 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.8104999999999998e-05 2024-05-08T08:52:22.535 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.536 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.536 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:22.536 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.879970+0000", 2024-05-08T08:52:22.536 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0046313220000000002 2024-05-08T08:52:22.536 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.536 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.536 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:22.536 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.879988+0000", 2024-05-08T08:52:22.536 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.7890999999999999e-05 2024-05-08T08:52:22.536 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.536 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.536 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:22.536 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.880039+0000", 2024-05-08T08:52:22.536 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1672000000000003e-05 2024-05-08T08:52:22.537 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.537 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.537 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:22.537 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:16.880086+0000", 2024-05-08T08:52:22.537 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6375999999999998e-05 2024-05-08T08:52:22.537 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:22.537 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:22.537 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:22.537 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.537 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.537 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4465.0:1966 3.1e 3:7e8411aa:::rbd_data.1140cf79ec85.0000000000000008:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-05-08T08:52:22.537 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:17.073457+0000", 2024-05-08T08:52:22.537 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 5.4599713100000002, 2024-05-08T08:52:22.537 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0058203980000000001, 2024-05-08T08:52:22.538 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:22.538 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:22.538 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:22.538 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:22.538 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4465", 2024-05-08T08:52:22.538 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/390275688", 2024-05-08T08:52:22.538 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1966 2024-05-08T08:52:22.538 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.538 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:22.538 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.538 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:22.538 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.073457+0000", 2024-05-08T08:52:22.538 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:22.538 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.539 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.539 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:22.539 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.073457+0000", 2024-05-08T08:52:22.539 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:22.539 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.539 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.539 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:22.539 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.073459+0000", 2024-05-08T08:52:22.539 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2359999999999999e-06 2024-05-08T08:52:22.539 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.539 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.539 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:22.539 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.073464+0000", 2024-05-08T08:52:22.539 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9089999999999999e-06 2024-05-08T08:52:22.540 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.540 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.540 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:22.540 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.073475+0000", 2024-05-08T08:52:22.540 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1328999999999999e-05 2024-05-08T08:52:22.540 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.540 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.540 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:22.540 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.073481+0000", 2024-05-08T08:52:22.540 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1700000000000002e-06 2024-05-08T08:52:22.540 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.540 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.540 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:22.540 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.073535+0000", 2024-05-08T08:52:22.540 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3544000000000001e-05 2024-05-08T08:52:22.541 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.541 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.541 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:22.541 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.073585+0000", 2024-05-08T08:52:22.541 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0692000000000002e-05 2024-05-08T08:52:22.541 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.541 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.541 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 3", 2024-05-08T08:52:22.541 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.073644+0000", 2024-05-08T08:52:22.541 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9015000000000003e-05 2024-05-08T08:52:22.541 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.541 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.541 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:22.541 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.074282+0000", 2024-05-08T08:52:22.542 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00063794299999999997 2024-05-08T08:52:22.542 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.542 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.542 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:22.542 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.079217+0000", 2024-05-08T08:52:22.542 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0049343369999999996 2024-05-08T08:52:22.542 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.542 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.542 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:22.542 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.079249+0000", 2024-05-08T08:52:22.542 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.2419000000000003e-05 2024-05-08T08:52:22.542 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.542 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.542 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:22.543 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:17.079278+0000", 2024-05-08T08:52:22.543 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8804000000000001e-05 2024-05-08T08:52:22.543 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:22.543 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:22.543 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:22.543 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.543 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.543 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4492.0:158 3.7 3:e6681e91:::rbd_data.114052026309.000000000000000a:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-05-08T08:52:22.543 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:19.024654+0000", 2024-05-08T08:52:22.543 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 3.5087749499999998, 2024-05-08T08:52:22.543 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0061168050000000003, 2024-05-08T08:52:22.543 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:22.543 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:22.543 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:22.543 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:22.544 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4492", 2024-05-08T08:52:22.544 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1076427944", 2024-05-08T08:52:22.544 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 158 2024-05-08T08:52:22.544 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.544 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:22.544 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.544 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:22.544 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.024654+0000", 2024-05-08T08:52:22.544 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:22.544 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.544 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.544 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:22.544 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.024654+0000", 2024-05-08T08:52:22.544 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:22.544 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.545 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.545 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:22.545 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.024655+0000", 2024-05-08T08:52:22.545 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.049e-06 2024-05-08T08:52:22.545 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.545 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.545 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:22.545 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.024661+0000", 2024-05-08T08:52:22.545 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1380000000000004e-06 2024-05-08T08:52:22.545 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.545 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.545 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:22.545 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.024681+0000", 2024-05-08T08:52:22.545 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0469999999999999e-05 2024-05-08T08:52:22.545 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.546 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.546 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:22.546 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.024699+0000", 2024-05-08T08:52:22.546 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.7935999999999999e-05 2024-05-08T08:52:22.546 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.546 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.546 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:22.546 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.024790+0000", 2024-05-08T08:52:22.546 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0152e-05 2024-05-08T08:52:22.546 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.546 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.546 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:22.546 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.024983+0000", 2024-05-08T08:52:22.546 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00019358 2024-05-08T08:52:22.547 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.547 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.547 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:22.547 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.030771+0000", 2024-05-08T08:52:22.547 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0057874800000000002 2024-05-08T08:52:22.547 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:22.547 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:22.547 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:22.547 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.547 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.547 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4492.0:178 3.7 3:e4b1bfac:::rbd_data.114052026309.0000000000000016:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-05-08T08:52:22.547 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:19.025199+0000", 2024-05-08T08:52:22.547 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 3.5082297809999998, 2024-05-08T08:52:22.547 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.010455720999999999, 2024-05-08T08:52:22.547 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:22.547 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:22.548 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:22.548 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:22.548 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4492", 2024-05-08T08:52:22.548 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1076427944", 2024-05-08T08:52:22.548 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 178 2024-05-08T08:52:22.548 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.548 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:22.548 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.548 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:22.549 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.025199+0000", 2024-05-08T08:52:22.549 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:22.549 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.549 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.549 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:22.549 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.025199+0000", 2024-05-08T08:52:22.549 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:22.549 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.549 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.549 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:22.549 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.025200+0000", 2024-05-08T08:52:22.549 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7199999999999998e-07 2024-05-08T08:52:22.549 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.549 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.550 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:22.550 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.025204+0000", 2024-05-08T08:52:22.550 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0509999999999998e-06 2024-05-08T08:52:22.550 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.550 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.550 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:22.550 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.025215+0000", 2024-05-08T08:52:22.550 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1677e-05 2024-05-08T08:52:22.550 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.550 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.550 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:22.550 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.025222+0000", 2024-05-08T08:52:22.550 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8210000000000004e-06 2024-05-08T08:52:22.550 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.551 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.551 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:22.551 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.025280+0000", 2024-05-08T08:52:22.551 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8270999999999998e-05 2024-05-08T08:52:22.551 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.551 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.551 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:22.551 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.025393+0000", 2024-05-08T08:52:22.551 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000112261 2024-05-08T08:52:22.551 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.551 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.551 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:22.551 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.035655+0000", 2024-05-08T08:52:22.551 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.010261968 2024-05-08T08:52:22.551 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:22.552 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:22.552 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:22.552 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.552 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.552 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4492.0:196 3.18 3:1ba353bf:::rbd_data.114052026309.0000000000000022:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-05-08T08:52:22.552 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:19.025247+0000", 2024-05-08T08:52:22.552 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 3.5081819479999998, 2024-05-08T08:52:22.552 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.007774329, 2024-05-08T08:52:22.552 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:22.552 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:22.552 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:22.552 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:22.552 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4492", 2024-05-08T08:52:22.552 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1076427944", 2024-05-08T08:52:22.553 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 196 2024-05-08T08:52:22.553 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.553 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:22.553 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.553 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:22.553 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.025247+0000", 2024-05-08T08:52:22.553 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:22.553 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.553 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.553 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:22.553 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.025247+0000", 2024-05-08T08:52:22.553 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:22.553 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.553 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.553 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:22.554 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.025247+0000", 2024-05-08T08:52:22.554 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5599999999999995e-07 2024-05-08T08:52:22.554 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.554 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.554 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:22.554 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.025251+0000", 2024-05-08T08:52:22.554 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3239999999999999e-06 2024-05-08T08:52:22.554 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.554 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.554 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:22.554 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.025267+0000", 2024-05-08T08:52:22.554 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6096e-05 2024-05-08T08:52:22.554 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.555 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.555 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:22.555 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.025273+0000", 2024-05-08T08:52:22.555 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3990000000000001e-06 2024-05-08T08:52:22.555 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.555 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.555 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:22.555 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.027242+0000", 2024-05-08T08:52:22.555 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0019685890000000002 2024-05-08T08:52:22.555 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.555 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.555 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:22.555 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.027342+0000", 2024-05-08T08:52:22.555 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010051399999999999 2024-05-08T08:52:22.556 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.556 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.556 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:22.556 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.033021+0000", 2024-05-08T08:52:22.556 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0056788510000000004 2024-05-08T08:52:22.556 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:22.556 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:22.556 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:22.556 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.556 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.556 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4492.0:216 3.18 3:1ba1d62d:::rbd_data.114052026309.000000000000002e:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-05-08T08:52:22.556 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:19.027010+0000", 2024-05-08T08:52:22.556 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 3.5064190499999999, 2024-05-08T08:52:22.556 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0061188279999999998, 2024-05-08T08:52:22.556 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:22.557 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:22.557 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:22.557 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:22.557 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4492", 2024-05-08T08:52:22.557 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1076427944", 2024-05-08T08:52:22.557 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 216 2024-05-08T08:52:22.557 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.557 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:22.557 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.557 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:22.557 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.027010+0000", 2024-05-08T08:52:22.557 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:22.557 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.557 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.557 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:22.558 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.027010+0000", 2024-05-08T08:52:22.558 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:22.558 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.558 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.558 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:22.558 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.027011+0000", 2024-05-08T08:52:22.558 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.5000000000000001e-07 2024-05-08T08:52:22.558 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.558 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.558 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:22.558 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.027015+0000", 2024-05-08T08:52:22.558 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6190000000000002e-06 2024-05-08T08:52:22.558 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.558 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.559 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:22.559 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.027026+0000", 2024-05-08T08:52:22.559 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0984e-05 2024-05-08T08:52:22.559 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.559 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.559 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:22.559 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.027034+0000", 2024-05-08T08:52:22.559 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0940000000000008e-06 2024-05-08T08:52:22.559 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.559 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.559 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:22.559 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.027759+0000", 2024-05-08T08:52:22.559 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00072523200000000003 2024-05-08T08:52:22.559 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.560 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.560 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:22.560 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.027882+0000", 2024-05-08T08:52:22.560 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012280200000000001 2024-05-08T08:52:22.560 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.560 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.560 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:22.560 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.033129+0000", 2024-05-08T08:52:22.560 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0052462469999999999 2024-05-08T08:52:22.560 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:22.560 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:22.560 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:22.560 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.560 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.560 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4492.0:217 3.8 3:14f1ba96:::rbd_data.114052026309.000000000000002f:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-05-08T08:52:22.561 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:19.027077+0000", 2024-05-08T08:52:22.561 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 3.5063519680000002, 2024-05-08T08:52:22.561 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0093162899999999996, 2024-05-08T08:52:22.561 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:22.561 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:22.561 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:22.561 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:22.561 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4492", 2024-05-08T08:52:22.561 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1076427944", 2024-05-08T08:52:22.561 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 217 2024-05-08T08:52:22.561 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.561 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:22.561 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.561 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:22.561 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.027077+0000", 2024-05-08T08:52:22.562 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:22.562 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.562 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.562 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:22.562 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.027077+0000", 2024-05-08T08:52:22.562 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:22.562 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.562 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.562 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:22.562 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.027078+0000", 2024-05-08T08:52:22.562 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9800000000000003e-07 2024-05-08T08:52:22.562 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.562 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.562 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:22.563 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.027082+0000", 2024-05-08T08:52:22.563 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8829999999999997e-06 2024-05-08T08:52:22.563 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.563 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.563 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:22.563 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.027093+0000", 2024-05-08T08:52:22.563 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0549e-05 2024-05-08T08:52:22.563 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.563 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.563 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:22.563 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.027100+0000", 2024-05-08T08:52:22.563 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1999999999999997e-06 2024-05-08T08:52:22.563 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.563 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.564 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:22.564 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.027859+0000", 2024-05-08T08:52:22.564 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00075841700000000005 2024-05-08T08:52:22.564 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.564 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.564 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:22.564 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.027971+0000", 2024-05-08T08:52:22.564 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000112522 2024-05-08T08:52:22.564 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.564 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.564 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:22.564 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.036393+0000", 2024-05-08T08:52:22.564 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0084219210000000006 2024-05-08T08:52:22.564 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:22.565 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:22.565 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:22.565 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.565 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.565 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "pg_update_log_missing(3.0 epoch 24/18 rep_tid 47 entries 24'11 (0'0) error 3:0366b645:::rbd_data.114052026309.0000000000000008:head by client.4492.0:154 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 22'7)", 2024-05-08T08:52:22.565 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:19.027274+0000", 2024-05-08T08:52:22.565 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 3.5061549030000001, 2024-05-08T08:52:22.565 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0053859049999999999, 2024-05-08T08:52:22.565 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:22.565 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:22.565 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "reached pg", 2024-05-08T08:52:22.565 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:22.565 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.565 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:22.565 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.027274+0000", 2024-05-08T08:52:22.566 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:22.566 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.566 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.566 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:22.566 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.027274+0000", 2024-05-08T08:52:22.566 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:22.566 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.566 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.566 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:22.566 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.027274+0000", 2024-05-08T08:52:22.566 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8200000000000001e-07 2024-05-08T08:52:22.566 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.566 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.566 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:22.567 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.027281+0000", 2024-05-08T08:52:22.567 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1300000000000003e-06 2024-05-08T08:52:22.567 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.567 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.567 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:22.567 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.027297+0000", 2024-05-08T08:52:22.567 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6113999999999999e-05 2024-05-08T08:52:22.567 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.567 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.567 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:22.567 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.027320+0000", 2024-05-08T08:52:22.567 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2439999999999999e-05 2024-05-08T08:52:22.567 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.567 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.567 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:22.567 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.027607+0000", 2024-05-08T08:52:22.568 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00028719299999999999 2024-05-08T08:52:22.568 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.568 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.568 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:22.568 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.032660+0000", 2024-05-08T08:52:22.568 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.005052646 2024-05-08T08:52:22.568 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:22.568 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:22.568 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:22.568 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.568 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.568 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "pg_update_log_missing(3.0 epoch 24/18 rep_tid 48 entries 24'12 (0'0) error 3:074bcfc1:::rbd_data.114052026309.000000000000000c:head by client.4492.0:161 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 22'7)", 2024-05-08T08:52:22.568 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:19.027335+0000", 2024-05-08T08:52:22.568 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 3.506093457, 2024-05-08T08:52:22.569 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0053387799999999996, 2024-05-08T08:52:22.569 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:22.569 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:22.569 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "reached pg", 2024-05-08T08:52:22.569 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:22.569 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.569 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:22.569 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.027335+0000", 2024-05-08T08:52:22.569 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:22.569 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.569 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.569 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:22.569 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.027335+0000", 2024-05-08T08:52:22.569 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:22.569 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.570 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.570 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:22.570 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.027336+0000", 2024-05-08T08:52:22.570 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.72e-07 2024-05-08T08:52:22.570 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.570 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.570 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:22.570 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.027342+0000", 2024-05-08T08:52:22.570 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9030000000000001e-06 2024-05-08T08:52:22.570 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.570 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.570 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:22.570 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.027355+0000", 2024-05-08T08:52:22.570 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.363e-05 2024-05-08T08:52:22.571 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.571 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.571 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:22.571 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.027367+0000", 2024-05-08T08:52:22.571 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2087e-05 2024-05-08T08:52:22.571 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.571 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.571 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:22.571 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.027729+0000", 2024-05-08T08:52:22.571 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00036192499999999997 2024-05-08T08:52:22.571 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.571 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.571 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:22.571 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.032674+0000", 2024-05-08T08:52:22.572 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0049448629999999999 2024-05-08T08:52:22.572 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:22.572 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:22.572 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:22.572 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.572 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.572 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "pg_update_log_missing(3.5 epoch 24/18 rep_tid 83 entries 24'3 (0'0) error 3:a4b5bd4b:::rbd_data.114052026309.000000000000002b:head by client.4492.0:210 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-05-08T08:52:22.572 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:19.027485+0000", 2024-05-08T08:52:22.572 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 3.5059436019999999, 2024-05-08T08:52:22.572 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0048204739999999999, 2024-05-08T08:52:22.572 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:22.572 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:22.572 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "reached pg", 2024-05-08T08:52:22.572 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:22.573 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.573 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:22.573 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.027485+0000", 2024-05-08T08:52:22.573 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:22.573 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.573 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.573 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:22.573 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.027485+0000", 2024-05-08T08:52:22.573 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:22.573 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.573 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.573 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:22.573 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.027486+0000", 2024-05-08T08:52:22.573 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6600000000000002e-07 2024-05-08T08:52:22.573 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.574 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.574 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:22.574 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.027493+0000", 2024-05-08T08:52:22.574 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.447e-06 2024-05-08T08:52:22.574 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.574 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.574 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:22.574 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.027504+0000", 2024-05-08T08:52:22.574 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1362000000000001e-05 2024-05-08T08:52:22.574 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.574 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.574 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:22.574 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.027516+0000", 2024-05-08T08:52:22.575 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1491999999999999e-05 2024-05-08T08:52:22.575 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.575 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.575 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:22.575 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.027647+0000", 2024-05-08T08:52:22.575 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013060099999999999 2024-05-08T08:52:22.575 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.575 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.575 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:22.575 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.032306+0000", 2024-05-08T08:52:22.575 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0046591059999999997 2024-05-08T08:52:22.575 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:22.575 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:22.575 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:22.576 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.576 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.576 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4492.0:228 3.7 3:e5b7b4f7:::rbd_object_map.114052026309.0000000000000004:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-05-08T08:52:22.576 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:19.028495+0000", 2024-05-08T08:52:22.576 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 3.5049334999999999, 2024-05-08T08:52:22.576 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0073659370000000004, 2024-05-08T08:52:22.576 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:22.576 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:22.576 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:22.576 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:22.576 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4492", 2024-05-08T08:52:22.576 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1076427944", 2024-05-08T08:52:22.576 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 228 2024-05-08T08:52:22.576 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.576 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:22.576 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.577 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:22.577 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.028495+0000", 2024-05-08T08:52:22.577 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:22.577 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.577 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.577 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:22.577 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.028495+0000", 2024-05-08T08:52:22.577 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:22.577 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.577 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.577 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:22.577 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.028496+0000", 2024-05-08T08:52:22.577 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5499999999999998e-07 2024-05-08T08:52:22.577 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.577 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.578 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:22.578 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.028501+0000", 2024-05-08T08:52:22.578 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7450000000000003e-06 2024-05-08T08:52:22.578 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.578 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.578 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:22.578 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.028508+0000", 2024-05-08T08:52:22.578 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.6260000000000003e-06 2024-05-08T08:52:22.578 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.578 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.578 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:22.578 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.028514+0000", 2024-05-08T08:52:22.578 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5949999999999998e-06 2024-05-08T08:52:22.578 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.579 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.579 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:22.579 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.028844+0000", 2024-05-08T08:52:22.579 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00033033900000000002 2024-05-08T08:52:22.579 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.579 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.579 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:22.579 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.028935+0000", 2024-05-08T08:52:22.579 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0704000000000004e-05 2024-05-08T08:52:22.579 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.579 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.579 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 1", 2024-05-08T08:52:22.579 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.029308+0000", 2024-05-08T08:52:22.579 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00037283200000000002 2024-05-08T08:52:22.579 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.579 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.580 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:22.580 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.034998+0000", 2024-05-08T08:52:22.580 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0056903910000000004 2024-05-08T08:52:22.580 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.580 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.580 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:22.580 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.035702+0000", 2024-05-08T08:52:22.580 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00070417600000000004 2024-05-08T08:52:22.580 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.580 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.580 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:22.580 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.035813+0000", 2024-05-08T08:52:22.580 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000111189 2024-05-08T08:52:22.580 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.580 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.581 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:22.581 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.035861+0000", 2024-05-08T08:52:22.581 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7784999999999997e-05 2024-05-08T08:52:22.581 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:22.581 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:22.581 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:22.581 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.581 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.581 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4492.0:276 3.7 3:e5b7b4f7:::rbd_object_map.114052026309.0000000000000004:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-05-08T08:52:22.581 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:19.030797+0000", 2024-05-08T08:52:22.581 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 3.5026314259999998, 2024-05-08T08:52:22.581 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0052181119999999996, 2024-05-08T08:52:22.581 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:22.581 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:22.582 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:22.582 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:22.582 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4492", 2024-05-08T08:52:22.582 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1076427944", 2024-05-08T08:52:22.582 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 276 2024-05-08T08:52:22.582 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.582 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:22.582 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.582 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:22.582 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.030797+0000", 2024-05-08T08:52:22.582 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:22.582 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.582 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.582 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:22.583 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.030797+0000", 2024-05-08T08:52:22.583 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:22.583 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.583 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.583 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:22.583 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.030798+0000", 2024-05-08T08:52:22.583 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4600000000000004e-07 2024-05-08T08:52:22.583 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.583 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.583 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:22.583 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.030804+0000", 2024-05-08T08:52:22.583 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.896e-06 2024-05-08T08:52:22.583 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.583 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.583 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:22.584 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.030814+0000", 2024-05-08T08:52:22.584 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.859e-06 2024-05-08T08:52:22.584 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.584 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.584 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:22.584 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.030829+0000", 2024-05-08T08:52:22.584 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5344000000000001e-05 2024-05-08T08:52:22.584 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.584 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.584 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:22.584 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.030882+0000", 2024-05-08T08:52:22.584 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2565999999999998e-05 2024-05-08T08:52:22.584 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.584 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.585 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:22.585 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.030969+0000", 2024-05-08T08:52:22.585 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7368999999999996e-05 2024-05-08T08:52:22.585 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.585 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.585 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 1", 2024-05-08T08:52:22.585 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.031324+0000", 2024-05-08T08:52:22.585 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00035461999999999999 2024-05-08T08:52:22.585 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.585 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.585 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:22.585 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.035003+0000", 2024-05-08T08:52:22.585 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.003679631 2024-05-08T08:52:22.585 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.585 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.586 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:22.586 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.035912+0000", 2024-05-08T08:52:22.586 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00090909499999999996 2024-05-08T08:52:22.586 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.586 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.586 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:22.586 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.035964+0000", 2024-05-08T08:52:22.586 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1941000000000003e-05 2024-05-08T08:52:22.586 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.586 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.586 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:22.586 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.036015+0000", 2024-05-08T08:52:22.586 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1044999999999997e-05 2024-05-08T08:52:22.586 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:22.587 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:22.587 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:22.587 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.587 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.587 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4492.0:283 3.7 3:e5b7b4f7:::rbd_object_map.114052026309.0000000000000004:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-05-08T08:52:22.587 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:19.031699+0000", 2024-05-08T08:52:22.587 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 3.5017295310000001, 2024-05-08T08:52:22.587 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0062351439999999998, 2024-05-08T08:52:22.587 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:22.587 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:22.587 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:22.587 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:22.587 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4492", 2024-05-08T08:52:22.587 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1076427944", 2024-05-08T08:52:22.588 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 283 2024-05-08T08:52:22.588 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.588 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:22.588 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.588 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:22.588 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.031699+0000", 2024-05-08T08:52:22.588 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:22.588 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.588 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.588 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:22.588 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.031699+0000", 2024-05-08T08:52:22.588 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:22.588 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.588 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.588 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:22.589 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.031700+0000", 2024-05-08T08:52:22.589 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4900000000000004e-07 2024-05-08T08:52:22.589 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.589 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.589 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:22.589 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.031706+0000", 2024-05-08T08:52:22.589 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4990000000000002e-06 2024-05-08T08:52:22.589 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.589 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.589 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:22.589 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.031721+0000", 2024-05-08T08:52:22.589 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5441e-05 2024-05-08T08:52:22.589 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.589 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.589 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:22.590 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.031729+0000", 2024-05-08T08:52:22.590 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.6799999999999993e-06 2024-05-08T08:52:22.590 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.590 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.590 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:22.590 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.031799+0000", 2024-05-08T08:52:22.590 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0674999999999997e-05 2024-05-08T08:52:22.590 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.590 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.590 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:22.590 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.031881+0000", 2024-05-08T08:52:22.590 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1063000000000006e-05 2024-05-08T08:52:22.590 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.590 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.590 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 1", 2024-05-08T08:52:22.591 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.032211+0000", 2024-05-08T08:52:22.591 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00033053800000000001 2024-05-08T08:52:22.591 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.591 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.591 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:22.591 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.036064+0000", 2024-05-08T08:52:22.591 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0038524950000000001 2024-05-08T08:52:22.591 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.591 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.591 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:22.591 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.037224+0000", 2024-05-08T08:52:22.591 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001160887 2024-05-08T08:52:22.591 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.591 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.592 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:22.592 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.037274+0000", 2024-05-08T08:52:22.592 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9188999999999999e-05 2024-05-08T08:52:22.592 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.592 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.592 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:22.592 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.037934+0000", 2024-05-08T08:52:22.592 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00066072799999999997 2024-05-08T08:52:22.592 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:22.592 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:22.592 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:22.592 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.592 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.592 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4492.0:298 3.18 3:1ba353bf:::rbd_data.114052026309.0000000000000022:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-05-08T08:52:22.592 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:19.033611+0000", 2024-05-08T08:52:22.593 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 3.4998180489999999, 2024-05-08T08:52:22.593 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0075427589999999996, 2024-05-08T08:52:22.593 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:22.593 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:22.593 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:22.593 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:22.593 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4492", 2024-05-08T08:52:22.593 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1076427944", 2024-05-08T08:52:22.593 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 298 2024-05-08T08:52:22.593 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.593 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:22.593 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.593 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:22.593 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.033611+0000", 2024-05-08T08:52:22.594 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:22.594 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.594 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.594 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:22.594 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.033611+0000", 2024-05-08T08:52:22.594 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:22.594 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.594 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.594 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:22.594 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.033612+0000", 2024-05-08T08:52:22.594 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1999999999999998e-07 2024-05-08T08:52:22.594 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.594 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.594 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:22.594 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.033616+0000", 2024-05-08T08:52:22.595 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7360000000000001e-06 2024-05-08T08:52:22.595 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.595 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.595 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:22.595 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.033626+0000", 2024-05-08T08:52:22.595 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6779999999999995e-06 2024-05-08T08:52:22.595 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.595 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.595 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:22.595 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.033633+0000", 2024-05-08T08:52:22.595 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.776e-06 2024-05-08T08:52:22.595 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.595 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.595 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:22.595 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.033687+0000", 2024-05-08T08:52:22.596 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4119999999999997e-05 2024-05-08T08:52:22.596 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.596 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.596 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:22.596 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.033770+0000", 2024-05-08T08:52:22.596 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3182000000000004e-05 2024-05-08T08:52:22.596 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.596 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.596 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 0", 2024-05-08T08:52:22.596 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.033875+0000", 2024-05-08T08:52:22.596 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010516 2024-05-08T08:52:22.596 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.596 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.596 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:22.597 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.039839+0000", 2024-05-08T08:52:22.597 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00596395 2024-05-08T08:52:22.597 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.597 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.597 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:22.597 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.041061+0000", 2024-05-08T08:52:22.597 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001221953 2024-05-08T08:52:22.597 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.597 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.597 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:22.597 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.041113+0000", 2024-05-08T08:52:22.597 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1635000000000003e-05 2024-05-08T08:52:22.597 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.597 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.598 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:22.598 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.041153+0000", 2024-05-08T08:52:22.598 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0748999999999999e-05 2024-05-08T08:52:22.598 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:22.598 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:22.598 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:22.598 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.598 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.598 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(client.4492.0:354 3.0 e24/18 3:04f88de1:::rbd_data.114052026309.0000000000000003:head v 24'13, mlcod=24'11)", 2024-05-08T08:52:22.598 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:19.037612+0000", 2024-05-08T08:52:22.598 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 3.4958171390000001, 2024-05-08T08:52:22.598 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0052023690000000001, 2024-05-08T08:52:22.598 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:22.598 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:22.599 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:22.599 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:22.599 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.599 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:22.599 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.037612+0000", 2024-05-08T08:52:22.599 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:22.599 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.599 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.599 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:22.599 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.037612+0000", 2024-05-08T08:52:22.599 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:22.599 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.599 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.600 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:22.600 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.037612+0000", 2024-05-08T08:52:22.600 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0800000000000004e-07 2024-05-08T08:52:22.600 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.600 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.600 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:22.600 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.037618+0000", 2024-05-08T08:52:22.600 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.626e-06 2024-05-08T08:52:22.600 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.600 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.600 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:22.600 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.037628+0000", 2024-05-08T08:52:22.600 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0125e-05 2024-05-08T08:52:22.600 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.600 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.600 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:22.601 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.037635+0000", 2024-05-08T08:52:22.601 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.798e-06 2024-05-08T08:52:22.601 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.601 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.601 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:22.601 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.037737+0000", 2024-05-08T08:52:22.601 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000101829 2024-05-08T08:52:22.601 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.601 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.601 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:22.601 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.037759+0000", 2024-05-08T08:52:22.601 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2671999999999999e-05 2024-05-08T08:52:22.601 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.601 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.601 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:22.602 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.042780+0000", 2024-05-08T08:52:22.602 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0050208869999999999 2024-05-08T08:52:22.602 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.602 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.602 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:22.602 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.042814+0000", 2024-05-08T08:52:22.602 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3824e-05 2024-05-08T08:52:22.602 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:22.602 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:22.602 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:22.602 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.602 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.602 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(client.4492.0:373 3.5 e24/18 3:a4b5bd4b:::rbd_data.114052026309.000000000000002b:head v 24'5, mlcod=24'3)", 2024-05-08T08:52:22.602 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:19.040089+0000", 2024-05-08T08:52:22.602 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 3.493339653, 2024-05-08T08:52:22.603 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.005476776, 2024-05-08T08:52:22.603 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:22.603 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:22.603 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:22.603 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:22.603 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.603 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:22.603 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.040089+0000", 2024-05-08T08:52:22.603 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:22.603 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.603 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.603 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:22.603 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.040089+0000", 2024-05-08T08:52:22.603 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:22.603 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.604 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.604 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:22.604 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.040090+0000", 2024-05-08T08:52:22.604 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8400000000000003e-07 2024-05-08T08:52:22.604 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.604 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.604 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:22.604 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.040118+0000", 2024-05-08T08:52:22.604 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8266e-05 2024-05-08T08:52:22.604 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.604 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.604 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:22.604 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.040139+0000", 2024-05-08T08:52:22.604 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0554e-05 2024-05-08T08:52:22.604 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.605 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.605 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:22.605 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.040147+0000", 2024-05-08T08:52:22.605 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8159999999999994e-06 2024-05-08T08:52:22.605 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.605 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.605 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:22.605 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.040192+0000", 2024-05-08T08:52:22.605 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5163000000000001e-05 2024-05-08T08:52:22.605 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.605 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.605 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:22.605 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.040210+0000", 2024-05-08T08:52:22.605 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.8287999999999998e-05 2024-05-08T08:52:22.605 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.606 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.606 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:22.606 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.045532+0000", 2024-05-08T08:52:22.606 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0053221020000000004 2024-05-08T08:52:22.606 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.606 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.606 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:22.606 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.045566+0000", 2024-05-08T08:52:22.606 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3803000000000001e-05 2024-05-08T08:52:22.606 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:22.606 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:22.606 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:22.606 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.606 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.606 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(client.4492.0:384 3.9 e24/18 3:9074a2d7:::rbd_data.114052026309.0000000000000036:head v 24'2, mlcod=0'0)", 2024-05-08T08:52:22.607 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:19.040227+0000", 2024-05-08T08:52:22.607 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 3.4932019140000001, 2024-05-08T08:52:22.607 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0052631889999999997, 2024-05-08T08:52:22.607 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:22.607 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:22.607 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:22.607 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:22.607 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.607 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:22.607 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.040227+0000", 2024-05-08T08:52:22.607 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:22.607 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.607 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.607 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:22.607 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.040227+0000", 2024-05-08T08:52:22.607 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:22.608 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.608 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.608 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:22.608 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.040227+0000", 2024-05-08T08:52:22.608 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0200000000000002e-07 2024-05-08T08:52:22.608 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.608 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.608 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:22.608 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.040233+0000", 2024-05-08T08:52:22.608 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2429999999999996e-06 2024-05-08T08:52:22.608 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.608 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.608 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:22.608 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.041466+0000", 2024-05-08T08:52:22.608 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0012328619999999999 2024-05-08T08:52:22.609 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.609 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.609 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:22.609 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.041475+0000", 2024-05-08T08:52:22.609 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0140000000000004e-06 2024-05-08T08:52:22.609 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.609 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.609 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:22.609 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.041542+0000", 2024-05-08T08:52:22.609 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7535999999999998e-05 2024-05-08T08:52:22.609 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.609 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.609 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:22.609 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.041561+0000", 2024-05-08T08:52:22.609 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.8918000000000001e-05 2024-05-08T08:52:22.610 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.610 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.610 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:22.610 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.045447+0000", 2024-05-08T08:52:22.610 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0038859049999999998 2024-05-08T08:52:22.610 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.610 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.610 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:22.610 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:19.045490+0000", 2024-05-08T08:52:22.610 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3108999999999997e-05 2024-05-08T08:52:22.610 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:22.610 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:22.610 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:22.610 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.611 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.611 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(client.4516.0:4529 3.5 e26/18 3:a38a8548:::rbd_object_map.1140ff5ddfe9:head v 26'372, mlcod=26'367)", 2024-05-08T08:52:22.611 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:21.423966+0000", 2024-05-08T08:52:22.611 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 1.109462401, 2024-05-08T08:52:22.611 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0053358110000000002, 2024-05-08T08:52:22.611 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:22.611 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:22.611 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:22.611 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:22.611 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.611 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:22.611 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.423966+0000", 2024-05-08T08:52:22.611 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:22.611 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.612 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.612 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:22.612 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.423966+0000", 2024-05-08T08:52:22.612 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:22.612 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.612 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.612 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:22.612 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.423967+0000", 2024-05-08T08:52:22.612 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6599999999999996e-07 2024-05-08T08:52:22.612 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.612 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.612 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:22.612 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.423974+0000", 2024-05-08T08:52:22.612 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7429999999999996e-06 2024-05-08T08:52:22.612 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.613 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.613 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:22.613 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.423986+0000", 2024-05-08T08:52:22.613 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2278e-05 2024-05-08T08:52:22.613 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.613 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.613 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:22.613 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.423992+0000", 2024-05-08T08:52:22.613 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3130000000000004e-06 2024-05-08T08:52:22.613 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.613 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.613 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:22.613 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.424035+0000", 2024-05-08T08:52:22.613 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3226000000000003e-05 2024-05-08T08:52:22.613 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.614 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.614 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:22.614 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.424052+0000", 2024-05-08T08:52:22.614 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6498999999999999e-05 2024-05-08T08:52:22.614 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.614 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.614 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:22.614 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.429269+0000", 2024-05-08T08:52:22.614 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0052168099999999997 2024-05-08T08:52:22.614 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.614 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.614 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:22.614 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.429302+0000", 2024-05-08T08:52:22.614 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3376e-05 2024-05-08T08:52:22.615 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:22.615 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:22.615 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:22.615 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.615 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.615 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(client.4516.0:4758 3.0 e26/18 3:053fa6db:::rbd_data.1140ff5ddfe9.000000000000002a:head v 26'49, mlcod=26'46)", 2024-05-08T08:52:22.615 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:21.470152+0000", 2024-05-08T08:52:22.615 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 1.0632763329999999, 2024-05-08T08:52:22.615 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0049072109999999999, 2024-05-08T08:52:22.615 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:22.615 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:22.615 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:22.615 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:22.615 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.615 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:22.616 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.470152+0000", 2024-05-08T08:52:22.616 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:22.616 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.616 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.616 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:22.616 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.470152+0000", 2024-05-08T08:52:22.616 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:22.616 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.616 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.616 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:22.616 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.470153+0000", 2024-05-08T08:52:22.616 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2200000000000004e-07 2024-05-08T08:52:22.616 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.616 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.616 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:22.617 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.470159+0000", 2024-05-08T08:52:22.617 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0599999999999996e-06 2024-05-08T08:52:22.617 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.617 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.617 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:22.617 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.470184+0000", 2024-05-08T08:52:22.617 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4854e-05 2024-05-08T08:52:22.617 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.617 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.617 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:22.617 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.470206+0000", 2024-05-08T08:52:22.617 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2082999999999998e-05 2024-05-08T08:52:22.617 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.617 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.617 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:22.618 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.470260+0000", 2024-05-08T08:52:22.618 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4126999999999999e-05 2024-05-08T08:52:22.618 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.618 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.618 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:22.618 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.470279+0000", 2024-05-08T08:52:22.618 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.8853000000000001e-05 2024-05-08T08:52:22.618 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.618 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.618 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:22.618 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.475015+0000", 2024-05-08T08:52:22.618 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0047359780000000001 2024-05-08T08:52:22.618 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.618 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.619 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:22.619 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.475060+0000", 2024-05-08T08:52:22.619 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4634e-05 2024-05-08T08:52:22.619 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:22.619 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:22.619 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:22.619 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.619 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.619 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4516.0:4777 3.18 3:1f77ca61:::rbd_data.1140ff5ddfe9.0000000000000050:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-05-08T08:52:22.619 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:21.470747+0000", 2024-05-08T08:52:22.619 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 1.062682127, 2024-05-08T08:52:22.619 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0049194169999999997, 2024-05-08T08:52:22.619 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:22.619 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:22.619 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:22.620 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:22.620 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4516", 2024-05-08T08:52:22.620 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1777277669", 2024-05-08T08:52:22.620 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 4777 2024-05-08T08:52:22.620 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.620 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:22.620 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.620 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:22.620 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.470747+0000", 2024-05-08T08:52:22.620 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:22.620 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.620 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.620 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:22.620 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.470747+0000", 2024-05-08T08:52:22.620 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:22.621 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.621 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.621 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:22.621 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.470748+0000", 2024-05-08T08:52:22.621 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1699999999999997e-07 2024-05-08T08:52:22.621 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.621 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.621 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:22.621 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.470754+0000", 2024-05-08T08:52:22.621 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7959999999999998e-06 2024-05-08T08:52:22.621 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.621 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.621 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:22.621 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.470768+0000", 2024-05-08T08:52:22.621 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.4151999999999999e-05 2024-05-08T08:52:22.622 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.622 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.622 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:22.622 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.470780+0000", 2024-05-08T08:52:22.622 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1558e-05 2024-05-08T08:52:22.622 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.622 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.622 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:22.622 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.470843+0000", 2024-05-08T08:52:22.622 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2512000000000001e-05 2024-05-08T08:52:22.622 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.622 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.622 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:22.622 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.470928+0000", 2024-05-08T08:52:22.623 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.5197000000000004e-05 2024-05-08T08:52:22.623 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.623 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.623 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 0", 2024-05-08T08:52:22.623 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.471065+0000", 2024-05-08T08:52:22.623 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013729000000000001 2024-05-08T08:52:22.623 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.623 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.623 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:22.623 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.475068+0000", 2024-05-08T08:52:22.623 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0040031939999999998 2024-05-08T08:52:22.623 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.623 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.623 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:22.623 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.475578+0000", 2024-05-08T08:52:22.624 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00050961599999999995 2024-05-08T08:52:22.624 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.624 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.624 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:22.624 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.475630+0000", 2024-05-08T08:52:22.624 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2422000000000001e-05 2024-05-08T08:52:22.624 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:22.624 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:22.624 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:22.624 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:21.475666+0000", 2024-05-08T08:52:22.624 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5763000000000003e-05 2024-05-08T08:52:22.624 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:22.624 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:22.624 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:22.624 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:22.625 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:22.625 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:52:22.627 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:52:22.627 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:52:22.629+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:52:22.628 INFO:teuthology.orchestra.run.smithi196.stdout:861 trunc from 0xd784600 to 0x72a9200 2024-05-08T08:52:22.628 INFO:teuthology.orchestra.run.smithi196.stdout:862 read 0x46c2600 thru 0x46c840f (0x5e10 bytes) 2024-05-08T08:52:22.628 INFO:teuthology.orchestra.run.smithi196.stdout:864 flatten 2024-05-08T08:52:22.628 INFO:teuthology.orchestra.run.smithi196.stdout:865 read 0x73e000 thru 0x74a295 (0xc296 bytes) 2024-05-08T08:52:22.628 INFO:teuthology.orchestra.run.smithi196.stdout:868 write 0x3b20000 thru 0x3b2f903 (0xf904 bytes) 2024-05-08T08:52:22.628 INFO:teuthology.orchestra.run.smithi196.stdout:869 write 0xd6e0200 thru 0xd6e8292 (0x8093 bytes) 2024-05-08T08:52:22.628 INFO:teuthology.orchestra.run.smithi196.stdout:870 clone 4 order 19 su 2048 sc 8 2024-05-08T08:52:22.726 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:22.730+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:22.734 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:52:22.734 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:52:22.734 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:52:22.734 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:52:22.743 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:52:22.827 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:22.831+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:22.888 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:52:22.888 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:52:22.888 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:52:22.888 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:52:22.888 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:52:22.897 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:52:22.939 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:22.942+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:22.956 INFO:teuthology.orchestra.run.smithi059.stderr:noscrub is set 2024-05-08T08:52:22.960 INFO:teuthology.orchestra.run.smithi059.stderr:set osd.5 primary-affinity to 0.279897 (8183432) 2024-05-08T08:52:23.030 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:52:23.033+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:52:23.047 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:52:23.047 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:52:23.047 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:52:23.047 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:52:23.047 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.047 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4189.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-05-08T08:52:23.047 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:51:58.895617+0000", 2024-05-08T08:52:23.047 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 24.153299134000001, 2024-05-08T08:52:23.047 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0098298229999999997, 2024-05-08T08:52:23.048 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:23.048 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:23.048 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:23.048 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:23.048 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4189", 2024-05-08T08:52:23.048 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/574763665", 2024-05-08T08:52:23.048 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 36 2024-05-08T08:52:23.048 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.048 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:23.048 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.048 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:23.048 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:58.895617+0000", 2024-05-08T08:52:23.048 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.048 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.048 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.049 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:23.049 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:58.895617+0000", 2024-05-08T08:52:23.049 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.049 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.049 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.049 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:23.049 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:58.895618+0000", 2024-05-08T08:52:23.049 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3499999999999996e-07 2024-05-08T08:52:23.049 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.049 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.049 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:23.049 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:58.895623+0000", 2024-05-08T08:52:23.049 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9400000000000001e-06 2024-05-08T08:52:23.049 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.050 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.050 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:23.050 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:58.895629+0000", 2024-05-08T08:52:23.050 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4849999999999997e-06 2024-05-08T08:52:23.050 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.050 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.050 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:23.050 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:58.895635+0000", 2024-05-08T08:52:23.050 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6300000000000003e-06 2024-05-08T08:52:23.050 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.050 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.050 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:23.050 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:58.899506+0000", 2024-05-08T08:52:23.050 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003871425 2024-05-08T08:52:23.051 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.051 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.051 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:23.051 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:58.899561+0000", 2024-05-08T08:52:23.051 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4214e-05 2024-05-08T08:52:23.051 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.051 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.051 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-05-08T08:52:23.051 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:58.899641+0000", 2024-05-08T08:52:23.051 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0527999999999999e-05 2024-05-08T08:52:23.051 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.051 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.051 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:52:23.052 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:58.900556+0000", 2024-05-08T08:52:23.052 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00091496000000000004 2024-05-08T08:52:23.052 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.052 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.052 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:23.052 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:58.905397+0000", 2024-05-08T08:52:23.052 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0048413700000000002 2024-05-08T08:52:23.052 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.052 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.052 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:23.052 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:58.905426+0000", 2024-05-08T08:52:23.052 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8158e-05 2024-05-08T08:52:23.052 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.052 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.053 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:23.053 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:58.905447+0000", 2024-05-08T08:52:23.053 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1478e-05 2024-05-08T08:52:23.053 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.053 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:23.053 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.053 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.053 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.053 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4189.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-05-08T08:52:23.053 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:51:58.895645+0000", 2024-05-08T08:52:23.053 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 24.153271103000002, 2024-05-08T08:52:23.053 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0099493129999999996, 2024-05-08T08:52:23.053 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:23.053 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:23.054 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:23.054 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:23.054 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4189", 2024-05-08T08:52:23.054 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/574763665", 2024-05-08T08:52:23.054 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 37 2024-05-08T08:52:23.054 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.054 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:23.054 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.054 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:23.054 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:58.895645+0000", 2024-05-08T08:52:23.054 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.054 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.054 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.055 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:23.055 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:58.895645+0000", 2024-05-08T08:52:23.055 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.055 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.055 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.055 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:23.055 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:58.895646+0000", 2024-05-08T08:52:23.055 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0999999999999999e-07 2024-05-08T08:52:23.055 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.055 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.055 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:23.055 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:58.895650+0000", 2024-05-08T08:52:23.055 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4279999999999998e-06 2024-05-08T08:52:23.055 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.056 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.056 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:23.056 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:58.895657+0000", 2024-05-08T08:52:23.056 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9029999999999998e-06 2024-05-08T08:52:23.056 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.056 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.056 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:23.056 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:58.895666+0000", 2024-05-08T08:52:23.056 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2779999999999999e-06 2024-05-08T08:52:23.056 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.056 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.056 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:23.056 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:58.899901+0000", 2024-05-08T08:52:23.056 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0042346689999999999 2024-05-08T08:52:23.057 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.057 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.057 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:23.057 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:58.899998+0000", 2024-05-08T08:52:23.057 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6903000000000004e-05 2024-05-08T08:52:23.057 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.057 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.057 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-05-08T08:52:23.057 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:58.900134+0000", 2024-05-08T08:52:23.057 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00013635599999999999 2024-05-08T08:52:23.057 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.057 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.057 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:52:23.058 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:58.900632+0000", 2024-05-08T08:52:23.058 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00049720500000000002 2024-05-08T08:52:23.058 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.058 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.058 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:23.058 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:58.905492+0000", 2024-05-08T08:52:23.058 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0048606200000000004 2024-05-08T08:52:23.058 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.058 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.058 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:23.058 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:58.905550+0000", 2024-05-08T08:52:23.058 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8171999999999998e-05 2024-05-08T08:52:23.058 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.059 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.059 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:23.059 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:58.905595+0000", 2024-05-08T08:52:23.059 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4268999999999999e-05 2024-05-08T08:52:23.059 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.059 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:23.059 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.059 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.059 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.059 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4189.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-05-08T08:52:23.059 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:51:59.117325+0000", 2024-05-08T08:52:23.059 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.931591825000002, 2024-05-08T08:52:23.059 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0085358469999999992, 2024-05-08T08:52:23.059 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:23.060 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:23.060 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:23.060 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:23.060 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4189", 2024-05-08T08:52:23.060 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/574763665", 2024-05-08T08:52:23.060 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 110 2024-05-08T08:52:23.060 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.060 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:23.060 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.060 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:23.060 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117325+0000", 2024-05-08T08:52:23.060 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.060 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.060 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.061 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:23.061 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117325+0000", 2024-05-08T08:52:23.061 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.061 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.061 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.061 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:23.061 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117325+0000", 2024-05-08T08:52:23.061 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6800000000000001e-07 2024-05-08T08:52:23.061 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.061 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.061 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:23.061 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117329+0000", 2024-05-08T08:52:23.061 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3150000000000002e-06 2024-05-08T08:52:23.062 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.062 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.062 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:23.062 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117336+0000", 2024-05-08T08:52:23.062 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3459999999999999e-06 2024-05-08T08:52:23.062 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.062 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.062 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:23.062 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117342+0000", 2024-05-08T08:52:23.062 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8529999999999997e-06 2024-05-08T08:52:23.062 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.062 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.062 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:23.063 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.120613+0000", 2024-05-08T08:52:23.063 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0032715370000000001 2024-05-08T08:52:23.063 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.063 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.063 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:23.063 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.120657+0000", 2024-05-08T08:52:23.063 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4292000000000002e-05 2024-05-08T08:52:23.063 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.063 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.063 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-05-08T08:52:23.063 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.120725+0000", 2024-05-08T08:52:23.063 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7880999999999996e-05 2024-05-08T08:52:23.063 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.063 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.064 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:52:23.064 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.122844+0000", 2024-05-08T08:52:23.064 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0021191840000000001 2024-05-08T08:52:23.064 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.064 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.064 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:23.064 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.125814+0000", 2024-05-08T08:52:23.064 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0029697529999999999 2024-05-08T08:52:23.064 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.064 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.064 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:23.064 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.125839+0000", 2024-05-08T08:52:23.064 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5221000000000001e-05 2024-05-08T08:52:23.064 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.065 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.065 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:23.065 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.125860+0000", 2024-05-08T08:52:23.065 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0996999999999999e-05 2024-05-08T08:52:23.065 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.065 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:23.065 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.065 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.065 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.065 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4189.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-05-08T08:52:23.065 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:51:59.117351+0000", 2024-05-08T08:52:23.065 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.931565465999999, 2024-05-08T08:52:23.065 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0085922610000000003, 2024-05-08T08:52:23.065 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:23.066 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:23.066 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:23.066 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:23.066 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4189", 2024-05-08T08:52:23.066 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/574763665", 2024-05-08T08:52:23.066 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 111 2024-05-08T08:52:23.066 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.066 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:23.066 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.066 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:23.066 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117351+0000", 2024-05-08T08:52:23.066 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.066 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.067 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.067 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:23.067 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117351+0000", 2024-05-08T08:52:23.067 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.067 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.067 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.067 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:23.067 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117351+0000", 2024-05-08T08:52:23.067 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5300000000000004e-07 2024-05-08T08:52:23.067 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.067 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.067 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:23.067 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117355+0000", 2024-05-08T08:52:23.067 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9979999999999997e-06 2024-05-08T08:52:23.068 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.068 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.068 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:23.068 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117363+0000", 2024-05-08T08:52:23.068 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7889999999999995e-06 2024-05-08T08:52:23.068 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.068 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.068 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:23.068 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117371+0000", 2024-05-08T08:52:23.068 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2849999999999997e-06 2024-05-08T08:52:23.068 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.068 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.068 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:23.068 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.121156+0000", 2024-05-08T08:52:23.068 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0037849720000000002 2024-05-08T08:52:23.069 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.069 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.069 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:23.069 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.121197+0000", 2024-05-08T08:52:23.069 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1743000000000003e-05 2024-05-08T08:52:23.069 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.069 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.069 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-05-08T08:52:23.069 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.121267+0000", 2024-05-08T08:52:23.069 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9754999999999999e-05 2024-05-08T08:52:23.069 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.069 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.069 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:52:23.069 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.122851+0000", 2024-05-08T08:52:23.070 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015838689999999999 2024-05-08T08:52:23.070 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.070 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.070 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:23.070 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.125892+0000", 2024-05-08T08:52:23.070 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0030415569999999999 2024-05-08T08:52:23.070 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.070 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.070 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:23.070 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.125917+0000", 2024-05-08T08:52:23.070 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4828e-05 2024-05-08T08:52:23.070 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.070 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.071 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:23.071 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.125943+0000", 2024-05-08T08:52:23.071 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5911999999999999e-05 2024-05-08T08:52:23.071 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.071 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:23.071 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.071 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.071 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.071 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4189.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-05-08T08:52:23.071 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:51:59.117383+0000", 2024-05-08T08:52:23.071 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.931533512000001, 2024-05-08T08:52:23.071 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0086381340000000004, 2024-05-08T08:52:23.071 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:23.071 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:23.071 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:23.071 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:23.072 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4189", 2024-05-08T08:52:23.072 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/574763665", 2024-05-08T08:52:23.072 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 112 2024-05-08T08:52:23.072 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.072 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:23.072 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.072 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:23.072 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117383+0000", 2024-05-08T08:52:23.072 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.072 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.072 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.072 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:23.072 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117383+0000", 2024-05-08T08:52:23.072 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.072 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.073 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.073 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:23.073 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117383+0000", 2024-05-08T08:52:23.073 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6499999999999999e-07 2024-05-08T08:52:23.073 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.073 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.073 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:23.073 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117438+0000", 2024-05-08T08:52:23.073 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4775000000000003e-05 2024-05-08T08:52:23.073 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.073 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.073 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:23.073 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117446+0000", 2024-05-08T08:52:23.073 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5950000000000001e-06 2024-05-08T08:52:23.073 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.074 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.074 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:23.074 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117452+0000", 2024-05-08T08:52:23.074 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2299999999999996e-06 2024-05-08T08:52:23.074 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.074 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.074 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:23.074 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.121500+0000", 2024-05-08T08:52:23.074 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0040479069999999999 2024-05-08T08:52:23.074 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.074 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.074 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:23.074 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.121547+0000", 2024-05-08T08:52:23.075 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7274000000000002e-05 2024-05-08T08:52:23.075 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.075 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.075 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-05-08T08:52:23.075 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.121618+0000", 2024-05-08T08:52:23.075 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0733000000000002e-05 2024-05-08T08:52:23.075 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.075 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.075 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:52:23.075 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.123852+0000", 2024-05-08T08:52:23.075 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0022343649999999999 2024-05-08T08:52:23.075 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.075 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.075 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:23.075 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.125975+0000", 2024-05-08T08:52:23.076 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0021230609999999999 2024-05-08T08:52:23.076 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.076 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.076 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:23.076 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.126000+0000", 2024-05-08T08:52:23.076 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4875000000000002e-05 2024-05-08T08:52:23.076 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.076 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.076 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:23.076 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.126021+0000", 2024-05-08T08:52:23.076 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0854000000000001e-05 2024-05-08T08:52:23.076 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.076 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:23.076 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.076 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.077 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.077 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4189.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-05-08T08:52:23.077 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:51:59.117464+0000", 2024-05-08T08:52:23.077 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.931452883999999, 2024-05-08T08:52:23.077 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0086338920000000006, 2024-05-08T08:52:23.077 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:23.077 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:23.077 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:23.077 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:23.077 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4189", 2024-05-08T08:52:23.077 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/574763665", 2024-05-08T08:52:23.077 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 113 2024-05-08T08:52:23.077 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.077 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:23.077 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.078 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:23.078 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117464+0000", 2024-05-08T08:52:23.078 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.078 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.078 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.078 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:23.078 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117464+0000", 2024-05-08T08:52:23.078 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.078 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.078 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.078 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:23.078 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117464+0000", 2024-05-08T08:52:23.078 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2e-07 2024-05-08T08:52:23.078 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.079 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.079 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:23.079 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117468+0000", 2024-05-08T08:52:23.079 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2610000000000004e-06 2024-05-08T08:52:23.079 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.079 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.079 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:23.079 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117475+0000", 2024-05-08T08:52:23.079 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3400000000000003e-06 2024-05-08T08:52:23.079 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.079 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.079 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:23.079 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117480+0000", 2024-05-08T08:52:23.079 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8740000000000003e-06 2024-05-08T08:52:23.080 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.080 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.080 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:23.080 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.122167+0000", 2024-05-08T08:52:23.080 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0046871889999999996 2024-05-08T08:52:23.080 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.080 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.080 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:23.080 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.122212+0000", 2024-05-08T08:52:23.080 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5488000000000002e-05 2024-05-08T08:52:23.080 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.080 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.080 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-05-08T08:52:23.080 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.122278+0000", 2024-05-08T08:52:23.081 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5690000000000003e-05 2024-05-08T08:52:23.081 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.081 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.081 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:52:23.081 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.123858+0000", 2024-05-08T08:52:23.081 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015798100000000001 2024-05-08T08:52:23.081 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.081 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.081 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:23.081 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.126052+0000", 2024-05-08T08:52:23.081 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002194714 2024-05-08T08:52:23.081 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.081 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.081 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:23.081 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.126078+0000", 2024-05-08T08:52:23.082 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5669000000000001e-05 2024-05-08T08:52:23.082 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.082 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.082 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:23.082 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.126097+0000", 2024-05-08T08:52:23.082 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9337000000000001e-05 2024-05-08T08:52:23.082 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.082 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:23.082 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.082 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.082 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.082 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4189.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-05-08T08:52:23.082 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:51:59.117493+0000", 2024-05-08T08:52:23.082 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.931423569, 2024-05-08T08:52:23.083 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0086786899999999993, 2024-05-08T08:52:23.083 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:23.083 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:23.083 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:23.083 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:23.083 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4189", 2024-05-08T08:52:23.083 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/574763665", 2024-05-08T08:52:23.083 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 114 2024-05-08T08:52:23.083 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.083 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:23.083 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.083 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:23.083 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117493+0000", 2024-05-08T08:52:23.083 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.083 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.084 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.084 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:23.084 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117493+0000", 2024-05-08T08:52:23.084 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.084 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.084 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.084 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:23.084 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117493+0000", 2024-05-08T08:52:23.084 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.06e-07 2024-05-08T08:52:23.084 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.084 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.084 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:23.084 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117498+0000", 2024-05-08T08:52:23.085 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2599999999999999e-06 2024-05-08T08:52:23.085 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.085 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.085 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:23.085 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117505+0000", 2024-05-08T08:52:23.085 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9870000000000002e-06 2024-05-08T08:52:23.085 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.085 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.085 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:23.085 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117510+0000", 2024-05-08T08:52:23.085 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9830000000000004e-06 2024-05-08T08:52:23.085 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.085 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.085 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:23.085 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.122513+0000", 2024-05-08T08:52:23.085 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0050029499999999999 2024-05-08T08:52:23.086 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.086 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.086 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:23.086 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.122555+0000", 2024-05-08T08:52:23.086 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2715e-05 2024-05-08T08:52:23.086 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.086 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.086 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-05-08T08:52:23.086 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.122623+0000", 2024-05-08T08:52:23.086 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7261999999999994e-05 2024-05-08T08:52:23.086 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.086 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.086 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:52:23.087 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.123865+0000", 2024-05-08T08:52:23.087 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0012428960000000001 2024-05-08T08:52:23.087 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.087 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.087 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:23.087 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.126126+0000", 2024-05-08T08:52:23.087 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0022607389999999999 2024-05-08T08:52:23.087 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.087 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.087 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:23.087 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.126151+0000", 2024-05-08T08:52:23.087 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5258000000000001e-05 2024-05-08T08:52:23.087 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.087 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.088 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:23.088 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.126172+0000", 2024-05-08T08:52:23.088 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0134000000000001e-05 2024-05-08T08:52:23.088 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.088 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:23.088 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.088 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.088 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.088 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4189.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-05-08T08:52:23.088 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:51:59.117577+0000", 2024-05-08T08:52:23.088 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.931339406999999, 2024-05-08T08:52:23.088 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010674029, 2024-05-08T08:52:23.088 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:23.088 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:23.088 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:23.089 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:23.089 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4189", 2024-05-08T08:52:23.089 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/574763665", 2024-05-08T08:52:23.089 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 115 2024-05-08T08:52:23.089 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.089 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:23.089 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.089 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:23.089 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117577+0000", 2024-05-08T08:52:23.089 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.089 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.089 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.089 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:23.089 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117577+0000", 2024-05-08T08:52:23.090 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.090 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.090 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.090 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:23.090 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117578+0000", 2024-05-08T08:52:23.090 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0299999999999999e-07 2024-05-08T08:52:23.090 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.090 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.090 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:23.090 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117622+0000", 2024-05-08T08:52:23.090 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4935999999999998e-05 2024-05-08T08:52:23.090 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.090 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.090 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:23.091 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117630+0000", 2024-05-08T08:52:23.091 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6899999999999992e-06 2024-05-08T08:52:23.091 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.091 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.091 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:23.091 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117636+0000", 2024-05-08T08:52:23.091 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3119999999999999e-06 2024-05-08T08:52:23.091 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.091 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.091 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:23.091 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.122880+0000", 2024-05-08T08:52:23.091 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0052432930000000004 2024-05-08T08:52:23.091 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.092 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.092 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:23.092 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.122923+0000", 2024-05-08T08:52:23.092 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3022999999999998e-05 2024-05-08T08:52:23.092 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.092 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.092 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-05-08T08:52:23.092 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.122997+0000", 2024-05-08T08:52:23.092 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4574999999999997e-05 2024-05-08T08:52:23.092 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.092 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.092 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:52:23.092 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.124183+0000", 2024-05-08T08:52:23.092 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011852099999999999 2024-05-08T08:52:23.093 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.093 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.093 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:23.093 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.128204+0000", 2024-05-08T08:52:23.093 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0040213649999999998 2024-05-08T08:52:23.093 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.093 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.093 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:23.093 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.128230+0000", 2024-05-08T08:52:23.093 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6210000000000001e-05 2024-05-08T08:52:23.093 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.093 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.093 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:23.094 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.128251+0000", 2024-05-08T08:52:23.094 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0911999999999999e-05 2024-05-08T08:52:23.094 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.094 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:23.094 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.094 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.094 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.094 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4189.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-05-08T08:52:23.094 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:51:59.117662+0000", 2024-05-08T08:52:23.094 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.931254900999999, 2024-05-08T08:52:23.094 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010668166999999999, 2024-05-08T08:52:23.094 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:23.094 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:23.095 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:23.095 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:23.095 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4189", 2024-05-08T08:52:23.095 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/574763665", 2024-05-08T08:52:23.095 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 116 2024-05-08T08:52:23.095 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.095 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:23.095 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.095 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:23.095 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117662+0000", 2024-05-08T08:52:23.095 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.095 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.095 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.095 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:23.095 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117662+0000", 2024-05-08T08:52:23.096 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.096 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.096 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.096 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:23.096 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117662+0000", 2024-05-08T08:52:23.096 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.82e-07 2024-05-08T08:52:23.096 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.096 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.096 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:23.096 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117668+0000", 2024-05-08T08:52:23.096 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4099999999999999e-06 2024-05-08T08:52:23.096 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.096 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.096 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:23.097 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117675+0000", 2024-05-08T08:52:23.097 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7230000000000004e-06 2024-05-08T08:52:23.097 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.097 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.097 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:23.097 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117680+0000", 2024-05-08T08:52:23.097 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8029999999999996e-06 2024-05-08T08:52:23.097 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.097 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.097 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:23.097 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.123525+0000", 2024-05-08T08:52:23.097 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0058446749999999997 2024-05-08T08:52:23.097 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.097 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.097 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:23.098 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.123569+0000", 2024-05-08T08:52:23.098 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3853999999999997e-05 2024-05-08T08:52:23.098 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.098 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.098 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-05-08T08:52:23.098 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.123638+0000", 2024-05-08T08:52:23.098 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9666999999999996e-05 2024-05-08T08:52:23.098 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.098 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.098 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:52:23.098 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.124242+0000", 2024-05-08T08:52:23.098 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00060348599999999997 2024-05-08T08:52:23.098 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.098 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.099 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:23.099 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.128282+0000", 2024-05-08T08:52:23.099 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0040407739999999996 2024-05-08T08:52:23.099 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.099 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.099 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:23.099 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.128309+0000", 2024-05-08T08:52:23.099 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6519000000000001e-05 2024-05-08T08:52:23.099 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.099 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.099 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:23.099 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.128330+0000", 2024-05-08T08:52:23.099 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0673999999999999e-05 2024-05-08T08:52:23.099 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.100 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:23.100 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.100 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.100 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.100 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4189.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-05-08T08:52:23.100 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:51:59.117691+0000", 2024-05-08T08:52:23.100 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.931225471000001, 2024-05-08T08:52:23.100 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010707118999999999, 2024-05-08T08:52:23.100 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:23.100 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:23.100 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:23.100 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:23.100 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4189", 2024-05-08T08:52:23.100 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/574763665", 2024-05-08T08:52:23.100 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 117 2024-05-08T08:52:23.101 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.101 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:23.101 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.101 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:23.101 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117691+0000", 2024-05-08T08:52:23.101 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.101 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.101 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.101 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:23.101 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117691+0000", 2024-05-08T08:52:23.101 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.101 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.101 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.102 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:23.102 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117691+0000", 2024-05-08T08:52:23.102 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4900000000000001e-07 2024-05-08T08:52:23.102 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.102 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.102 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:23.102 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117696+0000", 2024-05-08T08:52:23.102 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0509999999999996e-06 2024-05-08T08:52:23.102 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.102 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.102 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:23.102 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117703+0000", 2024-05-08T08:52:23.102 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1229999999999997e-06 2024-05-08T08:52:23.102 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.103 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.103 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:23.103 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.117708+0000", 2024-05-08T08:52:23.103 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5310000000000001e-06 2024-05-08T08:52:23.103 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.103 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.103 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:23.103 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.123886+0000", 2024-05-08T08:52:23.103 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0061775750000000003 2024-05-08T08:52:23.103 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.103 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.103 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:23.103 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.123937+0000", 2024-05-08T08:52:23.103 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1669999999999998e-05 2024-05-08T08:52:23.103 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.104 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.104 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-05-08T08:52:23.104 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.124009+0000", 2024-05-08T08:52:23.104 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1815000000000003e-05 2024-05-08T08:52:23.104 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.104 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.104 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:52:23.104 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.124474+0000", 2024-05-08T08:52:23.104 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00046435299999999998 2024-05-08T08:52:23.104 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.104 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.104 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:23.104 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.128354+0000", 2024-05-08T08:52:23.104 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003880175 2024-05-08T08:52:23.104 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.105 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.105 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:23.105 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.128379+0000", 2024-05-08T08:52:23.105 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5143999999999998e-05 2024-05-08T08:52:23.105 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.105 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.105 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:23.105 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.128398+0000", 2024-05-08T08:52:23.105 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9233e-05 2024-05-08T08:52:23.105 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.105 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:23.105 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.105 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.105 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.105 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4189.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e13)", 2024-05-08T08:52:23.106 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:51:59.444340+0000", 2024-05-08T08:52:23.106 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.604576435999999, 2024-05-08T08:52:23.106 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0085447240000000001, 2024-05-08T08:52:23.106 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:23.106 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:23.106 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:23.106 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:23.106 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4189", 2024-05-08T08:52:23.106 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/574763665", 2024-05-08T08:52:23.106 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 139 2024-05-08T08:52:23.106 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.106 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:23.106 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.106 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:23.107 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.444340+0000", 2024-05-08T08:52:23.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.107 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:23.107 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.444340+0000", 2024-05-08T08:52:23.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.107 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:23.107 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.444340+0000", 2024-05-08T08:52:23.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2300000000000001e-07 2024-05-08T08:52:23.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.107 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:23.108 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.444345+0000", 2024-05-08T08:52:23.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7770000000000002e-06 2024-05-08T08:52:23.108 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.108 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.108 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:23.108 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.444353+0000", 2024-05-08T08:52:23.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1610000000000003e-06 2024-05-08T08:52:23.108 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.108 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.108 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:23.108 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.444359+0000", 2024-05-08T08:52:23.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2789999999999998e-06 2024-05-08T08:52:23.108 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.108 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.108 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:23.108 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.450067+0000", 2024-05-08T08:52:23.109 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.005707869 2024-05-08T08:52:23.109 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.109 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.109 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:23.109 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.450141+0000", 2024-05-08T08:52:23.109 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4035000000000005e-05 2024-05-08T08:52:23.109 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.109 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.109 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-05-08T08:52:23.109 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.450261+0000", 2024-05-08T08:52:23.109 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000120219 2024-05-08T08:52:23.109 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.109 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.109 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:52:23.109 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.452245+0000", 2024-05-08T08:52:23.110 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0019839329999999998 2024-05-08T08:52:23.110 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.110 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.110 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:23.110 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.452800+0000", 2024-05-08T08:52:23.110 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00055537999999999996 2024-05-08T08:52:23.110 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.110 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.110 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:23.110 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.452849+0000", 2024-05-08T08:52:23.110 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8922999999999999e-05 2024-05-08T08:52:23.110 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.110 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.110 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:23.111 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.452885+0000", 2024-05-08T08:52:23.111 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5624999999999999e-05 2024-05-08T08:52:23.111 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.111 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:23.111 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.111 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.111 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.111 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4189.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e13)", 2024-05-08T08:52:23.111 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:51:59.444369+0000", 2024-05-08T08:52:23.111 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.604547140000001, 2024-05-08T08:52:23.111 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0086469149999999998, 2024-05-08T08:52:23.111 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:23.111 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:23.111 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:23.111 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:23.112 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4189", 2024-05-08T08:52:23.112 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/574763665", 2024-05-08T08:52:23.112 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 140 2024-05-08T08:52:23.112 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.112 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:23.112 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.112 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:23.112 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.444369+0000", 2024-05-08T08:52:23.112 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.112 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.112 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.112 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:23.112 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.444369+0000", 2024-05-08T08:52:23.112 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.113 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.113 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.113 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:23.113 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.444370+0000", 2024-05-08T08:52:23.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6600000000000002e-07 2024-05-08T08:52:23.113 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.113 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.113 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:23.113 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.444433+0000", 2024-05-08T08:52:23.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3433000000000001e-05 2024-05-08T08:52:23.113 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.113 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.113 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:23.113 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.444440+0000", 2024-05-08T08:52:23.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.985e-06 2024-05-08T08:52:23.114 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.114 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.114 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:23.114 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.444444+0000", 2024-05-08T08:52:23.114 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8670000000000004e-06 2024-05-08T08:52:23.114 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.114 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.114 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:23.114 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.450579+0000", 2024-05-08T08:52:23.114 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0061353969999999999 2024-05-08T08:52:23.114 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.114 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.115 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:23.115 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.450629+0000", 2024-05-08T08:52:23.115 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0062000000000003e-05 2024-05-08T08:52:23.115 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.115 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.115 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-05-08T08:52:23.115 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.450708+0000", 2024-05-08T08:52:23.115 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8576000000000001e-05 2024-05-08T08:52:23.115 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.115 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.115 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:52:23.115 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.452440+0000", 2024-05-08T08:52:23.115 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0017315620000000001 2024-05-08T08:52:23.115 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.116 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.116 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:23.116 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.452930+0000", 2024-05-08T08:52:23.116 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00049034199999999995 2024-05-08T08:52:23.116 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.116 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.116 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:23.116 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.452980+0000", 2024-05-08T08:52:23.116 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0436999999999998e-05 2024-05-08T08:52:23.116 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.116 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.116 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:23.116 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.453016+0000", 2024-05-08T08:52:23.116 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5787999999999997e-05 2024-05-08T08:52:23.116 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.117 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:23.117 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.117 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.117 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.117 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4189.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e13)", 2024-05-08T08:52:23.117 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:51:59.444452+0000", 2024-05-08T08:52:23.117 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.604464419999999, 2024-05-08T08:52:23.117 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0088408830000000008, 2024-05-08T08:52:23.117 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:23.117 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:23.117 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:23.117 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:23.117 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4189", 2024-05-08T08:52:23.118 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/574763665", 2024-05-08T08:52:23.118 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 141 2024-05-08T08:52:23.118 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.118 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:23.118 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.118 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:23.118 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.444452+0000", 2024-05-08T08:52:23.118 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.118 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.118 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.118 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:23.118 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.444452+0000", 2024-05-08T08:52:23.118 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.118 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.119 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.119 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:23.119 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.444452+0000", 2024-05-08T08:52:23.119 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.39e-07 2024-05-08T08:52:23.119 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.119 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.119 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:23.119 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.444455+0000", 2024-05-08T08:52:23.119 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4439999999999998e-06 2024-05-08T08:52:23.119 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.119 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.119 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:23.119 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.444458+0000", 2024-05-08T08:52:23.119 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5769999999999998e-06 2024-05-08T08:52:23.120 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.120 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.120 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:23.120 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.444461+0000", 2024-05-08T08:52:23.120 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6309999999999999e-06 2024-05-08T08:52:23.120 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.120 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.120 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:23.120 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.451765+0000", 2024-05-08T08:52:23.120 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007303749 2024-05-08T08:52:23.120 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.120 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.120 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:23.120 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.451841+0000", 2024-05-08T08:52:23.120 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6202e-05 2024-05-08T08:52:23.121 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.121 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.121 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-05-08T08:52:23.121 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.451965+0000", 2024-05-08T08:52:23.121 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012429699999999999 2024-05-08T08:52:23.121 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.121 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.121 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:23.121 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.453203+0000", 2024-05-08T08:52:23.121 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001237995 2024-05-08T08:52:23.121 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.121 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.121 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:52:23.122 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.453214+0000", 2024-05-08T08:52:23.122 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0841e-05 2024-05-08T08:52:23.122 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.122 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.122 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:23.122 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.453256+0000", 2024-05-08T08:52:23.122 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1597000000000002e-05 2024-05-08T08:52:23.122 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.122 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.122 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:23.122 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:51:59.453293+0000", 2024-05-08T08:52:23.122 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7110999999999998e-05 2024-05-08T08:52:23.122 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.122 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:23.122 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.123 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.123 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.123 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4465.0:2338 3.13 3:c9516f98:::rbd_data.1140cf79ec85.0000000000000036:head [write 745472~61440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-05-08T08:52:23.123 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:17.300089+0000", 2024-05-08T08:52:23.123 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.7488271040000001, 2024-05-08T08:52:23.123 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0093887469999999994, 2024-05-08T08:52:23.123 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:23.123 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:23.123 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:23.123 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:23.123 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4465", 2024-05-08T08:52:23.123 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/390275688", 2024-05-08T08:52:23.123 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2338 2024-05-08T08:52:23.123 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.124 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:23.124 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.124 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:23.124 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.300089+0000", 2024-05-08T08:52:23.124 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.124 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.124 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.124 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:23.124 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.300089+0000", 2024-05-08T08:52:23.124 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.124 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.124 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.124 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:23.124 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.300090+0000", 2024-05-08T08:52:23.125 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0900000000000001e-07 2024-05-08T08:52:23.125 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.125 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.125 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:23.125 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.300115+0000", 2024-05-08T08:52:23.125 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4896999999999999e-05 2024-05-08T08:52:23.125 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.125 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.125 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:23.125 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.300130+0000", 2024-05-08T08:52:23.125 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5129e-05 2024-05-08T08:52:23.125 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.125 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.125 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:23.126 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.300135+0000", 2024-05-08T08:52:23.126 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5040000000000002e-06 2024-05-08T08:52:23.126 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.126 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.126 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:23.126 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.304762+0000", 2024-05-08T08:52:23.126 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0046264599999999998 2024-05-08T08:52:23.126 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.126 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.126 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:23.126 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.304805+0000", 2024-05-08T08:52:23.126 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3016000000000003e-05 2024-05-08T08:52:23.126 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.127 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.127 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 5", 2024-05-08T08:52:23.127 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.304878+0000", 2024-05-08T08:52:23.127 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2622999999999999e-05 2024-05-08T08:52:23.127 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.127 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.127 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:52:23.127 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.309109+0000", 2024-05-08T08:52:23.127 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0042317930000000002 2024-05-08T08:52:23.127 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.127 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.127 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:23.127 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.309410+0000", 2024-05-08T08:52:23.127 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00030075600000000001 2024-05-08T08:52:23.128 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.128 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.128 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:23.128 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.309442+0000", 2024-05-08T08:52:23.128 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2112e-05 2024-05-08T08:52:23.128 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.128 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.128 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:23.128 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.309478+0000", 2024-05-08T08:52:23.128 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5848e-05 2024-05-08T08:52:23.128 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.128 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:23.128 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.128 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.129 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.129 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4465.0:2340 3.13 3:ce9cf5f5:::rbd_data.1140cf79ec85.0000000000000025:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3502080~28672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-05-08T08:52:23.129 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:17.300148+0000", 2024-05-08T08:52:23.129 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.7487680660000002, 2024-05-08T08:52:23.129 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013302347000000001, 2024-05-08T08:52:23.129 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:23.129 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:23.129 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:23.129 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:23.129 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4465", 2024-05-08T08:52:23.129 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/390275688", 2024-05-08T08:52:23.129 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2340 2024-05-08T08:52:23.129 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.129 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:23.129 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.130 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:23.130 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.300148+0000", 2024-05-08T08:52:23.130 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.130 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.130 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.130 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:23.130 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.300148+0000", 2024-05-08T08:52:23.130 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.130 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.130 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.130 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:23.130 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.300149+0000", 2024-05-08T08:52:23.130 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9599999999999999e-07 2024-05-08T08:52:23.130 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.131 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.131 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:23.131 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.300158+0000", 2024-05-08T08:52:23.131 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9020000000000005e-06 2024-05-08T08:52:23.131 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.131 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.131 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:23.131 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.300163+0000", 2024-05-08T08:52:23.131 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2789999999999998e-06 2024-05-08T08:52:23.131 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.131 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.131 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:23.131 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.300166+0000", 2024-05-08T08:52:23.131 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0910000000000001e-06 2024-05-08T08:52:23.131 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.132 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.132 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:23.132 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.307699+0000", 2024-05-08T08:52:23.132 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0075324520000000002 2024-05-08T08:52:23.132 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.132 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.132 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:23.132 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.307738+0000", 2024-05-08T08:52:23.132 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9551999999999997e-05 2024-05-08T08:52:23.132 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.132 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.132 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 5", 2024-05-08T08:52:23.132 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.307811+0000", 2024-05-08T08:52:23.132 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3207999999999994e-05 2024-05-08T08:52:23.132 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.133 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.133 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:52:23.133 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.309299+0000", 2024-05-08T08:52:23.133 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001487878 2024-05-08T08:52:23.133 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.133 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.133 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:23.133 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.313376+0000", 2024-05-08T08:52:23.133 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0040769810000000004 2024-05-08T08:52:23.133 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.133 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.133 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:23.133 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.313417+0000", 2024-05-08T08:52:23.133 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0386999999999998e-05 2024-05-08T08:52:23.133 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.134 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.134 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:23.134 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:17.313451+0000", 2024-05-08T08:52:23.134 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4121000000000001e-05 2024-05-08T08:52:23.134 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.134 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:23.134 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.134 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.134 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.134 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4492.0:365 3.13 3:ca666c48:::rbd_data.114052026309.0000000000000013:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-05-08T08:52:23.134 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:19.037321+0000", 2024-05-08T08:52:23.134 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.011595335, 2024-05-08T08:52:23.134 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0094752250000000003, 2024-05-08T08:52:23.134 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:23.134 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:23.135 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:23.135 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:23.135 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4492", 2024-05-08T08:52:23.135 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/1076427944", 2024-05-08T08:52:23.135 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 365 2024-05-08T08:52:23.135 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.135 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:23.135 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.135 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:23.135 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.037321+0000", 2024-05-08T08:52:23.135 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.135 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.135 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.135 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:23.135 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.037321+0000", 2024-05-08T08:52:23.136 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.136 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.136 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.136 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:23.136 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.037322+0000", 2024-05-08T08:52:23.136 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9400000000000004e-07 2024-05-08T08:52:23.136 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.136 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.136 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:23.136 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.037431+0000", 2024-05-08T08:52:23.136 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010942600000000001 2024-05-08T08:52:23.136 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.136 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.136 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:23.137 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.037445+0000", 2024-05-08T08:52:23.137 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3254e-05 2024-05-08T08:52:23.137 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.137 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.137 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:23.137 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.037452+0000", 2024-05-08T08:52:23.137 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8460000000000008e-06 2024-05-08T08:52:23.137 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.137 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.137 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:23.137 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.037509+0000", 2024-05-08T08:52:23.137 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6239000000000001e-05 2024-05-08T08:52:23.137 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.137 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.138 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:23.138 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.037592+0000", 2024-05-08T08:52:23.138 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3590000000000004e-05 2024-05-08T08:52:23.138 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.138 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.138 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 5", 2024-05-08T08:52:23.138 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.037801+0000", 2024-05-08T08:52:23.138 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020864399999999999 2024-05-08T08:52:23.138 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.138 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.138 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:52:23.138 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.043730+0000", 2024-05-08T08:52:23.138 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0059288580000000004 2024-05-08T08:52:23.138 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.139 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.139 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:23.139 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.046700+0000", 2024-05-08T08:52:23.139 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0029698390000000002 2024-05-08T08:52:23.139 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.139 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.139 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:23.139 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.046757+0000", 2024-05-08T08:52:23.139 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7311000000000001e-05 2024-05-08T08:52:23.139 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.139 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.139 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:23.139 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.046796+0000", 2024-05-08T08:52:23.139 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9424000000000001e-05 2024-05-08T08:52:23.139 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.139 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:23.140 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.140 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.140 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.140 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4492.0:366 3.13 3:ca666c48:::rbd_data.114052026309.0000000000000013:head [delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-05-08T08:52:23.140 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:19.037469+0000", 2024-05-08T08:52:23.140 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.0114478279999997, 2024-05-08T08:52:23.140 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010074655, 2024-05-08T08:52:23.140 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:23.140 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:23.140 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:23.140 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:23.140 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4492", 2024-05-08T08:52:23.140 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/1076427944", 2024-05-08T08:52:23.140 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 366 2024-05-08T08:52:23.141 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.141 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:23.141 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.141 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:23.141 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.037469+0000", 2024-05-08T08:52:23.141 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.141 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.141 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.141 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:23.141 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.037469+0000", 2024-05-08T08:52:23.141 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.141 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.141 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.141 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:23.142 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.037469+0000", 2024-05-08T08:52:23.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9699999999999995e-07 2024-05-08T08:52:23.142 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.142 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.142 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:23.142 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.037472+0000", 2024-05-08T08:52:23.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6979999999999998e-06 2024-05-08T08:52:23.142 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.142 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.142 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:23.142 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.037480+0000", 2024-05-08T08:52:23.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6350000000000006e-06 2024-05-08T08:52:23.142 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.142 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.143 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:23.143 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.037485+0000", 2024-05-08T08:52:23.143 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.694e-06 2024-05-08T08:52:23.143 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.143 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.143 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:23.143 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.041483+0000", 2024-05-08T08:52:23.143 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0039972469999999998 2024-05-08T08:52:23.143 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.143 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.143 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:23.143 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.041556+0000", 2024-05-08T08:52:23.143 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3820999999999997e-05 2024-05-08T08:52:23.143 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.144 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.144 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 5", 2024-05-08T08:52:23.144 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.041785+0000", 2024-05-08T08:52:23.144 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00022845400000000001 2024-05-08T08:52:23.144 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.144 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.144 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:52:23.144 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.046863+0000", 2024-05-08T08:52:23.144 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.005077827 2024-05-08T08:52:23.144 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.144 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.144 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:23.144 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.047458+0000", 2024-05-08T08:52:23.144 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00059532699999999999 2024-05-08T08:52:23.144 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.145 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.145 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:23.145 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.047505+0000", 2024-05-08T08:52:23.145 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7438999999999998e-05 2024-05-08T08:52:23.145 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.145 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.145 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:23.145 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.047543+0000", 2024-05-08T08:52:23.145 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7815999999999998e-05 2024-05-08T08:52:23.145 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.145 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:23.145 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.145 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.145 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.146 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4492.0:519 3.11 3:8cb8f602:::rbd_data.114052026309.0000000000000053:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-05-08T08:52:23.146 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:19.054388+0000", 2024-05-08T08:52:23.146 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.9945284029999999, 2024-05-08T08:52:23.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0094360179999999991, 2024-05-08T08:52:23.146 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:23.146 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:23.146 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:23.146 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:23.146 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4492", 2024-05-08T08:52:23.146 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/1076427944", 2024-05-08T08:52:23.146 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 519 2024-05-08T08:52:23.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.146 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:23.147 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.147 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:23.147 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.054388+0000", 2024-05-08T08:52:23.147 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.147 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.147 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.147 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:23.147 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.054388+0000", 2024-05-08T08:52:23.147 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.147 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.147 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.147 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:23.147 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.054389+0000", 2024-05-08T08:52:23.147 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8899999999999998e-07 2024-05-08T08:52:23.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.148 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.148 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:23.148 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.054393+0000", 2024-05-08T08:52:23.148 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5859999999999998e-06 2024-05-08T08:52:23.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.148 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.148 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:23.148 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.054403+0000", 2024-05-08T08:52:23.148 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3700000000000001e-06 2024-05-08T08:52:23.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.148 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.148 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:23.148 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.054408+0000", 2024-05-08T08:52:23.148 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0189999999999997e-06 2024-05-08T08:52:23.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.149 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:23.149 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.054454+0000", 2024-05-08T08:52:23.149 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5955999999999998e-05 2024-05-08T08:52:23.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.149 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:23.149 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.054528+0000", 2024-05-08T08:52:23.149 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3899999999999994e-05 2024-05-08T08:52:23.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.149 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 4", 2024-05-08T08:52:23.149 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.054704+0000", 2024-05-08T08:52:23.150 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00017592700000000001 2024-05-08T08:52:23.150 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.150 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.150 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:52:23.150 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.056144+0000", 2024-05-08T08:52:23.150 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001440074 2024-05-08T08:52:23.150 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.150 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.150 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:23.150 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.063725+0000", 2024-05-08T08:52:23.150 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0075815250000000004 2024-05-08T08:52:23.150 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.150 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.150 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:23.151 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.063782+0000", 2024-05-08T08:52:23.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6709000000000002e-05 2024-05-08T08:52:23.151 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.151 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.151 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:23.151 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.063824+0000", 2024-05-08T08:52:23.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2063e-05 2024-05-08T08:52:23.151 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.151 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:23.151 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.151 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.151 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.151 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4492.0:521 3.15 3:aec0efd2:::rbd_data.114052026309.000000000000005d:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-05-08T08:52:23.151 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:19.054422+0000", 2024-05-08T08:52:23.152 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.9944941410000001, 2024-05-08T08:52:23.152 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.011801782, 2024-05-08T08:52:23.152 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:23.152 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:23.152 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:23.152 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:23.152 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4492", 2024-05-08T08:52:23.152 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/1076427944", 2024-05-08T08:52:23.152 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 521 2024-05-08T08:52:23.152 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.152 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:23.152 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.152 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:23.152 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.054422+0000", 2024-05-08T08:52:23.152 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.153 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.153 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.153 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:23.153 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.054422+0000", 2024-05-08T08:52:23.153 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.153 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.153 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.153 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:23.153 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.054423+0000", 2024-05-08T08:52:23.153 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7100000000000002e-07 2024-05-08T08:52:23.153 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.153 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.153 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:23.153 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.054425+0000", 2024-05-08T08:52:23.154 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3479999999999998e-06 2024-05-08T08:52:23.154 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.154 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.154 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:23.154 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.054503+0000", 2024-05-08T08:52:23.154 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8308999999999999e-05 2024-05-08T08:52:23.154 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.154 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.154 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:23.154 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.054510+0000", 2024-05-08T08:52:23.154 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9909999999999996e-06 2024-05-08T08:52:23.154 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.154 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.155 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:23.155 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.054560+0000", 2024-05-08T08:52:23.155 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0049000000000001e-05 2024-05-08T08:52:23.155 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.155 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.155 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:23.155 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.054670+0000", 2024-05-08T08:52:23.155 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000109508 2024-05-08T08:52:23.155 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.155 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.155 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:23.155 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.066224+0000", 2024-05-08T08:52:23.155 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.011554106 2024-05-08T08:52:23.155 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.155 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:23.156 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.156 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.156 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.156 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4492.0:560 3.15 3:ac531b94:::rbd_data.114052026309.000000000000006a:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-05-08T08:52:23.156 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:19.056869+0000", 2024-05-08T08:52:23.156 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.992047704, 2024-05-08T08:52:23.156 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.012411800000000001, 2024-05-08T08:52:23.156 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:23.156 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:23.156 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:23.156 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:23.156 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4492", 2024-05-08T08:52:23.156 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/1076427944", 2024-05-08T08:52:23.156 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 560 2024-05-08T08:52:23.157 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.157 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:23.157 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.157 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:23.157 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.056869+0000", 2024-05-08T08:52:23.157 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.157 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.157 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.157 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:23.157 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.056869+0000", 2024-05-08T08:52:23.157 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.157 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.157 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.157 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:23.157 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.056869+0000", 2024-05-08T08:52:23.158 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6700000000000003e-07 2024-05-08T08:52:23.158 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.158 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.158 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:23.158 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.056880+0000", 2024-05-08T08:52:23.158 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0699e-05 2024-05-08T08:52:23.158 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.158 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.158 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:23.158 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.056891+0000", 2024-05-08T08:52:23.158 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0373e-05 2024-05-08T08:52:23.158 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.158 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.158 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:23.158 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.056897+0000", 2024-05-08T08:52:23.159 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8589999999999999e-06 2024-05-08T08:52:23.159 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.159 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.159 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:23.159 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.056999+0000", 2024-05-08T08:52:23.159 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000101464 2024-05-08T08:52:23.159 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.159 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.159 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:23.159 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.057097+0000", 2024-05-08T08:52:23.159 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8090000000000004e-05 2024-05-08T08:52:23.159 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.159 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.159 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:23.159 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.069281+0000", 2024-05-08T08:52:23.160 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.012183548000000001 2024-05-08T08:52:23.160 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.160 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:23.160 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.160 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.160 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:23.160 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:52:23.161 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:52:23.162 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:52:23.165+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:52:23.262 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:52:23.266+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:52:23.267 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:52:23.267 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:52:23.267 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.267 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4555.0:670 3.1d 3.9421bb3d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:52:23.267 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:22.971264+0000", 2024-05-08T08:52:23.267 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.299706578, 2024-05-08T08:52:23.267 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.29972874799999999, 2024-05-08T08:52:23.267 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:23.267 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:23.267 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "waiting for peered", 2024-05-08T08:52:23.267 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:23.267 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4555", 2024-05-08T08:52:23.267 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/1932461342", 2024-05-08T08:52:23.268 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 670 2024-05-08T08:52:23.268 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.268 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:23.268 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.268 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:23.268 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:22.971264+0000", 2024-05-08T08:52:23.268 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.268 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.268 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.268 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:23.268 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:22.971264+0000", 2024-05-08T08:52:23.268 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.268 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.268 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.269 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:23.269 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:22.971265+0000", 2024-05-08T08:52:23.269 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.28e-06 2024-05-08T08:52:23.269 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.269 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.269 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:23.269 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:22.971272+0000", 2024-05-08T08:52:23.269 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6449999999999999e-06 2024-05-08T08:52:23.269 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.269 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.269 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:23.269 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:22.971284+0000", 2024-05-08T08:52:23.269 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2167000000000001e-05 2024-05-08T08:52:23.269 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.269 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.270 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:23.270 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:22.971292+0000", 2024-05-08T08:52:23.270 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5780000000000001e-06 2024-05-08T08:52:23.270 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.270 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.270 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:23.270 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:22.971351+0000", 2024-05-08T08:52:23.270 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8687999999999998e-05 2024-05-08T08:52:23.270 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.270 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.270 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-05-08T08:52:23.270 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:22.971364+0000", 2024-05-08T08:52:23.270 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3909e-05 2024-05-08T08:52:23.270 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.271 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:23.271 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.271 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.271 INFO:teuthology.orchestra.run.smithi192.stdout: ], 2024-05-08T08:52:23.271 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 1 2024-05-08T08:52:23.271 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:52:23.275 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:52:23.363 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:23.367+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:23.380 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:52:23.381 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:52:23.381 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:52:23.381 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:52:23.381 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:52:23.389 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:52:23.474 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:23.477+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:23.505 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T08:52:23.537 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:52:23.538 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:52:23.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:52:23.538 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:52:23.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.538 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4416.0:9 3.1a 3:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-05-08T08:52:23.538 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:13.935120+0000", 2024-05-08T08:52:23.538 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.6047887410000001, 2024-05-08T08:52:23.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.008168804, 2024-05-08T08:52:23.538 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:23.538 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:23.538 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:23.538 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:23.539 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4416", 2024-05-08T08:52:23.539 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:52:23.539 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 9 2024-05-08T08:52:23.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.539 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:23.539 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.539 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:23.539 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:13.935120+0000", 2024-05-08T08:52:23.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.539 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.539 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:23.539 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:13.935120+0000", 2024-05-08T08:52:23.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.540 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.540 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:23.540 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:13.935121+0000", 2024-05-08T08:52:23.540 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.015e-06 2024-05-08T08:52:23.540 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.540 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.540 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:23.540 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:13.935125+0000", 2024-05-08T08:52:23.540 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9559999999999996e-06 2024-05-08T08:52:23.540 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.540 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.540 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:23.540 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:13.935135+0000", 2024-05-08T08:52:23.540 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5089999999999999e-06 2024-05-08T08:52:23.540 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.540 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.541 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:23.541 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:13.935143+0000", 2024-05-08T08:52:23.541 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4800000000000001e-06 2024-05-08T08:52:23.541 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.541 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.541 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:23.541 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:13.935184+0000", 2024-05-08T08:52:23.541 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0308000000000001e-05 2024-05-08T08:52:23.541 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.541 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.541 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op must wait for map", 2024-05-08T08:52:23.541 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:13.935188+0000", 2024-05-08T08:52:23.541 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3499999999999999e-06 2024-05-08T08:52:23.541 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.542 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.542 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:23.542 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:13.938333+0000", 2024-05-08T08:52:23.542 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031447799999999998 2024-05-08T08:52:23.542 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.542 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.542 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:23.542 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:13.938396+0000", 2024-05-08T08:52:23.542 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2738999999999997e-05 2024-05-08T08:52:23.542 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.542 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.542 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-05-08T08:52:23.542 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:13.938480+0000", 2024-05-08T08:52:23.542 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4808999999999994e-05 2024-05-08T08:52:23.542 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.543 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:23.543 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:13.939565+0000", 2024-05-08T08:52:23.543 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0010842740000000001 2024-05-08T08:52:23.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.543 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:52:23.543 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:13.943198+0000", 2024-05-08T08:52:23.543 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0036330500000000001 2024-05-08T08:52:23.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.543 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:23.543 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:13.943247+0000", 2024-05-08T08:52:23.543 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9224999999999997e-05 2024-05-08T08:52:23.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.544 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:23.544 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:13.943288+0000", 2024-05-08T08:52:23.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1309e-05 2024-05-08T08:52:23.544 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.544 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:23.544 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.544 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.544 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.544 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4492.0:194 3.1a 3:5d070cd8:::rbd_data.114052026309.0000000000000021:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-05-08T08:52:23.544 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:19.024960+0000", 2024-05-08T08:52:23.544 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.5149480109999995, 2024-05-08T08:52:23.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010185179000000001, 2024-05-08T08:52:23.544 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:23.544 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:23.545 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:23.545 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:23.545 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4492", 2024-05-08T08:52:23.545 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/1076427944", 2024-05-08T08:52:23.545 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 194 2024-05-08T08:52:23.545 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.545 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:23.545 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.545 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:23.545 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.024960+0000", 2024-05-08T08:52:23.545 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.545 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.545 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.545 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:23.545 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.024960+0000", 2024-05-08T08:52:23.546 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.546 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.546 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.546 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:23.546 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.024961+0000", 2024-05-08T08:52:23.546 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8899999999999998e-07 2024-05-08T08:52:23.546 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.546 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.546 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:23.546 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.024966+0000", 2024-05-08T08:52:23.546 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9799999999999998e-06 2024-05-08T08:52:23.546 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.546 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.547 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:23.547 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.024978+0000", 2024-05-08T08:52:23.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.188e-05 2024-05-08T08:52:23.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.547 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:23.547 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.024990+0000", 2024-05-08T08:52:23.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1725999999999999e-05 2024-05-08T08:52:23.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.547 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:23.547 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.025049+0000", 2024-05-08T08:52:23.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8959000000000003e-05 2024-05-08T08:52:23.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:23.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.025521+0000", 2024-05-08T08:52:23.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00047250299999999999 2024-05-08T08:52:23.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:23.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.035145+0000", 2024-05-08T08:52:23.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0096241420000000005 2024-05-08T08:52:23.548 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.548 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:23.548 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.548 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4492.0:519 3.11 e24/18 3:8cb8f602:::rbd_data.114052026309.0000000000000053:head v 24'22, mlcod=24'18)", 2024-05-08T08:52:23.549 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:19.055325+0000", 2024-05-08T08:52:23.549 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.4845836160000001, 2024-05-08T08:52:23.549 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0082626039999999998, 2024-05-08T08:52:23.549 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:23.549 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:23.549 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:23.549 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:23.549 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.549 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:23.549 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.055325+0000", 2024-05-08T08:52:23.549 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.549 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.549 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.549 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:23.550 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.055325+0000", 2024-05-08T08:52:23.550 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.550 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.550 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.550 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:23.550 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.055325+0000", 2024-05-08T08:52:23.550 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8100000000000003e-07 2024-05-08T08:52:23.550 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.550 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.550 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:23.550 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.055331+0000", 2024-05-08T08:52:23.550 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3990000000000003e-06 2024-05-08T08:52:23.550 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.550 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.551 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:23.551 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.055461+0000", 2024-05-08T08:52:23.551 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000130629 2024-05-08T08:52:23.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.551 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:23.551 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.055468+0000", 2024-05-08T08:52:23.551 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3570000000000003e-06 2024-05-08T08:52:23.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.551 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:23.551 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.055915+0000", 2024-05-08T08:52:23.551 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000447584 2024-05-08T08:52:23.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.552 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:23.552 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.055939+0000", 2024-05-08T08:52:23.552 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3496999999999999e-05 2024-05-08T08:52:23.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.552 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.552 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:23.552 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.063542+0000", 2024-05-08T08:52:23.552 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0076030439999999998 2024-05-08T08:52:23.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.552 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.552 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:23.552 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.063587+0000", 2024-05-08T08:52:23.552 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5513000000000003e-05 2024-05-08T08:52:23.552 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.553 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:23.553 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.553 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.553 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.553 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4492.0:526 3.1 e24/18 3:81bf20c4:::rbd_data.114052026309.0000000000000056:head v 24'26, mlcod=24'24)", 2024-05-08T08:52:23.553 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:19.055758+0000", 2024-05-08T08:52:23.553 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.4841498590000004, 2024-05-08T08:52:23.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0078676519999999993, 2024-05-08T08:52:23.553 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:23.553 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:23.553 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:23.553 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:23.553 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.553 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:23.554 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.055758+0000", 2024-05-08T08:52:23.554 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.554 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.554 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.554 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:23.554 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.055758+0000", 2024-05-08T08:52:23.554 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.554 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.554 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.554 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:23.554 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.055759+0000", 2024-05-08T08:52:23.554 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3399999999999999e-07 2024-05-08T08:52:23.554 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.554 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.555 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:23.555 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.055764+0000", 2024-05-08T08:52:23.555 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6909999999999996e-06 2024-05-08T08:52:23.555 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.555 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.555 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:23.555 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.055855+0000", 2024-05-08T08:52:23.555 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1566999999999999e-05 2024-05-08T08:52:23.555 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.555 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.555 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:23.555 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.055868+0000", 2024-05-08T08:52:23.555 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3124e-05 2024-05-08T08:52:23.555 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.555 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.556 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:23.556 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.056115+0000", 2024-05-08T08:52:23.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00024636599999999998 2024-05-08T08:52:23.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.556 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.556 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:23.556 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.056133+0000", 2024-05-08T08:52:23.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7830999999999999e-05 2024-05-08T08:52:23.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.556 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.556 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:23.556 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.063590+0000", 2024-05-08T08:52:23.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0074579010000000003 2024-05-08T08:52:23.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.557 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.557 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:23.557 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.063626+0000", 2024-05-08T08:52:23.557 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5638e-05 2024-05-08T08:52:23.557 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.557 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:23.557 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.557 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.557 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.557 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4492.0:541 3.9 3:97ed463e:::rbd_data.114052026309.000000000000004d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-05-08T08:52:23.557 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:19.056058+0000", 2024-05-08T08:52:23.557 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.4838503850000002, 2024-05-08T08:52:23.557 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0076644210000000003, 2024-05-08T08:52:23.558 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:23.558 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:23.558 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:23.558 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:23.558 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4492", 2024-05-08T08:52:23.558 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/1076427944", 2024-05-08T08:52:23.558 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 541 2024-05-08T08:52:23.558 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.558 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:23.558 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.558 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:23.558 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.056058+0000", 2024-05-08T08:52:23.558 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.559 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.559 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.559 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:23.559 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.056058+0000", 2024-05-08T08:52:23.559 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.559 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.559 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.559 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:23.559 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.056059+0000", 2024-05-08T08:52:23.559 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6899999999999996e-07 2024-05-08T08:52:23.559 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.559 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.559 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:23.560 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.056063+0000", 2024-05-08T08:52:23.560 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7049999999999998e-06 2024-05-08T08:52:23.560 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.560 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.560 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:23.560 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.056072+0000", 2024-05-08T08:52:23.560 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.585e-06 2024-05-08T08:52:23.560 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.560 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.560 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:23.560 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.056078+0000", 2024-05-08T08:52:23.560 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4690000000000002e-06 2024-05-08T08:52:23.560 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.560 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.561 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:23.561 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.056159+0000", 2024-05-08T08:52:23.561 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0980000000000001e-05 2024-05-08T08:52:23.561 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.561 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.561 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:23.561 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.056231+0000", 2024-05-08T08:52:23.561 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1853999999999993e-05 2024-05-08T08:52:23.561 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.561 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.561 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 2", 2024-05-08T08:52:23.561 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.056421+0000", 2024-05-08T08:52:23.561 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00019012099999999999 2024-05-08T08:52:23.561 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.562 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.562 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:23.562 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.062767+0000", 2024-05-08T08:52:23.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063456140000000003 2024-05-08T08:52:23.562 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.562 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.562 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:52:23.562 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.063634+0000", 2024-05-08T08:52:23.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00086653100000000003 2024-05-08T08:52:23.562 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.562 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.562 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:23.562 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.063682+0000", 2024-05-08T08:52:23.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8254000000000003e-05 2024-05-08T08:52:23.563 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.563 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.563 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:23.563 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.063722+0000", 2024-05-08T08:52:23.563 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0538999999999999e-05 2024-05-08T08:52:23.563 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.563 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:23.563 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.563 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.563 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.563 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4492.0:542 3.9 3:970ccd0c:::rbd_data.114052026309.000000000000004f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-05-08T08:52:23.563 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:19.056198+0000", 2024-05-08T08:52:23.563 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.4837105309999998, 2024-05-08T08:52:23.563 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0095430549999999999, 2024-05-08T08:52:23.563 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:23.564 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:23.564 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:23.564 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:23.564 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4492", 2024-05-08T08:52:23.564 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/1076427944", 2024-05-08T08:52:23.564 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 542 2024-05-08T08:52:23.564 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.564 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:23.564 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.564 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:23.564 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.056198+0000", 2024-05-08T08:52:23.564 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.564 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.564 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.565 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:23.565 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.056198+0000", 2024-05-08T08:52:23.565 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.565 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.565 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.565 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:23.565 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.056198+0000", 2024-05-08T08:52:23.565 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1999999999999999e-07 2024-05-08T08:52:23.565 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.565 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.565 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:23.565 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.056203+0000", 2024-05-08T08:52:23.565 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2220000000000004e-06 2024-05-08T08:52:23.566 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.566 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.566 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:23.566 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.056211+0000", 2024-05-08T08:52:23.566 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2390000000000002e-06 2024-05-08T08:52:23.566 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.566 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.566 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:23.566 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.056219+0000", 2024-05-08T08:52:23.566 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0020000000000006e-06 2024-05-08T08:52:23.566 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.566 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.566 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:23.566 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.056678+0000", 2024-05-08T08:52:23.567 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00045879599999999998 2024-05-08T08:52:23.567 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.567 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.567 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:23.567 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.056746+0000", 2024-05-08T08:52:23.567 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8112000000000001e-05 2024-05-08T08:52:23.567 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.567 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.567 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 2", 2024-05-08T08:52:23.567 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.056957+0000", 2024-05-08T08:52:23.567 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000211208 2024-05-08T08:52:23.567 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.567 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.567 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:52:23.568 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.063731+0000", 2024-05-08T08:52:23.568 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0067737420000000001 2024-05-08T08:52:23.568 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.568 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.568 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:23.568 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.065679+0000", 2024-05-08T08:52:23.568 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001948466 2024-05-08T08:52:23.568 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.568 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.568 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:23.568 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.065711+0000", 2024-05-08T08:52:23.568 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1902000000000001e-05 2024-05-08T08:52:23.568 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.569 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.569 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:23.570 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.065741+0000", 2024-05-08T08:52:23.570 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9646e-05 2024-05-08T08:52:23.570 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.570 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:23.571 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.571 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.571 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4492.0:676 3.3 3:c73d9490:::rbd_data.114052026309.0000000000000077:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-05-08T08:52:23.571 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:19.069584+0000", 2024-05-08T08:52:23.571 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.4703245430000003, 2024-05-08T08:52:23.571 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.011947223, 2024-05-08T08:52:23.571 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:23.571 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:23.572 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:23.572 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:23.572 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4492", 2024-05-08T08:52:23.572 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/1076427944", 2024-05-08T08:52:23.572 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 676 2024-05-08T08:52:23.572 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.572 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:23.572 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.572 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:23.572 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.069584+0000", 2024-05-08T08:52:23.572 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.573 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.573 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.573 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:23.573 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.069584+0000", 2024-05-08T08:52:23.573 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.573 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.573 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.573 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:23.573 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.069585+0000", 2024-05-08T08:52:23.573 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5000000000000001e-07 2024-05-08T08:52:23.574 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.574 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.574 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:23.574 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.069590+0000", 2024-05-08T08:52:23.574 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9200000000000003e-06 2024-05-08T08:52:23.574 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.574 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.574 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:23.574 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.069601+0000", 2024-05-08T08:52:23.574 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1501000000000001e-05 2024-05-08T08:52:23.574 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.574 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.574 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:23.574 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.069609+0000", 2024-05-08T08:52:23.575 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9789999999999993e-06 2024-05-08T08:52:23.575 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.575 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.575 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:23.575 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.069665+0000", 2024-05-08T08:52:23.575 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5538000000000003e-05 2024-05-08T08:52:23.575 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.575 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.575 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:23.575 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.069732+0000", 2024-05-08T08:52:23.575 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7681000000000005e-05 2024-05-08T08:52:23.575 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.576 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.576 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-05-08T08:52:23.576 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.069914+0000", 2024-05-08T08:52:23.576 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00018190700000000001 2024-05-08T08:52:23.576 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.576 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.576 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:52:23.576 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.074829+0000", 2024-05-08T08:52:23.576 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0049143370000000004 2024-05-08T08:52:23.576 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.576 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.576 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:23.577 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.081431+0000", 2024-05-08T08:52:23.577 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0066020319999999999 2024-05-08T08:52:23.577 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.577 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.577 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:23.577 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.081486+0000", 2024-05-08T08:52:23.577 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5537e-05 2024-05-08T08:52:23.577 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.577 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.577 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:23.578 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.081531+0000", 2024-05-08T08:52:23.578 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4841e-05 2024-05-08T08:52:23.578 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.578 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:23.578 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.578 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.578 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.578 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4492.0:1054 3.9 3:92f7a35e:::rbd_data.114052026309.00000000000000fd:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-05-08T08:52:23.578 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:19.105488+0000", 2024-05-08T08:52:23.578 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.4344198419999996, 2024-05-08T08:52:23.578 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0083466559999999992, 2024-05-08T08:52:23.578 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:23.578 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:23.578 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:23.578 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:23.579 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4492", 2024-05-08T08:52:23.579 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/1076427944", 2024-05-08T08:52:23.579 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1054 2024-05-08T08:52:23.579 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.579 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:23.579 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.579 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:23.579 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.105488+0000", 2024-05-08T08:52:23.579 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.579 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.579 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.579 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:23.579 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.105488+0000", 2024-05-08T08:52:23.579 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.579 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.580 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.580 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:23.580 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.105489+0000", 2024-05-08T08:52:23.580 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4e-07 2024-05-08T08:52:23.580 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.580 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.580 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:23.580 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.105494+0000", 2024-05-08T08:52:23.580 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8110000000000002e-06 2024-05-08T08:52:23.580 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.580 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.580 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:23.580 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.105508+0000", 2024-05-08T08:52:23.580 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4069e-05 2024-05-08T08:52:23.581 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.581 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.581 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:23.581 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.105516+0000", 2024-05-08T08:52:23.581 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4059999999999999e-06 2024-05-08T08:52:23.581 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.581 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.581 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:23.581 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.105608+0000", 2024-05-08T08:52:23.581 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2289999999999999e-05 2024-05-08T08:52:23.581 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.581 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.581 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:23.581 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.105734+0000", 2024-05-08T08:52:23.581 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012655500000000001 2024-05-08T08:52:23.582 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.582 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.582 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:23.582 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.113835+0000", 2024-05-08T08:52:23.582 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0081005850000000004 2024-05-08T08:52:23.582 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.582 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:23.582 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.582 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.582 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.582 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4492.0:1058 3.1a 3:5a2b8914:::rbd_data.114052026309.0000000000000100:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-05-08T08:52:23.582 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:19.105536+0000", 2024-05-08T08:52:23.582 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.4343722190000001, 2024-05-08T08:52:23.583 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0083339859999999998, 2024-05-08T08:52:23.583 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:23.583 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:23.583 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:23.583 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:23.583 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4492", 2024-05-08T08:52:23.583 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/1076427944", 2024-05-08T08:52:23.583 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1058 2024-05-08T08:52:23.583 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.583 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:23.583 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.583 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:23.583 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.105536+0000", 2024-05-08T08:52:23.583 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.584 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.584 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.584 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:23.584 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.105536+0000", 2024-05-08T08:52:23.584 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.584 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.584 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.584 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:23.584 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.105537+0000", 2024-05-08T08:52:23.584 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5400000000000001e-07 2024-05-08T08:52:23.584 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.584 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.584 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:23.584 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.105540+0000", 2024-05-08T08:52:23.584 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9689999999999999e-06 2024-05-08T08:52:23.585 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.585 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.585 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:23.585 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.105547+0000", 2024-05-08T08:52:23.585 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7209999999999994e-06 2024-05-08T08:52:23.585 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.585 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.585 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:23.585 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.105555+0000", 2024-05-08T08:52:23.585 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5800000000000003e-06 2024-05-08T08:52:23.585 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.585 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.585 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:23.585 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.105631+0000", 2024-05-08T08:52:23.586 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6161000000000005e-05 2024-05-08T08:52:23.586 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.586 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.586 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:23.586 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.105775+0000", 2024-05-08T08:52:23.586 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014406799999999999 2024-05-08T08:52:23.586 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.586 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.586 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:23.586 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.113870+0000", 2024-05-08T08:52:23.586 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0080949330000000003 2024-05-08T08:52:23.586 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.586 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:23.586 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.587 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.587 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.587 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "pg_update_log_missing(3.a epoch 24/18 rep_tid 489 entries 24'217 (0'0) error 3:502d98fa:::rbd_data.114052026309.00000000000000fa:head by client.4492.0:1049 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 24'215)", 2024-05-08T08:52:23.587 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:19.105829+0000", 2024-05-08T08:52:23.587 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.4340795789999996, 2024-05-08T08:52:23.587 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0080571949999999996, 2024-05-08T08:52:23.587 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:23.587 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:23.587 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-05-08T08:52:23.587 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:23.587 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.587 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:23.587 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.105829+0000", 2024-05-08T08:52:23.587 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.587 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.588 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.588 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:23.588 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.105829+0000", 2024-05-08T08:52:23.588 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.588 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.588 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.588 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:23.588 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.105829+0000", 2024-05-08T08:52:23.588 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9800000000000003e-07 2024-05-08T08:52:23.588 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.588 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.588 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:23.588 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.105837+0000", 2024-05-08T08:52:23.588 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6680000000000001e-06 2024-05-08T08:52:23.589 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.589 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.589 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:23.589 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.105855+0000", 2024-05-08T08:52:23.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8074e-05 2024-05-08T08:52:23.589 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.589 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.589 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:23.589 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.105869+0000", 2024-05-08T08:52:23.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4453999999999999e-05 2024-05-08T08:52:23.589 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.589 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.589 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:23.589 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.106069+0000", 2024-05-08T08:52:23.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00019910899999999999 2024-05-08T08:52:23.590 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.590 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.590 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:23.590 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.113886+0000", 2024-05-08T08:52:23.590 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007817292 2024-05-08T08:52:23.590 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.590 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:23.590 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.590 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.590 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.590 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "pg_update_log_missing(3.1c epoch 24/18 rep_tid 198 entries 24'31 (0'0) error 3:398e8057:::rbd_data.114052026309.00000000000000f9:head by client.4492.0:1047 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 24'29)", 2024-05-08T08:52:23.590 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:19.106130+0000", 2024-05-08T08:52:23.590 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.4337782179999996, 2024-05-08T08:52:23.590 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0076616130000000003, 2024-05-08T08:52:23.591 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:23.591 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:23.591 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-05-08T08:52:23.591 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:23.591 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.591 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:23.591 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.106130+0000", 2024-05-08T08:52:23.591 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.591 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.591 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.591 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:23.591 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.106130+0000", 2024-05-08T08:52:23.591 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.591 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.591 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.591 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:23.592 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.106131+0000", 2024-05-08T08:52:23.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7400000000000003e-07 2024-05-08T08:52:23.592 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.592 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.592 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:23.592 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.106138+0000", 2024-05-08T08:52:23.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7090000000000002e-06 2024-05-08T08:52:23.592 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.592 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.592 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:23.592 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.106155+0000", 2024-05-08T08:52:23.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7122e-05 2024-05-08T08:52:23.592 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.592 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.593 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:23.593 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.106168+0000", 2024-05-08T08:52:23.593 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2544000000000001e-05 2024-05-08T08:52:23.593 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.593 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.593 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:23.593 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.106254+0000", 2024-05-08T08:52:23.593 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5902999999999994e-05 2024-05-08T08:52:23.593 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.593 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.594 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:23.594 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.113792+0000", 2024-05-08T08:52:23.594 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0075377609999999996 2024-05-08T08:52:23.594 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.594 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:23.594 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.594 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.594 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.594 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "pg_update_log_missing(3.1d epoch 24/18 rep_tid 199 entries 24'22 (0'0) error 3:baedc9ab:::rbd_data.114052026309.00000000000000fc:head by client.4492.0:1051 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 24'20)", 2024-05-08T08:52:23.594 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:19.106193+0000", 2024-05-08T08:52:23.594 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.4337151109999997, 2024-05-08T08:52:23.594 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0076040470000000001, 2024-05-08T08:52:23.594 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:23.595 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:23.595 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-05-08T08:52:23.595 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:23.595 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.595 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:23.595 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.106193+0000", 2024-05-08T08:52:23.595 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.595 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.595 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.595 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:23.595 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.106193+0000", 2024-05-08T08:52:23.595 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.595 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.595 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.595 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:23.596 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.106193+0000", 2024-05-08T08:52:23.596 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5199999999999998e-07 2024-05-08T08:52:23.596 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.596 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:23.596 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.106199+0000", 2024-05-08T08:52:23.596 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4840000000000003e-06 2024-05-08T08:52:23.596 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.596 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:23.596 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.106213+0000", 2024-05-08T08:52:23.596 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3718e-05 2024-05-08T08:52:23.596 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.597 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:23.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.106224+0000", 2024-05-08T08:52:23.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1207e-05 2024-05-08T08:52:23.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.597 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:23.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.106289+0000", 2024-05-08T08:52:23.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5539999999999999e-05 2024-05-08T08:52:23.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.597 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:23.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.113797+0000", 2024-05-08T08:52:23.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007507746 2024-05-08T08:52:23.597 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.597 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:23.598 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.598 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.598 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "pg_update_log_missing(3.a epoch 24/18 rep_tid 490 entries 24'218 (0'0) error 3:54689ce5:::rbd_data.114052026309.00000000000000fb:head by client.4492.0:1050 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 24'215)", 2024-05-08T08:52:23.598 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:19.106223+0000", 2024-05-08T08:52:23.598 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.4336851470000003, 2024-05-08T08:52:23.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0076744270000000002, 2024-05-08T08:52:23.598 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:23.598 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:23.598 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-05-08T08:52:23.598 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:23.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:23.598 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.106223+0000", 2024-05-08T08:52:23.599 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.599 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.599 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.599 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:23.599 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.106223+0000", 2024-05-08T08:52:23.599 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.599 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.599 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.599 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:23.599 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.106224+0000", 2024-05-08T08:52:23.599 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2699999999999999e-07 2024-05-08T08:52:23.599 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.599 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.599 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:23.600 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.106231+0000", 2024-05-08T08:52:23.600 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6539999999999999e-06 2024-05-08T08:52:23.600 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.600 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.600 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:23.600 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.106249+0000", 2024-05-08T08:52:23.600 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7479e-05 2024-05-08T08:52:23.600 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.600 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.600 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:23.600 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.106262+0000", 2024-05-08T08:52:23.600 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3227999999999999e-05 2024-05-08T08:52:23.600 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.600 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.600 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:23.601 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.106353+0000", 2024-05-08T08:52:23.601 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0946000000000006e-05 2024-05-08T08:52:23.601 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.601 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:23.601 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.113898+0000", 2024-05-08T08:52:23.601 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0075444930000000002 2024-05-08T08:52:23.601 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.601 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:23.601 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.601 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.601 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "pg_update_log_missing(3.1c epoch 24/18 rep_tid 200 entries 24'32 (0'0) error 3:3a4bb9ad:::rbd_data.114052026309.00000000000000ff:head by client.4492.0:1056 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 24'29)", 2024-05-08T08:52:23.601 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:19.106242+0000", 2024-05-08T08:52:23.601 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.4336661570000002, 2024-05-08T08:52:23.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0075633330000000002, 2024-05-08T08:52:23.602 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:23.602 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:23.602 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-05-08T08:52:23.602 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:23.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:23.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.106242+0000", 2024-05-08T08:52:23.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:23.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.106242+0000", 2024-05-08T08:52:23.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.603 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.603 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:23.603 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.106242+0000", 2024-05-08T08:52:23.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.39e-07 2024-05-08T08:52:23.603 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.603 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:23.603 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.106247+0000", 2024-05-08T08:52:23.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8189999999999999e-06 2024-05-08T08:52:23.603 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.603 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:23.603 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.106257+0000", 2024-05-08T08:52:23.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6299999999999993e-06 2024-05-08T08:52:23.604 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.604 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.604 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:23.604 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.106269+0000", 2024-05-08T08:52:23.604 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2119999999999999e-05 2024-05-08T08:52:23.604 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.604 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.604 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:23.604 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.106397+0000", 2024-05-08T08:52:23.604 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000127806 2024-05-08T08:52:23.604 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.604 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.604 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:23.604 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.113805+0000", 2024-05-08T08:52:23.604 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007408619 2024-05-08T08:52:23.605 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.605 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:23.605 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.605 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.605 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.605 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4492.0:1094 3.1b e24/18 3:dbcdc403:::rbd_data.114052026309.00000000000000f0:head v 24'28, mlcod=24'25)", 2024-05-08T08:52:23.605 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:19.109431+0000", 2024-05-08T08:52:23.605 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.4304775269999999, 2024-05-08T08:52:23.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0092267200000000008, 2024-05-08T08:52:23.605 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:23.605 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:23.605 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:23.605 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:23.605 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.605 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:23.606 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.109431+0000", 2024-05-08T08:52:23.606 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.606 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.606 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:23.606 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.109431+0000", 2024-05-08T08:52:23.606 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.606 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.606 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:23.606 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.109431+0000", 2024-05-08T08:52:23.606 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5899999999999996e-07 2024-05-08T08:52:23.606 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.607 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:23.607 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.109438+0000", 2024-05-08T08:52:23.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7700000000000004e-06 2024-05-08T08:52:23.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.607 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:23.607 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.109450+0000", 2024-05-08T08:52:23.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1491999999999999e-05 2024-05-08T08:52:23.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.607 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:23.607 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.109456+0000", 2024-05-08T08:52:23.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.55e-06 2024-05-08T08:52:23.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.608 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:23.608 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.109506+0000", 2024-05-08T08:52:23.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0108999999999997e-05 2024-05-08T08:52:23.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.608 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:23.608 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.109528+0000", 2024-05-08T08:52:23.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2079e-05 2024-05-08T08:52:23.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.608 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:23.608 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.118621+0000", 2024-05-08T08:52:23.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0090922550000000005 2024-05-08T08:52:23.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.609 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:23.609 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.118657+0000", 2024-05-08T08:52:23.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6906000000000002e-05 2024-05-08T08:52:23.609 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.609 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:23.609 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.609 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4492.0:1105 3.14 e24/18 3:2c3f186e:::rbd_data.114052026309.00000000000000ed:head v 24'20, mlcod=24'18)", 2024-05-08T08:52:23.609 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:19.110037+0000", 2024-05-08T08:52:23.609 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.4298712650000001, 2024-05-08T08:52:23.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0086111020000000007, 2024-05-08T08:52:23.609 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:23.609 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:23.610 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:23.610 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:23.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.610 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:23.610 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.110037+0000", 2024-05-08T08:52:23.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.610 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.610 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:23.610 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.110037+0000", 2024-05-08T08:52:23.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.610 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.611 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:23.611 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.110038+0000", 2024-05-08T08:52:23.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0900000000000001e-07 2024-05-08T08:52:23.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.611 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:23.611 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.110042+0000", 2024-05-08T08:52:23.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4649999999999996e-06 2024-05-08T08:52:23.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.611 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:23.611 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.110052+0000", 2024-05-08T08:52:23.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6059999999999992e-06 2024-05-08T08:52:23.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.612 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:23.612 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.110058+0000", 2024-05-08T08:52:23.612 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2280000000000003e-06 2024-05-08T08:52:23.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.612 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:23.612 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.110103+0000", 2024-05-08T08:52:23.612 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5084999999999999e-05 2024-05-08T08:52:23.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.612 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:23.612 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.110123+0000", 2024-05-08T08:52:23.612 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9646000000000001e-05 2024-05-08T08:52:23.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.613 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:23.613 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.118612+0000", 2024-05-08T08:52:23.613 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0084894140000000007 2024-05-08T08:52:23.613 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.613 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.613 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:23.613 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.118648+0000", 2024-05-08T08:52:23.613 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6049e-05 2024-05-08T08:52:23.613 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.613 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:23.613 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.613 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.613 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.613 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4492.0:1106 3.a e24/18 3:517bbb4d:::rbd_data.114052026309.00000000000000eb:head v 24'219, mlcod=24'215)", 2024-05-08T08:52:23.613 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:19.110248+0000", 2024-05-08T08:52:23.614 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.429660621, 2024-05-08T08:52:23.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0085531319999999997, 2024-05-08T08:52:23.614 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:23.614 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:23.614 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:23.614 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:23.614 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.614 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:23.614 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.110248+0000", 2024-05-08T08:52:23.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.614 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.614 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.614 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:23.614 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.110248+0000", 2024-05-08T08:52:23.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:23.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.110248+0000", 2024-05-08T08:52:23.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9800000000000003e-07 2024-05-08T08:52:23.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:23.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.110253+0000", 2024-05-08T08:52:23.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8010000000000003e-06 2024-05-08T08:52:23.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:23.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.110262+0000", 2024-05-08T08:52:23.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2590000000000006e-06 2024-05-08T08:52:23.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:23.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.110270+0000", 2024-05-08T08:52:23.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7239999999999992e-06 2024-05-08T08:52:23.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:23.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.110323+0000", 2024-05-08T08:52:23.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3433999999999997e-05 2024-05-08T08:52:23.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:23.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.110343+0000", 2024-05-08T08:52:23.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9232000000000002e-05 2024-05-08T08:52:23.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.617 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.617 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:23.617 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.118771+0000", 2024-05-08T08:52:23.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0084282699999999999 2024-05-08T08:52:23.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.617 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.617 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:23.617 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.118801+0000", 2024-05-08T08:52:23.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9813999999999999e-05 2024-05-08T08:52:23.617 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.617 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:23.617 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.618 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.618 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4492.0:1132 3.9 3:92f7a35e:::rbd_data.114052026309.00000000000000fd:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-05-08T08:52:23.618 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:19.113992+0000", 2024-05-08T08:52:23.618 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.4259162820000002, 2024-05-08T08:52:23.618 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0076760209999999999, 2024-05-08T08:52:23.618 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:23.618 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:23.618 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:23.618 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:23.618 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4492", 2024-05-08T08:52:23.618 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/1076427944", 2024-05-08T08:52:23.618 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1132 2024-05-08T08:52:23.618 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.618 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:23.618 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.619 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:23.619 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.113992+0000", 2024-05-08T08:52:23.619 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.619 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.619 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.619 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:23.619 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.113992+0000", 2024-05-08T08:52:23.619 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.619 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.619 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.619 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:23.619 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.113993+0000", 2024-05-08T08:52:23.619 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0750000000000001e-06 2024-05-08T08:52:23.619 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.619 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.620 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:23.620 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.113998+0000", 2024-05-08T08:52:23.620 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2900000000000002e-06 2024-05-08T08:52:23.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.620 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:23.620 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.114009+0000", 2024-05-08T08:52:23.620 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0818999999999999e-05 2024-05-08T08:52:23.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.620 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:23.620 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.114016+0000", 2024-05-08T08:52:23.620 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8120000000000002e-06 2024-05-08T08:52:23.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:23.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.114063+0000", 2024-05-08T08:52:23.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7448999999999999e-05 2024-05-08T08:52:23.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:23.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.114136+0000", 2024-05-08T08:52:23.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2826999999999999e-05 2024-05-08T08:52:23.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 2", 2024-05-08T08:52:23.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.114334+0000", 2024-05-08T08:52:23.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000197283 2024-05-08T08:52:23.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.622 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.622 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:23.622 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.116805+0000", 2024-05-08T08:52:23.622 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0024716 2024-05-08T08:52:23.622 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.622 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.622 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:52:23.622 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.121541+0000", 2024-05-08T08:52:23.622 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0047357620000000001 2024-05-08T08:52:23.622 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.622 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.622 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:23.622 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.121625+0000", 2024-05-08T08:52:23.622 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3807000000000006e-05 2024-05-08T08:52:23.623 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.623 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.623 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:23.623 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.121668+0000", 2024-05-08T08:52:23.623 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3297000000000002e-05 2024-05-08T08:52:23.623 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.623 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:23.623 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.623 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.623 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.623 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4492.0:1133 3.1a 3:5a2b8914:::rbd_data.114052026309.0000000000000100:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-05-08T08:52:23.623 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:19.114034+0000", 2024-05-08T08:52:23.623 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.4258747390000002, 2024-05-08T08:52:23.623 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0077546009999999999, 2024-05-08T08:52:23.624 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:23.624 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:23.624 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:23.624 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:23.624 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4492", 2024-05-08T08:52:23.624 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/1076427944", 2024-05-08T08:52:23.624 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1133 2024-05-08T08:52:23.624 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.624 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:23.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.624 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:23.624 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.114034+0000", 2024-05-08T08:52:23.624 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.624 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.625 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:23.625 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.114034+0000", 2024-05-08T08:52:23.625 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.625 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.625 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.625 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:23.625 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.114034+0000", 2024-05-08T08:52:23.625 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2e-07 2024-05-08T08:52:23.625 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.625 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.625 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:23.625 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.114037+0000", 2024-05-08T08:52:23.625 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1990000000000002e-06 2024-05-08T08:52:23.625 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.625 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.626 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:23.626 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.114045+0000", 2024-05-08T08:52:23.626 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.469e-06 2024-05-08T08:52:23.626 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.626 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.626 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:23.626 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.114051+0000", 2024-05-08T08:52:23.626 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7700000000000004e-06 2024-05-08T08:52:23.626 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.626 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.626 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:23.626 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.114094+0000", 2024-05-08T08:52:23.626 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2491999999999999e-05 2024-05-08T08:52:23.626 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.627 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.627 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:23.627 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.114157+0000", 2024-05-08T08:52:23.627 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3178999999999999e-05 2024-05-08T08:52:23.627 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.627 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.627 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-05-08T08:52:23.627 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.114358+0000", 2024-05-08T08:52:23.627 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020135199999999999 2024-05-08T08:52:23.627 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.627 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.627 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:23.627 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.116018+0000", 2024-05-08T08:52:23.627 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016593630000000001 2024-05-08T08:52:23.627 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.628 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:52:23.628 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.121569+0000", 2024-05-08T08:52:23.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0055506899999999996 2024-05-08T08:52:23.628 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.628 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:23.628 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.121748+0000", 2024-05-08T08:52:23.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00017919600000000001 2024-05-08T08:52:23.628 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.628 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:23.628 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.121788+0000", 2024-05-08T08:52:23.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0370999999999997e-05 2024-05-08T08:52:23.628 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.629 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:23.629 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.629 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4492.0:1135 3.1a 3:597af2f9:::rbd_data.114052026309.0000000000000111:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-05-08T08:52:23.629 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:19.114086+0000", 2024-05-08T08:52:23.629 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.4258219820000004, 2024-05-08T08:52:23.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0078299119999999996, 2024-05-08T08:52:23.629 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:23.629 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:23.629 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:23.629 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:23.629 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4492", 2024-05-08T08:52:23.629 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/1076427944", 2024-05-08T08:52:23.629 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1135 2024-05-08T08:52:23.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.630 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:23.630 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.630 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:23.630 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.114086+0000", 2024-05-08T08:52:23.630 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.630 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.630 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:23.630 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.114086+0000", 2024-05-08T08:52:23.630 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:23.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.630 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.630 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:23.631 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.114087+0000", 2024-05-08T08:52:23.631 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9299999999999997e-07 2024-05-08T08:52:23.631 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.631 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.631 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:23.631 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.114090+0000", 2024-05-08T08:52:23.631 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3790000000000002e-06 2024-05-08T08:52:23.631 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.631 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.631 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:23.631 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.114097+0000", 2024-05-08T08:52:23.631 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1380000000000001e-06 2024-05-08T08:52:23.631 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.631 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.631 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:23.632 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.114103+0000", 2024-05-08T08:52:23.632 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8549999999999998e-06 2024-05-08T08:52:23.632 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.632 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.632 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:23.632 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.114681+0000", 2024-05-08T08:52:23.632 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00057759300000000005 2024-05-08T08:52:23.632 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.632 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.632 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:23.632 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.114802+0000", 2024-05-08T08:52:23.632 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012094 2024-05-08T08:52:23.632 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:23.632 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:23.632 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:23.633 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.121916+0000", 2024-05-08T08:52:23.633 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0071143140000000001 2024-05-08T08:52:23.633 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.633 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:23.633 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.633 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:23.633 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:23.633 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:52:23.634 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:52:23.637+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:52:23.635 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:52:23.645 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:52:23.645 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:52:23.663 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:52:23.663 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:52:23.680 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:52:23.680 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:52:23.698 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:52:23.698 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:52:23.715 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:52:23.715 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:52:23.733 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:52:23.733 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:52:23.735 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:52:23.738+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:52:23.782 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:23.782 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:23.791 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:52:23.835 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:52:23.839+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:52:23.936 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:23.939+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:23.937 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:23.937 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:23.947 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:52:24.037 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:52:24.040+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:52:24.093 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:24.094 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:24.102 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:52:24.138 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:52:24.142+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:52:24.224 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:24.224 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:24.233 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:52:24.239 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:24.243+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:24.340 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:24.343+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:24.351 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:24.351 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:24.361 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:52:24.391 INFO:teuthology.orchestra.run.smithi196.stdout:checking clone #2, image image_client.0-clone2 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent3 2024-05-08T08:52:24.441 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:52:24.444+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:52:24.487 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:24.487 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:24.500 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:52:24.542 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:24.545+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:24.643 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:52:24.646+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:52:24.698 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:24.699 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:24.710 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:52:24.744 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:24.747+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:24.845 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:24.848+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:24.869 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:24.869 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:24.882 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:52:24.946 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:52:24.949+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:52:24.976 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-05-08T08:52:25.039 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:25.039 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:25.047 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:25.050+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:25.053 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:52:25.148 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:25.151+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:25.190 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:25.190 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:25.199 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:52:25.249 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:52:25.252+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:52:25.309 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:25.309 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:25.318 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:52:25.362 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:52:25.365+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:52:25.362 INFO:teuthology.orchestra.run.smithi196.stdout:871 read 0x198e000 thru 0x19920b0 (0x40b1 bytes) 2024-05-08T08:52:25.363 INFO:teuthology.orchestra.run.smithi196.stdout:873 read 0x943e000 thru 0x944ace6 (0xcce7 bytes) 2024-05-08T08:52:25.363 INFO:teuthology.orchestra.run.smithi196.stdout:878 read 0xa425400 thru 0xa430791 (0xb392 bytes) 2024-05-08T08:52:25.364 INFO:teuthology.orchestra.run.smithi196.stdout:879 punch from 0x2caa200 to 0x2cb0000, (0x5e00 bytes) 2024-05-08T08:52:25.442 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:25.442 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:25.444 INFO:teuthology.orchestra.run.smithi196.stdout:880 write 0x71f9600 thru 0x7207e04 (0xe805 bytes) 2024-05-08T08:52:25.444 INFO:teuthology.orchestra.run.smithi196.stdout:881 read 0x92b0a00 thru 0x92bf191 (0xe792 bytes) 2024-05-08T08:52:25.444 INFO:teuthology.orchestra.run.smithi196.stdout:882 read 0xb9f0400 thru 0xb9f04a8 (0xa9 bytes) 2024-05-08T08:52:25.444 INFO:teuthology.orchestra.run.smithi196.stdout:885 punch from 0x5d88600 to 0x5d97000, (0xea00 bytes) 2024-05-08T08:52:25.450 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:52:25.454+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:52:25.453 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:52:25.552 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:52:25.555+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:52:25.588 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:25.588 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:25.596 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:52:25.682 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:25.685+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:25.744 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T08:52:25.748 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:25.748 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:25.753 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:52:25.757+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:52:25.758 INFO:teuthology.orchestra.run.smithi196.stdout:886 punch from 0xb15b800 to 0xb163800, (0x8000 bytes) 2024-05-08T08:52:25.760 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:52:25.768 INFO:teuthology.orchestra.run.smithi196.stdout:890 write 0x1e0ea00 thru 0x1e120e4 (0x36e5 bytes) 2024-05-08T08:52:25.769 INFO:teuthology.orchestra.run.smithi196.stdout:893 read 0x2236e00 thru 0x22437b2 (0xc9b3 bytes) 2024-05-08T08:52:25.774 INFO:teuthology.orchestra.run.smithi196.stdout:894 punch from 0xa944e00 to 0xa94c800, (0x7a00 bytes) 2024-05-08T08:52:25.855 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:52:25.858+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:52:25.886 INFO:teuthology.orchestra.run.smithi196.stdout:897 read 0x445b600 thru 0x445e016 (0x2a17 bytes) 2024-05-08T08:52:25.887 INFO:teuthology.orchestra.run.smithi196.stdout:898 read 0x22f7000 thru 0x22f9d5d (0x2d5e bytes) 2024-05-08T08:52:25.889 INFO:teuthology.orchestra.run.smithi196.stdout:899 punch from 0x6613400 to 0x6622a00, (0xf600 bytes) 2024-05-08T08:52:25.924 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:52:25.924 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:52:25.953 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:25.953 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:25.956 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:25.959+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:25.959 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:52:25.959 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:52:25.965 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:52:25.977 INFO:teuthology.orchestra.run.smithi059.stderr:nodeep-scrub is set 2024-05-08T08:52:25.983 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:52:25.984 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:52:25.987 INFO:teuthology.orchestra.run.smithi196.stdout:900 trunc from 0xd6e8400 to 0xd705a00 2024-05-08T08:52:25.995 INFO:teuthology.orchestra.run.smithi196.stdout:901 clone 5 order 25 su 16777216 sc 4 2024-05-08T08:52:26.002 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:52:26.003 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:52:26.030 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:52:26.030 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:52:26.047 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:52:26.047 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:52:26.056 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:52:26.060+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:52:26.110 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:52:26.110 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:52:26.111 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:52:26.111 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:52:26.119 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:52:26.158 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:52:26.162+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:52:26.224 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:52:26.225 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:52:26.225 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:52:26.225 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:52:26.225 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:52:26.233 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:52:26.278 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:26.281+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:26.339 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:52:26.339 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:52:26.340 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:52:26.340 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:52:26.340 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.340 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4416.0:9 3.1a 3:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-05-08T08:52:26.340 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:13.935120+0000", 2024-05-08T08:52:26.340 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 12.406801923, 2024-05-08T08:52:26.340 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.008168804, 2024-05-08T08:52:26.340 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:26.340 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:26.340 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:26.340 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:26.340 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4416", 2024-05-08T08:52:26.340 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:52:26.340 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 9 2024-05-08T08:52:26.341 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.341 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:26.341 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.341 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:26.341 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:13.935120+0000", 2024-05-08T08:52:26.341 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:26.341 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.341 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.341 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:26.341 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:13.935120+0000", 2024-05-08T08:52:26.341 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:26.341 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.341 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.341 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:26.341 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:13.935121+0000", 2024-05-08T08:52:26.342 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.015e-06 2024-05-08T08:52:26.342 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.342 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.342 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:26.342 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:13.935125+0000", 2024-05-08T08:52:26.342 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9559999999999996e-06 2024-05-08T08:52:26.342 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.342 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.342 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:26.342 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:13.935135+0000", 2024-05-08T08:52:26.342 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5089999999999999e-06 2024-05-08T08:52:26.342 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.342 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.342 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:26.343 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:13.935143+0000", 2024-05-08T08:52:26.343 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4800000000000001e-06 2024-05-08T08:52:26.343 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.343 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.343 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:26.343 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:13.935184+0000", 2024-05-08T08:52:26.343 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0308000000000001e-05 2024-05-08T08:52:26.343 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.343 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.343 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op must wait for map", 2024-05-08T08:52:26.344 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:13.935188+0000", 2024-05-08T08:52:26.344 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3499999999999999e-06 2024-05-08T08:52:26.344 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.344 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.344 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:26.344 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:13.938333+0000", 2024-05-08T08:52:26.344 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031447799999999998 2024-05-08T08:52:26.345 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.345 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:26.345 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:13.938396+0000", 2024-05-08T08:52:26.345 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2738999999999997e-05 2024-05-08T08:52:26.345 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.345 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-05-08T08:52:26.345 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:13.938480+0000", 2024-05-08T08:52:26.345 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4808999999999994e-05 2024-05-08T08:52:26.345 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.346 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:26.346 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:13.939565+0000", 2024-05-08T08:52:26.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0010842740000000001 2024-05-08T08:52:26.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.346 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:52:26.346 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:13.943198+0000", 2024-05-08T08:52:26.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0036330500000000001 2024-05-08T08:52:26.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.346 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:26.346 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:13.943247+0000", 2024-05-08T08:52:26.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9224999999999997e-05 2024-05-08T08:52:26.347 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.347 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.347 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:26.347 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:13.943288+0000", 2024-05-08T08:52:26.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1309e-05 2024-05-08T08:52:26.347 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.347 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:26.347 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.347 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.347 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.347 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4492.0:194 3.1a 3:5d070cd8:::rbd_data.114052026309.0000000000000021:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-05-08T08:52:26.347 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:19.024960+0000", 2024-05-08T08:52:26.347 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.316961193, 2024-05-08T08:52:26.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010185179000000001, 2024-05-08T08:52:26.348 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:26.348 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:26.348 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:26.348 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:26.348 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4492", 2024-05-08T08:52:26.348 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/1076427944", 2024-05-08T08:52:26.348 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 194 2024-05-08T08:52:26.348 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.348 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:26.348 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.348 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:26.348 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.024960+0000", 2024-05-08T08:52:26.348 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:26.348 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.348 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.349 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:26.349 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.024960+0000", 2024-05-08T08:52:26.349 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:26.349 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.349 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.349 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:26.349 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.024961+0000", 2024-05-08T08:52:26.349 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8899999999999998e-07 2024-05-08T08:52:26.349 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.349 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.349 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:26.349 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.024966+0000", 2024-05-08T08:52:26.349 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9799999999999998e-06 2024-05-08T08:52:26.349 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.349 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.350 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:26.350 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.024978+0000", 2024-05-08T08:52:26.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.188e-05 2024-05-08T08:52:26.350 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.350 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.350 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:26.350 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.024990+0000", 2024-05-08T08:52:26.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1725999999999999e-05 2024-05-08T08:52:26.350 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.350 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.350 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:26.350 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.025049+0000", 2024-05-08T08:52:26.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8959000000000003e-05 2024-05-08T08:52:26.350 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.351 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.351 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:26.351 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.025521+0000", 2024-05-08T08:52:26.351 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00047250299999999999 2024-05-08T08:52:26.351 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.351 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.351 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:26.351 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.035145+0000", 2024-05-08T08:52:26.351 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0096241420000000005 2024-05-08T08:52:26.351 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.351 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:26.351 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.351 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.351 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.351 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4492.0:519 3.11 e24/18 3:8cb8f602:::rbd_data.114052026309.0000000000000053:head v 24'22, mlcod=24'18)", 2024-05-08T08:52:26.352 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:19.055325+0000", 2024-05-08T08:52:26.352 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.2865967979999997, 2024-05-08T08:52:26.352 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0082626039999999998, 2024-05-08T08:52:26.352 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:26.352 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:26.352 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:26.352 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:26.352 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.352 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:26.352 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.055325+0000", 2024-05-08T08:52:26.352 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:26.352 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.352 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.352 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:26.353 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.055325+0000", 2024-05-08T08:52:26.353 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:26.353 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.353 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.353 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:26.353 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.055325+0000", 2024-05-08T08:52:26.353 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8100000000000003e-07 2024-05-08T08:52:26.353 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.353 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.353 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:26.353 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.055331+0000", 2024-05-08T08:52:26.353 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3990000000000003e-06 2024-05-08T08:52:26.353 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.353 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.353 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:26.353 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.055461+0000", 2024-05-08T08:52:26.354 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000130629 2024-05-08T08:52:26.354 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.354 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.354 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:26.354 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.055468+0000", 2024-05-08T08:52:26.354 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3570000000000003e-06 2024-05-08T08:52:26.354 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.354 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.354 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:26.354 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.055915+0000", 2024-05-08T08:52:26.354 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000447584 2024-05-08T08:52:26.354 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.354 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.355 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:26.355 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.055939+0000", 2024-05-08T08:52:26.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3496999999999999e-05 2024-05-08T08:52:26.355 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.355 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.355 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:26.355 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.063542+0000", 2024-05-08T08:52:26.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0076030439999999998 2024-05-08T08:52:26.355 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.355 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.355 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:26.355 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.063587+0000", 2024-05-08T08:52:26.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5513000000000003e-05 2024-05-08T08:52:26.355 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.355 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:26.356 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.356 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.356 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.356 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4492.0:526 3.1 e24/18 3:81bf20c4:::rbd_data.114052026309.0000000000000056:head v 24'26, mlcod=24'24)", 2024-05-08T08:52:26.356 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:19.055758+0000", 2024-05-08T08:52:26.356 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.286163041, 2024-05-08T08:52:26.356 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0078676519999999993, 2024-05-08T08:52:26.356 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:26.356 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:26.356 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:26.356 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:26.356 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.356 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:26.356 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.055758+0000", 2024-05-08T08:52:26.357 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:26.357 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.357 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.357 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:26.357 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.055758+0000", 2024-05-08T08:52:26.357 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:26.357 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.357 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.357 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:26.357 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.055759+0000", 2024-05-08T08:52:26.357 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3399999999999999e-07 2024-05-08T08:52:26.357 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.357 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.357 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:26.358 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.055764+0000", 2024-05-08T08:52:26.358 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6909999999999996e-06 2024-05-08T08:52:26.358 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.358 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.358 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:26.358 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.055855+0000", 2024-05-08T08:52:26.358 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1566999999999999e-05 2024-05-08T08:52:26.358 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.358 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.358 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:26.358 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.055868+0000", 2024-05-08T08:52:26.358 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3124e-05 2024-05-08T08:52:26.358 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.358 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.358 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:26.359 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.056115+0000", 2024-05-08T08:52:26.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00024636599999999998 2024-05-08T08:52:26.359 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.359 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.359 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:26.359 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.056133+0000", 2024-05-08T08:52:26.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7830999999999999e-05 2024-05-08T08:52:26.359 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.359 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.359 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:26.359 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.063590+0000", 2024-05-08T08:52:26.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0074579010000000003 2024-05-08T08:52:26.359 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.359 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.359 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:26.360 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.063626+0000", 2024-05-08T08:52:26.360 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5638e-05 2024-05-08T08:52:26.360 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.360 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:26.360 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.360 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.360 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.360 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4492.0:541 3.9 3:97ed463e:::rbd_data.114052026309.000000000000004d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-05-08T08:52:26.360 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:19.056058+0000", 2024-05-08T08:52:26.360 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.2858635669999998, 2024-05-08T08:52:26.360 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0076644210000000003, 2024-05-08T08:52:26.360 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:26.360 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:26.360 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:26.360 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:26.360 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4492", 2024-05-08T08:52:26.360 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/1076427944", 2024-05-08T08:52:26.361 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 541 2024-05-08T08:52:26.361 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.361 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:26.361 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.361 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:26.361 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.056058+0000", 2024-05-08T08:52:26.361 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:26.361 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.361 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.361 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:26.361 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.056058+0000", 2024-05-08T08:52:26.361 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:26.361 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.361 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.361 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:26.361 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.056059+0000", 2024-05-08T08:52:26.362 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6899999999999996e-07 2024-05-08T08:52:26.362 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.362 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.362 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:26.362 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.056063+0000", 2024-05-08T08:52:26.362 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7049999999999998e-06 2024-05-08T08:52:26.362 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.362 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.362 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:26.362 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.056072+0000", 2024-05-08T08:52:26.362 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.585e-06 2024-05-08T08:52:26.362 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.362 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.362 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:26.362 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.056078+0000", 2024-05-08T08:52:26.362 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4690000000000002e-06 2024-05-08T08:52:26.363 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.363 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.363 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:26.363 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.056159+0000", 2024-05-08T08:52:26.363 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0980000000000001e-05 2024-05-08T08:52:26.363 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.363 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.363 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:26.363 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.056231+0000", 2024-05-08T08:52:26.363 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1853999999999993e-05 2024-05-08T08:52:26.363 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.363 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.363 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 2", 2024-05-08T08:52:26.363 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.056421+0000", 2024-05-08T08:52:26.363 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00019012099999999999 2024-05-08T08:52:26.363 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.364 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.364 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:26.364 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.062767+0000", 2024-05-08T08:52:26.364 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063456140000000003 2024-05-08T08:52:26.364 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.364 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.364 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:52:26.364 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.063634+0000", 2024-05-08T08:52:26.364 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00086653100000000003 2024-05-08T08:52:26.364 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.364 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.364 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:26.364 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.063682+0000", 2024-05-08T08:52:26.364 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8254000000000003e-05 2024-05-08T08:52:26.364 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.365 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.365 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:26.365 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.063722+0000", 2024-05-08T08:52:26.365 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0538999999999999e-05 2024-05-08T08:52:26.365 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.365 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:26.365 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.365 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.365 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.365 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4492.0:542 3.9 3:970ccd0c:::rbd_data.114052026309.000000000000004f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-05-08T08:52:26.365 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:19.056198+0000", 2024-05-08T08:52:26.365 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.2857237130000003, 2024-05-08T08:52:26.365 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0095430549999999999, 2024-05-08T08:52:26.365 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:26.365 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:26.365 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:26.366 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:26.366 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4492", 2024-05-08T08:52:26.366 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/1076427944", 2024-05-08T08:52:26.366 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 542 2024-05-08T08:52:26.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.366 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:26.366 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.366 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:26.366 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.056198+0000", 2024-05-08T08:52:26.366 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:26.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.366 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.366 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:26.366 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.056198+0000", 2024-05-08T08:52:26.366 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:26.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.367 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.367 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:26.367 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.056198+0000", 2024-05-08T08:52:26.367 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1999999999999999e-07 2024-05-08T08:52:26.367 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.367 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.367 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:26.367 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.056203+0000", 2024-05-08T08:52:26.367 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2220000000000004e-06 2024-05-08T08:52:26.367 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.367 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.367 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:26.368 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.056211+0000", 2024-05-08T08:52:26.368 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2390000000000002e-06 2024-05-08T08:52:26.368 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.368 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.368 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:26.368 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.056219+0000", 2024-05-08T08:52:26.368 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0020000000000006e-06 2024-05-08T08:52:26.368 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.368 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.368 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:26.368 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.056678+0000", 2024-05-08T08:52:26.368 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00045879599999999998 2024-05-08T08:52:26.368 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.368 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.369 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:26.369 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.056746+0000", 2024-05-08T08:52:26.369 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8112000000000001e-05 2024-05-08T08:52:26.369 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.369 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.369 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 2", 2024-05-08T08:52:26.369 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.056957+0000", 2024-05-08T08:52:26.369 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000211208 2024-05-08T08:52:26.369 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.369 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.369 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:52:26.369 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.063731+0000", 2024-05-08T08:52:26.369 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0067737420000000001 2024-05-08T08:52:26.369 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.369 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.370 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:26.370 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.065679+0000", 2024-05-08T08:52:26.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001948466 2024-05-08T08:52:26.370 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.370 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.370 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:26.370 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.065711+0000", 2024-05-08T08:52:26.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1902000000000001e-05 2024-05-08T08:52:26.370 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.370 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.370 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:26.370 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.065741+0000", 2024-05-08T08:52:26.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9646e-05 2024-05-08T08:52:26.370 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.371 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:26.371 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.371 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.371 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.371 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4492.0:676 3.3 3:c73d9490:::rbd_data.114052026309.0000000000000077:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-05-08T08:52:26.371 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:19.069584+0000", 2024-05-08T08:52:26.371 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.2723377249999999, 2024-05-08T08:52:26.371 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.011947223, 2024-05-08T08:52:26.371 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:26.371 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:26.371 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:26.371 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:26.371 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4492", 2024-05-08T08:52:26.371 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/1076427944", 2024-05-08T08:52:26.371 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 676 2024-05-08T08:52:26.372 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.372 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:26.372 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.372 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:26.372 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.069584+0000", 2024-05-08T08:52:26.372 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:26.372 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.372 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.372 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:26.372 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.069584+0000", 2024-05-08T08:52:26.372 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:26.372 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.372 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.372 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:26.372 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.069585+0000", 2024-05-08T08:52:26.373 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5000000000000001e-07 2024-05-08T08:52:26.373 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.373 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.373 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:26.373 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.069590+0000", 2024-05-08T08:52:26.373 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9200000000000003e-06 2024-05-08T08:52:26.373 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.373 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.373 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:26.373 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.069601+0000", 2024-05-08T08:52:26.373 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1501000000000001e-05 2024-05-08T08:52:26.373 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.373 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.373 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:26.373 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.069609+0000", 2024-05-08T08:52:26.374 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9789999999999993e-06 2024-05-08T08:52:26.374 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.374 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.374 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:26.374 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.069665+0000", 2024-05-08T08:52:26.374 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5538000000000003e-05 2024-05-08T08:52:26.374 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.374 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.374 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:26.374 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.069732+0000", 2024-05-08T08:52:26.374 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7681000000000005e-05 2024-05-08T08:52:26.374 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.374 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.374 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-05-08T08:52:26.375 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.069914+0000", 2024-05-08T08:52:26.375 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00018190700000000001 2024-05-08T08:52:26.375 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.375 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.375 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:52:26.375 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.074829+0000", 2024-05-08T08:52:26.375 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0049143370000000004 2024-05-08T08:52:26.375 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.375 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.375 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:26.375 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.081431+0000", 2024-05-08T08:52:26.375 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0066020319999999999 2024-05-08T08:52:26.375 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.376 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.376 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:26.376 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.081486+0000", 2024-05-08T08:52:26.376 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5537e-05 2024-05-08T08:52:26.376 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.376 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.376 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:26.376 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.081531+0000", 2024-05-08T08:52:26.376 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4841e-05 2024-05-08T08:52:26.376 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.376 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:26.376 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.376 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.376 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.376 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4492.0:1054 3.9 3:92f7a35e:::rbd_data.114052026309.00000000000000fd:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-05-08T08:52:26.377 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:19.105488+0000", 2024-05-08T08:52:26.377 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.2364330240000001, 2024-05-08T08:52:26.377 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0083466559999999992, 2024-05-08T08:52:26.377 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:26.377 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:26.377 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:26.377 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:26.377 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4492", 2024-05-08T08:52:26.377 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/1076427944", 2024-05-08T08:52:26.377 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1054 2024-05-08T08:52:26.377 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.378 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:26.378 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.378 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:26.378 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.105488+0000", 2024-05-08T08:52:26.378 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:26.378 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.378 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.378 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:26.378 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.105488+0000", 2024-05-08T08:52:26.378 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:26.378 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.378 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.378 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:26.378 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.105489+0000", 2024-05-08T08:52:26.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4e-07 2024-05-08T08:52:26.379 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.379 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.379 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:26.379 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.105494+0000", 2024-05-08T08:52:26.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8110000000000002e-06 2024-05-08T08:52:26.379 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.379 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.379 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:26.379 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.105508+0000", 2024-05-08T08:52:26.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4069e-05 2024-05-08T08:52:26.379 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.379 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.379 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:26.380 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.105516+0000", 2024-05-08T08:52:26.380 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4059999999999999e-06 2024-05-08T08:52:26.380 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.380 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.380 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:26.380 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.105608+0000", 2024-05-08T08:52:26.380 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2289999999999999e-05 2024-05-08T08:52:26.380 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.380 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.380 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:26.380 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.105734+0000", 2024-05-08T08:52:26.380 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012655500000000001 2024-05-08T08:52:26.380 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.380 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.380 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:26.381 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.113835+0000", 2024-05-08T08:52:26.381 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0081005850000000004 2024-05-08T08:52:26.381 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.381 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:26.381 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.381 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.381 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.381 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4492.0:1058 3.1a 3:5a2b8914:::rbd_data.114052026309.0000000000000100:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-05-08T08:52:26.381 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:19.105536+0000", 2024-05-08T08:52:26.381 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.2363854009999997, 2024-05-08T08:52:26.381 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0083339859999999998, 2024-05-08T08:52:26.381 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:26.381 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:26.381 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:26.382 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:26.382 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4492", 2024-05-08T08:52:26.382 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/1076427944", 2024-05-08T08:52:26.382 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1058 2024-05-08T08:52:26.382 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.382 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:26.382 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.382 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:26.382 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.105536+0000", 2024-05-08T08:52:26.382 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:26.382 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.382 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.382 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:26.382 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.105536+0000", 2024-05-08T08:52:26.383 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:26.383 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.383 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.383 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:26.383 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.105537+0000", 2024-05-08T08:52:26.383 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5400000000000001e-07 2024-05-08T08:52:26.383 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.383 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.383 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:26.383 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.105540+0000", 2024-05-08T08:52:26.383 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9689999999999999e-06 2024-05-08T08:52:26.383 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.383 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.383 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:26.383 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.105547+0000", 2024-05-08T08:52:26.384 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7209999999999994e-06 2024-05-08T08:52:26.384 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.384 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:26.384 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.105555+0000", 2024-05-08T08:52:26.384 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5800000000000003e-06 2024-05-08T08:52:26.384 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.384 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:26.384 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.105631+0000", 2024-05-08T08:52:26.384 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6161000000000005e-05 2024-05-08T08:52:26.384 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.384 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:26.384 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.105775+0000", 2024-05-08T08:52:26.385 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014406799999999999 2024-05-08T08:52:26.385 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.385 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.385 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:26.385 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.113870+0000", 2024-05-08T08:52:26.385 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0080949330000000003 2024-05-08T08:52:26.385 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.385 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:26.385 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.385 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.385 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.385 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "pg_update_log_missing(3.a epoch 24/18 rep_tid 489 entries 24'217 (0'0) error 3:502d98fa:::rbd_data.114052026309.00000000000000fa:head by client.4492.0:1049 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 24'215)", 2024-05-08T08:52:26.385 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:19.105829+0000", 2024-05-08T08:52:26.385 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.2360927610000001, 2024-05-08T08:52:26.385 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0080571949999999996, 2024-05-08T08:52:26.386 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:26.386 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:26.386 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-05-08T08:52:26.386 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:26.386 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.386 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:26.386 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.105829+0000", 2024-05-08T08:52:26.386 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:26.386 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.386 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.386 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:26.386 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.105829+0000", 2024-05-08T08:52:26.386 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:26.386 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.386 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.387 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:26.387 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.105829+0000", 2024-05-08T08:52:26.387 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9800000000000003e-07 2024-05-08T08:52:26.387 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.387 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.387 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:26.387 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.105837+0000", 2024-05-08T08:52:26.387 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6680000000000001e-06 2024-05-08T08:52:26.387 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.387 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.387 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:26.387 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.105855+0000", 2024-05-08T08:52:26.387 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8074e-05 2024-05-08T08:52:26.387 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.387 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.388 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:26.388 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.105869+0000", 2024-05-08T08:52:26.388 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4453999999999999e-05 2024-05-08T08:52:26.388 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.388 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.388 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:26.388 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.106069+0000", 2024-05-08T08:52:26.388 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00019910899999999999 2024-05-08T08:52:26.388 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.388 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.388 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:26.388 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.113886+0000", 2024-05-08T08:52:26.388 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007817292 2024-05-08T08:52:26.388 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.388 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:26.389 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.389 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.389 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.389 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "pg_update_log_missing(3.1c epoch 24/18 rep_tid 198 entries 24'31 (0'0) error 3:398e8057:::rbd_data.114052026309.00000000000000f9:head by client.4492.0:1047 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 24'29)", 2024-05-08T08:52:26.389 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:19.106130+0000", 2024-05-08T08:52:26.389 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.2357914000000001, 2024-05-08T08:52:26.389 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0076616130000000003, 2024-05-08T08:52:26.389 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:26.389 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:26.389 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-05-08T08:52:26.389 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:26.389 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.389 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:26.389 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.106130+0000", 2024-05-08T08:52:26.389 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:26.389 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.390 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.390 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:26.390 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.106130+0000", 2024-05-08T08:52:26.390 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:26.390 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.390 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.390 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:26.390 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.106131+0000", 2024-05-08T08:52:26.390 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7400000000000003e-07 2024-05-08T08:52:26.390 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.390 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.390 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:26.390 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.106138+0000", 2024-05-08T08:52:26.390 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7090000000000002e-06 2024-05-08T08:52:26.391 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.391 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.391 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:26.391 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.106155+0000", 2024-05-08T08:52:26.391 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7122e-05 2024-05-08T08:52:26.391 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.391 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.391 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:26.391 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.106168+0000", 2024-05-08T08:52:26.391 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2544000000000001e-05 2024-05-08T08:52:26.391 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.391 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.391 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:26.391 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.106254+0000", 2024-05-08T08:52:26.391 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5902999999999994e-05 2024-05-08T08:52:26.392 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.392 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.392 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:26.392 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.113792+0000", 2024-05-08T08:52:26.392 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0075377609999999996 2024-05-08T08:52:26.392 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.392 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:26.392 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.392 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.392 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.392 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "pg_update_log_missing(3.1d epoch 24/18 rep_tid 199 entries 24'22 (0'0) error 3:baedc9ab:::rbd_data.114052026309.00000000000000fc:head by client.4492.0:1051 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 24'20)", 2024-05-08T08:52:26.392 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:19.106193+0000", 2024-05-08T08:52:26.392 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.2357282930000002, 2024-05-08T08:52:26.392 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0076040470000000001, 2024-05-08T08:52:26.392 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:26.393 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:26.393 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-05-08T08:52:26.393 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:26.393 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.393 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:26.393 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.106193+0000", 2024-05-08T08:52:26.393 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:26.393 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.393 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.393 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:26.393 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.106193+0000", 2024-05-08T08:52:26.393 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:26.393 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.393 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.393 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:26.394 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.106193+0000", 2024-05-08T08:52:26.394 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5199999999999998e-07 2024-05-08T08:52:26.394 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.394 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.394 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:26.394 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.106199+0000", 2024-05-08T08:52:26.394 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4840000000000003e-06 2024-05-08T08:52:26.394 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.394 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.394 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:26.394 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.106213+0000", 2024-05-08T08:52:26.394 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3718e-05 2024-05-08T08:52:26.394 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.395 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.395 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:26.395 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.106224+0000", 2024-05-08T08:52:26.395 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1207e-05 2024-05-08T08:52:26.395 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.395 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.395 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:26.395 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.106289+0000", 2024-05-08T08:52:26.395 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5539999999999999e-05 2024-05-08T08:52:26.395 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.395 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.395 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:26.395 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.113797+0000", 2024-05-08T08:52:26.395 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007507746 2024-05-08T08:52:26.396 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.396 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:26.396 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.396 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.396 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.396 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "pg_update_log_missing(3.a epoch 24/18 rep_tid 490 entries 24'218 (0'0) error 3:54689ce5:::rbd_data.114052026309.00000000000000fb:head by client.4492.0:1050 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 24'215)", 2024-05-08T08:52:26.396 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:19.106223+0000", 2024-05-08T08:52:26.396 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.2356983289999999, 2024-05-08T08:52:26.396 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0076744270000000002, 2024-05-08T08:52:26.396 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:26.396 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:26.396 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-05-08T08:52:26.396 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:26.396 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.396 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:26.397 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.106223+0000", 2024-05-08T08:52:26.397 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:26.397 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.397 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.397 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:26.397 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.106223+0000", 2024-05-08T08:52:26.397 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:26.397 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.397 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.397 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:26.397 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.106224+0000", 2024-05-08T08:52:26.397 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2699999999999999e-07 2024-05-08T08:52:26.397 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.397 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.398 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:26.398 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.106231+0000", 2024-05-08T08:52:26.398 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6539999999999999e-06 2024-05-08T08:52:26.398 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.398 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.398 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:26.398 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.106249+0000", 2024-05-08T08:52:26.398 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7479e-05 2024-05-08T08:52:26.398 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.398 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.398 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:26.398 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.106262+0000", 2024-05-08T08:52:26.398 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3227999999999999e-05 2024-05-08T08:52:26.398 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.399 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.399 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:26.399 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.106353+0000", 2024-05-08T08:52:26.399 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0946000000000006e-05 2024-05-08T08:52:26.399 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.399 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.399 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:26.399 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.113898+0000", 2024-05-08T08:52:26.399 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0075444930000000002 2024-05-08T08:52:26.399 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.399 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:26.399 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.399 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.399 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.399 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "pg_update_log_missing(3.1c epoch 24/18 rep_tid 200 entries 24'32 (0'0) error 3:3a4bb9ad:::rbd_data.114052026309.00000000000000ff:head by client.4492.0:1056 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 24'29)", 2024-05-08T08:52:26.400 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:19.106242+0000", 2024-05-08T08:52:26.400 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.2356793389999998, 2024-05-08T08:52:26.400 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0075633330000000002, 2024-05-08T08:52:26.400 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:26.400 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:26.400 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-05-08T08:52:26.400 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:26.400 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.400 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:26.400 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.106242+0000", 2024-05-08T08:52:26.400 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:26.401 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.401 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.401 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:26.401 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.106242+0000", 2024-05-08T08:52:26.401 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:26.401 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.401 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.401 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:26.401 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.106242+0000", 2024-05-08T08:52:26.401 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.39e-07 2024-05-08T08:52:26.401 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.401 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.401 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:26.401 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.106247+0000", 2024-05-08T08:52:26.402 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8189999999999999e-06 2024-05-08T08:52:26.402 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.402 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.402 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:26.402 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.106257+0000", 2024-05-08T08:52:26.402 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6299999999999993e-06 2024-05-08T08:52:26.402 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.402 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.402 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:26.402 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.106269+0000", 2024-05-08T08:52:26.402 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2119999999999999e-05 2024-05-08T08:52:26.402 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.402 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.403 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:26.403 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.106397+0000", 2024-05-08T08:52:26.403 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000127806 2024-05-08T08:52:26.403 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.403 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.403 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:26.403 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.113805+0000", 2024-05-08T08:52:26.403 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007408619 2024-05-08T08:52:26.403 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.403 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:26.403 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.403 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.403 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.403 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4492.0:1094 3.1b e24/18 3:dbcdc403:::rbd_data.114052026309.00000000000000f0:head v 24'28, mlcod=24'25)", 2024-05-08T08:52:26.403 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:19.109431+0000", 2024-05-08T08:52:26.404 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.2324907090000004, 2024-05-08T08:52:26.404 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0092267200000000008, 2024-05-08T08:52:26.404 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:26.404 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:26.404 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:26.404 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:26.404 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.404 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:26.404 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.109431+0000", 2024-05-08T08:52:26.404 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:26.404 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.404 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.404 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:26.404 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.109431+0000", 2024-05-08T08:52:26.405 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:26.405 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.405 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.405 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:26.405 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.109431+0000", 2024-05-08T08:52:26.405 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5899999999999996e-07 2024-05-08T08:52:26.405 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.405 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.405 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:26.405 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.109438+0000", 2024-05-08T08:52:26.405 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7700000000000004e-06 2024-05-08T08:52:26.405 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.405 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.405 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:26.406 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.109450+0000", 2024-05-08T08:52:26.406 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1491999999999999e-05 2024-05-08T08:52:26.406 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.406 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.406 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:26.406 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.109456+0000", 2024-05-08T08:52:26.406 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.55e-06 2024-05-08T08:52:26.406 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.406 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.406 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:26.406 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.109506+0000", 2024-05-08T08:52:26.406 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0108999999999997e-05 2024-05-08T08:52:26.406 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.406 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.407 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:26.407 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.109528+0000", 2024-05-08T08:52:26.407 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2079e-05 2024-05-08T08:52:26.407 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.407 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.407 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:26.407 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.118621+0000", 2024-05-08T08:52:26.407 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0090922550000000005 2024-05-08T08:52:26.407 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.407 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.407 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:26.407 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.118657+0000", 2024-05-08T08:52:26.407 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6906000000000002e-05 2024-05-08T08:52:26.407 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.407 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:26.408 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.408 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.408 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.408 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4492.0:1105 3.14 e24/18 3:2c3f186e:::rbd_data.114052026309.00000000000000ed:head v 24'20, mlcod=24'18)", 2024-05-08T08:52:26.408 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:19.110037+0000", 2024-05-08T08:52:26.408 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.2318844469999997, 2024-05-08T08:52:26.408 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0086111020000000007, 2024-05-08T08:52:26.408 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:26.408 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:26.408 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:26.408 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:26.408 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.408 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:26.409 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.110037+0000", 2024-05-08T08:52:26.409 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:26.409 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.409 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.409 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:26.409 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.110037+0000", 2024-05-08T08:52:26.409 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:26.409 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.409 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.409 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:26.409 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.110038+0000", 2024-05-08T08:52:26.409 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0900000000000001e-07 2024-05-08T08:52:26.409 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.409 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.409 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:26.410 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.110042+0000", 2024-05-08T08:52:26.410 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4649999999999996e-06 2024-05-08T08:52:26.410 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.410 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.410 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:26.410 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.110052+0000", 2024-05-08T08:52:26.410 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6059999999999992e-06 2024-05-08T08:52:26.410 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.410 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.410 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:26.410 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.110058+0000", 2024-05-08T08:52:26.410 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2280000000000003e-06 2024-05-08T08:52:26.410 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.411 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.411 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:26.411 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.110103+0000", 2024-05-08T08:52:26.411 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5084999999999999e-05 2024-05-08T08:52:26.411 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.411 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.411 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:26.411 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.110123+0000", 2024-05-08T08:52:26.411 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9646000000000001e-05 2024-05-08T08:52:26.411 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.411 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.411 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:26.411 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.118612+0000", 2024-05-08T08:52:26.411 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0084894140000000007 2024-05-08T08:52:26.412 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.412 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.412 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:26.412 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.118648+0000", 2024-05-08T08:52:26.412 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6049e-05 2024-05-08T08:52:26.412 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.412 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:26.412 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.412 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.412 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.412 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4492.0:1106 3.a e24/18 3:517bbb4d:::rbd_data.114052026309.00000000000000eb:head v 24'219, mlcod=24'215)", 2024-05-08T08:52:26.412 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:19.110248+0000", 2024-05-08T08:52:26.412 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.2316738029999996, 2024-05-08T08:52:26.412 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0085531319999999997, 2024-05-08T08:52:26.413 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:26.413 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:26.413 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:26.413 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:26.413 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.413 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:26.413 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.110248+0000", 2024-05-08T08:52:26.413 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:26.413 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.413 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.413 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:26.413 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.110248+0000", 2024-05-08T08:52:26.413 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:26.414 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.414 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.414 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:26.414 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.110248+0000", 2024-05-08T08:52:26.414 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9800000000000003e-07 2024-05-08T08:52:26.414 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.414 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.414 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:26.414 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.110253+0000", 2024-05-08T08:52:26.414 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8010000000000003e-06 2024-05-08T08:52:26.414 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.414 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.414 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:26.414 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.110262+0000", 2024-05-08T08:52:26.415 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2590000000000006e-06 2024-05-08T08:52:26.415 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.415 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.415 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:26.415 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.110270+0000", 2024-05-08T08:52:26.415 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7239999999999992e-06 2024-05-08T08:52:26.415 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.415 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.415 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:26.415 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.110323+0000", 2024-05-08T08:52:26.415 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3433999999999997e-05 2024-05-08T08:52:26.415 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.415 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.416 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:26.416 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.110343+0000", 2024-05-08T08:52:26.416 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9232000000000002e-05 2024-05-08T08:52:26.416 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.416 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.416 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:26.416 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.118771+0000", 2024-05-08T08:52:26.416 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0084282699999999999 2024-05-08T08:52:26.416 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.416 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.416 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:26.416 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.118801+0000", 2024-05-08T08:52:26.416 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9813999999999999e-05 2024-05-08T08:52:26.417 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.417 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:26.417 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.417 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.417 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.417 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4492.0:1132 3.9 3:92f7a35e:::rbd_data.114052026309.00000000000000fd:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-05-08T08:52:26.417 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:19.113992+0000", 2024-05-08T08:52:26.417 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.2279294639999998, 2024-05-08T08:52:26.417 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0076760209999999999, 2024-05-08T08:52:26.417 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:26.417 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:26.417 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:26.417 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:26.417 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4492", 2024-05-08T08:52:26.417 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/1076427944", 2024-05-08T08:52:26.418 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1132 2024-05-08T08:52:26.418 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.418 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:26.418 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.418 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:26.418 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.113992+0000", 2024-05-08T08:52:26.418 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:26.418 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.418 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.418 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:26.418 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.113992+0000", 2024-05-08T08:52:26.418 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:26.418 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.419 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.419 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:26.419 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.113993+0000", 2024-05-08T08:52:26.419 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0750000000000001e-06 2024-05-08T08:52:26.419 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.419 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.419 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:26.419 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.113998+0000", 2024-05-08T08:52:26.419 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2900000000000002e-06 2024-05-08T08:52:26.419 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.419 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.419 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:26.419 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.114009+0000", 2024-05-08T08:52:26.419 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0818999999999999e-05 2024-05-08T08:52:26.420 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.420 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.420 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:26.420 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.114016+0000", 2024-05-08T08:52:26.420 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8120000000000002e-06 2024-05-08T08:52:26.420 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.420 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.420 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:26.420 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.114063+0000", 2024-05-08T08:52:26.420 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7448999999999999e-05 2024-05-08T08:52:26.420 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.420 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.420 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:26.420 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.114136+0000", 2024-05-08T08:52:26.421 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2826999999999999e-05 2024-05-08T08:52:26.421 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.421 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.421 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 2", 2024-05-08T08:52:26.421 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.114334+0000", 2024-05-08T08:52:26.421 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000197283 2024-05-08T08:52:26.421 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.421 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.421 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:26.421 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.116805+0000", 2024-05-08T08:52:26.421 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0024716 2024-05-08T08:52:26.421 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.421 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.421 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:52:26.421 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.121541+0000", 2024-05-08T08:52:26.422 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0047357620000000001 2024-05-08T08:52:26.422 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.422 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.422 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:26.422 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.121625+0000", 2024-05-08T08:52:26.422 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3807000000000006e-05 2024-05-08T08:52:26.422 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.422 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.422 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:26.422 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.121668+0000", 2024-05-08T08:52:26.422 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3297000000000002e-05 2024-05-08T08:52:26.422 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.422 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:26.422 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.423 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.423 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.423 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4492.0:1133 3.1a 3:5a2b8914:::rbd_data.114052026309.0000000000000100:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-05-08T08:52:26.423 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:19.114034+0000", 2024-05-08T08:52:26.423 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.2278879209999998, 2024-05-08T08:52:26.423 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0077546009999999999, 2024-05-08T08:52:26.423 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:26.423 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:26.423 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:26.423 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:26.423 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4492", 2024-05-08T08:52:26.423 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/1076427944", 2024-05-08T08:52:26.423 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1133 2024-05-08T08:52:26.423 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.424 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:26.424 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.424 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:26.424 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.114034+0000", 2024-05-08T08:52:26.424 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:26.424 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.424 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.424 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:26.424 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.114034+0000", 2024-05-08T08:52:26.424 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:26.424 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.424 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.424 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:26.424 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.114034+0000", 2024-05-08T08:52:26.425 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2e-07 2024-05-08T08:52:26.425 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.425 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.425 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:26.425 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.114037+0000", 2024-05-08T08:52:26.425 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1990000000000002e-06 2024-05-08T08:52:26.425 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.425 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.425 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:26.425 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.114045+0000", 2024-05-08T08:52:26.425 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.469e-06 2024-05-08T08:52:26.425 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.425 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.425 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:26.425 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.114051+0000", 2024-05-08T08:52:26.425 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7700000000000004e-06 2024-05-08T08:52:26.426 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.426 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.426 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:26.426 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.114094+0000", 2024-05-08T08:52:26.426 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2491999999999999e-05 2024-05-08T08:52:26.426 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.426 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.426 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:26.426 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.114157+0000", 2024-05-08T08:52:26.426 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3178999999999999e-05 2024-05-08T08:52:26.426 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.426 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.426 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-05-08T08:52:26.426 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.114358+0000", 2024-05-08T08:52:26.426 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020135199999999999 2024-05-08T08:52:26.427 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.427 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.427 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:26.427 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.116018+0000", 2024-05-08T08:52:26.427 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016593630000000001 2024-05-08T08:52:26.427 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.427 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.427 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:52:26.427 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.121569+0000", 2024-05-08T08:52:26.427 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0055506899999999996 2024-05-08T08:52:26.427 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.427 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.427 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:26.427 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.121748+0000", 2024-05-08T08:52:26.428 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00017919600000000001 2024-05-08T08:52:26.428 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.428 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.428 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:26.428 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.121788+0000", 2024-05-08T08:52:26.428 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0370999999999997e-05 2024-05-08T08:52:26.428 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.428 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:26.428 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.428 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.428 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.428 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4492.0:1135 3.1a 3:597af2f9:::rbd_data.114052026309.0000000000000111:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-05-08T08:52:26.428 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:19.114086+0000", 2024-05-08T08:52:26.429 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.227835164, 2024-05-08T08:52:26.429 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0078299119999999996, 2024-05-08T08:52:26.429 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:26.429 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:26.429 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:26.429 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:26.429 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4492", 2024-05-08T08:52:26.429 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/1076427944", 2024-05-08T08:52:26.429 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1135 2024-05-08T08:52:26.429 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.429 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:26.429 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.429 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:26.429 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.114086+0000", 2024-05-08T08:52:26.429 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:26.430 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.430 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.430 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:26.430 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.114086+0000", 2024-05-08T08:52:26.430 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:26.430 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.430 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.430 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:26.430 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.114087+0000", 2024-05-08T08:52:26.430 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9299999999999997e-07 2024-05-08T08:52:26.430 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.430 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.430 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:26.430 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.114090+0000", 2024-05-08T08:52:26.431 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3790000000000002e-06 2024-05-08T08:52:26.431 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.431 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.431 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:26.431 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.114097+0000", 2024-05-08T08:52:26.431 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1380000000000001e-06 2024-05-08T08:52:26.431 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.431 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.431 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:26.431 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.114103+0000", 2024-05-08T08:52:26.431 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8549999999999998e-06 2024-05-08T08:52:26.431 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.431 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.431 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:26.431 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.114681+0000", 2024-05-08T08:52:26.432 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00057759300000000005 2024-05-08T08:52:26.432 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.432 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.432 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:26.432 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.114802+0000", 2024-05-08T08:52:26.432 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012094 2024-05-08T08:52:26.432 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.432 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.432 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:26.432 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.121916+0000", 2024-05-08T08:52:26.432 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0071143140000000001 2024-05-08T08:52:26.432 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.432 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:26.432 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.433 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.433 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:26.433 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:52:26.434 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:52:26.435 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:52:26.438+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:52:26.535 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:52:26.538+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:52:26.581 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:52:26.581 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:52:26.581 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:52:26.581 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:52:26.590 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:52:26.636 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:52:26.639+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:52:26.694 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:52:26.694 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:52:26.694 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:52:26.694 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:52:26.694 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:52:26.702 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:52:26.736 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:52:26.740+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:52:26.809 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:52:26.809 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:52:26.810 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:52:26.810 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:52:26.810 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.810 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4492.0:356 3.c 3:34e44dc8:::rbd_data.114052026309.0000000000000028:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-05-08T08:52:26.810 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:19.036633+0000", 2024-05-08T08:52:26.810 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.7747460449999997, 2024-05-08T08:52:26.810 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.011595787, 2024-05-08T08:52:26.810 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:26.810 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:26.810 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:26.810 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:26.810 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4492", 2024-05-08T08:52:26.810 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/1076427944", 2024-05-08T08:52:26.811 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 356 2024-05-08T08:52:26.811 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.811 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:26.811 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.811 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:26.811 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.036633+0000", 2024-05-08T08:52:26.811 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:26.811 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.811 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.811 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:26.811 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.036633+0000", 2024-05-08T08:52:26.811 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:26.811 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.811 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.812 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:26.812 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.036633+0000", 2024-05-08T08:52:26.812 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0500000000000003e-07 2024-05-08T08:52:26.812 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.812 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.812 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:26.812 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.036638+0000", 2024-05-08T08:52:26.812 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0579999999999999e-06 2024-05-08T08:52:26.812 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.812 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.812 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:26.812 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.036646+0000", 2024-05-08T08:52:26.812 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3280000000000006e-06 2024-05-08T08:52:26.812 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.812 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.813 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:26.813 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.036651+0000", 2024-05-08T08:52:26.813 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0529999999999997e-06 2024-05-08T08:52:26.813 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.813 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.813 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:26.813 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.036689+0000", 2024-05-08T08:52:26.813 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8179000000000002e-05 2024-05-08T08:52:26.813 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.813 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.813 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:26.813 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.036759+0000", 2024-05-08T08:52:26.813 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9548000000000006e-05 2024-05-08T08:52:26.813 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.813 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.814 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-05-08T08:52:26.814 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.036953+0000", 2024-05-08T08:52:26.814 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00019428800000000001 2024-05-08T08:52:26.814 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.814 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.814 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:52:26.814 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.041187+0000", 2024-05-08T08:52:26.814 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0042344499999999998 2024-05-08T08:52:26.814 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.814 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.814 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:26.814 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.048137+0000", 2024-05-08T08:52:26.814 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0069492340000000003 2024-05-08T08:52:26.814 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.815 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.815 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:26.815 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.048190+0000", 2024-05-08T08:52:26.815 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3091999999999999e-05 2024-05-08T08:52:26.815 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.815 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.815 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:26.815 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.048229+0000", 2024-05-08T08:52:26.815 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8852e-05 2024-05-08T08:52:26.815 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.815 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:26.815 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.815 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.815 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.816 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4492.0:374 3.5 3:a4b5bd4b:::rbd_data.114052026309.000000000000002b:head [delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-05-08T08:52:26.816 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:19.039055+0000", 2024-05-08T08:52:26.816 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.7723236, 2024-05-08T08:52:26.816 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010314874999999999, 2024-05-08T08:52:26.816 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:26.816 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:26.816 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:26.816 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:26.816 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4492", 2024-05-08T08:52:26.816 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/1076427944", 2024-05-08T08:52:26.816 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 374 2024-05-08T08:52:26.816 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.816 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:26.816 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.816 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:26.817 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.039055+0000", 2024-05-08T08:52:26.817 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:26.817 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.817 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.817 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:26.817 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.039055+0000", 2024-05-08T08:52:26.817 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:26.817 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.817 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.817 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:26.817 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.039056+0000", 2024-05-08T08:52:26.817 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5899999999999996e-07 2024-05-08T08:52:26.817 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.817 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.817 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:26.818 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.039058+0000", 2024-05-08T08:52:26.818 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6299999999999998e-06 2024-05-08T08:52:26.818 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.818 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.818 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:26.818 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.039065+0000", 2024-05-08T08:52:26.818 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6649999999999997e-06 2024-05-08T08:52:26.818 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.818 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.818 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:26.818 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.039070+0000", 2024-05-08T08:52:26.818 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1290000000000004e-06 2024-05-08T08:52:26.818 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.818 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.818 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:26.819 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.042068+0000", 2024-05-08T08:52:26.819 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0029976410000000001 2024-05-08T08:52:26.819 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.819 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.819 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:26.819 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.042140+0000", 2024-05-08T08:52:26.819 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2061e-05 2024-05-08T08:52:26.819 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.819 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.819 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 2", 2024-05-08T08:52:26.819 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.042290+0000", 2024-05-08T08:52:26.819 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015021500000000001 2024-05-08T08:52:26.819 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.819 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.820 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:26.820 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.047600+0000", 2024-05-08T08:52:26.820 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0053103289999999999 2024-05-08T08:52:26.820 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.820 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.820 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:52:26.820 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.049287+0000", 2024-05-08T08:52:26.820 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016864079999999999 2024-05-08T08:52:26.820 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.820 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.820 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:26.820 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.049329+0000", 2024-05-08T08:52:26.820 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2138999999999998e-05 2024-05-08T08:52:26.820 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.820 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.821 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:26.821 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.049370+0000", 2024-05-08T08:52:26.821 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1099000000000001e-05 2024-05-08T08:52:26.821 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.821 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:26.821 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.821 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.821 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.821 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4492.0:382 3.5 3:a2424f8d:::rbd_data.114052026309.0000000000000034:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-05-08T08:52:26.821 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:19.039273+0000", 2024-05-08T08:52:26.821 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.7721059029999999, 2024-05-08T08:52:26.821 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.016204234000000001, 2024-05-08T08:52:26.821 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:26.821 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:26.821 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:26.822 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:26.822 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4492", 2024-05-08T08:52:26.822 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/1076427944", 2024-05-08T08:52:26.822 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 382 2024-05-08T08:52:26.822 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.822 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:26.822 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.822 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:26.822 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.039273+0000", 2024-05-08T08:52:26.822 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:26.822 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.822 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.822 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:26.822 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.039273+0000", 2024-05-08T08:52:26.823 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:26.823 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.823 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.823 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:26.823 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.039273+0000", 2024-05-08T08:52:26.823 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7199999999999999e-07 2024-05-08T08:52:26.823 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.823 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.823 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:26.823 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.039276+0000", 2024-05-08T08:52:26.823 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0599999999999999e-06 2024-05-08T08:52:26.823 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.823 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.823 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:26.824 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.039283+0000", 2024-05-08T08:52:26.824 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2389999999999999e-06 2024-05-08T08:52:26.824 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.824 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.824 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:26.824 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.039288+0000", 2024-05-08T08:52:26.824 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9470000000000002e-06 2024-05-08T08:52:26.824 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.824 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.824 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:26.824 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.046899+0000", 2024-05-08T08:52:26.824 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0076113279999999997 2024-05-08T08:52:26.824 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.824 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.824 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:26.825 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.046981+0000", 2024-05-08T08:52:26.825 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2482000000000001e-05 2024-05-08T08:52:26.825 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.825 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.825 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 2", 2024-05-08T08:52:26.825 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.047169+0000", 2024-05-08T08:52:26.825 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00018705100000000001 2024-05-08T08:52:26.825 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.825 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.825 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:52:26.825 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.049953+0000", 2024-05-08T08:52:26.825 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0027841020000000001 2024-05-08T08:52:26.825 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.825 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.825 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:26.826 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.055388+0000", 2024-05-08T08:52:26.826 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.005435189 2024-05-08T08:52:26.826 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.826 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.826 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:26.826 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.055439+0000", 2024-05-08T08:52:26.826 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1669999999999998e-05 2024-05-08T08:52:26.826 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.826 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.826 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:26.826 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.055477+0000", 2024-05-08T08:52:26.826 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7694000000000002e-05 2024-05-08T08:52:26.826 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.827 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:26.827 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.827 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.827 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.827 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4492.0:387 3.5 3:a6ec6807:::rbd_data.114052026309.000000000000003d:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-05-08T08:52:26.827 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:19.041433+0000", 2024-05-08T08:52:26.827 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.7699456700000002, 2024-05-08T08:52:26.827 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.014154582000000001, 2024-05-08T08:52:26.827 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:26.827 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:26.827 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:26.827 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:26.827 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4492", 2024-05-08T08:52:26.827 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/1076427944", 2024-05-08T08:52:26.827 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 387 2024-05-08T08:52:26.828 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.828 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:26.828 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.828 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:26.828 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.041433+0000", 2024-05-08T08:52:26.828 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:26.828 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.828 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.828 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:26.828 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.041433+0000", 2024-05-08T08:52:26.828 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:26.828 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.828 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.828 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:26.828 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.041434+0000", 2024-05-08T08:52:26.829 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4099999999999998e-07 2024-05-08T08:52:26.829 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.829 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.829 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:26.829 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.041438+0000", 2024-05-08T08:52:26.829 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8210000000000003e-06 2024-05-08T08:52:26.829 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.829 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.829 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:26.829 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.041447+0000", 2024-05-08T08:52:26.829 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3340000000000008e-06 2024-05-08T08:52:26.829 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.829 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.829 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:26.829 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.041453+0000", 2024-05-08T08:52:26.830 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3960000000000003e-06 2024-05-08T08:52:26.830 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.830 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.830 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:26.830 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.047635+0000", 2024-05-08T08:52:26.830 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0061812860000000002 2024-05-08T08:52:26.830 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.830 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.830 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:26.830 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.047732+0000", 2024-05-08T08:52:26.830 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7187000000000003e-05 2024-05-08T08:52:26.830 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.830 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.831 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:26.831 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.055588+0000", 2024-05-08T08:52:26.831 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0078559170000000005 2024-05-08T08:52:26.831 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.831 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:26.831 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.831 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.831 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.831 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4492.0:406 3.5 3:a311dd56:::rbd_data.114052026309.0000000000000015:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-05-08T08:52:26.831 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:19.041676+0000", 2024-05-08T08:52:26.831 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.7697029679999998, 2024-05-08T08:52:26.831 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0140521, 2024-05-08T08:52:26.831 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:26.831 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:26.831 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:26.832 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:26.832 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4492", 2024-05-08T08:52:26.832 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/1076427944", 2024-05-08T08:52:26.832 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 406 2024-05-08T08:52:26.832 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.832 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:26.832 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.832 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:26.832 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.041676+0000", 2024-05-08T08:52:26.832 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:26.832 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.832 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.832 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:26.832 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.041676+0000", 2024-05-08T08:52:26.832 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:26.833 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.833 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.833 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:26.833 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.041676+0000", 2024-05-08T08:52:26.833 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9399999999999995e-07 2024-05-08T08:52:26.833 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.833 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.833 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:26.833 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.041679+0000", 2024-05-08T08:52:26.833 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5579999999999999e-06 2024-05-08T08:52:26.833 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.833 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.833 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:26.833 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.041686+0000", 2024-05-08T08:52:26.833 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7340000000000002e-06 2024-05-08T08:52:26.833 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.834 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.834 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:26.834 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.041691+0000", 2024-05-08T08:52:26.834 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.989e-06 2024-05-08T08:52:26.834 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.834 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.834 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:26.834 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.048570+0000", 2024-05-08T08:52:26.834 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00687924 2024-05-08T08:52:26.834 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.834 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.834 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:26.834 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.048638+0000", 2024-05-08T08:52:26.834 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8287999999999994e-05 2024-05-08T08:52:26.835 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.835 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.835 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 2", 2024-05-08T08:52:26.835 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.048818+0000", 2024-05-08T08:52:26.835 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000179839 2024-05-08T08:52:26.835 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.835 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.835 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:52:26.835 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.052160+0000", 2024-05-08T08:52:26.835 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0033416299999999999 2024-05-08T08:52:26.835 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.835 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.835 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:26.835 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.055627+0000", 2024-05-08T08:52:26.836 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0034672240000000001 2024-05-08T08:52:26.836 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.836 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.836 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:26.836 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.055692+0000", 2024-05-08T08:52:26.836 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5483999999999998e-05 2024-05-08T08:52:26.836 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.836 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.836 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:26.836 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.055728+0000", 2024-05-08T08:52:26.836 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5620000000000001e-05 2024-05-08T08:52:26.836 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.837 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:26.837 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.837 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.837 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.837 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4492.0:520 3.c 3:362b2d9b:::rbd_data.114052026309.000000000000005c:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-05-08T08:52:26.837 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:19.054078+0000", 2024-05-08T08:52:26.837 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.757301129, 2024-05-08T08:52:26.837 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.012133771999999999, 2024-05-08T08:52:26.837 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:26.837 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:26.837 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:26.837 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:26.838 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4492", 2024-05-08T08:52:26.838 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/1076427944", 2024-05-08T08:52:26.838 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 520 2024-05-08T08:52:26.838 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.838 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:26.838 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.838 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:26.838 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.054078+0000", 2024-05-08T08:52:26.838 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:26.838 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.838 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.838 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:26.838 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.054078+0000", 2024-05-08T08:52:26.839 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:26.839 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.839 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.839 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:26.839 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.054078+0000", 2024-05-08T08:52:26.839 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3e-07 2024-05-08T08:52:26.839 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.839 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.839 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:26.839 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.054082+0000", 2024-05-08T08:52:26.839 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7459999999999998e-06 2024-05-08T08:52:26.839 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.839 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.839 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:26.840 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.054092+0000", 2024-05-08T08:52:26.840 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5640000000000002e-06 2024-05-08T08:52:26.840 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.840 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.840 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:26.840 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.054099+0000", 2024-05-08T08:52:26.840 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5109999999999997e-06 2024-05-08T08:52:26.840 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.840 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.840 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:26.840 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.054822+0000", 2024-05-08T08:52:26.840 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00072319300000000001 2024-05-08T08:52:26.840 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.840 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.840 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:26.840 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.054950+0000", 2024-05-08T08:52:26.841 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000127226 2024-05-08T08:52:26.841 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.841 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.841 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:26.841 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.066211+0000", 2024-05-08T08:52:26.841 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.011261802 2024-05-08T08:52:26.841 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.841 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:26.841 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.841 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.841 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.841 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4492.0:809 3.10 3:0d5d94e1:::rbd_data.114052026309.000000000000009b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-05-08T08:52:26.841 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:19.080673+0000", 2024-05-08T08:52:26.841 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.7307062010000003, 2024-05-08T08:52:26.841 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010518886, 2024-05-08T08:52:26.841 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:26.842 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:26.842 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:26.842 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:26.842 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4492", 2024-05-08T08:52:26.842 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/1076427944", 2024-05-08T08:52:26.842 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 809 2024-05-08T08:52:26.842 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.842 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:26.842 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.842 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:26.842 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.080673+0000", 2024-05-08T08:52:26.842 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:26.842 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.842 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.842 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:26.843 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.080673+0000", 2024-05-08T08:52:26.843 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:26.843 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.843 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.843 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:26.843 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.080673+0000", 2024-05-08T08:52:26.843 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7800000000000001e-07 2024-05-08T08:52:26.843 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.843 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.843 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:26.843 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.080678+0000", 2024-05-08T08:52:26.843 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7199999999999997e-06 2024-05-08T08:52:26.844 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.844 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.844 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:26.844 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.080689+0000", 2024-05-08T08:52:26.844 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0539e-05 2024-05-08T08:52:26.844 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.844 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.844 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:26.844 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.080695+0000", 2024-05-08T08:52:26.844 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5490000000000003e-06 2024-05-08T08:52:26.844 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.844 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.844 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:26.844 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.080743+0000", 2024-05-08T08:52:26.845 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8111000000000001e-05 2024-05-08T08:52:26.845 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.845 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.845 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:26.845 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.080824+0000", 2024-05-08T08:52:26.845 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0996999999999997e-05 2024-05-08T08:52:26.845 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.845 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.845 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-05-08T08:52:26.845 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.081011+0000", 2024-05-08T08:52:26.845 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000187009 2024-05-08T08:52:26.845 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.845 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.846 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:52:26.846 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.088360+0000", 2024-05-08T08:52:26.846 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0073487179999999997 2024-05-08T08:52:26.846 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.846 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.846 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:26.846 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.091094+0000", 2024-05-08T08:52:26.846 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0027338990000000001 2024-05-08T08:52:26.846 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.846 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.846 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:26.846 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.091151+0000", 2024-05-08T08:52:26.846 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7525000000000002e-05 2024-05-08T08:52:26.847 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.847 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.847 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:26.847 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.091192+0000", 2024-05-08T08:52:26.847 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0040999999999998e-05 2024-05-08T08:52:26.847 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.847 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:26.847 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.847 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.847 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.847 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4492.0:843 3.10 3:0c70fbd7:::rbd_data.114052026309.00000000000000bd:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-05-08T08:52:26.847 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:19.084147+0000", 2024-05-08T08:52:26.847 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.7272319139999999, 2024-05-08T08:52:26.847 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.011722827999999999, 2024-05-08T08:52:26.847 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:26.848 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:26.848 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:26.848 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:26.848 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4492", 2024-05-08T08:52:26.848 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/1076427944", 2024-05-08T08:52:26.848 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 843 2024-05-08T08:52:26.848 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.848 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:26.848 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.848 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:26.848 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.084147+0000", 2024-05-08T08:52:26.848 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:26.848 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.848 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.848 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:26.849 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.084147+0000", 2024-05-08T08:52:26.849 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:26.849 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.849 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.849 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:26.849 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.084148+0000", 2024-05-08T08:52:26.849 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0500000000000002e-07 2024-05-08T08:52:26.849 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.849 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.849 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:26.849 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.084152+0000", 2024-05-08T08:52:26.849 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1230000000000002e-06 2024-05-08T08:52:26.849 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.850 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.850 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:26.850 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.084162+0000", 2024-05-08T08:52:26.850 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7389999999999994e-06 2024-05-08T08:52:26.850 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.850 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.850 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:26.850 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.084168+0000", 2024-05-08T08:52:26.850 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8619999999999997e-06 2024-05-08T08:52:26.850 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.850 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.850 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:26.850 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.084229+0000", 2024-05-08T08:52:26.850 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0557000000000003e-05 2024-05-08T08:52:26.850 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.851 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.851 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:26.851 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.084353+0000", 2024-05-08T08:52:26.851 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000124327 2024-05-08T08:52:26.851 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.851 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.851 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:26.851 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.095870+0000", 2024-05-08T08:52:26.851 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.011516415 2024-05-08T08:52:26.851 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.851 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:26.851 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.851 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.851 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.852 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4492.0:1305 3.10 3:0a9ad982:::rbd_data.114052026309.0000000000000145:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-05-08T08:52:26.852 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:19.132561+0000", 2024-05-08T08:52:26.852 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.6788177399999995, 2024-05-08T08:52:26.852 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013632557999999999, 2024-05-08T08:52:26.852 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:26.852 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:26.852 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:26.852 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:26.852 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4492", 2024-05-08T08:52:26.852 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/1076427944", 2024-05-08T08:52:26.852 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1305 2024-05-08T08:52:26.852 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.852 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:26.852 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.853 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:26.853 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.132561+0000", 2024-05-08T08:52:26.853 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:26.853 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.853 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.853 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:26.853 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.132561+0000", 2024-05-08T08:52:26.853 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:26.853 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.853 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.853 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:26.853 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.132562+0000", 2024-05-08T08:52:26.853 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4700000000000001e-07 2024-05-08T08:52:26.853 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.853 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.854 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:26.854 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.132567+0000", 2024-05-08T08:52:26.854 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3619999999999997e-06 2024-05-08T08:52:26.854 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.854 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.854 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:26.854 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.132578+0000", 2024-05-08T08:52:26.854 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1229999999999999e-05 2024-05-08T08:52:26.854 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.854 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.854 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:26.854 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.132586+0000", 2024-05-08T08:52:26.854 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8970000000000008e-06 2024-05-08T08:52:26.855 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.855 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.855 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:26.855 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.132690+0000", 2024-05-08T08:52:26.855 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104146 2024-05-08T08:52:26.855 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.855 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.855 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:26.855 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.132883+0000", 2024-05-08T08:52:26.855 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000192678 2024-05-08T08:52:26.855 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.855 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.855 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:26.855 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.146194+0000", 2024-05-08T08:52:26.856 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013310498 2024-05-08T08:52:26.856 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.856 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:26.856 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.856 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.856 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.856 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4492.0:1339 3.10 3:0ccbd17f:::rbd_data.114052026309.0000000000000138:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-05-08T08:52:26.856 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:19.135232+0000", 2024-05-08T08:52:26.856 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.6761469089999999, 2024-05-08T08:52:26.856 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.011094043, 2024-05-08T08:52:26.856 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:26.856 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:26.856 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:26.856 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:26.857 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4492", 2024-05-08T08:52:26.857 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/1076427944", 2024-05-08T08:52:26.857 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1339 2024-05-08T08:52:26.857 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.857 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:26.857 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.857 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:26.857 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.135232+0000", 2024-05-08T08:52:26.857 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:26.857 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.857 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.857 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:26.857 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.135232+0000", 2024-05-08T08:52:26.857 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:26.857 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.858 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.858 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:26.858 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.135233+0000", 2024-05-08T08:52:26.858 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2799999999999995e-07 2024-05-08T08:52:26.858 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.858 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.858 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:26.858 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.135238+0000", 2024-05-08T08:52:26.858 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7570000000000004e-06 2024-05-08T08:52:26.858 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.858 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.858 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:26.858 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.135249+0000", 2024-05-08T08:52:26.858 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1168999999999999e-05 2024-05-08T08:52:26.858 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.859 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.859 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:26.859 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.135259+0000", 2024-05-08T08:52:26.859 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0282000000000001e-05 2024-05-08T08:52:26.859 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.859 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.859 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:26.859 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.135301+0000", 2024-05-08T08:52:26.859 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.214e-05 2024-05-08T08:52:26.859 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.859 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.859 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:26.859 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.135391+0000", 2024-05-08T08:52:26.859 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9666000000000005e-05 2024-05-08T08:52:26.859 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.860 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.860 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-05-08T08:52:26.860 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.135601+0000", 2024-05-08T08:52:26.860 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021018499999999999 2024-05-08T08:52:26.860 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.860 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.860 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:52:26.860 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.136997+0000", 2024-05-08T08:52:26.860 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0013961049999999999 2024-05-08T08:52:26.860 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.860 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.860 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:26.860 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.146230+0000", 2024-05-08T08:52:26.860 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.009233425 2024-05-08T08:52:26.860 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.860 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.861 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:26.861 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.146277+0000", 2024-05-08T08:52:26.861 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6708000000000001e-05 2024-05-08T08:52:26.861 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.861 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.861 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:26.861 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.146326+0000", 2024-05-08T08:52:26.861 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8778e-05 2024-05-08T08:52:26.861 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.861 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:26.861 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.861 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.861 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.861 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4492.0:1340 3.c 3:30be3cd6:::rbd_data.114052026309.0000000000000139:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-05-08T08:52:26.861 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:19.135272+0000", 2024-05-08T08:52:26.862 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.676107011, 2024-05-08T08:52:26.862 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.011051986, 2024-05-08T08:52:26.862 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:26.862 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:26.862 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:26.862 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:26.862 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4492", 2024-05-08T08:52:26.862 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/1076427944", 2024-05-08T08:52:26.862 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1340 2024-05-08T08:52:26.862 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.862 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:26.862 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.862 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:26.862 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.135272+0000", 2024-05-08T08:52:26.863 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:26.863 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.863 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.863 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:26.863 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.135272+0000", 2024-05-08T08:52:26.863 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:26.863 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.863 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.863 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:26.863 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.135272+0000", 2024-05-08T08:52:26.863 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.37e-07 2024-05-08T08:52:26.863 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.863 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.863 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:26.863 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.135276+0000", 2024-05-08T08:52:26.864 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6619999999999998e-06 2024-05-08T08:52:26.864 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.864 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.864 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:26.864 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.135283+0000", 2024-05-08T08:52:26.864 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0659999999999998e-06 2024-05-08T08:52:26.864 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.864 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.864 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:26.864 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.135290+0000", 2024-05-08T08:52:26.864 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3100000000000003e-06 2024-05-08T08:52:26.864 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.864 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.864 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:26.864 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.135474+0000", 2024-05-08T08:52:26.865 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00018352700000000001 2024-05-08T08:52:26.865 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.865 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.865 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:26.865 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.135521+0000", 2024-05-08T08:52:26.865 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6863999999999997e-05 2024-05-08T08:52:26.865 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.865 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.865 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-05-08T08:52:26.865 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.135620+0000", 2024-05-08T08:52:26.865 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9253999999999995e-05 2024-05-08T08:52:26.865 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.865 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.865 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:52:26.866 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.136989+0000", 2024-05-08T08:52:26.866 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001368684 2024-05-08T08:52:26.866 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.866 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.866 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:26.866 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.146223+0000", 2024-05-08T08:52:26.866 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0092343970000000001 2024-05-08T08:52:26.866 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.866 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.866 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:26.866 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.146276+0000", 2024-05-08T08:52:26.866 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3059000000000001e-05 2024-05-08T08:52:26.866 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.866 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.867 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:26.867 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:19.146324+0000", 2024-05-08T08:52:26.867 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7626000000000001e-05 2024-05-08T08:52:26.867 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.867 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:26.867 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.867 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.867 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.867 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4516.0:5106 3.5 3:a38a8548:::rbd_object_map.1140ff5ddfe9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-05-08T08:52:26.867 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:21.796378+0000", 2024-05-08T08:52:26.867 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.0150011079999999, 2024-05-08T08:52:26.867 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.011599094000000001, 2024-05-08T08:52:26.867 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:26.868 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:26.868 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:26.868 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:26.868 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4516", 2024-05-08T08:52:26.868 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/1777277669", 2024-05-08T08:52:26.868 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5106 2024-05-08T08:52:26.868 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.868 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:26.868 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.868 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:26.868 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.796378+0000", 2024-05-08T08:52:26.868 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:26.868 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.868 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.868 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:26.869 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.796378+0000", 2024-05-08T08:52:26.869 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:26.869 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.869 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.869 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:26.869 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.796378+0000", 2024-05-08T08:52:26.869 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.11e-07 2024-05-08T08:52:26.869 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.869 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.869 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:26.869 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.796380+0000", 2024-05-08T08:52:26.869 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.888e-06 2024-05-08T08:52:26.869 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.869 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.869 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:26.870 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.796383+0000", 2024-05-08T08:52:26.870 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.943e-06 2024-05-08T08:52:26.870 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.870 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.870 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:26.870 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.796385+0000", 2024-05-08T08:52:26.870 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2859999999999998e-06 2024-05-08T08:52:26.870 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.870 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.870 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:26.870 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.804877+0000", 2024-05-08T08:52:26.870 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0084920550000000001 2024-05-08T08:52:26.870 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.870 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.871 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:26.871 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.804951+0000", 2024-05-08T08:52:26.871 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4105999999999998e-05 2024-05-08T08:52:26.871 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.871 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.871 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 2", 2024-05-08T08:52:26.871 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.805133+0000", 2024-05-08T08:52:26.871 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00018138200000000001 2024-05-08T08:52:26.871 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.871 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.871 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:52:26.871 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.806258+0000", 2024-05-08T08:52:26.871 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011249280000000001 2024-05-08T08:52:26.871 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.871 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.871 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:26.872 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.807876+0000", 2024-05-08T08:52:26.872 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016184089999999999 2024-05-08T08:52:26.872 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.872 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.872 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:26.872 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.807929+0000", 2024-05-08T08:52:26.872 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2809999999999998e-05 2024-05-08T08:52:26.872 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.872 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.872 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:26.872 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.807977+0000", 2024-05-08T08:52:26.872 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8075999999999998e-05 2024-05-08T08:52:26.872 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.872 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:26.872 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.873 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.873 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.873 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4516.0:5107 3.5 3:a38a8548:::rbd_object_map.1140ff5ddfe9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-05-08T08:52:26.873 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:21.796388+0000", 2024-05-08T08:52:26.873 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.0149908200000004, 2024-05-08T08:52:26.873 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.011709763999999999, 2024-05-08T08:52:26.873 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:26.873 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:26.873 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:26.873 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:26.873 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4516", 2024-05-08T08:52:26.873 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/1777277669", 2024-05-08T08:52:26.873 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5107 2024-05-08T08:52:26.873 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.873 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:26.874 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.874 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:26.874 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.796388+0000", 2024-05-08T08:52:26.874 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:26.874 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.874 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.874 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:26.874 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.796388+0000", 2024-05-08T08:52:26.874 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:26.874 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.874 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.874 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:26.874 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.796388+0000", 2024-05-08T08:52:26.874 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.11e-07 2024-05-08T08:52:26.875 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.875 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.875 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:26.875 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.796390+0000", 2024-05-08T08:52:26.875 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4580000000000001e-06 2024-05-08T08:52:26.875 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.875 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.875 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:26.875 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.796393+0000", 2024-05-08T08:52:26.875 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9229999999999998e-06 2024-05-08T08:52:26.875 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.875 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.875 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:26.875 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.796395+0000", 2024-05-08T08:52:26.876 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2510000000000001e-06 2024-05-08T08:52:26.876 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.876 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.876 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:26.876 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.805442+0000", 2024-05-08T08:52:26.876 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.009047645 2024-05-08T08:52:26.876 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.876 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.876 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:26.876 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.805488+0000", 2024-05-08T08:52:26.876 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5175e-05 2024-05-08T08:52:26.876 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.876 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.876 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 2", 2024-05-08T08:52:26.876 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.805734+0000", 2024-05-08T08:52:26.877 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00024666999999999999 2024-05-08T08:52:26.877 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.877 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.877 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:52:26.877 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.807990+0000", 2024-05-08T08:52:26.877 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0022559870000000001 2024-05-08T08:52:26.877 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.877 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.877 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:26.877 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.808025+0000", 2024-05-08T08:52:26.877 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4870999999999999e-05 2024-05-08T08:52:26.877 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.877 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.877 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:26.877 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.808053+0000", 2024-05-08T08:52:26.877 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7957e-05 2024-05-08T08:52:26.878 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.878 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.878 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:26.878 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.808098+0000", 2024-05-08T08:52:26.878 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4616000000000001e-05 2024-05-08T08:52:26.878 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.878 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:26.878 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.878 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.878 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.878 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4516.0:5108 3.5 3:a38a8548:::rbd_object_map.1140ff5ddfe9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-05-08T08:52:26.878 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:21.796398+0000", 2024-05-08T08:52:26.878 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.0149812210000002, 2024-05-08T08:52:26.878 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.011820819, 2024-05-08T08:52:26.878 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:26.879 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:26.879 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:26.879 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:26.879 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4516", 2024-05-08T08:52:26.879 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/1777277669", 2024-05-08T08:52:26.879 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5108 2024-05-08T08:52:26.879 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.879 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:26.879 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.879 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:26.879 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.796398+0000", 2024-05-08T08:52:26.879 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:26.879 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.879 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.879 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:26.879 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.796398+0000", 2024-05-08T08:52:26.880 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:26.880 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.880 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.880 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:26.880 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.796398+0000", 2024-05-08T08:52:26.880 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2000000000000001e-07 2024-05-08T08:52:26.880 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.880 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.880 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:26.880 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.796400+0000", 2024-05-08T08:52:26.880 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.889e-06 2024-05-08T08:52:26.880 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.880 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.880 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:26.880 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.796403+0000", 2024-05-08T08:52:26.881 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8959999999999999e-06 2024-05-08T08:52:26.881 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.881 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.881 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:26.881 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.796405+0000", 2024-05-08T08:52:26.881 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4059999999999999e-06 2024-05-08T08:52:26.881 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.881 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.881 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:26.881 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.806286+0000", 2024-05-08T08:52:26.881 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0098805309999999997 2024-05-08T08:52:26.881 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.881 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.881 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:26.881 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.806363+0000", 2024-05-08T08:52:26.882 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7164999999999997e-05 2024-05-08T08:52:26.882 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.882 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.882 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 2", 2024-05-08T08:52:26.882 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.806572+0000", 2024-05-08T08:52:26.882 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020897099999999999 2024-05-08T08:52:26.882 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.882 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.882 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:52:26.882 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.807995+0000", 2024-05-08T08:52:26.882 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0014230410000000001 2024-05-08T08:52:26.882 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.882 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.883 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:26.883 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.808145+0000", 2024-05-08T08:52:26.883 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015046999999999999 2024-05-08T08:52:26.883 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.883 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.883 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:26.883 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.808180+0000", 2024-05-08T08:52:26.883 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4499999999999998e-05 2024-05-08T08:52:26.883 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.883 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.883 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:26.883 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.808218+0000", 2024-05-08T08:52:26.883 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8729999999999997e-05 2024-05-08T08:52:26.883 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.884 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:26.884 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.884 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.884 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.884 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4516.0:5109 3.5 3:a38a8548:::rbd_object_map.1140ff5ddfe9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-05-08T08:52:26.884 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:21.796408+0000", 2024-05-08T08:52:26.884 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.0149710909999996, 2024-05-08T08:52:26.884 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.01328818, 2024-05-08T08:52:26.884 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:26.884 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:26.884 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:26.884 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:26.884 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4516", 2024-05-08T08:52:26.884 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/1777277669", 2024-05-08T08:52:26.885 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5109 2024-05-08T08:52:26.885 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.885 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:26.885 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.885 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:26.885 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.796408+0000", 2024-05-08T08:52:26.885 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:26.885 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.885 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.885 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:26.885 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.796408+0000", 2024-05-08T08:52:26.885 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:26.885 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.886 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.886 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:26.886 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.796408+0000", 2024-05-08T08:52:26.886 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.11e-07 2024-05-08T08:52:26.886 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.886 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.886 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:26.886 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.796410+0000", 2024-05-08T08:52:26.886 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5719999999999999e-06 2024-05-08T08:52:26.886 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.886 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.886 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:26.886 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.796412+0000", 2024-05-08T08:52:26.886 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8770000000000002e-06 2024-05-08T08:52:26.887 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.887 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.887 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:26.887 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.796415+0000", 2024-05-08T08:52:26.887 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4619999999999999e-06 2024-05-08T08:52:26.887 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.887 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.887 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:26.887 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.806987+0000", 2024-05-08T08:52:26.887 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010572109999999999 2024-05-08T08:52:26.887 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.887 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.887 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:26.887 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.807071+0000", 2024-05-08T08:52:26.887 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3554999999999995e-05 2024-05-08T08:52:26.888 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.888 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.888 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 2", 2024-05-08T08:52:26.888 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.807458+0000", 2024-05-08T08:52:26.888 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00038749200000000002 2024-05-08T08:52:26.888 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.888 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.888 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:52:26.888 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.809566+0000", 2024-05-08T08:52:26.888 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0021078870000000001 2024-05-08T08:52:26.888 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.888 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.888 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:26.888 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.809622+0000", 2024-05-08T08:52:26.889 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5905000000000001e-05 2024-05-08T08:52:26.889 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.889 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.889 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:26.889 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.809659+0000", 2024-05-08T08:52:26.889 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7626999999999997e-05 2024-05-08T08:52:26.889 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.889 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.889 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:26.889 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.809696+0000", 2024-05-08T08:52:26.889 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6482000000000001e-05 2024-05-08T08:52:26.889 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.889 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:26.889 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.890 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.890 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.890 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4516.0:5110 3.5 3:a38a8548:::rbd_object_map.1140ff5ddfe9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-05-08T08:52:26.890 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:21.796418+0000", 2024-05-08T08:52:26.890 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.0149609780000004, 2024-05-08T08:52:26.890 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.014988794999999999, 2024-05-08T08:52:26.890 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:26.890 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:26.890 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:26.890 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:26.890 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4516", 2024-05-08T08:52:26.890 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/1777277669", 2024-05-08T08:52:26.890 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5110 2024-05-08T08:52:26.890 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.891 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:26.891 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.891 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:26.891 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.796418+0000", 2024-05-08T08:52:26.891 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:26.891 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.891 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.891 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:26.891 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.796418+0000", 2024-05-08T08:52:26.891 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:26.891 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.891 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.891 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:26.891 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.796418+0000", 2024-05-08T08:52:26.891 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2000000000000001e-07 2024-05-08T08:52:26.892 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.892 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.892 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:26.892 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.796419+0000", 2024-05-08T08:52:26.892 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.31e-06 2024-05-08T08:52:26.892 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.892 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.892 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:26.892 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.796422+0000", 2024-05-08T08:52:26.892 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8490000000000002e-06 2024-05-08T08:52:26.892 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.892 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.892 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:26.892 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.796425+0000", 2024-05-08T08:52:26.893 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3649999999999998e-06 2024-05-08T08:52:26.893 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.893 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.893 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:26.893 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.808255+0000", 2024-05-08T08:52:26.893 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.01183009 2024-05-08T08:52:26.893 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.893 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.893 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:26.893 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.808322+0000", 2024-05-08T08:52:26.893 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7321000000000001e-05 2024-05-08T08:52:26.893 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.893 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.893 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 2", 2024-05-08T08:52:26.894 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.808556+0000", 2024-05-08T08:52:26.894 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00023366000000000001 2024-05-08T08:52:26.894 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.894 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.894 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:52:26.894 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.810382+0000", 2024-05-08T08:52:26.894 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00182614 2024-05-08T08:52:26.894 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.894 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.894 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:26.894 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.811309+0000", 2024-05-08T08:52:26.894 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00092687200000000003 2024-05-08T08:52:26.894 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.894 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.895 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:26.895 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.811352+0000", 2024-05-08T08:52:26.895 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3075999999999999e-05 2024-05-08T08:52:26.895 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.895 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.895 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:26.895 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.811407+0000", 2024-05-08T08:52:26.895 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4892000000000002e-05 2024-05-08T08:52:26.895 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.895 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:26.895 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.895 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.895 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.895 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4516.0:5111 3.5 3:a38a8548:::rbd_object_map.1140ff5ddfe9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-05-08T08:52:26.895 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:21.796427+0000", 2024-05-08T08:52:26.896 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.0149514540000002, 2024-05-08T08:52:26.896 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.015117148, 2024-05-08T08:52:26.896 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:26.896 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:26.896 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:26.896 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:26.896 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4516", 2024-05-08T08:52:26.896 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/1777277669", 2024-05-08T08:52:26.896 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5111 2024-05-08T08:52:26.896 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.896 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:26.896 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.896 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:26.897 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.796427+0000", 2024-05-08T08:52:26.897 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:26.897 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.897 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.897 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:26.897 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.796427+0000", 2024-05-08T08:52:26.897 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:26.897 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.897 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.897 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:26.897 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.796428+0000", 2024-05-08T08:52:26.897 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.41e-07 2024-05-08T08:52:26.897 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.897 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.898 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:26.898 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.796429+0000", 2024-05-08T08:52:26.898 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5689999999999999e-06 2024-05-08T08:52:26.898 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.898 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.898 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:26.898 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.796432+0000", 2024-05-08T08:52:26.898 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9799999999999998e-06 2024-05-08T08:52:26.898 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.898 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.898 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:26.898 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.796434+0000", 2024-05-08T08:52:26.898 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3070000000000001e-06 2024-05-08T08:52:26.898 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.899 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.899 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:26.899 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.808898+0000", 2024-05-08T08:52:26.899 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.01246307 2024-05-08T08:52:26.899 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.899 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.899 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:26.899 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.808979+0000", 2024-05-08T08:52:26.899 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1306999999999999e-05 2024-05-08T08:52:26.899 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.899 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.899 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 2", 2024-05-08T08:52:26.899 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.809269+0000", 2024-05-08T08:52:26.899 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00029058600000000001 2024-05-08T08:52:26.900 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.900 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.900 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:52:26.900 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.811416+0000", 2024-05-08T08:52:26.900 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0021466580000000001 2024-05-08T08:52:26.900 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.900 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.900 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:26.900 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.811458+0000", 2024-05-08T08:52:26.900 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1801000000000002e-05 2024-05-08T08:52:26.900 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.900 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.900 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:26.900 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.811507+0000", 2024-05-08T08:52:26.900 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8863000000000003e-05 2024-05-08T08:52:26.901 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.901 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.901 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:26.901 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.811545+0000", 2024-05-08T08:52:26.901 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7765999999999997e-05 2024-05-08T08:52:26.901 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.901 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:26.901 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.901 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.901 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.901 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4516.0:5112 3.5 3:a38a8548:::rbd_object_map.1140ff5ddfe9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-05-08T08:52:26.901 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:21.796437+0000", 2024-05-08T08:52:26.901 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.0149415890000002, 2024-05-08T08:52:26.901 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.015222421, 2024-05-08T08:52:26.901 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:26.901 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:26.902 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:26.902 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:26.902 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4516", 2024-05-08T08:52:26.902 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/1777277669", 2024-05-08T08:52:26.902 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5112 2024-05-08T08:52:26.902 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.902 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:26.902 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.902 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:26.902 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.796437+0000", 2024-05-08T08:52:26.902 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:26.902 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.902 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.903 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:26.903 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.796437+0000", 2024-05-08T08:52:26.903 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:26.903 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.903 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.903 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:26.903 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.796437+0000", 2024-05-08T08:52:26.903 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0900000000000001e-07 2024-05-08T08:52:26.903 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.903 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.903 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:26.903 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.796439+0000", 2024-05-08T08:52:26.903 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4079999999999999e-06 2024-05-08T08:52:26.903 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.903 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.904 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:26.904 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.796442+0000", 2024-05-08T08:52:26.904 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0690000000000002e-06 2024-05-08T08:52:26.904 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.904 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.904 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:26.904 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.796444+0000", 2024-05-08T08:52:26.904 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4250000000000001e-06 2024-05-08T08:52:26.904 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.904 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.904 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:26.904 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.809740+0000", 2024-05-08T08:52:26.904 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013295694 2024-05-08T08:52:26.904 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.905 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.905 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:26.905 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.809836+0000", 2024-05-08T08:52:26.905 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5996999999999996e-05 2024-05-08T08:52:26.905 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.905 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.905 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 2", 2024-05-08T08:52:26.905 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.810075+0000", 2024-05-08T08:52:26.905 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00023876799999999999 2024-05-08T08:52:26.905 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.905 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.905 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:26.905 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.811592+0000", 2024-05-08T08:52:26.905 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015170540000000001 2024-05-08T08:52:26.905 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.906 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.906 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:52:26.906 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.811600+0000", 2024-05-08T08:52:26.906 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1729999999999995e-06 2024-05-08T08:52:26.906 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.906 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.906 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:26.906 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.811626+0000", 2024-05-08T08:52:26.906 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6108000000000001e-05 2024-05-08T08:52:26.906 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.906 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.906 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:26.906 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.811660+0000", 2024-05-08T08:52:26.906 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3516000000000002e-05 2024-05-08T08:52:26.907 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.907 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:26.907 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.907 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.907 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.907 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4516.0:5113 3.5 3:a38a8548:::rbd_object_map.1140ff5ddfe9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-05-08T08:52:26.907 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:21.796448+0000", 2024-05-08T08:52:26.907 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.0149306779999998, 2024-05-08T08:52:26.907 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.016200105999999999, 2024-05-08T08:52:26.907 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:26.907 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:26.907 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:26.907 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:26.907 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4516", 2024-05-08T08:52:26.907 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/1777277669", 2024-05-08T08:52:26.908 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5113 2024-05-08T08:52:26.908 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.908 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:26.908 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.908 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:26.908 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.796448+0000", 2024-05-08T08:52:26.908 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:26.908 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.908 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.908 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:26.908 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.796448+0000", 2024-05-08T08:52:26.908 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:26.908 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.908 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.909 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:26.909 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.796448+0000", 2024-05-08T08:52:26.909 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.36e-07 2024-05-08T08:52:26.909 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.909 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.909 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:26.909 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.796450+0000", 2024-05-08T08:52:26.909 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7579999999999999e-06 2024-05-08T08:52:26.909 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.909 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.909 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:26.909 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.796453+0000", 2024-05-08T08:52:26.909 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.892e-06 2024-05-08T08:52:26.909 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.909 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.909 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:26.910 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.796455+0000", 2024-05-08T08:52:26.910 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4540000000000001e-06 2024-05-08T08:52:26.910 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.910 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.910 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:26.910 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.810426+0000", 2024-05-08T08:52:26.910 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013970774 2024-05-08T08:52:26.910 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.910 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.910 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:26.910 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.810510+0000", 2024-05-08T08:52:26.910 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3548999999999996e-05 2024-05-08T08:52:26.910 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.910 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.910 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 2", 2024-05-08T08:52:26.911 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.810863+0000", 2024-05-08T08:52:26.911 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00035304299999999999 2024-05-08T08:52:26.911 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.911 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.911 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:26.911 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.812530+0000", 2024-05-08T08:52:26.911 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001666701 2024-05-08T08:52:26.911 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.911 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.911 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:52:26.911 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.812551+0000", 2024-05-08T08:52:26.911 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1515999999999999e-05 2024-05-08T08:52:26.911 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.911 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.912 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:26.912 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.812601+0000", 2024-05-08T08:52:26.912 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0144e-05 2024-05-08T08:52:26.912 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.912 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.912 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:26.912 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.812648+0000", 2024-05-08T08:52:26.912 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7039000000000001e-05 2024-05-08T08:52:26.912 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.912 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:26.912 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.912 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.912 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.912 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4516.0:5114 3.5 3:a38a8548:::rbd_object_map.1140ff5ddfe9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-05-08T08:52:26.912 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:21.796459+0000", 2024-05-08T08:52:26.913 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.0149196930000004, 2024-05-08T08:52:26.913 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.017232346999999999, 2024-05-08T08:52:26.913 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:26.913 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:26.913 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:26.913 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:26.913 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4516", 2024-05-08T08:52:26.913 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/1777277669", 2024-05-08T08:52:26.913 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5114 2024-05-08T08:52:26.913 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.913 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:26.913 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.913 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:26.913 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.796459+0000", 2024-05-08T08:52:26.913 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:26.914 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.914 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.914 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:26.914 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.796459+0000", 2024-05-08T08:52:26.914 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:26.914 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.914 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.914 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:26.914 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.796459+0000", 2024-05-08T08:52:26.914 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.22e-07 2024-05-08T08:52:26.914 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.914 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.914 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:26.914 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.796464+0000", 2024-05-08T08:52:26.915 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2030000000000002e-06 2024-05-08T08:52:26.915 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.915 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.915 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:26.915 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.796467+0000", 2024-05-08T08:52:26.915 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.264e-06 2024-05-08T08:52:26.915 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.915 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.915 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:26.915 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.796469+0000", 2024-05-08T08:52:26.915 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.396e-06 2024-05-08T08:52:26.915 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.915 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.915 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:26.916 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.811698+0000", 2024-05-08T08:52:26.916 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.015228587 2024-05-08T08:52:26.916 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.916 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.916 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:26.916 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.811775+0000", 2024-05-08T08:52:26.916 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7144999999999994e-05 2024-05-08T08:52:26.916 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.916 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.916 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 2", 2024-05-08T08:52:26.916 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.812111+0000", 2024-05-08T08:52:26.916 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00033609299999999998 2024-05-08T08:52:26.916 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.916 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.916 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:52:26.916 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.813372+0000", 2024-05-08T08:52:26.917 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001260702 2024-05-08T08:52:26.917 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.917 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.917 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:26.917 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.813616+0000", 2024-05-08T08:52:26.917 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00024411100000000001 2024-05-08T08:52:26.917 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.917 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.917 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:26.917 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.813661+0000", 2024-05-08T08:52:26.917 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5077999999999998e-05 2024-05-08T08:52:26.917 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:26.917 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:26.918 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:26.918 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:21.813691+0000", 2024-05-08T08:52:26.918 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0546000000000002e-05 2024-05-08T08:52:26.918 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.918 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:26.918 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.918 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:26.918 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:26.918 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:52:26.920 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:52:26.923+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:52:26.920 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:52:27.020 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:27.023+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:27.047 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:52:27.047 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:52:27.047 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:52:27.047 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:52:27.054 INFO:teuthology.orchestra.run.smithi196.stdout:truncating image image_client.0-clone4 from 0xd705a00 (overlap 0xd6e8400) to 0x0 2024-05-08T08:52:27.057 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:52:27.121 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:27.125+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:27.222 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:27.226+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:27.238 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:52:27.238 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:52:27.239 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:52:27.239 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:52:27.239 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:52:27.252 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:52:27.323 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:52:27.326+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:52:27.425 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:27.428+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:27.426 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:52:27.426 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:52:27.426 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:52:27.426 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:52:27.426 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.426 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:13251 3.12 3:4b07cb6d:::rbd_data.1140782590a5.0000000000000004:6 [read 677888~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:27.426 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.285367+0000", 2024-05-08T08:52:27.426 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.14222615, 2024-05-08T08:52:27.427 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.052666170999999998, 2024-05-08T08:52:27.427 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:27.427 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:27.427 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:27.427 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:27.427 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:27.427 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:27.427 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 13251 2024-05-08T08:52:27.427 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.427 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:27.427 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.427 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:27.427 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.285367+0000", 2024-05-08T08:52:27.428 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:27.428 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.428 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.428 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:27.428 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.285367+0000", 2024-05-08T08:52:27.428 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:27.428 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.428 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.428 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:27.428 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.285367+0000", 2024-05-08T08:52:27.428 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.44e-07 2024-05-08T08:52:27.428 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.428 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.428 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:27.428 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.285370+0000", 2024-05-08T08:52:27.429 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3020000000000002e-06 2024-05-08T08:52:27.429 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.429 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.429 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:27.429 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.285376+0000", 2024-05-08T08:52:27.429 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1310000000000003e-06 2024-05-08T08:52:27.429 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.429 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.429 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:27.429 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.285382+0000", 2024-05-08T08:52:27.429 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7209999999999999e-06 2024-05-08T08:52:27.429 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.429 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.429 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:27.429 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.337859+0000", 2024-05-08T08:52:27.429 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.052477348 2024-05-08T08:52:27.430 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.430 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.430 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:27.430 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.337934+0000", 2024-05-08T08:52:27.430 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5228999999999994e-05 2024-05-08T08:52:27.430 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.430 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.430 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:27.430 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.338033+0000", 2024-05-08T08:52:27.430 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8895999999999996e-05 2024-05-08T08:52:27.430 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:27.430 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:27.430 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:27.430 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.430 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.431 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:13253 3.12 3:4b07cb6d:::rbd_data.1140782590a5.0000000000000004:6 [read 694272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:27.431 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.285389+0000", 2024-05-08T08:52:27.431 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.142204514, 2024-05-08T08:52:27.431 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.052881773, 2024-05-08T08:52:27.431 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:27.431 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:27.431 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:27.431 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:27.431 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:27.431 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:27.431 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 13253 2024-05-08T08:52:27.431 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.431 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:27.431 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.431 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:27.431 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.285389+0000", 2024-05-08T08:52:27.432 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:27.432 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.432 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.432 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:27.432 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.285389+0000", 2024-05-08T08:52:27.432 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:27.432 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.432 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.432 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:27.432 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.285389+0000", 2024-05-08T08:52:27.432 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6499999999999999e-07 2024-05-08T08:52:27.432 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.432 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.432 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:27.432 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.285391+0000", 2024-05-08T08:52:27.432 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3369999999999998e-06 2024-05-08T08:52:27.432 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.433 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.433 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:27.433 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.285398+0000", 2024-05-08T08:52:27.433 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3929999999999996e-06 2024-05-08T08:52:27.433 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.433 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.433 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:27.433 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.285402+0000", 2024-05-08T08:52:27.433 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0489999999999997e-06 2024-05-08T08:52:27.433 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.433 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.433 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:27.433 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.338080+0000", 2024-05-08T08:52:27.433 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.052677753000000001 2024-05-08T08:52:27.433 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.433 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.434 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:27.434 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.338163+0000", 2024-05-08T08:52:27.434 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3395999999999999e-05 2024-05-08T08:52:27.434 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.434 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.434 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:27.434 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.338270+0000", 2024-05-08T08:52:27.434 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010738000000000001 2024-05-08T08:52:27.434 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:27.434 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:27.434 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:27.434 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.434 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.434 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:13255 3.12 3:4b07cb6d:::rbd_data.1140782590a5.0000000000000004:6 [read 710656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:27.435 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.285426+0000", 2024-05-08T08:52:27.435 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.14216722500000001, 2024-05-08T08:52:27.435 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.053065543999999999, 2024-05-08T08:52:27.435 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:27.435 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:27.435 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:27.435 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:27.435 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:27.435 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:27.435 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 13255 2024-05-08T08:52:27.435 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.435 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:27.435 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.435 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:27.435 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.285426+0000", 2024-05-08T08:52:27.436 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:27.436 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.436 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.436 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:27.436 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.285426+0000", 2024-05-08T08:52:27.436 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:27.436 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.436 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.436 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:27.436 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.285426+0000", 2024-05-08T08:52:27.436 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0200000000000002e-07 2024-05-08T08:52:27.436 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.436 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.436 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:27.437 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.285429+0000", 2024-05-08T08:52:27.437 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.5799999999999999e-06 2024-05-08T08:52:27.437 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.437 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.437 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:27.437 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.285436+0000", 2024-05-08T08:52:27.437 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8630000000000002e-06 2024-05-08T08:52:27.437 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.437 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.437 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:27.437 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.285441+0000", 2024-05-08T08:52:27.437 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4979999999999997e-06 2024-05-08T08:52:27.437 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.438 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.438 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:27.438 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.338305+0000", 2024-05-08T08:52:27.438 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.052863922000000001 2024-05-08T08:52:27.438 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.438 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.438 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:27.438 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.338386+0000", 2024-05-08T08:52:27.438 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0877999999999994e-05 2024-05-08T08:52:27.438 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.438 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.438 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:27.438 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.338491+0000", 2024-05-08T08:52:27.438 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000105301 2024-05-08T08:52:27.439 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:27.439 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:27.439 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:27.439 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.439 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.439 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:13256 3.12 3:4b07cb6d:::rbd_data.1140782590a5.0000000000000004:6 [read 727040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:27.439 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.285449+0000", 2024-05-08T08:52:27.439 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.14214368399999999, 2024-05-08T08:52:27.439 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.053250638000000003, 2024-05-08T08:52:27.439 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:27.439 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:27.439 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:27.439 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:27.439 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:27.440 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:27.440 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 13256 2024-05-08T08:52:27.440 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.440 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:27.440 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.440 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:27.440 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.285449+0000", 2024-05-08T08:52:27.440 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:27.440 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.440 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.440 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:27.440 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.285449+0000", 2024-05-08T08:52:27.440 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:27.440 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.441 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.441 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:27.441 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.285450+0000", 2024-05-08T08:52:27.441 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8999999999999997e-07 2024-05-08T08:52:27.441 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.441 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.441 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:27.441 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.285452+0000", 2024-05-08T08:52:27.441 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.588e-06 2024-05-08T08:52:27.441 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.441 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.441 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:27.441 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.285459+0000", 2024-05-08T08:52:27.441 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.37e-06 2024-05-08T08:52:27.441 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.442 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.442 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:27.442 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.285464+0000", 2024-05-08T08:52:27.442 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.135e-06 2024-05-08T08:52:27.442 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.442 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.442 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:27.442 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.338522+0000", 2024-05-08T08:52:27.442 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.053057578000000001 2024-05-08T08:52:27.442 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.442 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.442 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:27.442 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.338608+0000", 2024-05-08T08:52:27.442 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.6249999999999996e-05 2024-05-08T08:52:27.443 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.443 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.443 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:27.443 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.338700+0000", 2024-05-08T08:52:27.443 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2226999999999996e-05 2024-05-08T08:52:27.443 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:27.443 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:27.443 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:27.443 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.443 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.443 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:13257 3.12 3:4b07cb6d:::rbd_data.1140782590a5.0000000000000004:6 [read 743424~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:27.443 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.285471+0000", 2024-05-08T08:52:27.443 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.14212197200000001, 2024-05-08T08:52:27.443 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.053429597000000002, 2024-05-08T08:52:27.443 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:27.444 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:27.444 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:27.444 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:27.444 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:27.444 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:27.444 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 13257 2024-05-08T08:52:27.444 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.444 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:27.444 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.444 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:27.444 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.285471+0000", 2024-05-08T08:52:27.444 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:27.444 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.444 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.445 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:27.445 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.285471+0000", 2024-05-08T08:52:27.445 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:27.445 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.445 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.445 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:27.445 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.286967+0000", 2024-05-08T08:52:27.445 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001496364 2024-05-08T08:52:27.445 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.445 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.445 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:27.445 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.286972+0000", 2024-05-08T08:52:27.445 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.228e-06 2024-05-08T08:52:27.445 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.446 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.446 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:27.446 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.286983+0000", 2024-05-08T08:52:27.446 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.081e-05 2024-05-08T08:52:27.446 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.446 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.446 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:27.446 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.286989+0000", 2024-05-08T08:52:27.446 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8759999999999999e-06 2024-05-08T08:52:27.446 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.446 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.446 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:27.446 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.338740+0000", 2024-05-08T08:52:27.446 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.051750610000000002 2024-05-08T08:52:27.446 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.447 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.447 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:27.447 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.338811+0000", 2024-05-08T08:52:27.447 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1489000000000006e-05 2024-05-08T08:52:27.447 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.447 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.447 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:27.447 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.338901+0000", 2024-05-08T08:52:27.447 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.9220000000000003e-05 2024-05-08T08:52:27.447 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:27.447 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:27.447 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:27.447 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.447 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.447 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:13264 3.12 3:4b07cb6d:::rbd_data.1140782590a5.0000000000000004:6 [read 808960~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:27.448 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.287077+0000", 2024-05-08T08:52:27.448 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.14051646500000001, 2024-05-08T08:52:27.448 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.052612976999999998, 2024-05-08T08:52:27.448 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:27.448 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:27.448 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:27.448 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:27.448 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:27.448 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:27.448 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 13264 2024-05-08T08:52:27.448 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.448 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:27.448 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.448 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:27.448 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287077+0000", 2024-05-08T08:52:27.449 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:27.449 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.449 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.449 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:27.449 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287077+0000", 2024-05-08T08:52:27.449 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:27.449 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.449 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.449 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:27.449 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287077+0000", 2024-05-08T08:52:27.449 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3500000000000002e-07 2024-05-08T08:52:27.449 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.449 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.449 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:27.449 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287079+0000", 2024-05-08T08:52:27.450 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3290000000000001e-06 2024-05-08T08:52:27.450 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.450 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.450 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:27.450 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287087+0000", 2024-05-08T08:52:27.450 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0520000000000002e-06 2024-05-08T08:52:27.450 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.450 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.450 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:27.450 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287093+0000", 2024-05-08T08:52:27.450 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1240000000000004e-06 2024-05-08T08:52:27.450 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.450 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.450 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:27.451 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.339531+0000", 2024-05-08T08:52:27.451 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.052438833999999997 2024-05-08T08:52:27.451 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.451 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.451 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:27.451 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.339602+0000", 2024-05-08T08:52:27.451 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0537999999999994e-05 2024-05-08T08:52:27.451 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.451 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.451 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:27.451 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.339690+0000", 2024-05-08T08:52:27.451 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7664999999999995e-05 2024-05-08T08:52:27.451 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:27.451 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:27.451 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:27.452 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.452 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.452 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:13266 3.12 3:4b07cb6d:::rbd_data.1140782590a5.0000000000000004:6 [read 825344~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:27.452 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.287099+0000", 2024-05-08T08:52:27.452 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.140493643, 2024-05-08T08:52:27.452 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.052762794000000002, 2024-05-08T08:52:27.452 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:27.452 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:27.452 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:27.452 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:27.452 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:27.452 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:27.452 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 13266 2024-05-08T08:52:27.452 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.452 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:27.453 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.453 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:27.453 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287099+0000", 2024-05-08T08:52:27.453 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:27.453 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.453 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.453 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:27.453 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287099+0000", 2024-05-08T08:52:27.453 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:27.453 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.453 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.453 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:27.453 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287100+0000", 2024-05-08T08:52:27.453 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.51e-07 2024-05-08T08:52:27.454 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.454 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.454 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:27.454 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287102+0000", 2024-05-08T08:52:27.454 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4269999999999998e-06 2024-05-08T08:52:27.454 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.454 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.454 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:27.454 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287110+0000", 2024-05-08T08:52:27.454 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.926e-06 2024-05-08T08:52:27.454 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.454 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.454 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:27.454 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287115+0000", 2024-05-08T08:52:27.455 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9280000000000001e-06 2024-05-08T08:52:27.455 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.455 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.455 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:27.455 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.339727+0000", 2024-05-08T08:52:27.455 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.052611611000000003 2024-05-08T08:52:27.455 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.455 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.455 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:27.455 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.339799+0000", 2024-05-08T08:52:27.455 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1830999999999997e-05 2024-05-08T08:52:27.455 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.455 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.455 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:27.456 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.339862+0000", 2024-05-08T08:52:27.456 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3520000000000002e-05 2024-05-08T08:52:27.456 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:27.456 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:27.456 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:27.456 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.456 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.456 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:13268 3.12 3:4b07cb6d:::rbd_data.1140782590a5.0000000000000004:6 [read 841728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:27.456 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.287122+0000", 2024-05-08T08:52:27.456 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.14047100600000001, 2024-05-08T08:52:27.456 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.052917905000000001, 2024-05-08T08:52:27.456 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:27.456 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:27.456 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:27.457 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:27.457 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:27.457 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:27.457 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 13268 2024-05-08T08:52:27.457 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.457 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:27.457 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.457 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:27.457 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287122+0000", 2024-05-08T08:52:27.457 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:27.457 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.457 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.457 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:27.457 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287122+0000", 2024-05-08T08:52:27.458 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:27.458 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.458 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.458 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:27.458 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287122+0000", 2024-05-08T08:52:27.458 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5100000000000001e-07 2024-05-08T08:52:27.458 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.458 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.458 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:27.458 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287125+0000", 2024-05-08T08:52:27.458 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2960000000000001e-06 2024-05-08T08:52:27.458 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.458 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.458 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:27.458 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287131+0000", 2024-05-08T08:52:27.459 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7059999999999998e-06 2024-05-08T08:52:27.459 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.459 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.459 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:27.459 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287136+0000", 2024-05-08T08:52:27.459 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9989999999999999e-06 2024-05-08T08:52:27.459 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.459 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.459 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:27.459 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.339887+0000", 2024-05-08T08:52:27.459 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.052751063000000001 2024-05-08T08:52:27.459 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.459 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.459 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:27.459 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.339936+0000", 2024-05-08T08:52:27.460 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8233999999999999e-05 2024-05-08T08:52:27.460 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.460 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.460 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:27.460 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.340040+0000", 2024-05-08T08:52:27.460 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000104256 2024-05-08T08:52:27.460 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:27.460 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:27.460 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:27.460 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.460 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.460 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:13270 3.12 3:4b07cb6d:::rbd_data.1140782590a5.0000000000000004:6 [read 858112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:27.460 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.287143+0000", 2024-05-08T08:52:27.460 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.140449871, 2024-05-08T08:52:27.460 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.053114280999999999, 2024-05-08T08:52:27.460 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:27.461 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:27.461 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:27.461 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:27.461 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:27.461 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:27.461 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 13270 2024-05-08T08:52:27.461 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.461 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:27.461 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.461 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:27.461 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287143+0000", 2024-05-08T08:52:27.461 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:27.461 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.461 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.462 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:27.462 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287143+0000", 2024-05-08T08:52:27.462 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:27.462 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.462 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.462 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:27.462 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287144+0000", 2024-05-08T08:52:27.462 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9200000000000002e-07 2024-05-08T08:52:27.462 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.462 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.462 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:27.462 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287146+0000", 2024-05-08T08:52:27.462 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.5059999999999998e-06 2024-05-08T08:52:27.462 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.462 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.463 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:27.463 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287152+0000", 2024-05-08T08:52:27.463 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2190000000000001e-06 2024-05-08T08:52:27.463 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.463 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.463 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:27.463 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287157+0000", 2024-05-08T08:52:27.463 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6909999999999996e-06 2024-05-08T08:52:27.463 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.463 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.463 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:27.463 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.340084+0000", 2024-05-08T08:52:27.463 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.052926829000000002 2024-05-08T08:52:27.463 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.463 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.464 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:27.464 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.340161+0000", 2024-05-08T08:52:27.464 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7541999999999997e-05 2024-05-08T08:52:27.464 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.464 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.464 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:27.464 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.340257+0000", 2024-05-08T08:52:27.464 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6101999999999996e-05 2024-05-08T08:52:27.464 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:27.464 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:27.464 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:27.464 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.464 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.464 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:13271 3.12 3:4b07cb6d:::rbd_data.1140782590a5.0000000000000004:6 [read 874496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:27.465 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.287165+0000", 2024-05-08T08:52:27.465 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.14042855400000001, 2024-05-08T08:52:27.465 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.053403058000000003, 2024-05-08T08:52:27.465 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:27.465 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:27.465 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:27.465 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:27.465 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:27.465 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:27.465 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 13271 2024-05-08T08:52:27.465 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.465 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:27.465 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.465 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:27.465 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287165+0000", 2024-05-08T08:52:27.465 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:27.466 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.466 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.466 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:27.466 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287165+0000", 2024-05-08T08:52:27.466 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:27.466 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.466 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.466 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:27.466 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287165+0000", 2024-05-08T08:52:27.466 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:52:27.466 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.466 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.466 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:27.466 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287167+0000", 2024-05-08T08:52:27.467 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3690000000000001e-06 2024-05-08T08:52:27.467 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.467 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.467 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:27.467 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287174+0000", 2024-05-08T08:52:27.467 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.601e-06 2024-05-08T08:52:27.467 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.467 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.467 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:27.467 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287179+0000", 2024-05-08T08:52:27.467 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6940000000000003e-06 2024-05-08T08:52:27.467 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.467 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.467 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:27.467 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.340302+0000", 2024-05-08T08:52:27.467 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.053123212000000003 2024-05-08T08:52:27.468 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.468 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.468 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:27.468 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.340384+0000", 2024-05-08T08:52:27.468 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.2406999999999999e-05 2024-05-08T08:52:27.468 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.468 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.468 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:27.468 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.340568+0000", 2024-05-08T08:52:27.468 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00018342099999999999 2024-05-08T08:52:27.468 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:27.468 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:27.468 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:27.468 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.469 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.469 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:13273 3.12 3:4b07cb6d:::rbd_data.1140782590a5.0000000000000004:6 [read 890880~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:27.469 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.287185+0000", 2024-05-08T08:52:27.469 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.140407853, 2024-05-08T08:52:27.469 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.053591804, 2024-05-08T08:52:27.469 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:27.469 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:27.469 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:27.469 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:27.469 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:27.469 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:27.469 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 13273 2024-05-08T08:52:27.469 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.469 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:27.469 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.470 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:27.470 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287185+0000", 2024-05-08T08:52:27.470 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:27.470 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.470 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.470 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:27.470 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287185+0000", 2024-05-08T08:52:27.470 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:27.470 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.470 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.470 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:27.470 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287186+0000", 2024-05-08T08:52:27.470 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6699999999999999e-07 2024-05-08T08:52:27.471 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.471 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.471 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:27.471 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287188+0000", 2024-05-08T08:52:27.471 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2089999999999999e-06 2024-05-08T08:52:27.471 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.471 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.471 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:27.471 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287194+0000", 2024-05-08T08:52:27.471 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3720000000000001e-06 2024-05-08T08:52:27.471 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.471 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.471 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:27.471 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287199+0000", 2024-05-08T08:52:27.471 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0440000000000003e-06 2024-05-08T08:52:27.472 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.472 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.472 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:27.472 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.340599+0000", 2024-05-08T08:52:27.472 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.053399874 2024-05-08T08:52:27.472 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.472 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.472 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:27.472 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.340682+0000", 2024-05-08T08:52:27.472 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3288999999999995e-05 2024-05-08T08:52:27.472 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.472 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.472 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:27.472 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.340777+0000", 2024-05-08T08:52:27.472 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4648999999999995e-05 2024-05-08T08:52:27.473 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:27.473 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:27.473 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:27.473 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.473 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.473 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:13274 3.12 3:4b07cb6d:::rbd_data.1140782590a5.0000000000000004:6 [read 907264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:27.473 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.287217+0000", 2024-05-08T08:52:27.473 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.14037591299999999, 2024-05-08T08:52:27.473 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.053784751999999998, 2024-05-08T08:52:27.473 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:27.473 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:27.473 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:27.473 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:27.473 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:27.473 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:27.474 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 13274 2024-05-08T08:52:27.474 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.474 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:27.474 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.474 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:27.474 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287217+0000", 2024-05-08T08:52:27.474 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:27.474 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.474 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.474 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:27.474 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287217+0000", 2024-05-08T08:52:27.474 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:27.474 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.474 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.475 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:27.475 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287218+0000", 2024-05-08T08:52:27.475 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4700000000000001e-07 2024-05-08T08:52:27.475 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.475 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.475 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:27.475 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287220+0000", 2024-05-08T08:52:27.475 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6649999999999999e-06 2024-05-08T08:52:27.475 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.475 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.475 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:27.475 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287228+0000", 2024-05-08T08:52:27.475 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1550000000000001e-06 2024-05-08T08:52:27.475 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.475 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.476 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:27.476 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287233+0000", 2024-05-08T08:52:27.476 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2190000000000003e-06 2024-05-08T08:52:27.476 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.476 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.476 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:27.476 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.340811+0000", 2024-05-08T08:52:27.476 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.053578308999999998 2024-05-08T08:52:27.476 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.476 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.476 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:27.476 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.340899+0000", 2024-05-08T08:52:27.476 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.8109000000000006e-05 2024-05-08T08:52:27.476 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.477 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.477 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:27.477 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.341002+0000", 2024-05-08T08:52:27.477 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000102748 2024-05-08T08:52:27.477 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:27.477 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:27.477 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:27.477 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.477 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.477 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:13275 3.12 3:4b07cb6d:::rbd_data.1140782590a5.0000000000000004:6 [read 923648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:27.477 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.287240+0000", 2024-05-08T08:52:27.477 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.14035271499999999, 2024-05-08T08:52:27.477 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.053965209, 2024-05-08T08:52:27.477 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:27.477 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:27.478 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:27.478 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:27.478 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:27.478 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:27.478 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 13275 2024-05-08T08:52:27.478 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.478 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:27.478 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.478 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:27.478 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287240+0000", 2024-05-08T08:52:27.478 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:27.478 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.478 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.478 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:27.479 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287240+0000", 2024-05-08T08:52:27.479 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:27.479 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.479 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.479 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:27.479 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287241+0000", 2024-05-08T08:52:27.479 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5400000000000002e-07 2024-05-08T08:52:27.479 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.479 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.479 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:27.479 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287243+0000", 2024-05-08T08:52:27.479 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.278e-06 2024-05-08T08:52:27.479 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.479 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.480 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:27.480 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287249+0000", 2024-05-08T08:52:27.480 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8769999999999998e-06 2024-05-08T08:52:27.480 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.480 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.480 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:27.480 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287254+0000", 2024-05-08T08:52:27.480 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3000000000000001e-06 2024-05-08T08:52:27.480 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.480 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.480 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:27.480 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.341042+0000", 2024-05-08T08:52:27.480 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.053787408000000002 2024-05-08T08:52:27.480 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.481 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.481 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:27.481 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.341114+0000", 2024-05-08T08:52:27.481 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2194999999999996e-05 2024-05-08T08:52:27.481 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.481 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.481 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:27.481 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.341206+0000", 2024-05-08T08:52:27.481 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1697e-05 2024-05-08T08:52:27.481 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:27.481 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:27.481 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:27.481 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.481 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.481 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:13276 3.12 3:4b07cb6d:::rbd_data.1140782590a5.0000000000000004:6 [read 940032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:27.482 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.287262+0000", 2024-05-08T08:52:27.482 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.140331434, 2024-05-08T08:52:27.482 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.054174808999999997, 2024-05-08T08:52:27.482 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:27.482 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:27.482 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:27.482 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:27.482 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:27.482 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:27.482 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 13276 2024-05-08T08:52:27.482 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.482 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:27.482 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.482 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:27.482 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287262+0000", 2024-05-08T08:52:27.483 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:27.483 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.483 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.483 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:27.483 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287262+0000", 2024-05-08T08:52:27.483 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:27.483 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.483 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.483 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:27.483 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287262+0000", 2024-05-08T08:52:27.483 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7300000000000002e-07 2024-05-08T08:52:27.483 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.483 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.483 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:27.483 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287264+0000", 2024-05-08T08:52:27.484 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3199999999999998e-06 2024-05-08T08:52:27.484 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.484 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.484 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:27.484 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287270+0000", 2024-05-08T08:52:27.484 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9569999999999999e-06 2024-05-08T08:52:27.484 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.484 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.484 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:27.484 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287275+0000", 2024-05-08T08:52:27.484 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1019999999999996e-06 2024-05-08T08:52:27.484 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.484 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.484 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:27.485 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.341245+0000", 2024-05-08T08:52:27.485 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.053969560999999999 2024-05-08T08:52:27.485 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.485 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.485 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:27.485 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.341328+0000", 2024-05-08T08:52:27.485 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3528000000000004e-05 2024-05-08T08:52:27.485 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.485 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.485 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:27.485 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.341436+0000", 2024-05-08T08:52:27.485 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010796799999999999 2024-05-08T08:52:27.485 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:27.485 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:27.486 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:27.486 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.486 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.486 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:13277 3.12 3:4b07cb6d:::rbd_data.1140782590a5.0000000000000004:6 [read 956416~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:27.486 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.287283+0000", 2024-05-08T08:52:27.486 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.14031037099999999, 2024-05-08T08:52:27.486 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.054381455000000002, 2024-05-08T08:52:27.486 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:27.486 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:27.486 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:27.486 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:27.486 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:27.486 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:27.486 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 13277 2024-05-08T08:52:27.486 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.487 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:27.487 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.487 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:27.487 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287283+0000", 2024-05-08T08:52:27.487 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:27.487 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.487 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.487 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:27.487 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287283+0000", 2024-05-08T08:52:27.487 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:27.487 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.487 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.487 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:27.488 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287283+0000", 2024-05-08T08:52:27.488 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9499999999999998e-07 2024-05-08T08:52:27.488 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.488 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.488 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:27.488 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287290+0000", 2024-05-08T08:52:27.488 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6080000000000001e-06 2024-05-08T08:52:27.488 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.488 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.488 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:27.488 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287303+0000", 2024-05-08T08:52:27.488 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2962999999999999e-05 2024-05-08T08:52:27.488 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.488 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.488 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:27.489 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287309+0000", 2024-05-08T08:52:27.489 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3300000000000004e-06 2024-05-08T08:52:27.489 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.489 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.489 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:27.489 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.341468+0000", 2024-05-08T08:52:27.489 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.054158975999999998 2024-05-08T08:52:27.489 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.489 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.489 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:27.489 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.341557+0000", 2024-05-08T08:52:27.489 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.9154999999999996e-05 2024-05-08T08:52:27.489 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.489 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.489 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:27.489 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.341664+0000", 2024-05-08T08:52:27.490 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010702800000000001 2024-05-08T08:52:27.490 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:27.490 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:27.490 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:27.490 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.490 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.490 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:13278 3.12 3:4b07cb6d:::rbd_data.1140782590a5.0000000000000004:6 [read 972800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:27.490 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.287317+0000", 2024-05-08T08:52:27.490 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.14027585200000001, 2024-05-08T08:52:27.490 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.054498129999999999, 2024-05-08T08:52:27.490 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:27.490 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:27.490 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:27.490 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:27.491 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:27.491 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:27.491 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 13278 2024-05-08T08:52:27.491 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.491 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:27.491 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.491 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:27.491 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287317+0000", 2024-05-08T08:52:27.491 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:27.491 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.491 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.491 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:27.491 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287317+0000", 2024-05-08T08:52:27.491 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:27.492 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.492 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.492 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:27.492 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287318+0000", 2024-05-08T08:52:27.492 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.08e-07 2024-05-08T08:52:27.492 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.492 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.492 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:27.492 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287320+0000", 2024-05-08T08:52:27.492 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8490000000000002e-06 2024-05-08T08:52:27.492 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.492 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.492 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:27.493 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287327+0000", 2024-05-08T08:52:27.493 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5690000000000001e-06 2024-05-08T08:52:27.493 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.493 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.493 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:27.493 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287332+0000", 2024-05-08T08:52:27.493 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4389999999999999e-06 2024-05-08T08:52:27.493 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.493 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.493 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:27.493 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.341693+0000", 2024-05-08T08:52:27.493 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.054360968000000003 2024-05-08T08:52:27.493 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.493 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.493 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:27.494 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.341752+0000", 2024-05-08T08:52:27.494 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8171000000000003e-05 2024-05-08T08:52:27.494 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.494 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.494 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:27.494 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.341815+0000", 2024-05-08T08:52:27.494 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3726000000000006e-05 2024-05-08T08:52:27.494 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:27.494 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:27.494 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:27.494 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.494 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.494 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:13279 3.12 3:4b07cb6d:::rbd_data.1140782590a5.0000000000000004:6 [read 989184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:27.495 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.287344+0000", 2024-05-08T08:52:27.495 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.140249187, 2024-05-08T08:52:27.495 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.054616273999999999, 2024-05-08T08:52:27.495 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:27.495 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:27.495 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:27.495 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:27.495 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:27.495 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:27.495 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 13279 2024-05-08T08:52:27.495 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.495 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:27.495 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.495 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:27.496 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287344+0000", 2024-05-08T08:52:27.496 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:27.496 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.496 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.496 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:27.496 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287344+0000", 2024-05-08T08:52:27.496 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:27.496 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.496 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.496 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:27.496 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287344+0000", 2024-05-08T08:52:27.496 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9000000000000002e-07 2024-05-08T08:52:27.496 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.496 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.497 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:27.497 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287347+0000", 2024-05-08T08:52:27.497 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.43e-06 2024-05-08T08:52:27.497 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.497 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.497 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:27.497 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287354+0000", 2024-05-08T08:52:27.497 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0219999999999999e-06 2024-05-08T08:52:27.497 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.497 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.497 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:27.497 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287359+0000", 2024-05-08T08:52:27.497 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1919999999999996e-06 2024-05-08T08:52:27.497 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.497 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.498 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:27.498 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.341846+0000", 2024-05-08T08:52:27.498 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.054487155000000002 2024-05-08T08:52:27.498 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.498 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.498 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:27.498 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.341895+0000", 2024-05-08T08:52:27.498 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8464000000000002e-05 2024-05-08T08:52:27.498 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.498 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.498 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:27.498 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.341960+0000", 2024-05-08T08:52:27.498 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5621000000000001e-05 2024-05-08T08:52:27.498 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:27.499 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:27.499 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:27.499 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.499 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.499 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:13280 3.12 3:4b07cb6d:::rbd_data.1140782590a5.0000000000000004:6 [read 1005568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:27.499 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.287366+0000", 2024-05-08T08:52:27.499 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.14022694499999999, 2024-05-08T08:52:27.499 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.054873488999999998, 2024-05-08T08:52:27.499 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:27.499 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:27.499 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:27.499 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:27.499 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:27.500 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:27.500 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 13280 2024-05-08T08:52:27.500 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.500 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:27.500 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.500 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:27.500 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287366+0000", 2024-05-08T08:52:27.500 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:27.500 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.500 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.500 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:27.500 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287366+0000", 2024-05-08T08:52:27.500 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:27.500 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.501 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.501 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:27.501 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287367+0000", 2024-05-08T08:52:27.501 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8299999999999997e-07 2024-05-08T08:52:27.501 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.501 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.501 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:27.501 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287369+0000", 2024-05-08T08:52:27.501 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.075e-06 2024-05-08T08:52:27.501 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.501 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.501 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:27.501 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287378+0000", 2024-05-08T08:52:27.502 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3989999999999993e-06 2024-05-08T08:52:27.502 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.502 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.502 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:27.502 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287384+0000", 2024-05-08T08:52:27.502 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4160000000000003e-06 2024-05-08T08:52:27.502 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.502 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.502 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:27.502 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.342002+0000", 2024-05-08T08:52:27.502 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.054618501 2024-05-08T08:52:27.502 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.502 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.503 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:27.503 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.342106+0000", 2024-05-08T08:52:27.503 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010372799999999999 2024-05-08T08:52:27.503 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.503 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.503 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:27.503 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.342240+0000", 2024-05-08T08:52:27.503 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013378700000000001 2024-05-08T08:52:27.503 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:27.503 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:27.503 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:27.503 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.503 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.503 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:13281 3.12 3:4b07cb6d:::rbd_data.1140782590a5.0000000000000004:6 [read 1021952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:27.504 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.287421+0000", 2024-05-08T08:52:27.504 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.14017195900000001, 2024-05-08T08:52:27.504 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.055062391000000002, 2024-05-08T08:52:27.504 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:27.504 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:27.504 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:27.504 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:27.504 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:27.504 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:27.504 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 13281 2024-05-08T08:52:27.504 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.504 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:27.504 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.504 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:27.504 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287421+0000", 2024-05-08T08:52:27.505 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:27.505 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.505 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.505 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:27.505 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287421+0000", 2024-05-08T08:52:27.505 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:27.505 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.505 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.505 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:27.505 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287422+0000", 2024-05-08T08:52:27.505 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1200000000000003e-07 2024-05-08T08:52:27.505 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.505 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.505 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:27.505 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287425+0000", 2024-05-08T08:52:27.506 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.875e-06 2024-05-08T08:52:27.506 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.506 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.506 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:27.506 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287432+0000", 2024-05-08T08:52:27.506 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3540000000000002e-06 2024-05-08T08:52:27.506 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.506 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.506 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:27.506 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287437+0000", 2024-05-08T08:52:27.506 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.429e-06 2024-05-08T08:52:27.506 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.506 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.506 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:27.506 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.342283+0000", 2024-05-08T08:52:27.507 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.054845452000000003 2024-05-08T08:52:27.507 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.507 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.507 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:27.507 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.342360+0000", 2024-05-08T08:52:27.507 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.6686000000000004e-05 2024-05-08T08:52:27.507 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.507 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.507 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:27.507 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.342484+0000", 2024-05-08T08:52:27.507 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012398300000000001 2024-05-08T08:52:27.507 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:27.507 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:27.507 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:27.507 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.508 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.508 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:13282 3.12 3:4b07cb6d:::rbd_data.1140782590a5.0000000000000004:6 [read 1038336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:27.508 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.287446+0000", 2024-05-08T08:52:27.508 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.140147093, 2024-05-08T08:52:27.508 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.055320428999999997, 2024-05-08T08:52:27.508 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:27.508 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:27.508 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:27.508 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:27.508 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:27.508 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:27.508 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 13282 2024-05-08T08:52:27.508 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.508 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:27.508 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.509 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:27.509 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287446+0000", 2024-05-08T08:52:27.509 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:27.509 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.509 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.509 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:27.509 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287446+0000", 2024-05-08T08:52:27.509 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:27.509 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.509 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.509 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:27.509 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287447+0000", 2024-05-08T08:52:27.509 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1399999999999997e-07 2024-05-08T08:52:27.509 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.509 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.510 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:27.510 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287449+0000", 2024-05-08T08:52:27.510 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4590000000000001e-06 2024-05-08T08:52:27.510 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.510 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.510 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:27.510 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287455+0000", 2024-05-08T08:52:27.510 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.246e-06 2024-05-08T08:52:27.510 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.510 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.510 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:27.510 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.287461+0000", 2024-05-08T08:52:27.510 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5040000000000002e-06 2024-05-08T08:52:27.510 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.511 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.511 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:27.511 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.342531+0000", 2024-05-08T08:52:27.511 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.055070024000000002 2024-05-08T08:52:27.511 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.511 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.511 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:27.511 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.342624+0000", 2024-05-08T08:52:27.511 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3105999999999999e-05 2024-05-08T08:52:27.511 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:27.511 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:27.511 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:27.511 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.342766+0000", 2024-05-08T08:52:27.511 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000142576 2024-05-08T08:52:27.511 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:27.512 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:27.512 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:27.512 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:27.512 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:27.512 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:52:27.513 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:52:27.557 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:52:27.560+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:52:27.626 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:27.630+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:27.676 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:52:27.677 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:52:27.677 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:52:27.677 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:52:27.686 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:52:27.727 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:27.730+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:27.828 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:52:27.831+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:52:27.857 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:52:27.857 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:52:27.857 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:52:27.858 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:52:27.858 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:52:27.871 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:52:27.929 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:27.932+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:27.977 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-05-08T08:52:27.977 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T08:52:27.978 INFO:tasks.thrashosds.thrasher:inject_pause on osd.1 2024-05-08T08:52:27.978 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-05-08T08:52:27.978 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-05-08T08:52:27.978 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok config set filestore_inject_stall 3 2024-05-08T08:52:27.993 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-05-08T08:52:28.029 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:52:28.033+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:52:28.060 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T08:52:28.070 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:52:28.070 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:52:28.070 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:52:28.071 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:52:28.071 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.071 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9944 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 749568~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:28.071 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.103900+0000", 2024-05-08T08:52:28.071 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.96834026699999998, 2024-05-08T08:52:28.071 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057007044, 2024-05-08T08:52:28.071 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:28.071 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:28.071 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:28.071 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:28.071 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:28.071 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:28.072 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9944 2024-05-08T08:52:28.072 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.072 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:28.072 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.072 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:28.072 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103900+0000", 2024-05-08T08:52:28.072 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:28.072 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.072 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.072 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:28.072 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103900+0000", 2024-05-08T08:52:28.072 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:28.072 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.073 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.073 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:28.073 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103900+0000", 2024-05-08T08:52:28.073 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6400000000000003e-07 2024-05-08T08:52:28.073 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.073 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.073 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:28.073 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103902+0000", 2024-05-08T08:52:28.073 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.858e-06 2024-05-08T08:52:28.073 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.073 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.073 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:28.073 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103907+0000", 2024-05-08T08:52:28.073 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3410000000000002e-06 2024-05-08T08:52:28.073 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.074 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.074 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:28.074 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103910+0000", 2024-05-08T08:52:28.074 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8540000000000001e-06 2024-05-08T08:52:28.074 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.074 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.074 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:28.074 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.160682+0000", 2024-05-08T08:52:28.074 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.056771822 2024-05-08T08:52:28.074 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.074 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.074 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:28.074 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.160777+0000", 2024-05-08T08:52:28.074 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4555999999999994e-05 2024-05-08T08:52:28.075 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.075 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.075 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:28.075 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.160907+0000", 2024-05-08T08:52:28.075 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000130149 2024-05-08T08:52:28.075 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:28.075 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:28.075 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:28.075 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.075 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.075 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9945 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 1441792~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:28.075 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.103914+0000", 2024-05-08T08:52:28.075 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.96832567400000003, 2024-05-08T08:52:28.075 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057211979000000003, 2024-05-08T08:52:28.076 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:28.076 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:28.076 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:28.076 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:28.076 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:28.076 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:28.076 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9945 2024-05-08T08:52:28.076 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.076 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:28.076 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.076 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:28.076 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103914+0000", 2024-05-08T08:52:28.076 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:28.076 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.076 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.077 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:28.077 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103914+0000", 2024-05-08T08:52:28.077 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:28.077 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.077 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.077 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:28.077 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103915+0000", 2024-05-08T08:52:28.077 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3099999999999999e-07 2024-05-08T08:52:28.077 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.077 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.077 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:28.077 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103916+0000", 2024-05-08T08:52:28.077 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.203e-06 2024-05-08T08:52:28.077 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.077 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.077 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:28.078 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103919+0000", 2024-05-08T08:52:28.078 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.117e-06 2024-05-08T08:52:28.078 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.078 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.078 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:28.078 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103921+0000", 2024-05-08T08:52:28.078 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4169999999999999e-06 2024-05-08T08:52:28.078 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.078 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.078 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:28.078 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.160959+0000", 2024-05-08T08:52:28.078 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057038074000000001 2024-05-08T08:52:28.078 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.078 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.079 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:28.079 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161041+0000", 2024-05-08T08:52:28.079 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1620999999999997e-05 2024-05-08T08:52:28.079 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.079 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.079 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:28.079 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161126+0000", 2024-05-08T08:52:28.079 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.5216000000000005e-05 2024-05-08T08:52:28.079 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:28.079 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:28.079 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:28.079 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.079 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.079 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9946 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 765952~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:28.079 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.103925+0000", 2024-05-08T08:52:28.079 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.96831530700000001, 2024-05-08T08:52:28.080 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057418770000000001, 2024-05-08T08:52:28.080 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:28.080 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:28.080 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:28.080 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:28.080 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:28.080 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:28.080 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9946 2024-05-08T08:52:28.080 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.080 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:28.080 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.080 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:28.080 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103925+0000", 2024-05-08T08:52:28.080 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:28.080 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.081 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.081 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:28.081 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103925+0000", 2024-05-08T08:52:28.081 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:28.081 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.081 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.081 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:28.081 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103925+0000", 2024-05-08T08:52:28.081 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9200000000000002e-07 2024-05-08T08:52:28.081 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.081 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.081 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:28.081 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103926+0000", 2024-05-08T08:52:28.081 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1620000000000001e-06 2024-05-08T08:52:28.081 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.082 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.082 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:28.082 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103929+0000", 2024-05-08T08:52:28.082 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9349999999999999e-06 2024-05-08T08:52:28.082 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.082 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.082 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:28.082 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103931+0000", 2024-05-08T08:52:28.082 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4099999999999998e-06 2024-05-08T08:52:28.082 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.082 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.082 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:28.082 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161161+0000", 2024-05-08T08:52:28.082 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057229176999999999 2024-05-08T08:52:28.082 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.083 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.083 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:28.083 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161235+0000", 2024-05-08T08:52:28.083 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4227000000000006e-05 2024-05-08T08:52:28.083 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.083 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.083 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:28.083 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161343+0000", 2024-05-08T08:52:28.083 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010856700000000001 2024-05-08T08:52:28.083 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:28.083 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:28.083 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:28.083 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.084 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.084 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9947 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 1458176~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:28.084 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.103935+0000", 2024-05-08T08:52:28.084 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.96830542900000005, 2024-05-08T08:52:28.084 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057541263000000002, 2024-05-08T08:52:28.084 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:28.084 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:28.084 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:28.084 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:28.084 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:28.084 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:28.084 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9947 2024-05-08T08:52:28.085 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.085 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:28.085 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.085 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:28.085 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103935+0000", 2024-05-08T08:52:28.085 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:28.085 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.085 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.085 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:28.085 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103935+0000", 2024-05-08T08:52:28.085 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:28.085 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.086 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.086 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:28.086 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103935+0000", 2024-05-08T08:52:28.086 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.11e-07 2024-05-08T08:52:28.086 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.086 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.086 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:28.086 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103940+0000", 2024-05-08T08:52:28.086 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1030000000000001e-06 2024-05-08T08:52:28.086 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.086 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.086 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:28.087 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103943+0000", 2024-05-08T08:52:28.087 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3869999999999999e-06 2024-05-08T08:52:28.087 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.087 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.087 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:28.087 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103951+0000", 2024-05-08T08:52:28.087 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.2150000000000001e-06 2024-05-08T08:52:28.087 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.087 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.087 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:28.087 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161377+0000", 2024-05-08T08:52:28.088 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057425925000000003 2024-05-08T08:52:28.088 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.088 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.088 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:28.088 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161431+0000", 2024-05-08T08:52:28.088 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3408000000000001e-05 2024-05-08T08:52:28.088 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.088 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.088 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:28.088 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161476+0000", 2024-05-08T08:52:28.088 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5014e-05 2024-05-08T08:52:28.088 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:28.089 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:28.089 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:28.089 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.089 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.089 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9948 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 782336~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:28.089 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.103965+0000", 2024-05-08T08:52:28.089 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.96827453900000005, 2024-05-08T08:52:28.089 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057746205000000002, 2024-05-08T08:52:28.089 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:28.089 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:28.089 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:28.090 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:28.090 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:28.090 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:28.090 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9948 2024-05-08T08:52:28.090 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.090 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:28.090 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.090 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:28.090 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103965+0000", 2024-05-08T08:52:28.090 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:28.090 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.090 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.091 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:28.091 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103965+0000", 2024-05-08T08:52:28.091 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:28.091 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.091 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.091 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:28.091 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103966+0000", 2024-05-08T08:52:28.091 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.44e-07 2024-05-08T08:52:28.091 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.091 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.091 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:28.091 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103968+0000", 2024-05-08T08:52:28.092 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4090000000000001e-06 2024-05-08T08:52:28.092 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.092 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.092 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:28.092 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103973+0000", 2024-05-08T08:52:28.092 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6820000000000002e-06 2024-05-08T08:52:28.092 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.092 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.092 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:28.092 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103976+0000", 2024-05-08T08:52:28.092 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.959e-06 2024-05-08T08:52:28.092 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.092 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.092 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:28.093 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161523+0000", 2024-05-08T08:52:28.093 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057547264000000001 2024-05-08T08:52:28.093 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.093 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.093 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:28.093 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161605+0000", 2024-05-08T08:52:28.093 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1854000000000006e-05 2024-05-08T08:52:28.093 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.093 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.093 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:28.093 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161712+0000", 2024-05-08T08:52:28.093 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000106493 2024-05-08T08:52:28.093 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:28.094 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:28.094 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:28.094 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.094 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.094 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9949 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 1474560~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:28.094 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.103980+0000", 2024-05-08T08:52:28.094 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.96825992000000005, 2024-05-08T08:52:28.094 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057852397, 2024-05-08T08:52:28.094 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:28.094 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:28.095 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:28.095 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:28.095 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:28.095 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:28.095 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9949 2024-05-08T08:52:28.095 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.095 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:28.095 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.095 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:28.095 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103980+0000", 2024-05-08T08:52:28.095 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:28.095 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.096 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.096 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:28.096 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103980+0000", 2024-05-08T08:52:28.096 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:28.096 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.096 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.096 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:28.096 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103980+0000", 2024-05-08T08:52:28.096 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.11e-07 2024-05-08T08:52:28.096 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.096 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.096 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:28.097 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103982+0000", 2024-05-08T08:52:28.097 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.269e-06 2024-05-08T08:52:28.097 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.097 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.097 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:28.097 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103985+0000", 2024-05-08T08:52:28.097 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0110000000000001e-06 2024-05-08T08:52:28.097 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.097 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.097 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:28.097 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103987+0000", 2024-05-08T08:52:28.097 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3020000000000002e-06 2024-05-08T08:52:28.097 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.098 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.098 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:28.098 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161744+0000", 2024-05-08T08:52:28.098 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057757115999999997 2024-05-08T08:52:28.098 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.098 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.098 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:28.098 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161791+0000", 2024-05-08T08:52:28.098 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6807000000000001e-05 2024-05-08T08:52:28.098 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.098 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.098 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:28.098 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161832+0000", 2024-05-08T08:52:28.098 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1681000000000003e-05 2024-05-08T08:52:28.099 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:28.099 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:28.099 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:28.099 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.099 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.099 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9950 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 798720~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:28.099 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.103990+0000", 2024-05-08T08:52:28.099 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.96824968499999997, 2024-05-08T08:52:28.099 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.058062890999999998, 2024-05-08T08:52:28.099 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:28.099 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:28.099 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:28.100 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:28.100 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:28.100 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:28.100 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9950 2024-05-08T08:52:28.100 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.100 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:28.100 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.100 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:28.100 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103990+0000", 2024-05-08T08:52:28.100 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:28.100 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.101 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.101 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:28.101 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103990+0000", 2024-05-08T08:52:28.101 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:28.101 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.101 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.101 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:28.101 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103991+0000", 2024-05-08T08:52:28.101 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1e-07 2024-05-08T08:52:28.101 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.101 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.101 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:28.101 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103992+0000", 2024-05-08T08:52:28.102 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.074e-06 2024-05-08T08:52:28.102 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.102 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.102 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:28.102 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103995+0000", 2024-05-08T08:52:28.102 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9529999999999999e-06 2024-05-08T08:52:28.102 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.102 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.102 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:28.102 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103997+0000", 2024-05-08T08:52:28.102 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3939999999999999e-06 2024-05-08T08:52:28.102 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.103 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.103 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:28.103 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161871+0000", 2024-05-08T08:52:28.103 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057873854000000002 2024-05-08T08:52:28.103 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.103 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.103 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:28.103 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161950+0000", 2024-05-08T08:52:28.103 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9433999999999999e-05 2024-05-08T08:52:28.103 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.103 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.103 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:28.103 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.162053+0000", 2024-05-08T08:52:28.103 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000102972 2024-05-08T08:52:28.103 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:28.104 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:28.104 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:28.104 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.104 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.104 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9951 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 1490944~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:28.104 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.104000+0000", 2024-05-08T08:52:28.104 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.96823988100000002, 2024-05-08T08:52:28.104 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.058313416999999999, 2024-05-08T08:52:28.104 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:28.104 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:28.104 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:28.104 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:28.104 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:28.104 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:28.105 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9951 2024-05-08T08:52:28.105 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.105 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:28.105 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.105 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:28.105 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.104000+0000", 2024-05-08T08:52:28.105 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:28.105 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.105 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.105 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:28.105 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.104000+0000", 2024-05-08T08:52:28.105 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:28.105 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.105 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.105 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:28.106 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.104962+0000", 2024-05-08T08:52:28.106 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00096230399999999998 2024-05-08T08:52:28.106 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.106 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.106 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:28.106 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.104978+0000", 2024-05-08T08:52:28.106 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5126999999999999e-05 2024-05-08T08:52:28.106 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.106 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.106 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:28.106 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.104987+0000", 2024-05-08T08:52:28.106 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8940000000000005e-06 2024-05-08T08:52:28.106 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.106 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.106 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:28.107 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.104994+0000", 2024-05-08T08:52:28.107 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5610000000000004e-06 2024-05-08T08:52:28.107 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.107 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.107 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:28.107 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.162114+0000", 2024-05-08T08:52:28.107 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057119717 2024-05-08T08:52:28.107 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.107 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.107 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:28.107 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.162214+0000", 2024-05-08T08:52:28.107 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.9982999999999995e-05 2024-05-08T08:52:28.107 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.107 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.107 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:28.108 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.162314+0000", 2024-05-08T08:52:28.108 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.9831e-05 2024-05-08T08:52:28.108 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:28.108 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:28.108 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:28.108 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.108 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.108 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9952 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 815104~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:28.108 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.105004+0000", 2024-05-08T08:52:28.108 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.96723562799999996, 2024-05-08T08:52:28.108 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057530855999999998, 2024-05-08T08:52:28.108 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:28.108 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:28.108 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:28.108 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:28.109 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:28.109 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:28.109 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9952 2024-05-08T08:52:28.109 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.109 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:28.109 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.109 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:28.109 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105004+0000", 2024-05-08T08:52:28.109 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:28.109 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.109 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.109 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:28.109 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105004+0000", 2024-05-08T08:52:28.109 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:28.109 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.110 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.110 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:28.110 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105005+0000", 2024-05-08T08:52:28.110 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9299999999999998e-07 2024-05-08T08:52:28.110 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.110 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.110 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:28.110 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105007+0000", 2024-05-08T08:52:28.110 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2979999999999999e-06 2024-05-08T08:52:28.110 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.110 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.110 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:28.110 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105017+0000", 2024-05-08T08:52:28.110 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5060000000000001e-06 2024-05-08T08:52:28.110 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.111 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.111 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:28.111 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105022+0000", 2024-05-08T08:52:28.111 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2619999999999998e-06 2024-05-08T08:52:28.111 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.111 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.111 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:28.111 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.162356+0000", 2024-05-08T08:52:28.111 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057334388999999999 2024-05-08T08:52:28.111 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.111 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.111 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:28.111 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.162437+0000", 2024-05-08T08:52:28.111 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0863000000000002e-05 2024-05-08T08:52:28.111 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.112 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.112 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:28.112 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.162535+0000", 2024-05-08T08:52:28.112 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8045000000000001e-05 2024-05-08T08:52:28.112 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:28.112 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:28.112 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:28.112 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.112 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.112 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9953 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 1507328~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:28.112 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.105064+0000", 2024-05-08T08:52:28.112 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.96717594100000004, 2024-05-08T08:52:28.112 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057721531999999999, 2024-05-08T08:52:28.112 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:28.112 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:28.112 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:28.113 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:28.113 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:28.113 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:28.113 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9953 2024-05-08T08:52:28.113 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.113 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:28.113 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.113 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:28.113 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105064+0000", 2024-05-08T08:52:28.113 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:28.113 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.113 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.113 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:28.113 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105064+0000", 2024-05-08T08:52:28.113 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:28.113 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.114 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.114 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:28.114 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105065+0000", 2024-05-08T08:52:28.114 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8800000000000002e-07 2024-05-08T08:52:28.114 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.114 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.114 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:28.114 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105067+0000", 2024-05-08T08:52:28.114 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7669999999999999e-06 2024-05-08T08:52:28.114 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.114 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.114 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:28.114 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105076+0000", 2024-05-08T08:52:28.114 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.5259999999999993e-06 2024-05-08T08:52:28.115 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.115 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.115 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:28.115 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105080+0000", 2024-05-08T08:52:28.115 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8e-06 2024-05-08T08:52:28.115 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.115 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.115 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:28.115 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.162596+0000", 2024-05-08T08:52:28.115 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057515822000000001 2024-05-08T08:52:28.115 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.115 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.116 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:28.116 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.162689+0000", 2024-05-08T08:52:28.116 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3258999999999996e-05 2024-05-08T08:52:28.116 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.116 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.116 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:28.116 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.162786+0000", 2024-05-08T08:52:28.116 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6769999999999997e-05 2024-05-08T08:52:28.116 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:28.116 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:28.116 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:28.116 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.117 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.117 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9954 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 1523712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:28.117 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.105088+0000", 2024-05-08T08:52:28.117 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.96715193200000005, 2024-05-08T08:52:28.117 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057925139, 2024-05-08T08:52:28.117 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:28.117 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:28.117 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:28.117 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:28.117 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:28.117 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:28.117 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9954 2024-05-08T08:52:28.117 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.118 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:28.118 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.118 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:28.118 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105088+0000", 2024-05-08T08:52:28.118 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:28.118 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.118 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.118 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:28.118 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105088+0000", 2024-05-08T08:52:28.118 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:28.118 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.118 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.118 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:28.118 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105089+0000", 2024-05-08T08:52:28.119 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9599999999999999e-07 2024-05-08T08:52:28.119 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.119 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.119 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:28.119 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105091+0000", 2024-05-08T08:52:28.119 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3649999999999998e-06 2024-05-08T08:52:28.119 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.119 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.119 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:28.119 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105107+0000", 2024-05-08T08:52:28.119 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6370000000000001e-05 2024-05-08T08:52:28.119 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.120 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.120 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:28.120 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105114+0000", 2024-05-08T08:52:28.120 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2190000000000001e-06 2024-05-08T08:52:28.120 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.120 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.120 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:28.120 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.162830+0000", 2024-05-08T08:52:28.120 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057716637000000001 2024-05-08T08:52:28.120 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.120 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.120 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:28.120 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.162924+0000", 2024-05-08T08:52:28.120 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3354000000000001e-05 2024-05-08T08:52:28.121 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.121 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.121 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:28.121 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.163013+0000", 2024-05-08T08:52:28.121 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.9697999999999994e-05 2024-05-08T08:52:28.121 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:28.121 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:28.121 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:28.121 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.121 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.121 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9955 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 831488~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:28.121 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.105124+0000", 2024-05-08T08:52:28.122 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.96711635799999995, 2024-05-08T08:52:28.122 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.058092618999999998, 2024-05-08T08:52:28.122 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:28.122 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:28.122 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:28.122 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:28.122 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:28.122 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:28.122 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9955 2024-05-08T08:52:28.122 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.122 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:28.122 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.122 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:28.122 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105124+0000", 2024-05-08T08:52:28.123 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:28.123 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.123 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.123 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:28.123 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105124+0000", 2024-05-08T08:52:28.123 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:28.123 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.123 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.123 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:28.123 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.106961+0000", 2024-05-08T08:52:28.123 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001837574 2024-05-08T08:52:28.123 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.124 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.124 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:28.124 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.106966+0000", 2024-05-08T08:52:28.124 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8439999999999997e-06 2024-05-08T08:52:28.124 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.124 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.124 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:28.124 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.106978+0000", 2024-05-08T08:52:28.124 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2330999999999999e-05 2024-05-08T08:52:28.124 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.124 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.124 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:28.125 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.106986+0000", 2024-05-08T08:52:28.125 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0439999999999996e-06 2024-05-08T08:52:28.125 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.125 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.125 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:28.125 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.163048+0000", 2024-05-08T08:52:28.125 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.056061182000000001 2024-05-08T08:52:28.125 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.125 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.125 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:28.125 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.163124+0000", 2024-05-08T08:52:28.125 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.6030000000000002e-05 2024-05-08T08:52:28.125 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.126 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.126 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:28.126 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.163216+0000", 2024-05-08T08:52:28.126 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2614000000000005e-05 2024-05-08T08:52:28.126 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:28.126 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:28.126 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:28.126 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.126 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.126 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9960 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 1589248~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:28.126 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.107116+0000", 2024-05-08T08:52:28.126 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.96512354, 2024-05-08T08:52:28.127 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.056889636, 2024-05-08T08:52:28.127 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:28.127 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:28.127 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:28.127 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:28.127 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:28.127 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:28.127 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9960 2024-05-08T08:52:28.127 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.127 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:28.127 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.127 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:28.127 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107116+0000", 2024-05-08T08:52:28.128 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:28.128 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.128 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.128 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:28.128 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107116+0000", 2024-05-08T08:52:28.128 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:28.128 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.128 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.128 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:28.128 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107117+0000", 2024-05-08T08:52:28.128 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4400000000000001e-07 2024-05-08T08:52:28.128 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.129 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.129 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:28.129 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107119+0000", 2024-05-08T08:52:28.129 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3750000000000001e-06 2024-05-08T08:52:28.129 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.129 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.129 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:28.129 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107126+0000", 2024-05-08T08:52:28.129 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3369999999999996e-06 2024-05-08T08:52:28.129 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.129 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.129 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:28.130 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107130+0000", 2024-05-08T08:52:28.130 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8620000000000002e-06 2024-05-08T08:52:28.130 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.130 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.130 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:28.130 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.163917+0000", 2024-05-08T08:52:28.130 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.056786634000000002 2024-05-08T08:52:28.130 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.130 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.130 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:28.130 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.163962+0000", 2024-05-08T08:52:28.131 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4521000000000003e-05 2024-05-08T08:52:28.131 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.131 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.131 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:28.131 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.164006+0000", 2024-05-08T08:52:28.131 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4462999999999998e-05 2024-05-08T08:52:28.131 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:28.131 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:28.131 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:28.131 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.131 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.131 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9961 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 864256~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:28.132 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.107138+0000", 2024-05-08T08:52:28.132 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.96510248099999996, 2024-05-08T08:52:28.132 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057108051999999999, 2024-05-08T08:52:28.132 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:28.132 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:28.132 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:28.132 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:28.132 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:28.132 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:28.132 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9961 2024-05-08T08:52:28.132 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.132 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:28.133 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.133 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:28.133 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107138+0000", 2024-05-08T08:52:28.133 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:28.133 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.133 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.133 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:28.133 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107138+0000", 2024-05-08T08:52:28.133 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:28.133 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.133 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.133 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:28.133 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107138+0000", 2024-05-08T08:52:28.134 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2399999999999999e-07 2024-05-08T08:52:28.134 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.134 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.134 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:28.134 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107140+0000", 2024-05-08T08:52:28.134 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1409999999999999e-06 2024-05-08T08:52:28.134 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.134 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.134 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:28.134 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107146+0000", 2024-05-08T08:52:28.134 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0639999999999999e-06 2024-05-08T08:52:28.134 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.135 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.135 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:28.135 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107151+0000", 2024-05-08T08:52:28.135 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.69e-06 2024-05-08T08:52:28.135 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.135 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.135 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:28.135 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.164051+0000", 2024-05-08T08:52:28.135 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.056900153000000002 2024-05-08T08:52:28.135 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.135 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.135 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:28.135 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.164136+0000", 2024-05-08T08:52:28.135 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.5086999999999994e-05 2024-05-08T08:52:28.135 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.135 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.136 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:28.136 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.164246+0000", 2024-05-08T08:52:28.136 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000109493 2024-05-08T08:52:28.136 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:28.136 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:28.136 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:28.136 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.136 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.136 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9962 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 1605632~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:28.136 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.107158+0000", 2024-05-08T08:52:28.136 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.96508240199999995, 2024-05-08T08:52:28.136 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057292698000000003, 2024-05-08T08:52:28.136 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:28.136 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:28.136 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:28.137 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:28.137 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:28.137 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:28.137 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9962 2024-05-08T08:52:28.137 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.137 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:28.137 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.137 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:28.137 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107158+0000", 2024-05-08T08:52:28.137 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:28.137 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.137 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.137 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:28.137 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107158+0000", 2024-05-08T08:52:28.137 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:28.137 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.137 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.138 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:28.138 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107158+0000", 2024-05-08T08:52:28.138 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8100000000000003e-07 2024-05-08T08:52:28.138 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.138 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.138 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:28.138 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107167+0000", 2024-05-08T08:52:28.138 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3079999999999997e-06 2024-05-08T08:52:28.138 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.138 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.138 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:28.138 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107175+0000", 2024-05-08T08:52:28.138 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2089999999999999e-06 2024-05-08T08:52:28.138 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.139 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.139 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:28.139 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107180+0000", 2024-05-08T08:52:28.139 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1120000000000004e-06 2024-05-08T08:52:28.139 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.139 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.139 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:28.139 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.164284+0000", 2024-05-08T08:52:28.139 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057104110999999999 2024-05-08T08:52:28.139 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.139 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.139 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:28.139 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.164370+0000", 2024-05-08T08:52:28.139 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.6549000000000001e-05 2024-05-08T08:52:28.139 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.139 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.140 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:28.140 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.164450+0000", 2024-05-08T08:52:28.140 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9827999999999995e-05 2024-05-08T08:52:28.140 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:28.140 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:28.140 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:28.140 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.140 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.140 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9963 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 880640~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:28.140 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.107188+0000", 2024-05-08T08:52:28.140 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.96505248399999999, 2024-05-08T08:52:28.140 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057560471000000002, 2024-05-08T08:52:28.140 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:28.140 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:28.140 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:28.140 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:28.141 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:28.141 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:28.141 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9963 2024-05-08T08:52:28.141 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.141 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:28.141 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.141 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:28.141 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107188+0000", 2024-05-08T08:52:28.141 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:28.141 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.141 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.141 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:28.141 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107188+0000", 2024-05-08T08:52:28.142 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:28.142 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.142 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.142 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:28.142 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.108964+0000", 2024-05-08T08:52:28.142 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001775993 2024-05-08T08:52:28.142 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.142 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.142 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:28.142 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.108969+0000", 2024-05-08T08:52:28.142 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1200000000000001e-06 2024-05-08T08:52:28.143 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.143 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.143 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:28.143 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.108981+0000", 2024-05-08T08:52:28.143 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2670000000000001e-05 2024-05-08T08:52:28.143 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.143 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.143 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:28.143 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.108989+0000", 2024-05-08T08:52:28.143 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3139999999999998e-06 2024-05-08T08:52:28.144 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.144 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.144 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:28.144 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.164504+0000", 2024-05-08T08:52:28.144 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.055515225000000001 2024-05-08T08:52:28.144 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.144 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.144 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:28.144 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.164606+0000", 2024-05-08T08:52:28.144 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010245 2024-05-08T08:52:28.144 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.144 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.144 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:28.145 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.164748+0000", 2024-05-08T08:52:28.145 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014169900000000001 2024-05-08T08:52:28.145 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:28.145 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:28.145 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:28.145 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.145 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.145 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9970 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 929792~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:28.145 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.109178+0000", 2024-05-08T08:52:28.145 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.96306220600000003, 2024-05-08T08:52:28.145 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057137975000000001, 2024-05-08T08:52:28.145 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:28.146 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:28.146 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:28.146 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:28.146 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:28.146 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:28.146 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9970 2024-05-08T08:52:28.146 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.146 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:28.146 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.146 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:28.146 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109178+0000", 2024-05-08T08:52:28.146 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:28.147 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.147 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.147 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:28.147 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109178+0000", 2024-05-08T08:52:28.147 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:28.147 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.147 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.147 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:28.147 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109178+0000", 2024-05-08T08:52:28.147 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7800000000000002e-07 2024-05-08T08:52:28.147 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.147 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.147 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:28.148 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109180+0000", 2024-05-08T08:52:28.148 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2199999999999999e-06 2024-05-08T08:52:28.148 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.148 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.148 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:28.148 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109186+0000", 2024-05-08T08:52:28.148 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9410000000000004e-06 2024-05-08T08:52:28.148 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.148 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.148 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:28.148 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109191+0000", 2024-05-08T08:52:28.149 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8729999999999998e-06 2024-05-08T08:52:28.149 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.149 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.149 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:28.149 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.166098+0000", 2024-05-08T08:52:28.149 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.056907124000000003 2024-05-08T08:52:28.149 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.149 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.149 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:28.149 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.166193+0000", 2024-05-08T08:52:28.149 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5018000000000004e-05 2024-05-08T08:52:28.149 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.150 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.150 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:28.150 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.166316+0000", 2024-05-08T08:52:28.150 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000122321 2024-05-08T08:52:28.150 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:28.150 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:28.150 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:28.150 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.150 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.150 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9971 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 1687552~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:28.150 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.109198+0000", 2024-05-08T08:52:28.151 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.96304152799999998, 2024-05-08T08:52:28.151 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057361875, 2024-05-08T08:52:28.151 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:28.151 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:28.151 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:28.151 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:28.151 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:28.151 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:28.151 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9971 2024-05-08T08:52:28.151 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.151 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:28.151 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.152 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:28.152 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109198+0000", 2024-05-08T08:52:28.152 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:28.152 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.152 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.152 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:28.152 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109198+0000", 2024-05-08T08:52:28.152 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:28.152 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.152 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.152 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:28.152 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109199+0000", 2024-05-08T08:52:28.153 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2800000000000002e-07 2024-05-08T08:52:28.153 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.153 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.153 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:28.153 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109208+0000", 2024-05-08T08:52:28.153 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4210000000000001e-06 2024-05-08T08:52:28.153 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.153 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.153 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:28.153 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109215+0000", 2024-05-08T08:52:28.153 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7209999999999997e-06 2024-05-08T08:52:28.153 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.154 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.154 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:28.154 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109220+0000", 2024-05-08T08:52:28.154 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.13e-06 2024-05-08T08:52:28.154 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.154 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.154 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:28.154 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.166353+0000", 2024-05-08T08:52:28.154 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057132693999999998 2024-05-08T08:52:28.154 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.154 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.155 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:28.155 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.166462+0000", 2024-05-08T08:52:28.155 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010940799999999999 2024-05-08T08:52:28.155 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.155 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.155 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:28.155 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.166560+0000", 2024-05-08T08:52:28.155 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8072999999999994e-05 2024-05-08T08:52:28.155 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:28.155 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:28.155 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:28.156 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.156 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.156 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9972 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 1703936~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:28.156 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.109228+0000", 2024-05-08T08:52:28.156 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.96301213600000002, 2024-05-08T08:52:28.156 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057541904999999997, 2024-05-08T08:52:28.156 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:28.156 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:28.156 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:28.156 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:28.157 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:28.157 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:28.157 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9972 2024-05-08T08:52:28.157 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.158 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:28.158 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.158 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:28.158 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109228+0000", 2024-05-08T08:52:28.158 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:28.158 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.158 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.158 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:28.158 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109228+0000", 2024-05-08T08:52:28.158 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:28.158 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.158 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.158 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:28.158 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109228+0000", 2024-05-08T08:52:28.158 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5699999999999998e-07 2024-05-08T08:52:28.159 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.159 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.159 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:28.159 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109231+0000", 2024-05-08T08:52:28.159 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2120000000000002e-06 2024-05-08T08:52:28.159 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.159 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.159 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:28.159 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109237+0000", 2024-05-08T08:52:28.159 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3099999999999997e-06 2024-05-08T08:52:28.159 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.159 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.159 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:28.159 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109243+0000", 2024-05-08T08:52:28.160 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7490000000000004e-06 2024-05-08T08:52:28.160 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.160 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.160 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:28.160 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.166609+0000", 2024-05-08T08:52:28.160 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057366372999999998 2024-05-08T08:52:28.160 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.160 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.160 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:28.160 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.166690+0000", 2024-05-08T08:52:28.160 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1426000000000003e-05 2024-05-08T08:52:28.160 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.160 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.160 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:28.160 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.166770+0000", 2024-05-08T08:52:28.161 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9377999999999998e-05 2024-05-08T08:52:28.161 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:28.161 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:28.161 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:28.161 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.161 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.161 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9973 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 946176~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:28.161 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.109249+0000", 2024-05-08T08:52:28.161 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.96299049400000003, 2024-05-08T08:52:28.161 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057760381, 2024-05-08T08:52:28.161 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:28.161 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:28.161 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:28.161 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:28.161 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:28.162 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:28.162 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9973 2024-05-08T08:52:28.162 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.162 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:28.162 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.162 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:28.162 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109249+0000", 2024-05-08T08:52:28.162 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:28.162 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.162 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.162 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:28.162 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109249+0000", 2024-05-08T08:52:28.162 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:28.162 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.163 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.163 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:28.163 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.110959+0000", 2024-05-08T08:52:28.163 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001709288 2024-05-08T08:52:28.163 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.163 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.163 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:28.163 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.110972+0000", 2024-05-08T08:52:28.163 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2719e-05 2024-05-08T08:52:28.163 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.163 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.163 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:28.163 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.110984+0000", 2024-05-08T08:52:28.163 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2001000000000001e-05 2024-05-08T08:52:28.163 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.163 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.164 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:28.164 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.110993+0000", 2024-05-08T08:52:28.164 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0880000000000001e-06 2024-05-08T08:52:28.164 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.164 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.164 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:28.164 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.166817+0000", 2024-05-08T08:52:28.164 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.055823971999999999 2024-05-08T08:52:28.164 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.164 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.164 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:28.164 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.166900+0000", 2024-05-08T08:52:28.164 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3838999999999995e-05 2024-05-08T08:52:28.164 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:28.164 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:28.165 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:28.165 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.167010+0000", 2024-05-08T08:52:28.165 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000109474 2024-05-08T08:52:28.165 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:28.165 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:28.165 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:28.165 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:28.165 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:28.165 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:52:28.167 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:52:28.168 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:52:28.168 INFO:teuthology.orchestra.run.smithi059.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-05-08T08:52:28.168 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:52:28.168 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:52:28.169+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:52:28.251 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:52:28.251 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:52:28.268 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:28.270+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:28.295 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:52:28.296 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:52:28.335 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:52:28.335 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:52:28.367 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:52:28.367 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:52:28.369 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:52:28.372+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:52:28.406 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:28.406 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:28.407 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:52:28.408 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:52:28.418 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:52:28.442 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:52:28.442 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:52:28.470 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:28.473+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:28.567 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:28.567 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:28.570 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:52:28.573+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:52:28.580 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:52:28.672 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:28.675+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:28.772 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:28.776+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:28.780 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:28.780 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:28.790 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:52:28.874 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:52:28.877+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:52:28.914 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:28.914 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:28.923 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:52:28.974 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:52:28.977+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:52:28.992 INFO:teuthology.orchestra.run.smithi059.stderr:noscrub is unset 2024-05-08T08:52:29.074 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:29.074 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:29.075 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:52:29.078+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:52:29.083 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:52:29.163 INFO:teuthology.orchestra.run.smithi196.stdout:checking clone #3, image image_client.0-clone3 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent4 2024-05-08T08:52:29.175 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:29.179+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:29.230 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:29.231 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:29.240 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:52:29.276 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:52:29.279+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:52:29.377 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:52:29.381+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:52:29.395 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:29.395 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:29.404 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:52:29.489 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:29.492+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:29.554 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:29.554 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:29.563 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:52:29.579 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:52:29.583+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:52:29.680 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:29.683+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:29.709 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:29.709 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:29.718 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:52:29.756 INFO:teuthology.orchestra.run.smithi196.stdout:902 punch from 0x8c5ce00 to 0x8c6ac00, (0xde00 bytes) 2024-05-08T08:52:29.781 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:52:29.784+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:52:29.862 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:29.863 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:29.871 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:52:29.882 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:52:29.886+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:52:29.976 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:29.976 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:29.986 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:52:29.987+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:52:29.987 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:52:30.084 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:52:30.088+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:52:30.092 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:30.092 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:30.101 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:52:30.185 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:52:30.188+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:52:30.217 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:30.217 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:30.226 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:52:30.310 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:52:30.313+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:52:30.376 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:30.376 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:30.384 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:52:30.387 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:30.390+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:30.461 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T08:52:30.512 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:52:30.515+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:52:30.536 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:30.536 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:30.545 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:52:30.588 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:52:30.592+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:52:30.593 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:52:30.593 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:52:30.612 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:52:30.612 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:52:30.640 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:52:30.641 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:52:30.659 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:52:30.659 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:52:30.677 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:52:30.678 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:52:30.689 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:30.692+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:30.695 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:52:30.696 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:52:30.705 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:52:30.705 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:52:30.705 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:52:30.705 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:52:30.714 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:52:30.790 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:30.793+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:30.864 INFO:teuthology.orchestra.run.smithi196.stdout:908 write 0x92f4000 thru 0x92f41f4 (0x1f5 bytes) 2024-05-08T08:52:30.865 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:52:30.865 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:52:30.865 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:52:30.865 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:52:30.865 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:52:30.874 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:52:30.875 INFO:teuthology.orchestra.run.smithi196.stdout:909 trunc from 0xd705a00 to 0x4642200 2024-05-08T08:52:30.891 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:30.894+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:30.992 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:52:30.993 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:52:30.993 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:52:30.993 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:52:30.993 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:30.993 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9944 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 749568~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:30.993 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.103900+0000", 2024-05-08T08:52:30.993 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 3.8903203550000001, 2024-05-08T08:52:30.993 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057007044, 2024-05-08T08:52:30.993 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:30.993 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:30.994 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:30.994 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:30.994 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:30.994 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:30.994 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9944 2024-05-08T08:52:30.994 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:30.994 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:30.994 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:30.994 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:30.994 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103900+0000", 2024-05-08T08:52:30.994 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:30.994 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:30.994 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:30.995 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:30.995 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103900+0000", 2024-05-08T08:52:30.995 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:30.995 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:30.995 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:30.995 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:30.995 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103900+0000", 2024-05-08T08:52:30.995 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6400000000000003e-07 2024-05-08T08:52:30.995 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:30.995 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:30.995 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:30.995 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103902+0000", 2024-05-08T08:52:30.995 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.858e-06 2024-05-08T08:52:30.995 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:30.996 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:30.996 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:30.996 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103907+0000", 2024-05-08T08:52:30.996 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3410000000000002e-06 2024-05-08T08:52:30.996 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:30.996 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:30.996 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:30.996 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103910+0000", 2024-05-08T08:52:30.996 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8540000000000001e-06 2024-05-08T08:52:30.996 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:30.996 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:30.996 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:30.996 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.160682+0000", 2024-05-08T08:52:30.996 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.056771822 2024-05-08T08:52:30.996 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:30.997 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:30.997 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:30.997 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.160777+0000", 2024-05-08T08:52:30.997 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4555999999999994e-05 2024-05-08T08:52:30.997 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:30.997 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:30.997 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:30.997 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.160907+0000", 2024-05-08T08:52:30.997 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000130149 2024-05-08T08:52:30.997 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:30.997 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:30.997 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:30.997 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:30.997 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:30.998 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9945 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 1441792~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:30.998 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.103914+0000", 2024-05-08T08:52:30.998 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 3.8903057620000001, 2024-05-08T08:52:30.998 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057211979000000003, 2024-05-08T08:52:30.998 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:30.998 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:30.998 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:30.998 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:30.998 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:30.998 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:30.998 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9945 2024-05-08T08:52:30.998 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:30.998 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:30.998 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:30.999 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:30.999 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103914+0000", 2024-05-08T08:52:30.999 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:30.999 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:30.999 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:30.999 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:30.999 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103914+0000", 2024-05-08T08:52:30.999 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:30.999 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:30.999 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:30.999 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:31.000 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103915+0000", 2024-05-08T08:52:31.000 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3099999999999999e-07 2024-05-08T08:52:31.000 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.000 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.000 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:31.000 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103916+0000", 2024-05-08T08:52:31.000 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.203e-06 2024-05-08T08:52:31.000 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.000 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.000 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:31.000 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103919+0000", 2024-05-08T08:52:31.000 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.117e-06 2024-05-08T08:52:31.000 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.000 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.001 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:31.001 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103921+0000", 2024-05-08T08:52:31.001 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4169999999999999e-06 2024-05-08T08:52:31.001 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.001 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.001 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:31.001 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.160959+0000", 2024-05-08T08:52:31.001 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057038074000000001 2024-05-08T08:52:31.001 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.001 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.001 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:31.001 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161041+0000", 2024-05-08T08:52:31.001 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1620999999999997e-05 2024-05-08T08:52:31.001 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.001 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.002 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:31.002 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161126+0000", 2024-05-08T08:52:31.002 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.5216000000000005e-05 2024-05-08T08:52:31.002 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:31.002 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:31.002 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:31.002 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.002 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.002 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9946 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 765952~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:31.002 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.103925+0000", 2024-05-08T08:52:31.002 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 3.8902953949999999, 2024-05-08T08:52:31.002 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057418770000000001, 2024-05-08T08:52:31.002 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:31.003 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:31.003 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:31.003 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:31.003 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:31.003 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:31.003 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9946 2024-05-08T08:52:31.003 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.003 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:31.004 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.004 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:31.004 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103925+0000", 2024-05-08T08:52:31.004 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:31.004 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.004 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.004 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:31.004 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103925+0000", 2024-05-08T08:52:31.004 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:31.004 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.004 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.004 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:31.004 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103925+0000", 2024-05-08T08:52:31.004 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9200000000000002e-07 2024-05-08T08:52:31.005 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.005 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.005 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:31.005 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103926+0000", 2024-05-08T08:52:31.005 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1620000000000001e-06 2024-05-08T08:52:31.005 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.005 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.005 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:31.005 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103929+0000", 2024-05-08T08:52:31.005 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9349999999999999e-06 2024-05-08T08:52:31.005 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.005 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.005 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:31.005 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103931+0000", 2024-05-08T08:52:31.006 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4099999999999998e-06 2024-05-08T08:52:31.006 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.006 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.006 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:31.006 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161161+0000", 2024-05-08T08:52:31.006 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057229176999999999 2024-05-08T08:52:31.006 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.006 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.006 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:31.006 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161235+0000", 2024-05-08T08:52:31.009 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4227000000000006e-05 2024-05-08T08:52:31.009 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.009 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.010 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:31.010 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161343+0000", 2024-05-08T08:52:31.010 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010856700000000001 2024-05-08T08:52:31.010 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:31.010 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:31.010 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:31.010 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.010 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.010 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9947 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 1458176~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:31.010 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.103935+0000", 2024-05-08T08:52:31.010 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 3.8902855170000001, 2024-05-08T08:52:31.011 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057541263000000002, 2024-05-08T08:52:31.011 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:31.011 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:31.011 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:31.011 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:31.011 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:31.011 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:31.011 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9947 2024-05-08T08:52:31.011 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.011 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:31.011 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.011 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:31.011 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103935+0000", 2024-05-08T08:52:31.012 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:31.012 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.012 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.012 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:31.012 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103935+0000", 2024-05-08T08:52:31.012 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:31.012 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.012 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.012 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:31.012 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103935+0000", 2024-05-08T08:52:31.012 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.11e-07 2024-05-08T08:52:31.012 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.012 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.012 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:31.013 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103940+0000", 2024-05-08T08:52:31.013 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1030000000000001e-06 2024-05-08T08:52:31.013 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.013 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.013 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:31.013 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103943+0000", 2024-05-08T08:52:31.013 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3869999999999999e-06 2024-05-08T08:52:31.013 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.013 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.013 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:31.013 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103951+0000", 2024-05-08T08:52:31.014 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.2150000000000001e-06 2024-05-08T08:52:31.014 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.014 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.014 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:31.014 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161377+0000", 2024-05-08T08:52:31.014 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057425925000000003 2024-05-08T08:52:31.014 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.015 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.015 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:31.015 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161431+0000", 2024-05-08T08:52:31.015 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3408000000000001e-05 2024-05-08T08:52:31.015 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.015 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.015 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:31.015 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161476+0000", 2024-05-08T08:52:31.015 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5014e-05 2024-05-08T08:52:31.015 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:31.015 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:31.015 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:31.015 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.016 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.016 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9948 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 782336~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:31.016 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.103965+0000", 2024-05-08T08:52:31.016 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 3.890254627, 2024-05-08T08:52:31.016 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057746205000000002, 2024-05-08T08:52:31.016 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:31.016 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:31.016 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:31.016 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:31.016 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:31.016 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:31.016 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9948 2024-05-08T08:52:31.016 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.016 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:31.016 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.017 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:31.017 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103965+0000", 2024-05-08T08:52:31.017 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:31.017 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.017 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.017 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:31.017 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103965+0000", 2024-05-08T08:52:31.017 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:31.017 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.017 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.017 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:31.017 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103966+0000", 2024-05-08T08:52:31.017 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.44e-07 2024-05-08T08:52:31.017 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.018 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.018 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:31.018 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103968+0000", 2024-05-08T08:52:31.018 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4090000000000001e-06 2024-05-08T08:52:31.018 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.018 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.018 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:31.018 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103973+0000", 2024-05-08T08:52:31.018 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6820000000000002e-06 2024-05-08T08:52:31.018 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.018 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.018 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:31.018 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103976+0000", 2024-05-08T08:52:31.018 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.959e-06 2024-05-08T08:52:31.019 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.019 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.019 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:31.019 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161523+0000", 2024-05-08T08:52:31.019 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057547264000000001 2024-05-08T08:52:31.019 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.019 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.019 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:31.019 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161605+0000", 2024-05-08T08:52:31.019 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1854000000000006e-05 2024-05-08T08:52:31.019 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.019 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.019 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:31.019 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161712+0000", 2024-05-08T08:52:31.019 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000106493 2024-05-08T08:52:31.020 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:31.020 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:31.020 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:31.020 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.020 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.020 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9949 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 1474560~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:31.021 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.103980+0000", 2024-05-08T08:52:31.021 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 3.8902400080000001, 2024-05-08T08:52:31.021 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057852397, 2024-05-08T08:52:31.021 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:31.021 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:31.021 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:31.021 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:31.021 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:31.021 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:31.021 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9949 2024-05-08T08:52:31.021 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.022 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:31.022 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.022 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:31.022 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103980+0000", 2024-05-08T08:52:31.022 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:31.022 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.022 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.022 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:31.022 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103980+0000", 2024-05-08T08:52:31.022 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:31.022 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.022 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.022 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:31.023 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103980+0000", 2024-05-08T08:52:31.023 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.11e-07 2024-05-08T08:52:31.023 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.023 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.023 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:31.023 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103982+0000", 2024-05-08T08:52:31.023 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.269e-06 2024-05-08T08:52:31.023 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.023 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.023 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:31.023 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103985+0000", 2024-05-08T08:52:31.023 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0110000000000001e-06 2024-05-08T08:52:31.023 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.023 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.024 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:31.024 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103987+0000", 2024-05-08T08:52:31.024 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3020000000000002e-06 2024-05-08T08:52:31.024 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.024 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.024 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:31.024 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161744+0000", 2024-05-08T08:52:31.024 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057757115999999997 2024-05-08T08:52:31.024 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.025 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.025 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:31.025 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161791+0000", 2024-05-08T08:52:31.025 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6807000000000001e-05 2024-05-08T08:52:31.025 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.025 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.025 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:31.025 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161832+0000", 2024-05-08T08:52:31.025 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1681000000000003e-05 2024-05-08T08:52:31.025 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:31.025 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:31.025 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:31.025 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.025 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.025 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9950 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 798720~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:31.026 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.103990+0000", 2024-05-08T08:52:31.026 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 3.8902297729999997, 2024-05-08T08:52:31.026 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.058062890999999998, 2024-05-08T08:52:31.026 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:31.026 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:31.026 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:31.026 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:31.026 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:31.026 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:31.026 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9950 2024-05-08T08:52:31.026 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.026 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:31.027 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.027 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:31.027 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103990+0000", 2024-05-08T08:52:31.027 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:31.027 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.027 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.027 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:31.027 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103990+0000", 2024-05-08T08:52:31.027 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:31.027 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.027 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.027 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:31.027 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103991+0000", 2024-05-08T08:52:31.027 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1e-07 2024-05-08T08:52:31.027 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.028 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.028 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:31.028 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103992+0000", 2024-05-08T08:52:31.028 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.074e-06 2024-05-08T08:52:31.028 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.028 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.028 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:31.028 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103995+0000", 2024-05-08T08:52:31.028 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9529999999999999e-06 2024-05-08T08:52:31.028 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.028 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.029 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:31.029 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103997+0000", 2024-05-08T08:52:31.029 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3939999999999999e-06 2024-05-08T08:52:31.029 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.029 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.029 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:31.029 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161871+0000", 2024-05-08T08:52:31.029 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057873854000000002 2024-05-08T08:52:31.029 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.029 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.029 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:31.029 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161950+0000", 2024-05-08T08:52:31.029 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9433999999999999e-05 2024-05-08T08:52:31.030 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.030 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.030 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:31.030 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.162053+0000", 2024-05-08T08:52:31.030 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000102972 2024-05-08T08:52:31.030 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:31.030 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:31.030 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:31.030 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.030 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.031 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9951 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 1490944~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:31.031 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.104000+0000", 2024-05-08T08:52:31.031 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 3.8902199689999999, 2024-05-08T08:52:31.031 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.058313416999999999, 2024-05-08T08:52:31.031 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:31.031 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:31.031 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:31.031 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:31.031 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:31.031 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:31.031 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9951 2024-05-08T08:52:31.031 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.031 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:31.032 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.032 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:31.032 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.104000+0000", 2024-05-08T08:52:31.032 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:31.032 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.032 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.032 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:31.032 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.104000+0000", 2024-05-08T08:52:31.032 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:31.032 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.032 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.032 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:31.032 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.104962+0000", 2024-05-08T08:52:31.032 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00096230399999999998 2024-05-08T08:52:31.032 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.033 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.033 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:31.033 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.104978+0000", 2024-05-08T08:52:31.033 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5126999999999999e-05 2024-05-08T08:52:31.033 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.033 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.033 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:31.033 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.104987+0000", 2024-05-08T08:52:31.033 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8940000000000005e-06 2024-05-08T08:52:31.033 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.033 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.033 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:31.033 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.104994+0000", 2024-05-08T08:52:31.033 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5610000000000004e-06 2024-05-08T08:52:31.034 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.034 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.034 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:31.034 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.162114+0000", 2024-05-08T08:52:31.034 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057119717 2024-05-08T08:52:31.034 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.034 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.034 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:31.034 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.162214+0000", 2024-05-08T08:52:31.035 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.9982999999999995e-05 2024-05-08T08:52:31.035 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.035 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.035 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:31.035 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.162314+0000", 2024-05-08T08:52:31.035 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.9831e-05 2024-05-08T08:52:31.035 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:31.035 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:31.035 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:31.035 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.035 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.035 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9952 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 815104~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:31.035 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.105004+0000", 2024-05-08T08:52:31.035 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 3.8892157159999998, 2024-05-08T08:52:31.035 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057530855999999998, 2024-05-08T08:52:31.036 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:31.036 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:31.036 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:31.036 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:31.036 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:31.036 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:31.036 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9952 2024-05-08T08:52:31.036 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.036 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:31.036 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.036 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:31.036 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105004+0000", 2024-05-08T08:52:31.036 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:31.036 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.037 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.037 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:31.037 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105004+0000", 2024-05-08T08:52:31.037 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:31.037 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.037 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.037 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:31.037 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105005+0000", 2024-05-08T08:52:31.037 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9299999999999998e-07 2024-05-08T08:52:31.037 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.037 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.037 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:31.037 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105007+0000", 2024-05-08T08:52:31.037 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2979999999999999e-06 2024-05-08T08:52:31.038 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.038 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.038 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:31.038 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105017+0000", 2024-05-08T08:52:31.038 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5060000000000001e-06 2024-05-08T08:52:31.038 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.038 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.038 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:31.038 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105022+0000", 2024-05-08T08:52:31.038 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2619999999999998e-06 2024-05-08T08:52:31.038 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.038 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.038 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:31.039 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.162356+0000", 2024-05-08T08:52:31.039 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057334388999999999 2024-05-08T08:52:31.039 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.039 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.039 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:31.039 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.162437+0000", 2024-05-08T08:52:31.039 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0863000000000002e-05 2024-05-08T08:52:31.039 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.039 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.039 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:31.039 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.162535+0000", 2024-05-08T08:52:31.039 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8045000000000001e-05 2024-05-08T08:52:31.040 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:31.040 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:31.040 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:31.040 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.040 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.040 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9953 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 1507328~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:31.040 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.105064+0000", 2024-05-08T08:52:31.040 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 3.889156029, 2024-05-08T08:52:31.040 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057721531999999999, 2024-05-08T08:52:31.040 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:31.040 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:31.040 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:31.040 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:31.040 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:31.041 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:31.041 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9953 2024-05-08T08:52:31.041 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.041 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:31.041 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.041 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:31.041 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105064+0000", 2024-05-08T08:52:31.041 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:31.041 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.041 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.042 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:31.042 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105064+0000", 2024-05-08T08:52:31.042 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:31.042 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.042 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.042 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:31.042 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105065+0000", 2024-05-08T08:52:31.042 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8800000000000002e-07 2024-05-08T08:52:31.042 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.042 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.042 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:31.042 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105067+0000", 2024-05-08T08:52:31.042 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7669999999999999e-06 2024-05-08T08:52:31.043 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.043 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.043 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:31.043 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105076+0000", 2024-05-08T08:52:31.043 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.5259999999999993e-06 2024-05-08T08:52:31.043 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.043 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.043 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:31.043 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105080+0000", 2024-05-08T08:52:31.043 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8e-06 2024-05-08T08:52:31.043 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.043 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.043 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:31.043 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.162596+0000", 2024-05-08T08:52:31.044 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057515822000000001 2024-05-08T08:52:31.044 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.044 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.044 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:31.044 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.162689+0000", 2024-05-08T08:52:31.044 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3258999999999996e-05 2024-05-08T08:52:31.044 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.044 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.044 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:31.044 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.162786+0000", 2024-05-08T08:52:31.044 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6769999999999997e-05 2024-05-08T08:52:31.044 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:31.045 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:31.045 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:31.045 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.045 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.045 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9954 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 1523712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:31.045 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.105088+0000", 2024-05-08T08:52:31.045 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 3.8891320199999999, 2024-05-08T08:52:31.045 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057925139, 2024-05-08T08:52:31.045 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:31.045 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:31.045 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:31.045 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:31.045 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:31.046 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:31.046 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9954 2024-05-08T08:52:31.046 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.046 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:31.046 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.046 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:31.046 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105088+0000", 2024-05-08T08:52:31.046 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:31.046 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.046 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.046 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:31.046 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105088+0000", 2024-05-08T08:52:31.046 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:31.047 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.047 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.047 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:31.047 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105089+0000", 2024-05-08T08:52:31.047 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9599999999999999e-07 2024-05-08T08:52:31.047 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.047 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.047 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:31.047 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105091+0000", 2024-05-08T08:52:31.047 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3649999999999998e-06 2024-05-08T08:52:31.047 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.047 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.047 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:31.047 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105107+0000", 2024-05-08T08:52:31.048 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6370000000000001e-05 2024-05-08T08:52:31.048 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.048 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.048 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:31.048 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105114+0000", 2024-05-08T08:52:31.048 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2190000000000001e-06 2024-05-08T08:52:31.048 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.048 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.048 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:31.048 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.162830+0000", 2024-05-08T08:52:31.048 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057716637000000001 2024-05-08T08:52:31.048 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.048 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.048 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:31.048 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.162924+0000", 2024-05-08T08:52:31.049 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3354000000000001e-05 2024-05-08T08:52:31.049 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.049 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.049 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:31.049 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.163013+0000", 2024-05-08T08:52:31.049 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.9697999999999994e-05 2024-05-08T08:52:31.049 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:31.049 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:31.049 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:31.049 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.049 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.049 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9955 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 831488~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:31.049 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.105124+0000", 2024-05-08T08:52:31.049 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 3.8890964459999999, 2024-05-08T08:52:31.049 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.058092618999999998, 2024-05-08T08:52:31.049 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:31.050 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:31.050 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:31.050 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:31.050 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:31.050 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:31.050 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9955 2024-05-08T08:52:31.050 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.050 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:31.050 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.050 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:31.050 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105124+0000", 2024-05-08T08:52:31.050 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:31.050 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.050 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.051 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:31.051 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105124+0000", 2024-05-08T08:52:31.051 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:31.051 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.051 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.051 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:31.051 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.106961+0000", 2024-05-08T08:52:31.051 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001837574 2024-05-08T08:52:31.051 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.051 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.051 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:31.051 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.106966+0000", 2024-05-08T08:52:31.052 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8439999999999997e-06 2024-05-08T08:52:31.052 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.052 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.052 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:31.052 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.106978+0000", 2024-05-08T08:52:31.052 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2330999999999999e-05 2024-05-08T08:52:31.052 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.052 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.052 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:31.052 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.106986+0000", 2024-05-08T08:52:31.052 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0439999999999996e-06 2024-05-08T08:52:31.052 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.052 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.052 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:31.052 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.163048+0000", 2024-05-08T08:52:31.053 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.056061182000000001 2024-05-08T08:52:31.053 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.053 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.053 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:31.053 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.163124+0000", 2024-05-08T08:52:31.053 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.6030000000000002e-05 2024-05-08T08:52:31.053 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.053 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.053 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:31.053 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.163216+0000", 2024-05-08T08:52:31.053 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2614000000000005e-05 2024-05-08T08:52:31.053 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:31.053 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:31.053 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:31.053 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.054 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.054 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9960 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 1589248~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:31.054 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.107116+0000", 2024-05-08T08:52:31.054 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 3.8871036280000002, 2024-05-08T08:52:31.054 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.056889636, 2024-05-08T08:52:31.054 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:31.054 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:31.054 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:31.054 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:31.054 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:31.054 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:31.054 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9960 2024-05-08T08:52:31.054 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.055 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:31.055 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.055 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:31.055 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107116+0000", 2024-05-08T08:52:31.055 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:31.055 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.055 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.055 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:31.055 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107116+0000", 2024-05-08T08:52:31.055 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:31.055 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.055 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.056 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:31.056 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107117+0000", 2024-05-08T08:52:31.056 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4400000000000001e-07 2024-05-08T08:52:31.056 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.056 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.056 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:31.056 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107119+0000", 2024-05-08T08:52:31.056 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3750000000000001e-06 2024-05-08T08:52:31.056 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.056 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.056 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:31.056 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107126+0000", 2024-05-08T08:52:31.056 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3369999999999996e-06 2024-05-08T08:52:31.056 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.057 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.057 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:31.057 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107130+0000", 2024-05-08T08:52:31.057 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8620000000000002e-06 2024-05-08T08:52:31.057 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.057 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.057 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:31.057 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.163917+0000", 2024-05-08T08:52:31.058 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.056786634000000002 2024-05-08T08:52:31.058 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.058 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.058 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:31.058 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.163962+0000", 2024-05-08T08:52:31.058 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4521000000000003e-05 2024-05-08T08:52:31.058 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.058 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.058 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:31.058 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.164006+0000", 2024-05-08T08:52:31.058 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4462999999999998e-05 2024-05-08T08:52:31.058 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:31.058 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:31.059 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:31.059 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.059 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.059 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9961 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 864256~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:31.059 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.107138+0000", 2024-05-08T08:52:31.059 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 3.8870825689999999, 2024-05-08T08:52:31.059 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057108051999999999, 2024-05-08T08:52:31.059 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:31.059 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:31.059 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:31.059 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:31.059 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:31.060 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:31.060 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9961 2024-05-08T08:52:31.060 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.060 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:31.060 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.060 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:31.060 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107138+0000", 2024-05-08T08:52:31.060 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:31.060 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.060 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.060 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:31.060 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107138+0000", 2024-05-08T08:52:31.060 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:31.061 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.061 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.061 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:31.061 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107138+0000", 2024-05-08T08:52:31.061 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2399999999999999e-07 2024-05-08T08:52:31.061 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.061 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.061 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:31.061 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107140+0000", 2024-05-08T08:52:31.061 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1409999999999999e-06 2024-05-08T08:52:31.061 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.061 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.061 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:31.061 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107146+0000", 2024-05-08T08:52:31.062 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0639999999999999e-06 2024-05-08T08:52:31.062 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.062 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.062 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:31.062 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107151+0000", 2024-05-08T08:52:31.062 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.69e-06 2024-05-08T08:52:31.062 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.062 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.062 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:31.062 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.164051+0000", 2024-05-08T08:52:31.063 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.056900153000000002 2024-05-08T08:52:31.063 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.063 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.063 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:31.063 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.164136+0000", 2024-05-08T08:52:31.063 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.5086999999999994e-05 2024-05-08T08:52:31.063 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.063 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.063 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:31.063 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.164246+0000", 2024-05-08T08:52:31.063 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000109493 2024-05-08T08:52:31.063 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:31.063 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:31.063 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:31.063 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.064 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.064 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9962 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 1605632~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:31.064 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.107158+0000", 2024-05-08T08:52:31.064 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 3.8870624899999999, 2024-05-08T08:52:31.064 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057292698000000003, 2024-05-08T08:52:31.064 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:31.064 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:31.064 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:31.064 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:31.064 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:31.064 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:31.064 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9962 2024-05-08T08:52:31.064 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.064 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:31.064 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.065 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:31.065 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107158+0000", 2024-05-08T08:52:31.065 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:31.065 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.065 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.065 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:31.065 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107158+0000", 2024-05-08T08:52:31.065 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:31.065 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.065 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.065 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:31.065 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107158+0000", 2024-05-08T08:52:31.066 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8100000000000003e-07 2024-05-08T08:52:31.066 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.066 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.066 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:31.066 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107167+0000", 2024-05-08T08:52:31.066 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3079999999999997e-06 2024-05-08T08:52:31.066 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.066 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.066 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:31.066 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107175+0000", 2024-05-08T08:52:31.066 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2089999999999999e-06 2024-05-08T08:52:31.066 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.066 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.067 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:31.067 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107180+0000", 2024-05-08T08:52:31.067 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1120000000000004e-06 2024-05-08T08:52:31.067 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.067 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.067 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:31.067 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.164284+0000", 2024-05-08T08:52:31.067 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057104110999999999 2024-05-08T08:52:31.067 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.067 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.067 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:31.067 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.164370+0000", 2024-05-08T08:52:31.067 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.6549000000000001e-05 2024-05-08T08:52:31.067 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.068 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.068 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:31.068 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.164450+0000", 2024-05-08T08:52:31.068 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9827999999999995e-05 2024-05-08T08:52:31.068 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:31.068 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:31.068 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:31.068 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.068 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.068 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9963 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 880640~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:31.068 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.107188+0000", 2024-05-08T08:52:31.068 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 3.8870325719999999, 2024-05-08T08:52:31.068 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057560471000000002, 2024-05-08T08:52:31.068 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:31.068 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:31.069 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:31.069 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:31.069 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:31.069 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:31.069 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9963 2024-05-08T08:52:31.069 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.069 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:31.069 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.069 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:31.069 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107188+0000", 2024-05-08T08:52:31.069 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:31.069 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.069 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.069 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:31.069 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107188+0000", 2024-05-08T08:52:31.069 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:31.069 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.070 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.070 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:31.070 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.108964+0000", 2024-05-08T08:52:31.070 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001775993 2024-05-08T08:52:31.070 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.070 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.070 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:31.070 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.108969+0000", 2024-05-08T08:52:31.070 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1200000000000001e-06 2024-05-08T08:52:31.070 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.070 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.070 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:31.070 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.108981+0000", 2024-05-08T08:52:31.070 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2670000000000001e-05 2024-05-08T08:52:31.071 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.071 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.071 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:31.071 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.108989+0000", 2024-05-08T08:52:31.071 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3139999999999998e-06 2024-05-08T08:52:31.071 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.071 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.071 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:31.071 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.164504+0000", 2024-05-08T08:52:31.071 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.055515225000000001 2024-05-08T08:52:31.071 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.071 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.071 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:31.071 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.164606+0000", 2024-05-08T08:52:31.071 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010245 2024-05-08T08:52:31.072 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.072 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.072 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:31.072 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.164748+0000", 2024-05-08T08:52:31.072 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014169900000000001 2024-05-08T08:52:31.072 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:31.072 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:31.072 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:31.072 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.072 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.072 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9970 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 929792~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:31.073 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.109178+0000", 2024-05-08T08:52:31.073 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 3.8850422939999998, 2024-05-08T08:52:31.073 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057137975000000001, 2024-05-08T08:52:31.073 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:31.073 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:31.073 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:31.073 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:31.073 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:31.073 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:31.073 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9970 2024-05-08T08:52:31.073 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.073 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:31.073 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.074 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:31.074 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109178+0000", 2024-05-08T08:52:31.074 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:31.074 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.074 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.074 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:31.074 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109178+0000", 2024-05-08T08:52:31.074 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:31.074 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.074 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.074 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:31.074 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109178+0000", 2024-05-08T08:52:31.074 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7800000000000002e-07 2024-05-08T08:52:31.074 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.075 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.075 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:31.075 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109180+0000", 2024-05-08T08:52:31.075 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2199999999999999e-06 2024-05-08T08:52:31.075 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.075 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.075 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:31.075 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109186+0000", 2024-05-08T08:52:31.075 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9410000000000004e-06 2024-05-08T08:52:31.075 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.075 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.075 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:31.075 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109191+0000", 2024-05-08T08:52:31.075 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8729999999999998e-06 2024-05-08T08:52:31.075 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.075 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.076 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:31.076 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.166098+0000", 2024-05-08T08:52:31.076 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.056907124000000003 2024-05-08T08:52:31.076 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.076 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.076 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:31.076 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.166193+0000", 2024-05-08T08:52:31.076 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5018000000000004e-05 2024-05-08T08:52:31.076 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.076 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.076 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:31.076 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.166316+0000", 2024-05-08T08:52:31.077 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000122321 2024-05-08T08:52:31.077 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:31.077 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:31.077 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:31.077 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.077 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.077 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9971 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 1687552~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:31.077 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.109198+0000", 2024-05-08T08:52:31.077 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 3.885021616, 2024-05-08T08:52:31.077 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057361875, 2024-05-08T08:52:31.077 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:31.077 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:31.077 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:31.077 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:31.077 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:31.078 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:31.078 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9971 2024-05-08T08:52:31.078 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.078 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:31.078 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.078 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:31.078 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109198+0000", 2024-05-08T08:52:31.078 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:31.078 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.078 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.078 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:31.078 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109198+0000", 2024-05-08T08:52:31.078 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:31.078 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.079 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.079 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:31.079 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109199+0000", 2024-05-08T08:52:31.079 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2800000000000002e-07 2024-05-08T08:52:31.079 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.079 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.079 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:31.079 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109208+0000", 2024-05-08T08:52:31.079 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4210000000000001e-06 2024-05-08T08:52:31.079 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.079 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.079 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:31.079 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109215+0000", 2024-05-08T08:52:31.079 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7209999999999997e-06 2024-05-08T08:52:31.079 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.080 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.080 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:31.080 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109220+0000", 2024-05-08T08:52:31.080 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.13e-06 2024-05-08T08:52:31.080 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.080 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.080 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:31.080 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.166353+0000", 2024-05-08T08:52:31.080 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057132693999999998 2024-05-08T08:52:31.080 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.080 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.080 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:31.080 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.166462+0000", 2024-05-08T08:52:31.080 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010940799999999999 2024-05-08T08:52:31.080 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.080 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.081 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:31.081 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.166560+0000", 2024-05-08T08:52:31.081 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8072999999999994e-05 2024-05-08T08:52:31.081 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:31.081 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:31.081 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:31.081 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.081 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.081 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9972 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 1703936~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:31.081 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.109228+0000", 2024-05-08T08:52:31.081 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 3.8849922239999999, 2024-05-08T08:52:31.081 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057541904999999997, 2024-05-08T08:52:31.081 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:31.081 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:31.081 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:31.082 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:31.082 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:31.082 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:31.082 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9972 2024-05-08T08:52:31.082 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.082 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:31.082 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.082 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:31.082 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109228+0000", 2024-05-08T08:52:31.082 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:31.082 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.082 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.083 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:31.083 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109228+0000", 2024-05-08T08:52:31.083 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:31.083 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.083 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.083 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:31.083 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109228+0000", 2024-05-08T08:52:31.083 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5699999999999998e-07 2024-05-08T08:52:31.083 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.083 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.083 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:31.083 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109231+0000", 2024-05-08T08:52:31.084 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2120000000000002e-06 2024-05-08T08:52:31.084 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.084 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.084 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:31.084 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109237+0000", 2024-05-08T08:52:31.084 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3099999999999997e-06 2024-05-08T08:52:31.084 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.084 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.084 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:31.084 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109243+0000", 2024-05-08T08:52:31.084 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7490000000000004e-06 2024-05-08T08:52:31.084 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.084 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.084 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:31.084 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.166609+0000", 2024-05-08T08:52:31.085 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057366372999999998 2024-05-08T08:52:31.085 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.085 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.085 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:31.085 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.166690+0000", 2024-05-08T08:52:31.085 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1426000000000003e-05 2024-05-08T08:52:31.085 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.085 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.085 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:31.085 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.166770+0000", 2024-05-08T08:52:31.085 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9377999999999998e-05 2024-05-08T08:52:31.085 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:31.085 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:31.085 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:31.085 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.085 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.086 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9973 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 946176~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:31.086 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.109249+0000", 2024-05-08T08:52:31.086 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 3.8849705820000002, 2024-05-08T08:52:31.086 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057760381, 2024-05-08T08:52:31.086 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:31.086 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:31.086 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:31.086 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:31.086 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:31.086 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:31.087 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9973 2024-05-08T08:52:31.087 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.087 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:31.087 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.087 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:31.087 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109249+0000", 2024-05-08T08:52:31.087 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:31.087 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.087 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.087 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:31.087 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109249+0000", 2024-05-08T08:52:31.087 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:31.087 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.087 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.087 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:31.087 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.110959+0000", 2024-05-08T08:52:31.088 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001709288 2024-05-08T08:52:31.088 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.088 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.088 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:31.088 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.110972+0000", 2024-05-08T08:52:31.088 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2719e-05 2024-05-08T08:52:31.088 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.088 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.088 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:31.088 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.110984+0000", 2024-05-08T08:52:31.088 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2001000000000001e-05 2024-05-08T08:52:31.088 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.088 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.088 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:31.088 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.110993+0000", 2024-05-08T08:52:31.089 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0880000000000001e-06 2024-05-08T08:52:31.089 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.089 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.089 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:31.089 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.166817+0000", 2024-05-08T08:52:31.089 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.055823971999999999 2024-05-08T08:52:31.089 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.089 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.089 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:31.089 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.166900+0000", 2024-05-08T08:52:31.089 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3838999999999995e-05 2024-05-08T08:52:31.089 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.089 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.089 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:31.089 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.167010+0000", 2024-05-08T08:52:31.090 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000109474 2024-05-08T08:52:31.090 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:31.090 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:31.090 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:31.090 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:31.090 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:31.090 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:52:31.091 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:52:30.994+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:52:31.092 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:52:31.093 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-05-08T08:52:31.094 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:52:31.098+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:52:31.196 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:52:31.199+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:52:31.246 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:52:31.247 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:52:31.247 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:52:31.247 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:52:31.256 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:52:31.297 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:31.300+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:31.366 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:52:31.366 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:52:31.367 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:52:31.367 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:52:31.367 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:52:31.375 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:52:31.422 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:52:31.421+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:52:31.498 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:52:31.502+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:52:31.509 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:52:31.510 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:52:31.510 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:52:31.510 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:52:31.510 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.510 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:19851 3.5 3:a5e62864:::rbd_data.1140782590a5.000000000000000e:6 [read 733184~2048] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:31.510 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.613045+0000", 2024-05-08T08:52:31.510 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 3.898204631, 2024-05-08T08:52:31.510 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.039558625, 2024-05-08T08:52:31.511 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:31.511 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:31.511 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:31.511 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:31.511 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:31.511 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:31.511 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 19851 2024-05-08T08:52:31.511 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.512 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:31.512 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.512 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:31.512 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613045+0000", 2024-05-08T08:52:31.512 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:31.512 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.512 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.512 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:31.512 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613045+0000", 2024-05-08T08:52:31.512 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:31.512 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.512 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.512 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:31.512 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613045+0000", 2024-05-08T08:52:31.513 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3900000000000005e-07 2024-05-08T08:52:31.513 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.513 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.513 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:31.513 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613047+0000", 2024-05-08T08:52:31.513 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3630000000000001e-06 2024-05-08T08:52:31.513 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.513 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.513 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:31.513 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613054+0000", 2024-05-08T08:52:31.513 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0779999999999998e-06 2024-05-08T08:52:31.513 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.513 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.513 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:31.513 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613060+0000", 2024-05-08T08:52:31.513 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2789999999999998e-06 2024-05-08T08:52:31.514 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.514 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.514 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:31.514 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.652522+0000", 2024-05-08T08:52:31.514 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.039462641999999999 2024-05-08T08:52:31.514 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.514 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.514 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:31.514 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.652558+0000", 2024-05-08T08:52:31.514 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5564e-05 2024-05-08T08:52:31.514 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.514 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.514 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:31.515 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.652603+0000", 2024-05-08T08:52:31.515 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5160000000000001e-05 2024-05-08T08:52:31.515 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:31.515 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:31.515 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:31.515 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.515 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.515 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:19853 3.5 3:a5e62864:::rbd_data.1140782590a5.000000000000000e:6 [read 749568~2048] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:31.515 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.613079+0000", 2024-05-08T08:52:31.515 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 3.8981704490000002, 2024-05-08T08:52:31.516 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.039675558, 2024-05-08T08:52:31.516 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:31.516 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:31.516 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:31.516 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:31.516 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:31.516 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:31.516 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 19853 2024-05-08T08:52:31.516 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.516 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:31.516 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.516 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:31.516 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613079+0000", 2024-05-08T08:52:31.516 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:31.517 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.517 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.517 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:31.517 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613079+0000", 2024-05-08T08:52:31.517 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:31.517 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.517 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.517 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:31.517 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613079+0000", 2024-05-08T08:52:31.517 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8699999999999995e-07 2024-05-08T08:52:31.517 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.517 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.517 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:31.517 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613082+0000", 2024-05-08T08:52:31.517 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6019999999999998e-06 2024-05-08T08:52:31.518 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.518 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.518 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:31.518 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613089+0000", 2024-05-08T08:52:31.518 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8669999999999996e-06 2024-05-08T08:52:31.518 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.518 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.518 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:31.518 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613094+0000", 2024-05-08T08:52:31.518 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2050000000000001e-06 2024-05-08T08:52:31.518 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.518 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.518 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:31.518 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.652618+0000", 2024-05-08T08:52:31.519 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.039523865999999998 2024-05-08T08:52:31.519 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.519 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.519 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:31.519 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.652659+0000", 2024-05-08T08:52:31.519 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1488e-05 2024-05-08T08:52:31.519 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.519 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.519 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:31.519 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.652754+0000", 2024-05-08T08:52:31.519 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4943000000000002e-05 2024-05-08T08:52:31.519 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:31.519 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:31.519 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:31.519 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.519 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.520 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:19855 3.5 3:a5e62864:::rbd_data.1140782590a5.000000000000000e:6 [read 765952~2048] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:31.520 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.613103+0000", 2024-05-08T08:52:31.520 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 3.8981463459999999, 2024-05-08T08:52:31.520 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.039895344999999999, 2024-05-08T08:52:31.520 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:31.520 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:31.520 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:31.520 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:31.520 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:31.520 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:31.520 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 19855 2024-05-08T08:52:31.520 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.520 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:31.520 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.521 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:31.521 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613103+0000", 2024-05-08T08:52:31.521 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:31.521 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.521 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.521 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:31.521 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613103+0000", 2024-05-08T08:52:31.521 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:31.521 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.522 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.522 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:31.522 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613103+0000", 2024-05-08T08:52:31.522 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3900000000000005e-07 2024-05-08T08:52:31.522 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.522 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.522 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:31.523 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613105+0000", 2024-05-08T08:52:31.523 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9549999999999999e-06 2024-05-08T08:52:31.523 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.523 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.523 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:31.523 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613112+0000", 2024-05-08T08:52:31.523 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3709999999999996e-06 2024-05-08T08:52:31.523 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.523 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.523 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:31.523 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613116+0000", 2024-05-08T08:52:31.524 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8040000000000001e-06 2024-05-08T08:52:31.524 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.524 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.524 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:31.524 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.652801+0000", 2024-05-08T08:52:31.524 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.039684589999999999 2024-05-08T08:52:31.524 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.524 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.524 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:31.524 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.652888+0000", 2024-05-08T08:52:31.524 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.6847000000000004e-05 2024-05-08T08:52:31.524 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.524 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.524 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:31.524 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.652998+0000", 2024-05-08T08:52:31.525 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000110239 2024-05-08T08:52:31.525 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:31.525 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:31.525 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:31.525 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.525 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.525 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:19857 3.5 3:a5e62864:::rbd_data.1140782590a5.000000000000000e:6 [read 782336~2048] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:31.525 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.613123+0000", 2024-05-08T08:52:31.525 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 3.8981257579999999, 2024-05-08T08:52:31.525 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.040094418999999999, 2024-05-08T08:52:31.525 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:31.525 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:31.525 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:31.525 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:31.525 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:31.526 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:31.526 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 19857 2024-05-08T08:52:31.526 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.526 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:31.526 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.526 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:31.526 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613123+0000", 2024-05-08T08:52:31.526 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:31.526 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.526 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.526 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:31.526 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613123+0000", 2024-05-08T08:52:31.526 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:31.527 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.527 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.527 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:31.527 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613124+0000", 2024-05-08T08:52:31.527 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9499999999999998e-07 2024-05-08T08:52:31.527 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.527 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.527 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:31.527 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613130+0000", 2024-05-08T08:52:31.527 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6730000000000003e-06 2024-05-08T08:52:31.527 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.527 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.527 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:31.527 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613137+0000", 2024-05-08T08:52:31.527 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7399999999999998e-06 2024-05-08T08:52:31.528 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.528 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.528 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:31.528 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613143+0000", 2024-05-08T08:52:31.528 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8289999999999996e-06 2024-05-08T08:52:31.528 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.528 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.528 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:31.528 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.653043+0000", 2024-05-08T08:52:31.528 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.039900401000000002 2024-05-08T08:52:31.528 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.528 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.528 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:31.528 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.653128+0000", 2024-05-08T08:52:31.528 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4532999999999999e-05 2024-05-08T08:52:31.528 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.529 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.529 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:31.529 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.653218+0000", 2024-05-08T08:52:31.529 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.9847999999999998e-05 2024-05-08T08:52:31.529 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:31.529 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:31.529 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:31.529 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.529 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.529 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:19859 3.5 3:a5e62864:::rbd_data.1140782590a5.000000000000000e:6 [read 798720~2048] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:31.529 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.613208+0000", 2024-05-08T08:52:31.529 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 3.8980413899999999, 2024-05-08T08:52:31.529 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.040110768999999998, 2024-05-08T08:52:31.529 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:31.529 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:31.530 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:31.530 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:31.530 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:31.530 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:31.530 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 19859 2024-05-08T08:52:31.530 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.530 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:31.530 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.530 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:31.530 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613208+0000", 2024-05-08T08:52:31.530 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:31.530 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.530 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.530 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:31.531 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613208+0000", 2024-05-08T08:52:31.531 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:31.531 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.531 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.531 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:31.531 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613208+0000", 2024-05-08T08:52:31.531 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5300000000000004e-07 2024-05-08T08:52:31.531 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.531 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.531 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:31.531 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613211+0000", 2024-05-08T08:52:31.531 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7829999999999999e-06 2024-05-08T08:52:31.531 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.532 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.532 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:31.532 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613220+0000", 2024-05-08T08:52:31.532 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7469999999999994e-06 2024-05-08T08:52:31.532 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.532 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.532 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:31.532 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613225+0000", 2024-05-08T08:52:31.532 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3489999999999999e-06 2024-05-08T08:52:31.532 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.532 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.532 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:31.533 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.653235+0000", 2024-05-08T08:52:31.533 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.040009939000000001 2024-05-08T08:52:31.533 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.533 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.533 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:31.533 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.653272+0000", 2024-05-08T08:52:31.533 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7039000000000002e-05 2024-05-08T08:52:31.533 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.533 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.534 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:31.534 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.653319+0000", 2024-05-08T08:52:31.534 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6359000000000001e-05 2024-05-08T08:52:31.534 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:31.534 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:31.534 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:31.534 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.534 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.534 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:19861 3.5 3:a5e62864:::rbd_data.1140782590a5.000000000000000e:6 [read 815104~2048] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:31.534 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.613247+0000", 2024-05-08T08:52:31.534 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 3.898001888, 2024-05-08T08:52:31.534 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.040184922999999997, 2024-05-08T08:52:31.534 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:31.534 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:31.534 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:31.535 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:31.535 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:31.535 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:31.535 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 19861 2024-05-08T08:52:31.535 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.535 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:31.535 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.535 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:31.535 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613247+0000", 2024-05-08T08:52:31.535 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:31.535 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.535 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.535 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:31.535 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613247+0000", 2024-05-08T08:52:31.536 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:31.536 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.536 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.536 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:31.536 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613248+0000", 2024-05-08T08:52:31.536 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1099999999999996e-07 2024-05-08T08:52:31.536 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.536 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.536 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:31.536 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613250+0000", 2024-05-08T08:52:31.536 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.525e-06 2024-05-08T08:52:31.536 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.536 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.536 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:31.536 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613259+0000", 2024-05-08T08:52:31.537 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.2889999999999998e-06 2024-05-08T08:52:31.537 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.537 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.537 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:31.537 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613264+0000", 2024-05-08T08:52:31.537 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2739999999999998e-06 2024-05-08T08:52:31.537 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.537 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.537 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:31.537 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.653333+0000", 2024-05-08T08:52:31.537 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.040069645000000001 2024-05-08T08:52:31.537 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.537 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.537 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:31.537 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.653367+0000", 2024-05-08T08:52:31.537 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3840999999999997e-05 2024-05-08T08:52:31.538 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.538 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.538 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:31.538 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.653432+0000", 2024-05-08T08:52:31.538 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4838000000000005e-05 2024-05-08T08:52:31.538 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:31.538 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:31.538 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:31.538 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.538 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.538 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:19863 3.5 3:a5e62864:::rbd_data.1140782590a5.000000000000000e:6 [read 831488~2048] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:31.538 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.613283+0000", 2024-05-08T08:52:31.538 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 3.8979666210000001, 2024-05-08T08:52:31.538 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.040385977000000003, 2024-05-08T08:52:31.538 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:31.539 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:31.539 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:31.539 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:31.539 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:31.539 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:31.539 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 19863 2024-05-08T08:52:31.539 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.539 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:31.539 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.539 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:31.539 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613283+0000", 2024-05-08T08:52:31.539 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:31.539 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.539 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.540 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:31.540 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613283+0000", 2024-05-08T08:52:31.540 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:31.540 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.540 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.540 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:31.540 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613283+0000", 2024-05-08T08:52:31.540 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2099999999999997e-07 2024-05-08T08:52:31.540 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.540 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.540 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:31.540 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613286+0000", 2024-05-08T08:52:31.540 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.5009999999999999e-06 2024-05-08T08:52:31.540 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.540 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.541 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:31.541 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613292+0000", 2024-05-08T08:52:31.541 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9199999999999998e-06 2024-05-08T08:52:31.541 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.541 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.541 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:31.541 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613298+0000", 2024-05-08T08:52:31.541 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2769999999999996e-06 2024-05-08T08:52:31.541 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.541 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.541 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:31.541 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.653479+0000", 2024-05-08T08:52:31.541 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.040181069 2024-05-08T08:52:31.542 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.542 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.542 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:31.542 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.653561+0000", 2024-05-08T08:52:31.542 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1806999999999998e-05 2024-05-08T08:52:31.542 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.542 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.542 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:31.542 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.653668+0000", 2024-05-08T08:52:31.543 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010788199999999999 2024-05-08T08:52:31.543 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:31.543 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:31.543 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:31.543 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.543 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.543 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:19864 3.5 3:a5e62864:::rbd_data.1140782590a5.000000000000000e:6 [read 847872~2048] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:31.543 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.613321+0000", 2024-05-08T08:52:31.543 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 3.8979285040000002, 2024-05-08T08:52:31.543 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.040567076000000001, 2024-05-08T08:52:31.543 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:31.543 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:31.543 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:31.543 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:31.544 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:31.544 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:31.544 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 19864 2024-05-08T08:52:31.544 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.544 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:31.544 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.544 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:31.544 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613321+0000", 2024-05-08T08:52:31.544 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:31.544 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.544 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.545 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:31.545 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613321+0000", 2024-05-08T08:52:31.545 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:31.545 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.545 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.545 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:31.545 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613321+0000", 2024-05-08T08:52:31.545 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.46e-07 2024-05-08T08:52:31.545 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.545 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.545 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:31.545 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613324+0000", 2024-05-08T08:52:31.545 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.464e-06 2024-05-08T08:52:31.545 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.545 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.545 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:31.546 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613331+0000", 2024-05-08T08:52:31.546 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1400000000000002e-06 2024-05-08T08:52:31.546 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.546 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.546 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:31.546 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613336+0000", 2024-05-08T08:52:31.546 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.626e-06 2024-05-08T08:52:31.546 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.546 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.546 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:31.546 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.653702+0000", 2024-05-08T08:52:31.546 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.040366022000000001 2024-05-08T08:52:31.546 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.546 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.547 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:31.547 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.653780+0000", 2024-05-08T08:52:31.547 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7708999999999997e-05 2024-05-08T08:52:31.547 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.547 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.547 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:31.547 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.653888+0000", 2024-05-08T08:52:31.547 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000107669 2024-05-08T08:52:31.547 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:31.547 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:31.547 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:31.547 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.547 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.548 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:19866 3.5 3:a5e62864:::rbd_data.1140782590a5.000000000000000e:6 [read 864256~2048] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:31.548 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.613343+0000", 2024-05-08T08:52:31.548 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 3.8979056349999999, 2024-05-08T08:52:31.548 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.040772782, 2024-05-08T08:52:31.548 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:31.548 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:31.548 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:31.548 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:31.548 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:31.548 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:31.548 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 19866 2024-05-08T08:52:31.548 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.548 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:31.548 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.548 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:31.548 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613343+0000", 2024-05-08T08:52:31.549 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:31.549 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.549 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.549 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:31.549 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613343+0000", 2024-05-08T08:52:31.549 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:31.549 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.549 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.549 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:31.549 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613344+0000", 2024-05-08T08:52:31.549 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5300000000000004e-07 2024-05-08T08:52:31.549 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.549 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.549 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:31.549 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613346+0000", 2024-05-08T08:52:31.549 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3290000000000001e-06 2024-05-08T08:52:31.550 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.550 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.550 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:31.550 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613353+0000", 2024-05-08T08:52:31.550 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4629999999999998e-06 2024-05-08T08:52:31.550 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.550 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.550 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:31.550 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613358+0000", 2024-05-08T08:52:31.550 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2000000000000002e-06 2024-05-08T08:52:31.550 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.550 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.550 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:31.550 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.653922+0000", 2024-05-08T08:52:31.550 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.040563752000000002 2024-05-08T08:52:31.551 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.551 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.551 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:31.551 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.654009+0000", 2024-05-08T08:52:31.551 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7090999999999997e-05 2024-05-08T08:52:31.551 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.551 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.551 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:31.551 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.654116+0000", 2024-05-08T08:52:31.551 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010729400000000001 2024-05-08T08:52:31.551 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:31.551 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:31.551 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:31.551 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.551 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.551 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:19868 3.5 3:a5e62864:::rbd_data.1140782590a5.000000000000000e:6 [read 880640~2048] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:31.552 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.613365+0000", 2024-05-08T08:52:31.552 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 3.897884355, 2024-05-08T08:52:31.552 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.040883700000000002, 2024-05-08T08:52:31.552 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:31.552 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:31.552 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:31.552 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:31.552 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:31.552 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:31.552 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 19868 2024-05-08T08:52:31.552 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.552 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:31.552 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.552 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:31.553 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613365+0000", 2024-05-08T08:52:31.553 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:31.553 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.553 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.553 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:31.553 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613365+0000", 2024-05-08T08:52:31.553 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:31.553 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.554 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.554 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:31.554 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613365+0000", 2024-05-08T08:52:31.554 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5400000000000001e-07 2024-05-08T08:52:31.554 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.554 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.554 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:31.554 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613368+0000", 2024-05-08T08:52:31.555 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3209999999999999e-06 2024-05-08T08:52:31.555 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.555 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.555 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:31.555 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613374+0000", 2024-05-08T08:52:31.555 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1199999999999999e-06 2024-05-08T08:52:31.555 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.555 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.555 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:31.555 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613379+0000", 2024-05-08T08:52:31.555 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1880000000000001e-06 2024-05-08T08:52:31.555 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.555 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.556 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:31.556 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.654152+0000", 2024-05-08T08:52:31.556 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.040772592000000003 2024-05-08T08:52:31.556 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.556 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.556 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:31.556 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.654195+0000", 2024-05-08T08:52:31.556 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3269999999999997e-05 2024-05-08T08:52:31.556 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.556 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.556 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:31.556 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.654248+0000", 2024-05-08T08:52:31.556 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3655e-05 2024-05-08T08:52:31.556 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:31.557 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:31.557 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:31.557 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.557 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.557 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:19870 3.5 3:a5e62864:::rbd_data.1140782590a5.000000000000000e:6 [read 897024~2048] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:31.557 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.613385+0000", 2024-05-08T08:52:31.557 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 3.8978640220000003, 2024-05-08T08:52:31.557 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.040967172000000003, 2024-05-08T08:52:31.557 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:31.557 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:31.557 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:31.557 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:31.557 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:31.557 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:31.557 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 19870 2024-05-08T08:52:31.558 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.558 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:31.558 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.558 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:31.558 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613385+0000", 2024-05-08T08:52:31.558 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:31.558 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.558 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.558 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:31.558 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613385+0000", 2024-05-08T08:52:31.558 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:31.558 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.558 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.558 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:31.559 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613385+0000", 2024-05-08T08:52:31.559 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8000000000000001e-07 2024-05-08T08:52:31.559 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.559 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.559 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:31.559 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613388+0000", 2024-05-08T08:52:31.559 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1519999999999999e-06 2024-05-08T08:52:31.559 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.559 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.559 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:31.559 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613395+0000", 2024-05-08T08:52:31.559 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0079999999999996e-06 2024-05-08T08:52:31.559 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.559 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.559 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:31.560 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613400+0000", 2024-05-08T08:52:31.560 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0209999999999999e-06 2024-05-08T08:52:31.560 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.560 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.560 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:31.560 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.654267+0000", 2024-05-08T08:52:31.560 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.040866966999999997 2024-05-08T08:52:31.561 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.561 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.561 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:31.561 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.654305+0000", 2024-05-08T08:52:31.561 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8043999999999997e-05 2024-05-08T08:52:31.561 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.561 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.561 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:31.561 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.654352+0000", 2024-05-08T08:52:31.561 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7599999999999998e-05 2024-05-08T08:52:31.561 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:31.561 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:31.561 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:31.561 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.561 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.562 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:19872 3.5 3:a5e62864:::rbd_data.1140782590a5.000000000000000e:6 [read 913408~2048] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:31.562 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.613406+0000", 2024-05-08T08:52:31.562 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 3.897842834, 2024-05-08T08:52:31.562 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.041046483000000002, 2024-05-08T08:52:31.562 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:31.562 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:31.562 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:31.562 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:31.562 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:31.562 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:31.562 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 19872 2024-05-08T08:52:31.562 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.562 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:31.562 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.563 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:31.563 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613406+0000", 2024-05-08T08:52:31.563 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:31.563 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.563 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.563 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:31.563 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613406+0000", 2024-05-08T08:52:31.563 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:31.563 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.563 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.563 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:31.563 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613407+0000", 2024-05-08T08:52:31.564 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4200000000000001e-07 2024-05-08T08:52:31.564 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.564 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.564 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:31.564 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613409+0000", 2024-05-08T08:52:31.564 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1799999999999999e-06 2024-05-08T08:52:31.564 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.564 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.564 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:31.564 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613415+0000", 2024-05-08T08:52:31.564 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2720000000000003e-06 2024-05-08T08:52:31.564 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.564 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.565 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:31.565 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613420+0000", 2024-05-08T08:52:31.565 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9760000000000003e-06 2024-05-08T08:52:31.565 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.565 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.565 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:31.565 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.654369+0000", 2024-05-08T08:52:31.565 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.040949103000000001 2024-05-08T08:52:31.565 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.566 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.566 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:31.566 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.654406+0000", 2024-05-08T08:52:31.566 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6807000000000002e-05 2024-05-08T08:52:31.566 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.566 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.566 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:31.566 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.654453+0000", 2024-05-08T08:52:31.566 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6703000000000003e-05 2024-05-08T08:52:31.566 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:31.566 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:31.566 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:31.566 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.566 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.567 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:19874 3.5 3:a5e62864:::rbd_data.1140782590a5.000000000000000e:6 [read 929792~2048] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:31.567 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.613426+0000", 2024-05-08T08:52:31.567 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 3.8978228430000001, 2024-05-08T08:52:31.567 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.041127155999999998, 2024-05-08T08:52:31.567 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:31.567 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:31.567 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:31.567 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:31.567 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:31.567 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:31.567 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 19874 2024-05-08T08:52:31.567 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.567 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:31.567 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.567 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:31.568 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613426+0000", 2024-05-08T08:52:31.568 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:31.568 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.568 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.568 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:31.568 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613426+0000", 2024-05-08T08:52:31.568 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:31.568 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.568 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.568 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:31.568 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613427+0000", 2024-05-08T08:52:31.568 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4799999999999999e-07 2024-05-08T08:52:31.568 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.568 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.569 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:31.569 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613429+0000", 2024-05-08T08:52:31.569 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.334e-06 2024-05-08T08:52:31.569 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.569 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.569 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:31.569 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613436+0000", 2024-05-08T08:52:31.569 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7560000000000002e-06 2024-05-08T08:52:31.569 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.569 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.569 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:31.569 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613440+0000", 2024-05-08T08:52:31.569 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7149999999999997e-06 2024-05-08T08:52:31.569 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.570 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.570 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:31.570 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.654469+0000", 2024-05-08T08:52:31.570 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.041028926 2024-05-08T08:52:31.570 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.570 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.570 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:31.570 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.654506+0000", 2024-05-08T08:52:31.570 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6879999999999999e-05 2024-05-08T08:52:31.570 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.570 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.570 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:31.570 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.654553+0000", 2024-05-08T08:52:31.571 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7197000000000002e-05 2024-05-08T08:52:31.571 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:31.571 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:31.571 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:31.571 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.571 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.571 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:19875 3.5 3:a5e62864:::rbd_data.1140782590a5.000000000000000e:6 [read 946176~2048] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:31.571 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.613448+0000", 2024-05-08T08:52:31.571 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 3.8978015660000001, 2024-05-08T08:52:31.571 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.041208306, 2024-05-08T08:52:31.571 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:31.571 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:31.571 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:31.571 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:31.572 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:31.572 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:31.572 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 19875 2024-05-08T08:52:31.572 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.572 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:31.572 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.572 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:31.572 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613448+0000", 2024-05-08T08:52:31.572 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:31.572 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.572 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.572 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:31.572 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613448+0000", 2024-05-08T08:52:31.572 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:31.572 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.573 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.573 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:31.573 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613448+0000", 2024-05-08T08:52:31.573 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.51e-07 2024-05-08T08:52:31.573 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.573 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.573 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:31.573 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613450+0000", 2024-05-08T08:52:31.573 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2299999999999998e-06 2024-05-08T08:52:31.573 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.573 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.573 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:31.573 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613459+0000", 2024-05-08T08:52:31.573 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1649999999999995e-06 2024-05-08T08:52:31.573 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.574 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.574 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:31.574 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613465+0000", 2024-05-08T08:52:31.574 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2079999999999999e-06 2024-05-08T08:52:31.574 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.574 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.574 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:31.574 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.654571+0000", 2024-05-08T08:52:31.575 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.041106530000000002 2024-05-08T08:52:31.575 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.575 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.575 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:31.575 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.654609+0000", 2024-05-08T08:52:31.575 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8204000000000002e-05 2024-05-08T08:52:31.575 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.575 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.575 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:31.575 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.654656+0000", 2024-05-08T08:52:31.575 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6517999999999997e-05 2024-05-08T08:52:31.575 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:31.576 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:31.576 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:31.576 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.576 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.576 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:19877 3.5 3:a5e62864:::rbd_data.1140782590a5.000000000000000e:6 [read 962560~2048] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:31.576 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.613472+0000", 2024-05-08T08:52:31.576 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 3.8977767829999999, 2024-05-08T08:52:31.576 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.041281796000000003, 2024-05-08T08:52:31.577 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:31.577 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:31.577 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:31.577 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:31.577 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:31.577 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:31.577 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 19877 2024-05-08T08:52:31.577 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.577 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:31.577 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.577 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:31.577 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613472+0000", 2024-05-08T08:52:31.577 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:31.578 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.578 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.578 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:31.578 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613472+0000", 2024-05-08T08:52:31.578 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:31.578 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.578 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.578 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:31.578 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613473+0000", 2024-05-08T08:52:31.578 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2599999999999998e-07 2024-05-08T08:52:31.578 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.578 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.578 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:31.579 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613475+0000", 2024-05-08T08:52:31.579 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1270000000000001e-06 2024-05-08T08:52:31.579 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.579 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.579 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:31.579 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613481+0000", 2024-05-08T08:52:31.579 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1850000000000001e-06 2024-05-08T08:52:31.579 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.579 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.579 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:31.579 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613486+0000", 2024-05-08T08:52:31.580 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0180000000000001e-06 2024-05-08T08:52:31.580 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.580 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.580 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:31.580 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.654671+0000", 2024-05-08T08:52:31.580 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.041185069999999997 2024-05-08T08:52:31.580 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.580 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.580 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:31.580 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.654708+0000", 2024-05-08T08:52:31.580 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6872999999999997e-05 2024-05-08T08:52:31.580 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.580 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.580 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:31.581 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.654754+0000", 2024-05-08T08:52:31.581 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6097000000000003e-05 2024-05-08T08:52:31.581 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:31.581 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:31.581 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:31.581 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.581 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.581 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:19879 3.5 3:a5e62864:::rbd_data.1140782590a5.000000000000000e:6 [read 978944~2048] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:31.581 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.613493+0000", 2024-05-08T08:52:31.581 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 3.8977561870000001, 2024-05-08T08:52:31.581 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.041366881000000001, 2024-05-08T08:52:31.581 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:31.581 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:31.581 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:31.582 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:31.582 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:31.582 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:31.582 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 19879 2024-05-08T08:52:31.582 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.582 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:31.582 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.582 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:31.582 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613493+0000", 2024-05-08T08:52:31.582 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:31.582 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.582 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.582 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:31.582 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613493+0000", 2024-05-08T08:52:31.583 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:31.583 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.583 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.583 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:31.583 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613493+0000", 2024-05-08T08:52:31.583 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9700000000000002e-07 2024-05-08T08:52:31.583 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.583 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.583 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:31.583 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613500+0000", 2024-05-08T08:52:31.583 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7440000000000001e-06 2024-05-08T08:52:31.583 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.583 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.583 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:31.584 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613513+0000", 2024-05-08T08:52:31.584 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3375e-05 2024-05-08T08:52:31.584 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.584 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.584 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:31.584 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613519+0000", 2024-05-08T08:52:31.584 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8599999999999998e-06 2024-05-08T08:52:31.584 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.584 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.584 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:31.584 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.654772+0000", 2024-05-08T08:52:31.584 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.041252240000000003 2024-05-08T08:52:31.584 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.584 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.584 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:31.585 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.654812+0000", 2024-05-08T08:52:31.585 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0386000000000003e-05 2024-05-08T08:52:31.585 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.585 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.585 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:31.585 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.654860+0000", 2024-05-08T08:52:31.585 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7879e-05 2024-05-08T08:52:31.585 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:31.585 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:31.585 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:31.585 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.585 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.586 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:19881 3.5 3:a5e62864:::rbd_data.1140782590a5.000000000000000e:6 [read 995328~2048] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:31.586 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.613527+0000", 2024-05-08T08:52:31.586 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 3.897721652, 2024-05-08T08:52:31.586 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.041465160000000001, 2024-05-08T08:52:31.586 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:31.586 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:31.586 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:31.586 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:31.586 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:31.587 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:31.587 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 19881 2024-05-08T08:52:31.587 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.587 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:31.587 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.587 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:31.587 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613527+0000", 2024-05-08T08:52:31.587 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:31.587 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.587 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.587 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:31.587 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613527+0000", 2024-05-08T08:52:31.587 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:31.587 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.587 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.587 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:31.588 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613528+0000", 2024-05-08T08:52:31.588 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1900000000000003e-07 2024-05-08T08:52:31.588 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.588 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.588 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:31.588 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613530+0000", 2024-05-08T08:52:31.588 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2970000000000002e-06 2024-05-08T08:52:31.588 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.588 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.588 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:31.588 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613538+0000", 2024-05-08T08:52:31.588 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2849999999999997e-06 2024-05-08T08:52:31.588 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.588 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.589 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:31.589 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613543+0000", 2024-05-08T08:52:31.589 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9370000000000003e-06 2024-05-08T08:52:31.589 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.589 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.589 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:31.589 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.654879+0000", 2024-05-08T08:52:31.589 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.041336154 2024-05-08T08:52:31.589 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.589 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.589 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:31.589 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.654913+0000", 2024-05-08T08:52:31.589 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4552999999999999e-05 2024-05-08T08:52:31.589 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.589 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.589 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:31.590 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.654993+0000", 2024-05-08T08:52:31.590 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9414999999999998e-05 2024-05-08T08:52:31.590 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:31.590 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:31.590 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:31.590 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.590 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.590 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:19883 3.5 3:a5e62864:::rbd_data.1140782590a5.000000000000000e:6 [read 1011712~2048] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:31.590 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.613550+0000", 2024-05-08T08:52:31.590 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 3.897699201, 2024-05-08T08:52:31.590 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.041671055999999998, 2024-05-08T08:52:31.590 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:31.590 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:31.590 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:31.591 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:31.591 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:31.591 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:31.591 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 19883 2024-05-08T08:52:31.591 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.591 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:31.591 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.591 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:31.591 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613550+0000", 2024-05-08T08:52:31.591 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:31.591 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.591 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.591 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:31.591 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613550+0000", 2024-05-08T08:52:31.591 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:31.592 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.592 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.592 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:31.592 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613550+0000", 2024-05-08T08:52:31.592 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.27e-07 2024-05-08T08:52:31.592 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.592 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.592 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:31.592 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613552+0000", 2024-05-08T08:52:31.592 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1280000000000002e-06 2024-05-08T08:52:31.592 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.592 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.592 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:31.592 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613558+0000", 2024-05-08T08:52:31.592 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9599999999999997e-06 2024-05-08T08:52:31.592 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.593 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.593 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:31.593 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613565+0000", 2024-05-08T08:52:31.593 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.161e-06 2024-05-08T08:52:31.593 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.593 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.593 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:31.593 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.655041+0000", 2024-05-08T08:52:31.593 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.041476811000000002 2024-05-08T08:52:31.593 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.593 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.593 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:31.593 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.655120+0000", 2024-05-08T08:52:31.593 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8594e-05 2024-05-08T08:52:31.594 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.594 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.594 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:31.594 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.655221+0000", 2024-05-08T08:52:31.594 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000100975 2024-05-08T08:52:31.594 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:31.594 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:31.594 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:31.595 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.595 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.595 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:19885 3.5 3:a5e62864:::rbd_data.1140782590a5.000000000000000e:6 [read 1028096~2048] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:31.595 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.613572+0000", 2024-05-08T08:52:31.595 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 3.8976776339999999, 2024-05-08T08:52:31.595 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.041849021, 2024-05-08T08:52:31.595 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:31.595 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:31.595 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:31.596 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:31.596 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:31.596 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:31.596 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 19885 2024-05-08T08:52:31.596 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.596 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:31.596 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.596 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:31.596 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613572+0000", 2024-05-08T08:52:31.596 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:31.596 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.596 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.596 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:31.597 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613572+0000", 2024-05-08T08:52:31.597 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:31.597 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.597 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.597 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:31.597 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613572+0000", 2024-05-08T08:52:31.597 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9699999999999996e-07 2024-05-08T08:52:31.597 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.597 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.597 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:31.598 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613574+0000", 2024-05-08T08:52:31.598 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3740000000000001e-06 2024-05-08T08:52:31.598 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.598 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.598 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:31.598 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613581+0000", 2024-05-08T08:52:31.598 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.528e-06 2024-05-08T08:52:31.598 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.598 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.599 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:31.599 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613586+0000", 2024-05-08T08:52:31.599 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9540000000000003e-06 2024-05-08T08:52:31.599 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.599 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.599 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:31.599 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.655253+0000", 2024-05-08T08:52:31.599 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.041667527000000003 2024-05-08T08:52:31.599 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.599 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.599 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:31.599 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.655326+0000", 2024-05-08T08:52:31.599 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.305e-05 2024-05-08T08:52:31.599 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.600 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.600 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:31.600 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.655421+0000", 2024-05-08T08:52:31.600 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4091000000000004e-05 2024-05-08T08:52:31.600 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:31.600 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:31.600 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:31.600 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.600 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.600 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:19887 3.5 3:a5e62864:::rbd_data.1140782590a5.000000000000000e:6 [read 1044480~2048] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:31.600 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.613593+0000", 2024-05-08T08:52:31.600 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 3.8976565949999999, 2024-05-08T08:52:31.600 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.042053713, 2024-05-08T08:52:31.601 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:31.601 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:31.601 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:31.601 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:31.601 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:31.601 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:31.601 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 19887 2024-05-08T08:52:31.601 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.601 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:31.601 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.601 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:31.601 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613593+0000", 2024-05-08T08:52:31.601 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:31.602 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.602 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.602 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:31.602 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613593+0000", 2024-05-08T08:52:31.602 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:31.602 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.602 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.602 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:31.602 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613593+0000", 2024-05-08T08:52:31.602 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6800000000000001e-07 2024-05-08T08:52:31.602 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.603 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.603 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:31.603 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613595+0000", 2024-05-08T08:52:31.603 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9759999999999998e-06 2024-05-08T08:52:31.603 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.603 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.603 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:31.603 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613608+0000", 2024-05-08T08:52:31.603 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3069e-05 2024-05-08T08:52:31.603 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.603 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.603 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:31.603 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.613613+0000", 2024-05-08T08:52:31.603 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4670000000000003e-06 2024-05-08T08:52:31.603 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.604 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.604 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:31.604 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.655465+0000", 2024-05-08T08:52:31.604 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.041851501999999999 2024-05-08T08:52:31.604 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.604 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.604 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:31.604 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.655547+0000", 2024-05-08T08:52:31.604 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.2002999999999994e-05 2024-05-08T08:52:31.604 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:31.604 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:31.604 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:31.605 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.655646+0000", 2024-05-08T08:52:31.605 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.9327999999999995e-05 2024-05-08T08:52:31.605 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:31.605 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:31.605 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:31.605 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:31.605 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:31.605 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:52:31.607 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:52:31.610+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:52:31.608 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:52:31.708 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:52:31.710+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:52:31.773 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:52:31.773 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:52:31.773 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:31.773 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:8889 3.a 3.4858f32a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:31.773 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.753510+0000", 2024-05-08T08:52:31.774 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.0227038, 2024-05-08T08:52:31.774 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.022721158000000002, 2024-05-08T08:52:31.774 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:31.774 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:31.774 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:31.775 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:31.775 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:31.775 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:31.775 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 8889 2024-05-08T08:52:31.775 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:31.775 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:31.776 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:31.776 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:31.776 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.753510+0000", 2024-05-08T08:52:31.776 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:31.776 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:31.776 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:31.776 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:31.777 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.753510+0000", 2024-05-08T08:52:31.777 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:31.777 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:31.777 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:31.777 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:31.777 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.753510+0000", 2024-05-08T08:52:31.778 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0200000000000002e-07 2024-05-08T08:52:31.778 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:31.778 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:31.778 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:31.778 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.753513+0000", 2024-05-08T08:52:31.778 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5409999999999999e-06 2024-05-08T08:52:31.779 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:31.779 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:31.779 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:31.779 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.753524+0000", 2024-05-08T08:52:31.779 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1121000000000001e-05 2024-05-08T08:52:31.779 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:31.780 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:31.780 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:31.780 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.753529+0000", 2024-05-08T08:52:31.780 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6380000000000001e-06 2024-05-08T08:52:31.780 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:31.780 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:31.780 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:31.781 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:31.781 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:31.781 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:8891 3.a 3.4858f32a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:31.781 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.753538+0000", 2024-05-08T08:52:31.781 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.022675796000000002, 2024-05-08T08:52:31.781 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.022729789, 2024-05-08T08:52:31.782 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:31.782 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:31.782 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:31.782 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:31.782 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:31.782 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:31.782 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 8891 2024-05-08T08:52:31.783 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:31.783 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:31.783 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:31.783 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:31.783 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.753538+0000", 2024-05-08T08:52:31.783 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:31.783 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:31.783 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:31.784 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:31.784 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.753538+0000", 2024-05-08T08:52:31.784 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:31.784 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:31.784 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:31.784 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:31.784 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.753538+0000", 2024-05-08T08:52:31.784 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8800000000000003e-07 2024-05-08T08:52:31.785 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:31.785 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:31.785 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:31.785 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.753541+0000", 2024-05-08T08:52:31.785 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.481e-06 2024-05-08T08:52:31.785 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:31.786 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:31.786 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:31.786 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.753547+0000", 2024-05-08T08:52:31.786 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3829999999999997e-06 2024-05-08T08:52:31.786 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:31.786 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:31.787 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:31.787 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.753552+0000", 2024-05-08T08:52:31.787 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1279999999999999e-06 2024-05-08T08:52:31.787 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:31.787 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:31.787 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:31.787 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:31.788 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:31.788 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:8880 3.a 3:54cf1a12:::rbd_data.1140782590a5.000000000000000d:6 [read 3061760~2048 out=2048b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:31.788 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.753078+0000", 2024-05-08T08:52:31.788 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.023135566, 2024-05-08T08:52:31.788 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.023225407, 2024-05-08T08:52:31.788 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:31.789 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:31.789 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:31.789 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:31.789 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:31.789 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:31.789 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 8880 2024-05-08T08:52:31.790 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:31.790 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:31.790 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:31.790 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:31.790 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.753078+0000", 2024-05-08T08:52:31.790 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:31.790 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:31.790 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:31.791 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:31.791 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.753078+0000", 2024-05-08T08:52:31.791 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:31.791 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:31.791 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:31.791 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:31.792 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.753078+0000", 2024-05-08T08:52:31.792 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0299999999999999e-07 2024-05-08T08:52:31.792 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:31.792 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:31.792 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:31.792 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.753082+0000", 2024-05-08T08:52:31.792 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2260000000000001e-06 2024-05-08T08:52:31.793 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:31.793 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:31.793 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:31.793 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.753089+0000", 2024-05-08T08:52:31.793 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5800000000000003e-06 2024-05-08T08:52:31.793 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:31.793 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:31.793 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:31.793 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.753095+0000", 2024-05-08T08:52:31.794 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5570000000000003e-06 2024-05-08T08:52:31.794 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:31.794 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:31.794 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:31.794 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.776229+0000", 2024-05-08T08:52:31.794 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.023134400999999999 2024-05-08T08:52:31.794 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:31.794 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:31.795 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:31.795 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.776266+0000", 2024-05-08T08:52:31.795 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7049999999999999e-05 2024-05-08T08:52:31.795 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:31.795 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:31.795 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:31.795 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:31.795 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:31.796 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:8882 3.a 3.4858f32a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:31.796 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.753417+0000", 2024-05-08T08:52:31.796 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.022796431999999998, 2024-05-08T08:52:31.796 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.022928653, 2024-05-08T08:52:31.796 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:31.796 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:31.797 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-05-08T08:52:31.797 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:31.797 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:31.797 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:31.797 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 8882 2024-05-08T08:52:31.797 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:31.797 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:31.798 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:31.798 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:31.798 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.753417+0000", 2024-05-08T08:52:31.798 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:31.798 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:31.799 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:31.799 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:31.799 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.753417+0000", 2024-05-08T08:52:31.799 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:31.799 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:31.799 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:31.800 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:31.800 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.753418+0000", 2024-05-08T08:52:31.800 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1500000000000003e-07 2024-05-08T08:52:31.800 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:31.800 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:31.800 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:31.801 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.753423+0000", 2024-05-08T08:52:31.801 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6480000000000002e-06 2024-05-08T08:52:31.801 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:31.801 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:31.801 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:31.801 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.753435+0000", 2024-05-08T08:52:31.802 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2082000000000001e-05 2024-05-08T08:52:31.802 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:31.802 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:31.802 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:31.802 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.753442+0000", 2024-05-08T08:52:31.802 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2309999999999999e-06 2024-05-08T08:52:31.803 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:31.803 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:31.803 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:31.803 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.776345+0000", 2024-05-08T08:52:31.803 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.022902696 2024-05-08T08:52:31.803 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:31.803 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:31.803 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:31.803 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:31.803 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:31.803 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:8884 3.a 3.4858f32a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:31.804 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.753454+0000", 2024-05-08T08:52:31.804 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.022759031999999998, 2024-05-08T08:52:31.804 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.022934519, 2024-05-08T08:52:31.804 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:31.804 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:31.804 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:31.804 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:31.804 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:31.804 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:31.804 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 8884 2024-05-08T08:52:31.804 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:31.804 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:31.804 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:31.804 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:31.805 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.753454+0000", 2024-05-08T08:52:31.805 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:31.805 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:31.805 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:31.805 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:31.805 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.753454+0000", 2024-05-08T08:52:31.805 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:31.805 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:31.805 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:31.805 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:31.805 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.753455+0000", 2024-05-08T08:52:31.805 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7599999999999997e-07 2024-05-08T08:52:31.805 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:31.805 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:31.805 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:31.806 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.753458+0000", 2024-05-08T08:52:31.806 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6910000000000002e-06 2024-05-08T08:52:31.806 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:31.806 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:31.806 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:31.806 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.753465+0000", 2024-05-08T08:52:31.806 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0879999999999997e-06 2024-05-08T08:52:31.806 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:31.806 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:31.806 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:31.806 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.753470+0000", 2024-05-08T08:52:31.806 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5169999999999999e-06 2024-05-08T08:52:31.806 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:31.806 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:31.807 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:31.807 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:31.807 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:31.807 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:8887 3.a 3.4858f32a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:31.807 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.753479+0000", 2024-05-08T08:52:31.807 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.022734923000000001, 2024-05-08T08:52:31.807 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.022937521999999998, 2024-05-08T08:52:31.807 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:31.807 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:31.807 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:31.807 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:31.807 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:31.807 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:31.807 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 8887 2024-05-08T08:52:31.807 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:31.808 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:31.808 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:31.808 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:31.808 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.753479+0000", 2024-05-08T08:52:31.808 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:31.808 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:31.808 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:31.808 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:31.808 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.753479+0000", 2024-05-08T08:52:31.808 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:31.808 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:31.808 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:31.808 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:31.808 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.753479+0000", 2024-05-08T08:52:31.808 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8400000000000005e-07 2024-05-08T08:52:31.809 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:31.809 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:31.809 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:31.809 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.753481+0000", 2024-05-08T08:52:31.809 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3120000000000001e-06 2024-05-08T08:52:31.809 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:31.809 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:31.809 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:31.809 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.753489+0000", 2024-05-08T08:52:31.809 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7929999999999998e-06 2024-05-08T08:52:31.809 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:31.809 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:31.809 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:31.809 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.753494+0000", 2024-05-08T08:52:31.809 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9010000000000002e-06 2024-05-08T08:52:31.810 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:31.810 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:31.810 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:31.810 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:31.810 INFO:teuthology.orchestra.run.smithi192.stdout: ], 2024-05-08T08:52:31.810 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 6 2024-05-08T08:52:31.810 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:52:31.811 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:52:31.813+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:52:31.812 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:52:31.912 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:52:31.915+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:52:31.917 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:52:31.917 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:52:31.917 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:52:31.917 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:52:31.917 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:52:31.926 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:52:32.004 INFO:teuthology.orchestra.run.smithi059.stderr:nodeep-scrub is unset 2024-05-08T08:52:32.012 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:32.016+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:32.034 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:52:32.034 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:52:32.034 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:52:32.034 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:52:32.035 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.035 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4736 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1783808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:32.035 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.585317+0000", 2024-05-08T08:52:32.035 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.45175090699999998, 2024-05-08T08:52:32.035 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035151422000000002, 2024-05-08T08:52:32.035 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:32.035 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:32.035 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:32.035 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:32.035 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:32.035 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:32.035 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4736 2024-05-08T08:52:32.035 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.035 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:32.035 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.036 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:32.036 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585317+0000", 2024-05-08T08:52:32.036 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.036 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.036 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.036 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:32.036 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585317+0000", 2024-05-08T08:52:32.036 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.036 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.036 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.036 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:32.036 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585318+0000", 2024-05-08T08:52:32.036 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5299999999999999e-07 2024-05-08T08:52:32.036 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.037 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.037 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:32.037 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585321+0000", 2024-05-08T08:52:32.037 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9960000000000002e-06 2024-05-08T08:52:32.037 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.037 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.037 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:32.037 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585331+0000", 2024-05-08T08:52:32.037 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9550000000000004e-06 2024-05-08T08:52:32.037 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.037 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.037 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:32.037 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585336+0000", 2024-05-08T08:52:32.037 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7980000000000002e-06 2024-05-08T08:52:32.037 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.038 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.038 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:32.038 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.620316+0000", 2024-05-08T08:52:32.038 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.034979606000000003 2024-05-08T08:52:32.038 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.038 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.038 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:32.038 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.620383+0000", 2024-05-08T08:52:32.038 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7387999999999999e-05 2024-05-08T08:52:32.038 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.038 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.038 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:32.038 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.620469+0000", 2024-05-08T08:52:32.038 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5226e-05 2024-05-08T08:52:32.038 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.039 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:32.039 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.039 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.039 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.039 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4738 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1800192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:32.039 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.585359+0000", 2024-05-08T08:52:32.039 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.451709113, 2024-05-08T08:52:32.039 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035337788000000002, 2024-05-08T08:52:32.039 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:32.039 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:32.039 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:32.039 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:32.039 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:32.039 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:32.039 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4738 2024-05-08T08:52:32.039 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.040 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:32.040 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.040 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:32.040 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585359+0000", 2024-05-08T08:52:32.040 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.040 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.040 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.040 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:32.040 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585359+0000", 2024-05-08T08:52:32.040 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.040 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.040 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.040 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:32.040 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585360+0000", 2024-05-08T08:52:32.040 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6899999999999998e-07 2024-05-08T08:52:32.040 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.040 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.041 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:32.041 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585362+0000", 2024-05-08T08:52:32.041 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4739999999999999e-06 2024-05-08T08:52:32.041 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.041 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.041 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:32.041 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585369+0000", 2024-05-08T08:52:32.041 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9310000000000002e-06 2024-05-08T08:52:32.041 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.041 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.041 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:32.041 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585375+0000", 2024-05-08T08:52:32.041 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9599999999999997e-06 2024-05-08T08:52:32.041 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.041 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.041 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:32.041 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.620510+0000", 2024-05-08T08:52:32.042 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035135176999999997 2024-05-08T08:52:32.042 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.042 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.042 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:32.042 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.620593+0000", 2024-05-08T08:52:32.042 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3057000000000001e-05 2024-05-08T08:52:32.042 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.042 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.042 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:32.042 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.620697+0000", 2024-05-08T08:52:32.042 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010372 2024-05-08T08:52:32.042 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.042 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:32.042 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.043 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.043 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.043 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4740 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1816576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:32.043 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.585472+0000", 2024-05-08T08:52:32.043 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.45159649099999999, 2024-05-08T08:52:32.043 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035430959999999997, 2024-05-08T08:52:32.043 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:32.043 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:32.043 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:32.043 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:32.043 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:32.043 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:32.043 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4740 2024-05-08T08:52:32.043 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.043 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:32.044 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.044 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:32.044 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585472+0000", 2024-05-08T08:52:32.044 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.044 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.044 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.044 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:32.044 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585472+0000", 2024-05-08T08:52:32.044 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.044 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.044 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.044 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:32.044 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585472+0000", 2024-05-08T08:52:32.044 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3200000000000005e-07 2024-05-08T08:52:32.045 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.045 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.045 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:32.045 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585476+0000", 2024-05-08T08:52:32.045 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2890000000000002e-06 2024-05-08T08:52:32.045 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.045 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.045 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:32.045 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585484+0000", 2024-05-08T08:52:32.045 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9589999999999995e-06 2024-05-08T08:52:32.045 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.045 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.045 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:32.046 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585489+0000", 2024-05-08T08:52:32.046 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4920000000000001e-06 2024-05-08T08:52:32.046 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.046 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.046 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:32.046 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.620728+0000", 2024-05-08T08:52:32.046 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035239165000000003 2024-05-08T08:52:32.046 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.046 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.046 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:32.046 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.620809+0000", 2024-05-08T08:52:32.046 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0611999999999993e-05 2024-05-08T08:52:32.047 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.047 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.047 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:32.047 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.620903+0000", 2024-05-08T08:52:32.047 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3811e-05 2024-05-08T08:52:32.047 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.047 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:32.047 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.047 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.047 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.047 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4742 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1832960~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:32.047 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.585513+0000", 2024-05-08T08:52:32.048 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.45155481800000002, 2024-05-08T08:52:32.048 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035588763000000002, 2024-05-08T08:52:32.048 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:32.048 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:32.048 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:32.048 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:32.048 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:32.048 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:32.048 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4742 2024-05-08T08:52:32.048 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.048 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:32.049 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.049 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:32.049 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585513+0000", 2024-05-08T08:52:32.049 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.049 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.049 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.049 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:32.049 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585513+0000", 2024-05-08T08:52:32.049 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.050 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.050 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.050 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:32.050 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585514+0000", 2024-05-08T08:52:32.050 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6100000000000001e-07 2024-05-08T08:52:32.050 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.050 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.050 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:32.050 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585516+0000", 2024-05-08T08:52:32.050 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.526e-06 2024-05-08T08:52:32.051 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.051 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.051 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:32.051 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585523+0000", 2024-05-08T08:52:32.051 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.562e-06 2024-05-08T08:52:32.051 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.051 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.051 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:32.051 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585528+0000", 2024-05-08T08:52:32.051 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7990000000000001e-06 2024-05-08T08:52:32.051 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.051 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.052 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:32.052 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.620939+0000", 2024-05-08T08:52:32.052 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035411419 2024-05-08T08:52:32.052 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.052 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.052 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:32.052 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621014+0000", 2024-05-08T08:52:32.052 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5017000000000004e-05 2024-05-08T08:52:32.052 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.052 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.052 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:32.053 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621102+0000", 2024-05-08T08:52:32.053 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7979000000000006e-05 2024-05-08T08:52:32.053 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.053 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:32.053 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.053 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.053 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.053 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4743 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1849344~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:32.053 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.585628+0000", 2024-05-08T08:52:32.053 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.45143993100000002, 2024-05-08T08:52:32.053 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035659962000000003, 2024-05-08T08:52:32.053 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:32.054 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:32.054 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:32.054 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:32.054 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:32.054 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:32.054 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4743 2024-05-08T08:52:32.054 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.054 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:32.054 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.054 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:32.054 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585628+0000", 2024-05-08T08:52:32.055 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.055 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.055 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.055 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:32.055 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585628+0000", 2024-05-08T08:52:32.055 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.055 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.055 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.055 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:32.055 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585629+0000", 2024-05-08T08:52:32.055 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2300000000000001e-07 2024-05-08T08:52:32.056 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.056 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.056 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:32.056 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585632+0000", 2024-05-08T08:52:32.056 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1370000000000002e-06 2024-05-08T08:52:32.056 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.056 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.056 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:32.056 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585641+0000", 2024-05-08T08:52:32.056 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5220000000000007e-06 2024-05-08T08:52:32.056 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.056 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.057 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:32.057 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585646+0000", 2024-05-08T08:52:32.057 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.609e-06 2024-05-08T08:52:32.057 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.057 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.057 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:32.057 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621132+0000", 2024-05-08T08:52:32.057 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035485472999999997 2024-05-08T08:52:32.057 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.057 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.057 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:32.058 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621204+0000", 2024-05-08T08:52:32.058 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2259000000000001e-05 2024-05-08T08:52:32.058 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.058 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.058 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:32.058 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621288+0000", 2024-05-08T08:52:32.058 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4338999999999993e-05 2024-05-08T08:52:32.058 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.058 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:32.058 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.058 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.059 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.059 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4746 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1865728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:32.059 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.585656+0000", 2024-05-08T08:52:32.059 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.45141224899999999, 2024-05-08T08:52:32.059 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035815909, 2024-05-08T08:52:32.059 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:32.059 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:32.059 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:32.059 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:32.059 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:32.059 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:32.059 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4746 2024-05-08T08:52:32.059 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.059 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:32.060 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.060 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:32.060 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585656+0000", 2024-05-08T08:52:32.060 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.060 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.060 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.060 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:32.060 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585656+0000", 2024-05-08T08:52:32.060 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.060 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.060 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.060 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:32.060 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585656+0000", 2024-05-08T08:52:32.060 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.08e-07 2024-05-08T08:52:32.061 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.061 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.061 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:32.061 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585658+0000", 2024-05-08T08:52:32.061 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0889999999999998e-06 2024-05-08T08:52:32.061 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.061 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.061 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:32.061 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585664+0000", 2024-05-08T08:52:32.061 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3990000000000003e-06 2024-05-08T08:52:32.061 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.061 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.061 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:32.062 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585668+0000", 2024-05-08T08:52:32.062 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3170000000000003e-06 2024-05-08T08:52:32.062 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.062 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.062 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:32.062 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621316+0000", 2024-05-08T08:52:32.062 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035647908999999998 2024-05-08T08:52:32.062 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.062 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.062 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:32.062 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621389+0000", 2024-05-08T08:52:32.062 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2531000000000001e-05 2024-05-08T08:52:32.062 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.063 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.063 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:32.063 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621472+0000", 2024-05-08T08:52:32.063 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3256000000000004e-05 2024-05-08T08:52:32.063 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.063 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:32.063 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.063 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.063 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.063 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4748 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1882112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:32.063 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.585697+0000", 2024-05-08T08:52:32.064 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.45137080299999999, 2024-05-08T08:52:32.064 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035962485000000002, 2024-05-08T08:52:32.064 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:32.064 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:32.064 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:32.064 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:32.064 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:32.064 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:32.064 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4748 2024-05-08T08:52:32.064 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.065 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:32.065 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.065 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:32.065 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585697+0000", 2024-05-08T08:52:32.065 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.065 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.065 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.065 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:32.065 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585697+0000", 2024-05-08T08:52:32.065 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.065 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.065 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.065 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:32.066 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585698+0000", 2024-05-08T08:52:32.066 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0900000000000001e-07 2024-05-08T08:52:32.066 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.066 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.066 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:32.066 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585701+0000", 2024-05-08T08:52:32.066 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0149999999999999e-06 2024-05-08T08:52:32.066 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.066 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.066 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:32.066 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585712+0000", 2024-05-08T08:52:32.066 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0655e-05 2024-05-08T08:52:32.066 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.066 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.067 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:32.067 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585717+0000", 2024-05-08T08:52:32.067 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3800000000000002e-06 2024-05-08T08:52:32.067 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.067 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.067 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:32.067 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621502+0000", 2024-05-08T08:52:32.067 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035784559000000001 2024-05-08T08:52:32.067 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.067 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.067 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:32.067 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621575+0000", 2024-05-08T08:52:32.067 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3269999999999995e-05 2024-05-08T08:52:32.067 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.067 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.067 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:32.068 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621660+0000", 2024-05-08T08:52:32.068 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4997e-05 2024-05-08T08:52:32.068 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.068 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:32.068 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.068 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.068 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.068 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4750 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1898496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:32.068 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.585810+0000", 2024-05-08T08:52:32.068 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.45125797299999998, 2024-05-08T08:52:32.068 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036034928000000001, 2024-05-08T08:52:32.068 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:32.068 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:32.068 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:32.068 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:32.068 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:32.069 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:32.069 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4750 2024-05-08T08:52:32.069 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.069 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:32.069 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.069 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:32.069 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585810+0000", 2024-05-08T08:52:32.069 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.069 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.069 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.069 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:32.069 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585810+0000", 2024-05-08T08:52:32.069 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.069 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.069 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.069 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:32.069 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585811+0000", 2024-05-08T08:52:32.070 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6499999999999999e-07 2024-05-08T08:52:32.070 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.070 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.070 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:32.070 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585814+0000", 2024-05-08T08:52:32.070 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1709999999999998e-06 2024-05-08T08:52:32.070 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.070 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.070 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:32.070 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585822+0000", 2024-05-08T08:52:32.070 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8909999999999995e-06 2024-05-08T08:52:32.070 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.070 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.070 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:32.070 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585827+0000", 2024-05-08T08:52:32.070 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5289999999999999e-06 2024-05-08T08:52:32.071 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.071 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.071 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:32.071 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621688+0000", 2024-05-08T08:52:32.071 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035861111000000001 2024-05-08T08:52:32.071 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.071 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.071 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:32.071 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621761+0000", 2024-05-08T08:52:32.071 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2211000000000004e-05 2024-05-08T08:52:32.071 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.071 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.071 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:32.071 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621845+0000", 2024-05-08T08:52:32.071 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4450000000000006e-05 2024-05-08T08:52:32.071 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.072 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:32.072 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.072 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.072 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.072 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4752 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1914880~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:32.072 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.585858+0000", 2024-05-08T08:52:32.072 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.451210004, 2024-05-08T08:52:32.072 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036180081000000003, 2024-05-08T08:52:32.072 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:32.072 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:32.072 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:32.072 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:32.072 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:32.072 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:32.072 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4752 2024-05-08T08:52:32.073 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.073 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:32.073 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.073 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:32.073 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585858+0000", 2024-05-08T08:52:32.073 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.073 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.073 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.073 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:32.073 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585858+0000", 2024-05-08T08:52:32.073 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.073 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.073 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.073 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:32.073 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585859+0000", 2024-05-08T08:52:32.073 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.82e-07 2024-05-08T08:52:32.074 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.074 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.074 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:32.074 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585861+0000", 2024-05-08T08:52:32.074 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.695e-06 2024-05-08T08:52:32.074 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.074 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.074 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:32.074 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585869+0000", 2024-05-08T08:52:32.074 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0299999999999994e-06 2024-05-08T08:52:32.074 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.074 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.074 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:32.074 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585874+0000", 2024-05-08T08:52:32.074 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9620000000000001e-06 2024-05-08T08:52:32.075 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.075 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.075 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:32.075 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621878+0000", 2024-05-08T08:52:32.075 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036004043999999999 2024-05-08T08:52:32.075 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.075 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.075 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:32.075 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621954+0000", 2024-05-08T08:52:32.075 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5414000000000007e-05 2024-05-08T08:52:32.075 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.075 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.075 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:32.075 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622038+0000", 2024-05-08T08:52:32.076 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4454000000000002e-05 2024-05-08T08:52:32.076 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.076 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:32.076 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.076 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.076 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.076 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4753 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1931264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:32.076 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.585947+0000", 2024-05-08T08:52:32.076 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.45112135599999997, 2024-05-08T08:52:32.076 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036267992999999998, 2024-05-08T08:52:32.076 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:32.076 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:32.076 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:32.076 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:32.077 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:32.077 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:32.077 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4753 2024-05-08T08:52:32.077 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.077 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:32.077 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.077 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:32.077 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585947+0000", 2024-05-08T08:52:32.077 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.077 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.077 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.077 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:32.077 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585947+0000", 2024-05-08T08:52:32.077 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.078 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.078 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.078 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:32.078 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585948+0000", 2024-05-08T08:52:32.078 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6000000000000002e-07 2024-05-08T08:52:32.078 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.078 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.078 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:32.078 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585952+0000", 2024-05-08T08:52:32.078 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7090000000000001e-06 2024-05-08T08:52:32.078 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.078 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.078 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:32.078 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585963+0000", 2024-05-08T08:52:32.079 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0740000000000001e-05 2024-05-08T08:52:32.079 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.079 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.079 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:32.079 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585970+0000", 2024-05-08T08:52:32.079 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3309999999999998e-06 2024-05-08T08:52:32.079 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.079 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.079 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:32.079 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622065+0000", 2024-05-08T08:52:32.079 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036094676999999999 2024-05-08T08:52:32.079 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.079 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.079 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:32.080 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622132+0000", 2024-05-08T08:52:32.080 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7214999999999999e-05 2024-05-08T08:52:32.080 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.080 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.080 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:32.080 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622215+0000", 2024-05-08T08:52:32.080 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2460999999999995e-05 2024-05-08T08:52:32.080 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.080 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:32.080 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.080 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.080 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.080 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4756 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1947648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:32.080 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586018+0000", 2024-05-08T08:52:32.081 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.45104971199999999, 2024-05-08T08:52:32.081 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036382335000000002, 2024-05-08T08:52:32.081 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:32.081 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:32.081 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:32.081 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:32.081 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:32.081 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:32.081 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4756 2024-05-08T08:52:32.081 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.081 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:32.081 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.081 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:32.081 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586018+0000", 2024-05-08T08:52:32.081 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.082 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.082 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.082 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:32.082 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586018+0000", 2024-05-08T08:52:32.082 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.082 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.082 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.082 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:32.082 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586019+0000", 2024-05-08T08:52:32.082 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9999999999999997e-07 2024-05-08T08:52:32.082 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.082 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.082 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:32.082 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586023+0000", 2024-05-08T08:52:32.083 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7330000000000001e-06 2024-05-08T08:52:32.083 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.083 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.083 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:32.083 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586031+0000", 2024-05-08T08:52:32.083 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4940000000000003e-06 2024-05-08T08:52:32.083 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.083 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.083 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:32.083 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586038+0000", 2024-05-08T08:52:32.083 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2539999999999998e-06 2024-05-08T08:52:32.083 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.083 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.083 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:32.084 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622243+0000", 2024-05-08T08:52:32.084 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036205009000000003 2024-05-08T08:52:32.084 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.084 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.084 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:32.084 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622313+0000", 2024-05-08T08:52:32.084 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9912000000000004e-05 2024-05-08T08:52:32.084 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.084 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.084 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:32.084 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622401+0000", 2024-05-08T08:52:32.084 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8232999999999994e-05 2024-05-08T08:52:32.084 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.084 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:32.084 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.085 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.085 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.085 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4758 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1964032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:32.085 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586080+0000", 2024-05-08T08:52:32.085 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.45098829299999998, 2024-05-08T08:52:32.085 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036547659000000003, 2024-05-08T08:52:32.085 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:32.085 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:32.085 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:32.085 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:32.085 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:32.085 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:32.085 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4758 2024-05-08T08:52:32.085 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.085 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:32.086 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.086 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:32.086 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586080+0000", 2024-05-08T08:52:32.086 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.086 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.086 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.086 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:32.086 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586080+0000", 2024-05-08T08:52:32.086 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.086 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.086 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.086 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:32.086 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586081+0000", 2024-05-08T08:52:32.086 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5499999999999998e-07 2024-05-08T08:52:32.087 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.087 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.087 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:32.087 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586084+0000", 2024-05-08T08:52:32.087 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9919999999999999e-06 2024-05-08T08:52:32.087 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.087 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.087 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:32.087 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586091+0000", 2024-05-08T08:52:32.087 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5440000000000001e-06 2024-05-08T08:52:32.087 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.087 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.087 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:32.087 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586098+0000", 2024-05-08T08:52:32.088 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8639999999999998e-06 2024-05-08T08:52:32.088 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.088 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.088 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:32.088 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622442+0000", 2024-05-08T08:52:32.088 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036344110999999998 2024-05-08T08:52:32.088 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.088 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.088 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:32.088 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622516+0000", 2024-05-08T08:52:32.088 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4089000000000002e-05 2024-05-08T08:52:32.088 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.088 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.088 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:32.088 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622628+0000", 2024-05-08T08:52:32.089 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00011140400000000001 2024-05-08T08:52:32.089 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.089 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:32.089 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.089 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.089 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.089 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4760 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1980416~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:32.089 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586178+0000", 2024-05-08T08:52:32.089 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.45088968099999999, 2024-05-08T08:52:32.089 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036647526999999999, 2024-05-08T08:52:32.089 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:32.089 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:32.089 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:32.089 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:32.089 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:32.090 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:32.090 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4760 2024-05-08T08:52:32.090 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.090 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:32.090 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.090 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:32.090 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586178+0000", 2024-05-08T08:52:32.090 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.090 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.090 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.090 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:32.090 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586178+0000", 2024-05-08T08:52:32.090 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.090 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.091 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.091 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:32.091 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586179+0000", 2024-05-08T08:52:32.091 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8599999999999997e-07 2024-05-08T08:52:32.091 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.091 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.091 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:32.091 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586182+0000", 2024-05-08T08:52:32.091 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.174e-06 2024-05-08T08:52:32.091 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.091 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.091 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:32.091 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586191+0000", 2024-05-08T08:52:32.091 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1869999999999997e-06 2024-05-08T08:52:32.092 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.092 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.092 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:32.092 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586197+0000", 2024-05-08T08:52:32.092 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2090000000000002e-06 2024-05-08T08:52:32.092 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.092 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.092 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:32.092 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622657+0000", 2024-05-08T08:52:32.092 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036459793999999997 2024-05-08T08:52:32.092 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.092 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.092 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:32.092 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622725+0000", 2024-05-08T08:52:32.093 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8374999999999995e-05 2024-05-08T08:52:32.093 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.093 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.093 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:32.093 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622826+0000", 2024-05-08T08:52:32.093 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010100200000000001 2024-05-08T08:52:32.093 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.093 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:32.093 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.093 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.093 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.093 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4762 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1996800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:32.093 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586299+0000", 2024-05-08T08:52:32.093 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.45076935000000001, 2024-05-08T08:52:32.094 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036720653999999998, 2024-05-08T08:52:32.094 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:32.094 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:32.094 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:32.094 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:32.094 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:32.094 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:32.094 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4762 2024-05-08T08:52:32.094 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.094 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:32.094 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.094 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:32.094 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586299+0000", 2024-05-08T08:52:32.094 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.095 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.095 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.095 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:32.095 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586299+0000", 2024-05-08T08:52:32.095 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.095 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.095 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.095 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:32.095 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586299+0000", 2024-05-08T08:52:32.095 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.99e-07 2024-05-08T08:52:32.095 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.095 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.095 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:32.095 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586303+0000", 2024-05-08T08:52:32.095 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1949999999999999e-06 2024-05-08T08:52:32.096 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.096 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.096 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:32.096 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586318+0000", 2024-05-08T08:52:32.096 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5401999999999999e-05 2024-05-08T08:52:32.096 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.096 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.096 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:32.096 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586324+0000", 2024-05-08T08:52:32.096 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8119999999999996e-06 2024-05-08T08:52:32.096 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.096 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.096 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:32.096 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622853+0000", 2024-05-08T08:52:32.096 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036529013999999999 2024-05-08T08:52:32.097 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.097 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.097 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:32.097 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622924+0000", 2024-05-08T08:52:32.097 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1654000000000002e-05 2024-05-08T08:52:32.097 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.097 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.097 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:32.097 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623019+0000", 2024-05-08T08:52:32.097 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4977999999999998e-05 2024-05-08T08:52:32.097 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.097 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:32.098 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.098 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.098 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.098 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4764 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 2013184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:32.098 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586506+0000", 2024-05-08T08:52:32.098 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.45056224499999997, 2024-05-08T08:52:32.098 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036859467, 2024-05-08T08:52:32.098 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:32.098 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:32.098 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:32.098 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:32.098 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:32.098 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:32.098 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4764 2024-05-08T08:52:32.099 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.099 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:32.099 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.099 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:32.099 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586506+0000", 2024-05-08T08:52:32.099 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.099 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.099 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.099 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:32.099 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586506+0000", 2024-05-08T08:52:32.099 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.099 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.099 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.099 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:32.099 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586507+0000", 2024-05-08T08:52:32.100 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9100000000000003e-07 2024-05-08T08:52:32.100 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.100 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.100 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:32.100 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586512+0000", 2024-05-08T08:52:32.100 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9470000000000002e-06 2024-05-08T08:52:32.100 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.100 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.100 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:32.100 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586521+0000", 2024-05-08T08:52:32.100 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3379999999999994e-06 2024-05-08T08:52:32.100 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.100 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.100 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:32.100 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586528+0000", 2024-05-08T08:52:32.101 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1289999999999998e-06 2024-05-08T08:52:32.101 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.101 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.101 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:32.101 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623095+0000", 2024-05-08T08:52:32.101 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036566411 2024-05-08T08:52:32.101 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.101 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.101 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:32.101 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623221+0000", 2024-05-08T08:52:32.101 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012622299999999999 2024-05-08T08:52:32.101 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.101 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.101 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:32.101 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623365+0000", 2024-05-08T08:52:32.102 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000144628 2024-05-08T08:52:32.102 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.102 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:32.102 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.102 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.102 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.102 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4766 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 2029568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:32.102 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586543+0000", 2024-05-08T08:52:32.102 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.45052475800000003, 2024-05-08T08:52:32.102 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.037033635000000002, 2024-05-08T08:52:32.102 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:32.102 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:32.102 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:32.102 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:32.103 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:32.103 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:32.103 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4766 2024-05-08T08:52:32.103 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.103 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:32.103 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.103 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:32.103 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586543+0000", 2024-05-08T08:52:32.103 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.103 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.103 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.103 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:32.103 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586543+0000", 2024-05-08T08:52:32.103 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.104 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.104 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.104 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:32.104 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586544+0000", 2024-05-08T08:52:32.104 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3099999999999998e-07 2024-05-08T08:52:32.104 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.104 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.104 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:32.104 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586547+0000", 2024-05-08T08:52:32.104 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.751e-06 2024-05-08T08:52:32.104 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.104 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.104 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:32.104 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586553+0000", 2024-05-08T08:52:32.104 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2870000000000001e-06 2024-05-08T08:52:32.105 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.105 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.105 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:32.105 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586558+0000", 2024-05-08T08:52:32.105 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5709999999999997e-06 2024-05-08T08:52:32.105 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.105 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.105 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:32.105 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623416+0000", 2024-05-08T08:52:32.105 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036857613999999997 2024-05-08T08:52:32.105 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.105 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.105 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:32.105 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623489+0000", 2024-05-08T08:52:32.106 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3172999999999999e-05 2024-05-08T08:52:32.106 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.106 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.106 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:32.106 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623577+0000", 2024-05-08T08:52:32.106 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7807999999999997e-05 2024-05-08T08:52:32.106 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.106 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:32.106 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.106 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.106 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.106 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4768 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 2045952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:32.106 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586566+0000", 2024-05-08T08:52:32.107 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.450502452, 2024-05-08T08:52:32.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.03726206, 2024-05-08T08:52:32.107 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:32.107 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:32.107 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:32.107 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:32.107 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:32.107 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:32.107 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4768 2024-05-08T08:52:32.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.107 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:32.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.107 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:32.107 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586566+0000", 2024-05-08T08:52:32.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.108 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.108 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.108 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:32.108 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586566+0000", 2024-05-08T08:52:32.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.108 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.108 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.108 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:32.108 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586566+0000", 2024-05-08T08:52:32.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8299999999999998e-07 2024-05-08T08:52:32.108 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.108 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.108 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:32.109 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586568+0000", 2024-05-08T08:52:32.109 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.063e-06 2024-05-08T08:52:32.109 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.109 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.109 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:32.109 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586574+0000", 2024-05-08T08:52:32.109 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8300000000000001e-06 2024-05-08T08:52:32.109 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.110 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.110 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:32.110 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586579+0000", 2024-05-08T08:52:32.110 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7559999999999999e-06 2024-05-08T08:52:32.110 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.110 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.110 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:32.110 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623612+0000", 2024-05-08T08:52:32.110 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.037033041000000003 2024-05-08T08:52:32.110 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.110 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.110 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:32.110 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623699+0000", 2024-05-08T08:52:32.110 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6817000000000006e-05 2024-05-08T08:52:32.111 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.111 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.111 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:32.111 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623828+0000", 2024-05-08T08:52:32.111 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012917000000000001 2024-05-08T08:52:32.111 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.111 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:32.111 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.111 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.111 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.111 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4770 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 2062336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:32.111 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586585+0000", 2024-05-08T08:52:32.111 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.45048326900000002, 2024-05-08T08:52:32.111 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.037449891999999999, 2024-05-08T08:52:32.111 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:32.112 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:32.112 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:32.112 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:32.112 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:32.112 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:32.112 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4770 2024-05-08T08:52:32.112 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.112 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:32.112 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.112 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:32.112 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586585+0000", 2024-05-08T08:52:32.112 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.112 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.112 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.113 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:32.113 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586585+0000", 2024-05-08T08:52:32.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.113 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.113 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.113 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:32.113 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586585+0000", 2024-05-08T08:52:32.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:52:32.113 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.113 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.113 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:32.113 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586587+0000", 2024-05-08T08:52:32.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.181e-06 2024-05-08T08:52:32.113 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.113 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.114 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:32.114 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586592+0000", 2024-05-08T08:52:32.114 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9710000000000003e-06 2024-05-08T08:52:32.114 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.114 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.114 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:32.114 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586598+0000", 2024-05-08T08:52:32.114 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.118e-06 2024-05-08T08:52:32.114 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.114 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.114 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:32.114 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623862+0000", 2024-05-08T08:52:32.114 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.037264711999999998 2024-05-08T08:52:32.114 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.114 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.115 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:32.115 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623941+0000", 2024-05-08T08:52:32.115 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9084000000000004e-05 2024-05-08T08:52:32.115 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.115 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.115 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:32.115 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.624035+0000", 2024-05-08T08:52:32.115 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3456999999999996e-05 2024-05-08T08:52:32.115 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.115 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:32.115 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.115 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.115 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.115 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4772 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 2078720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:32.115 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586621+0000", 2024-05-08T08:52:32.116 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.45044732100000001, 2024-05-08T08:52:32.116 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.037650391999999998, 2024-05-08T08:52:32.116 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:32.116 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:32.116 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:32.116 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:32.116 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:32.116 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:32.116 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4772 2024-05-08T08:52:32.116 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.116 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:32.116 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.116 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:32.116 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586621+0000", 2024-05-08T08:52:32.116 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.117 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.117 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.117 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:32.117 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586621+0000", 2024-05-08T08:52:32.117 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.117 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.117 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.117 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:32.117 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586621+0000", 2024-05-08T08:52:32.117 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:52:32.117 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.117 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.117 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:32.117 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586623+0000", 2024-05-08T08:52:32.118 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9240000000000001e-06 2024-05-08T08:52:32.118 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.118 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.118 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:32.118 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586628+0000", 2024-05-08T08:52:32.118 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6410000000000001e-06 2024-05-08T08:52:32.118 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.118 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.118 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:32.118 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586633+0000", 2024-05-08T08:52:32.118 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.763e-06 2024-05-08T08:52:32.118 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.118 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.118 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:32.119 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.624077+0000", 2024-05-08T08:52:32.119 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.037444709999999999 2024-05-08T08:52:32.119 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.119 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.119 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:32.119 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.624164+0000", 2024-05-08T08:52:32.119 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6769999999999998e-05 2024-05-08T08:52:32.119 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.119 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.119 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:32.119 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.624271+0000", 2024-05-08T08:52:32.119 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010723 2024-05-08T08:52:32.119 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.119 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:32.119 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.120 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.120 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.120 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4774 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 2095104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:32.120 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586640+0000", 2024-05-08T08:52:32.120 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.45042818299999998, 2024-05-08T08:52:32.120 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.037827178000000003, 2024-05-08T08:52:32.120 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:32.120 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:32.120 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:32.120 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:32.120 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:32.120 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:32.120 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4774 2024-05-08T08:52:32.120 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.120 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:32.121 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.121 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:32.121 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586640+0000", 2024-05-08T08:52:32.121 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.121 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.121 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.121 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:32.121 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586640+0000", 2024-05-08T08:52:32.121 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.121 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.121 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.121 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:32.121 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586640+0000", 2024-05-08T08:52:32.121 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5199999999999998e-07 2024-05-08T08:52:32.122 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.122 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.122 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:32.122 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586643+0000", 2024-05-08T08:52:32.122 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.232e-06 2024-05-08T08:52:32.122 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.122 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.122 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:32.122 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586649+0000", 2024-05-08T08:52:32.122 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9660000000000001e-06 2024-05-08T08:52:32.122 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.122 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.122 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:32.122 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586653+0000", 2024-05-08T08:52:32.122 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7350000000000004e-06 2024-05-08T08:52:32.123 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.123 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.123 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:32.123 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.624303+0000", 2024-05-08T08:52:32.123 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.037649647000000001 2024-05-08T08:52:32.123 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.123 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.123 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:32.123 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.624373+0000", 2024-05-08T08:52:32.123 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0415999999999998e-05 2024-05-08T08:52:32.123 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.123 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.123 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:32.123 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.624467+0000", 2024-05-08T08:52:32.123 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3830000000000001e-05 2024-05-08T08:52:32.124 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.124 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:32.124 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.124 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.124 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:32.124 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:52:32.126 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:52:32.126 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:52:32.128+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:52:32.226 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:32.229+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:32.271 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:52:32.271 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:52:32.271 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:52:32.271 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:52:32.279 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:52:32.327 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:52:32.330+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:52:32.425 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:52:32.425 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:52:32.425 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:52:32.425 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:52:32.425 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:52:32.428 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:32.431+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:32.433 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:52:32.529 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:32.533+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:32.584 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:52:32.584 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:52:32.584 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:52:32.584 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:52:32.584 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.585 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22069 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 749568~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:32.585 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857624+0000", 2024-05-08T08:52:32.585 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.7293981289999998, 2024-05-08T08:52:32.585 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046969958999999999, 2024-05-08T08:52:32.585 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:32.585 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:32.585 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:32.585 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:32.585 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:32.585 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:32.585 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22069 2024-05-08T08:52:32.585 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.585 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:32.585 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.585 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:32.586 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857624+0000", 2024-05-08T08:52:32.586 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.586 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.586 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.586 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:32.586 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857624+0000", 2024-05-08T08:52:32.586 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.586 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.586 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.586 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:32.586 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857625+0000", 2024-05-08T08:52:32.586 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.03e-07 2024-05-08T08:52:32.586 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.586 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.586 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:32.586 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857627+0000", 2024-05-08T08:52:32.587 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9369999999999998e-06 2024-05-08T08:52:32.587 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.587 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.587 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:32.587 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857633+0000", 2024-05-08T08:52:32.587 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1229999999999997e-06 2024-05-08T08:52:32.587 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.587 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.587 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:32.587 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857637+0000", 2024-05-08T08:52:32.587 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7820000000000001e-06 2024-05-08T08:52:32.587 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.587 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.587 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:32.587 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904457+0000", 2024-05-08T08:52:32.587 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046819715999999997 2024-05-08T08:52:32.588 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.588 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.588 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:32.588 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904524+0000", 2024-05-08T08:52:32.588 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6371000000000005e-05 2024-05-08T08:52:32.588 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.588 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.588 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:32.588 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904594+0000", 2024-05-08T08:52:32.588 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0627e-05 2024-05-08T08:52:32.588 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.588 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:32.588 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.588 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.588 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.588 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22071 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 765952~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:32.589 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857643+0000", 2024-05-08T08:52:32.589 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.729379046, 2024-05-08T08:52:32.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047106528000000002, 2024-05-08T08:52:32.589 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:32.589 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:32.589 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:32.589 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:32.589 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:32.589 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:32.589 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22071 2024-05-08T08:52:32.589 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.589 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:32.589 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.589 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:32.589 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857643+0000", 2024-05-08T08:52:32.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.590 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.590 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.590 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:32.590 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857643+0000", 2024-05-08T08:52:32.590 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.590 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.590 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.590 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:32.590 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857644+0000", 2024-05-08T08:52:32.590 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2000000000000001e-07 2024-05-08T08:52:32.590 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.590 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.590 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:32.590 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857646+0000", 2024-05-08T08:52:32.590 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0059999999999999e-06 2024-05-08T08:52:32.590 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.591 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.591 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:32.591 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857651+0000", 2024-05-08T08:52:32.591 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.84e-06 2024-05-08T08:52:32.591 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.591 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.591 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:32.591 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857657+0000", 2024-05-08T08:52:32.591 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5620000000000003e-06 2024-05-08T08:52:32.591 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.591 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.591 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:32.591 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904620+0000", 2024-05-08T08:52:32.591 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046962661000000003 2024-05-08T08:52:32.591 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.592 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.592 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:32.592 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904678+0000", 2024-05-08T08:52:32.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8656999999999998e-05 2024-05-08T08:52:32.592 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.592 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.592 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:32.592 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904750+0000", 2024-05-08T08:52:32.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1482000000000004e-05 2024-05-08T08:52:32.592 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.592 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:32.592 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.592 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.592 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.592 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22073 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 782336~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:32.592 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:52:32.593 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.7293550379999996, 2024-05-08T08:52:32.593 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047244234000000003, 2024-05-08T08:52:32.593 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:32.593 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:32.593 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:32.593 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:32.593 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:32.593 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:32.593 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22073 2024-05-08T08:52:32.593 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.593 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:32.593 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.593 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:32.593 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:52:32.593 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.593 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.594 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.594 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:32.594 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:52:32.594 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.594 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.594 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.594 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:32.594 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857668+0000", 2024-05-08T08:52:32.594 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7800000000000002e-07 2024-05-08T08:52:32.594 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.594 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.594 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:32.594 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857670+0000", 2024-05-08T08:52:32.594 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0820000000000001e-06 2024-05-08T08:52:32.594 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.594 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.595 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:32.595 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857678+0000", 2024-05-08T08:52:32.595 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7589999999999997e-06 2024-05-08T08:52:32.595 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.595 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.595 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:32.595 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857691+0000", 2024-05-08T08:52:32.595 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3672999999999999e-05 2024-05-08T08:52:32.595 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.595 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.595 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:32.595 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904780+0000", 2024-05-08T08:52:32.595 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047088616999999999 2024-05-08T08:52:32.595 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.595 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.595 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:32.596 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904840+0000", 2024-05-08T08:52:32.596 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0356999999999998e-05 2024-05-08T08:52:32.596 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.596 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:32.596 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904912+0000", 2024-05-08T08:52:32.596 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1367999999999998e-05 2024-05-08T08:52:32.596 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.596 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:32.596 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.596 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.596 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22075 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 798720~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:32.596 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:52:32.597 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.7293246409999998, 2024-05-08T08:52:32.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047384643999999997, 2024-05-08T08:52:32.597 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:32.597 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:32.597 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:32.597 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:32.597 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:32.597 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:32.597 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22075 2024-05-08T08:52:32.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.597 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:32.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.597 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:32.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:52:32.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:32.598 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:52:32.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.598 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:32.598 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:52:32.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7399999999999999e-07 2024-05-08T08:52:32.598 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:32.598 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857705+0000", 2024-05-08T08:52:32.599 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1980000000000004e-06 2024-05-08T08:52:32.599 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.599 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.599 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:32.599 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857713+0000", 2024-05-08T08:52:32.599 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7039999999999994e-06 2024-05-08T08:52:32.599 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.599 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.599 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:32.599 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857718+0000", 2024-05-08T08:52:32.599 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.865e-06 2024-05-08T08:52:32.599 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.599 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.599 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:32.599 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904941+0000", 2024-05-08T08:52:32.600 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047223198000000001 2024-05-08T08:52:32.600 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.600 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.600 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:32.600 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904999+0000", 2024-05-08T08:52:32.600 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8254999999999997e-05 2024-05-08T08:52:32.600 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.600 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.600 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:32.600 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905082+0000", 2024-05-08T08:52:32.600 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3049999999999999e-05 2024-05-08T08:52:32.600 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.600 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:32.600 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.600 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.600 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.600 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22077 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 815104~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:32.601 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:52:32.601 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.7292866299999998, 2024-05-08T08:52:32.601 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047519448999999998, 2024-05-08T08:52:32.601 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:32.601 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:32.601 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:32.601 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:32.601 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:32.601 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:32.601 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22077 2024-05-08T08:52:32.601 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.601 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:32.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.601 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:32.601 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:52:32.601 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:32.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:52:32.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:32.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:52:32.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.99e-07 2024-05-08T08:52:32.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:32.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857739+0000", 2024-05-08T08:52:32.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3599999999999999e-06 2024-05-08T08:52:32.603 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.603 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:32.603 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857745+0000", 2024-05-08T08:52:32.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5139999999999998e-06 2024-05-08T08:52:32.603 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.603 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:32.603 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857750+0000", 2024-05-08T08:52:32.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8550000000000001e-06 2024-05-08T08:52:32.603 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.603 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:32.603 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905110+0000", 2024-05-08T08:52:32.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047360126000000002 2024-05-08T08:52:32.603 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.604 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.604 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:32.604 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905175+0000", 2024-05-08T08:52:32.604 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4689000000000004e-05 2024-05-08T08:52:32.604 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.604 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.604 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:32.604 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905255+0000", 2024-05-08T08:52:32.604 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0306e-05 2024-05-08T08:52:32.604 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.604 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:32.604 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.604 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.604 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.604 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22079 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 831488~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:32.604 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:52:32.605 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.7292660819999996, 2024-05-08T08:52:32.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.04767499, 2024-05-08T08:52:32.605 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:32.605 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:32.605 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:32.605 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:32.605 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:32.605 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:32.605 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22079 2024-05-08T08:52:32.605 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.605 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:32.605 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.605 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:32.605 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:52:32.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.605 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.606 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:32.606 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:52:32.606 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.606 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.606 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:32.606 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857757+0000", 2024-05-08T08:52:32.606 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8299999999999997e-07 2024-05-08T08:52:32.606 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.606 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:32.606 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857759+0000", 2024-05-08T08:52:32.606 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1390000000000002e-06 2024-05-08T08:52:32.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.607 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:32.607 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857765+0000", 2024-05-08T08:52:32.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4609999999999996e-06 2024-05-08T08:52:32.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.607 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:32.607 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857770+0000", 2024-05-08T08:52:32.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6689999999999997e-06 2024-05-08T08:52:32.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.607 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:32.607 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905281+0000", 2024-05-08T08:52:32.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047510668999999998 2024-05-08T08:52:32.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.608 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:32.608 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905344+0000", 2024-05-08T08:52:32.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2797000000000002e-05 2024-05-08T08:52:32.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.608 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:32.608 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905431+0000", 2024-05-08T08:52:32.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7771999999999999e-05 2024-05-08T08:52:32.608 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.608 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:32.608 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.609 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22082 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 847872~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:32.609 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:52:32.609 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.7292463910000002, 2024-05-08T08:52:32.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047831934999999999, 2024-05-08T08:52:32.609 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:32.609 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:32.609 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:32.609 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:32.609 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:32.609 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:32.609 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22082 2024-05-08T08:52:32.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.609 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:32.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.609 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:32.609 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:52:32.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.610 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.610 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:32.610 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:52:32.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.610 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.610 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:32.610 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:52:32.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:52:32.610 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.610 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:32.610 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857778+0000", 2024-05-08T08:52:32.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0789999999999999e-06 2024-05-08T08:52:32.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.611 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:32.611 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857784+0000", 2024-05-08T08:52:32.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0480000000000004e-06 2024-05-08T08:52:32.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.611 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:32.611 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857790+0000", 2024-05-08T08:52:32.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2719999999999997e-06 2024-05-08T08:52:32.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.611 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:32.611 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905458+0000", 2024-05-08T08:52:32.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047668470999999997 2024-05-08T08:52:32.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.612 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:32.612 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905521+0000", 2024-05-08T08:52:32.612 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2952000000000003e-05 2024-05-08T08:52:32.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.612 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:32.612 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905608+0000", 2024-05-08T08:52:32.612 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6759000000000001e-05 2024-05-08T08:52:32.612 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.612 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:32.612 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.613 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22084 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 864256~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:32.613 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:52:32.613 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.7292224410000001, 2024-05-08T08:52:32.613 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047977748000000001, 2024-05-08T08:52:32.613 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:32.613 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:32.613 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:32.613 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:32.613 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:32.613 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:32.613 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22084 2024-05-08T08:52:32.613 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.613 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:32.613 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.613 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:32.614 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:52:32.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.614 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.614 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.614 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:32.614 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:52:32.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.614 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.614 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.614 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:32.614 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:52:32.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:52:32.614 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.614 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:32.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857802+0000", 2024-05-08T08:52:32.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0779999999999998e-06 2024-05-08T08:52:32.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:32.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857812+0000", 2024-05-08T08:52:32.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7939999999999997e-06 2024-05-08T08:52:32.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:32.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857817+0000", 2024-05-08T08:52:32.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8899999999999998e-06 2024-05-08T08:52:32.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:32.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905636+0000", 2024-05-08T08:52:32.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047819252 2024-05-08T08:52:32.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:32.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905700+0000", 2024-05-08T08:52:32.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4016999999999994e-05 2024-05-08T08:52:32.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:32.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905778+0000", 2024-05-08T08:52:32.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7306999999999997e-05 2024-05-08T08:52:32.616 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.616 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:32.616 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.617 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.617 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22086 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 880640~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:32.617 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:52:32.617 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.7291990909999999, 2024-05-08T08:52:32.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048119149, 2024-05-08T08:52:32.617 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:32.617 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:32.617 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:32.617 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:32.617 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:32.617 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:32.617 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22086 2024-05-08T08:52:32.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.617 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:32.618 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.618 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:32.618 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:52:32.618 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.618 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.618 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.618 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:32.618 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:52:32.618 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.618 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.618 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.618 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:32.618 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857824+0000", 2024-05-08T08:52:32.618 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4200000000000002e-07 2024-05-08T08:52:32.619 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.619 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.619 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:32.619 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857826+0000", 2024-05-08T08:52:32.619 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0590000000000001e-06 2024-05-08T08:52:32.619 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.619 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.619 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:32.619 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857832+0000", 2024-05-08T08:52:32.619 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8029999999999999e-06 2024-05-08T08:52:32.619 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.619 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.619 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:32.619 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857837+0000", 2024-05-08T08:52:32.619 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7929999999999997e-06 2024-05-08T08:52:32.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.620 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:32.620 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905803+0000", 2024-05-08T08:52:32.620 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047965761000000003 2024-05-08T08:52:32.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.620 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:32.620 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905862+0000", 2024-05-08T08:52:32.620 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9203000000000002e-05 2024-05-08T08:52:32.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.620 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:32.620 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905942+0000", 2024-05-08T08:52:32.620 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0187999999999999e-05 2024-05-08T08:52:32.621 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.621 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:32.621 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.621 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22088 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 897024~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:32.621 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:52:32.621 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.7291790389999999, 2024-05-08T08:52:32.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048256049000000002, 2024-05-08T08:52:32.621 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:32.621 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:32.621 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:32.621 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:32.621 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:32.621 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:32.621 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22088 2024-05-08T08:52:32.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.622 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:32.622 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.622 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:32.622 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:52:32.622 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.622 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.622 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.622 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:32.622 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:52:32.622 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.622 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.622 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.622 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:32.622 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857844+0000", 2024-05-08T08:52:32.623 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.46e-07 2024-05-08T08:52:32.623 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.623 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.623 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:32.623 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857846+0000", 2024-05-08T08:52:32.623 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0899999999999999e-06 2024-05-08T08:52:32.623 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.623 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.623 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:32.623 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857855+0000", 2024-05-08T08:52:32.623 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8219999999999996e-06 2024-05-08T08:52:32.623 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.623 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.623 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:32.623 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857860+0000", 2024-05-08T08:52:32.623 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2989999999999996e-06 2024-05-08T08:52:32.624 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.624 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:32.624 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905966+0000", 2024-05-08T08:52:32.624 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048106542000000002 2024-05-08T08:52:32.624 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.624 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:32.624 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906024+0000", 2024-05-08T08:52:32.624 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7958000000000003e-05 2024-05-08T08:52:32.624 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.624 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:32.624 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906099+0000", 2024-05-08T08:52:32.624 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4992000000000003e-05 2024-05-08T08:52:32.624 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.625 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:32.625 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.625 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.625 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.625 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22091 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 913408~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:32.625 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:52:32.625 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.7291552790000004, 2024-05-08T08:52:32.625 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048395290000000001, 2024-05-08T08:52:32.625 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:32.625 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:32.625 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:32.625 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:32.625 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:32.626 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:32.626 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22091 2024-05-08T08:52:32.626 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.626 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:32.626 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.626 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:32.626 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:52:32.626 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.626 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.626 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.626 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:32.626 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:52:32.626 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.626 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.627 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.627 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:32.627 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:52:32.627 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4900000000000001e-07 2024-05-08T08:52:32.627 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.627 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.627 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:32.627 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857870+0000", 2024-05-08T08:52:32.627 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2510000000000001e-06 2024-05-08T08:52:32.627 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.627 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.627 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:32.627 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857877+0000", 2024-05-08T08:52:32.627 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0809999999999996e-06 2024-05-08T08:52:32.627 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.627 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.628 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:32.628 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857881+0000", 2024-05-08T08:52:32.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.724e-06 2024-05-08T08:52:32.628 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.628 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:32.628 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906127+0000", 2024-05-08T08:52:32.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048245345000000002 2024-05-08T08:52:32.628 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.628 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:32.628 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906190+0000", 2024-05-08T08:52:32.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2762000000000007e-05 2024-05-08T08:52:32.628 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.629 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:32.629 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906262+0000", 2024-05-08T08:52:32.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2778e-05 2024-05-08T08:52:32.629 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.629 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:32.629 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.629 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22093 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 929792~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:32.629 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:52:32.629 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.7291329150000001, 2024-05-08T08:52:32.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048532875000000003, 2024-05-08T08:52:32.629 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:32.630 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:32.630 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:32.630 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:32.630 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:32.630 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:32.630 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22093 2024-05-08T08:52:32.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.630 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:32.630 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.630 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:32.630 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:52:32.630 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.630 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.631 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:32.631 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:52:32.631 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.631 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.631 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.631 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:32.631 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857890+0000", 2024-05-08T08:52:32.631 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-07 2024-05-08T08:52:32.631 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.631 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.631 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:32.631 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857899+0000", 2024-05-08T08:52:32.631 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6889999999999999e-06 2024-05-08T08:52:32.631 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.631 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.632 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:32.632 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857906+0000", 2024-05-08T08:52:32.632 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6490000000000002e-06 2024-05-08T08:52:32.632 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.632 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.632 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:32.632 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857917+0000", 2024-05-08T08:52:32.632 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0764999999999999e-05 2024-05-08T08:52:32.632 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.632 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.632 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:32.632 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906287+0000", 2024-05-08T08:52:32.632 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048370442 2024-05-08T08:52:32.632 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.632 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.633 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:32.633 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906349+0000", 2024-05-08T08:52:32.633 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1537000000000004e-05 2024-05-08T08:52:32.633 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.633 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.633 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:32.633 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906422+0000", 2024-05-08T08:52:32.633 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3431999999999998e-05 2024-05-08T08:52:32.633 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.633 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:32.633 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.633 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.633 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.633 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22095 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 946176~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:32.633 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:52:32.634 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.7290982110000002, 2024-05-08T08:52:32.634 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048657593999999998, 2024-05-08T08:52:32.634 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:32.634 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:32.634 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:32.634 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:32.634 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:32.634 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:32.634 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22095 2024-05-08T08:52:32.634 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.634 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:32.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.634 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:32.634 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:52:32.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.635 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.635 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:32.635 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:52:32.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.635 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.635 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:32.635 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857925+0000", 2024-05-08T08:52:32.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3099999999999998e-07 2024-05-08T08:52:32.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.635 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.635 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:32.635 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857927+0000", 2024-05-08T08:52:32.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.34e-06 2024-05-08T08:52:32.636 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.636 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:32.636 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857938+0000", 2024-05-08T08:52:32.636 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1426e-05 2024-05-08T08:52:32.636 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.636 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:32.636 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857943+0000", 2024-05-08T08:52:32.636 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5140000000000003e-06 2024-05-08T08:52:32.636 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.636 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:32.636 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906448+0000", 2024-05-08T08:52:32.636 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048504641000000001 2024-05-08T08:52:32.636 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.637 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.637 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:32.637 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906510+0000", 2024-05-08T08:52:32.637 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2663000000000006e-05 2024-05-08T08:52:32.637 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.637 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.637 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:32.637 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906582+0000", 2024-05-08T08:52:32.637 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1478999999999998e-05 2024-05-08T08:52:32.637 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.637 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:32.637 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.637 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.637 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.637 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22097 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 962560~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:32.637 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:52:32.638 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.7290705219999998, 2024-05-08T08:52:32.638 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048791353000000003, 2024-05-08T08:52:32.638 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:32.638 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:32.638 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:32.638 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:32.638 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:32.638 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:32.638 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22097 2024-05-08T08:52:32.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.638 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:32.638 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.638 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:32.638 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:52:32.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.639 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.639 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.639 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:32.639 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:52:32.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.639 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.639 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.639 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:32.639 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:52:32.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9200000000000002e-07 2024-05-08T08:52:32.639 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.639 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.639 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:32.639 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857954+0000", 2024-05-08T08:52:32.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1900000000000002e-06 2024-05-08T08:52:32.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.640 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.640 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:32.640 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857961+0000", 2024-05-08T08:52:32.640 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4989999999999999e-06 2024-05-08T08:52:32.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.640 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.640 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:32.640 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857971+0000", 2024-05-08T08:52:32.640 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0485999999999999e-05 2024-05-08T08:52:32.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.640 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.640 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:32.640 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906607+0000", 2024-05-08T08:52:32.640 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048635109000000003 2024-05-08T08:52:32.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.641 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:32.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906667+0000", 2024-05-08T08:52:32.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0439000000000002e-05 2024-05-08T08:52:32.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.641 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:32.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906743+0000", 2024-05-08T08:52:32.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6237999999999998e-05 2024-05-08T08:52:32.641 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.641 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:32.641 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.641 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22099 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 978944~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:32.641 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:52:32.641 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.7290443619999998, 2024-05-08T08:52:32.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048932055000000002, 2024-05-08T08:52:32.642 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:32.642 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:32.642 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:32.642 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:32.642 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:32.642 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:32.642 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22099 2024-05-08T08:52:32.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.642 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:32.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:32.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:52:32.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.643 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.643 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.643 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:32.643 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:52:32.643 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.643 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.643 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.643 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:32.643 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:52:32.643 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.27e-07 2024-05-08T08:52:32.643 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.643 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.643 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:32.643 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857987+0000", 2024-05-08T08:52:32.643 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5890000000000003e-06 2024-05-08T08:52:32.643 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.643 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.644 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:32.644 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857994+0000", 2024-05-08T08:52:32.644 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6619999999999999e-06 2024-05-08T08:52:32.644 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.644 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.644 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:32.644 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857998+0000", 2024-05-08T08:52:32.644 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7110000000000003e-06 2024-05-08T08:52:32.644 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.644 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.644 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:32.644 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906772+0000", 2024-05-08T08:52:32.644 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048773415000000001 2024-05-08T08:52:32.644 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.644 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.644 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:32.645 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906834+0000", 2024-05-08T08:52:32.645 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1752000000000001e-05 2024-05-08T08:52:32.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.645 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.645 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:32.645 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906910+0000", 2024-05-08T08:52:32.645 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6499000000000001e-05 2024-05-08T08:52:32.645 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.645 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:32.645 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.645 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.645 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22101 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 995328~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:32.645 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:52:32.645 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.7290172310000003, 2024-05-08T08:52:32.645 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049087364000000001, 2024-05-08T08:52:32.646 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:32.646 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:32.646 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:32.646 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:32.646 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:32.646 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:32.646 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22101 2024-05-08T08:52:32.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.646 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:32.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.646 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:32.646 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:52:32.646 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.647 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.647 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:32.647 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:52:32.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.647 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.647 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.647 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:32.647 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858006+0000", 2024-05-08T08:52:32.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1399999999999997e-07 2024-05-08T08:52:32.647 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.647 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.647 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:32.647 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858008+0000", 2024-05-08T08:52:32.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2060000000000001e-06 2024-05-08T08:52:32.647 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.648 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.648 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:32.648 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858017+0000", 2024-05-08T08:52:32.648 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1579999999999994e-06 2024-05-08T08:52:32.648 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.648 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.648 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:32.648 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858022+0000", 2024-05-08T08:52:32.648 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4199999999999998e-06 2024-05-08T08:52:32.648 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.648 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.648 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:32.648 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906940+0000", 2024-05-08T08:52:32.648 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048917627999999998 2024-05-08T08:52:32.648 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.649 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.649 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:32.649 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907002+0000", 2024-05-08T08:52:32.649 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2399999999999999e-05 2024-05-08T08:52:32.649 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.649 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.649 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:32.650 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907092+0000", 2024-05-08T08:52:32.650 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0137999999999997e-05 2024-05-08T08:52:32.650 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.650 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:32.650 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.650 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.650 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.650 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22103 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1011712~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:32.650 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:52:32.650 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.7289924460000003, 2024-05-08T08:52:32.650 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049228542, 2024-05-08T08:52:32.650 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:32.650 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:32.651 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:32.651 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:32.651 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:32.651 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:32.651 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22103 2024-05-08T08:52:32.651 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.651 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:32.651 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.651 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:32.651 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:52:32.651 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.651 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.651 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.651 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:32.651 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:52:32.652 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.652 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.652 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.652 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:32.652 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:52:32.652 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4299999999999999e-07 2024-05-08T08:52:32.652 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.652 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.652 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:32.652 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858032+0000", 2024-05-08T08:52:32.652 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0590000000000001e-06 2024-05-08T08:52:32.652 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.652 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.652 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:32.652 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858037+0000", 2024-05-08T08:52:32.652 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.977e-06 2024-05-08T08:52:32.653 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.653 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.653 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:32.653 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858042+0000", 2024-05-08T08:52:32.653 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7650000000000001e-06 2024-05-08T08:52:32.653 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.653 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.653 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:32.653 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907118+0000", 2024-05-08T08:52:32.653 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049076147000000001 2024-05-08T08:52:32.653 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.653 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.653 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:32.653 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907180+0000", 2024-05-08T08:52:32.653 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1971999999999996e-05 2024-05-08T08:52:32.654 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.654 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.654 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:32.654 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907258+0000", 2024-05-08T08:52:32.654 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8279e-05 2024-05-08T08:52:32.654 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.654 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:32.654 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.654 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.654 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.654 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22105 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1028096~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:32.654 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:52:32.654 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.7289732029999998, 2024-05-08T08:52:32.654 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049394609999999999, 2024-05-08T08:52:32.655 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:32.655 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:32.655 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:32.655 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:32.655 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:32.655 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:32.655 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22105 2024-05-08T08:52:32.655 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.655 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:32.655 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.655 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:32.655 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:52:32.655 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.655 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.655 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.655 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:32.656 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:52:32.656 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.656 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.656 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.656 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:32.656 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858050+0000", 2024-05-08T08:52:32.656 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5999999999999999e-07 2024-05-08T08:52:32.656 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.656 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.656 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:32.656 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858051+0000", 2024-05-08T08:52:32.656 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8870000000000001e-06 2024-05-08T08:52:32.656 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.656 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.656 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:32.656 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858057+0000", 2024-05-08T08:52:32.657 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8019999999999997e-06 2024-05-08T08:52:32.657 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.657 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.657 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:32.657 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858062+0000", 2024-05-08T08:52:32.657 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.561e-06 2024-05-08T08:52:32.657 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.657 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.657 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:32.657 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907289+0000", 2024-05-08T08:52:32.657 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049226957000000002 2024-05-08T08:52:32.657 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.657 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.657 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:32.657 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907356+0000", 2024-05-08T08:52:32.657 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6915000000000006e-05 2024-05-08T08:52:32.658 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.658 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.658 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:32.658 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907444+0000", 2024-05-08T08:52:32.658 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8127999999999994e-05 2024-05-08T08:52:32.658 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.658 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:32.658 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.658 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.658 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.658 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22107 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1044480~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:32.658 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:52:32.658 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.7289548630000002, 2024-05-08T08:52:32.658 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049799942999999999, 2024-05-08T08:52:32.659 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:32.659 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:32.659 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:32.659 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:32.659 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:32.659 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:32.659 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22107 2024-05-08T08:52:32.659 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.659 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:32.659 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.659 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:32.659 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:52:32.659 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.659 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.659 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.660 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:32.660 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:52:32.660 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.660 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.660 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.660 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:32.660 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:52:32.660 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4799999999999999e-07 2024-05-08T08:52:32.660 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.660 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.660 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:32.660 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858070+0000", 2024-05-08T08:52:32.660 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.012e-06 2024-05-08T08:52:32.660 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.660 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.661 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:32.661 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858077+0000", 2024-05-08T08:52:32.661 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6290000000000001e-06 2024-05-08T08:52:32.661 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.661 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.661 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:32.661 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858083+0000", 2024-05-08T08:52:32.661 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8880000000000002e-06 2024-05-08T08:52:32.661 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.661 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.661 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:32.661 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907671+0000", 2024-05-08T08:52:32.661 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049587191000000003 2024-05-08T08:52:32.661 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.661 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.662 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:32.662 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907763+0000", 2024-05-08T08:52:32.662 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2231000000000005e-05 2024-05-08T08:52:32.662 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.662 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.662 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:32.662 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907867+0000", 2024-05-08T08:52:32.662 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104644 2024-05-08T08:52:32.662 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.662 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:32.662 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.662 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.662 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.662 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22294 3.1c 3:3aad0389:::rbd_data.11406886b07b.0000000000000080:head [stat,delete] snapc 7=[7] RETRY=6 ondisk+retry+write+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:32.663 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:52:32.663 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.7283610149999999, 2024-05-08T08:52:32.663 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051152191, 2024-05-08T08:52:32.663 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:32.663 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:32.663 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:32.663 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:32.663 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:32.663 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:32.663 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22294 2024-05-08T08:52:32.663 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.663 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:32.663 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.663 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:32.663 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:52:32.663 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.664 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.664 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.664 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:32.664 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:52:32.664 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.664 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.664 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.664 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:32.664 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858662+0000", 2024-05-08T08:52:32.664 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4799999999999998e-07 2024-05-08T08:52:32.664 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.664 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.664 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:32.664 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858664+0000", 2024-05-08T08:52:32.665 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1229999999999998e-06 2024-05-08T08:52:32.665 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.665 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.665 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:32.665 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858670+0000", 2024-05-08T08:52:32.665 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3099999999999997e-06 2024-05-08T08:52:32.665 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.665 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.665 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:32.665 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858676+0000", 2024-05-08T08:52:32.665 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9750000000000004e-06 2024-05-08T08:52:32.665 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.665 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.665 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:32.665 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907901+0000", 2024-05-08T08:52:32.666 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049224520000000001 2024-05-08T08:52:32.666 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.666 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.666 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:32.666 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.908085+0000", 2024-05-08T08:52:32.666 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00018377500000000001 2024-05-08T08:52:32.666 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.666 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.666 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:32.666 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.909814+0000", 2024-05-08T08:52:32.666 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00172894 2024-05-08T08:52:32.666 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.666 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:32.666 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.667 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.667 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:32.667 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:52:32.668 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:52:32.669 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:52:32.670+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:52:32.707 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T08:52:32.813 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:52:32.814+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:52:32.911 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:32.914+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:32.983 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:52:32.983 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:52:32.984 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.984 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15737 3.16 3.5b6b6bd6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:32.984 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.951026+0000", 2024-05-08T08:52:32.984 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.030936172000000001, 2024-05-08T08:52:32.984 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030954428999999999, 2024-05-08T08:52:32.984 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:32.984 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:32.984 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:32.984 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:32.984 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:32.984 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:32.984 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15737 2024-05-08T08:52:32.984 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.984 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:32.985 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.985 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:32.985 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.951026+0000", 2024-05-08T08:52:32.985 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.985 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.985 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.985 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:32.985 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.951026+0000", 2024-05-08T08:52:32.985 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.985 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.985 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.985 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:32.985 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.951027+0000", 2024-05-08T08:52:32.985 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6800000000000001e-07 2024-05-08T08:52:32.985 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.985 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.986 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:32.986 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.951029+0000", 2024-05-08T08:52:32.986 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2199999999999999e-06 2024-05-08T08:52:32.986 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.986 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.986 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:32.986 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.951035+0000", 2024-05-08T08:52:32.986 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0850000000000002e-06 2024-05-08T08:52:32.986 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.986 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.986 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:32.986 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.951040+0000", 2024-05-08T08:52:32.986 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.792e-06 2024-05-08T08:52:32.987 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.987 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:32.987 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.987 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.987 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.987 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15800 3.16 3.83728616 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:32.987 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.957071+0000", 2024-05-08T08:52:32.987 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.024891242000000001, 2024-05-08T08:52:32.987 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.024970910999999998, 2024-05-08T08:52:32.987 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:32.987 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:32.987 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:32.987 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:32.987 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:32.988 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:32.988 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15800 2024-05-08T08:52:32.988 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.988 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:32.988 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.988 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:32.988 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.957071+0000", 2024-05-08T08:52:32.988 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.988 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.988 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.988 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:32.988 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.957071+0000", 2024-05-08T08:52:32.988 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.988 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.988 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.989 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:32.989 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.957072+0000", 2024-05-08T08:52:32.989 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5900000000000002e-07 2024-05-08T08:52:32.989 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.989 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.989 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:32.989 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.957074+0000", 2024-05-08T08:52:32.989 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2120000000000002e-06 2024-05-08T08:52:32.989 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.989 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.989 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:32.989 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.957080+0000", 2024-05-08T08:52:32.989 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8900000000000004e-06 2024-05-08T08:52:32.989 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.989 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.990 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:32.990 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.957084+0000", 2024-05-08T08:52:32.990 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7929999999999997e-06 2024-05-08T08:52:32.990 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.990 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:32.990 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.990 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.990 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.990 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15865 3.16 3.83728616 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:32.990 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.963046+0000", 2024-05-08T08:52:32.990 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.018916156999999999, 2024-05-08T08:52:32.990 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.019052935, 2024-05-08T08:52:32.990 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:32.990 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:32.991 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:32.991 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:32.991 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:32.991 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:32.991 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15865 2024-05-08T08:52:32.991 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.991 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:32.991 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.991 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:32.991 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.963046+0000", 2024-05-08T08:52:32.991 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.991 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.991 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.991 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:32.991 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.963046+0000", 2024-05-08T08:52:32.992 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.992 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.992 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.992 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:32.992 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.963047+0000", 2024-05-08T08:52:32.992 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4899999999999995e-07 2024-05-08T08:52:32.992 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.992 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.992 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:32.992 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.963050+0000", 2024-05-08T08:52:32.992 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.253e-06 2024-05-08T08:52:32.993 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.993 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.993 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:32.993 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.963057+0000", 2024-05-08T08:52:32.993 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9689999999999997e-06 2024-05-08T08:52:32.993 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.993 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.993 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:32.993 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.963063+0000", 2024-05-08T08:52:32.993 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8540000000000002e-06 2024-05-08T08:52:32.993 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.993 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:32.993 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.993 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.993 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.994 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15739 3.16 3.83728616 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:32.994 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.951047+0000", 2024-05-08T08:52:32.994 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.030915333, 2024-05-08T08:52:32.994 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031107072, 2024-05-08T08:52:32.994 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:32.994 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:32.994 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:32.994 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:32.994 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:32.994 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:32.994 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15739 2024-05-08T08:52:32.994 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.994 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:32.995 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.995 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:32.995 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.951047+0000", 2024-05-08T08:52:32.995 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.995 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.995 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.995 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:32.995 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.951047+0000", 2024-05-08T08:52:32.995 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.995 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.995 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.995 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:32.995 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.951047+0000", 2024-05-08T08:52:32.995 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9200000000000002e-07 2024-05-08T08:52:32.995 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.996 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.996 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:32.996 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.951050+0000", 2024-05-08T08:52:32.996 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2230000000000001e-06 2024-05-08T08:52:32.996 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.996 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.996 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:32.996 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.951056+0000", 2024-05-08T08:52:32.996 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8969999999999996e-06 2024-05-08T08:52:32.996 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.996 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.996 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:32.996 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.951061+0000", 2024-05-08T08:52:32.996 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1479999999999997e-06 2024-05-08T08:52:32.996 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.997 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:32.997 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:32.997 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.997 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.997 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15801 3.16 3.83728616 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:32.997 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.957091+0000", 2024-05-08T08:52:32.997 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.024871257000000001, 2024-05-08T08:52:32.997 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.025115558999999999, 2024-05-08T08:52:32.997 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:32.997 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:32.997 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:32.997 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:32.997 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:32.997 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:32.997 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15801 2024-05-08T08:52:32.998 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.998 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:32.998 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.998 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:32.998 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.957091+0000", 2024-05-08T08:52:32.998 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.998 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.998 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.998 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:32.998 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.957091+0000", 2024-05-08T08:52:32.998 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:32.998 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.998 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.998 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:32.999 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.957092+0000", 2024-05-08T08:52:32.999 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9800000000000004e-07 2024-05-08T08:52:32.999 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.999 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.999 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:32.999 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.957094+0000", 2024-05-08T08:52:32.999 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2189999999999998e-06 2024-05-08T08:52:32.999 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.999 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.999 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:32.999 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.957100+0000", 2024-05-08T08:52:32.999 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8989999999999998e-06 2024-05-08T08:52:32.999 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:32.999 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:32.999 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:32.999 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.957104+0000", 2024-05-08T08:52:33.000 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4469999999999999e-06 2024-05-08T08:52:33.000 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.000 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.000 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.000 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.000 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.000 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15741 3.16 3.83728616 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.000 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.951067+0000", 2024-05-08T08:52:33.000 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.030895418000000001, 2024-05-08T08:52:33.000 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031174729000000002, 2024-05-08T08:52:33.000 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.000 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.000 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:33.000 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.001 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.001 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.001 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15741 2024-05-08T08:52:33.001 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.001 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.001 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.001 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.001 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.951067+0000", 2024-05-08T08:52:33.001 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.001 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.001 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.001 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.001 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.951067+0000", 2024-05-08T08:52:33.001 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.001 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.002 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.002 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.002 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.951067+0000", 2024-05-08T08:52:33.002 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9400000000000001e-07 2024-05-08T08:52:33.002 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.002 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.002 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.002 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.951070+0000", 2024-05-08T08:52:33.002 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2840000000000001e-06 2024-05-08T08:52:33.002 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.002 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.002 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.002 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.951076+0000", 2024-05-08T08:52:33.002 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.891e-06 2024-05-08T08:52:33.003 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.003 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.003 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.003 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.951080+0000", 2024-05-08T08:52:33.003 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8559999999999998e-06 2024-05-08T08:52:33.003 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.003 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.003 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.003 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.003 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.003 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15803 3.16 3.83728616 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.003 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.957110+0000", 2024-05-08T08:52:33.003 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.024852170999999999, 2024-05-08T08:52:33.003 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.025169185, 2024-05-08T08:52:33.003 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.004 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.004 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:33.004 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.004 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.004 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.004 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15803 2024-05-08T08:52:33.004 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.004 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.004 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.004 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.004 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.957110+0000", 2024-05-08T08:52:33.004 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.004 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.004 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.004 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.005 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.957110+0000", 2024-05-08T08:52:33.005 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.005 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.005 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.005 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.005 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.957111+0000", 2024-05-08T08:52:33.005 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9200000000000002e-07 2024-05-08T08:52:33.005 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.005 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.005 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.005 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.957113+0000", 2024-05-08T08:52:33.005 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1969999999999999e-06 2024-05-08T08:52:33.005 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.005 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.005 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.006 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.957118+0000", 2024-05-08T08:52:33.006 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5400000000000003e-06 2024-05-08T08:52:33.006 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.006 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.006 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.006 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.957123+0000", 2024-05-08T08:52:33.006 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6800000000000001e-06 2024-05-08T08:52:33.006 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.006 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.006 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.006 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.006 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.006 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15743 3.16 3.83728616 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.006 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.951087+0000", 2024-05-08T08:52:33.007 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.030875354000000001, 2024-05-08T08:52:33.007 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.03122285, 2024-05-08T08:52:33.007 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.007 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.007 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:33.007 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.007 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.007 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.007 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15743 2024-05-08T08:52:33.007 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.007 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.007 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.007 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.007 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.951087+0000", 2024-05-08T08:52:33.008 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.008 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.008 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.008 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.008 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.951087+0000", 2024-05-08T08:52:33.008 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.008 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.008 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.008 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.008 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.951087+0000", 2024-05-08T08:52:33.008 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3000000000000001e-07 2024-05-08T08:52:33.008 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.008 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.008 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.008 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.951090+0000", 2024-05-08T08:52:33.009 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2730000000000001e-06 2024-05-08T08:52:33.009 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.009 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.009 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.009 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.951096+0000", 2024-05-08T08:52:33.009 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8660000000000002e-06 2024-05-08T08:52:33.009 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.009 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.009 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.009 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.951100+0000", 2024-05-08T08:52:33.009 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8910000000000003e-06 2024-05-08T08:52:33.009 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.009 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.009 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.009 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.010 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.010 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15805 3.16 3.83728616 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.010 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.957140+0000", 2024-05-08T08:52:33.010 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.024821899000000001, 2024-05-08T08:52:33.010 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.025219139000000002, 2024-05-08T08:52:33.010 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.010 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.010 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:33.010 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.010 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.010 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.010 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15805 2024-05-08T08:52:33.010 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.010 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.010 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.010 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.011 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.957140+0000", 2024-05-08T08:52:33.011 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.011 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.011 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.011 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.012 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:52:33.012 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:52:33.013 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.957140+0000", 2024-05-08T08:52:33.013 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.014 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.014 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.014 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.014 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.957141+0000", 2024-05-08T08:52:33.014 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3099999999999998e-07 2024-05-08T08:52:33.014 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.014 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.014 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.014 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.957144+0000", 2024-05-08T08:52:33.014 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5380000000000001e-06 2024-05-08T08:52:33.014 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.014 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.014 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.014 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.957151+0000", 2024-05-08T08:52:33.015 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5519999999999999e-06 2024-05-08T08:52:33.015 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.015 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.015 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.015 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.957156+0000", 2024-05-08T08:52:33.015 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2669999999999997e-06 2024-05-08T08:52:33.015 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.015 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.015 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.015 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.015 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.015 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15745 3.16 3.83728616 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.015 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.951107+0000", 2024-05-08T08:52:33.015 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.030855790000000001, 2024-05-08T08:52:33.016 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031309917999999999, 2024-05-08T08:52:33.016 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.016 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.016 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:33.016 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.016 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.016 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.016 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15745 2024-05-08T08:52:33.016 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.016 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.016 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.016 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.016 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.951107+0000", 2024-05-08T08:52:33.016 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.016 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.017 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.017 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.017 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.951107+0000", 2024-05-08T08:52:33.017 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.017 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.017 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.017 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.017 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.951107+0000", 2024-05-08T08:52:33.017 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7700000000000005e-07 2024-05-08T08:52:33.017 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.017 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.017 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.017 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.951109+0000", 2024-05-08T08:52:33.017 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.17e-06 2024-05-08T08:52:33.018 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.018 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.018 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.018 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.951115+0000", 2024-05-08T08:52:33.018 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0079999999999999e-06 2024-05-08T08:52:33.018 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.018 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.018 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.018 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.951120+0000", 2024-05-08T08:52:33.018 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8550000000000001e-06 2024-05-08T08:52:33.018 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.018 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.018 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.018 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.019 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.019 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15807 3.16 3.83728616 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.019 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.957163+0000", 2024-05-08T08:52:33.019 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.024799038999999998, 2024-05-08T08:52:33.019 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.025306708000000001, 2024-05-08T08:52:33.019 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.019 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.019 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:33.019 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.019 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.019 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.019 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15807 2024-05-08T08:52:33.019 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.019 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.020 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.020 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.020 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.957163+0000", 2024-05-08T08:52:33.020 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.020 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.020 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.020 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.020 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.957163+0000", 2024-05-08T08:52:33.020 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.020 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.020 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.020 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.020 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.957164+0000", 2024-05-08T08:52:33.020 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2399999999999999e-07 2024-05-08T08:52:33.020 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.021 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.021 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.021 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.957166+0000", 2024-05-08T08:52:33.021 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2110000000000001e-06 2024-05-08T08:52:33.021 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.021 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.021 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.021 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.957173+0000", 2024-05-08T08:52:33.021 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0500000000000003e-06 2024-05-08T08:52:33.021 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.021 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.021 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.021 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.957178+0000", 2024-05-08T08:52:33.021 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.729e-06 2024-05-08T08:52:33.022 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.022 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.022 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.022 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.022 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.022 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15747 3.16 3.83728616 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.022 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.951138+0000", 2024-05-08T08:52:33.022 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.030824463999999999, 2024-05-08T08:52:33.022 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031386279000000003, 2024-05-08T08:52:33.022 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.022 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.022 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:33.022 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.022 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.023 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.023 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15747 2024-05-08T08:52:33.023 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.023 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.023 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.023 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.023 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.951138+0000", 2024-05-08T08:52:33.023 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.023 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.023 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.023 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.023 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.951138+0000", 2024-05-08T08:52:33.023 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.023 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.024 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.024 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.024 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.951139+0000", 2024-05-08T08:52:33.024 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2799999999999996e-07 2024-05-08T08:52:33.024 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.024 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.024 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.024 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.951141+0000", 2024-05-08T08:52:33.024 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5560000000000002e-06 2024-05-08T08:52:33.024 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.024 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.024 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.024 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.951148+0000", 2024-05-08T08:52:33.024 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.409e-06 2024-05-08T08:52:33.024 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.025 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.025 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.025 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.951153+0000", 2024-05-08T08:52:33.025 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.237e-06 2024-05-08T08:52:33.025 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.025 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.025 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.025 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.025 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.025 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15809 3.16 3.83728616 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.025 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.957184+0000", 2024-05-08T08:52:33.025 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.024778679000000001, 2024-05-08T08:52:33.025 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.025373209000000001, 2024-05-08T08:52:33.025 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.026 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.026 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:33.026 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.026 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.026 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.026 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15809 2024-05-08T08:52:33.026 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.026 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.026 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.026 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.026 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.957184+0000", 2024-05-08T08:52:33.026 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.026 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.026 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.026 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.027 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.957184+0000", 2024-05-08T08:52:33.027 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.027 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.027 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.027 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.027 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.957184+0000", 2024-05-08T08:52:33.027 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3500000000000002e-07 2024-05-08T08:52:33.027 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.027 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.027 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.027 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.957186+0000", 2024-05-08T08:52:33.027 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0150000000000002e-06 2024-05-08T08:52:33.027 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.027 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.028 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.028 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.957192+0000", 2024-05-08T08:52:33.028 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7049999999999996e-06 2024-05-08T08:52:33.028 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.028 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.028 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.028 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.957198+0000", 2024-05-08T08:52:33.028 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8130000000000001e-06 2024-05-08T08:52:33.028 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.028 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.028 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.028 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.028 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.028 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15749 3.16 3.83728616 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.029 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.951160+0000", 2024-05-08T08:52:33.029 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.030802751999999999, 2024-05-08T08:52:33.029 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031424727, 2024-05-08T08:52:33.029 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.029 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.029 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:33.029 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.029 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.029 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.029 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15749 2024-05-08T08:52:33.029 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.029 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.029 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.029 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.029 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.951160+0000", 2024-05-08T08:52:33.030 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.030 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.030 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.030 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.030 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.951160+0000", 2024-05-08T08:52:33.030 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.030 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.030 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.030 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.030 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.951160+0000", 2024-05-08T08:52:33.030 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2900000000000004e-07 2024-05-08T08:52:33.030 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.030 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.030 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.031 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.951162+0000", 2024-05-08T08:52:33.031 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0269999999999998e-06 2024-05-08T08:52:33.031 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.031 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.031 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.031 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.951168+0000", 2024-05-08T08:52:33.031 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8420000000000001e-06 2024-05-08T08:52:33.031 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.031 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.031 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.031 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.951173+0000", 2024-05-08T08:52:33.031 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8489999999999997e-06 2024-05-08T08:52:33.031 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.031 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.031 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.032 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.032 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.032 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15811 3.16 3.83728616 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.032 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.957205+0000", 2024-05-08T08:52:33.032 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.024757174, 2024-05-08T08:52:33.032 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.025406201999999999, 2024-05-08T08:52:33.032 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.032 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.032 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:33.032 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.032 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.032 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.032 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15811 2024-05-08T08:52:33.032 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.032 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.033 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.033 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.033 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.957205+0000", 2024-05-08T08:52:33.033 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.033 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.033 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.033 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.033 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.957205+0000", 2024-05-08T08:52:33.033 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.033 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.033 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.033 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.033 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.957206+0000", 2024-05-08T08:52:33.033 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7500000000000001e-07 2024-05-08T08:52:33.033 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.034 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.034 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.034 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.957208+0000", 2024-05-08T08:52:33.034 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0669999999999999e-06 2024-05-08T08:52:33.034 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.034 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.034 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.034 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.957215+0000", 2024-05-08T08:52:33.034 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0210000000000002e-06 2024-05-08T08:52:33.034 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.034 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.034 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.034 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.957219+0000", 2024-05-08T08:52:33.034 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5279999999999997e-06 2024-05-08T08:52:33.034 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.035 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.035 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.035 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.035 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.035 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15685 3.16 3:6bd6d6da:::rbd_data.1140782590a5.0000000000000015:6 [read 2734080~2048 out=2048b] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.035 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.945175+0000", 2024-05-08T08:52:33.035 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.036787780999999999, 2024-05-08T08:52:33.035 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.037467276000000001, 2024-05-08T08:52:33.035 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.035 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.035 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:33.035 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.035 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.036 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.036 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15685 2024-05-08T08:52:33.036 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.036 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.036 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.036 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.036 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.945175+0000", 2024-05-08T08:52:33.036 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.036 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.036 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.036 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.036 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.945175+0000", 2024-05-08T08:52:33.036 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.036 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.036 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.037 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.037 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.945175+0000", 2024-05-08T08:52:33.037 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.39e-07 2024-05-08T08:52:33.037 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.037 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.037 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.037 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.945177+0000", 2024-05-08T08:52:33.037 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.759e-06 2024-05-08T08:52:33.037 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.037 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.037 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.037 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.945180+0000", 2024-05-08T08:52:33.037 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5370000000000002e-06 2024-05-08T08:52:33.037 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.037 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.038 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.038 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.945185+0000", 2024-05-08T08:52:33.038 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5399999999999997e-06 2024-05-08T08:52:33.038 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.038 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.038 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:33.038 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.982545+0000", 2024-05-08T08:52:33.038 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.037360141999999999 2024-05-08T08:52:33.038 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.038 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.038 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:33.038 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.982583+0000", 2024-05-08T08:52:33.038 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8191000000000001e-05 2024-05-08T08:52:33.038 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.038 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.039 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:33.039 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.982643+0000", 2024-05-08T08:52:33.039 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0118000000000003e-05 2024-05-08T08:52:33.039 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.039 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.039 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.039 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.039 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.039 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15751 3.16 3.83728616 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.039 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.951179+0000", 2024-05-08T08:52:33.039 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.030783324000000001, 2024-05-08T08:52:33.039 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031530760999999997, 2024-05-08T08:52:33.039 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.039 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.039 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:33.040 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.040 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.040 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.040 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15751 2024-05-08T08:52:33.040 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.040 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.040 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.040 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.040 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.951179+0000", 2024-05-08T08:52:33.040 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.040 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.040 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.040 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.040 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.951179+0000", 2024-05-08T08:52:33.040 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.041 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.041 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.041 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.041 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.951179+0000", 2024-05-08T08:52:33.041 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3799999999999998e-07 2024-05-08T08:52:33.041 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.041 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.041 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.041 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.951182+0000", 2024-05-08T08:52:33.041 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1449999999999998e-06 2024-05-08T08:52:33.041 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.041 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.041 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.041 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.951193+0000", 2024-05-08T08:52:33.041 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1158000000000001e-05 2024-05-08T08:52:33.042 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.042 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.042 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.042 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.951198+0000", 2024-05-08T08:52:33.042 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8899999999999998e-06 2024-05-08T08:52:33.042 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.042 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.042 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.042 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.042 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.042 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15813 3.16 3.83728616 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.042 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.957225+0000", 2024-05-08T08:52:33.042 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.024737254, 2024-05-08T08:52:33.043 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.025543461, 2024-05-08T08:52:33.043 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.043 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.043 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:33.043 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.043 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.043 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.043 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15813 2024-05-08T08:52:33.043 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.043 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.043 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.043 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.043 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.957225+0000", 2024-05-08T08:52:33.043 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.043 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.043 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.044 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.044 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.957225+0000", 2024-05-08T08:52:33.044 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.044 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.044 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.044 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.044 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.958936+0000", 2024-05-08T08:52:33.044 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0017108889999999999 2024-05-08T08:52:33.044 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.044 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.044 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.044 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.958941+0000", 2024-05-08T08:52:33.044 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6129999999999997e-06 2024-05-08T08:52:33.044 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.044 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.045 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.045 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.958950+0000", 2024-05-08T08:52:33.045 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2399999999999996e-06 2024-05-08T08:52:33.045 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.045 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.045 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.045 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.958957+0000", 2024-05-08T08:52:33.045 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6449999999999999e-06 2024-05-08T08:52:33.045 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.045 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.045 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.045 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.045 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.045 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15687 3.16 3:6bd6d6da:::rbd_data.1140782590a5.0000000000000015:6 [read 2750464~2048 out=2048b] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.045 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.945191+0000", 2024-05-08T08:52:33.045 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.036771757000000002, 2024-05-08T08:52:33.046 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.037635970999999997, 2024-05-08T08:52:33.046 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.046 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.046 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:33.046 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.046 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.046 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.046 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15687 2024-05-08T08:52:33.046 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.046 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.046 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.046 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.046 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.945191+0000", 2024-05-08T08:52:33.046 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.047 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.047 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.047 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.047 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.945191+0000", 2024-05-08T08:52:33.047 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.047 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.047 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.047 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.047 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.946936+0000", 2024-05-08T08:52:33.047 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0017458479999999999 2024-05-08T08:52:33.047 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.047 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.047 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.047 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.946941+0000", 2024-05-08T08:52:33.047 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0740000000000003e-06 2024-05-08T08:52:33.048 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.048 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.048 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.048 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.946951+0000", 2024-05-08T08:52:33.048 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0499000000000001e-05 2024-05-08T08:52:33.048 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.048 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.048 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.048 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.946958+0000", 2024-05-08T08:52:33.048 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7859999999999999e-06 2024-05-08T08:52:33.048 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.048 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.048 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:33.048 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.982666+0000", 2024-05-08T08:52:33.048 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035708482 2024-05-08T08:52:33.048 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.049 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.049 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:33.049 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.982742+0000", 2024-05-08T08:52:33.049 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5689e-05 2024-05-08T08:52:33.049 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.049 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.049 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.049 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.049 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.049 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15753 3.16 3.83728616 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.049 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.951209+0000", 2024-05-08T08:52:33.049 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.030753210999999999, 2024-05-08T08:52:33.049 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031665564, 2024-05-08T08:52:33.049 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.049 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.050 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:33.050 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.050 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.050 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.050 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15753 2024-05-08T08:52:33.050 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.050 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.050 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.050 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.050 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.951209+0000", 2024-05-08T08:52:33.050 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.050 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.050 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.050 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.050 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.951209+0000", 2024-05-08T08:52:33.051 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.051 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.051 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.051 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.051 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.952938+0000", 2024-05-08T08:52:33.051 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001729307 2024-05-08T08:52:33.051 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.051 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.051 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.051 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.952943+0000", 2024-05-08T08:52:33.051 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8169999999999998e-06 2024-05-08T08:52:33.051 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.051 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.051 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.051 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.952952+0000", 2024-05-08T08:52:33.051 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1849999999999993e-06 2024-05-08T08:52:33.052 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.052 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.052 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.052 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.952959+0000", 2024-05-08T08:52:33.052 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5699999999999998e-06 2024-05-08T08:52:33.052 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.052 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.052 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.052 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.052 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.052 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15816 3.16 3.83728616 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.052 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.958968+0000", 2024-05-08T08:52:33.052 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.022994522999999999, 2024-05-08T08:52:33.052 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.023934845, 2024-05-08T08:52:33.053 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.053 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.053 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:33.053 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.053 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.053 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.053 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15816 2024-05-08T08:52:33.053 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.053 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.053 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.053 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.053 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.958968+0000", 2024-05-08T08:52:33.053 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.053 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.053 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.054 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.054 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.958968+0000", 2024-05-08T08:52:33.054 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.054 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.054 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.054 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.054 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.958968+0000", 2024-05-08T08:52:33.054 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2600000000000002e-07 2024-05-08T08:52:33.054 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.054 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.054 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.054 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.958971+0000", 2024-05-08T08:52:33.054 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6000000000000001e-06 2024-05-08T08:52:33.055 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.055 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.055 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.055 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.958984+0000", 2024-05-08T08:52:33.055 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2843000000000001e-05 2024-05-08T08:52:33.055 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.055 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.055 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.055 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.958990+0000", 2024-05-08T08:52:33.055 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4010000000000002e-06 2024-05-08T08:52:33.055 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.055 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.055 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.055 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.055 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.056 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15689 3.16 3:6bd6d6da:::rbd_data.1140782590a5.0000000000000015:6 [read 2766848~2048 out=2048b] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.056 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.946969+0000", 2024-05-08T08:52:33.056 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.034993754000000002, 2024-05-08T08:52:33.056 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035975401999999997, 2024-05-08T08:52:33.056 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.056 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.056 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:33.056 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.056 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.056 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.056 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15689 2024-05-08T08:52:33.056 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.056 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.056 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.057 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.057 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.946969+0000", 2024-05-08T08:52:33.057 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.057 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.057 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.057 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.057 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.946969+0000", 2024-05-08T08:52:33.057 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.057 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.057 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.057 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.057 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.946969+0000", 2024-05-08T08:52:33.057 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5600000000000001e-07 2024-05-08T08:52:33.057 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.057 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.058 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.058 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.946971+0000", 2024-05-08T08:52:33.058 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.074e-06 2024-05-08T08:52:33.058 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.058 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.058 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.058 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.946978+0000", 2024-05-08T08:52:33.058 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6640000000000001e-06 2024-05-08T08:52:33.058 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.058 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.058 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.058 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.946983+0000", 2024-05-08T08:52:33.058 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.006e-06 2024-05-08T08:52:33.058 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.059 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.059 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:33.059 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.982862+0000", 2024-05-08T08:52:33.059 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035878738 2024-05-08T08:52:33.059 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.059 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.059 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:33.059 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.982904+0000", 2024-05-08T08:52:33.059 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1990999999999998e-05 2024-05-08T08:52:33.059 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.059 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.059 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.059 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.059 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.060 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15755 3.16 3.83728616 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.060 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.952977+0000", 2024-05-08T08:52:33.060 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.028985268000000002, 2024-05-08T08:52:33.060 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030022460000000001, 2024-05-08T08:52:33.060 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.060 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.060 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:33.060 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.061 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.062 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.062 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15755 2024-05-08T08:52:33.062 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.062 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.062 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.062 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.062 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.952977+0000", 2024-05-08T08:52:33.062 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.062 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.062 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.062 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.062 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.952977+0000", 2024-05-08T08:52:33.062 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.062 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.063 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.063 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.063 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.952978+0000", 2024-05-08T08:52:33.063 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3399999999999999e-07 2024-05-08T08:52:33.063 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.063 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.063 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.063 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.952981+0000", 2024-05-08T08:52:33.063 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9660000000000001e-06 2024-05-08T08:52:33.063 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.063 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.063 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.063 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.952988+0000", 2024-05-08T08:52:33.063 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0249999999999997e-06 2024-05-08T08:52:33.063 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.064 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.064 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.064 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.952993+0000", 2024-05-08T08:52:33.064 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2859999999999999e-06 2024-05-08T08:52:33.064 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.064 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.064 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.064 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.064 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.064 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15818 3.16 3.83728616 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.064 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.958998+0000", 2024-05-08T08:52:33.064 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.022964222999999999, 2024-05-08T08:52:33.064 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.024054953, 2024-05-08T08:52:33.064 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.064 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.064 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:33.065 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.065 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.065 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.065 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15818 2024-05-08T08:52:33.065 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.065 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.065 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.065 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.065 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.958998+0000", 2024-05-08T08:52:33.065 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.065 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.065 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.065 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.065 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.958998+0000", 2024-05-08T08:52:33.066 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.066 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.066 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.066 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.066 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.958999+0000", 2024-05-08T08:52:33.066 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3799999999999998e-07 2024-05-08T08:52:33.066 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.066 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.066 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.066 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.959001+0000", 2024-05-08T08:52:33.066 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5399999999999998e-06 2024-05-08T08:52:33.066 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.066 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.066 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.067 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.959007+0000", 2024-05-08T08:52:33.067 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2720000000000003e-06 2024-05-08T08:52:33.067 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.067 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.067 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.067 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.959013+0000", 2024-05-08T08:52:33.067 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9909999999999999e-06 2024-05-08T08:52:33.067 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.067 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.067 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.067 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.067 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.067 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15691 3.16 3:6bd6d6da:::rbd_data.1140782590a5.0000000000000015:6 [read 2783232~2048 out=2048b] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.067 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.946990+0000", 2024-05-08T08:52:33.068 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.034972498999999997, 2024-05-08T08:52:33.068 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036122709000000003, 2024-05-08T08:52:33.068 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.068 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.068 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:33.068 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.068 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.068 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.068 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15691 2024-05-08T08:52:33.068 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.068 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.068 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.068 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.068 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.946990+0000", 2024-05-08T08:52:33.068 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.069 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.069 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.069 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.069 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.946990+0000", 2024-05-08T08:52:33.069 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.069 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.069 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.069 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.069 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.946990+0000", 2024-05-08T08:52:33.069 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1300000000000001e-07 2024-05-08T08:52:33.069 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.069 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.069 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.069 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.946992+0000", 2024-05-08T08:52:33.070 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9649999999999998e-06 2024-05-08T08:52:33.070 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.070 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.070 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.070 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.946999+0000", 2024-05-08T08:52:33.070 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7549999999999997e-06 2024-05-08T08:52:33.070 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.070 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.070 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.070 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.947004+0000", 2024-05-08T08:52:33.070 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1070000000000004e-06 2024-05-08T08:52:33.070 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.070 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.070 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:33.071 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.982982+0000", 2024-05-08T08:52:33.071 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035977756999999999 2024-05-08T08:52:33.071 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.071 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.071 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:33.071 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.983059+0000", 2024-05-08T08:52:33.071 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7173000000000001e-05 2024-05-08T08:52:33.071 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.071 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.071 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.071 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.071 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.071 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15757 3.16 3.83728616 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.071 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.953001+0000", 2024-05-08T08:52:33.072 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.028961539000000001, 2024-05-08T08:52:33.072 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030172378, 2024-05-08T08:52:33.072 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.072 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.072 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:33.072 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.072 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.072 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.072 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15757 2024-05-08T08:52:33.072 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.072 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.072 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.072 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.072 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.953001+0000", 2024-05-08T08:52:33.072 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.073 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.073 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.073 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.073 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.953001+0000", 2024-05-08T08:52:33.073 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.073 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.073 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.073 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.073 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.953001+0000", 2024-05-08T08:52:33.073 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2100000000000002e-07 2024-05-08T08:52:33.073 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.073 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.073 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.073 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.953004+0000", 2024-05-08T08:52:33.074 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.334e-06 2024-05-08T08:52:33.074 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.074 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.074 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.074 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.953010+0000", 2024-05-08T08:52:33.074 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0390000000000001e-06 2024-05-08T08:52:33.074 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.074 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.074 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.074 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.953015+0000", 2024-05-08T08:52:33.074 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1680000000000003e-06 2024-05-08T08:52:33.074 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.074 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.074 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.075 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.075 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.075 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15820 3.16 3.83728616 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.075 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.959020+0000", 2024-05-08T08:52:33.075 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.022942272999999999, 2024-05-08T08:52:33.075 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.024181556, 2024-05-08T08:52:33.075 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.075 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.075 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:33.075 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.075 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.075 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.075 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15820 2024-05-08T08:52:33.075 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.075 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.076 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.076 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.076 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.959020+0000", 2024-05-08T08:52:33.076 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.076 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.076 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.076 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.076 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.959020+0000", 2024-05-08T08:52:33.076 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.076 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.076 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.076 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.076 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.959021+0000", 2024-05-08T08:52:33.076 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2e-07 2024-05-08T08:52:33.076 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.077 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.077 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.077 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.959023+0000", 2024-05-08T08:52:33.077 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2709999999999999e-06 2024-05-08T08:52:33.077 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.077 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.077 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.077 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.959030+0000", 2024-05-08T08:52:33.077 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9e-06 2024-05-08T08:52:33.077 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.077 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.077 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.077 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.959035+0000", 2024-05-08T08:52:33.077 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3770000000000004e-06 2024-05-08T08:52:33.077 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.078 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.078 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.078 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.078 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.078 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15693 3.16 3:6bd6d6da:::rbd_data.1140782590a5.0000000000000015:6 [read 2799616~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.078 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.947011+0000", 2024-05-08T08:52:33.078 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.034951771, 2024-05-08T08:52:33.078 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036226727, 2024-05-08T08:52:33.078 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.078 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.078 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:33.078 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.078 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.078 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.079 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15693 2024-05-08T08:52:33.079 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.079 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.079 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.079 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.079 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.947011+0000", 2024-05-08T08:52:33.079 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.079 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.079 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.079 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.079 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.947011+0000", 2024-05-08T08:52:33.079 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.079 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.079 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.079 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.079 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.947011+0000", 2024-05-08T08:52:33.080 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0699999999999998e-07 2024-05-08T08:52:33.080 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.080 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.080 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.080 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.947013+0000", 2024-05-08T08:52:33.080 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8959999999999999e-06 2024-05-08T08:52:33.080 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.080 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.080 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.080 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.947021+0000", 2024-05-08T08:52:33.080 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8080000000000005e-06 2024-05-08T08:52:33.080 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.080 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.080 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.080 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.947026+0000", 2024-05-08T08:52:33.081 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1800000000000004e-06 2024-05-08T08:52:33.081 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.081 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.081 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:33.081 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.983173+0000", 2024-05-08T08:52:33.081 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036147585000000003 2024-05-08T08:52:33.081 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.081 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.081 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:33.081 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.983216+0000", 2024-05-08T08:52:33.081 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2728000000000002e-05 2024-05-08T08:52:33.081 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.081 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.081 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.082 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.082 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.082 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15759 3.16 3.83728616 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.082 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.953021+0000", 2024-05-08T08:52:33.082 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.028941547000000001, 2024-05-08T08:52:33.082 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030259309000000002, 2024-05-08T08:52:33.082 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.082 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.082 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:33.082 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.082 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.082 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.082 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15759 2024-05-08T08:52:33.082 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.083 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.083 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.083 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.083 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.953021+0000", 2024-05-08T08:52:33.083 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.083 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.083 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.083 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.083 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.953021+0000", 2024-05-08T08:52:33.083 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.083 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.083 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.083 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.083 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.953021+0000", 2024-05-08T08:52:33.084 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.63e-07 2024-05-08T08:52:33.084 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.084 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.084 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.084 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.953023+0000", 2024-05-08T08:52:33.084 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0660000000000002e-06 2024-05-08T08:52:33.084 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.084 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.084 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.084 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.953029+0000", 2024-05-08T08:52:33.084 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7780000000000004e-06 2024-05-08T08:52:33.084 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.084 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.084 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.084 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.953034+0000", 2024-05-08T08:52:33.085 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7099999999999998e-06 2024-05-08T08:52:33.085 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.085 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.085 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.085 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.085 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.085 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15822 3.16 3.83728616 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.085 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.959042+0000", 2024-05-08T08:52:33.085 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.022920686999999999, 2024-05-08T08:52:33.085 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.024291249000000001, 2024-05-08T08:52:33.085 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.085 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.085 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:33.085 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.086 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.086 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.086 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15822 2024-05-08T08:52:33.086 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.086 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.086 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.086 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.086 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.959042+0000", 2024-05-08T08:52:33.086 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.086 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.086 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.086 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.086 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.959042+0000", 2024-05-08T08:52:33.086 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.087 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.087 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.087 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.087 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.959042+0000", 2024-05-08T08:52:33.087 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3799999999999998e-07 2024-05-08T08:52:33.087 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.087 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.087 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.087 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.959051+0000", 2024-05-08T08:52:33.087 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3260000000000002e-06 2024-05-08T08:52:33.087 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.087 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.087 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.087 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.959057+0000", 2024-05-08T08:52:33.088 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7880000000000003e-06 2024-05-08T08:52:33.088 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.088 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.088 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.088 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.959062+0000", 2024-05-08T08:52:33.088 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7360000000000001e-06 2024-05-08T08:52:33.088 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.088 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.088 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.088 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.088 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.088 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15694 3.16 3:6bd6d6da:::rbd_data.1140782590a5.0000000000000015:6 [read 2816000~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.088 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.947121+0000", 2024-05-08T08:52:33.088 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.034841631999999997, 2024-05-08T08:52:33.088 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036272444000000001, 2024-05-08T08:52:33.089 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.089 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.089 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:33.089 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.089 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.089 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.089 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15694 2024-05-08T08:52:33.089 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.089 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.089 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.089 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.089 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.947121+0000", 2024-05-08T08:52:33.089 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.089 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.089 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.090 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.090 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.947121+0000", 2024-05-08T08:52:33.090 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.090 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.090 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.090 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.090 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.947121+0000", 2024-05-08T08:52:33.090 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.06e-07 2024-05-08T08:52:33.090 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.090 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.090 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.090 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.947124+0000", 2024-05-08T08:52:33.090 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6860000000000002e-06 2024-05-08T08:52:33.090 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.091 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.091 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.091 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.947136+0000", 2024-05-08T08:52:33.091 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2007999999999999e-05 2024-05-08T08:52:33.091 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.091 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.091 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.091 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.947142+0000", 2024-05-08T08:52:33.091 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2349999999999996e-06 2024-05-08T08:52:33.091 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.091 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.091 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:33.091 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.983285+0000", 2024-05-08T08:52:33.091 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036142560999999997 2024-05-08T08:52:33.091 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.092 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.092 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:33.092 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.983361+0000", 2024-05-08T08:52:33.092 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5976999999999994e-05 2024-05-08T08:52:33.092 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.092 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.092 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.092 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.092 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.092 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15761 3.16 3.83728616 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.092 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.953041+0000", 2024-05-08T08:52:33.092 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.028921095000000001, 2024-05-08T08:52:33.092 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030416322999999999, 2024-05-08T08:52:33.092 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.092 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.092 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:33.093 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.093 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.093 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.093 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15761 2024-05-08T08:52:33.093 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.093 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.093 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.093 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.093 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.953041+0000", 2024-05-08T08:52:33.093 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.093 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.093 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.093 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.093 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.953041+0000", 2024-05-08T08:52:33.094 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.094 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.094 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.094 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.094 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.953042+0000", 2024-05-08T08:52:33.094 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9000000000000002e-07 2024-05-08T08:52:33.094 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.094 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.094 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.094 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.953044+0000", 2024-05-08T08:52:33.094 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4849999999999999e-06 2024-05-08T08:52:33.094 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.094 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.094 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.095 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.953049+0000", 2024-05-08T08:52:33.095 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6439999999999999e-06 2024-05-08T08:52:33.095 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.095 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.095 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.095 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.953052+0000", 2024-05-08T08:52:33.095 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6720000000000002e-06 2024-05-08T08:52:33.095 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.095 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.095 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.095 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.095 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.095 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15824 3.16 3.83728616 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.095 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.959069+0000", 2024-05-08T08:52:33.096 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.022893879999999998, 2024-05-08T08:52:33.096 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.024418703999999999, 2024-05-08T08:52:33.096 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.096 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.096 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:33.096 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.096 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.096 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.096 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15824 2024-05-08T08:52:33.096 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.096 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.096 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.096 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.096 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.959069+0000", 2024-05-08T08:52:33.096 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.097 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.097 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.097 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.097 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.959069+0000", 2024-05-08T08:52:33.097 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.097 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.097 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.097 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.097 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.959069+0000", 2024-05-08T08:52:33.097 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.32e-07 2024-05-08T08:52:33.097 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.097 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.097 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.097 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.959071+0000", 2024-05-08T08:52:33.097 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2450000000000001e-06 2024-05-08T08:52:33.098 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.098 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.098 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.098 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.959077+0000", 2024-05-08T08:52:33.098 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4879999999999998e-06 2024-05-08T08:52:33.098 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.098 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.098 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.098 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.959081+0000", 2024-05-08T08:52:33.098 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6779999999999999e-06 2024-05-08T08:52:33.098 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.098 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.098 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.098 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.099 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.099 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15697 3.16 3:6bd6d6da:::rbd_data.1140782590a5.0000000000000015:6 [read 2832384~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.099 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.947150+0000", 2024-05-08T08:52:33.099 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.034811996999999997, 2024-05-08T08:52:33.099 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036379029, 2024-05-08T08:52:33.099 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.099 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.099 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:33.099 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.099 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.099 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.099 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15697 2024-05-08T08:52:33.099 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.099 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.100 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.100 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.100 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.947150+0000", 2024-05-08T08:52:33.100 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.100 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.100 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.100 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.100 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.947150+0000", 2024-05-08T08:52:33.100 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.100 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.100 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.100 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.100 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.947151+0000", 2024-05-08T08:52:33.100 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4499999999999997e-07 2024-05-08T08:52:33.100 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.101 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.101 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.101 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.947153+0000", 2024-05-08T08:52:33.101 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.04e-06 2024-05-08T08:52:33.101 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.101 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.101 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.101 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.947159+0000", 2024-05-08T08:52:33.101 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4579999999999998e-06 2024-05-08T08:52:33.101 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.101 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.101 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.101 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.947164+0000", 2024-05-08T08:52:33.101 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9010000000000002e-06 2024-05-08T08:52:33.101 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.101 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.102 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:33.102 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.983470+0000", 2024-05-08T08:52:33.102 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036305207999999999 2024-05-08T08:52:33.102 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.102 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.102 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:33.102 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.983511+0000", 2024-05-08T08:52:33.102 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1755999999999998e-05 2024-05-08T08:52:33.102 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.102 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.102 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.102 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.102 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.102 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15763 3.16 3.83728616 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.103 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.953058+0000", 2024-05-08T08:52:33.103 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.028904033999999999, 2024-05-08T08:52:33.103 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030510576000000001, 2024-05-08T08:52:33.103 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.103 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.103 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:33.103 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.103 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.103 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.103 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15763 2024-05-08T08:52:33.103 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.103 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.103 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.103 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.103 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.953058+0000", 2024-05-08T08:52:33.103 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.104 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.104 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.104 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.104 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.953058+0000", 2024-05-08T08:52:33.104 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.104 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.104 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.104 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.104 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.953059+0000", 2024-05-08T08:52:33.104 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5999999999999999e-07 2024-05-08T08:52:33.104 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.104 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.104 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.104 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.953061+0000", 2024-05-08T08:52:33.104 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1390000000000002e-06 2024-05-08T08:52:33.104 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.105 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.105 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.105 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.953067+0000", 2024-05-08T08:52:33.105 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4590000000000003e-06 2024-05-08T08:52:33.105 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.105 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.105 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.105 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.953072+0000", 2024-05-08T08:52:33.105 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8509999999999998e-06 2024-05-08T08:52:33.105 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.105 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.105 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.105 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.105 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.105 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15827 3.16 3.83728616 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.106 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.959087+0000", 2024-05-08T08:52:33.106 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.022875083000000001, 2024-05-08T08:52:33.106 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.024540589000000002, 2024-05-08T08:52:33.106 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.106 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.106 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:33.106 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.106 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.106 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.106 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15827 2024-05-08T08:52:33.106 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.106 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.106 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.106 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.107 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.959087+0000", 2024-05-08T08:52:33.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.107 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.107 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.959087+0000", 2024-05-08T08:52:33.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.107 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.107 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.959088+0000", 2024-05-08T08:52:33.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9700000000000002e-07 2024-05-08T08:52:33.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.107 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.107 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.959090+0000", 2024-05-08T08:52:33.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9999999999999999e-06 2024-05-08T08:52:33.108 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.108 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.108 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.108 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.959095+0000", 2024-05-08T08:52:33.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5799999999999999e-06 2024-05-08T08:52:33.108 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.108 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.108 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.108 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.959100+0000", 2024-05-08T08:52:33.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5120000000000002e-06 2024-05-08T08:52:33.108 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.108 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.108 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.109 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.109 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.109 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15699 3.16 3:6bd6d6da:::rbd_data.1140782590a5.0000000000000015:6 [read 2848768~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.109 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.947171+0000", 2024-05-08T08:52:33.109 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.03479136, 2024-05-08T08:52:33.109 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036515666000000002, 2024-05-08T08:52:33.109 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.109 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.109 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:33.109 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.109 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.109 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.109 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15699 2024-05-08T08:52:33.109 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.109 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.110 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.110 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.110 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.947171+0000", 2024-05-08T08:52:33.110 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.110 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.110 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.110 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.110 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.947171+0000", 2024-05-08T08:52:33.110 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.110 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.110 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.110 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.110 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.947171+0000", 2024-05-08T08:52:33.110 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0200000000000003e-07 2024-05-08T08:52:33.111 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.111 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.111 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.111 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.947173+0000", 2024-05-08T08:52:33.111 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9700000000000002e-06 2024-05-08T08:52:33.111 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.111 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.111 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.111 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.947180+0000", 2024-05-08T08:52:33.111 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3470000000000004e-06 2024-05-08T08:52:33.111 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.111 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.111 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.111 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.947185+0000", 2024-05-08T08:52:33.111 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0300000000000001e-06 2024-05-08T08:52:33.111 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.112 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.112 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:33.112 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.983581+0000", 2024-05-08T08:52:33.112 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036396089 2024-05-08T08:52:33.112 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.112 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.112 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:33.112 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.983656+0000", 2024-05-08T08:52:33.112 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4750000000000001e-05 2024-05-08T08:52:33.112 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.112 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.112 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.112 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.112 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.112 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15765 3.16 3.83728616 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.112 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.953079+0000", 2024-05-08T08:52:33.113 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.028883644, 2024-05-08T08:52:33.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030675034, 2024-05-08T08:52:33.113 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.113 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.113 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:33.113 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.113 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.113 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.113 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15765 2024-05-08T08:52:33.113 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.113 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.113 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.113 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.113 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.953079+0000", 2024-05-08T08:52:33.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.114 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.114 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.114 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.114 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.953079+0000", 2024-05-08T08:52:33.114 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.114 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.114 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.114 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.114 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.953079+0000", 2024-05-08T08:52:33.114 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9900000000000001e-07 2024-05-08T08:52:33.114 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.114 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.114 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.114 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.953081+0000", 2024-05-08T08:52:33.115 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2450000000000001e-06 2024-05-08T08:52:33.115 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.115 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.115 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.115 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.953087+0000", 2024-05-08T08:52:33.115 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6790000000000002e-06 2024-05-08T08:52:33.115 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.115 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.115 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.115 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.953092+0000", 2024-05-08T08:52:33.115 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9359999999999998e-06 2024-05-08T08:52:33.115 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.115 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.115 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.115 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.115 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.116 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15829 3.16 3.83728616 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.116 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.959106+0000", 2024-05-08T08:52:33.116 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.022856877000000001, 2024-05-08T08:52:33.116 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.024677184000000001, 2024-05-08T08:52:33.116 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.116 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.116 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:33.116 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.116 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.116 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.116 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15829 2024-05-08T08:52:33.116 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.116 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.116 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.117 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.117 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.959106+0000", 2024-05-08T08:52:33.117 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.117 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.117 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.117 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.117 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.959106+0000", 2024-05-08T08:52:33.117 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.117 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.117 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.117 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.117 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.959106+0000", 2024-05-08T08:52:33.117 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9700000000000002e-07 2024-05-08T08:52:33.117 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.117 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.118 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.118 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.959108+0000", 2024-05-08T08:52:33.118 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.894e-06 2024-05-08T08:52:33.118 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.118 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.118 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.118 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.959113+0000", 2024-05-08T08:52:33.118 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6080000000000003e-06 2024-05-08T08:52:33.118 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.118 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.118 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.118 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.959118+0000", 2024-05-08T08:52:33.118 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7990000000000001e-06 2024-05-08T08:52:33.118 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.119 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.119 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.119 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.119 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.119 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15701 3.16 3:6bd6d6da:::rbd_data.1140782590a5.0000000000000015:6 [read 2865152~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.119 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.947192+0000", 2024-05-08T08:52:33.119 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.034770862999999999, 2024-05-08T08:52:33.119 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036629581000000001, 2024-05-08T08:52:33.119 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.119 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.119 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:33.119 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.119 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.119 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.120 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15701 2024-05-08T08:52:33.120 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.120 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.120 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.120 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.120 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.947192+0000", 2024-05-08T08:52:33.120 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.120 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.120 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.120 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.120 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.947192+0000", 2024-05-08T08:52:33.120 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.120 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.120 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.120 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.121 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.947192+0000", 2024-05-08T08:52:33.121 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9900000000000001e-07 2024-05-08T08:52:33.121 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.121 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.121 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.121 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.947194+0000", 2024-05-08T08:52:33.121 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0669999999999999e-06 2024-05-08T08:52:33.121 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.121 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.121 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.121 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.947200+0000", 2024-05-08T08:52:33.121 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4160000000000001e-06 2024-05-08T08:52:33.121 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.121 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.121 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.121 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.947205+0000", 2024-05-08T08:52:33.122 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5929999999999999e-06 2024-05-08T08:52:33.122 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.122 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.122 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:33.122 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.983765+0000", 2024-05-08T08:52:33.122 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036560256999999999 2024-05-08T08:52:33.122 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.122 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.122 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:33.122 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.983807+0000", 2024-05-08T08:52:33.122 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2123999999999999e-05 2024-05-08T08:52:33.122 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.122 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.123 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.123 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.123 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.123 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15767 3.16 3.83728616 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.123 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.953099+0000", 2024-05-08T08:52:33.123 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.028863654999999998, 2024-05-08T08:52:33.123 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030759353, 2024-05-08T08:52:33.123 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.123 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.123 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:33.123 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.123 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.123 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.123 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15767 2024-05-08T08:52:33.123 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.124 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.124 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.124 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.124 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.953099+0000", 2024-05-08T08:52:33.124 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.124 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.124 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.124 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.124 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.953099+0000", 2024-05-08T08:52:33.124 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.124 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.124 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.124 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.124 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.953099+0000", 2024-05-08T08:52:33.124 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9400000000000001e-07 2024-05-08T08:52:33.125 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.125 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.125 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.125 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.953101+0000", 2024-05-08T08:52:33.125 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9630000000000001e-06 2024-05-08T08:52:33.125 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.125 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.125 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.125 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.953107+0000", 2024-05-08T08:52:33.125 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8000000000000004e-06 2024-05-08T08:52:33.125 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.125 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.125 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.125 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.953112+0000", 2024-05-08T08:52:33.125 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7190000000000001e-06 2024-05-08T08:52:33.126 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.126 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.126 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.126 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.126 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.126 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15831 3.16 3.83728616 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.126 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.959124+0000", 2024-05-08T08:52:33.126 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.022838299999999999, 2024-05-08T08:52:33.126 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.024786644, 2024-05-08T08:52:33.126 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.126 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.126 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:33.126 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.127 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.127 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.127 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15831 2024-05-08T08:52:33.127 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.127 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.127 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.127 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.127 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.959124+0000", 2024-05-08T08:52:33.127 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.127 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.127 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.127 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.127 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.959124+0000", 2024-05-08T08:52:33.127 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.127 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.128 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.128 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.128 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.959124+0000", 2024-05-08T08:52:33.128 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8099999999999998e-07 2024-05-08T08:52:33.128 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.128 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.128 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.128 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.959126+0000", 2024-05-08T08:52:33.128 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9939999999999999e-06 2024-05-08T08:52:33.128 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.128 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.128 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.128 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.959132+0000", 2024-05-08T08:52:33.128 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.648e-06 2024-05-08T08:52:33.128 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.129 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.129 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.129 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.959137+0000", 2024-05-08T08:52:33.129 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4669999999999998e-06 2024-05-08T08:52:33.129 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.129 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.129 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.129 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.129 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.129 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15703 3.16 3:6bd6d6da:::rbd_data.1140782590a5.0000000000000015:6 [read 2881536~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.129 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.947212+0000", 2024-05-08T08:52:33.129 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.034750549999999998, 2024-05-08T08:52:33.129 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036765165000000002, 2024-05-08T08:52:33.129 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.129 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.130 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:33.130 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.130 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.130 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.130 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15703 2024-05-08T08:52:33.130 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.130 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.130 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.130 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.130 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.947212+0000", 2024-05-08T08:52:33.130 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.130 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.130 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.130 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.131 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.947212+0000", 2024-05-08T08:52:33.131 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.131 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.131 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.131 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.131 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.947212+0000", 2024-05-08T08:52:33.131 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8099999999999998e-07 2024-05-08T08:52:33.131 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.131 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.131 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.131 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.947220+0000", 2024-05-08T08:52:33.131 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6750000000000002e-06 2024-05-08T08:52:33.131 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.131 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.131 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.131 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.947227+0000", 2024-05-08T08:52:33.132 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6150000000000002e-06 2024-05-08T08:52:33.132 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.132 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.132 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.132 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.947232+0000", 2024-05-08T08:52:33.132 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0969999999999996e-06 2024-05-08T08:52:33.132 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.132 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.132 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:33.132 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.983880+0000", 2024-05-08T08:52:33.132 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036648728999999998 2024-05-08T08:52:33.132 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.132 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.132 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:33.132 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.983962+0000", 2024-05-08T08:52:33.132 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1584999999999999e-05 2024-05-08T08:52:33.133 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.133 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.133 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.133 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.133 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.133 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15769 3.16 3.83728616 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.133 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.953118+0000", 2024-05-08T08:52:33.133 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.028844644999999999, 2024-05-08T08:52:33.133 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030925865, 2024-05-08T08:52:33.133 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.133 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.133 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:33.133 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.133 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.133 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.133 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15769 2024-05-08T08:52:33.133 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.134 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.134 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.134 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.134 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.953118+0000", 2024-05-08T08:52:33.134 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.134 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.134 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.134 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.134 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.953118+0000", 2024-05-08T08:52:33.134 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.134 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.134 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.134 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.134 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.953118+0000", 2024-05-08T08:52:33.134 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7399999999999999e-07 2024-05-08T08:52:33.135 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.135 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.135 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.135 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.953120+0000", 2024-05-08T08:52:33.135 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0930000000000001e-06 2024-05-08T08:52:33.135 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.135 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.135 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.135 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.953126+0000", 2024-05-08T08:52:33.135 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9669999999999998e-06 2024-05-08T08:52:33.135 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.135 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.135 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.135 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.953131+0000", 2024-05-08T08:52:33.135 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7509999999999999e-06 2024-05-08T08:52:33.136 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.136 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.136 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.136 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.136 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.136 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15833 3.16 3.83728616 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.136 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.959142+0000", 2024-05-08T08:52:33.136 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.022820337, 2024-05-08T08:52:33.136 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.024943099999999999, 2024-05-08T08:52:33.136 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.136 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.136 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:33.136 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.136 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.136 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.137 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15833 2024-05-08T08:52:33.137 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.137 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.137 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.137 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.137 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.959142+0000", 2024-05-08T08:52:33.137 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.137 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.137 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.137 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.137 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.959142+0000", 2024-05-08T08:52:33.137 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.137 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.138 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.138 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.138 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.959142+0000", 2024-05-08T08:52:33.138 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4900000000000001e-07 2024-05-08T08:52:33.138 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.138 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.138 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.138 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.959144+0000", 2024-05-08T08:52:33.138 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7519999999999999e-06 2024-05-08T08:52:33.138 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.138 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.138 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.138 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.959149+0000", 2024-05-08T08:52:33.138 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.242e-06 2024-05-08T08:52:33.138 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.139 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.139 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.139 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.959154+0000", 2024-05-08T08:52:33.139 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5669999999999996e-06 2024-05-08T08:52:33.139 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.139 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.139 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.139 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.139 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.139 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15705 3.16 3:6bd6d6da:::rbd_data.1140782590a5.0000000000000015:6 [read 2897920~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.139 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.947314+0000", 2024-05-08T08:52:33.139 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.034648789999999999, 2024-05-08T08:52:33.139 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036807438999999997, 2024-05-08T08:52:33.139 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.139 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.140 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:33.140 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.140 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.140 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.140 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15705 2024-05-08T08:52:33.140 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.140 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.140 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.140 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.140 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.947314+0000", 2024-05-08T08:52:33.140 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.140 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.140 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.140 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.141 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.947314+0000", 2024-05-08T08:52:33.141 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.141 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.141 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.141 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.141 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.947317+0000", 2024-05-08T08:52:33.141 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5939999999999998e-06 2024-05-08T08:52:33.141 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.141 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.141 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.141 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.947320+0000", 2024-05-08T08:52:33.141 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6510000000000001e-06 2024-05-08T08:52:33.141 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.141 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.141 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.142 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.947395+0000", 2024-05-08T08:52:33.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4722999999999996e-05 2024-05-08T08:52:33.142 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.142 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.142 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.142 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.947402+0000", 2024-05-08T08:52:33.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9279999999999993e-06 2024-05-08T08:52:33.142 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.142 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.142 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:33.142 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.984078+0000", 2024-05-08T08:52:33.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036675312000000002 2024-05-08T08:52:33.142 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.142 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.142 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:33.143 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.984121+0000", 2024-05-08T08:52:33.143 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3529999999999998e-05 2024-05-08T08:52:33.143 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.143 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.143 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.143 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.143 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.143 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15771 3.16 3.83728616 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.143 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.953137+0000", 2024-05-08T08:52:33.143 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.028825349, 2024-05-08T08:52:33.143 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031022859999999999, 2024-05-08T08:52:33.143 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.143 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.143 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:33.143 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.143 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.144 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.144 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15771 2024-05-08T08:52:33.144 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.144 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.144 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.144 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.144 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.953137+0000", 2024-05-08T08:52:33.144 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.144 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.144 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.144 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.144 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.953137+0000", 2024-05-08T08:52:33.144 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.144 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.145 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.145 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.145 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.954938+0000", 2024-05-08T08:52:33.145 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001801216 2024-05-08T08:52:33.145 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.145 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.145 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.145 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.954943+0000", 2024-05-08T08:52:33.145 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4519999999999999e-06 2024-05-08T08:52:33.145 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.145 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.145 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.145 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.954952+0000", 2024-05-08T08:52:33.145 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3190000000000001e-06 2024-05-08T08:52:33.145 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.146 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.146 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.146 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.954959+0000", 2024-05-08T08:52:33.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.562e-06 2024-05-08T08:52:33.146 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.146 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.146 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.146 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.146 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15835 3.16 3.83728616 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.146 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.959160+0000", 2024-05-08T08:52:33.146 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.022802804999999999, 2024-05-08T08:52:33.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.025044770000000001, 2024-05-08T08:52:33.146 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.147 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.147 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:33.147 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.147 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.147 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.147 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15835 2024-05-08T08:52:33.147 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.147 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.147 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.147 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.147 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.959160+0000", 2024-05-08T08:52:33.147 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.147 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.147 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.147 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.147 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.959160+0000", 2024-05-08T08:52:33.148 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.148 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.148 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.148 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.960938+0000", 2024-05-08T08:52:33.148 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001778377 2024-05-08T08:52:33.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.148 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.148 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.148 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.960942+0000", 2024-05-08T08:52:33.148 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4240000000000003e-06 2024-05-08T08:52:33.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.148 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.148 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.148 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.960952+0000", 2024-05-08T08:52:33.149 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6069999999999997e-06 2024-05-08T08:52:33.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.149 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.149 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.960958+0000", 2024-05-08T08:52:33.149 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3339999999999998e-06 2024-05-08T08:52:33.149 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.149 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.149 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.149 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15707 3.16 3:6bd6d6da:::rbd_data.1140782590a5.0000000000000015:6 [read 2914304~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.149 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.947412+0000", 2024-05-08T08:52:33.149 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.034550105999999997, 2024-05-08T08:52:33.149 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036852299999999998, 2024-05-08T08:52:33.150 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.150 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.150 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-05-08T08:52:33.150 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.150 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.150 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.150 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15707 2024-05-08T08:52:33.150 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.150 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.150 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.150 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.150 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.947412+0000", 2024-05-08T08:52:33.150 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.150 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.151 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.151 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.151 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.947412+0000", 2024-05-08T08:52:33.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.151 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.151 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.151 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.151 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.947413+0000", 2024-05-08T08:52:33.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2e-07 2024-05-08T08:52:33.151 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.151 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.151 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.151 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.947415+0000", 2024-05-08T08:52:33.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5909999999999998e-06 2024-05-08T08:52:33.151 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.152 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.152 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.152 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.947471+0000", 2024-05-08T08:52:33.152 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5436999999999998e-05 2024-05-08T08:52:33.152 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.152 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.152 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.152 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.947477+0000", 2024-05-08T08:52:33.152 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5130000000000002e-06 2024-05-08T08:52:33.152 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.152 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.152 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:33.152 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.984193+0000", 2024-05-08T08:52:33.152 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036715431 2024-05-08T08:52:33.152 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.152 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.153 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:33.153 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.984268+0000", 2024-05-08T08:52:33.153 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4844000000000004e-05 2024-05-08T08:52:33.153 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.153 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.153 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.153 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.153 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.153 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15773 3.16 3.83728616 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.153 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.954971+0000", 2024-05-08T08:52:33.153 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.026991640000000001, 2024-05-08T08:52:33.153 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029358688000000001, 2024-05-08T08:52:33.153 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.153 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.153 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:33.154 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.154 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.154 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.154 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15773 2024-05-08T08:52:33.154 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.154 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.154 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.154 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.154 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.954971+0000", 2024-05-08T08:52:33.154 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.154 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.154 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.154 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.154 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.954971+0000", 2024-05-08T08:52:33.155 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.155 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.155 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.155 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.155 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.954971+0000", 2024-05-08T08:52:33.155 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9599999999999999e-07 2024-05-08T08:52:33.155 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.155 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.155 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.155 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.954974+0000", 2024-05-08T08:52:33.155 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6989999999999999e-06 2024-05-08T08:52:33.155 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.155 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.155 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.155 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.954981+0000", 2024-05-08T08:52:33.156 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6209999999999998e-06 2024-05-08T08:52:33.156 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.156 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.156 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.156 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.954986+0000", 2024-05-08T08:52:33.156 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4419999999999997e-06 2024-05-08T08:52:33.156 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.156 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.156 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.156 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.156 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.156 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15837 3.16 3.83728616 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.156 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.960971+0000", 2024-05-08T08:52:33.156 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.020991704, 2024-05-08T08:52:33.156 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.023407941000000002, 2024-05-08T08:52:33.156 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.157 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.157 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:33.157 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.157 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.157 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.157 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15837 2024-05-08T08:52:33.157 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.157 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.157 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.157 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.157 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.960971+0000", 2024-05-08T08:52:33.157 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.157 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.157 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.157 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.158 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.960971+0000", 2024-05-08T08:52:33.158 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.158 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.158 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.158 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.158 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.960971+0000", 2024-05-08T08:52:33.158 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1699999999999998e-07 2024-05-08T08:52:33.158 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.158 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.158 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.158 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.960974+0000", 2024-05-08T08:52:33.158 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3499999999999999e-06 2024-05-08T08:52:33.158 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.159 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.159 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.159 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.960980+0000", 2024-05-08T08:52:33.159 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5339999999999996e-06 2024-05-08T08:52:33.159 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.159 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.159 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.159 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.960985+0000", 2024-05-08T08:52:33.159 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.135e-06 2024-05-08T08:52:33.159 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.159 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.159 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.159 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.159 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.159 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15709 3.16 3:6bd6d6da:::rbd_data.1140782590a5.0000000000000015:6 [read 2930688~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.159 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.947486+0000", 2024-05-08T08:52:33.160 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.034476566, 2024-05-08T08:52:33.160 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036928804000000003, 2024-05-08T08:52:33.160 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.160 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.160 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-05-08T08:52:33.160 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.160 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.160 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.160 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15709 2024-05-08T08:52:33.160 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.160 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.160 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.160 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.160 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.947486+0000", 2024-05-08T08:52:33.160 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.161 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.161 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.161 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.161 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.947486+0000", 2024-05-08T08:52:33.161 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.161 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.161 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.161 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.161 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.947486+0000", 2024-05-08T08:52:33.161 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5700000000000002e-07 2024-05-08T08:52:33.161 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.161 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.161 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.161 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.947489+0000", 2024-05-08T08:52:33.161 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.65e-06 2024-05-08T08:52:33.161 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.162 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.162 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.162 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.947496+0000", 2024-05-08T08:52:33.162 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3220000000000004e-06 2024-05-08T08:52:33.162 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.162 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.162 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.162 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.947502+0000", 2024-05-08T08:52:33.162 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1909999999999999e-06 2024-05-08T08:52:33.162 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.162 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.162 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:33.162 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.984378+0000", 2024-05-08T08:52:33.163 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036876863000000003 2024-05-08T08:52:33.163 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.163 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.163 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.163 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.163 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.163 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15775 3.16 3.83728616 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.163 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.954993+0000", 2024-05-08T08:52:33.163 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.026969631000000001, 2024-05-08T08:52:33.163 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029458710999999999, 2024-05-08T08:52:33.163 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.163 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.163 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:33.163 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.163 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.163 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.164 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15775 2024-05-08T08:52:33.164 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.164 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.164 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.164 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.164 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.954993+0000", 2024-05-08T08:52:33.164 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.164 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.164 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.164 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.164 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.954993+0000", 2024-05-08T08:52:33.164 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.164 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.164 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.164 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.165 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.954993+0000", 2024-05-08T08:52:33.165 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4999999999999998e-07 2024-05-08T08:52:33.165 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.165 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.165 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.165 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.955000+0000", 2024-05-08T08:52:33.165 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1369999999999996e-06 2024-05-08T08:52:33.165 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.165 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.165 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.165 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.955007+0000", 2024-05-08T08:52:33.165 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5899999999999996e-06 2024-05-08T08:52:33.165 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.165 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.165 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.166 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.955012+0000", 2024-05-08T08:52:33.166 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0220000000000004e-06 2024-05-08T08:52:33.166 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.166 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.166 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.166 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.166 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.166 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15839 3.16 3.83728616 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.166 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.960992+0000", 2024-05-08T08:52:33.166 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.020970471000000001, 2024-05-08T08:52:33.166 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.023487151000000001, 2024-05-08T08:52:33.166 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.166 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.166 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:33.167 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.167 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.167 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.167 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15839 2024-05-08T08:52:33.167 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.167 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.167 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.167 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.167 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.960992+0000", 2024-05-08T08:52:33.167 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.167 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.167 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.167 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.167 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.960992+0000", 2024-05-08T08:52:33.167 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.168 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.168 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.168 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.168 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.960992+0000", 2024-05-08T08:52:33.168 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9000000000000002e-07 2024-05-08T08:52:33.168 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.168 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.168 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.168 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.960994+0000", 2024-05-08T08:52:33.168 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9470000000000002e-06 2024-05-08T08:52:33.168 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.168 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.168 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.168 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.961000+0000", 2024-05-08T08:52:33.168 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.485e-06 2024-05-08T08:52:33.169 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.169 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.169 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.169 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.961004+0000", 2024-05-08T08:52:33.169 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5800000000000002e-06 2024-05-08T08:52:33.169 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.169 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.169 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.169 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.169 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.169 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15711 3.16 3.5b6b6bd6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.169 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.947509+0000", 2024-05-08T08:52:33.169 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.034453875000000002, 2024-05-08T08:52:33.169 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.037007940000000003, 2024-05-08T08:52:33.169 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.170 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.170 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-05-08T08:52:33.170 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.170 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.170 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.170 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15711 2024-05-08T08:52:33.170 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.170 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.170 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.170 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.170 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.947509+0000", 2024-05-08T08:52:33.170 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.170 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.170 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.171 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.171 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.947509+0000", 2024-05-08T08:52:33.171 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.171 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.171 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.171 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.171 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.947509+0000", 2024-05-08T08:52:33.171 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1100000000000001e-07 2024-05-08T08:52:33.171 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.171 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.171 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.171 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.947511+0000", 2024-05-08T08:52:33.171 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0549999999999998e-06 2024-05-08T08:52:33.171 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.171 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.172 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.172 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.947518+0000", 2024-05-08T08:52:33.172 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6760000000000001e-06 2024-05-08T08:52:33.172 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.172 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.172 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.172 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.947523+0000", 2024-05-08T08:52:33.172 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0660000000000003e-06 2024-05-08T08:52:33.172 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.172 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.172 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:33.172 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.984518+0000", 2024-05-08T08:52:33.172 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036995778 2024-05-08T08:52:33.172 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.173 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.173 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.173 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.173 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.173 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15777 3.16 3.83728616 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.173 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.955019+0000", 2024-05-08T08:52:33.173 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.026943353999999999, 2024-05-08T08:52:33.173 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029556724999999999, 2024-05-08T08:52:33.173 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.173 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.173 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:33.173 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.173 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.173 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.173 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15777 2024-05-08T08:52:33.174 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.174 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.174 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.174 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.174 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.955019+0000", 2024-05-08T08:52:33.174 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.174 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.174 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.174 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.174 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.955019+0000", 2024-05-08T08:52:33.174 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.174 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.174 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.174 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.175 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.955019+0000", 2024-05-08T08:52:33.175 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:52:33.175 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.175 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.175 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.175 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.955021+0000", 2024-05-08T08:52:33.175 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9980000000000002e-06 2024-05-08T08:52:33.175 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.175 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.175 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.175 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.955027+0000", 2024-05-08T08:52:33.175 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8309999999999997e-06 2024-05-08T08:52:33.175 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.175 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.175 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.175 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.955032+0000", 2024-05-08T08:52:33.176 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-05-08T08:52:33.176 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T08:52:33.176 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-05-08T08:52:33.176 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-05-08T08:52:33.177 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:52:33.178 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:52:33.179 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:52:33.179 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:52:33.180 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:52:33.180 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:52:33.180 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9899999999999997e-06 2024-05-08T08:52:33.180 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.180 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.180 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.181 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.181 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.181 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15841 3.16 3.83728616 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.181 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.961010+0000", 2024-05-08T08:52:33.181 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.020952038999999999, 2024-05-08T08:52:33.181 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.023621781000000001, 2024-05-08T08:52:33.181 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.181 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.181 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:33.181 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.181 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.181 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.181 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15841 2024-05-08T08:52:33.181 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.181 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.182 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.182 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.182 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.961010+0000", 2024-05-08T08:52:33.182 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.182 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.182 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.182 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.182 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.961010+0000", 2024-05-08T08:52:33.182 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.182 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.182 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.182 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.182 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.961011+0000", 2024-05-08T08:52:33.182 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9499999999999998e-07 2024-05-08T08:52:33.183 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.183 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.183 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.183 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.961013+0000", 2024-05-08T08:52:33.183 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.114e-06 2024-05-08T08:52:33.183 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.183 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.183 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.183 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.961019+0000", 2024-05-08T08:52:33.183 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1480000000000003e-06 2024-05-08T08:52:33.183 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.183 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.183 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.183 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.961024+0000", 2024-05-08T08:52:33.183 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7090000000000001e-06 2024-05-08T08:52:33.184 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.184 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.184 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.184 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.184 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.184 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15713 3.16 3.5b6b6bd6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.184 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.947529+0000", 2024-05-08T08:52:33.184 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.034433053999999998, 2024-05-08T08:52:33.184 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.037157845000000002, 2024-05-08T08:52:33.184 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.184 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.184 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:33.184 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.184 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.184 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.185 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15713 2024-05-08T08:52:33.185 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.185 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.185 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.185 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.185 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.947529+0000", 2024-05-08T08:52:33.185 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.185 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.185 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.185 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.185 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.947529+0000", 2024-05-08T08:52:33.185 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.185 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.185 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.186 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.186 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.948945+0000", 2024-05-08T08:52:33.186 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0014157499999999999 2024-05-08T08:52:33.186 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.186 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.186 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.186 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.948950+0000", 2024-05-08T08:52:33.186 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1900000000000003e-06 2024-05-08T08:52:33.186 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.186 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.186 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.186 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.948966+0000", 2024-05-08T08:52:33.186 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6150999999999998e-05 2024-05-08T08:52:33.186 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.187 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.187 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.187 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.948975+0000", 2024-05-08T08:52:33.187 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6929999999999996e-06 2024-05-08T08:52:33.187 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.187 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.187 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.187 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.187 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.187 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15779 3.16 3.83728616 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.187 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.955039+0000", 2024-05-08T08:52:33.187 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.026923685999999999, 2024-05-08T08:52:33.187 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029690557999999999, 2024-05-08T08:52:33.187 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.188 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.188 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:33.188 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.188 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.188 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.188 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15779 2024-05-08T08:52:33.188 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.188 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.188 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.188 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.188 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.955039+0000", 2024-05-08T08:52:33.188 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.188 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.188 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.188 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.189 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.955039+0000", 2024-05-08T08:52:33.189 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.189 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.189 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.189 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.189 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.955039+0000", 2024-05-08T08:52:33.189 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3500000000000002e-07 2024-05-08T08:52:33.189 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.189 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.189 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.189 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.955042+0000", 2024-05-08T08:52:33.189 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.441e-06 2024-05-08T08:52:33.189 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.189 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.189 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.190 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.955047+0000", 2024-05-08T08:52:33.190 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8499999999999999e-06 2024-05-08T08:52:33.190 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.190 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.190 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.190 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.955052+0000", 2024-05-08T08:52:33.190 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7740000000000004e-06 2024-05-08T08:52:33.190 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.190 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.190 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.190 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.190 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.190 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15843 3.16 3.83728616 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.190 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.961030+0000", 2024-05-08T08:52:33.191 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.020932460999999999, 2024-05-08T08:52:33.191 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.023727531999999999, 2024-05-08T08:52:33.191 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.191 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.191 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:33.191 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.191 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.191 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.191 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15843 2024-05-08T08:52:33.191 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.191 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.191 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.191 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.191 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.961030+0000", 2024-05-08T08:52:33.192 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.192 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.192 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.192 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.192 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.961030+0000", 2024-05-08T08:52:33.192 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.192 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.192 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.192 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.192 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.961030+0000", 2024-05-08T08:52:33.192 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1100000000000001e-07 2024-05-08T08:52:33.192 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.192 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.192 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.192 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.961033+0000", 2024-05-08T08:52:33.193 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3149999999999999e-06 2024-05-08T08:52:33.193 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.193 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.193 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.193 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.961038+0000", 2024-05-08T08:52:33.193 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7969999999999997e-06 2024-05-08T08:52:33.193 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.193 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.193 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.193 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.961043+0000", 2024-05-08T08:52:33.193 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7729999999999999e-06 2024-05-08T08:52:33.193 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.193 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.193 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.193 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.193 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.194 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15715 3.16 3.5b6b6bd6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.194 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.948997+0000", 2024-05-08T08:52:33.194 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.032965876999999998, 2024-05-08T08:52:33.194 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035788024000000002, 2024-05-08T08:52:33.194 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.194 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.194 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:33.194 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.194 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.194 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.194 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15715 2024-05-08T08:52:33.194 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.194 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.195 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.195 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.195 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.948997+0000", 2024-05-08T08:52:33.195 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.195 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.195 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.195 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.195 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.948997+0000", 2024-05-08T08:52:33.195 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.195 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.195 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.195 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.195 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.948997+0000", 2024-05-08T08:52:33.195 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4099999999999998e-07 2024-05-08T08:52:33.196 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.196 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.196 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.196 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.949000+0000", 2024-05-08T08:52:33.196 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7999999999999999e-06 2024-05-08T08:52:33.196 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.196 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.196 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.196 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.949007+0000", 2024-05-08T08:52:33.196 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6499999999999999e-06 2024-05-08T08:52:33.196 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.196 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.196 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.196 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.949012+0000", 2024-05-08T08:52:33.197 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5049999999999998e-06 2024-05-08T08:52:33.197 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.197 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.197 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.197 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.197 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.197 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15781 3.16 3.83728616 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.197 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.955096+0000", 2024-05-08T08:52:33.197 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.026866766, 2024-05-08T08:52:33.197 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029719132999999998, 2024-05-08T08:52:33.197 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.197 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.197 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:33.197 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.197 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.198 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.198 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15781 2024-05-08T08:52:33.198 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.198 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.198 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.198 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.198 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.955096+0000", 2024-05-08T08:52:33.198 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.198 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.198 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.198 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.198 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.955096+0000", 2024-05-08T08:52:33.198 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.198 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.198 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.199 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.199 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.955096+0000", 2024-05-08T08:52:33.199 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6300000000000005e-07 2024-05-08T08:52:33.199 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.199 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.199 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.199 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.955099+0000", 2024-05-08T08:52:33.199 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.695e-06 2024-05-08T08:52:33.199 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.199 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.199 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.199 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.955106+0000", 2024-05-08T08:52:33.199 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5660000000000001e-06 2024-05-08T08:52:33.199 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.199 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.200 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.200 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.955112+0000", 2024-05-08T08:52:33.200 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4580000000000001e-06 2024-05-08T08:52:33.200 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.200 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.200 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.200 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.200 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.200 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15845 3.16 3.83728616 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.200 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.961049+0000", 2024-05-08T08:52:33.200 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.020912938999999998, 2024-05-08T08:52:33.200 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.023811506999999999, 2024-05-08T08:52:33.200 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.200 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.201 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:33.201 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.201 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.201 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.201 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15845 2024-05-08T08:52:33.201 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.201 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.201 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.201 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.201 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.961049+0000", 2024-05-08T08:52:33.201 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.201 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.201 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.201 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.201 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.961049+0000", 2024-05-08T08:52:33.202 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.202 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.202 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.202 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.202 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.961050+0000", 2024-05-08T08:52:33.202 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1699999999999998e-07 2024-05-08T08:52:33.202 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.202 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.202 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.202 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.961052+0000", 2024-05-08T08:52:33.202 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1110000000000002e-06 2024-05-08T08:52:33.202 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.202 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.202 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.203 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.961058+0000", 2024-05-08T08:52:33.203 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.925e-06 2024-05-08T08:52:33.203 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.203 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.203 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.203 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.961063+0000", 2024-05-08T08:52:33.203 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6149999999999999e-06 2024-05-08T08:52:33.203 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.203 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.203 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.203 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.203 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.203 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15718 3.16 3.5b6b6bd6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.203 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.949025+0000", 2024-05-08T08:52:33.204 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.032937554000000001, 2024-05-08T08:52:33.204 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035892500000000001, 2024-05-08T08:52:33.204 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.204 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.204 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:33.204 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.204 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.204 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.204 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15718 2024-05-08T08:52:33.204 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.204 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.204 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.204 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.204 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.949025+0000", 2024-05-08T08:52:33.204 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.204 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.205 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.205 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.205 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.949025+0000", 2024-05-08T08:52:33.205 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.205 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.205 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.205 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.205 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.949025+0000", 2024-05-08T08:52:33.205 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9999999999999998e-07 2024-05-08T08:52:33.205 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.205 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.205 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.205 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.949028+0000", 2024-05-08T08:52:33.205 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4030000000000001e-06 2024-05-08T08:52:33.205 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.206 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.206 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.206 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.949034+0000", 2024-05-08T08:52:33.206 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2790000000000004e-06 2024-05-08T08:52:33.206 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.206 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.206 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.206 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.949039+0000", 2024-05-08T08:52:33.206 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9250000000000003e-06 2024-05-08T08:52:33.206 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.206 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.206 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.206 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.206 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.207 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15783 3.16 3.83728616 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.207 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.955120+0000", 2024-05-08T08:52:33.207 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.026841883, 2024-05-08T08:52:33.207 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029856698000000001, 2024-05-08T08:52:33.207 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.207 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.207 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:33.207 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.207 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.207 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.207 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15783 2024-05-08T08:52:33.207 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.207 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.207 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.207 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.208 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.955120+0000", 2024-05-08T08:52:33.208 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.208 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.208 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.208 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.208 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.955120+0000", 2024-05-08T08:52:33.208 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.208 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.208 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.208 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.208 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.955121+0000", 2024-05-08T08:52:33.208 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5600000000000001e-07 2024-05-08T08:52:33.208 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.208 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.209 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.209 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.955123+0000", 2024-05-08T08:52:33.209 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8530000000000001e-06 2024-05-08T08:52:33.209 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.209 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.209 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.209 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.955129+0000", 2024-05-08T08:52:33.209 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.998e-06 2024-05-08T08:52:33.209 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.209 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.209 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.209 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.955133+0000", 2024-05-08T08:52:33.209 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.544e-06 2024-05-08T08:52:33.209 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.209 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.210 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.210 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.210 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.210 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15847 3.16 3.83728616 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.210 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.961069+0000", 2024-05-08T08:52:33.210 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.020893663999999999, 2024-05-08T08:52:33.210 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.024008338000000001, 2024-05-08T08:52:33.210 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.210 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.210 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:33.210 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.210 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.210 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.211 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15847 2024-05-08T08:52:33.211 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.211 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.211 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.211 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.211 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.961069+0000", 2024-05-08T08:52:33.211 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.211 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.211 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.211 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.211 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.961069+0000", 2024-05-08T08:52:33.211 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.211 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.211 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.212 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.212 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.961069+0000", 2024-05-08T08:52:33.212 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0200000000000003e-07 2024-05-08T08:52:33.212 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.212 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.212 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.212 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.961076+0000", 2024-05-08T08:52:33.212 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.878e-06 2024-05-08T08:52:33.212 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.212 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.212 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.212 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.961084+0000", 2024-05-08T08:52:33.212 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5079999999999999e-06 2024-05-08T08:52:33.212 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.212 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.213 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.213 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.961088+0000", 2024-05-08T08:52:33.213 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.651e-06 2024-05-08T08:52:33.213 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.213 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.213 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.213 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.213 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.213 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15720 3.16 3.5b6b6bd6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.213 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.949045+0000", 2024-05-08T08:52:33.213 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.032917077000000003, 2024-05-08T08:52:33.213 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036059910000000001, 2024-05-08T08:52:33.213 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.213 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.213 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:33.214 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.214 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.214 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.214 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15720 2024-05-08T08:52:33.214 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.214 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.214 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.214 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.214 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.949045+0000", 2024-05-08T08:52:33.214 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.214 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.214 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.214 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.214 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.949045+0000", 2024-05-08T08:52:33.214 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.215 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.215 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.215 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.215 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.949046+0000", 2024-05-08T08:52:33.215 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8099999999999998e-07 2024-05-08T08:52:33.215 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.215 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.215 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.215 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.949048+0000", 2024-05-08T08:52:33.215 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.255e-06 2024-05-08T08:52:33.215 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.215 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.215 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.215 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.949054+0000", 2024-05-08T08:52:33.215 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7139999999999998e-06 2024-05-08T08:52:33.216 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.216 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.216 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.216 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.949058+0000", 2024-05-08T08:52:33.216 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6229999999999996e-06 2024-05-08T08:52:33.216 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.216 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.216 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.216 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.216 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.216 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15785 3.16 3.83728616 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.216 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.955138+0000", 2024-05-08T08:52:33.216 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.026824601999999999, 2024-05-08T08:52:33.216 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029995939999999999, 2024-05-08T08:52:33.216 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.217 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.217 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:33.217 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.217 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.217 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.217 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15785 2024-05-08T08:52:33.217 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.217 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.217 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.217 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.217 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.955138+0000", 2024-05-08T08:52:33.217 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.217 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.217 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.217 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.217 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.955138+0000", 2024-05-08T08:52:33.218 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.218 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.218 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.218 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.218 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.955138+0000", 2024-05-08T08:52:33.218 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.35e-07 2024-05-08T08:52:33.218 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.218 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.218 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.218 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.955139+0000", 2024-05-08T08:52:33.218 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1939999999999999e-06 2024-05-08T08:52:33.218 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.218 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.218 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.219 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.955142+0000", 2024-05-08T08:52:33.219 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9069999999999999e-06 2024-05-08T08:52:33.219 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.219 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.219 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.219 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.955145+0000", 2024-05-08T08:52:33.219 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1410000000000001e-06 2024-05-08T08:52:33.219 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.219 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.219 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.219 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.219 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.219 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15849 3.16 3.83728616 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.219 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.961095+0000", 2024-05-08T08:52:33.219 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.020867245, 2024-05-08T08:52:33.219 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.024080727999999999, 2024-05-08T08:52:33.220 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.220 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.220 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:33.220 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.220 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.220 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.220 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15849 2024-05-08T08:52:33.220 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.220 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.220 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.220 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.220 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.961095+0000", 2024-05-08T08:52:33.220 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.220 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.220 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.221 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.221 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.961095+0000", 2024-05-08T08:52:33.221 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.221 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.221 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.221 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.221 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.961096+0000", 2024-05-08T08:52:33.221 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8500000000000002e-07 2024-05-08T08:52:33.221 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.221 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.221 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.221 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.961098+0000", 2024-05-08T08:52:33.221 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2900000000000001e-06 2024-05-08T08:52:33.221 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.221 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.222 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.222 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.961104+0000", 2024-05-08T08:52:33.222 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1129999999999997e-06 2024-05-08T08:52:33.222 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.222 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.222 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.222 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.961109+0000", 2024-05-08T08:52:33.222 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6450000000000004e-06 2024-05-08T08:52:33.222 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.222 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.222 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.222 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.222 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.222 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15722 3.16 3.5b6b6bd6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.223 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.949064+0000", 2024-05-08T08:52:33.223 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.032898295000000001, 2024-05-08T08:52:33.223 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036171897000000001, 2024-05-08T08:52:33.223 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.223 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.223 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:33.223 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.223 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.223 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.223 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15722 2024-05-08T08:52:33.223 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.223 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.223 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.223 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.224 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.949064+0000", 2024-05-08T08:52:33.224 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.224 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.224 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.224 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.224 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.949064+0000", 2024-05-08T08:52:33.224 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.224 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.224 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.224 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.224 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.949064+0000", 2024-05-08T08:52:33.224 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9099999999999999e-07 2024-05-08T08:52:33.224 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.224 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.224 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.225 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.949066+0000", 2024-05-08T08:52:33.225 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9910000000000001e-06 2024-05-08T08:52:33.225 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.225 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.225 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.225 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.949072+0000", 2024-05-08T08:52:33.225 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1710000000000001e-06 2024-05-08T08:52:33.225 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.225 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.225 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.225 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.949076+0000", 2024-05-08T08:52:33.225 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1239999999999998e-06 2024-05-08T08:52:33.225 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.225 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.225 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.226 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.226 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.226 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15786 3.16 3.83728616 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.226 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.955150+0000", 2024-05-08T08:52:33.226 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.026812266000000001, 2024-05-08T08:52:33.226 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030139562000000002, 2024-05-08T08:52:33.226 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.226 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.226 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:33.226 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.226 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.226 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.226 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15786 2024-05-08T08:52:33.226 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.227 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.227 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.227 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.227 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.955150+0000", 2024-05-08T08:52:33.227 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.227 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.227 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.227 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.227 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.955150+0000", 2024-05-08T08:52:33.227 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.227 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.227 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.227 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.227 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.955150+0000", 2024-05-08T08:52:33.227 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6399999999999998e-07 2024-05-08T08:52:33.228 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.228 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.228 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.228 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.955152+0000", 2024-05-08T08:52:33.228 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9180000000000001e-06 2024-05-08T08:52:33.228 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.228 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.228 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.228 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.955158+0000", 2024-05-08T08:52:33.228 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1170000000000003e-06 2024-05-08T08:52:33.228 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.228 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.228 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.228 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.955162+0000", 2024-05-08T08:52:33.228 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5029999999999999e-06 2024-05-08T08:52:33.229 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.229 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.229 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.229 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.229 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.229 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15851 3.16 3.83728616 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.229 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.961115+0000", 2024-05-08T08:52:33.229 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.020847193, 2024-05-08T08:52:33.229 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.024230538999999999, 2024-05-08T08:52:33.229 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.229 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.229 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:33.229 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.229 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.229 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.230 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15851 2024-05-08T08:52:33.230 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.230 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.230 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.230 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.230 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.961115+0000", 2024-05-08T08:52:33.230 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.230 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.230 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.230 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.230 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.961115+0000", 2024-05-08T08:52:33.230 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.230 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.230 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.231 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.231 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.961116+0000", 2024-05-08T08:52:33.231 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.32e-07 2024-05-08T08:52:33.231 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.231 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.231 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.231 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.961118+0000", 2024-05-08T08:52:33.231 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.199e-06 2024-05-08T08:52:33.231 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.231 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.231 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.231 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.961124+0000", 2024-05-08T08:52:33.231 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7559999999999996e-06 2024-05-08T08:52:33.231 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.232 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.232 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.232 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.961128+0000", 2024-05-08T08:52:33.232 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6619999999999996e-06 2024-05-08T08:52:33.232 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.232 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.232 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.232 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.232 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.232 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15724 3.16 3.5b6b6bd6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.232 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.949081+0000", 2024-05-08T08:52:33.232 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.032881071999999997, 2024-05-08T08:52:33.232 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036302971000000003, 2024-05-08T08:52:33.232 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.232 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.232 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:33.233 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.233 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.233 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.233 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15724 2024-05-08T08:52:33.233 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.233 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.233 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.233 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.233 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.949081+0000", 2024-05-08T08:52:33.233 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.233 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.233 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.233 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.233 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.949081+0000", 2024-05-08T08:52:33.233 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.234 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.234 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.234 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.234 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.949082+0000", 2024-05-08T08:52:33.234 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:52:33.234 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.234 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.234 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.234 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.949084+0000", 2024-05-08T08:52:33.234 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0219999999999999e-06 2024-05-08T08:52:33.234 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.234 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.234 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.234 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.949090+0000", 2024-05-08T08:52:33.235 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0700000000000003e-06 2024-05-08T08:52:33.235 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.235 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.235 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.235 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.949095+0000", 2024-05-08T08:52:33.235 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7820000000000001e-06 2024-05-08T08:52:33.235 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.235 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.235 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.235 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.235 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.235 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15788 3.16 3.83728616 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.235 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.955168+0000", 2024-05-08T08:52:33.235 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.026793906999999999, 2024-05-08T08:52:33.235 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030245365999999999, 2024-05-08T08:52:33.236 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.236 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.236 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:33.236 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.236 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.236 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.236 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15788 2024-05-08T08:52:33.236 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.236 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.236 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.236 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.236 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.955168+0000", 2024-05-08T08:52:33.236 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.236 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.237 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.237 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.237 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.955168+0000", 2024-05-08T08:52:33.237 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.237 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.237 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.237 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.237 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.955169+0000", 2024-05-08T08:52:33.237 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.72e-07 2024-05-08T08:52:33.237 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.237 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.237 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.237 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.955171+0000", 2024-05-08T08:52:33.237 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.153e-06 2024-05-08T08:52:33.237 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.237 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.238 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.238 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.955177+0000", 2024-05-08T08:52:33.238 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.716e-06 2024-05-08T08:52:33.238 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.238 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.238 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.238 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.955181+0000", 2024-05-08T08:52:33.238 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5140000000000003e-06 2024-05-08T08:52:33.238 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.238 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.238 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.238 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.238 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.239 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15853 3.16 3.83728616 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.239 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.961134+0000", 2024-05-08T08:52:33.239 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.020828010000000001, 2024-05-08T08:52:33.239 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.024306174999999999, 2024-05-08T08:52:33.239 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.239 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.239 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:33.239 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.239 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.239 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.239 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15853 2024-05-08T08:52:33.239 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.239 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.239 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.239 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.239 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.961134+0000", 2024-05-08T08:52:33.240 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.240 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.240 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.240 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.240 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.961134+0000", 2024-05-08T08:52:33.240 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.240 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.240 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.240 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.240 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.961135+0000", 2024-05-08T08:52:33.240 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.65e-07 2024-05-08T08:52:33.240 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.240 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.240 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.241 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.961137+0000", 2024-05-08T08:52:33.241 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.035e-06 2024-05-08T08:52:33.241 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.241 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.241 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.241 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.961142+0000", 2024-05-08T08:52:33.241 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4879999999999998e-06 2024-05-08T08:52:33.241 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.241 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.241 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.241 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.961147+0000", 2024-05-08T08:52:33.241 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7450000000000003e-06 2024-05-08T08:52:33.241 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.241 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.242 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.242 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.242 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.242 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15726 3.16 3.5b6b6bd6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.242 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.949101+0000", 2024-05-08T08:52:33.242 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.032861735000000003, 2024-05-08T08:52:33.242 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036370327000000001, 2024-05-08T08:52:33.242 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.242 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.242 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:33.242 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.242 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.242 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.242 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15726 2024-05-08T08:52:33.242 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.243 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.243 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.243 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.243 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.949101+0000", 2024-05-08T08:52:33.243 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.243 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.243 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.243 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.243 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.949101+0000", 2024-05-08T08:52:33.243 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.243 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.243 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.243 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.243 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.949101+0000", 2024-05-08T08:52:33.243 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9000000000000002e-07 2024-05-08T08:52:33.244 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.244 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.244 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.244 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.949103+0000", 2024-05-08T08:52:33.244 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9769999999999999e-06 2024-05-08T08:52:33.244 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.244 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.244 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.244 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.949108+0000", 2024-05-08T08:52:33.244 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0119999999999996e-06 2024-05-08T08:52:33.244 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.244 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.244 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.244 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.949112+0000", 2024-05-08T08:52:33.244 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2470000000000002e-06 2024-05-08T08:52:33.244 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.245 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.245 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.245 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.245 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.245 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15790 3.16 3.83728616 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.245 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.955187+0000", 2024-05-08T08:52:33.245 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.026775119, 2024-05-08T08:52:33.245 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030335089999999999, 2024-05-08T08:52:33.245 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.245 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.245 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:33.245 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.245 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.245 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.246 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15790 2024-05-08T08:52:33.246 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.246 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.246 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.246 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.246 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.955187+0000", 2024-05-08T08:52:33.246 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.246 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.246 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.246 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.246 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.955187+0000", 2024-05-08T08:52:33.246 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.246 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.246 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.247 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.247 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.956951+0000", 2024-05-08T08:52:33.247 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001763943 2024-05-08T08:52:33.247 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.247 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.247 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.247 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.956955+0000", 2024-05-08T08:52:33.247 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0899999999999998e-06 2024-05-08T08:52:33.247 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.247 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.247 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.247 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.956964+0000", 2024-05-08T08:52:33.247 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9199999999999993e-06 2024-05-08T08:52:33.247 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.247 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.248 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.248 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.956971+0000", 2024-05-08T08:52:33.248 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4799999999999998e-06 2024-05-08T08:52:33.248 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.248 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.248 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.248 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.248 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.248 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15855 3.16 3.83728616 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.248 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.961153+0000", 2024-05-08T08:52:33.248 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.020809283000000001, 2024-05-08T08:52:33.248 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.024424083999999999, 2024-05-08T08:52:33.248 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.248 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.249 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:33.249 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.249 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.249 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.249 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15855 2024-05-08T08:52:33.249 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.249 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.249 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.249 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.249 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.961153+0000", 2024-05-08T08:52:33.249 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.249 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.249 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.249 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.250 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.961153+0000", 2024-05-08T08:52:33.250 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.250 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.250 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.250 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.250 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.961153+0000", 2024-05-08T08:52:33.250 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7599999999999998e-07 2024-05-08T08:52:33.250 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.250 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.250 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.250 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.961156+0000", 2024-05-08T08:52:33.250 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0810000000000001e-06 2024-05-08T08:52:33.250 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.250 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.251 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.251 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.961161+0000", 2024-05-08T08:52:33.251 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8139999999999997e-06 2024-05-08T08:52:33.251 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.251 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.251 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.251 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.961166+0000", 2024-05-08T08:52:33.251 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5410000000000002e-06 2024-05-08T08:52:33.251 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.251 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.251 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.251 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.251 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.251 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15728 3.16 3.5b6b6bd6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.251 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.949119+0000", 2024-05-08T08:52:33.252 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.032843638000000001, 2024-05-08T08:52:33.252 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036515328, 2024-05-08T08:52:33.252 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.252 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.252 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:33.252 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.252 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.252 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.252 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15728 2024-05-08T08:52:33.252 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.252 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.252 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.252 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.252 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.949119+0000", 2024-05-08T08:52:33.252 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.253 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.253 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.253 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.253 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.949119+0000", 2024-05-08T08:52:33.253 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.253 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.253 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.253 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.253 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.949119+0000", 2024-05-08T08:52:33.253 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2899999999999999e-07 2024-05-08T08:52:33.253 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.253 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.253 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.253 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.949126+0000", 2024-05-08T08:52:33.253 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0140000000000001e-06 2024-05-08T08:52:33.254 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.254 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.254 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.254 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.949138+0000", 2024-05-08T08:52:33.254 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1851000000000001e-05 2024-05-08T08:52:33.254 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.254 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.254 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.254 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.949143+0000", 2024-05-08T08:52:33.254 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.31e-06 2024-05-08T08:52:33.254 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.254 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.254 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.254 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.255 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.255 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15792 3.16 3.83728616 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.255 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.956982+0000", 2024-05-08T08:52:33.255 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.024980347999999999, 2024-05-08T08:52:33.255 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028704632000000001, 2024-05-08T08:52:33.255 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.255 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.255 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:33.255 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.255 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.255 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.255 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15792 2024-05-08T08:52:33.255 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.255 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.255 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.255 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.256 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.956982+0000", 2024-05-08T08:52:33.256 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.256 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.256 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.256 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.256 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.956982+0000", 2024-05-08T08:52:33.256 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.256 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.256 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.256 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.256 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.956983+0000", 2024-05-08T08:52:33.256 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3799999999999997e-07 2024-05-08T08:52:33.256 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.256 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.256 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.257 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.956985+0000", 2024-05-08T08:52:33.257 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3879999999999998e-06 2024-05-08T08:52:33.257 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.257 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.257 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.257 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.956991+0000", 2024-05-08T08:52:33.257 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3799999999999999e-06 2024-05-08T08:52:33.257 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.257 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.257 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.257 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.956998+0000", 2024-05-08T08:52:33.257 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3720000000000001e-06 2024-05-08T08:52:33.257 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.257 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.257 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.257 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.258 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.258 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15857 3.16 3.83728616 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.258 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.961171+0000", 2024-05-08T08:52:33.258 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.020790942, 2024-05-08T08:52:33.258 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.024546385, 2024-05-08T08:52:33.258 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.258 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.258 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:33.258 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.258 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.258 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.258 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15857 2024-05-08T08:52:33.258 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.258 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.259 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.259 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.259 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.961171+0000", 2024-05-08T08:52:33.259 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.259 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.259 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.259 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.259 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.961171+0000", 2024-05-08T08:52:33.259 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.259 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.259 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.259 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.259 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.961172+0000", 2024-05-08T08:52:33.259 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.96e-07 2024-05-08T08:52:33.259 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.260 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.260 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.260 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.961174+0000", 2024-05-08T08:52:33.260 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8729999999999999e-06 2024-05-08T08:52:33.260 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.260 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.260 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.260 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.961179+0000", 2024-05-08T08:52:33.260 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6670000000000001e-06 2024-05-08T08:52:33.260 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.260 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.260 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.260 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.961184+0000", 2024-05-08T08:52:33.260 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6609999999999999e-06 2024-05-08T08:52:33.260 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.260 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.261 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.261 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.261 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.261 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15730 3.16 3.5b6b6bd6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.261 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.949151+0000", 2024-05-08T08:52:33.261 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.032811224999999999, 2024-05-08T08:52:33.261 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036593594, 2024-05-08T08:52:33.261 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.261 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.261 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:33.261 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.261 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.261 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.261 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15730 2024-05-08T08:52:33.261 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.261 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.261 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.262 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.262 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.949151+0000", 2024-05-08T08:52:33.262 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.262 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.262 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.262 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.262 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.949151+0000", 2024-05-08T08:52:33.262 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.262 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.262 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.262 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.262 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.950938+0000", 2024-05-08T08:52:33.262 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0017871600000000001 2024-05-08T08:52:33.262 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.263 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.263 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.263 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.950943+0000", 2024-05-08T08:52:33.263 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5349999999999998e-06 2024-05-08T08:52:33.263 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.263 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.265 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.265 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.950959+0000", 2024-05-08T08:52:33.266 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6490999999999998e-05 2024-05-08T08:52:33.266 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.266 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.266 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.266 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.950968+0000", 2024-05-08T08:52:33.266 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2700000000000004e-06 2024-05-08T08:52:33.266 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.266 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.266 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.266 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.266 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.266 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15794 3.16 3.83728616 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.266 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.957005+0000", 2024-05-08T08:52:33.267 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.024957638000000001, 2024-05-08T08:52:33.267 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028768307999999999, 2024-05-08T08:52:33.267 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.267 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.267 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:33.267 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.267 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.267 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.267 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15794 2024-05-08T08:52:33.267 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.267 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.267 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.267 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.267 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.957005+0000", 2024-05-08T08:52:33.267 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.268 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.268 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.268 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.268 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.957005+0000", 2024-05-08T08:52:33.268 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.268 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.268 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.268 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.268 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.957005+0000", 2024-05-08T08:52:33.268 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0200000000000003e-07 2024-05-08T08:52:33.268 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.268 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.268 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.268 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.957008+0000", 2024-05-08T08:52:33.268 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5399999999999998e-06 2024-05-08T08:52:33.269 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.269 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.269 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.269 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.957014+0000", 2024-05-08T08:52:33.269 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1859999999999997e-06 2024-05-08T08:52:33.269 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.269 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.269 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.269 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.957019+0000", 2024-05-08T08:52:33.269 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0799999999999996e-06 2024-05-08T08:52:33.269 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.269 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.269 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.269 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.269 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.270 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15859 3.16 3.83728616 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.270 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.961191+0000", 2024-05-08T08:52:33.270 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.020771030999999999, 2024-05-08T08:52:33.270 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.024612701000000001, 2024-05-08T08:52:33.270 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.270 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.270 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:33.270 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.270 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.270 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.270 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15859 2024-05-08T08:52:33.270 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.270 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.271 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.271 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.271 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.961191+0000", 2024-05-08T08:52:33.271 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.271 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.271 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.271 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.271 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.961191+0000", 2024-05-08T08:52:33.271 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.271 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.271 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.271 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.271 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.961192+0000", 2024-05-08T08:52:33.271 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8800000000000003e-07 2024-05-08T08:52:33.271 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.272 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.272 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.272 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.961194+0000", 2024-05-08T08:52:33.272 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0370000000000001e-06 2024-05-08T08:52:33.272 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.272 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.272 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.272 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.961200+0000", 2024-05-08T08:52:33.272 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.908e-06 2024-05-08T08:52:33.272 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.272 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.272 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.272 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.961204+0000", 2024-05-08T08:52:33.272 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6070000000000001e-06 2024-05-08T08:52:33.273 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.273 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.273 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.273 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.273 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.273 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15733 3.16 3.5b6b6bd6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.273 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.950981+0000", 2024-05-08T08:52:33.273 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.030980969000000001, 2024-05-08T08:52:33.273 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.034874346000000001, 2024-05-08T08:52:33.273 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.273 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.273 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:33.273 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.273 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.273 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.274 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15733 2024-05-08T08:52:33.274 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.274 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.274 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.274 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.274 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.950981+0000", 2024-05-08T08:52:33.274 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.274 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.274 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.274 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.274 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.950981+0000", 2024-05-08T08:52:33.274 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.274 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.274 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.275 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.275 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.950982+0000", 2024-05-08T08:52:33.275 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0100000000000005e-07 2024-05-08T08:52:33.275 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.275 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.275 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.275 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.950985+0000", 2024-05-08T08:52:33.275 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6819999999999999e-06 2024-05-08T08:52:33.275 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.275 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.275 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.275 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.950992+0000", 2024-05-08T08:52:33.275 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8159999999999996e-06 2024-05-08T08:52:33.275 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.275 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.276 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.276 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.950997+0000", 2024-05-08T08:52:33.276 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7440000000000004e-06 2024-05-08T08:52:33.276 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.276 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.276 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.276 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.276 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.276 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15796 3.16 3.83728616 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.276 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.957026+0000", 2024-05-08T08:52:33.276 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.024935967, 2024-05-08T08:52:33.276 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028885425999999999, 2024-05-08T08:52:33.276 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.276 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.276 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:33.277 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.277 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.277 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.277 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15796 2024-05-08T08:52:33.277 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.277 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.277 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.277 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.277 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.957026+0000", 2024-05-08T08:52:33.277 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.277 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.277 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.277 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.277 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.957026+0000", 2024-05-08T08:52:33.277 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.277 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.278 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.278 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.278 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.957027+0000", 2024-05-08T08:52:33.278 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8599999999999999e-07 2024-05-08T08:52:33.278 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.278 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.278 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.278 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.957029+0000", 2024-05-08T08:52:33.278 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3140000000000002e-06 2024-05-08T08:52:33.278 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.278 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.278 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.278 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.957035+0000", 2024-05-08T08:52:33.278 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8980000000000001e-06 2024-05-08T08:52:33.279 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.279 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.279 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.279 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.957040+0000", 2024-05-08T08:52:33.279 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6319999999999999e-06 2024-05-08T08:52:33.279 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.279 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.279 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.279 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.279 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.279 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15861 3.16 3.83728616 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.279 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.961210+0000", 2024-05-08T08:52:33.279 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.020752118, 2024-05-08T08:52:33.279 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.024763325999999999, 2024-05-08T08:52:33.279 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.279 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.280 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:33.280 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.280 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.280 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.280 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15861 2024-05-08T08:52:33.280 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.280 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.280 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.280 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.280 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.961210+0000", 2024-05-08T08:52:33.280 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.280 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.280 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.280 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.280 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.961210+0000", 2024-05-08T08:52:33.281 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.281 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.281 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.281 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.281 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.962937+0000", 2024-05-08T08:52:33.281 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001726518 2024-05-08T08:52:33.281 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.281 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.281 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.281 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.962941+0000", 2024-05-08T08:52:33.281 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5340000000000001e-06 2024-05-08T08:52:33.281 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.281 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.281 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.281 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.962951+0000", 2024-05-08T08:52:33.282 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9019999999999994e-06 2024-05-08T08:52:33.282 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.282 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.282 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.282 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.962958+0000", 2024-05-08T08:52:33.282 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9750000000000001e-06 2024-05-08T08:52:33.282 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.282 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.282 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.282 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.282 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.282 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15735 3.16 3.5b6b6bd6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.282 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.951005+0000", 2024-05-08T08:52:33.282 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.030957366, 2024-05-08T08:52:33.283 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035019939, 2024-05-08T08:52:33.283 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.283 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.283 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:33.283 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.283 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.283 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.283 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15735 2024-05-08T08:52:33.283 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.283 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.283 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.283 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.283 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.951005+0000", 2024-05-08T08:52:33.283 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.284 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.284 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.284 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.284 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.951005+0000", 2024-05-08T08:52:33.284 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.284 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.284 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.284 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.284 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.951005+0000", 2024-05-08T08:52:33.284 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.08e-07 2024-05-08T08:52:33.284 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.284 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.284 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.284 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.951008+0000", 2024-05-08T08:52:33.284 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4159999999999998e-06 2024-05-08T08:52:33.285 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.285 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.285 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.285 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.951014+0000", 2024-05-08T08:52:33.285 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2750000000000001e-06 2024-05-08T08:52:33.285 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.285 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.285 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.285 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.951019+0000", 2024-05-08T08:52:33.285 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1590000000000001e-06 2024-05-08T08:52:33.285 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.285 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.285 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.285 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.285 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.286 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15798 3.16 3.83728616 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.286 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.957046+0000", 2024-05-08T08:52:33.286 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.024916443, 2024-05-08T08:52:33.286 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029014630999999999, 2024-05-08T08:52:33.286 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.286 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.286 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:33.286 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.286 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.286 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.286 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15798 2024-05-08T08:52:33.286 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.286 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.286 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.287 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.287 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.957046+0000", 2024-05-08T08:52:33.287 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.287 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.287 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.287 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.287 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.957046+0000", 2024-05-08T08:52:33.287 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.287 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.287 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.287 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.287 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.957046+0000", 2024-05-08T08:52:33.287 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0400000000000002e-07 2024-05-08T08:52:33.287 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.287 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.288 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.288 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.957053+0000", 2024-05-08T08:52:33.288 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6889999999999998e-06 2024-05-08T08:52:33.288 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.288 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.288 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.288 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.957059+0000", 2024-05-08T08:52:33.288 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2790000000000004e-06 2024-05-08T08:52:33.288 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.288 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.288 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.288 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.957064+0000", 2024-05-08T08:52:33.288 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0030000000000002e-06 2024-05-08T08:52:33.288 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.289 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.289 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.289 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.289 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.289 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:15863 3.16 3.83728616 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:33.289 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.962970+0000", 2024-05-08T08:52:33.289 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.018992294999999999, 2024-05-08T08:52:33.289 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.023117919000000001, 2024-05-08T08:52:33.289 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:33.289 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:33.289 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:33.289 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:33.289 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:33.289 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:33.289 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 15863 2024-05-08T08:52:33.289 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.290 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:33.290 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.290 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:33.290 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.962970+0000", 2024-05-08T08:52:33.290 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.290 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.290 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.290 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:33.290 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.962970+0000", 2024-05-08T08:52:33.290 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:33.290 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.290 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.290 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:33.290 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.962971+0000", 2024-05-08T08:52:33.291 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4899999999999995e-07 2024-05-08T08:52:33.291 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.291 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.291 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:33.291 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.962973+0000", 2024-05-08T08:52:33.291 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6259999999999999e-06 2024-05-08T08:52:33.291 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.291 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.291 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:33.291 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.962980+0000", 2024-05-08T08:52:33.291 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1300000000000003e-06 2024-05-08T08:52:33.291 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:33.291 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:33.291 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:33.291 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.962985+0000", 2024-05-08T08:52:33.292 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0889999999999999e-06 2024-05-08T08:52:33.292 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.292 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:33.292 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.292 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:33.292 INFO:teuthology.orchestra.run.smithi192.stdout: ], 2024-05-08T08:52:33.292 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 90 2024-05-08T08:52:33.292 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:52:33.293 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:52:33.293 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:52:33.015+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:52:33.294 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:33.180+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:33.294 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:52:33.295 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:52:33.295 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:52:33.296 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:52:33.296 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:52:33.297+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:52:33.396 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:52:33.399+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:52:33.426 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:33.426 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:33.435 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:52:33.497 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:52:33.499+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:52:33.598 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:33.601+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:33.614 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:33.614 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:33.629 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:52:33.699 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:52:33.701+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:52:33.707 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:52:33.707 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:52:33.707 INFO:teuthology.orchestra.run.smithi059.stdout: "epoch": 33, 2024-05-08T08:52:33.707 INFO:teuthology.orchestra.run.smithi059.stdout: "fsid": "38dfae8c-5c5e-4b0f-a264-91f22c6c5554", 2024-05-08T08:52:33.707 INFO:teuthology.orchestra.run.smithi059.stdout: "created": "2024-05-08T08:51:45.571369+0000", 2024-05-08T08:52:33.707 INFO:teuthology.orchestra.run.smithi059.stdout: "modified": "2024-05-08T08:52:31.999420+0000", 2024-05-08T08:52:33.707 INFO:teuthology.orchestra.run.smithi059.stdout: "last_up_change": "2024-05-08T08:51:56.847706+0000", 2024-05-08T08:52:33.708 INFO:teuthology.orchestra.run.smithi059.stdout: "last_in_change": "2024-05-08T08:51:47.267430+0000", 2024-05-08T08:52:33.708 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-05-08T08:52:33.708 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_num": 5799936, 2024-05-08T08:52:33.708 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_set": [ 2024-05-08T08:52:33.708 INFO:teuthology.orchestra.run.smithi059.stdout: "pglog_hardlimit", 2024-05-08T08:52:33.708 INFO:teuthology.orchestra.run.smithi059.stdout: "purged_snapdirs", 2024-05-08T08:52:33.708 INFO:teuthology.orchestra.run.smithi059.stdout: "recovery_deletes", 2024-05-08T08:52:33.708 INFO:teuthology.orchestra.run.smithi059.stdout: "sortbitwise" 2024-05-08T08:52:33.708 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T08:52:33.708 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_version": 4, 2024-05-08T08:52:33.708 INFO:teuthology.orchestra.run.smithi059.stdout: "full_ratio": 0.94999998807907104, 2024-05-08T08:52:33.708 INFO:teuthology.orchestra.run.smithi059.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-05-08T08:52:33.708 INFO:teuthology.orchestra.run.smithi059.stdout: "nearfull_ratio": 0.85000002384185791, 2024-05-08T08:52:33.708 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_snapshot": "", 2024-05-08T08:52:33.708 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_max": 3, 2024-05-08T08:52:33.709 INFO:teuthology.orchestra.run.smithi059.stdout: "max_osd": 6, 2024-05-08T08:52:33.709 INFO:teuthology.orchestra.run.smithi059.stdout: "require_min_compat_client": "luminous", 2024-05-08T08:52:33.709 INFO:teuthology.orchestra.run.smithi059.stdout: "min_compat_client": "jewel", 2024-05-08T08:52:33.709 INFO:teuthology.orchestra.run.smithi059.stdout: "require_osd_release": "squid", 2024-05-08T08:52:33.709 INFO:teuthology.orchestra.run.smithi059.stdout: "allow_crimson": false, 2024-05-08T08:52:33.709 INFO:teuthology.orchestra.run.smithi059.stdout: "pools": [ 2024-05-08T08:52:33.709 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:33.709 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 1, 2024-05-08T08:52:33.709 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_name": ".mgr", 2024-05-08T08:52:33.709 INFO:teuthology.orchestra.run.smithi059.stdout: "create_time": "2024-05-08T08:51:57.787790+0000", 2024-05-08T08:52:33.709 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T08:52:33.709 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_names": "hashpspool", 2024-05-08T08:52:33.709 INFO:teuthology.orchestra.run.smithi059.stdout: "type": 1, 2024-05-08T08:52:33.709 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 2, 2024-05-08T08:52:33.709 INFO:teuthology.orchestra.run.smithi059.stdout: "min_size": 1, 2024-05-08T08:52:33.709 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_rule": 0, 2024-05-08T08:52:33.710 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_count": 0, 2024-05-08T08:52:33.710 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_target": 0, 2024-05-08T08:52:33.710 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_barrier": 0, 2024-05-08T08:52:33.710 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-08T08:52:33.710 INFO:teuthology.orchestra.run.smithi059.stdout: "object_hash": 2, 2024-05-08T08:52:33.710 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscale_mode": "off", 2024-05-08T08:52:33.710 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num": 1, 2024-05-08T08:52:33.710 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num": 1, 2024-05-08T08:52:33.710 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num_target": 1, 2024-05-08T08:52:33.710 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_target": 1, 2024-05-08T08:52:33.710 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_pending": 1, 2024-05-08T08:52:33.710 INFO:teuthology.orchestra.run.smithi059.stdout: "last_pg_merge_meta": { 2024-05-08T08:52:33.710 INFO:teuthology.orchestra.run.smithi059.stdout: "source_pgid": "0.0", 2024-05-08T08:52:33.710 INFO:teuthology.orchestra.run.smithi059.stdout: "ready_epoch": 0, 2024-05-08T08:52:33.711 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_started": 0, 2024-05-08T08:52:33.711 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_clean": 0, 2024-05-08T08:52:33.711 INFO:teuthology.orchestra.run.smithi059.stdout: "source_version": "0'0", 2024-05-08T08:52:33.711 INFO:teuthology.orchestra.run.smithi059.stdout: "target_version": "0'0" 2024-05-08T08:52:33.711 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:33.711 INFO:teuthology.orchestra.run.smithi059.stdout: "last_change": "14", 2024-05-08T08:52:33.711 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend": "0", 2024-05-08T08:52:33.711 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-08T08:52:33.711 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_preluminous": "0", 2024-05-08T08:52:33.711 INFO:teuthology.orchestra.run.smithi059.stdout: "auid": 0, 2024-05-08T08:52:33.711 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_mode": "selfmanaged", 2024-05-08T08:52:33.711 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_seq": 0, 2024-05-08T08:52:33.711 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_epoch": 0, 2024-05-08T08:52:33.711 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_snaps": [], 2024-05-08T08:52:33.711 INFO:teuthology.orchestra.run.smithi059.stdout: "removed_snaps": "[]", 2024-05-08T08:52:33.711 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_bytes": 0, 2024-05-08T08:52:33.712 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_objects": 0, 2024-05-08T08:52:33.712 INFO:teuthology.orchestra.run.smithi059.stdout: "tiers": [], 2024-05-08T08:52:33.712 INFO:teuthology.orchestra.run.smithi059.stdout: "tier_of": -1, 2024-05-08T08:52:33.712 INFO:teuthology.orchestra.run.smithi059.stdout: "read_tier": -1, 2024-05-08T08:52:33.712 INFO:teuthology.orchestra.run.smithi059.stdout: "write_tier": -1, 2024-05-08T08:52:33.712 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_mode": "none", 2024-05-08T08:52:33.712 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_bytes": 0, 2024-05-08T08:52:33.712 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_objects": 0, 2024-05-08T08:52:33.712 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-08T08:52:33.712 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-08T08:52:33.712 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-08T08:52:33.712 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_flush_age": 0, 2024-05-08T08:52:33.712 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_evict_age": 0, 2024-05-08T08:52:33.712 INFO:teuthology.orchestra.run.smithi059.stdout: "erasure_code_profile": "", 2024-05-08T08:52:33.712 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_params": { 2024-05-08T08:52:33.712 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "none" 2024-05-08T08:52:33.713 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:33.713 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_period": 0, 2024-05-08T08:52:33.713 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_count": 0, 2024-05-08T08:52:33.713 INFO:teuthology.orchestra.run.smithi059.stdout: "use_gmt_hitset": true, 2024-05-08T08:52:33.713 INFO:teuthology.orchestra.run.smithi059.stdout: "min_read_recency_for_promote": 0, 2024-05-08T08:52:33.713 INFO:teuthology.orchestra.run.smithi059.stdout: "min_write_recency_for_promote": 0, 2024-05-08T08:52:33.713 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_grade_decay_rate": 0, 2024-05-08T08:52:33.713 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_search_last_n": 0, 2024-05-08T08:52:33.713 INFO:teuthology.orchestra.run.smithi059.stdout: "grade_table": [], 2024-05-08T08:52:33.713 INFO:teuthology.orchestra.run.smithi059.stdout: "stripe_width": 0, 2024-05-08T08:52:33.713 INFO:teuthology.orchestra.run.smithi059.stdout: "expected_num_objects": 0, 2024-05-08T08:52:33.713 INFO:teuthology.orchestra.run.smithi059.stdout: "fast_read": false, 2024-05-08T08:52:33.713 INFO:teuthology.orchestra.run.smithi059.stdout: "options": { 2024-05-08T08:52:33.713 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_max": 32, 2024-05-08T08:52:33.713 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_min": 1 2024-05-08T08:52:33.713 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:33.714 INFO:teuthology.orchestra.run.smithi059.stdout: "application_metadata": { 2024-05-08T08:52:33.714 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": {} 2024-05-08T08:52:33.714 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:33.714 INFO:teuthology.orchestra.run.smithi059.stdout: "read_balance": { 2024-05-08T08:52:33.714 INFO:teuthology.orchestra.run.smithi059.stdout: "score_type": "Fair distribution", 2024-05-08T08:52:33.714 INFO:teuthology.orchestra.run.smithi059.stdout: "score_acting": 6.059999942779541, 2024-05-08T08:52:33.714 INFO:teuthology.orchestra.run.smithi059.stdout: "score_stable": 6.059999942779541, 2024-05-08T08:52:33.714 INFO:teuthology.orchestra.run.smithi059.stdout: "optimal_score": 0.33000001311302185, 2024-05-08T08:52:33.714 INFO:teuthology.orchestra.run.smithi059.stdout: "raw_score_acting": 2, 2024-05-08T08:52:33.714 INFO:teuthology.orchestra.run.smithi059.stdout: "raw_score_stable": 2, 2024-05-08T08:52:33.714 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity_weighted": 0.87819164991378784, 2024-05-08T08:52:33.714 INFO:teuthology.orchestra.run.smithi059.stdout: "average_primary_affinity": 0.87999999523162842, 2024-05-08T08:52:33.714 INFO:teuthology.orchestra.run.smithi059.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-05-08T08:52:33.714 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:33.715 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:33.715 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:33.715 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 2, 2024-05-08T08:52:33.715 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_name": "rbd", 2024-05-08T08:52:33.715 INFO:teuthology.orchestra.run.smithi059.stdout: "create_time": "2024-05-08T08:52:02.563677+0000", 2024-05-08T08:52:33.715 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T08:52:33.715 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_names": "hashpspool", 2024-05-08T08:52:33.715 INFO:teuthology.orchestra.run.smithi059.stdout: "type": 1, 2024-05-08T08:52:33.715 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 2, 2024-05-08T08:52:33.715 INFO:teuthology.orchestra.run.smithi059.stdout: "min_size": 1, 2024-05-08T08:52:33.715 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_rule": 0, 2024-05-08T08:52:33.716 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_count": 0, 2024-05-08T08:52:33.716 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_target": 0, 2024-05-08T08:52:33.716 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_barrier": 0, 2024-05-08T08:52:33.716 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-08T08:52:33.716 INFO:teuthology.orchestra.run.smithi059.stdout: "object_hash": 2, 2024-05-08T08:52:33.716 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscale_mode": "off", 2024-05-08T08:52:33.716 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num": 8, 2024-05-08T08:52:33.716 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num": 8, 2024-05-08T08:52:33.716 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num_target": 8, 2024-05-08T08:52:33.716 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_target": 8, 2024-05-08T08:52:33.716 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_pending": 8, 2024-05-08T08:52:33.716 INFO:teuthology.orchestra.run.smithi059.stdout: "last_pg_merge_meta": { 2024-05-08T08:52:33.716 INFO:teuthology.orchestra.run.smithi059.stdout: "source_pgid": "0.0", 2024-05-08T08:52:33.716 INFO:teuthology.orchestra.run.smithi059.stdout: "ready_epoch": 0, 2024-05-08T08:52:33.716 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_started": 0, 2024-05-08T08:52:33.717 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_clean": 0, 2024-05-08T08:52:33.717 INFO:teuthology.orchestra.run.smithi059.stdout: "source_version": "0'0", 2024-05-08T08:52:33.717 INFO:teuthology.orchestra.run.smithi059.stdout: "target_version": "0'0" 2024-05-08T08:52:33.717 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:33.717 INFO:teuthology.orchestra.run.smithi059.stdout: "last_change": "17", 2024-05-08T08:52:33.717 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend": "0", 2024-05-08T08:52:33.717 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-08T08:52:33.717 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_preluminous": "0", 2024-05-08T08:52:33.717 INFO:teuthology.orchestra.run.smithi059.stdout: "auid": 0, 2024-05-08T08:52:33.717 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_mode": "selfmanaged", 2024-05-08T08:52:33.717 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_seq": 0, 2024-05-08T08:52:33.717 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_epoch": 0, 2024-05-08T08:52:33.717 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_snaps": [], 2024-05-08T08:52:33.717 INFO:teuthology.orchestra.run.smithi059.stdout: "removed_snaps": "[]", 2024-05-08T08:52:33.717 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_bytes": 0, 2024-05-08T08:52:33.717 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_objects": 0, 2024-05-08T08:52:33.717 INFO:teuthology.orchestra.run.smithi059.stdout: "tiers": [], 2024-05-08T08:52:33.718 INFO:teuthology.orchestra.run.smithi059.stdout: "tier_of": -1, 2024-05-08T08:52:33.718 INFO:teuthology.orchestra.run.smithi059.stdout: "read_tier": -1, 2024-05-08T08:52:33.718 INFO:teuthology.orchestra.run.smithi059.stdout: "write_tier": -1, 2024-05-08T08:52:33.718 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_mode": "none", 2024-05-08T08:52:33.718 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_bytes": 0, 2024-05-08T08:52:33.718 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_objects": 0, 2024-05-08T08:52:33.718 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-08T08:52:33.718 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-08T08:52:33.718 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-08T08:52:33.718 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_flush_age": 0, 2024-05-08T08:52:33.718 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_evict_age": 0, 2024-05-08T08:52:33.718 INFO:teuthology.orchestra.run.smithi059.stdout: "erasure_code_profile": "", 2024-05-08T08:52:33.718 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_params": { 2024-05-08T08:52:33.718 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "none" 2024-05-08T08:52:33.719 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:33.719 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_period": 0, 2024-05-08T08:52:33.719 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_count": 0, 2024-05-08T08:52:33.719 INFO:teuthology.orchestra.run.smithi059.stdout: "use_gmt_hitset": true, 2024-05-08T08:52:33.719 INFO:teuthology.orchestra.run.smithi059.stdout: "min_read_recency_for_promote": 0, 2024-05-08T08:52:33.719 INFO:teuthology.orchestra.run.smithi059.stdout: "min_write_recency_for_promote": 0, 2024-05-08T08:52:33.719 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_grade_decay_rate": 0, 2024-05-08T08:52:33.719 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_search_last_n": 0, 2024-05-08T08:52:33.719 INFO:teuthology.orchestra.run.smithi059.stdout: "grade_table": [], 2024-05-08T08:52:33.719 INFO:teuthology.orchestra.run.smithi059.stdout: "stripe_width": 0, 2024-05-08T08:52:33.719 INFO:teuthology.orchestra.run.smithi059.stdout: "expected_num_objects": 0, 2024-05-08T08:52:33.719 INFO:teuthology.orchestra.run.smithi059.stdout: "fast_read": false, 2024-05-08T08:52:33.719 INFO:teuthology.orchestra.run.smithi059.stdout: "options": {}, 2024-05-08T08:52:33.719 INFO:teuthology.orchestra.run.smithi059.stdout: "application_metadata": { 2024-05-08T08:52:33.719 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd": {} 2024-05-08T08:52:33.719 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:33.719 INFO:teuthology.orchestra.run.smithi059.stdout: "read_balance": { 2024-05-08T08:52:33.720 INFO:teuthology.orchestra.run.smithi059.stdout: "score_type": "Fair distribution", 2024-05-08T08:52:33.720 INFO:teuthology.orchestra.run.smithi059.stdout: "score_acting": 2.25, 2024-05-08T08:52:33.720 INFO:teuthology.orchestra.run.smithi059.stdout: "score_stable": 2.25, 2024-05-08T08:52:33.720 INFO:teuthology.orchestra.run.smithi059.stdout: "optimal_score": 1, 2024-05-08T08:52:33.720 INFO:teuthology.orchestra.run.smithi059.stdout: "raw_score_acting": 2.25, 2024-05-08T08:52:33.720 INFO:teuthology.orchestra.run.smithi059.stdout: "raw_score_stable": 2.25, 2024-05-08T08:52:33.720 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity_weighted": 0.87819164991378784, 2024-05-08T08:52:33.720 INFO:teuthology.orchestra.run.smithi059.stdout: "average_primary_affinity": 0.87999999523162842, 2024-05-08T08:52:33.720 INFO:teuthology.orchestra.run.smithi059.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-05-08T08:52:33.720 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:33.720 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:33.720 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:33.720 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 3, 2024-05-08T08:52:33.720 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_name": "pool_client.0", 2024-05-08T08:52:33.720 INFO:teuthology.orchestra.run.smithi059.stdout: "create_time": "2024-05-08T08:52:09.955467+0000", 2024-05-08T08:52:33.721 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 8193, 2024-05-08T08:52:33.721 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-05-08T08:52:33.721 INFO:teuthology.orchestra.run.smithi059.stdout: "type": 1, 2024-05-08T08:52:33.721 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 2, 2024-05-08T08:52:33.721 INFO:teuthology.orchestra.run.smithi059.stdout: "min_size": 1, 2024-05-08T08:52:33.721 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_rule": 0, 2024-05-08T08:52:33.721 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_count": 0, 2024-05-08T08:52:33.721 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_target": 0, 2024-05-08T08:52:33.721 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_barrier": 0, 2024-05-08T08:52:33.721 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-08T08:52:33.721 INFO:teuthology.orchestra.run.smithi059.stdout: "object_hash": 2, 2024-05-08T08:52:33.721 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscale_mode": "off", 2024-05-08T08:52:33.721 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num": 32, 2024-05-08T08:52:33.721 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num": 32, 2024-05-08T08:52:33.721 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num_target": 32, 2024-05-08T08:52:33.721 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_target": 32, 2024-05-08T08:52:33.722 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_pending": 32, 2024-05-08T08:52:33.722 INFO:teuthology.orchestra.run.smithi059.stdout: "last_pg_merge_meta": { 2024-05-08T08:52:33.722 INFO:teuthology.orchestra.run.smithi059.stdout: "source_pgid": "0.0", 2024-05-08T08:52:33.722 INFO:teuthology.orchestra.run.smithi059.stdout: "ready_epoch": 0, 2024-05-08T08:52:33.722 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_started": 0, 2024-05-08T08:52:33.722 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_clean": 0, 2024-05-08T08:52:33.722 INFO:teuthology.orchestra.run.smithi059.stdout: "source_version": "0'0", 2024-05-08T08:52:33.722 INFO:teuthology.orchestra.run.smithi059.stdout: "target_version": "0'0" 2024-05-08T08:52:33.722 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:33.722 INFO:teuthology.orchestra.run.smithi059.stdout: "last_change": "31", 2024-05-08T08:52:33.723 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend": "0", 2024-05-08T08:52:33.723 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-08T08:52:33.723 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_preluminous": "0", 2024-05-08T08:52:33.723 INFO:teuthology.orchestra.run.smithi059.stdout: "auid": 0, 2024-05-08T08:52:33.723 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_mode": "selfmanaged", 2024-05-08T08:52:33.723 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_seq": 7, 2024-05-08T08:52:33.723 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_epoch": 31, 2024-05-08T08:52:33.723 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_snaps": [], 2024-05-08T08:52:33.723 INFO:teuthology.orchestra.run.smithi059.stdout: "removed_snaps": "[]", 2024-05-08T08:52:33.723 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_bytes": 0, 2024-05-08T08:52:33.723 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_objects": 0, 2024-05-08T08:52:33.723 INFO:teuthology.orchestra.run.smithi059.stdout: "tiers": [], 2024-05-08T08:52:33.723 INFO:teuthology.orchestra.run.smithi059.stdout: "tier_of": -1, 2024-05-08T08:52:33.723 INFO:teuthology.orchestra.run.smithi059.stdout: "read_tier": -1, 2024-05-08T08:52:33.723 INFO:teuthology.orchestra.run.smithi059.stdout: "write_tier": -1, 2024-05-08T08:52:33.724 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_mode": "none", 2024-05-08T08:52:33.724 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_bytes": 0, 2024-05-08T08:52:33.724 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_objects": 0, 2024-05-08T08:52:33.724 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-08T08:52:33.724 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-08T08:52:33.724 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-08T08:52:33.724 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_flush_age": 0, 2024-05-08T08:52:33.724 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_evict_age": 0, 2024-05-08T08:52:33.724 INFO:teuthology.orchestra.run.smithi059.stdout: "erasure_code_profile": "", 2024-05-08T08:52:33.724 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_params": { 2024-05-08T08:52:33.724 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "none" 2024-05-08T08:52:33.724 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:33.724 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_period": 0, 2024-05-08T08:52:33.724 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_count": 0, 2024-05-08T08:52:33.724 INFO:teuthology.orchestra.run.smithi059.stdout: "use_gmt_hitset": true, 2024-05-08T08:52:33.724 INFO:teuthology.orchestra.run.smithi059.stdout: "min_read_recency_for_promote": 0, 2024-05-08T08:52:33.724 INFO:teuthology.orchestra.run.smithi059.stdout: "min_write_recency_for_promote": 0, 2024-05-08T08:52:33.725 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_grade_decay_rate": 0, 2024-05-08T08:52:33.725 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_search_last_n": 0, 2024-05-08T08:52:33.725 INFO:teuthology.orchestra.run.smithi059.stdout: "grade_table": [], 2024-05-08T08:52:33.725 INFO:teuthology.orchestra.run.smithi059.stdout: "stripe_width": 0, 2024-05-08T08:52:33.725 INFO:teuthology.orchestra.run.smithi059.stdout: "expected_num_objects": 0, 2024-05-08T08:52:33.725 INFO:teuthology.orchestra.run.smithi059.stdout: "fast_read": false, 2024-05-08T08:52:33.725 INFO:teuthology.orchestra.run.smithi059.stdout: "options": {}, 2024-05-08T08:52:33.725 INFO:teuthology.orchestra.run.smithi059.stdout: "application_metadata": { 2024-05-08T08:52:33.725 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd": {} 2024-05-08T08:52:33.725 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:33.725 INFO:teuthology.orchestra.run.smithi059.stdout: "read_balance": { 2024-05-08T08:52:33.725 INFO:teuthology.orchestra.run.smithi059.stdout: "score_type": "Fair distribution", 2024-05-08T08:52:33.725 INFO:teuthology.orchestra.run.smithi059.stdout: "score_acting": 1.5, 2024-05-08T08:52:33.725 INFO:teuthology.orchestra.run.smithi059.stdout: "score_stable": 1.5, 2024-05-08T08:52:33.725 INFO:teuthology.orchestra.run.smithi059.stdout: "optimal_score": 1, 2024-05-08T08:52:33.726 INFO:teuthology.orchestra.run.smithi059.stdout: "raw_score_acting": 1.5, 2024-05-08T08:52:33.726 INFO:teuthology.orchestra.run.smithi059.stdout: "raw_score_stable": 1.5, 2024-05-08T08:52:33.726 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity_weighted": 0.87819164991378784, 2024-05-08T08:52:33.726 INFO:teuthology.orchestra.run.smithi059.stdout: "average_primary_affinity": 0.87999999523162842, 2024-05-08T08:52:33.726 INFO:teuthology.orchestra.run.smithi059.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-05-08T08:52:33.726 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:33.726 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:33.726 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T08:52:33.726 INFO:teuthology.orchestra.run.smithi059.stdout: "osds": [ 2024-05-08T08:52:33.726 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:33.726 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 0, 2024-05-08T08:52:33.726 INFO:teuthology.orchestra.run.smithi059.stdout: "uuid": "300801a5-1919-4b23-8c6d-bc7dad61cc4d", 2024-05-08T08:52:33.726 INFO:teuthology.orchestra.run.smithi059.stdout: "up": 1, 2024-05-08T08:52:33.726 INFO:teuthology.orchestra.run.smithi059.stdout: "in": 1, 2024-05-08T08:52:33.727 INFO:teuthology.orchestra.run.smithi059.stdout: "weight": 1, 2024-05-08T08:52:33.727 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity": 1, 2024-05-08T08:52:33.727 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_begin": 0, 2024-05-08T08:52:33.727 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_end": 0, 2024-05-08T08:52:33.727 INFO:teuthology.orchestra.run.smithi059.stdout: "up_from": 11, 2024-05-08T08:52:33.727 INFO:teuthology.orchestra.run.smithi059.stdout: "up_thru": 28, 2024-05-08T08:52:33.727 INFO:teuthology.orchestra.run.smithi059.stdout: "down_at": 0, 2024-05-08T08:52:33.727 INFO:teuthology.orchestra.run.smithi059.stdout: "lost_at": 0, 2024-05-08T08:52:33.727 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrs": { 2024-05-08T08:52:33.727 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:52:33.727 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:33.727 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:52:33.727 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6800", 2024-05-08T08:52:33.727 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 802160151 2024-05-08T08:52:33.727 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:33.728 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:33.728 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:52:33.728 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6801", 2024-05-08T08:52:33.728 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 802160151 2024-05-08T08:52:33.728 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:33.728 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:33.728 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:33.728 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addrs": { 2024-05-08T08:52:33.728 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:52:33.728 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:33.728 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:52:33.728 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6802", 2024-05-08T08:52:33.728 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 802160151 2024-05-08T08:52:33.728 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:33.729 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:33.729 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:52:33.729 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6803", 2024-05-08T08:52:33.729 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 802160151 2024-05-08T08:52:33.729 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:33.729 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:33.729 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:33.729 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addrs": { 2024-05-08T08:52:33.729 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:52:33.729 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:33.729 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:52:33.729 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6806", 2024-05-08T08:52:33.729 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 802160151 2024-05-08T08:52:33.729 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:33.729 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:33.730 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:52:33.730 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6807", 2024-05-08T08:52:33.730 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 802160151 2024-05-08T08:52:33.730 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:33.730 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:33.730 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:33.730 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addrs": { 2024-05-08T08:52:33.730 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:52:33.730 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:33.730 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:52:33.730 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6804", 2024-05-08T08:52:33.730 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 802160151 2024-05-08T08:52:33.730 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:33.730 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:33.731 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:52:33.731 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6805", 2024-05-08T08:52:33.731 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 802160151 2024-05-08T08:52:33.731 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:33.731 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:33.731 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:33.731 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "172.21.15.59:6801/802160151", 2024-05-08T08:52:33.731 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "172.21.15.59:6803/802160151", 2024-05-08T08:52:33.731 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addr": "172.21.15.59:6807/802160151", 2024-05-08T08:52:33.731 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addr": "172.21.15.59:6805/802160151", 2024-05-08T08:52:33.731 INFO:teuthology.orchestra.run.smithi059.stdout: "state": [ 2024-05-08T08:52:33.731 INFO:teuthology.orchestra.run.smithi059.stdout: "exists", 2024-05-08T08:52:33.731 INFO:teuthology.orchestra.run.smithi059.stdout: "up" 2024-05-08T08:52:33.731 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:33.731 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:33.732 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:33.732 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 1, 2024-05-08T08:52:33.732 INFO:teuthology.orchestra.run.smithi059.stdout: "uuid": "0fead0de-6a65-4eba-b400-eea5692702e0", 2024-05-08T08:52:33.732 INFO:teuthology.orchestra.run.smithi059.stdout: "up": 1, 2024-05-08T08:52:33.732 INFO:teuthology.orchestra.run.smithi059.stdout: "in": 1, 2024-05-08T08:52:33.732 INFO:teuthology.orchestra.run.smithi059.stdout: "weight": 1, 2024-05-08T08:52:33.732 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity": 1, 2024-05-08T08:52:33.732 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_begin": 0, 2024-05-08T08:52:33.732 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_end": 0, 2024-05-08T08:52:33.732 INFO:teuthology.orchestra.run.smithi059.stdout: "up_from": 11, 2024-05-08T08:52:33.732 INFO:teuthology.orchestra.run.smithi059.stdout: "up_thru": 28, 2024-05-08T08:52:33.732 INFO:teuthology.orchestra.run.smithi059.stdout: "down_at": 0, 2024-05-08T08:52:33.732 INFO:teuthology.orchestra.run.smithi059.stdout: "lost_at": 0, 2024-05-08T08:52:33.732 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrs": { 2024-05-08T08:52:33.732 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:52:33.733 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:33.733 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:52:33.733 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6816", 2024-05-08T08:52:33.733 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4116801011 2024-05-08T08:52:33.733 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:33.733 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:33.733 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:52:33.733 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6817", 2024-05-08T08:52:33.733 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4116801011 2024-05-08T08:52:33.733 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:33.733 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:33.733 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:33.733 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addrs": { 2024-05-08T08:52:33.733 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:52:33.734 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:33.734 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:52:33.734 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6818", 2024-05-08T08:52:33.734 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4116801011 2024-05-08T08:52:33.734 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:33.734 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:33.734 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:52:33.734 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6819", 2024-05-08T08:52:33.734 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4116801011 2024-05-08T08:52:33.734 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:33.734 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:33.734 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:33.734 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addrs": { 2024-05-08T08:52:33.735 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:52:33.735 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:33.735 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:52:33.735 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6822", 2024-05-08T08:52:33.735 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4116801011 2024-05-08T08:52:33.735 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:33.735 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:33.735 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:52:33.735 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6823", 2024-05-08T08:52:33.735 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4116801011 2024-05-08T08:52:33.735 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:33.735 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:33.735 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:33.735 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addrs": { 2024-05-08T08:52:33.735 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:52:33.735 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:33.735 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:52:33.736 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6820", 2024-05-08T08:52:33.736 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4116801011 2024-05-08T08:52:33.736 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:33.736 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:33.736 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:52:33.736 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6821", 2024-05-08T08:52:33.736 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4116801011 2024-05-08T08:52:33.736 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:33.736 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:33.736 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:33.736 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "172.21.15.59:6817/4116801011", 2024-05-08T08:52:33.736 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "172.21.15.59:6819/4116801011", 2024-05-08T08:52:33.736 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addr": "172.21.15.59:6823/4116801011", 2024-05-08T08:52:33.736 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addr": "172.21.15.59:6821/4116801011", 2024-05-08T08:52:33.736 INFO:teuthology.orchestra.run.smithi059.stdout: "state": [ 2024-05-08T08:52:33.736 INFO:teuthology.orchestra.run.smithi059.stdout: "exists", 2024-05-08T08:52:33.737 INFO:teuthology.orchestra.run.smithi059.stdout: "up" 2024-05-08T08:52:33.737 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:33.737 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:33.737 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:33.737 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 2, 2024-05-08T08:52:33.737 INFO:teuthology.orchestra.run.smithi059.stdout: "uuid": "1a233de6-1b82-4a51-8e96-4cd4abdf16c8", 2024-05-08T08:52:33.737 INFO:teuthology.orchestra.run.smithi059.stdout: "up": 1, 2024-05-08T08:52:33.737 INFO:teuthology.orchestra.run.smithi059.stdout: "in": 1, 2024-05-08T08:52:33.737 INFO:teuthology.orchestra.run.smithi059.stdout: "weight": 1, 2024-05-08T08:52:33.737 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity": 1, 2024-05-08T08:52:33.737 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_begin": 0, 2024-05-08T08:52:33.737 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_end": 0, 2024-05-08T08:52:33.737 INFO:teuthology.orchestra.run.smithi059.stdout: "up_from": 11, 2024-05-08T08:52:33.737 INFO:teuthology.orchestra.run.smithi059.stdout: "up_thru": 28, 2024-05-08T08:52:33.737 INFO:teuthology.orchestra.run.smithi059.stdout: "down_at": 0, 2024-05-08T08:52:33.737 INFO:teuthology.orchestra.run.smithi059.stdout: "lost_at": 0, 2024-05-08T08:52:33.738 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrs": { 2024-05-08T08:52:33.738 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:52:33.738 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:33.738 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:52:33.738 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6808", 2024-05-08T08:52:33.738 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1547661179 2024-05-08T08:52:33.738 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:33.738 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:33.738 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:52:33.738 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6809", 2024-05-08T08:52:33.738 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1547661179 2024-05-08T08:52:33.738 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:33.738 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:33.738 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:33.738 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addrs": { 2024-05-08T08:52:33.739 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:52:33.739 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:33.739 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:52:33.739 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6810", 2024-05-08T08:52:33.739 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1547661179 2024-05-08T08:52:33.739 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:33.739 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:33.739 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:52:33.739 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6811", 2024-05-08T08:52:33.739 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1547661179 2024-05-08T08:52:33.739 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:33.739 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:33.739 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:33.739 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addrs": { 2024-05-08T08:52:33.739 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:52:33.739 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:33.740 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:52:33.740 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6814", 2024-05-08T08:52:33.740 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1547661179 2024-05-08T08:52:33.740 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:33.740 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:33.740 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:52:33.740 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6815", 2024-05-08T08:52:33.740 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1547661179 2024-05-08T08:52:33.740 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:33.740 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:33.740 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:33.740 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addrs": { 2024-05-08T08:52:33.740 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:52:33.740 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:33.741 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:52:33.741 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6812", 2024-05-08T08:52:33.741 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1547661179 2024-05-08T08:52:33.741 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:33.741 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:33.741 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:52:33.741 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6813", 2024-05-08T08:52:33.741 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1547661179 2024-05-08T08:52:33.741 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:33.741 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:33.741 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:33.741 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "172.21.15.59:6809/1547661179", 2024-05-08T08:52:33.741 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "172.21.15.59:6811/1547661179", 2024-05-08T08:52:33.741 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addr": "172.21.15.59:6815/1547661179", 2024-05-08T08:52:33.741 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addr": "172.21.15.59:6813/1547661179", 2024-05-08T08:52:33.742 INFO:teuthology.orchestra.run.smithi059.stdout: "state": [ 2024-05-08T08:52:33.742 INFO:teuthology.orchestra.run.smithi059.stdout: "exists", 2024-05-08T08:52:33.742 INFO:teuthology.orchestra.run.smithi059.stdout: "up" 2024-05-08T08:52:33.742 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:33.742 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:33.742 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:33.742 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 3, 2024-05-08T08:52:33.742 INFO:teuthology.orchestra.run.smithi059.stdout: "uuid": "cc4870b8-dcd9-4e43-978d-8de6c6396abf", 2024-05-08T08:52:33.742 INFO:teuthology.orchestra.run.smithi059.stdout: "up": 1, 2024-05-08T08:52:33.742 INFO:teuthology.orchestra.run.smithi059.stdout: "in": 1, 2024-05-08T08:52:33.742 INFO:teuthology.orchestra.run.smithi059.stdout: "weight": 1, 2024-05-08T08:52:33.742 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity": 1, 2024-05-08T08:52:33.742 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_begin": 0, 2024-05-08T08:52:33.743 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_end": 0, 2024-05-08T08:52:33.743 INFO:teuthology.orchestra.run.smithi059.stdout: "up_from": 11, 2024-05-08T08:52:33.743 INFO:teuthology.orchestra.run.smithi059.stdout: "up_thru": 18, 2024-05-08T08:52:33.743 INFO:teuthology.orchestra.run.smithi059.stdout: "down_at": 0, 2024-05-08T08:52:33.743 INFO:teuthology.orchestra.run.smithi059.stdout: "lost_at": 0, 2024-05-08T08:52:33.743 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrs": { 2024-05-08T08:52:33.743 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:52:33.743 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:33.743 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:52:33.743 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6800", 2024-05-08T08:52:33.743 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4054400810 2024-05-08T08:52:33.743 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:33.743 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:33.743 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:52:33.743 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6801", 2024-05-08T08:52:33.744 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4054400810 2024-05-08T08:52:33.744 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:33.744 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:33.744 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:33.744 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addrs": { 2024-05-08T08:52:33.744 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:52:33.744 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:33.744 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:52:33.744 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6802", 2024-05-08T08:52:33.744 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4054400810 2024-05-08T08:52:33.744 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:33.744 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:33.744 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:52:33.744 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6803", 2024-05-08T08:52:33.744 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4054400810 2024-05-08T08:52:33.744 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:33.745 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:33.745 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:33.745 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addrs": { 2024-05-08T08:52:33.745 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:52:33.745 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:33.745 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:52:33.745 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6806", 2024-05-08T08:52:33.745 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4054400810 2024-05-08T08:52:33.745 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:33.745 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:33.745 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:52:33.745 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6807", 2024-05-08T08:52:33.745 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4054400810 2024-05-08T08:52:33.745 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:33.745 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:33.746 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:33.746 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addrs": { 2024-05-08T08:52:33.746 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:52:33.746 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:33.746 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:52:33.746 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6804", 2024-05-08T08:52:33.746 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4054400810 2024-05-08T08:52:33.746 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:33.746 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:33.746 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:52:33.746 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6805", 2024-05-08T08:52:33.746 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4054400810 2024-05-08T08:52:33.746 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:33.746 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:33.747 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:33.747 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "172.21.15.192:6801/4054400810", 2024-05-08T08:52:33.747 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "172.21.15.192:6803/4054400810", 2024-05-08T08:52:33.747 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addr": "172.21.15.192:6807/4054400810", 2024-05-08T08:52:33.747 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addr": "172.21.15.192:6805/4054400810", 2024-05-08T08:52:33.747 INFO:teuthology.orchestra.run.smithi059.stdout: "state": [ 2024-05-08T08:52:33.747 INFO:teuthology.orchestra.run.smithi059.stdout: "exists", 2024-05-08T08:52:33.747 INFO:teuthology.orchestra.run.smithi059.stdout: "up" 2024-05-08T08:52:33.747 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:33.747 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:33.747 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:33.747 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 4, 2024-05-08T08:52:33.747 INFO:teuthology.orchestra.run.smithi059.stdout: "uuid": "d1d05620-42fb-4107-a2f9-e83542d076ee", 2024-05-08T08:52:33.747 INFO:teuthology.orchestra.run.smithi059.stdout: "up": 1, 2024-05-08T08:52:33.747 INFO:teuthology.orchestra.run.smithi059.stdout: "in": 1, 2024-05-08T08:52:33.748 INFO:teuthology.orchestra.run.smithi059.stdout: "weight": 1, 2024-05-08T08:52:33.748 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity": 0.9892578125, 2024-05-08T08:52:33.748 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_begin": 0, 2024-05-08T08:52:33.748 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_end": 0, 2024-05-08T08:52:33.748 INFO:teuthology.orchestra.run.smithi059.stdout: "up_from": 11, 2024-05-08T08:52:33.748 INFO:teuthology.orchestra.run.smithi059.stdout: "up_thru": 28, 2024-05-08T08:52:33.748 INFO:teuthology.orchestra.run.smithi059.stdout: "down_at": 0, 2024-05-08T08:52:33.748 INFO:teuthology.orchestra.run.smithi059.stdout: "lost_at": 0, 2024-05-08T08:52:33.748 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrs": { 2024-05-08T08:52:33.748 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:52:33.748 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:33.748 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:52:33.748 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6808", 2024-05-08T08:52:33.748 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 58985565 2024-05-08T08:52:33.748 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:33.748 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:33.749 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:52:33.749 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6809", 2024-05-08T08:52:33.749 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 58985565 2024-05-08T08:52:33.749 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:33.749 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:33.749 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:33.749 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addrs": { 2024-05-08T08:52:33.749 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:52:33.749 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:33.749 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:52:33.749 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6810", 2024-05-08T08:52:33.749 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 58985565 2024-05-08T08:52:33.749 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:33.749 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:33.749 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:52:33.749 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6811", 2024-05-08T08:52:33.750 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 58985565 2024-05-08T08:52:33.750 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:33.750 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:33.750 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:33.750 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addrs": { 2024-05-08T08:52:33.750 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:52:33.750 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:33.750 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:52:33.750 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6814", 2024-05-08T08:52:33.750 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 58985565 2024-05-08T08:52:33.750 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:33.750 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:33.750 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:52:33.750 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6815", 2024-05-08T08:52:33.751 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 58985565 2024-05-08T08:52:33.751 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:33.751 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:33.751 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:33.751 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addrs": { 2024-05-08T08:52:33.751 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:52:33.751 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:33.751 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:52:33.751 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6812", 2024-05-08T08:52:33.751 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 58985565 2024-05-08T08:52:33.751 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:33.751 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:33.751 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:52:33.751 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6813", 2024-05-08T08:52:33.752 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 58985565 2024-05-08T08:52:33.752 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:33.752 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:33.752 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:33.752 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "172.21.15.192:6809/58985565", 2024-05-08T08:52:33.752 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "172.21.15.192:6811/58985565", 2024-05-08T08:52:33.752 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addr": "172.21.15.192:6815/58985565", 2024-05-08T08:52:33.752 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addr": "172.21.15.192:6813/58985565", 2024-05-08T08:52:33.752 INFO:teuthology.orchestra.run.smithi059.stdout: "state": [ 2024-05-08T08:52:33.752 INFO:teuthology.orchestra.run.smithi059.stdout: "exists", 2024-05-08T08:52:33.752 INFO:teuthology.orchestra.run.smithi059.stdout: "up" 2024-05-08T08:52:33.752 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:33.752 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:33.752 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:33.752 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 5, 2024-05-08T08:52:33.753 INFO:teuthology.orchestra.run.smithi059.stdout: "uuid": "ab26136c-31c3-4090-9fa2-988dba4f731f", 2024-05-08T08:52:33.753 INFO:teuthology.orchestra.run.smithi059.stdout: "up": 1, 2024-05-08T08:52:33.753 INFO:teuthology.orchestra.run.smithi059.stdout: "in": 1, 2024-05-08T08:52:33.753 INFO:teuthology.orchestra.run.smithi059.stdout: "weight": 1, 2024-05-08T08:52:33.753 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity": 0.2798919677734375, 2024-05-08T08:52:33.753 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_begin": 0, 2024-05-08T08:52:33.753 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_end": 0, 2024-05-08T08:52:33.753 INFO:teuthology.orchestra.run.smithi059.stdout: "up_from": 11, 2024-05-08T08:52:33.753 INFO:teuthology.orchestra.run.smithi059.stdout: "up_thru": 18, 2024-05-08T08:52:33.753 INFO:teuthology.orchestra.run.smithi059.stdout: "down_at": 0, 2024-05-08T08:52:33.753 INFO:teuthology.orchestra.run.smithi059.stdout: "lost_at": 0, 2024-05-08T08:52:33.753 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrs": { 2024-05-08T08:52:33.753 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:52:33.753 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:33.753 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:52:33.753 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6816", 2024-05-08T08:52:33.754 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1283571573 2024-05-08T08:52:33.754 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:33.754 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:33.754 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:52:33.754 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6817", 2024-05-08T08:52:33.754 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1283571573 2024-05-08T08:52:33.754 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:33.754 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:33.754 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:33.754 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addrs": { 2024-05-08T08:52:33.754 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:52:33.754 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:33.754 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:52:33.754 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6818", 2024-05-08T08:52:33.755 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1283571573 2024-05-08T08:52:33.755 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:33.755 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:33.755 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:52:33.755 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6819", 2024-05-08T08:52:33.755 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1283571573 2024-05-08T08:52:33.755 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:33.755 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:33.755 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:33.755 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addrs": { 2024-05-08T08:52:33.755 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:52:33.755 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:33.755 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:52:33.755 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6822", 2024-05-08T08:52:33.756 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1283571573 2024-05-08T08:52:33.756 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:33.756 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:33.756 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:52:33.756 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6823", 2024-05-08T08:52:33.756 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1283571573 2024-05-08T08:52:33.756 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:33.756 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:33.756 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:33.756 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addrs": { 2024-05-08T08:52:33.756 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:52:33.756 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:33.756 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:52:33.756 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6820", 2024-05-08T08:52:33.756 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1283571573 2024-05-08T08:52:33.756 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:33.757 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:33.757 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:52:33.757 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6821", 2024-05-08T08:52:33.757 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1283571573 2024-05-08T08:52:33.757 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:33.757 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:33.757 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:33.757 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "172.21.15.192:6817/1283571573", 2024-05-08T08:52:33.757 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "172.21.15.192:6819/1283571573", 2024-05-08T08:52:33.757 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addr": "172.21.15.192:6823/1283571573", 2024-05-08T08:52:33.757 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addr": "172.21.15.192:6821/1283571573", 2024-05-08T08:52:33.757 INFO:teuthology.orchestra.run.smithi059.stdout: "state": [ 2024-05-08T08:52:33.757 INFO:teuthology.orchestra.run.smithi059.stdout: "exists", 2024-05-08T08:52:33.757 INFO:teuthology.orchestra.run.smithi059.stdout: "up" 2024-05-08T08:52:33.757 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:33.758 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:33.758 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T08:52:33.758 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_xinfo": [ 2024-05-08T08:52:33.758 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:33.758 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 0, 2024-05-08T08:52:33.758 INFO:teuthology.orchestra.run.smithi059.stdout: "down_stamp": "0.000000", 2024-05-08T08:52:33.758 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_probability": 0, 2024-05-08T08:52:33.758 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_interval": 0, 2024-05-08T08:52:33.758 INFO:teuthology.orchestra.run.smithi059.stdout: "features": 4540701547738038271, 2024-05-08T08:52:33.758 INFO:teuthology.orchestra.run.smithi059.stdout: "old_weight": 0, 2024-05-08T08:52:33.758 INFO:teuthology.orchestra.run.smithi059.stdout: "last_purged_snaps_scrub": "2024-05-08T08:51:52.396946+0000", 2024-05-08T08:52:33.758 INFO:teuthology.orchestra.run.smithi059.stdout: "dead_epoch": 0 2024-05-08T08:52:33.758 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:33.758 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:33.759 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 1, 2024-05-08T08:52:33.759 INFO:teuthology.orchestra.run.smithi059.stdout: "down_stamp": "0.000000", 2024-05-08T08:52:33.759 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_probability": 0, 2024-05-08T08:52:33.759 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_interval": 0, 2024-05-08T08:52:33.759 INFO:teuthology.orchestra.run.smithi059.stdout: "features": 4540701547738038271, 2024-05-08T08:52:33.759 INFO:teuthology.orchestra.run.smithi059.stdout: "old_weight": 0, 2024-05-08T08:52:33.759 INFO:teuthology.orchestra.run.smithi059.stdout: "last_purged_snaps_scrub": "2024-05-08T08:51:52.453160+0000", 2024-05-08T08:52:33.759 INFO:teuthology.orchestra.run.smithi059.stdout: "dead_epoch": 0 2024-05-08T08:52:33.759 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:33.759 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:33.759 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 2, 2024-05-08T08:52:33.759 INFO:teuthology.orchestra.run.smithi059.stdout: "down_stamp": "0.000000", 2024-05-08T08:52:33.759 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_probability": 0, 2024-05-08T08:52:33.759 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_interval": 0, 2024-05-08T08:52:33.759 INFO:teuthology.orchestra.run.smithi059.stdout: "features": 4540701547738038271, 2024-05-08T08:52:33.760 INFO:teuthology.orchestra.run.smithi059.stdout: "old_weight": 0, 2024-05-08T08:52:33.760 INFO:teuthology.orchestra.run.smithi059.stdout: "last_purged_snaps_scrub": "2024-05-08T08:51:52.387606+0000", 2024-05-08T08:52:33.760 INFO:teuthology.orchestra.run.smithi059.stdout: "dead_epoch": 0 2024-05-08T08:52:33.760 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:33.760 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:33.760 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 3, 2024-05-08T08:52:33.760 INFO:teuthology.orchestra.run.smithi059.stdout: "down_stamp": "0.000000", 2024-05-08T08:52:33.760 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_probability": 0, 2024-05-08T08:52:33.760 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_interval": 0, 2024-05-08T08:52:33.760 INFO:teuthology.orchestra.run.smithi059.stdout: "features": 4540701547738038271, 2024-05-08T08:52:33.760 INFO:teuthology.orchestra.run.smithi059.stdout: "old_weight": 0, 2024-05-08T08:52:33.760 INFO:teuthology.orchestra.run.smithi059.stdout: "last_purged_snaps_scrub": "2024-05-08T08:51:52.360952+0000", 2024-05-08T08:52:33.760 INFO:teuthology.orchestra.run.smithi059.stdout: "dead_epoch": 0 2024-05-08T08:52:33.760 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:33.760 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:33.760 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 4, 2024-05-08T08:52:33.761 INFO:teuthology.orchestra.run.smithi059.stdout: "down_stamp": "0.000000", 2024-05-08T08:52:33.761 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_probability": 0, 2024-05-08T08:52:33.761 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_interval": 0, 2024-05-08T08:52:33.761 INFO:teuthology.orchestra.run.smithi059.stdout: "features": 4540701547738038271, 2024-05-08T08:52:33.761 INFO:teuthology.orchestra.run.smithi059.stdout: "old_weight": 0, 2024-05-08T08:52:33.761 INFO:teuthology.orchestra.run.smithi059.stdout: "last_purged_snaps_scrub": "2024-05-08T08:51:52.468681+0000", 2024-05-08T08:52:33.761 INFO:teuthology.orchestra.run.smithi059.stdout: "dead_epoch": 0 2024-05-08T08:52:33.761 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:33.761 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:33.761 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 5, 2024-05-08T08:52:33.761 INFO:teuthology.orchestra.run.smithi059.stdout: "down_stamp": "0.000000", 2024-05-08T08:52:33.761 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_probability": 0, 2024-05-08T08:52:33.761 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_interval": 0, 2024-05-08T08:52:33.761 INFO:teuthology.orchestra.run.smithi059.stdout: "features": 4540701547738038271, 2024-05-08T08:52:33.761 INFO:teuthology.orchestra.run.smithi059.stdout: "old_weight": 0, 2024-05-08T08:52:33.762 INFO:teuthology.orchestra.run.smithi059.stdout: "last_purged_snaps_scrub": "2024-05-08T08:51:52.401080+0000", 2024-05-08T08:52:33.762 INFO:teuthology.orchestra.run.smithi059.stdout: "dead_epoch": 0 2024-05-08T08:52:33.762 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:33.762 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T08:52:33.762 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_upmap": [], 2024-05-08T08:52:33.762 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_upmap_items": [], 2024-05-08T08:52:33.762 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_upmap_primaries": [], 2024-05-08T08:52:33.762 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_temp": [], 2024-05-08T08:52:33.762 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_temp": [], 2024-05-08T08:52:33.762 INFO:teuthology.orchestra.run.smithi059.stdout: "blocklist": {}, 2024-05-08T08:52:33.762 INFO:teuthology.orchestra.run.smithi059.stdout: "range_blocklist": {}, 2024-05-08T08:52:33.762 INFO:teuthology.orchestra.run.smithi059.stdout: "erasure_code_profiles": { 2024-05-08T08:52:33.762 INFO:teuthology.orchestra.run.smithi059.stdout: "default": { 2024-05-08T08:52:33.762 INFO:teuthology.orchestra.run.smithi059.stdout: "crush-failure-domain": "osd", 2024-05-08T08:52:33.762 INFO:teuthology.orchestra.run.smithi059.stdout: "k": "2", 2024-05-08T08:52:33.763 INFO:teuthology.orchestra.run.smithi059.stdout: "m": "1", 2024-05-08T08:52:33.763 INFO:teuthology.orchestra.run.smithi059.stdout: "plugin": "jerasure", 2024-05-08T08:52:33.763 INFO:teuthology.orchestra.run.smithi059.stdout: "technique": "reed_sol_van" 2024-05-08T08:52:33.763 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:33.763 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:33.763 INFO:teuthology.orchestra.run.smithi059.stdout: "removed_snaps_queue": [], 2024-05-08T08:52:33.763 INFO:teuthology.orchestra.run.smithi059.stdout: "new_removed_snaps": [], 2024-05-08T08:52:33.763 INFO:teuthology.orchestra.run.smithi059.stdout: "new_purged_snaps": [], 2024-05-08T08:52:33.763 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_node_flags": {}, 2024-05-08T08:52:33.763 INFO:teuthology.orchestra.run.smithi059.stdout: "device_class_flags": {}, 2024-05-08T08:52:33.763 INFO:teuthology.orchestra.run.smithi059.stdout: "stretch_mode": { 2024-05-08T08:52:33.763 INFO:teuthology.orchestra.run.smithi059.stdout: "stretch_mode_enabled": false, 2024-05-08T08:52:33.763 INFO:teuthology.orchestra.run.smithi059.stdout: "stretch_bucket_count": 0, 2024-05-08T08:52:33.763 INFO:teuthology.orchestra.run.smithi059.stdout: "degraded_stretch_mode": 0, 2024-05-08T08:52:33.763 INFO:teuthology.orchestra.run.smithi059.stdout: "recovering_stretch_mode": 0, 2024-05-08T08:52:33.763 INFO:teuthology.orchestra.run.smithi059.stdout: "stretch_mode_bucket": 0 2024-05-08T08:52:33.763 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:33.764 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:52:33.765 INFO:tasks.thrashosds.thrasher:j is {'epoch': 33, 'fsid': '38dfae8c-5c5e-4b0f-a264-91f22c6c5554', 'created': '2024-05-08T08:51:45.571369+0000', 'modified': '2024-05-08T08:52:31.999420+0000', 'last_up_change': '2024-05-08T08:51:56.847706+0000', 'last_in_change': '2024-05-08T08:51:47.267430+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T08:51:57.787790+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '14', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.8781916499137878, 'average_primary_affinity': 0.8799999952316284, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-05-08T08:52:02.563677+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.8781916499137878, 'average_primary_affinity': 0.8799999952316284, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-05-08T08:52:09.955467+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '31', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 7, 'snap_epoch': 31, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.8781916499137878, 'average_primary_affinity': 0.8799999952316284, 'average_primary_affinity_weighted': 0.8799999952316284}}], 'osds': [{'osd': 0, 'uuid': '300801a5-1919-4b23-8c6d-bc7dad61cc4d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 28, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6800', 'nonce': 802160151}, {'type': 'v1', 'addr': '172.21.15.59:6801', 'nonce': 802160151}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6802', 'nonce': 802160151}, {'type': 'v1', 'addr': '172.21.15.59:6803', 'nonce': 802160151}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6806', 'nonce': 802160151}, {'type': 'v1', 'addr': '172.21.15.59:6807', 'nonce': 802160151}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6804', 'nonce': 802160151}, {'type': 'v1', 'addr': '172.21.15.59:6805', 'nonce': 802160151}]}, 'public_addr': '172.21.15.59:6801/802160151', 'cluster_addr': '172.21.15.59:6803/802160151', 'heartbeat_back_addr': '172.21.15.59:6807/802160151', 'heartbeat_front_addr': '172.21.15.59:6805/802160151', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '0fead0de-6a65-4eba-b400-eea5692702e0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 28, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6816', 'nonce': 4116801011}, {'type': 'v1', 'addr': '172.21.15.59:6817', 'nonce': 4116801011}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6818', 'nonce': 4116801011}, {'type': 'v1', 'addr': '172.21.15.59:6819', 'nonce': 4116801011}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6822', 'nonce': 4116801011}, {'type': 'v1', 'addr': '172.21.15.59:6823', 'nonce': 4116801011}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6820', 'nonce': 4116801011}, {'type': 'v1', 'addr': '172.21.15.59:6821', 'nonce': 4116801011}]}, 'public_addr': '172.21.15.59:6817/4116801011', 'cluster_addr': '172.21.15.59:6819/4116801011', 'heartbeat_back_addr': '172.21.15.59:6823/4116801011', 'heartbeat_front_addr': '172.21.15.59:6821/4116801011', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '1a233de6-1b82-4a51-8e96-4cd4abdf16c8', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 28, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6808', 'nonce': 1547661179}, {'type': 'v1', 'addr': '172.21.15.59:6809', 'nonce': 1547661179}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6810', 'nonce': 1547661179}, {'type': 'v1', 'addr': '172.21.15.59:6811', 'nonce': 1547661179}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6814', 'nonce': 1547661179}, {'type': 'v1', 'addr': '172.21.15.59:6815', 'nonce': 1547661179}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6812', 'nonce': 1547661179}, {'type': 'v1', 'addr': '172.21.15.59:6813', 'nonce': 1547661179}]}, 'public_addr': '172.21.15.59:6809/1547661179', 'cluster_addr': '172.21.15.59:6811/1547661179', 'heartbeat_back_addr': '172.21.15.59:6815/1547661179', 'heartbeat_front_addr': '172.21.15.59:6813/1547661179', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'cc4870b8-dcd9-4e43-978d-8de6c6396abf', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 18, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6800', 'nonce': 4054400810}, {'type': 'v1', 'addr': '172.21.15.192:6801', 'nonce': 4054400810}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6802', 'nonce': 4054400810}, {'type': 'v1', 'addr': '172.21.15.192:6803', 'nonce': 4054400810}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6806', 'nonce': 4054400810}, {'type': 'v1', 'addr': '172.21.15.192:6807', 'nonce': 4054400810}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6804', 'nonce': 4054400810}, {'type': 'v1', 'addr': '172.21.15.192:6805', 'nonce': 4054400810}]}, 'public_addr': '172.21.15.192:6801/4054400810', 'cluster_addr': '172.21.15.192:6803/4054400810', 'heartbeat_back_addr': '172.21.15.192:6807/4054400810', 'heartbeat_front_addr': '172.21.15.192:6805/4054400810', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'd1d05620-42fb-4107-a2f9-e83542d076ee', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.9892578125, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 28, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6808', 'nonce': 58985565}, {'type': 'v1', 'addr': '172.21.15.192:6809', 'nonce': 58985565}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6810', 'nonce': 58985565}, {'type': 'v1', 'addr': '172.21.15.192:6811', 'nonce': 58985565}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6814', 'nonce': 58985565}, {'type': 'v1', 'addr': '172.21.15.192:6815', 'nonce': 58985565}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6812', 'nonce': 58985565}, {'type': 'v1', 'addr': '172.21.15.192:6813', 'nonce': 58985565}]}, 'public_addr': '172.21.15.192:6809/58985565', 'cluster_addr': '172.21.15.192:6811/58985565', 'heartbeat_back_addr': '172.21.15.192:6815/58985565', 'heartbeat_front_addr': '172.21.15.192:6813/58985565', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'ab26136c-31c3-4090-9fa2-988dba4f731f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.2798919677734375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 18, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6816', 'nonce': 1283571573}, {'type': 'v1', 'addr': '172.21.15.192:6817', 'nonce': 1283571573}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6818', 'nonce': 1283571573}, {'type': 'v1', 'addr': '172.21.15.192:6819', 'nonce': 1283571573}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6822', 'nonce': 1283571573}, {'type': 'v1', 'addr': '172.21.15.192:6823', 'nonce': 1283571573}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6820', 'nonce': 1283571573}, {'type': 'v1', 'addr': '172.21.15.192:6821', 'nonce': 1283571573}]}, 'public_addr': '172.21.15.192:6817/1283571573', 'cluster_addr': '172.21.15.192:6819/1283571573', 'heartbeat_back_addr': '172.21.15.192:6823/1283571573', 'heartbeat_front_addr': '172.21.15.192:6821/1283571573', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:51:52.396946+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:51:52.453160+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:51:52.387606+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:51:52.360952+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:51:52.468681+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:51:52.401080+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-05-08T08:52:33.765 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T08:52:33.766 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:33.766 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:33.767 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:52:33.806 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:52:33.809+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:52:33.901 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:52:33.903+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:52:33.917 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:33.917 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:33.927 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:52:34.002 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:52:34.005+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:52:34.018 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-05-08T08:52:34.079 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:34.079 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:34.089 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:52:34.114 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:52:34.115 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-05-08T08:52:34.116 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:52:34.116+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:52:34.128 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":46,"stamp":"2024-05-08T08:52:33.747812+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":139608075,"num_objects":1092,"num_object_clones":108,"num_object_copies":2184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1092,"num_whiteouts":878,"num_read":77150,"num_read_kb":149757,"num_write":3830,"num_write_kb":49436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4126,"ondisk_log_size":4126,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":199168,"kb_used_data":7932,"kb_used_omap":9,"kb_used_meta":191158,"kb_avail":566031872,"statfs":{"total":579820584960,"available":579616636928,"internally_reserved":0,"allocated":8122368,"data_stored":38718579,"data_compressed":1965632,"data_compressed_allocated":4390912,"data_compressed_original":37003264,"omap_allocated":9568,"internal_metadata":195746464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,10,8],"upper_bound":32},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":54644001,"num_objects":541,"num_object_clones":100,"num_object_copies":1082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":541,"num_whiteouts":432,"num_read":59287,"num_read_kb":93732,"num_write":929,"num_write_kb":32197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.759888"},"pg_stats":[{"pgid":"3.1f","version":"33'117","reported_seq":5324,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T08:52:32.757199+0000","last_change":"2024-05-08T08:52:13.934602+0000","last_active":"2024-05-08T08:52:32.757199+0000","last_peered":"2024-05-08T08:52:32.757199+0000","last_clean":"2024-05-08T08:52:32.757199+0000","last_became_active":"2024-05-08T08:52:11.931332+0000","last_became_peered":"2024-05-08T08:52:11.931332+0000","last_unstale":"2024-05-08T08:52:32.757199+0000","last_undegraded":"2024-05-08T08:52:32.757199+0000","last_fullsized":"2024-05-08T08:52:32.757199+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:06:24.407458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000691565,"stat_sum":{"num_bytes":24861712,"num_objects":36,"num_object_clones":5,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":26,"num_read":4073,"num_read_kb":1555,"num_write":207,"num_write_kb":20326,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"31'105","reported_seq":237,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T08:52:32.633501+0000","last_change":"2024-05-08T08:52:13.944947+0000","last_active":"2024-05-08T08:52:32.633501+0000","last_peered":"2024-05-08T08:52:32.633501+0000","last_clean":"2024-05-08T08:52:32.633501+0000","last_became_active":"2024-05-08T08:52:11.930844+0000","last_became_peered":"2024-05-08T08:52:11.930844+0000","last_unstale":"2024-05-08T08:52:32.633501+0000","last_undegraded":"2024-05-08T08:52:32.633501+0000","last_fullsized":"2024-05-08T08:52:32.633501+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:30:03.474421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040424900000000003,"stat_sum":{"num_bytes":1685504,"num_objects":40,"num_object_clones":4,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":34,"num_read":128,"num_read_kb":776,"num_write":70,"num_write_kb":479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"31'69","reported_seq":3629,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T08:52:32.708973+0000","last_change":"2024-05-08T08:52:23.967693+0000","last_active":"2024-05-08T08:52:32.708973+0000","last_peered":"2024-05-08T08:52:32.708973+0000","last_clean":"2024-05-08T08:52:32.708973+0000","last_became_active":"2024-05-08T08:52:23.966512+0000","last_became_peered":"2024-05-08T08:52:23.966512+0000","last_unstale":"2024-05-08T08:52:32.708973+0000","last_undegraded":"2024-05-08T08:52:32.708973+0000","last_fullsized":"2024-05-08T08:52:32.708973+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:28:02.461064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072111799999999998,"stat_sum":{"num_bytes":2545743,"num_objects":29,"num_object_clones":1,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":21,"num_read":3502,"num_read_kb":7694,"num_write":51,"num_write_kb":230,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"31'115","reported_seq":3268,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T08:52:32.008452+0000","last_change":"2024-05-08T08:52:23.966973+0000","last_active":"2024-05-08T08:52:32.008452+0000","last_peered":"2024-05-08T08:52:32.008452+0000","last_clean":"2024-05-08T08:52:32.008452+0000","last_became_active":"2024-05-08T08:52:23.966513+0000","last_became_peered":"2024-05-08T08:52:23.966513+0000","last_unstale":"2024-05-08T08:52:32.008452+0000","last_undegraded":"2024-05-08T08:52:32.008452+0000","last_fullsized":"2024-05-08T08:52:32.008452+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:08:50.043116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094844199999999997,"stat_sum":{"num_bytes":8574976,"num_objects":41,"num_object_clones":5,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":33,"num_read":3153,"num_read_kb":5966,"num_write":81,"num_write_kb":1189,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"31'84","reported_seq":6093,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T08:52:32.740212+0000","last_change":"2024-05-08T08:52:13.934974+0000","last_active":"2024-05-08T08:52:32.740212+0000","last_peered":"2024-05-08T08:52:32.740212+0000","last_clean":"2024-05-08T08:52:32.740212+0000","last_became_active":"2024-05-08T08:52:11.927963+0000","last_became_peered":"2024-05-08T08:52:11.927963+0000","last_unstale":"2024-05-08T08:52:32.740212+0000","last_undegraded":"2024-05-08T08:52:32.740212+0000","last_fullsized":"2024-05-08T08:52:32.740212+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:08:10.106626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000685079,"stat_sum":{"num_bytes":5629440,"num_objects":35,"num_object_clones":1,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":32,"num_read":5999,"num_read_kb":9269,"num_write":53,"num_write_kb":352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"31'170","reported_seq":3730,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T08:52:32.008353+0000","last_change":"2024-05-08T08:52:13.938821+0000","last_active":"2024-05-08T08:52:32.008353+0000","last_peered":"2024-05-08T08:52:32.008353+0000","last_clean":"2024-05-08T08:52:32.008353+0000","last_became_active":"2024-05-08T08:52:11.926106+0000","last_became_peered":"2024-05-08T08:52:11.926106+0000","last_unstale":"2024-05-08T08:52:32.008353+0000","last_undegraded":"2024-05-08T08:52:32.008353+0000","last_fullsized":"2024-05-08T08:52:32.008353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:57:48.385593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066525600000000003,"stat_sum":{"num_bytes":4185137,"num_objects":38,"num_object_clones":4,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":28,"num_read":3909,"num_read_kb":6254,"num_write":197,"num_write_kb":1766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"31'105","reported_seq":194,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T08:52:32.634684+0000","last_change":"2024-05-08T08:52:13.940847+0000","last_active":"2024-05-08T08:52:32.634684+0000","last_peered":"2024-05-08T08:52:32.634684+0000","last_clean":"2024-05-08T08:52:32.634684+0000","last_became_active":"2024-05-08T08:52:11.925809+0000","last_became_peered":"2024-05-08T08:52:11.925809+0000","last_unstale":"2024-05-08T08:52:32.634684+0000","last_undegraded":"2024-05-08T08:52:32.634684+0000","last_fullsized":"2024-05-08T08:52:32.634684+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:29:46.421472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082481000000000004,"stat_sum":{"num_bytes":1319936,"num_objects":37,"num_object_clones":3,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":34,"num_read":79,"num_read_kb":1299,"num_write":69,"num_write_kb":1441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"31'81","reported_seq":172,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T08:52:32.008514+0000","last_change":"2024-05-08T08:52:13.945184+0000","last_active":"2024-05-08T08:52:32.008514+0000","last_peered":"2024-05-08T08:52:32.008514+0000","last_clean":"2024-05-08T08:52:32.008514+0000","last_became_active":"2024-05-08T08:52:11.927366+0000","last_became_peered":"2024-05-08T08:52:11.927366+0000","last_unstale":"2024-05-08T08:52:32.008514+0000","last_undegraded":"2024-05-08T08:52:32.008514+0000","last_fullsized":"2024-05-08T08:52:32.008514+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:32:15.610690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057208900000000002,"stat_sum":{"num_bytes":1719808,"num_objects":32,"num_object_clones":3,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":28,"num_read":75,"num_read_kb":588,"num_write":50,"num_write_kb":734,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"31'361","reported_seq":513,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T08:52:32.515807+0000","last_change":"2024-05-08T08:52:13.935180+0000","last_active":"2024-05-08T08:52:32.515807+0000","last_peered":"2024-05-08T08:52:32.515807+0000","last_clean":"2024-05-08T08:52:32.515807+0000","last_became_active":"2024-05-08T08:52:11.930714+0000","last_became_peered":"2024-05-08T08:52:11.930714+0000","last_unstale":"2024-05-08T08:52:32.515807+0000","last_undegraded":"2024-05-08T08:52:32.515807+0000","last_fullsized":"2024-05-08T08:52:32.515807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":361,"log_dups_size":0,"ondisk_log_size":361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:23:10.125313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000866371,"stat_sum":{"num_bytes":1124449,"num_objects":24,"num_object_clones":2,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":18,"num_read":1636,"num_read_kb":1695,"num_write":550,"num_write_kb":1200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"31'93","reported_seq":8173,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T08:52:32.709643+0000","last_change":"2024-05-08T08:52:13.932215+0000","last_active":"2024-05-08T08:52:32.709643+0000","last_peered":"2024-05-08T08:52:32.709643+0000","last_clean":"2024-05-08T08:52:32.709643+0000","last_became_active":"2024-05-08T08:52:11.931213+0000","last_became_peered":"2024-05-08T08:52:11.931213+0000","last_unstale":"2024-05-08T08:52:32.709643+0000","last_undegraded":"2024-05-08T08:52:32.709643+0000","last_fullsized":"2024-05-08T08:52:32.709643+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:17:25.946639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084652399999999997,"stat_sum":{"num_bytes":8559771,"num_objects":30,"num_object_clones":3,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":20,"num_read":8065,"num_read_kb":10058,"num_write":79,"num_write_kb":1151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"32'88","reported_seq":3144,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T08:52:32.756297+0000","last_change":"2024-05-08T08:52:13.944999+0000","last_active":"2024-05-08T08:52:32.756297+0000","last_peered":"2024-05-08T08:52:32.756297+0000","last_clean":"2024-05-08T08:52:32.756297+0000","last_became_active":"2024-05-08T08:52:11.931695+0000","last_became_peered":"2024-05-08T08:52:11.931695+0000","last_unstale":"2024-05-08T08:52:32.756297+0000","last_undegraded":"2024-05-08T08:52:32.756297+0000","last_fullsized":"2024-05-08T08:52:32.756297+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:59:46.409551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063902800000000004,"stat_sum":{"num_bytes":6899200,"num_objects":32,"num_object_clones":3,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":26,"num_read":3044,"num_read_kb":5301,"num_write":62,"num_write_kb":340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"32'126","reported_seq":3103,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T08:52:32.005994+0000","last_change":"2024-05-08T08:52:13.944700+0000","last_active":"2024-05-08T08:52:32.005994+0000","last_peered":"2024-05-08T08:52:32.005994+0000","last_clean":"2024-05-08T08:52:32.005994+0000","last_became_active":"2024-05-08T08:52:11.932699+0000","last_became_peered":"2024-05-08T08:52:11.932699+0000","last_unstale":"2024-05-08T08:52:32.005994+0000","last_undegraded":"2024-05-08T08:52:32.005994+0000","last_fullsized":"2024-05-08T08:52:32.005994+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:06:42.141391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000430331,"stat_sum":{"num_bytes":1945616,"num_objects":34,"num_object_clones":5,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":26,"num_read":564,"num_read_kb":1019,"num_write":71,"num_write_kb":617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":44,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T08:52:32.006256+0000","last_change":"2024-05-08T08:52:03.893505+0000","last_active":"2024-05-08T08:52:32.006256+0000","last_peered":"2024-05-08T08:52:32.006256+0000","last_clean":"2024-05-08T08:52:32.006256+0000","last_became_active":"2024-05-08T08:52:03.893187+0000","last_became_peered":"2024-05-08T08:52:03.893187+0000","last_unstale":"2024-05-08T08:52:32.006256+0000","last_undegraded":"2024-05-08T08:52:32.006256+0000","last_fullsized":"2024-05-08T08:52:32.006256+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:51:04.039242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"31'73","reported_seq":5217,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T08:52:32.515765+0000","last_change":"2024-05-08T08:52:13.934894+0000","last_active":"2024-05-08T08:52:32.515765+0000","last_peered":"2024-05-08T08:52:32.515765+0000","last_clean":"2024-05-08T08:52:32.515765+0000","last_became_active":"2024-05-08T08:52:11.929184+0000","last_became_peered":"2024-05-08T08:52:11.929184+0000","last_unstale":"2024-05-08T08:52:32.515765+0000","last_undegraded":"2024-05-08T08:52:32.515765+0000","last_fullsized":"2024-05-08T08:52:32.515765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:14:07.878464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088867899999999997,"stat_sum":{"num_bytes":7365632,"num_objects":30,"num_object_clones":2,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":22,"num_read":5123,"num_read_kb":13640,"num_write":52,"num_write_kb":914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":44,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T08:52:32.634946+0000","last_change":"2024-05-08T08:52:03.896159+0000","last_active":"2024-05-08T08:52:32.634946+0000","last_peered":"2024-05-08T08:52:32.634946+0000","last_clean":"2024-05-08T08:52:32.634946+0000","last_became_active":"2024-05-08T08:52:03.895797+0000","last_became_peered":"2024-05-08T08:52:03.895797+0000","last_unstale":"2024-05-08T08:52:32.634946+0000","last_undegraded":"2024-05-08T08:52:32.634946+0000","last_fullsized":"2024-05-08T08:52:32.634946+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:34:33.513525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"31'86","reported_seq":154,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T08:52:32.008762+0000","last_change":"2024-05-08T08:52:13.944951+0000","last_active":"2024-05-08T08:52:32.008762+0000","last_peered":"2024-05-08T08:52:32.008762+0000","last_clean":"2024-05-08T08:52:32.008762+0000","last_became_active":"2024-05-08T08:52:11.926905+0000","last_became_peered":"2024-05-08T08:52:11.926905+0000","last_unstale":"2024-05-08T08:52:32.008762+0000","last_undegraded":"2024-05-08T08:52:32.008762+0000","last_fullsized":"2024-05-08T08:52:32.008762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:04:52.530792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044527,"stat_sum":{"num_bytes":2332818,"num_objects":34,"num_object_clones":6,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":27,"num_read":100,"num_read_kb":678,"num_write":64,"num_write_kb":722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":44,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T08:52:32.006138+0000","last_change":"2024-05-08T08:52:04.144764+0000","last_active":"2024-05-08T08:52:32.006138+0000","last_peered":"2024-05-08T08:52:32.006138+0000","last_clean":"2024-05-08T08:52:32.006138+0000","last_became_active":"2024-05-08T08:52:04.144444+0000","last_became_peered":"2024-05-08T08:52:04.144444+0000","last_unstale":"2024-05-08T08:52:32.006138+0000","last_undegraded":"2024-05-08T08:52:32.006138+0000","last_fullsized":"2024-05-08T08:52:32.006138+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:58:31.498922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"32'123","reported_seq":3149,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T08:52:32.635034+0000","last_change":"2024-05-08T08:52:13.940976+0000","last_active":"2024-05-08T08:52:32.635034+0000","last_peered":"2024-05-08T08:52:32.635034+0000","last_clean":"2024-05-08T08:52:32.635034+0000","last_became_active":"2024-05-08T08:52:11.926402+0000","last_became_peered":"2024-05-08T08:52:11.926402+0000","last_unstale":"2024-05-08T08:52:32.635034+0000","last_undegraded":"2024-05-08T08:52:32.635034+0000","last_fullsized":"2024-05-08T08:52:32.635034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:43:06.930287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000669351,"stat_sum":{"num_bytes":1548288,"num_objects":40,"num_object_clones":4,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":34,"num_read":3464,"num_read_kb":2632,"num_write":80,"num_write_kb":862,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":44,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T08:52:32.635001+0000","last_change":"2024-05-08T08:52:04.144543+0000","last_active":"2024-05-08T08:52:32.635001+0000","last_peered":"2024-05-08T08:52:32.635001+0000","last_clean":"2024-05-08T08:52:32.635001+0000","last_became_active":"2024-05-08T08:52:04.144242+0000","last_became_peered":"2024-05-08T08:52:04.144242+0000","last_unstale":"2024-05-08T08:52:32.635001+0000","last_undegraded":"2024-05-08T08:52:32.635001+0000","last_fullsized":"2024-05-08T08:52:32.635001+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:57:36.844633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"31'484","reported_seq":3246,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T08:52:32.008737+0000","last_change":"2024-05-08T08:52:23.967224+0000","last_active":"2024-05-08T08:52:32.008737+0000","last_peered":"2024-05-08T08:52:32.008737+0000","last_clean":"2024-05-08T08:52:32.008737+0000","last_became_active":"2024-05-08T08:52:23.966596+0000","last_became_peered":"2024-05-08T08:52:23.966596+0000","last_unstale":"2024-05-08T08:52:32.008737+0000","last_undegraded":"2024-05-08T08:52:32.008737+0000","last_fullsized":"2024-05-08T08:52:32.008737+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:38:48.025531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064078099999999999,"stat_sum":{"num_bytes":4546731,"num_objects":51,"num_object_clones":5,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":42,"num_read":4799,"num_read_kb":6109,"num_write":757,"num_write_kb":899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"31'57","reported_seq":105,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T08:52:32.008366+0000","last_change":"2024-05-08T08:52:13.938650+0000","last_active":"2024-05-08T08:52:32.008366+0000","last_peered":"2024-05-08T08:52:32.008366+0000","last_clean":"2024-05-08T08:52:32.008366+0000","last_became_active":"2024-05-08T08:52:11.927134+0000","last_became_peered":"2024-05-08T08:52:11.927134+0000","last_unstale":"2024-05-08T08:52:32.008366+0000","last_undegraded":"2024-05-08T08:52:32.008366+0000","last_fullsized":"2024-05-08T08:52:32.008366+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:48:15.963607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047037799999999997,"stat_sum":{"num_bytes":1220608,"num_objects":23,"num_object_clones":3,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":20,"num_read":24,"num_read_kb":36,"num_write":35,"num_write_kb":1067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T08:52:32.634863+0000","last_change":"2024-05-08T08:52:23.969093+0000","last_active":"2024-05-08T08:52:32.634863+0000","last_peered":"2024-05-08T08:52:32.634863+0000","last_clean":"2024-05-08T08:52:32.634863+0000","last_became_active":"2024-05-08T08:52:23.968782+0000","last_became_peered":"2024-05-08T08:52:23.968782+0000","last_unstale":"2024-05-08T08:52:32.634863+0000","last_undegraded":"2024-05-08T08:52:32.634863+0000","last_fullsized":"2024-05-08T08:52:32.634863+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:37:41.693728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"31'83","reported_seq":219,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T08:52:32.634664+0000","last_change":"2024-05-08T08:52:13.940504+0000","last_active":"2024-05-08T08:52:32.634664+0000","last_peered":"2024-05-08T08:52:32.634664+0000","last_clean":"2024-05-08T08:52:32.634664+0000","last_became_active":"2024-05-08T08:52:11.928301+0000","last_became_peered":"2024-05-08T08:52:11.928301+0000","last_unstale":"2024-05-08T08:52:32.634664+0000","last_undegraded":"2024-05-08T08:52:32.634664+0000","last_fullsized":"2024-05-08T08:52:32.634664+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:49:48.473134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046631800000000002,"stat_sum":{"num_bytes":1052688,"num_objects":32,"num_object_clones":3,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":27,"num_read":128,"num_read_kb":841,"num_write":54,"num_write_kb":796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":44,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T08:52:32.008558+0000","last_change":"2024-05-08T08:52:03.891768+0000","last_active":"2024-05-08T08:52:32.008558+0000","last_peered":"2024-05-08T08:52:32.008558+0000","last_clean":"2024-05-08T08:52:32.008558+0000","last_became_active":"2024-05-08T08:52:03.891374+0000","last_became_peered":"2024-05-08T08:52:03.891374+0000","last_unstale":"2024-05-08T08:52:32.008558+0000","last_undegraded":"2024-05-08T08:52:32.008558+0000","last_fullsized":"2024-05-08T08:52:32.008558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:18:10.240492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"31'214","reported_seq":7904,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T08:52:32.515539+0000","last_change":"2024-05-08T08:52:13.934317+0000","last_active":"2024-05-08T08:52:32.515539+0000","last_peered":"2024-05-08T08:52:32.515539+0000","last_clean":"2024-05-08T08:52:32.515539+0000","last_became_active":"2024-05-08T08:52:11.928598+0000","last_became_peered":"2024-05-08T08:52:11.928598+0000","last_unstale":"2024-05-08T08:52:32.515539+0000","last_undegraded":"2024-05-08T08:52:32.515539+0000","last_fullsized":"2024-05-08T08:52:32.515539+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":214,"log_dups_size":0,"ondisk_log_size":214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:04:30.756423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046440499999999998,"stat_sum":{"num_bytes":6546992,"num_objects":28,"num_object_clones":2,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":21,"num_read":8207,"num_read_kb":10533,"num_write":117,"num_write_kb":559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":44,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T08:52:32.005919+0000","last_change":"2024-05-08T08:52:03.891551+0000","last_active":"2024-05-08T08:52:32.005919+0000","last_peered":"2024-05-08T08:52:32.005919+0000","last_clean":"2024-05-08T08:52:32.005919+0000","last_became_active":"2024-05-08T08:52:03.891296+0000","last_became_peered":"2024-05-08T08:52:03.891296+0000","last_unstale":"2024-05-08T08:52:32.005919+0000","last_undegraded":"2024-05-08T08:52:32.005919+0000","last_fullsized":"2024-05-08T08:52:32.005919+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:54.806343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"31'86","reported_seq":361,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T08:52:32.634847+0000","last_change":"2024-05-08T08:52:13.940743+0000","last_active":"2024-05-08T08:52:32.634847+0000","last_peered":"2024-05-08T08:52:32.634847+0000","last_clean":"2024-05-08T08:52:32.634847+0000","last_became_active":"2024-05-08T08:52:11.928273+0000","last_became_peered":"2024-05-08T08:52:11.928273+0000","last_unstale":"2024-05-08T08:52:32.634847+0000","last_undegraded":"2024-05-08T08:52:32.634847+0000","last_fullsized":"2024-05-08T08:52:32.634847+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:36:06.878336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062135599999999999,"stat_sum":{"num_bytes":2326544,"num_objects":37,"num_object_clones":7,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":28,"num_read":274,"num_read_kb":2967,"num_write":59,"num_write_kb":889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":19,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T08:52:32.008619+0000","last_change":"2024-05-08T08:52:23.966794+0000","last_active":"2024-05-08T08:52:32.008619+0000","last_peered":"2024-05-08T08:52:32.008619+0000","last_clean":"2024-05-08T08:52:32.008619+0000","last_became_active":"2024-05-08T08:52:23.966544+0000","last_became_peered":"2024-05-08T08:52:23.966544+0000","last_unstale":"2024-05-08T08:52:32.008619+0000","last_undegraded":"2024-05-08T08:52:32.008619+0000","last_fullsized":"2024-05-08T08:52:32.008619+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:25:56.927420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":200,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T08:52:32.005926+0000","last_change":"2024-05-08T08:51:58.861899+0000","last_active":"2024-05-08T08:52:32.005926+0000","last_peered":"2024-05-08T08:52:32.005926+0000","last_clean":"2024-05-08T08:52:32.005926+0000","last_became_active":"2024-05-08T08:51:58.861596+0000","last_became_peered":"2024-05-08T08:51:58.861596+0000","last_unstale":"2024-05-08T08:52:32.005926+0000","last_undegraded":"2024-05-08T08:52:32.005926+0000","last_fullsized":"2024-05-08T08:52:32.005926+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:51:57.849666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:51:57.849666+0000","last_clean_scrub_stamp":"2024-05-08T08:51:57.849666+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:53:21.739950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"31'85","reported_seq":215,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T08:52:32.008561+0000","last_change":"2024-05-08T08:52:13.945362+0000","last_active":"2024-05-08T08:52:32.008561+0000","last_peered":"2024-05-08T08:52:32.008561+0000","last_clean":"2024-05-08T08:52:32.008561+0000","last_became_active":"2024-05-08T08:52:11.926975+0000","last_became_peered":"2024-05-08T08:52:11.926975+0000","last_unstale":"2024-05-08T08:52:32.008561+0000","last_undegraded":"2024-05-08T08:52:32.008561+0000","last_fullsized":"2024-05-08T08:52:32.008561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:39:50.543256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068218599999999999,"stat_sum":{"num_bytes":4125712,"num_objects":41,"num_object_clones":5,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":33,"num_read":128,"num_read_kb":928,"num_write":50,"num_write_kb":1280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"31'75","reported_seq":2795,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T08:52:32.757135+0000","last_change":"2024-05-08T08:52:13.938471+0000","last_active":"2024-05-08T08:52:32.757135+0000","last_peered":"2024-05-08T08:52:32.757135+0000","last_clean":"2024-05-08T08:52:32.757135+0000","last_became_active":"2024-05-08T08:52:11.927141+0000","last_became_peered":"2024-05-08T08:52:11.927141+0000","last_unstale":"2024-05-08T08:52:32.757135+0000","last_undegraded":"2024-05-08T08:52:32.757135+0000","last_fullsized":"2024-05-08T08:52:32.757135+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:38:46.412088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00030754000000000003,"stat_sum":{"num_bytes":4210688,"num_objects":31,"num_object_clones":3,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":26,"num_read":2703,"num_read_kb":4457,"num_write":48,"num_write_kb":899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"32'295","reported_seq":5522,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T08:52:32.006060+0000","last_change":"2024-05-08T08:52:13.945213+0000","last_active":"2024-05-08T08:52:32.006060+0000","last_peered":"2024-05-08T08:52:32.006060+0000","last_clean":"2024-05-08T08:52:32.006060+0000","last_became_active":"2024-05-08T08:52:11.932810+0000","last_became_peered":"2024-05-08T08:52:11.932810+0000","last_unstale":"2024-05-08T08:52:32.006060+0000","last_undegraded":"2024-05-08T08:52:32.006060+0000","last_fullsized":"2024-05-08T08:52:32.006060+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":295,"log_dups_size":0,"ondisk_log_size":295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:31:20.342047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070350100000000004,"stat_sum":{"num_bytes":4284416,"num_objects":37,"num_object_clones":3,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":29,"num_read":7503,"num_read_kb":10603,"num_write":164,"num_write_kb":745,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"31'93","reported_seq":449,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T08:52:32.634846+0000","last_change":"2024-05-08T08:52:13.940600+0000","last_active":"2024-05-08T08:52:32.634846+0000","last_peered":"2024-05-08T08:52:32.634846+0000","last_clean":"2024-05-08T08:52:32.634846+0000","last_became_active":"2024-05-08T08:52:11.927196+0000","last_became_peered":"2024-05-08T08:52:11.927196+0000","last_unstale":"2024-05-08T08:52:32.634846+0000","last_undegraded":"2024-05-08T08:52:32.634846+0000","last_fullsized":"2024-05-08T08:52:32.634846+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:55:14.642650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048296599999999998,"stat_sum":{"num_bytes":1239040,"num_objects":32,"num_object_clones":2,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":28,"num_read":343,"num_read_kb":2832,"num_write":69,"num_write_kb":754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"31'79","reported_seq":350,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T08:52:32.515664+0000","last_change":"2024-05-08T08:52:23.969521+0000","last_active":"2024-05-08T08:52:32.515664+0000","last_peered":"2024-05-08T08:52:32.515664+0000","last_clean":"2024-05-08T08:52:32.515664+0000","last_became_active":"2024-05-08T08:52:23.969067+0000","last_became_peered":"2024-05-08T08:52:23.969067+0000","last_unstale":"2024-05-08T08:52:32.515664+0000","last_undegraded":"2024-05-08T08:52:32.515664+0000","last_fullsized":"2024-05-08T08:52:32.515664+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:43:18.053011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048793000000000003,"stat_sum":{"num_bytes":1067008,"num_objects":30,"num_object_clones":1,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":26,"num_read":254,"num_read_kb":2212,"num_write":56,"num_write_kb":690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"31'93","reported_seq":449,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T08:52:32.006078+0000","last_change":"2024-05-08T08:52:13.945141+0000","last_active":"2024-05-08T08:52:32.006078+0000","last_peered":"2024-05-08T08:52:32.006078+0000","last_clean":"2024-05-08T08:52:32.006078+0000","last_became_active":"2024-05-08T08:52:11.932594+0000","last_became_peered":"2024-05-08T08:52:11.932594+0000","last_unstale":"2024-05-08T08:52:32.006078+0000","last_undegraded":"2024-05-08T08:52:32.006078+0000","last_fullsized":"2024-05-08T08:52:32.006078+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:15:02.652007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074388199999999996,"stat_sum":{"num_bytes":1680896,"num_objects":39,"num_object_clones":3,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":31,"num_read":345,"num_read_kb":3686,"num_write":64,"num_write_kb":913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"31'67","reported_seq":3329,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T08:52:32.758090+0000","last_change":"2024-05-08T08:52:23.966947+0000","last_active":"2024-05-08T08:52:32.758090+0000","last_peered":"2024-05-08T08:52:32.758090+0000","last_clean":"2024-05-08T08:52:32.758090+0000","last_became_active":"2024-05-08T08:52:23.966538+0000","last_became_peered":"2024-05-08T08:52:23.966538+0000","last_unstale":"2024-05-08T08:52:32.758090+0000","last_undegraded":"2024-05-08T08:52:32.758090+0000","last_fullsized":"2024-05-08T08:52:32.758090+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:54:22.956014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085079600000000002,"stat_sum":{"num_bytes":4915200,"num_objects":21,"num_object_clones":3,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":16,"num_read":3232,"num_read_kb":6138,"num_write":49,"num_write_kb":771,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"31'104","reported_seq":1357,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T08:52:32.486109+0000","last_change":"2024-05-08T08:52:13.932524+0000","last_active":"2024-05-08T08:52:32.486109+0000","last_peered":"2024-05-08T08:52:32.486109+0000","last_clean":"2024-05-08T08:52:32.486109+0000","last_became_active":"2024-05-08T08:52:11.931347+0000","last_became_peered":"2024-05-08T08:52:11.931347+0000","last_unstale":"2024-05-08T08:52:32.486109+0000","last_undegraded":"2024-05-08T08:52:32.486109+0000","last_fullsized":"2024-05-08T08:52:32.486109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:45:28.884429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010326370000000001,"stat_sum":{"num_bytes":4305038,"num_objects":24,"num_object_clones":1,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":21,"num_read":1451,"num_read_kb":10281,"num_write":126,"num_write_kb":802,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"31'83","reported_seq":173,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T08:52:32.515586+0000","last_change":"2024-05-08T08:52:23.970138+0000","last_active":"2024-05-08T08:52:32.515586+0000","last_peered":"2024-05-08T08:52:32.515586+0000","last_clean":"2024-05-08T08:52:32.515586+0000","last_became_active":"2024-05-08T08:52:23.969732+0000","last_became_peered":"2024-05-08T08:52:23.969732+0000","last_unstale":"2024-05-08T08:52:32.515586+0000","last_undegraded":"2024-05-08T08:52:32.515586+0000","last_fullsized":"2024-05-08T08:52:32.515586+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:44:10.131517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053593,"stat_sum":{"num_bytes":2360832,"num_objects":37,"num_object_clones":5,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":31,"num_read":79,"num_read_kb":1310,"num_write":50,"num_write_kb":906,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"31'102","reported_seq":259,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T08:52:32.005993+0000","last_change":"2024-05-08T08:52:13.944665+0000","last_active":"2024-05-08T08:52:32.005993+0000","last_peered":"2024-05-08T08:52:32.005993+0000","last_clean":"2024-05-08T08:52:32.005993+0000","last_became_active":"2024-05-08T08:52:11.932700+0000","last_became_peered":"2024-05-08T08:52:11.932700+0000","last_unstale":"2024-05-08T08:52:32.005993+0000","last_undegraded":"2024-05-08T08:52:32.005993+0000","last_fullsized":"2024-05-08T08:52:32.005993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:07:30.452157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041230699999999999,"stat_sum":{"num_bytes":4785152,"num_objects":38,"num_object_clones":3,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":33,"num_read":151,"num_read_kb":507,"num_write":68,"num_write_kb":365,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"31'85","reported_seq":3556,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T08:52:32.710756+0000","last_change":"2024-05-08T08:52:13.934541+0000","last_active":"2024-05-08T08:52:32.710756+0000","last_peered":"2024-05-08T08:52:32.710756+0000","last_clean":"2024-05-08T08:52:32.710756+0000","last_became_active":"2024-05-08T08:52:11.931202+0000","last_became_peered":"2024-05-08T08:52:11.931202+0000","last_unstale":"2024-05-08T08:52:32.710756+0000","last_undegraded":"2024-05-08T08:52:32.710756+0000","last_fullsized":"2024-05-08T08:52:32.710756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:22:05.269075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059220300000000002,"stat_sum":{"num_bytes":6540288,"num_objects":37,"num_object_clones":1,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":30,"num_read":3458,"num_read_kb":11403,"num_write":62,"num_write_kb":903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"33'128","reported_seq":1141,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T08:52:32.485722+0000","last_change":"2024-05-08T08:52:13.944894+0000","last_active":"2024-05-08T08:52:32.485722+0000","last_peered":"2024-05-08T08:52:32.485722+0000","last_clean":"2024-05-08T08:52:32.485722+0000","last_became_active":"2024-05-08T08:52:11.930434+0000","last_became_peered":"2024-05-08T08:52:11.930434+0000","last_unstale":"2024-05-08T08:52:32.485722+0000","last_undegraded":"2024-05-08T08:52:32.485722+0000","last_fullsized":"2024-05-08T08:52:32.485722+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:17:45.168077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059972400000000005,"stat_sum":{"num_bytes":3511844,"num_objects":40,"num_object_clones":7,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":27,"num_read":1041,"num_read_kb":6390,"num_write":107,"num_write_kb":1459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":139017707,"num_objects":1090,"num_object_clones":108,"num_object_copies":2180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1090,"num_whiteouts":878,"num_read":77034,"num_read_kb":149657,"num_write":3671,"num_write_kb":47020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":11698176,"data_stored":77029334,"data_compressed":2218746,"data_compressed_allocated":6914048,"data_compressed_original":74014720,"omap_allocated":0,"internal_metadata":0},"log_size":4009,"ondisk_log_size":4009,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":11046,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640265,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36780,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335060,"statfs":{"total":96636764160,"available":96599101440,"internally_reserved":0,"allocated":1478656,"data_stored":6480677,"data_compressed":419938,"data_compressed_allocated":843776,"data_compressed_original":6205440,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640268,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32500,"kb_used_data":1316,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339340,"statfs":{"total":96636764160,"available":96603484160,"internally_reserved":0,"allocated":1347584,"data_stored":7202930,"data_compressed":310279,"data_compressed_allocated":749568,"data_compressed_original":6926336,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640267,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32616,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339224,"statfs":{"total":96636764160,"available":96603365376,"internally_reserved":0,"allocated":1470464,"data_stored":5625663,"data_compressed":387015,"data_compressed_allocated":774144,"data_compressed_original":5283840,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640266,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32356,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339484,"statfs":{"total":96636764160,"available":96603631616,"internally_reserved":0,"allocated":1204224,"data_stored":5575951,"data_compressed":281567,"data_compressed_allocated":614400,"data_compressed_original":5316608,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,10,8],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640265,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32416,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339424,"statfs":{"total":96636764160,"available":96603570176,"internally_reserved":0,"allocated":1265664,"data_stored":7200608,"data_compressed":227181,"data_compressed_allocated":655360,"data_compressed_original":6922240,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640268,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32500,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339340,"statfs":{"total":96636764160,"available":96603484160,"internally_reserved":0,"allocated":1355776,"data_stored":6632750,"data_compressed":339652,"data_compressed_allocated":753664,"data_compressed_original":6348800,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5523,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5523,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3850240,"data_stored":26053793,"data_compressed":466209,"data_compressed_allocated":2015232,"data_compressed_original":24854528,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":921600,"data_stored":7071475,"data_compressed":227181,"data_compressed_allocated":655360,"data_compressed_original":6922240,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":847872,"data_stored":5441885,"data_compressed":281567,"data_compressed_allocated":614400,"data_compressed_original":5316608,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":3952640,"data_stored":25041773,"data_compressed":513572,"data_compressed_allocated":2035712,"data_compressed_original":23789568,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":991232,"data_stored":7068864,"data_compressed":310279,"data_compressed_allocated":749568,"data_compressed_original":6926336,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1134592,"data_stored":6351544,"data_compressed":419938,"data_compressed_allocated":843776,"data_compressed_original":6205440,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T08:52:34.130 INFO:tasks.thrashosds.thrasher:Setting 3.1d to [0, 5] 2024-05-08T08:52:34.130 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.1d', '0', '5'] 2024-05-08T08:52:34.130 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.1d 0 5 2024-05-08T08:52:34.208 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:34.208 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:34.208 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:52:34.211+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:52:34.219 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:52:34.299 INFO:teuthology.orchestra.run.smithi196.stdout:913 read 0x20a4200 thru 0x20a43ce (0x1cf bytes) 2024-05-08T08:52:34.301 INFO:teuthology.orchestra.run.smithi196.stdout:skipping zero length punch hole 2024-05-08T08:52:34.301 INFO:teuthology.orchestra.run.smithi196.stdout:919 read 0x2c80400 thru 0x2c8c35f (0xbf60 bytes) 2024-05-08T08:52:34.304 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:34.308+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:34.308 INFO:teuthology.orchestra.run.smithi196.stdout:920 trunc from 0x4642200 to 0x1009600 2024-05-08T08:52:34.339 INFO:teuthology.orchestra.run.smithi196.stdout:921 trunc from 0x1009600 to 0x792aa00 2024-05-08T08:52:34.347 INFO:teuthology.orchestra.run.smithi196.stdout:922 read 0x6a60200 thru 0x6a61154 (0xf55 bytes) 2024-05-08T08:52:34.348 INFO:teuthology.orchestra.run.smithi196.stdout:926 punch from 0x67da000 to 0x67e2000, (0x8000 bytes) 2024-05-08T08:52:34.361 INFO:teuthology.orchestra.run.smithi196.stdout:927 trunc from 0x792aa00 to 0xe825e00 2024-05-08T08:52:34.368 INFO:teuthology.orchestra.run.smithi196.stdout:928 punch from 0x3513800 to 0x3519000, (0x5800 bytes) 2024-05-08T08:52:34.374 INFO:teuthology.orchestra.run.smithi196.stdout:930 write 0xc752400 thru 0xc75b2fa (0x8efb bytes) 2024-05-08T08:52:34.374 INFO:teuthology.orchestra.run.smithi196.stdout:931 write 0x7eafe00 thru 0x7ebf265 (0xf466 bytes) 2024-05-08T08:52:34.386 INFO:teuthology.orchestra.run.smithi196.stdout:932 trunc from 0xe825e00 to 0x3ef9e00 2024-05-08T08:52:34.406 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:52:34.409+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:52:34.409 INFO:teuthology.orchestra.run.smithi196.stdout:935 read 0x3294400 thru 0x3296b77 (0x2778 bytes) 2024-05-08T08:52:34.410 INFO:teuthology.orchestra.run.smithi196.stdout:936 read 0x95c00 thru 0xa10ac (0xb4ad bytes) 2024-05-08T08:52:34.415 INFO:teuthology.orchestra.run.smithi196.stdout:937 read 0x28fba00 thru 0x2909027 (0xd628 bytes) 2024-05-08T08:52:34.415 INFO:teuthology.orchestra.run.smithi196.stdout:938 punch from 0x2b6ca00 to 0x2b75400, (0x8a00 bytes) 2024-05-08T08:52:34.415 INFO:teuthology.orchestra.run.smithi196.stdout:939 write 0xf9a800 thru 0xfa4bb5 (0xa3b6 bytes) 2024-05-08T08:52:34.416 INFO:teuthology.orchestra.run.smithi196.stdout:940 read 0x5a5400 thru 0x5aaeb3 (0x5ab4 bytes) 2024-05-08T08:52:34.428 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:34.428 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:34.429 INFO:teuthology.orchestra.run.smithi196.stdout:942 trunc from 0x3ef9e00 to 0xe7c0a00 2024-05-08T08:52:34.441 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:52:34.443 INFO:teuthology.orchestra.run.smithi196.stdout:943 write 0x37a5400 thru 0x37afdac (0xa9ad bytes) 2024-05-08T08:52:34.444 INFO:teuthology.orchestra.run.smithi196.stdout:945 punch from 0xd352800 to 0xd360800, (0xe000 bytes) 2024-05-08T08:52:34.460 INFO:teuthology.orchestra.run.smithi196.stdout:947 trunc from 0xe7c0a00 to 0x1b01000 2024-05-08T08:52:34.484 INFO:teuthology.orchestra.run.smithi196.stdout:948 read 0x7a0a00 thru 0x7a3fae (0x35af bytes) 2024-05-08T08:52:34.486 INFO:teuthology.orchestra.run.smithi196.stdout:950 read 0x14b7200 thru 0x14bb65a (0x445b bytes) 2024-05-08T08:52:34.505 INFO:teuthology.orchestra.run.smithi196.stdout:953 trunc from 0x1b01000 to 0xe202c00 2024-05-08T08:52:34.513 INFO:teuthology.orchestra.run.smithi196.stdout:954 write 0x45fcc00 thru 0x45ff5d6 (0x29d7 bytes) 2024-05-08T08:52:34.519 INFO:teuthology.orchestra.run.smithi196.stdout:955 read 0x34df400 thru 0x34ee43c (0xf03d bytes) 2024-05-08T08:52:34.519 INFO:teuthology.orchestra.run.smithi196.stdout:956 read 0xddc800 thru 0xde5484 (0x8c85 bytes) 2024-05-08T08:52:34.520 INFO:teuthology.orchestra.run.smithi196.stdout:958 write 0xd75ec00 thru 0xd76885b (0x9c5c bytes) 2024-05-08T08:52:34.520 INFO:teuthology.orchestra.run.smithi196.stdout:959 read 0xc8c4000 thru 0xc8ca287 (0x6288 bytes) 2024-05-08T08:52:34.520 INFO:teuthology.orchestra.run.smithi196.stdout:960 punch from 0x1426600 to 0x1434c00, (0xe600 bytes) 2024-05-08T08:52:34.525 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:52:34.529+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:52:34.528 INFO:teuthology.orchestra.run.smithi196.stdout:963 trunc from 0xe202c00 to 0x7a25e00 2024-05-08T08:52:34.543 INFO:teuthology.orchestra.run.smithi196.stdout:964 trunc from 0x7a25e00 to 0x9619200 2024-05-08T08:52:34.557 INFO:teuthology.orchestra.run.smithi196.stdout:965 read 0x6359a00 thru 0x6359b92 (0x193 bytes) 2024-05-08T08:52:34.557 INFO:teuthology.orchestra.run.smithi196.stdout:966 write 0x1ebbc00 thru 0x1ec9af7 (0xdef8 bytes) 2024-05-08T08:52:34.562 INFO:teuthology.orchestra.run.smithi196.stdout:967 write 0x1a2ee00 thru 0x1a37a5e (0x8c5f bytes) 2024-05-08T08:52:34.563 INFO:teuthology.orchestra.run.smithi196.stdout:968 trunc from 0x9619200 to 0x8456200 2024-05-08T08:52:34.578 INFO:teuthology.orchestra.run.smithi196.stdout:969 read 0x3c39a00 thru 0x3c47ec1 (0xe4c2 bytes) 2024-05-08T08:52:34.579 INFO:teuthology.orchestra.run.smithi196.stdout:970 read 0x1b46a00 thru 0x1b4b579 (0x4b7a bytes) 2024-05-08T08:52:34.593 INFO:teuthology.orchestra.run.smithi196.stdout:971 write 0x93e8c00 thru 0x93ee033 (0x5434 bytes) 2024-05-08T08:52:34.595 INFO:teuthology.orchestra.run.smithi196.stdout:973 write 0x6791e00 thru 0x679372a (0x192b bytes) 2024-05-08T08:52:34.595 INFO:teuthology.orchestra.run.smithi196.stdout:974 trunc from 0x93ee200 to 0xacff600 2024-05-08T08:52:34.597 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:34.597 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:34.607 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:52:34.608 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:34.610+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:34.608 INFO:teuthology.orchestra.run.smithi196.stdout:975 read 0x9903800 thru 0x990ce54 (0x9655 bytes) 2024-05-08T08:52:34.618 INFO:teuthology.orchestra.run.smithi196.stdout:978 trunc from 0xacff600 to 0x3ae5200 2024-05-08T08:52:34.645 INFO:teuthology.orchestra.run.smithi196.stdout:980 write 0x3fcdc00 thru 0x3fd6b53 (0x8f54 bytes) 2024-05-08T08:52:34.666 INFO:teuthology.orchestra.run.smithi196.stdout:983 write 0xbb6fa00 thru 0xbb6fd35 (0x336 bytes) 2024-05-08T08:52:34.673 INFO:teuthology.orchestra.run.smithi196.stdout:985 trunc from 0xbb6fe00 to 0x6c76800 2024-05-08T08:52:34.690 INFO:teuthology.orchestra.run.smithi196.stdout:986 punch from 0xb55000 to 0xb57c00, (0x2c00 bytes) 2024-05-08T08:52:34.692 INFO:teuthology.orchestra.run.smithi196.stdout:987 write 0x5e3c00 thru 0x5ebdbb (0x81bc bytes) 2024-05-08T08:52:34.694 INFO:teuthology.orchestra.run.smithi196.stdout:990 read 0x287f800 thru 0x288181d (0x201e bytes) 2024-05-08T08:52:34.694 INFO:teuthology.orchestra.run.smithi196.stdout:993 read 0x3552c00 thru 0x3552c9c (0x9d bytes) 2024-05-08T08:52:34.694 INFO:teuthology.orchestra.run.smithi196.stdout:994 read 0x5d85000 thru 0x5d9111c (0xc11d bytes) 2024-05-08T08:52:34.696 INFO:teuthology.orchestra.run.smithi196.stdout:997 trunc from 0x6c76800 to 0x58d3200 2024-05-08T08:52:34.707 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:34.710+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:34.711 INFO:teuthology.orchestra.run.smithi196.stdout:998 read 0x1b9c000 thru 0x1ba7f45 (0xbf46 bytes) 2024-05-08T08:52:34.712 INFO:teuthology.orchestra.run.smithi196.stdout:999 read 0x3acc800 thru 0x3ad3369 (0x6b6a bytes) 2024-05-08T08:52:34.712 INFO:teuthology.orchestra.run.smithi196.stdout:1002 write 0x4626000 thru 0x4634122 (0xe123 bytes) 2024-05-08T08:52:34.713 INFO:teuthology.orchestra.run.smithi196.stdout:1005 punch from 0x380c200 to 0x380f800, (0x3600 bytes) 2024-05-08T08:52:34.713 INFO:teuthology.orchestra.run.smithi196.stdout:1006 write 0x17e2400 thru 0x17f04fa (0xe0fb bytes) 2024-05-08T08:52:34.713 INFO:teuthology.orchestra.run.smithi196.stdout:1007 punch from 0x55e6800 to 0x55f1000, (0xa800 bytes) 2024-05-08T08:52:34.733 INFO:teuthology.orchestra.run.smithi196.stdout:1008 write 0xe1cf400 thru 0xe1d333e (0x3f3f bytes) 2024-05-08T08:52:34.733 INFO:teuthology.orchestra.run.smithi196.stdout:1009 clone 6 order 19 su 4096 sc 10 2024-05-08T08:52:34.766 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:34.766 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:34.775 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:52:34.819 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:52:34.822+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:52:34.885 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:34.885 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:34.893 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:52:34.909 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:34.912+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:35.005 INFO:teuthology.orchestra.run.smithi059.stderr:noscrub is set 2024-05-08T08:52:35.009 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:52:35.012+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:52:35.012 INFO:teuthology.orchestra.run.smithi059.stderr:set 3.1d pg_upmap mapping to [0,5] 2024-05-08T08:52:35.050 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:35.050 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:35.060 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:52:35.111 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:52:35.114+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:52:35.164 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:35.165 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:35.173 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:52:35.212 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:52:35.215+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:52:35.278 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:35.278 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:35.286 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:52:35.297 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T08:52:35.330 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:35.333+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:35.392 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:35.392 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:35.401 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:52:35.422 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:52:35.422 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:52:35.439 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:52:35.440 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:52:35.442 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:52:35.445+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:52:35.456 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:52:35.456 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:52:35.474 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:52:35.474 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:52:35.491 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:52:35.492 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:52:35.509 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:52:35.509 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:52:35.514 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:52:35.517+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:52:35.547 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:52:35.547 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:52:35.547 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:52:35.547 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:52:35.556 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:52:35.615 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:35.618+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:35.661 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:52:35.662 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:52:35.662 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:52:35.662 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:52:35.662 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:52:35.671 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:52:35.755 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:52:35.758+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:52:35.816 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:52:35.819+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:52:35.818 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:52:35.818 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:52:35.818 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:52:35.818 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:52:35.818 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.818 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22069 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 749568~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:35.819 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857624+0000", 2024-05-08T08:52:35.819 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.9631188579999996, 2024-05-08T08:52:35.819 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046969958999999999, 2024-05-08T08:52:35.819 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:35.819 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:35.819 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:35.819 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:35.819 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:35.819 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:35.819 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22069 2024-05-08T08:52:35.819 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.819 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:35.819 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.819 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:35.819 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857624+0000", 2024-05-08T08:52:35.820 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:35.820 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.820 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.820 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:35.820 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857624+0000", 2024-05-08T08:52:35.820 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:35.820 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.820 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.820 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:35.820 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857625+0000", 2024-05-08T08:52:35.820 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.03e-07 2024-05-08T08:52:35.820 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.820 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.820 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:35.820 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857627+0000", 2024-05-08T08:52:35.821 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9369999999999998e-06 2024-05-08T08:52:35.821 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.821 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.821 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:35.821 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857633+0000", 2024-05-08T08:52:35.821 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1229999999999997e-06 2024-05-08T08:52:35.821 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.821 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.821 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:35.821 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857637+0000", 2024-05-08T08:52:35.821 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7820000000000001e-06 2024-05-08T08:52:35.821 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.821 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.821 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:35.821 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904457+0000", 2024-05-08T08:52:35.822 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046819715999999997 2024-05-08T08:52:35.822 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.822 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.822 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:35.822 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904524+0000", 2024-05-08T08:52:35.822 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6371000000000005e-05 2024-05-08T08:52:35.822 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.822 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.822 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:35.822 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904594+0000", 2024-05-08T08:52:35.822 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0627e-05 2024-05-08T08:52:35.822 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:35.822 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:35.822 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:35.823 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.823 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.823 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22071 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 765952~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:35.823 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857643+0000", 2024-05-08T08:52:35.823 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.9630997749999999, 2024-05-08T08:52:35.823 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047106528000000002, 2024-05-08T08:52:35.823 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:35.823 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:35.823 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:35.823 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:35.823 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:35.823 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:35.823 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22071 2024-05-08T08:52:35.823 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.823 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:35.824 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.824 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:35.824 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857643+0000", 2024-05-08T08:52:35.824 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:35.824 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.824 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.824 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:35.824 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857643+0000", 2024-05-08T08:52:35.824 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:35.824 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.824 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.824 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:35.824 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857644+0000", 2024-05-08T08:52:35.824 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2000000000000001e-07 2024-05-08T08:52:35.824 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.825 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.825 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:35.825 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857646+0000", 2024-05-08T08:52:35.825 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0059999999999999e-06 2024-05-08T08:52:35.825 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.825 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.825 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:35.825 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857651+0000", 2024-05-08T08:52:35.825 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.84e-06 2024-05-08T08:52:35.825 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.825 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.825 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:35.825 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857657+0000", 2024-05-08T08:52:35.825 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5620000000000003e-06 2024-05-08T08:52:35.825 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.825 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.826 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:35.826 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904620+0000", 2024-05-08T08:52:35.826 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046962661000000003 2024-05-08T08:52:35.826 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.826 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.826 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:35.826 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904678+0000", 2024-05-08T08:52:35.826 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8656999999999998e-05 2024-05-08T08:52:35.826 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.826 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.826 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:35.826 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904750+0000", 2024-05-08T08:52:35.826 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1482000000000004e-05 2024-05-08T08:52:35.826 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:35.826 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:35.827 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:35.827 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.827 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.827 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22073 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 782336~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:35.827 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:52:35.827 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.9630757670000003, 2024-05-08T08:52:35.827 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047244234000000003, 2024-05-08T08:52:35.827 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:35.827 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:35.827 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:35.827 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:35.827 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:35.827 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:35.827 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22073 2024-05-08T08:52:35.827 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.828 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:35.828 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.828 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:35.828 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:52:35.828 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:35.828 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.828 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.828 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:35.828 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:52:35.828 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:35.828 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.828 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.828 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:35.828 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857668+0000", 2024-05-08T08:52:35.828 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7800000000000002e-07 2024-05-08T08:52:35.829 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.829 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.829 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:35.829 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857670+0000", 2024-05-08T08:52:35.829 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0820000000000001e-06 2024-05-08T08:52:35.829 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.829 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.829 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:35.829 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857678+0000", 2024-05-08T08:52:35.829 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7589999999999997e-06 2024-05-08T08:52:35.829 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.829 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.829 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:35.829 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857691+0000", 2024-05-08T08:52:35.829 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3672999999999999e-05 2024-05-08T08:52:35.829 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.830 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.830 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:35.830 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904780+0000", 2024-05-08T08:52:35.830 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047088616999999999 2024-05-08T08:52:35.830 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.830 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.830 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:35.830 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904840+0000", 2024-05-08T08:52:35.830 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0356999999999998e-05 2024-05-08T08:52:35.830 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.830 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.830 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:35.830 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904912+0000", 2024-05-08T08:52:35.830 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1367999999999998e-05 2024-05-08T08:52:35.830 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:35.831 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:35.831 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:35.831 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.831 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.831 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22075 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 798720~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:35.831 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:52:35.831 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.9630453699999997, 2024-05-08T08:52:35.831 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047384643999999997, 2024-05-08T08:52:35.831 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:35.831 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:35.831 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:35.831 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:35.831 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:35.831 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:35.831 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22075 2024-05-08T08:52:35.831 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.832 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:35.832 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.832 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:35.832 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:52:35.832 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:35.832 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.832 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.832 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:35.832 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:52:35.832 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:35.832 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.832 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.832 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:35.832 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:52:35.833 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7399999999999999e-07 2024-05-08T08:52:35.833 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.833 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.833 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:35.833 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857705+0000", 2024-05-08T08:52:35.833 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1980000000000004e-06 2024-05-08T08:52:35.833 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.833 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.833 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:35.833 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857713+0000", 2024-05-08T08:52:35.833 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7039999999999994e-06 2024-05-08T08:52:35.833 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.833 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.833 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:35.833 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857718+0000", 2024-05-08T08:52:35.834 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.865e-06 2024-05-08T08:52:35.834 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.834 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.834 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:35.834 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904941+0000", 2024-05-08T08:52:35.834 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047223198000000001 2024-05-08T08:52:35.834 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.834 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.834 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:35.834 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904999+0000", 2024-05-08T08:52:35.834 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8254999999999997e-05 2024-05-08T08:52:35.834 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.834 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.834 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:35.834 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905082+0000", 2024-05-08T08:52:35.835 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3049999999999999e-05 2024-05-08T08:52:35.835 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:35.835 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:35.835 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:35.835 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.835 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.835 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22077 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 815104~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:35.835 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:52:35.835 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.9630073589999997, 2024-05-08T08:52:35.835 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047519448999999998, 2024-05-08T08:52:35.835 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:35.835 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:35.835 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:35.835 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:35.836 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:35.836 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:35.836 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22077 2024-05-08T08:52:35.836 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.836 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:35.836 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.836 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:35.836 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:52:35.836 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:35.836 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.836 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.836 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:35.836 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:52:35.836 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:35.836 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.837 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.837 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:35.837 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:52:35.837 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.99e-07 2024-05-08T08:52:35.837 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.837 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.837 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:35.837 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857739+0000", 2024-05-08T08:52:35.837 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3599999999999999e-06 2024-05-08T08:52:35.837 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.837 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.837 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:35.837 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857745+0000", 2024-05-08T08:52:35.837 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5139999999999998e-06 2024-05-08T08:52:35.837 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.837 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.838 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:35.838 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857750+0000", 2024-05-08T08:52:35.838 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8550000000000001e-06 2024-05-08T08:52:35.838 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.838 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.838 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:35.838 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905110+0000", 2024-05-08T08:52:35.838 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047360126000000002 2024-05-08T08:52:35.838 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.838 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.838 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:35.838 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905175+0000", 2024-05-08T08:52:35.838 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4689000000000004e-05 2024-05-08T08:52:35.838 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.838 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.839 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:35.839 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905255+0000", 2024-05-08T08:52:35.839 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0306e-05 2024-05-08T08:52:35.839 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:35.839 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:35.839 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:35.839 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.839 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.839 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22079 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 831488~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:35.839 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:52:35.839 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.9629868110000004, 2024-05-08T08:52:35.839 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.04767499, 2024-05-08T08:52:35.839 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:35.839 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:35.839 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:35.839 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:35.840 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:35.840 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:35.840 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22079 2024-05-08T08:52:35.840 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.840 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:35.840 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.840 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:35.840 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:52:35.840 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:35.840 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.840 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.840 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:35.840 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:52:35.840 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:35.840 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.841 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.841 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:35.841 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857757+0000", 2024-05-08T08:52:35.841 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8299999999999997e-07 2024-05-08T08:52:35.841 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.841 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.841 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:35.841 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857759+0000", 2024-05-08T08:52:35.841 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1390000000000002e-06 2024-05-08T08:52:35.841 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.841 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.841 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:35.841 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857765+0000", 2024-05-08T08:52:35.841 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4609999999999996e-06 2024-05-08T08:52:35.841 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.842 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.842 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:35.842 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857770+0000", 2024-05-08T08:52:35.842 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6689999999999997e-06 2024-05-08T08:52:35.842 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.842 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.842 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:35.842 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905281+0000", 2024-05-08T08:52:35.842 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047510668999999998 2024-05-08T08:52:35.842 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.842 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.842 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:35.842 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905344+0000", 2024-05-08T08:52:35.842 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2797000000000002e-05 2024-05-08T08:52:35.842 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.843 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.843 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:35.843 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905431+0000", 2024-05-08T08:52:35.843 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7771999999999999e-05 2024-05-08T08:52:35.843 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:35.843 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:35.843 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:35.843 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.843 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.843 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22082 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 847872~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:35.843 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:52:35.843 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.9629671200000001, 2024-05-08T08:52:35.843 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047831934999999999, 2024-05-08T08:52:35.843 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:35.843 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:35.844 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:35.844 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:35.844 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:35.844 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:35.844 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22082 2024-05-08T08:52:35.844 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.844 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:35.844 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.844 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:35.844 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:52:35.844 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:35.844 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.844 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.844 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:35.844 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:52:35.844 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:35.845 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.845 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.845 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:35.845 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:52:35.845 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:52:35.845 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.845 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.845 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:35.845 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857778+0000", 2024-05-08T08:52:35.845 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0789999999999999e-06 2024-05-08T08:52:35.845 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.845 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.845 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:35.845 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857784+0000", 2024-05-08T08:52:35.845 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0480000000000004e-06 2024-05-08T08:52:35.846 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.846 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.846 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:35.846 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857790+0000", 2024-05-08T08:52:35.846 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2719999999999997e-06 2024-05-08T08:52:35.846 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.846 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.846 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:35.846 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905458+0000", 2024-05-08T08:52:35.846 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047668470999999997 2024-05-08T08:52:35.846 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.846 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.846 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:35.846 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905521+0000", 2024-05-08T08:52:35.846 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2952000000000003e-05 2024-05-08T08:52:35.847 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.847 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.847 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:35.847 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905608+0000", 2024-05-08T08:52:35.847 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6759000000000001e-05 2024-05-08T08:52:35.847 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:35.847 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:35.847 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:35.847 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.847 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.847 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22084 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 864256~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:35.847 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:52:35.847 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.96294317, 2024-05-08T08:52:35.847 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047977748000000001, 2024-05-08T08:52:35.847 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:35.847 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:35.848 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:35.848 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:35.848 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:35.848 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:35.848 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22084 2024-05-08T08:52:35.848 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.848 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:35.848 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.848 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:35.848 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:52:35.848 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:35.848 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.848 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.848 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:35.848 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:52:35.849 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:35.849 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.849 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.849 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:35.849 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:52:35.849 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:52:35.849 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.849 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.849 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:35.849 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857802+0000", 2024-05-08T08:52:35.849 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0779999999999998e-06 2024-05-08T08:52:35.849 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.849 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.849 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:35.849 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857812+0000", 2024-05-08T08:52:35.850 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7939999999999997e-06 2024-05-08T08:52:35.850 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.850 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.850 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:35.850 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857817+0000", 2024-05-08T08:52:35.850 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8899999999999998e-06 2024-05-08T08:52:35.850 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.850 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.850 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:35.850 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905636+0000", 2024-05-08T08:52:35.850 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047819252 2024-05-08T08:52:35.850 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.850 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.850 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:35.851 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905700+0000", 2024-05-08T08:52:35.851 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4016999999999994e-05 2024-05-08T08:52:35.851 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.851 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.851 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:35.851 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905778+0000", 2024-05-08T08:52:35.851 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7306999999999997e-05 2024-05-08T08:52:35.851 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:35.851 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:35.851 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:35.851 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.851 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.851 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22086 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 880640~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:35.851 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:52:35.851 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.9629198199999998, 2024-05-08T08:52:35.852 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048119149, 2024-05-08T08:52:35.852 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:35.852 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:35.852 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:35.852 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:35.852 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:35.852 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:35.852 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22086 2024-05-08T08:52:35.852 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.852 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:35.852 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.852 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:35.852 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:52:35.852 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:35.852 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.852 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.853 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:35.853 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:52:35.853 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:35.853 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.853 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.853 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:35.853 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857824+0000", 2024-05-08T08:52:35.853 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4200000000000002e-07 2024-05-08T08:52:35.853 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.853 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.853 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:35.853 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857826+0000", 2024-05-08T08:52:35.853 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0590000000000001e-06 2024-05-08T08:52:35.853 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.854 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.854 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:35.854 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857832+0000", 2024-05-08T08:52:35.854 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8029999999999999e-06 2024-05-08T08:52:35.854 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.854 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.854 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:35.854 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857837+0000", 2024-05-08T08:52:35.854 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7929999999999997e-06 2024-05-08T08:52:35.854 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.854 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.854 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:35.854 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905803+0000", 2024-05-08T08:52:35.854 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047965761000000003 2024-05-08T08:52:35.855 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.855 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.855 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:35.855 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905862+0000", 2024-05-08T08:52:35.855 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9203000000000002e-05 2024-05-08T08:52:35.855 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.855 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.855 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:35.855 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905942+0000", 2024-05-08T08:52:35.855 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0187999999999999e-05 2024-05-08T08:52:35.855 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:35.855 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:35.855 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:35.855 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.855 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.856 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22088 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 897024~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:35.856 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:52:35.856 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.9628997679999998, 2024-05-08T08:52:35.856 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048256049000000002, 2024-05-08T08:52:35.856 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:35.856 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:35.856 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:35.856 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:35.856 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:35.856 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:35.856 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22088 2024-05-08T08:52:35.856 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.856 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:35.856 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.856 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:35.857 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:52:35.857 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:35.857 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.857 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.857 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:35.857 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:52:35.857 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:35.857 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.857 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.857 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:35.857 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857844+0000", 2024-05-08T08:52:35.857 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.46e-07 2024-05-08T08:52:35.857 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.857 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.857 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:35.858 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857846+0000", 2024-05-08T08:52:35.858 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0899999999999999e-06 2024-05-08T08:52:35.858 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.858 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.858 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:35.858 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857855+0000", 2024-05-08T08:52:35.858 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8219999999999996e-06 2024-05-08T08:52:35.858 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.858 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.858 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:35.858 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857860+0000", 2024-05-08T08:52:35.858 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2989999999999996e-06 2024-05-08T08:52:35.858 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.858 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.859 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:35.859 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905966+0000", 2024-05-08T08:52:35.859 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048106542000000002 2024-05-08T08:52:35.859 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.859 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.859 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:35.859 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906024+0000", 2024-05-08T08:52:35.859 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7958000000000003e-05 2024-05-08T08:52:35.859 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.859 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.859 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:35.859 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906099+0000", 2024-05-08T08:52:35.859 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4992000000000003e-05 2024-05-08T08:52:35.859 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:35.859 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:35.859 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:35.860 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.860 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.860 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22091 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 913408~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:35.860 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:52:35.860 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.9628760080000003, 2024-05-08T08:52:35.860 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048395290000000001, 2024-05-08T08:52:35.860 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:35.860 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:35.860 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:35.860 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:35.860 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:35.860 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:35.860 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22091 2024-05-08T08:52:35.860 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.861 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:35.861 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.861 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:35.861 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:52:35.861 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:35.861 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.861 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.861 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:35.861 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:52:35.861 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:35.861 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.861 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.861 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:35.861 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:52:35.861 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4900000000000001e-07 2024-05-08T08:52:35.861 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.862 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.862 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:35.862 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857870+0000", 2024-05-08T08:52:35.862 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2510000000000001e-06 2024-05-08T08:52:35.862 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.862 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.862 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:35.862 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857877+0000", 2024-05-08T08:52:35.862 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0809999999999996e-06 2024-05-08T08:52:35.862 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.862 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.862 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:35.862 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857881+0000", 2024-05-08T08:52:35.862 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.724e-06 2024-05-08T08:52:35.863 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.863 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.863 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:35.863 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906127+0000", 2024-05-08T08:52:35.863 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048245345000000002 2024-05-08T08:52:35.863 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.863 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.863 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:35.863 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906190+0000", 2024-05-08T08:52:35.863 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2762000000000007e-05 2024-05-08T08:52:35.863 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.863 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.863 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:35.863 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906262+0000", 2024-05-08T08:52:35.863 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2778e-05 2024-05-08T08:52:35.864 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:35.864 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:35.864 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:35.864 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.864 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.864 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22093 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 929792~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:35.864 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:52:35.864 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.962853644, 2024-05-08T08:52:35.864 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048532875000000003, 2024-05-08T08:52:35.864 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:35.864 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:35.864 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:35.864 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:35.864 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:35.864 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:35.865 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22093 2024-05-08T08:52:35.865 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.865 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:35.865 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.865 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:35.865 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:52:35.865 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:35.865 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.865 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.865 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:35.865 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:52:35.865 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:35.865 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.865 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.865 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:35.865 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857890+0000", 2024-05-08T08:52:35.866 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-07 2024-05-08T08:52:35.866 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.866 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.866 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:35.866 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857899+0000", 2024-05-08T08:52:35.866 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6889999999999999e-06 2024-05-08T08:52:35.866 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.866 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.866 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:35.866 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857906+0000", 2024-05-08T08:52:35.866 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6490000000000002e-06 2024-05-08T08:52:35.866 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.866 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.866 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:35.866 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857917+0000", 2024-05-08T08:52:35.867 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0764999999999999e-05 2024-05-08T08:52:35.867 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.867 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.867 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:35.867 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906287+0000", 2024-05-08T08:52:35.867 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048370442 2024-05-08T08:52:35.867 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.867 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.867 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:35.867 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906349+0000", 2024-05-08T08:52:35.867 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1537000000000004e-05 2024-05-08T08:52:35.867 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.867 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.867 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:35.867 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906422+0000", 2024-05-08T08:52:35.868 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3431999999999998e-05 2024-05-08T08:52:35.868 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:35.868 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:35.868 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:35.868 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.868 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.868 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22095 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 946176~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:35.868 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:52:35.868 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.96281894, 2024-05-08T08:52:35.868 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048657593999999998, 2024-05-08T08:52:35.868 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:35.868 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:35.868 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:35.868 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:35.868 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:35.868 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:35.869 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22095 2024-05-08T08:52:35.869 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.869 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:35.869 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.869 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:35.869 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:52:35.869 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:35.869 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.869 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.869 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:35.869 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:52:35.869 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:35.869 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.869 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.869 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:35.870 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857925+0000", 2024-05-08T08:52:35.870 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3099999999999998e-07 2024-05-08T08:52:35.870 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.870 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.870 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:35.870 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857927+0000", 2024-05-08T08:52:35.870 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.34e-06 2024-05-08T08:52:35.870 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.870 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.870 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:35.870 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857938+0000", 2024-05-08T08:52:35.870 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1426e-05 2024-05-08T08:52:35.870 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.870 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.871 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:35.871 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857943+0000", 2024-05-08T08:52:35.871 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5140000000000003e-06 2024-05-08T08:52:35.871 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.871 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.871 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:35.871 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906448+0000", 2024-05-08T08:52:35.871 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048504641000000001 2024-05-08T08:52:35.871 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.871 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.871 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:35.871 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906510+0000", 2024-05-08T08:52:35.871 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2663000000000006e-05 2024-05-08T08:52:35.871 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.871 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.872 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:35.872 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906582+0000", 2024-05-08T08:52:35.872 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1478999999999998e-05 2024-05-08T08:52:35.872 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:35.872 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:35.872 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:35.872 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.872 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.872 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22097 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 962560~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:35.872 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:52:35.872 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.9627912509999996, 2024-05-08T08:52:35.872 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048791353000000003, 2024-05-08T08:52:35.872 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:35.872 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:35.872 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:35.873 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:35.873 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:35.873 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:35.873 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22097 2024-05-08T08:52:35.873 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.873 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:35.873 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.873 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:35.873 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:52:35.873 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:35.873 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.873 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.873 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:35.873 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:52:35.873 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:35.874 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.874 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.874 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:35.874 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:52:35.874 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9200000000000002e-07 2024-05-08T08:52:35.874 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.874 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.874 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:35.874 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857954+0000", 2024-05-08T08:52:35.874 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1900000000000002e-06 2024-05-08T08:52:35.874 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.874 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.874 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:35.874 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857961+0000", 2024-05-08T08:52:35.874 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4989999999999999e-06 2024-05-08T08:52:35.875 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.875 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.875 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:35.875 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857971+0000", 2024-05-08T08:52:35.875 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0485999999999999e-05 2024-05-08T08:52:35.875 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.875 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.875 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:35.875 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906607+0000", 2024-05-08T08:52:35.875 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048635109000000003 2024-05-08T08:52:35.875 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.875 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.875 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:35.875 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906667+0000", 2024-05-08T08:52:35.875 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0439000000000002e-05 2024-05-08T08:52:35.875 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.876 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.876 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:35.876 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906743+0000", 2024-05-08T08:52:35.876 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6237999999999998e-05 2024-05-08T08:52:35.876 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:35.876 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:35.876 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:35.876 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.876 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.876 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22099 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 978944~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:35.876 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:52:35.876 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.9627650909999996, 2024-05-08T08:52:35.876 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048932055000000002, 2024-05-08T08:52:35.876 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:35.876 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:35.877 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:35.877 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:35.877 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:35.877 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:35.877 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22099 2024-05-08T08:52:35.877 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.877 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:35.877 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.877 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:35.877 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:52:35.877 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:35.877 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.877 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.877 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:35.877 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:52:35.877 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:35.878 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.878 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.878 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:35.878 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:52:35.878 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.27e-07 2024-05-08T08:52:35.878 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.878 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.878 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:35.878 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857987+0000", 2024-05-08T08:52:35.878 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5890000000000003e-06 2024-05-08T08:52:35.878 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.878 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.878 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:35.878 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857994+0000", 2024-05-08T08:52:35.878 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6619999999999999e-06 2024-05-08T08:52:35.878 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.879 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.879 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:35.879 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857998+0000", 2024-05-08T08:52:35.879 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7110000000000003e-06 2024-05-08T08:52:35.879 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.879 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.879 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:35.879 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906772+0000", 2024-05-08T08:52:35.879 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048773415000000001 2024-05-08T08:52:35.879 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.879 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.879 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:35.879 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906834+0000", 2024-05-08T08:52:35.879 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1752000000000001e-05 2024-05-08T08:52:35.879 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.880 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.880 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:35.880 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906910+0000", 2024-05-08T08:52:35.880 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6499000000000001e-05 2024-05-08T08:52:35.880 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:35.880 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:35.880 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:35.880 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.880 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.880 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22101 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 995328~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:35.880 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:52:35.880 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.9627379600000001, 2024-05-08T08:52:35.880 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049087364000000001, 2024-05-08T08:52:35.880 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:35.880 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:35.880 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:35.880 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:35.881 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:35.881 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:35.881 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22101 2024-05-08T08:52:35.881 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.881 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:35.881 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.881 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:35.881 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:52:35.881 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:35.881 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.881 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.881 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:35.881 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:52:35.881 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:35.881 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.882 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.882 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:35.882 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858006+0000", 2024-05-08T08:52:35.882 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1399999999999997e-07 2024-05-08T08:52:35.882 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.882 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.882 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:35.882 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858008+0000", 2024-05-08T08:52:35.882 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2060000000000001e-06 2024-05-08T08:52:35.882 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.882 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.882 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:35.882 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858017+0000", 2024-05-08T08:52:35.882 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1579999999999994e-06 2024-05-08T08:52:35.883 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.883 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.883 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:35.883 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858022+0000", 2024-05-08T08:52:35.883 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4199999999999998e-06 2024-05-08T08:52:35.883 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.883 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.883 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:35.883 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906940+0000", 2024-05-08T08:52:35.883 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048917627999999998 2024-05-08T08:52:35.883 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.883 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.883 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:35.883 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907002+0000", 2024-05-08T08:52:35.883 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2399999999999999e-05 2024-05-08T08:52:35.884 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.884 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.884 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:35.884 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907092+0000", 2024-05-08T08:52:35.884 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0137999999999997e-05 2024-05-08T08:52:35.884 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:35.884 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:35.884 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:35.884 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.884 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.884 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22103 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1011712~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:35.884 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:52:35.884 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.9627131750000002, 2024-05-08T08:52:35.884 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049228542, 2024-05-08T08:52:35.885 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:35.885 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:35.885 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:35.885 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:35.885 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:35.885 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:35.885 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22103 2024-05-08T08:52:35.885 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.885 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:35.885 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.885 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:35.885 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:52:35.885 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:35.885 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.886 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.886 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:35.886 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:52:35.886 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:35.886 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.886 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.886 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:35.886 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:52:35.886 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4299999999999999e-07 2024-05-08T08:52:35.886 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.886 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.886 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:35.886 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858032+0000", 2024-05-08T08:52:35.886 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0590000000000001e-06 2024-05-08T08:52:35.886 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.887 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.887 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:35.887 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858037+0000", 2024-05-08T08:52:35.887 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.977e-06 2024-05-08T08:52:35.887 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.887 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.887 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:35.887 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858042+0000", 2024-05-08T08:52:35.887 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7650000000000001e-06 2024-05-08T08:52:35.887 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.887 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.887 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:35.887 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907118+0000", 2024-05-08T08:52:35.887 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049076147000000001 2024-05-08T08:52:35.887 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.888 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.888 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:35.888 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907180+0000", 2024-05-08T08:52:35.888 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1971999999999996e-05 2024-05-08T08:52:35.888 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.888 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.888 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:35.888 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907258+0000", 2024-05-08T08:52:35.888 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8279e-05 2024-05-08T08:52:35.888 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:35.888 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:35.888 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:35.888 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.888 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.889 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22105 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1028096~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:35.889 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:52:35.889 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.9626939319999996, 2024-05-08T08:52:35.889 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049394609999999999, 2024-05-08T08:52:35.889 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:35.889 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:35.889 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:35.889 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:35.889 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:35.889 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:35.889 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22105 2024-05-08T08:52:35.889 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.889 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:35.889 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.889 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:35.889 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:52:35.889 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:35.890 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.890 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.890 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:35.890 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:52:35.890 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:35.890 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.890 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.890 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:35.890 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858050+0000", 2024-05-08T08:52:35.890 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5999999999999999e-07 2024-05-08T08:52:35.890 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.890 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.890 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:35.890 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858051+0000", 2024-05-08T08:52:35.891 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8870000000000001e-06 2024-05-08T08:52:35.891 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.891 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.891 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:35.891 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858057+0000", 2024-05-08T08:52:35.891 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8019999999999997e-06 2024-05-08T08:52:35.891 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.891 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.891 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:35.891 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858062+0000", 2024-05-08T08:52:35.891 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.561e-06 2024-05-08T08:52:35.891 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.891 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.891 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:35.891 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907289+0000", 2024-05-08T08:52:35.892 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049226957000000002 2024-05-08T08:52:35.892 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.892 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.892 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:35.892 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907356+0000", 2024-05-08T08:52:35.892 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6915000000000006e-05 2024-05-08T08:52:35.892 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.892 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.892 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:35.892 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907444+0000", 2024-05-08T08:52:35.892 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8127999999999994e-05 2024-05-08T08:52:35.892 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:35.892 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:35.892 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:35.893 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.893 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.893 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22107 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1044480~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:35.893 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:52:35.893 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.9626755920000001, 2024-05-08T08:52:35.893 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049799942999999999, 2024-05-08T08:52:35.893 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:35.893 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:35.893 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:35.893 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:35.893 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:35.893 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:35.893 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22107 2024-05-08T08:52:35.893 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.894 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:35.894 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.894 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:35.894 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:52:35.894 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:35.894 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.894 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.894 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:35.894 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:52:35.894 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:35.894 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.894 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.894 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:35.894 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:52:35.894 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4799999999999999e-07 2024-05-08T08:52:35.894 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.895 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.895 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:35.895 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858070+0000", 2024-05-08T08:52:35.895 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.012e-06 2024-05-08T08:52:35.895 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.895 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.895 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:35.895 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858077+0000", 2024-05-08T08:52:35.895 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6290000000000001e-06 2024-05-08T08:52:35.895 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.895 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.895 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:35.895 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858083+0000", 2024-05-08T08:52:35.895 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8880000000000002e-06 2024-05-08T08:52:35.896 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.896 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.896 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:35.896 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907671+0000", 2024-05-08T08:52:35.896 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049587191000000003 2024-05-08T08:52:35.896 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.896 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.896 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:35.896 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907763+0000", 2024-05-08T08:52:35.896 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2231000000000005e-05 2024-05-08T08:52:35.896 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.896 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.896 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:35.896 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907867+0000", 2024-05-08T08:52:35.896 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104644 2024-05-08T08:52:35.896 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:35.897 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:35.897 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:35.897 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.897 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.897 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22294 3.1c 3:3aad0389:::rbd_data.11406886b07b.0000000000000080:head [stat,delete] snapc 7=[7] RETRY=6 ondisk+retry+write+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:35.897 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:52:35.897 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.9620817439999998, 2024-05-08T08:52:35.897 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051152191, 2024-05-08T08:52:35.897 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:35.897 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:35.897 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:35.897 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:35.897 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:35.897 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:35.898 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22294 2024-05-08T08:52:35.898 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.898 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:35.898 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.898 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:35.898 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:52:35.898 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:35.898 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.898 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.898 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:35.898 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:52:35.898 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:35.898 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.898 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.898 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:35.899 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858662+0000", 2024-05-08T08:52:35.899 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4799999999999998e-07 2024-05-08T08:52:35.899 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.899 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.899 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:35.899 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858664+0000", 2024-05-08T08:52:35.899 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1229999999999998e-06 2024-05-08T08:52:35.899 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.899 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.899 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:35.899 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858670+0000", 2024-05-08T08:52:35.899 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3099999999999997e-06 2024-05-08T08:52:35.899 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.899 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.899 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:35.900 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858676+0000", 2024-05-08T08:52:35.900 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9750000000000004e-06 2024-05-08T08:52:35.900 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.900 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.900 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:35.900 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907901+0000", 2024-05-08T08:52:35.900 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049224520000000001 2024-05-08T08:52:35.900 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.900 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.900 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:35.900 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.908085+0000", 2024-05-08T08:52:35.900 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00018377500000000001 2024-05-08T08:52:35.900 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:35.900 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:35.900 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:35.900 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.909814+0000", 2024-05-08T08:52:35.901 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00172894 2024-05-08T08:52:35.901 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:35.901 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:35.901 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:35.901 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:35.901 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:35.901 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:52:35.902 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:52:35.944 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:52:35.947+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:52:36.018 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:36.021+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:36.054 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:52:36.055 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:52:36.055 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:52:36.055 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:52:36.061 INFO:teuthology.orchestra.run.smithi196.stdout:truncating image image_client.0-clone5 from 0xe1d3400 (overlap 0x1009600) to 0x0 2024-05-08T08:52:36.065 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:52:36.150 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:36.153+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:36.213 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:52:36.214 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:52:36.214 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:52:36.214 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:52:36.214 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:52:36.219 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:36.222+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:36.222 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:52:36.320 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:52:36.324+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:52:36.376 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:52:36.376 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:52:36.376 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:52:36.376 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:52:36.376 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.376 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6690 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1093632~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:36.376 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:52:36.376 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.6631154869999998, 2024-05-08T08:52:36.376 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052462754, 2024-05-08T08:52:36.377 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:36.377 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:36.377 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:36.377 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:36.377 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:36.377 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:36.377 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6690 2024-05-08T08:52:36.377 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.377 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:36.377 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.377 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:36.377 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:52:36.377 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:36.377 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.377 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.378 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:36.378 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:52:36.378 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:36.378 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.378 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.378 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:36.378 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715346+0000", 2024-05-08T08:52:36.378 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.84e-07 2024-05-08T08:52:36.378 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.378 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.378 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:36.378 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715348+0000", 2024-05-08T08:52:36.378 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.114e-06 2024-05-08T08:52:36.378 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.378 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.379 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:36.379 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715353+0000", 2024-05-08T08:52:36.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5720000000000002e-06 2024-05-08T08:52:36.379 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.379 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.379 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.379 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715358+0000", 2024-05-08T08:52:36.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7140000000000001e-06 2024-05-08T08:52:36.379 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.379 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.379 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:36.379 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767638+0000", 2024-05-08T08:52:36.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052279562000000002 2024-05-08T08:52:36.379 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.380 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.380 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:36.380 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767713+0000", 2024-05-08T08:52:36.380 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5652e-05 2024-05-08T08:52:36.380 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.380 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.380 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:36.380 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767808+0000", 2024-05-08T08:52:36.380 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4755999999999999e-05 2024-05-08T08:52:36.380 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:36.380 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:36.380 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:36.380 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.380 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.380 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6692 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1110016~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:36.380 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:52:36.381 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.6630963970000003, 2024-05-08T08:52:36.381 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052666495000000001, 2024-05-08T08:52:36.381 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:36.381 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:36.381 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:36.381 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:36.381 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:36.381 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:36.381 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6692 2024-05-08T08:52:36.381 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.381 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:36.381 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.381 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:36.381 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:52:36.382 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:36.382 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.382 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.382 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:36.382 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:52:36.382 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:36.382 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.382 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.382 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:36.382 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:52:36.382 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8599999999999999e-07 2024-05-08T08:52:36.382 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.382 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.382 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:36.382 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715372+0000", 2024-05-08T08:52:36.383 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0990000000000001e-06 2024-05-08T08:52:36.383 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.383 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.383 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:36.383 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715378+0000", 2024-05-08T08:52:36.383 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0680000000000002e-06 2024-05-08T08:52:36.383 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.383 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.383 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.383 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715383+0000", 2024-05-08T08:52:36.383 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8740000000000003e-06 2024-05-08T08:52:36.383 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.383 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.383 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:36.383 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767848+0000", 2024-05-08T08:52:36.384 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052464668999999998 2024-05-08T08:52:36.384 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.384 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:36.384 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767927+0000", 2024-05-08T08:52:36.384 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9228999999999996e-05 2024-05-08T08:52:36.384 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.384 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:36.384 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768031+0000", 2024-05-08T08:52:36.384 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010417 2024-05-08T08:52:36.384 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:36.384 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:36.384 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:36.384 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.385 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6694 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1126400~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:36.385 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:52:36.385 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.6630715, 2024-05-08T08:52:36.385 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052848829, 2024-05-08T08:52:36.385 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:36.385 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:36.385 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:36.385 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:36.385 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:36.385 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:36.385 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6694 2024-05-08T08:52:36.385 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.385 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:36.385 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.385 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:36.385 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:52:36.386 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:36.386 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.386 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.386 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:36.386 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:52:36.386 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:36.386 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.386 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.386 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:36.386 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715390+0000", 2024-05-08T08:52:36.386 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9900000000000001e-07 2024-05-08T08:52:36.386 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.386 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.386 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:36.387 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715392+0000", 2024-05-08T08:52:36.387 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8279999999999999e-06 2024-05-08T08:52:36.387 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.387 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.387 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:36.387 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715398+0000", 2024-05-08T08:52:36.387 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7619999999999998e-06 2024-05-08T08:52:36.387 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.387 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.387 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.387 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715403+0000", 2024-05-08T08:52:36.387 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7790000000000003e-06 2024-05-08T08:52:36.387 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.387 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.387 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:36.387 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768067+0000", 2024-05-08T08:52:36.388 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052663785999999997 2024-05-08T08:52:36.388 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.388 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.388 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:36.388 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768142+0000", 2024-05-08T08:52:36.388 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5018999999999994e-05 2024-05-08T08:52:36.388 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.388 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.388 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:36.388 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768238+0000", 2024-05-08T08:52:36.388 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6255999999999995e-05 2024-05-08T08:52:36.388 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:36.388 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:36.388 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:36.388 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.388 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.389 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6696 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1142784~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:36.389 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:52:36.389 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.6630519919999998, 2024-05-08T08:52:36.389 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.053051057999999998, 2024-05-08T08:52:36.389 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:36.389 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:36.389 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:36.389 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:36.389 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:36.389 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:36.389 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6696 2024-05-08T08:52:36.389 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.389 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:36.389 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.390 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:36.390 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:52:36.390 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:36.390 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.390 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.390 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:36.390 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:52:36.390 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:36.390 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.390 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.390 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:36.390 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:52:36.390 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-05-08T08:52:36.390 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.391 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.391 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:36.391 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715411+0000", 2024-05-08T08:52:36.391 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9630000000000001e-06 2024-05-08T08:52:36.391 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.391 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.391 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:36.391 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715417+0000", 2024-05-08T08:52:36.391 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7250000000000002e-06 2024-05-08T08:52:36.391 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.391 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.391 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.391 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715421+0000", 2024-05-08T08:52:36.391 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4089999999999996e-06 2024-05-08T08:52:36.391 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.392 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.392 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:36.392 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768277+0000", 2024-05-08T08:52:36.392 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052855293999999997 2024-05-08T08:52:36.392 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.392 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.392 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:36.392 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768358+0000", 2024-05-08T08:52:36.392 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1720999999999999e-05 2024-05-08T08:52:36.392 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.392 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.392 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:36.392 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768460+0000", 2024-05-08T08:52:36.392 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010154600000000001 2024-05-08T08:52:36.392 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:36.393 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:36.393 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:36.393 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.393 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.393 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6698 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1159168~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:36.393 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:52:36.393 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.6630331250000001, 2024-05-08T08:52:36.393 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.053245726, 2024-05-08T08:52:36.393 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:36.393 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:36.393 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:36.393 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:36.393 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:36.393 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:36.393 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6698 2024-05-08T08:52:36.394 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.394 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:36.394 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.394 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:36.394 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:52:36.394 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:36.394 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.394 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.394 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:36.394 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:52:36.394 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:36.394 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.394 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.394 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:36.395 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716936+0000", 2024-05-08T08:52:36.395 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001508665 2024-05-08T08:52:36.395 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.395 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.395 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:36.395 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716941+0000", 2024-05-08T08:52:36.395 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1910000000000002e-06 2024-05-08T08:52:36.395 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.395 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.395 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:36.395 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716950+0000", 2024-05-08T08:52:36.395 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1900000000000001e-06 2024-05-08T08:52:36.395 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.395 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.395 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.395 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716958+0000", 2024-05-08T08:52:36.396 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8019999999999992e-06 2024-05-08T08:52:36.396 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.396 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.396 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:36.396 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768499+0000", 2024-05-08T08:52:36.396 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051540926000000001 2024-05-08T08:52:36.396 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.396 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.396 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:36.396 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768574+0000", 2024-05-08T08:52:36.396 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4988999999999996e-05 2024-05-08T08:52:36.396 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.396 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.396 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:36.397 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768673+0000", 2024-05-08T08:52:36.397 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9963000000000005e-05 2024-05-08T08:52:36.397 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:36.397 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:36.397 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:36.397 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.397 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.397 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6706 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1224704~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:36.397 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:52:36.397 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.6614278179999999, 2024-05-08T08:52:36.397 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052435598, 2024-05-08T08:52:36.397 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:36.397 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:36.397 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:36.397 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:36.397 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:36.398 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:36.398 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6706 2024-05-08T08:52:36.398 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.398 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:36.398 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.398 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:36.398 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:52:36.398 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:36.398 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.398 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.398 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:36.398 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:52:36.398 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:36.398 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.398 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.399 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:36.399 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:52:36.399 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:52:36.399 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.399 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.399 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:36.399 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717035+0000", 2024-05-08T08:52:36.399 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9290000000000001e-06 2024-05-08T08:52:36.399 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.399 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.399 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:36.399 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717041+0000", 2024-05-08T08:52:36.399 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6720000000000001e-06 2024-05-08T08:52:36.399 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.399 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.400 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.400 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717046+0000", 2024-05-08T08:52:36.400 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5650000000000003e-06 2024-05-08T08:52:36.400 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.400 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.400 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:36.400 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769311+0000", 2024-05-08T08:52:36.400 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052265645999999999 2024-05-08T08:52:36.400 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.400 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.400 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:36.400 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769379+0000", 2024-05-08T08:52:36.400 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7391000000000006e-05 2024-05-08T08:52:36.400 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.400 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.401 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:36.401 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769469+0000", 2024-05-08T08:52:36.401 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0025999999999995e-05 2024-05-08T08:52:36.401 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:36.401 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:36.401 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:36.401 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.401 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.401 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6708 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1241088~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:36.401 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:52:36.401 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.6614097140000004, 2024-05-08T08:52:36.401 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052640552, 2024-05-08T08:52:36.401 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:36.401 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:36.401 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:36.402 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:36.402 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:36.402 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:36.402 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6708 2024-05-08T08:52:36.402 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.402 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:36.402 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.402 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:36.402 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:52:36.402 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:36.402 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.402 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.402 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:36.402 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:52:36.403 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:36.403 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.403 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.403 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:36.403 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717052+0000", 2024-05-08T08:52:36.403 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:52:36.403 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.403 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.403 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:36.403 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717053+0000", 2024-05-08T08:52:36.403 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8330000000000001e-06 2024-05-08T08:52:36.403 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.403 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.403 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:36.403 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717058+0000", 2024-05-08T08:52:36.404 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.933e-06 2024-05-08T08:52:36.404 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.404 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.404 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.404 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717063+0000", 2024-05-08T08:52:36.404 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2180000000000001e-06 2024-05-08T08:52:36.404 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.404 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.404 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:36.404 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769507+0000", 2024-05-08T08:52:36.404 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052444866999999999 2024-05-08T08:52:36.404 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.404 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.404 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:36.404 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769589+0000", 2024-05-08T08:52:36.405 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2039000000000005e-05 2024-05-08T08:52:36.405 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.405 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.405 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:36.405 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769692+0000", 2024-05-08T08:52:36.405 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102252 2024-05-08T08:52:36.405 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:36.405 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:36.405 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:36.405 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.405 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.405 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6710 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1257472~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:36.405 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:52:36.405 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.6613916900000003, 2024-05-08T08:52:36.405 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.05283057, 2024-05-08T08:52:36.406 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:36.406 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:36.406 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:36.406 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:36.406 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:36.406 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:36.406 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6710 2024-05-08T08:52:36.406 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.406 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:36.406 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.406 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:36.406 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:52:36.406 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:36.406 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.406 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.407 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:36.407 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:52:36.407 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:36.407 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.407 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.407 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:36.407 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718951+0000", 2024-05-08T08:52:36.407 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0018817809999999999 2024-05-08T08:52:36.407 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.407 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.407 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:36.407 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718956+0000", 2024-05-08T08:52:36.407 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5399999999999997e-06 2024-05-08T08:52:36.407 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.407 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.408 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:36.408 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718964+0000", 2024-05-08T08:52:36.408 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6629999999999999e-06 2024-05-08T08:52:36.408 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.408 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.408 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.408 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718972+0000", 2024-05-08T08:52:36.408 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5229999999999998e-06 2024-05-08T08:52:36.408 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.408 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.408 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:36.408 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769729+0000", 2024-05-08T08:52:36.408 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050756959999999997 2024-05-08T08:52:36.408 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.409 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.409 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:36.409 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769803+0000", 2024-05-08T08:52:36.409 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4538999999999999e-05 2024-05-08T08:52:36.409 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.409 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.409 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:36.409 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769900+0000", 2024-05-08T08:52:36.409 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6564000000000006e-05 2024-05-08T08:52:36.409 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:36.409 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:36.409 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:36.409 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.409 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.409 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6726 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1388544~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:36.410 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:52:36.410 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.659316918, 2024-05-08T08:52:36.410 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052451506000000002, 2024-05-08T08:52:36.410 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:36.410 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:36.410 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:36.410 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:36.410 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:36.410 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:36.410 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6726 2024-05-08T08:52:36.410 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.410 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:36.410 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.410 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:36.411 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:52:36.411 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:36.411 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.411 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.411 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:36.411 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:52:36.411 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:36.411 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.411 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.411 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:36.411 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719200+0000", 2024-05-08T08:52:36.411 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5865999999999997e-05 2024-05-08T08:52:36.411 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.411 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.411 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:36.411 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719204+0000", 2024-05-08T08:52:36.412 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8630000000000001e-06 2024-05-08T08:52:36.412 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.412 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.412 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:36.412 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719215+0000", 2024-05-08T08:52:36.412 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1590000000000001e-05 2024-05-08T08:52:36.412 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.412 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.412 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.412 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719220+0000", 2024-05-08T08:52:36.412 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1619999999999999e-06 2024-05-08T08:52:36.412 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.412 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.412 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:36.412 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771425+0000", 2024-05-08T08:52:36.413 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052204249000000001 2024-05-08T08:52:36.413 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.413 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.413 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:36.413 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771498+0000", 2024-05-08T08:52:36.413 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3782999999999995e-05 2024-05-08T08:52:36.413 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.413 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.413 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:36.413 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771595+0000", 2024-05-08T08:52:36.413 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6992999999999998e-05 2024-05-08T08:52:36.413 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:36.413 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:36.413 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:36.413 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.413 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.414 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6728 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1404928~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:36.414 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:52:36.414 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.6592295129999997, 2024-05-08T08:52:36.414 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052590747, 2024-05-08T08:52:36.414 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:36.414 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:36.414 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:36.414 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:36.414 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:36.414 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:36.414 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6728 2024-05-08T08:52:36.414 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.414 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:36.414 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.415 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:36.415 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:52:36.415 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:36.415 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.415 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.415 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:36.415 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:52:36.415 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:36.415 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.415 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.415 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:36.415 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719232+0000", 2024-05-08T08:52:36.415 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4000000000000001e-07 2024-05-08T08:52:36.415 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.416 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.416 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:36.416 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719234+0000", 2024-05-08T08:52:36.416 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4499999999999998e-06 2024-05-08T08:52:36.416 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.416 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.416 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:36.416 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719240+0000", 2024-05-08T08:52:36.416 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5720000000000002e-06 2024-05-08T08:52:36.416 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.416 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.416 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.416 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719245+0000", 2024-05-08T08:52:36.416 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8690000000000003e-06 2024-05-08T08:52:36.416 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.416 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.417 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:36.417 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771636+0000", 2024-05-08T08:52:36.417 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052390979999999997 2024-05-08T08:52:36.417 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.417 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.417 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:36.417 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771717+0000", 2024-05-08T08:52:36.417 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1291999999999993e-05 2024-05-08T08:52:36.417 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.417 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.417 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:36.417 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771822+0000", 2024-05-08T08:52:36.417 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104944 2024-05-08T08:52:36.417 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:36.417 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:36.418 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:36.418 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.418 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.418 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6730 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1421312~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:36.418 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:52:36.418 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.6592092709999999, 2024-05-08T08:52:36.418 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052770477000000003, 2024-05-08T08:52:36.418 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:36.418 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:36.418 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:36.418 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:36.418 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:36.418 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:36.418 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6730 2024-05-08T08:52:36.418 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.419 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:36.419 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.419 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:36.419 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:52:36.419 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:36.419 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.419 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.419 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:36.419 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:52:36.419 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:36.419 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.419 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.419 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:36.419 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:52:36.419 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9000000000000002e-07 2024-05-08T08:52:36.419 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.420 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.420 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:36.420 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719254+0000", 2024-05-08T08:52:36.420 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4099999999999998e-06 2024-05-08T08:52:36.420 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.420 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.420 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:36.420 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719263+0000", 2024-05-08T08:52:36.420 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4109999999999996e-06 2024-05-08T08:52:36.420 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.420 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.420 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.420 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719268+0000", 2024-05-08T08:52:36.420 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2669999999999997e-06 2024-05-08T08:52:36.420 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.421 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.421 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:36.421 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771846+0000", 2024-05-08T08:52:36.421 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052577677000000003 2024-05-08T08:52:36.421 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.421 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.421 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:36.421 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771920+0000", 2024-05-08T08:52:36.421 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4460000000000002e-05 2024-05-08T08:52:36.421 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.421 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.421 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:36.421 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772022+0000", 2024-05-08T08:52:36.421 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010186199999999999 2024-05-08T08:52:36.421 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:36.421 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:36.422 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:36.422 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.422 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.422 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6732 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1437696~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:36.422 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:52:36.422 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.659184464, 2024-05-08T08:52:36.422 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052905575000000003, 2024-05-08T08:52:36.422 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:36.422 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:36.422 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:36.422 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:36.422 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:36.422 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:36.422 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6732 2024-05-08T08:52:36.422 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.423 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:36.423 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.423 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:36.423 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:52:36.423 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:36.423 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.423 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.423 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:36.423 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:52:36.423 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:36.423 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.423 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.423 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:36.423 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721392+0000", 2024-05-08T08:52:36.423 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002115287 2024-05-08T08:52:36.424 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.424 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.424 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:36.424 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721396+0000", 2024-05-08T08:52:36.424 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1679999999999997e-06 2024-05-08T08:52:36.424 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.424 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.424 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:36.424 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721404+0000", 2024-05-08T08:52:36.424 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4409999999999993e-06 2024-05-08T08:52:36.424 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.424 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.424 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.424 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721410+0000", 2024-05-08T08:52:36.424 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9150000000000001e-06 2024-05-08T08:52:36.424 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.425 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.425 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:36.425 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772061+0000", 2024-05-08T08:52:36.425 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050651068 2024-05-08T08:52:36.425 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.425 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.425 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:36.425 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772127+0000", 2024-05-08T08:52:36.425 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5424000000000002e-05 2024-05-08T08:52:36.425 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.425 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.425 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:36.425 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772182+0000", 2024-05-08T08:52:36.425 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5272000000000002e-05 2024-05-08T08:52:36.425 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:36.426 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:36.426 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:36.426 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.426 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.426 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6749 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1585152~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:36.426 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:52:36.426 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.6568836940000002, 2024-05-08T08:52:36.426 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052350264000000001, 2024-05-08T08:52:36.426 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:36.426 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:36.426 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:36.426 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:36.426 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:36.426 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:36.426 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6749 2024-05-08T08:52:36.427 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.427 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:36.427 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.427 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:36.427 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:52:36.427 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:36.427 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.427 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.427 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:36.427 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:52:36.427 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:36.427 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.427 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.427 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:36.427 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721578+0000", 2024-05-08T08:52:36.428 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0399999999999996e-07 2024-05-08T08:52:36.428 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.428 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.428 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:36.428 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721580+0000", 2024-05-08T08:52:36.428 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5450000000000002e-06 2024-05-08T08:52:36.428 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.428 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.428 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:36.428 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721587+0000", 2024-05-08T08:52:36.428 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0190000000000001e-06 2024-05-08T08:52:36.428 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.428 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.428 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.428 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721593+0000", 2024-05-08T08:52:36.428 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5659999999999997e-06 2024-05-08T08:52:36.429 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.429 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.429 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:36.429 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773759+0000", 2024-05-08T08:52:36.429 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052166553999999997 2024-05-08T08:52:36.429 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.429 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.429 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:36.429 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773833+0000", 2024-05-08T08:52:36.429 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3913999999999997e-05 2024-05-08T08:52:36.429 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.429 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.429 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:36.429 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773927+0000", 2024-05-08T08:52:36.429 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4061999999999995e-05 2024-05-08T08:52:36.430 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:36.430 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:36.430 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:36.430 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.430 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.430 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6751 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1601536~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:36.430 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:52:36.430 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.6568604149999997, 2024-05-08T08:52:36.430 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052461302000000001, 2024-05-08T08:52:36.430 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:36.430 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:36.430 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:36.430 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:36.430 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:36.431 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:36.431 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6751 2024-05-08T08:52:36.431 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.431 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:36.431 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.431 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:36.431 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:52:36.431 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:36.431 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.431 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.431 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:36.431 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:52:36.431 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:36.431 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.431 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.431 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:36.432 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722939+0000", 2024-05-08T08:52:36.432 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001338601 2024-05-08T08:52:36.432 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.432 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.432 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:36.432 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722944+0000", 2024-05-08T08:52:36.432 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4469999999999999e-06 2024-05-08T08:52:36.432 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.432 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.432 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:36.432 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722955+0000", 2024-05-08T08:52:36.432 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1892999999999999e-05 2024-05-08T08:52:36.432 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.432 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.432 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.433 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722963+0000", 2024-05-08T08:52:36.433 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1080000000000004e-06 2024-05-08T08:52:36.433 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.433 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.433 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:36.433 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773964+0000", 2024-05-08T08:52:36.433 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051001022 2024-05-08T08:52:36.433 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.433 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.433 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:36.433 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.774006+0000", 2024-05-08T08:52:36.433 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2729999999999999e-05 2024-05-08T08:52:36.433 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.433 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.433 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:36.434 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.774062+0000", 2024-05-08T08:52:36.434 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5501000000000003e-05 2024-05-08T08:52:36.434 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:36.434 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:36.434 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:36.434 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.434 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.434 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6760 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1683456~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:36.434 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:52:36.434 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.6553510390000001, 2024-05-08T08:52:36.434 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052571312000000002, 2024-05-08T08:52:36.434 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:36.434 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:36.434 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:36.434 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:36.435 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:36.435 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:36.435 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6760 2024-05-08T08:52:36.435 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.435 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:36.435 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.435 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:36.435 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:52:36.435 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:36.435 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.435 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.435 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:36.435 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:52:36.435 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:36.435 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.436 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.436 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:36.436 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:52:36.436 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2100000000000002e-07 2024-05-08T08:52:36.436 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.436 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.436 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:36.436 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723113+0000", 2024-05-08T08:52:36.436 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3149999999999999e-06 2024-05-08T08:52:36.436 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.436 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.436 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:36.436 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723118+0000", 2024-05-08T08:52:36.436 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8860000000000001e-06 2024-05-08T08:52:36.436 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.436 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.437 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.437 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723123+0000", 2024-05-08T08:52:36.437 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6770000000000003e-06 2024-05-08T08:52:36.437 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.437 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.437 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:36.437 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775414+0000", 2024-05-08T08:52:36.437 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052290692999999999 2024-05-08T08:52:36.437 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.437 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.437 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:36.437 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775459+0000", 2024-05-08T08:52:36.437 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5639999999999997e-05 2024-05-08T08:52:36.437 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.437 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.438 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:36.438 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775681+0000", 2024-05-08T08:52:36.438 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00022168000000000001 2024-05-08T08:52:36.438 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:36.438 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:36.438 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:36.438 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.438 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.438 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6761 3.16 3:6bd6d6da:::rbd_data.1140782590a5.0000000000000015:6 [read 1095680~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:36.438 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:52:36.438 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.6553318189999997, 2024-05-08T08:52:36.438 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052751275, 2024-05-08T08:52:36.438 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:36.438 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:36.438 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:36.439 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:36.439 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:36.439 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:36.439 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6761 2024-05-08T08:52:36.439 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.439 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:36.439 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.439 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:36.439 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:52:36.439 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:36.439 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.439 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.439 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:36.439 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:52:36.439 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:36.439 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.440 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.440 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:36.440 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724939+0000", 2024-05-08T08:52:36.440 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001809817 2024-05-08T08:52:36.440 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.440 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.440 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:36.440 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724943+0000", 2024-05-08T08:52:36.440 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4920000000000004e-06 2024-05-08T08:52:36.440 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.440 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.440 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:36.440 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724956+0000", 2024-05-08T08:52:36.440 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2427e-05 2024-05-08T08:52:36.440 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.441 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.441 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.441 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724963+0000", 2024-05-08T08:52:36.441 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2559999999999996e-06 2024-05-08T08:52:36.441 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.441 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.441 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:36.441 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775711+0000", 2024-05-08T08:52:36.441 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050747621 2024-05-08T08:52:36.441 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.441 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.441 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:36.441 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775785+0000", 2024-05-08T08:52:36.441 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4634000000000004e-05 2024-05-08T08:52:36.442 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.442 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.442 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:36.442 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775880+0000", 2024-05-08T08:52:36.442 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5027999999999999e-05 2024-05-08T08:52:36.442 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:36.442 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:36.442 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:36.442 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.442 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.442 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6770 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1765376~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:36.442 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:52:36.442 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.6533007240000002, 2024-05-08T08:52:36.442 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052339296, 2024-05-08T08:52:36.443 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:36.443 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:36.443 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:36.443 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:36.443 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:36.443 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:36.443 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6770 2024-05-08T08:52:36.443 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.443 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:36.443 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.443 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:36.443 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:52:36.443 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:36.443 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.443 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.444 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:36.444 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:52:36.444 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:36.444 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.444 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.444 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:36.444 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725161+0000", 2024-05-08T08:52:36.444 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-05-08T08:52:36.444 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.444 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.444 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:36.444 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725163+0000", 2024-05-08T08:52:36.444 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4370000000000001e-06 2024-05-08T08:52:36.444 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.444 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.445 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:36.445 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725169+0000", 2024-05-08T08:52:36.445 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8409999999999996e-06 2024-05-08T08:52:36.445 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.445 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.445 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.445 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725174+0000", 2024-05-08T08:52:36.445 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7249999999999997e-06 2024-05-08T08:52:36.445 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.445 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.445 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:36.445 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777343+0000", 2024-05-08T08:52:36.445 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052169633999999999 2024-05-08T08:52:36.445 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.445 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.445 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:36.446 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777409+0000", 2024-05-08T08:52:36.446 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6033999999999998e-05 2024-05-08T08:52:36.446 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.446 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.446 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:36.446 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777499+0000", 2024-05-08T08:52:36.446 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0224999999999998e-05 2024-05-08T08:52:36.446 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:36.446 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:36.446 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:36.446 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.446 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.446 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6771 3.16 3:6bd6d6da:::rbd_data.1140782590a5.0000000000000015:6 [read 1177600~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:36.446 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:52:36.446 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.6532809070000001, 2024-05-08T08:52:36.446 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052438604999999999, 2024-05-08T08:52:36.447 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:36.447 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:36.447 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:36.447 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:36.447 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:36.447 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:36.447 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6771 2024-05-08T08:52:36.447 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.447 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:36.447 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.447 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:36.447 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:52:36.447 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:36.447 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.447 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.448 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:36.448 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:52:36.448 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:36.448 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.448 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.448 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:36.448 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:52:36.448 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:52:36.448 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.448 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.448 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:36.448 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725188+0000", 2024-05-08T08:52:36.448 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1579999999999999e-06 2024-05-08T08:52:36.448 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.448 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.448 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:36.449 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725195+0000", 2024-05-08T08:52:36.449 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8909999999999995e-06 2024-05-08T08:52:36.449 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.449 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.449 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.449 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725200+0000", 2024-05-08T08:52:36.449 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9810000000000003e-06 2024-05-08T08:52:36.449 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.449 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.449 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:36.449 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777527+0000", 2024-05-08T08:52:36.449 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052326582000000003 2024-05-08T08:52:36.449 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.449 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.449 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:36.449 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777566+0000", 2024-05-08T08:52:36.449 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9447999999999999e-05 2024-05-08T08:52:36.450 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.450 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.450 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:36.450 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777619+0000", 2024-05-08T08:52:36.450 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2135000000000001e-05 2024-05-08T08:52:36.450 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:36.450 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:36.450 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:36.450 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.450 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.450 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6772 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1781760~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:36.450 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:52:36.450 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.6532528500000003, 2024-05-08T08:52:36.450 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052601799999999997, 2024-05-08T08:52:36.450 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:36.451 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:36.451 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:36.451 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:36.451 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:36.451 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:36.451 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6772 2024-05-08T08:52:36.451 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.451 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:36.451 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.451 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:36.451 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:52:36.451 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:36.451 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.451 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.451 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:36.451 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:52:36.452 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:36.452 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.452 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.452 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:36.452 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726939+0000", 2024-05-08T08:52:36.452 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001730493 2024-05-08T08:52:36.452 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.452 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.452 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:36.452 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726943+0000", 2024-05-08T08:52:36.452 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7330000000000003e-06 2024-05-08T08:52:36.452 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.452 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.452 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:36.452 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726954+0000", 2024-05-08T08:52:36.452 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1107000000000001e-05 2024-05-08T08:52:36.453 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.453 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.453 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.453 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726962+0000", 2024-05-08T08:52:36.453 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.503e-06 2024-05-08T08:52:36.453 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.453 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.453 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:36.453 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777653+0000", 2024-05-08T08:52:36.453 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050691542999999999 2024-05-08T08:52:36.453 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.453 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.453 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:36.453 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777719+0000", 2024-05-08T08:52:36.454 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5228000000000006e-05 2024-05-08T08:52:36.454 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.454 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.454 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:36.454 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777810+0000", 2024-05-08T08:52:36.454 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1193000000000006e-05 2024-05-08T08:52:36.454 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:36.454 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:36.454 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:36.454 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.454 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.454 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6782 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1880064~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:36.454 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:52:36.454 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.6512843430000004, 2024-05-08T08:52:36.454 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052326492000000002, 2024-05-08T08:52:36.455 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:36.455 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:36.455 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:36.455 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:36.455 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:36.455 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:36.455 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6782 2024-05-08T08:52:36.455 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.455 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:36.455 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.455 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:36.455 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:52:36.455 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:36.455 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.455 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.456 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:36.456 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:52:36.456 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:36.456 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.456 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.456 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:36.456 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728939+0000", 2024-05-08T08:52:36.456 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001762719 2024-05-08T08:52:36.456 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.456 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.456 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:36.456 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728944+0000", 2024-05-08T08:52:36.456 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.746e-06 2024-05-08T08:52:36.456 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.456 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.457 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:36.457 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728955+0000", 2024-05-08T08:52:36.457 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1043000000000001e-05 2024-05-08T08:52:36.457 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.457 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.457 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.457 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728963+0000", 2024-05-08T08:52:36.457 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5809999999999999e-06 2024-05-08T08:52:36.457 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.457 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.457 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:36.457 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779320+0000", 2024-05-08T08:52:36.457 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050357290999999998 2024-05-08T08:52:36.457 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.457 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.457 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:36.458 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779401+0000", 2024-05-08T08:52:36.458 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0574000000000005e-05 2024-05-08T08:52:36.458 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:36.458 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:36.458 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:36.458 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779503+0000", 2024-05-08T08:52:36.458 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102538 2024-05-08T08:52:36.458 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:36.458 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:36.458 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:36.458 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:36.458 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:36.458 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:52:36.460 INFO:teuthology.orchestra.run.smithi196.stdout:checking clone #4, image image_client.0-clone4 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent5 2024-05-08T08:52:36.460 INFO:teuthology.orchestra.run.smithi196.stdout:1011 punch from 0xca65c00 to 0xca72400, (0xc800 bytes) 2024-05-08T08:52:36.460 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:36.463+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:36.461 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:52:36.470 INFO:teuthology.orchestra.run.smithi196.stdout:1012 write 0x54bc600 thru 0x54cc0c7 (0xfac8 bytes) 2024-05-08T08:52:36.470 INFO:teuthology.orchestra.run.smithi196.stdout:1015 write 0xee8a00 thru 0xef4bf1 (0xc1f2 bytes) 2024-05-08T08:52:36.472 INFO:teuthology.orchestra.run.smithi196.stdout:1016 read 0x4a2aa00 thru 0x4a31755 (0x6d56 bytes) 2024-05-08T08:52:36.560 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:52:36.563+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:52:36.651 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:52:36.651 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:52:36.651 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.651 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1900 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.651 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.582564+0000", 2024-05-08T08:52:36.651 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.052664365999999997, 2024-05-08T08:52:36.651 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.052691623999999999, 2024-05-08T08:52:36.651 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.652 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.652 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.652 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.652 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.652 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.652 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1900 2024-05-08T08:52:36.652 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.652 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.652 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.652 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.652 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.582564+0000", 2024-05-08T08:52:36.652 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.652 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.652 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.652 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.653 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.582564+0000", 2024-05-08T08:52:36.653 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.653 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.653 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.653 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.653 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.582863+0000", 2024-05-08T08:52:36.653 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00029948000000000003 2024-05-08T08:52:36.653 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.653 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.653 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.653 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.582867+0000", 2024-05-08T08:52:36.653 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3359999999999997e-06 2024-05-08T08:52:36.653 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.653 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.653 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.654 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.582876+0000", 2024-05-08T08:52:36.654 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.6610000000000006e-06 2024-05-08T08:52:36.654 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.654 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.654 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.654 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.582882+0000", 2024-05-08T08:52:36.654 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0719999999999996e-06 2024-05-08T08:52:36.654 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.654 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.654 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.654 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.654 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.654 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1964 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.654 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.589873+0000", 2024-05-08T08:52:36.655 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.045355399999999997, 2024-05-08T08:52:36.655 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.045463953000000001, 2024-05-08T08:52:36.655 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.655 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.655 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.655 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.655 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.655 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.655 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1964 2024-05-08T08:52:36.655 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.655 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.655 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.655 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.655 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.589873+0000", 2024-05-08T08:52:36.655 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.655 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.656 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.656 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.656 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.589873+0000", 2024-05-08T08:52:36.656 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.656 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.656 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.656 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.656 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.590067+0000", 2024-05-08T08:52:36.656 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00019404200000000001 2024-05-08T08:52:36.656 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.656 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.656 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.656 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.590070+0000", 2024-05-08T08:52:36.656 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.2059999999999999e-06 2024-05-08T08:52:36.657 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.657 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.657 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.657 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.590076+0000", 2024-05-08T08:52:36.657 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6490000000000002e-06 2024-05-08T08:52:36.657 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.657 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.657 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.657 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.590081+0000", 2024-05-08T08:52:36.657 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.233e-06 2024-05-08T08:52:36.657 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.657 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.657 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.657 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.657 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.658 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2027 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.658 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.596540+0000", 2024-05-08T08:52:36.658 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.038687636999999997, 2024-05-08T08:52:36.658 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.038862157000000001, 2024-05-08T08:52:36.658 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.658 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.658 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.658 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.658 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.658 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.658 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2027 2024-05-08T08:52:36.658 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.658 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.658 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.659 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.659 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.596540+0000", 2024-05-08T08:52:36.659 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.659 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.659 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.659 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.659 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.596540+0000", 2024-05-08T08:52:36.659 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.659 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.659 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.659 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.659 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.596541+0000", 2024-05-08T08:52:36.659 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7300000000000002e-07 2024-05-08T08:52:36.659 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.660 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.660 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.660 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.596543+0000", 2024-05-08T08:52:36.660 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.289e-06 2024-05-08T08:52:36.660 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.660 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.660 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.660 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.596548+0000", 2024-05-08T08:52:36.660 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8890000000000001e-06 2024-05-08T08:52:36.660 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.660 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.660 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.660 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.596551+0000", 2024-05-08T08:52:36.660 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6040000000000001e-06 2024-05-08T08:52:36.660 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.660 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.661 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.661 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.661 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.661 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2091 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.661 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.603776+0000", 2024-05-08T08:52:36.661 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.031451623999999997, 2024-05-08T08:52:36.661 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.031695569, 2024-05-08T08:52:36.661 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.661 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.661 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.661 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.661 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.661 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.661 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2091 2024-05-08T08:52:36.661 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.662 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.662 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.662 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.662 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.603776+0000", 2024-05-08T08:52:36.662 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.662 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.662 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.662 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.662 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.603776+0000", 2024-05-08T08:52:36.662 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.662 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.662 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.662 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.662 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.603987+0000", 2024-05-08T08:52:36.663 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000210244 2024-05-08T08:52:36.663 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.663 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.663 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.663 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.603991+0000", 2024-05-08T08:52:36.663 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1459999999999998e-06 2024-05-08T08:52:36.663 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.663 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.663 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.663 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.604000+0000", 2024-05-08T08:52:36.663 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6949999999999995e-06 2024-05-08T08:52:36.663 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.663 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.663 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.663 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.604007+0000", 2024-05-08T08:52:36.664 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6810000000000001e-06 2024-05-08T08:52:36.664 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.664 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.664 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.664 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.664 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.664 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2156 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.664 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.610431+0000", 2024-05-08T08:52:36.664 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.024796612999999999, 2024-05-08T08:52:36.664 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.025107718000000001, 2024-05-08T08:52:36.664 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.664 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.664 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.664 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.664 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.665 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.665 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2156 2024-05-08T08:52:36.665 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.665 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.665 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.665 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.665 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.610431+0000", 2024-05-08T08:52:36.665 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.665 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.665 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.665 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.665 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.610431+0000", 2024-05-08T08:52:36.665 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.665 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.665 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.666 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.666 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.610432+0000", 2024-05-08T08:52:36.666 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8599999999999998e-07 2024-05-08T08:52:36.666 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.666 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.666 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.666 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.610435+0000", 2024-05-08T08:52:36.666 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.5299999999999999e-06 2024-05-08T08:52:36.666 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.666 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.666 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.666 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.610441+0000", 2024-05-08T08:52:36.666 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7159999999999997e-06 2024-05-08T08:52:36.666 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.667 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.667 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.667 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.610450+0000", 2024-05-08T08:52:36.667 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.6270000000000006e-06 2024-05-08T08:52:36.667 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.667 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.667 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.667 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.667 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.667 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2221 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.667 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.616987+0000", 2024-05-08T08:52:36.667 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.018241248000000002, 2024-05-08T08:52:36.667 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.018618961, 2024-05-08T08:52:36.667 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.668 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.668 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.668 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.668 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.668 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.668 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2221 2024-05-08T08:52:36.668 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.668 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.668 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.668 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.668 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.616987+0000", 2024-05-08T08:52:36.668 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.668 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.668 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.669 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.669 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.616987+0000", 2024-05-08T08:52:36.669 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.669 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.669 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.669 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.669 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.617163+0000", 2024-05-08T08:52:36.669 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000176818 2024-05-08T08:52:36.669 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.669 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.669 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.669 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.617168+0000", 2024-05-08T08:52:36.669 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0980000000000004e-06 2024-05-08T08:52:36.669 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.669 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.670 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.670 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.617179+0000", 2024-05-08T08:52:36.670 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1287000000000001e-05 2024-05-08T08:52:36.670 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.670 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.670 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.670 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.617186+0000", 2024-05-08T08:52:36.670 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1049999999999997e-06 2024-05-08T08:52:36.670 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.670 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.670 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.670 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.670 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.670 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2284 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.670 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.623000+0000", 2024-05-08T08:52:36.671 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.012228312, 2024-05-08T08:52:36.671 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.012673145, 2024-05-08T08:52:36.671 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.671 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.671 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.671 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.671 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.671 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.671 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2284 2024-05-08T08:52:36.671 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.671 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.671 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.671 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.671 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.623000+0000", 2024-05-08T08:52:36.671 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.672 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.672 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.672 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.672 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.623000+0000", 2024-05-08T08:52:36.672 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.672 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.672 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.672 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.672 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.623320+0000", 2024-05-08T08:52:36.672 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00031995 2024-05-08T08:52:36.672 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.672 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.672 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.672 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.623324+0000", 2024-05-08T08:52:36.672 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.498e-06 2024-05-08T08:52:36.673 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.673 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.673 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.673 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.623335+0000", 2024-05-08T08:52:36.673 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.095e-05 2024-05-08T08:52:36.673 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.673 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.673 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.673 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.623342+0000", 2024-05-08T08:52:36.673 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9689999999999997e-06 2024-05-08T08:52:36.673 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.673 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.673 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.673 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.673 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.673 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2349 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.674 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.630028+0000", 2024-05-08T08:52:36.674 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0052000070000000004, 2024-05-08T08:52:36.674 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.005710835, 2024-05-08T08:52:36.674 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.674 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.674 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.674 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.674 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.674 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.674 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2349 2024-05-08T08:52:36.674 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.674 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.674 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.674 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.675 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.630028+0000", 2024-05-08T08:52:36.675 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.675 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.675 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.675 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.675 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.630028+0000", 2024-05-08T08:52:36.675 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.675 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.675 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.675 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.675 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.630028+0000", 2024-05-08T08:52:36.675 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6399999999999998e-07 2024-05-08T08:52:36.675 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.675 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.675 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.675 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.630030+0000", 2024-05-08T08:52:36.676 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.717e-06 2024-05-08T08:52:36.676 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.676 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.676 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.676 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.630034+0000", 2024-05-08T08:52:36.676 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.985e-06 2024-05-08T08:52:36.676 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.676 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.676 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.676 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.630037+0000", 2024-05-08T08:52:36.676 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.931e-06 2024-05-08T08:52:36.676 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.676 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.676 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.676 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.677 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.677 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1902 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.677 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.582893+0000", 2024-05-08T08:52:36.677 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.052335048000000002, 2024-05-08T08:52:36.677 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.052910519000000003, 2024-05-08T08:52:36.677 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.677 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.677 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.677 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.677 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.677 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.677 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1902 2024-05-08T08:52:36.677 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.677 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.677 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.678 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.678 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.582893+0000", 2024-05-08T08:52:36.678 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.678 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.678 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.678 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.678 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.582893+0000", 2024-05-08T08:52:36.678 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.678 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.678 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.678 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.678 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.582961+0000", 2024-05-08T08:52:36.678 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7871000000000001e-05 2024-05-08T08:52:36.679 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.679 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.679 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.679 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.582965+0000", 2024-05-08T08:52:36.679 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1110000000000001e-06 2024-05-08T08:52:36.679 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.679 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.679 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.679 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.582975+0000", 2024-05-08T08:52:36.679 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8060000000000006e-06 2024-05-08T08:52:36.679 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.679 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.679 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.679 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.582981+0000", 2024-05-08T08:52:36.680 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.438e-06 2024-05-08T08:52:36.680 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.680 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.680 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.680 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.680 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.680 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1966 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.680 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.590089+0000", 2024-05-08T08:52:36.680 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.045139107999999997, 2024-05-08T08:52:36.680 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.045786108999999998, 2024-05-08T08:52:36.680 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.680 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.680 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.680 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.680 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.681 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.681 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1966 2024-05-08T08:52:36.681 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.681 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.681 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.681 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.681 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.590089+0000", 2024-05-08T08:52:36.681 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.681 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.681 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.681 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.681 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.590089+0000", 2024-05-08T08:52:36.681 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.681 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.681 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.681 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.682 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.590089+0000", 2024-05-08T08:52:36.682 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9999999999999998e-07 2024-05-08T08:52:36.682 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.682 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.682 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.682 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.590091+0000", 2024-05-08T08:52:36.682 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.4300000000000001e-06 2024-05-08T08:52:36.682 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.682 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.682 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.682 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.590094+0000", 2024-05-08T08:52:36.682 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3900000000000002e-06 2024-05-08T08:52:36.682 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.683 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.683 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.683 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.590097+0000", 2024-05-08T08:52:36.683 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.819e-06 2024-05-08T08:52:36.683 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.683 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.683 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.683 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.683 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.683 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2029 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.683 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.596556+0000", 2024-05-08T08:52:36.683 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.038671832000000003, 2024-05-08T08:52:36.683 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.039382523000000003, 2024-05-08T08:52:36.683 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.683 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.683 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.684 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.684 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.684 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.684 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2029 2024-05-08T08:52:36.684 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.684 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.684 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.684 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.684 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.596556+0000", 2024-05-08T08:52:36.684 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.684 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.684 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.684 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.684 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.596556+0000", 2024-05-08T08:52:36.684 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.685 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.685 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.685 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.685 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.596695+0000", 2024-05-08T08:52:36.685 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013909600000000001 2024-05-08T08:52:36.685 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.685 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.685 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.685 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.596699+0000", 2024-05-08T08:52:36.685 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7790000000000002e-06 2024-05-08T08:52:36.685 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.685 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.685 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.685 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.596707+0000", 2024-05-08T08:52:36.685 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7200000000000006e-06 2024-05-08T08:52:36.686 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.686 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.686 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.686 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.596716+0000", 2024-05-08T08:52:36.686 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2890000000000003e-06 2024-05-08T08:52:36.686 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.686 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.686 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.686 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.686 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.686 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2093 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.686 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.604017+0000", 2024-05-08T08:52:36.686 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.031210667000000001, 2024-05-08T08:52:36.686 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.031988359000000001, 2024-05-08T08:52:36.687 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.687 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.687 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.687 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.687 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.687 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.687 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2093 2024-05-08T08:52:36.687 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.687 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.687 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.687 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.687 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.604017+0000", 2024-05-08T08:52:36.687 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.687 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.687 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.687 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.688 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.604017+0000", 2024-05-08T08:52:36.688 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.688 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.688 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.688 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.688 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.604423+0000", 2024-05-08T08:52:36.688 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00040615399999999999 2024-05-08T08:52:36.688 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.688 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.688 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.688 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.604427+0000", 2024-05-08T08:52:36.688 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0990000000000001e-06 2024-05-08T08:52:36.688 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.688 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.688 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.688 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.604437+0000", 2024-05-08T08:52:36.689 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2289999999999992e-06 2024-05-08T08:52:36.689 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.689 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.689 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.689 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.604444+0000", 2024-05-08T08:52:36.689 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4769999999999997e-06 2024-05-08T08:52:36.689 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.689 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.689 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.689 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.689 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.689 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2158 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.689 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.610458+0000", 2024-05-08T08:52:36.689 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.024769965000000001, 2024-05-08T08:52:36.690 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.025604855999999999, 2024-05-08T08:52:36.690 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.690 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.690 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.690 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.690 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.690 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.690 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2158 2024-05-08T08:52:36.690 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.690 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.690 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.690 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.690 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.610458+0000", 2024-05-08T08:52:36.690 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.690 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.691 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.691 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.691 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.610458+0000", 2024-05-08T08:52:36.691 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.691 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.691 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.691 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.691 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.610801+0000", 2024-05-08T08:52:36.691 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00034264000000000001 2024-05-08T08:52:36.691 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.691 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.691 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.691 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.610804+0000", 2024-05-08T08:52:36.691 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3780000000000001e-06 2024-05-08T08:52:36.691 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.691 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.692 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.692 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.610811+0000", 2024-05-08T08:52:36.692 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9110000000000004e-06 2024-05-08T08:52:36.692 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.692 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.692 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.692 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.610815+0000", 2024-05-08T08:52:36.692 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0799999999999999e-06 2024-05-08T08:52:36.692 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.692 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.692 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.692 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.692 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.693 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2223 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.693 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.617197+0000", 2024-05-08T08:52:36.693 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.018030889000000001, 2024-05-08T08:52:36.693 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.018922972999999999, 2024-05-08T08:52:36.693 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.693 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.693 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.693 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.693 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.693 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.693 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2223 2024-05-08T08:52:36.693 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.693 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.693 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.693 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.694 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.617197+0000", 2024-05-08T08:52:36.694 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.694 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.694 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.694 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.694 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.617197+0000", 2024-05-08T08:52:36.694 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.694 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.694 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.694 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.694 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.617298+0000", 2024-05-08T08:52:36.694 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000100843 2024-05-08T08:52:36.694 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.695 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.695 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.695 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.617309+0000", 2024-05-08T08:52:36.695 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1289e-05 2024-05-08T08:52:36.695 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.695 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.695 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.695 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.617319+0000", 2024-05-08T08:52:36.695 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0291e-05 2024-05-08T08:52:36.695 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.695 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.695 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.695 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.617326+0000", 2024-05-08T08:52:36.695 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6030000000000001e-06 2024-05-08T08:52:36.695 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.696 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.696 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.696 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.696 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.696 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2286 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.696 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.623354+0000", 2024-05-08T08:52:36.696 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.011874371999999999, 2024-05-08T08:52:36.696 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.012813319, 2024-05-08T08:52:36.696 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.696 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.696 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.696 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.696 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.696 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.696 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2286 2024-05-08T08:52:36.696 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.697 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.697 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.697 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.697 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.623354+0000", 2024-05-08T08:52:36.697 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.697 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.697 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.697 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.697 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.623354+0000", 2024-05-08T08:52:36.697 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.697 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.697 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.697 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.697 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.623475+0000", 2024-05-08T08:52:36.698 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000120974 2024-05-08T08:52:36.698 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.698 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.698 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.698 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.623477+0000", 2024-05-08T08:52:36.698 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9610000000000001e-06 2024-05-08T08:52:36.698 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.698 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.698 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.698 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.623484+0000", 2024-05-08T08:52:36.698 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1029999999999998e-06 2024-05-08T08:52:36.698 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.698 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.698 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.698 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.623488+0000", 2024-05-08T08:52:36.699 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0509999999999998e-06 2024-05-08T08:52:36.699 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.699 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.699 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.699 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.699 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.699 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2351 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.699 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.630041+0000", 2024-05-08T08:52:36.699 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0051870620000000001, 2024-05-08T08:52:36.699 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0061817249999999999, 2024-05-08T08:52:36.699 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.699 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.699 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.699 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.700 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.700 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.700 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2351 2024-05-08T08:52:36.700 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.700 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.700 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.700 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.700 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.630041+0000", 2024-05-08T08:52:36.700 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.700 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.700 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.700 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.700 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.630041+0000", 2024-05-08T08:52:36.700 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.700 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.701 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.701 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.701 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.630395+0000", 2024-05-08T08:52:36.701 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000354407 2024-05-08T08:52:36.701 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.701 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.701 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.701 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.630399+0000", 2024-05-08T08:52:36.701 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9020000000000001e-06 2024-05-08T08:52:36.701 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.701 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.701 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.701 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.630409+0000", 2024-05-08T08:52:36.701 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.7569999999999999e-06 2024-05-08T08:52:36.701 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.701 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.702 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.702 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.630416+0000", 2024-05-08T08:52:36.702 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0199999999999997e-06 2024-05-08T08:52:36.702 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.702 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.702 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.702 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.702 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.702 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1904 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.702 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.582991+0000", 2024-05-08T08:52:36.702 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.052237036000000001, 2024-05-08T08:52:36.702 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.053287104000000002, 2024-05-08T08:52:36.702 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.703 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.703 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.703 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.703 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.703 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.703 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1904 2024-05-08T08:52:36.703 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.703 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.703 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.703 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.703 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.582991+0000", 2024-05-08T08:52:36.703 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.703 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.703 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.703 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.704 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.582991+0000", 2024-05-08T08:52:36.704 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.704 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.704 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.704 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.704 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.583252+0000", 2024-05-08T08:52:36.704 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00026129300000000001 2024-05-08T08:52:36.704 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.704 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.704 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.704 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.583257+0000", 2024-05-08T08:52:36.704 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4739999999999999e-06 2024-05-08T08:52:36.704 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.704 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.704 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.705 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.583268+0000", 2024-05-08T08:52:36.705 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1525999999999999e-05 2024-05-08T08:52:36.705 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.705 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.705 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.705 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.583275+0000", 2024-05-08T08:52:36.705 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9970000000000001e-06 2024-05-08T08:52:36.705 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.705 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.705 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.705 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.705 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.705 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1968 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.705 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.590101+0000", 2024-05-08T08:52:36.705 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.045127361999999997, 2024-05-08T08:52:36.706 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.046209944000000003, 2024-05-08T08:52:36.706 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.706 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.706 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.706 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.706 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.706 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.706 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1968 2024-05-08T08:52:36.706 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.706 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.706 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.706 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.706 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.590101+0000", 2024-05-08T08:52:36.706 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.707 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.707 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.707 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.707 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.590101+0000", 2024-05-08T08:52:36.707 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.707 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.707 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.707 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.707 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.590476+0000", 2024-05-08T08:52:36.707 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000375764 2024-05-08T08:52:36.707 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.707 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.707 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.707 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.590480+0000", 2024-05-08T08:52:36.707 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3050000000000001e-06 2024-05-08T08:52:36.708 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.708 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.708 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.708 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.590488+0000", 2024-05-08T08:52:36.708 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0870000000000007e-06 2024-05-08T08:52:36.708 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.708 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.708 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.708 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.590492+0000", 2024-05-08T08:52:36.708 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5129999999999998e-06 2024-05-08T08:52:36.708 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.708 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.708 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.708 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.708 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.709 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2031 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.709 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.596726+0000", 2024-05-08T08:52:36.709 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.03850191, 2024-05-08T08:52:36.709 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.039611988000000001, 2024-05-08T08:52:36.709 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.709 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.709 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.709 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.709 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.709 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.709 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2031 2024-05-08T08:52:36.709 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.709 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.709 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.709 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.710 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.596726+0000", 2024-05-08T08:52:36.710 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.710 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.710 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.710 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.710 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.596726+0000", 2024-05-08T08:52:36.710 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.710 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.710 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.710 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.710 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.596965+0000", 2024-05-08T08:52:36.710 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00023868000000000001 2024-05-08T08:52:36.710 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.710 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.711 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.711 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.596973+0000", 2024-05-08T08:52:36.711 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9400000000000002e-06 2024-05-08T08:52:36.711 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.711 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.711 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.711 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.596980+0000", 2024-05-08T08:52:36.711 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7190000000000001e-06 2024-05-08T08:52:36.711 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.711 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.711 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.711 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.596984+0000", 2024-05-08T08:52:36.711 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0849999999999999e-06 2024-05-08T08:52:36.711 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.711 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.712 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.712 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.712 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.712 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2095 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.712 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.604455+0000", 2024-05-08T08:52:36.712 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.030772523999999999, 2024-05-08T08:52:36.712 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.031909819999999998, 2024-05-08T08:52:36.712 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.712 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.712 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.712 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.712 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.712 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.712 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2095 2024-05-08T08:52:36.712 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.712 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.712 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.713 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.713 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.604455+0000", 2024-05-08T08:52:36.713 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.713 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.713 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.713 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.713 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.604455+0000", 2024-05-08T08:52:36.713 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.713 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.713 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.713 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.713 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.604456+0000", 2024-05-08T08:52:36.713 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0699999999999997e-07 2024-05-08T08:52:36.714 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.714 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.714 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.714 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.604458+0000", 2024-05-08T08:52:36.714 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3910000000000001e-06 2024-05-08T08:52:36.714 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.714 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.714 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.714 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.604468+0000", 2024-05-08T08:52:36.715 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0188e-05 2024-05-08T08:52:36.715 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.715 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.715 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.715 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.604474+0000", 2024-05-08T08:52:36.715 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8459999999999996e-06 2024-05-08T08:52:36.715 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.715 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.715 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.715 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.715 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.715 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2160 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.715 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.610823+0000", 2024-05-08T08:52:36.716 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.024404427999999999, 2024-05-08T08:52:36.716 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.025573192000000002, 2024-05-08T08:52:36.716 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.716 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.716 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.716 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.716 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.716 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.716 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2160 2024-05-08T08:52:36.716 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.716 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.716 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.717 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.717 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.610823+0000", 2024-05-08T08:52:36.717 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.717 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.717 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.717 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.717 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.610823+0000", 2024-05-08T08:52:36.717 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.717 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.717 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.717 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.717 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.610959+0000", 2024-05-08T08:52:36.717 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013522100000000001 2024-05-08T08:52:36.718 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.718 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.718 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.718 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.610963+0000", 2024-05-08T08:52:36.718 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2440000000000004e-06 2024-05-08T08:52:36.718 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.718 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.718 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.718 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.610979+0000", 2024-05-08T08:52:36.718 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6504999999999999e-05 2024-05-08T08:52:36.718 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.718 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.719 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.719 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.610988+0000", 2024-05-08T08:52:36.719 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.225e-06 2024-05-08T08:52:36.719 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.719 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.719 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.719 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.719 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.719 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2225 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.719 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.617335+0000", 2024-05-08T08:52:36.719 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.017892946, 2024-05-08T08:52:36.719 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.019100071, 2024-05-08T08:52:36.719 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.719 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.720 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.720 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.720 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.720 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.720 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2225 2024-05-08T08:52:36.720 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.720 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.720 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.720 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.720 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.617335+0000", 2024-05-08T08:52:36.720 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.720 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.721 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.721 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.721 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.617335+0000", 2024-05-08T08:52:36.721 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.721 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.721 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.721 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.721 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.617551+0000", 2024-05-08T08:52:36.721 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000216152 2024-05-08T08:52:36.721 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.721 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.721 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.722 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.617555+0000", 2024-05-08T08:52:36.722 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9210000000000002e-06 2024-05-08T08:52:36.722 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.722 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.722 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.722 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.617564+0000", 2024-05-08T08:52:36.722 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4770000000000003e-06 2024-05-08T08:52:36.722 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.722 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.722 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.722 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.617571+0000", 2024-05-08T08:52:36.722 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1829999999999997e-06 2024-05-08T08:52:36.723 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.723 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.723 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.723 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.723 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.723 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2288 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.723 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.623495+0000", 2024-05-08T08:52:36.723 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.011732638, 2024-05-08T08:52:36.723 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.012968746, 2024-05-08T08:52:36.723 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.723 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.723 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.724 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.724 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.724 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.724 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2288 2024-05-08T08:52:36.724 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.724 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.724 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.724 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.724 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.623495+0000", 2024-05-08T08:52:36.724 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.724 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.724 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.725 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.725 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.623495+0000", 2024-05-08T08:52:36.725 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.725 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.725 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.725 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.725 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.623771+0000", 2024-05-08T08:52:36.725 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00027615000000000002 2024-05-08T08:52:36.725 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.725 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.725 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.725 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.623776+0000", 2024-05-08T08:52:36.726 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8300000000000003e-06 2024-05-08T08:52:36.726 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.726 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.726 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.726 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.623785+0000", 2024-05-08T08:52:36.726 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.8990000000000007e-06 2024-05-08T08:52:36.726 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.726 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.726 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.726 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.623792+0000", 2024-05-08T08:52:36.726 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7700000000000004e-06 2024-05-08T08:52:36.726 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.727 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.727 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.727 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.727 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.727 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2353 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.727 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.630427+0000", 2024-05-08T08:52:36.727 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0048010020000000004, 2024-05-08T08:52:36.727 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.006064318, 2024-05-08T08:52:36.727 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.727 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.727 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.727 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.727 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.728 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.728 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2353 2024-05-08T08:52:36.728 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.728 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.728 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.728 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.728 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.630427+0000", 2024-05-08T08:52:36.728 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.728 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.728 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.728 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.728 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.630427+0000", 2024-05-08T08:52:36.729 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.729 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.729 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.729 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.729 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.630427+0000", 2024-05-08T08:52:36.729 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1099999999999996e-07 2024-05-08T08:52:36.729 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.729 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.729 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.729 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.630430+0000", 2024-05-08T08:52:36.729 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6749999999999998e-06 2024-05-08T08:52:36.729 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.729 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.730 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.730 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.630438+0000", 2024-05-08T08:52:36.730 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9680000000000006e-06 2024-05-08T08:52:36.730 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.730 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.730 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.730 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.630444+0000", 2024-05-08T08:52:36.730 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5249999999999996e-06 2024-05-08T08:52:36.730 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.730 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.730 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.730 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.731 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.731 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1906 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.731 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.583286+0000", 2024-05-08T08:52:36.731 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.051942220999999997, 2024-05-08T08:52:36.731 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.053233530000000001, 2024-05-08T08:52:36.731 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.731 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.731 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.731 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.731 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.731 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.731 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1906 2024-05-08T08:52:36.731 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.732 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.732 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.732 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.732 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.583286+0000", 2024-05-08T08:52:36.732 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.732 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.732 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.732 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.732 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.583286+0000", 2024-05-08T08:52:36.732 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.732 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.732 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.733 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.733 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.583429+0000", 2024-05-08T08:52:36.733 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014364 2024-05-08T08:52:36.733 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.733 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.733 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.733 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.583433+0000", 2024-05-08T08:52:36.733 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9949999999999999e-06 2024-05-08T08:52:36.733 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.733 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.733 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.733 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.583442+0000", 2024-05-08T08:52:36.733 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.8599999999999999e-06 2024-05-08T08:52:36.734 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.734 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.734 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.734 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.583449+0000", 2024-05-08T08:52:36.734 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7950000000000002e-06 2024-05-08T08:52:36.734 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.734 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.734 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.734 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.734 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.734 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1970 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.734 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.590500+0000", 2024-05-08T08:52:36.735 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.044727774999999997, 2024-05-08T08:52:36.735 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.046046262999999997, 2024-05-08T08:52:36.735 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.735 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.735 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.735 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.735 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.735 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.735 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1970 2024-05-08T08:52:36.735 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.735 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.735 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.735 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.736 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.590500+0000", 2024-05-08T08:52:36.736 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.736 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.736 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.736 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.736 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.590500+0000", 2024-05-08T08:52:36.736 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.736 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.736 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.736 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.736 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.590500+0000", 2024-05-08T08:52:36.736 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.2099999999999998e-07 2024-05-08T08:52:36.737 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.737 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.737 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.737 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.590502+0000", 2024-05-08T08:52:36.737 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.511e-06 2024-05-08T08:52:36.737 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.737 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.737 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.737 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.590506+0000", 2024-05-08T08:52:36.737 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0280000000000002e-06 2024-05-08T08:52:36.737 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.737 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.737 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.738 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.590510+0000", 2024-05-08T08:52:36.738 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5410000000000001e-06 2024-05-08T08:52:36.738 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.738 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.738 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.738 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.738 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.738 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2033 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.738 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.596992+0000", 2024-05-08T08:52:36.738 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.038235462999999997, 2024-05-08T08:52:36.738 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.039581491000000003, 2024-05-08T08:52:36.739 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.739 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.739 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.739 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.739 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.739 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.739 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2033 2024-05-08T08:52:36.739 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.739 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.739 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.739 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.739 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.596992+0000", 2024-05-08T08:52:36.739 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.740 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.740 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.740 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.740 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.596992+0000", 2024-05-08T08:52:36.740 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.740 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.740 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.740 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.740 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.597295+0000", 2024-05-08T08:52:36.740 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000302529 2024-05-08T08:52:36.740 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.740 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.741 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.741 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.597298+0000", 2024-05-08T08:52:36.741 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3909999999999998e-06 2024-05-08T08:52:36.741 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.741 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.741 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.741 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.597306+0000", 2024-05-08T08:52:36.741 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1679999999999998e-06 2024-05-08T08:52:36.741 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.741 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.741 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.741 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.597310+0000", 2024-05-08T08:52:36.741 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3850000000000004e-06 2024-05-08T08:52:36.741 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.742 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.742 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.742 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.742 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.742 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2097 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.742 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.604483+0000", 2024-05-08T08:52:36.742 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.030745393999999999, 2024-05-08T08:52:36.742 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.032118740999999999, 2024-05-08T08:52:36.742 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.742 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.742 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.742 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.743 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.743 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.743 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2097 2024-05-08T08:52:36.743 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.743 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.743 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.743 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.743 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.604483+0000", 2024-05-08T08:52:36.743 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.743 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.743 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.743 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.744 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.604483+0000", 2024-05-08T08:52:36.744 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.744 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.744 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.744 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.744 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.604565+0000", 2024-05-08T08:52:36.744 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.2238999999999996e-05 2024-05-08T08:52:36.744 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.744 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.744 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.744 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.604568+0000", 2024-05-08T08:52:36.744 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6059999999999999e-06 2024-05-08T08:52:36.744 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.745 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.745 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.745 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.604576+0000", 2024-05-08T08:52:36.745 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9519999999999994e-06 2024-05-08T08:52:36.745 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.745 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.745 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.745 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.604581+0000", 2024-05-08T08:52:36.745 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9729999999999997e-06 2024-05-08T08:52:36.745 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.745 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.745 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.745 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.746 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.746 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2162 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.746 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.610999+0000", 2024-05-08T08:52:36.746 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.024229363, 2024-05-08T08:52:36.746 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.025633857999999999, 2024-05-08T08:52:36.746 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.746 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.746 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.746 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.746 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.746 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.746 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2162 2024-05-08T08:52:36.747 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.747 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.747 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.747 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.747 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.610999+0000", 2024-05-08T08:52:36.747 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.747 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.747 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.747 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.747 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.610999+0000", 2024-05-08T08:52:36.747 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.747 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.747 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.748 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.748 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.611076+0000", 2024-05-08T08:52:36.748 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7719999999999994e-05 2024-05-08T08:52:36.748 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.748 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.748 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.748 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.611080+0000", 2024-05-08T08:52:36.748 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.738e-06 2024-05-08T08:52:36.748 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.748 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.748 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.748 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.611090+0000", 2024-05-08T08:52:36.749 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0404000000000001e-05 2024-05-08T08:52:36.749 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.749 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.749 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.749 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.611098+0000", 2024-05-08T08:52:36.749 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8269999999999998e-06 2024-05-08T08:52:36.749 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.749 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.749 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.749 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.749 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.749 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2226 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.749 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.617581+0000", 2024-05-08T08:52:36.749 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.01764659, 2024-05-08T08:52:36.750 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.019086924000000002, 2024-05-08T08:52:36.750 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.750 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.750 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.750 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.750 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.750 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.750 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2226 2024-05-08T08:52:36.750 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.750 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.750 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.751 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.751 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.617581+0000", 2024-05-08T08:52:36.751 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.751 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.751 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.751 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.751 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.617581+0000", 2024-05-08T08:52:36.751 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.751 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.751 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.751 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.751 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.617681+0000", 2024-05-08T08:52:36.751 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000100113 2024-05-08T08:52:36.752 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.752 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.752 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.752 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.617684+0000", 2024-05-08T08:52:36.752 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8619999999999999e-06 2024-05-08T08:52:36.752 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.752 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.752 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.752 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.617691+0000", 2024-05-08T08:52:36.752 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9199999999999998e-06 2024-05-08T08:52:36.752 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.752 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.753 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.753 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.617696+0000", 2024-05-08T08:52:36.753 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4000000000000002e-06 2024-05-08T08:52:36.753 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.753 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.753 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.753 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.753 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.753 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2290 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.753 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.623804+0000", 2024-05-08T08:52:36.753 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.011423579999999999, 2024-05-08T08:52:36.753 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.012892382000000001, 2024-05-08T08:52:36.754 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.754 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.754 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.754 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.754 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.754 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.754 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2290 2024-05-08T08:52:36.754 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.754 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.754 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.754 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.754 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.623804+0000", 2024-05-08T08:52:36.755 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.755 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.755 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.755 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.755 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.623804+0000", 2024-05-08T08:52:36.755 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.755 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.755 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.755 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.755 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.624033+0000", 2024-05-08T08:52:36.755 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00022851699999999999 2024-05-08T08:52:36.755 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.756 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.756 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.756 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.624038+0000", 2024-05-08T08:52:36.756 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1220000000000003e-06 2024-05-08T08:52:36.756 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.756 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.756 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.756 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.624048+0000", 2024-05-08T08:52:36.756 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.7229999999999999e-06 2024-05-08T08:52:36.756 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.756 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.756 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.756 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.624055+0000", 2024-05-08T08:52:36.757 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.267e-06 2024-05-08T08:52:36.757 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.757 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.757 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.757 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.757 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.757 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2355 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.757 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.630452+0000", 2024-05-08T08:52:36.757 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0047762880000000001, 2024-05-08T08:52:36.757 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.006271731, 2024-05-08T08:52:36.757 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.757 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.758 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.758 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.758 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.758 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.758 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2355 2024-05-08T08:52:36.758 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.758 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.758 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.758 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.758 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.630452+0000", 2024-05-08T08:52:36.758 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.759 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.759 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.759 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.759 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.630452+0000", 2024-05-08T08:52:36.759 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.759 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.759 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.759 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.759 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.630561+0000", 2024-05-08T08:52:36.759 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000109299 2024-05-08T08:52:36.759 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.759 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.760 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.760 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.630563+0000", 2024-05-08T08:52:36.760 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3120000000000001e-06 2024-05-08T08:52:36.760 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.760 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.760 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.760 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.630568+0000", 2024-05-08T08:52:36.760 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0039999999999999e-06 2024-05-08T08:52:36.760 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.760 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.760 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.760 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.630572+0000", 2024-05-08T08:52:36.760 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9140000000000001e-06 2024-05-08T08:52:36.761 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.761 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.761 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.761 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.761 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.761 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1908 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.761 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.583459+0000", 2024-05-08T08:52:36.761 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.051768574999999997, 2024-05-08T08:52:36.761 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.053291165000000001, 2024-05-08T08:52:36.761 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.761 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.761 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.762 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.762 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.762 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.762 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1908 2024-05-08T08:52:36.762 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.762 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.762 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.762 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.762 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.583459+0000", 2024-05-08T08:52:36.762 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.762 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.763 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.763 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.763 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.583459+0000", 2024-05-08T08:52:36.763 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.763 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.763 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.763 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.763 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.583679+0000", 2024-05-08T08:52:36.763 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000219521 2024-05-08T08:52:36.763 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.763 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.763 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.764 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.583683+0000", 2024-05-08T08:52:36.764 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0670000000000002e-06 2024-05-08T08:52:36.764 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.764 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.764 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.764 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.583692+0000", 2024-05-08T08:52:36.764 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.577e-06 2024-05-08T08:52:36.764 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.764 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.764 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.764 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.583699+0000", 2024-05-08T08:52:36.764 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5220000000000004e-06 2024-05-08T08:52:36.765 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.765 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.765 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.765 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.765 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.765 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1972 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.765 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.590513+0000", 2024-05-08T08:52:36.765 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.044714487999999997, 2024-05-08T08:52:36.765 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.046272323999999997, 2024-05-08T08:52:36.765 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.765 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.766 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.766 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.766 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.766 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.766 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1972 2024-05-08T08:52:36.766 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.766 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.766 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.766 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.766 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.590513+0000", 2024-05-08T08:52:36.766 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.766 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.767 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.767 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.767 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.590513+0000", 2024-05-08T08:52:36.767 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.767 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.767 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.767 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.767 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.590654+0000", 2024-05-08T08:52:36.767 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000140774 2024-05-08T08:52:36.767 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.767 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.767 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.767 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.590658+0000", 2024-05-08T08:52:36.768 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.805e-06 2024-05-08T08:52:36.768 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.768 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.768 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.768 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.590672+0000", 2024-05-08T08:52:36.768 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3929e-05 2024-05-08T08:52:36.768 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.768 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.768 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.768 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.590678+0000", 2024-05-08T08:52:36.768 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2859999999999996e-06 2024-05-08T08:52:36.768 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.769 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.769 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.769 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.769 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.769 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2035 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.769 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.597321+0000", 2024-05-08T08:52:36.769 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.037907312999999998, 2024-05-08T08:52:36.769 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.039498138000000002, 2024-05-08T08:52:36.769 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.769 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.769 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.769 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.770 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.770 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.770 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2035 2024-05-08T08:52:36.770 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.770 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.770 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.770 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.770 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.597321+0000", 2024-05-08T08:52:36.770 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.770 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.770 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.770 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.771 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.597321+0000", 2024-05-08T08:52:36.771 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.771 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.771 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.771 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.771 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.597321+0000", 2024-05-08T08:52:36.771 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1800000000000001e-07 2024-05-08T08:52:36.771 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.771 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.771 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.771 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.597323+0000", 2024-05-08T08:52:36.771 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.604e-06 2024-05-08T08:52:36.771 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.772 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.772 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.772 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.597326+0000", 2024-05-08T08:52:36.772 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3799999999999998e-06 2024-05-08T08:52:36.772 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.772 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.772 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.772 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.597329+0000", 2024-05-08T08:52:36.772 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7700000000000002e-06 2024-05-08T08:52:36.772 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.772 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.772 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.773 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.773 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.773 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2099 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.773 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.604596+0000", 2024-05-08T08:52:36.773 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.030632058, 2024-05-08T08:52:36.773 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.032250148999999999, 2024-05-08T08:52:36.773 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.773 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.773 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.773 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.773 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.773 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.774 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2099 2024-05-08T08:52:36.774 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.774 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.774 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.774 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.774 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.604596+0000", 2024-05-08T08:52:36.774 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.774 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.774 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.774 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.774 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.604596+0000", 2024-05-08T08:52:36.774 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.775 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.775 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.775 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.775 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.604959+0000", 2024-05-08T08:52:36.775 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00036355799999999999 2024-05-08T08:52:36.775 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.775 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.775 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.775 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.604964+0000", 2024-05-08T08:52:36.775 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3989999999999997e-06 2024-05-08T08:52:36.775 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.775 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.775 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.776 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.604972+0000", 2024-05-08T08:52:36.776 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.602e-06 2024-05-08T08:52:36.776 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.776 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.776 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.776 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.604979+0000", 2024-05-08T08:52:36.776 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3280000000000002e-06 2024-05-08T08:52:36.776 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.776 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.776 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.776 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.776 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.776 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2164 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.777 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.611109+0000", 2024-05-08T08:52:36.777 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.024119040000000001, 2024-05-08T08:52:36.777 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.025763814999999999, 2024-05-08T08:52:36.777 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.777 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.777 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.777 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.777 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.777 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.777 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2164 2024-05-08T08:52:36.777 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.777 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.778 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.778 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.778 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.611109+0000", 2024-05-08T08:52:36.778 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.778 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.778 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.778 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.778 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.611109+0000", 2024-05-08T08:52:36.778 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.778 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.778 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.779 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.779 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.611323+0000", 2024-05-08T08:52:36.779 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000214587 2024-05-08T08:52:36.779 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.779 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.779 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.779 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.611326+0000", 2024-05-08T08:52:36.779 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.988e-06 2024-05-08T08:52:36.779 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.779 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.779 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.779 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.611336+0000", 2024-05-08T08:52:36.779 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0939999999999997e-06 2024-05-08T08:52:36.780 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.780 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.780 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.780 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.611341+0000", 2024-05-08T08:52:36.780 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6790000000000002e-06 2024-05-08T08:52:36.780 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.780 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.780 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.780 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.780 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.780 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2228 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.780 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.617705+0000", 2024-05-08T08:52:36.780 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.017523321000000001, 2024-05-08T08:52:36.780 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.019194859000000002, 2024-05-08T08:52:36.781 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.781 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.781 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.781 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.781 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.781 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.781 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2228 2024-05-08T08:52:36.781 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.781 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.781 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.781 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.782 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.617705+0000", 2024-05-08T08:52:36.782 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.782 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.782 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.782 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.782 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.617705+0000", 2024-05-08T08:52:36.782 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.782 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.782 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.782 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.782 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.617941+0000", 2024-05-08T08:52:36.782 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00023642100000000001 2024-05-08T08:52:36.783 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.783 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.783 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.783 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.617946+0000", 2024-05-08T08:52:36.783 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6770000000000003e-06 2024-05-08T08:52:36.783 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.783 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.783 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.783 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.617963+0000", 2024-05-08T08:52:36.783 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.7110999999999999e-05 2024-05-08T08:52:36.783 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.783 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.783 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.784 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.617971+0000", 2024-05-08T08:52:36.784 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.5080000000000005e-06 2024-05-08T08:52:36.784 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.784 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.784 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.784 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.784 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.784 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2292 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.784 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.624066+0000", 2024-05-08T08:52:36.784 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.011161577000000001, 2024-05-08T08:52:36.784 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.012872929999999999, 2024-05-08T08:52:36.785 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.785 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.785 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.785 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.785 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.785 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.785 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2292 2024-05-08T08:52:36.785 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.785 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.785 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.785 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.785 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.624066+0000", 2024-05-08T08:52:36.785 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.786 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.786 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.786 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.786 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.624066+0000", 2024-05-08T08:52:36.786 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.786 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.786 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.786 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.786 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.624479+0000", 2024-05-08T08:52:36.786 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000412614 2024-05-08T08:52:36.786 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.787 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.787 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.787 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.624483+0000", 2024-05-08T08:52:36.787 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0269999999999997e-06 2024-05-08T08:52:36.787 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.787 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.787 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.787 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.624491+0000", 2024-05-08T08:52:36.787 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3270000000000001e-06 2024-05-08T08:52:36.787 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.787 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.787 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.787 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.624498+0000", 2024-05-08T08:52:36.788 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2060000000000001e-06 2024-05-08T08:52:36.788 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.788 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.788 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.788 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.788 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.788 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2357 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.788 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.630578+0000", 2024-05-08T08:52:36.788 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.004649645, 2024-05-08T08:52:36.788 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0063964080000000001, 2024-05-08T08:52:36.788 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.788 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.789 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.789 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.789 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.789 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.789 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2357 2024-05-08T08:52:36.789 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.789 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.789 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.789 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.789 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.630578+0000", 2024-05-08T08:52:36.789 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.789 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.790 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.790 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.790 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.630578+0000", 2024-05-08T08:52:36.790 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.790 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.790 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.790 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.790 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.630833+0000", 2024-05-08T08:52:36.790 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000254837 2024-05-08T08:52:36.790 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.790 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.790 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.791 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.630837+0000", 2024-05-08T08:52:36.791 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8770000000000003e-06 2024-05-08T08:52:36.791 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.791 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.791 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.791 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.630846+0000", 2024-05-08T08:52:36.791 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3200000000000006e-06 2024-05-08T08:52:36.791 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.791 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.791 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.791 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.630853+0000", 2024-05-08T08:52:36.791 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7830000000000001e-06 2024-05-08T08:52:36.792 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.792 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.792 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.792 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.792 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.792 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1910 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.792 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.583709+0000", 2024-05-08T08:52:36.792 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.051519118000000003, 2024-05-08T08:52:36.792 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.053294710000000002, 2024-05-08T08:52:36.792 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.792 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.792 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.793 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.793 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.793 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.793 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1910 2024-05-08T08:52:36.793 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.793 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.793 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.793 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.793 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.583709+0000", 2024-05-08T08:52:36.793 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.793 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.793 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.793 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.794 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.583709+0000", 2024-05-08T08:52:36.794 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.794 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.794 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.794 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.794 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.583935+0000", 2024-05-08T08:52:36.794 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00022625 2024-05-08T08:52:36.794 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.794 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.794 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.794 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.583939+0000", 2024-05-08T08:52:36.795 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1699999999999999e-06 2024-05-08T08:52:36.795 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.795 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.795 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.795 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.583948+0000", 2024-05-08T08:52:36.795 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7579999999999998e-06 2024-05-08T08:52:36.795 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.795 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.795 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.795 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.583961+0000", 2024-05-08T08:52:36.795 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2819999999999999e-05 2024-05-08T08:52:36.795 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.796 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.796 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.796 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.796 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.796 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1974 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.796 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.590697+0000", 2024-05-08T08:52:36.796 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.044531249000000002, 2024-05-08T08:52:36.796 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.046334049000000002, 2024-05-08T08:52:36.796 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.796 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.796 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.796 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.796 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.797 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.797 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1974 2024-05-08T08:52:36.797 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.797 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.797 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.797 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.797 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.590697+0000", 2024-05-08T08:52:36.797 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.797 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.797 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.797 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.797 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.590697+0000", 2024-05-08T08:52:36.798 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.798 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.798 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.798 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.798 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.590959+0000", 2024-05-08T08:52:36.798 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00026222099999999998 2024-05-08T08:52:36.798 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.798 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.798 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.798 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.590963+0000", 2024-05-08T08:52:36.798 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9779999999999999e-06 2024-05-08T08:52:36.799 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.799 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.799 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.799 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.590975+0000", 2024-05-08T08:52:36.799 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2595999999999999e-05 2024-05-08T08:52:36.799 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.799 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.799 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.799 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.590980+0000", 2024-05-08T08:52:36.799 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3229999999999999e-06 2024-05-08T08:52:36.799 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.799 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.799 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.800 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.800 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.800 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2037 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.800 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.597333+0000", 2024-05-08T08:52:36.800 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.037895024999999999, 2024-05-08T08:52:36.800 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.039736072999999997, 2024-05-08T08:52:36.800 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.800 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.800 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.800 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.800 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.800 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.801 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2037 2024-05-08T08:52:36.801 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.801 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.801 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.801 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.801 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.597333+0000", 2024-05-08T08:52:36.801 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.801 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.801 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.801 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.801 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.597333+0000", 2024-05-08T08:52:36.801 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.802 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.802 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.802 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.802 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.597570+0000", 2024-05-08T08:52:36.802 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00023747699999999999 2024-05-08T08:52:36.802 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.802 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.802 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.802 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.597575+0000", 2024-05-08T08:52:36.802 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2520000000000001e-06 2024-05-08T08:52:36.802 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.803 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.803 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.803 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.597585+0000", 2024-05-08T08:52:36.803 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0540000000000001e-05 2024-05-08T08:52:36.803 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.803 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.803 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.803 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.597592+0000", 2024-05-08T08:52:36.803 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8190000000000003e-06 2024-05-08T08:52:36.803 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.803 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.803 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.803 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.804 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.804 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2101 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.804 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.604990+0000", 2024-05-08T08:52:36.804 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.030238290000000001, 2024-05-08T08:52:36.804 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.032134891999999998, 2024-05-08T08:52:36.804 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.804 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.804 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.804 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.804 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.804 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.804 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2101 2024-05-08T08:52:36.805 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.805 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.805 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.805 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.805 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.604990+0000", 2024-05-08T08:52:36.805 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.805 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.805 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.805 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.805 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.604990+0000", 2024-05-08T08:52:36.805 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.805 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.805 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.806 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.806 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.605235+0000", 2024-05-08T08:52:36.806 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00024503200000000001 2024-05-08T08:52:36.806 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.806 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.806 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.806 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.605239+0000", 2024-05-08T08:52:36.806 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9829999999999998e-06 2024-05-08T08:52:36.806 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.806 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.806 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.806 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.605249+0000", 2024-05-08T08:52:36.807 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.042e-05 2024-05-08T08:52:36.807 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.807 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.807 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.807 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.605257+0000", 2024-05-08T08:52:36.807 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7379999999999994e-06 2024-05-08T08:52:36.807 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.807 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.807 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.807 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.807 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.807 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2166 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.807 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.611358+0000", 2024-05-08T08:52:36.808 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.023870025, 2024-05-08T08:52:36.808 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.025823342999999999, 2024-05-08T08:52:36.808 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.808 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.808 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.808 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.808 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.808 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.808 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2166 2024-05-08T08:52:36.808 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.808 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.808 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.808 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.809 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.611358+0000", 2024-05-08T08:52:36.809 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.809 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.809 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.809 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.809 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.611358+0000", 2024-05-08T08:52:36.809 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.809 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.809 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.809 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.809 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.611358+0000", 2024-05-08T08:52:36.809 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8100000000000003e-07 2024-05-08T08:52:36.809 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.809 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.810 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.810 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.611364+0000", 2024-05-08T08:52:36.810 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8359999999999997e-06 2024-05-08T08:52:36.810 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.810 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.810 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.810 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.611378+0000", 2024-05-08T08:52:36.810 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3835000000000001e-05 2024-05-08T08:52:36.810 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.810 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.810 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.810 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.611385+0000", 2024-05-08T08:52:36.811 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4289999999999998e-06 2024-05-08T08:52:36.811 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.811 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.811 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.811 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.811 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.811 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2231 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.811 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.617983+0000", 2024-05-08T08:52:36.811 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.01724477, 2024-05-08T08:52:36.811 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.019256016000000001, 2024-05-08T08:52:36.811 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.811 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.811 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.812 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.812 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.812 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.812 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2231 2024-05-08T08:52:36.812 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.812 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.812 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.812 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.812 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.617983+0000", 2024-05-08T08:52:36.812 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.812 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.813 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.813 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.813 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.617983+0000", 2024-05-08T08:52:36.813 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.813 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.813 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.813 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.813 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.617984+0000", 2024-05-08T08:52:36.813 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9800000000000003e-07 2024-05-08T08:52:36.813 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.813 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.813 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.813 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.617987+0000", 2024-05-08T08:52:36.814 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9129999999999999e-06 2024-05-08T08:52:36.814 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.814 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.814 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.814 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.617994+0000", 2024-05-08T08:52:36.814 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7970000000000001e-06 2024-05-08T08:52:36.814 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.814 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.814 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.814 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.618000+0000", 2024-05-08T08:52:36.814 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6500000000000001e-06 2024-05-08T08:52:36.814 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.815 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.815 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.815 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.815 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.815 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2294 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.815 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.624510+0000", 2024-05-08T08:52:36.815 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.010718373, 2024-05-08T08:52:36.815 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.012784480000000001, 2024-05-08T08:52:36.815 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.815 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.815 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.816 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.816 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.816 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.816 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2294 2024-05-08T08:52:36.816 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.816 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.816 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.816 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.816 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.624510+0000", 2024-05-08T08:52:36.816 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.816 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.816 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.816 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.817 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.624510+0000", 2024-05-08T08:52:36.817 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.817 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.817 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.817 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.817 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.624510+0000", 2024-05-08T08:52:36.817 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0699999999999997e-07 2024-05-08T08:52:36.817 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.817 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.817 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.817 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.624521+0000", 2024-05-08T08:52:36.817 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1089e-05 2024-05-08T08:52:36.818 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.818 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.818 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.818 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.624529+0000", 2024-05-08T08:52:36.818 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7540000000000006e-06 2024-05-08T08:52:36.818 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.818 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.818 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.818 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.624535+0000", 2024-05-08T08:52:36.818 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8769999999999998e-06 2024-05-08T08:52:36.818 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.818 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.819 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.819 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.819 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.819 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2359 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.819 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.630878+0000", 2024-05-08T08:52:36.819 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0043501080000000001, 2024-05-08T08:52:36.819 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0064449169999999997, 2024-05-08T08:52:36.819 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.819 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.819 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.819 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.819 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.819 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.820 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2359 2024-05-08T08:52:36.820 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.820 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.820 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.820 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.820 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.630878+0000", 2024-05-08T08:52:36.820 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.820 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.820 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.820 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.820 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.630878+0000", 2024-05-08T08:52:36.820 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.821 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.821 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.821 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.821 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.630878+0000", 2024-05-08T08:52:36.821 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4700000000000001e-07 2024-05-08T08:52:36.821 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.821 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.821 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.821 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.630881+0000", 2024-05-08T08:52:36.821 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.909e-06 2024-05-08T08:52:36.821 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.822 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.822 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.822 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.630889+0000", 2024-05-08T08:52:36.822 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.2160000000000006e-06 2024-05-08T08:52:36.822 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.822 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.822 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.822 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.630895+0000", 2024-05-08T08:52:36.822 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8170000000000004e-06 2024-05-08T08:52:36.822 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.822 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.822 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.823 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.823 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.823 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1912 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 10997760~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.823 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.583975+0000", 2024-05-08T08:52:36.823 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.051252773000000001, 2024-05-08T08:52:36.823 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.053384781999999999, 2024-05-08T08:52:36.823 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.823 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.823 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:36.823 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.823 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.823 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.823 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1912 2024-05-08T08:52:36.824 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.824 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.824 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.824 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.824 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.583975+0000", 2024-05-08T08:52:36.824 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.824 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.824 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.824 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.824 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.583975+0000", 2024-05-08T08:52:36.824 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.824 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.825 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.825 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.825 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.584192+0000", 2024-05-08T08:52:36.825 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00021662 2024-05-08T08:52:36.825 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.825 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.825 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.825 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.584203+0000", 2024-05-08T08:52:36.825 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1564e-05 2024-05-08T08:52:36.825 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.825 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.825 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.825 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.584212+0000", 2024-05-08T08:52:36.826 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0769999999999997e-06 2024-05-08T08:52:36.826 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.826 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.826 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.826 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.584219+0000", 2024-05-08T08:52:36.826 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5799999999999997e-06 2024-05-08T08:52:36.826 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.826 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.826 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:36.826 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.637311+0000", 2024-05-08T08:52:36.826 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.053092036000000002 2024-05-08T08:52:36.826 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.827 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.827 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:36.827 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.637363+0000", 2024-05-08T08:52:36.827 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1755000000000002e-05 2024-05-08T08:52:36.827 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.827 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.827 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.827 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.827 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.827 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1976 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.827 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.590988+0000", 2024-05-08T08:52:36.827 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.044239528, 2024-05-08T08:52:36.828 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.046408221999999999, 2024-05-08T08:52:36.828 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.828 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.828 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.828 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.828 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.828 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.828 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1976 2024-05-08T08:52:36.828 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.828 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.828 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.828 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.829 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.590988+0000", 2024-05-08T08:52:36.829 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.829 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.829 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.829 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.829 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.590988+0000", 2024-05-08T08:52:36.829 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.829 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.829 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.829 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.829 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.591257+0000", 2024-05-08T08:52:36.829 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00026833400000000001 2024-05-08T08:52:36.829 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.829 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.830 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.830 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.591261+0000", 2024-05-08T08:52:36.830 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9899999999999999e-06 2024-05-08T08:52:36.830 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.830 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.830 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.830 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.591272+0000", 2024-05-08T08:52:36.830 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0913e-05 2024-05-08T08:52:36.830 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.830 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.830 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.830 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.591280+0000", 2024-05-08T08:52:36.831 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.8729999999999996e-06 2024-05-08T08:52:36.831 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.831 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.831 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.831 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.831 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.831 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2039 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.831 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.597603+0000", 2024-05-08T08:52:36.831 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.037625100000000002, 2024-05-08T08:52:36.831 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.039821434000000003, 2024-05-08T08:52:36.831 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.831 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.831 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.832 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.832 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.832 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.832 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2039 2024-05-08T08:52:36.832 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.832 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.832 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.832 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.832 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.597603+0000", 2024-05-08T08:52:36.832 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.832 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.833 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.833 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.833 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.597603+0000", 2024-05-08T08:52:36.833 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.833 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.833 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.833 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.833 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.598028+0000", 2024-05-08T08:52:36.833 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00042475599999999998 2024-05-08T08:52:36.833 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.833 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.833 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.833 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.598032+0000", 2024-05-08T08:52:36.834 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6410000000000001e-06 2024-05-08T08:52:36.834 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.834 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.834 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.834 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.598043+0000", 2024-05-08T08:52:36.834 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1028e-05 2024-05-08T08:52:36.834 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.834 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.834 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.834 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.598051+0000", 2024-05-08T08:52:36.834 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5599999999999996e-06 2024-05-08T08:52:36.835 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.835 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.835 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.835 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.835 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.835 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2103 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.835 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.605268+0000", 2024-05-08T08:52:36.835 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.029959992000000001, 2024-05-08T08:52:36.835 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.032208220000000003, 2024-05-08T08:52:36.835 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.835 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.835 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.836 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.836 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.836 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.836 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2103 2024-05-08T08:52:36.836 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.836 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.836 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.836 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.836 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.605268+0000", 2024-05-08T08:52:36.836 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.836 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.836 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.836 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.837 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.605268+0000", 2024-05-08T08:52:36.837 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.837 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.837 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.837 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.837 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.605268+0000", 2024-05-08T08:52:36.837 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6599999999999996e-07 2024-05-08T08:52:36.837 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.837 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.837 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.837 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.605278+0000", 2024-05-08T08:52:36.837 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5740000000000002e-06 2024-05-08T08:52:36.838 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.838 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.838 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.838 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.605285+0000", 2024-05-08T08:52:36.838 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2989999999999999e-06 2024-05-08T08:52:36.838 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.838 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.838 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.838 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.605291+0000", 2024-05-08T08:52:36.838 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9009999999999999e-06 2024-05-08T08:52:36.838 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.838 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.839 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.839 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.839 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.839 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2168 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.839 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.611394+0000", 2024-05-08T08:52:36.839 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.023834226, 2024-05-08T08:52:36.839 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.026138431, 2024-05-08T08:52:36.839 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.839 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.839 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.839 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.839 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.839 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.840 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2168 2024-05-08T08:52:36.840 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.840 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.840 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.840 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.840 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.611394+0000", 2024-05-08T08:52:36.840 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.840 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.840 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.840 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.840 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.611394+0000", 2024-05-08T08:52:36.840 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.840 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.841 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.841 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.841 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.611838+0000", 2024-05-08T08:52:36.841 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00044431499999999998 2024-05-08T08:52:36.841 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.841 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.841 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.841 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.611841+0000", 2024-05-08T08:52:36.841 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3869999999999999e-06 2024-05-08T08:52:36.841 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.841 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.841 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.842 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.611849+0000", 2024-05-08T08:52:36.842 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2049999999999996e-06 2024-05-08T08:52:36.842 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.842 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.842 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.842 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.611853+0000", 2024-05-08T08:52:36.842 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7310000000000001e-06 2024-05-08T08:52:36.842 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.842 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.842 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.842 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.842 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.843 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2233 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.843 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.618008+0000", 2024-05-08T08:52:36.843 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.017220073999999998, 2024-05-08T08:52:36.843 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.019580046, 2024-05-08T08:52:36.843 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.843 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.843 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.843 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.843 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.843 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.843 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2233 2024-05-08T08:52:36.843 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.843 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.844 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.844 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.844 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.618008+0000", 2024-05-08T08:52:36.844 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.844 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.844 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.844 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.844 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.618008+0000", 2024-05-08T08:52:36.844 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.844 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.844 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.844 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.845 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.618195+0000", 2024-05-08T08:52:36.845 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000187441 2024-05-08T08:52:36.845 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.845 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.845 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.845 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.618199+0000", 2024-05-08T08:52:36.845 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9199999999999997e-06 2024-05-08T08:52:36.845 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.845 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.845 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.845 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.618208+0000", 2024-05-08T08:52:36.845 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2769999999999994e-06 2024-05-08T08:52:36.845 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.846 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.846 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.846 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.618216+0000", 2024-05-08T08:52:36.846 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3900000000000004e-06 2024-05-08T08:52:36.846 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.846 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.846 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.846 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.846 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.846 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2296 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.846 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.624543+0000", 2024-05-08T08:52:36.846 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.010684585999999999, 2024-05-08T08:52:36.847 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.013099433000000001, 2024-05-08T08:52:36.847 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.847 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.847 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.847 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.847 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.847 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.847 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2296 2024-05-08T08:52:36.847 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.847 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.847 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.847 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.848 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.624543+0000", 2024-05-08T08:52:36.848 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.848 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.848 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.848 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.848 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.624543+0000", 2024-05-08T08:52:36.848 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.848 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.848 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.848 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.848 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.624832+0000", 2024-05-08T08:52:36.848 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00028886500000000002 2024-05-08T08:52:36.848 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.849 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.849 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.849 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.624835+0000", 2024-05-08T08:52:36.849 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.315e-06 2024-05-08T08:52:36.849 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.849 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.849 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.849 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.624843+0000", 2024-05-08T08:52:36.849 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0140000000000001e-06 2024-05-08T08:52:36.849 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.849 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.849 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.849 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.624847+0000", 2024-05-08T08:52:36.850 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5539999999999999e-06 2024-05-08T08:52:36.850 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.850 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.850 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.850 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.850 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.850 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2361 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.850 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.630903+0000", 2024-05-08T08:52:36.850 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0043244700000000004, 2024-05-08T08:52:36.850 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0067947049999999998, 2024-05-08T08:52:36.850 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.851 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.851 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.851 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.851 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.851 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.851 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2361 2024-05-08T08:52:36.851 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.851 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.851 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.851 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.851 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.630903+0000", 2024-05-08T08:52:36.851 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.851 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.852 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.852 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.852 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.630903+0000", 2024-05-08T08:52:36.852 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.852 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.852 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.852 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.852 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.631330+0000", 2024-05-08T08:52:36.852 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00042650700000000001 2024-05-08T08:52:36.852 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.852 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.852 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.853 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.631334+0000", 2024-05-08T08:52:36.853 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1790000000000001e-06 2024-05-08T08:52:36.853 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.853 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.853 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.853 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.631344+0000", 2024-05-08T08:52:36.853 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0308e-05 2024-05-08T08:52:36.853 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.853 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.853 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.853 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.631352+0000", 2024-05-08T08:52:36.853 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3849999999999996e-06 2024-05-08T08:52:36.854 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.854 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.854 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.854 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.854 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.854 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1978 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.854 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.591292+0000", 2024-05-08T08:52:36.854 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.043935536999999997, 2024-05-08T08:52:36.854 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.046464085000000002, 2024-05-08T08:52:36.854 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.854 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.854 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.855 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.855 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.855 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.855 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1978 2024-05-08T08:52:36.855 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.855 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.855 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.855 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.855 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.591292+0000", 2024-05-08T08:52:36.855 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.855 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.855 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.856 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.856 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.591292+0000", 2024-05-08T08:52:36.856 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.856 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.856 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.856 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.856 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.591293+0000", 2024-05-08T08:52:36.856 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6300000000000005e-07 2024-05-08T08:52:36.856 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.856 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.856 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.856 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.591296+0000", 2024-05-08T08:52:36.857 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.61e-06 2024-05-08T08:52:36.857 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.857 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.857 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.857 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.591304+0000", 2024-05-08T08:52:36.857 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.2870000000000004e-06 2024-05-08T08:52:36.857 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.857 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.857 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.857 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.591310+0000", 2024-05-08T08:52:36.857 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0340000000000002e-06 2024-05-08T08:52:36.857 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.857 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.858 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.858 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.858 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.858 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2041 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.858 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.598062+0000", 2024-05-08T08:52:36.858 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.037165609000000002, 2024-05-08T08:52:36.858 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.039749932000000002, 2024-05-08T08:52:36.858 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.858 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.858 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.858 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.859 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.859 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.859 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2041 2024-05-08T08:52:36.859 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.859 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.859 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.859 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.859 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.598062+0000", 2024-05-08T08:52:36.859 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.859 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.859 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.859 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.859 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.598062+0000", 2024-05-08T08:52:36.860 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.860 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.860 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.860 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.860 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.598063+0000", 2024-05-08T08:52:36.860 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3600000000000004e-07 2024-05-08T08:52:36.860 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.860 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.860 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.860 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.598065+0000", 2024-05-08T08:52:36.860 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.5519999999999999e-06 2024-05-08T08:52:36.860 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.860 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.861 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.861 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.598073+0000", 2024-05-08T08:52:36.861 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7309999999999993e-06 2024-05-08T08:52:36.861 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.861 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.861 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.861 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.598079+0000", 2024-05-08T08:52:36.861 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1059999999999996e-06 2024-05-08T08:52:36.861 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.861 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.861 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.861 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.861 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.862 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2105 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.862 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.605299+0000", 2024-05-08T08:52:36.862 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.029928732, 2024-05-08T08:52:36.862 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.032568672, 2024-05-08T08:52:36.862 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.862 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.862 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.862 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.862 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.862 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.862 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2105 2024-05-08T08:52:36.862 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.863 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.863 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.863 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.863 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.605299+0000", 2024-05-08T08:52:36.863 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.863 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.863 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.863 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.863 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.605299+0000", 2024-05-08T08:52:36.863 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.863 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.863 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.863 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.864 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.605621+0000", 2024-05-08T08:52:36.864 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00032197500000000001 2024-05-08T08:52:36.864 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.864 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.864 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.864 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.605625+0000", 2024-05-08T08:52:36.864 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1559999999999997e-06 2024-05-08T08:52:36.864 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.864 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.864 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.864 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.605636+0000", 2024-05-08T08:52:36.864 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1063999999999999e-05 2024-05-08T08:52:36.865 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.865 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.865 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.865 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.605643+0000", 2024-05-08T08:52:36.865 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7569999999999998e-06 2024-05-08T08:52:36.865 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.865 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.865 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.865 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.865 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.865 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2170 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.865 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.611861+0000", 2024-05-08T08:52:36.865 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.023366443000000001, 2024-05-08T08:52:36.865 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.026036255000000001, 2024-05-08T08:52:36.866 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.866 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.866 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.866 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.866 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.866 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.866 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2170 2024-05-08T08:52:36.866 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.866 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.866 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.866 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.866 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.611861+0000", 2024-05-08T08:52:36.867 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.867 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.867 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.867 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.867 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.611861+0000", 2024-05-08T08:52:36.867 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.867 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.867 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.867 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.867 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.611862+0000", 2024-05-08T08:52:36.867 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5499999999999999e-07 2024-05-08T08:52:36.867 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.868 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.868 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.868 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.611863+0000", 2024-05-08T08:52:36.868 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.454e-06 2024-05-08T08:52:36.868 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.868 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.868 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.868 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.611867+0000", 2024-05-08T08:52:36.868 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7110000000000001e-06 2024-05-08T08:52:36.868 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.868 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.868 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.868 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.611870+0000", 2024-05-08T08:52:36.869 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8109999999999999e-06 2024-05-08T08:52:36.869 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.869 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.869 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.869 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.869 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.869 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2235 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.869 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.618228+0000", 2024-05-08T08:52:36.869 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.016999486000000001, 2024-05-08T08:52:36.869 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.019696689, 2024-05-08T08:52:36.869 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.869 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.869 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.870 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.870 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.870 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.870 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2235 2024-05-08T08:52:36.870 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.870 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.870 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.870 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.870 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.618228+0000", 2024-05-08T08:52:36.870 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.870 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.871 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.871 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.871 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.618228+0000", 2024-05-08T08:52:36.871 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.871 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.871 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.871 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.871 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.618229+0000", 2024-05-08T08:52:36.871 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8400000000000004e-07 2024-05-08T08:52:36.871 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.871 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.871 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.871 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.618232+0000", 2024-05-08T08:52:36.872 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.74e-06 2024-05-08T08:52:36.872 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.872 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.872 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.872 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.618239+0000", 2024-05-08T08:52:36.872 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3089999999999998e-06 2024-05-08T08:52:36.872 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.872 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.872 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.872 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.618245+0000", 2024-05-08T08:52:36.872 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8819999999999998e-06 2024-05-08T08:52:36.872 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.872 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.873 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.873 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.873 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.873 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2298 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.873 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.624855+0000", 2024-05-08T08:52:36.873 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.010373106999999999, 2024-05-08T08:52:36.873 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.013097123, 2024-05-08T08:52:36.873 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.873 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.873 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.873 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.873 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.873 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.874 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2298 2024-05-08T08:52:36.874 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.874 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.874 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.874 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.874 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.624855+0000", 2024-05-08T08:52:36.874 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.874 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.874 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.874 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.874 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.624855+0000", 2024-05-08T08:52:36.875 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.875 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.875 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.875 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.875 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.624967+0000", 2024-05-08T08:52:36.875 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000112397 2024-05-08T08:52:36.875 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.875 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.875 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.875 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.624971+0000", 2024-05-08T08:52:36.875 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5899999999999999e-06 2024-05-08T08:52:36.875 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.876 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.876 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.876 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.624978+0000", 2024-05-08T08:52:36.876 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7020000000000001e-06 2024-05-08T08:52:36.876 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.876 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.876 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.876 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.624984+0000", 2024-05-08T08:52:36.876 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5779999999999998e-06 2024-05-08T08:52:36.876 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.876 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.876 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.876 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.876 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.877 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2363 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.877 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.631365+0000", 2024-05-08T08:52:36.877 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.003863235, 2024-05-08T08:52:36.877 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0066202220000000003, 2024-05-08T08:52:36.877 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.877 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.877 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.877 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.877 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.877 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.877 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2363 2024-05-08T08:52:36.877 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.877 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.878 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.878 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.878 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.631365+0000", 2024-05-08T08:52:36.878 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.878 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.878 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.878 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.878 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.631365+0000", 2024-05-08T08:52:36.878 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.878 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.878 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.879 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.879 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.631365+0000", 2024-05-08T08:52:36.879 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1799999999999995e-07 2024-05-08T08:52:36.879 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.879 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.879 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.879 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.631368+0000", 2024-05-08T08:52:36.879 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6450000000000001e-06 2024-05-08T08:52:36.879 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.879 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.879 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.879 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.631375+0000", 2024-05-08T08:52:36.879 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5660000000000001e-06 2024-05-08T08:52:36.879 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.880 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.880 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.880 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.631381+0000", 2024-05-08T08:52:36.880 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8499999999999999e-06 2024-05-08T08:52:36.880 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.880 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.880 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.880 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.880 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.880 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1980 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.880 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.591329+0000", 2024-05-08T08:52:36.880 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.043899187999999999, 2024-05-08T08:52:36.881 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.046690833000000001, 2024-05-08T08:52:36.881 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.881 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.881 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.881 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.881 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.881 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.881 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1980 2024-05-08T08:52:36.881 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.881 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.881 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.881 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.881 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.591329+0000", 2024-05-08T08:52:36.882 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.882 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.882 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.882 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.882 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.591329+0000", 2024-05-08T08:52:36.882 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.882 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.882 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.882 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.882 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.591541+0000", 2024-05-08T08:52:36.882 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00021211299999999999 2024-05-08T08:52:36.882 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.883 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.883 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.883 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.591545+0000", 2024-05-08T08:52:36.883 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2139999999999998e-06 2024-05-08T08:52:36.883 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.883 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.883 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.883 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.591556+0000", 2024-05-08T08:52:36.883 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0807e-05 2024-05-08T08:52:36.883 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.883 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.884 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.884 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.591564+0000", 2024-05-08T08:52:36.884 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.6839999999999996e-06 2024-05-08T08:52:36.884 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.884 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.884 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.884 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.884 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.884 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2043 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.884 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.598087+0000", 2024-05-08T08:52:36.884 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.037140790999999999, 2024-05-08T08:52:36.884 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.039981379999999997, 2024-05-08T08:52:36.885 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.885 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.885 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.885 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.885 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.885 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.885 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2043 2024-05-08T08:52:36.885 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.885 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.885 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.885 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.885 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.598087+0000", 2024-05-08T08:52:36.886 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.886 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.886 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.886 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.886 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.598087+0000", 2024-05-08T08:52:36.886 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.886 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.886 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.886 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.886 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.598209+0000", 2024-05-08T08:52:36.886 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012235600000000001 2024-05-08T08:52:36.886 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.887 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.887 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.887 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.598213+0000", 2024-05-08T08:52:36.887 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8199999999999998e-06 2024-05-08T08:52:36.887 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.887 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.887 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.887 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.598222+0000", 2024-05-08T08:52:36.887 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.2400000000000007e-06 2024-05-08T08:52:36.887 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.887 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.887 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.888 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.598228+0000", 2024-05-08T08:52:36.888 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0939999999999996e-06 2024-05-08T08:52:36.888 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.888 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.888 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.888 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.888 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.888 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2107 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.888 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.605656+0000", 2024-05-08T08:52:36.888 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.029572290000000001, 2024-05-08T08:52:36.888 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.032441143999999998, 2024-05-08T08:52:36.888 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.889 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.889 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.889 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.889 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.889 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.889 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2107 2024-05-08T08:52:36.889 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.889 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.889 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.889 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.889 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.605656+0000", 2024-05-08T08:52:36.890 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.890 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.890 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.890 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.890 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.605656+0000", 2024-05-08T08:52:36.890 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.890 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.890 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.890 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.890 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.605656+0000", 2024-05-08T08:52:36.890 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6100000000000001e-07 2024-05-08T08:52:36.891 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.891 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.891 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.891 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.605659+0000", 2024-05-08T08:52:36.891 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4990000000000001e-06 2024-05-08T08:52:36.891 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.891 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.891 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.891 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.605673+0000", 2024-05-08T08:52:36.891 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.4783999999999999e-05 2024-05-08T08:52:36.891 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.891 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.891 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.892 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.605680+0000", 2024-05-08T08:52:36.892 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.285e-06 2024-05-08T08:52:36.892 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.892 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.892 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.892 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.892 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.892 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2172 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.892 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.611874+0000", 2024-05-08T08:52:36.892 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.023353538, 2024-05-08T08:52:36.892 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.026259286999999999, 2024-05-08T08:52:36.892 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.892 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.892 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.893 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.893 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.893 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.893 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2172 2024-05-08T08:52:36.893 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.893 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.893 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.893 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.893 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.611874+0000", 2024-05-08T08:52:36.893 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.893 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.893 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.893 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.894 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.611874+0000", 2024-05-08T08:52:36.894 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.894 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.894 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.894 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.894 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.612223+0000", 2024-05-08T08:52:36.894 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00034822999999999999 2024-05-08T08:52:36.894 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.894 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.894 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.894 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.612226+0000", 2024-05-08T08:52:36.894 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4860000000000002e-06 2024-05-08T08:52:36.895 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.895 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.895 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.895 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.612233+0000", 2024-05-08T08:52:36.895 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3839999999999999e-06 2024-05-08T08:52:36.895 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.895 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.895 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.895 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.612238+0000", 2024-05-08T08:52:36.895 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5129999999999998e-06 2024-05-08T08:52:36.895 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.895 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.895 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.896 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.896 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.896 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2237 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.896 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.618255+0000", 2024-05-08T08:52:36.896 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.016973062000000001, 2024-05-08T08:52:36.896 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.01993381, 2024-05-08T08:52:36.896 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.896 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.896 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.896 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.896 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.896 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.896 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2237 2024-05-08T08:52:36.896 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.897 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.897 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.897 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.897 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.618255+0000", 2024-05-08T08:52:36.897 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.897 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.897 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.897 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.897 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.618255+0000", 2024-05-08T08:52:36.897 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.897 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.897 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.897 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.897 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.618478+0000", 2024-05-08T08:52:36.898 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000223358 2024-05-08T08:52:36.898 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.898 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.898 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.898 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.618482+0000", 2024-05-08T08:52:36.898 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7749999999999999e-06 2024-05-08T08:52:36.898 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.898 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.898 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.898 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.618492+0000", 2024-05-08T08:52:36.898 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.7140000000000005e-06 2024-05-08T08:52:36.898 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.899 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.899 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.899 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.618499+0000", 2024-05-08T08:52:36.899 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0559999999999999e-06 2024-05-08T08:52:36.899 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.899 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.899 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.899 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.899 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.899 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2300 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.899 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.624994+0000", 2024-05-08T08:52:36.899 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.010234108, 2024-05-08T08:52:36.900 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.013329773, 2024-05-08T08:52:36.900 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.900 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.900 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.900 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.900 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.900 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.900 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2300 2024-05-08T08:52:36.900 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.900 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.900 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.900 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.900 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.624994+0000", 2024-05-08T08:52:36.901 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.901 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.901 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.901 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.901 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.624994+0000", 2024-05-08T08:52:36.901 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.901 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.901 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.901 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.901 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.625218+0000", 2024-05-08T08:52:36.901 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00022402699999999999 2024-05-08T08:52:36.901 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.901 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.902 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.902 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.625222+0000", 2024-05-08T08:52:36.902 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4179999999999999e-06 2024-05-08T08:52:36.902 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.902 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.902 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.902 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.625232+0000", 2024-05-08T08:52:36.902 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3880000000000006e-06 2024-05-08T08:52:36.902 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.902 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.902 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.902 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.625238+0000", 2024-05-08T08:52:36.903 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8369999999999999e-06 2024-05-08T08:52:36.903 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.903 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.903 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.903 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.903 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.903 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2365 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.903 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.631390+0000", 2024-05-08T08:52:36.903 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0038380630000000001, 2024-05-08T08:52:36.903 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0069750259999999996, 2024-05-08T08:52:36.903 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.903 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.904 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.904 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.904 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.904 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.904 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2365 2024-05-08T08:52:36.904 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.904 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.904 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.904 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.904 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.631390+0000", 2024-05-08T08:52:36.904 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.904 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.905 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.905 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.905 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.631390+0000", 2024-05-08T08:52:36.905 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.905 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.905 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.905 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.905 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.631433+0000", 2024-05-08T08:52:36.905 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3529000000000003e-05 2024-05-08T08:52:36.905 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.905 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.905 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.905 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.631437+0000", 2024-05-08T08:52:36.906 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.467e-06 2024-05-08T08:52:36.906 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.906 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.906 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.906 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.631445+0000", 2024-05-08T08:52:36.906 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.5129999999999996e-06 2024-05-08T08:52:36.906 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.906 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.906 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.906 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.631452+0000", 2024-05-08T08:52:36.906 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6409999999999996e-06 2024-05-08T08:52:36.906 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.906 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.907 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.907 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.907 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.907 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1982 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.907 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.591575+0000", 2024-05-08T08:52:36.907 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.043652651000000001, 2024-05-08T08:52:36.907 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.046828024000000003, 2024-05-08T08:52:36.907 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.907 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.907 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.907 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.907 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.907 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.908 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1982 2024-05-08T08:52:36.908 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.908 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.908 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.908 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.908 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.591575+0000", 2024-05-08T08:52:36.908 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.908 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.908 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.908 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.908 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.591575+0000", 2024-05-08T08:52:36.908 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.908 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.909 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.909 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.909 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.591576+0000", 2024-05-08T08:52:36.909 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4099999999999998e-07 2024-05-08T08:52:36.909 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.909 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.909 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.909 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.591579+0000", 2024-05-08T08:52:36.909 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.633e-06 2024-05-08T08:52:36.909 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.909 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.909 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.909 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.591586+0000", 2024-05-08T08:52:36.910 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5020000000000003e-06 2024-05-08T08:52:36.910 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.910 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.910 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.910 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.591592+0000", 2024-05-08T08:52:36.910 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2450000000000003e-06 2024-05-08T08:52:36.910 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.910 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.910 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.910 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.910 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.910 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2045 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.910 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.598238+0000", 2024-05-08T08:52:36.911 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.036989826000000003, 2024-05-08T08:52:36.911 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.040207001999999999, 2024-05-08T08:52:36.911 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.911 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.911 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.911 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.911 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.911 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.911 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2045 2024-05-08T08:52:36.911 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.911 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.911 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.912 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.912 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.598238+0000", 2024-05-08T08:52:36.912 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.912 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.912 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.912 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.912 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.598238+0000", 2024-05-08T08:52:36.912 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.912 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.912 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.912 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.912 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.598628+0000", 2024-05-08T08:52:36.912 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00038955800000000003 2024-05-08T08:52:36.913 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.913 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.913 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.913 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.598632+0000", 2024-05-08T08:52:36.913 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0969999999999999e-06 2024-05-08T08:52:36.913 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.913 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.913 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.913 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.598641+0000", 2024-05-08T08:52:36.913 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.8920000000000006e-06 2024-05-08T08:52:36.913 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.913 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.913 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.914 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.598647+0000", 2024-05-08T08:52:36.914 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5100000000000004e-06 2024-05-08T08:52:36.914 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.914 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.914 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.914 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.914 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.914 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2109 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.914 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.605689+0000", 2024-05-08T08:52:36.914 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.029538687000000001, 2024-05-08T08:52:36.914 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.032784213, 2024-05-08T08:52:36.914 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.915 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.915 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.915 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.915 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.915 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.915 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2109 2024-05-08T08:52:36.915 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.915 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.915 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.915 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.915 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.605689+0000", 2024-05-08T08:52:36.916 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.916 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.916 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.916 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.916 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.605689+0000", 2024-05-08T08:52:36.916 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.916 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.916 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.916 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.916 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.605760+0000", 2024-05-08T08:52:36.916 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0673999999999995e-05 2024-05-08T08:52:36.916 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.917 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.917 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.917 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.605764+0000", 2024-05-08T08:52:36.917 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1359999999999999e-06 2024-05-08T08:52:36.917 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.917 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.917 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.917 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.605774+0000", 2024-05-08T08:52:36.917 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.9189999999999994e-06 2024-05-08T08:52:36.917 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.917 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.917 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.918 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.605781+0000", 2024-05-08T08:52:36.918 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.725e-06 2024-05-08T08:52:36.918 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.918 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.918 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.918 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.918 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.918 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2174 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.918 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.612246+0000", 2024-05-08T08:52:36.918 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.022981963000000001, 2024-05-08T08:52:36.918 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.026265543999999998, 2024-05-08T08:52:36.918 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.919 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.919 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.919 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.919 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.919 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.919 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2174 2024-05-08T08:52:36.919 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.919 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.919 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.919 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.919 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.612246+0000", 2024-05-08T08:52:36.919 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.920 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.920 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.920 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.920 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.612246+0000", 2024-05-08T08:52:36.920 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.920 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.920 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.920 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.920 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.612480+0000", 2024-05-08T08:52:36.920 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000234155 2024-05-08T08:52:36.920 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.920 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.920 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.921 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.612488+0000", 2024-05-08T08:52:36.921 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.2400000000000007e-06 2024-05-08T08:52:36.921 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.921 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.921 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.921 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.612495+0000", 2024-05-08T08:52:36.921 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2929999999999997e-06 2024-05-08T08:52:36.921 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.921 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.921 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.921 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.612499+0000", 2024-05-08T08:52:36.921 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1659999999999996e-06 2024-05-08T08:52:36.922 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.922 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.922 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.922 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.922 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.922 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2239 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.922 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.618518+0000", 2024-05-08T08:52:36.922 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.016709924000000001, 2024-05-08T08:52:36.922 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.020047435999999998, 2024-05-08T08:52:36.922 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.922 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.922 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.923 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.923 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.923 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.923 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2239 2024-05-08T08:52:36.923 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.923 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.923 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.923 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.923 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.618518+0000", 2024-05-08T08:52:36.923 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.923 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.923 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.924 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.924 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.618518+0000", 2024-05-08T08:52:36.924 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.924 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.924 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.924 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.924 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.618519+0000", 2024-05-08T08:52:36.924 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7400000000000003e-07 2024-05-08T08:52:36.924 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.924 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.924 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.924 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.618521+0000", 2024-05-08T08:52:36.924 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8260000000000001e-06 2024-05-08T08:52:36.925 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.925 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.925 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.925 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.618530+0000", 2024-05-08T08:52:36.925 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.5299999999999996e-06 2024-05-08T08:52:36.925 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.925 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.925 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.925 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.618536+0000", 2024-05-08T08:52:36.925 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.144e-06 2024-05-08T08:52:36.925 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.925 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.925 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.926 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.926 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.926 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2302 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.926 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.625250+0000", 2024-05-08T08:52:36.926 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0099775739999999995, 2024-05-08T08:52:36.926 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.013374509, 2024-05-08T08:52:36.926 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.926 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.926 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.926 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.926 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.926 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.927 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2302 2024-05-08T08:52:36.927 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.927 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.927 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.927 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.927 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.625250+0000", 2024-05-08T08:52:36.927 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.927 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.927 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.927 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.927 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.625250+0000", 2024-05-08T08:52:36.927 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.927 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.928 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.928 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.928 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.625354+0000", 2024-05-08T08:52:36.928 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000103668 2024-05-08T08:52:36.928 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.928 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.928 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.928 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.625358+0000", 2024-05-08T08:52:36.928 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8720000000000004e-06 2024-05-08T08:52:36.928 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.928 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.928 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.928 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.625373+0000", 2024-05-08T08:52:36.929 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.556e-05 2024-05-08T08:52:36.929 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.929 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.929 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.929 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.625380+0000", 2024-05-08T08:52:36.929 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0049999999999998e-06 2024-05-08T08:52:36.929 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.929 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.929 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.929 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.929 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.929 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2367 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.930 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.631462+0000", 2024-05-08T08:52:36.930 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0037661740000000002, 2024-05-08T08:52:36.930 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0072184429999999997, 2024-05-08T08:52:36.930 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.930 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.930 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.930 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.930 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.930 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.930 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2367 2024-05-08T08:52:36.930 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.930 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.931 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.931 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.931 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.631462+0000", 2024-05-08T08:52:36.931 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.931 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.931 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.931 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.931 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.631462+0000", 2024-05-08T08:52:36.931 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.931 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.931 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.931 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.932 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.631862+0000", 2024-05-08T08:52:36.932 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00039992199999999998 2024-05-08T08:52:36.932 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.932 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.932 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.932 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.631872+0000", 2024-05-08T08:52:36.932 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0190999999999999e-05 2024-05-08T08:52:36.932 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.932 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.932 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.932 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.631882+0000", 2024-05-08T08:52:36.932 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0609e-05 2024-05-08T08:52:36.933 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.933 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.933 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.933 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.631890+0000", 2024-05-08T08:52:36.933 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1790000000000002e-06 2024-05-08T08:52:36.933 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.933 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.933 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.933 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.933 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.933 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1984 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.933 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.591600+0000", 2024-05-08T08:52:36.933 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.043627895, 2024-05-08T08:52:36.934 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.047139795999999998, 2024-05-08T08:52:36.934 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.934 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.934 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.934 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.934 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.934 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.934 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1984 2024-05-08T08:52:36.934 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.934 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.934 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.935 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.935 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.591600+0000", 2024-05-08T08:52:36.935 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.935 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.935 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.935 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.935 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.591600+0000", 2024-05-08T08:52:36.935 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.935 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.935 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.935 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.935 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.591763+0000", 2024-05-08T08:52:36.936 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000163432 2024-05-08T08:52:36.936 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.936 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.936 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.936 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.591776+0000", 2024-05-08T08:52:36.936 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2904999999999999e-05 2024-05-08T08:52:36.936 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.936 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.936 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.936 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.591789+0000", 2024-05-08T08:52:36.936 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2483999999999999e-05 2024-05-08T08:52:36.936 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.937 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.937 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.937 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.591797+0000", 2024-05-08T08:52:36.937 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.225e-06 2024-05-08T08:52:36.937 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.937 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.937 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.937 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.937 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.937 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2047 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.937 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.598658+0000", 2024-05-08T08:52:36.937 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.036570071000000003, 2024-05-08T08:52:36.937 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.040115703000000003, 2024-05-08T08:52:36.938 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.938 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.938 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.938 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.938 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.938 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.938 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2047 2024-05-08T08:52:36.938 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.938 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.938 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.938 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.938 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.598658+0000", 2024-05-08T08:52:36.939 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.939 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.939 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.939 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.939 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.598658+0000", 2024-05-08T08:52:36.939 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.939 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.939 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.939 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.939 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.598936+0000", 2024-05-08T08:52:36.939 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00027859899999999998 2024-05-08T08:52:36.939 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.939 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.940 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.940 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.598941+0000", 2024-05-08T08:52:36.940 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1520000000000002e-06 2024-05-08T08:52:36.940 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.940 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.940 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.940 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.598949+0000", 2024-05-08T08:52:36.940 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0450000000000001e-06 2024-05-08T08:52:36.940 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.940 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.940 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.940 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.598962+0000", 2024-05-08T08:52:36.940 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2935000000000001e-05 2024-05-08T08:52:36.941 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.941 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.941 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.941 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.941 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.941 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2111 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.941 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.605791+0000", 2024-05-08T08:52:36.941 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.029437093000000001, 2024-05-08T08:52:36.941 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.033041332, 2024-05-08T08:52:36.941 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.941 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.941 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.941 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.941 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.942 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.942 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2111 2024-05-08T08:52:36.942 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.942 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.942 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.942 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.942 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.605791+0000", 2024-05-08T08:52:36.942 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.942 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.942 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.942 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.943 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.605791+0000", 2024-05-08T08:52:36.943 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.943 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.943 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.943 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.943 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.605884+0000", 2024-05-08T08:52:36.943 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2923000000000005e-05 2024-05-08T08:52:36.943 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.943 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.943 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.943 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.605887+0000", 2024-05-08T08:52:36.943 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7069999999999998e-06 2024-05-08T08:52:36.944 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.944 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.944 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.944 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.605896+0000", 2024-05-08T08:52:36.944 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.2810000000000008e-06 2024-05-08T08:52:36.944 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.944 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.944 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.944 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.605902+0000", 2024-05-08T08:52:36.944 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1700000000000002e-06 2024-05-08T08:52:36.944 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.944 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.945 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.945 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.945 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.945 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2176 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.945 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.612506+0000", 2024-05-08T08:52:36.945 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.022722142000000001, 2024-05-08T08:52:36.945 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.026382909, 2024-05-08T08:52:36.945 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.945 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.945 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.945 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.945 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.945 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.946 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2176 2024-05-08T08:52:36.946 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.946 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.946 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.946 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.946 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.612506+0000", 2024-05-08T08:52:36.946 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.946 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.946 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.946 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.946 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.612506+0000", 2024-05-08T08:52:36.946 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.947 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.947 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.947 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.947 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.612506+0000", 2024-05-08T08:52:36.947 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4099999999999999e-07 2024-05-08T08:52:36.947 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.947 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.947 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.947 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.612508+0000", 2024-05-08T08:52:36.947 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.4750000000000001e-06 2024-05-08T08:52:36.947 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.947 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.947 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.948 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.612511+0000", 2024-05-08T08:52:36.948 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3699999999999999e-06 2024-05-08T08:52:36.948 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.948 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.948 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.948 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.612514+0000", 2024-05-08T08:52:36.948 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8270000000000002e-06 2024-05-08T08:52:36.948 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.948 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.948 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.948 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.948 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.948 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2241 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.949 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.618544+0000", 2024-05-08T08:52:36.949 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.016684034, 2024-05-08T08:52:36.949 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.020405218999999999, 2024-05-08T08:52:36.949 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.949 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.949 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.949 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.949 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.949 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.949 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2241 2024-05-08T08:52:36.949 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.949 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.949 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.950 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.950 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.618544+0000", 2024-05-08T08:52:36.950 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.950 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.950 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.950 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.950 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.618544+0000", 2024-05-08T08:52:36.950 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.950 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.950 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.950 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.950 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.618924+0000", 2024-05-08T08:52:36.951 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00038001099999999998 2024-05-08T08:52:36.951 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.951 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.951 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.951 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.618928+0000", 2024-05-08T08:52:36.951 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0879999999999997e-06 2024-05-08T08:52:36.951 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.951 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.951 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.951 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.618936+0000", 2024-05-08T08:52:36.951 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3559999999999993e-06 2024-05-08T08:52:36.951 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.951 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.951 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.952 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.618942+0000", 2024-05-08T08:52:36.952 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.139e-06 2024-05-08T08:52:36.952 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.952 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.952 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.952 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.952 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.952 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2304 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.952 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.625393+0000", 2024-05-08T08:52:36.952 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0098346170000000004, 2024-05-08T08:52:36.952 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.013617917, 2024-05-08T08:52:36.952 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.952 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.953 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.953 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.953 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.953 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.953 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2304 2024-05-08T08:52:36.953 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.953 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.953 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.953 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.953 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.625393+0000", 2024-05-08T08:52:36.953 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.953 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.954 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.954 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.954 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.625393+0000", 2024-05-08T08:52:36.954 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.954 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.954 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.954 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.954 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.625504+0000", 2024-05-08T08:52:36.954 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000111191 2024-05-08T08:52:36.954 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.954 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.954 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.955 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.625509+0000", 2024-05-08T08:52:36.955 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4619999999999998e-06 2024-05-08T08:52:36.955 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.955 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.955 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.955 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.625524+0000", 2024-05-08T08:52:36.955 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.49e-05 2024-05-08T08:52:36.955 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.955 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.955 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.955 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.625530+0000", 2024-05-08T08:52:36.955 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5150000000000003e-06 2024-05-08T08:52:36.956 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.956 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.956 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.956 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.956 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.956 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2369 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.956 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.631901+0000", 2024-05-08T08:52:36.956 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.003327258, 2024-05-08T08:52:36.956 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0071471859999999998, 2024-05-08T08:52:36.956 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.956 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.956 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.957 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.957 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.957 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.957 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2369 2024-05-08T08:52:36.957 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.957 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.957 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.957 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.957 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.631901+0000", 2024-05-08T08:52:36.957 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.957 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.957 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.957 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.958 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.631901+0000", 2024-05-08T08:52:36.958 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.958 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.958 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.958 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.958 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.631901+0000", 2024-05-08T08:52:36.958 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7700000000000005e-07 2024-05-08T08:52:36.958 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.958 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.958 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.958 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.631904+0000", 2024-05-08T08:52:36.958 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6010000000000002e-06 2024-05-08T08:52:36.959 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.959 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.959 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.959 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.631909+0000", 2024-05-08T08:52:36.959 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3820000000000003e-06 2024-05-08T08:52:36.959 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.959 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.959 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.959 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.631913+0000", 2024-05-08T08:52:36.959 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4680000000000001e-06 2024-05-08T08:52:36.959 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.959 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.959 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.960 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.960 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.960 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1986 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.960 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.591810+0000", 2024-05-08T08:52:36.960 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.043418352, 2024-05-08T08:52:36.960 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.047272555000000001, 2024-05-08T08:52:36.960 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.960 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.960 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.960 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.960 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.961 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.961 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1986 2024-05-08T08:52:36.961 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.961 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.961 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.961 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.961 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.591810+0000", 2024-05-08T08:52:36.961 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.961 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.961 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.961 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.962 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.591810+0000", 2024-05-08T08:52:36.962 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.962 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.962 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.962 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.962 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.591871+0000", 2024-05-08T08:52:36.962 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1710000000000004e-05 2024-05-08T08:52:36.962 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.962 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.962 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.962 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.591875+0000", 2024-05-08T08:52:36.962 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1470000000000003e-06 2024-05-08T08:52:36.963 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.963 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.963 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.963 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.591886+0000", 2024-05-08T08:52:36.963 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0603e-05 2024-05-08T08:52:36.963 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.963 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.963 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.963 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.591899+0000", 2024-05-08T08:52:36.963 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2561999999999999e-05 2024-05-08T08:52:36.963 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.963 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.964 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.964 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.964 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.964 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2049 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.964 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.598974+0000", 2024-05-08T08:52:36.964 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.036254173000000001, 2024-05-08T08:52:36.964 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.040160306, 2024-05-08T08:52:36.964 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.964 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.964 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.964 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.965 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.965 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.965 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2049 2024-05-08T08:52:36.965 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.965 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.965 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.965 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.965 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.598974+0000", 2024-05-08T08:52:36.965 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.965 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.965 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.965 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.966 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.598974+0000", 2024-05-08T08:52:36.966 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.966 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.966 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.966 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.966 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.598974+0000", 2024-05-08T08:52:36.966 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3799999999999997e-07 2024-05-08T08:52:36.966 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.966 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.966 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.966 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.598977+0000", 2024-05-08T08:52:36.966 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.723e-06 2024-05-08T08:52:36.967 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.967 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.967 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.967 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.598984+0000", 2024-05-08T08:52:36.967 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0180000000000004e-06 2024-05-08T08:52:36.967 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.967 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.967 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.967 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.598990+0000", 2024-05-08T08:52:36.967 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5099999999999998e-06 2024-05-08T08:52:36.967 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.967 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.967 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.968 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.968 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.968 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2113 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.968 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.605912+0000", 2024-05-08T08:52:36.968 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.029315903000000001, 2024-05-08T08:52:36.968 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.033273981000000001, 2024-05-08T08:52:36.968 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.968 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.968 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.968 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.968 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.968 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.968 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2113 2024-05-08T08:52:36.969 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.969 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.969 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.969 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.969 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.605912+0000", 2024-05-08T08:52:36.969 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.969 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.969 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.969 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.969 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.605912+0000", 2024-05-08T08:52:36.969 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.969 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.969 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.970 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.970 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.606262+0000", 2024-05-08T08:52:36.970 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00035025300000000002 2024-05-08T08:52:36.970 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.970 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.970 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.970 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.606266+0000", 2024-05-08T08:52:36.970 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9740000000000004e-06 2024-05-08T08:52:36.970 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.970 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.970 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.970 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.606275+0000", 2024-05-08T08:52:36.971 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.6510000000000007e-06 2024-05-08T08:52:36.971 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.971 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.971 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.971 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.606281+0000", 2024-05-08T08:52:36.971 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3280000000000002e-06 2024-05-08T08:52:36.971 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.971 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.971 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.971 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.971 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.971 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2178 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.972 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.612539+0000", 2024-05-08T08:52:36.972 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.022688898999999998, 2024-05-08T08:52:36.972 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.026705261000000001, 2024-05-08T08:52:36.972 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.972 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.972 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.972 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.972 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.972 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.972 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2178 2024-05-08T08:52:36.972 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.972 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.972 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.973 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.973 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.612539+0000", 2024-05-08T08:52:36.973 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.973 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.973 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.973 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.973 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.612539+0000", 2024-05-08T08:52:36.973 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.973 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.973 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.973 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.973 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.612864+0000", 2024-05-08T08:52:36.973 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00032451399999999998 2024-05-08T08:52:36.974 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.974 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.974 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.974 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.612866+0000", 2024-05-08T08:52:36.974 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6400000000000001e-06 2024-05-08T08:52:36.974 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.974 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.974 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.974 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.612872+0000", 2024-05-08T08:52:36.974 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5010000000000004e-06 2024-05-08T08:52:36.974 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.974 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.975 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.975 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.612875+0000", 2024-05-08T08:52:36.975 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7890000000000001e-06 2024-05-08T08:52:36.975 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.975 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.975 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.975 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.975 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.975 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2243 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.975 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.618960+0000", 2024-05-08T08:52:36.975 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.016267904999999999, 2024-05-08T08:52:36.975 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.020344806999999999, 2024-05-08T08:52:36.975 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.976 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.976 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.976 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.976 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.976 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.976 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2243 2024-05-08T08:52:36.976 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.976 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.976 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.976 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.976 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.618960+0000", 2024-05-08T08:52:36.976 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.976 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.977 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.977 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.977 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.618960+0000", 2024-05-08T08:52:36.977 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.977 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.977 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.977 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.977 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.619041+0000", 2024-05-08T08:52:36.977 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1259000000000002e-05 2024-05-08T08:52:36.977 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.977 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.977 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.977 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.619044+0000", 2024-05-08T08:52:36.978 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1559999999999999e-06 2024-05-08T08:52:36.978 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.978 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.978 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.978 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.619052+0000", 2024-05-08T08:52:36.978 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0630000000000006e-06 2024-05-08T08:52:36.978 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.978 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.978 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.978 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.619058+0000", 2024-05-08T08:52:36.978 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2460000000000003e-06 2024-05-08T08:52:36.978 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.978 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.979 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.979 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.979 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.979 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2306 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.979 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.625544+0000", 2024-05-08T08:52:36.979 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0096834269999999997, 2024-05-08T08:52:36.979 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.013816292000000001, 2024-05-08T08:52:36.979 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.979 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.979 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.979 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.979 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.980 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.980 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2306 2024-05-08T08:52:36.980 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.980 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.980 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.980 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.980 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.625544+0000", 2024-05-08T08:52:36.980 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.980 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.980 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.980 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.980 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.625544+0000", 2024-05-08T08:52:36.980 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.981 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.981 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.981 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.981 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.625797+0000", 2024-05-08T08:52:36.981 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00025206799999999999 2024-05-08T08:52:36.981 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.981 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.981 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.981 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.625800+0000", 2024-05-08T08:52:36.981 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3620000000000002e-06 2024-05-08T08:52:36.981 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.981 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.982 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.982 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.625807+0000", 2024-05-08T08:52:36.982 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6039999999999998e-06 2024-05-08T08:52:36.982 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.982 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.982 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.982 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.625811+0000", 2024-05-08T08:52:36.982 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.33e-06 2024-05-08T08:52:36.982 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.982 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.982 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.982 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.983 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.983 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2371 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.983 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.631919+0000", 2024-05-08T08:52:36.983 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0033091380000000001, 2024-05-08T08:52:36.983 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0074811900000000004, 2024-05-08T08:52:36.983 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.983 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.983 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.983 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.983 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.983 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.983 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2371 2024-05-08T08:52:36.983 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.984 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.984 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.984 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.984 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.631919+0000", 2024-05-08T08:52:36.984 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.984 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.984 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.984 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.984 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.631919+0000", 2024-05-08T08:52:36.984 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.984 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.984 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.985 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.985 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.632229+0000", 2024-05-08T08:52:36.985 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00031020400000000001 2024-05-08T08:52:36.985 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.985 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.985 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.985 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.632233+0000", 2024-05-08T08:52:36.985 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7120000000000002e-06 2024-05-08T08:52:36.985 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.985 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.985 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.985 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.632243+0000", 2024-05-08T08:52:36.985 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.9000000000000001e-06 2024-05-08T08:52:36.986 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.986 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.986 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.986 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.632251+0000", 2024-05-08T08:52:36.986 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0660000000000004e-06 2024-05-08T08:52:36.986 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.986 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.986 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.986 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.986 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.986 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1988 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.987 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.591910+0000", 2024-05-08T08:52:36.987 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.043318093000000002, 2024-05-08T08:52:36.987 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.047524495, 2024-05-08T08:52:36.987 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.987 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.987 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.987 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.987 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.987 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.987 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1988 2024-05-08T08:52:36.987 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.987 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.987 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.988 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.988 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.591910+0000", 2024-05-08T08:52:36.988 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.988 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.988 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.988 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.988 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.591910+0000", 2024-05-08T08:52:36.988 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.988 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.988 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.988 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.988 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.592101+0000", 2024-05-08T08:52:36.989 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00019075900000000001 2024-05-08T08:52:36.989 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.989 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.989 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.989 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.592105+0000", 2024-05-08T08:52:36.989 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0790000000000002e-06 2024-05-08T08:52:36.989 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.989 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.989 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.989 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.592124+0000", 2024-05-08T08:52:36.989 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9208e-05 2024-05-08T08:52:36.989 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.989 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.990 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.990 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.592132+0000", 2024-05-08T08:52:36.990 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9200000000000004e-06 2024-05-08T08:52:36.990 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.990 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.990 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.990 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.990 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.990 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2051 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.990 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.598996+0000", 2024-05-08T08:52:36.990 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.036231454000000003, 2024-05-08T08:52:36.990 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.040489065999999997, 2024-05-08T08:52:36.991 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.991 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.991 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.991 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.991 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.991 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.991 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2051 2024-05-08T08:52:36.991 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.991 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.991 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.991 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.991 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.598996+0000", 2024-05-08T08:52:36.991 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.992 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.992 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.992 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.992 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.598996+0000", 2024-05-08T08:52:36.992 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.992 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.992 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.992 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.992 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.599225+0000", 2024-05-08T08:52:36.992 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00022889899999999999 2024-05-08T08:52:36.992 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.992 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.992 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.992 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.599228+0000", 2024-05-08T08:52:36.993 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0520000000000002e-06 2024-05-08T08:52:36.993 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.993 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.993 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.993 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.599235+0000", 2024-05-08T08:52:36.993 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8469999999999998e-06 2024-05-08T08:52:36.993 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.993 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.993 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.993 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.599240+0000", 2024-05-08T08:52:36.993 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4569999999999998e-06 2024-05-08T08:52:36.993 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.994 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.994 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.994 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.994 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.994 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2115 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.994 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.606292+0000", 2024-05-08T08:52:36.994 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.028935857999999998, 2024-05-08T08:52:36.994 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.03322162, 2024-05-08T08:52:36.994 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.994 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.994 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.994 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.995 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.995 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.995 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2115 2024-05-08T08:52:36.995 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.995 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.995 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.995 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.995 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.606292+0000", 2024-05-08T08:52:36.995 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.995 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.995 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.995 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.995 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.606292+0000", 2024-05-08T08:52:36.996 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.996 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.996 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.996 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:36.996 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.606386+0000", 2024-05-08T08:52:36.996 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4291999999999998e-05 2024-05-08T08:52:36.996 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.996 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.996 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:36.996 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.606390+0000", 2024-05-08T08:52:36.996 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5200000000000002e-06 2024-05-08T08:52:36.996 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.997 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.997 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:36.997 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.606398+0000", 2024-05-08T08:52:36.997 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.6450000000000005e-06 2024-05-08T08:52:36.997 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.997 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.997 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:36.997 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.606403+0000", 2024-05-08T08:52:36.997 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.976e-06 2024-05-08T08:52:36.997 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.997 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:36.997 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:36.998 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.998 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.998 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2180 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:36.998 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.612882+0000", 2024-05-08T08:52:36.998 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.022345599000000001, 2024-05-08T08:52:36.998 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.026661114, 2024-05-08T08:52:36.998 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:36.998 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:36.998 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:36.998 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:36.998 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:36.998 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:36.999 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2180 2024-05-08T08:52:36.999 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.999 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:36.999 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.999 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:36.999 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.612882+0000", 2024-05-08T08:52:36.999 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.999 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:36.999 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:36.999 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:36.999 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.612882+0000", 2024-05-08T08:52:36.999 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:36.999 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.000 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.000 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.000 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.613127+0000", 2024-05-08T08:52:37.000 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00024471699999999998 2024-05-08T08:52:37.000 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.000 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.000 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.000 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.613131+0000", 2024-05-08T08:52:37.000 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2599999999999999e-06 2024-05-08T08:52:37.000 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.000 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.000 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.000 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.613141+0000", 2024-05-08T08:52:37.001 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4469999999999995e-06 2024-05-08T08:52:37.001 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.001 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.001 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.001 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.613147+0000", 2024-05-08T08:52:37.001 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1600000000000003e-06 2024-05-08T08:52:37.001 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.001 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.001 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.001 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.001 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.001 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2245 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.002 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.619066+0000", 2024-05-08T08:52:37.002 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.016162333000000001, 2024-05-08T08:52:37.002 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.02051648, 2024-05-08T08:52:37.002 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.002 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.002 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.002 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.002 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.002 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.002 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2245 2024-05-08T08:52:37.002 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.002 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.003 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.003 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.003 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.619066+0000", 2024-05-08T08:52:37.003 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.003 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.003 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.003 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.003 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.619066+0000", 2024-05-08T08:52:37.003 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.003 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.003 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.003 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.004 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.619278+0000", 2024-05-08T08:52:37.004 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00021220899999999999 2024-05-08T08:52:37.004 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.004 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.004 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.004 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.619281+0000", 2024-05-08T08:52:37.004 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4209999999999999e-06 2024-05-08T08:52:37.004 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.004 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.004 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.004 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.619289+0000", 2024-05-08T08:52:37.004 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9549999999999992e-06 2024-05-08T08:52:37.004 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.005 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.005 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.005 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.619295+0000", 2024-05-08T08:52:37.005 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5979999999999996e-06 2024-05-08T08:52:37.005 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.005 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.005 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.005 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.005 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.005 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2309 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.005 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.625819+0000", 2024-05-08T08:52:37.005 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0094090979999999994, 2024-05-08T08:52:37.006 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.013821676, 2024-05-08T08:52:37.006 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.006 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.006 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.006 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.006 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.006 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.006 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2309 2024-05-08T08:52:37.006 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.006 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.006 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.006 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.006 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.625819+0000", 2024-05-08T08:52:37.007 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.007 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.007 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.007 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.007 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.625819+0000", 2024-05-08T08:52:37.007 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.007 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.007 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.007 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.007 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.625988+0000", 2024-05-08T08:52:37.007 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00016923200000000001 2024-05-08T08:52:37.007 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.007 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.007 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.007 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.625992+0000", 2024-05-08T08:52:37.008 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2889999999999999e-06 2024-05-08T08:52:37.008 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.008 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.008 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.008 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.626001+0000", 2024-05-08T08:52:37.008 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.5329999999999994e-06 2024-05-08T08:52:37.008 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.008 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.008 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.008 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.626007+0000", 2024-05-08T08:52:37.008 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4289999999999998e-06 2024-05-08T08:52:37.008 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.008 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.009 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.009 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.009 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.009 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2373 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.009 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.632263+0000", 2024-05-08T08:52:37.009 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.002965016, 2024-05-08T08:52:37.009 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0074338640000000001, 2024-05-08T08:52:37.009 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.009 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.009 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.009 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.009 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.010 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.010 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2373 2024-05-08T08:52:37.010 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.010 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.010 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.010 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.010 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.632263+0000", 2024-05-08T08:52:37.010 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.010 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.010 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.010 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.010 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.632263+0000", 2024-05-08T08:52:37.011 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.011 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.011 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.011 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.011 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.632263+0000", 2024-05-08T08:52:37.011 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2e-07 2024-05-08T08:52:37.011 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.011 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.011 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.011 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.632266+0000", 2024-05-08T08:52:37.011 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.508e-06 2024-05-08T08:52:37.011 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.011 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.011 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.012 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.632273+0000", 2024-05-08T08:52:37.012 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.165e-06 2024-05-08T08:52:37.012 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.012 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.012 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.012 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.632279+0000", 2024-05-08T08:52:37.012 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8440000000000003e-06 2024-05-08T08:52:37.012 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.012 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.012 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.012 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.012 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.012 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1990 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.013 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.592143+0000", 2024-05-08T08:52:37.013 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.043084697999999998, 2024-05-08T08:52:37.013 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.047616397999999997, 2024-05-08T08:52:37.013 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.013 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.013 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.013 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.013 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.013 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.013 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1990 2024-05-08T08:52:37.013 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.013 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.013 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.013 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.014 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.592143+0000", 2024-05-08T08:52:37.014 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.014 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.014 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.014 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.014 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.592143+0000", 2024-05-08T08:52:37.014 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.014 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.014 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.014 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.014 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.592486+0000", 2024-05-08T08:52:37.014 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00034309999999999999 2024-05-08T08:52:37.015 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.015 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.015 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.015 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.592491+0000", 2024-05-08T08:52:37.015 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3189999999999996e-06 2024-05-08T08:52:37.015 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.015 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.015 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.015 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.592500+0000", 2024-05-08T08:52:37.015 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3379999999999994e-06 2024-05-08T08:52:37.015 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.015 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.015 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.016 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.592506+0000", 2024-05-08T08:52:37.016 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5100000000000004e-06 2024-05-08T08:52:37.016 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.016 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.016 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.016 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.016 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.016 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2053 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.016 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.599248+0000", 2024-05-08T08:52:37.016 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.035980285000000001, 2024-05-08T08:52:37.016 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.040562309999999997, 2024-05-08T08:52:37.016 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.016 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.017 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.017 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.017 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.017 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.017 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2053 2024-05-08T08:52:37.017 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.017 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.017 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.017 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.017 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.599248+0000", 2024-05-08T08:52:37.017 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.017 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.018 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.018 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.018 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.599248+0000", 2024-05-08T08:52:37.018 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.018 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.018 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.018 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.018 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.599248+0000", 2024-05-08T08:52:37.018 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.2000000000000001e-07 2024-05-08T08:52:37.018 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.018 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.019 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.019 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.599249+0000", 2024-05-08T08:52:37.019 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5290000000000001e-06 2024-05-08T08:52:37.019 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.019 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.019 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.019 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.599253+0000", 2024-05-08T08:52:37.019 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.579e-06 2024-05-08T08:52:37.019 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.019 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.019 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.019 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.599256+0000", 2024-05-08T08:52:37.020 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.588e-06 2024-05-08T08:52:37.020 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.020 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.020 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.020 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.020 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.020 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2117 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.020 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.606413+0000", 2024-05-08T08:52:37.020 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.028814749000000001, 2024-05-08T08:52:37.020 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.033428377000000002, 2024-05-08T08:52:37.020 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.020 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.020 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.021 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.021 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.021 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.021 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2117 2024-05-08T08:52:37.021 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.021 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.021 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.021 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.021 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.606413+0000", 2024-05-08T08:52:37.021 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.021 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.021 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.022 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.022 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.606413+0000", 2024-05-08T08:52:37.022 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.022 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.022 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.022 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.022 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.606913+0000", 2024-05-08T08:52:37.022 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00050018899999999995 2024-05-08T08:52:37.022 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.022 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.022 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.022 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.606917+0000", 2024-05-08T08:52:37.023 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.143e-06 2024-05-08T08:52:37.023 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.023 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.023 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.023 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.606931+0000", 2024-05-08T08:52:37.023 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3022e-05 2024-05-08T08:52:37.023 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.023 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.023 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.023 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.606937+0000", 2024-05-08T08:52:37.023 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4699999999999999e-06 2024-05-08T08:52:37.023 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.024 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.024 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.024 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.024 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.024 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2182 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.024 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.613157+0000", 2024-05-08T08:52:37.024 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.022070513999999999, 2024-05-08T08:52:37.024 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.026711571, 2024-05-08T08:52:37.024 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.024 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.024 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.024 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.025 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.025 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.025 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2182 2024-05-08T08:52:37.025 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.025 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.025 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.025 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.025 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.613157+0000", 2024-05-08T08:52:37.025 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.025 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.025 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.025 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.026 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.613157+0000", 2024-05-08T08:52:37.026 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.026 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.026 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.026 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.026 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.613158+0000", 2024-05-08T08:52:37.026 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5900000000000003e-07 2024-05-08T08:52:37.026 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.026 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.026 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.026 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.613160+0000", 2024-05-08T08:52:37.026 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.8059999999999999e-06 2024-05-08T08:52:37.027 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.027 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.027 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.027 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.613163+0000", 2024-05-08T08:52:37.027 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9110000000000003e-06 2024-05-08T08:52:37.027 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.027 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.027 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.027 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.613166+0000", 2024-05-08T08:52:37.027 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9890000000000001e-06 2024-05-08T08:52:37.027 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.027 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.027 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.028 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.028 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.028 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2247 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.028 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.619304+0000", 2024-05-08T08:52:37.028 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.015923759999999999, 2024-05-08T08:52:37.028 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.020597440000000002, 2024-05-08T08:52:37.028 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.028 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.028 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.028 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.028 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.029 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.029 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2247 2024-05-08T08:52:37.029 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.029 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.029 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.029 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.029 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.619304+0000", 2024-05-08T08:52:37.029 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.029 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.029 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.029 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.030 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.619304+0000", 2024-05-08T08:52:37.030 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.030 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.030 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.030 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.030 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.619527+0000", 2024-05-08T08:52:37.030 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00022265800000000001 2024-05-08T08:52:37.030 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.030 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.030 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.030 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.619535+0000", 2024-05-08T08:52:37.031 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0770000000000008e-06 2024-05-08T08:52:37.031 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.031 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.031 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.031 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.619542+0000", 2024-05-08T08:52:37.031 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6250000000000001e-06 2024-05-08T08:52:37.031 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.031 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.031 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.031 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.619546+0000", 2024-05-08T08:52:37.031 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0879999999999997e-06 2024-05-08T08:52:37.031 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.031 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.032 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.032 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.032 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.032 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2311 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.032 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.626018+0000", 2024-05-08T08:52:37.032 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0092101539999999999, 2024-05-08T08:52:37.032 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.01391089, 2024-05-08T08:52:37.032 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.032 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.032 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.032 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.032 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.032 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.033 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2311 2024-05-08T08:52:37.033 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.033 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.033 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.033 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.033 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.626018+0000", 2024-05-08T08:52:37.033 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.033 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.033 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.033 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.033 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.626018+0000", 2024-05-08T08:52:37.033 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.033 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.034 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.034 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.034 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.626398+0000", 2024-05-08T08:52:37.034 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00038014499999999997 2024-05-08T08:52:37.034 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.034 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.034 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.034 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.626402+0000", 2024-05-08T08:52:37.034 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9380000000000002e-06 2024-05-08T08:52:37.034 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.034 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.034 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.035 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.626411+0000", 2024-05-08T08:52:37.035 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1120000000000002e-06 2024-05-08T08:52:37.035 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.035 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.035 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.035 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.626417+0000", 2024-05-08T08:52:37.035 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5479999999999998e-06 2024-05-08T08:52:37.035 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.035 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.035 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.035 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.035 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.036 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2375 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.036 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.632287+0000", 2024-05-08T08:52:37.036 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0029413920000000001, 2024-05-08T08:52:37.036 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0076840700000000003, 2024-05-08T08:52:37.036 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.036 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.036 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.036 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.036 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.036 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.036 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2375 2024-05-08T08:52:37.036 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.037 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.037 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.037 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.037 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.632287+0000", 2024-05-08T08:52:37.037 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.037 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.037 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.037 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.037 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.632287+0000", 2024-05-08T08:52:37.037 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.037 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.037 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.038 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.038 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.632460+0000", 2024-05-08T08:52:37.038 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00017304700000000001 2024-05-08T08:52:37.038 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.038 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.038 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.038 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.632463+0000", 2024-05-08T08:52:37.038 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.309e-06 2024-05-08T08:52:37.038 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.038 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.038 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.038 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.632470+0000", 2024-05-08T08:52:37.038 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8390000000000001e-06 2024-05-08T08:52:37.039 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.039 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.039 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.039 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.632474+0000", 2024-05-08T08:52:37.039 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3830000000000002e-06 2024-05-08T08:52:37.039 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.039 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.039 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.039 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.039 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.039 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1992 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.039 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.592519+0000", 2024-05-08T08:52:37.040 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.042708662000000001, 2024-05-08T08:52:37.040 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.047490310000000001, 2024-05-08T08:52:37.040 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.040 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.040 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.040 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.040 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.040 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.040 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1992 2024-05-08T08:52:37.040 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.040 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.040 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.040 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.041 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.592519+0000", 2024-05-08T08:52:37.041 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.041 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.041 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.041 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.041 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.592519+0000", 2024-05-08T08:52:37.041 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.041 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.041 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.041 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.041 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.592611+0000", 2024-05-08T08:52:37.041 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1695999999999998e-05 2024-05-08T08:52:37.041 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.041 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.042 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.042 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.592615+0000", 2024-05-08T08:52:37.042 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0239999999999999e-06 2024-05-08T08:52:37.042 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.042 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.042 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.042 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.592623+0000", 2024-05-08T08:52:37.042 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4910000000000005e-06 2024-05-08T08:52:37.042 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.042 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.042 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.043 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.592630+0000", 2024-05-08T08:52:37.043 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5119999999999997e-06 2024-05-08T08:52:37.043 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.043 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.043 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.043 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.043 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.043 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2055 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.043 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.599261+0000", 2024-05-08T08:52:37.043 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.035966686999999997, 2024-05-08T08:52:37.043 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.040803915000000003, 2024-05-08T08:52:37.043 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.044 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.044 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.044 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.044 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.044 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.044 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2055 2024-05-08T08:52:37.044 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.044 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.044 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.045 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.045 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.599261+0000", 2024-05-08T08:52:37.045 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.045 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.045 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.045 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.045 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.599261+0000", 2024-05-08T08:52:37.045 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.045 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.045 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.045 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.045 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.599629+0000", 2024-05-08T08:52:37.046 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00036736499999999998 2024-05-08T08:52:37.046 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.046 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.046 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.046 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.599637+0000", 2024-05-08T08:52:37.046 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.6289999999999999e-06 2024-05-08T08:52:37.046 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.046 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.046 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.046 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.599644+0000", 2024-05-08T08:52:37.046 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8759999999999999e-06 2024-05-08T08:52:37.047 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.047 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.047 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.047 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.599648+0000", 2024-05-08T08:52:37.047 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1749999999999998e-06 2024-05-08T08:52:37.047 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.047 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.047 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.047 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.047 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.047 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2119 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.047 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.606950+0000", 2024-05-08T08:52:37.047 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.028277677000000001, 2024-05-08T08:52:37.048 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.033176259, 2024-05-08T08:52:37.048 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.048 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.048 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.048 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.048 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.048 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.048 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2119 2024-05-08T08:52:37.048 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.048 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.048 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.049 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.049 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.606950+0000", 2024-05-08T08:52:37.049 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.049 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.049 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.049 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.049 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.606950+0000", 2024-05-08T08:52:37.049 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.049 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.049 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.049 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.049 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.607181+0000", 2024-05-08T08:52:37.050 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00023032 2024-05-08T08:52:37.050 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.050 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.050 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.050 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.607184+0000", 2024-05-08T08:52:37.050 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.737e-06 2024-05-08T08:52:37.050 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.050 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.050 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.050 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.607194+0000", 2024-05-08T08:52:37.050 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6350000000000001e-06 2024-05-08T08:52:37.050 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.051 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.051 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.051 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.607200+0000", 2024-05-08T08:52:37.051 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2600000000000002e-06 2024-05-08T08:52:37.051 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.051 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.051 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.051 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.051 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.051 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2184 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.051 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.613170+0000", 2024-05-08T08:52:37.051 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.022057535999999999, 2024-05-08T08:52:37.052 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.027012172000000001, 2024-05-08T08:52:37.052 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.052 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.052 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.052 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.052 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.052 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.052 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2184 2024-05-08T08:52:37.052 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.052 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.052 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.052 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.052 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.613170+0000", 2024-05-08T08:52:37.053 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.053 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.053 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.053 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.053 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.613170+0000", 2024-05-08T08:52:37.053 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.053 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.053 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.053 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.053 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.613386+0000", 2024-05-08T08:52:37.053 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000215651 2024-05-08T08:52:37.054 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.054 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.054 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.054 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.613389+0000", 2024-05-08T08:52:37.054 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4829999999999999e-06 2024-05-08T08:52:37.054 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.054 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.054 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.054 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.613399+0000", 2024-05-08T08:52:37.054 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1409999999999994e-06 2024-05-08T08:52:37.054 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.054 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.055 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.055 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.613407+0000", 2024-05-08T08:52:37.055 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9400000000000002e-06 2024-05-08T08:52:37.055 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.055 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.055 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.055 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.055 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.055 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2249 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.055 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.619553+0000", 2024-05-08T08:52:37.055 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.015674695999999998, 2024-05-08T08:52:37.055 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.020689622000000001, 2024-05-08T08:52:37.056 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.056 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.056 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.056 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.056 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.056 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.056 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2249 2024-05-08T08:52:37.056 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.056 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.056 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.056 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.056 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.619553+0000", 2024-05-08T08:52:37.057 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.057 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.057 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.057 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.057 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.619553+0000", 2024-05-08T08:52:37.057 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.057 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.057 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.057 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.057 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.619662+0000", 2024-05-08T08:52:37.057 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000108902 2024-05-08T08:52:37.057 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.057 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.058 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.058 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.619665+0000", 2024-05-08T08:52:37.058 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3230000000000002e-06 2024-05-08T08:52:37.058 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.058 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.058 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.058 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.619674+0000", 2024-05-08T08:52:37.058 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.9420000000000001e-06 2024-05-08T08:52:37.058 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.058 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.058 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.059 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.619681+0000", 2024-05-08T08:52:37.059 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5769999999999999e-06 2024-05-08T08:52:37.059 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.059 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.059 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.059 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.059 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.059 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2313 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.059 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.626438+0000", 2024-05-08T08:52:37.059 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0087900070000000007, 2024-05-08T08:52:37.059 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.013840177, 2024-05-08T08:52:37.059 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.059 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.060 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.060 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.060 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.060 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.060 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2313 2024-05-08T08:52:37.060 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.060 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.060 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.060 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.060 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.626438+0000", 2024-05-08T08:52:37.060 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.060 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.060 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.061 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.061 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.626438+0000", 2024-05-08T08:52:37.061 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.061 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.061 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.061 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.061 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.626438+0000", 2024-05-08T08:52:37.061 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9100000000000004e-07 2024-05-08T08:52:37.061 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.061 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.061 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.061 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.626441+0000", 2024-05-08T08:52:37.062 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.898e-06 2024-05-08T08:52:37.062 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.062 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.062 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.062 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.626448+0000", 2024-05-08T08:52:37.062 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1019999999999999e-06 2024-05-08T08:52:37.062 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.062 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.062 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.062 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.626454+0000", 2024-05-08T08:52:37.062 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.755e-06 2024-05-08T08:52:37.062 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.063 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.063 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.063 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.063 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.063 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2377 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.063 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.632481+0000", 2024-05-08T08:52:37.063 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0027465189999999998, 2024-05-08T08:52:37.063 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0078264849999999993, 2024-05-08T08:52:37.063 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.063 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.063 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.063 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.064 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.064 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.064 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2377 2024-05-08T08:52:37.064 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.064 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.064 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.064 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.064 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.632481+0000", 2024-05-08T08:52:37.064 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.064 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.064 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.064 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.065 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.632481+0000", 2024-05-08T08:52:37.065 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.065 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.065 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.065 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.065 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.632581+0000", 2024-05-08T08:52:37.065 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.9983999999999997e-05 2024-05-08T08:52:37.065 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.065 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.065 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.065 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.632584+0000", 2024-05-08T08:52:37.065 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3989999999999998e-06 2024-05-08T08:52:37.066 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.066 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.066 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.066 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.632589+0000", 2024-05-08T08:52:37.066 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3909999999999997e-06 2024-05-08T08:52:37.066 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.066 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.066 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.066 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.632594+0000", 2024-05-08T08:52:37.066 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3989999999999997e-06 2024-05-08T08:52:37.066 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.066 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.067 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.067 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.067 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.067 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1994 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.067 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.592641+0000", 2024-05-08T08:52:37.067 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.042586659999999998, 2024-05-08T08:52:37.067 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.047701833999999999, 2024-05-08T08:52:37.067 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.067 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.067 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.067 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.067 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.067 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.068 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1994 2024-05-08T08:52:37.068 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.068 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.068 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.068 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.068 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.592641+0000", 2024-05-08T08:52:37.068 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.068 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.068 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.068 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.068 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.592641+0000", 2024-05-08T08:52:37.068 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.069 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.069 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.069 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.069 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.592860+0000", 2024-05-08T08:52:37.069 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00021901900000000001 2024-05-08T08:52:37.069 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.069 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.069 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.069 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.592864+0000", 2024-05-08T08:52:37.069 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4070000000000002e-06 2024-05-08T08:52:37.069 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.069 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.069 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.070 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.592876+0000", 2024-05-08T08:52:37.070 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1945e-05 2024-05-08T08:52:37.070 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.070 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.070 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.070 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.592880+0000", 2024-05-08T08:52:37.070 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.357e-06 2024-05-08T08:52:37.070 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.070 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.070 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.070 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.070 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.071 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2057 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.071 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.599657+0000", 2024-05-08T08:52:37.071 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.035571318999999997, 2024-05-08T08:52:37.071 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.040735680000000003, 2024-05-08T08:52:37.071 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.071 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.071 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.071 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.071 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.071 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.071 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2057 2024-05-08T08:52:37.071 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.071 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.072 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.072 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.072 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.599657+0000", 2024-05-08T08:52:37.072 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.072 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.072 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.072 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.072 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.599657+0000", 2024-05-08T08:52:37.072 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.072 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.072 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.072 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.072 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.599657+0000", 2024-05-08T08:52:37.073 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5699999999999998e-07 2024-05-08T08:52:37.073 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.073 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.073 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.073 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.599658+0000", 2024-05-08T08:52:37.073 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.522e-06 2024-05-08T08:52:37.073 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.073 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.073 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.073 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.599662+0000", 2024-05-08T08:52:37.073 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3809999999999999e-06 2024-05-08T08:52:37.073 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.074 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.074 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.074 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.599666+0000", 2024-05-08T08:52:37.074 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.687e-06 2024-05-08T08:52:37.074 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.074 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.074 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.074 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.074 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.074 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2121 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.074 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.607218+0000", 2024-05-08T08:52:37.074 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.028010258999999999, 2024-05-08T08:52:37.075 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.033214467999999997, 2024-05-08T08:52:37.075 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.075 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.075 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.075 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.075 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.075 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.075 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2121 2024-05-08T08:52:37.075 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.075 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.075 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.075 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.075 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.607218+0000", 2024-05-08T08:52:37.076 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.076 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.076 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.076 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.076 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.607218+0000", 2024-05-08T08:52:37.076 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.076 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.076 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.076 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.076 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.607218+0000", 2024-05-08T08:52:37.076 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1500000000000005e-07 2024-05-08T08:52:37.076 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.076 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.077 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.077 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.607221+0000", 2024-05-08T08:52:37.077 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.937e-06 2024-05-08T08:52:37.077 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.077 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.077 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.077 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.607228+0000", 2024-05-08T08:52:37.077 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1289999999999998e-06 2024-05-08T08:52:37.077 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.077 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.077 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.077 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.607234+0000", 2024-05-08T08:52:37.078 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3290000000000001e-06 2024-05-08T08:52:37.078 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.078 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.078 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.078 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.078 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.078 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2186 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.078 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.613420+0000", 2024-05-08T08:52:37.078 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.021808306, 2024-05-08T08:52:37.078 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.027071927999999999, 2024-05-08T08:52:37.078 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.079 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.079 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.079 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.079 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.079 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.079 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2186 2024-05-08T08:52:37.079 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.079 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.079 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.079 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.079 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.613420+0000", 2024-05-08T08:52:37.079 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.080 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.080 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.080 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.080 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.613420+0000", 2024-05-08T08:52:37.080 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.080 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.080 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.080 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.080 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.613641+0000", 2024-05-08T08:52:37.080 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00022182199999999999 2024-05-08T08:52:37.080 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.080 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.080 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.081 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.613646+0000", 2024-05-08T08:52:37.081 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6129999999999997e-06 2024-05-08T08:52:37.081 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.081 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.081 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.081 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.613657+0000", 2024-05-08T08:52:37.081 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0838e-05 2024-05-08T08:52:37.081 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.081 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.081 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.081 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.613665+0000", 2024-05-08T08:52:37.081 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4819999999999994e-06 2024-05-08T08:52:37.081 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.082 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.082 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.082 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.082 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.082 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2250 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.082 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.619693+0000", 2024-05-08T08:52:37.082 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.015534951999999999, 2024-05-08T08:52:37.082 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.020855662000000001, 2024-05-08T08:52:37.082 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.082 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.082 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.083 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.083 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.083 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.083 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2250 2024-05-08T08:52:37.083 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.083 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.083 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.083 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.083 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.619693+0000", 2024-05-08T08:52:37.083 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.083 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.083 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.084 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.084 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.619693+0000", 2024-05-08T08:52:37.084 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.084 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.084 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.084 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.084 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.619917+0000", 2024-05-08T08:52:37.084 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00022383 2024-05-08T08:52:37.084 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.084 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.084 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.084 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.619922+0000", 2024-05-08T08:52:37.085 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3859999999999998e-06 2024-05-08T08:52:37.085 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.085 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.085 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.085 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.619934+0000", 2024-05-08T08:52:37.085 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2004999999999999e-05 2024-05-08T08:52:37.085 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.085 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.085 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.085 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.619942+0000", 2024-05-08T08:52:37.085 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4070000000000004e-06 2024-05-08T08:52:37.085 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.086 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.086 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.086 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.086 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.086 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2315 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.086 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.626461+0000", 2024-05-08T08:52:37.086 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0087664609999999997, 2024-05-08T08:52:37.086 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.014148297000000001, 2024-05-08T08:52:37.086 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.086 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.086 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.086 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.087 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.087 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.087 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2315 2024-05-08T08:52:37.087 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.087 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.087 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.087 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.087 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.626461+0000", 2024-05-08T08:52:37.087 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.087 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.087 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.087 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.087 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.626461+0000", 2024-05-08T08:52:37.088 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.088 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.088 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.088 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.088 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.626703+0000", 2024-05-08T08:52:37.088 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000241191 2024-05-08T08:52:37.088 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.088 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.088 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.088 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.626706+0000", 2024-05-08T08:52:37.088 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1020000000000001e-06 2024-05-08T08:52:37.088 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.089 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.089 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.089 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.626712+0000", 2024-05-08T08:52:37.089 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0739999999999998e-06 2024-05-08T08:52:37.089 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.089 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.089 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.089 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.626716+0000", 2024-05-08T08:52:37.089 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.104e-06 2024-05-08T08:52:37.089 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.089 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.089 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.089 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.090 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.090 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2379 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.090 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.632600+0000", 2024-05-08T08:52:37.090 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0026278790000000001, 2024-05-08T08:52:37.090 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0080570280000000008, 2024-05-08T08:52:37.090 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.090 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.090 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.090 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.090 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.090 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.090 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2379 2024-05-08T08:52:37.091 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.091 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.091 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.091 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.091 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.632600+0000", 2024-05-08T08:52:37.091 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.091 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.091 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.091 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.091 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.632600+0000", 2024-05-08T08:52:37.091 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.091 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.091 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.092 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.092 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.632832+0000", 2024-05-08T08:52:37.092 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00023149800000000001 2024-05-08T08:52:37.092 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.092 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.092 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.092 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.632835+0000", 2024-05-08T08:52:37.092 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7100000000000001e-06 2024-05-08T08:52:37.092 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.092 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.092 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.092 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.632846+0000", 2024-05-08T08:52:37.093 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0361e-05 2024-05-08T08:52:37.093 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.093 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.093 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.093 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.632853+0000", 2024-05-08T08:52:37.093 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3699999999999997e-06 2024-05-08T08:52:37.093 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.093 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.093 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.093 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.093 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.093 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1996 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.094 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.592888+0000", 2024-05-08T08:52:37.094 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.042339851999999997, 2024-05-08T08:52:37.094 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.047807536999999997, 2024-05-08T08:52:37.094 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.094 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.094 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.094 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.094 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.094 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.094 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1996 2024-05-08T08:52:37.094 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.094 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.095 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.095 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.095 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.592888+0000", 2024-05-08T08:52:37.095 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.095 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.095 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.095 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.095 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.592888+0000", 2024-05-08T08:52:37.095 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.095 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.095 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.095 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.096 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.593068+0000", 2024-05-08T08:52:37.096 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00017966600000000001 2024-05-08T08:52:37.096 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.096 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.096 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.096 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.593072+0000", 2024-05-08T08:52:37.096 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3309999999999997e-06 2024-05-08T08:52:37.096 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.096 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.096 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.096 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.593084+0000", 2024-05-08T08:52:37.097 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.204e-05 2024-05-08T08:52:37.097 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.097 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.097 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.097 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.593092+0000", 2024-05-08T08:52:37.097 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.6000000000000001e-06 2024-05-08T08:52:37.097 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.097 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.097 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.097 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.097 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.097 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2059 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.097 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.599670+0000", 2024-05-08T08:52:37.098 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.035558330999999999, 2024-05-08T08:52:37.098 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.041067687999999998, 2024-05-08T08:52:37.098 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.098 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.098 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.098 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.098 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.098 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.098 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2059 2024-05-08T08:52:37.098 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.098 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.098 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.099 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.099 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.599670+0000", 2024-05-08T08:52:37.099 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.099 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.099 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.099 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.099 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.599670+0000", 2024-05-08T08:52:37.099 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.099 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.099 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.099 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.099 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.600034+0000", 2024-05-08T08:52:37.100 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00036467500000000001 2024-05-08T08:52:37.100 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.100 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.100 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.100 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.600038+0000", 2024-05-08T08:52:37.100 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.709e-06 2024-05-08T08:52:37.100 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.100 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.100 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.100 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.600047+0000", 2024-05-08T08:52:37.100 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.5499999999999995e-06 2024-05-08T08:52:37.100 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.100 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.101 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.101 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.600054+0000", 2024-05-08T08:52:37.101 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0820000000000001e-06 2024-05-08T08:52:37.101 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.101 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.101 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.101 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.101 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.101 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2123 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.101 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.607241+0000", 2024-05-08T08:52:37.101 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.027986404999999999, 2024-05-08T08:52:37.101 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.033524086000000002, 2024-05-08T08:52:37.101 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.102 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.102 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.102 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.102 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.102 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.102 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2123 2024-05-08T08:52:37.102 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.102 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.102 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.102 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.102 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.607241+0000", 2024-05-08T08:52:37.102 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.102 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.103 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.103 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.103 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.607241+0000", 2024-05-08T08:52:37.103 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.103 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.103 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.103 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.103 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.607437+0000", 2024-05-08T08:52:37.103 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00019505299999999999 2024-05-08T08:52:37.103 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.103 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.103 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.104 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.607441+0000", 2024-05-08T08:52:37.104 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3220000000000003e-06 2024-05-08T08:52:37.104 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.104 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.104 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.104 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.607453+0000", 2024-05-08T08:52:37.104 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1737e-05 2024-05-08T08:52:37.104 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.104 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.104 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.104 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.607460+0000", 2024-05-08T08:52:37.104 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4139999999999997e-06 2024-05-08T08:52:37.105 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.105 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.105 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.105 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.105 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.105 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2188 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.105 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.613678+0000", 2024-05-08T08:52:37.105 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.021550192999999999, 2024-05-08T08:52:37.105 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.027117858000000002, 2024-05-08T08:52:37.105 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.105 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.105 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.106 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.106 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.106 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.106 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2188 2024-05-08T08:52:37.106 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.106 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.106 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.106 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.106 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.613678+0000", 2024-05-08T08:52:37.106 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.106 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.107 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.107 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.107 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.613678+0000", 2024-05-08T08:52:37.107 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.107 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.107 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.107 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.107 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.613899+0000", 2024-05-08T08:52:37.107 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00022157500000000001 2024-05-08T08:52:37.107 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.107 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.107 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.107 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.613904+0000", 2024-05-08T08:52:37.107 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5650000000000003e-06 2024-05-08T08:52:37.108 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.108 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.108 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.108 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.613915+0000", 2024-05-08T08:52:37.108 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1396999999999999e-05 2024-05-08T08:52:37.108 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.108 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.108 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.108 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.613923+0000", 2024-05-08T08:52:37.108 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8019999999999992e-06 2024-05-08T08:52:37.108 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.108 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.108 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.109 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.109 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.109 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2252 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.109 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.619961+0000", 2024-05-08T08:52:37.109 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.015266686999999999, 2024-05-08T08:52:37.109 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.020861633000000001, 2024-05-08T08:52:37.109 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.109 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.109 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.109 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.109 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.109 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.109 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2252 2024-05-08T08:52:37.110 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.110 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.110 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.110 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.110 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.619961+0000", 2024-05-08T08:52:37.110 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.110 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.110 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.110 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.110 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.619961+0000", 2024-05-08T08:52:37.110 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.111 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.111 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.111 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.111 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.620039+0000", 2024-05-08T08:52:37.111 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7919999999999999e-05 2024-05-08T08:52:37.111 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.111 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.111 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.111 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.620043+0000", 2024-05-08T08:52:37.111 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0620000000000002e-06 2024-05-08T08:52:37.111 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.111 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.111 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.112 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.620053+0000", 2024-05-08T08:52:37.112 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3139999999999993e-06 2024-05-08T08:52:37.112 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.112 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.112 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.112 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.620059+0000", 2024-05-08T08:52:37.112 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8689999999999998e-06 2024-05-08T08:52:37.112 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.112 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.112 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.112 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.112 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.112 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2317 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.113 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.626724+0000", 2024-05-08T08:52:37.113 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0085043649999999998, 2024-05-08T08:52:37.113 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.014127859, 2024-05-08T08:52:37.113 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.113 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.113 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.113 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.113 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.113 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.113 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2317 2024-05-08T08:52:37.113 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.113 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.114 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.114 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.114 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.626724+0000", 2024-05-08T08:52:37.114 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.114 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.114 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.114 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.114 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.626724+0000", 2024-05-08T08:52:37.114 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.114 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.114 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.114 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.115 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.626959+0000", 2024-05-08T08:52:37.115 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00023558800000000001 2024-05-08T08:52:37.115 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.115 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.115 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.115 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.626963+0000", 2024-05-08T08:52:37.115 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2910000000000001e-06 2024-05-08T08:52:37.115 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.115 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.115 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.115 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.626972+0000", 2024-05-08T08:52:37.115 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.6580000000000008e-06 2024-05-08T08:52:37.116 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.116 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.116 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.116 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.626978+0000", 2024-05-08T08:52:37.116 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2600000000000002e-06 2024-05-08T08:52:37.116 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.116 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.116 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.116 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.116 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.116 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2381 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.116 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.632864+0000", 2024-05-08T08:52:37.117 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0023643850000000001, 2024-05-08T08:52:37.117 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.008030578, 2024-05-08T08:52:37.117 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.117 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.117 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.117 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.117 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.117 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.117 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2381 2024-05-08T08:52:37.117 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.117 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.117 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.117 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.118 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.632864+0000", 2024-05-08T08:52:37.118 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.118 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.118 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.118 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.118 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.632864+0000", 2024-05-08T08:52:37.118 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.118 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.118 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.118 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.118 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.632958+0000", 2024-05-08T08:52:37.119 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4925000000000003e-05 2024-05-08T08:52:37.119 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.119 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.119 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.119 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.632962+0000", 2024-05-08T08:52:37.119 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7639999999999999e-06 2024-05-08T08:52:37.119 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.119 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.119 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.119 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.632971+0000", 2024-05-08T08:52:37.119 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.5890000000000003e-06 2024-05-08T08:52:37.119 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.119 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.120 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.120 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.632975+0000", 2024-05-08T08:52:37.120 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1370000000000004e-06 2024-05-08T08:52:37.120 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.120 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.120 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.120 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.120 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.120 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1998 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.120 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.593103+0000", 2024-05-08T08:52:37.120 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.042124612999999998, 2024-05-08T08:52:37.120 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.047856724000000003, 2024-05-08T08:52:37.120 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.121 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.121 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.121 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.121 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.121 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.121 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1998 2024-05-08T08:52:37.121 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.121 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.121 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.121 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.121 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.593103+0000", 2024-05-08T08:52:37.121 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.122 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.122 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.122 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.122 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.593103+0000", 2024-05-08T08:52:37.122 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.122 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.122 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.122 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.122 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.593327+0000", 2024-05-08T08:52:37.122 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00022386199999999999 2024-05-08T08:52:37.122 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.122 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.122 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.123 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.593332+0000", 2024-05-08T08:52:37.123 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6720000000000003e-06 2024-05-08T08:52:37.123 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.123 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.123 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.123 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.593343+0000", 2024-05-08T08:52:37.123 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0854e-05 2024-05-08T08:52:37.123 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.123 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.123 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.123 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.593353+0000", 2024-05-08T08:52:37.123 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0456e-05 2024-05-08T08:52:37.124 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.124 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.124 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.124 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.124 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.124 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2061 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.124 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.600065+0000", 2024-05-08T08:52:37.124 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.035163152000000003, 2024-05-08T08:52:37.124 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.040952593000000002, 2024-05-08T08:52:37.124 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.124 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.124 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.124 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.125 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.125 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.125 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2061 2024-05-08T08:52:37.125 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.125 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.125 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.125 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.125 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.600065+0000", 2024-05-08T08:52:37.125 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.125 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.125 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.125 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.125 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.600065+0000", 2024-05-08T08:52:37.126 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.126 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.126 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.126 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.126 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.600065+0000", 2024-05-08T08:52:37.126 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5400000000000002e-07 2024-05-08T08:52:37.126 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.126 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.126 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.126 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.600068+0000", 2024-05-08T08:52:37.126 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.463e-06 2024-05-08T08:52:37.127 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.127 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.127 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.127 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.600074+0000", 2024-05-08T08:52:37.127 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5930000000000002e-06 2024-05-08T08:52:37.127 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.127 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.127 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.127 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.600080+0000", 2024-05-08T08:52:37.127 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8889999999999999e-06 2024-05-08T08:52:37.127 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.127 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.127 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.128 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.128 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.128 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2125 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.128 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.607471+0000", 2024-05-08T08:52:37.128 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.027756658999999999, 2024-05-08T08:52:37.128 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.033605923000000003, 2024-05-08T08:52:37.128 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.128 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.128 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.128 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.128 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.128 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.128 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2125 2024-05-08T08:52:37.129 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.129 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.129 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.129 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.129 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.607471+0000", 2024-05-08T08:52:37.129 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.129 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.129 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.129 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.129 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.607471+0000", 2024-05-08T08:52:37.129 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.129 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.130 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.130 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.130 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.607820+0000", 2024-05-08T08:52:37.130 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00034893199999999999 2024-05-08T08:52:37.130 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.130 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.130 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.130 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.607824+0000", 2024-05-08T08:52:37.130 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5839999999999999e-06 2024-05-08T08:52:37.130 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.130 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.130 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.131 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.607832+0000", 2024-05-08T08:52:37.131 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8450000000000003e-06 2024-05-08T08:52:37.131 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.131 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.131 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.131 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.607836+0000", 2024-05-08T08:52:37.131 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6369999999999998e-06 2024-05-08T08:52:37.131 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.131 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.131 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.131 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.131 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.131 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2190 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.132 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.613935+0000", 2024-05-08T08:52:37.132 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.021292744999999998, 2024-05-08T08:52:37.132 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.027186838000000001, 2024-05-08T08:52:37.132 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.132 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.132 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.132 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.132 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.132 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.132 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2190 2024-05-08T08:52:37.132 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.132 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.133 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.133 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.133 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.613935+0000", 2024-05-08T08:52:37.133 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.133 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.133 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.133 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.133 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.613935+0000", 2024-05-08T08:52:37.133 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.133 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.133 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.133 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.134 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.613936+0000", 2024-05-08T08:52:37.134 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4700000000000001e-07 2024-05-08T08:52:37.134 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.134 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.134 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.134 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.613938+0000", 2024-05-08T08:52:37.134 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6860000000000002e-06 2024-05-08T08:52:37.134 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.134 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.134 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.134 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.613946+0000", 2024-05-08T08:52:37.134 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9160000000000001e-06 2024-05-08T08:52:37.135 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.135 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.135 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.135 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.613958+0000", 2024-05-08T08:52:37.135 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2004999999999999e-05 2024-05-08T08:52:37.135 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.135 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.135 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.135 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.135 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.135 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2254 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.135 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.620071+0000", 2024-05-08T08:52:37.135 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.015156704999999999, 2024-05-08T08:52:37.136 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.02108169, 2024-05-08T08:52:37.136 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.136 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.136 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.136 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.136 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.136 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.136 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2254 2024-05-08T08:52:37.136 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.136 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.136 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.136 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.137 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.620071+0000", 2024-05-08T08:52:37.137 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.137 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.137 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.137 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.137 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.620071+0000", 2024-05-08T08:52:37.137 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.137 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.137 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.137 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.137 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.620167+0000", 2024-05-08T08:52:37.137 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6175999999999995e-05 2024-05-08T08:52:37.137 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.138 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.138 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.138 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.620171+0000", 2024-05-08T08:52:37.138 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1530000000000001e-06 2024-05-08T08:52:37.138 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.138 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.138 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.138 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.620178+0000", 2024-05-08T08:52:37.138 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2919999999999998e-06 2024-05-08T08:52:37.138 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.138 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.138 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.138 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.620183+0000", 2024-05-08T08:52:37.139 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7999999999999998e-06 2024-05-08T08:52:37.139 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.139 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.139 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.139 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.139 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.139 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2319 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.139 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.626989+0000", 2024-05-08T08:52:37.139 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0082391259999999994, 2024-05-08T08:52:37.139 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.014191096, 2024-05-08T08:52:37.139 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.139 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.140 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.140 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.140 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.140 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.140 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2319 2024-05-08T08:52:37.140 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.140 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.140 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.140 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.140 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.626989+0000", 2024-05-08T08:52:37.140 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.140 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.140 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.141 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.141 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.626989+0000", 2024-05-08T08:52:37.141 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.141 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.141 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.141 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.141 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.627081+0000", 2024-05-08T08:52:37.141 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2717e-05 2024-05-08T08:52:37.141 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.141 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.141 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.141 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.627089+0000", 2024-05-08T08:52:37.141 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1559999999999998e-06 2024-05-08T08:52:37.142 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.142 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.142 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.142 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.627095+0000", 2024-05-08T08:52:37.142 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6290000000000004e-06 2024-05-08T08:52:37.142 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.142 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.142 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.142 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.627099+0000", 2024-05-08T08:52:37.142 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0790000000000002e-06 2024-05-08T08:52:37.142 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.142 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.143 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.143 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.143 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.143 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2383 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.143 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.632982+0000", 2024-05-08T08:52:37.143 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0022454559999999998, 2024-05-08T08:52:37.143 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0082255850000000005, 2024-05-08T08:52:37.143 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.143 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.143 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.143 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.143 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.144 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.144 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2383 2024-05-08T08:52:37.144 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.144 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.144 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.144 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.144 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.632982+0000", 2024-05-08T08:52:37.144 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.144 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.144 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.144 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.144 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.632982+0000", 2024-05-08T08:52:37.145 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.145 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.145 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.145 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.145 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.633223+0000", 2024-05-08T08:52:37.145 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00024029100000000001 2024-05-08T08:52:37.145 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.145 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.145 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.145 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.633227+0000", 2024-05-08T08:52:37.145 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8720000000000004e-06 2024-05-08T08:52:37.145 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.146 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.146 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.146 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.633236+0000", 2024-05-08T08:52:37.146 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6549999999999999e-06 2024-05-08T08:52:37.146 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.146 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.146 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.146 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.633243+0000", 2024-05-08T08:52:37.146 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4609999999999996e-06 2024-05-08T08:52:37.146 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.146 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.146 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.147 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.147 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.147 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2000 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.147 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.593365+0000", 2024-05-08T08:52:37.147 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.041862459999999997, 2024-05-08T08:52:37.147 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.047876119000000002, 2024-05-08T08:52:37.147 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.147 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.147 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.147 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.147 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.147 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.148 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2000 2024-05-08T08:52:37.148 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.148 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.148 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.148 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.148 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.593365+0000", 2024-05-08T08:52:37.148 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.148 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.148 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.148 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.148 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.593365+0000", 2024-05-08T08:52:37.148 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.149 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.149 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.149 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.149 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.593580+0000", 2024-05-08T08:52:37.149 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00021491699999999999 2024-05-08T08:52:37.149 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.149 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.149 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.149 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.593584+0000", 2024-05-08T08:52:37.149 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0810000000000004e-06 2024-05-08T08:52:37.149 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.149 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.150 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.150 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.593594+0000", 2024-05-08T08:52:37.150 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8030000000000008e-06 2024-05-08T08:52:37.150 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.150 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.150 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.150 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.593603+0000", 2024-05-08T08:52:37.150 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.8389999999999996e-06 2024-05-08T08:52:37.150 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.150 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.150 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.150 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.151 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.151 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2063 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.151 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.600087+0000", 2024-05-08T08:52:37.151 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.035140434999999998, 2024-05-08T08:52:37.151 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.041202718999999999, 2024-05-08T08:52:37.151 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.151 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.151 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.151 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.151 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.151 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.151 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2063 2024-05-08T08:52:37.151 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.152 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.152 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.152 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.152 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.600087+0000", 2024-05-08T08:52:37.152 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.152 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.152 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.152 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.152 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.600087+0000", 2024-05-08T08:52:37.152 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.152 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.152 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.152 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.153 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.600288+0000", 2024-05-08T08:52:37.153 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00020024499999999999 2024-05-08T08:52:37.153 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.153 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.153 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.153 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.600291+0000", 2024-05-08T08:52:37.153 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4809999999999998e-06 2024-05-08T08:52:37.153 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.153 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.153 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.153 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.600298+0000", 2024-05-08T08:52:37.153 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5699999999999998e-06 2024-05-08T08:52:37.154 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.154 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.154 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.154 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.600302+0000", 2024-05-08T08:52:37.154 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0890000000000002e-06 2024-05-08T08:52:37.154 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.154 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.154 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.154 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.154 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.154 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2127 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.154 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.607846+0000", 2024-05-08T08:52:37.155 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.027382381000000001, 2024-05-08T08:52:37.155 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.033473029000000001, 2024-05-08T08:52:37.155 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.155 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.155 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.155 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.155 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.155 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.155 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2127 2024-05-08T08:52:37.155 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.155 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.155 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.155 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.156 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.607846+0000", 2024-05-08T08:52:37.156 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.156 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.156 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.156 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.156 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.607846+0000", 2024-05-08T08:52:37.156 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.156 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.156 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.156 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.156 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.608076+0000", 2024-05-08T08:52:37.156 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00023003499999999999 2024-05-08T08:52:37.156 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.157 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.157 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.157 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.608084+0000", 2024-05-08T08:52:37.157 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1519999999999992e-06 2024-05-08T08:52:37.157 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.157 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.157 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.157 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.608091+0000", 2024-05-08T08:52:37.157 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7759999999999997e-06 2024-05-08T08:52:37.157 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.157 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.157 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.157 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.608096+0000", 2024-05-08T08:52:37.158 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.318e-06 2024-05-08T08:52:37.158 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.158 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.158 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.158 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.158 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.158 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2192 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.158 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.613966+0000", 2024-05-08T08:52:37.158 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.021262132, 2024-05-08T08:52:37.158 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.027485140000000002, 2024-05-08T08:52:37.158 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.158 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.159 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.159 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.159 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.159 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.159 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2192 2024-05-08T08:52:37.159 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.159 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.159 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.159 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.159 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.613966+0000", 2024-05-08T08:52:37.159 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.159 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.160 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.160 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.160 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.613966+0000", 2024-05-08T08:52:37.160 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.160 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.160 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.160 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.160 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.614157+0000", 2024-05-08T08:52:37.160 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00019141199999999999 2024-05-08T08:52:37.160 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.160 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.160 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.160 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.614160+0000", 2024-05-08T08:52:37.161 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.252e-06 2024-05-08T08:52:37.161 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.161 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.161 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.161 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.614169+0000", 2024-05-08T08:52:37.161 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.8170000000000005e-06 2024-05-08T08:52:37.161 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.161 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.161 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.161 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.614176+0000", 2024-05-08T08:52:37.161 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5950000000000004e-06 2024-05-08T08:52:37.161 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.161 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.162 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.162 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.162 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.162 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2256 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.162 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.620191+0000", 2024-05-08T08:52:37.162 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.015036936000000001, 2024-05-08T08:52:37.162 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.021289057, 2024-05-08T08:52:37.162 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.162 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.162 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.162 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.162 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.163 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.163 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2256 2024-05-08T08:52:37.163 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.163 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.163 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.163 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.163 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.620191+0000", 2024-05-08T08:52:37.163 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.163 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.163 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.163 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.163 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.620191+0000", 2024-05-08T08:52:37.164 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.164 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.164 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.164 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.164 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.620425+0000", 2024-05-08T08:52:37.164 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00023372700000000001 2024-05-08T08:52:37.164 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.164 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.164 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.164 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.620429+0000", 2024-05-08T08:52:37.164 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1659999999999996e-06 2024-05-08T08:52:37.164 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.165 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.165 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.165 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.620437+0000", 2024-05-08T08:52:37.165 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9619999999999993e-06 2024-05-08T08:52:37.165 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.165 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.165 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.165 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.620443+0000", 2024-05-08T08:52:37.165 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.75e-06 2024-05-08T08:52:37.165 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.165 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.165 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.166 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.166 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.166 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2321 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.166 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.627124+0000", 2024-05-08T08:52:37.166 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0081037959999999999, 2024-05-08T08:52:37.166 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.014382621, 2024-05-08T08:52:37.166 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.166 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.166 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.166 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.166 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.166 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.167 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2321 2024-05-08T08:52:37.167 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.167 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.167 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.167 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.167 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.627124+0000", 2024-05-08T08:52:37.167 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.167 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.167 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.167 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.167 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.627124+0000", 2024-05-08T08:52:37.167 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.167 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.168 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.168 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.168 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.627596+0000", 2024-05-08T08:52:37.168 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00047191399999999998 2024-05-08T08:52:37.168 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.168 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.168 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.168 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.627601+0000", 2024-05-08T08:52:37.168 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7740000000000004e-06 2024-05-08T08:52:37.168 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.168 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.168 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.169 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.627611+0000", 2024-05-08T08:52:37.169 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0227999999999999e-05 2024-05-08T08:52:37.169 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.169 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.169 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.169 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.627622+0000", 2024-05-08T08:52:37.169 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0772e-05 2024-05-08T08:52:37.169 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.169 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.169 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.169 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.169 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.169 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2385 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.170 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.633254+0000", 2024-05-08T08:52:37.170 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0019736829999999999, 2024-05-08T08:52:37.170 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0082823919999999995, 2024-05-08T08:52:37.170 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.170 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.170 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.170 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.170 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.170 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.170 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2385 2024-05-08T08:52:37.170 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.170 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.171 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.171 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.171 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.633254+0000", 2024-05-08T08:52:37.171 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.171 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.171 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.171 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.171 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.633254+0000", 2024-05-08T08:52:37.171 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.171 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.171 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.171 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.171 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.633481+0000", 2024-05-08T08:52:37.172 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000226326 2024-05-08T08:52:37.172 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.172 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.172 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.172 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.633484+0000", 2024-05-08T08:52:37.172 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3349999999999998e-06 2024-05-08T08:52:37.172 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.172 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.172 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.172 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.633496+0000", 2024-05-08T08:52:37.172 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1736000000000001e-05 2024-05-08T08:52:37.172 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.172 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.173 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.173 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.633500+0000", 2024-05-08T08:52:37.173 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3880000000000002e-06 2024-05-08T08:52:37.173 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.173 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.173 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.173 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.173 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.173 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2002 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.173 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.593616+0000", 2024-05-08T08:52:37.173 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.041611500000000003, 2024-05-08T08:52:37.173 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.047953280000000001, 2024-05-08T08:52:37.174 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.174 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.174 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.174 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.174 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.174 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.174 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2002 2024-05-08T08:52:37.174 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.174 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.174 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.174 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.175 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.593616+0000", 2024-05-08T08:52:37.175 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.175 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.175 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.175 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.175 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.593616+0000", 2024-05-08T08:52:37.175 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.175 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.175 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.175 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.175 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.593840+0000", 2024-05-08T08:52:37.175 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00022362700000000001 2024-05-08T08:52:37.175 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.176 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.176 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.176 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.593844+0000", 2024-05-08T08:52:37.176 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2259999999999999e-06 2024-05-08T08:52:37.176 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.176 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.176 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.176 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.593855+0000", 2024-05-08T08:52:37.176 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0652999999999999e-05 2024-05-08T08:52:37.176 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.176 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.176 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.177 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.593861+0000", 2024-05-08T08:52:37.177 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5710000000000003e-06 2024-05-08T08:52:37.177 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.177 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.177 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.177 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.177 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.177 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2065 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.177 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.600309+0000", 2024-05-08T08:52:37.177 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.034918567999999997, 2024-05-08T08:52:37.177 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.041305173000000001, 2024-05-08T08:52:37.177 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.178 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.178 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.178 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.178 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.178 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.178 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2065 2024-05-08T08:52:37.178 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.178 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.178 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.178 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.178 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.600309+0000", 2024-05-08T08:52:37.178 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.179 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.179 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.179 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.179 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.600309+0000", 2024-05-08T08:52:37.179 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.179 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.179 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.179 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.179 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.600546+0000", 2024-05-08T08:52:37.179 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000236949 2024-05-08T08:52:37.179 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.179 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.179 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.180 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.600549+0000", 2024-05-08T08:52:37.180 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1599999999999998e-06 2024-05-08T08:52:37.180 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.180 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.180 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.180 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.600557+0000", 2024-05-08T08:52:37.180 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9410000000000007e-06 2024-05-08T08:52:37.180 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.180 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.180 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.180 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.600563+0000", 2024-05-08T08:52:37.180 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8279999999999999e-06 2024-05-08T08:52:37.181 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.181 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.181 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.181 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.181 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.181 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2129 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.181 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.608104+0000", 2024-05-08T08:52:37.181 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.027123669999999999, 2024-05-08T08:52:37.181 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.033538399000000003, 2024-05-08T08:52:37.181 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.181 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.181 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.181 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.182 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.182 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.182 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2129 2024-05-08T08:52:37.182 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.182 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.182 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.182 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.182 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.608104+0000", 2024-05-08T08:52:37.182 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.182 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.182 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.183 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.183 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.608104+0000", 2024-05-08T08:52:37.183 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.183 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.183 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.183 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.183 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.608105+0000", 2024-05-08T08:52:37.183 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1300000000000001e-07 2024-05-08T08:52:37.183 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.183 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.183 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.183 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.608106+0000", 2024-05-08T08:52:37.183 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.579e-06 2024-05-08T08:52:37.184 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.184 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.184 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.184 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.608110+0000", 2024-05-08T08:52:37.184 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.444e-06 2024-05-08T08:52:37.184 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.184 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.184 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.184 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.608112+0000", 2024-05-08T08:52:37.184 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7599999999999998e-06 2024-05-08T08:52:37.184 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.184 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.185 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.185 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.185 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.185 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2194 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.185 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.614185+0000", 2024-05-08T08:52:37.185 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.021042819000000001, 2024-05-08T08:52:37.185 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.027487047000000001, 2024-05-08T08:52:37.185 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.185 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.185 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.185 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.185 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.185 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.186 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2194 2024-05-08T08:52:37.186 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.186 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.186 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.186 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.186 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.614185+0000", 2024-05-08T08:52:37.186 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.186 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.186 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.186 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.186 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.614185+0000", 2024-05-08T08:52:37.186 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.187 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.187 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.187 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.187 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.614544+0000", 2024-05-08T08:52:37.187 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00035883600000000002 2024-05-08T08:52:37.187 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.187 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.187 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.187 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.614547+0000", 2024-05-08T08:52:37.187 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5489999999999998e-06 2024-05-08T08:52:37.187 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.187 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.187 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.188 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.614563+0000", 2024-05-08T08:52:37.188 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5710999999999999e-05 2024-05-08T08:52:37.188 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.188 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.188 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.188 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.614568+0000", 2024-05-08T08:52:37.188 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2700000000000004e-06 2024-05-08T08:52:37.188 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.188 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.188 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.188 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.188 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.188 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2258 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.189 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.620452+0000", 2024-05-08T08:52:37.189 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.014775437000000001, 2024-05-08T08:52:37.189 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.021246899999999999, 2024-05-08T08:52:37.189 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.189 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.189 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.189 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.189 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.189 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.189 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2258 2024-05-08T08:52:37.189 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.189 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.190 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.190 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.190 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.620452+0000", 2024-05-08T08:52:37.190 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.190 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.190 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.190 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.190 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.620452+0000", 2024-05-08T08:52:37.190 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.190 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.190 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.190 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.191 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.620810+0000", 2024-05-08T08:52:37.191 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00035777399999999999 2024-05-08T08:52:37.191 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.191 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.191 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.191 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.620813+0000", 2024-05-08T08:52:37.191 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.2210000000000002e-06 2024-05-08T08:52:37.191 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.191 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.191 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.191 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.620821+0000", 2024-05-08T08:52:37.192 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3119999999999996e-06 2024-05-08T08:52:37.192 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.192 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.192 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.192 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.620825+0000", 2024-05-08T08:52:37.192 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1500000000000001e-06 2024-05-08T08:52:37.192 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.192 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.192 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.192 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.192 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.192 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2323 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.192 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.627636+0000", 2024-05-08T08:52:37.193 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.007591899, 2024-05-08T08:52:37.193 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.014091892, 2024-05-08T08:52:37.193 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.193 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.193 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.193 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.193 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.193 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.193 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2323 2024-05-08T08:52:37.193 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.193 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.193 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.194 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.194 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.627636+0000", 2024-05-08T08:52:37.194 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.194 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.194 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.194 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.194 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.627636+0000", 2024-05-08T08:52:37.194 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.194 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.194 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.194 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.194 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.627636+0000", 2024-05-08T08:52:37.195 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3500000000000002e-07 2024-05-08T08:52:37.195 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.195 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.195 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.195 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.627639+0000", 2024-05-08T08:52:37.195 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7930000000000002e-06 2024-05-08T08:52:37.195 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.195 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.195 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.195 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.627646+0000", 2024-05-08T08:52:37.195 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8360000000000003e-06 2024-05-08T08:52:37.195 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.195 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.196 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.196 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.627655+0000", 2024-05-08T08:52:37.196 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.6629999999999999e-06 2024-05-08T08:52:37.196 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.196 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.196 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.196 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.196 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.196 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2387 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.196 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.633510+0000", 2024-05-08T08:52:37.196 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0017183490000000001, 2024-05-08T08:52:37.196 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0082458670000000005, 2024-05-08T08:52:37.196 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.197 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.197 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.197 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.197 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.197 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.197 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2387 2024-05-08T08:52:37.197 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.197 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.197 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.197 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.197 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.633510+0000", 2024-05-08T08:52:37.197 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.197 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.198 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.198 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.198 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.633510+0000", 2024-05-08T08:52:37.198 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.198 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.198 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.198 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.198 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.633739+0000", 2024-05-08T08:52:37.198 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00022921899999999999 2024-05-08T08:52:37.198 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.198 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.199 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.199 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.633742+0000", 2024-05-08T08:52:37.199 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8760000000000001e-06 2024-05-08T08:52:37.199 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.199 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.199 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.199 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.633750+0000", 2024-05-08T08:52:37.199 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8779999999999998e-06 2024-05-08T08:52:37.199 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.199 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.199 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.199 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.633754+0000", 2024-05-08T08:52:37.200 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4070000000000003e-06 2024-05-08T08:52:37.200 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.200 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.200 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.200 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.200 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.200 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2003 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.200 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.593873+0000", 2024-05-08T08:52:37.200 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.041355333000000001, 2024-05-08T08:52:37.200 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.047917296999999998, 2024-05-08T08:52:37.200 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.200 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.200 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.201 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.201 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.201 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.201 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2003 2024-05-08T08:52:37.201 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.201 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.201 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.201 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.201 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.593873+0000", 2024-05-08T08:52:37.201 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.201 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.201 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.201 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.202 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.593873+0000", 2024-05-08T08:52:37.202 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.202 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.202 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.202 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.202 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.594198+0000", 2024-05-08T08:52:37.202 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00032587799999999999 2024-05-08T08:52:37.202 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.202 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.202 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.202 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.594202+0000", 2024-05-08T08:52:37.202 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6150000000000001e-06 2024-05-08T08:52:37.202 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.203 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.203 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.203 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.594212+0000", 2024-05-08T08:52:37.203 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.9639999999999998e-06 2024-05-08T08:52:37.203 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.203 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.203 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.203 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.594220+0000", 2024-05-08T08:52:37.203 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7200000000000006e-06 2024-05-08T08:52:37.203 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.203 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.203 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.204 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.204 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.204 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2067 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.204 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.600573+0000", 2024-05-08T08:52:37.204 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.034655181, 2024-05-08T08:52:37.204 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.041257223000000003, 2024-05-08T08:52:37.204 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.204 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.204 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.204 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.204 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.204 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.205 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2067 2024-05-08T08:52:37.205 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.205 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.205 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.205 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.205 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.600573+0000", 2024-05-08T08:52:37.205 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.205 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.205 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.205 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.205 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.600573+0000", 2024-05-08T08:52:37.205 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.205 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.206 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.206 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.206 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.600923+0000", 2024-05-08T08:52:37.206 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00035074800000000001 2024-05-08T08:52:37.206 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.206 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.206 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.206 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.600927+0000", 2024-05-08T08:52:37.206 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0249999999999996e-06 2024-05-08T08:52:37.206 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.206 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.206 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.206 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.600936+0000", 2024-05-08T08:52:37.207 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0900000000000005e-06 2024-05-08T08:52:37.207 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.207 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.207 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.207 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.600942+0000", 2024-05-08T08:52:37.207 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.584e-06 2024-05-08T08:52:37.207 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.207 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.207 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.207 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.207 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.208 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2131 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.208 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.608116+0000", 2024-05-08T08:52:37.208 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.027111685, 2024-05-08T08:52:37.208 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.033744290000000003, 2024-05-08T08:52:37.208 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.208 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.208 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.208 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.208 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.208 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.208 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2131 2024-05-08T08:52:37.208 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.208 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.208 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.209 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.209 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.608116+0000", 2024-05-08T08:52:37.209 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.209 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.209 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.209 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.209 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.608116+0000", 2024-05-08T08:52:37.209 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.209 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.209 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.209 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.209 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.608463+0000", 2024-05-08T08:52:37.210 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00034637300000000003 2024-05-08T08:52:37.210 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.210 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.210 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.210 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.608467+0000", 2024-05-08T08:52:37.210 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9199999999999997e-06 2024-05-08T08:52:37.210 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.210 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.210 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.210 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.608477+0000", 2024-05-08T08:52:37.210 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0579e-05 2024-05-08T08:52:37.210 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.211 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.211 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.211 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.608485+0000", 2024-05-08T08:52:37.211 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.6620000000000005e-06 2024-05-08T08:52:37.211 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.211 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.211 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.211 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.211 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.211 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2196 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.211 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.614576+0000", 2024-05-08T08:52:37.211 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.020651424000000002, 2024-05-08T08:52:37.212 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.027317046000000001, 2024-05-08T08:52:37.212 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.212 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.212 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.212 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.212 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.212 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.212 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2196 2024-05-08T08:52:37.212 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.212 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.212 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.212 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.213 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.614576+0000", 2024-05-08T08:52:37.213 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.213 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.213 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.213 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.213 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.614576+0000", 2024-05-08T08:52:37.213 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.213 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.213 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.213 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.213 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.614671+0000", 2024-05-08T08:52:37.213 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4231e-05 2024-05-08T08:52:37.213 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.214 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.214 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.214 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.614673+0000", 2024-05-08T08:52:37.214 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2869999999999999e-06 2024-05-08T08:52:37.214 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.214 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.214 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.214 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.614678+0000", 2024-05-08T08:52:37.214 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0019999999999997e-06 2024-05-08T08:52:37.214 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.214 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.215 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.215 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.614681+0000", 2024-05-08T08:52:37.215 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.281e-06 2024-05-08T08:52:37.215 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.215 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.215 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.215 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.215 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.215 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2260 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.215 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.620833+0000", 2024-05-08T08:52:37.215 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.014395163000000001, 2024-05-08T08:52:37.215 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.021117178, 2024-05-08T08:52:37.215 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.216 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.216 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.216 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.216 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.216 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.216 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2260 2024-05-08T08:52:37.216 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.216 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.216 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.216 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.216 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.620833+0000", 2024-05-08T08:52:37.216 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.216 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.217 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.217 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.217 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.620833+0000", 2024-05-08T08:52:37.217 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.217 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.217 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.217 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.217 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.620960+0000", 2024-05-08T08:52:37.217 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012709599999999999 2024-05-08T08:52:37.217 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.217 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.217 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.217 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.620964+0000", 2024-05-08T08:52:37.218 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.754e-06 2024-05-08T08:52:37.218 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.218 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.218 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.218 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.620975+0000", 2024-05-08T08:52:37.218 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1798e-05 2024-05-08T08:52:37.218 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.218 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.218 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.218 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.620982+0000", 2024-05-08T08:52:37.218 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5579999999999997e-06 2024-05-08T08:52:37.218 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.219 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.219 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.219 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.219 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.219 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2325 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.219 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.627687+0000", 2024-05-08T08:52:37.219 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0075406010000000001, 2024-05-08T08:52:37.219 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.014324055, 2024-05-08T08:52:37.219 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.219 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.219 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.219 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.219 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.220 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.220 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2325 2024-05-08T08:52:37.220 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.220 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.220 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.220 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.220 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.627687+0000", 2024-05-08T08:52:37.220 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.220 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.220 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.220 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.220 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.627687+0000", 2024-05-08T08:52:37.221 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.221 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.221 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.221 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.221 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.627688+0000", 2024-05-08T08:52:37.221 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7e-07 2024-05-08T08:52:37.221 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.221 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.221 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.221 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.627690+0000", 2024-05-08T08:52:37.221 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.311e-06 2024-05-08T08:52:37.221 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.221 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.222 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.222 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.627695+0000", 2024-05-08T08:52:37.222 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9239999999999998e-06 2024-05-08T08:52:37.222 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.222 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.222 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.222 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.627698+0000", 2024-05-08T08:52:37.222 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3340000000000002e-06 2024-05-08T08:52:37.222 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.222 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.222 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.223 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.223 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.223 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2389 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.223 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.633762+0000", 2024-05-08T08:52:37.223 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0014661260000000001, 2024-05-08T08:52:37.223 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0083051170000000007, 2024-05-08T08:52:37.223 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.223 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.223 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.223 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.223 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.223 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.224 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2389 2024-05-08T08:52:37.224 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.224 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.224 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.224 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.224 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.633762+0000", 2024-05-08T08:52:37.224 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.224 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.224 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.224 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.224 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.633762+0000", 2024-05-08T08:52:37.224 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.225 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.225 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.225 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.225 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.633762+0000", 2024-05-08T08:52:37.225 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1300000000000001e-07 2024-05-08T08:52:37.225 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.225 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.225 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.225 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.633764+0000", 2024-05-08T08:52:37.225 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.4270000000000001e-06 2024-05-08T08:52:37.225 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.225 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.225 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.226 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.633768+0000", 2024-05-08T08:52:37.226 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6920000000000001e-06 2024-05-08T08:52:37.226 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.226 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.226 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.226 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.633771+0000", 2024-05-08T08:52:37.226 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.971e-06 2024-05-08T08:52:37.226 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.226 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.226 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.226 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.227 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.227 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2005 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.227 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.594231+0000", 2024-05-08T08:52:37.227 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.040997079999999998, 2024-05-08T08:52:37.227 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.047899518000000002, 2024-05-08T08:52:37.227 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.227 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.227 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.227 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.227 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.227 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.227 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2005 2024-05-08T08:52:37.227 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.228 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.228 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.228 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.228 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.594231+0000", 2024-05-08T08:52:37.228 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.228 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.228 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.228 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.228 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.594231+0000", 2024-05-08T08:52:37.228 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.228 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.228 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.228 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.229 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.594231+0000", 2024-05-08T08:52:37.229 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3399999999999999e-07 2024-05-08T08:52:37.229 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.229 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.229 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.229 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.594234+0000", 2024-05-08T08:52:37.229 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0960000000000001e-06 2024-05-08T08:52:37.229 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.229 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.229 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.229 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.594251+0000", 2024-05-08T08:52:37.229 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6331e-05 2024-05-08T08:52:37.230 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.230 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.230 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.230 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.594257+0000", 2024-05-08T08:52:37.230 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0399999999999998e-06 2024-05-08T08:52:37.230 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.230 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.230 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.230 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.230 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.230 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2069 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.230 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.600966+0000", 2024-05-08T08:52:37.231 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.034261708000000002, 2024-05-08T08:52:37.231 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.041200208000000002, 2024-05-08T08:52:37.231 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.231 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.231 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.231 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.231 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.231 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.231 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2069 2024-05-08T08:52:37.231 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.231 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.231 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.232 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.232 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.600966+0000", 2024-05-08T08:52:37.232 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.232 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.232 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.232 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.232 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.600966+0000", 2024-05-08T08:52:37.232 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.232 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.232 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.232 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.232 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.601151+0000", 2024-05-08T08:52:37.233 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000184883 2024-05-08T08:52:37.233 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.233 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.233 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.233 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.601155+0000", 2024-05-08T08:52:37.233 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5999999999999998e-06 2024-05-08T08:52:37.233 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.233 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.233 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.233 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.601164+0000", 2024-05-08T08:52:37.233 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.8890000000000008e-06 2024-05-08T08:52:37.233 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.234 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.234 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.234 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.601168+0000", 2024-05-08T08:52:37.234 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1869999999999999e-06 2024-05-08T08:52:37.234 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.234 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.234 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.234 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.234 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.234 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2133 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.234 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.608494+0000", 2024-05-08T08:52:37.234 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.026733891999999999, 2024-05-08T08:52:37.234 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.033703153999999999, 2024-05-08T08:52:37.235 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.235 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.235 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.235 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.235 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.235 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.235 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2133 2024-05-08T08:52:37.235 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.235 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.235 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.235 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.235 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.608494+0000", 2024-05-08T08:52:37.235 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.236 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.236 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.236 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.236 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.608494+0000", 2024-05-08T08:52:37.236 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.236 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.236 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.236 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.236 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.608495+0000", 2024-05-08T08:52:37.236 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0800000000000005e-07 2024-05-08T08:52:37.236 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.236 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.236 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.237 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.608497+0000", 2024-05-08T08:52:37.237 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1090000000000001e-06 2024-05-08T08:52:37.237 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.237 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.237 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.237 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.608501+0000", 2024-05-08T08:52:37.237 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1409999999999998e-06 2024-05-08T08:52:37.237 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.237 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.237 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.237 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.608504+0000", 2024-05-08T08:52:37.237 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.2789999999999999e-06 2024-05-08T08:52:37.237 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.238 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.238 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.238 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.238 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.238 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2198 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.238 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.614687+0000", 2024-05-08T08:52:37.238 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.020540801000000001, 2024-05-08T08:52:37.238 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.027537260000000001, 2024-05-08T08:52:37.238 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.238 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.238 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.239 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.239 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.239 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.239 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2198 2024-05-08T08:52:37.239 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.239 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.239 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.239 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.239 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.614687+0000", 2024-05-08T08:52:37.239 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.239 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.239 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.240 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.240 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.614687+0000", 2024-05-08T08:52:37.240 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.240 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.240 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.240 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.240 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.614937+0000", 2024-05-08T08:52:37.240 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00025003800000000001 2024-05-08T08:52:37.240 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.240 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.240 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.240 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.614944+0000", 2024-05-08T08:52:37.241 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9090000000000003e-06 2024-05-08T08:52:37.241 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.241 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.241 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.241 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.614950+0000", 2024-05-08T08:52:37.241 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.105e-06 2024-05-08T08:52:37.241 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.241 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.241 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.241 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.614961+0000", 2024-05-08T08:52:37.241 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.045e-05 2024-05-08T08:52:37.241 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.241 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.241 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.242 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.242 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.242 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2262 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.242 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.620992+0000", 2024-05-08T08:52:37.242 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.014235718, 2024-05-08T08:52:37.242 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.021260481000000001, 2024-05-08T08:52:37.242 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.242 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.242 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.242 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.242 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.242 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.243 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2262 2024-05-08T08:52:37.243 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.243 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.243 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.243 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.243 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.620992+0000", 2024-05-08T08:52:37.243 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.243 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.243 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.243 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.243 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.620992+0000", 2024-05-08T08:52:37.244 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.244 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.244 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.244 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.244 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.621200+0000", 2024-05-08T08:52:37.244 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00020773699999999999 2024-05-08T08:52:37.244 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.244 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.244 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.244 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.621203+0000", 2024-05-08T08:52:37.244 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.331e-06 2024-05-08T08:52:37.244 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.245 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.245 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.245 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.621209+0000", 2024-05-08T08:52:37.245 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9780000000000002e-06 2024-05-08T08:52:37.245 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.245 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.245 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.245 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.621213+0000", 2024-05-08T08:52:37.245 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0509999999999998e-06 2024-05-08T08:52:37.245 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.245 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.245 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.245 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.246 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.246 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2327 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.246 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.627703+0000", 2024-05-08T08:52:37.246 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0075249799999999997, 2024-05-08T08:52:37.246 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.014576820000000001, 2024-05-08T08:52:37.246 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.246 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.246 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.246 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.246 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.246 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.246 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2327 2024-05-08T08:52:37.247 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.247 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.247 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.247 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.247 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.627703+0000", 2024-05-08T08:52:37.247 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.247 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.247 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.247 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.247 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.627703+0000", 2024-05-08T08:52:37.247 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.247 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.247 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.248 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.248 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.627946+0000", 2024-05-08T08:52:37.248 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00024265499999999999 2024-05-08T08:52:37.248 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.248 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.248 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.248 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.627950+0000", 2024-05-08T08:52:37.248 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3819999999999997e-06 2024-05-08T08:52:37.248 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.248 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.248 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.248 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.627964+0000", 2024-05-08T08:52:37.249 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.4028e-05 2024-05-08T08:52:37.249 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.249 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.249 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.249 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.627969+0000", 2024-05-08T08:52:37.249 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8080000000000004e-06 2024-05-08T08:52:37.249 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.249 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.249 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.249 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.249 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.249 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2391 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.250 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.633775+0000", 2024-05-08T08:52:37.250 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0014528919999999999, 2024-05-08T08:52:37.250 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0085341730000000008, 2024-05-08T08:52:37.250 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.250 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.250 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.250 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.250 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.250 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.250 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2391 2024-05-08T08:52:37.250 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.250 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.251 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.251 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.251 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.633775+0000", 2024-05-08T08:52:37.251 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.251 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.251 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.251 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.251 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.633775+0000", 2024-05-08T08:52:37.251 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.251 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.251 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.251 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.251 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.633991+0000", 2024-05-08T08:52:37.252 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00021626999999999999 2024-05-08T08:52:37.252 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.252 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.252 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.252 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.634002+0000", 2024-05-08T08:52:37.252 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0569e-05 2024-05-08T08:52:37.252 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.252 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.252 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.252 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.634011+0000", 2024-05-08T08:52:37.252 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3740000000000004e-06 2024-05-08T08:52:37.252 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.253 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.253 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.253 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.634018+0000", 2024-05-08T08:52:37.253 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8959999999999997e-06 2024-05-08T08:52:37.253 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.253 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.253 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.253 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.253 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.253 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2007 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.253 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.594268+0000", 2024-05-08T08:52:37.253 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.040959978000000001, 2024-05-08T08:52:37.254 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.048086169999999998, 2024-05-08T08:52:37.254 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.254 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.254 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.254 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.254 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.254 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.254 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2007 2024-05-08T08:52:37.254 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.254 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.254 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.254 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.254 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.594268+0000", 2024-05-08T08:52:37.255 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.255 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.255 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.255 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.255 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.594268+0000", 2024-05-08T08:52:37.255 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.255 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.255 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.255 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.255 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.594479+0000", 2024-05-08T08:52:37.255 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000210841 2024-05-08T08:52:37.255 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.255 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.256 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.256 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.594487+0000", 2024-05-08T08:52:37.256 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4160000000000004e-06 2024-05-08T08:52:37.256 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.256 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.256 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.256 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.594502+0000", 2024-05-08T08:52:37.256 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5255999999999999e-05 2024-05-08T08:52:37.256 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.256 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.256 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.256 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.594509+0000", 2024-05-08T08:52:37.257 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.533e-06 2024-05-08T08:52:37.257 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.257 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.257 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.257 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.257 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.257 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2071 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.257 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.601176+0000", 2024-05-08T08:52:37.257 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.034052298000000002, 2024-05-08T08:52:37.257 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.041236994999999999, 2024-05-08T08:52:37.257 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.257 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.258 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.258 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.258 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.258 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.258 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2071 2024-05-08T08:52:37.258 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.258 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.258 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.258 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.258 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.601176+0000", 2024-05-08T08:52:37.258 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.258 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.259 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.259 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.259 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.601176+0000", 2024-05-08T08:52:37.259 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.259 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.259 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.259 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.259 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.601546+0000", 2024-05-08T08:52:37.259 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00037035699999999998 2024-05-08T08:52:37.259 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.259 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.259 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.259 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.601549+0000", 2024-05-08T08:52:37.260 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.2509999999999999e-06 2024-05-08T08:52:37.260 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.260 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.260 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.260 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.601556+0000", 2024-05-08T08:52:37.260 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2479999999999999e-06 2024-05-08T08:52:37.260 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.260 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.260 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.260 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.601561+0000", 2024-05-08T08:52:37.260 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3499999999999999e-06 2024-05-08T08:52:37.261 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.261 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.261 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.261 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.261 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.261 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2135 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.261 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.608508+0000", 2024-05-08T08:52:37.261 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.026719762000000001, 2024-05-08T08:52:37.261 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.033960728000000003, 2024-05-08T08:52:37.261 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.261 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.261 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.261 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.262 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.262 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.262 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2135 2024-05-08T08:52:37.262 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.262 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.262 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.262 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.262 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.608508+0000", 2024-05-08T08:52:37.262 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.262 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.262 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.262 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.263 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.608508+0000", 2024-05-08T08:52:37.263 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.263 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.263 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.263 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.263 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.608508+0000", 2024-05-08T08:52:37.263 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6800000000000002e-07 2024-05-08T08:52:37.263 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.263 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.263 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.263 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.608510+0000", 2024-05-08T08:52:37.263 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3599999999999999e-06 2024-05-08T08:52:37.264 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.264 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.264 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.264 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.608513+0000", 2024-05-08T08:52:37.264 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.2440000000000002e-06 2024-05-08T08:52:37.264 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.264 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.264 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.264 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.608516+0000", 2024-05-08T08:52:37.264 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6460000000000002e-06 2024-05-08T08:52:37.264 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.264 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.264 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.265 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.265 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.265 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2200 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.265 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.614975+0000", 2024-05-08T08:52:37.265 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.020253090000000001, 2024-05-08T08:52:37.265 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.027553675999999999, 2024-05-08T08:52:37.265 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.265 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.265 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.265 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.265 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.265 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.266 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2200 2024-05-08T08:52:37.266 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.266 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.266 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.266 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.266 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.614975+0000", 2024-05-08T08:52:37.266 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.266 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.266 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.266 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.266 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.614975+0000", 2024-05-08T08:52:37.266 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.267 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.267 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.267 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.267 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.614975+0000", 2024-05-08T08:52:37.267 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3500000000000002e-07 2024-05-08T08:52:37.267 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.267 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.267 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.267 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.614977+0000", 2024-05-08T08:52:37.267 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.7710000000000001e-06 2024-05-08T08:52:37.267 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.267 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.267 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.268 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.614982+0000", 2024-05-08T08:52:37.268 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3159999999999996e-06 2024-05-08T08:52:37.268 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.268 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.268 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.268 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.614985+0000", 2024-05-08T08:52:37.268 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0309999999999999e-06 2024-05-08T08:52:37.268 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.268 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.268 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.268 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.268 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.268 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2264 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.269 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.621221+0000", 2024-05-08T08:52:37.269 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.014006802, 2024-05-08T08:52:37.269 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.021347365, 2024-05-08T08:52:37.269 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.269 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.269 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.269 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.269 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.269 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.269 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2264 2024-05-08T08:52:37.269 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.269 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.269 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.270 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.270 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.621221+0000", 2024-05-08T08:52:37.270 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.270 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.270 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.270 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.270 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.621221+0000", 2024-05-08T08:52:37.270 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.270 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.270 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.270 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.270 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.621466+0000", 2024-05-08T08:52:37.271 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00024467400000000001 2024-05-08T08:52:37.271 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.271 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.271 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.271 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.621470+0000", 2024-05-08T08:52:37.271 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9099999999999998e-06 2024-05-08T08:52:37.271 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.271 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.271 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.271 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.621484+0000", 2024-05-08T08:52:37.271 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.4443999999999999e-05 2024-05-08T08:52:37.271 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.271 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.272 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.272 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.621491+0000", 2024-05-08T08:52:37.272 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8140000000000003e-06 2024-05-08T08:52:37.272 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.272 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.272 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.272 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.272 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.272 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2328 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.272 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.627977+0000", 2024-05-08T08:52:37.272 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0072505060000000003, 2024-05-08T08:52:37.272 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.014621518999999999, 2024-05-08T08:52:37.272 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.273 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.273 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.273 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.273 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.273 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.273 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2328 2024-05-08T08:52:37.273 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.273 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.273 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.273 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.273 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.627977+0000", 2024-05-08T08:52:37.273 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.273 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.274 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.274 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.274 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.627977+0000", 2024-05-08T08:52:37.274 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.274 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.274 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.274 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.274 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.628501+0000", 2024-05-08T08:52:37.274 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00052322600000000001 2024-05-08T08:52:37.274 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.274 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.274 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.275 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.628504+0000", 2024-05-08T08:52:37.275 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1810000000000001e-06 2024-05-08T08:52:37.275 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.275 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.275 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.275 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.628511+0000", 2024-05-08T08:52:37.275 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.447e-06 2024-05-08T08:52:37.275 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.275 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.275 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.275 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.628516+0000", 2024-05-08T08:52:37.275 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5600000000000004e-06 2024-05-08T08:52:37.276 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.276 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.276 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.276 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.276 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.276 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2393 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.276 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.634029+0000", 2024-05-08T08:52:37.276 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0011992629999999999, 2024-05-08T08:52:37.276 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0085982409999999995, 2024-05-08T08:52:37.276 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.276 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.276 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.277 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.277 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.277 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.277 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2393 2024-05-08T08:52:37.277 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.277 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.277 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.277 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.277 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.634029+0000", 2024-05-08T08:52:37.277 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.277 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.277 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.277 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.277 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.634029+0000", 2024-05-08T08:52:37.278 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.278 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.278 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.278 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.278 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.634121+0000", 2024-05-08T08:52:37.278 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2317000000000004e-05 2024-05-08T08:52:37.278 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.278 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.278 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.278 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.634125+0000", 2024-05-08T08:52:37.278 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.551e-06 2024-05-08T08:52:37.279 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.279 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.279 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.279 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.634133+0000", 2024-05-08T08:52:37.279 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7050000000000005e-06 2024-05-08T08:52:37.279 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.279 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.279 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.279 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.634140+0000", 2024-05-08T08:52:37.279 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6200000000000001e-06 2024-05-08T08:52:37.279 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.279 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.280 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.280 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.280 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.280 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2009 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.280 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.594520+0000", 2024-05-08T08:52:37.280 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.040708273000000003, 2024-05-08T08:52:37.280 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.048136681000000001, 2024-05-08T08:52:37.280 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.280 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.280 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.280 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.280 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.281 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.281 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2009 2024-05-08T08:52:37.281 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.281 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.281 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.281 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.281 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.594520+0000", 2024-05-08T08:52:37.281 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.281 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.281 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.281 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.281 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.594520+0000", 2024-05-08T08:52:37.281 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.282 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.282 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.282 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.282 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.594520+0000", 2024-05-08T08:52:37.282 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.82e-07 2024-05-08T08:52:37.282 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.282 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.282 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.282 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.594523+0000", 2024-05-08T08:52:37.282 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.101e-06 2024-05-08T08:52:37.282 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.282 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.283 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.283 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.594530+0000", 2024-05-08T08:52:37.283 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8569999999999997e-06 2024-05-08T08:52:37.283 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.283 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.283 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.283 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.594536+0000", 2024-05-08T08:52:37.283 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5520000000000004e-06 2024-05-08T08:52:37.283 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.283 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.283 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.283 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.284 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.284 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2073 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.284 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.601569+0000", 2024-05-08T08:52:37.284 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.033658891000000003, 2024-05-08T08:52:37.284 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.041114699999999997, 2024-05-08T08:52:37.284 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.284 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.284 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.284 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.284 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.284 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.284 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2073 2024-05-08T08:52:37.285 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.285 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.285 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.285 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.285 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.601569+0000", 2024-05-08T08:52:37.285 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.285 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.285 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.285 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.285 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.601569+0000", 2024-05-08T08:52:37.285 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.285 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.285 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.286 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.286 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.601569+0000", 2024-05-08T08:52:37.286 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.15e-07 2024-05-08T08:52:37.286 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.286 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.286 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.286 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.601571+0000", 2024-05-08T08:52:37.286 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5349999999999999e-06 2024-05-08T08:52:37.286 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.286 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.286 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.286 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.601574+0000", 2024-05-08T08:52:37.287 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6179999999999999e-06 2024-05-08T08:52:37.287 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.287 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.287 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.287 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.601577+0000", 2024-05-08T08:52:37.287 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.88e-06 2024-05-08T08:52:37.287 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.287 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.287 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.287 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.287 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.287 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2137 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.288 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.608519+0000", 2024-05-08T08:52:37.288 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.026708862999999999, 2024-05-08T08:52:37.288 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.034191676999999997, 2024-05-08T08:52:37.288 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.288 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.288 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.288 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.288 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.288 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.288 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2137 2024-05-08T08:52:37.288 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.288 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.288 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.289 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.289 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.608519+0000", 2024-05-08T08:52:37.289 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.289 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.289 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.289 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.289 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.608519+0000", 2024-05-08T08:52:37.289 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.289 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.289 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.289 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.289 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.608598+0000", 2024-05-08T08:52:37.289 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9111999999999997e-05 2024-05-08T08:52:37.290 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.290 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.290 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.290 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.608602+0000", 2024-05-08T08:52:37.290 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.856e-06 2024-05-08T08:52:37.290 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.290 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.290 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.290 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.608610+0000", 2024-05-08T08:52:37.290 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.6450000000000005e-06 2024-05-08T08:52:37.290 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.290 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.291 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.291 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.608615+0000", 2024-05-08T08:52:37.291 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3190000000000002e-06 2024-05-08T08:52:37.291 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.291 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.291 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.291 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.291 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.291 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2202 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.291 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.614990+0000", 2024-05-08T08:52:37.291 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.020238163999999999, 2024-05-08T08:52:37.291 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.027749060999999998, 2024-05-08T08:52:37.292 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.292 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.292 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.292 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.292 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.292 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.292 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2202 2024-05-08T08:52:37.292 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.292 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.292 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.292 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.292 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.614990+0000", 2024-05-08T08:52:37.292 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.293 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.293 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.293 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.293 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.614990+0000", 2024-05-08T08:52:37.293 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.293 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.293 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.293 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.293 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.615347+0000", 2024-05-08T08:52:37.293 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00035684200000000001 2024-05-08T08:52:37.293 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.293 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.294 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.294 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.615352+0000", 2024-05-08T08:52:37.294 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9520000000000002e-06 2024-05-08T08:52:37.294 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.294 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.294 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.294 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.615363+0000", 2024-05-08T08:52:37.294 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1766999999999999e-05 2024-05-08T08:52:37.294 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.294 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.294 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.294 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.615371+0000", 2024-05-08T08:52:37.295 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0450000000000001e-06 2024-05-08T08:52:37.295 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.295 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.295 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.295 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.295 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.295 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2266 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.295 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.621502+0000", 2024-05-08T08:52:37.295 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.013725810999999999, 2024-05-08T08:52:37.295 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.021263557999999998, 2024-05-08T08:52:37.295 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.295 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.295 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.296 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.296 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.296 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.296 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2266 2024-05-08T08:52:37.296 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.296 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.296 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.296 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.296 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.621502+0000", 2024-05-08T08:52:37.296 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.296 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.296 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.296 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.297 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.621502+0000", 2024-05-08T08:52:37.297 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.297 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.297 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.297 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.297 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.621503+0000", 2024-05-08T08:52:37.297 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8999999999999996e-07 2024-05-08T08:52:37.297 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.297 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.297 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.297 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.621506+0000", 2024-05-08T08:52:37.297 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8459999999999999e-06 2024-05-08T08:52:37.298 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.298 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.298 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.298 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.621513+0000", 2024-05-08T08:52:37.298 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1520000000000003e-06 2024-05-08T08:52:37.298 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.298 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.298 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.298 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.621518+0000", 2024-05-08T08:52:37.298 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7039999999999999e-06 2024-05-08T08:52:37.298 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.298 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.299 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.299 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.299 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.299 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2331 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.299 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.628524+0000", 2024-05-08T08:52:37.299 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0067040900000000002, 2024-05-08T08:52:37.299 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.01428665, 2024-05-08T08:52:37.299 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.299 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.299 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.299 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.299 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.300 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.300 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2331 2024-05-08T08:52:37.300 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.300 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.300 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.300 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.300 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.628524+0000", 2024-05-08T08:52:37.300 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.300 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.300 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.300 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.300 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.628524+0000", 2024-05-08T08:52:37.300 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.301 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.301 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.301 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.301 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.628524+0000", 2024-05-08T08:52:37.301 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3500000000000002e-07 2024-05-08T08:52:37.301 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.301 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.301 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.301 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.628526+0000", 2024-05-08T08:52:37.301 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.4130000000000001e-06 2024-05-08T08:52:37.301 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.301 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.301 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.302 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.628529+0000", 2024-05-08T08:52:37.302 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5619999999999999e-06 2024-05-08T08:52:37.302 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.302 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.302 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.302 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.628532+0000", 2024-05-08T08:52:37.302 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8090000000000001e-06 2024-05-08T08:52:37.302 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.302 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.302 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.302 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.302 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.303 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2395 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.303 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.634150+0000", 2024-05-08T08:52:37.303 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0010776799999999999, 2024-05-08T08:52:37.303 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0087154410000000009, 2024-05-08T08:52:37.303 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.303 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.303 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.303 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.303 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.303 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.303 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2395 2024-05-08T08:52:37.303 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.304 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.304 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.304 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.304 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.634150+0000", 2024-05-08T08:52:37.304 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.304 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.304 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.304 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.304 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.634150+0000", 2024-05-08T08:52:37.304 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.304 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.304 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.305 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.305 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.634511+0000", 2024-05-08T08:52:37.305 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00036072200000000001 2024-05-08T08:52:37.305 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.305 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.305 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.305 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.634514+0000", 2024-05-08T08:52:37.305 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1099999999999999e-06 2024-05-08T08:52:37.305 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.305 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.305 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.305 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.634528+0000", 2024-05-08T08:52:37.305 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.4042999999999999e-05 2024-05-08T08:52:37.306 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.306 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.306 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.306 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.634535+0000", 2024-05-08T08:52:37.306 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1459999999999999e-06 2024-05-08T08:52:37.306 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.306 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.306 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.306 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.306 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.306 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2011 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.306 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.594552+0000", 2024-05-08T08:52:37.307 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.040676194999999998, 2024-05-08T08:52:37.307 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.048374819999999999, 2024-05-08T08:52:37.307 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.307 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.307 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.307 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.307 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.307 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.307 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2011 2024-05-08T08:52:37.307 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.307 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.307 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.307 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.308 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.594552+0000", 2024-05-08T08:52:37.308 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.308 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.308 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.308 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.308 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.594552+0000", 2024-05-08T08:52:37.308 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.308 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.308 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.308 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.308 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.594733+0000", 2024-05-08T08:52:37.308 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000181509 2024-05-08T08:52:37.309 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.309 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.309 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.309 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.594737+0000", 2024-05-08T08:52:37.309 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6729999999999998e-06 2024-05-08T08:52:37.309 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.309 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.309 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.309 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.594746+0000", 2024-05-08T08:52:37.309 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.9460000000000004e-06 2024-05-08T08:52:37.309 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.309 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.309 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.310 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.594758+0000", 2024-05-08T08:52:37.310 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2181000000000001e-05 2024-05-08T08:52:37.310 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.310 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.310 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.310 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.310 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.310 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2075 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.310 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.601581+0000", 2024-05-08T08:52:37.310 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.033646584, 2024-05-08T08:52:37.310 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.041411372000000002, 2024-05-08T08:52:37.311 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.311 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.311 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.311 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.311 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.311 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.311 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2075 2024-05-08T08:52:37.311 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.311 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.311 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.311 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.311 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.601581+0000", 2024-05-08T08:52:37.311 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.312 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.312 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.312 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.312 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.601581+0000", 2024-05-08T08:52:37.312 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.312 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.312 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.312 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.312 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.601991+0000", 2024-05-08T08:52:37.312 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00040920399999999998 2024-05-08T08:52:37.312 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.312 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.313 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.313 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.601994+0000", 2024-05-08T08:52:37.313 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8589999999999998e-06 2024-05-08T08:52:37.313 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.313 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.313 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.313 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.602001+0000", 2024-05-08T08:52:37.313 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.097e-06 2024-05-08T08:52:37.313 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.313 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.313 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.313 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.602006+0000", 2024-05-08T08:52:37.313 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3200000000000001e-06 2024-05-08T08:52:37.314 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.314 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.314 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.314 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.314 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.314 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2139 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.314 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.608624+0000", 2024-05-08T08:52:37.314 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.026604060999999998, 2024-05-08T08:52:37.314 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.034410686000000003, 2024-05-08T08:52:37.314 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.314 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.314 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.315 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.315 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.315 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.315 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2139 2024-05-08T08:52:37.315 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.315 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.315 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.315 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.315 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.608624+0000", 2024-05-08T08:52:37.315 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.315 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.315 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.316 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.316 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.608624+0000", 2024-05-08T08:52:37.316 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.316 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.316 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.316 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.316 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.608959+0000", 2024-05-08T08:52:37.316 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000335323 2024-05-08T08:52:37.316 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.316 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.316 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.316 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.608964+0000", 2024-05-08T08:52:37.316 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4469999999999999e-06 2024-05-08T08:52:37.317 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.317 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.317 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.317 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.608974+0000", 2024-05-08T08:52:37.317 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0416e-05 2024-05-08T08:52:37.317 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.317 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.317 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.317 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.608981+0000", 2024-05-08T08:52:37.317 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0269999999999998e-06 2024-05-08T08:52:37.317 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.317 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.317 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.317 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.318 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.318 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2204 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.318 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.615384+0000", 2024-05-08T08:52:37.318 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.019844219999999999, 2024-05-08T08:52:37.318 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.027678711000000002, 2024-05-08T08:52:37.318 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.318 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.318 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.318 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.318 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.318 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.318 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2204 2024-05-08T08:52:37.318 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.319 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.319 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.319 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.319 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.615384+0000", 2024-05-08T08:52:37.319 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.319 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.319 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.319 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.319 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.615384+0000", 2024-05-08T08:52:37.319 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.319 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.319 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.320 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.320 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.615478+0000", 2024-05-08T08:52:37.320 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4810999999999997e-05 2024-05-08T08:52:37.320 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.320 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.320 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.320 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.615483+0000", 2024-05-08T08:52:37.320 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1069999999999998e-06 2024-05-08T08:52:37.320 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.320 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.320 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.320 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.615493+0000", 2024-05-08T08:52:37.321 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0227000000000001e-05 2024-05-08T08:52:37.321 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.321 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.321 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.321 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.615500+0000", 2024-05-08T08:52:37.321 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2910000000000002e-06 2024-05-08T08:52:37.321 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.321 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.321 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.321 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.321 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.321 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2268 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.322 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.621526+0000", 2024-05-08T08:52:37.322 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.013701875000000001, 2024-05-08T08:52:37.322 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.021566403000000001, 2024-05-08T08:52:37.322 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.322 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.322 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.322 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.322 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.322 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.322 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2268 2024-05-08T08:52:37.322 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.322 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.323 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.323 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.323 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.621526+0000", 2024-05-08T08:52:37.323 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.323 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.323 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.323 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.323 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.621526+0000", 2024-05-08T08:52:37.323 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.323 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.323 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.323 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.324 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.621718+0000", 2024-05-08T08:52:37.324 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00019165499999999999 2024-05-08T08:52:37.324 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.324 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.324 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.324 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.621722+0000", 2024-05-08T08:52:37.324 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7260000000000002e-06 2024-05-08T08:52:37.324 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.324 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.324 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.324 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.621737+0000", 2024-05-08T08:52:37.324 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.4371e-05 2024-05-08T08:52:37.325 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.325 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.325 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.325 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.621744+0000", 2024-05-08T08:52:37.325 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9809999999999997e-06 2024-05-08T08:52:37.325 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.325 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.325 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.325 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.325 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.325 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2333 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.325 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.628536+0000", 2024-05-08T08:52:37.326 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.006692173, 2024-05-08T08:52:37.326 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.014583904999999999, 2024-05-08T08:52:37.326 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.326 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.326 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.326 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.326 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.326 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.326 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2333 2024-05-08T08:52:37.326 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.326 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.326 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.326 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.327 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.628536+0000", 2024-05-08T08:52:37.327 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.327 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.327 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.327 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.327 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.628536+0000", 2024-05-08T08:52:37.327 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.327 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.327 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.327 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.327 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.628536+0000", 2024-05-08T08:52:37.327 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2399999999999999e-07 2024-05-08T08:52:37.327 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.328 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.328 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.328 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.628537+0000", 2024-05-08T08:52:37.328 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.217e-06 2024-05-08T08:52:37.328 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.328 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.328 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.328 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.628540+0000", 2024-05-08T08:52:37.328 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0419999999999998e-06 2024-05-08T08:52:37.328 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.328 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.328 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.328 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.628543+0000", 2024-05-08T08:52:37.328 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.469e-06 2024-05-08T08:52:37.329 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.329 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.329 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.329 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.329 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.329 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2397 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.329 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.634546+0000", 2024-05-08T08:52:37.329 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.000681801, 2024-05-08T08:52:37.329 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0086059420000000001, 2024-05-08T08:52:37.329 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.329 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.329 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.329 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.330 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.330 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.330 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2397 2024-05-08T08:52:37.330 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.330 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.330 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.330 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.330 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.634546+0000", 2024-05-08T08:52:37.330 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.330 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.330 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.330 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.330 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.634546+0000", 2024-05-08T08:52:37.331 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.331 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.331 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.331 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.331 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.634637+0000", 2024-05-08T08:52:37.331 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0426000000000005e-05 2024-05-08T08:52:37.331 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.331 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.331 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.331 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.634640+0000", 2024-05-08T08:52:37.331 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3129999999999999e-06 2024-05-08T08:52:37.331 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.331 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.332 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.332 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.634646+0000", 2024-05-08T08:52:37.332 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2199999999999997e-06 2024-05-08T08:52:37.332 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.332 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.332 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.332 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.634650+0000", 2024-05-08T08:52:37.332 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9009999999999996e-06 2024-05-08T08:52:37.332 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.332 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.332 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.332 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.333 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.333 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2013 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.333 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.594769+0000", 2024-05-08T08:52:37.333 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.040459356000000002, 2024-05-08T08:52:37.333 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.048439809, 2024-05-08T08:52:37.333 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.333 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.333 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.333 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.333 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.333 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.333 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2013 2024-05-08T08:52:37.334 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.334 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.334 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.334 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.334 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.594769+0000", 2024-05-08T08:52:37.334 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.334 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.334 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.334 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.334 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.594769+0000", 2024-05-08T08:52:37.334 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.334 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.335 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.335 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.335 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.595163+0000", 2024-05-08T08:52:37.335 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00039440900000000002 2024-05-08T08:52:37.335 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.335 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.335 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.335 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.595167+0000", 2024-05-08T08:52:37.335 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9190000000000001e-06 2024-05-08T08:52:37.335 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.335 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.335 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.335 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.595178+0000", 2024-05-08T08:52:37.336 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1525999999999999e-05 2024-05-08T08:52:37.336 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.336 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.336 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.336 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.595185+0000", 2024-05-08T08:52:37.336 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6919999999999996e-06 2024-05-08T08:52:37.336 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.336 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.336 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.336 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.336 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.337 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2077 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.337 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.602014+0000", 2024-05-08T08:52:37.337 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.033213558999999997, 2024-05-08T08:52:37.337 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.041254164000000003, 2024-05-08T08:52:37.337 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.337 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.337 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.337 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.337 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.337 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.337 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2077 2024-05-08T08:52:37.337 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.338 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.338 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.338 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.338 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.602014+0000", 2024-05-08T08:52:37.338 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.338 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.338 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.338 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.338 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.602014+0000", 2024-05-08T08:52:37.338 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.338 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.338 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.339 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.339 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.602015+0000", 2024-05-08T08:52:37.339 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.2099999999999998e-07 2024-05-08T08:52:37.339 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.339 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.339 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.339 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.602016+0000", 2024-05-08T08:52:37.339 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.4529999999999999e-06 2024-05-08T08:52:37.339 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.339 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.339 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.340 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.602020+0000", 2024-05-08T08:52:37.340 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5389999999999999e-06 2024-05-08T08:52:37.340 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.340 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.340 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.340 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.602022+0000", 2024-05-08T08:52:37.340 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8200000000000001e-06 2024-05-08T08:52:37.340 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.340 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.340 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.340 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.341 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.341 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2142 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.341 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.608992+0000", 2024-05-08T08:52:37.341 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.026235469000000001, 2024-05-08T08:52:37.341 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.034333759999999998, 2024-05-08T08:52:37.341 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.341 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.341 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.341 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.341 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.341 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.341 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2142 2024-05-08T08:52:37.341 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.342 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.342 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.342 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.342 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.608992+0000", 2024-05-08T08:52:37.342 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.342 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.342 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.342 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.342 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.608992+0000", 2024-05-08T08:52:37.342 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.342 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.342 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.343 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.343 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.609117+0000", 2024-05-08T08:52:37.343 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012444299999999999 2024-05-08T08:52:37.343 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.343 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.343 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.343 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.609120+0000", 2024-05-08T08:52:37.343 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5439999999999999e-06 2024-05-08T08:52:37.343 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.343 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.343 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.343 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.609128+0000", 2024-05-08T08:52:37.344 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5950000000000001e-06 2024-05-08T08:52:37.344 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.344 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.344 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.344 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.609132+0000", 2024-05-08T08:52:37.344 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4769999999999997e-06 2024-05-08T08:52:37.344 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.344 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.344 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.344 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.344 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.344 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2206 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.345 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.615512+0000", 2024-05-08T08:52:37.345 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.019715993000000001, 2024-05-08T08:52:37.345 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.027872971999999999, 2024-05-08T08:52:37.345 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.345 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.345 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.345 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.345 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.345 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.345 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2206 2024-05-08T08:52:37.345 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.345 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.346 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.346 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.346 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.615512+0000", 2024-05-08T08:52:37.346 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.346 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.346 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.346 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.346 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.615512+0000", 2024-05-08T08:52:37.346 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.346 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.346 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.347 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.347 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.615737+0000", 2024-05-08T08:52:37.347 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00022500499999999999 2024-05-08T08:52:37.347 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.347 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.347 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.347 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.615742+0000", 2024-05-08T08:52:37.347 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9450000000000001e-06 2024-05-08T08:52:37.347 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.347 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.347 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.347 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.615755+0000", 2024-05-08T08:52:37.347 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3208e-05 2024-05-08T08:52:37.348 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.348 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.348 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.348 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.615763+0000", 2024-05-08T08:52:37.348 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0970000000000006e-06 2024-05-08T08:52:37.348 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.348 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.348 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.348 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.348 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.348 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2270 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.348 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.621756+0000", 2024-05-08T08:52:37.349 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.013471942000000001, 2024-05-08T08:52:37.349 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.021657961999999999, 2024-05-08T08:52:37.349 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.349 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.349 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.349 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.349 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.349 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.349 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2270 2024-05-08T08:52:37.349 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.349 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.349 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.349 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.350 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.621756+0000", 2024-05-08T08:52:37.350 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.350 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.350 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.350 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.350 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.621756+0000", 2024-05-08T08:52:37.350 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.350 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.350 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.350 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.350 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.622108+0000", 2024-05-08T08:52:37.350 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00035209599999999998 2024-05-08T08:52:37.350 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.351 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.351 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.351 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.622120+0000", 2024-05-08T08:52:37.351 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.182e-05 2024-05-08T08:52:37.351 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.351 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.351 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.351 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.622131+0000", 2024-05-08T08:52:37.351 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0835999999999999e-05 2024-05-08T08:52:37.351 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.351 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.351 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.354 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.622138+0000", 2024-05-08T08:52:37.354 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4080000000000001e-06 2024-05-08T08:52:37.354 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.354 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.354 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.354 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.354 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.354 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2335 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.355 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.628548+0000", 2024-05-08T08:52:37.355 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0066798209999999998, 2024-05-08T08:52:37.355 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.014892677999999999, 2024-05-08T08:52:37.355 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.355 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.355 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.355 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.355 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.355 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.355 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2335 2024-05-08T08:52:37.355 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.355 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.356 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.356 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.356 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.628548+0000", 2024-05-08T08:52:37.356 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.356 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.356 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.356 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.356 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.628548+0000", 2024-05-08T08:52:37.356 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.356 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.356 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.356 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.356 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.628847+0000", 2024-05-08T08:52:37.356 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00029935200000000002 2024-05-08T08:52:37.356 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.356 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.357 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.357 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.628851+0000", 2024-05-08T08:52:37.357 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7330000000000001e-06 2024-05-08T08:52:37.357 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.357 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.357 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.357 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.628861+0000", 2024-05-08T08:52:37.357 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.7229999999999999e-06 2024-05-08T08:52:37.357 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.357 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.357 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.357 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.628869+0000", 2024-05-08T08:52:37.357 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0250000000000002e-06 2024-05-08T08:52:37.357 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.357 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.358 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.358 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.358 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.358 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2399 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.358 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.634657+0000", 2024-05-08T08:52:37.358 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.00057103500000000001, 2024-05-08T08:52:37.358 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0088130900000000009, 2024-05-08T08:52:37.358 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.358 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.358 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.358 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.358 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.358 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.358 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2399 2024-05-08T08:52:37.358 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.359 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.359 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.359 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.359 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.634657+0000", 2024-05-08T08:52:37.359 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.359 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.359 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.359 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.359 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.634657+0000", 2024-05-08T08:52:37.359 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.359 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.359 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.359 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.359 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.634965+0000", 2024-05-08T08:52:37.360 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00030851000000000001 2024-05-08T08:52:37.360 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.360 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.360 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.360 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.634968+0000", 2024-05-08T08:52:37.360 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0900000000000001e-06 2024-05-08T08:52:37.360 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.360 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.360 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.360 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.634978+0000", 2024-05-08T08:52:37.360 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4700000000000008e-06 2024-05-08T08:52:37.360 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.360 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.360 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.360 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.634984+0000", 2024-05-08T08:52:37.361 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8819999999999998e-06 2024-05-08T08:52:37.361 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.361 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.361 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.361 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.361 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.361 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2015 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.361 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.595196+0000", 2024-05-08T08:52:37.361 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.040032267000000003, 2024-05-08T08:52:37.361 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.048303116, 2024-05-08T08:52:37.361 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.361 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.361 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.361 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.361 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.362 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.362 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2015 2024-05-08T08:52:37.362 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.362 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.362 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.362 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.362 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.595196+0000", 2024-05-08T08:52:37.362 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.362 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.362 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.362 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.362 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.595196+0000", 2024-05-08T08:52:37.362 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.362 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.362 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.363 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.363 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.595196+0000", 2024-05-08T08:52:37.363 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1600000000000001e-07 2024-05-08T08:52:37.363 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.363 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.363 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.363 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.595199+0000", 2024-05-08T08:52:37.363 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8820000000000001e-06 2024-05-08T08:52:37.363 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.363 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.363 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.363 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.595207+0000", 2024-05-08T08:52:37.363 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8609999999999998e-06 2024-05-08T08:52:37.363 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.363 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.364 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.364 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.595213+0000", 2024-05-08T08:52:37.364 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7860000000000002e-06 2024-05-08T08:52:37.364 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.364 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.364 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.364 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.364 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.364 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2079 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.364 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.602027+0000", 2024-05-08T08:52:37.365 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.033201289000000002, 2024-05-08T08:52:37.365 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.041504714999999998, 2024-05-08T08:52:37.365 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.365 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.365 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.365 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.365 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.365 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.365 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2079 2024-05-08T08:52:37.365 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.365 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.365 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.365 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.365 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.602027+0000", 2024-05-08T08:52:37.365 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.365 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.366 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.366 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.366 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.602027+0000", 2024-05-08T08:52:37.366 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.366 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.366 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.366 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.366 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.602464+0000", 2024-05-08T08:52:37.366 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00043758600000000001 2024-05-08T08:52:37.366 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.366 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.366 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.366 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.602472+0000", 2024-05-08T08:52:37.366 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.6040000000000004e-06 2024-05-08T08:52:37.366 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.367 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.367 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.367 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.602481+0000", 2024-05-08T08:52:37.367 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.9500000000000007e-06 2024-05-08T08:52:37.367 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.367 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.367 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.367 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.602485+0000", 2024-05-08T08:52:37.367 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4070000000000003e-06 2024-05-08T08:52:37.367 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.367 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.367 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.367 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.367 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.367 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2144 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.368 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.609140+0000", 2024-05-08T08:52:37.368 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.026087419000000001, 2024-05-08T08:52:37.368 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.03444117, 2024-05-08T08:52:37.368 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.368 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.368 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.368 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.368 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.368 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.368 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2144 2024-05-08T08:52:37.368 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.368 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.368 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.368 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.368 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.609140+0000", 2024-05-08T08:52:37.369 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.369 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.369 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.369 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.369 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.609140+0000", 2024-05-08T08:52:37.369 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.369 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.369 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.369 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.369 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.609500+0000", 2024-05-08T08:52:37.369 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00035947300000000002 2024-05-08T08:52:37.369 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.369 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.369 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.370 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.609504+0000", 2024-05-08T08:52:37.370 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2309999999999998e-06 2024-05-08T08:52:37.370 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.370 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.370 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.370 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.609519+0000", 2024-05-08T08:52:37.370 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.52e-05 2024-05-08T08:52:37.370 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.370 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.370 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.370 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.609526+0000", 2024-05-08T08:52:37.370 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7320000000000001e-06 2024-05-08T08:52:37.370 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.371 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.371 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.371 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.371 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.371 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2208 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.371 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.615775+0000", 2024-05-08T08:52:37.371 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.019452712, 2024-05-08T08:52:37.371 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.027868955000000001, 2024-05-08T08:52:37.371 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.371 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.371 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.371 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.371 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.371 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.372 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2208 2024-05-08T08:52:37.372 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.372 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.372 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.372 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.372 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.615775+0000", 2024-05-08T08:52:37.372 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.372 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.372 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.372 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.372 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.615775+0000", 2024-05-08T08:52:37.372 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.372 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.372 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.372 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.373 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.615776+0000", 2024-05-08T08:52:37.373 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2699999999999998e-07 2024-05-08T08:52:37.373 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.373 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.373 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.373 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.615779+0000", 2024-05-08T08:52:37.373 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7819999999999998e-06 2024-05-08T08:52:37.373 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.373 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.373 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.373 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.615786+0000", 2024-05-08T08:52:37.373 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4850000000000003e-06 2024-05-08T08:52:37.373 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.373 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.373 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.374 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.615793+0000", 2024-05-08T08:52:37.374 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3260000000000001e-06 2024-05-08T08:52:37.374 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.374 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.374 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.374 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.374 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.374 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2272 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.374 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.622150+0000", 2024-05-08T08:52:37.374 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.013077738, 2024-05-08T08:52:37.374 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.021550462999999999, 2024-05-08T08:52:37.375 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.375 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.375 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.375 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.375 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.375 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.375 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2272 2024-05-08T08:52:37.375 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.375 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.375 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.375 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.375 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.622150+0000", 2024-05-08T08:52:37.375 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.376 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.376 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.376 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.376 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.622150+0000", 2024-05-08T08:52:37.376 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.376 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.376 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.376 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.376 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.622151+0000", 2024-05-08T08:52:37.376 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5799999999999999e-07 2024-05-08T08:52:37.376 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.376 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.376 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.376 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.622153+0000", 2024-05-08T08:52:37.376 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6429999999999999e-06 2024-05-08T08:52:37.377 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.377 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.377 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.377 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.622160+0000", 2024-05-08T08:52:37.377 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1350000000000003e-06 2024-05-08T08:52:37.377 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.377 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.377 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.377 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.622166+0000", 2024-05-08T08:52:37.377 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6930000000000004e-06 2024-05-08T08:52:37.377 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.377 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.377 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.377 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.377 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.378 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2337 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.378 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.628880+0000", 2024-05-08T08:52:37.378 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0063481559999999998, 2024-05-08T08:52:37.378 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.014880757999999999, 2024-05-08T08:52:37.378 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.378 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.378 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.378 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.378 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.378 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.378 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2337 2024-05-08T08:52:37.378 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.378 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.378 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.378 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.379 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.628880+0000", 2024-05-08T08:52:37.379 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.379 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.379 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.379 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.379 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.628880+0000", 2024-05-08T08:52:37.379 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.379 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.379 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.379 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.379 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.628880+0000", 2024-05-08T08:52:37.379 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8899999999999999e-07 2024-05-08T08:52:37.379 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.379 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.380 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.380 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.628883+0000", 2024-05-08T08:52:37.380 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.52e-06 2024-05-08T08:52:37.380 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.380 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.380 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.380 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.628890+0000", 2024-05-08T08:52:37.380 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2200000000000003e-06 2024-05-08T08:52:37.380 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.380 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.380 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.380 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.628896+0000", 2024-05-08T08:52:37.381 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7579999999999998e-06 2024-05-08T08:52:37.381 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.381 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.381 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.381 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.381 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.381 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2402 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.381 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.634995+0000", 2024-05-08T08:52:37.381 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.00023296699999999999, 2024-05-08T08:52:37.381 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0088013919999999999, 2024-05-08T08:52:37.381 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.381 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.381 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.382 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.382 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.382 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.382 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2402 2024-05-08T08:52:37.382 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.382 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.382 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.382 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.382 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.634995+0000", 2024-05-08T08:52:37.382 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.382 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.382 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.383 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.383 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.634995+0000", 2024-05-08T08:52:37.383 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.383 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.383 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.383 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.383 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.634996+0000", 2024-05-08T08:52:37.383 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4799999999999997e-07 2024-05-08T08:52:37.383 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.383 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.383 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.383 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.634999+0000", 2024-05-08T08:52:37.384 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.692e-06 2024-05-08T08:52:37.384 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.384 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.384 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.384 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.635008+0000", 2024-05-08T08:52:37.384 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.6230000000000003e-06 2024-05-08T08:52:37.384 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.384 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.384 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.384 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.635015+0000", 2024-05-08T08:52:37.384 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6560000000000003e-06 2024-05-08T08:52:37.384 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.384 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.385 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.385 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.385 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.385 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2017 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.385 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.595220+0000", 2024-05-08T08:52:37.385 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.040008292000000001, 2024-05-08T08:52:37.385 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.048609566999999999, 2024-05-08T08:52:37.385 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.385 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.385 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.385 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.386 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.386 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.386 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2017 2024-05-08T08:52:37.386 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.386 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.386 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.386 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.386 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.595220+0000", 2024-05-08T08:52:37.386 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.386 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.386 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.386 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.386 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.595220+0000", 2024-05-08T08:52:37.386 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.386 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.387 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.387 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.387 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.595335+0000", 2024-05-08T08:52:37.387 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000115058 2024-05-08T08:52:37.387 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.387 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.387 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.387 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.595338+0000", 2024-05-08T08:52:37.387 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4769999999999999e-06 2024-05-08T08:52:37.387 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.387 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.387 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.387 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.595347+0000", 2024-05-08T08:52:37.387 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0489999999999993e-06 2024-05-08T08:52:37.387 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.387 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.388 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.388 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.595354+0000", 2024-05-08T08:52:37.388 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4080000000000003e-06 2024-05-08T08:52:37.388 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.388 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.388 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.388 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.388 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.388 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2081 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.388 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.602526+0000", 2024-05-08T08:52:37.388 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.032702383000000002, 2024-05-08T08:52:37.388 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.041331126000000003, 2024-05-08T08:52:37.388 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.388 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.388 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.388 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.389 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.389 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.389 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2081 2024-05-08T08:52:37.389 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.389 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.389 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.389 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.389 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.602526+0000", 2024-05-08T08:52:37.389 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.389 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.389 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.389 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.389 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.602526+0000", 2024-05-08T08:52:37.389 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.389 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.390 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.390 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.390 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.602526+0000", 2024-05-08T08:52:37.390 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3000000000000001e-07 2024-05-08T08:52:37.390 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.390 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.390 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.390 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.602529+0000", 2024-05-08T08:52:37.390 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0520000000000002e-06 2024-05-08T08:52:37.390 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.390 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.390 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.390 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.602535+0000", 2024-05-08T08:52:37.390 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.874e-06 2024-05-08T08:52:37.391 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.391 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.391 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.391 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.602539+0000", 2024-05-08T08:52:37.391 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7359999999999999e-06 2024-05-08T08:52:37.391 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.391 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.391 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.391 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.391 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.391 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2146 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.391 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.609537+0000", 2024-05-08T08:52:37.391 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.025690517, 2024-05-08T08:52:37.391 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.034347637, 2024-05-08T08:52:37.391 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.392 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.392 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.392 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.392 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.392 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.392 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2146 2024-05-08T08:52:37.392 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.392 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.392 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.392 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.392 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.609537+0000", 2024-05-08T08:52:37.392 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.393 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.393 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.393 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.393 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.609537+0000", 2024-05-08T08:52:37.393 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.393 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.393 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.393 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.393 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.609538+0000", 2024-05-08T08:52:37.393 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3600000000000004e-07 2024-05-08T08:52:37.393 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.393 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.393 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.393 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.609541+0000", 2024-05-08T08:52:37.393 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6240000000000002e-06 2024-05-08T08:52:37.393 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.394 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.394 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.394 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.609547+0000", 2024-05-08T08:52:37.394 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6800000000000004e-06 2024-05-08T08:52:37.394 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.394 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.394 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.394 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.609552+0000", 2024-05-08T08:52:37.394 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0919999999999997e-06 2024-05-08T08:52:37.394 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.394 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.394 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.394 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.394 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.394 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2210 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.395 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.615808+0000", 2024-05-08T08:52:37.395 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.019420280000000002, 2024-05-08T08:52:37.395 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.02810441, 2024-05-08T08:52:37.395 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.395 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.395 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.395 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.395 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.395 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.395 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2210 2024-05-08T08:52:37.395 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.395 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.396 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.396 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.396 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.615808+0000", 2024-05-08T08:52:37.396 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.396 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.396 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.396 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.396 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.615808+0000", 2024-05-08T08:52:37.396 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.396 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.396 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.396 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.396 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.616118+0000", 2024-05-08T08:52:37.396 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000310489 2024-05-08T08:52:37.397 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.397 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.397 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.397 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.616121+0000", 2024-05-08T08:52:37.397 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9890000000000001e-06 2024-05-08T08:52:37.397 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.397 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.397 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.397 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.616130+0000", 2024-05-08T08:52:37.397 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0599999999999997e-06 2024-05-08T08:52:37.397 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.397 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.398 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.398 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.616138+0000", 2024-05-08T08:52:37.398 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5800000000000003e-06 2024-05-08T08:52:37.398 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.398 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.398 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.398 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.398 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.398 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2274 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.398 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.622174+0000", 2024-05-08T08:52:37.398 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.013053861, 2024-05-08T08:52:37.398 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.021770774999999999, 2024-05-08T08:52:37.398 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.399 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.399 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.399 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.399 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.399 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.399 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2274 2024-05-08T08:52:37.399 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.399 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.399 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.399 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.399 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.622174+0000", 2024-05-08T08:52:37.399 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.399 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.400 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.400 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.400 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.622174+0000", 2024-05-08T08:52:37.400 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.400 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.400 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.400 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.400 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.622494+0000", 2024-05-08T08:52:37.400 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00032034000000000001 2024-05-08T08:52:37.400 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.400 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.400 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.400 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.622497+0000", 2024-05-08T08:52:37.400 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8270000000000002e-06 2024-05-08T08:52:37.401 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.401 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.401 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.401 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.622506+0000", 2024-05-08T08:52:37.401 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3520000000000007e-06 2024-05-08T08:52:37.401 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.401 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.401 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.401 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.622510+0000", 2024-05-08T08:52:37.401 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3800000000000004e-06 2024-05-08T08:52:37.401 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.401 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.401 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.401 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.402 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.402 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2339 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.402 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.628903+0000", 2024-05-08T08:52:37.402 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0063246480000000004, 2024-05-08T08:52:37.402 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.015098469, 2024-05-08T08:52:37.402 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.402 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.402 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.402 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.402 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.402 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.402 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2339 2024-05-08T08:52:37.402 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.402 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.403 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.403 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.403 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.628903+0000", 2024-05-08T08:52:37.403 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.403 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.403 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.403 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.403 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.628903+0000", 2024-05-08T08:52:37.403 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.403 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.403 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.403 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.403 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.629241+0000", 2024-05-08T08:52:37.403 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00033767500000000001 2024-05-08T08:52:37.404 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.404 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.404 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.404 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.629244+0000", 2024-05-08T08:52:37.404 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.943e-06 2024-05-08T08:52:37.404 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.404 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.404 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.404 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.629252+0000", 2024-05-08T08:52:37.404 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0430000000000008e-06 2024-05-08T08:52:37.404 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.404 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.404 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.404 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.629256+0000", 2024-05-08T08:52:37.404 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5680000000000001e-06 2024-05-08T08:52:37.405 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.405 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.405 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.405 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.405 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.405 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2405 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.405 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.635024+0000", 2024-05-08T08:52:37.405 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.00020403700000000001, 2024-05-08T08:52:37.405 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.009036779, 2024-05-08T08:52:37.405 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.405 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.405 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.406 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.406 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.406 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.406 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2405 2024-05-08T08:52:37.406 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.406 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.406 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.406 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.406 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.635024+0000", 2024-05-08T08:52:37.406 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.406 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.406 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.407 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.407 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.635024+0000", 2024-05-08T08:52:37.407 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.407 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.407 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.407 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.407 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.635287+0000", 2024-05-08T08:52:37.407 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00026325900000000001 2024-05-08T08:52:37.407 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.407 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.407 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.407 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.635292+0000", 2024-05-08T08:52:37.407 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9259999999999999e-06 2024-05-08T08:52:37.407 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.408 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.408 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.408 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.635302+0000", 2024-05-08T08:52:37.408 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.9839999999999996e-06 2024-05-08T08:52:37.408 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.408 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.408 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.408 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.635309+0000", 2024-05-08T08:52:37.408 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1740000000000003e-06 2024-05-08T08:52:37.408 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.408 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.408 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.408 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.409 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.409 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2019 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.409 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.595364+0000", 2024-05-08T08:52:37.409 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.039864233999999998, 2024-05-08T08:52:37.409 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.048755844, 2024-05-08T08:52:37.409 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.409 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.409 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.409 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.409 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.409 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.409 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2019 2024-05-08T08:52:37.409 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.410 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.410 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.410 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.410 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.595364+0000", 2024-05-08T08:52:37.410 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.410 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.410 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.410 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.410 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.595364+0000", 2024-05-08T08:52:37.410 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.410 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.410 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.411 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.411 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.595784+0000", 2024-05-08T08:52:37.411 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000420247 2024-05-08T08:52:37.411 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.411 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.411 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.411 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.595788+0000", 2024-05-08T08:52:37.411 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6339999999999999e-06 2024-05-08T08:52:37.411 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.411 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.411 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.411 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.595794+0000", 2024-05-08T08:52:37.411 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.878e-06 2024-05-08T08:52:37.412 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.412 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.412 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.412 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.595799+0000", 2024-05-08T08:52:37.412 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6319999999999999e-06 2024-05-08T08:52:37.412 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.412 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.412 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.412 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.412 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.412 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2083 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.412 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.602545+0000", 2024-05-08T08:52:37.412 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.032682735999999997, 2024-05-08T08:52:37.413 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.041630233000000003, 2024-05-08T08:52:37.413 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.413 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.413 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.413 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.413 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.413 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.413 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2083 2024-05-08T08:52:37.413 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.413 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.413 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.414 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.414 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.602545+0000", 2024-05-08T08:52:37.414 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.414 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.414 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.414 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.414 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.602545+0000", 2024-05-08T08:52:37.414 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.414 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.414 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.414 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.414 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.602828+0000", 2024-05-08T08:52:37.414 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000283217 2024-05-08T08:52:37.415 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.415 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.415 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.415 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.602832+0000", 2024-05-08T08:52:37.415 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.844e-06 2024-05-08T08:52:37.415 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.415 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.415 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.415 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.602843+0000", 2024-05-08T08:52:37.415 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.03e-05 2024-05-08T08:52:37.415 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.415 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.415 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.415 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.602849+0000", 2024-05-08T08:52:37.416 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8399999999999997e-06 2024-05-08T08:52:37.416 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.416 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.416 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.416 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.416 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.416 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2148 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.416 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.609560+0000", 2024-05-08T08:52:37.416 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.025668218, 2024-05-08T08:52:37.416 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.034657625999999997, 2024-05-08T08:52:37.416 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.416 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.417 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.417 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.417 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.417 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.417 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2148 2024-05-08T08:52:37.417 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.417 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.417 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.417 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.417 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.609560+0000", 2024-05-08T08:52:37.417 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.417 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.417 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.417 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.417 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.609560+0000", 2024-05-08T08:52:37.417 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.418 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.418 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.418 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.418 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.609750+0000", 2024-05-08T08:52:37.418 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00019013999999999999 2024-05-08T08:52:37.418 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.418 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.418 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.418 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.609754+0000", 2024-05-08T08:52:37.418 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2749999999999997e-06 2024-05-08T08:52:37.418 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.418 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.418 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.418 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.609765+0000", 2024-05-08T08:52:37.419 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0767e-05 2024-05-08T08:52:37.419 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.419 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.419 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.419 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.609769+0000", 2024-05-08T08:52:37.419 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1180000000000002e-06 2024-05-08T08:52:37.419 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.419 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.419 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.419 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.419 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.419 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2212 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.419 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.616147+0000", 2024-05-08T08:52:37.419 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.019080588999999998, 2024-05-08T08:52:37.419 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.028102107000000001, 2024-05-08T08:52:37.419 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.420 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.420 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.420 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.420 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.420 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.420 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2212 2024-05-08T08:52:37.420 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.420 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.420 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.420 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.420 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.616147+0000", 2024-05-08T08:52:37.420 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.420 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.420 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.420 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.421 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.616147+0000", 2024-05-08T08:52:37.421 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.421 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.421 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.421 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.421 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.616378+0000", 2024-05-08T08:52:37.421 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00023115500000000001 2024-05-08T08:52:37.421 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.421 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.421 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.421 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.616382+0000", 2024-05-08T08:52:37.421 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.014e-06 2024-05-08T08:52:37.421 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.421 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.421 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.422 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.616393+0000", 2024-05-08T08:52:37.422 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0856000000000001e-05 2024-05-08T08:52:37.422 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.422 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.422 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.422 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.616401+0000", 2024-05-08T08:52:37.422 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9589999999999995e-06 2024-05-08T08:52:37.422 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.422 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.422 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.422 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.422 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.422 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2276 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.422 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.622518+0000", 2024-05-08T08:52:37.422 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.012710063000000001, 2024-05-08T08:52:37.423 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.021762581, 2024-05-08T08:52:37.423 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.423 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.423 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.423 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.423 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.423 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.423 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2276 2024-05-08T08:52:37.423 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.423 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.423 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.423 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.423 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.622518+0000", 2024-05-08T08:52:37.423 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.424 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.424 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.424 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.424 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.622518+0000", 2024-05-08T08:52:37.424 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.424 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.424 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.424 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.424 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.622518+0000", 2024-05-08T08:52:37.424 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.2300000000000002e-07 2024-05-08T08:52:37.424 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.424 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.424 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.424 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.622520+0000", 2024-05-08T08:52:37.425 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.44e-06 2024-05-08T08:52:37.425 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.425 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.425 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.425 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.622523+0000", 2024-05-08T08:52:37.425 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5990000000000002e-06 2024-05-08T08:52:37.425 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.425 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.425 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.425 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.622527+0000", 2024-05-08T08:52:37.425 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.608e-06 2024-05-08T08:52:37.425 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.425 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.425 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.425 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.426 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.426 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2341 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.426 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.629265+0000", 2024-05-08T08:52:37.426 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0059633409999999996, 2024-05-08T08:52:37.426 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.015042728999999999, 2024-05-08T08:52:37.426 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.426 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.426 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.426 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.426 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.427 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.427 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2341 2024-05-08T08:52:37.427 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.427 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.427 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.427 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.427 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.629265+0000", 2024-05-08T08:52:37.427 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.427 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.427 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.427 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.427 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.629265+0000", 2024-05-08T08:52:37.427 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.427 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.427 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.428 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.428 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.629265+0000", 2024-05-08T08:52:37.428 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3099999999999999e-07 2024-05-08T08:52:37.428 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.428 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.428 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.428 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.629266+0000", 2024-05-08T08:52:37.428 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.491e-06 2024-05-08T08:52:37.428 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.428 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.428 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.428 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.629270+0000", 2024-05-08T08:52:37.428 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.517e-06 2024-05-08T08:52:37.428 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.428 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.429 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.429 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.629273+0000", 2024-05-08T08:52:37.429 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8200000000000001e-06 2024-05-08T08:52:37.429 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.429 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.429 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.429 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.429 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.429 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2407 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.429 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.635322+0000", 2024-05-08T08:52:37.429 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 4294967295.9999061, 2024-05-08T08:52:37.429 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0090153679999999993, 2024-05-08T08:52:37.429 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.429 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.429 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.430 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.430 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.430 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.430 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2407 2024-05-08T08:52:37.430 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.430 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.430 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.430 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.430 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.635322+0000", 2024-05-08T08:52:37.430 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.430 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.430 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.430 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.431 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.635322+0000", 2024-05-08T08:52:37.431 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.431 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.431 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.431 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.431 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.635417+0000", 2024-05-08T08:52:37.431 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4654000000000005e-05 2024-05-08T08:52:37.431 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.431 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.431 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.431 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.635420+0000", 2024-05-08T08:52:37.431 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.467e-06 2024-05-08T08:52:37.431 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.431 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.432 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.432 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.635426+0000", 2024-05-08T08:52:37.432 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4389999999999997e-06 2024-05-08T08:52:37.432 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.432 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.432 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.432 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.635431+0000", 2024-05-08T08:52:37.432 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2780000000000004e-06 2024-05-08T08:52:37.432 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.432 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.432 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.432 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.432 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.432 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2021 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.433 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.595808+0000", 2024-05-08T08:52:37.433 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.039419838999999998, 2024-05-08T08:52:37.433 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.048557925000000002, 2024-05-08T08:52:37.433 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.433 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.433 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.433 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.433 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.433 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.433 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2021 2024-05-08T08:52:37.433 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.433 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.433 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.433 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.433 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.595808+0000", 2024-05-08T08:52:37.434 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.434 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.434 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.434 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.434 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.595808+0000", 2024-05-08T08:52:37.434 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.434 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.434 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.434 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.434 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.595808+0000", 2024-05-08T08:52:37.434 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1399999999999998e-07 2024-05-08T08:52:37.434 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.434 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.435 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.435 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.595810+0000", 2024-05-08T08:52:37.435 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5910000000000001e-06 2024-05-08T08:52:37.435 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.435 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.435 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.435 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.595814+0000", 2024-05-08T08:52:37.435 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8809999999999998e-06 2024-05-08T08:52:37.435 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.435 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.435 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.435 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.595817+0000", 2024-05-08T08:52:37.435 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9979999999999999e-06 2024-05-08T08:52:37.435 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.435 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.436 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.436 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.436 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.436 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2085 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.436 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.602860+0000", 2024-05-08T08:52:37.436 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.032368284999999997, 2024-05-08T08:52:37.436 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.041536539999999997, 2024-05-08T08:52:37.436 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.436 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.436 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.436 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.436 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.437 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.437 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2085 2024-05-08T08:52:37.437 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.437 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.437 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.437 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.437 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.602860+0000", 2024-05-08T08:52:37.437 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.437 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.438 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.438 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.438 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.602860+0000", 2024-05-08T08:52:37.438 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.438 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.438 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.438 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.438 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.603219+0000", 2024-05-08T08:52:37.438 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00035944599999999997 2024-05-08T08:52:37.438 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.438 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.438 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.439 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.603225+0000", 2024-05-08T08:52:37.439 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8880000000000002e-06 2024-05-08T08:52:37.439 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.439 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.439 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.439 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.603238+0000", 2024-05-08T08:52:37.439 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3497e-05 2024-05-08T08:52:37.439 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.439 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.439 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.439 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.603247+0000", 2024-05-08T08:52:37.440 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7339999999999997e-06 2024-05-08T08:52:37.440 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.440 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.440 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.440 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.440 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.440 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2150 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.440 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.609777+0000", 2024-05-08T08:52:37.440 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.025450823000000001, 2024-05-08T08:52:37.440 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.034659011000000003, 2024-05-08T08:52:37.440 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.440 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.440 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.441 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.441 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.441 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.441 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2150 2024-05-08T08:52:37.441 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.441 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.441 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.441 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.441 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.609777+0000", 2024-05-08T08:52:37.441 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.441 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.442 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.442 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.442 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.609777+0000", 2024-05-08T08:52:37.442 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.442 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.442 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.442 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.442 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.609777+0000", 2024-05-08T08:52:37.442 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5900000000000003e-07 2024-05-08T08:52:37.442 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.442 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.442 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.442 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.609779+0000", 2024-05-08T08:52:37.442 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.545e-06 2024-05-08T08:52:37.443 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.443 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.443 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.443 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.609783+0000", 2024-05-08T08:52:37.443 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8190000000000002e-06 2024-05-08T08:52:37.443 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.443 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.443 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.443 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.609786+0000", 2024-05-08T08:52:37.443 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8619999999999999e-06 2024-05-08T08:52:37.443 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.443 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.443 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.443 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.443 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.444 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2214 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.444 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.616412+0000", 2024-05-08T08:52:37.444 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.018815835, 2024-05-08T08:52:37.444 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.028082336999999999, 2024-05-08T08:52:37.444 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.444 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.444 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.444 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.444 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.444 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.444 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2214 2024-05-08T08:52:37.444 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.444 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.444 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.444 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.445 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.616412+0000", 2024-05-08T08:52:37.445 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.445 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.445 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.445 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.445 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.616412+0000", 2024-05-08T08:52:37.445 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.445 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.445 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.445 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.445 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.616413+0000", 2024-05-08T08:52:37.445 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1900000000000003e-07 2024-05-08T08:52:37.445 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.445 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.445 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.445 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.616415+0000", 2024-05-08T08:52:37.446 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6299999999999998e-06 2024-05-08T08:52:37.446 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.446 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.446 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.446 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.616422+0000", 2024-05-08T08:52:37.446 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5710000000000003e-06 2024-05-08T08:52:37.446 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.446 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.446 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.446 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.616427+0000", 2024-05-08T08:52:37.446 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3120000000000001e-06 2024-05-08T08:52:37.446 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.446 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.446 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.446 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.447 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.447 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2278 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.447 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.622531+0000", 2024-05-08T08:52:37.447 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.012697103, 2024-05-08T08:52:37.447 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.022019764000000001, 2024-05-08T08:52:37.447 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.447 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.447 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.447 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.447 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.447 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.448 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2278 2024-05-08T08:52:37.448 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.448 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.448 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.448 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.448 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.622531+0000", 2024-05-08T08:52:37.448 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.448 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.448 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.448 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.448 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.622531+0000", 2024-05-08T08:52:37.448 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.448 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.448 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.448 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.449 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.622531+0000", 2024-05-08T08:52:37.449 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5199999999999998e-07 2024-05-08T08:52:37.449 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.449 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.449 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.449 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.622532+0000", 2024-05-08T08:52:37.449 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.316e-06 2024-05-08T08:52:37.449 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.449 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.449 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.449 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.622536+0000", 2024-05-08T08:52:37.449 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0920000000000002e-06 2024-05-08T08:52:37.449 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.449 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.449 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.450 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.622538+0000", 2024-05-08T08:52:37.450 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6510000000000001e-06 2024-05-08T08:52:37.450 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.450 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.450 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.450 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.450 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.450 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2343 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.450 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.629277+0000", 2024-05-08T08:52:37.450 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0059512719999999996, 2024-05-08T08:52:37.450 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.015333055999999999, 2024-05-08T08:52:37.450 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.450 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.450 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.450 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.451 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.451 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.451 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2343 2024-05-08T08:52:37.451 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.451 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.451 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.451 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.451 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.629277+0000", 2024-05-08T08:52:37.451 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.451 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.451 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.451 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.451 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.629277+0000", 2024-05-08T08:52:37.451 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.451 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.451 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.452 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.452 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.629602+0000", 2024-05-08T08:52:37.452 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00032568099999999997 2024-05-08T08:52:37.452 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.452 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.452 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.452 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.629610+0000", 2024-05-08T08:52:37.452 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5560000000000002e-06 2024-05-08T08:52:37.452 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.452 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.452 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.452 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.629618+0000", 2024-05-08T08:52:37.452 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0029999999999994e-06 2024-05-08T08:52:37.452 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.452 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.452 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.453 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.629622+0000", 2024-05-08T08:52:37.453 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5499999999999996e-06 2024-05-08T08:52:37.453 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.453 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.453 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.453 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.453 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.453 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2409 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.453 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.635439+0000", 2024-05-08T08:52:37.453 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 4294967295.9997888, 2024-05-08T08:52:37.453 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0092200359999999992, 2024-05-08T08:52:37.453 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.453 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.453 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.453 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.453 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.454 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.454 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2409 2024-05-08T08:52:37.454 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.454 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.454 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.454 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.454 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.635439+0000", 2024-05-08T08:52:37.454 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.454 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.454 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.454 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.454 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.635439+0000", 2024-05-08T08:52:37.454 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.454 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.455 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.455 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.455 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.635868+0000", 2024-05-08T08:52:37.455 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00042878000000000002 2024-05-08T08:52:37.455 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.455 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.455 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.455 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.635872+0000", 2024-05-08T08:52:37.455 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5199999999999999e-06 2024-05-08T08:52:37.455 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.455 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.455 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.455 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.635882+0000", 2024-05-08T08:52:37.455 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2569999999999996e-06 2024-05-08T08:52:37.455 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.455 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.456 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.456 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.635888+0000", 2024-05-08T08:52:37.456 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7769999999999997e-06 2024-05-08T08:52:37.456 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.456 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.456 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.456 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.456 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.456 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2023 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.456 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.595821+0000", 2024-05-08T08:52:37.456 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.039406903, 2024-05-08T08:52:37.457 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.048870495999999999, 2024-05-08T08:52:37.457 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.457 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.457 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.457 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.457 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.457 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.457 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2023 2024-05-08T08:52:37.457 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.457 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.457 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.458 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.458 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.595821+0000", 2024-05-08T08:52:37.458 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.458 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.458 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.458 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.458 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.595821+0000", 2024-05-08T08:52:37.458 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.458 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.458 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.458 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.458 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.596165+0000", 2024-05-08T08:52:37.459 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00034443599999999999 2024-05-08T08:52:37.459 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.459 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.459 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.459 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.596170+0000", 2024-05-08T08:52:37.459 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5940000000000004e-06 2024-05-08T08:52:37.459 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.459 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.459 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.459 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.596182+0000", 2024-05-08T08:52:37.459 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1765e-05 2024-05-08T08:52:37.459 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.459 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.459 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.460 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.596194+0000", 2024-05-08T08:52:37.460 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1990999999999999e-05 2024-05-08T08:52:37.460 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.460 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.460 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.460 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.460 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.460 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2087 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.460 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.603266+0000", 2024-05-08T08:52:37.460 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.031962270000000001, 2024-05-08T08:52:37.460 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.041459650000000001, 2024-05-08T08:52:37.460 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.460 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.460 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.461 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.461 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.461 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.461 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2087 2024-05-08T08:52:37.461 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.461 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.461 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.461 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.461 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.603266+0000", 2024-05-08T08:52:37.461 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.461 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.461 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.461 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.461 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.603266+0000", 2024-05-08T08:52:37.461 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.462 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.462 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.462 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.462 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.603478+0000", 2024-05-08T08:52:37.462 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00021212800000000001 2024-05-08T08:52:37.462 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.462 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.462 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.462 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.603483+0000", 2024-05-08T08:52:37.462 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.48e-06 2024-05-08T08:52:37.462 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.462 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.462 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.462 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.603498+0000", 2024-05-08T08:52:37.463 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.4756999999999999e-05 2024-05-08T08:52:37.463 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.463 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.463 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.463 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.603507+0000", 2024-05-08T08:52:37.463 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7099999999999996e-06 2024-05-08T08:52:37.463 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.463 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.463 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.463 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.463 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.463 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2152 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.463 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.609794+0000", 2024-05-08T08:52:37.463 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.025434025999999998, 2024-05-08T08:52:37.464 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.034987296000000001, 2024-05-08T08:52:37.464 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.464 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.464 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.464 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.464 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.464 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.464 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2152 2024-05-08T08:52:37.464 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.464 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.464 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.464 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.465 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.609794+0000", 2024-05-08T08:52:37.465 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.465 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.465 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.465 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.465 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.609794+0000", 2024-05-08T08:52:37.465 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.465 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.465 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.465 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.465 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.610136+0000", 2024-05-08T08:52:37.465 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00034218300000000003 2024-05-08T08:52:37.466 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.466 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.466 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.466 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.610140+0000", 2024-05-08T08:52:37.466 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4390000000000002e-06 2024-05-08T08:52:37.466 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.467 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.467 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.467 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.610151+0000", 2024-05-08T08:52:37.467 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0145e-05 2024-05-08T08:52:37.467 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.467 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.468 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.468 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.610157+0000", 2024-05-08T08:52:37.468 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7499999999999997e-06 2024-05-08T08:52:37.468 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.468 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.468 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.468 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.468 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.469 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2216 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.469 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.616434+0000", 2024-05-08T08:52:37.469 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.018793863000000001, 2024-05-08T08:52:37.469 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.028407932, 2024-05-08T08:52:37.469 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.469 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.469 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.469 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.469 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.469 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.469 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2216 2024-05-08T08:52:37.469 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.469 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.470 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.470 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.470 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.616434+0000", 2024-05-08T08:52:37.470 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.470 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.470 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.470 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.470 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.616434+0000", 2024-05-08T08:52:37.470 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.470 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.470 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.470 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.471 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.616775+0000", 2024-05-08T08:52:37.471 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000341186 2024-05-08T08:52:37.471 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.471 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.471 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.471 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.616779+0000", 2024-05-08T08:52:37.471 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.2930000000000001e-06 2024-05-08T08:52:37.471 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.471 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.471 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.471 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.616786+0000", 2024-05-08T08:52:37.471 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0589999999999997e-06 2024-05-08T08:52:37.471 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.472 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.472 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.472 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.616790+0000", 2024-05-08T08:52:37.472 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0690000000000003e-06 2024-05-08T08:52:37.472 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.472 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.472 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.472 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.472 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.472 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2280 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.472 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.622543+0000", 2024-05-08T08:52:37.472 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.012684864000000001, 2024-05-08T08:52:37.472 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.022355595999999998, 2024-05-08T08:52:37.472 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.473 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.473 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.473 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.473 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.473 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.473 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2280 2024-05-08T08:52:37.473 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.473 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.473 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.473 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.473 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.622543+0000", 2024-05-08T08:52:37.473 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.473 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.473 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.473 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.474 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.622543+0000", 2024-05-08T08:52:37.474 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.474 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.474 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.474 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.474 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.622753+0000", 2024-05-08T08:52:37.474 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00020985200000000001 2024-05-08T08:52:37.474 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.474 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.474 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.474 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.622757+0000", 2024-05-08T08:52:37.474 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7239999999999998e-06 2024-05-08T08:52:37.474 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.475 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.475 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.475 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.622764+0000", 2024-05-08T08:52:37.475 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7889999999999995e-06 2024-05-08T08:52:37.475 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.475 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.475 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.475 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.622771+0000", 2024-05-08T08:52:37.475 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1639999999999998e-06 2024-05-08T08:52:37.475 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.475 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.475 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.475 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.475 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.476 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2345 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.476 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.629630+0000", 2024-05-08T08:52:37.476 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0055974379999999997, 2024-05-08T08:52:37.476 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.015339073999999999, 2024-05-08T08:52:37.476 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.476 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.476 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.476 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.476 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.476 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.476 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2345 2024-05-08T08:52:37.476 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.476 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.476 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.477 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.477 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.629630+0000", 2024-05-08T08:52:37.477 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.477 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.477 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.477 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.477 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.629630+0000", 2024-05-08T08:52:37.477 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.477 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.477 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.477 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.477 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.629631+0000", 2024-05-08T08:52:37.477 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6100000000000002e-07 2024-05-08T08:52:37.477 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.477 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.478 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.478 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.629633+0000", 2024-05-08T08:52:37.478 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.722e-06 2024-05-08T08:52:37.478 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.478 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.478 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.478 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.629637+0000", 2024-05-08T08:52:37.478 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4510000000000002e-06 2024-05-08T08:52:37.478 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.478 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.478 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.478 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.629641+0000", 2024-05-08T08:52:37.479 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7869999999999999e-06 2024-05-08T08:52:37.479 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.479 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.479 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.479 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.479 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.479 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2411 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.479 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.635900+0000", 2024-05-08T08:52:37.479 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 4294967295.9993277, 2024-05-08T08:52:37.479 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0091021379999999992, 2024-05-08T08:52:37.479 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.479 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.479 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.479 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.479 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.480 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.480 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2411 2024-05-08T08:52:37.480 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.480 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.480 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.480 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.480 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.635900+0000", 2024-05-08T08:52:37.480 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.480 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.480 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.480 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.480 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.635900+0000", 2024-05-08T08:52:37.481 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.481 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.481 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.481 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.481 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.635901+0000", 2024-05-08T08:52:37.481 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1600000000000001e-07 2024-05-08T08:52:37.481 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.481 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.481 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.481 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.635904+0000", 2024-05-08T08:52:37.481 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9699999999999999e-06 2024-05-08T08:52:37.481 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.481 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.481 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.481 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.635912+0000", 2024-05-08T08:52:37.481 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8490000000000006e-06 2024-05-08T08:52:37.482 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.482 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.482 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.482 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.635917+0000", 2024-05-08T08:52:37.482 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5770000000000001e-06 2024-05-08T08:52:37.482 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.482 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.482 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.482 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.482 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.482 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2025 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.482 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.596205+0000", 2024-05-08T08:52:37.482 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.039022988000000002, 2024-05-08T08:52:37.483 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.048831513, 2024-05-08T08:52:37.483 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.483 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.483 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.483 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.483 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.483 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.483 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2025 2024-05-08T08:52:37.483 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.483 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.483 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.483 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.483 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.596205+0000", 2024-05-08T08:52:37.483 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.484 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.484 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.484 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.484 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.596205+0000", 2024-05-08T08:52:37.484 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.484 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.484 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.484 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.484 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.596501+0000", 2024-05-08T08:52:37.484 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00029582700000000003 2024-05-08T08:52:37.484 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.484 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.484 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.484 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.596504+0000", 2024-05-08T08:52:37.484 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0079999999999998e-06 2024-05-08T08:52:37.485 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.485 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.485 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.485 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.596512+0000", 2024-05-08T08:52:37.485 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3210000000000005e-06 2024-05-08T08:52:37.485 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.485 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.485 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.485 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.596521+0000", 2024-05-08T08:52:37.485 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.541e-06 2024-05-08T08:52:37.485 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.485 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.485 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.485 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.486 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.486 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2089 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.486 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.603520+0000", 2024-05-08T08:52:37.486 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.031708262000000001, 2024-05-08T08:52:37.486 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.041544750999999998, 2024-05-08T08:52:37.486 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.486 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.486 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.486 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.486 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.486 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.486 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2089 2024-05-08T08:52:37.486 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.486 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.487 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.487 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.487 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.603520+0000", 2024-05-08T08:52:37.487 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.487 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.487 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.487 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.487 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.603520+0000", 2024-05-08T08:52:37.487 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.487 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.487 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.487 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.487 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.603733+0000", 2024-05-08T08:52:37.487 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000213186 2024-05-08T08:52:37.488 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.488 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.488 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.488 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.603739+0000", 2024-05-08T08:52:37.488 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.84e-06 2024-05-08T08:52:37.488 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.488 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.488 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.488 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.603755+0000", 2024-05-08T08:52:37.488 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6249999999999999e-05 2024-05-08T08:52:37.488 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.488 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.489 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.489 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.603763+0000", 2024-05-08T08:52:37.489 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4249999999999998e-06 2024-05-08T08:52:37.489 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.489 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.489 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.489 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.489 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.489 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2154 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.489 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.610172+0000", 2024-05-08T08:52:37.489 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.025055918, 2024-05-08T08:52:37.489 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.034930274999999997, 2024-05-08T08:52:37.489 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.489 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.489 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.490 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.490 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.490 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.490 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2154 2024-05-08T08:52:37.490 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.490 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.490 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.490 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.490 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.610172+0000", 2024-05-08T08:52:37.491 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.491 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.491 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.491 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.491 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.610172+0000", 2024-05-08T08:52:37.491 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.491 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.491 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.491 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.491 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.610398+0000", 2024-05-08T08:52:37.491 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00022618700000000001 2024-05-08T08:52:37.491 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.491 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.492 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.492 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.610402+0000", 2024-05-08T08:52:37.492 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.687e-06 2024-05-08T08:52:37.492 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.492 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.492 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.492 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.610410+0000", 2024-05-08T08:52:37.493 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9610000000000005e-06 2024-05-08T08:52:37.493 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.493 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.493 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.493 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.610416+0000", 2024-05-08T08:52:37.493 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.037e-06 2024-05-08T08:52:37.494 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.494 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.494 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.494 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.494 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.494 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2219 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.494 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.616797+0000", 2024-05-08T08:52:37.495 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.018430439, 2024-05-08T08:52:37.495 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.028360478000000001, 2024-05-08T08:52:37.495 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.495 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.495 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.495 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.495 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.495 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.495 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2219 2024-05-08T08:52:37.495 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.495 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.495 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.495 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.496 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.616797+0000", 2024-05-08T08:52:37.496 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.496 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.496 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.496 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.496 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.616797+0000", 2024-05-08T08:52:37.496 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.496 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.496 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.496 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.496 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.616958+0000", 2024-05-08T08:52:37.496 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000160872 2024-05-08T08:52:37.496 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.497 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.497 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.497 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.616962+0000", 2024-05-08T08:52:37.497 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0709999999999996e-06 2024-05-08T08:52:37.497 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.497 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.497 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.497 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.616971+0000", 2024-05-08T08:52:37.497 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1629999999999996e-06 2024-05-08T08:52:37.497 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.497 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.497 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.498 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.616976+0000", 2024-05-08T08:52:37.498 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9200000000000001e-06 2024-05-08T08:52:37.498 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.498 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.498 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.498 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.498 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.498 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2282 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.498 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.622781+0000", 2024-05-08T08:52:37.498 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.012447189000000001, 2024-05-08T08:52:37.498 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.022437238000000002, 2024-05-08T08:52:37.498 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.499 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.499 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.499 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.499 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.499 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.499 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2282 2024-05-08T08:52:37.499 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.499 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.499 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.499 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.499 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.622781+0000", 2024-05-08T08:52:37.500 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.500 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.500 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.500 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.500 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.622781+0000", 2024-05-08T08:52:37.500 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.500 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.500 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.500 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.500 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.622965+0000", 2024-05-08T08:52:37.500 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000183788 2024-05-08T08:52:37.500 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.501 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.501 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.501 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.622969+0000", 2024-05-08T08:52:37.501 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5619999999999997e-06 2024-05-08T08:52:37.501 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.501 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.501 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.501 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.622980+0000", 2024-05-08T08:52:37.501 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0742e-05 2024-05-08T08:52:37.501 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.501 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.501 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.501 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.622987+0000", 2024-05-08T08:52:37.501 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3889999999999999e-06 2024-05-08T08:52:37.501 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.502 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.502 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.502 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.502 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.502 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2347 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.502 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.629645+0000", 2024-05-08T08:52:37.502 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0055829920000000002, 2024-05-08T08:52:37.502 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.015628410999999998, 2024-05-08T08:52:37.502 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.502 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.502 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.502 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.502 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.502 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.503 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2347 2024-05-08T08:52:37.503 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.503 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.503 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.503 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.503 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.629645+0000", 2024-05-08T08:52:37.503 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.503 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.503 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.503 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.503 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.629645+0000", 2024-05-08T08:52:37.503 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.503 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.503 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.503 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.504 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.629996+0000", 2024-05-08T08:52:37.504 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00035072499999999997 2024-05-08T08:52:37.504 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.504 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.504 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.504 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.629999+0000", 2024-05-08T08:52:37.504 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4060000000000001e-06 2024-05-08T08:52:37.504 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.504 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.504 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.504 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.630015+0000", 2024-05-08T08:52:37.504 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5880999999999999e-05 2024-05-08T08:52:37.504 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.504 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.505 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.505 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.630020+0000", 2024-05-08T08:52:37.505 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8339999999999998e-06 2024-05-08T08:52:37.505 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.505 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.505 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.505 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.505 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.505 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:2413 3.1f 3.8f53afdf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:37.505 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.635925+0000", 2024-05-08T08:52:37.505 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 4294967295.9993033, 2024-05-08T08:52:37.505 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0094057020000000002, 2024-05-08T08:52:37.505 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:37.505 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:37.505 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:37.506 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:37.506 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:37.506 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:37.506 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2413 2024-05-08T08:52:37.506 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.506 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:37.506 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.506 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:37.506 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.635925+0000", 2024-05-08T08:52:37.506 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.506 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.506 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.506 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:37.506 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.635925+0000", 2024-05-08T08:52:37.507 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:37.507 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.507 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.507 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:37.507 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.636120+0000", 2024-05-08T08:52:37.507 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000195389 2024-05-08T08:52:37.507 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.507 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.507 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:37.507 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.636123+0000", 2024-05-08T08:52:37.507 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0730000000000001e-06 2024-05-08T08:52:37.507 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.507 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.507 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:37.507 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.636129+0000", 2024-05-08T08:52:37.508 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0159999999999997e-06 2024-05-08T08:52:37.508 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:37.508 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:37.508 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:37.508 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.636134+0000", 2024-05-08T08:52:37.508 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2079999999999999e-06 2024-05-08T08:52:37.508 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.508 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:37.508 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.508 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:37.508 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T08:52:37.508 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 231 2024-05-08T08:52:37.508 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:52:37.509 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-05-08T08:52:37.511 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:52:37.512 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:52:37.513+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:52:37.513 INFO:teuthology.orchestra.run.smithi196.stdout:1020 write 0xe33fe00 thru 0xe34f788 (0xf989 bytes) 2024-05-08T08:52:37.513 INFO:teuthology.orchestra.run.smithi196.stdout:1024 punch from 0x5eac00 to 0x5ed600, (0x2a00 bytes) 2024-05-08T08:52:37.513 INFO:teuthology.orchestra.run.smithi196.stdout:1025 read 0xc8c1200 thru 0xc8d086e (0xf66f bytes) 2024-05-08T08:52:37.513 INFO:teuthology.orchestra.run.smithi196.stdout:1027 read 0xcaaf800 thru 0xcab44a1 (0x4ca2 bytes) 2024-05-08T08:52:37.514 INFO:teuthology.orchestra.run.smithi196.stdout:1028 read 0xaa28000 thru 0xaa36c1a (0xec1b bytes) 2024-05-08T08:52:37.514 INFO:teuthology.orchestra.run.smithi196.stdout:1029 read 0xe1ffa00 thru 0xe200504 (0xb05 bytes) 2024-05-08T08:52:37.514 INFO:teuthology.orchestra.run.smithi196.stdout:1031 write 0x24a800 thru 0x259a41 (0xf242 bytes) 2024-05-08T08:52:37.514 INFO:teuthology.orchestra.run.smithi196.stdout:1032 read 0x3328200 thru 0x3336313 (0xe114 bytes) 2024-05-08T08:52:37.514 INFO:teuthology.orchestra.run.smithi196.stdout:1033 trunc from 0xe34f800 to 0xc8f6600 2024-05-08T08:52:37.514 INFO:teuthology.orchestra.run.smithi196.stdout:1034 write 0x51c2e00 thru 0x51c91c1 (0x63c2 bytes) 2024-05-08T08:52:37.514 INFO:teuthology.orchestra.run.smithi196.stdout:1035 write 0x5f4400 thru 0x60348b (0xf08c bytes) 2024-05-08T08:52:37.521 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T08:52:37.572 INFO:teuthology.orchestra.run.smithi196.stdout:1036 write 0xdd3c400 thru 0xdd41fb4 (0x5bb5 bytes) 2024-05-08T08:52:37.593 INFO:teuthology.orchestra.run.smithi196.stdout:1037 write 0xe46c000 thru 0xe475045 (0x9046 bytes) 2024-05-08T08:52:37.610 INFO:teuthology.orchestra.run.smithi196.stdout:1038 trunc from 0xe475200 to 0x13f2400 2024-05-08T08:52:37.611 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:52:37.613+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:52:37.711 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:52:37.714+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:52:37.724 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:52:37.724 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:52:37.724 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:52:37.724 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:52:37.725 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:52:37.735 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:52:37.739 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:52:37.739 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:52:37.773 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:52:37.773 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:52:37.808 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:52:37.808 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:52:37.812 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:52:37.815+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:52:37.836 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:52:37.836 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:52:37.860 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:52:37.861 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:52:37.891 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:52:37.891 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:52:37.899 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:37.899 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:37.911 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:52:37.912 INFO:teuthology.orchestra.run.smithi196.stdout:1039 clone 7 order 24 su 65536 sc 2 2024-05-08T08:52:37.913 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:37.916+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:38.013 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:38.017+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:38.017 INFO:teuthology.orchestra.run.smithi059.stderr:nodeep-scrub is set 2024-05-08T08:52:38.069 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:38.069 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:38.079 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:52:38.102 INFO:teuthology.orchestra.run.smithi196.stdout:truncating image image_client.0-clone6 from 0x13f2400 (overlap 0x13f2400) to 0x11f3a00 2024-05-08T08:52:38.114 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:38.118+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:38.215 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:52:38.218+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:52:38.930 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:38.931 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:38.931 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:52:38.932 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:52:38.933+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:52:38.934 INFO:teuthology.orchestra.run.smithi196.stdout:checking clone #5, image image_client.0-clone5 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent6 2024-05-08T08:52:38.935 INFO:teuthology.orchestra.run.smithi196.stdout:1041 write 0x74b6000 thru 0x74bb20b (0x520c bytes) 2024-05-08T08:52:38.935 INFO:teuthology.orchestra.run.smithi196.stdout:1043 write 0x296000 thru 0x2a3615 (0xd616 bytes) 2024-05-08T08:52:38.935 INFO:teuthology.orchestra.run.smithi196.stdout:1044 punch from 0x2b51400 to 0x2b57400, (0x6000 bytes) 2024-05-08T08:52:38.935 INFO:teuthology.orchestra.run.smithi196.stdout:1046 read 0x35fde00 thru 0x360c605 (0xe806 bytes) 2024-05-08T08:52:38.935 INFO:teuthology.orchestra.run.smithi196.stdout:1047 read 0xd39e00 thru 0xd445a4 (0xa7a5 bytes) 2024-05-08T08:52:38.935 INFO:teuthology.orchestra.run.smithi196.stdout:1048 trunc from 0x74bb400 to 0x7e2d200 2024-05-08T08:52:38.935 INFO:teuthology.orchestra.run.smithi196.stdout:1050 read 0x5945800 thru 0x594f631 (0x9e32 bytes) 2024-05-08T08:52:38.935 INFO:teuthology.orchestra.run.smithi196.stdout:1051 write 0x734f400 thru 0x7358ff8 (0x9bf9 bytes) 2024-05-08T08:52:38.935 INFO:teuthology.orchestra.run.smithi196.stdout:1053 read 0xe9b000 thru 0xea7d5f (0xcd60 bytes) 2024-05-08T08:52:38.935 INFO:teuthology.orchestra.run.smithi196.stdout:1054 write 0x590b800 thru 0x590cd67 (0x1568 bytes) 2024-05-08T08:52:38.935 INFO:teuthology.orchestra.run.smithi196.stdout:1056 read 0x4de6e00 thru 0x4ded0f9 (0x62fa bytes) 2024-05-08T08:52:38.935 INFO:teuthology.orchestra.run.smithi196.stdout:1057 punch from 0x5c08a00 to 0x5c0b000, (0x2600 bytes) 2024-05-08T08:52:38.936 INFO:teuthology.orchestra.run.smithi196.stdout:1062 trunc from 0x7e2d200 to 0xb2c3800 2024-05-08T08:52:38.936 INFO:teuthology.orchestra.run.smithi196.stdout:1064 write 0x3f64e00 thru 0x3f72689 (0xd88a bytes) 2024-05-08T08:52:38.936 INFO:teuthology.orchestra.run.smithi196.stdout:1069 read 0x619c000 thru 0x61a26bd (0x66be bytes) 2024-05-08T08:52:38.936 INFO:teuthology.orchestra.run.smithi196.stdout:1072 read 0x891ca00 thru 0x89265f0 (0x9bf1 bytes) 2024-05-08T08:52:38.936 INFO:teuthology.orchestra.run.smithi196.stdout:1075 trunc from 0xb2c3800 to 0x789b400 2024-05-08T08:52:38.936 INFO:teuthology.orchestra.run.smithi196.stdout:1077 write 0xb0e00 thru 0xbfd91 (0xef92 bytes) 2024-05-08T08:52:38.936 INFO:teuthology.orchestra.run.smithi196.stdout:1078 read 0x76fb000 thru 0x770878e (0xd78f bytes) 2024-05-08T08:52:38.936 INFO:teuthology.orchestra.run.smithi196.stdout:1080 punch from 0x2780e00 to 0x278be00, (0xb000 bytes) 2024-05-08T08:52:38.936 INFO:teuthology.orchestra.run.smithi196.stdout:1081 write 0xbecc600 thru 0xbed52b2 (0x8cb3 bytes) 2024-05-08T08:52:38.936 INFO:teuthology.orchestra.run.smithi196.stdout:1083 read 0xa77ac00 thru 0xa77e794 (0x3b95 bytes) 2024-05-08T08:52:38.936 INFO:teuthology.orchestra.run.smithi196.stdout:1084 write 0xb3c00 thru 0xb6b1e (0x2f1f bytes) 2024-05-08T08:52:38.936 INFO:teuthology.orchestra.run.smithi196.stdout:1087 read 0x8bad600 thru 0x8bb63c3 (0x8dc4 bytes) 2024-05-08T08:52:38.936 INFO:teuthology.orchestra.run.smithi196.stdout:1090 read 0x9d0c800 thru 0x9d1b92b (0xf12c bytes) 2024-05-08T08:52:38.936 INFO:teuthology.orchestra.run.smithi196.stdout:1092 trunc from 0xbed5400 to 0x9a21200 2024-05-08T08:52:38.937 INFO:teuthology.orchestra.run.smithi196.stdout:1093 read 0x5c9a600 thru 0x5ca83dd (0xddde bytes) 2024-05-08T08:52:38.937 INFO:teuthology.orchestra.run.smithi196.stdout:1094 punch from 0x273a00 to 0x283400, (0xfa00 bytes) 2024-05-08T08:52:38.937 INFO:teuthology.orchestra.run.smithi196.stdout:1096 read 0x8a26c00 thru 0x8a34678 (0xda79 bytes) 2024-05-08T08:52:38.937 INFO:teuthology.orchestra.run.smithi196.stdout:1098 punch from 0x4e2a000 to 0x4e39800, (0xf800 bytes) 2024-05-08T08:52:38.937 INFO:teuthology.orchestra.run.smithi196.stdout:1099 write 0x804ee00 thru 0x8052f6c (0x416d bytes) 2024-05-08T08:52:38.937 INFO:teuthology.orchestra.run.smithi196.stdout:1101 trunc from 0x9a21200 to 0x9e44a00 2024-05-08T08:52:38.937 INFO:teuthology.orchestra.run.smithi196.stdout:1102 trunc from 0x9e44a00 to 0x1a59e00 2024-05-08T08:52:38.937 INFO:teuthology.orchestra.run.smithi196.stdout:1103 read 0xea1800 thru 0xea8dfc (0x75fd bytes) 2024-05-08T08:52:38.938 INFO:teuthology.orchestra.run.smithi196.stdout:1104 read 0x133cc00 thru 0x1343f92 (0x7393 bytes) 2024-05-08T08:52:38.938 INFO:teuthology.orchestra.run.smithi196.stdout:1111 write 0xe55400 thru 0xe643ee (0xefef bytes) 2024-05-08T08:52:38.959 INFO:teuthology.orchestra.run.smithi196.stdout:1115 write 0xa474000 thru 0xa47a897 (0x6898 bytes) 2024-05-08T08:52:38.966 INFO:teuthology.orchestra.run.smithi196.stdout:1116 punch from 0x56bbc00 to 0x56c8c00, (0xd000 bytes) 2024-05-08T08:52:38.967 INFO:teuthology.orchestra.run.smithi196.stdout:1117 clone 8 order 24 su 2097152 sc 3 2024-05-08T08:52:39.030 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:52:39.033+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:52:39.095 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:39.095 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:39.105 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:52:39.116 INFO:teuthology.orchestra.run.smithi196.stdout:leaving image image_client.0-clone7 intact 2024-05-08T08:52:39.133 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:39.136+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:39.218 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:39.218 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:39.227 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:52:39.234 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:52:39.235+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:52:39.333 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:52:39.336+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:52:39.342 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:39.342 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:39.351 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:52:39.433 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:39.437+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:39.505 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:39.505 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:39.514 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:52:39.560 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:39.563+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:39.583 INFO:teuthology.orchestra.run.smithi196.stdout:checking clone #6, image image_client.0-clone6 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent7 2024-05-08T08:52:39.634 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:39.634 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:39.635 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:52:39.638+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:52:39.645 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:52:39.736 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:52:39.740+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:52:39.738 INFO:teuthology.orchestra.run.smithi196.stdout:1119 read 0x5985200 thru 0x5985cbe (0xabf bytes) 2024-05-08T08:52:39.755 INFO:teuthology.orchestra.run.smithi196.stdout:1121 write 0xbae4c00 thru 0xbaf332c (0xe72d bytes) 2024-05-08T08:52:39.756 INFO:teuthology.orchestra.run.smithi196.stdout:1123 read 0xa947200 thru 0xa9548f0 (0xd6f1 bytes) 2024-05-08T08:52:39.756 INFO:teuthology.orchestra.run.smithi196.stdout:1124 read 0x1d8c00 thru 0x1db752 (0x2b53 bytes) 2024-05-08T08:52:39.756 INFO:teuthology.orchestra.run.smithi196.stdout:1125 read 0x43f3e00 thru 0x43f93b4 (0x55b5 bytes) 2024-05-08T08:52:39.757 INFO:teuthology.orchestra.run.smithi196.stdout:1129 write 0x1a95000 thru 0x1a9bc73 (0x6c74 bytes) 2024-05-08T08:52:39.758 INFO:teuthology.orchestra.run.smithi196.stdout:1132 read 0x908d400 thru 0x909b29f (0xdea0 bytes) 2024-05-08T08:52:39.758 INFO:teuthology.orchestra.run.smithi196.stdout:1138 read 0x286ac00 thru 0x2875cf2 (0xb0f3 bytes) 2024-05-08T08:52:39.758 INFO:teuthology.orchestra.run.smithi196.stdout:1139 write 0x1fed200 thru 0x1ff676f (0x9570 bytes) 2024-05-08T08:52:39.758 INFO:teuthology.orchestra.run.smithi196.stdout:1140 flatten 2024-05-08T08:52:39.802 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:39.802 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:39.811 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:52:39.855 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:39.858+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:39.906 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T08:52:39.923 INFO:teuthology.orchestra.run.smithi196.stdout:1141 trunc from 0xbaf3400 to 0xae70a00 2024-05-08T08:52:39.924 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:39.924 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:39.934 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:52:39.934 INFO:teuthology.orchestra.run.smithi196.stdout:1144 read 0x72e4000 thru 0x72e4ce1 (0xce2 bytes) 2024-05-08T08:52:39.935 INFO:teuthology.orchestra.run.smithi196.stdout:1145 write 0x3ec6a00 thru 0x3ed1bba (0xb1bb bytes) 2024-05-08T08:52:39.936 INFO:teuthology.orchestra.run.smithi196.stdout:1146 write 0xd0b200 thru 0xd1559e (0xa39f bytes) 2024-05-08T08:52:39.937 INFO:teuthology.orchestra.run.smithi196.stdout:1151 punch from 0x162fa00 to 0x1630c00, (0x1200 bytes) 2024-05-08T08:52:39.938 INFO:teuthology.orchestra.run.smithi196.stdout:1153 read 0x784d000 thru 0x78548b7 (0x78b8 bytes) 2024-05-08T08:52:39.950 INFO:teuthology.orchestra.run.smithi196.stdout:1154 write 0xae79e00 thru 0xae8039b (0x659c bytes) 2024-05-08T08:52:39.963 INFO:teuthology.orchestra.run.smithi196.stdout:1157 trunc from 0xae80400 to 0xc3a000 2024-05-08T08:52:39.975 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:39.978+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:39.981 INFO:teuthology.orchestra.run.smithi196.stdout:1159 trunc from 0xc3a000 to 0x5dea00 2024-05-08T08:52:40.004 INFO:teuthology.orchestra.run.smithi196.stdout:1160 write 0x497a400 thru 0x49888ac (0xe4ad bytes) 2024-05-08T08:52:40.025 INFO:teuthology.orchestra.run.smithi196.stdout:1162 write 0xa603800 thru 0xa611c96 (0xe497 bytes) 2024-05-08T08:52:40.025 INFO:teuthology.orchestra.run.smithi196.stdout:1164 punch from 0x7c61200 to 0x7c6f200, (0xe000 bytes) 2024-05-08T08:52:40.027 INFO:teuthology.orchestra.run.smithi196.stdout:1165 punch from 0x20e6a00 to 0x20f3c00, (0xd200 bytes) 2024-05-08T08:52:40.027 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-05-08T08:52:40.027 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T08:52:40.028 INFO:tasks.thrashosds.thrasher:inject_pause on osd.0 2024-05-08T08:52:40.028 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-05-08T08:52:40.028 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-05-08T08:52:40.028 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok config set filestore_inject_stall 3 2024-05-08T08:52:40.034 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-05-08T08:52:40.041 INFO:teuthology.orchestra.run.smithi196.stdout:1166 write 0xe2f9e00 thru 0xe30034c (0x654d bytes) 2024-05-08T08:52:40.043 INFO:teuthology.orchestra.run.smithi196.stdout:1167 write 0x1d42c00 thru 0x1d47121 (0x4522 bytes) 2024-05-08T08:52:40.049 INFO:teuthology.orchestra.run.smithi196.stdout:1169 trunc from 0xe300400 to 0x95d0000 2024-05-08T08:52:40.069 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:52:40.069 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:52:40.070 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:52:40.072+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:52:40.074 INFO:teuthology.orchestra.run.smithi196.stdout:1171 read 0x48fdc00 thru 0x4905453 (0x7854 bytes) 2024-05-08T08:52:40.076 INFO:teuthology.orchestra.run.smithi196.stdout:1173 write 0x603ea00 thru 0x60476f3 (0x8cf4 bytes) 2024-05-08T08:52:40.076 INFO:teuthology.orchestra.run.smithi196.stdout:1174 write 0x10b4c00 thru 0x10b68ec (0x1ced bytes) 2024-05-08T08:52:40.076 INFO:teuthology.orchestra.run.smithi196.stdout:1175 punch from 0x3788c00 to 0x3790600, (0x7a00 bytes) 2024-05-08T08:52:40.077 INFO:teuthology.orchestra.run.smithi196.stdout:1176 write 0x5cf6000 thru 0x5cf6e4d (0xe4e bytes) 2024-05-08T08:52:40.077 INFO:teuthology.orchestra.run.smithi196.stdout:1177 punch from 0x1604800 to 0x160ea00, (0xa200 bytes) 2024-05-08T08:52:40.077 INFO:teuthology.orchestra.run.smithi196.stdout:1178 read 0x6b8bc00 thru 0x6b92a4c (0x6e4d bytes) 2024-05-08T08:52:40.078 INFO:teuthology.orchestra.run.smithi196.stdout:1181 punch from 0x432a800 to 0x432fe00, (0x5600 bytes) 2024-05-08T08:52:40.087 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:40.087 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:40.093 INFO:teuthology.orchestra.run.smithi196.stdout:1182 write 0xabaee00 thru 0xabbb6cd (0xc8ce bytes) 2024-05-08T08:52:40.093 INFO:teuthology.orchestra.run.smithi196.stdout:1185 read 0x1f5dc00 thru 0x1f652fb (0x76fc bytes) 2024-05-08T08:52:40.095 INFO:teuthology.orchestra.run.smithi196.stdout:1189 write 0x9ca7200 thru 0x9cb1106 (0x9f07 bytes) 2024-05-08T08:52:40.095 INFO:teuthology.orchestra.run.smithi196.stdout:1191 read 0x4c41600 thru 0x4c4e08f (0xca90 bytes) 2024-05-08T08:52:40.097 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:52:40.098 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:52:40.098 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:52:40.099 INFO:teuthology.orchestra.run.smithi196.stdout:1193 trunc from 0xabbb800 to 0xd1b5400 2024-05-08T08:52:40.113 INFO:teuthology.orchestra.run.smithi196.stdout:skipping zero length punch hole 2024-05-08T08:52:40.113 INFO:teuthology.orchestra.run.smithi196.stdout:1195 read 0x8610200 thru 0x86177dc (0x75dd bytes) 2024-05-08T08:52:40.114 INFO:teuthology.orchestra.run.smithi196.stdout:1196 write 0xb61d200 thru 0xb62a7cb (0xd5cc bytes) 2024-05-08T08:52:40.114 INFO:teuthology.orchestra.run.smithi196.stdout:1199 write 0xa006400 thru 0xa00ee71 (0x8a72 bytes) 2024-05-08T08:52:40.115 INFO:teuthology.orchestra.run.smithi196.stdout:1201 write 0x167c000 thru 0x167d942 (0x1943 bytes) 2024-05-08T08:52:40.116 INFO:teuthology.orchestra.run.smithi196.stdout:1203 read 0x7eede00 thru 0x7ef691a (0x8b1b bytes) 2024-05-08T08:52:40.116 INFO:teuthology.orchestra.run.smithi196.stdout:1204 write 0xa18aa00 thru 0xa18c625 (0x1c26 bytes) 2024-05-08T08:52:40.116 INFO:teuthology.orchestra.run.smithi196.stdout:1205 punch from 0xc7a8200 to 0xc7b2600, (0xa400 bytes) 2024-05-08T08:52:40.118 INFO:teuthology.orchestra.run.smithi196.stdout:1206 punch from 0x4812a00 to 0x4818400, (0x5a00 bytes) 2024-05-08T08:52:40.118 INFO:teuthology.orchestra.run.smithi196.stdout:1207 punch from 0x6b26000 to 0x6b26400, (0x400 bytes) 2024-05-08T08:52:40.130 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:52:40.131 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:52:40.133 INFO:teuthology.orchestra.run.smithi196.stdout:1210 write 0xd75d600 thru 0xd76ccca (0xf6cb bytes) 2024-05-08T08:52:40.135 INFO:teuthology.orchestra.run.smithi196.stdout:1213 write 0xd63bc00 thru 0xd64482c (0x8c2d bytes) 2024-05-08T08:52:40.135 INFO:teuthology.orchestra.run.smithi196.stdout:1214 write 0x2ddd200 thru 0x2de8f09 (0xbd0a bytes) 2024-05-08T08:52:40.135 INFO:teuthology.orchestra.run.smithi196.stdout:1217 read 0xa63be00 thru 0xa647907 (0xbb08 bytes) 2024-05-08T08:52:40.135 INFO:teuthology.orchestra.run.smithi196.stdout:1219 read 0x3dbc800 thru 0x3dc6ae4 (0xa2e5 bytes) 2024-05-08T08:52:40.140 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:52:40.144+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:52:40.142 INFO:teuthology.orchestra.run.smithi196.stdout:1220 trunc from 0xd76ce00 to 0x81e5000 2024-05-08T08:52:40.166 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:52:40.167 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:52:40.167 INFO:teuthology.orchestra.run.smithi196.stdout:1223 read 0x49dd600 thru 0x49ecc88 (0xf689 bytes) 2024-05-08T08:52:40.168 INFO:teuthology.orchestra.run.smithi196.stdout:1224 write 0x376ba00 thru 0x3771902 (0x5f03 bytes) 2024-05-08T08:52:40.180 INFO:teuthology.orchestra.run.smithi196.stdout:1230 write 0x878ee00 thru 0x8790d57 (0x1f58 bytes) 2024-05-08T08:52:40.181 INFO:teuthology.orchestra.run.smithi196.stdout:1232 punch from 0x7395a00 to 0x73a1600, (0xbc00 bytes) 2024-05-08T08:52:40.185 INFO:teuthology.orchestra.run.smithi196.stdout:1234 trunc from 0x8790e00 to 0x62dd800 2024-05-08T08:52:40.191 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:52:40.192 INFO:teuthology.orchestra.run.smithi059.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-05-08T08:52:40.192 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:52:40.199 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:52:40.199 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:52:40.204 INFO:teuthology.orchestra.run.smithi196.stdout:1236 read 0x1644200 thru 0x16526f3 (0xe4f4 bytes) 2024-05-08T08:52:40.214 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:52:40.214 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:52:40.214 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:52:40.214 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:52:40.214 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:52:40.222 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:52:40.222 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:52:40.224 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:52:40.225 INFO:teuthology.orchestra.run.smithi196.stdout:1237 write 0x903dc00 thru 0x904d2ac (0xf6ad bytes) 2024-05-08T08:52:40.234 INFO:teuthology.orchestra.run.smithi196.stdout:1238 write 0xfec800 thru 0xfeefc7 (0x27c8 bytes) 2024-05-08T08:52:40.238 INFO:teuthology.orchestra.run.smithi196.stdout:1239 trunc from 0x904d400 to 0x5786000 2024-05-08T08:52:40.262 INFO:teuthology.orchestra.run.smithi196.stdout:1241 trunc from 0x5786000 to 0xa36f000 2024-05-08T08:52:40.268 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:40.271+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:40.271 INFO:teuthology.orchestra.run.smithi196.stdout:1242 write 0x7c69000 thru 0x7c73e61 (0xae62 bytes) 2024-05-08T08:52:40.293 INFO:teuthology.orchestra.run.smithi196.stdout:1244 write 0xb48c200 thru 0xb48fb1d (0x391e bytes) 2024-05-08T08:52:40.293 INFO:teuthology.orchestra.run.smithi196.stdout:1247 read 0x5b63a00 thru 0x5b72a73 (0xf074 bytes) 2024-05-08T08:52:40.293 INFO:teuthology.orchestra.run.smithi196.stdout:1254 write 0x3baaa00 thru 0x3bafc56 (0x5257 bytes) 2024-05-08T08:52:40.293 INFO:teuthology.orchestra.run.smithi196.stdout:1255 write 0xfe1600 thru 0xfecf11 (0xb912 bytes) 2024-05-08T08:52:40.303 INFO:teuthology.orchestra.run.smithi196.stdout:1256 trunc from 0xb48fc00 to 0x3c0de00 2024-05-08T08:52:40.329 INFO:teuthology.orchestra.run.smithi196.stdout:1257 read 0x3b73a00 thru 0x3b73fcc (0x5cd bytes) 2024-05-08T08:52:40.329 INFO:teuthology.orchestra.run.smithi196.stdout:1259 read 0x22b1e00 thru 0x22b1fd5 (0x1d6 bytes) 2024-05-08T08:52:40.329 INFO:teuthology.orchestra.run.smithi196.stdout:1260 read 0x1e10800 thru 0x1e14db7 (0x45b8 bytes) 2024-05-08T08:52:40.331 INFO:teuthology.orchestra.run.smithi196.stdout:1261 trunc from 0x3c0de00 to 0x4dd7400 2024-05-08T08:52:40.336 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:52:40.336 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:52:40.336 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:52:40.336 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:52:40.336 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.336 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22069 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 749568~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:40.336 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857624+0000", 2024-05-08T08:52:40.336 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 12.480552536999999, 2024-05-08T08:52:40.336 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046969958999999999, 2024-05-08T08:52:40.336 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:40.336 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:40.336 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:40.337 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:40.337 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:40.337 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:40.337 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22069 2024-05-08T08:52:40.337 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.337 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:40.337 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.337 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:40.337 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857624+0000", 2024-05-08T08:52:40.337 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:40.337 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.337 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.337 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:40.337 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857624+0000", 2024-05-08T08:52:40.337 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:40.337 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.338 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.338 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:40.338 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857625+0000", 2024-05-08T08:52:40.338 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.03e-07 2024-05-08T08:52:40.338 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.338 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.338 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:40.338 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857627+0000", 2024-05-08T08:52:40.338 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9369999999999998e-06 2024-05-08T08:52:40.338 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.338 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.338 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:40.338 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857633+0000", 2024-05-08T08:52:40.338 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1229999999999997e-06 2024-05-08T08:52:40.339 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.339 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.339 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:40.339 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857637+0000", 2024-05-08T08:52:40.339 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7820000000000001e-06 2024-05-08T08:52:40.339 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.339 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.339 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:40.339 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904457+0000", 2024-05-08T08:52:40.339 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046819715999999997 2024-05-08T08:52:40.339 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.339 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.339 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:40.339 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904524+0000", 2024-05-08T08:52:40.339 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6371000000000005e-05 2024-05-08T08:52:40.340 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.340 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.340 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:40.340 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904594+0000", 2024-05-08T08:52:40.340 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0627e-05 2024-05-08T08:52:40.340 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.340 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:40.340 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.340 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.340 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.340 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22071 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 765952~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:40.340 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857643+0000", 2024-05-08T08:52:40.340 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 12.480533454, 2024-05-08T08:52:40.340 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047106528000000002, 2024-05-08T08:52:40.340 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:40.341 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:40.341 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:40.341 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:40.341 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:40.341 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:40.341 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22071 2024-05-08T08:52:40.341 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.341 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:40.341 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.341 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:40.341 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857643+0000", 2024-05-08T08:52:40.341 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:40.341 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.341 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.341 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:40.342 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857643+0000", 2024-05-08T08:52:40.342 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:40.342 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.342 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.342 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:40.342 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857644+0000", 2024-05-08T08:52:40.342 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2000000000000001e-07 2024-05-08T08:52:40.342 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.342 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.342 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:40.342 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857646+0000", 2024-05-08T08:52:40.342 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0059999999999999e-06 2024-05-08T08:52:40.342 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.342 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.343 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:40.343 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857651+0000", 2024-05-08T08:52:40.343 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.84e-06 2024-05-08T08:52:40.343 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.343 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.343 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:40.343 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857657+0000", 2024-05-08T08:52:40.343 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5620000000000003e-06 2024-05-08T08:52:40.343 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.343 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.343 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:40.343 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904620+0000", 2024-05-08T08:52:40.343 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046962661000000003 2024-05-08T08:52:40.343 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.343 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.344 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:40.344 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904678+0000", 2024-05-08T08:52:40.344 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8656999999999998e-05 2024-05-08T08:52:40.344 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.344 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.344 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:40.344 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904750+0000", 2024-05-08T08:52:40.344 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1482000000000004e-05 2024-05-08T08:52:40.344 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.344 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:40.344 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.344 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.344 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.344 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22073 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 782336~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:40.344 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:52:40.345 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 12.480509445999999, 2024-05-08T08:52:40.345 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047244234000000003, 2024-05-08T08:52:40.345 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:40.345 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:40.345 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:40.345 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:40.345 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:40.345 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:40.345 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22073 2024-05-08T08:52:40.345 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.345 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:40.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.345 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:40.345 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:52:40.345 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:40.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.346 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:40.346 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:52:40.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:40.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.346 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:40.346 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857668+0000", 2024-05-08T08:52:40.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7800000000000002e-07 2024-05-08T08:52:40.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.346 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:40.346 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857670+0000", 2024-05-08T08:52:40.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0820000000000001e-06 2024-05-08T08:52:40.347 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.347 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.347 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:40.347 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857678+0000", 2024-05-08T08:52:40.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7589999999999997e-06 2024-05-08T08:52:40.347 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.347 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.347 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:40.347 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857691+0000", 2024-05-08T08:52:40.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3672999999999999e-05 2024-05-08T08:52:40.347 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.347 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.347 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:40.347 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904780+0000", 2024-05-08T08:52:40.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047088616999999999 2024-05-08T08:52:40.347 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.348 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.348 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:40.348 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904840+0000", 2024-05-08T08:52:40.348 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0356999999999998e-05 2024-05-08T08:52:40.348 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.348 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.348 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:40.348 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904912+0000", 2024-05-08T08:52:40.348 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1367999999999998e-05 2024-05-08T08:52:40.348 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.348 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:40.348 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.348 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.348 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.348 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22075 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 798720~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:40.349 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:52:40.349 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 12.480479048999999, 2024-05-08T08:52:40.349 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047384643999999997, 2024-05-08T08:52:40.349 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:40.349 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:40.349 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:40.349 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:40.349 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:40.349 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:40.349 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22075 2024-05-08T08:52:40.349 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.349 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:40.349 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.349 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:40.350 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:52:40.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:40.350 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.350 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.350 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:40.350 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:52:40.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:40.350 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.350 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.350 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:40.350 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:52:40.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7399999999999999e-07 2024-05-08T08:52:40.350 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.350 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.351 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:40.351 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857705+0000", 2024-05-08T08:52:40.351 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1980000000000004e-06 2024-05-08T08:52:40.351 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.351 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.351 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:40.351 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857713+0000", 2024-05-08T08:52:40.351 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7039999999999994e-06 2024-05-08T08:52:40.351 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.351 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.351 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:40.351 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857718+0000", 2024-05-08T08:52:40.351 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.865e-06 2024-05-08T08:52:40.351 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.351 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.351 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:40.352 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904941+0000", 2024-05-08T08:52:40.352 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047223198000000001 2024-05-08T08:52:40.352 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.352 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.352 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:40.352 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904999+0000", 2024-05-08T08:52:40.352 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8254999999999997e-05 2024-05-08T08:52:40.352 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.352 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.352 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:40.352 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905082+0000", 2024-05-08T08:52:40.352 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3049999999999999e-05 2024-05-08T08:52:40.352 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.352 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:40.352 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.352 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.353 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.353 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22077 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 815104~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:40.353 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:52:40.353 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 12.480441038, 2024-05-08T08:52:40.353 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047519448999999998, 2024-05-08T08:52:40.353 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:40.353 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:40.353 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:40.353 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:40.353 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:40.353 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:40.353 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22077 2024-05-08T08:52:40.353 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.353 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:40.353 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.354 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:40.354 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:52:40.354 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:40.354 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.354 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.354 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:40.354 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:52:40.354 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:40.354 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.354 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.354 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:40.354 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:52:40.354 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.99e-07 2024-05-08T08:52:40.354 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.354 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.355 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:40.355 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857739+0000", 2024-05-08T08:52:40.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3599999999999999e-06 2024-05-08T08:52:40.355 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.355 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.355 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:40.355 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857745+0000", 2024-05-08T08:52:40.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5139999999999998e-06 2024-05-08T08:52:40.355 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.355 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.355 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:40.355 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857750+0000", 2024-05-08T08:52:40.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8550000000000001e-06 2024-05-08T08:52:40.355 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.355 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.355 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:40.356 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905110+0000", 2024-05-08T08:52:40.356 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047360126000000002 2024-05-08T08:52:40.356 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.356 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.356 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:40.356 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905175+0000", 2024-05-08T08:52:40.356 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4689000000000004e-05 2024-05-08T08:52:40.356 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.356 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.356 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:40.356 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905255+0000", 2024-05-08T08:52:40.356 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0306e-05 2024-05-08T08:52:40.356 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.356 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:40.356 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.356 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.357 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.357 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22079 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 831488~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:40.357 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:52:40.357 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 12.48042049, 2024-05-08T08:52:40.357 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.04767499, 2024-05-08T08:52:40.357 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:40.357 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:40.357 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:40.357 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:40.357 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:40.357 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:40.357 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22079 2024-05-08T08:52:40.357 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.357 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:40.357 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.357 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:40.358 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:52:40.358 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:40.358 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.358 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.358 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:40.358 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:52:40.358 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:40.358 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.358 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.358 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:40.358 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857757+0000", 2024-05-08T08:52:40.358 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8299999999999997e-07 2024-05-08T08:52:40.358 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.358 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.358 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:40.358 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857759+0000", 2024-05-08T08:52:40.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1390000000000002e-06 2024-05-08T08:52:40.359 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.359 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.359 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:40.359 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857765+0000", 2024-05-08T08:52:40.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4609999999999996e-06 2024-05-08T08:52:40.359 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.359 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.359 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:40.359 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857770+0000", 2024-05-08T08:52:40.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6689999999999997e-06 2024-05-08T08:52:40.359 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.359 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.359 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:40.359 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905281+0000", 2024-05-08T08:52:40.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047510668999999998 2024-05-08T08:52:40.360 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.360 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.360 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:40.360 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905344+0000", 2024-05-08T08:52:40.360 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2797000000000002e-05 2024-05-08T08:52:40.360 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.360 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.360 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:40.360 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905431+0000", 2024-05-08T08:52:40.360 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7771999999999999e-05 2024-05-08T08:52:40.360 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.360 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:40.360 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.360 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.360 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.361 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22082 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 847872~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:40.361 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:52:40.361 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 12.480400799, 2024-05-08T08:52:40.361 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047831934999999999, 2024-05-08T08:52:40.361 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:40.361 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:40.361 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:40.361 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:40.361 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:40.361 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:40.361 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22082 2024-05-08T08:52:40.361 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.361 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:40.361 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.361 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:40.362 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:52:40.362 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:40.362 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.362 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.362 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:40.362 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:52:40.362 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:40.362 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.362 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.362 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:40.362 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:52:40.362 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:52:40.362 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.362 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.363 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:40.363 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857778+0000", 2024-05-08T08:52:40.363 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0789999999999999e-06 2024-05-08T08:52:40.363 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.363 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.363 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:40.363 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857784+0000", 2024-05-08T08:52:40.363 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0480000000000004e-06 2024-05-08T08:52:40.363 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.363 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.363 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:40.363 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857790+0000", 2024-05-08T08:52:40.363 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2719999999999997e-06 2024-05-08T08:52:40.363 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.364 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.364 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:40.364 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905458+0000", 2024-05-08T08:52:40.364 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047668470999999997 2024-05-08T08:52:40.364 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.364 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.364 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:40.364 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905521+0000", 2024-05-08T08:52:40.364 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2952000000000003e-05 2024-05-08T08:52:40.364 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.364 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.364 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:40.364 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905608+0000", 2024-05-08T08:52:40.364 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6759000000000001e-05 2024-05-08T08:52:40.364 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.365 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:40.365 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.365 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.365 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.365 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22084 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 864256~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:40.365 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:52:40.365 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 12.480376849000001, 2024-05-08T08:52:40.365 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047977748000000001, 2024-05-08T08:52:40.365 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:40.365 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:40.365 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:40.365 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:40.365 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:40.365 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:40.365 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22084 2024-05-08T08:52:40.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.366 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:40.366 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.366 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:40.366 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:52:40.366 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:40.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.366 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.366 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:40.366 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:52:40.366 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:40.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.366 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.366 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:40.367 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:52:40.367 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:52:40.367 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.367 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.367 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:40.367 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857802+0000", 2024-05-08T08:52:40.367 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0779999999999998e-06 2024-05-08T08:52:40.367 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.367 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.367 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:40.367 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857812+0000", 2024-05-08T08:52:40.367 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7939999999999997e-06 2024-05-08T08:52:40.367 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.367 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.367 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:40.368 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857817+0000", 2024-05-08T08:52:40.368 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8899999999999998e-06 2024-05-08T08:52:40.368 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.368 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.368 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:40.368 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905636+0000", 2024-05-08T08:52:40.368 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047819252 2024-05-08T08:52:40.368 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.368 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.368 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:40.368 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905700+0000", 2024-05-08T08:52:40.368 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4016999999999994e-05 2024-05-08T08:52:40.368 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.368 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.368 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:40.368 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905778+0000", 2024-05-08T08:52:40.369 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7306999999999997e-05 2024-05-08T08:52:40.369 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.369 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:40.369 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.369 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.369 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.369 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22086 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 880640~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:40.369 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:52:40.369 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 12.480353499, 2024-05-08T08:52:40.369 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048119149, 2024-05-08T08:52:40.369 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:40.369 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:40.369 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:40.369 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:40.369 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:40.370 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:40.370 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22086 2024-05-08T08:52:40.370 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.370 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:40.370 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.370 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:40.370 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:52:40.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:40.370 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.370 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.370 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:40.370 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:52:40.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:40.370 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.370 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.370 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:40.371 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857824+0000", 2024-05-08T08:52:40.371 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4200000000000002e-07 2024-05-08T08:52:40.371 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.371 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.371 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:40.371 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857826+0000", 2024-05-08T08:52:40.371 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0590000000000001e-06 2024-05-08T08:52:40.371 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.371 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.371 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:40.371 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857832+0000", 2024-05-08T08:52:40.371 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8029999999999999e-06 2024-05-08T08:52:40.371 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.371 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.371 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:40.371 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857837+0000", 2024-05-08T08:52:40.372 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7929999999999997e-06 2024-05-08T08:52:40.372 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.372 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.372 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:40.372 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905803+0000", 2024-05-08T08:52:40.372 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047965761000000003 2024-05-08T08:52:40.372 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.372 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.372 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:40.372 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905862+0000", 2024-05-08T08:52:40.372 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9203000000000002e-05 2024-05-08T08:52:40.372 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.372 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.372 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:40.372 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905942+0000", 2024-05-08T08:52:40.373 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0187999999999999e-05 2024-05-08T08:52:40.373 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.373 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:40.373 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.373 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.373 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.373 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22088 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 897024~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:40.373 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:52:40.373 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 12.480333447, 2024-05-08T08:52:40.373 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048256049000000002, 2024-05-08T08:52:40.373 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:40.373 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:40.373 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:40.373 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:40.373 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:40.374 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:40.374 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22088 2024-05-08T08:52:40.374 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.374 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:40.374 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.374 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:40.374 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:52:40.374 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:40.374 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.374 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.374 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:40.374 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:52:40.374 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:40.375 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.375 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.375 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:40.375 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857844+0000", 2024-05-08T08:52:40.375 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.46e-07 2024-05-08T08:52:40.375 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.375 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.375 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:40.375 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857846+0000", 2024-05-08T08:52:40.375 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0899999999999999e-06 2024-05-08T08:52:40.375 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.375 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.375 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:40.375 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857855+0000", 2024-05-08T08:52:40.376 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8219999999999996e-06 2024-05-08T08:52:40.376 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.376 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.376 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:40.376 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857860+0000", 2024-05-08T08:52:40.376 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2989999999999996e-06 2024-05-08T08:52:40.376 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.376 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.376 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:40.376 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905966+0000", 2024-05-08T08:52:40.376 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048106542000000002 2024-05-08T08:52:40.376 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.376 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.376 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:40.376 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906024+0000", 2024-05-08T08:52:40.377 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7958000000000003e-05 2024-05-08T08:52:40.377 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.377 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.377 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:40.377 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906099+0000", 2024-05-08T08:52:40.377 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4992000000000003e-05 2024-05-08T08:52:40.377 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.377 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:40.377 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.377 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.377 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.377 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22091 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 913408~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:40.377 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:52:40.377 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 12.480309687, 2024-05-08T08:52:40.377 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048395290000000001, 2024-05-08T08:52:40.378 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:40.378 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:40.378 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:40.378 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:40.378 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:40.378 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:40.378 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22091 2024-05-08T08:52:40.378 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.378 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:40.378 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.378 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:40.378 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:52:40.378 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:40.378 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.378 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.379 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:40.379 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:52:40.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:40.379 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.379 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.379 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:40.379 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:52:40.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4900000000000001e-07 2024-05-08T08:52:40.379 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.379 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.379 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:40.379 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857870+0000", 2024-05-08T08:52:40.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2510000000000001e-06 2024-05-08T08:52:40.379 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.379 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.379 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:40.380 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857877+0000", 2024-05-08T08:52:40.380 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0809999999999996e-06 2024-05-08T08:52:40.380 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.380 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.380 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:40.380 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857881+0000", 2024-05-08T08:52:40.380 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.724e-06 2024-05-08T08:52:40.380 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.380 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.380 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:40.380 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906127+0000", 2024-05-08T08:52:40.380 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048245345000000002 2024-05-08T08:52:40.380 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.380 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.380 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:40.380 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906190+0000", 2024-05-08T08:52:40.381 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2762000000000007e-05 2024-05-08T08:52:40.381 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.381 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.381 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:40.381 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906262+0000", 2024-05-08T08:52:40.381 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2778e-05 2024-05-08T08:52:40.381 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.381 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:40.381 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.381 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.381 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.381 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22093 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 929792~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:40.381 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:52:40.381 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 12.480287323000001, 2024-05-08T08:52:40.381 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048532875000000003, 2024-05-08T08:52:40.381 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:40.382 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:40.382 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:40.382 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:40.382 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:40.382 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:40.382 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22093 2024-05-08T08:52:40.382 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.382 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:40.382 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.382 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:40.382 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:52:40.382 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:40.382 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.382 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.382 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:40.383 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:52:40.383 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:40.383 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.383 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.383 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:40.383 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857890+0000", 2024-05-08T08:52:40.383 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-07 2024-05-08T08:52:40.383 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.383 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.383 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:40.383 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857899+0000", 2024-05-08T08:52:40.383 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6889999999999999e-06 2024-05-08T08:52:40.383 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.383 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.383 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:40.383 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857906+0000", 2024-05-08T08:52:40.384 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6490000000000002e-06 2024-05-08T08:52:40.384 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.384 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:40.384 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857917+0000", 2024-05-08T08:52:40.384 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0764999999999999e-05 2024-05-08T08:52:40.384 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.384 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:40.384 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906287+0000", 2024-05-08T08:52:40.384 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048370442 2024-05-08T08:52:40.384 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.384 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:40.384 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906349+0000", 2024-05-08T08:52:40.384 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1537000000000004e-05 2024-05-08T08:52:40.385 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.385 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.385 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:40.385 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906422+0000", 2024-05-08T08:52:40.385 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3431999999999998e-05 2024-05-08T08:52:40.385 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.385 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:40.385 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.385 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.385 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.385 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22095 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 946176~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:40.385 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:52:40.385 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 12.480252619, 2024-05-08T08:52:40.385 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048657593999999998, 2024-05-08T08:52:40.385 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:40.385 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:40.386 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:40.386 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:40.386 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:40.386 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:40.386 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22095 2024-05-08T08:52:40.386 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.386 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:40.386 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.386 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:40.386 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:52:40.386 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:40.386 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.386 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.386 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:40.386 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:52:40.387 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:40.387 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.387 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.387 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:40.387 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857925+0000", 2024-05-08T08:52:40.387 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3099999999999998e-07 2024-05-08T08:52:40.387 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.387 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.387 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:40.387 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857927+0000", 2024-05-08T08:52:40.387 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.34e-06 2024-05-08T08:52:40.387 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.387 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.387 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:40.387 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857938+0000", 2024-05-08T08:52:40.388 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1426e-05 2024-05-08T08:52:40.388 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.388 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.388 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:40.388 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857943+0000", 2024-05-08T08:52:40.388 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5140000000000003e-06 2024-05-08T08:52:40.388 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.388 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.388 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:40.388 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906448+0000", 2024-05-08T08:52:40.388 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048504641000000001 2024-05-08T08:52:40.388 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.388 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.388 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:40.388 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906510+0000", 2024-05-08T08:52:40.388 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2663000000000006e-05 2024-05-08T08:52:40.389 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.389 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.389 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:40.389 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906582+0000", 2024-05-08T08:52:40.389 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1478999999999998e-05 2024-05-08T08:52:40.389 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.389 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:40.389 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.389 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.389 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.389 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22097 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 962560~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:40.389 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:52:40.389 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 12.48022493, 2024-05-08T08:52:40.389 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048791353000000003, 2024-05-08T08:52:40.389 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:40.390 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:40.390 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:40.390 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:40.390 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:40.390 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:40.390 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22097 2024-05-08T08:52:40.390 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.390 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:40.390 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.390 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:40.390 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:52:40.390 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:40.390 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.390 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.391 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:40.391 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:52:40.391 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:40.391 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.391 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.391 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:40.391 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:52:40.391 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9200000000000002e-07 2024-05-08T08:52:40.391 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.391 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.391 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:40.391 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857954+0000", 2024-05-08T08:52:40.391 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1900000000000002e-06 2024-05-08T08:52:40.391 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.391 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.392 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:40.392 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857961+0000", 2024-05-08T08:52:40.392 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4989999999999999e-06 2024-05-08T08:52:40.392 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.392 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.392 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:40.392 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857971+0000", 2024-05-08T08:52:40.392 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0485999999999999e-05 2024-05-08T08:52:40.392 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.392 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.392 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:40.392 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906607+0000", 2024-05-08T08:52:40.392 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048635109000000003 2024-05-08T08:52:40.392 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.392 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.392 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:40.393 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906667+0000", 2024-05-08T08:52:40.393 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0439000000000002e-05 2024-05-08T08:52:40.393 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.393 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.393 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:40.393 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906743+0000", 2024-05-08T08:52:40.393 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6237999999999998e-05 2024-05-08T08:52:40.393 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.393 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:40.393 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.393 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.393 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.393 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22099 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 978944~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:40.393 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:52:40.393 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 12.480198769999999, 2024-05-08T08:52:40.394 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048932055000000002, 2024-05-08T08:52:40.394 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:40.394 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:40.394 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:40.394 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:40.394 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:40.394 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:40.394 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22099 2024-05-08T08:52:40.394 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.394 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:40.394 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.394 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:40.394 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:52:40.394 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:40.394 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.395 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.395 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:40.395 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:52:40.395 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:40.395 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.395 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.395 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:40.395 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:52:40.395 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.27e-07 2024-05-08T08:52:40.395 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.395 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.395 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:40.395 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857987+0000", 2024-05-08T08:52:40.395 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5890000000000003e-06 2024-05-08T08:52:40.395 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.396 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.396 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:40.396 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857994+0000", 2024-05-08T08:52:40.396 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6619999999999999e-06 2024-05-08T08:52:40.396 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.396 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.396 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:40.396 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857998+0000", 2024-05-08T08:52:40.396 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7110000000000003e-06 2024-05-08T08:52:40.396 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.396 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.396 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:40.396 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906772+0000", 2024-05-08T08:52:40.396 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048773415000000001 2024-05-08T08:52:40.396 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.397 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.397 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:40.397 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906834+0000", 2024-05-08T08:52:40.397 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1752000000000001e-05 2024-05-08T08:52:40.397 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.397 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.397 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:40.397 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906910+0000", 2024-05-08T08:52:40.397 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6499000000000001e-05 2024-05-08T08:52:40.397 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.397 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:40.397 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.397 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.397 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.397 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22101 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 995328~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:40.398 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:52:40.398 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 12.480171639, 2024-05-08T08:52:40.398 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049087364000000001, 2024-05-08T08:52:40.398 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:40.398 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:40.398 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:40.398 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:40.398 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:40.398 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:40.398 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22101 2024-05-08T08:52:40.398 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.398 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:40.398 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.398 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:40.398 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:52:40.399 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:40.399 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.399 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.399 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:40.399 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:52:40.399 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:40.399 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.399 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.399 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:40.399 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858006+0000", 2024-05-08T08:52:40.399 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1399999999999997e-07 2024-05-08T08:52:40.399 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.399 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.399 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:40.399 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858008+0000", 2024-05-08T08:52:40.399 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2060000000000001e-06 2024-05-08T08:52:40.400 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.400 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.400 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:40.400 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858017+0000", 2024-05-08T08:52:40.400 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1579999999999994e-06 2024-05-08T08:52:40.400 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.400 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.400 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:40.400 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858022+0000", 2024-05-08T08:52:40.400 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4199999999999998e-06 2024-05-08T08:52:40.400 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.400 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.400 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:40.400 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906940+0000", 2024-05-08T08:52:40.400 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048917627999999998 2024-05-08T08:52:40.401 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.401 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.401 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:40.401 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907002+0000", 2024-05-08T08:52:40.401 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2399999999999999e-05 2024-05-08T08:52:40.401 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.401 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.401 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:40.401 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907092+0000", 2024-05-08T08:52:40.401 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0137999999999997e-05 2024-05-08T08:52:40.401 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.401 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:40.401 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.401 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.401 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.402 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22103 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1011712~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:40.402 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:52:40.402 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 12.480146854000001, 2024-05-08T08:52:40.402 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049228542, 2024-05-08T08:52:40.402 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:40.402 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:40.402 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:40.402 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:40.402 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:40.402 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:40.402 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22103 2024-05-08T08:52:40.402 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.402 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:40.402 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.403 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:40.403 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:52:40.403 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:40.403 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.403 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.403 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:40.403 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:52:40.403 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:40.403 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.403 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.403 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:40.403 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:52:40.403 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4299999999999999e-07 2024-05-08T08:52:40.403 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.403 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.404 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:40.404 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858032+0000", 2024-05-08T08:52:40.404 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0590000000000001e-06 2024-05-08T08:52:40.404 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.404 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.404 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:40.404 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858037+0000", 2024-05-08T08:52:40.404 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.977e-06 2024-05-08T08:52:40.404 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.404 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.404 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:40.404 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858042+0000", 2024-05-08T08:52:40.404 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7650000000000001e-06 2024-05-08T08:52:40.404 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.404 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.405 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:40.405 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907118+0000", 2024-05-08T08:52:40.405 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049076147000000001 2024-05-08T08:52:40.405 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.405 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.405 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:40.405 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907180+0000", 2024-05-08T08:52:40.405 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1971999999999996e-05 2024-05-08T08:52:40.405 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.405 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.405 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:40.405 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907258+0000", 2024-05-08T08:52:40.405 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8279e-05 2024-05-08T08:52:40.405 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.405 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:40.406 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.406 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.406 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.406 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22105 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1028096~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:40.406 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:52:40.406 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 12.480127611, 2024-05-08T08:52:40.406 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049394609999999999, 2024-05-08T08:52:40.406 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:40.406 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:40.406 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:40.406 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:40.406 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:40.406 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:40.406 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22105 2024-05-08T08:52:40.406 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.407 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:40.407 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.407 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:40.407 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:52:40.407 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:40.407 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.407 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.407 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:40.407 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:52:40.407 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:40.407 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.407 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.407 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:40.407 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858050+0000", 2024-05-08T08:52:40.407 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5999999999999999e-07 2024-05-08T08:52:40.408 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.408 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.408 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:40.408 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858051+0000", 2024-05-08T08:52:40.408 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8870000000000001e-06 2024-05-08T08:52:40.408 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.408 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.408 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:40.408 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858057+0000", 2024-05-08T08:52:40.408 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8019999999999997e-06 2024-05-08T08:52:40.408 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.408 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.408 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:40.408 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858062+0000", 2024-05-08T08:52:40.408 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.561e-06 2024-05-08T08:52:40.409 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.409 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.409 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:40.409 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907289+0000", 2024-05-08T08:52:40.409 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049226957000000002 2024-05-08T08:52:40.409 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.409 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.409 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:40.409 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907356+0000", 2024-05-08T08:52:40.409 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6915000000000006e-05 2024-05-08T08:52:40.409 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.409 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.409 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:40.409 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907444+0000", 2024-05-08T08:52:40.409 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8127999999999994e-05 2024-05-08T08:52:40.409 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.410 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:40.410 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.410 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.410 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.410 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22107 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1044480~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:40.410 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:52:40.410 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 12.480109271, 2024-05-08T08:52:40.410 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049799942999999999, 2024-05-08T08:52:40.410 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:40.410 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:40.410 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:40.410 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:40.410 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:40.410 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:40.411 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22107 2024-05-08T08:52:40.411 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.411 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:40.411 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.411 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:40.411 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:52:40.411 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:40.411 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.411 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.411 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:40.411 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:52:40.411 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:40.411 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.411 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.411 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:40.411 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:52:40.412 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4799999999999999e-07 2024-05-08T08:52:40.412 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.412 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.412 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:40.412 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858070+0000", 2024-05-08T08:52:40.412 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.012e-06 2024-05-08T08:52:40.412 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.412 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.412 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:40.412 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858077+0000", 2024-05-08T08:52:40.412 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6290000000000001e-06 2024-05-08T08:52:40.412 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.412 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.412 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:40.412 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858083+0000", 2024-05-08T08:52:40.413 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8880000000000002e-06 2024-05-08T08:52:40.413 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.413 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.413 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:40.413 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907671+0000", 2024-05-08T08:52:40.413 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049587191000000003 2024-05-08T08:52:40.413 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.413 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.413 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:40.413 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907763+0000", 2024-05-08T08:52:40.413 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2231000000000005e-05 2024-05-08T08:52:40.413 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.413 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.413 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:40.413 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907867+0000", 2024-05-08T08:52:40.413 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104644 2024-05-08T08:52:40.414 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.414 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:40.414 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.414 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.414 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.414 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22294 3.1c 3:3aad0389:::rbd_data.11406886b07b.0000000000000080:head [stat,delete] snapc 7=[7] RETRY=6 ondisk+retry+write+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:40.414 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:52:40.414 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 12.479515423, 2024-05-08T08:52:40.414 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051152191, 2024-05-08T08:52:40.414 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:40.414 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:40.414 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:40.414 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:40.414 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:40.414 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:40.415 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22294 2024-05-08T08:52:40.415 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.415 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:40.415 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.415 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:40.415 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:52:40.415 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:40.415 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.415 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.415 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:40.415 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:52:40.415 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:40.415 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.415 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.415 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:40.416 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858662+0000", 2024-05-08T08:52:40.416 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4799999999999998e-07 2024-05-08T08:52:40.416 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.416 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.416 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:40.416 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858664+0000", 2024-05-08T08:52:40.416 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1229999999999998e-06 2024-05-08T08:52:40.416 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.416 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.416 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:40.416 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858670+0000", 2024-05-08T08:52:40.416 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3099999999999997e-06 2024-05-08T08:52:40.416 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.416 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.416 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:40.417 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858676+0000", 2024-05-08T08:52:40.417 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9750000000000004e-06 2024-05-08T08:52:40.417 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.417 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.417 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:40.417 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907901+0000", 2024-05-08T08:52:40.417 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049224520000000001 2024-05-08T08:52:40.417 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.417 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.417 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:40.417 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.908085+0000", 2024-05-08T08:52:40.417 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00018377500000000001 2024-05-08T08:52:40.417 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.418 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.418 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:40.418 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.909814+0000", 2024-05-08T08:52:40.418 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00172894 2024-05-08T08:52:40.418 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.418 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:40.418 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.418 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.418 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:40.418 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:52:40.420 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:52:40.421 INFO:teuthology.orchestra.run.smithi196.stdout:1263 read 0xa83800 thru 0xa87b92 (0x4393 bytes) 2024-05-08T08:52:40.421 INFO:teuthology.orchestra.run.smithi196.stdout:1264 read 0x4d5c600 thru 0x4d65425 (0x8e26 bytes) 2024-05-08T08:52:40.421 INFO:teuthology.orchestra.run.smithi196.stdout:1265 read 0x376f600 thru 0x3772b36 (0x3537 bytes) 2024-05-08T08:52:40.421 INFO:teuthology.orchestra.run.smithi196.stdout:1266 punch from 0x5ffc00 to 0x602800, (0x2c00 bytes) 2024-05-08T08:52:40.421 INFO:teuthology.orchestra.run.smithi196.stdout:1269 write 0xc6af800 thru 0xc6b0a32 (0x1233 bytes) 2024-05-08T08:52:40.421 INFO:teuthology.orchestra.run.smithi196.stdout:1270 trunc from 0xc6b0c00 to 0x171f800 2024-05-08T08:52:40.421 INFO:teuthology.orchestra.run.smithi196.stdout:1272 punch from 0x5b9400 to 0x5bc800, (0x3400 bytes) 2024-05-08T08:52:40.422 INFO:teuthology.orchestra.run.smithi196.stdout:1273 read 0x121a00 thru 0x126efd (0x54fe bytes) 2024-05-08T08:52:40.422 INFO:teuthology.orchestra.run.smithi196.stdout:1275 read 0x1141200 thru 0x1148c45 (0x7a46 bytes) 2024-05-08T08:52:40.422 INFO:teuthology.orchestra.run.smithi196.stdout:1279 read 0x331000 thru 0x33ab71 (0x9b72 bytes) 2024-05-08T08:52:40.422 INFO:teuthology.orchestra.run.smithi196.stdout:1280 punch from 0xd5f800 to 0xd6a200, (0xaa00 bytes) 2024-05-08T08:52:40.422 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:52:40.422+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:52:40.435 INFO:teuthology.orchestra.run.smithi196.stdout:1282 write 0x9fb3c00 thru 0x9fbcde1 (0x91e2 bytes) 2024-05-08T08:52:40.435 INFO:teuthology.orchestra.run.smithi196.stdout:1285 write 0x699b400 thru 0x69a409e (0x8c9f bytes) 2024-05-08T08:52:40.435 INFO:teuthology.orchestra.run.smithi196.stdout:1286 punch from 0x4926600 to 0x4927e00, (0x1800 bytes) 2024-05-08T08:52:40.440 INFO:teuthology.orchestra.run.smithi196.stdout:1287 read 0x798e00 thru 0x79b0d0 (0x22d1 bytes) 2024-05-08T08:52:40.454 INFO:teuthology.orchestra.run.smithi196.stdout:1290 write 0xe9aba00 thru 0xe9b145f (0x5a60 bytes) 2024-05-08T08:52:40.458 INFO:teuthology.orchestra.run.smithi196.stdout:1294 trunc from 0xe9b1600 to 0xbc14c00 2024-05-08T08:52:40.475 INFO:teuthology.orchestra.run.smithi196.stdout:1296 read 0x1344200 thru 0x1347152 (0x2f53 bytes) 2024-05-08T08:52:40.478 INFO:teuthology.orchestra.run.smithi196.stdout:1297 read 0x97fb400 thru 0x9800c10 (0x5811 bytes) 2024-05-08T08:52:40.478 INFO:teuthology.orchestra.run.smithi196.stdout:1299 write 0x7196c00 thru 0x71a4096 (0xd497 bytes) 2024-05-08T08:52:40.478 INFO:teuthology.orchestra.run.smithi196.stdout:1301 punch from 0x2b06e00 to 0x2b16c00, (0xfe00 bytes) 2024-05-08T08:52:40.479 INFO:teuthology.orchestra.run.smithi196.stdout:1302 write 0xa7c8e00 thru 0xa7d2dba (0x9fbb bytes) 2024-05-08T08:52:40.479 INFO:teuthology.orchestra.run.smithi196.stdout:1304 write 0x9a2d600 thru 0x9a32817 (0x5218 bytes) 2024-05-08T08:52:40.487 INFO:teuthology.orchestra.run.smithi196.stdout:1305 trunc from 0xbc14c00 to 0x4ab6e00 2024-05-08T08:52:40.516 INFO:teuthology.orchestra.run.smithi196.stdout:1306 read 0xe3da00 thru 0xe44f1b (0x751c bytes) 2024-05-08T08:52:40.516 INFO:teuthology.orchestra.run.smithi196.stdout:1307 read 0x45cb800 thru 0x45cf02a (0x382b bytes) 2024-05-08T08:52:40.516 INFO:teuthology.orchestra.run.smithi196.stdout:1312 read 0x1866a00 thru 0x186c484 (0x5a85 bytes) 2024-05-08T08:52:40.517 INFO:teuthology.orchestra.run.smithi196.stdout:1316 punch from 0x1fa6a00 to 0x1fb3a00, (0xd000 bytes) 2024-05-08T08:52:40.520 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:40.523+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:40.534 INFO:teuthology.orchestra.run.smithi196.stdout:1318 write 0xc233c00 thru 0xc236278 (0x2679 bytes) 2024-05-08T08:52:40.536 INFO:teuthology.orchestra.run.smithi196.stdout:1325 punch from 0xb040c00 to 0xb046200, (0x5600 bytes) 2024-05-08T08:52:40.537 INFO:teuthology.orchestra.run.smithi196.stdout:1327 read 0x3214e00 thru 0x321e749 (0x994a bytes) 2024-05-08T08:52:40.537 INFO:teuthology.orchestra.run.smithi196.stdout:1329 read 0x16bc200 thru 0x16bd962 (0x1763 bytes) 2024-05-08T08:52:40.538 INFO:teuthology.orchestra.run.smithi196.stdout:1330 read 0x8abca00 thru 0x8ac2cb5 (0x62b6 bytes) 2024-05-08T08:52:40.538 INFO:teuthology.orchestra.run.smithi196.stdout:1331 read 0x4171800 thru 0x417ff03 (0xe704 bytes) 2024-05-08T08:52:40.538 INFO:teuthology.orchestra.run.smithi196.stdout:1336 write 0x2438800 thru 0x243ac71 (0x2472 bytes) 2024-05-08T08:52:40.539 INFO:teuthology.orchestra.run.smithi196.stdout:1340 write 0x93f3c00 thru 0x93fa47b (0x687c bytes) 2024-05-08T08:52:40.539 INFO:teuthology.orchestra.run.smithi196.stdout:1341 write 0x3dd600 thru 0x3dd60a (0xb bytes) 2024-05-08T08:52:40.540 INFO:teuthology.orchestra.run.smithi196.stdout:1342 read 0x8e75200 thru 0x8e7920e (0x400f bytes) 2024-05-08T08:52:40.540 INFO:teuthology.orchestra.run.smithi196.stdout:1345 write 0x6226800 thru 0x62336d5 (0xced6 bytes) 2024-05-08T08:52:40.540 INFO:teuthology.orchestra.run.smithi196.stdout:1348 read 0x43a00 thru 0x4c456 (0x8a57 bytes) 2024-05-08T08:52:40.541 INFO:teuthology.orchestra.run.smithi196.stdout:1353 punch from 0x8984a00 to 0x8986800, (0x1e00 bytes) 2024-05-08T08:52:40.541 INFO:teuthology.orchestra.run.smithi196.stdout:1354 write 0x1cc6800 thru 0x1cd40e6 (0xd8e7 bytes) 2024-05-08T08:52:40.542 INFO:teuthology.orchestra.run.smithi196.stdout:1355 punch from 0x7488e00 to 0x748d200, (0x4400 bytes) 2024-05-08T08:52:40.542 INFO:teuthology.orchestra.run.smithi196.stdout:1357 read 0xa923e00 thru 0xa928d23 (0x4f24 bytes) 2024-05-08T08:52:40.542 INFO:teuthology.orchestra.run.smithi196.stdout:1358 read 0x630fa00 thru 0x631309d (0x369e bytes) 2024-05-08T08:52:40.542 INFO:teuthology.orchestra.run.smithi196.stdout:1359 write 0xbb03a00 thru 0xbb08514 (0x4b15 bytes) 2024-05-08T08:52:40.542 INFO:teuthology.orchestra.run.smithi196.stdout:1360 write 0x439b200 thru 0x43a566d (0xa46e bytes) 2024-05-08T08:52:40.543 INFO:teuthology.orchestra.run.smithi196.stdout:1362 punch from 0x9599000 to 0x959a800, (0x1800 bytes) 2024-05-08T08:52:40.543 INFO:teuthology.orchestra.run.smithi196.stdout:1363 read 0x576c600 thru 0x5774949 (0x834a bytes) 2024-05-08T08:52:40.543 INFO:teuthology.orchestra.run.smithi196.stdout:1365 punch from 0x6946e00 to 0x6954000, (0xd200 bytes) 2024-05-08T08:52:40.543 INFO:teuthology.orchestra.run.smithi196.stdout:1366 write 0x9a40200 thru 0x9a48fa6 (0x8da7 bytes) 2024-05-08T08:52:40.560 INFO:teuthology.orchestra.run.smithi196.stdout:1370 write 0xca74400 thru 0xca80dec (0xc9ed bytes) 2024-05-08T08:52:40.575 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:52:40.575 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:52:40.575 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:52:40.575 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:52:40.575 INFO:teuthology.orchestra.run.smithi196.stdout:1371 write 0xcde9000 thru 0xcdf22c5 (0x92c6 bytes) 2024-05-08T08:52:40.576 INFO:teuthology.orchestra.run.smithi196.stdout:1372 write 0x226e200 thru 0x2275d7b (0x7b7c bytes) 2024-05-08T08:52:40.576 INFO:teuthology.orchestra.run.smithi196.stdout:1373 read 0x1eace00 thru 0x1ebbf62 (0xf163 bytes) 2024-05-08T08:52:40.584 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:52:40.589 INFO:teuthology.orchestra.run.smithi196.stdout:1375 trunc from 0xcdf2400 to 0x261c400 2024-05-08T08:52:40.618 INFO:teuthology.orchestra.run.smithi196.stdout:1377 punch from 0x52b600 to 0x532e00, (0x7800 bytes) 2024-05-08T08:52:40.618 INFO:teuthology.orchestra.run.smithi196.stdout:1379 read 0xdc3800 thru 0xdcda3c (0xa23d bytes) 2024-05-08T08:52:40.618 INFO:teuthology.orchestra.run.smithi196.stdout:1380 punch from 0x1b4c400 to 0x1b50400, (0x4000 bytes) 2024-05-08T08:52:40.618 INFO:teuthology.orchestra.run.smithi196.stdout:1382 punch from 0x1fa600 to 0x201200, (0x6c00 bytes) 2024-05-08T08:52:40.619 INFO:teuthology.orchestra.run.smithi196.stdout:1385 read 0xaf8800 thru 0xb058af (0xd0b0 bytes) 2024-05-08T08:52:40.620 INFO:teuthology.orchestra.run.smithi196.stdout:1387 read 0xf4ca00 thru 0xf5807a (0xb67b bytes) 2024-05-08T08:52:40.625 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:40.628+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:40.651 INFO:teuthology.orchestra.run.smithi196.stdout:1389 write 0xea6d000 thru 0xea7bd80 (0xed81 bytes) 2024-05-08T08:52:40.651 INFO:teuthology.orchestra.run.smithi196.stdout:1392 write 0xe46a800 thru 0xe4794cd (0xecce bytes) 2024-05-08T08:52:40.651 INFO:teuthology.orchestra.run.smithi196.stdout:1400 punch from 0xdfff200 to 0xe001a00, (0x2800 bytes) 2024-05-08T08:52:40.652 INFO:teuthology.orchestra.run.smithi196.stdout:1401 read 0x3963e00 thru 0x397108d (0xd28e bytes) 2024-05-08T08:52:40.652 INFO:teuthology.orchestra.run.smithi196.stdout:1402 write 0x45b7200 thru 0x45b9497 (0x2298 bytes) 2024-05-08T08:52:40.652 INFO:teuthology.orchestra.run.smithi196.stdout:1404 read 0xb53400 thru 0xb567ba (0x33bb bytes) 2024-05-08T08:52:40.652 INFO:teuthology.orchestra.run.smithi196.stdout:1405 write 0xb2cf000 thru 0xb2db0a9 (0xc0aa bytes) 2024-05-08T08:52:40.657 INFO:teuthology.orchestra.run.smithi196.stdout:1406 trunc from 0xea7be00 to 0xa30da00 2024-05-08T08:52:40.676 INFO:teuthology.orchestra.run.smithi196.stdout:1408 read 0x6569800 thru 0x657836f (0xeb70 bytes) 2024-05-08T08:52:40.677 INFO:teuthology.orchestra.run.smithi196.stdout:1411 clone 9 order 21 su 8192 sc 5 2024-05-08T08:52:40.722 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:40.725+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:40.731 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:52:40.732 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:52:40.732 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:52:40.732 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:52:40.732 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:52:40.740 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:52:40.825 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:52:40.828+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:52:40.888 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:52:40.888 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:52:40.888 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:52:40.888 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:52:40.888 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.888 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6690 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1093632~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:40.888 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:52:40.888 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.1751134810000003, 2024-05-08T08:52:40.888 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052462754, 2024-05-08T08:52:40.888 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:40.889 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:40.889 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:40.889 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:40.889 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:40.889 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:40.889 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6690 2024-05-08T08:52:40.889 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.889 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:40.889 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.889 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:40.889 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:52:40.889 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:40.889 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.889 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.889 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:40.890 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:52:40.890 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:40.890 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.890 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.890 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:40.890 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715346+0000", 2024-05-08T08:52:40.890 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.84e-07 2024-05-08T08:52:40.890 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.890 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.890 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:40.890 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715348+0000", 2024-05-08T08:52:40.890 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.114e-06 2024-05-08T08:52:40.890 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.890 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.890 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:40.891 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715353+0000", 2024-05-08T08:52:40.891 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5720000000000002e-06 2024-05-08T08:52:40.891 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.891 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.891 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:40.891 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715358+0000", 2024-05-08T08:52:40.891 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7140000000000001e-06 2024-05-08T08:52:40.891 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.891 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.891 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:40.891 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767638+0000", 2024-05-08T08:52:40.891 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052279562000000002 2024-05-08T08:52:40.891 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.891 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.891 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:40.892 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767713+0000", 2024-05-08T08:52:40.892 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5652e-05 2024-05-08T08:52:40.892 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.892 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.892 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:40.892 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767808+0000", 2024-05-08T08:52:40.892 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4755999999999999e-05 2024-05-08T08:52:40.892 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.892 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:40.892 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.892 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.892 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.892 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6692 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1110016~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:40.892 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:52:40.893 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.175094391, 2024-05-08T08:52:40.893 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052666495000000001, 2024-05-08T08:52:40.893 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:40.893 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:40.893 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:40.893 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:40.893 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:40.893 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:40.893 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6692 2024-05-08T08:52:40.893 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.893 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:40.893 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.893 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:40.893 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:52:40.893 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:40.893 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.894 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.894 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:40.894 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:52:40.894 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:40.894 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.894 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.894 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:40.894 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:52:40.894 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8599999999999999e-07 2024-05-08T08:52:40.894 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.894 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.894 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:40.894 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715372+0000", 2024-05-08T08:52:40.894 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0990000000000001e-06 2024-05-08T08:52:40.894 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.894 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.895 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:40.895 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715378+0000", 2024-05-08T08:52:40.895 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0680000000000002e-06 2024-05-08T08:52:40.895 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.895 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.895 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:40.895 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715383+0000", 2024-05-08T08:52:40.895 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8740000000000003e-06 2024-05-08T08:52:40.895 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.895 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.895 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:40.895 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767848+0000", 2024-05-08T08:52:40.895 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052464668999999998 2024-05-08T08:52:40.895 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.895 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.895 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:40.896 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767927+0000", 2024-05-08T08:52:40.896 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9228999999999996e-05 2024-05-08T08:52:40.896 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.896 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.896 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:40.896 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768031+0000", 2024-05-08T08:52:40.896 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010417 2024-05-08T08:52:40.896 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.896 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:40.896 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.896 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.896 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.896 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6694 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1126400~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:40.896 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:52:40.896 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.1750694940000006, 2024-05-08T08:52:40.896 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052848829, 2024-05-08T08:52:40.897 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:40.897 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:40.897 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:40.897 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:40.897 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:40.897 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:40.897 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6694 2024-05-08T08:52:40.897 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.897 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:40.897 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.897 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:40.897 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:52:40.897 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:40.897 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.897 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.897 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:40.898 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:52:40.898 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:40.898 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.898 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.898 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:40.898 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715390+0000", 2024-05-08T08:52:40.898 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9900000000000001e-07 2024-05-08T08:52:40.898 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.898 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.898 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:40.898 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715392+0000", 2024-05-08T08:52:40.898 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8279999999999999e-06 2024-05-08T08:52:40.898 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.898 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.898 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:40.899 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715398+0000", 2024-05-08T08:52:40.899 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7619999999999998e-06 2024-05-08T08:52:40.899 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.899 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.899 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:40.899 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715403+0000", 2024-05-08T08:52:40.899 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7790000000000003e-06 2024-05-08T08:52:40.899 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.899 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.899 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:40.899 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768067+0000", 2024-05-08T08:52:40.899 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052663785999999997 2024-05-08T08:52:40.899 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.899 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.900 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:40.900 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768142+0000", 2024-05-08T08:52:40.900 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5018999999999994e-05 2024-05-08T08:52:40.900 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.900 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.900 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:40.900 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768238+0000", 2024-05-08T08:52:40.900 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6255999999999995e-05 2024-05-08T08:52:40.900 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.900 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:40.900 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.900 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.900 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.900 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6696 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1142784~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:40.900 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:52:40.900 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.1750499859999994, 2024-05-08T08:52:40.901 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.053051057999999998, 2024-05-08T08:52:40.901 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:40.901 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:40.901 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:40.901 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:40.901 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:40.901 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:40.901 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6696 2024-05-08T08:52:40.901 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.901 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:40.901 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.901 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:40.901 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:52:40.901 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:40.901 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.901 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.902 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:40.902 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:52:40.902 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:40.902 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.902 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.902 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:40.902 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:52:40.902 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-05-08T08:52:40.902 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.902 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.902 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:40.902 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715411+0000", 2024-05-08T08:52:40.902 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9630000000000001e-06 2024-05-08T08:52:40.902 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.903 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.903 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:40.903 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715417+0000", 2024-05-08T08:52:40.903 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7250000000000002e-06 2024-05-08T08:52:40.903 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.903 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.903 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:40.903 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715421+0000", 2024-05-08T08:52:40.903 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4089999999999996e-06 2024-05-08T08:52:40.903 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.903 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.903 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:40.903 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768277+0000", 2024-05-08T08:52:40.903 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052855293999999997 2024-05-08T08:52:40.903 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.903 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.904 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:40.904 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768358+0000", 2024-05-08T08:52:40.904 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1720999999999999e-05 2024-05-08T08:52:40.904 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.904 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.904 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:40.904 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768460+0000", 2024-05-08T08:52:40.904 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010154600000000001 2024-05-08T08:52:40.904 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.904 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:40.904 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.904 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.904 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.904 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6698 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1159168~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:40.904 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:52:40.905 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.1750311189999998, 2024-05-08T08:52:40.905 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.053245726, 2024-05-08T08:52:40.905 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:40.905 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:40.905 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:40.905 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:40.905 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:40.905 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:40.905 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6698 2024-05-08T08:52:40.905 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.905 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:40.905 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.905 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:40.905 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:52:40.905 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:40.905 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.906 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.906 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:40.906 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:52:40.906 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:40.906 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.906 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.906 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:40.906 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716936+0000", 2024-05-08T08:52:40.906 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001508665 2024-05-08T08:52:40.906 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.906 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.906 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:40.906 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716941+0000", 2024-05-08T08:52:40.906 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1910000000000002e-06 2024-05-08T08:52:40.907 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.907 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.907 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:40.907 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716950+0000", 2024-05-08T08:52:40.907 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1900000000000001e-06 2024-05-08T08:52:40.907 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.907 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.907 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:40.907 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716958+0000", 2024-05-08T08:52:40.907 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8019999999999992e-06 2024-05-08T08:52:40.907 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.907 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.907 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:40.907 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768499+0000", 2024-05-08T08:52:40.907 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051540926000000001 2024-05-08T08:52:40.907 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.908 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.908 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:40.908 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768574+0000", 2024-05-08T08:52:40.908 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4988999999999996e-05 2024-05-08T08:52:40.908 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.908 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.908 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:40.908 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768673+0000", 2024-05-08T08:52:40.908 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9963000000000005e-05 2024-05-08T08:52:40.908 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.908 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:40.908 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.908 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.908 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.908 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6706 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1224704~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:40.908 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:52:40.909 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.1734258119999996, 2024-05-08T08:52:40.909 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052435598, 2024-05-08T08:52:40.909 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:40.909 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:40.909 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:40.909 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:40.909 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:40.909 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:40.909 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6706 2024-05-08T08:52:40.909 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.909 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:40.909 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.909 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:40.909 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:52:40.909 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:40.909 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.910 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.910 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:40.910 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:52:40.910 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:40.910 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.910 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.910 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:40.910 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:52:40.910 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:52:40.910 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.910 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.910 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:40.910 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717035+0000", 2024-05-08T08:52:40.910 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9290000000000001e-06 2024-05-08T08:52:40.911 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.911 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.911 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:40.911 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717041+0000", 2024-05-08T08:52:40.911 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6720000000000001e-06 2024-05-08T08:52:40.911 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.911 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.911 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:40.911 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717046+0000", 2024-05-08T08:52:40.911 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5650000000000003e-06 2024-05-08T08:52:40.911 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.911 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.911 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:40.911 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769311+0000", 2024-05-08T08:52:40.911 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052265645999999999 2024-05-08T08:52:40.912 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.912 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.912 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:40.912 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769379+0000", 2024-05-08T08:52:40.912 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7391000000000006e-05 2024-05-08T08:52:40.912 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.912 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.912 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:40.912 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769469+0000", 2024-05-08T08:52:40.912 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0025999999999995e-05 2024-05-08T08:52:40.912 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.912 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:40.912 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.912 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.912 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.912 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6708 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1241088~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:40.913 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:52:40.913 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.1734077079999992, 2024-05-08T08:52:40.913 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052640552, 2024-05-08T08:52:40.913 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:40.913 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:40.913 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:40.913 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:40.913 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:40.913 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:40.913 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6708 2024-05-08T08:52:40.913 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.913 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:40.913 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.913 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:40.913 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:52:40.914 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:40.914 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.914 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.914 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:40.914 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:52:40.914 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:40.914 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.914 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.914 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:40.914 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717052+0000", 2024-05-08T08:52:40.914 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:52:40.914 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.914 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.914 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:40.915 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717053+0000", 2024-05-08T08:52:40.915 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8330000000000001e-06 2024-05-08T08:52:40.915 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.915 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.915 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:40.915 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717058+0000", 2024-05-08T08:52:40.915 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.933e-06 2024-05-08T08:52:40.915 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.915 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.915 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:40.915 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717063+0000", 2024-05-08T08:52:40.915 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2180000000000001e-06 2024-05-08T08:52:40.915 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.915 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.915 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:40.915 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769507+0000", 2024-05-08T08:52:40.916 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052444866999999999 2024-05-08T08:52:40.916 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.916 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.916 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:40.916 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769589+0000", 2024-05-08T08:52:40.916 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2039000000000005e-05 2024-05-08T08:52:40.916 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.916 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.916 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:40.916 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769692+0000", 2024-05-08T08:52:40.916 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102252 2024-05-08T08:52:40.916 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.916 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:40.916 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.917 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.917 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.917 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6710 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1257472~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:40.917 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:52:40.917 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.173389684, 2024-05-08T08:52:40.917 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.05283057, 2024-05-08T08:52:40.917 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:40.917 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:40.917 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:40.917 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:40.917 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:40.917 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:40.917 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6710 2024-05-08T08:52:40.917 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.917 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:40.918 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.918 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:40.918 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:52:40.918 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:40.918 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.918 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.918 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:40.918 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:52:40.918 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:40.918 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.918 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.918 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:40.918 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718951+0000", 2024-05-08T08:52:40.918 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0018817809999999999 2024-05-08T08:52:40.919 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.919 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.919 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:40.919 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718956+0000", 2024-05-08T08:52:40.919 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5399999999999997e-06 2024-05-08T08:52:40.919 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.919 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.919 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:40.919 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718964+0000", 2024-05-08T08:52:40.919 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6629999999999999e-06 2024-05-08T08:52:40.919 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.919 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.919 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:40.919 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718972+0000", 2024-05-08T08:52:40.919 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5229999999999998e-06 2024-05-08T08:52:40.920 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.920 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.920 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:40.920 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769729+0000", 2024-05-08T08:52:40.920 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050756959999999997 2024-05-08T08:52:40.920 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.920 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.920 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:40.920 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769803+0000", 2024-05-08T08:52:40.920 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4538999999999999e-05 2024-05-08T08:52:40.920 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.920 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.920 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:40.920 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769900+0000", 2024-05-08T08:52:40.920 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6564000000000006e-05 2024-05-08T08:52:40.920 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.921 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:40.921 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.921 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.921 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.921 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6726 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1388544~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:40.921 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:52:40.921 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.1713149119999997, 2024-05-08T08:52:40.921 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052451506000000002, 2024-05-08T08:52:40.921 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:40.921 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:40.921 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:40.921 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:40.921 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:40.921 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:40.921 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6726 2024-05-08T08:52:40.921 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.921 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:40.922 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.922 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:40.922 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:52:40.922 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:40.922 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.922 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.922 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:40.922 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:52:40.922 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:40.922 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.922 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.922 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:40.922 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719200+0000", 2024-05-08T08:52:40.922 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5865999999999997e-05 2024-05-08T08:52:40.923 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.923 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.923 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:40.923 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719204+0000", 2024-05-08T08:52:40.923 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8630000000000001e-06 2024-05-08T08:52:40.923 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.923 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.923 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:40.923 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719215+0000", 2024-05-08T08:52:40.923 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1590000000000001e-05 2024-05-08T08:52:40.923 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.923 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.923 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:40.923 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719220+0000", 2024-05-08T08:52:40.923 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1619999999999999e-06 2024-05-08T08:52:40.923 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.924 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.924 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:40.924 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771425+0000", 2024-05-08T08:52:40.924 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052204249000000001 2024-05-08T08:52:40.924 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.924 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.924 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:40.924 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771498+0000", 2024-05-08T08:52:40.924 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3782999999999995e-05 2024-05-08T08:52:40.924 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.924 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.924 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:40.924 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771595+0000", 2024-05-08T08:52:40.924 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6992999999999998e-05 2024-05-08T08:52:40.924 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.924 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:40.925 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.925 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.925 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.925 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6728 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1404928~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:40.925 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:52:40.925 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.1712275069999993, 2024-05-08T08:52:40.925 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052590747, 2024-05-08T08:52:40.925 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:40.925 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:40.925 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:40.925 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:40.925 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:40.925 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:40.925 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6728 2024-05-08T08:52:40.925 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.925 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:40.925 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.926 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:40.926 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:52:40.926 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:40.926 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.926 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.926 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:40.926 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:52:40.926 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:40.926 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.926 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.926 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:40.926 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719232+0000", 2024-05-08T08:52:40.926 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4000000000000001e-07 2024-05-08T08:52:40.926 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.927 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.927 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:40.927 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719234+0000", 2024-05-08T08:52:40.927 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4499999999999998e-06 2024-05-08T08:52:40.927 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.927 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.927 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:40.927 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719240+0000", 2024-05-08T08:52:40.927 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5720000000000002e-06 2024-05-08T08:52:40.927 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.927 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.927 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:40.927 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719245+0000", 2024-05-08T08:52:40.927 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8690000000000003e-06 2024-05-08T08:52:40.927 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.927 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.928 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:40.928 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771636+0000", 2024-05-08T08:52:40.928 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052390979999999997 2024-05-08T08:52:40.928 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.928 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.928 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:40.928 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771717+0000", 2024-05-08T08:52:40.928 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1291999999999993e-05 2024-05-08T08:52:40.928 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.928 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.928 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:40.928 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771822+0000", 2024-05-08T08:52:40.928 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104944 2024-05-08T08:52:40.928 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.928 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:40.928 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.929 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.929 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.929 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6730 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1421312~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:40.929 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:52:40.929 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.1712072649999996, 2024-05-08T08:52:40.929 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052770477000000003, 2024-05-08T08:52:40.929 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:40.929 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:40.929 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:40.929 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:40.929 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:40.929 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:40.929 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6730 2024-05-08T08:52:40.929 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.929 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:40.929 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.930 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:40.930 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:52:40.930 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:40.930 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.930 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.930 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:40.930 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:52:40.930 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:40.930 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.930 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.930 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:40.930 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:52:40.930 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9000000000000002e-07 2024-05-08T08:52:40.930 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.930 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.931 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:40.931 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719254+0000", 2024-05-08T08:52:40.931 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4099999999999998e-06 2024-05-08T08:52:40.931 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.931 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.931 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:40.931 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719263+0000", 2024-05-08T08:52:40.931 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4109999999999996e-06 2024-05-08T08:52:40.931 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.931 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.931 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:40.931 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719268+0000", 2024-05-08T08:52:40.931 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2669999999999997e-06 2024-05-08T08:52:40.931 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.931 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.932 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:40.932 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771846+0000", 2024-05-08T08:52:40.932 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052577677000000003 2024-05-08T08:52:40.932 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.932 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.932 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:40.932 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771920+0000", 2024-05-08T08:52:40.932 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4460000000000002e-05 2024-05-08T08:52:40.932 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.932 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.932 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:40.932 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772022+0000", 2024-05-08T08:52:40.932 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010186199999999999 2024-05-08T08:52:40.932 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.932 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:40.932 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.933 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.933 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.933 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6732 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1437696~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:40.933 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:52:40.933 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.1711824580000005, 2024-05-08T08:52:40.933 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052905575000000003, 2024-05-08T08:52:40.933 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:40.933 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:40.933 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:40.933 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:40.933 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:40.933 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:40.933 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6732 2024-05-08T08:52:40.933 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.933 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:40.934 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.934 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:40.934 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:52:40.934 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:40.934 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.934 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.934 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:40.934 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:52:40.934 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:40.934 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.934 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.934 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:40.934 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721392+0000", 2024-05-08T08:52:40.934 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002115287 2024-05-08T08:52:40.934 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.935 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.935 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:40.935 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721396+0000", 2024-05-08T08:52:40.935 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1679999999999997e-06 2024-05-08T08:52:40.935 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.935 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.935 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:40.935 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721404+0000", 2024-05-08T08:52:40.935 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4409999999999993e-06 2024-05-08T08:52:40.935 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.935 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.935 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:40.935 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721410+0000", 2024-05-08T08:52:40.935 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9150000000000001e-06 2024-05-08T08:52:40.935 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.935 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.936 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:40.936 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772061+0000", 2024-05-08T08:52:40.936 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050651068 2024-05-08T08:52:40.936 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.936 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.936 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:40.936 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772127+0000", 2024-05-08T08:52:40.936 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5424000000000002e-05 2024-05-08T08:52:40.936 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.936 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.936 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:40.936 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772182+0000", 2024-05-08T08:52:40.936 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5272000000000002e-05 2024-05-08T08:52:40.936 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.936 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:40.937 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.937 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.937 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.937 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6749 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1585152~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:40.937 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:52:40.937 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.1688816880000008, 2024-05-08T08:52:40.937 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052350264000000001, 2024-05-08T08:52:40.937 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:40.937 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:40.937 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:40.937 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:40.937 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:40.937 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:40.937 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6749 2024-05-08T08:52:40.937 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.938 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:40.938 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.938 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:40.938 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:52:40.938 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:40.938 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.938 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.938 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:40.938 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:52:40.938 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:40.938 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.938 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.938 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:40.938 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721578+0000", 2024-05-08T08:52:40.939 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0399999999999996e-07 2024-05-08T08:52:40.939 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.939 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.939 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:40.939 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721580+0000", 2024-05-08T08:52:40.939 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5450000000000002e-06 2024-05-08T08:52:40.939 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.939 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.939 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:40.939 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721587+0000", 2024-05-08T08:52:40.939 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0190000000000001e-06 2024-05-08T08:52:40.939 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.939 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.939 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:40.939 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721593+0000", 2024-05-08T08:52:40.940 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5659999999999997e-06 2024-05-08T08:52:40.940 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.940 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.940 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:40.940 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773759+0000", 2024-05-08T08:52:40.940 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052166553999999997 2024-05-08T08:52:40.940 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.940 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.940 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:40.940 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773833+0000", 2024-05-08T08:52:40.940 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3913999999999997e-05 2024-05-08T08:52:40.940 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.940 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.940 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:40.940 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773927+0000", 2024-05-08T08:52:40.940 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4061999999999995e-05 2024-05-08T08:52:40.941 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.941 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:40.941 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.941 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.941 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.941 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6751 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1601536~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:40.941 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:52:40.941 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.1688584090000003, 2024-05-08T08:52:40.941 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052461302000000001, 2024-05-08T08:52:40.941 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:40.941 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:40.941 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:40.941 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:40.941 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:40.941 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:40.941 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6751 2024-05-08T08:52:40.942 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.942 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:40.942 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.942 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:40.942 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:52:40.942 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:40.942 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.942 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.942 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:40.942 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:52:40.942 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:40.942 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.942 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.942 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:40.943 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722939+0000", 2024-05-08T08:52:40.943 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001338601 2024-05-08T08:52:40.943 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.943 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.943 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:40.943 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722944+0000", 2024-05-08T08:52:40.943 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4469999999999999e-06 2024-05-08T08:52:40.943 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.943 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.943 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:40.943 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722955+0000", 2024-05-08T08:52:40.943 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1892999999999999e-05 2024-05-08T08:52:40.943 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.943 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.943 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:40.944 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722963+0000", 2024-05-08T08:52:40.944 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1080000000000004e-06 2024-05-08T08:52:40.944 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.944 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.944 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:40.944 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773964+0000", 2024-05-08T08:52:40.944 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051001022 2024-05-08T08:52:40.944 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.944 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.944 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:40.944 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.774006+0000", 2024-05-08T08:52:40.944 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2729999999999999e-05 2024-05-08T08:52:40.944 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.944 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.944 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:40.945 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.774062+0000", 2024-05-08T08:52:40.945 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5501000000000003e-05 2024-05-08T08:52:40.945 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.945 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:40.945 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.945 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.945 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.945 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6760 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1683456~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:40.945 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:52:40.945 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.1673490330000007, 2024-05-08T08:52:40.945 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052571312000000002, 2024-05-08T08:52:40.945 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:40.945 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:40.945 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:40.945 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:40.946 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:40.946 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:40.946 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6760 2024-05-08T08:52:40.946 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.946 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:40.946 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.946 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:40.946 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:52:40.946 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:40.946 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.946 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.946 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:40.946 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:52:40.946 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:40.946 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.946 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.947 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:40.947 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:52:40.947 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2100000000000002e-07 2024-05-08T08:52:40.947 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.947 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.947 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:40.947 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723113+0000", 2024-05-08T08:52:40.947 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3149999999999999e-06 2024-05-08T08:52:40.947 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.947 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.947 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:40.947 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723118+0000", 2024-05-08T08:52:40.947 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8860000000000001e-06 2024-05-08T08:52:40.947 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.947 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.948 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:40.948 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723123+0000", 2024-05-08T08:52:40.948 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6770000000000003e-06 2024-05-08T08:52:40.948 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.948 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.948 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:40.948 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775414+0000", 2024-05-08T08:52:40.948 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052290692999999999 2024-05-08T08:52:40.948 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.948 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.948 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:40.948 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775459+0000", 2024-05-08T08:52:40.948 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5639999999999997e-05 2024-05-08T08:52:40.948 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.948 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.948 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:40.949 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775681+0000", 2024-05-08T08:52:40.949 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00022168000000000001 2024-05-08T08:52:40.949 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.949 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:40.949 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.949 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.949 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.949 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6761 3.16 3:6bd6d6da:::rbd_data.1140782590a5.0000000000000015:6 [read 1095680~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:40.949 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:52:40.949 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.1673298130000003, 2024-05-08T08:52:40.949 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052751275, 2024-05-08T08:52:40.949 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:40.949 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:40.949 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:40.949 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:40.949 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:40.949 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:40.950 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6761 2024-05-08T08:52:40.950 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.950 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:40.950 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.950 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:40.950 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:52:40.950 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:40.950 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.950 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.950 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:40.950 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:52:40.950 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:40.950 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.950 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.950 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:40.951 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724939+0000", 2024-05-08T08:52:40.951 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001809817 2024-05-08T08:52:40.951 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.951 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.951 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:40.951 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724943+0000", 2024-05-08T08:52:40.951 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4920000000000004e-06 2024-05-08T08:52:40.951 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.951 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.951 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:40.951 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724956+0000", 2024-05-08T08:52:40.951 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2427e-05 2024-05-08T08:52:40.951 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.951 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.951 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:40.952 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724963+0000", 2024-05-08T08:52:40.952 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2559999999999996e-06 2024-05-08T08:52:40.952 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.952 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.952 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:40.952 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775711+0000", 2024-05-08T08:52:40.952 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050747621 2024-05-08T08:52:40.952 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.952 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.952 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:40.952 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775785+0000", 2024-05-08T08:52:40.952 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4634000000000004e-05 2024-05-08T08:52:40.952 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.952 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.952 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:40.952 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775880+0000", 2024-05-08T08:52:40.953 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5027999999999999e-05 2024-05-08T08:52:40.953 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.953 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:40.953 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.953 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.953 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.953 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6770 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1765376~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:40.953 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:52:40.953 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.1652987180000007, 2024-05-08T08:52:40.953 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052339296, 2024-05-08T08:52:40.953 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:40.953 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:40.953 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:40.953 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:40.953 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:40.954 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:40.954 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6770 2024-05-08T08:52:40.954 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.954 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:40.954 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.954 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:40.954 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:52:40.954 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:40.954 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.954 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.954 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:40.954 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:52:40.954 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:40.954 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.955 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.955 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:40.955 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725161+0000", 2024-05-08T08:52:40.955 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-05-08T08:52:40.955 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.955 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.955 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:40.955 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725163+0000", 2024-05-08T08:52:40.955 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4370000000000001e-06 2024-05-08T08:52:40.955 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.955 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.955 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:40.955 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725169+0000", 2024-05-08T08:52:40.955 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8409999999999996e-06 2024-05-08T08:52:40.955 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.955 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.955 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:40.956 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725174+0000", 2024-05-08T08:52:40.956 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7249999999999997e-06 2024-05-08T08:52:40.956 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.956 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.956 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:40.956 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777343+0000", 2024-05-08T08:52:40.956 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052169633999999999 2024-05-08T08:52:40.956 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.956 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.956 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:40.956 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777409+0000", 2024-05-08T08:52:40.956 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6033999999999998e-05 2024-05-08T08:52:40.956 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.956 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.956 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:40.957 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777499+0000", 2024-05-08T08:52:40.957 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0224999999999998e-05 2024-05-08T08:52:40.957 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.957 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:40.957 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.957 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.957 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.957 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6771 3.16 3:6bd6d6da:::rbd_data.1140782590a5.0000000000000015:6 [read 1177600~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:40.957 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:52:40.957 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.1652789010000006, 2024-05-08T08:52:40.957 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052438604999999999, 2024-05-08T08:52:40.957 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:40.957 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:40.957 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:40.957 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:40.957 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:40.958 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:40.958 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6771 2024-05-08T08:52:40.958 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.958 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:40.958 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.958 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:40.958 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:52:40.958 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:40.958 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.958 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.958 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:40.958 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:52:40.958 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:40.958 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.958 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.958 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:40.959 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:52:40.959 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:52:40.959 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.959 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.959 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:40.959 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725188+0000", 2024-05-08T08:52:40.959 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1579999999999999e-06 2024-05-08T08:52:40.959 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.959 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.959 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:40.959 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725195+0000", 2024-05-08T08:52:40.959 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8909999999999995e-06 2024-05-08T08:52:40.959 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.959 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.959 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:40.959 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725200+0000", 2024-05-08T08:52:40.960 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9810000000000003e-06 2024-05-08T08:52:40.960 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.960 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.960 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:40.960 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777527+0000", 2024-05-08T08:52:40.960 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052326582000000003 2024-05-08T08:52:40.960 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.960 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.960 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:40.960 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777566+0000", 2024-05-08T08:52:40.960 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9447999999999999e-05 2024-05-08T08:52:40.960 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.960 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.960 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:40.960 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777619+0000", 2024-05-08T08:52:40.960 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2135000000000001e-05 2024-05-08T08:52:40.961 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.961 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:40.961 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.961 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.961 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.961 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6772 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1781760~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:40.961 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:52:40.961 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.1652508439999991, 2024-05-08T08:52:40.961 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052601799999999997, 2024-05-08T08:52:40.961 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:40.961 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:40.961 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:40.961 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:40.961 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:40.961 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:40.962 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6772 2024-05-08T08:52:40.962 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.962 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:40.962 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.962 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:40.962 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:52:40.962 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:40.962 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.962 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.962 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:40.962 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:52:40.962 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:40.962 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.962 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.962 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:40.962 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726939+0000", 2024-05-08T08:52:40.963 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001730493 2024-05-08T08:52:40.963 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.963 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.963 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:40.963 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726943+0000", 2024-05-08T08:52:40.963 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7330000000000003e-06 2024-05-08T08:52:40.963 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.963 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.963 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:40.963 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726954+0000", 2024-05-08T08:52:40.963 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1107000000000001e-05 2024-05-08T08:52:40.963 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.963 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.963 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:40.963 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726962+0000", 2024-05-08T08:52:40.964 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.503e-06 2024-05-08T08:52:40.964 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.964 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.964 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:40.964 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777653+0000", 2024-05-08T08:52:40.964 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050691542999999999 2024-05-08T08:52:40.964 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.964 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.964 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:40.964 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777719+0000", 2024-05-08T08:52:40.964 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5228000000000006e-05 2024-05-08T08:52:40.964 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.964 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.964 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:40.964 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777810+0000", 2024-05-08T08:52:40.965 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1193000000000006e-05 2024-05-08T08:52:40.965 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.965 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:40.965 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.965 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.965 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.965 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6782 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1880064~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:40.965 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:52:40.965 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.1632823370000001, 2024-05-08T08:52:40.965 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052326492000000002, 2024-05-08T08:52:40.965 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:40.965 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:40.965 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:40.965 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:40.965 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:40.966 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:40.966 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6782 2024-05-08T08:52:40.966 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.966 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:40.966 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.966 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:40.966 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:52:40.966 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:40.966 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.966 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.966 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:40.966 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:52:40.966 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:40.966 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.966 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.966 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:40.967 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728939+0000", 2024-05-08T08:52:40.967 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001762719 2024-05-08T08:52:40.967 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.967 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.967 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:40.967 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728944+0000", 2024-05-08T08:52:40.967 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.746e-06 2024-05-08T08:52:40.967 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.967 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.967 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:40.967 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728955+0000", 2024-05-08T08:52:40.967 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1043000000000001e-05 2024-05-08T08:52:40.967 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.967 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.967 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:40.968 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728963+0000", 2024-05-08T08:52:40.968 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5809999999999999e-06 2024-05-08T08:52:40.968 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.968 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.968 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:40.968 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779320+0000", 2024-05-08T08:52:40.968 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050357290999999998 2024-05-08T08:52:40.968 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.968 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.968 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:40.968 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779401+0000", 2024-05-08T08:52:40.968 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0574000000000005e-05 2024-05-08T08:52:40.968 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:40.968 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:40.968 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:40.968 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779503+0000", 2024-05-08T08:52:40.969 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102538 2024-05-08T08:52:40.969 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.969 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:40.969 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.969 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:40.969 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:40.969 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:52:40.970 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:52:40.971 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:52:40.973+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:52:41.026 INFO:teuthology.orchestra.run.smithi059.stderr:noscrub is unset 2024-05-08T08:52:41.070 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:52:41.074+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:52:41.090 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:52:41.090 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:52:41.090 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:52:41.090 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:52:41.099 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:52:41.185 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:52:41.188+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:52:41.252 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:52:41.252 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:52:41.252 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:52:41.252 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:52:41.252 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:52:41.262 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:52:41.272 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:41.276+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:41.373 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:52:41.377+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:52:41.420 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:52:41.421 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:52:41.421 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:52:41.421 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:52:41.421 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.421 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4621.0:9089 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 23339520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:41.421 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:52:41.421 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 8.9300633919999992, 2024-05-08T08:52:41.421 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.13814907200000001, 2024-05-08T08:52:41.421 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:41.421 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:41.421 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:41.421 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:41.421 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4621", 2024-05-08T08:52:41.421 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:41.421 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9089 2024-05-08T08:52:41.422 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.422 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:41.422 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.422 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:41.422 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:52:41.422 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:41.422 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.422 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.422 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:41.422 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:52:41.422 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:41.422 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.422 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.423 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:41.423 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493005+0000", 2024-05-08T08:52:41.423 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1700000000000001e-06 2024-05-08T08:52:41.423 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.423 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.423 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:41.423 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.513963+0000", 2024-05-08T08:52:41.423 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.020957581999999999 2024-05-08T08:52:41.423 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.423 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.423 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:41.423 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.513995+0000", 2024-05-08T08:52:41.423 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1993999999999999e-05 2024-05-08T08:52:41.423 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.423 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.423 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:41.424 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514007+0000", 2024-05-08T08:52:41.424 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1581e-05 2024-05-08T08:52:41.424 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.424 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.424 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:41.424 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514122+0000", 2024-05-08T08:52:41.424 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000115282 2024-05-08T08:52:41.424 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.424 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.424 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op must wait for map", 2024-05-08T08:52:41.424 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514128+0000", 2024-05-08T08:52:41.424 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9950000000000002e-06 2024-05-08T08:52:41.424 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.424 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.425 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:41.425 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.515800+0000", 2024-05-08T08:52:41.425 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001671886 2024-05-08T08:52:41.425 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.425 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.425 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:41.425 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.515893+0000", 2024-05-08T08:52:41.425 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3490000000000001e-05 2024-05-08T08:52:41.425 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.425 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.425 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 3", 2024-05-08T08:52:41.425 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.517647+0000", 2024-05-08T08:52:41.425 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0017535630000000001 2024-05-08T08:52:41.425 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.425 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.426 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:41.426 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.627578+0000", 2024-05-08T08:52:41.426 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.109931557 2024-05-08T08:52:41.426 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.426 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.426 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:41.426 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631055+0000", 2024-05-08T08:52:41.426 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0034761420000000002 2024-05-08T08:52:41.426 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.426 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.426 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:41.426 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631096+0000", 2024-05-08T08:52:41.426 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1059000000000001e-05 2024-05-08T08:52:41.427 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.427 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.427 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:41.427 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631151+0000", 2024-05-08T08:52:41.427 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5770999999999998e-05 2024-05-08T08:52:41.427 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:41.427 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:41.427 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:41.427 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.427 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.427 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1062 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 10903552~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:41.427 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485389+0000", 2024-05-08T08:52:41.427 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 4.9376770429999999, 2024-05-08T08:52:41.427 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070031261999999997, 2024-05-08T08:52:41.428 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:41.428 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:41.428 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:41.428 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:41.428 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:41.428 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:41.428 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1062 2024-05-08T08:52:41.428 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.428 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:41.428 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.428 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:41.428 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485389+0000", 2024-05-08T08:52:41.428 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:41.428 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.428 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.429 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:41.429 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485389+0000", 2024-05-08T08:52:41.429 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:41.429 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.429 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.429 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:41.429 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485389+0000", 2024-05-08T08:52:41.429 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.51e-07 2024-05-08T08:52:41.429 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.429 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.429 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:41.429 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485392+0000", 2024-05-08T08:52:41.429 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.593e-06 2024-05-08T08:52:41.429 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.429 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.430 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:41.430 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485398+0000", 2024-05-08T08:52:41.430 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.793e-06 2024-05-08T08:52:41.430 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.430 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.430 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:41.430 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485404+0000", 2024-05-08T08:52:41.430 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0669999999999999e-06 2024-05-08T08:52:41.430 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.430 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.430 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:41.430 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555214+0000", 2024-05-08T08:52:41.430 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.069810935000000005 2024-05-08T08:52:41.430 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.430 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.431 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:41.431 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555306+0000", 2024-05-08T08:52:41.431 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1939000000000002e-05 2024-05-08T08:52:41.431 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.431 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.431 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:41.431 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555420+0000", 2024-05-08T08:52:41.431 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000113484 2024-05-08T08:52:41.431 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:41.431 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:41.431 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:41.431 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.431 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.431 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1065 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 10944512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:41.431 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485411+0000", 2024-05-08T08:52:41.432 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 4.937654706, 2024-05-08T08:52:41.432 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070281386000000001, 2024-05-08T08:52:41.432 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:41.432 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:41.432 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:41.432 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:41.432 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:41.432 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:41.432 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1065 2024-05-08T08:52:41.432 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.432 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:41.432 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.432 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:41.432 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485411+0000", 2024-05-08T08:52:41.432 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:41.432 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.433 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.433 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:41.433 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485411+0000", 2024-05-08T08:52:41.433 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:41.433 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.433 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.433 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:41.433 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485411+0000", 2024-05-08T08:52:41.433 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5600000000000001e-07 2024-05-08T08:52:41.433 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.433 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.433 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:41.433 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485414+0000", 2024-05-08T08:52:41.433 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2299999999999998e-06 2024-05-08T08:52:41.433 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.433 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.434 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:41.434 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485420+0000", 2024-05-08T08:52:41.434 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2179999999999996e-06 2024-05-08T08:52:41.434 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.434 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.434 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:41.434 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485425+0000", 2024-05-08T08:52:41.434 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9830000000000004e-06 2024-05-08T08:52:41.434 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.434 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.434 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:41.434 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555468+0000", 2024-05-08T08:52:41.434 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070043541000000001 2024-05-08T08:52:41.434 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.434 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.435 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:41.435 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555566+0000", 2024-05-08T08:52:41.435 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8082e-05 2024-05-08T08:52:41.435 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.435 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.435 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:41.435 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555692+0000", 2024-05-08T08:52:41.435 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000125976 2024-05-08T08:52:41.435 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:41.435 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:41.435 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:41.435 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.435 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.436 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1067 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 10985472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:41.436 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485431+0000", 2024-05-08T08:52:41.436 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 4.9376345710000002, 2024-05-08T08:52:41.436 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070412460999999996, 2024-05-08T08:52:41.436 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:41.436 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:41.436 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:41.436 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:41.436 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:41.436 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:41.436 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1067 2024-05-08T08:52:41.436 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.436 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:41.436 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.437 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:41.437 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485431+0000", 2024-05-08T08:52:41.437 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:41.437 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.437 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.437 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:41.437 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485431+0000", 2024-05-08T08:52:41.437 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:41.437 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.437 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.437 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:41.437 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485432+0000", 2024-05-08T08:52:41.437 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8299999999999998e-07 2024-05-08T08:52:41.438 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.438 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.438 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:41.438 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485434+0000", 2024-05-08T08:52:41.438 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0849999999999999e-06 2024-05-08T08:52:41.438 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.438 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.438 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:41.438 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485440+0000", 2024-05-08T08:52:41.438 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0020000000000003e-06 2024-05-08T08:52:41.438 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.438 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.438 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:41.438 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485445+0000", 2024-05-08T08:52:41.439 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5180000000000004e-06 2024-05-08T08:52:41.439 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.439 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.439 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:41.439 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555731+0000", 2024-05-08T08:52:41.439 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070286214 2024-05-08T08:52:41.439 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.439 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.439 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:41.439 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555787+0000", 2024-05-08T08:52:41.439 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5992999999999997e-05 2024-05-08T08:52:41.439 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.439 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.439 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:41.439 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555844+0000", 2024-05-08T08:52:41.439 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6266e-05 2024-05-08T08:52:41.440 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:41.440 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:41.440 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:41.440 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.440 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.440 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1071 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11026432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:41.440 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:52:41.440 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 4.9376139009999997, 2024-05-08T08:52:41.440 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070580037999999998, 2024-05-08T08:52:41.440 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:41.440 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:41.440 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:41.440 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:41.440 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:41.440 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:41.440 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1071 2024-05-08T08:52:41.441 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.441 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:41.441 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.441 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:41.441 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:52:41.441 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:41.441 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.441 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.441 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:41.441 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:52:41.441 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:41.441 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.441 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.441 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:41.441 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:52:41.442 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2800000000000002e-07 2024-05-08T08:52:41.442 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.442 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.442 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:41.442 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485454+0000", 2024-05-08T08:52:41.442 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2290000000000002e-06 2024-05-08T08:52:41.442 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.442 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.442 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:41.442 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485461+0000", 2024-05-08T08:52:41.442 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9920000000000001e-06 2024-05-08T08:52:41.442 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.442 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.442 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:41.442 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485467+0000", 2024-05-08T08:52:41.442 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3979999999999998e-06 2024-05-08T08:52:41.443 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.443 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.443 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:41.443 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555880+0000", 2024-05-08T08:52:41.443 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070413165 2024-05-08T08:52:41.443 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.443 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.443 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:41.443 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555926+0000", 2024-05-08T08:52:41.443 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6217000000000002e-05 2024-05-08T08:52:41.443 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.443 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.443 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:41.443 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556032+0000", 2024-05-08T08:52:41.444 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000105609 2024-05-08T08:52:41.444 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:41.444 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:41.444 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:41.444 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.444 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.444 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1073 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11067392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:41.444 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:52:41.444 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 4.9375926799999998, 2024-05-08T08:52:41.444 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070821404000000004, 2024-05-08T08:52:41.444 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:41.444 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:41.445 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:41.445 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:41.445 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:41.445 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:41.445 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1073 2024-05-08T08:52:41.445 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.445 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:41.445 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.445 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:41.445 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:52:41.445 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:41.445 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.445 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.445 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:41.445 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:52:41.446 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:41.446 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.446 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.446 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:41.446 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:52:41.446 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.08e-07 2024-05-08T08:52:41.446 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.446 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.446 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:41.446 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485476+0000", 2024-05-08T08:52:41.446 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3929999999999998e-06 2024-05-08T08:52:41.446 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.446 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.446 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:41.446 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485485+0000", 2024-05-08T08:52:41.447 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1880000000000008e-06 2024-05-08T08:52:41.447 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.447 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.447 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:41.447 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485494+0000", 2024-05-08T08:52:41.447 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3109999999999995e-06 2024-05-08T08:52:41.447 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.447 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.447 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:41.447 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556082+0000", 2024-05-08T08:52:41.447 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070587426999999994 2024-05-08T08:52:41.447 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.447 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.447 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:41.448 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556177+0000", 2024-05-08T08:52:41.448 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5051999999999997e-05 2024-05-08T08:52:41.448 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.448 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.448 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:41.448 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556294+0000", 2024-05-08T08:52:41.448 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000117625 2024-05-08T08:52:41.448 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:41.448 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:41.448 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:41.448 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.448 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.448 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1075 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11108352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:41.448 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:52:41.448 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 4.9375621030000003, 2024-05-08T08:52:41.449 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071045026999999997, 2024-05-08T08:52:41.449 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:41.449 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:41.449 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:41.449 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:41.449 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:41.449 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:41.449 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1075 2024-05-08T08:52:41.449 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.449 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:41.449 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.449 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:41.449 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:52:41.449 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:41.449 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.450 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.450 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:41.450 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:52:41.450 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:41.450 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.450 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.450 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:41.450 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:52:41.450 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.32e-07 2024-05-08T08:52:41.450 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.450 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.450 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:41.450 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485518+0000", 2024-05-08T08:52:41.451 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3769e-05 2024-05-08T08:52:41.451 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.451 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.451 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:41.451 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485527+0000", 2024-05-08T08:52:41.451 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6779999999999995e-06 2024-05-08T08:52:41.451 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.451 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.451 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:41.451 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485532+0000", 2024-05-08T08:52:41.451 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9830000000000004e-06 2024-05-08T08:52:41.451 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.451 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.452 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:41.452 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556340+0000", 2024-05-08T08:52:41.452 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070807743000000006 2024-05-08T08:52:41.452 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.452 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.452 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:41.452 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556435+0000", 2024-05-08T08:52:41.452 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4732e-05 2024-05-08T08:52:41.452 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.452 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.452 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:41.452 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556549+0000", 2024-05-08T08:52:41.453 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011369 2024-05-08T08:52:41.453 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:41.453 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:41.453 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:41.453 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.453 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.453 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1078 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11149312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:41.453 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:52:41.453 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 4.9375219040000005, 2024-05-08T08:52:41.453 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071167276000000002, 2024-05-08T08:52:41.453 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:41.454 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:41.454 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:41.454 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:41.454 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:41.454 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:41.454 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1078 2024-05-08T08:52:41.454 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.454 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:41.454 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.454 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:41.454 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:52:41.455 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:41.455 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.455 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.455 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:41.455 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:52:41.455 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:41.455 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.455 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.455 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:41.455 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:52:41.455 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5600000000000001e-07 2024-05-08T08:52:41.455 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.456 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.456 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:41.456 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485547+0000", 2024-05-08T08:52:41.456 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3590000000000002e-06 2024-05-08T08:52:41.456 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.456 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.456 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:41.456 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485553+0000", 2024-05-08T08:52:41.456 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2770000000000002e-06 2024-05-08T08:52:41.456 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.456 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.456 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:41.456 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485558+0000", 2024-05-08T08:52:41.456 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.95e-06 2024-05-08T08:52:41.457 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.457 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.457 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:41.457 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556582+0000", 2024-05-08T08:52:41.457 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071024444000000006 2024-05-08T08:52:41.457 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.457 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.457 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:41.457 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556637+0000", 2024-05-08T08:52:41.457 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4672000000000001e-05 2024-05-08T08:52:41.457 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.458 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.458 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:41.458 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556711+0000", 2024-05-08T08:52:41.458 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4117999999999997e-05 2024-05-08T08:52:41.458 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:41.458 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:41.458 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:41.458 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.458 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.458 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1080 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11190272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:41.458 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:52:41.458 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 4.9375011710000001, 2024-05-08T08:52:41.459 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071350721000000006, 2024-05-08T08:52:41.459 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:41.459 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:41.459 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:41.459 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:41.459 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:41.459 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:41.459 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1080 2024-05-08T08:52:41.459 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.459 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:41.459 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.459 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:41.459 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:52:41.460 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:41.460 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.460 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.460 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:41.460 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:52:41.460 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:41.460 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.460 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.460 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:41.460 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:52:41.460 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.03e-07 2024-05-08T08:52:41.460 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.460 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.461 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:41.461 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485567+0000", 2024-05-08T08:52:41.461 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1090000000000001e-06 2024-05-08T08:52:41.461 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.461 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.461 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:41.461 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485573+0000", 2024-05-08T08:52:41.461 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3049999999999998e-06 2024-05-08T08:52:41.461 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.461 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.461 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:41.462 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485578+0000", 2024-05-08T08:52:41.462 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0130000000000001e-06 2024-05-08T08:52:41.462 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.462 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.462 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:41.462 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556743+0000", 2024-05-08T08:52:41.462 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071164449000000005 2024-05-08T08:52:41.462 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.462 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.462 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:41.462 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556797+0000", 2024-05-08T08:52:41.463 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4589999999999997e-05 2024-05-08T08:52:41.463 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.463 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.463 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:41.463 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556915+0000", 2024-05-08T08:52:41.463 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000117852 2024-05-08T08:52:41.463 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:41.463 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:41.463 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:41.463 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.463 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.464 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1082 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11231232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:41.464 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:52:41.464 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 4.9374809280000003, 2024-05-08T08:52:41.464 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071591396000000002, 2024-05-08T08:52:41.464 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:41.464 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:41.464 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:41.464 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:41.464 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:41.464 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:41.464 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1082 2024-05-08T08:52:41.464 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.464 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:41.464 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.465 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:41.465 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:52:41.465 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:41.465 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.465 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.465 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:41.465 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:52:41.465 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:41.465 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.465 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.465 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:41.465 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:52:41.465 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1800000000000001e-07 2024-05-08T08:52:41.465 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.466 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.466 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:41.466 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485587+0000", 2024-05-08T08:52:41.466 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2589999999999999e-06 2024-05-08T08:52:41.466 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.466 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.466 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:41.466 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485594+0000", 2024-05-08T08:52:41.466 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0789999999999997e-06 2024-05-08T08:52:41.466 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.466 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.466 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:41.466 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485598+0000", 2024-05-08T08:52:41.467 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7480000000000001e-06 2024-05-08T08:52:41.467 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.467 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.467 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:41.467 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556971+0000", 2024-05-08T08:52:41.467 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071372774999999999 2024-05-08T08:52:41.467 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.467 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.467 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:41.467 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557069+0000", 2024-05-08T08:52:41.467 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.7732999999999994e-05 2024-05-08T08:52:41.467 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.467 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.467 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:41.468 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557176+0000", 2024-05-08T08:52:41.468 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000107384 2024-05-08T08:52:41.468 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:41.468 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:41.468 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:41.468 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.468 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.468 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1084 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11272192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:41.468 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:52:41.468 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 4.9374422889999998, 2024-05-08T08:52:41.468 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071808977999999996, 2024-05-08T08:52:41.468 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:41.468 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:41.468 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:41.469 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:41.469 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:41.469 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:41.469 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1084 2024-05-08T08:52:41.469 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.469 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:41.469 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.469 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:41.469 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:52:41.469 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:41.469 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.469 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.470 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:41.470 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:52:41.470 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:41.470 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.470 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.470 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:41.470 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485624+0000", 2024-05-08T08:52:41.470 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:52:41.470 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.470 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.470 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:41.470 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485626+0000", 2024-05-08T08:52:41.470 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9460000000000001e-06 2024-05-08T08:52:41.470 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.470 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.471 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:41.471 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485633+0000", 2024-05-08T08:52:41.471 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1149999999999996e-06 2024-05-08T08:52:41.471 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.471 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.471 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:41.471 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485638+0000", 2024-05-08T08:52:41.471 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5670000000000002e-06 2024-05-08T08:52:41.471 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.471 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.471 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:41.471 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557226+0000", 2024-05-08T08:52:41.471 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071587267999999996 2024-05-08T08:52:41.471 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.471 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.471 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:41.471 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557321+0000", 2024-05-08T08:52:41.472 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5785000000000005e-05 2024-05-08T08:52:41.472 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.472 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.472 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:41.472 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557432+0000", 2024-05-08T08:52:41.472 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011092800000000001 2024-05-08T08:52:41.472 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:41.472 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:41.472 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:41.472 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.472 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.472 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1086 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11313152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:41.472 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:52:41.472 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 4.9374189619999997, 2024-05-08T08:52:41.472 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071933132999999996, 2024-05-08T08:52:41.472 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:41.473 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:41.473 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:41.473 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:41.473 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:41.473 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:41.473 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1086 2024-05-08T08:52:41.473 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.473 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:41.473 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.473 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:41.473 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:52:41.473 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:41.473 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.473 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.473 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:41.473 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:52:41.473 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:41.474 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.474 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.474 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:41.474 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486328+0000", 2024-05-08T08:52:41.474 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00068107900000000002 2024-05-08T08:52:41.474 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.474 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.474 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:41.474 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486331+0000", 2024-05-08T08:52:41.474 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.585e-06 2024-05-08T08:52:41.474 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.474 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.474 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:41.474 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486343+0000", 2024-05-08T08:52:41.474 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1856e-05 2024-05-08T08:52:41.475 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.475 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.475 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:41.475 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486350+0000", 2024-05-08T08:52:41.475 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8600000000000004e-06 2024-05-08T08:52:41.475 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.475 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.475 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:41.475 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557470+0000", 2024-05-08T08:52:41.475 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071120326999999997 2024-05-08T08:52:41.475 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.475 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.475 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:41.475 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557514+0000", 2024-05-08T08:52:41.475 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3909000000000003e-05 2024-05-08T08:52:41.476 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.476 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.476 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:41.476 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557580+0000", 2024-05-08T08:52:41.476 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5517000000000003e-05 2024-05-08T08:52:41.476 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:41.476 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:41.476 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:41.476 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.476 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.476 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1089 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11354112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:41.476 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:52:41.476 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 4.9367060900000004, 2024-05-08T08:52:41.476 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071457381, 2024-05-08T08:52:41.476 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:41.477 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:41.477 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:41.477 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:41.477 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:41.477 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:41.477 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1089 2024-05-08T08:52:41.477 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.477 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:41.477 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.477 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:41.477 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:52:41.477 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:41.477 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.477 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.477 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:41.478 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:52:41.478 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:41.478 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.478 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.478 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:41.478 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486851+0000", 2024-05-08T08:52:41.478 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00049102 2024-05-08T08:52:41.478 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.478 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.478 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:41.478 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486854+0000", 2024-05-08T08:52:41.478 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6859999999999999e-06 2024-05-08T08:52:41.478 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.478 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.479 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:41.479 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486864+0000", 2024-05-08T08:52:41.479 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6080000000000002e-06 2024-05-08T08:52:41.479 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.479 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.479 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:41.479 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486870+0000", 2024-05-08T08:52:41.479 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.477e-06 2024-05-08T08:52:41.479 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.479 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.479 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:41.479 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557612+0000", 2024-05-08T08:52:41.479 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070741557999999996 2024-05-08T08:52:41.480 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.480 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.480 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:41.480 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557695+0000", 2024-05-08T08:52:41.480 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3300000000000005e-05 2024-05-08T08:52:41.480 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.480 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.480 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:41.480 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557817+0000", 2024-05-08T08:52:41.480 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012173200000000001 2024-05-08T08:52:41.480 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:41.480 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:41.480 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:41.481 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.481 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.481 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1091 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11395072~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:41.481 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:52:41.481 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 4.9361852949999996, 2024-05-08T08:52:41.481 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071184737999999997, 2024-05-08T08:52:41.481 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:41.481 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:41.481 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:41.481 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:41.481 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:41.481 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:41.482 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1091 2024-05-08T08:52:41.482 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.482 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:41.482 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.482 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:41.482 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:52:41.482 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:41.482 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.482 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.482 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:41.482 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:52:41.483 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:41.483 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.483 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.483 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:41.483 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486881+0000", 2024-05-08T08:52:41.483 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0699999999999997e-07 2024-05-08T08:52:41.483 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.483 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.483 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:41.483 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486883+0000", 2024-05-08T08:52:41.483 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.402e-06 2024-05-08T08:52:41.484 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.484 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.484 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:41.484 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486890+0000", 2024-05-08T08:52:41.484 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0480000000000001e-06 2024-05-08T08:52:41.484 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.484 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.484 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:41.484 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486896+0000", 2024-05-08T08:52:41.484 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7640000000000002e-06 2024-05-08T08:52:41.484 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.484 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.485 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:41.485 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557860+0000", 2024-05-08T08:52:41.485 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070963791999999998 2024-05-08T08:52:41.485 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.485 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.485 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:41.485 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557952+0000", 2024-05-08T08:52:41.485 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2082000000000004e-05 2024-05-08T08:52:41.485 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.485 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.485 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:41.485 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558065+0000", 2024-05-08T08:52:41.486 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000113143 2024-05-08T08:52:41.486 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:41.486 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:41.486 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:41.486 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.486 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.486 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1094 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11436032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:41.486 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:52:41.486 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 4.9361621270000002, 2024-05-08T08:52:41.486 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071400507000000002, 2024-05-08T08:52:41.486 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:41.487 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:41.487 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:41.487 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:41.487 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:41.487 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:41.487 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1094 2024-05-08T08:52:41.487 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.487 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:41.487 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.487 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:41.487 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:52:41.487 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:41.487 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.488 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.488 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:41.488 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:52:41.488 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:41.488 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.488 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.488 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:41.488 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487940+0000", 2024-05-08T08:52:41.488 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001036868 2024-05-08T08:52:41.488 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.488 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.488 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:41.489 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487944+0000", 2024-05-08T08:52:41.489 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7069999999999998e-06 2024-05-08T08:52:41.489 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.489 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.489 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:41.489 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487951+0000", 2024-05-08T08:52:41.489 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2649999999999999e-06 2024-05-08T08:52:41.489 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.489 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.489 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:41.489 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487961+0000", 2024-05-08T08:52:41.490 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4110000000000002e-06 2024-05-08T08:52:41.490 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.490 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.490 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:41.490 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558117+0000", 2024-05-08T08:52:41.490 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070156662999999994 2024-05-08T08:52:41.490 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.490 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.490 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:41.490 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558214+0000", 2024-05-08T08:52:41.490 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6444e-05 2024-05-08T08:52:41.491 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.491 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.491 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:41.491 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558304+0000", 2024-05-08T08:52:41.491 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0148999999999994e-05 2024-05-08T08:52:41.491 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:41.491 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:41.491 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:41.491 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.491 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.491 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1096 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11476992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:41.492 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:52:41.492 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 4.935095596, 2024-05-08T08:52:41.492 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070480074000000004, 2024-05-08T08:52:41.492 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:41.492 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:41.492 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:41.492 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:41.492 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:41.492 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:41.492 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1096 2024-05-08T08:52:41.493 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.493 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:41.493 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.493 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:41.493 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:52:41.493 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:41.493 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.493 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.493 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:41.493 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:52:41.493 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:41.494 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.494 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.494 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:41.494 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:52:41.494 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3700000000000001e-07 2024-05-08T08:52:41.494 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.494 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.494 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:41.494 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487973+0000", 2024-05-08T08:52:41.494 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0870000000000001e-06 2024-05-08T08:52:41.494 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.494 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.494 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:41.494 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487977+0000", 2024-05-08T08:52:41.495 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0600000000000001e-06 2024-05-08T08:52:41.495 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.495 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.495 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:41.495 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487980+0000", 2024-05-08T08:52:41.495 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1089999999999998e-06 2024-05-08T08:52:41.495 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.495 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.495 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:41.495 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558338+0000", 2024-05-08T08:52:41.495 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070358233000000006 2024-05-08T08:52:41.495 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.495 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.495 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:41.495 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558392+0000", 2024-05-08T08:52:41.496 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4537999999999999e-05 2024-05-08T08:52:41.496 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.496 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.496 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:41.496 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558450+0000", 2024-05-08T08:52:41.496 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7710000000000001e-05 2024-05-08T08:52:41.496 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:41.496 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:41.496 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:41.496 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.496 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.496 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1099 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11517952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:41.496 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:52:41.496 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 4.9350817200000003, 2024-05-08T08:52:41.496 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070720506000000002, 2024-05-08T08:52:41.497 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:41.497 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:41.497 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:41.497 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:41.497 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:41.497 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:41.497 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1099 2024-05-08T08:52:41.497 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.497 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:41.497 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.497 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:41.497 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:52:41.497 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:41.497 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.497 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.498 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:41.498 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:52:41.498 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:41.498 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.498 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.498 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:41.498 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:52:41.498 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3999999999999998e-07 2024-05-08T08:52:41.498 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.498 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.498 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:41.498 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487986+0000", 2024-05-08T08:52:41.498 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3519999999999999e-06 2024-05-08T08:52:41.498 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.498 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.499 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:41.499 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487989+0000", 2024-05-08T08:52:41.499 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.23e-06 2024-05-08T08:52:41.499 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.499 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.499 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:41.499 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487991+0000", 2024-05-08T08:52:41.499 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6350000000000002e-06 2024-05-08T08:52:41.499 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.499 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.499 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:41.499 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558489+0000", 2024-05-08T08:52:41.499 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070497924000000003 2024-05-08T08:52:41.499 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.500 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.500 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:41.500 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558583+0000", 2024-05-08T08:52:41.500 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3405000000000004e-05 2024-05-08T08:52:41.500 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.500 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.500 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:41.500 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558704+0000", 2024-05-08T08:52:41.500 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012171999999999999 2024-05-08T08:52:41.500 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:41.500 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:41.500 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:41.500 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.500 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.501 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1100 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11558912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:41.501 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:52:41.501 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 4.9350687210000004, 2024-05-08T08:52:41.501 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070962266999999996, 2024-05-08T08:52:41.501 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:41.501 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:41.501 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:41.501 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:41.501 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:41.501 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:41.501 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1100 2024-05-08T08:52:41.501 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.501 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:41.501 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.501 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:41.502 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:52:41.502 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:41.502 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.502 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.502 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:41.502 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:52:41.502 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:41.502 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.502 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.502 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:41.502 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488430+0000", 2024-05-08T08:52:41.502 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000432676 2024-05-08T08:52:41.502 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.502 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.503 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:41.503 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488433+0000", 2024-05-08T08:52:41.503 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.45e-06 2024-05-08T08:52:41.503 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.503 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.503 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:41.503 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488442+0000", 2024-05-08T08:52:41.503 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.687e-06 2024-05-08T08:52:41.503 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.503 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.503 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:41.503 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488446+0000", 2024-05-08T08:52:41.503 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4939999999999997e-06 2024-05-08T08:52:41.503 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.503 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.504 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:41.504 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558756+0000", 2024-05-08T08:52:41.504 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070310051999999998 2024-05-08T08:52:41.504 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.504 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.504 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:41.504 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558844+0000", 2024-05-08T08:52:41.504 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7690999999999998e-05 2024-05-08T08:52:41.504 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.504 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.504 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:41.504 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558959+0000", 2024-05-08T08:52:41.504 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000115217 2024-05-08T08:52:41.504 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:41.504 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:41.505 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:41.505 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.505 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.505 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1103 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11599872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:41.505 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:52:41.505 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 4.9346110459999997, 2024-05-08T08:52:41.505 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070777100999999995, 2024-05-08T08:52:41.505 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:41.505 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:41.505 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:41.505 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:41.505 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:41.505 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:41.505 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1103 2024-05-08T08:52:41.505 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.506 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:41.506 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.506 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:41.506 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:52:41.506 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:41.506 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.506 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.506 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:41.506 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:52:41.506 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:41.506 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.506 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.506 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:41.506 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488959+0000", 2024-05-08T08:52:41.506 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00050457200000000001 2024-05-08T08:52:41.507 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.507 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.507 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:41.507 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488969+0000", 2024-05-08T08:52:41.507 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.7159999999999998e-06 2024-05-08T08:52:41.507 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.507 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.507 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:41.507 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488981+0000", 2024-05-08T08:52:41.507 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2150000000000001e-05 2024-05-08T08:52:41.507 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.507 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.507 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:41.507 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488988+0000", 2024-05-08T08:52:41.507 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8759999999999999e-06 2024-05-08T08:52:41.508 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.508 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.508 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:41.508 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558992+0000", 2024-05-08T08:52:41.508 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070004358000000003 2024-05-08T08:52:41.508 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.508 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.508 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:41.508 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559093+0000", 2024-05-08T08:52:41.508 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000100299 2024-05-08T08:52:41.508 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.508 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.508 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:41.508 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559232+0000", 2024-05-08T08:52:41.508 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013913 2024-05-08T08:52:41.509 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:41.509 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:41.509 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:41.509 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.509 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.509 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1104 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11640832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:41.509 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:52:41.509 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 4.9340662310000001, 2024-05-08T08:52:41.509 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070521947000000001, 2024-05-08T08:52:41.509 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:41.509 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:41.509 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:41.509 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:41.509 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:41.509 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:41.509 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1104 2024-05-08T08:52:41.510 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.510 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:41.510 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.510 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:41.510 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:52:41.510 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:41.510 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.510 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.510 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:41.510 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:52:41.510 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:41.510 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.510 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.510 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:41.511 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489700+0000", 2024-05-08T08:52:41.511 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00070067600000000001 2024-05-08T08:52:41.511 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.511 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.511 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:41.511 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489704+0000", 2024-05-08T08:52:41.511 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1640000000000003e-06 2024-05-08T08:52:41.511 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.511 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.511 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:41.511 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489716+0000", 2024-05-08T08:52:41.511 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1379000000000001e-05 2024-05-08T08:52:41.511 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.511 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.511 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:41.512 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489724+0000", 2024-05-08T08:52:41.512 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8949999999999998e-06 2024-05-08T08:52:41.512 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.512 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.512 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:41.512 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559278+0000", 2024-05-08T08:52:41.512 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.069554151999999994 2024-05-08T08:52:41.512 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.512 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.512 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:41.512 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559387+0000", 2024-05-08T08:52:41.512 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000109258 2024-05-08T08:52:41.512 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.512 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.512 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:41.513 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559521+0000", 2024-05-08T08:52:41.513 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000134423 2024-05-08T08:52:41.513 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:41.513 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:41.513 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:41.513 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:41.513 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:41.513 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:52:41.515 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:52:41.515 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:52:41.517+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:52:41.615 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:41.619+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:41.665 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:52:41.665 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:52:41.665 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:52:41.665 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:52:41.672 INFO:teuthology.orchestra.run.smithi196.stdout:checking clone #7, image image_client.0-clone7 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent8 2024-05-08T08:52:41.674 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:52:41.715 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:41.719+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:41.783 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:52:41.783 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:52:41.784 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:52:41.784 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:52:41.784 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:52:41.792 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:52:41.836 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:52:41.839+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:52:41.917 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:41.920+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:41.925 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:52:41.925 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:52:41.925 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:52:41.925 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:52:41.926 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.926 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9944 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 749568~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:41.926 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.103900+0000", 2024-05-08T08:52:41.926 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 14.824175395999999, 2024-05-08T08:52:41.926 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057007044, 2024-05-08T08:52:41.926 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:41.926 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:41.926 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:41.926 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:41.926 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:41.926 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:41.926 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9944 2024-05-08T08:52:41.926 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.927 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:41.927 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.927 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:41.927 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103900+0000", 2024-05-08T08:52:41.927 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:41.927 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.927 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.927 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:41.927 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103900+0000", 2024-05-08T08:52:41.927 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:41.927 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.927 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.927 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:41.927 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103900+0000", 2024-05-08T08:52:41.928 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6400000000000003e-07 2024-05-08T08:52:41.928 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.928 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.928 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:41.928 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103902+0000", 2024-05-08T08:52:41.928 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.858e-06 2024-05-08T08:52:41.928 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.928 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.928 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:41.928 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103907+0000", 2024-05-08T08:52:41.928 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3410000000000002e-06 2024-05-08T08:52:41.928 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.928 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.928 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:41.929 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103910+0000", 2024-05-08T08:52:41.929 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8540000000000001e-06 2024-05-08T08:52:41.929 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.929 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.929 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:41.929 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.160682+0000", 2024-05-08T08:52:41.929 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.056771822 2024-05-08T08:52:41.929 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.929 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.929 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:41.929 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.160777+0000", 2024-05-08T08:52:41.929 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4555999999999994e-05 2024-05-08T08:52:41.929 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.929 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.929 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:41.930 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.160907+0000", 2024-05-08T08:52:41.930 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000130149 2024-05-08T08:52:41.930 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:41.930 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:41.930 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:41.930 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.930 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.930 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9945 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 1441792~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:41.930 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.103914+0000", 2024-05-08T08:52:41.930 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 14.824160803, 2024-05-08T08:52:41.930 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057211979000000003, 2024-05-08T08:52:41.930 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:41.930 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:41.930 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:41.931 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:41.931 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:41.931 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:41.931 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9945 2024-05-08T08:52:41.931 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.931 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:41.931 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.931 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:41.931 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103914+0000", 2024-05-08T08:52:41.931 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:41.931 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.931 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.931 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:41.931 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103914+0000", 2024-05-08T08:52:41.931 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:41.932 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.932 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.932 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:41.932 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103915+0000", 2024-05-08T08:52:41.932 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3099999999999999e-07 2024-05-08T08:52:41.932 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.932 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.932 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:41.932 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103916+0000", 2024-05-08T08:52:41.932 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.203e-06 2024-05-08T08:52:41.932 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.932 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.932 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:41.932 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103919+0000", 2024-05-08T08:52:41.932 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.117e-06 2024-05-08T08:52:41.933 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.933 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.933 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:41.933 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103921+0000", 2024-05-08T08:52:41.933 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4169999999999999e-06 2024-05-08T08:52:41.933 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.933 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.933 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:41.933 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.160959+0000", 2024-05-08T08:52:41.933 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057038074000000001 2024-05-08T08:52:41.933 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.933 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.933 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:41.933 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161041+0000", 2024-05-08T08:52:41.933 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1620999999999997e-05 2024-05-08T08:52:41.934 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.934 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.934 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:41.934 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161126+0000", 2024-05-08T08:52:41.934 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.5216000000000005e-05 2024-05-08T08:52:41.934 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:41.934 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:41.934 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:41.934 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.934 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.934 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9946 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 765952~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:41.934 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.103925+0000", 2024-05-08T08:52:41.934 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 14.824150436, 2024-05-08T08:52:41.934 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057418770000000001, 2024-05-08T08:52:41.935 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:41.935 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:41.935 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:41.935 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:41.935 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:41.935 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:41.935 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9946 2024-05-08T08:52:41.935 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.935 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:41.935 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.935 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:41.935 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103925+0000", 2024-05-08T08:52:41.935 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:41.935 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.935 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.935 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:41.936 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103925+0000", 2024-05-08T08:52:41.936 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:41.936 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.936 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.936 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:41.936 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103925+0000", 2024-05-08T08:52:41.936 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9200000000000002e-07 2024-05-08T08:52:41.936 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.936 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.936 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:41.936 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103926+0000", 2024-05-08T08:52:41.936 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1620000000000001e-06 2024-05-08T08:52:41.936 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.936 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.936 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:41.937 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103929+0000", 2024-05-08T08:52:41.937 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9349999999999999e-06 2024-05-08T08:52:41.937 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.937 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.937 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:41.937 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103931+0000", 2024-05-08T08:52:41.937 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4099999999999998e-06 2024-05-08T08:52:41.937 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.937 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.937 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:41.937 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161161+0000", 2024-05-08T08:52:41.937 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057229176999999999 2024-05-08T08:52:41.937 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.937 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.937 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:41.937 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161235+0000", 2024-05-08T08:52:41.938 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4227000000000006e-05 2024-05-08T08:52:41.938 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.938 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.938 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:41.938 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161343+0000", 2024-05-08T08:52:41.938 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010856700000000001 2024-05-08T08:52:41.938 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:41.938 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:41.938 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:41.938 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.938 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.938 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9947 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 1458176~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:41.938 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.103935+0000", 2024-05-08T08:52:41.938 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 14.824140558, 2024-05-08T08:52:41.939 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057541263000000002, 2024-05-08T08:52:41.939 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:41.939 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:41.939 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:41.939 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:41.939 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:41.939 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:41.939 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9947 2024-05-08T08:52:41.939 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.939 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:41.939 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.939 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:41.939 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103935+0000", 2024-05-08T08:52:41.939 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:41.939 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.940 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.940 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:41.940 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103935+0000", 2024-05-08T08:52:41.940 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:41.940 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.940 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.940 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:41.940 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103935+0000", 2024-05-08T08:52:41.940 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.11e-07 2024-05-08T08:52:41.940 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.940 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.940 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:41.940 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103940+0000", 2024-05-08T08:52:41.940 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1030000000000001e-06 2024-05-08T08:52:41.940 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.941 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.941 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:41.941 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103943+0000", 2024-05-08T08:52:41.941 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3869999999999999e-06 2024-05-08T08:52:41.941 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.941 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.941 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:41.941 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103951+0000", 2024-05-08T08:52:41.941 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.2150000000000001e-06 2024-05-08T08:52:41.941 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.941 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.941 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:41.941 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161377+0000", 2024-05-08T08:52:41.941 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057425925000000003 2024-05-08T08:52:41.941 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.942 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.942 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:41.942 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161431+0000", 2024-05-08T08:52:41.942 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3408000000000001e-05 2024-05-08T08:52:41.942 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.942 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.942 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:41.942 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161476+0000", 2024-05-08T08:52:41.942 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5014e-05 2024-05-08T08:52:41.942 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:41.942 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:41.942 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:41.942 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.942 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.943 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9948 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 782336~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:41.943 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.103965+0000", 2024-05-08T08:52:41.943 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 14.824109668, 2024-05-08T08:52:41.943 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057746205000000002, 2024-05-08T08:52:41.943 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:41.943 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:41.943 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:41.943 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:41.943 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:41.943 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:41.943 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9948 2024-05-08T08:52:41.943 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.943 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:41.943 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.944 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:41.944 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103965+0000", 2024-05-08T08:52:41.944 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:41.944 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.944 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.944 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:41.944 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103965+0000", 2024-05-08T08:52:41.944 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:41.944 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.944 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.944 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:41.944 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103966+0000", 2024-05-08T08:52:41.944 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.44e-07 2024-05-08T08:52:41.944 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.944 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.945 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:41.945 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103968+0000", 2024-05-08T08:52:41.945 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4090000000000001e-06 2024-05-08T08:52:41.945 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.945 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.945 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:41.945 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103973+0000", 2024-05-08T08:52:41.945 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6820000000000002e-06 2024-05-08T08:52:41.945 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.945 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.945 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:41.945 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103976+0000", 2024-05-08T08:52:41.945 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.959e-06 2024-05-08T08:52:41.945 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.946 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.946 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:41.946 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161523+0000", 2024-05-08T08:52:41.946 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057547264000000001 2024-05-08T08:52:41.946 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.946 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.946 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:41.946 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161605+0000", 2024-05-08T08:52:41.946 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1854000000000006e-05 2024-05-08T08:52:41.946 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.946 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.946 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:41.946 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161712+0000", 2024-05-08T08:52:41.946 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000106493 2024-05-08T08:52:41.947 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:41.947 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:41.947 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:41.947 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.947 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.947 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9949 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 1474560~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:41.947 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.103980+0000", 2024-05-08T08:52:41.947 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 14.824095049, 2024-05-08T08:52:41.947 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057852397, 2024-05-08T08:52:41.947 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:41.947 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:41.947 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:41.947 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:41.947 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:41.948 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:41.948 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9949 2024-05-08T08:52:41.948 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.948 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:41.948 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.948 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:41.948 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103980+0000", 2024-05-08T08:52:41.948 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:41.948 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.948 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.948 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:41.948 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103980+0000", 2024-05-08T08:52:41.948 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:41.948 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.948 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.949 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:41.949 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103980+0000", 2024-05-08T08:52:41.949 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.11e-07 2024-05-08T08:52:41.949 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.949 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.949 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:41.949 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103982+0000", 2024-05-08T08:52:41.949 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.269e-06 2024-05-08T08:52:41.949 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.949 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.949 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:41.949 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103985+0000", 2024-05-08T08:52:41.949 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0110000000000001e-06 2024-05-08T08:52:41.949 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.949 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.949 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:41.950 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103987+0000", 2024-05-08T08:52:41.950 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3020000000000002e-06 2024-05-08T08:52:41.950 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.950 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.950 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:41.950 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161744+0000", 2024-05-08T08:52:41.950 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057757115999999997 2024-05-08T08:52:41.950 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.950 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.950 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:41.950 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161791+0000", 2024-05-08T08:52:41.950 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6807000000000001e-05 2024-05-08T08:52:41.950 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.950 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.950 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:41.951 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161832+0000", 2024-05-08T08:52:41.951 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1681000000000003e-05 2024-05-08T08:52:41.951 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:41.951 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:41.951 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:41.951 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.951 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.951 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9950 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 798720~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:41.951 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.103990+0000", 2024-05-08T08:52:41.951 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 14.824084814000001, 2024-05-08T08:52:41.951 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.058062890999999998, 2024-05-08T08:52:41.951 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:41.951 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:41.951 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:41.951 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:41.952 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:41.952 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:41.952 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9950 2024-05-08T08:52:41.952 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.952 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:41.952 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.952 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:41.952 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103990+0000", 2024-05-08T08:52:41.952 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:41.952 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.952 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.952 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:41.952 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103990+0000", 2024-05-08T08:52:41.952 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:41.953 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.953 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.953 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:41.953 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103991+0000", 2024-05-08T08:52:41.953 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1e-07 2024-05-08T08:52:41.953 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.953 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.953 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:41.953 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103992+0000", 2024-05-08T08:52:41.953 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.074e-06 2024-05-08T08:52:41.953 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.953 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.953 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:41.953 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103995+0000", 2024-05-08T08:52:41.954 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9529999999999999e-06 2024-05-08T08:52:41.954 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.954 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.954 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:41.954 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103997+0000", 2024-05-08T08:52:41.954 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3939999999999999e-06 2024-05-08T08:52:41.954 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.954 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.954 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:41.954 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161871+0000", 2024-05-08T08:52:41.954 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057873854000000002 2024-05-08T08:52:41.954 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.954 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.955 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:41.955 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161950+0000", 2024-05-08T08:52:41.955 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9433999999999999e-05 2024-05-08T08:52:41.955 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.955 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.955 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:41.955 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.162053+0000", 2024-05-08T08:52:41.955 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000102972 2024-05-08T08:52:41.955 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:41.955 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:41.955 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:41.955 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.955 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.955 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9951 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 1490944~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:41.956 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.104000+0000", 2024-05-08T08:52:41.956 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 14.82407501, 2024-05-08T08:52:41.956 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.058313416999999999, 2024-05-08T08:52:41.956 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:41.956 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:41.956 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:41.956 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:41.956 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:41.956 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:41.956 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9951 2024-05-08T08:52:41.956 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.956 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:41.956 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.956 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:41.956 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.104000+0000", 2024-05-08T08:52:41.957 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:41.957 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.957 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.957 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:41.957 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.104000+0000", 2024-05-08T08:52:41.957 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:41.957 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.957 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.957 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:41.957 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.104962+0000", 2024-05-08T08:52:41.957 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00096230399999999998 2024-05-08T08:52:41.957 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.957 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.957 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:41.958 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.104978+0000", 2024-05-08T08:52:41.958 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5126999999999999e-05 2024-05-08T08:52:41.958 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.959 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.959 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:41.959 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.104987+0000", 2024-05-08T08:52:41.959 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8940000000000005e-06 2024-05-08T08:52:41.959 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.959 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.959 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:41.959 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.104994+0000", 2024-05-08T08:52:41.959 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5610000000000004e-06 2024-05-08T08:52:41.959 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.959 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.960 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:41.960 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.162114+0000", 2024-05-08T08:52:41.960 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057119717 2024-05-08T08:52:41.960 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.960 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.960 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:41.960 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.162214+0000", 2024-05-08T08:52:41.960 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.9982999999999995e-05 2024-05-08T08:52:41.960 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.960 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.960 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:41.960 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.162314+0000", 2024-05-08T08:52:41.961 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.9831e-05 2024-05-08T08:52:41.961 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:41.961 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:41.961 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:41.961 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.961 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.961 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9952 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 815104~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:41.961 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.105004+0000", 2024-05-08T08:52:41.961 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 14.823070757, 2024-05-08T08:52:41.961 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057530855999999998, 2024-05-08T08:52:41.961 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:41.962 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:41.962 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:41.962 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:41.962 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:41.962 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:41.962 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9952 2024-05-08T08:52:41.962 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.962 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:41.962 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.963 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:41.963 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105004+0000", 2024-05-08T08:52:41.963 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:41.963 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.963 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.963 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:41.963 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105004+0000", 2024-05-08T08:52:41.963 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:41.963 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.963 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.964 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:41.964 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105005+0000", 2024-05-08T08:52:41.964 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9299999999999998e-07 2024-05-08T08:52:41.964 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.964 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.964 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:41.964 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105007+0000", 2024-05-08T08:52:41.964 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2979999999999999e-06 2024-05-08T08:52:41.964 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.964 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.965 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:41.965 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105017+0000", 2024-05-08T08:52:41.965 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5060000000000001e-06 2024-05-08T08:52:41.965 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.965 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.965 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:41.965 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105022+0000", 2024-05-08T08:52:41.965 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2619999999999998e-06 2024-05-08T08:52:41.965 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.965 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.965 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:41.965 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.162356+0000", 2024-05-08T08:52:41.966 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057334388999999999 2024-05-08T08:52:41.966 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.966 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.966 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:41.966 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.162437+0000", 2024-05-08T08:52:41.966 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0863000000000002e-05 2024-05-08T08:52:41.966 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.966 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.966 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:41.966 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.162535+0000", 2024-05-08T08:52:41.966 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8045000000000001e-05 2024-05-08T08:52:41.966 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:41.967 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:41.967 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:41.967 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.967 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.967 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9953 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 1507328~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:41.967 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.105064+0000", 2024-05-08T08:52:41.967 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 14.82301107, 2024-05-08T08:52:41.967 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057721531999999999, 2024-05-08T08:52:41.968 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:41.968 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:41.968 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:41.968 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:41.968 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:41.968 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:41.968 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9953 2024-05-08T08:52:41.969 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.969 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:41.969 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.969 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:41.969 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105064+0000", 2024-05-08T08:52:41.969 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:41.969 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.969 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.969 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:41.970 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105064+0000", 2024-05-08T08:52:41.970 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:41.970 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.970 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.970 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:41.970 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105065+0000", 2024-05-08T08:52:41.970 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8800000000000002e-07 2024-05-08T08:52:41.970 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.970 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.970 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:41.970 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105067+0000", 2024-05-08T08:52:41.970 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7669999999999999e-06 2024-05-08T08:52:41.970 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.971 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.971 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:41.971 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105076+0000", 2024-05-08T08:52:41.971 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.5259999999999993e-06 2024-05-08T08:52:41.971 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.971 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.971 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:41.971 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105080+0000", 2024-05-08T08:52:41.971 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8e-06 2024-05-08T08:52:41.971 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.971 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.971 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:41.971 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.162596+0000", 2024-05-08T08:52:41.971 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057515822000000001 2024-05-08T08:52:41.972 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.972 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.972 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:41.972 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.162689+0000", 2024-05-08T08:52:41.972 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3258999999999996e-05 2024-05-08T08:52:41.972 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.972 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.972 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:41.972 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.162786+0000", 2024-05-08T08:52:41.972 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6769999999999997e-05 2024-05-08T08:52:41.972 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:41.972 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:41.972 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:41.972 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.973 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.973 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9954 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 1523712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:41.973 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.105088+0000", 2024-05-08T08:52:41.973 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 14.822987060999999, 2024-05-08T08:52:41.973 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057925139, 2024-05-08T08:52:41.973 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:41.973 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:41.973 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:41.973 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:41.973 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:41.973 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:41.973 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9954 2024-05-08T08:52:41.973 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.973 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:41.973 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.974 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:41.974 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105088+0000", 2024-05-08T08:52:41.974 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:41.974 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.974 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.974 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:41.974 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105088+0000", 2024-05-08T08:52:41.974 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:41.974 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.974 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.974 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:41.974 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105089+0000", 2024-05-08T08:52:41.974 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9599999999999999e-07 2024-05-08T08:52:41.974 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.974 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.975 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:41.975 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105091+0000", 2024-05-08T08:52:41.975 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3649999999999998e-06 2024-05-08T08:52:41.975 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.975 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.975 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:41.975 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105107+0000", 2024-05-08T08:52:41.975 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6370000000000001e-05 2024-05-08T08:52:41.975 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.975 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.975 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:41.975 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105114+0000", 2024-05-08T08:52:41.975 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2190000000000001e-06 2024-05-08T08:52:41.975 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.975 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.976 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:41.976 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.162830+0000", 2024-05-08T08:52:41.976 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057716637000000001 2024-05-08T08:52:41.976 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.976 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.976 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:41.976 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.162924+0000", 2024-05-08T08:52:41.976 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3354000000000001e-05 2024-05-08T08:52:41.976 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.976 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.976 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:41.976 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.163013+0000", 2024-05-08T08:52:41.976 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.9697999999999994e-05 2024-05-08T08:52:41.976 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:41.977 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:41.977 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:41.977 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.977 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.977 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9955 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 831488~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:41.977 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.105124+0000", 2024-05-08T08:52:41.977 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 14.822951486999999, 2024-05-08T08:52:41.977 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.058092618999999998, 2024-05-08T08:52:41.977 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:41.977 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:41.977 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:41.977 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:41.977 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:41.977 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:41.977 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9955 2024-05-08T08:52:41.978 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.978 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:41.978 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.978 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:41.978 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105124+0000", 2024-05-08T08:52:41.978 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:41.978 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.978 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.978 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:41.978 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105124+0000", 2024-05-08T08:52:41.978 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:41.978 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.978 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.978 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:41.979 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.106961+0000", 2024-05-08T08:52:41.979 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001837574 2024-05-08T08:52:41.979 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.979 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.979 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:41.979 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.106966+0000", 2024-05-08T08:52:41.979 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8439999999999997e-06 2024-05-08T08:52:41.979 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.979 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.979 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:41.979 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.106978+0000", 2024-05-08T08:52:41.979 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2330999999999999e-05 2024-05-08T08:52:41.979 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.979 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.979 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:41.980 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.106986+0000", 2024-05-08T08:52:41.980 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0439999999999996e-06 2024-05-08T08:52:41.980 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.980 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.980 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:41.980 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.163048+0000", 2024-05-08T08:52:41.980 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.056061182000000001 2024-05-08T08:52:41.980 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.980 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.980 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:41.980 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.163124+0000", 2024-05-08T08:52:41.980 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.6030000000000002e-05 2024-05-08T08:52:41.980 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.980 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.980 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:41.980 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.163216+0000", 2024-05-08T08:52:41.981 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2614000000000005e-05 2024-05-08T08:52:41.981 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:41.981 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:41.981 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:41.981 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.981 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.981 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9960 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 1589248~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:41.981 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.107116+0000", 2024-05-08T08:52:41.981 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 14.820958668999999, 2024-05-08T08:52:41.981 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.056889636, 2024-05-08T08:52:41.981 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:41.981 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:41.981 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:41.981 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:41.982 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:41.982 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:41.982 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9960 2024-05-08T08:52:41.982 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.982 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:41.982 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.982 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:41.982 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107116+0000", 2024-05-08T08:52:41.982 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:41.982 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.982 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.982 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:41.982 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107116+0000", 2024-05-08T08:52:41.982 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:41.983 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.983 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.983 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:41.983 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107117+0000", 2024-05-08T08:52:41.983 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4400000000000001e-07 2024-05-08T08:52:41.983 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.983 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.983 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:41.983 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107119+0000", 2024-05-08T08:52:41.983 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3750000000000001e-06 2024-05-08T08:52:41.983 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.983 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.983 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:41.983 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107126+0000", 2024-05-08T08:52:41.983 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3369999999999996e-06 2024-05-08T08:52:41.984 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.984 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.984 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:41.984 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107130+0000", 2024-05-08T08:52:41.984 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8620000000000002e-06 2024-05-08T08:52:41.984 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.984 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.984 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:41.984 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.163917+0000", 2024-05-08T08:52:41.984 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.056786634000000002 2024-05-08T08:52:41.984 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.984 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.984 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:41.984 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.163962+0000", 2024-05-08T08:52:41.984 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4521000000000003e-05 2024-05-08T08:52:41.985 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.985 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.985 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:41.985 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.164006+0000", 2024-05-08T08:52:41.985 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4462999999999998e-05 2024-05-08T08:52:41.985 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:41.985 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:41.985 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:41.985 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.985 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.985 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9961 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 864256~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:41.985 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.107138+0000", 2024-05-08T08:52:41.985 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 14.82093761, 2024-05-08T08:52:41.985 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057108051999999999, 2024-05-08T08:52:41.985 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:41.986 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:41.986 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:41.986 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:41.986 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:41.986 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:41.986 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9961 2024-05-08T08:52:41.986 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.986 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:41.986 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.986 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:41.986 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107138+0000", 2024-05-08T08:52:41.986 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:41.986 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.987 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.987 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:41.987 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107138+0000", 2024-05-08T08:52:41.987 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:41.987 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.987 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.987 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:41.987 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107138+0000", 2024-05-08T08:52:41.987 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2399999999999999e-07 2024-05-08T08:52:41.987 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.987 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.987 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:41.987 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107140+0000", 2024-05-08T08:52:41.987 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1409999999999999e-06 2024-05-08T08:52:41.987 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.988 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.988 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:41.988 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107146+0000", 2024-05-08T08:52:41.988 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0639999999999999e-06 2024-05-08T08:52:41.988 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.988 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.988 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:41.988 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107151+0000", 2024-05-08T08:52:41.988 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.69e-06 2024-05-08T08:52:41.988 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.988 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.988 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:41.988 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.164051+0000", 2024-05-08T08:52:41.988 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.056900153000000002 2024-05-08T08:52:41.988 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.989 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.989 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:41.989 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.164136+0000", 2024-05-08T08:52:41.989 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.5086999999999994e-05 2024-05-08T08:52:41.989 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.989 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.989 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:41.989 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.164246+0000", 2024-05-08T08:52:41.989 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000109493 2024-05-08T08:52:41.989 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:41.989 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:41.989 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:41.989 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.990 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.990 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9962 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 1605632~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:41.990 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.107158+0000", 2024-05-08T08:52:41.990 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 14.820917530999999, 2024-05-08T08:52:41.990 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057292698000000003, 2024-05-08T08:52:41.990 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:41.990 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:41.990 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:41.990 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:41.990 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:41.990 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:41.990 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9962 2024-05-08T08:52:41.990 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.990 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:41.991 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.991 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:41.991 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107158+0000", 2024-05-08T08:52:41.991 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:41.991 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.991 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.991 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:41.991 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107158+0000", 2024-05-08T08:52:41.991 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:41.991 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.991 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.991 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:41.991 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107158+0000", 2024-05-08T08:52:41.991 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8100000000000003e-07 2024-05-08T08:52:41.991 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.992 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.992 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:41.992 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107167+0000", 2024-05-08T08:52:41.992 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3079999999999997e-06 2024-05-08T08:52:41.992 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.992 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.992 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:41.992 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107175+0000", 2024-05-08T08:52:41.992 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2089999999999999e-06 2024-05-08T08:52:41.992 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.992 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.992 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:41.992 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107180+0000", 2024-05-08T08:52:41.992 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1120000000000004e-06 2024-05-08T08:52:41.992 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.993 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.993 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:41.993 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.164284+0000", 2024-05-08T08:52:41.993 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057104110999999999 2024-05-08T08:52:41.993 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.993 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.993 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:41.993 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.164370+0000", 2024-05-08T08:52:41.993 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.6549000000000001e-05 2024-05-08T08:52:41.993 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.993 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.993 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:41.993 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.164450+0000", 2024-05-08T08:52:41.993 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9827999999999995e-05 2024-05-08T08:52:41.993 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:41.994 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:41.994 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:41.994 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.994 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.994 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9963 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 880640~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:41.994 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.107188+0000", 2024-05-08T08:52:41.994 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 14.820887613, 2024-05-08T08:52:41.994 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057560471000000002, 2024-05-08T08:52:41.994 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:41.994 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:41.994 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:41.994 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:41.994 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:41.994 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:41.994 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9963 2024-05-08T08:52:41.995 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.995 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:41.995 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.995 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:41.995 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107188+0000", 2024-05-08T08:52:41.995 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:41.995 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.995 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.995 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:41.995 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107188+0000", 2024-05-08T08:52:41.995 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:41.995 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.995 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.995 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:41.995 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.108964+0000", 2024-05-08T08:52:41.995 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001775993 2024-05-08T08:52:41.996 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.996 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.996 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:41.996 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.108969+0000", 2024-05-08T08:52:41.996 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1200000000000001e-06 2024-05-08T08:52:41.996 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.996 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.996 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:41.996 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.108981+0000", 2024-05-08T08:52:41.996 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2670000000000001e-05 2024-05-08T08:52:41.996 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.996 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.996 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:41.996 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.108989+0000", 2024-05-08T08:52:41.997 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3139999999999998e-06 2024-05-08T08:52:41.997 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.997 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.997 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:41.997 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.164504+0000", 2024-05-08T08:52:41.997 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.055515225000000001 2024-05-08T08:52:41.997 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.997 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.997 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:41.997 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.164606+0000", 2024-05-08T08:52:41.997 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010245 2024-05-08T08:52:41.997 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.997 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.997 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:41.997 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.164748+0000", 2024-05-08T08:52:41.997 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014169900000000001 2024-05-08T08:52:41.998 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:41.998 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:41.998 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:41.998 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.998 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.998 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9970 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 929792~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:41.998 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.109178+0000", 2024-05-08T08:52:41.998 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 14.818897334999999, 2024-05-08T08:52:41.998 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057137975000000001, 2024-05-08T08:52:41.998 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:41.998 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:41.998 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:41.998 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:41.998 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:41.999 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:41.999 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9970 2024-05-08T08:52:41.999 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.999 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:41.999 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.999 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:41.999 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109178+0000", 2024-05-08T08:52:41.999 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:41.999 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.999 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:41.999 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:41.999 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109178+0000", 2024-05-08T08:52:41.999 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:41.999 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:41.999 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:42.000 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:42.000 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109178+0000", 2024-05-08T08:52:42.000 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7800000000000002e-07 2024-05-08T08:52:42.000 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:42.000 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:42.000 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:42.000 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109180+0000", 2024-05-08T08:52:42.000 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2199999999999999e-06 2024-05-08T08:52:42.000 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:42.000 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:42.000 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:42.000 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109186+0000", 2024-05-08T08:52:42.000 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9410000000000004e-06 2024-05-08T08:52:42.000 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:42.000 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:42.000 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:42.001 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109191+0000", 2024-05-08T08:52:42.001 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8729999999999998e-06 2024-05-08T08:52:42.001 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:42.001 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:42.001 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:42.001 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.166098+0000", 2024-05-08T08:52:42.001 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.056907124000000003 2024-05-08T08:52:42.001 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:42.001 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:42.001 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:42.001 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.166193+0000", 2024-05-08T08:52:42.001 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5018000000000004e-05 2024-05-08T08:52:42.001 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:42.001 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:42.001 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:42.002 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.166316+0000", 2024-05-08T08:52:42.002 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000122321 2024-05-08T08:52:42.002 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:42.002 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:42.002 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:42.002 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:42.002 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:42.002 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9971 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 1687552~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:42.002 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.109198+0000", 2024-05-08T08:52:42.002 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 14.818876657000001, 2024-05-08T08:52:42.002 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057361875, 2024-05-08T08:52:42.002 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:42.002 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:42.002 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:42.003 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:42.003 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:42.003 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:42.003 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9971 2024-05-08T08:52:42.003 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:42.003 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:42.003 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:42.003 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:42.003 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109198+0000", 2024-05-08T08:52:42.003 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:42.003 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:42.003 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:42.003 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:42.003 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109198+0000", 2024-05-08T08:52:42.003 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:42.004 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:42.004 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:42.004 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:42.004 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109199+0000", 2024-05-08T08:52:42.004 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2800000000000002e-07 2024-05-08T08:52:42.004 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:42.004 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:42.004 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:42.004 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109208+0000", 2024-05-08T08:52:42.004 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4210000000000001e-06 2024-05-08T08:52:42.004 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:42.004 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:42.004 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:42.004 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109215+0000", 2024-05-08T08:52:42.004 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7209999999999997e-06 2024-05-08T08:52:42.004 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:42.005 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:42.005 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:42.005 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109220+0000", 2024-05-08T08:52:42.005 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.13e-06 2024-05-08T08:52:42.005 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:42.005 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:42.005 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:42.005 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.166353+0000", 2024-05-08T08:52:42.005 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057132693999999998 2024-05-08T08:52:42.005 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:42.005 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:42.005 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:42.005 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.166462+0000", 2024-05-08T08:52:42.005 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010940799999999999 2024-05-08T08:52:42.005 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:42.005 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:42.006 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:42.006 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.166560+0000", 2024-05-08T08:52:42.006 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8072999999999994e-05 2024-05-08T08:52:42.006 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:42.006 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:42.006 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:42.006 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:42.006 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:42.006 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9972 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 1703936~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:42.006 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.109228+0000", 2024-05-08T08:52:42.006 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 14.818847265, 2024-05-08T08:52:42.006 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057541904999999997, 2024-05-08T08:52:42.006 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:42.006 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:42.007 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:42.007 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:42.007 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:42.007 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:42.007 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9972 2024-05-08T08:52:42.007 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:42.007 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:42.007 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:42.007 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:42.007 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109228+0000", 2024-05-08T08:52:42.007 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:42.007 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:42.007 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:42.007 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:42.007 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109228+0000", 2024-05-08T08:52:42.007 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:42.008 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:42.008 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:42.008 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:42.008 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109228+0000", 2024-05-08T08:52:42.008 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5699999999999998e-07 2024-05-08T08:52:42.008 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:42.008 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:42.008 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:42.008 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109231+0000", 2024-05-08T08:52:42.008 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2120000000000002e-06 2024-05-08T08:52:42.008 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:42.008 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:42.008 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:42.008 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109237+0000", 2024-05-08T08:52:42.008 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3099999999999997e-06 2024-05-08T08:52:42.008 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:42.009 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:42.009 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:42.009 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109243+0000", 2024-05-08T08:52:42.009 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7490000000000004e-06 2024-05-08T08:52:42.009 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:42.009 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:42.009 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:42.009 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.166609+0000", 2024-05-08T08:52:42.009 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057366372999999998 2024-05-08T08:52:42.009 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:42.009 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:42.009 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:42.009 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.166690+0000", 2024-05-08T08:52:42.009 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1426000000000003e-05 2024-05-08T08:52:42.009 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:42.009 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:42.010 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:42.010 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.166770+0000", 2024-05-08T08:52:42.010 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9377999999999998e-05 2024-05-08T08:52:42.010 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:42.010 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:42.010 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:42.010 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:42.010 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:42.010 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9973 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 946176~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:42.010 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.109249+0000", 2024-05-08T08:52:42.010 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 14.818825623, 2024-05-08T08:52:42.010 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057760381, 2024-05-08T08:52:42.010 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:42.011 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:42.011 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:42.011 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:42.011 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:42.011 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:42.011 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9973 2024-05-08T08:52:42.011 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:42.011 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:42.011 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:42.011 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:42.011 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109249+0000", 2024-05-08T08:52:42.011 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:42.011 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:42.011 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:42.012 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:42.012 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109249+0000", 2024-05-08T08:52:42.012 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:42.012 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:42.012 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:42.012 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:42.012 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.110959+0000", 2024-05-08T08:52:42.012 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001709288 2024-05-08T08:52:42.012 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:42.012 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:42.012 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:42.012 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.110972+0000", 2024-05-08T08:52:42.012 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2719e-05 2024-05-08T08:52:42.012 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:42.013 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:42.013 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:42.013 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.110984+0000", 2024-05-08T08:52:42.013 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2001000000000001e-05 2024-05-08T08:52:42.013 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:42.013 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:42.013 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:42.013 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.110993+0000", 2024-05-08T08:52:42.013 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0880000000000001e-06 2024-05-08T08:52:42.013 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:42.013 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:42.013 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:42.013 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.166817+0000", 2024-05-08T08:52:42.013 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.055823971999999999 2024-05-08T08:52:42.013 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:42.014 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:42.014 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:42.014 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.166900+0000", 2024-05-08T08:52:42.014 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3838999999999995e-05 2024-05-08T08:52:42.014 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:42.014 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:42.014 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:42.014 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.167010+0000", 2024-05-08T08:52:42.014 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000109474 2024-05-08T08:52:42.014 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:42.014 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:42.014 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:42.014 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:42.014 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:42.015 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:52:42.016 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:52:42.018 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:42.021+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:42.119 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:52:42.123+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:52:42.166 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:52:42.166 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:52:42.166 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:52:42.167 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:52:42.175 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:52:42.176 INFO:teuthology.orchestra.run.smithi196.stdout:1412 read 0x68f0a00 thru 0x68f1482 (0xa83 bytes) 2024-05-08T08:52:42.177 INFO:teuthology.orchestra.run.smithi196.stdout:1414 read 0x60ca00 thru 0x612d4f (0x6350 bytes) 2024-05-08T08:52:42.178 INFO:teuthology.orchestra.run.smithi196.stdout:1416 read 0x6968400 thru 0x696a0a0 (0x1ca1 bytes) 2024-05-08T08:52:42.183 INFO:teuthology.orchestra.run.smithi196.stdout:1417 trunc from 0xa30da00 to 0x6592800 2024-05-08T08:52:42.205 INFO:teuthology.orchestra.run.smithi196.stdout:1418 read 0x59c8600 thru 0x59ceb8d (0x658e bytes) 2024-05-08T08:52:42.206 INFO:teuthology.orchestra.run.smithi196.stdout:1419 read 0x166be00 thru 0x1673d98 (0x7f99 bytes) 2024-05-08T08:52:42.207 INFO:teuthology.orchestra.run.smithi196.stdout:1422 punch from 0x5749400 to 0x574b200, (0x1e00 bytes) 2024-05-08T08:52:42.220 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:42.223+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:42.236 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T08:52:42.257 INFO:teuthology.orchestra.run.smithi196.stdout:1424 write 0x84cd800 thru 0x84d1f56 (0x4757 bytes) 2024-05-08T08:52:42.257 INFO:teuthology.orchestra.run.smithi196.stdout:1425 write 0x7ec8800 thru 0x7ecca18 (0x4219 bytes) 2024-05-08T08:52:42.257 INFO:teuthology.orchestra.run.smithi196.stdout:1426 punch from 0x60d1200 to 0x60df400, (0xe200 bytes) 2024-05-08T08:52:42.317 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:52:42.317 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:52:42.317 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:52:42.317 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:52:42.317 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:52:42.320 INFO:teuthology.orchestra.run.smithi196.stdout:1427 write 0xca9d000 thru 0xcaaa65c (0xd65d bytes) 2024-05-08T08:52:42.320 INFO:teuthology.orchestra.run.smithi196.stdout:1428 read 0x97e7e00 thru 0x97f2b61 (0xad62 bytes) 2024-05-08T08:52:42.320 INFO:teuthology.orchestra.run.smithi196.stdout:1433 read 0x27d1c00 thru 0x27d7e4e (0x624f bytes) 2024-05-08T08:52:42.321 INFO:teuthology.orchestra.run.smithi196.stdout:1435 write 0x130e200 thru 0x1318b4a (0xa94b bytes) 2024-05-08T08:52:42.321 INFO:teuthology.orchestra.run.smithi196.stdout:1436 read 0x9f1bc00 thru 0x9f26f20 (0xb321 bytes) 2024-05-08T08:52:42.321 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:42.324+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:42.327 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:52:42.408 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:52:42.408 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:52:42.421 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:42.425+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:42.440 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:52:42.440 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:52:42.460 INFO:teuthology.orchestra.run.smithi196.stdout:1437 write 0xda4d800 thru 0xda4f3fd (0x1bfe bytes) 2024-05-08T08:52:42.461 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:52:42.461 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:52:42.476 INFO:teuthology.orchestra.run.smithi196.stdout:1438 trunc from 0xda4f400 to 0x1a5a400 2024-05-08T08:52:42.480 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:42.480 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:42.481 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:52:42.481 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:52:42.493 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:52:42.509 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:52:42.509 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:52:42.535 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:52:42.536 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:52:42.538 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:52:42.541+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:52:42.615 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:42.616 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:42.623 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:52:42.626+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:52:42.625 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:52:42.637 INFO:teuthology.orchestra.run.smithi196.stdout:1441 write 0xbdd3600 thru 0xbddb6f1 (0x80f2 bytes) 2024-05-08T08:52:42.638 INFO:teuthology.orchestra.run.smithi196.stdout:1446 write 0x46f8800 thru 0x4700d9d (0x859e bytes) 2024-05-08T08:52:42.638 INFO:teuthology.orchestra.run.smithi196.stdout:1447 punch from 0x7530600 to 0x7539c00, (0x9600 bytes) 2024-05-08T08:52:42.643 INFO:teuthology.orchestra.run.smithi196.stdout:1448 punch from 0xb502800 to 0xb50b600, (0x8e00 bytes) 2024-05-08T08:52:42.643 INFO:teuthology.orchestra.run.smithi196.stdout:1449 write 0xa758e00 thru 0xa7643eb (0xb5ec bytes) 2024-05-08T08:52:42.644 INFO:teuthology.orchestra.run.smithi196.stdout:1450 read 0x2866e00 thru 0x287029e (0x949f bytes) 2024-05-08T08:52:42.644 INFO:teuthology.orchestra.run.smithi196.stdout:1451 write 0x9ee8600 thru 0x9eee585 (0x5f86 bytes) 2024-05-08T08:52:42.644 INFO:teuthology.orchestra.run.smithi196.stdout:1454 punch from 0xb5bfe00 to 0xb5c9800, (0x9a00 bytes) 2024-05-08T08:52:42.644 INFO:teuthology.orchestra.run.smithi196.stdout:1455 trunc from 0xbddb800 to 0xb47fa00 2024-05-08T08:52:42.679 INFO:teuthology.orchestra.run.smithi196.stdout:1461 punch from 0xa226c00 to 0xa229a00, (0x2e00 bytes) 2024-05-08T08:52:42.685 INFO:teuthology.orchestra.run.smithi196.stdout:1463 write 0xac5da00 thru 0xac69df6 (0xc3f7 bytes) 2024-05-08T08:52:42.685 INFO:teuthology.orchestra.run.smithi196.stdout:1464 write 0x5269400 thru 0x5277b71 (0xe772 bytes) 2024-05-08T08:52:42.685 INFO:teuthology.orchestra.run.smithi196.stdout:1469 punch from 0x23ee200 to 0x23f1400, (0x3200 bytes) 2024-05-08T08:52:42.686 INFO:teuthology.orchestra.run.smithi196.stdout:1472 trunc from 0xb47fa00 to 0xba76c00 2024-05-08T08:52:42.720 INFO:teuthology.orchestra.run.smithi196.stdout:1474 trunc from 0xba76c00 to 0x76ea00 2024-05-08T08:52:42.724 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:42.727+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:42.796 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:42.796 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:42.809 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:52:42.825 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:52:42.828+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:52:42.875 INFO:teuthology.orchestra.run.smithi196.stdout:1476 trunc from 0x76ea00 to 0xc6f2800 2024-05-08T08:52:42.884 INFO:teuthology.orchestra.run.smithi196.stdout:1477 trunc from 0xc6f2800 to 0xd179a00 2024-05-08T08:52:42.900 INFO:teuthology.orchestra.run.smithi196.stdout:1478 punch from 0xcc1e600 to 0xcc23e00, (0x5800 bytes) 2024-05-08T08:52:42.902 INFO:teuthology.orchestra.run.smithi196.stdout:1479 read 0xae6c800 thru 0xae740f6 (0x78f7 bytes) 2024-05-08T08:52:42.912 INFO:teuthology.orchestra.run.smithi196.stdout:1483 trunc from 0xd179a00 to 0x4e2c800 2024-05-08T08:52:42.926 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:52:42.929+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:52:42.927 INFO:teuthology.orchestra.run.smithi196.stdout:1484 trunc from 0x4e2c800 to 0x60a8800 2024-05-08T08:52:42.951 INFO:teuthology.orchestra.run.smithi196.stdout:1485 write 0x9428400 thru 0x942f1a2 (0x6da3 bytes) 2024-05-08T08:52:42.965 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:42.965 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:42.975 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:52:42.977 INFO:teuthology.orchestra.run.smithi196.stdout:1486 write 0xe9aac00 thru 0xe9b2429 (0x782a bytes) 2024-05-08T08:52:42.978 INFO:teuthology.orchestra.run.smithi196.stdout:1487 read 0xc600a00 thru 0xc60dc5e (0xd25f bytes) 2024-05-08T08:52:42.991 INFO:teuthology.orchestra.run.smithi196.stdout:1491 trunc from 0xe9b2600 to 0x3ae5a00 2024-05-08T08:52:43.027 INFO:teuthology.orchestra.run.smithi196.stdout:1493 trunc from 0x3ae5a00 to 0xace9200 2024-05-08T08:52:43.027 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:52:43.030+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:52:43.037 INFO:teuthology.orchestra.run.smithi196.stdout:1495 read 0x2f7ce00 thru 0x2f7ff17 (0x3118 bytes) 2024-05-08T08:52:43.043 INFO:teuthology.orchestra.run.smithi196.stdout:1497 trunc from 0xace9200 to 0x72ac200 2024-05-08T08:52:43.044 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-05-08T08:52:43.054 INFO:teuthology.orchestra.run.smithi196.stdout:1498 punch from 0x43fc800 to 0x4400200, (0x3a00 bytes) 2024-05-08T08:52:43.057 INFO:teuthology.orchestra.run.smithi196.stdout:1501 punch from 0x3b32800 to 0x3b3d000, (0xa800 bytes) 2024-05-08T08:52:43.057 INFO:teuthology.orchestra.run.smithi196.stdout:1502 read 0x4b21600 thru 0x4b23f7f (0x2980 bytes) 2024-05-08T08:52:43.057 INFO:teuthology.orchestra.run.smithi196.stdout:1503 read 0x34da400 thru 0x34e4bdf (0xa7e0 bytes) 2024-05-08T08:52:43.057 INFO:teuthology.orchestra.run.smithi196.stdout:1507 punch from 0x2f9dc00 to 0x2fa2400, (0x4800 bytes) 2024-05-08T08:52:43.057 INFO:teuthology.orchestra.run.smithi196.stdout:1508 read 0x101fc00 thru 0x102fad3 (0xfed4 bytes) 2024-05-08T08:52:43.058 INFO:teuthology.orchestra.run.smithi196.stdout:1509 punch from 0x40b8600 to 0x40c6400, (0xde00 bytes) 2024-05-08T08:52:43.081 INFO:teuthology.orchestra.run.smithi196.stdout:1511 write 0xc18d600 thru 0xc18d678 (0x79 bytes) 2024-05-08T08:52:43.087 INFO:teuthology.orchestra.run.smithi196.stdout:1512 punch from 0x4834400 to 0x483e400, (0xa000 bytes) 2024-05-08T08:52:43.089 INFO:teuthology.orchestra.run.smithi196.stdout:1515 write 0x92d1a00 thru 0x92da2b2 (0x88b3 bytes) 2024-05-08T08:52:43.089 INFO:teuthology.orchestra.run.smithi196.stdout:1516 write 0xc108e00 thru 0xc10ab06 (0x1d07 bytes) 2024-05-08T08:52:43.089 INFO:teuthology.orchestra.run.smithi196.stdout:1519 write 0x17c3e00 thru 0x17d33ee (0xf5ef bytes) 2024-05-08T08:52:43.089 INFO:teuthology.orchestra.run.smithi196.stdout:1520 write 0xa1ab000 thru 0xa1b1807 (0x6808 bytes) 2024-05-08T08:52:43.089 INFO:teuthology.orchestra.run.smithi196.stdout:1523 punch from 0x121fe00 to 0x122d400, (0xd600 bytes) 2024-05-08T08:52:43.096 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:43.096 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:43.107 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:52:43.127 INFO:teuthology.orchestra.run.smithi196.stdout:1524 write 0xca8c800 thru 0xca9ad57 (0xe558 bytes) 2024-05-08T08:52:43.130 INFO:teuthology.orchestra.run.smithi196.stdout:1525 read 0xa220400 thru 0xa22e142 (0xdd43 bytes) 2024-05-08T08:52:43.131 INFO:teuthology.orchestra.run.smithi196.stdout:1526 read 0x90d9e00 thru 0x90e2ec4 (0x90c5 bytes) 2024-05-08T08:52:43.132 INFO:teuthology.orchestra.run.smithi196.stdout:1527 read 0xf2bc00 thru 0xf3480a (0x8c0b bytes) 2024-05-08T08:52:43.133 INFO:teuthology.orchestra.run.smithi196.stdout:1530 read 0x33fe200 thru 0x3400a75 (0x2876 bytes) 2024-05-08T08:52:43.146 INFO:teuthology.orchestra.run.smithi196.stdout:1531 trunc from 0xca9ae00 to 0x2bafa00 2024-05-08T08:52:43.150 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:43.154+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:43.212 INFO:teuthology.orchestra.run.smithi196.stdout:1532 write 0xe1c7000 thru 0xe1d0789 (0x978a bytes) 2024-05-08T08:52:43.212 INFO:teuthology.orchestra.run.smithi196.stdout:1534 punch from 0x4953c00 to 0x4955800, (0x1c00 bytes) 2024-05-08T08:52:43.213 INFO:teuthology.orchestra.run.smithi196.stdout:1535 punch from 0x4872a00 to 0x4882800, (0xfe00 bytes) 2024-05-08T08:52:43.217 INFO:teuthology.orchestra.run.smithi196.stdout:1537 trunc from 0xe1d0800 to 0x9c34e00 2024-05-08T08:52:43.227 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:43.227 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:43.229 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:52:43.232+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:52:43.237 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:52:43.248 INFO:teuthology.orchestra.run.smithi196.stdout:1539 write 0x234ac00 thru 0x234bf4a (0x134b bytes) 2024-05-08T08:52:43.248 INFO:teuthology.orchestra.run.smithi196.stdout:1541 punch from 0x1026400 to 0x102e000, (0x7c00 bytes) 2024-05-08T08:52:43.248 INFO:teuthology.orchestra.run.smithi196.stdout:1542 write 0x5d9b600 thru 0x5da4294 (0x8c95 bytes) 2024-05-08T08:52:43.248 INFO:teuthology.orchestra.run.smithi196.stdout:1543 read 0x274a000 thru 0x2750cb0 (0x6cb1 bytes) 2024-05-08T08:52:43.248 INFO:teuthology.orchestra.run.smithi196.stdout:1544 trunc from 0x9c34e00 to 0x93c5600 2024-05-08T08:52:43.271 INFO:teuthology.orchestra.run.smithi196.stdout:1547 punch from 0x5dbc800 to 0x5dc2600, (0x5e00 bytes) 2024-05-08T08:52:43.274 INFO:teuthology.orchestra.run.smithi196.stdout:1548 write 0x76a5800 thru 0x76b1807 (0xc008 bytes) 2024-05-08T08:52:43.274 INFO:teuthology.orchestra.run.smithi196.stdout:1551 write 0x6bc1600 thru 0x6bc1a99 (0x49a bytes) 2024-05-08T08:52:43.274 INFO:teuthology.orchestra.run.smithi196.stdout:1555 write 0x72d1600 thru 0x72dcc9e (0xb69f bytes) 2024-05-08T08:52:43.274 INFO:teuthology.orchestra.run.smithi196.stdout:1556 read 0x3ffd400 thru 0x400a123 (0xcd24 bytes) 2024-05-08T08:52:43.279 INFO:teuthology.orchestra.run.smithi196.stdout:1557 trunc from 0x93c5600 to 0xdaca600 2024-05-08T08:52:43.308 INFO:teuthology.orchestra.run.smithi196.stdout:1559 trunc from 0xdaca600 to 0x6e61200 2024-05-08T08:52:43.330 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:52:43.333+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:52:43.340 INFO:teuthology.orchestra.run.smithi196.stdout:1560 write 0xe50aa00 thru 0xe50fe89 (0x548a bytes) 2024-05-08T08:52:43.340 INFO:teuthology.orchestra.run.smithi196.stdout:1562 read 0x2661800 thru 0x266f82a (0xe02b bytes) 2024-05-08T08:52:43.341 INFO:teuthology.orchestra.run.smithi196.stdout:1563 read 0x2145a00 thru 0x215271e (0xcd1f bytes) 2024-05-08T08:52:43.341 INFO:teuthology.orchestra.run.smithi196.stdout:1566 write 0xaf63e00 thru 0xaf64624 (0x825 bytes) 2024-05-08T08:52:43.341 INFO:teuthology.orchestra.run.smithi196.stdout:1568 read 0x2d4c200 thru 0x2d5937f (0xd180 bytes) 2024-05-08T08:52:43.342 INFO:teuthology.orchestra.run.smithi196.stdout:1569 read 0xb1ad800 thru 0xb1bcaac (0xf2ad bytes) 2024-05-08T08:52:43.342 INFO:teuthology.orchestra.run.smithi196.stdout:1570 punch from 0x1b9de00 to 0x1ba7a00, (0x9c00 bytes) 2024-05-08T08:52:43.342 INFO:teuthology.orchestra.run.smithi196.stdout:1572 punch from 0x9551400 to 0x955ec00, (0xd800 bytes) 2024-05-08T08:52:43.342 INFO:teuthology.orchestra.run.smithi196.stdout:1573 read 0x7435a00 thru 0x743a77d (0x4d7e bytes) 2024-05-08T08:52:43.343 INFO:teuthology.orchestra.run.smithi196.stdout:1574 read 0x25b200 thru 0x25f6b5 (0x44b6 bytes) 2024-05-08T08:52:43.345 INFO:teuthology.orchestra.run.smithi196.stdout:1576 write 0x5879a00 thru 0x5884702 (0xad03 bytes) 2024-05-08T08:52:43.345 INFO:teuthology.orchestra.run.smithi196.stdout:1578 write 0x3321e00 thru 0x33291aa (0x73ab bytes) 2024-05-08T08:52:43.345 INFO:teuthology.orchestra.run.smithi196.stdout:1579 punch from 0x436fa00 to 0x4372000, (0x2600 bytes) 2024-05-08T08:52:43.345 INFO:teuthology.orchestra.run.smithi196.stdout:1580 read 0xd718000 thru 0xd72018d (0x818e bytes) 2024-05-08T08:52:43.345 INFO:teuthology.orchestra.run.smithi196.stdout:1581 write 0xe44c600 thru 0xe45b5f5 (0xeff6 bytes) 2024-05-08T08:52:43.345 INFO:teuthology.orchestra.run.smithi196.stdout:1584 punch from 0xaeb9400 to 0xaebee00, (0x5a00 bytes) 2024-05-08T08:52:43.359 INFO:teuthology.orchestra.run.smithi196.stdout:1585 trunc from 0xe510000 to 0x3ac8c00 2024-05-08T08:52:43.401 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:43.401 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:43.410 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:52:43.418 INFO:teuthology.orchestra.run.smithi196.stdout:1586 trunc from 0x3ac8c00 to 0x26df400 2024-05-08T08:52:43.431 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:52:43.434+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:52:43.435 INFO:teuthology.orchestra.run.smithi196.stdout:1587 punch from 0x2600400 to 0x260c400, (0xc000 bytes) 2024-05-08T08:52:43.459 INFO:teuthology.orchestra.run.smithi196.stdout:1589 write 0xb821200 thru 0xb82cd04 (0xbb05 bytes) 2024-05-08T08:52:43.459 INFO:teuthology.orchestra.run.smithi196.stdout:1590 write 0x6bc7a00 thru 0x6bcfc3c (0x823d bytes) 2024-05-08T08:52:43.463 INFO:teuthology.orchestra.run.smithi196.stdout:1591 trunc from 0xb82ce00 to 0x7b0dc00 2024-05-08T08:52:43.490 INFO:teuthology.orchestra.run.smithi196.stdout:1592 write 0x55b2600 thru 0x55b4e86 (0x2887 bytes) 2024-05-08T08:52:43.498 INFO:teuthology.orchestra.run.smithi196.stdout:1594 trunc from 0x7b0dc00 to 0x19e7800 2024-05-08T08:52:43.518 INFO:teuthology.orchestra.run.smithi196.stdout:1595 read 0xb21c00 thru 0xb2eeb9 (0xd2ba bytes) 2024-05-08T08:52:43.524 INFO:teuthology.orchestra.run.smithi196.stdout:1598 read 0x11bbc00 thru 0x11c5908 (0x9d09 bytes) 2024-05-08T08:52:43.525 INFO:teuthology.orchestra.run.smithi196.stdout:1600 write 0x1008e00 thru 0x1010be3 (0x7de4 bytes) 2024-05-08T08:52:43.530 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:43.530 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:43.530 INFO:teuthology.orchestra.run.smithi196.stdout:1601 trunc from 0x19e7800 to 0x5a10c00 2024-05-08T08:52:43.531 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:52:43.535+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:52:43.541 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:52:43.542 INFO:teuthology.orchestra.run.smithi196.stdout:1602 read 0x51ae400 thru 0x51bbd07 (0xd908 bytes) 2024-05-08T08:52:43.543 INFO:teuthology.orchestra.run.smithi196.stdout:1603 write 0x5094e00 thru 0x50974b5 (0x26b6 bytes) 2024-05-08T08:52:43.543 INFO:teuthology.orchestra.run.smithi196.stdout:1606 read 0x3024e00 thru 0x3031fed (0xd1ee bytes) 2024-05-08T08:52:43.558 INFO:teuthology.orchestra.run.smithi196.stdout:1609 write 0x5d63a00 thru 0x5d6f703 (0xbd04 bytes) 2024-05-08T08:52:43.559 INFO:teuthology.orchestra.run.smithi196.stdout:1611 punch from 0x39ed000 to 0x39fc400, (0xf400 bytes) 2024-05-08T08:52:43.560 INFO:teuthology.orchestra.run.smithi196.stdout:1612 read 0x5446200 thru 0x5455c7c (0xfa7d bytes) 2024-05-08T08:52:43.561 INFO:teuthology.orchestra.run.smithi196.stdout:1613 write 0x26f3200 thru 0x26fcd84 (0x9b85 bytes) 2024-05-08T08:52:43.561 INFO:teuthology.orchestra.run.smithi196.stdout:1614 write 0x59c8000 thru 0x59ca351 (0x2352 bytes) 2024-05-08T08:52:43.583 INFO:teuthology.orchestra.run.smithi196.stdout:1615 write 0x8c93e00 thru 0x8c9f494 (0xb695 bytes) 2024-05-08T08:52:43.599 INFO:teuthology.orchestra.run.smithi196.stdout:1616 write 0x9599800 thru 0x95a29b8 (0x91b9 bytes) 2024-05-08T08:52:43.600 INFO:teuthology.orchestra.run.smithi196.stdout:1624 read 0x91bf800 thru 0x91ce779 (0xef7a bytes) 2024-05-08T08:52:43.604 INFO:teuthology.orchestra.run.smithi196.stdout:1626 read 0x7b90200 thru 0x7b94ec2 (0x4cc3 bytes) 2024-05-08T08:52:43.604 INFO:teuthology.orchestra.run.smithi196.stdout:1627 read 0x3b3f200 thru 0x3b47fe0 (0x8de1 bytes) 2024-05-08T08:52:43.620 INFO:teuthology.orchestra.run.smithi196.stdout:1629 write 0xba8bc00 thru 0xba905a4 (0x49a5 bytes) 2024-05-08T08:52:43.626 INFO:teuthology.orchestra.run.smithi196.stdout:1630 write 0x4136c00 thru 0x4136ee8 (0x2e9 bytes) 2024-05-08T08:52:43.626 INFO:teuthology.orchestra.run.smithi196.stdout:1634 write 0xaca5a00 thru 0xaca939c (0x399d bytes) 2024-05-08T08:52:43.633 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:52:43.636+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:52:43.640 INFO:teuthology.orchestra.run.smithi196.stdout:1637 trunc from 0xba90600 to 0x3b1200 2024-05-08T08:52:43.670 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:43.670 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:43.679 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:52:43.702 INFO:teuthology.orchestra.run.smithi196.stdout:1640 write 0x7b29000 thru 0x7b37509 (0xe50a bytes) 2024-05-08T08:52:43.709 INFO:teuthology.orchestra.run.smithi196.stdout:1641 trunc from 0x7b37600 to 0xcbb0800 2024-05-08T08:52:43.733 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:52:43.736+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:52:43.742 INFO:teuthology.orchestra.run.smithi196.stdout:1648 trunc from 0xcbb0800 to 0x5bfb400 2024-05-08T08:52:43.792 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:43.792 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:43.801 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:52:43.842 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:43.845+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:43.911 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:43.911 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:43.920 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:52:43.935 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:43.938+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:44.011 INFO:teuthology.orchestra.run.smithi196.stdout:1651 read 0x36be200 thru 0x36cd7e5 (0xf5e6 bytes) 2024-05-08T08:52:44.020 INFO:teuthology.orchestra.run.smithi196.stdout:1655 trunc from 0x5bfb400 to 0xcb7ec00 2024-05-08T08:52:44.029 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:44.029 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:44.033 INFO:teuthology.orchestra.run.smithi196.stdout:1660 read 0xb8c9c00 thru 0xb8d019f (0x65a0 bytes) 2024-05-08T08:52:44.035 INFO:teuthology.orchestra.run.smithi196.stdout:1662 punch from 0xb96fe00 to 0xb97d400, (0xd600 bytes) 2024-05-08T08:52:44.035 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:52:44.038+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:52:44.036 INFO:teuthology.orchestra.run.smithi196.stdout:1663 read 0xbd64200 thru 0xbd6d331 (0x9132 bytes) 2024-05-08T08:52:44.037 INFO:teuthology.orchestra.run.smithi196.stdout:1665 write 0xa78800 thru 0xa86814 (0xe015 bytes) 2024-05-08T08:52:44.037 INFO:teuthology.orchestra.run.smithi196.stdout:1666 write 0x1cd7400 thru 0x1cdabaf (0x37b0 bytes) 2024-05-08T08:52:44.039 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:52:44.039 INFO:teuthology.orchestra.run.smithi196.stdout:1671 trunc from 0xcb7ec00 to 0xe78d800 2024-05-08T08:52:44.042 INFO:teuthology.orchestra.run.smithi059.stderr:nodeep-scrub is unset 2024-05-08T08:52:44.069 INFO:teuthology.orchestra.run.smithi196.stdout:1672 trunc from 0xe78d800 to 0xb7f3c00 2024-05-08T08:52:44.080 INFO:teuthology.orchestra.run.smithi196.stdout:1674 write 0x828be00 thru 0x8290875 (0x4a76 bytes) 2024-05-08T08:52:44.081 INFO:teuthology.orchestra.run.smithi196.stdout:1680 punch from 0x5ae1c00 to 0x5aea200, (0x8600 bytes) 2024-05-08T08:52:44.086 INFO:teuthology.orchestra.run.smithi196.stdout:1682 read 0x866fa00 thru 0x867701b (0x761c bytes) 2024-05-08T08:52:44.090 INFO:teuthology.orchestra.run.smithi196.stdout:1684 read 0x1e49600 thru 0x1e49fd4 (0x9d5 bytes) 2024-05-08T08:52:44.091 INFO:teuthology.orchestra.run.smithi196.stdout:1686 read 0x7ecb800 thru 0x7eda9e9 (0xf1ea bytes) 2024-05-08T08:52:44.091 INFO:teuthology.orchestra.run.smithi196.stdout:1687 read 0x35a0a00 thru 0x35a30b4 (0x26b5 bytes) 2024-05-08T08:52:44.091 INFO:teuthology.orchestra.run.smithi196.stdout:1688 write 0x766a400 thru 0x7674fec (0xabed bytes) 2024-05-08T08:52:44.115 INFO:teuthology.orchestra.run.smithi196.stdout:1689 write 0xbfefe00 thru 0xbff4493 (0x4694 bytes) 2024-05-08T08:52:44.116 INFO:teuthology.orchestra.run.smithi196.stdout:1691 read 0x2810600 thru 0x281f28d (0xec8e bytes) 2024-05-08T08:52:44.116 INFO:teuthology.orchestra.run.smithi196.stdout:1692 write 0x8a21c00 thru 0x8a26d3b (0x513c bytes) 2024-05-08T08:52:44.116 INFO:teuthology.orchestra.run.smithi196.stdout:1693 read 0xa797c00 thru 0xa797c79 (0x7a bytes) 2024-05-08T08:52:44.116 INFO:teuthology.orchestra.run.smithi196.stdout:1694 read 0xca6a00 thru 0xcb187e (0xae7f bytes) 2024-05-08T08:52:44.116 INFO:teuthology.orchestra.run.smithi196.stdout:1697 read 0x3596a00 thru 0x3597c3f (0x1240 bytes) 2024-05-08T08:52:44.116 INFO:teuthology.orchestra.run.smithi196.stdout:1698 write 0x5c0ac00 thru 0x5c133b6 (0x87b7 bytes) 2024-05-08T08:52:44.116 INFO:teuthology.orchestra.run.smithi196.stdout:1700 read 0x696e800 thru 0x696e93f (0x140 bytes) 2024-05-08T08:52:44.117 INFO:teuthology.orchestra.run.smithi196.stdout:1701 trunc from 0xbff4600 to 0xaf1fa00 2024-05-08T08:52:44.136 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:52:44.140+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:52:44.141 INFO:teuthology.orchestra.run.smithi196.stdout:1702 read 0x75b4c00 thru 0x75c040c (0xb80d bytes) 2024-05-08T08:52:44.143 INFO:teuthology.orchestra.run.smithi196.stdout:1704 write 0x3d92e00 thru 0x3d97028 (0x4229 bytes) 2024-05-08T08:52:44.143 INFO:teuthology.orchestra.run.smithi196.stdout:1707 read 0x2b46600 thru 0x2b52fd7 (0xc9d8 bytes) 2024-05-08T08:52:44.143 INFO:teuthology.orchestra.run.smithi196.stdout:1709 write 0x5705e00 thru 0x570ddba (0x7fbb bytes) 2024-05-08T08:52:44.143 INFO:teuthology.orchestra.run.smithi196.stdout:1713 write 0x173e000 thru 0x174661c (0x861d bytes) 2024-05-08T08:52:44.150 INFO:teuthology.orchestra.run.smithi196.stdout:1716 trunc from 0xaf1fa00 to 0x4e28200 2024-05-08T08:52:44.181 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:44.181 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:44.182 INFO:teuthology.orchestra.run.smithi196.stdout:1719 read 0x4058800 thru 0x4065ebf (0xd6c0 bytes) 2024-05-08T08:52:44.190 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:52:44.191 INFO:teuthology.orchestra.run.smithi196.stdout:1720 trunc from 0x4e28200 to 0xb09da00 2024-05-08T08:52:44.200 INFO:teuthology.orchestra.run.smithi196.stdout:1721 write 0x4883c00 thru 0x48895eb (0x59ec bytes) 2024-05-08T08:52:44.201 INFO:teuthology.orchestra.run.smithi196.stdout:1722 punch from 0x4d5f800 to 0x4d61e00, (0x2600 bytes) 2024-05-08T08:52:44.206 INFO:teuthology.orchestra.run.smithi196.stdout:1724 read 0x51d0a00 thru 0x51d99a7 (0x8fa8 bytes) 2024-05-08T08:52:44.208 INFO:teuthology.orchestra.run.smithi196.stdout:1725 trunc from 0xb09da00 to 0x9207200 2024-05-08T08:52:44.219 INFO:teuthology.orchestra.run.smithi196.stdout:1726 write 0xb86200 thru 0xb92c13 (0xca14 bytes) 2024-05-08T08:52:44.237 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:44.241+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:44.238 INFO:teuthology.orchestra.run.smithi196.stdout:1727 write 0xc4d4800 thru 0xc4e4641 (0xfe42 bytes) 2024-05-08T08:52:44.238 INFO:teuthology.orchestra.run.smithi196.stdout:1731 write 0x5ddfa00 thru 0x5de74eb (0x7aec bytes) 2024-05-08T08:52:44.238 INFO:teuthology.orchestra.run.smithi196.stdout:1732 punch from 0x6055c00 to 0x605f400, (0x9800 bytes) 2024-05-08T08:52:44.240 INFO:teuthology.orchestra.run.smithi196.stdout:1734 write 0x672f600 thru 0x673ee62 (0xf863 bytes) 2024-05-08T08:52:44.240 INFO:teuthology.orchestra.run.smithi196.stdout:1735 write 0x938e600 thru 0x939bec0 (0xd8c1 bytes) 2024-05-08T08:52:44.240 INFO:teuthology.orchestra.run.smithi196.stdout:1737 punch from 0x3152c00 to 0x3154800, (0x1c00 bytes) 2024-05-08T08:52:44.241 INFO:teuthology.orchestra.run.smithi196.stdout:1741 read 0x392e00 thru 0x39abad (0x7dae bytes) 2024-05-08T08:52:44.245 INFO:teuthology.orchestra.run.smithi196.stdout:1742 trunc from 0xc4e4800 to 0x97e0800 2024-05-08T08:52:44.281 INFO:teuthology.orchestra.run.smithi196.stdout:1744 read 0x93e1600 thru 0x93eb063 (0x9a64 bytes) 2024-05-08T08:52:44.282 INFO:teuthology.orchestra.run.smithi196.stdout:1745 write 0x18f8600 thru 0x18fad5f (0x2760 bytes) 2024-05-08T08:52:44.282 INFO:teuthology.orchestra.run.smithi196.stdout:1746 read 0x4c06e00 thru 0x4c0b523 (0x4724 bytes) 2024-05-08T08:52:44.282 INFO:teuthology.orchestra.run.smithi196.stdout:1747 read 0x9b1200 thru 0x9b99ef (0x87f0 bytes) 2024-05-08T08:52:44.282 INFO:teuthology.orchestra.run.smithi196.stdout:1750 write 0x4c0fa00 thru 0x4c1ae56 (0xb457 bytes) 2024-05-08T08:52:44.283 INFO:teuthology.orchestra.run.smithi196.stdout:1752 read 0xdc9200 thru 0xdd4bf8 (0xb9f9 bytes) 2024-05-08T08:52:44.286 INFO:teuthology.orchestra.run.smithi196.stdout:1753 trunc from 0x97e0800 to 0x72cd600 2024-05-08T08:52:44.305 INFO:teuthology.orchestra.run.smithi196.stdout:1754 write 0x594ae00 thru 0x5951388 (0x6589 bytes) 2024-05-08T08:52:44.321 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:44.322 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:44.331 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:52:44.334 INFO:teuthology.orchestra.run.smithi196.stdout:1755 write 0xb04fe00 thru 0xb05c2f1 (0xc4f2 bytes) 2024-05-08T08:52:44.334 INFO:teuthology.orchestra.run.smithi196.stdout:1756 read 0x42aac00 thru 0x42b884c (0xdc4d bytes) 2024-05-08T08:52:44.335 INFO:teuthology.orchestra.run.smithi196.stdout:1757 read 0x4517200 thru 0x45196b4 (0x24b5 bytes) 2024-05-08T08:52:44.335 INFO:teuthology.orchestra.run.smithi196.stdout:1759 read 0x30e5a00 thru 0x30eb01f (0x5620 bytes) 2024-05-08T08:52:44.335 INFO:teuthology.orchestra.run.smithi196.stdout:1761 punch from 0x842b000 to 0x842da00, (0x2a00 bytes) 2024-05-08T08:52:44.357 INFO:teuthology.orchestra.run.smithi196.stdout:1762 write 0xc961200 thru 0xc96c99d (0xb79e bytes) 2024-05-08T08:52:44.357 INFO:teuthology.orchestra.run.smithi196.stdout:1763 punch from 0x598b600 to 0x5992400, (0x6e00 bytes) 2024-05-08T08:52:44.358 INFO:teuthology.orchestra.run.smithi196.stdout:1765 punch from 0x4a8cc00 to 0x4a8dc00, (0x1000 bytes) 2024-05-08T08:52:44.367 INFO:teuthology.orchestra.run.smithi196.stdout:1766 trunc from 0xc96ca00 to 0x5b42800 2024-05-08T08:52:44.375 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:52:44.378+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:52:44.396 INFO:teuthology.orchestra.run.smithi196.stdout:1767 write 0x2964200 thru 0x29675b4 (0x33b5 bytes) 2024-05-08T08:52:44.397 INFO:teuthology.orchestra.run.smithi196.stdout:1768 read 0x1ef200 thru 0x1f4640 (0x5441 bytes) 2024-05-08T08:52:44.397 INFO:teuthology.orchestra.run.smithi196.stdout:1770 read 0x5914a00 thru 0x591ee2c (0xa42d bytes) 2024-05-08T08:52:44.422 INFO:teuthology.orchestra.run.smithi196.stdout:1772 write 0xb6f7600 thru 0xb7029cb (0xb3cc bytes) 2024-05-08T08:52:44.422 INFO:teuthology.orchestra.run.smithi196.stdout:1773 read 0x35c200 thru 0x35e0bb (0x1ebc bytes) 2024-05-08T08:52:44.422 INFO:teuthology.orchestra.run.smithi196.stdout:1774 write 0x6ae8400 thru 0x6af4da7 (0xc9a8 bytes) 2024-05-08T08:52:44.422 INFO:teuthology.orchestra.run.smithi196.stdout:1775 write 0x128e00 thru 0x130d6d (0x7f6e bytes) 2024-05-08T08:52:44.422 INFO:teuthology.orchestra.run.smithi196.stdout:1776 read 0x4566c00 thru 0x4570436 (0x9837 bytes) 2024-05-08T08:52:44.423 INFO:teuthology.orchestra.run.smithi196.stdout:1778 write 0x58a3000 thru 0x58ae205 (0xb206 bytes) 2024-05-08T08:52:44.423 INFO:teuthology.orchestra.run.smithi196.stdout:1780 read 0x13d1c00 thru 0x13d2b2b (0xf2c bytes) 2024-05-08T08:52:44.424 INFO:teuthology.orchestra.run.smithi196.stdout:1782 read 0x97a2800 thru 0x97aa021 (0x7822 bytes) 2024-05-08T08:52:44.424 INFO:teuthology.orchestra.run.smithi196.stdout:1785 read 0x1a58000 thru 0x1a59ea4 (0x1ea5 bytes) 2024-05-08T08:52:44.424 INFO:teuthology.orchestra.run.smithi196.stdout:1788 punch from 0x498a600 to 0x4998600, (0xe000 bytes) 2024-05-08T08:52:44.434 INFO:teuthology.orchestra.run.smithi196.stdout:1790 trunc from 0xb702a00 to 0x3ae0200 2024-05-08T08:52:44.439 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:52:44.442+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:52:44.464 INFO:teuthology.orchestra.run.smithi196.stdout:1791 read 0x1a65600 thru 0x1a6e1e2 (0x8be3 bytes) 2024-05-08T08:52:44.465 INFO:teuthology.orchestra.run.smithi196.stdout:1792 read 0x1507400 thru 0x1511145 (0x9d46 bytes) 2024-05-08T08:52:44.468 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:44.468 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:44.479 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:52:44.481 INFO:teuthology.orchestra.run.smithi196.stdout:1795 write 0x8ba9400 thru 0x8bab09b (0x1c9c bytes) 2024-05-08T08:52:44.499 INFO:teuthology.orchestra.run.smithi196.stdout:1796 write 0xd2e0a00 thru 0xd2e60be (0x56bf bytes) 2024-05-08T08:52:44.507 INFO:teuthology.orchestra.run.smithi196.stdout:1798 write 0x9d00a00 thru 0x9d0e445 (0xda46 bytes) 2024-05-08T08:52:44.507 INFO:teuthology.orchestra.run.smithi196.stdout:1800 punch from 0x1858600 to 0x1863c00, (0xb600 bytes) 2024-05-08T08:52:44.508 INFO:teuthology.orchestra.run.smithi196.stdout:1802 read 0x2082c00 thru 0x208829b (0x569c bytes) 2024-05-08T08:52:44.508 INFO:teuthology.orchestra.run.smithi196.stdout:1803 read 0x8a2fe00 thru 0x8a37880 (0x7a81 bytes) 2024-05-08T08:52:44.509 INFO:teuthology.orchestra.run.smithi196.stdout:1804 read 0x106ec00 thru 0x106ef71 (0x372 bytes) 2024-05-08T08:52:44.509 INFO:teuthology.orchestra.run.smithi196.stdout:1805 read 0x1211600 thru 0x1219a6d (0x846e bytes) 2024-05-08T08:52:44.510 INFO:teuthology.orchestra.run.smithi196.stdout:1806 read 0x8a4f800 thru 0x8a50caa (0x14ab bytes) 2024-05-08T08:52:44.510 INFO:teuthology.orchestra.run.smithi196.stdout:1808 punch from 0x7398c00 to 0x73a1200, (0x8600 bytes) 2024-05-08T08:52:44.523 INFO:teuthology.orchestra.run.smithi196.stdout:1809 trunc from 0xd2e6200 to 0x2a48800 2024-05-08T08:52:44.543 INFO:teuthology.orchestra.run.smithi196.stdout:1811 read 0x283d400 thru 0x284c0fe (0xecff bytes) 2024-05-08T08:52:44.544 INFO:teuthology.orchestra.run.smithi196.stdout:1812 read 0x27800 thru 0x3496b (0xd16c bytes) 2024-05-08T08:52:44.549 INFO:teuthology.orchestra.run.smithi196.stdout:1813 punch from 0x1157e00 to 0x115d000, (0x5200 bytes) 2024-05-08T08:52:44.549 INFO:teuthology.orchestra.run.smithi196.stdout:1817 punch from 0x194cc00 to 0x1952a00, (0x5e00 bytes) 2024-05-08T08:52:44.549 INFO:teuthology.orchestra.run.smithi196.stdout:1818 write 0xc94400 thru 0xca32cd (0xeece bytes) 2024-05-08T08:52:44.551 INFO:teuthology.orchestra.run.smithi196.stdout:1819 write 0x29e0200 thru 0x29e04bf (0x2c0 bytes) 2024-05-08T08:52:44.551 INFO:teuthology.orchestra.run.smithi196.stdout:1820 read 0x4a3400 thru 0x4aaa6a (0x766b bytes) 2024-05-08T08:52:44.552 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T08:52:44.564 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:52:44.567+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:52:44.567 INFO:teuthology.orchestra.run.smithi196.stdout:1823 write 0x2b65400 thru 0x2b6a365 (0x4f66 bytes) 2024-05-08T08:52:44.568 INFO:teuthology.orchestra.run.smithi196.stdout:1824 read 0x17fa800 thru 0x1801704 (0x6f05 bytes) 2024-05-08T08:52:44.568 INFO:teuthology.orchestra.run.smithi196.stdout:1829 read 0xe05000 thru 0xe10316 (0xb317 bytes) 2024-05-08T08:52:44.569 INFO:teuthology.orchestra.run.smithi196.stdout:1830 read 0x1cf5800 thru 0x1cfc179 (0x697a bytes) 2024-05-08T08:52:44.572 INFO:teuthology.orchestra.run.smithi196.stdout:1831 read 0x2414400 thru 0x241b486 (0x7087 bytes) 2024-05-08T08:52:44.580 INFO:teuthology.orchestra.run.smithi196.stdout:1833 trunc from 0x2b6a400 to 0x8e7dc00 2024-05-08T08:52:44.612 INFO:teuthology.orchestra.run.smithi196.stdout:1834 write 0xd7c8000 thru 0xd7d4146 (0xc147 bytes) 2024-05-08T08:52:44.613 INFO:teuthology.orchestra.run.smithi196.stdout:1836 read 0x594c400 thru 0x5952bea (0x67eb bytes) 2024-05-08T08:52:44.613 INFO:teuthology.orchestra.run.smithi196.stdout:1837 write 0x8bd6200 thru 0x8bdc1b9 (0x5fba bytes) 2024-05-08T08:52:44.613 INFO:teuthology.orchestra.run.smithi196.stdout:1838 read 0xb656e00 thru 0xb6614ce (0xa6cf bytes) 2024-05-08T08:52:44.613 INFO:teuthology.orchestra.run.smithi196.stdout:1840 read 0x8f91c00 thru 0x8f97181 (0x5582 bytes) 2024-05-08T08:52:44.613 INFO:teuthology.orchestra.run.smithi196.stdout:1841 punch from 0xa16c800 to 0xa177400, (0xac00 bytes) 2024-05-08T08:52:44.617 INFO:teuthology.orchestra.run.smithi196.stdout:1844 read 0x2182400 thru 0x21864c6 (0x40c7 bytes) 2024-05-08T08:52:44.617 INFO:teuthology.orchestra.run.smithi196.stdout:1846 read 0x56af400 thru 0x56bf02b (0xfc2c bytes) 2024-05-08T08:52:44.641 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:44.644+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:44.648 INFO:teuthology.orchestra.run.smithi196.stdout:1847 write 0xe51b000 thru 0xe51e7d9 (0x37da bytes) 2024-05-08T08:52:44.648 INFO:teuthology.orchestra.run.smithi196.stdout:1848 write 0x6f91c00 thru 0x6f9d2b7 (0xb6b8 bytes) 2024-05-08T08:52:44.648 INFO:teuthology.orchestra.run.smithi196.stdout:1854 write 0x7df9c00 thru 0x7e05900 (0xbd01 bytes) 2024-05-08T08:52:44.648 INFO:teuthology.orchestra.run.smithi196.stdout:1855 read 0xc56ca00 thru 0xc5724bc (0x5abd bytes) 2024-05-08T08:52:44.648 INFO:teuthology.orchestra.run.smithi196.stdout:1859 punch from 0x4352000 to 0x4352400, (0x400 bytes) 2024-05-08T08:52:44.648 INFO:teuthology.orchestra.run.smithi196.stdout:1862 write 0x467a600 thru 0x4687430 (0xce31 bytes) 2024-05-08T08:52:44.648 INFO:teuthology.orchestra.run.smithi196.stdout:1865 read 0xd830e00 thru 0xd83718c (0x638d bytes) 2024-05-08T08:52:44.649 INFO:teuthology.orchestra.run.smithi196.stdout:1866 write 0xcf2fc00 thru 0xcf30004 (0x405 bytes) 2024-05-08T08:52:44.649 INFO:teuthology.orchestra.run.smithi196.stdout:1869 punch from 0xcdc9600 to 0xcdd3000, (0x9a00 bytes) 2024-05-08T08:52:44.649 INFO:teuthology.orchestra.run.smithi196.stdout:1872 write 0x48ea00 thru 0x493167 (0x4768 bytes) 2024-05-08T08:52:44.649 INFO:teuthology.orchestra.run.smithi196.stdout:1875 write 0x5cec00 thru 0x5d6869 (0x7c6a bytes) 2024-05-08T08:52:44.650 INFO:teuthology.orchestra.run.smithi196.stdout:1881 write 0x441e00 thru 0x44f785 (0xd986 bytes) 2024-05-08T08:52:44.650 INFO:teuthology.orchestra.run.smithi196.stdout:1884 read 0xa5b1400 thru 0xa5bf94e (0xe54f bytes) 2024-05-08T08:52:44.659 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:44.659 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:44.662 INFO:teuthology.orchestra.run.smithi196.stdout:1886 trunc from 0xe51e800 to 0x4b4f600 2024-05-08T08:52:44.672 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:52:44.715 INFO:teuthology.orchestra.run.smithi196.stdout:1888 read 0x47de00 thru 0x48d539 (0xf73a bytes) 2024-05-08T08:52:44.722 INFO:teuthology.orchestra.run.smithi196.stdout:1891 trunc from 0x4b4f600 to 0x1c2ee00 2024-05-08T08:52:44.723 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:52:44.723 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:52:44.742 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:52:44.745+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:52:44.748 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:52:44.748 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:52:44.763 INFO:teuthology.orchestra.run.smithi196.stdout:1892 write 0x8ca1600 thru 0x8cab432 (0x9e33 bytes) 2024-05-08T08:52:44.763 INFO:teuthology.orchestra.run.smithi196.stdout:1893 write 0x630b600 thru 0x631444f (0x8e50 bytes) 2024-05-08T08:52:44.768 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:52:44.768 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:52:44.791 INFO:teuthology.orchestra.run.smithi196.stdout:1894 write 0xcdc9a00 thru 0xcdd5fad (0xc5ae bytes) 2024-05-08T08:52:44.792 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:52:44.792 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:52:44.797 INFO:teuthology.orchestra.run.smithi196.stdout:1895 read 0x5c4ba00 thru 0x5c57a53 (0xc054 bytes) 2024-05-08T08:52:44.798 INFO:teuthology.orchestra.run.smithi196.stdout:1897 read 0xc888e00 thru 0xc88fb00 (0x6d01 bytes) 2024-05-08T08:52:44.808 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:52:44.809 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:52:44.809 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:52:44.809 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:52:44.810 INFO:teuthology.orchestra.run.smithi196.stdout:1898 trunc from 0xcdd6000 to 0x3006600 2024-05-08T08:52:44.810 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:52:44.811 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:52:44.820 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:52:44.834 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:52:44.834 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:52:44.846 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:52:44.849+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:52:44.856 INFO:teuthology.orchestra.run.smithi196.stdout:1900 write 0x9893e00 thru 0x98972ed (0x34ee bytes) 2024-05-08T08:52:44.864 INFO:teuthology.orchestra.run.smithi196.stdout:1902 clone 10 order 24 su 4194304 sc 6 2024-05-08T08:52:44.944 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:52:44.947+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:52:44.947 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:52:44.947 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:52:44.947 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:52:44.947 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:52:44.948 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:52:44.956 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:52:45.045 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:45.048+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:45.111 INFO:teuthology.orchestra.run.smithi196.stdout:truncating image image_client.0-clone9 from 0x9897400 (overlap 0x3b1200) to 0x1d8800 2024-05-08T08:52:45.114 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:52:45.115 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:52:45.115 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:52:45.115 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:52:45.115 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.115 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7490 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 0~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:45.115 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.744226+0000", 2024-05-08T08:52:45.115 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 7.372678359, 2024-05-08T08:52:45.115 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057294818999999997, 2024-05-08T08:52:45.115 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:45.115 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:45.115 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:45.115 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:45.115 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:45.116 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:45.116 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7490 2024-05-08T08:52:45.116 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.116 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:45.116 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.116 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:45.116 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744226+0000", 2024-05-08T08:52:45.116 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:45.116 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.116 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.116 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:45.116 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744226+0000", 2024-05-08T08:52:45.116 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:45.116 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.116 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.117 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:45.117 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744226+0000", 2024-05-08T08:52:45.117 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3799999999999998e-07 2024-05-08T08:52:45.117 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.117 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.117 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:45.117 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744228+0000", 2024-05-08T08:52:45.117 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0710000000000002e-06 2024-05-08T08:52:45.117 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.117 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.117 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:45.117 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744233+0000", 2024-05-08T08:52:45.117 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0130000000000001e-06 2024-05-08T08:52:45.117 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.117 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.118 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:45.118 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744237+0000", 2024-05-08T08:52:45.118 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1930000000000002e-06 2024-05-08T08:52:45.118 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.118 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.118 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:45.118 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.801334+0000", 2024-05-08T08:52:45.118 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057097004999999999 2024-05-08T08:52:45.118 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.118 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.118 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:45.118 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.801414+0000", 2024-05-08T08:52:45.118 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0129000000000005e-05 2024-05-08T08:52:45.118 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.118 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.119 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:45.119 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.801521+0000", 2024-05-08T08:52:45.119 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010707 2024-05-08T08:52:45.119 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:45.119 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:45.119 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:45.119 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.119 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.119 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7491 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 40960~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:45.119 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.744242+0000", 2024-05-08T08:52:45.119 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 7.3726625280000002, 2024-05-08T08:52:45.119 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057541677999999999, 2024-05-08T08:52:45.119 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:45.119 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:45.119 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:45.120 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:45.120 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:45.120 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:45.120 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7491 2024-05-08T08:52:45.120 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.120 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:45.120 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.120 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:45.120 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744242+0000", 2024-05-08T08:52:45.120 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:45.120 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.120 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.120 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:45.120 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744242+0000", 2024-05-08T08:52:45.120 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:45.120 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.121 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.121 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:45.121 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744242+0000", 2024-05-08T08:52:45.121 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.2000000000000001e-07 2024-05-08T08:52:45.121 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.121 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.121 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:45.121 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744244+0000", 2024-05-08T08:52:45.121 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6959999999999999e-06 2024-05-08T08:52:45.121 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.121 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.121 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:45.121 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744247+0000", 2024-05-08T08:52:45.121 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3000000000000002e-06 2024-05-08T08:52:45.121 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.121 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.121 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:45.122 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744250+0000", 2024-05-08T08:52:45.122 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.5399999999999998e-06 2024-05-08T08:52:45.122 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.122 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.122 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:45.122 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.801564+0000", 2024-05-08T08:52:45.122 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057313809 2024-05-08T08:52:45.122 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.122 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.122 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:45.122 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.801671+0000", 2024-05-08T08:52:45.122 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000107333 2024-05-08T08:52:45.122 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.122 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.123 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:45.123 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.801784+0000", 2024-05-08T08:52:45.123 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011268 2024-05-08T08:52:45.123 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:45.123 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:45.123 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:45.123 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.123 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.123 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7492 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 81920~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:45.123 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.744253+0000", 2024-05-08T08:52:45.123 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 7.3726510510000001, 2024-05-08T08:52:45.123 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057761380000000001, 2024-05-08T08:52:45.123 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:45.123 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:45.123 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:45.123 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:45.124 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:45.124 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:45.124 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7492 2024-05-08T08:52:45.124 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.124 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:45.124 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.124 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:45.124 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744253+0000", 2024-05-08T08:52:45.124 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:45.124 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.124 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.124 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:45.124 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744253+0000", 2024-05-08T08:52:45.124 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:45.125 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.125 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.125 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:45.125 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.745991+0000", 2024-05-08T08:52:45.125 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001738021 2024-05-08T08:52:45.125 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.125 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.125 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:45.125 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.745997+0000", 2024-05-08T08:52:45.125 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5300000000000004e-06 2024-05-08T08:52:45.125 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.125 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.125 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:45.125 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.746006+0000", 2024-05-08T08:52:45.126 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2210000000000003e-06 2024-05-08T08:52:45.126 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.126 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.126 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:45.126 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.746026+0000", 2024-05-08T08:52:45.126 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9487000000000002e-05 2024-05-08T08:52:45.126 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.126 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.126 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:45.126 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.801815+0000", 2024-05-08T08:52:45.126 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.055789196999999999 2024-05-08T08:52:45.126 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.126 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.127 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:45.127 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.801908+0000", 2024-05-08T08:52:45.127 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3320999999999996e-05 2024-05-08T08:52:45.127 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.127 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.127 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:45.127 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.802015+0000", 2024-05-08T08:52:45.127 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000106603 2024-05-08T08:52:45.127 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:45.127 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:45.127 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:45.127 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.127 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.127 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7499 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 368640~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:45.127 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.746209+0000", 2024-05-08T08:52:45.128 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 7.3706951030000001, 2024-05-08T08:52:45.128 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057256452999999999, 2024-05-08T08:52:45.128 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:45.128 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:45.128 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:45.128 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:45.128 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:45.128 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:45.128 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7499 2024-05-08T08:52:45.128 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.128 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:45.128 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.128 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:45.128 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.746209+0000", 2024-05-08T08:52:45.128 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:45.129 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.129 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.129 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:45.129 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.746209+0000", 2024-05-08T08:52:45.129 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:45.129 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.129 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.129 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:45.129 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.746210+0000", 2024-05-08T08:52:45.129 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2100000000000002e-07 2024-05-08T08:52:45.129 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.129 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.129 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:45.130 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.746212+0000", 2024-05-08T08:52:45.130 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.922e-06 2024-05-08T08:52:45.130 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.130 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.130 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:45.130 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.746221+0000", 2024-05-08T08:52:45.130 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2900000000000008e-06 2024-05-08T08:52:45.130 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.130 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.130 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:45.130 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.746227+0000", 2024-05-08T08:52:45.130 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3740000000000003e-06 2024-05-08T08:52:45.130 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.130 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.131 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:45.131 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.803304+0000", 2024-05-08T08:52:45.131 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057076239000000001 2024-05-08T08:52:45.131 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.131 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.131 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:45.131 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.803380+0000", 2024-05-08T08:52:45.131 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.6309000000000004e-05 2024-05-08T08:52:45.131 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.131 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.131 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:45.131 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.803466+0000", 2024-05-08T08:52:45.131 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.5897999999999996e-05 2024-05-08T08:52:45.131 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:45.131 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:45.131 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:45.132 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.132 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.132 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7500 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 409600~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:45.132 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.746237+0000", 2024-05-08T08:52:45.132 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 7.3706675209999997, 2024-05-08T08:52:45.132 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057429380000000002, 2024-05-08T08:52:45.132 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:45.132 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:45.132 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:45.132 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:45.132 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:45.132 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:45.132 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7500 2024-05-08T08:52:45.132 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.132 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:45.133 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.133 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:45.133 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.746237+0000", 2024-05-08T08:52:45.133 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:45.133 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.133 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.133 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:45.133 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.746237+0000", 2024-05-08T08:52:45.133 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:45.133 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.133 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.133 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:45.133 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.747959+0000", 2024-05-08T08:52:45.133 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001721888 2024-05-08T08:52:45.133 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.134 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.134 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:45.134 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.747964+0000", 2024-05-08T08:52:45.134 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0030000000000002e-06 2024-05-08T08:52:45.134 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.134 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.134 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:45.134 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.747976+0000", 2024-05-08T08:52:45.134 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2656999999999999e-05 2024-05-08T08:52:45.134 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.134 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.134 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:45.134 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.747983+0000", 2024-05-08T08:52:45.134 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.866e-06 2024-05-08T08:52:45.135 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.135 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.135 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:45.135 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.803495+0000", 2024-05-08T08:52:45.135 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.055511392999999999 2024-05-08T08:52:45.135 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.135 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.135 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:45.135 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.803574+0000", 2024-05-08T08:52:45.135 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9215000000000006e-05 2024-05-08T08:52:45.135 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.135 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.135 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:45.136 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.803666+0000", 2024-05-08T08:52:45.136 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2357999999999999e-05 2024-05-08T08:52:45.136 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:45.136 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:45.136 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:45.136 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.136 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.136 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7508 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 737280~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:45.136 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.748180+0000", 2024-05-08T08:52:45.136 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 7.3687244639999996, 2024-05-08T08:52:45.136 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057419809000000002, 2024-05-08T08:52:45.136 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:45.137 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:45.137 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:45.137 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:45.137 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:45.137 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:45.137 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7508 2024-05-08T08:52:45.137 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.137 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:45.137 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.137 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:45.137 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748180+0000", 2024-05-08T08:52:45.137 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:45.137 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.138 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.138 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:45.138 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748180+0000", 2024-05-08T08:52:45.138 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:45.138 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.138 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.138 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:45.138 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748180+0000", 2024-05-08T08:52:45.138 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4499999999999997e-07 2024-05-08T08:52:45.138 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.138 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.138 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:45.138 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748183+0000", 2024-05-08T08:52:45.139 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1660000000000001e-06 2024-05-08T08:52:45.139 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.139 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.139 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:45.139 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748197+0000", 2024-05-08T08:52:45.139 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.4877e-05 2024-05-08T08:52:45.139 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.139 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.139 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:45.139 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748203+0000", 2024-05-08T08:52:45.139 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7470000000000002e-06 2024-05-08T08:52:45.139 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.139 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.139 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:45.140 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.805398+0000", 2024-05-08T08:52:45.140 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057194513000000002 2024-05-08T08:52:45.140 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.140 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.140 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:45.140 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.805544+0000", 2024-05-08T08:52:45.140 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000146522 2024-05-08T08:52:45.140 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.140 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.140 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:45.140 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.805600+0000", 2024-05-08T08:52:45.140 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5538999999999998e-05 2024-05-08T08:52:45.140 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:45.140 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:45.141 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:45.141 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.141 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.141 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7509 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 778240~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:45.141 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.748211+0000", 2024-05-08T08:52:45.141 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 7.3686933950000002, 2024-05-08T08:52:45.141 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057619034999999999, 2024-05-08T08:52:45.141 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:45.141 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:45.141 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:45.141 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:45.141 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:45.141 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:45.141 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7509 2024-05-08T08:52:45.141 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.141 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:45.142 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.142 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:45.142 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748211+0000", 2024-05-08T08:52:45.142 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:45.142 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.142 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.142 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:45.142 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748211+0000", 2024-05-08T08:52:45.142 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:45.142 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.142 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.142 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:45.142 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748212+0000", 2024-05-08T08:52:45.142 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8100000000000003e-07 2024-05-08T08:52:45.143 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.143 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.143 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:45.143 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748214+0000", 2024-05-08T08:52:45.143 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4669999999999998e-06 2024-05-08T08:52:45.143 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.143 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.143 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:45.143 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748221+0000", 2024-05-08T08:52:45.143 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5769999999999999e-06 2024-05-08T08:52:45.143 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.143 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.143 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:45.143 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748225+0000", 2024-05-08T08:52:45.143 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8060000000000002e-06 2024-05-08T08:52:45.143 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.144 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.144 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:45.144 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.805636+0000", 2024-05-08T08:52:45.144 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057410309 2024-05-08T08:52:45.144 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.144 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.144 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:45.144 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.805723+0000", 2024-05-08T08:52:45.144 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7435999999999994e-05 2024-05-08T08:52:45.144 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.144 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.144 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:45.144 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.805830+0000", 2024-05-08T08:52:45.144 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000106959 2024-05-08T08:52:45.144 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:45.145 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:45.145 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:45.145 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.145 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.145 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7510 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 819200~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:45.145 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.748232+0000", 2024-05-08T08:52:45.145 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 7.3686723650000001, 2024-05-08T08:52:45.145 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057710421999999997, 2024-05-08T08:52:45.145 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:45.145 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:45.145 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:45.145 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:45.145 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:45.145 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:45.146 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7510 2024-05-08T08:52:45.146 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.146 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:45.146 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.146 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:45.146 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748232+0000", 2024-05-08T08:52:45.146 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:45.146 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.146 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.146 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:45.146 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748232+0000", 2024-05-08T08:52:45.146 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:45.146 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.147 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.147 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:45.147 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.749959+0000", 2024-05-08T08:52:45.147 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001727 2024-05-08T08:52:45.147 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.147 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.147 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:45.147 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.749964+0000", 2024-05-08T08:52:45.147 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4700000000000004e-06 2024-05-08T08:52:45.147 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.147 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.147 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:45.148 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.749984+0000", 2024-05-08T08:52:45.148 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0068000000000002e-05 2024-05-08T08:52:45.148 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.148 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.148 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:45.148 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.749993+0000", 2024-05-08T08:52:45.148 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4720000000000001e-06 2024-05-08T08:52:45.148 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.148 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.148 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:45.148 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.805846+0000", 2024-05-08T08:52:45.148 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.055852770000000003 2024-05-08T08:52:45.149 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.149 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.149 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:45.149 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.805890+0000", 2024-05-08T08:52:45.149 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4323999999999998e-05 2024-05-08T08:52:45.149 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.149 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.149 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:45.149 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.805942+0000", 2024-05-08T08:52:45.149 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2318000000000003e-05 2024-05-08T08:52:45.149 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:45.149 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:45.149 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:45.150 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.150 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.150 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7515 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 1024000~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:45.150 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.750135+0000", 2024-05-08T08:52:45.150 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 7.3667689889999997, 2024-05-08T08:52:45.150 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057188125999999999, 2024-05-08T08:52:45.150 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:45.150 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:45.150 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:45.150 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:45.150 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:45.150 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:45.151 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7515 2024-05-08T08:52:45.151 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.151 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:45.151 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.151 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:45.151 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750135+0000", 2024-05-08T08:52:45.151 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:45.151 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.151 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.151 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:45.151 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750135+0000", 2024-05-08T08:52:45.151 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:45.152 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.152 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.152 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:45.152 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750136+0000", 2024-05-08T08:52:45.152 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8100000000000003e-07 2024-05-08T08:52:45.152 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.152 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.152 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:45.152 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750138+0000", 2024-05-08T08:52:45.152 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.339e-06 2024-05-08T08:52:45.152 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.152 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.152 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:45.153 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750147+0000", 2024-05-08T08:52:45.153 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7669999999999992e-06 2024-05-08T08:52:45.153 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.153 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.153 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:45.153 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750153+0000", 2024-05-08T08:52:45.153 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5029999999999997e-06 2024-05-08T08:52:45.153 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.153 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.153 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:45.153 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.807145+0000", 2024-05-08T08:52:45.153 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.056992174 2024-05-08T08:52:45.154 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.154 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.154 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:45.154 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.807231+0000", 2024-05-08T08:52:45.154 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.6347000000000005e-05 2024-05-08T08:52:45.154 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.154 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.154 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:45.154 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.807324+0000", 2024-05-08T08:52:45.154 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2515000000000004e-05 2024-05-08T08:52:45.154 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:45.154 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:45.154 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:45.154 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.155 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.155 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7516 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 1064960~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:45.155 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.750162+0000", 2024-05-08T08:52:45.155 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 7.3667421600000003, 2024-05-08T08:52:45.155 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057363971999999999, 2024-05-08T08:52:45.155 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:45.155 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:45.155 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:45.155 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:45.155 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:45.155 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:45.155 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7516 2024-05-08T08:52:45.155 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.155 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:45.156 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.156 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:45.156 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750162+0000", 2024-05-08T08:52:45.156 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:45.156 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.156 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.156 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:45.156 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750162+0000", 2024-05-08T08:52:45.156 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:45.156 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.156 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.156 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:45.156 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750163+0000", 2024-05-08T08:52:45.156 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.13e-07 2024-05-08T08:52:45.157 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.157 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.157 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:45.157 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750165+0000", 2024-05-08T08:52:45.157 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2309999999999999e-06 2024-05-08T08:52:45.157 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.157 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.157 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:45.157 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750174+0000", 2024-05-08T08:52:45.157 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4740000000000005e-06 2024-05-08T08:52:45.157 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.157 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.158 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:45.158 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750180+0000", 2024-05-08T08:52:45.158 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4760000000000003e-06 2024-05-08T08:52:45.158 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.158 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.158 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:45.158 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.807355+0000", 2024-05-08T08:52:45.158 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057175029000000002 2024-05-08T08:52:45.158 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.158 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.158 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:45.158 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.807437+0000", 2024-05-08T08:52:45.158 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.2268999999999994e-05 2024-05-08T08:52:45.158 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.159 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.159 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:45.159 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.807526+0000", 2024-05-08T08:52:45.159 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.8880000000000003e-05 2024-05-08T08:52:45.159 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:45.159 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:45.159 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:45.159 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.159 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.159 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7517 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 1105920~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:45.159 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.750188+0000", 2024-05-08T08:52:45.159 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 7.3667166670000004, 2024-05-08T08:52:45.159 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057547099999999997, 2024-05-08T08:52:45.159 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:45.159 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:45.160 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:45.160 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:45.160 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:45.160 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:45.160 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7517 2024-05-08T08:52:45.160 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.160 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:45.160 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.160 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:45.160 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750188+0000", 2024-05-08T08:52:45.160 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:45.160 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.160 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.160 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:45.160 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750188+0000", 2024-05-08T08:52:45.161 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:45.161 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.161 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.161 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:45.161 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750188+0000", 2024-05-08T08:52:45.161 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7199999999999999e-07 2024-05-08T08:52:45.161 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.161 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.161 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:45.161 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750190+0000", 2024-05-08T08:52:45.161 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.154e-06 2024-05-08T08:52:45.161 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.161 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.162 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:45.162 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750198+0000", 2024-05-08T08:52:45.162 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8320000000000006e-06 2024-05-08T08:52:45.162 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.162 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.162 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:45.162 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750203+0000", 2024-05-08T08:52:45.162 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1900000000000003e-06 2024-05-08T08:52:45.162 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.162 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.162 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:45.162 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.807555+0000", 2024-05-08T08:52:45.162 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057351148999999997 2024-05-08T08:52:45.162 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.163 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.163 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:45.163 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.807640+0000", 2024-05-08T08:52:45.163 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.5853999999999995e-05 2024-05-08T08:52:45.163 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.163 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.163 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:45.163 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.807735+0000", 2024-05-08T08:52:45.163 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4449000000000003e-05 2024-05-08T08:52:45.163 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:45.163 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:45.163 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:45.163 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.163 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.164 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7518 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 1146880~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:45.164 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.750212+0000", 2024-05-08T08:52:45.164 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 7.3666928670000003, 2024-05-08T08:52:45.164 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057664963999999999, 2024-05-08T08:52:45.164 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:45.164 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:45.164 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:45.164 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:45.164 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:45.164 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:45.164 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7518 2024-05-08T08:52:45.164 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.164 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:45.164 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.165 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:45.165 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750212+0000", 2024-05-08T08:52:45.165 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:45.165 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.165 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.165 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:45.165 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750212+0000", 2024-05-08T08:52:45.165 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:45.165 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.165 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.165 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:45.165 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750212+0000", 2024-05-08T08:52:45.165 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.46e-07 2024-05-08T08:52:45.165 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.165 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.166 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:45.166 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750220+0000", 2024-05-08T08:52:45.166 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5610000000000001e-06 2024-05-08T08:52:45.166 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.166 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.166 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:45.166 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750228+0000", 2024-05-08T08:52:45.166 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3180000000000007e-06 2024-05-08T08:52:45.166 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.166 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.166 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:45.166 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750234+0000", 2024-05-08T08:52:45.166 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0209999999999996e-06 2024-05-08T08:52:45.167 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.167 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.167 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:45.167 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.807765+0000", 2024-05-08T08:52:45.167 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057531481000000002 2024-05-08T08:52:45.167 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.167 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.167 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:45.167 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.807815+0000", 2024-05-08T08:52:45.167 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9282e-05 2024-05-08T08:52:45.167 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.167 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.167 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:45.167 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.807877+0000", 2024-05-08T08:52:45.167 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1854999999999996e-05 2024-05-08T08:52:45.168 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:45.168 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:45.168 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:45.168 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.168 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.168 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7519 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 1187840~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:45.168 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.750261+0000", 2024-05-08T08:52:45.168 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 7.3666433810000003, 2024-05-08T08:52:45.168 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057872519999999997, 2024-05-08T08:52:45.168 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:45.168 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:45.168 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:45.168 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:45.168 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:45.169 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:45.169 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7519 2024-05-08T08:52:45.169 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.169 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:45.169 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.169 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:45.169 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750261+0000", 2024-05-08T08:52:45.169 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:45.169 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.169 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.169 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:45.169 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750261+0000", 2024-05-08T08:52:45.169 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:45.170 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.170 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.170 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:45.170 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750262+0000", 2024-05-08T08:52:45.170 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7000000000000005e-07 2024-05-08T08:52:45.170 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.170 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.170 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:45.170 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750265+0000", 2024-05-08T08:52:45.170 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4589999999999998e-06 2024-05-08T08:52:45.170 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.170 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.170 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:45.170 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750274+0000", 2024-05-08T08:52:45.171 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.5459999999999992e-06 2024-05-08T08:52:45.171 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.171 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.171 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:45.171 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750280+0000", 2024-05-08T08:52:45.171 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0950000000000001e-06 2024-05-08T08:52:45.171 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.171 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.171 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:45.171 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.807917+0000", 2024-05-08T08:52:45.171 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057637715999999999 2024-05-08T08:52:45.171 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.171 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.172 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:45.172 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.808013+0000", 2024-05-08T08:52:45.172 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5569999999999995e-05 2024-05-08T08:52:45.172 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.172 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.172 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:45.172 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.808134+0000", 2024-05-08T08:52:45.172 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012056400000000001 2024-05-08T08:52:45.172 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:45.172 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:45.172 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:45.172 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.172 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.172 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7520 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 1228800~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:45.173 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.750289+0000", 2024-05-08T08:52:45.173 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 7.3666149369999996, 2024-05-08T08:52:45.173 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.058133788999999998, 2024-05-08T08:52:45.173 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:45.173 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:45.173 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:45.173 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:45.173 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:45.173 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:45.173 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7520 2024-05-08T08:52:45.173 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.173 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:45.173 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.173 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:45.174 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750289+0000", 2024-05-08T08:52:45.174 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:45.174 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.174 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.174 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:45.174 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750289+0000", 2024-05-08T08:52:45.174 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:45.174 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.174 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.174 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:45.174 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.751976+0000", 2024-05-08T08:52:45.174 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0016865389999999999 2024-05-08T08:52:45.175 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.175 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.175 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:45.175 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.751981+0000", 2024-05-08T08:52:45.175 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5719999999999996e-06 2024-05-08T08:52:45.175 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.175 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.175 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:45.175 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.751991+0000", 2024-05-08T08:52:45.175 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0788000000000001e-05 2024-05-08T08:52:45.175 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.175 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.175 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:45.175 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.751999+0000", 2024-05-08T08:52:45.175 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2690000000000002e-06 2024-05-08T08:52:45.176 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.176 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.176 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:45.176 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.808183+0000", 2024-05-08T08:52:45.176 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.056184686999999997 2024-05-08T08:52:45.176 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.176 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.176 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:45.176 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.808296+0000", 2024-05-08T08:52:45.176 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000112754 2024-05-08T08:52:45.176 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.176 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.177 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:45.177 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.808423+0000", 2024-05-08T08:52:45.177 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012718000000000001 2024-05-08T08:52:45.177 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:45.177 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:45.177 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:45.177 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.177 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.177 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7524 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 1392640~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:45.177 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.752121+0000", 2024-05-08T08:52:45.177 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 7.3647838879999998, 2024-05-08T08:52:45.177 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057199521000000003, 2024-05-08T08:52:45.177 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:45.177 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:45.177 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:45.178 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:45.178 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:45.178 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:45.178 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7524 2024-05-08T08:52:45.178 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.178 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:45.178 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.178 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:45.178 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752121+0000", 2024-05-08T08:52:45.178 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:45.178 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.178 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.178 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:45.179 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752121+0000", 2024-05-08T08:52:45.179 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:45.179 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.179 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.179 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:45.179 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752121+0000", 2024-05-08T08:52:45.179 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5999999999999999e-07 2024-05-08T08:52:45.179 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.179 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.179 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:45.179 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752123+0000", 2024-05-08T08:52:45.179 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3499999999999999e-06 2024-05-08T08:52:45.179 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.179 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.180 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:45.180 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752132+0000", 2024-05-08T08:52:45.180 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.5539999999999998e-06 2024-05-08T08:52:45.180 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.180 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.180 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:45.180 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752138+0000", 2024-05-08T08:52:45.180 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3640000000000004e-06 2024-05-08T08:52:45.180 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.181 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.181 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:45.181 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.809210+0000", 2024-05-08T08:52:45.181 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057071469999999999 2024-05-08T08:52:45.181 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.181 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.181 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:45.181 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.809258+0000", 2024-05-08T08:52:45.181 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8755000000000003e-05 2024-05-08T08:52:45.181 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.181 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.181 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:45.181 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.809320+0000", 2024-05-08T08:52:45.181 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1568000000000004e-05 2024-05-08T08:52:45.182 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:45.182 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:45.182 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:45.182 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.182 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.182 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7525 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 1433600~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:45.182 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.752146+0000", 2024-05-08T08:52:45.182 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 7.3647588439999998, 2024-05-08T08:52:45.182 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057400072000000003, 2024-05-08T08:52:45.182 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:45.182 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:45.182 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:45.182 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:45.183 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:45.183 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:45.183 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7525 2024-05-08T08:52:45.183 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.183 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:45.183 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.183 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:45.183 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752146+0000", 2024-05-08T08:52:45.183 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:45.183 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.183 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.183 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:45.183 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752146+0000", 2024-05-08T08:52:45.183 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:45.184 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.184 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.184 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:45.184 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752146+0000", 2024-05-08T08:52:45.184 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0400000000000002e-07 2024-05-08T08:52:45.184 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.184 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.184 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:45.184 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752148+0000", 2024-05-08T08:52:45.184 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3209999999999999e-06 2024-05-08T08:52:45.184 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.184 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.184 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:45.184 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752155+0000", 2024-05-08T08:52:45.185 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6109999999999999e-06 2024-05-08T08:52:45.185 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.185 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.185 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:45.185 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752161+0000", 2024-05-08T08:52:45.185 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.489e-06 2024-05-08T08:52:45.185 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.185 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.185 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:45.185 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.809360+0000", 2024-05-08T08:52:45.185 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057198860999999997 2024-05-08T08:52:45.185 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.185 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.185 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:45.186 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.809450+0000", 2024-05-08T08:52:45.186 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0136000000000006e-05 2024-05-08T08:52:45.186 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.186 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.186 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:45.186 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.809546+0000", 2024-05-08T08:52:45.186 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5249999999999998e-05 2024-05-08T08:52:45.186 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:45.186 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:45.186 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:45.186 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.186 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.186 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7526 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 1474560~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:45.187 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.752168+0000", 2024-05-08T08:52:45.187 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 7.3647361309999999, 2024-05-08T08:52:45.187 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057516818999999997, 2024-05-08T08:52:45.187 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:45.187 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:45.187 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:45.187 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:45.187 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:45.187 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:45.187 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7526 2024-05-08T08:52:45.187 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.187 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:45.187 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.188 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:45.188 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752168+0000", 2024-05-08T08:52:45.188 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:45.188 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.188 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.188 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:45.188 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752168+0000", 2024-05-08T08:52:45.188 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:45.188 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.188 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.188 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:45.188 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752169+0000", 2024-05-08T08:52:45.188 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4799999999999999e-07 2024-05-08T08:52:45.188 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.189 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.189 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:45.189 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752171+0000", 2024-05-08T08:52:45.189 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.289e-06 2024-05-08T08:52:45.189 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.189 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.189 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:45.189 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752177+0000", 2024-05-08T08:52:45.189 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3690000000000003e-06 2024-05-08T08:52:45.189 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.189 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.189 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:45.189 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752183+0000", 2024-05-08T08:52:45.189 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.152e-06 2024-05-08T08:52:45.190 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.190 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.190 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:45.190 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.809574+0000", 2024-05-08T08:52:45.190 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057391075 2024-05-08T08:52:45.190 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.190 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.190 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:45.190 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.809624+0000", 2024-05-08T08:52:45.190 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0673000000000001e-05 2024-05-08T08:52:45.190 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.190 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.190 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:45.190 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.809685+0000", 2024-05-08T08:52:45.191 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0813000000000002e-05 2024-05-08T08:52:45.191 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:45.191 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:45.191 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:45.191 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.191 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.191 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7527 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 1515520~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:45.191 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.752189+0000", 2024-05-08T08:52:45.191 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 7.3647152030000003, 2024-05-08T08:52:45.191 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057722168999999997, 2024-05-08T08:52:45.191 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:45.191 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:45.191 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:45.192 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:45.192 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:45.192 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:45.192 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7527 2024-05-08T08:52:45.192 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.192 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:45.192 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.192 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:45.192 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752189+0000", 2024-05-08T08:52:45.192 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:45.192 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.192 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.192 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:45.192 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752189+0000", 2024-05-08T08:52:45.192 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:45.193 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.193 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.193 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:45.193 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752190+0000", 2024-05-08T08:52:45.193 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8799999999999998e-07 2024-05-08T08:52:45.193 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.193 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.193 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:45.193 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752192+0000", 2024-05-08T08:52:45.193 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9759999999999998e-06 2024-05-08T08:52:45.193 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.193 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.193 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:45.193 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752199+0000", 2024-05-08T08:52:45.194 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3289999999999996e-06 2024-05-08T08:52:45.194 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.194 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.194 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:45.194 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752205+0000", 2024-05-08T08:52:45.194 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2759999999999997e-06 2024-05-08T08:52:45.194 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.194 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.194 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:45.194 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.809726+0000", 2024-05-08T08:52:45.194 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057521249000000003 2024-05-08T08:52:45.194 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.194 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.194 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:45.194 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.809815+0000", 2024-05-08T08:52:45.195 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.8987999999999996e-05 2024-05-08T08:52:45.195 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.195 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.195 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:45.195 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.809911+0000", 2024-05-08T08:52:45.195 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5963000000000003e-05 2024-05-08T08:52:45.195 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:45.195 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:45.195 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:45.195 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.195 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.195 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7528 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 1556480~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:45.195 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.752212+0000", 2024-05-08T08:52:45.195 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 7.3646924929999997, 2024-05-08T08:52:45.195 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057904956, 2024-05-08T08:52:45.196 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:45.196 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:45.196 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:45.196 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:45.196 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:45.196 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:45.196 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7528 2024-05-08T08:52:45.196 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.196 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:45.196 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.196 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:45.196 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752212+0000", 2024-05-08T08:52:45.196 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:45.196 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.196 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.196 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:45.197 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752212+0000", 2024-05-08T08:52:45.197 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:45.197 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.197 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.197 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:45.197 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752212+0000", 2024-05-08T08:52:45.197 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7800000000000002e-07 2024-05-08T08:52:45.197 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.197 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.197 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:45.197 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752215+0000", 2024-05-08T08:52:45.197 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2120000000000002e-06 2024-05-08T08:52:45.197 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.197 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.197 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:45.198 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752222+0000", 2024-05-08T08:52:45.198 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.6329999999999996e-06 2024-05-08T08:52:45.198 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.198 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.198 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:45.198 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752227+0000", 2024-05-08T08:52:45.198 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9289999999999997e-06 2024-05-08T08:52:45.198 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.198 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.198 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:45.198 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.809966+0000", 2024-05-08T08:52:45.198 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057738554999999997 2024-05-08T08:52:45.198 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.199 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.199 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:45.199 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.810052+0000", 2024-05-08T08:52:45.199 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.6540999999999997e-05 2024-05-08T08:52:45.199 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.199 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.199 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:45.199 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.810117+0000", 2024-05-08T08:52:45.199 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4608000000000002e-05 2024-05-08T08:52:45.199 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:45.199 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:45.199 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:45.199 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.199 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.199 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7529 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 1597440~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:45.199 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.752234+0000", 2024-05-08T08:52:45.200 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 7.36467048, 2024-05-08T08:52:45.200 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.058133851, 2024-05-08T08:52:45.200 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:45.200 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:45.200 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:45.200 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:45.200 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:45.200 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:45.200 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7529 2024-05-08T08:52:45.200 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.200 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:45.200 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.200 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:45.200 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752234+0000", 2024-05-08T08:52:45.201 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:45.201 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.201 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.201 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:45.201 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752234+0000", 2024-05-08T08:52:45.201 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:45.201 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.201 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.201 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:45.201 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.753959+0000", 2024-05-08T08:52:45.201 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0017251409999999999 2024-05-08T08:52:45.201 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.201 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.201 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:45.201 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.753963+0000", 2024-05-08T08:52:45.201 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2860000000000001e-06 2024-05-08T08:52:45.202 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.202 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.202 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:45.202 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.753973+0000", 2024-05-08T08:52:45.202 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5119999999999997e-06 2024-05-08T08:52:45.202 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.202 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.202 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:45.202 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.753979+0000", 2024-05-08T08:52:45.202 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3589999999999996e-06 2024-05-08T08:52:45.202 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.202 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.203 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:45.203 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.810182+0000", 2024-05-08T08:52:45.203 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.056202941999999999 2024-05-08T08:52:45.203 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.203 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.203 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:45.203 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.810270+0000", 2024-05-08T08:52:45.203 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7338999999999998e-05 2024-05-08T08:52:45.203 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:45.203 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:45.203 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:45.203 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.810368+0000", 2024-05-08T08:52:45.203 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8271999999999997e-05 2024-05-08T08:52:45.203 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:45.203 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:45.203 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:45.204 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:45.204 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:45.204 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:52:45.204 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-05-08T08:52:45.204 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T08:52:45.204 INFO:tasks.thrashosds.thrasher:Removing osd 4, in_osds are: [0, 1, 2, 3, 4, 5] 2024-05-08T08:52:45.205 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 4 2024-05-08T08:52:45.206 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:52:45.207 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:52:45.208+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:52:45.306 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:52:45.309+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:52:45.314 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:52:45.314 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:52:45.314 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:52:45.314 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:52:45.323 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:52:45.407 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:52:45.410+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:52:45.418 INFO:teuthology.orchestra.run.smithi196.stdout:checking clone #8, image image_client.0-clone8 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent9 2024-05-08T08:52:45.433 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:52:45.433 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:52:45.433 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:52:45.433 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:52:45.433 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:52:45.442 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:52:45.526 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:45.529+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:45.574 INFO:tasks.ceph.mon.a.smithi059.stderr:2024-05-08T08:52:45.577+0000 7fc9cc811640 -1 mon.a@0(leader).osd e40 definitely_dead 0 2024-05-08T08:52:45.601 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:52:45.602 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:52:45.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:52:45.602 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:52:45.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.602 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4738 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1800192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:45.602 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.585359+0000", 2024-05-08T08:52:45.602 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 14.018307989, 2024-05-08T08:52:45.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035337788000000002, 2024-05-08T08:52:45.602 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:45.602 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:45.602 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:45.602 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:45.603 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:45.603 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:45.603 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4738 2024-05-08T08:52:45.603 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.603 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:45.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.603 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:45.603 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585359+0000", 2024-05-08T08:52:45.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:45.603 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.603 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:45.603 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585359+0000", 2024-05-08T08:52:45.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:45.603 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.604 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.604 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:45.604 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585360+0000", 2024-05-08T08:52:45.604 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6899999999999998e-07 2024-05-08T08:52:45.604 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.604 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.604 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:45.604 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585362+0000", 2024-05-08T08:52:45.604 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4739999999999999e-06 2024-05-08T08:52:45.604 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.604 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.604 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:45.604 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585369+0000", 2024-05-08T08:52:45.604 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9310000000000002e-06 2024-05-08T08:52:45.605 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.605 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.605 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:45.605 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585375+0000", 2024-05-08T08:52:45.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9599999999999997e-06 2024-05-08T08:52:45.605 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.605 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.605 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:45.605 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.620510+0000", 2024-05-08T08:52:45.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035135176999999997 2024-05-08T08:52:45.605 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.605 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.605 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:45.605 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.620593+0000", 2024-05-08T08:52:45.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3057000000000001e-05 2024-05-08T08:52:45.606 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.606 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:45.606 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.620697+0000", 2024-05-08T08:52:45.606 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010372 2024-05-08T08:52:45.606 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:45.606 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:45.606 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:45.606 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.606 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4740 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1816576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:45.606 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.585472+0000", 2024-05-08T08:52:45.606 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 14.018195367000001, 2024-05-08T08:52:45.606 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035430959999999997, 2024-05-08T08:52:45.607 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:45.607 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:45.607 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:45.607 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:45.607 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:45.607 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:45.607 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4740 2024-05-08T08:52:45.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.607 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:45.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.607 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:45.607 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585472+0000", 2024-05-08T08:52:45.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:45.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.608 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:45.608 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585472+0000", 2024-05-08T08:52:45.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:45.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.608 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:45.608 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585472+0000", 2024-05-08T08:52:45.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3200000000000005e-07 2024-05-08T08:52:45.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.608 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:45.608 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585476+0000", 2024-05-08T08:52:45.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2890000000000002e-06 2024-05-08T08:52:45.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.609 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:45.609 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585484+0000", 2024-05-08T08:52:45.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9589999999999995e-06 2024-05-08T08:52:45.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.609 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:45.609 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585489+0000", 2024-05-08T08:52:45.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4920000000000001e-06 2024-05-08T08:52:45.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.609 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:45.609 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.620728+0000", 2024-05-08T08:52:45.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035239165000000003 2024-05-08T08:52:45.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.610 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:45.610 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.620809+0000", 2024-05-08T08:52:45.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0611999999999993e-05 2024-05-08T08:52:45.610 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.610 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:45.610 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.620903+0000", 2024-05-08T08:52:45.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3811e-05 2024-05-08T08:52:45.610 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:45.610 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:45.610 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:45.610 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.610 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4742 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1832960~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:45.610 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.585513+0000", 2024-05-08T08:52:45.611 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 14.018153694, 2024-05-08T08:52:45.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035588763000000002, 2024-05-08T08:52:45.611 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:45.611 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:45.611 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:45.611 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:45.611 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:45.611 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:45.611 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4742 2024-05-08T08:52:45.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.611 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:45.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.611 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:45.611 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585513+0000", 2024-05-08T08:52:45.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:45.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.612 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:45.612 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585513+0000", 2024-05-08T08:52:45.612 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:45.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.612 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:45.612 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585514+0000", 2024-05-08T08:52:45.612 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6100000000000001e-07 2024-05-08T08:52:45.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.613 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.613 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:45.613 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585516+0000", 2024-05-08T08:52:45.613 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.526e-06 2024-05-08T08:52:45.613 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.613 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.613 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:45.613 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585523+0000", 2024-05-08T08:52:45.613 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.562e-06 2024-05-08T08:52:45.613 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.613 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.613 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:45.613 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585528+0000", 2024-05-08T08:52:45.613 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7990000000000001e-06 2024-05-08T08:52:45.613 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.614 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.614 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:45.614 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.620939+0000", 2024-05-08T08:52:45.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035411419 2024-05-08T08:52:45.614 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.614 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.614 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:45.614 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621014+0000", 2024-05-08T08:52:45.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5017000000000004e-05 2024-05-08T08:52:45.614 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.614 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.614 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:45.614 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621102+0000", 2024-05-08T08:52:45.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7979000000000006e-05 2024-05-08T08:52:45.614 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:45.615 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:45.615 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:45.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.615 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4743 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1849344~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:45.615 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.585628+0000", 2024-05-08T08:52:45.615 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 14.018038807, 2024-05-08T08:52:45.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035659962000000003, 2024-05-08T08:52:45.615 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:45.615 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:45.615 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:45.615 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:45.615 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:45.615 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:45.615 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4743 2024-05-08T08:52:45.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.616 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:45.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:45.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585628+0000", 2024-05-08T08:52:45.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:45.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:45.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585628+0000", 2024-05-08T08:52:45.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:45.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:45.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585629+0000", 2024-05-08T08:52:45.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2300000000000001e-07 2024-05-08T08:52:45.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.617 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.617 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:45.617 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585632+0000", 2024-05-08T08:52:45.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1370000000000002e-06 2024-05-08T08:52:45.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.617 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.617 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:45.617 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585641+0000", 2024-05-08T08:52:45.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5220000000000007e-06 2024-05-08T08:52:45.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.617 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.617 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:45.617 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585646+0000", 2024-05-08T08:52:45.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.609e-06 2024-05-08T08:52:45.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.618 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.618 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:45.618 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621132+0000", 2024-05-08T08:52:45.618 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035485472999999997 2024-05-08T08:52:45.618 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.618 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.618 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:45.618 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621204+0000", 2024-05-08T08:52:45.618 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2259000000000001e-05 2024-05-08T08:52:45.618 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.618 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.618 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:45.618 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621288+0000", 2024-05-08T08:52:45.618 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4338999999999993e-05 2024-05-08T08:52:45.618 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:45.619 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:45.619 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:45.619 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.619 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.619 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4746 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1865728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:45.619 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.585656+0000", 2024-05-08T08:52:45.619 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 14.018011124999999, 2024-05-08T08:52:45.619 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035815909, 2024-05-08T08:52:45.619 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:45.619 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:45.619 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:45.619 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:45.619 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:45.619 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:45.619 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4746 2024-05-08T08:52:45.619 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.620 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:45.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.620 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:45.620 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585656+0000", 2024-05-08T08:52:45.620 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:45.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.620 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:45.620 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585656+0000", 2024-05-08T08:52:45.620 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:45.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.620 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:45.620 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585656+0000", 2024-05-08T08:52:45.620 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.08e-07 2024-05-08T08:52:45.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:45.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585658+0000", 2024-05-08T08:52:45.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0889999999999998e-06 2024-05-08T08:52:45.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:45.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585664+0000", 2024-05-08T08:52:45.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3990000000000003e-06 2024-05-08T08:52:45.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:45.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585668+0000", 2024-05-08T08:52:45.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3170000000000003e-06 2024-05-08T08:52:45.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:45.622 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621316+0000", 2024-05-08T08:52:45.622 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035647908999999998 2024-05-08T08:52:45.622 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.622 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.622 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:45.622 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621389+0000", 2024-05-08T08:52:45.622 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2531000000000001e-05 2024-05-08T08:52:45.622 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.622 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.622 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:45.622 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621472+0000", 2024-05-08T08:52:45.622 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3256000000000004e-05 2024-05-08T08:52:45.622 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:45.622 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:45.623 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:45.623 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.623 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.623 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4748 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1882112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:45.623 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.585697+0000", 2024-05-08T08:52:45.623 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 14.017969679, 2024-05-08T08:52:45.623 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035962485000000002, 2024-05-08T08:52:45.623 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:45.623 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:45.623 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:45.623 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:45.623 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:45.623 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:45.623 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4748 2024-05-08T08:52:45.624 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.624 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:45.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.624 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:45.624 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585697+0000", 2024-05-08T08:52:45.624 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:45.624 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.624 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:45.624 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585697+0000", 2024-05-08T08:52:45.624 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:45.624 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.624 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:45.624 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585698+0000", 2024-05-08T08:52:45.625 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0900000000000001e-07 2024-05-08T08:52:45.625 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.625 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.625 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:45.625 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585701+0000", 2024-05-08T08:52:45.625 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0149999999999999e-06 2024-05-08T08:52:45.625 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.625 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.625 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:45.625 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585712+0000", 2024-05-08T08:52:45.625 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0655e-05 2024-05-08T08:52:45.625 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.625 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.625 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:45.625 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585717+0000", 2024-05-08T08:52:45.625 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3800000000000002e-06 2024-05-08T08:52:45.625 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.626 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.626 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:45.626 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621502+0000", 2024-05-08T08:52:45.626 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035784559000000001 2024-05-08T08:52:45.626 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.626 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.626 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:45.626 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621575+0000", 2024-05-08T08:52:45.626 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3269999999999995e-05 2024-05-08T08:52:45.626 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.626 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.626 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:45.626 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621660+0000", 2024-05-08T08:52:45.626 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4997e-05 2024-05-08T08:52:45.627 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:45.627 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:45.627 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:45.627 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.627 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.627 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4750 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1898496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:45.627 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.585810+0000", 2024-05-08T08:52:45.627 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 14.017856848999999, 2024-05-08T08:52:45.627 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036034928000000001, 2024-05-08T08:52:45.627 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:45.627 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:45.627 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:45.627 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:45.627 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:45.627 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:45.628 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4750 2024-05-08T08:52:45.628 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.628 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:45.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.628 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:45.628 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585810+0000", 2024-05-08T08:52:45.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:45.628 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.628 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:45.628 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585810+0000", 2024-05-08T08:52:45.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:45.628 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.628 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:45.628 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585811+0000", 2024-05-08T08:52:45.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6499999999999999e-07 2024-05-08T08:52:45.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.629 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:45.629 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585814+0000", 2024-05-08T08:52:45.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1709999999999998e-06 2024-05-08T08:52:45.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.629 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:45.629 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585822+0000", 2024-05-08T08:52:45.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8909999999999995e-06 2024-05-08T08:52:45.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.629 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:45.630 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585827+0000", 2024-05-08T08:52:45.630 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5289999999999999e-06 2024-05-08T08:52:45.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.630 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.630 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:45.630 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621688+0000", 2024-05-08T08:52:45.630 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035861111000000001 2024-05-08T08:52:45.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.630 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.630 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:45.630 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621761+0000", 2024-05-08T08:52:45.630 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2211000000000004e-05 2024-05-08T08:52:45.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.630 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.630 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:45.631 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621845+0000", 2024-05-08T08:52:45.631 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4450000000000006e-05 2024-05-08T08:52:45.631 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:45.631 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:45.631 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:45.631 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.631 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.631 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4752 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1914880~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:45.631 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.585858+0000", 2024-05-08T08:52:45.631 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 14.01780888, 2024-05-08T08:52:45.631 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036180081000000003, 2024-05-08T08:52:45.631 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:45.631 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:45.631 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:45.631 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:45.631 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:45.631 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:45.632 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4752 2024-05-08T08:52:45.632 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.632 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:45.632 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.632 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:45.632 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585858+0000", 2024-05-08T08:52:45.632 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:45.632 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.632 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.632 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:45.632 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585858+0000", 2024-05-08T08:52:45.632 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:45.632 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.632 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.632 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:45.632 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585859+0000", 2024-05-08T08:52:45.633 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.82e-07 2024-05-08T08:52:45.633 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.633 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.633 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:45.633 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585861+0000", 2024-05-08T08:52:45.633 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.695e-06 2024-05-08T08:52:45.633 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.633 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.633 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:45.633 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585869+0000", 2024-05-08T08:52:45.633 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0299999999999994e-06 2024-05-08T08:52:45.633 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.633 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.633 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:45.633 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585874+0000", 2024-05-08T08:52:45.633 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9620000000000001e-06 2024-05-08T08:52:45.634 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.634 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:45.634 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621878+0000", 2024-05-08T08:52:45.634 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036004043999999999 2024-05-08T08:52:45.634 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.634 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:45.634 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621954+0000", 2024-05-08T08:52:45.634 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5414000000000007e-05 2024-05-08T08:52:45.634 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.634 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:45.635 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622038+0000", 2024-05-08T08:52:45.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4454000000000002e-05 2024-05-08T08:52:45.635 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:45.635 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:45.635 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:45.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.635 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.635 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4753 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1931264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:45.635 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.585947+0000", 2024-05-08T08:52:45.635 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 14.017720232, 2024-05-08T08:52:45.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036267992999999998, 2024-05-08T08:52:45.635 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:45.635 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:45.635 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:45.635 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:45.636 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:45.636 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:45.636 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4753 2024-05-08T08:52:45.636 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.636 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:45.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.636 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:45.636 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585947+0000", 2024-05-08T08:52:45.636 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:45.636 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.636 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:45.636 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585947+0000", 2024-05-08T08:52:45.636 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:45.636 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.636 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:45.637 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585948+0000", 2024-05-08T08:52:45.637 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6000000000000002e-07 2024-05-08T08:52:45.637 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.637 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.637 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:45.637 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585952+0000", 2024-05-08T08:52:45.637 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7090000000000001e-06 2024-05-08T08:52:45.637 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.637 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.637 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:45.637 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585963+0000", 2024-05-08T08:52:45.637 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0740000000000001e-05 2024-05-08T08:52:45.637 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.637 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.637 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:45.637 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585970+0000", 2024-05-08T08:52:45.638 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3309999999999998e-06 2024-05-08T08:52:45.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.638 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.638 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:45.638 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622065+0000", 2024-05-08T08:52:45.638 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036094676999999999 2024-05-08T08:52:45.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.638 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.638 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:45.638 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622132+0000", 2024-05-08T08:52:45.638 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7214999999999999e-05 2024-05-08T08:52:45.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.638 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.638 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:45.638 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622215+0000", 2024-05-08T08:52:45.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2460999999999995e-05 2024-05-08T08:52:45.639 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:45.639 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:45.639 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:45.639 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.639 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.639 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4756 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1947648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:45.639 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586018+0000", 2024-05-08T08:52:45.639 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 14.017648588, 2024-05-08T08:52:45.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036382335000000002, 2024-05-08T08:52:45.639 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:45.639 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:45.639 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:45.639 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:45.639 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:45.639 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:45.640 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4756 2024-05-08T08:52:45.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.640 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:45.640 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.640 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:45.640 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586018+0000", 2024-05-08T08:52:45.640 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:45.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.640 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.640 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:45.640 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586018+0000", 2024-05-08T08:52:45.640 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:45.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.640 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.640 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:45.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586019+0000", 2024-05-08T08:52:45.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9999999999999997e-07 2024-05-08T08:52:45.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.641 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:45.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586023+0000", 2024-05-08T08:52:45.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7330000000000001e-06 2024-05-08T08:52:45.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.641 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:45.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586031+0000", 2024-05-08T08:52:45.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4940000000000003e-06 2024-05-08T08:52:45.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.641 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:45.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586038+0000", 2024-05-08T08:52:45.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2539999999999998e-06 2024-05-08T08:52:45.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:45.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622243+0000", 2024-05-08T08:52:45.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036205009000000003 2024-05-08T08:52:45.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:45.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622313+0000", 2024-05-08T08:52:45.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9912000000000004e-05 2024-05-08T08:52:45.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:45.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622401+0000", 2024-05-08T08:52:45.643 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8232999999999994e-05 2024-05-08T08:52:45.643 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:45.643 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:45.643 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:45.643 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.643 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.643 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4758 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1964032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:45.643 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586080+0000", 2024-05-08T08:52:45.643 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 14.017587169, 2024-05-08T08:52:45.643 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036547659000000003, 2024-05-08T08:52:45.643 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:45.643 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:45.643 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:45.643 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:45.643 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:45.643 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:45.643 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4758 2024-05-08T08:52:45.644 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.644 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:45.644 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.644 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:45.644 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586080+0000", 2024-05-08T08:52:45.644 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:45.644 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.644 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.644 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:45.644 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586080+0000", 2024-05-08T08:52:45.644 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:45.644 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.644 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.644 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:45.644 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586081+0000", 2024-05-08T08:52:45.645 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5499999999999998e-07 2024-05-08T08:52:45.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.645 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.645 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:45.645 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586084+0000", 2024-05-08T08:52:45.645 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9919999999999999e-06 2024-05-08T08:52:45.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.645 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.645 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:45.645 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586091+0000", 2024-05-08T08:52:45.645 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5440000000000001e-06 2024-05-08T08:52:45.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.645 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.645 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:45.645 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586098+0000", 2024-05-08T08:52:45.646 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8639999999999998e-06 2024-05-08T08:52:45.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.646 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:45.646 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622442+0000", 2024-05-08T08:52:45.646 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036344110999999998 2024-05-08T08:52:45.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.646 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:45.646 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622516+0000", 2024-05-08T08:52:45.646 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4089000000000002e-05 2024-05-08T08:52:45.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.646 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:45.646 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622628+0000", 2024-05-08T08:52:45.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00011140400000000001 2024-05-08T08:52:45.647 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:45.647 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:45.647 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:45.647 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.647 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.647 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4760 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1980416~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:45.647 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586178+0000", 2024-05-08T08:52:45.647 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 14.017488557, 2024-05-08T08:52:45.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036647526999999999, 2024-05-08T08:52:45.647 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:45.647 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:45.647 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:45.647 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:45.647 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:45.647 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:45.648 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4760 2024-05-08T08:52:45.648 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.648 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:45.648 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.648 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:45.648 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586178+0000", 2024-05-08T08:52:45.648 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:45.648 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.648 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.648 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:45.648 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586178+0000", 2024-05-08T08:52:45.648 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:45.648 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.648 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.648 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:45.649 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586179+0000", 2024-05-08T08:52:45.649 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8599999999999997e-07 2024-05-08T08:52:45.649 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.649 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.649 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:45.649 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586182+0000", 2024-05-08T08:52:45.649 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.174e-06 2024-05-08T08:52:45.649 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.649 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.649 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:45.649 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586191+0000", 2024-05-08T08:52:45.649 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1869999999999997e-06 2024-05-08T08:52:45.649 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.649 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.650 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:45.650 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586197+0000", 2024-05-08T08:52:45.650 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2090000000000002e-06 2024-05-08T08:52:45.650 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.650 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.650 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:45.650 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622657+0000", 2024-05-08T08:52:45.650 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036459793999999997 2024-05-08T08:52:45.650 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.650 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.650 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:45.650 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622725+0000", 2024-05-08T08:52:45.650 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8374999999999995e-05 2024-05-08T08:52:45.650 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.650 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.651 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:45.651 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622826+0000", 2024-05-08T08:52:45.651 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010100200000000001 2024-05-08T08:52:45.651 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:45.651 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:45.651 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:45.651 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.651 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.651 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4762 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1996800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:45.651 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586299+0000", 2024-05-08T08:52:45.651 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 14.017368226, 2024-05-08T08:52:45.651 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036720653999999998, 2024-05-08T08:52:45.651 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:45.651 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:45.651 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:45.651 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:45.652 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:45.652 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:45.652 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4762 2024-05-08T08:52:45.652 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.652 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:45.652 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.652 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:45.652 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586299+0000", 2024-05-08T08:52:45.652 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:45.652 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.652 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.652 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:45.652 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586299+0000", 2024-05-08T08:52:45.653 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:45.653 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.653 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.653 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:45.653 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586299+0000", 2024-05-08T08:52:45.653 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.99e-07 2024-05-08T08:52:45.653 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.653 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.653 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:45.653 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586303+0000", 2024-05-08T08:52:45.653 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1949999999999999e-06 2024-05-08T08:52:45.653 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.653 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.654 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:45.654 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586318+0000", 2024-05-08T08:52:45.654 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5401999999999999e-05 2024-05-08T08:52:45.654 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.654 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.654 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:45.654 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586324+0000", 2024-05-08T08:52:45.654 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8119999999999996e-06 2024-05-08T08:52:45.654 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.654 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.654 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:45.654 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622853+0000", 2024-05-08T08:52:45.655 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036529013999999999 2024-05-08T08:52:45.655 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.655 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.655 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:45.655 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622924+0000", 2024-05-08T08:52:45.655 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1654000000000002e-05 2024-05-08T08:52:45.655 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.655 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.655 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:45.655 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623019+0000", 2024-05-08T08:52:45.655 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4977999999999998e-05 2024-05-08T08:52:45.655 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:45.655 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:45.655 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:45.656 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.656 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.656 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4764 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 2013184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:45.656 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586506+0000", 2024-05-08T08:52:45.656 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 14.017161121000001, 2024-05-08T08:52:45.656 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036859467, 2024-05-08T08:52:45.656 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:45.656 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:45.656 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:45.656 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:45.656 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:45.656 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:45.656 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4764 2024-05-08T08:52:45.656 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.656 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:45.657 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.657 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:45.657 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586506+0000", 2024-05-08T08:52:45.657 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:45.657 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.657 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.657 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:45.657 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586506+0000", 2024-05-08T08:52:45.657 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:45.657 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.657 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.657 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:45.657 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586507+0000", 2024-05-08T08:52:45.658 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9100000000000003e-07 2024-05-08T08:52:45.658 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.658 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.658 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:45.658 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586512+0000", 2024-05-08T08:52:45.658 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9470000000000002e-06 2024-05-08T08:52:45.658 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.658 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.658 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:45.658 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586521+0000", 2024-05-08T08:52:45.658 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3379999999999994e-06 2024-05-08T08:52:45.658 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.658 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.658 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:45.659 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586528+0000", 2024-05-08T08:52:45.659 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1289999999999998e-06 2024-05-08T08:52:45.659 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.659 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.659 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:45.659 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623095+0000", 2024-05-08T08:52:45.659 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036566411 2024-05-08T08:52:45.659 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.659 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.659 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:45.659 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623221+0000", 2024-05-08T08:52:45.659 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012622299999999999 2024-05-08T08:52:45.659 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.659 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.659 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:45.660 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623365+0000", 2024-05-08T08:52:45.660 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000144628 2024-05-08T08:52:45.660 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:45.660 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:45.660 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:45.660 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.660 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.660 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4766 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 2029568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:45.660 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586543+0000", 2024-05-08T08:52:45.660 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 14.017123634000001, 2024-05-08T08:52:45.660 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.037033635000000002, 2024-05-08T08:52:45.660 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:45.660 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:45.660 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:45.661 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:45.661 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:45.661 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:45.661 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4766 2024-05-08T08:52:45.661 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.661 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:45.661 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.661 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:45.661 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586543+0000", 2024-05-08T08:52:45.661 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:45.661 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.661 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.661 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:45.661 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586543+0000", 2024-05-08T08:52:45.661 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:45.662 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.662 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.662 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:45.662 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586544+0000", 2024-05-08T08:52:45.662 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3099999999999998e-07 2024-05-08T08:52:45.662 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.662 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.662 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:45.662 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586547+0000", 2024-05-08T08:52:45.662 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.751e-06 2024-05-08T08:52:45.662 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.662 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.662 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:45.662 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586553+0000", 2024-05-08T08:52:45.663 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2870000000000001e-06 2024-05-08T08:52:45.663 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.663 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.663 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:45.663 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586558+0000", 2024-05-08T08:52:45.663 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5709999999999997e-06 2024-05-08T08:52:45.663 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.663 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.663 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:45.663 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623416+0000", 2024-05-08T08:52:45.663 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036857613999999997 2024-05-08T08:52:45.663 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.663 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.663 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:45.663 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623489+0000", 2024-05-08T08:52:45.664 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3172999999999999e-05 2024-05-08T08:52:45.664 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.664 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.664 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:45.664 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623577+0000", 2024-05-08T08:52:45.664 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7807999999999997e-05 2024-05-08T08:52:45.664 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:45.664 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:45.664 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:45.664 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.664 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.664 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4768 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 2045952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:45.664 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586566+0000", 2024-05-08T08:52:45.664 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 14.017101328000001, 2024-05-08T08:52:45.664 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.03726206, 2024-05-08T08:52:45.664 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:45.665 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:45.665 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:45.665 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:45.665 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:45.665 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:45.665 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4768 2024-05-08T08:52:45.665 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.665 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:45.665 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.665 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:45.665 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586566+0000", 2024-05-08T08:52:45.665 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:45.665 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.665 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.665 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:45.666 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586566+0000", 2024-05-08T08:52:45.666 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:45.666 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.666 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.666 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:45.666 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586566+0000", 2024-05-08T08:52:45.666 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8299999999999998e-07 2024-05-08T08:52:45.666 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.666 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.666 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:45.666 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586568+0000", 2024-05-08T08:52:45.666 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.063e-06 2024-05-08T08:52:45.666 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.666 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.666 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:45.667 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586574+0000", 2024-05-08T08:52:45.667 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8300000000000001e-06 2024-05-08T08:52:45.667 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.667 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.667 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:45.667 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586579+0000", 2024-05-08T08:52:45.667 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7559999999999999e-06 2024-05-08T08:52:45.667 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.667 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.667 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:45.667 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623612+0000", 2024-05-08T08:52:45.667 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.037033041000000003 2024-05-08T08:52:45.667 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.667 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.668 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:45.668 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623699+0000", 2024-05-08T08:52:45.668 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6817000000000006e-05 2024-05-08T08:52:45.668 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.668 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.668 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:45.668 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623828+0000", 2024-05-08T08:52:45.668 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012917000000000001 2024-05-08T08:52:45.668 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:45.668 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:45.668 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:45.668 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.668 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.668 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4770 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 2062336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:45.668 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586585+0000", 2024-05-08T08:52:45.668 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 14.017082145, 2024-05-08T08:52:45.669 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.037449891999999999, 2024-05-08T08:52:45.669 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:45.669 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:45.669 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:45.669 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:45.669 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:45.669 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:45.669 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4770 2024-05-08T08:52:45.669 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.669 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:45.669 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.669 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:45.669 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586585+0000", 2024-05-08T08:52:45.669 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:45.670 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.670 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.670 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:45.670 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586585+0000", 2024-05-08T08:52:45.670 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:45.670 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.670 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.670 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:45.670 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586585+0000", 2024-05-08T08:52:45.670 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:52:45.670 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.670 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.670 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:45.670 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586587+0000", 2024-05-08T08:52:45.670 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.181e-06 2024-05-08T08:52:45.671 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.671 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.671 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:45.671 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586592+0000", 2024-05-08T08:52:45.671 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9710000000000003e-06 2024-05-08T08:52:45.671 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.671 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.671 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:45.671 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586598+0000", 2024-05-08T08:52:45.671 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.118e-06 2024-05-08T08:52:45.671 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.671 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.671 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:45.671 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623862+0000", 2024-05-08T08:52:45.671 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.037264711999999998 2024-05-08T08:52:45.672 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.672 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.672 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:45.672 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623941+0000", 2024-05-08T08:52:45.672 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9084000000000004e-05 2024-05-08T08:52:45.672 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.672 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.672 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:45.672 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.624035+0000", 2024-05-08T08:52:45.672 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3456999999999996e-05 2024-05-08T08:52:45.672 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:45.672 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:45.672 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:45.672 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.672 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.673 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4772 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 2078720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:45.673 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586621+0000", 2024-05-08T08:52:45.673 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 14.017046197000001, 2024-05-08T08:52:45.673 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.037650391999999998, 2024-05-08T08:52:45.673 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:45.673 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:45.673 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:45.673 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:45.673 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:45.673 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:45.673 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4772 2024-05-08T08:52:45.673 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.673 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:45.673 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.673 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:45.674 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586621+0000", 2024-05-08T08:52:45.674 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:45.674 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.674 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.674 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:45.674 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586621+0000", 2024-05-08T08:52:45.674 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:45.674 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.674 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.674 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:45.674 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586621+0000", 2024-05-08T08:52:45.674 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:52:45.674 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.675 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.675 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:45.675 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586623+0000", 2024-05-08T08:52:45.675 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9240000000000001e-06 2024-05-08T08:52:45.675 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.675 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.675 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:45.675 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586628+0000", 2024-05-08T08:52:45.675 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6410000000000001e-06 2024-05-08T08:52:45.675 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.675 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.675 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:45.675 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586633+0000", 2024-05-08T08:52:45.675 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.763e-06 2024-05-08T08:52:45.675 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.676 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.676 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:45.676 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.624077+0000", 2024-05-08T08:52:45.676 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.037444709999999999 2024-05-08T08:52:45.676 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.676 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.676 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:45.676 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.624164+0000", 2024-05-08T08:52:45.676 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6769999999999998e-05 2024-05-08T08:52:45.676 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.676 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.676 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:45.676 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.624271+0000", 2024-05-08T08:52:45.676 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010723 2024-05-08T08:52:45.676 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:45.677 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:45.677 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:45.677 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.677 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.677 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4774 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 2095104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:45.677 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586640+0000", 2024-05-08T08:52:45.677 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 14.017027059, 2024-05-08T08:52:45.677 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.037827178000000003, 2024-05-08T08:52:45.677 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:45.677 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:45.677 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:45.677 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:45.677 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:45.677 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:45.677 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4774 2024-05-08T08:52:45.678 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.678 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:45.678 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.678 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:45.678 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586640+0000", 2024-05-08T08:52:45.678 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:45.678 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.678 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.678 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:45.678 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586640+0000", 2024-05-08T08:52:45.678 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:45.678 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.678 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.678 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:45.678 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586640+0000", 2024-05-08T08:52:45.679 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5199999999999998e-07 2024-05-08T08:52:45.679 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.679 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.679 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:45.679 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586643+0000", 2024-05-08T08:52:45.679 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.232e-06 2024-05-08T08:52:45.679 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.679 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.679 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:45.679 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586649+0000", 2024-05-08T08:52:45.679 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9660000000000001e-06 2024-05-08T08:52:45.679 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.679 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.679 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:45.679 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586653+0000", 2024-05-08T08:52:45.680 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7350000000000004e-06 2024-05-08T08:52:45.680 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.680 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.680 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:45.680 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.624303+0000", 2024-05-08T08:52:45.680 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.037649647000000001 2024-05-08T08:52:45.680 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.680 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.680 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:45.680 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.624373+0000", 2024-05-08T08:52:45.680 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0415999999999998e-05 2024-05-08T08:52:45.680 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.680 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.680 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:45.680 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.624467+0000", 2024-05-08T08:52:45.681 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3830000000000001e-05 2024-05-08T08:52:45.681 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:45.681 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:45.681 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:45.681 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.681 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.681 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4621.0:9089 3.1f e33/18 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:head v 33'117, mlcod=31'115)", 2024-05-08T08:52:45.681 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:52:45.681 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 13.084048847, 2024-05-08T08:52:45.681 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.111244524, 2024-05-08T08:52:45.681 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:45.681 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:45.681 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:45.681 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:45.681 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.681 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:45.682 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:52:45.682 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:45.682 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.682 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.682 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:45.682 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:52:45.682 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:45.682 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.682 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.682 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:45.682 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:52:45.682 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5900000000000001e-07 2024-05-08T08:52:45.682 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.682 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.683 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:45.683 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539600+0000", 2024-05-08T08:52:45.683 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.019981364000000001 2024-05-08T08:52:45.683 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.683 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.683 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:45.683 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539624+0000", 2024-05-08T08:52:45.683 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4043e-05 2024-05-08T08:52:45.683 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.683 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.683 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:45.683 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539635+0000", 2024-05-08T08:52:45.683 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1511e-05 2024-05-08T08:52:45.683 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.683 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.684 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:45.684 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539728+0000", 2024-05-08T08:52:45.684 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2498999999999996e-05 2024-05-08T08:52:45.684 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.684 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.684 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:45.684 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539746+0000", 2024-05-08T08:52:45.684 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8289000000000001e-05 2024-05-08T08:52:45.684 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.684 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.684 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:45.684 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.630822+0000", 2024-05-08T08:52:45.684 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.091076283999999993 2024-05-08T08:52:45.684 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:45.684 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:45.685 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:45.685 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.630863+0000", 2024-05-08T08:52:45.685 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0374999999999999e-05 2024-05-08T08:52:45.685 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:45.685 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:45.685 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:45.685 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:45.685 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:45.685 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:52:45.686 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:52:45.689+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:52:45.687 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:52:45.787 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:45.791+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:45.833 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:52:45.833 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:52:45.833 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:52:45.833 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:52:45.842 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:52:45.878 INFO:teuthology.orchestra.run.smithi196.stdout:1904 read 0x7edc800 thru 0x7ee05dd (0x3dde bytes) 2024-05-08T08:52:45.879 INFO:teuthology.orchestra.run.smithi196.stdout:1905 punch from 0x68af800 to 0x68b6800, (0x7000 bytes) 2024-05-08T08:52:45.888 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:52:45.891+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:52:45.895 INFO:teuthology.orchestra.run.smithi196.stdout:1907 trunc from 0x9897400 to 0x3084000 2024-05-08T08:52:45.977 INFO:teuthology.orchestra.run.smithi196.stdout:1908 read 0x1c81a00 thru 0x1c85ae0 (0x40e1 bytes) 2024-05-08T08:52:45.990 INFO:teuthology.orchestra.run.smithi196.stdout:1911 write 0x565d600 thru 0x56649d1 (0x73d2 bytes) 2024-05-08T08:52:45.990 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:52:45.993+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:52:45.993 INFO:teuthology.orchestra.run.smithi196.stdout:1912 write 0x37a8000 thru 0x37a9b8e (0x1b8f bytes) 2024-05-08T08:52:45.993 INFO:teuthology.orchestra.run.smithi196.stdout:1913 read 0x14fd000 thru 0x15073f4 (0xa3f5 bytes) 2024-05-08T08:52:45.994 INFO:teuthology.orchestra.run.smithi196.stdout:1915 read 0x31d8800 thru 0x31e6428 (0xdc29 bytes) 2024-05-08T08:52:45.994 INFO:teuthology.orchestra.run.smithi196.stdout:1916 read 0x254ae00 thru 0x254c7cd (0x19ce bytes) 2024-05-08T08:52:45.994 INFO:teuthology.orchestra.run.smithi196.stdout:1917 punch from 0x19ef000 to 0x19f7e00, (0x8e00 bytes) 2024-05-08T08:52:45.995 INFO:teuthology.orchestra.run.smithi196.stdout:1918 write 0x240f800 thru 0x2417b6c (0x836d bytes) 2024-05-08T08:52:45.996 INFO:teuthology.orchestra.run.smithi196.stdout:1920 write 0x1696200 thru 0x169d01c (0x6e1d bytes) 2024-05-08T08:52:45.997 INFO:teuthology.orchestra.run.smithi196.stdout:1922 read 0x644a00 thru 0x6548f5 (0xfef6 bytes) 2024-05-08T08:52:45.997 INFO:teuthology.orchestra.run.smithi196.stdout:1924 read 0x4e26a00 thru 0x4e2826e (0x186f bytes) 2024-05-08T08:52:46.001 INFO:teuthology.orchestra.run.smithi196.stdout:1925 trunc from 0x5664a00 to 0x889a400 2024-05-08T08:52:46.009 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:52:46.010 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:52:46.010 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:52:46.010 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:52:46.010 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:52:46.013 INFO:teuthology.orchestra.run.smithi196.stdout:1926 read 0x53fea00 thru 0x540e73f (0xfd40 bytes) 2024-05-08T08:52:46.015 INFO:teuthology.orchestra.run.smithi196.stdout:1927 punch from 0x3e0da00 to 0x3e1b200, (0xd800 bytes) 2024-05-08T08:52:46.019 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:52:46.031 INFO:teuthology.orchestra.run.smithi196.stdout:1928 write 0x952e400 thru 0x95301bd (0x1dbe bytes) 2024-05-08T08:52:46.031 INFO:teuthology.orchestra.run.smithi196.stdout:1929 write 0x1900c00 thru 0x19026a1 (0x1aa2 bytes) 2024-05-08T08:52:46.035 INFO:teuthology.orchestra.run.smithi196.stdout:1934 punch from 0x1c16c00 to 0x1c21000, (0xa400 bytes) 2024-05-08T08:52:46.035 INFO:teuthology.orchestra.run.smithi196.stdout:1935 write 0x6cc8a00 thru 0x6cd1656 (0x8c57 bytes) 2024-05-08T08:52:46.035 INFO:teuthology.orchestra.run.smithi196.stdout:1936 read 0x1786600 thru 0x178b129 (0x4b2a bytes) 2024-05-08T08:52:46.036 INFO:teuthology.orchestra.run.smithi196.stdout:1938 write 0x986600 thru 0x98b880 (0x5281 bytes) 2024-05-08T08:52:46.043 INFO:teuthology.orchestra.run.smithi196.stdout:1939 trunc from 0x9530200 to 0x3ebfc00 2024-05-08T08:52:46.044 INFO:teuthology.orchestra.run.smithi059.stderr:marked out osd.4. 2024-05-08T08:52:46.060 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-05-08T08:52:46.105 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:52:46.108+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:52:46.131 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:52:46.131 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:52:46.131 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:52:46.132 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:52:46.132 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.132 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22069 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 749568~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:46.132 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857624+0000", 2024-05-08T08:52:46.132 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.276087083, 2024-05-08T08:52:46.132 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046969958999999999, 2024-05-08T08:52:46.132 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:46.132 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:46.133 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:46.133 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:46.133 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:46.133 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:46.133 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22069 2024-05-08T08:52:46.133 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.133 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:46.133 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.133 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:46.133 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857624+0000", 2024-05-08T08:52:46.133 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:46.133 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.133 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.133 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:46.133 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857624+0000", 2024-05-08T08:52:46.134 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:46.134 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.134 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.134 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:46.134 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857625+0000", 2024-05-08T08:52:46.134 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.03e-07 2024-05-08T08:52:46.134 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.134 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.134 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:46.135 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857627+0000", 2024-05-08T08:52:46.135 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9369999999999998e-06 2024-05-08T08:52:46.135 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.135 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.135 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:46.135 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857633+0000", 2024-05-08T08:52:46.135 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1229999999999997e-06 2024-05-08T08:52:46.135 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.135 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.135 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:46.135 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857637+0000", 2024-05-08T08:52:46.135 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7820000000000001e-06 2024-05-08T08:52:46.135 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.135 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.135 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:46.136 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904457+0000", 2024-05-08T08:52:46.136 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046819715999999997 2024-05-08T08:52:46.136 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.136 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.136 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:46.136 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904524+0000", 2024-05-08T08:52:46.138 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6371000000000005e-05 2024-05-08T08:52:46.138 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.138 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.138 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:46.138 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904594+0000", 2024-05-08T08:52:46.138 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0627e-05 2024-05-08T08:52:46.139 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.139 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:46.139 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.139 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.139 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.139 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22071 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 765952~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:46.139 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857643+0000", 2024-05-08T08:52:46.139 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.276067999999999, 2024-05-08T08:52:46.139 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047106528000000002, 2024-05-08T08:52:46.139 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:46.139 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:46.139 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:46.139 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:46.139 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:46.139 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:46.140 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22071 2024-05-08T08:52:46.140 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.140 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:46.140 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.140 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:46.140 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857643+0000", 2024-05-08T08:52:46.140 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:46.140 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.140 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.140 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:46.141 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857643+0000", 2024-05-08T08:52:46.141 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:46.141 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.141 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.141 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:46.141 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857644+0000", 2024-05-08T08:52:46.141 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2000000000000001e-07 2024-05-08T08:52:46.141 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.141 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.141 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:46.141 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857646+0000", 2024-05-08T08:52:46.141 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0059999999999999e-06 2024-05-08T08:52:46.141 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.141 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.141 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:46.142 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857651+0000", 2024-05-08T08:52:46.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.84e-06 2024-05-08T08:52:46.142 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.142 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.142 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:46.142 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857657+0000", 2024-05-08T08:52:46.143 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5620000000000003e-06 2024-05-08T08:52:46.143 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.143 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.143 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:46.143 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904620+0000", 2024-05-08T08:52:46.143 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046962661000000003 2024-05-08T08:52:46.143 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.143 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.143 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:46.143 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904678+0000", 2024-05-08T08:52:46.144 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8656999999999998e-05 2024-05-08T08:52:46.144 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.144 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.144 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:46.144 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904750+0000", 2024-05-08T08:52:46.144 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1482000000000004e-05 2024-05-08T08:52:46.144 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.144 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:46.144 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.144 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.144 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.144 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22073 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 782336~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:46.144 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:52:46.144 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.276043992000002, 2024-05-08T08:52:46.145 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047244234000000003, 2024-05-08T08:52:46.145 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:46.145 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:46.145 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:46.145 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:46.145 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:46.145 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:46.145 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22073 2024-05-08T08:52:46.145 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.145 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:46.145 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.145 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:46.145 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:52:46.145 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:46.145 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.145 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.146 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:46.146 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:52:46.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:46.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.146 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.146 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:46.146 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857668+0000", 2024-05-08T08:52:46.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7800000000000002e-07 2024-05-08T08:52:46.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.146 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.146 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:46.146 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857670+0000", 2024-05-08T08:52:46.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0820000000000001e-06 2024-05-08T08:52:46.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.146 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.147 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:46.147 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857678+0000", 2024-05-08T08:52:46.147 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7589999999999997e-06 2024-05-08T08:52:46.147 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.147 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.147 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:46.147 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857691+0000", 2024-05-08T08:52:46.147 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3672999999999999e-05 2024-05-08T08:52:46.147 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.147 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.147 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:46.147 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904780+0000", 2024-05-08T08:52:46.147 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047088616999999999 2024-05-08T08:52:46.147 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.147 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.147 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:46.148 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904840+0000", 2024-05-08T08:52:46.148 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0356999999999998e-05 2024-05-08T08:52:46.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.148 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.148 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:46.148 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904912+0000", 2024-05-08T08:52:46.148 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1367999999999998e-05 2024-05-08T08:52:46.148 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.148 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:46.148 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.148 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.148 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22075 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 798720~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:46.148 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:52:46.149 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.276013594999998, 2024-05-08T08:52:46.149 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047384643999999997, 2024-05-08T08:52:46.149 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:46.149 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:46.149 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:46.149 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:46.149 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:46.149 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:46.149 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22075 2024-05-08T08:52:46.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.149 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:46.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.149 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:46.149 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:52:46.149 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:46.150 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.150 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.150 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:46.150 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:52:46.150 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:46.150 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.150 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.150 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:46.150 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:52:46.150 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7399999999999999e-07 2024-05-08T08:52:46.150 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.150 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.150 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:46.150 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857705+0000", 2024-05-08T08:52:46.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1980000000000004e-06 2024-05-08T08:52:46.151 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.151 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.151 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:46.151 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857713+0000", 2024-05-08T08:52:46.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7039999999999994e-06 2024-05-08T08:52:46.151 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.151 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.151 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:46.151 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857718+0000", 2024-05-08T08:52:46.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.865e-06 2024-05-08T08:52:46.151 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.151 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.151 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:46.151 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904941+0000", 2024-05-08T08:52:46.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047223198000000001 2024-05-08T08:52:46.151 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.152 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.152 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:46.152 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904999+0000", 2024-05-08T08:52:46.152 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8254999999999997e-05 2024-05-08T08:52:46.152 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.152 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.152 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:46.152 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905082+0000", 2024-05-08T08:52:46.152 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3049999999999999e-05 2024-05-08T08:52:46.152 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.152 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:46.152 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.152 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.152 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.152 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22077 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 815104~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:46.152 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:52:46.153 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.275975584000001, 2024-05-08T08:52:46.153 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047519448999999998, 2024-05-08T08:52:46.153 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:46.153 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:46.153 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:46.153 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:46.153 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:46.153 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:46.153 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22077 2024-05-08T08:52:46.153 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.153 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:46.153 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.153 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:46.153 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:52:46.153 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:46.154 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.154 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.154 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:46.154 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:52:46.154 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:46.154 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.154 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.154 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:46.154 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:52:46.154 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.99e-07 2024-05-08T08:52:46.154 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.154 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.154 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:46.154 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857739+0000", 2024-05-08T08:52:46.155 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3599999999999999e-06 2024-05-08T08:52:46.155 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.155 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.155 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:46.155 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857745+0000", 2024-05-08T08:52:46.155 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5139999999999998e-06 2024-05-08T08:52:46.155 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.155 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.155 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:46.155 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857750+0000", 2024-05-08T08:52:46.155 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8550000000000001e-06 2024-05-08T08:52:46.155 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.155 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.155 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:46.155 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905110+0000", 2024-05-08T08:52:46.156 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047360126000000002 2024-05-08T08:52:46.156 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.156 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.156 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:46.156 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905175+0000", 2024-05-08T08:52:46.156 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4689000000000004e-05 2024-05-08T08:52:46.156 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.156 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.156 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:46.156 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905255+0000", 2024-05-08T08:52:46.156 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0306e-05 2024-05-08T08:52:46.156 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.156 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:46.156 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.156 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.156 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.157 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22079 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 831488~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:46.157 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:52:46.157 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.275955035999999, 2024-05-08T08:52:46.157 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.04767499, 2024-05-08T08:52:46.157 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:46.157 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:46.157 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:46.157 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:46.157 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:46.157 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:46.157 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22079 2024-05-08T08:52:46.157 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.157 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:46.157 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.157 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:46.157 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:52:46.157 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:46.158 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.158 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.158 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:46.158 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:52:46.158 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:46.158 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.158 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.158 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:46.158 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857757+0000", 2024-05-08T08:52:46.158 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8299999999999997e-07 2024-05-08T08:52:46.158 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.158 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.158 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:46.158 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857759+0000", 2024-05-08T08:52:46.158 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1390000000000002e-06 2024-05-08T08:52:46.159 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.159 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.159 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:46.159 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857765+0000", 2024-05-08T08:52:46.159 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4609999999999996e-06 2024-05-08T08:52:46.159 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.159 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.159 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:46.159 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857770+0000", 2024-05-08T08:52:46.159 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6689999999999997e-06 2024-05-08T08:52:46.159 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.159 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.159 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:46.159 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905281+0000", 2024-05-08T08:52:46.159 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047510668999999998 2024-05-08T08:52:46.159 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.160 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.160 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:46.160 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905344+0000", 2024-05-08T08:52:46.160 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2797000000000002e-05 2024-05-08T08:52:46.160 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.160 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.160 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:46.160 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905431+0000", 2024-05-08T08:52:46.160 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7771999999999999e-05 2024-05-08T08:52:46.160 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.160 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:46.160 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.160 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.160 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.160 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22082 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 847872~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:46.160 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:52:46.161 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.275935345000001, 2024-05-08T08:52:46.161 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047831934999999999, 2024-05-08T08:52:46.161 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:46.161 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:46.161 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:46.161 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:46.161 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:46.161 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:46.161 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22082 2024-05-08T08:52:46.161 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.161 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:46.161 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.161 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:46.161 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:52:46.161 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:46.161 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.161 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.162 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:46.162 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:52:46.162 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:46.162 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.162 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.162 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:46.162 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:52:46.162 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:52:46.162 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.162 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.162 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:46.162 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857778+0000", 2024-05-08T08:52:46.162 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0789999999999999e-06 2024-05-08T08:52:46.162 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.162 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.163 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:46.163 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857784+0000", 2024-05-08T08:52:46.163 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0480000000000004e-06 2024-05-08T08:52:46.163 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.163 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.163 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:46.163 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857790+0000", 2024-05-08T08:52:46.163 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2719999999999997e-06 2024-05-08T08:52:46.163 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.163 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.163 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:46.163 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905458+0000", 2024-05-08T08:52:46.163 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047668470999999997 2024-05-08T08:52:46.164 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.164 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.164 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:46.164 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905521+0000", 2024-05-08T08:52:46.164 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2952000000000003e-05 2024-05-08T08:52:46.164 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.164 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.164 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:46.164 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905608+0000", 2024-05-08T08:52:46.164 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6759000000000001e-05 2024-05-08T08:52:46.164 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.164 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:46.164 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.164 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.165 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.165 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22084 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 864256~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:46.165 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:52:46.165 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.275911395000001, 2024-05-08T08:52:46.165 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047977748000000001, 2024-05-08T08:52:46.165 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:46.165 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:46.165 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:46.165 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:46.165 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:46.165 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:46.165 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22084 2024-05-08T08:52:46.165 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.166 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:46.166 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.166 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:46.166 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:52:46.166 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:46.166 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.166 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.166 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:46.166 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:52:46.166 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:46.166 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.166 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.166 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:46.167 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:52:46.167 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:52:46.167 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.167 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.167 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:46.167 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857802+0000", 2024-05-08T08:52:46.167 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0779999999999998e-06 2024-05-08T08:52:46.167 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.167 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.167 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:46.167 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857812+0000", 2024-05-08T08:52:46.167 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7939999999999997e-06 2024-05-08T08:52:46.167 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.168 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.168 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:46.168 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857817+0000", 2024-05-08T08:52:46.168 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8899999999999998e-06 2024-05-08T08:52:46.168 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.168 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.168 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:46.168 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905636+0000", 2024-05-08T08:52:46.168 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047819252 2024-05-08T08:52:46.168 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.168 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.168 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:46.168 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905700+0000", 2024-05-08T08:52:46.168 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4016999999999994e-05 2024-05-08T08:52:46.168 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.169 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.169 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:46.169 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905778+0000", 2024-05-08T08:52:46.169 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7306999999999997e-05 2024-05-08T08:52:46.169 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.169 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:46.169 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.169 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.169 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.169 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22086 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 880640~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:46.169 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:52:46.169 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.275888044999999, 2024-05-08T08:52:46.169 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048119149, 2024-05-08T08:52:46.169 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:46.170 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:46.170 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:46.170 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:46.170 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:46.170 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:46.170 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22086 2024-05-08T08:52:46.170 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.170 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:46.170 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.170 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:46.170 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:52:46.170 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:46.170 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.170 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.171 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:46.171 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:52:46.171 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:46.171 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.171 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.171 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:46.171 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857824+0000", 2024-05-08T08:52:46.171 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4200000000000002e-07 2024-05-08T08:52:46.171 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.171 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.171 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:46.171 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857826+0000", 2024-05-08T08:52:46.171 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0590000000000001e-06 2024-05-08T08:52:46.171 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.172 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.172 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:46.172 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857832+0000", 2024-05-08T08:52:46.172 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8029999999999999e-06 2024-05-08T08:52:46.172 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.172 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.172 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:46.172 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857837+0000", 2024-05-08T08:52:46.172 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7929999999999997e-06 2024-05-08T08:52:46.172 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.172 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.172 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:46.172 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905803+0000", 2024-05-08T08:52:46.172 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047965761000000003 2024-05-08T08:52:46.172 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.172 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.173 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:46.173 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905862+0000", 2024-05-08T08:52:46.173 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9203000000000002e-05 2024-05-08T08:52:46.173 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.173 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.173 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:46.173 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905942+0000", 2024-05-08T08:52:46.173 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0187999999999999e-05 2024-05-08T08:52:46.173 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.173 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:46.173 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.173 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.173 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.173 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22088 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 897024~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:46.173 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:52:46.174 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.275867992999999, 2024-05-08T08:52:46.174 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048256049000000002, 2024-05-08T08:52:46.174 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:46.174 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:46.174 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:46.174 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:46.174 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:46.174 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:46.174 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22088 2024-05-08T08:52:46.174 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.174 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:46.174 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.174 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:46.174 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:52:46.174 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:46.175 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.175 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.175 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:46.175 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:52:46.175 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:46.175 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.175 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.175 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:46.175 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857844+0000", 2024-05-08T08:52:46.175 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.46e-07 2024-05-08T08:52:46.175 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.175 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.175 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:46.175 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857846+0000", 2024-05-08T08:52:46.175 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0899999999999999e-06 2024-05-08T08:52:46.176 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.176 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.176 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:46.176 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857855+0000", 2024-05-08T08:52:46.176 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8219999999999996e-06 2024-05-08T08:52:46.176 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.176 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.176 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:46.176 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857860+0000", 2024-05-08T08:52:46.176 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2989999999999996e-06 2024-05-08T08:52:46.176 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.176 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.176 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:46.176 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905966+0000", 2024-05-08T08:52:46.176 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048106542000000002 2024-05-08T08:52:46.177 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.177 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.177 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:46.177 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906024+0000", 2024-05-08T08:52:46.177 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7958000000000003e-05 2024-05-08T08:52:46.177 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.177 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.177 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:46.177 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906099+0000", 2024-05-08T08:52:46.177 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4992000000000003e-05 2024-05-08T08:52:46.177 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.177 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:46.177 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.177 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.178 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.178 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22091 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 913408~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:46.178 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:52:46.178 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.275844233000001, 2024-05-08T08:52:46.178 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048395290000000001, 2024-05-08T08:52:46.178 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:46.178 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:46.178 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:46.178 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:46.178 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:46.178 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:46.178 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22091 2024-05-08T08:52:46.178 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.178 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:46.178 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.179 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:46.179 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:52:46.179 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:46.179 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.179 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.179 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:46.179 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:52:46.179 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:46.179 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.179 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.179 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:46.179 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:52:46.179 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4900000000000001e-07 2024-05-08T08:52:46.179 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.179 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.179 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:46.180 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857870+0000", 2024-05-08T08:52:46.180 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2510000000000001e-06 2024-05-08T08:52:46.180 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.180 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.180 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:46.180 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857877+0000", 2024-05-08T08:52:46.180 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0809999999999996e-06 2024-05-08T08:52:46.180 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.180 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.180 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:46.180 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857881+0000", 2024-05-08T08:52:46.180 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.724e-06 2024-05-08T08:52:46.180 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.180 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.180 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:46.180 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906127+0000", 2024-05-08T08:52:46.181 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048245345000000002 2024-05-08T08:52:46.181 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.181 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.181 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:46.181 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906190+0000", 2024-05-08T08:52:46.181 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2762000000000007e-05 2024-05-08T08:52:46.181 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.181 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.181 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:46.181 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906262+0000", 2024-05-08T08:52:46.181 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2778e-05 2024-05-08T08:52:46.181 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.181 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:46.181 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.181 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.181 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.181 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22093 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 929792~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:46.182 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:52:46.182 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.275821869000001, 2024-05-08T08:52:46.182 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048532875000000003, 2024-05-08T08:52:46.182 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:46.182 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:46.182 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:46.182 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:46.182 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:46.182 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:46.182 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22093 2024-05-08T08:52:46.182 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.182 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:46.182 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.182 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:46.182 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:52:46.183 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:46.183 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.183 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.183 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:46.183 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:52:46.183 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:46.183 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.183 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.183 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:46.183 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857890+0000", 2024-05-08T08:52:46.183 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-07 2024-05-08T08:52:46.183 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.183 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.183 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:46.183 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857899+0000", 2024-05-08T08:52:46.184 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6889999999999999e-06 2024-05-08T08:52:46.184 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.184 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.184 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:46.184 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857906+0000", 2024-05-08T08:52:46.184 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6490000000000002e-06 2024-05-08T08:52:46.184 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.184 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.184 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:46.184 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857917+0000", 2024-05-08T08:52:46.184 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0764999999999999e-05 2024-05-08T08:52:46.184 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.184 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.184 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:46.184 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906287+0000", 2024-05-08T08:52:46.185 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048370442 2024-05-08T08:52:46.185 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.185 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.185 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:46.185 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906349+0000", 2024-05-08T08:52:46.185 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1537000000000004e-05 2024-05-08T08:52:46.185 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.185 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.185 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:46.185 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906422+0000", 2024-05-08T08:52:46.185 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3431999999999998e-05 2024-05-08T08:52:46.185 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.185 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:46.185 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.185 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.185 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.186 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22095 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 946176~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:46.186 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:52:46.186 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.275787165000001, 2024-05-08T08:52:46.186 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048657593999999998, 2024-05-08T08:52:46.186 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:46.186 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:46.186 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:46.186 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:46.186 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:46.186 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:46.186 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22095 2024-05-08T08:52:46.186 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.186 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:46.186 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.187 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:46.187 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:52:46.187 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:46.187 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.187 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.187 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:46.187 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:52:46.187 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:46.187 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.187 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.187 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:46.187 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857925+0000", 2024-05-08T08:52:46.187 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3099999999999998e-07 2024-05-08T08:52:46.187 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.188 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.188 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:46.188 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857927+0000", 2024-05-08T08:52:46.188 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.34e-06 2024-05-08T08:52:46.188 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.188 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.188 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:46.188 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857938+0000", 2024-05-08T08:52:46.188 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1426e-05 2024-05-08T08:52:46.188 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.188 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.188 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:46.188 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857943+0000", 2024-05-08T08:52:46.188 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5140000000000003e-06 2024-05-08T08:52:46.188 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.188 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.189 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:46.189 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906448+0000", 2024-05-08T08:52:46.189 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048504641000000001 2024-05-08T08:52:46.189 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.189 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.189 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:46.189 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906510+0000", 2024-05-08T08:52:46.189 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2663000000000006e-05 2024-05-08T08:52:46.189 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.189 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.189 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:46.189 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906582+0000", 2024-05-08T08:52:46.189 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1478999999999998e-05 2024-05-08T08:52:46.189 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.189 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:46.189 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.190 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.190 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.190 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22097 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 962560~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:46.190 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:52:46.190 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.275759476000001, 2024-05-08T08:52:46.190 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048791353000000003, 2024-05-08T08:52:46.190 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:46.190 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:46.190 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:46.190 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:46.190 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:46.190 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:46.190 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22097 2024-05-08T08:52:46.190 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.191 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:46.191 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.191 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:46.191 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:52:46.191 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:46.191 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.191 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.191 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:46.191 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:52:46.191 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:46.191 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.191 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.191 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:46.191 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:52:46.191 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9200000000000002e-07 2024-05-08T08:52:46.191 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.192 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.192 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:46.192 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857954+0000", 2024-05-08T08:52:46.192 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1900000000000002e-06 2024-05-08T08:52:46.192 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.192 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.192 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:46.192 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857961+0000", 2024-05-08T08:52:46.192 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4989999999999999e-06 2024-05-08T08:52:46.192 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.192 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.192 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:46.192 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857971+0000", 2024-05-08T08:52:46.192 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0485999999999999e-05 2024-05-08T08:52:46.192 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.192 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.193 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:46.193 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906607+0000", 2024-05-08T08:52:46.193 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048635109000000003 2024-05-08T08:52:46.193 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.193 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.193 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:46.193 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906667+0000", 2024-05-08T08:52:46.193 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0439000000000002e-05 2024-05-08T08:52:46.193 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.193 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.193 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:46.193 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906743+0000", 2024-05-08T08:52:46.193 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6237999999999998e-05 2024-05-08T08:52:46.193 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.193 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:46.194 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.194 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.194 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.194 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22099 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 978944~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:46.194 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:52:46.194 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.275733316, 2024-05-08T08:52:46.194 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048932055000000002, 2024-05-08T08:52:46.194 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:46.194 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:46.194 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:46.194 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:46.194 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:46.194 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:46.194 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22099 2024-05-08T08:52:46.195 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.195 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:46.195 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.195 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:46.195 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:52:46.195 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:46.195 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.195 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.195 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:46.195 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:52:46.195 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:46.195 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.195 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.195 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:46.195 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:52:46.195 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.27e-07 2024-05-08T08:52:46.195 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.196 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.196 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:46.196 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857987+0000", 2024-05-08T08:52:46.196 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5890000000000003e-06 2024-05-08T08:52:46.196 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.196 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.196 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:46.196 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857994+0000", 2024-05-08T08:52:46.196 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6619999999999999e-06 2024-05-08T08:52:46.196 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.196 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.196 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:46.196 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857998+0000", 2024-05-08T08:52:46.196 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7110000000000003e-06 2024-05-08T08:52:46.196 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.196 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.196 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:46.197 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906772+0000", 2024-05-08T08:52:46.197 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048773415000000001 2024-05-08T08:52:46.197 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.197 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.197 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:46.197 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906834+0000", 2024-05-08T08:52:46.197 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1752000000000001e-05 2024-05-08T08:52:46.197 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.197 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.197 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:46.197 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906910+0000", 2024-05-08T08:52:46.197 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6499000000000001e-05 2024-05-08T08:52:46.197 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.197 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:46.197 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.198 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.198 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.198 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22101 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 995328~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:46.198 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:52:46.198 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.275706185000001, 2024-05-08T08:52:46.198 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049087364000000001, 2024-05-08T08:52:46.198 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:46.198 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:46.198 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:46.198 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:46.198 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:46.198 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:46.198 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22101 2024-05-08T08:52:46.198 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.198 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:46.199 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.199 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:46.199 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:52:46.199 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:46.199 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.199 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.199 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:46.199 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:52:46.199 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:46.199 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.199 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.199 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:46.199 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858006+0000", 2024-05-08T08:52:46.199 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1399999999999997e-07 2024-05-08T08:52:46.199 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.199 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.199 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:46.199 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858008+0000", 2024-05-08T08:52:46.200 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2060000000000001e-06 2024-05-08T08:52:46.200 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.200 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.200 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:46.200 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858017+0000", 2024-05-08T08:52:46.200 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1579999999999994e-06 2024-05-08T08:52:46.200 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.200 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.200 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:46.200 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858022+0000", 2024-05-08T08:52:46.200 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4199999999999998e-06 2024-05-08T08:52:46.200 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.200 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.200 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:46.200 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906940+0000", 2024-05-08T08:52:46.201 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048917627999999998 2024-05-08T08:52:46.201 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.201 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.201 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:46.201 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907002+0000", 2024-05-08T08:52:46.201 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2399999999999999e-05 2024-05-08T08:52:46.201 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.201 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.201 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:46.201 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907092+0000", 2024-05-08T08:52:46.201 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0137999999999997e-05 2024-05-08T08:52:46.201 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.201 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:46.201 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.201 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.201 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.202 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22103 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1011712~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:46.202 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:52:46.202 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.2756814, 2024-05-08T08:52:46.202 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049228542, 2024-05-08T08:52:46.202 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:46.202 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:46.202 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:46.202 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:46.202 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:46.202 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:46.202 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22103 2024-05-08T08:52:46.202 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.202 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:46.202 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.203 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:46.203 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:52:46.203 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:46.203 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.203 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.203 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:46.203 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:52:46.203 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:46.203 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.203 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.203 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:46.203 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:52:46.203 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4299999999999999e-07 2024-05-08T08:52:46.203 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.203 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.203 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:46.204 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858032+0000", 2024-05-08T08:52:46.204 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0590000000000001e-06 2024-05-08T08:52:46.204 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.204 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.204 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:46.204 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858037+0000", 2024-05-08T08:52:46.204 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.977e-06 2024-05-08T08:52:46.204 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.204 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.204 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:46.204 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858042+0000", 2024-05-08T08:52:46.204 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7650000000000001e-06 2024-05-08T08:52:46.204 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.204 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.205 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:46.205 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907118+0000", 2024-05-08T08:52:46.205 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049076147000000001 2024-05-08T08:52:46.205 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.205 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.205 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:46.205 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907180+0000", 2024-05-08T08:52:46.205 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1971999999999996e-05 2024-05-08T08:52:46.205 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.205 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.205 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:46.205 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907258+0000", 2024-05-08T08:52:46.205 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8279e-05 2024-05-08T08:52:46.205 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.205 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:46.205 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.206 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.206 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.206 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22105 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1028096~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:46.206 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:52:46.206 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.275662156999999, 2024-05-08T08:52:46.206 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049394609999999999, 2024-05-08T08:52:46.206 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:46.206 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:46.206 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:46.206 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:46.206 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:46.206 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:46.206 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22105 2024-05-08T08:52:46.206 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.207 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:46.207 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.207 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:46.207 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:52:46.207 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:46.207 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.207 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.207 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:46.207 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:52:46.207 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:46.207 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.207 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.207 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:46.207 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858050+0000", 2024-05-08T08:52:46.207 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5999999999999999e-07 2024-05-08T08:52:46.208 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.208 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.208 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:46.208 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858051+0000", 2024-05-08T08:52:46.208 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8870000000000001e-06 2024-05-08T08:52:46.208 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.208 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.208 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:46.208 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858057+0000", 2024-05-08T08:52:46.208 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8019999999999997e-06 2024-05-08T08:52:46.208 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.208 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.208 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:46.208 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858062+0000", 2024-05-08T08:52:46.208 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.561e-06 2024-05-08T08:52:46.208 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.209 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.209 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:46.209 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907289+0000", 2024-05-08T08:52:46.209 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049226957000000002 2024-05-08T08:52:46.209 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.209 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.209 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:46.209 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907356+0000", 2024-05-08T08:52:46.209 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6915000000000006e-05 2024-05-08T08:52:46.209 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.209 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.209 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:46.209 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907444+0000", 2024-05-08T08:52:46.209 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8127999999999994e-05 2024-05-08T08:52:46.209 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.210 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:46.210 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.210 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.210 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.210 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22107 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1044480~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:46.210 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:52:46.210 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.275643816999999, 2024-05-08T08:52:46.210 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049799942999999999, 2024-05-08T08:52:46.210 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:46.210 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:46.210 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:46.210 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:46.210 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:46.210 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:46.210 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22107 2024-05-08T08:52:46.211 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.211 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:46.211 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.211 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:46.211 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:52:46.211 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:46.211 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.211 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.211 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:46.211 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:52:46.211 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:46.211 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.211 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.211 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:46.211 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:52:46.211 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4799999999999999e-07 2024-05-08T08:52:46.211 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.212 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.212 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:46.212 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858070+0000", 2024-05-08T08:52:46.212 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.012e-06 2024-05-08T08:52:46.212 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.212 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.212 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:46.212 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858077+0000", 2024-05-08T08:52:46.212 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6290000000000001e-06 2024-05-08T08:52:46.212 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.212 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.212 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:46.212 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858083+0000", 2024-05-08T08:52:46.212 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8880000000000002e-06 2024-05-08T08:52:46.213 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.213 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.213 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:46.213 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907671+0000", 2024-05-08T08:52:46.213 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049587191000000003 2024-05-08T08:52:46.213 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.213 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.213 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:46.213 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907763+0000", 2024-05-08T08:52:46.213 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2231000000000005e-05 2024-05-08T08:52:46.213 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.213 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.213 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:46.213 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907867+0000", 2024-05-08T08:52:46.213 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104644 2024-05-08T08:52:46.213 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.213 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:46.214 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.214 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.214 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.214 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22294 3.1c 3:3aad0389:::rbd_data.11406886b07b.0000000000000080:head [stat,delete] snapc 7=[7] RETRY=6 ondisk+retry+write+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:46.214 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:52:46.214 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.275049969000001, 2024-05-08T08:52:46.214 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051152191, 2024-05-08T08:52:46.214 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:46.214 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:46.214 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:46.214 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:46.214 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:46.214 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:46.214 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22294 2024-05-08T08:52:46.215 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.215 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:46.215 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.215 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:46.215 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:52:46.215 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:46.215 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.215 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.215 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:46.215 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:52:46.215 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:46.215 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.215 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.215 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:46.215 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858662+0000", 2024-05-08T08:52:46.216 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4799999999999998e-07 2024-05-08T08:52:46.216 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.216 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.216 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:46.216 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858664+0000", 2024-05-08T08:52:46.216 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1229999999999998e-06 2024-05-08T08:52:46.216 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.216 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.216 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:46.216 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858670+0000", 2024-05-08T08:52:46.216 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3099999999999997e-06 2024-05-08T08:52:46.216 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.216 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.216 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:46.216 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858676+0000", 2024-05-08T08:52:46.217 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9750000000000004e-06 2024-05-08T08:52:46.217 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.217 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.217 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:46.217 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907901+0000", 2024-05-08T08:52:46.217 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049224520000000001 2024-05-08T08:52:46.217 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.217 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.217 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:46.217 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.908085+0000", 2024-05-08T08:52:46.217 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00018377500000000001 2024-05-08T08:52:46.217 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.217 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.217 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:46.217 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.909814+0000", 2024-05-08T08:52:46.217 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00172894 2024-05-08T08:52:46.218 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.218 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:46.218 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.218 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.218 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:46.218 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:52:46.220 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:52:46.220 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:46.223+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:46.320 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:46.323+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:46.366 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:52:46.366 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:52:46.366 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:52:46.366 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:52:46.374 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:52:46.458 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:52:46.461+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:52:46.519 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:52:46.519 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:52:46.520 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:52:46.520 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:52:46.520 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:52:46.522 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:46.525+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:46.528 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:52:46.623 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:52:46.626+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:52:46.675 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:52:46.675 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:52:46.675 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:52:46.675 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:52:46.675 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.675 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6690 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1093632~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:46.675 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:52:46.675 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 14.961990144, 2024-05-08T08:52:46.675 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052462754, 2024-05-08T08:52:46.676 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:46.676 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:46.676 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:46.676 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:46.676 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:46.676 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:46.676 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6690 2024-05-08T08:52:46.676 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.676 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:46.676 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.676 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:46.677 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:52:46.677 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:46.677 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.677 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.677 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:46.677 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:52:46.677 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:46.677 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.677 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.677 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:46.677 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715346+0000", 2024-05-08T08:52:46.677 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.84e-07 2024-05-08T08:52:46.677 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.678 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.678 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:46.678 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715348+0000", 2024-05-08T08:52:46.678 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.114e-06 2024-05-08T08:52:46.678 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.678 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.678 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:46.678 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715353+0000", 2024-05-08T08:52:46.678 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5720000000000002e-06 2024-05-08T08:52:46.678 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.678 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.678 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:46.679 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715358+0000", 2024-05-08T08:52:46.679 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7140000000000001e-06 2024-05-08T08:52:46.679 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.679 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.679 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:46.679 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767638+0000", 2024-05-08T08:52:46.679 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052279562000000002 2024-05-08T08:52:46.680 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.680 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.680 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:46.680 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767713+0000", 2024-05-08T08:52:46.680 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5652e-05 2024-05-08T08:52:46.680 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.680 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.680 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:46.680 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767808+0000", 2024-05-08T08:52:46.680 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4755999999999999e-05 2024-05-08T08:52:46.680 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.680 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:46.680 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.681 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.681 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.681 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6692 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1110016~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:46.681 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:52:46.681 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 14.961971053999999, 2024-05-08T08:52:46.681 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052666495000000001, 2024-05-08T08:52:46.681 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:46.681 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:46.681 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:46.681 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:46.681 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:46.681 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:46.681 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6692 2024-05-08T08:52:46.681 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.682 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:46.682 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.682 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:46.682 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:52:46.682 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:46.682 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.682 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.682 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:46.682 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:52:46.682 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:46.682 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.682 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.682 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:46.683 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:52:46.683 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8599999999999999e-07 2024-05-08T08:52:46.683 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.683 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.683 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:46.683 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715372+0000", 2024-05-08T08:52:46.683 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0990000000000001e-06 2024-05-08T08:52:46.683 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.683 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.683 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:46.683 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715378+0000", 2024-05-08T08:52:46.683 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0680000000000002e-06 2024-05-08T08:52:46.683 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.683 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.683 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:46.683 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715383+0000", 2024-05-08T08:52:46.684 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8740000000000003e-06 2024-05-08T08:52:46.684 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.684 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.684 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:46.684 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767848+0000", 2024-05-08T08:52:46.684 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052464668999999998 2024-05-08T08:52:46.684 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.684 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.684 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:46.684 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767927+0000", 2024-05-08T08:52:46.684 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9228999999999996e-05 2024-05-08T08:52:46.684 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.684 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.684 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:46.684 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768031+0000", 2024-05-08T08:52:46.684 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010417 2024-05-08T08:52:46.685 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.685 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:46.685 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.685 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.685 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.685 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6694 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1126400~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:46.685 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:52:46.685 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 14.961946157, 2024-05-08T08:52:46.685 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052848829, 2024-05-08T08:52:46.685 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:46.685 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:46.685 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:46.685 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:46.685 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:46.685 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:46.686 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6694 2024-05-08T08:52:46.686 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.686 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:46.686 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.686 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:46.686 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:52:46.686 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:46.686 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.686 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.686 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:46.686 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:52:46.686 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:46.686 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.686 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.686 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:46.687 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715390+0000", 2024-05-08T08:52:46.687 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9900000000000001e-07 2024-05-08T08:52:46.687 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.687 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.687 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:46.687 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715392+0000", 2024-05-08T08:52:46.687 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8279999999999999e-06 2024-05-08T08:52:46.687 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.687 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.687 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:46.687 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715398+0000", 2024-05-08T08:52:46.687 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7619999999999998e-06 2024-05-08T08:52:46.687 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.687 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.687 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:46.687 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715403+0000", 2024-05-08T08:52:46.688 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7790000000000003e-06 2024-05-08T08:52:46.688 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.688 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.688 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:46.688 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768067+0000", 2024-05-08T08:52:46.688 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052663785999999997 2024-05-08T08:52:46.688 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.688 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.688 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:46.688 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768142+0000", 2024-05-08T08:52:46.688 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5018999999999994e-05 2024-05-08T08:52:46.688 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.688 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.688 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:46.688 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768238+0000", 2024-05-08T08:52:46.688 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6255999999999995e-05 2024-05-08T08:52:46.689 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.689 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:46.689 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.689 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.689 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.689 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6696 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1142784~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:46.689 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:52:46.689 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 14.961926649, 2024-05-08T08:52:46.689 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.053051057999999998, 2024-05-08T08:52:46.689 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:46.689 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:46.689 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:46.689 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:46.689 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:46.689 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:46.690 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6696 2024-05-08T08:52:46.690 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.690 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:46.690 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.690 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:46.690 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:52:46.691 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:46.691 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.691 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.691 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:46.691 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:52:46.691 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:46.691 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.692 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.692 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:46.692 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:52:46.692 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-05-08T08:52:46.692 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.692 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.693 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:46.693 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715411+0000", 2024-05-08T08:52:46.693 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9630000000000001e-06 2024-05-08T08:52:46.693 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.693 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.693 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:46.693 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715417+0000", 2024-05-08T08:52:46.694 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7250000000000002e-06 2024-05-08T08:52:46.694 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.694 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.694 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:46.694 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715421+0000", 2024-05-08T08:52:46.694 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4089999999999996e-06 2024-05-08T08:52:46.694 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.694 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.694 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:46.694 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768277+0000", 2024-05-08T08:52:46.695 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052855293999999997 2024-05-08T08:52:46.695 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.695 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.695 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:46.695 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768358+0000", 2024-05-08T08:52:46.695 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1720999999999999e-05 2024-05-08T08:52:46.695 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.695 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.695 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:46.695 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768460+0000", 2024-05-08T08:52:46.695 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010154600000000001 2024-05-08T08:52:46.695 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.695 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:46.695 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.696 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.696 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.696 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6698 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1159168~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:46.696 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:52:46.696 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 14.961907782000001, 2024-05-08T08:52:46.696 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.053245726, 2024-05-08T08:52:46.696 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:46.696 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:46.696 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:46.696 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:46.696 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:46.696 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:46.696 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6698 2024-05-08T08:52:46.696 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.696 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:46.697 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.697 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:46.697 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:52:46.697 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:46.697 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.697 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.697 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:46.697 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:52:46.697 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:46.697 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.697 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.697 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:46.697 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716936+0000", 2024-05-08T08:52:46.697 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001508665 2024-05-08T08:52:46.697 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.697 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.698 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:46.698 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716941+0000", 2024-05-08T08:52:46.698 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1910000000000002e-06 2024-05-08T08:52:46.698 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.698 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.698 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:46.698 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716950+0000", 2024-05-08T08:52:46.698 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1900000000000001e-06 2024-05-08T08:52:46.698 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.698 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.698 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:46.698 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716958+0000", 2024-05-08T08:52:46.698 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8019999999999992e-06 2024-05-08T08:52:46.698 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.699 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.699 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:46.699 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768499+0000", 2024-05-08T08:52:46.699 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051540926000000001 2024-05-08T08:52:46.699 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.699 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.699 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:46.699 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768574+0000", 2024-05-08T08:52:46.699 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4988999999999996e-05 2024-05-08T08:52:46.699 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.699 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.699 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:46.699 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768673+0000", 2024-05-08T08:52:46.699 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9963000000000005e-05 2024-05-08T08:52:46.699 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.700 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:46.700 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.700 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.700 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.700 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6706 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1224704~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:46.700 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:52:46.700 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 14.960302475000001, 2024-05-08T08:52:46.700 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052435598, 2024-05-08T08:52:46.700 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:46.700 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:46.700 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:46.700 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:46.700 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:46.700 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:46.700 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6706 2024-05-08T08:52:46.700 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.701 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:46.701 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.701 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:46.701 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:52:46.701 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:46.701 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.701 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.701 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:46.701 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:52:46.701 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:46.701 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.701 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.701 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:46.701 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:52:46.701 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:52:46.702 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.702 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.702 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:46.702 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717035+0000", 2024-05-08T08:52:46.702 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9290000000000001e-06 2024-05-08T08:52:46.702 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.702 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.702 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:46.702 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717041+0000", 2024-05-08T08:52:46.702 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6720000000000001e-06 2024-05-08T08:52:46.702 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.702 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.702 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:46.702 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717046+0000", 2024-05-08T08:52:46.703 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5650000000000003e-06 2024-05-08T08:52:46.703 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.703 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.703 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:46.703 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769311+0000", 2024-05-08T08:52:46.703 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052265645999999999 2024-05-08T08:52:46.703 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.703 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.703 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:46.703 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769379+0000", 2024-05-08T08:52:46.703 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7391000000000006e-05 2024-05-08T08:52:46.703 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.703 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.703 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:46.703 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769469+0000", 2024-05-08T08:52:46.704 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0025999999999995e-05 2024-05-08T08:52:46.704 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.704 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:46.704 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.704 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.704 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.704 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6708 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1241088~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:46.704 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:52:46.704 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 14.960284371, 2024-05-08T08:52:46.704 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052640552, 2024-05-08T08:52:46.704 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:46.704 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:46.704 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:46.705 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:46.705 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:46.705 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:46.705 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6708 2024-05-08T08:52:46.705 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.705 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:46.705 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.705 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:46.705 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:52:46.705 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:46.705 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.705 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.705 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:46.705 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:52:46.705 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:46.706 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.706 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.706 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:46.706 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717052+0000", 2024-05-08T08:52:46.706 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:52:46.706 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.706 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.706 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:46.706 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717053+0000", 2024-05-08T08:52:46.706 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8330000000000001e-06 2024-05-08T08:52:46.706 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.706 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.706 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:46.707 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717058+0000", 2024-05-08T08:52:46.707 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.933e-06 2024-05-08T08:52:46.707 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.707 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.707 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:46.707 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717063+0000", 2024-05-08T08:52:46.707 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2180000000000001e-06 2024-05-08T08:52:46.707 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.707 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.708 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:46.708 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769507+0000", 2024-05-08T08:52:46.708 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052444866999999999 2024-05-08T08:52:46.708 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.708 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.708 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:46.708 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769589+0000", 2024-05-08T08:52:46.708 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2039000000000005e-05 2024-05-08T08:52:46.708 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.708 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.708 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:46.708 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769692+0000", 2024-05-08T08:52:46.708 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102252 2024-05-08T08:52:46.708 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.708 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:46.709 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.709 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.709 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.709 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6710 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1257472~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:46.709 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:52:46.709 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 14.960266346999999, 2024-05-08T08:52:46.709 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.05283057, 2024-05-08T08:52:46.709 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:46.709 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:46.709 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:46.709 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:46.710 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:46.710 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:46.710 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6710 2024-05-08T08:52:46.710 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.710 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:46.710 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.710 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:46.710 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:52:46.710 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:46.710 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.710 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.710 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:46.710 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:52:46.710 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:46.711 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.711 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.711 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:46.711 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718951+0000", 2024-05-08T08:52:46.711 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0018817809999999999 2024-05-08T08:52:46.711 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.711 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.711 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:46.711 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718956+0000", 2024-05-08T08:52:46.711 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5399999999999997e-06 2024-05-08T08:52:46.711 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.711 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.711 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:46.711 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718964+0000", 2024-05-08T08:52:46.712 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6629999999999999e-06 2024-05-08T08:52:46.712 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.712 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.712 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:46.712 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718972+0000", 2024-05-08T08:52:46.712 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5229999999999998e-06 2024-05-08T08:52:46.712 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.712 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.712 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:46.712 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769729+0000", 2024-05-08T08:52:46.712 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050756959999999997 2024-05-08T08:52:46.713 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.713 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.713 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:46.713 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769803+0000", 2024-05-08T08:52:46.713 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4538999999999999e-05 2024-05-08T08:52:46.713 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.713 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.713 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:46.713 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769900+0000", 2024-05-08T08:52:46.713 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6564000000000006e-05 2024-05-08T08:52:46.713 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.713 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:46.714 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.714 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.714 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.714 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6726 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1388544~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:46.714 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:52:46.714 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 14.958191575000001, 2024-05-08T08:52:46.714 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052451506000000002, 2024-05-08T08:52:46.714 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:46.714 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:46.714 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:46.714 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:46.715 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:46.715 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:46.715 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6726 2024-05-08T08:52:46.715 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.715 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:46.715 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.715 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:46.715 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:52:46.715 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:46.715 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.715 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.715 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:46.715 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:52:46.715 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:46.716 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.716 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.716 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:46.716 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719200+0000", 2024-05-08T08:52:46.716 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5865999999999997e-05 2024-05-08T08:52:46.716 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.716 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.716 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:46.716 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719204+0000", 2024-05-08T08:52:46.716 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8630000000000001e-06 2024-05-08T08:52:46.716 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.716 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.716 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:46.716 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719215+0000", 2024-05-08T08:52:46.717 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1590000000000001e-05 2024-05-08T08:52:46.717 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.717 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.717 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:46.717 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719220+0000", 2024-05-08T08:52:46.717 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1619999999999999e-06 2024-05-08T08:52:46.717 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.717 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.717 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:46.717 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771425+0000", 2024-05-08T08:52:46.717 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052204249000000001 2024-05-08T08:52:46.717 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.717 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.717 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:46.718 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771498+0000", 2024-05-08T08:52:46.718 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3782999999999995e-05 2024-05-08T08:52:46.718 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.718 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.718 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:46.718 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771595+0000", 2024-05-08T08:52:46.718 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6992999999999998e-05 2024-05-08T08:52:46.718 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.718 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:46.718 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.718 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.718 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.718 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6728 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1404928~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:46.719 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:52:46.719 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 14.95810417, 2024-05-08T08:52:46.719 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052590747, 2024-05-08T08:52:46.719 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:46.719 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:46.719 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:46.719 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:46.719 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:46.719 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:46.719 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6728 2024-05-08T08:52:46.719 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.719 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:46.719 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.719 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:46.719 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:52:46.719 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:46.720 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.720 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.720 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:46.720 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:52:46.720 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:46.720 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.720 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.720 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:46.720 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719232+0000", 2024-05-08T08:52:46.720 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4000000000000001e-07 2024-05-08T08:52:46.720 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.720 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.720 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:46.720 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719234+0000", 2024-05-08T08:52:46.721 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4499999999999998e-06 2024-05-08T08:52:46.721 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.721 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.721 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:46.721 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719240+0000", 2024-05-08T08:52:46.721 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5720000000000002e-06 2024-05-08T08:52:46.721 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.721 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.721 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:46.721 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719245+0000", 2024-05-08T08:52:46.721 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8690000000000003e-06 2024-05-08T08:52:46.721 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.721 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.721 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:46.721 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771636+0000", 2024-05-08T08:52:46.722 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052390979999999997 2024-05-08T08:52:46.722 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.722 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.722 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:46.722 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771717+0000", 2024-05-08T08:52:46.722 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1291999999999993e-05 2024-05-08T08:52:46.722 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.722 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.722 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:46.722 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771822+0000", 2024-05-08T08:52:46.722 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104944 2024-05-08T08:52:46.722 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.722 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:46.723 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.723 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.723 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.723 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6730 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1421312~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:46.723 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:52:46.723 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 14.958083928000001, 2024-05-08T08:52:46.723 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052770477000000003, 2024-05-08T08:52:46.723 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:46.723 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:46.723 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:46.723 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:46.723 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:46.723 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:46.723 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6730 2024-05-08T08:52:46.723 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.723 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:46.724 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.724 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:46.724 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:52:46.724 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:46.724 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.724 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.724 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:46.724 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:52:46.724 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:46.724 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.724 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.724 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:46.724 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:52:46.724 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9000000000000002e-07 2024-05-08T08:52:46.724 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.724 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.725 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:46.725 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719254+0000", 2024-05-08T08:52:46.725 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4099999999999998e-06 2024-05-08T08:52:46.725 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.725 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.725 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:46.725 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719263+0000", 2024-05-08T08:52:46.725 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4109999999999996e-06 2024-05-08T08:52:46.725 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.725 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.725 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:46.725 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719268+0000", 2024-05-08T08:52:46.725 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2669999999999997e-06 2024-05-08T08:52:46.725 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.725 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.725 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:46.726 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771846+0000", 2024-05-08T08:52:46.726 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052577677000000003 2024-05-08T08:52:46.726 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.726 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.726 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:46.726 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771920+0000", 2024-05-08T08:52:46.726 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4460000000000002e-05 2024-05-08T08:52:46.726 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.726 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.726 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:46.726 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772022+0000", 2024-05-08T08:52:46.726 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010186199999999999 2024-05-08T08:52:46.726 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.727 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:46.727 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.727 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.727 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.727 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6732 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1437696~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:46.727 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:52:46.727 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 14.958059121, 2024-05-08T08:52:46.727 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052905575000000003, 2024-05-08T08:52:46.728 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:46.728 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:46.728 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:46.728 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:46.728 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:46.728 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:46.728 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6732 2024-05-08T08:52:46.729 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.729 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:46.729 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.729 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:46.729 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:52:46.729 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:46.729 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.729 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.729 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:46.729 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:52:46.729 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:46.730 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.730 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.730 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:46.730 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721392+0000", 2024-05-08T08:52:46.730 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002115287 2024-05-08T08:52:46.730 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.730 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.730 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:46.730 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721396+0000", 2024-05-08T08:52:46.730 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1679999999999997e-06 2024-05-08T08:52:46.730 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.730 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.730 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:46.730 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721404+0000", 2024-05-08T08:52:46.731 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4409999999999993e-06 2024-05-08T08:52:46.731 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.731 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.731 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:46.731 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721410+0000", 2024-05-08T08:52:46.731 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9150000000000001e-06 2024-05-08T08:52:46.731 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.731 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.731 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:46.731 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772061+0000", 2024-05-08T08:52:46.731 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050651068 2024-05-08T08:52:46.731 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.731 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.731 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:46.731 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772127+0000", 2024-05-08T08:52:46.732 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5424000000000002e-05 2024-05-08T08:52:46.732 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.732 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.732 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:46.732 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772182+0000", 2024-05-08T08:52:46.732 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5272000000000002e-05 2024-05-08T08:52:46.732 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.732 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:46.732 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.732 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.732 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.732 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6749 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1585152~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:46.732 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:52:46.732 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 14.955758351, 2024-05-08T08:52:46.732 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052350264000000001, 2024-05-08T08:52:46.732 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:46.733 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:46.733 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:46.733 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:46.733 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:46.733 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:46.733 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6749 2024-05-08T08:52:46.733 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.733 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:46.733 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.733 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:46.733 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:52:46.733 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:46.733 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.733 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.733 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:46.733 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:52:46.734 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:46.734 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.734 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.734 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:46.734 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721578+0000", 2024-05-08T08:52:46.734 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0399999999999996e-07 2024-05-08T08:52:46.734 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.734 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.734 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:46.734 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721580+0000", 2024-05-08T08:52:46.734 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5450000000000002e-06 2024-05-08T08:52:46.734 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.734 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.734 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:46.735 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721587+0000", 2024-05-08T08:52:46.735 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0190000000000001e-06 2024-05-08T08:52:46.735 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.735 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.735 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:46.735 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721593+0000", 2024-05-08T08:52:46.735 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5659999999999997e-06 2024-05-08T08:52:46.735 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.735 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.735 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:46.735 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773759+0000", 2024-05-08T08:52:46.735 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052166553999999997 2024-05-08T08:52:46.735 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.735 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.735 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:46.736 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773833+0000", 2024-05-08T08:52:46.736 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3913999999999997e-05 2024-05-08T08:52:46.736 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.736 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.736 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:46.736 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773927+0000", 2024-05-08T08:52:46.736 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4061999999999995e-05 2024-05-08T08:52:46.736 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.736 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:46.736 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.736 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.736 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.736 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6751 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1601536~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:46.736 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:52:46.736 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 14.955735072, 2024-05-08T08:52:46.737 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052461302000000001, 2024-05-08T08:52:46.737 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:46.737 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:46.737 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:46.737 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:46.737 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:46.737 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:46.737 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6751 2024-05-08T08:52:46.737 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.737 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:46.738 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.738 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:46.738 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:52:46.738 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:46.738 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.738 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.738 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:46.738 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:52:46.738 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:46.738 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.739 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.739 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:46.739 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722939+0000", 2024-05-08T08:52:46.739 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001338601 2024-05-08T08:52:46.739 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.739 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.739 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:46.739 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722944+0000", 2024-05-08T08:52:46.739 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4469999999999999e-06 2024-05-08T08:52:46.739 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.739 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.739 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:46.739 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722955+0000", 2024-05-08T08:52:46.739 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1892999999999999e-05 2024-05-08T08:52:46.739 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.740 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.740 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:46.740 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722963+0000", 2024-05-08T08:52:46.740 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1080000000000004e-06 2024-05-08T08:52:46.740 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.740 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.740 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:46.740 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773964+0000", 2024-05-08T08:52:46.740 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051001022 2024-05-08T08:52:46.740 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.740 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.740 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:46.740 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.774006+0000", 2024-05-08T08:52:46.740 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2729999999999999e-05 2024-05-08T08:52:46.740 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.741 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.741 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:46.741 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.774062+0000", 2024-05-08T08:52:46.741 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5501000000000003e-05 2024-05-08T08:52:46.741 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.741 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:46.741 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.741 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.741 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.741 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6760 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1683456~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:46.741 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:52:46.741 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 14.954225696, 2024-05-08T08:52:46.741 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052571312000000002, 2024-05-08T08:52:46.741 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:46.741 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:46.742 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:46.742 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:46.742 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:46.742 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:46.742 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6760 2024-05-08T08:52:46.742 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.742 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:46.742 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.742 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:46.742 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:52:46.742 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:46.742 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.742 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.742 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:46.743 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:52:46.743 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:46.743 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.743 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.743 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:46.743 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:52:46.743 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2100000000000002e-07 2024-05-08T08:52:46.743 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.743 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.743 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:46.743 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723113+0000", 2024-05-08T08:52:46.743 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3149999999999999e-06 2024-05-08T08:52:46.743 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.743 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.743 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:46.744 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723118+0000", 2024-05-08T08:52:46.744 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8860000000000001e-06 2024-05-08T08:52:46.744 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.744 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.744 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:46.744 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723123+0000", 2024-05-08T08:52:46.744 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6770000000000003e-06 2024-05-08T08:52:46.744 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.744 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.744 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:46.744 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775414+0000", 2024-05-08T08:52:46.744 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052290692999999999 2024-05-08T08:52:46.744 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.744 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.744 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:46.744 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775459+0000", 2024-05-08T08:52:46.745 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5639999999999997e-05 2024-05-08T08:52:46.745 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.745 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.745 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:46.745 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775681+0000", 2024-05-08T08:52:46.745 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00022168000000000001 2024-05-08T08:52:46.745 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.745 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:46.745 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.746 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.746 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.746 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6761 3.16 3:6bd6d6da:::rbd_data.1140782590a5.0000000000000015:6 [read 1095680~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:46.746 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:52:46.746 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 14.954206476, 2024-05-08T08:52:46.746 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052751275, 2024-05-08T08:52:46.747 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:46.747 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:46.747 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:46.747 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:46.747 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:46.747 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:46.748 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6761 2024-05-08T08:52:46.748 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.748 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:46.748 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.748 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:46.748 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:52:46.748 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:46.749 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.749 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.749 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:46.749 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:52:46.749 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:46.749 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.749 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.750 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:46.750 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724939+0000", 2024-05-08T08:52:46.750 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001809817 2024-05-08T08:52:46.750 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.750 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.750 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:46.750 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724943+0000", 2024-05-08T08:52:46.751 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4920000000000004e-06 2024-05-08T08:52:46.751 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.751 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.751 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:46.751 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724956+0000", 2024-05-08T08:52:46.751 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2427e-05 2024-05-08T08:52:46.752 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.752 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.752 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:46.752 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724963+0000", 2024-05-08T08:52:46.752 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2559999999999996e-06 2024-05-08T08:52:46.752 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.753 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.753 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:46.753 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775711+0000", 2024-05-08T08:52:46.753 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050747621 2024-05-08T08:52:46.753 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.753 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.754 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:46.754 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775785+0000", 2024-05-08T08:52:46.754 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4634000000000004e-05 2024-05-08T08:52:46.754 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.754 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.754 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:46.754 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775880+0000", 2024-05-08T08:52:46.755 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5027999999999999e-05 2024-05-08T08:52:46.755 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.755 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:46.755 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.755 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.755 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.756 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6770 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1765376~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:46.756 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:52:46.756 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 14.952175381, 2024-05-08T08:52:46.756 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052339296, 2024-05-08T08:52:46.756 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:46.756 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:46.757 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:46.757 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:46.757 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:46.757 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:46.757 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6770 2024-05-08T08:52:46.757 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.757 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:46.758 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.758 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:46.758 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:52:46.761 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:46.761 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.762 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.762 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:46.762 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:52:46.762 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:46.762 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.762 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.762 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:46.762 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725161+0000", 2024-05-08T08:52:46.762 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-05-08T08:52:46.762 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.762 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.763 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:46.763 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725163+0000", 2024-05-08T08:52:46.763 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4370000000000001e-06 2024-05-08T08:52:46.763 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.763 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.763 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:46.763 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725169+0000", 2024-05-08T08:52:46.763 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8409999999999996e-06 2024-05-08T08:52:46.764 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.764 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.764 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:46.764 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725174+0000", 2024-05-08T08:52:46.764 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7249999999999997e-06 2024-05-08T08:52:46.764 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.764 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.764 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:46.764 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777343+0000", 2024-05-08T08:52:46.764 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052169633999999999 2024-05-08T08:52:46.764 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.765 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.765 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:46.765 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777409+0000", 2024-05-08T08:52:46.765 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6033999999999998e-05 2024-05-08T08:52:46.765 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.765 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.765 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:46.765 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777499+0000", 2024-05-08T08:52:46.765 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0224999999999998e-05 2024-05-08T08:52:46.765 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.765 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:46.765 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.765 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.766 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.766 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6771 3.16 3:6bd6d6da:::rbd_data.1140782590a5.0000000000000015:6 [read 1177600~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:46.766 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:52:46.766 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 14.952155564, 2024-05-08T08:52:46.766 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052438604999999999, 2024-05-08T08:52:46.766 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:46.766 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:46.766 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:46.766 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:46.766 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:46.766 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:46.766 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6771 2024-05-08T08:52:46.766 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.766 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:46.767 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.767 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:46.767 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:52:46.767 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:46.767 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.767 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.767 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:46.767 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:52:46.767 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:46.767 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.767 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.767 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:46.767 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:52:46.767 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:52:46.767 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.767 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.767 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:46.768 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725188+0000", 2024-05-08T08:52:46.768 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1579999999999999e-06 2024-05-08T08:52:46.768 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.768 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.768 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:46.768 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725195+0000", 2024-05-08T08:52:46.768 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8909999999999995e-06 2024-05-08T08:52:46.768 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.768 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.769 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:46.769 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725200+0000", 2024-05-08T08:52:46.769 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9810000000000003e-06 2024-05-08T08:52:46.769 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.769 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.769 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:46.769 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777527+0000", 2024-05-08T08:52:46.769 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052326582000000003 2024-05-08T08:52:46.769 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.769 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.769 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:46.769 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777566+0000", 2024-05-08T08:52:46.769 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9447999999999999e-05 2024-05-08T08:52:46.769 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.769 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.769 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:46.770 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777619+0000", 2024-05-08T08:52:46.770 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2135000000000001e-05 2024-05-08T08:52:46.770 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.770 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:46.770 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.770 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.770 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.770 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6772 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1781760~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:46.770 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:52:46.770 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 14.952127507, 2024-05-08T08:52:46.771 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052601799999999997, 2024-05-08T08:52:46.771 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:46.771 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:46.771 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:46.771 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:46.771 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:46.771 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:46.771 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6772 2024-05-08T08:52:46.771 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.771 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:46.771 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.771 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:46.771 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:52:46.771 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:46.771 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.771 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.772 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:46.772 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:52:46.772 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:46.772 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.772 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.772 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:46.772 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726939+0000", 2024-05-08T08:52:46.772 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001730493 2024-05-08T08:52:46.772 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.772 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.772 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:46.772 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726943+0000", 2024-05-08T08:52:46.772 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7330000000000003e-06 2024-05-08T08:52:46.772 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.773 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.773 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:46.773 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726954+0000", 2024-05-08T08:52:46.773 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1107000000000001e-05 2024-05-08T08:52:46.773 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.773 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.773 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:46.773 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726962+0000", 2024-05-08T08:52:46.773 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.503e-06 2024-05-08T08:52:46.773 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.773 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.773 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:46.773 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777653+0000", 2024-05-08T08:52:46.773 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050691542999999999 2024-05-08T08:52:46.774 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.774 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.774 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:46.774 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777719+0000", 2024-05-08T08:52:46.774 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5228000000000006e-05 2024-05-08T08:52:46.774 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.774 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.774 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:46.774 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777810+0000", 2024-05-08T08:52:46.774 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1193000000000006e-05 2024-05-08T08:52:46.774 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.774 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:46.774 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.774 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.775 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.775 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6782 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1880064~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:46.775 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:52:46.775 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 14.950158999999999, 2024-05-08T08:52:46.775 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052326492000000002, 2024-05-08T08:52:46.775 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:46.775 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:46.775 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:46.775 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:46.775 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:46.775 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:46.775 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6782 2024-05-08T08:52:46.775 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.775 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:46.775 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.775 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:46.776 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:52:46.776 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:46.776 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.776 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.776 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:46.776 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:52:46.776 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:46.776 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.776 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.776 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:46.776 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728939+0000", 2024-05-08T08:52:46.776 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001762719 2024-05-08T08:52:46.777 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.777 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.777 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:46.777 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728944+0000", 2024-05-08T08:52:46.777 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.746e-06 2024-05-08T08:52:46.777 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.777 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.777 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:46.777 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728955+0000", 2024-05-08T08:52:46.777 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1043000000000001e-05 2024-05-08T08:52:46.777 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.777 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.777 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:46.777 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728963+0000", 2024-05-08T08:52:46.778 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5809999999999999e-06 2024-05-08T08:52:46.778 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.778 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.778 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:46.778 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779320+0000", 2024-05-08T08:52:46.778 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050357290999999998 2024-05-08T08:52:46.778 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.778 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.778 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:46.778 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779401+0000", 2024-05-08T08:52:46.778 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0574000000000005e-05 2024-05-08T08:52:46.778 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:46.778 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:46.778 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:46.779 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779503+0000", 2024-05-08T08:52:46.779 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102538 2024-05-08T08:52:46.779 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.779 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:46.779 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.779 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:46.779 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:46.779 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:52:46.781 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:52:46.784+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:52:46.781 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:52:46.846 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T08:52:46.891 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:52:46.894+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:52:46.895 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:52:46.895 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:52:46.895 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:52:46.895 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:52:46.904 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:52:46.974 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:52:46.975 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:52:46.982 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:46.986+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:46.992 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:52:46.993 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:52:47.010 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:52:47.010 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:52:47.028 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:52:47.028 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:52:47.046 INFO:teuthology.orchestra.run.smithi059.stderr:noscrub is set 2024-05-08T08:52:47.061 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:52:47.061 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:52:47.077 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:47.077 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:47.081 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:52:47.082 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:52:47.083 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:47.086+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:47.087 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:52:47.184 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:47.188+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:47.244 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:47.244 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:47.253 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:52:47.286 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:47.289+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:47.363 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:47.363 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:47.372 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:52:47.387 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:47.391+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:47.482 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:47.482 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:47.488 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:52:47.491+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:52:47.490 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:52:47.589 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:52:47.592+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:52:47.641 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:47.641 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:47.649 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:52:47.691 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:52:47.694+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:52:47.791 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:47.795+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:47.800 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:47.800 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:47.809 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:52:47.892 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:52:47.895+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:52:47.917 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:47.918 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:47.926 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:52:48.011 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:52:48.014+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:52:48.091 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:48.091 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:48.095 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:52:48.099+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:52:48.101 INFO:teuthology.orchestra.run.smithi196.stdout:1940 write 0x7f8a800 thru 0x7f96e5e (0xc65f bytes) 2024-05-08T08:52:48.104 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:52:48.117 INFO:teuthology.orchestra.run.smithi196.stdout:1942 trunc from 0x7f97000 to 0xdbed000 2024-05-08T08:52:48.140 INFO:teuthology.orchestra.run.smithi196.stdout:1946 write 0x1be1800 thru 0x1be91df (0x79e0 bytes) 2024-05-08T08:52:48.141 INFO:teuthology.orchestra.run.smithi196.stdout:1947 punch from 0x4bd9c00 to 0x4be1000, (0x7400 bytes) 2024-05-08T08:52:48.141 INFO:teuthology.orchestra.run.smithi196.stdout:1948 write 0x9645200 thru 0x964bb48 (0x6949 bytes) 2024-05-08T08:52:48.142 INFO:teuthology.orchestra.run.smithi196.stdout:1952 punch from 0x9183c00 to 0x918a600, (0x6a00 bytes) 2024-05-08T08:52:48.142 INFO:teuthology.orchestra.run.smithi196.stdout:1956 punch from 0x6296200 to 0x6299e00, (0x3c00 bytes) 2024-05-08T08:52:48.142 INFO:teuthology.orchestra.run.smithi196.stdout:1957 write 0x117400 thru 0x11e054 (0x6c55 bytes) 2024-05-08T08:52:48.144 INFO:teuthology.orchestra.run.smithi196.stdout:1958 read 0x7a66e00 thru 0x7a6907d (0x227e bytes) 2024-05-08T08:52:48.162 INFO:teuthology.orchestra.run.smithi196.stdout:1959 trunc from 0xdbed000 to 0x2949200 2024-05-08T08:52:48.195 INFO:teuthology.orchestra.run.smithi196.stdout:1962 trunc from 0x2949200 to 0x6d39400 2024-05-08T08:52:48.197 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:52:48.200+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:52:48.213 INFO:teuthology.orchestra.run.smithi196.stdout:1965 punch from 0x2af6400 to 0x2afa600, (0x4200 bytes) 2024-05-08T08:52:48.215 INFO:teuthology.orchestra.run.smithi196.stdout:1966 read 0x3889c00 thru 0x3891e87 (0x8288 bytes) 2024-05-08T08:52:48.234 INFO:teuthology.orchestra.run.smithi196.stdout:1967 write 0x77e6c00 thru 0x77eb373 (0x4774 bytes) 2024-05-08T08:52:48.236 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:48.236 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:48.246 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:52:48.259 INFO:teuthology.orchestra.run.smithi196.stdout:1968 write 0xa60fa00 thru 0xa611f9b (0x259c bytes) 2024-05-08T08:52:48.260 INFO:teuthology.orchestra.run.smithi196.stdout:1969 read 0x774200 thru 0x783ca8 (0xfaa9 bytes) 2024-05-08T08:52:48.283 INFO:teuthology.orchestra.run.smithi196.stdout:1970 write 0xc6ab200 thru 0xc6ac39c (0x119d bytes) 2024-05-08T08:52:48.283 INFO:teuthology.orchestra.run.smithi196.stdout:1971 read 0x9546200 thru 0x9555d01 (0xfb02 bytes) 2024-05-08T08:52:48.287 INFO:teuthology.orchestra.run.smithi196.stdout:1973 trunc from 0xc6ac400 to 0x9c48a00 2024-05-08T08:52:48.298 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:52:48.301+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:52:48.314 INFO:teuthology.orchestra.run.smithi196.stdout:1975 read 0x45b5a00 thru 0x45c3c1a (0xe21b bytes) 2024-05-08T08:52:48.314 INFO:teuthology.orchestra.run.smithi196.stdout:1977 write 0x5200 thru 0xe0fe (0x8eff bytes) 2024-05-08T08:52:48.315 INFO:teuthology.orchestra.run.smithi196.stdout:1978 write 0x457ca00 thru 0x458279a (0x5d9b bytes) 2024-05-08T08:52:48.315 INFO:teuthology.orchestra.run.smithi196.stdout:1981 write 0x485ea00 thru 0x486c7a0 (0xdda1 bytes) 2024-05-08T08:52:48.317 INFO:teuthology.orchestra.run.smithi196.stdout:1982 read 0x20f7800 thru 0x20f8392 (0xb93 bytes) 2024-05-08T08:52:48.317 INFO:teuthology.orchestra.run.smithi196.stdout:1983 write 0x330f400 thru 0x331677f (0x7380 bytes) 2024-05-08T08:52:48.317 INFO:teuthology.orchestra.run.smithi196.stdout:1984 write 0x9606a00 thru 0x9612bdb (0xc1dc bytes) 2024-05-08T08:52:48.318 INFO:teuthology.orchestra.run.smithi196.stdout:1986 read 0x5168a00 thru 0x516b417 (0x2a18 bytes) 2024-05-08T08:52:48.318 INFO:teuthology.orchestra.run.smithi196.stdout:1988 punch from 0x8953e00 to 0x895ee00, (0xb000 bytes) 2024-05-08T08:52:48.319 INFO:teuthology.orchestra.run.smithi196.stdout:1989 write 0x9418000 thru 0x941d6d5 (0x56d6 bytes) 2024-05-08T08:52:48.319 INFO:teuthology.orchestra.run.smithi196.stdout:1990 write 0x3d06800 thru 0x3d09abe (0x32bf bytes) 2024-05-08T08:52:48.341 INFO:teuthology.orchestra.run.smithi196.stdout:1991 write 0xe48aa00 thru 0xe48f344 (0x4945 bytes) 2024-05-08T08:52:48.342 INFO:teuthology.orchestra.run.smithi196.stdout:1993 write 0x41bd400 thru 0x41c1637 (0x4238 bytes) 2024-05-08T08:52:48.342 INFO:teuthology.orchestra.run.smithi196.stdout:1994 write 0x40df200 thru 0x40e25b5 (0x33b6 bytes) 2024-05-08T08:52:48.346 INFO:teuthology.orchestra.run.smithi196.stdout:1995 trunc from 0xe48f400 to 0xd8ad600 2024-05-08T08:52:48.359 INFO:teuthology.orchestra.run.smithi196.stdout:1997 trunc from 0xd8ad600 to 0xdc3b800 2024-05-08T08:52:48.366 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:48.366 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:48.370 INFO:teuthology.orchestra.run.smithi196.stdout:2001 write 0x9fc800 thru 0xa07d14 (0xb515 bytes) 2024-05-08T08:52:48.372 INFO:teuthology.orchestra.run.smithi196.stdout:2002 write 0xedf600 thru 0xeef326 (0xfd27 bytes) 2024-05-08T08:52:48.373 INFO:teuthology.orchestra.run.smithi196.stdout:2004 write 0x8d25400 thru 0x8d2b19f (0x5da0 bytes) 2024-05-08T08:52:48.376 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:52:48.377 INFO:teuthology.orchestra.run.smithi196.stdout:2007 trunc from 0xdc3b800 to 0xbc0dc00 2024-05-08T08:52:48.404 INFO:teuthology.orchestra.run.smithi196.stdout:2008 read 0x7187600 thru 0x71956ff (0xe100 bytes) 2024-05-08T08:52:48.404 INFO:teuthology.orchestra.run.smithi196.stdout:2010 read 0x4f97400 thru 0x4fa3f84 (0xcb85 bytes) 2024-05-08T08:52:48.404 INFO:teuthology.orchestra.run.smithi196.stdout:2014 write 0xa16a400 thru 0xa1792ad (0xeeae bytes) 2024-05-08T08:52:48.404 INFO:teuthology.orchestra.run.smithi196.stdout:2017 read 0x4eaa800 thru 0x4eb3126 (0x8927 bytes) 2024-05-08T08:52:48.404 INFO:teuthology.orchestra.run.smithi196.stdout:2020 punch from 0x2edf400 to 0x2ee1000, (0x1c00 bytes) 2024-05-08T08:52:48.404 INFO:teuthology.orchestra.run.smithi196.stdout:2021 write 0x4d7e800 thru 0x4d86ae2 (0x82e3 bytes) 2024-05-08T08:52:48.404 INFO:teuthology.orchestra.run.smithi196.stdout:2023 write 0xb529000 thru 0xb52e87a (0x587b bytes) 2024-05-08T08:52:48.405 INFO:teuthology.orchestra.run.smithi196.stdout:2024 punch from 0xba9a000 to 0xbaa3c00, (0x9c00 bytes) 2024-05-08T08:52:48.405 INFO:teuthology.orchestra.run.smithi196.stdout:2026 read 0xbbb0a00 thru 0xbbb1e8b (0x148c bytes) 2024-05-08T08:52:48.419 INFO:teuthology.orchestra.run.smithi196.stdout:2028 write 0xde98400 thru 0xde9b186 (0x2d87 bytes) 2024-05-08T08:52:48.422 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:48.425+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:48.426 INFO:teuthology.orchestra.run.smithi196.stdout:2035 read 0x18db000 thru 0x18e20d4 (0x70d5 bytes) 2024-05-08T08:52:48.438 INFO:teuthology.orchestra.run.smithi196.stdout:2037 trunc from 0xde9b200 to 0x47e5a00 2024-05-08T08:52:48.479 INFO:teuthology.orchestra.run.smithi196.stdout:2038 write 0xcefe400 thru 0xcf07908 (0x9509 bytes) 2024-05-08T08:52:48.486 INFO:teuthology.orchestra.run.smithi196.stdout:2040 read 0x7c66800 thru 0x7c6b6d6 (0x4ed7 bytes) 2024-05-08T08:52:48.486 INFO:teuthology.orchestra.run.smithi196.stdout:2042 write 0x636800 thru 0x6393fa (0x2bfb bytes) 2024-05-08T08:52:48.486 INFO:teuthology.orchestra.run.smithi196.stdout:2043 punch from 0x7c88600 to 0x7c8f000, (0x6a00 bytes) 2024-05-08T08:52:48.487 INFO:teuthology.orchestra.run.smithi196.stdout:2044 write 0x7aee000 thru 0x7afcace (0xeacf bytes) 2024-05-08T08:52:48.487 INFO:teuthology.orchestra.run.smithi196.stdout:2048 punch from 0xa112600 to 0xa120400, (0xde00 bytes) 2024-05-08T08:52:48.487 INFO:teuthology.orchestra.run.smithi196.stdout:2049 read 0xadc1400 thru 0xadd0f8e (0xfb8f bytes) 2024-05-08T08:52:48.488 INFO:teuthology.orchestra.run.smithi196.stdout:2050 write 0x86ba00 thru 0x875491 (0x9a92 bytes) 2024-05-08T08:52:48.488 INFO:teuthology.orchestra.run.smithi196.stdout:2052 write 0x1448c00 thru 0x144ac98 (0x2099 bytes) 2024-05-08T08:52:48.489 INFO:teuthology.orchestra.run.smithi196.stdout:2053 read 0x2b7d000 thru 0x2b8a8e3 (0xd8e4 bytes) 2024-05-08T08:52:48.489 INFO:teuthology.orchestra.run.smithi196.stdout:2055 punch from 0xbeec600 to 0xbeed600, (0x1000 bytes) 2024-05-08T08:52:48.489 INFO:teuthology.orchestra.run.smithi196.stdout:2057 read 0x43fca00 thru 0x44070b1 (0xa6b2 bytes) 2024-05-08T08:52:48.492 INFO:teuthology.orchestra.run.smithi196.stdout:2061 punch from 0xb4f0c00 to 0xb4f6600, (0x5a00 bytes) 2024-05-08T08:52:48.494 INFO:teuthology.orchestra.run.smithi196.stdout:2062 trunc from 0xcf07a00 to 0xbc3ac00 2024-05-08T08:52:48.500 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:48.503+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:48.502 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:48.502 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:48.511 INFO:teuthology.orchestra.run.smithi196.stdout:2063 read 0x2aa9200 thru 0x2ab3fc6 (0xadc7 bytes) 2024-05-08T08:52:48.512 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:52:48.512 INFO:teuthology.orchestra.run.smithi196.stdout:2065 read 0x2072c00 thru 0x2076df2 (0x41f3 bytes) 2024-05-08T08:52:48.523 INFO:teuthology.orchestra.run.smithi196.stdout:2066 write 0xcdcda00 thru 0xcdd6a67 (0x9068 bytes) 2024-05-08T08:52:48.523 INFO:teuthology.orchestra.run.smithi196.stdout:2067 punch from 0xb2bac00 to 0xb2bf000, (0x4400 bytes) 2024-05-08T08:52:48.524 INFO:teuthology.orchestra.run.smithi196.stdout:2068 read 0x1ebd400 thru 0x1ecc55a (0xf15b bytes) 2024-05-08T08:52:48.526 INFO:teuthology.orchestra.run.smithi196.stdout:2071 trunc from 0xcdd6c00 to 0xb7f7800 2024-05-08T08:52:48.546 INFO:teuthology.orchestra.run.smithi196.stdout:2073 trunc from 0xb7f7800 to 0x873fe00 2024-05-08T08:52:48.562 INFO:teuthology.orchestra.run.smithi196.stdout:2076 read 0x8028600 thru 0x80342fd (0xbcfe bytes) 2024-05-08T08:52:48.563 INFO:teuthology.orchestra.run.smithi196.stdout:2078 write 0x6581200 thru 0x6581c0b (0xa0c bytes) 2024-05-08T08:52:48.564 INFO:teuthology.orchestra.run.smithi196.stdout:2083 read 0xce6400 thru 0xcef6c5 (0x92c6 bytes) 2024-05-08T08:52:48.564 INFO:teuthology.orchestra.run.smithi196.stdout:2087 read 0x6aacc00 thru 0x6ab6180 (0x9581 bytes) 2024-05-08T08:52:48.567 INFO:teuthology.orchestra.run.smithi196.stdout:2089 trunc from 0x873fe00 to 0xa35c600 2024-05-08T08:52:48.577 INFO:teuthology.orchestra.run.smithi196.stdout:2091 write 0x4ffa200 thru 0x4ffde06 (0x3c07 bytes) 2024-05-08T08:52:48.578 INFO:teuthology.orchestra.run.smithi196.stdout:2092 write 0x273d600 thru 0x274ab44 (0xd545 bytes) 2024-05-08T08:52:48.579 INFO:teuthology.orchestra.run.smithi196.stdout:2093 read 0x2042c00 thru 0x204c9c9 (0x9dca bytes) 2024-05-08T08:52:48.587 INFO:teuthology.orchestra.run.smithi196.stdout:2097 trunc from 0xa35c600 to 0x3d3f200 2024-05-08T08:52:48.601 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:52:48.604+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:52:48.632 INFO:teuthology.orchestra.run.smithi196.stdout:2099 write 0xa547e00 thru 0xa556351 (0xe552 bytes) 2024-05-08T08:52:48.640 INFO:teuthology.orchestra.run.smithi196.stdout:2101 trunc from 0xa556400 to 0x4b1bc00 2024-05-08T08:52:48.660 INFO:teuthology.orchestra.run.smithi196.stdout:2102 read 0x1983000 thru 0x198330f (0x310 bytes) 2024-05-08T08:52:48.662 INFO:teuthology.orchestra.run.smithi196.stdout:2104 read 0x2336600 thru 0x233ea27 (0x8428 bytes) 2024-05-08T08:52:48.662 INFO:teuthology.orchestra.run.smithi196.stdout:2105 read 0xf46800 thru 0xf4766b (0xe6c bytes) 2024-05-08T08:52:48.662 INFO:teuthology.orchestra.run.smithi196.stdout:2106 read 0x143c000 thru 0x1443ed6 (0x7ed7 bytes) 2024-05-08T08:52:48.663 INFO:teuthology.orchestra.run.smithi196.stdout:2107 punch from 0x392b600 to 0x392fa00, (0x4400 bytes) 2024-05-08T08:52:48.663 INFO:teuthology.orchestra.run.smithi196.stdout:truncating to largest ever: 0xee1a600 2024-05-08T08:52:48.667 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:48.668 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:48.675 INFO:teuthology.orchestra.run.smithi196.stdout:2109 trunc from 0x4b1bc00 to 0xee1a600 2024-05-08T08:52:48.677 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:52:48.695 INFO:teuthology.orchestra.run.smithi196.stdout:2110 trunc from 0xee1a600 to 0x6182c00 2024-05-08T08:52:48.702 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:52:48.705+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:52:48.728 INFO:teuthology.orchestra.run.smithi196.stdout:2112 write 0xed4b200 thru 0xed51f48 (0x6d49 bytes) 2024-05-08T08:52:48.728 INFO:teuthology.orchestra.run.smithi196.stdout:2113 read 0x3109400 thru 0x31127a7 (0x93a8 bytes) 2024-05-08T08:52:48.729 INFO:teuthology.orchestra.run.smithi196.stdout:2116 read 0x3cf6400 thru 0x3d0269d (0xc29e bytes) 2024-05-08T08:52:48.730 INFO:teuthology.orchestra.run.smithi196.stdout:2117 write 0x5b65e00 thru 0x5b6b119 (0x531a bytes) 2024-05-08T08:52:48.731 INFO:teuthology.orchestra.run.smithi196.stdout:2118 write 0xa6dda00 thru 0xa6e086e (0x2e6f bytes) 2024-05-08T08:52:48.731 INFO:teuthology.orchestra.run.smithi196.stdout:2119 clone 11 order 21 su 524288 sc 6 2024-05-08T08:52:48.783 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:48.783 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:48.792 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:52:48.802 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:52:48.805+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:52:48.897 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:48.897 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:48.903 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:48.906+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:48.905 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:52:49.005 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:52:49.008+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:52:49.052 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:49.052 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:49.061 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:52:49.070 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-05-08T08:52:49.095 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T08:52:49.115 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:52:49.118+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:52:49.206 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:52:49.210+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:52:49.207 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:49.207 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:49.215 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:52:49.255 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:52:49.255 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:52:49.273 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:52:49.273 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:52:49.290 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:52:49.290 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:52:49.307 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:52:49.310+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:52:49.308 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:52:49.308 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:52:49.321 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:49.321 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:49.325 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:52:49.326 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:52:49.331 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:52:49.344 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:52:49.344 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:52:49.408 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:52:49.411+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:52:49.442 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:52:49.442 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:52:49.443 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:52:49.443 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:52:49.451 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:52:49.509 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:49.512+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:49.601 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:52:49.601 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:52:49.601 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:52:49.601 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:52:49.601 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:52:49.610 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:52:49.610 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:52:49.612+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:52:49.711 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:52:49.714+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:52:49.761 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:52:49.761 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:52:49.761 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:52:49.761 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:52:49.761 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.761 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4621.0:9089 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 23339520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:49.761 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:52:49.762 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 17.270332452000002, 2024-05-08T08:52:49.762 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.13814907200000001, 2024-05-08T08:52:49.762 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:49.762 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:49.762 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:49.762 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:49.762 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4621", 2024-05-08T08:52:49.762 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:49.762 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9089 2024-05-08T08:52:49.762 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.762 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:49.762 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.762 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:49.762 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:52:49.763 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:49.763 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.763 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.763 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:49.763 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:52:49.763 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:49.763 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.763 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.763 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:49.763 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493005+0000", 2024-05-08T08:52:49.763 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1700000000000001e-06 2024-05-08T08:52:49.763 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.763 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.763 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:49.763 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.513963+0000", 2024-05-08T08:52:49.764 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.020957581999999999 2024-05-08T08:52:49.764 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.764 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.764 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:49.764 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.513995+0000", 2024-05-08T08:52:49.764 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1993999999999999e-05 2024-05-08T08:52:49.764 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.764 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.764 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:49.764 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514007+0000", 2024-05-08T08:52:49.764 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1581e-05 2024-05-08T08:52:49.764 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.764 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.764 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:49.764 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514122+0000", 2024-05-08T08:52:49.764 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000115282 2024-05-08T08:52:49.765 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.765 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.765 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op must wait for map", 2024-05-08T08:52:49.765 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514128+0000", 2024-05-08T08:52:49.765 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9950000000000002e-06 2024-05-08T08:52:49.765 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.765 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.765 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:49.765 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.515800+0000", 2024-05-08T08:52:49.765 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001671886 2024-05-08T08:52:49.765 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.765 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.765 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:49.765 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.515893+0000", 2024-05-08T08:52:49.765 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3490000000000001e-05 2024-05-08T08:52:49.765 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.766 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.766 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 3", 2024-05-08T08:52:49.766 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.517647+0000", 2024-05-08T08:52:49.766 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0017535630000000001 2024-05-08T08:52:49.766 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.766 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.766 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:49.766 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.627578+0000", 2024-05-08T08:52:49.766 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.109931557 2024-05-08T08:52:49.766 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.766 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.766 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:49.766 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631055+0000", 2024-05-08T08:52:49.766 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0034761420000000002 2024-05-08T08:52:49.767 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.767 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.767 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:49.767 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631096+0000", 2024-05-08T08:52:49.767 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1059000000000001e-05 2024-05-08T08:52:49.767 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.767 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.767 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:49.767 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631151+0000", 2024-05-08T08:52:49.767 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5770999999999998e-05 2024-05-08T08:52:49.767 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:49.767 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:49.767 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:49.767 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.767 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.767 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1062 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 10903552~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:49.768 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485389+0000", 2024-05-08T08:52:49.768 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 13.277946103, 2024-05-08T08:52:49.768 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070031261999999997, 2024-05-08T08:52:49.768 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:49.768 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:49.768 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:49.768 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:49.768 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:49.768 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:49.768 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1062 2024-05-08T08:52:49.768 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.768 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:49.768 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.768 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:49.769 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485389+0000", 2024-05-08T08:52:49.769 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:49.769 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.769 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.769 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:49.769 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485389+0000", 2024-05-08T08:52:49.769 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:49.769 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.769 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.769 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:49.770 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485389+0000", 2024-05-08T08:52:49.770 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.51e-07 2024-05-08T08:52:49.770 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.770 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.770 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:49.770 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485392+0000", 2024-05-08T08:52:49.770 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.593e-06 2024-05-08T08:52:49.770 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.770 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.770 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:49.770 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485398+0000", 2024-05-08T08:52:49.770 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.793e-06 2024-05-08T08:52:49.770 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.770 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.770 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:49.771 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485404+0000", 2024-05-08T08:52:49.771 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0669999999999999e-06 2024-05-08T08:52:49.771 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.771 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.771 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:49.771 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555214+0000", 2024-05-08T08:52:49.771 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.069810935000000005 2024-05-08T08:52:49.771 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.771 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.771 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:49.771 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555306+0000", 2024-05-08T08:52:49.771 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1939000000000002e-05 2024-05-08T08:52:49.772 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.772 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.772 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:49.772 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555420+0000", 2024-05-08T08:52:49.772 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000113484 2024-05-08T08:52:49.772 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:49.772 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:49.772 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:49.772 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.772 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.772 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1065 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 10944512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:49.772 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485411+0000", 2024-05-08T08:52:49.772 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 13.277923766000001, 2024-05-08T08:52:49.772 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070281386000000001, 2024-05-08T08:52:49.772 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:49.773 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:49.773 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:49.773 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:49.773 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:49.773 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:49.773 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1065 2024-05-08T08:52:49.773 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.773 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:49.773 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.773 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:49.773 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485411+0000", 2024-05-08T08:52:49.773 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:49.773 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.773 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.774 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:49.774 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485411+0000", 2024-05-08T08:52:49.774 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:49.774 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.774 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.774 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:49.774 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485411+0000", 2024-05-08T08:52:49.774 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5600000000000001e-07 2024-05-08T08:52:49.774 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.774 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.774 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:49.774 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485414+0000", 2024-05-08T08:52:49.774 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2299999999999998e-06 2024-05-08T08:52:49.774 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.775 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.775 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:49.775 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485420+0000", 2024-05-08T08:52:49.775 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2179999999999996e-06 2024-05-08T08:52:49.775 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.775 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.775 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:49.775 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485425+0000", 2024-05-08T08:52:49.775 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9830000000000004e-06 2024-05-08T08:52:49.775 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.775 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.775 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:49.775 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555468+0000", 2024-05-08T08:52:49.776 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070043541000000001 2024-05-08T08:52:49.776 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.776 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.776 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:49.776 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555566+0000", 2024-05-08T08:52:49.776 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8082e-05 2024-05-08T08:52:49.776 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.776 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.776 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:49.776 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555692+0000", 2024-05-08T08:52:49.776 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000125976 2024-05-08T08:52:49.776 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:49.776 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:49.776 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:49.776 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.776 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.777 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1067 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 10985472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:49.777 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485431+0000", 2024-05-08T08:52:49.777 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 13.277903630999999, 2024-05-08T08:52:49.777 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070412460999999996, 2024-05-08T08:52:49.777 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:49.777 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:49.777 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:49.777 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:49.777 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:49.777 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:49.777 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1067 2024-05-08T08:52:49.777 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.777 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:49.777 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.777 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:49.777 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485431+0000", 2024-05-08T08:52:49.777 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:49.778 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.778 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.778 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:49.778 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485431+0000", 2024-05-08T08:52:49.778 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:49.778 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.778 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.778 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:49.778 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485432+0000", 2024-05-08T08:52:49.778 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8299999999999998e-07 2024-05-08T08:52:49.778 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.778 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.778 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:49.778 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485434+0000", 2024-05-08T08:52:49.778 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0849999999999999e-06 2024-05-08T08:52:49.779 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.779 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.779 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:49.779 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485440+0000", 2024-05-08T08:52:49.779 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0020000000000003e-06 2024-05-08T08:52:49.779 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.779 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.779 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:49.779 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485445+0000", 2024-05-08T08:52:49.779 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5180000000000004e-06 2024-05-08T08:52:49.779 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.779 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.779 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:49.779 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555731+0000", 2024-05-08T08:52:49.779 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070286214 2024-05-08T08:52:49.780 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.780 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.780 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:49.780 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555787+0000", 2024-05-08T08:52:49.780 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5992999999999997e-05 2024-05-08T08:52:49.780 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.780 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.780 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:49.780 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555844+0000", 2024-05-08T08:52:49.780 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6266e-05 2024-05-08T08:52:49.780 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:49.780 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:49.780 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:49.780 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.780 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.780 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1071 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11026432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:49.781 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:52:49.781 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 13.277882961, 2024-05-08T08:52:49.781 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070580037999999998, 2024-05-08T08:52:49.781 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:49.781 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:49.781 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:49.781 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:49.781 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:49.781 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:49.781 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1071 2024-05-08T08:52:49.781 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.781 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:49.781 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.781 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:49.782 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:52:49.782 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:49.782 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.782 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.782 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:49.782 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:52:49.782 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:49.782 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.782 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.782 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:49.782 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:52:49.782 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2800000000000002e-07 2024-05-08T08:52:49.782 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.782 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.782 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:49.783 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485454+0000", 2024-05-08T08:52:49.783 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2290000000000002e-06 2024-05-08T08:52:49.783 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.783 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.783 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:49.783 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485461+0000", 2024-05-08T08:52:49.783 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9920000000000001e-06 2024-05-08T08:52:49.783 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.783 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.783 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:49.783 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485467+0000", 2024-05-08T08:52:49.783 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3979999999999998e-06 2024-05-08T08:52:49.784 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.784 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.784 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:49.784 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555880+0000", 2024-05-08T08:52:49.784 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070413165 2024-05-08T08:52:49.784 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.784 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.784 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:49.784 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555926+0000", 2024-05-08T08:52:49.784 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6217000000000002e-05 2024-05-08T08:52:49.784 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.784 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.784 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:49.784 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556032+0000", 2024-05-08T08:52:49.784 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000105609 2024-05-08T08:52:49.785 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:49.785 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:49.785 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:49.785 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.785 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.785 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1073 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11067392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:49.785 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:52:49.785 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 13.277861740000001, 2024-05-08T08:52:49.785 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070821404000000004, 2024-05-08T08:52:49.785 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:49.785 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:49.785 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:49.785 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:49.785 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:49.786 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:49.786 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1073 2024-05-08T08:52:49.786 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.786 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:49.786 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.786 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:49.786 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:52:49.786 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:49.786 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.786 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.786 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:49.786 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:52:49.786 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:49.786 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.787 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.787 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:49.787 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:52:49.787 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.08e-07 2024-05-08T08:52:49.787 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.787 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.787 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:49.787 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485476+0000", 2024-05-08T08:52:49.787 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3929999999999998e-06 2024-05-08T08:52:49.787 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.787 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.787 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:49.787 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485485+0000", 2024-05-08T08:52:49.787 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1880000000000008e-06 2024-05-08T08:52:49.788 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.788 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.788 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:49.788 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485494+0000", 2024-05-08T08:52:49.788 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3109999999999995e-06 2024-05-08T08:52:49.788 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.788 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.788 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:49.789 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556082+0000", 2024-05-08T08:52:49.789 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070587426999999994 2024-05-08T08:52:49.789 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.789 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.789 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:49.789 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556177+0000", 2024-05-08T08:52:49.789 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5051999999999997e-05 2024-05-08T08:52:49.789 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.789 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.789 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:49.789 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556294+0000", 2024-05-08T08:52:49.789 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000117625 2024-05-08T08:52:49.789 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:49.789 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:49.789 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:49.790 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.790 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.790 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1075 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11108352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:49.790 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:52:49.790 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 13.277831163, 2024-05-08T08:52:49.790 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071045026999999997, 2024-05-08T08:52:49.790 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:49.790 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:49.790 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:49.790 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:49.790 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:49.790 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:49.790 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1075 2024-05-08T08:52:49.790 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.791 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:49.791 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.791 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:49.791 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:52:49.791 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:49.791 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.791 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.791 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:49.791 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:52:49.791 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:49.791 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.791 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.791 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:49.791 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:52:49.792 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.32e-07 2024-05-08T08:52:49.792 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.792 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.792 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:49.792 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485518+0000", 2024-05-08T08:52:49.792 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3769e-05 2024-05-08T08:52:49.792 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.792 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.792 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:49.792 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485527+0000", 2024-05-08T08:52:49.792 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6779999999999995e-06 2024-05-08T08:52:49.792 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.792 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.792 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:49.792 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485532+0000", 2024-05-08T08:52:49.793 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9830000000000004e-06 2024-05-08T08:52:49.793 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.793 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.793 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:49.793 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556340+0000", 2024-05-08T08:52:49.793 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070807743000000006 2024-05-08T08:52:49.793 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.793 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.793 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:49.793 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556435+0000", 2024-05-08T08:52:49.793 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4732e-05 2024-05-08T08:52:49.793 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.793 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.793 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:49.794 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556549+0000", 2024-05-08T08:52:49.794 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011369 2024-05-08T08:52:49.794 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:49.794 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:49.794 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:49.794 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.794 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.794 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1078 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11149312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:49.794 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:52:49.794 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 13.277790963999999, 2024-05-08T08:52:49.794 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071167276000000002, 2024-05-08T08:52:49.794 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:49.794 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:49.794 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:49.795 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:49.795 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:49.795 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:49.795 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1078 2024-05-08T08:52:49.795 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.795 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:49.795 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.795 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:49.795 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:52:49.795 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:49.795 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.795 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.795 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:49.795 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:52:49.795 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:49.795 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.796 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.796 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:49.796 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:52:49.796 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5600000000000001e-07 2024-05-08T08:52:49.796 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.796 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.796 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:49.796 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485547+0000", 2024-05-08T08:52:49.796 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3590000000000002e-06 2024-05-08T08:52:49.796 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.796 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.796 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:49.796 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485553+0000", 2024-05-08T08:52:49.796 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2770000000000002e-06 2024-05-08T08:52:49.796 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.796 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.797 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:49.797 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485558+0000", 2024-05-08T08:52:49.797 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.95e-06 2024-05-08T08:52:49.797 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.797 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.797 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:49.797 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556582+0000", 2024-05-08T08:52:49.797 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071024444000000006 2024-05-08T08:52:49.797 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.797 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.797 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:49.797 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556637+0000", 2024-05-08T08:52:49.797 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4672000000000001e-05 2024-05-08T08:52:49.797 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.797 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.797 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:49.798 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556711+0000", 2024-05-08T08:52:49.798 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4117999999999997e-05 2024-05-08T08:52:49.798 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:49.798 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:49.798 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:49.798 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.798 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.798 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1080 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11190272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:49.798 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:52:49.798 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 13.277770231, 2024-05-08T08:52:49.798 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071350721000000006, 2024-05-08T08:52:49.798 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:49.798 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:49.798 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:49.798 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:49.799 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:49.799 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:49.799 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1080 2024-05-08T08:52:49.799 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.799 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:49.799 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.799 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:49.799 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:52:49.799 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:49.799 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.799 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.799 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:49.799 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:52:49.799 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:49.799 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.799 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.800 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:49.800 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:52:49.800 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.03e-07 2024-05-08T08:52:49.800 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.800 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.800 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:49.800 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485567+0000", 2024-05-08T08:52:49.800 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1090000000000001e-06 2024-05-08T08:52:49.800 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.800 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.800 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:49.800 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485573+0000", 2024-05-08T08:52:49.800 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3049999999999998e-06 2024-05-08T08:52:49.800 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.800 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.800 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:49.801 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485578+0000", 2024-05-08T08:52:49.801 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0130000000000001e-06 2024-05-08T08:52:49.801 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.801 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.801 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:49.801 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556743+0000", 2024-05-08T08:52:49.801 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071164449000000005 2024-05-08T08:52:49.801 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.801 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.801 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:49.801 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556797+0000", 2024-05-08T08:52:49.801 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4589999999999997e-05 2024-05-08T08:52:49.801 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.801 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.801 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:49.802 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556915+0000", 2024-05-08T08:52:49.802 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000117852 2024-05-08T08:52:49.802 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:49.802 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:49.802 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:49.802 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.802 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.802 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1082 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11231232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:49.802 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:52:49.802 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 13.277749988, 2024-05-08T08:52:49.802 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071591396000000002, 2024-05-08T08:52:49.802 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:49.802 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:49.803 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:49.803 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:49.803 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:49.803 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:49.803 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1082 2024-05-08T08:52:49.803 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.803 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:49.803 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.803 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:49.803 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:52:49.803 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:49.803 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.803 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.803 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:49.803 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:52:49.804 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:49.804 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.804 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.804 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:49.804 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:52:49.804 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1800000000000001e-07 2024-05-08T08:52:49.804 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.804 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.804 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:49.804 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485587+0000", 2024-05-08T08:52:49.804 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2589999999999999e-06 2024-05-08T08:52:49.804 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.804 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.804 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:49.805 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485594+0000", 2024-05-08T08:52:49.805 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0789999999999997e-06 2024-05-08T08:52:49.805 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.805 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.805 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:49.805 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485598+0000", 2024-05-08T08:52:49.805 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7480000000000001e-06 2024-05-08T08:52:49.805 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.805 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.805 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:49.805 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556971+0000", 2024-05-08T08:52:49.805 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071372774999999999 2024-05-08T08:52:49.805 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.805 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.805 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:49.805 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557069+0000", 2024-05-08T08:52:49.805 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.7732999999999994e-05 2024-05-08T08:52:49.806 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.806 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.806 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:49.806 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557176+0000", 2024-05-08T08:52:49.806 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000107384 2024-05-08T08:52:49.806 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:49.806 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:49.806 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:49.806 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.806 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.806 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1084 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11272192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:49.806 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:52:49.806 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 13.277711349, 2024-05-08T08:52:49.806 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071808977999999996, 2024-05-08T08:52:49.807 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:49.807 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:49.807 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:49.807 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:49.807 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:49.807 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:49.807 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1084 2024-05-08T08:52:49.807 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.807 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:49.807 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.807 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:49.807 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:52:49.807 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:49.807 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.807 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.807 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:49.808 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:52:49.808 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:49.808 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.808 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.808 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:49.808 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485624+0000", 2024-05-08T08:52:49.808 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:52:49.808 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.808 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.808 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:49.808 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485626+0000", 2024-05-08T08:52:49.808 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9460000000000001e-06 2024-05-08T08:52:49.808 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.808 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.808 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:49.808 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485633+0000", 2024-05-08T08:52:49.808 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1149999999999996e-06 2024-05-08T08:52:49.809 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.809 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.809 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:49.809 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485638+0000", 2024-05-08T08:52:49.809 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5670000000000002e-06 2024-05-08T08:52:49.809 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.809 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.809 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:49.809 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557226+0000", 2024-05-08T08:52:49.809 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071587267999999996 2024-05-08T08:52:49.809 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.809 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.809 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:49.809 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557321+0000", 2024-05-08T08:52:49.809 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5785000000000005e-05 2024-05-08T08:52:49.809 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.810 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.810 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:49.810 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557432+0000", 2024-05-08T08:52:49.810 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011092800000000001 2024-05-08T08:52:49.810 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:49.810 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:49.810 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:49.810 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.810 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.810 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1086 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11313152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:49.810 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:52:49.810 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 13.277688022, 2024-05-08T08:52:49.810 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071933132999999996, 2024-05-08T08:52:49.810 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:49.810 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:49.811 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:49.811 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:49.811 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:49.811 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:49.811 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1086 2024-05-08T08:52:49.811 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.811 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:49.811 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.811 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:49.811 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:52:49.811 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:49.811 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.811 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.811 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:49.811 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:52:49.811 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:49.812 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.812 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.812 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:49.812 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486328+0000", 2024-05-08T08:52:49.812 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00068107900000000002 2024-05-08T08:52:49.812 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.812 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.812 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:49.812 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486331+0000", 2024-05-08T08:52:49.812 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.585e-06 2024-05-08T08:52:49.812 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.812 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.812 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:49.812 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486343+0000", 2024-05-08T08:52:49.812 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1856e-05 2024-05-08T08:52:49.812 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.813 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.813 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:49.813 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486350+0000", 2024-05-08T08:52:49.813 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8600000000000004e-06 2024-05-08T08:52:49.813 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.813 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.813 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:49.813 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557470+0000", 2024-05-08T08:52:49.813 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071120326999999997 2024-05-08T08:52:49.813 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.813 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.813 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:49.813 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557514+0000", 2024-05-08T08:52:49.813 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3909000000000003e-05 2024-05-08T08:52:49.813 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.814 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.814 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:49.814 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557580+0000", 2024-05-08T08:52:49.814 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5517000000000003e-05 2024-05-08T08:52:49.814 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:49.814 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:49.814 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:49.814 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.814 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.814 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1089 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11354112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:49.814 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:52:49.814 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 13.27697515, 2024-05-08T08:52:49.814 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071457381, 2024-05-08T08:52:49.814 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:49.815 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:49.815 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:49.815 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:49.815 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:49.815 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:49.815 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1089 2024-05-08T08:52:49.815 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.815 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:49.815 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.815 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:49.815 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:52:49.815 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:49.815 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.815 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.815 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:49.815 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:52:49.815 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:49.816 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.816 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.816 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:49.816 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486851+0000", 2024-05-08T08:52:49.816 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00049102 2024-05-08T08:52:49.816 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.816 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.816 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:49.816 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486854+0000", 2024-05-08T08:52:49.816 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6859999999999999e-06 2024-05-08T08:52:49.816 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.816 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.816 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:49.816 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486864+0000", 2024-05-08T08:52:49.816 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6080000000000002e-06 2024-05-08T08:52:49.816 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.817 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.817 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:49.817 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486870+0000", 2024-05-08T08:52:49.817 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.477e-06 2024-05-08T08:52:49.817 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.817 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.817 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:49.817 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557612+0000", 2024-05-08T08:52:49.817 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070741557999999996 2024-05-08T08:52:49.817 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.817 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.817 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:49.817 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557695+0000", 2024-05-08T08:52:49.817 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3300000000000005e-05 2024-05-08T08:52:49.818 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.818 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.818 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:49.818 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557817+0000", 2024-05-08T08:52:49.818 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012173200000000001 2024-05-08T08:52:49.818 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:49.818 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:49.818 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:49.818 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.818 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.818 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1091 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11395072~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:49.818 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:52:49.818 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 13.276454355, 2024-05-08T08:52:49.818 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071184737999999997, 2024-05-08T08:52:49.818 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:49.819 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:49.819 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:49.819 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:49.819 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:49.819 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:49.819 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1091 2024-05-08T08:52:49.819 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.819 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:49.819 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.819 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:49.819 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:52:49.819 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:49.819 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.819 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.819 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:49.820 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:52:49.820 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:49.820 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.820 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.820 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:49.820 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486881+0000", 2024-05-08T08:52:49.820 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0699999999999997e-07 2024-05-08T08:52:49.820 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.820 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.820 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:49.820 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486883+0000", 2024-05-08T08:52:49.820 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.402e-06 2024-05-08T08:52:49.820 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.820 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.821 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:49.821 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486890+0000", 2024-05-08T08:52:49.821 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0480000000000001e-06 2024-05-08T08:52:49.821 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.821 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.821 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:49.821 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486896+0000", 2024-05-08T08:52:49.821 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7640000000000002e-06 2024-05-08T08:52:49.821 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.821 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.821 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:49.821 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557860+0000", 2024-05-08T08:52:49.821 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070963791999999998 2024-05-08T08:52:49.821 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.821 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.822 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:49.822 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557952+0000", 2024-05-08T08:52:49.822 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2082000000000004e-05 2024-05-08T08:52:49.822 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.822 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.822 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:49.822 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558065+0000", 2024-05-08T08:52:49.822 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000113143 2024-05-08T08:52:49.822 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:49.822 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:49.822 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:49.822 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.822 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.822 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1094 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11436032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:49.823 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:52:49.823 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 13.276431187, 2024-05-08T08:52:49.823 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071400507000000002, 2024-05-08T08:52:49.823 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:49.823 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:49.823 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:49.823 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:49.823 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:49.823 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:49.823 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1094 2024-05-08T08:52:49.823 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.823 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:49.823 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.823 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:49.823 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:52:49.824 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:49.824 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.824 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.824 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:49.824 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:52:49.824 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:49.824 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.824 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.824 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:49.824 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487940+0000", 2024-05-08T08:52:49.824 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001036868 2024-05-08T08:52:49.824 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.824 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.824 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:49.825 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487944+0000", 2024-05-08T08:52:49.825 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7069999999999998e-06 2024-05-08T08:52:49.825 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.825 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.825 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:49.825 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487951+0000", 2024-05-08T08:52:49.825 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2649999999999999e-06 2024-05-08T08:52:49.825 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.825 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.825 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:49.825 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487961+0000", 2024-05-08T08:52:49.825 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4110000000000002e-06 2024-05-08T08:52:49.825 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.825 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.825 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:49.825 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558117+0000", 2024-05-08T08:52:49.826 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070156662999999994 2024-05-08T08:52:49.826 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.826 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.826 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:49.826 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558214+0000", 2024-05-08T08:52:49.826 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6444e-05 2024-05-08T08:52:49.826 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.826 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.826 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:49.826 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558304+0000", 2024-05-08T08:52:49.826 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0148999999999994e-05 2024-05-08T08:52:49.826 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:49.826 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:49.826 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:49.827 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.827 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.827 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1096 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11476992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:49.827 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:52:49.827 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 13.275364656000001, 2024-05-08T08:52:49.827 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070480074000000004, 2024-05-08T08:52:49.827 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:49.827 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:49.827 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:49.827 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:49.827 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:49.827 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:49.827 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1096 2024-05-08T08:52:49.827 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.828 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:49.828 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.828 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:49.828 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:52:49.828 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:49.828 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.828 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.828 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:49.828 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:52:49.828 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:49.828 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.828 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.828 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:49.828 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:52:49.828 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3700000000000001e-07 2024-05-08T08:52:49.828 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.829 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.829 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:49.829 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487973+0000", 2024-05-08T08:52:49.829 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0870000000000001e-06 2024-05-08T08:52:49.829 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.829 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.829 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:49.829 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487977+0000", 2024-05-08T08:52:49.829 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0600000000000001e-06 2024-05-08T08:52:49.829 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.829 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.829 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:49.829 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487980+0000", 2024-05-08T08:52:49.829 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1089999999999998e-06 2024-05-08T08:52:49.829 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.829 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.830 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:49.830 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558338+0000", 2024-05-08T08:52:49.830 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070358233000000006 2024-05-08T08:52:49.830 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.830 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.830 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:49.830 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558392+0000", 2024-05-08T08:52:49.830 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4537999999999999e-05 2024-05-08T08:52:49.830 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.830 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.830 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:49.830 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558450+0000", 2024-05-08T08:52:49.830 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7710000000000001e-05 2024-05-08T08:52:49.830 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:49.830 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:49.831 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:49.831 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.831 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.831 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1099 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11517952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:49.831 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:52:49.831 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 13.27535078, 2024-05-08T08:52:49.831 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070720506000000002, 2024-05-08T08:52:49.831 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:49.831 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:49.831 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:49.831 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:49.831 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:49.831 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:49.831 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1099 2024-05-08T08:52:49.831 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.832 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:49.832 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.832 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:49.832 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:52:49.832 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:49.832 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.832 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.832 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:49.832 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:52:49.832 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:49.832 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.832 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.832 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:49.832 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:52:49.832 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3999999999999998e-07 2024-05-08T08:52:49.833 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.833 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.833 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:49.833 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487986+0000", 2024-05-08T08:52:49.833 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3519999999999999e-06 2024-05-08T08:52:49.833 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.833 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.833 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:49.833 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487989+0000", 2024-05-08T08:52:49.833 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.23e-06 2024-05-08T08:52:49.833 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.833 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.833 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:49.833 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487991+0000", 2024-05-08T08:52:49.833 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6350000000000002e-06 2024-05-08T08:52:49.834 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.834 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.834 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:49.834 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558489+0000", 2024-05-08T08:52:49.834 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070497924000000003 2024-05-08T08:52:49.834 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.834 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.834 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:49.834 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558583+0000", 2024-05-08T08:52:49.834 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3405000000000004e-05 2024-05-08T08:52:49.834 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.834 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.834 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:49.834 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558704+0000", 2024-05-08T08:52:49.835 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012171999999999999 2024-05-08T08:52:49.835 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:49.835 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:49.835 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:49.835 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.835 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.835 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1100 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11558912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:49.835 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:52:49.835 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 13.275337780999999, 2024-05-08T08:52:49.835 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070962266999999996, 2024-05-08T08:52:49.835 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:49.835 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:49.835 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:49.835 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:49.835 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:49.836 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:49.836 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1100 2024-05-08T08:52:49.836 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.836 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:49.836 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.836 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:49.836 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:52:49.836 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:49.836 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.836 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.836 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:49.836 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:52:49.836 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:49.836 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.836 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.837 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:49.837 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488430+0000", 2024-05-08T08:52:49.837 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000432676 2024-05-08T08:52:49.837 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.837 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.837 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:49.837 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488433+0000", 2024-05-08T08:52:49.837 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.45e-06 2024-05-08T08:52:49.837 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.837 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.837 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:49.837 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488442+0000", 2024-05-08T08:52:49.837 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.687e-06 2024-05-08T08:52:49.837 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.837 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.838 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:49.838 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488446+0000", 2024-05-08T08:52:49.838 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4939999999999997e-06 2024-05-08T08:52:49.838 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.838 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.838 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:49.838 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558756+0000", 2024-05-08T08:52:49.838 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070310051999999998 2024-05-08T08:52:49.838 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.838 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.838 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:49.838 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558844+0000", 2024-05-08T08:52:49.838 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7690999999999998e-05 2024-05-08T08:52:49.838 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.838 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.839 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:49.839 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558959+0000", 2024-05-08T08:52:49.839 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000115217 2024-05-08T08:52:49.839 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:49.839 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:49.839 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:49.839 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.839 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.839 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1103 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11599872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:49.839 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:52:49.839 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 13.274880105999999, 2024-05-08T08:52:49.839 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070777100999999995, 2024-05-08T08:52:49.839 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:49.839 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:49.839 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:49.840 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:49.840 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:49.840 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:49.840 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1103 2024-05-08T08:52:49.840 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.840 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:49.840 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.840 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:49.840 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:52:49.840 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:49.840 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.840 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.840 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:49.840 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:52:49.840 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:49.841 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.841 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.841 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:49.841 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488959+0000", 2024-05-08T08:52:49.841 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00050457200000000001 2024-05-08T08:52:49.841 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.841 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.841 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:49.841 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488969+0000", 2024-05-08T08:52:49.841 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.7159999999999998e-06 2024-05-08T08:52:49.841 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.841 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.841 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:49.841 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488981+0000", 2024-05-08T08:52:49.841 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2150000000000001e-05 2024-05-08T08:52:49.842 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.842 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.842 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:49.842 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488988+0000", 2024-05-08T08:52:49.842 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8759999999999999e-06 2024-05-08T08:52:49.842 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.842 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.842 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:49.842 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558992+0000", 2024-05-08T08:52:49.842 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070004358000000003 2024-05-08T08:52:49.842 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.842 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.842 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:49.843 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559093+0000", 2024-05-08T08:52:49.843 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000100299 2024-05-08T08:52:49.843 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.843 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.843 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:49.843 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559232+0000", 2024-05-08T08:52:49.843 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013913 2024-05-08T08:52:49.843 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:49.843 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:49.843 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:49.843 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.843 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.843 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1104 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11640832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:49.843 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:52:49.843 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 13.274335291, 2024-05-08T08:52:49.844 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070521947000000001, 2024-05-08T08:52:49.844 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:49.844 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:49.844 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:49.844 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:49.844 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:49.844 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:49.844 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1104 2024-05-08T08:52:49.844 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.844 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:49.844 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.844 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:49.844 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:52:49.844 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:49.844 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.845 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.845 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:49.845 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:52:49.845 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:49.845 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.845 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.845 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:49.845 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489700+0000", 2024-05-08T08:52:49.845 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00070067600000000001 2024-05-08T08:52:49.845 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.845 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.845 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:49.845 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489704+0000", 2024-05-08T08:52:49.845 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1640000000000003e-06 2024-05-08T08:52:49.846 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.846 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.846 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:49.846 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489716+0000", 2024-05-08T08:52:49.846 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1379000000000001e-05 2024-05-08T08:52:49.846 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.846 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.846 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:49.846 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489724+0000", 2024-05-08T08:52:49.846 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8949999999999998e-06 2024-05-08T08:52:49.846 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.846 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.846 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:49.846 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559278+0000", 2024-05-08T08:52:49.847 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.069554151999999994 2024-05-08T08:52:49.847 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.847 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.847 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:49.847 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559387+0000", 2024-05-08T08:52:49.847 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000109258 2024-05-08T08:52:49.847 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:49.847 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:49.847 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:49.847 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559521+0000", 2024-05-08T08:52:49.847 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000134423 2024-05-08T08:52:49.847 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:49.847 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:49.847 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:49.847 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:49.848 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:49.848 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:52:49.849 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:52:49.850 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:52:49.852+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:52:49.950 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:52:49.953+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:52:50.000 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:52:50.000 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:52:50.001 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:52:50.001 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:52:50.009 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:52:50.050 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:52:50.054+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:52:50.057 INFO:teuthology.orchestra.run.smithi059.stderr:nodeep-scrub is set 2024-05-08T08:52:50.140 INFO:teuthology.orchestra.run.smithi196.stdout:truncating image image_client.0-clone10 from 0xed52000 (overlap 0x2949200) to 0x1ce6600 2024-05-08T08:52:50.151 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:52:50.154+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:52:50.171 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:52:50.171 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:52:50.171 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:52:50.171 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:52:50.171 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:52:50.182 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:52:50.252 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:50.255+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:50.336 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:52:50.337 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:52:50.337 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:52:50.337 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:52:50.337 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.337 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9944 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 749568~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:50.337 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.103900+0000", 2024-05-08T08:52:50.337 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 23.235244141999999, 2024-05-08T08:52:50.337 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057007044, 2024-05-08T08:52:50.337 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:50.337 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:50.337 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:50.338 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:50.338 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:50.338 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:50.338 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9944 2024-05-08T08:52:50.338 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.338 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:50.338 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.338 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:50.338 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103900+0000", 2024-05-08T08:52:50.338 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.338 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.338 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.338 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:50.338 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103900+0000", 2024-05-08T08:52:50.339 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.339 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.339 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.339 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:50.339 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103900+0000", 2024-05-08T08:52:50.339 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6400000000000003e-07 2024-05-08T08:52:50.339 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.339 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.339 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:50.339 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103902+0000", 2024-05-08T08:52:50.339 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.858e-06 2024-05-08T08:52:50.339 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.339 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.339 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:50.339 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103907+0000", 2024-05-08T08:52:50.340 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3410000000000002e-06 2024-05-08T08:52:50.340 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.340 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.340 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:50.340 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103910+0000", 2024-05-08T08:52:50.340 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8540000000000001e-06 2024-05-08T08:52:50.340 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.340 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.340 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:50.340 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.160682+0000", 2024-05-08T08:52:50.340 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.056771822 2024-05-08T08:52:50.340 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.340 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.340 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:50.340 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.160777+0000", 2024-05-08T08:52:50.341 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4555999999999994e-05 2024-05-08T08:52:50.341 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.341 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.341 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:50.341 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.160907+0000", 2024-05-08T08:52:50.341 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000130149 2024-05-08T08:52:50.341 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.341 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:50.341 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.341 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.341 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.341 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9945 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 1441792~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:50.341 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.103914+0000", 2024-05-08T08:52:50.341 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 23.235229549, 2024-05-08T08:52:50.341 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057211979000000003, 2024-05-08T08:52:50.342 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:50.342 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:50.342 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:50.342 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:50.342 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:50.342 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:50.342 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9945 2024-05-08T08:52:50.342 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.342 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:50.342 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.342 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:50.342 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103914+0000", 2024-05-08T08:52:50.342 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.343 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.343 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.343 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:50.343 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103914+0000", 2024-05-08T08:52:50.343 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.343 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.343 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.343 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:50.343 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103915+0000", 2024-05-08T08:52:50.343 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3099999999999999e-07 2024-05-08T08:52:50.343 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.343 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.343 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:50.343 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103916+0000", 2024-05-08T08:52:50.343 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.203e-06 2024-05-08T08:52:50.344 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.344 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.344 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:50.344 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103919+0000", 2024-05-08T08:52:50.344 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.117e-06 2024-05-08T08:52:50.344 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.344 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.344 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:50.344 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103921+0000", 2024-05-08T08:52:50.344 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4169999999999999e-06 2024-05-08T08:52:50.344 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.344 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.344 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:50.344 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.160959+0000", 2024-05-08T08:52:50.344 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057038074000000001 2024-05-08T08:52:50.345 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.345 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.345 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:50.345 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161041+0000", 2024-05-08T08:52:50.345 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1620999999999997e-05 2024-05-08T08:52:50.345 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.345 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.345 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:50.345 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161126+0000", 2024-05-08T08:52:50.345 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.5216000000000005e-05 2024-05-08T08:52:50.345 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.345 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:50.345 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.345 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.345 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.346 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9946 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 765952~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:50.346 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.103925+0000", 2024-05-08T08:52:50.346 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 23.235219182000002, 2024-05-08T08:52:50.346 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057418770000000001, 2024-05-08T08:52:50.346 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:50.346 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:50.346 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:50.346 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:50.346 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:50.346 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:50.346 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9946 2024-05-08T08:52:50.346 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.346 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:50.347 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.347 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:50.347 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103925+0000", 2024-05-08T08:52:50.347 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.347 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.347 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.347 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:50.347 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103925+0000", 2024-05-08T08:52:50.347 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.347 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.347 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.347 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:50.347 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103925+0000", 2024-05-08T08:52:50.347 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9200000000000002e-07 2024-05-08T08:52:50.347 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.347 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.348 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:50.348 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103926+0000", 2024-05-08T08:52:50.348 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1620000000000001e-06 2024-05-08T08:52:50.348 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.348 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.348 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:50.348 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103929+0000", 2024-05-08T08:52:50.348 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9349999999999999e-06 2024-05-08T08:52:50.348 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.348 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.348 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:50.348 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103931+0000", 2024-05-08T08:52:50.348 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4099999999999998e-06 2024-05-08T08:52:50.348 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.349 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.349 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:50.349 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161161+0000", 2024-05-08T08:52:50.349 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057229176999999999 2024-05-08T08:52:50.349 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.349 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.349 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:50.349 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161235+0000", 2024-05-08T08:52:50.349 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4227000000000006e-05 2024-05-08T08:52:50.349 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.349 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.349 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:50.349 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161343+0000", 2024-05-08T08:52:50.349 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010856700000000001 2024-05-08T08:52:50.349 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.350 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:50.350 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.350 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.350 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.350 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9947 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 1458176~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:50.350 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.103935+0000", 2024-05-08T08:52:50.350 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 23.235209304000001, 2024-05-08T08:52:50.350 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057541263000000002, 2024-05-08T08:52:50.350 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:50.350 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:50.350 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:50.350 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:50.351 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:50.351 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:50.351 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9947 2024-05-08T08:52:50.351 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.351 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:50.351 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.351 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:50.351 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103935+0000", 2024-05-08T08:52:50.351 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.351 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.351 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.351 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:50.351 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103935+0000", 2024-05-08T08:52:50.351 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.351 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.351 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.352 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:50.352 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103935+0000", 2024-05-08T08:52:50.352 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.11e-07 2024-05-08T08:52:50.352 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.352 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.352 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:50.352 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103940+0000", 2024-05-08T08:52:50.352 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1030000000000001e-06 2024-05-08T08:52:50.352 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.352 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.352 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:50.352 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103943+0000", 2024-05-08T08:52:50.352 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3869999999999999e-06 2024-05-08T08:52:50.352 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.353 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.353 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:50.353 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103951+0000", 2024-05-08T08:52:50.353 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.2150000000000001e-06 2024-05-08T08:52:50.353 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.353 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.353 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:50.353 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161377+0000", 2024-05-08T08:52:50.353 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057425925000000003 2024-05-08T08:52:50.353 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.353 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.353 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:50.353 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161431+0000", 2024-05-08T08:52:50.353 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3408000000000001e-05 2024-05-08T08:52:50.353 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.354 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.354 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:50.354 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161476+0000", 2024-05-08T08:52:50.354 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5014e-05 2024-05-08T08:52:50.354 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.354 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:50.354 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.354 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.354 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.354 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9948 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 782336~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:50.354 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.103965+0000", 2024-05-08T08:52:50.354 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 23.235178414, 2024-05-08T08:52:50.354 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057746205000000002, 2024-05-08T08:52:50.354 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:50.355 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:50.355 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:50.355 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:50.355 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:50.355 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:50.355 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9948 2024-05-08T08:52:50.355 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.355 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:50.355 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.355 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:50.355 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103965+0000", 2024-05-08T08:52:50.355 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.355 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.355 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.356 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:50.356 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103965+0000", 2024-05-08T08:52:50.356 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.356 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.356 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.356 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:50.356 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103966+0000", 2024-05-08T08:52:50.356 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.44e-07 2024-05-08T08:52:50.356 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.356 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.356 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:50.356 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103968+0000", 2024-05-08T08:52:50.356 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4090000000000001e-06 2024-05-08T08:52:50.356 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.356 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.357 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:50.357 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103973+0000", 2024-05-08T08:52:50.357 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6820000000000002e-06 2024-05-08T08:52:50.357 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.357 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.357 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:50.357 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103976+0000", 2024-05-08T08:52:50.357 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.959e-06 2024-05-08T08:52:50.357 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.357 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.357 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:50.357 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161523+0000", 2024-05-08T08:52:50.357 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057547264000000001 2024-05-08T08:52:50.357 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.358 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.358 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:50.358 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161605+0000", 2024-05-08T08:52:50.358 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1854000000000006e-05 2024-05-08T08:52:50.358 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.358 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.358 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:50.358 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161712+0000", 2024-05-08T08:52:50.358 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000106493 2024-05-08T08:52:50.358 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.358 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:50.358 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.358 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.358 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.359 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9949 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 1474560~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:50.359 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.103980+0000", 2024-05-08T08:52:50.359 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 23.235163794999998, 2024-05-08T08:52:50.359 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057852397, 2024-05-08T08:52:50.359 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:50.359 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:50.359 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:50.359 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:50.359 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:50.359 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:50.359 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9949 2024-05-08T08:52:50.359 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.359 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:50.359 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.360 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:50.360 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103980+0000", 2024-05-08T08:52:50.360 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.360 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.360 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.360 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:50.360 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103980+0000", 2024-05-08T08:52:50.360 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.360 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.360 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.360 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:50.360 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103980+0000", 2024-05-08T08:52:50.360 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.11e-07 2024-05-08T08:52:50.360 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.360 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.361 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:50.361 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103982+0000", 2024-05-08T08:52:50.361 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.269e-06 2024-05-08T08:52:50.361 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.361 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.361 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:50.361 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103985+0000", 2024-05-08T08:52:50.361 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0110000000000001e-06 2024-05-08T08:52:50.361 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.361 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.361 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:50.361 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103987+0000", 2024-05-08T08:52:50.361 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3020000000000002e-06 2024-05-08T08:52:50.361 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.361 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.361 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:50.362 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161744+0000", 2024-05-08T08:52:50.362 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057757115999999997 2024-05-08T08:52:50.362 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.362 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.362 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:50.362 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161791+0000", 2024-05-08T08:52:50.362 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6807000000000001e-05 2024-05-08T08:52:50.362 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.362 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.362 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:50.362 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161832+0000", 2024-05-08T08:52:50.362 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1681000000000003e-05 2024-05-08T08:52:50.362 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.362 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:50.363 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.363 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.363 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.363 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9950 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 798720~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:50.363 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.103990+0000", 2024-05-08T08:52:50.363 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 23.235153560000001, 2024-05-08T08:52:50.363 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.058062890999999998, 2024-05-08T08:52:50.363 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:50.363 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:50.363 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:50.363 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:50.363 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:50.363 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:50.363 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9950 2024-05-08T08:52:50.364 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.364 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:50.364 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.364 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:50.364 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103990+0000", 2024-05-08T08:52:50.364 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.364 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.364 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.364 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:50.364 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103990+0000", 2024-05-08T08:52:50.364 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.364 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.364 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.364 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:50.365 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103991+0000", 2024-05-08T08:52:50.365 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1e-07 2024-05-08T08:52:50.365 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.365 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.365 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:50.365 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103992+0000", 2024-05-08T08:52:50.365 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.074e-06 2024-05-08T08:52:50.365 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.365 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.365 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:50.365 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103995+0000", 2024-05-08T08:52:50.365 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9529999999999999e-06 2024-05-08T08:52:50.365 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.365 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.366 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:50.366 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103997+0000", 2024-05-08T08:52:50.366 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3939999999999999e-06 2024-05-08T08:52:50.366 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.366 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.366 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:50.366 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161871+0000", 2024-05-08T08:52:50.366 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057873854000000002 2024-05-08T08:52:50.366 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.366 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.366 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:50.366 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161950+0000", 2024-05-08T08:52:50.366 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9433999999999999e-05 2024-05-08T08:52:50.366 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.367 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.367 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:50.367 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.162053+0000", 2024-05-08T08:52:50.367 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000102972 2024-05-08T08:52:50.367 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.367 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:50.367 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.367 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.367 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.367 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9951 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 1490944~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:50.367 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.104000+0000", 2024-05-08T08:52:50.367 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 23.235143755999999, 2024-05-08T08:52:50.367 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.058313416999999999, 2024-05-08T08:52:50.367 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:50.367 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:50.368 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:50.368 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:50.368 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:50.368 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:50.368 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9951 2024-05-08T08:52:50.368 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.368 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:50.368 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.368 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:50.368 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.104000+0000", 2024-05-08T08:52:50.368 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.368 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.368 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.368 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:50.369 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.104000+0000", 2024-05-08T08:52:50.369 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.369 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.369 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.369 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:50.369 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.104962+0000", 2024-05-08T08:52:50.369 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00096230399999999998 2024-05-08T08:52:50.369 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.369 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.369 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:50.369 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.104978+0000", 2024-05-08T08:52:50.369 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5126999999999999e-05 2024-05-08T08:52:50.369 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.369 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.369 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:50.370 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.104987+0000", 2024-05-08T08:52:50.370 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8940000000000005e-06 2024-05-08T08:52:50.370 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.370 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.370 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:50.370 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.104994+0000", 2024-05-08T08:52:50.370 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5610000000000004e-06 2024-05-08T08:52:50.370 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.370 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.370 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:50.370 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.162114+0000", 2024-05-08T08:52:50.370 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057119717 2024-05-08T08:52:50.370 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.371 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.371 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:50.371 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.162214+0000", 2024-05-08T08:52:50.371 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.9982999999999995e-05 2024-05-08T08:52:50.371 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.371 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.371 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:50.371 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.162314+0000", 2024-05-08T08:52:50.371 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.9831e-05 2024-05-08T08:52:50.371 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.371 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:50.371 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.371 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.371 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.372 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9952 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 815104~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:50.372 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.105004+0000", 2024-05-08T08:52:50.372 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 23.234139503000002, 2024-05-08T08:52:50.372 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057530855999999998, 2024-05-08T08:52:50.372 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:50.372 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:50.372 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:50.372 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:50.372 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:50.372 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:50.372 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9952 2024-05-08T08:52:50.372 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.372 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:50.372 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.373 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:50.373 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105004+0000", 2024-05-08T08:52:50.373 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.373 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.373 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.373 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:50.373 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105004+0000", 2024-05-08T08:52:50.373 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.373 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.373 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.373 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:50.373 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105005+0000", 2024-05-08T08:52:50.373 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9299999999999998e-07 2024-05-08T08:52:50.373 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.374 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.374 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:50.374 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105007+0000", 2024-05-08T08:52:50.374 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2979999999999999e-06 2024-05-08T08:52:50.374 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.374 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.374 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:50.374 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105017+0000", 2024-05-08T08:52:50.374 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5060000000000001e-06 2024-05-08T08:52:50.374 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.374 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.374 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:50.374 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105022+0000", 2024-05-08T08:52:50.374 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2619999999999998e-06 2024-05-08T08:52:50.374 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.375 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.375 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:50.375 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.162356+0000", 2024-05-08T08:52:50.375 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057334388999999999 2024-05-08T08:52:50.375 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.375 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.375 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:50.375 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.162437+0000", 2024-05-08T08:52:50.375 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0863000000000002e-05 2024-05-08T08:52:50.375 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.375 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.375 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:50.375 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.162535+0000", 2024-05-08T08:52:50.375 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8045000000000001e-05 2024-05-08T08:52:50.375 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.375 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:50.376 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.376 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.376 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.376 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9953 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 1507328~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:50.376 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.105064+0000", 2024-05-08T08:52:50.376 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 23.234079816000001, 2024-05-08T08:52:50.376 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057721531999999999, 2024-05-08T08:52:50.376 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:50.376 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:50.376 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:50.376 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:50.376 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:50.376 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:50.376 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9953 2024-05-08T08:52:50.376 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.377 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:50.377 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.377 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:50.377 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105064+0000", 2024-05-08T08:52:50.377 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.377 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.377 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.377 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:50.377 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105064+0000", 2024-05-08T08:52:50.377 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.377 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.377 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.377 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:50.377 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105065+0000", 2024-05-08T08:52:50.378 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8800000000000002e-07 2024-05-08T08:52:50.378 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.378 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.378 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:50.378 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105067+0000", 2024-05-08T08:52:50.378 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7669999999999999e-06 2024-05-08T08:52:50.378 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.378 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.378 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:50.378 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105076+0000", 2024-05-08T08:52:50.378 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.5259999999999993e-06 2024-05-08T08:52:50.378 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.378 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.378 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:50.379 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105080+0000", 2024-05-08T08:52:50.379 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8e-06 2024-05-08T08:52:50.379 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.379 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.379 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:50.379 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.162596+0000", 2024-05-08T08:52:50.379 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057515822000000001 2024-05-08T08:52:50.379 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.379 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.379 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:50.379 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.162689+0000", 2024-05-08T08:52:50.379 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3258999999999996e-05 2024-05-08T08:52:50.379 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.379 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.380 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:50.380 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.162786+0000", 2024-05-08T08:52:50.380 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6769999999999997e-05 2024-05-08T08:52:50.380 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.380 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:50.380 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.380 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.380 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.380 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9954 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 1523712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:50.380 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.105088+0000", 2024-05-08T08:52:50.380 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 23.234055807000001, 2024-05-08T08:52:50.380 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057925139, 2024-05-08T08:52:50.380 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:50.380 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:50.380 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:50.380 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:50.381 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:50.381 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:50.381 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9954 2024-05-08T08:52:50.381 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.381 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:50.381 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.381 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:50.381 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105088+0000", 2024-05-08T08:52:50.381 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.381 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.381 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.381 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:50.381 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105088+0000", 2024-05-08T08:52:50.381 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.381 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.381 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.382 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:50.382 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105089+0000", 2024-05-08T08:52:50.382 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9599999999999999e-07 2024-05-08T08:52:50.382 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.382 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.382 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:50.382 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105091+0000", 2024-05-08T08:52:50.382 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3649999999999998e-06 2024-05-08T08:52:50.382 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.382 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.382 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:50.382 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105107+0000", 2024-05-08T08:52:50.382 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6370000000000001e-05 2024-05-08T08:52:50.382 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.383 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.383 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:50.383 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105114+0000", 2024-05-08T08:52:50.383 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2190000000000001e-06 2024-05-08T08:52:50.383 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.383 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.383 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:50.383 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.162830+0000", 2024-05-08T08:52:50.383 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057716637000000001 2024-05-08T08:52:50.383 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.383 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.383 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:50.383 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.162924+0000", 2024-05-08T08:52:50.383 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3354000000000001e-05 2024-05-08T08:52:50.384 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.384 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.384 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:50.384 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.163013+0000", 2024-05-08T08:52:50.384 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.9697999999999994e-05 2024-05-08T08:52:50.384 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.384 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:50.384 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.384 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.384 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.384 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9955 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 831488~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:50.384 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.105124+0000", 2024-05-08T08:52:50.384 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 23.234020232999999, 2024-05-08T08:52:50.384 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.058092618999999998, 2024-05-08T08:52:50.384 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:50.385 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:50.385 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:50.385 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:50.385 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:50.385 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:50.385 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9955 2024-05-08T08:52:50.385 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.385 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:50.385 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.385 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:50.385 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105124+0000", 2024-05-08T08:52:50.385 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.385 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.385 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.385 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:50.386 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105124+0000", 2024-05-08T08:52:50.386 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.386 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.386 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.386 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:50.386 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.106961+0000", 2024-05-08T08:52:50.386 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001837574 2024-05-08T08:52:50.386 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.386 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.386 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:50.386 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.106966+0000", 2024-05-08T08:52:50.386 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8439999999999997e-06 2024-05-08T08:52:50.386 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.386 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.387 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:50.387 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.106978+0000", 2024-05-08T08:52:50.387 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2330999999999999e-05 2024-05-08T08:52:50.387 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.387 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.387 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:50.387 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.106986+0000", 2024-05-08T08:52:50.387 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0439999999999996e-06 2024-05-08T08:52:50.387 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.387 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.387 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:50.387 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.163048+0000", 2024-05-08T08:52:50.387 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.056061182000000001 2024-05-08T08:52:50.387 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.388 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.388 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:50.388 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.163124+0000", 2024-05-08T08:52:50.388 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.6030000000000002e-05 2024-05-08T08:52:50.388 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.388 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.388 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:50.388 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.163216+0000", 2024-05-08T08:52:50.388 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2614000000000005e-05 2024-05-08T08:52:50.388 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.388 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:50.388 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.388 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.388 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.388 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9960 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 1589248~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:50.389 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.107116+0000", 2024-05-08T08:52:50.389 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 23.232027415000001, 2024-05-08T08:52:50.389 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.056889636, 2024-05-08T08:52:50.389 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:50.389 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:50.389 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:50.389 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:50.389 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:50.389 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:50.389 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9960 2024-05-08T08:52:50.389 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.389 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:50.389 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.389 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:50.390 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107116+0000", 2024-05-08T08:52:50.390 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.390 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.390 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.390 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:50.390 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107116+0000", 2024-05-08T08:52:50.390 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.390 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.390 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.390 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:50.390 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107117+0000", 2024-05-08T08:52:50.390 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4400000000000001e-07 2024-05-08T08:52:50.390 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.391 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.391 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:50.391 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107119+0000", 2024-05-08T08:52:50.391 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3750000000000001e-06 2024-05-08T08:52:50.391 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.391 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.391 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:50.391 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107126+0000", 2024-05-08T08:52:50.391 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3369999999999996e-06 2024-05-08T08:52:50.391 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.391 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.391 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:50.391 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107130+0000", 2024-05-08T08:52:50.391 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8620000000000002e-06 2024-05-08T08:52:50.391 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.391 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.392 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:50.392 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.163917+0000", 2024-05-08T08:52:50.392 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.056786634000000002 2024-05-08T08:52:50.392 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.392 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.392 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:50.392 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.163962+0000", 2024-05-08T08:52:50.392 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4521000000000003e-05 2024-05-08T08:52:50.392 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.392 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.392 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:50.392 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.164006+0000", 2024-05-08T08:52:50.392 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4462999999999998e-05 2024-05-08T08:52:50.392 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.392 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:50.393 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.393 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.393 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.393 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9961 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 864256~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:50.393 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.107138+0000", 2024-05-08T08:52:50.393 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 23.232006355999999, 2024-05-08T08:52:50.393 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057108051999999999, 2024-05-08T08:52:50.393 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:50.393 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:50.393 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:50.393 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:50.393 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:50.393 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:50.393 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9961 2024-05-08T08:52:50.394 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.394 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:50.394 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.394 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:50.394 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107138+0000", 2024-05-08T08:52:50.394 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.394 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.394 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.394 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:50.394 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107138+0000", 2024-05-08T08:52:50.394 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.394 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.394 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.394 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:50.394 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107138+0000", 2024-05-08T08:52:50.395 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2399999999999999e-07 2024-05-08T08:52:50.395 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.395 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.395 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:50.395 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107140+0000", 2024-05-08T08:52:50.395 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1409999999999999e-06 2024-05-08T08:52:50.395 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.395 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.395 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:50.395 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107146+0000", 2024-05-08T08:52:50.395 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0639999999999999e-06 2024-05-08T08:52:50.395 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.395 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.395 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:50.395 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107151+0000", 2024-05-08T08:52:50.395 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.69e-06 2024-05-08T08:52:50.396 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.396 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.396 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:50.396 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.164051+0000", 2024-05-08T08:52:50.396 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.056900153000000002 2024-05-08T08:52:50.396 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.396 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.396 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:50.396 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.164136+0000", 2024-05-08T08:52:50.396 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.5086999999999994e-05 2024-05-08T08:52:50.396 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.396 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.396 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:50.396 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.164246+0000", 2024-05-08T08:52:50.396 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000109493 2024-05-08T08:52:50.396 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.396 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:50.397 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.397 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.397 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.397 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9962 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 1605632~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:50.397 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.107158+0000", 2024-05-08T08:52:50.397 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 23.231986277000001, 2024-05-08T08:52:50.397 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057292698000000003, 2024-05-08T08:52:50.397 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:50.397 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:50.397 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:50.397 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:50.397 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:50.397 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:50.397 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9962 2024-05-08T08:52:50.398 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.398 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:50.398 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.398 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:50.398 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107158+0000", 2024-05-08T08:52:50.398 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.398 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.398 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.398 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:50.398 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107158+0000", 2024-05-08T08:52:50.398 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.398 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.398 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.398 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:50.399 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107158+0000", 2024-05-08T08:52:50.399 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8100000000000003e-07 2024-05-08T08:52:50.399 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.399 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.399 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:50.399 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107167+0000", 2024-05-08T08:52:50.399 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3079999999999997e-06 2024-05-08T08:52:50.399 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.399 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.399 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:50.399 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107175+0000", 2024-05-08T08:52:50.399 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2089999999999999e-06 2024-05-08T08:52:50.399 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.399 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.400 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:50.400 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107180+0000", 2024-05-08T08:52:50.400 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1120000000000004e-06 2024-05-08T08:52:50.400 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.400 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.400 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:50.400 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.164284+0000", 2024-05-08T08:52:50.400 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057104110999999999 2024-05-08T08:52:50.400 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.400 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.400 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:50.400 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.164370+0000", 2024-05-08T08:52:50.400 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.6549000000000001e-05 2024-05-08T08:52:50.400 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.400 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.401 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:50.401 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.164450+0000", 2024-05-08T08:52:50.401 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9827999999999995e-05 2024-05-08T08:52:50.401 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.401 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:50.401 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.401 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.401 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.401 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9963 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 880640~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:50.401 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.107188+0000", 2024-05-08T08:52:50.401 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 23.231956359000002, 2024-05-08T08:52:50.401 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057560471000000002, 2024-05-08T08:52:50.401 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:50.401 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:50.402 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:50.402 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:50.402 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:50.402 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:50.402 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9963 2024-05-08T08:52:50.402 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.402 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:50.402 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.402 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:50.402 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107188+0000", 2024-05-08T08:52:50.402 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.402 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.402 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.402 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:50.402 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107188+0000", 2024-05-08T08:52:50.403 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.403 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.403 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.403 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:50.403 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.108964+0000", 2024-05-08T08:52:50.403 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001775993 2024-05-08T08:52:50.403 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.403 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.403 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:50.403 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.108969+0000", 2024-05-08T08:52:50.403 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1200000000000001e-06 2024-05-08T08:52:50.403 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.403 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.403 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:50.404 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.108981+0000", 2024-05-08T08:52:50.404 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2670000000000001e-05 2024-05-08T08:52:50.404 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.404 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.404 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:50.404 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.108989+0000", 2024-05-08T08:52:50.404 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3139999999999998e-06 2024-05-08T08:52:50.404 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.404 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.404 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:50.404 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.164504+0000", 2024-05-08T08:52:50.404 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.055515225000000001 2024-05-08T08:52:50.404 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.404 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.405 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:50.405 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.164606+0000", 2024-05-08T08:52:50.405 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010245 2024-05-08T08:52:50.405 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.405 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.405 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:50.405 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.164748+0000", 2024-05-08T08:52:50.405 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014169900000000001 2024-05-08T08:52:50.405 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.405 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:50.405 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.405 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.405 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.405 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9970 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 929792~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:50.405 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.109178+0000", 2024-05-08T08:52:50.406 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 23.229966081000001, 2024-05-08T08:52:50.406 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057137975000000001, 2024-05-08T08:52:50.406 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:50.406 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:50.406 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:50.406 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:50.406 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:50.406 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:50.406 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9970 2024-05-08T08:52:50.406 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.406 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:50.406 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.406 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:50.407 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109178+0000", 2024-05-08T08:52:50.407 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.407 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.407 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.407 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:50.407 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109178+0000", 2024-05-08T08:52:50.407 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.407 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.407 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.407 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:50.407 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109178+0000", 2024-05-08T08:52:50.407 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7800000000000002e-07 2024-05-08T08:52:50.407 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.407 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.408 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:50.408 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109180+0000", 2024-05-08T08:52:50.408 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2199999999999999e-06 2024-05-08T08:52:50.408 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.408 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.408 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:50.408 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109186+0000", 2024-05-08T08:52:50.408 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9410000000000004e-06 2024-05-08T08:52:50.408 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.408 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.408 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:50.408 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109191+0000", 2024-05-08T08:52:50.408 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8729999999999998e-06 2024-05-08T08:52:50.408 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.409 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.409 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:50.409 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.166098+0000", 2024-05-08T08:52:50.409 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.056907124000000003 2024-05-08T08:52:50.409 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.409 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.409 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:50.409 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.166193+0000", 2024-05-08T08:52:50.409 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5018000000000004e-05 2024-05-08T08:52:50.409 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.409 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.409 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:50.409 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.166316+0000", 2024-05-08T08:52:50.409 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000122321 2024-05-08T08:52:50.409 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.410 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:50.410 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.410 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.410 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.410 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9971 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 1687552~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:50.410 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.109198+0000", 2024-05-08T08:52:50.410 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 23.229945402999999, 2024-05-08T08:52:50.410 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057361875, 2024-05-08T08:52:50.410 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:50.410 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:50.410 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:50.410 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:50.410 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:50.411 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:50.411 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9971 2024-05-08T08:52:50.411 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.411 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:50.411 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.411 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:50.411 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109198+0000", 2024-05-08T08:52:50.411 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.411 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.411 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.411 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:50.411 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109198+0000", 2024-05-08T08:52:50.411 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.411 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.411 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.411 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:50.412 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109199+0000", 2024-05-08T08:52:50.412 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2800000000000002e-07 2024-05-08T08:52:50.412 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.412 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.412 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:50.412 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109208+0000", 2024-05-08T08:52:50.412 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4210000000000001e-06 2024-05-08T08:52:50.412 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.412 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.412 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:50.412 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109215+0000", 2024-05-08T08:52:50.412 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7209999999999997e-06 2024-05-08T08:52:50.412 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.412 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.412 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:50.412 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109220+0000", 2024-05-08T08:52:50.413 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.13e-06 2024-05-08T08:52:50.413 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.413 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.413 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:50.413 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.166353+0000", 2024-05-08T08:52:50.413 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057132693999999998 2024-05-08T08:52:50.413 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.413 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.413 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:50.413 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.166462+0000", 2024-05-08T08:52:50.413 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010940799999999999 2024-05-08T08:52:50.413 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.413 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.413 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:50.413 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.166560+0000", 2024-05-08T08:52:50.413 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8072999999999994e-05 2024-05-08T08:52:50.414 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.414 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:50.414 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.414 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.414 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.414 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9972 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 1703936~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:50.414 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.109228+0000", 2024-05-08T08:52:50.414 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 23.229916011, 2024-05-08T08:52:50.414 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057541904999999997, 2024-05-08T08:52:50.414 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:50.414 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:50.414 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:50.414 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:50.414 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:50.415 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:50.415 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9972 2024-05-08T08:52:50.415 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.415 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:50.415 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.415 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:50.415 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109228+0000", 2024-05-08T08:52:50.415 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.415 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.415 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.415 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:50.415 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109228+0000", 2024-05-08T08:52:50.415 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.415 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.415 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.416 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:50.416 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109228+0000", 2024-05-08T08:52:50.416 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5699999999999998e-07 2024-05-08T08:52:50.416 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.416 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.416 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:50.416 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109231+0000", 2024-05-08T08:52:50.416 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2120000000000002e-06 2024-05-08T08:52:50.416 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.416 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.416 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:50.416 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109237+0000", 2024-05-08T08:52:50.416 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3099999999999997e-06 2024-05-08T08:52:50.416 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.417 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.417 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:50.417 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109243+0000", 2024-05-08T08:52:50.417 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7490000000000004e-06 2024-05-08T08:52:50.417 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.417 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.417 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:50.417 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.166609+0000", 2024-05-08T08:52:50.417 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057366372999999998 2024-05-08T08:52:50.417 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.417 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.417 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:50.417 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.166690+0000", 2024-05-08T08:52:50.417 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1426000000000003e-05 2024-05-08T08:52:50.417 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.418 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.418 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:50.418 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.166770+0000", 2024-05-08T08:52:50.418 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9377999999999998e-05 2024-05-08T08:52:50.418 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.418 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:50.418 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.419 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.419 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.419 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9973 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 946176~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:50.419 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.109249+0000", 2024-05-08T08:52:50.419 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 23.229894369, 2024-05-08T08:52:50.419 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057760381, 2024-05-08T08:52:50.419 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:50.419 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:50.419 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:50.419 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:50.419 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:50.419 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:50.419 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9973 2024-05-08T08:52:50.420 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.420 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:50.420 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.420 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:50.420 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109249+0000", 2024-05-08T08:52:50.420 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.420 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.420 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.420 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:50.420 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109249+0000", 2024-05-08T08:52:50.420 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.420 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.420 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.420 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:50.420 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.110959+0000", 2024-05-08T08:52:50.421 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001709288 2024-05-08T08:52:50.421 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.421 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.421 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:50.421 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.110972+0000", 2024-05-08T08:52:50.421 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2719e-05 2024-05-08T08:52:50.421 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.421 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.421 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:50.421 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.110984+0000", 2024-05-08T08:52:50.421 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2001000000000001e-05 2024-05-08T08:52:50.421 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.421 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.421 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:50.421 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.110993+0000", 2024-05-08T08:52:50.421 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0880000000000001e-06 2024-05-08T08:52:50.422 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.422 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.422 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:50.422 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.166817+0000", 2024-05-08T08:52:50.422 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.055823971999999999 2024-05-08T08:52:50.422 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.422 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.422 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:50.422 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.166900+0000", 2024-05-08T08:52:50.422 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3838999999999995e-05 2024-05-08T08:52:50.422 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.422 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.422 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:50.422 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.167010+0000", 2024-05-08T08:52:50.423 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000109474 2024-05-08T08:52:50.423 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.423 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:50.423 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.423 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.423 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:50.423 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:52:50.425 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:52:50.425 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:52:50.427+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:52:50.493 INFO:teuthology.orchestra.run.smithi196.stdout:checking clone #9, image image_client.0-clone9 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent10 2024-05-08T08:52:50.525 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:50.529+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:50.564 INFO:teuthology.orchestra.run.smithi196.stdout:2122 trunc from 0xed52000 to 0xe94de00 2024-05-08T08:52:50.586 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:52:50.586 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:52:50.586 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.586 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4848.0:176 3.18 3:189c3b51:::rbd_object_map.1140f00e9ce3:head [call lock.assert_locked in=44b,call rbd.object_map_resize in=29b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-05-08T08:52:50.586 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:50.588927+0000", 2024-05-08T08:52:50.586 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.00059386900000000002, 2024-05-08T08:52:50.586 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00061302500000000001, 2024-05-08T08:52:50.586 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:50.586 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:50.586 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "waiting for sub ops", 2024-05-08T08:52:50.587 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:50.587 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4848", 2024-05-08T08:52:50.587 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3282077120", 2024-05-08T08:52:50.587 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 176 2024-05-08T08:52:50.587 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.587 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:50.587 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.587 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:50.587 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:50.588927+0000", 2024-05-08T08:52:50.587 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.587 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.587 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.587 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:50.587 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:50.588927+0000", 2024-05-08T08:52:50.587 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.588 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.588 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.588 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:50.588 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:50.588928+0000", 2024-05-08T08:52:50.588 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.9999999999999996e-07 2024-05-08T08:52:50.588 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.588 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.588 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:50.588 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:50.588932+0000", 2024-05-08T08:52:50.588 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1500000000000001e-06 2024-05-08T08:52:50.588 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.588 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.588 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:50.588 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:50.588942+0000", 2024-05-08T08:52:50.589 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0251999999999999e-05 2024-05-08T08:52:50.589 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.589 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.589 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:50.589 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:50.588947+0000", 2024-05-08T08:52:50.589 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0579999999999997e-06 2024-05-08T08:52:50.589 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.589 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.589 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:50.589 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:50.588997+0000", 2024-05-08T08:52:50.589 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9891000000000001e-05 2024-05-08T08:52:50.589 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.589 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.589 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:50.589 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:50.589050+0000", 2024-05-08T08:52:50.590 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2960000000000001e-05 2024-05-08T08:52:50.590 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.590 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.590 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 0", 2024-05-08T08:52:50.590 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:50.589194+0000", 2024-05-08T08:52:50.590 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014387499999999999 2024-05-08T08:52:50.590 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.590 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:50.590 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.590 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.590 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T08:52:50.590 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 1 2024-05-08T08:52:50.590 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:52:50.591 INFO:teuthology.orchestra.run.smithi196.stdout:2123 write 0xc293800 thru 0xc2a356a (0xfd6b bytes) 2024-05-08T08:52:50.595 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:52:50.598 INFO:teuthology.orchestra.run.smithi196.stdout:2124 punch from 0x2ccac00 to 0x2cda000, (0xf400 bytes) 2024-05-08T08:52:50.610 INFO:teuthology.orchestra.run.smithi196.stdout:2125 trunc from 0xe94de00 to 0x4e94c00 2024-05-08T08:52:50.640 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:52:50.643+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:52:50.657 INFO:teuthology.orchestra.run.smithi196.stdout:2127 write 0xda5de00 thru 0xda67f0a (0xa10b bytes) 2024-05-08T08:52:50.664 INFO:teuthology.orchestra.run.smithi196.stdout:2131 write 0xc335600 thru 0xc339b57 (0x4558 bytes) 2024-05-08T08:52:50.664 INFO:teuthology.orchestra.run.smithi196.stdout:2132 punch from 0x88eda00 to 0x88f7200, (0x9800 bytes) 2024-05-08T08:52:50.664 INFO:teuthology.orchestra.run.smithi196.stdout:2133 punch from 0x8a6c200 to 0x8a72200, (0x6000 bytes) 2024-05-08T08:52:50.665 INFO:teuthology.orchestra.run.smithi196.stdout:2134 trunc from 0xda68000 to 0xdf6d800 2024-05-08T08:52:50.680 INFO:teuthology.orchestra.run.smithi196.stdout:2136 trunc from 0xdf6d800 to 0xd43c200 2024-05-08T08:52:50.698 INFO:teuthology.orchestra.run.smithi196.stdout:2139 punch from 0x1933600 to 0x193f800, (0xc200 bytes) 2024-05-08T08:52:50.712 INFO:teuthology.orchestra.run.smithi196.stdout:2143 trunc from 0xd43c200 to 0x2668400 2024-05-08T08:52:50.727 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:50.730+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:50.734 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:52:50.735 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:52:50.735 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:52:50.735 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:52:50.735 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:52:50.744 INFO:teuthology.orchestra.run.smithi196.stdout:2144 read 0x41a200 thru 0x429d9f (0xfba0 bytes) 2024-05-08T08:52:50.745 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:52:50.754 INFO:teuthology.orchestra.run.smithi196.stdout:2146 write 0x3256000 thru 0x32625c5 (0xc5c6 bytes) 2024-05-08T08:52:50.755 INFO:teuthology.orchestra.run.smithi196.stdout:2148 read 0x2d31400 thru 0x2d3f6b6 (0xe2b7 bytes) 2024-05-08T08:52:50.775 INFO:teuthology.orchestra.run.smithi196.stdout:2149 write 0x92ab800 thru 0x92b678a (0xaf8b bytes) 2024-05-08T08:52:50.776 INFO:teuthology.orchestra.run.smithi196.stdout:2152 punch from 0x47ecc00 to 0x47f4c00, (0x8000 bytes) 2024-05-08T08:52:50.800 INFO:teuthology.orchestra.run.smithi196.stdout:2154 write 0xcdbe800 thru 0xcdc4100 (0x5901 bytes) 2024-05-08T08:52:50.800 INFO:teuthology.orchestra.run.smithi196.stdout:2158 read 0xa1f7c00 thru 0xa202a1b (0xae1c bytes) 2024-05-08T08:52:50.800 INFO:teuthology.orchestra.run.smithi196.stdout:2159 read 0x6d71a00 thru 0x6d817c4 (0xfdc5 bytes) 2024-05-08T08:52:50.800 INFO:teuthology.orchestra.run.smithi196.stdout:2162 read 0x9551e00 thru 0x955d3a1 (0xb5a2 bytes) 2024-05-08T08:52:50.815 INFO:teuthology.orchestra.run.smithi196.stdout:2163 write 0xe2e7600 thru 0xe2f2e21 (0xb822 bytes) 2024-05-08T08:52:50.816 INFO:teuthology.orchestra.run.smithi196.stdout:2164 write 0x2a4e600 thru 0x2a5b1e3 (0xcbe4 bytes) 2024-05-08T08:52:50.825 INFO:teuthology.orchestra.run.smithi196.stdout:2165 trunc from 0xe2f3000 to 0x6750400 2024-05-08T08:52:50.827 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:50.831+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:50.862 INFO:teuthology.orchestra.run.smithi196.stdout:2167 write 0xe52b200 thru 0xe539cb7 (0xeab8 bytes) 2024-05-08T08:52:50.862 INFO:teuthology.orchestra.run.smithi196.stdout:2168 read 0xa463000 thru 0xa46b265 (0x8266 bytes) 2024-05-08T08:52:50.862 INFO:teuthology.orchestra.run.smithi196.stdout:2170 write 0x7e73400 thru 0x7e7b313 (0x7f14 bytes) 2024-05-08T08:52:50.862 INFO:teuthology.orchestra.run.smithi196.stdout:2173 write 0x8830600 thru 0x8837f06 (0x7907 bytes) 2024-05-08T08:52:50.862 INFO:teuthology.orchestra.run.smithi196.stdout:2174 write 0xd85f400 thru 0xd867c86 (0x8887 bytes) 2024-05-08T08:52:50.872 INFO:teuthology.orchestra.run.smithi196.stdout:2179 trunc from 0xe539e00 to 0x6339e00 2024-05-08T08:52:50.904 INFO:teuthology.orchestra.run.smithi196.stdout:2180 trunc from 0x6339e00 to 0xd3b1200 2024-05-08T08:52:50.914 INFO:teuthology.orchestra.run.smithi196.stdout:2181 write 0x403a000 thru 0x403cb9a (0x2b9b bytes) 2024-05-08T08:52:50.916 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:52:50.916 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:52:50.917 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:52:50.917 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:52:50.917 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.917 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7490 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 0~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:50.917 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.744226+0000", 2024-05-08T08:52:50.917 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 13.174413981000001, 2024-05-08T08:52:50.917 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057294818999999997, 2024-05-08T08:52:50.917 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:50.918 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:50.918 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:50.918 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:50.918 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:50.918 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:50.918 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7490 2024-05-08T08:52:50.918 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.918 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:50.918 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.918 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:50.918 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744226+0000", 2024-05-08T08:52:50.918 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.918 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.919 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.919 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:50.919 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744226+0000", 2024-05-08T08:52:50.919 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.919 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.919 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.919 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:50.919 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744226+0000", 2024-05-08T08:52:50.919 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3799999999999998e-07 2024-05-08T08:52:50.919 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.919 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.919 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:50.919 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744228+0000", 2024-05-08T08:52:50.919 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0710000000000002e-06 2024-05-08T08:52:50.919 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.920 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.920 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:50.920 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744233+0000", 2024-05-08T08:52:50.920 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0130000000000001e-06 2024-05-08T08:52:50.920 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.920 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.920 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:50.920 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744237+0000", 2024-05-08T08:52:50.920 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1930000000000002e-06 2024-05-08T08:52:50.920 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.920 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.920 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:50.920 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.801334+0000", 2024-05-08T08:52:50.920 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057097004999999999 2024-05-08T08:52:50.920 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.921 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.921 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:50.921 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.801414+0000", 2024-05-08T08:52:50.921 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0129000000000005e-05 2024-05-08T08:52:50.921 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.921 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.921 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:50.921 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.801521+0000", 2024-05-08T08:52:50.921 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010707 2024-05-08T08:52:50.921 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.921 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:50.921 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.921 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.921 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.921 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7491 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 40960~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:50.922 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.744242+0000", 2024-05-08T08:52:50.922 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 13.17439815, 2024-05-08T08:52:50.922 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057541677999999999, 2024-05-08T08:52:50.922 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:50.922 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:50.922 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:50.922 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:50.922 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:50.922 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:50.922 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7491 2024-05-08T08:52:50.922 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.922 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:50.922 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.923 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:50.923 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744242+0000", 2024-05-08T08:52:50.923 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.923 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.923 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.923 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:50.923 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744242+0000", 2024-05-08T08:52:50.923 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.923 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.923 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.923 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:50.923 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744242+0000", 2024-05-08T08:52:50.923 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.2000000000000001e-07 2024-05-08T08:52:50.923 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.923 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.923 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:50.924 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744244+0000", 2024-05-08T08:52:50.924 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6959999999999999e-06 2024-05-08T08:52:50.924 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.924 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.924 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:50.924 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744247+0000", 2024-05-08T08:52:50.924 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3000000000000002e-06 2024-05-08T08:52:50.924 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.924 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.924 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:50.924 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744250+0000", 2024-05-08T08:52:50.924 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.5399999999999998e-06 2024-05-08T08:52:50.924 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.924 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.925 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:50.925 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.801564+0000", 2024-05-08T08:52:50.925 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057313809 2024-05-08T08:52:50.925 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.925 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.925 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:50.925 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.801671+0000", 2024-05-08T08:52:50.925 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000107333 2024-05-08T08:52:50.925 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.925 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.925 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:50.925 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.801784+0000", 2024-05-08T08:52:50.925 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011268 2024-05-08T08:52:50.925 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.926 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:50.926 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.926 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.926 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.926 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7492 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 81920~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:50.926 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.744253+0000", 2024-05-08T08:52:50.926 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 13.174386673000001, 2024-05-08T08:52:50.926 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057761380000000001, 2024-05-08T08:52:50.926 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:50.926 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:50.926 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:50.926 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:50.926 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:50.926 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:50.927 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7492 2024-05-08T08:52:50.927 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.927 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:50.927 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.927 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:50.927 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744253+0000", 2024-05-08T08:52:50.927 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.927 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.927 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.927 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:50.927 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744253+0000", 2024-05-08T08:52:50.927 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.927 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.927 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.927 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:50.928 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.745991+0000", 2024-05-08T08:52:50.928 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001738021 2024-05-08T08:52:50.928 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.928 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.928 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:50.928 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.745997+0000", 2024-05-08T08:52:50.928 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5300000000000004e-06 2024-05-08T08:52:50.928 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.928 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.928 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:50.928 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.746006+0000", 2024-05-08T08:52:50.928 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2210000000000003e-06 2024-05-08T08:52:50.928 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.928 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.928 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:50.929 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.746026+0000", 2024-05-08T08:52:50.929 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9487000000000002e-05 2024-05-08T08:52:50.929 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.929 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.929 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:50.929 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.801815+0000", 2024-05-08T08:52:50.929 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.055789196999999999 2024-05-08T08:52:50.929 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.929 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.929 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:50.929 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.801908+0000", 2024-05-08T08:52:50.929 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3320999999999996e-05 2024-05-08T08:52:50.929 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.929 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.929 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:50.930 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.802015+0000", 2024-05-08T08:52:50.930 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000106603 2024-05-08T08:52:50.930 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.930 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:50.930 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.930 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.930 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.930 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7499 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 368640~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:50.930 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.746209+0000", 2024-05-08T08:52:50.930 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 13.172430725, 2024-05-08T08:52:50.930 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057256452999999999, 2024-05-08T08:52:50.930 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:50.930 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:50.931 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:50.931 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:50.931 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:50.931 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:50.931 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7499 2024-05-08T08:52:50.931 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.931 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:50.931 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.931 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:50.931 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.746209+0000", 2024-05-08T08:52:50.931 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.931 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.931 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.931 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:50.932 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.746209+0000", 2024-05-08T08:52:50.932 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.932 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.932 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.932 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:50.932 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.746210+0000", 2024-05-08T08:52:50.932 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2100000000000002e-07 2024-05-08T08:52:50.932 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.932 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.932 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:50.932 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.746212+0000", 2024-05-08T08:52:50.932 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.922e-06 2024-05-08T08:52:50.932 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.932 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.932 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:50.933 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.746221+0000", 2024-05-08T08:52:50.933 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2900000000000008e-06 2024-05-08T08:52:50.933 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.933 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.933 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:50.933 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.746227+0000", 2024-05-08T08:52:50.933 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3740000000000003e-06 2024-05-08T08:52:50.933 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.933 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.933 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:50.933 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.803304+0000", 2024-05-08T08:52:50.933 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057076239000000001 2024-05-08T08:52:50.933 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.933 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.933 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:50.934 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.803380+0000", 2024-05-08T08:52:50.934 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.6309000000000004e-05 2024-05-08T08:52:50.934 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.934 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.934 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:50.934 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.803466+0000", 2024-05-08T08:52:50.934 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.5897999999999996e-05 2024-05-08T08:52:50.934 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.934 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:50.934 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.934 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.934 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.934 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7500 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 409600~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:50.934 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.746237+0000", 2024-05-08T08:52:50.935 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 13.172403143, 2024-05-08T08:52:50.935 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057429380000000002, 2024-05-08T08:52:50.935 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:50.935 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:50.935 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:50.935 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:50.935 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:50.935 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:50.935 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7500 2024-05-08T08:52:50.935 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.935 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:50.935 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.935 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:50.935 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.746237+0000", 2024-05-08T08:52:50.936 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.936 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.936 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.936 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:50.936 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.746237+0000", 2024-05-08T08:52:50.936 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.936 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.936 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.936 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:50.936 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.747959+0000", 2024-05-08T08:52:50.936 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001721888 2024-05-08T08:52:50.936 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.936 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.936 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:50.936 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.747964+0000", 2024-05-08T08:52:50.937 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0030000000000002e-06 2024-05-08T08:52:50.937 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.937 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.937 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:50.937 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.747976+0000", 2024-05-08T08:52:50.937 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2656999999999999e-05 2024-05-08T08:52:50.937 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.937 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.937 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:50.937 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.747983+0000", 2024-05-08T08:52:50.937 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.866e-06 2024-05-08T08:52:50.937 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.937 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.937 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:50.937 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.803495+0000", 2024-05-08T08:52:50.938 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.055511392999999999 2024-05-08T08:52:50.938 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.938 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.938 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:50.938 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.803574+0000", 2024-05-08T08:52:50.938 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9215000000000006e-05 2024-05-08T08:52:50.938 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.938 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.938 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:50.938 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.803666+0000", 2024-05-08T08:52:50.938 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2357999999999999e-05 2024-05-08T08:52:50.938 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.938 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:50.938 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.939 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.939 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.939 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7508 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 737280~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:50.939 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.748180+0000", 2024-05-08T08:52:50.939 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 13.170460086, 2024-05-08T08:52:50.939 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057419809000000002, 2024-05-08T08:52:50.939 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:50.939 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:50.939 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:50.939 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:50.939 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:50.939 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:50.939 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7508 2024-05-08T08:52:50.939 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.939 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:50.940 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.940 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:50.940 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748180+0000", 2024-05-08T08:52:50.940 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.940 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.940 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.940 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:50.940 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748180+0000", 2024-05-08T08:52:50.940 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.940 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.940 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.940 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:50.940 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748180+0000", 2024-05-08T08:52:50.940 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4499999999999997e-07 2024-05-08T08:52:50.940 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.941 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.941 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:50.941 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748183+0000", 2024-05-08T08:52:50.941 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1660000000000001e-06 2024-05-08T08:52:50.941 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.941 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.941 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:50.941 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748197+0000", 2024-05-08T08:52:50.941 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.4877e-05 2024-05-08T08:52:50.941 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.941 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.941 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:50.941 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748203+0000", 2024-05-08T08:52:50.941 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7470000000000002e-06 2024-05-08T08:52:50.941 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.942 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.942 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:50.942 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.805398+0000", 2024-05-08T08:52:50.942 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057194513000000002 2024-05-08T08:52:50.942 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.942 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.942 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:50.942 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.805544+0000", 2024-05-08T08:52:50.942 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000146522 2024-05-08T08:52:50.942 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.942 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.942 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:50.942 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.805600+0000", 2024-05-08T08:52:50.942 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5538999999999998e-05 2024-05-08T08:52:50.943 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.943 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:50.943 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.943 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.943 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.943 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7509 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 778240~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:50.943 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.748211+0000", 2024-05-08T08:52:50.943 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 13.170429017, 2024-05-08T08:52:50.943 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057619034999999999, 2024-05-08T08:52:50.943 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:50.943 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:50.943 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:50.943 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:50.943 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:50.943 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:50.944 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7509 2024-05-08T08:52:50.944 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.944 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:50.944 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.944 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:50.944 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748211+0000", 2024-05-08T08:52:50.944 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.944 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.944 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.944 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:50.944 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748211+0000", 2024-05-08T08:52:50.944 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.944 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.945 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.945 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:50.945 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748212+0000", 2024-05-08T08:52:50.945 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8100000000000003e-07 2024-05-08T08:52:50.945 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.945 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.945 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:50.945 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748214+0000", 2024-05-08T08:52:50.945 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4669999999999998e-06 2024-05-08T08:52:50.945 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.945 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.945 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:50.945 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748221+0000", 2024-05-08T08:52:50.945 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5769999999999999e-06 2024-05-08T08:52:50.945 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.946 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.946 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:50.946 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748225+0000", 2024-05-08T08:52:50.946 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8060000000000002e-06 2024-05-08T08:52:50.946 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.946 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.946 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:50.946 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.805636+0000", 2024-05-08T08:52:50.946 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057410309 2024-05-08T08:52:50.946 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.946 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.946 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:50.946 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.805723+0000", 2024-05-08T08:52:50.946 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7435999999999994e-05 2024-05-08T08:52:50.947 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.947 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.947 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:50.947 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.805830+0000", 2024-05-08T08:52:50.947 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000106959 2024-05-08T08:52:50.947 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.947 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:50.947 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.947 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.947 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.947 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7510 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 819200~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:50.947 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.748232+0000", 2024-05-08T08:52:50.947 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 13.170407987000001, 2024-05-08T08:52:50.947 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057710421999999997, 2024-05-08T08:52:50.948 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:50.948 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:50.948 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:50.948 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:50.948 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:50.948 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:50.948 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7510 2024-05-08T08:52:50.948 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.948 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:50.948 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.948 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:50.948 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748232+0000", 2024-05-08T08:52:50.948 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.948 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.948 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.949 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:50.949 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748232+0000", 2024-05-08T08:52:50.949 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.949 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.949 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.949 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:50.949 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.749959+0000", 2024-05-08T08:52:50.949 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001727 2024-05-08T08:52:50.949 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.949 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.949 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:50.949 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.749964+0000", 2024-05-08T08:52:50.949 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4700000000000004e-06 2024-05-08T08:52:50.949 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.950 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.950 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:50.950 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.749984+0000", 2024-05-08T08:52:50.950 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0068000000000002e-05 2024-05-08T08:52:50.950 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.950 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.950 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:50.950 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.749993+0000", 2024-05-08T08:52:50.950 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4720000000000001e-06 2024-05-08T08:52:50.950 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.950 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.950 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:50.950 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.805846+0000", 2024-05-08T08:52:50.950 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.055852770000000003 2024-05-08T08:52:50.950 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.951 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.951 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:50.951 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.805890+0000", 2024-05-08T08:52:50.951 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4323999999999998e-05 2024-05-08T08:52:50.951 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.951 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.951 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:50.951 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.805942+0000", 2024-05-08T08:52:50.951 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2318000000000003e-05 2024-05-08T08:52:50.951 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.951 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:50.951 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.951 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.951 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.951 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7515 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 1024000~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:50.952 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.750135+0000", 2024-05-08T08:52:50.952 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 13.168504610999999, 2024-05-08T08:52:50.952 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057188125999999999, 2024-05-08T08:52:50.952 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:50.952 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:50.952 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:50.952 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:50.952 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:50.952 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:50.952 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7515 2024-05-08T08:52:50.952 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.952 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:50.952 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.952 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:50.953 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750135+0000", 2024-05-08T08:52:50.953 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.953 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.953 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.953 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:50.953 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750135+0000", 2024-05-08T08:52:50.953 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.953 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.953 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.953 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:50.953 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750136+0000", 2024-05-08T08:52:50.953 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8100000000000003e-07 2024-05-08T08:52:50.953 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.953 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.953 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:50.954 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750138+0000", 2024-05-08T08:52:50.954 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.339e-06 2024-05-08T08:52:50.954 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.954 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.954 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:50.954 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750147+0000", 2024-05-08T08:52:50.954 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7669999999999992e-06 2024-05-08T08:52:50.954 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.954 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.954 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:50.954 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750153+0000", 2024-05-08T08:52:50.954 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5029999999999997e-06 2024-05-08T08:52:50.954 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.954 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.954 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:50.955 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.807145+0000", 2024-05-08T08:52:50.955 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.056992174 2024-05-08T08:52:50.955 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.955 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.955 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:50.955 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.807231+0000", 2024-05-08T08:52:50.955 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.6347000000000005e-05 2024-05-08T08:52:50.955 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.955 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.955 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:50.955 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.807324+0000", 2024-05-08T08:52:50.955 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2515000000000004e-05 2024-05-08T08:52:50.955 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.955 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:50.956 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.956 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.956 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.956 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7516 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 1064960~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:50.956 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.750162+0000", 2024-05-08T08:52:50.956 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 13.168477782, 2024-05-08T08:52:50.956 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057363971999999999, 2024-05-08T08:52:50.956 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:50.956 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:50.956 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:50.956 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:50.956 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:50.956 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:50.956 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7516 2024-05-08T08:52:50.956 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.957 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:50.957 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.957 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:50.957 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750162+0000", 2024-05-08T08:52:50.957 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.957 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.957 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.957 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:50.957 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750162+0000", 2024-05-08T08:52:50.957 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.957 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.957 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.957 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:50.957 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750163+0000", 2024-05-08T08:52:50.958 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.13e-07 2024-05-08T08:52:50.958 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.958 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.958 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:50.958 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750165+0000", 2024-05-08T08:52:50.958 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2309999999999999e-06 2024-05-08T08:52:50.958 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.958 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.958 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:50.958 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750174+0000", 2024-05-08T08:52:50.958 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4740000000000005e-06 2024-05-08T08:52:50.958 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.958 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.958 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:50.959 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750180+0000", 2024-05-08T08:52:50.959 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4760000000000003e-06 2024-05-08T08:52:50.959 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.959 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.959 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:50.959 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.807355+0000", 2024-05-08T08:52:50.959 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057175029000000002 2024-05-08T08:52:50.959 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.959 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.959 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:50.959 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.807437+0000", 2024-05-08T08:52:50.959 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.2268999999999994e-05 2024-05-08T08:52:50.959 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.959 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.960 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:50.960 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.807526+0000", 2024-05-08T08:52:50.960 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.8880000000000003e-05 2024-05-08T08:52:50.960 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.960 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:50.960 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.960 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.960 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.960 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7517 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 1105920~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:50.960 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.750188+0000", 2024-05-08T08:52:50.960 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 13.168452288999999, 2024-05-08T08:52:50.960 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057547099999999997, 2024-05-08T08:52:50.960 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:50.960 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:50.960 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:50.961 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:50.961 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:50.961 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:50.961 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7517 2024-05-08T08:52:50.961 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.961 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:50.961 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.961 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:50.961 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750188+0000", 2024-05-08T08:52:50.961 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.961 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.961 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.961 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:50.961 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750188+0000", 2024-05-08T08:52:50.962 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.962 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.962 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.962 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:50.962 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750188+0000", 2024-05-08T08:52:50.962 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7199999999999999e-07 2024-05-08T08:52:50.962 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.962 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.962 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:50.962 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750190+0000", 2024-05-08T08:52:50.962 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.154e-06 2024-05-08T08:52:50.962 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.962 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.962 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:50.963 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750198+0000", 2024-05-08T08:52:50.963 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8320000000000006e-06 2024-05-08T08:52:50.963 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.963 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.963 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:50.963 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750203+0000", 2024-05-08T08:52:50.963 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1900000000000003e-06 2024-05-08T08:52:50.963 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.963 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.963 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:50.963 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.807555+0000", 2024-05-08T08:52:50.963 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057351148999999997 2024-05-08T08:52:50.963 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.963 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.963 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:50.964 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.807640+0000", 2024-05-08T08:52:50.964 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.5853999999999995e-05 2024-05-08T08:52:50.964 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.964 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.964 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:50.964 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.807735+0000", 2024-05-08T08:52:50.964 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4449000000000003e-05 2024-05-08T08:52:50.964 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.964 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:50.964 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.964 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.964 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.964 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7518 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 1146880~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:50.964 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.750212+0000", 2024-05-08T08:52:50.964 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 13.168428489, 2024-05-08T08:52:50.965 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057664963999999999, 2024-05-08T08:52:50.965 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:50.965 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:50.965 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:50.965 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:50.965 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:50.965 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:50.965 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7518 2024-05-08T08:52:50.965 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.965 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:50.965 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.965 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:50.965 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750212+0000", 2024-05-08T08:52:50.965 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.965 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.966 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.966 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:50.966 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750212+0000", 2024-05-08T08:52:50.966 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.966 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.966 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.966 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:50.966 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750212+0000", 2024-05-08T08:52:50.966 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.46e-07 2024-05-08T08:52:50.966 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.966 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.966 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:50.966 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750220+0000", 2024-05-08T08:52:50.966 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5610000000000001e-06 2024-05-08T08:52:50.967 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.967 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.967 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:50.967 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750228+0000", 2024-05-08T08:52:50.967 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3180000000000007e-06 2024-05-08T08:52:50.967 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.967 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.967 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:50.967 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750234+0000", 2024-05-08T08:52:50.967 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0209999999999996e-06 2024-05-08T08:52:50.967 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.967 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.967 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:50.967 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.807765+0000", 2024-05-08T08:52:50.967 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057531481000000002 2024-05-08T08:52:50.967 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.968 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.968 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:50.968 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.807815+0000", 2024-05-08T08:52:50.968 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9282e-05 2024-05-08T08:52:50.968 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.968 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.968 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:50.968 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.807877+0000", 2024-05-08T08:52:50.968 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1854999999999996e-05 2024-05-08T08:52:50.968 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.968 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:50.968 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.968 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.968 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.969 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7519 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 1187840~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:50.969 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.750261+0000", 2024-05-08T08:52:50.969 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 13.168379003, 2024-05-08T08:52:50.969 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057872519999999997, 2024-05-08T08:52:50.969 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:50.969 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:50.969 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:50.969 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:50.969 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:50.969 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:50.969 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7519 2024-05-08T08:52:50.969 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.969 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:50.969 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.969 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:50.970 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750261+0000", 2024-05-08T08:52:50.970 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.970 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.970 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.970 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:50.970 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750261+0000", 2024-05-08T08:52:50.970 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.970 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.970 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.970 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:50.970 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750262+0000", 2024-05-08T08:52:50.970 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7000000000000005e-07 2024-05-08T08:52:50.970 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.970 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.971 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:50.971 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750265+0000", 2024-05-08T08:52:50.971 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4589999999999998e-06 2024-05-08T08:52:50.971 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.971 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.971 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:50.971 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750274+0000", 2024-05-08T08:52:50.971 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.5459999999999992e-06 2024-05-08T08:52:50.971 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.971 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.971 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:50.971 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750280+0000", 2024-05-08T08:52:50.971 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0950000000000001e-06 2024-05-08T08:52:50.971 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.972 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.972 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:50.972 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.807917+0000", 2024-05-08T08:52:50.972 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057637715999999999 2024-05-08T08:52:50.972 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.972 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.972 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:50.972 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.808013+0000", 2024-05-08T08:52:50.972 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5569999999999995e-05 2024-05-08T08:52:50.972 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.972 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.972 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:50.972 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.808134+0000", 2024-05-08T08:52:50.972 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012056400000000001 2024-05-08T08:52:50.972 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.973 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:50.973 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.973 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.973 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.973 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7520 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 1228800~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:50.973 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.750289+0000", 2024-05-08T08:52:50.973 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 13.168350559, 2024-05-08T08:52:50.973 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.058133788999999998, 2024-05-08T08:52:50.973 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:50.973 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:50.973 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:50.973 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:50.973 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:50.973 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:50.973 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7520 2024-05-08T08:52:50.974 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.974 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:50.974 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.974 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:50.974 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750289+0000", 2024-05-08T08:52:50.974 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.974 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.974 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.974 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:50.974 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750289+0000", 2024-05-08T08:52:50.974 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.974 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.974 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.974 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:50.975 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.751976+0000", 2024-05-08T08:52:50.975 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0016865389999999999 2024-05-08T08:52:50.975 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.975 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.975 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:50.975 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.751981+0000", 2024-05-08T08:52:50.975 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5719999999999996e-06 2024-05-08T08:52:50.975 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.975 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.975 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:50.975 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.751991+0000", 2024-05-08T08:52:50.975 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0788000000000001e-05 2024-05-08T08:52:50.975 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.975 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.975 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:50.976 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.751999+0000", 2024-05-08T08:52:50.976 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2690000000000002e-06 2024-05-08T08:52:50.976 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.976 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.976 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:50.976 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.808183+0000", 2024-05-08T08:52:50.976 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.056184686999999997 2024-05-08T08:52:50.976 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.976 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.976 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:50.976 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.808296+0000", 2024-05-08T08:52:50.976 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000112754 2024-05-08T08:52:50.976 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.976 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.977 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:50.977 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.808423+0000", 2024-05-08T08:52:50.977 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012718000000000001 2024-05-08T08:52:50.977 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.977 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:50.977 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.977 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.977 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.977 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7524 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 1392640~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:50.977 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.752121+0000", 2024-05-08T08:52:50.977 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 13.166519510000001, 2024-05-08T08:52:50.977 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057199521000000003, 2024-05-08T08:52:50.977 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:50.977 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:50.977 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:50.978 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:50.978 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:50.978 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:50.978 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7524 2024-05-08T08:52:50.978 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.978 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:50.978 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.978 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:50.978 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752121+0000", 2024-05-08T08:52:50.978 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.978 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.978 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.978 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:50.978 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752121+0000", 2024-05-08T08:52:50.979 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.979 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.979 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.979 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:50.979 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752121+0000", 2024-05-08T08:52:50.979 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5999999999999999e-07 2024-05-08T08:52:50.979 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.979 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.979 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:50.979 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752123+0000", 2024-05-08T08:52:50.979 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3499999999999999e-06 2024-05-08T08:52:50.979 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.979 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.979 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:50.979 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752132+0000", 2024-05-08T08:52:50.980 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.5539999999999998e-06 2024-05-08T08:52:50.980 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.980 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.980 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:50.980 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752138+0000", 2024-05-08T08:52:50.980 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3640000000000004e-06 2024-05-08T08:52:50.980 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.980 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.980 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:50.980 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.809210+0000", 2024-05-08T08:52:50.980 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057071469999999999 2024-05-08T08:52:50.980 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.980 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.980 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:50.980 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.809258+0000", 2024-05-08T08:52:50.980 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8755000000000003e-05 2024-05-08T08:52:50.981 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.981 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.981 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:50.981 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.809320+0000", 2024-05-08T08:52:50.981 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1568000000000004e-05 2024-05-08T08:52:50.981 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.981 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:50.981 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.981 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.981 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.981 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7525 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 1433600~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:50.981 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.752146+0000", 2024-05-08T08:52:50.981 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 13.166494466, 2024-05-08T08:52:50.981 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057400072000000003, 2024-05-08T08:52:50.982 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:50.982 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:50.982 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:50.982 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:50.982 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:50.982 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:50.982 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7525 2024-05-08T08:52:50.982 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.982 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:50.982 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.982 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:50.982 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752146+0000", 2024-05-08T08:52:50.982 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.982 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.982 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.983 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:50.983 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752146+0000", 2024-05-08T08:52:50.983 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.983 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.983 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.983 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:50.983 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752146+0000", 2024-05-08T08:52:50.983 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0400000000000002e-07 2024-05-08T08:52:50.983 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.983 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.983 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:50.983 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752148+0000", 2024-05-08T08:52:50.983 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3209999999999999e-06 2024-05-08T08:52:50.983 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.984 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.984 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:50.984 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752155+0000", 2024-05-08T08:52:50.984 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6109999999999999e-06 2024-05-08T08:52:50.984 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.984 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.984 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:50.984 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752161+0000", 2024-05-08T08:52:50.984 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.489e-06 2024-05-08T08:52:50.984 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.984 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.984 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:50.984 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.809360+0000", 2024-05-08T08:52:50.984 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057198860999999997 2024-05-08T08:52:50.984 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.985 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.985 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:50.985 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.809450+0000", 2024-05-08T08:52:50.985 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0136000000000006e-05 2024-05-08T08:52:50.985 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.985 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.985 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:50.985 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.809546+0000", 2024-05-08T08:52:50.985 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5249999999999998e-05 2024-05-08T08:52:50.985 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.985 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:50.985 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.985 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.985 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.986 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7526 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 1474560~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:50.986 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.752168+0000", 2024-05-08T08:52:50.986 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 13.166471753, 2024-05-08T08:52:50.986 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057516818999999997, 2024-05-08T08:52:50.986 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:50.986 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:50.986 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:50.986 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:50.986 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:50.986 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:50.986 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7526 2024-05-08T08:52:50.986 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.986 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:50.986 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.987 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:50.987 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752168+0000", 2024-05-08T08:52:50.987 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.987 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.987 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.987 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:50.987 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752168+0000", 2024-05-08T08:52:50.987 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.987 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.987 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.987 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:50.987 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752169+0000", 2024-05-08T08:52:50.987 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4799999999999999e-07 2024-05-08T08:52:50.987 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.987 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.988 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:50.988 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752171+0000", 2024-05-08T08:52:50.988 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.289e-06 2024-05-08T08:52:50.988 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.988 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.988 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:50.988 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752177+0000", 2024-05-08T08:52:50.988 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3690000000000003e-06 2024-05-08T08:52:50.988 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.988 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.988 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:50.988 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752183+0000", 2024-05-08T08:52:50.988 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.152e-06 2024-05-08T08:52:50.988 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.988 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.989 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:50.989 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.809574+0000", 2024-05-08T08:52:50.989 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057391075 2024-05-08T08:52:50.989 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.989 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.989 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:50.989 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.809624+0000", 2024-05-08T08:52:50.989 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0673000000000001e-05 2024-05-08T08:52:50.989 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.989 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.989 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:50.989 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.809685+0000", 2024-05-08T08:52:50.989 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0813000000000002e-05 2024-05-08T08:52:50.989 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.989 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:50.990 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.990 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.990 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.990 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7527 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 1515520~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:50.990 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.752189+0000", 2024-05-08T08:52:50.990 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 13.166450825, 2024-05-08T08:52:50.990 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057722168999999997, 2024-05-08T08:52:50.990 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:50.990 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:50.990 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:50.990 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:50.990 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:50.990 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:50.990 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7527 2024-05-08T08:52:50.991 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.991 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:50.991 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.991 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:50.991 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752189+0000", 2024-05-08T08:52:50.991 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.991 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.991 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.991 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:50.991 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752189+0000", 2024-05-08T08:52:50.991 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.991 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.991 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.991 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:50.992 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752190+0000", 2024-05-08T08:52:50.992 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8799999999999998e-07 2024-05-08T08:52:50.992 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.992 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.992 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:50.992 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752192+0000", 2024-05-08T08:52:50.992 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9759999999999998e-06 2024-05-08T08:52:50.992 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.992 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.992 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:50.992 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752199+0000", 2024-05-08T08:52:50.992 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3289999999999996e-06 2024-05-08T08:52:50.992 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.992 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.992 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:50.993 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752205+0000", 2024-05-08T08:52:50.993 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2759999999999997e-06 2024-05-08T08:52:50.993 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.993 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.993 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:50.993 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.809726+0000", 2024-05-08T08:52:50.993 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057521249000000003 2024-05-08T08:52:50.993 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.993 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.993 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:50.993 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.809815+0000", 2024-05-08T08:52:50.993 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.8987999999999996e-05 2024-05-08T08:52:50.993 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.993 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.993 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:50.994 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.809911+0000", 2024-05-08T08:52:50.994 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5963000000000003e-05 2024-05-08T08:52:50.994 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.994 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:50.994 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.994 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.994 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.994 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7528 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 1556480~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:50.994 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.752212+0000", 2024-05-08T08:52:50.994 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 13.166428115, 2024-05-08T08:52:50.994 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057904956, 2024-05-08T08:52:50.994 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:50.994 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:50.994 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:50.994 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:50.995 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:50.995 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:50.995 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7528 2024-05-08T08:52:50.995 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.995 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:50.995 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.995 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:50.995 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752212+0000", 2024-05-08T08:52:50.995 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.995 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.995 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.995 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:50.995 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752212+0000", 2024-05-08T08:52:50.995 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.996 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.996 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.996 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:50.996 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752212+0000", 2024-05-08T08:52:50.996 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7800000000000002e-07 2024-05-08T08:52:50.996 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.996 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.996 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:50.996 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752215+0000", 2024-05-08T08:52:50.996 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2120000000000002e-06 2024-05-08T08:52:50.996 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.996 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.996 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:50.996 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752222+0000", 2024-05-08T08:52:50.996 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.6329999999999996e-06 2024-05-08T08:52:50.996 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.997 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.997 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:50.997 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752227+0000", 2024-05-08T08:52:50.997 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9289999999999997e-06 2024-05-08T08:52:50.997 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.997 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.997 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:50.997 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.809966+0000", 2024-05-08T08:52:50.997 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057738554999999997 2024-05-08T08:52:50.997 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.997 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.997 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:50.997 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.810052+0000", 2024-05-08T08:52:50.997 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.6540999999999997e-05 2024-05-08T08:52:50.998 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.998 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.998 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:50.998 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.810117+0000", 2024-05-08T08:52:50.998 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4608000000000002e-05 2024-05-08T08:52:50.998 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.998 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:50.998 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:50.998 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.998 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.998 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7529 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 1597440~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:50.998 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.752234+0000", 2024-05-08T08:52:50.998 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 13.166406102, 2024-05-08T08:52:50.999 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.058133851, 2024-05-08T08:52:50.999 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:50.999 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:50.999 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:50.999 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:50.999 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:50.999 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:50.999 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7529 2024-05-08T08:52:50.999 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:50.999 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:50.999 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:50.999 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:50.999 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752234+0000", 2024-05-08T08:52:50.999 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:50.999 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:51.000 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:51.000 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:51.000 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752234+0000", 2024-05-08T08:52:51.000 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:51.000 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:51.000 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:51.000 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:51.000 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.753959+0000", 2024-05-08T08:52:51.000 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0017251409999999999 2024-05-08T08:52:51.000 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:51.000 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:51.000 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:51.000 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.753963+0000", 2024-05-08T08:52:51.000 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2860000000000001e-06 2024-05-08T08:52:51.000 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:51.000 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:51.001 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:51.001 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.753973+0000", 2024-05-08T08:52:51.001 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5119999999999997e-06 2024-05-08T08:52:51.001 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:51.001 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:51.001 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:51.001 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.753979+0000", 2024-05-08T08:52:51.001 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3589999999999996e-06 2024-05-08T08:52:51.001 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:51.001 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:51.001 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:51.001 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.810182+0000", 2024-05-08T08:52:51.001 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.056202941999999999 2024-05-08T08:52:51.001 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:51.001 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:51.002 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:51.002 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.810270+0000", 2024-05-08T08:52:51.002 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7338999999999998e-05 2024-05-08T08:52:51.002 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:51.002 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:51.002 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:51.002 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.810368+0000", 2024-05-08T08:52:51.002 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8271999999999997e-05 2024-05-08T08:52:51.002 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:51.002 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:51.002 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:51.002 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:51.002 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:51.002 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:52:51.004 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:51.007+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:51.005 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:52:51.006 INFO:teuthology.orchestra.run.smithi196.stdout:2183 trunc from 0xd3b1200 to 0x1aa1400 2024-05-08T08:52:51.006 INFO:teuthology.orchestra.run.smithi196.stdout:2185 read 0xcbca00 thru 0xcc8c1f (0xc220 bytes) 2024-05-08T08:52:51.006 INFO:teuthology.orchestra.run.smithi196.stdout:2187 write 0x211e800 thru 0x21260a7 (0x78a8 bytes) 2024-05-08T08:52:51.006 INFO:teuthology.orchestra.run.smithi196.stdout:2191 read 0x1d75e00 thru 0x1d78818 (0x2a19 bytes) 2024-05-08T08:52:51.006 INFO:teuthology.orchestra.run.smithi196.stdout:2193 write 0x1dd8000 thru 0x1de2946 (0xa947 bytes) 2024-05-08T08:52:51.006 INFO:teuthology.orchestra.run.smithi196.stdout:2194 write 0xdf84c00 thru 0xdf8f30a (0xa70b bytes) 2024-05-08T08:52:51.006 INFO:teuthology.orchestra.run.smithi196.stdout:2198 read 0x2a69200 thru 0x2a765e0 (0xd3e1 bytes) 2024-05-08T08:52:51.007 INFO:teuthology.orchestra.run.smithi196.stdout:2201 read 0x5e80000 thru 0x5e83150 (0x3151 bytes) 2024-05-08T08:52:51.007 INFO:teuthology.orchestra.run.smithi196.stdout:2203 write 0x2067200 thru 0x206b04e (0x3e4f bytes) 2024-05-08T08:52:51.007 INFO:teuthology.orchestra.run.smithi196.stdout:2204 write 0xbc98400 thru 0xbc996a5 (0x12a6 bytes) 2024-05-08T08:52:51.007 INFO:teuthology.orchestra.run.smithi196.stdout:2205 read 0x5119400 thru 0x5119e65 (0xa66 bytes) 2024-05-08T08:52:51.019 INFO:teuthology.orchestra.run.smithi196.stdout:2207 trunc from 0xdf8f400 to 0x8a5a00 2024-05-08T08:52:51.064 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 5] out_osds: [4] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-05-08T08:52:51.064 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T08:52:51.064 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-05-08T08:52:51.064 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-05-08T08:52:51.066 INFO:teuthology.orchestra.run.smithi196.stdout:2209 read 0x1e2e00 thru 0x1eda3c (0xac3d bytes) 2024-05-08T08:52:51.068 INFO:teuthology.orchestra.run.smithi196.stdout:2210 read 0x5e1c00 thru 0x5ea5d3 (0x89d4 bytes) 2024-05-08T08:52:51.068 INFO:teuthology.orchestra.run.smithi196.stdout:2212 read 0x792600 thru 0x7978c9 (0x52ca bytes) 2024-05-08T08:52:51.081 INFO:teuthology.orchestra.run.smithi196.stdout:2214 write 0x1ec8800 thru 0x1ed0934 (0x8135 bytes) 2024-05-08T08:52:51.103 INFO:teuthology.orchestra.run.smithi196.stdout:2215 write 0x94b4e00 thru 0x94c21ad (0xd3ae bytes) 2024-05-08T08:52:51.104 INFO:teuthology.orchestra.run.smithi196.stdout:2216 read 0x7ddf800 thru 0x7ded878 (0xe079 bytes) 2024-05-08T08:52:51.104 INFO:teuthology.orchestra.run.smithi196.stdout:2217 read 0x6769200 thru 0x6776846 (0xd647 bytes) 2024-05-08T08:52:51.104 INFO:teuthology.orchestra.run.smithi196.stdout:2222 punch from 0x941bc00 to 0x942a400, (0xe800 bytes) 2024-05-08T08:52:51.105 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:52:51.108+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:52:51.109 INFO:teuthology.orchestra.run.smithi196.stdout:2223 trunc from 0x94c2200 to 0xcc66400 2024-05-08T08:52:51.121 INFO:teuthology.orchestra.run.smithi196.stdout:2224 punch from 0x478c200 to 0x478c600, (0x400 bytes) 2024-05-08T08:52:51.122 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:52:51.122 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:52:51.122 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:52:51.122 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:52:51.127 INFO:teuthology.orchestra.run.smithi196.stdout:2226 read 0xafa5400 thru 0xafaa390 (0x4f91 bytes) 2024-05-08T08:52:51.127 INFO:teuthology.orchestra.run.smithi196.stdout:2228 trunc from 0xcc66400 to 0xd3b6a00 2024-05-08T08:52:51.131 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:52:51.144 INFO:teuthology.orchestra.run.smithi196.stdout:2229 trunc from 0xd3b6a00 to 0x9c0a400 2024-05-08T08:52:51.155 INFO:teuthology.orchestra.run.smithi196.stdout:2230 punch from 0x2bd1200 to 0x2bd7e00, (0x6c00 bytes) 2024-05-08T08:52:51.162 INFO:teuthology.orchestra.run.smithi196.stdout:2232 read 0x6b48800 thru 0x6b4d4ef (0x4cf0 bytes) 2024-05-08T08:52:51.163 INFO:teuthology.orchestra.run.smithi196.stdout:2233 write 0x7365400 thru 0x7367055 (0x1c56 bytes) 2024-05-08T08:52:51.168 INFO:teuthology.orchestra.run.smithi196.stdout:2234 trunc from 0x9c0a400 to 0xe81ba00 2024-05-08T08:52:51.180 INFO:teuthology.orchestra.run.smithi196.stdout:2236 clone 12 order 19 su 32768 sc 15 2024-05-08T08:52:51.215 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:51.218+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:51.276 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:52:51.277 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:52:51.277 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:52:51.277 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:52:51.277 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:52:51.285 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:52:51.326 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:52:51.329+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:52:51.356 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T08:52:51.384 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:52:51.384 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:52:51.384 INFO:teuthology.orchestra.run.smithi059.stdout: "epoch": 45, 2024-05-08T08:52:51.384 INFO:teuthology.orchestra.run.smithi059.stdout: "fsid": "38dfae8c-5c5e-4b0f-a264-91f22c6c5554", 2024-05-08T08:52:51.384 INFO:teuthology.orchestra.run.smithi059.stdout: "created": "2024-05-08T08:51:45.571369+0000", 2024-05-08T08:52:51.384 INFO:teuthology.orchestra.run.smithi059.stdout: "modified": "2024-05-08T08:52:50.060366+0000", 2024-05-08T08:52:51.384 INFO:teuthology.orchestra.run.smithi059.stdout: "last_up_change": "2024-05-08T08:51:56.847706+0000", 2024-05-08T08:52:51.385 INFO:teuthology.orchestra.run.smithi059.stdout: "last_in_change": "2024-05-08T08:52:46.045870+0000", 2024-05-08T08:52:51.385 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-05-08T08:52:51.385 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_num": 5806080, 2024-05-08T08:52:51.385 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_set": [ 2024-05-08T08:52:51.385 INFO:teuthology.orchestra.run.smithi059.stdout: "nodeep-scrub", 2024-05-08T08:52:51.385 INFO:teuthology.orchestra.run.smithi059.stdout: "noscrub", 2024-05-08T08:52:51.385 INFO:teuthology.orchestra.run.smithi059.stdout: "pglog_hardlimit", 2024-05-08T08:52:51.385 INFO:teuthology.orchestra.run.smithi059.stdout: "purged_snapdirs", 2024-05-08T08:52:51.385 INFO:teuthology.orchestra.run.smithi059.stdout: "recovery_deletes", 2024-05-08T08:52:51.385 INFO:teuthology.orchestra.run.smithi059.stdout: "sortbitwise" 2024-05-08T08:52:51.385 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T08:52:51.385 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_version": 4, 2024-05-08T08:52:51.385 INFO:teuthology.orchestra.run.smithi059.stdout: "full_ratio": 0.94999998807907104, 2024-05-08T08:52:51.385 INFO:teuthology.orchestra.run.smithi059.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-05-08T08:52:51.385 INFO:teuthology.orchestra.run.smithi059.stdout: "nearfull_ratio": 0.85000002384185791, 2024-05-08T08:52:51.386 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_snapshot": "", 2024-05-08T08:52:51.386 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_max": 3, 2024-05-08T08:52:51.386 INFO:teuthology.orchestra.run.smithi059.stdout: "max_osd": 6, 2024-05-08T08:52:51.386 INFO:teuthology.orchestra.run.smithi059.stdout: "require_min_compat_client": "luminous", 2024-05-08T08:52:51.386 INFO:teuthology.orchestra.run.smithi059.stdout: "min_compat_client": "luminous", 2024-05-08T08:52:51.386 INFO:teuthology.orchestra.run.smithi059.stdout: "require_osd_release": "squid", 2024-05-08T08:52:51.386 INFO:teuthology.orchestra.run.smithi059.stdout: "allow_crimson": false, 2024-05-08T08:52:51.386 INFO:teuthology.orchestra.run.smithi059.stdout: "pools": [ 2024-05-08T08:52:51.386 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:51.386 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 1, 2024-05-08T08:52:51.386 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_name": ".mgr", 2024-05-08T08:52:51.386 INFO:teuthology.orchestra.run.smithi059.stdout: "create_time": "2024-05-08T08:51:57.787790+0000", 2024-05-08T08:52:51.386 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T08:52:51.387 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_names": "hashpspool", 2024-05-08T08:52:51.387 INFO:teuthology.orchestra.run.smithi059.stdout: "type": 1, 2024-05-08T08:52:51.387 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 2, 2024-05-08T08:52:51.387 INFO:teuthology.orchestra.run.smithi059.stdout: "min_size": 1, 2024-05-08T08:52:51.387 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_rule": 0, 2024-05-08T08:52:51.387 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_count": 0, 2024-05-08T08:52:51.387 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_target": 0, 2024-05-08T08:52:51.387 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_barrier": 0, 2024-05-08T08:52:51.387 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-08T08:52:51.387 INFO:teuthology.orchestra.run.smithi059.stdout: "object_hash": 2, 2024-05-08T08:52:51.387 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscale_mode": "off", 2024-05-08T08:52:51.387 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num": 1, 2024-05-08T08:52:51.387 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num": 1, 2024-05-08T08:52:51.387 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num_target": 1, 2024-05-08T08:52:51.387 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_target": 1, 2024-05-08T08:52:51.388 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_pending": 1, 2024-05-08T08:52:51.388 INFO:teuthology.orchestra.run.smithi059.stdout: "last_pg_merge_meta": { 2024-05-08T08:52:51.388 INFO:teuthology.orchestra.run.smithi059.stdout: "source_pgid": "0.0", 2024-05-08T08:52:51.388 INFO:teuthology.orchestra.run.smithi059.stdout: "ready_epoch": 0, 2024-05-08T08:52:51.388 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_started": 0, 2024-05-08T08:52:51.388 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_clean": 0, 2024-05-08T08:52:51.388 INFO:teuthology.orchestra.run.smithi059.stdout: "source_version": "0'0", 2024-05-08T08:52:51.388 INFO:teuthology.orchestra.run.smithi059.stdout: "target_version": "0'0" 2024-05-08T08:52:51.388 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:51.388 INFO:teuthology.orchestra.run.smithi059.stdout: "last_change": "14", 2024-05-08T08:52:51.388 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend": "0", 2024-05-08T08:52:51.388 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-08T08:52:51.388 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_preluminous": "0", 2024-05-08T08:52:51.388 INFO:teuthology.orchestra.run.smithi059.stdout: "auid": 0, 2024-05-08T08:52:51.388 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_mode": "selfmanaged", 2024-05-08T08:52:51.389 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_seq": 0, 2024-05-08T08:52:51.389 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_epoch": 0, 2024-05-08T08:52:51.389 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_snaps": [], 2024-05-08T08:52:51.389 INFO:teuthology.orchestra.run.smithi059.stdout: "removed_snaps": "[]", 2024-05-08T08:52:51.389 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_bytes": 0, 2024-05-08T08:52:51.389 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_objects": 0, 2024-05-08T08:52:51.389 INFO:teuthology.orchestra.run.smithi059.stdout: "tiers": [], 2024-05-08T08:52:51.389 INFO:teuthology.orchestra.run.smithi059.stdout: "tier_of": -1, 2024-05-08T08:52:51.389 INFO:teuthology.orchestra.run.smithi059.stdout: "read_tier": -1, 2024-05-08T08:52:51.389 INFO:teuthology.orchestra.run.smithi059.stdout: "write_tier": -1, 2024-05-08T08:52:51.389 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_mode": "none", 2024-05-08T08:52:51.389 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_bytes": 0, 2024-05-08T08:52:51.389 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_objects": 0, 2024-05-08T08:52:51.389 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-08T08:52:51.390 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-08T08:52:51.390 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-08T08:52:51.390 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_flush_age": 0, 2024-05-08T08:52:51.390 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_evict_age": 0, 2024-05-08T08:52:51.390 INFO:teuthology.orchestra.run.smithi059.stdout: "erasure_code_profile": "", 2024-05-08T08:52:51.390 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_params": { 2024-05-08T08:52:51.390 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "none" 2024-05-08T08:52:51.390 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:51.390 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_period": 0, 2024-05-08T08:52:51.390 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_count": 0, 2024-05-08T08:52:51.390 INFO:teuthology.orchestra.run.smithi059.stdout: "use_gmt_hitset": true, 2024-05-08T08:52:51.390 INFO:teuthology.orchestra.run.smithi059.stdout: "min_read_recency_for_promote": 0, 2024-05-08T08:52:51.390 INFO:teuthology.orchestra.run.smithi059.stdout: "min_write_recency_for_promote": 0, 2024-05-08T08:52:51.390 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_grade_decay_rate": 0, 2024-05-08T08:52:51.391 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_search_last_n": 0, 2024-05-08T08:52:51.391 INFO:teuthology.orchestra.run.smithi059.stdout: "grade_table": [], 2024-05-08T08:52:51.391 INFO:teuthology.orchestra.run.smithi059.stdout: "stripe_width": 0, 2024-05-08T08:52:51.391 INFO:teuthology.orchestra.run.smithi059.stdout: "expected_num_objects": 0, 2024-05-08T08:52:51.391 INFO:teuthology.orchestra.run.smithi059.stdout: "fast_read": false, 2024-05-08T08:52:51.391 INFO:teuthology.orchestra.run.smithi059.stdout: "options": { 2024-05-08T08:52:51.391 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_max": 32, 2024-05-08T08:52:51.391 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_min": 1 2024-05-08T08:52:51.391 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:51.391 INFO:teuthology.orchestra.run.smithi059.stdout: "application_metadata": { 2024-05-08T08:52:51.391 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": {} 2024-05-08T08:52:51.391 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:51.391 INFO:teuthology.orchestra.run.smithi059.stdout: "read_balance": { 2024-05-08T08:52:51.391 INFO:teuthology.orchestra.run.smithi059.stdout: "score_type": "Fair distribution", 2024-05-08T08:52:51.391 INFO:teuthology.orchestra.run.smithi059.stdout: "score_acting": 6.059999942779541, 2024-05-08T08:52:51.391 INFO:teuthology.orchestra.run.smithi059.stdout: "score_stable": 6.059999942779541, 2024-05-08T08:52:51.392 INFO:teuthology.orchestra.run.smithi059.stdout: "optimal_score": 0.33000001311302185, 2024-05-08T08:52:51.392 INFO:teuthology.orchestra.run.smithi059.stdout: "raw_score_acting": 2, 2024-05-08T08:52:51.392 INFO:teuthology.orchestra.run.smithi059.stdout: "raw_score_stable": 2, 2024-05-08T08:52:51.392 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity_weighted": 0.87819164991378784, 2024-05-08T08:52:51.392 INFO:teuthology.orchestra.run.smithi059.stdout: "average_primary_affinity": 0.87999999523162842, 2024-05-08T08:52:51.392 INFO:teuthology.orchestra.run.smithi059.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-05-08T08:52:51.392 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:51.392 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:51.392 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:51.392 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 2, 2024-05-08T08:52:51.392 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_name": "rbd", 2024-05-08T08:52:51.392 INFO:teuthology.orchestra.run.smithi059.stdout: "create_time": "2024-05-08T08:52:02.563677+0000", 2024-05-08T08:52:51.392 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T08:52:51.392 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_names": "hashpspool", 2024-05-08T08:52:51.392 INFO:teuthology.orchestra.run.smithi059.stdout: "type": 1, 2024-05-08T08:52:51.393 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 2, 2024-05-08T08:52:51.393 INFO:teuthology.orchestra.run.smithi059.stdout: "min_size": 1, 2024-05-08T08:52:51.393 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_rule": 0, 2024-05-08T08:52:51.393 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_count": 0, 2024-05-08T08:52:51.393 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_target": 0, 2024-05-08T08:52:51.393 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_barrier": 0, 2024-05-08T08:52:51.393 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-08T08:52:51.393 INFO:teuthology.orchestra.run.smithi059.stdout: "object_hash": 2, 2024-05-08T08:52:51.393 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscale_mode": "off", 2024-05-08T08:52:51.393 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num": 8, 2024-05-08T08:52:51.393 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num": 8, 2024-05-08T08:52:51.393 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num_target": 8, 2024-05-08T08:52:51.393 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_target": 8, 2024-05-08T08:52:51.393 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_pending": 8, 2024-05-08T08:52:51.394 INFO:teuthology.orchestra.run.smithi059.stdout: "last_pg_merge_meta": { 2024-05-08T08:52:51.394 INFO:teuthology.orchestra.run.smithi059.stdout: "source_pgid": "0.0", 2024-05-08T08:52:51.394 INFO:teuthology.orchestra.run.smithi059.stdout: "ready_epoch": 0, 2024-05-08T08:52:51.394 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_started": 0, 2024-05-08T08:52:51.394 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_clean": 0, 2024-05-08T08:52:51.394 INFO:teuthology.orchestra.run.smithi059.stdout: "source_version": "0'0", 2024-05-08T08:52:51.394 INFO:teuthology.orchestra.run.smithi059.stdout: "target_version": "0'0" 2024-05-08T08:52:51.394 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:51.394 INFO:teuthology.orchestra.run.smithi059.stdout: "last_change": "17", 2024-05-08T08:52:51.394 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend": "0", 2024-05-08T08:52:51.394 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-08T08:52:51.394 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_preluminous": "0", 2024-05-08T08:52:51.394 INFO:teuthology.orchestra.run.smithi059.stdout: "auid": 0, 2024-05-08T08:52:51.395 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_mode": "selfmanaged", 2024-05-08T08:52:51.395 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_seq": 0, 2024-05-08T08:52:51.395 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_epoch": 0, 2024-05-08T08:52:51.395 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_snaps": [], 2024-05-08T08:52:51.395 INFO:teuthology.orchestra.run.smithi059.stdout: "removed_snaps": "[]", 2024-05-08T08:52:51.395 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_bytes": 0, 2024-05-08T08:52:51.395 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_objects": 0, 2024-05-08T08:52:51.395 INFO:teuthology.orchestra.run.smithi059.stdout: "tiers": [], 2024-05-08T08:52:51.395 INFO:teuthology.orchestra.run.smithi059.stdout: "tier_of": -1, 2024-05-08T08:52:51.395 INFO:teuthology.orchestra.run.smithi059.stdout: "read_tier": -1, 2024-05-08T08:52:51.395 INFO:teuthology.orchestra.run.smithi059.stdout: "write_tier": -1, 2024-05-08T08:52:51.395 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_mode": "none", 2024-05-08T08:52:51.395 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_bytes": 0, 2024-05-08T08:52:51.395 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_objects": 0, 2024-05-08T08:52:51.395 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-08T08:52:51.395 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-08T08:52:51.396 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-08T08:52:51.396 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_flush_age": 0, 2024-05-08T08:52:51.396 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_evict_age": 0, 2024-05-08T08:52:51.396 INFO:teuthology.orchestra.run.smithi059.stdout: "erasure_code_profile": "", 2024-05-08T08:52:51.396 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_params": { 2024-05-08T08:52:51.396 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "none" 2024-05-08T08:52:51.396 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:51.396 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_period": 0, 2024-05-08T08:52:51.396 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_count": 0, 2024-05-08T08:52:51.396 INFO:teuthology.orchestra.run.smithi059.stdout: "use_gmt_hitset": true, 2024-05-08T08:52:51.396 INFO:teuthology.orchestra.run.smithi059.stdout: "min_read_recency_for_promote": 0, 2024-05-08T08:52:51.396 INFO:teuthology.orchestra.run.smithi059.stdout: "min_write_recency_for_promote": 0, 2024-05-08T08:52:51.396 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_grade_decay_rate": 0, 2024-05-08T08:52:51.396 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_search_last_n": 0, 2024-05-08T08:52:51.397 INFO:teuthology.orchestra.run.smithi059.stdout: "grade_table": [], 2024-05-08T08:52:51.397 INFO:teuthology.orchestra.run.smithi059.stdout: "stripe_width": 0, 2024-05-08T08:52:51.397 INFO:teuthology.orchestra.run.smithi059.stdout: "expected_num_objects": 0, 2024-05-08T08:52:51.397 INFO:teuthology.orchestra.run.smithi059.stdout: "fast_read": false, 2024-05-08T08:52:51.397 INFO:teuthology.orchestra.run.smithi059.stdout: "options": {}, 2024-05-08T08:52:51.397 INFO:teuthology.orchestra.run.smithi059.stdout: "application_metadata": { 2024-05-08T08:52:51.397 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd": {} 2024-05-08T08:52:51.397 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:51.397 INFO:teuthology.orchestra.run.smithi059.stdout: "read_balance": { 2024-05-08T08:52:51.397 INFO:teuthology.orchestra.run.smithi059.stdout: "score_type": "Fair distribution", 2024-05-08T08:52:51.397 INFO:teuthology.orchestra.run.smithi059.stdout: "score_acting": 2.2699999809265137, 2024-05-08T08:52:51.397 INFO:teuthology.orchestra.run.smithi059.stdout: "score_stable": 2.2699999809265137, 2024-05-08T08:52:51.397 INFO:teuthology.orchestra.run.smithi059.stdout: "optimal_score": 0.82999998331069946, 2024-05-08T08:52:51.397 INFO:teuthology.orchestra.run.smithi059.stdout: "raw_score_acting": 1.8799999952316284, 2024-05-08T08:52:51.397 INFO:teuthology.orchestra.run.smithi059.stdout: "raw_score_stable": 1.8799999952316284, 2024-05-08T08:52:51.398 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity_weighted": 0.87819164991378784, 2024-05-08T08:52:51.398 INFO:teuthology.orchestra.run.smithi059.stdout: "average_primary_affinity": 0.87999999523162842, 2024-05-08T08:52:51.398 INFO:teuthology.orchestra.run.smithi059.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-05-08T08:52:51.398 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:51.398 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:51.398 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:51.398 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 3, 2024-05-08T08:52:51.398 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_name": "pool_client.0", 2024-05-08T08:52:51.398 INFO:teuthology.orchestra.run.smithi059.stdout: "create_time": "2024-05-08T08:52:09.955467+0000", 2024-05-08T08:52:51.398 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 8193, 2024-05-08T08:52:51.398 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-05-08T08:52:51.398 INFO:teuthology.orchestra.run.smithi059.stdout: "type": 1, 2024-05-08T08:52:51.398 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 2, 2024-05-08T08:52:51.398 INFO:teuthology.orchestra.run.smithi059.stdout: "min_size": 1, 2024-05-08T08:52:51.399 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_rule": 0, 2024-05-08T08:52:51.399 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_count": 0, 2024-05-08T08:52:51.399 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_target": 0, 2024-05-08T08:52:51.399 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_barrier": 0, 2024-05-08T08:52:51.399 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-08T08:52:51.399 INFO:teuthology.orchestra.run.smithi059.stdout: "object_hash": 2, 2024-05-08T08:52:51.399 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscale_mode": "off", 2024-05-08T08:52:51.399 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num": 32, 2024-05-08T08:52:51.399 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num": 32, 2024-05-08T08:52:51.399 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num_target": 32, 2024-05-08T08:52:51.399 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_target": 32, 2024-05-08T08:52:51.399 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_pending": 32, 2024-05-08T08:52:51.399 INFO:teuthology.orchestra.run.smithi059.stdout: "last_pg_merge_meta": { 2024-05-08T08:52:51.399 INFO:teuthology.orchestra.run.smithi059.stdout: "source_pgid": "0.0", 2024-05-08T08:52:51.399 INFO:teuthology.orchestra.run.smithi059.stdout: "ready_epoch": 0, 2024-05-08T08:52:51.400 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_started": 0, 2024-05-08T08:52:51.400 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_clean": 0, 2024-05-08T08:52:51.400 INFO:teuthology.orchestra.run.smithi059.stdout: "source_version": "0'0", 2024-05-08T08:52:51.400 INFO:teuthology.orchestra.run.smithi059.stdout: "target_version": "0'0" 2024-05-08T08:52:51.400 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:51.400 INFO:teuthology.orchestra.run.smithi059.stdout: "last_change": "44", 2024-05-08T08:52:51.400 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend": "0", 2024-05-08T08:52:51.400 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-08T08:52:51.400 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_preluminous": "0", 2024-05-08T08:52:51.400 INFO:teuthology.orchestra.run.smithi059.stdout: "auid": 0, 2024-05-08T08:52:51.400 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_mode": "selfmanaged", 2024-05-08T08:52:51.400 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_seq": 13, 2024-05-08T08:52:51.400 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_epoch": 44, 2024-05-08T08:52:51.401 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_snaps": [], 2024-05-08T08:52:51.401 INFO:teuthology.orchestra.run.smithi059.stdout: "removed_snaps": "[]", 2024-05-08T08:52:51.401 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_bytes": 0, 2024-05-08T08:52:51.401 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_objects": 0, 2024-05-08T08:52:51.401 INFO:teuthology.orchestra.run.smithi059.stdout: "tiers": [], 2024-05-08T08:52:51.401 INFO:teuthology.orchestra.run.smithi059.stdout: "tier_of": -1, 2024-05-08T08:52:51.401 INFO:teuthology.orchestra.run.smithi059.stdout: "read_tier": -1, 2024-05-08T08:52:51.401 INFO:teuthology.orchestra.run.smithi059.stdout: "write_tier": -1, 2024-05-08T08:52:51.401 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_mode": "none", 2024-05-08T08:52:51.401 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_bytes": 0, 2024-05-08T08:52:51.401 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_objects": 0, 2024-05-08T08:52:51.401 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-08T08:52:51.401 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-08T08:52:51.401 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-08T08:52:51.401 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_flush_age": 0, 2024-05-08T08:52:51.402 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_evict_age": 0, 2024-05-08T08:52:51.402 INFO:teuthology.orchestra.run.smithi059.stdout: "erasure_code_profile": "", 2024-05-08T08:52:51.402 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_params": { 2024-05-08T08:52:51.402 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "none" 2024-05-08T08:52:51.402 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:51.402 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_period": 0, 2024-05-08T08:52:51.402 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_count": 0, 2024-05-08T08:52:51.402 INFO:teuthology.orchestra.run.smithi059.stdout: "use_gmt_hitset": true, 2024-05-08T08:52:51.402 INFO:teuthology.orchestra.run.smithi059.stdout: "min_read_recency_for_promote": 0, 2024-05-08T08:52:51.402 INFO:teuthology.orchestra.run.smithi059.stdout: "min_write_recency_for_promote": 0, 2024-05-08T08:52:51.402 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_grade_decay_rate": 0, 2024-05-08T08:52:51.402 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_search_last_n": 0, 2024-05-08T08:52:51.402 INFO:teuthology.orchestra.run.smithi059.stdout: "grade_table": [], 2024-05-08T08:52:51.402 INFO:teuthology.orchestra.run.smithi059.stdout: "stripe_width": 0, 2024-05-08T08:52:51.403 INFO:teuthology.orchestra.run.smithi059.stdout: "expected_num_objects": 0, 2024-05-08T08:52:51.403 INFO:teuthology.orchestra.run.smithi059.stdout: "fast_read": false, 2024-05-08T08:52:51.403 INFO:teuthology.orchestra.run.smithi059.stdout: "options": {}, 2024-05-08T08:52:51.403 INFO:teuthology.orchestra.run.smithi059.stdout: "application_metadata": { 2024-05-08T08:52:51.403 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd": {} 2024-05-08T08:52:51.403 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:51.403 INFO:teuthology.orchestra.run.smithi059.stdout: "read_balance": { 2024-05-08T08:52:51.403 INFO:teuthology.orchestra.run.smithi059.stdout: "score_type": "Fair distribution", 2024-05-08T08:52:51.403 INFO:teuthology.orchestra.run.smithi059.stdout: "score_acting": 1.8799999952316284, 2024-05-08T08:52:51.403 INFO:teuthology.orchestra.run.smithi059.stdout: "score_stable": 1.8799999952316284, 2024-05-08T08:52:51.403 INFO:teuthology.orchestra.run.smithi059.stdout: "optimal_score": 0.82999998331069946, 2024-05-08T08:52:51.403 INFO:teuthology.orchestra.run.smithi059.stdout: "raw_score_acting": 1.559999942779541, 2024-05-08T08:52:51.403 INFO:teuthology.orchestra.run.smithi059.stdout: "raw_score_stable": 1.559999942779541, 2024-05-08T08:52:51.403 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity_weighted": 0.87819164991378784, 2024-05-08T08:52:51.403 INFO:teuthology.orchestra.run.smithi059.stdout: "average_primary_affinity": 0.87999999523162842, 2024-05-08T08:52:51.404 INFO:teuthology.orchestra.run.smithi059.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-05-08T08:52:51.404 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:51.404 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:51.404 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T08:52:51.404 INFO:teuthology.orchestra.run.smithi059.stdout: "osds": [ 2024-05-08T08:52:51.404 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:51.404 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 0, 2024-05-08T08:52:51.404 INFO:teuthology.orchestra.run.smithi059.stdout: "uuid": "300801a5-1919-4b23-8c6d-bc7dad61cc4d", 2024-05-08T08:52:51.404 INFO:teuthology.orchestra.run.smithi059.stdout: "up": 1, 2024-05-08T08:52:51.404 INFO:teuthology.orchestra.run.smithi059.stdout: "in": 1, 2024-05-08T08:52:51.404 INFO:teuthology.orchestra.run.smithi059.stdout: "weight": 1, 2024-05-08T08:52:51.404 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity": 1, 2024-05-08T08:52:51.404 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_begin": 0, 2024-05-08T08:52:51.404 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_end": 0, 2024-05-08T08:52:51.405 INFO:teuthology.orchestra.run.smithi059.stdout: "up_from": 11, 2024-05-08T08:52:51.405 INFO:teuthology.orchestra.run.smithi059.stdout: "up_thru": 44, 2024-05-08T08:52:51.405 INFO:teuthology.orchestra.run.smithi059.stdout: "down_at": 0, 2024-05-08T08:52:51.405 INFO:teuthology.orchestra.run.smithi059.stdout: "lost_at": 0, 2024-05-08T08:52:51.405 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrs": { 2024-05-08T08:52:51.405 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:52:51.405 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:51.405 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:52:51.405 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6800", 2024-05-08T08:52:51.405 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 802160151 2024-05-08T08:52:51.405 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:51.405 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:51.405 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:52:51.405 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6801", 2024-05-08T08:52:51.405 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 802160151 2024-05-08T08:52:51.406 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:51.406 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:51.406 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:51.406 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addrs": { 2024-05-08T08:52:51.406 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:52:51.406 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:51.406 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:52:51.406 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6802", 2024-05-08T08:52:51.406 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 802160151 2024-05-08T08:52:51.406 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:51.406 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:51.406 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:52:51.406 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6803", 2024-05-08T08:52:51.407 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 802160151 2024-05-08T08:52:51.407 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:51.407 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:51.407 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:51.407 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addrs": { 2024-05-08T08:52:51.407 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:52:51.407 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:51.407 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:52:51.407 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6806", 2024-05-08T08:52:51.407 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 802160151 2024-05-08T08:52:51.407 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:51.407 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:51.407 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:52:51.407 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6807", 2024-05-08T08:52:51.407 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 802160151 2024-05-08T08:52:51.408 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:51.408 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:51.408 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:51.408 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addrs": { 2024-05-08T08:52:51.408 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:52:51.408 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:51.408 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:52:51.408 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6804", 2024-05-08T08:52:51.408 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 802160151 2024-05-08T08:52:51.408 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:51.408 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:51.408 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:52:51.408 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6805", 2024-05-08T08:52:51.408 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 802160151 2024-05-08T08:52:51.408 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:51.409 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:51.409 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:51.409 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "172.21.15.59:6801/802160151", 2024-05-08T08:52:51.409 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "172.21.15.59:6803/802160151", 2024-05-08T08:52:51.409 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addr": "172.21.15.59:6807/802160151", 2024-05-08T08:52:51.409 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addr": "172.21.15.59:6805/802160151", 2024-05-08T08:52:51.409 INFO:teuthology.orchestra.run.smithi059.stdout: "state": [ 2024-05-08T08:52:51.409 INFO:teuthology.orchestra.run.smithi059.stdout: "exists", 2024-05-08T08:52:51.409 INFO:teuthology.orchestra.run.smithi059.stdout: "up" 2024-05-08T08:52:51.409 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:51.409 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:51.409 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:51.409 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 1, 2024-05-08T08:52:51.409 INFO:teuthology.orchestra.run.smithi059.stdout: "uuid": "0fead0de-6a65-4eba-b400-eea5692702e0", 2024-05-08T08:52:51.409 INFO:teuthology.orchestra.run.smithi059.stdout: "up": 1, 2024-05-08T08:52:51.410 INFO:teuthology.orchestra.run.smithi059.stdout: "in": 1, 2024-05-08T08:52:51.410 INFO:teuthology.orchestra.run.smithi059.stdout: "weight": 1, 2024-05-08T08:52:51.410 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity": 1, 2024-05-08T08:52:51.410 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_begin": 0, 2024-05-08T08:52:51.410 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_end": 0, 2024-05-08T08:52:51.410 INFO:teuthology.orchestra.run.smithi059.stdout: "up_from": 11, 2024-05-08T08:52:51.410 INFO:teuthology.orchestra.run.smithi059.stdout: "up_thru": 44, 2024-05-08T08:52:51.410 INFO:teuthology.orchestra.run.smithi059.stdout: "down_at": 0, 2024-05-08T08:52:51.410 INFO:teuthology.orchestra.run.smithi059.stdout: "lost_at": 0, 2024-05-08T08:52:51.410 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrs": { 2024-05-08T08:52:51.410 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:52:51.410 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:51.410 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:52:51.410 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6816", 2024-05-08T08:52:51.411 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4116801011 2024-05-08T08:52:51.411 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:51.411 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:51.411 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:52:51.411 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6817", 2024-05-08T08:52:51.411 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4116801011 2024-05-08T08:52:51.411 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:51.411 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:51.411 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:51.411 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addrs": { 2024-05-08T08:52:51.411 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:52:51.411 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:51.411 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:52:51.411 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6818", 2024-05-08T08:52:51.411 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4116801011 2024-05-08T08:52:51.412 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:51.412 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:51.412 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:52:51.412 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6819", 2024-05-08T08:52:51.412 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4116801011 2024-05-08T08:52:51.412 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:51.412 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:51.412 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:51.412 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addrs": { 2024-05-08T08:52:51.412 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:52:51.412 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:51.412 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:52:51.412 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6822", 2024-05-08T08:52:51.412 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4116801011 2024-05-08T08:52:51.412 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:51.413 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:51.413 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:52:51.413 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6823", 2024-05-08T08:52:51.413 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4116801011 2024-05-08T08:52:51.413 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:51.413 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:51.413 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:51.413 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addrs": { 2024-05-08T08:52:51.413 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:52:51.413 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:51.413 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:52:51.413 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6820", 2024-05-08T08:52:51.413 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4116801011 2024-05-08T08:52:51.413 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:51.414 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:51.414 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:52:51.414 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6821", 2024-05-08T08:52:51.414 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4116801011 2024-05-08T08:52:51.414 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:51.414 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:51.414 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:51.414 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "172.21.15.59:6817/4116801011", 2024-05-08T08:52:51.414 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "172.21.15.59:6819/4116801011", 2024-05-08T08:52:51.414 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addr": "172.21.15.59:6823/4116801011", 2024-05-08T08:52:51.414 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addr": "172.21.15.59:6821/4116801011", 2024-05-08T08:52:51.414 INFO:teuthology.orchestra.run.smithi059.stdout: "state": [ 2024-05-08T08:52:51.414 INFO:teuthology.orchestra.run.smithi059.stdout: "exists", 2024-05-08T08:52:51.414 INFO:teuthology.orchestra.run.smithi059.stdout: "up" 2024-05-08T08:52:51.414 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:51.415 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:51.415 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:51.415 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 2, 2024-05-08T08:52:51.415 INFO:teuthology.orchestra.run.smithi059.stdout: "uuid": "1a233de6-1b82-4a51-8e96-4cd4abdf16c8", 2024-05-08T08:52:51.415 INFO:teuthology.orchestra.run.smithi059.stdout: "up": 1, 2024-05-08T08:52:51.415 INFO:teuthology.orchestra.run.smithi059.stdout: "in": 1, 2024-05-08T08:52:51.415 INFO:teuthology.orchestra.run.smithi059.stdout: "weight": 1, 2024-05-08T08:52:51.415 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity": 1, 2024-05-08T08:52:51.415 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_begin": 0, 2024-05-08T08:52:51.415 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_end": 0, 2024-05-08T08:52:51.415 INFO:teuthology.orchestra.run.smithi059.stdout: "up_from": 11, 2024-05-08T08:52:51.415 INFO:teuthology.orchestra.run.smithi059.stdout: "up_thru": 44, 2024-05-08T08:52:51.415 INFO:teuthology.orchestra.run.smithi059.stdout: "down_at": 0, 2024-05-08T08:52:51.415 INFO:teuthology.orchestra.run.smithi059.stdout: "lost_at": 0, 2024-05-08T08:52:51.415 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrs": { 2024-05-08T08:52:51.416 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:52:51.416 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:51.416 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:52:51.416 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6808", 2024-05-08T08:52:51.416 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1547661179 2024-05-08T08:52:51.416 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:51.416 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:51.416 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:52:51.416 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6809", 2024-05-08T08:52:51.416 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1547661179 2024-05-08T08:52:51.416 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:51.416 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:51.416 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:51.416 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addrs": { 2024-05-08T08:52:51.416 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:52:51.417 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:51.417 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:52:51.417 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6810", 2024-05-08T08:52:51.417 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1547661179 2024-05-08T08:52:51.417 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:51.417 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:51.417 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:52:51.417 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6811", 2024-05-08T08:52:51.417 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1547661179 2024-05-08T08:52:51.417 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:51.417 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:51.417 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:51.417 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addrs": { 2024-05-08T08:52:51.417 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:52:51.418 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:51.418 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:52:51.418 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6814", 2024-05-08T08:52:51.418 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1547661179 2024-05-08T08:52:51.418 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:51.418 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:51.418 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:52:51.418 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6815", 2024-05-08T08:52:51.418 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1547661179 2024-05-08T08:52:51.418 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:51.418 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:51.418 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:51.418 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addrs": { 2024-05-08T08:52:51.418 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:52:51.419 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:51.419 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:52:51.419 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6812", 2024-05-08T08:52:51.419 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1547661179 2024-05-08T08:52:51.419 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:51.419 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:51.419 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:52:51.419 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6813", 2024-05-08T08:52:51.419 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1547661179 2024-05-08T08:52:51.419 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:51.419 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:51.419 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:51.419 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "172.21.15.59:6809/1547661179", 2024-05-08T08:52:51.419 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "172.21.15.59:6811/1547661179", 2024-05-08T08:52:51.419 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addr": "172.21.15.59:6815/1547661179", 2024-05-08T08:52:51.420 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addr": "172.21.15.59:6813/1547661179", 2024-05-08T08:52:51.420 INFO:teuthology.orchestra.run.smithi059.stdout: "state": [ 2024-05-08T08:52:51.420 INFO:teuthology.orchestra.run.smithi059.stdout: "exists", 2024-05-08T08:52:51.420 INFO:teuthology.orchestra.run.smithi059.stdout: "up" 2024-05-08T08:52:51.420 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:51.420 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:51.420 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:51.420 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 3, 2024-05-08T08:52:51.420 INFO:teuthology.orchestra.run.smithi059.stdout: "uuid": "cc4870b8-dcd9-4e43-978d-8de6c6396abf", 2024-05-08T08:52:51.420 INFO:teuthology.orchestra.run.smithi059.stdout: "up": 1, 2024-05-08T08:52:51.420 INFO:teuthology.orchestra.run.smithi059.stdout: "in": 1, 2024-05-08T08:52:51.420 INFO:teuthology.orchestra.run.smithi059.stdout: "weight": 1, 2024-05-08T08:52:51.420 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity": 1, 2024-05-08T08:52:51.420 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_begin": 0, 2024-05-08T08:52:51.420 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_end": 0, 2024-05-08T08:52:51.421 INFO:teuthology.orchestra.run.smithi059.stdout: "up_from": 11, 2024-05-08T08:52:51.421 INFO:teuthology.orchestra.run.smithi059.stdout: "up_thru": 44, 2024-05-08T08:52:51.421 INFO:teuthology.orchestra.run.smithi059.stdout: "down_at": 0, 2024-05-08T08:52:51.421 INFO:teuthology.orchestra.run.smithi059.stdout: "lost_at": 0, 2024-05-08T08:52:51.421 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrs": { 2024-05-08T08:52:51.421 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:52:51.421 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:51.421 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:52:51.421 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6800", 2024-05-08T08:52:51.421 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4054400810 2024-05-08T08:52:51.421 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:51.421 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:51.421 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:52:51.421 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6801", 2024-05-08T08:52:51.421 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4054400810 2024-05-08T08:52:51.422 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:51.422 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:51.422 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:51.422 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addrs": { 2024-05-08T08:52:51.422 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:52:51.422 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:51.422 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:52:51.422 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6802", 2024-05-08T08:52:51.422 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4054400810 2024-05-08T08:52:51.422 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:51.422 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:51.422 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:52:51.422 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6803", 2024-05-08T08:52:51.423 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4054400810 2024-05-08T08:52:51.423 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:51.423 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:51.423 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:51.423 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addrs": { 2024-05-08T08:52:51.423 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:52:51.423 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:51.423 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:52:51.423 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6806", 2024-05-08T08:52:51.423 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4054400810 2024-05-08T08:52:51.423 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:51.423 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:51.423 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:52:51.423 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6807", 2024-05-08T08:52:51.423 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4054400810 2024-05-08T08:52:51.424 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:51.424 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:51.424 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:51.424 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addrs": { 2024-05-08T08:52:51.424 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:52:51.424 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:51.424 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:52:51.424 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6804", 2024-05-08T08:52:51.424 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4054400810 2024-05-08T08:52:51.424 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:51.424 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:51.424 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:52:51.424 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6805", 2024-05-08T08:52:51.424 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4054400810 2024-05-08T08:52:51.424 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:51.425 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:51.425 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:51.425 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "172.21.15.192:6801/4054400810", 2024-05-08T08:52:51.425 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "172.21.15.192:6803/4054400810", 2024-05-08T08:52:51.425 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addr": "172.21.15.192:6807/4054400810", 2024-05-08T08:52:51.425 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addr": "172.21.15.192:6805/4054400810", 2024-05-08T08:52:51.425 INFO:teuthology.orchestra.run.smithi059.stdout: "state": [ 2024-05-08T08:52:51.425 INFO:teuthology.orchestra.run.smithi059.stdout: "exists", 2024-05-08T08:52:51.425 INFO:teuthology.orchestra.run.smithi059.stdout: "up" 2024-05-08T08:52:51.425 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:51.425 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:51.425 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:51.425 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 4, 2024-05-08T08:52:51.425 INFO:teuthology.orchestra.run.smithi059.stdout: "uuid": "d1d05620-42fb-4107-a2f9-e83542d076ee", 2024-05-08T08:52:51.425 INFO:teuthology.orchestra.run.smithi059.stdout: "up": 1, 2024-05-08T08:52:51.426 INFO:teuthology.orchestra.run.smithi059.stdout: "in": 0, 2024-05-08T08:52:51.426 INFO:teuthology.orchestra.run.smithi059.stdout: "weight": 0, 2024-05-08T08:52:51.426 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity": 0.9892578125, 2024-05-08T08:52:51.426 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_begin": 0, 2024-05-08T08:52:51.426 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_end": 0, 2024-05-08T08:52:51.426 INFO:teuthology.orchestra.run.smithi059.stdout: "up_from": 11, 2024-05-08T08:52:51.426 INFO:teuthology.orchestra.run.smithi059.stdout: "up_thru": 43, 2024-05-08T08:52:51.426 INFO:teuthology.orchestra.run.smithi059.stdout: "down_at": 0, 2024-05-08T08:52:51.426 INFO:teuthology.orchestra.run.smithi059.stdout: "lost_at": 0, 2024-05-08T08:52:51.426 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrs": { 2024-05-08T08:52:51.426 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:52:51.426 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:51.426 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:52:51.426 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6808", 2024-05-08T08:52:51.427 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 58985565 2024-05-08T08:52:51.427 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:51.427 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:51.427 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:52:51.427 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6809", 2024-05-08T08:52:51.427 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 58985565 2024-05-08T08:52:51.427 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:51.427 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:51.427 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:51.427 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addrs": { 2024-05-08T08:52:51.427 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:52:51.427 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:51.427 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:52:51.427 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6810", 2024-05-08T08:52:51.427 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 58985565 2024-05-08T08:52:51.428 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:51.428 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:51.428 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:52:51.428 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6811", 2024-05-08T08:52:51.428 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 58985565 2024-05-08T08:52:51.428 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:51.428 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:51.428 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:51.428 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addrs": { 2024-05-08T08:52:51.428 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:52:51.428 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:51.428 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:52:51.428 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6814", 2024-05-08T08:52:51.428 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 58985565 2024-05-08T08:52:51.428 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:51.429 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:51.429 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:52:51.429 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6815", 2024-05-08T08:52:51.429 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 58985565 2024-05-08T08:52:51.429 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:51.429 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:51.429 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:51.429 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addrs": { 2024-05-08T08:52:51.429 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:52:51.429 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:51.429 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:52:51.429 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6812", 2024-05-08T08:52:51.429 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 58985565 2024-05-08T08:52:51.429 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:51.429 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:51.429 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:52:51.430 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6813", 2024-05-08T08:52:51.430 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 58985565 2024-05-08T08:52:51.430 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:51.430 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:51.430 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:51.430 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "172.21.15.192:6809/58985565", 2024-05-08T08:52:51.430 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "172.21.15.192:6811/58985565", 2024-05-08T08:52:51.430 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addr": "172.21.15.192:6815/58985565", 2024-05-08T08:52:51.430 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addr": "172.21.15.192:6813/58985565", 2024-05-08T08:52:51.430 INFO:teuthology.orchestra.run.smithi059.stdout: "state": [ 2024-05-08T08:52:51.430 INFO:teuthology.orchestra.run.smithi059.stdout: "exists", 2024-05-08T08:52:51.430 INFO:teuthology.orchestra.run.smithi059.stdout: "up" 2024-05-08T08:52:51.430 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:51.430 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:51.431 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:51.431 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 5, 2024-05-08T08:52:51.431 INFO:teuthology.orchestra.run.smithi059.stdout: "uuid": "ab26136c-31c3-4090-9fa2-988dba4f731f", 2024-05-08T08:52:51.431 INFO:teuthology.orchestra.run.smithi059.stdout: "up": 1, 2024-05-08T08:52:51.431 INFO:teuthology.orchestra.run.smithi059.stdout: "in": 1, 2024-05-08T08:52:51.431 INFO:teuthology.orchestra.run.smithi059.stdout: "weight": 1, 2024-05-08T08:52:51.431 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity": 0.2798919677734375, 2024-05-08T08:52:51.431 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_begin": 0, 2024-05-08T08:52:51.431 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_end": 0, 2024-05-08T08:52:51.431 INFO:teuthology.orchestra.run.smithi059.stdout: "up_from": 11, 2024-05-08T08:52:51.431 INFO:teuthology.orchestra.run.smithi059.stdout: "up_thru": 18, 2024-05-08T08:52:51.431 INFO:teuthology.orchestra.run.smithi059.stdout: "down_at": 0, 2024-05-08T08:52:51.431 INFO:teuthology.orchestra.run.smithi059.stdout: "lost_at": 0, 2024-05-08T08:52:51.431 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrs": { 2024-05-08T08:52:51.431 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:52:51.432 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:51.432 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:52:51.432 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6816", 2024-05-08T08:52:51.432 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1283571573 2024-05-08T08:52:51.432 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:51.432 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:51.432 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:52:51.432 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6817", 2024-05-08T08:52:51.432 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1283571573 2024-05-08T08:52:51.432 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:51.432 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:51.432 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:51.432 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addrs": { 2024-05-08T08:52:51.432 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:52:51.433 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:51.433 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:52:51.433 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6818", 2024-05-08T08:52:51.433 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1283571573 2024-05-08T08:52:51.433 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:51.433 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:51.433 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:52:51.433 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6819", 2024-05-08T08:52:51.433 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1283571573 2024-05-08T08:52:51.433 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:51.433 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:51.433 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:51.433 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addrs": { 2024-05-08T08:52:51.433 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:52:51.433 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:51.434 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:52:51.434 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6822", 2024-05-08T08:52:51.434 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1283571573 2024-05-08T08:52:51.434 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:51.434 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:51.434 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:52:51.434 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6823", 2024-05-08T08:52:51.434 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1283571573 2024-05-08T08:52:51.434 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:51.434 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:51.434 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:51.434 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addrs": { 2024-05-08T08:52:51.434 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:52:51.434 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:51.435 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:52:51.435 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6820", 2024-05-08T08:52:51.435 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1283571573 2024-05-08T08:52:51.435 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:51.435 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:51.435 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:52:51.435 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6821", 2024-05-08T08:52:51.435 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1283571573 2024-05-08T08:52:51.435 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:51.435 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:51.435 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:51.435 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "172.21.15.192:6817/1283571573", 2024-05-08T08:52:51.435 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "172.21.15.192:6819/1283571573", 2024-05-08T08:52:51.435 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addr": "172.21.15.192:6823/1283571573", 2024-05-08T08:52:51.435 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addr": "172.21.15.192:6821/1283571573", 2024-05-08T08:52:51.436 INFO:teuthology.orchestra.run.smithi059.stdout: "state": [ 2024-05-08T08:52:51.436 INFO:teuthology.orchestra.run.smithi059.stdout: "exists", 2024-05-08T08:52:51.436 INFO:teuthology.orchestra.run.smithi059.stdout: "up" 2024-05-08T08:52:51.436 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:51.436 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:51.436 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T08:52:51.436 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_xinfo": [ 2024-05-08T08:52:51.436 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:51.436 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 0, 2024-05-08T08:52:51.436 INFO:teuthology.orchestra.run.smithi059.stdout: "down_stamp": "0.000000", 2024-05-08T08:52:51.436 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_probability": 0, 2024-05-08T08:52:51.436 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_interval": 0, 2024-05-08T08:52:51.436 INFO:teuthology.orchestra.run.smithi059.stdout: "features": 4540701547738038271, 2024-05-08T08:52:51.436 INFO:teuthology.orchestra.run.smithi059.stdout: "old_weight": 0, 2024-05-08T08:52:51.436 INFO:teuthology.orchestra.run.smithi059.stdout: "last_purged_snaps_scrub": "2024-05-08T08:51:52.396946+0000", 2024-05-08T08:52:51.437 INFO:teuthology.orchestra.run.smithi059.stdout: "dead_epoch": 0 2024-05-08T08:52:51.437 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:51.437 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:51.437 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 1, 2024-05-08T08:52:51.437 INFO:teuthology.orchestra.run.smithi059.stdout: "down_stamp": "0.000000", 2024-05-08T08:52:51.437 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_probability": 0, 2024-05-08T08:52:51.437 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_interval": 0, 2024-05-08T08:52:51.437 INFO:teuthology.orchestra.run.smithi059.stdout: "features": 4540701547738038271, 2024-05-08T08:52:51.437 INFO:teuthology.orchestra.run.smithi059.stdout: "old_weight": 0, 2024-05-08T08:52:51.437 INFO:teuthology.orchestra.run.smithi059.stdout: "last_purged_snaps_scrub": "2024-05-08T08:51:52.453160+0000", 2024-05-08T08:52:51.437 INFO:teuthology.orchestra.run.smithi059.stdout: "dead_epoch": 0 2024-05-08T08:52:51.437 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:51.437 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:51.437 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 2, 2024-05-08T08:52:51.438 INFO:teuthology.orchestra.run.smithi059.stdout: "down_stamp": "0.000000", 2024-05-08T08:52:51.438 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_probability": 0, 2024-05-08T08:52:51.438 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_interval": 0, 2024-05-08T08:52:51.438 INFO:teuthology.orchestra.run.smithi059.stdout: "features": 4540701547738038271, 2024-05-08T08:52:51.438 INFO:teuthology.orchestra.run.smithi059.stdout: "old_weight": 0, 2024-05-08T08:52:51.438 INFO:teuthology.orchestra.run.smithi059.stdout: "last_purged_snaps_scrub": "2024-05-08T08:51:52.387606+0000", 2024-05-08T08:52:51.438 INFO:teuthology.orchestra.run.smithi059.stdout: "dead_epoch": 0 2024-05-08T08:52:51.438 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:51.438 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:51.438 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 3, 2024-05-08T08:52:51.438 INFO:teuthology.orchestra.run.smithi059.stdout: "down_stamp": "0.000000", 2024-05-08T08:52:51.438 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_probability": 0, 2024-05-08T08:52:51.438 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_interval": 0, 2024-05-08T08:52:51.438 INFO:teuthology.orchestra.run.smithi059.stdout: "features": 4540701547738038271, 2024-05-08T08:52:51.439 INFO:teuthology.orchestra.run.smithi059.stdout: "old_weight": 0, 2024-05-08T08:52:51.439 INFO:teuthology.orchestra.run.smithi059.stdout: "last_purged_snaps_scrub": "2024-05-08T08:51:52.360952+0000", 2024-05-08T08:52:51.439 INFO:teuthology.orchestra.run.smithi059.stdout: "dead_epoch": 0 2024-05-08T08:52:51.439 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:51.439 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:51.439 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 4, 2024-05-08T08:52:51.439 INFO:teuthology.orchestra.run.smithi059.stdout: "down_stamp": "0.000000", 2024-05-08T08:52:51.439 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_probability": 0, 2024-05-08T08:52:51.439 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_interval": 0, 2024-05-08T08:52:51.439 INFO:teuthology.orchestra.run.smithi059.stdout: "features": 4540701547738038271, 2024-05-08T08:52:51.439 INFO:teuthology.orchestra.run.smithi059.stdout: "old_weight": 65536, 2024-05-08T08:52:51.439 INFO:teuthology.orchestra.run.smithi059.stdout: "last_purged_snaps_scrub": "2024-05-08T08:51:52.468681+0000", 2024-05-08T08:52:51.439 INFO:teuthology.orchestra.run.smithi059.stdout: "dead_epoch": 0 2024-05-08T08:52:51.439 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:51.440 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:51.440 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 5, 2024-05-08T08:52:51.440 INFO:teuthology.orchestra.run.smithi059.stdout: "down_stamp": "0.000000", 2024-05-08T08:52:51.440 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_probability": 0, 2024-05-08T08:52:51.440 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_interval": 0, 2024-05-08T08:52:51.440 INFO:teuthology.orchestra.run.smithi059.stdout: "features": 4540701547738038271, 2024-05-08T08:52:51.440 INFO:teuthology.orchestra.run.smithi059.stdout: "old_weight": 0, 2024-05-08T08:52:51.440 INFO:teuthology.orchestra.run.smithi059.stdout: "last_purged_snaps_scrub": "2024-05-08T08:51:52.401080+0000", 2024-05-08T08:52:51.440 INFO:teuthology.orchestra.run.smithi059.stdout: "dead_epoch": 0 2024-05-08T08:52:51.440 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:51.440 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T08:52:51.440 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_upmap": [ 2024-05-08T08:52:51.440 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:51.440 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "3.1d", 2024-05-08T08:52:51.440 INFO:teuthology.orchestra.run.smithi059.stdout: "osds": [ 2024-05-08T08:52:51.440 INFO:teuthology.orchestra.run.smithi059.stdout: 0, 2024-05-08T08:52:51.441 INFO:teuthology.orchestra.run.smithi059.stdout: 5 2024-05-08T08:52:51.441 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:51.441 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:51.441 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T08:52:51.441 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_upmap_items": [], 2024-05-08T08:52:51.441 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_upmap_primaries": [], 2024-05-08T08:52:51.441 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_temp": [], 2024-05-08T08:52:51.441 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_temp": [], 2024-05-08T08:52:51.441 INFO:teuthology.orchestra.run.smithi059.stdout: "blocklist": {}, 2024-05-08T08:52:51.441 INFO:teuthology.orchestra.run.smithi059.stdout: "range_blocklist": {}, 2024-05-08T08:52:51.441 INFO:teuthology.orchestra.run.smithi059.stdout: "erasure_code_profiles": { 2024-05-08T08:52:51.441 INFO:teuthology.orchestra.run.smithi059.stdout: "default": { 2024-05-08T08:52:51.441 INFO:teuthology.orchestra.run.smithi059.stdout: "crush-failure-domain": "osd", 2024-05-08T08:52:51.441 INFO:teuthology.orchestra.run.smithi059.stdout: "k": "2", 2024-05-08T08:52:51.441 INFO:teuthology.orchestra.run.smithi059.stdout: "m": "1", 2024-05-08T08:52:51.442 INFO:teuthology.orchestra.run.smithi059.stdout: "plugin": "jerasure", 2024-05-08T08:52:51.442 INFO:teuthology.orchestra.run.smithi059.stdout: "technique": "reed_sol_van" 2024-05-08T08:52:51.442 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:51.442 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:51.442 INFO:teuthology.orchestra.run.smithi059.stdout: "removed_snaps_queue": [], 2024-05-08T08:52:51.442 INFO:teuthology.orchestra.run.smithi059.stdout: "new_removed_snaps": [], 2024-05-08T08:52:51.442 INFO:teuthology.orchestra.run.smithi059.stdout: "new_purged_snaps": [], 2024-05-08T08:52:51.442 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_node_flags": {}, 2024-05-08T08:52:51.442 INFO:teuthology.orchestra.run.smithi059.stdout: "device_class_flags": {}, 2024-05-08T08:52:51.442 INFO:teuthology.orchestra.run.smithi059.stdout: "stretch_mode": { 2024-05-08T08:52:51.442 INFO:teuthology.orchestra.run.smithi059.stdout: "stretch_mode_enabled": false, 2024-05-08T08:52:51.442 INFO:teuthology.orchestra.run.smithi059.stdout: "stretch_bucket_count": 0, 2024-05-08T08:52:51.442 INFO:teuthology.orchestra.run.smithi059.stdout: "degraded_stretch_mode": 0, 2024-05-08T08:52:51.442 INFO:teuthology.orchestra.run.smithi059.stdout: "recovering_stretch_mode": 0, 2024-05-08T08:52:51.443 INFO:teuthology.orchestra.run.smithi059.stdout: "stretch_mode_bucket": 0 2024-05-08T08:52:51.443 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:51.443 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:52:51.445 INFO:tasks.thrashosds.thrasher:j is {'epoch': 45, 'fsid': '38dfae8c-5c5e-4b0f-a264-91f22c6c5554', 'created': '2024-05-08T08:51:45.571369+0000', 'modified': '2024-05-08T08:52:50.060366+0000', 'last_up_change': '2024-05-08T08:51:56.847706+0000', 'last_in_change': '2024-05-08T08:52:46.045870+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T08:51:57.787790+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '14', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.8781916499137878, 'average_primary_affinity': 0.8799999952316284, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-05-08T08:52:02.563677+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.2699999809265137, 'score_stable': 2.2699999809265137, 'optimal_score': 0.8299999833106995, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.8781916499137878, 'average_primary_affinity': 0.8799999952316284, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-05-08T08:52:09.955467+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 13, 'snap_epoch': 44, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 0.8299999833106995, 'raw_score_acting': 1.559999942779541, 'raw_score_stable': 1.559999942779541, 'primary_affinity_weighted': 0.8781916499137878, 'average_primary_affinity': 0.8799999952316284, 'average_primary_affinity_weighted': 0.8799999952316284}}], 'osds': [{'osd': 0, 'uuid': '300801a5-1919-4b23-8c6d-bc7dad61cc4d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 44, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6800', 'nonce': 802160151}, {'type': 'v1', 'addr': '172.21.15.59:6801', 'nonce': 802160151}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6802', 'nonce': 802160151}, {'type': 'v1', 'addr': '172.21.15.59:6803', 'nonce': 802160151}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6806', 'nonce': 802160151}, {'type': 'v1', 'addr': '172.21.15.59:6807', 'nonce': 802160151}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6804', 'nonce': 802160151}, {'type': 'v1', 'addr': '172.21.15.59:6805', 'nonce': 802160151}]}, 'public_addr': '172.21.15.59:6801/802160151', 'cluster_addr': '172.21.15.59:6803/802160151', 'heartbeat_back_addr': '172.21.15.59:6807/802160151', 'heartbeat_front_addr': '172.21.15.59:6805/802160151', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '0fead0de-6a65-4eba-b400-eea5692702e0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 44, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6816', 'nonce': 4116801011}, {'type': 'v1', 'addr': '172.21.15.59:6817', 'nonce': 4116801011}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6818', 'nonce': 4116801011}, {'type': 'v1', 'addr': '172.21.15.59:6819', 'nonce': 4116801011}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6822', 'nonce': 4116801011}, {'type': 'v1', 'addr': '172.21.15.59:6823', 'nonce': 4116801011}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6820', 'nonce': 4116801011}, {'type': 'v1', 'addr': '172.21.15.59:6821', 'nonce': 4116801011}]}, 'public_addr': '172.21.15.59:6817/4116801011', 'cluster_addr': '172.21.15.59:6819/4116801011', 'heartbeat_back_addr': '172.21.15.59:6823/4116801011', 'heartbeat_front_addr': '172.21.15.59:6821/4116801011', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '1a233de6-1b82-4a51-8e96-4cd4abdf16c8', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 44, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6808', 'nonce': 1547661179}, {'type': 'v1', 'addr': '172.21.15.59:6809', 'nonce': 1547661179}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6810', 'nonce': 1547661179}, {'type': 'v1', 'addr': '172.21.15.59:6811', 'nonce': 1547661179}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6814', 'nonce': 1547661179}, {'type': 'v1', 'addr': '172.21.15.59:6815', 'nonce': 1547661179}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6812', 'nonce': 1547661179}, {'type': 'v1', 'addr': '172.21.15.59:6813', 'nonce': 1547661179}]}, 'public_addr': '172.21.15.59:6809/1547661179', 'cluster_addr': '172.21.15.59:6811/1547661179', 'heartbeat_back_addr': '172.21.15.59:6815/1547661179', 'heartbeat_front_addr': '172.21.15.59:6813/1547661179', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'cc4870b8-dcd9-4e43-978d-8de6c6396abf', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 44, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6800', 'nonce': 4054400810}, {'type': 'v1', 'addr': '172.21.15.192:6801', 'nonce': 4054400810}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6802', 'nonce': 4054400810}, {'type': 'v1', 'addr': '172.21.15.192:6803', 'nonce': 4054400810}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6806', 'nonce': 4054400810}, {'type': 'v1', 'addr': '172.21.15.192:6807', 'nonce': 4054400810}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6804', 'nonce': 4054400810}, {'type': 'v1', 'addr': '172.21.15.192:6805', 'nonce': 4054400810}]}, 'public_addr': '172.21.15.192:6801/4054400810', 'cluster_addr': '172.21.15.192:6803/4054400810', 'heartbeat_back_addr': '172.21.15.192:6807/4054400810', 'heartbeat_front_addr': '172.21.15.192:6805/4054400810', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'd1d05620-42fb-4107-a2f9-e83542d076ee', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 0.9892578125, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 43, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6808', 'nonce': 58985565}, {'type': 'v1', 'addr': '172.21.15.192:6809', 'nonce': 58985565}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6810', 'nonce': 58985565}, {'type': 'v1', 'addr': '172.21.15.192:6811', 'nonce': 58985565}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6814', 'nonce': 58985565}, {'type': 'v1', 'addr': '172.21.15.192:6815', 'nonce': 58985565}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6812', 'nonce': 58985565}, {'type': 'v1', 'addr': '172.21.15.192:6813', 'nonce': 58985565}]}, 'public_addr': '172.21.15.192:6809/58985565', 'cluster_addr': '172.21.15.192:6811/58985565', 'heartbeat_back_addr': '172.21.15.192:6815/58985565', 'heartbeat_front_addr': '172.21.15.192:6813/58985565', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'ab26136c-31c3-4090-9fa2-988dba4f731f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.2798919677734375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 18, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6816', 'nonce': 1283571573}, {'type': 'v1', 'addr': '172.21.15.192:6817', 'nonce': 1283571573}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6818', 'nonce': 1283571573}, {'type': 'v1', 'addr': '172.21.15.192:6819', 'nonce': 1283571573}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6822', 'nonce': 1283571573}, {'type': 'v1', 'addr': '172.21.15.192:6823', 'nonce': 1283571573}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6820', 'nonce': 1283571573}, {'type': 'v1', 'addr': '172.21.15.192:6821', 'nonce': 1283571573}]}, 'public_addr': '172.21.15.192:6817/1283571573', 'cluster_addr': '172.21.15.192:6819/1283571573', 'heartbeat_back_addr': '172.21.15.192:6823/1283571573', 'heartbeat_front_addr': '172.21.15.192:6821/1283571573', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:51:52.396946+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:51:52.453160+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:51:52.387606+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:51:52.360952+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-05-08T08:51:52.468681+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:51:52.401080+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.1d', 'osds': [0, 5]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-05-08T08:52:51.445 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T08:52:51.445 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:52:51.447+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:52:51.447 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:52:51.447 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:52:51.447 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:52:51.447 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:52:51.447 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.447 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4742 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1832960~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:51.447 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.585513+0000", 2024-05-08T08:52:51.447 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 19.849215863000001, 2024-05-08T08:52:51.447 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035588763000000002, 2024-05-08T08:52:51.447 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:51.448 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:51.448 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:51.448 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:51.448 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:51.448 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:51.448 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4742 2024-05-08T08:52:51.448 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.448 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:51.448 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.448 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:51.448 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585513+0000", 2024-05-08T08:52:51.448 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:51.448 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.448 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.448 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:51.449 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585513+0000", 2024-05-08T08:52:51.449 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:51.449 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.449 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.449 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:51.449 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585514+0000", 2024-05-08T08:52:51.449 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6100000000000001e-07 2024-05-08T08:52:51.449 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.449 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.449 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:51.449 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585516+0000", 2024-05-08T08:52:51.449 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.526e-06 2024-05-08T08:52:51.449 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.449 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.449 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:51.450 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585523+0000", 2024-05-08T08:52:51.450 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.562e-06 2024-05-08T08:52:51.450 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.450 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.450 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:51.450 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585528+0000", 2024-05-08T08:52:51.450 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7990000000000001e-06 2024-05-08T08:52:51.450 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.450 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.450 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:51.450 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.620939+0000", 2024-05-08T08:52:51.450 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035411419 2024-05-08T08:52:51.450 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.451 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.451 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:51.451 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621014+0000", 2024-05-08T08:52:51.451 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5017000000000004e-05 2024-05-08T08:52:51.451 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.451 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.451 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:51.451 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621102+0000", 2024-05-08T08:52:51.451 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7979000000000006e-05 2024-05-08T08:52:51.451 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:51.451 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:51.451 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:51.451 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.451 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.451 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4743 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1849344~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:51.452 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.585628+0000", 2024-05-08T08:52:51.452 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 19.849100975999999, 2024-05-08T08:52:51.452 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035659962000000003, 2024-05-08T08:52:51.452 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:51.452 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:51.452 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:51.452 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:51.452 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:51.452 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:51.452 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4743 2024-05-08T08:52:51.452 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.452 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:51.452 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.452 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:51.453 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585628+0000", 2024-05-08T08:52:51.453 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:51.453 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.453 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.453 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:51.453 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585628+0000", 2024-05-08T08:52:51.453 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:51.453 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.453 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.453 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:51.453 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585629+0000", 2024-05-08T08:52:51.453 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2300000000000001e-07 2024-05-08T08:52:51.453 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.453 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.453 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:51.454 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585632+0000", 2024-05-08T08:52:51.454 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1370000000000002e-06 2024-05-08T08:52:51.454 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.454 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.454 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:51.454 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585641+0000", 2024-05-08T08:52:51.454 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5220000000000007e-06 2024-05-08T08:52:51.454 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.454 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.454 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:51.454 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585646+0000", 2024-05-08T08:52:51.454 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.609e-06 2024-05-08T08:52:51.454 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.454 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.455 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:51.455 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621132+0000", 2024-05-08T08:52:51.455 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035485472999999997 2024-05-08T08:52:51.455 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.455 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.455 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:51.455 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621204+0000", 2024-05-08T08:52:51.455 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2259000000000001e-05 2024-05-08T08:52:51.455 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.455 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.455 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:51.455 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621288+0000", 2024-05-08T08:52:51.455 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4338999999999993e-05 2024-05-08T08:52:51.455 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:51.455 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:51.455 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:51.456 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.456 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.456 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4746 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1865728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:51.456 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.585656+0000", 2024-05-08T08:52:51.456 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 19.849073294, 2024-05-08T08:52:51.456 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035815909, 2024-05-08T08:52:51.456 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:51.456 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:51.456 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:51.456 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:51.456 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:51.456 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:51.456 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4746 2024-05-08T08:52:51.456 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.456 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:51.457 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.457 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:51.457 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585656+0000", 2024-05-08T08:52:51.457 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:51.457 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.457 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.457 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:51.457 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585656+0000", 2024-05-08T08:52:51.457 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:51.457 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.457 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.457 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:51.457 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585656+0000", 2024-05-08T08:52:51.457 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.08e-07 2024-05-08T08:52:51.457 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.458 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.458 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:51.458 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585658+0000", 2024-05-08T08:52:51.458 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0889999999999998e-06 2024-05-08T08:52:51.458 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.458 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.458 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:51.458 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585664+0000", 2024-05-08T08:52:51.458 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3990000000000003e-06 2024-05-08T08:52:51.458 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.458 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.458 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:51.462 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585668+0000", 2024-05-08T08:52:51.462 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3170000000000003e-06 2024-05-08T08:52:51.462 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.462 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.462 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:51.462 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621316+0000", 2024-05-08T08:52:51.462 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035647908999999998 2024-05-08T08:52:51.462 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.462 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.462 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:51.462 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621389+0000", 2024-05-08T08:52:51.463 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2531000000000001e-05 2024-05-08T08:52:51.463 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.463 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.463 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:51.463 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621472+0000", 2024-05-08T08:52:51.463 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3256000000000004e-05 2024-05-08T08:52:51.463 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:51.463 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:51.463 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:51.463 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.463 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.463 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4748 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1882112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:51.463 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.585697+0000", 2024-05-08T08:52:51.463 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 19.849031847999999, 2024-05-08T08:52:51.463 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035962485000000002, 2024-05-08T08:52:51.464 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:51.464 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:51.464 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:51.464 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:51.464 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:51.464 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:51.464 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4748 2024-05-08T08:52:51.464 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.464 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:51.464 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.464 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:51.464 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585697+0000", 2024-05-08T08:52:51.464 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:51.464 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.464 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.465 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:51.465 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585697+0000", 2024-05-08T08:52:51.465 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:51.465 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.465 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.465 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:51.465 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585698+0000", 2024-05-08T08:52:51.465 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0900000000000001e-07 2024-05-08T08:52:51.465 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.465 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.465 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:51.465 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585701+0000", 2024-05-08T08:52:51.465 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0149999999999999e-06 2024-05-08T08:52:51.465 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.465 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.466 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:51.466 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585712+0000", 2024-05-08T08:52:51.466 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0655e-05 2024-05-08T08:52:51.466 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.466 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.466 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:51.466 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585717+0000", 2024-05-08T08:52:51.466 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3800000000000002e-06 2024-05-08T08:52:51.466 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.466 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.466 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:51.466 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621502+0000", 2024-05-08T08:52:51.466 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035784559000000001 2024-05-08T08:52:51.467 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.467 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.467 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:51.467 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621575+0000", 2024-05-08T08:52:51.467 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3269999999999995e-05 2024-05-08T08:52:51.467 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.467 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.467 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:51.467 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621660+0000", 2024-05-08T08:52:51.467 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4997e-05 2024-05-08T08:52:51.467 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:51.467 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:51.467 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:51.467 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.467 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.468 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4750 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1898496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:51.468 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.585810+0000", 2024-05-08T08:52:51.468 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 19.848919018, 2024-05-08T08:52:51.468 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036034928000000001, 2024-05-08T08:52:51.468 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:51.468 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:51.468 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:51.468 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:51.468 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:51.468 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:51.468 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4750 2024-05-08T08:52:51.468 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.468 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:51.468 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.469 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:51.469 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585810+0000", 2024-05-08T08:52:51.469 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:51.469 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.469 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.469 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:51.469 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585810+0000", 2024-05-08T08:52:51.469 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:51.469 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.469 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.469 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:51.469 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585811+0000", 2024-05-08T08:52:51.469 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6499999999999999e-07 2024-05-08T08:52:51.469 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.469 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.470 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:51.470 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585814+0000", 2024-05-08T08:52:51.470 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1709999999999998e-06 2024-05-08T08:52:51.470 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.470 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.470 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:51.470 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585822+0000", 2024-05-08T08:52:51.470 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8909999999999995e-06 2024-05-08T08:52:51.470 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.470 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.470 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:51.470 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585827+0000", 2024-05-08T08:52:51.471 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5289999999999999e-06 2024-05-08T08:52:51.471 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.471 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.471 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:51.471 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621688+0000", 2024-05-08T08:52:51.471 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035861111000000001 2024-05-08T08:52:51.471 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.471 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.471 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:51.471 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621761+0000", 2024-05-08T08:52:51.471 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2211000000000004e-05 2024-05-08T08:52:51.471 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.471 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.471 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:51.471 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621845+0000", 2024-05-08T08:52:51.471 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4450000000000006e-05 2024-05-08T08:52:51.472 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:51.472 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:51.472 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:51.472 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.472 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.472 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4752 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1914880~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:51.472 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.585858+0000", 2024-05-08T08:52:51.472 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 19.848871049, 2024-05-08T08:52:51.472 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036180081000000003, 2024-05-08T08:52:51.472 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:51.472 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:51.472 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:51.472 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:51.472 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:51.473 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:51.473 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4752 2024-05-08T08:52:51.473 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.473 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:51.473 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.473 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:51.473 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585858+0000", 2024-05-08T08:52:51.473 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:51.473 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.473 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.473 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:51.473 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585858+0000", 2024-05-08T08:52:51.473 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:51.473 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.473 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.474 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:51.474 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585859+0000", 2024-05-08T08:52:51.474 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.82e-07 2024-05-08T08:52:51.474 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.474 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.474 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:51.474 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585861+0000", 2024-05-08T08:52:51.474 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.695e-06 2024-05-08T08:52:51.474 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.474 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.474 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:51.474 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585869+0000", 2024-05-08T08:52:51.474 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0299999999999994e-06 2024-05-08T08:52:51.474 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.475 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.475 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:51.475 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585874+0000", 2024-05-08T08:52:51.475 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9620000000000001e-06 2024-05-08T08:52:51.475 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.475 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.475 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:51.475 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621878+0000", 2024-05-08T08:52:51.475 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036004043999999999 2024-05-08T08:52:51.475 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.475 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.475 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:51.475 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621954+0000", 2024-05-08T08:52:51.475 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5414000000000007e-05 2024-05-08T08:52:51.475 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.475 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.476 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:51.476 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622038+0000", 2024-05-08T08:52:51.476 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4454000000000002e-05 2024-05-08T08:52:51.476 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:51.476 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:51.476 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:51.476 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.476 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.476 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4753 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1931264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:51.476 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.585947+0000", 2024-05-08T08:52:51.476 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 19.848782401000001, 2024-05-08T08:52:51.476 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036267992999999998, 2024-05-08T08:52:51.476 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:51.476 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:51.476 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:51.477 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:51.477 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:51.477 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:51.477 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4753 2024-05-08T08:52:51.477 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.477 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:51.477 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.477 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:51.477 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585947+0000", 2024-05-08T08:52:51.477 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:51.477 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.477 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.477 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:51.477 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585947+0000", 2024-05-08T08:52:51.478 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:51.478 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.478 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.478 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:51.478 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585948+0000", 2024-05-08T08:52:51.478 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6000000000000002e-07 2024-05-08T08:52:51.478 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.478 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.478 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:51.478 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585952+0000", 2024-05-08T08:52:51.478 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7090000000000001e-06 2024-05-08T08:52:51.478 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.479 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.479 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:51.479 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585963+0000", 2024-05-08T08:52:51.479 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0740000000000001e-05 2024-05-08T08:52:51.479 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.479 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.479 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:51.479 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585970+0000", 2024-05-08T08:52:51.479 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3309999999999998e-06 2024-05-08T08:52:51.479 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.479 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.479 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:51.479 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622065+0000", 2024-05-08T08:52:51.479 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036094676999999999 2024-05-08T08:52:51.479 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.480 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.480 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:51.480 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622132+0000", 2024-05-08T08:52:51.480 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7214999999999999e-05 2024-05-08T08:52:51.480 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.480 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.480 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:51.480 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622215+0000", 2024-05-08T08:52:51.480 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2460999999999995e-05 2024-05-08T08:52:51.480 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:51.480 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:51.480 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:51.480 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.480 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.480 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4756 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1947648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:51.481 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586018+0000", 2024-05-08T08:52:51.481 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 19.848710756999999, 2024-05-08T08:52:51.481 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036382335000000002, 2024-05-08T08:52:51.481 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:51.481 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:51.481 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:51.481 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:51.481 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:51.481 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:51.481 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4756 2024-05-08T08:52:51.481 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.481 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:51.481 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.481 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:51.481 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586018+0000", 2024-05-08T08:52:51.482 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:51.482 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.482 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.482 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:51.482 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586018+0000", 2024-05-08T08:52:51.482 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:51.482 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.482 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.482 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:51.482 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586019+0000", 2024-05-08T08:52:51.482 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9999999999999997e-07 2024-05-08T08:52:51.482 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.482 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.482 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:51.482 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586023+0000", 2024-05-08T08:52:51.483 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7330000000000001e-06 2024-05-08T08:52:51.483 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.483 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.483 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:51.483 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586031+0000", 2024-05-08T08:52:51.483 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4940000000000003e-06 2024-05-08T08:52:51.483 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.483 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.483 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:51.483 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586038+0000", 2024-05-08T08:52:51.483 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2539999999999998e-06 2024-05-08T08:52:51.483 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.483 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.483 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:51.483 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622243+0000", 2024-05-08T08:52:51.484 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036205009000000003 2024-05-08T08:52:51.484 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.484 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.484 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:51.484 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622313+0000", 2024-05-08T08:52:51.484 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9912000000000004e-05 2024-05-08T08:52:51.484 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.484 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.484 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:51.484 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622401+0000", 2024-05-08T08:52:51.484 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8232999999999994e-05 2024-05-08T08:52:51.484 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:51.484 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:51.484 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:51.484 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.485 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.485 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4758 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1964032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:51.485 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586080+0000", 2024-05-08T08:52:51.485 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 19.848649338000001, 2024-05-08T08:52:51.485 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036547659000000003, 2024-05-08T08:52:51.485 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:51.485 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:51.485 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:51.485 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:51.485 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:51.485 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:51.485 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4758 2024-05-08T08:52:51.485 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.485 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:51.485 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.485 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:51.486 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586080+0000", 2024-05-08T08:52:51.486 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:51.486 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.486 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.486 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:51.486 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586080+0000", 2024-05-08T08:52:51.486 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:51.486 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.486 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.486 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:51.486 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586081+0000", 2024-05-08T08:52:51.486 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5499999999999998e-07 2024-05-08T08:52:51.486 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.486 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.486 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:51.487 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586084+0000", 2024-05-08T08:52:51.487 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9919999999999999e-06 2024-05-08T08:52:51.487 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.487 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.487 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:51.487 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586091+0000", 2024-05-08T08:52:51.487 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5440000000000001e-06 2024-05-08T08:52:51.487 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.487 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.487 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:51.487 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586098+0000", 2024-05-08T08:52:51.487 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8639999999999998e-06 2024-05-08T08:52:51.487 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.487 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.487 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:51.487 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622442+0000", 2024-05-08T08:52:51.488 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036344110999999998 2024-05-08T08:52:51.488 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.488 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.488 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:51.488 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622516+0000", 2024-05-08T08:52:51.488 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4089000000000002e-05 2024-05-08T08:52:51.488 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.488 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.488 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:51.488 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622628+0000", 2024-05-08T08:52:51.488 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00011140400000000001 2024-05-08T08:52:51.488 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:51.488 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:51.488 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:51.488 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.489 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.489 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4760 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1980416~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:51.489 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586178+0000", 2024-05-08T08:52:51.489 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 19.848550725999999, 2024-05-08T08:52:51.489 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036647526999999999, 2024-05-08T08:52:51.489 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:51.489 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:51.489 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:51.489 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:51.489 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:51.489 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:51.489 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4760 2024-05-08T08:52:51.489 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.489 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:51.489 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.490 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:51.490 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586178+0000", 2024-05-08T08:52:51.490 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:51.490 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.490 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.490 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:51.490 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586178+0000", 2024-05-08T08:52:51.490 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:51.490 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.490 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.490 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:51.490 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586179+0000", 2024-05-08T08:52:51.490 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8599999999999997e-07 2024-05-08T08:52:51.490 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:51.491 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586182+0000", 2024-05-08T08:52:51.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.174e-06 2024-05-08T08:52:51.491 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:51.491 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586191+0000", 2024-05-08T08:52:51.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1869999999999997e-06 2024-05-08T08:52:51.491 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:51.491 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586197+0000", 2024-05-08T08:52:51.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2090000000000002e-06 2024-05-08T08:52:51.491 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.492 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:51.492 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622657+0000", 2024-05-08T08:52:51.492 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036459793999999997 2024-05-08T08:52:51.492 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.492 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.492 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:51.492 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622725+0000", 2024-05-08T08:52:51.492 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8374999999999995e-05 2024-05-08T08:52:51.492 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.492 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.492 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:51.492 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622826+0000", 2024-05-08T08:52:51.492 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010100200000000001 2024-05-08T08:52:51.492 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:51.493 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:51.493 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:51.493 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.493 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.493 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4762 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1996800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:51.493 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586299+0000", 2024-05-08T08:52:51.493 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 19.848430395000001, 2024-05-08T08:52:51.493 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036720653999999998, 2024-05-08T08:52:51.493 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:51.493 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:51.493 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:51.493 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:51.493 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:51.493 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:51.493 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4762 2024-05-08T08:52:51.493 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.494 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:51.494 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.494 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:51.494 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586299+0000", 2024-05-08T08:52:51.494 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:51.494 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.494 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.494 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:51.494 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586299+0000", 2024-05-08T08:52:51.494 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:51.494 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.494 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.494 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:51.494 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586299+0000", 2024-05-08T08:52:51.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.99e-07 2024-05-08T08:52:51.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.495 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:51.495 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586303+0000", 2024-05-08T08:52:51.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1949999999999999e-06 2024-05-08T08:52:51.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.495 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:51.495 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586318+0000", 2024-05-08T08:52:51.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5401999999999999e-05 2024-05-08T08:52:51.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.495 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:51.495 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586324+0000", 2024-05-08T08:52:51.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8119999999999996e-06 2024-05-08T08:52:51.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.496 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:51.496 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622853+0000", 2024-05-08T08:52:51.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036529013999999999 2024-05-08T08:52:51.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.496 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:51.496 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622924+0000", 2024-05-08T08:52:51.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1654000000000002e-05 2024-05-08T08:52:51.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.496 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:51.496 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623019+0000", 2024-05-08T08:52:51.497 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4977999999999998e-05 2024-05-08T08:52:51.497 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:51.497 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:51.497 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:51.497 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.497 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.497 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4764 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 2013184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:51.497 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586506+0000", 2024-05-08T08:52:51.497 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 19.84822329, 2024-05-08T08:52:51.497 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036859467, 2024-05-08T08:52:51.497 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:51.497 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:51.497 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:51.497 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:51.497 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:51.497 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:51.498 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4764 2024-05-08T08:52:51.498 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.498 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:51.498 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.498 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:51.498 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586506+0000", 2024-05-08T08:52:51.498 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:51.498 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.498 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.498 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:51.498 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586506+0000", 2024-05-08T08:52:51.498 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:51.498 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.498 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.499 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:51.499 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586507+0000", 2024-05-08T08:52:51.499 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9100000000000003e-07 2024-05-08T08:52:51.499 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.499 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.499 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:51.499 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586512+0000", 2024-05-08T08:52:51.499 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9470000000000002e-06 2024-05-08T08:52:51.499 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.499 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.499 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:51.499 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586521+0000", 2024-05-08T08:52:51.499 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3379999999999994e-06 2024-05-08T08:52:51.500 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.500 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.500 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:51.500 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586528+0000", 2024-05-08T08:52:51.500 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1289999999999998e-06 2024-05-08T08:52:51.500 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.500 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.500 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:51.500 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623095+0000", 2024-05-08T08:52:51.500 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036566411 2024-05-08T08:52:51.500 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.500 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.500 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:51.500 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623221+0000", 2024-05-08T08:52:51.501 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012622299999999999 2024-05-08T08:52:51.501 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.501 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.501 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:51.501 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623365+0000", 2024-05-08T08:52:51.501 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000144628 2024-05-08T08:52:51.501 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:51.501 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:51.501 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:51.501 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.501 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.501 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4766 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 2029568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:51.501 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586543+0000", 2024-05-08T08:52:51.501 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 19.848185803, 2024-05-08T08:52:51.501 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.037033635000000002, 2024-05-08T08:52:51.502 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:51.502 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:51.502 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:51.502 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:51.502 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:51.502 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:51.502 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4766 2024-05-08T08:52:51.502 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.502 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:51.502 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.502 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:51.502 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586543+0000", 2024-05-08T08:52:51.502 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:51.502 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.503 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:51.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586543+0000", 2024-05-08T08:52:51.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:51.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.503 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:51.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586544+0000", 2024-05-08T08:52:51.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3099999999999998e-07 2024-05-08T08:52:51.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.503 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:51.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586547+0000", 2024-05-08T08:52:51.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.751e-06 2024-05-08T08:52:51.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.504 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.504 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:51.504 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586553+0000", 2024-05-08T08:52:51.504 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2870000000000001e-06 2024-05-08T08:52:51.504 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.504 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.504 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:51.504 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586558+0000", 2024-05-08T08:52:51.504 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5709999999999997e-06 2024-05-08T08:52:51.504 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.504 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.504 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:51.504 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623416+0000", 2024-05-08T08:52:51.504 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036857613999999997 2024-05-08T08:52:51.504 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.505 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.505 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:51.505 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623489+0000", 2024-05-08T08:52:51.505 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3172999999999999e-05 2024-05-08T08:52:51.505 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.505 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.505 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:51.505 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623577+0000", 2024-05-08T08:52:51.505 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7807999999999997e-05 2024-05-08T08:52:51.505 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:51.505 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:51.505 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:51.505 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.505 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.506 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4768 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 2045952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:51.506 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586566+0000", 2024-05-08T08:52:51.506 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 19.848163497000002, 2024-05-08T08:52:51.506 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.03726206, 2024-05-08T08:52:51.506 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:51.506 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:51.506 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:51.506 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:51.506 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:51.506 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:51.506 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4768 2024-05-08T08:52:51.507 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.507 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:51.507 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.507 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:51.507 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586566+0000", 2024-05-08T08:52:51.507 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:51.507 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.507 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.507 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:51.507 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586566+0000", 2024-05-08T08:52:51.507 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:51.507 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.507 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.507 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:51.507 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586566+0000", 2024-05-08T08:52:51.508 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8299999999999998e-07 2024-05-08T08:52:51.508 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.508 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.508 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:51.508 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586568+0000", 2024-05-08T08:52:51.508 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.063e-06 2024-05-08T08:52:51.508 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.508 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.508 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:51.508 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586574+0000", 2024-05-08T08:52:51.508 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8300000000000001e-06 2024-05-08T08:52:51.508 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.508 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.508 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:51.509 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586579+0000", 2024-05-08T08:52:51.509 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7559999999999999e-06 2024-05-08T08:52:51.509 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.509 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.509 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:51.509 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623612+0000", 2024-05-08T08:52:51.509 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.037033041000000003 2024-05-08T08:52:51.509 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.509 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.509 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:51.509 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623699+0000", 2024-05-08T08:52:51.509 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6817000000000006e-05 2024-05-08T08:52:51.509 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.509 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.509 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:51.510 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623828+0000", 2024-05-08T08:52:51.510 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012917000000000001 2024-05-08T08:52:51.510 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:51.510 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:51.510 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:51.510 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.510 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.510 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4770 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 2062336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:51.510 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586585+0000", 2024-05-08T08:52:51.510 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 19.848144313999999, 2024-05-08T08:52:51.510 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.037449891999999999, 2024-05-08T08:52:51.510 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:51.510 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:51.510 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:51.511 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:51.511 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:51.511 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:51.511 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4770 2024-05-08T08:52:51.511 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.511 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:51.511 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.511 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:51.511 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586585+0000", 2024-05-08T08:52:51.511 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:51.511 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.511 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.511 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:51.511 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586585+0000", 2024-05-08T08:52:51.511 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:51.512 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.512 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.512 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:51.512 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586585+0000", 2024-05-08T08:52:51.512 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:52:51.512 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.512 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.512 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:51.512 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586587+0000", 2024-05-08T08:52:51.512 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.181e-06 2024-05-08T08:52:51.512 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.512 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.512 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:51.512 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586592+0000", 2024-05-08T08:52:51.512 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9710000000000003e-06 2024-05-08T08:52:51.513 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.513 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.513 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:51.513 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586598+0000", 2024-05-08T08:52:51.513 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.118e-06 2024-05-08T08:52:51.513 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.513 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.513 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:51.513 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623862+0000", 2024-05-08T08:52:51.513 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.037264711999999998 2024-05-08T08:52:51.513 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.513 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.513 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:51.513 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623941+0000", 2024-05-08T08:52:51.514 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9084000000000004e-05 2024-05-08T08:52:51.514 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.514 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.514 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:51.514 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.624035+0000", 2024-05-08T08:52:51.514 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3456999999999996e-05 2024-05-08T08:52:51.514 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:51.514 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:51.514 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:51.514 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.514 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.514 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4772 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 2078720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:51.514 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586621+0000", 2024-05-08T08:52:51.514 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 19.848108365999998, 2024-05-08T08:52:51.514 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.037650391999999998, 2024-05-08T08:52:51.515 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:51.515 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:51.515 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:51.515 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:51.515 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:51.515 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:51.515 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4772 2024-05-08T08:52:51.515 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.515 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:51.515 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.515 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:51.515 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586621+0000", 2024-05-08T08:52:51.515 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:51.515 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.515 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.515 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:51.516 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586621+0000", 2024-05-08T08:52:51.516 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:51.516 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.516 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.516 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:51.516 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586621+0000", 2024-05-08T08:52:51.516 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:52:51.516 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.516 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.516 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:51.516 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586623+0000", 2024-05-08T08:52:51.516 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9240000000000001e-06 2024-05-08T08:52:51.516 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.516 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.516 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:51.517 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586628+0000", 2024-05-08T08:52:51.517 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6410000000000001e-06 2024-05-08T08:52:51.517 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.517 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.517 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:51.517 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586633+0000", 2024-05-08T08:52:51.517 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.763e-06 2024-05-08T08:52:51.517 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.517 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.517 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:51.517 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.624077+0000", 2024-05-08T08:52:51.517 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.037444709999999999 2024-05-08T08:52:51.517 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.517 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.518 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:51.518 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.624164+0000", 2024-05-08T08:52:51.518 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6769999999999998e-05 2024-05-08T08:52:51.518 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.518 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.518 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:51.518 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.624271+0000", 2024-05-08T08:52:51.518 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010723 2024-05-08T08:52:51.518 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:51.518 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:51.518 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:51.518 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.518 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.519 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4774 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 2095104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:51.519 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586640+0000", 2024-05-08T08:52:51.519 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 19.848089227999999, 2024-05-08T08:52:51.519 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.037827178000000003, 2024-05-08T08:52:51.519 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:51.519 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:51.519 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:51.519 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:51.519 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:51.519 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:51.519 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4774 2024-05-08T08:52:51.519 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.519 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:51.519 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.519 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:51.520 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586640+0000", 2024-05-08T08:52:51.520 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:51.520 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.520 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.520 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:51.520 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586640+0000", 2024-05-08T08:52:51.520 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:51.520 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.520 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.520 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:51.520 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586640+0000", 2024-05-08T08:52:51.520 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5199999999999998e-07 2024-05-08T08:52:51.520 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.520 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.520 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:51.521 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586643+0000", 2024-05-08T08:52:51.521 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.232e-06 2024-05-08T08:52:51.521 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.521 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.521 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:51.521 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586649+0000", 2024-05-08T08:52:51.521 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9660000000000001e-06 2024-05-08T08:52:51.521 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.521 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.521 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:51.521 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586653+0000", 2024-05-08T08:52:51.521 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7350000000000004e-06 2024-05-08T08:52:51.521 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.521 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.521 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:51.522 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.624303+0000", 2024-05-08T08:52:51.522 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.037649647000000001 2024-05-08T08:52:51.522 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.522 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.522 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:51.522 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.624373+0000", 2024-05-08T08:52:51.522 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0415999999999998e-05 2024-05-08T08:52:51.522 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.522 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.522 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:51.522 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.624467+0000", 2024-05-08T08:52:51.522 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3830000000000001e-05 2024-05-08T08:52:51.522 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:51.522 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:51.523 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:51.523 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.523 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.523 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4621.0:9089 3.1f e33/18 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:head v 33'117, mlcod=31'115)", 2024-05-08T08:52:51.523 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:52:51.523 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.915111016000001, 2024-05-08T08:52:51.523 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.111244524, 2024-05-08T08:52:51.523 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:51.523 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:51.523 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:51.523 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:51.523 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.523 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:51.523 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:52:51.524 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:51.524 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.524 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.524 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:51.524 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:52:51.524 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:51.524 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.524 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.524 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:51.524 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:52:51.524 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5900000000000001e-07 2024-05-08T08:52:51.524 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.524 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.524 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:51.524 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539600+0000", 2024-05-08T08:52:51.525 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.019981364000000001 2024-05-08T08:52:51.525 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.525 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.525 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:51.525 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539624+0000", 2024-05-08T08:52:51.525 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4043e-05 2024-05-08T08:52:51.525 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.525 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.525 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:51.525 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539635+0000", 2024-05-08T08:52:51.525 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1511e-05 2024-05-08T08:52:51.525 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.525 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.525 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:51.525 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539728+0000", 2024-05-08T08:52:51.525 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2498999999999996e-05 2024-05-08T08:52:51.526 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.526 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.526 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:51.526 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539746+0000", 2024-05-08T08:52:51.526 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8289000000000001e-05 2024-05-08T08:52:51.526 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.526 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.526 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:51.526 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.630822+0000", 2024-05-08T08:52:51.526 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.091076283999999993 2024-05-08T08:52:51.526 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.526 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.526 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:51.526 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.630863+0000", 2024-05-08T08:52:51.527 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0374999999999999e-05 2024-05-08T08:52:51.527 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:51.527 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:51.527 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:51.527 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.527 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.527 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4795.0:230 3.1f e40/18 3:f8db1a56:::rbd_data.11402b26385.0000000000000000:head v 40'171, mlcod=39'168)", 2024-05-08T08:52:51.527 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:45.927773+0000", 2024-05-08T08:52:51.527 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.5069559330000004, 2024-05-08T08:52:51.527 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045892405999999997, 2024-05-08T08:52:51.527 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:51.527 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:51.527 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:51.527 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:51.527 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.528 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:51.528 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.927773+0000", 2024-05-08T08:52:51.528 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:51.528 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.528 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.528 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:51.528 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.927773+0000", 2024-05-08T08:52:51.528 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:51.528 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.528 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.528 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:51.528 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.927774+0000", 2024-05-08T08:52:51.528 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0500000000000002e-07 2024-05-08T08:52:51.528 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.529 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.529 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:51.529 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.931581+0000", 2024-05-08T08:52:51.529 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0038071799999999999 2024-05-08T08:52:51.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.529 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.529 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:51.529 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.931599+0000", 2024-05-08T08:52:51.529 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7479e-05 2024-05-08T08:52:51.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.529 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.529 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:51.529 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.931624+0000", 2024-05-08T08:52:51.529 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5372999999999999e-05 2024-05-08T08:52:51.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.529 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.530 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:51.530 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.931698+0000", 2024-05-08T08:52:51.530 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4043999999999998e-05 2024-05-08T08:52:51.530 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.530 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.530 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:51.530 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.931725+0000", 2024-05-08T08:52:51.530 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6883e-05 2024-05-08T08:52:51.530 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.530 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.530 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:51.530 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.973623+0000", 2024-05-08T08:52:51.530 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.041898283000000001 2024-05-08T08:52:51.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.531 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.531 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:51.531 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.973666+0000", 2024-05-08T08:52:51.531 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2358999999999999e-05 2024-05-08T08:52:51.531 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:51.531 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:51.531 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:51.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.531 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.531 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4195.0:4 3.11 3:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-05-08T08:52:51.531 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:49.764654+0000", 2024-05-08T08:52:51.531 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 1.670075475, 2024-05-08T08:52:51.531 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.30714275499999999, 2024-05-08T08:52:51.531 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:51.532 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:51.532 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-05-08T08:52:51.532 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:51.532 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4195", 2024-05-08T08:52:51.532 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/59179521", 2024-05-08T08:52:51.532 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4 2024-05-08T08:52:51.532 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.532 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:51.532 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.532 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:51.532 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764654+0000", 2024-05-08T08:52:51.532 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:51.532 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.532 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.533 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:51.533 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764654+0000", 2024-05-08T08:52:51.533 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:51.533 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.533 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.533 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:51.533 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764655+0000", 2024-05-08T08:52:51.533 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.452e-06 2024-05-08T08:52:51.533 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.533 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.533 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:51.533 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764661+0000", 2024-05-08T08:52:51.533 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5369999999999997e-06 2024-05-08T08:52:51.533 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.533 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.534 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:51.534 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764671+0000", 2024-05-08T08:52:51.534 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.028e-05 2024-05-08T08:52:51.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.534 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.534 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:51.534 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764680+0000", 2024-05-08T08:52:51.534 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0629999999999995e-06 2024-05-08T08:52:51.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.534 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.534 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:51.534 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764753+0000", 2024-05-08T08:52:51.534 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2679e-05 2024-05-08T08:52:51.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.534 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.535 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-05-08T08:52:51.535 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764765+0000", 2024-05-08T08:52:51.535 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2415000000000001e-05 2024-05-08T08:52:51.535 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.535 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.535 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:51.535 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:50.071692+0000", 2024-05-08T08:52:51.535 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.306927324 2024-05-08T08:52:51.535 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:51.535 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:51.535 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:51.535 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:50.071796+0000", 2024-05-08T08:52:51.535 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104005 2024-05-08T08:52:51.535 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:51.536 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:51.536 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:51.536 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:51.536 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:51.536 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:52:51.536 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:52:51.537 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:52:51.538 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:52:51.538 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:52:51.538 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:52:51.539 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:52:51.539 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:52:51.541 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:52:51.542 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:52:51.560 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:52:51.561 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:52:51.578 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:52:51.579 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:52:51.583 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:51.586+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:51.645 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:51.645 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:51.646 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:52:51.649+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:52:51.654 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:52:51.746 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:52:51.749+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:52:51.801 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:51.801 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:51.809 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:52:51.837 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:52:51.838 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-05-08T08:52:51.849 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":67,"stamp":"2024-05-08T08:52:51.750502+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":438758371,"num_objects":1223,"num_object_clones":141,"num_object_copies":2446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1223,"num_whiteouts":896,"num_read":131655,"num_read_kb":406458,"num_write":7312,"num_write_kb":170208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":71,"num_bytes_recovered":2516559,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6881,"ondisk_log_size":6881,"up":82,"acting":73,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":73,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":471859200,"kb_used":202564,"kb_used_data":21684,"kb_used_omap":7,"kb_used_meta":180792,"kb_avail":471656636,"statfs":{"total":483183820800,"available":482976395264,"internally_reserved":0,"allocated":22204416,"data_stored":151605261,"data_compressed":5033463,"data_compressed_allocated":13688832,"data_compressed_original":146698240,"omap_allocated":7973,"internal_metadata":185131227},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":92685908,"num_objects":46,"num_object_clones":18,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":13,"num_read":12019,"num_read_kb":59415,"num_write":1826,"num_write_kb":16707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":2343424,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.699515"},"pg_stats":[{"pgid":"3.1f","version":"41'174","reported_seq":11252,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-05-08T08:52:47.053629+0000","last_change":"2024-05-08T08:52:13.934602+0000","last_active":"2024-05-08T08:52:47.053629+0000","last_peered":"2024-05-08T08:52:47.053629+0000","last_clean":"2024-05-08T08:52:47.053629+0000","last_became_active":"2024-05-08T08:52:11.931332+0000","last_became_peered":"2024-05-08T08:52:11.931332+0000","last_unstale":"2024-05-08T08:52:47.053629+0000","last_undegraded":"2024-05-08T08:52:47.053629+0000","last_fullsized":"2024-05-08T08:52:47.053629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:16:57.420792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000691565,"stat_sum":{"num_bytes":38931472,"num_objects":40,"num_object_clones":7,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":28,"num_read":9939,"num_read_kb":23296,"num_write":268,"num_write_kb":26782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"41'174","reported_seq":1294,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-05-08T08:52:47.055513+0000","last_change":"2024-05-08T08:52:13.944947+0000","last_active":"2024-05-08T08:52:47.055513+0000","last_peered":"2024-05-08T08:52:47.055513+0000","last_clean":"2024-05-08T08:52:47.055513+0000","last_became_active":"2024-05-08T08:52:11.930844+0000","last_became_peered":"2024-05-08T08:52:11.930844+0000","last_unstale":"2024-05-08T08:52:47.055513+0000","last_undegraded":"2024-05-08T08:52:47.055513+0000","last_fullsized":"2024-05-08T08:52:47.055513+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:34:26.156206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040424900000000003,"stat_sum":{"num_bytes":3758080,"num_objects":44,"num_object_clones":5,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":35,"num_read":1273,"num_read_kb":3348,"num_write":120,"num_write_kb":3318,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"39'84","reported_seq":158,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-05-08T08:52:47.053412+0000","last_change":"2024-05-08T08:52:36.057025+0000","last_active":"2024-05-08T08:52:47.053412+0000","last_peered":"2024-05-08T08:52:47.053412+0000","last_clean":"2024-05-08T08:52:47.053412+0000","last_became_active":"2024-05-08T08:52:36.024295+0000","last_became_peered":"2024-05-08T08:52:36.024295+0000","last_unstale":"2024-05-08T08:52:47.053412+0000","last_undegraded":"2024-05-08T08:52:47.053412+0000","last_fullsized":"2024-05-08T08:52:47.053412+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:06:21.473327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072111799999999998,"stat_sum":{"num_bytes":2545743,"num_objects":29,"num_object_clones":1,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":21,"num_read":3050,"num_read_kb":8027,"num_write":66,"num_write_kb":287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":173135,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"40'214","reported_seq":7156,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-05-08T08:52:45.911087+0000","last_change":"2024-05-08T08:52:47.053736+0000","last_active":"2024-05-08T08:52:47.053736+0000","last_peered":"2024-05-08T08:52:45.911087+0000","last_clean":"2024-05-08T08:52:45.911087+0000","last_became_active":"2024-05-08T08:52:23.966513+0000","last_became_peered":"2024-05-08T08:52:23.966513+0000","last_unstale":"2024-05-08T08:52:45.911087+0000","last_undegraded":"2024-05-08T08:52:45.911087+0000","last_fullsized":"2024-05-08T08:52:45.911087+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":214,"log_dups_size":0,"ondisk_log_size":214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094844199999999997,"stat_sum":{"num_bytes":36842533,"num_objects":47,"num_object_clones":6,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":33,"num_read":6936,"num_read_kb":47238,"num_write":310,"num_write_kb":18775,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,5],"up_primary":1,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"40'231","reported_seq":7512,"reported_epoch":41,"state":"remapped+peering","last_fresh":"2024-05-08T08:52:46.051409+0000","last_change":"2024-05-08T08:52:47.054367+0000","last_active":"2024-05-08T08:52:46.051409+0000","last_peered":"2024-05-08T08:52:45.963005+0000","last_clean":"2024-05-08T08:52:45.963005+0000","last_became_active":"2024-05-08T08:52:11.927963+0000","last_became_peered":"2024-05-08T08:52:11.927963+0000","last_unstale":"2024-05-08T08:52:46.051409+0000","last_undegraded":"2024-05-08T08:52:46.051409+0000","last_fullsized":"2024-05-08T08:52:46.051409+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":231,"log_dups_size":0,"ondisk_log_size":231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000685079,"stat_sum":{"num_bytes":24642604,"num_objects":41,"num_object_clones":3,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":34,"num_read":7879,"num_read_kb":12197,"num_write":299,"num_write_kb":17200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"39'181","reported_seq":5518,"reported_epoch":42,"state":"remapped+peering","last_fresh":"2024-05-08T08:52:47.058356+0000","last_change":"2024-05-08T08:52:47.058356+0000","last_active":"2024-05-08T08:52:46.054171+0000","last_peered":"2024-05-08T08:52:44.661708+0000","last_clean":"2024-05-08T08:52:44.661708+0000","last_became_active":"2024-05-08T08:52:11.926106+0000","last_became_peered":"2024-05-08T08:52:11.926106+0000","last_unstale":"2024-05-08T08:52:47.058356+0000","last_undegraded":"2024-05-08T08:52:47.058356+0000","last_fullsized":"2024-05-08T08:52:47.058356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066525600000000003,"stat_sum":{"num_bytes":4709425,"num_objects":39,"num_object_clones":4,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":28,"num_read":5665,"num_read_kb":9400,"num_write":210,"num_write_kb":2398,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"40'152","reported_seq":1012,"reported_epoch":41,"state":"remapped+peering","last_fresh":"2024-05-08T08:52:46.056672+0000","last_change":"2024-05-08T08:52:47.059139+0000","last_active":"2024-05-08T08:52:46.056672+0000","last_peered":"2024-05-08T08:52:45.925372+0000","last_clean":"2024-05-08T08:52:45.925372+0000","last_became_active":"2024-05-08T08:52:11.925809+0000","last_became_peered":"2024-05-08T08:52:11.925809+0000","last_unstale":"2024-05-08T08:52:46.056672+0000","last_undegraded":"2024-05-08T08:52:46.056672+0000","last_fullsized":"2024-05-08T08:52:46.056672+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082481000000000004,"stat_sum":{"num_bytes":4611584,"num_objects":42,"num_object_clones":5,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":35,"num_read":823,"num_read_kb":8290,"num_write":117,"num_write_kb":5214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"39'142","reported_seq":4133,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-05-08T08:52:47.055364+0000","last_change":"2024-05-08T08:52:13.945184+0000","last_active":"2024-05-08T08:52:47.055364+0000","last_peered":"2024-05-08T08:52:47.055364+0000","last_clean":"2024-05-08T08:52:47.055364+0000","last_became_active":"2024-05-08T08:52:11.927366+0000","last_became_peered":"2024-05-08T08:52:11.927366+0000","last_unstale":"2024-05-08T08:52:47.055364+0000","last_undegraded":"2024-05-08T08:52:47.055364+0000","last_fullsized":"2024-05-08T08:52:47.055364+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":142,"log_dups_size":0,"ondisk_log_size":142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:22:11.408947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057208900000000002,"stat_sum":{"num_bytes":56109568,"num_objects":41,"num_object_clones":6,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":30,"num_read":3977,"num_read_kb":17064,"num_write":115,"num_write_kb":14997,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"41'441","reported_seq":805,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-05-08T08:52:47.055879+0000","last_change":"2024-05-08T08:52:13.935180+0000","last_active":"2024-05-08T08:52:47.055879+0000","last_peered":"2024-05-08T08:52:47.055879+0000","last_clean":"2024-05-08T08:52:47.055879+0000","last_became_active":"2024-05-08T08:52:11.930714+0000","last_became_peered":"2024-05-08T08:52:11.930714+0000","last_unstale":"2024-05-08T08:52:47.055879+0000","last_undegraded":"2024-05-08T08:52:47.055879+0000","last_fullsized":"2024-05-08T08:52:47.055879+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":441,"log_dups_size":0,"ondisk_log_size":441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:50.872615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000866371,"stat_sum":{"num_bytes":1124449,"num_objects":25,"num_object_clones":2,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":18,"num_read":1975,"num_read_kb":2923,"num_write":622,"num_write_kb":1447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"41'180","reported_seq":11870,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-05-08T08:52:47.056772+0000","last_change":"2024-05-08T08:52:13.932215+0000","last_active":"2024-05-08T08:52:47.056772+0000","last_peered":"2024-05-08T08:52:47.056772+0000","last_clean":"2024-05-08T08:52:47.056772+0000","last_became_active":"2024-05-08T08:52:11.931213+0000","last_became_peered":"2024-05-08T08:52:11.931213+0000","last_unstale":"2024-05-08T08:52:47.056772+0000","last_undegraded":"2024-05-08T08:52:47.056772+0000","last_fullsized":"2024-05-08T08:52:47.056772+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:18:26.978636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084652399999999997,"stat_sum":{"num_bytes":25823936,"num_objects":37,"num_object_clones":5,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":21,"num_read":11767,"num_read_kb":44177,"num_write":239,"num_write_kb":12819,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"39'134","reported_seq":3510,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-05-08T08:52:47.053248+0000","last_change":"2024-05-08T08:52:13.944999+0000","last_active":"2024-05-08T08:52:47.053248+0000","last_peered":"2024-05-08T08:52:47.053248+0000","last_clean":"2024-05-08T08:52:47.053248+0000","last_became_active":"2024-05-08T08:52:11.931695+0000","last_became_peered":"2024-05-08T08:52:11.931695+0000","last_unstale":"2024-05-08T08:52:47.053248+0000","last_undegraded":"2024-05-08T08:52:47.053248+0000","last_fullsized":"2024-05-08T08:52:47.053248+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:06:48.814777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063902800000000004,"stat_sum":{"num_bytes":6899200,"num_objects":32,"num_object_clones":3,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":26,"num_read":3343,"num_read_kb":6277,"num_write":108,"num_write_kb":621,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"40'212","reported_seq":3522,"reported_epoch":41,"state":"remapped+peering","last_fresh":"2024-05-08T08:52:46.054886+0000","last_change":"2024-05-08T08:52:47.053681+0000","last_active":"2024-05-08T08:52:46.054886+0000","last_peered":"2024-05-08T08:52:46.026849+0000","last_clean":"2024-05-08T08:52:46.026849+0000","last_became_active":"2024-05-08T08:52:11.932699+0000","last_became_peered":"2024-05-08T08:52:11.932699+0000","last_unstale":"2024-05-08T08:52:46.054886+0000","last_undegraded":"2024-05-08T08:52:46.054886+0000","last_fullsized":"2024-05-08T08:52:46.054886+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":212,"log_dups_size":0,"ondisk_log_size":212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000430331,"stat_sum":{"num_bytes":17586704,"num_objects":37,"num_object_clones":6,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":27,"num_read":1242,"num_read_kb":2765,"num_write":155,"num_write_kb":3356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":67,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-05-08T08:52:47.057420+0000","last_change":"2024-05-08T08:52:47.057420+0000","last_active":"2024-05-08T08:52:47.057420+0000","last_peered":"2024-05-08T08:52:47.057420+0000","last_clean":"2024-05-08T08:52:47.057420+0000","last_became_active":"2024-05-08T08:52:47.057207+0000","last_became_peered":"2024-05-08T08:52:47.057207+0000","last_unstale":"2024-05-08T08:52:47.057420+0000","last_undegraded":"2024-05-08T08:52:47.057420+0000","last_fullsized":"2024-05-08T08:52:47.057420+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:47:42.112090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"39'117","reported_seq":8891,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-05-08T08:52:47.053929+0000","last_change":"2024-05-08T08:52:13.934894+0000","last_active":"2024-05-08T08:52:47.053929+0000","last_peered":"2024-05-08T08:52:47.053929+0000","last_clean":"2024-05-08T08:52:47.053929+0000","last_became_active":"2024-05-08T08:52:11.929184+0000","last_became_peered":"2024-05-08T08:52:11.929184+0000","last_unstale":"2024-05-08T08:52:47.053929+0000","last_undegraded":"2024-05-08T08:52:47.053929+0000","last_fullsized":"2024-05-08T08:52:47.053929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:32:23.508696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088867899999999997,"stat_sum":{"num_bytes":22045696,"num_objects":31,"num_object_clones":2,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":22,"num_read":8733,"num_read_kb":45100,"num_write":148,"num_write_kb":9051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":62,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-05-08T08:52:47.059612+0000","last_change":"2024-05-08T08:52:03.896159+0000","last_active":"2024-05-08T08:52:47.059612+0000","last_peered":"2024-05-08T08:52:47.059612+0000","last_clean":"2024-05-08T08:52:47.059612+0000","last_became_active":"2024-05-08T08:52:03.895797+0000","last_became_peered":"2024-05-08T08:52:03.895797+0000","last_unstale":"2024-05-08T08:52:47.059612+0000","last_undegraded":"2024-05-08T08:52:47.059612+0000","last_fullsized":"2024-05-08T08:52:47.059612+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:34:33.513525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"39'124","reported_seq":1028,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-05-08T08:52:47.055989+0000","last_change":"2024-05-08T08:52:13.944951+0000","last_active":"2024-05-08T08:52:47.055989+0000","last_peered":"2024-05-08T08:52:47.055989+0000","last_clean":"2024-05-08T08:52:47.055989+0000","last_became_active":"2024-05-08T08:52:11.926905+0000","last_became_peered":"2024-05-08T08:52:11.926905+0000","last_unstale":"2024-05-08T08:52:47.055989+0000","last_undegraded":"2024-05-08T08:52:47.055989+0000","last_fullsized":"2024-05-08T08:52:47.055989+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:22:13.499222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044527,"stat_sum":{"num_bytes":5048466,"num_objects":42,"num_object_clones":8,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":27,"num_read":929,"num_read_kb":6490,"num_write":108,"num_write_kb":3387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":62,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-05-08T08:52:47.053381+0000","last_change":"2024-05-08T08:52:04.144764+0000","last_active":"2024-05-08T08:52:47.053381+0000","last_peered":"2024-05-08T08:52:47.053381+0000","last_clean":"2024-05-08T08:52:47.053381+0000","last_became_active":"2024-05-08T08:52:04.144444+0000","last_became_peered":"2024-05-08T08:52:04.144444+0000","last_unstale":"2024-05-08T08:52:47.053381+0000","last_undegraded":"2024-05-08T08:52:47.053381+0000","last_fullsized":"2024-05-08T08:52:47.053381+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:58:31.498922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"41'207","reported_seq":3775,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-05-08T08:52:47.059626+0000","last_change":"2024-05-08T08:52:13.940976+0000","last_active":"2024-05-08T08:52:47.059626+0000","last_peered":"2024-05-08T08:52:47.059626+0000","last_clean":"2024-05-08T08:52:47.059626+0000","last_became_active":"2024-05-08T08:52:11.926402+0000","last_became_peered":"2024-05-08T08:52:11.926402+0000","last_unstale":"2024-05-08T08:52:47.059626+0000","last_undegraded":"2024-05-08T08:52:47.059626+0000","last_fullsized":"2024-05-08T08:52:47.059626+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":207,"log_dups_size":0,"ondisk_log_size":207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:34:36.190224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000669351,"stat_sum":{"num_bytes":2588672,"num_objects":42,"num_object_clones":4,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":34,"num_read":4311,"num_read_kb":6354,"num_write":166,"num_write_kb":1941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":62,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-05-08T08:52:47.059716+0000","last_change":"2024-05-08T08:52:04.144543+0000","last_active":"2024-05-08T08:52:47.059716+0000","last_peered":"2024-05-08T08:52:47.059716+0000","last_clean":"2024-05-08T08:52:47.059716+0000","last_became_active":"2024-05-08T08:52:04.144242+0000","last_became_peered":"2024-05-08T08:52:04.144242+0000","last_unstale":"2024-05-08T08:52:47.059716+0000","last_undegraded":"2024-05-08T08:52:47.059716+0000","last_fullsized":"2024-05-08T08:52:47.059716+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:57:36.844633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"40'531","reported_seq":4134,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-05-08T08:52:47.056328+0000","last_change":"2024-05-08T08:52:23.967224+0000","last_active":"2024-05-08T08:52:47.056328+0000","last_peered":"2024-05-08T08:52:47.056328+0000","last_clean":"2024-05-08T08:52:47.056328+0000","last_became_active":"2024-05-08T08:52:23.966596+0000","last_became_peered":"2024-05-08T08:52:23.966596+0000","last_unstale":"2024-05-08T08:52:47.056328+0000","last_undegraded":"2024-05-08T08:52:47.056328+0000","last_fullsized":"2024-05-08T08:52:47.056328+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":531,"log_dups_size":0,"ondisk_log_size":531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:12:59.365231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064078099999999999,"stat_sum":{"num_bytes":6713531,"num_objects":56,"num_object_clones":6,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":43,"num_read":5631,"num_read_kb":8629,"num_write":805,"num_write_kb":3145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"39'122","reported_seq":612,"reported_epoch":42,"state":"remapped+peering","last_fresh":"2024-05-08T08:52:47.053673+0000","last_change":"2024-05-08T08:52:47.053673+0000","last_active":"2024-05-08T08:52:46.051542+0000","last_peered":"2024-05-08T08:52:44.802494+0000","last_clean":"2024-05-08T08:52:44.802494+0000","last_became_active":"2024-05-08T08:52:11.927134+0000","last_became_peered":"2024-05-08T08:52:11.927134+0000","last_unstale":"2024-05-08T08:52:47.053673+0000","last_undegraded":"2024-05-08T08:52:47.053673+0000","last_fullsized":"2024-05-08T08:52:47.053673+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047037799999999997,"stat_sum":{"num_bytes":2269184,"num_objects":25,"num_object_clones":3,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":20,"num_read":452,"num_read_kb":5462,"num_write":104,"num_write_kb":2330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":37,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-05-08T08:52:47.059078+0000","last_change":"2024-05-08T08:52:23.969093+0000","last_active":"2024-05-08T08:52:47.059078+0000","last_peered":"2024-05-08T08:52:47.059078+0000","last_clean":"2024-05-08T08:52:47.059078+0000","last_became_active":"2024-05-08T08:52:23.968782+0000","last_became_peered":"2024-05-08T08:52:23.968782+0000","last_unstale":"2024-05-08T08:52:47.059078+0000","last_undegraded":"2024-05-08T08:52:47.059078+0000","last_fullsized":"2024-05-08T08:52:47.059078+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:37:41.693728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"40'129","reported_seq":601,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-05-08T08:52:47.058759+0000","last_change":"2024-05-08T08:52:13.940504+0000","last_active":"2024-05-08T08:52:47.058759+0000","last_peered":"2024-05-08T08:52:47.058759+0000","last_clean":"2024-05-08T08:52:47.058759+0000","last_became_active":"2024-05-08T08:52:11.928301+0000","last_became_peered":"2024-05-08T08:52:11.928301+0000","last_unstale":"2024-05-08T08:52:47.058759+0000","last_undegraded":"2024-05-08T08:52:47.058759+0000","last_fullsized":"2024-05-08T08:52:47.058759+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:24:36.121873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046631800000000002,"stat_sum":{"num_bytes":20586564,"num_objects":40,"num_object_clones":5,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":29,"num_read":468,"num_read_kb":3878,"num_write":104,"num_write_kb":6332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":62,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-05-08T08:52:47.057014+0000","last_change":"2024-05-08T08:52:03.891768+0000","last_active":"2024-05-08T08:52:47.057014+0000","last_peered":"2024-05-08T08:52:47.057014+0000","last_clean":"2024-05-08T08:52:47.057014+0000","last_became_active":"2024-05-08T08:52:03.891374+0000","last_became_peered":"2024-05-08T08:52:03.891374+0000","last_unstale":"2024-05-08T08:52:47.057014+0000","last_undegraded":"2024-05-08T08:52:47.057014+0000","last_fullsized":"2024-05-08T08:52:47.057014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:18:10.240492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"40'439","reported_seq":13779,"reported_epoch":41,"state":"remapped+peering","last_fresh":"2024-05-08T08:52:46.051172+0000","last_change":"2024-05-08T08:52:47.052784+0000","last_active":"2024-05-08T08:52:46.051172+0000","last_peered":"2024-05-08T08:52:45.963007+0000","last_clean":"2024-05-08T08:52:45.963007+0000","last_became_active":"2024-05-08T08:52:11.928598+0000","last_became_peered":"2024-05-08T08:52:11.928598+0000","last_unstale":"2024-05-08T08:52:46.051172+0000","last_undegraded":"2024-05-08T08:52:46.051172+0000","last_fullsized":"2024-05-08T08:52:46.051172+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046440499999999998,"stat_sum":{"num_bytes":9303600,"num_objects":34,"num_object_clones":4,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":22,"num_read":14406,"num_read_kb":18265,"num_write":252,"num_write_kb":3132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":62,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-05-08T08:52:47.052433+0000","last_change":"2024-05-08T08:52:03.891551+0000","last_active":"2024-05-08T08:52:47.052433+0000","last_peered":"2024-05-08T08:52:47.052433+0000","last_clean":"2024-05-08T08:52:47.052433+0000","last_became_active":"2024-05-08T08:52:03.891296+0000","last_became_peered":"2024-05-08T08:52:03.891296+0000","last_unstale":"2024-05-08T08:52:47.052433+0000","last_undegraded":"2024-05-08T08:52:47.052433+0000","last_fullsized":"2024-05-08T08:52:47.052433+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:54.806343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"40'110","reported_seq":467,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-05-08T08:52:47.058983+0000","last_change":"2024-05-08T08:52:13.940743+0000","last_active":"2024-05-08T08:52:47.058983+0000","last_peered":"2024-05-08T08:52:47.058983+0000","last_clean":"2024-05-08T08:52:47.058983+0000","last_became_active":"2024-05-08T08:52:11.928273+0000","last_became_peered":"2024-05-08T08:52:11.928273+0000","last_unstale":"2024-05-08T08:52:47.058983+0000","last_undegraded":"2024-05-08T08:52:47.058983+0000","last_fullsized":"2024-05-08T08:52:47.058983+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:00:40.626465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062135599999999999,"stat_sum":{"num_bytes":2875506,"num_objects":41,"num_object_clones":8,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":29,"num_read":348,"num_read_kb":3390,"num_write":82,"num_write_kb":987,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":37,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-05-08T08:52:47.055411+0000","last_change":"2024-05-08T08:52:23.966794+0000","last_active":"2024-05-08T08:52:47.055411+0000","last_peered":"2024-05-08T08:52:47.055411+0000","last_clean":"2024-05-08T08:52:47.055411+0000","last_became_active":"2024-05-08T08:52:23.966544+0000","last_became_peered":"2024-05-08T08:52:23.966544+0000","last_unstale":"2024-05-08T08:52:47.055411+0000","last_undegraded":"2024-05-08T08:52:47.055411+0000","last_fullsized":"2024-05-08T08:52:47.055411+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:25:56.927420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":218,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-05-08T08:52:47.052520+0000","last_change":"2024-05-08T08:51:58.861899+0000","last_active":"2024-05-08T08:52:47.052520+0000","last_peered":"2024-05-08T08:52:47.052520+0000","last_clean":"2024-05-08T08:52:47.052520+0000","last_became_active":"2024-05-08T08:51:58.861596+0000","last_became_peered":"2024-05-08T08:51:58.861596+0000","last_unstale":"2024-05-08T08:52:47.052520+0000","last_undegraded":"2024-05-08T08:52:47.052520+0000","last_fullsized":"2024-05-08T08:52:47.052520+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:51:57.849666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:51:57.849666+0000","last_clean_scrub_stamp":"2024-05-08T08:51:57.849666+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:53:21.739950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"39'117","reported_seq":848,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-05-08T08:52:47.055425+0000","last_change":"2024-05-08T08:52:13.945362+0000","last_active":"2024-05-08T08:52:47.055425+0000","last_peered":"2024-05-08T08:52:47.055425+0000","last_clean":"2024-05-08T08:52:47.055425+0000","last_became_active":"2024-05-08T08:52:11.926975+0000","last_became_peered":"2024-05-08T08:52:11.926975+0000","last_unstale":"2024-05-08T08:52:47.055425+0000","last_undegraded":"2024-05-08T08:52:47.055425+0000","last_fullsized":"2024-05-08T08:52:47.055425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:54:32.846588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068218599999999999,"stat_sum":{"num_bytes":17097744,"num_objects":45,"num_object_clones":6,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":33,"num_read":714,"num_read_kb":4864,"num_write":84,"num_write_kb":2455,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"40'164","reported_seq":4471,"reported_epoch":42,"state":"remapped+peering","last_fresh":"2024-05-08T08:52:47.056684+0000","last_change":"2024-05-08T08:52:47.056684+0000","last_active":"2024-05-08T08:52:46.053341+0000","last_peered":"2024-05-08T08:52:45.891633+0000","last_clean":"2024-05-08T08:52:45.891633+0000","last_became_active":"2024-05-08T08:52:11.927141+0000","last_became_peered":"2024-05-08T08:52:11.927141+0000","last_unstale":"2024-05-08T08:52:47.056684+0000","last_undegraded":"2024-05-08T08:52:47.056684+0000","last_fullsized":"2024-05-08T08:52:47.056684+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00030754000000000003,"stat_sum":{"num_bytes":5255168,"num_objects":35,"num_object_clones":4,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":26,"num_read":4756,"num_read_kb":10488,"num_write":117,"num_write_kb":2110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"40'422","reported_seq":7570,"reported_epoch":41,"state":"remapped+peering","last_fresh":"2024-05-08T08:52:46.055113+0000","last_change":"2024-05-08T08:52:47.053284+0000","last_active":"2024-05-08T08:52:46.055113+0000","last_peered":"2024-05-08T08:52:45.963007+0000","last_clean":"2024-05-08T08:52:45.963007+0000","last_became_active":"2024-05-08T08:52:11.932810+0000","last_became_peered":"2024-05-08T08:52:11.932810+0000","last_unstale":"2024-05-08T08:52:46.055113+0000","last_undegraded":"2024-05-08T08:52:46.055113+0000","last_fullsized":"2024-05-08T08:52:46.055113+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":422,"log_dups_size":0,"ondisk_log_size":422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070350100000000004,"stat_sum":{"num_bytes":12296192,"num_objects":40,"num_object_clones":4,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":30,"num_read":10305,"num_read_kb":13877,"num_write":262,"num_write_kb":1232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"41'151","reported_seq":1036,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-05-08T08:52:47.058941+0000","last_change":"2024-05-08T08:52:13.940600+0000","last_active":"2024-05-08T08:52:47.058941+0000","last_peered":"2024-05-08T08:52:47.058941+0000","last_clean":"2024-05-08T08:52:47.058941+0000","last_became_active":"2024-05-08T08:52:11.927196+0000","last_became_peered":"2024-05-08T08:52:11.927196+0000","last_unstale":"2024-05-08T08:52:47.058941+0000","last_undegraded":"2024-05-08T08:52:47.058941+0000","last_fullsized":"2024-05-08T08:52:47.058941+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:47:32.235549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048296599999999998,"stat_sum":{"num_bytes":22111248,"num_objects":37,"num_object_clones":3,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":28,"num_read":864,"num_read_kb":4839,"num_write":128,"num_write_kb":1742,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"38'89","reported_seq":395,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-05-08T08:52:47.052718+0000","last_change":"2024-05-08T08:52:23.969521+0000","last_active":"2024-05-08T08:52:47.052718+0000","last_peered":"2024-05-08T08:52:47.052718+0000","last_clean":"2024-05-08T08:52:47.052718+0000","last_became_active":"2024-05-08T08:52:23.969067+0000","last_became_peered":"2024-05-08T08:52:23.969067+0000","last_unstale":"2024-05-08T08:52:47.052718+0000","last_undegraded":"2024-05-08T08:52:47.052718+0000","last_fullsized":"2024-05-08T08:52:47.052718+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:28:21.982639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048793000000000003,"stat_sum":{"num_bytes":1067008,"num_objects":30,"num_object_clones":1,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":26,"num_read":268,"num_read_kb":2268,"num_write":66,"num_write_kb":726,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"40'158","reported_seq":676,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-05-08T08:52:47.053268+0000","last_change":"2024-05-08T08:52:13.945141+0000","last_active":"2024-05-08T08:52:47.053268+0000","last_peered":"2024-05-08T08:52:47.053268+0000","last_clean":"2024-05-08T08:52:47.053268+0000","last_became_active":"2024-05-08T08:52:11.932594+0000","last_became_peered":"2024-05-08T08:52:11.932594+0000","last_unstale":"2024-05-08T08:52:47.053268+0000","last_undegraded":"2024-05-08T08:52:47.053268+0000","last_fullsized":"2024-05-08T08:52:47.053268+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":158,"log_dups_size":0,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:37:25.081217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074388199999999996,"stat_sum":{"num_bytes":2221568,"num_objects":41,"num_object_clones":4,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":32,"num_read":485,"num_read_kb":4904,"num_write":128,"num_write_kb":1336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"40'97","reported_seq":86,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-05-08T08:52:47.139209+0000","last_change":"2024-05-08T08:52:47.137016+0000","last_active":"2024-05-08T08:52:47.139209+0000","last_peered":"2024-05-08T08:52:47.139209+0000","last_clean":"2024-05-08T08:52:47.139209+0000","last_became_active":"2024-05-08T08:52:47.071804+0000","last_became_peered":"2024-05-08T08:52:47.071804+0000","last_unstale":"2024-05-08T08:52:47.139209+0000","last_undegraded":"2024-05-08T08:52:47.139209+0000","last_fullsized":"2024-05-08T08:52:47.139209+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:11:51.357761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085079600000000002,"stat_sum":{"num_bytes":7188480,"num_objects":25,"num_object_clones":4,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":16,"num_read":5307,"num_read_kb":16911,"num_write":81,"num_write_kb":2931,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":2343424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.f","version":"37'117","reported_seq":1877,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-05-08T08:52:47.056779+0000","last_change":"2024-05-08T08:52:13.932524+0000","last_active":"2024-05-08T08:52:47.056779+0000","last_peered":"2024-05-08T08:52:47.056779+0000","last_clean":"2024-05-08T08:52:47.056779+0000","last_became_active":"2024-05-08T08:52:11.931347+0000","last_became_peered":"2024-05-08T08:52:11.931347+0000","last_unstale":"2024-05-08T08:52:47.056779+0000","last_undegraded":"2024-05-08T08:52:47.056779+0000","last_fullsized":"2024-05-08T08:52:47.056779+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:38:18.433195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010326370000000001,"stat_sum":{"num_bytes":7685827,"num_objects":29,"num_object_clones":1,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":21,"num_read":1963,"num_read_kb":14063,"num_write":141,"num_write_kb":1378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"39'133","reported_seq":2195,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-05-08T08:52:47.052072+0000","last_change":"2024-05-08T08:52:23.970138+0000","last_active":"2024-05-08T08:52:47.052072+0000","last_peered":"2024-05-08T08:52:47.052072+0000","last_clean":"2024-05-08T08:52:47.052072+0000","last_became_active":"2024-05-08T08:52:23.969732+0000","last_became_peered":"2024-05-08T08:52:23.969732+0000","last_unstale":"2024-05-08T08:52:47.052072+0000","last_undegraded":"2024-05-08T08:52:47.052072+0000","last_fullsized":"2024-05-08T08:52:47.052072+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:37:55.961844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053593,"stat_sum":{"num_bytes":16504320,"num_objects":42,"num_object_clones":6,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":31,"num_read":2037,"num_read_kb":17923,"num_write":103,"num_write_kb":8464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"39'132","reported_seq":346,"reported_epoch":41,"state":"remapped+peering","last_fresh":"2024-05-08T08:52:46.054908+0000","last_change":"2024-05-08T08:52:47.053595+0000","last_active":"2024-05-08T08:52:46.054908+0000","last_peered":"2024-05-08T08:52:45.047323+0000","last_clean":"2024-05-08T08:52:45.047323+0000","last_became_active":"2024-05-08T08:52:11.932700+0000","last_became_peered":"2024-05-08T08:52:11.932700+0000","last_unstale":"2024-05-08T08:52:46.054908+0000","last_undegraded":"2024-05-08T08:52:46.054908+0000","last_fullsized":"2024-05-08T08:52:46.054908+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":132,"log_dups_size":0,"ondisk_log_size":132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041230699999999999,"stat_sum":{"num_bytes":4785152,"num_objects":38,"num_object_clones":3,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":33,"num_read":187,"num_read_kb":1020,"num_write":98,"num_write_kb":501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"41'855","reported_seq":5688,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-05-08T08:52:47.052309+0000","last_change":"2024-05-08T08:52:13.934541+0000","last_active":"2024-05-08T08:52:47.052309+0000","last_peered":"2024-05-08T08:52:47.052309+0000","last_clean":"2024-05-08T08:52:47.052309+0000","last_became_active":"2024-05-08T08:52:11.931202+0000","last_became_peered":"2024-05-08T08:52:11.931202+0000","last_unstale":"2024-05-08T08:52:47.052309+0000","last_undegraded":"2024-05-08T08:52:47.052309+0000","last_fullsized":"2024-05-08T08:52:47.052309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":855,"log_dups_size":0,"ondisk_log_size":855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:28:22.248419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059220300000000002,"stat_sum":{"num_bytes":20709486,"num_objects":47,"num_object_clones":3,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":31,"num_read":9470,"num_read_kb":18307,"num_write":1302,"num_write_kb":3538,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"41'231","reported_seq":1978,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-05-08T08:52:47.052900+0000","last_change":"2024-05-08T08:52:13.944894+0000","last_active":"2024-05-08T08:52:47.052900+0000","last_peered":"2024-05-08T08:52:47.052900+0000","last_clean":"2024-05-08T08:52:47.052900+0000","last_became_active":"2024-05-08T08:52:11.930434+0000","last_became_peered":"2024-05-08T08:52:11.930434+0000","last_unstale":"2024-05-08T08:52:47.052900+0000","last_undegraded":"2024-05-08T08:52:47.052900+0000","last_fullsized":"2024-05-08T08:52:47.052900+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":231,"log_dups_size":0,"ondisk_log_size":231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:57:04.271478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059972400000000005,"stat_sum":{"num_bytes":26229293,"num_objects":47,"num_object_clones":9,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":27,"num_read":2036,"num_read_kb":14324,"num_write":245,"num_write_kb":3860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":438168003,"num_objects":1221,"num_object_clones":141,"num_object_copies":2442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1221,"num_whiteouts":896,"num_read":131539,"num_read_kb":406358,"num_write":7153,"num_write_kb":167792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":71,"num_bytes_recovered":2516559,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":22585344,"data_stored":175077254,"data_compressed":5882702,"data_compressed_allocated":15966208,"data_compressed_original":170771456,"omap_allocated":0,"internal_metadata":0},"log_size":6764,"ondisk_log_size":6764,"up":64,"acting":55,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":11046,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640268,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40188,"kb_used_data":4844,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94331652,"statfs":{"total":96636764160,"available":96595611648,"internally_reserved":0,"allocated":4960256,"data_stored":39063527,"data_compressed":1391401,"data_compressed_allocated":3796992,"data_compressed_original":38425600,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":0,"seq":47244640271,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640270,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45928,"kb_used_data":6424,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94325912,"statfs":{"total":96636764160,"available":96589733888,"internally_reserved":0,"allocated":6578176,"data_stored":39527059,"data_compressed":1396597,"data_compressed_allocated":3743744,"data_compressed_original":37778432,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640269,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37452,"kb_used_data":2112,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334388,"statfs":{"total":96636764160,"available":96598413312,"internally_reserved":0,"allocated":2162688,"data_stored":15482326,"data_compressed":522193,"data_compressed_allocated":1294336,"data_compressed_original":15056896,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640268,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33660,"kb_used_data":2472,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338180,"statfs":{"total":96636764160,"available":96602296320,"internally_reserved":0,"allocated":2531328,"data_stored":20195278,"data_compressed":571762,"data_compressed_allocated":1622016,"data_compressed_original":19745792,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640271,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45336,"kb_used_data":5832,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94326504,"statfs":{"total":96636764160,"available":96590340096,"internally_reserved":0,"allocated":5971968,"data_stored":37337071,"data_compressed":1151510,"data_compressed_allocated":3231744,"data_compressed_original":35691520,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5523,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5523,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5447680,"data_stored":36565402,"data_compressed":1145987,"data_compressed_allocated":3194880,"data_compressed_original":35118080,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2064384,"data_stored":20013977,"data_compressed":571762,"data_compressed_allocated":1622016,"data_compressed_original":19745792,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1695744,"data_stored":15301025,"data_compressed":522193,"data_compressed_allocated":1294336,"data_compressed_original":15056896,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":6053888,"data_stored":38755390,"data_compressed":1391074,"data_compressed_allocated":3706880,"data_compressed_original":37204992,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":2830336,"data_stored":25559234,"data_compressed":860285,"data_compressed_allocated":2351104,"data_compressed_original":25220096,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4493312,"data_stored":38882226,"data_compressed":1391401,"data_compressed_allocated":3796992,"data_compressed_original":38425600,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T08:52:51.851 INFO:tasks.thrashosds.thrasher:Setting 3.17 to [2, 3] 2024-05-08T08:52:51.851 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.17', '2', '3'] 2024-05-08T08:52:51.851 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.17 2 3 2024-05-08T08:52:51.854 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:51.856+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:51.913 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:51.914 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:51.922 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:52:51.964 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:51.967+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:52.049 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:52:52.053+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:52:52.068 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:52.068 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:52.077 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:52:52.080 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-05-08T08:52:52.150 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:52:52.153+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:52:52.177 INFO:teuthology.orchestra.run.smithi196.stdout:truncating image image_client.0-clone11 from 0xe81ba00 (overlap 0x8a5a00) to 0x7c8400 2024-05-08T08:52:52.185 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:52.185 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:52.195 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:52:52.279 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:52:52.282+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:52:52.341 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:52.341 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:52.349 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:52:52.391 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:52.394+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:52.455 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:52.456+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:52.470 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:52.470 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:52.479 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:52:52.553 INFO:teuthology.orchestra.run.smithi196.stdout:checking clone #10, image image_client.0-clone10 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent11 2024-05-08T08:52:52.554 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:52:52.557+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:52:52.597 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:52.597 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:52.607 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:52:52.691 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:52:52.694+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:52:52.717 INFO:teuthology.orchestra.run.smithi196.stdout:2238 trunc from 0xe81ba00 to 0xab39000 2024-05-08T08:52:52.749 INFO:teuthology.orchestra.run.smithi196.stdout:2240 write 0x2a68200 thru 0x2a6ddde (0x5bdf bytes) 2024-05-08T08:52:52.750 INFO:teuthology.orchestra.run.smithi196.stdout:2241 write 0x6c83c00 thru 0x6c84d51 (0x1152 bytes) 2024-05-08T08:52:52.750 INFO:teuthology.orchestra.run.smithi196.stdout:2243 write 0x4c13400 thru 0x4c20bb1 (0xd7b2 bytes) 2024-05-08T08:52:52.755 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:52:52.758+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:52:52.760 INFO:teuthology.orchestra.run.smithi196.stdout:2244 trunc from 0xab39000 to 0x1b0e800 2024-05-08T08:52:52.766 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:52.766 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:52.778 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:52:52.817 INFO:teuthology.orchestra.run.smithi196.stdout:2245 trunc from 0x1b0e800 to 0x5e74800 2024-05-08T08:52:52.826 INFO:teuthology.orchestra.run.smithi196.stdout:2246 write 0x295ec00 thru 0x2969494 (0xa895 bytes) 2024-05-08T08:52:52.827 INFO:teuthology.orchestra.run.smithi196.stdout:2247 write 0x4d1c600 thru 0x4d26a2f (0xa430 bytes) 2024-05-08T08:52:52.836 INFO:teuthology.orchestra.run.smithi196.stdout:2251 trunc from 0x5e74800 to 0xd35c000 2024-05-08T08:52:52.854 INFO:teuthology.orchestra.run.smithi196.stdout:2253 read 0xb2e0800 thru 0xb2ea973 (0xa174 bytes) 2024-05-08T08:52:52.856 INFO:teuthology.orchestra.run.smithi196.stdout:2254 trunc from 0xd35c000 to 0xe22ee00 2024-05-08T08:52:52.856 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:52.860+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:52.876 INFO:teuthology.orchestra.run.smithi196.stdout:2255 trunc from 0xe22ee00 to 0x5a8a400 2024-05-08T08:52:52.892 INFO:teuthology.orchestra.run.smithi196.stdout:2256 trunc from 0x5a8a400 to 0x9f3400 2024-05-08T08:52:52.938 INFO:teuthology.orchestra.run.smithi196.stdout:2258 trunc from 0x9f3400 to 0x1ef7000 2024-05-08T08:52:52.948 INFO:teuthology.orchestra.run.smithi196.stdout:2259 read 0xe98600 thru 0xea2eef (0xa8f0 bytes) 2024-05-08T08:52:52.952 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:52.953 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:52.957 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:52:52.960+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:52:52.962 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:52:52.963 INFO:teuthology.orchestra.run.smithi196.stdout:2260 trunc from 0x1ef7000 to 0xdbb9400 2024-05-08T08:52:52.973 INFO:teuthology.orchestra.run.smithi196.stdout:2262 write 0x1a99a00 thru 0x1aa92b1 (0xf8b2 bytes) 2024-05-08T08:52:52.974 INFO:teuthology.orchestra.run.smithi196.stdout:2263 read 0xb5d9400 thru 0xb5da115 (0xd16 bytes) 2024-05-08T08:52:52.979 INFO:teuthology.orchestra.run.smithi196.stdout:2265 write 0x6497400 thru 0x64a3feb (0xcbec bytes) 2024-05-08T08:52:52.979 INFO:teuthology.orchestra.run.smithi196.stdout:2267 trunc from 0xdbb9400 to 0xa903a00 2024-05-08T08:52:52.999 INFO:teuthology.orchestra.run.smithi196.stdout:2268 write 0x6b6d600 thru 0x6b7aae0 (0xd4e1 bytes) 2024-05-08T08:52:53.005 INFO:teuthology.orchestra.run.smithi196.stdout:2272 trunc from 0xa903a00 to 0xb584400 2024-05-08T08:52:53.017 INFO:teuthology.orchestra.run.smithi196.stdout:2274 write 0x17a1800 thru 0x17ad774 (0xbf75 bytes) 2024-05-08T08:52:53.019 INFO:teuthology.orchestra.run.smithi196.stdout:2275 write 0xb3fe00 thru 0xb464af (0x66b0 bytes) 2024-05-08T08:52:53.019 INFO:teuthology.orchestra.run.smithi196.stdout:2279 read 0x87d9a00 thru 0x87dd8c2 (0x3ec3 bytes) 2024-05-08T08:52:53.023 INFO:teuthology.orchestra.run.smithi196.stdout:2280 trunc from 0xb584400 to 0xcd38c00 2024-05-08T08:52:53.038 INFO:teuthology.orchestra.run.smithi196.stdout:2281 trunc from 0xcd38c00 to 0xa66e200 2024-05-08T08:52:53.048 INFO:teuthology.orchestra.run.smithi196.stdout:2282 write 0x92cd600 thru 0x92db7ed (0xe1ee bytes) 2024-05-08T08:52:53.051 INFO:teuthology.orchestra.run.smithi196.stdout:2284 trunc from 0xa66e200 to 0xc0c2200 2024-05-08T08:52:53.058 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:53.061+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:53.064 INFO:teuthology.orchestra.run.smithi196.stdout:2290 read 0x3d2d400 thru 0x3d2edb4 (0x19b5 bytes) 2024-05-08T08:52:53.066 INFO:teuthology.orchestra.run.smithi196.stdout:2291 write 0x2730800 thru 0x273cd1f (0xc520 bytes) 2024-05-08T08:52:53.066 INFO:teuthology.orchestra.run.smithi196.stdout:2293 write 0x45e0400 thru 0x45eb951 (0xb552 bytes) 2024-05-08T08:52:53.066 INFO:teuthology.orchestra.run.smithi196.stdout:2294 read 0xaa6d800 thru 0xaa7b4c4 (0xdcc5 bytes) 2024-05-08T08:52:53.066 INFO:teuthology.orchestra.run.smithi196.stdout:2295 read 0xa171c00 thru 0xa172e0b (0x120c bytes) 2024-05-08T08:52:53.066 INFO:teuthology.orchestra.run.smithi196.stdout:2297 read 0xb7aa200 thru 0xb7ad858 (0x3659 bytes) 2024-05-08T08:52:53.066 INFO:teuthology.orchestra.run.smithi196.stdout:2298 read 0x2528800 thru 0x252ec9a (0x649b bytes) 2024-05-08T08:52:53.067 INFO:teuthology.orchestra.run.smithi196.stdout:2299 clone 13 order 21 su 16384 sc 13 2024-05-08T08:52:53.083 INFO:teuthology.orchestra.run.smithi059.stderr:noscrub is unset 2024-05-08T08:52:53.086 INFO:teuthology.orchestra.run.smithi059.stderr:set 3.17 pg_upmap mapping to [2,3] 2024-05-08T08:52:53.134 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:53.134 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:53.143 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:52:53.185 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:52:53.188+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:52:53.259 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:52:53.263+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:52:53.298 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:53.298 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:53.303 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:52:53.388 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:52:53.391+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:52:53.454 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:53.454 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:53.463 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:52:53.463 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:53.466+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:53.562 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:52:53.566+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:52:53.590 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T08:52:53.613 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:53.613 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:53.623 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:52:53.665 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:52:53.668+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:52:53.718 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:52:53.719 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:52:53.737 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:52:53.737 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:52:53.755 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:52:53.755 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:52:53.764 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:52:53.767+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:52:53.772 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:52:53.773 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:52:53.779 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:52:53.779 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:52:53.779 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:52:53.780 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:52:53.780 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.780 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7490 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 0~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:53.780 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.744226+0000", 2024-05-08T08:52:53.780 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 16.037692308, 2024-05-08T08:52:53.780 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057294818999999997, 2024-05-08T08:52:53.780 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:53.780 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:53.780 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:53.780 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:53.780 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:53.780 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:53.780 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7490 2024-05-08T08:52:53.780 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.780 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:53.781 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.781 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:53.781 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744226+0000", 2024-05-08T08:52:53.781 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:53.781 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.781 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.781 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:53.781 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744226+0000", 2024-05-08T08:52:53.781 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:53.781 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.781 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.781 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:53.781 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744226+0000", 2024-05-08T08:52:53.781 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3799999999999998e-07 2024-05-08T08:52:53.782 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.782 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.782 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:53.782 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744228+0000", 2024-05-08T08:52:53.782 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0710000000000002e-06 2024-05-08T08:52:53.782 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.782 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.782 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:53.782 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744233+0000", 2024-05-08T08:52:53.782 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0130000000000001e-06 2024-05-08T08:52:53.782 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.783 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.783 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:53.783 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744237+0000", 2024-05-08T08:52:53.783 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1930000000000002e-06 2024-05-08T08:52:53.783 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.783 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.783 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:53.783 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.801334+0000", 2024-05-08T08:52:53.783 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057097004999999999 2024-05-08T08:52:53.783 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.783 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.783 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:53.783 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.801414+0000", 2024-05-08T08:52:53.783 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0129000000000005e-05 2024-05-08T08:52:53.784 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.784 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.784 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:53.784 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.801521+0000", 2024-05-08T08:52:53.784 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010707 2024-05-08T08:52:53.784 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:53.784 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:53.784 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:53.784 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.784 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.784 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7491 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 40960~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:53.784 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.744242+0000", 2024-05-08T08:52:53.784 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 16.037676477000002, 2024-05-08T08:52:53.784 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057541677999999999, 2024-05-08T08:52:53.784 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:53.785 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:53.785 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:53.785 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:53.785 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:53.785 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:53.785 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7491 2024-05-08T08:52:53.785 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.785 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:53.785 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.785 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:53.785 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744242+0000", 2024-05-08T08:52:53.785 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:53.785 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.785 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.785 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:53.785 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744242+0000", 2024-05-08T08:52:53.786 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:53.786 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.786 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.786 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:53.786 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744242+0000", 2024-05-08T08:52:53.786 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.2000000000000001e-07 2024-05-08T08:52:53.786 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.786 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.786 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:53.786 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744244+0000", 2024-05-08T08:52:53.786 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6959999999999999e-06 2024-05-08T08:52:53.786 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.786 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.786 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:53.787 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744247+0000", 2024-05-08T08:52:53.787 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3000000000000002e-06 2024-05-08T08:52:53.787 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.787 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.787 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:53.787 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744250+0000", 2024-05-08T08:52:53.787 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.5399999999999998e-06 2024-05-08T08:52:53.787 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.787 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.787 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:53.787 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.801564+0000", 2024-05-08T08:52:53.787 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057313809 2024-05-08T08:52:53.787 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.788 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.788 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:53.788 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.801671+0000", 2024-05-08T08:52:53.788 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000107333 2024-05-08T08:52:53.788 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.788 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.788 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:53.788 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.801784+0000", 2024-05-08T08:52:53.788 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011268 2024-05-08T08:52:53.788 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:53.788 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:53.788 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:53.788 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.789 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.789 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7492 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 81920~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:53.789 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.744253+0000", 2024-05-08T08:52:53.789 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 16.037665000000001, 2024-05-08T08:52:53.789 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057761380000000001, 2024-05-08T08:52:53.789 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:53.789 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:53.789 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:53.789 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:53.789 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:53.789 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:53.789 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7492 2024-05-08T08:52:53.789 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.789 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:53.789 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.790 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:53.790 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744253+0000", 2024-05-08T08:52:53.790 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:53.790 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.790 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.790 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:53.790 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744253+0000", 2024-05-08T08:52:53.790 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:53.790 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.790 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.790 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:53.790 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.745991+0000", 2024-05-08T08:52:53.790 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001738021 2024-05-08T08:52:53.790 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.790 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.791 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:53.791 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.745997+0000", 2024-05-08T08:52:53.791 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5300000000000004e-06 2024-05-08T08:52:53.791 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.791 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.791 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:53.791 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.746006+0000", 2024-05-08T08:52:53.791 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2210000000000003e-06 2024-05-08T08:52:53.791 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.791 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.791 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:53.791 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.746026+0000", 2024-05-08T08:52:53.791 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9487000000000002e-05 2024-05-08T08:52:53.791 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.792 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.792 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:53.792 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.801815+0000", 2024-05-08T08:52:53.792 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.055789196999999999 2024-05-08T08:52:53.792 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.792 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.792 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:53.792 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.801908+0000", 2024-05-08T08:52:53.792 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3320999999999996e-05 2024-05-08T08:52:53.792 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.792 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.792 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:53.792 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.802015+0000", 2024-05-08T08:52:53.792 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000106603 2024-05-08T08:52:53.793 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:53.793 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:53.793 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:53.793 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.793 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.793 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7499 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 368640~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:53.793 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.746209+0000", 2024-05-08T08:52:53.793 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 16.035709052000001, 2024-05-08T08:52:53.793 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057256452999999999, 2024-05-08T08:52:53.793 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:53.793 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:53.793 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:53.793 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:53.793 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:53.794 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:53.794 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7499 2024-05-08T08:52:53.794 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.794 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:53.794 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.794 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:53.794 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.746209+0000", 2024-05-08T08:52:53.794 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:53.794 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.794 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.794 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:53.794 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.746209+0000", 2024-05-08T08:52:53.794 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:53.795 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.795 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.795 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:53.795 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.746210+0000", 2024-05-08T08:52:53.795 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2100000000000002e-07 2024-05-08T08:52:53.795 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.795 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.795 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:53.795 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.746212+0000", 2024-05-08T08:52:53.795 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.922e-06 2024-05-08T08:52:53.795 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.795 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.795 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:53.795 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.746221+0000", 2024-05-08T08:52:53.795 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2900000000000008e-06 2024-05-08T08:52:53.796 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.796 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.796 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:53.796 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.746227+0000", 2024-05-08T08:52:53.796 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3740000000000003e-06 2024-05-08T08:52:53.796 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.796 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.796 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:53.796 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.803304+0000", 2024-05-08T08:52:53.796 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057076239000000001 2024-05-08T08:52:53.796 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.796 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.796 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:53.796 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.803380+0000", 2024-05-08T08:52:53.796 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.6309000000000004e-05 2024-05-08T08:52:53.796 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.797 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.797 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:53.797 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.803466+0000", 2024-05-08T08:52:53.797 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.5897999999999996e-05 2024-05-08T08:52:53.797 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:53.797 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:53.797 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:53.797 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.797 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.797 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7500 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 409600~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:53.797 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.746237+0000", 2024-05-08T08:52:53.797 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 16.03568147, 2024-05-08T08:52:53.797 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057429380000000002, 2024-05-08T08:52:53.797 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:53.797 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:53.798 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:53.798 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:53.798 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:53.798 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:53.798 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7500 2024-05-08T08:52:53.798 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.798 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:53.798 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.798 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:53.798 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.746237+0000", 2024-05-08T08:52:53.798 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:53.798 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.798 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.798 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:53.798 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.746237+0000", 2024-05-08T08:52:53.799 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:53.799 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.799 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.799 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:53.799 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.747959+0000", 2024-05-08T08:52:53.799 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001721888 2024-05-08T08:52:53.799 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.799 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.799 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:53.799 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.747964+0000", 2024-05-08T08:52:53.799 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0030000000000002e-06 2024-05-08T08:52:53.800 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.800 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.800 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:53.800 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.747976+0000", 2024-05-08T08:52:53.800 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2656999999999999e-05 2024-05-08T08:52:53.800 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.800 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.800 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:53.800 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.747983+0000", 2024-05-08T08:52:53.800 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.866e-06 2024-05-08T08:52:53.800 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.800 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.800 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:53.800 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.803495+0000", 2024-05-08T08:52:53.801 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.055511392999999999 2024-05-08T08:52:53.801 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.801 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.801 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:53.801 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.803574+0000", 2024-05-08T08:52:53.801 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9215000000000006e-05 2024-05-08T08:52:53.801 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.801 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.801 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:53.801 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.803666+0000", 2024-05-08T08:52:53.801 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2357999999999999e-05 2024-05-08T08:52:53.801 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:53.801 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:53.802 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:53.802 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.802 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.802 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7508 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 737280~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:53.802 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.748180+0000", 2024-05-08T08:52:53.802 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 16.033738412999998, 2024-05-08T08:52:53.802 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057419809000000002, 2024-05-08T08:52:53.802 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:53.802 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:53.802 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:53.802 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:53.802 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:53.802 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:53.802 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7508 2024-05-08T08:52:53.803 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.803 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:53.803 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.803 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:53.803 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748180+0000", 2024-05-08T08:52:53.803 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:53.803 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.803 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.803 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:53.803 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748180+0000", 2024-05-08T08:52:53.803 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:53.803 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.803 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.803 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:53.804 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748180+0000", 2024-05-08T08:52:53.804 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4499999999999997e-07 2024-05-08T08:52:53.804 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.804 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.804 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:53.804 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748183+0000", 2024-05-08T08:52:53.804 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1660000000000001e-06 2024-05-08T08:52:53.804 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.804 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.804 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:53.804 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748197+0000", 2024-05-08T08:52:53.804 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.4877e-05 2024-05-08T08:52:53.804 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.804 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.804 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:53.805 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748203+0000", 2024-05-08T08:52:53.805 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7470000000000002e-06 2024-05-08T08:52:53.805 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.805 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.805 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:53.805 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.805398+0000", 2024-05-08T08:52:53.805 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057194513000000002 2024-05-08T08:52:53.805 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.805 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.805 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:53.805 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.805544+0000", 2024-05-08T08:52:53.805 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000146522 2024-05-08T08:52:53.805 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.805 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.805 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:53.806 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.805600+0000", 2024-05-08T08:52:53.806 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5538999999999998e-05 2024-05-08T08:52:53.806 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:53.806 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:53.806 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:53.806 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.806 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.806 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7509 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 778240~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:53.806 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.748211+0000", 2024-05-08T08:52:53.806 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 16.033707344, 2024-05-08T08:52:53.806 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057619034999999999, 2024-05-08T08:52:53.806 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:53.806 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:53.806 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:53.807 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:53.807 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:53.807 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:53.807 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7509 2024-05-08T08:52:53.807 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.807 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:53.807 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.807 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:53.807 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748211+0000", 2024-05-08T08:52:53.807 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:53.807 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.807 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.807 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:53.807 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748211+0000", 2024-05-08T08:52:53.808 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:53.808 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.808 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.808 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:53.808 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748212+0000", 2024-05-08T08:52:53.808 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8100000000000003e-07 2024-05-08T08:52:53.808 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.808 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.808 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:53.808 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748214+0000", 2024-05-08T08:52:53.808 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4669999999999998e-06 2024-05-08T08:52:53.808 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.808 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.809 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:53.809 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748221+0000", 2024-05-08T08:52:53.809 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5769999999999999e-06 2024-05-08T08:52:53.809 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.809 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.809 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:53.809 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748225+0000", 2024-05-08T08:52:53.809 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8060000000000002e-06 2024-05-08T08:52:53.809 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.809 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.809 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:53.809 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.805636+0000", 2024-05-08T08:52:53.809 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057410309 2024-05-08T08:52:53.809 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.809 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.809 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:53.810 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.805723+0000", 2024-05-08T08:52:53.810 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7435999999999994e-05 2024-05-08T08:52:53.810 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.810 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.810 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:53.810 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.805830+0000", 2024-05-08T08:52:53.810 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000106959 2024-05-08T08:52:53.810 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:53.810 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:53.810 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:53.810 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.810 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.810 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7510 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 819200~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:53.811 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.748232+0000", 2024-05-08T08:52:53.811 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 16.033686314000001, 2024-05-08T08:52:53.811 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057710421999999997, 2024-05-08T08:52:53.811 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:53.811 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:53.811 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:53.811 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:53.811 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:53.811 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:53.811 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7510 2024-05-08T08:52:53.811 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.811 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:53.811 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.811 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:53.811 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748232+0000", 2024-05-08T08:52:53.812 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:53.812 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.812 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.812 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:53.812 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748232+0000", 2024-05-08T08:52:53.812 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:53.812 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.812 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.812 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:53.812 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.749959+0000", 2024-05-08T08:52:53.812 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001727 2024-05-08T08:52:53.812 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.812 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.812 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:53.812 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.749964+0000", 2024-05-08T08:52:53.812 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4700000000000004e-06 2024-05-08T08:52:53.813 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.813 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.813 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:53.813 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.749984+0000", 2024-05-08T08:52:53.813 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0068000000000002e-05 2024-05-08T08:52:53.813 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.813 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.813 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:53.813 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.749993+0000", 2024-05-08T08:52:53.813 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4720000000000001e-06 2024-05-08T08:52:53.813 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.813 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.813 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:53.813 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.805846+0000", 2024-05-08T08:52:53.813 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.055852770000000003 2024-05-08T08:52:53.813 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.814 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.814 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:53.814 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.805890+0000", 2024-05-08T08:52:53.814 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4323999999999998e-05 2024-05-08T08:52:53.814 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.814 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.814 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:53.814 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.805942+0000", 2024-05-08T08:52:53.814 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2318000000000003e-05 2024-05-08T08:52:53.814 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:53.814 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:53.814 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:53.814 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.814 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.815 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7515 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 1024000~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:53.815 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.750135+0000", 2024-05-08T08:52:53.815 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 16.031782937999999, 2024-05-08T08:52:53.815 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057188125999999999, 2024-05-08T08:52:53.815 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:53.815 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:53.815 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:53.815 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:53.815 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:53.815 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:53.815 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7515 2024-05-08T08:52:53.815 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.815 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:53.815 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.815 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:53.816 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750135+0000", 2024-05-08T08:52:53.816 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:53.816 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.816 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.816 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:53.816 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750135+0000", 2024-05-08T08:52:53.816 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:53.816 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.816 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.816 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:53.816 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750136+0000", 2024-05-08T08:52:53.816 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8100000000000003e-07 2024-05-08T08:52:53.816 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.816 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.816 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:53.816 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750138+0000", 2024-05-08T08:52:53.817 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.339e-06 2024-05-08T08:52:53.817 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.817 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.817 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:53.817 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750147+0000", 2024-05-08T08:52:53.817 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7669999999999992e-06 2024-05-08T08:52:53.817 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.817 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.817 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:53.817 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750153+0000", 2024-05-08T08:52:53.817 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5029999999999997e-06 2024-05-08T08:52:53.817 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.817 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.817 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:53.817 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.807145+0000", 2024-05-08T08:52:53.817 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.056992174 2024-05-08T08:52:53.818 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.818 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.818 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:53.818 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.807231+0000", 2024-05-08T08:52:53.818 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.6347000000000005e-05 2024-05-08T08:52:53.818 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.818 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.818 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:53.818 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.807324+0000", 2024-05-08T08:52:53.818 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2515000000000004e-05 2024-05-08T08:52:53.818 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:53.818 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:53.818 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:53.818 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.818 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.819 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7516 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 1064960~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:53.819 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.750162+0000", 2024-05-08T08:52:53.819 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 16.031756109, 2024-05-08T08:52:53.819 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057363971999999999, 2024-05-08T08:52:53.819 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:53.819 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:53.819 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:53.819 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:53.819 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:53.819 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:53.819 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7516 2024-05-08T08:52:53.819 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.819 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:53.819 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.819 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:53.819 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750162+0000", 2024-05-08T08:52:53.820 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:53.820 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.820 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.820 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:53.820 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750162+0000", 2024-05-08T08:52:53.820 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:53.820 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.820 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.820 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:53.820 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750163+0000", 2024-05-08T08:52:53.820 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.13e-07 2024-05-08T08:52:53.820 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.820 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.820 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:53.820 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750165+0000", 2024-05-08T08:52:53.820 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2309999999999999e-06 2024-05-08T08:52:53.821 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.821 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.821 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:53.821 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750174+0000", 2024-05-08T08:52:53.821 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4740000000000005e-06 2024-05-08T08:52:53.821 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.821 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.821 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:53.821 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750180+0000", 2024-05-08T08:52:53.821 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4760000000000003e-06 2024-05-08T08:52:53.821 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.821 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.821 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:53.821 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.807355+0000", 2024-05-08T08:52:53.821 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057175029000000002 2024-05-08T08:52:53.821 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.822 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.822 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:53.822 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.807437+0000", 2024-05-08T08:52:53.822 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.2268999999999994e-05 2024-05-08T08:52:53.822 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.822 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.822 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:53.822 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.807526+0000", 2024-05-08T08:52:53.822 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.8880000000000003e-05 2024-05-08T08:52:53.822 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:53.822 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:53.822 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:53.822 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.822 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.822 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7517 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 1105920~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:53.823 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.750188+0000", 2024-05-08T08:52:53.823 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 16.031730616000001, 2024-05-08T08:52:53.823 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057547099999999997, 2024-05-08T08:52:53.823 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:53.823 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:53.823 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:53.823 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:53.823 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:53.823 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:53.823 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7517 2024-05-08T08:52:53.823 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.823 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:53.823 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.823 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:53.823 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750188+0000", 2024-05-08T08:52:53.824 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:53.824 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.824 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.824 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:53.824 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750188+0000", 2024-05-08T08:52:53.824 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:53.824 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.824 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.824 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:53.824 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750188+0000", 2024-05-08T08:52:53.824 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7199999999999999e-07 2024-05-08T08:52:53.824 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.824 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.824 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:53.824 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750190+0000", 2024-05-08T08:52:53.825 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.154e-06 2024-05-08T08:52:53.825 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.825 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.825 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:53.825 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750198+0000", 2024-05-08T08:52:53.825 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8320000000000006e-06 2024-05-08T08:52:53.825 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.825 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.825 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:53.825 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750203+0000", 2024-05-08T08:52:53.825 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1900000000000003e-06 2024-05-08T08:52:53.825 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.825 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.825 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:53.825 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.807555+0000", 2024-05-08T08:52:53.826 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057351148999999997 2024-05-08T08:52:53.826 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.826 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.826 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:53.826 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.807640+0000", 2024-05-08T08:52:53.826 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.5853999999999995e-05 2024-05-08T08:52:53.826 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.826 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.826 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:53.826 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.807735+0000", 2024-05-08T08:52:53.826 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4449000000000003e-05 2024-05-08T08:52:53.826 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:53.826 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:53.826 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:53.827 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.827 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.827 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7518 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 1146880~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:53.827 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.750212+0000", 2024-05-08T08:52:53.827 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 16.031706816, 2024-05-08T08:52:53.827 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057664963999999999, 2024-05-08T08:52:53.827 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:53.827 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:53.827 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:53.827 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:53.827 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:53.827 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:53.827 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7518 2024-05-08T08:52:53.827 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.828 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:53.828 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.828 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:53.828 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750212+0000", 2024-05-08T08:52:53.828 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:53.828 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.828 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.828 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:53.828 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750212+0000", 2024-05-08T08:52:53.828 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:53.828 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.828 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.828 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:53.828 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750212+0000", 2024-05-08T08:52:53.829 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.46e-07 2024-05-08T08:52:53.829 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.829 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.829 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:53.829 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750220+0000", 2024-05-08T08:52:53.829 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5610000000000001e-06 2024-05-08T08:52:53.829 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.829 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.829 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:53.829 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750228+0000", 2024-05-08T08:52:53.829 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3180000000000007e-06 2024-05-08T08:52:53.829 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.829 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.829 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:53.829 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750234+0000", 2024-05-08T08:52:53.829 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0209999999999996e-06 2024-05-08T08:52:53.830 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.830 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.830 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:53.830 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.807765+0000", 2024-05-08T08:52:53.830 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057531481000000002 2024-05-08T08:52:53.830 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.830 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.830 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:53.830 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.807815+0000", 2024-05-08T08:52:53.830 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9282e-05 2024-05-08T08:52:53.830 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.830 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.830 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:53.830 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.807877+0000", 2024-05-08T08:52:53.830 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1854999999999996e-05 2024-05-08T08:52:53.831 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:53.831 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:53.831 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:53.831 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.831 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.831 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7519 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 1187840~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:53.831 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.750261+0000", 2024-05-08T08:52:53.831 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 16.031657330000002, 2024-05-08T08:52:53.831 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057872519999999997, 2024-05-08T08:52:53.831 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:53.831 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:53.831 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:53.831 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:53.831 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:53.832 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:53.832 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7519 2024-05-08T08:52:53.832 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.832 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:53.832 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.832 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:53.832 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750261+0000", 2024-05-08T08:52:53.832 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:53.832 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.832 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.832 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:53.832 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750261+0000", 2024-05-08T08:52:53.832 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:53.832 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.832 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.832 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:53.833 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750262+0000", 2024-05-08T08:52:53.833 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7000000000000005e-07 2024-05-08T08:52:53.833 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.833 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.833 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:53.833 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750265+0000", 2024-05-08T08:52:53.833 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4589999999999998e-06 2024-05-08T08:52:53.833 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.833 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.833 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:53.833 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750274+0000", 2024-05-08T08:52:53.833 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.5459999999999992e-06 2024-05-08T08:52:53.833 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.833 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.834 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:53.834 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750280+0000", 2024-05-08T08:52:53.834 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0950000000000001e-06 2024-05-08T08:52:53.834 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.834 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.834 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:53.834 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.807917+0000", 2024-05-08T08:52:53.834 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057637715999999999 2024-05-08T08:52:53.834 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.834 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.834 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:53.834 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.808013+0000", 2024-05-08T08:52:53.834 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5569999999999995e-05 2024-05-08T08:52:53.834 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.835 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.835 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:53.835 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.808134+0000", 2024-05-08T08:52:53.835 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012056400000000001 2024-05-08T08:52:53.835 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:53.835 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:53.835 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:53.835 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.835 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.835 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7520 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 1228800~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:53.835 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.750289+0000", 2024-05-08T08:52:53.835 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 16.031628886, 2024-05-08T08:52:53.835 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.058133788999999998, 2024-05-08T08:52:53.835 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:53.835 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:53.836 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:53.836 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:53.836 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:53.836 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:53.836 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7520 2024-05-08T08:52:53.836 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.836 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:53.836 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.836 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:53.836 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750289+0000", 2024-05-08T08:52:53.836 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:53.836 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.836 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.836 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:53.837 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750289+0000", 2024-05-08T08:52:53.837 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:53.837 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.837 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.837 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:53.837 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.751976+0000", 2024-05-08T08:52:53.837 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0016865389999999999 2024-05-08T08:52:53.837 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.837 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.837 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:53.837 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.751981+0000", 2024-05-08T08:52:53.837 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5719999999999996e-06 2024-05-08T08:52:53.837 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.837 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.838 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:53.838 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.751991+0000", 2024-05-08T08:52:53.838 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0788000000000001e-05 2024-05-08T08:52:53.838 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.838 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.838 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:53.838 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.751999+0000", 2024-05-08T08:52:53.838 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2690000000000002e-06 2024-05-08T08:52:53.838 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.838 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.838 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:53.838 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.808183+0000", 2024-05-08T08:52:53.838 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.056184686999999997 2024-05-08T08:52:53.838 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.838 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.839 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:53.839 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.808296+0000", 2024-05-08T08:52:53.839 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000112754 2024-05-08T08:52:53.839 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.839 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.839 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:53.839 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.808423+0000", 2024-05-08T08:52:53.839 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012718000000000001 2024-05-08T08:52:53.839 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:53.839 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:53.839 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:53.839 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.839 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.839 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7524 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 1392640~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:53.839 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.752121+0000", 2024-05-08T08:52:53.839 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 16.029797837, 2024-05-08T08:52:53.839 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057199521000000003, 2024-05-08T08:52:53.840 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:53.840 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:53.840 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:53.840 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:53.840 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:53.840 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:53.840 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7524 2024-05-08T08:52:53.840 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.840 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:53.840 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.840 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:53.840 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752121+0000", 2024-05-08T08:52:53.840 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:53.840 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.840 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.840 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:53.840 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752121+0000", 2024-05-08T08:52:53.841 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:53.841 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.841 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.841 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:53.841 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752121+0000", 2024-05-08T08:52:53.841 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5999999999999999e-07 2024-05-08T08:52:53.841 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.841 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.841 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:53.841 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752123+0000", 2024-05-08T08:52:53.841 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3499999999999999e-06 2024-05-08T08:52:53.841 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.841 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.841 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:53.841 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752132+0000", 2024-05-08T08:52:53.841 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.5539999999999998e-06 2024-05-08T08:52:53.841 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.842 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.842 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:53.842 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752138+0000", 2024-05-08T08:52:53.842 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3640000000000004e-06 2024-05-08T08:52:53.842 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.842 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.842 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:53.842 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.809210+0000", 2024-05-08T08:52:53.842 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057071469999999999 2024-05-08T08:52:53.842 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.842 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.842 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:53.842 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.809258+0000", 2024-05-08T08:52:53.842 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8755000000000003e-05 2024-05-08T08:52:53.842 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.843 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.843 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:53.843 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.809320+0000", 2024-05-08T08:52:53.843 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1568000000000004e-05 2024-05-08T08:52:53.843 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:53.843 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:53.843 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:53.843 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.843 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.843 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7525 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 1433600~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:53.843 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.752146+0000", 2024-05-08T08:52:53.843 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 16.029772792999999, 2024-05-08T08:52:53.843 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057400072000000003, 2024-05-08T08:52:53.843 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:53.843 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:53.843 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:53.844 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:53.844 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:53.844 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:53.844 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7525 2024-05-08T08:52:53.844 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.844 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:53.844 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.844 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:53.844 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752146+0000", 2024-05-08T08:52:53.844 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:53.844 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.844 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.844 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:53.844 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752146+0000", 2024-05-08T08:52:53.844 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:53.845 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.845 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.845 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:53.845 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752146+0000", 2024-05-08T08:52:53.845 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0400000000000002e-07 2024-05-08T08:52:53.845 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.845 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.845 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:53.845 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752148+0000", 2024-05-08T08:52:53.845 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3209999999999999e-06 2024-05-08T08:52:53.845 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.845 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.845 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:53.845 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752155+0000", 2024-05-08T08:52:53.845 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6109999999999999e-06 2024-05-08T08:52:53.846 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.846 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.846 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:53.846 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752161+0000", 2024-05-08T08:52:53.846 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.489e-06 2024-05-08T08:52:53.846 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.846 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.846 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:53.846 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.809360+0000", 2024-05-08T08:52:53.846 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057198860999999997 2024-05-08T08:52:53.846 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.846 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.846 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:53.846 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.809450+0000", 2024-05-08T08:52:53.847 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0136000000000006e-05 2024-05-08T08:52:53.847 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.847 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.847 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:53.847 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.809546+0000", 2024-05-08T08:52:53.847 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5249999999999998e-05 2024-05-08T08:52:53.847 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:53.847 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:53.847 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:53.847 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.847 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.847 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7526 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 1474560~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:53.847 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.752168+0000", 2024-05-08T08:52:53.847 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 16.029750079999999, 2024-05-08T08:52:53.847 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057516818999999997, 2024-05-08T08:52:53.847 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:53.848 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:53.848 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:53.848 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:53.848 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:53.848 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:53.848 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7526 2024-05-08T08:52:53.848 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.848 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:53.848 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.848 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:53.848 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752168+0000", 2024-05-08T08:52:53.848 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:53.848 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.848 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.848 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:53.849 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752168+0000", 2024-05-08T08:52:53.849 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:53.849 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.849 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.849 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:53.849 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752169+0000", 2024-05-08T08:52:53.849 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4799999999999999e-07 2024-05-08T08:52:53.849 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.849 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.849 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:53.849 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752171+0000", 2024-05-08T08:52:53.849 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.289e-06 2024-05-08T08:52:53.849 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.849 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.849 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:53.849 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752177+0000", 2024-05-08T08:52:53.850 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3690000000000003e-06 2024-05-08T08:52:53.850 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.850 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.850 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:53.850 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752183+0000", 2024-05-08T08:52:53.850 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.152e-06 2024-05-08T08:52:53.850 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.850 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.850 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:53.850 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.809574+0000", 2024-05-08T08:52:53.850 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057391075 2024-05-08T08:52:53.850 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.850 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.850 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:53.850 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.809624+0000", 2024-05-08T08:52:53.851 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0673000000000001e-05 2024-05-08T08:52:53.851 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.851 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.851 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:53.851 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.809685+0000", 2024-05-08T08:52:53.851 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0813000000000002e-05 2024-05-08T08:52:53.851 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:53.851 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:53.851 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:53.851 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.851 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.851 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7527 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 1515520~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:53.851 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.752189+0000", 2024-05-08T08:52:53.851 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 16.029729152000002, 2024-05-08T08:52:53.851 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057722168999999997, 2024-05-08T08:52:53.852 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:53.852 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:53.852 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:53.852 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:53.852 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:53.852 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:53.852 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7527 2024-05-08T08:52:53.852 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.852 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:53.852 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.852 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:53.852 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752189+0000", 2024-05-08T08:52:53.852 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:53.852 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.852 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.852 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:53.853 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752189+0000", 2024-05-08T08:52:53.853 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:53.853 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.853 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.853 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:53.853 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752190+0000", 2024-05-08T08:52:53.853 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8799999999999998e-07 2024-05-08T08:52:53.853 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.853 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.853 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:53.853 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752192+0000", 2024-05-08T08:52:53.853 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9759999999999998e-06 2024-05-08T08:52:53.853 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.853 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.853 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:53.854 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752199+0000", 2024-05-08T08:52:53.854 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3289999999999996e-06 2024-05-08T08:52:53.854 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.854 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.854 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:53.854 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752205+0000", 2024-05-08T08:52:53.854 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2759999999999997e-06 2024-05-08T08:52:53.854 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.854 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.854 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:53.854 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.809726+0000", 2024-05-08T08:52:53.854 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057521249000000003 2024-05-08T08:52:53.854 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.854 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.855 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:53.855 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.809815+0000", 2024-05-08T08:52:53.855 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.8987999999999996e-05 2024-05-08T08:52:53.855 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.855 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.855 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:53.855 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.809911+0000", 2024-05-08T08:52:53.855 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5963000000000003e-05 2024-05-08T08:52:53.855 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:53.855 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:53.855 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:53.855 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.855 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.855 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7528 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 1556480~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:53.855 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.752212+0000", 2024-05-08T08:52:53.855 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 16.029706441999998, 2024-05-08T08:52:53.856 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057904956, 2024-05-08T08:52:53.856 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:53.856 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:53.856 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:53.856 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:53.856 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:53.856 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:53.856 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7528 2024-05-08T08:52:53.856 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.856 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:53.856 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.856 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:53.856 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752212+0000", 2024-05-08T08:52:53.856 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:53.856 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.857 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.857 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:53.857 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752212+0000", 2024-05-08T08:52:53.857 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:53.857 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.857 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.857 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:53.857 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752212+0000", 2024-05-08T08:52:53.857 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7800000000000002e-07 2024-05-08T08:52:53.857 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.857 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.857 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:53.857 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752215+0000", 2024-05-08T08:52:53.857 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2120000000000002e-06 2024-05-08T08:52:53.857 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.858 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.858 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:53.858 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752222+0000", 2024-05-08T08:52:53.858 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.6329999999999996e-06 2024-05-08T08:52:53.858 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.858 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.858 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:53.858 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752227+0000", 2024-05-08T08:52:53.858 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9289999999999997e-06 2024-05-08T08:52:53.858 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.858 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.858 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:53.858 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.809966+0000", 2024-05-08T08:52:53.858 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057738554999999997 2024-05-08T08:52:53.859 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.859 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.859 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:53.859 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.810052+0000", 2024-05-08T08:52:53.859 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.6540999999999997e-05 2024-05-08T08:52:53.859 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.859 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.859 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:53.859 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.810117+0000", 2024-05-08T08:52:53.859 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4608000000000002e-05 2024-05-08T08:52:53.859 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:53.859 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:53.859 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:53.859 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.859 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.859 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7529 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 1597440~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:53.860 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.752234+0000", 2024-05-08T08:52:53.860 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 16.029684429, 2024-05-08T08:52:53.860 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.058133851, 2024-05-08T08:52:53.860 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:53.860 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:53.860 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:53.860 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:53.860 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:53.860 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:53.860 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7529 2024-05-08T08:52:53.860 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.860 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:53.860 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.860 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:53.860 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752234+0000", 2024-05-08T08:52:53.860 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:53.861 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.861 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.861 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:53.861 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752234+0000", 2024-05-08T08:52:53.861 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:53.861 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.861 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.861 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:53.861 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.753959+0000", 2024-05-08T08:52:53.861 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0017251409999999999 2024-05-08T08:52:53.861 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.861 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.861 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:53.861 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.753963+0000", 2024-05-08T08:52:53.861 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2860000000000001e-06 2024-05-08T08:52:53.861 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.862 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.862 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:53.862 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.753973+0000", 2024-05-08T08:52:53.862 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5119999999999997e-06 2024-05-08T08:52:53.862 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.862 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.862 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:53.862 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.753979+0000", 2024-05-08T08:52:53.862 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3589999999999996e-06 2024-05-08T08:52:53.862 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.862 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.862 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:53.862 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.810182+0000", 2024-05-08T08:52:53.862 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.056202941999999999 2024-05-08T08:52:53.862 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.863 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.863 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:53.863 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.810270+0000", 2024-05-08T08:52:53.863 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7338999999999998e-05 2024-05-08T08:52:53.863 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:53.863 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:53.863 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:53.863 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.810368+0000", 2024-05-08T08:52:53.863 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8271999999999997e-05 2024-05-08T08:52:53.863 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:53.863 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:53.863 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:53.863 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:53.863 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:53.863 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:52:53.865 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:52:53.865 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:52:53.866 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:52:53.866 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:52:53.867 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:52:53.867 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:52:53.868+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:52:53.966 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:52:53.969+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:52:53.977 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:52:53.977 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:52:53.977 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:52:53.977 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:52:53.986 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:52:54.067 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:52:54.071+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:52:54.091 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:52:54.092 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:52:54.092 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:52:54.092 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:52:54.092 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:52:54.102 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:52:54.169 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:52:54.172+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:52:54.209 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:52:54.209 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:52:54.209 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:52:54.209 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:52:54.209 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.209 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4742 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1832960~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:54.209 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.585513+0000", 2024-05-08T08:52:54.209 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 22.625573364000001, 2024-05-08T08:52:54.209 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035588763000000002, 2024-05-08T08:52:54.209 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:54.209 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:54.210 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:54.210 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:54.210 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:54.210 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:54.210 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4742 2024-05-08T08:52:54.210 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.210 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:54.210 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.210 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:54.211 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585513+0000", 2024-05-08T08:52:54.211 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:54.211 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.211 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.211 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:54.211 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585513+0000", 2024-05-08T08:52:54.211 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:54.211 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.211 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.212 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:54.212 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585514+0000", 2024-05-08T08:52:54.212 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6100000000000001e-07 2024-05-08T08:52:54.212 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.212 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.212 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:54.212 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585516+0000", 2024-05-08T08:52:54.212 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.526e-06 2024-05-08T08:52:54.212 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.212 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.212 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:54.212 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585523+0000", 2024-05-08T08:52:54.212 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.562e-06 2024-05-08T08:52:54.212 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.213 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.213 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:54.213 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585528+0000", 2024-05-08T08:52:54.213 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7990000000000001e-06 2024-05-08T08:52:54.213 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.213 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.213 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:54.213 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.620939+0000", 2024-05-08T08:52:54.213 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035411419 2024-05-08T08:52:54.213 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.213 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.213 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:54.213 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621014+0000", 2024-05-08T08:52:54.213 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5017000000000004e-05 2024-05-08T08:52:54.214 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.214 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.214 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:54.214 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621102+0000", 2024-05-08T08:52:54.214 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7979000000000006e-05 2024-05-08T08:52:54.214 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.214 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:54.214 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.214 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.214 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.214 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4743 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1849344~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:54.215 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.585628+0000", 2024-05-08T08:52:54.215 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 22.625458476999999, 2024-05-08T08:52:54.215 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035659962000000003, 2024-05-08T08:52:54.215 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:54.215 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:54.215 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:54.215 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:54.215 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:54.215 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:54.215 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4743 2024-05-08T08:52:54.215 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.215 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:54.215 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.215 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:54.216 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585628+0000", 2024-05-08T08:52:54.216 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:54.216 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.216 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.216 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:54.216 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585628+0000", 2024-05-08T08:52:54.216 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:54.216 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.216 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.216 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:54.216 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585629+0000", 2024-05-08T08:52:54.216 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2300000000000001e-07 2024-05-08T08:52:54.216 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.216 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.217 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:54.217 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585632+0000", 2024-05-08T08:52:54.217 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1370000000000002e-06 2024-05-08T08:52:54.217 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.217 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.217 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:54.217 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585641+0000", 2024-05-08T08:52:54.217 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5220000000000007e-06 2024-05-08T08:52:54.217 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.217 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.217 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:54.217 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585646+0000", 2024-05-08T08:52:54.217 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.609e-06 2024-05-08T08:52:54.217 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.217 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.218 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:54.218 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621132+0000", 2024-05-08T08:52:54.218 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035485472999999997 2024-05-08T08:52:54.218 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.218 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.218 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:54.218 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621204+0000", 2024-05-08T08:52:54.218 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2259000000000001e-05 2024-05-08T08:52:54.218 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.218 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.218 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:54.218 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621288+0000", 2024-05-08T08:52:54.218 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4338999999999993e-05 2024-05-08T08:52:54.218 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.219 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:54.219 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.219 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.219 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.219 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4746 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1865728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:54.219 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.585656+0000", 2024-05-08T08:52:54.219 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 22.625430795, 2024-05-08T08:52:54.219 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035815909, 2024-05-08T08:52:54.219 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:54.219 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:54.219 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:54.219 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:54.219 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:54.219 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:54.219 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4746 2024-05-08T08:52:54.220 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.220 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:54.220 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.220 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:54.220 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585656+0000", 2024-05-08T08:52:54.220 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:54.220 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.220 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.220 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:54.220 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585656+0000", 2024-05-08T08:52:54.220 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:54.220 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.220 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.220 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:54.220 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585656+0000", 2024-05-08T08:52:54.220 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.08e-07 2024-05-08T08:52:54.221 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.221 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.221 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:54.221 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585658+0000", 2024-05-08T08:52:54.221 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0889999999999998e-06 2024-05-08T08:52:54.221 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.221 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.221 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:54.221 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585664+0000", 2024-05-08T08:52:54.221 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3990000000000003e-06 2024-05-08T08:52:54.221 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.221 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.221 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:54.221 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585668+0000", 2024-05-08T08:52:54.221 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3170000000000003e-06 2024-05-08T08:52:54.222 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.222 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.222 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:54.222 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621316+0000", 2024-05-08T08:52:54.222 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035647908999999998 2024-05-08T08:52:54.222 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.222 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.222 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:54.222 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621389+0000", 2024-05-08T08:52:54.222 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2531000000000001e-05 2024-05-08T08:52:54.222 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.222 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.222 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:54.222 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621472+0000", 2024-05-08T08:52:54.222 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3256000000000004e-05 2024-05-08T08:52:54.223 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.223 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:54.223 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.223 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.223 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.223 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4748 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1882112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:54.223 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.585697+0000", 2024-05-08T08:52:54.223 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 22.625389348999999, 2024-05-08T08:52:54.223 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035962485000000002, 2024-05-08T08:52:54.223 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:54.223 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:54.223 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:54.223 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:54.223 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:54.224 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:54.224 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4748 2024-05-08T08:52:54.224 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.224 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:54.224 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.224 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:54.224 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585697+0000", 2024-05-08T08:52:54.224 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:54.224 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.224 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.224 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:54.224 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585697+0000", 2024-05-08T08:52:54.224 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:54.224 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.225 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.225 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:54.225 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585698+0000", 2024-05-08T08:52:54.225 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0900000000000001e-07 2024-05-08T08:52:54.225 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.225 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.225 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:54.225 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585701+0000", 2024-05-08T08:52:54.225 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0149999999999999e-06 2024-05-08T08:52:54.225 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.225 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.225 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:54.225 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585712+0000", 2024-05-08T08:52:54.225 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0655e-05 2024-05-08T08:52:54.225 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.226 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.226 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:54.226 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585717+0000", 2024-05-08T08:52:54.226 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3800000000000002e-06 2024-05-08T08:52:54.226 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.226 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.226 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:54.226 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621502+0000", 2024-05-08T08:52:54.226 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035784559000000001 2024-05-08T08:52:54.226 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.226 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.226 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:54.226 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621575+0000", 2024-05-08T08:52:54.226 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3269999999999995e-05 2024-05-08T08:52:54.227 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.227 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.227 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:54.227 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621660+0000", 2024-05-08T08:52:54.227 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4997e-05 2024-05-08T08:52:54.227 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.227 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:54.227 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.227 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.227 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.227 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4750 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1898496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:54.227 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.585810+0000", 2024-05-08T08:52:54.227 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 22.625276519, 2024-05-08T08:52:54.227 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036034928000000001, 2024-05-08T08:52:54.227 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:54.228 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:54.228 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:54.228 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:54.228 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:54.228 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:54.228 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4750 2024-05-08T08:52:54.228 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.228 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:54.228 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.228 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:54.228 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585810+0000", 2024-05-08T08:52:54.228 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:54.228 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.228 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.228 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:54.229 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585810+0000", 2024-05-08T08:52:54.229 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:54.229 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.229 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.229 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:54.229 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585811+0000", 2024-05-08T08:52:54.229 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6499999999999999e-07 2024-05-08T08:52:54.229 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.229 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.229 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:54.229 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585814+0000", 2024-05-08T08:52:54.229 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1709999999999998e-06 2024-05-08T08:52:54.229 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.229 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.229 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:54.230 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585822+0000", 2024-05-08T08:52:54.230 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8909999999999995e-06 2024-05-08T08:52:54.230 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.230 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.230 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:54.230 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585827+0000", 2024-05-08T08:52:54.230 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5289999999999999e-06 2024-05-08T08:52:54.230 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.230 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.230 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:54.230 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621688+0000", 2024-05-08T08:52:54.230 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035861111000000001 2024-05-08T08:52:54.230 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.230 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.231 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:54.231 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621761+0000", 2024-05-08T08:52:54.231 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2211000000000004e-05 2024-05-08T08:52:54.231 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.231 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.231 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:54.231 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621845+0000", 2024-05-08T08:52:54.231 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4450000000000006e-05 2024-05-08T08:52:54.231 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.231 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:54.231 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.231 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.231 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.231 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4752 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1914880~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:54.231 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.585858+0000", 2024-05-08T08:52:54.232 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 22.625228549999999, 2024-05-08T08:52:54.232 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036180081000000003, 2024-05-08T08:52:54.232 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:54.232 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:54.232 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:54.232 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:54.232 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:54.232 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:54.232 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4752 2024-05-08T08:52:54.232 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.232 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:54.232 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.232 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:54.232 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585858+0000", 2024-05-08T08:52:54.233 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:54.233 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.233 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.233 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:54.233 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585858+0000", 2024-05-08T08:52:54.233 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:54.233 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.233 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.233 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:54.233 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585859+0000", 2024-05-08T08:52:54.233 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.82e-07 2024-05-08T08:52:54.233 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.233 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.233 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:54.233 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585861+0000", 2024-05-08T08:52:54.234 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.695e-06 2024-05-08T08:52:54.234 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.234 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.234 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:54.234 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585869+0000", 2024-05-08T08:52:54.234 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0299999999999994e-06 2024-05-08T08:52:54.234 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.234 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.234 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:54.234 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585874+0000", 2024-05-08T08:52:54.234 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9620000000000001e-06 2024-05-08T08:52:54.234 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.234 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.234 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:54.234 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621878+0000", 2024-05-08T08:52:54.235 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036004043999999999 2024-05-08T08:52:54.235 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.235 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.235 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:54.235 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621954+0000", 2024-05-08T08:52:54.235 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5414000000000007e-05 2024-05-08T08:52:54.235 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.235 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.235 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:54.235 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622038+0000", 2024-05-08T08:52:54.235 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4454000000000002e-05 2024-05-08T08:52:54.235 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.235 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:54.235 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.235 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.236 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.236 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4753 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1931264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:54.236 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.585947+0000", 2024-05-08T08:52:54.236 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 22.625139902000001, 2024-05-08T08:52:54.236 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036267992999999998, 2024-05-08T08:52:54.236 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:54.236 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:54.236 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:54.236 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:54.236 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:54.236 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:54.236 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4753 2024-05-08T08:52:54.236 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.236 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:54.236 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.237 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:54.237 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585947+0000", 2024-05-08T08:52:54.237 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:54.237 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.237 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.237 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:54.237 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585947+0000", 2024-05-08T08:52:54.237 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:54.237 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.237 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.237 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:54.237 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585948+0000", 2024-05-08T08:52:54.237 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6000000000000002e-07 2024-05-08T08:52:54.237 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.237 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.238 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:54.238 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585952+0000", 2024-05-08T08:52:54.238 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7090000000000001e-06 2024-05-08T08:52:54.238 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.238 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.238 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:54.238 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585963+0000", 2024-05-08T08:52:54.238 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0740000000000001e-05 2024-05-08T08:52:54.238 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.238 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.238 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:54.238 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585970+0000", 2024-05-08T08:52:54.238 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3309999999999998e-06 2024-05-08T08:52:54.238 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.239 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.239 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:54.239 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622065+0000", 2024-05-08T08:52:54.239 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036094676999999999 2024-05-08T08:52:54.239 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.239 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.239 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:54.239 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622132+0000", 2024-05-08T08:52:54.239 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7214999999999999e-05 2024-05-08T08:52:54.239 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.239 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.239 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:54.239 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622215+0000", 2024-05-08T08:52:54.239 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2460999999999995e-05 2024-05-08T08:52:54.240 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.240 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:54.240 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.240 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.240 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.240 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4756 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1947648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:54.240 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586018+0000", 2024-05-08T08:52:54.240 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 22.625068257999999, 2024-05-08T08:52:54.240 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036382335000000002, 2024-05-08T08:52:54.240 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:54.240 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:54.240 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:54.240 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:54.240 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:54.241 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:54.241 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4756 2024-05-08T08:52:54.241 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.241 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:54.241 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.241 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:54.241 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586018+0000", 2024-05-08T08:52:54.241 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:54.241 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.241 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.241 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:54.241 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586018+0000", 2024-05-08T08:52:54.241 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:54.241 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.241 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.242 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:54.242 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586019+0000", 2024-05-08T08:52:54.242 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9999999999999997e-07 2024-05-08T08:52:54.242 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.242 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.242 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:54.242 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586023+0000", 2024-05-08T08:52:54.242 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7330000000000001e-06 2024-05-08T08:52:54.242 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.242 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.242 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:54.242 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586031+0000", 2024-05-08T08:52:54.242 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4940000000000003e-06 2024-05-08T08:52:54.242 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.243 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.243 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:54.243 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586038+0000", 2024-05-08T08:52:54.243 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2539999999999998e-06 2024-05-08T08:52:54.243 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.243 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.243 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:54.243 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622243+0000", 2024-05-08T08:52:54.243 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036205009000000003 2024-05-08T08:52:54.243 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.243 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.243 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:54.243 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622313+0000", 2024-05-08T08:52:54.243 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9912000000000004e-05 2024-05-08T08:52:54.243 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.244 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.244 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:54.244 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622401+0000", 2024-05-08T08:52:54.244 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8232999999999994e-05 2024-05-08T08:52:54.244 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.244 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:54.244 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.244 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.244 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.244 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4758 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1964032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:54.244 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586080+0000", 2024-05-08T08:52:54.244 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 22.625006839000001, 2024-05-08T08:52:54.244 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036547659000000003, 2024-05-08T08:52:54.244 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:54.244 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:54.245 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:54.245 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:54.245 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:54.245 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:54.245 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4758 2024-05-08T08:52:54.245 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.245 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:54.245 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.245 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:54.245 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586080+0000", 2024-05-08T08:52:54.245 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:54.245 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.245 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.245 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:54.246 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586080+0000", 2024-05-08T08:52:54.246 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:54.246 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.246 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.246 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:54.246 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586081+0000", 2024-05-08T08:52:54.246 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5499999999999998e-07 2024-05-08T08:52:54.246 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.246 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.246 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:54.246 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586084+0000", 2024-05-08T08:52:54.246 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9919999999999999e-06 2024-05-08T08:52:54.246 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.246 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.246 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:54.247 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586091+0000", 2024-05-08T08:52:54.247 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5440000000000001e-06 2024-05-08T08:52:54.247 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.247 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.247 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:54.247 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586098+0000", 2024-05-08T08:52:54.247 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8639999999999998e-06 2024-05-08T08:52:54.247 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.247 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.247 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:54.247 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622442+0000", 2024-05-08T08:52:54.247 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036344110999999998 2024-05-08T08:52:54.247 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.247 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.248 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:54.248 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622516+0000", 2024-05-08T08:52:54.248 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4089000000000002e-05 2024-05-08T08:52:54.248 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.248 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.248 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:54.248 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622628+0000", 2024-05-08T08:52:54.248 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00011140400000000001 2024-05-08T08:52:54.248 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.248 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:54.248 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.248 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.248 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.248 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4760 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1980416~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:54.248 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586178+0000", 2024-05-08T08:52:54.249 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 22.624908226999999, 2024-05-08T08:52:54.249 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036647526999999999, 2024-05-08T08:52:54.249 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:54.249 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:54.249 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:54.249 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:54.249 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:54.249 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:54.249 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4760 2024-05-08T08:52:54.249 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.249 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:54.249 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.249 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:54.249 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586178+0000", 2024-05-08T08:52:54.249 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:54.250 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.250 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.250 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:54.250 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586178+0000", 2024-05-08T08:52:54.250 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:54.250 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.250 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.250 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:54.250 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586179+0000", 2024-05-08T08:52:54.250 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8599999999999997e-07 2024-05-08T08:52:54.250 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.250 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.250 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:54.250 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586182+0000", 2024-05-08T08:52:54.251 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.174e-06 2024-05-08T08:52:54.251 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.251 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.251 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:54.251 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586191+0000", 2024-05-08T08:52:54.251 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1869999999999997e-06 2024-05-08T08:52:54.251 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.251 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.251 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:54.251 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586197+0000", 2024-05-08T08:52:54.251 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2090000000000002e-06 2024-05-08T08:52:54.251 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.251 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.251 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:54.252 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622657+0000", 2024-05-08T08:52:54.252 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036459793999999997 2024-05-08T08:52:54.252 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.252 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.252 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:54.252 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622725+0000", 2024-05-08T08:52:54.252 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8374999999999995e-05 2024-05-08T08:52:54.252 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.252 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.252 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:54.252 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622826+0000", 2024-05-08T08:52:54.252 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010100200000000001 2024-05-08T08:52:54.252 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.252 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:54.252 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.253 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.253 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.253 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4762 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1996800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:54.253 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586299+0000", 2024-05-08T08:52:54.253 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 22.624787896000001, 2024-05-08T08:52:54.253 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036720653999999998, 2024-05-08T08:52:54.253 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:54.253 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:54.253 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:54.253 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:54.253 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:54.253 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:54.253 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4762 2024-05-08T08:52:54.253 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.253 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:54.253 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.254 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:54.254 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586299+0000", 2024-05-08T08:52:54.254 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:54.254 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.254 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.254 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:54.254 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586299+0000", 2024-05-08T08:52:54.254 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:54.254 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.254 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.254 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:54.254 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586299+0000", 2024-05-08T08:52:54.254 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.99e-07 2024-05-08T08:52:54.255 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.255 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.255 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:54.255 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586303+0000", 2024-05-08T08:52:54.255 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1949999999999999e-06 2024-05-08T08:52:54.255 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.255 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.255 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:54.255 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586318+0000", 2024-05-08T08:52:54.255 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5401999999999999e-05 2024-05-08T08:52:54.255 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.255 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.255 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:54.255 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586324+0000", 2024-05-08T08:52:54.255 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8119999999999996e-06 2024-05-08T08:52:54.255 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.256 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.256 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:54.256 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622853+0000", 2024-05-08T08:52:54.256 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036529013999999999 2024-05-08T08:52:54.256 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.256 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.256 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:54.256 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622924+0000", 2024-05-08T08:52:54.256 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1654000000000002e-05 2024-05-08T08:52:54.256 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.256 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.256 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:54.256 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623019+0000", 2024-05-08T08:52:54.256 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4977999999999998e-05 2024-05-08T08:52:54.256 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.257 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:54.257 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.257 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.257 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.257 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4764 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 2013184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:54.257 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586506+0000", 2024-05-08T08:52:54.257 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 22.624580791, 2024-05-08T08:52:54.257 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036859467, 2024-05-08T08:52:54.257 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:54.257 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:54.257 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:54.257 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:54.257 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:54.257 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:54.257 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4764 2024-05-08T08:52:54.257 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.258 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:54.258 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.258 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:54.258 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586506+0000", 2024-05-08T08:52:54.258 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:54.258 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.258 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.258 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:54.258 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586506+0000", 2024-05-08T08:52:54.258 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:54.258 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.258 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.258 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:54.259 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586507+0000", 2024-05-08T08:52:54.259 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9100000000000003e-07 2024-05-08T08:52:54.259 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.259 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.259 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:54.259 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586512+0000", 2024-05-08T08:52:54.259 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9470000000000002e-06 2024-05-08T08:52:54.259 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.259 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.259 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:54.259 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586521+0000", 2024-05-08T08:52:54.259 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3379999999999994e-06 2024-05-08T08:52:54.259 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.259 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.259 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:54.260 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586528+0000", 2024-05-08T08:52:54.260 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1289999999999998e-06 2024-05-08T08:52:54.260 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.260 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.260 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:54.260 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623095+0000", 2024-05-08T08:52:54.260 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036566411 2024-05-08T08:52:54.260 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.260 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.260 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:54.260 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623221+0000", 2024-05-08T08:52:54.260 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012622299999999999 2024-05-08T08:52:54.260 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.260 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.260 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:54.261 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623365+0000", 2024-05-08T08:52:54.261 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000144628 2024-05-08T08:52:54.261 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.261 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:54.261 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.261 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.261 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.261 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4766 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 2029568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:54.261 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586543+0000", 2024-05-08T08:52:54.261 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 22.624543303999999, 2024-05-08T08:52:54.261 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.037033635000000002, 2024-05-08T08:52:54.261 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:54.261 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:54.261 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:54.262 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:54.262 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:54.262 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:54.262 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4766 2024-05-08T08:52:54.262 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.262 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:54.262 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.262 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:54.262 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586543+0000", 2024-05-08T08:52:54.263 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:54.263 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.263 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.263 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:54.263 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586543+0000", 2024-05-08T08:52:54.263 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:54.263 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.263 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.263 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:54.263 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586544+0000", 2024-05-08T08:52:54.263 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3099999999999998e-07 2024-05-08T08:52:54.263 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.263 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.264 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:54.264 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586547+0000", 2024-05-08T08:52:54.264 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.751e-06 2024-05-08T08:52:54.264 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.264 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.264 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:54.264 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586553+0000", 2024-05-08T08:52:54.264 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2870000000000001e-06 2024-05-08T08:52:54.264 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.264 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.264 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:54.265 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586558+0000", 2024-05-08T08:52:54.265 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5709999999999997e-06 2024-05-08T08:52:54.265 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.265 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.265 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:54.265 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623416+0000", 2024-05-08T08:52:54.265 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036857613999999997 2024-05-08T08:52:54.265 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.265 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.265 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:54.265 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623489+0000", 2024-05-08T08:52:54.265 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3172999999999999e-05 2024-05-08T08:52:54.265 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.265 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.266 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:54.266 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623577+0000", 2024-05-08T08:52:54.266 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7807999999999997e-05 2024-05-08T08:52:54.266 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.266 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:54.266 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.266 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.266 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.266 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4768 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 2045952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:54.266 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586566+0000", 2024-05-08T08:52:54.266 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 22.624520998000001, 2024-05-08T08:52:54.267 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.03726206, 2024-05-08T08:52:54.267 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:54.267 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:54.267 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:54.267 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:54.267 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:54.267 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:54.267 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4768 2024-05-08T08:52:54.267 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.267 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:54.267 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.267 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:54.267 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586566+0000", 2024-05-08T08:52:54.268 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:54.268 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.268 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.268 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:54.268 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586566+0000", 2024-05-08T08:52:54.268 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:54.268 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.268 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.268 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:54.268 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586566+0000", 2024-05-08T08:52:54.268 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8299999999999998e-07 2024-05-08T08:52:54.268 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.268 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.269 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:54.269 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586568+0000", 2024-05-08T08:52:54.269 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.063e-06 2024-05-08T08:52:54.269 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.269 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.269 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:54.269 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586574+0000", 2024-05-08T08:52:54.269 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8300000000000001e-06 2024-05-08T08:52:54.269 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.269 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.269 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:54.269 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586579+0000", 2024-05-08T08:52:54.269 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7559999999999999e-06 2024-05-08T08:52:54.269 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.270 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.270 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:54.270 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623612+0000", 2024-05-08T08:52:54.270 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.037033041000000003 2024-05-08T08:52:54.270 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.270 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.270 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:54.270 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623699+0000", 2024-05-08T08:52:54.270 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6817000000000006e-05 2024-05-08T08:52:54.270 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.270 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.270 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:54.271 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623828+0000", 2024-05-08T08:52:54.271 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012917000000000001 2024-05-08T08:52:54.271 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.271 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:54.271 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.271 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.271 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.271 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4770 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 2062336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:54.271 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586585+0000", 2024-05-08T08:52:54.271 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 22.624501814999999, 2024-05-08T08:52:54.271 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.037449891999999999, 2024-05-08T08:52:54.271 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:54.271 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:54.271 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:54.272 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:54.272 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:54.272 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:54.272 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4770 2024-05-08T08:52:54.272 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.272 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:54.272 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.272 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:54.272 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586585+0000", 2024-05-08T08:52:54.272 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:54.272 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.272 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.272 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:54.272 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586585+0000", 2024-05-08T08:52:54.273 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:54.273 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.273 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.273 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:54.273 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586585+0000", 2024-05-08T08:52:54.273 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:52:54.273 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.273 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.273 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:54.273 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586587+0000", 2024-05-08T08:52:54.273 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.181e-06 2024-05-08T08:52:54.273 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.274 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.274 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:54.274 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586592+0000", 2024-05-08T08:52:54.274 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9710000000000003e-06 2024-05-08T08:52:54.274 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.274 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.274 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:54.274 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586598+0000", 2024-05-08T08:52:54.274 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.118e-06 2024-05-08T08:52:54.274 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.274 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.274 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:54.275 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623862+0000", 2024-05-08T08:52:54.275 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.037264711999999998 2024-05-08T08:52:54.275 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.275 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.275 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:54.275 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623941+0000", 2024-05-08T08:52:54.275 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9084000000000004e-05 2024-05-08T08:52:54.275 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.275 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.275 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:54.275 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.624035+0000", 2024-05-08T08:52:54.275 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3456999999999996e-05 2024-05-08T08:52:54.276 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.276 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:54.276 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.276 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.276 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.276 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4772 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 2078720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:54.276 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586621+0000", 2024-05-08T08:52:54.276 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 22.624465867000001, 2024-05-08T08:52:54.276 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.037650391999999998, 2024-05-08T08:52:54.276 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:54.276 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:54.276 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:54.276 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:54.277 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:54.277 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:54.277 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4772 2024-05-08T08:52:54.277 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.277 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:54.277 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.277 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:54.277 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586621+0000", 2024-05-08T08:52:54.277 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:54.277 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.277 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.277 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:54.277 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586621+0000", 2024-05-08T08:52:54.278 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:54.278 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.278 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.278 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:54.278 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586621+0000", 2024-05-08T08:52:54.278 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:52:54.278 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.278 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.278 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:54.278 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586623+0000", 2024-05-08T08:52:54.278 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9240000000000001e-06 2024-05-08T08:52:54.278 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.279 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.279 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:54.279 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586628+0000", 2024-05-08T08:52:54.279 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6410000000000001e-06 2024-05-08T08:52:54.279 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.279 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.279 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:54.279 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586633+0000", 2024-05-08T08:52:54.279 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.763e-06 2024-05-08T08:52:54.279 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.279 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.279 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:54.279 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.624077+0000", 2024-05-08T08:52:54.280 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.037444709999999999 2024-05-08T08:52:54.280 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.280 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.280 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:54.280 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.624164+0000", 2024-05-08T08:52:54.280 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6769999999999998e-05 2024-05-08T08:52:54.280 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.280 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.280 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:54.280 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.624271+0000", 2024-05-08T08:52:54.280 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010723 2024-05-08T08:52:54.280 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.281 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:54.281 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.281 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.281 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.281 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4774 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 2095104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:54.281 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586640+0000", 2024-05-08T08:52:54.281 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 22.624446728999999, 2024-05-08T08:52:54.281 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.037827178000000003, 2024-05-08T08:52:54.281 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:54.281 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:54.281 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:54.281 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:54.282 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:54.282 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:54.282 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4774 2024-05-08T08:52:54.282 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.282 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:54.282 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.282 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:54.282 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586640+0000", 2024-05-08T08:52:54.282 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:54.282 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.282 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.283 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:54.283 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586640+0000", 2024-05-08T08:52:54.283 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:54.283 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.283 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.283 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:54.283 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586640+0000", 2024-05-08T08:52:54.283 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5199999999999998e-07 2024-05-08T08:52:54.283 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.283 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.283 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:54.283 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586643+0000", 2024-05-08T08:52:54.284 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.232e-06 2024-05-08T08:52:54.284 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.284 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.284 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:54.284 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586649+0000", 2024-05-08T08:52:54.284 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9660000000000001e-06 2024-05-08T08:52:54.284 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.284 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.284 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:54.284 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586653+0000", 2024-05-08T08:52:54.284 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7350000000000004e-06 2024-05-08T08:52:54.284 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.285 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.285 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:54.285 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.624303+0000", 2024-05-08T08:52:54.285 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.037649647000000001 2024-05-08T08:52:54.285 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.285 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.285 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:54.285 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.624373+0000", 2024-05-08T08:52:54.285 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0415999999999998e-05 2024-05-08T08:52:54.285 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.285 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.285 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:54.286 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.624467+0000", 2024-05-08T08:52:54.286 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3830000000000001e-05 2024-05-08T08:52:54.286 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.286 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:54.286 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.286 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.286 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.286 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4621.0:9089 3.1f e33/18 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:head v 33'117, mlcod=31'115)", 2024-05-08T08:52:54.286 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:52:54.286 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 21.691468517000001, 2024-05-08T08:52:54.286 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.111244524, 2024-05-08T08:52:54.287 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:54.287 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:54.287 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:54.287 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:54.287 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.287 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:54.287 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:52:54.287 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:54.287 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.287 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.287 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:54.287 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:52:54.288 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:54.288 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.288 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.288 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:54.288 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:52:54.288 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5900000000000001e-07 2024-05-08T08:52:54.288 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.288 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.288 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:54.288 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539600+0000", 2024-05-08T08:52:54.288 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.019981364000000001 2024-05-08T08:52:54.288 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.288 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.289 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:54.289 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539624+0000", 2024-05-08T08:52:54.289 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4043e-05 2024-05-08T08:52:54.289 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.289 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.289 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:54.289 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539635+0000", 2024-05-08T08:52:54.289 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1511e-05 2024-05-08T08:52:54.289 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.289 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.289 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:54.290 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539728+0000", 2024-05-08T08:52:54.290 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2498999999999996e-05 2024-05-08T08:52:54.290 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.290 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.290 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:54.290 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539746+0000", 2024-05-08T08:52:54.290 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8289000000000001e-05 2024-05-08T08:52:54.290 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.290 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.290 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:54.290 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.630822+0000", 2024-05-08T08:52:54.291 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.091076283999999993 2024-05-08T08:52:54.291 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.291 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.291 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:54.291 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.630863+0000", 2024-05-08T08:52:54.291 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0374999999999999e-05 2024-05-08T08:52:54.291 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.291 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:54.291 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.291 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.291 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.291 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4795.0:230 3.1f e40/18 3:f8db1a56:::rbd_data.11402b26385.0000000000000000:head v 40'171, mlcod=39'168)", 2024-05-08T08:52:54.291 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:45.927773+0000", 2024-05-08T08:52:54.292 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 8.2833134340000001, 2024-05-08T08:52:54.292 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045892405999999997, 2024-05-08T08:52:54.292 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:54.292 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:54.292 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:54.292 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:54.292 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.292 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:54.292 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.927773+0000", 2024-05-08T08:52:54.292 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:54.292 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.292 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.293 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:54.293 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.927773+0000", 2024-05-08T08:52:54.293 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:54.293 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.293 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.293 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:54.293 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.927774+0000", 2024-05-08T08:52:54.293 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0500000000000002e-07 2024-05-08T08:52:54.293 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.293 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.293 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:54.293 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.931581+0000", 2024-05-08T08:52:54.293 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0038071799999999999 2024-05-08T08:52:54.294 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.294 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.294 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:54.294 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.931599+0000", 2024-05-08T08:52:54.294 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7479e-05 2024-05-08T08:52:54.294 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.294 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.294 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:54.294 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.931624+0000", 2024-05-08T08:52:54.294 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5372999999999999e-05 2024-05-08T08:52:54.294 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.295 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.295 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:54.295 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.931698+0000", 2024-05-08T08:52:54.295 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4043999999999998e-05 2024-05-08T08:52:54.295 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.295 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.295 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:54.295 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.931725+0000", 2024-05-08T08:52:54.295 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6883e-05 2024-05-08T08:52:54.295 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.295 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.295 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:52:54.296 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.973623+0000", 2024-05-08T08:52:54.296 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.041898283000000001 2024-05-08T08:52:54.296 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.296 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.296 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:54.296 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.973666+0000", 2024-05-08T08:52:54.296 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2358999999999999e-05 2024-05-08T08:52:54.296 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.296 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:54.296 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.296 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.296 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.297 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4195.0:4 3.11 3:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-05-08T08:52:54.297 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:49.764654+0000", 2024-05-08T08:52:54.297 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.4464329759999996, 2024-05-08T08:52:54.297 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.30714275499999999, 2024-05-08T08:52:54.297 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:54.297 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:54.297 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-05-08T08:52:54.297 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:54.297 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4195", 2024-05-08T08:52:54.297 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/59179521", 2024-05-08T08:52:54.297 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4 2024-05-08T08:52:54.297 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.298 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:54.298 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.298 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:54.298 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764654+0000", 2024-05-08T08:52:54.298 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:54.298 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.298 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.298 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:54.298 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764654+0000", 2024-05-08T08:52:54.298 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:54.298 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.298 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.299 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:54.299 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764655+0000", 2024-05-08T08:52:54.299 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.452e-06 2024-05-08T08:52:54.299 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.299 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.299 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:54.299 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764661+0000", 2024-05-08T08:52:54.299 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5369999999999997e-06 2024-05-08T08:52:54.299 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.299 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.299 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:54.299 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764671+0000", 2024-05-08T08:52:54.300 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.028e-05 2024-05-08T08:52:54.300 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.300 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.300 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:54.300 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764680+0000", 2024-05-08T08:52:54.300 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0629999999999995e-06 2024-05-08T08:52:54.300 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.300 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.300 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:54.300 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764753+0000", 2024-05-08T08:52:54.300 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2679e-05 2024-05-08T08:52:54.300 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.301 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.301 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-05-08T08:52:54.301 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764765+0000", 2024-05-08T08:52:54.301 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2415000000000001e-05 2024-05-08T08:52:54.301 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.301 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.301 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:54.301 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:50.071692+0000", 2024-05-08T08:52:54.301 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.306927324 2024-05-08T08:52:54.301 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.301 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.301 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:54.302 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:50.071796+0000", 2024-05-08T08:52:54.302 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104005 2024-05-08T08:52:54.302 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.302 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:54.302 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.302 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.302 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:54.302 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:52:54.304 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:52:54.305 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:52:54.307+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:52:54.404 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:52:54.408+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:52:54.410 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:52:54.410 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:52:54.410 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:52:54.410 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:52:54.419 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:52:54.506 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:52:54.509+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:52:54.523 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:52:54.523 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:52:54.523 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:52:54.523 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:52:54.523 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:52:54.532 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:52:54.606 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:54.609+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:54.639 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:52:54.639 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:52:54.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:52:54.639 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:52:54.639 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.640 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22069 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 749568~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:54.640 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857624+0000", 2024-05-08T08:52:54.640 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 26.783161155999998, 2024-05-08T08:52:54.640 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046969958999999999, 2024-05-08T08:52:54.640 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:54.640 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:54.641 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:54.641 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:54.641 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:54.641 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:54.641 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22069 2024-05-08T08:52:54.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.642 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:54.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:54.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857624+0000", 2024-05-08T08:52:54.643 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:54.643 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.643 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.643 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:54.643 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857624+0000", 2024-05-08T08:52:54.644 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:54.644 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.644 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.644 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:54.644 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857625+0000", 2024-05-08T08:52:54.644 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.03e-07 2024-05-08T08:52:54.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.645 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.645 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:54.645 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857627+0000", 2024-05-08T08:52:54.645 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9369999999999998e-06 2024-05-08T08:52:54.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.646 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:54.646 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857633+0000", 2024-05-08T08:52:54.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1229999999999997e-06 2024-05-08T08:52:54.647 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.647 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.648 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:54.648 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857637+0000", 2024-05-08T08:52:54.648 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7820000000000001e-06 2024-05-08T08:52:54.648 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.648 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.648 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:54.648 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904457+0000", 2024-05-08T08:52:54.648 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046819715999999997 2024-05-08T08:52:54.648 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.648 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.649 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:54.649 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904524+0000", 2024-05-08T08:52:54.649 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6371000000000005e-05 2024-05-08T08:52:54.649 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.649 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.649 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:54.649 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904594+0000", 2024-05-08T08:52:54.649 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0627e-05 2024-05-08T08:52:54.649 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.649 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:54.649 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.649 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.649 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.649 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22071 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 765952~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:54.649 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857643+0000", 2024-05-08T08:52:54.650 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 26.783142073, 2024-05-08T08:52:54.650 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047106528000000002, 2024-05-08T08:52:54.650 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:54.650 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:54.650 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:54.650 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:54.650 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:54.650 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:54.650 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22071 2024-05-08T08:52:54.650 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.650 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:54.650 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.650 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:54.651 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857643+0000", 2024-05-08T08:52:54.651 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:54.651 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.651 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.651 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:54.651 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857643+0000", 2024-05-08T08:52:54.651 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:54.651 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.651 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.651 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:54.651 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857644+0000", 2024-05-08T08:52:54.651 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2000000000000001e-07 2024-05-08T08:52:54.651 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.652 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.652 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:54.652 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857646+0000", 2024-05-08T08:52:54.652 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0059999999999999e-06 2024-05-08T08:52:54.652 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.652 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.652 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:54.652 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857651+0000", 2024-05-08T08:52:54.652 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.84e-06 2024-05-08T08:52:54.652 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.653 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.653 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:54.653 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857657+0000", 2024-05-08T08:52:54.653 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5620000000000003e-06 2024-05-08T08:52:54.653 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.653 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.653 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:54.653 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904620+0000", 2024-05-08T08:52:54.653 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046962661000000003 2024-05-08T08:52:54.654 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.654 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.654 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:54.654 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904678+0000", 2024-05-08T08:52:54.654 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8656999999999998e-05 2024-05-08T08:52:54.654 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.655 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.655 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:54.655 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904750+0000", 2024-05-08T08:52:54.655 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1482000000000004e-05 2024-05-08T08:52:54.655 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.655 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:54.655 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.655 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.655 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.655 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22073 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 782336~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:54.655 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:52:54.655 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 26.783118065, 2024-05-08T08:52:54.655 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047244234000000003, 2024-05-08T08:52:54.655 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:54.656 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:54.656 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:54.656 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:54.656 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:54.656 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:54.656 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22073 2024-05-08T08:52:54.656 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.656 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:54.656 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.656 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:54.656 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:52:54.656 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:54.656 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.656 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.657 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:54.657 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:52:54.657 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:54.657 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.657 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.657 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:54.657 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857668+0000", 2024-05-08T08:52:54.657 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7800000000000002e-07 2024-05-08T08:52:54.657 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.657 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.657 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:54.657 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857670+0000", 2024-05-08T08:52:54.657 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0820000000000001e-06 2024-05-08T08:52:54.658 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.658 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.658 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:54.658 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857678+0000", 2024-05-08T08:52:54.658 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7589999999999997e-06 2024-05-08T08:52:54.658 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.658 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.658 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:54.658 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857691+0000", 2024-05-08T08:52:54.659 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3672999999999999e-05 2024-05-08T08:52:54.659 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.659 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.659 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:54.659 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904780+0000", 2024-05-08T08:52:54.659 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047088616999999999 2024-05-08T08:52:54.659 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.659 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.659 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:54.659 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904840+0000", 2024-05-08T08:52:54.659 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0356999999999998e-05 2024-05-08T08:52:54.659 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.659 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.659 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:54.659 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904912+0000", 2024-05-08T08:52:54.660 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1367999999999998e-05 2024-05-08T08:52:54.660 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.660 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:54.660 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.660 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.660 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.660 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22075 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 798720~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:54.660 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:52:54.660 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 26.783087668, 2024-05-08T08:52:54.661 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047384643999999997, 2024-05-08T08:52:54.661 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:54.661 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:54.661 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:54.661 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:54.661 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:54.661 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:54.661 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22075 2024-05-08T08:52:54.661 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.661 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:54.661 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.661 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:54.661 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:52:54.662 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:54.662 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.662 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.662 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:54.662 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:52:54.662 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:54.662 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.662 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.662 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:54.662 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:52:54.662 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7399999999999999e-07 2024-05-08T08:52:54.662 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.662 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.662 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:54.663 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857705+0000", 2024-05-08T08:52:54.663 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1980000000000004e-06 2024-05-08T08:52:54.663 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.663 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.663 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:54.663 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857713+0000", 2024-05-08T08:52:54.663 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7039999999999994e-06 2024-05-08T08:52:54.663 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.663 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.663 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:54.663 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857718+0000", 2024-05-08T08:52:54.663 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.865e-06 2024-05-08T08:52:54.663 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.663 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.664 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:54.664 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904941+0000", 2024-05-08T08:52:54.664 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047223198000000001 2024-05-08T08:52:54.664 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.664 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.664 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:54.664 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904999+0000", 2024-05-08T08:52:54.664 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8254999999999997e-05 2024-05-08T08:52:54.664 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.664 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.664 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:54.664 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905082+0000", 2024-05-08T08:52:54.665 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3049999999999999e-05 2024-05-08T08:52:54.665 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.665 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:54.665 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.665 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.665 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.665 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22077 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 815104~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:54.665 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:52:54.665 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 26.783049656999999, 2024-05-08T08:52:54.665 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047519448999999998, 2024-05-08T08:52:54.665 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:54.665 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:54.665 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:54.665 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:54.666 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:54.666 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:54.666 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22077 2024-05-08T08:52:54.666 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.666 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:54.666 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.666 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:54.666 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:52:54.666 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:54.666 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.666 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.666 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:54.666 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:52:54.667 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:54.667 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.667 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.667 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:54.667 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:52:54.667 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.99e-07 2024-05-08T08:52:54.667 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.667 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.667 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:54.667 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857739+0000", 2024-05-08T08:52:54.667 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3599999999999999e-06 2024-05-08T08:52:54.667 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.667 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.667 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:54.667 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857745+0000", 2024-05-08T08:52:54.668 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5139999999999998e-06 2024-05-08T08:52:54.668 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.668 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.668 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:54.668 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857750+0000", 2024-05-08T08:52:54.668 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8550000000000001e-06 2024-05-08T08:52:54.668 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.668 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.668 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:54.668 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905110+0000", 2024-05-08T08:52:54.668 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047360126000000002 2024-05-08T08:52:54.668 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.668 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.668 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:54.668 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905175+0000", 2024-05-08T08:52:54.669 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4689000000000004e-05 2024-05-08T08:52:54.669 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.669 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.669 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:54.669 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905255+0000", 2024-05-08T08:52:54.669 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0306e-05 2024-05-08T08:52:54.669 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.669 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:54.669 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.669 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.669 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.669 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22079 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 831488~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:54.669 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:52:54.669 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 26.783029109000001, 2024-05-08T08:52:54.669 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.04767499, 2024-05-08T08:52:54.669 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:54.670 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:54.670 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:54.670 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:54.670 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:54.670 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:54.670 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22079 2024-05-08T08:52:54.670 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.670 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:54.670 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.670 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:54.670 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:52:54.670 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:54.670 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.670 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.671 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:54.671 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:52:54.671 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:54.671 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.671 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.671 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:54.671 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857757+0000", 2024-05-08T08:52:54.671 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8299999999999997e-07 2024-05-08T08:52:54.671 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.671 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.671 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:54.671 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857759+0000", 2024-05-08T08:52:54.671 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1390000000000002e-06 2024-05-08T08:52:54.671 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.671 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.671 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:54.671 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857765+0000", 2024-05-08T08:52:54.672 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4609999999999996e-06 2024-05-08T08:52:54.672 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.672 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.672 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:54.672 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857770+0000", 2024-05-08T08:52:54.672 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6689999999999997e-06 2024-05-08T08:52:54.672 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.672 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.672 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:54.672 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905281+0000", 2024-05-08T08:52:54.672 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047510668999999998 2024-05-08T08:52:54.672 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.672 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.672 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:54.672 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905344+0000", 2024-05-08T08:52:54.672 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2797000000000002e-05 2024-05-08T08:52:54.673 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.673 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.673 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:54.673 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905431+0000", 2024-05-08T08:52:54.673 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7771999999999999e-05 2024-05-08T08:52:54.673 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.673 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:54.673 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.673 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.673 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.673 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22082 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 847872~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:54.673 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:52:54.673 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 26.783009417999999, 2024-05-08T08:52:54.673 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047831934999999999, 2024-05-08T08:52:54.673 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:54.674 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:54.674 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:54.674 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:54.674 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:54.674 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:54.674 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22082 2024-05-08T08:52:54.674 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.674 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:54.674 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.674 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:54.674 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:52:54.674 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:54.675 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.675 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.675 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:54.675 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:52:54.675 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:54.675 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.675 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.675 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:54.675 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:52:54.675 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:52:54.675 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.675 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.675 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:54.676 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857778+0000", 2024-05-08T08:52:54.676 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0789999999999999e-06 2024-05-08T08:52:54.676 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.676 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.676 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:54.676 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857784+0000", 2024-05-08T08:52:54.676 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0480000000000004e-06 2024-05-08T08:52:54.676 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.676 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.676 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:54.676 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857790+0000", 2024-05-08T08:52:54.676 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2719999999999997e-06 2024-05-08T08:52:54.676 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.676 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.676 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:54.677 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905458+0000", 2024-05-08T08:52:54.677 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047668470999999997 2024-05-08T08:52:54.677 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.677 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.677 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:54.677 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905521+0000", 2024-05-08T08:52:54.677 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2952000000000003e-05 2024-05-08T08:52:54.677 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.677 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.677 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:54.677 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905608+0000", 2024-05-08T08:52:54.677 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6759000000000001e-05 2024-05-08T08:52:54.677 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.677 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:54.678 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.678 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.678 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.678 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22084 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 864256~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:54.678 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:52:54.678 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 26.782985468, 2024-05-08T08:52:54.678 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047977748000000001, 2024-05-08T08:52:54.678 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:54.678 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:54.678 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:54.678 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:54.678 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:54.678 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:54.678 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22084 2024-05-08T08:52:54.679 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.679 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:54.679 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.679 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:54.679 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:52:54.679 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:54.679 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.679 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.679 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:54.679 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:52:54.679 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:54.679 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.679 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.679 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:54.679 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:52:54.680 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:52:54.680 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.680 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.680 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:54.680 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857802+0000", 2024-05-08T08:52:54.680 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0779999999999998e-06 2024-05-08T08:52:54.680 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.680 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.680 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:54.680 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857812+0000", 2024-05-08T08:52:54.680 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7939999999999997e-06 2024-05-08T08:52:54.680 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.680 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.680 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:54.680 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857817+0000", 2024-05-08T08:52:54.681 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8899999999999998e-06 2024-05-08T08:52:54.681 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.681 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.681 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:54.681 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905636+0000", 2024-05-08T08:52:54.681 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047819252 2024-05-08T08:52:54.681 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.681 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.681 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:54.681 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905700+0000", 2024-05-08T08:52:54.681 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4016999999999994e-05 2024-05-08T08:52:54.681 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.681 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.681 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:54.681 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905778+0000", 2024-05-08T08:52:54.682 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7306999999999997e-05 2024-05-08T08:52:54.682 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.682 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:54.682 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.682 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.682 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.682 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22086 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 880640~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:54.682 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:52:54.682 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 26.782962118, 2024-05-08T08:52:54.682 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048119149, 2024-05-08T08:52:54.682 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:54.682 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:54.682 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:54.682 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:54.683 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:54.683 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:54.683 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22086 2024-05-08T08:52:54.683 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.683 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:54.683 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.683 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:54.683 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:52:54.683 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:54.683 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.683 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.683 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:54.683 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:52:54.683 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:54.683 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.684 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.684 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:54.684 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857824+0000", 2024-05-08T08:52:54.684 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4200000000000002e-07 2024-05-08T08:52:54.684 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.684 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.684 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:54.684 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857826+0000", 2024-05-08T08:52:54.684 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0590000000000001e-06 2024-05-08T08:52:54.684 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.684 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.684 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:54.684 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857832+0000", 2024-05-08T08:52:54.684 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8029999999999999e-06 2024-05-08T08:52:54.684 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.685 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.685 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:54.685 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857837+0000", 2024-05-08T08:52:54.685 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7929999999999997e-06 2024-05-08T08:52:54.685 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.685 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.685 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:54.685 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905803+0000", 2024-05-08T08:52:54.685 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047965761000000003 2024-05-08T08:52:54.685 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.685 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.685 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:54.685 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905862+0000", 2024-05-08T08:52:54.685 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9203000000000002e-05 2024-05-08T08:52:54.685 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.686 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.686 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:54.686 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905942+0000", 2024-05-08T08:52:54.686 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0187999999999999e-05 2024-05-08T08:52:54.686 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.686 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:54.686 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.686 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.686 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.686 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22088 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 897024~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:54.686 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:52:54.686 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 26.782942066, 2024-05-08T08:52:54.686 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048256049000000002, 2024-05-08T08:52:54.686 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:54.687 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:54.687 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:54.687 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:54.687 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:54.687 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:54.687 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22088 2024-05-08T08:52:54.687 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.687 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:54.687 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.687 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:54.687 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:52:54.687 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:54.687 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.687 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.687 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:54.688 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:52:54.688 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:54.688 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.688 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.688 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:54.688 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857844+0000", 2024-05-08T08:52:54.688 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.46e-07 2024-05-08T08:52:54.688 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.688 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.688 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:54.688 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857846+0000", 2024-05-08T08:52:54.688 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0899999999999999e-06 2024-05-08T08:52:54.688 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.688 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.688 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:54.689 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857855+0000", 2024-05-08T08:52:54.689 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8219999999999996e-06 2024-05-08T08:52:54.689 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.689 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.689 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:54.689 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857860+0000", 2024-05-08T08:52:54.689 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2989999999999996e-06 2024-05-08T08:52:54.689 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.689 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.689 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:54.689 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905966+0000", 2024-05-08T08:52:54.689 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048106542000000002 2024-05-08T08:52:54.689 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.689 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.689 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:54.690 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906024+0000", 2024-05-08T08:52:54.690 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7958000000000003e-05 2024-05-08T08:52:54.690 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.690 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.690 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:54.690 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906099+0000", 2024-05-08T08:52:54.690 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4992000000000003e-05 2024-05-08T08:52:54.690 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.690 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:54.690 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.690 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.690 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.690 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22091 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 913408~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:54.690 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:52:54.691 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 26.782918305999999, 2024-05-08T08:52:54.691 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048395290000000001, 2024-05-08T08:52:54.691 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:54.691 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:54.691 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:54.691 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:54.691 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:54.691 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:54.691 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22091 2024-05-08T08:52:54.691 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.691 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:54.691 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.691 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:54.691 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:52:54.691 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:54.692 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.692 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.692 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:54.692 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:52:54.692 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:54.692 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.692 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.692 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:54.692 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:52:54.692 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4900000000000001e-07 2024-05-08T08:52:54.692 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.692 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.692 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:54.692 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857870+0000", 2024-05-08T08:52:54.692 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2510000000000001e-06 2024-05-08T08:52:54.693 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.693 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.693 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:54.693 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857877+0000", 2024-05-08T08:52:54.693 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0809999999999996e-06 2024-05-08T08:52:54.693 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.693 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.693 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:54.693 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857881+0000", 2024-05-08T08:52:54.693 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.724e-06 2024-05-08T08:52:54.693 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.693 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.693 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:54.693 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906127+0000", 2024-05-08T08:52:54.693 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048245345000000002 2024-05-08T08:52:54.693 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.694 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.694 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:54.694 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906190+0000", 2024-05-08T08:52:54.694 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2762000000000007e-05 2024-05-08T08:52:54.694 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.694 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.694 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:54.694 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906262+0000", 2024-05-08T08:52:54.694 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2778e-05 2024-05-08T08:52:54.694 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.694 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:54.694 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.694 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.694 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.695 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22093 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 929792~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:54.695 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:52:54.695 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 26.782895942, 2024-05-08T08:52:54.695 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048532875000000003, 2024-05-08T08:52:54.695 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:54.695 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:54.695 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:54.695 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:54.695 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:54.695 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:54.695 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22093 2024-05-08T08:52:54.695 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.695 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:54.695 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.696 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:54.696 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:52:54.696 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:54.696 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.696 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.696 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:54.696 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:52:54.696 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:54.696 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.696 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.696 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:54.697 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857890+0000", 2024-05-08T08:52:54.697 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-07 2024-05-08T08:52:54.697 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.697 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.697 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:54.697 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857899+0000", 2024-05-08T08:52:54.697 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6889999999999999e-06 2024-05-08T08:52:54.697 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.697 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.697 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:54.697 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857906+0000", 2024-05-08T08:52:54.697 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6490000000000002e-06 2024-05-08T08:52:54.697 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.698 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.698 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:54.698 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857917+0000", 2024-05-08T08:52:54.698 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0764999999999999e-05 2024-05-08T08:52:54.698 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.698 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.698 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:54.698 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906287+0000", 2024-05-08T08:52:54.698 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048370442 2024-05-08T08:52:54.698 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.698 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.698 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:54.698 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906349+0000", 2024-05-08T08:52:54.698 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1537000000000004e-05 2024-05-08T08:52:54.698 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.699 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.699 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:54.699 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906422+0000", 2024-05-08T08:52:54.699 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3431999999999998e-05 2024-05-08T08:52:54.699 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.699 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:54.699 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.699 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.699 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.699 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22095 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 946176~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:54.699 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:52:54.699 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 26.782861237999999, 2024-05-08T08:52:54.699 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048657593999999998, 2024-05-08T08:52:54.699 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:54.699 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:54.700 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:54.700 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:54.700 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:54.700 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:54.700 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22095 2024-05-08T08:52:54.700 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.700 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:54.700 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.700 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:54.700 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:52:54.700 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:54.700 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.700 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.700 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:54.700 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:52:54.701 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:54.701 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.701 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.701 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:54.701 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857925+0000", 2024-05-08T08:52:54.701 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3099999999999998e-07 2024-05-08T08:52:54.701 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.701 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.701 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:54.701 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857927+0000", 2024-05-08T08:52:54.701 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.34e-06 2024-05-08T08:52:54.701 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.701 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.701 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:54.701 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857938+0000", 2024-05-08T08:52:54.702 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1426e-05 2024-05-08T08:52:54.702 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.702 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.702 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:54.702 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857943+0000", 2024-05-08T08:52:54.702 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5140000000000003e-06 2024-05-08T08:52:54.702 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.702 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.702 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:54.702 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906448+0000", 2024-05-08T08:52:54.702 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048504641000000001 2024-05-08T08:52:54.702 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.702 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.702 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:54.703 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906510+0000", 2024-05-08T08:52:54.703 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2663000000000006e-05 2024-05-08T08:52:54.703 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.703 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.703 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:54.703 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906582+0000", 2024-05-08T08:52:54.703 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1478999999999998e-05 2024-05-08T08:52:54.703 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.703 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:54.703 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.703 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.703 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.703 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22097 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 962560~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:54.703 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:52:54.704 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 26.782833548999999, 2024-05-08T08:52:54.704 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048791353000000003, 2024-05-08T08:52:54.704 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:54.704 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:54.704 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:54.704 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:54.704 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:54.704 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:54.704 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22097 2024-05-08T08:52:54.704 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.704 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:54.704 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.704 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:54.704 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:52:54.705 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:54.705 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.705 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.705 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:54.705 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:52:54.705 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:54.705 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.705 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.705 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:54.705 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:52:54.705 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9200000000000002e-07 2024-05-08T08:52:54.705 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.705 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.705 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:54.705 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857954+0000", 2024-05-08T08:52:54.705 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1900000000000002e-06 2024-05-08T08:52:54.706 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.706 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.706 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:54.706 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857961+0000", 2024-05-08T08:52:54.706 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4989999999999999e-06 2024-05-08T08:52:54.706 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.706 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.706 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:54.706 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857971+0000", 2024-05-08T08:52:54.706 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0485999999999999e-05 2024-05-08T08:52:54.706 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.706 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.706 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:54.706 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906607+0000", 2024-05-08T08:52:54.707 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048635109000000003 2024-05-08T08:52:54.707 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.707 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.707 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:54.707 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906667+0000", 2024-05-08T08:52:54.707 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0439000000000002e-05 2024-05-08T08:52:54.707 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.707 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.707 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:54.707 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906743+0000", 2024-05-08T08:52:54.707 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6237999999999998e-05 2024-05-08T08:52:54.707 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.707 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:54.707 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.708 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.708 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.708 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22099 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 978944~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:54.708 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:52:54.708 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 26.782807388999998, 2024-05-08T08:52:54.708 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048932055000000002, 2024-05-08T08:52:54.708 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:54.708 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:54.708 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:54.708 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:54.708 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:54.708 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:54.708 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22099 2024-05-08T08:52:54.708 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.708 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:54.709 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.709 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:54.709 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:52:54.709 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:54.709 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.709 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.709 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:54.709 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:52:54.709 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:54.709 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.709 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.709 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:54.709 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:52:54.709 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.27e-07 2024-05-08T08:52:54.709 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.710 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.710 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:54.710 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857987+0000", 2024-05-08T08:52:54.710 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5890000000000003e-06 2024-05-08T08:52:54.710 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.710 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.710 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:54.710 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857994+0000", 2024-05-08T08:52:54.710 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6619999999999999e-06 2024-05-08T08:52:54.710 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.710 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.710 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:54.710 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857998+0000", 2024-05-08T08:52:54.710 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7110000000000003e-06 2024-05-08T08:52:54.711 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.711 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.711 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:54.711 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906772+0000", 2024-05-08T08:52:54.711 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048773415000000001 2024-05-08T08:52:54.711 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.711 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.711 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:54.711 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906834+0000", 2024-05-08T08:52:54.711 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1752000000000001e-05 2024-05-08T08:52:54.711 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.711 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.711 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:54.711 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906910+0000", 2024-05-08T08:52:54.712 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6499000000000001e-05 2024-05-08T08:52:54.712 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.712 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:54.712 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.712 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.712 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.712 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22101 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 995328~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:54.712 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:52:54.712 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 26.782780257999999, 2024-05-08T08:52:54.712 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049087364000000001, 2024-05-08T08:52:54.712 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:54.712 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:54.712 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:54.712 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:54.712 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:54.713 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:54.713 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22101 2024-05-08T08:52:54.713 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.713 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:54.713 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.713 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:54.713 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:52:54.713 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:54.713 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.713 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.714 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:54.714 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:52:54.714 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:54.714 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.714 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.714 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:54.714 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858006+0000", 2024-05-08T08:52:54.714 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1399999999999997e-07 2024-05-08T08:52:54.714 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.714 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.714 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:54.714 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858008+0000", 2024-05-08T08:52:54.714 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2060000000000001e-06 2024-05-08T08:52:54.715 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.715 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.715 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:54.715 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858017+0000", 2024-05-08T08:52:54.715 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1579999999999994e-06 2024-05-08T08:52:54.715 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.715 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.715 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:54.715 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858022+0000", 2024-05-08T08:52:54.715 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4199999999999998e-06 2024-05-08T08:52:54.715 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.715 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.715 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:54.715 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906940+0000", 2024-05-08T08:52:54.715 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048917627999999998 2024-05-08T08:52:54.715 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.716 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.716 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:54.716 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907002+0000", 2024-05-08T08:52:54.716 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2399999999999999e-05 2024-05-08T08:52:54.716 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.716 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.716 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:54.716 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907092+0000", 2024-05-08T08:52:54.716 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0137999999999997e-05 2024-05-08T08:52:54.716 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.716 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:54.716 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.716 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.716 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.716 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22103 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1011712~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:54.717 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:52:54.717 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 26.782755473000002, 2024-05-08T08:52:54.717 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049228542, 2024-05-08T08:52:54.717 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:54.717 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:54.717 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:54.717 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:54.717 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:54.717 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:54.717 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22103 2024-05-08T08:52:54.717 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.717 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:54.717 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.717 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:54.717 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:52:54.717 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:54.718 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.718 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.718 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:54.718 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:52:54.718 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:54.718 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.718 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.718 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:54.718 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:52:54.718 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4299999999999999e-07 2024-05-08T08:52:54.718 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.718 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.718 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:54.718 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858032+0000", 2024-05-08T08:52:54.718 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0590000000000001e-06 2024-05-08T08:52:54.719 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.719 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.719 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:54.719 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858037+0000", 2024-05-08T08:52:54.719 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.977e-06 2024-05-08T08:52:54.719 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.719 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.719 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:54.719 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858042+0000", 2024-05-08T08:52:54.719 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7650000000000001e-06 2024-05-08T08:52:54.719 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.719 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.719 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:54.719 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907118+0000", 2024-05-08T08:52:54.719 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049076147000000001 2024-05-08T08:52:54.720 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.720 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.720 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:54.720 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907180+0000", 2024-05-08T08:52:54.720 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1971999999999996e-05 2024-05-08T08:52:54.720 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.720 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.720 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:54.720 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907258+0000", 2024-05-08T08:52:54.720 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8279e-05 2024-05-08T08:52:54.720 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.720 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:54.720 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.720 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.720 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.720 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22105 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1028096~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:54.721 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:52:54.721 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 26.782736230000001, 2024-05-08T08:52:54.721 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049394609999999999, 2024-05-08T08:52:54.721 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:54.721 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:54.721 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:54.721 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:54.721 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:54.721 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:54.721 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22105 2024-05-08T08:52:54.721 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.721 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:54.721 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.721 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:54.721 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:52:54.722 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:54.722 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.722 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.722 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:54.722 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:52:54.722 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:54.722 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.722 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.722 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:54.722 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858050+0000", 2024-05-08T08:52:54.722 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5999999999999999e-07 2024-05-08T08:52:54.722 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.722 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.722 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:54.722 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858051+0000", 2024-05-08T08:52:54.723 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8870000000000001e-06 2024-05-08T08:52:54.723 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.723 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.723 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:54.723 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858057+0000", 2024-05-08T08:52:54.723 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8019999999999997e-06 2024-05-08T08:52:54.723 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.723 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.723 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:54.723 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858062+0000", 2024-05-08T08:52:54.723 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.561e-06 2024-05-08T08:52:54.723 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.723 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.723 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:54.724 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907289+0000", 2024-05-08T08:52:54.724 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049226957000000002 2024-05-08T08:52:54.724 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.724 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.724 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:54.724 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907356+0000", 2024-05-08T08:52:54.724 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6915000000000006e-05 2024-05-08T08:52:54.724 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.724 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.724 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:54.724 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907444+0000", 2024-05-08T08:52:54.724 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8127999999999994e-05 2024-05-08T08:52:54.724 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.724 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:54.725 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.725 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.725 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.725 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22107 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1044480~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:54.725 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:52:54.725 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 26.782717890000001, 2024-05-08T08:52:54.725 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049799942999999999, 2024-05-08T08:52:54.725 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:54.725 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:54.725 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:54.725 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:54.725 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:54.725 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:54.725 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22107 2024-05-08T08:52:54.725 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.726 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:54.726 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.726 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:54.726 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:52:54.726 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:54.726 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.726 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.726 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:54.726 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:52:54.726 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:54.726 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.726 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.726 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:54.730 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:52:54.730 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4799999999999999e-07 2024-05-08T08:52:54.730 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.730 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.730 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:54.730 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858070+0000", 2024-05-08T08:52:54.730 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.012e-06 2024-05-08T08:52:54.730 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.730 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.730 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:54.730 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858077+0000", 2024-05-08T08:52:54.731 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6290000000000001e-06 2024-05-08T08:52:54.731 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.731 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.731 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:54.731 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858083+0000", 2024-05-08T08:52:54.731 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8880000000000002e-06 2024-05-08T08:52:54.731 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.731 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.731 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:54.731 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907671+0000", 2024-05-08T08:52:54.731 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049587191000000003 2024-05-08T08:52:54.731 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.731 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.731 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:54.731 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907763+0000", 2024-05-08T08:52:54.732 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2231000000000005e-05 2024-05-08T08:52:54.732 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.732 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.732 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:54.732 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907867+0000", 2024-05-08T08:52:54.732 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104644 2024-05-08T08:52:54.732 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.732 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:54.732 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.732 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.732 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.732 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22294 3.1c 3:3aad0389:::rbd_data.11406886b07b.0000000000000080:head [stat,delete] snapc 7=[7] RETRY=6 ondisk+retry+write+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:54.732 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:52:54.732 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 26.782124042, 2024-05-08T08:52:54.732 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051152191, 2024-05-08T08:52:54.732 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:54.733 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:54.733 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:54.733 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:54.733 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:52:54.733 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:54.733 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22294 2024-05-08T08:52:54.733 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.733 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:54.733 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.733 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:54.733 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:52:54.733 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:54.733 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.733 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.733 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:54.734 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:52:54.734 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:54.734 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.734 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.734 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:54.734 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858662+0000", 2024-05-08T08:52:54.734 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4799999999999998e-07 2024-05-08T08:52:54.734 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.734 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.734 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:54.734 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858664+0000", 2024-05-08T08:52:54.734 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1229999999999998e-06 2024-05-08T08:52:54.734 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.734 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.735 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:54.735 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858670+0000", 2024-05-08T08:52:54.735 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3099999999999997e-06 2024-05-08T08:52:54.735 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.735 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.735 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:54.735 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858676+0000", 2024-05-08T08:52:54.735 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9750000000000004e-06 2024-05-08T08:52:54.735 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.735 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.735 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:54.735 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907901+0000", 2024-05-08T08:52:54.735 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049224520000000001 2024-05-08T08:52:54.735 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.735 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.735 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:54.736 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.908085+0000", 2024-05-08T08:52:54.736 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00018377500000000001 2024-05-08T08:52:54.736 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:54.736 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:54.736 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:54.736 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.909814+0000", 2024-05-08T08:52:54.736 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00172894 2024-05-08T08:52:54.736 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.736 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:54.736 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.736 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:54.736 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:54.736 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:52:54.738 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:52:54.739 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:52:54.741+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:52:54.838 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:52:54.841+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:52:54.885 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:52:54.885 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:52:54.885 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:52:54.885 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:52:54.894 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:52:54.940 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:54.943+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:55.001 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:52:55.001 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:52:55.001 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:52:55.001 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:52:55.001 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:52:55.010 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:52:55.040 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:52:55.044+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:52:55.105 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-05-08T08:52:55.149 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:52:55.152+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:52:55.160 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:52:55.160 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:52:55.160 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:52:55.160 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:52:55.160 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.160 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6690 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1093632~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:55.160 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:52:55.160 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.447139283999999, 2024-05-08T08:52:55.160 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052462754, 2024-05-08T08:52:55.161 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:55.161 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:55.161 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:55.161 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:55.161 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:55.161 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:55.161 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6690 2024-05-08T08:52:55.161 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.161 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:55.161 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.161 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:55.161 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:52:55.161 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:55.161 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.161 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.162 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:55.162 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:52:55.162 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:55.162 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.162 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.162 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:55.162 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715346+0000", 2024-05-08T08:52:55.162 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.84e-07 2024-05-08T08:52:55.162 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.162 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.162 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:55.162 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715348+0000", 2024-05-08T08:52:55.162 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.114e-06 2024-05-08T08:52:55.162 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.163 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.163 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:55.163 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715353+0000", 2024-05-08T08:52:55.163 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5720000000000002e-06 2024-05-08T08:52:55.163 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.163 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.163 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:55.163 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715358+0000", 2024-05-08T08:52:55.163 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7140000000000001e-06 2024-05-08T08:52:55.163 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.163 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.163 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:55.163 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767638+0000", 2024-05-08T08:52:55.163 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052279562000000002 2024-05-08T08:52:55.164 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.164 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.164 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:55.164 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767713+0000", 2024-05-08T08:52:55.164 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5652e-05 2024-05-08T08:52:55.164 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.164 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.164 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:55.164 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767808+0000", 2024-05-08T08:52:55.164 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4755999999999999e-05 2024-05-08T08:52:55.164 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:55.164 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:55.164 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:55.164 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.164 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.165 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6692 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1110016~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:55.165 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:52:55.165 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.447120194, 2024-05-08T08:52:55.165 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052666495000000001, 2024-05-08T08:52:55.165 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:55.165 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:55.165 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:55.165 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:55.165 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:55.165 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:55.165 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6692 2024-05-08T08:52:55.165 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.165 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:55.165 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.165 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:55.166 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:52:55.166 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:55.166 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.166 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.166 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:55.166 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:52:55.166 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:55.166 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.166 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.166 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:55.166 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:52:55.166 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8599999999999999e-07 2024-05-08T08:52:55.166 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.166 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.167 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:55.167 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715372+0000", 2024-05-08T08:52:55.167 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0990000000000001e-06 2024-05-08T08:52:55.167 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.167 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.167 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:55.167 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715378+0000", 2024-05-08T08:52:55.167 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0680000000000002e-06 2024-05-08T08:52:55.167 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.167 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.167 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:55.167 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715383+0000", 2024-05-08T08:52:55.167 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8740000000000003e-06 2024-05-08T08:52:55.167 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.167 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.168 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:55.168 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767848+0000", 2024-05-08T08:52:55.168 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052464668999999998 2024-05-08T08:52:55.168 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.168 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.168 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:55.168 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767927+0000", 2024-05-08T08:52:55.168 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9228999999999996e-05 2024-05-08T08:52:55.168 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.168 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.168 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:55.168 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768031+0000", 2024-05-08T08:52:55.168 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010417 2024-05-08T08:52:55.168 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:55.168 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:55.168 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:55.169 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.169 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.169 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6694 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1126400~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:55.169 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:52:55.169 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.447095297000001, 2024-05-08T08:52:55.169 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052848829, 2024-05-08T08:52:55.169 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:55.169 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:55.169 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:55.169 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:55.169 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:55.169 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:55.169 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6694 2024-05-08T08:52:55.169 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.169 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:55.170 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.170 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:55.170 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:52:55.170 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:55.170 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.170 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.170 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:55.170 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:52:55.170 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:55.170 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.170 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.170 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:55.170 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715390+0000", 2024-05-08T08:52:55.170 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9900000000000001e-07 2024-05-08T08:52:55.170 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.171 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.171 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:55.171 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715392+0000", 2024-05-08T08:52:55.171 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8279999999999999e-06 2024-05-08T08:52:55.171 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.171 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.171 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:55.171 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715398+0000", 2024-05-08T08:52:55.171 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7619999999999998e-06 2024-05-08T08:52:55.171 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.171 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.171 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:55.171 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715403+0000", 2024-05-08T08:52:55.171 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7790000000000003e-06 2024-05-08T08:52:55.171 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.172 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.172 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:55.172 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768067+0000", 2024-05-08T08:52:55.172 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052663785999999997 2024-05-08T08:52:55.172 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.172 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.172 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:55.172 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768142+0000", 2024-05-08T08:52:55.172 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5018999999999994e-05 2024-05-08T08:52:55.172 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.172 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.172 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:55.172 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768238+0000", 2024-05-08T08:52:55.172 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6255999999999995e-05 2024-05-08T08:52:55.172 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:55.173 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:55.173 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:55.173 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.173 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.173 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6696 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1142784~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:55.173 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:52:55.173 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.447075788999999, 2024-05-08T08:52:55.173 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.053051057999999998, 2024-05-08T08:52:55.173 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:55.173 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:55.173 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:55.173 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:55.173 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:55.173 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:55.174 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6696 2024-05-08T08:52:55.174 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.174 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:55.174 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.174 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:55.174 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:52:55.174 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:55.174 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.174 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.174 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:55.174 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:52:55.174 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:55.174 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.174 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.174 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:55.175 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:52:55.175 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-05-08T08:52:55.175 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.175 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.175 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:55.175 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715411+0000", 2024-05-08T08:52:55.175 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9630000000000001e-06 2024-05-08T08:52:55.175 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.175 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.175 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:55.175 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715417+0000", 2024-05-08T08:52:55.175 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7250000000000002e-06 2024-05-08T08:52:55.175 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.175 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.175 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:55.176 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715421+0000", 2024-05-08T08:52:55.176 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4089999999999996e-06 2024-05-08T08:52:55.176 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.176 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.176 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:55.176 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768277+0000", 2024-05-08T08:52:55.176 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052855293999999997 2024-05-08T08:52:55.176 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.176 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.176 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:55.176 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768358+0000", 2024-05-08T08:52:55.176 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1720999999999999e-05 2024-05-08T08:52:55.176 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.176 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.176 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:55.177 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768460+0000", 2024-05-08T08:52:55.177 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010154600000000001 2024-05-08T08:52:55.177 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:55.178 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:55.178 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:55.178 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.178 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.178 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6698 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1159168~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:55.178 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:52:55.178 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.447056922000002, 2024-05-08T08:52:55.178 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.053245726, 2024-05-08T08:52:55.178 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:55.178 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:55.178 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:55.178 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:55.178 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:55.178 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:55.178 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6698 2024-05-08T08:52:55.179 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.179 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:55.179 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.179 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:55.179 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:52:55.179 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:55.179 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.179 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.179 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:55.179 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:52:55.179 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:55.179 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.179 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.179 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:55.179 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716936+0000", 2024-05-08T08:52:55.180 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001508665 2024-05-08T08:52:55.180 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.180 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.180 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:55.180 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716941+0000", 2024-05-08T08:52:55.180 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1910000000000002e-06 2024-05-08T08:52:55.180 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.180 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.180 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:55.180 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716950+0000", 2024-05-08T08:52:55.180 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1900000000000001e-06 2024-05-08T08:52:55.180 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.180 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.180 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:55.180 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716958+0000", 2024-05-08T08:52:55.181 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8019999999999992e-06 2024-05-08T08:52:55.181 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.181 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.181 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:55.181 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768499+0000", 2024-05-08T08:52:55.181 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051540926000000001 2024-05-08T08:52:55.181 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.181 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.181 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:55.181 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768574+0000", 2024-05-08T08:52:55.181 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4988999999999996e-05 2024-05-08T08:52:55.181 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.181 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.181 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:55.181 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768673+0000", 2024-05-08T08:52:55.181 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9963000000000005e-05 2024-05-08T08:52:55.182 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:55.182 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:55.182 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:55.182 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.182 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.182 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6706 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1224704~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:55.182 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:52:55.182 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.445451615, 2024-05-08T08:52:55.182 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052435598, 2024-05-08T08:52:55.182 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:55.182 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:55.182 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:55.182 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:55.182 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:55.183 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:55.183 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6706 2024-05-08T08:52:55.183 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.183 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:55.183 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.183 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:55.183 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:52:55.183 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:55.183 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.183 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.183 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:55.183 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:52:55.183 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:55.183 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.183 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.183 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:55.184 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:52:55.184 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:52:55.184 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.184 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.184 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:55.184 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717035+0000", 2024-05-08T08:52:55.184 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9290000000000001e-06 2024-05-08T08:52:55.184 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.184 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.184 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:55.184 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717041+0000", 2024-05-08T08:52:55.184 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6720000000000001e-06 2024-05-08T08:52:55.184 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.184 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.184 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:55.185 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717046+0000", 2024-05-08T08:52:55.185 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5650000000000003e-06 2024-05-08T08:52:55.185 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.185 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.185 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:55.185 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769311+0000", 2024-05-08T08:52:55.185 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052265645999999999 2024-05-08T08:52:55.185 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.185 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.185 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:55.185 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769379+0000", 2024-05-08T08:52:55.185 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7391000000000006e-05 2024-05-08T08:52:55.186 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.186 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.186 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:55.186 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769469+0000", 2024-05-08T08:52:55.186 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0025999999999995e-05 2024-05-08T08:52:55.186 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:55.186 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:55.186 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:55.186 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.186 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.186 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6708 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1241088~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:55.186 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:52:55.186 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.445433511000001, 2024-05-08T08:52:55.186 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052640552, 2024-05-08T08:52:55.186 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:55.187 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:55.187 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:55.187 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:55.187 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:55.187 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:55.187 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6708 2024-05-08T08:52:55.187 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.187 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:55.187 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.188 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:55.188 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:52:55.188 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:55.188 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.188 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.188 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:55.188 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:52:55.188 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:55.188 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.188 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.188 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:55.188 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717052+0000", 2024-05-08T08:52:55.188 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:52:55.188 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.189 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.189 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:55.189 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717053+0000", 2024-05-08T08:52:55.189 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8330000000000001e-06 2024-05-08T08:52:55.189 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.189 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.189 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:55.189 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717058+0000", 2024-05-08T08:52:55.189 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.933e-06 2024-05-08T08:52:55.189 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.189 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.189 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:55.189 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717063+0000", 2024-05-08T08:52:55.189 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2180000000000001e-06 2024-05-08T08:52:55.189 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.190 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.190 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:55.190 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769507+0000", 2024-05-08T08:52:55.190 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052444866999999999 2024-05-08T08:52:55.190 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.190 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.190 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:55.190 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769589+0000", 2024-05-08T08:52:55.190 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2039000000000005e-05 2024-05-08T08:52:55.190 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.190 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.190 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:55.190 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769692+0000", 2024-05-08T08:52:55.190 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102252 2024-05-08T08:52:55.191 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:55.191 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:55.191 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:55.191 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.191 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.191 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6710 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1257472~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:55.191 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:52:55.191 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.445415486999998, 2024-05-08T08:52:55.191 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.05283057, 2024-05-08T08:52:55.191 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:55.192 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:55.192 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:55.192 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:55.192 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:55.192 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:55.192 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6710 2024-05-08T08:52:55.192 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.192 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:55.192 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.192 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:55.192 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:52:55.192 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:55.193 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.193 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.193 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:55.193 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:52:55.193 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:55.193 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.193 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.193 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:55.193 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718951+0000", 2024-05-08T08:52:55.193 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0018817809999999999 2024-05-08T08:52:55.193 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.193 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.193 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:55.193 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718956+0000", 2024-05-08T08:52:55.194 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5399999999999997e-06 2024-05-08T08:52:55.194 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.194 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.194 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:55.194 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718964+0000", 2024-05-08T08:52:55.194 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6629999999999999e-06 2024-05-08T08:52:55.194 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.194 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.194 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:55.194 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718972+0000", 2024-05-08T08:52:55.195 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5229999999999998e-06 2024-05-08T08:52:55.195 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.195 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.195 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:55.195 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769729+0000", 2024-05-08T08:52:55.195 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050756959999999997 2024-05-08T08:52:55.195 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.195 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.195 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:55.195 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769803+0000", 2024-05-08T08:52:55.195 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4538999999999999e-05 2024-05-08T08:52:55.195 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.195 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.195 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:55.195 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769900+0000", 2024-05-08T08:52:55.195 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6564000000000006e-05 2024-05-08T08:52:55.196 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:55.196 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:55.196 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:55.196 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.196 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.196 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6726 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1388544~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:55.196 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:52:55.196 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.443340715000001, 2024-05-08T08:52:55.196 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052451506000000002, 2024-05-08T08:52:55.196 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:55.196 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:55.196 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:55.197 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:55.197 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:55.197 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:55.197 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6726 2024-05-08T08:52:55.197 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.197 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:55.197 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.197 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:55.197 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:52:55.197 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:55.197 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.197 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.197 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:55.197 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:52:55.198 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:55.198 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.198 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.198 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:55.198 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719200+0000", 2024-05-08T08:52:55.199 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5865999999999997e-05 2024-05-08T08:52:55.199 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.199 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.199 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:55.199 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719204+0000", 2024-05-08T08:52:55.199 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8630000000000001e-06 2024-05-08T08:52:55.199 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.199 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.199 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:55.199 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719215+0000", 2024-05-08T08:52:55.199 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1590000000000001e-05 2024-05-08T08:52:55.199 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.199 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.199 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:55.200 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719220+0000", 2024-05-08T08:52:55.200 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1619999999999999e-06 2024-05-08T08:52:55.200 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.200 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.200 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:55.200 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771425+0000", 2024-05-08T08:52:55.200 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052204249000000001 2024-05-08T08:52:55.200 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.200 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.200 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:55.200 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771498+0000", 2024-05-08T08:52:55.200 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3782999999999995e-05 2024-05-08T08:52:55.200 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.200 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.200 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:55.201 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771595+0000", 2024-05-08T08:52:55.201 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6992999999999998e-05 2024-05-08T08:52:55.201 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:55.201 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:55.201 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:55.201 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.201 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.201 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6728 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1404928~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:55.201 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:52:55.201 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.443253309999999, 2024-05-08T08:52:55.201 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052590747, 2024-05-08T08:52:55.201 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:55.201 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:55.201 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:55.201 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:55.202 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:55.202 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:55.202 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6728 2024-05-08T08:52:55.202 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.202 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:55.202 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.202 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:55.202 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:52:55.202 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:55.202 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.202 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.202 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:55.202 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:52:55.203 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:55.203 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.203 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.203 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:55.203 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719232+0000", 2024-05-08T08:52:55.203 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4000000000000001e-07 2024-05-08T08:52:55.203 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.203 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.203 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:55.203 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719234+0000", 2024-05-08T08:52:55.203 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4499999999999998e-06 2024-05-08T08:52:55.203 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.203 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.203 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:55.203 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719240+0000", 2024-05-08T08:52:55.204 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5720000000000002e-06 2024-05-08T08:52:55.204 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.204 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.204 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:55.204 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719245+0000", 2024-05-08T08:52:55.204 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8690000000000003e-06 2024-05-08T08:52:55.204 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.204 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.204 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:55.204 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771636+0000", 2024-05-08T08:52:55.204 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052390979999999997 2024-05-08T08:52:55.204 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.204 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.204 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:55.204 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771717+0000", 2024-05-08T08:52:55.205 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1291999999999993e-05 2024-05-08T08:52:55.205 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.205 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.205 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:55.205 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771822+0000", 2024-05-08T08:52:55.205 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104944 2024-05-08T08:52:55.205 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:55.205 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:55.205 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:55.205 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.205 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.205 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6730 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1421312~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:55.205 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:52:55.205 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.443233068000001, 2024-05-08T08:52:55.205 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052770477000000003, 2024-05-08T08:52:55.206 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:55.206 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:55.206 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:55.206 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:55.206 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:55.206 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:55.206 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6730 2024-05-08T08:52:55.206 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.206 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:55.206 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.206 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:55.206 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:52:55.206 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:55.206 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.207 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.207 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:55.207 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:52:55.207 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:55.207 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.207 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.207 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:55.207 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:52:55.207 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9000000000000002e-07 2024-05-08T08:52:55.207 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.207 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.207 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:55.207 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719254+0000", 2024-05-08T08:52:55.207 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4099999999999998e-06 2024-05-08T08:52:55.207 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.208 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.208 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:55.208 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719263+0000", 2024-05-08T08:52:55.208 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4109999999999996e-06 2024-05-08T08:52:55.208 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.208 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.208 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:55.208 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719268+0000", 2024-05-08T08:52:55.208 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2669999999999997e-06 2024-05-08T08:52:55.208 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.208 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.208 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:55.208 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771846+0000", 2024-05-08T08:52:55.208 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052577677000000003 2024-05-08T08:52:55.209 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.209 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.209 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:55.209 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771920+0000", 2024-05-08T08:52:55.209 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4460000000000002e-05 2024-05-08T08:52:55.209 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.209 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.209 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:55.209 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772022+0000", 2024-05-08T08:52:55.209 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010186199999999999 2024-05-08T08:52:55.209 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:55.209 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:55.209 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:55.209 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.209 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.209 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6732 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1437696~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:55.210 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:52:55.210 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.443208260999999, 2024-05-08T08:52:55.210 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052905575000000003, 2024-05-08T08:52:55.210 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:55.210 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:55.210 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:55.210 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:55.210 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:55.210 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:55.210 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6732 2024-05-08T08:52:55.210 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.210 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:55.210 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.211 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:55.211 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:52:55.211 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:55.211 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.211 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.211 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:55.211 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:52:55.211 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:55.211 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.211 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.211 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:55.211 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721392+0000", 2024-05-08T08:52:55.211 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002115287 2024-05-08T08:52:55.211 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.211 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.212 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:55.212 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721396+0000", 2024-05-08T08:52:55.212 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1679999999999997e-06 2024-05-08T08:52:55.212 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.212 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.212 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:55.212 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721404+0000", 2024-05-08T08:52:55.212 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4409999999999993e-06 2024-05-08T08:52:55.212 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.212 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.212 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:55.212 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721410+0000", 2024-05-08T08:52:55.212 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9150000000000001e-06 2024-05-08T08:52:55.212 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.212 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.212 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:55.213 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772061+0000", 2024-05-08T08:52:55.213 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050651068 2024-05-08T08:52:55.213 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.213 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.213 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:55.213 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772127+0000", 2024-05-08T08:52:55.213 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5424000000000002e-05 2024-05-08T08:52:55.213 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.213 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.213 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:55.213 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772182+0000", 2024-05-08T08:52:55.213 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5272000000000002e-05 2024-05-08T08:52:55.213 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:55.213 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:55.213 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:55.214 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.214 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.214 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6749 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1585152~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:55.214 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:52:55.214 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.440907491000001, 2024-05-08T08:52:55.214 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052350264000000001, 2024-05-08T08:52:55.214 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:55.214 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:55.214 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:55.214 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:55.214 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:55.214 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:55.214 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6749 2024-05-08T08:52:55.214 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.215 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:55.215 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.215 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:55.215 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:52:55.215 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:55.215 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.215 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.215 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:55.215 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:52:55.215 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:55.215 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.215 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.215 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:55.215 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721578+0000", 2024-05-08T08:52:55.215 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0399999999999996e-07 2024-05-08T08:52:55.216 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.216 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.216 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:55.216 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721580+0000", 2024-05-08T08:52:55.216 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5450000000000002e-06 2024-05-08T08:52:55.216 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.216 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.216 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:55.216 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721587+0000", 2024-05-08T08:52:55.216 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0190000000000001e-06 2024-05-08T08:52:55.216 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.216 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.216 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:55.216 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721593+0000", 2024-05-08T08:52:55.216 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5659999999999997e-06 2024-05-08T08:52:55.217 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.217 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.217 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:55.217 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773759+0000", 2024-05-08T08:52:55.217 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052166553999999997 2024-05-08T08:52:55.217 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.217 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.217 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:55.217 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773833+0000", 2024-05-08T08:52:55.217 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3913999999999997e-05 2024-05-08T08:52:55.217 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.217 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.217 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:55.217 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773927+0000", 2024-05-08T08:52:55.217 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4061999999999995e-05 2024-05-08T08:52:55.218 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:55.218 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:55.218 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:55.218 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.218 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.218 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6751 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1601536~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:55.218 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:52:55.218 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.440884212, 2024-05-08T08:52:55.218 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052461302000000001, 2024-05-08T08:52:55.218 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:55.218 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:55.218 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:55.218 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:55.218 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:55.219 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:55.219 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6751 2024-05-08T08:52:55.219 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.219 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:55.219 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.219 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:55.219 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:52:55.219 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:55.219 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.219 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.219 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:55.219 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:52:55.219 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:55.219 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.219 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.220 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:55.220 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722939+0000", 2024-05-08T08:52:55.220 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001338601 2024-05-08T08:52:55.220 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.220 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.220 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:55.220 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722944+0000", 2024-05-08T08:52:55.220 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4469999999999999e-06 2024-05-08T08:52:55.220 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.220 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.220 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:55.220 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722955+0000", 2024-05-08T08:52:55.220 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1892999999999999e-05 2024-05-08T08:52:55.220 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.220 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.221 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:55.221 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722963+0000", 2024-05-08T08:52:55.221 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1080000000000004e-06 2024-05-08T08:52:55.221 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.221 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.221 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:55.221 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773964+0000", 2024-05-08T08:52:55.221 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051001022 2024-05-08T08:52:55.221 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.221 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.221 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:55.221 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.774006+0000", 2024-05-08T08:52:55.221 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2729999999999999e-05 2024-05-08T08:52:55.221 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.221 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.221 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:55.222 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.774062+0000", 2024-05-08T08:52:55.222 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5501000000000003e-05 2024-05-08T08:52:55.222 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:55.222 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:55.222 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:55.222 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.222 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.222 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6760 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1683456~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:55.222 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:52:55.222 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.439374835999999, 2024-05-08T08:52:55.222 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052571312000000002, 2024-05-08T08:52:55.222 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:55.222 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:55.222 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:55.223 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:55.223 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:55.223 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:55.223 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6760 2024-05-08T08:52:55.223 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.223 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:55.223 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.223 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:55.223 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:52:55.223 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:55.223 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.223 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.223 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:55.223 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:52:55.223 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:55.224 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.224 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.224 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:55.224 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:52:55.224 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2100000000000002e-07 2024-05-08T08:52:55.224 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.224 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.224 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:55.224 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723113+0000", 2024-05-08T08:52:55.224 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3149999999999999e-06 2024-05-08T08:52:55.224 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.224 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.224 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:55.224 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723118+0000", 2024-05-08T08:52:55.224 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8860000000000001e-06 2024-05-08T08:52:55.225 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.225 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.225 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:55.225 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723123+0000", 2024-05-08T08:52:55.225 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6770000000000003e-06 2024-05-08T08:52:55.225 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.225 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.225 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:55.225 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775414+0000", 2024-05-08T08:52:55.225 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052290692999999999 2024-05-08T08:52:55.225 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.225 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.225 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:55.225 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775459+0000", 2024-05-08T08:52:55.225 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5639999999999997e-05 2024-05-08T08:52:55.226 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.226 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.226 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:55.226 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775681+0000", 2024-05-08T08:52:55.226 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00022168000000000001 2024-05-08T08:52:55.226 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:55.226 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:55.226 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:55.226 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.226 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.226 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6761 3.16 3:6bd6d6da:::rbd_data.1140782590a5.0000000000000015:6 [read 1095680~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:55.226 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:52:55.226 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.439355616, 2024-05-08T08:52:55.226 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052751275, 2024-05-08T08:52:55.226 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:55.227 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:55.227 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:55.227 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:55.227 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:55.227 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:55.227 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6761 2024-05-08T08:52:55.227 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.227 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:55.227 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.227 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:55.227 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:52:55.227 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:55.227 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.227 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.227 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:55.228 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:52:55.228 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:55.228 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.228 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.228 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:55.228 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724939+0000", 2024-05-08T08:52:55.228 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001809817 2024-05-08T08:52:55.228 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.228 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.228 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:55.228 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724943+0000", 2024-05-08T08:52:55.228 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4920000000000004e-06 2024-05-08T08:52:55.228 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.228 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.229 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:55.229 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724956+0000", 2024-05-08T08:52:55.229 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2427e-05 2024-05-08T08:52:55.229 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.229 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.229 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:55.229 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724963+0000", 2024-05-08T08:52:55.229 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2559999999999996e-06 2024-05-08T08:52:55.229 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.229 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.229 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:55.229 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775711+0000", 2024-05-08T08:52:55.229 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050747621 2024-05-08T08:52:55.229 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.229 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.229 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:55.230 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775785+0000", 2024-05-08T08:52:55.230 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4634000000000004e-05 2024-05-08T08:52:55.230 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.230 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.230 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:55.230 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775880+0000", 2024-05-08T08:52:55.230 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5027999999999999e-05 2024-05-08T08:52:55.230 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:55.230 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:55.230 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:55.230 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.230 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.230 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6770 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1765376~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:55.230 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:52:55.231 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.437324521000001, 2024-05-08T08:52:55.231 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052339296, 2024-05-08T08:52:55.231 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:55.231 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:55.231 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:55.231 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:55.231 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:55.231 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:55.231 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6770 2024-05-08T08:52:55.231 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.231 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:55.231 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.231 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:55.231 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:52:55.231 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:55.231 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.232 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.232 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:55.232 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:52:55.232 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:55.232 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.232 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.232 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:55.232 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725161+0000", 2024-05-08T08:52:55.232 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-05-08T08:52:55.232 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.232 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.232 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:55.232 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725163+0000", 2024-05-08T08:52:55.232 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4370000000000001e-06 2024-05-08T08:52:55.232 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.233 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.233 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:55.233 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725169+0000", 2024-05-08T08:52:55.233 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8409999999999996e-06 2024-05-08T08:52:55.233 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.233 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.233 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:55.233 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725174+0000", 2024-05-08T08:52:55.233 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7249999999999997e-06 2024-05-08T08:52:55.233 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.233 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.233 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:55.233 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777343+0000", 2024-05-08T08:52:55.233 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052169633999999999 2024-05-08T08:52:55.233 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.234 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.234 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:55.234 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777409+0000", 2024-05-08T08:52:55.234 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6033999999999998e-05 2024-05-08T08:52:55.234 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.234 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.234 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:55.234 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777499+0000", 2024-05-08T08:52:55.234 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0224999999999998e-05 2024-05-08T08:52:55.234 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:55.234 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:55.234 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:55.234 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.234 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.235 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6771 3.16 3:6bd6d6da:::rbd_data.1140782590a5.0000000000000015:6 [read 1177600~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:55.235 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:52:55.235 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.437304703999999, 2024-05-08T08:52:55.235 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052438604999999999, 2024-05-08T08:52:55.235 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:55.235 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:55.235 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:55.235 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:55.235 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:55.235 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:55.235 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6771 2024-05-08T08:52:55.235 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.235 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:55.235 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.235 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:55.236 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:52:55.236 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:55.236 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.236 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.236 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:55.236 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:52:55.236 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:55.236 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.236 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.236 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:55.236 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:52:55.236 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:52:55.236 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.236 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.237 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:55.237 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725188+0000", 2024-05-08T08:52:55.237 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1579999999999999e-06 2024-05-08T08:52:55.237 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.237 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.237 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:55.237 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725195+0000", 2024-05-08T08:52:55.237 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8909999999999995e-06 2024-05-08T08:52:55.237 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.237 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.237 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:55.237 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725200+0000", 2024-05-08T08:52:55.237 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9810000000000003e-06 2024-05-08T08:52:55.237 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.237 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.238 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:55.238 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777527+0000", 2024-05-08T08:52:55.238 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052326582000000003 2024-05-08T08:52:55.238 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.238 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.238 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:55.238 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777566+0000", 2024-05-08T08:52:55.238 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9447999999999999e-05 2024-05-08T08:52:55.238 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.238 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.238 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:55.238 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777619+0000", 2024-05-08T08:52:55.238 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2135000000000001e-05 2024-05-08T08:52:55.238 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:55.238 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:55.239 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:55.239 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.239 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.239 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6772 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1781760~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:55.239 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:52:55.239 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.437276647000001, 2024-05-08T08:52:55.239 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052601799999999997, 2024-05-08T08:52:55.239 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:55.239 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:55.239 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:55.239 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:55.239 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:55.239 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:55.239 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6772 2024-05-08T08:52:55.239 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.240 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:55.240 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.240 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:55.240 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:52:55.240 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:55.240 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.240 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.240 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:55.240 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:52:55.240 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:55.240 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.240 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.240 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:55.240 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726939+0000", 2024-05-08T08:52:55.240 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001730493 2024-05-08T08:52:55.241 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.241 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.241 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:55.241 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726943+0000", 2024-05-08T08:52:55.241 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7330000000000003e-06 2024-05-08T08:52:55.241 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.241 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.241 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:55.241 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726954+0000", 2024-05-08T08:52:55.241 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1107000000000001e-05 2024-05-08T08:52:55.241 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.241 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.241 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:55.241 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726962+0000", 2024-05-08T08:52:55.241 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.503e-06 2024-05-08T08:52:55.242 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.242 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.242 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:55.242 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777653+0000", 2024-05-08T08:52:55.242 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050691542999999999 2024-05-08T08:52:55.242 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.242 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.242 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:55.242 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777719+0000", 2024-05-08T08:52:55.242 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5228000000000006e-05 2024-05-08T08:52:55.242 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.242 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.242 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:55.242 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777810+0000", 2024-05-08T08:52:55.243 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1193000000000006e-05 2024-05-08T08:52:55.243 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:55.243 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:55.243 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:55.243 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.243 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.243 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6782 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1880064~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:55.243 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:52:55.243 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.43530814, 2024-05-08T08:52:55.243 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052326492000000002, 2024-05-08T08:52:55.243 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:55.243 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:55.243 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:55.243 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:55.243 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:55.244 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:55.244 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6782 2024-05-08T08:52:55.244 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.244 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:55.244 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.244 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:55.244 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:52:55.244 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:55.244 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.244 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.244 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:55.244 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:52:55.244 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:55.244 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.244 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.244 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:55.245 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728939+0000", 2024-05-08T08:52:55.245 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001762719 2024-05-08T08:52:55.245 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.245 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.245 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:55.245 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728944+0000", 2024-05-08T08:52:55.245 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.746e-06 2024-05-08T08:52:55.245 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.245 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.245 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:55.245 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728955+0000", 2024-05-08T08:52:55.245 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1043000000000001e-05 2024-05-08T08:52:55.245 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.245 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.245 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:55.246 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728963+0000", 2024-05-08T08:52:55.246 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5809999999999999e-06 2024-05-08T08:52:55.246 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.246 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.246 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:55.246 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779320+0000", 2024-05-08T08:52:55.246 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050357290999999998 2024-05-08T08:52:55.246 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.246 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.246 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:55.246 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779401+0000", 2024-05-08T08:52:55.246 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0574000000000005e-05 2024-05-08T08:52:55.246 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:55.246 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:55.246 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:55.247 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779503+0000", 2024-05-08T08:52:55.247 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102538 2024-05-08T08:52:55.247 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:55.247 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:55.247 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:55.247 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:55.247 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:55.247 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:52:55.249 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:52:55.249 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:52:55.251+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:52:55.349 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:55.353+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:55.367 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:52:55.367 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:52:55.367 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.367 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:8641 3.17 3:e8a0a3e4:::rbd_header.11402b26385:head [call rbd.get_data_pool in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-05-08T08:52:55.368 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:55.370962+0000", 2024-05-08T08:52:55.368 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.000120253, 2024-05-08T08:52:55.368 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013801800000000001, 2024-05-08T08:52:55.368 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:55.368 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:55.368 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "reached pg", 2024-05-08T08:52:55.368 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:55.368 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:52:55.368 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:52:55.368 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 8641 2024-05-08T08:52:55.368 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.368 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:55.368 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.368 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:55.368 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.370962+0000", 2024-05-08T08:52:55.369 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:55.369 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.369 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.369 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:55.369 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.370962+0000", 2024-05-08T08:52:55.369 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:55.369 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.369 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.369 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:55.369 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.370963+0000", 2024-05-08T08:52:55.369 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0690000000000001e-06 2024-05-08T08:52:55.369 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.369 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.369 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:55.370 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.370968+0000", 2024-05-08T08:52:55.370 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5589999999999996e-06 2024-05-08T08:52:55.370 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.370 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.370 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:55.370 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.370982+0000", 2024-05-08T08:52:55.370 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.4171e-05 2024-05-08T08:52:55.370 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.370 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.370 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:55.370 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.370990+0000", 2024-05-08T08:52:55.370 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7149999999999998e-06 2024-05-08T08:52:55.370 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.370 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.371 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:55.371 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.371054+0000", 2024-05-08T08:52:55.371 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4067999999999997e-05 2024-05-08T08:52:55.371 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:55.371 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:55.371 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:55.371 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.371 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.371 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "MOSDPGRecoveryDeleteReply(3.17 e49,48 3:ec85a88a:::rbd_data.1140cf79ec85.000000000000000c:head,22'263)", 2024-05-08T08:52:55.371 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:55.371022+0000", 2024-05-08T08:52:55.371 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 6.0009000000000001e-05, 2024-05-08T08:52:55.371 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012837699999999999, 2024-05-08T08:52:55.371 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:55.371 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:55.371 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:55.372 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:55.372 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.372 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:55.372 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.371022+0000", 2024-05-08T08:52:55.372 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:55.372 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.372 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.372 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:55.372 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.371022+0000", 2024-05-08T08:52:55.372 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:55.372 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.372 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.372 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:55.372 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.371022+0000", 2024-05-08T08:52:55.372 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2800000000000002e-07 2024-05-08T08:52:55.373 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.373 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.373 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:55.373 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.371029+0000", 2024-05-08T08:52:55.373 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1860000000000003e-06 2024-05-08T08:52:55.373 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.373 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.373 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:55.373 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.371040+0000", 2024-05-08T08:52:55.373 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0122999999999999e-05 2024-05-08T08:52:55.373 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.373 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.373 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:55.373 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.371047+0000", 2024-05-08T08:52:55.374 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.951e-06 2024-05-08T08:52:55.374 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:55.374 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:55.374 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:55.374 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.374 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.374 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "MOSDPGRecoveryDeleteReply(3.17 e49,48 3:ec85a88a:::rbd_data.1140cf79ec85.000000000000000c:head,22'263)", 2024-05-08T08:52:55.374 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:55.371144+0000", 2024-05-08T08:52:55.374 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 4294967295.9999375, 2024-05-08T08:52:55.374 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6789000000000001e-05, 2024-05-08T08:52:55.374 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:55.374 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:55.374 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:55.374 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:55.375 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.375 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:55.375 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.371144+0000", 2024-05-08T08:52:55.375 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:55.375 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.375 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.375 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:55.375 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.371144+0000", 2024-05-08T08:52:55.375 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:55.375 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.375 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.375 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:55.375 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.371145+0000", 2024-05-08T08:52:55.375 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3499999999999996e-07 2024-05-08T08:52:55.375 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.375 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.376 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:55.376 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.371150+0000", 2024-05-08T08:52:55.376 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8380000000000001e-06 2024-05-08T08:52:55.376 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.376 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.376 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:55.376 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.371162+0000", 2024-05-08T08:52:55.376 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1939999999999999e-05 2024-05-08T08:52:55.376 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.376 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.376 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:55.376 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.371170+0000", 2024-05-08T08:52:55.376 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8790000000000003e-06 2024-05-08T08:52:55.376 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:55.377 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:55.377 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:55.377 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:55.377 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T08:52:55.377 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 3 2024-05-08T08:52:55.377 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:52:55.379 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:52:55.432 INFO:teuthology.orchestra.run.smithi196.stdout:truncating image image_client.0-clone12 from 0xc0c2200 (overlap 0x9f3400) to 0x1fd600 2024-05-08T08:52:55.450 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:52:55.452+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:52:55.530 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:52:55.530 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:52:55.530 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:52:55.530 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:52:55.530 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:52:55.539 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:52:55.550 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:55.552+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:55.651 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:52:55.654+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:52:55.652 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:52:55.652 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:52:55.652 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:52:55.653 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:52:55.653 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.653 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4621.0:9089 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 23339520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:55.653 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:52:55.653 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 23.161754636000001, 2024-05-08T08:52:55.653 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.13814907200000001, 2024-05-08T08:52:55.653 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:55.653 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:55.653 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:55.653 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:55.653 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4621", 2024-05-08T08:52:55.653 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:55.653 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9089 2024-05-08T08:52:55.653 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.654 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:55.654 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.654 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:55.654 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:52:55.654 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:55.654 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.654 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.654 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:55.654 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:52:55.654 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:55.654 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.654 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.654 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:55.655 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493005+0000", 2024-05-08T08:52:55.655 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1700000000000001e-06 2024-05-08T08:52:55.655 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.655 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.655 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:55.655 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.513963+0000", 2024-05-08T08:52:55.655 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.020957581999999999 2024-05-08T08:52:55.655 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.655 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.655 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:55.655 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.513995+0000", 2024-05-08T08:52:55.655 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1993999999999999e-05 2024-05-08T08:52:55.655 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.655 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.655 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:55.656 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514007+0000", 2024-05-08T08:52:55.656 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1581e-05 2024-05-08T08:52:55.656 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.656 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.656 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:55.656 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514122+0000", 2024-05-08T08:52:55.656 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000115282 2024-05-08T08:52:55.656 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.656 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.656 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op must wait for map", 2024-05-08T08:52:55.656 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514128+0000", 2024-05-08T08:52:55.656 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9950000000000002e-06 2024-05-08T08:52:55.656 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.656 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.656 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:55.657 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.515800+0000", 2024-05-08T08:52:55.657 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001671886 2024-05-08T08:52:55.657 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.657 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.657 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:55.657 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.515893+0000", 2024-05-08T08:52:55.657 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3490000000000001e-05 2024-05-08T08:52:55.657 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.657 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.657 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 3", 2024-05-08T08:52:55.657 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.517647+0000", 2024-05-08T08:52:55.657 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0017535630000000001 2024-05-08T08:52:55.657 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.657 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.657 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:55.658 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.627578+0000", 2024-05-08T08:52:55.658 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.109931557 2024-05-08T08:52:55.658 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.658 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.658 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:55.658 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631055+0000", 2024-05-08T08:52:55.658 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0034761420000000002 2024-05-08T08:52:55.658 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.658 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.658 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:55.658 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631096+0000", 2024-05-08T08:52:55.658 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1059000000000001e-05 2024-05-08T08:52:55.658 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.658 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.659 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:55.659 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631151+0000", 2024-05-08T08:52:55.659 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5770999999999998e-05 2024-05-08T08:52:55.659 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:55.659 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:55.659 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:55.659 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.659 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.659 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1065 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 10944512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:55.659 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485411+0000", 2024-05-08T08:52:55.659 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 19.16934595, 2024-05-08T08:52:55.659 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070281386000000001, 2024-05-08T08:52:55.659 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:55.659 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:55.659 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:55.660 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:55.660 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:55.660 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:55.660 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1065 2024-05-08T08:52:55.660 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.660 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:55.660 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.660 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:55.660 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485411+0000", 2024-05-08T08:52:55.660 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:55.660 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.660 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.660 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:55.660 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485411+0000", 2024-05-08T08:52:55.660 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:55.661 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.661 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.661 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:55.661 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485411+0000", 2024-05-08T08:52:55.661 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5600000000000001e-07 2024-05-08T08:52:55.661 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.661 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.661 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:55.661 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485414+0000", 2024-05-08T08:52:55.661 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2299999999999998e-06 2024-05-08T08:52:55.661 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.661 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.661 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:55.661 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485420+0000", 2024-05-08T08:52:55.661 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2179999999999996e-06 2024-05-08T08:52:55.662 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.662 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.662 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:55.662 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485425+0000", 2024-05-08T08:52:55.662 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9830000000000004e-06 2024-05-08T08:52:55.662 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.662 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.662 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:55.662 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555468+0000", 2024-05-08T08:52:55.662 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070043541000000001 2024-05-08T08:52:55.663 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.663 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.663 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:55.663 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555566+0000", 2024-05-08T08:52:55.663 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8082e-05 2024-05-08T08:52:55.663 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.663 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.663 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:55.663 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555692+0000", 2024-05-08T08:52:55.663 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000125976 2024-05-08T08:52:55.663 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:55.663 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:55.663 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:55.663 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.663 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.663 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1067 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 10985472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:55.664 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485431+0000", 2024-05-08T08:52:55.664 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 19.169325815000001, 2024-05-08T08:52:55.664 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070412460999999996, 2024-05-08T08:52:55.664 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:55.664 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:55.664 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:55.664 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:55.664 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:55.664 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:55.664 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1067 2024-05-08T08:52:55.664 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.664 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:55.665 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.665 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:55.665 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485431+0000", 2024-05-08T08:52:55.665 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:55.665 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.665 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.665 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:55.665 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485431+0000", 2024-05-08T08:52:55.665 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:55.665 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.665 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.666 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:55.666 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485432+0000", 2024-05-08T08:52:55.666 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8299999999999998e-07 2024-05-08T08:52:55.666 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.666 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.666 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:55.666 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485434+0000", 2024-05-08T08:52:55.666 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0849999999999999e-06 2024-05-08T08:52:55.666 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.666 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.666 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:55.666 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485440+0000", 2024-05-08T08:52:55.666 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0020000000000003e-06 2024-05-08T08:52:55.667 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.667 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.667 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:55.667 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485445+0000", 2024-05-08T08:52:55.667 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5180000000000004e-06 2024-05-08T08:52:55.667 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.667 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.667 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:55.667 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555731+0000", 2024-05-08T08:52:55.667 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070286214 2024-05-08T08:52:55.667 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.667 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.667 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:55.667 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555787+0000", 2024-05-08T08:52:55.668 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5992999999999997e-05 2024-05-08T08:52:55.668 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.668 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.668 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:55.668 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555844+0000", 2024-05-08T08:52:55.668 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6266e-05 2024-05-08T08:52:55.668 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:55.668 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:55.668 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:55.668 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.668 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.668 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1071 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11026432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:55.668 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:52:55.668 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 19.169305144999999, 2024-05-08T08:52:55.668 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070580037999999998, 2024-05-08T08:52:55.669 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:55.669 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:55.669 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:55.669 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:55.669 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:55.669 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:55.669 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1071 2024-05-08T08:52:55.669 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.669 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:55.669 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.669 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:55.669 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:52:55.669 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:55.669 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.669 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.670 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:55.670 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:52:55.670 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:55.670 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.670 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.670 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:55.670 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:52:55.670 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2800000000000002e-07 2024-05-08T08:52:55.670 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.670 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.670 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:55.670 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485454+0000", 2024-05-08T08:52:55.670 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2290000000000002e-06 2024-05-08T08:52:55.670 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.671 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.671 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:55.671 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485461+0000", 2024-05-08T08:52:55.671 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9920000000000001e-06 2024-05-08T08:52:55.671 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.671 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.671 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:55.671 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485467+0000", 2024-05-08T08:52:55.671 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3979999999999998e-06 2024-05-08T08:52:55.671 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.671 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.671 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:55.671 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555880+0000", 2024-05-08T08:52:55.671 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070413165 2024-05-08T08:52:55.672 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.672 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.672 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:55.672 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555926+0000", 2024-05-08T08:52:55.672 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6217000000000002e-05 2024-05-08T08:52:55.672 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.672 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.672 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:55.672 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556032+0000", 2024-05-08T08:52:55.672 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000105609 2024-05-08T08:52:55.672 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:55.672 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:55.673 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:55.673 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.673 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.673 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1073 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11067392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:55.673 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:52:55.673 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 19.169283923999998, 2024-05-08T08:52:55.673 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070821404000000004, 2024-05-08T08:52:55.673 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:55.673 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:55.673 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:55.673 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:55.673 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:55.673 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:55.673 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1073 2024-05-08T08:52:55.674 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.674 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:55.674 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.674 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:55.674 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:52:55.674 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:55.674 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.674 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.674 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:55.674 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:52:55.674 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:55.674 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.674 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.674 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:55.675 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:52:55.675 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.08e-07 2024-05-08T08:52:55.675 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.675 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.675 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:55.675 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485476+0000", 2024-05-08T08:52:55.675 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3929999999999998e-06 2024-05-08T08:52:55.675 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.675 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.675 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:55.675 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485485+0000", 2024-05-08T08:52:55.675 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1880000000000008e-06 2024-05-08T08:52:55.675 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.675 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.675 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:55.676 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485494+0000", 2024-05-08T08:52:55.676 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3109999999999995e-06 2024-05-08T08:52:55.676 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.676 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.676 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:55.676 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556082+0000", 2024-05-08T08:52:55.676 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070587426999999994 2024-05-08T08:52:55.676 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.676 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.676 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:55.676 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556177+0000", 2024-05-08T08:52:55.676 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5051999999999997e-05 2024-05-08T08:52:55.676 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.676 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.677 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:55.677 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556294+0000", 2024-05-08T08:52:55.677 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000117625 2024-05-08T08:52:55.677 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:55.677 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:55.677 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:55.677 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.677 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.677 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1075 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11108352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:55.677 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:52:55.677 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 19.169253347000001, 2024-05-08T08:52:55.677 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071045026999999997, 2024-05-08T08:52:55.677 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:55.677 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:55.677 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:55.678 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:55.678 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:55.678 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:55.678 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1075 2024-05-08T08:52:55.678 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.678 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:55.678 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.678 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:55.678 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:52:55.678 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:55.678 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.678 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.678 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:55.678 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:52:55.678 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:55.679 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.679 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.679 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:55.679 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:52:55.679 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.32e-07 2024-05-08T08:52:55.679 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.679 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.679 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:55.679 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485518+0000", 2024-05-08T08:52:55.679 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3769e-05 2024-05-08T08:52:55.679 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.679 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.679 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:55.679 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485527+0000", 2024-05-08T08:52:55.679 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6779999999999995e-06 2024-05-08T08:52:55.680 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.680 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.680 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:55.680 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485532+0000", 2024-05-08T08:52:55.680 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9830000000000004e-06 2024-05-08T08:52:55.680 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.680 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.680 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:55.680 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556340+0000", 2024-05-08T08:52:55.680 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070807743000000006 2024-05-08T08:52:55.680 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.680 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.680 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:55.680 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556435+0000", 2024-05-08T08:52:55.681 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4732e-05 2024-05-08T08:52:55.681 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.681 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.681 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:55.681 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556549+0000", 2024-05-08T08:52:55.681 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011369 2024-05-08T08:52:55.681 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:55.681 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:55.681 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:55.681 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.681 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.681 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1078 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11149312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:55.681 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:52:55.681 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 19.169213148000001, 2024-05-08T08:52:55.682 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071167276000000002, 2024-05-08T08:52:55.682 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:55.682 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:55.682 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:55.682 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:55.682 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:55.682 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:55.682 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1078 2024-05-08T08:52:55.682 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.682 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:55.682 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.682 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:55.682 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:52:55.682 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:55.683 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.683 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.683 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:55.683 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:52:55.683 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:55.683 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.683 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.683 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:55.683 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:52:55.683 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5600000000000001e-07 2024-05-08T08:52:55.683 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.683 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.683 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:55.683 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485547+0000", 2024-05-08T08:52:55.683 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3590000000000002e-06 2024-05-08T08:52:55.684 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.684 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.684 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:55.684 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485553+0000", 2024-05-08T08:52:55.684 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2770000000000002e-06 2024-05-08T08:52:55.684 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.684 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.684 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:55.684 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485558+0000", 2024-05-08T08:52:55.684 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.95e-06 2024-05-08T08:52:55.684 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.684 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.684 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:55.684 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556582+0000", 2024-05-08T08:52:55.684 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071024444000000006 2024-05-08T08:52:55.685 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.685 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.685 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:55.685 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556637+0000", 2024-05-08T08:52:55.685 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4672000000000001e-05 2024-05-08T08:52:55.685 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.685 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.685 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:55.685 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556711+0000", 2024-05-08T08:52:55.685 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4117999999999997e-05 2024-05-08T08:52:55.685 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:55.685 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:55.685 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:55.685 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.686 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.686 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1080 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11190272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:55.686 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:52:55.686 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 19.169192415000001, 2024-05-08T08:52:55.686 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071350721000000006, 2024-05-08T08:52:55.686 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:55.686 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:55.686 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:55.686 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:55.686 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:55.686 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:55.686 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1080 2024-05-08T08:52:55.686 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.686 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:55.687 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.687 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:55.687 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:52:55.687 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:55.687 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.687 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.687 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:55.687 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:52:55.687 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:55.687 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.687 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.687 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:55.687 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:52:55.687 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.03e-07 2024-05-08T08:52:55.688 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.688 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.688 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:55.688 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485567+0000", 2024-05-08T08:52:55.688 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1090000000000001e-06 2024-05-08T08:52:55.688 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.688 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.688 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:55.688 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485573+0000", 2024-05-08T08:52:55.688 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3049999999999998e-06 2024-05-08T08:52:55.688 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.688 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.688 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:55.688 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485578+0000", 2024-05-08T08:52:55.688 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0130000000000001e-06 2024-05-08T08:52:55.689 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.689 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.689 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:55.689 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556743+0000", 2024-05-08T08:52:55.689 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071164449000000005 2024-05-08T08:52:55.689 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.689 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.689 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:55.689 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556797+0000", 2024-05-08T08:52:55.689 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4589999999999997e-05 2024-05-08T08:52:55.689 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.689 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.689 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:55.689 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556915+0000", 2024-05-08T08:52:55.689 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000117852 2024-05-08T08:52:55.690 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:55.690 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:55.690 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:55.690 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.690 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.690 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1082 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11231232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:55.690 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:52:55.690 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 19.169172172, 2024-05-08T08:52:55.690 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071591396000000002, 2024-05-08T08:52:55.690 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:55.690 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:55.690 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:55.690 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:55.690 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:55.691 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:55.691 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1082 2024-05-08T08:52:55.691 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.691 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:55.691 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.691 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:55.691 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:52:55.691 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:55.691 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.691 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.691 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:55.691 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:52:55.691 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:55.691 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.691 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.692 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:55.692 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:52:55.692 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1800000000000001e-07 2024-05-08T08:52:55.692 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.692 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.692 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:55.692 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485587+0000", 2024-05-08T08:52:55.692 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2589999999999999e-06 2024-05-08T08:52:55.692 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.692 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.692 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:55.692 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485594+0000", 2024-05-08T08:52:55.692 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0789999999999997e-06 2024-05-08T08:52:55.692 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.692 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.693 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:55.693 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485598+0000", 2024-05-08T08:52:55.693 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7480000000000001e-06 2024-05-08T08:52:55.693 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.693 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.693 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:55.693 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556971+0000", 2024-05-08T08:52:55.693 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071372774999999999 2024-05-08T08:52:55.693 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.693 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.693 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:55.693 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557069+0000", 2024-05-08T08:52:55.693 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.7732999999999994e-05 2024-05-08T08:52:55.693 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.694 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.694 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:55.694 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557176+0000", 2024-05-08T08:52:55.694 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000107384 2024-05-08T08:52:55.694 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:55.694 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:55.694 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:55.694 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.694 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.694 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1084 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11272192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:55.694 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:52:55.694 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 19.169133533, 2024-05-08T08:52:55.694 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071808977999999996, 2024-05-08T08:52:55.694 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:55.695 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:55.695 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:55.695 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:55.695 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:55.695 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:55.695 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1084 2024-05-08T08:52:55.695 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.695 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:55.695 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.695 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:55.695 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:52:55.695 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:55.695 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.695 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.695 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:55.696 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:52:55.696 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:55.696 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.696 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.696 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:55.696 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485624+0000", 2024-05-08T08:52:55.696 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:52:55.696 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.696 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.696 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:55.696 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485626+0000", 2024-05-08T08:52:55.696 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9460000000000001e-06 2024-05-08T08:52:55.696 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.696 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.697 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:55.697 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485633+0000", 2024-05-08T08:52:55.697 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1149999999999996e-06 2024-05-08T08:52:55.697 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.697 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.697 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:55.697 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485638+0000", 2024-05-08T08:52:55.697 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5670000000000002e-06 2024-05-08T08:52:55.697 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.697 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.697 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:55.697 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557226+0000", 2024-05-08T08:52:55.697 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071587267999999996 2024-05-08T08:52:55.697 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.697 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.698 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:55.698 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557321+0000", 2024-05-08T08:52:55.698 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5785000000000005e-05 2024-05-08T08:52:55.698 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.698 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.698 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:55.698 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557432+0000", 2024-05-08T08:52:55.698 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011092800000000001 2024-05-08T08:52:55.698 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:55.698 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:55.698 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:55.698 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.698 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.698 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1086 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11313152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:55.699 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:52:55.699 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 19.169110205999999, 2024-05-08T08:52:55.699 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071933132999999996, 2024-05-08T08:52:55.699 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:55.699 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:55.699 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:55.699 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:55.699 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:55.699 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:55.699 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1086 2024-05-08T08:52:55.699 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.699 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:55.699 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.699 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:55.699 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:52:55.700 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:55.700 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.700 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.700 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:55.700 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:52:55.700 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:55.700 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.700 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.700 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:55.700 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486328+0000", 2024-05-08T08:52:55.700 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00068107900000000002 2024-05-08T08:52:55.700 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.700 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.700 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:55.701 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486331+0000", 2024-05-08T08:52:55.701 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.585e-06 2024-05-08T08:52:55.701 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.701 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.701 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:55.701 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486343+0000", 2024-05-08T08:52:55.701 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1856e-05 2024-05-08T08:52:55.701 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.701 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.701 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:55.701 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486350+0000", 2024-05-08T08:52:55.701 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8600000000000004e-06 2024-05-08T08:52:55.701 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.701 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.701 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:55.702 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557470+0000", 2024-05-08T08:52:55.702 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071120326999999997 2024-05-08T08:52:55.702 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.702 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.702 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:55.702 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557514+0000", 2024-05-08T08:52:55.702 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3909000000000003e-05 2024-05-08T08:52:55.702 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.702 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.702 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:55.702 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557580+0000", 2024-05-08T08:52:55.702 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5517000000000003e-05 2024-05-08T08:52:55.702 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:55.702 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:55.703 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:55.703 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.703 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.703 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1089 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11354112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:55.703 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:52:55.703 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 19.168397334000002, 2024-05-08T08:52:55.703 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071457381, 2024-05-08T08:52:55.703 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:55.703 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:55.703 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:55.703 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:55.703 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:55.703 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:55.703 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1089 2024-05-08T08:52:55.703 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.703 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:55.704 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.704 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:55.704 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:52:55.704 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:55.704 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.704 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.704 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:55.704 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:52:55.704 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:55.704 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.704 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.704 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:55.704 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486851+0000", 2024-05-08T08:52:55.704 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00049102 2024-05-08T08:52:55.705 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.705 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.705 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:55.705 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486854+0000", 2024-05-08T08:52:55.705 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6859999999999999e-06 2024-05-08T08:52:55.705 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.705 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.705 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:55.705 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486864+0000", 2024-05-08T08:52:55.705 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6080000000000002e-06 2024-05-08T08:52:55.705 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.705 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.705 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:55.705 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486870+0000", 2024-05-08T08:52:55.705 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.477e-06 2024-05-08T08:52:55.706 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.706 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.706 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:55.706 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557612+0000", 2024-05-08T08:52:55.706 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070741557999999996 2024-05-08T08:52:55.706 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.706 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.706 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:55.706 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557695+0000", 2024-05-08T08:52:55.706 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3300000000000005e-05 2024-05-08T08:52:55.706 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.706 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.706 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:55.706 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557817+0000", 2024-05-08T08:52:55.706 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012173200000000001 2024-05-08T08:52:55.707 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:55.707 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:55.707 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:55.707 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.707 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.707 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1091 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11395072~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:55.707 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:52:55.707 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 19.167876539000002, 2024-05-08T08:52:55.707 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071184737999999997, 2024-05-08T08:52:55.707 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:55.707 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:55.707 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:55.707 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:55.708 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:55.708 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:55.708 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1091 2024-05-08T08:52:55.708 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.708 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:55.708 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.708 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:55.708 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:52:55.708 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:55.708 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.708 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.708 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:55.708 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:52:55.708 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:55.708 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.709 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.709 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:55.709 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486881+0000", 2024-05-08T08:52:55.709 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0699999999999997e-07 2024-05-08T08:52:55.709 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.709 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.709 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:55.709 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486883+0000", 2024-05-08T08:52:55.709 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.402e-06 2024-05-08T08:52:55.709 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.709 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.709 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:55.709 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486890+0000", 2024-05-08T08:52:55.709 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0480000000000001e-06 2024-05-08T08:52:55.709 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.710 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.710 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:55.710 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486896+0000", 2024-05-08T08:52:55.710 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7640000000000002e-06 2024-05-08T08:52:55.710 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.710 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.710 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:55.710 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557860+0000", 2024-05-08T08:52:55.710 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070963791999999998 2024-05-08T08:52:55.710 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.710 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.710 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:55.710 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557952+0000", 2024-05-08T08:52:55.710 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2082000000000004e-05 2024-05-08T08:52:55.711 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.711 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.711 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:55.711 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558065+0000", 2024-05-08T08:52:55.711 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000113143 2024-05-08T08:52:55.711 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:55.711 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:55.711 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:55.711 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.711 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.711 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1094 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11436032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:55.711 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:52:55.711 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 19.167853371, 2024-05-08T08:52:55.711 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071400507000000002, 2024-05-08T08:52:55.711 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:55.712 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:55.712 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:55.712 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:55.712 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:55.712 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:55.712 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1094 2024-05-08T08:52:55.712 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.712 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:55.712 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.712 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:55.712 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:52:55.712 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:55.712 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.712 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.712 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:55.713 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:52:55.713 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:55.713 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.713 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.713 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:55.713 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487940+0000", 2024-05-08T08:52:55.713 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001036868 2024-05-08T08:52:55.713 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.713 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.713 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:55.713 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487944+0000", 2024-05-08T08:52:55.713 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7069999999999998e-06 2024-05-08T08:52:55.713 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.713 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.713 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:55.714 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487951+0000", 2024-05-08T08:52:55.714 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2649999999999999e-06 2024-05-08T08:52:55.714 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.714 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.714 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:55.714 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487961+0000", 2024-05-08T08:52:55.714 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4110000000000002e-06 2024-05-08T08:52:55.714 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.714 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.714 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:55.714 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558117+0000", 2024-05-08T08:52:55.714 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070156662999999994 2024-05-08T08:52:55.714 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.714 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.714 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:55.715 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558214+0000", 2024-05-08T08:52:55.715 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6444e-05 2024-05-08T08:52:55.715 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.715 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.715 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:55.715 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558304+0000", 2024-05-08T08:52:55.715 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0148999999999994e-05 2024-05-08T08:52:55.715 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:55.715 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:55.715 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:55.715 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.715 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.715 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1096 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11476992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:55.715 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:52:55.715 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 19.16678684, 2024-05-08T08:52:55.716 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070480074000000004, 2024-05-08T08:52:55.716 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:55.716 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:55.716 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:55.716 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:55.716 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:55.716 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:55.716 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1096 2024-05-08T08:52:55.716 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.716 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:55.716 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.716 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:55.716 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:52:55.716 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:55.716 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.717 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.717 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:55.717 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:52:55.717 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:55.717 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.717 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.717 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:55.717 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:52:55.717 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3700000000000001e-07 2024-05-08T08:52:55.717 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.717 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.717 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:55.717 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487973+0000", 2024-05-08T08:52:55.717 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0870000000000001e-06 2024-05-08T08:52:55.717 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.718 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.718 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:55.718 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487977+0000", 2024-05-08T08:52:55.718 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0600000000000001e-06 2024-05-08T08:52:55.718 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.718 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.718 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:55.718 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487980+0000", 2024-05-08T08:52:55.718 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1089999999999998e-06 2024-05-08T08:52:55.718 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.718 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.718 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:55.718 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558338+0000", 2024-05-08T08:52:55.718 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070358233000000006 2024-05-08T08:52:55.718 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.719 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.719 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:55.719 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558392+0000", 2024-05-08T08:52:55.719 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4537999999999999e-05 2024-05-08T08:52:55.719 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.719 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.719 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:55.719 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558450+0000", 2024-05-08T08:52:55.719 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7710000000000001e-05 2024-05-08T08:52:55.719 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:55.719 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:55.719 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:55.719 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.719 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.720 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1099 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11517952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:55.720 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:52:55.720 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 19.166772964, 2024-05-08T08:52:55.720 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070720506000000002, 2024-05-08T08:52:55.720 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:55.720 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:55.720 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:55.720 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:55.720 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:55.720 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:55.720 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1099 2024-05-08T08:52:55.720 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.720 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:55.720 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.721 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:55.721 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:52:55.721 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:55.721 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.721 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.721 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:55.721 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:52:55.721 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:55.721 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.721 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.721 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:55.721 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:52:55.721 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3999999999999998e-07 2024-05-08T08:52:55.721 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.721 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.722 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:55.722 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487986+0000", 2024-05-08T08:52:55.722 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3519999999999999e-06 2024-05-08T08:52:55.722 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.722 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.722 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:55.722 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487989+0000", 2024-05-08T08:52:55.722 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.23e-06 2024-05-08T08:52:55.722 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.722 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.722 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:55.722 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487991+0000", 2024-05-08T08:52:55.722 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6350000000000002e-06 2024-05-08T08:52:55.722 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.723 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.723 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:55.723 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558489+0000", 2024-05-08T08:52:55.723 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070497924000000003 2024-05-08T08:52:55.723 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.723 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.723 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:55.723 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558583+0000", 2024-05-08T08:52:55.723 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3405000000000004e-05 2024-05-08T08:52:55.723 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.723 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.723 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:55.723 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558704+0000", 2024-05-08T08:52:55.723 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012171999999999999 2024-05-08T08:52:55.723 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:55.724 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:55.724 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:55.724 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.724 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.724 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1100 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11558912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:55.724 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:52:55.724 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 19.166759965000001, 2024-05-08T08:52:55.724 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070962266999999996, 2024-05-08T08:52:55.724 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:55.724 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:55.724 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:55.724 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:55.724 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:55.724 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:55.725 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1100 2024-05-08T08:52:55.725 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.725 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:55.725 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.725 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:55.725 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:52:55.725 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:55.725 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.725 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.725 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:55.725 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:52:55.725 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:55.725 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.725 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.725 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:55.726 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488430+0000", 2024-05-08T08:52:55.726 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000432676 2024-05-08T08:52:55.726 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.726 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.726 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:55.726 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488433+0000", 2024-05-08T08:52:55.726 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.45e-06 2024-05-08T08:52:55.726 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.726 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.726 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:55.726 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488442+0000", 2024-05-08T08:52:55.726 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.687e-06 2024-05-08T08:52:55.726 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.727 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.727 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:55.727 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488446+0000", 2024-05-08T08:52:55.727 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4939999999999997e-06 2024-05-08T08:52:55.727 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.727 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.727 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:55.727 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558756+0000", 2024-05-08T08:52:55.727 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070310051999999998 2024-05-08T08:52:55.727 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.727 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.727 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:55.727 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558844+0000", 2024-05-08T08:52:55.727 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7690999999999998e-05 2024-05-08T08:52:55.727 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.728 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.728 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:55.728 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558959+0000", 2024-05-08T08:52:55.728 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000115217 2024-05-08T08:52:55.728 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:55.728 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:55.728 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:55.728 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.728 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.728 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1103 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11599872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:55.728 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:52:55.728 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 19.166302290000001, 2024-05-08T08:52:55.728 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070777100999999995, 2024-05-08T08:52:55.728 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:55.728 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:55.729 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:55.729 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:55.729 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:55.729 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:55.729 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1103 2024-05-08T08:52:55.729 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.729 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:55.729 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.729 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:55.729 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:52:55.729 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:55.729 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.729 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.729 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:55.730 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:52:55.730 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:55.730 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.730 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.730 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:55.730 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488959+0000", 2024-05-08T08:52:55.730 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00050457200000000001 2024-05-08T08:52:55.730 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.730 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.730 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:55.730 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488969+0000", 2024-05-08T08:52:55.730 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.7159999999999998e-06 2024-05-08T08:52:55.730 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.730 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.730 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:55.731 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488981+0000", 2024-05-08T08:52:55.731 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2150000000000001e-05 2024-05-08T08:52:55.731 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.731 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.731 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:55.731 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488988+0000", 2024-05-08T08:52:55.731 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8759999999999999e-06 2024-05-08T08:52:55.731 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.731 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.731 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:55.731 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558992+0000", 2024-05-08T08:52:55.731 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070004358000000003 2024-05-08T08:52:55.731 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.732 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.733 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:55.733 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559093+0000", 2024-05-08T08:52:55.733 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000100299 2024-05-08T08:52:55.733 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.733 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.733 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:55.733 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559232+0000", 2024-05-08T08:52:55.733 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013913 2024-05-08T08:52:55.733 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:55.733 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:55.733 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:55.734 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.734 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.734 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1104 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11640832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:55.734 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:52:55.734 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 19.165757474999999, 2024-05-08T08:52:55.734 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070521947000000001, 2024-05-08T08:52:55.734 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:55.734 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:55.734 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:55.734 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:55.734 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:55.734 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:55.734 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1104 2024-05-08T08:52:55.734 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.734 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:55.735 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.735 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:55.735 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:52:55.735 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:55.735 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.735 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.735 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:55.735 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:52:55.735 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:55.735 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.735 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.735 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:55.735 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489700+0000", 2024-05-08T08:52:55.735 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00070067600000000001 2024-05-08T08:52:55.736 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.736 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.736 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:55.736 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489704+0000", 2024-05-08T08:52:55.736 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1640000000000003e-06 2024-05-08T08:52:55.737 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.737 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.737 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:55.737 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489716+0000", 2024-05-08T08:52:55.737 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1379000000000001e-05 2024-05-08T08:52:55.737 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.737 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.737 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:55.737 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489724+0000", 2024-05-08T08:52:55.737 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8949999999999998e-06 2024-05-08T08:52:55.737 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.737 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.737 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:55.737 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559278+0000", 2024-05-08T08:52:55.737 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.069554151999999994 2024-05-08T08:52:55.738 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.738 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.738 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:55.738 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559387+0000", 2024-05-08T08:52:55.738 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000109258 2024-05-08T08:52:55.738 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.738 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.738 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:55.738 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559521+0000", 2024-05-08T08:52:55.738 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000134423 2024-05-08T08:52:55.738 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:55.738 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:55.738 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:55.738 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.738 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.739 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:8636 3.17 3:e8a0a3e4:::rbd_header.11402b26385:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-05-08T08:52:55.739 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:54.150971+0000", 2024-05-08T08:52:55.739 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 1.503785481, 2024-05-08T08:52:55.739 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.217273233, 2024-05-08T08:52:55.739 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:55.739 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:55.739 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:55.739 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:55.739 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:52:55.739 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:52:55.739 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 8636 2024-05-08T08:52:55.739 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.739 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:55.739 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.739 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:55.740 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150971+0000", 2024-05-08T08:52:55.740 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:55.740 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.740 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.740 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:55.740 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150971+0000", 2024-05-08T08:52:55.740 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:55.740 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.740 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.740 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:55.740 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150972+0000", 2024-05-08T08:52:55.740 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.71e-07 2024-05-08T08:52:55.740 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.740 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.740 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:55.741 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150977+0000", 2024-05-08T08:52:55.741 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.152e-06 2024-05-08T08:52:55.741 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.741 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.741 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:55.741 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150990+0000", 2024-05-08T08:52:55.741 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2364999999999999e-05 2024-05-08T08:52:55.741 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.741 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.741 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:55.741 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150998+0000", 2024-05-08T08:52:55.741 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0739999999999993e-06 2024-05-08T08:52:55.741 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.741 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.741 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:55.741 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.151054+0000", 2024-05-08T08:52:55.742 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5949000000000003e-05 2024-05-08T08:52:55.742 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.742 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.742 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:52:55.742 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.151060+0000", 2024-05-08T08:52:55.742 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6120000000000004e-06 2024-05-08T08:52:55.742 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.742 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.742 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:55.742 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.367640+0000", 2024-05-08T08:52:55.742 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2165792739999999 2024-05-08T08:52:55.742 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.742 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.742 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:55.743 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.367839+0000", 2024-05-08T08:52:55.743 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000199185 2024-05-08T08:52:55.743 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.743 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.743 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:55.743 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.368245+0000", 2024-05-08T08:52:55.743 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00040575099999999998 2024-05-08T08:52:55.743 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:55.743 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:55.743 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:55.743 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:55.743 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:55.743 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:52:55.745 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:52:55.786 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:52:55.789+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:52:55.853 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:55.856+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:55.868 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T08:52:55.888 INFO:teuthology.orchestra.run.smithi196.stdout:checking clone #11, image image_client.0-clone11 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent12 2024-05-08T08:52:55.903 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:52:55.903 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:52:55.904 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.904 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4796.0:101 3.1a 3:5f64fd19:::rbd_data.1140f00e9ce3.0000000000000004:head [sparse-read 270336~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-05-08T08:52:55.904 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:55.906032+0000", 2024-05-08T08:52:55.904 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0010613910000000001, 2024-05-08T08:52:55.904 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0010834830000000001, 2024-05-08T08:52:55.904 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:55.904 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:55.904 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:55.904 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:55.904 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4796", 2024-05-08T08:52:55.904 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1803394514", 2024-05-08T08:52:55.904 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 101 2024-05-08T08:52:55.904 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.905 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:55.905 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.905 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:55.905 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.906032+0000", 2024-05-08T08:52:55.905 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:55.905 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.905 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.905 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:55.905 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.906032+0000", 2024-05-08T08:52:55.905 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:55.905 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.905 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.905 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:55.905 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.906033+0000", 2024-05-08T08:52:55.905 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7499999999999999e-07 2024-05-08T08:52:55.906 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.906 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.906 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:55.906 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.906038+0000", 2024-05-08T08:52:55.906 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4159999999999997e-06 2024-05-08T08:52:55.906 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.906 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.906 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:55.906 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.906051+0000", 2024-05-08T08:52:55.906 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3899e-05 2024-05-08T08:52:55.906 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.906 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.906 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:55.907 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.906059+0000", 2024-05-08T08:52:55.907 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7449999999999995e-06 2024-05-08T08:52:55.907 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.907 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.907 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:55.907 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.906482+0000", 2024-05-08T08:52:55.907 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00042282399999999998 2024-05-08T08:52:55.907 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.907 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.907 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:55.907 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.906523+0000", 2024-05-08T08:52:55.907 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0729999999999998e-05 2024-05-08T08:52:55.907 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:55.908 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:55.908 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:55.908 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.908 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.908 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4796.0:102 3.1a 3.98bf26fa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-05-08T08:52:55.908 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:55.906835+0000", 2024-05-08T08:52:55.908 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.00025900100000000002, 2024-05-08T08:52:55.908 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000321207, 2024-05-08T08:52:55.908 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:55.908 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:55.908 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:52:55.908 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:55.908 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4796", 2024-05-08T08:52:55.908 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1803394514", 2024-05-08T08:52:55.909 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 102 2024-05-08T08:52:55.909 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.909 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:55.909 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.909 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:55.909 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.906835+0000", 2024-05-08T08:52:55.909 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:55.909 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.909 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.909 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:55.909 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.906835+0000", 2024-05-08T08:52:55.909 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:55.909 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.909 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.910 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:55.910 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.906835+0000", 2024-05-08T08:52:55.910 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.6099999999999999e-07 2024-05-08T08:52:55.910 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.910 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.910 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:55.910 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.906840+0000", 2024-05-08T08:52:55.910 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1949999999999996e-06 2024-05-08T08:52:55.910 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.910 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.910 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:55.910 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.906855+0000", 2024-05-08T08:52:55.910 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5296000000000001e-05 2024-05-08T08:52:55.911 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:55.911 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:55.911 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:55.911 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.906863+0000", 2024-05-08T08:52:55.911 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1580000000000005e-06 2024-05-08T08:52:55.911 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:55.911 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:55.911 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:55.911 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:55.911 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T08:52:55.911 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 2 2024-05-08T08:52:55.911 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:52:55.914 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:52:55.954 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:55.956+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:56.000 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:52:56.000 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:52:56.001 INFO:teuthology.orchestra.run.smithi196.stdout:2301 trunc from 0xc0c2200 to 0x4f48800 2024-05-08T08:52:56.028 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:52:56.028 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:52:56.047 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:52:56.047 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:52:56.055 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:52:56.058+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:52:56.055 INFO:teuthology.orchestra.run.smithi196.stdout:2303 write 0xcf01200 thru 0xcf02c91 (0x1a92 bytes) 2024-05-08T08:52:56.074 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:52:56.075 INFO:teuthology.orchestra.run.smithi196.stdout:2306 write 0xecc5c00 thru 0xecc66fb (0xafc bytes) 2024-05-08T08:52:56.075 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:52:56.077 INFO:teuthology.orchestra.run.smithi196.stdout:2307 trunc from 0xecc6800 to 0xc5de200 2024-05-08T08:52:56.083 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:56.083 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:56.095 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:52:56.097 INFO:teuthology.orchestra.run.smithi196.stdout:2308 read 0x69afc00 thru 0x69b1ef8 (0x22f9 bytes) 2024-05-08T08:52:56.097 INFO:teuthology.orchestra.run.smithi196.stdout:2312 punch from 0x22dc200 to 0x22e6400, (0xa200 bytes) 2024-05-08T08:52:56.099 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:52:56.099 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:52:56.100 INFO:teuthology.orchestra.run.smithi196.stdout:2313 read 0xb27b600 thru 0xb27c19d (0xb9e bytes) 2024-05-08T08:52:56.100 INFO:teuthology.orchestra.run.smithi196.stdout:2315 punch from 0x5f15800 to 0x5f1b400, (0x5c00 bytes) 2024-05-08T08:52:56.100 INFO:teuthology.orchestra.run.smithi196.stdout:2316 punch from 0x7e2e400 to 0x7e3dc00, (0xf800 bytes) 2024-05-08T08:52:56.101 INFO:teuthology.orchestra.run.smithi196.stdout:2318 write 0x3579800 thru 0x357f563 (0x5d64 bytes) 2024-05-08T08:52:56.101 INFO:teuthology.orchestra.run.smithi196.stdout:2319 read 0x459ee00 thru 0x45a1bc6 (0x2dc7 bytes) 2024-05-08T08:52:56.101 INFO:teuthology.orchestra.run.smithi196.stdout:2320 write 0x38c7800 thru 0x38c8113 (0x914 bytes) 2024-05-08T08:52:56.109 INFO:teuthology.orchestra.run.smithi196.stdout:2321 trunc from 0xc5de200 to 0x4eb1800 2024-05-08T08:52:56.117 INFO:teuthology.orchestra.run.smithi059.stderr:nodeep-scrub is unset 2024-05-08T08:52:56.127 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:52:56.127 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:52:56.155 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:56.158+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:56.217 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:56.217 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:56.226 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:52:56.270 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:52:56.273+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:52:56.335 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:56.336 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:56.344 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:52:56.385 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:52:56.388+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:52:56.459 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:56.461+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:56.493 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:56.493 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:56.502 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:52:56.560 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:52:56.563+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:52:56.611 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:56.612 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:56.620 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:52:56.662 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:52:56.665+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:52:56.738 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:56.738 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:56.748 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:52:56.791 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:56.793+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:56.853 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:56.853 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:56.862 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:52:56.863 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:52:56.866+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:52:56.964 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:52:56.967+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:52:57.007 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:57.007 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:57.015 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:52:57.066 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:52:57.069+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:52:57.122 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:57.122 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:57.133 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:52:57.153 INFO:teuthology.orchestra.run.smithi196.stdout:2324 trunc from 0x4eb1800 to 0x4ffd400 2024-05-08T08:52:57.164 INFO:teuthology.orchestra.run.smithi196.stdout:2325 punch from 0x119f000 to 0x11ace00, (0xde00 bytes) 2024-05-08T08:52:57.171 INFO:teuthology.orchestra.run.smithi196.stdout:2326 write 0x37d8400 thru 0x37e5756 (0xd357 bytes) 2024-05-08T08:52:57.177 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:57.179+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:57.216 INFO:teuthology.orchestra.run.smithi196.stdout:2327 write 0x6a59000 thru 0x6a67f76 (0xef77 bytes) 2024-05-08T08:52:57.238 INFO:teuthology.orchestra.run.smithi196.stdout:2328 write 0xa57a400 thru 0xa586e40 (0xca41 bytes) 2024-05-08T08:52:57.239 INFO:teuthology.orchestra.run.smithi196.stdout:2329 read 0xf7e00 thru 0xfcaea (0x4ceb bytes) 2024-05-08T08:52:57.251 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:57.251 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:57.255 INFO:teuthology.orchestra.run.smithi196.stdout:2330 trunc from 0xa587000 to 0x996600 2024-05-08T08:52:57.262 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:52:57.303 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:57.306+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:57.333 INFO:teuthology.orchestra.run.smithi196.stdout:2331 read 0x4b5000 thru 0x4b8329 (0x332a bytes) 2024-05-08T08:52:57.345 INFO:teuthology.orchestra.run.smithi196.stdout:2336 write 0x21ede00 thru 0x21f23ed (0x45ee bytes) 2024-05-08T08:52:57.358 INFO:teuthology.orchestra.run.smithi196.stdout:2338 trunc from 0x21f2400 to 0xbbf6c00 2024-05-08T08:52:57.368 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:57.370+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:57.384 INFO:teuthology.orchestra.run.smithi196.stdout:2339 write 0xc261a00 thru 0xc2656bc (0x3cbd bytes) 2024-05-08T08:52:57.385 INFO:teuthology.orchestra.run.smithi196.stdout:2340 read 0x1c6ee00 thru 0x1c749e0 (0x5be1 bytes) 2024-05-08T08:52:57.402 INFO:teuthology.orchestra.run.smithi196.stdout:2341 write 0xcc90e00 thru 0xcc9728f (0x6490 bytes) 2024-05-08T08:52:57.407 INFO:teuthology.orchestra.run.smithi196.stdout:2342 punch from 0x1a8c200 to 0x1a8e200, (0x2000 bytes) 2024-05-08T08:52:57.408 INFO:teuthology.orchestra.run.smithi196.stdout:2344 read 0x8b96c00 thru 0x8ba0df5 (0xa1f6 bytes) 2024-05-08T08:52:57.408 INFO:teuthology.orchestra.run.smithi196.stdout:2347 read 0xb76d200 thru 0xb77067f (0x3480 bytes) 2024-05-08T08:52:57.408 INFO:teuthology.orchestra.run.smithi196.stdout:2348 write 0x65b2a00 thru 0x65c1712 (0xed13 bytes) 2024-05-08T08:52:57.409 INFO:teuthology.orchestra.run.smithi196.stdout:2349 trunc from 0xcc97400 to 0xda81200 2024-05-08T08:52:57.417 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:57.418 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:57.427 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:52:57.428 INFO:teuthology.orchestra.run.smithi196.stdout:2351 read 0x2fa9a00 thru 0x2fad22f (0x3830 bytes) 2024-05-08T08:52:57.434 INFO:teuthology.orchestra.run.smithi196.stdout:2352 write 0xbd6800 thru 0xbdd701 (0x6f02 bytes) 2024-05-08T08:52:57.435 INFO:teuthology.orchestra.run.smithi196.stdout:2353 trunc from 0xda81200 to 0xd14c600 2024-05-08T08:52:57.446 INFO:teuthology.orchestra.run.smithi196.stdout:2354 punch from 0x23fee00 to 0x2400800, (0x1a00 bytes) 2024-05-08T08:52:57.449 INFO:teuthology.orchestra.run.smithi196.stdout:2359 punch from 0x3288000 to 0x3294200, (0xc200 bytes) 2024-05-08T08:52:57.449 INFO:teuthology.orchestra.run.smithi196.stdout:2360 write 0x8c2ce00 thru 0x8c394b9 (0xc6ba bytes) 2024-05-08T08:52:57.449 INFO:teuthology.orchestra.run.smithi196.stdout:2361 read 0x9630800 thru 0x963c25f (0xba60 bytes) 2024-05-08T08:52:57.449 INFO:teuthology.orchestra.run.smithi196.stdout:2362 read 0x3f0ec00 thru 0x3f192a0 (0xa6a1 bytes) 2024-05-08T08:52:57.449 INFO:teuthology.orchestra.run.smithi196.stdout:2365 write 0x745dc00 thru 0x746c5f8 (0xe9f9 bytes) 2024-05-08T08:52:57.449 INFO:teuthology.orchestra.run.smithi196.stdout:2366 punch from 0x655a600 to 0x6561a00, (0x7400 bytes) 2024-05-08T08:52:57.469 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:52:57.473+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:52:57.477 INFO:teuthology.orchestra.run.smithi196.stdout:2367 write 0xd16ca00 thru 0xd17c247 (0xf848 bytes) 2024-05-08T08:52:57.478 INFO:teuthology.orchestra.run.smithi196.stdout:2369 read 0x740d600 thru 0x74136df (0x60e0 bytes) 2024-05-08T08:52:57.478 INFO:teuthology.orchestra.run.smithi196.stdout:2370 read 0x23a0200 thru 0x23ae363 (0xe164 bytes) 2024-05-08T08:52:57.478 INFO:teuthology.orchestra.run.smithi196.stdout:2371 trunc from 0xd17c400 to 0xd14f600 2024-05-08T08:52:57.497 INFO:teuthology.orchestra.run.smithi196.stdout:2372 write 0xc9d800 thru 0xc9fe37 (0x2638 bytes) 2024-05-08T08:52:57.515 INFO:teuthology.orchestra.run.smithi196.stdout:2373 write 0xee11000 thru 0xee20298 (0xf299 bytes) 2024-05-08T08:52:57.517 INFO:teuthology.orchestra.run.smithi196.stdout:2374 write 0xeaf3a00 thru 0xeb0237d (0xe97e bytes) 2024-05-08T08:52:57.517 INFO:teuthology.orchestra.run.smithi196.stdout:2375 write 0xa02e400 thru 0xa0393f5 (0xaff6 bytes) 2024-05-08T08:52:57.530 INFO:teuthology.orchestra.run.smithi196.stdout:2378 trunc from 0xee20400 to 0x4b36e00 2024-05-08T08:52:57.570 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:57.572+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:57.584 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:57.584 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:57.591 INFO:teuthology.orchestra.run.smithi196.stdout:2379 write 0x6bb7a00 thru 0x6bbe9fe (0x6fff bytes) 2024-05-08T08:52:57.592 INFO:teuthology.orchestra.run.smithi196.stdout:2381 read 0xd04000 thru 0xd085e9 (0x45ea bytes) 2024-05-08T08:52:57.592 INFO:teuthology.orchestra.run.smithi196.stdout:2383 punch from 0x45ba200 to 0x45c7800, (0xd600 bytes) 2024-05-08T08:52:57.593 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:52:57.594 INFO:teuthology.orchestra.run.smithi196.stdout:2384 write 0x36ace00 thru 0x36bbc0c (0xee0d bytes) 2024-05-08T08:52:57.594 INFO:teuthology.orchestra.run.smithi196.stdout:2386 write 0x11fe800 thru 0x12053bd (0x6bbe bytes) 2024-05-08T08:52:57.616 INFO:teuthology.orchestra.run.smithi196.stdout:2387 write 0xb849200 thru 0xb856814 (0xd615 bytes) 2024-05-08T08:52:57.617 INFO:teuthology.orchestra.run.smithi196.stdout:2388 read 0x3833400 thru 0x38420cb (0xeccc bytes) 2024-05-08T08:52:57.617 INFO:teuthology.orchestra.run.smithi196.stdout:2390 punch from 0x262cc00 to 0x262f800, (0x2c00 bytes) 2024-05-08T08:52:57.618 INFO:teuthology.orchestra.run.smithi196.stdout:2391 read 0x3613400 thru 0x361f074 (0xbc75 bytes) 2024-05-08T08:52:57.619 INFO:teuthology.orchestra.run.smithi196.stdout:2393 write 0x5f5f800 thru 0x5f69047 (0x9848 bytes) 2024-05-08T08:52:57.627 INFO:teuthology.orchestra.run.smithi196.stdout:2394 trunc from 0xb856a00 to 0x4f39c00 2024-05-08T08:52:57.653 INFO:teuthology.orchestra.run.smithi196.stdout:2395 punch from 0x3a96000 to 0x3a9ce00, (0x6e00 bytes) 2024-05-08T08:52:57.655 INFO:teuthology.orchestra.run.smithi196.stdout:2396 punch from 0x4c21c00 to 0x4c2b400, (0x9800 bytes) 2024-05-08T08:52:57.655 INFO:teuthology.orchestra.run.smithi196.stdout:2397 read 0x413e800 thru 0x41470d0 (0x88d1 bytes) 2024-05-08T08:52:57.656 INFO:teuthology.orchestra.run.smithi196.stdout:2399 punch from 0x34aca00 to 0x34ad000, (0x600 bytes) 2024-05-08T08:52:57.656 INFO:teuthology.orchestra.run.smithi196.stdout:2402 read 0x1dd9e00 thru 0x1de6081 (0xc282 bytes) 2024-05-08T08:52:57.662 INFO:teuthology.orchestra.run.smithi196.stdout:2404 trunc from 0x4f39c00 to 0x94e8800 2024-05-08T08:52:57.671 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:52:57.674+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:52:57.678 INFO:teuthology.orchestra.run.smithi196.stdout:2405 punch from 0x4997c00 to 0x499f600, (0x7a00 bytes) 2024-05-08T08:52:57.678 INFO:teuthology.orchestra.run.smithi196.stdout:2407 write 0x8e8ca00 thru 0x8e91ae4 (0x50e5 bytes) 2024-05-08T08:52:57.678 INFO:teuthology.orchestra.run.smithi196.stdout:2410 punch from 0x6ade400 to 0x6ae8a00, (0xa600 bytes) 2024-05-08T08:52:57.679 INFO:teuthology.orchestra.run.smithi196.stdout:2411 trunc from 0x94e8800 to 0xab39c00 2024-05-08T08:52:57.700 INFO:teuthology.orchestra.run.smithi196.stdout:2413 read 0x97c8000 thru 0x97d5ec7 (0xdec8 bytes) 2024-05-08T08:52:57.701 INFO:teuthology.orchestra.run.smithi196.stdout:2417 read 0x6c1600 thru 0x6c7d59 (0x675a bytes) 2024-05-08T08:52:57.702 INFO:teuthology.orchestra.run.smithi196.stdout:2418 trunc from 0xab39c00 to 0x9a2e600 2024-05-08T08:52:57.713 INFO:teuthology.orchestra.run.smithi196.stdout:2424 write 0x9900800 thru 0x9904617 (0x3e18 bytes) 2024-05-08T08:52:57.713 INFO:teuthology.orchestra.run.smithi196.stdout:2425 read 0x55f0c00 thru 0x55f4ba0 (0x3fa1 bytes) 2024-05-08T08:52:57.713 INFO:teuthology.orchestra.run.smithi196.stdout:2427 write 0x2aa6200 thru 0x2aad36f (0x7170 bytes) 2024-05-08T08:52:57.735 INFO:teuthology.orchestra.run.smithi196.stdout:2429 write 0xafca200 thru 0xafd4b3a (0xa93b bytes) 2024-05-08T08:52:57.735 INFO:teuthology.orchestra.run.smithi196.stdout:2430 punch from 0x185a00 to 0x190600, (0xac00 bytes) 2024-05-08T08:52:57.737 INFO:teuthology.orchestra.run.smithi196.stdout:2437 read 0x60efe00 thru 0x60f96ee (0x98ef bytes) 2024-05-08T08:52:57.737 INFO:teuthology.orchestra.run.smithi196.stdout:2438 clone 14 order 22 su 1048576 sc 3 2024-05-08T08:52:57.748 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:57.748 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:57.758 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:52:57.802 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:57.804+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:57.863 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:57.863 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:57.871 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:52:57.913 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:57.915+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:57.973 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:52:57.976+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:52:57.981 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:57.981 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:57.990 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:52:58.074 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:52:58.076+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:52:58.102 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 5] out_osds: [4] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-05-08T08:52:58.102 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T08:52:58.102 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-05-08T08:52:58.103 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-05-08T08:52:58.134 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-05-08T08:52:58.139 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T08:52:58.150 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:52:58.150 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:52:58.161 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:52:58.176 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:52:58.178+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:52:58.244 INFO:teuthology.orchestra.run.smithi196.stdout:truncating image image_client.0-clone13 from 0xafd4c00 (overlap 0x996600) to 0x3d5c00 2024-05-08T08:52:58.277 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:52:58.280+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:52:58.316 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:52:58.317 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:52:58.337 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:52:58.338 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:52:58.356 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:52:58.356 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:52:58.360 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:52:58.361 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:52:58.361 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:52:58.361 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:52:58.361 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.361 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4621.0:9089 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 23339520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:52:58.361 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:52:58.361 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 25.869199169000002, 2024-05-08T08:52:58.362 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.13814907200000001, 2024-05-08T08:52:58.362 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:58.362 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:58.362 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:52:58.362 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:58.362 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4621", 2024-05-08T08:52:58.362 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:58.362 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9089 2024-05-08T08:52:58.362 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.362 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:58.362 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.363 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:58.363 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:52:58.363 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:58.363 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.363 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.363 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:58.363 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:52:58.363 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:58.363 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.364 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.364 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:58.365 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493005+0000", 2024-05-08T08:52:58.365 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1700000000000001e-06 2024-05-08T08:52:58.365 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.365 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.365 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:58.365 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.513963+0000", 2024-05-08T08:52:58.365 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.020957581999999999 2024-05-08T08:52:58.365 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.365 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.366 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:58.366 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.513995+0000", 2024-05-08T08:52:58.366 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1993999999999999e-05 2024-05-08T08:52:58.366 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.366 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.366 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:58.366 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514007+0000", 2024-05-08T08:52:58.366 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1581e-05 2024-05-08T08:52:58.366 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.366 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.366 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:58.366 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514122+0000", 2024-05-08T08:52:58.366 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000115282 2024-05-08T08:52:58.367 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.367 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.367 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op must wait for map", 2024-05-08T08:52:58.367 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514128+0000", 2024-05-08T08:52:58.367 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9950000000000002e-06 2024-05-08T08:52:58.367 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.367 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.367 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:58.367 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.515800+0000", 2024-05-08T08:52:58.367 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001671886 2024-05-08T08:52:58.367 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.367 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.367 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:58.367 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.515893+0000", 2024-05-08T08:52:58.367 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3490000000000001e-05 2024-05-08T08:52:58.367 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.368 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.368 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 3", 2024-05-08T08:52:58.368 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.517647+0000", 2024-05-08T08:52:58.368 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0017535630000000001 2024-05-08T08:52:58.368 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.368 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.368 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:52:58.368 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.627578+0000", 2024-05-08T08:52:58.368 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.109931557 2024-05-08T08:52:58.368 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.368 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.368 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:52:58.369 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631055+0000", 2024-05-08T08:52:58.369 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0034761420000000002 2024-05-08T08:52:58.369 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.369 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.369 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:52:58.369 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631096+0000", 2024-05-08T08:52:58.369 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1059000000000001e-05 2024-05-08T08:52:58.369 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.369 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.369 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:58.370 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631151+0000", 2024-05-08T08:52:58.370 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5770999999999998e-05 2024-05-08T08:52:58.370 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.370 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:58.370 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.370 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.370 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.370 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1065 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 10944512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:58.370 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485411+0000", 2024-05-08T08:52:58.370 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 21.876790483000001, 2024-05-08T08:52:58.370 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070281386000000001, 2024-05-08T08:52:58.370 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:58.370 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:58.370 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:58.371 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:58.371 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:58.371 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:58.371 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1065 2024-05-08T08:52:58.371 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.371 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:58.371 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.371 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:58.371 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485411+0000", 2024-05-08T08:52:58.371 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:58.371 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.372 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.372 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:58.372 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485411+0000", 2024-05-08T08:52:58.372 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:58.372 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.372 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.373 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:58.373 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485411+0000", 2024-05-08T08:52:58.373 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5600000000000001e-07 2024-05-08T08:52:58.373 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.373 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.373 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:58.374 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485414+0000", 2024-05-08T08:52:58.374 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2299999999999998e-06 2024-05-08T08:52:58.374 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.374 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.374 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:58.374 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485420+0000", 2024-05-08T08:52:58.374 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2179999999999996e-06 2024-05-08T08:52:58.374 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.375 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.375 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:58.375 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485425+0000", 2024-05-08T08:52:58.375 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9830000000000004e-06 2024-05-08T08:52:58.375 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.375 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.375 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:58.375 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555468+0000", 2024-05-08T08:52:58.375 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070043541000000001 2024-05-08T08:52:58.375 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.375 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.375 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:58.376 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555566+0000", 2024-05-08T08:52:58.376 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8082e-05 2024-05-08T08:52:58.376 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.376 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.376 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:58.376 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555692+0000", 2024-05-08T08:52:58.376 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000125976 2024-05-08T08:52:58.376 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.376 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:58.376 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.376 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.376 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.376 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1067 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 10985472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:58.376 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485431+0000", 2024-05-08T08:52:58.376 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 21.876770348000001, 2024-05-08T08:52:58.377 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070412460999999996, 2024-05-08T08:52:58.377 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:58.377 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:58.377 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:58.377 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:58.377 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:58.377 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:58.377 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1067 2024-05-08T08:52:58.377 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.377 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:58.377 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.377 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:58.377 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485431+0000", 2024-05-08T08:52:58.377 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:58.377 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.377 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.378 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:58.378 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485431+0000", 2024-05-08T08:52:58.378 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:58.378 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.378 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.378 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:58.378 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485432+0000", 2024-05-08T08:52:58.378 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8299999999999998e-07 2024-05-08T08:52:58.378 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.378 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.378 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:58.378 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485434+0000", 2024-05-08T08:52:58.378 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0849999999999999e-06 2024-05-08T08:52:58.379 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.379 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.379 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:58.379 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485440+0000", 2024-05-08T08:52:58.379 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0020000000000003e-06 2024-05-08T08:52:58.379 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.379 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.379 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:58.379 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485445+0000", 2024-05-08T08:52:58.379 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5180000000000004e-06 2024-05-08T08:52:58.379 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.379 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.379 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:58.379 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555731+0000", 2024-05-08T08:52:58.380 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070286214 2024-05-08T08:52:58.380 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.380 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.380 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:58.380 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555787+0000", 2024-05-08T08:52:58.380 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5992999999999997e-05 2024-05-08T08:52:58.380 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.380 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.380 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:58.380 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555844+0000", 2024-05-08T08:52:58.380 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6266e-05 2024-05-08T08:52:58.380 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.380 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:58.380 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.381 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.381 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.381 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1071 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11026432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:58.381 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:52:58.381 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 21.876749677999999, 2024-05-08T08:52:58.381 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070580037999999998, 2024-05-08T08:52:58.381 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:58.381 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:58.381 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:58.381 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:58.381 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:58.381 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:58.381 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1071 2024-05-08T08:52:58.381 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.382 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:58.382 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.382 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:58.382 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:52:58.382 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:58.382 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.382 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.382 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:58.382 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:52:58.382 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:58.382 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.382 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.382 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:58.383 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:52:58.383 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2800000000000002e-07 2024-05-08T08:52:58.383 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.383 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.383 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:58.383 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485454+0000", 2024-05-08T08:52:58.383 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2290000000000002e-06 2024-05-08T08:52:58.383 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.383 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.383 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:58.383 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485461+0000", 2024-05-08T08:52:58.383 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9920000000000001e-06 2024-05-08T08:52:58.384 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.384 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.384 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:58.384 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485467+0000", 2024-05-08T08:52:58.384 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3979999999999998e-06 2024-05-08T08:52:58.384 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.384 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.384 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:58.384 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555880+0000", 2024-05-08T08:52:58.385 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070413165 2024-05-08T08:52:58.385 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.385 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.386 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:58.386 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555926+0000", 2024-05-08T08:52:58.386 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6217000000000002e-05 2024-05-08T08:52:58.386 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.386 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.387 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:58.387 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556032+0000", 2024-05-08T08:52:58.387 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000105609 2024-05-08T08:52:58.387 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.387 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:58.388 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.388 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.388 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.388 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1073 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11067392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:58.388 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:52:58.388 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 21.876728456999999, 2024-05-08T08:52:58.389 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070821404000000004, 2024-05-08T08:52:58.389 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:58.389 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:58.389 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:58.389 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:58.390 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:58.390 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:58.390 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1073 2024-05-08T08:52:58.390 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.390 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:58.390 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.391 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:58.391 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:52:58.391 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:58.391 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.391 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.391 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:58.392 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:52:58.392 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:58.392 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.392 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.392 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:58.393 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:52:58.393 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.08e-07 2024-05-08T08:52:58.393 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.393 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.393 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:58.393 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485476+0000", 2024-05-08T08:52:58.393 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3929999999999998e-06 2024-05-08T08:52:58.394 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.394 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.394 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:58.394 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485485+0000", 2024-05-08T08:52:58.394 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1880000000000008e-06 2024-05-08T08:52:58.394 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.395 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.395 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:58.395 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485494+0000", 2024-05-08T08:52:58.395 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3109999999999995e-06 2024-05-08T08:52:58.395 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.395 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.396 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:58.396 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556082+0000", 2024-05-08T08:52:58.396 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070587426999999994 2024-05-08T08:52:58.396 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.396 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.396 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:58.396 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556177+0000", 2024-05-08T08:52:58.397 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5051999999999997e-05 2024-05-08T08:52:58.397 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.397 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.397 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:58.397 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556294+0000", 2024-05-08T08:52:58.397 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000117625 2024-05-08T08:52:58.398 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.398 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:58.398 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.398 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.398 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.398 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1075 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11108352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:58.398 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:52:58.399 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 21.876697880000002, 2024-05-08T08:52:58.399 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071045026999999997, 2024-05-08T08:52:58.399 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:58.399 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:58.399 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:58.399 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:58.400 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:58.400 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:58.400 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1075 2024-05-08T08:52:58.400 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.400 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:58.400 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.400 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:58.401 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:52:58.401 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:58.401 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.401 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.401 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:58.401 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:52:58.401 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:58.401 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.402 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.402 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:58.402 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:52:58.402 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.32e-07 2024-05-08T08:52:58.402 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.402 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.402 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:58.402 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485518+0000", 2024-05-08T08:52:58.403 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3769e-05 2024-05-08T08:52:58.403 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.403 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.403 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:58.403 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485527+0000", 2024-05-08T08:52:58.403 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6779999999999995e-06 2024-05-08T08:52:58.403 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.404 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.404 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:58.404 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485532+0000", 2024-05-08T08:52:58.404 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9830000000000004e-06 2024-05-08T08:52:58.404 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.404 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.405 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:58.405 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556340+0000", 2024-05-08T08:52:58.405 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070807743000000006 2024-05-08T08:52:58.405 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.405 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.405 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:58.405 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556435+0000", 2024-05-08T08:52:58.406 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4732e-05 2024-05-08T08:52:58.406 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.406 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.406 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:58.406 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556549+0000", 2024-05-08T08:52:58.407 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011369 2024-05-08T08:52:58.407 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.407 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:58.407 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.407 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.407 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.408 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1078 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11149312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:58.408 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:52:58.408 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 21.876657681000001, 2024-05-08T08:52:58.408 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071167276000000002, 2024-05-08T08:52:58.408 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:58.408 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:58.409 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:58.409 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:58.409 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:58.409 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:58.409 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1078 2024-05-08T08:52:58.409 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.409 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:58.409 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.410 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:58.410 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:52:58.410 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:58.410 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.410 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.410 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:58.410 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:52:58.410 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:58.411 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.411 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.411 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:58.411 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:52:58.411 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5600000000000001e-07 2024-05-08T08:52:58.411 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.411 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.411 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:58.411 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485547+0000", 2024-05-08T08:52:58.412 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3590000000000002e-06 2024-05-08T08:52:58.412 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.412 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.412 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:58.412 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485553+0000", 2024-05-08T08:52:58.413 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2770000000000002e-06 2024-05-08T08:52:58.413 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.413 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.413 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:58.413 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485558+0000", 2024-05-08T08:52:58.413 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.95e-06 2024-05-08T08:52:58.413 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.414 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.414 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:58.414 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556582+0000", 2024-05-08T08:52:58.414 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071024444000000006 2024-05-08T08:52:58.414 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.415 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.415 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:58.415 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556637+0000", 2024-05-08T08:52:58.415 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4672000000000001e-05 2024-05-08T08:52:58.415 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.415 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.415 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:58.416 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556711+0000", 2024-05-08T08:52:58.416 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4117999999999997e-05 2024-05-08T08:52:58.416 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.416 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:58.416 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.416 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.417 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.417 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1080 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11190272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:58.417 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:52:58.417 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 21.876636948000002, 2024-05-08T08:52:58.417 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071350721000000006, 2024-05-08T08:52:58.417 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:58.417 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:58.417 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:58.417 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:58.417 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:58.418 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:58.418 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1080 2024-05-08T08:52:58.418 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.418 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:58.418 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.418 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:58.418 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:52:58.418 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:58.418 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.418 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.418 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:58.418 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:52:58.418 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:58.419 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.419 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.419 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:58.419 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:52:58.419 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.03e-07 2024-05-08T08:52:58.419 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.419 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.419 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:58.419 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485567+0000", 2024-05-08T08:52:58.419 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1090000000000001e-06 2024-05-08T08:52:58.419 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.419 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.419 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:58.419 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485573+0000", 2024-05-08T08:52:58.420 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3049999999999998e-06 2024-05-08T08:52:58.420 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.420 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.420 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:58.420 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485578+0000", 2024-05-08T08:52:58.420 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0130000000000001e-06 2024-05-08T08:52:58.420 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.420 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.420 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:58.420 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556743+0000", 2024-05-08T08:52:58.420 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071164449000000005 2024-05-08T08:52:58.420 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.420 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.420 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:58.421 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556797+0000", 2024-05-08T08:52:58.421 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4589999999999997e-05 2024-05-08T08:52:58.421 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.421 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.421 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:58.421 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556915+0000", 2024-05-08T08:52:58.421 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000117852 2024-05-08T08:52:58.421 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.421 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:58.421 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.421 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.421 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.421 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1082 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11231232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:58.421 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:52:58.421 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 21.876616705, 2024-05-08T08:52:58.421 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071591396000000002, 2024-05-08T08:52:58.422 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:58.422 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:58.422 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:58.422 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:58.422 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:58.422 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:58.422 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1082 2024-05-08T08:52:58.422 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.422 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:58.422 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.422 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:58.422 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:52:58.422 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:58.422 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.423 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.423 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:58.423 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:52:58.423 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:58.423 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.423 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.423 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:58.423 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:52:58.423 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1800000000000001e-07 2024-05-08T08:52:58.423 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.423 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.427 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:58.428 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485587+0000", 2024-05-08T08:52:58.428 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2589999999999999e-06 2024-05-08T08:52:58.428 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.428 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.428 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:58.428 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485594+0000", 2024-05-08T08:52:58.428 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0789999999999997e-06 2024-05-08T08:52:58.428 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.428 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.428 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:58.428 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485598+0000", 2024-05-08T08:52:58.429 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7480000000000001e-06 2024-05-08T08:52:58.429 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.429 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.429 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:58.429 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556971+0000", 2024-05-08T08:52:58.429 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071372774999999999 2024-05-08T08:52:58.429 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.429 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.429 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:58.429 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557069+0000", 2024-05-08T08:52:58.429 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.7732999999999994e-05 2024-05-08T08:52:58.429 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.429 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.429 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:58.430 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557176+0000", 2024-05-08T08:52:58.430 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000107384 2024-05-08T08:52:58.430 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.430 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:58.430 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.430 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.430 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.430 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1084 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11272192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:58.430 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:52:58.430 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 21.876578066, 2024-05-08T08:52:58.431 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071808977999999996, 2024-05-08T08:52:58.431 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:58.431 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:58.431 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:58.431 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:58.431 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:58.431 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:58.431 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1084 2024-05-08T08:52:58.432 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.432 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:58.432 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.432 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:58.432 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:52:58.432 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:58.432 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.432 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.432 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:58.433 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:52:58.433 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:58.433 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.433 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.433 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:58.433 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485624+0000", 2024-05-08T08:52:58.433 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:52:58.433 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.433 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.433 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:58.433 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485626+0000", 2024-05-08T08:52:58.433 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9460000000000001e-06 2024-05-08T08:52:58.433 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.433 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.434 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:58.434 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485633+0000", 2024-05-08T08:52:58.434 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1149999999999996e-06 2024-05-08T08:52:58.434 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.434 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.434 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:58.434 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485638+0000", 2024-05-08T08:52:58.434 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5670000000000002e-06 2024-05-08T08:52:58.434 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.434 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.434 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:58.434 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557226+0000", 2024-05-08T08:52:58.434 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071587267999999996 2024-05-08T08:52:58.434 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.435 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.435 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:58.435 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557321+0000", 2024-05-08T08:52:58.435 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5785000000000005e-05 2024-05-08T08:52:58.435 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.435 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.435 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:58.435 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557432+0000", 2024-05-08T08:52:58.435 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011092800000000001 2024-05-08T08:52:58.435 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.435 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:58.435 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.435 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.435 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.436 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1086 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11313152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:58.436 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:52:58.436 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 21.876554738999999, 2024-05-08T08:52:58.436 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071933132999999996, 2024-05-08T08:52:58.436 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:58.436 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:58.436 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:58.436 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:58.436 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:58.436 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:58.436 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1086 2024-05-08T08:52:58.436 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.436 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:58.436 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.436 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:58.437 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:52:58.437 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:58.437 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.437 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.437 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:58.437 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:52:58.437 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:58.437 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.437 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.437 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:58.437 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486328+0000", 2024-05-08T08:52:58.437 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00068107900000000002 2024-05-08T08:52:58.437 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.437 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.438 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:58.438 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486331+0000", 2024-05-08T08:52:58.438 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.585e-06 2024-05-08T08:52:58.438 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.438 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.438 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:58.438 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486343+0000", 2024-05-08T08:52:58.438 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1856e-05 2024-05-08T08:52:58.438 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.438 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.438 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:58.438 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486350+0000", 2024-05-08T08:52:58.438 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8600000000000004e-06 2024-05-08T08:52:58.438 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.439 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.439 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:58.439 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557470+0000", 2024-05-08T08:52:58.439 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071120326999999997 2024-05-08T08:52:58.439 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.439 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.439 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:58.439 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557514+0000", 2024-05-08T08:52:58.439 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3909000000000003e-05 2024-05-08T08:52:58.439 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.439 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.439 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:58.439 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557580+0000", 2024-05-08T08:52:58.439 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5517000000000003e-05 2024-05-08T08:52:58.439 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.439 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:58.440 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.440 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.440 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.440 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1089 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11354112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:58.440 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:52:58.440 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 21.875841866999998, 2024-05-08T08:52:58.440 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071457381, 2024-05-08T08:52:58.440 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:58.440 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:58.440 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:58.440 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:58.440 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:58.440 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:58.440 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1089 2024-05-08T08:52:58.440 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.440 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:58.441 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.441 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:58.441 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:52:58.441 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:58.441 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.441 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.441 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:58.441 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:52:58.441 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:58.441 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.441 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.441 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:58.441 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486851+0000", 2024-05-08T08:52:58.441 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00049102 2024-05-08T08:52:58.441 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.441 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.442 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:58.442 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486854+0000", 2024-05-08T08:52:58.442 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6859999999999999e-06 2024-05-08T08:52:58.442 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.442 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.442 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:58.442 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486864+0000", 2024-05-08T08:52:58.442 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6080000000000002e-06 2024-05-08T08:52:58.442 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.442 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.442 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:58.442 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486870+0000", 2024-05-08T08:52:58.442 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.477e-06 2024-05-08T08:52:58.442 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.443 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.443 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:58.443 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557612+0000", 2024-05-08T08:52:58.443 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070741557999999996 2024-05-08T08:52:58.443 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.443 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.443 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:58.443 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557695+0000", 2024-05-08T08:52:58.443 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3300000000000005e-05 2024-05-08T08:52:58.443 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.443 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.443 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:58.443 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557817+0000", 2024-05-08T08:52:58.443 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012173200000000001 2024-05-08T08:52:58.443 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.444 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:58.444 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.444 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.444 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.444 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1091 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11395072~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:58.444 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:52:58.444 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 21.875321071999998, 2024-05-08T08:52:58.444 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071184737999999997, 2024-05-08T08:52:58.444 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:58.444 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:58.444 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:58.444 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:58.444 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:58.444 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:58.445 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1091 2024-05-08T08:52:58.445 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.445 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:58.445 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.445 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:58.445 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:52:58.445 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:58.445 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.445 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.445 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:58.445 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:52:58.445 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:58.445 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.445 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.445 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:58.446 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486881+0000", 2024-05-08T08:52:58.446 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0699999999999997e-07 2024-05-08T08:52:58.446 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.446 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.446 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:58.446 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486883+0000", 2024-05-08T08:52:58.446 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.402e-06 2024-05-08T08:52:58.446 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.446 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.446 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:58.446 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486890+0000", 2024-05-08T08:52:58.446 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0480000000000001e-06 2024-05-08T08:52:58.446 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.446 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.446 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:58.447 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486896+0000", 2024-05-08T08:52:58.447 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7640000000000002e-06 2024-05-08T08:52:58.447 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.447 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.447 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:58.447 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557860+0000", 2024-05-08T08:52:58.447 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070963791999999998 2024-05-08T08:52:58.447 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.447 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.447 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:58.447 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557952+0000", 2024-05-08T08:52:58.447 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2082000000000004e-05 2024-05-08T08:52:58.447 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.447 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.447 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:58.447 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558065+0000", 2024-05-08T08:52:58.447 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000113143 2024-05-08T08:52:58.448 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.448 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:58.448 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.448 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.448 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.448 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1094 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11436032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:58.448 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:52:58.448 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 21.875297904, 2024-05-08T08:52:58.448 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071400507000000002, 2024-05-08T08:52:58.448 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:58.448 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:58.448 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:58.448 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:58.448 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:58.449 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:58.449 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1094 2024-05-08T08:52:58.449 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.449 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:58.449 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.449 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:58.449 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:52:58.449 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:58.449 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.449 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.449 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:58.449 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:52:58.449 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:58.449 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.449 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.450 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:58.450 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487940+0000", 2024-05-08T08:52:58.450 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001036868 2024-05-08T08:52:58.450 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.450 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.450 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:58.450 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487944+0000", 2024-05-08T08:52:58.450 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7069999999999998e-06 2024-05-08T08:52:58.450 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.450 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.450 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:58.450 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487951+0000", 2024-05-08T08:52:58.450 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2649999999999999e-06 2024-05-08T08:52:58.450 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.451 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.451 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:58.451 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487961+0000", 2024-05-08T08:52:58.451 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4110000000000002e-06 2024-05-08T08:52:58.451 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.451 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.451 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:58.451 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558117+0000", 2024-05-08T08:52:58.451 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070156662999999994 2024-05-08T08:52:58.451 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.451 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.451 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:58.451 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558214+0000", 2024-05-08T08:52:58.451 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6444e-05 2024-05-08T08:52:58.451 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.452 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.452 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:58.452 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558304+0000", 2024-05-08T08:52:58.452 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0148999999999994e-05 2024-05-08T08:52:58.452 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.452 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:58.452 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.452 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.452 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.452 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1096 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11476992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:58.452 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:52:58.452 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 21.874231373000001, 2024-05-08T08:52:58.452 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070480074000000004, 2024-05-08T08:52:58.452 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:58.452 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:58.453 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:58.453 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:58.453 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:58.453 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:58.453 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1096 2024-05-08T08:52:58.453 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.453 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:58.453 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.453 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:58.453 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:52:58.453 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:58.453 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.453 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.453 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:58.453 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:52:58.454 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:58.454 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.454 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.454 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:58.454 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:52:58.454 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3700000000000001e-07 2024-05-08T08:52:58.454 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.454 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.454 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:58.454 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487973+0000", 2024-05-08T08:52:58.454 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0870000000000001e-06 2024-05-08T08:52:58.454 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.454 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.454 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:58.455 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487977+0000", 2024-05-08T08:52:58.455 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0600000000000001e-06 2024-05-08T08:52:58.455 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.455 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.455 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:58.455 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487980+0000", 2024-05-08T08:52:58.455 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1089999999999998e-06 2024-05-08T08:52:58.455 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.455 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.455 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:58.455 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558338+0000", 2024-05-08T08:52:58.455 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070358233000000006 2024-05-08T08:52:58.455 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.455 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.455 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:58.455 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558392+0000", 2024-05-08T08:52:58.456 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4537999999999999e-05 2024-05-08T08:52:58.456 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.456 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.456 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:58.456 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558450+0000", 2024-05-08T08:52:58.456 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7710000000000001e-05 2024-05-08T08:52:58.456 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.456 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:58.456 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.456 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.456 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.456 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1099 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11517952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:58.456 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:52:58.456 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 21.874217497, 2024-05-08T08:52:58.456 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070720506000000002, 2024-05-08T08:52:58.457 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:58.457 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:58.457 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:58.457 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:58.457 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:58.457 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:58.457 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1099 2024-05-08T08:52:58.457 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.457 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:58.457 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.457 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:58.457 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:52:58.457 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:58.457 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.457 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.457 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:58.458 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:52:58.458 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:58.458 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.458 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.458 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:58.458 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:52:58.458 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3999999999999998e-07 2024-05-08T08:52:58.458 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.458 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.458 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:58.458 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487986+0000", 2024-05-08T08:52:58.458 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3519999999999999e-06 2024-05-08T08:52:58.458 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.458 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.458 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:58.459 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487989+0000", 2024-05-08T08:52:58.459 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.23e-06 2024-05-08T08:52:58.459 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.459 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.459 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:58.459 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487991+0000", 2024-05-08T08:52:58.459 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6350000000000002e-06 2024-05-08T08:52:58.459 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.459 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.459 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:58.459 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558489+0000", 2024-05-08T08:52:58.459 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070497924000000003 2024-05-08T08:52:58.459 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.459 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.459 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:58.460 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558583+0000", 2024-05-08T08:52:58.460 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3405000000000004e-05 2024-05-08T08:52:58.460 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.460 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.460 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:58.460 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558704+0000", 2024-05-08T08:52:58.460 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012171999999999999 2024-05-08T08:52:58.460 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.460 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:58.460 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.460 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.460 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.460 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1100 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11558912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:58.460 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:52:58.460 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 21.874204498000001, 2024-05-08T08:52:58.460 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070962266999999996, 2024-05-08T08:52:58.461 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:58.461 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:58.461 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:58.461 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:58.461 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:58.461 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:58.461 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1100 2024-05-08T08:52:58.461 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.461 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:58.461 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.461 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:58.461 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:52:58.461 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:58.461 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.461 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.461 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:58.462 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:52:58.462 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:58.462 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.462 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.462 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:58.462 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488430+0000", 2024-05-08T08:52:58.462 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000432676 2024-05-08T08:52:58.462 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.462 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.462 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:58.462 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488433+0000", 2024-05-08T08:52:58.462 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.45e-06 2024-05-08T08:52:58.462 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.463 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.463 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:58.463 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488442+0000", 2024-05-08T08:52:58.463 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.687e-06 2024-05-08T08:52:58.463 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.463 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.463 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:58.463 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488446+0000", 2024-05-08T08:52:58.463 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4939999999999997e-06 2024-05-08T08:52:58.463 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.463 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.463 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:58.463 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558756+0000", 2024-05-08T08:52:58.463 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070310051999999998 2024-05-08T08:52:58.464 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.464 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.464 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:58.464 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558844+0000", 2024-05-08T08:52:58.464 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7690999999999998e-05 2024-05-08T08:52:58.464 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.464 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.464 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:58.464 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558959+0000", 2024-05-08T08:52:58.464 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000115217 2024-05-08T08:52:58.464 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.464 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:58.464 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.464 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.465 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.465 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1103 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11599872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:58.465 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:52:58.465 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 21.873746823000001, 2024-05-08T08:52:58.465 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070777100999999995, 2024-05-08T08:52:58.465 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:58.465 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:58.465 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:58.465 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:58.465 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:58.465 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:58.465 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1103 2024-05-08T08:52:58.465 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.465 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:58.466 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.466 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:58.466 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:52:58.466 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:58.466 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.466 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.466 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:58.466 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:52:58.466 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:58.466 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.466 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.466 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:58.466 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488959+0000", 2024-05-08T08:52:58.466 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00050457200000000001 2024-05-08T08:52:58.466 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.467 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.467 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:58.467 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488969+0000", 2024-05-08T08:52:58.467 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.7159999999999998e-06 2024-05-08T08:52:58.467 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.467 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.467 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:58.467 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488981+0000", 2024-05-08T08:52:58.467 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2150000000000001e-05 2024-05-08T08:52:58.467 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.467 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.467 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:58.467 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488988+0000", 2024-05-08T08:52:58.467 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8759999999999999e-06 2024-05-08T08:52:58.468 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.468 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.468 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:58.468 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558992+0000", 2024-05-08T08:52:58.468 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070004358000000003 2024-05-08T08:52:58.468 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.468 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.468 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:58.468 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559093+0000", 2024-05-08T08:52:58.468 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000100299 2024-05-08T08:52:58.468 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.468 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.468 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:58.468 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559232+0000", 2024-05-08T08:52:58.468 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013913 2024-05-08T08:52:58.469 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.469 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:58.469 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.469 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.469 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.469 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1104 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11640832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:58.469 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:52:58.469 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 21.873202008, 2024-05-08T08:52:58.469 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070521947000000001, 2024-05-08T08:52:58.469 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:58.469 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:58.469 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:58.469 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:58.469 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:58.470 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:58.470 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1104 2024-05-08T08:52:58.470 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.470 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:58.470 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.470 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:58.470 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:52:58.470 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:58.470 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.470 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.470 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:58.470 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:52:58.470 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:58.470 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.471 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.471 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:58.471 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489700+0000", 2024-05-08T08:52:58.471 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00070067600000000001 2024-05-08T08:52:58.471 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.471 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.471 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:58.471 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489704+0000", 2024-05-08T08:52:58.471 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1640000000000003e-06 2024-05-08T08:52:58.471 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.471 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.471 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:58.471 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489716+0000", 2024-05-08T08:52:58.471 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1379000000000001e-05 2024-05-08T08:52:58.471 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.472 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.472 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:58.472 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489724+0000", 2024-05-08T08:52:58.472 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8949999999999998e-06 2024-05-08T08:52:58.472 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.472 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.472 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:58.472 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559278+0000", 2024-05-08T08:52:58.472 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.069554151999999994 2024-05-08T08:52:58.472 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.472 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.472 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:58.472 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559387+0000", 2024-05-08T08:52:58.472 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000109258 2024-05-08T08:52:58.473 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.473 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.473 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:58.473 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559521+0000", 2024-05-08T08:52:58.473 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000134423 2024-05-08T08:52:58.473 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.473 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:58.473 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.473 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.473 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.473 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:8636 3.17 3:e8a0a3e4:::rbd_header.11402b26385:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-05-08T08:52:58.473 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:54.150971+0000", 2024-05-08T08:52:58.473 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 4.2112300139999999, 2024-05-08T08:52:58.473 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.217273233, 2024-05-08T08:52:58.474 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:58.474 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:58.474 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:58.474 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:58.474 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:52:58.474 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:52:58.474 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 8636 2024-05-08T08:52:58.474 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.474 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:58.474 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.474 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:58.474 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150971+0000", 2024-05-08T08:52:58.474 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:58.474 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.474 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.475 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:58.475 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150971+0000", 2024-05-08T08:52:58.475 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:58.475 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.475 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.475 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:58.475 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150972+0000", 2024-05-08T08:52:58.475 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.71e-07 2024-05-08T08:52:58.475 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.475 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.475 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:58.475 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150977+0000", 2024-05-08T08:52:58.475 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.152e-06 2024-05-08T08:52:58.475 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.475 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.476 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:58.476 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150990+0000", 2024-05-08T08:52:58.476 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2364999999999999e-05 2024-05-08T08:52:58.476 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.476 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.476 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:58.476 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150998+0000", 2024-05-08T08:52:58.476 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0739999999999993e-06 2024-05-08T08:52:58.476 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.476 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.476 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:58.476 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.151054+0000", 2024-05-08T08:52:58.476 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5949000000000003e-05 2024-05-08T08:52:58.476 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.477 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.477 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:52:58.477 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.151060+0000", 2024-05-08T08:52:58.477 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6120000000000004e-06 2024-05-08T08:52:58.477 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.477 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.477 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:58.477 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.367640+0000", 2024-05-08T08:52:58.477 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2165792739999999 2024-05-08T08:52:58.477 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.477 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.477 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:58.477 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.367839+0000", 2024-05-08T08:52:58.477 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000199185 2024-05-08T08:52:58.478 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.478 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.478 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:58.478 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.368245+0000", 2024-05-08T08:52:58.478 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00040575099999999998 2024-05-08T08:52:58.478 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.478 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:58.478 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.478 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.478 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:58.478 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:52:58.480 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:52:58.480 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:52:58.481 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:52:58.481 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:52:58.481 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:52:58.482 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:52:58.482 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:52:58.483 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:52:58.482+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:52:58.485 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:52:58.485 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:52:58.485 INFO:teuthology.orchestra.run.smithi059.stdout: "epoch": 52, 2024-05-08T08:52:58.485 INFO:teuthology.orchestra.run.smithi059.stdout: "fsid": "38dfae8c-5c5e-4b0f-a264-91f22c6c5554", 2024-05-08T08:52:58.485 INFO:teuthology.orchestra.run.smithi059.stdout: "created": "2024-05-08T08:51:45.571369+0000", 2024-05-08T08:52:58.485 INFO:teuthology.orchestra.run.smithi059.stdout: "modified": "2024-05-08T08:52:58.118482+0000", 2024-05-08T08:52:58.485 INFO:teuthology.orchestra.run.smithi059.stdout: "last_up_change": "2024-05-08T08:51:56.847706+0000", 2024-05-08T08:52:58.485 INFO:teuthology.orchestra.run.smithi059.stdout: "last_in_change": "2024-05-08T08:52:46.045870+0000", 2024-05-08T08:52:58.485 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-05-08T08:52:58.486 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_num": 5799936, 2024-05-08T08:52:58.486 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_set": [ 2024-05-08T08:52:58.486 INFO:teuthology.orchestra.run.smithi059.stdout: "pglog_hardlimit", 2024-05-08T08:52:58.486 INFO:teuthology.orchestra.run.smithi059.stdout: "purged_snapdirs", 2024-05-08T08:52:58.486 INFO:teuthology.orchestra.run.smithi059.stdout: "recovery_deletes", 2024-05-08T08:52:58.486 INFO:teuthology.orchestra.run.smithi059.stdout: "sortbitwise" 2024-05-08T08:52:58.486 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T08:52:58.486 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_version": 4, 2024-05-08T08:52:58.486 INFO:teuthology.orchestra.run.smithi059.stdout: "full_ratio": 0.94999998807907104, 2024-05-08T08:52:58.486 INFO:teuthology.orchestra.run.smithi059.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-05-08T08:52:58.486 INFO:teuthology.orchestra.run.smithi059.stdout: "nearfull_ratio": 0.85000002384185791, 2024-05-08T08:52:58.486 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_snapshot": "", 2024-05-08T08:52:58.486 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_max": 3, 2024-05-08T08:52:58.486 INFO:teuthology.orchestra.run.smithi059.stdout: "max_osd": 6, 2024-05-08T08:52:58.487 INFO:teuthology.orchestra.run.smithi059.stdout: "require_min_compat_client": "luminous", 2024-05-08T08:52:58.487 INFO:teuthology.orchestra.run.smithi059.stdout: "min_compat_client": "luminous", 2024-05-08T08:52:58.487 INFO:teuthology.orchestra.run.smithi059.stdout: "require_osd_release": "squid", 2024-05-08T08:52:58.487 INFO:teuthology.orchestra.run.smithi059.stdout: "allow_crimson": false, 2024-05-08T08:52:58.487 INFO:teuthology.orchestra.run.smithi059.stdout: "pools": [ 2024-05-08T08:52:58.487 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.487 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 1, 2024-05-08T08:52:58.487 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_name": ".mgr", 2024-05-08T08:52:58.487 INFO:teuthology.orchestra.run.smithi059.stdout: "create_time": "2024-05-08T08:51:57.787790+0000", 2024-05-08T08:52:58.487 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T08:52:58.487 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_names": "hashpspool", 2024-05-08T08:52:58.487 INFO:teuthology.orchestra.run.smithi059.stdout: "type": 1, 2024-05-08T08:52:58.487 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 2, 2024-05-08T08:52:58.487 INFO:teuthology.orchestra.run.smithi059.stdout: "min_size": 1, 2024-05-08T08:52:58.488 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_rule": 0, 2024-05-08T08:52:58.488 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_count": 0, 2024-05-08T08:52:58.488 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_target": 0, 2024-05-08T08:52:58.488 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_barrier": 0, 2024-05-08T08:52:58.488 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-08T08:52:58.488 INFO:teuthology.orchestra.run.smithi059.stdout: "object_hash": 2, 2024-05-08T08:52:58.488 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscale_mode": "off", 2024-05-08T08:52:58.488 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num": 1, 2024-05-08T08:52:58.488 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num": 1, 2024-05-08T08:52:58.488 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num_target": 1, 2024-05-08T08:52:58.488 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_target": 1, 2024-05-08T08:52:58.488 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_pending": 1, 2024-05-08T08:52:58.488 INFO:teuthology.orchestra.run.smithi059.stdout: "last_pg_merge_meta": { 2024-05-08T08:52:58.488 INFO:teuthology.orchestra.run.smithi059.stdout: "source_pgid": "0.0", 2024-05-08T08:52:58.489 INFO:teuthology.orchestra.run.smithi059.stdout: "ready_epoch": 0, 2024-05-08T08:52:58.489 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_started": 0, 2024-05-08T08:52:58.489 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_clean": 0, 2024-05-08T08:52:58.489 INFO:teuthology.orchestra.run.smithi059.stdout: "source_version": "0'0", 2024-05-08T08:52:58.489 INFO:teuthology.orchestra.run.smithi059.stdout: "target_version": "0'0" 2024-05-08T08:52:58.489 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.489 INFO:teuthology.orchestra.run.smithi059.stdout: "last_change": "14", 2024-05-08T08:52:58.489 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend": "0", 2024-05-08T08:52:58.489 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-08T08:52:58.489 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_preluminous": "0", 2024-05-08T08:52:58.489 INFO:teuthology.orchestra.run.smithi059.stdout: "auid": 0, 2024-05-08T08:52:58.489 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_mode": "selfmanaged", 2024-05-08T08:52:58.489 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_seq": 0, 2024-05-08T08:52:58.489 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_epoch": 0, 2024-05-08T08:52:58.490 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_snaps": [], 2024-05-08T08:52:58.490 INFO:teuthology.orchestra.run.smithi059.stdout: "removed_snaps": "[]", 2024-05-08T08:52:58.490 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_bytes": 0, 2024-05-08T08:52:58.490 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_objects": 0, 2024-05-08T08:52:58.490 INFO:teuthology.orchestra.run.smithi059.stdout: "tiers": [], 2024-05-08T08:52:58.490 INFO:teuthology.orchestra.run.smithi059.stdout: "tier_of": -1, 2024-05-08T08:52:58.490 INFO:teuthology.orchestra.run.smithi059.stdout: "read_tier": -1, 2024-05-08T08:52:58.490 INFO:teuthology.orchestra.run.smithi059.stdout: "write_tier": -1, 2024-05-08T08:52:58.490 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_mode": "none", 2024-05-08T08:52:58.490 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_bytes": 0, 2024-05-08T08:52:58.490 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_objects": 0, 2024-05-08T08:52:58.490 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-08T08:52:58.490 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-08T08:52:58.491 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-08T08:52:58.491 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_flush_age": 0, 2024-05-08T08:52:58.491 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_evict_age": 0, 2024-05-08T08:52:58.491 INFO:teuthology.orchestra.run.smithi059.stdout: "erasure_code_profile": "", 2024-05-08T08:52:58.491 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_params": { 2024-05-08T08:52:58.491 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "none" 2024-05-08T08:52:58.491 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.491 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_period": 0, 2024-05-08T08:52:58.491 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_count": 0, 2024-05-08T08:52:58.491 INFO:teuthology.orchestra.run.smithi059.stdout: "use_gmt_hitset": true, 2024-05-08T08:52:58.491 INFO:teuthology.orchestra.run.smithi059.stdout: "min_read_recency_for_promote": 0, 2024-05-08T08:52:58.491 INFO:teuthology.orchestra.run.smithi059.stdout: "min_write_recency_for_promote": 0, 2024-05-08T08:52:58.491 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_grade_decay_rate": 0, 2024-05-08T08:52:58.491 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_search_last_n": 0, 2024-05-08T08:52:58.492 INFO:teuthology.orchestra.run.smithi059.stdout: "grade_table": [], 2024-05-08T08:52:58.492 INFO:teuthology.orchestra.run.smithi059.stdout: "stripe_width": 0, 2024-05-08T08:52:58.492 INFO:teuthology.orchestra.run.smithi059.stdout: "expected_num_objects": 0, 2024-05-08T08:52:58.492 INFO:teuthology.orchestra.run.smithi059.stdout: "fast_read": false, 2024-05-08T08:52:58.492 INFO:teuthology.orchestra.run.smithi059.stdout: "options": { 2024-05-08T08:52:58.492 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_max": 32, 2024-05-08T08:52:58.492 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_min": 1 2024-05-08T08:52:58.492 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.492 INFO:teuthology.orchestra.run.smithi059.stdout: "application_metadata": { 2024-05-08T08:52:58.492 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": {} 2024-05-08T08:52:58.492 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.492 INFO:teuthology.orchestra.run.smithi059.stdout: "read_balance": { 2024-05-08T08:52:58.492 INFO:teuthology.orchestra.run.smithi059.stdout: "score_type": "Fair distribution", 2024-05-08T08:52:58.492 INFO:teuthology.orchestra.run.smithi059.stdout: "score_acting": 6.059999942779541, 2024-05-08T08:52:58.493 INFO:teuthology.orchestra.run.smithi059.stdout: "score_stable": 6.059999942779541, 2024-05-08T08:52:58.493 INFO:teuthology.orchestra.run.smithi059.stdout: "optimal_score": 0.33000001311302185, 2024-05-08T08:52:58.493 INFO:teuthology.orchestra.run.smithi059.stdout: "raw_score_acting": 2, 2024-05-08T08:52:58.493 INFO:teuthology.orchestra.run.smithi059.stdout: "raw_score_stable": 2, 2024-05-08T08:52:58.493 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity_weighted": 0.87819164991378784, 2024-05-08T08:52:58.493 INFO:teuthology.orchestra.run.smithi059.stdout: "average_primary_affinity": 0.87999999523162842, 2024-05-08T08:52:58.493 INFO:teuthology.orchestra.run.smithi059.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-05-08T08:52:58.493 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.493 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.493 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.493 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 2, 2024-05-08T08:52:58.493 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_name": "rbd", 2024-05-08T08:52:58.493 INFO:teuthology.orchestra.run.smithi059.stdout: "create_time": "2024-05-08T08:52:02.563677+0000", 2024-05-08T08:52:58.493 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T08:52:58.494 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_names": "hashpspool", 2024-05-08T08:52:58.494 INFO:teuthology.orchestra.run.smithi059.stdout: "type": 1, 2024-05-08T08:52:58.494 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 2, 2024-05-08T08:52:58.494 INFO:teuthology.orchestra.run.smithi059.stdout: "min_size": 1, 2024-05-08T08:52:58.494 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_rule": 0, 2024-05-08T08:52:58.494 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_count": 0, 2024-05-08T08:52:58.494 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_target": 0, 2024-05-08T08:52:58.494 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_barrier": 0, 2024-05-08T08:52:58.494 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-08T08:52:58.494 INFO:teuthology.orchestra.run.smithi059.stdout: "object_hash": 2, 2024-05-08T08:52:58.494 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscale_mode": "off", 2024-05-08T08:52:58.494 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num": 8, 2024-05-08T08:52:58.494 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num": 8, 2024-05-08T08:52:58.494 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num_target": 8, 2024-05-08T08:52:58.495 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_target": 8, 2024-05-08T08:52:58.495 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_pending": 8, 2024-05-08T08:52:58.495 INFO:teuthology.orchestra.run.smithi059.stdout: "last_pg_merge_meta": { 2024-05-08T08:52:58.495 INFO:teuthology.orchestra.run.smithi059.stdout: "source_pgid": "0.0", 2024-05-08T08:52:58.495 INFO:teuthology.orchestra.run.smithi059.stdout: "ready_epoch": 0, 2024-05-08T08:52:58.495 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_started": 0, 2024-05-08T08:52:58.495 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_clean": 0, 2024-05-08T08:52:58.495 INFO:teuthology.orchestra.run.smithi059.stdout: "source_version": "0'0", 2024-05-08T08:52:58.495 INFO:teuthology.orchestra.run.smithi059.stdout: "target_version": "0'0" 2024-05-08T08:52:58.495 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.495 INFO:teuthology.orchestra.run.smithi059.stdout: "last_change": "17", 2024-05-08T08:52:58.495 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend": "0", 2024-05-08T08:52:58.495 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-08T08:52:58.495 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_preluminous": "0", 2024-05-08T08:52:58.495 INFO:teuthology.orchestra.run.smithi059.stdout: "auid": 0, 2024-05-08T08:52:58.496 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_mode": "selfmanaged", 2024-05-08T08:52:58.496 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_seq": 0, 2024-05-08T08:52:58.496 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_epoch": 0, 2024-05-08T08:52:58.496 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_snaps": [], 2024-05-08T08:52:58.496 INFO:teuthology.orchestra.run.smithi059.stdout: "removed_snaps": "[]", 2024-05-08T08:52:58.496 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_bytes": 0, 2024-05-08T08:52:58.496 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_objects": 0, 2024-05-08T08:52:58.496 INFO:teuthology.orchestra.run.smithi059.stdout: "tiers": [], 2024-05-08T08:52:58.496 INFO:teuthology.orchestra.run.smithi059.stdout: "tier_of": -1, 2024-05-08T08:52:58.496 INFO:teuthology.orchestra.run.smithi059.stdout: "read_tier": -1, 2024-05-08T08:52:58.496 INFO:teuthology.orchestra.run.smithi059.stdout: "write_tier": -1, 2024-05-08T08:52:58.496 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_mode": "none", 2024-05-08T08:52:58.496 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_bytes": 0, 2024-05-08T08:52:58.496 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_objects": 0, 2024-05-08T08:52:58.497 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-08T08:52:58.497 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-08T08:52:58.497 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-08T08:52:58.497 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_flush_age": 0, 2024-05-08T08:52:58.497 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_evict_age": 0, 2024-05-08T08:52:58.497 INFO:teuthology.orchestra.run.smithi059.stdout: "erasure_code_profile": "", 2024-05-08T08:52:58.497 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_params": { 2024-05-08T08:52:58.497 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "none" 2024-05-08T08:52:58.497 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.497 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_period": 0, 2024-05-08T08:52:58.497 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_count": 0, 2024-05-08T08:52:58.497 INFO:teuthology.orchestra.run.smithi059.stdout: "use_gmt_hitset": true, 2024-05-08T08:52:58.497 INFO:teuthology.orchestra.run.smithi059.stdout: "min_read_recency_for_promote": 0, 2024-05-08T08:52:58.497 INFO:teuthology.orchestra.run.smithi059.stdout: "min_write_recency_for_promote": 0, 2024-05-08T08:52:58.498 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_grade_decay_rate": 0, 2024-05-08T08:52:58.498 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_search_last_n": 0, 2024-05-08T08:52:58.498 INFO:teuthology.orchestra.run.smithi059.stdout: "grade_table": [], 2024-05-08T08:52:58.498 INFO:teuthology.orchestra.run.smithi059.stdout: "stripe_width": 0, 2024-05-08T08:52:58.498 INFO:teuthology.orchestra.run.smithi059.stdout: "expected_num_objects": 0, 2024-05-08T08:52:58.498 INFO:teuthology.orchestra.run.smithi059.stdout: "fast_read": false, 2024-05-08T08:52:58.498 INFO:teuthology.orchestra.run.smithi059.stdout: "options": {}, 2024-05-08T08:52:58.498 INFO:teuthology.orchestra.run.smithi059.stdout: "application_metadata": { 2024-05-08T08:52:58.498 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd": {} 2024-05-08T08:52:58.498 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.498 INFO:teuthology.orchestra.run.smithi059.stdout: "read_balance": { 2024-05-08T08:52:58.498 INFO:teuthology.orchestra.run.smithi059.stdout: "score_type": "Fair distribution", 2024-05-08T08:52:58.498 INFO:teuthology.orchestra.run.smithi059.stdout: "score_acting": 2.2699999809265137, 2024-05-08T08:52:58.498 INFO:teuthology.orchestra.run.smithi059.stdout: "score_stable": 2.2699999809265137, 2024-05-08T08:52:58.499 INFO:teuthology.orchestra.run.smithi059.stdout: "optimal_score": 0.82999998331069946, 2024-05-08T08:52:58.499 INFO:teuthology.orchestra.run.smithi059.stdout: "raw_score_acting": 1.8799999952316284, 2024-05-08T08:52:58.499 INFO:teuthology.orchestra.run.smithi059.stdout: "raw_score_stable": 1.8799999952316284, 2024-05-08T08:52:58.499 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity_weighted": 0.87819164991378784, 2024-05-08T08:52:58.499 INFO:teuthology.orchestra.run.smithi059.stdout: "average_primary_affinity": 0.87999999523162842, 2024-05-08T08:52:58.499 INFO:teuthology.orchestra.run.smithi059.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-05-08T08:52:58.499 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.499 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.499 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.499 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 3, 2024-05-08T08:52:58.499 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_name": "pool_client.0", 2024-05-08T08:52:58.499 INFO:teuthology.orchestra.run.smithi059.stdout: "create_time": "2024-05-08T08:52:09.955467+0000", 2024-05-08T08:52:58.499 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 8193, 2024-05-08T08:52:58.499 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-05-08T08:52:58.499 INFO:teuthology.orchestra.run.smithi059.stdout: "type": 1, 2024-05-08T08:52:58.500 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 2, 2024-05-08T08:52:58.500 INFO:teuthology.orchestra.run.smithi059.stdout: "min_size": 1, 2024-05-08T08:52:58.500 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_rule": 0, 2024-05-08T08:52:58.500 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_count": 0, 2024-05-08T08:52:58.500 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_target": 0, 2024-05-08T08:52:58.500 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_barrier": 0, 2024-05-08T08:52:58.500 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-08T08:52:58.500 INFO:teuthology.orchestra.run.smithi059.stdout: "object_hash": 2, 2024-05-08T08:52:58.500 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscale_mode": "off", 2024-05-08T08:52:58.500 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num": 32, 2024-05-08T08:52:58.500 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num": 32, 2024-05-08T08:52:58.500 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num_target": 32, 2024-05-08T08:52:58.500 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_target": 32, 2024-05-08T08:52:58.500 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_pending": 32, 2024-05-08T08:52:58.501 INFO:teuthology.orchestra.run.smithi059.stdout: "last_pg_merge_meta": { 2024-05-08T08:52:58.501 INFO:teuthology.orchestra.run.smithi059.stdout: "source_pgid": "0.0", 2024-05-08T08:52:58.501 INFO:teuthology.orchestra.run.smithi059.stdout: "ready_epoch": 0, 2024-05-08T08:52:58.501 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_started": 0, 2024-05-08T08:52:58.501 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_clean": 0, 2024-05-08T08:52:58.501 INFO:teuthology.orchestra.run.smithi059.stdout: "source_version": "0'0", 2024-05-08T08:52:58.501 INFO:teuthology.orchestra.run.smithi059.stdout: "target_version": "0'0" 2024-05-08T08:52:58.501 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.501 INFO:teuthology.orchestra.run.smithi059.stdout: "last_change": "52", 2024-05-08T08:52:58.501 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend": "0", 2024-05-08T08:52:58.501 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-08T08:52:58.501 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_preluminous": "0", 2024-05-08T08:52:58.501 INFO:teuthology.orchestra.run.smithi059.stdout: "auid": 0, 2024-05-08T08:52:58.501 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_mode": "selfmanaged", 2024-05-08T08:52:58.502 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_seq": 16, 2024-05-08T08:52:58.502 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_epoch": 52, 2024-05-08T08:52:58.502 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_snaps": [], 2024-05-08T08:52:58.502 INFO:teuthology.orchestra.run.smithi059.stdout: "removed_snaps": "[]", 2024-05-08T08:52:58.502 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_bytes": 0, 2024-05-08T08:52:58.502 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_objects": 0, 2024-05-08T08:52:58.502 INFO:teuthology.orchestra.run.smithi059.stdout: "tiers": [], 2024-05-08T08:52:58.502 INFO:teuthology.orchestra.run.smithi059.stdout: "tier_of": -1, 2024-05-08T08:52:58.502 INFO:teuthology.orchestra.run.smithi059.stdout: "read_tier": -1, 2024-05-08T08:52:58.502 INFO:teuthology.orchestra.run.smithi059.stdout: "write_tier": -1, 2024-05-08T08:52:58.502 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_mode": "none", 2024-05-08T08:52:58.502 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_bytes": 0, 2024-05-08T08:52:58.502 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_objects": 0, 2024-05-08T08:52:58.502 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-08T08:52:58.503 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-08T08:52:58.503 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-08T08:52:58.503 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_flush_age": 0, 2024-05-08T08:52:58.503 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_evict_age": 0, 2024-05-08T08:52:58.503 INFO:teuthology.orchestra.run.smithi059.stdout: "erasure_code_profile": "", 2024-05-08T08:52:58.503 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_params": { 2024-05-08T08:52:58.503 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "none" 2024-05-08T08:52:58.503 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.503 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_period": 0, 2024-05-08T08:52:58.503 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_count": 0, 2024-05-08T08:52:58.503 INFO:teuthology.orchestra.run.smithi059.stdout: "use_gmt_hitset": true, 2024-05-08T08:52:58.503 INFO:teuthology.orchestra.run.smithi059.stdout: "min_read_recency_for_promote": 0, 2024-05-08T08:52:58.503 INFO:teuthology.orchestra.run.smithi059.stdout: "min_write_recency_for_promote": 0, 2024-05-08T08:52:58.503 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_grade_decay_rate": 0, 2024-05-08T08:52:58.504 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_search_last_n": 0, 2024-05-08T08:52:58.504 INFO:teuthology.orchestra.run.smithi059.stdout: "grade_table": [], 2024-05-08T08:52:58.504 INFO:teuthology.orchestra.run.smithi059.stdout: "stripe_width": 0, 2024-05-08T08:52:58.504 INFO:teuthology.orchestra.run.smithi059.stdout: "expected_num_objects": 0, 2024-05-08T08:52:58.504 INFO:teuthology.orchestra.run.smithi059.stdout: "fast_read": false, 2024-05-08T08:52:58.504 INFO:teuthology.orchestra.run.smithi059.stdout: "options": {}, 2024-05-08T08:52:58.504 INFO:teuthology.orchestra.run.smithi059.stdout: "application_metadata": { 2024-05-08T08:52:58.504 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd": {} 2024-05-08T08:52:58.504 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.504 INFO:teuthology.orchestra.run.smithi059.stdout: "read_balance": { 2024-05-08T08:52:58.504 INFO:teuthology.orchestra.run.smithi059.stdout: "score_type": "Fair distribution", 2024-05-08T08:52:58.504 INFO:teuthology.orchestra.run.smithi059.stdout: "score_acting": 1.7000000476837158, 2024-05-08T08:52:58.505 INFO:teuthology.orchestra.run.smithi059.stdout: "score_stable": 1.7000000476837158, 2024-05-08T08:52:58.505 INFO:teuthology.orchestra.run.smithi059.stdout: "optimal_score": 0.82999998331069946, 2024-05-08T08:52:58.505 INFO:teuthology.orchestra.run.smithi059.stdout: "raw_score_acting": 1.4099999666213989, 2024-05-08T08:52:58.505 INFO:teuthology.orchestra.run.smithi059.stdout: "raw_score_stable": 1.4099999666213989, 2024-05-08T08:52:58.505 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity_weighted": 0.87819164991378784, 2024-05-08T08:52:58.505 INFO:teuthology.orchestra.run.smithi059.stdout: "average_primary_affinity": 0.87999999523162842, 2024-05-08T08:52:58.505 INFO:teuthology.orchestra.run.smithi059.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-05-08T08:52:58.505 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.505 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.505 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T08:52:58.505 INFO:teuthology.orchestra.run.smithi059.stdout: "osds": [ 2024-05-08T08:52:58.505 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.505 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 0, 2024-05-08T08:52:58.505 INFO:teuthology.orchestra.run.smithi059.stdout: "uuid": "300801a5-1919-4b23-8c6d-bc7dad61cc4d", 2024-05-08T08:52:58.506 INFO:teuthology.orchestra.run.smithi059.stdout: "up": 1, 2024-05-08T08:52:58.506 INFO:teuthology.orchestra.run.smithi059.stdout: "in": 1, 2024-05-08T08:52:58.506 INFO:teuthology.orchestra.run.smithi059.stdout: "weight": 1, 2024-05-08T08:52:58.506 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity": 1, 2024-05-08T08:52:58.506 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_begin": 0, 2024-05-08T08:52:58.506 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_end": 0, 2024-05-08T08:52:58.506 INFO:teuthology.orchestra.run.smithi059.stdout: "up_from": 11, 2024-05-08T08:52:58.506 INFO:teuthology.orchestra.run.smithi059.stdout: "up_thru": 49, 2024-05-08T08:52:58.506 INFO:teuthology.orchestra.run.smithi059.stdout: "down_at": 0, 2024-05-08T08:52:58.506 INFO:teuthology.orchestra.run.smithi059.stdout: "lost_at": 0, 2024-05-08T08:52:58.506 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrs": { 2024-05-08T08:52:58.506 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:52:58.506 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.506 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:52:58.507 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6800", 2024-05-08T08:52:58.507 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 802160151 2024-05-08T08:52:58.507 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.507 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.507 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:52:58.507 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6801", 2024-05-08T08:52:58.507 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 802160151 2024-05-08T08:52:58.507 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.507 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:58.507 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.507 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addrs": { 2024-05-08T08:52:58.507 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:52:58.507 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.507 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:52:58.507 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6802", 2024-05-08T08:52:58.508 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 802160151 2024-05-08T08:52:58.508 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.508 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.508 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:52:58.508 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6803", 2024-05-08T08:52:58.508 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 802160151 2024-05-08T08:52:58.508 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.508 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:58.508 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.508 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addrs": { 2024-05-08T08:52:58.508 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:52:58.508 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.508 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:52:58.508 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6806", 2024-05-08T08:52:58.509 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 802160151 2024-05-08T08:52:58.509 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.509 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.509 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:52:58.509 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6807", 2024-05-08T08:52:58.509 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 802160151 2024-05-08T08:52:58.509 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.509 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:58.509 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.509 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addrs": { 2024-05-08T08:52:58.509 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:52:58.509 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.509 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:52:58.509 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6804", 2024-05-08T08:52:58.509 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 802160151 2024-05-08T08:52:58.510 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.510 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.510 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:52:58.510 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6805", 2024-05-08T08:52:58.510 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 802160151 2024-05-08T08:52:58.510 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.510 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:58.510 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.510 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "172.21.15.59:6801/802160151", 2024-05-08T08:52:58.510 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "172.21.15.59:6803/802160151", 2024-05-08T08:52:58.510 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addr": "172.21.15.59:6807/802160151", 2024-05-08T08:52:58.510 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addr": "172.21.15.59:6805/802160151", 2024-05-08T08:52:58.510 INFO:teuthology.orchestra.run.smithi059.stdout: "state": [ 2024-05-08T08:52:58.511 INFO:teuthology.orchestra.run.smithi059.stdout: "exists", 2024-05-08T08:52:58.511 INFO:teuthology.orchestra.run.smithi059.stdout: "up" 2024-05-08T08:52:58.511 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:58.511 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.511 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.511 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 1, 2024-05-08T08:52:58.511 INFO:teuthology.orchestra.run.smithi059.stdout: "uuid": "0fead0de-6a65-4eba-b400-eea5692702e0", 2024-05-08T08:52:58.511 INFO:teuthology.orchestra.run.smithi059.stdout: "up": 1, 2024-05-08T08:52:58.511 INFO:teuthology.orchestra.run.smithi059.stdout: "in": 1, 2024-05-08T08:52:58.511 INFO:teuthology.orchestra.run.smithi059.stdout: "weight": 1, 2024-05-08T08:52:58.511 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity": 1, 2024-05-08T08:52:58.511 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_begin": 0, 2024-05-08T08:52:58.511 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_end": 0, 2024-05-08T08:52:58.511 INFO:teuthology.orchestra.run.smithi059.stdout: "up_from": 11, 2024-05-08T08:52:58.511 INFO:teuthology.orchestra.run.smithi059.stdout: "up_thru": 44, 2024-05-08T08:52:58.511 INFO:teuthology.orchestra.run.smithi059.stdout: "down_at": 0, 2024-05-08T08:52:58.512 INFO:teuthology.orchestra.run.smithi059.stdout: "lost_at": 0, 2024-05-08T08:52:58.512 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrs": { 2024-05-08T08:52:58.512 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:52:58.512 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.512 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:52:58.512 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6816", 2024-05-08T08:52:58.512 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4116801011 2024-05-08T08:52:58.512 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.512 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.512 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:52:58.512 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6817", 2024-05-08T08:52:58.512 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4116801011 2024-05-08T08:52:58.512 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.512 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:58.512 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.513 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addrs": { 2024-05-08T08:52:58.513 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:52:58.513 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.513 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:52:58.513 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6818", 2024-05-08T08:52:58.513 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4116801011 2024-05-08T08:52:58.513 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.513 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.513 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:52:58.513 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6819", 2024-05-08T08:52:58.513 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4116801011 2024-05-08T08:52:58.513 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.513 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:58.513 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.514 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addrs": { 2024-05-08T08:52:58.514 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:52:58.514 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.514 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:52:58.514 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6822", 2024-05-08T08:52:58.514 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4116801011 2024-05-08T08:52:58.514 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.514 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.514 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:52:58.514 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6823", 2024-05-08T08:52:58.514 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4116801011 2024-05-08T08:52:58.514 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.514 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:58.514 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.515 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addrs": { 2024-05-08T08:52:58.515 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:52:58.515 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.515 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:52:58.515 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6820", 2024-05-08T08:52:58.515 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4116801011 2024-05-08T08:52:58.515 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.515 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.515 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:52:58.515 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6821", 2024-05-08T08:52:58.515 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4116801011 2024-05-08T08:52:58.515 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.515 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:58.515 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.515 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "172.21.15.59:6817/4116801011", 2024-05-08T08:52:58.515 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "172.21.15.59:6819/4116801011", 2024-05-08T08:52:58.516 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addr": "172.21.15.59:6823/4116801011", 2024-05-08T08:52:58.516 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addr": "172.21.15.59:6821/4116801011", 2024-05-08T08:52:58.516 INFO:teuthology.orchestra.run.smithi059.stdout: "state": [ 2024-05-08T08:52:58.516 INFO:teuthology.orchestra.run.smithi059.stdout: "exists", 2024-05-08T08:52:58.516 INFO:teuthology.orchestra.run.smithi059.stdout: "up" 2024-05-08T08:52:58.516 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:58.516 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.516 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.516 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 2, 2024-05-08T08:52:58.516 INFO:teuthology.orchestra.run.smithi059.stdout: "uuid": "1a233de6-1b82-4a51-8e96-4cd4abdf16c8", 2024-05-08T08:52:58.516 INFO:teuthology.orchestra.run.smithi059.stdout: "up": 1, 2024-05-08T08:52:58.516 INFO:teuthology.orchestra.run.smithi059.stdout: "in": 1, 2024-05-08T08:52:58.516 INFO:teuthology.orchestra.run.smithi059.stdout: "weight": 1, 2024-05-08T08:52:58.516 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity": 1, 2024-05-08T08:52:58.516 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_begin": 0, 2024-05-08T08:52:58.517 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_end": 0, 2024-05-08T08:52:58.517 INFO:teuthology.orchestra.run.smithi059.stdout: "up_from": 11, 2024-05-08T08:52:58.517 INFO:teuthology.orchestra.run.smithi059.stdout: "up_thru": 50, 2024-05-08T08:52:58.517 INFO:teuthology.orchestra.run.smithi059.stdout: "down_at": 0, 2024-05-08T08:52:58.517 INFO:teuthology.orchestra.run.smithi059.stdout: "lost_at": 0, 2024-05-08T08:52:58.517 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrs": { 2024-05-08T08:52:58.517 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:52:58.517 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.517 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:52:58.517 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6808", 2024-05-08T08:52:58.517 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1547661179 2024-05-08T08:52:58.517 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.517 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.517 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:52:58.517 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6809", 2024-05-08T08:52:58.517 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1547661179 2024-05-08T08:52:58.518 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.518 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:58.518 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.518 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addrs": { 2024-05-08T08:52:58.518 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:52:58.518 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.518 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:52:58.518 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6810", 2024-05-08T08:52:58.518 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1547661179 2024-05-08T08:52:58.518 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.518 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.518 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:52:58.518 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6811", 2024-05-08T08:52:58.518 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1547661179 2024-05-08T08:52:58.519 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.519 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:58.519 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.519 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addrs": { 2024-05-08T08:52:58.519 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:52:58.519 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.519 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:52:58.519 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6814", 2024-05-08T08:52:58.519 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1547661179 2024-05-08T08:52:58.519 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.519 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.519 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:52:58.519 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6815", 2024-05-08T08:52:58.519 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1547661179 2024-05-08T08:52:58.519 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.520 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:58.520 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.520 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addrs": { 2024-05-08T08:52:58.520 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:52:58.520 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.520 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:52:58.520 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6812", 2024-05-08T08:52:58.520 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1547661179 2024-05-08T08:52:58.520 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.520 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.520 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:52:58.520 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6813", 2024-05-08T08:52:58.520 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1547661179 2024-05-08T08:52:58.520 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.520 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:58.521 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.521 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "172.21.15.59:6809/1547661179", 2024-05-08T08:52:58.521 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "172.21.15.59:6811/1547661179", 2024-05-08T08:52:58.521 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addr": "172.21.15.59:6815/1547661179", 2024-05-08T08:52:58.521 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addr": "172.21.15.59:6813/1547661179", 2024-05-08T08:52:58.521 INFO:teuthology.orchestra.run.smithi059.stdout: "state": [ 2024-05-08T08:52:58.521 INFO:teuthology.orchestra.run.smithi059.stdout: "exists", 2024-05-08T08:52:58.521 INFO:teuthology.orchestra.run.smithi059.stdout: "up" 2024-05-08T08:52:58.521 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:58.521 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.521 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.521 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 3, 2024-05-08T08:52:58.521 INFO:teuthology.orchestra.run.smithi059.stdout: "uuid": "cc4870b8-dcd9-4e43-978d-8de6c6396abf", 2024-05-08T08:52:58.521 INFO:teuthology.orchestra.run.smithi059.stdout: "up": 1, 2024-05-08T08:52:58.521 INFO:teuthology.orchestra.run.smithi059.stdout: "in": 1, 2024-05-08T08:52:58.522 INFO:teuthology.orchestra.run.smithi059.stdout: "weight": 1, 2024-05-08T08:52:58.522 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity": 1, 2024-05-08T08:52:58.522 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_begin": 0, 2024-05-08T08:52:58.522 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_end": 0, 2024-05-08T08:52:58.522 INFO:teuthology.orchestra.run.smithi059.stdout: "up_from": 11, 2024-05-08T08:52:58.522 INFO:teuthology.orchestra.run.smithi059.stdout: "up_thru": 44, 2024-05-08T08:52:58.522 INFO:teuthology.orchestra.run.smithi059.stdout: "down_at": 0, 2024-05-08T08:52:58.522 INFO:teuthology.orchestra.run.smithi059.stdout: "lost_at": 0, 2024-05-08T08:52:58.522 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrs": { 2024-05-08T08:52:58.522 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:52:58.522 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.522 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:52:58.522 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6800", 2024-05-08T08:52:58.522 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4054400810 2024-05-08T08:52:58.523 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.523 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.523 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:52:58.523 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6801", 2024-05-08T08:52:58.523 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4054400810 2024-05-08T08:52:58.523 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.523 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:58.523 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.523 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addrs": { 2024-05-08T08:52:58.523 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:52:58.523 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.523 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:52:58.523 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6802", 2024-05-08T08:52:58.523 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4054400810 2024-05-08T08:52:58.523 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.524 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.524 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:52:58.524 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6803", 2024-05-08T08:52:58.524 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4054400810 2024-05-08T08:52:58.524 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.524 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:58.524 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.524 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addrs": { 2024-05-08T08:52:58.524 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:52:58.524 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.524 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:52:58.524 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6806", 2024-05-08T08:52:58.524 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4054400810 2024-05-08T08:52:58.524 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.525 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.525 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:52:58.525 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6807", 2024-05-08T08:52:58.525 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4054400810 2024-05-08T08:52:58.525 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.525 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:58.525 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.525 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addrs": { 2024-05-08T08:52:58.525 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:52:58.525 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.525 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:52:58.525 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6804", 2024-05-08T08:52:58.525 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4054400810 2024-05-08T08:52:58.525 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.525 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.526 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:52:58.526 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6805", 2024-05-08T08:52:58.526 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4054400810 2024-05-08T08:52:58.526 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.526 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:58.526 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.526 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "172.21.15.192:6801/4054400810", 2024-05-08T08:52:58.526 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "172.21.15.192:6803/4054400810", 2024-05-08T08:52:58.526 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addr": "172.21.15.192:6807/4054400810", 2024-05-08T08:52:58.526 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addr": "172.21.15.192:6805/4054400810", 2024-05-08T08:52:58.526 INFO:teuthology.orchestra.run.smithi059.stdout: "state": [ 2024-05-08T08:52:58.526 INFO:teuthology.orchestra.run.smithi059.stdout: "exists", 2024-05-08T08:52:58.526 INFO:teuthology.orchestra.run.smithi059.stdout: "up" 2024-05-08T08:52:58.527 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:58.527 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.527 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.527 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 4, 2024-05-08T08:52:58.527 INFO:teuthology.orchestra.run.smithi059.stdout: "uuid": "d1d05620-42fb-4107-a2f9-e83542d076ee", 2024-05-08T08:52:58.527 INFO:teuthology.orchestra.run.smithi059.stdout: "up": 1, 2024-05-08T08:52:58.527 INFO:teuthology.orchestra.run.smithi059.stdout: "in": 0, 2024-05-08T08:52:58.527 INFO:teuthology.orchestra.run.smithi059.stdout: "weight": 0, 2024-05-08T08:52:58.527 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity": 0.9892578125, 2024-05-08T08:52:58.527 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_begin": 0, 2024-05-08T08:52:58.527 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_end": 0, 2024-05-08T08:52:58.527 INFO:teuthology.orchestra.run.smithi059.stdout: "up_from": 11, 2024-05-08T08:52:58.527 INFO:teuthology.orchestra.run.smithi059.stdout: "up_thru": 43, 2024-05-08T08:52:58.527 INFO:teuthology.orchestra.run.smithi059.stdout: "down_at": 0, 2024-05-08T08:52:58.527 INFO:teuthology.orchestra.run.smithi059.stdout: "lost_at": 0, 2024-05-08T08:52:58.528 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrs": { 2024-05-08T08:52:58.528 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:52:58.528 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.528 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:52:58.528 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6808", 2024-05-08T08:52:58.528 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 58985565 2024-05-08T08:52:58.528 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.528 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.528 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:52:58.528 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6809", 2024-05-08T08:52:58.528 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 58985565 2024-05-08T08:52:58.528 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.528 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:58.528 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.528 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addrs": { 2024-05-08T08:52:58.529 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:52:58.529 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.529 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:52:58.529 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6810", 2024-05-08T08:52:58.529 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 58985565 2024-05-08T08:52:58.529 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.529 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.529 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:52:58.529 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6811", 2024-05-08T08:52:58.529 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 58985565 2024-05-08T08:52:58.529 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.529 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:58.529 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.529 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addrs": { 2024-05-08T08:52:58.529 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:52:58.530 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.530 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:52:58.530 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6814", 2024-05-08T08:52:58.530 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 58985565 2024-05-08T08:52:58.530 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.530 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.530 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:52:58.530 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6815", 2024-05-08T08:52:58.530 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 58985565 2024-05-08T08:52:58.530 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.530 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:58.530 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.530 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addrs": { 2024-05-08T08:52:58.530 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:52:58.531 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.531 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:52:58.531 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6812", 2024-05-08T08:52:58.531 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 58985565 2024-05-08T08:52:58.531 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.531 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.531 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:52:58.531 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6813", 2024-05-08T08:52:58.531 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 58985565 2024-05-08T08:52:58.531 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.531 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:58.531 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.531 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "172.21.15.192:6809/58985565", 2024-05-08T08:52:58.531 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "172.21.15.192:6811/58985565", 2024-05-08T08:52:58.531 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addr": "172.21.15.192:6815/58985565", 2024-05-08T08:52:58.532 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addr": "172.21.15.192:6813/58985565", 2024-05-08T08:52:58.532 INFO:teuthology.orchestra.run.smithi059.stdout: "state": [ 2024-05-08T08:52:58.532 INFO:teuthology.orchestra.run.smithi059.stdout: "exists", 2024-05-08T08:52:58.532 INFO:teuthology.orchestra.run.smithi059.stdout: "up" 2024-05-08T08:52:58.532 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:58.532 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.532 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.532 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 5, 2024-05-08T08:52:58.532 INFO:teuthology.orchestra.run.smithi059.stdout: "uuid": "ab26136c-31c3-4090-9fa2-988dba4f731f", 2024-05-08T08:52:58.532 INFO:teuthology.orchestra.run.smithi059.stdout: "up": 1, 2024-05-08T08:52:58.532 INFO:teuthology.orchestra.run.smithi059.stdout: "in": 1, 2024-05-08T08:52:58.532 INFO:teuthology.orchestra.run.smithi059.stdout: "weight": 1, 2024-05-08T08:52:58.532 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity": 0.2798919677734375, 2024-05-08T08:52:58.532 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_begin": 0, 2024-05-08T08:52:58.532 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_end": 0, 2024-05-08T08:52:58.533 INFO:teuthology.orchestra.run.smithi059.stdout: "up_from": 11, 2024-05-08T08:52:58.533 INFO:teuthology.orchestra.run.smithi059.stdout: "up_thru": 18, 2024-05-08T08:52:58.533 INFO:teuthology.orchestra.run.smithi059.stdout: "down_at": 0, 2024-05-08T08:52:58.533 INFO:teuthology.orchestra.run.smithi059.stdout: "lost_at": 0, 2024-05-08T08:52:58.533 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrs": { 2024-05-08T08:52:58.533 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:52:58.533 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.533 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:52:58.533 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6816", 2024-05-08T08:52:58.533 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1283571573 2024-05-08T08:52:58.533 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.533 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.533 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:52:58.533 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6817", 2024-05-08T08:52:58.533 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1283571573 2024-05-08T08:52:58.533 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.534 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:58.534 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.534 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addrs": { 2024-05-08T08:52:58.534 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:52:58.534 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.534 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:52:58.534 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6818", 2024-05-08T08:52:58.534 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1283571573 2024-05-08T08:52:58.534 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.534 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.534 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:52:58.534 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6819", 2024-05-08T08:52:58.534 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1283571573 2024-05-08T08:52:58.534 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.535 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:58.535 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.535 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addrs": { 2024-05-08T08:52:58.535 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:52:58.535 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.535 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:52:58.535 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6822", 2024-05-08T08:52:58.535 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1283571573 2024-05-08T08:52:58.535 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.535 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.535 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:52:58.535 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6823", 2024-05-08T08:52:58.535 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1283571573 2024-05-08T08:52:58.535 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.535 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:58.536 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.536 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addrs": { 2024-05-08T08:52:58.536 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:52:58.536 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.536 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:52:58.536 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6820", 2024-05-08T08:52:58.536 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1283571573 2024-05-08T08:52:58.536 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.536 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.536 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:52:58.536 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6821", 2024-05-08T08:52:58.536 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1283571573 2024-05-08T08:52:58.536 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.536 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:58.536 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.536 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "172.21.15.192:6817/1283571573", 2024-05-08T08:52:58.537 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "172.21.15.192:6819/1283571573", 2024-05-08T08:52:58.537 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addr": "172.21.15.192:6823/1283571573", 2024-05-08T08:52:58.537 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addr": "172.21.15.192:6821/1283571573", 2024-05-08T08:52:58.537 INFO:teuthology.orchestra.run.smithi059.stdout: "state": [ 2024-05-08T08:52:58.537 INFO:teuthology.orchestra.run.smithi059.stdout: "exists", 2024-05-08T08:52:58.537 INFO:teuthology.orchestra.run.smithi059.stdout: "up" 2024-05-08T08:52:58.537 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:58.537 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.537 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T08:52:58.537 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_xinfo": [ 2024-05-08T08:52:58.537 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.537 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 0, 2024-05-08T08:52:58.537 INFO:teuthology.orchestra.run.smithi059.stdout: "down_stamp": "0.000000", 2024-05-08T08:52:58.537 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_probability": 0, 2024-05-08T08:52:58.538 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_interval": 0, 2024-05-08T08:52:58.538 INFO:teuthology.orchestra.run.smithi059.stdout: "features": 4540701547738038271, 2024-05-08T08:52:58.538 INFO:teuthology.orchestra.run.smithi059.stdout: "old_weight": 0, 2024-05-08T08:52:58.538 INFO:teuthology.orchestra.run.smithi059.stdout: "last_purged_snaps_scrub": "2024-05-08T08:51:52.396946+0000", 2024-05-08T08:52:58.538 INFO:teuthology.orchestra.run.smithi059.stdout: "dead_epoch": 0 2024-05-08T08:52:58.538 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.538 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.538 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 1, 2024-05-08T08:52:58.538 INFO:teuthology.orchestra.run.smithi059.stdout: "down_stamp": "0.000000", 2024-05-08T08:52:58.538 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_probability": 0, 2024-05-08T08:52:58.538 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_interval": 0, 2024-05-08T08:52:58.538 INFO:teuthology.orchestra.run.smithi059.stdout: "features": 4540701547738038271, 2024-05-08T08:52:58.538 INFO:teuthology.orchestra.run.smithi059.stdout: "old_weight": 0, 2024-05-08T08:52:58.538 INFO:teuthology.orchestra.run.smithi059.stdout: "last_purged_snaps_scrub": "2024-05-08T08:51:52.453160+0000", 2024-05-08T08:52:58.539 INFO:teuthology.orchestra.run.smithi059.stdout: "dead_epoch": 0 2024-05-08T08:52:58.539 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.539 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.539 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 2, 2024-05-08T08:52:58.539 INFO:teuthology.orchestra.run.smithi059.stdout: "down_stamp": "0.000000", 2024-05-08T08:52:58.539 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_probability": 0, 2024-05-08T08:52:58.539 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_interval": 0, 2024-05-08T08:52:58.539 INFO:teuthology.orchestra.run.smithi059.stdout: "features": 4540701547738038271, 2024-05-08T08:52:58.539 INFO:teuthology.orchestra.run.smithi059.stdout: "old_weight": 0, 2024-05-08T08:52:58.539 INFO:teuthology.orchestra.run.smithi059.stdout: "last_purged_snaps_scrub": "2024-05-08T08:51:52.387606+0000", 2024-05-08T08:52:58.539 INFO:teuthology.orchestra.run.smithi059.stdout: "dead_epoch": 0 2024-05-08T08:52:58.539 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.539 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.539 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 3, 2024-05-08T08:52:58.539 INFO:teuthology.orchestra.run.smithi059.stdout: "down_stamp": "0.000000", 2024-05-08T08:52:58.539 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_probability": 0, 2024-05-08T08:52:58.540 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_interval": 0, 2024-05-08T08:52:58.540 INFO:teuthology.orchestra.run.smithi059.stdout: "features": 4540701547738038271, 2024-05-08T08:52:58.540 INFO:teuthology.orchestra.run.smithi059.stdout: "old_weight": 0, 2024-05-08T08:52:58.540 INFO:teuthology.orchestra.run.smithi059.stdout: "last_purged_snaps_scrub": "2024-05-08T08:51:52.360952+0000", 2024-05-08T08:52:58.540 INFO:teuthology.orchestra.run.smithi059.stdout: "dead_epoch": 0 2024-05-08T08:52:58.540 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.540 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.540 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 4, 2024-05-08T08:52:58.540 INFO:teuthology.orchestra.run.smithi059.stdout: "down_stamp": "0.000000", 2024-05-08T08:52:58.540 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_probability": 0, 2024-05-08T08:52:58.540 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_interval": 0, 2024-05-08T08:52:58.540 INFO:teuthology.orchestra.run.smithi059.stdout: "features": 4540701547738038271, 2024-05-08T08:52:58.540 INFO:teuthology.orchestra.run.smithi059.stdout: "old_weight": 65536, 2024-05-08T08:52:58.540 INFO:teuthology.orchestra.run.smithi059.stdout: "last_purged_snaps_scrub": "2024-05-08T08:51:52.468681+0000", 2024-05-08T08:52:58.540 INFO:teuthology.orchestra.run.smithi059.stdout: "dead_epoch": 0 2024-05-08T08:52:58.541 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.541 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.541 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 5, 2024-05-08T08:52:58.541 INFO:teuthology.orchestra.run.smithi059.stdout: "down_stamp": "0.000000", 2024-05-08T08:52:58.541 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_probability": 0, 2024-05-08T08:52:58.541 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_interval": 0, 2024-05-08T08:52:58.541 INFO:teuthology.orchestra.run.smithi059.stdout: "features": 4540701547738038271, 2024-05-08T08:52:58.541 INFO:teuthology.orchestra.run.smithi059.stdout: "old_weight": 0, 2024-05-08T08:52:58.541 INFO:teuthology.orchestra.run.smithi059.stdout: "last_purged_snaps_scrub": "2024-05-08T08:51:52.401080+0000", 2024-05-08T08:52:58.541 INFO:teuthology.orchestra.run.smithi059.stdout: "dead_epoch": 0 2024-05-08T08:52:58.541 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.541 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T08:52:58.541 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_upmap": [ 2024-05-08T08:52:58.541 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.541 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "3.17", 2024-05-08T08:52:58.541 INFO:teuthology.orchestra.run.smithi059.stdout: "osds": [ 2024-05-08T08:52:58.542 INFO:teuthology.orchestra.run.smithi059.stdout: 2, 2024-05-08T08:52:58.542 INFO:teuthology.orchestra.run.smithi059.stdout: 3 2024-05-08T08:52:58.542 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:58.542 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.542 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.542 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "3.1d", 2024-05-08T08:52:58.542 INFO:teuthology.orchestra.run.smithi059.stdout: "osds": [ 2024-05-08T08:52:58.542 INFO:teuthology.orchestra.run.smithi059.stdout: 0, 2024-05-08T08:52:58.542 INFO:teuthology.orchestra.run.smithi059.stdout: 5 2024-05-08T08:52:58.542 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:58.542 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.542 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T08:52:58.542 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_upmap_items": [], 2024-05-08T08:52:58.542 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_upmap_primaries": [], 2024-05-08T08:52:58.542 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_temp": [], 2024-05-08T08:52:58.543 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_temp": [], 2024-05-08T08:52:58.543 INFO:teuthology.orchestra.run.smithi059.stdout: "blocklist": {}, 2024-05-08T08:52:58.543 INFO:teuthology.orchestra.run.smithi059.stdout: "range_blocklist": {}, 2024-05-08T08:52:58.543 INFO:teuthology.orchestra.run.smithi059.stdout: "erasure_code_profiles": { 2024-05-08T08:52:58.543 INFO:teuthology.orchestra.run.smithi059.stdout: "default": { 2024-05-08T08:52:58.543 INFO:teuthology.orchestra.run.smithi059.stdout: "crush-failure-domain": "osd", 2024-05-08T08:52:58.543 INFO:teuthology.orchestra.run.smithi059.stdout: "k": "2", 2024-05-08T08:52:58.543 INFO:teuthology.orchestra.run.smithi059.stdout: "m": "1", 2024-05-08T08:52:58.543 INFO:teuthology.orchestra.run.smithi059.stdout: "plugin": "jerasure", 2024-05-08T08:52:58.543 INFO:teuthology.orchestra.run.smithi059.stdout: "technique": "reed_sol_van" 2024-05-08T08:52:58.543 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.543 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.543 INFO:teuthology.orchestra.run.smithi059.stdout: "removed_snaps_queue": [], 2024-05-08T08:52:58.543 INFO:teuthology.orchestra.run.smithi059.stdout: "new_removed_snaps": [], 2024-05-08T08:52:58.543 INFO:teuthology.orchestra.run.smithi059.stdout: "new_purged_snaps": [], 2024-05-08T08:52:58.544 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_node_flags": {}, 2024-05-08T08:52:58.544 INFO:teuthology.orchestra.run.smithi059.stdout: "device_class_flags": {}, 2024-05-08T08:52:58.544 INFO:teuthology.orchestra.run.smithi059.stdout: "stretch_mode": { 2024-05-08T08:52:58.544 INFO:teuthology.orchestra.run.smithi059.stdout: "stretch_mode_enabled": false, 2024-05-08T08:52:58.544 INFO:teuthology.orchestra.run.smithi059.stdout: "stretch_bucket_count": 0, 2024-05-08T08:52:58.544 INFO:teuthology.orchestra.run.smithi059.stdout: "degraded_stretch_mode": 0, 2024-05-08T08:52:58.544 INFO:teuthology.orchestra.run.smithi059.stdout: "recovering_stretch_mode": 0, 2024-05-08T08:52:58.544 INFO:teuthology.orchestra.run.smithi059.stdout: "stretch_mode_bucket": 0 2024-05-08T08:52:58.544 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.544 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:52:58.546 INFO:tasks.thrashosds.thrasher:j is {'epoch': 52, 'fsid': '38dfae8c-5c5e-4b0f-a264-91f22c6c5554', 'created': '2024-05-08T08:51:45.571369+0000', 'modified': '2024-05-08T08:52:58.118482+0000', 'last_up_change': '2024-05-08T08:51:56.847706+0000', 'last_in_change': '2024-05-08T08:52:46.045870+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T08:51:57.787790+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '14', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.8781916499137878, 'average_primary_affinity': 0.8799999952316284, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-05-08T08:52:02.563677+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.2699999809265137, 'score_stable': 2.2699999809265137, 'optimal_score': 0.8299999833106995, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.8781916499137878, 'average_primary_affinity': 0.8799999952316284, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-05-08T08:52:09.955467+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '52', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 16, 'snap_epoch': 52, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.7000000476837158, 'score_stable': 1.7000000476837158, 'optimal_score': 0.8299999833106995, 'raw_score_acting': 1.409999966621399, 'raw_score_stable': 1.409999966621399, 'primary_affinity_weighted': 0.8781916499137878, 'average_primary_affinity': 0.8799999952316284, 'average_primary_affinity_weighted': 0.8799999952316284}}], 'osds': [{'osd': 0, 'uuid': '300801a5-1919-4b23-8c6d-bc7dad61cc4d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 49, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6800', 'nonce': 802160151}, {'type': 'v1', 'addr': '172.21.15.59:6801', 'nonce': 802160151}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6802', 'nonce': 802160151}, {'type': 'v1', 'addr': '172.21.15.59:6803', 'nonce': 802160151}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6806', 'nonce': 802160151}, {'type': 'v1', 'addr': '172.21.15.59:6807', 'nonce': 802160151}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6804', 'nonce': 802160151}, {'type': 'v1', 'addr': '172.21.15.59:6805', 'nonce': 802160151}]}, 'public_addr': '172.21.15.59:6801/802160151', 'cluster_addr': '172.21.15.59:6803/802160151', 'heartbeat_back_addr': '172.21.15.59:6807/802160151', 'heartbeat_front_addr': '172.21.15.59:6805/802160151', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '0fead0de-6a65-4eba-b400-eea5692702e0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 44, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6816', 'nonce': 4116801011}, {'type': 'v1', 'addr': '172.21.15.59:6817', 'nonce': 4116801011}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6818', 'nonce': 4116801011}, {'type': 'v1', 'addr': '172.21.15.59:6819', 'nonce': 4116801011}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6822', 'nonce': 4116801011}, {'type': 'v1', 'addr': '172.21.15.59:6823', 'nonce': 4116801011}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6820', 'nonce': 4116801011}, {'type': 'v1', 'addr': '172.21.15.59:6821', 'nonce': 4116801011}]}, 'public_addr': '172.21.15.59:6817/4116801011', 'cluster_addr': '172.21.15.59:6819/4116801011', 'heartbeat_back_addr': '172.21.15.59:6823/4116801011', 'heartbeat_front_addr': '172.21.15.59:6821/4116801011', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '1a233de6-1b82-4a51-8e96-4cd4abdf16c8', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 50, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6808', 'nonce': 1547661179}, {'type': 'v1', 'addr': '172.21.15.59:6809', 'nonce': 1547661179}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6810', 'nonce': 1547661179}, {'type': 'v1', 'addr': '172.21.15.59:6811', 'nonce': 1547661179}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6814', 'nonce': 1547661179}, {'type': 'v1', 'addr': '172.21.15.59:6815', 'nonce': 1547661179}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6812', 'nonce': 1547661179}, {'type': 'v1', 'addr': '172.21.15.59:6813', 'nonce': 1547661179}]}, 'public_addr': '172.21.15.59:6809/1547661179', 'cluster_addr': '172.21.15.59:6811/1547661179', 'heartbeat_back_addr': '172.21.15.59:6815/1547661179', 'heartbeat_front_addr': '172.21.15.59:6813/1547661179', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'cc4870b8-dcd9-4e43-978d-8de6c6396abf', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 44, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6800', 'nonce': 4054400810}, {'type': 'v1', 'addr': '172.21.15.192:6801', 'nonce': 4054400810}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6802', 'nonce': 4054400810}, {'type': 'v1', 'addr': '172.21.15.192:6803', 'nonce': 4054400810}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6806', 'nonce': 4054400810}, {'type': 'v1', 'addr': '172.21.15.192:6807', 'nonce': 4054400810}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6804', 'nonce': 4054400810}, {'type': 'v1', 'addr': '172.21.15.192:6805', 'nonce': 4054400810}]}, 'public_addr': '172.21.15.192:6801/4054400810', 'cluster_addr': '172.21.15.192:6803/4054400810', 'heartbeat_back_addr': '172.21.15.192:6807/4054400810', 'heartbeat_front_addr': '172.21.15.192:6805/4054400810', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'd1d05620-42fb-4107-a2f9-e83542d076ee', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 0.9892578125, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 43, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6808', 'nonce': 58985565}, {'type': 'v1', 'addr': '172.21.15.192:6809', 'nonce': 58985565}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6810', 'nonce': 58985565}, {'type': 'v1', 'addr': '172.21.15.192:6811', 'nonce': 58985565}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6814', 'nonce': 58985565}, {'type': 'v1', 'addr': '172.21.15.192:6815', 'nonce': 58985565}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6812', 'nonce': 58985565}, {'type': 'v1', 'addr': '172.21.15.192:6813', 'nonce': 58985565}]}, 'public_addr': '172.21.15.192:6809/58985565', 'cluster_addr': '172.21.15.192:6811/58985565', 'heartbeat_back_addr': '172.21.15.192:6815/58985565', 'heartbeat_front_addr': '172.21.15.192:6813/58985565', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'ab26136c-31c3-4090-9fa2-988dba4f731f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.2798919677734375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 18, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6816', 'nonce': 1283571573}, {'type': 'v1', 'addr': '172.21.15.192:6817', 'nonce': 1283571573}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6818', 'nonce': 1283571573}, {'type': 'v1', 'addr': '172.21.15.192:6819', 'nonce': 1283571573}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6822', 'nonce': 1283571573}, {'type': 'v1', 'addr': '172.21.15.192:6823', 'nonce': 1283571573}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6820', 'nonce': 1283571573}, {'type': 'v1', 'addr': '172.21.15.192:6821', 'nonce': 1283571573}]}, 'public_addr': '172.21.15.192:6817/1283571573', 'cluster_addr': '172.21.15.192:6819/1283571573', 'heartbeat_back_addr': '172.21.15.192:6823/1283571573', 'heartbeat_front_addr': '172.21.15.192:6821/1283571573', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:51:52.396946+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:51:52.453160+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:51:52.387606+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:51:52.360952+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-05-08T08:51:52.468681+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:51:52.401080+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.17', 'osds': [2, 3]}, {'pgid': '3.1d', 'osds': [0, 5]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-05-08T08:52:58.546 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T08:52:58.587 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:52:58.590+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:52:58.655 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:52:58.655 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:52:58.655 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:52:58.655 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:52:58.664 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:52:58.681 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:58.684+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:58.781 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:52:58.785+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:52:58.783 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:52:58.783 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:52:58.784 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:52:58.784 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:52:58.784 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:52:58.793 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:52:58.882 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:52:58.885+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:52:58.896 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:52:58.897 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-05-08T08:52:58.908 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":77,"stamp":"2024-05-08T08:52:58.122862+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":551583616,"num_objects":1410,"num_object_clones":208,"num_object_copies":2820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1410,"num_whiteouts":937,"num_read":148049,"num_read_kb":496004,"num_write":9035,"num_write_kb":197644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":637,"num_bytes_recovered":27285362,"num_keys_recovered":109,"num_objects_omap":16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8373,"ondisk_log_size":8373,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":83,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":471859200,"kb_used":206828,"kb_used_data":30128,"kb_used_omap":7,"kb_used_meta":176632,"kb_avail":471652372,"statfs":{"total":483183820800,"available":482972028928,"internally_reserved":0,"allocated":30851072,"data_stored":231094573,"data_compressed":10645902,"data_compressed_allocated":20578304,"data_compressed_original":224974336,"omap_allocated":7973,"internal_metadata":180871387},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":60146298,"num_objects":130,"num_object_clones":47,"num_object_copies":260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":130,"num_whiteouts":33,"num_read":8615,"num_read_kb":49905,"num_write":1050,"num_write_kb":20470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":607329,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.023936"},"pg_stats":[{"pgid":"3.1f","version":"51'208","reported_seq":12540,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-08T08:52:57.663207+0000","last_change":"2024-05-08T08:52:13.934602+0000","last_active":"2024-05-08T08:52:57.663207+0000","last_peered":"2024-05-08T08:52:57.663207+0000","last_clean":"2024-05-08T08:52:57.663207+0000","last_became_active":"2024-05-08T08:52:11.931332+0000","last_became_peered":"2024-05-08T08:52:11.931332+0000","last_unstale":"2024-05-08T08:52:57.663207+0000","last_undegraded":"2024-05-08T08:52:57.663207+0000","last_fullsized":"2024-05-08T08:52:57.663207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:39:40.980675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000691565,"stat_sum":{"num_bytes":46172215,"num_objects":48,"num_object_clones":10,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":28,"num_read":11206,"num_read_kb":52686,"num_write":301,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"51'193","reported_seq":1413,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-08T08:52:57.752738+0000","last_change":"2024-05-08T08:52:13.944947+0000","last_active":"2024-05-08T08:52:57.752738+0000","last_peered":"2024-05-08T08:52:57.752738+0000","last_clean":"2024-05-08T08:52:57.752738+0000","last_became_active":"2024-05-08T08:52:11.930844+0000","last_became_peered":"2024-05-08T08:52:11.930844+0000","last_unstale":"2024-05-08T08:52:57.752738+0000","last_undegraded":"2024-05-08T08:52:57.752738+0000","last_fullsized":"2024-05-08T08:52:57.752738+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":193,"log_dups_size":0,"ondisk_log_size":193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:48:01.884805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040424900000000003,"stat_sum":{"num_bytes":7480320,"num_objects":48,"num_object_clones":6,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":35,"num_read":1354,"num_read_kb":4273,"num_write":138,"num_write_kb":4043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"51'116","reported_seq":418,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-08T08:52:57.760921+0000","last_change":"2024-05-08T08:52:36.057025+0000","last_active":"2024-05-08T08:52:57.760921+0000","last_peered":"2024-05-08T08:52:57.760921+0000","last_clean":"2024-05-08T08:52:57.760921+0000","last_became_active":"2024-05-08T08:52:36.024295+0000","last_became_peered":"2024-05-08T08:52:36.024295+0000","last_unstale":"2024-05-08T08:52:57.760921+0000","last_undegraded":"2024-05-08T08:52:57.760921+0000","last_fullsized":"2024-05-08T08:52:57.760921+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:50:58.179634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072111799999999998,"stat_sum":{"num_bytes":6808143,"num_objects":36,"num_object_clones":2,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":21,"num_read":3269,"num_read_kb":10614,"num_write":102,"num_write_kb":3391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":173135,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"51'231","reported_seq":7288,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-08T08:52:57.660878+0000","last_change":"2024-05-08T08:52:50.070942+0000","last_active":"2024-05-08T08:52:57.660878+0000","last_peered":"2024-05-08T08:52:57.660878+0000","last_clean":"2024-05-08T08:52:57.660878+0000","last_became_active":"2024-05-08T08:52:50.069533+0000","last_became_peered":"2024-05-08T08:52:50.069533+0000","last_unstale":"2024-05-08T08:52:57.660878+0000","last_undegraded":"2024-05-08T08:52:57.660878+0000","last_fullsized":"2024-05-08T08:52:57.660878+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":231,"log_dups_size":0,"ondisk_log_size":231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:39:58.535337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094844199999999997,"stat_sum":{"num_bytes":38824997,"num_objects":48,"num_object_clones":6,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":33,"num_read":7035,"num_read_kb":47767,"num_write":328,"num_write_kb":18949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":13917221,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"51'249","reported_seq":7764,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-08T08:52:57.729939+0000","last_change":"2024-05-08T08:52:50.069794+0000","last_active":"2024-05-08T08:52:57.729939+0000","last_peered":"2024-05-08T08:52:57.729939+0000","last_clean":"2024-05-08T08:52:57.729939+0000","last_became_active":"2024-05-08T08:52:50.069306+0000","last_became_peered":"2024-05-08T08:52:50.069306+0000","last_unstale":"2024-05-08T08:52:57.729939+0000","last_undegraded":"2024-05-08T08:52:57.729939+0000","last_fullsized":"2024-05-08T08:52:57.729939+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:59:10.960083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000685079,"stat_sum":{"num_bytes":26357292,"num_objects":50,"num_object_clones":7,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":38,"num_read":7995,"num_read_kb":13174,"num_write":314,"num_write_kb":18022,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":146988,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"51'251","reported_seq":6976,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-08T08:52:57.738227+0000","last_change":"2024-05-08T08:52:50.072642+0000","last_active":"2024-05-08T08:52:57.738227+0000","last_peered":"2024-05-08T08:52:57.738227+0000","last_clean":"2024-05-08T08:52:57.738227+0000","last_became_active":"2024-05-08T08:52:50.072082+0000","last_became_peered":"2024-05-08T08:52:50.072082+0000","last_unstale":"2024-05-08T08:52:57.738227+0000","last_undegraded":"2024-05-08T08:52:57.738227+0000","last_fullsized":"2024-05-08T08:52:57.738227+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":251,"log_dups_size":0,"ondisk_log_size":251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:35:14.361761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066525600000000003,"stat_sum":{"num_bytes":9804353,"num_objects":48,"num_object_clones":7,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":30,"num_read":7268,"num_read_kb":13326,"num_write":262,"num_write_kb":3060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":1862193,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"51'170","reported_seq":1266,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-08T08:52:57.677449+0000","last_change":"2024-05-08T08:52:50.069911+0000","last_active":"2024-05-08T08:52:57.677449+0000","last_peered":"2024-05-08T08:52:57.677449+0000","last_clean":"2024-05-08T08:52:57.677449+0000","last_became_active":"2024-05-08T08:52:50.069328+0000","last_became_peered":"2024-05-08T08:52:50.069328+0000","last_unstale":"2024-05-08T08:52:57.677449+0000","last_undegraded":"2024-05-08T08:52:57.677449+0000","last_fullsized":"2024-05-08T08:52:57.677449+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:18:04.037069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082481000000000004,"stat_sum":{"num_bytes":5234176,"num_objects":45,"num_object_clones":6,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":35,"num_read":919,"num_read_kb":9602,"num_write":134,"num_write_kb":5779,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":1896960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"49'264","reported_seq":4331,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-08T08:52:57.291760+0000","last_change":"2024-05-08T08:52:13.945184+0000","last_active":"2024-05-08T08:52:57.291760+0000","last_peered":"2024-05-08T08:52:57.291760+0000","last_clean":"2024-05-08T08:52:57.291760+0000","last_became_active":"2024-05-08T08:52:11.927366+0000","last_became_peered":"2024-05-08T08:52:11.927366+0000","last_unstale":"2024-05-08T08:52:57.291760+0000","last_undegraded":"2024-05-08T08:52:57.291760+0000","last_fullsized":"2024-05-08T08:52:57.291760+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":264,"log_dups_size":0,"ondisk_log_size":264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:21:56.653689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057208900000000002,"stat_sum":{"num_bytes":58095652,"num_objects":50,"num_object_clones":10,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":34,"num_read":4584,"num_read_kb":17820,"num_write":310,"num_write_kb":15242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"51'540","reported_seq":2118,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-08T08:52:57.755950+0000","last_change":"2024-05-08T08:52:57.123662+0000","last_active":"2024-05-08T08:52:57.755950+0000","last_peered":"2024-05-08T08:52:57.755950+0000","last_clean":"2024-05-08T08:52:57.755950+0000","last_became_active":"2024-05-08T08:52:57.122685+0000","last_became_peered":"2024-05-08T08:52:57.122685+0000","last_unstale":"2024-05-08T08:52:57.755950+0000","last_undegraded":"2024-05-08T08:52:57.755950+0000","last_fullsized":"2024-05-08T08:52:57.755950+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":540,"log_dups_size":0,"ondisk_log_size":540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:38:50.133955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000866371,"stat_sum":{"num_bytes":2926689,"num_objects":33,"num_object_clones":5,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":20,"num_read":4105,"num_read_kb":6101,"num_write":682,"num_write_kb":2903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":607329,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.16","version":"51'268","reported_seq":12038,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-08T08:52:57.633666+0000","last_change":"2024-05-08T08:52:13.932215+0000","last_active":"2024-05-08T08:52:57.633666+0000","last_peered":"2024-05-08T08:52:57.633666+0000","last_clean":"2024-05-08T08:52:57.633666+0000","last_became_active":"2024-05-08T08:52:11.931213+0000","last_became_peered":"2024-05-08T08:52:11.931213+0000","last_unstale":"2024-05-08T08:52:57.633666+0000","last_undegraded":"2024-05-08T08:52:57.633666+0000","last_fullsized":"2024-05-08T08:52:57.633666+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":268,"log_dups_size":0,"ondisk_log_size":268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:26:06.315005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084652399999999997,"stat_sum":{"num_bytes":26086079,"num_objects":39,"num_object_clones":6,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":22,"num_read":12183,"num_read_kb":45117,"num_write":373,"num_write_kb":13131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"51'269","reported_seq":3717,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-08T08:52:57.608678+0000","last_change":"2024-05-08T08:52:13.944999+0000","last_active":"2024-05-08T08:52:57.608678+0000","last_peered":"2024-05-08T08:52:57.608678+0000","last_clean":"2024-05-08T08:52:57.608678+0000","last_became_active":"2024-05-08T08:52:11.931695+0000","last_became_peered":"2024-05-08T08:52:11.931695+0000","last_unstale":"2024-05-08T08:52:57.608678+0000","last_undegraded":"2024-05-08T08:52:57.608678+0000","last_fullsized":"2024-05-08T08:52:57.608678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":269,"log_dups_size":0,"ondisk_log_size":269,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:58:06.324563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063902800000000004,"stat_sum":{"num_bytes":10027046,"num_objects":38,"num_object_clones":5,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":28,"num_read":4018,"num_read_kb":7890,"num_write":335,"num_write_kb":967,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"51'249","reported_seq":4101,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-08T08:52:57.731624+0000","last_change":"2024-05-08T08:52:50.068267+0000","last_active":"2024-05-08T08:52:57.731624+0000","last_peered":"2024-05-08T08:52:57.731624+0000","last_clean":"2024-05-08T08:52:57.731624+0000","last_became_active":"2024-05-08T08:52:50.067650+0000","last_became_peered":"2024-05-08T08:52:50.067650+0000","last_unstale":"2024-05-08T08:52:57.731624+0000","last_undegraded":"2024-05-08T08:52:57.731624+0000","last_fullsized":"2024-05-08T08:52:57.731624+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:08:07.260593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000430331,"stat_sum":{"num_bytes":20665936,"num_objects":46,"num_object_clones":10,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":28,"num_read":1665,"num_read_kb":6939,"num_write":190,"num_write_kb":4722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":593936,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-08T08:52:57.121524+0000","last_change":"2024-05-08T08:52:47.057420+0000","last_active":"2024-05-08T08:52:57.121524+0000","last_peered":"2024-05-08T08:52:57.121524+0000","last_clean":"2024-05-08T08:52:57.121524+0000","last_became_active":"2024-05-08T08:52:47.057207+0000","last_became_peered":"2024-05-08T08:52:47.057207+0000","last_unstale":"2024-05-08T08:52:57.121524+0000","last_undegraded":"2024-05-08T08:52:57.121524+0000","last_fullsized":"2024-05-08T08:52:57.121524+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:47:42.112090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"51'207","reported_seq":9929,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-08T08:52:57.750870+0000","last_change":"2024-05-08T08:52:13.934894+0000","last_active":"2024-05-08T08:52:57.750870+0000","last_peered":"2024-05-08T08:52:57.750870+0000","last_clean":"2024-05-08T08:52:57.750870+0000","last_became_active":"2024-05-08T08:52:11.929184+0000","last_became_peered":"2024-05-08T08:52:11.929184+0000","last_unstale":"2024-05-08T08:52:57.750870+0000","last_undegraded":"2024-05-08T08:52:57.750870+0000","last_fullsized":"2024-05-08T08:52:57.750870+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":207,"log_dups_size":0,"ondisk_log_size":207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:46:30.248165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088867899999999997,"stat_sum":{"num_bytes":26354688,"num_objects":38,"num_object_clones":4,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":23,"num_read":10559,"num_read_kb":47588,"num_write":203,"num_write_kb":9690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":80,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-08T08:52:57.126685+0000","last_change":"2024-05-08T08:52:03.896159+0000","last_active":"2024-05-08T08:52:57.126685+0000","last_peered":"2024-05-08T08:52:57.126685+0000","last_clean":"2024-05-08T08:52:57.126685+0000","last_became_active":"2024-05-08T08:52:03.895797+0000","last_became_peered":"2024-05-08T08:52:03.895797+0000","last_unstale":"2024-05-08T08:52:57.126685+0000","last_undegraded":"2024-05-08T08:52:57.126685+0000","last_fullsized":"2024-05-08T08:52:57.126685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:34:33.513525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"51'143","reported_seq":1878,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-08T08:52:57.729888+0000","last_change":"2024-05-08T08:52:13.944951+0000","last_active":"2024-05-08T08:52:57.729888+0000","last_peered":"2024-05-08T08:52:57.729888+0000","last_clean":"2024-05-08T08:52:57.729888+0000","last_became_active":"2024-05-08T08:52:11.926905+0000","last_became_peered":"2024-05-08T08:52:11.926905+0000","last_unstale":"2024-05-08T08:52:57.729888+0000","last_undegraded":"2024-05-08T08:52:57.729888+0000","last_fullsized":"2024-05-08T08:52:57.729888+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:27:43.077591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044527,"stat_sum":{"num_bytes":8833170,"num_objects":52,"num_object_clones":12,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":28,"num_read":1753,"num_read_kb":18863,"num_write":132,"num_write_kb":6376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":80,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-08T08:52:57.121418+0000","last_change":"2024-05-08T08:52:04.144764+0000","last_active":"2024-05-08T08:52:57.121418+0000","last_peered":"2024-05-08T08:52:57.121418+0000","last_clean":"2024-05-08T08:52:57.121418+0000","last_became_active":"2024-05-08T08:52:04.144444+0000","last_became_peered":"2024-05-08T08:52:04.144444+0000","last_unstale":"2024-05-08T08:52:57.121418+0000","last_undegraded":"2024-05-08T08:52:57.121418+0000","last_fullsized":"2024-05-08T08:52:57.121418+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:58:31.498922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"51'229","reported_seq":4128,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-08T08:52:57.741081+0000","last_change":"2024-05-08T08:52:13.940976+0000","last_active":"2024-05-08T08:52:57.741081+0000","last_peered":"2024-05-08T08:52:57.741081+0000","last_clean":"2024-05-08T08:52:57.741081+0000","last_became_active":"2024-05-08T08:52:11.926402+0000","last_became_peered":"2024-05-08T08:52:11.926402+0000","last_unstale":"2024-05-08T08:52:57.741081+0000","last_undegraded":"2024-05-08T08:52:57.741081+0000","last_fullsized":"2024-05-08T08:52:57.741081+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":229,"log_dups_size":0,"ondisk_log_size":229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:22:35.720720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000669351,"stat_sum":{"num_bytes":5825024,"num_objects":48,"num_object_clones":6,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":35,"num_read":4626,"num_read_kb":8016,"num_write":191,"num_write_kb":3481,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":80,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-08T08:52:57.126535+0000","last_change":"2024-05-08T08:52:04.144543+0000","last_active":"2024-05-08T08:52:57.126535+0000","last_peered":"2024-05-08T08:52:57.126535+0000","last_clean":"2024-05-08T08:52:57.126535+0000","last_became_active":"2024-05-08T08:52:04.144242+0000","last_became_peered":"2024-05-08T08:52:04.144242+0000","last_unstale":"2024-05-08T08:52:57.126535+0000","last_undegraded":"2024-05-08T08:52:57.126535+0000","last_fullsized":"2024-05-08T08:52:57.126535+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:57:36.844633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"51'542","reported_seq":4183,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-08T08:52:57.560976+0000","last_change":"2024-05-08T08:52:23.967224+0000","last_active":"2024-05-08T08:52:57.560976+0000","last_peered":"2024-05-08T08:52:57.560976+0000","last_clean":"2024-05-08T08:52:57.560976+0000","last_became_active":"2024-05-08T08:52:23.966596+0000","last_became_peered":"2024-05-08T08:52:23.966596+0000","last_unstale":"2024-05-08T08:52:57.560976+0000","last_undegraded":"2024-05-08T08:52:57.560976+0000","last_fullsized":"2024-05-08T08:52:57.560976+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":542,"log_dups_size":0,"ondisk_log_size":542,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:16:44.024259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064078099999999999,"stat_sum":{"num_bytes":6713531,"num_objects":56,"num_object_clones":6,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":43,"num_read":5650,"num_read_kb":8788,"num_write":816,"num_write_kb":3234,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"51'148","reported_seq":806,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-08T08:52:57.750868+0000","last_change":"2024-05-08T08:52:50.070065+0000","last_active":"2024-05-08T08:52:57.750868+0000","last_peered":"2024-05-08T08:52:57.750868+0000","last_clean":"2024-05-08T08:52:57.750868+0000","last_became_active":"2024-05-08T08:52:50.069576+0000","last_became_peered":"2024-05-08T08:52:50.069576+0000","last_unstale":"2024-05-08T08:52:57.750868+0000","last_undegraded":"2024-05-08T08:52:57.750868+0000","last_fullsized":"2024-05-08T08:52:57.750868+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:12:54.645531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047037799999999997,"stat_sum":{"num_bytes":6234112,"num_objects":30,"num_object_clones":4,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":21,"num_read":523,"num_read_kb":5826,"num_write":129,"num_write_kb":2458,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":2101248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":55,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-08T08:52:57.126279+0000","last_change":"2024-05-08T08:52:23.969093+0000","last_active":"2024-05-08T08:52:57.126279+0000","last_peered":"2024-05-08T08:52:57.126279+0000","last_clean":"2024-05-08T08:52:57.126279+0000","last_became_active":"2024-05-08T08:52:23.968782+0000","last_became_peered":"2024-05-08T08:52:23.968782+0000","last_unstale":"2024-05-08T08:52:57.126279+0000","last_undegraded":"2024-05-08T08:52:57.126279+0000","last_fullsized":"2024-05-08T08:52:57.126279+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:37:41.693728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"51'151","reported_seq":755,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-08T08:52:57.743253+0000","last_change":"2024-05-08T08:52:13.940504+0000","last_active":"2024-05-08T08:52:57.743253+0000","last_peered":"2024-05-08T08:52:57.743253+0000","last_clean":"2024-05-08T08:52:57.743253+0000","last_became_active":"2024-05-08T08:52:11.928301+0000","last_became_peered":"2024-05-08T08:52:11.928301+0000","last_unstale":"2024-05-08T08:52:57.743253+0000","last_undegraded":"2024-05-08T08:52:57.743253+0000","last_fullsized":"2024-05-08T08:52:57.743253+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:23:02.850455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046631800000000002,"stat_sum":{"num_bytes":24315972,"num_objects":44,"num_object_clones":6,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":30,"num_read":590,"num_read_kb":4842,"num_write":127,"num_write_kb":7312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":80,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-08T08:52:57.121775+0000","last_change":"2024-05-08T08:52:03.891768+0000","last_active":"2024-05-08T08:52:57.121775+0000","last_peered":"2024-05-08T08:52:57.121775+0000","last_clean":"2024-05-08T08:52:57.121775+0000","last_became_active":"2024-05-08T08:52:03.891374+0000","last_became_peered":"2024-05-08T08:52:03.891374+0000","last_unstale":"2024-05-08T08:52:57.121775+0000","last_undegraded":"2024-05-08T08:52:57.121775+0000","last_fullsized":"2024-05-08T08:52:57.121775+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:18:10.240492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"51'481","reported_seq":14229,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-08T08:52:57.530506+0000","last_change":"2024-05-08T08:52:50.071097+0000","last_active":"2024-05-08T08:52:57.530506+0000","last_peered":"2024-05-08T08:52:57.530506+0000","last_clean":"2024-05-08T08:52:57.530506+0000","last_became_active":"2024-05-08T08:52:50.070396+0000","last_became_peered":"2024-05-08T08:52:50.070396+0000","last_unstale":"2024-05-08T08:52:57.530506+0000","last_undegraded":"2024-05-08T08:52:57.530506+0000","last_fullsized":"2024-05-08T08:52:57.530506+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":481,"log_dups_size":0,"ondisk_log_size":481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:21:47.141027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046440499999999998,"stat_sum":{"num_bytes":26714672,"num_objects":42,"num_object_clones":8,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":26,"num_read":15037,"num_read_kb":19720,"num_write":286,"num_write_kb":3475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":1271856,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":80,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-08T08:52:57.121158+0000","last_change":"2024-05-08T08:52:03.891551+0000","last_active":"2024-05-08T08:52:57.121158+0000","last_peered":"2024-05-08T08:52:57.121158+0000","last_clean":"2024-05-08T08:52:57.121158+0000","last_became_active":"2024-05-08T08:52:03.891296+0000","last_became_peered":"2024-05-08T08:52:03.891296+0000","last_unstale":"2024-05-08T08:52:57.121158+0000","last_undegraded":"2024-05-08T08:52:57.121158+0000","last_fullsized":"2024-05-08T08:52:57.121158+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:54.806343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"51'122","reported_seq":554,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-08T08:52:57.731189+0000","last_change":"2024-05-08T08:52:13.940743+0000","last_active":"2024-05-08T08:52:57.731189+0000","last_peered":"2024-05-08T08:52:57.731189+0000","last_clean":"2024-05-08T08:52:57.731189+0000","last_became_active":"2024-05-08T08:52:11.928273+0000","last_became_peered":"2024-05-08T08:52:11.928273+0000","last_unstale":"2024-05-08T08:52:57.731189+0000","last_undegraded":"2024-05-08T08:52:57.731189+0000","last_fullsized":"2024-05-08T08:52:57.731189+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:45:36.030620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062135599999999999,"stat_sum":{"num_bytes":7776370,"num_objects":46,"num_object_clones":9,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":30,"num_read":421,"num_read_kb":3623,"num_write":96,"num_write_kb":1142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":55,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-08T08:52:57.121591+0000","last_change":"2024-05-08T08:52:23.966794+0000","last_active":"2024-05-08T08:52:57.121591+0000","last_peered":"2024-05-08T08:52:57.121591+0000","last_clean":"2024-05-08T08:52:57.121591+0000","last_became_active":"2024-05-08T08:52:23.966544+0000","last_became_peered":"2024-05-08T08:52:23.966544+0000","last_unstale":"2024-05-08T08:52:57.121591+0000","last_undegraded":"2024-05-08T08:52:57.121591+0000","last_fullsized":"2024-05-08T08:52:57.121591+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:25:56.927420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":236,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-08T08:52:57.121168+0000","last_change":"2024-05-08T08:51:58.861899+0000","last_active":"2024-05-08T08:52:57.121168+0000","last_peered":"2024-05-08T08:52:57.121168+0000","last_clean":"2024-05-08T08:52:57.121168+0000","last_became_active":"2024-05-08T08:51:58.861596+0000","last_became_peered":"2024-05-08T08:51:58.861596+0000","last_unstale":"2024-05-08T08:52:57.121168+0000","last_undegraded":"2024-05-08T08:52:57.121168+0000","last_fullsized":"2024-05-08T08:52:57.121168+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:51:57.849666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:51:57.849666+0000","last_clean_scrub_stamp":"2024-05-08T08:51:57.849666+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:53:21.739950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"51'120","reported_seq":872,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-08T08:52:57.273324+0000","last_change":"2024-05-08T08:52:13.945362+0000","last_active":"2024-05-08T08:52:57.273324+0000","last_peered":"2024-05-08T08:52:57.273324+0000","last_clean":"2024-05-08T08:52:57.273324+0000","last_became_active":"2024-05-08T08:52:11.926975+0000","last_became_peered":"2024-05-08T08:52:11.926975+0000","last_unstale":"2024-05-08T08:52:57.273324+0000","last_undegraded":"2024-05-08T08:52:57.273324+0000","last_fullsized":"2024-05-08T08:52:57.273324+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:02:56.219529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068218599999999999,"stat_sum":{"num_bytes":19044368,"num_objects":46,"num_object_clones":6,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":33,"num_read":715,"num_read_kb":4864,"num_write":87,"num_write_kb":2483,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"51'194","reported_seq":4683,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-08T08:52:57.563548+0000","last_change":"2024-05-08T08:52:50.069736+0000","last_active":"2024-05-08T08:52:57.563548+0000","last_peered":"2024-05-08T08:52:57.563548+0000","last_clean":"2024-05-08T08:52:57.563548+0000","last_became_active":"2024-05-08T08:52:50.069112+0000","last_became_peered":"2024-05-08T08:52:50.069112+0000","last_unstale":"2024-05-08T08:52:57.563548+0000","last_undegraded":"2024-05-08T08:52:57.563548+0000","last_fullsized":"2024-05-08T08:52:57.563548+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":194,"log_dups_size":0,"ondisk_log_size":194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:44:55.434855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00030754000000000003,"stat_sum":{"num_bytes":6563328,"num_objects":40,"num_object_clones":6,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":28,"num_read":4813,"num_read_kb":11006,"num_write":145,"num_write_kb":2310,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":1878528,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"51'452","reported_seq":7871,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-08T08:52:57.729887+0000","last_change":"2024-05-08T08:52:50.072624+0000","last_active":"2024-05-08T08:52:57.729887+0000","last_peered":"2024-05-08T08:52:57.729887+0000","last_clean":"2024-05-08T08:52:57.729887+0000","last_became_active":"2024-05-08T08:52:50.071883+0000","last_became_peered":"2024-05-08T08:52:50.071883+0000","last_unstale":"2024-05-08T08:52:57.729887+0000","last_undegraded":"2024-05-08T08:52:57.729887+0000","last_fullsized":"2024-05-08T08:52:57.729887+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":452,"log_dups_size":0,"ondisk_log_size":452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:55:37.705068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070350100000000004,"stat_sum":{"num_bytes":15769748,"num_objects":46,"num_object_clones":5,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":30,"num_read":10526,"num_read_kb":15321,"num_write":305,"num_write_kb":2551,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":54,"num_bytes_recovered":426496,"num_keys_recovered":28,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"51'252","reported_seq":2230,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-08T08:52:57.755038+0000","last_change":"2024-05-08T08:52:13.940600+0000","last_active":"2024-05-08T08:52:57.755038+0000","last_peered":"2024-05-08T08:52:57.755038+0000","last_clean":"2024-05-08T08:52:57.755038+0000","last_became_active":"2024-05-08T08:52:11.927196+0000","last_became_peered":"2024-05-08T08:52:11.927196+0000","last_unstale":"2024-05-08T08:52:57.755038+0000","last_undegraded":"2024-05-08T08:52:57.755038+0000","last_fullsized":"2024-05-08T08:52:57.755038+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":252,"log_dups_size":0,"ondisk_log_size":252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:18:24.470315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048296599999999998,"stat_sum":{"num_bytes":27166240,"num_objects":42,"num_object_clones":4,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":28,"num_read":2334,"num_read_kb":6340,"num_write":192,"num_write_kb":2017,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"51'116","reported_seq":542,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-08T08:52:57.664551+0000","last_change":"2024-05-08T08:52:23.969521+0000","last_active":"2024-05-08T08:52:57.664551+0000","last_peered":"2024-05-08T08:52:57.664551+0000","last_clean":"2024-05-08T08:52:57.664551+0000","last_became_active":"2024-05-08T08:52:23.969067+0000","last_became_peered":"2024-05-08T08:52:23.969067+0000","last_unstale":"2024-05-08T08:52:57.664551+0000","last_undegraded":"2024-05-08T08:52:57.664551+0000","last_fullsized":"2024-05-08T08:52:57.664551+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:45:20.047377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048793000000000003,"stat_sum":{"num_bytes":2731520,"num_objects":39,"num_object_clones":5,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":30,"num_read":378,"num_read_kb":2770,"num_write":89,"num_write_kb":932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"51'180","reported_seq":836,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-08T08:52:57.729785+0000","last_change":"2024-05-08T08:52:13.945141+0000","last_active":"2024-05-08T08:52:57.729785+0000","last_peered":"2024-05-08T08:52:57.729785+0000","last_clean":"2024-05-08T08:52:57.729785+0000","last_became_active":"2024-05-08T08:52:11.932594+0000","last_became_peered":"2024-05-08T08:52:11.932594+0000","last_unstale":"2024-05-08T08:52:57.729785+0000","last_undegraded":"2024-05-08T08:52:57.729785+0000","last_fullsized":"2024-05-08T08:52:57.729785+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:12:59.906297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074388199999999996,"stat_sum":{"num_bytes":3361792,"num_objects":46,"num_object_clones":6,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":34,"num_read":610,"num_read_kb":5928,"num_write":149,"num_write_kb":2213,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"51'123","reported_seq":302,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-08T08:52:57.659964+0000","last_change":"2024-05-08T08:52:47.137016+0000","last_active":"2024-05-08T08:52:57.659964+0000","last_peered":"2024-05-08T08:52:57.659964+0000","last_clean":"2024-05-08T08:52:57.659964+0000","last_became_active":"2024-05-08T08:52:47.071804+0000","last_became_peered":"2024-05-08T08:52:47.071804+0000","last_unstale":"2024-05-08T08:52:57.659964+0000","last_undegraded":"2024-05-08T08:52:57.659964+0000","last_fullsized":"2024-05-08T08:52:57.659964+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:54:29.464720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085079600000000002,"stat_sum":{"num_bytes":10514432,"num_objects":28,"num_object_clones":5,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":16,"num_read":5486,"num_read_kb":20232,"num_write":107,"num_write_kb":3740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":2343424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.f","version":"51'142","reported_seq":1974,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-08T08:52:57.563587+0000","last_change":"2024-05-08T08:52:13.932524+0000","last_active":"2024-05-08T08:52:57.563587+0000","last_peered":"2024-05-08T08:52:57.563587+0000","last_clean":"2024-05-08T08:52:57.563587+0000","last_became_active":"2024-05-08T08:52:11.931347+0000","last_became_peered":"2024-05-08T08:52:11.931347+0000","last_unstale":"2024-05-08T08:52:57.563587+0000","last_undegraded":"2024-05-08T08:52:57.563587+0000","last_fullsized":"2024-05-08T08:52:57.563587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":142,"log_dups_size":0,"ondisk_log_size":142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:04:47.939660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010326370000000001,"stat_sum":{"num_bytes":8210115,"num_objects":33,"num_object_clones":3,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":23,"num_read":2023,"num_read_kb":14491,"num_write":164,"num_write_kb":1465,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"51'148","reported_seq":2321,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-08T08:52:57.754810+0000","last_change":"2024-05-08T08:52:23.970138+0000","last_active":"2024-05-08T08:52:57.754810+0000","last_peered":"2024-05-08T08:52:57.754810+0000","last_clean":"2024-05-08T08:52:57.754810+0000","last_became_active":"2024-05-08T08:52:23.969732+0000","last_became_peered":"2024-05-08T08:52:23.969732+0000","last_unstale":"2024-05-08T08:52:57.754810+0000","last_undegraded":"2024-05-08T08:52:57.754810+0000","last_fullsized":"2024-05-08T08:52:57.754810+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:31:21.011701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053593,"stat_sum":{"num_bytes":17634816,"num_objects":45,"num_object_clones":7,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":32,"num_read":2135,"num_read_kb":18763,"num_write":118,"num_write_kb":9293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"51'166","reported_seq":702,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-08T08:52:57.731081+0000","last_change":"2024-05-08T08:52:50.071476+0000","last_active":"2024-05-08T08:52:57.731081+0000","last_peered":"2024-05-08T08:52:57.731081+0000","last_clean":"2024-05-08T08:52:57.731081+0000","last_became_active":"2024-05-08T08:52:50.070771+0000","last_became_peered":"2024-05-08T08:52:50.070771+0000","last_unstale":"2024-05-08T08:52:57.731081+0000","last_undegraded":"2024-05-08T08:52:57.731081+0000","last_fullsized":"2024-05-08T08:52:57.731081+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:23:17.474522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041230699999999999,"stat_sum":{"num_bytes":9478144,"num_objects":45,"num_object_clones":4,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":33,"num_read":384,"num_read_kb":2623,"num_write":137,"num_write_kb":2159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":66048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"51'1052","reported_seq":6407,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-08T08:52:57.748091+0000","last_change":"2024-05-08T08:52:13.934541+0000","last_active":"2024-05-08T08:52:57.748091+0000","last_peered":"2024-05-08T08:52:57.748091+0000","last_clean":"2024-05-08T08:52:57.748091+0000","last_became_active":"2024-05-08T08:52:11.931202+0000","last_became_peered":"2024-05-08T08:52:11.931202+0000","last_unstale":"2024-05-08T08:52:57.748091+0000","last_undegraded":"2024-05-08T08:52:57.748091+0000","last_fullsized":"2024-05-08T08:52:57.748091+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":1052,"log_dups_size":0,"ondisk_log_size":1052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:10:49.289725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059220300000000002,"stat_sum":{"num_bytes":22436519,"num_objects":53,"num_object_clones":6,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":31,"num_read":11306,"num_read_kb":23332,"num_write":1641,"num_write_kb":5059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"51'280","reported_seq":2436,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-08T08:52:57.742861+0000","last_change":"2024-05-08T08:52:13.944894+0000","last_active":"2024-05-08T08:52:57.742861+0000","last_peered":"2024-05-08T08:52:57.742861+0000","last_clean":"2024-05-08T08:52:57.742861+0000","last_became_active":"2024-05-08T08:52:11.930434+0000","last_became_peered":"2024-05-08T08:52:11.930434+0000","last_unstale":"2024-05-08T08:52:57.742861+0000","last_undegraded":"2024-05-08T08:52:57.742861+0000","last_fullsized":"2024-05-08T08:52:57.742861+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":280,"log_dups_size":0,"ondisk_log_size":280,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:11:38.520825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059972400000000005,"stat_sum":{"num_bytes":36831789,"num_objects":60,"num_object_clones":16,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":31,"num_read":2463,"num_read_kb":17659,"num_write":293,"num_write_kb":5489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":550993248,"num_objects":1408,"num_object_clones":208,"num_object_copies":2816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1408,"num_whiteouts":937,"num_read":147933,"num_read_kb":495904,"num_write":8876,"num_write_kb":195228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":637,"num_bytes_recovered":27285362,"num_keys_recovered":109,"num_objects_omap":16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32366592,"data_stored":267800450,"data_compressed":12087216,"data_compressed_allocated":24203264,"data_compressed_original":262300160,"omap_allocated":0,"internal_metadata":0},"log_size":8256,"ondisk_log_size":8256,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":11046,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640270,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45692,"kb_used_data":6192,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94326148,"statfs":{"total":96636764160,"available":96589975552,"internally_reserved":0,"allocated":6340608,"data_stored":48625067,"data_compressed":2408826,"data_compressed_allocated":4825088,"data_compressed_original":47758848,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":0,"seq":47244640273,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640272,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47824,"kb_used_data":8328,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94324016,"statfs":{"total":96636764160,"available":96587792384,"internally_reserved":0,"allocated":8527872,"data_stored":59114053,"data_compressed":3074742,"data_compressed_allocated":5271552,"data_compressed_original":57048064,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640271,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39064,"kb_used_data":3720,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94332776,"statfs":{"total":96636764160,"available":96596762624,"internally_reserved":0,"allocated":3809280,"data_stored":30406306,"data_compressed":1325927,"data_compressed_allocated":2621440,"data_compressed_original":29791744,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640270,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40232,"kb_used_data":4888,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94331608,"statfs":{"total":96636764160,"available":96595566592,"internally_reserved":0,"allocated":5005312,"data_stored":43907197,"data_compressed":1697987,"data_compressed_allocated":3735552,"data_compressed_original":43209216,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640273,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34016,"kb_used_data":7000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94337824,"statfs":{"total":96636764160,"available":96601931776,"internally_reserved":0,"allocated":7168000,"data_stored":49041950,"data_compressed":2138420,"data_compressed_allocated":4124672,"data_compressed_original":47166464,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5523,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5523,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6856704,"data_stored":51182067,"data_compressed":2250706,"data_compressed_allocated":4333568,"data_compressed_original":49497088,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4874240,"data_stored":47030386,"data_compressed":1847555,"data_compressed_allocated":4100096,"data_compressed_original":46514688,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3350528,"data_stored":30344343,"data_compressed":1364115,"data_compressed_allocated":2699264,"data_compressed_original":29937664,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8454144,"data_stored":63108644,"data_compressed":3266326,"data_compressed_allocated":5722112,"data_compressed_original":61243392,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":2830336,"data_stored":25559234,"data_compressed":860285,"data_compressed_allocated":2351104,"data_compressed_original":25220096,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":6000640,"data_stored":50575776,"data_compressed":2498229,"data_compressed_allocated":4997120,"data_compressed_original":49887232,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T08:52:58.910 INFO:tasks.thrashosds.thrasher:Setting 3.4 to [1, 5] 2024-05-08T08:52:58.910 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.4', '1', '5'] 2024-05-08T08:52:58.910 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.4 1 5 2024-05-08T08:52:58.945 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:52:58.945 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:52:58.945 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:52:58.945 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:52:58.945 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.946 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9944 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 749568~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:58.946 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.103900+0000", 2024-05-08T08:52:58.946 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 31.843943337999999, 2024-05-08T08:52:58.946 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057007044, 2024-05-08T08:52:58.946 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:58.946 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:58.946 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:58.946 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:58.946 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:58.946 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:58.946 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9944 2024-05-08T08:52:58.946 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.946 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:58.946 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.946 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:58.947 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103900+0000", 2024-05-08T08:52:58.947 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:58.947 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.947 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.947 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:58.947 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103900+0000", 2024-05-08T08:52:58.947 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:58.947 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.947 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.947 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:58.947 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103900+0000", 2024-05-08T08:52:58.947 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6400000000000003e-07 2024-05-08T08:52:58.947 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.947 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.947 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:58.948 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103902+0000", 2024-05-08T08:52:58.948 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.858e-06 2024-05-08T08:52:58.948 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.948 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.948 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:58.948 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103907+0000", 2024-05-08T08:52:58.948 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3410000000000002e-06 2024-05-08T08:52:58.948 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.948 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.948 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:58.948 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103910+0000", 2024-05-08T08:52:58.948 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8540000000000001e-06 2024-05-08T08:52:58.948 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.948 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.948 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:58.949 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.160682+0000", 2024-05-08T08:52:58.949 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.056771822 2024-05-08T08:52:58.949 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.949 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.949 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:58.949 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.160777+0000", 2024-05-08T08:52:58.949 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4555999999999994e-05 2024-05-08T08:52:58.949 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.949 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.949 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:58.949 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.160907+0000", 2024-05-08T08:52:58.949 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000130149 2024-05-08T08:52:58.949 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.949 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:58.949 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.949 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.950 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.950 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9945 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 1441792~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:58.950 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.103914+0000", 2024-05-08T08:52:58.950 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 31.843928744999999, 2024-05-08T08:52:58.950 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057211979000000003, 2024-05-08T08:52:58.950 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:58.950 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:58.950 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:58.950 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:58.950 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:58.950 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:58.950 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9945 2024-05-08T08:52:58.950 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.950 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:58.950 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.951 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:58.951 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103914+0000", 2024-05-08T08:52:58.951 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:58.951 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.951 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.951 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:58.951 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103914+0000", 2024-05-08T08:52:58.951 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:58.951 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.951 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.951 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:58.951 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103915+0000", 2024-05-08T08:52:58.951 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3099999999999999e-07 2024-05-08T08:52:58.951 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.951 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.951 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:58.952 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103916+0000", 2024-05-08T08:52:58.952 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.203e-06 2024-05-08T08:52:58.952 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.952 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.952 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:58.952 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103919+0000", 2024-05-08T08:52:58.952 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.117e-06 2024-05-08T08:52:58.952 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.952 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.952 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:58.952 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103921+0000", 2024-05-08T08:52:58.952 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4169999999999999e-06 2024-05-08T08:52:58.952 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.952 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.952 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:58.952 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.160959+0000", 2024-05-08T08:52:58.952 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057038074000000001 2024-05-08T08:52:58.953 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.953 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.953 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:58.953 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161041+0000", 2024-05-08T08:52:58.953 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1620999999999997e-05 2024-05-08T08:52:58.953 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.953 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.953 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:58.953 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161126+0000", 2024-05-08T08:52:58.953 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.5216000000000005e-05 2024-05-08T08:52:58.953 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.953 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:58.953 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.953 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.953 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.953 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9946 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 765952~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:58.954 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.103925+0000", 2024-05-08T08:52:58.954 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 31.843918378000001, 2024-05-08T08:52:58.954 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057418770000000001, 2024-05-08T08:52:58.954 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:58.954 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:58.954 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:58.954 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:58.954 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:58.954 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:58.954 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9946 2024-05-08T08:52:58.954 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.954 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:58.954 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.954 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:58.954 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103925+0000", 2024-05-08T08:52:58.955 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:58.955 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.955 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.955 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:58.955 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103925+0000", 2024-05-08T08:52:58.955 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:58.955 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.955 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.955 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:58.955 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103925+0000", 2024-05-08T08:52:58.955 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9200000000000002e-07 2024-05-08T08:52:58.955 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.955 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.955 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:58.955 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103926+0000", 2024-05-08T08:52:58.955 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1620000000000001e-06 2024-05-08T08:52:58.956 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.956 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.956 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:58.956 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103929+0000", 2024-05-08T08:52:58.956 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9349999999999999e-06 2024-05-08T08:52:58.956 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.956 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.956 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:58.956 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103931+0000", 2024-05-08T08:52:58.956 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4099999999999998e-06 2024-05-08T08:52:58.956 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.956 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.956 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:58.956 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161161+0000", 2024-05-08T08:52:58.956 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057229176999999999 2024-05-08T08:52:58.957 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.957 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.957 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:58.957 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161235+0000", 2024-05-08T08:52:58.957 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4227000000000006e-05 2024-05-08T08:52:58.957 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.957 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.957 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:58.957 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161343+0000", 2024-05-08T08:52:58.957 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010856700000000001 2024-05-08T08:52:58.957 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.957 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:58.957 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.957 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.957 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.957 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9947 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 1458176~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:58.958 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.103935+0000", 2024-05-08T08:52:58.958 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 31.843908500000001, 2024-05-08T08:52:58.958 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057541263000000002, 2024-05-08T08:52:58.958 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:58.958 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:58.958 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:58.958 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:58.958 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:58.958 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:58.958 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9947 2024-05-08T08:52:58.958 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.958 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:58.958 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.958 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:58.959 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103935+0000", 2024-05-08T08:52:58.959 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:58.959 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.959 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.959 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:58.959 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103935+0000", 2024-05-08T08:52:58.959 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:58.959 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.959 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.959 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:58.959 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103935+0000", 2024-05-08T08:52:58.959 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.11e-07 2024-05-08T08:52:58.959 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.959 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.960 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:58.960 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103940+0000", 2024-05-08T08:52:58.960 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1030000000000001e-06 2024-05-08T08:52:58.960 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.960 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.960 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:58.960 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103943+0000", 2024-05-08T08:52:58.960 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3869999999999999e-06 2024-05-08T08:52:58.960 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.960 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.960 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:58.960 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103951+0000", 2024-05-08T08:52:58.960 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.2150000000000001e-06 2024-05-08T08:52:58.960 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.960 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.961 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:58.961 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161377+0000", 2024-05-08T08:52:58.961 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057425925000000003 2024-05-08T08:52:58.961 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.961 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.961 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:58.961 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161431+0000", 2024-05-08T08:52:58.961 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3408000000000001e-05 2024-05-08T08:52:58.961 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.961 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.961 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:58.961 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161476+0000", 2024-05-08T08:52:58.961 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5014e-05 2024-05-08T08:52:58.961 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.961 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:58.961 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.961 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.962 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.962 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9948 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 782336~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:58.962 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.103965+0000", 2024-05-08T08:52:58.962 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 31.84387761, 2024-05-08T08:52:58.962 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057746205000000002, 2024-05-08T08:52:58.962 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:58.962 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:58.962 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:58.962 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:58.962 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:58.962 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:58.962 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9948 2024-05-08T08:52:58.962 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.962 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:58.963 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.963 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:58.963 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103965+0000", 2024-05-08T08:52:58.963 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:58.963 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.963 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.963 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:58.963 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103965+0000", 2024-05-08T08:52:58.963 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:58.963 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.963 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.963 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:58.963 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103966+0000", 2024-05-08T08:52:58.963 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.44e-07 2024-05-08T08:52:58.963 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.964 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.964 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:58.964 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103968+0000", 2024-05-08T08:52:58.964 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4090000000000001e-06 2024-05-08T08:52:58.964 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.964 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.964 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:58.964 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103973+0000", 2024-05-08T08:52:58.964 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6820000000000002e-06 2024-05-08T08:52:58.964 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.964 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.964 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:58.964 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103976+0000", 2024-05-08T08:52:58.964 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.959e-06 2024-05-08T08:52:58.964 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.964 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.965 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:58.965 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161523+0000", 2024-05-08T08:52:58.965 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057547264000000001 2024-05-08T08:52:58.965 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.965 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.965 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:58.965 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161605+0000", 2024-05-08T08:52:58.965 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1854000000000006e-05 2024-05-08T08:52:58.965 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.965 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.965 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:58.965 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161712+0000", 2024-05-08T08:52:58.965 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000106493 2024-05-08T08:52:58.965 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.965 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:58.965 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.966 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.966 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.966 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9949 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 1474560~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:58.966 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.103980+0000", 2024-05-08T08:52:58.966 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 31.843862991000002, 2024-05-08T08:52:58.966 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057852397, 2024-05-08T08:52:58.966 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:58.966 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:58.966 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:58.966 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:58.966 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:58.966 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:58.966 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9949 2024-05-08T08:52:58.966 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.966 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:58.967 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.967 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:58.967 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103980+0000", 2024-05-08T08:52:58.967 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:58.967 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.967 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.967 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:58.967 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103980+0000", 2024-05-08T08:52:58.967 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:58.967 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.967 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.967 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:58.967 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103980+0000", 2024-05-08T08:52:58.967 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.11e-07 2024-05-08T08:52:58.967 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.968 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.968 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:58.968 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103982+0000", 2024-05-08T08:52:58.968 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.269e-06 2024-05-08T08:52:58.968 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.968 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.968 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:58.968 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103985+0000", 2024-05-08T08:52:58.968 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0110000000000001e-06 2024-05-08T08:52:58.968 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.968 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.968 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:58.968 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103987+0000", 2024-05-08T08:52:58.968 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3020000000000002e-06 2024-05-08T08:52:58.968 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.968 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.969 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:58.969 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161744+0000", 2024-05-08T08:52:58.969 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057757115999999997 2024-05-08T08:52:58.969 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.969 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.969 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:58.969 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161791+0000", 2024-05-08T08:52:58.969 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6807000000000001e-05 2024-05-08T08:52:58.969 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.969 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.969 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:58.969 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161832+0000", 2024-05-08T08:52:58.969 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1681000000000003e-05 2024-05-08T08:52:58.969 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.969 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:58.969 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.969 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.970 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.970 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9950 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 798720~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:58.970 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.103990+0000", 2024-05-08T08:52:58.970 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 31.843852756, 2024-05-08T08:52:58.970 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.058062890999999998, 2024-05-08T08:52:58.970 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:58.970 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:58.970 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:58.970 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:58.970 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:58.970 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:58.970 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9950 2024-05-08T08:52:58.970 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.970 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:58.971 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.971 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:58.971 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103990+0000", 2024-05-08T08:52:58.971 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:58.971 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.971 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.971 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:58.971 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103990+0000", 2024-05-08T08:52:58.971 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:58.971 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.971 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.971 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:58.971 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103991+0000", 2024-05-08T08:52:58.971 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1e-07 2024-05-08T08:52:58.971 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.971 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.972 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:58.972 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103992+0000", 2024-05-08T08:52:58.972 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.074e-06 2024-05-08T08:52:58.972 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.972 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.972 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:58.972 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103995+0000", 2024-05-08T08:52:58.972 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9529999999999999e-06 2024-05-08T08:52:58.972 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.972 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.972 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:58.972 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103997+0000", 2024-05-08T08:52:58.972 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3939999999999999e-06 2024-05-08T08:52:58.972 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.972 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.972 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:58.973 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161871+0000", 2024-05-08T08:52:58.973 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057873854000000002 2024-05-08T08:52:58.973 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.973 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.973 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:58.973 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161950+0000", 2024-05-08T08:52:58.973 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9433999999999999e-05 2024-05-08T08:52:58.973 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.973 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.973 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:58.973 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.162053+0000", 2024-05-08T08:52:58.973 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000102972 2024-05-08T08:52:58.973 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.973 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:58.973 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.973 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.973 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.974 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9951 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 1490944~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:58.974 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.104000+0000", 2024-05-08T08:52:58.974 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 31.843842951999999, 2024-05-08T08:52:58.974 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.058313416999999999, 2024-05-08T08:52:58.974 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:58.974 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:58.974 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:58.974 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:58.974 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:58.974 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:58.974 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9951 2024-05-08T08:52:58.974 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.974 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:58.974 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.975 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:58.975 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.104000+0000", 2024-05-08T08:52:58.975 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:58.975 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.975 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.975 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:58.975 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.104000+0000", 2024-05-08T08:52:58.975 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:58.975 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.975 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.975 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:58.975 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.104962+0000", 2024-05-08T08:52:58.975 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00096230399999999998 2024-05-08T08:52:58.975 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.975 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.976 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:58.976 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.104978+0000", 2024-05-08T08:52:58.976 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5126999999999999e-05 2024-05-08T08:52:58.976 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.976 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.976 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:58.976 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.104987+0000", 2024-05-08T08:52:58.976 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8940000000000005e-06 2024-05-08T08:52:58.976 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.976 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.976 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:58.976 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.104994+0000", 2024-05-08T08:52:58.976 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5610000000000004e-06 2024-05-08T08:52:58.976 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.976 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.977 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:58.977 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.162114+0000", 2024-05-08T08:52:58.977 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057119717 2024-05-08T08:52:58.977 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.977 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.977 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:58.977 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.162214+0000", 2024-05-08T08:52:58.977 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.9982999999999995e-05 2024-05-08T08:52:58.977 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.977 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.977 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:58.977 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.162314+0000", 2024-05-08T08:52:58.977 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.9831e-05 2024-05-08T08:52:58.977 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.977 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:58.977 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.978 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.978 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.978 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9952 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 815104~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:58.978 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.105004+0000", 2024-05-08T08:52:58.978 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 31.842838699000001, 2024-05-08T08:52:58.978 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057530855999999998, 2024-05-08T08:52:58.978 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:58.978 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:58.978 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:58.978 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:58.978 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:58.978 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:58.978 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9952 2024-05-08T08:52:58.978 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.978 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:58.979 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.979 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:58.979 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105004+0000", 2024-05-08T08:52:58.979 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:58.979 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.979 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.979 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:58.979 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105004+0000", 2024-05-08T08:52:58.979 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:58.979 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.979 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.979 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:58.979 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105005+0000", 2024-05-08T08:52:58.979 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9299999999999998e-07 2024-05-08T08:52:58.979 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.980 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.980 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:58.980 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105007+0000", 2024-05-08T08:52:58.980 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2979999999999999e-06 2024-05-08T08:52:58.980 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.980 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.980 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:58.980 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105017+0000", 2024-05-08T08:52:58.980 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5060000000000001e-06 2024-05-08T08:52:58.980 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.980 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.980 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:58.980 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105022+0000", 2024-05-08T08:52:58.980 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2619999999999998e-06 2024-05-08T08:52:58.980 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.980 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.981 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:58.981 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.162356+0000", 2024-05-08T08:52:58.981 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057334388999999999 2024-05-08T08:52:58.981 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.981 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.981 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:58.981 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.162437+0000", 2024-05-08T08:52:58.981 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0863000000000002e-05 2024-05-08T08:52:58.981 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.981 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.981 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:58.981 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.162535+0000", 2024-05-08T08:52:58.981 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8045000000000001e-05 2024-05-08T08:52:58.981 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.981 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:58.981 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.981 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.982 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.982 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9953 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 1507328~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:58.982 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.105064+0000", 2024-05-08T08:52:58.982 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 31.842779012000001, 2024-05-08T08:52:58.982 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057721531999999999, 2024-05-08T08:52:58.982 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:58.982 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:58.982 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:58.982 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:58.982 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:58.982 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:58.982 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9953 2024-05-08T08:52:58.982 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.982 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:58.983 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.983 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:58.983 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105064+0000", 2024-05-08T08:52:58.983 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:58.983 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.983 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.983 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:58.983 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105064+0000", 2024-05-08T08:52:58.983 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:58.983 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.983 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.983 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:58.983 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105065+0000", 2024-05-08T08:52:58.983 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8800000000000002e-07 2024-05-08T08:52:58.983 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.983 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.984 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:58.984 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105067+0000", 2024-05-08T08:52:58.984 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7669999999999999e-06 2024-05-08T08:52:58.984 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.984 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.984 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:58.984 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105076+0000", 2024-05-08T08:52:58.984 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.5259999999999993e-06 2024-05-08T08:52:58.984 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.984 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.984 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:58.984 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105080+0000", 2024-05-08T08:52:58.984 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8e-06 2024-05-08T08:52:58.984 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.984 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.985 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:58.985 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.162596+0000", 2024-05-08T08:52:58.985 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057515822000000001 2024-05-08T08:52:58.985 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.985 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.985 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:58.985 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.162689+0000", 2024-05-08T08:52:58.985 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3258999999999996e-05 2024-05-08T08:52:58.985 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.985 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.985 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:58.985 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.162786+0000", 2024-05-08T08:52:58.985 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6769999999999997e-05 2024-05-08T08:52:58.985 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.985 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:58.985 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.986 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.986 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.986 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9954 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 1523712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:58.986 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.105088+0000", 2024-05-08T08:52:58.986 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 31.842755003000001, 2024-05-08T08:52:58.986 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057925139, 2024-05-08T08:52:58.986 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:58.986 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:58.986 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:58.986 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:58.986 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:58.986 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:58.986 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9954 2024-05-08T08:52:58.986 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.986 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:58.986 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.987 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:58.987 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105088+0000", 2024-05-08T08:52:58.987 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:58.987 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.987 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.987 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:58.987 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105088+0000", 2024-05-08T08:52:58.987 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:58.987 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.987 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.987 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:58.987 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105089+0000", 2024-05-08T08:52:58.987 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9599999999999999e-07 2024-05-08T08:52:58.987 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.987 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.988 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:58.988 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105091+0000", 2024-05-08T08:52:58.988 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3649999999999998e-06 2024-05-08T08:52:58.988 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.988 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.988 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:58.988 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105107+0000", 2024-05-08T08:52:58.988 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6370000000000001e-05 2024-05-08T08:52:58.988 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.988 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.988 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:58.988 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105114+0000", 2024-05-08T08:52:58.988 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2190000000000001e-06 2024-05-08T08:52:58.988 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.989 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.989 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:58.989 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.162830+0000", 2024-05-08T08:52:58.989 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057716637000000001 2024-05-08T08:52:58.989 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.989 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.989 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:58.989 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.162924+0000", 2024-05-08T08:52:58.989 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3354000000000001e-05 2024-05-08T08:52:58.989 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.989 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.989 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:58.989 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.163013+0000", 2024-05-08T08:52:58.989 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.9697999999999994e-05 2024-05-08T08:52:58.989 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.989 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:58.990 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.990 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.990 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.990 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9955 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 831488~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:58.990 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.105124+0000", 2024-05-08T08:52:58.990 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 31.842719428999999, 2024-05-08T08:52:58.990 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.058092618999999998, 2024-05-08T08:52:58.990 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:58.990 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:58.990 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:58.990 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:58.990 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:58.990 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:58.990 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9955 2024-05-08T08:52:58.990 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.990 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:58.991 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.991 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:58.991 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105124+0000", 2024-05-08T08:52:58.991 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:58.991 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.991 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.991 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:58.991 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105124+0000", 2024-05-08T08:52:58.991 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:58.991 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.991 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.991 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:58.991 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.106961+0000", 2024-05-08T08:52:58.991 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001837574 2024-05-08T08:52:58.991 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.991 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.992 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:58.992 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.106966+0000", 2024-05-08T08:52:58.992 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8439999999999997e-06 2024-05-08T08:52:58.992 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.992 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.992 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:58.992 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.106978+0000", 2024-05-08T08:52:58.992 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2330999999999999e-05 2024-05-08T08:52:58.992 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.992 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.992 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:58.992 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.106986+0000", 2024-05-08T08:52:58.992 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0439999999999996e-06 2024-05-08T08:52:58.992 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.992 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.992 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:58.992 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.163048+0000", 2024-05-08T08:52:58.993 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.056061182000000001 2024-05-08T08:52:58.993 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.993 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.993 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:58.993 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.163124+0000", 2024-05-08T08:52:58.993 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.6030000000000002e-05 2024-05-08T08:52:58.993 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.993 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.993 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:58.993 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.163216+0000", 2024-05-08T08:52:58.993 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2614000000000005e-05 2024-05-08T08:52:58.993 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.993 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:58.993 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.994 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.994 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.994 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9960 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 1589248~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:58.994 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.107116+0000", 2024-05-08T08:52:58.994 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 31.840726611000001, 2024-05-08T08:52:58.994 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.056889636, 2024-05-08T08:52:58.994 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:58.994 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:58.994 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:58.994 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:58.994 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:58.994 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:58.994 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9960 2024-05-08T08:52:58.994 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.995 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:58.995 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.995 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:58.995 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107116+0000", 2024-05-08T08:52:58.995 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:58.995 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.995 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.995 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:58.995 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107116+0000", 2024-05-08T08:52:58.995 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:58.995 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.995 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.995 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:58.995 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107117+0000", 2024-05-08T08:52:58.995 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4400000000000001e-07 2024-05-08T08:52:58.996 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.996 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.996 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:58.996 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107119+0000", 2024-05-08T08:52:58.996 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3750000000000001e-06 2024-05-08T08:52:58.996 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.996 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.996 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:58.996 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107126+0000", 2024-05-08T08:52:58.996 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3369999999999996e-06 2024-05-08T08:52:58.996 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.996 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.996 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:58.996 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107130+0000", 2024-05-08T08:52:58.996 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8620000000000002e-06 2024-05-08T08:52:58.996 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.997 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.997 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:58.997 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.163917+0000", 2024-05-08T08:52:58.997 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.056786634000000002 2024-05-08T08:52:58.997 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.997 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.997 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:58.997 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.163962+0000", 2024-05-08T08:52:58.997 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4521000000000003e-05 2024-05-08T08:52:58.997 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.997 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.997 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:58.997 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.164006+0000", 2024-05-08T08:52:58.997 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4462999999999998e-05 2024-05-08T08:52:58.997 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.997 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:58.998 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:58.998 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.998 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.998 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9961 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 864256~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:58.998 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.107138+0000", 2024-05-08T08:52:58.998 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 31.840705551999999, 2024-05-08T08:52:58.998 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057108051999999999, 2024-05-08T08:52:58.998 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:58.998 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:58.998 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:58.998 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:58.998 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:58.998 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:58.998 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9961 2024-05-08T08:52:58.999 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.999 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:58.999 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.999 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:58.999 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107138+0000", 2024-05-08T08:52:58.999 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:58.999 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.999 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.999 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:58.999 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107138+0000", 2024-05-08T08:52:58.999 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:58.999 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:58.999 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:58.999 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:58.999 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107138+0000", 2024-05-08T08:52:58.999 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2399999999999999e-07 2024-05-08T08:52:59.000 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.000 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.000 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:59.000 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107140+0000", 2024-05-08T08:52:59.000 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1409999999999999e-06 2024-05-08T08:52:59.000 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.000 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.000 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:59.000 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107146+0000", 2024-05-08T08:52:59.000 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0639999999999999e-06 2024-05-08T08:52:59.000 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.000 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.000 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:59.000 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107151+0000", 2024-05-08T08:52:59.000 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.69e-06 2024-05-08T08:52:59.000 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.001 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.001 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:59.001 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.164051+0000", 2024-05-08T08:52:59.001 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.056900153000000002 2024-05-08T08:52:59.001 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.001 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.001 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:59.001 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.164136+0000", 2024-05-08T08:52:59.001 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.5086999999999994e-05 2024-05-08T08:52:59.001 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.001 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.001 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:59.001 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.164246+0000", 2024-05-08T08:52:59.001 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000109493 2024-05-08T08:52:59.001 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:59.001 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:59.002 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:59.002 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.002 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.002 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9962 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 1605632~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:59.002 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.107158+0000", 2024-05-08T08:52:59.002 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 31.840685473000001, 2024-05-08T08:52:59.002 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057292698000000003, 2024-05-08T08:52:59.002 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:59.002 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:59.002 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:59.002 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:59.002 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:59.002 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:59.002 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9962 2024-05-08T08:52:59.002 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.003 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:59.003 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.003 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:59.003 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107158+0000", 2024-05-08T08:52:59.003 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:59.003 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.003 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.003 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:59.003 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107158+0000", 2024-05-08T08:52:59.003 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:59.003 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.003 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.003 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:59.003 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107158+0000", 2024-05-08T08:52:59.003 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8100000000000003e-07 2024-05-08T08:52:59.003 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.004 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.004 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:59.004 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107167+0000", 2024-05-08T08:52:59.004 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3079999999999997e-06 2024-05-08T08:52:59.004 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.004 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.004 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:59.004 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107175+0000", 2024-05-08T08:52:59.004 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2089999999999999e-06 2024-05-08T08:52:59.004 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.004 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.004 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:59.004 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107180+0000", 2024-05-08T08:52:59.004 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1120000000000004e-06 2024-05-08T08:52:59.004 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.004 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.005 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:59.005 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.164284+0000", 2024-05-08T08:52:59.005 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057104110999999999 2024-05-08T08:52:59.005 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.005 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.005 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:59.005 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.164370+0000", 2024-05-08T08:52:59.005 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.6549000000000001e-05 2024-05-08T08:52:59.005 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.005 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.005 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:59.005 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.164450+0000", 2024-05-08T08:52:59.005 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9827999999999995e-05 2024-05-08T08:52:59.005 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:59.005 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:59.005 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:59.005 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.006 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.006 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9963 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 880640~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:59.006 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.107188+0000", 2024-05-08T08:52:59.006 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 31.840655555000001, 2024-05-08T08:52:59.006 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057560471000000002, 2024-05-08T08:52:59.006 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:59.006 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:59.006 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:59.006 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:59.006 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:59.006 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:59.006 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9963 2024-05-08T08:52:59.006 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.006 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:59.006 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.006 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:59.007 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107188+0000", 2024-05-08T08:52:59.007 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:59.007 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.007 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.007 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:59.007 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107188+0000", 2024-05-08T08:52:59.007 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:59.007 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.007 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.007 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:59.007 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.108964+0000", 2024-05-08T08:52:59.007 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001775993 2024-05-08T08:52:59.007 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.007 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.007 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:59.007 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.108969+0000", 2024-05-08T08:52:59.008 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1200000000000001e-06 2024-05-08T08:52:59.008 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.008 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.008 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:59.008 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.108981+0000", 2024-05-08T08:52:59.008 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2670000000000001e-05 2024-05-08T08:52:59.008 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.008 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.008 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:59.008 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.108989+0000", 2024-05-08T08:52:59.008 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3139999999999998e-06 2024-05-08T08:52:59.008 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.008 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.008 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:59.008 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.164504+0000", 2024-05-08T08:52:59.008 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.055515225000000001 2024-05-08T08:52:59.008 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.009 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.009 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:59.009 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.164606+0000", 2024-05-08T08:52:59.009 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010245 2024-05-08T08:52:59.009 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.009 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.009 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:59.009 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.164748+0000", 2024-05-08T08:52:59.009 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014169900000000001 2024-05-08T08:52:59.009 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:59.009 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:59.009 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:59.009 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.009 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.009 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9970 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 929792~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:59.009 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.109178+0000", 2024-05-08T08:52:59.010 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 31.838665277, 2024-05-08T08:52:59.010 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057137975000000001, 2024-05-08T08:52:59.010 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:59.010 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:59.010 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:59.010 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:59.010 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:59.010 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:59.010 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9970 2024-05-08T08:52:59.010 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.010 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:59.010 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.010 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:59.010 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109178+0000", 2024-05-08T08:52:59.010 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:59.010 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.011 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.011 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:59.011 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109178+0000", 2024-05-08T08:52:59.011 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:59.011 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.011 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.011 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:59.011 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109178+0000", 2024-05-08T08:52:59.011 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7800000000000002e-07 2024-05-08T08:52:59.011 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.011 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.011 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:59.011 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109180+0000", 2024-05-08T08:52:59.011 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2199999999999999e-06 2024-05-08T08:52:59.011 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.011 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.011 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:59.012 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109186+0000", 2024-05-08T08:52:59.012 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9410000000000004e-06 2024-05-08T08:52:59.012 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.012 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.012 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:59.012 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109191+0000", 2024-05-08T08:52:59.012 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8729999999999998e-06 2024-05-08T08:52:59.012 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.012 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.012 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:59.012 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.166098+0000", 2024-05-08T08:52:59.012 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.056907124000000003 2024-05-08T08:52:59.012 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.012 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.012 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:59.013 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.166193+0000", 2024-05-08T08:52:59.013 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5018000000000004e-05 2024-05-08T08:52:59.013 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.013 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.013 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:59.013 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.166316+0000", 2024-05-08T08:52:59.013 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000122321 2024-05-08T08:52:59.013 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:59.013 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:59.013 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:59.013 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.013 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.013 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9971 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 1687552~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:59.013 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.109198+0000", 2024-05-08T08:52:59.013 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 31.838644598999998, 2024-05-08T08:52:59.014 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057361875, 2024-05-08T08:52:59.014 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:59.014 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:59.014 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:59.014 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:59.014 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:59.014 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:59.014 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9971 2024-05-08T08:52:59.014 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.014 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:59.014 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.014 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:59.014 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109198+0000", 2024-05-08T08:52:59.014 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:59.015 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.015 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.015 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:59.015 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109198+0000", 2024-05-08T08:52:59.015 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:59.015 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.015 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.015 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:59.015 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109199+0000", 2024-05-08T08:52:59.015 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2800000000000002e-07 2024-05-08T08:52:59.015 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.015 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.015 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:59.015 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109208+0000", 2024-05-08T08:52:59.015 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4210000000000001e-06 2024-05-08T08:52:59.016 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.016 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.016 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:59.016 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109215+0000", 2024-05-08T08:52:59.016 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7209999999999997e-06 2024-05-08T08:52:59.016 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.016 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.016 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:59.016 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109220+0000", 2024-05-08T08:52:59.016 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.13e-06 2024-05-08T08:52:59.016 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.016 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.016 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:59.016 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.166353+0000", 2024-05-08T08:52:59.016 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057132693999999998 2024-05-08T08:52:59.016 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.017 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.017 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:59.017 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.166462+0000", 2024-05-08T08:52:59.017 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010940799999999999 2024-05-08T08:52:59.017 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.017 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.017 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:59.017 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.166560+0000", 2024-05-08T08:52:59.017 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8072999999999994e-05 2024-05-08T08:52:59.017 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:59.017 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:59.017 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:59.017 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.017 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.017 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9972 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 1703936~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:59.017 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.109228+0000", 2024-05-08T08:52:59.017 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 31.838615207, 2024-05-08T08:52:59.018 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057541904999999997, 2024-05-08T08:52:59.018 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:59.018 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:59.018 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:59.018 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:59.018 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:59.018 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:59.018 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9972 2024-05-08T08:52:59.018 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.018 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:59.018 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.018 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:59.018 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109228+0000", 2024-05-08T08:52:59.018 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:59.018 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.018 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.018 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:59.019 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109228+0000", 2024-05-08T08:52:59.019 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:59.019 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.019 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.019 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:59.019 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109228+0000", 2024-05-08T08:52:59.019 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5699999999999998e-07 2024-05-08T08:52:59.019 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.019 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.019 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:59.019 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109231+0000", 2024-05-08T08:52:59.019 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2120000000000002e-06 2024-05-08T08:52:59.019 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.019 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.019 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:59.019 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109237+0000", 2024-05-08T08:52:59.020 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3099999999999997e-06 2024-05-08T08:52:59.020 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.020 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.020 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:59.020 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109243+0000", 2024-05-08T08:52:59.020 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7490000000000004e-06 2024-05-08T08:52:59.020 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.020 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.020 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:59.020 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.166609+0000", 2024-05-08T08:52:59.020 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057366372999999998 2024-05-08T08:52:59.020 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.020 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.020 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:59.020 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.166690+0000", 2024-05-08T08:52:59.021 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1426000000000003e-05 2024-05-08T08:52:59.021 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.021 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.021 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:59.021 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.166770+0000", 2024-05-08T08:52:59.021 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9377999999999998e-05 2024-05-08T08:52:59.021 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:59.021 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:59.021 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:59.021 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.021 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.021 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9973 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 946176~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:52:59.021 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.109249+0000", 2024-05-08T08:52:59.021 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 31.838593565, 2024-05-08T08:52:59.021 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057760381, 2024-05-08T08:52:59.021 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:59.021 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:59.022 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:59.022 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:59.022 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:52:59.022 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:52:59.022 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9973 2024-05-08T08:52:59.022 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.022 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:59.022 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.022 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:59.022 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109249+0000", 2024-05-08T08:52:59.022 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:59.022 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.022 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.022 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:59.022 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109249+0000", 2024-05-08T08:52:59.022 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:59.023 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.023 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.023 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:59.023 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.110959+0000", 2024-05-08T08:52:59.023 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001709288 2024-05-08T08:52:59.023 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.023 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.023 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:59.023 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.110972+0000", 2024-05-08T08:52:59.023 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2719e-05 2024-05-08T08:52:59.023 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.023 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.023 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:59.023 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.110984+0000", 2024-05-08T08:52:59.023 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2001000000000001e-05 2024-05-08T08:52:59.023 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.023 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.024 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:59.024 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.110993+0000", 2024-05-08T08:52:59.024 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0880000000000001e-06 2024-05-08T08:52:59.024 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.024 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.024 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:59.024 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.166817+0000", 2024-05-08T08:52:59.024 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.055823971999999999 2024-05-08T08:52:59.024 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.024 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.024 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:59.024 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.166900+0000", 2024-05-08T08:52:59.024 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3838999999999995e-05 2024-05-08T08:52:59.024 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.024 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.024 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:59.025 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.167010+0000", 2024-05-08T08:52:59.025 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000109474 2024-05-08T08:52:59.025 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:59.025 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:59.025 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:59.025 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:59.025 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:59.025 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:52:59.026 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:52:59.027 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:52:59.029+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:52:59.120 INFO:teuthology.orchestra.run.smithi196.stdout:checking clone #12, image image_client.0-clone12 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent13 2024-05-08T08:52:59.124 INFO:teuthology.orchestra.run.smithi059.stderr:noscrub is set 2024-05-08T08:52:59.127 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:59.130+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:59.190 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:52:59.190 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:52:59.190 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.190 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(client.4832.0:215 3.1a e53/44 3:5d683759:::rbd_header.1140525c891b:head v 53'256, mlcod=52'254)", 2024-05-08T08:52:59.190 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:59.193623+0000", 2024-05-08T08:52:59.190 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.00056230400000000002, 2024-05-08T08:52:59.191 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00057821299999999995, 2024-05-08T08:52:59.191 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:59.191 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:59.191 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:59.191 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:59.191 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.191 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:59.191 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:59.193623+0000", 2024-05-08T08:52:59.191 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:59.191 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.191 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.191 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:59.191 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:59.193623+0000", 2024-05-08T08:52:59.191 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:59.191 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.191 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.192 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:59.192 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:59.193623+0000", 2024-05-08T08:52:59.192 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.8799999999999999e-07 2024-05-08T08:52:59.192 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.192 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.192 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:59.192 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:59.193628+0000", 2024-05-08T08:52:59.192 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.639e-06 2024-05-08T08:52:59.192 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.192 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.192 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:59.192 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:59.193640+0000", 2024-05-08T08:52:59.192 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2388999999999999e-05 2024-05-08T08:52:59.192 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.192 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.193 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:59.193 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:59.193648+0000", 2024-05-08T08:52:59.193 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1550000000000007e-06 2024-05-08T08:52:59.193 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.193 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.193 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:59.193 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:59.193709+0000", 2024-05-08T08:52:59.193 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0146000000000003e-05 2024-05-08T08:52:59.193 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.193 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.193 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:59.193 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:59.193732+0000", 2024-05-08T08:52:59.193 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.368e-05 2024-05-08T08:52:59.193 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:59.193 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:59.194 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:59.194 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:59.194 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T08:52:59.194 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 1 2024-05-08T08:52:59.194 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:52:59.199 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:52:59.201 INFO:teuthology.orchestra.run.smithi196.stdout:2443 trunc from 0xafd4c00 to 0xb49f800 2024-05-08T08:52:59.227 INFO:teuthology.orchestra.run.smithi196.stdout:2444 write 0xe832000 thru 0xe83770f (0x5710 bytes) 2024-05-08T08:52:59.228 INFO:teuthology.orchestra.run.smithi196.stdout:2445 write 0xe08ba00 thru 0xe099e07 (0xe408 bytes) 2024-05-08T08:52:59.228 INFO:teuthology.orchestra.run.smithi196.stdout:2447 read 0x6528000 thru 0x652c226 (0x4227 bytes) 2024-05-08T08:52:59.239 INFO:teuthology.orchestra.run.smithi196.stdout:2448 trunc from 0xe837800 to 0x4ddc600 2024-05-08T08:52:59.243 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:52:59.246+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:52:59.273 INFO:teuthology.orchestra.run.smithi196.stdout:2452 punch from 0x127c400 to 0x1286600, (0xa200 bytes) 2024-05-08T08:52:59.293 INFO:teuthology.orchestra.run.smithi196.stdout:2453 write 0x55f1200 thru 0x55fde87 (0xcc88 bytes) 2024-05-08T08:52:59.299 INFO:teuthology.orchestra.run.smithi196.stdout:2456 trunc from 0x55fe000 to 0xa1ba00 2024-05-08T08:52:59.328 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:59.332+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:59.331 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:52:59.331 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:52:59.331 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:52:59.331 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:52:59.331 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:52:59.343 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:52:59.368 INFO:teuthology.orchestra.run.smithi196.stdout:2459 read 0x989600 thru 0x9934a6 (0x9ea7 bytes) 2024-05-08T08:52:59.376 INFO:teuthology.orchestra.run.smithi196.stdout:2463 clone 15 order 22 su 262144 sc 4 2024-05-08T08:52:59.429 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:52:59.433+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:52:59.495 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:52:59.495 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:52:59.495 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:52:59.495 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:52:59.495 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.495 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7490 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 0~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:59.495 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.744226+0000", 2024-05-08T08:52:59.495 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 21.752743483, 2024-05-08T08:52:59.496 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057294818999999997, 2024-05-08T08:52:59.496 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:59.496 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:59.496 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:59.496 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:59.496 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:59.496 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:59.497 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7490 2024-05-08T08:52:59.497 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.497 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:59.497 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.497 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:59.497 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744226+0000", 2024-05-08T08:52:59.497 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:59.497 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.497 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.497 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:59.497 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744226+0000", 2024-05-08T08:52:59.497 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:59.497 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.497 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.498 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:59.498 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744226+0000", 2024-05-08T08:52:59.498 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3799999999999998e-07 2024-05-08T08:52:59.498 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.498 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.498 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:59.498 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744228+0000", 2024-05-08T08:52:59.498 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0710000000000002e-06 2024-05-08T08:52:59.498 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.498 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.498 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:59.498 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744233+0000", 2024-05-08T08:52:59.498 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0130000000000001e-06 2024-05-08T08:52:59.499 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.499 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.499 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:59.499 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744237+0000", 2024-05-08T08:52:59.499 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1930000000000002e-06 2024-05-08T08:52:59.499 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.499 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.499 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:59.499 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.801334+0000", 2024-05-08T08:52:59.499 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057097004999999999 2024-05-08T08:52:59.499 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.499 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.499 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:59.499 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.801414+0000", 2024-05-08T08:52:59.499 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0129000000000005e-05 2024-05-08T08:52:59.500 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.500 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.500 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:59.500 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.801521+0000", 2024-05-08T08:52:59.500 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010707 2024-05-08T08:52:59.500 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:59.500 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:59.500 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:59.500 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.500 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.500 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7491 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 40960~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:59.500 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.744242+0000", 2024-05-08T08:52:59.500 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 21.752727652000001, 2024-05-08T08:52:59.500 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057541677999999999, 2024-05-08T08:52:59.500 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:59.501 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:59.501 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:59.501 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:59.501 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:59.501 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:59.501 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7491 2024-05-08T08:52:59.501 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.501 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:59.501 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.501 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:59.501 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744242+0000", 2024-05-08T08:52:59.501 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:59.501 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.501 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.501 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:59.502 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744242+0000", 2024-05-08T08:52:59.502 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:59.502 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.502 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.502 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:59.502 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744242+0000", 2024-05-08T08:52:59.502 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.2000000000000001e-07 2024-05-08T08:52:59.502 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.502 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.502 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:59.502 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744244+0000", 2024-05-08T08:52:59.502 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6959999999999999e-06 2024-05-08T08:52:59.502 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.502 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.503 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:59.503 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744247+0000", 2024-05-08T08:52:59.503 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3000000000000002e-06 2024-05-08T08:52:59.503 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.503 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.503 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:59.503 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744250+0000", 2024-05-08T08:52:59.503 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.5399999999999998e-06 2024-05-08T08:52:59.503 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.503 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.503 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:59.503 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.801564+0000", 2024-05-08T08:52:59.503 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057313809 2024-05-08T08:52:59.503 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.503 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.504 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:59.504 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.801671+0000", 2024-05-08T08:52:59.504 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000107333 2024-05-08T08:52:59.504 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.504 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.504 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:59.504 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.801784+0000", 2024-05-08T08:52:59.504 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011268 2024-05-08T08:52:59.504 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:59.505 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:59.505 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:59.505 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.505 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.505 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7492 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 81920~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:59.505 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.744253+0000", 2024-05-08T08:52:59.505 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 21.752716175, 2024-05-08T08:52:59.505 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057761380000000001, 2024-05-08T08:52:59.505 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:59.505 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:59.505 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:59.505 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:59.505 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:59.505 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:59.505 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7492 2024-05-08T08:52:59.506 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.506 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:59.506 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.506 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:59.506 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744253+0000", 2024-05-08T08:52:59.506 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:59.506 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.506 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.506 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:59.506 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744253+0000", 2024-05-08T08:52:59.507 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:59.507 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.507 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.507 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:59.507 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.745991+0000", 2024-05-08T08:52:59.507 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001738021 2024-05-08T08:52:59.507 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.507 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.507 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:59.507 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.745997+0000", 2024-05-08T08:52:59.507 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5300000000000004e-06 2024-05-08T08:52:59.507 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.507 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.507 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:59.508 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.746006+0000", 2024-05-08T08:52:59.508 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2210000000000003e-06 2024-05-08T08:52:59.508 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.508 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.508 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:59.508 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.746026+0000", 2024-05-08T08:52:59.508 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9487000000000002e-05 2024-05-08T08:52:59.508 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.508 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.508 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:59.508 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.801815+0000", 2024-05-08T08:52:59.508 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.055789196999999999 2024-05-08T08:52:59.508 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.508 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.509 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:59.509 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.801908+0000", 2024-05-08T08:52:59.509 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3320999999999996e-05 2024-05-08T08:52:59.509 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.509 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.509 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:59.509 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.802015+0000", 2024-05-08T08:52:59.509 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000106603 2024-05-08T08:52:59.509 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:59.509 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:59.509 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:59.509 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.509 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.509 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7499 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 368640~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:59.509 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.746209+0000", 2024-05-08T08:52:59.510 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 21.750760227000001, 2024-05-08T08:52:59.510 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057256452999999999, 2024-05-08T08:52:59.510 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:59.510 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:59.510 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:59.510 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:59.510 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:59.510 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:59.510 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7499 2024-05-08T08:52:59.510 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.510 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:59.510 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.510 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:59.511 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.746209+0000", 2024-05-08T08:52:59.511 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:59.511 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.511 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.511 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:59.511 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.746209+0000", 2024-05-08T08:52:59.511 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:59.511 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.511 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.511 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:59.511 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.746210+0000", 2024-05-08T08:52:59.511 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2100000000000002e-07 2024-05-08T08:52:59.511 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.511 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.511 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:59.512 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.746212+0000", 2024-05-08T08:52:59.512 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.922e-06 2024-05-08T08:52:59.512 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.512 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.512 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:59.512 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.746221+0000", 2024-05-08T08:52:59.512 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2900000000000008e-06 2024-05-08T08:52:59.512 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.512 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.512 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:59.512 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.746227+0000", 2024-05-08T08:52:59.512 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3740000000000003e-06 2024-05-08T08:52:59.512 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.512 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.513 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:59.513 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.803304+0000", 2024-05-08T08:52:59.513 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057076239000000001 2024-05-08T08:52:59.513 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.513 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.513 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:59.513 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.803380+0000", 2024-05-08T08:52:59.513 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.6309000000000004e-05 2024-05-08T08:52:59.513 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.513 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.513 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:59.513 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.803466+0000", 2024-05-08T08:52:59.513 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.5897999999999996e-05 2024-05-08T08:52:59.513 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:59.514 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:59.514 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:59.514 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.514 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.514 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7500 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 409600~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:59.514 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.746237+0000", 2024-05-08T08:52:59.514 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 21.750732644999999, 2024-05-08T08:52:59.514 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057429380000000002, 2024-05-08T08:52:59.514 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:59.514 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:59.514 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:59.514 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:59.515 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:59.515 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:59.515 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7500 2024-05-08T08:52:59.515 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.515 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:59.515 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.515 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:59.515 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.746237+0000", 2024-05-08T08:52:59.515 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:59.515 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.515 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.515 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:59.515 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.746237+0000", 2024-05-08T08:52:59.516 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:59.516 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.516 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.516 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:59.516 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.747959+0000", 2024-05-08T08:52:59.516 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001721888 2024-05-08T08:52:59.516 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.516 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.516 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:59.516 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.747964+0000", 2024-05-08T08:52:59.517 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0030000000000002e-06 2024-05-08T08:52:59.517 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.517 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.517 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:59.517 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.747976+0000", 2024-05-08T08:52:59.517 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2656999999999999e-05 2024-05-08T08:52:59.517 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.517 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.517 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:59.517 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.747983+0000", 2024-05-08T08:52:59.517 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.866e-06 2024-05-08T08:52:59.518 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.518 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.518 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:59.518 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.803495+0000", 2024-05-08T08:52:59.518 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.055511392999999999 2024-05-08T08:52:59.518 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.518 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.518 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:59.518 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.803574+0000", 2024-05-08T08:52:59.518 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9215000000000006e-05 2024-05-08T08:52:59.518 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.518 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.518 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:59.518 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.803666+0000", 2024-05-08T08:52:59.519 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2357999999999999e-05 2024-05-08T08:52:59.519 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:59.519 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:59.519 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:59.519 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.519 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.519 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7508 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 737280~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:59.519 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.748180+0000", 2024-05-08T08:52:59.519 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 21.748789588000001, 2024-05-08T08:52:59.519 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057419809000000002, 2024-05-08T08:52:59.519 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:59.519 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:59.519 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:59.519 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:59.520 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:59.520 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:59.520 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7508 2024-05-08T08:52:59.520 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.520 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:59.520 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.520 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:59.520 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748180+0000", 2024-05-08T08:52:59.520 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:59.520 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.520 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.520 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:59.520 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748180+0000", 2024-05-08T08:52:59.520 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:59.520 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.521 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.521 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:59.521 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748180+0000", 2024-05-08T08:52:59.521 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4499999999999997e-07 2024-05-08T08:52:59.521 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.521 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.521 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:59.521 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748183+0000", 2024-05-08T08:52:59.521 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1660000000000001e-06 2024-05-08T08:52:59.521 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.521 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.521 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:59.521 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748197+0000", 2024-05-08T08:52:59.521 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.4877e-05 2024-05-08T08:52:59.521 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.522 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.522 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:59.522 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748203+0000", 2024-05-08T08:52:59.522 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7470000000000002e-06 2024-05-08T08:52:59.522 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.522 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.522 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:59.522 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.805398+0000", 2024-05-08T08:52:59.522 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057194513000000002 2024-05-08T08:52:59.522 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.522 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.522 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:59.522 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.805544+0000", 2024-05-08T08:52:59.523 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000146522 2024-05-08T08:52:59.523 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.523 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.523 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:59.523 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.805600+0000", 2024-05-08T08:52:59.523 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5538999999999998e-05 2024-05-08T08:52:59.523 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:59.523 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:59.523 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:59.523 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.523 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.523 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7509 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 778240~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:59.523 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.748211+0000", 2024-05-08T08:52:59.523 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 21.748758518999999, 2024-05-08T08:52:59.524 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057619034999999999, 2024-05-08T08:52:59.524 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:59.524 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:59.524 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:59.524 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:59.524 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:59.524 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:59.524 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7509 2024-05-08T08:52:59.524 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.524 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:59.524 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.524 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:59.524 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748211+0000", 2024-05-08T08:52:59.525 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:59.525 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.525 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.525 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:59.525 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748211+0000", 2024-05-08T08:52:59.525 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:59.525 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.525 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.525 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:59.525 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748212+0000", 2024-05-08T08:52:59.526 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8100000000000003e-07 2024-05-08T08:52:59.526 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.526 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.526 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:59.526 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748214+0000", 2024-05-08T08:52:59.526 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4669999999999998e-06 2024-05-08T08:52:59.526 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.526 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.526 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:59.526 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748221+0000", 2024-05-08T08:52:59.526 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5769999999999999e-06 2024-05-08T08:52:59.526 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.526 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.526 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:59.527 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748225+0000", 2024-05-08T08:52:59.527 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8060000000000002e-06 2024-05-08T08:52:59.527 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.527 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.527 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:59.527 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.805636+0000", 2024-05-08T08:52:59.527 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057410309 2024-05-08T08:52:59.527 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.527 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.527 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:59.527 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.805723+0000", 2024-05-08T08:52:59.528 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7435999999999994e-05 2024-05-08T08:52:59.528 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.528 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.528 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:59.528 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.805830+0000", 2024-05-08T08:52:59.528 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000106959 2024-05-08T08:52:59.528 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:59.528 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:59.528 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:59.528 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.529 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.529 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7510 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 819200~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:59.529 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.748232+0000", 2024-05-08T08:52:59.529 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 21.748737489, 2024-05-08T08:52:59.529 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057710421999999997, 2024-05-08T08:52:59.529 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:59.529 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:59.529 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:59.529 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:59.529 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:59.529 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:59.529 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7510 2024-05-08T08:52:59.529 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.529 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:59.529 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.529 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:59.530 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748232+0000", 2024-05-08T08:52:59.530 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:59.530 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.530 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.530 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:59.530 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748232+0000", 2024-05-08T08:52:59.530 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:59.530 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.530 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.530 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:59.530 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.749959+0000", 2024-05-08T08:52:59.530 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001727 2024-05-08T08:52:59.530 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.531 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.531 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:59.531 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.749964+0000", 2024-05-08T08:52:59.531 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4700000000000004e-06 2024-05-08T08:52:59.531 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.531 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.531 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:59.531 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.749984+0000", 2024-05-08T08:52:59.531 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0068000000000002e-05 2024-05-08T08:52:59.531 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.531 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.531 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:59.531 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.749993+0000", 2024-05-08T08:52:59.531 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4720000000000001e-06 2024-05-08T08:52:59.531 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.532 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.532 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:59.532 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.805846+0000", 2024-05-08T08:52:59.532 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.055852770000000003 2024-05-08T08:52:59.532 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.532 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.532 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:59.532 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.805890+0000", 2024-05-08T08:52:59.532 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4323999999999998e-05 2024-05-08T08:52:59.532 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.532 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.532 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:59.532 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.805942+0000", 2024-05-08T08:52:59.532 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2318000000000003e-05 2024-05-08T08:52:59.532 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:59.533 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:59.533 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:59.533 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.533 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.533 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7515 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 1024000~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:59.533 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.750135+0000", 2024-05-08T08:52:59.533 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 21.746834112999998, 2024-05-08T08:52:59.533 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057188125999999999, 2024-05-08T08:52:59.533 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:59.533 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:59.533 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:59.533 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:59.533 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:59.533 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:59.533 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7515 2024-05-08T08:52:59.534 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.534 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:59.534 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.534 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:59.534 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750135+0000", 2024-05-08T08:52:59.534 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:59.534 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.534 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.534 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:59.534 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750135+0000", 2024-05-08T08:52:59.534 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:59.534 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.534 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.535 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:59.535 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750136+0000", 2024-05-08T08:52:59.535 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8100000000000003e-07 2024-05-08T08:52:59.535 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.535 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.535 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:59.535 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750138+0000", 2024-05-08T08:52:59.535 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.339e-06 2024-05-08T08:52:59.535 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.535 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.536 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:59.536 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750147+0000", 2024-05-08T08:52:59.536 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7669999999999992e-06 2024-05-08T08:52:59.536 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.536 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.536 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:59.536 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750153+0000", 2024-05-08T08:52:59.536 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5029999999999997e-06 2024-05-08T08:52:59.536 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.536 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.536 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:59.536 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.807145+0000", 2024-05-08T08:52:59.536 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.056992174 2024-05-08T08:52:59.537 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.537 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.537 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:59.537 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.807231+0000", 2024-05-08T08:52:59.537 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.6347000000000005e-05 2024-05-08T08:52:59.537 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.537 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.537 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:59.537 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.807324+0000", 2024-05-08T08:52:59.537 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2515000000000004e-05 2024-05-08T08:52:59.537 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:59.537 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:59.537 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:59.537 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.538 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.538 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7516 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 1064960~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:59.538 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.750162+0000", 2024-05-08T08:52:59.538 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 21.746807283999999, 2024-05-08T08:52:59.538 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057363971999999999, 2024-05-08T08:52:59.538 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:59.538 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:59.538 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:59.538 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:59.538 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:59.538 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:59.539 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7516 2024-05-08T08:52:59.539 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.539 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:59.539 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.539 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:59.539 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750162+0000", 2024-05-08T08:52:59.539 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:59.539 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.539 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.539 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:59.539 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750162+0000", 2024-05-08T08:52:59.539 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:59.539 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.539 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.540 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:59.540 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750163+0000", 2024-05-08T08:52:59.540 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.13e-07 2024-05-08T08:52:59.540 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.540 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.540 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:59.540 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750165+0000", 2024-05-08T08:52:59.540 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2309999999999999e-06 2024-05-08T08:52:59.540 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.540 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.540 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:59.540 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750174+0000", 2024-05-08T08:52:59.540 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4740000000000005e-06 2024-05-08T08:52:59.540 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.540 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.541 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:59.541 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750180+0000", 2024-05-08T08:52:59.541 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4760000000000003e-06 2024-05-08T08:52:59.541 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.541 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.541 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:59.541 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.807355+0000", 2024-05-08T08:52:59.541 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057175029000000002 2024-05-08T08:52:59.541 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.541 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.541 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:59.541 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.807437+0000", 2024-05-08T08:52:59.541 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.2268999999999994e-05 2024-05-08T08:52:59.541 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.541 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.542 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:59.542 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.807526+0000", 2024-05-08T08:52:59.542 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.8880000000000003e-05 2024-05-08T08:52:59.542 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:59.542 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:59.542 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:59.542 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.542 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.542 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7517 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 1105920~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:59.542 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.750188+0000", 2024-05-08T08:52:59.542 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 21.746781791, 2024-05-08T08:52:59.543 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057547099999999997, 2024-05-08T08:52:59.543 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:59.543 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:59.543 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:59.543 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:59.543 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:59.543 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:59.543 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7517 2024-05-08T08:52:59.543 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.543 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:59.543 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.543 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:59.544 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750188+0000", 2024-05-08T08:52:59.544 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:59.544 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.544 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.544 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:59.544 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750188+0000", 2024-05-08T08:52:59.544 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:59.544 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.544 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.544 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:59.544 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750188+0000", 2024-05-08T08:52:59.544 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7199999999999999e-07 2024-05-08T08:52:59.544 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.544 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.545 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:59.545 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750190+0000", 2024-05-08T08:52:59.545 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.154e-06 2024-05-08T08:52:59.545 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.545 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.545 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:59.545 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750198+0000", 2024-05-08T08:52:59.545 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8320000000000006e-06 2024-05-08T08:52:59.545 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.545 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.545 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:59.545 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750203+0000", 2024-05-08T08:52:59.546 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1900000000000003e-06 2024-05-08T08:52:59.546 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.546 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.546 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:59.546 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.807555+0000", 2024-05-08T08:52:59.546 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057351148999999997 2024-05-08T08:52:59.546 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.546 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.546 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:59.546 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.807640+0000", 2024-05-08T08:52:59.546 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.5853999999999995e-05 2024-05-08T08:52:59.547 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.547 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.547 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:59.547 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.807735+0000", 2024-05-08T08:52:59.547 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4449000000000003e-05 2024-05-08T08:52:59.547 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:59.547 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:59.547 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:59.547 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.547 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.547 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7518 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 1146880~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:59.547 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.750212+0000", 2024-05-08T08:52:59.547 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 21.746757990999999, 2024-05-08T08:52:59.547 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057664963999999999, 2024-05-08T08:52:59.547 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:59.548 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:59.548 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:59.548 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:59.548 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:59.548 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:59.548 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7518 2024-05-08T08:52:59.548 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.548 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:59.548 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.548 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:59.548 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750212+0000", 2024-05-08T08:52:59.549 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:59.549 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.549 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.549 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:59.549 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750212+0000", 2024-05-08T08:52:59.549 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:59.549 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.549 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.549 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:59.549 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750212+0000", 2024-05-08T08:52:59.549 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.46e-07 2024-05-08T08:52:59.549 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.549 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.549 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:59.550 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750220+0000", 2024-05-08T08:52:59.550 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5610000000000001e-06 2024-05-08T08:52:59.550 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.550 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.550 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:59.550 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750228+0000", 2024-05-08T08:52:59.550 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3180000000000007e-06 2024-05-08T08:52:59.550 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.550 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.550 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:59.550 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750234+0000", 2024-05-08T08:52:59.550 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0209999999999996e-06 2024-05-08T08:52:59.550 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.551 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.551 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:59.551 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.807765+0000", 2024-05-08T08:52:59.551 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057531481000000002 2024-05-08T08:52:59.551 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.551 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.551 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:59.551 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.807815+0000", 2024-05-08T08:52:59.551 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9282e-05 2024-05-08T08:52:59.551 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.551 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.551 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:59.551 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.807877+0000", 2024-05-08T08:52:59.551 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1854999999999996e-05 2024-05-08T08:52:59.551 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:59.552 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:59.552 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:59.552 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.552 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.552 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7519 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 1187840~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:59.552 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.750261+0000", 2024-05-08T08:52:59.552 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 21.746708505000001, 2024-05-08T08:52:59.552 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057872519999999997, 2024-05-08T08:52:59.552 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:59.552 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:59.552 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:59.552 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:59.552 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:59.552 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:59.552 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7519 2024-05-08T08:52:59.553 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.553 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:59.553 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.553 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:59.553 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750261+0000", 2024-05-08T08:52:59.553 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:59.553 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.553 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.553 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:59.553 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750261+0000", 2024-05-08T08:52:59.553 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:59.553 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.553 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.553 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:59.554 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750262+0000", 2024-05-08T08:52:59.554 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7000000000000005e-07 2024-05-08T08:52:59.554 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.554 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.554 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:59.554 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750265+0000", 2024-05-08T08:52:59.554 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4589999999999998e-06 2024-05-08T08:52:59.554 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.554 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.554 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:59.554 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750274+0000", 2024-05-08T08:52:59.554 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.5459999999999992e-06 2024-05-08T08:52:59.555 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.555 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.555 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:59.555 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750280+0000", 2024-05-08T08:52:59.555 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0950000000000001e-06 2024-05-08T08:52:59.555 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.555 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.555 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:59.555 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.807917+0000", 2024-05-08T08:52:59.555 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057637715999999999 2024-05-08T08:52:59.555 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.556 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.556 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:59.556 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.808013+0000", 2024-05-08T08:52:59.556 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5569999999999995e-05 2024-05-08T08:52:59.556 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.556 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.556 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:59.556 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.808134+0000", 2024-05-08T08:52:59.556 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012056400000000001 2024-05-08T08:52:59.556 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:59.556 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:59.556 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:59.556 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.557 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.557 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7520 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 1228800~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:59.557 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.750289+0000", 2024-05-08T08:52:59.557 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 21.746680060999999, 2024-05-08T08:52:59.557 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.058133788999999998, 2024-05-08T08:52:59.557 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:59.557 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:59.557 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:59.557 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:59.557 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:59.557 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:59.557 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7520 2024-05-08T08:52:59.558 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.558 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:59.558 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.558 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:59.558 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750289+0000", 2024-05-08T08:52:59.558 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:59.558 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.558 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.558 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:59.558 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750289+0000", 2024-05-08T08:52:59.559 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:59.559 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.559 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.559 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:59.559 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.751976+0000", 2024-05-08T08:52:59.559 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0016865389999999999 2024-05-08T08:52:59.559 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.559 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.559 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:59.559 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.751981+0000", 2024-05-08T08:52:59.559 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5719999999999996e-06 2024-05-08T08:52:59.559 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.560 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.560 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:59.560 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.751991+0000", 2024-05-08T08:52:59.560 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0788000000000001e-05 2024-05-08T08:52:59.560 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.560 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.560 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:59.560 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.751999+0000", 2024-05-08T08:52:59.560 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2690000000000002e-06 2024-05-08T08:52:59.560 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.560 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.560 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:59.560 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.808183+0000", 2024-05-08T08:52:59.560 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.056184686999999997 2024-05-08T08:52:59.561 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.561 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.561 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:59.561 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.808296+0000", 2024-05-08T08:52:59.561 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000112754 2024-05-08T08:52:59.561 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.561 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.561 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:59.561 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.808423+0000", 2024-05-08T08:52:59.561 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012718000000000001 2024-05-08T08:52:59.561 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:59.561 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:59.561 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:59.561 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.562 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.562 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7524 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 1392640~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:59.562 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.752121+0000", 2024-05-08T08:52:59.562 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 21.744849012, 2024-05-08T08:52:59.562 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057199521000000003, 2024-05-08T08:52:59.562 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:59.562 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:59.562 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:59.562 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:59.562 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:59.563 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:59.563 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7524 2024-05-08T08:52:59.563 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.563 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:59.563 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.563 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:59.563 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752121+0000", 2024-05-08T08:52:59.563 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:59.563 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.563 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.563 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:59.563 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752121+0000", 2024-05-08T08:52:59.563 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:59.563 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.564 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.564 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:59.564 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752121+0000", 2024-05-08T08:52:59.564 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5999999999999999e-07 2024-05-08T08:52:59.564 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.564 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.564 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:59.564 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752123+0000", 2024-05-08T08:52:59.564 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3499999999999999e-06 2024-05-08T08:52:59.564 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.564 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.564 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:59.564 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752132+0000", 2024-05-08T08:52:59.565 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.5539999999999998e-06 2024-05-08T08:52:59.565 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.565 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.565 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:59.565 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752138+0000", 2024-05-08T08:52:59.565 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3640000000000004e-06 2024-05-08T08:52:59.565 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.565 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.565 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:59.565 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.809210+0000", 2024-05-08T08:52:59.565 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057071469999999999 2024-05-08T08:52:59.565 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.565 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.565 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:59.565 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.809258+0000", 2024-05-08T08:52:59.566 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8755000000000003e-05 2024-05-08T08:52:59.566 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.566 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.566 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:59.566 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.809320+0000", 2024-05-08T08:52:59.566 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1568000000000004e-05 2024-05-08T08:52:59.566 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:59.566 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:59.566 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:59.566 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.566 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.566 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7525 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 1433600~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:59.566 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.752146+0000", 2024-05-08T08:52:59.567 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 21.744823967999999, 2024-05-08T08:52:59.567 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057400072000000003, 2024-05-08T08:52:59.567 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:59.567 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:59.567 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:59.567 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:59.567 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:59.567 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:59.567 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7525 2024-05-08T08:52:59.567 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.567 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:59.567 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.567 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:59.567 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752146+0000", 2024-05-08T08:52:59.568 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:59.568 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.568 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.568 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:59.568 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752146+0000", 2024-05-08T08:52:59.568 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:59.568 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.568 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.568 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:59.568 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752146+0000", 2024-05-08T08:52:59.568 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0400000000000002e-07 2024-05-08T08:52:59.568 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.568 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.569 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:59.569 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752148+0000", 2024-05-08T08:52:59.569 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3209999999999999e-06 2024-05-08T08:52:59.569 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.569 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.569 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:59.569 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752155+0000", 2024-05-08T08:52:59.569 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6109999999999999e-06 2024-05-08T08:52:59.569 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.569 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.570 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:59.570 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752161+0000", 2024-05-08T08:52:59.570 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.489e-06 2024-05-08T08:52:59.570 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.570 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.570 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:59.570 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.809360+0000", 2024-05-08T08:52:59.570 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057198860999999997 2024-05-08T08:52:59.570 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.570 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.570 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:59.570 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.809450+0000", 2024-05-08T08:52:59.570 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0136000000000006e-05 2024-05-08T08:52:59.571 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.571 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.571 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:59.571 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.809546+0000", 2024-05-08T08:52:59.571 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5249999999999998e-05 2024-05-08T08:52:59.571 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:59.571 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:59.571 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:59.571 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.571 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.571 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7526 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 1474560~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:59.571 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.752168+0000", 2024-05-08T08:52:59.571 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 21.744801254999999, 2024-05-08T08:52:59.571 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057516818999999997, 2024-05-08T08:52:59.571 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:59.572 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:59.572 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:59.572 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:59.572 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:59.572 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:59.572 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7526 2024-05-08T08:52:59.572 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.572 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:59.572 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.572 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:59.572 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752168+0000", 2024-05-08T08:52:59.572 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:59.572 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.572 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.573 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:59.573 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752168+0000", 2024-05-08T08:52:59.573 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:59.573 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.573 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.573 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:59.573 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752169+0000", 2024-05-08T08:52:59.573 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4799999999999999e-07 2024-05-08T08:52:59.573 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.573 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.573 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:59.573 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752171+0000", 2024-05-08T08:52:59.573 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.289e-06 2024-05-08T08:52:59.574 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.574 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.574 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:59.574 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752177+0000", 2024-05-08T08:52:59.574 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3690000000000003e-06 2024-05-08T08:52:59.574 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.574 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.574 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:59.574 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752183+0000", 2024-05-08T08:52:59.574 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.152e-06 2024-05-08T08:52:59.574 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.574 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.575 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:59.575 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.809574+0000", 2024-05-08T08:52:59.575 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057391075 2024-05-08T08:52:59.575 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.575 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.575 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:59.575 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.809624+0000", 2024-05-08T08:52:59.575 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0673000000000001e-05 2024-05-08T08:52:59.575 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.575 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.575 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:59.575 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.809685+0000", 2024-05-08T08:52:59.575 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0813000000000002e-05 2024-05-08T08:52:59.575 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:59.575 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:59.576 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:59.576 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.576 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.576 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7527 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 1515520~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:59.576 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.752189+0000", 2024-05-08T08:52:59.576 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 21.744780327000001, 2024-05-08T08:52:59.576 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057722168999999997, 2024-05-08T08:52:59.576 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:59.576 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:59.576 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:59.577 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:59.577 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:59.577 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:59.577 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7527 2024-05-08T08:52:59.577 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.577 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:59.577 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.577 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:59.577 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752189+0000", 2024-05-08T08:52:59.577 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:59.577 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.577 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.577 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:59.577 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752189+0000", 2024-05-08T08:52:59.578 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:59.578 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.578 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.578 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:59.578 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752190+0000", 2024-05-08T08:52:59.578 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8799999999999998e-07 2024-05-08T08:52:59.578 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.578 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.578 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:59.578 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752192+0000", 2024-05-08T08:52:59.578 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9759999999999998e-06 2024-05-08T08:52:59.578 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.578 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.579 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:59.579 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752199+0000", 2024-05-08T08:52:59.579 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3289999999999996e-06 2024-05-08T08:52:59.579 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.579 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.579 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:59.579 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752205+0000", 2024-05-08T08:52:59.579 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2759999999999997e-06 2024-05-08T08:52:59.579 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.579 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.579 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:59.580 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.809726+0000", 2024-05-08T08:52:59.580 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057521249000000003 2024-05-08T08:52:59.580 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.580 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.580 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:59.580 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.809815+0000", 2024-05-08T08:52:59.580 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.8987999999999996e-05 2024-05-08T08:52:59.580 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.580 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.580 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:59.580 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.809911+0000", 2024-05-08T08:52:59.580 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5963000000000003e-05 2024-05-08T08:52:59.580 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:59.580 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:59.581 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:59.581 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.581 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.581 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7528 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 1556480~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:59.581 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.752212+0000", 2024-05-08T08:52:59.581 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 21.744757617000001, 2024-05-08T08:52:59.581 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057904956, 2024-05-08T08:52:59.581 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:59.581 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:59.581 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:59.581 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:59.581 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:59.581 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:59.581 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7528 2024-05-08T08:52:59.581 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.581 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:59.582 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.582 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:59.582 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752212+0000", 2024-05-08T08:52:59.582 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:59.582 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.582 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.582 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:59.582 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752212+0000", 2024-05-08T08:52:59.582 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:59.582 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.582 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.582 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:59.582 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752212+0000", 2024-05-08T08:52:59.583 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7800000000000002e-07 2024-05-08T08:52:59.583 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.583 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.583 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:59.583 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752215+0000", 2024-05-08T08:52:59.583 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2120000000000002e-06 2024-05-08T08:52:59.583 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.583 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.583 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:59.583 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752222+0000", 2024-05-08T08:52:59.583 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.6329999999999996e-06 2024-05-08T08:52:59.583 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.583 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.584 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:59.584 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752227+0000", 2024-05-08T08:52:59.584 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9289999999999997e-06 2024-05-08T08:52:59.584 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.584 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.584 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:59.584 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.809966+0000", 2024-05-08T08:52:59.584 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057738554999999997 2024-05-08T08:52:59.584 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.584 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.584 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:59.584 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.810052+0000", 2024-05-08T08:52:59.584 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.6540999999999997e-05 2024-05-08T08:52:59.584 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.585 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.585 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:59.585 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.810117+0000", 2024-05-08T08:52:59.585 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4608000000000002e-05 2024-05-08T08:52:59.585 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:59.585 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:59.585 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:59.585 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.585 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.585 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7529 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 1597440~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:52:59.585 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.752234+0000", 2024-05-08T08:52:59.585 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 21.744735603999999, 2024-05-08T08:52:59.585 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.058133851, 2024-05-08T08:52:59.585 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:52:59.585 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:52:59.586 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:52:59.586 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:52:59.586 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:52:59.586 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:52:59.586 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7529 2024-05-08T08:52:59.586 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.586 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:52:59.586 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.586 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:52:59.586 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752234+0000", 2024-05-08T08:52:59.587 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:59.587 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.587 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.587 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:52:59.587 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752234+0000", 2024-05-08T08:52:59.587 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:52:59.587 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.587 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.587 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:52:59.587 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.753959+0000", 2024-05-08T08:52:59.587 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0017251409999999999 2024-05-08T08:52:59.587 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.587 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.587 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:52:59.588 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.753963+0000", 2024-05-08T08:52:59.588 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2860000000000001e-06 2024-05-08T08:52:59.588 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.588 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.588 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:52:59.588 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.753973+0000", 2024-05-08T08:52:59.588 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5119999999999997e-06 2024-05-08T08:52:59.588 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.588 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.588 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:52:59.588 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.753979+0000", 2024-05-08T08:52:59.588 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3589999999999996e-06 2024-05-08T08:52:59.588 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.588 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.589 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:52:59.589 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.810182+0000", 2024-05-08T08:52:59.589 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.056202941999999999 2024-05-08T08:52:59.589 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.589 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.589 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:52:59.589 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.810270+0000", 2024-05-08T08:52:59.589 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7338999999999998e-05 2024-05-08T08:52:59.589 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:52:59.589 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:52:59.589 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:52:59.589 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.810368+0000", 2024-05-08T08:52:59.589 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8271999999999997e-05 2024-05-08T08:52:59.589 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:59.590 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:59.590 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:59.590 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:52:59.590 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:52:59.590 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:52:59.592 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:52:59.592 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:52:59.594+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:52:59.693 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:52:59.696+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:52:59.698 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:52:59.698 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:52:59.698 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:52:59.699 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:52:59.707 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:52:59.794 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:52:59.797+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:52:59.851 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:52:59.851 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:52:59.851 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:52:59.851 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:52:59.851 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:52:59.860 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:52:59.894 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:52:59.897+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:52:59.968 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:52:59.969 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:52:59.969 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:52:59.969 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:52:59.969 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:59.969 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4742 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1832960~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:59.969 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.585513+0000", 2024-05-08T08:52:59.969 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 28.385326095, 2024-05-08T08:52:59.969 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035588763000000002, 2024-05-08T08:52:59.969 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:59.969 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:59.969 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:59.969 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:59.969 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:59.969 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:59.970 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4742 2024-05-08T08:52:59.970 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:59.970 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:59.970 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:59.970 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:59.970 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585513+0000", 2024-05-08T08:52:59.970 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:59.970 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:59.970 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:59.970 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:59.970 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585513+0000", 2024-05-08T08:52:59.971 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:59.971 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:59.971 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:59.971 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:59.971 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585514+0000", 2024-05-08T08:52:59.971 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6100000000000001e-07 2024-05-08T08:52:59.971 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:59.971 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:59.971 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:59.971 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585516+0000", 2024-05-08T08:52:59.971 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.526e-06 2024-05-08T08:52:59.971 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:59.971 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:59.972 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:59.972 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585523+0000", 2024-05-08T08:52:59.972 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.562e-06 2024-05-08T08:52:59.972 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:59.972 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:59.972 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:59.972 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585528+0000", 2024-05-08T08:52:59.972 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7990000000000001e-06 2024-05-08T08:52:59.972 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:59.972 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:59.972 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:59.973 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.620939+0000", 2024-05-08T08:52:59.973 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035411419 2024-05-08T08:52:59.973 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:59.973 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:59.973 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:59.973 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621014+0000", 2024-05-08T08:52:59.973 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5017000000000004e-05 2024-05-08T08:52:59.973 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:59.973 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:59.973 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:59.973 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621102+0000", 2024-05-08T08:52:59.973 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7979000000000006e-05 2024-05-08T08:52:59.973 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:59.974 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:59.974 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:59.974 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:59.974 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:59.974 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4743 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1849344~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:59.974 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.585628+0000", 2024-05-08T08:52:59.974 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 28.385211208000001, 2024-05-08T08:52:59.974 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035659962000000003, 2024-05-08T08:52:59.974 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:59.974 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:59.974 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:59.974 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:59.974 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:59.974 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:59.975 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4743 2024-05-08T08:52:59.975 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:59.975 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:59.975 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:59.975 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:59.975 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585628+0000", 2024-05-08T08:52:59.975 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:59.975 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:59.975 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:59.975 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:59.975 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585628+0000", 2024-05-08T08:52:59.975 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:59.975 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:59.975 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:59.975 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:59.976 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585629+0000", 2024-05-08T08:52:59.976 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2300000000000001e-07 2024-05-08T08:52:59.976 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:59.976 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:59.976 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:59.976 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585632+0000", 2024-05-08T08:52:59.976 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1370000000000002e-06 2024-05-08T08:52:59.976 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:59.976 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:59.976 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:59.976 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585641+0000", 2024-05-08T08:52:59.977 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5220000000000007e-06 2024-05-08T08:52:59.977 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:59.977 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:59.977 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:59.977 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585646+0000", 2024-05-08T08:52:59.977 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.609e-06 2024-05-08T08:52:59.977 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:59.977 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:59.977 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:59.977 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621132+0000", 2024-05-08T08:52:59.977 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035485472999999997 2024-05-08T08:52:59.977 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:59.978 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:59.978 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:59.978 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621204+0000", 2024-05-08T08:52:59.978 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2259000000000001e-05 2024-05-08T08:52:59.978 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:59.978 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:59.978 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:59.978 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621288+0000", 2024-05-08T08:52:59.978 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4338999999999993e-05 2024-05-08T08:52:59.978 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:59.978 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:59.978 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:59.978 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:59.978 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:59.979 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4746 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1865728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:59.979 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.585656+0000", 2024-05-08T08:52:59.979 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 28.385183525999999, 2024-05-08T08:52:59.979 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035815909, 2024-05-08T08:52:59.979 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:59.979 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:59.979 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:59.979 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:59.979 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:59.979 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:59.979 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4746 2024-05-08T08:52:59.979 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:59.979 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:59.979 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:59.980 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:59.980 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585656+0000", 2024-05-08T08:52:59.980 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:59.980 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:59.980 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:59.980 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:59.980 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585656+0000", 2024-05-08T08:52:59.980 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:59.980 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:59.980 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:59.980 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:59.980 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585656+0000", 2024-05-08T08:52:59.980 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.08e-07 2024-05-08T08:52:59.980 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:59.980 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:59.981 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:59.981 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585658+0000", 2024-05-08T08:52:59.981 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0889999999999998e-06 2024-05-08T08:52:59.981 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:59.981 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:59.981 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:59.981 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585664+0000", 2024-05-08T08:52:59.981 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3990000000000003e-06 2024-05-08T08:52:59.981 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:59.981 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:59.981 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:59.981 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585668+0000", 2024-05-08T08:52:59.982 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3170000000000003e-06 2024-05-08T08:52:59.982 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:59.982 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:59.982 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:59.982 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621316+0000", 2024-05-08T08:52:59.982 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035647908999999998 2024-05-08T08:52:59.982 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:59.982 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:59.982 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:59.982 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621389+0000", 2024-05-08T08:52:59.982 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2531000000000001e-05 2024-05-08T08:52:59.982 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:59.982 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:59.983 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:59.983 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621472+0000", 2024-05-08T08:52:59.983 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3256000000000004e-05 2024-05-08T08:52:59.983 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:59.983 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:59.983 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:59.983 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:59.983 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:59.983 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4748 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1882112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:59.983 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.585697+0000", 2024-05-08T08:52:59.983 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 28.385142080000001, 2024-05-08T08:52:59.984 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035962485000000002, 2024-05-08T08:52:59.984 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:59.984 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:59.984 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:59.984 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:59.984 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:59.984 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:59.984 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4748 2024-05-08T08:52:59.984 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:59.984 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:59.984 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:59.985 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:59.985 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585697+0000", 2024-05-08T08:52:59.985 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:59.985 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:59.985 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:59.985 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:59.985 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585697+0000", 2024-05-08T08:52:59.985 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:59.985 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:59.985 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:59.985 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:59.985 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585698+0000", 2024-05-08T08:52:59.985 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0900000000000001e-07 2024-05-08T08:52:59.985 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:59.986 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:59.986 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:59.986 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585701+0000", 2024-05-08T08:52:59.986 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0149999999999999e-06 2024-05-08T08:52:59.986 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:59.986 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:59.986 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:59.986 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585712+0000", 2024-05-08T08:52:59.986 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0655e-05 2024-05-08T08:52:59.986 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:59.986 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:59.986 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:59.986 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585717+0000", 2024-05-08T08:52:59.986 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3800000000000002e-06 2024-05-08T08:52:59.986 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:59.987 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:59.987 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:59.987 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621502+0000", 2024-05-08T08:52:59.987 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035784559000000001 2024-05-08T08:52:59.987 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:59.987 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:59.987 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:59.987 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621575+0000", 2024-05-08T08:52:59.987 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3269999999999995e-05 2024-05-08T08:52:59.987 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:59.987 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:59.987 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:59.988 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621660+0000", 2024-05-08T08:52:59.988 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4997e-05 2024-05-08T08:52:59.988 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:59.988 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:59.988 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:59.988 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:59.988 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:59.988 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4750 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1898496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:59.988 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.585810+0000", 2024-05-08T08:52:59.988 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 28.385029249999999, 2024-05-08T08:52:59.988 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036034928000000001, 2024-05-08T08:52:59.988 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:59.988 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:59.989 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:59.989 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:59.989 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:59.989 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:59.989 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4750 2024-05-08T08:52:59.989 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:59.989 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:59.989 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:59.989 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:59.989 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585810+0000", 2024-05-08T08:52:59.989 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:59.989 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:59.990 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:59.990 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:59.990 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585810+0000", 2024-05-08T08:52:59.990 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:59.990 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:59.990 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:59.990 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:59.990 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585811+0000", 2024-05-08T08:52:59.990 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6499999999999999e-07 2024-05-08T08:52:59.990 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:59.990 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:59.990 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:59.990 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585814+0000", 2024-05-08T08:52:59.990 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1709999999999998e-06 2024-05-08T08:52:59.991 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:59.991 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:59.991 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:59.991 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585822+0000", 2024-05-08T08:52:59.991 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8909999999999995e-06 2024-05-08T08:52:59.991 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:59.991 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:59.991 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:59.991 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585827+0000", 2024-05-08T08:52:59.991 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5289999999999999e-06 2024-05-08T08:52:59.991 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:59.991 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:59.992 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:59.992 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621688+0000", 2024-05-08T08:52:59.992 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035861111000000001 2024-05-08T08:52:59.992 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:59.992 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:59.992 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:59.992 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621761+0000", 2024-05-08T08:52:59.992 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2211000000000004e-05 2024-05-08T08:52:59.992 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:59.992 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:59.992 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:59.993 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621845+0000", 2024-05-08T08:52:59.993 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4450000000000006e-05 2024-05-08T08:52:59.993 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:59.993 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:59.993 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:59.993 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:59.993 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:59.993 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4752 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1914880~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:59.993 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.585858+0000", 2024-05-08T08:52:59.993 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 28.384981281000002, 2024-05-08T08:52:59.993 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036180081000000003, 2024-05-08T08:52:59.993 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:59.994 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:59.994 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:59.994 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:59.994 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:59.994 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:59.994 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4752 2024-05-08T08:52:59.994 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:59.994 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:59.994 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:59.994 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:59.994 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585858+0000", 2024-05-08T08:52:59.995 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:59.995 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:59.995 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:59.995 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:52:59.995 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585858+0000", 2024-05-08T08:52:59.995 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:52:59.995 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:59.995 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:59.995 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:52:59.995 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585859+0000", 2024-05-08T08:52:59.995 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.82e-07 2024-05-08T08:52:59.995 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:59.995 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:59.995 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:52:59.995 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585861+0000", 2024-05-08T08:52:59.996 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.695e-06 2024-05-08T08:52:59.996 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:59.996 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:59.996 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:52:59.996 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585869+0000", 2024-05-08T08:52:59.996 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0299999999999994e-06 2024-05-08T08:52:59.996 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:59.996 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:59.996 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:52:59.996 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585874+0000", 2024-05-08T08:52:59.996 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9620000000000001e-06 2024-05-08T08:52:59.996 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:59.996 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:59.997 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:52:59.997 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621878+0000", 2024-05-08T08:52:59.997 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036004043999999999 2024-05-08T08:52:59.997 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:59.997 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:59.997 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:52:59.997 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621954+0000", 2024-05-08T08:52:59.997 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5414000000000007e-05 2024-05-08T08:52:59.997 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:59.997 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:59.998 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:52:59.998 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622038+0000", 2024-05-08T08:52:59.998 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4454000000000002e-05 2024-05-08T08:52:59.998 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:59.998 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:52:59.998 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:52:59.998 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:59.998 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:59.998 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4753 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1931264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:52:59.998 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.585947+0000", 2024-05-08T08:52:59.998 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 28.384892633, 2024-05-08T08:52:59.998 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036267992999999998, 2024-05-08T08:52:59.998 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:52:59.998 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:52:59.999 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:52:59.999 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:52:59.999 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:52:59.999 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:52:59.999 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4753 2024-05-08T08:52:59.999 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:52:59.999 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:52:59.999 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:52:59.999 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:52:59.999 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585947+0000", 2024-05-08T08:53:00.000 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:00.000 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.000 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.000 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:00.000 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585947+0000", 2024-05-08T08:53:00.000 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:00.000 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.000 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.000 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:00.000 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585948+0000", 2024-05-08T08:53:00.000 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6000000000000002e-07 2024-05-08T08:53:00.000 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.000 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.000 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:00.001 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585952+0000", 2024-05-08T08:53:00.001 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7090000000000001e-06 2024-05-08T08:53:00.001 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.001 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.001 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:00.001 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585963+0000", 2024-05-08T08:53:00.001 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0740000000000001e-05 2024-05-08T08:53:00.001 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.001 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.001 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:00.001 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585970+0000", 2024-05-08T08:53:00.001 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3309999999999998e-06 2024-05-08T08:53:00.001 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.001 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.001 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:00.002 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622065+0000", 2024-05-08T08:53:00.002 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036094676999999999 2024-05-08T08:53:00.002 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.002 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.002 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:00.002 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622132+0000", 2024-05-08T08:53:00.002 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7214999999999999e-05 2024-05-08T08:53:00.002 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.002 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.002 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:00.002 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622215+0000", 2024-05-08T08:53:00.002 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2460999999999995e-05 2024-05-08T08:53:00.002 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:00.002 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:00.003 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:00.003 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.003 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.003 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4756 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1947648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:00.003 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586018+0000", 2024-05-08T08:53:00.003 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 28.384820989000001, 2024-05-08T08:53:00.003 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036382335000000002, 2024-05-08T08:53:00.003 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:00.003 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:00.003 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:00.003 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:00.003 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:00.004 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:00.004 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4756 2024-05-08T08:53:00.004 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.004 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:00.004 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.004 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:00.004 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586018+0000", 2024-05-08T08:53:00.004 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:00.004 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.004 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.004 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:00.005 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586018+0000", 2024-05-08T08:53:00.005 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:00.005 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.005 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.005 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:00.005 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586019+0000", 2024-05-08T08:53:00.005 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9999999999999997e-07 2024-05-08T08:53:00.005 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.005 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.005 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:00.005 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586023+0000", 2024-05-08T08:53:00.005 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7330000000000001e-06 2024-05-08T08:53:00.005 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.005 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.006 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:00.006 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586031+0000", 2024-05-08T08:53:00.006 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4940000000000003e-06 2024-05-08T08:53:00.006 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.006 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.006 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:00.006 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586038+0000", 2024-05-08T08:53:00.006 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2539999999999998e-06 2024-05-08T08:53:00.006 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.006 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.006 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:00.006 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622243+0000", 2024-05-08T08:53:00.006 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036205009000000003 2024-05-08T08:53:00.007 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.007 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.007 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:00.007 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622313+0000", 2024-05-08T08:53:00.007 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9912000000000004e-05 2024-05-08T08:53:00.007 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.007 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.007 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:00.007 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622401+0000", 2024-05-08T08:53:00.007 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8232999999999994e-05 2024-05-08T08:53:00.007 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:00.007 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:00.007 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:00.007 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.007 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.008 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4758 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1964032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:00.008 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586080+0000", 2024-05-08T08:53:00.008 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 28.38475957, 2024-05-08T08:53:00.008 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036547659000000003, 2024-05-08T08:53:00.008 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:00.008 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:00.008 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:00.008 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:00.008 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:00.008 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:00.008 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4758 2024-05-08T08:53:00.008 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.008 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:00.008 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.009 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:00.009 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586080+0000", 2024-05-08T08:53:00.009 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:00.009 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.009 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.009 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:00.009 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586080+0000", 2024-05-08T08:53:00.009 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:00.010 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.010 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.010 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:00.010 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586081+0000", 2024-05-08T08:53:00.010 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5499999999999998e-07 2024-05-08T08:53:00.010 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.010 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.010 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:00.010 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586084+0000", 2024-05-08T08:53:00.010 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9919999999999999e-06 2024-05-08T08:53:00.010 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.010 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.010 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:00.010 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586091+0000", 2024-05-08T08:53:00.011 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5440000000000001e-06 2024-05-08T08:53:00.011 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.011 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.011 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:00.011 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586098+0000", 2024-05-08T08:53:00.011 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8639999999999998e-06 2024-05-08T08:53:00.011 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.011 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.011 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:00.011 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622442+0000", 2024-05-08T08:53:00.011 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036344110999999998 2024-05-08T08:53:00.012 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.012 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.012 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:00.012 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622516+0000", 2024-05-08T08:53:00.012 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4089000000000002e-05 2024-05-08T08:53:00.012 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.012 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.012 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:00.012 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622628+0000", 2024-05-08T08:53:00.012 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00011140400000000001 2024-05-08T08:53:00.012 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:00.012 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:00.012 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:00.012 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.012 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.013 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4760 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1980416~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:00.013 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586178+0000", 2024-05-08T08:53:00.013 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 28.384660958000001, 2024-05-08T08:53:00.013 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036647526999999999, 2024-05-08T08:53:00.013 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:00.013 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:00.013 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:00.013 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:00.013 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:00.013 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:00.013 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4760 2024-05-08T08:53:00.013 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.013 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:00.014 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.014 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:00.014 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586178+0000", 2024-05-08T08:53:00.014 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:00.014 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.014 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.014 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:00.014 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586178+0000", 2024-05-08T08:53:00.014 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:00.014 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.014 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.015 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:00.015 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586179+0000", 2024-05-08T08:53:00.015 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8599999999999997e-07 2024-05-08T08:53:00.015 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.015 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.015 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:00.015 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586182+0000", 2024-05-08T08:53:00.015 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.174e-06 2024-05-08T08:53:00.015 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.015 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.015 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:00.016 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586191+0000", 2024-05-08T08:53:00.016 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1869999999999997e-06 2024-05-08T08:53:00.016 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.016 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.016 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:00.016 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586197+0000", 2024-05-08T08:53:00.016 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2090000000000002e-06 2024-05-08T08:53:00.016 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.016 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.016 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:00.016 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622657+0000", 2024-05-08T08:53:00.016 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036459793999999997 2024-05-08T08:53:00.016 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.017 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.017 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:00.017 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622725+0000", 2024-05-08T08:53:00.017 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8374999999999995e-05 2024-05-08T08:53:00.017 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.017 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.017 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:00.017 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622826+0000", 2024-05-08T08:53:00.017 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010100200000000001 2024-05-08T08:53:00.017 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:00.017 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:00.017 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:00.017 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.017 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.018 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4762 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1996800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:00.018 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586299+0000", 2024-05-08T08:53:00.018 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 28.384540627, 2024-05-08T08:53:00.018 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036720653999999998, 2024-05-08T08:53:00.018 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:00.018 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:00.018 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:00.018 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:00.018 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:00.018 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:00.018 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4762 2024-05-08T08:53:00.018 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.018 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:00.018 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.019 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:00.019 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586299+0000", 2024-05-08T08:53:00.019 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:00.019 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.019 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.019 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:00.019 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586299+0000", 2024-05-08T08:53:00.019 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:00.019 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.019 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.019 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:00.019 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586299+0000", 2024-05-08T08:53:00.020 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.99e-07 2024-05-08T08:53:00.020 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.020 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.020 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:00.020 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586303+0000", 2024-05-08T08:53:00.020 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1949999999999999e-06 2024-05-08T08:53:00.020 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.020 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.020 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:00.021 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586318+0000", 2024-05-08T08:53:00.021 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5401999999999999e-05 2024-05-08T08:53:00.021 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.021 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.021 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:00.021 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586324+0000", 2024-05-08T08:53:00.021 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8119999999999996e-06 2024-05-08T08:53:00.021 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.021 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.021 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:00.021 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622853+0000", 2024-05-08T08:53:00.022 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036529013999999999 2024-05-08T08:53:00.022 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.022 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.022 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:00.022 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622924+0000", 2024-05-08T08:53:00.022 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1654000000000002e-05 2024-05-08T08:53:00.022 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.022 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.022 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:00.022 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623019+0000", 2024-05-08T08:53:00.022 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4977999999999998e-05 2024-05-08T08:53:00.022 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:00.022 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:00.022 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:00.022 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.023 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.023 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4764 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 2013184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:00.023 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586506+0000", 2024-05-08T08:53:00.023 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 28.384333521999999, 2024-05-08T08:53:00.023 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036859467, 2024-05-08T08:53:00.023 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:00.023 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:00.023 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:00.023 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:00.023 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:00.023 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:00.023 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4764 2024-05-08T08:53:00.023 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.023 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:00.024 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.024 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:00.024 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586506+0000", 2024-05-08T08:53:00.024 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:00.024 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.024 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.024 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:00.024 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586506+0000", 2024-05-08T08:53:00.024 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:00.025 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.025 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.025 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:00.025 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586507+0000", 2024-05-08T08:53:00.025 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9100000000000003e-07 2024-05-08T08:53:00.025 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.025 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.025 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:00.025 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586512+0000", 2024-05-08T08:53:00.025 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9470000000000002e-06 2024-05-08T08:53:00.026 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.026 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.026 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:00.026 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586521+0000", 2024-05-08T08:53:00.026 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3379999999999994e-06 2024-05-08T08:53:00.026 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.027 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.027 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:00.027 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586528+0000", 2024-05-08T08:53:00.027 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1289999999999998e-06 2024-05-08T08:53:00.027 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.027 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.028 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:00.028 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623095+0000", 2024-05-08T08:53:00.028 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036566411 2024-05-08T08:53:00.028 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.028 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.028 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:00.028 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623221+0000", 2024-05-08T08:53:00.029 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012622299999999999 2024-05-08T08:53:00.029 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.029 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.029 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:00.029 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623365+0000", 2024-05-08T08:53:00.029 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000144628 2024-05-08T08:53:00.030 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:00.030 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:00.030 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:00.030 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.030 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.031 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4766 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 2029568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:00.031 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586543+0000", 2024-05-08T08:53:00.031 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 28.384296034999998, 2024-05-08T08:53:00.031 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.037033635000000002, 2024-05-08T08:53:00.031 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:00.031 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:00.031 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:00.032 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:00.032 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:00.032 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:00.032 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4766 2024-05-08T08:53:00.032 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.032 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:00.033 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.033 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:00.033 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586543+0000", 2024-05-08T08:53:00.033 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:00.033 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.033 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.034 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:00.034 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586543+0000", 2024-05-08T08:53:00.034 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:00.034 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.034 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.035 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:00.035 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586544+0000", 2024-05-08T08:53:00.035 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3099999999999998e-07 2024-05-08T08:53:00.035 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.035 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.035 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:00.035 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586547+0000", 2024-05-08T08:53:00.035 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.751e-06 2024-05-08T08:53:00.035 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.036 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.036 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:00.036 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586553+0000", 2024-05-08T08:53:00.036 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2870000000000001e-06 2024-05-08T08:53:00.036 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.036 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.036 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:00.037 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586558+0000", 2024-05-08T08:53:00.037 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5709999999999997e-06 2024-05-08T08:53:00.037 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.037 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.037 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:00.037 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623416+0000", 2024-05-08T08:53:00.037 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036857613999999997 2024-05-08T08:53:00.038 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.038 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.038 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:00.038 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623489+0000", 2024-05-08T08:53:00.038 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3172999999999999e-05 2024-05-08T08:53:00.038 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.038 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.038 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:00.038 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623577+0000", 2024-05-08T08:53:00.038 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7807999999999997e-05 2024-05-08T08:53:00.038 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:00.038 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:00.038 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:00.039 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.039 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.039 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4768 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 2045952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:00.039 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586566+0000", 2024-05-08T08:53:00.039 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 28.384273729, 2024-05-08T08:53:00.039 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.03726206, 2024-05-08T08:53:00.039 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:00.039 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:00.039 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:00.039 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:00.039 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:00.039 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:00.039 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4768 2024-05-08T08:53:00.039 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.040 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:00.040 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.040 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:00.040 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586566+0000", 2024-05-08T08:53:00.040 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:00.040 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.040 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.040 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:00.040 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586566+0000", 2024-05-08T08:53:00.040 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:00.040 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.040 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.040 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:00.040 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586566+0000", 2024-05-08T08:53:00.040 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8299999999999998e-07 2024-05-08T08:53:00.041 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.041 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.041 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:00.041 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586568+0000", 2024-05-08T08:53:00.041 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.063e-06 2024-05-08T08:53:00.041 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.041 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.041 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:00.041 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586574+0000", 2024-05-08T08:53:00.041 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8300000000000001e-06 2024-05-08T08:53:00.041 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.041 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.041 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:00.041 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586579+0000", 2024-05-08T08:53:00.041 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7559999999999999e-06 2024-05-08T08:53:00.042 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.042 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.042 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:00.042 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623612+0000", 2024-05-08T08:53:00.042 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.037033041000000003 2024-05-08T08:53:00.042 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.042 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.042 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:00.042 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623699+0000", 2024-05-08T08:53:00.042 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6817000000000006e-05 2024-05-08T08:53:00.042 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.042 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.042 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:00.043 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623828+0000", 2024-05-08T08:53:00.043 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012917000000000001 2024-05-08T08:53:00.043 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:00.043 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:00.043 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:00.043 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.043 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.043 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4770 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 2062336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:00.043 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586585+0000", 2024-05-08T08:53:00.043 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 28.384254546000001, 2024-05-08T08:53:00.043 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.037449891999999999, 2024-05-08T08:53:00.043 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:00.043 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:00.043 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:00.044 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:00.044 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:00.044 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:00.044 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4770 2024-05-08T08:53:00.044 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.044 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:00.044 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.044 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:00.044 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586585+0000", 2024-05-08T08:53:00.044 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:00.044 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.044 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.044 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:00.044 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586585+0000", 2024-05-08T08:53:00.045 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:00.045 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.045 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.045 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:00.045 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586585+0000", 2024-05-08T08:53:00.045 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:53:00.045 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.045 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.045 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:00.045 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586587+0000", 2024-05-08T08:53:00.045 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.181e-06 2024-05-08T08:53:00.045 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.045 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.045 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:00.045 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586592+0000", 2024-05-08T08:53:00.046 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9710000000000003e-06 2024-05-08T08:53:00.046 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.046 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.046 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:00.046 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586598+0000", 2024-05-08T08:53:00.046 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.118e-06 2024-05-08T08:53:00.046 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.046 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.046 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:00.046 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623862+0000", 2024-05-08T08:53:00.046 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.037264711999999998 2024-05-08T08:53:00.046 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.047 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.047 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:00.047 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623941+0000", 2024-05-08T08:53:00.047 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9084000000000004e-05 2024-05-08T08:53:00.047 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.047 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.047 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:00.047 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.624035+0000", 2024-05-08T08:53:00.048 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3456999999999996e-05 2024-05-08T08:53:00.048 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:00.048 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:00.048 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:00.048 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.048 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.048 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4772 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 2078720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:00.048 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586621+0000", 2024-05-08T08:53:00.049 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 28.384218598, 2024-05-08T08:53:00.049 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.037650391999999998, 2024-05-08T08:53:00.049 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:00.049 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:00.049 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:00.049 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:00.049 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:00.049 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:00.050 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4772 2024-05-08T08:53:00.050 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.050 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:00.050 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.050 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:00.050 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586621+0000", 2024-05-08T08:53:00.050 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:00.050 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.051 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.051 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:00.051 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586621+0000", 2024-05-08T08:53:00.051 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:00.051 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.051 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.052 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:00.052 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586621+0000", 2024-05-08T08:53:00.052 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:53:00.052 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.052 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.052 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:00.053 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586623+0000", 2024-05-08T08:53:00.053 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9240000000000001e-06 2024-05-08T08:53:00.053 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.053 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.053 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:00.053 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586628+0000", 2024-05-08T08:53:00.053 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6410000000000001e-06 2024-05-08T08:53:00.054 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.054 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.054 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:00.054 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586633+0000", 2024-05-08T08:53:00.054 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.763e-06 2024-05-08T08:53:00.054 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.054 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.055 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:00.055 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.624077+0000", 2024-05-08T08:53:00.055 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.037444709999999999 2024-05-08T08:53:00.055 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.055 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.055 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:00.055 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.624164+0000", 2024-05-08T08:53:00.055 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6769999999999998e-05 2024-05-08T08:53:00.056 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.056 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.056 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:00.056 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.624271+0000", 2024-05-08T08:53:00.056 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010723 2024-05-08T08:53:00.056 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:00.057 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:00.057 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:00.057 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.057 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.057 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4774 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 2095104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:00.057 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586640+0000", 2024-05-08T08:53:00.057 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 28.384199460000001, 2024-05-08T08:53:00.058 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.037827178000000003, 2024-05-08T08:53:00.058 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:00.058 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:00.058 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:00.058 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:00.058 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:00.059 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:00.059 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4774 2024-05-08T08:53:00.059 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.059 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:00.059 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.059 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:00.059 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586640+0000", 2024-05-08T08:53:00.059 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:00.059 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.059 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.059 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:00.060 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586640+0000", 2024-05-08T08:53:00.060 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:00.060 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.060 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.060 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:00.060 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586640+0000", 2024-05-08T08:53:00.060 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5199999999999998e-07 2024-05-08T08:53:00.060 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.060 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.060 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:00.060 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586643+0000", 2024-05-08T08:53:00.060 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.232e-06 2024-05-08T08:53:00.060 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.060 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.060 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:00.061 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586649+0000", 2024-05-08T08:53:00.061 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9660000000000001e-06 2024-05-08T08:53:00.061 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.061 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.061 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:00.061 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586653+0000", 2024-05-08T08:53:00.061 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7350000000000004e-06 2024-05-08T08:53:00.061 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.061 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.061 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:00.061 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.624303+0000", 2024-05-08T08:53:00.061 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.037649647000000001 2024-05-08T08:53:00.061 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.061 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.062 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:00.062 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.624373+0000", 2024-05-08T08:53:00.062 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0415999999999998e-05 2024-05-08T08:53:00.062 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.062 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.062 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:00.062 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.624467+0000", 2024-05-08T08:53:00.062 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3830000000000001e-05 2024-05-08T08:53:00.062 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:00.062 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:00.062 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:00.062 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.062 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.063 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4621.0:9089 3.1f e33/18 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:head v 33'117, mlcod=31'115)", 2024-05-08T08:53:00.063 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:53:00.063 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 27.451221248, 2024-05-08T08:53:00.063 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.111244524, 2024-05-08T08:53:00.063 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:00.063 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:00.063 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:53:00.063 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:00.063 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.063 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:00.063 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:53:00.063 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:00.063 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.063 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.063 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:00.064 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:53:00.064 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:00.064 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.064 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.064 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:00.064 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:53:00.064 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5900000000000001e-07 2024-05-08T08:53:00.064 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.064 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.064 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:00.064 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539600+0000", 2024-05-08T08:53:00.064 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.019981364000000001 2024-05-08T08:53:00.064 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.064 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.065 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:00.065 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539624+0000", 2024-05-08T08:53:00.065 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4043e-05 2024-05-08T08:53:00.065 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.065 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.065 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:00.065 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539635+0000", 2024-05-08T08:53:00.065 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1511e-05 2024-05-08T08:53:00.065 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.065 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.065 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:00.065 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539728+0000", 2024-05-08T08:53:00.065 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2498999999999996e-05 2024-05-08T08:53:00.065 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.065 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.066 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:00.066 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539746+0000", 2024-05-08T08:53:00.066 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8289000000000001e-05 2024-05-08T08:53:00.066 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.066 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.066 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:53:00.066 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.630822+0000", 2024-05-08T08:53:00.066 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.091076283999999993 2024-05-08T08:53:00.066 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.067 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.067 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:00.067 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.630863+0000", 2024-05-08T08:53:00.067 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0374999999999999e-05 2024-05-08T08:53:00.067 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:00.067 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:00.067 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:00.067 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.067 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.067 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4795.0:230 3.1f e40/18 3:f8db1a56:::rbd_data.11402b26385.0000000000000000:head v 40'171, mlcod=39'168)", 2024-05-08T08:53:00.067 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:45.927773+0000", 2024-05-08T08:53:00.067 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 14.043066165000001, 2024-05-08T08:53:00.067 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045892405999999997, 2024-05-08T08:53:00.067 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:00.068 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:00.068 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:53:00.068 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:00.068 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.068 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:00.068 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.927773+0000", 2024-05-08T08:53:00.068 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:00.068 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.068 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.068 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:00.068 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.927773+0000", 2024-05-08T08:53:00.068 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:00.068 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.068 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.068 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:00.069 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.927774+0000", 2024-05-08T08:53:00.069 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0500000000000002e-07 2024-05-08T08:53:00.069 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.069 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.069 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:00.069 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.931581+0000", 2024-05-08T08:53:00.069 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0038071799999999999 2024-05-08T08:53:00.069 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.069 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.069 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:00.069 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.931599+0000", 2024-05-08T08:53:00.070 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7479e-05 2024-05-08T08:53:00.070 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.070 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.070 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:00.070 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.931624+0000", 2024-05-08T08:53:00.070 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5372999999999999e-05 2024-05-08T08:53:00.070 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.070 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.070 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:00.070 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.931698+0000", 2024-05-08T08:53:00.070 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4043999999999998e-05 2024-05-08T08:53:00.070 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.071 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.071 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:00.071 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.931725+0000", 2024-05-08T08:53:00.071 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6883e-05 2024-05-08T08:53:00.071 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.071 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.071 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:53:00.071 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.973623+0000", 2024-05-08T08:53:00.071 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.041898283000000001 2024-05-08T08:53:00.071 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.071 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.071 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:00.071 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.973666+0000", 2024-05-08T08:53:00.071 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2358999999999999e-05 2024-05-08T08:53:00.072 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:00.072 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:00.072 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:00.072 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.072 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.072 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4195.0:4 3.11 3:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-05-08T08:53:00.072 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:49.764654+0000", 2024-05-08T08:53:00.072 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 10.206185706999999, 2024-05-08T08:53:00.072 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.30714275499999999, 2024-05-08T08:53:00.072 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:00.072 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:00.072 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-05-08T08:53:00.072 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:00.073 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4195", 2024-05-08T08:53:00.073 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/59179521", 2024-05-08T08:53:00.073 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4 2024-05-08T08:53:00.073 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.073 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:00.073 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.073 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:00.073 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764654+0000", 2024-05-08T08:53:00.074 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:00.074 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.074 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.074 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:00.074 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764654+0000", 2024-05-08T08:53:00.074 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:00.074 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.074 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.074 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:00.074 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764655+0000", 2024-05-08T08:53:00.074 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.452e-06 2024-05-08T08:53:00.075 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.075 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.075 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:00.075 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764661+0000", 2024-05-08T08:53:00.075 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5369999999999997e-06 2024-05-08T08:53:00.075 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.075 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.075 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:00.075 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764671+0000", 2024-05-08T08:53:00.075 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.028e-05 2024-05-08T08:53:00.075 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.076 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.076 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:00.076 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764680+0000", 2024-05-08T08:53:00.076 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0629999999999995e-06 2024-05-08T08:53:00.076 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.076 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.076 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:00.076 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764753+0000", 2024-05-08T08:53:00.076 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2679e-05 2024-05-08T08:53:00.076 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.076 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.076 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-05-08T08:53:00.076 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764765+0000", 2024-05-08T08:53:00.077 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2415000000000001e-05 2024-05-08T08:53:00.077 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.077 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.077 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:00.077 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:50.071692+0000", 2024-05-08T08:53:00.077 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.306927324 2024-05-08T08:53:00.077 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.077 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.077 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:00.077 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:50.071796+0000", 2024-05-08T08:53:00.077 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104005 2024-05-08T08:53:00.077 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:00.077 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:00.078 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:00.078 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:00.078 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:00.078 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:53:00.079 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:53:00.080 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:00.082+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:00.133 INFO:teuthology.orchestra.run.smithi059.stderr:set 3.4 pg_upmap mapping to [1,5] 2024-05-08T08:53:00.179 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:00.183+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:00.189 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:53:00.189 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:53:00.189 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:53:00.189 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:53:00.199 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:53:00.281 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:53:00.284+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:53:00.344 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:53:00.344 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:53:00.345 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:53:00.345 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:53:00.345 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:53:00.354 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:53:00.400 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:00.403+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:00.462 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:53:00.463 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:53:00.463 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:53:00.463 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:53:00.463 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.463 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22069 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 749568~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:00.464 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857624+0000", 2024-05-08T08:53:00.464 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 32.606572700000001, 2024-05-08T08:53:00.464 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046969958999999999, 2024-05-08T08:53:00.464 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:00.464 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:00.464 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:00.465 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:00.465 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:00.465 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:00.465 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22069 2024-05-08T08:53:00.465 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.465 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:00.466 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.466 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:00.466 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857624+0000", 2024-05-08T08:53:00.466 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:00.466 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.466 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.466 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:00.466 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857624+0000", 2024-05-08T08:53:00.467 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:00.467 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.467 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.467 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:00.467 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857625+0000", 2024-05-08T08:53:00.470 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.03e-07 2024-05-08T08:53:00.470 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.471 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.471 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:00.471 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857627+0000", 2024-05-08T08:53:00.471 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9369999999999998e-06 2024-05-08T08:53:00.471 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.471 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.472 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:00.472 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857633+0000", 2024-05-08T08:53:00.472 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1229999999999997e-06 2024-05-08T08:53:00.472 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.472 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.472 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:00.473 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857637+0000", 2024-05-08T08:53:00.473 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7820000000000001e-06 2024-05-08T08:53:00.473 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.473 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.473 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:00.473 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904457+0000", 2024-05-08T08:53:00.474 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046819715999999997 2024-05-08T08:53:00.474 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.474 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.474 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:00.474 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904524+0000", 2024-05-08T08:53:00.474 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6371000000000005e-05 2024-05-08T08:53:00.475 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.475 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.475 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:00.475 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904594+0000", 2024-05-08T08:53:00.475 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0627e-05 2024-05-08T08:53:00.476 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:00.476 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:00.476 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:00.476 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.476 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.476 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22071 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 765952~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:00.477 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857643+0000", 2024-05-08T08:53:00.477 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 32.606553617000003, 2024-05-08T08:53:00.477 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047106528000000002, 2024-05-08T08:53:00.477 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:00.477 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:00.477 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:00.478 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:00.478 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:00.478 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:00.478 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22071 2024-05-08T08:53:00.478 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.478 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:00.479 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.479 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:00.479 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857643+0000", 2024-05-08T08:53:00.479 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:00.479 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.479 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.480 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:00.480 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857643+0000", 2024-05-08T08:53:00.480 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:00.480 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.480 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.481 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:00.481 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857644+0000", 2024-05-08T08:53:00.481 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2000000000000001e-07 2024-05-08T08:53:00.481 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.481 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.481 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:00.482 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857646+0000", 2024-05-08T08:53:00.482 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0059999999999999e-06 2024-05-08T08:53:00.482 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.482 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.482 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:00.482 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857651+0000", 2024-05-08T08:53:00.483 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.84e-06 2024-05-08T08:53:00.483 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.483 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.483 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:00.483 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857657+0000", 2024-05-08T08:53:00.483 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5620000000000003e-06 2024-05-08T08:53:00.484 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.484 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.484 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:00.484 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904620+0000", 2024-05-08T08:53:00.484 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046962661000000003 2024-05-08T08:53:00.484 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.484 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.484 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:00.485 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904678+0000", 2024-05-08T08:53:00.485 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8656999999999998e-05 2024-05-08T08:53:00.485 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.485 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.485 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:00.485 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904750+0000", 2024-05-08T08:53:00.485 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1482000000000004e-05 2024-05-08T08:53:00.485 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:00.486 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:00.486 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:00.486 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.486 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.486 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22073 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 782336~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:00.486 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:53:00.486 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 32.606529608999999, 2024-05-08T08:53:00.486 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047244234000000003, 2024-05-08T08:53:00.487 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:00.487 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:00.487 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:00.487 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:00.487 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:00.487 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:00.487 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22073 2024-05-08T08:53:00.488 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.488 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:00.488 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.488 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:00.488 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:53:00.488 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:00.488 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.489 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.489 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:00.489 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:53:00.489 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:00.489 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.489 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.489 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:00.489 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857668+0000", 2024-05-08T08:53:00.489 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7800000000000002e-07 2024-05-08T08:53:00.490 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.490 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.490 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:00.490 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857670+0000", 2024-05-08T08:53:00.490 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0820000000000001e-06 2024-05-08T08:53:00.490 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.490 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.490 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:00.490 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857678+0000", 2024-05-08T08:53:00.490 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7589999999999997e-06 2024-05-08T08:53:00.490 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.490 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:00.491 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857691+0000", 2024-05-08T08:53:00.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3672999999999999e-05 2024-05-08T08:53:00.491 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:00.491 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904780+0000", 2024-05-08T08:53:00.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047088616999999999 2024-05-08T08:53:00.491 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:00.491 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904840+0000", 2024-05-08T08:53:00.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0356999999999998e-05 2024-05-08T08:53:00.491 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.492 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.492 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:00.492 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904912+0000", 2024-05-08T08:53:00.492 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1367999999999998e-05 2024-05-08T08:53:00.492 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:00.492 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:00.492 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:00.492 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.492 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.492 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22075 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 798720~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:00.492 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:53:00.492 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 32.606499212000003, 2024-05-08T08:53:00.492 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047384643999999997, 2024-05-08T08:53:00.492 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:00.493 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:00.493 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:00.493 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:00.493 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:00.493 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:00.493 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22075 2024-05-08T08:53:00.493 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.493 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:00.493 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.493 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:00.493 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:53:00.493 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:00.493 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.494 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.494 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:00.494 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:53:00.494 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:00.494 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.494 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.494 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:00.494 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:53:00.494 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7399999999999999e-07 2024-05-08T08:53:00.494 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.494 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.494 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:00.494 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857705+0000", 2024-05-08T08:53:00.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1980000000000004e-06 2024-05-08T08:53:00.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.495 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:00.495 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857713+0000", 2024-05-08T08:53:00.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7039999999999994e-06 2024-05-08T08:53:00.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.495 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:00.495 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857718+0000", 2024-05-08T08:53:00.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.865e-06 2024-05-08T08:53:00.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.496 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:00.496 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904941+0000", 2024-05-08T08:53:00.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047223198000000001 2024-05-08T08:53:00.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.496 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:00.496 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904999+0000", 2024-05-08T08:53:00.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8254999999999997e-05 2024-05-08T08:53:00.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.496 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:00.496 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905082+0000", 2024-05-08T08:53:00.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3049999999999999e-05 2024-05-08T08:53:00.497 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:00.497 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:00.497 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:00.497 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.497 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.497 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22077 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 815104~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:00.497 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:53:00.497 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 32.606461201000002, 2024-05-08T08:53:00.497 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047519448999999998, 2024-05-08T08:53:00.497 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:00.497 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:00.498 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:00.498 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:00.498 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:00.498 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:00.498 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22077 2024-05-08T08:53:00.498 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.498 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:00.498 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.498 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:00.498 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:53:00.498 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:00.499 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.499 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.499 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:00.499 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:53:00.499 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:00.499 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.499 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.499 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:00.499 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:53:00.499 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.99e-07 2024-05-08T08:53:00.499 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.500 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.500 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:00.500 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857739+0000", 2024-05-08T08:53:00.500 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3599999999999999e-06 2024-05-08T08:53:00.500 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.500 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.500 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:00.500 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857745+0000", 2024-05-08T08:53:00.500 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5139999999999998e-06 2024-05-08T08:53:00.500 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.500 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.501 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:00.501 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857750+0000", 2024-05-08T08:53:00.501 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8550000000000001e-06 2024-05-08T08:53:00.501 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.501 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.501 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:00.502 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905110+0000", 2024-05-08T08:53:00.502 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047360126000000002 2024-05-08T08:53:00.502 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.502 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.502 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:00.502 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905175+0000", 2024-05-08T08:53:00.502 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4689000000000004e-05 2024-05-08T08:53:00.502 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.502 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.502 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:00.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905255+0000", 2024-05-08T08:53:00.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0306e-05 2024-05-08T08:53:00.503 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:00.503 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:00.503 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:00.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.503 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22079 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 831488~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:00.503 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:53:00.503 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 32.606440653, 2024-05-08T08:53:00.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.04767499, 2024-05-08T08:53:00.503 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:00.503 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:00.504 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:00.504 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:00.504 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:00.504 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:00.504 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22079 2024-05-08T08:53:00.504 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.504 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:00.504 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.504 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:00.504 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:53:00.504 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:00.504 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.504 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.505 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:00.505 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:53:00.505 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:00.505 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.505 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.505 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:00.505 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857757+0000", 2024-05-08T08:53:00.505 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8299999999999997e-07 2024-05-08T08:53:00.505 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.505 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.505 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:00.505 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857759+0000", 2024-05-08T08:53:00.505 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1390000000000002e-06 2024-05-08T08:53:00.505 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.505 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.506 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:00.506 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857765+0000", 2024-05-08T08:53:00.506 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4609999999999996e-06 2024-05-08T08:53:00.506 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.506 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.506 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:00.506 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857770+0000", 2024-05-08T08:53:00.506 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6689999999999997e-06 2024-05-08T08:53:00.506 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.506 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.506 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:00.506 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905281+0000", 2024-05-08T08:53:00.507 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047510668999999998 2024-05-08T08:53:00.507 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.507 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.507 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:00.507 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905344+0000", 2024-05-08T08:53:00.507 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2797000000000002e-05 2024-05-08T08:53:00.507 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.507 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.507 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:00.507 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905431+0000", 2024-05-08T08:53:00.507 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7771999999999999e-05 2024-05-08T08:53:00.507 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:00.507 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:00.507 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:00.508 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.508 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.508 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22082 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 847872~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:00.508 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:53:00.508 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 32.606420962000001, 2024-05-08T08:53:00.508 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047831934999999999, 2024-05-08T08:53:00.508 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:00.508 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:00.508 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:00.508 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:00.508 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:00.508 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:00.508 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22082 2024-05-08T08:53:00.508 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.509 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:00.509 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.509 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:00.509 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:53:00.509 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:00.509 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.509 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.509 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:00.509 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:53:00.509 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:00.509 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.509 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.509 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:00.509 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:53:00.510 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:53:00.510 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.510 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.510 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:00.510 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857778+0000", 2024-05-08T08:53:00.510 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0789999999999999e-06 2024-05-08T08:53:00.510 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.510 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.510 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:00.510 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857784+0000", 2024-05-08T08:53:00.510 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0480000000000004e-06 2024-05-08T08:53:00.510 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.510 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.511 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:00.511 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857790+0000", 2024-05-08T08:53:00.511 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2719999999999997e-06 2024-05-08T08:53:00.511 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.511 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.511 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:00.511 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905458+0000", 2024-05-08T08:53:00.511 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047668470999999997 2024-05-08T08:53:00.511 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.511 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.511 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:00.512 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905521+0000", 2024-05-08T08:53:00.512 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2952000000000003e-05 2024-05-08T08:53:00.512 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.512 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.512 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:00.512 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905608+0000", 2024-05-08T08:53:00.512 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6759000000000001e-05 2024-05-08T08:53:00.512 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:00.512 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:00.512 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:00.512 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.512 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.512 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22084 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 864256~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:00.512 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:53:00.513 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 32.606397012000002, 2024-05-08T08:53:00.513 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047977748000000001, 2024-05-08T08:53:00.513 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:00.513 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:00.513 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:00.513 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:00.513 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:00.513 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:00.513 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22084 2024-05-08T08:53:00.513 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.513 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:00.513 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.513 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:00.513 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:53:00.514 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:00.514 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.514 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.514 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:00.514 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:53:00.514 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:00.514 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.514 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.515 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:00.515 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:53:00.515 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:53:00.515 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.515 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.515 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:00.515 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857802+0000", 2024-05-08T08:53:00.515 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0779999999999998e-06 2024-05-08T08:53:00.515 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.515 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.515 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:00.515 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857812+0000", 2024-05-08T08:53:00.515 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7939999999999997e-06 2024-05-08T08:53:00.515 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.516 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.516 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:00.516 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857817+0000", 2024-05-08T08:53:00.516 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8899999999999998e-06 2024-05-08T08:53:00.516 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.516 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.516 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:00.516 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905636+0000", 2024-05-08T08:53:00.516 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047819252 2024-05-08T08:53:00.516 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.516 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.516 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:00.517 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905700+0000", 2024-05-08T08:53:00.517 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4016999999999994e-05 2024-05-08T08:53:00.517 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.517 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.517 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:00.517 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905778+0000", 2024-05-08T08:53:00.517 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7306999999999997e-05 2024-05-08T08:53:00.517 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:00.517 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:00.517 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:00.517 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.517 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.517 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22086 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 880640~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:00.517 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:53:00.518 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 32.606373662000003, 2024-05-08T08:53:00.518 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048119149, 2024-05-08T08:53:00.518 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:00.518 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:00.518 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:00.518 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:00.518 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:00.518 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:00.518 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22086 2024-05-08T08:53:00.518 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.518 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:00.518 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.518 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:00.519 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:53:00.519 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:00.519 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.519 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.519 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:00.519 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:53:00.519 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:00.519 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.519 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.519 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:00.519 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857824+0000", 2024-05-08T08:53:00.519 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4200000000000002e-07 2024-05-08T08:53:00.519 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.519 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.519 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:00.520 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857826+0000", 2024-05-08T08:53:00.520 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0590000000000001e-06 2024-05-08T08:53:00.520 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.520 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.520 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:00.520 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857832+0000", 2024-05-08T08:53:00.520 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8029999999999999e-06 2024-05-08T08:53:00.520 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.520 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.520 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:00.520 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857837+0000", 2024-05-08T08:53:00.520 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7929999999999997e-06 2024-05-08T08:53:00.520 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.520 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.520 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:00.520 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905803+0000", 2024-05-08T08:53:00.520 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047965761000000003 2024-05-08T08:53:00.521 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.521 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.521 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:00.521 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905862+0000", 2024-05-08T08:53:00.521 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9203000000000002e-05 2024-05-08T08:53:00.521 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.521 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.521 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:00.521 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905942+0000", 2024-05-08T08:53:00.521 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0187999999999999e-05 2024-05-08T08:53:00.521 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:00.521 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:00.521 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:00.521 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.522 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.522 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22088 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 897024~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:00.522 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:53:00.522 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 32.606353609999999, 2024-05-08T08:53:00.522 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048256049000000002, 2024-05-08T08:53:00.522 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:00.522 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:00.522 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:00.522 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:00.522 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:00.522 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:00.522 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22088 2024-05-08T08:53:00.522 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.522 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:00.523 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.523 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:00.523 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:53:00.523 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:00.523 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.523 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.523 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:00.523 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:53:00.523 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:00.523 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.523 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.523 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:00.523 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857844+0000", 2024-05-08T08:53:00.523 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.46e-07 2024-05-08T08:53:00.523 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.524 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.524 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:00.524 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857846+0000", 2024-05-08T08:53:00.524 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0899999999999999e-06 2024-05-08T08:53:00.524 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.524 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.524 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:00.524 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857855+0000", 2024-05-08T08:53:00.524 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8219999999999996e-06 2024-05-08T08:53:00.524 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.524 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.524 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:00.524 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857860+0000", 2024-05-08T08:53:00.524 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2989999999999996e-06 2024-05-08T08:53:00.524 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.525 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.525 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:00.525 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905966+0000", 2024-05-08T08:53:00.525 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048106542000000002 2024-05-08T08:53:00.525 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.525 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.525 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:00.525 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906024+0000", 2024-05-08T08:53:00.525 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7958000000000003e-05 2024-05-08T08:53:00.525 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.525 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.525 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:00.525 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906099+0000", 2024-05-08T08:53:00.525 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4992000000000003e-05 2024-05-08T08:53:00.525 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:00.526 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:00.526 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:00.526 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.526 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.526 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22091 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 913408~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:00.526 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:53:00.526 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 32.606329850000002, 2024-05-08T08:53:00.526 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048395290000000001, 2024-05-08T08:53:00.526 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:00.526 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:00.526 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:00.526 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:00.526 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:00.526 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:00.527 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22091 2024-05-08T08:53:00.527 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.527 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:00.527 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.527 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:00.527 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:53:00.527 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:00.527 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.527 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.527 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:00.527 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:53:00.527 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:00.527 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.527 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.527 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:00.528 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:53:00.528 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4900000000000001e-07 2024-05-08T08:53:00.528 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.528 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.528 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:00.528 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857870+0000", 2024-05-08T08:53:00.528 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2510000000000001e-06 2024-05-08T08:53:00.528 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.528 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.528 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:00.528 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857877+0000", 2024-05-08T08:53:00.528 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0809999999999996e-06 2024-05-08T08:53:00.528 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.528 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.529 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:00.529 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857881+0000", 2024-05-08T08:53:00.529 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.724e-06 2024-05-08T08:53:00.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.529 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.529 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:00.529 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906127+0000", 2024-05-08T08:53:00.529 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048245345000000002 2024-05-08T08:53:00.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.529 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.529 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:00.529 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906190+0000", 2024-05-08T08:53:00.530 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2762000000000007e-05 2024-05-08T08:53:00.530 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.530 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.530 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:00.530 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906262+0000", 2024-05-08T08:53:00.530 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2778e-05 2024-05-08T08:53:00.530 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:00.530 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:00.530 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:00.530 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.530 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.530 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22093 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 929792~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:00.531 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:53:00.531 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 32.606307485999999, 2024-05-08T08:53:00.531 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048532875000000003, 2024-05-08T08:53:00.531 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:00.531 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:00.531 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:00.531 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:00.531 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:00.531 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:00.531 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22093 2024-05-08T08:53:00.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.531 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:00.531 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.531 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:00.532 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:53:00.532 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:00.532 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.532 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.532 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:00.532 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:53:00.532 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:00.532 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.532 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.532 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:00.532 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857890+0000", 2024-05-08T08:53:00.533 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-07 2024-05-08T08:53:00.533 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.533 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.533 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:00.533 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857899+0000", 2024-05-08T08:53:00.533 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6889999999999999e-06 2024-05-08T08:53:00.533 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.533 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.533 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:00.533 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857906+0000", 2024-05-08T08:53:00.533 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6490000000000002e-06 2024-05-08T08:53:00.533 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.533 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.533 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:00.534 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857917+0000", 2024-05-08T08:53:00.534 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0764999999999999e-05 2024-05-08T08:53:00.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.534 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.534 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:00.534 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906287+0000", 2024-05-08T08:53:00.534 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048370442 2024-05-08T08:53:00.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.534 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.534 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:00.534 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906349+0000", 2024-05-08T08:53:00.534 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1537000000000004e-05 2024-05-08T08:53:00.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.534 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.535 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:00.535 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906422+0000", 2024-05-08T08:53:00.535 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3431999999999998e-05 2024-05-08T08:53:00.535 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:00.535 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:00.535 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:00.535 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.535 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.535 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22095 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 946176~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:00.535 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:53:00.535 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 32.606272781999998, 2024-05-08T08:53:00.535 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048657593999999998, 2024-05-08T08:53:00.535 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:00.535 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:00.536 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:00.536 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:00.536 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:00.536 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:00.536 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22095 2024-05-08T08:53:00.536 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.536 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:00.536 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.536 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:00.536 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:53:00.536 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:00.536 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.536 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.536 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:00.537 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:53:00.537 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:00.537 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.537 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.537 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:00.537 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857925+0000", 2024-05-08T08:53:00.537 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3099999999999998e-07 2024-05-08T08:53:00.537 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.537 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.537 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:00.537 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857927+0000", 2024-05-08T08:53:00.537 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.34e-06 2024-05-08T08:53:00.537 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.537 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.538 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:00.538 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857938+0000", 2024-05-08T08:53:00.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1426e-05 2024-05-08T08:53:00.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.538 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:00.538 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857943+0000", 2024-05-08T08:53:00.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5140000000000003e-06 2024-05-08T08:53:00.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.538 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:00.538 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906448+0000", 2024-05-08T08:53:00.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048504641000000001 2024-05-08T08:53:00.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.539 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.539 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:00.539 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906510+0000", 2024-05-08T08:53:00.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2663000000000006e-05 2024-05-08T08:53:00.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.539 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.539 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:00.539 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906582+0000", 2024-05-08T08:53:00.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1478999999999998e-05 2024-05-08T08:53:00.539 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:00.539 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:00.539 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:00.540 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.540 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.540 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22097 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 962560~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:00.540 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:53:00.540 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 32.606245092999998, 2024-05-08T08:53:00.540 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048791353000000003, 2024-05-08T08:53:00.540 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:00.540 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:00.540 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:00.540 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:00.540 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:00.540 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:00.540 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22097 2024-05-08T08:53:00.540 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.540 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:00.540 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.541 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:00.541 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:53:00.541 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:00.541 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.541 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.541 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:00.541 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:53:00.541 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:00.541 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.541 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.541 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:00.541 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:53:00.541 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9200000000000002e-07 2024-05-08T08:53:00.541 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.541 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.541 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:00.542 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857954+0000", 2024-05-08T08:53:00.542 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1900000000000002e-06 2024-05-08T08:53:00.542 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.542 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.542 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:00.542 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857961+0000", 2024-05-08T08:53:00.542 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4989999999999999e-06 2024-05-08T08:53:00.542 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.542 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.542 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:00.542 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857971+0000", 2024-05-08T08:53:00.542 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0485999999999999e-05 2024-05-08T08:53:00.542 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.542 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.543 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:00.543 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906607+0000", 2024-05-08T08:53:00.543 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048635109000000003 2024-05-08T08:53:00.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.543 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:00.543 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906667+0000", 2024-05-08T08:53:00.543 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0439000000000002e-05 2024-05-08T08:53:00.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.543 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:00.543 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906743+0000", 2024-05-08T08:53:00.543 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6237999999999998e-05 2024-05-08T08:53:00.543 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:00.543 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:00.544 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:00.544 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.544 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.544 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22099 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 978944~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:00.544 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:53:00.544 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 32.606218933000001, 2024-05-08T08:53:00.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048932055000000002, 2024-05-08T08:53:00.544 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:00.544 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:00.544 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:00.544 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:00.544 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:00.544 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:00.544 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22099 2024-05-08T08:53:00.544 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.544 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:00.545 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.545 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:00.545 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:53:00.545 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:00.545 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.545 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.545 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:00.545 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:53:00.545 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:00.545 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.545 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.545 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:00.545 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:53:00.545 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.27e-07 2024-05-08T08:53:00.546 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.546 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.546 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:00.546 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857987+0000", 2024-05-08T08:53:00.546 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5890000000000003e-06 2024-05-08T08:53:00.546 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.546 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.546 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:00.546 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857994+0000", 2024-05-08T08:53:00.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6619999999999999e-06 2024-05-08T08:53:00.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.547 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:00.547 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857998+0000", 2024-05-08T08:53:00.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7110000000000003e-06 2024-05-08T08:53:00.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.547 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:00.547 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906772+0000", 2024-05-08T08:53:00.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048773415000000001 2024-05-08T08:53:00.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.547 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:00.547 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906834+0000", 2024-05-08T08:53:00.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1752000000000001e-05 2024-05-08T08:53:00.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:00.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906910+0000", 2024-05-08T08:53:00.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6499000000000001e-05 2024-05-08T08:53:00.548 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:00.548 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:00.549 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:00.549 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.549 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.549 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22101 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 995328~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:00.549 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:53:00.549 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 32.606191801999998, 2024-05-08T08:53:00.549 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049087364000000001, 2024-05-08T08:53:00.549 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:00.549 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:00.549 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:00.550 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:00.550 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:00.550 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:00.550 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22101 2024-05-08T08:53:00.550 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.550 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:00.550 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.550 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:00.550 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:53:00.550 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:00.550 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.550 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.550 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:00.551 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:53:00.551 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:00.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.551 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:00.551 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858006+0000", 2024-05-08T08:53:00.551 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1399999999999997e-07 2024-05-08T08:53:00.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.551 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:00.551 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858008+0000", 2024-05-08T08:53:00.551 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2060000000000001e-06 2024-05-08T08:53:00.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.552 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:00.552 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858017+0000", 2024-05-08T08:53:00.552 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1579999999999994e-06 2024-05-08T08:53:00.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.552 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.552 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:00.552 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858022+0000", 2024-05-08T08:53:00.552 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4199999999999998e-06 2024-05-08T08:53:00.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.552 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.552 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:00.552 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906940+0000", 2024-05-08T08:53:00.552 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048917627999999998 2024-05-08T08:53:00.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.552 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.553 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:00.553 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907002+0000", 2024-05-08T08:53:00.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2399999999999999e-05 2024-05-08T08:53:00.553 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.553 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.553 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:00.553 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907092+0000", 2024-05-08T08:53:00.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0137999999999997e-05 2024-05-08T08:53:00.553 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:00.553 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:00.553 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:00.553 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.553 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.553 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22103 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1011712~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:00.554 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:53:00.554 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 32.606167016999997, 2024-05-08T08:53:00.554 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049228542, 2024-05-08T08:53:00.554 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:00.554 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:00.554 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:00.554 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:00.554 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:00.554 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:00.554 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22103 2024-05-08T08:53:00.554 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.554 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:00.554 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.555 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:00.555 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:53:00.555 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:00.555 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.555 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.555 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:00.555 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:53:00.555 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:00.555 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.555 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.555 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:00.555 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:53:00.555 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4299999999999999e-07 2024-05-08T08:53:00.555 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.555 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.556 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:00.556 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858032+0000", 2024-05-08T08:53:00.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0590000000000001e-06 2024-05-08T08:53:00.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.556 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.556 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:00.556 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858037+0000", 2024-05-08T08:53:00.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.977e-06 2024-05-08T08:53:00.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.556 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.556 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:00.556 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858042+0000", 2024-05-08T08:53:00.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7650000000000001e-06 2024-05-08T08:53:00.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.557 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.557 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:00.557 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907118+0000", 2024-05-08T08:53:00.557 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049076147000000001 2024-05-08T08:53:00.557 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.557 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.557 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:00.557 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907180+0000", 2024-05-08T08:53:00.557 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1971999999999996e-05 2024-05-08T08:53:00.557 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.557 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.557 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:00.557 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907258+0000", 2024-05-08T08:53:00.557 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8279e-05 2024-05-08T08:53:00.558 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:00.558 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:00.558 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:00.558 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.558 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.558 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22105 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1028096~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:00.558 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:53:00.558 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 32.606147774, 2024-05-08T08:53:00.558 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049394609999999999, 2024-05-08T08:53:00.558 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:00.558 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:00.558 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:00.558 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:00.558 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:00.559 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:00.559 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22105 2024-05-08T08:53:00.559 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.559 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:00.559 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.559 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:00.559 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:53:00.559 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:00.559 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.559 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.559 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:00.559 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:53:00.559 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:00.559 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.559 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.560 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:00.560 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858050+0000", 2024-05-08T08:53:00.560 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5999999999999999e-07 2024-05-08T08:53:00.560 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.560 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.560 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:00.560 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858051+0000", 2024-05-08T08:53:00.560 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8870000000000001e-06 2024-05-08T08:53:00.560 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.560 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.560 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:00.560 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858057+0000", 2024-05-08T08:53:00.560 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8019999999999997e-06 2024-05-08T08:53:00.560 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.561 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.561 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:00.561 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858062+0000", 2024-05-08T08:53:00.561 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.561e-06 2024-05-08T08:53:00.561 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.561 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.561 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:00.561 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907289+0000", 2024-05-08T08:53:00.561 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049226957000000002 2024-05-08T08:53:00.561 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.561 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.561 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:00.561 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907356+0000", 2024-05-08T08:53:00.561 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6915000000000006e-05 2024-05-08T08:53:00.561 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.562 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.562 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:00.562 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907444+0000", 2024-05-08T08:53:00.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8127999999999994e-05 2024-05-08T08:53:00.562 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:00.562 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:00.562 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:00.562 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.562 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.562 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22107 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1044480~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:00.562 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:53:00.562 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 32.606129434000003, 2024-05-08T08:53:00.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049799942999999999, 2024-05-08T08:53:00.563 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:00.563 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:00.563 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:00.563 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:00.563 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:00.563 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:00.563 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22107 2024-05-08T08:53:00.563 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.563 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:00.563 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.563 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:00.563 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:53:00.563 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:00.563 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.563 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.564 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:00.564 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:53:00.564 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:00.564 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.564 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.564 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:00.564 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:53:00.564 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4799999999999999e-07 2024-05-08T08:53:00.564 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.564 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.564 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:00.564 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858070+0000", 2024-05-08T08:53:00.564 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.012e-06 2024-05-08T08:53:00.564 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.564 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.565 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:00.565 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858077+0000", 2024-05-08T08:53:00.565 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6290000000000001e-06 2024-05-08T08:53:00.565 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.565 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.565 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:00.565 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858083+0000", 2024-05-08T08:53:00.565 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8880000000000002e-06 2024-05-08T08:53:00.565 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.565 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.565 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:00.565 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907671+0000", 2024-05-08T08:53:00.565 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049587191000000003 2024-05-08T08:53:00.565 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.566 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.566 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:00.566 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907763+0000", 2024-05-08T08:53:00.566 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2231000000000005e-05 2024-05-08T08:53:00.566 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.566 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.566 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:00.566 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907867+0000", 2024-05-08T08:53:00.566 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104644 2024-05-08T08:53:00.566 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:00.566 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:00.566 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:00.566 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.566 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.567 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22294 3.1c 3:3aad0389:::rbd_data.11406886b07b.0000000000000080:head [stat,delete] snapc 7=[7] RETRY=6 ondisk+retry+write+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:00.567 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:53:00.567 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 32.605535586000002, 2024-05-08T08:53:00.567 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051152191, 2024-05-08T08:53:00.567 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:00.567 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:00.567 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:00.567 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:00.567 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:00.567 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:00.567 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22294 2024-05-08T08:53:00.567 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.567 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:00.567 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.567 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:00.568 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:53:00.568 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:00.568 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.568 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.568 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:00.568 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:53:00.568 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:00.568 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.568 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.568 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:00.568 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858662+0000", 2024-05-08T08:53:00.568 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4799999999999998e-07 2024-05-08T08:53:00.568 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.568 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.568 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:00.569 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858664+0000", 2024-05-08T08:53:00.569 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1229999999999998e-06 2024-05-08T08:53:00.569 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.569 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.569 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:00.569 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858670+0000", 2024-05-08T08:53:00.569 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3099999999999997e-06 2024-05-08T08:53:00.569 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.569 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.569 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:00.569 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858676+0000", 2024-05-08T08:53:00.569 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9750000000000004e-06 2024-05-08T08:53:00.569 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.569 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.569 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:00.570 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907901+0000", 2024-05-08T08:53:00.570 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049224520000000001 2024-05-08T08:53:00.570 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.570 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.570 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:00.570 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.908085+0000", 2024-05-08T08:53:00.570 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00018377500000000001 2024-05-08T08:53:00.570 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:00.570 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:00.570 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:00.570 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.909814+0000", 2024-05-08T08:53:00.570 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00172894 2024-05-08T08:53:00.570 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:00.570 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:00.570 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:00.571 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:00.571 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:00.571 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:53:00.571 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T08:53:00.573 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:53:00.575+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:53:00.573 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:53:00.677 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:53:00.680+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:53:00.705 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:53:00.706 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:53:00.720 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:53:00.720 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:53:00.720 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:53:00.721 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:53:00.723 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:53:00.723 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:53:00.729 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:53:00.739 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:53:00.740 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:53:00.757 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:53:00.757 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:53:00.774 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:53:00.775 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:53:00.777 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:00.780+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:00.792 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:53:00.792 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:53:00.835 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:00.835 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:00.843 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:53:00.882 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:00.881+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:00.948 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:00.948 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:00.957 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:53:00.979 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:00.982+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:01.069 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:01.069 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:01.078 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:53:01.079 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:01.082+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:01.144 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-05-08T08:53:01.188 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:01.191+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:01.188 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:01.188 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:01.198 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:53:01.282 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:01.285+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:01.351 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:01.351 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:01.362 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:53:01.382 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:01.385+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:01.393 INFO:teuthology.orchestra.run.smithi196.stdout:truncating image image_client.0-clone14 from 0xa1ba00 (overlap 0xa1ba00) to 0x918c00 2024-05-08T08:53:01.483 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:01.486+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:01.520 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:01.520 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:01.530 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:53:01.584 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:01.587+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:01.644 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:01.644 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:01.654 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:53:01.698 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:53:01.701+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:53:01.769 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:01.769 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:01.778 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:53:01.819 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:01.822+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:01.887 INFO:teuthology.orchestra.run.smithi196.stdout:checking clone #13, image image_client.0-clone13 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent14 2024-05-08T08:53:01.887 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:01.890+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:01.939 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:01.939 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:01.948 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:53:01.988 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:01.991+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:01.988 INFO:teuthology.orchestra.run.smithi196.stdout:2464 write 0x2ad7600 thru 0x2adee9b (0x789c bytes) 2024-05-08T08:53:01.989 INFO:teuthology.orchestra.run.smithi196.stdout:2465 flatten 2024-05-08T08:53:02.069 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:02.070 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:02.070 INFO:teuthology.orchestra.run.smithi196.stdout:2466 write 0x11bbc00 thru 0x11c679a (0xab9b bytes) 2024-05-08T08:53:02.070 INFO:teuthology.orchestra.run.smithi196.stdout:2473 punch from 0x134fe00 to 0x1359000, (0x9200 bytes) 2024-05-08T08:53:02.070 INFO:teuthology.orchestra.run.smithi196.stdout:2476 punch from 0x2378000 to 0x2378400, (0x400 bytes) 2024-05-08T08:53:02.071 INFO:teuthology.orchestra.run.smithi196.stdout:2477 read 0x7ddc00 thru 0x7de924 (0xd25 bytes) 2024-05-08T08:53:02.071 INFO:teuthology.orchestra.run.smithi196.stdout:2482 read 0x12d2000 thru 0x12dc9e0 (0xa9e1 bytes) 2024-05-08T08:53:02.071 INFO:teuthology.orchestra.run.smithi196.stdout:2485 read 0x2338e00 thru 0x233b824 (0x2a25 bytes) 2024-05-08T08:53:02.071 INFO:teuthology.orchestra.run.smithi196.stdout:2487 punch from 0x2376e00 to 0x237d600, (0x6800 bytes) 2024-05-08T08:53:02.075 INFO:teuthology.orchestra.run.smithi196.stdout:2489 read 0xf61c00 thru 0xf6d0ac (0xb4ad bytes) 2024-05-08T08:53:02.075 INFO:teuthology.orchestra.run.smithi196.stdout:2491 trunc from 0x2adf000 to 0x2de7400 2024-05-08T08:53:02.080 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:53:02.099 INFO:teuthology.orchestra.run.smithi196.stdout:2492 write 0x5cf1c00 thru 0x5cf302f (0x1430 bytes) 2024-05-08T08:53:02.105 INFO:teuthology.orchestra.run.smithi196.stdout:2493 write 0x3900c00 thru 0x390a594 (0x9995 bytes) 2024-05-08T08:53:02.118 INFO:teuthology.orchestra.run.smithi196.stdout:2494 write 0x622ba00 thru 0x622f6c1 (0x3cc2 bytes) 2024-05-08T08:53:02.119 INFO:teuthology.orchestra.run.smithi196.stdout:2496 read 0x5ded200 thru 0x5df2c01 (0x5a02 bytes) 2024-05-08T08:53:02.120 INFO:teuthology.orchestra.run.smithi196.stdout:2498 read 0xb41e00 thru 0xb49bc3 (0x7dc4 bytes) 2024-05-08T08:53:02.123 INFO:teuthology.orchestra.run.smithi196.stdout:2499 read 0x4e62400 thru 0x4e7199a (0xf59b bytes) 2024-05-08T08:53:02.123 INFO:teuthology.orchestra.run.smithi196.stdout:2502 read 0x21600 thru 0x22789 (0x118a bytes) 2024-05-08T08:53:02.123 INFO:teuthology.orchestra.run.smithi196.stdout:2505 read 0x61f8c00 thru 0x6200115 (0x7516 bytes) 2024-05-08T08:53:02.123 INFO:teuthology.orchestra.run.smithi196.stdout:2506 write 0x5b98400 thru 0x5ba0110 (0x7d11 bytes) 2024-05-08T08:53:02.124 INFO:teuthology.orchestra.run.smithi196.stdout:2507 trunc from 0x622f800 to 0x3ec9c00 2024-05-08T08:53:02.124 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:02.127+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:02.165 INFO:teuthology.orchestra.run.smithi196.stdout:2510 write 0xb0c6000 thru 0xb0c8b1b (0x2b1c bytes) 2024-05-08T08:53:02.165 INFO:teuthology.orchestra.run.smithi196.stdout:2511 write 0x6a6d800 thru 0x6a7c619 (0xee1a bytes) 2024-05-08T08:53:02.166 INFO:teuthology.orchestra.run.smithi196.stdout:2512 punch from 0x52ef800 to 0x52f4000, (0x4800 bytes) 2024-05-08T08:53:02.166 INFO:teuthology.orchestra.run.smithi196.stdout:2514 read 0x2171400 thru 0x217865b (0x725c bytes) 2024-05-08T08:53:02.167 INFO:teuthology.orchestra.run.smithi196.stdout:2515 read 0x4eba400 thru 0x4ec085b (0x645c bytes) 2024-05-08T08:53:02.167 INFO:teuthology.orchestra.run.smithi196.stdout:2516 write 0x7b9d400 thru 0x7bac407 (0xf008 bytes) 2024-05-08T08:53:02.167 INFO:teuthology.orchestra.run.smithi196.stdout:2517 read 0x7bab800 thru 0x7bab9aa (0x1ab bytes) 2024-05-08T08:53:02.185 INFO:teuthology.orchestra.run.smithi196.stdout:2518 write 0xbd1f400 thru 0xbd2156e (0x216f bytes) 2024-05-08T08:53:02.187 INFO:teuthology.orchestra.run.smithi196.stdout:2519 read 0x287f400 thru 0x288e062 (0xec63 bytes) 2024-05-08T08:53:02.187 INFO:teuthology.orchestra.run.smithi196.stdout:2521 write 0x5a82400 thru 0x5a8524b (0x2e4c bytes) 2024-05-08T08:53:02.188 INFO:teuthology.orchestra.run.smithi196.stdout:2523 punch from 0x6baea00 to 0x6bbae00, (0xc400 bytes) 2024-05-08T08:53:02.189 INFO:teuthology.orchestra.run.smithi196.stdout:2524 write 0x7d1ea00 thru 0x7d20d77 (0x2378 bytes) 2024-05-08T08:53:02.189 INFO:teuthology.orchestra.run.smithi196.stdout:2526 write 0x70b7e00 thru 0x70c3c5a (0xbe5b bytes) 2024-05-08T08:53:02.189 INFO:teuthology.orchestra.run.smithi196.stdout:2530 read 0x1e86200 thru 0x1e8715c (0xf5d bytes) 2024-05-08T08:53:02.190 INFO:teuthology.orchestra.run.smithi196.stdout:2531 write 0xb105600 thru 0xb11257a (0xcf7b bytes) 2024-05-08T08:53:02.190 INFO:teuthology.orchestra.run.smithi196.stdout:2533 punch from 0x995400 to 0x998600, (0x3200 bytes) 2024-05-08T08:53:02.190 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:53:02.193+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:53:02.209 INFO:teuthology.orchestra.run.smithi196.stdout:2534 write 0xc8d6400 thru 0xc8d66bd (0x2be bytes) 2024-05-08T08:53:02.210 INFO:teuthology.orchestra.run.smithi196.stdout:2538 read 0x37dfa00 thru 0x37e3d88 (0x4389 bytes) 2024-05-08T08:53:02.210 INFO:teuthology.orchestra.run.smithi196.stdout:2540 write 0x424000 thru 0x42d4da (0x94db bytes) 2024-05-08T08:53:02.212 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:02.213 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:02.214 INFO:teuthology.orchestra.run.smithi196.stdout:2544 write 0xe53000 thru 0xe6173c (0xe73d bytes) 2024-05-08T08:53:02.223 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:53:02.230 INFO:teuthology.orchestra.run.smithi196.stdout:2546 write 0xd685000 thru 0xd68e22c (0x922d bytes) 2024-05-08T08:53:02.238 INFO:teuthology.orchestra.run.smithi196.stdout:2548 read 0x77c0400 thru 0x77c52a5 (0x4ea6 bytes) 2024-05-08T08:53:02.245 INFO:teuthology.orchestra.run.smithi196.stdout:2550 trunc from 0xd68e400 to 0x7d82400 2024-05-08T08:53:02.273 INFO:teuthology.orchestra.run.smithi196.stdout:2551 trunc from 0x7d82400 to 0xade9600 2024-05-08T08:53:02.284 INFO:teuthology.orchestra.run.smithi196.stdout:2552 write 0xae4600 thru 0xaf0f58 (0xc959 bytes) 2024-05-08T08:53:02.291 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:02.294+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:02.292 INFO:teuthology.orchestra.run.smithi196.stdout:2553 read 0x137dc00 thru 0x13842fe (0x66ff bytes) 2024-05-08T08:53:02.310 INFO:teuthology.orchestra.run.smithi196.stdout:2554 write 0xe2f1800 thru 0xe2f225e (0xa5f bytes) 2024-05-08T08:53:02.315 INFO:teuthology.orchestra.run.smithi196.stdout:2555 punch from 0xe97000 to 0xea3c00, (0xcc00 bytes) 2024-05-08T08:53:02.323 INFO:teuthology.orchestra.run.smithi196.stdout:2556 trunc from 0xe2f2400 to 0x4f7f400 2024-05-08T08:53:02.335 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:02.335 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:02.344 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:53:02.355 INFO:teuthology.orchestra.run.smithi059.stderr:nodeep-scrub is set 2024-05-08T08:53:02.429 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:02.432+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:02.491 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:02.491 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:02.492 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:02.495+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:02.500 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:53:02.593 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:02.596+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:02.645 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:02.646 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:02.654 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:53:02.698 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:02.701+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:02.760 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:02.760 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:02.769 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:53:02.803 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T08:53:02.810 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:02.813+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:02.896 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:02.899+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:02.914 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:02.914 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:02.923 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:53:02.933 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:53:02.933 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:53:02.951 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:53:02.951 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:53:02.968 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:53:02.968 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:53:02.985 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:53:02.986 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:53:02.997 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:03.000+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:03.003 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:53:03.003 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:53:03.021 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:53:03.021 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:53:03.031 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:53:03.032 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:53:03.032 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:53:03.032 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:53:03.032 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.032 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22069 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 749568~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:03.032 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857624+0000", 2024-05-08T08:53:03.032 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 35.176039762999999, 2024-05-08T08:53:03.032 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046969958999999999, 2024-05-08T08:53:03.032 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:03.032 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:03.032 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:03.032 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:03.032 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:03.032 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:03.033 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22069 2024-05-08T08:53:03.033 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.033 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:03.033 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.033 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:03.033 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857624+0000", 2024-05-08T08:53:03.033 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:03.033 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.033 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.033 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:03.033 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857624+0000", 2024-05-08T08:53:03.033 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:03.033 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.033 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.034 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:03.034 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857625+0000", 2024-05-08T08:53:03.034 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.03e-07 2024-05-08T08:53:03.034 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.034 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.034 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:03.034 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857627+0000", 2024-05-08T08:53:03.034 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9369999999999998e-06 2024-05-08T08:53:03.034 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.034 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.034 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:03.034 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857633+0000", 2024-05-08T08:53:03.034 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1229999999999997e-06 2024-05-08T08:53:03.035 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.035 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.035 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:03.035 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857637+0000", 2024-05-08T08:53:03.035 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7820000000000001e-06 2024-05-08T08:53:03.035 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.035 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.035 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:03.035 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904457+0000", 2024-05-08T08:53:03.035 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046819715999999997 2024-05-08T08:53:03.036 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.036 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.036 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:03.036 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904524+0000", 2024-05-08T08:53:03.036 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6371000000000005e-05 2024-05-08T08:53:03.036 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.036 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.036 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:03.036 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904594+0000", 2024-05-08T08:53:03.036 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0627e-05 2024-05-08T08:53:03.036 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.036 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:03.036 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.036 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.037 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.037 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22071 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 765952~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:03.037 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857643+0000", 2024-05-08T08:53:03.037 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 35.176020680000001, 2024-05-08T08:53:03.037 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047106528000000002, 2024-05-08T08:53:03.037 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:03.037 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:03.037 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:03.037 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:03.037 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:03.037 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:03.037 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22071 2024-05-08T08:53:03.037 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.037 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:03.037 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.038 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:03.038 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857643+0000", 2024-05-08T08:53:03.038 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:03.038 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.038 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.038 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:03.038 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857643+0000", 2024-05-08T08:53:03.038 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:03.038 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.038 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.038 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:03.038 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857644+0000", 2024-05-08T08:53:03.038 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2000000000000001e-07 2024-05-08T08:53:03.038 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.039 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.039 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:03.039 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857646+0000", 2024-05-08T08:53:03.039 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0059999999999999e-06 2024-05-08T08:53:03.039 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.039 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.039 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:03.039 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857651+0000", 2024-05-08T08:53:03.039 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.84e-06 2024-05-08T08:53:03.039 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.039 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.039 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:03.039 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857657+0000", 2024-05-08T08:53:03.039 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5620000000000003e-06 2024-05-08T08:53:03.040 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.040 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.040 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:03.040 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904620+0000", 2024-05-08T08:53:03.040 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046962661000000003 2024-05-08T08:53:03.040 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.040 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.040 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:03.040 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904678+0000", 2024-05-08T08:53:03.040 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8656999999999998e-05 2024-05-08T08:53:03.040 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.041 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.041 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:03.041 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904750+0000", 2024-05-08T08:53:03.041 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1482000000000004e-05 2024-05-08T08:53:03.041 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.041 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:03.041 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.041 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.041 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.041 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22073 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 782336~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:03.041 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:53:03.041 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 35.175996671999997, 2024-05-08T08:53:03.041 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047244234000000003, 2024-05-08T08:53:03.041 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:03.042 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:03.042 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:03.042 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:03.042 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:03.042 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:03.042 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22073 2024-05-08T08:53:03.042 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.042 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:03.042 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.042 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:03.042 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:53:03.042 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:03.042 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.042 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.043 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:03.043 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:53:03.043 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:03.043 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.043 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.043 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:03.043 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857668+0000", 2024-05-08T08:53:03.043 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7800000000000002e-07 2024-05-08T08:53:03.043 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.043 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.044 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:03.044 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857670+0000", 2024-05-08T08:53:03.044 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0820000000000001e-06 2024-05-08T08:53:03.044 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.044 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.044 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:03.044 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857678+0000", 2024-05-08T08:53:03.044 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7589999999999997e-06 2024-05-08T08:53:03.044 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.044 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.044 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:03.045 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857691+0000", 2024-05-08T08:53:03.045 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3672999999999999e-05 2024-05-08T08:53:03.045 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.045 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.045 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:03.045 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904780+0000", 2024-05-08T08:53:03.045 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047088616999999999 2024-05-08T08:53:03.045 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.045 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.046 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:03.046 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904840+0000", 2024-05-08T08:53:03.046 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0356999999999998e-05 2024-05-08T08:53:03.046 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.046 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.046 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:03.046 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904912+0000", 2024-05-08T08:53:03.046 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1367999999999998e-05 2024-05-08T08:53:03.046 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.047 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:03.047 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.047 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.047 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.047 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22075 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 798720~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:03.047 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:53:03.047 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 35.175966275, 2024-05-08T08:53:03.047 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047384643999999997, 2024-05-08T08:53:03.047 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:03.047 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:03.047 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:03.047 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:03.047 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:03.048 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:03.048 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22075 2024-05-08T08:53:03.048 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.048 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:03.048 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.048 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:03.048 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:53:03.048 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:03.048 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.048 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.048 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:03.048 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:53:03.048 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:03.048 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.049 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.049 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:03.049 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:53:03.049 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7399999999999999e-07 2024-05-08T08:53:03.049 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.049 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.049 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:03.049 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857705+0000", 2024-05-08T08:53:03.049 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1980000000000004e-06 2024-05-08T08:53:03.049 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.049 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.049 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:03.050 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857713+0000", 2024-05-08T08:53:03.050 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7039999999999994e-06 2024-05-08T08:53:03.050 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.050 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.050 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:03.050 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857718+0000", 2024-05-08T08:53:03.050 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.865e-06 2024-05-08T08:53:03.050 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.050 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.050 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:03.050 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904941+0000", 2024-05-08T08:53:03.050 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047223198000000001 2024-05-08T08:53:03.050 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.051 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.051 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:03.051 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904999+0000", 2024-05-08T08:53:03.051 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8254999999999997e-05 2024-05-08T08:53:03.051 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.051 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.051 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:03.051 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905082+0000", 2024-05-08T08:53:03.051 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3049999999999999e-05 2024-05-08T08:53:03.051 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.051 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:03.051 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.051 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.051 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.052 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22077 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 815104~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:03.052 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:53:03.052 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 35.175928264, 2024-05-08T08:53:03.052 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047519448999999998, 2024-05-08T08:53:03.052 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:03.052 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:03.052 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:03.052 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:03.052 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:03.052 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:03.052 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22077 2024-05-08T08:53:03.052 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.052 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:03.052 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.052 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:03.053 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:53:03.053 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:03.053 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.053 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.053 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:03.053 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:53:03.053 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:03.053 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.053 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.053 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:03.053 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:53:03.053 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.99e-07 2024-05-08T08:53:03.053 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.053 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.053 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:03.054 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857739+0000", 2024-05-08T08:53:03.054 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3599999999999999e-06 2024-05-08T08:53:03.054 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.054 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.054 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:03.054 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857745+0000", 2024-05-08T08:53:03.054 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5139999999999998e-06 2024-05-08T08:53:03.054 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.054 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.054 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:03.054 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857750+0000", 2024-05-08T08:53:03.054 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8550000000000001e-06 2024-05-08T08:53:03.054 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.054 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.055 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:03.055 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905110+0000", 2024-05-08T08:53:03.055 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047360126000000002 2024-05-08T08:53:03.055 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.055 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.055 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:03.055 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905175+0000", 2024-05-08T08:53:03.055 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4689000000000004e-05 2024-05-08T08:53:03.055 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.055 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.055 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:03.055 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905255+0000", 2024-05-08T08:53:03.055 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0306e-05 2024-05-08T08:53:03.055 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.055 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:03.056 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.056 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.056 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.056 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22079 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 831488~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:03.056 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:53:03.056 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 35.175907715999998, 2024-05-08T08:53:03.056 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.04767499, 2024-05-08T08:53:03.056 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:03.056 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:03.056 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:03.057 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:03.057 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:03.057 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:03.057 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22079 2024-05-08T08:53:03.057 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.057 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:03.057 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.057 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:03.057 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:53:03.057 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:03.057 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.058 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.058 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:03.058 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:53:03.058 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:03.058 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.058 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.058 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:03.058 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857757+0000", 2024-05-08T08:53:03.058 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8299999999999997e-07 2024-05-08T08:53:03.058 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.058 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.058 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:03.059 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857759+0000", 2024-05-08T08:53:03.059 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1390000000000002e-06 2024-05-08T08:53:03.059 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.059 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.059 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:03.059 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857765+0000", 2024-05-08T08:53:03.059 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4609999999999996e-06 2024-05-08T08:53:03.059 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.059 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.059 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:03.059 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857770+0000", 2024-05-08T08:53:03.059 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6689999999999997e-06 2024-05-08T08:53:03.059 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.059 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.059 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:03.060 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905281+0000", 2024-05-08T08:53:03.060 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047510668999999998 2024-05-08T08:53:03.060 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.060 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.060 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:03.060 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905344+0000", 2024-05-08T08:53:03.060 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2797000000000002e-05 2024-05-08T08:53:03.060 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.060 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.060 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:03.060 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905431+0000", 2024-05-08T08:53:03.061 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7771999999999999e-05 2024-05-08T08:53:03.061 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.061 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:03.061 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.061 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.061 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.061 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22082 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 847872~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:03.061 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:53:03.061 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 35.175888024999999, 2024-05-08T08:53:03.061 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047831934999999999, 2024-05-08T08:53:03.061 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:03.061 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:03.061 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:03.061 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:03.062 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:03.062 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:03.062 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22082 2024-05-08T08:53:03.062 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.062 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:03.062 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.062 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:03.062 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:53:03.062 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:03.062 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.062 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.062 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:03.062 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:53:03.063 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:03.063 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.063 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.063 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:03.063 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:53:03.063 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:53:03.063 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.063 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.063 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:03.063 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857778+0000", 2024-05-08T08:53:03.063 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0789999999999999e-06 2024-05-08T08:53:03.063 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.063 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.063 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:03.063 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857784+0000", 2024-05-08T08:53:03.064 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0480000000000004e-06 2024-05-08T08:53:03.064 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.064 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.064 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:03.064 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857790+0000", 2024-05-08T08:53:03.064 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2719999999999997e-06 2024-05-08T08:53:03.064 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.064 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.064 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:03.064 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905458+0000", 2024-05-08T08:53:03.064 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047668470999999997 2024-05-08T08:53:03.064 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.064 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.064 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:03.064 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905521+0000", 2024-05-08T08:53:03.065 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2952000000000003e-05 2024-05-08T08:53:03.065 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.065 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.065 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:03.065 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905608+0000", 2024-05-08T08:53:03.065 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6759000000000001e-05 2024-05-08T08:53:03.065 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.065 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:03.065 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.065 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.065 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.065 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22084 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 864256~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:03.065 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:53:03.065 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 35.175864075, 2024-05-08T08:53:03.065 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047977748000000001, 2024-05-08T08:53:03.066 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:03.066 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:03.066 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:03.066 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:03.066 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:03.066 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:03.066 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22084 2024-05-08T08:53:03.066 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.066 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:03.066 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.066 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:03.066 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:53:03.066 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:03.066 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.067 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.067 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:03.067 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:53:03.067 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:03.067 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.067 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.067 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:03.067 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:53:03.067 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:53:03.067 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.067 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.068 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:03.068 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857802+0000", 2024-05-08T08:53:03.068 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0779999999999998e-06 2024-05-08T08:53:03.068 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.068 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.068 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:03.068 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857812+0000", 2024-05-08T08:53:03.068 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7939999999999997e-06 2024-05-08T08:53:03.068 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.068 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.068 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:03.068 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857817+0000", 2024-05-08T08:53:03.068 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8899999999999998e-06 2024-05-08T08:53:03.068 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.069 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.069 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:03.069 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905636+0000", 2024-05-08T08:53:03.069 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047819252 2024-05-08T08:53:03.069 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.069 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.069 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:03.069 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905700+0000", 2024-05-08T08:53:03.069 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4016999999999994e-05 2024-05-08T08:53:03.069 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.069 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.069 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:03.069 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905778+0000", 2024-05-08T08:53:03.069 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7306999999999997e-05 2024-05-08T08:53:03.069 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.070 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:03.070 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.070 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.070 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.070 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22086 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 880640~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:03.070 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:53:03.070 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 35.175840725, 2024-05-08T08:53:03.070 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048119149, 2024-05-08T08:53:03.070 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:03.070 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:03.070 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:03.070 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:03.070 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:03.070 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:03.071 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22086 2024-05-08T08:53:03.071 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.071 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:03.071 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.071 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:03.071 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:53:03.071 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:03.071 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.071 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.071 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:03.071 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:53:03.071 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:03.071 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.071 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.071 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:03.072 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857824+0000", 2024-05-08T08:53:03.072 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4200000000000002e-07 2024-05-08T08:53:03.072 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.072 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.072 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:03.072 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857826+0000", 2024-05-08T08:53:03.072 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0590000000000001e-06 2024-05-08T08:53:03.072 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.072 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.072 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:03.072 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857832+0000", 2024-05-08T08:53:03.072 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8029999999999999e-06 2024-05-08T08:53:03.072 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.072 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.072 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:03.072 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857837+0000", 2024-05-08T08:53:03.073 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7929999999999997e-06 2024-05-08T08:53:03.073 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.073 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.073 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:03.073 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905803+0000", 2024-05-08T08:53:03.073 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047965761000000003 2024-05-08T08:53:03.073 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.073 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.073 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:03.073 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905862+0000", 2024-05-08T08:53:03.073 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9203000000000002e-05 2024-05-08T08:53:03.073 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.073 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.073 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:03.073 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905942+0000", 2024-05-08T08:53:03.074 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0187999999999999e-05 2024-05-08T08:53:03.074 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.074 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:03.074 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.074 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.074 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.074 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22088 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 897024~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:03.074 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:53:03.074 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 35.175820672999997, 2024-05-08T08:53:03.074 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048256049000000002, 2024-05-08T08:53:03.074 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:03.074 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:03.075 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:03.075 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:03.075 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:03.075 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:03.075 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22088 2024-05-08T08:53:03.075 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.075 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:03.075 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.076 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:03.076 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:53:03.076 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:03.076 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.077 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.077 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:03.078 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:53:03.078 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:03.078 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.078 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.078 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:03.078 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857844+0000", 2024-05-08T08:53:03.078 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.46e-07 2024-05-08T08:53:03.078 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.078 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.079 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:03.079 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857846+0000", 2024-05-08T08:53:03.079 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0899999999999999e-06 2024-05-08T08:53:03.079 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.079 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.079 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:03.079 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857855+0000", 2024-05-08T08:53:03.079 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8219999999999996e-06 2024-05-08T08:53:03.079 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.079 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.079 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:03.080 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857860+0000", 2024-05-08T08:53:03.080 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2989999999999996e-06 2024-05-08T08:53:03.080 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.080 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.080 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:03.080 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905966+0000", 2024-05-08T08:53:03.080 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048106542000000002 2024-05-08T08:53:03.080 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.080 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.080 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:03.080 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906024+0000", 2024-05-08T08:53:03.081 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7958000000000003e-05 2024-05-08T08:53:03.081 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.081 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.081 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:03.081 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906099+0000", 2024-05-08T08:53:03.081 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4992000000000003e-05 2024-05-08T08:53:03.081 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.081 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:03.081 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.081 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.082 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.082 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22091 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 913408~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:03.082 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:53:03.082 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 35.175796912999999, 2024-05-08T08:53:03.082 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048395290000000001, 2024-05-08T08:53:03.082 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:03.082 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:03.082 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:03.083 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:03.083 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:03.083 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:03.083 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22091 2024-05-08T08:53:03.083 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.083 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:03.083 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.083 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:03.083 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:53:03.083 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:03.083 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.083 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.083 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:03.084 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:53:03.084 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:03.084 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.084 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.084 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:03.084 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:53:03.084 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4900000000000001e-07 2024-05-08T08:53:03.084 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.084 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.084 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:03.084 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857870+0000", 2024-05-08T08:53:03.085 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2510000000000001e-06 2024-05-08T08:53:03.085 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.085 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.085 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:03.085 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857877+0000", 2024-05-08T08:53:03.085 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0809999999999996e-06 2024-05-08T08:53:03.085 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.085 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.085 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:03.085 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857881+0000", 2024-05-08T08:53:03.085 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.724e-06 2024-05-08T08:53:03.085 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.085 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.086 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:03.086 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906127+0000", 2024-05-08T08:53:03.086 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048245345000000002 2024-05-08T08:53:03.086 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.086 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.087 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:03.087 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906190+0000", 2024-05-08T08:53:03.087 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2762000000000007e-05 2024-05-08T08:53:03.087 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.087 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.087 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:03.087 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906262+0000", 2024-05-08T08:53:03.087 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2778e-05 2024-05-08T08:53:03.087 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.087 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:03.087 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.088 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.088 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.088 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22093 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 929792~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:03.088 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:53:03.088 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 35.175774549000003, 2024-05-08T08:53:03.088 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048532875000000003, 2024-05-08T08:53:03.088 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:03.088 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:03.088 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:03.088 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:03.088 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:03.088 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:03.089 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22093 2024-05-08T08:53:03.089 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.089 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:03.089 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.089 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:03.089 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:53:03.089 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:03.089 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.089 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.089 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:03.089 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:53:03.089 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:03.089 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.090 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.090 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:03.090 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857890+0000", 2024-05-08T08:53:03.090 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-07 2024-05-08T08:53:03.090 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.090 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.090 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:03.090 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857899+0000", 2024-05-08T08:53:03.090 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6889999999999999e-06 2024-05-08T08:53:03.090 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.090 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.090 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:03.090 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857906+0000", 2024-05-08T08:53:03.091 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6490000000000002e-06 2024-05-08T08:53:03.091 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.091 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.091 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:03.091 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857917+0000", 2024-05-08T08:53:03.091 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0764999999999999e-05 2024-05-08T08:53:03.091 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.091 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.091 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:03.091 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906287+0000", 2024-05-08T08:53:03.091 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048370442 2024-05-08T08:53:03.091 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.091 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.092 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:03.092 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906349+0000", 2024-05-08T08:53:03.092 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1537000000000004e-05 2024-05-08T08:53:03.092 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.092 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.092 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:03.092 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906422+0000", 2024-05-08T08:53:03.092 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3431999999999998e-05 2024-05-08T08:53:03.092 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.092 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:03.092 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.092 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.092 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.092 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22095 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 946176~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:03.092 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:53:03.092 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 35.175739845000003, 2024-05-08T08:53:03.093 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048657593999999998, 2024-05-08T08:53:03.093 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:03.093 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:03.093 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:03.093 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:03.093 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:03.093 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:03.093 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22095 2024-05-08T08:53:03.093 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.093 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:03.093 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.094 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:03.094 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:53:03.094 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:03.094 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.094 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.094 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:03.094 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:53:03.094 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:03.094 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.095 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.095 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:03.095 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857925+0000", 2024-05-08T08:53:03.095 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3099999999999998e-07 2024-05-08T08:53:03.095 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.095 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.095 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:03.095 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857927+0000", 2024-05-08T08:53:03.095 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.34e-06 2024-05-08T08:53:03.096 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.096 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.096 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:03.096 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857938+0000", 2024-05-08T08:53:03.096 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1426e-05 2024-05-08T08:53:03.096 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.096 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.096 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:03.096 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857943+0000", 2024-05-08T08:53:03.097 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5140000000000003e-06 2024-05-08T08:53:03.097 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.097 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.097 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:03.097 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906448+0000", 2024-05-08T08:53:03.097 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048504641000000001 2024-05-08T08:53:03.097 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.097 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.098 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:03.098 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906510+0000", 2024-05-08T08:53:03.098 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2663000000000006e-05 2024-05-08T08:53:03.098 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.098 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.098 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:03.098 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906582+0000", 2024-05-08T08:53:03.098 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1478999999999998e-05 2024-05-08T08:53:03.098 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.099 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:03.099 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.099 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.099 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.099 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22097 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 962560~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:03.099 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:53:03.099 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 35.175712156000003, 2024-05-08T08:53:03.100 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048791353000000003, 2024-05-08T08:53:03.100 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:03.100 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:03.100 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:03.100 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:03.100 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:03.100 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:03.100 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22097 2024-05-08T08:53:03.100 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.100 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:03.100 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.100 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:03.100 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:53:03.100 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:03.100 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.101 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.101 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:03.101 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:53:03.101 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:03.101 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.101 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.101 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:03.101 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:53:03.101 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9200000000000002e-07 2024-05-08T08:53:03.101 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.101 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.101 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:03.101 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857954+0000", 2024-05-08T08:53:03.101 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1900000000000002e-06 2024-05-08T08:53:03.102 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.102 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.102 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:03.102 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857961+0000", 2024-05-08T08:53:03.102 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4989999999999999e-06 2024-05-08T08:53:03.102 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.102 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.102 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:03.102 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857971+0000", 2024-05-08T08:53:03.102 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0485999999999999e-05 2024-05-08T08:53:03.102 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.102 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.102 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:03.102 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906607+0000", 2024-05-08T08:53:03.103 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048635109000000003 2024-05-08T08:53:03.103 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.103 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.103 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:03.103 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906667+0000", 2024-05-08T08:53:03.103 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0439000000000002e-05 2024-05-08T08:53:03.103 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.103 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.103 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:03.103 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906743+0000", 2024-05-08T08:53:03.103 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6237999999999998e-05 2024-05-08T08:53:03.103 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.103 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:03.103 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.103 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.104 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.104 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22099 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 978944~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:03.104 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:53:03.104 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 35.175685995999999, 2024-05-08T08:53:03.104 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048932055000000002, 2024-05-08T08:53:03.104 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:03.104 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:03.104 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:03.104 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:03.104 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:03.104 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:03.104 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22099 2024-05-08T08:53:03.105 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.105 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:03.105 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.105 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:03.105 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:53:03.105 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:03.105 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.105 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.105 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:03.105 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:53:03.106 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:03.106 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.106 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.106 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:03.106 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:53:03.106 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.27e-07 2024-05-08T08:53:03.106 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.106 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.106 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:03.106 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857987+0000", 2024-05-08T08:53:03.106 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5890000000000003e-06 2024-05-08T08:53:03.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.107 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:03.107 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857994+0000", 2024-05-08T08:53:03.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6619999999999999e-06 2024-05-08T08:53:03.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.107 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:03.107 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857998+0000", 2024-05-08T08:53:03.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7110000000000003e-06 2024-05-08T08:53:03.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.107 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:03.107 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906772+0000", 2024-05-08T08:53:03.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048773415000000001 2024-05-08T08:53:03.108 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.108 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.108 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:03.108 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906834+0000", 2024-05-08T08:53:03.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1752000000000001e-05 2024-05-08T08:53:03.108 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.108 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.108 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:03.108 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906910+0000", 2024-05-08T08:53:03.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6499000000000001e-05 2024-05-08T08:53:03.108 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.108 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:03.108 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.109 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.109 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.109 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22101 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 995328~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:03.109 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:53:03.109 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 35.175658865000003, 2024-05-08T08:53:03.109 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049087364000000001, 2024-05-08T08:53:03.109 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:03.109 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:03.109 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:03.109 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:03.109 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:03.109 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:03.109 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22101 2024-05-08T08:53:03.109 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.110 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:03.110 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.110 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:03.110 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:53:03.110 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:03.110 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.110 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.110 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:03.110 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:53:03.110 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:03.110 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.110 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.110 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:03.111 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858006+0000", 2024-05-08T08:53:03.111 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1399999999999997e-07 2024-05-08T08:53:03.111 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.111 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.111 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:03.111 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858008+0000", 2024-05-08T08:53:03.111 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2060000000000001e-06 2024-05-08T08:53:03.111 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.111 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.111 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:03.111 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858017+0000", 2024-05-08T08:53:03.112 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1579999999999994e-06 2024-05-08T08:53:03.112 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.112 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.112 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:03.112 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858022+0000", 2024-05-08T08:53:03.112 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4199999999999998e-06 2024-05-08T08:53:03.112 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.112 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.112 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:03.112 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906940+0000", 2024-05-08T08:53:03.112 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048917627999999998 2024-05-08T08:53:03.112 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.112 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.113 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:03.113 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907002+0000", 2024-05-08T08:53:03.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2399999999999999e-05 2024-05-08T08:53:03.113 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.113 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.113 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:03.113 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907092+0000", 2024-05-08T08:53:03.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0137999999999997e-05 2024-05-08T08:53:03.113 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.113 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:03.113 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.113 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.113 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.113 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22103 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1011712~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:03.113 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:53:03.114 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 35.175634080000002, 2024-05-08T08:53:03.114 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049228542, 2024-05-08T08:53:03.114 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:03.114 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:03.114 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:03.114 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:03.114 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:03.114 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:03.114 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22103 2024-05-08T08:53:03.114 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.114 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:03.114 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.114 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:03.114 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:53:03.115 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:03.115 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.115 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.115 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:03.115 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:53:03.115 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:03.115 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.115 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.115 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:03.115 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:53:03.116 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4299999999999999e-07 2024-05-08T08:53:03.116 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.116 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.116 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:03.116 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858032+0000", 2024-05-08T08:53:03.116 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0590000000000001e-06 2024-05-08T08:53:03.116 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.116 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.116 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:03.116 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858037+0000", 2024-05-08T08:53:03.116 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.977e-06 2024-05-08T08:53:03.116 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.116 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.116 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:03.116 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858042+0000", 2024-05-08T08:53:03.117 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7650000000000001e-06 2024-05-08T08:53:03.117 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.117 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.117 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:03.117 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907118+0000", 2024-05-08T08:53:03.117 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049076147000000001 2024-05-08T08:53:03.117 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.117 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.117 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:03.117 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907180+0000", 2024-05-08T08:53:03.117 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1971999999999996e-05 2024-05-08T08:53:03.117 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.117 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.117 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:03.117 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907258+0000", 2024-05-08T08:53:03.118 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8279e-05 2024-05-08T08:53:03.118 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.118 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:03.118 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.118 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.118 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.118 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22105 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1028096~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:03.118 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:53:03.118 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 35.175614836999998, 2024-05-08T08:53:03.118 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049394609999999999, 2024-05-08T08:53:03.118 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:03.118 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:03.118 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:03.119 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:03.119 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:03.119 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:03.119 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22105 2024-05-08T08:53:03.119 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.119 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:03.119 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.119 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:03.119 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:53:03.119 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:03.119 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.119 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.119 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:03.119 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:53:03.119 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:03.120 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.120 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.120 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:03.120 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858050+0000", 2024-05-08T08:53:03.120 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5999999999999999e-07 2024-05-08T08:53:03.120 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.120 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.120 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:03.120 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858051+0000", 2024-05-08T08:53:03.120 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8870000000000001e-06 2024-05-08T08:53:03.120 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.120 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.120 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:03.120 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858057+0000", 2024-05-08T08:53:03.120 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8019999999999997e-06 2024-05-08T08:53:03.121 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.121 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.121 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:03.121 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858062+0000", 2024-05-08T08:53:03.121 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.561e-06 2024-05-08T08:53:03.121 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.121 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.121 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:03.121 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907289+0000", 2024-05-08T08:53:03.121 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049226957000000002 2024-05-08T08:53:03.121 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.121 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.121 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:03.121 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907356+0000", 2024-05-08T08:53:03.121 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6915000000000006e-05 2024-05-08T08:53:03.122 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.122 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.122 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:03.122 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907444+0000", 2024-05-08T08:53:03.122 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8127999999999994e-05 2024-05-08T08:53:03.122 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.122 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:03.122 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.122 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.122 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.122 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22107 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1044480~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:03.122 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:53:03.122 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 35.175596497000001, 2024-05-08T08:53:03.123 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049799942999999999, 2024-05-08T08:53:03.123 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:03.123 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:03.123 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:03.123 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:03.123 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:03.123 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:03.123 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22107 2024-05-08T08:53:03.123 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.123 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:03.123 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.123 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:03.123 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:53:03.123 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:03.123 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.124 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.124 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:03.124 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:53:03.124 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:03.124 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.124 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.124 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:03.124 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:53:03.124 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4799999999999999e-07 2024-05-08T08:53:03.124 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.124 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.124 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:03.124 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858070+0000", 2024-05-08T08:53:03.124 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.012e-06 2024-05-08T08:53:03.125 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.125 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.125 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:03.125 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858077+0000", 2024-05-08T08:53:03.125 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6290000000000001e-06 2024-05-08T08:53:03.125 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.125 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.125 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:03.125 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858083+0000", 2024-05-08T08:53:03.125 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8880000000000002e-06 2024-05-08T08:53:03.125 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.125 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.125 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:03.126 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907671+0000", 2024-05-08T08:53:03.126 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049587191000000003 2024-05-08T08:53:03.126 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.126 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.126 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:03.126 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907763+0000", 2024-05-08T08:53:03.126 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2231000000000005e-05 2024-05-08T08:53:03.126 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.126 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.126 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:03.126 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907867+0000", 2024-05-08T08:53:03.127 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104644 2024-05-08T08:53:03.127 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.127 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:03.127 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.127 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.127 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.127 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22294 3.1c 3:3aad0389:::rbd_data.11406886b07b.0000000000000080:head [stat,delete] snapc 7=[7] RETRY=6 ondisk+retry+write+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:03.127 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:53:03.127 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 35.175002649, 2024-05-08T08:53:03.127 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051152191, 2024-05-08T08:53:03.127 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:03.127 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:03.128 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:03.128 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:03.128 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:03.128 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:03.128 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22294 2024-05-08T08:53:03.128 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.128 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:03.128 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.128 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:03.128 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:53:03.128 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:03.128 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.128 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.128 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:03.128 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:53:03.129 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:03.129 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.129 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.129 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:03.129 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858662+0000", 2024-05-08T08:53:03.129 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4799999999999998e-07 2024-05-08T08:53:03.129 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.129 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.129 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:03.129 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858664+0000", 2024-05-08T08:53:03.129 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1229999999999998e-06 2024-05-08T08:53:03.129 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.129 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.129 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:03.130 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858670+0000", 2024-05-08T08:53:03.130 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3099999999999997e-06 2024-05-08T08:53:03.130 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.130 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.130 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:03.130 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858676+0000", 2024-05-08T08:53:03.130 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9750000000000004e-06 2024-05-08T08:53:03.130 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.130 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.130 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:03.131 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907901+0000", 2024-05-08T08:53:03.131 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049224520000000001 2024-05-08T08:53:03.131 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.131 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.131 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:03.131 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.908085+0000", 2024-05-08T08:53:03.131 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00018377500000000001 2024-05-08T08:53:03.131 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.131 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.131 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:03.131 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.909814+0000", 2024-05-08T08:53:03.131 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00172894 2024-05-08T08:53:03.131 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.131 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:03.131 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.132 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.132 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:03.132 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:53:03.134 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:53:03.134 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:03.136+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:03.234 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:03.237+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:03.281 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:53:03.281 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:53:03.281 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:53:03.281 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:53:03.289 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:53:03.335 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:03.338+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:03.373 INFO:teuthology.orchestra.run.smithi196.stdout:2557 write 0xa7bae00 thru 0xa7bd8c1 (0x2ac2 bytes) 2024-05-08T08:53:03.378 INFO:teuthology.orchestra.run.smithi196.stdout:2559 read 0x54b9600 thru 0x54c9598 (0xff99 bytes) 2024-05-08T08:53:03.388 INFO:teuthology.orchestra.run.smithi196.stdout:2560 write 0xaabd800 thru 0xaacbe90 (0xe691 bytes) 2024-05-08T08:53:03.395 INFO:teuthology.orchestra.run.smithi196.stdout:2561 read 0x7ae4400 thru 0x7aebb44 (0x7745 bytes) 2024-05-08T08:53:03.395 INFO:teuthology.orchestra.run.smithi196.stdout:2563 write 0x1c3ee00 thru 0x1c45e1f (0x7020 bytes) 2024-05-08T08:53:03.397 INFO:teuthology.orchestra.run.smithi196.stdout:2567 punch from 0x1877a00 to 0x1883000, (0xb600 bytes) 2024-05-08T08:53:03.398 INFO:teuthology.orchestra.run.smithi196.stdout:2568 read 0xa4c4c00 thru 0xa4d1af3 (0xcef4 bytes) 2024-05-08T08:53:03.398 INFO:teuthology.orchestra.run.smithi196.stdout:2569 punch from 0x6459000 to 0x6463400, (0xa400 bytes) 2024-05-08T08:53:03.398 INFO:teuthology.orchestra.run.smithi196.stdout:2570 punch from 0x422b200 to 0x4232c00, (0x7a00 bytes) 2024-05-08T08:53:03.398 INFO:teuthology.orchestra.run.smithi196.stdout:2574 punch from 0x4e50400 to 0x4e5d200, (0xce00 bytes) 2024-05-08T08:53:03.417 INFO:teuthology.orchestra.run.smithi196.stdout:2576 write 0xb02c400 thru 0xb02e1ed (0x1dee bytes) 2024-05-08T08:53:03.425 INFO:teuthology.orchestra.run.smithi196.stdout:2577 punch from 0x4e30400 to 0x4e36600, (0x6200 bytes) 2024-05-08T08:53:03.427 INFO:teuthology.orchestra.run.smithi196.stdout:2578 write 0x6602600 thru 0x6606903 (0x4304 bytes) 2024-05-08T08:53:03.427 INFO:teuthology.orchestra.run.smithi196.stdout:2579 read 0x8168800 thru 0x81691c8 (0x9c9 bytes) 2024-05-08T08:53:03.427 INFO:teuthology.orchestra.run.smithi196.stdout:2580 write 0x1de1c00 thru 0x1de3fb7 (0x23b8 bytes) 2024-05-08T08:53:03.434 INFO:teuthology.orchestra.run.smithi196.stdout:2583 trunc from 0xb02e200 to 0x75c3200 2024-05-08T08:53:03.435 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:03.438+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:03.442 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:53:03.442 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:53:03.442 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:53:03.442 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:53:03.442 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:53:03.451 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:53:03.452 INFO:teuthology.orchestra.run.smithi196.stdout:2584 write 0x549e000 thru 0x54a0842 (0x2843 bytes) 2024-05-08T08:53:03.455 INFO:teuthology.orchestra.run.smithi196.stdout:2588 trunc from 0x75c3200 to 0x639ec00 2024-05-08T08:53:03.475 INFO:teuthology.orchestra.run.smithi196.stdout:2590 read 0x58de800 thru 0x58e62cf (0x7ad0 bytes) 2024-05-08T08:53:03.475 INFO:teuthology.orchestra.run.smithi196.stdout:2591 read 0x32f1a00 thru 0x32fd40f (0xba10 bytes) 2024-05-08T08:53:03.486 INFO:teuthology.orchestra.run.smithi196.stdout:2593 write 0x7ac6a00 thru 0x7ad4ee6 (0xe4e7 bytes) 2024-05-08T08:53:03.487 INFO:teuthology.orchestra.run.smithi196.stdout:2594 punch from 0x594b600 to 0x5957e00, (0xc800 bytes) 2024-05-08T08:53:03.488 INFO:teuthology.orchestra.run.smithi196.stdout:2595 read 0x51f1a00 thru 0x51fb075 (0x9676 bytes) 2024-05-08T08:53:03.488 INFO:teuthology.orchestra.run.smithi196.stdout:2596 punch from 0xdf5e00 to 0xdf7400, (0x1600 bytes) 2024-05-08T08:53:03.495 INFO:teuthology.orchestra.run.smithi196.stdout:2598 trunc from 0x7ad5000 to 0xbf99e00 2024-05-08T08:53:03.513 INFO:teuthology.orchestra.run.smithi196.stdout:2599 write 0xd5e4e00 thru 0xd5e75f0 (0x27f1 bytes) 2024-05-08T08:53:03.526 INFO:teuthology.orchestra.run.smithi196.stdout:2601 write 0xea32e00 thru 0xea3e825 (0xba26 bytes) 2024-05-08T08:53:03.527 INFO:teuthology.orchestra.run.smithi196.stdout:2602 punch from 0xca14c00 to 0xca23200, (0xe600 bytes) 2024-05-08T08:53:03.532 INFO:teuthology.orchestra.run.smithi196.stdout:2604 read 0xdffa400 thru 0xe0082d8 (0xded9 bytes) 2024-05-08T08:53:03.534 INFO:teuthology.orchestra.run.smithi196.stdout:2605 punch from 0x646b200 to 0x646e800, (0x3600 bytes) 2024-05-08T08:53:03.536 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:03.539+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:03.537 INFO:teuthology.orchestra.run.smithi196.stdout:2606 trunc from 0xea3ea00 to 0xc5d7800 2024-05-08T08:53:03.550 INFO:teuthology.orchestra.run.smithi196.stdout:2607 punch from 0xa5d1400 to 0xa5dec00, (0xd800 bytes) 2024-05-08T08:53:03.553 INFO:teuthology.orchestra.run.smithi196.stdout:2608 write 0xb61ce00 thru 0xb61efb3 (0x21b4 bytes) 2024-05-08T08:53:03.553 INFO:teuthology.orchestra.run.smithi196.stdout:2612 write 0x1a29a00 thru 0x1a29bb1 (0x1b2 bytes) 2024-05-08T08:53:03.565 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:53:03.565 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:53:03.566 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:53:03.566 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:53:03.566 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.566 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6690 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1093632~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:03.566 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:53:03.566 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 31.852668917999999, 2024-05-08T08:53:03.566 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052462754, 2024-05-08T08:53:03.566 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:03.567 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:03.567 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:03.567 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:03.567 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:03.567 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:03.567 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6690 2024-05-08T08:53:03.567 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.567 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:03.567 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.567 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:03.567 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:53:03.567 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:03.568 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.568 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.568 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:03.568 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:53:03.568 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:03.568 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.569 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.569 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:03.569 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715346+0000", 2024-05-08T08:53:03.569 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.84e-07 2024-05-08T08:53:03.569 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.569 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.569 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:03.570 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715348+0000", 2024-05-08T08:53:03.570 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.114e-06 2024-05-08T08:53:03.570 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.570 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.570 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:03.571 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715353+0000", 2024-05-08T08:53:03.571 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5720000000000002e-06 2024-05-08T08:53:03.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.571 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.571 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:03.572 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715358+0000", 2024-05-08T08:53:03.572 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7140000000000001e-06 2024-05-08T08:53:03.572 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.572 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.572 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:03.572 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767638+0000", 2024-05-08T08:53:03.572 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052279562000000002 2024-05-08T08:53:03.572 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.573 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.573 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:03.573 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767713+0000", 2024-05-08T08:53:03.573 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5652e-05 2024-05-08T08:53:03.573 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.573 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.573 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:03.573 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767808+0000", 2024-05-08T08:53:03.573 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4755999999999999e-05 2024-05-08T08:53:03.573 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.573 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:03.573 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.573 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.573 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.574 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6692 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1110016~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:03.574 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:53:03.574 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 31.852649828000001, 2024-05-08T08:53:03.574 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052666495000000001, 2024-05-08T08:53:03.574 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:03.574 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:03.574 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:03.574 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:03.574 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:03.574 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:03.574 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6692 2024-05-08T08:53:03.574 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.574 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:03.574 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.575 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:03.575 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:53:03.575 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:03.575 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.575 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.575 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:03.575 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:53:03.575 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:03.576 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.576 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.576 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:03.576 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:53:03.576 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8599999999999999e-07 2024-05-08T08:53:03.577 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.577 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.577 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:03.577 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715372+0000", 2024-05-08T08:53:03.577 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0990000000000001e-06 2024-05-08T08:53:03.577 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.578 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.578 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:03.578 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715378+0000", 2024-05-08T08:53:03.578 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0680000000000002e-06 2024-05-08T08:53:03.578 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.578 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.578 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:03.578 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715383+0000", 2024-05-08T08:53:03.578 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8740000000000003e-06 2024-05-08T08:53:03.578 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.578 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.578 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:03.578 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767848+0000", 2024-05-08T08:53:03.579 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052464668999999998 2024-05-08T08:53:03.579 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.579 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.579 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:03.579 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767927+0000", 2024-05-08T08:53:03.579 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9228999999999996e-05 2024-05-08T08:53:03.579 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.579 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.579 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:03.579 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768031+0000", 2024-05-08T08:53:03.579 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010417 2024-05-08T08:53:03.579 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.579 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:03.579 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.580 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.580 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.580 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6694 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1126400~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:03.580 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:53:03.580 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 31.852624931000001, 2024-05-08T08:53:03.580 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052848829, 2024-05-08T08:53:03.580 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:03.580 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:03.580 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:03.581 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:03.581 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:03.581 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:03.581 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6694 2024-05-08T08:53:03.581 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.581 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:03.581 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.581 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:03.581 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:53:03.581 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:03.581 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.581 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.581 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:03.581 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:53:03.581 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:03.582 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.582 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.582 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:03.582 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715390+0000", 2024-05-08T08:53:03.582 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9900000000000001e-07 2024-05-08T08:53:03.582 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.582 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.582 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:03.582 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715392+0000", 2024-05-08T08:53:03.582 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8279999999999999e-06 2024-05-08T08:53:03.582 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.582 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.583 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:03.583 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715398+0000", 2024-05-08T08:53:03.583 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7619999999999998e-06 2024-05-08T08:53:03.583 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.583 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.583 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:03.583 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715403+0000", 2024-05-08T08:53:03.583 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7790000000000003e-06 2024-05-08T08:53:03.583 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.583 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.583 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:03.583 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768067+0000", 2024-05-08T08:53:03.584 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052663785999999997 2024-05-08T08:53:03.584 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.584 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.584 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:03.584 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768142+0000", 2024-05-08T08:53:03.584 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5018999999999994e-05 2024-05-08T08:53:03.584 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.584 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.584 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:03.584 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768238+0000", 2024-05-08T08:53:03.584 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6255999999999995e-05 2024-05-08T08:53:03.584 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.584 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:03.584 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.584 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.584 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.585 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6696 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1142784~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:03.585 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:53:03.585 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 31.852605423, 2024-05-08T08:53:03.585 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.053051057999999998, 2024-05-08T08:53:03.585 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:03.585 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:03.585 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:03.585 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:03.585 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:03.585 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:03.585 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6696 2024-05-08T08:53:03.585 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.585 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:03.585 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.586 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:03.586 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:53:03.586 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:03.586 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.586 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.586 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:03.586 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:53:03.586 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:03.586 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.586 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.586 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:03.586 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:53:03.586 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-05-08T08:53:03.587 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.587 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.587 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:03.587 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715411+0000", 2024-05-08T08:53:03.587 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9630000000000001e-06 2024-05-08T08:53:03.587 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.587 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.587 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:03.587 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715417+0000", 2024-05-08T08:53:03.587 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7250000000000002e-06 2024-05-08T08:53:03.587 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.587 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.587 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:03.587 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715421+0000", 2024-05-08T08:53:03.587 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4089999999999996e-06 2024-05-08T08:53:03.588 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.588 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.588 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:03.588 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768277+0000", 2024-05-08T08:53:03.588 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052855293999999997 2024-05-08T08:53:03.588 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.588 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.588 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:03.588 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768358+0000", 2024-05-08T08:53:03.588 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1720999999999999e-05 2024-05-08T08:53:03.588 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.588 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.588 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:03.588 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768460+0000", 2024-05-08T08:53:03.588 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010154600000000001 2024-05-08T08:53:03.589 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.589 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:03.589 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.589 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.589 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.589 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6698 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1159168~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:03.589 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:53:03.589 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 31.852586555999999, 2024-05-08T08:53:03.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.053245726, 2024-05-08T08:53:03.590 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:03.590 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:03.590 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:03.590 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:03.590 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:03.590 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:03.590 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6698 2024-05-08T08:53:03.590 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.590 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:03.590 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.591 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:03.591 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:53:03.591 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:03.591 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.591 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.591 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:03.592 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:53:03.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:03.592 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.592 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.592 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:03.592 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716936+0000", 2024-05-08T08:53:03.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001508665 2024-05-08T08:53:03.592 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.592 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.593 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:03.593 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716941+0000", 2024-05-08T08:53:03.593 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1910000000000002e-06 2024-05-08T08:53:03.593 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.593 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.593 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:03.593 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716950+0000", 2024-05-08T08:53:03.593 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1900000000000001e-06 2024-05-08T08:53:03.593 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.593 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.593 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:03.593 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716958+0000", 2024-05-08T08:53:03.593 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8019999999999992e-06 2024-05-08T08:53:03.593 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.594 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.594 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:03.594 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768499+0000", 2024-05-08T08:53:03.594 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051540926000000001 2024-05-08T08:53:03.594 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.594 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.594 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:03.594 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768574+0000", 2024-05-08T08:53:03.594 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4988999999999996e-05 2024-05-08T08:53:03.594 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.594 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.594 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:03.594 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768673+0000", 2024-05-08T08:53:03.594 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9963000000000005e-05 2024-05-08T08:53:03.595 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.595 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:03.595 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.595 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.595 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.595 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6706 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1224704~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:03.595 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:53:03.595 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 31.850981249, 2024-05-08T08:53:03.595 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052435598, 2024-05-08T08:53:03.595 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:03.595 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:03.595 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:03.595 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:03.595 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:03.596 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:03.596 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6706 2024-05-08T08:53:03.596 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.596 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:03.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.596 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:03.596 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:53:03.596 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:03.596 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.596 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:03.596 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:53:03.596 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:03.596 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.597 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:03.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:53:03.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:53:03.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.597 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:03.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717035+0000", 2024-05-08T08:53:03.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9290000000000001e-06 2024-05-08T08:53:03.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.597 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:03.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717041+0000", 2024-05-08T08:53:03.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6720000000000001e-06 2024-05-08T08:53:03.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:03.598 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717046+0000", 2024-05-08T08:53:03.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5650000000000003e-06 2024-05-08T08:53:03.598 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:03.598 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769311+0000", 2024-05-08T08:53:03.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052265645999999999 2024-05-08T08:53:03.598 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:03.598 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769379+0000", 2024-05-08T08:53:03.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7391000000000006e-05 2024-05-08T08:53:03.598 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.599 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.599 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:03.599 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769469+0000", 2024-05-08T08:53:03.599 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0025999999999995e-05 2024-05-08T08:53:03.599 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.599 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:03.599 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.599 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.599 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.599 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6708 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1241088~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:03.599 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:53:03.599 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 31.850963145000001, 2024-05-08T08:53:03.599 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052640552, 2024-05-08T08:53:03.599 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:03.599 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:03.600 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:03.600 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:03.600 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:03.600 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:03.600 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6708 2024-05-08T08:53:03.600 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.600 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:03.600 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.600 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:03.600 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:53:03.600 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:03.600 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.600 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.601 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:03.601 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:53:03.601 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:03.601 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.601 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:03.601 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717052+0000", 2024-05-08T08:53:03.601 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:53:03.601 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.601 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:03.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717053+0000", 2024-05-08T08:53:03.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8330000000000001e-06 2024-05-08T08:53:03.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:03.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717058+0000", 2024-05-08T08:53:03.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.933e-06 2024-05-08T08:53:03.603 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.603 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:03.603 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717063+0000", 2024-05-08T08:53:03.604 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2180000000000001e-06 2024-05-08T08:53:03.604 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.604 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.604 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:03.604 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769507+0000", 2024-05-08T08:53:03.604 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052444866999999999 2024-05-08T08:53:03.605 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.605 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.605 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:03.605 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769589+0000", 2024-05-08T08:53:03.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2039000000000005e-05 2024-05-08T08:53:03.606 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.606 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:03.606 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769692+0000", 2024-05-08T08:53:03.606 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102252 2024-05-08T08:53:03.607 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.607 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:03.607 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.607 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6710 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1257472~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:03.608 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:53:03.608 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 31.850945120999999, 2024-05-08T08:53:03.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.05283057, 2024-05-08T08:53:03.608 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:03.608 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:03.609 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:03.609 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:03.609 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:03.609 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:03.609 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6710 2024-05-08T08:53:03.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.610 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:03.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.610 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:03.610 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:53:03.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:03.610 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.611 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:03.611 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:53:03.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:03.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.612 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:03.612 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718951+0000", 2024-05-08T08:53:03.612 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0018817809999999999 2024-05-08T08:53:03.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.612 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:03.612 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718956+0000", 2024-05-08T08:53:03.613 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5399999999999997e-06 2024-05-08T08:53:03.613 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.613 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.613 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:03.613 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718964+0000", 2024-05-08T08:53:03.613 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6629999999999999e-06 2024-05-08T08:53:03.614 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.614 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.614 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:03.614 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718972+0000", 2024-05-08T08:53:03.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5229999999999998e-06 2024-05-08T08:53:03.614 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:03.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769729+0000", 2024-05-08T08:53:03.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050756959999999997 2024-05-08T08:53:03.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:03.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769803+0000", 2024-05-08T08:53:03.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4538999999999999e-05 2024-05-08T08:53:03.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:03.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769900+0000", 2024-05-08T08:53:03.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6564000000000006e-05 2024-05-08T08:53:03.617 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.617 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:03.617 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.617 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.617 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6726 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1388544~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:03.617 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:53:03.617 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 31.848870348999998, 2024-05-08T08:53:03.618 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052451506000000002, 2024-05-08T08:53:03.618 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:03.618 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:03.618 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:03.618 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:03.618 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:03.618 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:03.619 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6726 2024-05-08T08:53:03.619 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.619 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:03.619 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.619 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:03.619 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:53:03.619 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:03.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.620 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:03.620 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:53:03.620 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:03.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:03.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719200+0000", 2024-05-08T08:53:03.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5865999999999997e-05 2024-05-08T08:53:03.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:03.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719204+0000", 2024-05-08T08:53:03.622 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8630000000000001e-06 2024-05-08T08:53:03.622 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.622 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.622 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:03.623 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719215+0000", 2024-05-08T08:53:03.623 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1590000000000001e-05 2024-05-08T08:53:03.623 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.623 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.623 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:03.623 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719220+0000", 2024-05-08T08:53:03.624 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1619999999999999e-06 2024-05-08T08:53:03.624 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.624 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:03.624 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771425+0000", 2024-05-08T08:53:03.625 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052204249000000001 2024-05-08T08:53:03.625 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.625 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.625 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:03.625 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771498+0000", 2024-05-08T08:53:03.625 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3782999999999995e-05 2024-05-08T08:53:03.626 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.626 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.626 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:03.626 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771595+0000", 2024-05-08T08:53:03.626 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6992999999999998e-05 2024-05-08T08:53:03.626 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.626 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:03.627 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.627 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.627 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.627 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6728 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1404928~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:03.627 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:53:03.627 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 31.848782944, 2024-05-08T08:53:03.627 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052590747, 2024-05-08T08:53:03.628 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:03.628 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:03.628 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:03.628 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:03.628 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:03.628 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:03.628 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6728 2024-05-08T08:53:03.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.629 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:03.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.629 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:03.629 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:53:03.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:03.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.630 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.630 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:03.630 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:53:03.630 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:03.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.631 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.631 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:03.631 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719232+0000", 2024-05-08T08:53:03.631 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4000000000000001e-07 2024-05-08T08:53:03.631 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.631 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.632 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:03.632 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719234+0000", 2024-05-08T08:53:03.632 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4499999999999998e-06 2024-05-08T08:53:03.632 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.632 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.632 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:03.633 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719240+0000", 2024-05-08T08:53:03.633 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5720000000000002e-06 2024-05-08T08:53:03.633 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.633 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.633 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:03.633 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719245+0000", 2024-05-08T08:53:03.633 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8690000000000003e-06 2024-05-08T08:53:03.633 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.633 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.633 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:03.633 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771636+0000", 2024-05-08T08:53:03.633 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052390979999999997 2024-05-08T08:53:03.633 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.633 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.634 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:03.634 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771717+0000", 2024-05-08T08:53:03.634 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1291999999999993e-05 2024-05-08T08:53:03.634 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.634 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:03.634 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771822+0000", 2024-05-08T08:53:03.634 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104944 2024-05-08T08:53:03.634 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.634 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:03.634 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.634 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.635 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6730 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1421312~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:03.635 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:53:03.635 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 31.848762701999998, 2024-05-08T08:53:03.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052770477000000003, 2024-05-08T08:53:03.635 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:03.635 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:03.635 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:03.635 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:03.635 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:03.635 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:03.635 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6730 2024-05-08T08:53:03.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.635 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:03.635 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.636 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:03.636 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:53:03.636 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:03.636 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.636 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:03.636 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:53:03.636 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:03.636 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.636 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:03.636 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:53:03.636 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9000000000000002e-07 2024-05-08T08:53:03.636 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.637 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:03.637 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719254+0000", 2024-05-08T08:53:03.637 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4099999999999998e-06 2024-05-08T08:53:03.637 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.637 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.637 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:03.637 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719263+0000", 2024-05-08T08:53:03.637 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4109999999999996e-06 2024-05-08T08:53:03.637 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.637 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.637 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:03.637 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719268+0000", 2024-05-08T08:53:03.637 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2669999999999997e-06 2024-05-08T08:53:03.637 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.638 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.638 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:03.638 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771846+0000", 2024-05-08T08:53:03.638 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052577677000000003 2024-05-08T08:53:03.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.638 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.638 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:03.638 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771920+0000", 2024-05-08T08:53:03.638 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4460000000000002e-05 2024-05-08T08:53:03.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.638 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.638 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:03.638 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772022+0000", 2024-05-08T08:53:03.638 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010186199999999999 2024-05-08T08:53:03.639 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.639 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:03.639 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.639 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.639 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.639 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6732 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1437696~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:03.639 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:53:03.639 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 31.848737894999999, 2024-05-08T08:53:03.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052905575000000003, 2024-05-08T08:53:03.639 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:03.639 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:03.639 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:03.639 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:03.639 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:03.639 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:03.640 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6732 2024-05-08T08:53:03.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.640 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:03.640 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.640 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:03.640 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:53:03.640 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:03.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.640 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.640 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:03.640 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:53:03.640 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:03.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.640 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.640 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:03.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721392+0000", 2024-05-08T08:53:03.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002115287 2024-05-08T08:53:03.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.641 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:03.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721396+0000", 2024-05-08T08:53:03.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1679999999999997e-06 2024-05-08T08:53:03.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.641 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:03.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721404+0000", 2024-05-08T08:53:03.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4409999999999993e-06 2024-05-08T08:53:03.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:03.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721410+0000", 2024-05-08T08:53:03.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9150000000000001e-06 2024-05-08T08:53:03.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:03.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772061+0000", 2024-05-08T08:53:03.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050651068 2024-05-08T08:53:03.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:03.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772127+0000", 2024-05-08T08:53:03.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5424000000000002e-05 2024-05-08T08:53:03.643 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.643 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.643 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:03.643 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772182+0000", 2024-05-08T08:53:03.643 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5272000000000002e-05 2024-05-08T08:53:03.643 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.644 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:03.644 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.644 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.644 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.644 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6749 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1585152~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:03.644 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:53:03.644 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 31.846437125000001, 2024-05-08T08:53:03.644 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052350264000000001, 2024-05-08T08:53:03.644 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:03.644 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:03.644 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:03.644 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:03.644 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:03.644 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:03.645 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6749 2024-05-08T08:53:03.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.645 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:03.645 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.645 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:03.645 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:53:03.645 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:03.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.645 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.645 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:03.645 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:53:03.645 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:03.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.646 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:03.646 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721578+0000", 2024-05-08T08:53:03.646 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0399999999999996e-07 2024-05-08T08:53:03.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.646 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:03.646 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721580+0000", 2024-05-08T08:53:03.646 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5450000000000002e-06 2024-05-08T08:53:03.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.646 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:03.647 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721587+0000", 2024-05-08T08:53:03.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0190000000000001e-06 2024-05-08T08:53:03.647 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.647 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.647 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:03.647 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721593+0000", 2024-05-08T08:53:03.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5659999999999997e-06 2024-05-08T08:53:03.647 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.647 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.647 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:03.647 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773759+0000", 2024-05-08T08:53:03.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052166553999999997 2024-05-08T08:53:03.647 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.648 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.648 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:03.648 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773833+0000", 2024-05-08T08:53:03.648 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3913999999999997e-05 2024-05-08T08:53:03.648 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.648 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.648 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:03.648 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773927+0000", 2024-05-08T08:53:03.648 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4061999999999995e-05 2024-05-08T08:53:03.648 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.648 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:03.648 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.648 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.648 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.648 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6751 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1601536~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:03.649 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:53:03.649 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 31.846413846000001, 2024-05-08T08:53:03.649 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052461302000000001, 2024-05-08T08:53:03.649 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:03.649 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:03.649 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:03.649 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:03.649 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:03.649 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:03.649 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6751 2024-05-08T08:53:03.649 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.649 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:03.649 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.650 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:03.650 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:53:03.650 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:03.650 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.650 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.650 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:03.650 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:53:03.650 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:03.650 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.650 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.650 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:03.650 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722939+0000", 2024-05-08T08:53:03.650 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001338601 2024-05-08T08:53:03.650 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.651 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.651 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:03.651 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722944+0000", 2024-05-08T08:53:03.651 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4469999999999999e-06 2024-05-08T08:53:03.651 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.651 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.651 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:03.651 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722955+0000", 2024-05-08T08:53:03.651 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1892999999999999e-05 2024-05-08T08:53:03.651 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.651 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.651 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:03.651 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722963+0000", 2024-05-08T08:53:03.651 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1080000000000004e-06 2024-05-08T08:53:03.652 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.652 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.652 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:03.652 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773964+0000", 2024-05-08T08:53:03.652 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051001022 2024-05-08T08:53:03.652 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.652 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.652 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:03.652 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.774006+0000", 2024-05-08T08:53:03.652 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2729999999999999e-05 2024-05-08T08:53:03.652 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.652 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.652 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:03.652 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.774062+0000", 2024-05-08T08:53:03.652 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5501000000000003e-05 2024-05-08T08:53:03.653 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.653 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:03.653 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.653 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.653 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.653 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6760 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1683456~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:03.653 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:53:03.653 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 31.844904469999999, 2024-05-08T08:53:03.653 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052571312000000002, 2024-05-08T08:53:03.653 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:03.653 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:03.653 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:03.653 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:03.654 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:03.654 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:03.654 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6760 2024-05-08T08:53:03.654 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.654 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:03.654 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.655 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:03.655 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:53:03.655 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:03.655 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.655 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.655 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:03.655 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:53:03.655 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:03.655 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.655 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.655 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:03.655 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:53:03.655 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2100000000000002e-07 2024-05-08T08:53:03.655 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.655 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.656 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:03.656 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723113+0000", 2024-05-08T08:53:03.656 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3149999999999999e-06 2024-05-08T08:53:03.656 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.656 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.656 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:03.656 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723118+0000", 2024-05-08T08:53:03.656 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8860000000000001e-06 2024-05-08T08:53:03.656 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.656 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.656 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:03.656 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723123+0000", 2024-05-08T08:53:03.656 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6770000000000003e-06 2024-05-08T08:53:03.656 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.656 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.657 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:03.657 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775414+0000", 2024-05-08T08:53:03.657 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052290692999999999 2024-05-08T08:53:03.657 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.657 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.657 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:03.657 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775459+0000", 2024-05-08T08:53:03.657 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5639999999999997e-05 2024-05-08T08:53:03.657 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.657 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.657 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:03.657 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775681+0000", 2024-05-08T08:53:03.657 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00022168000000000001 2024-05-08T08:53:03.657 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.657 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:03.658 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.658 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.658 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.658 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6761 3.16 3:6bd6d6da:::rbd_data.1140782590a5.0000000000000015:6 [read 1095680~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:03.658 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:53:03.658 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 31.844885250000001, 2024-05-08T08:53:03.658 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052751275, 2024-05-08T08:53:03.658 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:03.658 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:03.658 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:03.658 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:03.658 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:03.658 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:03.659 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6761 2024-05-08T08:53:03.659 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.659 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:03.659 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.659 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:03.659 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:53:03.659 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:03.659 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.659 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.659 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:03.659 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:53:03.659 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:03.659 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.659 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.659 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:03.660 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724939+0000", 2024-05-08T08:53:03.660 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001809817 2024-05-08T08:53:03.660 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.660 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.660 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:03.660 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724943+0000", 2024-05-08T08:53:03.660 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4920000000000004e-06 2024-05-08T08:53:03.660 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.660 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.660 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:03.660 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724956+0000", 2024-05-08T08:53:03.660 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2427e-05 2024-05-08T08:53:03.660 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.660 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.661 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:03.661 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724963+0000", 2024-05-08T08:53:03.661 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2559999999999996e-06 2024-05-08T08:53:03.661 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.661 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.661 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:03.661 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775711+0000", 2024-05-08T08:53:03.661 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050747621 2024-05-08T08:53:03.661 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.661 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.661 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:03.661 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775785+0000", 2024-05-08T08:53:03.661 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4634000000000004e-05 2024-05-08T08:53:03.661 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.661 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.662 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:03.662 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775880+0000", 2024-05-08T08:53:03.662 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5027999999999999e-05 2024-05-08T08:53:03.662 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.662 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:03.662 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.662 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.662 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.662 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6770 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1765376~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:03.662 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:53:03.662 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 31.842854155000001, 2024-05-08T08:53:03.662 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052339296, 2024-05-08T08:53:03.662 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:03.662 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:03.663 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:03.663 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:03.663 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:03.663 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:03.663 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6770 2024-05-08T08:53:03.663 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.663 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:03.663 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.663 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:03.663 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:53:03.663 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:03.663 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.663 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.663 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:03.664 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:53:03.664 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:03.664 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.664 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.664 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:03.664 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725161+0000", 2024-05-08T08:53:03.664 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-05-08T08:53:03.664 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.664 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.664 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:03.664 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725163+0000", 2024-05-08T08:53:03.664 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4370000000000001e-06 2024-05-08T08:53:03.664 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.664 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.665 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:03.665 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725169+0000", 2024-05-08T08:53:03.665 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8409999999999996e-06 2024-05-08T08:53:03.665 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.665 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.665 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:03.665 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725174+0000", 2024-05-08T08:53:03.665 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7249999999999997e-06 2024-05-08T08:53:03.666 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.666 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.666 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:03.666 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777343+0000", 2024-05-08T08:53:03.666 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052169633999999999 2024-05-08T08:53:03.666 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.666 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.666 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:03.666 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777409+0000", 2024-05-08T08:53:03.666 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6033999999999998e-05 2024-05-08T08:53:03.666 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.666 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.666 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:03.667 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777499+0000", 2024-05-08T08:53:03.667 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0224999999999998e-05 2024-05-08T08:53:03.667 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.667 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:03.667 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.667 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.667 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.667 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6771 3.16 3:6bd6d6da:::rbd_data.1140782590a5.0000000000000015:6 [read 1177600~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:03.667 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:53:03.667 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 31.842834337999999, 2024-05-08T08:53:03.667 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052438604999999999, 2024-05-08T08:53:03.667 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:03.667 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:03.667 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:03.667 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:03.668 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:03.668 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:03.668 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6771 2024-05-08T08:53:03.668 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.668 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:03.668 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.668 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:03.668 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:53:03.668 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:03.668 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.668 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.668 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:03.668 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:53:03.668 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:03.669 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.669 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.669 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:03.669 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:53:03.669 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:53:03.669 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.669 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.669 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:03.669 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725188+0000", 2024-05-08T08:53:03.669 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1579999999999999e-06 2024-05-08T08:53:03.669 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.669 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.669 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:03.669 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725195+0000", 2024-05-08T08:53:03.670 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8909999999999995e-06 2024-05-08T08:53:03.670 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.670 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.670 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:03.670 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725200+0000", 2024-05-08T08:53:03.670 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9810000000000003e-06 2024-05-08T08:53:03.670 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.670 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.670 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:03.670 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777527+0000", 2024-05-08T08:53:03.670 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052326582000000003 2024-05-08T08:53:03.670 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.670 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.670 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:03.671 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777566+0000", 2024-05-08T08:53:03.671 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9447999999999999e-05 2024-05-08T08:53:03.671 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.671 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.671 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:03.671 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777619+0000", 2024-05-08T08:53:03.671 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2135000000000001e-05 2024-05-08T08:53:03.671 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.671 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:03.671 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.671 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.672 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.672 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6772 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1781760~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:03.672 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:53:03.672 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 31.842806281000001, 2024-05-08T08:53:03.672 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052601799999999997, 2024-05-08T08:53:03.672 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:03.672 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:03.672 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:03.672 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:03.672 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:03.672 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:03.672 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6772 2024-05-08T08:53:03.673 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.673 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:03.673 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.673 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:03.673 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:53:03.673 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:03.673 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.673 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.673 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:03.673 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:53:03.673 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:03.673 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.673 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.673 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:03.673 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726939+0000", 2024-05-08T08:53:03.674 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001730493 2024-05-08T08:53:03.674 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.674 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.674 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:03.674 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726943+0000", 2024-05-08T08:53:03.674 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7330000000000003e-06 2024-05-08T08:53:03.674 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.674 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.674 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:03.674 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726954+0000", 2024-05-08T08:53:03.674 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1107000000000001e-05 2024-05-08T08:53:03.674 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.675 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.675 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:03.675 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726962+0000", 2024-05-08T08:53:03.675 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.503e-06 2024-05-08T08:53:03.675 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.675 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.675 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:03.675 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777653+0000", 2024-05-08T08:53:03.675 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050691542999999999 2024-05-08T08:53:03.675 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.676 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.676 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:03.676 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777719+0000", 2024-05-08T08:53:03.676 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5228000000000006e-05 2024-05-08T08:53:03.676 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.676 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.676 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:03.676 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777810+0000", 2024-05-08T08:53:03.676 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1193000000000006e-05 2024-05-08T08:53:03.676 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.676 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:03.677 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.677 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.677 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.677 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6782 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1880064~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:03.677 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:53:03.677 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 31.840837774000001, 2024-05-08T08:53:03.677 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052326492000000002, 2024-05-08T08:53:03.677 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:03.677 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:03.677 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:03.677 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:03.677 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:03.677 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:03.677 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6782 2024-05-08T08:53:03.677 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.678 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:03.678 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.678 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:03.678 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:53:03.678 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:03.678 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.678 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.678 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:03.678 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:53:03.678 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:03.678 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.678 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.678 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:03.679 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728939+0000", 2024-05-08T08:53:03.679 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001762719 2024-05-08T08:53:03.679 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.679 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.679 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:03.679 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728944+0000", 2024-05-08T08:53:03.679 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.746e-06 2024-05-08T08:53:03.679 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.679 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.679 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:03.679 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728955+0000", 2024-05-08T08:53:03.679 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1043000000000001e-05 2024-05-08T08:53:03.679 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.679 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.679 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:03.680 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728963+0000", 2024-05-08T08:53:03.680 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5809999999999999e-06 2024-05-08T08:53:03.680 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.680 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.680 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:03.680 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779320+0000", 2024-05-08T08:53:03.680 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050357290999999998 2024-05-08T08:53:03.680 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.680 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.680 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:03.680 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779401+0000", 2024-05-08T08:53:03.680 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0574000000000005e-05 2024-05-08T08:53:03.680 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:03.681 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:03.681 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:03.681 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779503+0000", 2024-05-08T08:53:03.681 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102538 2024-05-08T08:53:03.681 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.681 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:03.681 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.681 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:03.681 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:03.681 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:53:03.684 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:03.685+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:03.684 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:53:03.686 INFO:teuthology.orchestra.run.smithi196.stdout:2613 write 0xdb52600 thru 0xdb5e1ff (0xbc00 bytes) 2024-05-08T08:53:03.686 INFO:teuthology.orchestra.run.smithi196.stdout:2614 read 0xb4d3a00 thru 0xb4db9e4 (0x7fe5 bytes) 2024-05-08T08:53:03.686 INFO:teuthology.orchestra.run.smithi196.stdout:2615 write 0xbf1e400 thru 0xbf2c624 (0xe225 bytes) 2024-05-08T08:53:03.686 INFO:teuthology.orchestra.run.smithi196.stdout:2618 read 0x994c800 thru 0x994e9da (0x21db bytes) 2024-05-08T08:53:03.686 INFO:teuthology.orchestra.run.smithi196.stdout:2623 write 0xd70a800 thru 0xd7116ff (0x6f00 bytes) 2024-05-08T08:53:03.687 INFO:teuthology.orchestra.run.smithi196.stdout:2624 read 0xd4f9c00 thru 0xd4ffb74 (0x5f75 bytes) 2024-05-08T08:53:03.687 INFO:teuthology.orchestra.run.smithi196.stdout:2628 trunc from 0xdb5e200 to 0xda27400 2024-05-08T08:53:03.687 INFO:teuthology.orchestra.run.smithi196.stdout:2629 read 0x8a7ba00 thru 0x8a7efda (0x35db bytes) 2024-05-08T08:53:03.687 INFO:teuthology.orchestra.run.smithi196.stdout:2630 punch from 0x677a800 to 0x677ce00, (0x2600 bytes) 2024-05-08T08:53:03.687 INFO:teuthology.orchestra.run.smithi196.stdout:2631 trunc from 0xda27400 to 0x4932e00 2024-05-08T08:53:03.687 INFO:teuthology.orchestra.run.smithi196.stdout:skipping zero length punch hole 2024-05-08T08:53:03.687 INFO:teuthology.orchestra.run.smithi196.stdout:2636 write 0x1b0200 thru 0x1b81e6 (0x7fe7 bytes) 2024-05-08T08:53:03.687 INFO:teuthology.orchestra.run.smithi196.stdout:2640 trunc from 0x4932e00 to 0xdd89800 2024-05-08T08:53:03.687 INFO:teuthology.orchestra.run.smithi196.stdout:2641 write 0x3ab6c00 thru 0x3abde15 (0x7216 bytes) 2024-05-08T08:53:03.687 INFO:teuthology.orchestra.run.smithi196.stdout:2643 read 0x760400 thru 0x760a29 (0x62a bytes) 2024-05-08T08:53:03.687 INFO:teuthology.orchestra.run.smithi196.stdout:2645 write 0x7f2d400 thru 0x7f2dacc (0x6cd bytes) 2024-05-08T08:53:03.687 INFO:teuthology.orchestra.run.smithi196.stdout:2646 read 0x1d10e00 thru 0x1d17754 (0x6955 bytes) 2024-05-08T08:53:03.688 INFO:teuthology.orchestra.run.smithi196.stdout:2647 read 0x41fc400 thru 0x41fe22c (0x1e2d bytes) 2024-05-08T08:53:03.688 INFO:teuthology.orchestra.run.smithi196.stdout:2649 clone 16 order 23 su 4194304 sc 8 2024-05-08T08:53:03.783 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:03.787+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:03.797 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:03.797 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:53:03.797 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:53:03.797 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:03.805 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:53:03.884 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:03.887+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:03.914 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:03.914 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:53:03.915 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:53:03.915 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:53:03.915 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:03.924 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:53:03.985 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:03.989+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:04.035 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:04.035 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:53:04.036 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:53:04.036 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:53:04.036 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.036 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4621.0:9089 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 23339520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:53:04.036 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:53:04.036 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 31.544427623000001, 2024-05-08T08:53:04.036 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.13814907200000001, 2024-05-08T08:53:04.036 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:04.036 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:04.036 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:53:04.036 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:04.037 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4621", 2024-05-08T08:53:04.037 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:04.037 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9089 2024-05-08T08:53:04.037 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.037 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:04.037 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.037 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:04.037 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:53:04.037 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:04.037 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.037 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.037 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:04.037 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:53:04.037 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:04.037 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.038 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.038 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:04.038 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493005+0000", 2024-05-08T08:53:04.038 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1700000000000001e-06 2024-05-08T08:53:04.038 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.038 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.038 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:04.038 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.513963+0000", 2024-05-08T08:53:04.038 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.020957581999999999 2024-05-08T08:53:04.039 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.039 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.039 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:04.039 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.513995+0000", 2024-05-08T08:53:04.039 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1993999999999999e-05 2024-05-08T08:53:04.039 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.039 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.039 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:04.039 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514007+0000", 2024-05-08T08:53:04.040 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1581e-05 2024-05-08T08:53:04.040 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.040 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.040 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:04.040 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514122+0000", 2024-05-08T08:53:04.040 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000115282 2024-05-08T08:53:04.040 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.040 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.040 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op must wait for map", 2024-05-08T08:53:04.040 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514128+0000", 2024-05-08T08:53:04.040 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9950000000000002e-06 2024-05-08T08:53:04.040 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.041 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.041 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:04.041 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.515800+0000", 2024-05-08T08:53:04.041 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001671886 2024-05-08T08:53:04.041 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.041 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.041 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:04.041 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.515893+0000", 2024-05-08T08:53:04.041 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3490000000000001e-05 2024-05-08T08:53:04.041 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.041 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.041 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 3", 2024-05-08T08:53:04.042 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.517647+0000", 2024-05-08T08:53:04.042 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0017535630000000001 2024-05-08T08:53:04.042 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.042 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.042 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:53:04.042 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.627578+0000", 2024-05-08T08:53:04.042 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.109931557 2024-05-08T08:53:04.042 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.042 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.042 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:53:04.042 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631055+0000", 2024-05-08T08:53:04.042 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0034761420000000002 2024-05-08T08:53:04.042 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.043 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.043 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:53:04.043 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631096+0000", 2024-05-08T08:53:04.043 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1059000000000001e-05 2024-05-08T08:53:04.043 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.043 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.043 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:04.043 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631151+0000", 2024-05-08T08:53:04.043 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5770999999999998e-05 2024-05-08T08:53:04.043 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.043 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:04.043 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.043 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.043 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.043 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1065 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 10944512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:04.044 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485411+0000", 2024-05-08T08:53:04.044 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 27.552018937, 2024-05-08T08:53:04.044 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070281386000000001, 2024-05-08T08:53:04.044 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:04.044 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:04.044 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:04.044 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:04.044 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:04.044 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:04.044 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1065 2024-05-08T08:53:04.044 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.044 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:04.044 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.044 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:04.045 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485411+0000", 2024-05-08T08:53:04.045 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:04.045 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.045 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.045 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:04.045 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485411+0000", 2024-05-08T08:53:04.045 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:04.045 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.045 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.045 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:04.045 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485411+0000", 2024-05-08T08:53:04.045 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5600000000000001e-07 2024-05-08T08:53:04.045 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.045 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.046 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:04.046 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485414+0000", 2024-05-08T08:53:04.046 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2299999999999998e-06 2024-05-08T08:53:04.046 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.046 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.046 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:04.046 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485420+0000", 2024-05-08T08:53:04.046 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2179999999999996e-06 2024-05-08T08:53:04.046 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.046 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.046 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:04.046 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485425+0000", 2024-05-08T08:53:04.046 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9830000000000004e-06 2024-05-08T08:53:04.047 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.047 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.047 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:04.047 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555468+0000", 2024-05-08T08:53:04.047 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070043541000000001 2024-05-08T08:53:04.047 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.047 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.047 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:04.047 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555566+0000", 2024-05-08T08:53:04.047 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8082e-05 2024-05-08T08:53:04.047 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.047 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.047 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:04.047 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555692+0000", 2024-05-08T08:53:04.047 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000125976 2024-05-08T08:53:04.047 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.048 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:04.048 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.048 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.048 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.048 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1067 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 10985472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:04.048 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485431+0000", 2024-05-08T08:53:04.048 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 27.551998802, 2024-05-08T08:53:04.048 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070412460999999996, 2024-05-08T08:53:04.048 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:04.048 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:04.048 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:04.048 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:04.048 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:04.048 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:04.048 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1067 2024-05-08T08:53:04.048 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.049 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:04.049 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.049 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:04.049 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485431+0000", 2024-05-08T08:53:04.049 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:04.049 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.049 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.049 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:04.049 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485431+0000", 2024-05-08T08:53:04.049 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:04.049 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.049 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.049 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:04.049 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485432+0000", 2024-05-08T08:53:04.049 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8299999999999998e-07 2024-05-08T08:53:04.049 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.050 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.050 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:04.050 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485434+0000", 2024-05-08T08:53:04.050 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0849999999999999e-06 2024-05-08T08:53:04.050 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.050 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.050 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:04.050 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485440+0000", 2024-05-08T08:53:04.050 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0020000000000003e-06 2024-05-08T08:53:04.050 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.050 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.050 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:04.050 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485445+0000", 2024-05-08T08:53:04.050 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5180000000000004e-06 2024-05-08T08:53:04.050 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.051 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.051 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:04.051 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555731+0000", 2024-05-08T08:53:04.051 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070286214 2024-05-08T08:53:04.051 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.051 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.051 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:04.051 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555787+0000", 2024-05-08T08:53:04.051 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5992999999999997e-05 2024-05-08T08:53:04.051 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.051 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.051 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:04.051 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555844+0000", 2024-05-08T08:53:04.051 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6266e-05 2024-05-08T08:53:04.051 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.052 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:04.052 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.052 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.052 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.052 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1071 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11026432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:04.052 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:53:04.052 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 27.551978131999999, 2024-05-08T08:53:04.052 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070580037999999998, 2024-05-08T08:53:04.052 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:04.052 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:04.052 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:04.052 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:04.052 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:04.052 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:04.052 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1071 2024-05-08T08:53:04.052 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.053 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:04.053 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.053 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:04.053 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:53:04.053 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:04.053 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.053 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.053 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:04.053 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:53:04.053 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:04.053 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.053 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.053 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:04.053 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:53:04.053 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2800000000000002e-07 2024-05-08T08:53:04.054 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.054 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.054 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:04.054 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485454+0000", 2024-05-08T08:53:04.054 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2290000000000002e-06 2024-05-08T08:53:04.054 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.054 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.054 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:04.054 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485461+0000", 2024-05-08T08:53:04.054 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9920000000000001e-06 2024-05-08T08:53:04.054 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.054 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.054 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:04.054 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485467+0000", 2024-05-08T08:53:04.055 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3979999999999998e-06 2024-05-08T08:53:04.055 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.055 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.055 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:04.055 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555880+0000", 2024-05-08T08:53:04.055 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070413165 2024-05-08T08:53:04.055 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.055 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.055 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:04.055 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555926+0000", 2024-05-08T08:53:04.055 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6217000000000002e-05 2024-05-08T08:53:04.055 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.055 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.055 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:04.055 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556032+0000", 2024-05-08T08:53:04.055 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000105609 2024-05-08T08:53:04.056 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.056 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:04.056 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.056 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.056 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.056 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1073 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11067392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:04.056 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:53:04.056 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 27.551956911000001, 2024-05-08T08:53:04.056 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070821404000000004, 2024-05-08T08:53:04.056 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:04.056 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:04.056 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:04.056 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:04.057 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:04.057 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:04.057 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1073 2024-05-08T08:53:04.057 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.057 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:04.057 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.057 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:04.057 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:53:04.057 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:04.057 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.057 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.058 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:04.058 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:53:04.058 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:04.058 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.058 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.058 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:04.058 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:53:04.058 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.08e-07 2024-05-08T08:53:04.058 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.058 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.058 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:04.058 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485476+0000", 2024-05-08T08:53:04.059 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3929999999999998e-06 2024-05-08T08:53:04.059 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.059 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.059 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:04.059 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485485+0000", 2024-05-08T08:53:04.059 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1880000000000008e-06 2024-05-08T08:53:04.059 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.059 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.059 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:04.059 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485494+0000", 2024-05-08T08:53:04.059 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3109999999999995e-06 2024-05-08T08:53:04.059 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.059 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.059 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:04.060 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556082+0000", 2024-05-08T08:53:04.060 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070587426999999994 2024-05-08T08:53:04.060 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.060 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.060 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:04.060 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556177+0000", 2024-05-08T08:53:04.060 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5051999999999997e-05 2024-05-08T08:53:04.060 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.061 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.061 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:04.061 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556294+0000", 2024-05-08T08:53:04.061 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000117625 2024-05-08T08:53:04.061 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.061 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:04.061 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.061 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.061 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.062 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1075 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11108352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:04.062 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:53:04.062 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 27.551926334000001, 2024-05-08T08:53:04.062 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071045026999999997, 2024-05-08T08:53:04.062 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:04.062 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:04.062 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:04.062 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:04.062 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:04.062 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:04.062 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1075 2024-05-08T08:53:04.063 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.063 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:04.063 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.063 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:04.063 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:53:04.063 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:04.063 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.063 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.063 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:04.063 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:53:04.063 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:04.063 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.064 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.064 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:04.064 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:53:04.064 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.32e-07 2024-05-08T08:53:04.064 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.064 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.064 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:04.064 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485518+0000", 2024-05-08T08:53:04.064 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3769e-05 2024-05-08T08:53:04.064 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.065 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.065 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:04.065 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485527+0000", 2024-05-08T08:53:04.065 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6779999999999995e-06 2024-05-08T08:53:04.065 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.065 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.065 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:04.065 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485532+0000", 2024-05-08T08:53:04.065 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9830000000000004e-06 2024-05-08T08:53:04.065 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.065 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.066 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:04.066 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556340+0000", 2024-05-08T08:53:04.066 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070807743000000006 2024-05-08T08:53:04.066 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.066 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.066 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:04.066 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556435+0000", 2024-05-08T08:53:04.066 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4732e-05 2024-05-08T08:53:04.066 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.066 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.066 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:04.066 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556549+0000", 2024-05-08T08:53:04.066 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011369 2024-05-08T08:53:04.067 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.067 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:04.067 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.067 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.067 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.067 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1078 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11149312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:04.067 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:53:04.067 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 27.551886135, 2024-05-08T08:53:04.067 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071167276000000002, 2024-05-08T08:53:04.067 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:04.067 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:04.067 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:04.067 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:04.067 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:04.067 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:04.067 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1078 2024-05-08T08:53:04.068 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.068 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:04.068 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.068 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:04.068 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:53:04.068 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:04.068 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.068 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.068 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:04.068 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:53:04.068 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:04.068 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.068 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.069 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:04.069 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:53:04.069 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5600000000000001e-07 2024-05-08T08:53:04.069 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.069 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.069 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:04.069 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485547+0000", 2024-05-08T08:53:04.069 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3590000000000002e-06 2024-05-08T08:53:04.069 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.069 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.069 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:04.069 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485553+0000", 2024-05-08T08:53:04.069 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2770000000000002e-06 2024-05-08T08:53:04.069 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.069 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.069 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:04.069 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485558+0000", 2024-05-08T08:53:04.070 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.95e-06 2024-05-08T08:53:04.070 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.070 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.070 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:04.070 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556582+0000", 2024-05-08T08:53:04.070 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071024444000000006 2024-05-08T08:53:04.070 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.070 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.070 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:04.070 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556637+0000", 2024-05-08T08:53:04.070 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4672000000000001e-05 2024-05-08T08:53:04.070 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.070 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.070 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:04.071 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556711+0000", 2024-05-08T08:53:04.071 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4117999999999997e-05 2024-05-08T08:53:04.071 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.071 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:04.071 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.071 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.071 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.071 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1080 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11190272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:04.071 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:53:04.071 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 27.551865402000001, 2024-05-08T08:53:04.071 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071350721000000006, 2024-05-08T08:53:04.071 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:04.071 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:04.071 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:04.071 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:04.071 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:04.071 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:04.072 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1080 2024-05-08T08:53:04.072 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.072 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:04.072 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.072 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:04.072 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:53:04.072 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:04.072 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.072 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.072 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:04.072 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:53:04.072 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:04.072 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.072 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.073 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:04.073 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:53:04.073 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.03e-07 2024-05-08T08:53:04.073 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.073 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.073 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:04.073 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485567+0000", 2024-05-08T08:53:04.073 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1090000000000001e-06 2024-05-08T08:53:04.073 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.073 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.073 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:04.073 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485573+0000", 2024-05-08T08:53:04.073 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3049999999999998e-06 2024-05-08T08:53:04.073 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.074 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.074 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:04.074 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485578+0000", 2024-05-08T08:53:04.074 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0130000000000001e-06 2024-05-08T08:53:04.074 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.074 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.074 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:04.074 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556743+0000", 2024-05-08T08:53:04.074 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071164449000000005 2024-05-08T08:53:04.074 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.074 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.074 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:04.074 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556797+0000", 2024-05-08T08:53:04.075 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4589999999999997e-05 2024-05-08T08:53:04.075 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.075 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.075 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:04.075 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556915+0000", 2024-05-08T08:53:04.075 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000117852 2024-05-08T08:53:04.075 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.075 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:04.075 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.075 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.075 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.075 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1082 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11231232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:04.075 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:53:04.075 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 27.551845158999999, 2024-05-08T08:53:04.076 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071591396000000002, 2024-05-08T08:53:04.076 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:04.076 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:04.076 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:04.076 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:04.076 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:04.076 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:04.076 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1082 2024-05-08T08:53:04.076 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.076 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:04.076 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.076 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:04.077 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:53:04.077 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:04.077 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.077 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.077 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:04.077 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:53:04.077 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:04.077 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.077 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.077 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:04.077 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:53:04.078 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1800000000000001e-07 2024-05-08T08:53:04.078 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.078 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.078 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:04.078 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485587+0000", 2024-05-08T08:53:04.078 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2589999999999999e-06 2024-05-08T08:53:04.078 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.078 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.078 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:04.078 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485594+0000", 2024-05-08T08:53:04.078 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0789999999999997e-06 2024-05-08T08:53:04.078 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.079 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.079 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:04.079 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485598+0000", 2024-05-08T08:53:04.079 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7480000000000001e-06 2024-05-08T08:53:04.079 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.079 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.079 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:04.079 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556971+0000", 2024-05-08T08:53:04.079 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071372774999999999 2024-05-08T08:53:04.079 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.079 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.079 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:04.080 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557069+0000", 2024-05-08T08:53:04.080 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.7732999999999994e-05 2024-05-08T08:53:04.080 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.080 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.080 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:04.080 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557176+0000", 2024-05-08T08:53:04.080 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000107384 2024-05-08T08:53:04.080 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.080 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:04.080 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.081 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.081 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.081 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1084 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11272192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:04.081 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:53:04.081 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 27.55180652, 2024-05-08T08:53:04.081 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071808977999999996, 2024-05-08T08:53:04.081 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:04.081 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:04.081 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:04.081 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:04.082 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:04.082 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:04.082 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1084 2024-05-08T08:53:04.082 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.082 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:04.082 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.082 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:04.082 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:53:04.082 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:04.082 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.083 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.083 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:04.083 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:53:04.083 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:04.083 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.083 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.083 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:04.083 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485624+0000", 2024-05-08T08:53:04.083 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:53:04.083 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.083 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.084 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:04.084 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485626+0000", 2024-05-08T08:53:04.084 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9460000000000001e-06 2024-05-08T08:53:04.084 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.084 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.084 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:04.084 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485633+0000", 2024-05-08T08:53:04.084 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1149999999999996e-06 2024-05-08T08:53:04.084 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.084 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.085 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:04.085 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485638+0000", 2024-05-08T08:53:04.085 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5670000000000002e-06 2024-05-08T08:53:04.085 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.085 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.085 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:04.085 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557226+0000", 2024-05-08T08:53:04.085 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071587267999999996 2024-05-08T08:53:04.085 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.085 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.085 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:04.085 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557321+0000", 2024-05-08T08:53:04.086 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5785000000000005e-05 2024-05-08T08:53:04.086 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.086 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.086 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:04.086 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557432+0000", 2024-05-08T08:53:04.086 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011092800000000001 2024-05-08T08:53:04.086 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.086 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:04.086 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.086 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.086 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.087 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1086 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11313152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:04.087 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:53:04.087 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 27.551783192999999, 2024-05-08T08:53:04.087 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071933132999999996, 2024-05-08T08:53:04.087 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:04.087 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:04.087 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:04.087 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:04.087 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:04.087 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:04.088 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1086 2024-05-08T08:53:04.088 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.088 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:04.088 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.088 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:04.088 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:53:04.088 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:04.088 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.088 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.088 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:04.089 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:53:04.089 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:04.089 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.089 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.089 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:04.089 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486328+0000", 2024-05-08T08:53:04.089 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00068107900000000002 2024-05-08T08:53:04.089 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.089 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.089 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:04.089 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486331+0000", 2024-05-08T08:53:04.090 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.585e-06 2024-05-08T08:53:04.090 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.090 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.090 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:04.090 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486343+0000", 2024-05-08T08:53:04.090 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1856e-05 2024-05-08T08:53:04.090 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.090 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.090 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:04.090 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486350+0000", 2024-05-08T08:53:04.090 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8600000000000004e-06 2024-05-08T08:53:04.091 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.091 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.091 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:04.091 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557470+0000", 2024-05-08T08:53:04.091 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071120326999999997 2024-05-08T08:53:04.091 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.091 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.091 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:04.091 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557514+0000", 2024-05-08T08:53:04.091 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3909000000000003e-05 2024-05-08T08:53:04.091 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.092 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.092 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:04.092 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557580+0000", 2024-05-08T08:53:04.092 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5517000000000003e-05 2024-05-08T08:53:04.092 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.092 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:04.092 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.092 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.092 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.092 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1089 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11354112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:04.092 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:53:04.092 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 27.551070321000001, 2024-05-08T08:53:04.092 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071457381, 2024-05-08T08:53:04.093 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:04.093 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:04.093 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:04.093 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:04.093 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:04.093 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:04.093 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1089 2024-05-08T08:53:04.093 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.093 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:04.093 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.093 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:04.094 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:53:04.094 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:04.094 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.094 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.094 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:04.094 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:53:04.094 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:04.094 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.094 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.094 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:04.095 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486851+0000", 2024-05-08T08:53:04.095 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00049102 2024-05-08T08:53:04.095 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.095 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.095 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:04.095 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486854+0000", 2024-05-08T08:53:04.095 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6859999999999999e-06 2024-05-08T08:53:04.095 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.095 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.095 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:04.096 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486864+0000", 2024-05-08T08:53:04.096 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6080000000000002e-06 2024-05-08T08:53:04.096 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.096 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.096 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:04.096 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486870+0000", 2024-05-08T08:53:04.096 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.477e-06 2024-05-08T08:53:04.096 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.096 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.096 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:04.096 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557612+0000", 2024-05-08T08:53:04.097 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070741557999999996 2024-05-08T08:53:04.097 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.097 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.097 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:04.097 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557695+0000", 2024-05-08T08:53:04.097 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3300000000000005e-05 2024-05-08T08:53:04.097 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.097 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.097 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:04.097 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557817+0000", 2024-05-08T08:53:04.098 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012173200000000001 2024-05-08T08:53:04.098 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.098 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:04.098 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.098 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.098 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.098 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1091 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11395072~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:04.098 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:53:04.098 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 27.550549526000001, 2024-05-08T08:53:04.098 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071184737999999997, 2024-05-08T08:53:04.098 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:04.099 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:04.099 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:04.099 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:04.099 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:04.099 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:04.099 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1091 2024-05-08T08:53:04.099 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.099 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:04.099 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.099 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:04.099 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:53:04.100 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:04.100 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.100 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.100 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:04.100 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:53:04.100 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:04.100 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.100 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.100 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:04.100 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486881+0000", 2024-05-08T08:53:04.101 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0699999999999997e-07 2024-05-08T08:53:04.101 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.101 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.101 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:04.101 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486883+0000", 2024-05-08T08:53:04.101 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.402e-06 2024-05-08T08:53:04.101 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.101 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.101 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:04.101 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486890+0000", 2024-05-08T08:53:04.102 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0480000000000001e-06 2024-05-08T08:53:04.102 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.102 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.102 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:04.102 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486896+0000", 2024-05-08T08:53:04.102 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7640000000000002e-06 2024-05-08T08:53:04.102 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.102 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.102 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:04.102 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557860+0000", 2024-05-08T08:53:04.103 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070963791999999998 2024-05-08T08:53:04.103 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.103 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.103 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:04.103 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557952+0000", 2024-05-08T08:53:04.103 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2082000000000004e-05 2024-05-08T08:53:04.103 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.103 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.103 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:04.103 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558065+0000", 2024-05-08T08:53:04.103 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000113143 2024-05-08T08:53:04.104 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.104 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:04.104 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.104 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.104 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.104 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1094 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11436032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:04.104 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:53:04.104 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 27.550526357999999, 2024-05-08T08:53:04.104 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071400507000000002, 2024-05-08T08:53:04.104 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:04.104 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:04.105 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:04.105 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:04.105 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:04.105 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:04.105 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1094 2024-05-08T08:53:04.105 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.105 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:04.105 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.105 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:04.106 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:53:04.106 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:04.106 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.106 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.106 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:04.106 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:53:04.106 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:04.106 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.106 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.106 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:04.106 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487940+0000", 2024-05-08T08:53:04.107 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001036868 2024-05-08T08:53:04.107 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.107 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.107 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:04.107 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487944+0000", 2024-05-08T08:53:04.107 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7069999999999998e-06 2024-05-08T08:53:04.107 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.107 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.107 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:04.107 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487951+0000", 2024-05-08T08:53:04.107 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2649999999999999e-06 2024-05-08T08:53:04.108 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.108 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.108 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:04.108 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487961+0000", 2024-05-08T08:53:04.108 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4110000000000002e-06 2024-05-08T08:53:04.108 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.108 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.108 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:04.108 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558117+0000", 2024-05-08T08:53:04.108 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070156662999999994 2024-05-08T08:53:04.108 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.109 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.109 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:04.109 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558214+0000", 2024-05-08T08:53:04.109 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6444e-05 2024-05-08T08:53:04.109 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.109 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.109 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:04.109 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558304+0000", 2024-05-08T08:53:04.109 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0148999999999994e-05 2024-05-08T08:53:04.109 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.109 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:04.109 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.109 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.110 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.110 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1096 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11476992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:04.110 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:53:04.110 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 27.549459827, 2024-05-08T08:53:04.110 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070480074000000004, 2024-05-08T08:53:04.110 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:04.110 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:04.110 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:04.110 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:04.110 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:04.110 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:04.110 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1096 2024-05-08T08:53:04.111 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.111 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:04.111 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.111 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:04.111 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:53:04.111 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:04.111 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.111 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.111 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:04.111 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:53:04.111 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:04.112 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.112 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.112 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:04.112 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:53:04.112 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3700000000000001e-07 2024-05-08T08:53:04.112 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.112 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.112 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:04.112 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487973+0000", 2024-05-08T08:53:04.112 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0870000000000001e-06 2024-05-08T08:53:04.112 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.112 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.113 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:04.113 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487977+0000", 2024-05-08T08:53:04.113 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0600000000000001e-06 2024-05-08T08:53:04.113 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.113 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.113 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:04.113 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487980+0000", 2024-05-08T08:53:04.113 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1089999999999998e-06 2024-05-08T08:53:04.113 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.113 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.113 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:04.113 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558338+0000", 2024-05-08T08:53:04.114 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070358233000000006 2024-05-08T08:53:04.114 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.114 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.114 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:04.114 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558392+0000", 2024-05-08T08:53:04.114 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4537999999999999e-05 2024-05-08T08:53:04.114 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.114 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.114 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:04.114 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558450+0000", 2024-05-08T08:53:04.114 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7710000000000001e-05 2024-05-08T08:53:04.115 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.115 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:04.115 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.115 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.115 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.115 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1099 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11517952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:04.115 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:53:04.115 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 27.549445950999999, 2024-05-08T08:53:04.115 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070720506000000002, 2024-05-08T08:53:04.115 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:04.115 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:04.115 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:04.116 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:04.116 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:04.116 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:04.116 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1099 2024-05-08T08:53:04.116 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.116 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:04.116 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.116 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:04.116 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:53:04.116 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:04.116 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.116 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.117 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:04.117 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:53:04.117 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:04.117 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.117 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.117 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:04.117 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:53:04.117 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3999999999999998e-07 2024-05-08T08:53:04.117 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.117 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.117 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:04.117 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487986+0000", 2024-05-08T08:53:04.117 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3519999999999999e-06 2024-05-08T08:53:04.117 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.117 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.118 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:04.118 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487989+0000", 2024-05-08T08:53:04.118 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.23e-06 2024-05-08T08:53:04.118 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.118 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.118 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:04.118 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487991+0000", 2024-05-08T08:53:04.118 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6350000000000002e-06 2024-05-08T08:53:04.118 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.118 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.118 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:04.118 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558489+0000", 2024-05-08T08:53:04.118 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070497924000000003 2024-05-08T08:53:04.118 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.118 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.119 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:04.119 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558583+0000", 2024-05-08T08:53:04.119 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3405000000000004e-05 2024-05-08T08:53:04.119 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.119 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.119 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:04.119 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558704+0000", 2024-05-08T08:53:04.119 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012171999999999999 2024-05-08T08:53:04.119 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.119 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:04.119 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.119 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.119 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.119 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1100 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11558912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:04.119 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:53:04.119 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 27.549432952, 2024-05-08T08:53:04.120 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070962266999999996, 2024-05-08T08:53:04.120 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:04.120 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:04.120 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:04.120 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:04.120 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:04.120 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:04.120 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1100 2024-05-08T08:53:04.120 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.120 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:04.120 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.120 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:04.120 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:53:04.120 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:04.120 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.120 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.120 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:04.121 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:53:04.121 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:04.121 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.121 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.121 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:04.121 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488430+0000", 2024-05-08T08:53:04.121 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000432676 2024-05-08T08:53:04.121 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.121 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.121 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:04.121 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488433+0000", 2024-05-08T08:53:04.121 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.45e-06 2024-05-08T08:53:04.121 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.121 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.122 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:04.122 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488442+0000", 2024-05-08T08:53:04.122 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.687e-06 2024-05-08T08:53:04.122 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.122 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.122 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:04.122 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488446+0000", 2024-05-08T08:53:04.122 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4939999999999997e-06 2024-05-08T08:53:04.122 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.122 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.122 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:04.122 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558756+0000", 2024-05-08T08:53:04.122 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070310051999999998 2024-05-08T08:53:04.123 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.123 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.123 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:04.123 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558844+0000", 2024-05-08T08:53:04.123 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7690999999999998e-05 2024-05-08T08:53:04.123 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.123 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.123 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:04.123 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558959+0000", 2024-05-08T08:53:04.123 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000115217 2024-05-08T08:53:04.123 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.123 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:04.123 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.123 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.123 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.124 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1103 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11599872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:04.124 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:53:04.124 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 27.548975277, 2024-05-08T08:53:04.124 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070777100999999995, 2024-05-08T08:53:04.124 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:04.124 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:04.124 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:04.124 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:04.124 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:04.124 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:04.124 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1103 2024-05-08T08:53:04.124 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.124 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:04.124 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.124 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:04.124 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:53:04.125 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:04.125 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.125 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.125 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:04.125 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:53:04.125 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:04.125 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.125 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.125 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:04.125 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488959+0000", 2024-05-08T08:53:04.125 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00050457200000000001 2024-05-08T08:53:04.125 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.125 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.125 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:04.125 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488969+0000", 2024-05-08T08:53:04.125 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.7159999999999998e-06 2024-05-08T08:53:04.125 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.126 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.126 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:04.126 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488981+0000", 2024-05-08T08:53:04.126 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2150000000000001e-05 2024-05-08T08:53:04.126 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.126 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.126 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:04.126 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488988+0000", 2024-05-08T08:53:04.126 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8759999999999999e-06 2024-05-08T08:53:04.126 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.126 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.126 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:04.126 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558992+0000", 2024-05-08T08:53:04.127 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070004358000000003 2024-05-08T08:53:04.127 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.127 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.127 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:04.127 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559093+0000", 2024-05-08T08:53:04.127 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000100299 2024-05-08T08:53:04.127 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.127 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.127 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:04.127 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559232+0000", 2024-05-08T08:53:04.127 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013913 2024-05-08T08:53:04.127 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.127 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:04.127 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.127 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.127 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.127 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1104 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11640832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:04.128 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:53:04.128 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 27.548430461999999, 2024-05-08T08:53:04.128 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070521947000000001, 2024-05-08T08:53:04.128 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:04.128 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:04.128 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:04.128 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:04.128 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:04.128 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:04.128 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1104 2024-05-08T08:53:04.128 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.128 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:04.128 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.128 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:04.128 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:53:04.129 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:04.129 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.129 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.129 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:04.129 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:53:04.129 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:04.129 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.129 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.129 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:04.129 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489700+0000", 2024-05-08T08:53:04.129 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00070067600000000001 2024-05-08T08:53:04.129 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.129 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.129 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:04.130 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489704+0000", 2024-05-08T08:53:04.130 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1640000000000003e-06 2024-05-08T08:53:04.130 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.130 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.130 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:04.130 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489716+0000", 2024-05-08T08:53:04.130 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1379000000000001e-05 2024-05-08T08:53:04.130 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.130 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.130 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:04.130 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489724+0000", 2024-05-08T08:53:04.130 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8949999999999998e-06 2024-05-08T08:53:04.130 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.130 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.130 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:04.131 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559278+0000", 2024-05-08T08:53:04.131 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.069554151999999994 2024-05-08T08:53:04.131 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.131 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.131 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:04.131 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559387+0000", 2024-05-08T08:53:04.131 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000109258 2024-05-08T08:53:04.131 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.131 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.131 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:04.131 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559521+0000", 2024-05-08T08:53:04.131 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000134423 2024-05-08T08:53:04.131 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.131 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:04.131 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.132 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.132 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.132 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:8636 3.17 3:e8a0a3e4:::rbd_header.11402b26385:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-05-08T08:53:04.132 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:54.150971+0000", 2024-05-08T08:53:04.132 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 9.8864584680000007, 2024-05-08T08:53:04.132 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.217273233, 2024-05-08T08:53:04.132 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:04.132 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:04.132 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:04.132 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:04.133 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:04.133 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:04.133 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 8636 2024-05-08T08:53:04.133 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.133 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:04.133 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.133 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:04.133 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150971+0000", 2024-05-08T08:53:04.133 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:04.133 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.134 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.134 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:04.134 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150971+0000", 2024-05-08T08:53:04.134 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:04.134 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.134 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.134 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:04.134 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150972+0000", 2024-05-08T08:53:04.134 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.71e-07 2024-05-08T08:53:04.134 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.135 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.135 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:04.135 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150977+0000", 2024-05-08T08:53:04.135 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.152e-06 2024-05-08T08:53:04.135 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.135 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.135 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:04.135 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150990+0000", 2024-05-08T08:53:04.135 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2364999999999999e-05 2024-05-08T08:53:04.135 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.136 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.136 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:04.136 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150998+0000", 2024-05-08T08:53:04.136 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0739999999999993e-06 2024-05-08T08:53:04.136 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.136 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.136 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:04.136 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.151054+0000", 2024-05-08T08:53:04.136 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5949000000000003e-05 2024-05-08T08:53:04.136 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.136 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.136 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:53:04.137 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.151060+0000", 2024-05-08T08:53:04.137 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6120000000000004e-06 2024-05-08T08:53:04.137 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.137 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.137 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:04.137 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.367640+0000", 2024-05-08T08:53:04.137 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2165792739999999 2024-05-08T08:53:04.137 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.137 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.137 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:04.138 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.367839+0000", 2024-05-08T08:53:04.138 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000199185 2024-05-08T08:53:04.138 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.138 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.138 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:04.138 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.368245+0000", 2024-05-08T08:53:04.138 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00040575099999999998 2024-05-08T08:53:04.138 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.138 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:04.138 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.138 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.138 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:04.139 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:04.141 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:53:04.141 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:04.143+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:04.241 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:53:04.244+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:53:04.291 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:04.291 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:53:04.291 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:53:04.292 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:04.300 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:53:04.345 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:04.345+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:04.369 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-05-08T08:53:04.443 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:04.446+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:04.462 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:04.462 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:53:04.462 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:53:04.462 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:53:04.462 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:04.472 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:53:04.544 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:04.547+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:04.585 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:04.585 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:53:04.585 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:53:04.586 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:53:04.586 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.586 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9944 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 749568~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:04.586 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.103900+0000", 2024-05-08T08:53:04.586 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 37.484103392999998, 2024-05-08T08:53:04.586 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057007044, 2024-05-08T08:53:04.586 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:04.586 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:04.586 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:04.586 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:04.586 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:53:04.586 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:04.586 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9944 2024-05-08T08:53:04.586 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.587 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:04.587 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.587 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:04.587 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103900+0000", 2024-05-08T08:53:04.587 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:04.587 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.587 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.587 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:04.587 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103900+0000", 2024-05-08T08:53:04.587 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:04.587 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.587 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.587 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:04.587 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103900+0000", 2024-05-08T08:53:04.587 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6400000000000003e-07 2024-05-08T08:53:04.587 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.588 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.588 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:04.588 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103902+0000", 2024-05-08T08:53:04.588 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.858e-06 2024-05-08T08:53:04.588 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.588 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.588 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:04.588 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103907+0000", 2024-05-08T08:53:04.588 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3410000000000002e-06 2024-05-08T08:53:04.588 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.588 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.588 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:04.588 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103910+0000", 2024-05-08T08:53:04.588 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8540000000000001e-06 2024-05-08T08:53:04.589 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.589 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.589 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:04.589 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.160682+0000", 2024-05-08T08:53:04.589 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.056771822 2024-05-08T08:53:04.589 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.589 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.589 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:04.589 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.160777+0000", 2024-05-08T08:53:04.589 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4555999999999994e-05 2024-05-08T08:53:04.589 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.589 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.589 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:04.589 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.160907+0000", 2024-05-08T08:53:04.589 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000130149 2024-05-08T08:53:04.589 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.590 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:04.590 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.590 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.590 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.590 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9945 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 1441792~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:04.590 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.103914+0000", 2024-05-08T08:53:04.590 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 37.484088800000002, 2024-05-08T08:53:04.590 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057211979000000003, 2024-05-08T08:53:04.590 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:04.590 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:04.590 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:04.590 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:04.590 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:53:04.590 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:04.591 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9945 2024-05-08T08:53:04.591 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.591 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:04.591 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.591 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:04.591 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103914+0000", 2024-05-08T08:53:04.591 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:04.591 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.591 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.591 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:04.591 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103914+0000", 2024-05-08T08:53:04.591 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:04.591 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.591 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.591 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:04.591 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103915+0000", 2024-05-08T08:53:04.592 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3099999999999999e-07 2024-05-08T08:53:04.592 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.592 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.592 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:04.592 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103916+0000", 2024-05-08T08:53:04.592 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.203e-06 2024-05-08T08:53:04.592 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.592 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.592 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:04.592 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103919+0000", 2024-05-08T08:53:04.592 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.117e-06 2024-05-08T08:53:04.592 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.592 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.592 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:04.592 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103921+0000", 2024-05-08T08:53:04.593 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4169999999999999e-06 2024-05-08T08:53:04.593 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.593 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.593 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:04.593 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.160959+0000", 2024-05-08T08:53:04.593 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057038074000000001 2024-05-08T08:53:04.593 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.593 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.593 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:04.593 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161041+0000", 2024-05-08T08:53:04.593 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1620999999999997e-05 2024-05-08T08:53:04.593 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.593 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.593 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:04.593 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161126+0000", 2024-05-08T08:53:04.594 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.5216000000000005e-05 2024-05-08T08:53:04.594 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.594 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:04.594 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.594 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.594 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.594 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9946 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 765952~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:04.594 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.103925+0000", 2024-05-08T08:53:04.594 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 37.484078433000001, 2024-05-08T08:53:04.594 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057418770000000001, 2024-05-08T08:53:04.594 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:04.594 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:04.594 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:04.594 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:04.594 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:53:04.595 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:04.595 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9946 2024-05-08T08:53:04.595 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.595 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:04.595 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.595 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:04.595 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103925+0000", 2024-05-08T08:53:04.595 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:04.595 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.595 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.595 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:04.595 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103925+0000", 2024-05-08T08:53:04.595 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:04.595 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.595 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.596 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:04.596 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103925+0000", 2024-05-08T08:53:04.596 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9200000000000002e-07 2024-05-08T08:53:04.596 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.596 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.596 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:04.596 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103926+0000", 2024-05-08T08:53:04.596 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1620000000000001e-06 2024-05-08T08:53:04.596 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.596 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.596 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:04.596 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103929+0000", 2024-05-08T08:53:04.596 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9349999999999999e-06 2024-05-08T08:53:04.596 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.596 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.597 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:04.597 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103931+0000", 2024-05-08T08:53:04.597 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4099999999999998e-06 2024-05-08T08:53:04.597 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.597 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.597 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:04.597 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161161+0000", 2024-05-08T08:53:04.597 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057229176999999999 2024-05-08T08:53:04.597 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.597 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.597 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:04.597 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161235+0000", 2024-05-08T08:53:04.597 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4227000000000006e-05 2024-05-08T08:53:04.597 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.597 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.598 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:04.598 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161343+0000", 2024-05-08T08:53:04.598 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010856700000000001 2024-05-08T08:53:04.598 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.598 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:04.598 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.598 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.598 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.598 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9947 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 1458176~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:04.598 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.103935+0000", 2024-05-08T08:53:04.598 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 37.484068555, 2024-05-08T08:53:04.598 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057541263000000002, 2024-05-08T08:53:04.598 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:04.598 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:04.599 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:04.599 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:04.599 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:53:04.599 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:04.599 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9947 2024-05-08T08:53:04.599 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.599 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:04.599 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.599 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:04.599 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103935+0000", 2024-05-08T08:53:04.599 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:04.599 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.599 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.599 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:04.599 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103935+0000", 2024-05-08T08:53:04.599 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:04.600 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.600 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.600 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:04.600 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103935+0000", 2024-05-08T08:53:04.600 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.11e-07 2024-05-08T08:53:04.600 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.600 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.600 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:04.600 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103940+0000", 2024-05-08T08:53:04.600 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1030000000000001e-06 2024-05-08T08:53:04.600 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.600 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.600 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:04.600 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103943+0000", 2024-05-08T08:53:04.600 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3869999999999999e-06 2024-05-08T08:53:04.601 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.601 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.601 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:04.601 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103951+0000", 2024-05-08T08:53:04.601 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.2150000000000001e-06 2024-05-08T08:53:04.601 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.601 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.601 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:04.601 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161377+0000", 2024-05-08T08:53:04.601 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057425925000000003 2024-05-08T08:53:04.601 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.601 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.601 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:04.601 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161431+0000", 2024-05-08T08:53:04.601 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3408000000000001e-05 2024-05-08T08:53:04.601 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.602 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.602 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:04.602 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161476+0000", 2024-05-08T08:53:04.602 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5014e-05 2024-05-08T08:53:04.602 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.602 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:04.602 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.602 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.602 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.602 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9948 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 782336~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:04.602 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.103965+0000", 2024-05-08T08:53:04.602 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 37.484037665000002, 2024-05-08T08:53:04.602 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057746205000000002, 2024-05-08T08:53:04.602 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:04.603 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:04.603 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:04.603 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:04.603 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:53:04.603 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:04.603 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9948 2024-05-08T08:53:04.603 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.603 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:04.603 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.603 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:04.603 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103965+0000", 2024-05-08T08:53:04.603 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:04.603 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.603 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.603 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:04.604 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103965+0000", 2024-05-08T08:53:04.604 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:04.604 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.604 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.604 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:04.604 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103966+0000", 2024-05-08T08:53:04.604 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.44e-07 2024-05-08T08:53:04.604 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.604 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.604 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:04.604 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103968+0000", 2024-05-08T08:53:04.604 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4090000000000001e-06 2024-05-08T08:53:04.604 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.604 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.604 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:04.604 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103973+0000", 2024-05-08T08:53:04.605 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6820000000000002e-06 2024-05-08T08:53:04.605 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.605 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.605 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:04.605 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103976+0000", 2024-05-08T08:53:04.605 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.959e-06 2024-05-08T08:53:04.605 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.605 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.605 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:04.605 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161523+0000", 2024-05-08T08:53:04.605 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057547264000000001 2024-05-08T08:53:04.605 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.605 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.605 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:04.605 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161605+0000", 2024-05-08T08:53:04.606 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1854000000000006e-05 2024-05-08T08:53:04.606 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.606 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.606 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:04.606 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161712+0000", 2024-05-08T08:53:04.606 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000106493 2024-05-08T08:53:04.606 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.606 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:04.606 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.606 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.606 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.606 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9949 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 1474560~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:04.606 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.103980+0000", 2024-05-08T08:53:04.606 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 37.484023045999997, 2024-05-08T08:53:04.606 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057852397, 2024-05-08T08:53:04.607 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:04.607 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:04.607 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:04.607 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:04.607 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:53:04.607 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:04.607 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9949 2024-05-08T08:53:04.607 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.607 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:04.607 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.607 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:04.607 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103980+0000", 2024-05-08T08:53:04.607 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:04.608 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.608 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.608 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:04.608 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103980+0000", 2024-05-08T08:53:04.608 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:04.608 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.608 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.608 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:04.608 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103980+0000", 2024-05-08T08:53:04.608 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.11e-07 2024-05-08T08:53:04.608 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.608 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.609 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:04.609 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103982+0000", 2024-05-08T08:53:04.609 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.269e-06 2024-05-08T08:53:04.609 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.609 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.609 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:04.609 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103985+0000", 2024-05-08T08:53:04.609 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0110000000000001e-06 2024-05-08T08:53:04.609 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.609 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.609 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:04.609 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103987+0000", 2024-05-08T08:53:04.609 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3020000000000002e-06 2024-05-08T08:53:04.609 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.609 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.610 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:04.610 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161744+0000", 2024-05-08T08:53:04.610 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057757115999999997 2024-05-08T08:53:04.610 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.610 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.610 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:04.610 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161791+0000", 2024-05-08T08:53:04.610 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6807000000000001e-05 2024-05-08T08:53:04.610 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.610 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.611 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:04.611 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161832+0000", 2024-05-08T08:53:04.611 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1681000000000003e-05 2024-05-08T08:53:04.611 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.611 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:04.611 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.611 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.611 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.611 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9950 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 798720~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:04.611 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.103990+0000", 2024-05-08T08:53:04.611 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 37.484012810999999, 2024-05-08T08:53:04.611 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.058062890999999998, 2024-05-08T08:53:04.612 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:04.612 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:04.612 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:04.612 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:04.612 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:53:04.612 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:04.612 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9950 2024-05-08T08:53:04.612 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.612 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:04.612 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.612 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:04.612 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103990+0000", 2024-05-08T08:53:04.612 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:04.612 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.612 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.613 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:04.613 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103990+0000", 2024-05-08T08:53:04.613 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:04.613 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.613 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.613 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:04.613 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103991+0000", 2024-05-08T08:53:04.613 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1e-07 2024-05-08T08:53:04.613 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.613 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.613 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:04.613 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103992+0000", 2024-05-08T08:53:04.614 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.074e-06 2024-05-08T08:53:04.614 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.614 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.614 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:04.614 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103995+0000", 2024-05-08T08:53:04.614 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9529999999999999e-06 2024-05-08T08:53:04.614 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.614 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.614 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:04.614 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.103997+0000", 2024-05-08T08:53:04.614 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3939999999999999e-06 2024-05-08T08:53:04.614 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.615 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.615 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:04.615 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161871+0000", 2024-05-08T08:53:04.615 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057873854000000002 2024-05-08T08:53:04.615 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.615 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.615 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:04.615 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.161950+0000", 2024-05-08T08:53:04.615 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9433999999999999e-05 2024-05-08T08:53:04.615 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.615 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.615 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:04.616 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.162053+0000", 2024-05-08T08:53:04.616 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000102972 2024-05-08T08:53:04.616 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.616 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:04.616 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.616 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.616 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.616 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9951 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 1490944~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:04.616 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.104000+0000", 2024-05-08T08:53:04.616 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 37.484003006999998, 2024-05-08T08:53:04.616 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.058313416999999999, 2024-05-08T08:53:04.616 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:04.617 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:04.617 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:04.617 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:04.617 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:53:04.617 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:04.617 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9951 2024-05-08T08:53:04.617 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.617 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:04.617 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.617 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:04.617 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.104000+0000", 2024-05-08T08:53:04.617 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:04.617 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.618 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.618 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:04.618 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.104000+0000", 2024-05-08T08:53:04.618 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:04.618 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.618 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.618 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:04.618 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.104962+0000", 2024-05-08T08:53:04.618 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00096230399999999998 2024-05-08T08:53:04.618 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.618 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.618 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:04.619 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.104978+0000", 2024-05-08T08:53:04.619 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5126999999999999e-05 2024-05-08T08:53:04.619 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.619 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.619 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:04.619 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.104987+0000", 2024-05-08T08:53:04.619 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8940000000000005e-06 2024-05-08T08:53:04.619 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.619 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.619 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:04.619 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.104994+0000", 2024-05-08T08:53:04.620 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5610000000000004e-06 2024-05-08T08:53:04.620 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.620 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.620 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:04.620 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.162114+0000", 2024-05-08T08:53:04.620 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057119717 2024-05-08T08:53:04.620 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.620 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.620 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:04.620 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.162214+0000", 2024-05-08T08:53:04.620 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.9982999999999995e-05 2024-05-08T08:53:04.620 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.621 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.621 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:04.621 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.162314+0000", 2024-05-08T08:53:04.621 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.9831e-05 2024-05-08T08:53:04.621 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.621 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:04.621 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.621 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.621 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.621 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9952 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 815104~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:04.621 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.105004+0000", 2024-05-08T08:53:04.621 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 37.482998754, 2024-05-08T08:53:04.622 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057530855999999998, 2024-05-08T08:53:04.622 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:04.622 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:04.622 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:04.622 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:04.622 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:53:04.622 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:04.622 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9952 2024-05-08T08:53:04.622 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.622 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:04.622 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.622 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:04.622 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105004+0000", 2024-05-08T08:53:04.623 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:04.623 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.623 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.623 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:04.623 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105004+0000", 2024-05-08T08:53:04.623 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:04.623 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.623 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.623 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:04.623 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105005+0000", 2024-05-08T08:53:04.623 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9299999999999998e-07 2024-05-08T08:53:04.623 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.623 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.624 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:04.624 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105007+0000", 2024-05-08T08:53:04.624 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2979999999999999e-06 2024-05-08T08:53:04.624 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.624 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.624 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:04.624 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105017+0000", 2024-05-08T08:53:04.624 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5060000000000001e-06 2024-05-08T08:53:04.624 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.624 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.624 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:04.624 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105022+0000", 2024-05-08T08:53:04.624 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2619999999999998e-06 2024-05-08T08:53:04.624 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.624 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.625 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:04.625 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.162356+0000", 2024-05-08T08:53:04.625 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057334388999999999 2024-05-08T08:53:04.625 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.625 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.625 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:04.625 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.162437+0000", 2024-05-08T08:53:04.625 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0863000000000002e-05 2024-05-08T08:53:04.625 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.625 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.625 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:04.625 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.162535+0000", 2024-05-08T08:53:04.625 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8045000000000001e-05 2024-05-08T08:53:04.625 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.626 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:04.626 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.626 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.626 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.626 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9953 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 1507328~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:04.626 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.105064+0000", 2024-05-08T08:53:04.626 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 37.482939066999997, 2024-05-08T08:53:04.626 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057721531999999999, 2024-05-08T08:53:04.626 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:04.626 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:04.626 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:04.626 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:04.626 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:53:04.626 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:04.626 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9953 2024-05-08T08:53:04.627 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.627 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:04.627 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.627 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:04.627 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105064+0000", 2024-05-08T08:53:04.627 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:04.627 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.627 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.627 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:04.627 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105064+0000", 2024-05-08T08:53:04.627 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:04.627 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.628 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.628 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:04.628 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105065+0000", 2024-05-08T08:53:04.628 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8800000000000002e-07 2024-05-08T08:53:04.628 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.628 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.628 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:04.628 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105067+0000", 2024-05-08T08:53:04.628 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7669999999999999e-06 2024-05-08T08:53:04.628 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.628 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.628 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:04.628 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105076+0000", 2024-05-08T08:53:04.629 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.5259999999999993e-06 2024-05-08T08:53:04.629 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.629 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.629 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:04.629 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105080+0000", 2024-05-08T08:53:04.629 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8e-06 2024-05-08T08:53:04.629 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.629 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.630 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:04.630 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.162596+0000", 2024-05-08T08:53:04.630 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057515822000000001 2024-05-08T08:53:04.630 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.630 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.630 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:04.630 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.162689+0000", 2024-05-08T08:53:04.631 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3258999999999996e-05 2024-05-08T08:53:04.631 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.631 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.631 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:04.631 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.162786+0000", 2024-05-08T08:53:04.631 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6769999999999997e-05 2024-05-08T08:53:04.631 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.631 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:04.631 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.631 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.631 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.631 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9954 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 1523712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:04.632 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.105088+0000", 2024-05-08T08:53:04.632 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 37.482915058000003, 2024-05-08T08:53:04.632 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057925139, 2024-05-08T08:53:04.632 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:04.632 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:04.632 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:04.632 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:04.632 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:53:04.632 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:04.632 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9954 2024-05-08T08:53:04.632 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.632 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:04.633 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.633 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:04.633 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105088+0000", 2024-05-08T08:53:04.633 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:04.633 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.633 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.633 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:04.633 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105088+0000", 2024-05-08T08:53:04.633 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:04.633 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.633 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.633 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:04.633 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105089+0000", 2024-05-08T08:53:04.634 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9599999999999999e-07 2024-05-08T08:53:04.634 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.634 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.634 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:04.634 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105091+0000", 2024-05-08T08:53:04.634 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3649999999999998e-06 2024-05-08T08:53:04.634 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.634 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.634 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:04.634 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105107+0000", 2024-05-08T08:53:04.634 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6370000000000001e-05 2024-05-08T08:53:04.634 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.635 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.635 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:04.635 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105114+0000", 2024-05-08T08:53:04.635 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2190000000000001e-06 2024-05-08T08:53:04.635 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.635 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.635 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:04.635 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.162830+0000", 2024-05-08T08:53:04.635 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057716637000000001 2024-05-08T08:53:04.635 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.635 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.635 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:04.636 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.162924+0000", 2024-05-08T08:53:04.636 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3354000000000001e-05 2024-05-08T08:53:04.636 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.636 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.636 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:04.636 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.163013+0000", 2024-05-08T08:53:04.636 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.9697999999999994e-05 2024-05-08T08:53:04.636 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.636 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:04.636 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.636 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.636 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.636 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9955 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 831488~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:04.637 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.105124+0000", 2024-05-08T08:53:04.637 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 37.482879484000001, 2024-05-08T08:53:04.637 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.058092618999999998, 2024-05-08T08:53:04.637 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:04.637 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:04.637 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:04.637 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:04.637 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:53:04.637 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:04.637 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9955 2024-05-08T08:53:04.637 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.637 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:04.637 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.637 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:04.638 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105124+0000", 2024-05-08T08:53:04.638 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:04.638 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.638 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.638 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:04.638 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.105124+0000", 2024-05-08T08:53:04.638 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:04.638 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.638 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.638 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:04.638 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.106961+0000", 2024-05-08T08:53:04.638 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001837574 2024-05-08T08:53:04.638 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.638 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.638 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:04.639 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.106966+0000", 2024-05-08T08:53:04.639 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8439999999999997e-06 2024-05-08T08:53:04.639 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.639 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.639 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:04.639 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.106978+0000", 2024-05-08T08:53:04.639 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2330999999999999e-05 2024-05-08T08:53:04.639 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.639 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.639 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:04.639 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.106986+0000", 2024-05-08T08:53:04.639 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0439999999999996e-06 2024-05-08T08:53:04.639 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.639 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.639 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:04.640 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.163048+0000", 2024-05-08T08:53:04.640 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.056061182000000001 2024-05-08T08:53:04.640 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.640 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.640 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:04.640 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.163124+0000", 2024-05-08T08:53:04.640 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.6030000000000002e-05 2024-05-08T08:53:04.640 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.640 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.640 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:04.640 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.163216+0000", 2024-05-08T08:53:04.640 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2614000000000005e-05 2024-05-08T08:53:04.640 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.640 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:04.641 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.641 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.641 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.641 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9960 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 1589248~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:04.641 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.107116+0000", 2024-05-08T08:53:04.641 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 37.480886666000004, 2024-05-08T08:53:04.641 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.056889636, 2024-05-08T08:53:04.641 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:04.641 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:04.641 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:04.641 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:04.641 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:53:04.641 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:04.641 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9960 2024-05-08T08:53:04.641 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.642 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:04.642 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.642 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:04.642 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107116+0000", 2024-05-08T08:53:04.642 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:04.642 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.642 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.642 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:04.642 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107116+0000", 2024-05-08T08:53:04.642 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:04.642 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.642 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.642 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:04.642 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107117+0000", 2024-05-08T08:53:04.642 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4400000000000001e-07 2024-05-08T08:53:04.643 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.643 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.643 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:04.643 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107119+0000", 2024-05-08T08:53:04.643 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3750000000000001e-06 2024-05-08T08:53:04.643 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.643 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.643 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:04.643 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107126+0000", 2024-05-08T08:53:04.643 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3369999999999996e-06 2024-05-08T08:53:04.643 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.643 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.643 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:04.643 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107130+0000", 2024-05-08T08:53:04.644 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8620000000000002e-06 2024-05-08T08:53:04.644 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.644 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.644 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:04.644 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.163917+0000", 2024-05-08T08:53:04.644 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.056786634000000002 2024-05-08T08:53:04.644 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.644 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.644 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:04.644 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.163962+0000", 2024-05-08T08:53:04.644 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4521000000000003e-05 2024-05-08T08:53:04.644 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.644 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.644 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:04.644 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.164006+0000", 2024-05-08T08:53:04.645 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4462999999999998e-05 2024-05-08T08:53:04.645 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.645 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:04.645 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.645 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.645 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.645 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9961 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 864256~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:04.645 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.107138+0000", 2024-05-08T08:53:04.645 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 37.480865606999998, 2024-05-08T08:53:04.645 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057108051999999999, 2024-05-08T08:53:04.645 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:04.645 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:04.645 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:04.645 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:04.645 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:53:04.646 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:04.646 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9961 2024-05-08T08:53:04.646 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.646 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:04.646 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.646 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:04.646 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107138+0000", 2024-05-08T08:53:04.646 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:04.646 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.646 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.646 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:04.646 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107138+0000", 2024-05-08T08:53:04.646 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:04.646 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.647 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.647 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:04.647 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107138+0000", 2024-05-08T08:53:04.647 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2399999999999999e-07 2024-05-08T08:53:04.647 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.647 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.647 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:04.647 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107140+0000", 2024-05-08T08:53:04.647 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1409999999999999e-06 2024-05-08T08:53:04.647 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.647 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.647 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:04.647 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107146+0000", 2024-05-08T08:53:04.648 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0639999999999999e-06 2024-05-08T08:53:04.648 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.648 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.648 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:04.648 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107151+0000", 2024-05-08T08:53:04.648 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.69e-06 2024-05-08T08:53:04.648 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.648 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.648 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:04.648 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.164051+0000", 2024-05-08T08:53:04.648 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.056900153000000002 2024-05-08T08:53:04.648 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.648 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.648 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:04.649 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.164136+0000", 2024-05-08T08:53:04.649 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.5086999999999994e-05 2024-05-08T08:53:04.649 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.649 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.649 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:04.649 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.164246+0000", 2024-05-08T08:53:04.649 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000109493 2024-05-08T08:53:04.649 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.649 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:04.649 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.649 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.649 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.649 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9962 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 1605632~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:04.650 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.107158+0000", 2024-05-08T08:53:04.650 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 37.480845528000003, 2024-05-08T08:53:04.650 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057292698000000003, 2024-05-08T08:53:04.650 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:04.650 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:04.650 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:04.650 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:04.650 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:53:04.650 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:04.650 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9962 2024-05-08T08:53:04.650 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.650 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:04.650 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.650 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:04.651 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107158+0000", 2024-05-08T08:53:04.651 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:04.651 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.651 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.651 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:04.651 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107158+0000", 2024-05-08T08:53:04.651 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:04.651 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.651 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.651 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:04.651 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107158+0000", 2024-05-08T08:53:04.651 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8100000000000003e-07 2024-05-08T08:53:04.651 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.652 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.652 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:04.652 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107167+0000", 2024-05-08T08:53:04.652 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3079999999999997e-06 2024-05-08T08:53:04.652 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.652 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.652 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:04.652 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107175+0000", 2024-05-08T08:53:04.652 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2089999999999999e-06 2024-05-08T08:53:04.652 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.652 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.652 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:04.652 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107180+0000", 2024-05-08T08:53:04.652 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1120000000000004e-06 2024-05-08T08:53:04.652 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.653 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.653 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:04.653 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.164284+0000", 2024-05-08T08:53:04.653 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057104110999999999 2024-05-08T08:53:04.653 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.653 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.653 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:04.653 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.164370+0000", 2024-05-08T08:53:04.653 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.6549000000000001e-05 2024-05-08T08:53:04.653 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.653 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.653 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:04.653 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.164450+0000", 2024-05-08T08:53:04.653 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9827999999999995e-05 2024-05-08T08:53:04.654 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.654 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:04.654 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.654 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.654 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.654 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9963 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 880640~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:04.654 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.107188+0000", 2024-05-08T08:53:04.654 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 37.480815610000001, 2024-05-08T08:53:04.654 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057560471000000002, 2024-05-08T08:53:04.654 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:04.654 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:04.654 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:04.654 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:04.654 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:53:04.654 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:04.655 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9963 2024-05-08T08:53:04.655 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.655 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:04.655 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.655 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:04.655 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107188+0000", 2024-05-08T08:53:04.655 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:04.655 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.655 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.655 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:04.655 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.107188+0000", 2024-05-08T08:53:04.655 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:04.655 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.655 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.656 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:04.656 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.108964+0000", 2024-05-08T08:53:04.656 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001775993 2024-05-08T08:53:04.656 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.656 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.656 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:04.656 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.108969+0000", 2024-05-08T08:53:04.656 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1200000000000001e-06 2024-05-08T08:53:04.656 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.656 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.656 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:04.656 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.108981+0000", 2024-05-08T08:53:04.656 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2670000000000001e-05 2024-05-08T08:53:04.657 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.657 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.657 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:04.657 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.108989+0000", 2024-05-08T08:53:04.657 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3139999999999998e-06 2024-05-08T08:53:04.657 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.657 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.657 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:04.657 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.164504+0000", 2024-05-08T08:53:04.657 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.055515225000000001 2024-05-08T08:53:04.657 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.657 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.657 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:04.658 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.164606+0000", 2024-05-08T08:53:04.658 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010245 2024-05-08T08:53:04.658 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.658 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.658 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:04.658 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.164748+0000", 2024-05-08T08:53:04.658 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014169900000000001 2024-05-08T08:53:04.658 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.658 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:04.658 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.658 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.658 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.659 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9970 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 929792~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:04.659 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.109178+0000", 2024-05-08T08:53:04.659 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 37.478825332, 2024-05-08T08:53:04.659 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057137975000000001, 2024-05-08T08:53:04.659 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:04.659 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:04.659 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:04.659 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:04.659 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:53:04.659 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:04.659 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9970 2024-05-08T08:53:04.659 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.659 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:04.660 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.660 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:04.660 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109178+0000", 2024-05-08T08:53:04.660 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:04.660 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.660 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.660 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:04.660 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109178+0000", 2024-05-08T08:53:04.660 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:04.660 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.660 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.660 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:04.661 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109178+0000", 2024-05-08T08:53:04.661 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7800000000000002e-07 2024-05-08T08:53:04.661 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.661 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.661 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:04.661 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109180+0000", 2024-05-08T08:53:04.661 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2199999999999999e-06 2024-05-08T08:53:04.661 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.661 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.661 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:04.661 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109186+0000", 2024-05-08T08:53:04.661 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9410000000000004e-06 2024-05-08T08:53:04.661 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.661 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.662 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:04.662 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109191+0000", 2024-05-08T08:53:04.662 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8729999999999998e-06 2024-05-08T08:53:04.662 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.662 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.662 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:04.662 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.166098+0000", 2024-05-08T08:53:04.662 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.056907124000000003 2024-05-08T08:53:04.662 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.662 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.662 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:04.662 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.166193+0000", 2024-05-08T08:53:04.663 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5018000000000004e-05 2024-05-08T08:53:04.663 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.663 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.663 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:04.663 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.166316+0000", 2024-05-08T08:53:04.663 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000122321 2024-05-08T08:53:04.663 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.663 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:04.663 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.663 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.663 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.663 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9971 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 1687552~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:04.663 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.109198+0000", 2024-05-08T08:53:04.664 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 37.478804654000001, 2024-05-08T08:53:04.664 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057361875, 2024-05-08T08:53:04.664 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:04.664 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:04.664 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:04.664 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:04.664 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:53:04.664 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:04.664 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9971 2024-05-08T08:53:04.664 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.664 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:04.664 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.665 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:04.665 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109198+0000", 2024-05-08T08:53:04.665 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:04.665 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.665 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.665 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:04.665 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109198+0000", 2024-05-08T08:53:04.665 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:04.665 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.665 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.665 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:04.665 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109199+0000", 2024-05-08T08:53:04.665 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2800000000000002e-07 2024-05-08T08:53:04.665 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.666 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.666 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:04.666 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109208+0000", 2024-05-08T08:53:04.666 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4210000000000001e-06 2024-05-08T08:53:04.666 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.666 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.666 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:04.666 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109215+0000", 2024-05-08T08:53:04.666 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7209999999999997e-06 2024-05-08T08:53:04.666 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.666 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.666 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:04.666 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109220+0000", 2024-05-08T08:53:04.666 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.13e-06 2024-05-08T08:53:04.666 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.667 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.667 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:04.667 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.166353+0000", 2024-05-08T08:53:04.667 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057132693999999998 2024-05-08T08:53:04.667 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.667 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.667 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:04.667 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.166462+0000", 2024-05-08T08:53:04.667 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010940799999999999 2024-05-08T08:53:04.667 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.667 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.667 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:04.667 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.166560+0000", 2024-05-08T08:53:04.667 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8072999999999994e-05 2024-05-08T08:53:04.667 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.668 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:04.668 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.668 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.668 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.668 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9972 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 1703936~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:04.668 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.109228+0000", 2024-05-08T08:53:04.668 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 37.478775261999999, 2024-05-08T08:53:04.668 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057541904999999997, 2024-05-08T08:53:04.668 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:04.668 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:04.668 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:04.668 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:04.668 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:53:04.668 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:04.668 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9972 2024-05-08T08:53:04.669 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.669 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:04.669 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.669 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:04.669 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109228+0000", 2024-05-08T08:53:04.669 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:04.669 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.669 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.669 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:04.669 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109228+0000", 2024-05-08T08:53:04.669 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:04.669 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.669 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.669 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:04.669 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109228+0000", 2024-05-08T08:53:04.669 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5699999999999998e-07 2024-05-08T08:53:04.670 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.670 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.670 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:04.670 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109231+0000", 2024-05-08T08:53:04.670 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2120000000000002e-06 2024-05-08T08:53:04.670 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.670 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.670 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:04.670 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109237+0000", 2024-05-08T08:53:04.670 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3099999999999997e-06 2024-05-08T08:53:04.670 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.670 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.670 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:04.670 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109243+0000", 2024-05-08T08:53:04.670 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7490000000000004e-06 2024-05-08T08:53:04.671 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.671 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.671 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:04.671 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.166609+0000", 2024-05-08T08:53:04.671 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057366372999999998 2024-05-08T08:53:04.671 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.671 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.671 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:04.671 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.166690+0000", 2024-05-08T08:53:04.671 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1426000000000003e-05 2024-05-08T08:53:04.671 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.671 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.671 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:04.671 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.166770+0000", 2024-05-08T08:53:04.671 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9377999999999998e-05 2024-05-08T08:53:04.672 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.672 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:04.672 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.672 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.672 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.672 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4576.0:9973 3.4 3:23845394:::rbd_data.1140782590a5.0000000000000002:6 [read 946176~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:04.672 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:27.109249+0000", 2024-05-08T08:53:04.672 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 37.478753619999999, 2024-05-08T08:53:04.672 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057760381, 2024-05-08T08:53:04.672 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:04.672 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:04.672 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:04.672 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:04.672 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4576", 2024-05-08T08:53:04.672 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:04.672 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9973 2024-05-08T08:53:04.673 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.673 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:04.673 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.673 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:04.673 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109249+0000", 2024-05-08T08:53:04.673 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:04.673 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.673 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.673 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:04.673 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.109249+0000", 2024-05-08T08:53:04.673 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:04.673 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.673 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.673 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:04.673 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.110959+0000", 2024-05-08T08:53:04.674 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001709288 2024-05-08T08:53:04.674 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.674 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.674 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:04.674 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.110972+0000", 2024-05-08T08:53:04.674 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2719e-05 2024-05-08T08:53:04.674 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.674 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.674 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:04.674 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.110984+0000", 2024-05-08T08:53:04.674 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2001000000000001e-05 2024-05-08T08:53:04.674 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.674 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.674 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:04.674 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.110993+0000", 2024-05-08T08:53:04.675 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0880000000000001e-06 2024-05-08T08:53:04.675 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.675 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.675 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:04.675 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.166817+0000", 2024-05-08T08:53:04.675 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.055823971999999999 2024-05-08T08:53:04.675 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.675 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.675 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:04.675 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.166900+0000", 2024-05-08T08:53:04.675 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3838999999999995e-05 2024-05-08T08:53:04.675 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:04.675 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:04.676 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:04.676 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:27.167010+0000", 2024-05-08T08:53:04.676 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000109474 2024-05-08T08:53:04.676 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.676 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:04.676 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.676 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:04.676 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:04.676 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:04.678 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:53:04.678 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:04.680+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:04.778 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:04.781+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:04.833 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:04.833 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:53:04.833 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:53:04.833 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:04.843 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:53:04.878 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:04.881+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:04.899 INFO:teuthology.orchestra.run.smithi196.stdout:checking clone #14, image image_client.0-clone14 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent15 2024-05-08T08:53:04.979 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:04.982+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:04.998 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:04.998 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:53:04.998 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:53:04.998 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:53:04.998 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:05.007 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:53:05.015 INFO:teuthology.orchestra.run.smithi196.stdout:2651 read 0xd15a600 thru 0xd15f696 (0x5097 bytes) 2024-05-08T08:53:05.020 INFO:teuthology.orchestra.run.smithi196.stdout:2652 trunc from 0xdd89800 to 0x9a8e800 2024-05-08T08:53:05.080 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:05.083+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:05.101 INFO:teuthology.orchestra.run.smithi196.stdout:2653 read 0x3959400 thru 0x3966199 (0xcd9a bytes) 2024-05-08T08:53:05.121 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:05.121 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:53:05.121 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:53:05.121 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:53:05.121 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.121 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7490 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 0~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:05.121 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.744226+0000", 2024-05-08T08:53:05.121 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 27.379210077, 2024-05-08T08:53:05.121 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057294818999999997, 2024-05-08T08:53:05.121 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:05.122 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:05.122 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:05.122 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:05.122 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:05.122 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:05.122 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7490 2024-05-08T08:53:05.122 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.122 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:05.122 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.122 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:05.122 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744226+0000", 2024-05-08T08:53:05.122 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:05.123 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.123 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.123 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:05.123 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744226+0000", 2024-05-08T08:53:05.123 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:05.123 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.123 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.123 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:05.124 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744226+0000", 2024-05-08T08:53:05.124 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3799999999999998e-07 2024-05-08T08:53:05.124 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.124 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.124 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:05.124 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744228+0000", 2024-05-08T08:53:05.125 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0710000000000002e-06 2024-05-08T08:53:05.125 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.125 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.125 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:05.125 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744233+0000", 2024-05-08T08:53:05.125 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0130000000000001e-06 2024-05-08T08:53:05.125 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.125 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.125 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:05.125 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744237+0000", 2024-05-08T08:53:05.125 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1930000000000002e-06 2024-05-08T08:53:05.125 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.125 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.125 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:05.125 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.801334+0000", 2024-05-08T08:53:05.126 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057097004999999999 2024-05-08T08:53:05.126 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.126 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.126 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:05.126 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.801414+0000", 2024-05-08T08:53:05.126 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0129000000000005e-05 2024-05-08T08:53:05.126 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.126 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.126 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:05.126 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.801521+0000", 2024-05-08T08:53:05.126 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010707 2024-05-08T08:53:05.126 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:05.126 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:05.127 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:05.127 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.127 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.127 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7491 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 40960~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:05.127 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.744242+0000", 2024-05-08T08:53:05.127 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 27.379194246000001, 2024-05-08T08:53:05.127 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057541677999999999, 2024-05-08T08:53:05.127 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:05.127 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:05.127 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:05.127 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:05.127 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:05.127 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:05.127 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7491 2024-05-08T08:53:05.128 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.128 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:05.128 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.128 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:05.128 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744242+0000", 2024-05-08T08:53:05.128 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:05.128 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.128 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.128 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:05.128 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744242+0000", 2024-05-08T08:53:05.128 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:05.128 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.128 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.128 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:05.128 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744242+0000", 2024-05-08T08:53:05.129 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.2000000000000001e-07 2024-05-08T08:53:05.129 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.129 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.129 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:05.129 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744244+0000", 2024-05-08T08:53:05.129 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6959999999999999e-06 2024-05-08T08:53:05.129 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.129 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.129 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:05.129 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744247+0000", 2024-05-08T08:53:05.129 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3000000000000002e-06 2024-05-08T08:53:05.129 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.129 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.130 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:05.130 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744250+0000", 2024-05-08T08:53:05.130 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.5399999999999998e-06 2024-05-08T08:53:05.130 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.130 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.130 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:05.130 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.801564+0000", 2024-05-08T08:53:05.132 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057313809 2024-05-08T08:53:05.132 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.132 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.132 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:05.133 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.801671+0000", 2024-05-08T08:53:05.133 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000107333 2024-05-08T08:53:05.133 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.133 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.133 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:05.133 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.801784+0000", 2024-05-08T08:53:05.133 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011268 2024-05-08T08:53:05.133 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:05.133 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:05.133 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:05.133 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.133 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.133 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7492 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 81920~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:05.133 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.744253+0000", 2024-05-08T08:53:05.133 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 27.379182769, 2024-05-08T08:53:05.134 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057761380000000001, 2024-05-08T08:53:05.134 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:05.134 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:05.134 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:05.134 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:05.134 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:05.135 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:05.135 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7492 2024-05-08T08:53:05.135 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.135 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:05.135 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.135 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:05.135 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744253+0000", 2024-05-08T08:53:05.135 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:05.135 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.135 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.135 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:05.136 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744253+0000", 2024-05-08T08:53:05.136 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:05.136 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.136 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.136 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:05.136 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.745991+0000", 2024-05-08T08:53:05.136 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001738021 2024-05-08T08:53:05.136 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.136 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.136 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:05.136 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.745997+0000", 2024-05-08T08:53:05.136 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5300000000000004e-06 2024-05-08T08:53:05.136 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.136 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.136 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:05.136 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.746006+0000", 2024-05-08T08:53:05.137 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2210000000000003e-06 2024-05-08T08:53:05.137 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.137 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.137 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:05.137 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.746026+0000", 2024-05-08T08:53:05.137 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9487000000000002e-05 2024-05-08T08:53:05.137 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.137 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.137 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:05.137 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.801815+0000", 2024-05-08T08:53:05.137 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.055789196999999999 2024-05-08T08:53:05.137 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.137 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.137 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:05.138 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.801908+0000", 2024-05-08T08:53:05.138 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3320999999999996e-05 2024-05-08T08:53:05.138 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.138 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.138 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:05.138 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.802015+0000", 2024-05-08T08:53:05.138 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000106603 2024-05-08T08:53:05.138 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:05.138 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:05.138 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:05.138 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.138 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.138 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7499 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 368640~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:05.138 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.746209+0000", 2024-05-08T08:53:05.139 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 27.377226821000001, 2024-05-08T08:53:05.139 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057256452999999999, 2024-05-08T08:53:05.139 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:05.139 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:05.139 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:05.139 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:05.139 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:05.139 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:05.139 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7499 2024-05-08T08:53:05.139 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.139 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:05.139 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.139 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:05.139 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.746209+0000", 2024-05-08T08:53:05.139 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:05.140 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.140 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.140 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:05.140 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.746209+0000", 2024-05-08T08:53:05.140 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:05.140 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.140 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.140 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:05.140 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.746210+0000", 2024-05-08T08:53:05.140 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2100000000000002e-07 2024-05-08T08:53:05.140 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.140 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.140 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:05.140 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.746212+0000", 2024-05-08T08:53:05.140 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.922e-06 2024-05-08T08:53:05.141 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.141 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.141 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:05.141 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.746221+0000", 2024-05-08T08:53:05.141 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2900000000000008e-06 2024-05-08T08:53:05.141 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.141 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.141 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:05.141 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.746227+0000", 2024-05-08T08:53:05.141 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3740000000000003e-06 2024-05-08T08:53:05.141 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.141 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.141 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:05.141 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.803304+0000", 2024-05-08T08:53:05.142 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057076239000000001 2024-05-08T08:53:05.142 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.142 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.142 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:05.142 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.803380+0000", 2024-05-08T08:53:05.142 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.6309000000000004e-05 2024-05-08T08:53:05.142 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.142 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.142 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:05.142 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.803466+0000", 2024-05-08T08:53:05.142 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.5897999999999996e-05 2024-05-08T08:53:05.142 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:05.142 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:05.142 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:05.142 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.143 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.143 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7500 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 409600~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:05.143 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.746237+0000", 2024-05-08T08:53:05.143 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 27.377199238999999, 2024-05-08T08:53:05.143 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057429380000000002, 2024-05-08T08:53:05.143 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:05.143 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:05.143 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:05.143 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:05.143 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:05.143 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:05.143 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7500 2024-05-08T08:53:05.143 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.143 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:05.143 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.144 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:05.144 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.746237+0000", 2024-05-08T08:53:05.144 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:05.144 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.144 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.144 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:05.144 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.746237+0000", 2024-05-08T08:53:05.144 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:05.144 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.144 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.144 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:05.144 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.747959+0000", 2024-05-08T08:53:05.144 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001721888 2024-05-08T08:53:05.144 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.144 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.145 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:05.145 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.747964+0000", 2024-05-08T08:53:05.145 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0030000000000002e-06 2024-05-08T08:53:05.145 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.145 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.145 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:05.145 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.747976+0000", 2024-05-08T08:53:05.145 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2656999999999999e-05 2024-05-08T08:53:05.145 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.145 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.146 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:05.146 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.747983+0000", 2024-05-08T08:53:05.146 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.866e-06 2024-05-08T08:53:05.146 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.146 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.146 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:05.146 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.803495+0000", 2024-05-08T08:53:05.146 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.055511392999999999 2024-05-08T08:53:05.146 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.146 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.146 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:05.146 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.803574+0000", 2024-05-08T08:53:05.146 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9215000000000006e-05 2024-05-08T08:53:05.147 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.147 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.147 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:05.147 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.803666+0000", 2024-05-08T08:53:05.147 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2357999999999999e-05 2024-05-08T08:53:05.147 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:05.147 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:05.147 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:05.147 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.147 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.147 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7508 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 737280~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:05.147 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.748180+0000", 2024-05-08T08:53:05.147 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 27.375256182000001, 2024-05-08T08:53:05.147 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057419809000000002, 2024-05-08T08:53:05.147 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:05.148 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:05.148 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:05.148 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:05.148 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:05.148 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:05.148 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7508 2024-05-08T08:53:05.148 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.148 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:05.148 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.148 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:05.148 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748180+0000", 2024-05-08T08:53:05.148 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:05.148 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.148 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.148 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:05.148 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748180+0000", 2024-05-08T08:53:05.149 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:05.149 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.149 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.149 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:05.149 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748180+0000", 2024-05-08T08:53:05.149 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4499999999999997e-07 2024-05-08T08:53:05.149 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.149 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.149 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:05.149 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748183+0000", 2024-05-08T08:53:05.149 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1660000000000001e-06 2024-05-08T08:53:05.149 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.149 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.149 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:05.149 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748197+0000", 2024-05-08T08:53:05.150 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.4877e-05 2024-05-08T08:53:05.150 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.150 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.150 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:05.150 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748203+0000", 2024-05-08T08:53:05.150 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7470000000000002e-06 2024-05-08T08:53:05.150 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.150 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.150 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:05.150 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.805398+0000", 2024-05-08T08:53:05.150 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057194513000000002 2024-05-08T08:53:05.150 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.150 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.150 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:05.150 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.805544+0000", 2024-05-08T08:53:05.151 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000146522 2024-05-08T08:53:05.151 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.151 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.151 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:05.151 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.805600+0000", 2024-05-08T08:53:05.151 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5538999999999998e-05 2024-05-08T08:53:05.151 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:05.151 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:05.151 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:05.151 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.151 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.151 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7509 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 778240~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:05.151 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.748211+0000", 2024-05-08T08:53:05.151 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 27.375225112999999, 2024-05-08T08:53:05.151 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057619034999999999, 2024-05-08T08:53:05.152 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:05.152 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:05.152 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:05.152 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:05.152 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:05.152 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:05.152 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7509 2024-05-08T08:53:05.152 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.152 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:05.152 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.152 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:05.153 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748211+0000", 2024-05-08T08:53:05.153 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:05.153 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.153 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.153 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:05.153 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748211+0000", 2024-05-08T08:53:05.153 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:05.153 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.153 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.153 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:05.153 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748212+0000", 2024-05-08T08:53:05.154 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8100000000000003e-07 2024-05-08T08:53:05.154 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.154 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.154 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:05.154 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748214+0000", 2024-05-08T08:53:05.154 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4669999999999998e-06 2024-05-08T08:53:05.154 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.154 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.154 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:05.154 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748221+0000", 2024-05-08T08:53:05.154 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5769999999999999e-06 2024-05-08T08:53:05.154 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.154 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.154 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:05.155 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748225+0000", 2024-05-08T08:53:05.155 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8060000000000002e-06 2024-05-08T08:53:05.155 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.155 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.155 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:05.155 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.805636+0000", 2024-05-08T08:53:05.155 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057410309 2024-05-08T08:53:05.156 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.156 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.156 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:05.156 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.805723+0000", 2024-05-08T08:53:05.156 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7435999999999994e-05 2024-05-08T08:53:05.156 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.156 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.156 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:05.156 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.805830+0000", 2024-05-08T08:53:05.156 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000106959 2024-05-08T08:53:05.156 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:05.157 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:05.157 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:05.157 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.157 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.157 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7510 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 819200~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:05.157 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.748232+0000", 2024-05-08T08:53:05.157 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 27.375204083, 2024-05-08T08:53:05.157 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057710421999999997, 2024-05-08T08:53:05.157 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:05.157 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:05.157 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:05.157 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:05.157 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:05.157 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:05.157 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7510 2024-05-08T08:53:05.158 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.158 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:05.158 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.158 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:05.158 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748232+0000", 2024-05-08T08:53:05.158 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:05.158 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.158 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.158 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:05.158 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748232+0000", 2024-05-08T08:53:05.158 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:05.158 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.158 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.158 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:05.159 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.749959+0000", 2024-05-08T08:53:05.159 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001727 2024-05-08T08:53:05.159 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.159 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.159 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:05.159 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.749964+0000", 2024-05-08T08:53:05.159 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4700000000000004e-06 2024-05-08T08:53:05.159 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.159 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.159 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:05.159 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.749984+0000", 2024-05-08T08:53:05.159 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0068000000000002e-05 2024-05-08T08:53:05.159 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.159 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.159 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:05.160 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.749993+0000", 2024-05-08T08:53:05.160 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4720000000000001e-06 2024-05-08T08:53:05.160 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.160 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.160 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:05.160 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.805846+0000", 2024-05-08T08:53:05.160 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.055852770000000003 2024-05-08T08:53:05.160 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.160 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.161 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:05.161 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.805890+0000", 2024-05-08T08:53:05.161 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4323999999999998e-05 2024-05-08T08:53:05.161 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.161 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.161 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:05.161 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.805942+0000", 2024-05-08T08:53:05.161 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2318000000000003e-05 2024-05-08T08:53:05.161 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:05.161 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:05.161 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:05.161 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.161 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.161 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7515 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 1024000~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:05.162 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.750135+0000", 2024-05-08T08:53:05.162 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 27.373300706999999, 2024-05-08T08:53:05.162 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057188125999999999, 2024-05-08T08:53:05.162 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:05.162 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:05.162 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:05.162 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:05.162 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:05.162 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:05.162 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7515 2024-05-08T08:53:05.162 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.162 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:05.162 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.163 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:05.163 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750135+0000", 2024-05-08T08:53:05.163 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:05.163 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.163 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.163 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:05.163 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750135+0000", 2024-05-08T08:53:05.163 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:05.163 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.163 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.163 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:05.163 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750136+0000", 2024-05-08T08:53:05.163 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8100000000000003e-07 2024-05-08T08:53:05.163 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.163 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.164 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:05.164 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750138+0000", 2024-05-08T08:53:05.164 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.339e-06 2024-05-08T08:53:05.164 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.164 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.164 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:05.164 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750147+0000", 2024-05-08T08:53:05.164 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7669999999999992e-06 2024-05-08T08:53:05.164 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.164 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.164 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:05.164 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750153+0000", 2024-05-08T08:53:05.164 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5029999999999997e-06 2024-05-08T08:53:05.164 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.164 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.164 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:05.165 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.807145+0000", 2024-05-08T08:53:05.165 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.056992174 2024-05-08T08:53:05.165 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.165 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.165 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:05.165 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.807231+0000", 2024-05-08T08:53:05.165 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.6347000000000005e-05 2024-05-08T08:53:05.165 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.165 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.165 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:05.165 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.807324+0000", 2024-05-08T08:53:05.165 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2515000000000004e-05 2024-05-08T08:53:05.165 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:05.166 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:05.166 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:05.166 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.166 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.166 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7516 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 1064960~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:05.166 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.750162+0000", 2024-05-08T08:53:05.166 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 27.373273877999999, 2024-05-08T08:53:05.166 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057363971999999999, 2024-05-08T08:53:05.167 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:05.167 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:05.167 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:05.167 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:05.167 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:05.167 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:05.167 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7516 2024-05-08T08:53:05.167 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.167 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:05.167 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.167 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:05.167 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750162+0000", 2024-05-08T08:53:05.167 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:05.168 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.168 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.168 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:05.168 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750162+0000", 2024-05-08T08:53:05.168 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:05.168 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.168 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.168 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:05.168 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750163+0000", 2024-05-08T08:53:05.168 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.13e-07 2024-05-08T08:53:05.168 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.168 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.168 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:05.168 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750165+0000", 2024-05-08T08:53:05.169 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2309999999999999e-06 2024-05-08T08:53:05.169 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.169 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.169 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:05.169 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750174+0000", 2024-05-08T08:53:05.169 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4740000000000005e-06 2024-05-08T08:53:05.169 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.169 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.169 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:05.169 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750180+0000", 2024-05-08T08:53:05.169 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4760000000000003e-06 2024-05-08T08:53:05.169 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.169 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.169 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:05.169 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.807355+0000", 2024-05-08T08:53:05.170 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057175029000000002 2024-05-08T08:53:05.170 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.170 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.170 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:05.170 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.807437+0000", 2024-05-08T08:53:05.170 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.2268999999999994e-05 2024-05-08T08:53:05.170 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.170 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.170 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:05.170 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.807526+0000", 2024-05-08T08:53:05.170 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.8880000000000003e-05 2024-05-08T08:53:05.171 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:05.171 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:05.171 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:05.171 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.171 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.171 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7517 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 1105920~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:05.171 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.750188+0000", 2024-05-08T08:53:05.171 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 27.373248385, 2024-05-08T08:53:05.171 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057547099999999997, 2024-05-08T08:53:05.171 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:05.171 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:05.171 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:05.171 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:05.172 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:05.172 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:05.172 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7517 2024-05-08T08:53:05.172 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.172 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:05.172 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.172 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:05.172 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750188+0000", 2024-05-08T08:53:05.172 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:05.172 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.172 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.172 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:05.172 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750188+0000", 2024-05-08T08:53:05.172 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:05.173 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.173 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.173 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:05.173 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750188+0000", 2024-05-08T08:53:05.173 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7199999999999999e-07 2024-05-08T08:53:05.173 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.173 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.173 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:05.173 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750190+0000", 2024-05-08T08:53:05.174 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.154e-06 2024-05-08T08:53:05.174 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.174 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.174 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:05.174 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750198+0000", 2024-05-08T08:53:05.174 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8320000000000006e-06 2024-05-08T08:53:05.174 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.174 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.174 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:05.174 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750203+0000", 2024-05-08T08:53:05.174 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1900000000000003e-06 2024-05-08T08:53:05.175 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.175 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.175 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:05.175 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.807555+0000", 2024-05-08T08:53:05.175 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057351148999999997 2024-05-08T08:53:05.175 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.175 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.175 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:05.175 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.807640+0000", 2024-05-08T08:53:05.175 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.5853999999999995e-05 2024-05-08T08:53:05.175 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.175 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.175 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:05.176 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.807735+0000", 2024-05-08T08:53:05.176 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4449000000000003e-05 2024-05-08T08:53:05.176 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:05.176 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:05.176 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:05.176 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.176 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.176 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7518 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 1146880~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:05.176 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.750212+0000", 2024-05-08T08:53:05.176 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 27.373224584999999, 2024-05-08T08:53:05.176 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057664963999999999, 2024-05-08T08:53:05.176 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:05.176 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:05.176 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:05.176 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:05.177 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:05.177 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:05.177 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7518 2024-05-08T08:53:05.177 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.177 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:05.177 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.177 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:05.177 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750212+0000", 2024-05-08T08:53:05.178 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:05.178 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.178 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.178 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:05.178 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750212+0000", 2024-05-08T08:53:05.178 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:05.178 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.178 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.178 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:05.178 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750212+0000", 2024-05-08T08:53:05.178 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.46e-07 2024-05-08T08:53:05.178 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.179 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.179 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:05.179 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750220+0000", 2024-05-08T08:53:05.179 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5610000000000001e-06 2024-05-08T08:53:05.179 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.179 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.179 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:05.179 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750228+0000", 2024-05-08T08:53:05.179 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3180000000000007e-06 2024-05-08T08:53:05.179 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.179 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.179 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:05.179 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750234+0000", 2024-05-08T08:53:05.179 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0209999999999996e-06 2024-05-08T08:53:05.180 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.180 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.180 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:05.180 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.807765+0000", 2024-05-08T08:53:05.180 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057531481000000002 2024-05-08T08:53:05.180 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.180 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.180 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:05.180 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.807815+0000", 2024-05-08T08:53:05.180 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9282e-05 2024-05-08T08:53:05.180 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.180 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.180 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:05.180 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.807877+0000", 2024-05-08T08:53:05.180 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1854999999999996e-05 2024-05-08T08:53:05.181 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:05.181 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:05.181 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:05.181 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.181 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.181 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7519 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 1187840~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:05.181 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.750261+0000", 2024-05-08T08:53:05.181 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 27.373175099000001, 2024-05-08T08:53:05.181 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057872519999999997, 2024-05-08T08:53:05.181 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:05.181 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:05.181 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:05.181 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:05.181 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:05.181 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:05.182 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7519 2024-05-08T08:53:05.182 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.182 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:05.182 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.182 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:05.182 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750261+0000", 2024-05-08T08:53:05.182 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:05.182 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.182 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.182 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:05.182 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750261+0000", 2024-05-08T08:53:05.182 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:05.182 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.182 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.183 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:05.183 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750262+0000", 2024-05-08T08:53:05.183 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7000000000000005e-07 2024-05-08T08:53:05.183 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.183 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.183 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:05.183 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750265+0000", 2024-05-08T08:53:05.183 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4589999999999998e-06 2024-05-08T08:53:05.183 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.183 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.183 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:05.183 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750274+0000", 2024-05-08T08:53:05.183 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.5459999999999992e-06 2024-05-08T08:53:05.184 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.184 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.184 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:05.184 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750280+0000", 2024-05-08T08:53:05.184 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0950000000000001e-06 2024-05-08T08:53:05.184 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.184 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.184 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:05.184 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.807917+0000", 2024-05-08T08:53:05.184 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057637715999999999 2024-05-08T08:53:05.184 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.184 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.184 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:05.184 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.808013+0000", 2024-05-08T08:53:05.184 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5569999999999995e-05 2024-05-08T08:53:05.185 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.185 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.185 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:05.185 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.808134+0000", 2024-05-08T08:53:05.185 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012056400000000001 2024-05-08T08:53:05.185 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:05.185 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:05.185 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:05.185 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.185 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.185 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7520 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 1228800~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:05.186 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.750289+0000", 2024-05-08T08:53:05.186 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 27.373146654999999, 2024-05-08T08:53:05.186 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.058133788999999998, 2024-05-08T08:53:05.186 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:05.186 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:05.186 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:05.186 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:05.186 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:05.187 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:05.187 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7520 2024-05-08T08:53:05.187 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.187 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:05.187 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.187 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:05.187 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750289+0000", 2024-05-08T08:53:05.187 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:05.187 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.188 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.188 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:05.188 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750289+0000", 2024-05-08T08:53:05.188 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:05.188 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.188 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.188 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:05.188 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.751976+0000", 2024-05-08T08:53:05.188 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0016865389999999999 2024-05-08T08:53:05.189 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.189 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.189 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:05.189 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.751981+0000", 2024-05-08T08:53:05.189 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5719999999999996e-06 2024-05-08T08:53:05.189 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.189 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.189 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:05.189 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.751991+0000", 2024-05-08T08:53:05.190 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0788000000000001e-05 2024-05-08T08:53:05.190 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.190 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.190 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:05.190 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.751999+0000", 2024-05-08T08:53:05.190 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2690000000000002e-06 2024-05-08T08:53:05.190 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.191 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.191 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:05.191 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.808183+0000", 2024-05-08T08:53:05.191 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.056184686999999997 2024-05-08T08:53:05.191 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.191 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.191 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:05.191 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.808296+0000", 2024-05-08T08:53:05.191 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000112754 2024-05-08T08:53:05.192 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.192 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.192 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:05.192 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.808423+0000", 2024-05-08T08:53:05.192 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012718000000000001 2024-05-08T08:53:05.192 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:05.192 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:05.192 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:05.192 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.193 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.193 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7524 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 1392640~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:05.193 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.752121+0000", 2024-05-08T08:53:05.193 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 27.371315606, 2024-05-08T08:53:05.193 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057199521000000003, 2024-05-08T08:53:05.193 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:05.193 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:05.193 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:05.193 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:05.194 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:05.194 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:05.194 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7524 2024-05-08T08:53:05.194 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.194 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:05.194 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.194 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:05.195 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752121+0000", 2024-05-08T08:53:05.195 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:05.195 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.195 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.195 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:05.195 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752121+0000", 2024-05-08T08:53:05.195 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:05.195 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.195 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.196 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:05.196 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752121+0000", 2024-05-08T08:53:05.196 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5999999999999999e-07 2024-05-08T08:53:05.196 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.196 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.196 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:05.196 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752123+0000", 2024-05-08T08:53:05.196 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3499999999999999e-06 2024-05-08T08:53:05.196 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.197 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.197 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:05.197 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752132+0000", 2024-05-08T08:53:05.197 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.5539999999999998e-06 2024-05-08T08:53:05.197 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.197 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.197 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:05.197 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752138+0000", 2024-05-08T08:53:05.197 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3640000000000004e-06 2024-05-08T08:53:05.197 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.198 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.198 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:05.198 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.809210+0000", 2024-05-08T08:53:05.198 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057071469999999999 2024-05-08T08:53:05.198 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.198 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.198 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:05.198 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.809258+0000", 2024-05-08T08:53:05.198 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8755000000000003e-05 2024-05-08T08:53:05.198 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.199 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.199 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:05.199 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.809320+0000", 2024-05-08T08:53:05.199 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1568000000000004e-05 2024-05-08T08:53:05.199 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:05.199 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:05.199 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:05.199 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.199 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.199 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7525 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 1433600~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:05.200 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.752146+0000", 2024-05-08T08:53:05.200 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 27.371290561999999, 2024-05-08T08:53:05.200 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057400072000000003, 2024-05-08T08:53:05.200 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:05.200 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:05.200 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:05.200 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:05.200 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:05.200 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:05.200 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7525 2024-05-08T08:53:05.200 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.200 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:05.200 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.201 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:05.201 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752146+0000", 2024-05-08T08:53:05.201 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:05.201 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.201 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.201 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:05.201 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752146+0000", 2024-05-08T08:53:05.201 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:05.201 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.201 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.201 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:05.201 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752146+0000", 2024-05-08T08:53:05.201 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0400000000000002e-07 2024-05-08T08:53:05.201 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.201 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.201 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:05.202 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752148+0000", 2024-05-08T08:53:05.202 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3209999999999999e-06 2024-05-08T08:53:05.202 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.202 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.202 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:05.202 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752155+0000", 2024-05-08T08:53:05.202 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6109999999999999e-06 2024-05-08T08:53:05.202 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.202 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.202 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:05.202 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752161+0000", 2024-05-08T08:53:05.202 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.489e-06 2024-05-08T08:53:05.202 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.203 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.203 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:05.203 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.809360+0000", 2024-05-08T08:53:05.203 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057198860999999997 2024-05-08T08:53:05.203 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.203 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.203 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:05.203 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.809450+0000", 2024-05-08T08:53:05.203 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0136000000000006e-05 2024-05-08T08:53:05.203 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.203 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.203 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:05.203 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.809546+0000", 2024-05-08T08:53:05.203 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5249999999999998e-05 2024-05-08T08:53:05.203 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:05.204 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:05.204 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:05.204 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.204 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.204 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7526 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 1474560~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:05.204 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.752168+0000", 2024-05-08T08:53:05.204 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 27.371267848999999, 2024-05-08T08:53:05.204 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057516818999999997, 2024-05-08T08:53:05.204 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:05.204 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:05.204 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:05.204 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:05.204 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:05.204 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:05.204 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7526 2024-05-08T08:53:05.205 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.205 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:05.205 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.205 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:05.205 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752168+0000", 2024-05-08T08:53:05.205 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:05.205 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.205 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.205 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:05.205 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752168+0000", 2024-05-08T08:53:05.205 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:05.205 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.205 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.205 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:05.205 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752169+0000", 2024-05-08T08:53:05.205 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4799999999999999e-07 2024-05-08T08:53:05.206 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.206 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.206 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:05.206 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752171+0000", 2024-05-08T08:53:05.206 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.289e-06 2024-05-08T08:53:05.206 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.206 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.206 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:05.206 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752177+0000", 2024-05-08T08:53:05.206 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3690000000000003e-06 2024-05-08T08:53:05.206 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.206 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.206 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:05.206 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752183+0000", 2024-05-08T08:53:05.207 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.152e-06 2024-05-08T08:53:05.207 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.207 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.207 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:05.207 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.809574+0000", 2024-05-08T08:53:05.207 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057391075 2024-05-08T08:53:05.207 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.207 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.207 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:05.207 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.809624+0000", 2024-05-08T08:53:05.207 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0673000000000001e-05 2024-05-08T08:53:05.207 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.207 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.207 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:05.207 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.809685+0000", 2024-05-08T08:53:05.208 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0813000000000002e-05 2024-05-08T08:53:05.208 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:05.208 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:05.208 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:05.208 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.208 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.208 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7527 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 1515520~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:05.208 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.752189+0000", 2024-05-08T08:53:05.208 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 27.371246921000001, 2024-05-08T08:53:05.208 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057722168999999997, 2024-05-08T08:53:05.208 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:05.208 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:05.209 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:05.209 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:05.209 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:05.209 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:05.209 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7527 2024-05-08T08:53:05.209 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.209 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:05.209 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.209 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:05.210 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752189+0000", 2024-05-08T08:53:05.210 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:05.210 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.210 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.210 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:05.210 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752189+0000", 2024-05-08T08:53:05.210 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:05.210 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.210 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.210 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:05.211 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752190+0000", 2024-05-08T08:53:05.211 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8799999999999998e-07 2024-05-08T08:53:05.211 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.211 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.211 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:05.211 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752192+0000", 2024-05-08T08:53:05.211 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9759999999999998e-06 2024-05-08T08:53:05.211 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.211 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.211 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:05.211 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752199+0000", 2024-05-08T08:53:05.211 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3289999999999996e-06 2024-05-08T08:53:05.211 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.212 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.212 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:05.212 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752205+0000", 2024-05-08T08:53:05.212 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2759999999999997e-06 2024-05-08T08:53:05.212 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.212 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.212 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:05.212 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.809726+0000", 2024-05-08T08:53:05.212 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057521249000000003 2024-05-08T08:53:05.212 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.212 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.212 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:05.212 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.809815+0000", 2024-05-08T08:53:05.212 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.8987999999999996e-05 2024-05-08T08:53:05.213 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.213 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.213 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:05.213 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.809911+0000", 2024-05-08T08:53:05.213 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5963000000000003e-05 2024-05-08T08:53:05.213 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:05.213 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:05.213 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:05.213 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.213 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.213 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7528 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 1556480~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:05.213 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.752212+0000", 2024-05-08T08:53:05.214 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 27.371224211000001, 2024-05-08T08:53:05.214 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057904956, 2024-05-08T08:53:05.214 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:05.214 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:05.214 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:05.214 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:05.214 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:05.214 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:05.214 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7528 2024-05-08T08:53:05.214 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.214 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:05.214 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.214 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:05.215 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752212+0000", 2024-05-08T08:53:05.215 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:05.215 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.215 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.215 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:05.215 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752212+0000", 2024-05-08T08:53:05.215 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:05.215 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.215 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.215 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:05.215 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752212+0000", 2024-05-08T08:53:05.215 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7800000000000002e-07 2024-05-08T08:53:05.216 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.216 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.216 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:05.216 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752215+0000", 2024-05-08T08:53:05.216 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2120000000000002e-06 2024-05-08T08:53:05.216 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.216 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.216 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:05.216 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752222+0000", 2024-05-08T08:53:05.216 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.6329999999999996e-06 2024-05-08T08:53:05.216 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.216 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.216 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:05.216 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752227+0000", 2024-05-08T08:53:05.217 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9289999999999997e-06 2024-05-08T08:53:05.217 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.217 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.217 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:05.217 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.809966+0000", 2024-05-08T08:53:05.217 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057738554999999997 2024-05-08T08:53:05.217 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.217 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.217 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:05.217 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.810052+0000", 2024-05-08T08:53:05.217 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.6540999999999997e-05 2024-05-08T08:53:05.217 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.217 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.217 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:05.217 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.810117+0000", 2024-05-08T08:53:05.218 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4608000000000002e-05 2024-05-08T08:53:05.218 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:05.218 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:05.218 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:05.218 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.218 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.218 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7529 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 1597440~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:05.218 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.752234+0000", 2024-05-08T08:53:05.218 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 27.371202197999999, 2024-05-08T08:53:05.218 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.058133851, 2024-05-08T08:53:05.218 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:05.218 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:05.219 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:05.219 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:05.219 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:05.219 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:05.219 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7529 2024-05-08T08:53:05.219 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.219 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:05.219 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.219 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:05.219 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752234+0000", 2024-05-08T08:53:05.219 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:05.219 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.219 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.219 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:05.219 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752234+0000", 2024-05-08T08:53:05.220 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:05.220 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.220 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.220 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:05.220 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.753959+0000", 2024-05-08T08:53:05.220 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0017251409999999999 2024-05-08T08:53:05.220 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.220 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.220 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:05.220 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.753963+0000", 2024-05-08T08:53:05.221 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2860000000000001e-06 2024-05-08T08:53:05.221 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.221 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.221 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:05.221 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.753973+0000", 2024-05-08T08:53:05.221 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5119999999999997e-06 2024-05-08T08:53:05.221 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.221 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.221 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:05.221 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.753979+0000", 2024-05-08T08:53:05.221 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3589999999999996e-06 2024-05-08T08:53:05.221 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.222 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.222 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:05.222 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.810182+0000", 2024-05-08T08:53:05.222 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.056202941999999999 2024-05-08T08:53:05.222 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.222 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.222 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:05.222 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.810270+0000", 2024-05-08T08:53:05.222 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7338999999999998e-05 2024-05-08T08:53:05.222 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.222 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.222 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:05.222 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.810368+0000", 2024-05-08T08:53:05.222 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8271999999999997e-05 2024-05-08T08:53:05.223 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:05.223 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:05.223 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:05.223 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:05.223 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:05.223 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:05.223 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T08:53:05.224 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 5] out_osds: [4] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-05-08T08:53:05.224 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T08:53:05.224 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-05-08T08:53:05.224 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-05-08T08:53:05.225 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:05.228+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:05.227 INFO:teuthology.orchestra.run.smithi196.stdout:2655 write 0x297d400 thru 0x2981a27 (0x4628 bytes) 2024-05-08T08:53:05.227 INFO:teuthology.orchestra.run.smithi196.stdout:2660 read 0x2d33000 thru 0x2d39752 (0x6753 bytes) 2024-05-08T08:53:05.227 INFO:teuthology.orchestra.run.smithi196.stdout:2661 punch from 0x5deec00 to 0x5df5a00, (0x6e00 bytes) 2024-05-08T08:53:05.227 INFO:teuthology.orchestra.run.smithi196.stdout:2662 write 0xe329c00 thru 0xe338af3 (0xeef4 bytes) 2024-05-08T08:53:05.227 INFO:teuthology.orchestra.run.smithi196.stdout:2664 trunc from 0xe338c00 to 0x60fa600 2024-05-08T08:53:05.227 INFO:teuthology.orchestra.run.smithi196.stdout:2665 read 0x3f12000 thru 0x3f1f1e8 (0xd1e9 bytes) 2024-05-08T08:53:05.227 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:53:05.230 INFO:teuthology.orchestra.run.smithi196.stdout:2666 read 0xb47800 thru 0xb4c309 (0x4b0a bytes) 2024-05-08T08:53:05.231 INFO:teuthology.orchestra.run.smithi196.stdout:2669 read 0x3be7200 thru 0x3bf3106 (0xbf07 bytes) 2024-05-08T08:53:05.232 INFO:teuthology.orchestra.run.smithi196.stdout:2670 write 0x257fe00 thru 0x258176a (0x196b bytes) 2024-05-08T08:53:05.232 INFO:teuthology.orchestra.run.smithi196.stdout:2671 read 0x406cc00 thru 0x4076f4c (0xa34d bytes) 2024-05-08T08:53:05.232 INFO:teuthology.orchestra.run.smithi196.stdout:2673 read 0xb56c00 thru 0xb5cf18 (0x6319 bytes) 2024-05-08T08:53:05.232 INFO:teuthology.orchestra.run.smithi196.stdout:2674 write 0x1e5ac00 thru 0x1e5dea8 (0x32a9 bytes) 2024-05-08T08:53:05.233 INFO:teuthology.orchestra.run.smithi196.stdout:2675 read 0x3435a00 thru 0x343a2d9 (0x48da bytes) 2024-05-08T08:53:05.234 INFO:teuthology.orchestra.run.smithi196.stdout:2677 trunc from 0x60fa600 to 0x4687c00 2024-05-08T08:53:05.314 INFO:teuthology.orchestra.run.smithi196.stdout:2678 trunc from 0x4687c00 to 0x9f24200 2024-05-08T08:53:05.326 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:05.329+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:05.327 INFO:teuthology.orchestra.run.smithi196.stdout:2679 write 0x1070200 thru 0x107c35f (0xc160 bytes) 2024-05-08T08:53:05.328 INFO:teuthology.orchestra.run.smithi196.stdout:2682 write 0x5d8000 thru 0x5dfc04 (0x7c05 bytes) 2024-05-08T08:53:05.329 INFO:teuthology.orchestra.run.smithi196.stdout:2683 read 0x1bb4000 thru 0x1bb4ec4 (0xec5 bytes) 2024-05-08T08:53:05.338 INFO:teuthology.orchestra.run.smithi196.stdout:2686 trunc from 0x9f24200 to 0x2b0d800 2024-05-08T08:53:05.343 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:53:05.343 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:53:05.343 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:05.344 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.5052.0:497 3.1f e59/18 3:fb92de62:::rbd_data.1140b92ee0d3.0000000000000004:head v 59'211, mlcod=51'208)", 2024-05-08T08:53:05.344 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:05.346896+0000", 2024-05-08T08:53:05.344 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.00055446900000000003, 2024-05-08T08:53:05.344 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00056999800000000005, 2024-05-08T08:53:05.344 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:05.344 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:05.344 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:05.344 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:05.344 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:05.344 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:05.344 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:05.346896+0000", 2024-05-08T08:53:05.344 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:05.344 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:05.344 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:05.344 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:05.345 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:05.346896+0000", 2024-05-08T08:53:05.345 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:05.345 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:05.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:05.345 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:05.345 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:05.346896+0000", 2024-05-08T08:53:05.345 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7100000000000001e-07 2024-05-08T08:53:05.345 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:05.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:05.345 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:05.345 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:05.347084+0000", 2024-05-08T08:53:05.345 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000188004 2024-05-08T08:53:05.345 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:05.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:05.345 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:05.345 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:05.347095+0000", 2024-05-08T08:53:05.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0545e-05 2024-05-08T08:53:05.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:05.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:05.346 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:05.346 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:05.347100+0000", 2024-05-08T08:53:05.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9080000000000003e-06 2024-05-08T08:53:05.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:05.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:05.346 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:05.346 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:05.347154+0000", 2024-05-08T08:53:05.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4579e-05 2024-05-08T08:53:05.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:05.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:05.346 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:05.347 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:05.347173+0000", 2024-05-08T08:53:05.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8952000000000001e-05 2024-05-08T08:53:05.347 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:05.347 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:05.347 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:05.347 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:05.347 INFO:teuthology.orchestra.run.smithi192.stdout: ], 2024-05-08T08:53:05.347 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 1 2024-05-08T08:53:05.347 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:53:05.353 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:53:05.370 INFO:teuthology.orchestra.run.smithi059.stderr:noscrub is unset 2024-05-08T08:53:05.407 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:53:05.407 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:53:05.427 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:53:05.427 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:53:05.438 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:05.441+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:05.443 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:53:05.444 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:53:05.461 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:53:05.462 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:53:05.479 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:53:05.479 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:53:05.498 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:53:05.498 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:53:05.500 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:05.500 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:05.509 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:53:05.550 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:05.553+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:05.594 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:53:05.595 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:05.595 INFO:teuthology.orchestra.run.smithi059.stdout: "epoch": 60, 2024-05-08T08:53:05.595 INFO:teuthology.orchestra.run.smithi059.stdout: "fsid": "38dfae8c-5c5e-4b0f-a264-91f22c6c5554", 2024-05-08T08:53:05.595 INFO:teuthology.orchestra.run.smithi059.stdout: "created": "2024-05-08T08:51:45.571369+0000", 2024-05-08T08:53:05.595 INFO:teuthology.orchestra.run.smithi059.stdout: "modified": "2024-05-08T08:53:05.367783+0000", 2024-05-08T08:53:05.595 INFO:teuthology.orchestra.run.smithi059.stdout: "last_up_change": "2024-05-08T08:51:56.847706+0000", 2024-05-08T08:53:05.595 INFO:teuthology.orchestra.run.smithi059.stdout: "last_in_change": "2024-05-08T08:52:46.045870+0000", 2024-05-08T08:53:05.595 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-05-08T08:53:05.595 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_num": 5804032, 2024-05-08T08:53:05.595 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_set": [ 2024-05-08T08:53:05.596 INFO:teuthology.orchestra.run.smithi059.stdout: "nodeep-scrub", 2024-05-08T08:53:05.596 INFO:teuthology.orchestra.run.smithi059.stdout: "pglog_hardlimit", 2024-05-08T08:53:05.596 INFO:teuthology.orchestra.run.smithi059.stdout: "purged_snapdirs", 2024-05-08T08:53:05.596 INFO:teuthology.orchestra.run.smithi059.stdout: "recovery_deletes", 2024-05-08T08:53:05.596 INFO:teuthology.orchestra.run.smithi059.stdout: "sortbitwise" 2024-05-08T08:53:05.596 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T08:53:05.596 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_version": 4, 2024-05-08T08:53:05.596 INFO:teuthology.orchestra.run.smithi059.stdout: "full_ratio": 0.94999998807907104, 2024-05-08T08:53:05.596 INFO:teuthology.orchestra.run.smithi059.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-05-08T08:53:05.596 INFO:teuthology.orchestra.run.smithi059.stdout: "nearfull_ratio": 0.85000002384185791, 2024-05-08T08:53:05.596 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_snapshot": "", 2024-05-08T08:53:05.596 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_max": 3, 2024-05-08T08:53:05.596 INFO:teuthology.orchestra.run.smithi059.stdout: "max_osd": 6, 2024-05-08T08:53:05.597 INFO:teuthology.orchestra.run.smithi059.stdout: "require_min_compat_client": "luminous", 2024-05-08T08:53:05.597 INFO:teuthology.orchestra.run.smithi059.stdout: "min_compat_client": "luminous", 2024-05-08T08:53:05.597 INFO:teuthology.orchestra.run.smithi059.stdout: "require_osd_release": "squid", 2024-05-08T08:53:05.597 INFO:teuthology.orchestra.run.smithi059.stdout: "allow_crimson": false, 2024-05-08T08:53:05.597 INFO:teuthology.orchestra.run.smithi059.stdout: "pools": [ 2024-05-08T08:53:05.597 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.597 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 1, 2024-05-08T08:53:05.598 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_name": ".mgr", 2024-05-08T08:53:05.598 INFO:teuthology.orchestra.run.smithi059.stdout: "create_time": "2024-05-08T08:51:57.787790+0000", 2024-05-08T08:53:05.598 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T08:53:05.598 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_names": "hashpspool", 2024-05-08T08:53:05.598 INFO:teuthology.orchestra.run.smithi059.stdout: "type": 1, 2024-05-08T08:53:05.598 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 2, 2024-05-08T08:53:05.598 INFO:teuthology.orchestra.run.smithi059.stdout: "min_size": 1, 2024-05-08T08:53:05.598 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_rule": 0, 2024-05-08T08:53:05.598 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_count": 0, 2024-05-08T08:53:05.599 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_target": 0, 2024-05-08T08:53:05.599 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_barrier": 0, 2024-05-08T08:53:05.599 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-08T08:53:05.599 INFO:teuthology.orchestra.run.smithi059.stdout: "object_hash": 2, 2024-05-08T08:53:05.599 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscale_mode": "off", 2024-05-08T08:53:05.599 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num": 1, 2024-05-08T08:53:05.599 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num": 1, 2024-05-08T08:53:05.599 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num_target": 1, 2024-05-08T08:53:05.599 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_target": 1, 2024-05-08T08:53:05.599 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_pending": 1, 2024-05-08T08:53:05.599 INFO:teuthology.orchestra.run.smithi059.stdout: "last_pg_merge_meta": { 2024-05-08T08:53:05.599 INFO:teuthology.orchestra.run.smithi059.stdout: "source_pgid": "0.0", 2024-05-08T08:53:05.600 INFO:teuthology.orchestra.run.smithi059.stdout: "ready_epoch": 0, 2024-05-08T08:53:05.600 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_started": 0, 2024-05-08T08:53:05.600 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_clean": 0, 2024-05-08T08:53:05.600 INFO:teuthology.orchestra.run.smithi059.stdout: "source_version": "0'0", 2024-05-08T08:53:05.600 INFO:teuthology.orchestra.run.smithi059.stdout: "target_version": "0'0" 2024-05-08T08:53:05.600 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.600 INFO:teuthology.orchestra.run.smithi059.stdout: "last_change": "14", 2024-05-08T08:53:05.600 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend": "0", 2024-05-08T08:53:05.600 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-08T08:53:05.600 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_preluminous": "0", 2024-05-08T08:53:05.600 INFO:teuthology.orchestra.run.smithi059.stdout: "auid": 0, 2024-05-08T08:53:05.600 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_mode": "selfmanaged", 2024-05-08T08:53:05.600 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_seq": 0, 2024-05-08T08:53:05.601 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_epoch": 0, 2024-05-08T08:53:05.601 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_snaps": [], 2024-05-08T08:53:05.601 INFO:teuthology.orchestra.run.smithi059.stdout: "removed_snaps": "[]", 2024-05-08T08:53:05.601 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_bytes": 0, 2024-05-08T08:53:05.601 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_objects": 0, 2024-05-08T08:53:05.601 INFO:teuthology.orchestra.run.smithi059.stdout: "tiers": [], 2024-05-08T08:53:05.601 INFO:teuthology.orchestra.run.smithi059.stdout: "tier_of": -1, 2024-05-08T08:53:05.601 INFO:teuthology.orchestra.run.smithi059.stdout: "read_tier": -1, 2024-05-08T08:53:05.601 INFO:teuthology.orchestra.run.smithi059.stdout: "write_tier": -1, 2024-05-08T08:53:05.601 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_mode": "none", 2024-05-08T08:53:05.601 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_bytes": 0, 2024-05-08T08:53:05.601 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_objects": 0, 2024-05-08T08:53:05.601 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-08T08:53:05.602 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-08T08:53:05.602 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-08T08:53:05.602 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_flush_age": 0, 2024-05-08T08:53:05.602 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_evict_age": 0, 2024-05-08T08:53:05.602 INFO:teuthology.orchestra.run.smithi059.stdout: "erasure_code_profile": "", 2024-05-08T08:53:05.602 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_params": { 2024-05-08T08:53:05.602 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "none" 2024-05-08T08:53:05.602 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.603 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_period": 0, 2024-05-08T08:53:05.603 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_count": 0, 2024-05-08T08:53:05.603 INFO:teuthology.orchestra.run.smithi059.stdout: "use_gmt_hitset": true, 2024-05-08T08:53:05.603 INFO:teuthology.orchestra.run.smithi059.stdout: "min_read_recency_for_promote": 0, 2024-05-08T08:53:05.603 INFO:teuthology.orchestra.run.smithi059.stdout: "min_write_recency_for_promote": 0, 2024-05-08T08:53:05.603 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_grade_decay_rate": 0, 2024-05-08T08:53:05.603 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_search_last_n": 0, 2024-05-08T08:53:05.603 INFO:teuthology.orchestra.run.smithi059.stdout: "grade_table": [], 2024-05-08T08:53:05.603 INFO:teuthology.orchestra.run.smithi059.stdout: "stripe_width": 0, 2024-05-08T08:53:05.603 INFO:teuthology.orchestra.run.smithi059.stdout: "expected_num_objects": 0, 2024-05-08T08:53:05.603 INFO:teuthology.orchestra.run.smithi059.stdout: "fast_read": false, 2024-05-08T08:53:05.604 INFO:teuthology.orchestra.run.smithi059.stdout: "options": { 2024-05-08T08:53:05.604 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_max": 32, 2024-05-08T08:53:05.604 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_min": 1 2024-05-08T08:53:05.604 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.604 INFO:teuthology.orchestra.run.smithi059.stdout: "application_metadata": { 2024-05-08T08:53:05.604 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": {} 2024-05-08T08:53:05.604 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.604 INFO:teuthology.orchestra.run.smithi059.stdout: "read_balance": { 2024-05-08T08:53:05.604 INFO:teuthology.orchestra.run.smithi059.stdout: "score_type": "Fair distribution", 2024-05-08T08:53:05.604 INFO:teuthology.orchestra.run.smithi059.stdout: "score_acting": 6.059999942779541, 2024-05-08T08:53:05.604 INFO:teuthology.orchestra.run.smithi059.stdout: "score_stable": 6.059999942779541, 2024-05-08T08:53:05.604 INFO:teuthology.orchestra.run.smithi059.stdout: "optimal_score": 0.33000001311302185, 2024-05-08T08:53:05.605 INFO:teuthology.orchestra.run.smithi059.stdout: "raw_score_acting": 2, 2024-05-08T08:53:05.605 INFO:teuthology.orchestra.run.smithi059.stdout: "raw_score_stable": 2, 2024-05-08T08:53:05.605 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity_weighted": 0.87819164991378784, 2024-05-08T08:53:05.605 INFO:teuthology.orchestra.run.smithi059.stdout: "average_primary_affinity": 0.87999999523162842, 2024-05-08T08:53:05.605 INFO:teuthology.orchestra.run.smithi059.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-05-08T08:53:05.605 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:05.605 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.606 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.606 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 2, 2024-05-08T08:53:05.606 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_name": "rbd", 2024-05-08T08:53:05.606 INFO:teuthology.orchestra.run.smithi059.stdout: "create_time": "2024-05-08T08:52:02.563677+0000", 2024-05-08T08:53:05.606 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T08:53:05.606 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_names": "hashpspool", 2024-05-08T08:53:05.606 INFO:teuthology.orchestra.run.smithi059.stdout: "type": 1, 2024-05-08T08:53:05.606 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 2, 2024-05-08T08:53:05.606 INFO:teuthology.orchestra.run.smithi059.stdout: "min_size": 1, 2024-05-08T08:53:05.606 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_rule": 0, 2024-05-08T08:53:05.606 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_count": 0, 2024-05-08T08:53:05.606 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_target": 0, 2024-05-08T08:53:05.607 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_barrier": 0, 2024-05-08T08:53:05.607 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-08T08:53:05.607 INFO:teuthology.orchestra.run.smithi059.stdout: "object_hash": 2, 2024-05-08T08:53:05.607 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscale_mode": "off", 2024-05-08T08:53:05.607 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num": 8, 2024-05-08T08:53:05.607 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num": 8, 2024-05-08T08:53:05.607 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num_target": 8, 2024-05-08T08:53:05.607 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_target": 8, 2024-05-08T08:53:05.607 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_pending": 8, 2024-05-08T08:53:05.607 INFO:teuthology.orchestra.run.smithi059.stdout: "last_pg_merge_meta": { 2024-05-08T08:53:05.607 INFO:teuthology.orchestra.run.smithi059.stdout: "source_pgid": "0.0", 2024-05-08T08:53:05.608 INFO:teuthology.orchestra.run.smithi059.stdout: "ready_epoch": 0, 2024-05-08T08:53:05.608 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_started": 0, 2024-05-08T08:53:05.608 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_clean": 0, 2024-05-08T08:53:05.608 INFO:teuthology.orchestra.run.smithi059.stdout: "source_version": "0'0", 2024-05-08T08:53:05.608 INFO:teuthology.orchestra.run.smithi059.stdout: "target_version": "0'0" 2024-05-08T08:53:05.608 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.608 INFO:teuthology.orchestra.run.smithi059.stdout: "last_change": "17", 2024-05-08T08:53:05.608 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend": "0", 2024-05-08T08:53:05.609 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-08T08:53:05.609 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_preluminous": "0", 2024-05-08T08:53:05.609 INFO:teuthology.orchestra.run.smithi059.stdout: "auid": 0, 2024-05-08T08:53:05.609 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_mode": "selfmanaged", 2024-05-08T08:53:05.609 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_seq": 0, 2024-05-08T08:53:05.609 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_epoch": 0, 2024-05-08T08:53:05.609 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_snaps": [], 2024-05-08T08:53:05.609 INFO:teuthology.orchestra.run.smithi059.stdout: "removed_snaps": "[]", 2024-05-08T08:53:05.609 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_bytes": 0, 2024-05-08T08:53:05.609 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_objects": 0, 2024-05-08T08:53:05.609 INFO:teuthology.orchestra.run.smithi059.stdout: "tiers": [], 2024-05-08T08:53:05.609 INFO:teuthology.orchestra.run.smithi059.stdout: "tier_of": -1, 2024-05-08T08:53:05.609 INFO:teuthology.orchestra.run.smithi059.stdout: "read_tier": -1, 2024-05-08T08:53:05.610 INFO:teuthology.orchestra.run.smithi059.stdout: "write_tier": -1, 2024-05-08T08:53:05.610 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_mode": "none", 2024-05-08T08:53:05.610 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_bytes": 0, 2024-05-08T08:53:05.610 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_objects": 0, 2024-05-08T08:53:05.610 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-08T08:53:05.610 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-08T08:53:05.610 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-08T08:53:05.610 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_flush_age": 0, 2024-05-08T08:53:05.610 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_evict_age": 0, 2024-05-08T08:53:05.610 INFO:teuthology.orchestra.run.smithi059.stdout: "erasure_code_profile": "", 2024-05-08T08:53:05.611 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_params": { 2024-05-08T08:53:05.611 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "none" 2024-05-08T08:53:05.611 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.611 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_period": 0, 2024-05-08T08:53:05.611 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_count": 0, 2024-05-08T08:53:05.611 INFO:teuthology.orchestra.run.smithi059.stdout: "use_gmt_hitset": true, 2024-05-08T08:53:05.611 INFO:teuthology.orchestra.run.smithi059.stdout: "min_read_recency_for_promote": 0, 2024-05-08T08:53:05.611 INFO:teuthology.orchestra.run.smithi059.stdout: "min_write_recency_for_promote": 0, 2024-05-08T08:53:05.611 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_grade_decay_rate": 0, 2024-05-08T08:53:05.611 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_search_last_n": 0, 2024-05-08T08:53:05.611 INFO:teuthology.orchestra.run.smithi059.stdout: "grade_table": [], 2024-05-08T08:53:05.611 INFO:teuthology.orchestra.run.smithi059.stdout: "stripe_width": 0, 2024-05-08T08:53:05.611 INFO:teuthology.orchestra.run.smithi059.stdout: "expected_num_objects": 0, 2024-05-08T08:53:05.612 INFO:teuthology.orchestra.run.smithi059.stdout: "fast_read": false, 2024-05-08T08:53:05.612 INFO:teuthology.orchestra.run.smithi059.stdout: "options": {}, 2024-05-08T08:53:05.612 INFO:teuthology.orchestra.run.smithi059.stdout: "application_metadata": { 2024-05-08T08:53:05.612 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd": {} 2024-05-08T08:53:05.612 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.612 INFO:teuthology.orchestra.run.smithi059.stdout: "read_balance": { 2024-05-08T08:53:05.612 INFO:teuthology.orchestra.run.smithi059.stdout: "score_type": "Fair distribution", 2024-05-08T08:53:05.612 INFO:teuthology.orchestra.run.smithi059.stdout: "score_acting": 2.2699999809265137, 2024-05-08T08:53:05.612 INFO:teuthology.orchestra.run.smithi059.stdout: "score_stable": 2.2699999809265137, 2024-05-08T08:53:05.612 INFO:teuthology.orchestra.run.smithi059.stdout: "optimal_score": 0.82999998331069946, 2024-05-08T08:53:05.613 INFO:teuthology.orchestra.run.smithi059.stdout: "raw_score_acting": 1.8799999952316284, 2024-05-08T08:53:05.613 INFO:teuthology.orchestra.run.smithi059.stdout: "raw_score_stable": 1.8799999952316284, 2024-05-08T08:53:05.613 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity_weighted": 0.87819164991378784, 2024-05-08T08:53:05.613 INFO:teuthology.orchestra.run.smithi059.stdout: "average_primary_affinity": 0.87999999523162842, 2024-05-08T08:53:05.613 INFO:teuthology.orchestra.run.smithi059.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-05-08T08:53:05.613 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:05.613 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.613 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.613 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 3, 2024-05-08T08:53:05.613 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_name": "pool_client.0", 2024-05-08T08:53:05.613 INFO:teuthology.orchestra.run.smithi059.stdout: "create_time": "2024-05-08T08:52:09.955467+0000", 2024-05-08T08:53:05.613 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 8193, 2024-05-08T08:53:05.613 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-05-08T08:53:05.614 INFO:teuthology.orchestra.run.smithi059.stdout: "type": 1, 2024-05-08T08:53:05.614 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 2, 2024-05-08T08:53:05.614 INFO:teuthology.orchestra.run.smithi059.stdout: "min_size": 1, 2024-05-08T08:53:05.614 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_rule": 0, 2024-05-08T08:53:05.614 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_count": 0, 2024-05-08T08:53:05.614 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_target": 0, 2024-05-08T08:53:05.614 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_barrier": 0, 2024-05-08T08:53:05.614 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-08T08:53:05.614 INFO:teuthology.orchestra.run.smithi059.stdout: "object_hash": 2, 2024-05-08T08:53:05.614 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscale_mode": "off", 2024-05-08T08:53:05.614 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num": 32, 2024-05-08T08:53:05.614 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num": 32, 2024-05-08T08:53:05.614 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num_target": 32, 2024-05-08T08:53:05.614 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_target": 32, 2024-05-08T08:53:05.615 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_pending": 32, 2024-05-08T08:53:05.615 INFO:teuthology.orchestra.run.smithi059.stdout: "last_pg_merge_meta": { 2024-05-08T08:53:05.615 INFO:teuthology.orchestra.run.smithi059.stdout: "source_pgid": "0.0", 2024-05-08T08:53:05.615 INFO:teuthology.orchestra.run.smithi059.stdout: "ready_epoch": 0, 2024-05-08T08:53:05.615 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_started": 0, 2024-05-08T08:53:05.615 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_clean": 0, 2024-05-08T08:53:05.615 INFO:teuthology.orchestra.run.smithi059.stdout: "source_version": "0'0", 2024-05-08T08:53:05.615 INFO:teuthology.orchestra.run.smithi059.stdout: "target_version": "0'0" 2024-05-08T08:53:05.615 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.615 INFO:teuthology.orchestra.run.smithi059.stdout: "last_change": "59", 2024-05-08T08:53:05.615 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend": "0", 2024-05-08T08:53:05.615 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-08T08:53:05.615 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_preluminous": "0", 2024-05-08T08:53:05.616 INFO:teuthology.orchestra.run.smithi059.stdout: "auid": 0, 2024-05-08T08:53:05.616 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_mode": "selfmanaged", 2024-05-08T08:53:05.616 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_seq": 18, 2024-05-08T08:53:05.616 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_epoch": 59, 2024-05-08T08:53:05.616 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_snaps": [], 2024-05-08T08:53:05.616 INFO:teuthology.orchestra.run.smithi059.stdout: "removed_snaps": "[]", 2024-05-08T08:53:05.616 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_bytes": 0, 2024-05-08T08:53:05.616 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_objects": 0, 2024-05-08T08:53:05.617 INFO:teuthology.orchestra.run.smithi059.stdout: "tiers": [], 2024-05-08T08:53:05.617 INFO:teuthology.orchestra.run.smithi059.stdout: "tier_of": -1, 2024-05-08T08:53:05.617 INFO:teuthology.orchestra.run.smithi059.stdout: "read_tier": -1, 2024-05-08T08:53:05.617 INFO:teuthology.orchestra.run.smithi059.stdout: "write_tier": -1, 2024-05-08T08:53:05.617 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_mode": "none", 2024-05-08T08:53:05.617 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_bytes": 0, 2024-05-08T08:53:05.617 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_objects": 0, 2024-05-08T08:53:05.617 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-08T08:53:05.617 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-08T08:53:05.617 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-08T08:53:05.617 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_flush_age": 0, 2024-05-08T08:53:05.617 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_evict_age": 0, 2024-05-08T08:53:05.618 INFO:teuthology.orchestra.run.smithi059.stdout: "erasure_code_profile": "", 2024-05-08T08:53:05.618 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_params": { 2024-05-08T08:53:05.618 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "none" 2024-05-08T08:53:05.618 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.618 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_period": 0, 2024-05-08T08:53:05.618 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_count": 0, 2024-05-08T08:53:05.619 INFO:teuthology.orchestra.run.smithi059.stdout: "use_gmt_hitset": true, 2024-05-08T08:53:05.619 INFO:teuthology.orchestra.run.smithi059.stdout: "min_read_recency_for_promote": 0, 2024-05-08T08:53:05.619 INFO:teuthology.orchestra.run.smithi059.stdout: "min_write_recency_for_promote": 0, 2024-05-08T08:53:05.619 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_grade_decay_rate": 0, 2024-05-08T08:53:05.619 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_search_last_n": 0, 2024-05-08T08:53:05.619 INFO:teuthology.orchestra.run.smithi059.stdout: "grade_table": [], 2024-05-08T08:53:05.619 INFO:teuthology.orchestra.run.smithi059.stdout: "stripe_width": 0, 2024-05-08T08:53:05.619 INFO:teuthology.orchestra.run.smithi059.stdout: "expected_num_objects": 0, 2024-05-08T08:53:05.619 INFO:teuthology.orchestra.run.smithi059.stdout: "fast_read": false, 2024-05-08T08:53:05.619 INFO:teuthology.orchestra.run.smithi059.stdout: "options": {}, 2024-05-08T08:53:05.619 INFO:teuthology.orchestra.run.smithi059.stdout: "application_metadata": { 2024-05-08T08:53:05.620 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd": {} 2024-05-08T08:53:05.620 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.620 INFO:teuthology.orchestra.run.smithi059.stdout: "read_balance": { 2024-05-08T08:53:05.620 INFO:teuthology.orchestra.run.smithi059.stdout: "score_type": "Fair distribution", 2024-05-08T08:53:05.620 INFO:teuthology.orchestra.run.smithi059.stdout: "score_acting": 1.7000000476837158, 2024-05-08T08:53:05.620 INFO:teuthology.orchestra.run.smithi059.stdout: "score_stable": 1.7000000476837158, 2024-05-08T08:53:05.620 INFO:teuthology.orchestra.run.smithi059.stdout: "optimal_score": 0.82999998331069946, 2024-05-08T08:53:05.620 INFO:teuthology.orchestra.run.smithi059.stdout: "raw_score_acting": 1.4099999666213989, 2024-05-08T08:53:05.620 INFO:teuthology.orchestra.run.smithi059.stdout: "raw_score_stable": 1.4099999666213989, 2024-05-08T08:53:05.620 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity_weighted": 0.87819164991378784, 2024-05-08T08:53:05.620 INFO:teuthology.orchestra.run.smithi059.stdout: "average_primary_affinity": 0.87999999523162842, 2024-05-08T08:53:05.620 INFO:teuthology.orchestra.run.smithi059.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-05-08T08:53:05.620 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:05.621 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:05.621 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T08:53:05.621 INFO:teuthology.orchestra.run.smithi059.stdout: "osds": [ 2024-05-08T08:53:05.621 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.621 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 0, 2024-05-08T08:53:05.621 INFO:teuthology.orchestra.run.smithi059.stdout: "uuid": "300801a5-1919-4b23-8c6d-bc7dad61cc4d", 2024-05-08T08:53:05.621 INFO:teuthology.orchestra.run.smithi059.stdout: "up": 1, 2024-05-08T08:53:05.621 INFO:teuthology.orchestra.run.smithi059.stdout: "in": 1, 2024-05-08T08:53:05.621 INFO:teuthology.orchestra.run.smithi059.stdout: "weight": 1, 2024-05-08T08:53:05.621 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity": 1, 2024-05-08T08:53:05.621 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_begin": 0, 2024-05-08T08:53:05.621 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_end": 0, 2024-05-08T08:53:05.621 INFO:teuthology.orchestra.run.smithi059.stdout: "up_from": 11, 2024-05-08T08:53:05.622 INFO:teuthology.orchestra.run.smithi059.stdout: "up_thru": 49, 2024-05-08T08:53:05.622 INFO:teuthology.orchestra.run.smithi059.stdout: "down_at": 0, 2024-05-08T08:53:05.622 INFO:teuthology.orchestra.run.smithi059.stdout: "lost_at": 0, 2024-05-08T08:53:05.622 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrs": { 2024-05-08T08:53:05.622 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:53:05.622 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.622 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:53:05.622 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6800", 2024-05-08T08:53:05.622 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 802160151 2024-05-08T08:53:05.622 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.622 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.622 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:53:05.623 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6801", 2024-05-08T08:53:05.623 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 802160151 2024-05-08T08:53:05.623 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:05.623 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:05.623 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.623 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addrs": { 2024-05-08T08:53:05.623 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:53:05.623 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.623 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:53:05.623 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6802", 2024-05-08T08:53:05.623 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 802160151 2024-05-08T08:53:05.623 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.623 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.624 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:53:05.624 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6803", 2024-05-08T08:53:05.624 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 802160151 2024-05-08T08:53:05.624 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:05.624 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:05.624 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.624 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addrs": { 2024-05-08T08:53:05.624 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:53:05.624 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.624 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:53:05.624 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6806", 2024-05-08T08:53:05.624 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 802160151 2024-05-08T08:53:05.624 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.624 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.625 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:53:05.625 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6807", 2024-05-08T08:53:05.625 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 802160151 2024-05-08T08:53:05.625 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:05.625 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:05.625 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.625 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addrs": { 2024-05-08T08:53:05.625 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:53:05.625 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.625 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:53:05.625 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6804", 2024-05-08T08:53:05.625 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 802160151 2024-05-08T08:53:05.625 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.625 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.626 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:53:05.626 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6805", 2024-05-08T08:53:05.626 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 802160151 2024-05-08T08:53:05.626 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:05.626 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:05.626 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.626 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "172.21.15.59:6801/802160151", 2024-05-08T08:53:05.626 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "172.21.15.59:6803/802160151", 2024-05-08T08:53:05.626 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addr": "172.21.15.59:6807/802160151", 2024-05-08T08:53:05.626 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addr": "172.21.15.59:6805/802160151", 2024-05-08T08:53:05.627 INFO:teuthology.orchestra.run.smithi059.stdout: "state": [ 2024-05-08T08:53:05.627 INFO:teuthology.orchestra.run.smithi059.stdout: "exists", 2024-05-08T08:53:05.627 INFO:teuthology.orchestra.run.smithi059.stdout: "up" 2024-05-08T08:53:05.627 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:05.627 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.627 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.627 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 1, 2024-05-08T08:53:05.627 INFO:teuthology.orchestra.run.smithi059.stdout: "uuid": "0fead0de-6a65-4eba-b400-eea5692702e0", 2024-05-08T08:53:05.627 INFO:teuthology.orchestra.run.smithi059.stdout: "up": 1, 2024-05-08T08:53:05.627 INFO:teuthology.orchestra.run.smithi059.stdout: "in": 1, 2024-05-08T08:53:05.627 INFO:teuthology.orchestra.run.smithi059.stdout: "weight": 1, 2024-05-08T08:53:05.627 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity": 1, 2024-05-08T08:53:05.628 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_begin": 0, 2024-05-08T08:53:05.628 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_end": 0, 2024-05-08T08:53:05.628 INFO:teuthology.orchestra.run.smithi059.stdout: "up_from": 11, 2024-05-08T08:53:05.628 INFO:teuthology.orchestra.run.smithi059.stdout: "up_thru": 57, 2024-05-08T08:53:05.628 INFO:teuthology.orchestra.run.smithi059.stdout: "down_at": 0, 2024-05-08T08:53:05.628 INFO:teuthology.orchestra.run.smithi059.stdout: "lost_at": 0, 2024-05-08T08:53:05.628 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrs": { 2024-05-08T08:53:05.628 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:53:05.628 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.628 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:53:05.628 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6816", 2024-05-08T08:53:05.628 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4116801011 2024-05-08T08:53:05.629 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.629 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.629 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:53:05.629 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6817", 2024-05-08T08:53:05.629 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4116801011 2024-05-08T08:53:05.629 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:05.629 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:05.629 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.629 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addrs": { 2024-05-08T08:53:05.629 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:53:05.629 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.629 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:53:05.629 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6818", 2024-05-08T08:53:05.630 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4116801011 2024-05-08T08:53:05.630 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.630 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.630 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:53:05.630 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6819", 2024-05-08T08:53:05.630 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4116801011 2024-05-08T08:53:05.631 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:05.631 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:05.631 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.631 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addrs": { 2024-05-08T08:53:05.631 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:53:05.631 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.631 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:53:05.631 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6822", 2024-05-08T08:53:05.631 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4116801011 2024-05-08T08:53:05.631 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.631 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.631 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:53:05.632 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6823", 2024-05-08T08:53:05.632 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4116801011 2024-05-08T08:53:05.632 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:05.632 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:05.632 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.632 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addrs": { 2024-05-08T08:53:05.632 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:53:05.632 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.632 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:53:05.632 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6820", 2024-05-08T08:53:05.632 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4116801011 2024-05-08T08:53:05.632 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.632 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.633 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:53:05.633 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6821", 2024-05-08T08:53:05.633 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4116801011 2024-05-08T08:53:05.633 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:05.633 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:05.633 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.633 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "172.21.15.59:6817/4116801011", 2024-05-08T08:53:05.633 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "172.21.15.59:6819/4116801011", 2024-05-08T08:53:05.633 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addr": "172.21.15.59:6823/4116801011", 2024-05-08T08:53:05.633 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addr": "172.21.15.59:6821/4116801011", 2024-05-08T08:53:05.633 INFO:teuthology.orchestra.run.smithi059.stdout: "state": [ 2024-05-08T08:53:05.633 INFO:teuthology.orchestra.run.smithi059.stdout: "exists", 2024-05-08T08:53:05.633 INFO:teuthology.orchestra.run.smithi059.stdout: "up" 2024-05-08T08:53:05.634 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:05.634 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.634 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.634 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 2, 2024-05-08T08:53:05.634 INFO:teuthology.orchestra.run.smithi059.stdout: "uuid": "1a233de6-1b82-4a51-8e96-4cd4abdf16c8", 2024-05-08T08:53:05.634 INFO:teuthology.orchestra.run.smithi059.stdout: "up": 1, 2024-05-08T08:53:05.634 INFO:teuthology.orchestra.run.smithi059.stdout: "in": 1, 2024-05-08T08:53:05.634 INFO:teuthology.orchestra.run.smithi059.stdout: "weight": 1, 2024-05-08T08:53:05.634 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity": 1, 2024-05-08T08:53:05.634 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_begin": 0, 2024-05-08T08:53:05.634 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_end": 0, 2024-05-08T08:53:05.634 INFO:teuthology.orchestra.run.smithi059.stdout: "up_from": 11, 2024-05-08T08:53:05.634 INFO:teuthology.orchestra.run.smithi059.stdout: "up_thru": 50, 2024-05-08T08:53:05.634 INFO:teuthology.orchestra.run.smithi059.stdout: "down_at": 0, 2024-05-08T08:53:05.635 INFO:teuthology.orchestra.run.smithi059.stdout: "lost_at": 0, 2024-05-08T08:53:05.635 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrs": { 2024-05-08T08:53:05.635 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:53:05.635 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.635 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:53:05.635 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6808", 2024-05-08T08:53:05.635 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1547661179 2024-05-08T08:53:05.635 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.635 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.635 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:53:05.635 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6809", 2024-05-08T08:53:05.635 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1547661179 2024-05-08T08:53:05.635 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:05.636 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:05.636 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.636 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addrs": { 2024-05-08T08:53:05.636 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:53:05.636 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.636 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:53:05.636 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6810", 2024-05-08T08:53:05.636 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1547661179 2024-05-08T08:53:05.636 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.636 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.636 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:53:05.636 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6811", 2024-05-08T08:53:05.636 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1547661179 2024-05-08T08:53:05.637 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:05.637 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:05.637 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.637 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addrs": { 2024-05-08T08:53:05.637 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:53:05.637 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.638 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:53:05.638 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6814", 2024-05-08T08:53:05.638 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1547661179 2024-05-08T08:53:05.638 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.638 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.638 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:53:05.638 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6815", 2024-05-08T08:53:05.638 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1547661179 2024-05-08T08:53:05.638 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:05.638 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:05.638 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.638 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addrs": { 2024-05-08T08:53:05.638 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:53:05.639 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.639 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:53:05.639 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6812", 2024-05-08T08:53:05.639 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1547661179 2024-05-08T08:53:05.639 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.639 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.639 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:53:05.639 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6813", 2024-05-08T08:53:05.639 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1547661179 2024-05-08T08:53:05.639 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:05.639 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:05.639 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.639 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "172.21.15.59:6809/1547661179", 2024-05-08T08:53:05.640 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "172.21.15.59:6811/1547661179", 2024-05-08T08:53:05.640 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addr": "172.21.15.59:6815/1547661179", 2024-05-08T08:53:05.640 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addr": "172.21.15.59:6813/1547661179", 2024-05-08T08:53:05.640 INFO:teuthology.orchestra.run.smithi059.stdout: "state": [ 2024-05-08T08:53:05.640 INFO:teuthology.orchestra.run.smithi059.stdout: "exists", 2024-05-08T08:53:05.640 INFO:teuthology.orchestra.run.smithi059.stdout: "up" 2024-05-08T08:53:05.640 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:05.640 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.640 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.640 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 3, 2024-05-08T08:53:05.640 INFO:teuthology.orchestra.run.smithi059.stdout: "uuid": "cc4870b8-dcd9-4e43-978d-8de6c6396abf", 2024-05-08T08:53:05.640 INFO:teuthology.orchestra.run.smithi059.stdout: "up": 1, 2024-05-08T08:53:05.640 INFO:teuthology.orchestra.run.smithi059.stdout: "in": 1, 2024-05-08T08:53:05.641 INFO:teuthology.orchestra.run.smithi059.stdout: "weight": 1, 2024-05-08T08:53:05.641 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity": 1, 2024-05-08T08:53:05.641 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_begin": 0, 2024-05-08T08:53:05.641 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_end": 0, 2024-05-08T08:53:05.641 INFO:teuthology.orchestra.run.smithi059.stdout: "up_from": 11, 2024-05-08T08:53:05.641 INFO:teuthology.orchestra.run.smithi059.stdout: "up_thru": 44, 2024-05-08T08:53:05.641 INFO:teuthology.orchestra.run.smithi059.stdout: "down_at": 0, 2024-05-08T08:53:05.641 INFO:teuthology.orchestra.run.smithi059.stdout: "lost_at": 0, 2024-05-08T08:53:05.641 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrs": { 2024-05-08T08:53:05.641 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:53:05.641 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.641 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:53:05.641 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6800", 2024-05-08T08:53:05.642 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4054400810 2024-05-08T08:53:05.642 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.642 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.642 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:53:05.642 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6801", 2024-05-08T08:53:05.642 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4054400810 2024-05-08T08:53:05.642 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:05.642 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:05.642 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.642 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addrs": { 2024-05-08T08:53:05.642 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:53:05.642 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.643 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:53:05.643 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6802", 2024-05-08T08:53:05.643 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4054400810 2024-05-08T08:53:05.643 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.643 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.643 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:53:05.643 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6803", 2024-05-08T08:53:05.643 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4054400810 2024-05-08T08:53:05.643 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:05.643 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:05.643 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.643 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addrs": { 2024-05-08T08:53:05.643 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:53:05.644 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.644 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:53:05.644 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6806", 2024-05-08T08:53:05.644 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4054400810 2024-05-08T08:53:05.644 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.644 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.644 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:53:05.644 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6807", 2024-05-08T08:53:05.644 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4054400810 2024-05-08T08:53:05.644 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:05.645 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:05.645 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.645 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addrs": { 2024-05-08T08:53:05.645 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:53:05.645 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.645 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:53:05.645 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6804", 2024-05-08T08:53:05.645 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4054400810 2024-05-08T08:53:05.645 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.645 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.645 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:53:05.645 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6805", 2024-05-08T08:53:05.645 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4054400810 2024-05-08T08:53:05.646 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:05.646 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:05.646 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.646 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "172.21.15.192:6801/4054400810", 2024-05-08T08:53:05.646 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "172.21.15.192:6803/4054400810", 2024-05-08T08:53:05.646 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addr": "172.21.15.192:6807/4054400810", 2024-05-08T08:53:05.646 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addr": "172.21.15.192:6805/4054400810", 2024-05-08T08:53:05.646 INFO:teuthology.orchestra.run.smithi059.stdout: "state": [ 2024-05-08T08:53:05.646 INFO:teuthology.orchestra.run.smithi059.stdout: "exists", 2024-05-08T08:53:05.646 INFO:teuthology.orchestra.run.smithi059.stdout: "up" 2024-05-08T08:53:05.646 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:05.646 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.647 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.647 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 4, 2024-05-08T08:53:05.647 INFO:teuthology.orchestra.run.smithi059.stdout: "uuid": "d1d05620-42fb-4107-a2f9-e83542d076ee", 2024-05-08T08:53:05.647 INFO:teuthology.orchestra.run.smithi059.stdout: "up": 1, 2024-05-08T08:53:05.647 INFO:teuthology.orchestra.run.smithi059.stdout: "in": 0, 2024-05-08T08:53:05.647 INFO:teuthology.orchestra.run.smithi059.stdout: "weight": 0, 2024-05-08T08:53:05.647 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity": 0.9892578125, 2024-05-08T08:53:05.647 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_begin": 0, 2024-05-08T08:53:05.647 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_end": 0, 2024-05-08T08:53:05.647 INFO:teuthology.orchestra.run.smithi059.stdout: "up_from": 11, 2024-05-08T08:53:05.647 INFO:teuthology.orchestra.run.smithi059.stdout: "up_thru": 43, 2024-05-08T08:53:05.648 INFO:teuthology.orchestra.run.smithi059.stdout: "down_at": 0, 2024-05-08T08:53:05.648 INFO:teuthology.orchestra.run.smithi059.stdout: "lost_at": 0, 2024-05-08T08:53:05.648 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrs": { 2024-05-08T08:53:05.648 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:53:05.648 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.648 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:53:05.648 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6808", 2024-05-08T08:53:05.648 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 58985565 2024-05-08T08:53:05.648 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.649 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.649 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:53:05.649 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6809", 2024-05-08T08:53:05.649 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 58985565 2024-05-08T08:53:05.649 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:05.649 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:05.649 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.649 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addrs": { 2024-05-08T08:53:05.649 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:53:05.649 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.649 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:53:05.649 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6810", 2024-05-08T08:53:05.649 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 58985565 2024-05-08T08:53:05.650 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.650 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.650 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:53:05.650 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6811", 2024-05-08T08:53:05.650 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 58985565 2024-05-08T08:53:05.650 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:05.650 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:05.650 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.650 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addrs": { 2024-05-08T08:53:05.650 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:53:05.650 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.650 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:53:05.651 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6814", 2024-05-08T08:53:05.651 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 58985565 2024-05-08T08:53:05.651 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.651 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.651 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:53:05.651 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6815", 2024-05-08T08:53:05.651 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 58985565 2024-05-08T08:53:05.651 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:05.651 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:05.651 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.651 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addrs": { 2024-05-08T08:53:05.651 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:53:05.651 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.652 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:53:05.652 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6812", 2024-05-08T08:53:05.652 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 58985565 2024-05-08T08:53:05.652 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.652 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.652 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:53:05.652 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6813", 2024-05-08T08:53:05.652 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 58985565 2024-05-08T08:53:05.652 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:05.652 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:05.652 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.652 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "172.21.15.192:6809/58985565", 2024-05-08T08:53:05.652 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "172.21.15.192:6811/58985565", 2024-05-08T08:53:05.652 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addr": "172.21.15.192:6815/58985565", 2024-05-08T08:53:05.652 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addr": "172.21.15.192:6813/58985565", 2024-05-08T08:53:05.653 INFO:teuthology.orchestra.run.smithi059.stdout: "state": [ 2024-05-08T08:53:05.653 INFO:teuthology.orchestra.run.smithi059.stdout: "exists", 2024-05-08T08:53:05.653 INFO:teuthology.orchestra.run.smithi059.stdout: "up" 2024-05-08T08:53:05.653 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:05.653 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.653 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.653 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 5, 2024-05-08T08:53:05.653 INFO:teuthology.orchestra.run.smithi059.stdout: "uuid": "ab26136c-31c3-4090-9fa2-988dba4f731f", 2024-05-08T08:53:05.653 INFO:teuthology.orchestra.run.smithi059.stdout: "up": 1, 2024-05-08T08:53:05.653 INFO:teuthology.orchestra.run.smithi059.stdout: "in": 1, 2024-05-08T08:53:05.653 INFO:teuthology.orchestra.run.smithi059.stdout: "weight": 1, 2024-05-08T08:53:05.653 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity": 0.2798919677734375, 2024-05-08T08:53:05.653 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_begin": 0, 2024-05-08T08:53:05.653 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_end": 0, 2024-05-08T08:53:05.654 INFO:teuthology.orchestra.run.smithi059.stdout: "up_from": 11, 2024-05-08T08:53:05.654 INFO:teuthology.orchestra.run.smithi059.stdout: "up_thru": 18, 2024-05-08T08:53:05.654 INFO:teuthology.orchestra.run.smithi059.stdout: "down_at": 0, 2024-05-08T08:53:05.654 INFO:teuthology.orchestra.run.smithi059.stdout: "lost_at": 0, 2024-05-08T08:53:05.654 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrs": { 2024-05-08T08:53:05.654 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:53:05.654 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.654 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:53:05.654 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6816", 2024-05-08T08:53:05.654 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1283571573 2024-05-08T08:53:05.654 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.654 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.655 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:53:05.655 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6817", 2024-05-08T08:53:05.655 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1283571573 2024-05-08T08:53:05.655 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:05.655 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:05.655 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.655 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addrs": { 2024-05-08T08:53:05.655 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:53:05.655 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.655 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:53:05.655 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6818", 2024-05-08T08:53:05.655 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1283571573 2024-05-08T08:53:05.656 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.656 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.656 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:53:05.656 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6819", 2024-05-08T08:53:05.656 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1283571573 2024-05-08T08:53:05.656 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:05.656 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:05.656 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.656 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addrs": { 2024-05-08T08:53:05.656 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:53:05.656 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.656 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:53:05.656 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6822", 2024-05-08T08:53:05.657 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1283571573 2024-05-08T08:53:05.657 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.657 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.657 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:53:05.657 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6823", 2024-05-08T08:53:05.657 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1283571573 2024-05-08T08:53:05.657 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:05.657 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:05.657 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.657 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addrs": { 2024-05-08T08:53:05.657 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:53:05.657 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.657 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:53:05.657 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6820", 2024-05-08T08:53:05.658 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1283571573 2024-05-08T08:53:05.658 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.658 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.658 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:53:05.658 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6821", 2024-05-08T08:53:05.658 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1283571573 2024-05-08T08:53:05.658 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:05.658 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:05.659 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.659 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "172.21.15.192:6817/1283571573", 2024-05-08T08:53:05.659 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "172.21.15.192:6819/1283571573", 2024-05-08T08:53:05.659 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addr": "172.21.15.192:6823/1283571573", 2024-05-08T08:53:05.659 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addr": "172.21.15.192:6821/1283571573", 2024-05-08T08:53:05.659 INFO:teuthology.orchestra.run.smithi059.stdout: "state": [ 2024-05-08T08:53:05.659 INFO:teuthology.orchestra.run.smithi059.stdout: "exists", 2024-05-08T08:53:05.659 INFO:teuthology.orchestra.run.smithi059.stdout: "up" 2024-05-08T08:53:05.659 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:05.659 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:05.659 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T08:53:05.659 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_xinfo": [ 2024-05-08T08:53:05.660 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.660 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 0, 2024-05-08T08:53:05.660 INFO:teuthology.orchestra.run.smithi059.stdout: "down_stamp": "0.000000", 2024-05-08T08:53:05.660 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_probability": 0, 2024-05-08T08:53:05.660 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_interval": 0, 2024-05-08T08:53:05.660 INFO:teuthology.orchestra.run.smithi059.stdout: "features": 4540701547738038271, 2024-05-08T08:53:05.660 INFO:teuthology.orchestra.run.smithi059.stdout: "old_weight": 0, 2024-05-08T08:53:05.660 INFO:teuthology.orchestra.run.smithi059.stdout: "last_purged_snaps_scrub": "2024-05-08T08:51:52.396946+0000", 2024-05-08T08:53:05.660 INFO:teuthology.orchestra.run.smithi059.stdout: "dead_epoch": 0 2024-05-08T08:53:05.660 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.660 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.660 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 1, 2024-05-08T08:53:05.661 INFO:teuthology.orchestra.run.smithi059.stdout: "down_stamp": "0.000000", 2024-05-08T08:53:05.661 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_probability": 0, 2024-05-08T08:53:05.661 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_interval": 0, 2024-05-08T08:53:05.661 INFO:teuthology.orchestra.run.smithi059.stdout: "features": 4540701547738038271, 2024-05-08T08:53:05.661 INFO:teuthology.orchestra.run.smithi059.stdout: "old_weight": 0, 2024-05-08T08:53:05.661 INFO:teuthology.orchestra.run.smithi059.stdout: "last_purged_snaps_scrub": "2024-05-08T08:51:52.453160+0000", 2024-05-08T08:53:05.661 INFO:teuthology.orchestra.run.smithi059.stdout: "dead_epoch": 0 2024-05-08T08:53:05.661 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.661 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.661 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 2, 2024-05-08T08:53:05.661 INFO:teuthology.orchestra.run.smithi059.stdout: "down_stamp": "0.000000", 2024-05-08T08:53:05.661 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_probability": 0, 2024-05-08T08:53:05.661 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_interval": 0, 2024-05-08T08:53:05.662 INFO:teuthology.orchestra.run.smithi059.stdout: "features": 4540701547738038271, 2024-05-08T08:53:05.662 INFO:teuthology.orchestra.run.smithi059.stdout: "old_weight": 0, 2024-05-08T08:53:05.662 INFO:teuthology.orchestra.run.smithi059.stdout: "last_purged_snaps_scrub": "2024-05-08T08:51:52.387606+0000", 2024-05-08T08:53:05.662 INFO:teuthology.orchestra.run.smithi059.stdout: "dead_epoch": 0 2024-05-08T08:53:05.662 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.662 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.662 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 3, 2024-05-08T08:53:05.662 INFO:teuthology.orchestra.run.smithi059.stdout: "down_stamp": "0.000000", 2024-05-08T08:53:05.662 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_probability": 0, 2024-05-08T08:53:05.662 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_interval": 0, 2024-05-08T08:53:05.662 INFO:teuthology.orchestra.run.smithi059.stdout: "features": 4540701547738038271, 2024-05-08T08:53:05.662 INFO:teuthology.orchestra.run.smithi059.stdout: "old_weight": 0, 2024-05-08T08:53:05.662 INFO:teuthology.orchestra.run.smithi059.stdout: "last_purged_snaps_scrub": "2024-05-08T08:51:52.360952+0000", 2024-05-08T08:53:05.663 INFO:teuthology.orchestra.run.smithi059.stdout: "dead_epoch": 0 2024-05-08T08:53:05.663 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.663 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.663 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 4, 2024-05-08T08:53:05.663 INFO:teuthology.orchestra.run.smithi059.stdout: "down_stamp": "0.000000", 2024-05-08T08:53:05.663 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_probability": 0, 2024-05-08T08:53:05.663 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_interval": 0, 2024-05-08T08:53:05.663 INFO:teuthology.orchestra.run.smithi059.stdout: "features": 4540701547738038271, 2024-05-08T08:53:05.663 INFO:teuthology.orchestra.run.smithi059.stdout: "old_weight": 65536, 2024-05-08T08:53:05.663 INFO:teuthology.orchestra.run.smithi059.stdout: "last_purged_snaps_scrub": "2024-05-08T08:51:52.468681+0000", 2024-05-08T08:53:05.663 INFO:teuthology.orchestra.run.smithi059.stdout: "dead_epoch": 0 2024-05-08T08:53:05.663 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.664 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.664 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 5, 2024-05-08T08:53:05.664 INFO:teuthology.orchestra.run.smithi059.stdout: "down_stamp": "0.000000", 2024-05-08T08:53:05.664 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_probability": 0, 2024-05-08T08:53:05.664 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_interval": 0, 2024-05-08T08:53:05.664 INFO:teuthology.orchestra.run.smithi059.stdout: "features": 4540701547738038271, 2024-05-08T08:53:05.664 INFO:teuthology.orchestra.run.smithi059.stdout: "old_weight": 0, 2024-05-08T08:53:05.664 INFO:teuthology.orchestra.run.smithi059.stdout: "last_purged_snaps_scrub": "2024-05-08T08:51:52.401080+0000", 2024-05-08T08:53:05.664 INFO:teuthology.orchestra.run.smithi059.stdout: "dead_epoch": 0 2024-05-08T08:53:05.664 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:05.664 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T08:53:05.664 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_upmap": [ 2024-05-08T08:53:05.664 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.664 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "3.4", 2024-05-08T08:53:05.665 INFO:teuthology.orchestra.run.smithi059.stdout: "osds": [ 2024-05-08T08:53:05.665 INFO:teuthology.orchestra.run.smithi059.stdout: 1, 2024-05-08T08:53:05.665 INFO:teuthology.orchestra.run.smithi059.stdout: 5 2024-05-08T08:53:05.665 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:05.665 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.665 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.665 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "3.17", 2024-05-08T08:53:05.665 INFO:teuthology.orchestra.run.smithi059.stdout: "osds": [ 2024-05-08T08:53:05.665 INFO:teuthology.orchestra.run.smithi059.stdout: 2, 2024-05-08T08:53:05.665 INFO:teuthology.orchestra.run.smithi059.stdout: 3 2024-05-08T08:53:05.665 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:05.665 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.665 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:05.665 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "3.1d", 2024-05-08T08:53:05.665 INFO:teuthology.orchestra.run.smithi059.stdout: "osds": [ 2024-05-08T08:53:05.666 INFO:teuthology.orchestra.run.smithi059.stdout: 0, 2024-05-08T08:53:05.666 INFO:teuthology.orchestra.run.smithi059.stdout: 5 2024-05-08T08:53:05.666 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:05.666 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:05.666 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T08:53:05.666 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_upmap_items": [], 2024-05-08T08:53:05.666 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_upmap_primaries": [], 2024-05-08T08:53:05.666 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_temp": [], 2024-05-08T08:53:05.666 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_temp": [], 2024-05-08T08:53:05.666 INFO:teuthology.orchestra.run.smithi059.stdout: "blocklist": {}, 2024-05-08T08:53:05.666 INFO:teuthology.orchestra.run.smithi059.stdout: "range_blocklist": {}, 2024-05-08T08:53:05.666 INFO:teuthology.orchestra.run.smithi059.stdout: "erasure_code_profiles": { 2024-05-08T08:53:05.667 INFO:teuthology.orchestra.run.smithi059.stdout: "default": { 2024-05-08T08:53:05.667 INFO:teuthology.orchestra.run.smithi059.stdout: "crush-failure-domain": "osd", 2024-05-08T08:53:05.667 INFO:teuthology.orchestra.run.smithi059.stdout: "k": "2", 2024-05-08T08:53:05.667 INFO:teuthology.orchestra.run.smithi059.stdout: "m": "1", 2024-05-08T08:53:05.667 INFO:teuthology.orchestra.run.smithi059.stdout: "plugin": "jerasure", 2024-05-08T08:53:05.667 INFO:teuthology.orchestra.run.smithi059.stdout: "technique": "reed_sol_van" 2024-05-08T08:53:05.667 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:05.667 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:05.667 INFO:teuthology.orchestra.run.smithi059.stdout: "removed_snaps_queue": [], 2024-05-08T08:53:05.667 INFO:teuthology.orchestra.run.smithi059.stdout: "new_removed_snaps": [], 2024-05-08T08:53:05.667 INFO:teuthology.orchestra.run.smithi059.stdout: "new_purged_snaps": [], 2024-05-08T08:53:05.667 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_node_flags": {}, 2024-05-08T08:53:05.667 INFO:teuthology.orchestra.run.smithi059.stdout: "device_class_flags": {}, 2024-05-08T08:53:05.667 INFO:teuthology.orchestra.run.smithi059.stdout: "stretch_mode": { 2024-05-08T08:53:05.667 INFO:teuthology.orchestra.run.smithi059.stdout: "stretch_mode_enabled": false, 2024-05-08T08:53:05.668 INFO:teuthology.orchestra.run.smithi059.stdout: "stretch_bucket_count": 0, 2024-05-08T08:53:05.668 INFO:teuthology.orchestra.run.smithi059.stdout: "degraded_stretch_mode": 0, 2024-05-08T08:53:05.668 INFO:teuthology.orchestra.run.smithi059.stdout: "recovering_stretch_mode": 0, 2024-05-08T08:53:05.668 INFO:teuthology.orchestra.run.smithi059.stdout: "stretch_mode_bucket": 0 2024-05-08T08:53:05.668 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:05.668 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:05.670 INFO:teuthology.orchestra.run.smithi196.stdout:2687 read 0x1437e00 thru 0x143dcbb (0x5ebc bytes) 2024-05-08T08:53:05.670 INFO:teuthology.orchestra.run.smithi196.stdout:2688 punch from 0x48e00 to 0x4c400, (0x3600 bytes) 2024-05-08T08:53:05.671 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:05.671 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:05.672 INFO:tasks.thrashosds.thrasher:j is {'epoch': 60, 'fsid': '38dfae8c-5c5e-4b0f-a264-91f22c6c5554', 'created': '2024-05-08T08:51:45.571369+0000', 'modified': '2024-05-08T08:53:05.367783+0000', 'last_up_change': '2024-05-08T08:51:56.847706+0000', 'last_in_change': '2024-05-08T08:52:46.045870+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T08:51:57.787790+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '14', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.8781916499137878, 'average_primary_affinity': 0.8799999952316284, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-05-08T08:52:02.563677+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.2699999809265137, 'score_stable': 2.2699999809265137, 'optimal_score': 0.8299999833106995, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.8781916499137878, 'average_primary_affinity': 0.8799999952316284, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-05-08T08:52:09.955467+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 18, 'snap_epoch': 59, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.7000000476837158, 'score_stable': 1.7000000476837158, 'optimal_score': 0.8299999833106995, 'raw_score_acting': 1.409999966621399, 'raw_score_stable': 1.409999966621399, 'primary_affinity_weighted': 0.8781916499137878, 'average_primary_affinity': 0.8799999952316284, 'average_primary_affinity_weighted': 0.8799999952316284}}], 'osds': [{'osd': 0, 'uuid': '300801a5-1919-4b23-8c6d-bc7dad61cc4d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 49, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6800', 'nonce': 802160151}, {'type': 'v1', 'addr': '172.21.15.59:6801', 'nonce': 802160151}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6802', 'nonce': 802160151}, {'type': 'v1', 'addr': '172.21.15.59:6803', 'nonce': 802160151}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6806', 'nonce': 802160151}, {'type': 'v1', 'addr': '172.21.15.59:6807', 'nonce': 802160151}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6804', 'nonce': 802160151}, {'type': 'v1', 'addr': '172.21.15.59:6805', 'nonce': 802160151}]}, 'public_addr': '172.21.15.59:6801/802160151', 'cluster_addr': '172.21.15.59:6803/802160151', 'heartbeat_back_addr': '172.21.15.59:6807/802160151', 'heartbeat_front_addr': '172.21.15.59:6805/802160151', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '0fead0de-6a65-4eba-b400-eea5692702e0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 57, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6816', 'nonce': 4116801011}, {'type': 'v1', 'addr': '172.21.15.59:6817', 'nonce': 4116801011}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6818', 'nonce': 4116801011}, {'type': 'v1', 'addr': '172.21.15.59:6819', 'nonce': 4116801011}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6822', 'nonce': 4116801011}, {'type': 'v1', 'addr': '172.21.15.59:6823', 'nonce': 4116801011}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6820', 'nonce': 4116801011}, {'type': 'v1', 'addr': '172.21.15.59:6821', 'nonce': 4116801011}]}, 'public_addr': '172.21.15.59:6817/4116801011', 'cluster_addr': '172.21.15.59:6819/4116801011', 'heartbeat_back_addr': '172.21.15.59:6823/4116801011', 'heartbeat_front_addr': '172.21.15.59:6821/4116801011', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '1a233de6-1b82-4a51-8e96-4cd4abdf16c8', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 50, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6808', 'nonce': 1547661179}, {'type': 'v1', 'addr': '172.21.15.59:6809', 'nonce': 1547661179}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6810', 'nonce': 1547661179}, {'type': 'v1', 'addr': '172.21.15.59:6811', 'nonce': 1547661179}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6814', 'nonce': 1547661179}, {'type': 'v1', 'addr': '172.21.15.59:6815', 'nonce': 1547661179}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6812', 'nonce': 1547661179}, {'type': 'v1', 'addr': '172.21.15.59:6813', 'nonce': 1547661179}]}, 'public_addr': '172.21.15.59:6809/1547661179', 'cluster_addr': '172.21.15.59:6811/1547661179', 'heartbeat_back_addr': '172.21.15.59:6815/1547661179', 'heartbeat_front_addr': '172.21.15.59:6813/1547661179', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'cc4870b8-dcd9-4e43-978d-8de6c6396abf', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 44, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6800', 'nonce': 4054400810}, {'type': 'v1', 'addr': '172.21.15.192:6801', 'nonce': 4054400810}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6802', 'nonce': 4054400810}, {'type': 'v1', 'addr': '172.21.15.192:6803', 'nonce': 4054400810}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6806', 'nonce': 4054400810}, {'type': 'v1', 'addr': '172.21.15.192:6807', 'nonce': 4054400810}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6804', 'nonce': 4054400810}, {'type': 'v1', 'addr': '172.21.15.192:6805', 'nonce': 4054400810}]}, 'public_addr': '172.21.15.192:6801/4054400810', 'cluster_addr': '172.21.15.192:6803/4054400810', 'heartbeat_back_addr': '172.21.15.192:6807/4054400810', 'heartbeat_front_addr': '172.21.15.192:6805/4054400810', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'd1d05620-42fb-4107-a2f9-e83542d076ee', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 0.9892578125, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 43, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6808', 'nonce': 58985565}, {'type': 'v1', 'addr': '172.21.15.192:6809', 'nonce': 58985565}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6810', 'nonce': 58985565}, {'type': 'v1', 'addr': '172.21.15.192:6811', 'nonce': 58985565}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6814', 'nonce': 58985565}, {'type': 'v1', 'addr': '172.21.15.192:6815', 'nonce': 58985565}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6812', 'nonce': 58985565}, {'type': 'v1', 'addr': '172.21.15.192:6813', 'nonce': 58985565}]}, 'public_addr': '172.21.15.192:6809/58985565', 'cluster_addr': '172.21.15.192:6811/58985565', 'heartbeat_back_addr': '172.21.15.192:6815/58985565', 'heartbeat_front_addr': '172.21.15.192:6813/58985565', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'ab26136c-31c3-4090-9fa2-988dba4f731f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.2798919677734375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 18, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6816', 'nonce': 1283571573}, {'type': 'v1', 'addr': '172.21.15.192:6817', 'nonce': 1283571573}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6818', 'nonce': 1283571573}, {'type': 'v1', 'addr': '172.21.15.192:6819', 'nonce': 1283571573}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6822', 'nonce': 1283571573}, {'type': 'v1', 'addr': '172.21.15.192:6823', 'nonce': 1283571573}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6820', 'nonce': 1283571573}, {'type': 'v1', 'addr': '172.21.15.192:6821', 'nonce': 1283571573}]}, 'public_addr': '172.21.15.192:6817/1283571573', 'cluster_addr': '172.21.15.192:6819/1283571573', 'heartbeat_back_addr': '172.21.15.192:6823/1283571573', 'heartbeat_front_addr': '172.21.15.192:6821/1283571573', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:51:52.396946+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:51:52.453160+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:51:52.387606+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:51:52.360952+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-05-08T08:51:52.468681+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:51:52.401080+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.4', 'osds': [1, 5]}, {'pgid': '3.17', 'osds': [2, 3]}, {'pgid': '3.1d', 'osds': [0, 5]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-05-08T08:53:05.672 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T08:53:05.673 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:53:05.673 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:05.672+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:05.694 INFO:teuthology.orchestra.run.smithi196.stdout:2690 write 0x3668a00 thru 0x3677a1e (0xf01f bytes) 2024-05-08T08:53:05.715 INFO:teuthology.orchestra.run.smithi196.stdout:2691 write 0x6ff8e00 thru 0x7004dcb (0xbfcc bytes) 2024-05-08T08:53:05.715 INFO:teuthology.orchestra.run.smithi196.stdout:2692 write 0x47aba00 thru 0x47afd60 (0x4361 bytes) 2024-05-08T08:53:05.722 INFO:teuthology.orchestra.run.smithi196.stdout:2693 trunc from 0x7004e00 to 0xd34c400 2024-05-08T08:53:05.746 INFO:teuthology.orchestra.run.smithi196.stdout:2694 write 0x8b6ea00 thru 0x8b761c6 (0x77c7 bytes) 2024-05-08T08:53:05.746 INFO:teuthology.orchestra.run.smithi196.stdout:2695 read 0xe53400 thru 0xe54155 (0xd56 bytes) 2024-05-08T08:53:05.747 INFO:teuthology.orchestra.run.smithi196.stdout:2696 read 0x41bc800 thru 0x41c6739 (0x9f3a bytes) 2024-05-08T08:53:05.755 INFO:teuthology.orchestra.run.smithi196.stdout:2697 trunc from 0xd34c400 to 0x6c28600 2024-05-08T08:53:05.770 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:05.773+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:05.784 INFO:teuthology.orchestra.run.smithi196.stdout:2698 trunc from 0x6c28600 to 0x717400 2024-05-08T08:53:05.815 INFO:teuthology.orchestra.run.smithi196.stdout:2700 punch from 0x6dc200 to 0x6e6c00, (0xaa00 bytes) 2024-05-08T08:53:05.827 INFO:teuthology.orchestra.run.smithi196.stdout:2701 trunc from 0x717400 to 0x956e600 2024-05-08T08:53:05.827 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:05.827 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:05.838 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:53:05.852 INFO:teuthology.orchestra.run.smithi196.stdout:2702 write 0xeb2fe00 thru 0xeb3275f (0x2960 bytes) 2024-05-08T08:53:05.853 INFO:teuthology.orchestra.run.smithi196.stdout:2703 read 0xddfca00 thru 0xddfd61a (0xc1b bytes) 2024-05-08T08:53:05.853 INFO:teuthology.orchestra.run.smithi196.stdout:2705 read 0xef1800 thru 0xf0034a (0xeb4b bytes) 2024-05-08T08:53:05.853 INFO:teuthology.orchestra.run.smithi196.stdout:2706 read 0x719c600 thru 0x71a112c (0x4b2d bytes) 2024-05-08T08:53:05.854 INFO:teuthology.orchestra.run.smithi196.stdout:2707 write 0x82ca400 thru 0x82cd51e (0x311f bytes) 2024-05-08T08:53:05.854 INFO:teuthology.orchestra.run.smithi196.stdout:2708 write 0xd087000 thru 0xd087218 (0x219 bytes) 2024-05-08T08:53:05.854 INFO:teuthology.orchestra.run.smithi196.stdout:2709 read 0x33b7600 thru 0x33c47cc (0xd1cd bytes) 2024-05-08T08:53:05.854 INFO:teuthology.orchestra.run.smithi196.stdout:2710 read 0x7c7be00 thru 0x7c82985 (0x6b86 bytes) 2024-05-08T08:53:05.854 INFO:teuthology.orchestra.run.smithi196.stdout:2711 read 0x6e0ba00 thru 0x6e141d2 (0x87d3 bytes) 2024-05-08T08:53:05.854 INFO:teuthology.orchestra.run.smithi196.stdout:2712 punch from 0x4d3a600 to 0x4d45600, (0xb000 bytes) 2024-05-08T08:53:05.857 INFO:teuthology.orchestra.run.smithi196.stdout:2714 read 0x3e7b000 thru 0x3e8433f (0x9340 bytes) 2024-05-08T08:53:05.857 INFO:teuthology.orchestra.run.smithi196.stdout:2716 write 0x78c3000 thru 0x78cbdcd (0x8dce bytes) 2024-05-08T08:53:05.858 INFO:teuthology.orchestra.run.smithi196.stdout:2718 punch from 0x2158400 to 0x215c800, (0x4400 bytes) 2024-05-08T08:53:05.858 INFO:teuthology.orchestra.run.smithi196.stdout:2719 read 0x909b200 thru 0x90a0ebd (0x5cbe bytes) 2024-05-08T08:53:05.858 INFO:teuthology.orchestra.run.smithi196.stdout:2721 read 0x424fe00 thru 0x4256fcc (0x71cd bytes) 2024-05-08T08:53:05.858 INFO:teuthology.orchestra.run.smithi196.stdout:2724 write 0xa3e4400 thru 0xa3e8291 (0x3e92 bytes) 2024-05-08T08:53:05.859 INFO:teuthology.orchestra.run.smithi196.stdout:2728 read 0xa921e00 thru 0xa924672 (0x2873 bytes) 2024-05-08T08:53:05.859 INFO:teuthology.orchestra.run.smithi196.stdout:2730 read 0x7170000 thru 0x71727ce (0x27cf bytes) 2024-05-08T08:53:05.859 INFO:teuthology.orchestra.run.smithi196.stdout:2733 write 0x14a1600 thru 0x14a9048 (0x7a49 bytes) 2024-05-08T08:53:05.871 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:05.874+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:05.873 INFO:teuthology.orchestra.run.smithi196.stdout:2736 trunc from 0xeb32800 to 0x2706200 2024-05-08T08:53:05.896 INFO:teuthology.orchestra.run.smithi196.stdout:2739 read 0x221b600 thru 0x2229801 (0xe202 bytes) 2024-05-08T08:53:05.897 INFO:teuthology.orchestra.run.smithi196.stdout:2743 read 0x1fb5a00 thru 0x1fbca4d (0x704e bytes) 2024-05-08T08:53:05.898 INFO:teuthology.orchestra.run.smithi196.stdout:2744 read 0x190ba00 thru 0x1911a98 (0x6099 bytes) 2024-05-08T08:53:05.898 INFO:teuthology.orchestra.run.smithi196.stdout:2746 read 0x1a72600 thru 0x1a80fe3 (0xe9e4 bytes) 2024-05-08T08:53:05.899 INFO:teuthology.orchestra.run.smithi196.stdout:2747 trunc from 0x2706200 to 0x363ce00 2024-05-08T08:53:05.908 INFO:teuthology.orchestra.run.smithi196.stdout:2750 write 0x2cbb400 thru 0x2ccab42 (0xf743 bytes) 2024-05-08T08:53:05.909 INFO:teuthology.orchestra.run.smithi196.stdout:2752 read 0xf59800 thru 0xf5fec2 (0x66c3 bytes) 2024-05-08T08:53:05.912 INFO:teuthology.orchestra.run.smithi196.stdout:2753 trunc from 0x363ce00 to 0x5c95400 2024-05-08T08:53:05.926 INFO:teuthology.orchestra.run.smithi196.stdout:2754 read 0x19f000 thru 0x1a2603 (0x3604 bytes) 2024-05-08T08:53:05.928 INFO:teuthology.orchestra.run.smithi196.stdout:2755 write 0x10f200 thru 0x117691 (0x8492 bytes) 2024-05-08T08:53:05.929 INFO:teuthology.orchestra.run.smithi196.stdout:2756 read 0x31ad200 thru 0x31bc9de (0xf7df bytes) 2024-05-08T08:53:05.929 INFO:teuthology.orchestra.run.smithi196.stdout:2758 read 0x27d3a00 thru 0x27d4ecb (0x14cc bytes) 2024-05-08T08:53:05.930 INFO:teuthology.orchestra.run.smithi196.stdout:2759 flatten 2024-05-08T08:53:05.941 INFO:teuthology.orchestra.run.smithi196.stdout:2760 punch from 0x275600 to 0x27c000, (0x6a00 bytes) 2024-05-08T08:53:05.942 INFO:teuthology.orchestra.run.smithi196.stdout:2762 punch from 0x41c1600 to 0x41c9800, (0x8200 bytes) 2024-05-08T08:53:05.967 INFO:teuthology.orchestra.run.smithi196.stdout:2763 write 0xea27a00 thru 0xea2c47b (0x4a7c bytes) 2024-05-08T08:53:05.967 INFO:teuthology.orchestra.run.smithi196.stdout:2765 write 0xdba3a00 thru 0xdbaddee (0xa3ef bytes) 2024-05-08T08:53:05.968 INFO:teuthology.orchestra.run.smithi196.stdout:2767 trunc from 0xea2c600 to 0xde89a00 2024-05-08T08:53:05.972 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:05.975+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:05.982 INFO:teuthology.orchestra.run.smithi196.stdout:2768 read 0x1703200 thru 0x170a7e1 (0x75e2 bytes) 2024-05-08T08:53:05.983 INFO:teuthology.orchestra.run.smithi196.stdout:2769 read 0x3b9ee00 thru 0x3ba431b (0x551c bytes) 2024-05-08T08:53:05.983 INFO:teuthology.orchestra.run.smithi196.stdout:2770 clone 17 order 21 su 32768 sc 15 2024-05-08T08:53:05.997 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:05.997 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:06.006 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:53:06.037 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:53:06.037 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-05-08T08:53:06.048 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":89,"stamp":"2024-05-08T08:53:05.752717+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":584237618,"num_objects":1479,"num_object_clones":256,"num_object_copies":2958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1479,"num_whiteouts":970,"num_read":154167,"num_read_kb":534522,"num_write":9364,"num_write_kb":218491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":707,"num_bytes_recovered":29906802,"num_keys_recovered":120,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8715,"ondisk_log_size":8715,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":471859200,"kb_used":235552,"kb_used_data":36636,"kb_used_omap":7,"kb_used_meta":198840,"kb_avail":471623648,"statfs":{"total":483183820800,"available":482942615552,"internally_reserved":0,"allocated":37515264,"data_stored":309523713,"data_compressed":14165949,"data_compressed_allocated":26300416,"data_compressed_original":302920704,"omap_allocated":7973,"internal_metadata":203612379},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":32654002,"num_objects":69,"num_object_clones":48,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":33,"num_read":6118,"num_read_kb":38518,"num_write":329,"num_write_kb":20847,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":2621440,"num_keys_recovered":11,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000705"},"pg_stats":[{"pgid":"3.1f","version":"52'210","reported_seq":12584,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-08T08:53:02.358121+0000","last_change":"2024-05-08T08:52:13.934602+0000","last_active":"2024-05-08T08:53:02.358121+0000","last_peered":"2024-05-08T08:53:02.358121+0000","last_clean":"2024-05-08T08:53:02.358121+0000","last_became_active":"2024-05-08T08:52:11.931332+0000","last_became_peered":"2024-05-08T08:52:11.931332+0000","last_unstale":"2024-05-08T08:53:02.358121+0000","last_undegraded":"2024-05-08T08:53:02.358121+0000","last_fullsized":"2024-05-08T08:53:02.358121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":210,"log_dups_size":0,"ondisk_log_size":210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:39:32.246936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000691565,"stat_sum":{"num_bytes":46172215,"num_objects":49,"num_object_clones":11,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":29,"num_read":11245,"num_read_kb":52972,"num_write":302,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"56'246","reported_seq":1542,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-08T08:53:02.360995+0000","last_change":"2024-05-08T08:52:13.944947+0000","last_active":"2024-05-08T08:53:02.360995+0000","last_peered":"2024-05-08T08:53:02.360995+0000","last_clean":"2024-05-08T08:53:02.360995+0000","last_became_active":"2024-05-08T08:52:11.930844+0000","last_became_peered":"2024-05-08T08:52:11.930844+0000","last_unstale":"2024-05-08T08:53:02.360995+0000","last_undegraded":"2024-05-08T08:53:02.360995+0000","last_fullsized":"2024-05-08T08:53:02.360995+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":246,"log_dups_size":0,"ondisk_log_size":246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:02:47.883860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040424900000000003,"stat_sum":{"num_bytes":11287606,"num_objects":53,"num_object_clones":8,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":37,"num_read":1652,"num_read_kb":6809,"num_write":221,"num_write_kb":6738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"56'129","reported_seq":637,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-08T08:53:02.358444+0000","last_change":"2024-05-08T08:52:36.057025+0000","last_active":"2024-05-08T08:53:02.358444+0000","last_peered":"2024-05-08T08:53:02.358444+0000","last_clean":"2024-05-08T08:53:02.358444+0000","last_became_active":"2024-05-08T08:52:36.024295+0000","last_became_peered":"2024-05-08T08:52:36.024295+0000","last_unstale":"2024-05-08T08:53:02.358444+0000","last_undegraded":"2024-05-08T08:53:02.358444+0000","last_fullsized":"2024-05-08T08:53:02.358444+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:35:37.147016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072111799999999998,"stat_sum":{"num_bytes":9556559,"num_objects":42,"num_object_clones":7,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":23,"num_read":3469,"num_read_kb":14567,"num_write":110,"num_write_kb":3444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":173135,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"56'267","reported_seq":7937,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-08T08:53:02.358525+0000","last_change":"2024-05-08T08:52:50.070942+0000","last_active":"2024-05-08T08:53:02.358525+0000","last_peered":"2024-05-08T08:53:02.358525+0000","last_clean":"2024-05-08T08:53:02.358525+0000","last_became_active":"2024-05-08T08:52:50.069533+0000","last_became_peered":"2024-05-08T08:52:50.069533+0000","last_unstale":"2024-05-08T08:53:02.358525+0000","last_undegraded":"2024-05-08T08:53:02.358525+0000","last_fullsized":"2024-05-08T08:53:02.358525+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":267,"log_dups_size":0,"ondisk_log_size":267,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:01:48.220278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094844199999999997,"stat_sum":{"num_bytes":41446437,"num_objects":51,"num_object_clones":7,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":34,"num_read":8017,"num_read_kb":51155,"num_write":356,"num_write_kb":21609,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":13917221,"num_keys_recovered":22,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"56'253","reported_seq":7840,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-08T08:53:02.358362+0000","last_change":"2024-05-08T08:52:50.069794+0000","last_active":"2024-05-08T08:53:02.358362+0000","last_peered":"2024-05-08T08:53:02.358362+0000","last_clean":"2024-05-08T08:53:02.358362+0000","last_became_active":"2024-05-08T08:52:50.069306+0000","last_became_peered":"2024-05-08T08:52:50.069306+0000","last_unstale":"2024-05-08T08:53:02.358362+0000","last_undegraded":"2024-05-08T08:53:02.358362+0000","last_fullsized":"2024-05-08T08:53:02.358362+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":253,"log_dups_size":0,"ondisk_log_size":253,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:21:55.107288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000685079,"stat_sum":{"num_bytes":30289452,"num_objects":52,"num_object_clones":8,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":38,"num_read":8055,"num_read_kb":14484,"num_write":317,"num_write_kb":18061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":146988,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"53'258","reported_seq":7279,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-08T08:53:02.358543+0000","last_change":"2024-05-08T08:52:50.072642+0000","last_active":"2024-05-08T08:53:02.358543+0000","last_peered":"2024-05-08T08:53:02.358543+0000","last_clean":"2024-05-08T08:53:02.358543+0000","last_became_active":"2024-05-08T08:52:50.072082+0000","last_became_peered":"2024-05-08T08:52:50.072082+0000","last_unstale":"2024-05-08T08:53:02.358543+0000","last_undegraded":"2024-05-08T08:53:02.358543+0000","last_fullsized":"2024-05-08T08:53:02.358543+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":258,"log_dups_size":0,"ondisk_log_size":258,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:37:41.343825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066525600000000003,"stat_sum":{"num_bytes":12950081,"num_objects":50,"num_object_clones":8,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":31,"num_read":7792,"num_read_kb":16911,"num_write":265,"num_write_kb":6132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":1862193,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"56'177","reported_seq":1326,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-08T08:53:02.358305+0000","last_change":"2024-05-08T08:52:50.069911+0000","last_active":"2024-05-08T08:53:02.358305+0000","last_peered":"2024-05-08T08:53:02.358305+0000","last_clean":"2024-05-08T08:53:02.358305+0000","last_became_active":"2024-05-08T08:52:50.069328+0000","last_became_peered":"2024-05-08T08:52:50.069328+0000","last_unstale":"2024-05-08T08:53:02.358305+0000","last_undegraded":"2024-05-08T08:53:02.358305+0000","last_fullsized":"2024-05-08T08:53:02.358305+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:20:57.740544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082481000000000004,"stat_sum":{"num_bytes":8493056,"num_objects":48,"num_object_clones":8,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":36,"num_read":963,"num_read_kb":12868,"num_write":140,"num_write_kb":8851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":1896960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"56'266","reported_seq":4347,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-08T08:53:02.360854+0000","last_change":"2024-05-08T08:52:13.945184+0000","last_active":"2024-05-08T08:53:02.360854+0000","last_peered":"2024-05-08T08:53:02.360854+0000","last_clean":"2024-05-08T08:53:02.360854+0000","last_became_active":"2024-05-08T08:52:11.927366+0000","last_became_peered":"2024-05-08T08:52:11.927366+0000","last_unstale":"2024-05-08T08:53:02.360854+0000","last_undegraded":"2024-05-08T08:53:02.360854+0000","last_fullsized":"2024-05-08T08:53:02.360854+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":266,"log_dups_size":0,"ondisk_log_size":266,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:03:59.636906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057208900000000002,"stat_sum":{"num_bytes":58095652,"num_objects":50,"num_object_clones":10,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":34,"num_read":4584,"num_read_kb":17820,"num_write":312,"num_write_kb":15291,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"52'544","reported_seq":2314,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-08T08:53:02.361119+0000","last_change":"2024-05-08T08:52:57.123662+0000","last_active":"2024-05-08T08:53:02.361119+0000","last_peered":"2024-05-08T08:53:02.361119+0000","last_clean":"2024-05-08T08:53:02.361119+0000","last_became_active":"2024-05-08T08:52:57.122685+0000","last_became_peered":"2024-05-08T08:52:57.122685+0000","last_unstale":"2024-05-08T08:53:02.361119+0000","last_undegraded":"2024-05-08T08:53:02.361119+0000","last_fullsized":"2024-05-08T08:53:02.361119+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":544,"log_dups_size":0,"ondisk_log_size":544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:46:02.981683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000866371,"stat_sum":{"num_bytes":2926689,"num_objects":35,"num_object_clones":7,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":21,"num_read":4526,"num_read_kb":7807,"num_write":684,"num_write_kb":2903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":607329,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.16","version":"56'275","reported_seq":12152,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-08T08:53:02.358756+0000","last_change":"2024-05-08T08:52:13.932215+0000","last_active":"2024-05-08T08:53:02.358756+0000","last_peered":"2024-05-08T08:53:02.358756+0000","last_clean":"2024-05-08T08:53:02.358756+0000","last_became_active":"2024-05-08T08:52:11.931213+0000","last_became_peered":"2024-05-08T08:52:11.931213+0000","last_unstale":"2024-05-08T08:53:02.358756+0000","last_undegraded":"2024-05-08T08:53:02.358756+0000","last_fullsized":"2024-05-08T08:53:02.358756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":275,"log_dups_size":0,"ondisk_log_size":275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:57:15.366217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084652399999999997,"stat_sum":{"num_bytes":29847743,"num_objects":41,"num_object_clones":7,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":22,"num_read":12278,"num_read_kb":49177,"num_write":379,"num_write_kb":16832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"56'273","reported_seq":3746,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-08T08:53:02.486279+0000","last_change":"2024-05-08T08:52:13.944999+0000","last_active":"2024-05-08T08:53:02.486279+0000","last_peered":"2024-05-08T08:53:02.486279+0000","last_clean":"2024-05-08T08:53:02.486279+0000","last_became_active":"2024-05-08T08:52:11.931695+0000","last_became_peered":"2024-05-08T08:52:11.931695+0000","last_unstale":"2024-05-08T08:53:02.486279+0000","last_undegraded":"2024-05-08T08:53:02.486279+0000","last_fullsized":"2024-05-08T08:53:02.486279+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":273,"log_dups_size":0,"ondisk_log_size":273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:36:59.310035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063902800000000004,"stat_sum":{"num_bytes":10027046,"num_objects":39,"num_object_clones":6,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":29,"num_read":4031,"num_read_kb":7915,"num_write":338,"num_write_kb":970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"56'255","reported_seq":4185,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-08T08:53:02.486259+0000","last_change":"2024-05-08T08:52:50.068267+0000","last_active":"2024-05-08T08:53:02.486259+0000","last_peered":"2024-05-08T08:53:02.486259+0000","last_clean":"2024-05-08T08:53:02.486259+0000","last_became_active":"2024-05-08T08:52:50.067650+0000","last_became_peered":"2024-05-08T08:52:50.067650+0000","last_unstale":"2024-05-08T08:53:02.486259+0000","last_undegraded":"2024-05-08T08:53:02.486259+0000","last_fullsized":"2024-05-08T08:53:02.486259+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":255,"log_dups_size":0,"ondisk_log_size":255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:28:25.045928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000430331,"stat_sum":{"num_bytes":20665936,"num_objects":47,"num_object_clones":11,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":29,"num_read":1742,"num_read_kb":7222,"num_write":195,"num_write_kb":4723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":593936,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":96,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-08T08:53:02.486501+0000","last_change":"2024-05-08T08:52:47.057420+0000","last_active":"2024-05-08T08:53:02.486501+0000","last_peered":"2024-05-08T08:53:02.486501+0000","last_clean":"2024-05-08T08:53:02.486501+0000","last_became_active":"2024-05-08T08:52:47.057207+0000","last_became_peered":"2024-05-08T08:52:47.057207+0000","last_unstale":"2024-05-08T08:53:02.486501+0000","last_undegraded":"2024-05-08T08:53:02.486501+0000","last_fullsized":"2024-05-08T08:53:02.486501+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:47:42.112090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"53'213","reported_seq":10061,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-08T08:53:02.358112+0000","last_change":"2024-05-08T08:52:13.934894+0000","last_active":"2024-05-08T08:53:02.358112+0000","last_peered":"2024-05-08T08:53:02.358112+0000","last_clean":"2024-05-08T08:53:02.358112+0000","last_became_active":"2024-05-08T08:52:11.929184+0000","last_became_peered":"2024-05-08T08:52:11.929184+0000","last_unstale":"2024-05-08T08:53:02.358112+0000","last_undegraded":"2024-05-08T08:53:02.358112+0000","last_fullsized":"2024-05-08T08:53:02.358112+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":213,"log_dups_size":0,"ondisk_log_size":213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:14:58.934728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088867899999999997,"stat_sum":{"num_bytes":26354688,"num_objects":40,"num_object_clones":6,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":25,"num_read":10913,"num_read_kb":48082,"num_write":207,"num_write_kb":9748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":92,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-08T08:53:02.358711+0000","last_change":"2024-05-08T08:52:03.896159+0000","last_active":"2024-05-08T08:53:02.358711+0000","last_peered":"2024-05-08T08:53:02.358711+0000","last_clean":"2024-05-08T08:53:02.358711+0000","last_became_active":"2024-05-08T08:52:03.895797+0000","last_became_peered":"2024-05-08T08:52:03.895797+0000","last_unstale":"2024-05-08T08:53:02.358711+0000","last_undegraded":"2024-05-08T08:53:02.358711+0000","last_fullsized":"2024-05-08T08:53:02.358711+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:34:33.513525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"56'149","reported_seq":1995,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-08T08:53:02.361205+0000","last_change":"2024-05-08T08:52:13.944951+0000","last_active":"2024-05-08T08:53:02.361205+0000","last_peered":"2024-05-08T08:53:02.361205+0000","last_clean":"2024-05-08T08:53:02.361205+0000","last_became_active":"2024-05-08T08:52:11.926905+0000","last_became_peered":"2024-05-08T08:52:11.926905+0000","last_unstale":"2024-05-08T08:53:02.361205+0000","last_undegraded":"2024-05-08T08:53:02.361205+0000","last_fullsized":"2024-05-08T08:53:02.361205+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:02:49.037937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044527,"stat_sum":{"num_bytes":8833170,"num_objects":54,"num_object_clones":14,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":29,"num_read":1853,"num_read_kb":20424,"num_write":136,"num_write_kb":6436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":91,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-08T08:53:02.486375+0000","last_change":"2024-05-08T08:52:04.144764+0000","last_active":"2024-05-08T08:53:02.486375+0000","last_peered":"2024-05-08T08:53:02.486375+0000","last_clean":"2024-05-08T08:53:02.486375+0000","last_became_active":"2024-05-08T08:52:04.144444+0000","last_became_peered":"2024-05-08T08:52:04.144444+0000","last_unstale":"2024-05-08T08:53:02.486375+0000","last_undegraded":"2024-05-08T08:53:02.486375+0000","last_fullsized":"2024-05-08T08:53:02.486375+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:58:31.498922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"56'275","reported_seq":4867,"reported_epoch":56,"state":"peering","last_fresh":"2024-05-08T08:53:02.350846+0000","last_change":"2024-05-08T08:53:02.359370+0000","last_active":"2024-05-08T08:53:02.359370+0000","last_peered":"2024-05-08T08:53:02.350846+0000","last_clean":"2024-05-08T08:52:59.395903+0000","last_became_active":"2024-05-08T08:53:01.359662+0000","last_became_peered":"2024-05-08T08:53:01.359662+0000","last_unstale":"2024-05-08T08:53:02.350846+0000","last_undegraded":"2024-05-08T08:53:02.350846+0000","last_fullsized":"2024-05-08T08:53:01.354679+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":275,"log_dups_size":0,"ondisk_log_size":275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000669351,"stat_sum":{"num_bytes":9171456,"num_objects":53,"num_object_clones":8,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":36,"num_read":5529,"num_read_kb":10233,"num_write":218,"num_write_kb":5922,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":2621440,"num_keys_recovered":11,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":92,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-08T08:53:02.358815+0000","last_change":"2024-05-08T08:52:04.144543+0000","last_active":"2024-05-08T08:53:02.358815+0000","last_peered":"2024-05-08T08:53:02.358815+0000","last_clean":"2024-05-08T08:53:02.358815+0000","last_became_active":"2024-05-08T08:52:04.144242+0000","last_became_peered":"2024-05-08T08:52:04.144242+0000","last_unstale":"2024-05-08T08:53:02.358815+0000","last_undegraded":"2024-05-08T08:53:02.358815+0000","last_fullsized":"2024-05-08T08:53:02.358815+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:57:36.844633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"56'546","reported_seq":4212,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-08T08:53:02.361038+0000","last_change":"2024-05-08T08:52:23.967224+0000","last_active":"2024-05-08T08:53:02.361038+0000","last_peered":"2024-05-08T08:53:02.361038+0000","last_clean":"2024-05-08T08:53:02.361038+0000","last_became_active":"2024-05-08T08:52:23.966596+0000","last_became_peered":"2024-05-08T08:52:23.966596+0000","last_unstale":"2024-05-08T08:53:02.361038+0000","last_undegraded":"2024-05-08T08:53:02.361038+0000","last_fullsized":"2024-05-08T08:53:02.361038+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":546,"log_dups_size":0,"ondisk_log_size":546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:06:03.034446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064078099999999999,"stat_sum":{"num_bytes":6713531,"num_objects":56,"num_object_clones":6,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":43,"num_read":5661,"num_read_kb":8810,"num_write":820,"num_write_kb":3258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"56'179","reported_seq":861,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-08T08:53:02.358333+0000","last_change":"2024-05-08T08:52:50.070065+0000","last_active":"2024-05-08T08:53:02.358333+0000","last_peered":"2024-05-08T08:53:02.358333+0000","last_clean":"2024-05-08T08:53:02.358333+0000","last_became_active":"2024-05-08T08:52:50.069576+0000","last_became_peered":"2024-05-08T08:52:50.069576+0000","last_unstale":"2024-05-08T08:53:02.358333+0000","last_undegraded":"2024-05-08T08:53:02.358333+0000","last_fullsized":"2024-05-08T08:53:02.358333+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":179,"log_dups_size":0,"ondisk_log_size":179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:38:15.806692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047037799999999997,"stat_sum":{"num_bytes":6234147,"num_objects":34,"num_object_clones":7,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":24,"num_read":651,"num_read_kb":5995,"num_write":172,"num_write_kb":2496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":2101248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":67,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-08T08:53:02.358711+0000","last_change":"2024-05-08T08:52:23.969093+0000","last_active":"2024-05-08T08:53:02.358711+0000","last_peered":"2024-05-08T08:53:02.358711+0000","last_clean":"2024-05-08T08:53:02.358711+0000","last_became_active":"2024-05-08T08:52:23.968782+0000","last_became_peered":"2024-05-08T08:52:23.968782+0000","last_unstale":"2024-05-08T08:53:02.358711+0000","last_undegraded":"2024-05-08T08:53:02.358711+0000","last_fullsized":"2024-05-08T08:53:02.358711+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:37:41.693728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"52'155","reported_seq":838,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-08T08:53:02.358333+0000","last_change":"2024-05-08T08:52:13.940504+0000","last_active":"2024-05-08T08:53:02.358333+0000","last_peered":"2024-05-08T08:53:02.358333+0000","last_clean":"2024-05-08T08:53:02.358333+0000","last_became_active":"2024-05-08T08:52:11.928301+0000","last_became_peered":"2024-05-08T08:52:11.928301+0000","last_unstale":"2024-05-08T08:53:02.358333+0000","last_undegraded":"2024-05-08T08:53:02.358333+0000","last_fullsized":"2024-05-08T08:53:02.358333+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:01:28.066503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046631800000000002,"stat_sum":{"num_bytes":24315972,"num_objects":46,"num_object_clones":8,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":31,"num_read":658,"num_read_kb":6106,"num_write":129,"num_write_kb":7312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":92,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-08T08:53:02.360907+0000","last_change":"2024-05-08T08:52:03.891768+0000","last_active":"2024-05-08T08:53:02.360907+0000","last_peered":"2024-05-08T08:53:02.360907+0000","last_clean":"2024-05-08T08:53:02.360907+0000","last_became_active":"2024-05-08T08:52:03.891374+0000","last_became_peered":"2024-05-08T08:52:03.891374+0000","last_unstale":"2024-05-08T08:53:02.360907+0000","last_undegraded":"2024-05-08T08:53:02.360907+0000","last_fullsized":"2024-05-08T08:53:02.360907+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:18:10.240492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"56'484","reported_seq":14351,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-08T08:53:02.358178+0000","last_change":"2024-05-08T08:52:50.071097+0000","last_active":"2024-05-08T08:53:02.358178+0000","last_peered":"2024-05-08T08:53:02.358178+0000","last_clean":"2024-05-08T08:53:02.358178+0000","last_became_active":"2024-05-08T08:52:50.070396+0000","last_became_peered":"2024-05-08T08:52:50.070396+0000","last_unstale":"2024-05-08T08:53:02.358178+0000","last_undegraded":"2024-05-08T08:53:02.358178+0000","last_fullsized":"2024-05-08T08:53:02.358178+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:25:27.444422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046440499999999998,"stat_sum":{"num_bytes":27603539,"num_objects":44,"num_object_clones":8,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":26,"num_read":15391,"num_read_kb":20018,"num_write":290,"num_write_kb":3515,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":1271856,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":91,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-08T08:53:02.486187+0000","last_change":"2024-05-08T08:52:03.891551+0000","last_active":"2024-05-08T08:53:02.486187+0000","last_peered":"2024-05-08T08:53:02.486187+0000","last_clean":"2024-05-08T08:53:02.486187+0000","last_became_active":"2024-05-08T08:52:03.891296+0000","last_became_peered":"2024-05-08T08:52:03.891296+0000","last_unstale":"2024-05-08T08:53:02.486187+0000","last_undegraded":"2024-05-08T08:53:02.486187+0000","last_fullsized":"2024-05-08T08:53:02.486187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:54.806343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"52'130","reported_seq":596,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-08T08:53:02.358634+0000","last_change":"2024-05-08T08:52:13.940743+0000","last_active":"2024-05-08T08:53:02.358634+0000","last_peered":"2024-05-08T08:53:02.358634+0000","last_clean":"2024-05-08T08:53:02.358634+0000","last_became_active":"2024-05-08T08:52:11.928273+0000","last_became_peered":"2024-05-08T08:52:11.928273+0000","last_unstale":"2024-05-08T08:53:02.358634+0000","last_undegraded":"2024-05-08T08:53:02.358634+0000","last_fullsized":"2024-05-08T08:53:02.358634+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:32:19.155511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062135599999999999,"stat_sum":{"num_bytes":7776443,"num_objects":51,"num_object_clones":12,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":33,"num_read":466,"num_read_kb":3643,"num_write":101,"num_write_kb":1144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":67,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-08T08:53:02.361001+0000","last_change":"2024-05-08T08:52:23.966794+0000","last_active":"2024-05-08T08:53:02.361001+0000","last_peered":"2024-05-08T08:53:02.361001+0000","last_clean":"2024-05-08T08:53:02.361001+0000","last_became_active":"2024-05-08T08:52:23.966544+0000","last_became_peered":"2024-05-08T08:52:23.966544+0000","last_unstale":"2024-05-08T08:53:02.361001+0000","last_undegraded":"2024-05-08T08:53:02.361001+0000","last_fullsized":"2024-05-08T08:53:02.361001+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:25:56.927420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":247,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-08T08:53:02.486187+0000","last_change":"2024-05-08T08:51:58.861899+0000","last_active":"2024-05-08T08:53:02.486187+0000","last_peered":"2024-05-08T08:53:02.486187+0000","last_clean":"2024-05-08T08:53:02.486187+0000","last_became_active":"2024-05-08T08:51:58.861596+0000","last_became_peered":"2024-05-08T08:51:58.861596+0000","last_unstale":"2024-05-08T08:53:02.486187+0000","last_undegraded":"2024-05-08T08:53:02.486187+0000","last_fullsized":"2024-05-08T08:53:02.486187+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:51:57.849666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:51:57.849666+0000","last_clean_scrub_stamp":"2024-05-08T08:51:57.849666+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:53:21.739950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"53'124","reported_seq":895,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-08T08:53:02.361042+0000","last_change":"2024-05-08T08:52:13.945362+0000","last_active":"2024-05-08T08:53:02.361042+0000","last_peered":"2024-05-08T08:53:02.361042+0000","last_clean":"2024-05-08T08:53:02.361042+0000","last_became_active":"2024-05-08T08:52:11.926975+0000","last_became_peered":"2024-05-08T08:52:11.926975+0000","last_unstale":"2024-05-08T08:53:02.361042+0000","last_undegraded":"2024-05-08T08:53:02.361042+0000","last_fullsized":"2024-05-08T08:53:02.361042+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:31:12.890505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068218599999999999,"stat_sum":{"num_bytes":19044368,"num_objects":47,"num_object_clones":7,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":34,"num_read":722,"num_read_kb":4877,"num_write":92,"num_write_kb":2555,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"56'198","reported_seq":4707,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-08T08:53:02.360872+0000","last_change":"2024-05-08T08:52:50.069736+0000","last_active":"2024-05-08T08:53:02.360872+0000","last_peered":"2024-05-08T08:53:02.360872+0000","last_clean":"2024-05-08T08:53:02.360872+0000","last_became_active":"2024-05-08T08:52:50.069112+0000","last_became_peered":"2024-05-08T08:52:50.069112+0000","last_unstale":"2024-05-08T08:53:02.360872+0000","last_undegraded":"2024-05-08T08:53:02.360872+0000","last_fullsized":"2024-05-08T08:53:02.360872+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":198,"log_dups_size":0,"ondisk_log_size":198,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:28:09.386209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00030754000000000003,"stat_sum":{"num_bytes":7601152,"num_objects":42,"num_object_clones":7,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":29,"num_read":4820,"num_read_kb":11038,"num_write":148,"num_write_kb":2337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":1878528,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"56'462","reported_seq":7986,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-08T08:53:02.486432+0000","last_change":"2024-05-08T08:52:50.072624+0000","last_active":"2024-05-08T08:53:02.486432+0000","last_peered":"2024-05-08T08:53:02.486432+0000","last_clean":"2024-05-08T08:53:02.486432+0000","last_became_active":"2024-05-08T08:52:50.071883+0000","last_became_peered":"2024-05-08T08:52:50.071883+0000","last_unstale":"2024-05-08T08:53:02.486432+0000","last_undegraded":"2024-05-08T08:53:02.486432+0000","last_fullsized":"2024-05-08T08:53:02.486432+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":462,"log_dups_size":0,"ondisk_log_size":462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:49:59.173527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070350100000000004,"stat_sum":{"num_bytes":15769748,"num_objects":48,"num_object_clones":7,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":31,"num_read":10634,"num_read_kb":16804,"num_write":313,"num_write_kb":2569,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":54,"num_bytes_recovered":426496,"num_keys_recovered":28,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"56'267","reported_seq":2510,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-08T08:53:02.358071+0000","last_change":"2024-05-08T08:52:13.940600+0000","last_active":"2024-05-08T08:53:02.358071+0000","last_peered":"2024-05-08T08:53:02.358071+0000","last_clean":"2024-05-08T08:53:02.358071+0000","last_became_active":"2024-05-08T08:52:11.927196+0000","last_became_peered":"2024-05-08T08:52:11.927196+0000","last_unstale":"2024-05-08T08:53:02.358071+0000","last_undegraded":"2024-05-08T08:53:02.358071+0000","last_fullsized":"2024-05-08T08:53:02.358071+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":267,"log_dups_size":0,"ondisk_log_size":267,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:36:12.600383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048296599999999998,"stat_sum":{"num_bytes":27166240,"num_objects":44,"num_object_clones":6,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":30,"num_read":2827,"num_read_kb":6751,"num_write":200,"num_write_kb":2022,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"52'118","reported_seq":563,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-08T08:53:02.358150+0000","last_change":"2024-05-08T08:52:23.969521+0000","last_active":"2024-05-08T08:53:02.358150+0000","last_peered":"2024-05-08T08:53:02.358150+0000","last_clean":"2024-05-08T08:53:02.358150+0000","last_became_active":"2024-05-08T08:52:23.969067+0000","last_became_peered":"2024-05-08T08:52:23.969067+0000","last_unstale":"2024-05-08T08:53:02.358150+0000","last_undegraded":"2024-05-08T08:53:02.358150+0000","last_fullsized":"2024-05-08T08:53:02.358150+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":118,"log_dups_size":0,"ondisk_log_size":118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:18:09.413877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048793000000000003,"stat_sum":{"num_bytes":2731520,"num_objects":40,"num_object_clones":6,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":31,"num_read":384,"num_read_kb":2770,"num_write":90,"num_write_kb":932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"56'185","reported_seq":920,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-08T08:53:02.486280+0000","last_change":"2024-05-08T08:52:13.945141+0000","last_active":"2024-05-08T08:53:02.486280+0000","last_peered":"2024-05-08T08:53:02.486280+0000","last_clean":"2024-05-08T08:53:02.486280+0000","last_became_active":"2024-05-08T08:52:11.932594+0000","last_became_peered":"2024-05-08T08:52:11.932594+0000","last_unstale":"2024-05-08T08:53:02.486280+0000","last_undegraded":"2024-05-08T08:53:02.486280+0000","last_fullsized":"2024-05-08T08:53:02.486280+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":185,"log_dups_size":0,"ondisk_log_size":185,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:49:56.612128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074388199999999996,"stat_sum":{"num_bytes":3361792,"num_objects":47,"num_object_clones":7,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":34,"num_read":678,"num_read_kb":7280,"num_write":153,"num_write_kb":2274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"52'125","reported_seq":323,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-08T08:53:02.358553+0000","last_change":"2024-05-08T08:52:47.137016+0000","last_active":"2024-05-08T08:53:02.358553+0000","last_peered":"2024-05-08T08:53:02.358553+0000","last_clean":"2024-05-08T08:53:02.358553+0000","last_became_active":"2024-05-08T08:52:47.071804+0000","last_became_peered":"2024-05-08T08:52:47.071804+0000","last_unstale":"2024-05-08T08:53:02.358553+0000","last_undegraded":"2024-05-08T08:53:02.358553+0000","last_fullsized":"2024-05-08T08:53:02.358553+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":125,"log_dups_size":0,"ondisk_log_size":125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:09:09.823033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085079600000000002,"stat_sum":{"num_bytes":10514432,"num_objects":29,"num_object_clones":6,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":17,"num_read":5492,"num_read_kb":20233,"num_write":108,"num_write_kb":3740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":2343424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.f","version":"56'150","reported_seq":2000,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-08T08:53:02.358788+0000","last_change":"2024-05-08T08:52:13.932524+0000","last_active":"2024-05-08T08:53:02.358788+0000","last_peered":"2024-05-08T08:53:02.358788+0000","last_clean":"2024-05-08T08:53:02.358788+0000","last_became_active":"2024-05-08T08:52:11.931347+0000","last_became_peered":"2024-05-08T08:52:11.931347+0000","last_unstale":"2024-05-08T08:53:02.358788+0000","last_undegraded":"2024-05-08T08:53:02.358788+0000","last_fullsized":"2024-05-08T08:53:02.358788+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:35:48.381065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010326370000000001,"stat_sum":{"num_bytes":8210115,"num_objects":33,"num_object_clones":3,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":23,"num_read":2028,"num_read_kb":14504,"num_write":172,"num_write_kb":1596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"52'150","reported_seq":2397,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-08T08:53:02.357984+0000","last_change":"2024-05-08T08:52:23.970138+0000","last_active":"2024-05-08T08:53:02.357984+0000","last_peered":"2024-05-08T08:53:02.357984+0000","last_clean":"2024-05-08T08:53:02.357984+0000","last_became_active":"2024-05-08T08:52:23.969732+0000","last_became_peered":"2024-05-08T08:52:23.969732+0000","last_unstale":"2024-05-08T08:53:02.357984+0000","last_undegraded":"2024-05-08T08:53:02.357984+0000","last_fullsized":"2024-05-08T08:53:02.357984+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:42:25.367308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053593,"stat_sum":{"num_bytes":17634816,"num_objects":46,"num_object_clones":8,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":32,"num_read":2197,"num_read_kb":20079,"num_write":119,"num_write_kb":9293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"56'179","reported_seq":807,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-08T08:53:02.486259+0000","last_change":"2024-05-08T08:52:50.071476+0000","last_active":"2024-05-08T08:53:02.486259+0000","last_peered":"2024-05-08T08:53:02.486259+0000","last_clean":"2024-05-08T08:53:02.486259+0000","last_became_active":"2024-05-08T08:52:50.070771+0000","last_became_peered":"2024-05-08T08:52:50.070771+0000","last_unstale":"2024-05-08T08:53:02.486259+0000","last_undegraded":"2024-05-08T08:53:02.486259+0000","last_fullsized":"2024-05-08T08:53:02.486259+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":179,"log_dups_size":0,"ondisk_log_size":179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:43:38.337890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041230699999999999,"stat_sum":{"num_bytes":9478144,"num_objects":50,"num_object_clones":9,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":37,"num_read":469,"num_read_kb":4109,"num_write":145,"num_write_kb":2168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":66048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"52'1070","reported_seq":6558,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-08T08:53:02.358120+0000","last_change":"2024-05-08T08:52:13.934541+0000","last_active":"2024-05-08T08:53:02.358120+0000","last_peered":"2024-05-08T08:53:02.358120+0000","last_clean":"2024-05-08T08:53:02.358120+0000","last_became_active":"2024-05-08T08:52:11.931202+0000","last_became_peered":"2024-05-08T08:52:11.931202+0000","last_unstale":"2024-05-08T08:53:02.358120+0000","last_undegraded":"2024-05-08T08:53:02.358120+0000","last_fullsized":"2024-05-08T08:53:02.358120+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":1070,"log_dups_size":0,"ondisk_log_size":1070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:20:49.193373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059220300000000002,"stat_sum":{"num_bytes":22436500,"num_objects":53,"num_object_clones":6,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":31,"num_read":11782,"num_read_kb":23855,"num_write":1674,"num_write_kb":5091,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"56'286","reported_seq":2531,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-08T08:53:02.486405+0000","last_change":"2024-05-08T08:52:13.944894+0000","last_active":"2024-05-08T08:53:02.486405+0000","last_peered":"2024-05-08T08:53:02.486405+0000","last_clean":"2024-05-08T08:53:02.486405+0000","last_became_active":"2024-05-08T08:52:11.930434+0000","last_became_peered":"2024-05-08T08:52:11.930434+0000","last_unstale":"2024-05-08T08:53:02.486405+0000","last_undegraded":"2024-05-08T08:53:02.486405+0000","last_fullsized":"2024-05-08T08:53:02.486405+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":286,"log_dups_size":0,"ondisk_log_size":286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:55:18.055134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059972400000000005,"stat_sum":{"num_bytes":40937005,"num_objects":63,"num_object_clones":17,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":31,"num_read":2542,"num_read_kb":19104,"num_write":299,"num_write_kb":7973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":583647250,"num_objects":1477,"num_object_clones":256,"num_object_copies":2954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1477,"num_whiteouts":970,"num_read":154051,"num_read_kb":534422,"num_write":9205,"num_write_kb":216075,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":707,"num_bytes_recovered":29906802,"num_keys_recovered":120,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":36970496,"data_stored":332605670,"data_compressed":15036819,"data_compressed_allocated":28590080,"data_compressed_original":327013888,"omap_allocated":0,"internal_metadata":0},"log_size":8598,"ondisk_log_size":8598,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":11046,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640271,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47540,"kb_used_data":8032,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94324300,"statfs":{"total":96636764160,"available":96588083200,"internally_reserved":0,"allocated":8224768,"data_stored":70122210,"data_compressed":3413816,"data_compressed_allocated":6451200,"data_compressed_original":69106688,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":0,"seq":47244640274,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640273,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53812,"kb_used_data":10156,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94318028,"statfs":{"total":96636764160,"available":96581660672,"internally_reserved":0,"allocated":10399744,"data_stored":84270474,"data_compressed":4168939,"data_compressed_allocated":6971392,"data_compressed_original":82120704,"omap_allocated":1593,"internal_metadata":44693959},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640272,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39880,"kb_used_data":4540,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94331960,"statfs":{"total":96636764160,"available":96595927040,"internally_reserved":0,"allocated":4648960,"data_stored":38040994,"data_compressed":1743929,"data_compressed_allocated":3317760,"data_compressed_original":37369344,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640271,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41788,"kb_used_data":6452,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94330052,"statfs":{"total":96636764160,"available":96593973248,"internally_reserved":0,"allocated":6606848,"data_stored":63045044,"data_compressed":2421776,"data_compressed_allocated":5152768,"data_compressed_original":62254592,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640274,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52532,"kb_used_data":7456,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94319308,"statfs":{"total":96636764160,"available":96582971392,"internally_reserved":0,"allocated":7634944,"data_stored":54044991,"data_compressed":2417489,"data_compressed_allocated":4407296,"data_compressed_original":52069376,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5523,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5523,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6914048,"data_stored":53141542,"data_compressed":2411888,"data_compressed_allocated":4366336,"data_compressed_original":51450880,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":5955584,"data_stored":62753467,"data_compressed":2432648,"data_compressed_allocated":5160960,"data_compressed_original":62232576,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3997696,"data_stored":37782256,"data_compressed":1743929,"data_compressed_allocated":3317760,"data_compressed_original":37369344,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":9756672,"data_stored":83534954,"data_compressed":4201462,"data_compressed_allocated":6979584,"data_compressed_original":81650688,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":2830336,"data_stored":25559234,"data_compressed":860285,"data_compressed_allocated":2351104,"data_compressed_original":25220096,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":7516160,"data_stored":69834217,"data_compressed":3386607,"data_compressed_allocated":6414336,"data_compressed_original":69090304,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T08:53:06.051 INFO:tasks.thrashosds.thrasher:Setting 3.17 to [1, 0] 2024-05-08T08:53:06.051 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.17', '1', '0'] 2024-05-08T08:53:06.051 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.17 1 0 2024-05-08T08:53:06.090 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:06.093+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:06.152 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:06.152 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:06.161 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:53:06.205 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:06.208+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:06.266 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:06.266 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:06.274 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:53:06.275 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:06.277+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:06.376 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:06.379+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:06.428 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:06.428 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:06.438 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:53:06.477 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:06.480+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:06.577 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:06.580+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:06.585 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:06.585 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:06.594 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:53:06.679 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:06.682+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:06.707 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:06.707 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:06.716 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:53:06.779 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:06.783+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:06.794 INFO:teuthology.orchestra.run.smithi196.stdout:checking clone #15, image image_client.0-clone15 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent16 2024-05-08T08:53:06.829 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:06.829 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:06.838 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:53:06.880 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:06.883+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:06.981 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:06.984+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:07.013 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:07.013 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:07.023 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:53:07.108 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:07.111+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:07.182 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:53:07.185+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:53:07.193 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:07.193 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:07.202 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:53:07.284 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:07.287+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:07.352 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:07.352 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:07.360 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:53:07.378 INFO:teuthology.orchestra.run.smithi059.stderr:set 3.17 pg_upmap mapping to [1,0] 2024-05-08T08:53:07.385 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:07.388+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:07.386 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-05-08T08:53:07.432 INFO:teuthology.orchestra.run.smithi196.stdout:2771 trunc from 0xde89a00 to 0x760f000 2024-05-08T08:53:07.485 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:53:07.489+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:53:07.486 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:07.486 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:07.494 INFO:teuthology.orchestra.run.smithi196.stdout:2774 write 0xe2ddc00 thru 0xe2ddef5 (0x2f6 bytes) 2024-05-08T08:53:07.496 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:53:07.500 INFO:teuthology.orchestra.run.smithi196.stdout:2775 read 0x72c4400 thru 0x72c6067 (0x1c68 bytes) 2024-05-08T08:53:07.500 INFO:teuthology.orchestra.run.smithi196.stdout:2776 write 0x6c82a00 thru 0x6c8b055 (0x8656 bytes) 2024-05-08T08:53:07.509 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T08:53:07.515 INFO:teuthology.orchestra.run.smithi196.stdout:2779 trunc from 0xe2de000 to 0x1d45e00 2024-05-08T08:53:07.586 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:07.589+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:07.654 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:07.654 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:07.667 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:53:07.674 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:53:07.674 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:53:07.693 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:53:07.693 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:53:07.693 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:07.696+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:07.710 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:53:07.710 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:53:07.728 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:53:07.728 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:53:07.746 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:53:07.746 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:53:07.764 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:53:07.764 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:53:07.786 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:07.786 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:53:07.786 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:53:07.786 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:53:07.786 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:07.788 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:07.791+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:07.795 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:53:07.889 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:07.892+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:07.947 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:07.947 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:53:07.948 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:53:07.948 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:53:07.948 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.948 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7490 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 0~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:07.948 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.744226+0000", 2024-05-08T08:53:07.948 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 30.205754663, 2024-05-08T08:53:07.948 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057294818999999997, 2024-05-08T08:53:07.948 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:07.948 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:07.948 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:07.948 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:07.948 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:07.948 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:07.949 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7490 2024-05-08T08:53:07.949 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.949 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:07.949 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.949 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:07.949 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744226+0000", 2024-05-08T08:53:07.949 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:07.949 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.949 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.949 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:07.949 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744226+0000", 2024-05-08T08:53:07.949 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:07.949 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.949 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.949 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:07.950 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744226+0000", 2024-05-08T08:53:07.950 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3799999999999998e-07 2024-05-08T08:53:07.950 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.950 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.950 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:07.950 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744228+0000", 2024-05-08T08:53:07.950 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0710000000000002e-06 2024-05-08T08:53:07.950 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.950 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.950 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:07.950 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744233+0000", 2024-05-08T08:53:07.950 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0130000000000001e-06 2024-05-08T08:53:07.950 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.950 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.951 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:07.951 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744237+0000", 2024-05-08T08:53:07.951 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1930000000000002e-06 2024-05-08T08:53:07.951 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.951 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.951 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:07.951 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.801334+0000", 2024-05-08T08:53:07.951 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057097004999999999 2024-05-08T08:53:07.951 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.951 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.951 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:07.951 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.801414+0000", 2024-05-08T08:53:07.951 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0129000000000005e-05 2024-05-08T08:53:07.951 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.951 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.952 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:07.952 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.801521+0000", 2024-05-08T08:53:07.952 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010707 2024-05-08T08:53:07.952 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:07.952 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:07.952 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:07.952 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.952 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.952 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7491 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 40960~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:07.952 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.744242+0000", 2024-05-08T08:53:07.952 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 30.205738832000002, 2024-05-08T08:53:07.952 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057541677999999999, 2024-05-08T08:53:07.952 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:07.952 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:07.952 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:07.953 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:07.953 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:07.953 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:07.953 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7491 2024-05-08T08:53:07.953 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.953 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:07.953 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.953 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:07.953 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744242+0000", 2024-05-08T08:53:07.953 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:07.953 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.953 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.953 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:07.953 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744242+0000", 2024-05-08T08:53:07.953 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:07.954 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.954 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.954 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:07.954 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744242+0000", 2024-05-08T08:53:07.954 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.2000000000000001e-07 2024-05-08T08:53:07.954 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.954 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.954 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:07.954 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744244+0000", 2024-05-08T08:53:07.954 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6959999999999999e-06 2024-05-08T08:53:07.954 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.954 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.954 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:07.954 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744247+0000", 2024-05-08T08:53:07.954 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3000000000000002e-06 2024-05-08T08:53:07.955 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.955 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.955 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:07.955 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744250+0000", 2024-05-08T08:53:07.955 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.5399999999999998e-06 2024-05-08T08:53:07.955 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.955 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.955 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:07.955 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.801564+0000", 2024-05-08T08:53:07.955 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057313809 2024-05-08T08:53:07.955 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.955 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.955 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:07.955 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.801671+0000", 2024-05-08T08:53:07.955 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000107333 2024-05-08T08:53:07.956 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.956 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.956 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:07.956 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.801784+0000", 2024-05-08T08:53:07.956 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011268 2024-05-08T08:53:07.956 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:07.956 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:07.956 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:07.956 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.956 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.956 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7492 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 81920~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:07.956 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.744253+0000", 2024-05-08T08:53:07.956 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 30.205727355000001, 2024-05-08T08:53:07.957 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057761380000000001, 2024-05-08T08:53:07.957 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:07.957 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:07.957 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:07.957 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:07.957 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:07.957 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:07.957 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7492 2024-05-08T08:53:07.957 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.957 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:07.957 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.957 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:07.957 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744253+0000", 2024-05-08T08:53:07.957 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:07.957 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.957 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.958 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:07.958 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.744253+0000", 2024-05-08T08:53:07.958 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:07.958 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.958 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.958 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:07.958 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.745991+0000", 2024-05-08T08:53:07.958 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001738021 2024-05-08T08:53:07.958 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.958 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.958 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:07.958 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.745997+0000", 2024-05-08T08:53:07.958 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5300000000000004e-06 2024-05-08T08:53:07.959 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.959 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.959 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:07.959 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.746006+0000", 2024-05-08T08:53:07.959 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2210000000000003e-06 2024-05-08T08:53:07.959 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.959 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.959 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:07.959 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.746026+0000", 2024-05-08T08:53:07.959 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9487000000000002e-05 2024-05-08T08:53:07.959 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.959 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.959 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:07.959 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.801815+0000", 2024-05-08T08:53:07.959 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.055789196999999999 2024-05-08T08:53:07.960 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.960 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.960 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:07.960 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.801908+0000", 2024-05-08T08:53:07.960 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3320999999999996e-05 2024-05-08T08:53:07.960 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.960 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.960 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:07.960 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.802015+0000", 2024-05-08T08:53:07.960 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000106603 2024-05-08T08:53:07.960 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:07.960 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:07.960 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:07.960 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.960 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.961 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7499 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 368640~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:07.961 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.746209+0000", 2024-05-08T08:53:07.961 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 30.203771407000001, 2024-05-08T08:53:07.961 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057256452999999999, 2024-05-08T08:53:07.961 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:07.961 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:07.961 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:07.961 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:07.961 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:07.961 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:07.961 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7499 2024-05-08T08:53:07.961 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.961 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:07.961 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.962 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:07.962 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.746209+0000", 2024-05-08T08:53:07.962 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:07.962 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.962 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.962 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:07.962 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.746209+0000", 2024-05-08T08:53:07.962 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:07.962 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.962 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.962 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:07.962 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.746210+0000", 2024-05-08T08:53:07.962 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2100000000000002e-07 2024-05-08T08:53:07.963 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.963 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.963 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:07.963 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.746212+0000", 2024-05-08T08:53:07.963 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.922e-06 2024-05-08T08:53:07.963 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.963 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.963 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:07.963 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.746221+0000", 2024-05-08T08:53:07.963 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2900000000000008e-06 2024-05-08T08:53:07.963 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.963 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.963 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:07.963 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.746227+0000", 2024-05-08T08:53:07.964 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3740000000000003e-06 2024-05-08T08:53:07.964 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.964 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.964 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:07.964 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.803304+0000", 2024-05-08T08:53:07.964 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057076239000000001 2024-05-08T08:53:07.964 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.964 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.964 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:07.964 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.803380+0000", 2024-05-08T08:53:07.964 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.6309000000000004e-05 2024-05-08T08:53:07.964 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.964 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.964 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:07.964 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.803466+0000", 2024-05-08T08:53:07.965 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.5897999999999996e-05 2024-05-08T08:53:07.965 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:07.965 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:07.965 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:07.965 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.965 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.965 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7500 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 409600~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:07.965 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.746237+0000", 2024-05-08T08:53:07.965 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 30.203743825, 2024-05-08T08:53:07.965 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057429380000000002, 2024-05-08T08:53:07.965 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:07.965 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:07.965 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:07.965 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:07.966 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:07.966 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:07.966 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7500 2024-05-08T08:53:07.966 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.966 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:07.966 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.966 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:07.966 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.746237+0000", 2024-05-08T08:53:07.966 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:07.966 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.966 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.966 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:07.966 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.746237+0000", 2024-05-08T08:53:07.967 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:07.967 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.967 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.967 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:07.967 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.747959+0000", 2024-05-08T08:53:07.967 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001721888 2024-05-08T08:53:07.967 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.967 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.967 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:07.967 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.747964+0000", 2024-05-08T08:53:07.967 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0030000000000002e-06 2024-05-08T08:53:07.967 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.967 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.967 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:07.967 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.747976+0000", 2024-05-08T08:53:07.968 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2656999999999999e-05 2024-05-08T08:53:07.968 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.968 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.968 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:07.968 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.747983+0000", 2024-05-08T08:53:07.968 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.866e-06 2024-05-08T08:53:07.968 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.968 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.968 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:07.968 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.803495+0000", 2024-05-08T08:53:07.968 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.055511392999999999 2024-05-08T08:53:07.968 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.968 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.968 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:07.969 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.803574+0000", 2024-05-08T08:53:07.969 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9215000000000006e-05 2024-05-08T08:53:07.969 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.969 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.969 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:07.969 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.803666+0000", 2024-05-08T08:53:07.969 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2357999999999999e-05 2024-05-08T08:53:07.969 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:07.969 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:07.969 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:07.969 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.969 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.969 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7508 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 737280~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:07.969 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.748180+0000", 2024-05-08T08:53:07.969 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 30.201800767999998, 2024-05-08T08:53:07.970 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057419809000000002, 2024-05-08T08:53:07.970 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:07.970 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:07.970 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:07.970 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:07.970 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:07.970 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:07.970 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7508 2024-05-08T08:53:07.970 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.970 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:07.970 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.970 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:07.970 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748180+0000", 2024-05-08T08:53:07.970 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:07.971 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.971 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.971 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:07.971 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748180+0000", 2024-05-08T08:53:07.971 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:07.971 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.971 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.971 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:07.971 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748180+0000", 2024-05-08T08:53:07.971 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4499999999999997e-07 2024-05-08T08:53:07.971 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.971 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.971 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:07.971 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748183+0000", 2024-05-08T08:53:07.971 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1660000000000001e-06 2024-05-08T08:53:07.972 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.972 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.972 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:07.972 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748197+0000", 2024-05-08T08:53:07.972 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.4877e-05 2024-05-08T08:53:07.972 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.972 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.972 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:07.972 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748203+0000", 2024-05-08T08:53:07.972 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7470000000000002e-06 2024-05-08T08:53:07.972 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.972 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.972 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:07.972 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.805398+0000", 2024-05-08T08:53:07.972 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057194513000000002 2024-05-08T08:53:07.972 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.973 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.973 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:07.973 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.805544+0000", 2024-05-08T08:53:07.973 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000146522 2024-05-08T08:53:07.973 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.973 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.973 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:07.973 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.805600+0000", 2024-05-08T08:53:07.973 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5538999999999998e-05 2024-05-08T08:53:07.973 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:07.973 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:07.973 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:07.973 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.973 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.973 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7509 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 778240~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:07.974 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.748211+0000", 2024-05-08T08:53:07.974 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 30.201769699, 2024-05-08T08:53:07.974 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057619034999999999, 2024-05-08T08:53:07.974 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:07.974 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:07.974 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:07.974 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:07.974 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:07.974 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:07.974 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7509 2024-05-08T08:53:07.974 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.974 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:07.974 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.975 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:07.975 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748211+0000", 2024-05-08T08:53:07.975 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:07.975 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.975 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.975 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:07.975 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748211+0000", 2024-05-08T08:53:07.975 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:07.975 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.975 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.975 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:07.975 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748212+0000", 2024-05-08T08:53:07.975 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8100000000000003e-07 2024-05-08T08:53:07.976 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.976 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.976 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:07.976 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748214+0000", 2024-05-08T08:53:07.976 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4669999999999998e-06 2024-05-08T08:53:07.976 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.976 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.976 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:07.976 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748221+0000", 2024-05-08T08:53:07.976 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5769999999999999e-06 2024-05-08T08:53:07.976 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.976 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.976 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:07.976 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748225+0000", 2024-05-08T08:53:07.977 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8060000000000002e-06 2024-05-08T08:53:07.977 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.977 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.977 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:07.977 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.805636+0000", 2024-05-08T08:53:07.977 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057410309 2024-05-08T08:53:07.977 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.977 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.977 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:07.977 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.805723+0000", 2024-05-08T08:53:07.977 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7435999999999994e-05 2024-05-08T08:53:07.977 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.977 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.977 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:07.977 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.805830+0000", 2024-05-08T08:53:07.978 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000106959 2024-05-08T08:53:07.978 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:07.978 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:07.978 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:07.978 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.978 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.978 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7510 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 819200~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:07.978 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.748232+0000", 2024-05-08T08:53:07.978 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 30.201748669000001, 2024-05-08T08:53:07.978 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057710421999999997, 2024-05-08T08:53:07.978 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:07.978 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:07.978 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:07.979 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:07.979 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:07.979 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:07.979 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7510 2024-05-08T08:53:07.979 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.979 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:07.979 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.979 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:07.979 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748232+0000", 2024-05-08T08:53:07.979 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:07.979 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.979 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.979 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:07.979 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.748232+0000", 2024-05-08T08:53:07.980 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:07.980 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.980 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.980 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:07.980 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.749959+0000", 2024-05-08T08:53:07.980 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001727 2024-05-08T08:53:07.980 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.980 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.980 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:07.980 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.749964+0000", 2024-05-08T08:53:07.980 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4700000000000004e-06 2024-05-08T08:53:07.980 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.980 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.980 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:07.980 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.749984+0000", 2024-05-08T08:53:07.981 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0068000000000002e-05 2024-05-08T08:53:07.981 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.981 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.981 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:07.981 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.749993+0000", 2024-05-08T08:53:07.981 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4720000000000001e-06 2024-05-08T08:53:07.981 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.981 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.981 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:07.981 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.805846+0000", 2024-05-08T08:53:07.981 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.055852770000000003 2024-05-08T08:53:07.981 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.981 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.981 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:07.981 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.805890+0000", 2024-05-08T08:53:07.982 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4323999999999998e-05 2024-05-08T08:53:07.982 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.982 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.982 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:07.982 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.805942+0000", 2024-05-08T08:53:07.982 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2318000000000003e-05 2024-05-08T08:53:07.982 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:07.982 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:07.982 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:07.982 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.982 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.982 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7515 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 1024000~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:07.982 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.750135+0000", 2024-05-08T08:53:07.983 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 30.199845292999999, 2024-05-08T08:53:07.983 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057188125999999999, 2024-05-08T08:53:07.983 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:07.983 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:07.983 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:07.983 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:07.983 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:07.983 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:07.983 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7515 2024-05-08T08:53:07.983 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.983 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:07.983 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.983 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:07.983 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750135+0000", 2024-05-08T08:53:07.983 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:07.984 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.984 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.984 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:07.984 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750135+0000", 2024-05-08T08:53:07.984 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:07.984 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.984 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.984 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:07.984 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750136+0000", 2024-05-08T08:53:07.984 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8100000000000003e-07 2024-05-08T08:53:07.984 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.984 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.984 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:07.984 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750138+0000", 2024-05-08T08:53:07.984 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.339e-06 2024-05-08T08:53:07.985 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.985 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.985 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:07.985 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750147+0000", 2024-05-08T08:53:07.985 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7669999999999992e-06 2024-05-08T08:53:07.985 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.985 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.985 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:07.985 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750153+0000", 2024-05-08T08:53:07.985 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5029999999999997e-06 2024-05-08T08:53:07.985 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.985 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.985 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:07.985 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.807145+0000", 2024-05-08T08:53:07.985 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.056992174 2024-05-08T08:53:07.985 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.986 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.986 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:07.986 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.807231+0000", 2024-05-08T08:53:07.986 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.6347000000000005e-05 2024-05-08T08:53:07.986 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.986 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.986 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:07.986 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.807324+0000", 2024-05-08T08:53:07.986 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2515000000000004e-05 2024-05-08T08:53:07.986 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:07.986 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:07.986 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:07.986 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.987 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.987 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7516 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 1064960~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:07.987 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.750162+0000", 2024-05-08T08:53:07.987 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 30.199818464, 2024-05-08T08:53:07.987 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057363971999999999, 2024-05-08T08:53:07.987 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:07.987 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:07.987 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:07.987 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:07.987 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:07.987 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:07.987 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7516 2024-05-08T08:53:07.987 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.987 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:07.987 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.988 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:07.988 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750162+0000", 2024-05-08T08:53:07.988 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:07.988 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.988 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.988 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:07.988 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750162+0000", 2024-05-08T08:53:07.988 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:07.988 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.988 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.988 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:07.988 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750163+0000", 2024-05-08T08:53:07.988 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.13e-07 2024-05-08T08:53:07.988 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.988 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.989 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:07.989 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750165+0000", 2024-05-08T08:53:07.989 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2309999999999999e-06 2024-05-08T08:53:07.989 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.989 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.989 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:07.989 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750174+0000", 2024-05-08T08:53:07.989 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4740000000000005e-06 2024-05-08T08:53:07.989 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.989 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.989 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:07.989 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750180+0000", 2024-05-08T08:53:07.989 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4760000000000003e-06 2024-05-08T08:53:07.989 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.989 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.990 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:07.990 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.807355+0000", 2024-05-08T08:53:07.990 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057175029000000002 2024-05-08T08:53:07.990 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.990 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.990 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:07.990 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.807437+0000", 2024-05-08T08:53:07.990 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.2268999999999994e-05 2024-05-08T08:53:07.990 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.990 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.990 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:07.990 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.807526+0000", 2024-05-08T08:53:07.990 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.8880000000000003e-05 2024-05-08T08:53:07.990 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:07.991 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:07.991 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:07.991 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.991 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.991 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7517 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 1105920~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:07.991 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.750188+0000", 2024-05-08T08:53:07.991 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 30.199792971000001, 2024-05-08T08:53:07.991 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057547099999999997, 2024-05-08T08:53:07.991 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:07.991 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:07.991 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:07.991 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:07.991 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:07.991 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:07.992 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7517 2024-05-08T08:53:07.992 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.992 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:07.992 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.992 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:07.992 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750188+0000", 2024-05-08T08:53:07.992 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:07.992 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.992 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.992 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:07.992 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750188+0000", 2024-05-08T08:53:07.992 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:07.992 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.992 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.992 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:07.992 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750188+0000", 2024-05-08T08:53:07.993 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7199999999999999e-07 2024-05-08T08:53:07.993 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.993 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.993 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:07.993 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750190+0000", 2024-05-08T08:53:07.993 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.154e-06 2024-05-08T08:53:07.993 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.993 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.993 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:07.993 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750198+0000", 2024-05-08T08:53:07.993 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8320000000000006e-06 2024-05-08T08:53:07.993 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.993 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.993 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:07.993 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750203+0000", 2024-05-08T08:53:07.994 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1900000000000003e-06 2024-05-08T08:53:07.994 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.994 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.994 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:07.994 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.807555+0000", 2024-05-08T08:53:07.994 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057351148999999997 2024-05-08T08:53:07.994 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.994 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.994 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:07.994 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.807640+0000", 2024-05-08T08:53:07.994 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.5853999999999995e-05 2024-05-08T08:53:07.994 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.994 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.994 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:07.995 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.807735+0000", 2024-05-08T08:53:07.995 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4449000000000003e-05 2024-05-08T08:53:07.995 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:07.995 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:07.995 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:07.995 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.995 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.995 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7518 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 1146880~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:07.995 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.750212+0000", 2024-05-08T08:53:07.995 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 30.199769171, 2024-05-08T08:53:07.995 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057664963999999999, 2024-05-08T08:53:07.995 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:07.995 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:07.995 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:07.996 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:07.996 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:07.996 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:07.996 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7518 2024-05-08T08:53:07.996 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.996 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:07.996 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.996 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:07.996 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750212+0000", 2024-05-08T08:53:07.996 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:07.996 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.996 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.996 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:07.996 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750212+0000", 2024-05-08T08:53:07.996 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:07.997 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.997 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.997 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:07.997 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750212+0000", 2024-05-08T08:53:07.997 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.46e-07 2024-05-08T08:53:07.997 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.997 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.997 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:07.997 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750220+0000", 2024-05-08T08:53:07.997 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5610000000000001e-06 2024-05-08T08:53:07.997 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.997 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.997 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:07.997 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750228+0000", 2024-05-08T08:53:07.997 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3180000000000007e-06 2024-05-08T08:53:07.997 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.998 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.998 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:07.998 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750234+0000", 2024-05-08T08:53:07.998 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0209999999999996e-06 2024-05-08T08:53:07.998 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.998 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.998 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:07.998 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.807765+0000", 2024-05-08T08:53:07.998 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057531481000000002 2024-05-08T08:53:07.998 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.998 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.998 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:07.998 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.807815+0000", 2024-05-08T08:53:07.998 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9282e-05 2024-05-08T08:53:07.999 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.999 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.999 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:07.999 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.807877+0000", 2024-05-08T08:53:07.999 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1854999999999996e-05 2024-05-08T08:53:07.999 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:07.999 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:07.999 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:07.999 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:07.999 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:07.999 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7519 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 1187840~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:07.999 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.750261+0000", 2024-05-08T08:53:07.999 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 30.199719685000002, 2024-05-08T08:53:07.999 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057872519999999997, 2024-05-08T08:53:08.000 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:08.000 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:08.000 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:08.000 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:08.000 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:08.000 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:08.000 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7519 2024-05-08T08:53:08.000 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:08.000 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:08.000 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:08.000 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:08.000 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750261+0000", 2024-05-08T08:53:08.000 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:08.000 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:08.001 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:08.001 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:08.001 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750261+0000", 2024-05-08T08:53:08.001 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:08.001 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:08.001 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:08.001 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:08.001 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750262+0000", 2024-05-08T08:53:08.001 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7000000000000005e-07 2024-05-08T08:53:08.001 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:08.001 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:08.001 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:08.001 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750265+0000", 2024-05-08T08:53:08.001 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4589999999999998e-06 2024-05-08T08:53:08.001 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:08.001 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:08.002 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:08.002 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750274+0000", 2024-05-08T08:53:08.002 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.5459999999999992e-06 2024-05-08T08:53:08.002 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:08.002 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:08.002 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:08.002 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750280+0000", 2024-05-08T08:53:08.002 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0950000000000001e-06 2024-05-08T08:53:08.002 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:08.002 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:08.002 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:08.002 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.807917+0000", 2024-05-08T08:53:08.002 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057637715999999999 2024-05-08T08:53:08.002 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:08.003 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:08.003 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:08.003 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.808013+0000", 2024-05-08T08:53:08.003 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5569999999999995e-05 2024-05-08T08:53:08.003 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:08.003 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:08.003 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:08.003 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.808134+0000", 2024-05-08T08:53:08.003 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012056400000000001 2024-05-08T08:53:08.003 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:08.003 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:08.003 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:08.003 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:08.003 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:08.004 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7520 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 1228800~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:08.004 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.750289+0000", 2024-05-08T08:53:08.004 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 30.199691241, 2024-05-08T08:53:08.004 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.058133788999999998, 2024-05-08T08:53:08.004 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:08.004 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:08.004 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:08.004 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:08.004 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:08.004 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:08.004 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7520 2024-05-08T08:53:08.004 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:08.004 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:08.004 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:08.005 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:08.005 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750289+0000", 2024-05-08T08:53:08.005 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:08.005 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:08.005 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:08.005 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:08.005 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.750289+0000", 2024-05-08T08:53:08.005 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:08.005 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:08.005 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:08.005 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:08.005 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.751976+0000", 2024-05-08T08:53:08.005 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0016865389999999999 2024-05-08T08:53:08.005 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:08.005 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:08.006 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:08.006 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.751981+0000", 2024-05-08T08:53:08.006 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5719999999999996e-06 2024-05-08T08:53:08.006 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:08.006 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:08.006 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:08.006 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.751991+0000", 2024-05-08T08:53:08.006 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0788000000000001e-05 2024-05-08T08:53:08.006 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:08.006 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:08.006 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:08.006 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.751999+0000", 2024-05-08T08:53:08.006 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2690000000000002e-06 2024-05-08T08:53:08.006 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:08.007 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:08.007 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:08.007 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.808183+0000", 2024-05-08T08:53:08.007 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.056184686999999997 2024-05-08T08:53:08.007 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:08.007 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:08.007 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:08.007 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.808296+0000", 2024-05-08T08:53:08.007 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000112754 2024-05-08T08:53:08.007 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:08.007 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:08.007 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:08.007 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.808423+0000", 2024-05-08T08:53:08.007 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012718000000000001 2024-05-08T08:53:08.008 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:08.008 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:08.008 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:08.008 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:08.008 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:08.008 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7524 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 1392640~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:08.008 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.752121+0000", 2024-05-08T08:53:08.008 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 30.197860192, 2024-05-08T08:53:08.008 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057199521000000003, 2024-05-08T08:53:08.008 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:08.008 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:08.008 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:08.008 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:08.008 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:08.008 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:08.009 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7524 2024-05-08T08:53:08.009 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:08.009 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:08.009 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:08.009 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:08.009 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752121+0000", 2024-05-08T08:53:08.009 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:08.009 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:08.009 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:08.009 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:08.009 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752121+0000", 2024-05-08T08:53:08.009 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:08.009 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:08.009 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:08.009 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:08.009 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752121+0000", 2024-05-08T08:53:08.009 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5999999999999999e-07 2024-05-08T08:53:08.010 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:08.010 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:08.010 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:08.010 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752123+0000", 2024-05-08T08:53:08.010 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3499999999999999e-06 2024-05-08T08:53:08.010 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:08.010 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:08.010 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:08.010 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752132+0000", 2024-05-08T08:53:08.010 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.5539999999999998e-06 2024-05-08T08:53:08.010 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:08.010 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:08.010 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:08.010 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752138+0000", 2024-05-08T08:53:08.010 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3640000000000004e-06 2024-05-08T08:53:08.011 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:08.011 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:08.011 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:08.011 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.809210+0000", 2024-05-08T08:53:08.011 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057071469999999999 2024-05-08T08:53:08.011 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:08.011 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:08.011 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:08.011 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.809258+0000", 2024-05-08T08:53:08.011 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8755000000000003e-05 2024-05-08T08:53:08.011 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:08.011 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:08.011 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:08.011 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.809320+0000", 2024-05-08T08:53:08.012 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1568000000000004e-05 2024-05-08T08:53:08.012 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:08.012 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:08.012 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:08.012 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:08.012 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:08.012 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7525 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 1433600~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:08.012 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.752146+0000", 2024-05-08T08:53:08.012 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 30.197835147999999, 2024-05-08T08:53:08.012 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057400072000000003, 2024-05-08T08:53:08.012 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:08.012 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:08.012 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:08.012 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:08.013 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:08.013 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:08.013 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7525 2024-05-08T08:53:08.013 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:08.013 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:08.013 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:08.013 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:08.013 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752146+0000", 2024-05-08T08:53:08.013 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:08.013 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:08.013 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:08.013 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:08.013 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752146+0000", 2024-05-08T08:53:08.013 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:08.013 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:08.014 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:08.014 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:08.014 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752146+0000", 2024-05-08T08:53:08.014 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0400000000000002e-07 2024-05-08T08:53:08.014 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:08.014 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:08.014 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:08.014 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752148+0000", 2024-05-08T08:53:08.014 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3209999999999999e-06 2024-05-08T08:53:08.014 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:08.014 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:08.014 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:08.014 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752155+0000", 2024-05-08T08:53:08.014 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6109999999999999e-06 2024-05-08T08:53:08.015 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:08.015 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:08.015 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:08.015 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752161+0000", 2024-05-08T08:53:08.015 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.489e-06 2024-05-08T08:53:08.015 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:08.015 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:08.015 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:08.015 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.809360+0000", 2024-05-08T08:53:08.015 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057198860999999997 2024-05-08T08:53:08.015 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:08.015 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:08.015 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:08.016 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.809450+0000", 2024-05-08T08:53:08.016 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0136000000000006e-05 2024-05-08T08:53:08.016 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:08.016 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:08.016 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:08.016 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.809546+0000", 2024-05-08T08:53:08.016 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5249999999999998e-05 2024-05-08T08:53:08.016 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:08.016 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:08.016 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:08.016 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:08.016 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:08.016 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7526 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 1474560~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:08.016 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.752168+0000", 2024-05-08T08:53:08.017 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 30.197812434999999, 2024-05-08T08:53:08.017 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057516818999999997, 2024-05-08T08:53:08.017 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:08.017 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:08.017 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:08.017 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:08.017 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:08.017 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:08.017 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7526 2024-05-08T08:53:08.017 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:08.017 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:08.017 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:08.017 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:08.017 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752168+0000", 2024-05-08T08:53:08.018 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:08.018 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:08.018 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:08.018 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:08.018 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752168+0000", 2024-05-08T08:53:08.018 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:08.018 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:08.018 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:08.018 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:08.018 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752169+0000", 2024-05-08T08:53:08.018 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4799999999999999e-07 2024-05-08T08:53:08.018 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:08.018 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:08.019 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:08.019 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752171+0000", 2024-05-08T08:53:08.019 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.289e-06 2024-05-08T08:53:08.019 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:08.019 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:08.019 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:08.019 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752177+0000", 2024-05-08T08:53:08.019 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3690000000000003e-06 2024-05-08T08:53:08.019 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:08.019 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:08.019 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:08.019 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752183+0000", 2024-05-08T08:53:08.019 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.152e-06 2024-05-08T08:53:08.019 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:08.019 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:08.020 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:08.020 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.809574+0000", 2024-05-08T08:53:08.020 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057391075 2024-05-08T08:53:08.020 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:08.020 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:08.020 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:08.020 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.809624+0000", 2024-05-08T08:53:08.020 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0673000000000001e-05 2024-05-08T08:53:08.020 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:08.020 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:08.020 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:08.020 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.809685+0000", 2024-05-08T08:53:08.020 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0813000000000002e-05 2024-05-08T08:53:08.020 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:08.020 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:08.021 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:08.021 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:08.021 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:08.021 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7527 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 1515520~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:08.021 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.752189+0000", 2024-05-08T08:53:08.021 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 30.197791507000002, 2024-05-08T08:53:08.021 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057722168999999997, 2024-05-08T08:53:08.021 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:08.021 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:08.021 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:08.021 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:08.021 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:08.021 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:08.021 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7527 2024-05-08T08:53:08.022 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:08.022 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:08.022 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:08.022 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:08.022 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752189+0000", 2024-05-08T08:53:08.022 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:08.022 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:08.022 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:08.022 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:08.022 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752189+0000", 2024-05-08T08:53:08.022 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:08.022 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:08.022 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:08.022 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:08.022 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752190+0000", 2024-05-08T08:53:08.023 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8799999999999998e-07 2024-05-08T08:53:08.023 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:08.023 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:08.023 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:08.023 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752192+0000", 2024-05-08T08:53:08.023 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9759999999999998e-06 2024-05-08T08:53:08.023 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:08.023 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:08.023 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:08.023 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752199+0000", 2024-05-08T08:53:08.023 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3289999999999996e-06 2024-05-08T08:53:08.023 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:08.023 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:08.023 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:08.023 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752205+0000", 2024-05-08T08:53:08.024 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2759999999999997e-06 2024-05-08T08:53:08.024 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:08.024 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:08.024 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:08.024 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.809726+0000", 2024-05-08T08:53:08.024 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057521249000000003 2024-05-08T08:53:08.024 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:08.024 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:08.024 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:08.024 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.809815+0000", 2024-05-08T08:53:08.024 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.8987999999999996e-05 2024-05-08T08:53:08.024 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:08.024 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:08.024 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:08.024 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.809911+0000", 2024-05-08T08:53:08.025 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5963000000000003e-05 2024-05-08T08:53:08.025 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:08.025 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:08.025 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:08.025 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:08.025 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:08.025 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7528 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 1556480~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:08.025 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.752212+0000", 2024-05-08T08:53:08.025 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 30.197768796999998, 2024-05-08T08:53:08.025 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057904956, 2024-05-08T08:53:08.025 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:08.025 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:08.025 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:08.025 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:08.025 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:08.026 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:08.026 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7528 2024-05-08T08:53:08.026 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:08.026 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:08.026 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:08.026 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:08.026 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752212+0000", 2024-05-08T08:53:08.026 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:08.026 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:08.026 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:08.026 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:08.026 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752212+0000", 2024-05-08T08:53:08.026 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:08.026 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:08.027 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:08.027 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:08.027 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752212+0000", 2024-05-08T08:53:08.027 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7800000000000002e-07 2024-05-08T08:53:08.027 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:08.027 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:08.027 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:08.027 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752215+0000", 2024-05-08T08:53:08.027 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2120000000000002e-06 2024-05-08T08:53:08.027 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:08.027 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:08.027 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:08.027 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752222+0000", 2024-05-08T08:53:08.027 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.6329999999999996e-06 2024-05-08T08:53:08.028 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:08.028 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:08.028 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:08.028 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752227+0000", 2024-05-08T08:53:08.028 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9289999999999997e-06 2024-05-08T08:53:08.028 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:08.028 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:08.028 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:08.028 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.809966+0000", 2024-05-08T08:53:08.028 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057738554999999997 2024-05-08T08:53:08.028 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:08.028 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:08.028 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:08.028 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.810052+0000", 2024-05-08T08:53:08.028 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.6540999999999997e-05 2024-05-08T08:53:08.029 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:08.029 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:08.029 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:08.029 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.810117+0000", 2024-05-08T08:53:08.029 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4608000000000002e-05 2024-05-08T08:53:08.029 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:08.029 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:08.029 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:08.029 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:08.029 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:08.029 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:7529 3.18 3:1e75630d:::rbd_data.11402f152156.0000000000000001:8 [read 1597440~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:08.029 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:37.752234+0000", 2024-05-08T08:53:08.029 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 30.197746784, 2024-05-08T08:53:08.029 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.058133851, 2024-05-08T08:53:08.029 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:08.030 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:08.030 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:08.030 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:08.030 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:08.030 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:08.030 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7529 2024-05-08T08:53:08.030 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:08.030 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:08.030 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:08.030 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:08.030 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752234+0000", 2024-05-08T08:53:08.030 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:08.031 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:08.031 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:08.031 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:08.031 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.752234+0000", 2024-05-08T08:53:08.031 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:08.031 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:08.031 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:08.031 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:08.031 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.753959+0000", 2024-05-08T08:53:08.031 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0017251409999999999 2024-05-08T08:53:08.031 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:08.031 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:08.031 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:08.031 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.753963+0000", 2024-05-08T08:53:08.032 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2860000000000001e-06 2024-05-08T08:53:08.032 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:08.032 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:08.032 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:08.032 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.753973+0000", 2024-05-08T08:53:08.032 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5119999999999997e-06 2024-05-08T08:53:08.032 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:08.032 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:08.032 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:08.032 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.753979+0000", 2024-05-08T08:53:08.032 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3589999999999996e-06 2024-05-08T08:53:08.032 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:08.032 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:08.032 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:08.032 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.810182+0000", 2024-05-08T08:53:08.033 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.056202941999999999 2024-05-08T08:53:08.033 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:08.033 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:08.033 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:08.033 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.810270+0000", 2024-05-08T08:53:08.033 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7338999999999998e-05 2024-05-08T08:53:08.033 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:08.033 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:08.033 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:08.033 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:37.810368+0000", 2024-05-08T08:53:08.033 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8271999999999997e-05 2024-05-08T08:53:08.033 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:08.033 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:08.033 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:08.034 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:08.034 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:08.034 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:08.035 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:53:08.036 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:08.038+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:08.135 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:53:08.139+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:53:08.143 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:53:08.143 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:53:08.143 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:53:08.143 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:53:08.152 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:53:08.237 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:08.240+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:08.256 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:53:08.257 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:53:08.257 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:53:08.257 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:53:08.257 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:53:08.265 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:53:08.339 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:08.341+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:08.371 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:53:08.372 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:53:08.372 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:53:08.372 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:53:08.372 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.372 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4742 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1832960~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:08.372 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.585513+0000", 2024-05-08T08:53:08.372 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 36.788351159999998, 2024-05-08T08:53:08.372 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035588763000000002, 2024-05-08T08:53:08.372 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:08.372 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:08.372 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:08.372 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:08.372 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:08.372 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:08.372 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4742 2024-05-08T08:53:08.373 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.373 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:08.373 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.373 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:08.373 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585513+0000", 2024-05-08T08:53:08.373 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:08.373 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.373 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.373 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:08.373 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585513+0000", 2024-05-08T08:53:08.373 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:08.373 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.373 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.373 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:08.373 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585514+0000", 2024-05-08T08:53:08.374 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6100000000000001e-07 2024-05-08T08:53:08.374 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.374 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.374 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:08.374 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585516+0000", 2024-05-08T08:53:08.374 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.526e-06 2024-05-08T08:53:08.374 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.374 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.374 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:08.374 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585523+0000", 2024-05-08T08:53:08.374 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.562e-06 2024-05-08T08:53:08.374 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.374 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.375 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:08.375 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585528+0000", 2024-05-08T08:53:08.375 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7990000000000001e-06 2024-05-08T08:53:08.375 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.375 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.375 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:08.375 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.620939+0000", 2024-05-08T08:53:08.375 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035411419 2024-05-08T08:53:08.375 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.375 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.375 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:08.375 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621014+0000", 2024-05-08T08:53:08.375 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5017000000000004e-05 2024-05-08T08:53:08.375 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.376 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.376 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:08.376 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621102+0000", 2024-05-08T08:53:08.376 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7979000000000006e-05 2024-05-08T08:53:08.376 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.376 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:08.376 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.376 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.376 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.376 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4743 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1849344~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:08.376 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.585628+0000", 2024-05-08T08:53:08.376 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 36.788236273000003, 2024-05-08T08:53:08.376 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035659962000000003, 2024-05-08T08:53:08.377 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:08.377 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:08.377 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:08.377 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:08.377 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:08.377 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:08.377 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4743 2024-05-08T08:53:08.377 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.377 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:08.377 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.377 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:08.377 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585628+0000", 2024-05-08T08:53:08.377 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:08.377 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.378 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.378 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:08.378 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585628+0000", 2024-05-08T08:53:08.378 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:08.378 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.378 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.378 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:08.378 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585629+0000", 2024-05-08T08:53:08.378 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2300000000000001e-07 2024-05-08T08:53:08.378 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.378 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.379 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:08.379 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585632+0000", 2024-05-08T08:53:08.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1370000000000002e-06 2024-05-08T08:53:08.379 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.379 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.379 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:08.379 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585641+0000", 2024-05-08T08:53:08.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5220000000000007e-06 2024-05-08T08:53:08.379 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.379 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.379 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:08.380 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585646+0000", 2024-05-08T08:53:08.380 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.609e-06 2024-05-08T08:53:08.380 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.380 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.380 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:08.380 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621132+0000", 2024-05-08T08:53:08.381 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035485472999999997 2024-05-08T08:53:08.381 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.381 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.381 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:08.381 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621204+0000", 2024-05-08T08:53:08.381 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2259000000000001e-05 2024-05-08T08:53:08.381 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.381 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.381 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:08.381 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621288+0000", 2024-05-08T08:53:08.381 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4338999999999993e-05 2024-05-08T08:53:08.381 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.381 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:08.381 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.382 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.382 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.382 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4746 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1865728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:08.382 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.585656+0000", 2024-05-08T08:53:08.382 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 36.788208591, 2024-05-08T08:53:08.382 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035815909, 2024-05-08T08:53:08.382 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:08.382 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:08.382 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:08.382 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:08.382 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:08.382 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:08.382 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4746 2024-05-08T08:53:08.383 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.383 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:08.383 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.383 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:08.383 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585656+0000", 2024-05-08T08:53:08.383 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:08.383 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.383 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.383 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:08.383 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585656+0000", 2024-05-08T08:53:08.384 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:08.384 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.384 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:08.384 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585656+0000", 2024-05-08T08:53:08.384 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.08e-07 2024-05-08T08:53:08.384 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.384 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:08.384 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585658+0000", 2024-05-08T08:53:08.384 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0889999999999998e-06 2024-05-08T08:53:08.384 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.384 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:08.384 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585664+0000", 2024-05-08T08:53:08.385 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3990000000000003e-06 2024-05-08T08:53:08.385 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.385 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.385 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:08.385 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585668+0000", 2024-05-08T08:53:08.385 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3170000000000003e-06 2024-05-08T08:53:08.385 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.385 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.385 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:08.385 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621316+0000", 2024-05-08T08:53:08.385 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035647908999999998 2024-05-08T08:53:08.385 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.385 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.385 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:08.385 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621389+0000", 2024-05-08T08:53:08.385 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2531000000000001e-05 2024-05-08T08:53:08.386 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.386 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.386 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:08.386 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621472+0000", 2024-05-08T08:53:08.386 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3256000000000004e-05 2024-05-08T08:53:08.386 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.386 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:08.386 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.386 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.386 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.386 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4748 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1882112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:08.386 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.585697+0000", 2024-05-08T08:53:08.386 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 36.788167145000003, 2024-05-08T08:53:08.386 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035962485000000002, 2024-05-08T08:53:08.387 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:08.387 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:08.387 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:08.387 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:08.387 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:08.387 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:08.387 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4748 2024-05-08T08:53:08.387 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.387 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:08.387 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.387 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:08.387 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585697+0000", 2024-05-08T08:53:08.387 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:08.387 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.387 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.387 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:08.388 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585697+0000", 2024-05-08T08:53:08.388 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:08.388 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.388 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.388 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:08.388 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585698+0000", 2024-05-08T08:53:08.388 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0900000000000001e-07 2024-05-08T08:53:08.388 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.388 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.388 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:08.388 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585701+0000", 2024-05-08T08:53:08.388 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0149999999999999e-06 2024-05-08T08:53:08.388 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.388 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.389 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:08.389 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585712+0000", 2024-05-08T08:53:08.389 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0655e-05 2024-05-08T08:53:08.389 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.389 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.389 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:08.389 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585717+0000", 2024-05-08T08:53:08.389 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3800000000000002e-06 2024-05-08T08:53:08.389 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.389 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.389 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:08.389 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621502+0000", 2024-05-08T08:53:08.389 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035784559000000001 2024-05-08T08:53:08.389 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.389 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.390 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:08.390 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621575+0000", 2024-05-08T08:53:08.390 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3269999999999995e-05 2024-05-08T08:53:08.390 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.390 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.390 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:08.390 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621660+0000", 2024-05-08T08:53:08.390 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4997e-05 2024-05-08T08:53:08.390 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.390 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:08.390 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.390 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.390 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.390 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4750 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1898496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:08.391 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.585810+0000", 2024-05-08T08:53:08.391 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 36.788054314999997, 2024-05-08T08:53:08.391 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036034928000000001, 2024-05-08T08:53:08.391 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:08.391 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:08.391 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:08.391 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:08.391 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:08.391 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:08.391 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4750 2024-05-08T08:53:08.391 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.391 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:08.391 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.391 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:08.391 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585810+0000", 2024-05-08T08:53:08.391 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:08.392 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.392 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.392 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:08.392 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585810+0000", 2024-05-08T08:53:08.392 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:08.392 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.392 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.392 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:08.392 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585811+0000", 2024-05-08T08:53:08.392 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6499999999999999e-07 2024-05-08T08:53:08.392 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.392 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.392 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:08.392 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585814+0000", 2024-05-08T08:53:08.392 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1709999999999998e-06 2024-05-08T08:53:08.393 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.393 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.393 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:08.393 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585822+0000", 2024-05-08T08:53:08.393 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8909999999999995e-06 2024-05-08T08:53:08.393 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.393 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.393 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:08.393 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585827+0000", 2024-05-08T08:53:08.393 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5289999999999999e-06 2024-05-08T08:53:08.393 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.393 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.393 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:08.393 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621688+0000", 2024-05-08T08:53:08.393 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035861111000000001 2024-05-08T08:53:08.394 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.394 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.394 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:08.394 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621761+0000", 2024-05-08T08:53:08.394 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2211000000000004e-05 2024-05-08T08:53:08.394 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.394 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.394 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:08.394 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621845+0000", 2024-05-08T08:53:08.394 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4450000000000006e-05 2024-05-08T08:53:08.394 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.394 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:08.394 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.394 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.395 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.395 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4752 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1914880~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:08.395 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.585858+0000", 2024-05-08T08:53:08.395 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 36.788006346000003, 2024-05-08T08:53:08.395 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036180081000000003, 2024-05-08T08:53:08.395 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:08.395 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:08.395 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:08.395 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:08.395 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:08.395 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:08.396 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4752 2024-05-08T08:53:08.396 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.396 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:08.396 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.396 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:08.396 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585858+0000", 2024-05-08T08:53:08.396 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:08.396 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.396 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.396 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:08.397 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585858+0000", 2024-05-08T08:53:08.397 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:08.397 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.397 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.397 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:08.397 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585859+0000", 2024-05-08T08:53:08.397 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.82e-07 2024-05-08T08:53:08.397 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.397 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.397 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:08.397 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585861+0000", 2024-05-08T08:53:08.397 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.695e-06 2024-05-08T08:53:08.397 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.397 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.397 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:08.398 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585869+0000", 2024-05-08T08:53:08.398 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0299999999999994e-06 2024-05-08T08:53:08.398 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.398 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.398 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:08.398 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585874+0000", 2024-05-08T08:53:08.398 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9620000000000001e-06 2024-05-08T08:53:08.398 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.398 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.398 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:08.398 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621878+0000", 2024-05-08T08:53:08.398 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036004043999999999 2024-05-08T08:53:08.398 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.399 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.399 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:08.399 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621954+0000", 2024-05-08T08:53:08.399 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5414000000000007e-05 2024-05-08T08:53:08.399 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.399 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.399 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:08.399 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622038+0000", 2024-05-08T08:53:08.399 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4454000000000002e-05 2024-05-08T08:53:08.399 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.399 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:08.399 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.399 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.399 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.399 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4753 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1931264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:08.400 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.585947+0000", 2024-05-08T08:53:08.400 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 36.787917698000001, 2024-05-08T08:53:08.400 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036267992999999998, 2024-05-08T08:53:08.400 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:08.400 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:08.400 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:08.400 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:08.400 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:08.400 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:08.400 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4753 2024-05-08T08:53:08.400 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.400 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:08.400 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.400 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:08.401 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585947+0000", 2024-05-08T08:53:08.401 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:08.401 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.401 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.401 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:08.401 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585947+0000", 2024-05-08T08:53:08.401 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:08.401 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.401 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.401 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:08.401 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585948+0000", 2024-05-08T08:53:08.401 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6000000000000002e-07 2024-05-08T08:53:08.401 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.401 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.401 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:08.402 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585952+0000", 2024-05-08T08:53:08.402 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7090000000000001e-06 2024-05-08T08:53:08.402 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.402 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.402 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:08.402 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585963+0000", 2024-05-08T08:53:08.402 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0740000000000001e-05 2024-05-08T08:53:08.402 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.402 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.402 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:08.402 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585970+0000", 2024-05-08T08:53:08.402 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3309999999999998e-06 2024-05-08T08:53:08.402 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.403 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.403 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:08.403 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622065+0000", 2024-05-08T08:53:08.403 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036094676999999999 2024-05-08T08:53:08.403 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.403 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.403 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:08.403 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622132+0000", 2024-05-08T08:53:08.403 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7214999999999999e-05 2024-05-08T08:53:08.403 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.403 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.403 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:08.403 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622215+0000", 2024-05-08T08:53:08.403 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2460999999999995e-05 2024-05-08T08:53:08.403 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.404 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:08.404 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.404 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.404 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.404 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4756 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1947648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:08.404 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586018+0000", 2024-05-08T08:53:08.404 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 36.787846053999999, 2024-05-08T08:53:08.404 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036382335000000002, 2024-05-08T08:53:08.404 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:08.404 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:08.404 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:08.404 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:08.404 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:08.404 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:08.404 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4756 2024-05-08T08:53:08.405 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.405 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:08.405 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.405 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:08.405 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586018+0000", 2024-05-08T08:53:08.405 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:08.405 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.405 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.405 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:08.405 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586018+0000", 2024-05-08T08:53:08.405 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:08.405 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.405 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.405 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:08.405 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586019+0000", 2024-05-08T08:53:08.406 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9999999999999997e-07 2024-05-08T08:53:08.406 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.406 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.406 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:08.406 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586023+0000", 2024-05-08T08:53:08.406 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7330000000000001e-06 2024-05-08T08:53:08.406 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.406 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.406 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:08.406 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586031+0000", 2024-05-08T08:53:08.406 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4940000000000003e-06 2024-05-08T08:53:08.406 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.406 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.406 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:08.407 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586038+0000", 2024-05-08T08:53:08.407 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2539999999999998e-06 2024-05-08T08:53:08.407 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.407 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.407 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:08.407 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622243+0000", 2024-05-08T08:53:08.407 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036205009000000003 2024-05-08T08:53:08.407 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.407 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.407 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:08.407 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622313+0000", 2024-05-08T08:53:08.407 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9912000000000004e-05 2024-05-08T08:53:08.407 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.407 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.407 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:08.407 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622401+0000", 2024-05-08T08:53:08.408 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8232999999999994e-05 2024-05-08T08:53:08.408 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.408 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:08.408 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.408 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.408 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.408 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4758 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1964032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:08.408 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586080+0000", 2024-05-08T08:53:08.408 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 36.787784635000001, 2024-05-08T08:53:08.408 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036547659000000003, 2024-05-08T08:53:08.408 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:08.408 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:08.408 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:08.408 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:08.409 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:08.409 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:08.409 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4758 2024-05-08T08:53:08.409 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.409 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:08.409 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.409 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:08.409 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586080+0000", 2024-05-08T08:53:08.409 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:08.409 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.409 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.410 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:08.410 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586080+0000", 2024-05-08T08:53:08.410 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:08.410 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.410 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.410 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:08.410 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586081+0000", 2024-05-08T08:53:08.410 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5499999999999998e-07 2024-05-08T08:53:08.410 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.410 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.410 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:08.410 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586084+0000", 2024-05-08T08:53:08.410 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9919999999999999e-06 2024-05-08T08:53:08.410 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.410 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.411 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:08.411 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586091+0000", 2024-05-08T08:53:08.411 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5440000000000001e-06 2024-05-08T08:53:08.411 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.411 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.411 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:08.411 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586098+0000", 2024-05-08T08:53:08.411 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8639999999999998e-06 2024-05-08T08:53:08.411 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.411 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.411 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:08.411 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622442+0000", 2024-05-08T08:53:08.411 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036344110999999998 2024-05-08T08:53:08.411 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.411 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.412 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:08.412 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622516+0000", 2024-05-08T08:53:08.412 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4089000000000002e-05 2024-05-08T08:53:08.412 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.412 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.412 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:08.412 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622628+0000", 2024-05-08T08:53:08.412 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00011140400000000001 2024-05-08T08:53:08.412 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.412 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:08.412 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.412 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.412 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.413 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4760 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1980416~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:08.413 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586178+0000", 2024-05-08T08:53:08.413 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 36.787686022999999, 2024-05-08T08:53:08.413 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036647526999999999, 2024-05-08T08:53:08.413 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:08.413 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:08.413 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:08.413 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:08.413 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:08.413 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:08.413 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4760 2024-05-08T08:53:08.413 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.413 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:08.414 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.414 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:08.414 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586178+0000", 2024-05-08T08:53:08.414 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:08.414 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.414 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.414 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:08.414 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586178+0000", 2024-05-08T08:53:08.414 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:08.414 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.414 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.414 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:08.414 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586179+0000", 2024-05-08T08:53:08.415 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8599999999999997e-07 2024-05-08T08:53:08.415 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.415 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.415 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:08.415 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586182+0000", 2024-05-08T08:53:08.415 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.174e-06 2024-05-08T08:53:08.415 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.415 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.415 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:08.415 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586191+0000", 2024-05-08T08:53:08.415 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1869999999999997e-06 2024-05-08T08:53:08.415 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.415 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.415 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:08.416 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586197+0000", 2024-05-08T08:53:08.416 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2090000000000002e-06 2024-05-08T08:53:08.416 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.416 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.416 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:08.416 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622657+0000", 2024-05-08T08:53:08.416 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036459793999999997 2024-05-08T08:53:08.416 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.416 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.416 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:08.416 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622725+0000", 2024-05-08T08:53:08.416 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8374999999999995e-05 2024-05-08T08:53:08.416 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.416 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.416 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:08.417 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622826+0000", 2024-05-08T08:53:08.417 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010100200000000001 2024-05-08T08:53:08.417 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.417 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:08.417 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.417 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.417 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.417 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4762 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1996800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:08.417 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586299+0000", 2024-05-08T08:53:08.417 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 36.787565692000001, 2024-05-08T08:53:08.417 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036720653999999998, 2024-05-08T08:53:08.417 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:08.417 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:08.418 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:08.418 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:08.418 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:08.418 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:08.418 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4762 2024-05-08T08:53:08.418 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.418 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:08.418 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.418 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:08.418 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586299+0000", 2024-05-08T08:53:08.418 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:08.418 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.418 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.418 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:08.419 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586299+0000", 2024-05-08T08:53:08.419 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:08.419 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.419 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.419 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:08.419 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586299+0000", 2024-05-08T08:53:08.419 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.99e-07 2024-05-08T08:53:08.419 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.419 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.419 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:08.419 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586303+0000", 2024-05-08T08:53:08.419 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1949999999999999e-06 2024-05-08T08:53:08.419 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.420 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.420 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:08.420 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586318+0000", 2024-05-08T08:53:08.420 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5401999999999999e-05 2024-05-08T08:53:08.420 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.420 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.420 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:08.420 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586324+0000", 2024-05-08T08:53:08.420 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8119999999999996e-06 2024-05-08T08:53:08.420 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.420 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.420 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:08.420 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622853+0000", 2024-05-08T08:53:08.420 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036529013999999999 2024-05-08T08:53:08.420 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.421 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.421 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:08.421 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622924+0000", 2024-05-08T08:53:08.421 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1654000000000002e-05 2024-05-08T08:53:08.421 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.421 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.421 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:08.421 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623019+0000", 2024-05-08T08:53:08.421 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4977999999999998e-05 2024-05-08T08:53:08.421 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.421 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:08.421 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.421 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.421 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.421 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4764 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 2013184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:08.422 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586506+0000", 2024-05-08T08:53:08.422 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 36.787358587, 2024-05-08T08:53:08.422 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036859467, 2024-05-08T08:53:08.422 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:08.422 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:08.422 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:08.422 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:08.422 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:08.422 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:08.422 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4764 2024-05-08T08:53:08.422 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.422 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:08.423 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.423 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:08.423 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586506+0000", 2024-05-08T08:53:08.423 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:08.423 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.423 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.423 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:08.423 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586506+0000", 2024-05-08T08:53:08.423 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:08.423 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.423 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.423 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:08.423 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586507+0000", 2024-05-08T08:53:08.423 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9100000000000003e-07 2024-05-08T08:53:08.423 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.424 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.424 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:08.424 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586512+0000", 2024-05-08T08:53:08.424 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9470000000000002e-06 2024-05-08T08:53:08.424 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.424 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.424 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:08.424 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586521+0000", 2024-05-08T08:53:08.424 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3379999999999994e-06 2024-05-08T08:53:08.424 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.424 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.424 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:08.424 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586528+0000", 2024-05-08T08:53:08.424 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1289999999999998e-06 2024-05-08T08:53:08.424 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.425 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.425 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:08.425 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623095+0000", 2024-05-08T08:53:08.425 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036566411 2024-05-08T08:53:08.425 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.425 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.425 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:08.425 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623221+0000", 2024-05-08T08:53:08.425 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012622299999999999 2024-05-08T08:53:08.425 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.425 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.425 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:08.425 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623365+0000", 2024-05-08T08:53:08.425 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000144628 2024-05-08T08:53:08.426 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.426 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:08.426 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.426 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.426 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.426 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4766 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 2029568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:08.426 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586543+0000", 2024-05-08T08:53:08.426 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 36.7873211, 2024-05-08T08:53:08.426 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.037033635000000002, 2024-05-08T08:53:08.426 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:08.426 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:08.426 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:08.426 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:08.426 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:08.426 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:08.427 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4766 2024-05-08T08:53:08.427 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.427 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:08.427 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.427 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:08.427 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586543+0000", 2024-05-08T08:53:08.427 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:08.427 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.427 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.427 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:08.427 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586543+0000", 2024-05-08T08:53:08.427 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:08.427 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.427 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.428 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:08.428 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586544+0000", 2024-05-08T08:53:08.428 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3099999999999998e-07 2024-05-08T08:53:08.428 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.428 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.428 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:08.428 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586547+0000", 2024-05-08T08:53:08.428 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.751e-06 2024-05-08T08:53:08.428 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.428 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.428 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:08.428 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586553+0000", 2024-05-08T08:53:08.428 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2870000000000001e-06 2024-05-08T08:53:08.428 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.428 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.429 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:08.429 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586558+0000", 2024-05-08T08:53:08.429 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5709999999999997e-06 2024-05-08T08:53:08.429 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.429 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.429 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:08.429 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623416+0000", 2024-05-08T08:53:08.429 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036857613999999997 2024-05-08T08:53:08.429 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.429 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.429 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:08.429 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623489+0000", 2024-05-08T08:53:08.429 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3172999999999999e-05 2024-05-08T08:53:08.429 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.429 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.430 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:08.430 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623577+0000", 2024-05-08T08:53:08.430 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7807999999999997e-05 2024-05-08T08:53:08.430 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.430 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:08.430 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.430 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.430 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.430 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4768 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 2045952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:08.430 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586566+0000", 2024-05-08T08:53:08.430 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 36.787298794000002, 2024-05-08T08:53:08.430 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.03726206, 2024-05-08T08:53:08.430 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:08.431 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:08.431 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:08.431 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:08.431 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:08.431 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:08.431 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4768 2024-05-08T08:53:08.431 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.431 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:08.431 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.431 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:08.431 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586566+0000", 2024-05-08T08:53:08.431 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:08.431 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.431 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.431 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:08.432 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586566+0000", 2024-05-08T08:53:08.432 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:08.432 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.432 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.432 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:08.432 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586566+0000", 2024-05-08T08:53:08.432 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8299999999999998e-07 2024-05-08T08:53:08.432 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.432 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.432 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:08.432 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586568+0000", 2024-05-08T08:53:08.432 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.063e-06 2024-05-08T08:53:08.432 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.432 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.432 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:08.433 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586574+0000", 2024-05-08T08:53:08.433 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8300000000000001e-06 2024-05-08T08:53:08.433 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.433 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.433 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:08.433 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586579+0000", 2024-05-08T08:53:08.433 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7559999999999999e-06 2024-05-08T08:53:08.433 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.433 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.433 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:08.433 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623612+0000", 2024-05-08T08:53:08.433 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.037033041000000003 2024-05-08T08:53:08.433 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.433 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.433 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:08.434 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623699+0000", 2024-05-08T08:53:08.434 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6817000000000006e-05 2024-05-08T08:53:08.434 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.434 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.434 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:08.434 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623828+0000", 2024-05-08T08:53:08.434 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012917000000000001 2024-05-08T08:53:08.434 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.434 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:08.434 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.434 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.434 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.434 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4770 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 2062336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:08.434 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586585+0000", 2024-05-08T08:53:08.435 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 36.787279611000002, 2024-05-08T08:53:08.435 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.037449891999999999, 2024-05-08T08:53:08.435 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:08.435 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:08.435 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:08.435 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:08.435 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:08.435 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:08.435 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4770 2024-05-08T08:53:08.435 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.435 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:08.435 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.435 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:08.435 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586585+0000", 2024-05-08T08:53:08.435 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:08.436 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.436 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.436 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:08.436 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586585+0000", 2024-05-08T08:53:08.436 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:08.436 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.436 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.436 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:08.436 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586585+0000", 2024-05-08T08:53:08.436 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:53:08.436 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.436 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.436 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:08.436 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586587+0000", 2024-05-08T08:53:08.436 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.181e-06 2024-05-08T08:53:08.437 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.437 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.437 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:08.437 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586592+0000", 2024-05-08T08:53:08.437 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9710000000000003e-06 2024-05-08T08:53:08.437 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.437 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.437 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:08.437 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586598+0000", 2024-05-08T08:53:08.437 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.118e-06 2024-05-08T08:53:08.437 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.437 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.437 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:08.437 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623862+0000", 2024-05-08T08:53:08.438 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.037264711999999998 2024-05-08T08:53:08.438 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.438 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.438 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:08.438 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623941+0000", 2024-05-08T08:53:08.438 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9084000000000004e-05 2024-05-08T08:53:08.438 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.438 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.438 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:08.438 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.624035+0000", 2024-05-08T08:53:08.438 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3456999999999996e-05 2024-05-08T08:53:08.438 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.438 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:08.438 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.439 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.439 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.439 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4772 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 2078720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:08.439 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586621+0000", 2024-05-08T08:53:08.439 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 36.787243662999998, 2024-05-08T08:53:08.439 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.037650391999999998, 2024-05-08T08:53:08.439 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:08.439 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:08.439 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:08.439 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:08.439 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:08.439 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:08.439 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4772 2024-05-08T08:53:08.439 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.439 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:08.440 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.440 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:08.440 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586621+0000", 2024-05-08T08:53:08.440 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:08.440 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.440 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.440 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:08.440 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586621+0000", 2024-05-08T08:53:08.440 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:08.440 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.440 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.440 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:08.440 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586621+0000", 2024-05-08T08:53:08.440 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:53:08.440 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.440 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.441 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:08.441 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586623+0000", 2024-05-08T08:53:08.441 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9240000000000001e-06 2024-05-08T08:53:08.441 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.441 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.441 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:08.441 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586628+0000", 2024-05-08T08:53:08.441 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6410000000000001e-06 2024-05-08T08:53:08.441 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.441 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.441 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:08.441 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586633+0000", 2024-05-08T08:53:08.441 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.763e-06 2024-05-08T08:53:08.441 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.441 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.442 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:08.442 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.624077+0000", 2024-05-08T08:53:08.442 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.037444709999999999 2024-05-08T08:53:08.442 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.442 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.442 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:08.442 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.624164+0000", 2024-05-08T08:53:08.442 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6769999999999998e-05 2024-05-08T08:53:08.442 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.442 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.442 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:08.442 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.624271+0000", 2024-05-08T08:53:08.442 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010723 2024-05-08T08:53:08.443 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.443 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:08.443 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.443 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.443 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.443 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4774 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 2095104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:08.443 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586640+0000", 2024-05-08T08:53:08.443 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 36.787224524999999, 2024-05-08T08:53:08.443 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.037827178000000003, 2024-05-08T08:53:08.443 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:08.443 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:08.443 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:08.443 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:08.443 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:08.443 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:08.444 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4774 2024-05-08T08:53:08.444 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.444 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:08.444 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.444 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:08.444 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586640+0000", 2024-05-08T08:53:08.444 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:08.444 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.444 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.444 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:08.444 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586640+0000", 2024-05-08T08:53:08.444 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:08.444 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.445 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.445 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:08.445 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586640+0000", 2024-05-08T08:53:08.445 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5199999999999998e-07 2024-05-08T08:53:08.445 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.445 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.445 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:08.445 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586643+0000", 2024-05-08T08:53:08.445 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.232e-06 2024-05-08T08:53:08.445 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.445 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.445 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:08.445 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586649+0000", 2024-05-08T08:53:08.445 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9660000000000001e-06 2024-05-08T08:53:08.446 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.446 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.446 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:08.446 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586653+0000", 2024-05-08T08:53:08.446 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7350000000000004e-06 2024-05-08T08:53:08.446 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.446 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.446 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:08.446 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.624303+0000", 2024-05-08T08:53:08.446 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.037649647000000001 2024-05-08T08:53:08.446 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.446 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.446 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:08.446 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.624373+0000", 2024-05-08T08:53:08.447 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0415999999999998e-05 2024-05-08T08:53:08.447 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.447 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.447 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:08.447 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.624467+0000", 2024-05-08T08:53:08.447 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3830000000000001e-05 2024-05-08T08:53:08.447 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.447 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:08.447 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.447 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.447 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.448 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4621.0:9089 3.1f e33/18 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:head v 33'117, mlcod=31'115)", 2024-05-08T08:53:08.448 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:53:08.448 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 35.854246312999997, 2024-05-08T08:53:08.448 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.111244524, 2024-05-08T08:53:08.448 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:08.448 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:08.448 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:53:08.448 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:08.448 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.448 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:08.448 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:53:08.448 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:08.448 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.448 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.448 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:08.449 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:53:08.449 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:08.449 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.449 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.449 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:08.449 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:53:08.449 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5900000000000001e-07 2024-05-08T08:53:08.449 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.449 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.449 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:08.449 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539600+0000", 2024-05-08T08:53:08.449 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.019981364000000001 2024-05-08T08:53:08.449 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.449 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.450 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:08.450 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539624+0000", 2024-05-08T08:53:08.450 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4043e-05 2024-05-08T08:53:08.450 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.450 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.450 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:08.450 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539635+0000", 2024-05-08T08:53:08.450 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1511e-05 2024-05-08T08:53:08.450 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.450 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.450 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:08.450 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539728+0000", 2024-05-08T08:53:08.451 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2498999999999996e-05 2024-05-08T08:53:08.451 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.451 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.451 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:08.451 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539746+0000", 2024-05-08T08:53:08.451 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8289000000000001e-05 2024-05-08T08:53:08.451 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.451 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.451 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:53:08.451 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.630822+0000", 2024-05-08T08:53:08.451 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.091076283999999993 2024-05-08T08:53:08.451 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.451 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.452 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:08.452 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.630863+0000", 2024-05-08T08:53:08.452 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0374999999999999e-05 2024-05-08T08:53:08.452 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.452 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:08.452 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.452 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.452 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.452 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4795.0:230 3.1f e40/18 3:f8db1a56:::rbd_data.11402b26385.0000000000000000:head v 40'171, mlcod=39'168)", 2024-05-08T08:53:08.452 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:45.927773+0000", 2024-05-08T08:53:08.452 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 22.44609123, 2024-05-08T08:53:08.452 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045892405999999997, 2024-05-08T08:53:08.452 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:08.452 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:08.452 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:53:08.452 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:08.453 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.453 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:08.453 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.927773+0000", 2024-05-08T08:53:08.453 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:08.453 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.453 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.453 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:08.453 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.927773+0000", 2024-05-08T08:53:08.453 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:08.453 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.453 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.453 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:08.453 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.927774+0000", 2024-05-08T08:53:08.454 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0500000000000002e-07 2024-05-08T08:53:08.454 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.454 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.454 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:08.454 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.931581+0000", 2024-05-08T08:53:08.454 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0038071799999999999 2024-05-08T08:53:08.454 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.454 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.454 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:08.454 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.931599+0000", 2024-05-08T08:53:08.454 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7479e-05 2024-05-08T08:53:08.455 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.455 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.455 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:08.455 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.931624+0000", 2024-05-08T08:53:08.455 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5372999999999999e-05 2024-05-08T08:53:08.455 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.455 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.455 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:08.455 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.931698+0000", 2024-05-08T08:53:08.455 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4043999999999998e-05 2024-05-08T08:53:08.455 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.456 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.456 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:08.456 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.931725+0000", 2024-05-08T08:53:08.456 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6883e-05 2024-05-08T08:53:08.456 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.456 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.456 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:53:08.457 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.973623+0000", 2024-05-08T08:53:08.457 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.041898283000000001 2024-05-08T08:53:08.457 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.457 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.457 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:08.457 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.973666+0000", 2024-05-08T08:53:08.457 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2358999999999999e-05 2024-05-08T08:53:08.457 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.457 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:08.457 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.457 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.457 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.457 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4195.0:4 3.11 3:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-05-08T08:53:08.457 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:49.764654+0000", 2024-05-08T08:53:08.458 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.609210772000001, 2024-05-08T08:53:08.458 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.30714275499999999, 2024-05-08T08:53:08.458 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:08.458 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:08.458 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-05-08T08:53:08.458 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:08.458 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4195", 2024-05-08T08:53:08.458 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/59179521", 2024-05-08T08:53:08.458 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4 2024-05-08T08:53:08.458 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.458 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:08.458 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.458 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:08.459 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764654+0000", 2024-05-08T08:53:08.459 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:08.459 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.459 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.459 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:08.459 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764654+0000", 2024-05-08T08:53:08.459 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:08.459 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.459 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.459 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:08.459 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764655+0000", 2024-05-08T08:53:08.459 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.452e-06 2024-05-08T08:53:08.459 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.460 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.460 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:08.460 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764661+0000", 2024-05-08T08:53:08.460 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5369999999999997e-06 2024-05-08T08:53:08.460 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.460 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.460 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:08.460 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764671+0000", 2024-05-08T08:53:08.460 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.028e-05 2024-05-08T08:53:08.460 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.460 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.460 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:08.460 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764680+0000", 2024-05-08T08:53:08.461 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0629999999999995e-06 2024-05-08T08:53:08.461 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.461 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.461 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:08.461 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764753+0000", 2024-05-08T08:53:08.461 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2679e-05 2024-05-08T08:53:08.461 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.461 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.461 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-05-08T08:53:08.461 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764765+0000", 2024-05-08T08:53:08.461 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2415000000000001e-05 2024-05-08T08:53:08.462 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.462 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.462 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:08.462 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:50.071692+0000", 2024-05-08T08:53:08.462 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.306927324 2024-05-08T08:53:08.462 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.462 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.462 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:08.462 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:50.071796+0000", 2024-05-08T08:53:08.462 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104005 2024-05-08T08:53:08.462 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.463 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:08.463 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.463 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.463 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:08.463 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:53:08.465 INFO:teuthology.orchestra.run.smithi059.stderr:nodeep-scrub is unset 2024-05-08T08:53:08.466 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:53:08.466 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:08.467+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:08.565 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:08.568+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:08.612 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:53:08.612 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:53:08.612 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:53:08.612 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:53:08.621 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:53:08.666 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:08.669+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:08.765 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:53:08.765 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:53:08.765 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:53:08.765 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:53:08.765 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:53:08.767 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:53:08.770+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:53:08.774 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:53:08.868 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:08.871+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:08.880 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:53:08.880 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:53:08.880 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:53:08.880 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:53:08.880 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.880 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22069 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 749568~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:08.881 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857624+0000", 2024-05-08T08:53:08.881 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.024977788999998, 2024-05-08T08:53:08.881 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046969958999999999, 2024-05-08T08:53:08.881 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:08.881 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:08.881 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:08.881 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:08.881 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:08.881 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:08.881 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22069 2024-05-08T08:53:08.881 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.881 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:08.881 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.881 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:08.881 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857624+0000", 2024-05-08T08:53:08.881 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:08.882 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.882 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.882 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:08.882 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857624+0000", 2024-05-08T08:53:08.882 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:08.882 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.882 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.882 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:08.882 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857625+0000", 2024-05-08T08:53:08.882 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.03e-07 2024-05-08T08:53:08.882 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.882 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.883 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:08.883 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857627+0000", 2024-05-08T08:53:08.883 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9369999999999998e-06 2024-05-08T08:53:08.883 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.883 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.883 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:08.883 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857633+0000", 2024-05-08T08:53:08.883 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1229999999999997e-06 2024-05-08T08:53:08.883 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.883 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.883 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:08.883 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857637+0000", 2024-05-08T08:53:08.883 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7820000000000001e-06 2024-05-08T08:53:08.883 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.884 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.884 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:08.884 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904457+0000", 2024-05-08T08:53:08.884 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046819715999999997 2024-05-08T08:53:08.884 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.884 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.884 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:08.884 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904524+0000", 2024-05-08T08:53:08.884 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6371000000000005e-05 2024-05-08T08:53:08.884 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.884 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.884 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:08.884 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904594+0000", 2024-05-08T08:53:08.884 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0627e-05 2024-05-08T08:53:08.885 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.885 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:08.885 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.885 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.885 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.885 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22071 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 765952~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:08.885 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857643+0000", 2024-05-08T08:53:08.885 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.024958706, 2024-05-08T08:53:08.885 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047106528000000002, 2024-05-08T08:53:08.885 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:08.885 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:08.885 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:08.885 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:08.885 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:08.885 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:08.886 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22071 2024-05-08T08:53:08.886 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.886 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:08.886 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.886 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:08.886 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857643+0000", 2024-05-08T08:53:08.886 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:08.886 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.886 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.886 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:08.886 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857643+0000", 2024-05-08T08:53:08.886 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:08.886 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.886 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.886 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:08.887 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857644+0000", 2024-05-08T08:53:08.887 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2000000000000001e-07 2024-05-08T08:53:08.887 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.887 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.887 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:08.887 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857646+0000", 2024-05-08T08:53:08.887 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0059999999999999e-06 2024-05-08T08:53:08.887 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.887 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.887 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:08.887 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857651+0000", 2024-05-08T08:53:08.887 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.84e-06 2024-05-08T08:53:08.887 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.887 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.887 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:08.888 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857657+0000", 2024-05-08T08:53:08.888 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5620000000000003e-06 2024-05-08T08:53:08.888 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.888 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.888 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:08.888 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904620+0000", 2024-05-08T08:53:08.888 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046962661000000003 2024-05-08T08:53:08.888 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.888 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.888 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:08.888 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904678+0000", 2024-05-08T08:53:08.888 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8656999999999998e-05 2024-05-08T08:53:08.888 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.888 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.888 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:08.889 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904750+0000", 2024-05-08T08:53:08.889 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1482000000000004e-05 2024-05-08T08:53:08.889 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.889 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:08.889 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.889 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.889 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.889 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22073 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 782336~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:08.889 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:53:08.889 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.024934698000003, 2024-05-08T08:53:08.889 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047244234000000003, 2024-05-08T08:53:08.889 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:08.889 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:08.889 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:08.889 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:08.889 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:08.890 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:08.890 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22073 2024-05-08T08:53:08.890 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.890 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:08.890 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.890 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:08.890 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:53:08.890 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:08.890 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.890 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.890 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:08.890 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:53:08.890 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:08.890 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.890 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.891 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:08.891 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857668+0000", 2024-05-08T08:53:08.891 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7800000000000002e-07 2024-05-08T08:53:08.891 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.891 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.891 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:08.891 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857670+0000", 2024-05-08T08:53:08.891 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0820000000000001e-06 2024-05-08T08:53:08.891 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.891 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.891 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:08.891 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857678+0000", 2024-05-08T08:53:08.891 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7589999999999997e-06 2024-05-08T08:53:08.891 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.891 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.891 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:08.891 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857691+0000", 2024-05-08T08:53:08.892 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3672999999999999e-05 2024-05-08T08:53:08.892 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.892 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.892 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:08.892 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904780+0000", 2024-05-08T08:53:08.892 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047088616999999999 2024-05-08T08:53:08.892 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.892 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.892 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:08.892 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904840+0000", 2024-05-08T08:53:08.892 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0356999999999998e-05 2024-05-08T08:53:08.892 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.892 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.892 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:08.892 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904912+0000", 2024-05-08T08:53:08.892 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1367999999999998e-05 2024-05-08T08:53:08.892 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.893 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:08.893 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.893 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.893 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.893 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22075 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 798720~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:08.893 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:53:08.893 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.024904300999999, 2024-05-08T08:53:08.893 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047384643999999997, 2024-05-08T08:53:08.893 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:08.893 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:08.893 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:08.893 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:08.893 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:08.893 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:08.894 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22075 2024-05-08T08:53:08.894 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.894 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:08.894 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.894 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:08.894 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:53:08.894 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:08.894 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.894 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.894 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:08.894 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:53:08.894 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:08.894 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.894 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.895 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:08.895 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:53:08.895 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7399999999999999e-07 2024-05-08T08:53:08.895 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.895 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.895 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:08.895 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857705+0000", 2024-05-08T08:53:08.895 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1980000000000004e-06 2024-05-08T08:53:08.895 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.895 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.895 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:08.895 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857713+0000", 2024-05-08T08:53:08.895 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7039999999999994e-06 2024-05-08T08:53:08.895 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.895 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.896 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:08.896 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857718+0000", 2024-05-08T08:53:08.896 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.865e-06 2024-05-08T08:53:08.896 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.896 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.896 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:08.896 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904941+0000", 2024-05-08T08:53:08.896 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047223198000000001 2024-05-08T08:53:08.896 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.896 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.896 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:08.896 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904999+0000", 2024-05-08T08:53:08.896 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8254999999999997e-05 2024-05-08T08:53:08.896 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.897 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.897 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:08.897 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905082+0000", 2024-05-08T08:53:08.897 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3049999999999999e-05 2024-05-08T08:53:08.897 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.897 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:08.897 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.897 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.897 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.897 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22077 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 815104~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:08.897 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:53:08.897 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.024866289999999, 2024-05-08T08:53:08.897 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047519448999999998, 2024-05-08T08:53:08.897 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:08.897 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:08.898 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:08.898 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:08.898 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:08.898 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:08.898 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22077 2024-05-08T08:53:08.898 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.898 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:08.898 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.898 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:08.898 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:53:08.898 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:08.898 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.898 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.898 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:08.899 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:53:08.899 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:08.899 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.899 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.899 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:08.899 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:53:08.899 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.99e-07 2024-05-08T08:53:08.899 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.899 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.899 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:08.899 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857739+0000", 2024-05-08T08:53:08.899 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3599999999999999e-06 2024-05-08T08:53:08.899 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.899 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.899 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:08.899 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857745+0000", 2024-05-08T08:53:08.900 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5139999999999998e-06 2024-05-08T08:53:08.900 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.900 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.900 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:08.900 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857750+0000", 2024-05-08T08:53:08.900 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8550000000000001e-06 2024-05-08T08:53:08.900 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.900 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.900 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:08.900 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905110+0000", 2024-05-08T08:53:08.900 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047360126000000002 2024-05-08T08:53:08.900 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.900 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.900 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:08.900 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905175+0000", 2024-05-08T08:53:08.900 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4689000000000004e-05 2024-05-08T08:53:08.901 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.901 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.901 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:08.901 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905255+0000", 2024-05-08T08:53:08.901 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0306e-05 2024-05-08T08:53:08.901 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.901 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:08.901 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.901 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.901 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.901 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22079 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 831488~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:08.901 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:53:08.901 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.024845741999997, 2024-05-08T08:53:08.901 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.04767499, 2024-05-08T08:53:08.901 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:08.901 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:08.901 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:08.902 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:08.902 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:08.902 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:08.902 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22079 2024-05-08T08:53:08.902 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.902 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:08.902 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.902 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:08.902 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:53:08.902 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:08.902 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.902 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.902 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:08.902 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:53:08.903 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:08.903 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.903 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.903 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:08.903 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857757+0000", 2024-05-08T08:53:08.903 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8299999999999997e-07 2024-05-08T08:53:08.903 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.903 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.903 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:08.903 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857759+0000", 2024-05-08T08:53:08.903 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1390000000000002e-06 2024-05-08T08:53:08.903 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.903 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.903 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:08.903 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857765+0000", 2024-05-08T08:53:08.903 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4609999999999996e-06 2024-05-08T08:53:08.904 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.904 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.904 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:08.904 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857770+0000", 2024-05-08T08:53:08.904 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6689999999999997e-06 2024-05-08T08:53:08.904 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.904 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.904 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:08.904 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905281+0000", 2024-05-08T08:53:08.904 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047510668999999998 2024-05-08T08:53:08.904 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.904 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.904 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:08.904 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905344+0000", 2024-05-08T08:53:08.904 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2797000000000002e-05 2024-05-08T08:53:08.905 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.905 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.905 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:08.905 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905431+0000", 2024-05-08T08:53:08.905 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7771999999999999e-05 2024-05-08T08:53:08.905 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.905 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:08.905 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.905 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.905 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.905 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22082 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 847872~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:08.905 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:53:08.905 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.024826050999998, 2024-05-08T08:53:08.905 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047831934999999999, 2024-05-08T08:53:08.906 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:08.906 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:08.906 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:08.906 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:08.906 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:08.906 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:08.906 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22082 2024-05-08T08:53:08.906 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.906 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:08.906 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.906 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:08.906 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:53:08.906 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:08.906 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.907 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.907 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:08.907 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:53:08.907 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:08.907 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.907 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.907 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:08.907 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:53:08.907 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:53:08.907 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.907 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.907 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:08.907 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857778+0000", 2024-05-08T08:53:08.907 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0789999999999999e-06 2024-05-08T08:53:08.907 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.908 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.908 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:08.908 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857784+0000", 2024-05-08T08:53:08.908 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0480000000000004e-06 2024-05-08T08:53:08.908 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.908 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.908 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:08.908 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857790+0000", 2024-05-08T08:53:08.908 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2719999999999997e-06 2024-05-08T08:53:08.908 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.908 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.908 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:08.908 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905458+0000", 2024-05-08T08:53:08.908 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047668470999999997 2024-05-08T08:53:08.908 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.908 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.909 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:08.909 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905521+0000", 2024-05-08T08:53:08.909 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2952000000000003e-05 2024-05-08T08:53:08.909 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.909 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.909 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:08.909 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905608+0000", 2024-05-08T08:53:08.909 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6759000000000001e-05 2024-05-08T08:53:08.909 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.909 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:08.909 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.909 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.909 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.909 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22084 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 864256~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:08.909 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:53:08.910 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.024802100999999, 2024-05-08T08:53:08.910 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047977748000000001, 2024-05-08T08:53:08.910 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:08.910 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:08.910 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:08.910 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:08.910 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:08.910 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:08.910 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22084 2024-05-08T08:53:08.910 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.910 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:08.910 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.910 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:08.910 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:53:08.910 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:08.911 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.911 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.911 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:08.911 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:53:08.911 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:08.911 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.911 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.911 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:08.911 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:53:08.911 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:53:08.911 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.911 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.911 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:08.911 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857802+0000", 2024-05-08T08:53:08.911 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0779999999999998e-06 2024-05-08T08:53:08.911 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.912 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.912 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:08.912 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857812+0000", 2024-05-08T08:53:08.912 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7939999999999997e-06 2024-05-08T08:53:08.912 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.912 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.912 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:08.912 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857817+0000", 2024-05-08T08:53:08.912 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8899999999999998e-06 2024-05-08T08:53:08.912 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.912 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.912 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:08.912 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905636+0000", 2024-05-08T08:53:08.912 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047819252 2024-05-08T08:53:08.912 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.912 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.912 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:08.913 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905700+0000", 2024-05-08T08:53:08.913 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4016999999999994e-05 2024-05-08T08:53:08.913 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.913 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.913 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:08.913 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905778+0000", 2024-05-08T08:53:08.913 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7306999999999997e-05 2024-05-08T08:53:08.913 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.913 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:08.913 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.913 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.913 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.913 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22086 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 880640~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:08.913 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:53:08.913 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.024778750999999, 2024-05-08T08:53:08.913 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048119149, 2024-05-08T08:53:08.914 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:08.914 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:08.914 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:08.914 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:08.914 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:08.914 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:08.914 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22086 2024-05-08T08:53:08.914 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.914 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:08.914 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.914 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:08.914 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:53:08.914 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:08.914 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.915 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.915 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:08.915 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:53:08.915 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:08.915 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.915 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.915 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:08.915 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857824+0000", 2024-05-08T08:53:08.915 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4200000000000002e-07 2024-05-08T08:53:08.915 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.915 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.915 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:08.915 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857826+0000", 2024-05-08T08:53:08.915 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0590000000000001e-06 2024-05-08T08:53:08.916 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.916 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.916 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:08.916 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857832+0000", 2024-05-08T08:53:08.916 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8029999999999999e-06 2024-05-08T08:53:08.916 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.916 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.916 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:08.916 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857837+0000", 2024-05-08T08:53:08.916 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7929999999999997e-06 2024-05-08T08:53:08.916 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.916 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.916 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:08.916 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905803+0000", 2024-05-08T08:53:08.916 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047965761000000003 2024-05-08T08:53:08.917 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.917 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.917 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:08.917 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905862+0000", 2024-05-08T08:53:08.917 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9203000000000002e-05 2024-05-08T08:53:08.917 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.917 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.917 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:08.917 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905942+0000", 2024-05-08T08:53:08.917 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0187999999999999e-05 2024-05-08T08:53:08.917 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.917 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:08.917 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.917 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.918 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.918 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22088 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 897024~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:08.918 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:53:08.918 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.024758699000003, 2024-05-08T08:53:08.918 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048256049000000002, 2024-05-08T08:53:08.918 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:08.918 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:08.918 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:08.918 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:08.918 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:08.918 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:08.918 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22088 2024-05-08T08:53:08.918 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.918 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:08.919 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.919 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:08.919 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:53:08.919 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:08.919 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.919 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.919 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:08.919 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:53:08.919 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:08.919 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.919 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.919 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:08.919 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857844+0000", 2024-05-08T08:53:08.919 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.46e-07 2024-05-08T08:53:08.920 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.920 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.920 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:08.920 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857846+0000", 2024-05-08T08:53:08.920 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0899999999999999e-06 2024-05-08T08:53:08.920 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.920 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.920 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:08.920 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857855+0000", 2024-05-08T08:53:08.920 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8219999999999996e-06 2024-05-08T08:53:08.920 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.920 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.920 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:08.920 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857860+0000", 2024-05-08T08:53:08.921 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2989999999999996e-06 2024-05-08T08:53:08.921 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.921 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.921 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:08.921 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905966+0000", 2024-05-08T08:53:08.921 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048106542000000002 2024-05-08T08:53:08.921 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.921 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.921 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:08.921 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906024+0000", 2024-05-08T08:53:08.921 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7958000000000003e-05 2024-05-08T08:53:08.921 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.921 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.921 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:08.921 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906099+0000", 2024-05-08T08:53:08.921 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4992000000000003e-05 2024-05-08T08:53:08.922 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.922 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:08.922 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.922 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.922 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.922 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22091 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 913408~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:08.922 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:53:08.922 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.024734938999998, 2024-05-08T08:53:08.922 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048395290000000001, 2024-05-08T08:53:08.922 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:08.922 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:08.922 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:08.922 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:08.922 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:08.923 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:08.923 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22091 2024-05-08T08:53:08.923 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.923 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:08.923 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.923 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:08.923 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:53:08.923 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:08.923 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.923 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.923 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:08.923 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:53:08.923 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:08.924 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.924 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.924 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:08.924 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:53:08.924 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4900000000000001e-07 2024-05-08T08:53:08.924 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.924 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.924 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:08.924 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857870+0000", 2024-05-08T08:53:08.924 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2510000000000001e-06 2024-05-08T08:53:08.924 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.924 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.924 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:08.924 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857877+0000", 2024-05-08T08:53:08.924 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0809999999999996e-06 2024-05-08T08:53:08.924 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.925 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.925 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:08.925 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857881+0000", 2024-05-08T08:53:08.925 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.724e-06 2024-05-08T08:53:08.925 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.925 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.925 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:08.925 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906127+0000", 2024-05-08T08:53:08.925 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048245345000000002 2024-05-08T08:53:08.925 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.925 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.925 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:08.925 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906190+0000", 2024-05-08T08:53:08.925 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2762000000000007e-05 2024-05-08T08:53:08.925 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.926 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.926 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:08.926 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906262+0000", 2024-05-08T08:53:08.926 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2778e-05 2024-05-08T08:53:08.926 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.926 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:08.926 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.926 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.926 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.926 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22093 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 929792~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:08.926 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:53:08.926 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.024712575000002, 2024-05-08T08:53:08.926 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048532875000000003, 2024-05-08T08:53:08.926 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:08.927 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:08.927 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:08.927 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:08.927 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:08.927 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:08.927 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22093 2024-05-08T08:53:08.927 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.927 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:08.927 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.927 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:08.927 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:53:08.927 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:08.927 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.927 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.927 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:08.927 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:53:08.927 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:08.928 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.928 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.928 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:08.928 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857890+0000", 2024-05-08T08:53:08.928 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-07 2024-05-08T08:53:08.928 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.928 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.928 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:08.928 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857899+0000", 2024-05-08T08:53:08.928 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6889999999999999e-06 2024-05-08T08:53:08.928 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.928 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.928 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:08.929 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857906+0000", 2024-05-08T08:53:08.929 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6490000000000002e-06 2024-05-08T08:53:08.929 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.929 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.929 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:08.929 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857917+0000", 2024-05-08T08:53:08.929 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0764999999999999e-05 2024-05-08T08:53:08.929 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.929 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.929 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:08.929 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906287+0000", 2024-05-08T08:53:08.929 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048370442 2024-05-08T08:53:08.929 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.929 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.929 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:08.930 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906349+0000", 2024-05-08T08:53:08.930 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1537000000000004e-05 2024-05-08T08:53:08.930 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.930 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.930 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:08.930 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906422+0000", 2024-05-08T08:53:08.930 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3431999999999998e-05 2024-05-08T08:53:08.930 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.930 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:08.930 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.930 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.930 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.930 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22095 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 946176~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:08.930 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:53:08.931 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.024677871000002, 2024-05-08T08:53:08.931 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048657593999999998, 2024-05-08T08:53:08.931 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:08.931 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:08.931 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:08.931 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:08.931 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:08.931 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:08.931 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22095 2024-05-08T08:53:08.931 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.931 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:08.931 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.931 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:08.931 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:53:08.931 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:08.931 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.932 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.932 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:08.932 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:53:08.932 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:08.932 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.932 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.932 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:08.932 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857925+0000", 2024-05-08T08:53:08.932 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3099999999999998e-07 2024-05-08T08:53:08.932 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.932 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.932 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:08.932 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857927+0000", 2024-05-08T08:53:08.932 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.34e-06 2024-05-08T08:53:08.932 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.933 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.933 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:08.933 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857938+0000", 2024-05-08T08:53:08.933 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1426e-05 2024-05-08T08:53:08.933 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.933 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.933 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:08.933 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857943+0000", 2024-05-08T08:53:08.933 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5140000000000003e-06 2024-05-08T08:53:08.933 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.933 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.933 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:08.933 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906448+0000", 2024-05-08T08:53:08.933 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048504641000000001 2024-05-08T08:53:08.934 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.934 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.934 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:08.934 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906510+0000", 2024-05-08T08:53:08.934 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2663000000000006e-05 2024-05-08T08:53:08.934 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.934 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.934 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:08.934 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906582+0000", 2024-05-08T08:53:08.934 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1478999999999998e-05 2024-05-08T08:53:08.934 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.934 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:08.934 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.934 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.935 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.935 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22097 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 962560~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:08.935 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:53:08.935 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.024650182000002, 2024-05-08T08:53:08.935 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048791353000000003, 2024-05-08T08:53:08.935 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:08.935 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:08.935 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:08.935 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:08.935 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:08.935 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:08.935 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22097 2024-05-08T08:53:08.935 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.935 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:08.936 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.936 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:08.936 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:53:08.936 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:08.936 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.936 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.936 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:08.936 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:53:08.936 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:08.936 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.936 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.936 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:08.936 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:53:08.936 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9200000000000002e-07 2024-05-08T08:53:08.936 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.937 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.937 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:08.937 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857954+0000", 2024-05-08T08:53:08.937 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1900000000000002e-06 2024-05-08T08:53:08.937 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.937 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.937 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:08.937 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857961+0000", 2024-05-08T08:53:08.937 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4989999999999999e-06 2024-05-08T08:53:08.937 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.937 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.937 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:08.937 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857971+0000", 2024-05-08T08:53:08.937 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0485999999999999e-05 2024-05-08T08:53:08.937 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.937 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.938 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:08.938 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906607+0000", 2024-05-08T08:53:08.938 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048635109000000003 2024-05-08T08:53:08.938 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.938 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.938 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:08.938 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906667+0000", 2024-05-08T08:53:08.938 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0439000000000002e-05 2024-05-08T08:53:08.938 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.938 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.938 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:08.938 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906743+0000", 2024-05-08T08:53:08.938 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6237999999999998e-05 2024-05-08T08:53:08.938 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.938 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:08.939 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.939 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.939 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.939 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22099 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 978944~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:08.939 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:53:08.939 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.024624021999998, 2024-05-08T08:53:08.939 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048932055000000002, 2024-05-08T08:53:08.939 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:08.939 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:08.939 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:08.939 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:08.939 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:08.939 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:08.939 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22099 2024-05-08T08:53:08.939 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.940 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:08.940 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.940 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:08.940 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:53:08.940 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:08.940 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.940 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.940 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:08.940 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:53:08.940 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:08.940 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.940 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.940 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:08.940 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:53:08.940 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.27e-07 2024-05-08T08:53:08.940 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.941 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.941 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:08.941 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857987+0000", 2024-05-08T08:53:08.941 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5890000000000003e-06 2024-05-08T08:53:08.941 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.941 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.941 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:08.941 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857994+0000", 2024-05-08T08:53:08.941 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6619999999999999e-06 2024-05-08T08:53:08.941 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.941 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.941 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:08.941 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857998+0000", 2024-05-08T08:53:08.941 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7110000000000003e-06 2024-05-08T08:53:08.941 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.941 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.942 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:08.942 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906772+0000", 2024-05-08T08:53:08.942 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048773415000000001 2024-05-08T08:53:08.942 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.942 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.942 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:08.942 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906834+0000", 2024-05-08T08:53:08.942 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1752000000000001e-05 2024-05-08T08:53:08.942 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.942 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.942 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:08.942 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906910+0000", 2024-05-08T08:53:08.942 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6499000000000001e-05 2024-05-08T08:53:08.942 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.942 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:08.942 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.943 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.943 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.943 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22101 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 995328~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:08.943 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:53:08.943 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.024596891000002, 2024-05-08T08:53:08.943 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049087364000000001, 2024-05-08T08:53:08.943 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:08.943 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:08.943 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:08.943 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:08.943 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:08.943 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:08.943 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22101 2024-05-08T08:53:08.943 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.943 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:08.944 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.944 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:08.944 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:53:08.944 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:08.944 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.944 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.944 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:08.944 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:53:08.944 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:08.944 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.944 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.944 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:08.944 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858006+0000", 2024-05-08T08:53:08.944 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1399999999999997e-07 2024-05-08T08:53:08.944 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.944 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.945 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:08.945 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858008+0000", 2024-05-08T08:53:08.945 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2060000000000001e-06 2024-05-08T08:53:08.945 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.945 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.945 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:08.945 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858017+0000", 2024-05-08T08:53:08.945 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1579999999999994e-06 2024-05-08T08:53:08.945 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.945 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.945 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:08.945 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858022+0000", 2024-05-08T08:53:08.945 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4199999999999998e-06 2024-05-08T08:53:08.945 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.945 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.945 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:08.946 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906940+0000", 2024-05-08T08:53:08.946 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048917627999999998 2024-05-08T08:53:08.946 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.946 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.946 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:08.946 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907002+0000", 2024-05-08T08:53:08.946 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2399999999999999e-05 2024-05-08T08:53:08.946 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.946 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.946 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:08.946 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907092+0000", 2024-05-08T08:53:08.946 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0137999999999997e-05 2024-05-08T08:53:08.946 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.946 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:08.946 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.947 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.947 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.947 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22103 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1011712~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:08.947 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:53:08.947 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.024572106000001, 2024-05-08T08:53:08.947 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049228542, 2024-05-08T08:53:08.947 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:08.947 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:08.947 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:08.947 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:08.947 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:08.947 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:08.947 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22103 2024-05-08T08:53:08.947 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.947 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:08.948 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.948 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:08.948 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:53:08.948 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:08.948 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.948 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.948 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:08.948 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:53:08.948 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:08.948 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.948 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.948 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:08.948 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:53:08.948 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4299999999999999e-07 2024-05-08T08:53:08.948 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.948 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.949 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:08.949 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858032+0000", 2024-05-08T08:53:08.949 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0590000000000001e-06 2024-05-08T08:53:08.949 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.949 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.949 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:08.949 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858037+0000", 2024-05-08T08:53:08.949 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.977e-06 2024-05-08T08:53:08.949 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.949 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.949 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:08.949 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858042+0000", 2024-05-08T08:53:08.949 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7650000000000001e-06 2024-05-08T08:53:08.949 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.949 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.950 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:08.950 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907118+0000", 2024-05-08T08:53:08.950 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049076147000000001 2024-05-08T08:53:08.950 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.950 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.950 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:08.950 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907180+0000", 2024-05-08T08:53:08.950 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1971999999999996e-05 2024-05-08T08:53:08.950 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.950 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.950 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:08.950 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907258+0000", 2024-05-08T08:53:08.950 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8279e-05 2024-05-08T08:53:08.950 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.950 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:08.951 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.951 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.951 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.951 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22105 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1028096~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:08.951 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:53:08.951 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.024552862999997, 2024-05-08T08:53:08.951 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049394609999999999, 2024-05-08T08:53:08.951 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:08.951 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:08.951 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:08.951 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:08.951 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:08.951 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:08.951 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22105 2024-05-08T08:53:08.951 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.952 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:08.952 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.952 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:08.952 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:53:08.952 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:08.952 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.952 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.952 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:08.952 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:53:08.952 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:08.952 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.952 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.952 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:08.952 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858050+0000", 2024-05-08T08:53:08.952 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5999999999999999e-07 2024-05-08T08:53:08.952 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.953 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.953 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:08.953 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858051+0000", 2024-05-08T08:53:08.953 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8870000000000001e-06 2024-05-08T08:53:08.953 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.953 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.953 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:08.953 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858057+0000", 2024-05-08T08:53:08.953 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8019999999999997e-06 2024-05-08T08:53:08.953 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.953 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.953 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:08.953 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858062+0000", 2024-05-08T08:53:08.953 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.561e-06 2024-05-08T08:53:08.953 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.954 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.954 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:08.954 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907289+0000", 2024-05-08T08:53:08.954 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049226957000000002 2024-05-08T08:53:08.954 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.954 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.954 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:08.954 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907356+0000", 2024-05-08T08:53:08.954 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6915000000000006e-05 2024-05-08T08:53:08.954 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.954 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.954 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:08.954 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907444+0000", 2024-05-08T08:53:08.954 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8127999999999994e-05 2024-05-08T08:53:08.955 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.955 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:08.955 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.955 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.955 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.955 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22107 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1044480~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:08.955 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:53:08.955 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.024534523, 2024-05-08T08:53:08.955 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049799942999999999, 2024-05-08T08:53:08.955 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:08.955 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:08.955 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:08.955 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:08.955 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:08.955 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:08.956 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22107 2024-05-08T08:53:08.956 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.956 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:08.956 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.956 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:08.956 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:53:08.956 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:08.956 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.956 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.956 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:08.956 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:53:08.956 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:08.956 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.956 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.956 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:08.956 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:53:08.957 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4799999999999999e-07 2024-05-08T08:53:08.957 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.957 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.957 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:08.957 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858070+0000", 2024-05-08T08:53:08.957 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.012e-06 2024-05-08T08:53:08.957 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.957 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.957 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:08.957 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858077+0000", 2024-05-08T08:53:08.957 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6290000000000001e-06 2024-05-08T08:53:08.957 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.957 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.957 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:08.957 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858083+0000", 2024-05-08T08:53:08.957 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8880000000000002e-06 2024-05-08T08:53:08.958 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.958 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.958 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:08.958 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907671+0000", 2024-05-08T08:53:08.958 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049587191000000003 2024-05-08T08:53:08.958 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.958 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.958 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:08.958 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907763+0000", 2024-05-08T08:53:08.958 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2231000000000005e-05 2024-05-08T08:53:08.958 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.958 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.958 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:08.958 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907867+0000", 2024-05-08T08:53:08.958 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104644 2024-05-08T08:53:08.958 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.959 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:08.959 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.959 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.959 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.959 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22294 3.1c 3:3aad0389:::rbd_data.11406886b07b.0000000000000080:head [stat,delete] snapc 7=[7] RETRY=6 ondisk+retry+write+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:08.959 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:53:08.959 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.023940674999999, 2024-05-08T08:53:08.959 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051152191, 2024-05-08T08:53:08.959 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:08.959 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:08.959 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:08.959 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:08.959 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:08.959 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:08.959 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22294 2024-05-08T08:53:08.960 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.960 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:08.960 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.960 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:08.960 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:53:08.960 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:08.960 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.960 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.960 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:08.960 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:53:08.960 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:08.960 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.960 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.960 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:08.960 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858662+0000", 2024-05-08T08:53:08.961 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4799999999999998e-07 2024-05-08T08:53:08.961 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.961 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.961 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:08.961 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858664+0000", 2024-05-08T08:53:08.961 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1229999999999998e-06 2024-05-08T08:53:08.961 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.961 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.961 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:08.961 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858670+0000", 2024-05-08T08:53:08.961 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3099999999999997e-06 2024-05-08T08:53:08.961 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.961 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.961 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:08.961 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858676+0000", 2024-05-08T08:53:08.961 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9750000000000004e-06 2024-05-08T08:53:08.962 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.962 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.962 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:08.962 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907901+0000", 2024-05-08T08:53:08.962 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049224520000000001 2024-05-08T08:53:08.962 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.962 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.962 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:08.962 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.908085+0000", 2024-05-08T08:53:08.962 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00018377500000000001 2024-05-08T08:53:08.962 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:08.962 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:08.962 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:08.962 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.909814+0000", 2024-05-08T08:53:08.962 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00172894 2024-05-08T08:53:08.962 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.963 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:08.963 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.963 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:08.963 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:08.963 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:53:08.964 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:53:08.968 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:08.972+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:09.069 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:53:09.158 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:53:09.158 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:53:09.158 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:53:09.159 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:09.073+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:09.160 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:53:09.204 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:09.207+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:09.265 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:53:09.265 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:53:09.265 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:53:09.265 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:53:09.265 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:53:09.270 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:09.273+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:09.273 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:53:09.371 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:09.374+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:09.422 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:53:09.422 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:53:09.422 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:53:09.422 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:53:09.423 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.423 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6690 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1093632~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:09.423 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:53:09.423 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.709479158999997, 2024-05-08T08:53:09.423 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052462754, 2024-05-08T08:53:09.423 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:09.423 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:09.423 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:09.423 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:09.423 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:09.423 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:09.423 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6690 2024-05-08T08:53:09.423 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.423 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:09.423 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.424 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:09.424 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:53:09.424 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:09.424 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.424 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.424 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:09.424 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:53:09.424 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:09.424 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.424 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.424 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:09.424 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715346+0000", 2024-05-08T08:53:09.424 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.84e-07 2024-05-08T08:53:09.424 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.425 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.425 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:09.425 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715348+0000", 2024-05-08T08:53:09.425 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.114e-06 2024-05-08T08:53:09.425 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.425 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.425 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:09.425 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715353+0000", 2024-05-08T08:53:09.425 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5720000000000002e-06 2024-05-08T08:53:09.425 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.425 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.425 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:09.425 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715358+0000", 2024-05-08T08:53:09.425 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7140000000000001e-06 2024-05-08T08:53:09.425 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.425 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.426 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:09.426 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767638+0000", 2024-05-08T08:53:09.426 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052279562000000002 2024-05-08T08:53:09.426 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.426 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.426 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:09.426 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767713+0000", 2024-05-08T08:53:09.426 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5652e-05 2024-05-08T08:53:09.426 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.426 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.426 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:09.426 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767808+0000", 2024-05-08T08:53:09.426 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4755999999999999e-05 2024-05-08T08:53:09.426 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:09.426 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:09.427 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:09.427 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.427 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.427 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6692 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1110016~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:09.427 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:53:09.427 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.709460069000002, 2024-05-08T08:53:09.427 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052666495000000001, 2024-05-08T08:53:09.427 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:09.427 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:09.427 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:09.427 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:09.427 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:09.427 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:09.428 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6692 2024-05-08T08:53:09.428 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.428 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:09.428 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.428 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:09.428 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:53:09.428 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:09.428 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.428 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.428 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:09.428 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:53:09.428 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:09.428 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.428 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.428 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:09.428 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:53:09.429 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8599999999999999e-07 2024-05-08T08:53:09.429 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.429 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.429 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:09.429 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715372+0000", 2024-05-08T08:53:09.429 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0990000000000001e-06 2024-05-08T08:53:09.429 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.429 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.429 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:09.429 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715378+0000", 2024-05-08T08:53:09.429 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0680000000000002e-06 2024-05-08T08:53:09.429 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.429 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.429 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:09.429 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715383+0000", 2024-05-08T08:53:09.430 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8740000000000003e-06 2024-05-08T08:53:09.430 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.430 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.430 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:09.430 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767848+0000", 2024-05-08T08:53:09.430 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052464668999999998 2024-05-08T08:53:09.430 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.430 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.430 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:09.430 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767927+0000", 2024-05-08T08:53:09.430 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9228999999999996e-05 2024-05-08T08:53:09.431 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.431 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.431 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:09.431 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768031+0000", 2024-05-08T08:53:09.431 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010417 2024-05-08T08:53:09.431 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:09.431 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:09.431 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:09.431 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.431 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.431 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6694 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1126400~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:09.431 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:53:09.431 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.709435171999999, 2024-05-08T08:53:09.431 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052848829, 2024-05-08T08:53:09.431 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:09.431 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:09.432 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:09.432 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:09.432 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:09.432 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:09.432 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6694 2024-05-08T08:53:09.432 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.432 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:09.432 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.432 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:09.432 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:53:09.432 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:09.432 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.432 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.432 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:09.432 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:53:09.433 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:09.433 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.433 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.433 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:09.433 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715390+0000", 2024-05-08T08:53:09.433 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9900000000000001e-07 2024-05-08T08:53:09.433 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.433 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.433 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:09.433 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715392+0000", 2024-05-08T08:53:09.433 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8279999999999999e-06 2024-05-08T08:53:09.433 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.433 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.433 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:09.433 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715398+0000", 2024-05-08T08:53:09.433 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7619999999999998e-06 2024-05-08T08:53:09.433 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.434 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.434 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:09.434 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715403+0000", 2024-05-08T08:53:09.434 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7790000000000003e-06 2024-05-08T08:53:09.434 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.434 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.434 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:09.434 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768067+0000", 2024-05-08T08:53:09.434 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052663785999999997 2024-05-08T08:53:09.434 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.434 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.434 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:09.434 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768142+0000", 2024-05-08T08:53:09.434 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5018999999999994e-05 2024-05-08T08:53:09.435 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.435 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.435 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:09.435 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768238+0000", 2024-05-08T08:53:09.435 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6255999999999995e-05 2024-05-08T08:53:09.435 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:09.435 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:09.435 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:09.435 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.435 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.435 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6696 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1142784~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:09.435 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:53:09.435 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.709415663999998, 2024-05-08T08:53:09.435 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.053051057999999998, 2024-05-08T08:53:09.436 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:09.436 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:09.436 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:09.436 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:09.436 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:09.436 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:09.436 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6696 2024-05-08T08:53:09.436 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.436 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:09.436 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.436 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:09.436 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:53:09.436 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:09.436 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.436 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.436 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:09.437 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:53:09.437 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:09.437 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.437 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.437 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:09.437 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:53:09.437 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-05-08T08:53:09.437 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.437 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.437 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:09.437 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715411+0000", 2024-05-08T08:53:09.437 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9630000000000001e-06 2024-05-08T08:53:09.437 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.437 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.437 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:09.438 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715417+0000", 2024-05-08T08:53:09.438 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7250000000000002e-06 2024-05-08T08:53:09.438 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.438 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.438 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:09.438 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715421+0000", 2024-05-08T08:53:09.438 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4089999999999996e-06 2024-05-08T08:53:09.438 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.438 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.438 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:09.438 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768277+0000", 2024-05-08T08:53:09.438 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052855293999999997 2024-05-08T08:53:09.438 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.438 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.439 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:09.439 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768358+0000", 2024-05-08T08:53:09.439 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1720999999999999e-05 2024-05-08T08:53:09.439 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.439 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.439 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:09.439 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768460+0000", 2024-05-08T08:53:09.439 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010154600000000001 2024-05-08T08:53:09.439 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:09.439 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:09.439 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:09.439 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.439 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.439 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6698 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1159168~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:09.439 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:53:09.439 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.709396796999997, 2024-05-08T08:53:09.440 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.053245726, 2024-05-08T08:53:09.440 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:09.440 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:09.440 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:09.440 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:09.440 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:09.440 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:09.440 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6698 2024-05-08T08:53:09.440 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.440 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:09.440 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.440 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:09.440 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:53:09.440 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:09.440 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.440 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.441 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:09.441 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:53:09.441 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:09.441 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.441 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.441 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:09.441 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716936+0000", 2024-05-08T08:53:09.441 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001508665 2024-05-08T08:53:09.441 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.441 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.441 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:09.441 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716941+0000", 2024-05-08T08:53:09.441 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1910000000000002e-06 2024-05-08T08:53:09.441 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.441 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.441 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:09.441 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716950+0000", 2024-05-08T08:53:09.442 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1900000000000001e-06 2024-05-08T08:53:09.442 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.442 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.442 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:09.442 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716958+0000", 2024-05-08T08:53:09.442 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8019999999999992e-06 2024-05-08T08:53:09.442 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.442 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.442 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:09.442 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768499+0000", 2024-05-08T08:53:09.442 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051540926000000001 2024-05-08T08:53:09.442 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.442 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.442 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:09.443 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768574+0000", 2024-05-08T08:53:09.443 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4988999999999996e-05 2024-05-08T08:53:09.443 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.443 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.443 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:09.443 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768673+0000", 2024-05-08T08:53:09.443 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9963000000000005e-05 2024-05-08T08:53:09.443 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:09.443 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:09.443 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:09.443 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.443 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.443 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6706 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1224704~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:09.443 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:53:09.443 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.707791489999998, 2024-05-08T08:53:09.444 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052435598, 2024-05-08T08:53:09.444 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:09.444 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:09.444 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:09.444 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:09.444 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:09.444 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:09.444 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6706 2024-05-08T08:53:09.444 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.444 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:09.444 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.444 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:09.445 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:53:09.445 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:09.445 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.445 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.445 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:09.445 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:53:09.445 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:09.445 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.445 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.445 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:09.445 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:53:09.445 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:53:09.445 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.445 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.445 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:09.446 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717035+0000", 2024-05-08T08:53:09.446 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9290000000000001e-06 2024-05-08T08:53:09.446 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.446 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.446 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:09.446 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717041+0000", 2024-05-08T08:53:09.446 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6720000000000001e-06 2024-05-08T08:53:09.446 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.446 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.446 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:09.446 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717046+0000", 2024-05-08T08:53:09.446 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5650000000000003e-06 2024-05-08T08:53:09.446 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.446 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.446 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:09.447 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769311+0000", 2024-05-08T08:53:09.447 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052265645999999999 2024-05-08T08:53:09.447 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.447 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.447 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:09.447 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769379+0000", 2024-05-08T08:53:09.447 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7391000000000006e-05 2024-05-08T08:53:09.447 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.447 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.447 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:09.447 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769469+0000", 2024-05-08T08:53:09.447 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0025999999999995e-05 2024-05-08T08:53:09.447 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:09.447 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:09.447 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:09.447 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.447 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.448 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6708 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1241088~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:09.448 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:53:09.448 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.707773386, 2024-05-08T08:53:09.448 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052640552, 2024-05-08T08:53:09.448 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:09.448 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:09.448 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:09.448 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:09.448 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:09.448 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:09.448 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6708 2024-05-08T08:53:09.448 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.448 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:09.448 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.449 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:09.449 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:53:09.449 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:09.449 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.449 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.449 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:09.449 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:53:09.449 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:09.449 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.449 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.449 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:09.449 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717052+0000", 2024-05-08T08:53:09.449 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:53:09.449 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.450 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.450 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:09.450 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717053+0000", 2024-05-08T08:53:09.450 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8330000000000001e-06 2024-05-08T08:53:09.450 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.450 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.450 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:09.450 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717058+0000", 2024-05-08T08:53:09.450 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.933e-06 2024-05-08T08:53:09.450 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.450 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.450 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:09.450 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717063+0000", 2024-05-08T08:53:09.450 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2180000000000001e-06 2024-05-08T08:53:09.450 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.451 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.451 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:09.451 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769507+0000", 2024-05-08T08:53:09.451 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052444866999999999 2024-05-08T08:53:09.451 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.451 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.451 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:09.451 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769589+0000", 2024-05-08T08:53:09.451 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2039000000000005e-05 2024-05-08T08:53:09.451 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.451 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.451 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:09.451 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769692+0000", 2024-05-08T08:53:09.451 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102252 2024-05-08T08:53:09.451 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:09.451 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:09.452 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:09.452 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.452 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.452 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6710 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1257472~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:09.452 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:53:09.452 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.707755362, 2024-05-08T08:53:09.452 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.05283057, 2024-05-08T08:53:09.452 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:09.452 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:09.452 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:09.452 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:09.452 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:09.452 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:09.453 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6710 2024-05-08T08:53:09.453 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.453 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:09.453 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.453 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:09.453 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:53:09.453 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:09.453 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.453 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.453 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:09.454 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:53:09.454 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:09.454 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.454 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.454 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:09.454 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718951+0000", 2024-05-08T08:53:09.454 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0018817809999999999 2024-05-08T08:53:09.454 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.454 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.454 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:09.454 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718956+0000", 2024-05-08T08:53:09.454 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5399999999999997e-06 2024-05-08T08:53:09.455 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.455 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.455 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:09.455 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718964+0000", 2024-05-08T08:53:09.455 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6629999999999999e-06 2024-05-08T08:53:09.455 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.455 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.455 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:09.455 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718972+0000", 2024-05-08T08:53:09.455 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5229999999999998e-06 2024-05-08T08:53:09.455 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.455 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.455 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:09.455 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769729+0000", 2024-05-08T08:53:09.456 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050756959999999997 2024-05-08T08:53:09.456 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.456 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.456 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:09.456 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769803+0000", 2024-05-08T08:53:09.456 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4538999999999999e-05 2024-05-08T08:53:09.456 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.456 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.456 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:09.456 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769900+0000", 2024-05-08T08:53:09.456 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6564000000000006e-05 2024-05-08T08:53:09.456 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:09.456 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:09.456 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:09.457 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.457 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.457 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6726 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1388544~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:09.457 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:53:09.457 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.70568059, 2024-05-08T08:53:09.457 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052451506000000002, 2024-05-08T08:53:09.457 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:09.457 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:09.457 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:09.457 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:09.457 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:09.457 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:09.457 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6726 2024-05-08T08:53:09.457 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.458 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:09.458 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.458 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:09.458 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:53:09.458 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:09.458 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.458 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.458 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:09.458 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:53:09.458 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:09.458 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.458 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.458 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:09.459 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719200+0000", 2024-05-08T08:53:09.459 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5865999999999997e-05 2024-05-08T08:53:09.459 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.459 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.459 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:09.459 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719204+0000", 2024-05-08T08:53:09.459 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8630000000000001e-06 2024-05-08T08:53:09.459 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.459 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.459 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:09.459 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719215+0000", 2024-05-08T08:53:09.459 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1590000000000001e-05 2024-05-08T08:53:09.459 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.459 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.460 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:09.460 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719220+0000", 2024-05-08T08:53:09.460 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1619999999999999e-06 2024-05-08T08:53:09.460 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.460 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.460 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:09.460 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771425+0000", 2024-05-08T08:53:09.460 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052204249000000001 2024-05-08T08:53:09.460 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.460 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.460 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:09.460 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771498+0000", 2024-05-08T08:53:09.461 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3782999999999995e-05 2024-05-08T08:53:09.461 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.461 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.461 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:09.461 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771595+0000", 2024-05-08T08:53:09.461 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6992999999999998e-05 2024-05-08T08:53:09.461 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:09.461 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:09.461 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:09.461 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.461 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.461 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6728 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1404928~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:09.461 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:53:09.461 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.705593184999998, 2024-05-08T08:53:09.461 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052590747, 2024-05-08T08:53:09.461 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:09.461 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:09.462 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:09.462 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:09.462 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:09.462 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:09.462 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6728 2024-05-08T08:53:09.462 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.462 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:09.462 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.462 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:09.462 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:53:09.462 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:09.462 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.462 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.462 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:09.463 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:53:09.463 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:09.463 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.463 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.463 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:09.463 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719232+0000", 2024-05-08T08:53:09.463 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4000000000000001e-07 2024-05-08T08:53:09.463 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.463 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.463 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:09.463 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719234+0000", 2024-05-08T08:53:09.463 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4499999999999998e-06 2024-05-08T08:53:09.463 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.463 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.463 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:09.464 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719240+0000", 2024-05-08T08:53:09.464 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5720000000000002e-06 2024-05-08T08:53:09.464 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.464 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.464 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:09.464 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719245+0000", 2024-05-08T08:53:09.464 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8690000000000003e-06 2024-05-08T08:53:09.464 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.464 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.464 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:09.464 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771636+0000", 2024-05-08T08:53:09.464 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052390979999999997 2024-05-08T08:53:09.464 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.464 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.464 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:09.464 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771717+0000", 2024-05-08T08:53:09.465 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1291999999999993e-05 2024-05-08T08:53:09.465 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.465 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.465 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:09.465 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771822+0000", 2024-05-08T08:53:09.465 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104944 2024-05-08T08:53:09.465 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:09.465 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:09.465 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:09.465 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.465 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.465 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6730 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1421312~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:09.465 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:53:09.465 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.705572943, 2024-05-08T08:53:09.465 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052770477000000003, 2024-05-08T08:53:09.465 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:09.466 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:09.466 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:09.466 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:09.466 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:09.466 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:09.466 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6730 2024-05-08T08:53:09.466 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.466 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:09.466 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.466 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:09.466 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:53:09.466 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:09.466 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.466 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.467 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:09.467 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:53:09.467 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:09.467 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.467 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.467 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:09.467 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:53:09.467 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9000000000000002e-07 2024-05-08T08:53:09.467 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.467 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.467 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:09.467 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719254+0000", 2024-05-08T08:53:09.467 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4099999999999998e-06 2024-05-08T08:53:09.467 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.467 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.468 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:09.468 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719263+0000", 2024-05-08T08:53:09.468 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4109999999999996e-06 2024-05-08T08:53:09.468 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.468 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.468 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:09.468 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719268+0000", 2024-05-08T08:53:09.468 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2669999999999997e-06 2024-05-08T08:53:09.468 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.468 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.468 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:09.468 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771846+0000", 2024-05-08T08:53:09.468 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052577677000000003 2024-05-08T08:53:09.468 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.468 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.468 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:09.468 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771920+0000", 2024-05-08T08:53:09.469 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4460000000000002e-05 2024-05-08T08:53:09.469 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.469 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.469 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:09.469 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772022+0000", 2024-05-08T08:53:09.469 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010186199999999999 2024-05-08T08:53:09.469 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:09.469 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:09.469 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:09.469 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.469 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.469 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6732 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1437696~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:09.469 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:53:09.469 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.705548135999997, 2024-05-08T08:53:09.469 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052905575000000003, 2024-05-08T08:53:09.470 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:09.470 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:09.470 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:09.470 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:09.470 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:09.470 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:09.470 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6732 2024-05-08T08:53:09.470 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.470 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:09.470 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.470 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:09.470 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:53:09.470 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:09.470 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.471 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.471 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:09.471 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:53:09.471 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:09.471 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.471 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.471 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:09.471 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721392+0000", 2024-05-08T08:53:09.471 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002115287 2024-05-08T08:53:09.471 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.471 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.471 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:09.471 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721396+0000", 2024-05-08T08:53:09.471 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1679999999999997e-06 2024-05-08T08:53:09.471 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.472 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.472 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:09.472 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721404+0000", 2024-05-08T08:53:09.472 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4409999999999993e-06 2024-05-08T08:53:09.472 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.472 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.472 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:09.472 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721410+0000", 2024-05-08T08:53:09.472 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9150000000000001e-06 2024-05-08T08:53:09.472 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.472 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.472 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:09.472 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772061+0000", 2024-05-08T08:53:09.472 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050651068 2024-05-08T08:53:09.472 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.473 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.473 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:09.473 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772127+0000", 2024-05-08T08:53:09.473 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5424000000000002e-05 2024-05-08T08:53:09.473 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.473 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.473 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:09.473 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772182+0000", 2024-05-08T08:53:09.473 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5272000000000002e-05 2024-05-08T08:53:09.473 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:09.473 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:09.473 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:09.473 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.473 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.473 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6749 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1585152~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:09.473 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:53:09.474 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.703247365999999, 2024-05-08T08:53:09.474 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052350264000000001, 2024-05-08T08:53:09.474 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:09.474 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:09.474 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:09.474 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:09.474 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:09.474 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:09.474 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6749 2024-05-08T08:53:09.474 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.474 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:09.474 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.474 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:09.475 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:53:09.475 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:09.475 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.475 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.475 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:09.475 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:53:09.475 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:09.475 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.475 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.475 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:09.475 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721578+0000", 2024-05-08T08:53:09.475 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0399999999999996e-07 2024-05-08T08:53:09.475 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.475 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.475 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:09.476 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721580+0000", 2024-05-08T08:53:09.476 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5450000000000002e-06 2024-05-08T08:53:09.476 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.476 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.476 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:09.476 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721587+0000", 2024-05-08T08:53:09.476 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0190000000000001e-06 2024-05-08T08:53:09.476 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.476 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.476 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:09.476 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721593+0000", 2024-05-08T08:53:09.476 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5659999999999997e-06 2024-05-08T08:53:09.476 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.476 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.476 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:09.477 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773759+0000", 2024-05-08T08:53:09.477 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052166553999999997 2024-05-08T08:53:09.477 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.477 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.477 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:09.477 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773833+0000", 2024-05-08T08:53:09.477 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3913999999999997e-05 2024-05-08T08:53:09.477 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.477 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.477 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:09.477 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773927+0000", 2024-05-08T08:53:09.477 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4061999999999995e-05 2024-05-08T08:53:09.477 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:09.477 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:09.477 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:09.477 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.478 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.478 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6751 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1601536~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:09.478 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:53:09.478 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.703224087000002, 2024-05-08T08:53:09.478 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052461302000000001, 2024-05-08T08:53:09.478 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:09.478 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:09.478 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:09.478 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:09.478 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:09.478 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:09.478 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6751 2024-05-08T08:53:09.478 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.478 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:09.478 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.479 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:09.479 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:53:09.479 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:09.479 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.479 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.479 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:09.479 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:53:09.479 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:09.479 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.479 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.479 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:09.479 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722939+0000", 2024-05-08T08:53:09.479 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001338601 2024-05-08T08:53:09.481 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.481 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.481 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:09.481 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722944+0000", 2024-05-08T08:53:09.481 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4469999999999999e-06 2024-05-08T08:53:09.481 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.481 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.481 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:09.481 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722955+0000", 2024-05-08T08:53:09.481 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1892999999999999e-05 2024-05-08T08:53:09.482 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.482 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.482 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:09.482 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722963+0000", 2024-05-08T08:53:09.482 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1080000000000004e-06 2024-05-08T08:53:09.482 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.482 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.482 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:09.482 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773964+0000", 2024-05-08T08:53:09.482 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051001022 2024-05-08T08:53:09.482 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.482 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.482 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:09.483 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.774006+0000", 2024-05-08T08:53:09.483 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2729999999999999e-05 2024-05-08T08:53:09.483 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.483 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.483 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:09.483 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.774062+0000", 2024-05-08T08:53:09.483 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5501000000000003e-05 2024-05-08T08:53:09.483 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:09.483 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:09.483 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:09.483 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.483 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.483 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6760 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1683456~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:09.483 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:53:09.483 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.701714711000001, 2024-05-08T08:53:09.484 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052571312000000002, 2024-05-08T08:53:09.484 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:09.484 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:09.484 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:09.484 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:09.484 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:09.484 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:09.484 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6760 2024-05-08T08:53:09.484 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.484 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:09.484 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.484 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:09.484 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:53:09.484 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:09.484 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.485 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.485 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:09.485 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:53:09.485 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:09.485 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.485 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.485 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:09.485 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:53:09.485 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2100000000000002e-07 2024-05-08T08:53:09.485 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.485 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.485 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:09.485 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723113+0000", 2024-05-08T08:53:09.485 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3149999999999999e-06 2024-05-08T08:53:09.486 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.486 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.486 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:09.486 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723118+0000", 2024-05-08T08:53:09.486 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8860000000000001e-06 2024-05-08T08:53:09.486 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.486 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.486 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:09.486 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723123+0000", 2024-05-08T08:53:09.486 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6770000000000003e-06 2024-05-08T08:53:09.486 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.486 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.486 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:09.486 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775414+0000", 2024-05-08T08:53:09.487 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052290692999999999 2024-05-08T08:53:09.487 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.487 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.487 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:09.487 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775459+0000", 2024-05-08T08:53:09.487 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5639999999999997e-05 2024-05-08T08:53:09.487 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.487 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.487 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:09.487 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775681+0000", 2024-05-08T08:53:09.487 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00022168000000000001 2024-05-08T08:53:09.487 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:09.487 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:09.487 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:09.487 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.488 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.488 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6761 3.16 3:6bd6d6da:::rbd_data.1140782590a5.0000000000000015:6 [read 1095680~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:09.488 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:53:09.488 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.701695491000002, 2024-05-08T08:53:09.488 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052751275, 2024-05-08T08:53:09.488 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:09.488 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:09.488 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:09.488 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:09.488 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:09.488 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:09.488 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6761 2024-05-08T08:53:09.488 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.488 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:09.489 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.489 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:09.489 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:53:09.489 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:09.489 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.489 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.489 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:09.489 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:53:09.489 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:09.489 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.489 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.489 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:09.489 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724939+0000", 2024-05-08T08:53:09.489 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001809817 2024-05-08T08:53:09.490 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.490 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.490 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:09.490 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724943+0000", 2024-05-08T08:53:09.490 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4920000000000004e-06 2024-05-08T08:53:09.490 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.490 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.490 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:09.490 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724956+0000", 2024-05-08T08:53:09.490 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2427e-05 2024-05-08T08:53:09.490 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.490 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.490 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:09.490 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724963+0000", 2024-05-08T08:53:09.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2559999999999996e-06 2024-05-08T08:53:09.491 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:09.491 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775711+0000", 2024-05-08T08:53:09.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050747621 2024-05-08T08:53:09.491 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:09.491 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775785+0000", 2024-05-08T08:53:09.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4634000000000004e-05 2024-05-08T08:53:09.491 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:09.491 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775880+0000", 2024-05-08T08:53:09.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5027999999999999e-05 2024-05-08T08:53:09.492 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:09.492 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:09.492 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:09.492 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.492 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.492 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6770 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1765376~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:09.492 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:53:09.492 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.699664396000003, 2024-05-08T08:53:09.492 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052339296, 2024-05-08T08:53:09.492 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:09.492 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:09.492 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:09.492 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:09.492 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:09.492 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:09.493 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6770 2024-05-08T08:53:09.493 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.493 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:09.493 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.493 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:09.493 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:53:09.493 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:09.493 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.493 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.493 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:09.493 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:53:09.493 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:09.493 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.493 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.493 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:09.493 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725161+0000", 2024-05-08T08:53:09.493 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-05-08T08:53:09.494 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.494 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.494 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:09.494 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725163+0000", 2024-05-08T08:53:09.494 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4370000000000001e-06 2024-05-08T08:53:09.494 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.494 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.494 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:09.494 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725169+0000", 2024-05-08T08:53:09.494 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8409999999999996e-06 2024-05-08T08:53:09.494 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.494 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.494 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:09.494 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725174+0000", 2024-05-08T08:53:09.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7249999999999997e-06 2024-05-08T08:53:09.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.495 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:09.495 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777343+0000", 2024-05-08T08:53:09.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052169633999999999 2024-05-08T08:53:09.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.495 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:09.495 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777409+0000", 2024-05-08T08:53:09.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6033999999999998e-05 2024-05-08T08:53:09.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.495 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:09.495 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777499+0000", 2024-05-08T08:53:09.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0224999999999998e-05 2024-05-08T08:53:09.496 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:09.496 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:09.496 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:09.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.496 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6771 3.16 3:6bd6d6da:::rbd_data.1140782590a5.0000000000000015:6 [read 1177600~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:09.496 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:53:09.496 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.699644579000001, 2024-05-08T08:53:09.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052438604999999999, 2024-05-08T08:53:09.496 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:09.496 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:09.496 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:09.496 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:09.497 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:09.497 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:09.497 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6771 2024-05-08T08:53:09.497 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.497 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:09.497 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.497 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:09.497 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:53:09.497 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:09.497 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.497 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.497 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:09.497 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:53:09.497 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:09.498 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.498 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.498 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:09.498 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:53:09.498 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:53:09.498 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.498 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.498 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:09.498 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725188+0000", 2024-05-08T08:53:09.498 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1579999999999999e-06 2024-05-08T08:53:09.498 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.498 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.498 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:09.498 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725195+0000", 2024-05-08T08:53:09.499 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8909999999999995e-06 2024-05-08T08:53:09.499 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.499 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.499 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:09.499 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725200+0000", 2024-05-08T08:53:09.499 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9810000000000003e-06 2024-05-08T08:53:09.499 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.499 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.499 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:09.499 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777527+0000", 2024-05-08T08:53:09.499 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052326582000000003 2024-05-08T08:53:09.499 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.499 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.499 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:09.499 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777566+0000", 2024-05-08T08:53:09.499 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9447999999999999e-05 2024-05-08T08:53:09.499 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.500 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.500 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:09.500 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777619+0000", 2024-05-08T08:53:09.500 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2135000000000001e-05 2024-05-08T08:53:09.500 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:09.500 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:09.500 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:09.500 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.500 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.500 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6772 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1781760~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:09.500 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:53:09.500 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.699616521999999, 2024-05-08T08:53:09.500 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052601799999999997, 2024-05-08T08:53:09.500 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:09.500 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:09.500 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:09.501 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:09.501 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:09.501 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:09.501 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6772 2024-05-08T08:53:09.501 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.501 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:09.501 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.501 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:09.501 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:53:09.501 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:09.501 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.501 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.501 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:09.501 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:53:09.501 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:09.501 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.502 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.502 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:09.502 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726939+0000", 2024-05-08T08:53:09.502 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001730493 2024-05-08T08:53:09.502 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.502 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.502 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:09.502 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726943+0000", 2024-05-08T08:53:09.502 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7330000000000003e-06 2024-05-08T08:53:09.502 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.502 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.502 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:09.502 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726954+0000", 2024-05-08T08:53:09.502 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1107000000000001e-05 2024-05-08T08:53:09.502 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.503 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:09.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726962+0000", 2024-05-08T08:53:09.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.503e-06 2024-05-08T08:53:09.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.503 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:09.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777653+0000", 2024-05-08T08:53:09.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050691542999999999 2024-05-08T08:53:09.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.503 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:09.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777719+0000", 2024-05-08T08:53:09.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5228000000000006e-05 2024-05-08T08:53:09.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.504 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:09.504 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777810+0000", 2024-05-08T08:53:09.504 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1193000000000006e-05 2024-05-08T08:53:09.504 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:09.504 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:09.504 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:09.504 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.504 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.504 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6782 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1880064~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:09.504 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:53:09.504 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.697648014999999, 2024-05-08T08:53:09.504 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052326492000000002, 2024-05-08T08:53:09.504 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:09.504 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:09.504 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:09.505 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:09.505 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:09.505 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:09.505 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6782 2024-05-08T08:53:09.505 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.505 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:09.505 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.505 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:09.505 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:53:09.505 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:09.505 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.505 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.505 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:09.505 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:53:09.505 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:09.505 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.506 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.506 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:09.506 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728939+0000", 2024-05-08T08:53:09.506 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001762719 2024-05-08T08:53:09.506 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.506 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.506 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:09.506 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728944+0000", 2024-05-08T08:53:09.506 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.746e-06 2024-05-08T08:53:09.506 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.506 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.506 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:09.506 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728955+0000", 2024-05-08T08:53:09.506 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1043000000000001e-05 2024-05-08T08:53:09.506 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.507 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.507 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:09.507 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728963+0000", 2024-05-08T08:53:09.507 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5809999999999999e-06 2024-05-08T08:53:09.507 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.507 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.507 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:09.507 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779320+0000", 2024-05-08T08:53:09.507 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050357290999999998 2024-05-08T08:53:09.507 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.507 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.507 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:09.507 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779401+0000", 2024-05-08T08:53:09.507 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0574000000000005e-05 2024-05-08T08:53:09.507 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:09.507 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:09.508 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:09.508 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779503+0000", 2024-05-08T08:53:09.508 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102538 2024-05-08T08:53:09.508 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:09.508 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:09.508 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:09.508 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:09.508 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:09.508 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:53:09.510 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:53:09.510 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:09.512+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:09.567 INFO:teuthology.orchestra.run.smithi196.stdout:2780 punch from 0xe9fe00 to 0xeaaa00, (0xac00 bytes) 2024-05-08T08:53:09.584 INFO:teuthology.orchestra.run.smithi196.stdout:2781 trunc from 0x1d45e00 to 0x93bce00 2024-05-08T08:53:09.599 INFO:teuthology.orchestra.run.smithi196.stdout:2783 trunc from 0x93bce00 to 0xcd48a00 2024-05-08T08:53:09.608 INFO:teuthology.orchestra.run.smithi196.stdout:2788 read 0x53fc200 thru 0x54046ac (0x84ad bytes) 2024-05-08T08:53:09.609 INFO:teuthology.orchestra.run.smithi196.stdout:2789 write 0xbaab600 thru 0xbab979a (0xe19b bytes) 2024-05-08T08:53:09.610 INFO:teuthology.orchestra.run.smithi196.stdout:2790 read 0xa83ca00 thru 0xa84b7d3 (0xedd4 bytes) 2024-05-08T08:53:09.610 INFO:teuthology.orchestra.run.smithi196.stdout:2792 punch from 0x23b6400 to 0x23b9e00, (0x3a00 bytes) 2024-05-08T08:53:09.610 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:09.613+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:09.613 INFO:teuthology.orchestra.run.smithi196.stdout:2793 trunc from 0xcd48a00 to 0xb33b400 2024-05-08T08:53:09.631 INFO:teuthology.orchestra.run.smithi196.stdout:2799 flatten 2024-05-08T08:53:09.640 INFO:teuthology.orchestra.run.smithi196.stdout:2800 punch from 0x601ce00 to 0x6022800, (0x5a00 bytes) 2024-05-08T08:53:09.641 INFO:teuthology.orchestra.run.smithi196.stdout:2801 write 0x5b92600 thru 0x5ba02e3 (0xdce4 bytes) 2024-05-08T08:53:09.641 INFO:teuthology.orchestra.run.smithi196.stdout:2802 write 0x55ac000 thru 0x55ae89a (0x289b bytes) 2024-05-08T08:53:09.642 INFO:teuthology.orchestra.run.smithi196.stdout:2804 read 0x6b6d000 thru 0x6b75454 (0x8455 bytes) 2024-05-08T08:53:09.642 INFO:teuthology.orchestra.run.smithi196.stdout:2806 write 0x992d800 thru 0x993bfad (0xe7ae bytes) 2024-05-08T08:53:09.642 INFO:teuthology.orchestra.run.smithi196.stdout:2807 write 0x22b0000 thru 0x22b1430 (0x1431 bytes) 2024-05-08T08:53:09.646 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:09.646 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:53:09.646 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:53:09.647 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:09.648 INFO:teuthology.orchestra.run.smithi196.stdout:2808 trunc from 0xb33b400 to 0x65e3400 2024-05-08T08:53:09.658 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:53:09.673 INFO:teuthology.orchestra.run.smithi196.stdout:2809 read 0x135ea00 thru 0x1363619 (0x4c1a bytes) 2024-05-08T08:53:09.698 INFO:teuthology.orchestra.run.smithi196.stdout:2812 write 0xd5ae600 thru 0xd5b3ff8 (0x59f9 bytes) 2024-05-08T08:53:09.699 INFO:teuthology.orchestra.run.smithi196.stdout:2815 write 0x9fcf200 thru 0x9fd10ad (0x1eae bytes) 2024-05-08T08:53:09.699 INFO:teuthology.orchestra.run.smithi196.stdout:2816 write 0x3f29c00 thru 0x3f37287 (0xd688 bytes) 2024-05-08T08:53:09.699 INFO:teuthology.orchestra.run.smithi196.stdout:2817 read 0x9123a00 thru 0x91323be (0xe9bf bytes) 2024-05-08T08:53:09.699 INFO:teuthology.orchestra.run.smithi196.stdout:2818 write 0x3fa1e00 thru 0x3faa825 (0x8a26 bytes) 2024-05-08T08:53:09.699 INFO:teuthology.orchestra.run.smithi196.stdout:2819 read 0x8f6a800 thru 0x8f7456e (0x9d6f bytes) 2024-05-08T08:53:09.699 INFO:teuthology.orchestra.run.smithi196.stdout:2820 punch from 0x253fa00 to 0x2541200, (0x1800 bytes) 2024-05-08T08:53:09.700 INFO:teuthology.orchestra.run.smithi196.stdout:2821 read 0x2508c00 thru 0x2512a54 (0x9e55 bytes) 2024-05-08T08:53:09.701 INFO:teuthology.orchestra.run.smithi196.stdout:2826 write 0x818c400 thru 0x8198a0c (0xc60d bytes) 2024-05-08T08:53:09.701 INFO:teuthology.orchestra.run.smithi196.stdout:2827 write 0x423b200 thru 0x423ee55 (0x3c56 bytes) 2024-05-08T08:53:09.701 INFO:teuthology.orchestra.run.smithi196.stdout:2830 write 0x3e10a00 thru 0x3e1dfe5 (0xd5e6 bytes) 2024-05-08T08:53:09.710 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:09.713+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:09.715 INFO:teuthology.orchestra.run.smithi196.stdout:2831 trunc from 0xd5b4000 to 0x1f33800 2024-05-08T08:53:09.742 INFO:teuthology.orchestra.run.smithi196.stdout:2832 read 0xc00000 thru 0xc0e305 (0xe306 bytes) 2024-05-08T08:53:09.758 INFO:teuthology.orchestra.run.smithi196.stdout:2833 write 0x6f4f000 thru 0x6f5cbf8 (0xdbf9 bytes) 2024-05-08T08:53:09.759 INFO:teuthology.orchestra.run.smithi196.stdout:2835 write 0x4871c00 thru 0x4873b17 (0x1f18 bytes) 2024-05-08T08:53:09.759 INFO:teuthology.orchestra.run.smithi196.stdout:2838 write 0x1556c00 thru 0x1561a43 (0xae44 bytes) 2024-05-08T08:53:09.760 INFO:teuthology.orchestra.run.smithi196.stdout:2843 read 0x51c7600 thru 0x51ced42 (0x7743 bytes) 2024-05-08T08:53:09.760 INFO:teuthology.orchestra.run.smithi196.stdout:2844 punch from 0x51d9800 to 0x51e2400, (0x8c00 bytes) 2024-05-08T08:53:09.760 INFO:teuthology.orchestra.run.smithi196.stdout:2846 read 0x1d2f800 thru 0x1d3f02e (0xf82f bytes) 2024-05-08T08:53:09.766 INFO:teuthology.orchestra.run.smithi196.stdout:2847 read 0x26ca400 thru 0x26cffa6 (0x5ba7 bytes) 2024-05-08T08:53:09.767 INFO:teuthology.orchestra.run.smithi196.stdout:2848 punch from 0xb0e800 to 0xb1c200, (0xda00 bytes) 2024-05-08T08:53:09.767 INFO:teuthology.orchestra.run.smithi196.stdout:2850 read 0x33d4400 thru 0x33d8159 (0x3d5a bytes) 2024-05-08T08:53:09.767 INFO:teuthology.orchestra.run.smithi196.stdout:2852 write 0x385a000 thru 0x385eb0a (0x4b0b bytes) 2024-05-08T08:53:09.776 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T08:53:09.783 INFO:teuthology.orchestra.run.smithi196.stdout:2854 write 0xbea4a00 thru 0xbeb2c03 (0xe204 bytes) 2024-05-08T08:53:09.785 INFO:teuthology.orchestra.run.smithi196.stdout:2855 read 0x80e7400 thru 0x80eb430 (0x4031 bytes) 2024-05-08T08:53:09.785 INFO:teuthology.orchestra.run.smithi196.stdout:2857 punch from 0x6431200 to 0x6434800, (0x3600 bytes) 2024-05-08T08:53:09.788 INFO:teuthology.orchestra.run.smithi196.stdout:2860 trunc from 0xbeb2e00 to 0xd7c1e00 2024-05-08T08:53:09.793 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:09.793 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:53:09.793 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:53:09.793 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:53:09.793 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:09.799 INFO:teuthology.orchestra.run.smithi196.stdout:2861 write 0x85b5e00 thru 0x85bc93b (0x6b3c bytes) 2024-05-08T08:53:09.801 INFO:teuthology.orchestra.run.smithi196.stdout:2862 read 0x379aa00 thru 0x379b56a (0xb6b bytes) 2024-05-08T08:53:09.804 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:53:09.811 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:09.814+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:09.814 INFO:teuthology.orchestra.run.smithi196.stdout:2863 write 0xd8e2c00 thru 0xd8e35f0 (0x9f1 bytes) 2024-05-08T08:53:09.820 INFO:teuthology.orchestra.run.smithi196.stdout:2864 trunc from 0xd8e3600 to 0x8798c00 2024-05-08T08:53:09.838 INFO:teuthology.orchestra.run.smithi196.stdout:2872 read 0xd9f800 thru 0xdaa74a (0xaf4b bytes) 2024-05-08T08:53:09.859 INFO:teuthology.orchestra.run.smithi196.stdout:2874 write 0xa998400 thru 0xa9a648b (0xe08c bytes) 2024-05-08T08:53:09.860 INFO:teuthology.orchestra.run.smithi196.stdout:2875 write 0x6eb8e00 thru 0x6ec527f (0xc480 bytes) 2024-05-08T08:53:09.863 INFO:teuthology.orchestra.run.smithi196.stdout:2880 trunc from 0xa9a6600 to 0xd752000 2024-05-08T08:53:09.877 INFO:teuthology.orchestra.run.smithi196.stdout:2882 clone 18 order 18 su 1024 sc 9 2024-05-08T08:53:09.911 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:53:09.914+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:53:09.917 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:53:09.917 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:53:09.935 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:53:09.935 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:53:09.952 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:53:09.953 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:53:09.963 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:09.963 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:09.971 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:53:09.971 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:53:09.972 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:53:09.989 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:53:09.990 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:53:10.007 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:53:10.007 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:53:10.012 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:10.015+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:10.099 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:10.101 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:10.109 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:53:10.114 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:10.117+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:10.214 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:10.218+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:10.218 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:10.218 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:10.227 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:53:10.315 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:10.318+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:10.335 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:10.336 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:10.345 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:53:10.416 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:10.419+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:10.460 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:10.460 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:10.467 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-05-08T08:53:10.469 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:53:10.517 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:10.520+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:10.617 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:53:10.621+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:53:10.621 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:10.621 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:10.630 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:53:10.718 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:10.721+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:10.779 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:10.780 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:10.788 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:53:10.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T08:53:10.823+0000 7f64ae177640 0 --2- 172.21.15.59:0/3677967346 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f64a8071050 0x7f64a81c8c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-05-08T08:53:10.832 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:10.835+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:10.895 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:10.896 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:10.904 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:53:10.946 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:10.949+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:11.021 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:11.024+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:11.050 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:11.050 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:11.059 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:53:11.122 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:11.125+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:11.205 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:11.206 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:11.214 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:53:11.258 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:11.261+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:11.319 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:11.319 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:11.324 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:11.328+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:11.328 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:53:11.349 INFO:teuthology.orchestra.run.smithi059.stderr:noscrub is set 2024-05-08T08:53:11.426 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:11.429+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:11.476 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:11.477 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:11.486 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:53:11.527 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:11.530+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:11.557 INFO:teuthology.orchestra.run.smithi196.stdout:checking clone #16, image image_client.0-clone16 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent17 2024-05-08T08:53:11.628 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:11.631+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:11.631 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:11.631 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:11.641 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:53:11.729 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:53:11.732+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:53:11.755 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:11.755 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:11.764 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:53:11.830 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:11.833+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:11.869 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:11.870 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:11.878 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:53:11.962 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:11.965+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:12.019 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T08:53:12.024 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:12.024 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:12.032 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:53:12.064 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:12.067+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:12.133 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:12.136+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:12.136 INFO:teuthology.orchestra.run.smithi196.stdout:2884 write 0xee03000 thru 0xee1044d (0xd44e bytes) 2024-05-08T08:53:12.138 INFO:teuthology.orchestra.run.smithi196.stdout:2885 read 0x769400 thru 0x771631 (0x8232 bytes) 2024-05-08T08:53:12.144 INFO:teuthology.orchestra.run.smithi196.stdout:2887 write 0xd94c400 thru 0xd9504f0 (0x40f1 bytes) 2024-05-08T08:53:12.145 INFO:teuthology.orchestra.run.smithi196.stdout:2888 flatten 2024-05-08T08:53:12.156 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:53:12.156 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:53:12.205 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:53:12.206 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:53:12.216 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:12.217 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:53:12.217 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.217 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2177 3.1b 3.41ca47db (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.217 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.208399+0000", 2024-05-08T08:53:12.217 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0031958379999999999, 2024-05-08T08:53:12.217 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0032228479999999999, 2024-05-08T08:53:12.217 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.217 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.217 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.217 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.217 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.217 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.217 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2177 2024-05-08T08:53:12.218 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.218 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.218 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.218 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.218 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208399+0000", 2024-05-08T08:53:12.218 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.218 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.218 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.218 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.218 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208399+0000", 2024-05-08T08:53:12.218 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.219 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.219 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.219 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.219 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208400+0000", 2024-05-08T08:53:12.219 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0299999999999999e-07 2024-05-08T08:53:12.219 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.219 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.219 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.219 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208402+0000", 2024-05-08T08:53:12.219 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3929999999999998e-06 2024-05-08T08:53:12.220 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.220 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.220 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.220 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208408+0000", 2024-05-08T08:53:12.220 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1360000000000002e-06 2024-05-08T08:53:12.220 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.220 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.220 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.220 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208413+0000", 2024-05-08T08:53:12.220 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9699999999999998e-06 2024-05-08T08:53:12.220 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.221 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.221 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.221 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.221 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.221 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2360 3.3 3.629e4dc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.221 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.209256+0000", 2024-05-08T08:53:12.221 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0023391639999999999, 2024-05-08T08:53:12.221 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0024320710000000001, 2024-05-08T08:53:12.221 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.221 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.221 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.221 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.222 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.222 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.222 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2360 2024-05-08T08:53:12.222 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.222 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.222 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.222 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.222 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209256+0000", 2024-05-08T08:53:12.222 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.222 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.222 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.222 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.223 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209256+0000", 2024-05-08T08:53:12.223 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.223 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.223 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.223 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.223 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209256+0000", 2024-05-08T08:53:12.223 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5600000000000001e-07 2024-05-08T08:53:12.223 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.223 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.223 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.223 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209262+0000", 2024-05-08T08:53:12.223 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0510000000000002e-06 2024-05-08T08:53:12.223 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.223 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.223 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.223 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209267+0000", 2024-05-08T08:53:12.224 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3519999999999997e-06 2024-05-08T08:53:12.224 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.224 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.224 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.224 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209272+0000", 2024-05-08T08:53:12.224 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6399999999999996e-06 2024-05-08T08:53:12.224 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.224 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.224 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.224 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.225 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.225 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2445 3.1b 3.41ca47db (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.225 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.209915+0000", 2024-05-08T08:53:12.225 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.001679732, 2024-05-08T08:53:12.225 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0018289199999999999, 2024-05-08T08:53:12.225 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.225 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.226 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.226 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.226 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.226 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.226 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2445 2024-05-08T08:53:12.226 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.226 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.226 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.226 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.226 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209915+0000", 2024-05-08T08:53:12.226 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.227 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.227 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.227 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.227 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209915+0000", 2024-05-08T08:53:12.227 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.227 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.227 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.227 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.227 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209916+0000", 2024-05-08T08:53:12.227 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1100000000000001e-07 2024-05-08T08:53:12.227 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.228 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.228 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.228 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209918+0000", 2024-05-08T08:53:12.228 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0159999999999998e-06 2024-05-08T08:53:12.228 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.228 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.228 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.229 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209923+0000", 2024-05-08T08:53:12.229 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8150000000000002e-06 2024-05-08T08:53:12.229 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.229 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.229 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.229 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209927+0000", 2024-05-08T08:53:12.229 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9940000000000002e-06 2024-05-08T08:53:12.229 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.229 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.229 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.229 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.229 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.230 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2178 3.1b 3.41ca47db (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.230 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.208420+0000", 2024-05-08T08:53:12.230 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.003174732, 2024-05-08T08:53:12.230 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0033784510000000002, 2024-05-08T08:53:12.230 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.230 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.230 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.230 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.230 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.230 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.230 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2178 2024-05-08T08:53:12.230 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.230 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.231 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.231 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.231 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208420+0000", 2024-05-08T08:53:12.231 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.231 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.231 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.231 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.231 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208420+0000", 2024-05-08T08:53:12.231 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.232 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.232 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.232 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.232 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208421+0000", 2024-05-08T08:53:12.232 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2599999999999998e-07 2024-05-08T08:53:12.232 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.232 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.232 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.232 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208423+0000", 2024-05-08T08:53:12.232 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1689999999999999e-06 2024-05-08T08:53:12.232 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.232 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.233 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.233 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208429+0000", 2024-05-08T08:53:12.233 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0920000000000003e-06 2024-05-08T08:53:12.233 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.233 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.233 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.233 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208434+0000", 2024-05-08T08:53:12.233 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9710000000000003e-06 2024-05-08T08:53:12.233 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.233 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.233 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.233 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.233 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.233 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2361 3.3 3.629e4dc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.233 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.209279+0000", 2024-05-08T08:53:12.234 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.002315592, 2024-05-08T08:53:12.234 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0025740369999999999, 2024-05-08T08:53:12.234 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.234 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.234 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.234 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.234 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.234 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.234 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2361 2024-05-08T08:53:12.234 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.235 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.235 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.235 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.235 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209279+0000", 2024-05-08T08:53:12.235 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.235 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.235 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.235 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.235 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209279+0000", 2024-05-08T08:53:12.235 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.235 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.235 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.235 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.235 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209280+0000", 2024-05-08T08:53:12.236 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9499999999999998e-07 2024-05-08T08:53:12.236 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.236 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.236 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.236 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209282+0000", 2024-05-08T08:53:12.236 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.125e-06 2024-05-08T08:53:12.236 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.236 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.236 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.236 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209288+0000", 2024-05-08T08:53:12.236 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4869999999999999e-06 2024-05-08T08:53:12.237 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.237 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.237 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.237 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209293+0000", 2024-05-08T08:53:12.237 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7299999999999996e-06 2024-05-08T08:53:12.237 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.237 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.237 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.237 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.237 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.237 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2499 3.12 3.27e58c72 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.237 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.209934+0000", 2024-05-08T08:53:12.237 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0016612860000000001, 2024-05-08T08:53:12.237 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0019752419999999999, 2024-05-08T08:53:12.238 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.238 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.238 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.238 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.239 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.239 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.239 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2499 2024-05-08T08:53:12.239 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.239 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.239 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.239 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.239 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209934+0000", 2024-05-08T08:53:12.239 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.239 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.239 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.239 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.239 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209934+0000", 2024-05-08T08:53:12.239 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.240 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.240 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.240 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.240 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209934+0000", 2024-05-08T08:53:12.240 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5499999999999999e-07 2024-05-08T08:53:12.240 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.240 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.240 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.240 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209936+0000", 2024-05-08T08:53:12.240 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0789999999999999e-06 2024-05-08T08:53:12.240 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.240 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.240 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.240 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209942+0000", 2024-05-08T08:53:12.240 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9510000000000003e-06 2024-05-08T08:53:12.241 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.241 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.241 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.241 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209946+0000", 2024-05-08T08:53:12.241 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9090000000000002e-06 2024-05-08T08:53:12.241 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.241 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.241 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.241 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.241 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.241 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2179 3.1b 3.41ca47db (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.241 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.208440+0000", 2024-05-08T08:53:12.241 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0031546080000000002, 2024-05-08T08:53:12.241 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0035360510000000001, 2024-05-08T08:53:12.241 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.242 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.242 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.242 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.242 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.242 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.242 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2179 2024-05-08T08:53:12.242 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.242 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.242 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.242 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.242 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208440+0000", 2024-05-08T08:53:12.242 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.242 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.242 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.243 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.243 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208440+0000", 2024-05-08T08:53:12.243 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.243 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.243 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.243 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.243 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208441+0000", 2024-05-08T08:53:12.243 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6499999999999999e-07 2024-05-08T08:53:12.243 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.243 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.243 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.243 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208443+0000", 2024-05-08T08:53:12.243 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9039999999999999e-06 2024-05-08T08:53:12.243 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.244 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.244 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.244 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208449+0000", 2024-05-08T08:53:12.244 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0229999999999998e-06 2024-05-08T08:53:12.244 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.244 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.244 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.244 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208453+0000", 2024-05-08T08:53:12.244 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.797e-06 2024-05-08T08:53:12.244 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.244 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.244 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.244 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.245 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.245 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2362 3.3 3.629e4dc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.245 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.209299+0000", 2024-05-08T08:53:12.245 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0022953639999999998, 2024-05-08T08:53:12.245 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0027342680000000002, 2024-05-08T08:53:12.245 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.245 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.245 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.245 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.245 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.245 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.245 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2362 2024-05-08T08:53:12.245 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.245 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.245 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.245 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.246 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209299+0000", 2024-05-08T08:53:12.246 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.246 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.246 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.246 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.246 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209299+0000", 2024-05-08T08:53:12.246 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.246 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.246 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.246 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.246 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209300+0000", 2024-05-08T08:53:12.246 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:53:12.246 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.246 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.246 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.247 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209302+0000", 2024-05-08T08:53:12.247 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1169999999999998e-06 2024-05-08T08:53:12.247 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.247 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.247 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.247 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209308+0000", 2024-05-08T08:53:12.247 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2910000000000004e-06 2024-05-08T08:53:12.247 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.247 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.247 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.247 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209313+0000", 2024-05-08T08:53:12.247 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.763e-06 2024-05-08T08:53:12.247 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.247 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.247 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.247 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.248 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.248 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2500 3.12 3.27e58c72 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.248 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.209952+0000", 2024-05-08T08:53:12.248 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0016429249999999999, 2024-05-08T08:53:12.248 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0021363179999999999, 2024-05-08T08:53:12.248 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.248 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.248 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.248 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.248 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.248 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.248 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2500 2024-05-08T08:53:12.248 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.248 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.248 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.248 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.249 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209952+0000", 2024-05-08T08:53:12.249 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.249 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.249 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.249 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.249 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209952+0000", 2024-05-08T08:53:12.249 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.249 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.249 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.249 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.249 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209952+0000", 2024-05-08T08:53:12.249 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9900000000000001e-07 2024-05-08T08:53:12.249 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.249 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.249 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.250 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209959+0000", 2024-05-08T08:53:12.250 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4230000000000002e-06 2024-05-08T08:53:12.250 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.250 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.250 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.250 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209965+0000", 2024-05-08T08:53:12.250 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.117e-06 2024-05-08T08:53:12.250 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.250 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.250 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.250 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209970+0000", 2024-05-08T08:53:12.250 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6920000000000001e-06 2024-05-08T08:53:12.250 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.250 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.250 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.251 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.251 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.251 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2180 3.1b 3.41ca47db (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.251 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.208460+0000", 2024-05-08T08:53:12.251 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0031350290000000001, 2024-05-08T08:53:12.251 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0036851700000000002, 2024-05-08T08:53:12.251 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.251 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.251 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.251 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.251 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.251 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.251 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2180 2024-05-08T08:53:12.251 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.252 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.252 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.252 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.252 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208460+0000", 2024-05-08T08:53:12.252 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.252 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.252 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.252 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.252 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208460+0000", 2024-05-08T08:53:12.252 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.252 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.252 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.252 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.252 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208460+0000", 2024-05-08T08:53:12.252 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4099999999999999e-07 2024-05-08T08:53:12.253 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.253 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.253 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.253 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208462+0000", 2024-05-08T08:53:12.253 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0099999999999998e-06 2024-05-08T08:53:12.253 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.253 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.253 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.253 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208468+0000", 2024-05-08T08:53:12.253 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0909999999999998e-06 2024-05-08T08:53:12.253 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.253 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.253 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.253 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208473+0000", 2024-05-08T08:53:12.254 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5020000000000003e-06 2024-05-08T08:53:12.254 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.254 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.254 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.254 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.254 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.254 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2363 3.3 3.629e4dc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.254 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.209319+0000", 2024-05-08T08:53:12.254 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0022755729999999999, 2024-05-08T08:53:12.254 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0028794329999999998, 2024-05-08T08:53:12.254 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.254 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.254 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.255 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.255 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.255 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.255 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2363 2024-05-08T08:53:12.255 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.255 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.255 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.255 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.255 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209319+0000", 2024-05-08T08:53:12.255 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.255 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.255 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.255 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.255 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209319+0000", 2024-05-08T08:53:12.255 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.256 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.256 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.256 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.256 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209320+0000", 2024-05-08T08:53:12.256 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9700000000000002e-07 2024-05-08T08:53:12.256 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.256 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.256 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.256 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209322+0000", 2024-05-08T08:53:12.256 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1569999999999998e-06 2024-05-08T08:53:12.256 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.256 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.256 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.256 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209328+0000", 2024-05-08T08:53:12.257 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8839999999999999e-06 2024-05-08T08:53:12.257 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.257 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.257 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.257 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209332+0000", 2024-05-08T08:53:12.257 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5159999999999996e-06 2024-05-08T08:53:12.257 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.257 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.257 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.257 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.257 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.257 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2501 3.12 3.27e58c72 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.257 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.209977+0000", 2024-05-08T08:53:12.257 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.001617734, 2024-05-08T08:53:12.258 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0022785079999999998, 2024-05-08T08:53:12.258 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.258 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.258 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.258 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.258 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.258 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.258 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2501 2024-05-08T08:53:12.258 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.258 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.258 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.258 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.258 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209977+0000", 2024-05-08T08:53:12.258 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.259 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.259 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.259 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.259 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209977+0000", 2024-05-08T08:53:12.259 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.259 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.259 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.259 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.259 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209978+0000", 2024-05-08T08:53:12.259 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4400000000000001e-07 2024-05-08T08:53:12.259 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.259 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.259 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.259 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209980+0000", 2024-05-08T08:53:12.260 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.373e-06 2024-05-08T08:53:12.260 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.260 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.260 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.260 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209986+0000", 2024-05-08T08:53:12.260 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8769999999999998e-06 2024-05-08T08:53:12.260 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.260 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.260 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.260 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209991+0000", 2024-05-08T08:53:12.260 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8049999999999997e-06 2024-05-08T08:53:12.260 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.260 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.260 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.260 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.261 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.261 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2181 3.1b 3.41ca47db (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.261 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.208479+0000", 2024-05-08T08:53:12.261 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0031157569999999998, 2024-05-08T08:53:12.261 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0038318829999999999, 2024-05-08T08:53:12.261 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.261 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.261 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.261 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.261 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.261 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.261 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2181 2024-05-08T08:53:12.261 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.261 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.261 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.262 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.262 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208479+0000", 2024-05-08T08:53:12.262 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.262 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.262 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.262 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.262 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208479+0000", 2024-05-08T08:53:12.262 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.262 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.262 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.262 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.262 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208480+0000", 2024-05-08T08:53:12.262 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6800000000000001e-07 2024-05-08T08:53:12.262 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.263 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.263 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.263 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208482+0000", 2024-05-08T08:53:12.263 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1409999999999999e-06 2024-05-08T08:53:12.263 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.263 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.263 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.263 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208488+0000", 2024-05-08T08:53:12.263 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1480000000000003e-06 2024-05-08T08:53:12.263 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.263 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.263 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.263 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208493+0000", 2024-05-08T08:53:12.263 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.656e-06 2024-05-08T08:53:12.264 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.264 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.264 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.264 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.264 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.264 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2364 3.3 3.629e4dc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.264 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.209338+0000", 2024-05-08T08:53:12.264 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0022570089999999999, 2024-05-08T08:53:12.264 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.003027325, 2024-05-08T08:53:12.264 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.264 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.264 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.264 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.264 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.265 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.265 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2364 2024-05-08T08:53:12.265 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.265 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.265 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.265 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.265 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209338+0000", 2024-05-08T08:53:12.265 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.265 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.265 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.265 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.265 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209338+0000", 2024-05-08T08:53:12.265 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.265 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.266 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.266 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.266 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209338+0000", 2024-05-08T08:53:12.266 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.72e-07 2024-05-08T08:53:12.266 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.266 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.266 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.266 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209340+0000", 2024-05-08T08:53:12.266 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9099999999999999e-06 2024-05-08T08:53:12.266 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.266 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.266 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.266 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209346+0000", 2024-05-08T08:53:12.266 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.541e-06 2024-05-08T08:53:12.266 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.267 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.267 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.267 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209350+0000", 2024-05-08T08:53:12.267 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3980000000000001e-06 2024-05-08T08:53:12.267 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.267 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.267 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.267 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.267 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.267 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2502 3.12 3.27e58c72 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.267 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.209996+0000", 2024-05-08T08:53:12.267 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0015986469999999999, 2024-05-08T08:53:12.267 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0024227820000000001, 2024-05-08T08:53:12.267 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.268 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.268 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.268 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.268 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.268 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.268 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2502 2024-05-08T08:53:12.268 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.268 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.268 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.268 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.268 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209996+0000", 2024-05-08T08:53:12.268 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.268 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.268 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.269 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.269 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209996+0000", 2024-05-08T08:53:12.269 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.269 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.269 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.269 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.269 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209997+0000", 2024-05-08T08:53:12.269 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4200000000000002e-07 2024-05-08T08:53:12.269 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.269 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.269 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.269 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.210005+0000", 2024-05-08T08:53:12.269 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3229999999999998e-06 2024-05-08T08:53:12.269 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.269 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.270 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.270 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.210011+0000", 2024-05-08T08:53:12.270 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9560000000000002e-06 2024-05-08T08:53:12.270 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.270 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.270 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.270 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.210016+0000", 2024-05-08T08:53:12.270 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8219999999999997e-06 2024-05-08T08:53:12.270 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.270 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.270 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.270 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.270 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.270 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2182 3.1b 3.41ca47db (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.271 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.208499+0000", 2024-05-08T08:53:12.271 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.003096073, 2024-05-08T08:53:12.271 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0039746479999999999, 2024-05-08T08:53:12.271 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.271 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.271 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.271 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.271 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.271 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.271 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2182 2024-05-08T08:53:12.271 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.271 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.271 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.271 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.271 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208499+0000", 2024-05-08T08:53:12.272 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.272 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.272 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.272 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.272 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208499+0000", 2024-05-08T08:53:12.272 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.272 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.272 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.272 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.272 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208499+0000", 2024-05-08T08:53:12.272 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2599999999999998e-07 2024-05-08T08:53:12.272 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.272 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.272 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.272 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208501+0000", 2024-05-08T08:53:12.273 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9180000000000001e-06 2024-05-08T08:53:12.273 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.273 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.273 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.273 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208507+0000", 2024-05-08T08:53:12.273 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1310000000000003e-06 2024-05-08T08:53:12.273 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.273 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.273 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.273 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208512+0000", 2024-05-08T08:53:12.273 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6990000000000002e-06 2024-05-08T08:53:12.273 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.273 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.273 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.273 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.273 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.274 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2365 3.3 3.629e4dc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.274 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.209355+0000", 2024-05-08T08:53:12.274 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.00223939, 2024-05-08T08:53:12.274 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.003172734, 2024-05-08T08:53:12.274 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.274 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.274 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.274 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.274 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.274 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.274 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2365 2024-05-08T08:53:12.274 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.274 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.274 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.275 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.275 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209355+0000", 2024-05-08T08:53:12.275 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.275 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.275 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.275 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.275 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209355+0000", 2024-05-08T08:53:12.275 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.275 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.275 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.275 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.275 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209356+0000", 2024-05-08T08:53:12.275 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7399999999999999e-07 2024-05-08T08:53:12.275 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.276 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.276 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.276 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209358+0000", 2024-05-08T08:53:12.276 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.8959999999999999e-06 2024-05-08T08:53:12.276 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.276 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.276 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.276 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209364+0000", 2024-05-08T08:53:12.276 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7939999999999999e-06 2024-05-08T08:53:12.276 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.276 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.276 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.276 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209368+0000", 2024-05-08T08:53:12.276 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7559999999999999e-06 2024-05-08T08:53:12.276 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.277 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.277 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.277 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.277 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.277 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2503 3.12 3.27e58c72 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.277 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.210022+0000", 2024-05-08T08:53:12.277 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0015730939999999999, 2024-05-08T08:53:12.277 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0025678770000000001, 2024-05-08T08:53:12.277 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.277 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.277 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.277 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.277 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.277 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.278 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2503 2024-05-08T08:53:12.278 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.278 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.278 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.278 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.278 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.210022+0000", 2024-05-08T08:53:12.278 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.278 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.278 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.278 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.278 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.210022+0000", 2024-05-08T08:53:12.278 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.278 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.279 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.279 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.279 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.210022+0000", 2024-05-08T08:53:12.279 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3700000000000001e-07 2024-05-08T08:53:12.279 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.279 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.279 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.279 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.210024+0000", 2024-05-08T08:53:12.279 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.8560000000000001e-06 2024-05-08T08:53:12.279 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.279 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.279 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.279 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.210030+0000", 2024-05-08T08:53:12.279 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3489999999999997e-06 2024-05-08T08:53:12.279 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.280 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.280 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.280 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.210038+0000", 2024-05-08T08:53:12.280 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5989999999999996e-06 2024-05-08T08:53:12.280 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.280 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.280 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.280 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.280 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.280 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2183 3.1b 3.41ca47db (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.280 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.208518+0000", 2024-05-08T08:53:12.280 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0030763660000000001, 2024-05-08T08:53:12.280 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0041310950000000004, 2024-05-08T08:53:12.280 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.280 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.280 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.281 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.281 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.281 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.281 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2183 2024-05-08T08:53:12.281 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.281 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.281 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.281 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.281 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208518+0000", 2024-05-08T08:53:12.281 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.281 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.281 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.281 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.281 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208518+0000", 2024-05-08T08:53:12.281 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.282 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.282 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.282 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.282 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208519+0000", 2024-05-08T08:53:12.282 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1600000000000002e-07 2024-05-08T08:53:12.282 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.282 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.282 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.282 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208521+0000", 2024-05-08T08:53:12.282 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.091e-06 2024-05-08T08:53:12.282 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.282 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.282 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.282 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208527+0000", 2024-05-08T08:53:12.283 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8209999999999998e-06 2024-05-08T08:53:12.283 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.283 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.283 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.283 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208537+0000", 2024-05-08T08:53:12.283 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0003999999999999e-05 2024-05-08T08:53:12.283 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.283 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.283 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.283 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.283 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.283 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2366 3.3 3.629e4dc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.283 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.209375+0000", 2024-05-08T08:53:12.283 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0022200269999999999, 2024-05-08T08:53:12.283 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.003329093, 2024-05-08T08:53:12.284 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.284 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.284 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.284 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.284 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.284 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.284 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2366 2024-05-08T08:53:12.284 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.284 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.284 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.284 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.284 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209375+0000", 2024-05-08T08:53:12.284 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.284 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.284 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.285 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.285 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209375+0000", 2024-05-08T08:53:12.285 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.285 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.285 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.285 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.285 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209375+0000", 2024-05-08T08:53:12.285 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8000000000000001e-07 2024-05-08T08:53:12.285 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.285 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.285 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.285 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209377+0000", 2024-05-08T08:53:12.285 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1789999999999998e-06 2024-05-08T08:53:12.285 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.285 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.286 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.286 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209383+0000", 2024-05-08T08:53:12.286 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9599999999999997e-06 2024-05-08T08:53:12.286 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.286 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.286 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.286 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209388+0000", 2024-05-08T08:53:12.286 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6149999999999999e-06 2024-05-08T08:53:12.286 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.286 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.286 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.286 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.286 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.286 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2504 3.12 3.27e58c72 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.286 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.210062+0000", 2024-05-08T08:53:12.287 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.001532414, 2024-05-08T08:53:12.287 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0026957970000000002, 2024-05-08T08:53:12.287 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.287 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.287 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.287 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.287 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.287 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.287 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2504 2024-05-08T08:53:12.287 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.287 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.287 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.287 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.287 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.210062+0000", 2024-05-08T08:53:12.287 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.287 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.288 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.288 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.288 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.210062+0000", 2024-05-08T08:53:12.288 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.288 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.288 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.288 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.288 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.210063+0000", 2024-05-08T08:53:12.288 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3900000000000005e-07 2024-05-08T08:53:12.288 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.288 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.288 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.288 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.210066+0000", 2024-05-08T08:53:12.288 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.5679999999999998e-06 2024-05-08T08:53:12.288 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.289 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.289 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.289 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.210073+0000", 2024-05-08T08:53:12.289 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.6809999999999998e-06 2024-05-08T08:53:12.289 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.289 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.289 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.289 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.210079+0000", 2024-05-08T08:53:12.289 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5979999999999996e-06 2024-05-08T08:53:12.289 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.289 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.289 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.289 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.289 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.290 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2184 3.1b 3.41ca47db (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.290 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.208731+0000", 2024-05-08T08:53:12.290 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0028639799999999999, 2024-05-08T08:53:12.290 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0040815219999999998, 2024-05-08T08:53:12.290 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.290 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.290 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.290 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.290 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.290 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.290 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2184 2024-05-08T08:53:12.290 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.290 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.291 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.291 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.291 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208731+0000", 2024-05-08T08:53:12.291 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.291 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.291 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.291 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.291 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208731+0000", 2024-05-08T08:53:12.291 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.291 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.291 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.291 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.291 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208731+0000", 2024-05-08T08:53:12.291 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9299999999999998e-07 2024-05-08T08:53:12.292 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.292 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.292 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.292 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208735+0000", 2024-05-08T08:53:12.292 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3400000000000002e-06 2024-05-08T08:53:12.292 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.292 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.292 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.292 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208752+0000", 2024-05-08T08:53:12.292 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.7614000000000001e-05 2024-05-08T08:53:12.292 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.292 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.292 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.292 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208758+0000", 2024-05-08T08:53:12.292 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5829999999999997e-06 2024-05-08T08:53:12.293 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.293 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.293 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.293 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.293 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.293 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2367 3.3 3.629e4dc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.293 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.209393+0000", 2024-05-08T08:53:12.293 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0022014729999999998, 2024-05-08T08:53:12.293 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0034737499999999998, 2024-05-08T08:53:12.293 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.293 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.293 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.293 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.293 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.293 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.294 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2367 2024-05-08T08:53:12.294 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.294 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.294 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.294 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.294 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209393+0000", 2024-05-08T08:53:12.294 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.294 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.294 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.294 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.294 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209393+0000", 2024-05-08T08:53:12.294 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.294 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.295 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.295 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.295 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209394+0000", 2024-05-08T08:53:12.295 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.58e-07 2024-05-08T08:53:12.295 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.295 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.295 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.295 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209396+0000", 2024-05-08T08:53:12.295 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.782e-06 2024-05-08T08:53:12.295 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.295 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.295 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.295 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209401+0000", 2024-05-08T08:53:12.295 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4600000000000002e-06 2024-05-08T08:53:12.295 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.295 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.296 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.296 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209405+0000", 2024-05-08T08:53:12.296 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3930000000000001e-06 2024-05-08T08:53:12.296 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.296 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.296 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.296 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.296 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.296 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2505 3.12 3.27e58c72 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.296 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.210093+0000", 2024-05-08T08:53:12.296 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.001501874, 2024-05-08T08:53:12.296 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0028314550000000001, 2024-05-08T08:53:12.296 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.297 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.297 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.297 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.297 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.297 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.297 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2505 2024-05-08T08:53:12.297 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.297 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.297 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.297 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.297 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.210093+0000", 2024-05-08T08:53:12.297 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.297 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.297 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.297 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.298 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.210093+0000", 2024-05-08T08:53:12.298 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.298 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.298 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.298 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.298 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.210094+0000", 2024-05-08T08:53:12.298 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0900000000000001e-07 2024-05-08T08:53:12.298 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.298 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.298 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.298 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.210096+0000", 2024-05-08T08:53:12.298 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4530000000000001e-06 2024-05-08T08:53:12.298 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.298 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.299 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.299 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.210103+0000", 2024-05-08T08:53:12.299 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6680000000000004e-06 2024-05-08T08:53:12.299 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.299 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.299 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.299 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.210108+0000", 2024-05-08T08:53:12.299 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7010000000000001e-06 2024-05-08T08:53:12.299 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.299 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.299 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.299 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.299 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.299 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:1917 3.1b 3:dbe25382:::rbd_data.1140100c9703.0000000000000004:14 [read 38912~1024 out=1024b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.299 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.205571+0000", 2024-05-08T08:53:12.300 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0060240299999999997, 2024-05-08T08:53:12.300 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0074189010000000003, 2024-05-08T08:53:12.300 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.300 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.300 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:12.300 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.300 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.300 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.300 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1917 2024-05-08T08:53:12.300 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.300 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.300 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.300 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.300 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205571+0000", 2024-05-08T08:53:12.300 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.300 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.300 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.301 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.301 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205571+0000", 2024-05-08T08:53:12.301 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.301 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.301 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.301 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.301 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205571+0000", 2024-05-08T08:53:12.301 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9999999999999998e-07 2024-05-08T08:53:12.301 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.301 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.301 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.301 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205574+0000", 2024-05-08T08:53:12.301 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.356e-06 2024-05-08T08:53:12.301 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.301 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.302 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.302 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205580+0000", 2024-05-08T08:53:12.302 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0240000000000003e-06 2024-05-08T08:53:12.302 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.302 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.302 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.302 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205585+0000", 2024-05-08T08:53:12.302 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0340000000000004e-06 2024-05-08T08:53:12.302 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.302 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.302 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:12.302 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.210948+0000", 2024-05-08T08:53:12.302 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0053629539999999996 2024-05-08T08:53:12.302 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.302 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.303 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:12.303 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.211026+0000", 2024-05-08T08:53:12.303 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8353e-05 2024-05-08T08:53:12.303 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.303 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.303 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.303 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.303 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.303 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2100 3.3 3.629e4dc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.303 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.207046+0000", 2024-05-08T08:53:12.303 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0045490119999999998, 2024-05-08T08:53:12.303 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0060220960000000002, 2024-05-08T08:53:12.303 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.303 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.303 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.304 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.304 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.304 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.304 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2100 2024-05-08T08:53:12.304 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.304 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.304 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.304 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.304 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207046+0000", 2024-05-08T08:53:12.304 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.304 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.304 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.304 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.305 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207046+0000", 2024-05-08T08:53:12.305 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.305 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.305 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.305 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.305 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207046+0000", 2024-05-08T08:53:12.305 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.75e-07 2024-05-08T08:53:12.305 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.305 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.305 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.305 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207049+0000", 2024-05-08T08:53:12.305 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.5380000000000001e-06 2024-05-08T08:53:12.305 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.305 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.305 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.305 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207241+0000", 2024-05-08T08:53:12.306 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00019172300000000001 2024-05-08T08:53:12.306 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.306 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.306 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.306 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207257+0000", 2024-05-08T08:53:12.306 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6433999999999999e-05 2024-05-08T08:53:12.306 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.306 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.306 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.306 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.306 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.306 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2185 3.1b 3.41ca47db (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.306 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.208769+0000", 2024-05-08T08:53:12.306 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0028262610000000001, 2024-05-08T08:53:12.307 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0043644340000000004, 2024-05-08T08:53:12.307 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.307 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.307 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.307 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.307 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.307 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.307 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2185 2024-05-08T08:53:12.307 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.307 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.307 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.307 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.307 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208769+0000", 2024-05-08T08:53:12.307 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.307 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.308 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.308 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.308 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208769+0000", 2024-05-08T08:53:12.308 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.308 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.308 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.308 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.308 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208769+0000", 2024-05-08T08:53:12.308 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6199999999999998e-07 2024-05-08T08:53:12.308 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.308 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.308 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.308 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208777+0000", 2024-05-08T08:53:12.308 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.6229999999999997e-06 2024-05-08T08:53:12.309 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.309 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.309 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.309 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208784+0000", 2024-05-08T08:53:12.309 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8099999999999998e-06 2024-05-08T08:53:12.309 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.309 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.309 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.309 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208790+0000", 2024-05-08T08:53:12.309 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6389999999999997e-06 2024-05-08T08:53:12.309 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.309 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.309 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.309 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.309 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.310 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2368 3.3 3.629e4dc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.310 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.209411+0000", 2024-05-08T08:53:12.310 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0021841740000000001, 2024-05-08T08:53:12.310 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0037768620000000002, 2024-05-08T08:53:12.310 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.310 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.310 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.310 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.310 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.310 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.310 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2368 2024-05-08T08:53:12.310 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.310 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.310 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.310 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.311 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209411+0000", 2024-05-08T08:53:12.311 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.311 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.311 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.311 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.311 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209411+0000", 2024-05-08T08:53:12.311 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.311 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.311 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.311 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.311 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209411+0000", 2024-05-08T08:53:12.311 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4400000000000001e-07 2024-05-08T08:53:12.311 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.312 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.312 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.312 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209413+0000", 2024-05-08T08:53:12.312 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.7260000000000001e-06 2024-05-08T08:53:12.312 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.312 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.312 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.312 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209418+0000", 2024-05-08T08:53:12.312 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5400000000000003e-06 2024-05-08T08:53:12.312 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.312 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.312 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.312 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209423+0000", 2024-05-08T08:53:12.312 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.515e-06 2024-05-08T08:53:12.313 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.313 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.313 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.313 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.313 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.313 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2506 3.12 3.27e58c72 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.313 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.210115+0000", 2024-05-08T08:53:12.313 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.001479916, 2024-05-08T08:53:12.313 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0031317160000000001, 2024-05-08T08:53:12.313 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.313 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.313 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.313 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.314 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.314 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.314 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2506 2024-05-08T08:53:12.314 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.314 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.314 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.314 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.314 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.210115+0000", 2024-05-08T08:53:12.314 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.314 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.314 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.314 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.314 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.210115+0000", 2024-05-08T08:53:12.315 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.315 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.315 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.315 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.315 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.210115+0000", 2024-05-08T08:53:12.315 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.39e-07 2024-05-08T08:53:12.315 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.315 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.315 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.315 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.210117+0000", 2024-05-08T08:53:12.315 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0389999999999999e-06 2024-05-08T08:53:12.315 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.315 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.315 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.315 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.210124+0000", 2024-05-08T08:53:12.316 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8940000000000004e-06 2024-05-08T08:53:12.316 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.316 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.316 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.316 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.210129+0000", 2024-05-08T08:53:12.316 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9599999999999999e-06 2024-05-08T08:53:12.316 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.316 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.316 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.316 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.316 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.316 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:1918 3.1b 3.41ca47db (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.316 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.205593+0000", 2024-05-08T08:53:12.316 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0060022030000000002, 2024-05-08T08:53:12.317 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0077094049999999999, 2024-05-08T08:53:12.317 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.317 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.317 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.317 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.317 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.317 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.317 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1918 2024-05-08T08:53:12.317 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.317 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.317 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.317 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.317 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205593+0000", 2024-05-08T08:53:12.318 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.318 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.318 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.318 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.318 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205593+0000", 2024-05-08T08:53:12.318 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.318 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.318 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.318 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.318 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205593+0000", 2024-05-08T08:53:12.318 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9099999999999999e-07 2024-05-08T08:53:12.318 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.318 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.318 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.319 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205595+0000", 2024-05-08T08:53:12.319 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1160000000000002e-06 2024-05-08T08:53:12.319 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.319 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.319 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.319 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205602+0000", 2024-05-08T08:53:12.319 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2139999999999999e-06 2024-05-08T08:53:12.319 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.319 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.319 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.319 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205607+0000", 2024-05-08T08:53:12.319 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.95e-06 2024-05-08T08:53:12.319 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.319 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.319 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.320 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.320 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.320 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2101 3.3 3.629e4dc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.320 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.207268+0000", 2024-05-08T08:53:12.320 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0043271810000000003, 2024-05-08T08:53:12.320 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0060881390000000002, 2024-05-08T08:53:12.320 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.320 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.320 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.320 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.320 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.320 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.320 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2101 2024-05-08T08:53:12.320 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.320 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.321 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.321 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.321 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207268+0000", 2024-05-08T08:53:12.321 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.321 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.321 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.321 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.321 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207268+0000", 2024-05-08T08:53:12.321 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.321 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.321 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.321 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.321 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207268+0000", 2024-05-08T08:53:12.322 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.99e-07 2024-05-08T08:53:12.322 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.322 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.322 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.322 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207271+0000", 2024-05-08T08:53:12.322 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1259999999999998e-06 2024-05-08T08:53:12.322 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.322 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.322 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.322 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207279+0000", 2024-05-08T08:53:12.322 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2010000000000002e-06 2024-05-08T08:53:12.322 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.322 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.322 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.323 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207284+0000", 2024-05-08T08:53:12.323 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7219999999999996e-06 2024-05-08T08:53:12.323 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.323 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.323 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.323 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.323 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.323 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2186 3.1b 3.41ca47db (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.323 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.208798+0000", 2024-05-08T08:53:12.323 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0027970500000000001, 2024-05-08T08:53:12.323 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0046124750000000004, 2024-05-08T08:53:12.323 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.323 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.323 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.324 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.324 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.324 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.324 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2186 2024-05-08T08:53:12.324 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.324 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.324 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.324 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.324 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208798+0000", 2024-05-08T08:53:12.324 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.324 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.324 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.324 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.324 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208798+0000", 2024-05-08T08:53:12.325 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.325 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.325 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.325 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.325 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208798+0000", 2024-05-08T08:53:12.325 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8699999999999995e-07 2024-05-08T08:53:12.325 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.325 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.325 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.325 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208801+0000", 2024-05-08T08:53:12.325 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2259999999999999e-06 2024-05-08T08:53:12.325 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.325 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.326 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.326 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208807+0000", 2024-05-08T08:53:12.326 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9800000000000003e-06 2024-05-08T08:53:12.326 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.326 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.326 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.326 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208811+0000", 2024-05-08T08:53:12.326 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4850000000000003e-06 2024-05-08T08:53:12.326 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.326 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.326 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.326 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.326 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.326 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2369 3.3 3.629e4dc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.327 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.209428+0000", 2024-05-08T08:53:12.327 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0021667560000000002, 2024-05-08T08:53:12.327 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.004039094, 2024-05-08T08:53:12.327 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.327 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.327 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.327 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.327 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.327 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.327 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2369 2024-05-08T08:53:12.327 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.327 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.327 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.327 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.328 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209428+0000", 2024-05-08T08:53:12.328 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.328 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.328 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.328 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.328 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209428+0000", 2024-05-08T08:53:12.328 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.328 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.328 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.328 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.328 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209428+0000", 2024-05-08T08:53:12.328 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.77e-07 2024-05-08T08:53:12.328 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.328 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.328 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.329 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209430+0000", 2024-05-08T08:53:12.329 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.863e-06 2024-05-08T08:53:12.329 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.329 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.329 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.329 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209436+0000", 2024-05-08T08:53:12.329 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4700000000000001e-06 2024-05-08T08:53:12.329 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.329 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.329 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.329 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209440+0000", 2024-05-08T08:53:12.329 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5399999999999997e-06 2024-05-08T08:53:12.329 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.330 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.330 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.330 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.330 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.330 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2507 3.12 3.27e58c72 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.330 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.210136+0000", 2024-05-08T08:53:12.330 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.001458647, 2024-05-08T08:53:12.330 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0033854750000000002, 2024-05-08T08:53:12.330 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.330 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.330 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.330 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.330 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.330 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.330 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2507 2024-05-08T08:53:12.331 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.331 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.331 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.331 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.331 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.210136+0000", 2024-05-08T08:53:12.331 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.331 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.331 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.331 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.331 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.210136+0000", 2024-05-08T08:53:12.331 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.331 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.331 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.331 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.331 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.210137+0000", 2024-05-08T08:53:12.331 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:53:12.332 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.332 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.332 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.332 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.210139+0000", 2024-05-08T08:53:12.332 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1600000000000001e-06 2024-05-08T08:53:12.332 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.332 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.332 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.332 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.210145+0000", 2024-05-08T08:53:12.332 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9399999999999999e-06 2024-05-08T08:53:12.332 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.332 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.332 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.332 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.210150+0000", 2024-05-08T08:53:12.333 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9919999999999998e-06 2024-05-08T08:53:12.333 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.333 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.333 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.333 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.333 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.333 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:1919 3.1b 3.41ca47db (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.333 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.205614+0000", 2024-05-08T08:53:12.333 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0059807200000000001, 2024-05-08T08:53:12.333 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0079690260000000006, 2024-05-08T08:53:12.333 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.333 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.333 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.333 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.334 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.334 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.334 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1919 2024-05-08T08:53:12.334 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.334 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.334 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.334 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.334 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205614+0000", 2024-05-08T08:53:12.334 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.334 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.334 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.334 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.334 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205614+0000", 2024-05-08T08:53:12.334 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.335 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.335 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.335 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.335 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205615+0000", 2024-05-08T08:53:12.335 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9299999999999999e-07 2024-05-08T08:53:12.335 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.335 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.335 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.335 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205617+0000", 2024-05-08T08:53:12.335 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2500000000000001e-06 2024-05-08T08:53:12.335 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.335 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.335 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.335 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205623+0000", 2024-05-08T08:53:12.335 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1600000000000003e-06 2024-05-08T08:53:12.336 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.336 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.336 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.336 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205628+0000", 2024-05-08T08:53:12.336 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9509999999999997e-06 2024-05-08T08:53:12.336 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.336 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.336 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.336 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.336 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.336 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2102 3.3 3.629e4dc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.336 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.207292+0000", 2024-05-08T08:53:12.336 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0043028390000000001, 2024-05-08T08:53:12.336 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.006347826, 2024-05-08T08:53:12.336 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.336 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.337 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.337 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.337 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.337 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.337 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2102 2024-05-08T08:53:12.337 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.337 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.337 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.337 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.337 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207292+0000", 2024-05-08T08:53:12.337 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.337 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.337 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.337 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.337 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207292+0000", 2024-05-08T08:53:12.338 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.338 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.338 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.338 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.338 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207292+0000", 2024-05-08T08:53:12.338 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2500000000000001e-07 2024-05-08T08:53:12.338 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.338 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.338 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.338 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207295+0000", 2024-05-08T08:53:12.338 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2409999999999998e-06 2024-05-08T08:53:12.338 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.338 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.338 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.339 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207301+0000", 2024-05-08T08:53:12.339 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3940000000000001e-06 2024-05-08T08:53:12.339 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.339 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.339 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.339 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207306+0000", 2024-05-08T08:53:12.339 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.045e-06 2024-05-08T08:53:12.339 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.339 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.339 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.339 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.339 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.339 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2187 3.1b 3.41ca47db (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.339 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.208818+0000", 2024-05-08T08:53:12.339 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0027770799999999999, 2024-05-08T08:53:12.340 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0048764259999999997, 2024-05-08T08:53:12.340 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.340 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.340 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.340 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.340 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.340 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.340 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2187 2024-05-08T08:53:12.340 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.340 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.340 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.340 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.340 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208818+0000", 2024-05-08T08:53:12.340 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.340 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.341 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.341 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.341 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208818+0000", 2024-05-08T08:53:12.341 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.341 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.341 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.341 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.341 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208818+0000", 2024-05-08T08:53:12.341 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7100000000000002e-07 2024-05-08T08:53:12.341 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.341 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.341 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.341 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208820+0000", 2024-05-08T08:53:12.341 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0190000000000001e-06 2024-05-08T08:53:12.341 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.342 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.342 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.342 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208827+0000", 2024-05-08T08:53:12.342 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.241e-06 2024-05-08T08:53:12.342 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.342 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.342 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.342 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208831+0000", 2024-05-08T08:53:12.342 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9459999999999997e-06 2024-05-08T08:53:12.342 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.342 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.342 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.342 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.342 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.342 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2370 3.3 3.629e4dc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.343 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.209446+0000", 2024-05-08T08:53:12.343 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.002149078, 2024-05-08T08:53:12.343 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.004304441, 2024-05-08T08:53:12.343 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.343 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.343 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.343 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.343 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.343 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.343 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2370 2024-05-08T08:53:12.343 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.343 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.343 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.343 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.343 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209446+0000", 2024-05-08T08:53:12.344 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.344 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.344 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.344 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.344 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209446+0000", 2024-05-08T08:53:12.344 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.344 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.344 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.344 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.344 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209446+0000", 2024-05-08T08:53:12.344 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7099999999999997e-07 2024-05-08T08:53:12.344 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.344 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.344 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.344 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209448+0000", 2024-05-08T08:53:12.345 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.852e-06 2024-05-08T08:53:12.345 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.345 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.345 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.345 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209453+0000", 2024-05-08T08:53:12.345 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3890000000000004e-06 2024-05-08T08:53:12.345 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.345 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.345 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.345 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209458+0000", 2024-05-08T08:53:12.345 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.549e-06 2024-05-08T08:53:12.345 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.345 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.345 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.345 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.346 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.346 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2508 3.12 3.27e58c72 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.346 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.210156+0000", 2024-05-08T08:53:12.346 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.001438943, 2024-05-08T08:53:12.346 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.003649504, 2024-05-08T08:53:12.346 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.346 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.346 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.346 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.346 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.346 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.346 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2508 2024-05-08T08:53:12.346 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.347 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.347 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.347 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.347 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.210156+0000", 2024-05-08T08:53:12.347 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.347 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.347 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.347 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.347 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.210156+0000", 2024-05-08T08:53:12.347 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.347 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.347 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.347 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.347 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.210156+0000", 2024-05-08T08:53:12.347 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7899999999999998e-07 2024-05-08T08:53:12.348 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.348 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.348 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.348 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.210159+0000", 2024-05-08T08:53:12.348 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.03e-06 2024-05-08T08:53:12.348 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.348 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.348 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.348 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.210165+0000", 2024-05-08T08:53:12.348 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0719999999999996e-06 2024-05-08T08:53:12.348 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.348 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.348 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.348 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.210170+0000", 2024-05-08T08:53:12.348 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0900000000000004e-06 2024-05-08T08:53:12.349 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.349 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.349 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.349 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.349 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.349 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:1920 3.1b 3.41ca47db (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.349 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.205634+0000", 2024-05-08T08:53:12.349 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0059608200000000004, 2024-05-08T08:53:12.349 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0082293739999999994, 2024-05-08T08:53:12.349 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.349 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.349 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.349 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.349 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.350 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.350 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1920 2024-05-08T08:53:12.350 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.350 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.350 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.351 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.351 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205634+0000", 2024-05-08T08:53:12.351 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.351 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.351 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.351 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.352 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205634+0000", 2024-05-08T08:53:12.352 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.352 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.352 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.352 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.352 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205634+0000", 2024-05-08T08:53:12.352 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.2399999999999999e-07 2024-05-08T08:53:12.352 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.352 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.352 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.352 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205636+0000", 2024-05-08T08:53:12.352 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0820000000000001e-06 2024-05-08T08:53:12.352 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.352 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.352 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.353 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205642+0000", 2024-05-08T08:53:12.353 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.823e-06 2024-05-08T08:53:12.353 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.353 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.353 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.353 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205647+0000", 2024-05-08T08:53:12.353 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9989999999999999e-06 2024-05-08T08:53:12.353 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.353 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.353 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.353 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.353 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.353 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2103 3.3 3.629e4dc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.353 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.207313+0000", 2024-05-08T08:53:12.353 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0042823419999999997, 2024-05-08T08:53:12.354 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0066086929999999997, 2024-05-08T08:53:12.354 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.354 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.354 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.354 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.354 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.354 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.354 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2103 2024-05-08T08:53:12.354 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.354 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.354 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.354 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.354 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207313+0000", 2024-05-08T08:53:12.354 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.355 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.355 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.355 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.355 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207313+0000", 2024-05-08T08:53:12.355 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.355 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.355 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.355 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.355 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207313+0000", 2024-05-08T08:53:12.355 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2199999999999999e-07 2024-05-08T08:53:12.355 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.355 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.355 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.355 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207315+0000", 2024-05-08T08:53:12.356 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.996e-06 2024-05-08T08:53:12.356 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.356 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.356 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.356 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207321+0000", 2024-05-08T08:53:12.356 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7819999999999999e-06 2024-05-08T08:53:12.356 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.356 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.356 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.356 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207326+0000", 2024-05-08T08:53:12.356 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2480000000000004e-06 2024-05-08T08:53:12.357 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.357 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.357 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.357 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.357 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.357 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2188 3.1b 3.41ca47db (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.357 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.208838+0000", 2024-05-08T08:53:12.357 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.002756831, 2024-05-08T08:53:12.357 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0051432190000000001, 2024-05-08T08:53:12.357 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.357 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.358 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.358 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.358 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.358 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.358 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2188 2024-05-08T08:53:12.358 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.358 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.358 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.358 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.358 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208838+0000", 2024-05-08T08:53:12.358 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.358 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.358 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.359 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.359 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208838+0000", 2024-05-08T08:53:12.359 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.359 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.359 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.359 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.359 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208838+0000", 2024-05-08T08:53:12.359 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9200000000000002e-07 2024-05-08T08:53:12.359 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.359 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.359 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.359 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208840+0000", 2024-05-08T08:53:12.359 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.8649999999999999e-06 2024-05-08T08:53:12.359 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.359 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.360 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.360 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208846+0000", 2024-05-08T08:53:12.360 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8050000000000003e-06 2024-05-08T08:53:12.360 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.360 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.360 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.360 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208851+0000", 2024-05-08T08:53:12.360 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5879999999999999e-06 2024-05-08T08:53:12.360 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.360 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.360 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.360 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.360 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.360 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2371 3.3 3.629e4dc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.360 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.209463+0000", 2024-05-08T08:53:12.360 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0021314400000000001, 2024-05-08T08:53:12.361 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0045746709999999998, 2024-05-08T08:53:12.361 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.361 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.361 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.361 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.361 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.361 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.361 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2371 2024-05-08T08:53:12.361 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.361 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.361 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.361 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.361 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209463+0000", 2024-05-08T08:53:12.361 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.361 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.362 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.362 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.362 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209463+0000", 2024-05-08T08:53:12.362 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.362 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.362 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.362 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.362 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209464+0000", 2024-05-08T08:53:12.362 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6100000000000002e-07 2024-05-08T08:53:12.362 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.362 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.362 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.362 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209466+0000", 2024-05-08T08:53:12.363 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.939e-06 2024-05-08T08:53:12.363 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.363 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.363 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.363 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209471+0000", 2024-05-08T08:53:12.363 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7440000000000004e-06 2024-05-08T08:53:12.363 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.363 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.363 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.363 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209476+0000", 2024-05-08T08:53:12.363 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6569999999999996e-06 2024-05-08T08:53:12.363 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.363 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.364 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.364 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.364 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.364 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:1921 3.1b 3.41ca47db (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.364 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.205654+0000", 2024-05-08T08:53:12.364 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.005940557, 2024-05-08T08:53:12.364 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0084395379999999999, 2024-05-08T08:53:12.364 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.364 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.364 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.364 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.364 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.364 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.364 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1921 2024-05-08T08:53:12.364 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.365 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.365 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.365 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.365 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205654+0000", 2024-05-08T08:53:12.365 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.365 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.365 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.365 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.365 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205654+0000", 2024-05-08T08:53:12.365 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.365 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.365 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.365 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.365 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205655+0000", 2024-05-08T08:53:12.365 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1600000000000002e-07 2024-05-08T08:53:12.366 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.366 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.366 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.366 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205657+0000", 2024-05-08T08:53:12.366 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0700000000000001e-06 2024-05-08T08:53:12.366 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.366 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.366 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.366 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205665+0000", 2024-05-08T08:53:12.366 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7239999999999992e-06 2024-05-08T08:53:12.366 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.366 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.366 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.367 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205669+0000", 2024-05-08T08:53:12.367 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.78e-06 2024-05-08T08:53:12.367 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.367 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.367 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.367 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.367 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.367 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2104 3.3 3.629e4dc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.367 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.207333+0000", 2024-05-08T08:53:12.367 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0042620920000000003, 2024-05-08T08:53:12.367 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0068164439999999996, 2024-05-08T08:53:12.367 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.367 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.367 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.368 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.368 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.368 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.368 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2104 2024-05-08T08:53:12.368 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.368 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.368 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.368 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.368 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207333+0000", 2024-05-08T08:53:12.368 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.368 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.368 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.368 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.368 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207333+0000", 2024-05-08T08:53:12.368 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.369 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.369 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.369 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.369 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207333+0000", 2024-05-08T08:53:12.369 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6699999999999999e-07 2024-05-08T08:53:12.369 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.369 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.369 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.369 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207335+0000", 2024-05-08T08:53:12.369 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9630000000000001e-06 2024-05-08T08:53:12.369 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.369 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.369 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.369 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207341+0000", 2024-05-08T08:53:12.370 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2659999999999998e-06 2024-05-08T08:53:12.370 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.370 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.370 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.370 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207346+0000", 2024-05-08T08:53:12.370 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7230000000000004e-06 2024-05-08T08:53:12.370 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.370 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.370 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.370 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.370 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.370 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2189 3.1b 3.41ca47db (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.370 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.208858+0000", 2024-05-08T08:53:12.370 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0027372299999999998, 2024-05-08T08:53:12.370 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0053476249999999999, 2024-05-08T08:53:12.371 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.371 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.371 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.371 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.371 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.371 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.371 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2189 2024-05-08T08:53:12.371 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.371 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.371 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.371 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.371 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208858+0000", 2024-05-08T08:53:12.371 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.371 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.371 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.372 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.372 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208858+0000", 2024-05-08T08:53:12.372 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.372 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.372 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.372 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.372 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208858+0000", 2024-05-08T08:53:12.372 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3099999999999998e-07 2024-05-08T08:53:12.372 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.372 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.372 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.372 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208860+0000", 2024-05-08T08:53:12.372 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2900000000000001e-06 2024-05-08T08:53:12.373 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.373 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.373 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.373 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208866+0000", 2024-05-08T08:53:12.373 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0750000000000003e-06 2024-05-08T08:53:12.373 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.373 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.373 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.373 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208871+0000", 2024-05-08T08:53:12.373 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6949999999999999e-06 2024-05-08T08:53:12.373 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.373 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.373 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.373 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.373 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.374 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2372 3.3 3.629e4dc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.374 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.209483+0000", 2024-05-08T08:53:12.374 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0021121590000000002, 2024-05-08T08:53:12.374 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0047794999999999999, 2024-05-08T08:53:12.374 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.374 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.374 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.374 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.374 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.374 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.374 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2372 2024-05-08T08:53:12.374 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.374 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.375 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.375 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.375 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209483+0000", 2024-05-08T08:53:12.375 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.375 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.375 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.375 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.375 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209483+0000", 2024-05-08T08:53:12.375 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.375 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.375 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.375 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.375 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209483+0000", 2024-05-08T08:53:12.376 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3000000000000001e-07 2024-05-08T08:53:12.376 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.376 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.376 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.376 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209490+0000", 2024-05-08T08:53:12.376 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0219999999999999e-06 2024-05-08T08:53:12.376 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.376 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.376 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.376 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209497+0000", 2024-05-08T08:53:12.376 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4160000000000001e-06 2024-05-08T08:53:12.376 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.376 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.377 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.377 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209502+0000", 2024-05-08T08:53:12.377 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1100000000000002e-06 2024-05-08T08:53:12.377 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.377 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.377 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.377 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.377 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.377 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:1922 3.1b 3.41ca47db (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.377 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.205676+0000", 2024-05-08T08:53:12.377 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.005918782, 2024-05-08T08:53:12.377 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0086421599999999994, 2024-05-08T08:53:12.377 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.377 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.378 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.378 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.378 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.378 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.378 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1922 2024-05-08T08:53:12.378 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.378 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.378 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.378 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.378 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205676+0000", 2024-05-08T08:53:12.378 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.378 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.378 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.378 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.379 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205676+0000", 2024-05-08T08:53:12.379 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.379 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.379 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.379 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.379 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205676+0000", 2024-05-08T08:53:12.379 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9799999999999999e-07 2024-05-08T08:53:12.379 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.379 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.379 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.379 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205684+0000", 2024-05-08T08:53:12.379 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.6279999999999996e-06 2024-05-08T08:53:12.379 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.379 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.380 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.380 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205690+0000", 2024-05-08T08:53:12.380 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0809999999999999e-06 2024-05-08T08:53:12.380 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.380 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.380 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.380 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205695+0000", 2024-05-08T08:53:12.380 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8679999999999998e-06 2024-05-08T08:53:12.380 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.380 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.380 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.380 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.380 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.380 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2105 3.3 3.629e4dc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.381 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.207352+0000", 2024-05-08T08:53:12.381 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0042424300000000002, 2024-05-08T08:53:12.381 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0070216790000000003, 2024-05-08T08:53:12.381 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.381 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.381 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.381 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.381 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.381 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.381 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2105 2024-05-08T08:53:12.381 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.381 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.381 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.381 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.381 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207352+0000", 2024-05-08T08:53:12.382 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.382 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.382 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.382 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.382 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207352+0000", 2024-05-08T08:53:12.382 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.382 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.382 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.382 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.382 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207353+0000", 2024-05-08T08:53:12.382 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3500000000000002e-07 2024-05-08T08:53:12.382 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.382 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.382 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.383 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207355+0000", 2024-05-08T08:53:12.383 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9810000000000002e-06 2024-05-08T08:53:12.383 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.383 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.383 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.383 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207361+0000", 2024-05-08T08:53:12.383 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.993e-06 2024-05-08T08:53:12.383 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.383 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.383 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.383 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207366+0000", 2024-05-08T08:53:12.383 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0080000000000002e-06 2024-05-08T08:53:12.383 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.384 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.384 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.384 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.384 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.384 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2373 3.3 3.629e4dc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.384 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.209508+0000", 2024-05-08T08:53:12.384 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0020867860000000002, 2024-05-08T08:53:12.384 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0049224899999999999, 2024-05-08T08:53:12.384 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.384 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.384 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.384 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.384 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.384 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.384 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2373 2024-05-08T08:53:12.385 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.385 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.385 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.385 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.385 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209508+0000", 2024-05-08T08:53:12.385 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.385 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.385 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.385 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.385 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209508+0000", 2024-05-08T08:53:12.385 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.385 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.385 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.385 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.386 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209509+0000", 2024-05-08T08:53:12.386 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3099999999999998e-07 2024-05-08T08:53:12.386 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.386 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.386 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.386 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209510+0000", 2024-05-08T08:53:12.386 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.889e-06 2024-05-08T08:53:12.386 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.386 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.386 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.386 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209516+0000", 2024-05-08T08:53:12.386 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2299999999999999e-06 2024-05-08T08:53:12.386 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.386 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.387 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.387 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209520+0000", 2024-05-08T08:53:12.387 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5090000000000004e-06 2024-05-08T08:53:12.387 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.387 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.387 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.387 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.387 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.387 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:1923 3.1b 3.41ca47db (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.387 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.205703+0000", 2024-05-08T08:53:12.387 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0058914869999999999, 2024-05-08T08:53:12.387 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.008782794, 2024-05-08T08:53:12.387 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.387 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.387 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.387 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.388 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.388 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.388 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1923 2024-05-08T08:53:12.388 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.388 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.388 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.388 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.388 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205703+0000", 2024-05-08T08:53:12.388 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.388 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.388 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.388 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.388 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205703+0000", 2024-05-08T08:53:12.388 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.388 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.389 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.389 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.389 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205704+0000", 2024-05-08T08:53:12.389 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6100000000000001e-07 2024-05-08T08:53:12.389 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.389 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.389 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.389 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205706+0000", 2024-05-08T08:53:12.389 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3630000000000001e-06 2024-05-08T08:53:12.389 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.389 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.389 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.389 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205712+0000", 2024-05-08T08:53:12.389 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.105e-06 2024-05-08T08:53:12.389 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.390 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.390 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.390 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205722+0000", 2024-05-08T08:53:12.390 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2259999999999994e-06 2024-05-08T08:53:12.390 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.390 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.390 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.390 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.390 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.390 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2106 3.3 3.629e4dc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.390 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.207372+0000", 2024-05-08T08:53:12.390 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0042225609999999997, 2024-05-08T08:53:12.391 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0071689229999999998, 2024-05-08T08:53:12.391 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.391 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.391 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.391 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.391 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.391 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.391 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2106 2024-05-08T08:53:12.391 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.391 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.391 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.391 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.391 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207372+0000", 2024-05-08T08:53:12.391 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.391 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.392 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.392 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.392 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207372+0000", 2024-05-08T08:53:12.392 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.392 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.392 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.392 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.392 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207373+0000", 2024-05-08T08:53:12.392 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4700000000000002e-07 2024-05-08T08:53:12.392 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.392 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.392 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.392 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207375+0000", 2024-05-08T08:53:12.392 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1629999999999999e-06 2024-05-08T08:53:12.392 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.392 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.392 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.393 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207381+0000", 2024-05-08T08:53:12.393 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7799999999999997e-06 2024-05-08T08:53:12.393 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.393 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.393 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.393 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207391+0000", 2024-05-08T08:53:12.393 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.063e-05 2024-05-08T08:53:12.393 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.393 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.393 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.393 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.393 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.393 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2374 3.3 3.629e4dc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.393 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.209527+0000", 2024-05-08T08:53:12.394 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0020682180000000001, 2024-05-08T08:53:12.394 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0050706420000000002, 2024-05-08T08:53:12.394 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.394 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.394 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.394 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.394 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.394 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.394 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2374 2024-05-08T08:53:12.394 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.394 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.394 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.394 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.395 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209527+0000", 2024-05-08T08:53:12.395 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.395 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.395 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.395 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.395 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209527+0000", 2024-05-08T08:53:12.395 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.395 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.395 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.395 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.395 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209527+0000", 2024-05-08T08:53:12.395 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7e-07 2024-05-08T08:53:12.395 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.395 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.395 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.395 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209529+0000", 2024-05-08T08:53:12.396 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0669999999999999e-06 2024-05-08T08:53:12.396 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.396 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.396 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.396 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209535+0000", 2024-05-08T08:53:12.396 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8930000000000002e-06 2024-05-08T08:53:12.396 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.396 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.396 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.396 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209545+0000", 2024-05-08T08:53:12.396 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.927e-06 2024-05-08T08:53:12.396 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.396 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.396 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.396 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.396 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.397 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:1924 3.1b 3.41ca47db (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.397 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.205728+0000", 2024-05-08T08:53:12.397 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.005866596, 2024-05-08T08:53:12.397 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0089243880000000001, 2024-05-08T08:53:12.397 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.397 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.397 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.397 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.397 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.397 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.397 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1924 2024-05-08T08:53:12.397 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.397 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.397 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.397 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.398 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205728+0000", 2024-05-08T08:53:12.398 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.398 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.398 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.398 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.398 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205728+0000", 2024-05-08T08:53:12.398 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.398 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.398 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.398 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.398 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205729+0000", 2024-05-08T08:53:12.398 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1899999999999998e-07 2024-05-08T08:53:12.398 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.398 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.398 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.399 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205731+0000", 2024-05-08T08:53:12.399 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3760000000000002e-06 2024-05-08T08:53:12.399 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.399 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.399 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.399 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205737+0000", 2024-05-08T08:53:12.399 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9889999999999997e-06 2024-05-08T08:53:12.399 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.399 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.399 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.399 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205742+0000", 2024-05-08T08:53:12.399 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7770000000000002e-06 2024-05-08T08:53:12.399 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.399 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.399 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.400 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.400 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.400 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2107 3.3 3.629e4dc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.400 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.207399+0000", 2024-05-08T08:53:12.400 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0041960410000000002, 2024-05-08T08:53:12.400 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0073095210000000002, 2024-05-08T08:53:12.400 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.400 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.400 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.400 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.400 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.400 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.400 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2107 2024-05-08T08:53:12.400 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.400 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.401 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.401 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.401 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207399+0000", 2024-05-08T08:53:12.401 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.401 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.401 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.401 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.401 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207399+0000", 2024-05-08T08:53:12.401 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.401 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.401 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.401 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.401 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207399+0000", 2024-05-08T08:53:12.401 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2399999999999999e-07 2024-05-08T08:53:12.402 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.402 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.402 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.402 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207401+0000", 2024-05-08T08:53:12.402 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1940000000000001e-06 2024-05-08T08:53:12.402 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.402 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.402 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.402 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207407+0000", 2024-05-08T08:53:12.402 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8819999999999998e-06 2024-05-08T08:53:12.402 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.402 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.402 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.402 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207412+0000", 2024-05-08T08:53:12.402 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5669999999999996e-06 2024-05-08T08:53:12.403 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.403 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.403 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.403 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.403 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.403 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2375 3.3 3.629e4dc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.403 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.209553+0000", 2024-05-08T08:53:12.403 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.002042262, 2024-05-08T08:53:12.403 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0052117140000000001, 2024-05-08T08:53:12.403 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.403 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.404 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.404 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.404 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.404 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.404 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2375 2024-05-08T08:53:12.404 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.404 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.404 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.404 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.404 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209553+0000", 2024-05-08T08:53:12.404 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.404 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.404 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.404 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.404 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209553+0000", 2024-05-08T08:53:12.405 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.405 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.405 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.405 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.405 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209553+0000", 2024-05-08T08:53:12.405 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4999999999999998e-07 2024-05-08T08:53:12.405 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.405 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.405 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.405 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209556+0000", 2024-05-08T08:53:12.405 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4940000000000002e-06 2024-05-08T08:53:12.405 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.405 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.406 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.406 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209563+0000", 2024-05-08T08:53:12.406 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4479999999999997e-06 2024-05-08T08:53:12.406 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.406 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.406 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.406 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209568+0000", 2024-05-08T08:53:12.406 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3329999999999996e-06 2024-05-08T08:53:12.406 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.406 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.406 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.406 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.406 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.406 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:1925 3.1b 3.41ca47db (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.407 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.205749+0000", 2024-05-08T08:53:12.407 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0058454880000000002, 2024-05-08T08:53:12.407 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0090714400000000001, 2024-05-08T08:53:12.407 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.407 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.407 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.407 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.407 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.407 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.407 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1925 2024-05-08T08:53:12.407 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.407 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.407 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.407 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.407 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205749+0000", 2024-05-08T08:53:12.408 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.408 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.408 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.408 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.408 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205749+0000", 2024-05-08T08:53:12.408 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.408 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.408 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.408 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.408 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205750+0000", 2024-05-08T08:53:12.408 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.84e-07 2024-05-08T08:53:12.408 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.408 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.408 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.408 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205752+0000", 2024-05-08T08:53:12.409 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1380000000000001e-06 2024-05-08T08:53:12.409 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.409 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.409 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.409 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205766+0000", 2024-05-08T08:53:12.409 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3930999999999999e-05 2024-05-08T08:53:12.409 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.409 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.409 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.409 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205772+0000", 2024-05-08T08:53:12.409 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3810000000000004e-06 2024-05-08T08:53:12.409 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.409 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.409 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.409 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.410 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.410 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2108 3.3 3.629e4dc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.410 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.207426+0000", 2024-05-08T08:53:12.410 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0041693520000000003, 2024-05-08T08:53:12.410 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0074509600000000004, 2024-05-08T08:53:12.410 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.410 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.410 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.410 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.410 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.410 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.410 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2108 2024-05-08T08:53:12.410 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.410 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.411 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.411 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.411 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207426+0000", 2024-05-08T08:53:12.411 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.411 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.411 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.411 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.411 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207426+0000", 2024-05-08T08:53:12.411 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.411 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.411 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.411 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.411 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207615+0000", 2024-05-08T08:53:12.411 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00018952199999999999 2024-05-08T08:53:12.411 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.412 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.412 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.412 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207635+0000", 2024-05-08T08:53:12.412 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0146e-05 2024-05-08T08:53:12.412 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.412 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.412 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.412 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207642+0000", 2024-05-08T08:53:12.412 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9970000000000001e-06 2024-05-08T08:53:12.412 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.412 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.412 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.412 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207648+0000", 2024-05-08T08:53:12.412 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2020000000000001e-06 2024-05-08T08:53:12.412 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.413 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.413 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.413 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.413 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.413 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2429 3.1b 3.41ca47db (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.413 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.209576+0000", 2024-05-08T08:53:12.413 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0020190160000000002, 2024-05-08T08:53:12.413 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0053568239999999996, 2024-05-08T08:53:12.413 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.413 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.413 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.413 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.413 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.413 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.413 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2429 2024-05-08T08:53:12.414 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.414 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.414 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.414 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.414 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209576+0000", 2024-05-08T08:53:12.414 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.414 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.414 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.414 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.414 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209576+0000", 2024-05-08T08:53:12.414 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.414 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.414 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.414 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.415 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209576+0000", 2024-05-08T08:53:12.415 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0499999999999999e-07 2024-05-08T08:53:12.415 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.415 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.415 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.415 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209579+0000", 2024-05-08T08:53:12.415 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3250000000000002e-06 2024-05-08T08:53:12.415 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.415 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.415 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.415 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209586+0000", 2024-05-08T08:53:12.415 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0500000000000003e-06 2024-05-08T08:53:12.415 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.415 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.415 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.416 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209591+0000", 2024-05-08T08:53:12.416 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.643e-06 2024-05-08T08:53:12.416 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.416 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.416 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.416 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.416 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.416 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:1926 3.1b 3.41ca47db (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.416 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.205785+0000", 2024-05-08T08:53:12.416 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0058098849999999999, 2024-05-08T08:53:12.416 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0092192119999999992, 2024-05-08T08:53:12.416 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.416 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.416 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.416 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.417 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.417 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.417 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1926 2024-05-08T08:53:12.417 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.417 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.417 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.417 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.417 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205785+0000", 2024-05-08T08:53:12.417 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.417 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.417 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.417 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.417 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205785+0000", 2024-05-08T08:53:12.417 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.417 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.418 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.418 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.418 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205786+0000", 2024-05-08T08:53:12.418 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8100000000000003e-07 2024-05-08T08:53:12.418 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.418 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.418 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.418 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205793+0000", 2024-05-08T08:53:12.418 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3379999999999999e-06 2024-05-08T08:53:12.418 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.418 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.418 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.418 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205803+0000", 2024-05-08T08:53:12.419 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6320000000000003e-06 2024-05-08T08:53:12.419 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.419 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.419 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.419 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205807+0000", 2024-05-08T08:53:12.419 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9269999999999996e-06 2024-05-08T08:53:12.419 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.419 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.419 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.419 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.419 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.419 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2109 3.3 3.629e4dc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.419 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.207661+0000", 2024-05-08T08:53:12.419 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0039339939999999997, 2024-05-08T08:53:12.419 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.007399353, 2024-05-08T08:53:12.420 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.420 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.420 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.420 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.420 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.420 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.420 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2109 2024-05-08T08:53:12.420 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.420 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.420 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.420 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.420 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207661+0000", 2024-05-08T08:53:12.420 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.420 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.421 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.421 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.421 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207661+0000", 2024-05-08T08:53:12.421 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.421 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.421 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.421 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.421 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207661+0000", 2024-05-08T08:53:12.421 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6899999999999997e-07 2024-05-08T08:53:12.421 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.421 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.421 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.421 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207668+0000", 2024-05-08T08:53:12.421 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5100000000000004e-06 2024-05-08T08:53:12.421 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.422 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.422 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.422 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207675+0000", 2024-05-08T08:53:12.422 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6989999999999997e-06 2024-05-08T08:53:12.422 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.422 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.422 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.422 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207680+0000", 2024-05-08T08:53:12.422 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3059999999999997e-06 2024-05-08T08:53:12.422 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.422 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.422 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.422 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.422 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.423 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2430 3.1b 3.41ca47db (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.423 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.209599+0000", 2024-05-08T08:53:12.423 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0019962859999999999, 2024-05-08T08:53:12.423 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0055177799999999999, 2024-05-08T08:53:12.423 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.423 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.423 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.423 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.423 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.423 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.423 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2430 2024-05-08T08:53:12.423 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.423 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.423 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.423 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.424 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209599+0000", 2024-05-08T08:53:12.424 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.424 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.424 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.424 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.424 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209599+0000", 2024-05-08T08:53:12.424 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.424 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.424 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.424 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.424 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209599+0000", 2024-05-08T08:53:12.424 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.15e-07 2024-05-08T08:53:12.424 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.424 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.425 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.425 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209605+0000", 2024-05-08T08:53:12.425 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4420000000000003e-06 2024-05-08T08:53:12.425 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.425 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.425 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.425 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209611+0000", 2024-05-08T08:53:12.425 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0279999999999997e-06 2024-05-08T08:53:12.425 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.425 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.425 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.425 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209616+0000", 2024-05-08T08:53:12.425 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9649999999999999e-06 2024-05-08T08:53:12.425 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.425 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.426 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.426 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.426 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.426 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:1927 3.1b 3.41ca47db (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.426 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.205815+0000", 2024-05-08T08:53:12.426 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0057800380000000004, 2024-05-08T08:53:12.426 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0093585130000000006, 2024-05-08T08:53:12.426 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.426 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.426 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.426 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.426 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.426 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.426 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1927 2024-05-08T08:53:12.427 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.427 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.427 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.427 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.427 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205815+0000", 2024-05-08T08:53:12.427 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.427 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.427 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.427 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.427 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205815+0000", 2024-05-08T08:53:12.427 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.427 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.427 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.427 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.428 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205815+0000", 2024-05-08T08:53:12.428 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0600000000000001e-07 2024-05-08T08:53:12.428 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.428 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.428 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.428 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205818+0000", 2024-05-08T08:53:12.428 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.5330000000000001e-06 2024-05-08T08:53:12.428 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.428 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.428 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.428 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205825+0000", 2024-05-08T08:53:12.428 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5650000000000004e-06 2024-05-08T08:53:12.428 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.428 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.428 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.428 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205830+0000", 2024-05-08T08:53:12.429 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3259999999999997e-06 2024-05-08T08:53:12.429 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.429 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.429 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.429 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.429 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.429 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2110 3.3 3.629e4dc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.429 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.207687+0000", 2024-05-08T08:53:12.429 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0039075459999999996, 2024-05-08T08:53:12.429 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.007541814, 2024-05-08T08:53:12.429 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.429 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.429 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.429 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.429 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.430 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.430 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2110 2024-05-08T08:53:12.430 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.430 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.430 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.430 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.430 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207687+0000", 2024-05-08T08:53:12.430 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.430 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.430 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.430 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.430 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207687+0000", 2024-05-08T08:53:12.430 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.430 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.431 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.431 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.431 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207688+0000", 2024-05-08T08:53:12.431 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7e-07 2024-05-08T08:53:12.431 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.431 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.431 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.431 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207690+0000", 2024-05-08T08:53:12.431 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3159999999999999e-06 2024-05-08T08:53:12.431 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.431 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.431 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.431 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207696+0000", 2024-05-08T08:53:12.431 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1489999999999999e-06 2024-05-08T08:53:12.432 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.432 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.432 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.432 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207701+0000", 2024-05-08T08:53:12.432 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6770000000000003e-06 2024-05-08T08:53:12.432 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.432 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.432 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.432 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.432 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.432 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2431 3.1b 3.41ca47db (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.432 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.209624+0000", 2024-05-08T08:53:12.432 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0019705830000000001, 2024-05-08T08:53:12.432 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0056603870000000002, 2024-05-08T08:53:12.433 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.433 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.433 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.433 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.433 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.433 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.433 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2431 2024-05-08T08:53:12.433 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.433 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.433 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.433 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.433 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209624+0000", 2024-05-08T08:53:12.433 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.433 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.434 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.434 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.434 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209624+0000", 2024-05-08T08:53:12.434 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.434 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.434 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.434 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.434 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209625+0000", 2024-05-08T08:53:12.434 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2100000000000002e-07 2024-05-08T08:53:12.434 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.434 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.434 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.434 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209627+0000", 2024-05-08T08:53:12.434 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3630000000000001e-06 2024-05-08T08:53:12.435 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.435 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.435 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.435 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209634+0000", 2024-05-08T08:53:12.435 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4389999999999997e-06 2024-05-08T08:53:12.435 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.435 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.435 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.435 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209638+0000", 2024-05-08T08:53:12.435 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8029999999999996e-06 2024-05-08T08:53:12.435 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.435 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.435 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.435 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.435 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.436 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:1928 3.1b 3.41ca47db (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.436 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.205838+0000", 2024-05-08T08:53:12.436 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0057569270000000002, 2024-05-08T08:53:12.436 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0095027989999999993, 2024-05-08T08:53:12.436 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.436 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.436 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.436 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.436 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.436 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.436 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1928 2024-05-08T08:53:12.436 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.436 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.436 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.437 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.437 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205838+0000", 2024-05-08T08:53:12.437 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.437 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.437 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.437 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.437 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205838+0000", 2024-05-08T08:53:12.437 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.437 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.437 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.437 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.437 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205838+0000", 2024-05-08T08:53:12.437 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8799999999999998e-07 2024-05-08T08:53:12.437 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.437 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.438 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.438 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205840+0000", 2024-05-08T08:53:12.438 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9300000000000002e-06 2024-05-08T08:53:12.438 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.438 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.438 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.438 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205846+0000", 2024-05-08T08:53:12.438 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.015e-06 2024-05-08T08:53:12.438 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.438 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.438 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.438 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205850+0000", 2024-05-08T08:53:12.438 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2169999999999996e-06 2024-05-08T08:53:12.438 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.439 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.439 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.439 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.439 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.439 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2111 3.3 3.629e4dc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.439 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.207708+0000", 2024-05-08T08:53:12.439 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0038872289999999999, 2024-05-08T08:53:12.439 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0076884270000000003, 2024-05-08T08:53:12.439 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.439 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.439 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.439 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.439 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.439 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.439 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2111 2024-05-08T08:53:12.439 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.440 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.440 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.440 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.440 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207708+0000", 2024-05-08T08:53:12.440 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.440 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.440 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.440 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.440 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207708+0000", 2024-05-08T08:53:12.440 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.440 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.440 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.440 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.440 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207708+0000", 2024-05-08T08:53:12.441 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5200000000000002e-07 2024-05-08T08:53:12.441 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.441 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.441 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.441 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207710+0000", 2024-05-08T08:53:12.441 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0310000000000001e-06 2024-05-08T08:53:12.441 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.441 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.441 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.441 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207716+0000", 2024-05-08T08:53:12.441 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0800000000000002e-06 2024-05-08T08:53:12.441 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.441 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.441 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.441 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207721+0000", 2024-05-08T08:53:12.441 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7269999999999998e-06 2024-05-08T08:53:12.442 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.442 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.442 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.442 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.442 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.442 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2432 3.1b 3.41ca47db (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.442 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.209645+0000", 2024-05-08T08:53:12.442 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.001949416, 2024-05-08T08:53:12.442 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0058057600000000001, 2024-05-08T08:53:12.442 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.442 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.442 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.442 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.443 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.443 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.443 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2432 2024-05-08T08:53:12.443 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.443 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.443 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.443 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.443 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209645+0000", 2024-05-08T08:53:12.443 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.443 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.443 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.443 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.443 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209645+0000", 2024-05-08T08:53:12.443 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.444 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.444 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.444 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.444 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209646+0000", 2024-05-08T08:53:12.444 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9799999999999999e-07 2024-05-08T08:53:12.444 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.444 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.444 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.444 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209648+0000", 2024-05-08T08:53:12.444 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2069999999999998e-06 2024-05-08T08:53:12.444 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.444 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.444 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.445 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 5] out_osds: [4] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-05-08T08:53:12.445 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T08:53:12.445 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T08:53:12.447 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:12.449+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:12.448 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209655+0000", 2024-05-08T08:53:12.448 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6900000000000003e-06 2024-05-08T08:53:12.448 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.448 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.448 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.448 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209660+0000", 2024-05-08T08:53:12.448 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9250000000000003e-06 2024-05-08T08:53:12.448 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.448 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.449 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.449 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.449 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.449 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:1929 3.1b 3.41ca47db (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.449 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.205857+0000", 2024-05-08T08:53:12.449 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0057379249999999996, 2024-05-08T08:53:12.449 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0097323949999999996, 2024-05-08T08:53:12.449 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.449 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.449 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.449 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.449 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.449 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.449 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1929 2024-05-08T08:53:12.450 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.450 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.450 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.450 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.450 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205857+0000", 2024-05-08T08:53:12.450 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.450 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.450 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.450 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.450 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205857+0000", 2024-05-08T08:53:12.450 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.450 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.450 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.450 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.451 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205857+0000", 2024-05-08T08:53:12.451 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.96e-07 2024-05-08T08:53:12.451 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.451 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.451 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.451 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205859+0000", 2024-05-08T08:53:12.451 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.08e-06 2024-05-08T08:53:12.451 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.451 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.451 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.451 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205867+0000", 2024-05-08T08:53:12.451 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2239999999999998e-06 2024-05-08T08:53:12.451 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.451 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.452 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.452 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205874+0000", 2024-05-08T08:53:12.452 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0690000000000004e-06 2024-05-08T08:53:12.452 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.452 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.452 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.452 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.452 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.452 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2112 3.3 3.629e4dc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.452 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.207727+0000", 2024-05-08T08:53:12.452 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.003867569, 2024-05-08T08:53:12.452 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0079188699999999997, 2024-05-08T08:53:12.452 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.452 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.452 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.453 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.453 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.453 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.453 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2112 2024-05-08T08:53:12.453 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.453 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.453 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.453 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.453 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207727+0000", 2024-05-08T08:53:12.453 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.453 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.453 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.453 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.454 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207727+0000", 2024-05-08T08:53:12.454 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.454 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.454 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.454 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.454 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207728+0000", 2024-05-08T08:53:12.454 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.2899999999999999e-07 2024-05-08T08:53:12.454 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.454 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.454 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.454 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207730+0000", 2024-05-08T08:53:12.454 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9190000000000002e-06 2024-05-08T08:53:12.454 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.455 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.455 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.455 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207735+0000", 2024-05-08T08:53:12.455 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9039999999999997e-06 2024-05-08T08:53:12.455 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.455 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.455 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.455 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207740+0000", 2024-05-08T08:53:12.455 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6e-06 2024-05-08T08:53:12.455 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.455 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.455 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.455 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.455 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.455 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2433 3.1b 3.41ca47db (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.456 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.209667+0000", 2024-05-08T08:53:12.456 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0019279830000000001, 2024-05-08T08:53:12.456 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0060398960000000003, 2024-05-08T08:53:12.456 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.456 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.456 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.456 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.456 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.456 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.456 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2433 2024-05-08T08:53:12.456 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.456 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.456 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.456 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.457 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209667+0000", 2024-05-08T08:53:12.457 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.457 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.457 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.457 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.457 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209667+0000", 2024-05-08T08:53:12.457 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.457 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.457 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.457 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.457 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209667+0000", 2024-05-08T08:53:12.457 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9200000000000002e-07 2024-05-08T08:53:12.457 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.457 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.457 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.458 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209670+0000", 2024-05-08T08:53:12.458 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.43e-06 2024-05-08T08:53:12.458 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.458 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.458 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.458 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209676+0000", 2024-05-08T08:53:12.458 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1929999999999998e-06 2024-05-08T08:53:12.458 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.458 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.458 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.458 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209681+0000", 2024-05-08T08:53:12.458 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1370000000000001e-06 2024-05-08T08:53:12.458 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.458 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.459 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.459 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.459 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.459 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:1930 3.1b 3.41ca47db (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.459 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.205882+0000", 2024-05-08T08:53:12.459 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0057124970000000004, 2024-05-08T08:53:12.459 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0098809099999999997, 2024-05-08T08:53:12.459 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.459 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.459 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.459 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.459 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.459 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.459 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1930 2024-05-08T08:53:12.459 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.460 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.460 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.460 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.460 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205882+0000", 2024-05-08T08:53:12.460 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.460 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.460 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.460 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.460 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205882+0000", 2024-05-08T08:53:12.460 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.460 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.460 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.460 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.461 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205883+0000", 2024-05-08T08:53:12.461 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9999999999999997e-07 2024-05-08T08:53:12.461 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.461 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.461 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.461 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205885+0000", 2024-05-08T08:53:12.461 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3590000000000002e-06 2024-05-08T08:53:12.461 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.461 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.461 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.461 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205893+0000", 2024-05-08T08:53:12.461 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3640000000000001e-06 2024-05-08T08:53:12.461 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.461 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.461 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.462 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.205898+0000", 2024-05-08T08:53:12.462 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2310000000000004e-06 2024-05-08T08:53:12.462 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.462 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.462 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.462 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.462 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.462 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2113 3.3 3.629e4dc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.462 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.207747+0000", 2024-05-08T08:53:12.462 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0038483179999999999, 2024-05-08T08:53:12.462 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0080762739999999996, 2024-05-08T08:53:12.462 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.462 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.463 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.463 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.463 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.463 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.463 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2113 2024-05-08T08:53:12.463 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.463 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.463 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.463 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.463 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207747+0000", 2024-05-08T08:53:12.463 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.463 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.463 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.463 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.464 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207747+0000", 2024-05-08T08:53:12.464 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.464 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.464 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.464 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.464 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207747+0000", 2024-05-08T08:53:12.464 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4799999999999999e-07 2024-05-08T08:53:12.464 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.464 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.464 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.464 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207749+0000", 2024-05-08T08:53:12.464 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4830000000000002e-06 2024-05-08T08:53:12.464 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.464 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.464 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.465 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207756+0000", 2024-05-08T08:53:12.465 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2269999999999998e-06 2024-05-08T08:53:12.465 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.465 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.465 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.465 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207760+0000", 2024-05-08T08:53:12.465 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7190000000000001e-06 2024-05-08T08:53:12.465 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.465 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.465 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.465 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.465 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.465 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2434 3.1b 3.41ca47db (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.466 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.209688+0000", 2024-05-08T08:53:12.466 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0019065320000000001, 2024-05-08T08:53:12.466 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0061908229999999998, 2024-05-08T08:53:12.466 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.466 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.466 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.466 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.466 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.466 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.466 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2434 2024-05-08T08:53:12.466 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.466 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.466 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.466 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.467 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209688+0000", 2024-05-08T08:53:12.467 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.467 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.467 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.467 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.467 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209688+0000", 2024-05-08T08:53:12.467 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.467 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.467 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.467 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.467 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209689+0000", 2024-05-08T08:53:12.467 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.03e-07 2024-05-08T08:53:12.467 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.467 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.467 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.468 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209694+0000", 2024-05-08T08:53:12.468 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5860000000000004e-06 2024-05-08T08:53:12.468 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.468 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.468 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.468 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209701+0000", 2024-05-08T08:53:12.468 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.024e-06 2024-05-08T08:53:12.468 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.468 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.468 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.468 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209706+0000", 2024-05-08T08:53:12.468 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9479999999999999e-06 2024-05-08T08:53:12.468 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.469 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.469 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.469 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.469 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.469 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2114 3.3 3.629e4dc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.469 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.207767+0000", 2024-05-08T08:53:12.469 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0038281249999999999, 2024-05-08T08:53:12.469 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0081690700000000005, 2024-05-08T08:53:12.469 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.469 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.469 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.469 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.469 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.469 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.470 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2114 2024-05-08T08:53:12.470 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.470 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.470 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.470 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.470 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207767+0000", 2024-05-08T08:53:12.470 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.470 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.470 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.470 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.470 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207767+0000", 2024-05-08T08:53:12.470 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.470 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.470 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.471 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.471 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207767+0000", 2024-05-08T08:53:12.471 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.58e-07 2024-05-08T08:53:12.471 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.471 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.471 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.471 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207769+0000", 2024-05-08T08:53:12.471 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0700000000000001e-06 2024-05-08T08:53:12.471 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.471 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.471 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.471 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207775+0000", 2024-05-08T08:53:12.471 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9920000000000004e-06 2024-05-08T08:53:12.471 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.471 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.472 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.472 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207780+0000", 2024-05-08T08:53:12.472 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7890000000000002e-06 2024-05-08T08:53:12.472 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.472 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.472 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.472 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.472 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.472 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2435 3.1b 3.41ca47db (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.472 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.209714+0000", 2024-05-08T08:53:12.472 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0018809020000000001, 2024-05-08T08:53:12.472 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0062946800000000004, 2024-05-08T08:53:12.472 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.472 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.472 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.473 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.473 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.473 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.473 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2435 2024-05-08T08:53:12.473 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.473 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.473 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.473 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.473 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209714+0000", 2024-05-08T08:53:12.473 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.473 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.473 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.473 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.473 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209714+0000", 2024-05-08T08:53:12.473 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.473 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.474 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.474 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.474 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209714+0000", 2024-05-08T08:53:12.474 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4000000000000002e-07 2024-05-08T08:53:12.474 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.474 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.474 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.474 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209717+0000", 2024-05-08T08:53:12.474 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.182e-06 2024-05-08T08:53:12.474 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.474 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.474 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.474 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209724+0000", 2024-05-08T08:53:12.474 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.08e-06 2024-05-08T08:53:12.475 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.475 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.475 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.475 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209729+0000", 2024-05-08T08:53:12.475 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2410000000000003e-06 2024-05-08T08:53:12.475 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.475 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.475 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.475 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.475 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.475 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2115 3.3 3.629e4dc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.475 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.207787+0000", 2024-05-08T08:53:12.475 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0038081759999999999, 2024-05-08T08:53:12.475 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0082784940000000008, 2024-05-08T08:53:12.475 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.476 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.476 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.476 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.476 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.476 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.476 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2115 2024-05-08T08:53:12.476 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.476 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.476 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.476 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.476 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207787+0000", 2024-05-08T08:53:12.476 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.476 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.476 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.477 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.477 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207787+0000", 2024-05-08T08:53:12.477 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.477 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.477 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.477 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.477 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207787+0000", 2024-05-08T08:53:12.477 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4000000000000002e-07 2024-05-08T08:53:12.477 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.477 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.477 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.477 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207789+0000", 2024-05-08T08:53:12.477 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.125e-06 2024-05-08T08:53:12.478 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.478 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.478 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.478 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207802+0000", 2024-05-08T08:53:12.478 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2374999999999999e-05 2024-05-08T08:53:12.478 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.478 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.478 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.478 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.207998+0000", 2024-05-08T08:53:12.478 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00019639599999999999 2024-05-08T08:53:12.478 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.478 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.478 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.478 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.479 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.479 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2436 3.1b 3.41ca47db (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.479 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.209736+0000", 2024-05-08T08:53:12.479 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.001858852, 2024-05-08T08:53:12.479 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0063899359999999997, 2024-05-08T08:53:12.479 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.479 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.479 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.479 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.479 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.479 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.479 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2436 2024-05-08T08:53:12.479 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.479 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.479 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.480 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.480 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209736+0000", 2024-05-08T08:53:12.480 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.480 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.480 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.480 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.480 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209736+0000", 2024-05-08T08:53:12.480 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.480 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.480 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.480 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.480 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209736+0000", 2024-05-08T08:53:12.480 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8099999999999998e-07 2024-05-08T08:53:12.480 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.481 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.481 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.481 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209739+0000", 2024-05-08T08:53:12.481 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2800000000000002e-06 2024-05-08T08:53:12.481 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.481 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.481 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.481 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209745+0000", 2024-05-08T08:53:12.481 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8540000000000002e-06 2024-05-08T08:53:12.481 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.481 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.481 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.481 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209749+0000", 2024-05-08T08:53:12.481 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9529999999999998e-06 2024-05-08T08:53:12.482 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.482 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.482 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.482 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.482 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.482 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2116 3.3 3.629e4dc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.482 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.208020+0000", 2024-05-08T08:53:12.482 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.003574991, 2024-05-08T08:53:12.482 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0081613250000000005, 2024-05-08T08:53:12.482 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.482 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.482 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.482 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.483 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.483 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.483 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2116 2024-05-08T08:53:12.483 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.483 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.483 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.483 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.483 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208020+0000", 2024-05-08T08:53:12.483 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.483 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.483 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.483 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.483 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208020+0000", 2024-05-08T08:53:12.483 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.483 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.484 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.484 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.484 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208020+0000", 2024-05-08T08:53:12.484 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0200000000000002e-07 2024-05-08T08:53:12.484 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.484 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.484 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.484 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208024+0000", 2024-05-08T08:53:12.484 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.275e-06 2024-05-08T08:53:12.484 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.484 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.484 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.484 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208031+0000", 2024-05-08T08:53:12.484 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1740000000000003e-06 2024-05-08T08:53:12.485 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.485 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.485 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.485 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208037+0000", 2024-05-08T08:53:12.485 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7100000000000004e-06 2024-05-08T08:53:12.485 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.485 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.485 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.485 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.485 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.485 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2437 3.1b 3.41ca47db (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.485 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.209757+0000", 2024-05-08T08:53:12.485 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0018382680000000001, 2024-05-08T08:53:12.485 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0064865840000000001, 2024-05-08T08:53:12.485 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.486 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.486 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.486 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.486 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.486 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.486 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2437 2024-05-08T08:53:12.486 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.486 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.486 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.486 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.486 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209757+0000", 2024-05-08T08:53:12.486 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.486 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.486 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.487 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.487 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209757+0000", 2024-05-08T08:53:12.487 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.487 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.487 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.487 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.487 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209757+0000", 2024-05-08T08:53:12.487 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9400000000000001e-07 2024-05-08T08:53:12.487 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.487 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.487 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.487 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209766+0000", 2024-05-08T08:53:12.487 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0100000000000001e-06 2024-05-08T08:53:12.487 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.488 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.488 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.488 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209774+0000", 2024-05-08T08:53:12.488 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7510000000000008e-06 2024-05-08T08:53:12.488 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.488 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.488 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.488 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209779+0000", 2024-05-08T08:53:12.488 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.933e-06 2024-05-08T08:53:12.488 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.488 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.488 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.488 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.489 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.489 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2117 3.3 3.629e4dc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.489 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.208044+0000", 2024-05-08T08:53:12.489 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.003550554, 2024-05-08T08:53:12.489 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0082545970000000007, 2024-05-08T08:53:12.489 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.489 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.489 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.489 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.489 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.489 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.489 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2117 2024-05-08T08:53:12.489 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.489 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.489 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.490 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.490 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208044+0000", 2024-05-08T08:53:12.490 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.490 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.490 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.490 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.490 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208044+0000", 2024-05-08T08:53:12.490 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.490 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.490 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.490 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.490 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208045+0000", 2024-05-08T08:53:12.491 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.15e-07 2024-05-08T08:53:12.491 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.491 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.491 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.491 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208047+0000", 2024-05-08T08:53:12.491 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3149999999999999e-06 2024-05-08T08:53:12.491 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.491 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.491 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.491 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208054+0000", 2024-05-08T08:53:12.491 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5130000000000002e-06 2024-05-08T08:53:12.491 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.491 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.491 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.491 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208059+0000", 2024-05-08T08:53:12.492 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2989999999999996e-06 2024-05-08T08:53:12.492 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.492 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.492 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.492 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.492 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.492 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2438 3.1b 3.41ca47db (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.492 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.209786+0000", 2024-05-08T08:53:12.492 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0018086840000000001, 2024-05-08T08:53:12.492 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0065714800000000002, 2024-05-08T08:53:12.492 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.492 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.492 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.492 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.492 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.493 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.493 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2438 2024-05-08T08:53:12.493 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.493 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.493 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.493 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.493 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209786+0000", 2024-05-08T08:53:12.493 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.493 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.493 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.493 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.493 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209786+0000", 2024-05-08T08:53:12.493 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.493 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.493 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.493 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.494 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209787+0000", 2024-05-08T08:53:12.494 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1199999999999998e-07 2024-05-08T08:53:12.494 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.494 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.494 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.494 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209789+0000", 2024-05-08T08:53:12.494 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1519999999999999e-06 2024-05-08T08:53:12.494 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.494 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.494 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.494 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209794+0000", 2024-05-08T08:53:12.494 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1460000000000004e-06 2024-05-08T08:53:12.494 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.495 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.495 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.495 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209798+0000", 2024-05-08T08:53:12.495 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4399999999999998e-06 2024-05-08T08:53:12.495 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.495 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.495 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.495 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.495 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.495 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2118 3.3 3.629e4dc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.495 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.208066+0000", 2024-05-08T08:53:12.495 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0035293550000000001, 2024-05-08T08:53:12.495 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0083471389999999999, 2024-05-08T08:53:12.495 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.496 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.496 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.496 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.496 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.496 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.496 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2118 2024-05-08T08:53:12.496 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.496 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.496 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.496 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.496 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208066+0000", 2024-05-08T08:53:12.496 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.496 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.496 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.496 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.497 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208066+0000", 2024-05-08T08:53:12.497 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.497 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.497 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.497 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.497 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208066+0000", 2024-05-08T08:53:12.497 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4200000000000001e-07 2024-05-08T08:53:12.497 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.497 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.497 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.497 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208068+0000", 2024-05-08T08:53:12.497 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.215e-06 2024-05-08T08:53:12.497 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.497 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.497 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.498 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208074+0000", 2024-05-08T08:53:12.498 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1789999999999996e-06 2024-05-08T08:53:12.498 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.498 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.498 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.498 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208079+0000", 2024-05-08T08:53:12.498 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4660000000000001e-06 2024-05-08T08:53:12.498 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.498 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.498 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.498 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.498 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.498 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2439 3.1b 3.41ca47db (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.499 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.209803+0000", 2024-05-08T08:53:12.499 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0017914610000000001, 2024-05-08T08:53:12.499 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0066698119999999998, 2024-05-08T08:53:12.499 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.499 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.499 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.499 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.499 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.499 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.499 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2439 2024-05-08T08:53:12.499 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.499 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.499 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.499 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.499 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209803+0000", 2024-05-08T08:53:12.499 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.500 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.500 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.500 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.500 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209803+0000", 2024-05-08T08:53:12.500 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.500 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.500 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.500 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.500 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209804+0000", 2024-05-08T08:53:12.500 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1e-07 2024-05-08T08:53:12.500 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.500 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.500 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.500 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209805+0000", 2024-05-08T08:53:12.501 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.559e-06 2024-05-08T08:53:12.501 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.501 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.501 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.501 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209810+0000", 2024-05-08T08:53:12.501 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8770000000000001e-06 2024-05-08T08:53:12.501 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.501 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.501 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.501 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209814+0000", 2024-05-08T08:53:12.501 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0860000000000004e-06 2024-05-08T08:53:12.501 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.501 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.501 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.501 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.502 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.502 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2172 3.1b 3.41ca47db (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.502 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.208085+0000", 2024-05-08T08:53:12.502 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0035095809999999999, 2024-05-08T08:53:12.502 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0084616120000000003, 2024-05-08T08:53:12.502 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.502 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.502 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.502 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.502 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.502 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.502 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2172 2024-05-08T08:53:12.502 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.502 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.503 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.503 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.503 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208085+0000", 2024-05-08T08:53:12.503 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.503 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.503 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.503 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.503 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208085+0000", 2024-05-08T08:53:12.503 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.503 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.503 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.503 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.503 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208086+0000", 2024-05-08T08:53:12.504 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2e-07 2024-05-08T08:53:12.504 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.504 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.504 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.504 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208088+0000", 2024-05-08T08:53:12.504 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9719999999999999e-06 2024-05-08T08:53:12.504 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.504 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.504 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.504 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208094+0000", 2024-05-08T08:53:12.504 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1639999999999998e-06 2024-05-08T08:53:12.504 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.504 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.505 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.505 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208099+0000", 2024-05-08T08:53:12.505 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8339999999999998e-06 2024-05-08T08:53:12.505 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.505 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.505 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.505 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.505 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.505 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2440 3.1b 3.41ca47db (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.505 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.209820+0000", 2024-05-08T08:53:12.505 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0017751209999999999, 2024-05-08T08:53:12.505 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0067891389999999996, 2024-05-08T08:53:12.505 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.505 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.506 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.506 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.506 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.506 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.506 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2440 2024-05-08T08:53:12.506 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.506 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.506 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.506 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.506 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209820+0000", 2024-05-08T08:53:12.506 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.506 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.507 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.507 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.507 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209820+0000", 2024-05-08T08:53:12.507 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.507 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.507 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.507 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.507 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209820+0000", 2024-05-08T08:53:12.507 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4700000000000002e-07 2024-05-08T08:53:12.507 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.507 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.507 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.507 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209822+0000", 2024-05-08T08:53:12.507 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.934e-06 2024-05-08T08:53:12.508 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.508 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.508 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.508 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209828+0000", 2024-05-08T08:53:12.508 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5659999999999997e-06 2024-05-08T08:53:12.508 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.508 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.508 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.508 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209832+0000", 2024-05-08T08:53:12.508 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4610000000000001e-06 2024-05-08T08:53:12.508 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.508 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.508 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.508 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.508 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.509 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2173 3.1b 3.41ca47db (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.509 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.208106+0000", 2024-05-08T08:53:12.509 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0034893110000000001, 2024-05-08T08:53:12.509 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0085588599999999997, 2024-05-08T08:53:12.509 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.509 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.509 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.509 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.509 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.509 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.509 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2173 2024-05-08T08:53:12.509 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.509 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.509 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.510 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.510 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208106+0000", 2024-05-08T08:53:12.510 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.510 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.510 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.510 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.510 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208106+0000", 2024-05-08T08:53:12.510 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.510 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.510 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.510 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.510 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208106+0000", 2024-05-08T08:53:12.510 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.72e-07 2024-05-08T08:53:12.511 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.511 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.511 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.511 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208113+0000", 2024-05-08T08:53:12.511 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5299999999999999e-06 2024-05-08T08:53:12.511 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.511 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.511 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.511 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208121+0000", 2024-05-08T08:53:12.511 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3780000000000003e-06 2024-05-08T08:53:12.511 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.511 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.511 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.511 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208127+0000", 2024-05-08T08:53:12.512 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6699999999999999e-06 2024-05-08T08:53:12.512 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.512 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.512 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.512 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.512 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.512 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2441 3.1b 3.41ca47db (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.512 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.209838+0000", 2024-05-08T08:53:12.512 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.001756482, 2024-05-08T08:53:12.512 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0068855820000000003, 2024-05-08T08:53:12.512 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.512 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.512 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.512 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.512 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.513 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.513 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2441 2024-05-08T08:53:12.513 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.513 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.513 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.513 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.513 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209838+0000", 2024-05-08T08:53:12.513 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.513 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.513 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.513 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.513 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209838+0000", 2024-05-08T08:53:12.513 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.513 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.513 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.514 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.514 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209839+0000", 2024-05-08T08:53:12.514 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8200000000000001e-07 2024-05-08T08:53:12.514 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.514 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.514 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.514 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209841+0000", 2024-05-08T08:53:12.514 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1689999999999999e-06 2024-05-08T08:53:12.514 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.514 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.514 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.514 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209847+0000", 2024-05-08T08:53:12.514 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7699999999999998e-06 2024-05-08T08:53:12.514 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.515 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.515 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.515 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209851+0000", 2024-05-08T08:53:12.515 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9820000000000002e-06 2024-05-08T08:53:12.515 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.515 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.515 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.515 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.515 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.515 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2174 3.1b 3.41ca47db (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.515 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.208134+0000", 2024-05-08T08:53:12.515 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0034611939999999999, 2024-05-08T08:53:12.515 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0086475159999999992, 2024-05-08T08:53:12.515 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.515 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.516 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.516 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.516 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.516 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.516 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2174 2024-05-08T08:53:12.516 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.516 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.516 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.516 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.516 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208134+0000", 2024-05-08T08:53:12.516 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.516 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.516 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.516 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.516 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208134+0000", 2024-05-08T08:53:12.517 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.517 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.517 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.517 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.517 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208134+0000", 2024-05-08T08:53:12.517 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5299999999999999e-07 2024-05-08T08:53:12.517 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.517 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.517 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.517 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208136+0000", 2024-05-08T08:53:12.517 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.159e-06 2024-05-08T08:53:12.517 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.517 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.517 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.518 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208143+0000", 2024-05-08T08:53:12.518 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2389999999999999e-06 2024-05-08T08:53:12.518 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.518 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.518 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.518 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208147+0000", 2024-05-08T08:53:12.518 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5040000000000004e-06 2024-05-08T08:53:12.518 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.518 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.518 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.518 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.518 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.518 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2357 3.3 3.629e4dc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.518 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.209202+0000", 2024-05-08T08:53:12.519 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0023926149999999998, 2024-05-08T08:53:12.519 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0076427439999999999, 2024-05-08T08:53:12.519 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.519 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.519 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.519 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.519 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.519 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.519 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2357 2024-05-08T08:53:12.519 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.519 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.519 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.519 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.520 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209202+0000", 2024-05-08T08:53:12.520 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.520 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.520 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.520 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.520 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209202+0000", 2024-05-08T08:53:12.520 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.520 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.520 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.520 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.520 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209203+0000", 2024-05-08T08:53:12.520 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.53e-07 2024-05-08T08:53:12.520 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.520 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.520 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.521 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209204+0000", 2024-05-08T08:53:12.521 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.8679999999999999e-06 2024-05-08T08:53:12.521 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.521 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.521 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.521 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209210+0000", 2024-05-08T08:53:12.521 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4410000000000001e-06 2024-05-08T08:53:12.521 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.521 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.521 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.521 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209214+0000", 2024-05-08T08:53:12.521 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.515e-06 2024-05-08T08:53:12.521 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.521 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.521 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.522 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.522 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.522 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2442 3.1b 3.41ca47db (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.522 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.209857+0000", 2024-05-08T08:53:12.522 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.001738149, 2024-05-08T08:53:12.522 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0070452830000000003, 2024-05-08T08:53:12.522 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.522 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.522 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.522 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.522 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.522 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.522 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2442 2024-05-08T08:53:12.523 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.523 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.523 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.523 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.523 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209857+0000", 2024-05-08T08:53:12.523 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.523 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.523 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.523 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.523 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209857+0000", 2024-05-08T08:53:12.523 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.523 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.523 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.523 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.524 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209857+0000", 2024-05-08T08:53:12.524 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9299999999999999e-07 2024-05-08T08:53:12.524 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.524 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.524 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.524 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209859+0000", 2024-05-08T08:53:12.524 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0700000000000001e-06 2024-05-08T08:53:12.524 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.524 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.524 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.524 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209865+0000", 2024-05-08T08:53:12.524 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5369999999999997e-06 2024-05-08T08:53:12.524 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.525 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.525 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.525 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209869+0000", 2024-05-08T08:53:12.525 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5990000000000004e-06 2024-05-08T08:53:12.525 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.525 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.525 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.525 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.525 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.525 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2175 3.1b 3.41ca47db (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.525 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.208153+0000", 2024-05-08T08:53:12.525 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0034414609999999998, 2024-05-08T08:53:12.525 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0088114749999999992, 2024-05-08T08:53:12.525 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.525 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.526 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.526 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.526 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.526 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.526 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2175 2024-05-08T08:53:12.526 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.526 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.526 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.526 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.526 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208153+0000", 2024-05-08T08:53:12.526 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.526 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.526 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.526 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.527 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208153+0000", 2024-05-08T08:53:12.527 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.527 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.527 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.527 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.527 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208154+0000", 2024-05-08T08:53:12.527 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9799999999999999e-07 2024-05-08T08:53:12.527 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.527 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.527 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.527 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208156+0000", 2024-05-08T08:53:12.527 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.013e-06 2024-05-08T08:53:12.527 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.527 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.528 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.528 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208162+0000", 2024-05-08T08:53:12.528 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1460000000000001e-06 2024-05-08T08:53:12.528 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.528 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.528 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.528 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208167+0000", 2024-05-08T08:53:12.528 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5889999999999996e-06 2024-05-08T08:53:12.528 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.528 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.528 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.528 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.528 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.528 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2358 3.3 3.629e4dc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.528 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.209220+0000", 2024-05-08T08:53:12.529 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0023745369999999999, 2024-05-08T08:53:12.529 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0078052590000000002, 2024-05-08T08:53:12.529 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.529 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.529 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.529 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.529 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.529 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.529 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2358 2024-05-08T08:53:12.529 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.529 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.529 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.529 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.529 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209220+0000", 2024-05-08T08:53:12.530 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.530 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.530 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.530 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.530 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209220+0000", 2024-05-08T08:53:12.530 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.530 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.530 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.530 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.530 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209221+0000", 2024-05-08T08:53:12.530 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3000000000000002e-07 2024-05-08T08:53:12.530 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.530 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.530 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.531 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209222+0000", 2024-05-08T08:53:12.531 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.8020000000000001e-06 2024-05-08T08:53:12.531 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.531 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.531 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.531 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209228+0000", 2024-05-08T08:53:12.531 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4689999999999996e-06 2024-05-08T08:53:12.531 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.531 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.531 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.531 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209232+0000", 2024-05-08T08:53:12.531 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5179999999999998e-06 2024-05-08T08:53:12.531 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.531 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.531 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.532 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.532 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.532 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2443 3.1b 3.41ca47db (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.532 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.209875+0000", 2024-05-08T08:53:12.532 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.001719372, 2024-05-08T08:53:12.532 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0072063190000000001, 2024-05-08T08:53:12.532 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.532 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.532 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.532 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.532 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.532 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.532 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2443 2024-05-08T08:53:12.532 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.532 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.533 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.533 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.533 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209875+0000", 2024-05-08T08:53:12.533 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.533 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.533 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.533 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.533 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209875+0000", 2024-05-08T08:53:12.533 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.533 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.533 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.533 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.533 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209876+0000", 2024-05-08T08:53:12.533 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.96e-07 2024-05-08T08:53:12.533 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.534 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.534 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.534 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209878+0000", 2024-05-08T08:53:12.534 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2450000000000001e-06 2024-05-08T08:53:12.534 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.534 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.534 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.534 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209884+0000", 2024-05-08T08:53:12.534 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0599999999999996e-06 2024-05-08T08:53:12.534 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.534 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.534 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.534 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209889+0000", 2024-05-08T08:53:12.535 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7979999999999996e-06 2024-05-08T08:53:12.535 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.535 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.535 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.535 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.535 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.535 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2176 3.1b 3.41ca47db (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.535 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.208362+0000", 2024-05-08T08:53:12.535 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0032329849999999999, 2024-05-08T08:53:12.535 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0087800380000000004, 2024-05-08T08:53:12.535 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.535 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.535 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.535 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.535 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.536 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.536 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2176 2024-05-08T08:53:12.536 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.536 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.536 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.536 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.536 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208362+0000", 2024-05-08T08:53:12.536 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.536 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.536 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.536 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.536 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208362+0000", 2024-05-08T08:53:12.536 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.536 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.536 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.536 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.537 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208362+0000", 2024-05-08T08:53:12.537 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7100000000000002e-07 2024-05-08T08:53:12.537 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.537 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.537 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.537 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208377+0000", 2024-05-08T08:53:12.537 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.4112e-05 2024-05-08T08:53:12.537 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.537 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.537 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.537 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208383+0000", 2024-05-08T08:53:12.537 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8589999999999999e-06 2024-05-08T08:53:12.537 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.537 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.537 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.537 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.208389+0000", 2024-05-08T08:53:12.538 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8289999999999996e-06 2024-05-08T08:53:12.538 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.538 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.538 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.538 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.538 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.538 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2359 3.3 3.629e4dc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.538 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.209238+0000", 2024-05-08T08:53:12.538 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0023568920000000002, 2024-05-08T08:53:12.538 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0079606650000000004, 2024-05-08T08:53:12.538 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.538 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.538 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.539 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.539 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.539 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.539 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2359 2024-05-08T08:53:12.539 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.539 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.539 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.539 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.539 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209238+0000", 2024-05-08T08:53:12.539 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.539 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.539 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.539 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.539 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209238+0000", 2024-05-08T08:53:12.539 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.539 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.540 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.540 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.540 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209238+0000", 2024-05-08T08:53:12.540 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6399999999999998e-07 2024-05-08T08:53:12.540 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.540 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.540 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.540 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209240+0000", 2024-05-08T08:53:12.540 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.8059999999999999e-06 2024-05-08T08:53:12.540 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.540 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.540 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.540 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209246+0000", 2024-05-08T08:53:12.540 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5400000000000003e-06 2024-05-08T08:53:12.541 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.541 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.541 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.541 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209250+0000", 2024-05-08T08:53:12.541 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5240000000000002e-06 2024-05-08T08:53:12.541 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.541 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.541 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.541 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.541 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.541 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:2444 3.1b 3.41ca47db (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:12.541 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:12.209895+0000", 2024-05-08T08:53:12.541 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0016995319999999999, 2024-05-08T08:53:12.541 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0073620279999999996, 2024-05-08T08:53:12.541 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.541 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.542 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:12.542 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.542 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:12.542 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:12.542 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2444 2024-05-08T08:53:12.542 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.542 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.542 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.542 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.542 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209895+0000", 2024-05-08T08:53:12.542 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.542 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.542 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.542 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.543 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209895+0000", 2024-05-08T08:53:12.543 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.543 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.543 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.543 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.543 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209896+0000", 2024-05-08T08:53:12.543 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9799999999999999e-07 2024-05-08T08:53:12.543 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.543 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.543 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.543 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209898+0000", 2024-05-08T08:53:12.543 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1610000000000001e-06 2024-05-08T08:53:12.543 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.543 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.543 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.543 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209904+0000", 2024-05-08T08:53:12.544 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2720000000000003e-06 2024-05-08T08:53:12.544 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.544 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.544 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.544 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:12.209909+0000", 2024-05-08T08:53:12.544 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6800000000000001e-06 2024-05-08T08:53:12.544 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.544 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:12.544 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.544 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:12.544 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T08:53:12.544 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 97 2024-05-08T08:53:12.544 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:12.545 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:53:12.546 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:53:12.546 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:53:12.546 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:53:12.547 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:53:12.547 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:53:12.547 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:53:12.548 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:53:12.549 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:53:12.550 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:12.551+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:12.650 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:12.653+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:12.750 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:12.754+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:12.788 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:12.788 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:53:12.788 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:53:12.788 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:53:12.788 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:12.802 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:53:12.852 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:12.855+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:12.953 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:12.956+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:12.993 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:12.994 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:53:12.994 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:53:12.994 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:53:12.994 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.994 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4621.0:9089 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 23339520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:53:12.994 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:53:12.994 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 40.500880100000003, 2024-05-08T08:53:12.994 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.13814907200000001, 2024-05-08T08:53:12.994 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:12.994 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:12.994 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:53:12.995 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:12.995 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4621", 2024-05-08T08:53:12.995 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:12.995 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9089 2024-05-08T08:53:12.995 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.995 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:12.995 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.995 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:12.995 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:53:12.995 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.995 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.995 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.995 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:12.995 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:53:12.995 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:12.996 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.996 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.996 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:12.996 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493005+0000", 2024-05-08T08:53:12.996 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1700000000000001e-06 2024-05-08T08:53:12.996 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.996 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.996 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:12.996 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.513963+0000", 2024-05-08T08:53:12.996 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.020957581999999999 2024-05-08T08:53:12.996 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.996 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.996 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:12.996 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.513995+0000", 2024-05-08T08:53:12.996 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1993999999999999e-05 2024-05-08T08:53:12.997 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.997 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.997 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:12.997 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514007+0000", 2024-05-08T08:53:12.997 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1581e-05 2024-05-08T08:53:12.997 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.997 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.997 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:12.997 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514122+0000", 2024-05-08T08:53:12.997 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000115282 2024-05-08T08:53:12.997 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.997 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.997 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op must wait for map", 2024-05-08T08:53:12.997 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514128+0000", 2024-05-08T08:53:12.998 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9950000000000002e-06 2024-05-08T08:53:12.998 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.998 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.998 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:12.998 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.515800+0000", 2024-05-08T08:53:12.998 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001671886 2024-05-08T08:53:12.998 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.998 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.998 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:12.998 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.515893+0000", 2024-05-08T08:53:12.998 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3490000000000001e-05 2024-05-08T08:53:12.998 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.998 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.998 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 3", 2024-05-08T08:53:12.998 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.517647+0000", 2024-05-08T08:53:12.999 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0017535630000000001 2024-05-08T08:53:12.999 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:12.999 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:12.999 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:53:12.999 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.627578+0000", 2024-05-08T08:53:12.999 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.109931557 2024-05-08T08:53:12.999 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.000 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.000 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:53:13.000 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631055+0000", 2024-05-08T08:53:13.000 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0034761420000000002 2024-05-08T08:53:13.000 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.000 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.000 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:53:13.000 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631096+0000", 2024-05-08T08:53:13.000 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1059000000000001e-05 2024-05-08T08:53:13.000 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.000 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.000 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:13.000 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631151+0000", 2024-05-08T08:53:13.000 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5770999999999998e-05 2024-05-08T08:53:13.000 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.001 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.001 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.001 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.001 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.001 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1065 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 10944512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:13.001 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485411+0000", 2024-05-08T08:53:13.001 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 36.508471413999999, 2024-05-08T08:53:13.001 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070281386000000001, 2024-05-08T08:53:13.001 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.001 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.001 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:13.001 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.001 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:13.001 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:13.001 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1065 2024-05-08T08:53:13.002 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.002 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.002 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.002 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.002 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485411+0000", 2024-05-08T08:53:13.002 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.002 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.002 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.002 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.002 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485411+0000", 2024-05-08T08:53:13.002 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.002 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.002 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.002 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.003 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485411+0000", 2024-05-08T08:53:13.003 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5600000000000001e-07 2024-05-08T08:53:13.003 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.003 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.003 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.003 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485414+0000", 2024-05-08T08:53:13.003 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2299999999999998e-06 2024-05-08T08:53:13.003 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.003 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.003 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.003 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485420+0000", 2024-05-08T08:53:13.003 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2179999999999996e-06 2024-05-08T08:53:13.003 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.003 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.003 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.003 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485425+0000", 2024-05-08T08:53:13.004 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9830000000000004e-06 2024-05-08T08:53:13.004 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.004 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.004 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:13.004 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555468+0000", 2024-05-08T08:53:13.004 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070043541000000001 2024-05-08T08:53:13.004 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.004 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.004 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:13.004 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555566+0000", 2024-05-08T08:53:13.004 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8082e-05 2024-05-08T08:53:13.004 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.004 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.004 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:13.005 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555692+0000", 2024-05-08T08:53:13.005 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000125976 2024-05-08T08:53:13.005 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.005 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.005 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.005 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.005 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.005 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1067 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 10985472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:13.005 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485431+0000", 2024-05-08T08:53:13.005 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 36.508451278999999, 2024-05-08T08:53:13.005 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070412460999999996, 2024-05-08T08:53:13.005 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.005 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.005 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:13.005 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.006 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:13.006 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:13.006 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1067 2024-05-08T08:53:13.006 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.006 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.006 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.006 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.006 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485431+0000", 2024-05-08T08:53:13.006 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.006 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.006 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.006 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.006 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485431+0000", 2024-05-08T08:53:13.006 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.007 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.007 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.007 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.007 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485432+0000", 2024-05-08T08:53:13.007 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8299999999999998e-07 2024-05-08T08:53:13.007 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.007 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.007 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.007 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485434+0000", 2024-05-08T08:53:13.007 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0849999999999999e-06 2024-05-08T08:53:13.007 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.007 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.007 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.007 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485440+0000", 2024-05-08T08:53:13.007 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0020000000000003e-06 2024-05-08T08:53:13.008 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.008 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.008 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.008 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485445+0000", 2024-05-08T08:53:13.008 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5180000000000004e-06 2024-05-08T08:53:13.008 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.008 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.008 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:13.008 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555731+0000", 2024-05-08T08:53:13.008 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070286214 2024-05-08T08:53:13.008 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.008 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.008 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:13.008 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555787+0000", 2024-05-08T08:53:13.008 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5992999999999997e-05 2024-05-08T08:53:13.009 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.009 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.009 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:13.009 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555844+0000", 2024-05-08T08:53:13.009 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6266e-05 2024-05-08T08:53:13.009 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.009 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.009 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.009 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.009 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.009 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1071 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11026432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:13.009 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:53:13.009 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 36.508430609000001, 2024-05-08T08:53:13.009 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070580037999999998, 2024-05-08T08:53:13.009 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.010 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.010 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:13.010 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.010 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:13.010 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:13.010 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1071 2024-05-08T08:53:13.010 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.010 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.010 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.010 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.010 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:53:13.010 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.010 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.010 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.011 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.011 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:53:13.011 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.011 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.011 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.011 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.011 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:53:13.011 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2800000000000002e-07 2024-05-08T08:53:13.011 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.011 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.011 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.011 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485454+0000", 2024-05-08T08:53:13.011 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2290000000000002e-06 2024-05-08T08:53:13.011 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.011 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.012 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.012 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485461+0000", 2024-05-08T08:53:13.012 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9920000000000001e-06 2024-05-08T08:53:13.012 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.012 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.012 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.012 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485467+0000", 2024-05-08T08:53:13.012 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3979999999999998e-06 2024-05-08T08:53:13.012 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.012 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.012 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:13.012 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555880+0000", 2024-05-08T08:53:13.012 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070413165 2024-05-08T08:53:13.012 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.013 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.013 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:13.013 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555926+0000", 2024-05-08T08:53:13.013 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6217000000000002e-05 2024-05-08T08:53:13.013 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.013 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.013 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:13.013 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556032+0000", 2024-05-08T08:53:13.013 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000105609 2024-05-08T08:53:13.013 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.013 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.013 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.013 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.013 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.013 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1073 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11067392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:13.013 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:53:13.014 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 36.508409387999997, 2024-05-08T08:53:13.014 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070821404000000004, 2024-05-08T08:53:13.014 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.014 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.014 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:13.014 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.014 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:13.014 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:13.014 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1073 2024-05-08T08:53:13.014 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.014 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.014 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.014 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.015 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:53:13.015 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.015 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.015 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.015 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.015 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:53:13.015 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.015 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.015 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.015 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.015 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:53:13.015 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.08e-07 2024-05-08T08:53:13.015 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.015 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.015 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.015 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485476+0000", 2024-05-08T08:53:13.015 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3929999999999998e-06 2024-05-08T08:53:13.016 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.016 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.016 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.016 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485485+0000", 2024-05-08T08:53:13.016 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1880000000000008e-06 2024-05-08T08:53:13.016 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.016 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.016 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.016 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485494+0000", 2024-05-08T08:53:13.016 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3109999999999995e-06 2024-05-08T08:53:13.016 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.016 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.016 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:13.016 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556082+0000", 2024-05-08T08:53:13.016 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070587426999999994 2024-05-08T08:53:13.016 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.016 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.017 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:13.017 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556177+0000", 2024-05-08T08:53:13.017 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5051999999999997e-05 2024-05-08T08:53:13.017 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.017 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.017 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:13.017 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556294+0000", 2024-05-08T08:53:13.017 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000117625 2024-05-08T08:53:13.017 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.017 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.017 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.017 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.017 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.017 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1075 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11108352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:13.018 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:53:13.018 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 36.508378811, 2024-05-08T08:53:13.018 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071045026999999997, 2024-05-08T08:53:13.018 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.018 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.018 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:13.018 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.018 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:13.018 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:13.018 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1075 2024-05-08T08:53:13.018 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.018 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.018 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.018 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.018 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:53:13.019 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.019 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.019 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.019 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.019 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:53:13.019 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.019 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.019 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.019 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.019 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:53:13.019 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.32e-07 2024-05-08T08:53:13.019 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.019 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.019 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.020 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485518+0000", 2024-05-08T08:53:13.020 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3769e-05 2024-05-08T08:53:13.020 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.020 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.020 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.020 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485527+0000", 2024-05-08T08:53:13.020 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6779999999999995e-06 2024-05-08T08:53:13.020 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.020 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.020 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.020 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485532+0000", 2024-05-08T08:53:13.020 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9830000000000004e-06 2024-05-08T08:53:13.020 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.020 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.021 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:13.021 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556340+0000", 2024-05-08T08:53:13.021 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070807743000000006 2024-05-08T08:53:13.021 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.021 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.021 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:13.021 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556435+0000", 2024-05-08T08:53:13.021 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4732e-05 2024-05-08T08:53:13.021 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.021 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.021 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:13.021 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556549+0000", 2024-05-08T08:53:13.021 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011369 2024-05-08T08:53:13.021 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.021 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.022 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.022 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.022 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.022 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1078 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11149312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:13.022 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:53:13.022 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 36.508338612000003, 2024-05-08T08:53:13.022 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071167276000000002, 2024-05-08T08:53:13.022 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.022 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.022 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:13.022 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.022 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:13.022 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:13.023 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1078 2024-05-08T08:53:13.023 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.023 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.023 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.023 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.023 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:53:13.023 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.023 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.023 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.023 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.023 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:53:13.023 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.023 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.023 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.023 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.024 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:53:13.024 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5600000000000001e-07 2024-05-08T08:53:13.024 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.024 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.024 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.024 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485547+0000", 2024-05-08T08:53:13.024 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3590000000000002e-06 2024-05-08T08:53:13.024 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.024 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.024 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.024 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485553+0000", 2024-05-08T08:53:13.024 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2770000000000002e-06 2024-05-08T08:53:13.024 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.024 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.025 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.025 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485558+0000", 2024-05-08T08:53:13.025 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.95e-06 2024-05-08T08:53:13.025 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.025 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.025 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:13.025 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556582+0000", 2024-05-08T08:53:13.025 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071024444000000006 2024-05-08T08:53:13.025 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.025 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.025 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:13.025 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556637+0000", 2024-05-08T08:53:13.025 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4672000000000001e-05 2024-05-08T08:53:13.026 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.026 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.026 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:13.026 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556711+0000", 2024-05-08T08:53:13.026 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4117999999999997e-05 2024-05-08T08:53:13.026 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.026 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.026 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.026 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.026 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.026 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1080 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11190272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:13.027 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:53:13.027 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 36.508317879000003, 2024-05-08T08:53:13.027 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071350721000000006, 2024-05-08T08:53:13.027 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.027 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.027 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:13.027 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.027 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:13.027 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:13.027 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1080 2024-05-08T08:53:13.027 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.027 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.027 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.027 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.028 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:53:13.028 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.028 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.028 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.028 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.028 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:53:13.028 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.028 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.028 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.028 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.028 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:53:13.028 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.03e-07 2024-05-08T08:53:13.028 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.029 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.029 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.029 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485567+0000", 2024-05-08T08:53:13.029 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1090000000000001e-06 2024-05-08T08:53:13.029 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.029 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.029 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.029 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485573+0000", 2024-05-08T08:53:13.029 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3049999999999998e-06 2024-05-08T08:53:13.029 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.029 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.029 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.029 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485578+0000", 2024-05-08T08:53:13.029 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0130000000000001e-06 2024-05-08T08:53:13.030 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.030 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.030 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:13.030 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556743+0000", 2024-05-08T08:53:13.030 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071164449000000005 2024-05-08T08:53:13.030 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.030 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.030 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:13.030 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556797+0000", 2024-05-08T08:53:13.030 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4589999999999997e-05 2024-05-08T08:53:13.030 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.030 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.030 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:13.031 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556915+0000", 2024-05-08T08:53:13.031 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000117852 2024-05-08T08:53:13.031 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.031 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.031 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.031 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.031 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.031 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1082 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11231232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:13.031 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:53:13.031 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 36.508297636000002, 2024-05-08T08:53:13.031 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071591396000000002, 2024-05-08T08:53:13.031 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.031 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.031 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:13.032 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.032 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:13.032 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:13.032 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1082 2024-05-08T08:53:13.032 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.032 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.032 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.032 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.032 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:53:13.032 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.032 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.032 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.032 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.033 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:53:13.033 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.033 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.033 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.033 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.033 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:53:13.033 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1800000000000001e-07 2024-05-08T08:53:13.033 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.033 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.033 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.033 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485587+0000", 2024-05-08T08:53:13.033 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2589999999999999e-06 2024-05-08T08:53:13.033 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.034 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.034 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.034 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485594+0000", 2024-05-08T08:53:13.034 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0789999999999997e-06 2024-05-08T08:53:13.034 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.034 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.034 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.034 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485598+0000", 2024-05-08T08:53:13.034 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7480000000000001e-06 2024-05-08T08:53:13.034 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.034 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.034 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:13.034 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556971+0000", 2024-05-08T08:53:13.034 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071372774999999999 2024-05-08T08:53:13.034 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.035 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.035 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:13.035 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557069+0000", 2024-05-08T08:53:13.035 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.7732999999999994e-05 2024-05-08T08:53:13.035 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.035 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.035 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:13.035 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557176+0000", 2024-05-08T08:53:13.035 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000107384 2024-05-08T08:53:13.035 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.035 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.035 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.035 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.035 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.036 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1084 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11272192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:13.036 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:53:13.036 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 36.508258996999999, 2024-05-08T08:53:13.036 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071808977999999996, 2024-05-08T08:53:13.036 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.036 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.036 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:13.036 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.036 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:13.036 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:13.036 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1084 2024-05-08T08:53:13.036 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.036 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.036 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.037 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.037 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:53:13.037 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.037 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.037 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.037 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.037 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:53:13.037 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.037 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.037 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.037 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.037 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485624+0000", 2024-05-08T08:53:13.037 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:53:13.037 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.037 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.038 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.038 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485626+0000", 2024-05-08T08:53:13.038 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9460000000000001e-06 2024-05-08T08:53:13.038 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.038 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.038 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.038 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485633+0000", 2024-05-08T08:53:13.038 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1149999999999996e-06 2024-05-08T08:53:13.038 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.038 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.038 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.038 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485638+0000", 2024-05-08T08:53:13.038 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5670000000000002e-06 2024-05-08T08:53:13.038 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.039 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.039 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:13.039 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557226+0000", 2024-05-08T08:53:13.039 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071587267999999996 2024-05-08T08:53:13.039 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.039 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.039 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:13.039 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557321+0000", 2024-05-08T08:53:13.039 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5785000000000005e-05 2024-05-08T08:53:13.039 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.039 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.039 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:13.039 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557432+0000", 2024-05-08T08:53:13.039 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011092800000000001 2024-05-08T08:53:13.039 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.040 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.040 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.040 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.040 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.040 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1086 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11313152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:13.040 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:53:13.040 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 36.508235669999998, 2024-05-08T08:53:13.040 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071933132999999996, 2024-05-08T08:53:13.040 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.040 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.040 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:13.040 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.040 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:13.040 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:13.040 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1086 2024-05-08T08:53:13.041 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.041 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.041 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.041 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.041 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:53:13.041 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.041 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.041 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.041 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.041 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:53:13.042 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.042 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.042 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.042 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.042 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486328+0000", 2024-05-08T08:53:13.042 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00068107900000000002 2024-05-08T08:53:13.042 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.042 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.042 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.042 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486331+0000", 2024-05-08T08:53:13.042 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.585e-06 2024-05-08T08:53:13.042 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.042 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.042 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.043 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486343+0000", 2024-05-08T08:53:13.043 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1856e-05 2024-05-08T08:53:13.043 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.043 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.043 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.043 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486350+0000", 2024-05-08T08:53:13.043 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8600000000000004e-06 2024-05-08T08:53:13.043 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.043 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.043 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:13.044 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557470+0000", 2024-05-08T08:53:13.044 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071120326999999997 2024-05-08T08:53:13.044 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.044 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.044 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:13.044 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557514+0000", 2024-05-08T08:53:13.044 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3909000000000003e-05 2024-05-08T08:53:13.044 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.044 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.044 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:13.044 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557580+0000", 2024-05-08T08:53:13.044 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5517000000000003e-05 2024-05-08T08:53:13.044 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.044 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.045 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.045 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.045 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.045 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1089 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11354112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:13.045 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:53:13.045 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 36.507522797999997, 2024-05-08T08:53:13.045 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071457381, 2024-05-08T08:53:13.045 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.045 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.045 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:13.045 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.045 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:13.045 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:13.045 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1089 2024-05-08T08:53:13.045 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.046 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.046 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.046 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.046 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:53:13.046 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.046 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.046 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.046 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.046 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:53:13.046 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.046 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.046 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.047 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.047 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486851+0000", 2024-05-08T08:53:13.047 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00049102 2024-05-08T08:53:13.047 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.047 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.047 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.047 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486854+0000", 2024-05-08T08:53:13.047 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6859999999999999e-06 2024-05-08T08:53:13.047 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.047 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.047 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.047 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486864+0000", 2024-05-08T08:53:13.048 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6080000000000002e-06 2024-05-08T08:53:13.048 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.048 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.048 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.048 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486870+0000", 2024-05-08T08:53:13.048 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.477e-06 2024-05-08T08:53:13.048 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.048 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.048 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:13.048 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557612+0000", 2024-05-08T08:53:13.048 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070741557999999996 2024-05-08T08:53:13.048 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.049 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.049 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:13.049 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557695+0000", 2024-05-08T08:53:13.049 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3300000000000005e-05 2024-05-08T08:53:13.049 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.049 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.049 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:13.049 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557817+0000", 2024-05-08T08:53:13.049 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012173200000000001 2024-05-08T08:53:13.049 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.049 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.049 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.050 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.050 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.050 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1091 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11395072~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:13.050 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:53:13.050 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 36.507002002999997, 2024-05-08T08:53:13.050 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071184737999999997, 2024-05-08T08:53:13.050 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.050 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.050 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:13.050 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.050 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:13.050 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:13.050 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1091 2024-05-08T08:53:13.050 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.051 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.051 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.051 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.051 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:53:13.051 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.051 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.051 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.051 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.051 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:53:13.051 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.051 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.051 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.051 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.051 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486881+0000", 2024-05-08T08:53:13.051 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0699999999999997e-07 2024-05-08T08:53:13.052 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.052 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.052 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.052 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486883+0000", 2024-05-08T08:53:13.052 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.402e-06 2024-05-08T08:53:13.052 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.052 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.052 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.052 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486890+0000", 2024-05-08T08:53:13.052 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0480000000000001e-06 2024-05-08T08:53:13.052 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.052 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.052 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.052 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486896+0000", 2024-05-08T08:53:13.052 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7640000000000002e-06 2024-05-08T08:53:13.053 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.053 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.053 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:13.053 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557860+0000", 2024-05-08T08:53:13.053 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070963791999999998 2024-05-08T08:53:13.053 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.053 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.053 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:13.053 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557952+0000", 2024-05-08T08:53:13.053 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2082000000000004e-05 2024-05-08T08:53:13.053 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.053 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.053 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:13.053 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558065+0000", 2024-05-08T08:53:13.054 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000113143 2024-05-08T08:53:13.054 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.054 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.054 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.054 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.054 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.054 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1094 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11436032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:13.055 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:53:13.055 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 36.506978834999998, 2024-05-08T08:53:13.055 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071400507000000002, 2024-05-08T08:53:13.055 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.055 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.055 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:13.055 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.056 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:13.056 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:13.056 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1094 2024-05-08T08:53:13.056 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.056 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.056 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.057 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.057 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:53:13.057 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.057 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.057 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.057 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.057 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:53:13.058 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.058 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.058 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.058 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.058 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487940+0000", 2024-05-08T08:53:13.059 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001036868 2024-05-08T08:53:13.059 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.059 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.059 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.059 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487944+0000", 2024-05-08T08:53:13.059 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7069999999999998e-06 2024-05-08T08:53:13.060 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.060 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.060 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.060 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487951+0000", 2024-05-08T08:53:13.060 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2649999999999999e-06 2024-05-08T08:53:13.060 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.061 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.061 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.061 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487961+0000", 2024-05-08T08:53:13.061 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4110000000000002e-06 2024-05-08T08:53:13.062 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.062 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.062 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:13.062 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558117+0000", 2024-05-08T08:53:13.062 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070156662999999994 2024-05-08T08:53:13.062 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.063 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.063 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:13.063 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558214+0000", 2024-05-08T08:53:13.063 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6444e-05 2024-05-08T08:53:13.063 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.063 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.064 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:13.064 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558304+0000", 2024-05-08T08:53:13.064 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0148999999999994e-05 2024-05-08T08:53:13.064 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.064 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.064 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.065 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.065 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.065 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1096 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11476992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:13.065 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:53:13.065 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 36.505912303999999, 2024-05-08T08:53:13.065 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070480074000000004, 2024-05-08T08:53:13.066 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.066 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.066 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:13.066 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.066 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:13.066 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:13.067 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1096 2024-05-08T08:53:13.067 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.067 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.067 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.067 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.067 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:53:13.068 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.068 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.068 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.068 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.068 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:53:13.068 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.069 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.069 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.069 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.069 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:53:13.069 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3700000000000001e-07 2024-05-08T08:53:13.069 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.069 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.070 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.070 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487973+0000", 2024-05-08T08:53:13.070 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0870000000000001e-06 2024-05-08T08:53:13.070 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.070 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.070 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.071 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487977+0000", 2024-05-08T08:53:13.071 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0600000000000001e-06 2024-05-08T08:53:13.071 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.071 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.071 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.071 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487980+0000", 2024-05-08T08:53:13.071 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1089999999999998e-06 2024-05-08T08:53:13.072 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.072 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.072 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:13.072 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558338+0000", 2024-05-08T08:53:13.072 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070358233000000006 2024-05-08T08:53:13.072 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.072 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.073 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:13.073 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558392+0000", 2024-05-08T08:53:13.073 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4537999999999999e-05 2024-05-08T08:53:13.073 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.073 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.073 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:13.074 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558450+0000", 2024-05-08T08:53:13.074 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7710000000000001e-05 2024-05-08T08:53:13.074 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.074 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.074 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.074 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.075 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.075 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1099 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11517952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:13.075 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:53:13.075 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 36.505898428000002, 2024-05-08T08:53:13.075 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070720506000000002, 2024-05-08T08:53:13.075 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.076 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.076 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:13.076 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.076 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:13.076 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:13.076 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1099 2024-05-08T08:53:13.077 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.077 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.077 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.077 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.077 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:53:13.077 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.077 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.078 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.078 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.078 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:53:13.078 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.078 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.078 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.079 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.079 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:53:13.079 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3999999999999998e-07 2024-05-08T08:53:13.079 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.079 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.079 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.080 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487986+0000", 2024-05-08T08:53:13.080 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3519999999999999e-06 2024-05-08T08:53:13.080 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.080 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.080 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.080 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487989+0000", 2024-05-08T08:53:13.081 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.23e-06 2024-05-08T08:53:13.081 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.081 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.081 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.081 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487991+0000", 2024-05-08T08:53:13.081 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6350000000000002e-06 2024-05-08T08:53:13.081 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.082 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.082 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:13.082 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558489+0000", 2024-05-08T08:53:13.082 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070497924000000003 2024-05-08T08:53:13.082 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.082 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.082 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:13.083 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558583+0000", 2024-05-08T08:53:13.083 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3405000000000004e-05 2024-05-08T08:53:13.083 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.083 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.083 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:13.083 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558704+0000", 2024-05-08T08:53:13.083 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012171999999999999 2024-05-08T08:53:13.084 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.084 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.084 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.084 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.084 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.084 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1100 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11558912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:13.085 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:53:13.085 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 36.505885429000003, 2024-05-08T08:53:13.085 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070962266999999996, 2024-05-08T08:53:13.085 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.085 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.085 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:13.086 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.086 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:13.086 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:13.086 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1100 2024-05-08T08:53:13.086 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.086 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.087 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.087 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.087 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:53:13.087 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.087 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.087 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.087 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.088 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:53:13.088 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.088 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.088 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.088 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.088 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488430+0000", 2024-05-08T08:53:13.089 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000432676 2024-05-08T08:53:13.089 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.089 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.089 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.089 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488433+0000", 2024-05-08T08:53:13.089 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.45e-06 2024-05-08T08:53:13.089 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.090 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.090 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.090 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488442+0000", 2024-05-08T08:53:13.090 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.687e-06 2024-05-08T08:53:13.090 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.090 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.091 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.091 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488446+0000", 2024-05-08T08:53:13.091 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4939999999999997e-06 2024-05-08T08:53:13.091 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.091 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.091 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:13.092 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558756+0000", 2024-05-08T08:53:13.092 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070310051999999998 2024-05-08T08:53:13.092 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.092 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.092 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:13.092 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558844+0000", 2024-05-08T08:53:13.092 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7690999999999998e-05 2024-05-08T08:53:13.093 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.093 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.093 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:13.093 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558959+0000", 2024-05-08T08:53:13.093 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000115217 2024-05-08T08:53:13.093 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.093 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.094 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.094 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.094 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.094 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1103 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11599872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:13.094 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:53:13.094 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 36.505427754000003, 2024-05-08T08:53:13.095 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070777100999999995, 2024-05-08T08:53:13.095 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.095 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.095 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:13.095 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.095 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:13.095 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:13.096 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1103 2024-05-08T08:53:13.096 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.096 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.096 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.096 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.096 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:53:13.096 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.097 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.097 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.097 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.097 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:53:13.097 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.097 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.098 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.098 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.098 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488959+0000", 2024-05-08T08:53:13.098 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00050457200000000001 2024-05-08T08:53:13.098 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.098 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.098 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.098 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488969+0000", 2024-05-08T08:53:13.099 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.7159999999999998e-06 2024-05-08T08:53:13.099 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.099 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.099 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.099 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488981+0000", 2024-05-08T08:53:13.099 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2150000000000001e-05 2024-05-08T08:53:13.099 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.099 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.099 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.100 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488988+0000", 2024-05-08T08:53:13.100 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8759999999999999e-06 2024-05-08T08:53:13.100 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.100 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.100 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:13.100 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558992+0000", 2024-05-08T08:53:13.100 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070004358000000003 2024-05-08T08:53:13.100 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.100 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.101 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:13.101 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559093+0000", 2024-05-08T08:53:13.101 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000100299 2024-05-08T08:53:13.101 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.101 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.101 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:13.101 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559232+0000", 2024-05-08T08:53:13.101 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013913 2024-05-08T08:53:13.101 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.101 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.102 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.102 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.102 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.102 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1104 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11640832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:13.102 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:53:13.102 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 36.504882938999998, 2024-05-08T08:53:13.102 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070521947000000001, 2024-05-08T08:53:13.102 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.103 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.103 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:13.103 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.103 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:13.103 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:13.103 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1104 2024-05-08T08:53:13.103 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.103 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.103 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.104 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.104 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:53:13.104 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.104 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.104 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.104 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.104 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:53:13.104 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.104 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.104 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.104 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.104 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489700+0000", 2024-05-08T08:53:13.104 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00070067600000000001 2024-05-08T08:53:13.104 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.104 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.104 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.105 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489704+0000", 2024-05-08T08:53:13.105 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1640000000000003e-06 2024-05-08T08:53:13.105 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.105 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.105 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.105 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489716+0000", 2024-05-08T08:53:13.105 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1379000000000001e-05 2024-05-08T08:53:13.105 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.105 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.105 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.105 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489724+0000", 2024-05-08T08:53:13.105 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8949999999999998e-06 2024-05-08T08:53:13.105 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.105 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.105 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:13.105 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559278+0000", 2024-05-08T08:53:13.106 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.069554151999999994 2024-05-08T08:53:13.106 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.106 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.106 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:13.106 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559387+0000", 2024-05-08T08:53:13.106 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000109258 2024-05-08T08:53:13.106 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.106 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.106 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:13.106 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559521+0000", 2024-05-08T08:53:13.106 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000134423 2024-05-08T08:53:13.106 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.106 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.106 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.107 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.107 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.107 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:8636 3.17 3:e8a0a3e4:::rbd_header.11402b26385:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-05-08T08:53:13.107 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:54.150971+0000", 2024-05-08T08:53:13.107 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 18.842910945, 2024-05-08T08:53:13.107 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.217273233, 2024-05-08T08:53:13.107 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.107 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.107 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:13.107 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.107 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:13.107 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:13.107 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 8636 2024-05-08T08:53:13.107 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.107 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.107 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.107 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.108 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150971+0000", 2024-05-08T08:53:13.108 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.108 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.108 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.108 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.108 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150971+0000", 2024-05-08T08:53:13.108 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.108 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.108 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.108 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.108 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150972+0000", 2024-05-08T08:53:13.108 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.71e-07 2024-05-08T08:53:13.108 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.108 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.108 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.108 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150977+0000", 2024-05-08T08:53:13.108 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.152e-06 2024-05-08T08:53:13.109 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.109 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.109 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.109 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150990+0000", 2024-05-08T08:53:13.109 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2364999999999999e-05 2024-05-08T08:53:13.109 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.109 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.109 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.109 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150998+0000", 2024-05-08T08:53:13.109 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0739999999999993e-06 2024-05-08T08:53:13.109 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.109 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.109 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:13.109 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.151054+0000", 2024-05-08T08:53:13.109 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5949000000000003e-05 2024-05-08T08:53:13.109 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.109 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.109 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:53:13.110 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.151060+0000", 2024-05-08T08:53:13.110 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6120000000000004e-06 2024-05-08T08:53:13.110 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.110 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.110 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:13.110 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.367640+0000", 2024-05-08T08:53:13.110 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2165792739999999 2024-05-08T08:53:13.110 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.110 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.110 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:13.110 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.367839+0000", 2024-05-08T08:53:13.110 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000199185 2024-05-08T08:53:13.110 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.110 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.110 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:13.111 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.368245+0000", 2024-05-08T08:53:13.111 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00040575099999999998 2024-05-08T08:53:13.111 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.111 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.111 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.111 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.111 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.111 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:13.112 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:53:13.119 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:53:13.119+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:53:13.184 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:53:13.185 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-05-08T08:53:13.207 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":98,"stamp":"2024-05-08T08:53:11.753567+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":666141374,"num_objects":1479,"num_object_clones":249,"num_object_copies":2958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1479,"num_whiteouts":949,"num_read":155983,"num_read_kb":580047,"num_write":9191,"num_write_kb":227811,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":663,"num_bytes_recovered":29299473,"num_keys_recovered":108,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8608,"ondisk_log_size":8608,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":471859200,"kb_used":221556,"kb_used_data":38852,"kb_used_omap":7,"kb_used_meta":182200,"kb_avail":471637644,"statfs":{"total":483183820800,"available":482956947456,"internally_reserved":0,"allocated":39784448,"data_stored":322133245,"data_compressed":15175643,"data_compressed_allocated":28110848,"data_compressed_original":315355648,"omap_allocated":7973,"internal_metadata":186573019},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":81903756,"num_objects":0,"num_object_clones":-7,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":-21,"num_read":1816,"num_read_kb":45525,"num_write":-173,"num_write_kb":9320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-44,"num_bytes_recovered":-607329,"num_keys_recovered":-12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000582"},"pg_stats":[{"pgid":"3.1f","version":"60'213","reported_seq":12601,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-08T08:53:07.381477+0000","last_change":"2024-05-08T08:52:13.934602+0000","last_active":"2024-05-08T08:53:07.381477+0000","last_peered":"2024-05-08T08:53:07.381477+0000","last_clean":"2024-05-08T08:53:07.381477+0000","last_became_active":"2024-05-08T08:52:11.931332+0000","last_became_peered":"2024-05-08T08:52:11.931332+0000","last_unstale":"2024-05-08T08:53:07.381477+0000","last_undegraded":"2024-05-08T08:53:07.381477+0000","last_fullsized":"2024-05-08T08:53:07.381477+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":213,"log_dups_size":0,"ondisk_log_size":213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:36:22.688129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000691565,"stat_sum":{"num_bytes":46172215,"num_objects":49,"num_object_clones":11,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":29,"num_read":11251,"num_read_kb":52974,"num_write":307,"num_write_kb":28270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"60'299","reported_seq":1893,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-08T08:53:07.467017+0000","last_change":"2024-05-08T08:52:13.944947+0000","last_active":"2024-05-08T08:53:07.467017+0000","last_peered":"2024-05-08T08:53:07.467017+0000","last_clean":"2024-05-08T08:53:07.467017+0000","last_became_active":"2024-05-08T08:52:11.930844+0000","last_became_peered":"2024-05-08T08:52:11.930844+0000","last_unstale":"2024-05-08T08:53:07.467017+0000","last_undegraded":"2024-05-08T08:53:07.467017+0000","last_fullsized":"2024-05-08T08:53:07.467017+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":299,"log_dups_size":0,"ondisk_log_size":299,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:45:49.447564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040424900000000003,"stat_sum":{"num_bytes":13161535,"num_objects":54,"num_object_clones":8,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":37,"num_read":2147,"num_read_kb":17473,"num_write":303,"num_write_kb":8610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"62'131","reported_seq":710,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-08T08:53:07.520050+0000","last_change":"2024-05-08T08:52:36.057025+0000","last_active":"2024-05-08T08:53:07.520050+0000","last_peered":"2024-05-08T08:53:07.520050+0000","last_clean":"2024-05-08T08:53:07.520050+0000","last_became_active":"2024-05-08T08:52:36.024295+0000","last_became_peered":"2024-05-08T08:52:36.024295+0000","last_unstale":"2024-05-08T08:53:07.520050+0000","last_undegraded":"2024-05-08T08:53:07.520050+0000","last_fullsized":"2024-05-08T08:53:07.520050+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:08:44.120289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072111799999999998,"stat_sum":{"num_bytes":11522681,"num_objects":44,"num_object_clones":7,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":23,"num_read":3533,"num_read_kb":14709,"num_write":112,"num_write_kb":3445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":173135,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"61'291","reported_seq":8681,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-08T08:53:07.386245+0000","last_change":"2024-05-08T08:52:50.070942+0000","last_active":"2024-05-08T08:53:07.386245+0000","last_peered":"2024-05-08T08:53:07.386245+0000","last_clean":"2024-05-08T08:53:07.386245+0000","last_became_active":"2024-05-08T08:52:50.069533+0000","last_became_peered":"2024-05-08T08:52:50.069533+0000","last_unstale":"2024-05-08T08:53:07.386245+0000","last_undegraded":"2024-05-08T08:53:07.386245+0000","last_fullsized":"2024-05-08T08:53:07.386245+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":291,"log_dups_size":0,"ondisk_log_size":291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:31:07.508249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094844199999999997,"stat_sum":{"num_bytes":48208437,"num_objects":54,"num_object_clones":7,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":34,"num_read":8783,"num_read_kb":62488,"num_write":375,"num_write_kb":21757,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":13917221,"num_keys_recovered":22,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"62'255","reported_seq":7910,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-08T08:53:07.524442+0000","last_change":"2024-05-08T08:52:50.069794+0000","last_active":"2024-05-08T08:53:07.524442+0000","last_peered":"2024-05-08T08:53:07.524442+0000","last_clean":"2024-05-08T08:53:07.524442+0000","last_became_active":"2024-05-08T08:52:50.069306+0000","last_became_peered":"2024-05-08T08:52:50.069306+0000","last_unstale":"2024-05-08T08:53:07.524442+0000","last_undegraded":"2024-05-08T08:53:07.524442+0000","last_fullsized":"2024-05-08T08:53:07.524442+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":255,"log_dups_size":0,"ondisk_log_size":255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:10:54.156325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000685079,"stat_sum":{"num_bytes":32255532,"num_objects":53,"num_object_clones":8,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":38,"num_read":8113,"num_read_kb":14677,"num_write":319,"num_write_kb":18083,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":146988,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"53'258","reported_seq":7519,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-08T08:53:07.498006+0000","last_change":"2024-05-08T08:52:50.072642+0000","last_active":"2024-05-08T08:53:07.498006+0000","last_peered":"2024-05-08T08:53:07.498006+0000","last_clean":"2024-05-08T08:53:07.498006+0000","last_became_active":"2024-05-08T08:52:50.072082+0000","last_became_peered":"2024-05-08T08:52:50.072082+0000","last_unstale":"2024-05-08T08:53:07.498006+0000","last_undegraded":"2024-05-08T08:53:07.498006+0000","last_fullsized":"2024-05-08T08:53:07.498006+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":258,"log_dups_size":0,"ondisk_log_size":258,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:20:14.748757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066525600000000003,"stat_sum":{"num_bytes":12950081,"num_objects":50,"num_object_clones":8,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":31,"num_read":8068,"num_read_kb":20234,"num_write":265,"num_write_kb":6132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":1862193,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"59'178","reported_seq":1379,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-08T08:53:07.386013+0000","last_change":"2024-05-08T08:52:50.069911+0000","last_active":"2024-05-08T08:53:07.386013+0000","last_peered":"2024-05-08T08:53:07.386013+0000","last_clean":"2024-05-08T08:53:07.386013+0000","last_became_active":"2024-05-08T08:52:50.069328+0000","last_became_peered":"2024-05-08T08:52:50.069328+0000","last_unstale":"2024-05-08T08:53:07.386013+0000","last_undegraded":"2024-05-08T08:53:07.386013+0000","last_fullsized":"2024-05-08T08:53:07.386013+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":178,"log_dups_size":0,"ondisk_log_size":178,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:38:12.590534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082481000000000004,"stat_sum":{"num_bytes":8493072,"num_objects":49,"num_object_clones":8,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":36,"num_read":1008,"num_read_kb":15955,"num_write":141,"num_write_kb":8852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":1896960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"60'271","reported_seq":4366,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-08T08:53:07.388412+0000","last_change":"2024-05-08T08:52:13.945184+0000","last_active":"2024-05-08T08:53:07.388412+0000","last_peered":"2024-05-08T08:53:07.388412+0000","last_clean":"2024-05-08T08:53:07.388412+0000","last_became_active":"2024-05-08T08:52:11.927366+0000","last_became_peered":"2024-05-08T08:52:11.927366+0000","last_unstale":"2024-05-08T08:53:07.388412+0000","last_undegraded":"2024-05-08T08:53:07.388412+0000","last_fullsized":"2024-05-08T08:53:07.388412+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:00:06.209506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057208900000000002,"stat_sum":{"num_bytes":58095652,"num_objects":50,"num_object_clones":10,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":34,"num_read":4587,"num_read_kb":17820,"num_write":317,"num_write_kb":15323,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"unknown","last_fresh":"2024-05-08T08:53:07.386313+0000","last_change":"2024-05-08T08:53:07.759074+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T08:53:07.386313+0000","last_undegraded":"2024-05-08T08:53:07.386313+0000","last_fullsized":"2024-05-08T08:53:07.386313+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"61'334","reported_seq":12897,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-08T08:53:07.438342+0000","last_change":"2024-05-08T08:52:13.932215+0000","last_active":"2024-05-08T08:53:07.438342+0000","last_peered":"2024-05-08T08:53:07.438342+0000","last_clean":"2024-05-08T08:53:07.438342+0000","last_became_active":"2024-05-08T08:52:11.931213+0000","last_became_peered":"2024-05-08T08:52:11.931213+0000","last_unstale":"2024-05-08T08:53:07.438342+0000","last_undegraded":"2024-05-08T08:53:07.438342+0000","last_fullsized":"2024-05-08T08:53:07.438342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":334,"log_dups_size":0,"ondisk_log_size":334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:26:20.626302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084652399999999997,"stat_sum":{"num_bytes":29847743,"num_objects":42,"num_object_clones":7,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":22,"num_read":13628,"num_read_kb":53527,"num_write":419,"num_write_kb":16926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"62'277","reported_seq":3764,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-08T08:53:07.519112+0000","last_change":"2024-05-08T08:52:13.944999+0000","last_active":"2024-05-08T08:53:07.519112+0000","last_peered":"2024-05-08T08:53:07.519112+0000","last_clean":"2024-05-08T08:53:07.519112+0000","last_became_active":"2024-05-08T08:52:11.931695+0000","last_became_peered":"2024-05-08T08:52:11.931695+0000","last_unstale":"2024-05-08T08:53:07.519112+0000","last_undegraded":"2024-05-08T08:53:07.519112+0000","last_fullsized":"2024-05-08T08:53:07.519112+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":277,"log_dups_size":0,"ondisk_log_size":277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:04:58.903868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063902800000000004,"stat_sum":{"num_bytes":11993126,"num_objects":40,"num_object_clones":6,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":29,"num_read":4036,"num_read_kb":7979,"num_write":344,"num_write_kb":1022,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"60'260","reported_seq":4214,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-08T08:53:07.384007+0000","last_change":"2024-05-08T08:52:50.068267+0000","last_active":"2024-05-08T08:53:07.384007+0000","last_peered":"2024-05-08T08:53:07.384007+0000","last_clean":"2024-05-08T08:53:07.384007+0000","last_became_active":"2024-05-08T08:52:50.067650+0000","last_became_peered":"2024-05-08T08:52:50.067650+0000","last_unstale":"2024-05-08T08:53:07.384007+0000","last_undegraded":"2024-05-08T08:53:07.384007+0000","last_fullsized":"2024-05-08T08:53:07.384007+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":260,"log_dups_size":0,"ondisk_log_size":260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:23:57.991585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000430331,"stat_sum":{"num_bytes":25691728,"num_objects":48,"num_object_clones":11,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":29,"num_read":1757,"num_read_kb":7262,"num_write":203,"num_write_kb":4912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":593936,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":106,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-08T08:53:07.384174+0000","last_change":"2024-05-08T08:52:47.057420+0000","last_active":"2024-05-08T08:53:07.384174+0000","last_peered":"2024-05-08T08:53:07.384174+0000","last_clean":"2024-05-08T08:53:07.384174+0000","last_became_active":"2024-05-08T08:52:47.057207+0000","last_became_peered":"2024-05-08T08:52:47.057207+0000","last_unstale":"2024-05-08T08:53:07.384174+0000","last_undegraded":"2024-05-08T08:53:07.384174+0000","last_fullsized":"2024-05-08T08:53:07.384174+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:47:42.112090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"62'214","reported_seq":10096,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-08T08:53:07.494859+0000","last_change":"2024-05-08T08:52:13.934894+0000","last_active":"2024-05-08T08:53:07.494859+0000","last_peered":"2024-05-08T08:53:07.494859+0000","last_clean":"2024-05-08T08:53:07.494859+0000","last_became_active":"2024-05-08T08:52:11.929184+0000","last_became_peered":"2024-05-08T08:52:11.929184+0000","last_unstale":"2024-05-08T08:53:07.494859+0000","last_undegraded":"2024-05-08T08:53:07.494859+0000","last_fullsized":"2024-05-08T08:53:07.494859+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":214,"log_dups_size":0,"ondisk_log_size":214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:56:43.550288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088867899999999997,"stat_sum":{"num_bytes":28320768,"num_objects":41,"num_object_clones":6,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":25,"num_read":10984,"num_read_kb":48176,"num_write":208,"num_write_kb":9748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":102,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-08T08:53:07.386421+0000","last_change":"2024-05-08T08:52:03.896159+0000","last_active":"2024-05-08T08:53:07.386421+0000","last_peered":"2024-05-08T08:53:07.386421+0000","last_clean":"2024-05-08T08:53:07.386421+0000","last_became_active":"2024-05-08T08:52:03.895797+0000","last_became_peered":"2024-05-08T08:52:03.895797+0000","last_unstale":"2024-05-08T08:53:07.386421+0000","last_undegraded":"2024-05-08T08:53:07.386421+0000","last_fullsized":"2024-05-08T08:53:07.386421+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:34:33.513525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"58'156","reported_seq":2021,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-08T08:53:07.389123+0000","last_change":"2024-05-08T08:52:13.944951+0000","last_active":"2024-05-08T08:53:07.389123+0000","last_peered":"2024-05-08T08:53:07.389123+0000","last_clean":"2024-05-08T08:53:07.389123+0000","last_became_active":"2024-05-08T08:52:11.926905+0000","last_became_peered":"2024-05-08T08:52:11.926905+0000","last_unstale":"2024-05-08T08:53:07.389123+0000","last_undegraded":"2024-05-08T08:53:07.389123+0000","last_fullsized":"2024-05-08T08:53:07.389123+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:58:13.662805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044527,"stat_sum":{"num_bytes":8833170,"num_objects":54,"num_object_clones":14,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":29,"num_read":1860,"num_read_kb":20444,"num_write":143,"num_write_kb":6551,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":101,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-08T08:53:07.384203+0000","last_change":"2024-05-08T08:52:04.144764+0000","last_active":"2024-05-08T08:53:07.384203+0000","last_peered":"2024-05-08T08:53:07.384203+0000","last_clean":"2024-05-08T08:53:07.384203+0000","last_became_active":"2024-05-08T08:52:04.144444+0000","last_became_peered":"2024-05-08T08:52:04.144444+0000","last_unstale":"2024-05-08T08:53:07.384203+0000","last_undegraded":"2024-05-08T08:53:07.384203+0000","last_fullsized":"2024-05-08T08:53:07.384203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:58:31.498922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"61'323","reported_seq":5512,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-08T08:53:07.386348+0000","last_change":"2024-05-08T08:53:03.367237+0000","last_active":"2024-05-08T08:53:07.386348+0000","last_peered":"2024-05-08T08:53:07.386348+0000","last_clean":"2024-05-08T08:53:07.386348+0000","last_became_active":"2024-05-08T08:53:03.366185+0000","last_became_peered":"2024-05-08T08:53:03.366185+0000","last_unstale":"2024-05-08T08:53:07.386348+0000","last_undegraded":"2024-05-08T08:53:07.386348+0000","last_fullsized":"2024-05-08T08:53:07.386348+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":323,"log_dups_size":0,"ondisk_log_size":323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:26:57.501526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000669351,"stat_sum":{"num_bytes":11568640,"num_objects":53,"num_object_clones":8,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":36,"num_read":6682,"num_read_kb":16210,"num_write":240,"num_write_kb":6018,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":2621440,"num_keys_recovered":11,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":102,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-08T08:53:07.386928+0000","last_change":"2024-05-08T08:52:04.144543+0000","last_active":"2024-05-08T08:53:07.386928+0000","last_peered":"2024-05-08T08:53:07.386928+0000","last_clean":"2024-05-08T08:53:07.386928+0000","last_became_active":"2024-05-08T08:52:04.144242+0000","last_became_peered":"2024-05-08T08:52:04.144242+0000","last_unstale":"2024-05-08T08:53:07.386928+0000","last_undegraded":"2024-05-08T08:53:07.386928+0000","last_fullsized":"2024-05-08T08:53:07.386928+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:57:36.844633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"62'549","reported_seq":4236,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-08T08:53:07.519095+0000","last_change":"2024-05-08T08:52:23.967224+0000","last_active":"2024-05-08T08:53:07.519095+0000","last_peered":"2024-05-08T08:53:07.519095+0000","last_clean":"2024-05-08T08:53:07.519095+0000","last_became_active":"2024-05-08T08:52:23.966596+0000","last_became_peered":"2024-05-08T08:52:23.966596+0000","last_unstale":"2024-05-08T08:53:07.519095+0000","last_undegraded":"2024-05-08T08:53:07.519095+0000","last_fullsized":"2024-05-08T08:53:07.519095+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":549,"log_dups_size":0,"ondisk_log_size":549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:36:19.824935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064078099999999999,"stat_sum":{"num_bytes":10612971,"num_objects":59,"num_object_clones":6,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":43,"num_read":5676,"num_read_kb":8878,"num_write":823,"num_write_kb":3259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"62'201","reported_seq":896,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-08T08:53:07.493936+0000","last_change":"2024-05-08T08:52:50.070065+0000","last_active":"2024-05-08T08:53:07.493936+0000","last_peered":"2024-05-08T08:53:07.493936+0000","last_clean":"2024-05-08T08:53:07.493936+0000","last_became_active":"2024-05-08T08:52:50.069576+0000","last_became_peered":"2024-05-08T08:52:50.069576+0000","last_unstale":"2024-05-08T08:53:07.493936+0000","last_undegraded":"2024-05-08T08:53:07.493936+0000","last_fullsized":"2024-05-08T08:53:07.493936+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:52:30.448923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047037799999999997,"stat_sum":{"num_bytes":6234211,"num_objects":35,"num_object_clones":7,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":24,"num_read":758,"num_read_kb":6080,"num_write":210,"num_write_kb":2531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":2101248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":77,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-08T08:53:07.386256+0000","last_change":"2024-05-08T08:52:23.969093+0000","last_active":"2024-05-08T08:53:07.386256+0000","last_peered":"2024-05-08T08:53:07.386256+0000","last_clean":"2024-05-08T08:53:07.386256+0000","last_became_active":"2024-05-08T08:52:23.968782+0000","last_became_peered":"2024-05-08T08:52:23.968782+0000","last_unstale":"2024-05-08T08:53:07.386256+0000","last_undegraded":"2024-05-08T08:53:07.386256+0000","last_fullsized":"2024-05-08T08:53:07.386256+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:37:41.693728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"62'159","reported_seq":911,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-08T08:53:07.520040+0000","last_change":"2024-05-08T08:52:13.940504+0000","last_active":"2024-05-08T08:53:07.520040+0000","last_peered":"2024-05-08T08:53:07.520040+0000","last_clean":"2024-05-08T08:53:07.520040+0000","last_became_active":"2024-05-08T08:52:11.928301+0000","last_became_peered":"2024-05-08T08:52:11.928301+0000","last_unstale":"2024-05-08T08:53:07.520040+0000","last_undegraded":"2024-05-08T08:53:07.520040+0000","last_fullsized":"2024-05-08T08:53:07.520040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:32:26.693456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046631800000000002,"stat_sum":{"num_bytes":31656004,"num_objects":49,"num_object_clones":8,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":31,"num_read":717,"num_read_kb":6239,"num_write":133,"num_write_kb":7318,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":102,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-08T08:53:07.388768+0000","last_change":"2024-05-08T08:52:03.891768+0000","last_active":"2024-05-08T08:53:07.388768+0000","last_peered":"2024-05-08T08:53:07.388768+0000","last_clean":"2024-05-08T08:53:07.388768+0000","last_became_active":"2024-05-08T08:52:03.891374+0000","last_became_peered":"2024-05-08T08:52:03.891374+0000","last_unstale":"2024-05-08T08:53:07.388768+0000","last_undegraded":"2024-05-08T08:53:07.388768+0000","last_fullsized":"2024-05-08T08:53:07.388768+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:18:10.240492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"62'487","reported_seq":14444,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-08T08:53:07.472294+0000","last_change":"2024-05-08T08:52:50.071097+0000","last_active":"2024-05-08T08:53:07.472294+0000","last_peered":"2024-05-08T08:53:07.472294+0000","last_clean":"2024-05-08T08:53:07.472294+0000","last_became_active":"2024-05-08T08:52:50.070396+0000","last_became_peered":"2024-05-08T08:52:50.070396+0000","last_unstale":"2024-05-08T08:53:07.472294+0000","last_undegraded":"2024-05-08T08:53:07.472294+0000","last_fullsized":"2024-05-08T08:53:07.472294+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:05:06.760605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046440499999999998,"stat_sum":{"num_bytes":31818323,"num_objects":45,"num_object_clones":8,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":26,"num_read":15518,"num_read_kb":20350,"num_write":293,"num_write_kb":3573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":1271856,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":101,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-08T08:53:07.383884+0000","last_change":"2024-05-08T08:52:03.891551+0000","last_active":"2024-05-08T08:53:07.383884+0000","last_peered":"2024-05-08T08:53:07.383884+0000","last_clean":"2024-05-08T08:53:07.383884+0000","last_became_active":"2024-05-08T08:52:03.891296+0000","last_became_peered":"2024-05-08T08:52:03.891296+0000","last_unstale":"2024-05-08T08:53:07.383884+0000","last_undegraded":"2024-05-08T08:53:07.383884+0000","last_fullsized":"2024-05-08T08:53:07.383884+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:54.806343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"61'224","reported_seq":712,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-08T08:53:07.386237+0000","last_change":"2024-05-08T08:52:13.940743+0000","last_active":"2024-05-08T08:53:07.386237+0000","last_peered":"2024-05-08T08:53:07.386237+0000","last_clean":"2024-05-08T08:53:07.386237+0000","last_became_active":"2024-05-08T08:52:11.928273+0000","last_became_peered":"2024-05-08T08:52:11.928273+0000","last_unstale":"2024-05-08T08:53:07.386237+0000","last_undegraded":"2024-05-08T08:53:07.386237+0000","last_fullsized":"2024-05-08T08:53:07.386237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":224,"log_dups_size":0,"ondisk_log_size":224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:58:50.222915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062135599999999999,"stat_sum":{"num_bytes":7776485,"num_objects":52,"num_object_clones":12,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":33,"num_read":975,"num_read_kb":4050,"num_write":267,"num_write_kb":1308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":77,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-08T08:53:07.388771+0000","last_change":"2024-05-08T08:52:23.966794+0000","last_active":"2024-05-08T08:53:07.388771+0000","last_peered":"2024-05-08T08:53:07.388771+0000","last_clean":"2024-05-08T08:53:07.388771+0000","last_became_active":"2024-05-08T08:52:23.966544+0000","last_became_peered":"2024-05-08T08:52:23.966544+0000","last_unstale":"2024-05-08T08:53:07.388771+0000","last_undegraded":"2024-05-08T08:53:07.388771+0000","last_fullsized":"2024-05-08T08:53:07.388771+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:25:56.927420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":257,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-08T08:53:07.383889+0000","last_change":"2024-05-08T08:51:58.861899+0000","last_active":"2024-05-08T08:53:07.383889+0000","last_peered":"2024-05-08T08:53:07.383889+0000","last_clean":"2024-05-08T08:53:07.383889+0000","last_became_active":"2024-05-08T08:51:58.861596+0000","last_became_peered":"2024-05-08T08:51:58.861596+0000","last_unstale":"2024-05-08T08:53:07.383889+0000","last_undegraded":"2024-05-08T08:53:07.383889+0000","last_fullsized":"2024-05-08T08:53:07.383889+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:51:57.849666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:51:57.849666+0000","last_clean_scrub_stamp":"2024-05-08T08:51:57.849666+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:53:21.739950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"62'127","reported_seq":912,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-08T08:53:07.493610+0000","last_change":"2024-05-08T08:52:13.945362+0000","last_active":"2024-05-08T08:53:07.493610+0000","last_peered":"2024-05-08T08:53:07.493610+0000","last_clean":"2024-05-08T08:53:07.493610+0000","last_became_active":"2024-05-08T08:52:11.926975+0000","last_became_peered":"2024-05-08T08:52:11.926975+0000","last_unstale":"2024-05-08T08:53:07.493610+0000","last_undegraded":"2024-05-08T08:53:07.493610+0000","last_fullsized":"2024-05-08T08:53:07.493610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:46:20.475744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068218599999999999,"stat_sum":{"num_bytes":21010448,"num_objects":48,"num_object_clones":7,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":34,"num_read":725,"num_read_kb":4913,"num_write":95,"num_write_kb":2557,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"60'215","reported_seq":5242,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-08T08:53:07.518164+0000","last_change":"2024-05-08T08:52:50.069736+0000","last_active":"2024-05-08T08:53:07.518164+0000","last_peered":"2024-05-08T08:53:07.518164+0000","last_clean":"2024-05-08T08:53:07.518164+0000","last_became_active":"2024-05-08T08:52:50.069112+0000","last_became_peered":"2024-05-08T08:52:50.069112+0000","last_unstale":"2024-05-08T08:53:07.518164+0000","last_undegraded":"2024-05-08T08:53:07.518164+0000","last_fullsized":"2024-05-08T08:53:07.518164+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":215,"log_dups_size":0,"ondisk_log_size":215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:14:53.908535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00030754000000000003,"stat_sum":{"num_bytes":22947328,"num_objects":45,"num_object_clones":7,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":29,"num_read":5336,"num_read_kb":17758,"num_write":169,"num_write_kb":10827,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":1878528,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"62'470","reported_seq":8015,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-08T08:53:07.506746+0000","last_change":"2024-05-08T08:52:50.072624+0000","last_active":"2024-05-08T08:53:07.506746+0000","last_peered":"2024-05-08T08:53:07.506746+0000","last_clean":"2024-05-08T08:53:07.506746+0000","last_became_active":"2024-05-08T08:52:50.071883+0000","last_became_peered":"2024-05-08T08:52:50.071883+0000","last_unstale":"2024-05-08T08:53:07.506746+0000","last_undegraded":"2024-05-08T08:53:07.506746+0000","last_fullsized":"2024-05-08T08:53:07.506746+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":470,"log_dups_size":0,"ondisk_log_size":470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:54:10.648460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070350100000000004,"stat_sum":{"num_bytes":17735828,"num_objects":49,"num_object_clones":7,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":31,"num_read":10650,"num_read_kb":16865,"num_write":321,"num_write_kb":2629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":54,"num_bytes_recovered":426496,"num_keys_recovered":28,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"62'272","reported_seq":2557,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-08T08:53:07.519260+0000","last_change":"2024-05-08T08:52:13.940600+0000","last_active":"2024-05-08T08:53:07.519260+0000","last_peered":"2024-05-08T08:53:07.519260+0000","last_clean":"2024-05-08T08:53:07.519260+0000","last_became_active":"2024-05-08T08:52:11.927196+0000","last_became_peered":"2024-05-08T08:52:11.927196+0000","last_unstale":"2024-05-08T08:53:07.519260+0000","last_undegraded":"2024-05-08T08:53:07.519260+0000","last_fullsized":"2024-05-08T08:53:07.519260+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":272,"log_dups_size":0,"ondisk_log_size":272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:00:28.674407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048296599999999998,"stat_sum":{"num_bytes":29128224,"num_objects":45,"num_object_clones":6,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":30,"num_read":2907,"num_read_kb":6869,"num_write":205,"num_write_kb":2054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"62'126","reported_seq":585,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-08T08:53:07.494368+0000","last_change":"2024-05-08T08:52:23.969521+0000","last_active":"2024-05-08T08:53:07.494368+0000","last_peered":"2024-05-08T08:53:07.494368+0000","last_clean":"2024-05-08T08:53:07.494368+0000","last_became_active":"2024-05-08T08:52:23.969067+0000","last_became_peered":"2024-05-08T08:52:23.969067+0000","last_unstale":"2024-05-08T08:53:07.494368+0000","last_undegraded":"2024-05-08T08:53:07.494368+0000","last_fullsized":"2024-05-08T08:53:07.494368+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:48:01.189965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048793000000000003,"stat_sum":{"num_bytes":4697600,"num_objects":41,"num_object_clones":6,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":31,"num_read":389,"num_read_kb":2806,"num_write":98,"num_write_kb":952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"60'202","reported_seq":993,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-08T08:53:07.384158+0000","last_change":"2024-05-08T08:52:13.945141+0000","last_active":"2024-05-08T08:53:07.384158+0000","last_peered":"2024-05-08T08:53:07.384158+0000","last_clean":"2024-05-08T08:53:07.384158+0000","last_became_active":"2024-05-08T08:52:11.932594+0000","last_became_peered":"2024-05-08T08:52:11.932594+0000","last_unstale":"2024-05-08T08:53:07.384158+0000","last_undegraded":"2024-05-08T08:53:07.384158+0000","last_fullsized":"2024-05-08T08:53:07.384158+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":202,"log_dups_size":0,"ondisk_log_size":202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:09:04.441251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074388199999999996,"stat_sum":{"num_bytes":7556096,"num_objects":48,"num_object_clones":7,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":34,"num_read":725,"num_read_kb":7383,"num_write":170,"num_write_kb":2517,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"59'127","reported_seq":366,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-08T08:53:07.386311+0000","last_change":"2024-05-08T08:52:47.137016+0000","last_active":"2024-05-08T08:53:07.386311+0000","last_peered":"2024-05-08T08:53:07.386311+0000","last_clean":"2024-05-08T08:53:07.386311+0000","last_became_active":"2024-05-08T08:52:47.071804+0000","last_became_peered":"2024-05-08T08:52:47.071804+0000","last_unstale":"2024-05-08T08:53:07.386311+0000","last_undegraded":"2024-05-08T08:53:07.386311+0000","last_fullsized":"2024-05-08T08:53:07.386311+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:19:54.992081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085079600000000002,"stat_sum":{"num_bytes":10514432,"num_objects":29,"num_object_clones":6,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":17,"num_read":5522,"num_read_kb":20527,"num_write":110,"num_write_kb":3740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":2343424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.f","version":"62'166","reported_seq":2113,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-08T08:53:07.498386+0000","last_change":"2024-05-08T08:52:13.932524+0000","last_active":"2024-05-08T08:53:07.498386+0000","last_peered":"2024-05-08T08:53:07.498386+0000","last_clean":"2024-05-08T08:53:07.498386+0000","last_became_active":"2024-05-08T08:52:11.931347+0000","last_became_peered":"2024-05-08T08:52:11.931347+0000","last_unstale":"2024-05-08T08:53:07.498386+0000","last_undegraded":"2024-05-08T08:53:07.498386+0000","last_fullsized":"2024-05-08T08:53:07.498386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:02:28.575511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010326370000000001,"stat_sum":{"num_bytes":12068547,"num_objects":35,"num_object_clones":3,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":23,"num_read":2221,"num_read_kb":14668,"num_write":185,"num_write_kb":1675,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"62'156","reported_seq":2471,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-08T08:53:07.522004+0000","last_change":"2024-05-08T08:52:23.970138+0000","last_active":"2024-05-08T08:53:07.522004+0000","last_peered":"2024-05-08T08:53:07.522004+0000","last_clean":"2024-05-08T08:53:07.522004+0000","last_became_active":"2024-05-08T08:52:23.969732+0000","last_became_peered":"2024-05-08T08:52:23.969732+0000","last_unstale":"2024-05-08T08:53:07.522004+0000","last_undegraded":"2024-05-08T08:53:07.522004+0000","last_fullsized":"2024-05-08T08:53:07.522004+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:39:56.250514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053593,"stat_sum":{"num_bytes":22476288,"num_objects":48,"num_object_clones":8,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":32,"num_read":2256,"num_read_kb":20222,"num_write":124,"num_write_kb":9335,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"58'180","reported_seq":872,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-08T08:53:07.383898+0000","last_change":"2024-05-08T08:52:50.071476+0000","last_active":"2024-05-08T08:53:07.383898+0000","last_peered":"2024-05-08T08:53:07.383898+0000","last_clean":"2024-05-08T08:53:07.383898+0000","last_became_active":"2024-05-08T08:52:50.070771+0000","last_became_peered":"2024-05-08T08:52:50.070771+0000","last_unstale":"2024-05-08T08:53:07.383898+0000","last_undegraded":"2024-05-08T08:53:07.383898+0000","last_fullsized":"2024-05-08T08:53:07.383898+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:24:45.166436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041230699999999999,"stat_sum":{"num_bytes":13597696,"num_objects":51,"num_object_clones":9,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":37,"num_read":521,"num_read_kb":4177,"num_write":146,"num_write_kb":2170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":66048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"62'1075","reported_seq":6603,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-08T08:53:07.457928+0000","last_change":"2024-05-08T08:52:13.934541+0000","last_active":"2024-05-08T08:53:07.457928+0000","last_peered":"2024-05-08T08:53:07.457928+0000","last_clean":"2024-05-08T08:53:07.457928+0000","last_became_active":"2024-05-08T08:52:11.931202+0000","last_became_peered":"2024-05-08T08:52:11.931202+0000","last_unstale":"2024-05-08T08:53:07.457928+0000","last_undegraded":"2024-05-08T08:53:07.457928+0000","last_fullsized":"2024-05-08T08:53:07.457928+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":1075,"log_dups_size":0,"ondisk_log_size":1075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:45:08.156933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059220300000000002,"stat_sum":{"num_bytes":24402580,"num_objects":54,"num_object_clones":6,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":31,"num_read":11863,"num_read_kb":23961,"num_write":1679,"num_write_kb":5193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"59'295","reported_seq":2680,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-08T08:53:07.384083+0000","last_change":"2024-05-08T08:52:13.944894+0000","last_active":"2024-05-08T08:53:07.384083+0000","last_peered":"2024-05-08T08:53:07.384083+0000","last_clean":"2024-05-08T08:53:07.384083+0000","last_became_active":"2024-05-08T08:52:11.930434+0000","last_became_peered":"2024-05-08T08:52:11.930434+0000","last_unstale":"2024-05-08T08:53:07.384083+0000","last_undegraded":"2024-05-08T08:53:07.384083+0000","last_fullsized":"2024-05-08T08:53:07.384083+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":295,"log_dups_size":0,"ondisk_log_size":295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:28:20.154553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059972400000000005,"stat_sum":{"num_bytes":44203565,"num_objects":63,"num_object_clones":17,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":31,"num_read":2671,"num_read_kb":24273,"num_write":308,"num_write_kb":8108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":665551006,"num_objects":1477,"num_object_clones":249,"num_object_copies":2954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1477,"num_whiteouts":949,"num_read":155867,"num_read_kb":579947,"num_write":9032,"num_write_kb":225395,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":663,"num_bytes_recovered":29299473,"num_keys_recovered":108,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":39051264,"data_stored":345178816,"data_compressed":16068094,"data_compressed_allocated":30437376,"data_compressed_original":339494400,"omap_allocated":0,"internal_metadata":0},"log_size":8491,"ondisk_log_size":8491,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":11046,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640272,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48576,"kb_used_data":9076,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94323264,"statfs":{"total":96636764160,"available":96587022336,"internally_reserved":0,"allocated":9293824,"data_stored":77763147,"data_compressed":3925578,"data_compressed_allocated":7401472,"data_compressed_original":76694528,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":0,"seq":47244640275,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640274,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37672,"kb_used_data":10436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94334168,"statfs":{"total":96636764160,"available":96598188032,"internally_reserved":0,"allocated":10686464,"data_stored":84592817,"data_compressed":4315445,"data_compressed_allocated":7139328,"data_compressed_original":82387968,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640273,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40796,"kb_used_data":5236,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94331044,"statfs":{"total":96636764160,"available":96594989056,"internally_reserved":0,"allocated":5361664,"data_stored":42587824,"data_compressed":2049832,"data_compressed_allocated":3956736,"data_compressed_original":41893376,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640272,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41892,"kb_used_data":6552,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94329948,"statfs":{"total":96636764160,"available":96593866752,"internally_reserved":0,"allocated":6709248,"data_stored":63087315,"data_compressed":2432752,"data_compressed_allocated":5169152,"data_compressed_original":62269440,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640275,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52620,"kb_used_data":7552,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94319220,"statfs":{"total":96636764160,"available":96582881280,"internally_reserved":0,"allocated":7733248,"data_stored":54102142,"data_compressed":2452036,"data_compressed_allocated":4444160,"data_compressed_original":52110336,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5523,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5523,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":7008256,"data_stored":53267088,"data_compressed":2468119,"data_compressed_allocated":4431872,"data_compressed_original":51569664,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":6008832,"data_stored":62807813,"data_compressed":2432752,"data_compressed_allocated":5169152,"data_compressed_original":62269440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4648960,"data_stored":42303273,"data_compressed":2049832,"data_compressed_allocated":3956736,"data_compressed_original":41893376,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":9949184,"data_stored":83744963,"data_compressed":4323414,"data_compressed_allocated":7118848,"data_compressed_original":81835008,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":2830336,"data_stored":25559234,"data_compressed":860285,"data_compressed_allocated":2351104,"data_compressed_original":25220096,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8605696,"data_stored":77496445,"data_compressed":3933692,"data_compressed_allocated":7409664,"data_compressed_original":76706816,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T08:53:13.214 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:13.217+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:13.314 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:13.317+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:13.327 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:13.327 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:53:13.327 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.327 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:26506 3.4 3.134fb004 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.327 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.269363+0000", 2024-05-08T08:53:13.327 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.045116120000000003, 2024-05-08T08:53:13.327 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.045141140000000003, 2024-05-08T08:53:13.327 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.327 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.327 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.327 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.327 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.328 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.328 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 26506 2024-05-08T08:53:13.328 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.328 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.328 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.328 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.328 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.269363+0000", 2024-05-08T08:53:13.328 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.328 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.328 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.328 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.329 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.269363+0000", 2024-05-08T08:53:13.329 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.329 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.329 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.329 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.329 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.269364+0000", 2024-05-08T08:53:13.329 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.63e-07 2024-05-08T08:53:13.329 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.329 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.329 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.329 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.269366+0000", 2024-05-08T08:53:13.329 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.39e-06 2024-05-08T08:53:13.329 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.329 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.330 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.330 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.269373+0000", 2024-05-08T08:53:13.330 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1269999999999997e-06 2024-05-08T08:53:13.330 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.330 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.330 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.330 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.269378+0000", 2024-05-08T08:53:13.330 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4269999999999999e-06 2024-05-08T08:53:13.330 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.330 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.330 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.330 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.330 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.330 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:26758 3.4 3.134fb004 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.331 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.283653+0000", 2024-05-08T08:53:13.331 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.030825997000000001, 2024-05-08T08:53:13.331 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.030889206999999998, 2024-05-08T08:53:13.331 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.331 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.331 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.331 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.331 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.331 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.331 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 26758 2024-05-08T08:53:13.331 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.331 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.331 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.331 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.332 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283653+0000", 2024-05-08T08:53:13.332 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.332 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.332 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.332 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.332 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283653+0000", 2024-05-08T08:53:13.332 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.332 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.332 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.332 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.332 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283653+0000", 2024-05-08T08:53:13.332 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.23e-07 2024-05-08T08:53:13.332 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.332 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.333 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.333 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283655+0000", 2024-05-08T08:53:13.333 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1370000000000001e-06 2024-05-08T08:53:13.333 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.333 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.333 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.333 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283658+0000", 2024-05-08T08:53:13.333 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9639999999999999e-06 2024-05-08T08:53:13.333 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.333 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.333 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.333 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283660+0000", 2024-05-08T08:53:13.333 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.5390000000000002e-06 2024-05-08T08:53:13.333 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.334 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.334 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.334 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.334 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.334 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:11615 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.334 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.292372+0000", 2024-05-08T08:53:13.334 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.022107437000000001, 2024-05-08T08:53:13.334 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.022200078000000002, 2024-05-08T08:53:13.334 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.334 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.334 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.334 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.334 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:13.335 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:13.335 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 11615 2024-05-08T08:53:13.335 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.335 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.335 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.335 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.335 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.292372+0000", 2024-05-08T08:53:13.335 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.335 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.335 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.335 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.335 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.292372+0000", 2024-05-08T08:53:13.335 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.336 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.336 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.336 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.336 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.292373+0000", 2024-05-08T08:53:13.336 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8899999999999998e-07 2024-05-08T08:53:13.336 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.336 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.336 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.336 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.292377+0000", 2024-05-08T08:53:13.336 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9259999999999999e-06 2024-05-08T08:53:13.336 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.336 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.336 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.336 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.292386+0000", 2024-05-08T08:53:13.337 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.6580000000000008e-06 2024-05-08T08:53:13.337 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.337 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.337 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.337 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.292393+0000", 2024-05-08T08:53:13.337 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6000000000000003e-06 2024-05-08T08:53:13.337 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.337 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.337 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.337 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.337 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.337 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27291 3.4 3.134fb004 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.337 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.302553+0000", 2024-05-08T08:53:13.337 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.011926153, 2024-05-08T08:53:13.337 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.012046786, 2024-05-08T08:53:13.338 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.338 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.338 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.338 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.338 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.338 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.338 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27291 2024-05-08T08:53:13.338 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.338 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.338 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.338 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.338 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302553+0000", 2024-05-08T08:53:13.338 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.338 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.338 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.339 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.339 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302553+0000", 2024-05-08T08:53:13.339 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.339 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.339 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.339 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.339 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302554+0000", 2024-05-08T08:53:13.339 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7100000000000002e-07 2024-05-08T08:53:13.339 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.339 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.339 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.339 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302556+0000", 2024-05-08T08:53:13.339 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7389999999999999e-06 2024-05-08T08:53:13.339 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.339 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.339 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.339 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302564+0000", 2024-05-08T08:53:13.340 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5229999999999998e-06 2024-05-08T08:53:13.340 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.340 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.340 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.340 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302570+0000", 2024-05-08T08:53:13.340 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9819999999999996e-06 2024-05-08T08:53:13.340 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.340 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.340 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.340 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.340 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.340 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27391 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.340 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.303582+0000", 2024-05-08T08:53:13.340 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.010896964, 2024-05-08T08:53:13.341 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.011044771, 2024-05-08T08:53:13.341 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.341 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.341 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.341 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.341 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.341 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.341 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27391 2024-05-08T08:53:13.341 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.341 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.341 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.341 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.341 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303582+0000", 2024-05-08T08:53:13.341 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.342 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.342 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.342 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.342 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303582+0000", 2024-05-08T08:53:13.342 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.342 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.342 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.342 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.342 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303583+0000", 2024-05-08T08:53:13.342 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.58e-07 2024-05-08T08:53:13.342 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.342 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.342 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.342 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303585+0000", 2024-05-08T08:53:13.343 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6539999999999999e-06 2024-05-08T08:53:13.343 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.343 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.343 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.343 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303591+0000", 2024-05-08T08:53:13.343 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8900000000000004e-06 2024-05-08T08:53:13.343 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.343 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.343 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.343 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303596+0000", 2024-05-08T08:53:13.343 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8760000000000004e-06 2024-05-08T08:53:13.343 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.343 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.343 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.344 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.344 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.344 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:26507 3.4 3.134fb004 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.344 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.269386+0000", 2024-05-08T08:53:13.344 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.045093112999999997, 2024-05-08T08:53:13.344 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.045269546000000001, 2024-05-08T08:53:13.344 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.344 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.344 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.344 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.344 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.344 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.344 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 26507 2024-05-08T08:53:13.344 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.345 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.345 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.345 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.345 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.269386+0000", 2024-05-08T08:53:13.345 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.345 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.345 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.345 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.345 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.269386+0000", 2024-05-08T08:53:13.345 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.345 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.345 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.345 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.346 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.269386+0000", 2024-05-08T08:53:13.346 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0499999999999999e-07 2024-05-08T08:53:13.346 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.346 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.346 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.346 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.269389+0000", 2024-05-08T08:53:13.346 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2699999999999999e-06 2024-05-08T08:53:13.346 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.346 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.346 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.346 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.269395+0000", 2024-05-08T08:53:13.346 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5060000000000001e-06 2024-05-08T08:53:13.346 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.346 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.347 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.347 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.269401+0000", 2024-05-08T08:53:13.347 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5790000000000003e-06 2024-05-08T08:53:13.347 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.347 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.347 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.347 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.347 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.347 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:11607 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.347 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.277902+0000", 2024-05-08T08:53:13.347 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.036577426000000003, 2024-05-08T08:53:13.347 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.036785213999999997, 2024-05-08T08:53:13.347 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.347 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.348 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.348 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.348 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:13.348 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:13.348 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 11607 2024-05-08T08:53:13.348 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.348 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.348 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.348 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.348 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.277902+0000", 2024-05-08T08:53:13.348 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.348 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.348 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.348 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.348 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.277902+0000", 2024-05-08T08:53:13.349 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.349 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.349 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.349 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.349 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.277903+0000", 2024-05-08T08:53:13.349 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.71e-07 2024-05-08T08:53:13.349 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.349 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.349 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.349 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.277908+0000", 2024-05-08T08:53:13.349 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9699999999999998e-06 2024-05-08T08:53:13.349 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.349 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.349 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.349 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.277918+0000", 2024-05-08T08:53:13.349 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0190000000000001e-05 2024-05-08T08:53:13.350 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.350 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.350 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.350 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.277925+0000", 2024-05-08T08:53:13.350 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7490000000000001e-06 2024-05-08T08:53:13.350 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.350 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.350 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.350 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.351 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.351 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:26759 3.4 3.134fb004 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.351 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.283663+0000", 2024-05-08T08:53:13.351 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.030816018000000001, 2024-05-08T08:53:13.351 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.031074924, 2024-05-08T08:53:13.351 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.351 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.351 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.351 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.351 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.351 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.351 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 26759 2024-05-08T08:53:13.351 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.351 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.351 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.351 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.352 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283663+0000", 2024-05-08T08:53:13.352 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.352 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.352 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.352 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.352 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283663+0000", 2024-05-08T08:53:13.352 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.352 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.352 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.352 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.352 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283663+0000", 2024-05-08T08:53:13.352 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.04e-07 2024-05-08T08:53:13.352 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.352 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.352 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.353 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283664+0000", 2024-05-08T08:53:13.353 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1090000000000001e-06 2024-05-08T08:53:13.353 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.353 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.353 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.353 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283667+0000", 2024-05-08T08:53:13.353 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9019999999999999e-06 2024-05-08T08:53:13.353 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.353 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.353 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.353 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283670+0000", 2024-05-08T08:53:13.353 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4360000000000001e-06 2024-05-08T08:53:13.353 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.353 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.353 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.354 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.354 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.354 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27292 3.4 3.134fb004 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.354 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.302578+0000", 2024-05-08T08:53:13.354 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.01190132, 2024-05-08T08:53:13.354 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.012194491999999999, 2024-05-08T08:53:13.354 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.354 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.354 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.354 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.354 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.354 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.354 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27292 2024-05-08T08:53:13.354 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.354 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.355 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.355 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.355 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302578+0000", 2024-05-08T08:53:13.355 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.355 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.355 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.355 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.355 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302578+0000", 2024-05-08T08:53:13.355 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.355 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.355 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.355 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.355 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302578+0000", 2024-05-08T08:53:13.356 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7e-07 2024-05-08T08:53:13.356 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.356 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.356 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.356 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302587+0000", 2024-05-08T08:53:13.356 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.6260000000000001e-06 2024-05-08T08:53:13.356 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.356 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.356 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.356 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302594+0000", 2024-05-08T08:53:13.356 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7229999999999998e-06 2024-05-08T08:53:13.356 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.356 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.356 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.356 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302599+0000", 2024-05-08T08:53:13.356 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9370000000000003e-06 2024-05-08T08:53:13.357 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.357 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.357 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.357 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.357 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.357 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27395 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.357 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.303602+0000", 2024-05-08T08:53:13.357 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.010877474999999999, 2024-05-08T08:53:13.357 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.01120086, 2024-05-08T08:53:13.357 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.357 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.357 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.357 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.357 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.357 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.358 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27395 2024-05-08T08:53:13.358 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.358 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.358 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.358 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.358 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303602+0000", 2024-05-08T08:53:13.358 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.358 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.358 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.358 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.358 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303602+0000", 2024-05-08T08:53:13.358 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.358 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.358 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.359 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.359 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303602+0000", 2024-05-08T08:53:13.359 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4400000000000001e-07 2024-05-08T08:53:13.359 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.359 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.359 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.359 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303604+0000", 2024-05-08T08:53:13.359 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4260000000000002e-06 2024-05-08T08:53:13.359 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.359 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.359 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.359 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303610+0000", 2024-05-08T08:53:13.359 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9440000000000002e-06 2024-05-08T08:53:13.359 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.359 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.360 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.360 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303615+0000", 2024-05-08T08:53:13.360 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5179999999999998e-06 2024-05-08T08:53:13.360 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.360 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.360 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.360 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.360 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.360 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27445 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.360 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.305421+0000", 2024-05-08T08:53:13.360 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0090580160000000003, 2024-05-08T08:53:13.360 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0094085680000000008, 2024-05-08T08:53:13.360 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.360 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.360 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.361 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.361 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.361 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.361 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27445 2024-05-08T08:53:13.361 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.361 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.361 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.361 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.361 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.305421+0000", 2024-05-08T08:53:13.361 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.361 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.361 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.361 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.361 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.305421+0000", 2024-05-08T08:53:13.361 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.362 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.362 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.362 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.362 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.305422+0000", 2024-05-08T08:53:13.362 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2500000000000001e-07 2024-05-08T08:53:13.362 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.362 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.362 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.362 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.305424+0000", 2024-05-08T08:53:13.362 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6180000000000002e-06 2024-05-08T08:53:13.362 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.362 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.362 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.362 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.305430+0000", 2024-05-08T08:53:13.362 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9519999999999999e-06 2024-05-08T08:53:13.363 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.363 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.363 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.363 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.305435+0000", 2024-05-08T08:53:13.363 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6770000000000003e-06 2024-05-08T08:53:13.363 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.363 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.363 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.363 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.363 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.363 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:26508 3.4 3.134fb004 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.363 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.269410+0000", 2024-05-08T08:53:13.363 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.045068872000000003, 2024-05-08T08:53:13.363 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.045446612999999997, 2024-05-08T08:53:13.363 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.363 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.364 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.364 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.364 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.364 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.364 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 26508 2024-05-08T08:53:13.364 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.364 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.364 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.364 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.364 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.269410+0000", 2024-05-08T08:53:13.364 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.364 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.364 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.364 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.364 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.269410+0000", 2024-05-08T08:53:13.365 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.365 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.365 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.365 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.365 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.269411+0000", 2024-05-08T08:53:13.365 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1399999999999997e-07 2024-05-08T08:53:13.365 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.365 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.365 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.365 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.269413+0000", 2024-05-08T08:53:13.365 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2989999999999999e-06 2024-05-08T08:53:13.365 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.365 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.365 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.365 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.269420+0000", 2024-05-08T08:53:13.366 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9059999999999996e-06 2024-05-08T08:53:13.366 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.366 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.366 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.366 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.269425+0000", 2024-05-08T08:53:13.366 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1050000000000003e-06 2024-05-08T08:53:13.366 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.366 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.366 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.366 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.366 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.366 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:26760 3.4 3.134fb004 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.366 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.283673+0000", 2024-05-08T08:53:13.366 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.030806257, 2024-05-08T08:53:13.366 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.031224704999999998, 2024-05-08T08:53:13.367 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.367 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.367 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.367 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.367 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.367 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.367 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 26760 2024-05-08T08:53:13.367 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.367 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.367 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.367 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.367 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283673+0000", 2024-05-08T08:53:13.367 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.367 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.367 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.367 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.368 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283673+0000", 2024-05-08T08:53:13.368 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.368 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.368 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.368 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.368 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283673+0000", 2024-05-08T08:53:13.368 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0900000000000001e-07 2024-05-08T08:53:13.368 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.368 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.368 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.368 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283674+0000", 2024-05-08T08:53:13.368 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0839999999999999e-06 2024-05-08T08:53:13.368 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.368 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.369 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.369 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283677+0000", 2024-05-08T08:53:13.369 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.942e-06 2024-05-08T08:53:13.369 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.369 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.369 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.369 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283680+0000", 2024-05-08T08:53:13.369 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.88e-06 2024-05-08T08:53:13.369 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.369 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.369 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.369 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.369 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.369 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:26974 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.369 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.289366+0000", 2024-05-08T08:53:13.370 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.02511292, 2024-05-08T08:53:13.370 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.025592518000000002, 2024-05-08T08:53:13.370 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.370 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.370 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.370 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.370 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.370 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.370 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 26974 2024-05-08T08:53:13.370 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.370 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.370 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.370 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.370 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289366+0000", 2024-05-08T08:53:13.371 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.371 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.371 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.371 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.371 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289366+0000", 2024-05-08T08:53:13.371 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.371 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.371 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.371 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.371 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289367+0000", 2024-05-08T08:53:13.371 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1399999999999997e-07 2024-05-08T08:53:13.371 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.371 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.371 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.371 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289370+0000", 2024-05-08T08:53:13.372 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0529999999999998e-06 2024-05-08T08:53:13.372 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.372 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.372 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.372 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289376+0000", 2024-05-08T08:53:13.372 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.302e-06 2024-05-08T08:53:13.372 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.372 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.372 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.372 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289382+0000", 2024-05-08T08:53:13.372 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8549999999999998e-06 2024-05-08T08:53:13.372 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.372 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.372 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.372 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.373 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.373 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27293 3.4 3.134fb004 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.373 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.302607+0000", 2024-05-08T08:53:13.373 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.011872554, 2024-05-08T08:53:13.373 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.012408850000000001, 2024-05-08T08:53:13.373 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.373 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.373 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.373 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.373 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.373 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.373 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27293 2024-05-08T08:53:13.373 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.373 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.373 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.373 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.374 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302607+0000", 2024-05-08T08:53:13.374 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.374 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.374 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.374 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.374 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302607+0000", 2024-05-08T08:53:13.374 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.374 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.374 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.374 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.374 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302607+0000", 2024-05-08T08:53:13.374 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0200000000000003e-07 2024-05-08T08:53:13.374 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.374 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.375 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.375 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302609+0000", 2024-05-08T08:53:13.375 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.441e-06 2024-05-08T08:53:13.375 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.375 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.375 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.375 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302616+0000", 2024-05-08T08:53:13.375 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3999999999999997e-06 2024-05-08T08:53:13.375 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.375 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.375 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.375 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302621+0000", 2024-05-08T08:53:13.375 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2079999999999999e-06 2024-05-08T08:53:13.375 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.375 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.376 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.376 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.376 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.376 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:11584 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.376 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.269477+0000", 2024-05-08T08:53:13.376 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.045001742999999997, 2024-05-08T08:53:13.376 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.045597115000000001, 2024-05-08T08:53:13.376 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.376 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.376 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.376 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.376 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:13.376 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:13.376 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 11584 2024-05-08T08:53:13.376 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.376 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.377 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.377 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.377 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.269477+0000", 2024-05-08T08:53:13.377 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.377 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.377 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.377 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.377 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.269477+0000", 2024-05-08T08:53:13.377 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.377 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.377 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.377 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.377 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.269478+0000", 2024-05-08T08:53:13.378 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.8000000000000004e-07 2024-05-08T08:53:13.378 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.378 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.378 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.378 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.269483+0000", 2024-05-08T08:53:13.378 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4399999999999998e-06 2024-05-08T08:53:13.378 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.378 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.378 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.378 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.269493+0000", 2024-05-08T08:53:13.378 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0665e-05 2024-05-08T08:53:13.378 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.378 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.378 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.379 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.269504+0000", 2024-05-08T08:53:13.379 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0305999999999999e-05 2024-05-08T08:53:13.379 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.379 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.379 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.379 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.379 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.379 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:11608 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.379 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.278125+0000", 2024-05-08T08:53:13.379 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.036354232, 2024-05-08T08:53:13.379 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.037006139, 2024-05-08T08:53:13.379 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.379 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.381 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.381 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.381 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:13.381 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:13.381 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 11608 2024-05-08T08:53:13.381 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.381 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.381 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.381 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.381 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.278125+0000", 2024-05-08T08:53:13.381 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.381 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.381 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.382 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.382 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.278125+0000", 2024-05-08T08:53:13.382 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.382 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.382 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.382 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.382 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.278126+0000", 2024-05-08T08:53:13.382 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7199999999999997e-07 2024-05-08T08:53:13.382 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.382 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.382 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.382 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.278130+0000", 2024-05-08T08:53:13.382 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5070000000000002e-06 2024-05-08T08:53:13.383 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.383 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.383 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.383 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.278139+0000", 2024-05-08T08:53:13.383 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7690000000000002e-06 2024-05-08T08:53:13.383 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.383 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.383 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.383 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.278145+0000", 2024-05-08T08:53:13.383 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2949999999999999e-06 2024-05-08T08:53:13.383 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.383 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.383 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.383 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.383 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.384 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:26761 3.4 3.134fb004 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.384 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.283683+0000", 2024-05-08T08:53:13.384 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.030795914000000001, 2024-05-08T08:53:13.384 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.031510007999999999, 2024-05-08T08:53:13.384 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.384 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.384 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.384 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.384 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.384 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.384 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 26761 2024-05-08T08:53:13.385 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.385 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.385 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.385 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.385 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283683+0000", 2024-05-08T08:53:13.385 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.385 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.385 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.385 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.385 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283683+0000", 2024-05-08T08:53:13.385 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.385 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.385 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.385 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.386 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283684+0000", 2024-05-08T08:53:13.386 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8000000000000002e-07 2024-05-08T08:53:13.386 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.386 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.386 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.386 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283685+0000", 2024-05-08T08:53:13.386 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0669999999999999e-06 2024-05-08T08:53:13.386 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.386 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.386 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.386 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283688+0000", 2024-05-08T08:53:13.386 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9249999999999999e-06 2024-05-08T08:53:13.386 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.386 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.387 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.387 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283690+0000", 2024-05-08T08:53:13.387 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4700000000000001e-06 2024-05-08T08:53:13.387 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.387 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.387 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.387 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.387 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.387 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27011 3.4 3.134fb004 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.387 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.289389+0000", 2024-05-08T08:53:13.387 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.025090092000000001, 2024-05-08T08:53:13.387 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.025863259, 2024-05-08T08:53:13.387 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.387 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.387 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.388 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.388 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.388 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.388 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27011 2024-05-08T08:53:13.388 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.388 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.388 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.388 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.388 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289389+0000", 2024-05-08T08:53:13.388 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.388 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.388 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.388 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.388 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289389+0000", 2024-05-08T08:53:13.388 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.389 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.389 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.389 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.389 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289389+0000", 2024-05-08T08:53:13.389 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0200000000000003e-07 2024-05-08T08:53:13.389 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.389 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.389 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.389 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289392+0000", 2024-05-08T08:53:13.389 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1239999999999999e-06 2024-05-08T08:53:13.389 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.389 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.389 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.389 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289398+0000", 2024-05-08T08:53:13.389 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1070000000000001e-06 2024-05-08T08:53:13.390 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.390 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.390 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.390 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289403+0000", 2024-05-08T08:53:13.390 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9509999999999997e-06 2024-05-08T08:53:13.390 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.390 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.390 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.390 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.390 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.390 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27294 3.4 3.134fb004 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.390 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.302630+0000", 2024-05-08T08:53:13.391 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.011849525, 2024-05-08T08:53:13.391 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.012678465999999999, 2024-05-08T08:53:13.391 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.391 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.391 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.391 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.391 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.391 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.391 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27294 2024-05-08T08:53:13.391 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.391 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.391 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.391 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.391 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302630+0000", 2024-05-08T08:53:13.391 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.391 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.392 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.392 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.392 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302630+0000", 2024-05-08T08:53:13.392 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.392 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.392 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.392 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.392 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302630+0000", 2024-05-08T08:53:13.392 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7599999999999998e-07 2024-05-08T08:53:13.392 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.392 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.392 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.392 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302632+0000", 2024-05-08T08:53:13.392 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0700000000000001e-06 2024-05-08T08:53:13.392 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.393 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.393 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.393 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302639+0000", 2024-05-08T08:53:13.393 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9600000000000003e-06 2024-05-08T08:53:13.393 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.393 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.393 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.393 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302644+0000", 2024-05-08T08:53:13.393 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9450000000000001e-06 2024-05-08T08:53:13.393 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.393 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.393 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.393 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.393 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.393 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:11585 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.394 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.269709+0000", 2024-05-08T08:53:13.394 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.044770123000000002, 2024-05-08T08:53:13.394 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.045654568999999999, 2024-05-08T08:53:13.394 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.394 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.394 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.394 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.394 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:13.394 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:13.394 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 11585 2024-05-08T08:53:13.394 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.394 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.394 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.395 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.395 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.269709+0000", 2024-05-08T08:53:13.395 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.395 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.395 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.395 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.395 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.269709+0000", 2024-05-08T08:53:13.395 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.395 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.395 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.395 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.395 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.269710+0000", 2024-05-08T08:53:13.395 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.09e-07 2024-05-08T08:53:13.395 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.395 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.396 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.396 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.269714+0000", 2024-05-08T08:53:13.396 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2719999999999999e-06 2024-05-08T08:53:13.396 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.396 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.396 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.396 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.269725+0000", 2024-05-08T08:53:13.396 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0607e-05 2024-05-08T08:53:13.396 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.396 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.396 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.396 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.269732+0000", 2024-05-08T08:53:13.396 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0840000000000003e-06 2024-05-08T08:53:13.396 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.396 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.397 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.397 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.397 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.397 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:11609 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.397 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.278420+0000", 2024-05-08T08:53:13.397 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.036058791999999999, 2024-05-08T08:53:13.397 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.037000401000000002, 2024-05-08T08:53:13.397 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.397 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.397 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.397 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.397 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:13.397 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:13.397 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 11609 2024-05-08T08:53:13.398 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.398 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.398 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.398 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.398 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.278420+0000", 2024-05-08T08:53:13.398 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.398 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.398 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.398 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.398 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.278420+0000", 2024-05-08T08:53:13.398 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.398 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.398 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.398 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.398 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.278421+0000", 2024-05-08T08:53:13.399 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1800000000000005e-07 2024-05-08T08:53:13.399 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.399 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.399 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.399 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.278426+0000", 2024-05-08T08:53:13.399 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6580000000000001e-06 2024-05-08T08:53:13.399 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.399 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.399 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.399 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.278434+0000", 2024-05-08T08:53:13.399 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.6349999999999995e-06 2024-05-08T08:53:13.399 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.399 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.399 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.399 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.278441+0000", 2024-05-08T08:53:13.399 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0970000000000002e-06 2024-05-08T08:53:13.400 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.400 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.400 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.400 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.400 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.400 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:26762 3.4 3.134fb004 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.400 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.283693+0000", 2024-05-08T08:53:13.400 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.030785964999999998, 2024-05-08T08:53:13.400 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.031782453000000002, 2024-05-08T08:53:13.400 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.400 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.400 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.400 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.400 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.400 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.400 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 26762 2024-05-08T08:53:13.400 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.401 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.401 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.401 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.401 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283693+0000", 2024-05-08T08:53:13.401 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.401 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.401 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.401 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.401 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283693+0000", 2024-05-08T08:53:13.401 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.401 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.401 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.401 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.401 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283693+0000", 2024-05-08T08:53:13.401 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2100000000000001e-07 2024-05-08T08:53:13.401 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.401 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.401 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.402 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283695+0000", 2024-05-08T08:53:13.402 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.079e-06 2024-05-08T08:53:13.402 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.402 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.402 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.402 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283697+0000", 2024-05-08T08:53:13.402 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8109999999999999e-06 2024-05-08T08:53:13.402 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.402 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.402 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.402 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283700+0000", 2024-05-08T08:53:13.402 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4729999999999999e-06 2024-05-08T08:53:13.402 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.403 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.403 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.403 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.403 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.403 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27012 3.4 3.134fb004 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.403 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.289410+0000", 2024-05-08T08:53:13.403 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.025069681, 2024-05-08T08:53:13.403 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.026123231, 2024-05-08T08:53:13.403 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.403 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.403 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.403 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.403 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.403 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.404 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27012 2024-05-08T08:53:13.404 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.404 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.404 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.404 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.404 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289410+0000", 2024-05-08T08:53:13.404 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.404 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.404 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.404 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.404 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289410+0000", 2024-05-08T08:53:13.404 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.404 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.404 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.405 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.405 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289410+0000", 2024-05-08T08:53:13.405 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9700000000000002e-07 2024-05-08T08:53:13.405 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.405 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.405 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.405 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289412+0000", 2024-05-08T08:53:13.405 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9570000000000001e-06 2024-05-08T08:53:13.405 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.405 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.405 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.405 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289418+0000", 2024-05-08T08:53:13.405 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2129999999999996e-06 2024-05-08T08:53:13.405 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.405 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.406 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.406 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289425+0000", 2024-05-08T08:53:13.406 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9439999999999999e-06 2024-05-08T08:53:13.406 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.406 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.406 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.406 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.406 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.406 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27295 3.4 3.134fb004 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.406 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.302651+0000", 2024-05-08T08:53:13.406 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.011828600999999999, 2024-05-08T08:53:13.406 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.012938921000000001, 2024-05-08T08:53:13.406 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.406 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.406 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.407 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.407 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.407 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.407 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27295 2024-05-08T08:53:13.407 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.407 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.407 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.407 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.407 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302651+0000", 2024-05-08T08:53:13.407 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.407 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.407 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.407 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.407 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302651+0000", 2024-05-08T08:53:13.407 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.407 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.408 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.408 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.408 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302651+0000", 2024-05-08T08:53:13.408 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.77e-07 2024-05-08T08:53:13.408 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.408 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.408 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.408 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302653+0000", 2024-05-08T08:53:13.408 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0990000000000001e-06 2024-05-08T08:53:13.408 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.408 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.408 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.408 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302659+0000", 2024-05-08T08:53:13.409 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2060000000000004e-06 2024-05-08T08:53:13.409 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.409 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.409 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.409 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302665+0000", 2024-05-08T08:53:13.409 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9479999999999996e-06 2024-05-08T08:53:13.409 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.409 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.409 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.409 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.409 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.409 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:11586 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.409 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.269926+0000", 2024-05-08T08:53:13.409 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.04455315, 2024-05-08T08:53:13.409 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.045721418999999999, 2024-05-08T08:53:13.410 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.410 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.410 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.410 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.410 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:13.410 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:13.410 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 11586 2024-05-08T08:53:13.410 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.410 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.410 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.410 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.410 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.269926+0000", 2024-05-08T08:53:13.410 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.410 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.411 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.411 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.411 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.269926+0000", 2024-05-08T08:53:13.411 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.411 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.411 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.411 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.411 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.269927+0000", 2024-05-08T08:53:13.411 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3399999999999998e-07 2024-05-08T08:53:13.411 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.411 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.411 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.411 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.269931+0000", 2024-05-08T08:53:13.412 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4479999999999996e-06 2024-05-08T08:53:13.412 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.412 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.412 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.412 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.269942+0000", 2024-05-08T08:53:13.412 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1e-05 2024-05-08T08:53:13.412 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.412 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.412 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.412 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.269949+0000", 2024-05-08T08:53:13.412 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0910000000000004e-06 2024-05-08T08:53:13.412 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.412 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.412 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.412 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.413 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.413 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:11610 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.413 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.278789+0000", 2024-05-08T08:53:13.413 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.035690245000000002, 2024-05-08T08:53:13.413 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.036917942000000002, 2024-05-08T08:53:13.413 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.413 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.413 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.413 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.413 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:13.413 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:13.413 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 11610 2024-05-08T08:53:13.413 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.413 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.413 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.414 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.414 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.278789+0000", 2024-05-08T08:53:13.414 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.414 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.414 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.414 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.414 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.278789+0000", 2024-05-08T08:53:13.414 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.414 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.414 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.414 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.414 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.278790+0000", 2024-05-08T08:53:13.414 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0699999999999996e-07 2024-05-08T08:53:13.414 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.415 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.415 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.415 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.278795+0000", 2024-05-08T08:53:13.415 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8960000000000002e-06 2024-05-08T08:53:13.415 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.415 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.415 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.415 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.278807+0000", 2024-05-08T08:53:13.415 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2585000000000001e-05 2024-05-08T08:53:13.415 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.415 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.415 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.415 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.278815+0000", 2024-05-08T08:53:13.416 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2710000000000003e-06 2024-05-08T08:53:13.416 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.416 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.416 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.416 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.416 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.416 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:26763 3.4 3.134fb004 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.416 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.283703+0000", 2024-05-08T08:53:13.416 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.030776106000000001, 2024-05-08T08:53:13.416 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.032065774999999998, 2024-05-08T08:53:13.416 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.416 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.416 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.416 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.416 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.417 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.417 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 26763 2024-05-08T08:53:13.417 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.417 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.417 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.417 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.417 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283703+0000", 2024-05-08T08:53:13.417 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.417 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.417 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.417 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.417 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283703+0000", 2024-05-08T08:53:13.417 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.417 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.417 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.418 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.418 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283703+0000", 2024-05-08T08:53:13.418 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.05e-07 2024-05-08T08:53:13.418 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.418 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.418 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.418 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283705+0000", 2024-05-08T08:53:13.418 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.22e-06 2024-05-08T08:53:13.418 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.418 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.418 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.418 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283707+0000", 2024-05-08T08:53:13.418 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.994e-06 2024-05-08T08:53:13.418 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.418 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.419 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.419 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283710+0000", 2024-05-08T08:53:13.419 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.373e-06 2024-05-08T08:53:13.419 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.419 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.419 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.419 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.419 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.419 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27013 3.4 3.134fb004 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.419 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.289432+0000", 2024-05-08T08:53:13.419 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.025047421, 2024-05-08T08:53:13.419 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.026396418000000001, 2024-05-08T08:53:13.419 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.419 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.419 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.420 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.420 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.420 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.420 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27013 2024-05-08T08:53:13.420 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.420 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.420 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.420 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.420 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289432+0000", 2024-05-08T08:53:13.420 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.420 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.420 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.420 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.420 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289432+0000", 2024-05-08T08:53:13.420 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.421 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.421 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.421 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.421 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289432+0000", 2024-05-08T08:53:13.421 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7000000000000005e-07 2024-05-08T08:53:13.421 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.421 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.421 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.421 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289440+0000", 2024-05-08T08:53:13.421 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8709999999999997e-06 2024-05-08T08:53:13.421 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.421 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.421 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.421 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289447+0000", 2024-05-08T08:53:13.422 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3459999999999999e-06 2024-05-08T08:53:13.422 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.422 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.422 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.422 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289452+0000", 2024-05-08T08:53:13.422 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1769999999999997e-06 2024-05-08T08:53:13.422 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.422 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.422 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.422 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.422 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.422 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27296 3.4 3.134fb004 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.422 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.302671+0000", 2024-05-08T08:53:13.423 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.011807722, 2024-05-08T08:53:13.423 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.013214538, 2024-05-08T08:53:13.423 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.423 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.423 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.423 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.423 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.423 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.423 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27296 2024-05-08T08:53:13.423 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.423 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.423 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.423 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.423 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302671+0000", 2024-05-08T08:53:13.423 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.424 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.424 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.424 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.424 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302671+0000", 2024-05-08T08:53:13.424 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.424 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.424 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.424 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.424 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302672+0000", 2024-05-08T08:53:13.424 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5199999999999998e-07 2024-05-08T08:53:13.424 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.424 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.424 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.424 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302674+0000", 2024-05-08T08:53:13.424 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1890000000000001e-06 2024-05-08T08:53:13.425 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.425 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.425 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.425 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302680+0000", 2024-05-08T08:53:13.425 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9309999999999996e-06 2024-05-08T08:53:13.425 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.425 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.425 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.425 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302684+0000", 2024-05-08T08:53:13.425 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4179999999999999e-06 2024-05-08T08:53:13.425 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.425 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.425 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.426 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.426 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.426 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:11587 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.426 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.270144+0000", 2024-05-08T08:53:13.426 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.044335262, 2024-05-08T08:53:13.426 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.045797196999999998, 2024-05-08T08:53:13.426 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.426 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.426 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.426 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.426 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:13.426 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:13.426 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 11587 2024-05-08T08:53:13.426 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.427 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.427 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.427 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.427 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.270144+0000", 2024-05-08T08:53:13.427 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.427 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.427 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.427 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.427 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.270144+0000", 2024-05-08T08:53:13.427 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.427 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.427 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.427 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.427 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.270145+0000", 2024-05-08T08:53:13.428 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0599999999999999e-07 2024-05-08T08:53:13.428 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.428 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.428 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.428 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.270149+0000", 2024-05-08T08:53:13.428 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2270000000000004e-06 2024-05-08T08:53:13.428 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.428 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.428 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.428 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.270160+0000", 2024-05-08T08:53:13.428 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0829000000000001e-05 2024-05-08T08:53:13.428 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.428 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.428 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.428 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.270167+0000", 2024-05-08T08:53:13.429 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0430000000000002e-06 2024-05-08T08:53:13.429 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.429 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.429 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.429 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.429 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.429 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:11611 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.429 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.279132+0000", 2024-05-08T08:53:13.429 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.035347591999999997, 2024-05-08T08:53:13.429 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.03686946, 2024-05-08T08:53:13.429 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.429 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.429 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.430 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.430 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:13.430 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:13.430 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 11611 2024-05-08T08:53:13.430 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.430 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.430 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.430 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.430 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.279132+0000", 2024-05-08T08:53:13.430 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.430 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.430 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.430 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.430 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.279132+0000", 2024-05-08T08:53:13.431 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.431 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.431 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.431 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.431 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.279132+0000", 2024-05-08T08:53:13.431 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9999999999999997e-07 2024-05-08T08:53:13.431 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.431 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.431 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.431 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.279137+0000", 2024-05-08T08:53:13.431 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.792e-06 2024-05-08T08:53:13.431 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.431 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.431 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.431 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.279150+0000", 2024-05-08T08:53:13.432 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2447e-05 2024-05-08T08:53:13.432 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.432 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.432 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.432 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.279157+0000", 2024-05-08T08:53:13.432 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1600000000000001e-06 2024-05-08T08:53:13.432 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.432 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.432 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.432 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.432 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.432 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:26764 3.4 3.134fb004 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.432 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.283713+0000", 2024-05-08T08:53:13.432 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.030766074000000001, 2024-05-08T08:53:13.432 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.032342414999999999, 2024-05-08T08:53:13.433 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.433 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.433 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.433 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.433 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.433 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.433 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 26764 2024-05-08T08:53:13.433 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.433 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.433 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.433 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.433 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283713+0000", 2024-05-08T08:53:13.433 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.433 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.434 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.434 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.434 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283713+0000", 2024-05-08T08:53:13.434 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.434 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.434 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.434 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.434 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283713+0000", 2024-05-08T08:53:13.434 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0699999999999999e-07 2024-05-08T08:53:13.434 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.434 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.434 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.434 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283714+0000", 2024-05-08T08:53:13.434 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.136e-06 2024-05-08T08:53:13.435 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.435 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.435 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.435 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283717+0000", 2024-05-08T08:53:13.435 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9560000000000002e-06 2024-05-08T08:53:13.435 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.435 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.435 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.435 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283720+0000", 2024-05-08T08:53:13.435 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.5780000000000001e-06 2024-05-08T08:53:13.435 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.435 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.435 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.435 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.435 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.436 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27014 3.4 3.134fb004 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.436 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.289459+0000", 2024-05-08T08:53:13.436 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.025020499000000002, 2024-05-08T08:53:13.436 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.026651681, 2024-05-08T08:53:13.436 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.436 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.436 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.436 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.436 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.436 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.436 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27014 2024-05-08T08:53:13.436 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.436 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.436 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.436 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.437 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289459+0000", 2024-05-08T08:53:13.437 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.437 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.437 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.437 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.437 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289459+0000", 2024-05-08T08:53:13.437 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.437 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.437 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.437 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.437 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289459+0000", 2024-05-08T08:53:13.437 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.01e-07 2024-05-08T08:53:13.437 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.437 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.437 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.438 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289461+0000", 2024-05-08T08:53:13.438 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.119e-06 2024-05-08T08:53:13.438 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.438 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.438 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.438 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289467+0000", 2024-05-08T08:53:13.438 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1449999999999996e-06 2024-05-08T08:53:13.438 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.438 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.438 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.438 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289472+0000", 2024-05-08T08:53:13.438 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7709999999999997e-06 2024-05-08T08:53:13.438 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.438 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.438 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.439 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.439 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.439 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27297 3.4 3.134fb004 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.439 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.302691+0000", 2024-05-08T08:53:13.439 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.011788363, 2024-05-08T08:53:13.439 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.013482968999999999, 2024-05-08T08:53:13.439 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.439 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.439 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.439 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.439 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.439 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.439 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27297 2024-05-08T08:53:13.439 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.440 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.440 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.440 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.440 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302691+0000", 2024-05-08T08:53:13.440 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.440 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.440 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.440 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.440 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302691+0000", 2024-05-08T08:53:13.440 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.440 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.440 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.440 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.440 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302691+0000", 2024-05-08T08:53:13.440 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7800000000000002e-07 2024-05-08T08:53:13.441 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.441 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.441 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.441 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302693+0000", 2024-05-08T08:53:13.441 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2110000000000001e-06 2024-05-08T08:53:13.441 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.441 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.441 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.441 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302699+0000", 2024-05-08T08:53:13.441 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8889999999999999e-06 2024-05-08T08:53:13.441 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.441 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.441 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.441 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302704+0000", 2024-05-08T08:53:13.441 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8239999999999999e-06 2024-05-08T08:53:13.441 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.441 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.442 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.442 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.442 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.442 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27401 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.442 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.303951+0000", 2024-05-08T08:53:13.442 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.010528186, 2024-05-08T08:53:13.442 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.012280259999999999, 2024-05-08T08:53:13.442 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.442 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.442 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.442 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.442 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.442 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.442 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27401 2024-05-08T08:53:13.443 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.443 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.443 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.443 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.443 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303951+0000", 2024-05-08T08:53:13.443 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.443 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.443 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.443 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.443 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303951+0000", 2024-05-08T08:53:13.443 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.443 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.443 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.443 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.443 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303952+0000", 2024-05-08T08:53:13.444 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9699999999999995e-07 2024-05-08T08:53:13.444 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.444 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.444 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.444 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303961+0000", 2024-05-08T08:53:13.444 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.9500000000000007e-06 2024-05-08T08:53:13.444 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.444 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.444 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.444 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303969+0000", 2024-05-08T08:53:13.444 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.2649999999999996e-06 2024-05-08T08:53:13.444 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.444 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.444 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.445 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303975+0000", 2024-05-08T08:53:13.445 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5649999999999998e-06 2024-05-08T08:53:13.445 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.445 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.445 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.445 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.445 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.445 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27450 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.445 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.305528+0000", 2024-05-08T08:53:13.445 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0089508230000000001, 2024-05-08T08:53:13.445 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.010757387, 2024-05-08T08:53:13.445 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.445 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.446 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.446 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.446 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.446 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.446 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27450 2024-05-08T08:53:13.446 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.446 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.446 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.446 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.446 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.305528+0000", 2024-05-08T08:53:13.446 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.446 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.446 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.446 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.446 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.305528+0000", 2024-05-08T08:53:13.447 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.447 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.447 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.447 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.447 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.305529+0000", 2024-05-08T08:53:13.447 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6600000000000002e-07 2024-05-08T08:53:13.447 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.447 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.447 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.447 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.305531+0000", 2024-05-08T08:53:13.447 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.61e-06 2024-05-08T08:53:13.447 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.447 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.447 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.447 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.305538+0000", 2024-05-08T08:53:13.448 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1349999999999997e-06 2024-05-08T08:53:13.448 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.448 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.448 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.448 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.305542+0000", 2024-05-08T08:53:13.448 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4139999999999996e-06 2024-05-08T08:53:13.448 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.448 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.448 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.448 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.448 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.448 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27543 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.448 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.308709+0000", 2024-05-08T08:53:13.448 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0057700010000000003, 2024-05-08T08:53:13.448 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0076433450000000002, 2024-05-08T08:53:13.448 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.449 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.449 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.449 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.449 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.449 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.449 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27543 2024-05-08T08:53:13.449 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.449 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.449 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.449 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.449 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.308709+0000", 2024-05-08T08:53:13.449 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.449 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.449 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.449 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.449 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.308709+0000", 2024-05-08T08:53:13.450 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.450 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.450 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.450 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.450 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.308710+0000", 2024-05-08T08:53:13.450 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2100000000000002e-07 2024-05-08T08:53:13.450 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.450 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.450 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.450 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.308715+0000", 2024-05-08T08:53:13.450 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.767e-06 2024-05-08T08:53:13.450 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.450 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.450 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.451 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.308722+0000", 2024-05-08T08:53:13.451 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6880000000000002e-06 2024-05-08T08:53:13.451 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.451 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.451 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.451 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.308727+0000", 2024-05-08T08:53:13.451 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0660000000000003e-06 2024-05-08T08:53:13.451 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.451 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.451 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.451 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.451 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.451 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:11588 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.451 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.270379+0000", 2024-05-08T08:53:13.451 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.044100018999999997, 2024-05-08T08:53:13.451 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.046028342999999999, 2024-05-08T08:53:13.452 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.452 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.452 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.452 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.452 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:13.452 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:13.452 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 11588 2024-05-08T08:53:13.452 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.452 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.452 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.452 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.452 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.270379+0000", 2024-05-08T08:53:13.452 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.452 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.452 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.453 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.453 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.270379+0000", 2024-05-08T08:53:13.453 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.453 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.453 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.453 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.453 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.270380+0000", 2024-05-08T08:53:13.453 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.78e-07 2024-05-08T08:53:13.453 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.453 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.453 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.453 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.270385+0000", 2024-05-08T08:53:13.453 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5959999999999997e-06 2024-05-08T08:53:13.453 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.454 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.454 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.454 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.270396+0000", 2024-05-08T08:53:13.454 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1785e-05 2024-05-08T08:53:13.454 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.454 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.454 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.454 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.270404+0000", 2024-05-08T08:53:13.454 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5490000000000001e-06 2024-05-08T08:53:13.454 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.454 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.454 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.454 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.455 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.455 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:11612 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.455 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.279535+0000", 2024-05-08T08:53:13.455 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.034944586, 2024-05-08T08:53:13.455 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.036932526, 2024-05-08T08:53:13.455 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.455 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.455 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.455 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.455 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:13.455 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:13.455 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 11612 2024-05-08T08:53:13.455 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.456 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.456 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.456 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.456 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.279535+0000", 2024-05-08T08:53:13.456 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.456 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.456 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.456 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.456 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.279535+0000", 2024-05-08T08:53:13.456 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.456 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.456 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.456 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.456 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.279535+0000", 2024-05-08T08:53:13.456 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.37e-07 2024-05-08T08:53:13.457 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.457 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.457 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.457 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.279540+0000", 2024-05-08T08:53:13.457 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8550000000000001e-06 2024-05-08T08:53:13.457 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.457 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.457 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.457 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.279558+0000", 2024-05-08T08:53:13.457 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.7368e-05 2024-05-08T08:53:13.457 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.457 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.457 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.457 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.279565+0000", 2024-05-08T08:53:13.457 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.075e-06 2024-05-08T08:53:13.457 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.458 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.458 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.458 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.458 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.458 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:26765 3.4 3.134fb004 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.458 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.283723+0000", 2024-05-08T08:53:13.458 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.030756171999999998, 2024-05-08T08:53:13.458 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.032799478, 2024-05-08T08:53:13.458 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.458 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.458 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.458 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.458 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.458 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.459 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 26765 2024-05-08T08:53:13.459 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.459 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.459 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.459 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.459 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283723+0000", 2024-05-08T08:53:13.459 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.459 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.459 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.459 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.459 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283723+0000", 2024-05-08T08:53:13.459 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.459 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.459 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.459 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.460 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283723+0000", 2024-05-08T08:53:13.460 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2000000000000001e-07 2024-05-08T08:53:13.460 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.460 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.460 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.460 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283727+0000", 2024-05-08T08:53:13.460 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.245e-06 2024-05-08T08:53:13.460 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.460 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.460 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.460 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283731+0000", 2024-05-08T08:53:13.460 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.2100000000000002e-06 2024-05-08T08:53:13.460 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.460 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.461 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.461 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283733+0000", 2024-05-08T08:53:13.461 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.5110000000000002e-06 2024-05-08T08:53:13.461 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.461 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.461 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.461 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.461 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.461 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27015 3.4 3.134fb004 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.461 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.289478+0000", 2024-05-08T08:53:13.461 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.025000773, 2024-05-08T08:53:13.461 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.027102654, 2024-05-08T08:53:13.461 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.461 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.461 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.461 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.462 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.462 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.462 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27015 2024-05-08T08:53:13.462 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.462 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.462 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.462 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.462 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289478+0000", 2024-05-08T08:53:13.462 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.462 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.462 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.462 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.462 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289478+0000", 2024-05-08T08:53:13.462 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.463 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.463 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.463 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.463 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289479+0000", 2024-05-08T08:53:13.463 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.34e-07 2024-05-08T08:53:13.463 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.463 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.463 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.463 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289481+0000", 2024-05-08T08:53:13.463 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1610000000000001e-06 2024-05-08T08:53:13.463 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.463 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.463 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.463 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289487+0000", 2024-05-08T08:53:13.463 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9270000000000002e-06 2024-05-08T08:53:13.464 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.464 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.464 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.464 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289492+0000", 2024-05-08T08:53:13.464 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0799999999999996e-06 2024-05-08T08:53:13.464 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.464 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.464 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.464 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.464 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.464 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27232 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.464 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.296868+0000", 2024-05-08T08:53:13.464 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.017611104999999998, 2024-05-08T08:53:13.464 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.019771094999999999, 2024-05-08T08:53:13.464 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.465 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.465 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.465 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.465 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.465 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.465 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27232 2024-05-08T08:53:13.465 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.465 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.465 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.465 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.465 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.296868+0000", 2024-05-08T08:53:13.465 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.465 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.465 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.465 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.466 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.296868+0000", 2024-05-08T08:53:13.466 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.466 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.466 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.466 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.466 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.296869+0000", 2024-05-08T08:53:13.466 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4899999999999995e-07 2024-05-08T08:53:13.466 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.466 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.466 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.466 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.296873+0000", 2024-05-08T08:53:13.466 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1799999999999998e-06 2024-05-08T08:53:13.466 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.467 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.467 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.467 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.296881+0000", 2024-05-08T08:53:13.467 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3000000000000002e-06 2024-05-08T08:53:13.467 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.467 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.467 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.467 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.296887+0000", 2024-05-08T08:53:13.467 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2269999999999998e-06 2024-05-08T08:53:13.467 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.467 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.467 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.467 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.468 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.468 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27298 3.4 3.134fb004 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.468 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.302710+0000", 2024-05-08T08:53:13.468 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.011768905, 2024-05-08T08:53:13.468 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.013991485, 2024-05-08T08:53:13.468 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.468 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.468 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.468 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.468 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.468 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.468 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27298 2024-05-08T08:53:13.468 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.468 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.469 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.469 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.469 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302710+0000", 2024-05-08T08:53:13.469 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.469 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.469 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.469 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.469 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302710+0000", 2024-05-08T08:53:13.469 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.469 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.469 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.469 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.469 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302711+0000", 2024-05-08T08:53:13.469 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5600000000000001e-07 2024-05-08T08:53:13.470 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.470 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.470 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.470 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302713+0000", 2024-05-08T08:53:13.470 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1969999999999999e-06 2024-05-08T08:53:13.470 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.470 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.470 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.470 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302726+0000", 2024-05-08T08:53:13.470 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3456e-05 2024-05-08T08:53:13.470 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.470 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.470 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.471 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302732+0000", 2024-05-08T08:53:13.471 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1929999999999998e-06 2024-05-08T08:53:13.471 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.471 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.471 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.471 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.471 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.471 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:11617 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.471 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.309108+0000", 2024-05-08T08:53:13.471 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0053709689999999997, 2024-05-08T08:53:13.471 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0076505169999999999, 2024-05-08T08:53:13.471 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.471 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.471 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.471 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.472 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:13.472 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:13.472 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 11617 2024-05-08T08:53:13.472 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.472 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.472 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.472 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.472 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.309108+0000", 2024-05-08T08:53:13.472 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.472 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.472 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.472 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.473 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.309108+0000", 2024-05-08T08:53:13.473 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.473 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.473 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.473 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.473 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.309109+0000", 2024-05-08T08:53:13.473 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3600000000000002e-07 2024-05-08T08:53:13.473 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.473 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.473 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.473 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.309114+0000", 2024-05-08T08:53:13.473 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0749999999999997e-06 2024-05-08T08:53:13.474 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.474 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.474 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.474 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.309123+0000", 2024-05-08T08:53:13.474 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1889999999999996e-06 2024-05-08T08:53:13.474 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.474 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.474 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.474 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.309130+0000", 2024-05-08T08:53:13.474 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5910000000000001e-06 2024-05-08T08:53:13.474 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.474 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.474 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.474 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.475 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.475 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:11589 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.475 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.270586+0000", 2024-05-08T08:53:13.475 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.043893303000000002, 2024-05-08T08:53:13.475 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.046233110000000001, 2024-05-08T08:53:13.475 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.475 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.475 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.475 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.475 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:13.475 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:13.475 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 11589 2024-05-08T08:53:13.475 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.475 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.476 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.476 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.476 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.270586+0000", 2024-05-08T08:53:13.476 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.476 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.476 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.476 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.476 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.270586+0000", 2024-05-08T08:53:13.476 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.476 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.476 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.476 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.476 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.270587+0000", 2024-05-08T08:53:13.476 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.2500000000000004e-07 2024-05-08T08:53:13.476 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.477 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.477 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.477 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.270591+0000", 2024-05-08T08:53:13.477 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5499999999999996e-06 2024-05-08T08:53:13.477 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.477 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.477 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.477 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.270602+0000", 2024-05-08T08:53:13.477 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0903e-05 2024-05-08T08:53:13.477 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.477 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.477 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.477 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.270610+0000", 2024-05-08T08:53:13.477 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4120000000000004e-06 2024-05-08T08:53:13.477 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.477 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.478 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.478 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.478 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.478 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:26766 3.4 3.134fb004 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.478 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.283736+0000", 2024-05-08T08:53:13.478 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.030742753000000001, 2024-05-08T08:53:13.478 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.033137137999999997, 2024-05-08T08:53:13.478 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.478 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.478 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.478 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.478 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.478 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.478 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 26766 2024-05-08T08:53:13.479 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.479 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.479 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.479 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.479 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283736+0000", 2024-05-08T08:53:13.479 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.479 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.479 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.479 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.479 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283736+0000", 2024-05-08T08:53:13.479 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.479 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.479 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.479 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.479 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283737+0000", 2024-05-08T08:53:13.480 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1400000000000001e-07 2024-05-08T08:53:13.480 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.480 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.480 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.480 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283738+0000", 2024-05-08T08:53:13.480 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.125e-06 2024-05-08T08:53:13.480 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.480 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.480 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.480 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283741+0000", 2024-05-08T08:53:13.480 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0910000000000001e-06 2024-05-08T08:53:13.480 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.480 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.480 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.481 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283743+0000", 2024-05-08T08:53:13.481 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4770000000000002e-06 2024-05-08T08:53:13.481 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.481 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.481 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.481 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.481 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.481 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27016 3.4 3.134fb004 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.481 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.289499+0000", 2024-05-08T08:53:13.481 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.024980441999999999, 2024-05-08T08:53:13.481 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.027432435000000002, 2024-05-08T08:53:13.481 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.481 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.481 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.481 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.482 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.482 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.482 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27016 2024-05-08T08:53:13.482 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.482 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.482 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.482 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.482 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289499+0000", 2024-05-08T08:53:13.482 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.482 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.482 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.482 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.482 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289499+0000", 2024-05-08T08:53:13.482 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.482 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.483 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.483 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.483 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289499+0000", 2024-05-08T08:53:13.483 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4099999999999999e-07 2024-05-08T08:53:13.483 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.483 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.483 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.483 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289501+0000", 2024-05-08T08:53:13.483 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1009999999999999e-06 2024-05-08T08:53:13.483 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.483 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.483 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.483 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289508+0000", 2024-05-08T08:53:13.483 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0500000000000003e-06 2024-05-08T08:53:13.484 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.484 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.484 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.484 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289513+0000", 2024-05-08T08:53:13.484 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8890000000000001e-06 2024-05-08T08:53:13.484 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.484 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.484 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.484 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.484 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.484 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:11616 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.484 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.297830+0000", 2024-05-08T08:53:13.484 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.016649595999999999, 2024-05-08T08:53:13.484 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.019160817, 2024-05-08T08:53:13.484 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.485 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.485 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.485 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.485 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:13.485 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:13.485 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 11616 2024-05-08T08:53:13.485 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.485 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.485 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.485 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.485 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.297830+0000", 2024-05-08T08:53:13.485 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.485 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.485 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.486 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.486 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.297830+0000", 2024-05-08T08:53:13.486 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.486 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.486 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.486 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.486 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.297830+0000", 2024-05-08T08:53:13.486 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.2799999999999995e-07 2024-05-08T08:53:13.486 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.486 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.486 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.486 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.297836+0000", 2024-05-08T08:53:13.486 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.451e-06 2024-05-08T08:53:13.486 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.486 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.487 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.487 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.297845+0000", 2024-05-08T08:53:13.487 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0170000000000002e-06 2024-05-08T08:53:13.487 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.487 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.487 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.487 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.297852+0000", 2024-05-08T08:53:13.487 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9700000000000002e-06 2024-05-08T08:53:13.487 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.487 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.487 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.487 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.487 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.487 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27299 3.4 3.134fb004 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.487 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.302740+0000", 2024-05-08T08:53:13.488 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.011738843000000001, 2024-05-08T08:53:13.488 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.014307554, 2024-05-08T08:53:13.488 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.488 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.488 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.488 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.488 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.488 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.488 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27299 2024-05-08T08:53:13.488 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.488 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.488 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.488 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.488 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302740+0000", 2024-05-08T08:53:13.488 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.489 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.489 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.489 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.489 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302740+0000", 2024-05-08T08:53:13.489 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.489 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.489 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.489 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.489 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302741+0000", 2024-05-08T08:53:13.489 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9100000000000004e-07 2024-05-08T08:53:13.489 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.489 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.489 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.490 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302744+0000", 2024-05-08T08:53:13.490 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6460000000000002e-06 2024-05-08T08:53:13.490 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.490 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.490 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.490 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302750+0000", 2024-05-08T08:53:13.490 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5760000000000002e-06 2024-05-08T08:53:13.490 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.490 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.490 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.490 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302755+0000", 2024-05-08T08:53:13.490 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9779999999999996e-06 2024-05-08T08:53:13.490 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.490 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.491 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.491 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.491 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.491 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27403 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.491 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.304008+0000", 2024-05-08T08:53:13.491 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.010471667, 2024-05-08T08:53:13.491 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.013093051, 2024-05-08T08:53:13.491 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.491 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.491 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.491 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.491 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.491 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.491 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27403 2024-05-08T08:53:13.491 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.492 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.492 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.492 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.492 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.304008+0000", 2024-05-08T08:53:13.492 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.492 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.492 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.492 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.492 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.304008+0000", 2024-05-08T08:53:13.492 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.492 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.492 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.492 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.492 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.304008+0000", 2024-05-08T08:53:13.492 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4999999999999998e-07 2024-05-08T08:53:13.493 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.493 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.493 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.493 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.304010+0000", 2024-05-08T08:53:13.493 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.559e-06 2024-05-08T08:53:13.493 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.493 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.493 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.493 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.304017+0000", 2024-05-08T08:53:13.493 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1600000000000003e-06 2024-05-08T08:53:13.493 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.493 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.493 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.493 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.304023+0000", 2024-05-08T08:53:13.493 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6139999999999997e-06 2024-05-08T08:53:13.494 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.494 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.494 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.494 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.494 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.494 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27544 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.494 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.309012+0000", 2024-05-08T08:53:13.494 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0054666960000000001, 2024-05-08T08:53:13.494 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0081448849999999993, 2024-05-08T08:53:13.494 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.494 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.494 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.494 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.495 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.495 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.495 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27544 2024-05-08T08:53:13.495 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.495 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.495 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.495 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.495 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.309012+0000", 2024-05-08T08:53:13.495 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.495 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.495 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.495 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.495 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.309012+0000", 2024-05-08T08:53:13.495 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.495 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.496 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.496 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.496 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.309013+0000", 2024-05-08T08:53:13.496 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2900000000000003e-07 2024-05-08T08:53:13.496 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.496 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.496 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.496 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.309018+0000", 2024-05-08T08:53:13.496 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.476e-06 2024-05-08T08:53:13.496 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.496 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.496 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.496 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.310381+0000", 2024-05-08T08:53:13.496 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001363544 2024-05-08T08:53:13.496 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.497 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.497 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.497 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.310389+0000", 2024-05-08T08:53:13.497 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.447e-06 2024-05-08T08:53:13.497 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.497 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.497 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.497 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.497 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.497 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:11590 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.497 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.271035+0000", 2024-05-08T08:53:13.497 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.043443949000000003, 2024-05-08T08:53:13.497 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.046182444000000003, 2024-05-08T08:53:13.497 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.498 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.498 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.498 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.498 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:13.498 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:13.498 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 11590 2024-05-08T08:53:13.498 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.498 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.498 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.498 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.498 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.271035+0000", 2024-05-08T08:53:13.498 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.498 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.498 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.499 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.499 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.271035+0000", 2024-05-08T08:53:13.499 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.499 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.499 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.499 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.499 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.271036+0000", 2024-05-08T08:53:13.499 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0100000000000003e-07 2024-05-08T08:53:13.499 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.499 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.499 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.499 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.271041+0000", 2024-05-08T08:53:13.499 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.028e-06 2024-05-08T08:53:13.499 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.499 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.500 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.500 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.271053+0000", 2024-05-08T08:53:13.500 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2081e-05 2024-05-08T08:53:13.500 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.500 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.500 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.500 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.271061+0000", 2024-05-08T08:53:13.500 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.6020000000000002e-06 2024-05-08T08:53:13.500 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.500 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.500 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.500 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.500 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.500 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:26767 3.4 3.134fb004 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.500 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.283747+0000", 2024-05-08T08:53:13.500 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.030732684, 2024-05-08T08:53:13.501 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.033525483000000002, 2024-05-08T08:53:13.501 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.501 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.501 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.501 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.501 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.501 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.501 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 26767 2024-05-08T08:53:13.501 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.501 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.501 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.501 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.501 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283747+0000", 2024-05-08T08:53:13.502 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.502 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.502 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.502 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.502 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283747+0000", 2024-05-08T08:53:13.502 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.502 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.502 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.502 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.502 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283747+0000", 2024-05-08T08:53:13.502 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2600000000000001e-07 2024-05-08T08:53:13.502 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.502 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.502 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.502 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283748+0000", 2024-05-08T08:53:13.503 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.141e-06 2024-05-08T08:53:13.503 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.503 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.503 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.503 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283751+0000", 2024-05-08T08:53:13.503 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9699999999999999e-06 2024-05-08T08:53:13.503 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.503 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.503 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.503 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283754+0000", 2024-05-08T08:53:13.503 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9739999999999998e-06 2024-05-08T08:53:13.503 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.503 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.503 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.503 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.504 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.504 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27017 3.4 3.134fb004 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.504 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.289520+0000", 2024-05-08T08:53:13.504 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.024959391000000001, 2024-05-08T08:53:13.504 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.027809489, 2024-05-08T08:53:13.504 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.504 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.504 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.504 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.504 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.504 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.504 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27017 2024-05-08T08:53:13.504 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.504 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.504 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.505 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.505 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289520+0000", 2024-05-08T08:53:13.505 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.505 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.505 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.505 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.505 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289520+0000", 2024-05-08T08:53:13.505 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.505 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.505 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.505 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.505 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289520+0000", 2024-05-08T08:53:13.505 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2599999999999998e-07 2024-05-08T08:53:13.505 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.505 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.506 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.506 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289522+0000", 2024-05-08T08:53:13.506 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1639999999999999e-06 2024-05-08T08:53:13.506 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.506 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.506 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.506 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289529+0000", 2024-05-08T08:53:13.506 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.139e-06 2024-05-08T08:53:13.506 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.506 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.506 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.506 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289533+0000", 2024-05-08T08:53:13.506 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5700000000000003e-06 2024-05-08T08:53:13.506 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.506 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.507 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.507 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.507 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.507 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27308 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.507 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.302761+0000", 2024-05-08T08:53:13.507 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.011717702, 2024-05-08T08:53:13.507 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.014623284, 2024-05-08T08:53:13.507 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.507 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.507 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.507 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.507 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.507 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.507 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27308 2024-05-08T08:53:13.507 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.507 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.508 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.508 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.508 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302761+0000", 2024-05-08T08:53:13.508 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.508 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.508 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.508 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.508 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302761+0000", 2024-05-08T08:53:13.508 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.508 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.508 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.508 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.508 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302762+0000", 2024-05-08T08:53:13.508 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4900000000000001e-07 2024-05-08T08:53:13.509 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.509 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.509 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.509 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302765+0000", 2024-05-08T08:53:13.509 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.819e-06 2024-05-08T08:53:13.509 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.509 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.509 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.509 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302770+0000", 2024-05-08T08:53:13.509 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7409999999999998e-06 2024-05-08T08:53:13.509 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.509 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.509 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.509 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302775+0000", 2024-05-08T08:53:13.509 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7859999999999996e-06 2024-05-08T08:53:13.510 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.510 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.510 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.510 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.510 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.510 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27453 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.510 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.305600+0000", 2024-05-08T08:53:13.510 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0088787959999999996, 2024-05-08T08:53:13.510 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.011846160999999999, 2024-05-08T08:53:13.510 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.510 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.510 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.510 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.510 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.510 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.511 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27453 2024-05-08T08:53:13.511 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.511 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.511 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.511 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.511 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.305600+0000", 2024-05-08T08:53:13.511 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.511 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.511 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.511 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.511 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.305600+0000", 2024-05-08T08:53:13.511 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.511 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.511 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.511 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.511 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.305601+0000", 2024-05-08T08:53:13.512 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7899999999999999e-07 2024-05-08T08:53:13.512 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.512 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.512 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.512 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.305604+0000", 2024-05-08T08:53:13.512 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1020000000000001e-06 2024-05-08T08:53:13.512 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.512 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.512 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.512 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.305612+0000", 2024-05-08T08:53:13.512 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5560000000000002e-06 2024-05-08T08:53:13.512 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.512 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.512 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.512 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.305616+0000", 2024-05-08T08:53:13.513 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8760000000000004e-06 2024-05-08T08:53:13.513 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.513 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.513 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.513 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.513 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.513 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27546 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.513 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.310403+0000", 2024-05-08T08:53:13.513 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0040765330000000002, 2024-05-08T08:53:13.513 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.007101558, 2024-05-08T08:53:13.513 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.513 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.513 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.513 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.514 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.514 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.514 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27546 2024-05-08T08:53:13.514 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.514 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.514 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.514 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.514 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.310403+0000", 2024-05-08T08:53:13.514 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.514 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.514 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.514 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.514 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.310403+0000", 2024-05-08T08:53:13.514 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.514 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.515 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.515 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.515 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.310403+0000", 2024-05-08T08:53:13.515 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4499999999999996e-07 2024-05-08T08:53:13.515 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.515 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.515 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.515 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.310408+0000", 2024-05-08T08:53:13.515 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5639999999999998e-06 2024-05-08T08:53:13.515 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.515 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.515 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.515 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.310417+0000", 2024-05-08T08:53:13.515 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.5860000000000005e-06 2024-05-08T08:53:13.515 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.516 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.516 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.516 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.310422+0000", 2024-05-08T08:53:13.516 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7939999999999999e-06 2024-05-08T08:53:13.516 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.516 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.516 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.516 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.516 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.516 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:11591 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.516 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.271176+0000", 2024-05-08T08:53:13.516 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.043303493999999998, 2024-05-08T08:53:13.516 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.046389443000000002, 2024-05-08T08:53:13.516 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.516 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.517 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.517 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.517 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:13.517 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:13.517 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 11591 2024-05-08T08:53:13.517 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.517 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.517 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.517 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.517 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.271176+0000", 2024-05-08T08:53:13.517 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.517 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.517 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.517 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.517 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.271176+0000", 2024-05-08T08:53:13.517 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.517 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.518 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.518 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.518 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.271177+0000", 2024-05-08T08:53:13.518 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4200000000000005e-07 2024-05-08T08:53:13.518 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.518 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.518 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.518 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.271181+0000", 2024-05-08T08:53:13.518 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.673e-06 2024-05-08T08:53:13.518 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.518 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.518 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.518 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.271193+0000", 2024-05-08T08:53:13.518 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.173e-05 2024-05-08T08:53:13.519 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.519 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.519 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.519 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.271200+0000", 2024-05-08T08:53:13.519 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4560000000000003e-06 2024-05-08T08:53:13.519 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.519 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.519 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.519 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.519 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.519 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:26768 3.4 3.134fb004 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.519 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.283757+0000", 2024-05-08T08:53:13.519 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.030722055000000002, 2024-05-08T08:53:13.519 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.033865972000000001, 2024-05-08T08:53:13.519 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.519 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.520 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.520 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.520 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.520 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.520 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 26768 2024-05-08T08:53:13.520 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.520 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.520 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.520 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.520 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283757+0000", 2024-05-08T08:53:13.520 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.520 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.520 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.520 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.520 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283757+0000", 2024-05-08T08:53:13.521 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.521 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.521 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.521 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.521 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283757+0000", 2024-05-08T08:53:13.521 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0599999999999999e-07 2024-05-08T08:53:13.521 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.521 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.521 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.521 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283758+0000", 2024-05-08T08:53:13.521 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.114e-06 2024-05-08T08:53:13.521 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.521 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.521 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.521 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283761+0000", 2024-05-08T08:53:13.522 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9560000000000002e-06 2024-05-08T08:53:13.522 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.522 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.522 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.522 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283764+0000", 2024-05-08T08:53:13.522 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.5040000000000001e-06 2024-05-08T08:53:13.522 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.522 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.522 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.522 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.522 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.522 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27018 3.4 3.134fb004 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.522 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.289540+0000", 2024-05-08T08:53:13.522 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.024939144999999999, 2024-05-08T08:53:13.523 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.028141943999999999, 2024-05-08T08:53:13.523 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.523 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.523 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.523 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.523 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.523 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.523 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27018 2024-05-08T08:53:13.523 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.523 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.523 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.523 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.523 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289540+0000", 2024-05-08T08:53:13.523 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.523 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.523 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.523 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.524 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289540+0000", 2024-05-08T08:53:13.524 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.524 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.524 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.524 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.524 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289540+0000", 2024-05-08T08:53:13.524 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8500000000000002e-07 2024-05-08T08:53:13.524 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.524 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.524 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.524 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289543+0000", 2024-05-08T08:53:13.524 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2790000000000001e-06 2024-05-08T08:53:13.524 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.524 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.524 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.525 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289549+0000", 2024-05-08T08:53:13.525 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0480000000000004e-06 2024-05-08T08:53:13.525 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.525 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.525 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.525 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289554+0000", 2024-05-08T08:53:13.525 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0320000000000003e-06 2024-05-08T08:53:13.525 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.525 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.525 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.525 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.525 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.525 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27233 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.525 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.302012+0000", 2024-05-08T08:53:13.525 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.012467354, 2024-05-08T08:53:13.526 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.015723797000000001, 2024-05-08T08:53:13.526 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.526 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.526 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.526 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.526 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.526 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.526 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27233 2024-05-08T08:53:13.526 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.526 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.526 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.526 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.526 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302012+0000", 2024-05-08T08:53:13.526 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.526 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.526 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.527 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.527 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302012+0000", 2024-05-08T08:53:13.527 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.527 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.527 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.527 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.527 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302013+0000", 2024-05-08T08:53:13.527 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5600000000000004e-07 2024-05-08T08:53:13.527 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.527 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.527 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.527 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302017+0000", 2024-05-08T08:53:13.527 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5639999999999998e-06 2024-05-08T08:53:13.528 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.528 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.528 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.528 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302028+0000", 2024-05-08T08:53:13.528 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0393e-05 2024-05-08T08:53:13.528 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.528 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.528 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.528 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302036+0000", 2024-05-08T08:53:13.528 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7080000000000003e-06 2024-05-08T08:53:13.528 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.528 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.528 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.528 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.528 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.528 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27312 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.529 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.302782+0000", 2024-05-08T08:53:13.529 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.011697172, 2024-05-08T08:53:13.529 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.015010584, 2024-05-08T08:53:13.529 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.529 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.529 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.529 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.529 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.529 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.529 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27312 2024-05-08T08:53:13.529 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.529 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.529 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.529 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.529 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302782+0000", 2024-05-08T08:53:13.530 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.530 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.530 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.530 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.530 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302782+0000", 2024-05-08T08:53:13.530 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.530 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.530 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.530 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.530 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302782+0000", 2024-05-08T08:53:13.530 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5900000000000003e-07 2024-05-08T08:53:13.530 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.530 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.530 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.530 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302785+0000", 2024-05-08T08:53:13.531 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9059999999999998e-06 2024-05-08T08:53:13.531 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.531 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.531 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.531 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302797+0000", 2024-05-08T08:53:13.531 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2177e-05 2024-05-08T08:53:13.531 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.531 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.531 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.531 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302803+0000", 2024-05-08T08:53:13.531 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4060000000000004e-06 2024-05-08T08:53:13.531 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.531 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.531 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.531 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.531 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.532 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27548 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.532 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.310431+0000", 2024-05-08T08:53:13.532 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0040479610000000001, 2024-05-08T08:53:13.532 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0074143519999999999, 2024-05-08T08:53:13.532 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.532 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.532 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.532 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.532 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.532 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.532 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27548 2024-05-08T08:53:13.532 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.532 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.532 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.532 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.532 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.310431+0000", 2024-05-08T08:53:13.533 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.533 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.533 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.533 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.533 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.310431+0000", 2024-05-08T08:53:13.533 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.533 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.533 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.533 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.533 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.310432+0000", 2024-05-08T08:53:13.533 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0900000000000002e-07 2024-05-08T08:53:13.533 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.533 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.533 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.534 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.310435+0000", 2024-05-08T08:53:13.534 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.954e-06 2024-05-08T08:53:13.534 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.534 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.534 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.534 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.310441+0000", 2024-05-08T08:53:13.534 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5830000000000003e-06 2024-05-08T08:53:13.534 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.534 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.534 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.534 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.310447+0000", 2024-05-08T08:53:13.534 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4099999999999999e-06 2024-05-08T08:53:13.534 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.535 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.535 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.535 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.535 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.535 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:11592 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.535 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.271332+0000", 2024-05-08T08:53:13.535 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.043147120999999997, 2024-05-08T08:53:13.535 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.046570406000000002, 2024-05-08T08:53:13.535 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.535 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.535 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.535 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.535 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:13.535 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:13.535 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 11592 2024-05-08T08:53:13.536 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.536 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.536 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.536 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.536 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.271332+0000", 2024-05-08T08:53:13.536 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.536 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.536 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.536 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.536 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.271332+0000", 2024-05-08T08:53:13.536 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.536 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.536 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.536 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.536 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.271333+0000", 2024-05-08T08:53:13.536 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3499999999999995e-07 2024-05-08T08:53:13.537 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.537 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.537 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.537 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.271337+0000", 2024-05-08T08:53:13.537 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4730000000000002e-06 2024-05-08T08:53:13.537 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.537 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.537 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.537 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.271349+0000", 2024-05-08T08:53:13.537 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1413000000000001e-05 2024-05-08T08:53:13.537 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.537 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.537 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.537 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.271356+0000", 2024-05-08T08:53:13.537 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3980000000000002e-06 2024-05-08T08:53:13.537 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.538 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.538 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.538 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.538 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.538 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:26769 3.4 3.134fb004 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.538 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.283768+0000", 2024-05-08T08:53:13.538 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.030710774999999999, 2024-05-08T08:53:13.538 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.034192727999999999, 2024-05-08T08:53:13.538 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.538 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.538 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.538 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.538 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.538 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.538 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 26769 2024-05-08T08:53:13.539 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.539 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.539 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.539 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.539 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283768+0000", 2024-05-08T08:53:13.539 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.539 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.539 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.539 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.539 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283768+0000", 2024-05-08T08:53:13.539 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.539 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.539 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.539 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.539 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283769+0000", 2024-05-08T08:53:13.539 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0800000000000001e-07 2024-05-08T08:53:13.540 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.540 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.540 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.540 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283770+0000", 2024-05-08T08:53:13.540 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.141e-06 2024-05-08T08:53:13.540 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.540 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.540 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.540 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283773+0000", 2024-05-08T08:53:13.540 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0400000000000001e-06 2024-05-08T08:53:13.540 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.540 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.540 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.540 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283775+0000", 2024-05-08T08:53:13.540 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4509999999999999e-06 2024-05-08T08:53:13.540 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.541 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.541 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.541 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.541 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.541 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27019 3.4 3.134fb004 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.541 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.289560+0000", 2024-05-08T08:53:13.541 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.024919081999999999, 2024-05-08T08:53:13.541 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.028464990999999999, 2024-05-08T08:53:13.541 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.541 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.541 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.541 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.541 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.541 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.541 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27019 2024-05-08T08:53:13.541 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.542 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.542 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.542 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.542 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289560+0000", 2024-05-08T08:53:13.542 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.542 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.542 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.542 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.542 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289560+0000", 2024-05-08T08:53:13.542 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.542 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.542 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.542 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.542 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289560+0000", 2024-05-08T08:53:13.542 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:53:13.542 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.543 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.543 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.543 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289562+0000", 2024-05-08T08:53:13.543 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9740000000000001e-06 2024-05-08T08:53:13.543 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.543 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.543 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.543 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289569+0000", 2024-05-08T08:53:13.543 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2339999999999999e-06 2024-05-08T08:53:13.543 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.543 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.543 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.543 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289573+0000", 2024-05-08T08:53:13.543 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7960000000000003e-06 2024-05-08T08:53:13.543 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.544 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.544 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.544 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.544 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.544 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27234 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.544 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.302050+0000", 2024-05-08T08:53:13.544 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.012429677, 2024-05-08T08:53:13.544 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.016030685999999999, 2024-05-08T08:53:13.544 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.544 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.544 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.544 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.544 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.544 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.544 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27234 2024-05-08T08:53:13.545 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.545 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.545 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.545 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.545 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302050+0000", 2024-05-08T08:53:13.545 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.545 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.545 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.545 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.545 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302050+0000", 2024-05-08T08:53:13.545 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.545 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.545 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.545 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.545 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302050+0000", 2024-05-08T08:53:13.545 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8400000000000005e-07 2024-05-08T08:53:13.545 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.546 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.546 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.546 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302053+0000", 2024-05-08T08:53:13.546 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.27e-06 2024-05-08T08:53:13.546 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.546 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.546 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.546 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302069+0000", 2024-05-08T08:53:13.546 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6110999999999999e-05 2024-05-08T08:53:13.546 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.546 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.546 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.546 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302076+0000", 2024-05-08T08:53:13.546 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3990000000000001e-06 2024-05-08T08:53:13.546 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.547 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.547 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.547 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.547 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.547 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27315 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.547 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.302811+0000", 2024-05-08T08:53:13.547 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.011668236, 2024-05-08T08:53:13.547 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.015323348000000001, 2024-05-08T08:53:13.547 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.547 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.547 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.547 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.547 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.547 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.547 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27315 2024-05-08T08:53:13.548 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.548 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.548 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.548 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.548 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302811+0000", 2024-05-08T08:53:13.548 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.548 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.548 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.548 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.548 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302811+0000", 2024-05-08T08:53:13.548 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.548 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.548 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.548 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.548 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302811+0000", 2024-05-08T08:53:13.548 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9400000000000001e-07 2024-05-08T08:53:13.549 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.549 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.549 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.549 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302814+0000", 2024-05-08T08:53:13.549 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7769999999999998e-06 2024-05-08T08:53:13.549 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.549 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.549 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.549 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302820+0000", 2024-05-08T08:53:13.549 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3090000000000001e-06 2024-05-08T08:53:13.549 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.549 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.549 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.549 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302825+0000", 2024-05-08T08:53:13.549 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8400000000000002e-06 2024-05-08T08:53:13.549 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.550 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.550 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.550 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.550 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.550 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27406 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.550 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.304075+0000", 2024-05-08T08:53:13.550 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.010404518, 2024-05-08T08:53:13.550 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.014130148, 2024-05-08T08:53:13.550 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.550 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.550 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.550 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.550 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.550 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.550 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27406 2024-05-08T08:53:13.551 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.551 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.551 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.551 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.551 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.304075+0000", 2024-05-08T08:53:13.551 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.551 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.551 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.551 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.551 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.304075+0000", 2024-05-08T08:53:13.551 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.551 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.551 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.551 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.551 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.304075+0000", 2024-05-08T08:53:13.551 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6199999999999999e-07 2024-05-08T08:53:13.552 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.552 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.552 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.552 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.304078+0000", 2024-05-08T08:53:13.552 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.5849999999999998e-06 2024-05-08T08:53:13.552 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.552 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.552 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.552 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.304084+0000", 2024-05-08T08:53:13.552 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1859999999999997e-06 2024-05-08T08:53:13.552 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.552 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.552 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.552 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.304088+0000", 2024-05-08T08:53:13.552 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.437e-06 2024-05-08T08:53:13.553 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.553 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.553 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.553 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.553 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.553 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27584 3.4 3.134fb004 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.553 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.310454+0000", 2024-05-08T08:53:13.553 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0040256210000000001, 2024-05-08T08:53:13.553 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0078048930000000002, 2024-05-08T08:53:13.553 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.553 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.553 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.553 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.553 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.553 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.554 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27584 2024-05-08T08:53:13.554 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.554 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.554 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.554 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.554 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.310454+0000", 2024-05-08T08:53:13.554 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.554 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.554 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.554 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.554 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.310454+0000", 2024-05-08T08:53:13.554 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.554 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.554 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.555 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.555 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.310454+0000", 2024-05-08T08:53:13.555 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9999999999999998e-07 2024-05-08T08:53:13.555 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.555 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.555 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.555 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.310456+0000", 2024-05-08T08:53:13.555 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3810000000000002e-06 2024-05-08T08:53:13.555 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.555 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.555 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.555 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.310462+0000", 2024-05-08T08:53:13.555 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8189999999999997e-06 2024-05-08T08:53:13.555 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.555 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.556 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.556 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.310467+0000", 2024-05-08T08:53:13.556 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8720000000000001e-06 2024-05-08T08:53:13.556 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.556 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.556 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.556 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.556 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.556 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:11593 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.556 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.271456+0000", 2024-05-08T08:53:13.556 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.043022811000000001, 2024-05-08T08:53:13.556 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.046857087999999998, 2024-05-08T08:53:13.556 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.556 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.556 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.556 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.557 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:13.557 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:13.557 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 11593 2024-05-08T08:53:13.557 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.557 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.557 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.557 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.557 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.271456+0000", 2024-05-08T08:53:13.557 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.557 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.557 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.557 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.557 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.271456+0000", 2024-05-08T08:53:13.557 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.557 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.557 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.558 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.558 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.271457+0000", 2024-05-08T08:53:13.558 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.6000000000000002e-07 2024-05-08T08:53:13.558 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.558 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.558 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.558 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.271462+0000", 2024-05-08T08:53:13.558 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7249999999999997e-06 2024-05-08T08:53:13.558 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.558 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.558 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.558 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.271473+0000", 2024-05-08T08:53:13.558 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0542e-05 2024-05-08T08:53:13.558 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.558 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.559 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.559 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.271480+0000", 2024-05-08T08:53:13.559 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3679999999999996e-06 2024-05-08T08:53:13.559 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.559 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.559 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.559 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.559 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.559 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:26770 3.4 3.134fb004 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.559 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.283779+0000", 2024-05-08T08:53:13.559 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.030700385, 2024-05-08T08:53:13.559 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.034593345999999997, 2024-05-08T08:53:13.559 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.559 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.559 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.559 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.560 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.560 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.560 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 26770 2024-05-08T08:53:13.560 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.560 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.560 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.560 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.560 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283779+0000", 2024-05-08T08:53:13.560 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.560 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.560 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.560 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.560 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283779+0000", 2024-05-08T08:53:13.560 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.560 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.561 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.561 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.561 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283779+0000", 2024-05-08T08:53:13.561 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2000000000000001e-07 2024-05-08T08:53:13.561 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.561 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.561 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.561 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283780+0000", 2024-05-08T08:53:13.561 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2589999999999999e-06 2024-05-08T08:53:13.561 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.561 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.561 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.561 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283783+0000", 2024-05-08T08:53:13.561 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9730000000000002e-06 2024-05-08T08:53:13.561 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.561 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.562 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.562 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283786+0000", 2024-05-08T08:53:13.562 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3470000000000001e-06 2024-05-08T08:53:13.562 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.562 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.562 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.562 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.562 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.562 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27020 3.4 3.134fb004 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.562 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.289580+0000", 2024-05-08T08:53:13.562 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.024898983, 2024-05-08T08:53:13.562 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.028846587999999999, 2024-05-08T08:53:13.562 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.562 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.563 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.563 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.563 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.563 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.563 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27020 2024-05-08T08:53:13.563 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.563 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.563 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.563 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.563 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289580+0000", 2024-05-08T08:53:13.563 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.563 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.563 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.563 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.563 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289580+0000", 2024-05-08T08:53:13.563 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.564 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.564 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.564 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.564 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289581+0000", 2024-05-08T08:53:13.564 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5699999999999998e-07 2024-05-08T08:53:13.564 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.564 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.564 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.564 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289583+0000", 2024-05-08T08:53:13.564 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0209999999999998e-06 2024-05-08T08:53:13.564 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.564 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.564 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.565 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289590+0000", 2024-05-08T08:53:13.565 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1230000000000002e-06 2024-05-08T08:53:13.565 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.565 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.565 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.565 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289595+0000", 2024-05-08T08:53:13.565 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8910000000000003e-06 2024-05-08T08:53:13.565 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.565 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.565 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.565 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.565 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.565 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27235 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.565 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.302085+0000", 2024-05-08T08:53:13.565 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.012394066, 2024-05-08T08:53:13.565 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.016399720999999999, 2024-05-08T08:53:13.566 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.566 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.566 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.566 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.566 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.566 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.566 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27235 2024-05-08T08:53:13.566 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.566 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.566 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.566 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.566 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302085+0000", 2024-05-08T08:53:13.566 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.566 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.566 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.567 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.567 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302085+0000", 2024-05-08T08:53:13.567 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.567 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.567 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.567 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.567 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302086+0000", 2024-05-08T08:53:13.567 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2799999999999996e-07 2024-05-08T08:53:13.567 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.567 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.567 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.567 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302089+0000", 2024-05-08T08:53:13.567 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.2729999999999998e-06 2024-05-08T08:53:13.567 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.567 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.568 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.568 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302096+0000", 2024-05-08T08:53:13.568 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9670000000000004e-06 2024-05-08T08:53:13.568 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.568 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.568 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.568 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302101+0000", 2024-05-08T08:53:13.568 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4149999999999998e-06 2024-05-08T08:53:13.568 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.568 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.568 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.568 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.569 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.569 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27321 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.569 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.302832+0000", 2024-05-08T08:53:13.569 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.011647075999999999, 2024-05-08T08:53:13.569 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.015709460000000001, 2024-05-08T08:53:13.569 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.569 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.569 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.569 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.569 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.569 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.569 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27321 2024-05-08T08:53:13.569 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.569 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.570 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.570 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.570 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302832+0000", 2024-05-08T08:53:13.570 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.570 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.570 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.570 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.570 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302832+0000", 2024-05-08T08:53:13.570 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.570 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.570 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.570 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.571 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302833+0000", 2024-05-08T08:53:13.571 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0400000000000002e-07 2024-05-08T08:53:13.571 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.571 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.571 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.571 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302835+0000", 2024-05-08T08:53:13.571 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6460000000000002e-06 2024-05-08T08:53:13.571 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.571 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.571 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.571 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302841+0000", 2024-05-08T08:53:13.572 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1489999999999999e-06 2024-05-08T08:53:13.572 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.572 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.572 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.572 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302847+0000", 2024-05-08T08:53:13.572 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7590000000000003e-06 2024-05-08T08:53:13.572 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.572 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.572 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.572 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.572 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.572 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27456 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.572 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.305664+0000", 2024-05-08T08:53:13.573 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0088147659999999999, 2024-05-08T08:53:13.573 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.012931633, 2024-05-08T08:53:13.573 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.573 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.573 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.573 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.573 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.573 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.573 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27456 2024-05-08T08:53:13.573 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.573 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.573 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.573 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.573 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.305664+0000", 2024-05-08T08:53:13.573 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.574 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.574 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.574 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.574 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.305664+0000", 2024-05-08T08:53:13.574 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.574 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.574 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.574 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.574 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.305665+0000", 2024-05-08T08:53:13.574 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8799999999999998e-07 2024-05-08T08:53:13.574 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.574 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.574 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.574 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.305668+0000", 2024-05-08T08:53:13.574 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9749999999999999e-06 2024-05-08T08:53:13.575 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.575 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.575 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.575 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.305674+0000", 2024-05-08T08:53:13.575 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.28e-06 2024-05-08T08:53:13.575 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.575 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.575 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.575 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.305679+0000", 2024-05-08T08:53:13.575 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5859999999999998e-06 2024-05-08T08:53:13.575 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.575 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.575 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.575 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.576 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.576 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27585 3.4 3.134fb004 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.576 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.310475+0000", 2024-05-08T08:53:13.576 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0040046320000000002, 2024-05-08T08:53:13.576 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0081757059999999996, 2024-05-08T08:53:13.576 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.576 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.576 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.576 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.576 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.576 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.576 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27585 2024-05-08T08:53:13.576 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.576 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.576 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.576 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.577 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.310475+0000", 2024-05-08T08:53:13.577 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.577 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.577 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.577 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.577 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.310475+0000", 2024-05-08T08:53:13.577 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.577 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.577 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.577 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.577 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.310475+0000", 2024-05-08T08:53:13.577 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4400000000000001e-07 2024-05-08T08:53:13.577 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.577 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.577 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.578 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.310477+0000", 2024-05-08T08:53:13.578 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.165e-06 2024-05-08T08:53:13.578 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.578 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.578 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.578 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.310483+0000", 2024-05-08T08:53:13.578 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7899999999999996e-06 2024-05-08T08:53:13.578 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.578 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.578 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.578 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.310489+0000", 2024-05-08T08:53:13.578 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2230000000000004e-06 2024-05-08T08:53:13.578 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.578 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.579 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.579 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.579 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.579 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:11594 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.579 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.271688+0000", 2024-05-08T08:53:13.579 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.042790863999999998, 2024-05-08T08:53:13.579 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.047016506, 2024-05-08T08:53:13.579 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.579 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.579 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.579 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.579 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:13.579 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:13.579 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 11594 2024-05-08T08:53:13.579 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.580 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.580 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.580 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.580 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.271688+0000", 2024-05-08T08:53:13.580 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.580 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.580 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.580 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.580 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.271688+0000", 2024-05-08T08:53:13.580 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.580 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.580 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.580 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.580 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.271689+0000", 2024-05-08T08:53:13.580 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3500000000000005e-07 2024-05-08T08:53:13.581 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.581 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.581 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.581 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.271694+0000", 2024-05-08T08:53:13.581 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5719999999999996e-06 2024-05-08T08:53:13.581 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.581 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.581 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.581 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.271705+0000", 2024-05-08T08:53:13.581 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0902e-05 2024-05-08T08:53:13.581 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.581 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.581 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.581 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.271712+0000", 2024-05-08T08:53:13.581 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3470000000000001e-06 2024-05-08T08:53:13.582 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.582 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.582 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.582 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.582 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.582 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:26771 3.4 3.134fb004 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.582 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.283789+0000", 2024-05-08T08:53:13.582 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.030690433999999999, 2024-05-08T08:53:13.582 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.034970323999999997, 2024-05-08T08:53:13.582 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.582 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.582 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.582 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.582 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.582 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.583 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 26771 2024-05-08T08:53:13.583 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.583 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.583 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.583 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.583 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283789+0000", 2024-05-08T08:53:13.583 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.583 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.583 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.583 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.583 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283789+0000", 2024-05-08T08:53:13.583 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.583 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.583 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.583 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.584 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283789+0000", 2024-05-08T08:53:13.584 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0699999999999999e-07 2024-05-08T08:53:13.584 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.584 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.584 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.584 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283790+0000", 2024-05-08T08:53:13.584 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.192e-06 2024-05-08T08:53:13.584 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.584 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.584 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.584 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283793+0000", 2024-05-08T08:53:13.584 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0180000000000002e-06 2024-05-08T08:53:13.584 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.584 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.584 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.585 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283795+0000", 2024-05-08T08:53:13.585 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3e-06 2024-05-08T08:53:13.585 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.585 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.585 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.585 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.585 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.585 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27021 3.4 3.134fb004 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.585 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.289601+0000", 2024-05-08T08:53:13.585 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.024878069999999999, 2024-05-08T08:53:13.585 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.029214505000000002, 2024-05-08T08:53:13.585 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.585 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.585 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.585 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.586 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.586 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.586 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27021 2024-05-08T08:53:13.586 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.586 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.586 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.586 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.586 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289601+0000", 2024-05-08T08:53:13.586 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.586 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.586 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.586 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.586 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289601+0000", 2024-05-08T08:53:13.586 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.587 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.587 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.587 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.587 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289601+0000", 2024-05-08T08:53:13.587 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5900000000000003e-07 2024-05-08T08:53:13.587 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.587 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.587 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.587 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289604+0000", 2024-05-08T08:53:13.587 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0999999999999998e-06 2024-05-08T08:53:13.587 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.587 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.587 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.587 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289610+0000", 2024-05-08T08:53:13.587 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.241e-06 2024-05-08T08:53:13.588 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.588 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.588 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.588 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289615+0000", 2024-05-08T08:53:13.588 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8280000000000002e-06 2024-05-08T08:53:13.588 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.588 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.588 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.588 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.588 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.588 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27236 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.588 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.302109+0000", 2024-05-08T08:53:13.588 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.012369998, 2024-05-08T08:53:13.589 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.016762557000000001, 2024-05-08T08:53:13.589 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.589 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.589 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.589 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.589 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.589 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.589 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27236 2024-05-08T08:53:13.589 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.589 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.589 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.589 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.589 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302109+0000", 2024-05-08T08:53:13.589 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.589 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.590 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.590 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.590 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302109+0000", 2024-05-08T08:53:13.590 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.590 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.590 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.590 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.590 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302110+0000", 2024-05-08T08:53:13.590 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6199999999999998e-07 2024-05-08T08:53:13.590 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.590 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.590 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.590 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302112+0000", 2024-05-08T08:53:13.590 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.5639999999999999e-06 2024-05-08T08:53:13.590 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.591 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.591 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.591 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302124+0000", 2024-05-08T08:53:13.591 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2105e-05 2024-05-08T08:53:13.591 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.591 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.591 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.591 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302130+0000", 2024-05-08T08:53:13.591 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4140000000000002e-06 2024-05-08T08:53:13.591 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.591 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.591 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.591 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.591 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.592 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27326 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.592 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.302854+0000", 2024-05-08T08:53:13.592 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.011625672, 2024-05-08T08:53:13.592 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.016072936999999999, 2024-05-08T08:53:13.592 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.592 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.592 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.592 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.592 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.592 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.592 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27326 2024-05-08T08:53:13.592 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.592 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.592 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.592 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.593 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302854+0000", 2024-05-08T08:53:13.593 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.593 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.593 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.593 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.593 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302854+0000", 2024-05-08T08:53:13.593 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.593 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.593 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.593 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.593 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302854+0000", 2024-05-08T08:53:13.593 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.2500000000000001e-07 2024-05-08T08:53:13.593 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.593 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.594 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.594 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302861+0000", 2024-05-08T08:53:13.594 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8340000000000001e-06 2024-05-08T08:53:13.594 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.594 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.594 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.594 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302867+0000", 2024-05-08T08:53:13.594 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.567e-06 2024-05-08T08:53:13.594 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.594 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.594 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.594 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302873+0000", 2024-05-08T08:53:13.594 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4620000000000004e-06 2024-05-08T08:53:13.594 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.594 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.595 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.595 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.595 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.595 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27586 3.4 3.134fb004 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.595 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.310495+0000", 2024-05-08T08:53:13.595 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0039842139999999998, 2024-05-08T08:53:13.595 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0084905469999999993, 2024-05-08T08:53:13.595 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.595 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.595 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.595 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.595 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.595 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.595 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27586 2024-05-08T08:53:13.595 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.596 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.596 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.596 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.596 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.310495+0000", 2024-05-08T08:53:13.596 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.596 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.596 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.596 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.596 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.310495+0000", 2024-05-08T08:53:13.596 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.596 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.596 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.596 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.596 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.310495+0000", 2024-05-08T08:53:13.596 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4999999999999998e-07 2024-05-08T08:53:13.597 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.597 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.597 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.597 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.310498+0000", 2024-05-08T08:53:13.597 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2419999999999999e-06 2024-05-08T08:53:13.597 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.597 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.597 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.597 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.310503+0000", 2024-05-08T08:53:13.597 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6289999999999998e-06 2024-05-08T08:53:13.597 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.597 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.597 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.597 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.310508+0000", 2024-05-08T08:53:13.597 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9620000000000001e-06 2024-05-08T08:53:13.598 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.598 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.598 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.598 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.598 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.598 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:11595 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.598 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.271927+0000", 2024-05-08T08:53:13.598 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.042552313000000001, 2024-05-08T08:53:13.598 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.047114811999999999, 2024-05-08T08:53:13.598 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.598 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.598 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.598 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.598 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:13.599 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:13.599 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 11595 2024-05-08T08:53:13.599 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.599 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.599 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.599 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.599 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.271927+0000", 2024-05-08T08:53:13.599 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.599 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.599 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.599 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.599 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.271927+0000", 2024-05-08T08:53:13.599 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.599 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.599 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.600 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.600 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.271928+0000", 2024-05-08T08:53:13.600 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0720000000000001e-06 2024-05-08T08:53:13.600 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.600 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.600 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.600 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.271933+0000", 2024-05-08T08:53:13.600 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2979999999999999e-06 2024-05-08T08:53:13.600 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.600 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.600 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.600 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.271945+0000", 2024-05-08T08:53:13.600 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1582e-05 2024-05-08T08:53:13.600 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.601 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.601 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.601 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.271952+0000", 2024-05-08T08:53:13.601 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5220000000000001e-06 2024-05-08T08:53:13.601 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.601 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.601 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.601 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.601 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.601 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27022 3.4 3.134fb004 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.601 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.289621+0000", 2024-05-08T08:53:13.601 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.024858293, 2024-05-08T08:53:13.602 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.029475251000000001, 2024-05-08T08:53:13.602 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.602 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.602 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.602 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.602 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.602 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.602 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27022 2024-05-08T08:53:13.602 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.602 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.602 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.602 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.603 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289621+0000", 2024-05-08T08:53:13.603 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.603 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.603 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.603 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.603 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289621+0000", 2024-05-08T08:53:13.603 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.603 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.603 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.603 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.603 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289621+0000", 2024-05-08T08:53:13.603 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.22e-07 2024-05-08T08:53:13.603 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.603 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.604 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.604 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289623+0000", 2024-05-08T08:53:13.604 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.131e-06 2024-05-08T08:53:13.604 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.604 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.604 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.604 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289629+0000", 2024-05-08T08:53:13.604 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7780000000000004e-06 2024-05-08T08:53:13.604 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.604 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.604 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.604 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289634+0000", 2024-05-08T08:53:13.604 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6430000000000003e-06 2024-05-08T08:53:13.605 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.605 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.605 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.605 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.605 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.605 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27237 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.605 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.302138+0000", 2024-05-08T08:53:13.605 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.012341255000000001, 2024-05-08T08:53:13.605 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.017014576999999999, 2024-05-08T08:53:13.605 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.605 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.605 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.605 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.605 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.605 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.606 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27237 2024-05-08T08:53:13.606 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.606 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.606 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.606 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.606 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302138+0000", 2024-05-08T08:53:13.606 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.606 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.606 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.606 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.606 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302138+0000", 2024-05-08T08:53:13.606 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.606 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.606 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.607 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.607 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302138+0000", 2024-05-08T08:53:13.607 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5600000000000001e-07 2024-05-08T08:53:13.607 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.607 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.607 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.607 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302141+0000", 2024-05-08T08:53:13.607 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8490000000000002e-06 2024-05-08T08:53:13.607 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.607 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.607 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.607 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302148+0000", 2024-05-08T08:53:13.607 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5799999999999997e-06 2024-05-08T08:53:13.607 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.607 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.607 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.608 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302153+0000", 2024-05-08T08:53:13.608 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2249999999999999e-06 2024-05-08T08:53:13.608 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.608 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.608 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.608 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.608 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.608 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27332 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.608 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.302880+0000", 2024-05-08T08:53:13.608 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.011599418, 2024-05-08T08:53:13.608 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.016331595000000001, 2024-05-08T08:53:13.608 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.609 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.609 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.609 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.609 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.609 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.609 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27332 2024-05-08T08:53:13.609 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.609 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.609 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.609 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.609 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302880+0000", 2024-05-08T08:53:13.609 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.609 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.610 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.610 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.610 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302880+0000", 2024-05-08T08:53:13.610 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.610 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.610 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.610 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.610 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302880+0000", 2024-05-08T08:53:13.610 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8200000000000001e-07 2024-05-08T08:53:13.610 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.610 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.610 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.610 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302883+0000", 2024-05-08T08:53:13.611 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9040000000000001e-06 2024-05-08T08:53:13.611 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.611 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.611 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.611 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302889+0000", 2024-05-08T08:53:13.611 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2870000000000001e-06 2024-05-08T08:53:13.611 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.611 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.611 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.611 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302894+0000", 2024-05-08T08:53:13.611 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8840000000000002e-06 2024-05-08T08:53:13.611 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.611 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.612 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.612 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.612 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.612 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27587 3.4 3.134fb004 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.612 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.310514+0000", 2024-05-08T08:53:13.612 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0039650040000000003, 2024-05-08T08:53:13.612 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0087520440000000005, 2024-05-08T08:53:13.612 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.612 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.612 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.612 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.612 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.612 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.612 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27587 2024-05-08T08:53:13.612 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.612 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.612 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.613 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.613 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.310514+0000", 2024-05-08T08:53:13.613 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.613 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.613 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.613 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.613 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.310514+0000", 2024-05-08T08:53:13.613 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.613 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.613 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.613 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.613 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.310515+0000", 2024-05-08T08:53:13.613 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4400000000000001e-07 2024-05-08T08:53:13.613 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.613 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.614 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.614 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.310516+0000", 2024-05-08T08:53:13.614 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.889e-06 2024-05-08T08:53:13.614 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.614 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.614 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.614 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.310522+0000", 2024-05-08T08:53:13.614 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6099999999999997e-06 2024-05-08T08:53:13.614 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.614 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.614 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.614 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311072+0000", 2024-05-08T08:53:13.614 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00054982399999999997 2024-05-08T08:53:13.614 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.615 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.615 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.615 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.615 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.615 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:11596 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.615 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.272156+0000", 2024-05-08T08:53:13.615 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.04232325, 2024-05-08T08:53:13.615 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.047164615999999999, 2024-05-08T08:53:13.615 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.615 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.615 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.615 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.615 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:13.616 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:13.616 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 11596 2024-05-08T08:53:13.616 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.616 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.616 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.616 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.616 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.272156+0000", 2024-05-08T08:53:13.616 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.616 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.616 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.616 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.616 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.272156+0000", 2024-05-08T08:53:13.616 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.616 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.617 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.617 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.617 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.272157+0000", 2024-05-08T08:53:13.617 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4799999999999997e-07 2024-05-08T08:53:13.617 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.617 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.617 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.617 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.272161+0000", 2024-05-08T08:53:13.617 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.318e-06 2024-05-08T08:53:13.617 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.617 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.617 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.617 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.272171+0000", 2024-05-08T08:53:13.617 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.004e-05 2024-05-08T08:53:13.617 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.618 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.618 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.618 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.272178+0000", 2024-05-08T08:53:13.618 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0709999999999997e-06 2024-05-08T08:53:13.618 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.618 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.618 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.618 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.618 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.618 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27023 3.4 3.134fb004 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.618 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.289883+0000", 2024-05-08T08:53:13.618 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.024596034999999999, 2024-05-08T08:53:13.618 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.029491719, 2024-05-08T08:53:13.618 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.619 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.619 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.619 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.619 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.619 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.619 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27023 2024-05-08T08:53:13.619 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.619 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.619 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.619 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.619 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289883+0000", 2024-05-08T08:53:13.619 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.619 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.619 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.619 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.620 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289883+0000", 2024-05-08T08:53:13.620 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.620 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.620 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.620 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.620 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289884+0000", 2024-05-08T08:53:13.620 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6199999999999997e-07 2024-05-08T08:53:13.620 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.620 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.620 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.620 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289887+0000", 2024-05-08T08:53:13.620 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5590000000000001e-06 2024-05-08T08:53:13.620 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.620 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.620 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.621 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289897+0000", 2024-05-08T08:53:13.621 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3349999999999996e-06 2024-05-08T08:53:13.621 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.621 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.621 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.621 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289903+0000", 2024-05-08T08:53:13.621 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0329999999999997e-06 2024-05-08T08:53:13.621 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.621 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.621 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.621 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.621 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.621 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27238 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.621 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.302160+0000", 2024-05-08T08:53:13.621 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.012319438, 2024-05-08T08:53:13.622 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.017269791, 2024-05-08T08:53:13.622 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.622 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.622 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.622 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.622 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.622 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.622 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27238 2024-05-08T08:53:13.622 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.622 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.622 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.622 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.622 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302160+0000", 2024-05-08T08:53:13.622 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.623 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.623 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.623 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.623 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302160+0000", 2024-05-08T08:53:13.623 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.623 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.623 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.623 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.623 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302160+0000", 2024-05-08T08:53:13.623 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:53:13.623 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.623 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.623 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.623 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302163+0000", 2024-05-08T08:53:13.624 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.5189999999999999e-06 2024-05-08T08:53:13.624 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.624 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.624 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.624 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302169+0000", 2024-05-08T08:53:13.624 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1e-06 2024-05-08T08:53:13.624 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.624 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.624 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.624 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302173+0000", 2024-05-08T08:53:13.624 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6650000000000002e-06 2024-05-08T08:53:13.624 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.624 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.624 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.625 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.625 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.625 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27334 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.625 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.302901+0000", 2024-05-08T08:53:13.625 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.011577756, 2024-05-08T08:53:13.625 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.016581775, 2024-05-08T08:53:13.625 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.625 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.625 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.625 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.625 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.625 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.625 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27334 2024-05-08T08:53:13.625 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.626 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.626 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.626 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.626 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302901+0000", 2024-05-08T08:53:13.626 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.626 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.626 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.626 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.626 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302901+0000", 2024-05-08T08:53:13.626 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.626 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.626 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.626 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.627 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302902+0000", 2024-05-08T08:53:13.627 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.13e-07 2024-05-08T08:53:13.627 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.627 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.627 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.627 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302904+0000", 2024-05-08T08:53:13.627 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.5459999999999998e-06 2024-05-08T08:53:13.627 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.627 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.627 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.627 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302911+0000", 2024-05-08T08:53:13.627 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7490000000000001e-06 2024-05-08T08:53:13.627 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.627 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.628 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.628 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302916+0000", 2024-05-08T08:53:13.628 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1399999999999999e-06 2024-05-08T08:53:13.628 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.628 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.628 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.628 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.628 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.628 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27459 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.628 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.305724+0000", 2024-05-08T08:53:13.628 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0087550990000000006, 2024-05-08T08:53:13.628 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.013820463, 2024-05-08T08:53:13.628 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.628 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.629 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.629 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.629 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.629 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.629 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27459 2024-05-08T08:53:13.629 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.629 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.629 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.629 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.629 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.305724+0000", 2024-05-08T08:53:13.629 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.629 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.629 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.630 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.630 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.305724+0000", 2024-05-08T08:53:13.630 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.630 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.630 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.630 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.630 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.305724+0000", 2024-05-08T08:53:13.630 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1399999999999998e-07 2024-05-08T08:53:13.630 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.630 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.630 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.630 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.305727+0000", 2024-05-08T08:53:13.630 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8339999999999999e-06 2024-05-08T08:53:13.631 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.631 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.631 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.631 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.305734+0000", 2024-05-08T08:53:13.631 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3459999999999999e-06 2024-05-08T08:53:13.631 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.631 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.631 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.631 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.305738+0000", 2024-05-08T08:53:13.631 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8609999999999997e-06 2024-05-08T08:53:13.631 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.631 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.631 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.632 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.632 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.632 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27588 3.4 3.134fb004 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.632 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.311133+0000", 2024-05-08T08:53:13.632 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.003346477, 2024-05-08T08:53:13.632 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0084666269999999991, 2024-05-08T08:53:13.632 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.632 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.632 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.632 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.632 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.632 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.632 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27588 2024-05-08T08:53:13.632 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.633 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.633 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.633 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.633 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311133+0000", 2024-05-08T08:53:13.633 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.633 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.633 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.633 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.633 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311133+0000", 2024-05-08T08:53:13.633 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.633 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.633 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.633 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.633 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311133+0000", 2024-05-08T08:53:13.634 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3900000000000004e-07 2024-05-08T08:53:13.634 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.634 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.634 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.634 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311137+0000", 2024-05-08T08:53:13.634 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8380000000000004e-06 2024-05-08T08:53:13.634 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.634 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.634 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.634 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311146+0000", 2024-05-08T08:53:13.634 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1330000000000005e-06 2024-05-08T08:53:13.634 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.634 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.635 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.635 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311152+0000", 2024-05-08T08:53:13.635 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9839999999999998e-06 2024-05-08T08:53:13.635 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.635 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.635 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.635 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.635 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.635 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:11597 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.635 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.272482+0000", 2024-05-08T08:53:13.635 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.041997367000000001, 2024-05-08T08:53:13.635 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.047172448999999998, 2024-05-08T08:53:13.635 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.635 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.636 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.636 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.636 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:13.636 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:13.636 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 11597 2024-05-08T08:53:13.636 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.636 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.636 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.636 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.636 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.272482+0000", 2024-05-08T08:53:13.636 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.636 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.636 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.636 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.637 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.272482+0000", 2024-05-08T08:53:13.637 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.637 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.637 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.637 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.637 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.272483+0000", 2024-05-08T08:53:13.637 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.2999999999999999e-07 2024-05-08T08:53:13.637 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.637 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.637 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.637 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.272487+0000", 2024-05-08T08:53:13.637 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6589999999999998e-06 2024-05-08T08:53:13.637 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.637 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.637 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.638 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.272496+0000", 2024-05-08T08:53:13.638 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.371e-06 2024-05-08T08:53:13.638 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.638 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.638 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.638 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.272502+0000", 2024-05-08T08:53:13.638 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2269999999999998e-06 2024-05-08T08:53:13.638 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.638 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.638 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.638 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.638 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.638 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27024 3.4 3.134fb004 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.638 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.289913+0000", 2024-05-08T08:53:13.638 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.024566114, 2024-05-08T08:53:13.639 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.029796466000000001, 2024-05-08T08:53:13.639 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.639 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.639 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.639 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.639 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.639 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.639 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27024 2024-05-08T08:53:13.639 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.639 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.639 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.639 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.639 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289913+0000", 2024-05-08T08:53:13.639 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.640 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.640 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.640 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.640 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289913+0000", 2024-05-08T08:53:13.640 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.640 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.640 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.640 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.640 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289914+0000", 2024-05-08T08:53:13.640 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9999999999999998e-07 2024-05-08T08:53:13.640 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.640 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.640 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.640 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289916+0000", 2024-05-08T08:53:13.640 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.396e-06 2024-05-08T08:53:13.641 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.641 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.641 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.641 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289922+0000", 2024-05-08T08:53:13.641 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2230000000000004e-06 2024-05-08T08:53:13.641 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.641 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.641 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.641 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289927+0000", 2024-05-08T08:53:13.641 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0579999999999997e-06 2024-05-08T08:53:13.641 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.641 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.641 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.641 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.641 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.641 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27239 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.642 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.302180+0000", 2024-05-08T08:53:13.642 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.012299087, 2024-05-08T08:53:13.642 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.017583946999999999, 2024-05-08T08:53:13.642 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.642 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.642 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.642 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.642 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.642 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.642 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27239 2024-05-08T08:53:13.642 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.642 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.642 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.642 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.642 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302180+0000", 2024-05-08T08:53:13.642 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.643 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.643 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.643 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.643 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302180+0000", 2024-05-08T08:53:13.643 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.643 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.643 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.643 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.643 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302181+0000", 2024-05-08T08:53:13.643 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:53:13.643 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.643 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.643 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.643 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302183+0000", 2024-05-08T08:53:13.643 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7420000000000002e-06 2024-05-08T08:53:13.644 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.644 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.644 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.644 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302190+0000", 2024-05-08T08:53:13.644 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4080000000000003e-06 2024-05-08T08:53:13.644 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.644 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.644 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.644 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302196+0000", 2024-05-08T08:53:13.644 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0039999999999996e-06 2024-05-08T08:53:13.644 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.644 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.644 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.644 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.645 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.645 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27340 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.645 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.302923+0000", 2024-05-08T08:53:13.645 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.011556162, 2024-05-08T08:53:13.645 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.016895124000000001, 2024-05-08T08:53:13.645 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.645 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.645 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.645 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.645 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.645 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.645 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27340 2024-05-08T08:53:13.645 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.646 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.646 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.646 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.646 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302923+0000", 2024-05-08T08:53:13.646 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.646 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.646 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.646 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.646 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302923+0000", 2024-05-08T08:53:13.646 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.646 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.646 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.646 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.647 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302923+0000", 2024-05-08T08:53:13.647 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7399999999999999e-07 2024-05-08T08:53:13.647 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.647 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.647 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.647 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302926+0000", 2024-05-08T08:53:13.647 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.5459999999999998e-06 2024-05-08T08:53:13.647 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.647 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.647 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.647 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302932+0000", 2024-05-08T08:53:13.647 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5289999999999997e-06 2024-05-08T08:53:13.647 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.647 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.647 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.648 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302938+0000", 2024-05-08T08:53:13.648 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.93e-06 2024-05-08T08:53:13.648 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.648 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.648 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.648 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.648 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.648 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27589 3.4 3.134fb004 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.648 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.311162+0000", 2024-05-08T08:53:13.648 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0033172380000000001, 2024-05-08T08:53:13.648 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0087118300000000003, 2024-05-08T08:53:13.648 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.648 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.648 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.649 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.649 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.649 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.649 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27589 2024-05-08T08:53:13.649 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.649 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.649 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.649 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.649 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311162+0000", 2024-05-08T08:53:13.649 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.649 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.649 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.649 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.649 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311162+0000", 2024-05-08T08:53:13.650 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.650 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.650 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.650 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.650 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311162+0000", 2024-05-08T08:53:13.650 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3000000000000001e-07 2024-05-08T08:53:13.650 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.650 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.650 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.650 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311165+0000", 2024-05-08T08:53:13.650 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2520000000000002e-06 2024-05-08T08:53:13.650 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.650 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.650 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.650 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311171+0000", 2024-05-08T08:53:13.651 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2480000000000001e-06 2024-05-08T08:53:13.651 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.651 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.651 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.651 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311176+0000", 2024-05-08T08:53:13.651 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.87e-06 2024-05-08T08:53:13.651 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.651 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.651 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.651 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.651 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.651 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:11598 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.651 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.272708+0000", 2024-05-08T08:53:13.651 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.041770938000000001, 2024-05-08T08:53:13.651 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.047223066000000001, 2024-05-08T08:53:13.652 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.652 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.652 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.652 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.652 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:13.652 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:13.652 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 11598 2024-05-08T08:53:13.652 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.652 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.652 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.652 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.652 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.272708+0000", 2024-05-08T08:53:13.652 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.652 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.652 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.653 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.653 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.272708+0000", 2024-05-08T08:53:13.653 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.653 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.653 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.653 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.653 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.272709+0000", 2024-05-08T08:53:13.653 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.02e-07 2024-05-08T08:53:13.653 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.653 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.653 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.653 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.272714+0000", 2024-05-08T08:53:13.653 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5750000000000002e-06 2024-05-08T08:53:13.653 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.653 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.654 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.654 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.272723+0000", 2024-05-08T08:53:13.654 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6630000000000005e-06 2024-05-08T08:53:13.654 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.654 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.654 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.654 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.272730+0000", 2024-05-08T08:53:13.654 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.888e-06 2024-05-08T08:53:13.654 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.654 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.654 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.654 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.654 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.655 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27025 3.4 3.134fb004 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.655 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.289935+0000", 2024-05-08T08:53:13.655 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.024544244, 2024-05-08T08:53:13.655 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.030056355, 2024-05-08T08:53:13.655 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.655 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.655 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.655 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.655 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.655 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.655 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27025 2024-05-08T08:53:13.655 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.655 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.655 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.655 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.656 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289935+0000", 2024-05-08T08:53:13.656 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.656 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.656 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.656 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.656 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289935+0000", 2024-05-08T08:53:13.656 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.656 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.656 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.656 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.656 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289935+0000", 2024-05-08T08:53:13.656 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4499999999999998e-07 2024-05-08T08:53:13.656 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.656 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.657 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.657 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289945+0000", 2024-05-08T08:53:13.657 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.9839999999999996e-06 2024-05-08T08:53:13.657 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.657 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.657 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.657 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289958+0000", 2024-05-08T08:53:13.657 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2975e-05 2024-05-08T08:53:13.657 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.657 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.657 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.657 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289964+0000", 2024-05-08T08:53:13.657 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0140000000000004e-06 2024-05-08T08:53:13.657 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.657 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.657 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.658 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.658 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.658 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27240 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.658 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.302202+0000", 2024-05-08T08:53:13.658 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.012277078, 2024-05-08T08:53:13.658 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.017846267999999998, 2024-05-08T08:53:13.658 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.658 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.658 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.658 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.658 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.658 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.658 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27240 2024-05-08T08:53:13.658 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.659 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.659 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.659 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.659 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302202+0000", 2024-05-08T08:53:13.659 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.659 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.659 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.659 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.659 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302202+0000", 2024-05-08T08:53:13.659 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.659 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.659 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.659 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.659 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302203+0000", 2024-05-08T08:53:13.659 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9999999999999998e-07 2024-05-08T08:53:13.660 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.660 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.660 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.660 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302205+0000", 2024-05-08T08:53:13.660 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4480000000000001e-06 2024-05-08T08:53:13.660 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.660 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.660 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.660 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302211+0000", 2024-05-08T08:53:13.660 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0510000000000002e-06 2024-05-08T08:53:13.660 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.660 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.660 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.660 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302217+0000", 2024-05-08T08:53:13.661 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6849999999999998e-06 2024-05-08T08:53:13.661 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.661 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.661 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.661 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.661 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.661 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27341 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.661 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.302945+0000", 2024-05-08T08:53:13.661 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.011534264000000001, 2024-05-08T08:53:13.661 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.017158834000000001, 2024-05-08T08:53:13.661 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.661 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.661 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.661 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.662 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.662 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.662 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27341 2024-05-08T08:53:13.662 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.662 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.662 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.662 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.662 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302945+0000", 2024-05-08T08:53:13.662 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.662 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.662 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.662 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.662 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302945+0000", 2024-05-08T08:53:13.663 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.663 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.663 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.663 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.663 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302945+0000", 2024-05-08T08:53:13.663 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4499999999999997e-07 2024-05-08T08:53:13.663 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.663 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.663 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.663 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302948+0000", 2024-05-08T08:53:13.663 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.5019999999999999e-06 2024-05-08T08:53:13.663 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.663 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.663 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.663 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302967+0000", 2024-05-08T08:53:13.664 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.8913e-05 2024-05-08T08:53:13.664 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.664 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.664 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.664 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302973+0000", 2024-05-08T08:53:13.664 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7819999999999999e-06 2024-05-08T08:53:13.664 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.664 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.664 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.664 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.664 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.664 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27590 3.4 3.134fb004 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.665 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.311182+0000", 2024-05-08T08:53:13.665 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0032969610000000002, 2024-05-08T08:53:13.665 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0089753490000000005, 2024-05-08T08:53:13.665 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.665 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.665 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.665 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.665 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.665 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.665 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27590 2024-05-08T08:53:13.665 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.665 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.666 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.666 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.666 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311182+0000", 2024-05-08T08:53:13.666 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.666 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.666 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.666 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.666 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311182+0000", 2024-05-08T08:53:13.666 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.666 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.666 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.666 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.666 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311183+0000", 2024-05-08T08:53:13.666 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9099999999999999e-07 2024-05-08T08:53:13.667 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.667 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.667 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.667 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311185+0000", 2024-05-08T08:53:13.667 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9690000000000001e-06 2024-05-08T08:53:13.667 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.667 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.667 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.667 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311190+0000", 2024-05-08T08:53:13.667 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6609999999999997e-06 2024-05-08T08:53:13.667 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.667 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.667 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.667 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311195+0000", 2024-05-08T08:53:13.667 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6920000000000001e-06 2024-05-08T08:53:13.668 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.668 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.668 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.668 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.668 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.668 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:11599 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.668 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.272930+0000", 2024-05-08T08:53:13.668 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.041549379999999997, 2024-05-08T08:53:13.668 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.047282204000000001, 2024-05-08T08:53:13.668 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.668 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.668 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.668 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.668 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:13.668 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:13.669 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 11599 2024-05-08T08:53:13.669 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.669 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.669 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.669 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.669 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.272930+0000", 2024-05-08T08:53:13.669 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.669 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.669 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.669 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.669 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.272930+0000", 2024-05-08T08:53:13.669 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.669 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.669 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.669 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.670 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.272931+0000", 2024-05-08T08:53:13.670 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8100000000000002e-07 2024-05-08T08:53:13.670 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.670 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.670 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.670 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.272935+0000", 2024-05-08T08:53:13.670 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5050000000000001e-06 2024-05-08T08:53:13.670 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.670 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.670 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.670 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.272944+0000", 2024-05-08T08:53:13.670 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7430000000000008e-06 2024-05-08T08:53:13.670 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.670 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.671 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.671 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.272950+0000", 2024-05-08T08:53:13.671 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.443e-06 2024-05-08T08:53:13.671 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.671 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.671 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.671 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.671 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.671 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:11613 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.671 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.285268+0000", 2024-05-08T08:53:13.671 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.029211603999999999, 2024-05-08T08:53:13.671 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.035000590999999998, 2024-05-08T08:53:13.671 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.671 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.671 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.672 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.672 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:13.672 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:13.672 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 11613 2024-05-08T08:53:13.672 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.672 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.672 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.672 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.672 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.285268+0000", 2024-05-08T08:53:13.672 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.672 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.672 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.672 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.672 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.285268+0000", 2024-05-08T08:53:13.673 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.673 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.673 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.673 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.673 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.285269+0000", 2024-05-08T08:53:13.673 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0610000000000001e-06 2024-05-08T08:53:13.673 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.673 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.673 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.673 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.285274+0000", 2024-05-08T08:53:13.673 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3820000000000003e-06 2024-05-08T08:53:13.674 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.674 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.674 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.674 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.285284+0000", 2024-05-08T08:53:13.674 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0427e-05 2024-05-08T08:53:13.674 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.674 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.674 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.674 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.285292+0000", 2024-05-08T08:53:13.674 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2189999999999998e-06 2024-05-08T08:53:13.674 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.674 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.674 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.674 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.675 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.675 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27026 3.4 3.134fb004 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.675 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.289973+0000", 2024-05-08T08:53:13.675 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.024505905000000001, 2024-05-08T08:53:13.675 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.030349333999999999, 2024-05-08T08:53:13.675 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.675 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.675 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.675 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.675 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.675 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.675 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27026 2024-05-08T08:53:13.675 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.675 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.675 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.676 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.676 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289973+0000", 2024-05-08T08:53:13.676 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.676 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.676 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.676 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.676 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289973+0000", 2024-05-08T08:53:13.676 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.676 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.676 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.676 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.676 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289974+0000", 2024-05-08T08:53:13.676 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1100000000000001e-07 2024-05-08T08:53:13.676 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.677 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.677 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.677 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289976+0000", 2024-05-08T08:53:13.677 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.435e-06 2024-05-08T08:53:13.677 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.677 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.677 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.677 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289983+0000", 2024-05-08T08:53:13.677 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7349999999999999e-06 2024-05-08T08:53:13.677 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.677 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.677 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.677 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289990+0000", 2024-05-08T08:53:13.677 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6980000000000001e-06 2024-05-08T08:53:13.678 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.678 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.678 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.678 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.678 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.678 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27241 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.678 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.302224+0000", 2024-05-08T08:53:13.678 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.012255139999999999, 2024-05-08T08:53:13.678 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.018157485000000001, 2024-05-08T08:53:13.678 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.678 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.678 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.678 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.679 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.679 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.679 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27241 2024-05-08T08:53:13.679 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.679 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.679 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.679 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.679 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302224+0000", 2024-05-08T08:53:13.679 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.679 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.679 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.679 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.679 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302224+0000", 2024-05-08T08:53:13.679 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.679 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.679 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.680 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.680 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302224+0000", 2024-05-08T08:53:13.680 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.84e-07 2024-05-08T08:53:13.680 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.680 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.680 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.680 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302227+0000", 2024-05-08T08:53:13.680 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.65e-06 2024-05-08T08:53:13.680 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.680 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.680 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.680 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302234+0000", 2024-05-08T08:53:13.680 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.686e-06 2024-05-08T08:53:13.680 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.680 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.680 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.681 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302239+0000", 2024-05-08T08:53:13.681 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0919999999999997e-06 2024-05-08T08:53:13.681 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.681 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.681 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.681 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.681 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.681 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27343 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.681 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.302981+0000", 2024-05-08T08:53:13.681 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.011498145, 2024-05-08T08:53:13.681 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.017456238999999998, 2024-05-08T08:53:13.681 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.681 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.681 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.682 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.682 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.682 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.682 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27343 2024-05-08T08:53:13.682 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.682 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.682 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.682 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.682 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302981+0000", 2024-05-08T08:53:13.682 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.682 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.682 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.682 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.683 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302981+0000", 2024-05-08T08:53:13.683 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.683 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.683 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.683 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.683 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302981+0000", 2024-05-08T08:53:13.683 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4799999999999999e-07 2024-05-08T08:53:13.683 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.683 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.683 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.683 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302985+0000", 2024-05-08T08:53:13.683 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3220000000000001e-06 2024-05-08T08:53:13.683 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.683 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.684 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.684 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302992+0000", 2024-05-08T08:53:13.684 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7299999999999999e-06 2024-05-08T08:53:13.684 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.684 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.684 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.684 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302996+0000", 2024-05-08T08:53:13.684 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7820000000000001e-06 2024-05-08T08:53:13.684 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.684 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.684 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.684 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.684 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.684 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27413 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.684 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.304667+0000", 2024-05-08T08:53:13.685 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0098123140000000008, 2024-05-08T08:53:13.685 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.015826895000000001, 2024-05-08T08:53:13.685 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.685 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.685 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.685 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.685 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.685 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.685 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27413 2024-05-08T08:53:13.685 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.685 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.685 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.685 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.685 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.304667+0000", 2024-05-08T08:53:13.686 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.686 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.686 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.686 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.686 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.304667+0000", 2024-05-08T08:53:13.686 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.686 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.686 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.686 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.686 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.304667+0000", 2024-05-08T08:53:13.686 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3000000000000001e-07 2024-05-08T08:53:13.686 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.686 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.686 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.686 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.304671+0000", 2024-05-08T08:53:13.687 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.2380000000000002e-06 2024-05-08T08:53:13.687 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.687 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.687 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.687 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.304677+0000", 2024-05-08T08:53:13.687 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7789999999999998e-06 2024-05-08T08:53:13.687 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.687 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.687 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.687 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.304683+0000", 2024-05-08T08:53:13.687 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3310000000000003e-06 2024-05-08T08:53:13.687 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.687 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.687 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.687 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.687 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.688 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27462 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.688 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.305784+0000", 2024-05-08T08:53:13.688 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0086952990000000001, 2024-05-08T08:53:13.688 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.014764882, 2024-05-08T08:53:13.688 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.688 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.688 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.688 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.688 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.688 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.688 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27462 2024-05-08T08:53:13.688 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.688 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.688 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.688 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.688 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.305784+0000", 2024-05-08T08:53:13.689 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.689 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.689 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.689 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.689 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.305784+0000", 2024-05-08T08:53:13.689 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.689 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.689 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.689 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.689 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.305784+0000", 2024-05-08T08:53:13.689 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9299999999999999e-07 2024-05-08T08:53:13.689 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.689 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.690 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.690 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.305787+0000", 2024-05-08T08:53:13.690 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9550000000000001e-06 2024-05-08T08:53:13.690 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.690 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.690 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.690 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.305793+0000", 2024-05-08T08:53:13.691 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9519999999999999e-06 2024-05-08T08:53:13.691 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.691 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.691 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.691 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.305800+0000", 2024-05-08T08:53:13.691 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3740000000000003e-06 2024-05-08T08:53:13.691 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.691 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.691 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.691 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.691 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.691 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27591 3.4 3.134fb004 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.691 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.311201+0000", 2024-05-08T08:53:13.692 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0032782380000000002, 2024-05-08T08:53:13.692 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0094036069999999996, 2024-05-08T08:53:13.692 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.692 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.692 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.692 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.692 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.692 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.692 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27591 2024-05-08T08:53:13.692 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.692 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.692 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.692 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.692 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311201+0000", 2024-05-08T08:53:13.693 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.693 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.693 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.693 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.693 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311201+0000", 2024-05-08T08:53:13.693 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.693 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.693 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.693 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.693 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311201+0000", 2024-05-08T08:53:13.693 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.46e-07 2024-05-08T08:53:13.693 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.693 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.693 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.694 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311203+0000", 2024-05-08T08:53:13.694 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.939e-06 2024-05-08T08:53:13.694 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.694 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.694 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.694 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311209+0000", 2024-05-08T08:53:13.694 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8760000000000002e-06 2024-05-08T08:53:13.694 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.694 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.694 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.694 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311214+0000", 2024-05-08T08:53:13.694 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4950000000000002e-06 2024-05-08T08:53:13.694 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.694 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.695 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.695 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.695 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.695 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:11600 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.695 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.273161+0000", 2024-05-08T08:53:13.695 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.041318131000000001, 2024-05-08T08:53:13.695 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.04750132, 2024-05-08T08:53:13.695 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.695 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.695 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.695 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.695 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:13.695 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:13.695 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 11600 2024-05-08T08:53:13.696 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.696 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.696 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.696 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.696 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.273161+0000", 2024-05-08T08:53:13.696 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.696 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.696 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.696 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.696 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.273161+0000", 2024-05-08T08:53:13.696 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.696 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.696 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.696 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.696 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.273162+0000", 2024-05-08T08:53:13.696 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9500000000000001e-07 2024-05-08T08:53:13.696 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.697 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.697 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.697 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.273167+0000", 2024-05-08T08:53:13.697 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8550000000000001e-06 2024-05-08T08:53:13.697 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.697 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.697 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.697 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.273193+0000", 2024-05-08T08:53:13.697 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.5834e-05 2024-05-08T08:53:13.697 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.697 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.697 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.697 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.273204+0000", 2024-05-08T08:53:13.697 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1780000000000001e-05 2024-05-08T08:53:13.697 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.697 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.698 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.698 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.698 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.698 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27027 3.4 3.134fb004 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.698 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.289997+0000", 2024-05-08T08:53:13.698 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.024482423, 2024-05-08T08:53:13.698 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.030722711, 2024-05-08T08:53:13.698 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.698 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.698 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.698 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.698 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.698 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.698 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27027 2024-05-08T08:53:13.698 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.699 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.699 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.699 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.699 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289997+0000", 2024-05-08T08:53:13.699 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.699 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.699 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.699 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.699 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289997+0000", 2024-05-08T08:53:13.699 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.699 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.699 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.699 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.700 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289997+0000", 2024-05-08T08:53:13.700 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9499999999999998e-07 2024-05-08T08:53:13.700 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.700 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.700 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.700 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289999+0000", 2024-05-08T08:53:13.700 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2510000000000001e-06 2024-05-08T08:53:13.700 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.700 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.700 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.700 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.290006+0000", 2024-05-08T08:53:13.700 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2230000000000004e-06 2024-05-08T08:53:13.700 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.700 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.701 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.701 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.290011+0000", 2024-05-08T08:53:13.701 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8960000000000002e-06 2024-05-08T08:53:13.701 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.701 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.701 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.701 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.701 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.701 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27242 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.701 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.302247+0000", 2024-05-08T08:53:13.701 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.012232593999999999, 2024-05-08T08:53:13.701 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.018532567999999999, 2024-05-08T08:53:13.702 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.702 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.702 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.702 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.702 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.702 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.702 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27242 2024-05-08T08:53:13.702 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.702 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.702 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.702 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.703 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302247+0000", 2024-05-08T08:53:13.703 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.703 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.703 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.703 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.703 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302247+0000", 2024-05-08T08:53:13.703 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.703 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.703 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.703 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.703 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302247+0000", 2024-05-08T08:53:13.703 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7800000000000002e-07 2024-05-08T08:53:13.703 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.703 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.703 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.703 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302250+0000", 2024-05-08T08:53:13.704 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6280000000000001e-06 2024-05-08T08:53:13.704 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.704 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.704 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.704 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302256+0000", 2024-05-08T08:53:13.704 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4710000000000004e-06 2024-05-08T08:53:13.704 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.704 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.704 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.704 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302261+0000", 2024-05-08T08:53:13.704 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.203e-06 2024-05-08T08:53:13.704 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.704 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.704 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.705 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.705 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.705 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27346 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.705 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.303002+0000", 2024-05-08T08:53:13.705 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.011476708, 2024-05-08T08:53:13.705 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.017834006999999999, 2024-05-08T08:53:13.705 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.705 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.705 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.705 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.705 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.705 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.705 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27346 2024-05-08T08:53:13.705 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.705 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.706 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.706 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.706 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303002+0000", 2024-05-08T08:53:13.706 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.706 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.706 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.706 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.706 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303002+0000", 2024-05-08T08:53:13.706 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.706 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.706 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.706 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.706 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303003+0000", 2024-05-08T08:53:13.707 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5600000000000001e-07 2024-05-08T08:53:13.707 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.707 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.707 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.707 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303005+0000", 2024-05-08T08:53:13.707 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6579999999999998e-06 2024-05-08T08:53:13.707 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.707 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.707 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.707 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303012+0000", 2024-05-08T08:53:13.707 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5250000000000002e-06 2024-05-08T08:53:13.707 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.707 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.707 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.708 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303017+0000", 2024-05-08T08:53:13.708 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8659999999999997e-06 2024-05-08T08:53:13.708 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.708 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.708 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.708 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.708 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.708 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27592 3.4 3.134fb004 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.708 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.311220+0000", 2024-05-08T08:53:13.708 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0032595380000000002, 2024-05-08T08:53:13.708 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0096763979999999992, 2024-05-08T08:53:13.708 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.708 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.708 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.709 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.709 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.709 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.709 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27592 2024-05-08T08:53:13.709 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.709 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.709 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.709 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.709 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311220+0000", 2024-05-08T08:53:13.709 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.709 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.709 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.709 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.709 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311220+0000", 2024-05-08T08:53:13.709 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.710 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.710 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.710 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.710 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311220+0000", 2024-05-08T08:53:13.710 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4799999999999999e-07 2024-05-08T08:53:13.710 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.710 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.710 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.710 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311222+0000", 2024-05-08T08:53:13.710 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9420000000000002e-06 2024-05-08T08:53:13.710 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.710 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.710 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.710 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311228+0000", 2024-05-08T08:53:13.711 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8259999999999998e-06 2024-05-08T08:53:13.711 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.711 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.711 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.711 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311233+0000", 2024-05-08T08:53:13.711 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7480000000000001e-06 2024-05-08T08:53:13.711 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.711 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.711 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.711 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.711 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.711 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:11601 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.711 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.273568+0000", 2024-05-08T08:53:13.711 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.040911239000000002, 2024-05-08T08:53:13.711 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.047393707, 2024-05-08T08:53:13.711 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.712 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.712 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "reached pg", 2024-05-08T08:53:13.712 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.712 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:13.712 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:13.712 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 11601 2024-05-08T08:53:13.712 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.712 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.712 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.712 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.712 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.273568+0000", 2024-05-08T08:53:13.712 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.712 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.713 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.713 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.713 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.273568+0000", 2024-05-08T08:53:13.713 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.713 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.713 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.713 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.713 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.273569+0000", 2024-05-08T08:53:13.713 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.004e-06 2024-05-08T08:53:13.713 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.713 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.713 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.713 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.273574+0000", 2024-05-08T08:53:13.714 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8060000000000002e-06 2024-05-08T08:53:13.714 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.714 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.714 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.714 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.273585+0000", 2024-05-08T08:53:13.714 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0754000000000001e-05 2024-05-08T08:53:13.714 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.714 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.714 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.714 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.273602+0000", 2024-05-08T08:53:13.714 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.7447000000000001e-05 2024-05-08T08:53:13.714 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.714 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.715 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:13.715 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.320949+0000", 2024-05-08T08:53:13.715 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.047347332999999998 2024-05-08T08:53:13.715 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.715 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.715 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.715 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.715 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.715 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27028 3.4 3.134fb004 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.715 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.290017+0000", 2024-05-08T08:53:13.715 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.024462184000000001, 2024-05-08T08:53:13.715 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.031008483, 2024-05-08T08:53:13.715 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.715 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.715 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.715 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.716 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.716 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.716 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27028 2024-05-08T08:53:13.716 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.716 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.716 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.716 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.716 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.290017+0000", 2024-05-08T08:53:13.716 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.716 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.716 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.716 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.717 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.290017+0000", 2024-05-08T08:53:13.717 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.717 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.717 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.717 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.717 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.290017+0000", 2024-05-08T08:53:13.717 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4499999999999998e-07 2024-05-08T08:53:13.717 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.717 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.717 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.717 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.290019+0000", 2024-05-08T08:53:13.717 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.99e-06 2024-05-08T08:53:13.717 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.717 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.717 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.718 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.290025+0000", 2024-05-08T08:53:13.718 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9039999999999997e-06 2024-05-08T08:53:13.718 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.718 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.718 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.718 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.290030+0000", 2024-05-08T08:53:13.718 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0010000000000001e-06 2024-05-08T08:53:13.718 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.718 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.718 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.718 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.718 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.718 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27243 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.718 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.302268+0000", 2024-05-08T08:53:13.719 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.012211547999999999, 2024-05-08T08:53:13.719 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.018815274, 2024-05-08T08:53:13.719 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.719 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.719 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.719 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.719 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.719 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.719 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27243 2024-05-08T08:53:13.719 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.719 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.719 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.719 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.719 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302268+0000", 2024-05-08T08:53:13.719 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.720 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.720 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.720 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.720 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302268+0000", 2024-05-08T08:53:13.720 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.720 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.720 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.720 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.720 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302268+0000", 2024-05-08T08:53:13.720 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9299999999999998e-07 2024-05-08T08:53:13.720 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.721 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.721 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.721 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302271+0000", 2024-05-08T08:53:13.721 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6070000000000002e-06 2024-05-08T08:53:13.721 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.721 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.721 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.721 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302278+0000", 2024-05-08T08:53:13.721 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2169999999999997e-06 2024-05-08T08:53:13.721 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.721 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.721 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.722 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302286+0000", 2024-05-08T08:53:13.722 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1810000000000001e-06 2024-05-08T08:53:13.722 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.722 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.722 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.722 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.722 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.722 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27348 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.722 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.303023+0000", 2024-05-08T08:53:13.722 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.011455793000000001, 2024-05-08T08:53:13.722 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.018114728, 2024-05-08T08:53:13.722 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.722 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.723 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.723 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.723 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.723 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.723 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27348 2024-05-08T08:53:13.723 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.723 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.723 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.723 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.723 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303023+0000", 2024-05-08T08:53:13.723 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.723 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.723 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.724 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.724 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303023+0000", 2024-05-08T08:53:13.724 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.724 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.724 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.724 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.724 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303024+0000", 2024-05-08T08:53:13.724 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7899999999999999e-07 2024-05-08T08:53:13.724 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.724 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.724 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.724 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303027+0000", 2024-05-08T08:53:13.724 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7549999999999999e-06 2024-05-08T08:53:13.724 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.725 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.725 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.725 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303033+0000", 2024-05-08T08:53:13.725 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1240000000000002e-06 2024-05-08T08:53:13.725 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.725 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.725 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.725 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303038+0000", 2024-05-08T08:53:13.725 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8999999999999997e-06 2024-05-08T08:53:13.725 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.725 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.725 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.725 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.725 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.725 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27415 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.725 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.304713+0000", 2024-05-08T08:53:13.726 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0097665029999999993, 2024-05-08T08:53:13.726 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.016484337000000002, 2024-05-08T08:53:13.726 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.726 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.726 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.726 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.726 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.726 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.726 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27415 2024-05-08T08:53:13.726 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.726 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.726 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.726 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.726 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.304713+0000", 2024-05-08T08:53:13.727 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.727 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.727 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.727 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.727 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.304713+0000", 2024-05-08T08:53:13.727 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.727 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.727 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.727 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.727 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.304719+0000", 2024-05-08T08:53:13.727 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0000000000000002e-06 2024-05-08T08:53:13.727 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.727 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.727 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.728 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.304722+0000", 2024-05-08T08:53:13.728 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.439e-06 2024-05-08T08:53:13.728 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.728 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.728 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.728 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.304915+0000", 2024-05-08T08:53:13.728 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00019307300000000001 2024-05-08T08:53:13.728 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.728 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.728 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.728 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.304932+0000", 2024-05-08T08:53:13.728 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6555e-05 2024-05-08T08:53:13.728 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.729 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.729 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.729 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.729 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.729 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27464 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.729 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.305829+0000", 2024-05-08T08:53:13.729 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0086503320000000002, 2024-05-08T08:53:13.729 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.01542636, 2024-05-08T08:53:13.729 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.729 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.729 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.729 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.729 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.729 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.729 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27464 2024-05-08T08:53:13.729 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.730 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.730 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.730 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.730 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.305829+0000", 2024-05-08T08:53:13.730 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.730 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.730 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.730 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.730 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.305829+0000", 2024-05-08T08:53:13.730 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.730 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.730 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.730 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.731 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.305829+0000", 2024-05-08T08:53:13.731 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4400000000000001e-07 2024-05-08T08:53:13.731 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.731 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.731 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.731 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.305837+0000", 2024-05-08T08:53:13.731 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0960000000000001e-06 2024-05-08T08:53:13.731 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.731 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.731 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.731 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.305844+0000", 2024-05-08T08:53:13.731 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3330000000000002e-06 2024-05-08T08:53:13.731 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.731 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.731 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.731 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.305849+0000", 2024-05-08T08:53:13.732 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0030000000000002e-06 2024-05-08T08:53:13.732 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.732 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.732 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.732 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.732 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.732 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27593 3.4 3.134fb004 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.732 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.311239+0000", 2024-05-08T08:53:13.732 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0032406079999999999, 2024-05-08T08:53:13.732 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.010075405000000001, 2024-05-08T08:53:13.732 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.732 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.732 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.732 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.732 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.732 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.733 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27593 2024-05-08T08:53:13.733 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.733 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.733 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.733 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.733 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311239+0000", 2024-05-08T08:53:13.733 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.733 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.733 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.733 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.733 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311239+0000", 2024-05-08T08:53:13.733 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.734 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.734 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.734 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.734 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311239+0000", 2024-05-08T08:53:13.734 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5699999999999998e-07 2024-05-08T08:53:13.734 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.734 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.734 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.734 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311241+0000", 2024-05-08T08:53:13.734 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.979e-06 2024-05-08T08:53:13.734 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.734 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.735 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.735 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311248+0000", 2024-05-08T08:53:13.735 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8530000000000003e-06 2024-05-08T08:53:13.735 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.735 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.735 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.735 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311252+0000", 2024-05-08T08:53:13.735 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4179999999999999e-06 2024-05-08T08:53:13.735 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.735 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.735 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.735 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.735 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.735 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27029 3.4 3.134fb004 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.735 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.290037+0000", 2024-05-08T08:53:13.736 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.024442635000000001, 2024-05-08T08:53:13.736 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.031339552, 2024-05-08T08:53:13.736 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.736 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.736 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.736 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.736 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.736 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.736 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27029 2024-05-08T08:53:13.736 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.736 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.736 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.736 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.736 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.290037+0000", 2024-05-08T08:53:13.736 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.737 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.737 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.737 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.737 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.290037+0000", 2024-05-08T08:53:13.737 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.737 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.737 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.737 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.737 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.290037+0000", 2024-05-08T08:53:13.737 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7099999999999997e-07 2024-05-08T08:53:13.737 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.737 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.737 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.737 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.290039+0000", 2024-05-08T08:53:13.737 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1160000000000002e-06 2024-05-08T08:53:13.738 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.738 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.738 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.738 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.290045+0000", 2024-05-08T08:53:13.738 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9839999999999998e-06 2024-05-08T08:53:13.738 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.738 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.738 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.738 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.290051+0000", 2024-05-08T08:53:13.738 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7080000000000002e-06 2024-05-08T08:53:13.738 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.738 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.738 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.738 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.739 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.739 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27244 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.739 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.302294+0000", 2024-05-08T08:53:13.739 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.012185204999999999, 2024-05-08T08:53:13.739 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.019117999, 2024-05-08T08:53:13.739 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.739 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.739 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.739 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.739 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.739 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.739 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27244 2024-05-08T08:53:13.739 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.739 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.739 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.740 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.740 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302294+0000", 2024-05-08T08:53:13.740 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.740 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.740 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.740 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.740 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302294+0000", 2024-05-08T08:53:13.740 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.740 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.740 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.740 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.740 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302294+0000", 2024-05-08T08:53:13.740 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2399999999999999e-07 2024-05-08T08:53:13.740 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.740 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.740 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.741 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302301+0000", 2024-05-08T08:53:13.741 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8530000000000003e-06 2024-05-08T08:53:13.741 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.741 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.741 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.741 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302308+0000", 2024-05-08T08:53:13.741 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7780000000000002e-06 2024-05-08T08:53:13.741 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.741 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.741 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.741 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302313+0000", 2024-05-08T08:53:13.741 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0719999999999999e-06 2024-05-08T08:53:13.741 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.741 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.742 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.742 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.742 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.742 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27351 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.742 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.303044+0000", 2024-05-08T08:53:13.742 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.011435486999999999, 2024-05-08T08:53:13.742 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.018399728000000001, 2024-05-08T08:53:13.742 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.742 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.742 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.742 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.742 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.742 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.743 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27351 2024-05-08T08:53:13.743 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.743 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.743 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.743 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.743 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303044+0000", 2024-05-08T08:53:13.743 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.743 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.743 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.743 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.743 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303044+0000", 2024-05-08T08:53:13.743 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.743 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.743 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.743 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.744 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303044+0000", 2024-05-08T08:53:13.744 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0699999999999998e-07 2024-05-08T08:53:13.744 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.744 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.744 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.744 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303047+0000", 2024-05-08T08:53:13.744 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7609999999999999e-06 2024-05-08T08:53:13.744 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.744 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.744 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.744 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303053+0000", 2024-05-08T08:53:13.744 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9529999999999996e-06 2024-05-08T08:53:13.745 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.745 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.745 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.745 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303058+0000", 2024-05-08T08:53:13.745 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9119999999999997e-06 2024-05-08T08:53:13.745 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.745 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.745 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.745 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.745 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.745 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27594 3.4 3.134fb004 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.745 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.311258+0000", 2024-05-08T08:53:13.745 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0032209489999999999, 2024-05-08T08:53:13.746 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.010239791999999999, 2024-05-08T08:53:13.746 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.746 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.746 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.746 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.746 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.746 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.746 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27594 2024-05-08T08:53:13.746 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.746 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.746 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.746 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.746 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311258+0000", 2024-05-08T08:53:13.747 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.747 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.747 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.747 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.747 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311258+0000", 2024-05-08T08:53:13.747 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.747 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.747 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.747 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.747 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311259+0000", 2024-05-08T08:53:13.747 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3799999999999998e-07 2024-05-08T08:53:13.747 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.747 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.747 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.748 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311261+0000", 2024-05-08T08:53:13.748 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0370000000000001e-06 2024-05-08T08:53:13.748 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.748 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.748 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.748 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311266+0000", 2024-05-08T08:53:13.748 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8470000000000001e-06 2024-05-08T08:53:13.748 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.748 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.748 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.748 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311271+0000", 2024-05-08T08:53:13.748 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7899999999999999e-06 2024-05-08T08:53:13.748 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.748 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.749 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.749 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.749 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.749 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27245 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.749 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.302321+0000", 2024-05-08T08:53:13.749 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.01215839, 2024-05-08T08:53:13.749 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.019232978000000001, 2024-05-08T08:53:13.749 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.749 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.749 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.749 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.749 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.749 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.749 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27245 2024-05-08T08:53:13.749 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.750 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.750 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.750 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.750 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302321+0000", 2024-05-08T08:53:13.750 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.750 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.750 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.750 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.750 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302321+0000", 2024-05-08T08:53:13.750 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.750 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.750 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.750 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.751 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302321+0000", 2024-05-08T08:53:13.751 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4400000000000001e-07 2024-05-08T08:53:13.751 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.751 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.751 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.751 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302324+0000", 2024-05-08T08:53:13.751 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4600000000000002e-06 2024-05-08T08:53:13.751 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.751 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.751 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.751 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302330+0000", 2024-05-08T08:53:13.751 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3840000000000002e-06 2024-05-08T08:53:13.751 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.752 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.752 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.752 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302336+0000", 2024-05-08T08:53:13.752 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2720000000000003e-06 2024-05-08T08:53:13.752 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.752 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.752 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.752 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.752 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.752 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27357 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.752 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.303064+0000", 2024-05-08T08:53:13.752 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.011415028000000001, 2024-05-08T08:53:13.752 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.018545496000000002, 2024-05-08T08:53:13.753 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.753 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.753 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.753 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.753 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.753 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.753 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27357 2024-05-08T08:53:13.753 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.753 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.753 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.753 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.753 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303064+0000", 2024-05-08T08:53:13.753 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.753 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.753 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.754 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.754 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303064+0000", 2024-05-08T08:53:13.754 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.754 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.754 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.754 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.754 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303065+0000", 2024-05-08T08:53:13.754 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.01e-07 2024-05-08T08:53:13.754 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.754 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.754 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.754 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303067+0000", 2024-05-08T08:53:13.754 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.5639999999999999e-06 2024-05-08T08:53:13.755 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.755 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.755 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.755 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303073+0000", 2024-05-08T08:53:13.755 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8259999999999998e-06 2024-05-08T08:53:13.755 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.755 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.755 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.755 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303078+0000", 2024-05-08T08:53:13.755 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8010000000000003e-06 2024-05-08T08:53:13.755 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.755 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.755 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.755 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.755 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.756 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27417 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.756 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.304976+0000", 2024-05-08T08:53:13.756 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0095036610000000001, 2024-05-08T08:53:13.756 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.016690608999999999, 2024-05-08T08:53:13.756 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.756 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.756 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.756 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.756 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.756 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.756 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27417 2024-05-08T08:53:13.756 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.756 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.756 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.756 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.756 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.304976+0000", 2024-05-08T08:53:13.757 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.757 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.757 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.757 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.757 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.304976+0000", 2024-05-08T08:53:13.757 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.757 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.757 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.757 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.757 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.304976+0000", 2024-05-08T08:53:13.757 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8100000000000003e-07 2024-05-08T08:53:13.757 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.757 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.757 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.757 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.304979+0000", 2024-05-08T08:53:13.757 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.2650000000000001e-06 2024-05-08T08:53:13.758 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.758 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.758 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.758 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.304986+0000", 2024-05-08T08:53:13.758 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5050000000000004e-06 2024-05-08T08:53:13.758 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.758 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.758 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.758 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.304991+0000", 2024-05-08T08:53:13.758 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.882e-06 2024-05-08T08:53:13.758 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.758 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.758 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.758 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.759 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.759 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27595 3.4 3.134fb004 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.759 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.311288+0000", 2024-05-08T08:53:13.759 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0031914669999999999, 2024-05-08T08:53:13.759 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.010434643, 2024-05-08T08:53:13.759 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.759 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.759 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.759 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.759 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.759 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.759 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27595 2024-05-08T08:53:13.759 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.759 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.759 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.759 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.760 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311288+0000", 2024-05-08T08:53:13.760 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.760 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.760 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.760 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.760 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311288+0000", 2024-05-08T08:53:13.760 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.760 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.760 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.760 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.760 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311288+0000", 2024-05-08T08:53:13.760 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4700000000000001e-07 2024-05-08T08:53:13.760 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.760 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.760 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.761 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311291+0000", 2024-05-08T08:53:13.761 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.5170000000000002e-06 2024-05-08T08:53:13.761 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.761 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.761 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.761 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311297+0000", 2024-05-08T08:53:13.761 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3589999999999996e-06 2024-05-08T08:53:13.761 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.761 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.761 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.761 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311302+0000", 2024-05-08T08:53:13.761 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9960000000000001e-06 2024-05-08T08:53:13.761 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.761 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.761 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.761 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.761 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.762 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27246 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.762 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.302344+0000", 2024-05-08T08:53:13.762 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.012135527, 2024-05-08T08:53:13.762 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.019433477000000001, 2024-05-08T08:53:13.762 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.762 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.762 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.762 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.762 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.762 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.762 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27246 2024-05-08T08:53:13.762 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.762 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.762 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.762 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.763 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302344+0000", 2024-05-08T08:53:13.763 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.763 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.763 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.763 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.763 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302344+0000", 2024-05-08T08:53:13.763 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.763 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.763 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.763 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.763 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302344+0000", 2024-05-08T08:53:13.763 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1199999999999998e-07 2024-05-08T08:53:13.763 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.763 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.763 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.763 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302347+0000", 2024-05-08T08:53:13.763 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8289999999999999e-06 2024-05-08T08:53:13.764 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.764 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.764 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.764 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302355+0000", 2024-05-08T08:53:13.764 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.6179999999999997e-06 2024-05-08T08:53:13.764 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.764 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.764 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.764 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302360+0000", 2024-05-08T08:53:13.764 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4779999999999999e-06 2024-05-08T08:53:13.764 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.764 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.764 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.764 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.764 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.764 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27360 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.765 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.303084+0000", 2024-05-08T08:53:13.765 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.011395438000000001, 2024-05-08T08:53:13.765 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.018750718, 2024-05-08T08:53:13.765 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.765 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.765 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.765 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.765 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.765 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.765 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27360 2024-05-08T08:53:13.765 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.765 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.765 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.765 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.765 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303084+0000", 2024-05-08T08:53:13.765 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.765 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.766 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.766 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.766 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303084+0000", 2024-05-08T08:53:13.766 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.766 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.766 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.766 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.766 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303084+0000", 2024-05-08T08:53:13.766 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5499999999999999e-07 2024-05-08T08:53:13.766 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.766 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.766 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.766 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303087+0000", 2024-05-08T08:53:13.766 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7030000000000002e-06 2024-05-08T08:53:13.766 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.767 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.767 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.767 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303093+0000", 2024-05-08T08:53:13.767 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3720000000000001e-06 2024-05-08T08:53:13.767 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.767 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.767 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.767 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303098+0000", 2024-05-08T08:53:13.767 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9540000000000003e-06 2024-05-08T08:53:13.767 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.767 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.767 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.767 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.767 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.767 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27596 3.4 3.134fb004 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.767 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.311309+0000", 2024-05-08T08:53:13.767 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0031704020000000001, 2024-05-08T08:53:13.768 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.010583156, 2024-05-08T08:53:13.768 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.768 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.768 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.768 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.768 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.768 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.768 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27596 2024-05-08T08:53:13.768 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.768 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.768 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.768 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.768 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311309+0000", 2024-05-08T08:53:13.768 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.768 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.769 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.769 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.769 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311309+0000", 2024-05-08T08:53:13.769 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.769 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.769 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.769 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.769 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311309+0000", 2024-05-08T08:53:13.769 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2300000000000002e-07 2024-05-08T08:53:13.769 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.769 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.769 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.769 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311311+0000", 2024-05-08T08:53:13.769 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.035e-06 2024-05-08T08:53:13.769 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.769 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.769 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.770 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311317+0000", 2024-05-08T08:53:13.770 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6010000000000002e-06 2024-05-08T08:53:13.770 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.770 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.770 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.770 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311321+0000", 2024-05-08T08:53:13.770 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5220000000000001e-06 2024-05-08T08:53:13.770 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.770 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.770 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.770 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.770 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.770 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:26509 3.4 3:200df2c8:::rbd_data.1140100c9703.0000000000000002:14 [read 1682432~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.770 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.272227+0000", 2024-05-08T08:53:13.771 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.042251974999999997, 2024-05-08T08:53:13.771 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.049731549, 2024-05-08T08:53:13.771 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.771 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.771 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:13.771 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.771 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.771 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.771 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 26509 2024-05-08T08:53:13.771 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.771 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.771 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.771 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.771 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.272227+0000", 2024-05-08T08:53:13.771 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.772 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.772 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.772 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.772 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.272227+0000", 2024-05-08T08:53:13.772 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.772 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.772 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.772 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.772 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.272228+0000", 2024-05-08T08:53:13.772 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0500000000000003e-07 2024-05-08T08:53:13.772 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.772 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.772 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.772 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.272238+0000", 2024-05-08T08:53:13.773 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8640000000000008e-06 2024-05-08T08:53:13.773 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.773 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.773 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.773 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.274562+0000", 2024-05-08T08:53:13.773 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0023237549999999998 2024-05-08T08:53:13.773 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.773 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.773 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.773 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.274569+0000", 2024-05-08T08:53:13.773 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9879999999999998e-06 2024-05-08T08:53:13.773 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.773 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.773 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:13.773 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.321849+0000", 2024-05-08T08:53:13.773 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.047280741000000001 2024-05-08T08:53:13.774 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.774 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.774 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:13.774 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.321920+0000", 2024-05-08T08:53:13.774 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1227000000000001e-05 2024-05-08T08:53:13.774 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.774 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.774 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.774 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.774 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.774 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27283 3.4 3.134fb004 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.774 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.302367+0000", 2024-05-08T08:53:13.774 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.012111985, 2024-05-08T08:53:13.774 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.019700592999999999, 2024-05-08T08:53:13.775 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.775 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.775 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.775 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.775 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.775 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.775 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27283 2024-05-08T08:53:13.775 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.775 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.775 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.775 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.775 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302367+0000", 2024-05-08T08:53:13.775 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.775 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.775 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.775 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.775 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302367+0000", 2024-05-08T08:53:13.776 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.776 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.776 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.776 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.776 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302368+0000", 2024-05-08T08:53:13.776 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9500000000000002e-07 2024-05-08T08:53:13.776 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.776 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.776 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.776 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302370+0000", 2024-05-08T08:53:13.776 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6699999999999998e-06 2024-05-08T08:53:13.776 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.777 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.777 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.777 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302377+0000", 2024-05-08T08:53:13.777 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4099999999999996e-06 2024-05-08T08:53:13.777 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.777 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.777 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.777 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302382+0000", 2024-05-08T08:53:13.777 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7190000000000001e-06 2024-05-08T08:53:13.777 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.777 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.777 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.777 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.777 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.777 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27366 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.777 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.303173+0000", 2024-05-08T08:53:13.778 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.011305934, 2024-05-08T08:53:13.778 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.018951108000000001, 2024-05-08T08:53:13.778 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.778 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.778 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.778 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.778 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.778 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.778 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27366 2024-05-08T08:53:13.778 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.778 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.778 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.778 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.778 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303173+0000", 2024-05-08T08:53:13.779 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.779 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.779 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.779 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.779 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303173+0000", 2024-05-08T08:53:13.779 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.779 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.779 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.779 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.779 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303174+0000", 2024-05-08T08:53:13.779 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8400000000000004e-07 2024-05-08T08:53:13.779 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.779 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.779 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.779 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303178+0000", 2024-05-08T08:53:13.779 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.692e-06 2024-05-08T08:53:13.780 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.780 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.780 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.780 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303188+0000", 2024-05-08T08:53:13.780 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0400999999999999e-05 2024-05-08T08:53:13.780 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.780 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.780 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.780 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303194+0000", 2024-05-08T08:53:13.780 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.567e-06 2024-05-08T08:53:13.780 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.780 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.780 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.780 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.780 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.780 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27419 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.781 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.305018+0000", 2024-05-08T08:53:13.781 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0094611729999999998, 2024-05-08T08:53:13.781 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.017169654999999999, 2024-05-08T08:53:13.781 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.781 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.781 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.781 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.781 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.781 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.781 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27419 2024-05-08T08:53:13.781 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.781 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.781 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.781 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.781 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.305018+0000", 2024-05-08T08:53:13.782 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.782 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.782 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.782 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.782 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.305018+0000", 2024-05-08T08:53:13.782 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.782 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.782 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.782 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.782 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.305018+0000", 2024-05-08T08:53:13.782 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2e-07 2024-05-08T08:53:13.782 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.782 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.782 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.782 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.305021+0000", 2024-05-08T08:53:13.783 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.5739999999999998e-06 2024-05-08T08:53:13.783 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.783 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.783 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.783 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.305027+0000", 2024-05-08T08:53:13.783 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1859999999999997e-06 2024-05-08T08:53:13.783 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.783 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.783 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.783 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.305032+0000", 2024-05-08T08:53:13.783 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6410000000000001e-06 2024-05-08T08:53:13.783 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.783 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.783 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.783 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.784 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.784 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27597 3.4 3.134fb004 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.784 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.311327+0000", 2024-05-08T08:53:13.784 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0031519439999999998, 2024-05-08T08:53:13.784 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.010916426, 2024-05-08T08:53:13.784 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.784 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.784 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.784 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.784 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.784 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.784 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27597 2024-05-08T08:53:13.784 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.784 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.785 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.785 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.785 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311327+0000", 2024-05-08T08:53:13.785 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.785 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.785 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.785 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.785 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311327+0000", 2024-05-08T08:53:13.785 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.785 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.785 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.785 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.785 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311328+0000", 2024-05-08T08:53:13.785 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3500000000000002e-07 2024-05-08T08:53:13.785 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.786 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.786 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.786 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311330+0000", 2024-05-08T08:53:13.786 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0779999999999998e-06 2024-05-08T08:53:13.786 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.786 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.786 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.786 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311335+0000", 2024-05-08T08:53:13.786 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5679999999999999e-06 2024-05-08T08:53:13.786 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.786 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.786 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.786 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311340+0000", 2024-05-08T08:53:13.786 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6249999999999998e-06 2024-05-08T08:53:13.787 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.787 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.787 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.787 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.787 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.787 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27284 3.4 3.134fb004 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.787 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.302388+0000", 2024-05-08T08:53:13.787 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.012090979999999999, 2024-05-08T08:53:13.787 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.019916752999999999, 2024-05-08T08:53:13.787 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.787 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.787 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.787 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.787 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.787 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.787 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27284 2024-05-08T08:53:13.788 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.788 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.788 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.788 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.788 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302388+0000", 2024-05-08T08:53:13.788 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.788 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.788 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.788 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.788 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302388+0000", 2024-05-08T08:53:13.788 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.788 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.788 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.788 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.788 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302389+0000", 2024-05-08T08:53:13.789 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9399999999999995e-07 2024-05-08T08:53:13.789 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.789 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.789 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.789 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302391+0000", 2024-05-08T08:53:13.789 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.295e-06 2024-05-08T08:53:13.789 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.789 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.789 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.789 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302397+0000", 2024-05-08T08:53:13.789 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0630000000000002e-06 2024-05-08T08:53:13.789 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.789 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.790 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.790 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302402+0000", 2024-05-08T08:53:13.790 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0420000000000002e-06 2024-05-08T08:53:13.790 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.790 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.790 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.790 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.790 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.790 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27372 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.790 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.303205+0000", 2024-05-08T08:53:13.790 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.011274510999999999, 2024-05-08T08:53:13.790 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.019157240999999998, 2024-05-08T08:53:13.790 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.791 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.791 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.791 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.791 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.791 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.791 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27372 2024-05-08T08:53:13.791 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.791 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.791 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.791 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.791 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303205+0000", 2024-05-08T08:53:13.791 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.791 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.791 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.791 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.792 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303205+0000", 2024-05-08T08:53:13.792 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.792 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.792 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.792 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.792 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303205+0000", 2024-05-08T08:53:13.792 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4000000000000002e-07 2024-05-08T08:53:13.792 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.792 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.792 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.792 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303215+0000", 2024-05-08T08:53:13.792 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8390000000000002e-06 2024-05-08T08:53:13.792 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.792 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.792 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.792 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303223+0000", 2024-05-08T08:53:13.793 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7940000000000003e-06 2024-05-08T08:53:13.793 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.793 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.793 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.793 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303229+0000", 2024-05-08T08:53:13.793 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8109999999999999e-06 2024-05-08T08:53:13.793 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.793 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.793 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.793 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.793 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.793 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27598 3.4 3.134fb004 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.793 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.311346+0000", 2024-05-08T08:53:13.793 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0031335320000000001, 2024-05-08T08:53:13.793 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.011071892, 2024-05-08T08:53:13.793 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.793 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.794 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.794 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.794 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.794 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.794 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27598 2024-05-08T08:53:13.794 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.794 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.794 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.794 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.794 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311346+0000", 2024-05-08T08:53:13.794 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.795 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.795 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.795 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.795 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311346+0000", 2024-05-08T08:53:13.795 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.795 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.795 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.795 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.795 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311346+0000", 2024-05-08T08:53:13.795 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.2800000000000003e-07 2024-05-08T08:53:13.795 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.795 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.795 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.795 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311348+0000", 2024-05-08T08:53:13.795 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.8109999999999999e-06 2024-05-08T08:53:13.795 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.796 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.796 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.796 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311353+0000", 2024-05-08T08:53:13.796 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5350000000000004e-06 2024-05-08T08:53:13.796 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.796 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.796 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.796 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311360+0000", 2024-05-08T08:53:13.796 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3300000000000004e-06 2024-05-08T08:53:13.796 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.796 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.796 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.796 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.796 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.796 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:26510 3.4 3:200df2c8:::rbd_data.1140100c9703.0000000000000002:14 [read 1691648~1024 out=1024b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.797 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.277324+0000", 2024-05-08T08:53:13.797 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.03715504, 2024-05-08T08:53:13.797 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.045158721999999998, 2024-05-08T08:53:13.797 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.797 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.797 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:13.797 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.797 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.797 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.797 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 26510 2024-05-08T08:53:13.797 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.797 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.797 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.797 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.797 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.277324+0000", 2024-05-08T08:53:13.798 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.798 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.798 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.798 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.798 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.277324+0000", 2024-05-08T08:53:13.798 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.798 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.798 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.798 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.798 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.277325+0000", 2024-05-08T08:53:13.798 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0999999999999998e-07 2024-05-08T08:53:13.798 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.799 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.799 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.799 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.277329+0000", 2024-05-08T08:53:13.799 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4020000000000004e-06 2024-05-08T08:53:13.799 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.799 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.799 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.799 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.277340+0000", 2024-05-08T08:53:13.799 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0993e-05 2024-05-08T08:53:13.799 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.799 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.799 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.799 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.277348+0000", 2024-05-08T08:53:13.799 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7330000000000003e-06 2024-05-08T08:53:13.799 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.800 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.800 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:13.800 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.322141+0000", 2024-05-08T08:53:13.800 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.044792683999999999 2024-05-08T08:53:13.800 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.800 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.800 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:13.800 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.322214+0000", 2024-05-08T08:53:13.800 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3273000000000001e-05 2024-05-08T08:53:13.800 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.800 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.800 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:13.800 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.322485+0000", 2024-05-08T08:53:13.800 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00027091499999999999 2024-05-08T08:53:13.801 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.801 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.801 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.801 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.801 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.801 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:26717 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.801 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.283584+0000", 2024-05-08T08:53:13.801 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.030894852, 2024-05-08T08:53:13.801 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.038972291999999999, 2024-05-08T08:53:13.801 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.801 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.801 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.801 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.801 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.802 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.802 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 26717 2024-05-08T08:53:13.802 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.802 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.802 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.802 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.802 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283584+0000", 2024-05-08T08:53:13.802 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.802 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.802 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.802 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.802 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283584+0000", 2024-05-08T08:53:13.802 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.802 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.803 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.803 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.803 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283585+0000", 2024-05-08T08:53:13.803 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3799999999999999e-07 2024-05-08T08:53:13.803 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.803 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.803 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.803 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283587+0000", 2024-05-08T08:53:13.803 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.041e-06 2024-05-08T08:53:13.803 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.803 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.803 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.803 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283590+0000", 2024-05-08T08:53:13.803 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1669999999999999e-06 2024-05-08T08:53:13.804 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.804 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.804 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.804 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283592+0000", 2024-05-08T08:53:13.804 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4609999999999998e-06 2024-05-08T08:53:13.804 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.804 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.804 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.804 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.804 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.804 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27285 3.4 3.134fb004 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.804 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.302409+0000", 2024-05-08T08:53:13.804 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.012070452000000001, 2024-05-08T08:53:13.804 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.020203539, 2024-05-08T08:53:13.804 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.805 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.805 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.805 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.805 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.805 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.805 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27285 2024-05-08T08:53:13.805 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.805 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.805 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.805 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.805 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302409+0000", 2024-05-08T08:53:13.805 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.805 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.805 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.805 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.806 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302409+0000", 2024-05-08T08:53:13.806 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.806 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.806 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.806 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.806 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302409+0000", 2024-05-08T08:53:13.806 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.39e-07 2024-05-08T08:53:13.806 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.806 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.806 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.806 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302411+0000", 2024-05-08T08:53:13.806 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0559999999999999e-06 2024-05-08T08:53:13.806 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.806 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.807 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.807 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302417+0000", 2024-05-08T08:53:13.807 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0109999999999997e-06 2024-05-08T08:53:13.807 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.807 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.807 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.807 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302423+0000", 2024-05-08T08:53:13.807 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7960000000000001e-06 2024-05-08T08:53:13.807 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.807 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.807 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.807 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.807 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.807 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27376 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.808 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.303236+0000", 2024-05-08T08:53:13.808 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.011242825999999999, 2024-05-08T08:53:13.808 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.019432646000000001, 2024-05-08T08:53:13.808 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.808 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.808 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.808 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.808 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.808 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.808 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27376 2024-05-08T08:53:13.808 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.808 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.808 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.809 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.809 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303236+0000", 2024-05-08T08:53:13.809 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.809 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.809 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.809 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.809 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303236+0000", 2024-05-08T08:53:13.809 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.809 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.809 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.809 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.809 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303237+0000", 2024-05-08T08:53:13.809 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8000000000000001e-07 2024-05-08T08:53:13.810 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.810 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.810 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.810 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303240+0000", 2024-05-08T08:53:13.810 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8370000000000001e-06 2024-05-08T08:53:13.810 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.810 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.810 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.810 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303246+0000", 2024-05-08T08:53:13.810 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3419999999999996e-06 2024-05-08T08:53:13.810 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.810 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.811 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.811 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303251+0000", 2024-05-08T08:53:13.811 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.69e-06 2024-05-08T08:53:13.811 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.811 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.811 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.811 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.811 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.811 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27599 3.4 3.134fb004 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.811 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.311366+0000", 2024-05-08T08:53:13.811 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0031133829999999999, 2024-05-08T08:53:13.811 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.011376002, 2024-05-08T08:53:13.811 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.812 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.812 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.812 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.812 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.812 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.812 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27599 2024-05-08T08:53:13.812 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.812 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.812 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.812 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.812 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311366+0000", 2024-05-08T08:53:13.812 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.812 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.812 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.813 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.813 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311366+0000", 2024-05-08T08:53:13.813 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.813 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.813 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.813 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.813 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311366+0000", 2024-05-08T08:53:13.813 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.34e-07 2024-05-08T08:53:13.813 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.813 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.813 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.814 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311373+0000", 2024-05-08T08:53:13.814 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8159999999999996e-06 2024-05-08T08:53:13.814 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.814 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.814 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.814 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311379+0000", 2024-05-08T08:53:13.814 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6339999999999998e-06 2024-05-08T08:53:13.814 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.814 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.814 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.814 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311383+0000", 2024-05-08T08:53:13.815 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6600000000000003e-06 2024-05-08T08:53:13.815 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.815 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.815 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.815 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.815 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.815 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:26753 3.4 3.134fb004 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.815 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.283596+0000", 2024-05-08T08:53:13.815 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.030883175999999998, 2024-05-08T08:53:13.815 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.039228805999999998, 2024-05-08T08:53:13.815 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.815 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.815 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.816 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.816 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.816 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.816 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 26753 2024-05-08T08:53:13.816 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.816 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.816 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.816 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.816 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283596+0000", 2024-05-08T08:53:13.816 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.816 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.816 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.816 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.816 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283596+0000", 2024-05-08T08:53:13.816 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.816 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.817 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.817 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.817 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283596+0000", 2024-05-08T08:53:13.817 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2000000000000001e-07 2024-05-08T08:53:13.817 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.817 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.817 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.817 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283597+0000", 2024-05-08T08:53:13.817 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1429999999999999e-06 2024-05-08T08:53:13.817 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.817 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.817 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.817 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283600+0000", 2024-05-08T08:53:13.817 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9409999999999999e-06 2024-05-08T08:53:13.818 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.818 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.818 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.818 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283604+0000", 2024-05-08T08:53:13.818 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7110000000000001e-06 2024-05-08T08:53:13.818 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.818 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.818 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.818 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.818 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.818 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:11614 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.818 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.289204+0000", 2024-05-08T08:53:13.818 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.025275590000000001, 2024-05-08T08:53:13.818 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.033678495000000003, 2024-05-08T08:53:13.819 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.819 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.819 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.819 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.819 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:13.819 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:13.819 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 11614 2024-05-08T08:53:13.819 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.819 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.819 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.819 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.819 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289204+0000", 2024-05-08T08:53:13.819 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.819 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.819 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.820 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.820 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289204+0000", 2024-05-08T08:53:13.820 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.820 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.820 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.820 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.820 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289205+0000", 2024-05-08T08:53:13.820 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1429999999999999e-06 2024-05-08T08:53:13.820 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.820 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.820 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.820 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289210+0000", 2024-05-08T08:53:13.820 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1070000000000004e-06 2024-05-08T08:53:13.820 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.820 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.821 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.821 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289221+0000", 2024-05-08T08:53:13.821 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0779e-05 2024-05-08T08:53:13.821 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.821 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.821 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.821 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.289230+0000", 2024-05-08T08:53:13.821 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.9619999999999999e-06 2024-05-08T08:53:13.821 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.821 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.821 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.821 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.821 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.821 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27286 3.4 3.134fb004 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.821 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.302430+0000", 2024-05-08T08:53:13.821 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.012049538, 2024-05-08T08:53:13.822 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.020508209999999999, 2024-05-08T08:53:13.822 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.822 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.822 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.822 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.822 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.822 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.822 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27286 2024-05-08T08:53:13.822 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.822 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.822 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.822 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.822 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302430+0000", 2024-05-08T08:53:13.822 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.823 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.823 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.823 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.823 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302430+0000", 2024-05-08T08:53:13.823 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.823 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.823 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.823 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.823 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302430+0000", 2024-05-08T08:53:13.823 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6699999999999999e-07 2024-05-08T08:53:13.823 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.823 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.823 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.823 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302432+0000", 2024-05-08T08:53:13.823 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.323e-06 2024-05-08T08:53:13.823 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.824 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.824 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.824 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302438+0000", 2024-05-08T08:53:13.824 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0290000000000002e-06 2024-05-08T08:53:13.824 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.824 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.824 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.824 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302444+0000", 2024-05-08T08:53:13.824 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.198e-06 2024-05-08T08:53:13.824 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.824 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.824 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.824 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.825 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.825 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27379 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.825 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.303259+0000", 2024-05-08T08:53:13.825 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.011220638999999999, 2024-05-08T08:53:13.825 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.019741913, 2024-05-08T08:53:13.825 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.825 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.825 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.825 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.825 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.825 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.825 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27379 2024-05-08T08:53:13.825 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.825 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.825 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.826 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.826 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303259+0000", 2024-05-08T08:53:13.826 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.826 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.826 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.826 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.826 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303259+0000", 2024-05-08T08:53:13.826 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.826 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.826 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.826 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.826 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303259+0000", 2024-05-08T08:53:13.826 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.72e-07 2024-05-08T08:53:13.826 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.827 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.827 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.827 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303262+0000", 2024-05-08T08:53:13.827 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7870000000000002e-06 2024-05-08T08:53:13.827 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.827 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.827 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.827 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303268+0000", 2024-05-08T08:53:13.827 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2330000000000003e-06 2024-05-08T08:53:13.827 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.827 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.827 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.827 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303273+0000", 2024-05-08T08:53:13.827 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.198e-06 2024-05-08T08:53:13.827 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.827 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.828 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.828 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.828 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.828 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27600 3.4 3.134fb004 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.828 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.311391+0000", 2024-05-08T08:53:13.828 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0030882499999999998, 2024-05-08T08:53:13.828 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.011667422, 2024-05-08T08:53:13.828 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.828 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.828 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.828 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.828 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.828 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.828 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27600 2024-05-08T08:53:13.828 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.829 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.829 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.829 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.829 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311391+0000", 2024-05-08T08:53:13.829 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.829 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.829 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.829 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.829 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311391+0000", 2024-05-08T08:53:13.829 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.829 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.829 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.829 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.829 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311391+0000", 2024-05-08T08:53:13.829 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.15e-07 2024-05-08T08:53:13.829 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.830 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.830 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.830 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311394+0000", 2024-05-08T08:53:13.830 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3010000000000001e-06 2024-05-08T08:53:13.830 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.830 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.830 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.830 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311400+0000", 2024-05-08T08:53:13.830 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6680000000000004e-06 2024-05-08T08:53:13.830 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.830 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.830 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.830 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311406+0000", 2024-05-08T08:53:13.830 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2530000000000004e-06 2024-05-08T08:53:13.831 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.831 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.831 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.831 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.831 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.831 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:26754 3.4 3.134fb004 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.831 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.283610+0000", 2024-05-08T08:53:13.831 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.030868782000000001, 2024-05-08T08:53:13.831 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.039500254999999998, 2024-05-08T08:53:13.831 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.831 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.831 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.831 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.831 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.832 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.832 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 26754 2024-05-08T08:53:13.832 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.832 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.832 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.832 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.832 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283610+0000", 2024-05-08T08:53:13.832 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.832 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.832 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.832 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.832 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283610+0000", 2024-05-08T08:53:13.832 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.832 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.832 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.833 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.833 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283611+0000", 2024-05-08T08:53:13.833 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1199999999999999e-07 2024-05-08T08:53:13.833 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.833 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.833 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.833 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283612+0000", 2024-05-08T08:53:13.833 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.243e-06 2024-05-08T08:53:13.833 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.833 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.833 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.833 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283615+0000", 2024-05-08T08:53:13.833 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1669999999999999e-06 2024-05-08T08:53:13.833 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.833 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.834 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.834 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283618+0000", 2024-05-08T08:53:13.834 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.2590000000000001e-06 2024-05-08T08:53:13.834 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.834 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.834 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.834 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.834 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.834 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27287 3.4 3.134fb004 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.834 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.302450+0000", 2024-05-08T08:53:13.834 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.01202949, 2024-05-08T08:53:13.834 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.020720816, 2024-05-08T08:53:13.834 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.834 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.835 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.835 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.835 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.835 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.835 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27287 2024-05-08T08:53:13.835 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.835 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.835 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.835 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.835 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302450+0000", 2024-05-08T08:53:13.835 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.835 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.835 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.835 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.835 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302450+0000", 2024-05-08T08:53:13.836 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.836 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.836 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.836 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.836 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302450+0000", 2024-05-08T08:53:13.836 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9099999999999999e-07 2024-05-08T08:53:13.836 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.836 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.836 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.836 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302452+0000", 2024-05-08T08:53:13.836 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9080000000000002e-06 2024-05-08T08:53:13.836 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.836 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.836 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.836 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302458+0000", 2024-05-08T08:53:13.836 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0499999999999997e-06 2024-05-08T08:53:13.837 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.837 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.837 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.837 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302463+0000", 2024-05-08T08:53:13.837 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0100000000000003e-06 2024-05-08T08:53:13.837 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.837 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.837 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.837 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.837 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.837 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27381 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.837 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.303280+0000", 2024-05-08T08:53:13.837 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.011199218, 2024-05-08T08:53:13.837 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.019946247, 2024-05-08T08:53:13.837 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.838 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.838 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.838 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.838 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.838 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.838 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27381 2024-05-08T08:53:13.838 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.838 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.838 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.838 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.838 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303280+0000", 2024-05-08T08:53:13.838 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.838 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.838 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.838 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.839 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303280+0000", 2024-05-08T08:53:13.839 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.839 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.839 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.839 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.839 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303280+0000", 2024-05-08T08:53:13.839 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.46e-07 2024-05-08T08:53:13.839 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.839 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.839 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.839 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303284+0000", 2024-05-08T08:53:13.839 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1609999999999996e-06 2024-05-08T08:53:13.839 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.839 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.839 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.839 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303292+0000", 2024-05-08T08:53:13.839 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4789999999999999e-06 2024-05-08T08:53:13.840 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.840 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.840 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.840 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303297+0000", 2024-05-08T08:53:13.840 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9810000000000003e-06 2024-05-08T08:53:13.840 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.840 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.840 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.840 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.840 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.840 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27601 3.4 3.134fb004 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.840 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.311412+0000", 2024-05-08T08:53:13.840 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0030666920000000002, 2024-05-08T08:53:13.840 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.011868035000000001, 2024-05-08T08:53:13.841 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.841 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.841 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.841 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.841 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.841 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.841 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27601 2024-05-08T08:53:13.841 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.841 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.841 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.841 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.841 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311412+0000", 2024-05-08T08:53:13.841 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.841 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.841 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.842 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.842 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311412+0000", 2024-05-08T08:53:13.842 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.842 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.842 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.842 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.842 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311413+0000", 2024-05-08T08:53:13.842 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5199999999999998e-07 2024-05-08T08:53:13.842 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.842 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.842 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.842 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311415+0000", 2024-05-08T08:53:13.842 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1449999999999998e-06 2024-05-08T08:53:13.842 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.842 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.842 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.843 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311421+0000", 2024-05-08T08:53:13.843 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4289999999999998e-06 2024-05-08T08:53:13.843 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.843 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.843 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.843 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.311426+0000", 2024-05-08T08:53:13.843 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9810000000000003e-06 2024-05-08T08:53:13.843 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.843 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.843 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.843 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.843 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.843 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:26755 3.4 3.134fb004 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.843 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.283622+0000", 2024-05-08T08:53:13.843 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.030857465000000001, 2024-05-08T08:53:13.844 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.039713225999999997, 2024-05-08T08:53:13.844 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.844 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.844 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.844 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.844 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.844 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.844 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 26755 2024-05-08T08:53:13.844 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.844 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.844 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.844 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.844 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283622+0000", 2024-05-08T08:53:13.844 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.844 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.845 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.845 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.845 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283622+0000", 2024-05-08T08:53:13.845 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.845 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.845 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.845 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.845 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283622+0000", 2024-05-08T08:53:13.845 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0900000000000001e-07 2024-05-08T08:53:13.845 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.845 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.845 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.845 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283623+0000", 2024-05-08T08:53:13.845 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1689999999999999e-06 2024-05-08T08:53:13.845 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.845 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.846 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.846 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283627+0000", 2024-05-08T08:53:13.846 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.676e-06 2024-05-08T08:53:13.846 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.846 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.846 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.846 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283629+0000", 2024-05-08T08:53:13.846 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.605e-06 2024-05-08T08:53:13.846 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.846 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.846 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.846 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.846 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.846 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27288 3.4 3.134fb004 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.846 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.302470+0000", 2024-05-08T08:53:13.846 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.012009651999999999, 2024-05-08T08:53:13.847 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.020920234999999999, 2024-05-08T08:53:13.847 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.847 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.847 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.847 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.847 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.847 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.847 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27288 2024-05-08T08:53:13.847 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.847 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.847 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.847 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.847 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302470+0000", 2024-05-08T08:53:13.847 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.847 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.847 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.847 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.848 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302470+0000", 2024-05-08T08:53:13.848 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.848 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.848 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.848 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.848 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302470+0000", 2024-05-08T08:53:13.848 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.89e-07 2024-05-08T08:53:13.848 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.848 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.848 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.848 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302472+0000", 2024-05-08T08:53:13.848 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0509999999999999e-06 2024-05-08T08:53:13.848 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.848 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.848 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.848 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302478+0000", 2024-05-08T08:53:13.849 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9050000000000002e-06 2024-05-08T08:53:13.849 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.849 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.849 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.849 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302483+0000", 2024-05-08T08:53:13.849 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7849999999999999e-06 2024-05-08T08:53:13.849 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.849 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.849 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.849 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.849 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.849 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27383 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.849 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.303305+0000", 2024-05-08T08:53:13.849 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.011174307, 2024-05-08T08:53:13.849 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.020139909000000001, 2024-05-08T08:53:13.850 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.850 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.850 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.850 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.850 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.850 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.850 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27383 2024-05-08T08:53:13.850 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.850 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.850 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.850 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.850 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303305+0000", 2024-05-08T08:53:13.850 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.850 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.850 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.851 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.851 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303305+0000", 2024-05-08T08:53:13.851 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.851 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.851 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.851 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.851 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303305+0000", 2024-05-08T08:53:13.851 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7899999999999998e-07 2024-05-08T08:53:13.851 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.851 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.851 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.851 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303308+0000", 2024-05-08T08:53:13.851 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.779e-06 2024-05-08T08:53:13.851 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.851 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.851 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.852 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303316+0000", 2024-05-08T08:53:13.852 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7409999999999992e-06 2024-05-08T08:53:13.852 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.852 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.852 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.852 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303521+0000", 2024-05-08T08:53:13.852 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00020498399999999999 2024-05-08T08:53:13.852 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.852 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.852 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.852 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.852 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.852 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:26756 3.4 3.134fb004 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.852 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.283633+0000", 2024-05-08T08:53:13.852 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.03084657, 2024-05-08T08:53:13.852 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.039869092000000002, 2024-05-08T08:53:13.853 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.853 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.853 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.853 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.853 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.853 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.853 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 26756 2024-05-08T08:53:13.853 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.853 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.853 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.853 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.853 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283633+0000", 2024-05-08T08:53:13.853 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.853 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.853 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.854 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.854 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283633+0000", 2024-05-08T08:53:13.854 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.854 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.854 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.854 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.854 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283633+0000", 2024-05-08T08:53:13.854 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.04e-07 2024-05-08T08:53:13.854 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.854 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.854 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.854 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283634+0000", 2024-05-08T08:53:13.855 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1400000000000001e-06 2024-05-08T08:53:13.855 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.855 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.855 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.855 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283637+0000", 2024-05-08T08:53:13.855 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.033e-06 2024-05-08T08:53:13.855 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.855 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.855 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.855 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283640+0000", 2024-05-08T08:53:13.855 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6220000000000001e-06 2024-05-08T08:53:13.855 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.855 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.855 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.856 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.856 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.856 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27289 3.4 3.134fb004 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.856 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.302489+0000", 2024-05-08T08:53:13.856 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.011990417, 2024-05-08T08:53:13.856 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.021071006, 2024-05-08T08:53:13.856 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.856 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.856 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.856 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.856 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.856 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.856 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27289 2024-05-08T08:53:13.856 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.856 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.857 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.857 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.857 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302489+0000", 2024-05-08T08:53:13.857 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.857 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.857 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.857 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.857 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302489+0000", 2024-05-08T08:53:13.857 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.857 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.857 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.857 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.857 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302489+0000", 2024-05-08T08:53:13.857 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:53:13.857 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.857 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.858 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.858 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302491+0000", 2024-05-08T08:53:13.858 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9149999999999999e-06 2024-05-08T08:53:13.858 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.858 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.858 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.858 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302497+0000", 2024-05-08T08:53:13.858 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.964e-06 2024-05-08T08:53:13.858 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.858 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.858 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.858 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302503+0000", 2024-05-08T08:53:13.858 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.502e-06 2024-05-08T08:53:13.858 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.858 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.859 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.859 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.859 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.859 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27386 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.859 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.303535+0000", 2024-05-08T08:53:13.859 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.010944103, 2024-05-08T08:53:13.859 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.020082499, 2024-05-08T08:53:13.859 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.859 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.859 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.859 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.859 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.859 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.859 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27386 2024-05-08T08:53:13.859 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.859 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.859 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.860 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.860 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303535+0000", 2024-05-08T08:53:13.860 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.860 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.860 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.860 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.860 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303535+0000", 2024-05-08T08:53:13.860 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.860 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.860 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.860 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.860 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303536+0000", 2024-05-08T08:53:13.860 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4199999999999995e-07 2024-05-08T08:53:13.860 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.860 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.860 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.860 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303540+0000", 2024-05-08T08:53:13.861 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4479999999999996e-06 2024-05-08T08:53:13.861 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.861 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.861 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.861 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303548+0000", 2024-05-08T08:53:13.861 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8329999999999994e-06 2024-05-08T08:53:13.861 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.861 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.861 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.861 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303554+0000", 2024-05-08T08:53:13.861 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3940000000000001e-06 2024-05-08T08:53:13.861 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.861 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.861 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.861 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.861 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.861 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27442 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.862 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.305359+0000", 2024-05-08T08:53:13.862 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0091200940000000005, 2024-05-08T08:53:13.862 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.018317304, 2024-05-08T08:53:13.862 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.862 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.862 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.862 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.862 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.862 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.862 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27442 2024-05-08T08:53:13.862 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.862 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.862 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.862 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.862 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.305359+0000", 2024-05-08T08:53:13.863 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.863 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.863 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.863 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.863 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.305359+0000", 2024-05-08T08:53:13.863 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.863 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.863 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.863 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.863 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.305360+0000", 2024-05-08T08:53:13.863 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.08e-07 2024-05-08T08:53:13.863 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.863 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.863 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.864 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.305362+0000", 2024-05-08T08:53:13.864 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7329999999999999e-06 2024-05-08T08:53:13.864 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.864 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.864 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.864 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.305369+0000", 2024-05-08T08:53:13.864 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4470000000000003e-06 2024-05-08T08:53:13.864 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.864 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.864 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.864 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.305374+0000", 2024-05-08T08:53:13.864 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1059999999999999e-06 2024-05-08T08:53:13.865 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.865 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.865 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.865 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.865 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.865 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:26757 3.4 3.134fb004 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.865 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.283643+0000", 2024-05-08T08:53:13.865 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.030836393, 2024-05-08T08:53:13.865 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.040089161999999998, 2024-05-08T08:53:13.865 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.865 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.865 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.865 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.865 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.865 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.866 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 26757 2024-05-08T08:53:13.866 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.866 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.866 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.866 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.866 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283643+0000", 2024-05-08T08:53:13.866 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.866 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.866 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.866 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.866 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283643+0000", 2024-05-08T08:53:13.866 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.866 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.867 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.867 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.867 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283643+0000", 2024-05-08T08:53:13.867 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7599999999999998e-07 2024-05-08T08:53:13.867 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.867 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.867 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.867 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283644+0000", 2024-05-08T08:53:13.867 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2100000000000001e-06 2024-05-08T08:53:13.867 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.867 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.867 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.867 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283647+0000", 2024-05-08T08:53:13.867 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9830000000000001e-06 2024-05-08T08:53:13.868 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.868 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.868 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.868 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.283650+0000", 2024-05-08T08:53:13.868 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.497e-06 2024-05-08T08:53:13.868 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.868 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.868 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.868 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.868 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.868 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27290 3.4 3.134fb004 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.868 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.302509+0000", 2024-05-08T08:53:13.868 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.011969899000000001, 2024-05-08T08:53:13.868 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.021277134, 2024-05-08T08:53:13.869 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.869 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.869 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.869 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.869 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.869 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.869 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27290 2024-05-08T08:53:13.869 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.869 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.869 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.869 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.869 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302509+0000", 2024-05-08T08:53:13.869 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.869 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.870 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.870 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.870 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302509+0000", 2024-05-08T08:53:13.870 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.870 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.870 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.870 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.870 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302510+0000", 2024-05-08T08:53:13.870 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6699999999999999e-07 2024-05-08T08:53:13.870 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.870 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.870 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.870 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302512+0000", 2024-05-08T08:53:13.870 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2330000000000001e-06 2024-05-08T08:53:13.871 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.871 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.871 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.871 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302519+0000", 2024-05-08T08:53:13.871 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9779999999999999e-06 2024-05-08T08:53:13.871 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.871 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.871 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.871 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.302524+0000", 2024-05-08T08:53:13.871 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9339999999999997e-06 2024-05-08T08:53:13.871 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.871 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.871 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.871 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.872 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.872 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:27390 3.4 3.8748ad64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:13.872 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.303562+0000", 2024-05-08T08:53:13.872 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.010917534, 2024-05-08T08:53:13.872 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.020280600999999999, 2024-05-08T08:53:13.872 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:13.872 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:13.872 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:13.872 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:13.872 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:13.872 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:13.872 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 27390 2024-05-08T08:53:13.872 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.872 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:13.872 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.873 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:13.873 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303562+0000", 2024-05-08T08:53:13.873 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.873 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.873 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.873 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:13.873 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303562+0000", 2024-05-08T08:53:13.873 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:13.873 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.873 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.873 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:13.873 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303562+0000", 2024-05-08T08:53:13.873 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9400000000000001e-07 2024-05-08T08:53:13.873 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.873 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.873 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:13.874 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303565+0000", 2024-05-08T08:53:13.874 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.621e-06 2024-05-08T08:53:13.874 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.874 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.874 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:13.874 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303571+0000", 2024-05-08T08:53:13.874 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2260000000000002e-06 2024-05-08T08:53:13.874 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:13.874 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:13.874 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:13.874 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.303576+0000", 2024-05-08T08:53:13.874 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1080000000000001e-06 2024-05-08T08:53:13.874 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.874 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:13.874 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.875 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:13.875 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T08:53:13.875 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 168 2024-05-08T08:53:13.875 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:13.875 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-05-08T08:53:13.877 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:53:13.877 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:13.880+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:13.977 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:13.980+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:14.063 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:14.063 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:53:14.063 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:53:14.063 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:53:14.063 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:14.076 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:53:14.078 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:14.081+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:14.181 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:14.184+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:14.236 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:14.236 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:53:14.236 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:53:14.236 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:53:14.236 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.236 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:22471 3.4 3:26b512e1:::rbd_header.1140133d334a:head [notify cookie 139902587499760] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:14.236 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.123196+0000", 2024-05-08T08:53:14.237 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 1.1145537569999999, 2024-05-08T08:53:14.237 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.061461114999999997, 2024-05-08T08:53:14.237 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:14.237 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:14.237 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:14.237 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:14.237 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:14.237 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:14.237 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 22471 2024-05-08T08:53:14.237 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.237 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:14.238 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.238 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:14.238 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123196+0000", 2024-05-08T08:53:14.238 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:14.238 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.238 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.238 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:14.238 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123196+0000", 2024-05-08T08:53:14.238 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:14.238 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.239 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.239 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:14.239 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123196+0000", 2024-05-08T08:53:14.239 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.2800000000000003e-07 2024-05-08T08:53:14.239 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.239 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.239 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:14.239 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123198+0000", 2024-05-08T08:53:14.239 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4040000000000002e-06 2024-05-08T08:53:14.239 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.239 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.240 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:14.240 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123204+0000", 2024-05-08T08:53:14.240 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8749999999999997e-06 2024-05-08T08:53:14.240 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.240 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.240 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:14.240 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123209+0000", 2024-05-08T08:53:14.240 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1209999999999998e-06 2024-05-08T08:53:14.240 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.240 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.240 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:14.240 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.184473+0000", 2024-05-08T08:53:14.240 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.061263347000000003 2024-05-08T08:53:14.241 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.241 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.241 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:14.241 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.184542+0000", 2024-05-08T08:53:14.241 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9023000000000006e-05 2024-05-08T08:53:14.241 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.241 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.241 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:14.241 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.184657+0000", 2024-05-08T08:53:14.241 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000115017 2024-05-08T08:53:14.241 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:14.241 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:14.242 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:14.242 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.242 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.242 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:22508 3.4 3:200df2c8:::rbd_data.1140100c9703.0000000000000002:14 [read 1283072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:14.242 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.123216+0000", 2024-05-08T08:53:14.242 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 1.1145335199999999, 2024-05-08T08:53:14.242 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.061633735000000002, 2024-05-08T08:53:14.242 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:14.242 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:14.242 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:14.242 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:14.243 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:14.243 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:14.243 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 22508 2024-05-08T08:53:14.243 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.243 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:14.243 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.243 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:14.243 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123216+0000", 2024-05-08T08:53:14.243 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:14.243 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.243 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.243 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:14.244 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123216+0000", 2024-05-08T08:53:14.244 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:14.244 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.244 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.244 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:14.244 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123216+0000", 2024-05-08T08:53:14.244 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.39e-07 2024-05-08T08:53:14.244 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.245 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.245 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:14.245 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123219+0000", 2024-05-08T08:53:14.245 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.571e-06 2024-05-08T08:53:14.245 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.245 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.245 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:14.245 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123225+0000", 2024-05-08T08:53:14.245 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9819999999999996e-06 2024-05-08T08:53:14.245 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.245 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.245 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:14.246 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123232+0000", 2024-05-08T08:53:14.246 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8669999999999996e-06 2024-05-08T08:53:14.246 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.246 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.246 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:14.246 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.184687+0000", 2024-05-08T08:53:14.246 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.061455009999999997 2024-05-08T08:53:14.246 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.246 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.246 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:14.246 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.184763+0000", 2024-05-08T08:53:14.246 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.6385999999999997e-05 2024-05-08T08:53:14.247 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.247 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.247 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:14.247 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.184850+0000", 2024-05-08T08:53:14.247 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.6479999999999999e-05 2024-05-08T08:53:14.247 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:14.247 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:14.247 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:14.247 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.247 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.247 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:22509 3.4 3:200df2c8:::rbd_data.1140100c9703.0000000000000002:14 [read 1292288~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:14.247 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.123239+0000", 2024-05-08T08:53:14.247 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 1.114510866, 2024-05-08T08:53:14.247 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.061804586000000002, 2024-05-08T08:53:14.248 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:14.248 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:14.248 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:14.248 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:14.248 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:14.248 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:14.248 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 22509 2024-05-08T08:53:14.248 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.248 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:14.248 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.248 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:14.248 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123239+0000", 2024-05-08T08:53:14.248 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:14.248 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.248 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.249 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:14.249 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123239+0000", 2024-05-08T08:53:14.249 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:14.249 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.249 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.249 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:14.249 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123239+0000", 2024-05-08T08:53:14.249 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4000000000000002e-07 2024-05-08T08:53:14.249 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.249 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.249 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:14.249 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123241+0000", 2024-05-08T08:53:14.249 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3209999999999999e-06 2024-05-08T08:53:14.249 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.249 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.249 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:14.250 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123249+0000", 2024-05-08T08:53:14.250 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.96e-06 2024-05-08T08:53:14.250 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.250 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.250 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:14.250 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123254+0000", 2024-05-08T08:53:14.250 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7280000000000003e-06 2024-05-08T08:53:14.250 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.250 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.250 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:14.250 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.184879+0000", 2024-05-08T08:53:14.250 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.061625231000000003 2024-05-08T08:53:14.250 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.250 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.250 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:14.251 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.184958+0000", 2024-05-08T08:53:14.251 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8397000000000002e-05 2024-05-08T08:53:14.251 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.251 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.251 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:14.251 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.185043+0000", 2024-05-08T08:53:14.251 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.5408999999999995e-05 2024-05-08T08:53:14.251 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:14.251 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:14.251 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:14.251 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.251 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.251 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:22510 3.4 3:200df2c8:::rbd_data.1140100c9703.0000000000000002:14 [read 1301504~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:14.251 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.123263+0000", 2024-05-08T08:53:14.251 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 1.1144865269999999, 2024-05-08T08:53:14.252 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.061964539999999999, 2024-05-08T08:53:14.252 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:14.252 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:14.252 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:14.252 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:14.252 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:14.252 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:14.252 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 22510 2024-05-08T08:53:14.252 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.252 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:14.252 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.252 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:14.253 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123263+0000", 2024-05-08T08:53:14.253 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:14.253 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.253 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.253 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:14.253 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123263+0000", 2024-05-08T08:53:14.253 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:14.253 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.253 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.253 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:14.253 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123263+0000", 2024-05-08T08:53:14.253 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1100000000000001e-07 2024-05-08T08:53:14.253 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.253 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.253 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:14.254 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123266+0000", 2024-05-08T08:53:14.254 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7970000000000001e-06 2024-05-08T08:53:14.254 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.254 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.254 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:14.254 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123272+0000", 2024-05-08T08:53:14.254 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8819999999999998e-06 2024-05-08T08:53:14.254 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.254 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.254 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:14.254 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123277+0000", 2024-05-08T08:53:14.254 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1209999999999998e-06 2024-05-08T08:53:14.254 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.254 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.255 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:14.255 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.185074+0000", 2024-05-08T08:53:14.255 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.061796901000000001 2024-05-08T08:53:14.255 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.255 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.255 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:14.255 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.185144+0000", 2024-05-08T08:53:14.255 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0265999999999994e-05 2024-05-08T08:53:14.255 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.255 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.255 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:14.255 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.185228+0000", 2024-05-08T08:53:14.255 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3162000000000001e-05 2024-05-08T08:53:14.255 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:14.255 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:14.255 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:14.256 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.256 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.256 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:22511 3.4 3:200df2c8:::rbd_data.1140100c9703.0000000000000002:14 [read 1312768~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:14.256 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.123284+0000", 2024-05-08T08:53:14.256 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 1.1144651370000001, 2024-05-08T08:53:14.256 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.062125073000000003, 2024-05-08T08:53:14.256 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:14.256 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:14.256 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:14.256 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:14.256 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:14.256 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:14.256 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 22511 2024-05-08T08:53:14.256 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.256 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:14.257 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.257 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:14.257 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123284+0000", 2024-05-08T08:53:14.257 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:14.257 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.257 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.257 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:14.257 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123284+0000", 2024-05-08T08:53:14.257 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:14.257 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.257 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.257 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:14.257 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123285+0000", 2024-05-08T08:53:14.257 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7099999999999997e-07 2024-05-08T08:53:14.257 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.258 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.258 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:14.258 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123287+0000", 2024-05-08T08:53:14.258 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2859999999999998e-06 2024-05-08T08:53:14.258 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.258 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.258 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:14.258 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123293+0000", 2024-05-08T08:53:14.258 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0240000000000003e-06 2024-05-08T08:53:14.258 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.259 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.259 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:14.259 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123298+0000", 2024-05-08T08:53:14.259 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1279999999999999e-06 2024-05-08T08:53:14.259 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.259 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.259 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:14.259 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.185257+0000", 2024-05-08T08:53:14.259 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.061958930000000002 2024-05-08T08:53:14.259 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.259 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.259 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:14.259 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.185326+0000", 2024-05-08T08:53:14.259 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8817000000000002e-05 2024-05-08T08:53:14.260 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.260 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.260 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:14.260 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.185409+0000", 2024-05-08T08:53:14.260 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3516999999999993e-05 2024-05-08T08:53:14.260 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:14.260 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:14.260 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:14.260 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.260 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.260 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:22512 3.4 3:200df2c8:::rbd_data.1140100c9703.0000000000000002:14 [read 1321984~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:14.260 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.123305+0000", 2024-05-08T08:53:14.260 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 1.114444419, 2024-05-08T08:53:14.260 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.062288156999999997, 2024-05-08T08:53:14.260 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:14.261 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:14.261 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:14.261 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:14.261 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:14.261 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:14.261 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 22512 2024-05-08T08:53:14.261 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.261 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:14.261 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.261 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:14.261 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123305+0000", 2024-05-08T08:53:14.261 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:14.261 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.261 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.262 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:14.262 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123305+0000", 2024-05-08T08:53:14.262 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:14.262 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.262 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.262 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:14.262 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123305+0000", 2024-05-08T08:53:14.262 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6600000000000002e-07 2024-05-08T08:53:14.262 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.262 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.262 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:14.262 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123308+0000", 2024-05-08T08:53:14.262 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2450000000000001e-06 2024-05-08T08:53:14.262 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.263 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.263 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:14.263 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123314+0000", 2024-05-08T08:53:14.263 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2580000000000001e-06 2024-05-08T08:53:14.263 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.263 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.263 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:14.263 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123319+0000", 2024-05-08T08:53:14.263 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6009999999999997e-06 2024-05-08T08:53:14.263 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.263 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.263 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:14.263 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.185440+0000", 2024-05-08T08:53:14.263 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.062121434000000003 2024-05-08T08:53:14.263 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.263 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.264 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:14.264 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.185511+0000", 2024-05-08T08:53:14.264 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1478999999999998e-05 2024-05-08T08:53:14.264 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.264 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.264 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:14.264 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.185593+0000", 2024-05-08T08:53:14.264 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1773999999999993e-05 2024-05-08T08:53:14.264 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:14.264 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:14.264 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:14.264 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.264 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.264 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:22513 3.4 3:200df2c8:::rbd_data.1140100c9703.0000000000000002:14 [read 1331200~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:14.264 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.123326+0000", 2024-05-08T08:53:14.265 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 1.11442387, 2024-05-08T08:53:14.265 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.062453781, 2024-05-08T08:53:14.265 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:14.265 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:14.265 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:14.265 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:14.265 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:14.265 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:14.266 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 22513 2024-05-08T08:53:14.267 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.267 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:14.267 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.267 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:14.267 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123326+0000", 2024-05-08T08:53:14.267 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:14.267 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.267 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.267 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:14.267 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123326+0000", 2024-05-08T08:53:14.267 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:14.267 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.267 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.267 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:14.268 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123326+0000", 2024-05-08T08:53:14.268 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.77e-07 2024-05-08T08:53:14.268 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.268 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.268 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:14.268 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123328+0000", 2024-05-08T08:53:14.268 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0940000000000002e-06 2024-05-08T08:53:14.268 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.268 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.268 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:14.268 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123334+0000", 2024-05-08T08:53:14.268 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6629999999999998e-06 2024-05-08T08:53:14.268 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.268 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.268 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:14.269 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123339+0000", 2024-05-08T08:53:14.269 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7500000000000003e-06 2024-05-08T08:53:14.269 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.269 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.269 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:14.269 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.185625+0000", 2024-05-08T08:53:14.269 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.062286672000000001 2024-05-08T08:53:14.269 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.269 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.269 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:14.269 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.185697+0000", 2024-05-08T08:53:14.269 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1853000000000005e-05 2024-05-08T08:53:14.269 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.269 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.269 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:14.270 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.185779+0000", 2024-05-08T08:53:14.270 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.2372000000000003e-05 2024-05-08T08:53:14.270 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:14.270 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:14.270 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:14.270 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.270 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.270 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:22514 3.4 3:200df2c8:::rbd_data.1140100c9703.0000000000000002:14 [read 1340416~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:14.270 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.123345+0000", 2024-05-08T08:53:14.270 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 1.1144046670000001, 2024-05-08T08:53:14.270 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.062624377999999994, 2024-05-08T08:53:14.270 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:14.270 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:14.270 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:14.270 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:14.271 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:14.271 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:14.271 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 22514 2024-05-08T08:53:14.271 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.271 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:14.271 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.271 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:14.271 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123345+0000", 2024-05-08T08:53:14.271 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:14.272 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.272 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.272 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:14.272 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123345+0000", 2024-05-08T08:53:14.272 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:14.272 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.272 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.272 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:14.272 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123345+0000", 2024-05-08T08:53:14.272 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4999999999999998e-07 2024-05-08T08:53:14.272 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.273 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.273 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:14.273 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123347+0000", 2024-05-08T08:53:14.273 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0250000000000001e-06 2024-05-08T08:53:14.273 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.273 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.273 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:14.273 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123353+0000", 2024-05-08T08:53:14.273 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0159999999999997e-06 2024-05-08T08:53:14.273 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.273 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.273 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:14.273 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123358+0000", 2024-05-08T08:53:14.273 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4750000000000004e-06 2024-05-08T08:53:14.274 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.274 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.274 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:14.274 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.185809+0000", 2024-05-08T08:53:14.274 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.062451602000000002 2024-05-08T08:53:14.274 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.274 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.274 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:14.274 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.185882+0000", 2024-05-08T08:53:14.274 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2259000000000001e-05 2024-05-08T08:53:14.274 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.274 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.274 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:14.274 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.185969+0000", 2024-05-08T08:53:14.275 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7651000000000005e-05 2024-05-08T08:53:14.275 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:14.275 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:14.275 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:14.275 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.275 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.275 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:22515 3.4 3:200df2c8:::rbd_data.1140100c9703.0000000000000002:14 [read 1351680~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:14.275 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.123364+0000", 2024-05-08T08:53:14.275 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 1.1143854120000001, 2024-05-08T08:53:14.275 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.062847945000000002, 2024-05-08T08:53:14.275 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:14.275 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:14.275 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:14.275 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:14.275 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:14.276 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:14.276 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 22515 2024-05-08T08:53:14.276 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.276 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:14.276 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.276 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:14.276 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123364+0000", 2024-05-08T08:53:14.276 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:14.276 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.276 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.276 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:14.276 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123364+0000", 2024-05-08T08:53:14.276 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:14.276 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.276 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.277 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:14.277 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123364+0000", 2024-05-08T08:53:14.277 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.2000000000000001e-07 2024-05-08T08:53:14.277 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.277 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.277 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:14.277 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123372+0000", 2024-05-08T08:53:14.277 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8650000000000001e-06 2024-05-08T08:53:14.277 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.277 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.277 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:14.277 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123379+0000", 2024-05-08T08:53:14.277 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.64e-06 2024-05-08T08:53:14.277 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.277 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.277 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:14.278 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123384+0000", 2024-05-08T08:53:14.278 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9830000000000004e-06 2024-05-08T08:53:14.278 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.278 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.278 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:14.278 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.185999+0000", 2024-05-08T08:53:14.278 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.062615547999999993 2024-05-08T08:53:14.278 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.278 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.278 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:14.278 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.186091+0000", 2024-05-08T08:53:14.278 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1576999999999994e-05 2024-05-08T08:53:14.278 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.278 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.279 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:14.279 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.186212+0000", 2024-05-08T08:53:14.279 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000121012 2024-05-08T08:53:14.279 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:14.279 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:14.279 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:14.279 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.279 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.279 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:22516 3.4 3:200df2c8:::rbd_data.1140100c9703.0000000000000002:14 [read 1360896~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:14.279 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.123391+0000", 2024-05-08T08:53:14.279 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 1.114358468, 2024-05-08T08:53:14.279 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.063010391999999998, 2024-05-08T08:53:14.279 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:14.279 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:14.279 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:14.280 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:14.280 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:14.280 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:14.280 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 22516 2024-05-08T08:53:14.280 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.280 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:14.280 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.280 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:14.280 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123391+0000", 2024-05-08T08:53:14.280 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:14.280 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.280 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.280 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:14.280 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123391+0000", 2024-05-08T08:53:14.281 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:14.281 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.281 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.281 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:14.281 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123391+0000", 2024-05-08T08:53:14.281 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0699999999999998e-07 2024-05-08T08:53:14.281 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.281 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.281 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:14.281 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123394+0000", 2024-05-08T08:53:14.281 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0930000000000001e-06 2024-05-08T08:53:14.281 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.281 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.281 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:14.281 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123399+0000", 2024-05-08T08:53:14.282 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7980000000000002e-06 2024-05-08T08:53:14.282 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.282 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.282 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:14.282 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123404+0000", 2024-05-08T08:53:14.282 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7169999999999999e-06 2024-05-08T08:53:14.282 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.282 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.282 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:14.282 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.186250+0000", 2024-05-08T08:53:14.282 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.062845711999999998 2024-05-08T08:53:14.282 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.282 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.282 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:14.283 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.186319+0000", 2024-05-08T08:53:14.283 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.97e-05 2024-05-08T08:53:14.283 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.283 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.283 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:14.283 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.186401+0000", 2024-05-08T08:53:14.283 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1965000000000006e-05 2024-05-08T08:53:14.283 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:14.283 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:14.283 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:14.283 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.283 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.283 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:22517 3.4 3:200df2c8:::rbd_data.1140100c9703.0000000000000002:14 [read 1370112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:14.283 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.123412+0000", 2024-05-08T08:53:14.283 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 1.114337889, 2024-05-08T08:53:14.284 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.063168632000000002, 2024-05-08T08:53:14.284 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:14.284 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:14.284 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:14.284 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:14.284 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:14.284 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:14.284 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 22517 2024-05-08T08:53:14.284 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.284 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:14.284 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.284 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:14.284 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123412+0000", 2024-05-08T08:53:14.284 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:14.284 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.285 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.285 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:14.285 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123412+0000", 2024-05-08T08:53:14.285 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:14.285 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.285 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.285 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:14.285 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123412+0000", 2024-05-08T08:53:14.285 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8500000000000002e-07 2024-05-08T08:53:14.285 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.285 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.285 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:14.285 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123414+0000", 2024-05-08T08:53:14.285 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0930000000000001e-06 2024-05-08T08:53:14.285 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.286 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.286 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:14.286 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123421+0000", 2024-05-08T08:53:14.286 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9999999999999999e-06 2024-05-08T08:53:14.286 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.286 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.286 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:14.286 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123426+0000", 2024-05-08T08:53:14.286 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.95e-06 2024-05-08T08:53:14.286 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.286 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.286 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:14.286 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.186429+0000", 2024-05-08T08:53:14.286 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.063003346000000002 2024-05-08T08:53:14.287 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.287 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.287 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:14.287 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.186501+0000", 2024-05-08T08:53:14.287 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1575000000000005e-05 2024-05-08T08:53:14.287 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.287 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.287 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:14.287 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.186580+0000", 2024-05-08T08:53:14.287 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9283000000000006e-05 2024-05-08T08:53:14.287 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:14.287 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:14.287 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:14.287 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.288 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.288 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:22518 3.4 3:200df2c8:::rbd_data.1140100c9703.0000000000000002:14 [read 1381376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:14.288 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.123432+0000", 2024-05-08T08:53:14.288 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 1.1143170090000001, 2024-05-08T08:53:14.288 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.063331088999999993, 2024-05-08T08:53:14.288 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:14.288 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:14.288 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:14.288 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:14.288 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:14.288 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:14.288 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 22518 2024-05-08T08:53:14.288 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.288 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:14.288 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.289 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:14.289 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123432+0000", 2024-05-08T08:53:14.289 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:14.289 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.289 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.289 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:14.289 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123432+0000", 2024-05-08T08:53:14.289 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:14.289 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.289 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.289 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:14.289 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123433+0000", 2024-05-08T08:53:14.289 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4099999999999999e-07 2024-05-08T08:53:14.289 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.289 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.289 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:14.290 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123435+0000", 2024-05-08T08:53:14.290 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9379999999999999e-06 2024-05-08T08:53:14.290 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.290 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.290 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:14.290 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123441+0000", 2024-05-08T08:53:14.290 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9290000000000003e-06 2024-05-08T08:53:14.290 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.290 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.290 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:14.290 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123447+0000", 2024-05-08T08:53:14.290 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8279999999999999e-06 2024-05-08T08:53:14.290 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.290 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.291 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:14.291 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.186609+0000", 2024-05-08T08:53:14.291 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.063162067000000002 2024-05-08T08:53:14.291 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.291 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.291 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:14.291 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.186679+0000", 2024-05-08T08:53:14.291 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9721999999999994e-05 2024-05-08T08:53:14.291 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.291 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.291 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:14.291 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.186764+0000", 2024-05-08T08:53:14.291 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.5063999999999997e-05 2024-05-08T08:53:14.291 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:14.292 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:14.292 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:14.292 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.292 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.292 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:22519 3.4 3:200df2c8:::rbd_data.1140100c9703.0000000000000002:14 [read 1390592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:14.292 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.123453+0000", 2024-05-08T08:53:14.292 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 1.1142960829999999, 2024-05-08T08:53:14.292 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.063507447999999994, 2024-05-08T08:53:14.292 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:14.292 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:14.292 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:14.292 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:14.292 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:14.292 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:14.292 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 22519 2024-05-08T08:53:14.292 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.293 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:14.293 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.293 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:14.293 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123453+0000", 2024-05-08T08:53:14.293 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:14.293 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.293 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.293 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:14.293 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123453+0000", 2024-05-08T08:53:14.293 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:14.293 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.293 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.293 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:14.293 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123454+0000", 2024-05-08T08:53:14.293 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5100000000000001e-07 2024-05-08T08:53:14.294 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.294 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.294 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:14.294 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123456+0000", 2024-05-08T08:53:14.294 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0789999999999999e-06 2024-05-08T08:53:14.294 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.294 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.294 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:14.294 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123462+0000", 2024-05-08T08:53:14.294 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1990000000000003e-06 2024-05-08T08:53:14.294 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.294 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.294 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:14.294 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123466+0000", 2024-05-08T08:53:14.295 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4449999999999998e-06 2024-05-08T08:53:14.295 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.295 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.295 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:14.295 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.186793+0000", 2024-05-08T08:53:14.295 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.063326128999999995 2024-05-08T08:53:14.295 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.295 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.295 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:14.295 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.186868+0000", 2024-05-08T08:53:14.295 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5475000000000005e-05 2024-05-08T08:53:14.295 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.295 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.295 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:14.296 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.186961+0000", 2024-05-08T08:53:14.296 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2769999999999994e-05 2024-05-08T08:53:14.296 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:14.296 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:14.296 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:14.296 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.296 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.296 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:22520 3.4 3:200df2c8:::rbd_data.1140100c9703.0000000000000002:14 [read 1399808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:14.296 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.123473+0000", 2024-05-08T08:53:14.296 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 1.114276445, 2024-05-08T08:53:14.296 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.063690254000000002, 2024-05-08T08:53:14.296 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:14.296 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:14.296 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:14.296 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:14.297 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:14.297 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:14.297 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 22520 2024-05-08T08:53:14.297 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.297 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:14.297 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.297 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:14.297 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123473+0000", 2024-05-08T08:53:14.297 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:14.297 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.297 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.297 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:14.297 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123473+0000", 2024-05-08T08:53:14.297 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:14.297 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.298 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.298 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:14.298 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123474+0000", 2024-05-08T08:53:14.298 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0100000000000005e-07 2024-05-08T08:53:14.298 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.298 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.298 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:14.298 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123476+0000", 2024-05-08T08:53:14.298 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0420000000000001e-06 2024-05-08T08:53:14.298 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.298 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.298 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:14.298 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123482+0000", 2024-05-08T08:53:14.298 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2670000000000003e-06 2024-05-08T08:53:14.298 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.299 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.299 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:14.299 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123487+0000", 2024-05-08T08:53:14.299 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6500000000000004e-06 2024-05-08T08:53:14.299 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.299 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.299 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:14.299 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.186990+0000", 2024-05-08T08:53:14.299 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.063503967999999994 2024-05-08T08:53:14.299 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.299 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.299 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:14.299 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.187077+0000", 2024-05-08T08:53:14.299 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.6415000000000005e-05 2024-05-08T08:53:14.299 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.300 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.300 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:14.300 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.187163+0000", 2024-05-08T08:53:14.300 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.6410999999999996e-05 2024-05-08T08:53:14.300 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:14.300 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:14.300 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:14.300 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.300 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.300 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:22521 3.4 3:200df2c8:::rbd_data.1140100c9703.0000000000000002:14 [read 1411072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:14.300 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.123493+0000", 2024-05-08T08:53:14.300 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 1.114256001, 2024-05-08T08:53:14.300 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.063877505000000001, 2024-05-08T08:53:14.300 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:14.300 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:14.301 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:14.301 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:14.301 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:14.301 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:14.301 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 22521 2024-05-08T08:53:14.301 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.301 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:14.301 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.301 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:14.301 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123493+0000", 2024-05-08T08:53:14.301 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:14.301 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.301 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.301 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:14.301 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123493+0000", 2024-05-08T08:53:14.302 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:14.302 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.302 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.302 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:14.302 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123494+0000", 2024-05-08T08:53:14.302 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8700000000000001e-07 2024-05-08T08:53:14.302 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.302 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.302 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:14.302 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123496+0000", 2024-05-08T08:53:14.302 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.973e-06 2024-05-08T08:53:14.302 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.302 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.302 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:14.303 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123502+0000", 2024-05-08T08:53:14.303 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2979999999999997e-06 2024-05-08T08:53:14.303 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.303 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.303 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:14.303 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123507+0000", 2024-05-08T08:53:14.303 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9949999999999996e-06 2024-05-08T08:53:14.303 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.303 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.303 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:14.303 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.187194+0000", 2024-05-08T08:53:14.303 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.063686417999999995 2024-05-08T08:53:14.303 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.303 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.303 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:14.304 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.187279+0000", 2024-05-08T08:53:14.304 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.5771000000000003e-05 2024-05-08T08:53:14.304 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.304 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.304 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:14.304 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.187371+0000", 2024-05-08T08:53:14.304 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1663000000000007e-05 2024-05-08T08:53:14.304 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:14.304 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:14.304 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:14.304 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.304 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.304 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:22522 3.4 3:200df2c8:::rbd_data.1140100c9703.0000000000000002:14 [read 1420288~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:14.304 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.123514+0000", 2024-05-08T08:53:14.304 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 1.11423505, 2024-05-08T08:53:14.305 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.064058437999999995, 2024-05-08T08:53:14.305 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:14.305 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:14.305 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:14.305 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:14.305 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:14.305 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:14.305 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 22522 2024-05-08T08:53:14.305 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.305 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:14.305 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.305 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:14.305 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123514+0000", 2024-05-08T08:53:14.305 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:14.305 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.306 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.306 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:14.306 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123514+0000", 2024-05-08T08:53:14.306 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:14.306 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.306 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.306 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:14.306 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123515+0000", 2024-05-08T08:53:14.306 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5199999999999998e-07 2024-05-08T08:53:14.306 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.306 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.306 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:14.306 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123517+0000", 2024-05-08T08:53:14.306 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1330000000000002e-06 2024-05-08T08:53:14.306 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.307 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.307 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:14.307 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123523+0000", 2024-05-08T08:53:14.307 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0950000000000001e-06 2024-05-08T08:53:14.307 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.307 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.307 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:14.307 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123528+0000", 2024-05-08T08:53:14.307 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5299999999999998e-06 2024-05-08T08:53:14.307 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.307 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.307 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:14.307 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.187401+0000", 2024-05-08T08:53:14.308 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.063873616999999994 2024-05-08T08:53:14.308 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.308 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.308 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:14.308 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.187477+0000", 2024-05-08T08:53:14.308 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5740000000000003e-05 2024-05-08T08:53:14.308 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.308 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.308 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:14.308 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.187573+0000", 2024-05-08T08:53:14.308 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5971000000000007e-05 2024-05-08T08:53:14.308 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:14.308 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:14.308 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:14.308 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.308 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.309 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:22523 3.4 3:200df2c8:::rbd_data.1140100c9703.0000000000000002:14 [read 1429504~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:14.309 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.123533+0000", 2024-05-08T08:53:14.309 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 1.1142160000000001, 2024-05-08T08:53:14.309 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.064248632999999999, 2024-05-08T08:53:14.309 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:14.309 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:14.309 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:14.309 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:14.309 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:14.309 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:14.309 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 22523 2024-05-08T08:53:14.309 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.309 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:14.309 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.310 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:14.310 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123533+0000", 2024-05-08T08:53:14.310 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:14.310 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.310 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.310 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:14.310 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123533+0000", 2024-05-08T08:53:14.310 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:14.310 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.310 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.310 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:14.310 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123534+0000", 2024-05-08T08:53:14.310 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.39e-07 2024-05-08T08:53:14.310 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.310 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.311 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:14.311 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123536+0000", 2024-05-08T08:53:14.311 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.951e-06 2024-05-08T08:53:14.311 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.311 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.311 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:14.311 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123542+0000", 2024-05-08T08:53:14.311 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.801e-06 2024-05-08T08:53:14.311 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.311 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.311 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:14.311 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123546+0000", 2024-05-08T08:53:14.311 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.454e-06 2024-05-08T08:53:14.312 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.312 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.312 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:14.312 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.187616+0000", 2024-05-08T08:53:14.312 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.064069599000000005 2024-05-08T08:53:14.312 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.312 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.312 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:14.312 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.187688+0000", 2024-05-08T08:53:14.312 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2557000000000003e-05 2024-05-08T08:53:14.312 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.312 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.312 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:14.312 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.187782+0000", 2024-05-08T08:53:14.313 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3931999999999994e-05 2024-05-08T08:53:14.313 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:14.313 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:14.313 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:14.313 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.313 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.313 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:22524 3.4 3:200df2c8:::rbd_data.1140100c9703.0000000000000002:14 [read 1438720~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:14.313 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.123552+0000", 2024-05-08T08:53:14.313 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 1.1141976790000001, 2024-05-08T08:53:14.313 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.064451531000000006, 2024-05-08T08:53:14.313 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:14.313 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:14.313 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:14.313 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:14.314 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:14.314 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:14.314 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 22524 2024-05-08T08:53:14.314 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.314 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:14.314 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.314 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:14.314 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123552+0000", 2024-05-08T08:53:14.314 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:14.314 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.314 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.314 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:14.314 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123552+0000", 2024-05-08T08:53:14.314 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:14.314 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.315 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.315 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:14.315 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123552+0000", 2024-05-08T08:53:14.315 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9400000000000001e-07 2024-05-08T08:53:14.315 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.315 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.315 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:14.315 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123554+0000", 2024-05-08T08:53:14.315 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.945e-06 2024-05-08T08:53:14.315 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.315 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.315 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:14.315 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123560+0000", 2024-05-08T08:53:14.315 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6999999999999996e-06 2024-05-08T08:53:14.316 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.316 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.316 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:14.316 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123564+0000", 2024-05-08T08:53:14.316 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5360000000000003e-06 2024-05-08T08:53:14.316 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.316 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.316 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:14.316 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.187824+0000", 2024-05-08T08:53:14.316 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.064260203000000002 2024-05-08T08:53:14.316 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.316 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.316 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:14.316 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.187896+0000", 2024-05-08T08:53:14.316 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1104000000000002e-05 2024-05-08T08:53:14.317 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.317 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.317 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:14.317 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.188003+0000", 2024-05-08T08:53:14.317 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000107749 2024-05-08T08:53:14.317 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:14.317 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:14.317 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:14.317 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.317 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.317 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:22780 3.4 3:200df2c8:::rbd_data.1140100c9703.0000000000000002:14 [read 1430528~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:14.317 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.130416+0000", 2024-05-08T08:53:14.317 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 1.107333297, 2024-05-08T08:53:14.317 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.061508963, 2024-05-08T08:53:14.317 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:14.318 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:14.318 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:14.318 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:14.318 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:14.318 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:14.318 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 22780 2024-05-08T08:53:14.318 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.318 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:14.318 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.318 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:14.318 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.130416+0000", 2024-05-08T08:53:14.318 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:14.318 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.318 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.318 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:14.319 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.130416+0000", 2024-05-08T08:53:14.319 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:14.319 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.319 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.319 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:14.319 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.130417+0000", 2024-05-08T08:53:14.319 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6300000000000001e-07 2024-05-08T08:53:14.319 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.319 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.319 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:14.319 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.130419+0000", 2024-05-08T08:53:14.319 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0059999999999999e-06 2024-05-08T08:53:14.319 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.319 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.319 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:14.319 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.130425+0000", 2024-05-08T08:53:14.320 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0399999999999998e-06 2024-05-08T08:53:14.320 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.320 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.320 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:14.320 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.130430+0000", 2024-05-08T08:53:14.320 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1850000000000003e-06 2024-05-08T08:53:14.320 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.320 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.320 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:14.320 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.191770+0000", 2024-05-08T08:53:14.320 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.061340266999999997 2024-05-08T08:53:14.320 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.320 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.320 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:14.320 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.191839+0000", 2024-05-08T08:53:14.320 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8520000000000001e-05 2024-05-08T08:53:14.321 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.321 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.321 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:14.321 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.191925+0000", 2024-05-08T08:53:14.321 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.6582000000000006e-05 2024-05-08T08:53:14.321 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:14.321 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:14.321 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:14.321 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.321 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.321 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:22781 3.4 3:200df2c8:::rbd_data.1140100c9703.0000000000000002:14 [read 1439744~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:14.321 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.130436+0000", 2024-05-08T08:53:14.321 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 1.1073131519999999, 2024-05-08T08:53:14.321 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.061768243, 2024-05-08T08:53:14.322 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:14.322 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:14.322 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:14.322 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:14.322 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:14.322 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:14.322 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 22781 2024-05-08T08:53:14.322 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.322 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:14.322 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.322 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:14.322 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.130436+0000", 2024-05-08T08:53:14.322 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:14.322 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.322 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.323 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:14.323 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.130436+0000", 2024-05-08T08:53:14.323 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:14.323 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.323 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.323 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:14.323 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.130437+0000", 2024-05-08T08:53:14.323 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6100000000000002e-07 2024-05-08T08:53:14.323 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.323 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.323 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:14.323 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.130444+0000", 2024-05-08T08:53:14.323 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.6629999999999993e-06 2024-05-08T08:53:14.323 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.323 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.324 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:14.324 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.130451+0000", 2024-05-08T08:53:14.324 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.007e-06 2024-05-08T08:53:14.324 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.324 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.324 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:14.324 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.130457+0000", 2024-05-08T08:53:14.324 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1719999999999998e-06 2024-05-08T08:53:14.324 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.324 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.324 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:14.324 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.191964+0000", 2024-05-08T08:53:14.324 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.061507305999999998 2024-05-08T08:53:14.324 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.324 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.325 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:14.325 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.192045+0000", 2024-05-08T08:53:14.325 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1347999999999994e-05 2024-05-08T08:53:14.325 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:14.325 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:14.325 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:14.325 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.192205+0000", 2024-05-08T08:53:14.325 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00015938599999999999 2024-05-08T08:53:14.325 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:14.325 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:14.325 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:14.325 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:14.325 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:14.325 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:14.326 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:53:14.328 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:14.331+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:14.429 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:14.432+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:14.479 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:14.479 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:53:14.479 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:53:14.479 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:14.493 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:53:14.538 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:14.541+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:14.550 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T08:53:14.630 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:14.633+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:14.644 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:14.645 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:53:14.645 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:53:14.645 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:53:14.645 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:14.657 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:53:14.732 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:14.734+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:14.735 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:53:14.735 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:53:14.754 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:53:14.754 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:53:14.787 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:53:14.788 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:53:14.808 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:14.808 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:14.813 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:53:14.814 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:53:14.819 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:53:14.861 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:14.864+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:14.926 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:14.926 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:14.933 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:53:14.936+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:53:14.935 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:53:15.034 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:15.037+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:15.040 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:15.040 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:15.041 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:53:15.041 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:53:15.049 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:53:15.067 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:53:15.067 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:53:15.136 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:15.139+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:15.155 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:15.156 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:15.164 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:53:15.236 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:15.239+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:15.270 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:15.270 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:15.279 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:53:15.339 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:15.340+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:15.423 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:15.424 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:15.427 INFO:teuthology.orchestra.run.smithi059.stderr:nodeep-scrub is set 2024-05-08T08:53:15.434 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:53:15.478 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:15.481+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:15.539 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:15.539 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:15.539 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:15.542+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:15.548 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:53:15.640 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:15.643+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:15.694 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:15.694 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:15.703 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:53:15.742 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:15.744+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:15.843 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:15.845+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:15.854 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:15.854 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:15.863 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:53:15.945 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:53:15.946+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:53:15.968 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:15.968 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:15.977 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:53:16.044 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:16.048+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:16.145 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:16.148+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:16.148 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:16.149 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:16.160 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:53:16.246 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:16.249+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:16.332 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:16.332 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:16.346 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:53:16.347 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:16.350+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:16.448 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:16.451+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:16.530 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:16.530 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:16.541 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:53:16.548 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:16.551+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:16.649 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:16.652+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:16.722 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:16.722 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:16.734 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:53:16.776 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:16.779+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:16.851 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:16.854+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:16.901 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:16.901 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:16.910 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:53:16.920 INFO:teuthology.orchestra.run.smithi196.stdout:2889 write 0xd903800 thru 0xd907241 (0x3a42 bytes) 2024-05-08T08:53:16.921 INFO:teuthology.orchestra.run.smithi196.stdout:2891 read 0xb347200 thru 0xb351e37 (0xac38 bytes) 2024-05-08T08:53:16.921 INFO:teuthology.orchestra.run.smithi196.stdout:2892 punch from 0x5aa6800 to 0x5ab0c00, (0xa400 bytes) 2024-05-08T08:53:16.922 INFO:teuthology.orchestra.run.smithi196.stdout:2893 write 0xcac4600 thru 0xcad104f (0xca50 bytes) 2024-05-08T08:53:16.936 INFO:teuthology.orchestra.run.smithi196.stdout:2895 trunc from 0xee10600 to 0x33d6a00 2024-05-08T08:53:16.952 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:16.955+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:17.000 INFO:teuthology.orchestra.run.smithi196.stdout:2896 read 0x1a1f400 thru 0x1a2d776 (0xe377 bytes) 2024-05-08T08:53:17.024 INFO:teuthology.orchestra.run.smithi196.stdout:2898 write 0xda43000 thru 0xda4bcc6 (0x8cc7 bytes) 2024-05-08T08:53:17.025 INFO:teuthology.orchestra.run.smithi196.stdout:2901 read 0x9227400 thru 0x922f33f (0x7f40 bytes) 2024-05-08T08:53:17.029 INFO:teuthology.orchestra.run.smithi196.stdout:2911 trunc from 0xda4be00 to 0xa34be00 2024-05-08T08:53:17.053 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:53:17.056+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:53:17.055 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:17.055 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:17.060 INFO:teuthology.orchestra.run.smithi196.stdout:2913 punch from 0x688200 to 0x697e00, (0xfc00 bytes) 2024-05-08T08:53:17.061 INFO:teuthology.orchestra.run.smithi196.stdout:2914 write 0x6864000 thru 0x6866ae5 (0x2ae6 bytes) 2024-05-08T08:53:17.066 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:53:17.083 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T08:53:17.087 INFO:teuthology.orchestra.run.smithi196.stdout:2915 write 0xe636a00 thru 0xe64545d (0xea5e bytes) 2024-05-08T08:53:17.088 INFO:teuthology.orchestra.run.smithi196.stdout:2916 punch from 0x6895200 to 0x689ae00, (0x5c00 bytes) 2024-05-08T08:53:17.088 INFO:teuthology.orchestra.run.smithi196.stdout:2921 read 0x27c8200 thru 0x27cd7f2 (0x55f3 bytes) 2024-05-08T08:53:17.099 INFO:teuthology.orchestra.run.smithi196.stdout:2922 trunc from 0xe645600 to 0x5924400 2024-05-08T08:53:17.133 INFO:teuthology.orchestra.run.smithi196.stdout:2923 read 0x267c000 thru 0x267f7d4 (0x37d5 bytes) 2024-05-08T08:53:17.152 INFO:teuthology.orchestra.run.smithi196.stdout:2926 write 0x9347600 thru 0x934ba19 (0x441a bytes) 2024-05-08T08:53:17.153 INFO:teuthology.orchestra.run.smithi196.stdout:2928 write 0x7618a00 thru 0x7624aee (0xc0ef bytes) 2024-05-08T08:53:17.154 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:17.157+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:17.185 INFO:teuthology.orchestra.run.smithi196.stdout:2931 write 0xa055800 thru 0xa060c86 (0xb487 bytes) 2024-05-08T08:53:17.214 INFO:teuthology.orchestra.run.smithi196.stdout:2934 write 0xd807a00 thru 0xd807ff5 (0x5f6 bytes) 2024-05-08T08:53:17.216 INFO:teuthology.orchestra.run.smithi196.stdout:2936 write 0x6467000 thru 0x646af78 (0x3f79 bytes) 2024-05-08T08:53:17.216 INFO:teuthology.orchestra.run.smithi196.stdout:2940 read 0x6f2a200 thru 0x6f2e3dc (0x41dd bytes) 2024-05-08T08:53:17.216 INFO:teuthology.orchestra.run.smithi196.stdout:2942 write 0x5c15600 thru 0x5c202a2 (0xaca3 bytes) 2024-05-08T08:53:17.216 INFO:teuthology.orchestra.run.smithi196.stdout:2943 read 0x445ae00 thru 0x4461f92 (0x7193 bytes) 2024-05-08T08:53:17.216 INFO:teuthology.orchestra.run.smithi196.stdout:2946 write 0x1cb5200 thru 0x1cb9a03 (0x4804 bytes) 2024-05-08T08:53:17.216 INFO:teuthology.orchestra.run.smithi196.stdout:2947 write 0x3cafc00 thru 0x3cb9917 (0x9d18 bytes) 2024-05-08T08:53:17.216 INFO:teuthology.orchestra.run.smithi196.stdout:2948 read 0xb412a00 thru 0xb42137d (0xe97e bytes) 2024-05-08T08:53:17.216 INFO:teuthology.orchestra.run.smithi196.stdout:2950 write 0xadbe000 thru 0xadcb5b3 (0xd5b4 bytes) 2024-05-08T08:53:17.216 INFO:teuthology.orchestra.run.smithi196.stdout:2955 read 0x8a9ca00 thru 0x8aa9675 (0xcc76 bytes) 2024-05-08T08:53:17.216 INFO:teuthology.orchestra.run.smithi196.stdout:2956 punch from 0x713a200 to 0x7141600, (0x7400 bytes) 2024-05-08T08:53:17.217 INFO:teuthology.orchestra.run.smithi196.stdout:2957 read 0x75be600 thru 0x75bf27e (0xc7f bytes) 2024-05-08T08:53:17.218 INFO:teuthology.orchestra.run.smithi196.stdout:2958 write 0x2130600 thru 0x2133fd2 (0x39d3 bytes) 2024-05-08T08:53:17.218 INFO:teuthology.orchestra.run.smithi196.stdout:2959 write 0x76a0600 thru 0x76adbe1 (0xd5e2 bytes) 2024-05-08T08:53:17.221 INFO:teuthology.orchestra.run.smithi196.stdout:2960 write 0x5baae00 thru 0x5bb6119 (0xb31a bytes) 2024-05-08T08:53:17.221 INFO:teuthology.orchestra.run.smithi196.stdout:2964 write 0x32f4800 thru 0x32ff736 (0xaf37 bytes) 2024-05-08T08:53:17.221 INFO:teuthology.orchestra.run.smithi196.stdout:2966 punch from 0xb9f4c00 to 0xba04800, (0xfc00 bytes) 2024-05-08T08:53:17.222 INFO:teuthology.orchestra.run.smithi196.stdout:2967 write 0xcb9000 thru 0xcba10e (0x110f bytes) 2024-05-08T08:53:17.222 INFO:teuthology.orchestra.run.smithi196.stdout:2968 punch from 0x56b5000 to 0x56bac00, (0x5c00 bytes) 2024-05-08T08:53:17.223 INFO:teuthology.orchestra.run.smithi196.stdout:2969 write 0x3760c00 thru 0x37677cd (0x6bce bytes) 2024-05-08T08:53:17.223 INFO:teuthology.orchestra.run.smithi196.stdout:2972 punch from 0xae5200 to 0xae8e00, (0x3c00 bytes) 2024-05-08T08:53:17.223 INFO:teuthology.orchestra.run.smithi196.stdout:2974 write 0xcf24400 thru 0xcf2fe1d (0xba1e bytes) 2024-05-08T08:53:17.223 INFO:teuthology.orchestra.run.smithi196.stdout:2977 read 0x852a800 thru 0x8538c31 (0xe432 bytes) 2024-05-08T08:53:17.223 INFO:teuthology.orchestra.run.smithi196.stdout:2978 read 0x86bcc00 thru 0x86c0802 (0x3c03 bytes) 2024-05-08T08:53:17.223 INFO:teuthology.orchestra.run.smithi196.stdout:2981 punch from 0x84c4800 to 0x84c8c00, (0x4400 bytes) 2024-05-08T08:53:17.224 INFO:teuthology.orchestra.run.smithi196.stdout:2983 punch from 0x2c01000 to 0x2c0b400, (0xa400 bytes) 2024-05-08T08:53:17.224 INFO:teuthology.orchestra.run.smithi196.stdout:2985 write 0xa43d200 thru 0xa44b52d (0xe32e bytes) 2024-05-08T08:53:17.225 INFO:teuthology.orchestra.run.smithi196.stdout:2987 write 0x6c64e00 thru 0x6c73f19 (0xf11a bytes) 2024-05-08T08:53:17.225 INFO:teuthology.orchestra.run.smithi196.stdout:2988 write 0x7e91600 thru 0x7e9e72f (0xd130 bytes) 2024-05-08T08:53:17.225 INFO:teuthology.orchestra.run.smithi196.stdout:2989 read 0x1d60c00 thru 0x1d6662f (0x5a30 bytes) 2024-05-08T08:53:17.225 INFO:teuthology.orchestra.run.smithi196.stdout:2990 read 0x317a800 thru 0x3188612 (0xde13 bytes) 2024-05-08T08:53:17.225 INFO:teuthology.orchestra.run.smithi196.stdout:2992 read 0x8480a00 thru 0x848ddca (0xd3cb bytes) 2024-05-08T08:53:17.225 INFO:teuthology.orchestra.run.smithi196.stdout:2996 read 0xcb44200 thru 0xcb4aab2 (0x68b3 bytes) 2024-05-08T08:53:17.225 INFO:teuthology.orchestra.run.smithi196.stdout:2998 punch from 0x680de00 to 0x6819800, (0xba00 bytes) 2024-05-08T08:53:17.225 INFO:teuthology.orchestra.run.smithi196.stdout:3000 read 0x149e800 thru 0x14ae171 (0xf972 bytes) 2024-05-08T08:53:17.229 INFO:teuthology.orchestra.run.smithi196.stdout:3002 write 0x51c6800 thru 0x51d320b (0xca0c bytes) 2024-05-08T08:53:17.230 INFO:teuthology.orchestra.run.smithi196.stdout:3003 read 0x5f34400 thru 0x5f36b90 (0x2791 bytes) 2024-05-08T08:53:17.230 INFO:teuthology.orchestra.run.smithi196.stdout:3005 punch from 0x3bc0e00 to 0x3bc7200, (0x6400 bytes) 2024-05-08T08:53:17.230 INFO:teuthology.orchestra.run.smithi196.stdout:3006 punch from 0x2f8dc00 to 0x2f9ca00, (0xee00 bytes) 2024-05-08T08:53:17.240 INFO:teuthology.orchestra.run.smithi196.stdout:3009 trunc from 0xd808000 to 0x4c99e00 2024-05-08T08:53:17.255 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:17.258+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:17.269 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:17.269 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:17.282 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:53:17.295 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:53:17.295 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:53:17.322 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:53:17.322 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:53:17.350 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:53:17.351 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:53:17.356 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:17.359+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:17.379 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:53:17.379 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:53:17.406 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:53:17.406 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:53:17.444 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:53:17.444 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:53:17.448 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-05-08T08:53:17.451 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:17.451 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:53:17.451 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:53:17.451 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:53:17.451 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:17.462 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:17.465+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:17.465 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:53:17.523 INFO:teuthology.orchestra.run.smithi196.stdout:3013 read 0x3616e00 thru 0x3625ab1 (0xecb2 bytes) 2024-05-08T08:53:17.533 INFO:teuthology.orchestra.run.smithi196.stdout:3015 trunc from 0x4c99e00 to 0xb79fa00 2024-05-08T08:53:17.546 INFO:teuthology.orchestra.run.smithi196.stdout:3016 write 0x75d0e00 thru 0x75d3673 (0x2874 bytes) 2024-05-08T08:53:17.548 INFO:teuthology.orchestra.run.smithi196.stdout:3017 read 0x94d6c00 thru 0x94dea46 (0x7e47 bytes) 2024-05-08T08:53:17.548 INFO:teuthology.orchestra.run.smithi196.stdout:3019 write 0xb2ce200 thru 0xb2d4b25 (0x6926 bytes) 2024-05-08T08:53:17.548 INFO:teuthology.orchestra.run.smithi196.stdout:3020 clone 19 order 19 su 2048 sc 4 2024-05-08T08:53:17.558 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:17.560+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:17.644 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:17.644 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:53:17.645 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:53:17.645 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:53:17.645 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.645 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:437 3.17 3:ea93a512:::rbd_data.1140100c9703.0000000000000077:head [set-alloc-hint object_size 2097152 write_size 2097152,write 1170432~1024] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e63)", 2024-05-08T08:53:17.645 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:08.390804+0000", 2024-05-08T08:53:17.645 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 9.2557327740000002, 2024-05-08T08:53:17.645 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0406068690000001, 2024-05-08T08:53:17.645 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:17.645 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:17.645 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:53:17.645 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:17.645 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:17.645 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:17.645 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 437 2024-05-08T08:53:17.645 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.646 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:17.646 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.646 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:17.646 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390804+0000", 2024-05-08T08:53:17.646 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:17.646 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.646 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.646 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:17.646 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390804+0000", 2024-05-08T08:53:17.646 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:17.646 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.646 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.646 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:17.646 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390806+0000", 2024-05-08T08:53:17.647 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.666e-06 2024-05-08T08:53:17.647 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.647 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.647 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:17.647 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390818+0000", 2024-05-08T08:53:17.647 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2187999999999999e-05 2024-05-08T08:53:17.647 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.647 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.647 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:17.647 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390834+0000", 2024-05-08T08:53:17.647 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6191000000000001e-05 2024-05-08T08:53:17.647 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.647 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.647 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:17.648 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390845+0000", 2024-05-08T08:53:17.648 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0831e-05 2024-05-08T08:53:17.648 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.648 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.648 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:17.648 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390922+0000", 2024-05-08T08:53:17.648 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7253e-05 2024-05-08T08:53:17.648 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.648 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.648 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op must wait for map", 2024-05-08T08:53:17.648 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390930+0000", 2024-05-08T08:53:17.648 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8259999999999993e-06 2024-05-08T08:53:17.648 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.648 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.648 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:17.649 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392155+0000", 2024-05-08T08:53:17.649 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0012256649999999999 2024-05-08T08:53:17.649 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.649 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.649 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:53:17.649 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392165+0000", 2024-05-08T08:53:17.649 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5799999999999998e-06 2024-05-08T08:53:17.649 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.649 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.649 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:17.649 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.429596+0000", 2024-05-08T08:53:17.649 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0374313639999999 2024-05-08T08:53:17.649 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.649 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.649 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:17.650 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.429728+0000", 2024-05-08T08:53:17.650 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000131552 2024-05-08T08:53:17.650 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.650 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.650 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 0,1", 2024-05-08T08:53:17.650 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.429860+0000", 2024-05-08T08:53:17.650 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000132137 2024-05-08T08:53:17.650 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.650 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.650 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:53:17.650 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.430811+0000", 2024-05-08T08:53:17.650 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00095040200000000004 2024-05-08T08:53:17.650 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.650 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.650 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:53:17.651 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431126+0000", 2024-05-08T08:53:17.651 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000315006 2024-05-08T08:53:17.651 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.651 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.651 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:53:17.651 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431272+0000", 2024-05-08T08:53:17.651 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000146487 2024-05-08T08:53:17.651 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.651 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.651 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:53:17.651 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431315+0000", 2024-05-08T08:53:17.651 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2697000000000001e-05 2024-05-08T08:53:17.651 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.651 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.651 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:17.652 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431411+0000", 2024-05-08T08:53:17.652 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6024000000000001e-05 2024-05-08T08:53:17.652 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:17.652 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:17.652 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:17.652 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.652 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.652 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:439 3.17 3:ea93a512:::rbd_data.1140100c9703.0000000000000077:head [read 1163264~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-05-08T08:53:17.652 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:08.390871+0000", 2024-05-08T08:53:17.652 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 9.2556652629999991, 2024-05-08T08:53:17.652 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0408790590000001, 2024-05-08T08:53:17.652 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:17.652 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:17.652 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:17.652 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:17.653 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:17.653 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:17.653 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 439 2024-05-08T08:53:17.653 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.653 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:17.653 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.653 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:17.653 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390871+0000", 2024-05-08T08:53:17.653 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:17.653 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.653 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.653 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:17.653 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390871+0000", 2024-05-08T08:53:17.653 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:17.653 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.653 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.654 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:17.654 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390872+0000", 2024-05-08T08:53:17.654 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6300000000000005e-07 2024-05-08T08:53:17.654 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.654 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.654 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:17.654 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390875+0000", 2024-05-08T08:53:17.654 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3459999999999998e-06 2024-05-08T08:53:17.654 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.654 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.654 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:17.654 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390891+0000", 2024-05-08T08:53:17.654 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5255e-05 2024-05-08T08:53:17.654 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.655 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.655 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:17.655 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390897+0000", 2024-05-08T08:53:17.655 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2360000000000001e-06 2024-05-08T08:53:17.655 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.655 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.655 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:17.655 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390995+0000", 2024-05-08T08:53:17.655 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8522000000000003e-05 2024-05-08T08:53:17.655 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.655 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.655 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting_for_map not empty", 2024-05-08T08:53:17.655 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.391005+0000", 2024-05-08T08:53:17.655 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1460000000000002e-06 2024-05-08T08:53:17.655 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.655 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.656 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:17.656 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392191+0000", 2024-05-08T08:53:17.656 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0011860270000000001 2024-05-08T08:53:17.656 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.656 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.656 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:53:17.656 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392197+0000", 2024-05-08T08:53:17.656 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4289999999999998e-06 2024-05-08T08:53:17.656 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.656 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.656 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:17.656 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.430596+0000", 2024-05-08T08:53:17.656 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0383989090000001 2024-05-08T08:53:17.656 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.656 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.657 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for rw locks", 2024-05-08T08:53:17.657 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.430676+0000", 2024-05-08T08:53:17.657 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9781000000000001e-05 2024-05-08T08:53:17.657 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.657 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.657 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:17.657 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431460+0000", 2024-05-08T08:53:17.657 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00078427300000000004 2024-05-08T08:53:17.657 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.657 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.657 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:17.657 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431540+0000", 2024-05-08T08:53:17.657 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0224999999999998e-05 2024-05-08T08:53:17.657 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.658 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.658 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:17.658 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431750+0000", 2024-05-08T08:53:17.658 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00021024699999999999 2024-05-08T08:53:17.658 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:17.658 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:17.658 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:17.658 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.658 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.658 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:945 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 4096000~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:17.658 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:53:17.658 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 8.1955156959999993, 2024-05-08T08:53:17.658 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086774741000000002, 2024-05-08T08:53:17.658 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:17.658 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:17.659 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:17.659 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:17.659 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:17.659 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:17.659 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 945 2024-05-08T08:53:17.659 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.659 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:17.659 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.659 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:17.659 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:53:17.659 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:17.659 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.659 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.659 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:17.659 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:53:17.660 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:17.660 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.660 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.660 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:17.660 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:53:17.660 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3499999999999996e-07 2024-05-08T08:53:17.660 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.660 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.660 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:17.660 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451024+0000", 2024-05-08T08:53:17.660 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.627e-06 2024-05-08T08:53:17.660 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.660 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.660 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:17.660 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451037+0000", 2024-05-08T08:53:17.661 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2431e-05 2024-05-08T08:53:17.661 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.661 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.661 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:17.661 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451042+0000", 2024-05-08T08:53:17.661 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8409999999999996e-06 2024-05-08T08:53:17.661 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.661 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.661 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:17.661 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537608+0000", 2024-05-08T08:53:17.661 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086565243 2024-05-08T08:53:17.661 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.661 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.661 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:17.662 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537687+0000", 2024-05-08T08:53:17.662 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9519999999999998e-05 2024-05-08T08:53:17.662 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.662 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.662 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:17.662 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537796+0000", 2024-05-08T08:53:17.662 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000108544 2024-05-08T08:53:17.662 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:17.662 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:17.662 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:17.662 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.662 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.662 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:964 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 458752~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:17.662 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451051+0000", 2024-05-08T08:53:17.663 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 8.1954855230000003, 2024-05-08T08:53:17.663 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086990490000000004, 2024-05-08T08:53:17.663 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:17.663 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:17.663 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:17.663 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:17.663 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:17.663 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:17.663 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 964 2024-05-08T08:53:17.663 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.663 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:17.663 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.663 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:17.663 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451051+0000", 2024-05-08T08:53:17.664 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:17.664 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.664 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.664 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:17.664 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451051+0000", 2024-05-08T08:53:17.664 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:17.664 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.664 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.664 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:17.664 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451052+0000", 2024-05-08T08:53:17.664 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7299999999999996e-07 2024-05-08T08:53:17.664 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.664 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.664 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:17.664 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451055+0000", 2024-05-08T08:53:17.665 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.2770000000000001e-06 2024-05-08T08:53:17.665 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.665 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.665 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:17.665 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451061+0000", 2024-05-08T08:53:17.665 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4060000000000002e-06 2024-05-08T08:53:17.665 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.665 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.665 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:17.665 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451066+0000", 2024-05-08T08:53:17.665 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0839999999999999e-06 2024-05-08T08:53:17.665 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.665 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.665 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:17.665 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537839+0000", 2024-05-08T08:53:17.665 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086772962999999995 2024-05-08T08:53:17.666 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.666 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.666 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:17.666 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537934+0000", 2024-05-08T08:53:17.666 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4742999999999997e-05 2024-05-08T08:53:17.666 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.666 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.666 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:17.666 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538042+0000", 2024-05-08T08:53:17.666 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000107444 2024-05-08T08:53:17.666 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:17.666 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:17.666 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:17.666 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.667 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.667 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:965 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 950272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:17.667 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451075+0000", 2024-05-08T08:53:17.667 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 8.1954614610000007, 2024-05-08T08:53:17.667 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087230196999999995, 2024-05-08T08:53:17.667 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:17.667 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:17.667 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:17.667 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:17.667 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:17.667 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:17.667 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 965 2024-05-08T08:53:17.667 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.667 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:17.667 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.668 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:17.668 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451075+0000", 2024-05-08T08:53:17.668 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:17.668 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.668 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.668 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:17.668 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451075+0000", 2024-05-08T08:53:17.668 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:17.668 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.668 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.668 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:17.668 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451076+0000", 2024-05-08T08:53:17.668 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5000000000000002e-07 2024-05-08T08:53:17.668 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.668 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.668 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:17.668 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451078+0000", 2024-05-08T08:53:17.669 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.5639999999999999e-06 2024-05-08T08:53:17.669 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.669 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.669 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:17.669 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451085+0000", 2024-05-08T08:53:17.669 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5640000000000002e-06 2024-05-08T08:53:17.669 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.669 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.669 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:17.669 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451090+0000", 2024-05-08T08:53:17.669 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1699999999999996e-06 2024-05-08T08:53:17.669 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.669 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.669 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:17.669 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538106+0000", 2024-05-08T08:53:17.670 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087015888999999999 2024-05-08T08:53:17.670 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.670 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.670 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:17.670 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538190+0000", 2024-05-08T08:53:17.670 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3961999999999993e-05 2024-05-08T08:53:17.670 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.670 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.670 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:17.670 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538305+0000", 2024-05-08T08:53:17.670 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011529800000000001 2024-05-08T08:53:17.670 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:17.670 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:17.670 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:17.671 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.671 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.671 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:966 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 1441792~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:17.671 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:53:17.671 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 8.1954380879999995, 2024-05-08T08:53:17.671 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087446323000000006, 2024-05-08T08:53:17.671 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:17.671 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:17.671 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:17.671 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:17.671 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:17.671 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:17.671 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 966 2024-05-08T08:53:17.671 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.671 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:17.671 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.672 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:17.672 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:53:17.672 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:17.672 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.672 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.672 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:17.672 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:53:17.672 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:17.672 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.672 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.672 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:17.672 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:53:17.672 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3700000000000001e-07 2024-05-08T08:53:17.672 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.672 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.672 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:17.673 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451101+0000", 2024-05-08T08:53:17.673 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2560000000000001e-06 2024-05-08T08:53:17.673 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.673 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.673 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:17.673 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451107+0000", 2024-05-08T08:53:17.673 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1070000000000001e-06 2024-05-08T08:53:17.673 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.673 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.673 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:17.673 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451112+0000", 2024-05-08T08:53:17.673 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1420000000000001e-06 2024-05-08T08:53:17.673 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.673 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.673 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:17.673 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538341+0000", 2024-05-08T08:53:17.674 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087228302999999993 2024-05-08T08:53:17.674 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.674 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.674 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:17.674 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538424+0000", 2024-05-08T08:53:17.674 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3497000000000004e-05 2024-05-08T08:53:17.674 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.674 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.674 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:17.674 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538545+0000", 2024-05-08T08:53:17.674 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000120581 2024-05-08T08:53:17.674 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:17.674 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:17.674 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:17.675 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.675 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.675 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:967 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:17.675 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451120+0000", 2024-05-08T08:53:17.675 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 8.1954166130000008, 2024-05-08T08:53:17.675 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087632725999999994, 2024-05-08T08:53:17.675 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:17.675 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:17.675 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:17.675 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:17.675 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:17.675 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:17.675 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 967 2024-05-08T08:53:17.675 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.675 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:17.675 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.676 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:17.676 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451120+0000", 2024-05-08T08:53:17.676 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:17.676 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.676 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.676 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:17.676 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451120+0000", 2024-05-08T08:53:17.676 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:17.676 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.676 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.676 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:17.676 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451121+0000", 2024-05-08T08:53:17.676 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8800000000000002e-07 2024-05-08T08:53:17.676 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.676 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.677 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:17.677 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451130+0000", 2024-05-08T08:53:17.677 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2299999999999997e-06 2024-05-08T08:53:17.677 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.677 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.677 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:17.677 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451137+0000", 2024-05-08T08:53:17.677 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4669999999999998e-06 2024-05-08T08:53:17.677 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.677 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.677 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:17.677 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451143+0000", 2024-05-08T08:53:17.677 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2839999999999997e-06 2024-05-08T08:53:17.677 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.677 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.678 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:17.678 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538575+0000", 2024-05-08T08:53:17.678 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087432789999999996 2024-05-08T08:53:17.678 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.678 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.678 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:17.678 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538648+0000", 2024-05-08T08:53:17.678 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2877e-05 2024-05-08T08:53:17.678 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.678 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.678 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:17.678 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538753+0000", 2024-05-08T08:53:17.678 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010449 2024-05-08T08:53:17.678 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:17.678 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:17.679 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:17.679 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.679 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.679 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:968 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 2424832~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:17.679 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451152+0000", 2024-05-08T08:53:17.679 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 8.1953845279999999, 2024-05-08T08:53:17.679 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087813325999999997, 2024-05-08T08:53:17.679 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:17.679 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:17.679 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:17.679 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:17.679 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:17.679 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:17.679 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 968 2024-05-08T08:53:17.679 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.679 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:17.680 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.680 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:17.680 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451152+0000", 2024-05-08T08:53:17.680 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:17.680 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.680 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.680 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:17.680 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451152+0000", 2024-05-08T08:53:17.680 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:17.680 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.680 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.680 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:17.680 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451153+0000", 2024-05-08T08:53:17.680 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3099999999999998e-07 2024-05-08T08:53:17.680 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.681 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.681 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:17.681 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451155+0000", 2024-05-08T08:53:17.681 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4760000000000001e-06 2024-05-08T08:53:17.681 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.681 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.681 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:17.681 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451162+0000", 2024-05-08T08:53:17.681 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5969999999999997e-06 2024-05-08T08:53:17.681 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.681 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.681 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:17.681 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451167+0000", 2024-05-08T08:53:17.681 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.977e-06 2024-05-08T08:53:17.681 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.681 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.682 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:17.682 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538781+0000", 2024-05-08T08:53:17.682 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087614846999999996 2024-05-08T08:53:17.682 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.682 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.682 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:17.682 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538857+0000", 2024-05-08T08:53:17.682 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5358000000000006e-05 2024-05-08T08:53:17.682 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.682 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.682 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:17.682 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538965+0000", 2024-05-08T08:53:17.682 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010864 2024-05-08T08:53:17.682 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:17.682 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:17.683 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:17.683 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.683 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.683 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:969 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 2916352~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:17.683 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:53:17.683 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 8.195362823, 2024-05-08T08:53:17.683 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087998980000000004, 2024-05-08T08:53:17.683 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:17.683 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:17.683 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:17.683 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:17.683 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:17.683 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:17.683 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 969 2024-05-08T08:53:17.684 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.684 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:17.684 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.684 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:17.684 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:53:17.684 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:17.684 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.684 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.684 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:17.684 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:53:17.684 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:17.684 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.684 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.684 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:17.684 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:53:17.685 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3599999999999999e-07 2024-05-08T08:53:17.685 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.685 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.685 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:17.685 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451177+0000", 2024-05-08T08:53:17.685 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4590000000000001e-06 2024-05-08T08:53:17.685 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.685 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.685 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:17.685 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451184+0000", 2024-05-08T08:53:17.685 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4109999999999999e-06 2024-05-08T08:53:17.685 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.685 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.685 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:17.685 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451191+0000", 2024-05-08T08:53:17.685 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7279999999999998e-06 2024-05-08T08:53:17.686 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.686 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.686 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:17.686 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538997+0000", 2024-05-08T08:53:17.686 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087806170000000003 2024-05-08T08:53:17.686 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.686 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.686 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:17.686 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539069+0000", 2024-05-08T08:53:17.686 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2373999999999995e-05 2024-05-08T08:53:17.686 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.686 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.686 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:17.686 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539173+0000", 2024-05-08T08:53:17.687 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000103402 2024-05-08T08:53:17.687 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:17.687 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:17.687 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:17.687 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.687 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.687 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:970 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 3407872~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:17.687 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451198+0000", 2024-05-08T08:53:17.687 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 8.1953382260000005, 2024-05-08T08:53:17.687 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088183368999999998, 2024-05-08T08:53:17.687 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:17.687 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:17.687 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:17.687 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:17.687 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:17.687 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:17.688 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 970 2024-05-08T08:53:17.688 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.688 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:17.688 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.688 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:17.688 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451198+0000", 2024-05-08T08:53:17.688 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:17.688 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.688 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.688 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:17.688 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451198+0000", 2024-05-08T08:53:17.688 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:17.688 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.688 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.688 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:17.689 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451199+0000", 2024-05-08T08:53:17.689 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7400000000000003e-07 2024-05-08T08:53:17.689 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.689 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.689 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:17.689 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451211+0000", 2024-05-08T08:53:17.689 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2109999999999999e-05 2024-05-08T08:53:17.689 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.689 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.689 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:17.689 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451219+0000", 2024-05-08T08:53:17.689 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4170000000000003e-06 2024-05-08T08:53:17.689 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.689 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.689 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:17.689 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451224+0000", 2024-05-08T08:53:17.690 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3990000000000003e-06 2024-05-08T08:53:17.690 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.690 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.690 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:17.690 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539204+0000", 2024-05-08T08:53:17.690 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087980099000000006 2024-05-08T08:53:17.690 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.690 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.690 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:17.693 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539276+0000", 2024-05-08T08:53:17.693 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1513000000000005e-05 2024-05-08T08:53:17.693 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.693 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.693 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:17.693 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539382+0000", 2024-05-08T08:53:17.694 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010615699999999999 2024-05-08T08:53:17.694 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:17.694 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:17.694 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:17.694 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.694 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.694 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:971 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 3899392~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:17.694 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451233+0000", 2024-05-08T08:53:17.694 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 8.1953035189999994, 2024-05-08T08:53:17.694 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088431525999999996, 2024-05-08T08:53:17.694 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:17.694 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:17.694 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:17.695 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:17.695 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:17.695 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:17.695 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 971 2024-05-08T08:53:17.695 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.695 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:17.695 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.695 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:17.695 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451233+0000", 2024-05-08T08:53:17.695 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:17.695 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.695 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.695 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:17.695 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451233+0000", 2024-05-08T08:53:17.695 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:17.695 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.695 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.696 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:17.696 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451234+0000", 2024-05-08T08:53:17.696 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8500000000000001e-07 2024-05-08T08:53:17.696 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.696 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.696 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:17.696 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451237+0000", 2024-05-08T08:53:17.696 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8320000000000002e-06 2024-05-08T08:53:17.696 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.696 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.696 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:17.696 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451243+0000", 2024-05-08T08:53:17.696 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6749999999999996e-06 2024-05-08T08:53:17.696 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.696 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.696 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:17.696 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451249+0000", 2024-05-08T08:53:17.697 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1229999999999997e-06 2024-05-08T08:53:17.697 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.697 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.697 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:17.697 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539412+0000", 2024-05-08T08:53:17.697 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088162668999999999 2024-05-08T08:53:17.697 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.697 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.697 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:17.697 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539505+0000", 2024-05-08T08:53:17.697 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3240999999999997e-05 2024-05-08T08:53:17.697 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.697 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.697 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:17.697 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539665+0000", 2024-05-08T08:53:17.698 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00015930100000000001 2024-05-08T08:53:17.698 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:17.698 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:17.698 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:17.698 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.698 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.698 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:972 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 196608~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:17.698 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451257+0000", 2024-05-08T08:53:17.698 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 8.1952795740000006, 2024-05-08T08:53:17.698 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088719928000000003, 2024-05-08T08:53:17.698 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:17.698 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:17.698 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:17.698 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:17.699 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:17.699 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:17.699 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 972 2024-05-08T08:53:17.699 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.699 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:17.699 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.699 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:17.699 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451257+0000", 2024-05-08T08:53:17.699 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:17.699 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.699 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.699 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:17.699 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451257+0000", 2024-05-08T08:53:17.700 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:17.700 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.700 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.700 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:17.700 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451258+0000", 2024-05-08T08:53:17.700 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7999999999999996e-07 2024-05-08T08:53:17.700 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.700 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.700 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:17.700 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451260+0000", 2024-05-08T08:53:17.700 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4609999999999998e-06 2024-05-08T08:53:17.700 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.700 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.700 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:17.700 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451267+0000", 2024-05-08T08:53:17.701 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.81e-06 2024-05-08T08:53:17.701 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.701 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.701 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:17.701 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451273+0000", 2024-05-08T08:53:17.701 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0859999999999998e-06 2024-05-08T08:53:17.701 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.701 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.701 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:17.701 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539718+0000", 2024-05-08T08:53:17.701 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088445059000000006 2024-05-08T08:53:17.701 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.701 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.701 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:17.701 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539834+0000", 2024-05-08T08:53:17.701 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000116426 2024-05-08T08:53:17.702 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.702 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.702 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:17.702 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539977+0000", 2024-05-08T08:53:17.702 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014260600000000001 2024-05-08T08:53:17.702 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:17.702 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:17.702 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:17.702 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.702 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.702 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:973 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 688128~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:17.702 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:53:17.702 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 8.1952569900000007, 2024-05-08T08:53:17.702 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088918496, 2024-05-08T08:53:17.702 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:17.703 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:17.703 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:17.703 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:17.703 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:17.703 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:17.703 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 973 2024-05-08T08:53:17.703 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.703 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:17.703 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.703 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:17.703 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:53:17.703 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:17.703 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.703 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.703 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:17.704 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:53:17.704 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:17.704 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.704 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.704 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:17.704 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:53:17.704 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9099999999999999e-07 2024-05-08T08:53:17.704 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.704 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.704 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:17.704 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451282+0000", 2024-05-08T08:53:17.704 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1550000000000001e-06 2024-05-08T08:53:17.704 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.704 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.704 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:17.705 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451288+0000", 2024-05-08T08:53:17.705 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8889999999999999e-06 2024-05-08T08:53:17.705 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.705 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.705 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:17.705 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451294+0000", 2024-05-08T08:53:17.705 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0850000000000002e-06 2024-05-08T08:53:17.705 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.705 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.705 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:17.705 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540011+0000", 2024-05-08T08:53:17.705 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088717314000000005 2024-05-08T08:53:17.705 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.705 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.705 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:17.706 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540089+0000", 2024-05-08T08:53:17.706 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7438e-05 2024-05-08T08:53:17.706 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.706 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.706 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:17.706 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540198+0000", 2024-05-08T08:53:17.706 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000109224 2024-05-08T08:53:17.706 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:17.706 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:17.706 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:17.706 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.706 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.706 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:974 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 1179648~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:17.707 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451301+0000", 2024-05-08T08:53:17.707 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 8.1952353569999996, 2024-05-08T08:53:17.707 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089105078000000004, 2024-05-08T08:53:17.707 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:17.707 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:17.707 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:17.707 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:17.707 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:17.707 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:17.707 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 974 2024-05-08T08:53:17.707 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.707 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:17.707 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.707 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:17.707 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451301+0000", 2024-05-08T08:53:17.708 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:17.708 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.708 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.708 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:17.708 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451301+0000", 2024-05-08T08:53:17.708 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:17.708 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.708 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.708 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:17.708 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451302+0000", 2024-05-08T08:53:17.708 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7800000000000002e-07 2024-05-08T08:53:17.708 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.708 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.708 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:17.708 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451304+0000", 2024-05-08T08:53:17.708 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.65e-06 2024-05-08T08:53:17.708 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.709 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.709 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:17.709 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451311+0000", 2024-05-08T08:53:17.709 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.472e-06 2024-05-08T08:53:17.709 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.709 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.709 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:17.709 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451316+0000", 2024-05-08T08:53:17.709 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9189999999999998e-06 2024-05-08T08:53:17.709 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.709 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.709 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:17.709 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540234+0000", 2024-05-08T08:53:17.709 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088917752000000003 2024-05-08T08:53:17.709 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.709 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.710 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:17.710 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540308+0000", 2024-05-08T08:53:17.710 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4758000000000005e-05 2024-05-08T08:53:17.710 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.710 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.710 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:17.710 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540406+0000", 2024-05-08T08:53:17.710 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8048999999999996e-05 2024-05-08T08:53:17.710 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:17.710 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:17.710 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:17.710 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.710 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.710 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:975 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 1671168~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:17.711 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:53:17.711 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 8.1951930270000002, 2024-05-08T08:53:17.711 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089294139999999994, 2024-05-08T08:53:17.711 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:17.711 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:17.711 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:17.711 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:17.711 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:17.711 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:17.711 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 975 2024-05-08T08:53:17.711 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.711 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:17.711 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.711 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:17.711 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:53:17.712 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:17.712 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.712 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.712 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:17.712 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:53:17.712 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:17.712 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.712 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.712 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:17.712 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:53:17.712 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1299999999999999e-07 2024-05-08T08:53:17.712 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.712 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.712 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:17.713 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451348+0000", 2024-05-08T08:53:17.713 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.484e-06 2024-05-08T08:53:17.713 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.713 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.713 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:17.713 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451356+0000", 2024-05-08T08:53:17.713 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8499999999999994e-06 2024-05-08T08:53:17.713 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.713 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.713 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:17.713 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451366+0000", 2024-05-08T08:53:17.713 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.9850000000000001e-06 2024-05-08T08:53:17.713 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.713 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.713 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:17.714 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540448+0000", 2024-05-08T08:53:17.714 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089082254 2024-05-08T08:53:17.714 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.714 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.714 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:17.714 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540528+0000", 2024-05-08T08:53:17.714 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0234999999999993e-05 2024-05-08T08:53:17.714 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.714 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.714 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:17.714 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540638+0000", 2024-05-08T08:53:17.714 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010951900000000001 2024-05-08T08:53:17.714 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:17.714 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:17.715 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:17.715 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.715 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.715 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:976 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 2162688~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:17.715 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451375+0000", 2024-05-08T08:53:17.715 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 8.1951612800000007, 2024-05-08T08:53:17.715 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089472745000000006, 2024-05-08T08:53:17.715 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:17.715 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:17.715 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:17.715 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:17.715 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:17.715 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:17.715 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 976 2024-05-08T08:53:17.715 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.716 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:17.716 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.716 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:17.716 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451375+0000", 2024-05-08T08:53:17.716 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:17.716 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.716 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.716 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:17.716 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451375+0000", 2024-05-08T08:53:17.716 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:17.716 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.716 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.716 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:17.716 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451376+0000", 2024-05-08T08:53:17.716 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2399999999999998e-07 2024-05-08T08:53:17.717 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.717 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.717 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:17.717 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451379+0000", 2024-05-08T08:53:17.717 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6359999999999998e-06 2024-05-08T08:53:17.717 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.717 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.717 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:17.717 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451387+0000", 2024-05-08T08:53:17.717 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9929999999999995e-06 2024-05-08T08:53:17.717 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.717 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.717 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:17.717 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451393+0000", 2024-05-08T08:53:17.717 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4699999999999999e-06 2024-05-08T08:53:17.718 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.718 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.718 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:17.718 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540677+0000", 2024-05-08T08:53:17.718 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089283504999999999 2024-05-08T08:53:17.718 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.718 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.718 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:17.718 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540743+0000", 2024-05-08T08:53:17.718 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6303999999999994e-05 2024-05-08T08:53:17.718 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.719 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.719 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:17.719 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540848+0000", 2024-05-08T08:53:17.719 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000105213 2024-05-08T08:53:17.719 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:17.719 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:17.719 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:17.719 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.719 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.719 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:977 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 2654208~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:17.719 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:53:17.719 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 8.1951358479999996, 2024-05-08T08:53:17.719 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089698473000000001, 2024-05-08T08:53:17.719 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:17.719 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:17.720 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:17.720 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:17.720 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:17.720 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:17.720 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 977 2024-05-08T08:53:17.720 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.720 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:17.720 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.720 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:17.720 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:53:17.720 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:17.720 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.720 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.721 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:17.721 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:53:17.721 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:17.721 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.721 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.721 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:17.721 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:53:17.721 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2899999999999999e-07 2024-05-08T08:53:17.721 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.721 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.721 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:17.721 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451404+0000", 2024-05-08T08:53:17.721 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6649999999999999e-06 2024-05-08T08:53:17.721 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.722 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.722 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:17.722 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451410+0000", 2024-05-08T08:53:17.722 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3269999999999997e-06 2024-05-08T08:53:17.722 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.722 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.722 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:17.722 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451415+0000", 2024-05-08T08:53:17.722 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9529999999999998e-06 2024-05-08T08:53:17.722 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.722 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.722 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:17.722 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540912+0000", 2024-05-08T08:53:17.722 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089496950000000006 2024-05-08T08:53:17.722 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.723 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.723 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:17.723 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541000+0000", 2024-05-08T08:53:17.723 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7429000000000006e-05 2024-05-08T08:53:17.723 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.723 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.723 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:17.723 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541099+0000", 2024-05-08T08:53:17.723 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.9720000000000001e-05 2024-05-08T08:53:17.723 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:17.723 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:17.723 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:17.723 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.723 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.724 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:978 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 3145728~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:17.724 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:53:17.724 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 8.1951146399999999, 2024-05-08T08:53:17.724 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089900635000000007, 2024-05-08T08:53:17.724 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:17.724 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:17.724 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:17.724 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:17.724 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:17.724 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:17.724 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 978 2024-05-08T08:53:17.724 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.724 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:17.724 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.724 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:17.725 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:53:17.725 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:17.725 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.725 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.725 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:17.725 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:53:17.725 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:17.725 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.725 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.725 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:17.725 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:53:17.725 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9400000000000001e-07 2024-05-08T08:53:17.725 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.725 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.726 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:17.726 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451425+0000", 2024-05-08T08:53:17.726 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2460000000000002e-06 2024-05-08T08:53:17.726 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.726 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.726 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:17.726 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451431+0000", 2024-05-08T08:53:17.726 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2419999999999997e-06 2024-05-08T08:53:17.726 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.726 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.726 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:17.726 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451436+0000", 2024-05-08T08:53:17.726 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.887e-06 2024-05-08T08:53:17.726 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.727 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.727 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:17.727 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541152+0000", 2024-05-08T08:53:17.727 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089716194999999999 2024-05-08T08:53:17.727 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.727 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.727 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:17.727 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541224+0000", 2024-05-08T08:53:17.727 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2416000000000006e-05 2024-05-08T08:53:17.727 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.727 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.727 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:17.727 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541323+0000", 2024-05-08T08:53:17.727 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8255e-05 2024-05-08T08:53:17.728 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:17.728 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:17.728 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:17.728 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.728 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.728 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:979 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 3637248~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:17.728 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451462+0000", 2024-05-08T08:53:17.728 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 8.1950741909999998, 2024-05-08T08:53:17.728 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.090036441999999994, 2024-05-08T08:53:17.728 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:17.728 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:17.728 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:17.728 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:17.728 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:17.728 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:17.728 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 979 2024-05-08T08:53:17.729 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.729 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:17.729 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.729 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:17.729 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451462+0000", 2024-05-08T08:53:17.729 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:17.729 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.729 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.729 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:17.729 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451462+0000", 2024-05-08T08:53:17.729 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:17.729 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.729 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.729 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:17.729 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451463+0000", 2024-05-08T08:53:17.730 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4300000000000002e-07 2024-05-08T08:53:17.730 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.730 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.730 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:17.730 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451471+0000", 2024-05-08T08:53:17.730 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0800000000000006e-06 2024-05-08T08:53:17.730 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.730 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.730 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:17.730 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451481+0000", 2024-05-08T08:53:17.730 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.38e-06 2024-05-08T08:53:17.730 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.730 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.731 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:17.731 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451487+0000", 2024-05-08T08:53:17.731 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4590000000000003e-06 2024-05-08T08:53:17.731 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.731 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.731 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:17.731 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541350+0000", 2024-05-08T08:53:17.731 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089862599000000001 2024-05-08T08:53:17.731 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.731 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.731 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:17.731 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541407+0000", 2024-05-08T08:53:17.731 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7667999999999997e-05 2024-05-08T08:53:17.731 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.732 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.732 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:17.732 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541499+0000", 2024-05-08T08:53:17.732 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1513000000000003e-05 2024-05-08T08:53:17.732 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:17.732 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:17.732 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:17.732 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.732 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.732 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:980 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 4128768~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:17.732 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451496+0000", 2024-05-08T08:53:17.732 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 8.1950404349999992, 2024-05-08T08:53:17.732 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.090258985, 2024-05-08T08:53:17.732 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:17.732 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:17.733 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:17.733 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:17.733 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:17.733 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:17.733 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 980 2024-05-08T08:53:17.733 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.733 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:17.733 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.733 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:17.733 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451496+0000", 2024-05-08T08:53:17.733 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:17.733 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.733 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.733 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:17.734 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451496+0000", 2024-05-08T08:53:17.734 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:17.734 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.734 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.734 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:17.734 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451497+0000", 2024-05-08T08:53:17.734 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7599999999999997e-07 2024-05-08T08:53:17.734 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.734 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.734 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:17.734 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451500+0000", 2024-05-08T08:53:17.734 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7930000000000002e-06 2024-05-08T08:53:17.734 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.734 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.734 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:17.735 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451506+0000", 2024-05-08T08:53:17.735 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6980000000000001e-06 2024-05-08T08:53:17.735 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.735 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.735 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:17.735 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451512+0000", 2024-05-08T08:53:17.735 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2490000000000001e-06 2024-05-08T08:53:17.735 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.735 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.735 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:17.735 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541550+0000", 2024-05-08T08:53:17.735 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.090037961999999999 2024-05-08T08:53:17.735 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.735 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.736 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:17.736 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541647+0000", 2024-05-08T08:53:17.736 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6924999999999998e-05 2024-05-08T08:53:17.736 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:17.736 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:17.736 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:17.736 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541755+0000", 2024-05-08T08:53:17.736 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000108682 2024-05-08T08:53:17.736 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:17.736 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:17.736 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:17.736 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:17.736 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:17.736 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:17.738 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:53:17.739 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:53:17.741+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:53:17.838 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:17.841+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:17.844 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:53:17.844 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:53:17.844 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:53:17.844 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:53:17.852 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:53:17.939 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:53:17.942+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:53:17.956 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:53:17.956 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:53:17.956 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:53:17.956 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:53:17.956 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:53:17.965 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:53:18.039 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:18.042+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:18.071 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:53:18.071 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:53:18.072 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:53:18.072 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:53:18.072 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.072 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4742 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1832960~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:18.072 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.585513+0000", 2024-05-08T08:53:18.072 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 46.488331494000001, 2024-05-08T08:53:18.072 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035588763000000002, 2024-05-08T08:53:18.072 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:18.072 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:18.072 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:18.072 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:18.072 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:18.072 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:18.072 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4742 2024-05-08T08:53:18.072 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.073 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:18.073 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.073 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:18.073 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585513+0000", 2024-05-08T08:53:18.073 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:18.073 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.073 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.073 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:18.073 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585513+0000", 2024-05-08T08:53:18.073 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:18.073 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.073 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.073 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:18.073 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585514+0000", 2024-05-08T08:53:18.074 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6100000000000001e-07 2024-05-08T08:53:18.074 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.074 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.074 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:18.074 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585516+0000", 2024-05-08T08:53:18.074 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.526e-06 2024-05-08T08:53:18.074 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.074 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.074 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:18.074 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585523+0000", 2024-05-08T08:53:18.074 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.562e-06 2024-05-08T08:53:18.074 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.074 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.075 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:18.075 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585528+0000", 2024-05-08T08:53:18.075 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7990000000000001e-06 2024-05-08T08:53:18.075 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.075 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.075 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:18.075 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.620939+0000", 2024-05-08T08:53:18.075 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035411419 2024-05-08T08:53:18.075 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.075 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.075 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:18.075 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621014+0000", 2024-05-08T08:53:18.075 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5017000000000004e-05 2024-05-08T08:53:18.075 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.075 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.075 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:18.076 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621102+0000", 2024-05-08T08:53:18.076 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7979000000000006e-05 2024-05-08T08:53:18.076 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.076 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:18.076 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.076 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.076 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.076 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4743 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1849344~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:18.076 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.585628+0000", 2024-05-08T08:53:18.076 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 46.488216606999998, 2024-05-08T08:53:18.076 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035659962000000003, 2024-05-08T08:53:18.076 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:18.076 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:18.076 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:18.076 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:18.077 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:18.077 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:18.077 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4743 2024-05-08T08:53:18.077 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.077 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:18.077 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.077 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:18.077 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585628+0000", 2024-05-08T08:53:18.077 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:18.077 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.077 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.077 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:18.077 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585628+0000", 2024-05-08T08:53:18.077 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:18.077 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.078 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.078 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:18.078 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585629+0000", 2024-05-08T08:53:18.078 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2300000000000001e-07 2024-05-08T08:53:18.078 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.078 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.078 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:18.078 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585632+0000", 2024-05-08T08:53:18.078 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1370000000000002e-06 2024-05-08T08:53:18.078 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.078 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.078 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:18.078 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585641+0000", 2024-05-08T08:53:18.078 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5220000000000007e-06 2024-05-08T08:53:18.079 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.079 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.079 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:18.079 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585646+0000", 2024-05-08T08:53:18.079 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.609e-06 2024-05-08T08:53:18.079 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.079 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.079 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:18.079 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621132+0000", 2024-05-08T08:53:18.079 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035485472999999997 2024-05-08T08:53:18.079 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.079 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.079 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:18.079 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621204+0000", 2024-05-08T08:53:18.079 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2259000000000001e-05 2024-05-08T08:53:18.079 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.080 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.080 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:18.080 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621288+0000", 2024-05-08T08:53:18.080 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4338999999999993e-05 2024-05-08T08:53:18.080 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.080 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:18.080 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.080 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.080 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.080 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4746 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1865728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:18.080 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.585656+0000", 2024-05-08T08:53:18.080 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 46.488188925000003, 2024-05-08T08:53:18.080 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035815909, 2024-05-08T08:53:18.080 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:18.080 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:18.081 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:18.081 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:18.081 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:18.081 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:18.081 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4746 2024-05-08T08:53:18.081 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.081 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:18.081 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.081 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:18.081 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585656+0000", 2024-05-08T08:53:18.081 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:18.081 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.081 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.081 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:18.081 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585656+0000", 2024-05-08T08:53:18.081 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:18.082 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.082 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.082 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:18.082 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585656+0000", 2024-05-08T08:53:18.082 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.08e-07 2024-05-08T08:53:18.082 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.082 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.082 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:18.082 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585658+0000", 2024-05-08T08:53:18.082 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0889999999999998e-06 2024-05-08T08:53:18.082 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.082 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.082 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:18.082 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585664+0000", 2024-05-08T08:53:18.082 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3990000000000003e-06 2024-05-08T08:53:18.083 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.083 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.083 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:18.083 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585668+0000", 2024-05-08T08:53:18.083 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3170000000000003e-06 2024-05-08T08:53:18.083 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.083 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.083 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:18.083 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621316+0000", 2024-05-08T08:53:18.083 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035647908999999998 2024-05-08T08:53:18.083 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.083 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.083 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:18.083 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621389+0000", 2024-05-08T08:53:18.083 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2531000000000001e-05 2024-05-08T08:53:18.084 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.084 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.084 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:18.084 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621472+0000", 2024-05-08T08:53:18.084 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3256000000000004e-05 2024-05-08T08:53:18.084 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.084 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:18.084 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.084 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.084 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.084 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4748 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1882112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:18.084 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.585697+0000", 2024-05-08T08:53:18.084 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 46.488147478999998, 2024-05-08T08:53:18.084 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035962485000000002, 2024-05-08T08:53:18.084 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:18.085 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:18.085 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:18.085 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:18.085 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:18.085 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:18.085 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4748 2024-05-08T08:53:18.085 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.085 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:18.085 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.085 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:18.085 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585697+0000", 2024-05-08T08:53:18.085 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:18.085 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.085 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.085 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:18.086 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585697+0000", 2024-05-08T08:53:18.086 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:18.086 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.086 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.086 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:18.086 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585698+0000", 2024-05-08T08:53:18.086 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0900000000000001e-07 2024-05-08T08:53:18.086 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.086 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.086 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:18.086 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585701+0000", 2024-05-08T08:53:18.086 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0149999999999999e-06 2024-05-08T08:53:18.086 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.086 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.087 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:18.087 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585712+0000", 2024-05-08T08:53:18.087 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0655e-05 2024-05-08T08:53:18.087 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.087 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.087 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:18.087 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585717+0000", 2024-05-08T08:53:18.087 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3800000000000002e-06 2024-05-08T08:53:18.087 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.087 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.087 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:18.087 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621502+0000", 2024-05-08T08:53:18.087 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035784559000000001 2024-05-08T08:53:18.087 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.087 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.088 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:18.088 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621575+0000", 2024-05-08T08:53:18.088 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3269999999999995e-05 2024-05-08T08:53:18.088 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.088 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.088 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:18.088 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621660+0000", 2024-05-08T08:53:18.088 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4997e-05 2024-05-08T08:53:18.088 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.088 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:18.088 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.088 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.088 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.088 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4750 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1898496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:18.088 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.585810+0000", 2024-05-08T08:53:18.089 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 46.488034648999999, 2024-05-08T08:53:18.089 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036034928000000001, 2024-05-08T08:53:18.089 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:18.089 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:18.089 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:18.089 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:18.089 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:18.089 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:18.089 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4750 2024-05-08T08:53:18.089 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.089 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:18.089 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.089 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:18.089 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585810+0000", 2024-05-08T08:53:18.089 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:18.090 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.090 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.090 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:18.090 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585810+0000", 2024-05-08T08:53:18.090 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:18.090 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.090 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.090 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:18.090 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585811+0000", 2024-05-08T08:53:18.090 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6499999999999999e-07 2024-05-08T08:53:18.090 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.090 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.090 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:18.090 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585814+0000", 2024-05-08T08:53:18.091 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1709999999999998e-06 2024-05-08T08:53:18.091 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.091 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.091 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:18.091 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585822+0000", 2024-05-08T08:53:18.091 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8909999999999995e-06 2024-05-08T08:53:18.091 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.091 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.091 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:18.091 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585827+0000", 2024-05-08T08:53:18.091 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5289999999999999e-06 2024-05-08T08:53:18.091 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.091 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.091 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:18.091 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621688+0000", 2024-05-08T08:53:18.092 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035861111000000001 2024-05-08T08:53:18.092 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.092 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.092 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:18.092 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621761+0000", 2024-05-08T08:53:18.092 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2211000000000004e-05 2024-05-08T08:53:18.092 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.092 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.092 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:18.092 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621845+0000", 2024-05-08T08:53:18.092 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4450000000000006e-05 2024-05-08T08:53:18.092 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.092 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:18.092 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.092 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.092 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.093 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4752 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1914880~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:18.093 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.585858+0000", 2024-05-08T08:53:18.093 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 46.487986679999999, 2024-05-08T08:53:18.093 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036180081000000003, 2024-05-08T08:53:18.093 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:18.093 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:18.093 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:18.093 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:18.093 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:18.093 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:18.093 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4752 2024-05-08T08:53:18.093 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.093 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:18.093 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.093 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:18.094 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585858+0000", 2024-05-08T08:53:18.094 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:18.094 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.094 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.094 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:18.094 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585858+0000", 2024-05-08T08:53:18.094 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:18.094 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.094 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.094 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:18.094 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585859+0000", 2024-05-08T08:53:18.094 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.82e-07 2024-05-08T08:53:18.094 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.094 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.094 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:18.095 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585861+0000", 2024-05-08T08:53:18.095 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.695e-06 2024-05-08T08:53:18.095 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.095 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.095 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:18.095 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585869+0000", 2024-05-08T08:53:18.095 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0299999999999994e-06 2024-05-08T08:53:18.095 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.095 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.095 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:18.095 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585874+0000", 2024-05-08T08:53:18.095 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9620000000000001e-06 2024-05-08T08:53:18.095 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.095 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.095 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:18.095 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621878+0000", 2024-05-08T08:53:18.096 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036004043999999999 2024-05-08T08:53:18.096 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.096 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.096 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:18.096 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621954+0000", 2024-05-08T08:53:18.096 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5414000000000007e-05 2024-05-08T08:53:18.096 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.096 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.096 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:18.096 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622038+0000", 2024-05-08T08:53:18.096 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4454000000000002e-05 2024-05-08T08:53:18.096 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.096 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:18.096 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.096 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.097 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.097 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4753 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1931264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:18.097 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.585947+0000", 2024-05-08T08:53:18.097 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 46.487898031999997, 2024-05-08T08:53:18.097 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036267992999999998, 2024-05-08T08:53:18.097 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:18.097 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:18.097 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:18.097 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:18.097 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:18.097 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:18.097 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4753 2024-05-08T08:53:18.097 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.097 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:18.097 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.098 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:18.098 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585947+0000", 2024-05-08T08:53:18.098 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:18.098 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.098 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.098 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:18.098 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585947+0000", 2024-05-08T08:53:18.098 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:18.098 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.098 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.098 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:18.098 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585948+0000", 2024-05-08T08:53:18.098 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6000000000000002e-07 2024-05-08T08:53:18.098 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.098 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.099 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:18.099 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585952+0000", 2024-05-08T08:53:18.099 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7090000000000001e-06 2024-05-08T08:53:18.099 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.099 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.099 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:18.099 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585963+0000", 2024-05-08T08:53:18.099 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0740000000000001e-05 2024-05-08T08:53:18.099 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.099 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.099 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:18.099 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585970+0000", 2024-05-08T08:53:18.099 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3309999999999998e-06 2024-05-08T08:53:18.099 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.100 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.100 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:18.100 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622065+0000", 2024-05-08T08:53:18.100 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036094676999999999 2024-05-08T08:53:18.100 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.100 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.100 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:18.100 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622132+0000", 2024-05-08T08:53:18.100 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7214999999999999e-05 2024-05-08T08:53:18.100 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.100 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.100 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:18.100 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622215+0000", 2024-05-08T08:53:18.100 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2460999999999995e-05 2024-05-08T08:53:18.101 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.101 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:18.101 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.101 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.101 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.101 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4756 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1947648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:18.101 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586018+0000", 2024-05-08T08:53:18.101 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 46.487826388000002, 2024-05-08T08:53:18.101 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036382335000000002, 2024-05-08T08:53:18.101 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:18.101 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:18.101 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:18.101 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:18.101 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:18.101 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:18.101 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4756 2024-05-08T08:53:18.102 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.102 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:18.102 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.102 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:18.102 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586018+0000", 2024-05-08T08:53:18.102 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:18.102 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.102 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.102 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:18.102 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586018+0000", 2024-05-08T08:53:18.102 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:18.102 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.102 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.102 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:18.103 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586019+0000", 2024-05-08T08:53:18.103 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9999999999999997e-07 2024-05-08T08:53:18.103 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.103 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.103 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:18.103 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586023+0000", 2024-05-08T08:53:18.103 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7330000000000001e-06 2024-05-08T08:53:18.103 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.103 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.103 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:18.103 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586031+0000", 2024-05-08T08:53:18.103 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4940000000000003e-06 2024-05-08T08:53:18.103 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.103 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.103 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:18.103 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586038+0000", 2024-05-08T08:53:18.104 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2539999999999998e-06 2024-05-08T08:53:18.104 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.104 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.104 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:18.104 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622243+0000", 2024-05-08T08:53:18.104 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036205009000000003 2024-05-08T08:53:18.104 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.104 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.104 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:18.104 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622313+0000", 2024-05-08T08:53:18.104 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9912000000000004e-05 2024-05-08T08:53:18.104 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.104 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.104 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:18.104 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622401+0000", 2024-05-08T08:53:18.105 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8232999999999994e-05 2024-05-08T08:53:18.105 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.105 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:18.105 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.105 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.105 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.105 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4758 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1964032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:18.105 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586080+0000", 2024-05-08T08:53:18.105 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 46.487764968999997, 2024-05-08T08:53:18.105 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036547659000000003, 2024-05-08T08:53:18.105 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:18.105 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:18.105 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:18.105 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:18.106 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:18.106 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:18.106 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4758 2024-05-08T08:53:18.106 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.106 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:18.106 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.106 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:18.106 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586080+0000", 2024-05-08T08:53:18.106 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:18.106 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.106 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.106 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:18.106 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586080+0000", 2024-05-08T08:53:18.106 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:18.106 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.107 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:18.107 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586081+0000", 2024-05-08T08:53:18.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5499999999999998e-07 2024-05-08T08:53:18.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.107 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:18.107 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586084+0000", 2024-05-08T08:53:18.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9919999999999999e-06 2024-05-08T08:53:18.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.107 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:18.107 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586091+0000", 2024-05-08T08:53:18.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5440000000000001e-06 2024-05-08T08:53:18.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.108 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:18.108 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586098+0000", 2024-05-08T08:53:18.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8639999999999998e-06 2024-05-08T08:53:18.108 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.108 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.108 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:18.108 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622442+0000", 2024-05-08T08:53:18.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036344110999999998 2024-05-08T08:53:18.108 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.108 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.108 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:18.108 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622516+0000", 2024-05-08T08:53:18.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4089000000000002e-05 2024-05-08T08:53:18.108 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.108 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.108 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:18.109 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622628+0000", 2024-05-08T08:53:18.109 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00011140400000000001 2024-05-08T08:53:18.109 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.109 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:18.109 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.109 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.109 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.109 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4760 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1980416~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:18.109 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586178+0000", 2024-05-08T08:53:18.109 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 46.487666357000002, 2024-05-08T08:53:18.109 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036647526999999999, 2024-05-08T08:53:18.109 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:18.109 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:18.109 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:18.109 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:18.110 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:18.110 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:18.110 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4760 2024-05-08T08:53:18.110 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.110 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:18.110 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.110 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:18.110 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586178+0000", 2024-05-08T08:53:18.110 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:18.110 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.110 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.110 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:18.110 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586178+0000", 2024-05-08T08:53:18.110 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:18.110 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.111 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.111 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:18.111 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586179+0000", 2024-05-08T08:53:18.111 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8599999999999997e-07 2024-05-08T08:53:18.111 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.111 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.111 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:18.111 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586182+0000", 2024-05-08T08:53:18.111 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.174e-06 2024-05-08T08:53:18.111 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.111 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.111 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:18.111 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586191+0000", 2024-05-08T08:53:18.111 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1869999999999997e-06 2024-05-08T08:53:18.111 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.111 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.112 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:18.112 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586197+0000", 2024-05-08T08:53:18.112 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2090000000000002e-06 2024-05-08T08:53:18.112 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.112 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.112 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:18.112 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622657+0000", 2024-05-08T08:53:18.112 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036459793999999997 2024-05-08T08:53:18.112 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.112 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.112 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:18.112 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622725+0000", 2024-05-08T08:53:18.112 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8374999999999995e-05 2024-05-08T08:53:18.112 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.112 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.113 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:18.113 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622826+0000", 2024-05-08T08:53:18.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010100200000000001 2024-05-08T08:53:18.113 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.113 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:18.113 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.113 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.113 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.113 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4762 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1996800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:18.113 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586299+0000", 2024-05-08T08:53:18.113 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 46.487546025999997, 2024-05-08T08:53:18.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036720653999999998, 2024-05-08T08:53:18.113 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:18.113 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:18.113 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:18.114 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:18.114 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:18.114 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:18.114 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4762 2024-05-08T08:53:18.114 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.114 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:18.114 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.114 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:18.114 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586299+0000", 2024-05-08T08:53:18.114 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:18.114 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.114 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.114 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:18.114 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586299+0000", 2024-05-08T08:53:18.115 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:18.115 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.115 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.115 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:18.115 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586299+0000", 2024-05-08T08:53:18.115 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.99e-07 2024-05-08T08:53:18.115 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.115 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.115 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:18.115 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586303+0000", 2024-05-08T08:53:18.115 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1949999999999999e-06 2024-05-08T08:53:18.115 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.115 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.115 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:18.115 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586318+0000", 2024-05-08T08:53:18.116 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5401999999999999e-05 2024-05-08T08:53:18.116 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.116 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.116 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:18.116 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586324+0000", 2024-05-08T08:53:18.116 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8119999999999996e-06 2024-05-08T08:53:18.116 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.116 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.116 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:18.116 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622853+0000", 2024-05-08T08:53:18.116 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036529013999999999 2024-05-08T08:53:18.116 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.116 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.116 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:18.117 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622924+0000", 2024-05-08T08:53:18.117 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1654000000000002e-05 2024-05-08T08:53:18.117 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.117 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.117 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:18.117 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623019+0000", 2024-05-08T08:53:18.117 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4977999999999998e-05 2024-05-08T08:53:18.117 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.117 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:18.117 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.117 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.117 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.117 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4764 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 2013184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:18.117 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586506+0000", 2024-05-08T08:53:18.118 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 46.487338921000003, 2024-05-08T08:53:18.118 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036859467, 2024-05-08T08:53:18.118 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:18.118 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:18.118 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:18.118 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:18.118 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:18.118 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:18.118 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4764 2024-05-08T08:53:18.118 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.118 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:18.118 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.118 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:18.118 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586506+0000", 2024-05-08T08:53:18.118 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:18.119 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.119 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.119 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:18.119 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586506+0000", 2024-05-08T08:53:18.119 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:18.119 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.119 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.119 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:18.119 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586507+0000", 2024-05-08T08:53:18.119 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9100000000000003e-07 2024-05-08T08:53:18.119 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.119 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.119 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:18.119 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586512+0000", 2024-05-08T08:53:18.120 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9470000000000002e-06 2024-05-08T08:53:18.120 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.120 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.120 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:18.120 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586521+0000", 2024-05-08T08:53:18.120 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3379999999999994e-06 2024-05-08T08:53:18.120 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.120 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.120 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:18.120 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586528+0000", 2024-05-08T08:53:18.120 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1289999999999998e-06 2024-05-08T08:53:18.120 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.120 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.120 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:18.120 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623095+0000", 2024-05-08T08:53:18.121 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036566411 2024-05-08T08:53:18.121 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.121 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.121 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:18.121 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623221+0000", 2024-05-08T08:53:18.121 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012622299999999999 2024-05-08T08:53:18.121 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.121 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.121 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:18.121 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623365+0000", 2024-05-08T08:53:18.121 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000144628 2024-05-08T08:53:18.121 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.121 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:18.121 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.122 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.122 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.122 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4766 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 2029568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:18.122 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586543+0000", 2024-05-08T08:53:18.122 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 46.487301434000003, 2024-05-08T08:53:18.122 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.037033635000000002, 2024-05-08T08:53:18.122 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:18.122 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:18.122 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:18.122 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:18.122 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:18.122 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:18.122 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4766 2024-05-08T08:53:18.122 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.123 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:18.123 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.123 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:18.123 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586543+0000", 2024-05-08T08:53:18.123 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:18.123 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.123 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.123 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:18.123 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586543+0000", 2024-05-08T08:53:18.123 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:18.123 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.123 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.123 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:18.123 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586544+0000", 2024-05-08T08:53:18.123 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3099999999999998e-07 2024-05-08T08:53:18.124 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.124 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.124 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:18.124 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586547+0000", 2024-05-08T08:53:18.124 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.751e-06 2024-05-08T08:53:18.124 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.124 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.124 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:18.124 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586553+0000", 2024-05-08T08:53:18.124 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2870000000000001e-06 2024-05-08T08:53:18.124 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.125 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.125 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:18.125 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586558+0000", 2024-05-08T08:53:18.125 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5709999999999997e-06 2024-05-08T08:53:18.125 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.125 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.125 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:18.125 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623416+0000", 2024-05-08T08:53:18.125 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036857613999999997 2024-05-08T08:53:18.125 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.125 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.125 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:18.125 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623489+0000", 2024-05-08T08:53:18.125 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3172999999999999e-05 2024-05-08T08:53:18.126 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.126 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.126 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:18.126 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623577+0000", 2024-05-08T08:53:18.126 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7807999999999997e-05 2024-05-08T08:53:18.126 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.126 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:18.126 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.126 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.126 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.126 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4768 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 2045952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:18.126 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586566+0000", 2024-05-08T08:53:18.126 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 46.487279127999997, 2024-05-08T08:53:18.127 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.03726206, 2024-05-08T08:53:18.127 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:18.127 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:18.127 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:18.127 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:18.127 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:18.127 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:18.127 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4768 2024-05-08T08:53:18.127 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.127 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:18.127 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.127 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:18.127 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586566+0000", 2024-05-08T08:53:18.127 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:18.127 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.128 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.128 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:18.128 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586566+0000", 2024-05-08T08:53:18.128 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:18.128 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.128 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.128 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:18.128 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586566+0000", 2024-05-08T08:53:18.128 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8299999999999998e-07 2024-05-08T08:53:18.128 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.128 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.128 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:18.128 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586568+0000", 2024-05-08T08:53:18.128 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.063e-06 2024-05-08T08:53:18.128 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.129 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.129 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:18.129 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586574+0000", 2024-05-08T08:53:18.129 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8300000000000001e-06 2024-05-08T08:53:18.129 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.129 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.129 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:18.129 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586579+0000", 2024-05-08T08:53:18.129 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7559999999999999e-06 2024-05-08T08:53:18.129 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.129 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.129 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:18.129 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623612+0000", 2024-05-08T08:53:18.129 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.037033041000000003 2024-05-08T08:53:18.129 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.130 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.130 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:18.130 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623699+0000", 2024-05-08T08:53:18.130 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6817000000000006e-05 2024-05-08T08:53:18.130 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.130 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.130 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:18.130 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623828+0000", 2024-05-08T08:53:18.130 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012917000000000001 2024-05-08T08:53:18.130 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.130 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:18.130 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.130 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.130 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.131 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4770 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 2062336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:18.131 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586585+0000", 2024-05-08T08:53:18.131 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 46.487259944999998, 2024-05-08T08:53:18.131 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.037449891999999999, 2024-05-08T08:53:18.131 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:18.131 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:18.131 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:18.131 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:18.131 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:18.131 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:18.131 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4770 2024-05-08T08:53:18.131 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.131 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:18.131 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.131 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:18.132 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586585+0000", 2024-05-08T08:53:18.132 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:18.132 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.132 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.132 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:18.132 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586585+0000", 2024-05-08T08:53:18.132 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:18.132 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.132 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.132 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:18.132 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586585+0000", 2024-05-08T08:53:18.132 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:53:18.132 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.132 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.132 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:18.133 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586587+0000", 2024-05-08T08:53:18.133 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.181e-06 2024-05-08T08:53:18.133 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.133 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.133 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:18.133 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586592+0000", 2024-05-08T08:53:18.133 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9710000000000003e-06 2024-05-08T08:53:18.133 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.133 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.133 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:18.133 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586598+0000", 2024-05-08T08:53:18.133 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.118e-06 2024-05-08T08:53:18.133 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.133 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.133 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:18.133 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623862+0000", 2024-05-08T08:53:18.134 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.037264711999999998 2024-05-08T08:53:18.134 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.134 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.134 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:18.134 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623941+0000", 2024-05-08T08:53:18.134 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9084000000000004e-05 2024-05-08T08:53:18.134 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.134 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.134 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:18.134 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.624035+0000", 2024-05-08T08:53:18.134 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3456999999999996e-05 2024-05-08T08:53:18.134 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.134 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:18.134 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.135 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.135 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.135 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4772 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 2078720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:18.135 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586621+0000", 2024-05-08T08:53:18.135 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 46.487223997000001, 2024-05-08T08:53:18.135 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.037650391999999998, 2024-05-08T08:53:18.135 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:18.135 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:18.135 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:18.135 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:18.135 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:18.135 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:18.135 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4772 2024-05-08T08:53:18.135 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.135 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:18.135 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.136 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:18.136 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586621+0000", 2024-05-08T08:53:18.136 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:18.136 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.136 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.136 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:18.136 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586621+0000", 2024-05-08T08:53:18.136 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:18.136 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.136 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.136 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:18.136 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586621+0000", 2024-05-08T08:53:18.136 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:53:18.136 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.136 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.137 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:18.137 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586623+0000", 2024-05-08T08:53:18.137 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9240000000000001e-06 2024-05-08T08:53:18.137 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.137 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.137 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:18.137 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586628+0000", 2024-05-08T08:53:18.137 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6410000000000001e-06 2024-05-08T08:53:18.137 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.137 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.137 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:18.137 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586633+0000", 2024-05-08T08:53:18.137 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.763e-06 2024-05-08T08:53:18.137 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.137 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.138 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:18.138 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.624077+0000", 2024-05-08T08:53:18.138 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.037444709999999999 2024-05-08T08:53:18.138 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.138 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.138 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:18.138 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.624164+0000", 2024-05-08T08:53:18.138 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6769999999999998e-05 2024-05-08T08:53:18.138 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.138 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.138 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:18.138 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.624271+0000", 2024-05-08T08:53:18.138 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010723 2024-05-08T08:53:18.138 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.138 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:18.139 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.139 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.139 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.139 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4774 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 2095104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:18.139 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586640+0000", 2024-05-08T08:53:18.139 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 46.487204859000002, 2024-05-08T08:53:18.139 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.037827178000000003, 2024-05-08T08:53:18.139 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:18.139 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:18.139 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:18.139 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:18.139 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:18.139 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:18.139 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4774 2024-05-08T08:53:18.140 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.140 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:18.140 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.140 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:18.140 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586640+0000", 2024-05-08T08:53:18.140 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:18.140 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.140 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.140 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:18.140 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586640+0000", 2024-05-08T08:53:18.140 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:18.140 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.140 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.140 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:18.140 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586640+0000", 2024-05-08T08:53:18.140 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5199999999999998e-07 2024-05-08T08:53:18.141 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.141 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.141 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:18.141 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586643+0000", 2024-05-08T08:53:18.141 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.232e-06 2024-05-08T08:53:18.141 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.141 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.141 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:18.141 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586649+0000", 2024-05-08T08:53:18.141 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9660000000000001e-06 2024-05-08T08:53:18.141 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.141 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.141 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:18.141 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586653+0000", 2024-05-08T08:53:18.141 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7350000000000004e-06 2024-05-08T08:53:18.142 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.142 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.142 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:18.142 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.624303+0000", 2024-05-08T08:53:18.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.037649647000000001 2024-05-08T08:53:18.142 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.142 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.142 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:18.142 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.624373+0000", 2024-05-08T08:53:18.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0415999999999998e-05 2024-05-08T08:53:18.142 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.142 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.142 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:18.142 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.624467+0000", 2024-05-08T08:53:18.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3830000000000001e-05 2024-05-08T08:53:18.143 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.143 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:18.143 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.143 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.143 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.143 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4621.0:9089 3.1f e33/18 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:head v 33'117, mlcod=31'115)", 2024-05-08T08:53:18.143 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:53:18.143 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 45.554226647, 2024-05-08T08:53:18.143 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.111244524, 2024-05-08T08:53:18.143 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:18.143 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:18.143 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:53:18.143 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:18.143 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.143 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:18.144 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:53:18.144 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:18.144 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.144 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.144 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:18.144 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:53:18.144 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:18.144 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.144 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.144 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:18.144 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:53:18.144 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5900000000000001e-07 2024-05-08T08:53:18.144 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.144 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.144 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:18.145 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539600+0000", 2024-05-08T08:53:18.145 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.019981364000000001 2024-05-08T08:53:18.145 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.145 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.145 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:18.145 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539624+0000", 2024-05-08T08:53:18.145 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4043e-05 2024-05-08T08:53:18.145 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.145 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.145 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:18.145 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539635+0000", 2024-05-08T08:53:18.145 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1511e-05 2024-05-08T08:53:18.145 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.145 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.145 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:18.145 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539728+0000", 2024-05-08T08:53:18.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2498999999999996e-05 2024-05-08T08:53:18.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.146 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.146 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:18.146 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539746+0000", 2024-05-08T08:53:18.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8289000000000001e-05 2024-05-08T08:53:18.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.146 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.146 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:53:18.146 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.630822+0000", 2024-05-08T08:53:18.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.091076283999999993 2024-05-08T08:53:18.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.146 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.146 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:18.146 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.630863+0000", 2024-05-08T08:53:18.147 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0374999999999999e-05 2024-05-08T08:53:18.147 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.147 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:18.147 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.147 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.147 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.147 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4795.0:230 3.1f e40/18 3:f8db1a56:::rbd_data.11402b26385.0000000000000000:head v 40'171, mlcod=39'168)", 2024-05-08T08:53:18.147 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:45.927773+0000", 2024-05-08T08:53:18.147 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 32.146071564000003, 2024-05-08T08:53:18.147 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045892405999999997, 2024-05-08T08:53:18.147 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:18.147 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:18.147 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:53:18.147 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:18.147 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.148 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:18.148 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.927773+0000", 2024-05-08T08:53:18.148 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:18.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.148 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.148 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:18.148 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.927773+0000", 2024-05-08T08:53:18.148 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:18.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.148 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.148 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:18.148 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.927774+0000", 2024-05-08T08:53:18.148 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0500000000000002e-07 2024-05-08T08:53:18.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.148 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.148 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:18.149 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.931581+0000", 2024-05-08T08:53:18.149 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0038071799999999999 2024-05-08T08:53:18.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.149 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:18.149 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.931599+0000", 2024-05-08T08:53:18.149 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7479e-05 2024-05-08T08:53:18.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.149 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:18.149 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.931624+0000", 2024-05-08T08:53:18.149 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5372999999999999e-05 2024-05-08T08:53:18.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.150 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:18.150 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.931698+0000", 2024-05-08T08:53:18.150 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4043999999999998e-05 2024-05-08T08:53:18.150 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.150 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.150 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:18.150 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.931725+0000", 2024-05-08T08:53:18.150 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6883e-05 2024-05-08T08:53:18.150 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.150 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.150 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:53:18.150 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.973623+0000", 2024-05-08T08:53:18.150 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.041898283000000001 2024-05-08T08:53:18.150 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.150 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.151 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:18.151 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.973666+0000", 2024-05-08T08:53:18.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2358999999999999e-05 2024-05-08T08:53:18.151 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.151 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:18.151 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.151 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.151 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.151 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4195.0:4 3.11 3:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-05-08T08:53:18.151 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:49.764654+0000", 2024-05-08T08:53:18.151 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 28.309191106, 2024-05-08T08:53:18.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.30714275499999999, 2024-05-08T08:53:18.151 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:18.151 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:18.152 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-05-08T08:53:18.152 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:18.152 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4195", 2024-05-08T08:53:18.152 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/59179521", 2024-05-08T08:53:18.152 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4 2024-05-08T08:53:18.152 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.152 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:18.152 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.152 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:18.152 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764654+0000", 2024-05-08T08:53:18.152 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:18.152 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.152 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.152 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:18.152 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764654+0000", 2024-05-08T08:53:18.153 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:18.153 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.153 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.153 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:18.153 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764655+0000", 2024-05-08T08:53:18.153 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.452e-06 2024-05-08T08:53:18.153 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.153 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.153 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:18.153 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764661+0000", 2024-05-08T08:53:18.153 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5369999999999997e-06 2024-05-08T08:53:18.153 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.153 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.153 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:18.153 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764671+0000", 2024-05-08T08:53:18.154 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.028e-05 2024-05-08T08:53:18.154 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.154 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.154 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:18.154 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764680+0000", 2024-05-08T08:53:18.154 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0629999999999995e-06 2024-05-08T08:53:18.154 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.154 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.154 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:18.154 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764753+0000", 2024-05-08T08:53:18.154 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2679e-05 2024-05-08T08:53:18.154 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.154 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.154 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-05-08T08:53:18.154 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764765+0000", 2024-05-08T08:53:18.155 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2415000000000001e-05 2024-05-08T08:53:18.155 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.155 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.155 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:18.155 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:50.071692+0000", 2024-05-08T08:53:18.155 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.306927324 2024-05-08T08:53:18.155 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.155 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.155 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:18.155 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:50.071796+0000", 2024-05-08T08:53:18.155 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104005 2024-05-08T08:53:18.155 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.155 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:18.155 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.155 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.155 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:18.156 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:53:18.157 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:53:18.158 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:18.160+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:18.210 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 5] out_osds: [4] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-05-08T08:53:18.210 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T08:53:18.210 INFO:tasks.thrashosds.thrasher:Adding osd 4 2024-05-08T08:53:18.210 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 4 2024-05-08T08:53:18.258 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:18.261+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:18.303 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:53:18.303 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:53:18.303 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:53:18.303 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:53:18.311 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:53:18.359 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:53:18.362+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:53:18.417 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:53:18.417 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:53:18.417 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:53:18.417 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:53:18.417 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:53:18.426 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:53:18.440 INFO:teuthology.orchestra.run.smithi059.stderr:noscrub is unset 2024-05-08T08:53:18.497 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:18.500+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:18.537 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:53:18.537 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:53:18.537 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:53:18.537 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:53:18.537 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.537 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22069 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 749568~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:18.537 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857624+0000", 2024-05-08T08:53:18.537 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 50.681580068999999, 2024-05-08T08:53:18.537 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046969958999999999, 2024-05-08T08:53:18.537 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:18.537 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:18.537 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:18.538 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:18.538 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:18.538 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:18.538 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22069 2024-05-08T08:53:18.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.538 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:18.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.538 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:18.538 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857624+0000", 2024-05-08T08:53:18.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:18.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.538 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:18.539 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857624+0000", 2024-05-08T08:53:18.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:18.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.539 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.539 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:18.539 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857625+0000", 2024-05-08T08:53:18.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.03e-07 2024-05-08T08:53:18.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.539 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.539 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:18.539 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857627+0000", 2024-05-08T08:53:18.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9369999999999998e-06 2024-05-08T08:53:18.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.539 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.540 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:18.540 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857633+0000", 2024-05-08T08:53:18.540 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1229999999999997e-06 2024-05-08T08:53:18.540 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.540 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.540 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:18.540 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857637+0000", 2024-05-08T08:53:18.540 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7820000000000001e-06 2024-05-08T08:53:18.540 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.540 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.540 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:18.540 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904457+0000", 2024-05-08T08:53:18.540 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046819715999999997 2024-05-08T08:53:18.540 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.540 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.540 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:18.541 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904524+0000", 2024-05-08T08:53:18.541 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6371000000000005e-05 2024-05-08T08:53:18.541 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.541 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.541 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:18.541 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904594+0000", 2024-05-08T08:53:18.541 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0627e-05 2024-05-08T08:53:18.541 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.541 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:18.541 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.541 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.541 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.541 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22071 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 765952~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:18.541 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857643+0000", 2024-05-08T08:53:18.541 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 50.681560986000001, 2024-05-08T08:53:18.542 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047106528000000002, 2024-05-08T08:53:18.542 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:18.542 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:18.542 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:18.542 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:18.542 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:18.542 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:18.542 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22071 2024-05-08T08:53:18.542 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.542 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:18.542 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.542 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:18.542 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857643+0000", 2024-05-08T08:53:18.543 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:18.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.543 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:18.543 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857643+0000", 2024-05-08T08:53:18.543 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:18.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.543 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:18.543 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857644+0000", 2024-05-08T08:53:18.543 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2000000000000001e-07 2024-05-08T08:53:18.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.544 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:18.544 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857646+0000", 2024-05-08T08:53:18.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0059999999999999e-06 2024-05-08T08:53:18.544 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.544 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.544 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:18.544 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857651+0000", 2024-05-08T08:53:18.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.84e-06 2024-05-08T08:53:18.544 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.544 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.544 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:18.544 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857657+0000", 2024-05-08T08:53:18.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5620000000000003e-06 2024-05-08T08:53:18.544 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.544 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.545 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:18.545 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904620+0000", 2024-05-08T08:53:18.545 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046962661000000003 2024-05-08T08:53:18.545 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.545 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.545 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:18.545 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904678+0000", 2024-05-08T08:53:18.545 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8656999999999998e-05 2024-05-08T08:53:18.545 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.545 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.545 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:18.545 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904750+0000", 2024-05-08T08:53:18.545 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1482000000000004e-05 2024-05-08T08:53:18.545 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.546 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:18.546 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.546 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.546 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.546 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22073 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 782336~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:18.546 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:53:18.546 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 50.681536977999997, 2024-05-08T08:53:18.546 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047244234000000003, 2024-05-08T08:53:18.546 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:18.546 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:18.546 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:18.546 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:18.546 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:18.546 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:18.546 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22073 2024-05-08T08:53:18.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.547 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:18.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.547 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:18.547 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:53:18.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:18.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.547 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:18.547 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:53:18.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:18.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.547 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:18.547 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857668+0000", 2024-05-08T08:53:18.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7800000000000002e-07 2024-05-08T08:53:18.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:18.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857670+0000", 2024-05-08T08:53:18.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0820000000000001e-06 2024-05-08T08:53:18.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:18.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857678+0000", 2024-05-08T08:53:18.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7589999999999997e-06 2024-05-08T08:53:18.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:18.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857691+0000", 2024-05-08T08:53:18.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3672999999999999e-05 2024-05-08T08:53:18.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.549 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.549 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:18.549 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904780+0000", 2024-05-08T08:53:18.549 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047088616999999999 2024-05-08T08:53:18.549 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.549 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.549 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:18.549 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904840+0000", 2024-05-08T08:53:18.549 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0356999999999998e-05 2024-05-08T08:53:18.549 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.549 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.549 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:18.549 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904912+0000", 2024-05-08T08:53:18.549 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1367999999999998e-05 2024-05-08T08:53:18.549 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.550 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:18.550 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.550 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.550 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.550 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22075 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 798720~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:18.550 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:53:18.550 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 50.681506581000001, 2024-05-08T08:53:18.550 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047384643999999997, 2024-05-08T08:53:18.550 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:18.550 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:18.550 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:18.550 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:18.550 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:18.551 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:18.551 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22075 2024-05-08T08:53:18.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.551 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:18.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.551 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:18.551 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:53:18.551 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:18.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.551 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:18.551 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:53:18.551 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:18.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.552 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:18.552 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:53:18.552 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7399999999999999e-07 2024-05-08T08:53:18.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.552 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.552 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:18.552 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857705+0000", 2024-05-08T08:53:18.552 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1980000000000004e-06 2024-05-08T08:53:18.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.552 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.552 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:18.552 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857713+0000", 2024-05-08T08:53:18.552 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7039999999999994e-06 2024-05-08T08:53:18.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.552 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.552 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:18.553 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857718+0000", 2024-05-08T08:53:18.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.865e-06 2024-05-08T08:53:18.553 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.553 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.553 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:18.553 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904941+0000", 2024-05-08T08:53:18.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047223198000000001 2024-05-08T08:53:18.553 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.553 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.553 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:18.553 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904999+0000", 2024-05-08T08:53:18.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8254999999999997e-05 2024-05-08T08:53:18.553 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.553 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.554 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:18.554 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905082+0000", 2024-05-08T08:53:18.554 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3049999999999999e-05 2024-05-08T08:53:18.554 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.554 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:18.554 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.554 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.554 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.554 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22077 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 815104~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:18.554 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:53:18.554 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 50.68146857, 2024-05-08T08:53:18.554 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047519448999999998, 2024-05-08T08:53:18.554 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:18.554 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:18.555 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:18.555 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:18.555 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:18.555 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:18.555 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22077 2024-05-08T08:53:18.555 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.555 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:18.555 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.555 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:18.555 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:53:18.555 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:18.555 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.555 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.555 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:18.555 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:53:18.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:18.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.556 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.556 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:18.556 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:53:18.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.99e-07 2024-05-08T08:53:18.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.556 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.556 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:18.556 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857739+0000", 2024-05-08T08:53:18.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3599999999999999e-06 2024-05-08T08:53:18.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.556 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.556 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:18.556 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857745+0000", 2024-05-08T08:53:18.557 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5139999999999998e-06 2024-05-08T08:53:18.557 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.557 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.557 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:18.557 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857750+0000", 2024-05-08T08:53:18.557 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8550000000000001e-06 2024-05-08T08:53:18.557 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.557 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.557 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:18.557 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905110+0000", 2024-05-08T08:53:18.557 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047360126000000002 2024-05-08T08:53:18.557 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.557 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.557 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:18.557 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905175+0000", 2024-05-08T08:53:18.558 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4689000000000004e-05 2024-05-08T08:53:18.558 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.558 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.558 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:18.558 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905255+0000", 2024-05-08T08:53:18.558 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0306e-05 2024-05-08T08:53:18.558 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.558 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:18.558 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.558 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.558 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.558 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22079 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 831488~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:18.558 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:53:18.558 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 50.681448021999998, 2024-05-08T08:53:18.559 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.04767499, 2024-05-08T08:53:18.559 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:18.559 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:18.559 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:18.559 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:18.559 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:18.559 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:18.559 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22079 2024-05-08T08:53:18.559 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.559 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:18.559 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.559 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:18.559 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:53:18.559 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:18.559 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.560 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.560 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:18.560 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:53:18.560 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:18.560 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.560 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.560 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:18.560 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857757+0000", 2024-05-08T08:53:18.560 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8299999999999997e-07 2024-05-08T08:53:18.560 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.560 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.560 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:18.560 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857759+0000", 2024-05-08T08:53:18.560 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1390000000000002e-06 2024-05-08T08:53:18.560 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.560 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.561 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:18.561 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857765+0000", 2024-05-08T08:53:18.561 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4609999999999996e-06 2024-05-08T08:53:18.561 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.561 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.561 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:18.561 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857770+0000", 2024-05-08T08:53:18.561 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6689999999999997e-06 2024-05-08T08:53:18.561 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.561 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.561 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:18.561 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905281+0000", 2024-05-08T08:53:18.561 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047510668999999998 2024-05-08T08:53:18.561 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.561 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.562 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:18.562 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905344+0000", 2024-05-08T08:53:18.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2797000000000002e-05 2024-05-08T08:53:18.562 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.562 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.562 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:18.562 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905431+0000", 2024-05-08T08:53:18.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7771999999999999e-05 2024-05-08T08:53:18.562 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.562 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:18.562 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.562 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.562 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.562 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22082 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 847872~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:18.563 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:53:18.563 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 50.681428330999999, 2024-05-08T08:53:18.563 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047831934999999999, 2024-05-08T08:53:18.563 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:18.563 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:18.563 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:18.563 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:18.563 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:18.563 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:18.563 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22082 2024-05-08T08:53:18.563 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.563 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:18.563 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.563 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:18.563 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:53:18.564 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:18.564 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.564 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.564 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:18.564 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:53:18.564 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:18.564 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.564 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.564 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:18.564 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:53:18.564 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:53:18.564 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.564 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.564 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:18.564 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857778+0000", 2024-05-08T08:53:18.565 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0789999999999999e-06 2024-05-08T08:53:18.565 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.565 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.565 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:18.565 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857784+0000", 2024-05-08T08:53:18.565 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0480000000000004e-06 2024-05-08T08:53:18.565 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.565 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.565 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:18.565 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857790+0000", 2024-05-08T08:53:18.565 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2719999999999997e-06 2024-05-08T08:53:18.565 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.565 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.565 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:18.565 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905458+0000", 2024-05-08T08:53:18.566 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047668470999999997 2024-05-08T08:53:18.566 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.566 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.566 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:18.566 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905521+0000", 2024-05-08T08:53:18.566 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2952000000000003e-05 2024-05-08T08:53:18.566 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.566 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.566 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:18.566 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905608+0000", 2024-05-08T08:53:18.566 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6759000000000001e-05 2024-05-08T08:53:18.566 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.566 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:18.566 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.567 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.567 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.567 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22084 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 864256~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:18.567 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:53:18.567 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 50.681404381, 2024-05-08T08:53:18.567 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047977748000000001, 2024-05-08T08:53:18.567 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:18.567 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:18.567 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:18.567 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:18.567 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:18.567 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:18.567 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22084 2024-05-08T08:53:18.567 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.568 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:18.568 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.568 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:18.568 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:53:18.568 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:18.568 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.568 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.568 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:18.568 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:53:18.568 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:18.568 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.568 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.568 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:18.568 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:53:18.568 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:53:18.568 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.569 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.569 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:18.569 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857802+0000", 2024-05-08T08:53:18.569 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0779999999999998e-06 2024-05-08T08:53:18.569 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.569 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.569 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:18.569 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857812+0000", 2024-05-08T08:53:18.569 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7939999999999997e-06 2024-05-08T08:53:18.569 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.569 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.569 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:18.569 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857817+0000", 2024-05-08T08:53:18.569 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8899999999999998e-06 2024-05-08T08:53:18.569 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.570 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.570 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:18.570 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905636+0000", 2024-05-08T08:53:18.570 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047819252 2024-05-08T08:53:18.570 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.570 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.570 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:18.570 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905700+0000", 2024-05-08T08:53:18.570 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4016999999999994e-05 2024-05-08T08:53:18.570 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.570 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.570 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:18.570 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905778+0000", 2024-05-08T08:53:18.570 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7306999999999997e-05 2024-05-08T08:53:18.570 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.571 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:18.571 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.571 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.571 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22086 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 880640~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:18.571 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:53:18.571 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 50.681381031000001, 2024-05-08T08:53:18.571 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048119149, 2024-05-08T08:53:18.571 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:18.571 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:18.571 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:18.571 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:18.571 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:18.571 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:18.571 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22086 2024-05-08T08:53:18.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.572 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:18.572 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.572 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:18.572 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:53:18.572 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:18.572 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.572 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.572 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:18.572 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:53:18.572 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:18.572 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.572 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.572 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:18.572 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857824+0000", 2024-05-08T08:53:18.573 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4200000000000002e-07 2024-05-08T08:53:18.573 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.573 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.573 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:18.573 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857826+0000", 2024-05-08T08:53:18.573 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0590000000000001e-06 2024-05-08T08:53:18.573 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.573 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.573 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:18.573 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857832+0000", 2024-05-08T08:53:18.573 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8029999999999999e-06 2024-05-08T08:53:18.573 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.573 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.573 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:18.573 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857837+0000", 2024-05-08T08:53:18.574 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7929999999999997e-06 2024-05-08T08:53:18.574 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.574 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.574 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:18.574 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905803+0000", 2024-05-08T08:53:18.574 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047965761000000003 2024-05-08T08:53:18.574 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.574 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.574 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:18.574 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905862+0000", 2024-05-08T08:53:18.574 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9203000000000002e-05 2024-05-08T08:53:18.574 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.574 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.574 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:18.575 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905942+0000", 2024-05-08T08:53:18.575 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0187999999999999e-05 2024-05-08T08:53:18.575 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.575 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:18.575 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.575 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.575 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.575 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22088 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 897024~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:18.575 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:53:18.575 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 50.681360978999997, 2024-05-08T08:53:18.575 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048256049000000002, 2024-05-08T08:53:18.575 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:18.575 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:18.575 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:18.575 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:18.575 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:18.575 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:18.576 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22088 2024-05-08T08:53:18.576 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.576 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:18.576 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.576 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:18.576 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:53:18.576 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:18.576 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.576 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.576 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:18.576 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:53:18.576 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:18.576 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.576 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.576 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:18.577 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857844+0000", 2024-05-08T08:53:18.577 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.46e-07 2024-05-08T08:53:18.577 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.577 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.577 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:18.577 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857846+0000", 2024-05-08T08:53:18.577 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0899999999999999e-06 2024-05-08T08:53:18.577 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.577 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.577 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:18.577 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857855+0000", 2024-05-08T08:53:18.577 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8219999999999996e-06 2024-05-08T08:53:18.577 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.577 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.577 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:18.578 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857860+0000", 2024-05-08T08:53:18.578 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2989999999999996e-06 2024-05-08T08:53:18.578 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.578 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.578 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:18.578 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905966+0000", 2024-05-08T08:53:18.578 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048106542000000002 2024-05-08T08:53:18.578 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.578 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.578 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:18.578 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906024+0000", 2024-05-08T08:53:18.578 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7958000000000003e-05 2024-05-08T08:53:18.578 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.578 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.579 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:18.579 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906099+0000", 2024-05-08T08:53:18.579 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4992000000000003e-05 2024-05-08T08:53:18.579 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.579 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:18.579 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.579 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.579 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.579 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22091 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 913408~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:18.579 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:53:18.579 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 50.681337219, 2024-05-08T08:53:18.579 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048395290000000001, 2024-05-08T08:53:18.579 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:18.579 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:18.580 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:18.580 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:18.580 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:18.580 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:18.580 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22091 2024-05-08T08:53:18.580 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.580 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:18.580 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.580 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:18.580 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:53:18.580 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:18.580 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.580 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.580 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:18.581 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:53:18.581 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:18.581 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.581 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.581 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:18.581 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:53:18.581 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4900000000000001e-07 2024-05-08T08:53:18.581 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.581 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.581 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:18.581 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857870+0000", 2024-05-08T08:53:18.581 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2510000000000001e-06 2024-05-08T08:53:18.581 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.581 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.581 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:18.581 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857877+0000", 2024-05-08T08:53:18.582 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0809999999999996e-06 2024-05-08T08:53:18.582 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.582 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.582 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:18.582 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857881+0000", 2024-05-08T08:53:18.582 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.724e-06 2024-05-08T08:53:18.582 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.582 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.582 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:18.582 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906127+0000", 2024-05-08T08:53:18.582 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048245345000000002 2024-05-08T08:53:18.582 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.582 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.582 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:18.583 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906190+0000", 2024-05-08T08:53:18.583 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2762000000000007e-05 2024-05-08T08:53:18.583 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.583 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.583 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:18.583 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906262+0000", 2024-05-08T08:53:18.583 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2778e-05 2024-05-08T08:53:18.583 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.583 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:18.583 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.583 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.583 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.583 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22093 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 929792~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:18.583 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:53:18.583 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 50.681314854999997, 2024-05-08T08:53:18.584 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048532875000000003, 2024-05-08T08:53:18.584 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:18.584 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:18.584 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:18.584 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:18.584 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:18.584 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:18.584 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22093 2024-05-08T08:53:18.584 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.584 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:18.584 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.584 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:18.584 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:53:18.584 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:18.585 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.585 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.585 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:18.585 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:53:18.585 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:18.585 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.585 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.585 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:18.585 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857890+0000", 2024-05-08T08:53:18.585 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-07 2024-05-08T08:53:18.585 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.585 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.585 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:18.585 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857899+0000", 2024-05-08T08:53:18.585 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6889999999999999e-06 2024-05-08T08:53:18.585 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.586 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.586 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:18.586 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857906+0000", 2024-05-08T08:53:18.586 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6490000000000002e-06 2024-05-08T08:53:18.586 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.586 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.586 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:18.586 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857917+0000", 2024-05-08T08:53:18.586 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0764999999999999e-05 2024-05-08T08:53:18.586 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.586 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.586 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:18.586 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906287+0000", 2024-05-08T08:53:18.587 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048370442 2024-05-08T08:53:18.587 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.587 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.587 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:18.587 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906349+0000", 2024-05-08T08:53:18.587 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1537000000000004e-05 2024-05-08T08:53:18.587 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.587 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.587 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:18.587 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906422+0000", 2024-05-08T08:53:18.587 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3431999999999998e-05 2024-05-08T08:53:18.587 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.587 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:18.587 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.587 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.588 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.588 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22095 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 946176~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:18.588 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:53:18.588 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 50.681280151000003, 2024-05-08T08:53:18.588 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048657593999999998, 2024-05-08T08:53:18.588 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:18.588 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:18.588 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:18.588 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:18.588 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:18.588 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:18.588 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22095 2024-05-08T08:53:18.588 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.588 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:18.588 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.588 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:18.589 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:53:18.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:18.589 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.589 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.589 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:18.589 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:53:18.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:18.589 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.589 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.589 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:18.589 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857925+0000", 2024-05-08T08:53:18.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3099999999999998e-07 2024-05-08T08:53:18.589 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.589 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.589 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:18.590 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857927+0000", 2024-05-08T08:53:18.590 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.34e-06 2024-05-08T08:53:18.590 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.590 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.590 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:18.590 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857938+0000", 2024-05-08T08:53:18.590 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1426e-05 2024-05-08T08:53:18.590 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.590 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.590 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:18.590 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857943+0000", 2024-05-08T08:53:18.590 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5140000000000003e-06 2024-05-08T08:53:18.590 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.590 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.591 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:18.591 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906448+0000", 2024-05-08T08:53:18.591 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048504641000000001 2024-05-08T08:53:18.591 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.591 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.591 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:18.591 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906510+0000", 2024-05-08T08:53:18.591 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2663000000000006e-05 2024-05-08T08:53:18.591 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.591 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.591 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:18.591 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906582+0000", 2024-05-08T08:53:18.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1478999999999998e-05 2024-05-08T08:53:18.592 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.592 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:18.592 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.592 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.592 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.592 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22097 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 962560~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:18.592 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:53:18.592 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 50.681252462000003, 2024-05-08T08:53:18.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048791353000000003, 2024-05-08T08:53:18.592 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:18.592 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:18.592 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:18.593 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:18.593 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:18.593 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:18.593 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22097 2024-05-08T08:53:18.593 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.593 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:18.593 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.593 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:18.593 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:53:18.593 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:18.593 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.593 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.593 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:18.593 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:53:18.593 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:18.594 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.594 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.594 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:18.594 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:53:18.594 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9200000000000002e-07 2024-05-08T08:53:18.594 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.594 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.594 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:18.594 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857954+0000", 2024-05-08T08:53:18.594 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1900000000000002e-06 2024-05-08T08:53:18.594 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.594 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.594 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:18.594 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857961+0000", 2024-05-08T08:53:18.595 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4989999999999999e-06 2024-05-08T08:53:18.595 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.595 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.595 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:18.595 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857971+0000", 2024-05-08T08:53:18.595 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0485999999999999e-05 2024-05-08T08:53:18.595 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.595 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.595 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:18.595 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906607+0000", 2024-05-08T08:53:18.595 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048635109000000003 2024-05-08T08:53:18.595 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.595 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.595 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:18.596 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906667+0000", 2024-05-08T08:53:18.596 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0439000000000002e-05 2024-05-08T08:53:18.596 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.596 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:18.596 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906743+0000", 2024-05-08T08:53:18.596 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6237999999999998e-05 2024-05-08T08:53:18.596 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.596 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:18.596 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.596 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.596 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22099 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 978944~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:18.596 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:53:18.596 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 50.681226301999999, 2024-05-08T08:53:18.596 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048932055000000002, 2024-05-08T08:53:18.597 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:18.597 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:18.597 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:18.597 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:18.597 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:18.597 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:18.597 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22099 2024-05-08T08:53:18.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.597 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:18.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.597 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:18.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:53:18.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:18.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:18.598 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:53:18.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:18.598 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:18.598 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:53:18.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.27e-07 2024-05-08T08:53:18.598 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:18.598 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857987+0000", 2024-05-08T08:53:18.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5890000000000003e-06 2024-05-08T08:53:18.599 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.599 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.599 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:18.599 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857994+0000", 2024-05-08T08:53:18.599 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6619999999999999e-06 2024-05-08T08:53:18.599 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.599 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.599 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:18.599 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857998+0000", 2024-05-08T08:53:18.599 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7110000000000003e-06 2024-05-08T08:53:18.599 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.599 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.599 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:18.599 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906772+0000", 2024-05-08T08:53:18.599 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048773415000000001 2024-05-08T08:53:18.599 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.600 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.600 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:18.600 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906834+0000", 2024-05-08T08:53:18.600 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1752000000000001e-05 2024-05-08T08:53:18.600 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.600 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.600 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:18.600 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906910+0000", 2024-05-08T08:53:18.600 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6499000000000001e-05 2024-05-08T08:53:18.600 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.600 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:18.600 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.600 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.600 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.601 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22101 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 995328~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:18.601 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:53:18.601 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 50.681199171000003, 2024-05-08T08:53:18.601 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049087364000000001, 2024-05-08T08:53:18.601 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:18.601 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:18.601 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:18.601 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:18.601 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:18.601 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:18.601 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22101 2024-05-08T08:53:18.601 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.601 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:18.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:18.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:53:18.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:18.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:18.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:53:18.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:18.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:18.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858006+0000", 2024-05-08T08:53:18.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1399999999999997e-07 2024-05-08T08:53:18.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.603 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:18.603 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858008+0000", 2024-05-08T08:53:18.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2060000000000001e-06 2024-05-08T08:53:18.603 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.603 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:18.603 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858017+0000", 2024-05-08T08:53:18.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1579999999999994e-06 2024-05-08T08:53:18.603 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.603 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:18.603 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858022+0000", 2024-05-08T08:53:18.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4199999999999998e-06 2024-05-08T08:53:18.604 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.604 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.604 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:18.604 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906940+0000", 2024-05-08T08:53:18.604 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048917627999999998 2024-05-08T08:53:18.604 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.604 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.604 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:18.604 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907002+0000", 2024-05-08T08:53:18.604 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2399999999999999e-05 2024-05-08T08:53:18.604 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.604 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.604 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:18.604 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907092+0000", 2024-05-08T08:53:18.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0137999999999997e-05 2024-05-08T08:53:18.605 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.605 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:18.605 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.605 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.605 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.605 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22103 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1011712~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:18.605 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:53:18.605 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 50.681174386000002, 2024-05-08T08:53:18.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049228542, 2024-05-08T08:53:18.605 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:18.605 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:18.605 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:18.605 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:18.606 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:18.606 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:18.606 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22103 2024-05-08T08:53:18.606 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.606 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:18.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.606 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:18.606 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:53:18.606 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:18.606 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.606 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:18.606 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:53:18.606 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:18.606 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.607 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:18.607 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:53:18.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4299999999999999e-07 2024-05-08T08:53:18.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.607 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:18.607 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858032+0000", 2024-05-08T08:53:18.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0590000000000001e-06 2024-05-08T08:53:18.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.607 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:18.607 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858037+0000", 2024-05-08T08:53:18.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.977e-06 2024-05-08T08:53:18.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.608 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:18.608 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858042+0000", 2024-05-08T08:53:18.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7650000000000001e-06 2024-05-08T08:53:18.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.608 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:18.608 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907118+0000", 2024-05-08T08:53:18.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049076147000000001 2024-05-08T08:53:18.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.608 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:18.608 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907180+0000", 2024-05-08T08:53:18.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1971999999999996e-05 2024-05-08T08:53:18.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.609 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:18.609 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907258+0000", 2024-05-08T08:53:18.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8279e-05 2024-05-08T08:53:18.609 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.609 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:18.609 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.609 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22105 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1028096~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:18.609 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:53:18.609 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 50.681155142999998, 2024-05-08T08:53:18.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049394609999999999, 2024-05-08T08:53:18.609 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:18.610 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:18.610 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:18.610 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:18.610 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:18.610 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:18.610 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22105 2024-05-08T08:53:18.610 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.610 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:18.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.610 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:18.610 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:53:18.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:18.610 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.611 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:18.611 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:53:18.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:18.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.611 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:18.611 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858050+0000", 2024-05-08T08:53:18.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5999999999999999e-07 2024-05-08T08:53:18.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.611 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:18.611 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858051+0000", 2024-05-08T08:53:18.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8870000000000001e-06 2024-05-08T08:53:18.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.612 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:18.612 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858057+0000", 2024-05-08T08:53:18.612 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8019999999999997e-06 2024-05-08T08:53:18.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.612 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:18.612 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858062+0000", 2024-05-08T08:53:18.612 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.561e-06 2024-05-08T08:53:18.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.612 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:18.612 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907289+0000", 2024-05-08T08:53:18.612 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049226957000000002 2024-05-08T08:53:18.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.613 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:18.613 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907356+0000", 2024-05-08T08:53:18.613 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6915000000000006e-05 2024-05-08T08:53:18.613 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.613 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.613 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:18.613 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907444+0000", 2024-05-08T08:53:18.613 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8127999999999994e-05 2024-05-08T08:53:18.613 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.613 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:18.613 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.613 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.613 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.613 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22107 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1044480~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:18.614 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:53:18.614 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 50.681136803000001, 2024-05-08T08:53:18.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049799942999999999, 2024-05-08T08:53:18.614 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:18.614 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:18.614 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:18.614 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:18.614 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:18.614 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:18.614 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22107 2024-05-08T08:53:18.614 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.614 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:18.614 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.614 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:18.614 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:53:18.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:18.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:18.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:53:18.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:18.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:18.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:53:18.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4799999999999999e-07 2024-05-08T08:53:18.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:18.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858070+0000", 2024-05-08T08:53:18.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.012e-06 2024-05-08T08:53:18.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:18.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858077+0000", 2024-05-08T08:53:18.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6290000000000001e-06 2024-05-08T08:53:18.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:18.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858083+0000", 2024-05-08T08:53:18.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8880000000000002e-06 2024-05-08T08:53:18.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:18.617 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907671+0000", 2024-05-08T08:53:18.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049587191000000003 2024-05-08T08:53:18.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.617 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.617 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:18.617 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907763+0000", 2024-05-08T08:53:18.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2231000000000005e-05 2024-05-08T08:53:18.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.617 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.617 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:18.617 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907867+0000", 2024-05-08T08:53:18.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104644 2024-05-08T08:53:18.617 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.617 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:18.617 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.618 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.618 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.618 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22294 3.1c 3:3aad0389:::rbd_data.11406886b07b.0000000000000080:head [stat,delete] snapc 7=[7] RETRY=6 ondisk+retry+write+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:18.618 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:53:18.618 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 50.680542955, 2024-05-08T08:53:18.618 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051152191, 2024-05-08T08:53:18.618 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:18.618 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:18.618 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:18.618 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:18.618 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:18.618 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:18.618 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22294 2024-05-08T08:53:18.618 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.618 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:18.619 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.619 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:18.619 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:53:18.619 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:18.619 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.619 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.619 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:18.619 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:53:18.619 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:18.619 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.619 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.619 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:18.619 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858662+0000", 2024-05-08T08:53:18.619 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4799999999999998e-07 2024-05-08T08:53:18.619 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.620 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:18.620 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858664+0000", 2024-05-08T08:53:18.620 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1229999999999998e-06 2024-05-08T08:53:18.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.620 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:18.620 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858670+0000", 2024-05-08T08:53:18.620 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3099999999999997e-06 2024-05-08T08:53:18.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.620 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:18.620 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858676+0000", 2024-05-08T08:53:18.620 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9750000000000004e-06 2024-05-08T08:53:18.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:18.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907901+0000", 2024-05-08T08:53:18.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049224520000000001 2024-05-08T08:53:18.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:18.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.908085+0000", 2024-05-08T08:53:18.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00018377500000000001 2024-05-08T08:53:18.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:18.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:18.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:18.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.909814+0000", 2024-05-08T08:53:18.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00172894 2024-05-08T08:53:18.622 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.622 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:18.622 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.622 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:18.622 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:18.622 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:53:18.623 INFO:tasks.ceph.mon.a.smithi059.stderr:2024-05-08T08:53:18.540+0000 7fc9cc811640 -1 mon.a@0(leader).osd e68 definitely_dead 0 2024-05-08T08:53:18.624 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:53:18.624 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:18.626+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:18.725 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:18.727+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:18.771 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:53:18.771 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:53:18.771 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:53:18.771 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:53:18.780 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:53:18.825 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:18.827+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:18.871 INFO:teuthology.orchestra.run.smithi196.stdout:checking clone #17, image image_client.0-clone17 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent18 2024-05-08T08:53:18.889 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:53:18.889 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:53:18.889 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:53:18.889 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:53:18.889 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:53:18.897 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:53:18.939 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:18.942+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:19.026 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:19.029+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:19.071 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:53:19.071 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:53:19.071 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:53:19.071 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:53:19.071 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.071 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6690 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1093632~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:19.071 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:53:19.071 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 47.357952513999997, 2024-05-08T08:53:19.072 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052462754, 2024-05-08T08:53:19.072 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:19.072 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:19.072 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:19.072 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:19.072 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:19.072 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:19.072 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6690 2024-05-08T08:53:19.072 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.072 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:19.072 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.072 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:19.072 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:53:19.073 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:19.073 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.073 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.073 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:19.073 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:53:19.073 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:19.073 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.073 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.073 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:19.073 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715346+0000", 2024-05-08T08:53:19.073 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.84e-07 2024-05-08T08:53:19.073 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.073 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.073 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:19.074 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715348+0000", 2024-05-08T08:53:19.074 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.114e-06 2024-05-08T08:53:19.074 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.074 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.074 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:19.074 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715353+0000", 2024-05-08T08:53:19.074 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5720000000000002e-06 2024-05-08T08:53:19.074 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.074 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.074 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:19.074 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715358+0000", 2024-05-08T08:53:19.074 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7140000000000001e-06 2024-05-08T08:53:19.074 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.074 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.075 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:19.075 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767638+0000", 2024-05-08T08:53:19.075 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052279562000000002 2024-05-08T08:53:19.075 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.075 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.075 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:19.075 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767713+0000", 2024-05-08T08:53:19.075 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5652e-05 2024-05-08T08:53:19.075 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.075 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.075 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:19.075 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767808+0000", 2024-05-08T08:53:19.075 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4755999999999999e-05 2024-05-08T08:53:19.075 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:19.075 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:19.076 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:19.076 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.076 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.076 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6692 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1110016~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:19.076 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:53:19.076 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 47.357933424000002, 2024-05-08T08:53:19.076 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052666495000000001, 2024-05-08T08:53:19.076 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:19.076 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:19.076 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:19.076 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:19.076 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:19.076 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:19.076 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6692 2024-05-08T08:53:19.077 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.077 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:19.077 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.077 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:19.077 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:53:19.077 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:19.077 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.077 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.077 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:19.077 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:53:19.077 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:19.077 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.077 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.077 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:19.078 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:53:19.078 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8599999999999999e-07 2024-05-08T08:53:19.078 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.078 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.078 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:19.078 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715372+0000", 2024-05-08T08:53:19.078 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0990000000000001e-06 2024-05-08T08:53:19.078 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.078 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.078 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:19.078 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715378+0000", 2024-05-08T08:53:19.078 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0680000000000002e-06 2024-05-08T08:53:19.078 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.078 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.078 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:19.079 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715383+0000", 2024-05-08T08:53:19.079 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8740000000000003e-06 2024-05-08T08:53:19.079 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.079 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.079 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:19.079 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767848+0000", 2024-05-08T08:53:19.079 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052464668999999998 2024-05-08T08:53:19.079 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.079 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.079 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:19.079 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767927+0000", 2024-05-08T08:53:19.079 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9228999999999996e-05 2024-05-08T08:53:19.079 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.079 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.079 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:19.079 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768031+0000", 2024-05-08T08:53:19.080 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010417 2024-05-08T08:53:19.080 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:19.080 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:19.080 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:19.080 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.080 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.080 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6694 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1126400~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:19.080 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:53:19.080 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 47.357908526999999, 2024-05-08T08:53:19.080 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052848829, 2024-05-08T08:53:19.080 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:19.080 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:19.080 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:19.080 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:19.080 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:19.081 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:19.081 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6694 2024-05-08T08:53:19.081 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.081 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:19.081 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.081 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:19.081 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:53:19.081 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:19.081 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.081 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.081 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:19.081 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:53:19.081 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:19.081 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.082 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.082 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:19.082 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715390+0000", 2024-05-08T08:53:19.082 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9900000000000001e-07 2024-05-08T08:53:19.082 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.082 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.082 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:19.082 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715392+0000", 2024-05-08T08:53:19.082 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8279999999999999e-06 2024-05-08T08:53:19.082 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.082 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.082 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:19.082 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715398+0000", 2024-05-08T08:53:19.082 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7619999999999998e-06 2024-05-08T08:53:19.082 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.083 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.083 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:19.083 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715403+0000", 2024-05-08T08:53:19.083 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7790000000000003e-06 2024-05-08T08:53:19.083 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.083 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.083 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:19.083 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768067+0000", 2024-05-08T08:53:19.083 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052663785999999997 2024-05-08T08:53:19.083 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.084 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.084 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:19.084 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768142+0000", 2024-05-08T08:53:19.084 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5018999999999994e-05 2024-05-08T08:53:19.084 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.084 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.084 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:19.084 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768238+0000", 2024-05-08T08:53:19.084 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6255999999999995e-05 2024-05-08T08:53:19.084 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:19.084 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:19.084 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:19.084 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.085 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.085 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6696 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1142784~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:19.085 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:53:19.085 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 47.357889018999998, 2024-05-08T08:53:19.085 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.053051057999999998, 2024-05-08T08:53:19.085 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:19.085 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:19.085 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:19.085 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:19.085 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:19.085 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:19.085 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6696 2024-05-08T08:53:19.085 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.085 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:19.086 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.086 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:19.086 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:53:19.086 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:19.086 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.086 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.086 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:19.086 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:53:19.086 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:19.086 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.086 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.086 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:19.086 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:53:19.086 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-05-08T08:53:19.087 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.087 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.087 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:19.087 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715411+0000", 2024-05-08T08:53:19.087 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9630000000000001e-06 2024-05-08T08:53:19.087 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.087 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.087 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:19.087 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715417+0000", 2024-05-08T08:53:19.087 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7250000000000002e-06 2024-05-08T08:53:19.087 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.087 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.087 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:19.087 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715421+0000", 2024-05-08T08:53:19.087 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4089999999999996e-06 2024-05-08T08:53:19.087 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.088 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.088 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:19.088 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768277+0000", 2024-05-08T08:53:19.088 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052855293999999997 2024-05-08T08:53:19.088 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.088 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.088 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:19.088 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768358+0000", 2024-05-08T08:53:19.088 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1720999999999999e-05 2024-05-08T08:53:19.088 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.088 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.088 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:19.088 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768460+0000", 2024-05-08T08:53:19.088 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010154600000000001 2024-05-08T08:53:19.088 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:19.088 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:19.088 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:19.089 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.089 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.089 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6698 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1159168~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:19.089 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:53:19.089 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 47.357870151999997, 2024-05-08T08:53:19.089 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.053245726, 2024-05-08T08:53:19.089 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:19.089 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:19.089 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:19.089 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:19.089 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:19.089 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:19.089 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6698 2024-05-08T08:53:19.089 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.089 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:19.090 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.090 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:19.090 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:53:19.090 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:19.090 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.090 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.090 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:19.090 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:53:19.090 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:19.090 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.090 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.090 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:19.090 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716936+0000", 2024-05-08T08:53:19.090 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001508665 2024-05-08T08:53:19.091 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.091 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.091 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:19.091 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716941+0000", 2024-05-08T08:53:19.091 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1910000000000002e-06 2024-05-08T08:53:19.091 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.091 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.091 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:19.091 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716950+0000", 2024-05-08T08:53:19.091 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1900000000000001e-06 2024-05-08T08:53:19.091 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.091 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.091 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:19.091 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716958+0000", 2024-05-08T08:53:19.092 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8019999999999992e-06 2024-05-08T08:53:19.092 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.092 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.092 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:19.092 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768499+0000", 2024-05-08T08:53:19.092 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051540926000000001 2024-05-08T08:53:19.092 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.092 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.092 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:19.092 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768574+0000", 2024-05-08T08:53:19.092 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4988999999999996e-05 2024-05-08T08:53:19.092 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.092 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.092 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:19.092 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768673+0000", 2024-05-08T08:53:19.093 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9963000000000005e-05 2024-05-08T08:53:19.093 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:19.093 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:19.093 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:19.093 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.093 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.093 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6706 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1224704~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:19.093 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:53:19.093 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 47.356264844999998, 2024-05-08T08:53:19.093 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052435598, 2024-05-08T08:53:19.093 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:19.093 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:19.093 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:19.093 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:19.093 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:19.094 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:19.094 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6706 2024-05-08T08:53:19.094 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.094 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:19.094 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.094 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:19.094 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:53:19.094 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:19.094 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.094 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.094 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:19.094 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:53:19.094 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:19.094 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.094 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.095 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:19.095 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:53:19.095 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:53:19.095 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.095 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.095 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:19.095 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717035+0000", 2024-05-08T08:53:19.095 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9290000000000001e-06 2024-05-08T08:53:19.095 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.095 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.095 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:19.095 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717041+0000", 2024-05-08T08:53:19.095 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6720000000000001e-06 2024-05-08T08:53:19.095 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.095 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.096 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:19.096 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717046+0000", 2024-05-08T08:53:19.096 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5650000000000003e-06 2024-05-08T08:53:19.096 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.096 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.096 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:19.096 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769311+0000", 2024-05-08T08:53:19.096 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052265645999999999 2024-05-08T08:53:19.096 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.096 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.096 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:19.096 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769379+0000", 2024-05-08T08:53:19.096 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7391000000000006e-05 2024-05-08T08:53:19.096 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.096 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.097 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:19.097 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769469+0000", 2024-05-08T08:53:19.097 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0025999999999995e-05 2024-05-08T08:53:19.097 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:19.097 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:19.097 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:19.097 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.097 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.097 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6708 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1241088~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:19.097 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:53:19.097 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 47.356246741, 2024-05-08T08:53:19.097 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052640552, 2024-05-08T08:53:19.097 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:19.097 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:19.097 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:19.098 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:19.098 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:19.098 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:19.098 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6708 2024-05-08T08:53:19.098 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.098 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:19.098 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.098 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:19.098 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:53:19.098 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:19.098 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.098 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.098 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:19.099 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:53:19.099 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:19.099 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.099 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.099 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:19.099 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717052+0000", 2024-05-08T08:53:19.099 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:53:19.099 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.099 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.099 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:19.099 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717053+0000", 2024-05-08T08:53:19.099 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8330000000000001e-06 2024-05-08T08:53:19.099 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.099 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.099 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:19.099 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717058+0000", 2024-05-08T08:53:19.100 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.933e-06 2024-05-08T08:53:19.100 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.100 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.100 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:19.100 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717063+0000", 2024-05-08T08:53:19.100 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2180000000000001e-06 2024-05-08T08:53:19.100 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.100 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.100 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:19.100 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769507+0000", 2024-05-08T08:53:19.100 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052444866999999999 2024-05-08T08:53:19.100 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.100 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.100 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:19.100 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769589+0000", 2024-05-08T08:53:19.101 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2039000000000005e-05 2024-05-08T08:53:19.101 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.101 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.101 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:19.101 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769692+0000", 2024-05-08T08:53:19.101 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102252 2024-05-08T08:53:19.101 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:19.101 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:19.101 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:19.101 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.101 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.101 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6710 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1257472~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:19.101 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:53:19.101 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 47.356228717, 2024-05-08T08:53:19.102 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.05283057, 2024-05-08T08:53:19.102 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:19.102 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:19.102 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:19.102 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:19.102 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:19.102 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:19.102 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6710 2024-05-08T08:53:19.102 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.102 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:19.102 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.102 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:19.102 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:53:19.103 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:19.103 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.103 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.103 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:19.103 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:53:19.103 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:19.103 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.103 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.103 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:19.103 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718951+0000", 2024-05-08T08:53:19.103 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0018817809999999999 2024-05-08T08:53:19.103 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.103 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.103 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:19.103 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718956+0000", 2024-05-08T08:53:19.103 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5399999999999997e-06 2024-05-08T08:53:19.104 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.104 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.104 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:19.104 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718964+0000", 2024-05-08T08:53:19.104 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6629999999999999e-06 2024-05-08T08:53:19.104 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.104 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.104 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:19.104 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718972+0000", 2024-05-08T08:53:19.104 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5229999999999998e-06 2024-05-08T08:53:19.104 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.104 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.104 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:19.104 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769729+0000", 2024-05-08T08:53:19.105 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050756959999999997 2024-05-08T08:53:19.105 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.105 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.105 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:19.105 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769803+0000", 2024-05-08T08:53:19.105 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4538999999999999e-05 2024-05-08T08:53:19.105 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.105 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.105 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:19.105 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769900+0000", 2024-05-08T08:53:19.105 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6564000000000006e-05 2024-05-08T08:53:19.105 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:19.105 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:19.105 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:19.105 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.106 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.106 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6726 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1388544~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:19.106 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:53:19.106 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 47.354153945, 2024-05-08T08:53:19.106 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052451506000000002, 2024-05-08T08:53:19.106 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:19.106 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:19.106 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:19.106 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:19.106 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:19.106 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:19.106 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6726 2024-05-08T08:53:19.106 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.106 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:19.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.107 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:19.107 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:53:19.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:19.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.107 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:19.107 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:53:19.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:19.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.107 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:19.107 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719200+0000", 2024-05-08T08:53:19.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5865999999999997e-05 2024-05-08T08:53:19.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.107 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:19.108 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719204+0000", 2024-05-08T08:53:19.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8630000000000001e-06 2024-05-08T08:53:19.108 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.108 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.108 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:19.108 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719215+0000", 2024-05-08T08:53:19.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1590000000000001e-05 2024-05-08T08:53:19.108 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.108 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.108 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:19.108 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719220+0000", 2024-05-08T08:53:19.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1619999999999999e-06 2024-05-08T08:53:19.108 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.108 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.108 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:19.108 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771425+0000", 2024-05-08T08:53:19.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052204249000000001 2024-05-08T08:53:19.109 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.109 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.109 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:19.109 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771498+0000", 2024-05-08T08:53:19.109 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3782999999999995e-05 2024-05-08T08:53:19.109 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.109 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.109 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:19.109 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771595+0000", 2024-05-08T08:53:19.109 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6992999999999998e-05 2024-05-08T08:53:19.109 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:19.109 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:19.109 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:19.109 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.109 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.109 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6728 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1404928~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:19.110 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:53:19.110 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 47.354066539999998, 2024-05-08T08:53:19.110 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052590747, 2024-05-08T08:53:19.110 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:19.110 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:19.110 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:19.110 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:19.110 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:19.110 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:19.110 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6728 2024-05-08T08:53:19.110 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.110 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:19.110 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.110 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:19.110 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:53:19.111 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:19.111 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.111 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.111 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:19.111 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:53:19.111 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:19.111 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.111 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.111 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:19.111 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719232+0000", 2024-05-08T08:53:19.111 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4000000000000001e-07 2024-05-08T08:53:19.111 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.111 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.111 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:19.111 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719234+0000", 2024-05-08T08:53:19.111 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4499999999999998e-06 2024-05-08T08:53:19.112 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.112 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.112 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:19.112 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719240+0000", 2024-05-08T08:53:19.112 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5720000000000002e-06 2024-05-08T08:53:19.112 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.112 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.112 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:19.112 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719245+0000", 2024-05-08T08:53:19.112 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8690000000000003e-06 2024-05-08T08:53:19.112 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.112 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.112 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:19.112 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771636+0000", 2024-05-08T08:53:19.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052390979999999997 2024-05-08T08:53:19.113 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.113 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.113 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:19.113 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771717+0000", 2024-05-08T08:53:19.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1291999999999993e-05 2024-05-08T08:53:19.113 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.113 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.113 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:19.113 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771822+0000", 2024-05-08T08:53:19.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104944 2024-05-08T08:53:19.113 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:19.113 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:19.113 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:19.113 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.114 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.114 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6730 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1421312~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:19.114 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:53:19.114 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 47.354046298, 2024-05-08T08:53:19.114 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052770477000000003, 2024-05-08T08:53:19.114 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:19.114 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:19.114 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:19.114 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:19.114 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:19.114 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:19.114 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6730 2024-05-08T08:53:19.114 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.115 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:19.115 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.115 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:19.115 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:53:19.115 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:19.115 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.115 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.115 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:19.115 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:53:19.115 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:19.115 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.115 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.115 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:19.115 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:53:19.115 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9000000000000002e-07 2024-05-08T08:53:19.115 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.116 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.116 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:19.116 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719254+0000", 2024-05-08T08:53:19.116 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4099999999999998e-06 2024-05-08T08:53:19.116 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.116 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.116 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:19.116 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719263+0000", 2024-05-08T08:53:19.116 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4109999999999996e-06 2024-05-08T08:53:19.116 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.116 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.116 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:19.116 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719268+0000", 2024-05-08T08:53:19.116 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2669999999999997e-06 2024-05-08T08:53:19.117 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.117 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.117 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:19.117 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771846+0000", 2024-05-08T08:53:19.117 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052577677000000003 2024-05-08T08:53:19.117 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.117 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.117 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:19.117 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771920+0000", 2024-05-08T08:53:19.117 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4460000000000002e-05 2024-05-08T08:53:19.117 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.117 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.117 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:19.117 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772022+0000", 2024-05-08T08:53:19.117 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010186199999999999 2024-05-08T08:53:19.117 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:19.118 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:19.118 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:19.118 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.118 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.118 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6732 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1437696~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:19.118 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:53:19.118 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 47.354021490999997, 2024-05-08T08:53:19.118 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052905575000000003, 2024-05-08T08:53:19.118 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:19.118 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:19.118 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:19.118 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:19.118 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:19.118 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:19.118 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6732 2024-05-08T08:53:19.119 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.119 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:19.119 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.119 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:19.119 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:53:19.119 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:19.119 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.119 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.119 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:19.119 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:53:19.119 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:19.119 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.119 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.119 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:19.119 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721392+0000", 2024-05-08T08:53:19.119 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002115287 2024-05-08T08:53:19.119 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.120 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.120 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:19.120 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721396+0000", 2024-05-08T08:53:19.120 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1679999999999997e-06 2024-05-08T08:53:19.120 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.120 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.120 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:19.120 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721404+0000", 2024-05-08T08:53:19.120 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4409999999999993e-06 2024-05-08T08:53:19.120 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.120 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.120 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:19.120 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721410+0000", 2024-05-08T08:53:19.120 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9150000000000001e-06 2024-05-08T08:53:19.120 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.120 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.121 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:19.121 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772061+0000", 2024-05-08T08:53:19.121 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050651068 2024-05-08T08:53:19.121 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.121 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.121 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:19.121 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772127+0000", 2024-05-08T08:53:19.121 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5424000000000002e-05 2024-05-08T08:53:19.121 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.121 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.121 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:19.121 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772182+0000", 2024-05-08T08:53:19.121 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5272000000000002e-05 2024-05-08T08:53:19.121 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:19.121 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:19.122 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:19.122 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.122 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.122 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6749 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1585152~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:19.122 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:53:19.122 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 47.351720721, 2024-05-08T08:53:19.122 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052350264000000001, 2024-05-08T08:53:19.122 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:19.122 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:19.122 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:19.122 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:19.122 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:19.122 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:19.123 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6749 2024-05-08T08:53:19.123 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.123 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:19.123 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.123 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:19.123 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:53:19.123 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:19.123 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.123 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.123 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:19.123 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:53:19.123 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:19.123 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.123 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.123 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:19.124 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721578+0000", 2024-05-08T08:53:19.124 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0399999999999996e-07 2024-05-08T08:53:19.124 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.124 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.124 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:19.124 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721580+0000", 2024-05-08T08:53:19.124 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5450000000000002e-06 2024-05-08T08:53:19.124 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.124 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.124 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:19.124 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721587+0000", 2024-05-08T08:53:19.124 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0190000000000001e-06 2024-05-08T08:53:19.124 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.124 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.125 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:19.125 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721593+0000", 2024-05-08T08:53:19.125 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5659999999999997e-06 2024-05-08T08:53:19.125 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.125 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.125 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:19.125 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773759+0000", 2024-05-08T08:53:19.125 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052166553999999997 2024-05-08T08:53:19.125 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.125 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.125 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:19.125 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773833+0000", 2024-05-08T08:53:19.125 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3913999999999997e-05 2024-05-08T08:53:19.125 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.125 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.126 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:19.126 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773927+0000", 2024-05-08T08:53:19.126 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4061999999999995e-05 2024-05-08T08:53:19.126 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:19.126 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:19.126 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:19.126 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.126 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.126 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6751 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1601536~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:19.126 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:53:19.126 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 47.351697442000003, 2024-05-08T08:53:19.126 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052461302000000001, 2024-05-08T08:53:19.126 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:19.126 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:19.127 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:19.127 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:19.127 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:19.127 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:19.127 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6751 2024-05-08T08:53:19.127 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.127 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:19.127 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.127 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:19.127 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:53:19.127 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:19.127 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.127 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.127 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:19.127 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:53:19.128 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:19.128 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.128 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.128 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:19.128 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722939+0000", 2024-05-08T08:53:19.128 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001338601 2024-05-08T08:53:19.128 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.128 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.128 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:19.128 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722944+0000", 2024-05-08T08:53:19.128 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4469999999999999e-06 2024-05-08T08:53:19.128 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.128 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.128 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:19.128 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722955+0000", 2024-05-08T08:53:19.128 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1892999999999999e-05 2024-05-08T08:53:19.129 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.129 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.129 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:19.129 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722963+0000", 2024-05-08T08:53:19.129 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1080000000000004e-06 2024-05-08T08:53:19.129 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.129 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.129 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:19.129 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773964+0000", 2024-05-08T08:53:19.129 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051001022 2024-05-08T08:53:19.129 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.129 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.129 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:19.129 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.774006+0000", 2024-05-08T08:53:19.129 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2729999999999999e-05 2024-05-08T08:53:19.129 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.130 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.130 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:19.130 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.774062+0000", 2024-05-08T08:53:19.130 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5501000000000003e-05 2024-05-08T08:53:19.130 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:19.130 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:19.130 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:19.130 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.130 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.130 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6760 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1683456~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:19.130 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:53:19.130 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 47.350188066000001, 2024-05-08T08:53:19.130 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052571312000000002, 2024-05-08T08:53:19.130 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:19.131 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:19.131 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:19.131 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:19.131 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:19.131 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:19.131 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6760 2024-05-08T08:53:19.131 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.131 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:19.131 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.131 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:19.131 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:53:19.131 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:19.131 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.131 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.131 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:19.132 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:53:19.132 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:19.132 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.132 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.132 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:19.132 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:53:19.132 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2100000000000002e-07 2024-05-08T08:53:19.132 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.132 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.132 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:19.132 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723113+0000", 2024-05-08T08:53:19.132 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3149999999999999e-06 2024-05-08T08:53:19.132 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.132 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.132 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:19.133 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723118+0000", 2024-05-08T08:53:19.133 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8860000000000001e-06 2024-05-08T08:53:19.133 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.133 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.133 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:19.133 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723123+0000", 2024-05-08T08:53:19.133 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6770000000000003e-06 2024-05-08T08:53:19.133 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.133 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.133 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:19.133 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775414+0000", 2024-05-08T08:53:19.133 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052290692999999999 2024-05-08T08:53:19.133 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.133 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.134 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:19.134 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775459+0000", 2024-05-08T08:53:19.134 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5639999999999997e-05 2024-05-08T08:53:19.134 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.134 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.134 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:19.134 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775681+0000", 2024-05-08T08:53:19.134 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00022168000000000001 2024-05-08T08:53:19.134 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:19.134 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:19.134 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:19.134 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.134 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.134 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6761 3.16 3:6bd6d6da:::rbd_data.1140782590a5.0000000000000015:6 [read 1095680~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:19.134 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:53:19.135 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 47.350168846000003, 2024-05-08T08:53:19.135 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052751275, 2024-05-08T08:53:19.135 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:19.135 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:19.135 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:19.135 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:19.135 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:19.135 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:19.135 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6761 2024-05-08T08:53:19.135 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.135 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:19.135 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.135 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:19.135 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:53:19.135 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:19.136 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.136 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.136 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:19.136 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:53:19.136 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:19.136 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.136 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.136 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:19.136 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724939+0000", 2024-05-08T08:53:19.136 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001809817 2024-05-08T08:53:19.136 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.136 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.136 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:19.136 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724943+0000", 2024-05-08T08:53:19.136 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4920000000000004e-06 2024-05-08T08:53:19.136 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.137 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.137 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:19.137 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724956+0000", 2024-05-08T08:53:19.137 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2427e-05 2024-05-08T08:53:19.137 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.137 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.137 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:19.137 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724963+0000", 2024-05-08T08:53:19.137 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2559999999999996e-06 2024-05-08T08:53:19.137 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.137 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.137 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:19.137 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775711+0000", 2024-05-08T08:53:19.137 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050747621 2024-05-08T08:53:19.138 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.138 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.138 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:19.138 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775785+0000", 2024-05-08T08:53:19.138 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4634000000000004e-05 2024-05-08T08:53:19.138 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.138 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.138 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:19.138 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775880+0000", 2024-05-08T08:53:19.138 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5027999999999999e-05 2024-05-08T08:53:19.138 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:19.138 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:19.138 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:19.138 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.139 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.139 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6770 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1765376~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:19.139 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:53:19.139 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 47.348137751000003, 2024-05-08T08:53:19.139 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052339296, 2024-05-08T08:53:19.139 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:19.139 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:19.139 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:19.139 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:19.139 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:19.139 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:19.139 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6770 2024-05-08T08:53:19.139 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.139 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:19.139 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.139 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:19.139 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:53:19.140 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:19.140 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.140 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.140 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:19.140 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:53:19.140 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:19.140 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.140 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.140 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:19.140 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725161+0000", 2024-05-08T08:53:19.140 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-05-08T08:53:19.140 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.140 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.140 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:19.140 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725163+0000", 2024-05-08T08:53:19.141 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4370000000000001e-06 2024-05-08T08:53:19.141 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.141 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.141 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:19.141 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725169+0000", 2024-05-08T08:53:19.141 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8409999999999996e-06 2024-05-08T08:53:19.141 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.141 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.141 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:19.141 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725174+0000", 2024-05-08T08:53:19.141 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7249999999999997e-06 2024-05-08T08:53:19.141 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.141 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.141 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:19.141 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777343+0000", 2024-05-08T08:53:19.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052169633999999999 2024-05-08T08:53:19.142 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.142 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.142 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:19.142 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777409+0000", 2024-05-08T08:53:19.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6033999999999998e-05 2024-05-08T08:53:19.142 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.142 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.142 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:19.142 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777499+0000", 2024-05-08T08:53:19.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0224999999999998e-05 2024-05-08T08:53:19.142 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:19.142 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:19.143 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:19.143 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.143 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.143 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6771 3.16 3:6bd6d6da:::rbd_data.1140782590a5.0000000000000015:6 [read 1177600~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:19.143 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:53:19.143 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 47.348117934000001, 2024-05-08T08:53:19.143 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052438604999999999, 2024-05-08T08:53:19.143 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:19.143 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:19.143 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:19.143 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:19.143 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:19.143 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:19.143 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6771 2024-05-08T08:53:19.144 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.144 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:19.144 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.144 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:19.144 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:53:19.144 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:19.144 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.144 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.144 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:19.144 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:53:19.144 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:19.144 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.144 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.144 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:19.144 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:53:19.145 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:53:19.145 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.145 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.145 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:19.145 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725188+0000", 2024-05-08T08:53:19.145 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1579999999999999e-06 2024-05-08T08:53:19.145 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.145 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.145 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:19.145 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725195+0000", 2024-05-08T08:53:19.145 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8909999999999995e-06 2024-05-08T08:53:19.145 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.145 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.145 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:19.145 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725200+0000", 2024-05-08T08:53:19.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9810000000000003e-06 2024-05-08T08:53:19.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.146 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.146 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:19.146 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777527+0000", 2024-05-08T08:53:19.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052326582000000003 2024-05-08T08:53:19.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.146 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.146 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:19.146 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777566+0000", 2024-05-08T08:53:19.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9447999999999999e-05 2024-05-08T08:53:19.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.146 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.146 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:19.146 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777619+0000", 2024-05-08T08:53:19.147 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2135000000000001e-05 2024-05-08T08:53:19.147 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:19.147 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:19.147 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:19.147 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.147 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.147 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6772 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1781760~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:19.147 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:53:19.147 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 47.348089877, 2024-05-08T08:53:19.147 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052601799999999997, 2024-05-08T08:53:19.147 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:19.147 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:19.147 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:19.147 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:19.147 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:19.148 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:19.148 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6772 2024-05-08T08:53:19.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.148 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:19.148 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.148 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:19.148 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:53:19.148 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:19.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.148 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.148 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:19.148 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:53:19.148 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:19.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.148 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.148 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:19.149 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726939+0000", 2024-05-08T08:53:19.149 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001730493 2024-05-08T08:53:19.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.149 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:19.149 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726943+0000", 2024-05-08T08:53:19.149 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7330000000000003e-06 2024-05-08T08:53:19.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.149 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:19.149 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726954+0000", 2024-05-08T08:53:19.149 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1107000000000001e-05 2024-05-08T08:53:19.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.149 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:19.149 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726962+0000", 2024-05-08T08:53:19.150 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.503e-06 2024-05-08T08:53:19.150 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.150 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.150 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:19.150 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777653+0000", 2024-05-08T08:53:19.150 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050691542999999999 2024-05-08T08:53:19.150 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.150 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.150 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:19.150 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777719+0000", 2024-05-08T08:53:19.150 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5228000000000006e-05 2024-05-08T08:53:19.150 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.150 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.150 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:19.150 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777810+0000", 2024-05-08T08:53:19.150 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1193000000000006e-05 2024-05-08T08:53:19.150 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:19.151 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:19.151 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:19.151 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.151 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.151 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6782 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1880064~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:19.151 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:53:19.151 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 47.346121369999999, 2024-05-08T08:53:19.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052326492000000002, 2024-05-08T08:53:19.151 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:19.151 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:19.151 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:19.151 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:19.151 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:19.151 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:19.151 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6782 2024-05-08T08:53:19.152 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.152 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:19.152 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.152 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:19.152 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:53:19.152 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:19.152 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.152 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.152 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:19.152 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:53:19.152 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:19.152 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.152 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.152 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:19.152 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728939+0000", 2024-05-08T08:53:19.152 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001762719 2024-05-08T08:53:19.153 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.153 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.153 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:19.153 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728944+0000", 2024-05-08T08:53:19.153 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.746e-06 2024-05-08T08:53:19.153 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.153 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.153 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:19.153 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728955+0000", 2024-05-08T08:53:19.153 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1043000000000001e-05 2024-05-08T08:53:19.153 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.153 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.153 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:19.153 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728963+0000", 2024-05-08T08:53:19.154 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5809999999999999e-06 2024-05-08T08:53:19.154 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.154 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.154 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:19.154 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779320+0000", 2024-05-08T08:53:19.154 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050357290999999998 2024-05-08T08:53:19.154 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.154 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.154 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:19.154 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779401+0000", 2024-05-08T08:53:19.154 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0574000000000005e-05 2024-05-08T08:53:19.154 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:19.154 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:19.154 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:19.154 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779503+0000", 2024-05-08T08:53:19.155 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102538 2024-05-08T08:53:19.155 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:19.155 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:19.155 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:19.155 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:19.155 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:19.155 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:53:19.156 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:19.159+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:19.157 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:53:19.257 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:19.260+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:19.311 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:19.311 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:53:19.311 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:19.311 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5194.0:6375 3.1f 3:fe9f0015:::rbd_data.1140100c9703.0000000000000053:head [read 524288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-05-08T08:53:19.311 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:19.314631+0000", 2024-05-08T08:53:19.312 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.000142977, 2024-05-08T08:53:19.312 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00015974399999999999, 2024-05-08T08:53:19.312 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:19.312 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:19.312 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "reached pg", 2024-05-08T08:53:19.312 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:19.312 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5194", 2024-05-08T08:53:19.312 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2693363123", 2024-05-08T08:53:19.312 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 6375 2024-05-08T08:53:19.312 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:19.312 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:19.313 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:19.313 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:19.313 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:19.314631+0000", 2024-05-08T08:53:19.313 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:19.313 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:19.313 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:19.313 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:19.313 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:19.314631+0000", 2024-05-08T08:53:19.313 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:19.313 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:19.314 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:19.314 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:19.314 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:19.314631+0000", 2024-05-08T08:53:19.314 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1399999999999997e-07 2024-05-08T08:53:19.314 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:19.314 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:19.314 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:19.314 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:19.314634+0000", 2024-05-08T08:53:19.314 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6259999999999999e-06 2024-05-08T08:53:19.314 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:19.314 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:19.315 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:19.315 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:19.314641+0000", 2024-05-08T08:53:19.315 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.374e-06 2024-05-08T08:53:19.315 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:19.315 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:19.315 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:19.315 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:19.314649+0000", 2024-05-08T08:53:19.315 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2030000000000003e-06 2024-05-08T08:53:19.315 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:19.315 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:19.315 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:19.315 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:19.314697+0000", 2024-05-08T08:53:19.316 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8362999999999998e-05 2024-05-08T08:53:19.316 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:19.316 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:19.316 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:19.316 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:19.314790+0000", 2024-05-08T08:53:19.316 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3564000000000001e-05 2024-05-08T08:53:19.316 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:19.316 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:19.316 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:19.316 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:19.317 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:19.317 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5194.0:6376 3.1f 3:fe9f0015:::rbd_data.1140100c9703.0000000000000053:head [read 528384~28672] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-05-08T08:53:19.317 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:19.314661+0000", 2024-05-08T08:53:19.317 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.00011301700000000001, 2024-05-08T08:53:19.317 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00020444300000000001, 2024-05-08T08:53:19.317 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:19.317 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:19.317 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "reached pg", 2024-05-08T08:53:19.317 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:19.317 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5194", 2024-05-08T08:53:19.317 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2693363123", 2024-05-08T08:53:19.317 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 6376 2024-05-08T08:53:19.318 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:19.318 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:19.318 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:19.318 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:19.318 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:19.314661+0000", 2024-05-08T08:53:19.318 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:19.318 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:19.318 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:19.318 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:19.318 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:19.314661+0000", 2024-05-08T08:53:19.319 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:19.319 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:19.319 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:19.319 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:19.319 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:19.314661+0000", 2024-05-08T08:53:19.319 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.75e-07 2024-05-08T08:53:19.319 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:19.319 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:19.319 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:19.319 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:19.314664+0000", 2024-05-08T08:53:19.319 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.531e-06 2024-05-08T08:53:19.319 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:19.320 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:19.320 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:19.320 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:19.314673+0000", 2024-05-08T08:53:19.320 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7269999999999996e-06 2024-05-08T08:53:19.320 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:19.320 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:19.320 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:19.320 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:19.314678+0000", 2024-05-08T08:53:19.320 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.31e-06 2024-05-08T08:53:19.320 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:19.320 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:19.320 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:19.321 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:19.314815+0000", 2024-05-08T08:53:19.321 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000137291 2024-05-08T08:53:19.321 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:19.321 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:19.321 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:19.321 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:19.321 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:19.321 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5194.0:6379 3.1f 3.a9b312bf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-05-08T08:53:19.321 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:19.314717+0000", 2024-05-08T08:53:19.321 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 5.6539000000000002e-05, 2024-05-08T08:53:19.321 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00021196299999999999, 2024-05-08T08:53:19.321 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:19.322 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:19.322 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "reached pg", 2024-05-08T08:53:19.322 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:19.322 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5194", 2024-05-08T08:53:19.322 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2693363123", 2024-05-08T08:53:19.322 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 6379 2024-05-08T08:53:19.322 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:19.322 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:19.322 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:19.322 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:19.322 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:19.314717+0000", 2024-05-08T08:53:19.322 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:19.322 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:19.323 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:19.323 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:19.323 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:19.314717+0000", 2024-05-08T08:53:19.323 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:19.323 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:19.323 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:19.323 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:19.323 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:19.314718+0000", 2024-05-08T08:53:19.323 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0100000000000005e-07 2024-05-08T08:53:19.323 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:19.323 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:19.323 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:19.324 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:19.314720+0000", 2024-05-08T08:53:19.324 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7140000000000002e-06 2024-05-08T08:53:19.324 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:19.324 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:19.324 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:19.324 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:19.314728+0000", 2024-05-08T08:53:19.324 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0020000000000006e-06 2024-05-08T08:53:19.324 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:19.324 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:19.324 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:19.324 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:19.314735+0000", 2024-05-08T08:53:19.325 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3770000000000001e-06 2024-05-08T08:53:19.325 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:19.325 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:19.325 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:19.325 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:19.314923+0000", 2024-05-08T08:53:19.325 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00018778800000000001 2024-05-08T08:53:19.325 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:19.325 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:19.325 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:19.325 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:19.325 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T08:53:19.326 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 3 2024-05-08T08:53:19.326 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:19.327 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:53:19.357 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:19.360+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:19.441 INFO:teuthology.orchestra.run.smithi059.stderr:marked in osd.4. 2024-05-08T08:53:19.458 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:19.462+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:19.460 INFO:tasks.thrashosds.thrasher:Added osd 4 2024-05-08T08:53:19.462 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T08:53:19.472 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:19.472 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:53:19.472 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:53:19.472 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:53:19.472 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:19.482 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:53:19.560 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:19.563+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:19.588 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:53:19.588 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:53:19.606 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:53:19.607 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:53:19.623 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:53:19.624 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:53:19.634 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:19.634 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:19.641 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:53:19.641 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:53:19.643 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:53:19.659 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:53:19.659 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:53:19.660 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:19.663+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:19.676 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:53:19.677 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:53:19.753 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:19.754 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:19.761 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:53:19.764+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:53:19.762 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:53:19.862 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:19.865+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:19.913 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:19.913 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:19.921 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:53:19.962 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:19.966+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:20.063 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:20.066+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:20.071 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:20.071 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:20.080 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:53:20.165 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:20.168+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:20.231 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:20.231 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:20.240 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:53:20.264 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:20.268+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:20.365 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:53:20.368+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:53:20.392 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:20.392 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:20.401 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:53:20.457 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-05-08T08:53:20.485 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:53:20.488+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:53:20.554 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:20.554 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:20.563 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:53:20.567 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:20.570+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:20.668 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:20.671+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:20.680 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:20.680 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:20.689 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:53:20.773 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:20.776+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:20.835 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:20.835 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:20.844 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:53:20.869 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:20.873+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:20.970 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:20.973+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:20.988 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:20.988 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:20.997 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:53:21.070 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:21.073+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:21.102 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:21.102 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:21.111 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:53:21.195 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:21.198+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:21.255 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:21.256 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:21.264 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:53:21.305 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:21.308+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:21.350 INFO:teuthology.orchestra.run.smithi059.stderr:nodeep-scrub is unset 2024-05-08T08:53:21.374 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:53:21.376+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:53:21.414 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:21.415 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:21.423 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:53:21.474 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:21.477+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:21.545 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:21.545 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:21.554 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:53:21.596 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:21.599+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:21.675 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:21.678+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:21.689 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T08:53:21.689 INFO:teuthology.orchestra.run.smithi196.stdout:3021 read 0x6f09c00 thru 0x6f1837b (0xe77c bytes) 2024-05-08T08:53:21.690 INFO:teuthology.orchestra.run.smithi196.stdout:3024 read 0x1324000 thru 0x13287c4 (0x47c5 bytes) 2024-05-08T08:53:21.691 INFO:teuthology.orchestra.run.smithi196.stdout:3025 write 0x777fe00 thru 0x7789c98 (0x9e99 bytes) 2024-05-08T08:53:21.692 INFO:teuthology.orchestra.run.smithi196.stdout:3027 read 0x2506800 thru 0x250dd85 (0x7586 bytes) 2024-05-08T08:53:21.692 INFO:teuthology.orchestra.run.smithi196.stdout:3028 punch from 0x8810a00 to 0x881f000, (0xe600 bytes) 2024-05-08T08:53:21.720 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:21.720 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:21.732 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:53:21.735 INFO:teuthology.orchestra.run.smithi196.stdout:3032 write 0xf0ee00 thru 0xf1d4e9 (0xe6ea bytes) 2024-05-08T08:53:21.735 INFO:teuthology.orchestra.run.smithi196.stdout:3034 read 0xaae6600 thru 0xaaeab6f (0x4570 bytes) 2024-05-08T08:53:21.744 INFO:teuthology.orchestra.run.smithi196.stdout:3035 trunc from 0xb79fa00 to 0x37d1200 2024-05-08T08:53:21.816 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:21.819+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:21.846 INFO:teuthology.orchestra.run.smithi196.stdout:3036 write 0x5dede00 thru 0x5df950a (0xb70b bytes) 2024-05-08T08:53:21.847 INFO:teuthology.orchestra.run.smithi196.stdout:3037 read 0x49d9c00 thru 0x49e69b4 (0xcdb5 bytes) 2024-05-08T08:53:21.847 INFO:teuthology.orchestra.run.smithi196.stdout:3040 read 0x3392800 thru 0x3394417 (0x1c18 bytes) 2024-05-08T08:53:21.848 INFO:teuthology.orchestra.run.smithi196.stdout:3041 punch from 0x1e5ae00 to 0x1e68200, (0xd400 bytes) 2024-05-08T08:53:21.852 INFO:teuthology.orchestra.run.smithi196.stdout:3042 punch from 0x2d1c00 to 0x2dfa00, (0xde00 bytes) 2024-05-08T08:53:21.870 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:53:21.870 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:53:21.872 INFO:teuthology.orchestra.run.smithi196.stdout:3043 trunc from 0x5df9600 to 0x8b6e600 2024-05-08T08:53:21.876 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:21.879+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:21.896 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:21.897 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:21.906 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:53:21.911 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:53:21.911 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:53:21.952 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:53:21.953 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:53:21.977 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:21.980+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:21.982 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:53:21.982 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:53:22.016 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:53:22.016 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:53:22.044 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:53:22.044 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:53:22.053 INFO:teuthology.orchestra.run.smithi196.stdout:3044 write 0x1968a00 thru 0x196d218 (0x4819 bytes) 2024-05-08T08:53:22.053 INFO:teuthology.orchestra.run.smithi196.stdout:3045 read 0xe8de00 thru 0xe97d6f (0x9f70 bytes) 2024-05-08T08:53:22.053 INFO:teuthology.orchestra.run.smithi196.stdout:3047 read 0x4365e00 thru 0x436b396 (0x5597 bytes) 2024-05-08T08:53:22.057 INFO:teuthology.orchestra.run.smithi196.stdout:3048 trunc from 0x8b6e600 to 0x5ff0400 2024-05-08T08:53:22.078 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:22.081+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:22.082 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:22.082 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:53:22.082 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.082 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5185.0:4271 3.1b 3:dc4f7061:::rbd_data.1140133d334a.0000000000000003:15 [read 39936~2048 out=2048b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-05-08T08:53:22.082 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:22.082724+0000", 2024-05-08T08:53:22.082 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.001806399, 2024-05-08T08:53:22.082 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0018349709999999999, 2024-05-08T08:53:22.082 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:22.082 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:22.082 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:22.083 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:22.083 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5185", 2024-05-08T08:53:22.083 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2726006659", 2024-05-08T08:53:22.083 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 4271 2024-05-08T08:53:22.083 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.083 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:22.083 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.083 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:22.083 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:22.082724+0000", 2024-05-08T08:53:22.083 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:22.083 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.083 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.083 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:22.083 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:22.082724+0000", 2024-05-08T08:53:22.083 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:22.083 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.084 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.084 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:22.084 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:22.082725+0000", 2024-05-08T08:53:22.084 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3300000000000002e-07 2024-05-08T08:53:22.084 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.084 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.084 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:22.084 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:22.082727+0000", 2024-05-08T08:53:22.084 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4600000000000002e-06 2024-05-08T08:53:22.084 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.084 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.084 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:22.084 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:22.082734+0000", 2024-05-08T08:53:22.084 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7109999999999998e-06 2024-05-08T08:53:22.085 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.085 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.085 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:22.085 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:22.082739+0000", 2024-05-08T08:53:22.085 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0170000000000004e-06 2024-05-08T08:53:22.085 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.085 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.085 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:22.085 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:22.084161+0000", 2024-05-08T08:53:22.085 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0014217349999999999 2024-05-08T08:53:22.085 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.085 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.085 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:22.085 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:22.084237+0000", 2024-05-08T08:53:22.086 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.6301e-05 2024-05-08T08:53:22.086 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:22.086 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:22.086 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:22.086 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.086 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.086 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5185.0:4291 3.3 3:c50e2de4:::rbd_data.1140133d334a.0000000000000008:15 [read 43008~1024 out=1024b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-05-08T08:53:22.086 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:22.082815+0000", 2024-05-08T08:53:22.086 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.001716197, 2024-05-08T08:53:22.086 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0018256380000000001, 2024-05-08T08:53:22.086 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:22.086 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:22.086 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:22.086 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:22.086 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5185", 2024-05-08T08:53:22.087 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2726006659", 2024-05-08T08:53:22.087 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 4291 2024-05-08T08:53:22.087 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.087 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:22.087 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.087 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:22.087 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:22.082815+0000", 2024-05-08T08:53:22.087 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:22.087 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.087 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.087 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:22.087 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:22.082815+0000", 2024-05-08T08:53:22.087 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:22.087 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.087 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.087 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:22.088 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:22.082815+0000", 2024-05-08T08:53:22.088 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9099999999999999e-07 2024-05-08T08:53:22.088 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.088 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.088 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:22.088 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:22.082817+0000", 2024-05-08T08:53:22.088 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.928e-06 2024-05-08T08:53:22.088 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.088 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.088 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:22.088 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:22.082823+0000", 2024-05-08T08:53:22.088 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6720000000000001e-06 2024-05-08T08:53:22.088 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.088 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.088 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:22.089 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:22.082827+0000", 2024-05-08T08:53:22.089 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7249999999999997e-06 2024-05-08T08:53:22.089 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.089 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.089 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:22.089 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:22.084402+0000", 2024-05-08T08:53:22.089 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0015743829999999999 2024-05-08T08:53:22.089 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.089 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.089 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:22.089 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:22.084498+0000", 2024-05-08T08:53:22.089 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5853000000000005e-05 2024-05-08T08:53:22.089 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:22.089 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:22.089 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:22.089 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.090 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.090 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5185.0:4294 3.1b 3.860ef23b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-05-08T08:53:22.090 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:22.082837+0000", 2024-05-08T08:53:22.090 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0016934960000000001, 2024-05-08T08:53:22.090 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0018691650000000001, 2024-05-08T08:53:22.090 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:22.090 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:22.090 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:22.090 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:22.090 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5185", 2024-05-08T08:53:22.090 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2726006659", 2024-05-08T08:53:22.090 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 4294 2024-05-08T08:53:22.090 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.090 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:22.090 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.090 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:22.091 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:22.082837+0000", 2024-05-08T08:53:22.091 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:22.091 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.091 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.091 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:22.091 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:22.082837+0000", 2024-05-08T08:53:22.091 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:22.091 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.091 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.091 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:22.091 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:22.082838+0000", 2024-05-08T08:53:22.091 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3599999999999999e-07 2024-05-08T08:53:22.091 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.091 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.091 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:22.091 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:22.082840+0000", 2024-05-08T08:53:22.092 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.03e-06 2024-05-08T08:53:22.092 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.092 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.092 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:22.092 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:22.082845+0000", 2024-05-08T08:53:22.092 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4729999999999999e-06 2024-05-08T08:53:22.092 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.092 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.092 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:22.092 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:22.082850+0000", 2024-05-08T08:53:22.092 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5889999999999996e-06 2024-05-08T08:53:22.092 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:22.092 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:22.092 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:22.093 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.093 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.093 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5185.0:4296 3.1b 3.860ef23b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-05-08T08:53:22.093 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:22.082859+0000", 2024-05-08T08:53:22.093 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.001671975, 2024-05-08T08:53:22.093 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001903739, 2024-05-08T08:53:22.093 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:22.093 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:22.093 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:22.093 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:22.093 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5185", 2024-05-08T08:53:22.093 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2726006659", 2024-05-08T08:53:22.093 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 4296 2024-05-08T08:53:22.093 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.094 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:22.094 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.094 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:22.094 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:22.082859+0000", 2024-05-08T08:53:22.094 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:22.094 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.094 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.094 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:22.094 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:22.082859+0000", 2024-05-08T08:53:22.094 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:22.094 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.094 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.094 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:22.094 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:22.082859+0000", 2024-05-08T08:53:22.094 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0400000000000002e-07 2024-05-08T08:53:22.095 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.095 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.095 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:22.095 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:22.082861+0000", 2024-05-08T08:53:22.095 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1730000000000002e-06 2024-05-08T08:53:22.095 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.095 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.095 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:22.095 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:22.082868+0000", 2024-05-08T08:53:22.095 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5220000000000004e-06 2024-05-08T08:53:22.095 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.095 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.095 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:22.095 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:22.082873+0000", 2024-05-08T08:53:22.095 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1970000000000004e-06 2024-05-08T08:53:22.096 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:22.096 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:22.096 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:22.096 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.096 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.096 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5185.0:4309 3.1b 3.860ef23b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-05-08T08:53:22.096 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:22.082884+0000", 2024-05-08T08:53:22.096 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0016466289999999999, 2024-05-08T08:53:22.096 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001931707, 2024-05-08T08:53:22.096 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:22.096 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:22.096 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:22.096 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:22.096 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5185", 2024-05-08T08:53:22.096 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2726006659", 2024-05-08T08:53:22.097 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 4309 2024-05-08T08:53:22.097 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.097 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:22.097 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.097 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:22.097 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:22.082884+0000", 2024-05-08T08:53:22.097 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:22.097 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.097 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.097 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:22.097 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:22.082884+0000", 2024-05-08T08:53:22.097 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:22.097 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.097 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.098 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:22.098 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:22.082885+0000", 2024-05-08T08:53:22.098 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7899999999999999e-07 2024-05-08T08:53:22.098 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.098 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.098 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:22.098 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:22.082888+0000", 2024-05-08T08:53:22.098 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7979999999999999e-06 2024-05-08T08:53:22.098 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.098 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.098 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:22.098 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:22.082894+0000", 2024-05-08T08:53:22.098 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9359999999999996e-06 2024-05-08T08:53:22.098 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.098 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.098 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:22.099 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:22.082900+0000", 2024-05-08T08:53:22.099 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3059999999999997e-06 2024-05-08T08:53:22.099 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:22.099 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:22.099 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:22.099 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.099 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.099 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5185.0:4311 3.3 3.27b470a3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-05-08T08:53:22.099 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:22.082906+0000", 2024-05-08T08:53:22.099 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.00162474, 2024-05-08T08:53:22.099 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001957351, 2024-05-08T08:53:22.099 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:22.099 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:22.099 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:22.099 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:22.099 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5185", 2024-05-08T08:53:22.100 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2726006659", 2024-05-08T08:53:22.100 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 4311 2024-05-08T08:53:22.100 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.100 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:22.100 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.100 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:22.100 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:22.082906+0000", 2024-05-08T08:53:22.100 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:22.100 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.100 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.100 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:22.100 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:22.082906+0000", 2024-05-08T08:53:22.100 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:22.100 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.100 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.100 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:22.100 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:22.082907+0000", 2024-05-08T08:53:22.101 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9499999999999998e-07 2024-05-08T08:53:22.101 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.101 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.101 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:22.101 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:22.082908+0000", 2024-05-08T08:53:22.101 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.903e-06 2024-05-08T08:53:22.101 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.101 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.101 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:22.101 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:22.082913+0000", 2024-05-08T08:53:22.101 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0250000000000002e-06 2024-05-08T08:53:22.101 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.101 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.101 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:22.102 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:22.082918+0000", 2024-05-08T08:53:22.102 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5759999999999999e-06 2024-05-08T08:53:22.102 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:22.102 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:22.102 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:22.102 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.102 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.102 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5185.0:4312 3.3 3.27b470a3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-05-08T08:53:22.102 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:22.082924+0000", 2024-05-08T08:53:22.102 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.001606838, 2024-05-08T08:53:22.102 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0019914960000000002, 2024-05-08T08:53:22.102 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:22.102 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:22.102 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:22.102 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:22.103 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5185", 2024-05-08T08:53:22.103 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2726006659", 2024-05-08T08:53:22.103 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 4312 2024-05-08T08:53:22.103 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.103 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:22.103 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.103 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:22.103 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:22.082924+0000", 2024-05-08T08:53:22.103 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:22.103 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.103 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.103 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:22.103 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:22.082924+0000", 2024-05-08T08:53:22.103 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:22.103 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.103 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.104 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:22.104 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:22.082924+0000", 2024-05-08T08:53:22.104 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.2599999999999998e-07 2024-05-08T08:53:22.104 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.104 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.104 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:22.104 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:22.082926+0000", 2024-05-08T08:53:22.104 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.762e-06 2024-05-08T08:53:22.104 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.104 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.104 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:22.104 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:22.082931+0000", 2024-05-08T08:53:22.104 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0189999999999997e-06 2024-05-08T08:53:22.104 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.104 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.104 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:22.105 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:22.082935+0000", 2024-05-08T08:53:22.105 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7720000000000001e-06 2024-05-08T08:53:22.105 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:22.105 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:22.105 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:22.105 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.105 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.105 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5185.0:4321 3.3 3.27b470a3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-05-08T08:53:22.105 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:22.082941+0000", 2024-05-08T08:53:22.105 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.001590132, 2024-05-08T08:53:22.105 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.002032846, 2024-05-08T08:53:22.105 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:22.105 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:22.105 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:22.105 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:22.105 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5185", 2024-05-08T08:53:22.106 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2726006659", 2024-05-08T08:53:22.106 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 4321 2024-05-08T08:53:22.106 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.106 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:22.106 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.106 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:22.106 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:22.082941+0000", 2024-05-08T08:53:22.106 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:22.106 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.106 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.106 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:22.106 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:22.082941+0000", 2024-05-08T08:53:22.106 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:22.106 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.107 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.107 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:22.107 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:22.082941+0000", 2024-05-08T08:53:22.107 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0499999999999999e-07 2024-05-08T08:53:22.107 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.107 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.107 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:22.107 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:22.082943+0000", 2024-05-08T08:53:22.107 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.7710000000000001e-06 2024-05-08T08:53:22.107 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.107 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.107 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:22.107 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:22.082948+0000", 2024-05-08T08:53:22.107 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4729999999999999e-06 2024-05-08T08:53:22.107 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.107 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.107 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:22.108 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:22.082959+0000", 2024-05-08T08:53:22.108 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0689999999999999e-05 2024-05-08T08:53:22.108 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:22.108 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:22.108 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:22.108 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.108 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.108 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5185.0:4322 3.3 3.27b470a3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-05-08T08:53:22.108 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:22.082973+0000", 2024-05-08T08:53:22.108 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0015574, 2024-05-08T08:53:22.108 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0020541880000000002, 2024-05-08T08:53:22.108 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:22.108 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:22.108 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:22.108 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:22.108 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5185", 2024-05-08T08:53:22.109 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2726006659", 2024-05-08T08:53:22.109 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 4322 2024-05-08T08:53:22.109 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.109 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:22.109 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.109 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:22.109 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:22.082973+0000", 2024-05-08T08:53:22.109 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:22.109 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.109 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.109 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:22.109 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:22.082973+0000", 2024-05-08T08:53:22.109 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:22.109 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.109 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.109 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:22.110 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:22.082974+0000", 2024-05-08T08:53:22.110 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7999999999999996e-07 2024-05-08T08:53:22.110 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.110 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.110 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:22.110 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:22.082977+0000", 2024-05-08T08:53:22.110 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.599e-06 2024-05-08T08:53:22.110 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.110 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.110 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:22.110 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:22.082983+0000", 2024-05-08T08:53:22.110 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.55e-06 2024-05-08T08:53:22.110 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.110 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.110 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:22.111 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:22.082988+0000", 2024-05-08T08:53:22.111 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.361e-06 2024-05-08T08:53:22.111 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:22.111 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:22.111 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:22.111 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.111 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.111 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5185.0:4323 3.1f 3:fcd5c0b9:::rbd_object_map.114035b37483:head [call lock.assert_locked in=44b,call rbd.object_map_resize in=29b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-05-08T08:53:22.111 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:22.084226+0000", 2024-05-08T08:53:22.111 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.000305035, 2024-05-08T08:53:22.111 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00085777, 2024-05-08T08:53:22.111 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:22.111 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:22.111 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "waiting for sub ops", 2024-05-08T08:53:22.111 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:22.111 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5185", 2024-05-08T08:53:22.112 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2726006659", 2024-05-08T08:53:22.112 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 4323 2024-05-08T08:53:22.112 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.112 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:22.112 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.112 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:22.112 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:22.084226+0000", 2024-05-08T08:53:22.112 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:22.112 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.112 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.112 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:22.112 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:22.084226+0000", 2024-05-08T08:53:22.112 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:22.112 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.112 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.113 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:22.113 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:22.084227+0000", 2024-05-08T08:53:22.113 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8100000000000001e-07 2024-05-08T08:53:22.113 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.113 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.113 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:22.113 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:22.084232+0000", 2024-05-08T08:53:22.113 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.344e-06 2024-05-08T08:53:22.113 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.113 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.113 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:22.113 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:22.084244+0000", 2024-05-08T08:53:22.113 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2254e-05 2024-05-08T08:53:22.113 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.113 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.113 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:22.114 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:22.084252+0000", 2024-05-08T08:53:22.114 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.503e-06 2024-05-08T08:53:22.114 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.114 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.114 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:22.114 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:22.084317+0000", 2024-05-08T08:53:22.114 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4820999999999995e-05 2024-05-08T08:53:22.114 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.114 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.114 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:22.114 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:22.084400+0000", 2024-05-08T08:53:22.114 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.2857999999999998e-05 2024-05-08T08:53:22.114 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.114 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.114 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 3", 2024-05-08T08:53:22.115 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:22.084658+0000", 2024-05-08T08:53:22.115 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000258514 2024-05-08T08:53:22.115 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:22.115 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:22.115 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:22.115 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:22.115 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T08:53:22.115 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 10 2024-05-08T08:53:22.115 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:22.116 INFO:teuthology.orchestra.run.smithi196.stdout:3049 read 0x47b2c00 thru 0x47bb706 (0x8b07 bytes) 2024-05-08T08:53:22.116 INFO:teuthology.orchestra.run.smithi196.stdout:3050 read 0x4f01200 thru 0x4f0d732 (0xc533 bytes) 2024-05-08T08:53:22.116 INFO:teuthology.orchestra.run.smithi196.stdout:3051 read 0x15fb000 thru 0x15ff7ef (0x47f0 bytes) 2024-05-08T08:53:22.116 INFO:teuthology.orchestra.run.smithi196.stdout:3052 read 0x4468c00 thru 0x446a4bc (0x18bd bytes) 2024-05-08T08:53:22.117 INFO:teuthology.orchestra.run.smithi196.stdout:3053 write 0x334a600 thru 0x335327f (0x8c80 bytes) 2024-05-08T08:53:22.117 INFO:teuthology.orchestra.run.smithi196.stdout:3057 clone 20 order 18 su 131072 sc 5 2024-05-08T08:53:22.117 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:53:22.178 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:22.181+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:22.279 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:22.282+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:22.281 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:22.282 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:53:22.282 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:53:22.282 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:53:22.282 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:22.292 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:53:22.380 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:22.383+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:22.449 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:22.450 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:53:22.450 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:53:22.450 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:53:22.450 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.450 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4621.0:9089 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 23339520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:53:22.450 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:53:22.450 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 49.958554386000003, 2024-05-08T08:53:22.450 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.13814907200000001, 2024-05-08T08:53:22.450 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:22.451 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:22.451 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:53:22.451 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:22.451 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4621", 2024-05-08T08:53:22.451 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:22.451 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9089 2024-05-08T08:53:22.451 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.451 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:22.452 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.452 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:22.452 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:53:22.452 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:22.452 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.452 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.452 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:22.452 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:53:22.452 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:22.452 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.452 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.452 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:22.453 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493005+0000", 2024-05-08T08:53:22.453 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1700000000000001e-06 2024-05-08T08:53:22.453 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.453 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.453 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:22.453 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.513963+0000", 2024-05-08T08:53:22.453 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.020957581999999999 2024-05-08T08:53:22.453 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.453 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.453 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:22.453 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.513995+0000", 2024-05-08T08:53:22.453 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1993999999999999e-05 2024-05-08T08:53:22.453 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.454 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.454 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:22.454 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514007+0000", 2024-05-08T08:53:22.454 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1581e-05 2024-05-08T08:53:22.454 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.454 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.454 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:22.454 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514122+0000", 2024-05-08T08:53:22.454 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000115282 2024-05-08T08:53:22.454 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.454 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.455 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op must wait for map", 2024-05-08T08:53:22.455 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514128+0000", 2024-05-08T08:53:22.455 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9950000000000002e-06 2024-05-08T08:53:22.455 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.455 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.455 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:22.455 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.515800+0000", 2024-05-08T08:53:22.455 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001671886 2024-05-08T08:53:22.455 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.455 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.455 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:22.455 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.515893+0000", 2024-05-08T08:53:22.455 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3490000000000001e-05 2024-05-08T08:53:22.456 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.456 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.456 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 3", 2024-05-08T08:53:22.456 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.517647+0000", 2024-05-08T08:53:22.456 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0017535630000000001 2024-05-08T08:53:22.456 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.456 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.456 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:53:22.456 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.627578+0000", 2024-05-08T08:53:22.456 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.109931557 2024-05-08T08:53:22.456 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.456 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.457 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:53:22.457 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631055+0000", 2024-05-08T08:53:22.457 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0034761420000000002 2024-05-08T08:53:22.457 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.457 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.457 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:53:22.457 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631096+0000", 2024-05-08T08:53:22.457 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1059000000000001e-05 2024-05-08T08:53:22.457 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.457 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.457 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:22.457 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631151+0000", 2024-05-08T08:53:22.457 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5770999999999998e-05 2024-05-08T08:53:22.458 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:22.458 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:22.458 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:22.458 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.458 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.458 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1065 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 10944512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:22.458 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485411+0000", 2024-05-08T08:53:22.458 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 45.966145699999998, 2024-05-08T08:53:22.458 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070281386000000001, 2024-05-08T08:53:22.458 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:22.458 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:22.458 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:22.459 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:22.459 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:22.459 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:22.459 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1065 2024-05-08T08:53:22.459 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.459 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:22.459 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.459 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:22.459 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485411+0000", 2024-05-08T08:53:22.459 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:22.459 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.459 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.459 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:22.460 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485411+0000", 2024-05-08T08:53:22.460 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:22.460 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.460 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.460 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:22.460 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485411+0000", 2024-05-08T08:53:22.460 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5600000000000001e-07 2024-05-08T08:53:22.460 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.460 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.460 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:22.460 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485414+0000", 2024-05-08T08:53:22.460 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2299999999999998e-06 2024-05-08T08:53:22.460 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.461 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.461 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:22.461 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485420+0000", 2024-05-08T08:53:22.461 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2179999999999996e-06 2024-05-08T08:53:22.461 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.461 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.461 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:22.461 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485425+0000", 2024-05-08T08:53:22.461 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9830000000000004e-06 2024-05-08T08:53:22.461 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.461 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.461 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:22.461 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555468+0000", 2024-05-08T08:53:22.461 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070043541000000001 2024-05-08T08:53:22.462 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.462 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.462 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:22.462 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555566+0000", 2024-05-08T08:53:22.462 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8082e-05 2024-05-08T08:53:22.462 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.462 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.462 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:22.462 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555692+0000", 2024-05-08T08:53:22.462 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000125976 2024-05-08T08:53:22.462 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:22.462 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:22.463 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:22.463 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.463 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.463 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1067 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 10985472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:22.463 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485431+0000", 2024-05-08T08:53:22.463 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 45.966125564999999, 2024-05-08T08:53:22.463 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070412460999999996, 2024-05-08T08:53:22.463 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:22.463 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:22.463 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:22.463 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:22.463 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:22.464 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:22.464 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1067 2024-05-08T08:53:22.464 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.464 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:22.464 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.464 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:22.464 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485431+0000", 2024-05-08T08:53:22.464 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:22.464 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.464 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.464 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:22.464 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485431+0000", 2024-05-08T08:53:22.465 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:22.465 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.465 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.465 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:22.465 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485432+0000", 2024-05-08T08:53:22.465 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8299999999999998e-07 2024-05-08T08:53:22.465 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.465 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.465 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:22.465 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485434+0000", 2024-05-08T08:53:22.465 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0849999999999999e-06 2024-05-08T08:53:22.465 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.466 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.466 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:22.466 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485440+0000", 2024-05-08T08:53:22.466 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0020000000000003e-06 2024-05-08T08:53:22.466 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.466 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.466 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:22.466 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485445+0000", 2024-05-08T08:53:22.466 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5180000000000004e-06 2024-05-08T08:53:22.466 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.467 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.467 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:22.467 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555731+0000", 2024-05-08T08:53:22.467 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070286214 2024-05-08T08:53:22.467 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.467 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.467 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:22.467 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555787+0000", 2024-05-08T08:53:22.467 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5992999999999997e-05 2024-05-08T08:53:22.467 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.467 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.467 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:22.468 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555844+0000", 2024-05-08T08:53:22.468 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6266e-05 2024-05-08T08:53:22.468 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:22.468 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:22.468 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:22.468 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.468 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.468 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1071 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11026432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:22.468 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:53:22.468 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 45.966104895000001, 2024-05-08T08:53:22.468 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070580037999999998, 2024-05-08T08:53:22.469 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:22.469 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:22.469 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:22.469 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:22.469 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:22.469 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:22.469 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1071 2024-05-08T08:53:22.469 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.469 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:22.469 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.469 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:22.469 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:53:22.470 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:22.470 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.470 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.470 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:22.470 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:53:22.470 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:22.470 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.470 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.470 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:22.470 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:53:22.470 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2800000000000002e-07 2024-05-08T08:53:22.470 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.471 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.471 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:22.471 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485454+0000", 2024-05-08T08:53:22.471 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2290000000000002e-06 2024-05-08T08:53:22.471 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.471 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.471 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:22.471 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485461+0000", 2024-05-08T08:53:22.471 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9920000000000001e-06 2024-05-08T08:53:22.471 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.471 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.471 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:22.472 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485467+0000", 2024-05-08T08:53:22.472 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3979999999999998e-06 2024-05-08T08:53:22.472 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.472 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.472 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:22.472 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555880+0000", 2024-05-08T08:53:22.472 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070413165 2024-05-08T08:53:22.472 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.472 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.472 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:22.472 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555926+0000", 2024-05-08T08:53:22.472 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6217000000000002e-05 2024-05-08T08:53:22.472 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.472 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.473 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:22.473 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556032+0000", 2024-05-08T08:53:22.473 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000105609 2024-05-08T08:53:22.473 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:22.473 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:22.473 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:22.473 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.473 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.473 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1073 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11067392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:22.473 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:53:22.473 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 45.966083673999997, 2024-05-08T08:53:22.473 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070821404000000004, 2024-05-08T08:53:22.473 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:22.474 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:22.474 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:22.474 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:22.474 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:22.474 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:22.474 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1073 2024-05-08T08:53:22.474 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.474 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:22.474 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.474 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:22.474 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:53:22.474 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:22.475 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.475 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.475 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:22.475 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:53:22.475 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:22.475 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.475 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.475 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:22.475 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:53:22.475 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.08e-07 2024-05-08T08:53:22.475 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.475 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.476 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:22.476 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485476+0000", 2024-05-08T08:53:22.476 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3929999999999998e-06 2024-05-08T08:53:22.476 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.476 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.476 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:22.476 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485485+0000", 2024-05-08T08:53:22.476 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1880000000000008e-06 2024-05-08T08:53:22.476 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.476 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.476 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:22.476 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485494+0000", 2024-05-08T08:53:22.476 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3109999999999995e-06 2024-05-08T08:53:22.476 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.477 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.477 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:22.477 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556082+0000", 2024-05-08T08:53:22.477 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070587426999999994 2024-05-08T08:53:22.477 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.477 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.477 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:22.477 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556177+0000", 2024-05-08T08:53:22.477 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5051999999999997e-05 2024-05-08T08:53:22.477 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.477 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.477 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:22.477 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556294+0000", 2024-05-08T08:53:22.477 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000117625 2024-05-08T08:53:22.478 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:22.478 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:22.478 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:22.478 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.478 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.478 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1075 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11108352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:22.478 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:53:22.478 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 45.966053097, 2024-05-08T08:53:22.478 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071045026999999997, 2024-05-08T08:53:22.478 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:22.478 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:22.478 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:22.479 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:22.479 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:22.479 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:22.479 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1075 2024-05-08T08:53:22.479 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.479 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:22.479 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.479 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:22.479 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:53:22.479 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:22.479 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.479 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.479 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:22.480 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:53:22.480 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:22.480 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.480 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.480 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:22.480 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:53:22.480 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.32e-07 2024-05-08T08:53:22.480 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.480 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.480 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:22.480 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485518+0000", 2024-05-08T08:53:22.480 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3769e-05 2024-05-08T08:53:22.481 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.481 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.481 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:22.481 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485527+0000", 2024-05-08T08:53:22.481 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6779999999999995e-06 2024-05-08T08:53:22.481 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.481 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.481 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:22.481 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485532+0000", 2024-05-08T08:53:22.481 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9830000000000004e-06 2024-05-08T08:53:22.481 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.482 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.482 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:22.482 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556340+0000", 2024-05-08T08:53:22.482 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070807743000000006 2024-05-08T08:53:22.482 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.482 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.482 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:22.482 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556435+0000", 2024-05-08T08:53:22.482 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4732e-05 2024-05-08T08:53:22.482 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.482 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.483 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:22.483 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556549+0000", 2024-05-08T08:53:22.483 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011369 2024-05-08T08:53:22.483 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:22.483 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:22.483 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:22.483 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.483 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.483 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1078 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11149312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:22.483 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:53:22.483 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 45.966012898000002, 2024-05-08T08:53:22.483 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071167276000000002, 2024-05-08T08:53:22.483 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:22.484 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:22.484 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:22.484 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:22.484 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:22.484 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:22.484 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1078 2024-05-08T08:53:22.484 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.484 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:22.484 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.484 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:22.484 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:53:22.484 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:22.485 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.485 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.485 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:22.485 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:53:22.485 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:22.485 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.485 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.485 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:22.485 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:53:22.485 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5600000000000001e-07 2024-05-08T08:53:22.485 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.485 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.485 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:22.486 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485547+0000", 2024-05-08T08:53:22.486 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3590000000000002e-06 2024-05-08T08:53:22.486 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.486 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.486 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:22.486 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485553+0000", 2024-05-08T08:53:22.486 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2770000000000002e-06 2024-05-08T08:53:22.486 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.486 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.486 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:22.486 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485558+0000", 2024-05-08T08:53:22.486 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.95e-06 2024-05-08T08:53:22.486 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.487 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.487 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:22.487 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556582+0000", 2024-05-08T08:53:22.487 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071024444000000006 2024-05-08T08:53:22.487 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.487 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.487 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:22.487 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556637+0000", 2024-05-08T08:53:22.487 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4672000000000001e-05 2024-05-08T08:53:22.487 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.487 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.487 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:22.487 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556711+0000", 2024-05-08T08:53:22.488 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4117999999999997e-05 2024-05-08T08:53:22.488 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:22.488 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:22.488 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:22.488 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.488 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.488 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1080 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11190272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:22.488 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:53:22.488 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 45.965992165000003, 2024-05-08T08:53:22.488 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071350721000000006, 2024-05-08T08:53:22.488 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:22.488 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:22.488 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:22.488 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:22.489 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:22.489 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:22.489 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1080 2024-05-08T08:53:22.489 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.489 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:22.489 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.489 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:22.489 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:53:22.489 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:22.489 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.489 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.489 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:22.489 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:53:22.490 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:22.490 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.490 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.490 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:22.490 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:53:22.490 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.03e-07 2024-05-08T08:53:22.490 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.490 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.490 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:22.490 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485567+0000", 2024-05-08T08:53:22.490 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1090000000000001e-06 2024-05-08T08:53:22.490 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.491 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.491 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:22.491 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485573+0000", 2024-05-08T08:53:22.491 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3049999999999998e-06 2024-05-08T08:53:22.491 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.491 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.491 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:22.491 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485578+0000", 2024-05-08T08:53:22.491 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0130000000000001e-06 2024-05-08T08:53:22.491 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.491 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.491 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:22.491 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556743+0000", 2024-05-08T08:53:22.492 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071164449000000005 2024-05-08T08:53:22.492 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.492 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.492 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:22.492 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556797+0000", 2024-05-08T08:53:22.492 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4589999999999997e-05 2024-05-08T08:53:22.492 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.492 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.492 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:22.492 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556915+0000", 2024-05-08T08:53:22.492 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000117852 2024-05-08T08:53:22.492 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:22.493 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:22.493 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:22.493 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.493 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.493 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1082 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11231232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:22.493 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:53:22.493 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 45.965971922000001, 2024-05-08T08:53:22.493 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071591396000000002, 2024-05-08T08:53:22.493 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:22.493 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:22.493 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:22.494 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:22.494 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:22.494 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:22.494 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1082 2024-05-08T08:53:22.494 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.494 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:22.494 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.494 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:22.494 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:53:22.494 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:22.494 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.495 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.495 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:22.495 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:53:22.495 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:22.495 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.495 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.495 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:22.495 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:53:22.495 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1800000000000001e-07 2024-05-08T08:53:22.495 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.495 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.496 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:22.496 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485587+0000", 2024-05-08T08:53:22.496 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2589999999999999e-06 2024-05-08T08:53:22.496 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.496 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.496 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:22.496 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485594+0000", 2024-05-08T08:53:22.496 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0789999999999997e-06 2024-05-08T08:53:22.496 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.496 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.496 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:22.496 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485598+0000", 2024-05-08T08:53:22.496 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7480000000000001e-06 2024-05-08T08:53:22.497 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.497 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.497 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:22.497 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556971+0000", 2024-05-08T08:53:22.497 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071372774999999999 2024-05-08T08:53:22.497 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.497 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.497 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:22.497 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557069+0000", 2024-05-08T08:53:22.497 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.7732999999999994e-05 2024-05-08T08:53:22.497 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.497 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.498 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:22.498 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557176+0000", 2024-05-08T08:53:22.498 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000107384 2024-05-08T08:53:22.498 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:22.498 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:22.498 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:22.498 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.498 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.498 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1084 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11272192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:22.498 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:53:22.499 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 45.965933282999998, 2024-05-08T08:53:22.499 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071808977999999996, 2024-05-08T08:53:22.499 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:22.499 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:22.499 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:22.499 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:22.499 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:22.499 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:22.499 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1084 2024-05-08T08:53:22.499 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.499 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:22.499 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.499 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:22.500 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:53:22.500 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:22.500 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.500 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.500 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:22.500 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:53:22.500 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:22.500 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.500 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.500 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:22.500 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485624+0000", 2024-05-08T08:53:22.500 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:53:22.500 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.500 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.500 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:22.501 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485626+0000", 2024-05-08T08:53:22.501 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9460000000000001e-06 2024-05-08T08:53:22.501 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.501 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.501 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:22.501 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485633+0000", 2024-05-08T08:53:22.501 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1149999999999996e-06 2024-05-08T08:53:22.501 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.501 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.501 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:22.501 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485638+0000", 2024-05-08T08:53:22.501 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5670000000000002e-06 2024-05-08T08:53:22.501 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.501 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.502 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:22.502 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557226+0000", 2024-05-08T08:53:22.502 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071587267999999996 2024-05-08T08:53:22.502 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.502 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.502 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:22.502 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557321+0000", 2024-05-08T08:53:22.502 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5785000000000005e-05 2024-05-08T08:53:22.502 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.502 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.502 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:22.502 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557432+0000", 2024-05-08T08:53:22.503 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011092800000000001 2024-05-08T08:53:22.503 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:22.503 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:22.503 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:22.503 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.503 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.503 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1086 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11313152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:22.503 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:53:22.503 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 45.965909955999997, 2024-05-08T08:53:22.503 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071933132999999996, 2024-05-08T08:53:22.503 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:22.503 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:22.503 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:22.503 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:22.504 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:22.504 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:22.504 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1086 2024-05-08T08:53:22.504 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.504 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:22.504 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.504 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:22.504 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:53:22.504 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:22.504 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.504 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.504 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:22.504 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:53:22.504 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:22.504 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.505 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.505 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:22.505 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486328+0000", 2024-05-08T08:53:22.505 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00068107900000000002 2024-05-08T08:53:22.505 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.505 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.505 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:22.505 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486331+0000", 2024-05-08T08:53:22.505 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.585e-06 2024-05-08T08:53:22.505 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.505 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.505 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:22.505 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486343+0000", 2024-05-08T08:53:22.505 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1856e-05 2024-05-08T08:53:22.506 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.506 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.506 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:22.506 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486350+0000", 2024-05-08T08:53:22.506 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8600000000000004e-06 2024-05-08T08:53:22.506 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.506 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.506 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:22.506 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557470+0000", 2024-05-08T08:53:22.506 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071120326999999997 2024-05-08T08:53:22.506 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.506 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.507 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:22.507 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557514+0000", 2024-05-08T08:53:22.507 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3909000000000003e-05 2024-05-08T08:53:22.507 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.507 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.507 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:22.507 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557580+0000", 2024-05-08T08:53:22.507 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5517000000000003e-05 2024-05-08T08:53:22.507 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:22.507 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:22.507 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:22.507 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.507 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.507 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1089 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11354112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:22.508 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:53:22.508 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 45.965197084000003, 2024-05-08T08:53:22.508 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071457381, 2024-05-08T08:53:22.508 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:22.508 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:22.508 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:22.508 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:22.508 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:22.508 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:22.508 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1089 2024-05-08T08:53:22.508 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.508 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:22.508 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.508 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:22.509 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:53:22.509 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:22.509 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.509 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.509 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:22.509 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:53:22.509 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:22.509 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.509 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.509 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:22.509 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486851+0000", 2024-05-08T08:53:22.509 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00049102 2024-05-08T08:53:22.509 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.509 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.510 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:22.510 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486854+0000", 2024-05-08T08:53:22.510 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6859999999999999e-06 2024-05-08T08:53:22.510 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.510 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.510 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:22.510 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486864+0000", 2024-05-08T08:53:22.510 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6080000000000002e-06 2024-05-08T08:53:22.510 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.510 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.510 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:22.510 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486870+0000", 2024-05-08T08:53:22.510 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.477e-06 2024-05-08T08:53:22.511 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.511 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.511 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:22.511 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557612+0000", 2024-05-08T08:53:22.511 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070741557999999996 2024-05-08T08:53:22.511 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.511 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.511 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:22.511 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557695+0000", 2024-05-08T08:53:22.511 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3300000000000005e-05 2024-05-08T08:53:22.511 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.511 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.511 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:22.511 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557817+0000", 2024-05-08T08:53:22.512 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012173200000000001 2024-05-08T08:53:22.512 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:22.512 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:22.512 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:22.512 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.512 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.512 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1091 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11395072~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:22.512 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:53:22.512 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 45.964676289000003, 2024-05-08T08:53:22.512 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071184737999999997, 2024-05-08T08:53:22.512 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:22.512 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:22.512 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:22.512 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:22.513 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:22.513 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:22.513 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1091 2024-05-08T08:53:22.513 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.513 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:22.513 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.513 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:22.513 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:53:22.513 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:22.513 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.513 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.513 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:22.513 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:53:22.513 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:22.514 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.514 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.514 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:22.514 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486881+0000", 2024-05-08T08:53:22.514 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0699999999999997e-07 2024-05-08T08:53:22.514 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.514 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.514 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:22.514 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486883+0000", 2024-05-08T08:53:22.514 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.402e-06 2024-05-08T08:53:22.514 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.515 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.515 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:22.515 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486890+0000", 2024-05-08T08:53:22.515 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0480000000000001e-06 2024-05-08T08:53:22.515 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.515 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.515 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:22.515 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486896+0000", 2024-05-08T08:53:22.515 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7640000000000002e-06 2024-05-08T08:53:22.515 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.515 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.515 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:22.515 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557860+0000", 2024-05-08T08:53:22.515 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070963791999999998 2024-05-08T08:53:22.515 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.516 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.516 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:22.516 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557952+0000", 2024-05-08T08:53:22.516 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2082000000000004e-05 2024-05-08T08:53:22.516 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.516 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.516 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:22.516 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558065+0000", 2024-05-08T08:53:22.516 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000113143 2024-05-08T08:53:22.516 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:22.516 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:22.516 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:22.516 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.516 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.517 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1094 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11436032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:22.517 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:53:22.517 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 45.964653120999998, 2024-05-08T08:53:22.517 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071400507000000002, 2024-05-08T08:53:22.517 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:22.517 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:22.517 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:22.517 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:22.517 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:22.517 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:22.517 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1094 2024-05-08T08:53:22.517 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.517 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:22.517 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.518 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:22.518 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:53:22.518 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:22.518 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.518 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.518 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:22.518 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:53:22.518 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:22.518 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.518 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.518 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:22.518 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487940+0000", 2024-05-08T08:53:22.518 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001036868 2024-05-08T08:53:22.519 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.519 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.519 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:22.519 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487944+0000", 2024-05-08T08:53:22.519 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7069999999999998e-06 2024-05-08T08:53:22.519 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.519 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.519 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:22.519 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487951+0000", 2024-05-08T08:53:22.519 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2649999999999999e-06 2024-05-08T08:53:22.519 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.519 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.519 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:22.519 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487961+0000", 2024-05-08T08:53:22.520 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4110000000000002e-06 2024-05-08T08:53:22.520 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.520 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.520 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:22.520 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558117+0000", 2024-05-08T08:53:22.520 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070156662999999994 2024-05-08T08:53:22.520 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.520 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.520 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:22.520 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558214+0000", 2024-05-08T08:53:22.520 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6444e-05 2024-05-08T08:53:22.520 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.520 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.520 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:22.521 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558304+0000", 2024-05-08T08:53:22.521 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0148999999999994e-05 2024-05-08T08:53:22.521 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:22.521 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:22.521 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:22.521 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.521 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.521 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1096 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11476992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:22.521 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:53:22.521 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 45.963586589999998, 2024-05-08T08:53:22.521 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070480074000000004, 2024-05-08T08:53:22.521 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:22.521 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:22.521 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:22.522 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:22.522 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:22.522 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:22.522 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1096 2024-05-08T08:53:22.522 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.522 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:22.522 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.522 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:22.522 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:53:22.522 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:22.522 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.522 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.522 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:22.523 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:53:22.523 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:22.523 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.523 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.523 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:22.523 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:53:22.523 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3700000000000001e-07 2024-05-08T08:53:22.523 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.523 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.523 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:22.523 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487973+0000", 2024-05-08T08:53:22.523 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0870000000000001e-06 2024-05-08T08:53:22.523 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.523 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.524 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:22.524 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487977+0000", 2024-05-08T08:53:22.524 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0600000000000001e-06 2024-05-08T08:53:22.524 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.524 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.524 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:22.524 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487980+0000", 2024-05-08T08:53:22.524 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1089999999999998e-06 2024-05-08T08:53:22.524 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.524 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.524 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:22.524 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558338+0000", 2024-05-08T08:53:22.524 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070358233000000006 2024-05-08T08:53:22.524 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.525 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.525 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:22.525 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558392+0000", 2024-05-08T08:53:22.525 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4537999999999999e-05 2024-05-08T08:53:22.525 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.525 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.525 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:22.525 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558450+0000", 2024-05-08T08:53:22.525 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7710000000000001e-05 2024-05-08T08:53:22.525 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:22.525 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:22.525 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:22.525 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.525 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.526 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1099 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11517952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:22.526 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:53:22.526 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 45.963572714000001, 2024-05-08T08:53:22.526 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070720506000000002, 2024-05-08T08:53:22.526 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:22.526 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:22.526 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:22.526 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:22.526 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:22.526 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:22.526 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1099 2024-05-08T08:53:22.526 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.526 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:22.527 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.527 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:22.527 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:53:22.527 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:22.527 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.527 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.527 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:22.527 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:53:22.527 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:22.527 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.527 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.527 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:22.527 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:53:22.527 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3999999999999998e-07 2024-05-08T08:53:22.527 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.528 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.528 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:22.528 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487986+0000", 2024-05-08T08:53:22.528 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3519999999999999e-06 2024-05-08T08:53:22.528 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.528 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.528 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:22.528 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487989+0000", 2024-05-08T08:53:22.528 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.23e-06 2024-05-08T08:53:22.528 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.528 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.528 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:22.528 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487991+0000", 2024-05-08T08:53:22.528 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6350000000000002e-06 2024-05-08T08:53:22.529 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.529 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.529 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:22.529 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558489+0000", 2024-05-08T08:53:22.529 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070497924000000003 2024-05-08T08:53:22.529 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.529 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.529 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:22.529 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558583+0000", 2024-05-08T08:53:22.529 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3405000000000004e-05 2024-05-08T08:53:22.529 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.529 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.529 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:22.529 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558704+0000", 2024-05-08T08:53:22.530 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012171999999999999 2024-05-08T08:53:22.530 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:22.530 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:22.530 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:22.530 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.530 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.530 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1100 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11558912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:22.530 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:53:22.530 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 45.963559715000002, 2024-05-08T08:53:22.530 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070962266999999996, 2024-05-08T08:53:22.530 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:22.530 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:22.531 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:22.531 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:22.531 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:22.531 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:22.531 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1100 2024-05-08T08:53:22.531 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.531 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:22.531 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.531 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:22.531 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:53:22.531 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:22.531 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.531 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.531 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:22.532 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:53:22.532 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:22.532 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.532 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.532 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:22.532 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488430+0000", 2024-05-08T08:53:22.532 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000432676 2024-05-08T08:53:22.532 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.532 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.532 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:22.532 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488433+0000", 2024-05-08T08:53:22.532 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.45e-06 2024-05-08T08:53:22.532 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.532 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.532 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:22.533 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488442+0000", 2024-05-08T08:53:22.533 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.687e-06 2024-05-08T08:53:22.533 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.533 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.533 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:22.533 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488446+0000", 2024-05-08T08:53:22.533 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4939999999999997e-06 2024-05-08T08:53:22.533 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.533 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.533 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:22.533 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558756+0000", 2024-05-08T08:53:22.533 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070310051999999998 2024-05-08T08:53:22.533 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.534 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.534 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:22.534 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558844+0000", 2024-05-08T08:53:22.534 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7690999999999998e-05 2024-05-08T08:53:22.534 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.534 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.534 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:22.534 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558959+0000", 2024-05-08T08:53:22.534 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000115217 2024-05-08T08:53:22.534 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:22.534 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:22.534 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:22.534 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.535 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.535 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1103 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11599872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:22.535 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:53:22.535 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 45.963102040000003, 2024-05-08T08:53:22.535 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070777100999999995, 2024-05-08T08:53:22.535 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:22.535 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:22.535 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:22.535 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:22.535 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:22.535 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:22.535 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1103 2024-05-08T08:53:22.535 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.535 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:22.536 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.536 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:22.536 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:53:22.536 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:22.536 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.536 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.536 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:22.536 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:53:22.536 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:22.536 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.536 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.536 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:22.536 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488959+0000", 2024-05-08T08:53:22.537 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00050457200000000001 2024-05-08T08:53:22.537 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.537 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.537 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:22.537 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488969+0000", 2024-05-08T08:53:22.537 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.7159999999999998e-06 2024-05-08T08:53:22.537 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.537 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.537 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:22.537 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488981+0000", 2024-05-08T08:53:22.537 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2150000000000001e-05 2024-05-08T08:53:22.537 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.537 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.538 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:22.538 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488988+0000", 2024-05-08T08:53:22.538 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8759999999999999e-06 2024-05-08T08:53:22.538 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.538 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.538 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:22.538 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558992+0000", 2024-05-08T08:53:22.538 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070004358000000003 2024-05-08T08:53:22.538 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.538 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.538 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:22.539 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559093+0000", 2024-05-08T08:53:22.539 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000100299 2024-05-08T08:53:22.539 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.539 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.539 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:22.539 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559232+0000", 2024-05-08T08:53:22.539 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013913 2024-05-08T08:53:22.539 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:22.539 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:22.539 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:22.539 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.539 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.539 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1104 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11640832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:22.540 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:53:22.540 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 45.962557224999998, 2024-05-08T08:53:22.540 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070521947000000001, 2024-05-08T08:53:22.540 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:22.540 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:22.540 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:22.540 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:22.540 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:22.540 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:22.540 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1104 2024-05-08T08:53:22.540 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.540 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:22.540 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.541 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:22.541 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:53:22.541 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:22.541 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.541 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.541 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:22.541 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:53:22.541 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:22.541 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.541 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.541 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:22.541 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489700+0000", 2024-05-08T08:53:22.541 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00070067600000000001 2024-05-08T08:53:22.542 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.542 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.542 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:22.542 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489704+0000", 2024-05-08T08:53:22.542 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1640000000000003e-06 2024-05-08T08:53:22.542 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.542 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.542 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:22.542 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489716+0000", 2024-05-08T08:53:22.542 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1379000000000001e-05 2024-05-08T08:53:22.542 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.542 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.542 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:22.543 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489724+0000", 2024-05-08T08:53:22.543 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8949999999999998e-06 2024-05-08T08:53:22.543 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.543 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.543 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:22.543 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559278+0000", 2024-05-08T08:53:22.543 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.069554151999999994 2024-05-08T08:53:22.543 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.543 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.543 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:22.543 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559387+0000", 2024-05-08T08:53:22.543 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000109258 2024-05-08T08:53:22.543 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.544 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.544 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:22.544 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559521+0000", 2024-05-08T08:53:22.544 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000134423 2024-05-08T08:53:22.544 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:22.544 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:22.544 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:22.544 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.544 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.544 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:8636 3.17 3:e8a0a3e4:::rbd_header.11402b26385:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-05-08T08:53:22.544 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:54.150971+0000", 2024-05-08T08:53:22.544 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 28.300585230999999, 2024-05-08T08:53:22.544 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.217273233, 2024-05-08T08:53:22.545 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:22.545 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:22.545 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:22.545 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:22.545 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:22.545 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:22.545 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 8636 2024-05-08T08:53:22.545 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.545 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:22.545 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.545 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:22.545 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150971+0000", 2024-05-08T08:53:22.546 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:22.546 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.546 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.546 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:22.546 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150971+0000", 2024-05-08T08:53:22.546 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:22.546 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.546 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.546 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:22.546 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150972+0000", 2024-05-08T08:53:22.546 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.71e-07 2024-05-08T08:53:22.546 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.546 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.547 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:22.547 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150977+0000", 2024-05-08T08:53:22.547 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.152e-06 2024-05-08T08:53:22.547 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.547 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.547 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:22.547 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150990+0000", 2024-05-08T08:53:22.547 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2364999999999999e-05 2024-05-08T08:53:22.547 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.547 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.547 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:22.547 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150998+0000", 2024-05-08T08:53:22.547 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0739999999999993e-06 2024-05-08T08:53:22.547 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.548 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.548 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:22.548 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.151054+0000", 2024-05-08T08:53:22.548 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5949000000000003e-05 2024-05-08T08:53:22.548 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.548 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.548 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:53:22.548 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.151060+0000", 2024-05-08T08:53:22.548 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6120000000000004e-06 2024-05-08T08:53:22.548 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.548 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.548 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:22.548 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.367640+0000", 2024-05-08T08:53:22.548 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2165792739999999 2024-05-08T08:53:22.549 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.549 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.549 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:22.549 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.367839+0000", 2024-05-08T08:53:22.549 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000199185 2024-05-08T08:53:22.549 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.549 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.549 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:22.549 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.368245+0000", 2024-05-08T08:53:22.549 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00040575099999999998 2024-05-08T08:53:22.549 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:22.549 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:22.549 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:22.550 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:22.550 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:22.550 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:22.551 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:53:22.552 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:22.554+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:22.652 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:22.655+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:22.709 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:22.709 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:53:22.709 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:53:22.709 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:22.718 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:53:22.762 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:22.765+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:22.833 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:22.833 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:53:22.833 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:53:22.833 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:53:22.833 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:22.843 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:53:22.884 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:22.887+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:22.955 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:22.958+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:22.994 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:22.994 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:53:22.994 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:53:22.994 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:53:22.995 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.995 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:22471 3.4 3:26b512e1:::rbd_header.1140133d334a:head [notify cookie 139902587499760] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:22.995 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.123196+0000", 2024-05-08T08:53:22.995 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 9.8732319900000007, 2024-05-08T08:53:22.995 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.061461114999999997, 2024-05-08T08:53:22.995 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:22.995 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:22.995 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:22.995 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:22.995 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:22.995 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:22.995 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 22471 2024-05-08T08:53:22.996 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.996 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:22.996 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.996 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:22.996 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123196+0000", 2024-05-08T08:53:22.996 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:22.996 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.996 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.996 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:22.996 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123196+0000", 2024-05-08T08:53:22.996 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:22.996 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.996 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.996 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:22.996 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123196+0000", 2024-05-08T08:53:22.997 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.2800000000000003e-07 2024-05-08T08:53:22.997 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.997 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.997 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:22.997 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123198+0000", 2024-05-08T08:53:22.997 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4040000000000002e-06 2024-05-08T08:53:22.997 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.997 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.997 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:22.997 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123204+0000", 2024-05-08T08:53:22.997 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8749999999999997e-06 2024-05-08T08:53:22.997 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.997 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.997 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:22.998 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123209+0000", 2024-05-08T08:53:22.998 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1209999999999998e-06 2024-05-08T08:53:22.998 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.998 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.998 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:22.998 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.184473+0000", 2024-05-08T08:53:22.998 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.061263347000000003 2024-05-08T08:53:22.998 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.998 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.998 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:22.998 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.184542+0000", 2024-05-08T08:53:22.998 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9023000000000006e-05 2024-05-08T08:53:22.998 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.998 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.998 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:22.999 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.184657+0000", 2024-05-08T08:53:22.999 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000115017 2024-05-08T08:53:22.999 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:22.999 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:22.999 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:22.999 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:22.999 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:22.999 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:22508 3.4 3:200df2c8:::rbd_data.1140100c9703.0000000000000002:14 [read 1283072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:22.999 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.123216+0000", 2024-05-08T08:53:22.999 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 9.8732117529999996, 2024-05-08T08:53:22.999 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.061633735000000002, 2024-05-08T08:53:22.999 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:22.999 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:22.999 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:23.000 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:23.000 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:23.000 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:23.000 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 22508 2024-05-08T08:53:23.000 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.000 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:23.000 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.000 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:23.000 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123216+0000", 2024-05-08T08:53:23.000 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:23.000 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.000 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.000 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:23.000 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123216+0000", 2024-05-08T08:53:23.000 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:23.001 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.001 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.001 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:23.001 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123216+0000", 2024-05-08T08:53:23.001 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.39e-07 2024-05-08T08:53:23.001 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.001 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.001 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:23.001 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123219+0000", 2024-05-08T08:53:23.001 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.571e-06 2024-05-08T08:53:23.001 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.001 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.001 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:23.001 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123225+0000", 2024-05-08T08:53:23.001 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9819999999999996e-06 2024-05-08T08:53:23.001 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.002 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.002 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:23.002 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123232+0000", 2024-05-08T08:53:23.002 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8669999999999996e-06 2024-05-08T08:53:23.002 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.002 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.002 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:23.002 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.184687+0000", 2024-05-08T08:53:23.002 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.061455009999999997 2024-05-08T08:53:23.002 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.002 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.002 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:23.002 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.184763+0000", 2024-05-08T08:53:23.003 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.6385999999999997e-05 2024-05-08T08:53:23.003 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.003 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.003 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:23.003 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.184850+0000", 2024-05-08T08:53:23.003 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.6479999999999999e-05 2024-05-08T08:53:23.003 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:23.003 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:23.003 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:23.003 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.003 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.003 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:22509 3.4 3:200df2c8:::rbd_data.1140100c9703.0000000000000002:14 [read 1292288~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:23.003 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.123239+0000", 2024-05-08T08:53:23.003 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 9.8731890989999993, 2024-05-08T08:53:23.003 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.061804586000000002, 2024-05-08T08:53:23.004 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:23.004 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:23.004 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:23.004 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:23.004 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:23.004 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:23.004 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 22509 2024-05-08T08:53:23.004 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.004 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:23.004 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.004 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:23.004 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123239+0000", 2024-05-08T08:53:23.004 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:23.004 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.004 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.005 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:23.005 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123239+0000", 2024-05-08T08:53:23.005 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:23.005 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.005 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.005 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:23.005 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123239+0000", 2024-05-08T08:53:23.005 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4000000000000002e-07 2024-05-08T08:53:23.005 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.005 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.005 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:23.005 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123241+0000", 2024-05-08T08:53:23.005 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3209999999999999e-06 2024-05-08T08:53:23.005 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.005 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.006 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:23.006 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123249+0000", 2024-05-08T08:53:23.006 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.96e-06 2024-05-08T08:53:23.006 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.006 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.006 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:23.006 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123254+0000", 2024-05-08T08:53:23.006 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7280000000000003e-06 2024-05-08T08:53:23.006 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.006 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.006 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:23.006 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.184879+0000", 2024-05-08T08:53:23.006 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.061625231000000003 2024-05-08T08:53:23.006 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.007 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.007 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:23.007 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.184958+0000", 2024-05-08T08:53:23.007 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8397000000000002e-05 2024-05-08T08:53:23.007 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.007 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.007 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:23.007 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.185043+0000", 2024-05-08T08:53:23.007 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.5408999999999995e-05 2024-05-08T08:53:23.007 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:23.007 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:23.007 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:23.007 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.007 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.007 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:22510 3.4 3:200df2c8:::rbd_data.1140100c9703.0000000000000002:14 [read 1301504~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:23.007 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.123263+0000", 2024-05-08T08:53:23.008 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 9.8731647599999999, 2024-05-08T08:53:23.008 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.061964539999999999, 2024-05-08T08:53:23.008 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:23.008 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:23.008 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:23.008 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:23.008 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:23.008 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:23.008 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 22510 2024-05-08T08:53:23.008 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.008 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:23.008 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.008 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:23.008 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123263+0000", 2024-05-08T08:53:23.008 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:23.009 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.009 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.009 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:23.009 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123263+0000", 2024-05-08T08:53:23.009 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:23.009 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.009 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.009 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:23.009 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123263+0000", 2024-05-08T08:53:23.009 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1100000000000001e-07 2024-05-08T08:53:23.009 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.009 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.009 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:23.009 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123266+0000", 2024-05-08T08:53:23.010 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7970000000000001e-06 2024-05-08T08:53:23.010 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.010 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.010 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:23.010 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123272+0000", 2024-05-08T08:53:23.010 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8819999999999998e-06 2024-05-08T08:53:23.010 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.010 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.010 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:23.010 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123277+0000", 2024-05-08T08:53:23.010 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1209999999999998e-06 2024-05-08T08:53:23.010 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.010 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.010 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:23.011 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.185074+0000", 2024-05-08T08:53:23.011 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.061796901000000001 2024-05-08T08:53:23.011 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.011 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.011 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:23.011 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.185144+0000", 2024-05-08T08:53:23.011 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0265999999999994e-05 2024-05-08T08:53:23.011 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.011 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.011 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:23.011 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.185228+0000", 2024-05-08T08:53:23.011 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3162000000000001e-05 2024-05-08T08:53:23.011 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:23.011 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:23.011 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:23.012 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.012 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.012 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:22511 3.4 3:200df2c8:::rbd_data.1140100c9703.0000000000000002:14 [read 1312768~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:23.012 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.123284+0000", 2024-05-08T08:53:23.012 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 9.8731433699999993, 2024-05-08T08:53:23.012 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.062125073000000003, 2024-05-08T08:53:23.012 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:23.012 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:23.012 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:23.012 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:23.012 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:23.012 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:23.012 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 22511 2024-05-08T08:53:23.012 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.012 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:23.013 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.013 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:23.013 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123284+0000", 2024-05-08T08:53:23.013 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:23.013 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.013 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.013 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:23.013 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123284+0000", 2024-05-08T08:53:23.013 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:23.013 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.013 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.013 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:23.013 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123285+0000", 2024-05-08T08:53:23.013 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7099999999999997e-07 2024-05-08T08:53:23.013 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.014 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.014 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:23.014 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123287+0000", 2024-05-08T08:53:23.014 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2859999999999998e-06 2024-05-08T08:53:23.014 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.014 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.014 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:23.014 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123293+0000", 2024-05-08T08:53:23.014 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0240000000000003e-06 2024-05-08T08:53:23.014 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.014 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.014 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:23.014 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123298+0000", 2024-05-08T08:53:23.014 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1279999999999999e-06 2024-05-08T08:53:23.014 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.015 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.015 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:23.015 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.185257+0000", 2024-05-08T08:53:23.015 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.061958930000000002 2024-05-08T08:53:23.015 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.015 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.015 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:23.015 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.185326+0000", 2024-05-08T08:53:23.015 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8817000000000002e-05 2024-05-08T08:53:23.015 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.015 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.015 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:23.015 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.185409+0000", 2024-05-08T08:53:23.015 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3516999999999993e-05 2024-05-08T08:53:23.015 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:23.016 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:23.016 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:23.016 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.016 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.016 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:22512 3.4 3:200df2c8:::rbd_data.1140100c9703.0000000000000002:14 [read 1321984~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:23.016 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.123305+0000", 2024-05-08T08:53:23.016 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 9.8731226519999993, 2024-05-08T08:53:23.016 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.062288156999999997, 2024-05-08T08:53:23.016 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:23.016 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:23.016 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:23.016 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:23.016 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:23.016 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:23.016 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 22512 2024-05-08T08:53:23.017 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.017 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:23.017 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.017 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:23.017 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123305+0000", 2024-05-08T08:53:23.017 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:23.017 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.017 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.017 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:23.017 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123305+0000", 2024-05-08T08:53:23.017 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:23.017 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.017 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.017 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:23.017 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123305+0000", 2024-05-08T08:53:23.017 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6600000000000002e-07 2024-05-08T08:53:23.018 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.018 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.018 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:23.018 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123308+0000", 2024-05-08T08:53:23.018 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2450000000000001e-06 2024-05-08T08:53:23.018 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.018 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.018 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:23.018 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123314+0000", 2024-05-08T08:53:23.018 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2580000000000001e-06 2024-05-08T08:53:23.018 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.018 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.018 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:23.018 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123319+0000", 2024-05-08T08:53:23.018 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6009999999999997e-06 2024-05-08T08:53:23.019 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.019 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.019 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:23.019 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.185440+0000", 2024-05-08T08:53:23.019 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.062121434000000003 2024-05-08T08:53:23.019 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.019 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.019 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:23.019 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.185511+0000", 2024-05-08T08:53:23.019 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1478999999999998e-05 2024-05-08T08:53:23.019 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.019 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.019 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:23.019 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.185593+0000", 2024-05-08T08:53:23.019 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1773999999999993e-05 2024-05-08T08:53:23.019 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:23.020 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:23.020 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:23.020 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.020 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.020 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:22513 3.4 3:200df2c8:::rbd_data.1140100c9703.0000000000000002:14 [read 1331200~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:23.020 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.123326+0000", 2024-05-08T08:53:23.020 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 9.8731021030000008, 2024-05-08T08:53:23.020 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.062453781, 2024-05-08T08:53:23.020 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:23.020 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:23.020 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:23.020 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:23.020 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:23.020 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:23.020 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 22513 2024-05-08T08:53:23.020 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.021 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:23.021 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.021 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:23.021 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123326+0000", 2024-05-08T08:53:23.021 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:23.021 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.021 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.021 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:23.021 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123326+0000", 2024-05-08T08:53:23.021 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:23.021 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.021 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.021 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:23.021 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123326+0000", 2024-05-08T08:53:23.022 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.77e-07 2024-05-08T08:53:23.022 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.022 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.022 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:23.022 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123328+0000", 2024-05-08T08:53:23.022 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0940000000000002e-06 2024-05-08T08:53:23.022 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.022 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.022 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:23.022 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123334+0000", 2024-05-08T08:53:23.022 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6629999999999998e-06 2024-05-08T08:53:23.022 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.022 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.022 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:23.023 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123339+0000", 2024-05-08T08:53:23.023 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7500000000000003e-06 2024-05-08T08:53:23.023 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.023 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.023 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:23.023 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.185625+0000", 2024-05-08T08:53:23.023 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.062286672000000001 2024-05-08T08:53:23.023 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.023 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.023 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:23.023 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.185697+0000", 2024-05-08T08:53:23.023 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1853000000000005e-05 2024-05-08T08:53:23.023 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.023 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.023 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:23.024 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.185779+0000", 2024-05-08T08:53:23.024 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.2372000000000003e-05 2024-05-08T08:53:23.024 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:23.024 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:23.024 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:23.024 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.024 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.024 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:22514 3.4 3:200df2c8:::rbd_data.1140100c9703.0000000000000002:14 [read 1340416~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:23.024 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.123345+0000", 2024-05-08T08:53:23.024 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 9.8730829, 2024-05-08T08:53:23.024 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.062624377999999994, 2024-05-08T08:53:23.024 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:23.024 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:23.024 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:23.025 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:23.025 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:23.025 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:23.025 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 22514 2024-05-08T08:53:23.025 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.025 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:23.025 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.025 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:23.025 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123345+0000", 2024-05-08T08:53:23.025 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:23.025 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.025 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.025 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:23.025 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123345+0000", 2024-05-08T08:53:23.025 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:23.025 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.026 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.026 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:23.026 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123345+0000", 2024-05-08T08:53:23.026 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4999999999999998e-07 2024-05-08T08:53:23.026 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.026 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.026 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:23.026 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123347+0000", 2024-05-08T08:53:23.026 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0250000000000001e-06 2024-05-08T08:53:23.026 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.026 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.026 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:23.026 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123353+0000", 2024-05-08T08:53:23.026 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0159999999999997e-06 2024-05-08T08:53:23.026 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.027 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.027 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:23.027 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123358+0000", 2024-05-08T08:53:23.027 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4750000000000004e-06 2024-05-08T08:53:23.027 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.027 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.027 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:23.027 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.185809+0000", 2024-05-08T08:53:23.027 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.062451602000000002 2024-05-08T08:53:23.027 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.027 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.027 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:23.027 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.185882+0000", 2024-05-08T08:53:23.027 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2259000000000001e-05 2024-05-08T08:53:23.027 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.027 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.028 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:23.028 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.185969+0000", 2024-05-08T08:53:23.028 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7651000000000005e-05 2024-05-08T08:53:23.028 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:23.028 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:23.028 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:23.028 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.028 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.028 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:22515 3.4 3:200df2c8:::rbd_data.1140100c9703.0000000000000002:14 [read 1351680~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:23.028 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.123364+0000", 2024-05-08T08:53:23.028 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 9.8730636450000002, 2024-05-08T08:53:23.028 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.062847945000000002, 2024-05-08T08:53:23.028 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:23.028 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:23.028 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:23.028 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:23.029 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:23.029 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:23.029 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 22515 2024-05-08T08:53:23.029 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.029 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:23.029 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.029 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:23.029 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123364+0000", 2024-05-08T08:53:23.029 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:23.029 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.029 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.029 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:23.029 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123364+0000", 2024-05-08T08:53:23.029 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:23.029 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.029 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.030 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:23.030 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123364+0000", 2024-05-08T08:53:23.030 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.2000000000000001e-07 2024-05-08T08:53:23.030 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.030 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.030 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:23.030 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123372+0000", 2024-05-08T08:53:23.030 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8650000000000001e-06 2024-05-08T08:53:23.030 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.030 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.030 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:23.030 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123379+0000", 2024-05-08T08:53:23.030 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.64e-06 2024-05-08T08:53:23.030 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.030 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.031 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:23.031 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123384+0000", 2024-05-08T08:53:23.031 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9830000000000004e-06 2024-05-08T08:53:23.031 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.031 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.031 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:23.031 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.185999+0000", 2024-05-08T08:53:23.031 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.062615547999999993 2024-05-08T08:53:23.031 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.031 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.031 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:23.031 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.186091+0000", 2024-05-08T08:53:23.031 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1576999999999994e-05 2024-05-08T08:53:23.031 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.031 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.031 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:23.031 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.186212+0000", 2024-05-08T08:53:23.032 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000121012 2024-05-08T08:53:23.032 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:23.032 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:23.032 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:23.032 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.032 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.032 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:22516 3.4 3:200df2c8:::rbd_data.1140100c9703.0000000000000002:14 [read 1360896~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:23.032 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.123391+0000", 2024-05-08T08:53:23.032 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 9.8730367010000002, 2024-05-08T08:53:23.032 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.063010391999999998, 2024-05-08T08:53:23.032 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:23.032 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:23.032 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:23.032 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:23.032 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:23.032 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:23.033 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 22516 2024-05-08T08:53:23.033 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.033 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:23.033 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.033 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:23.033 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123391+0000", 2024-05-08T08:53:23.033 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:23.033 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.033 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.033 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:23.033 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123391+0000", 2024-05-08T08:53:23.033 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:23.033 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.033 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.034 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:23.034 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123391+0000", 2024-05-08T08:53:23.034 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0699999999999998e-07 2024-05-08T08:53:23.034 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.034 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.034 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:23.034 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123394+0000", 2024-05-08T08:53:23.034 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0930000000000001e-06 2024-05-08T08:53:23.034 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.034 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.034 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:23.034 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123399+0000", 2024-05-08T08:53:23.034 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7980000000000002e-06 2024-05-08T08:53:23.034 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.035 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.035 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:23.035 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123404+0000", 2024-05-08T08:53:23.035 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7169999999999999e-06 2024-05-08T08:53:23.035 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.035 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.035 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:23.035 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.186250+0000", 2024-05-08T08:53:23.035 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.062845711999999998 2024-05-08T08:53:23.035 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.035 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.035 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:23.035 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.186319+0000", 2024-05-08T08:53:23.035 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.97e-05 2024-05-08T08:53:23.035 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.035 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.035 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:23.036 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.186401+0000", 2024-05-08T08:53:23.036 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1965000000000006e-05 2024-05-08T08:53:23.036 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:23.036 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:23.036 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:23.036 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.036 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.036 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:22517 3.4 3:200df2c8:::rbd_data.1140100c9703.0000000000000002:14 [read 1370112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:23.036 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.123412+0000", 2024-05-08T08:53:23.036 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 9.8730161219999992, 2024-05-08T08:53:23.036 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.063168632000000002, 2024-05-08T08:53:23.036 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:23.036 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:23.036 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:23.037 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:23.037 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:23.037 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:23.037 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 22517 2024-05-08T08:53:23.037 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.037 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:23.037 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.037 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:23.037 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123412+0000", 2024-05-08T08:53:23.037 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:23.037 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.037 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.037 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:23.037 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123412+0000", 2024-05-08T08:53:23.037 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:23.037 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.038 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.038 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:23.038 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123412+0000", 2024-05-08T08:53:23.038 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8500000000000002e-07 2024-05-08T08:53:23.038 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.038 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.038 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:23.038 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123414+0000", 2024-05-08T08:53:23.038 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0930000000000001e-06 2024-05-08T08:53:23.038 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.038 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.038 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:23.038 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123421+0000", 2024-05-08T08:53:23.038 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9999999999999999e-06 2024-05-08T08:53:23.038 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.039 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.039 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:23.039 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123426+0000", 2024-05-08T08:53:23.039 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.95e-06 2024-05-08T08:53:23.039 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.039 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.039 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:23.039 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.186429+0000", 2024-05-08T08:53:23.039 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.063003346000000002 2024-05-08T08:53:23.039 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.039 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.039 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:23.039 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.186501+0000", 2024-05-08T08:53:23.039 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1575000000000005e-05 2024-05-08T08:53:23.040 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.040 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.040 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:23.040 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.186580+0000", 2024-05-08T08:53:23.040 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9283000000000006e-05 2024-05-08T08:53:23.040 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:23.040 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:23.040 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:23.040 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.040 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.040 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:22518 3.4 3:200df2c8:::rbd_data.1140100c9703.0000000000000002:14 [read 1381376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:23.040 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.123432+0000", 2024-05-08T08:53:23.040 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 9.872995242, 2024-05-08T08:53:23.040 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.063331088999999993, 2024-05-08T08:53:23.041 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:23.041 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:23.041 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:23.041 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:23.041 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:23.041 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:23.041 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 22518 2024-05-08T08:53:23.041 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.041 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:23.041 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.041 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:23.041 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123432+0000", 2024-05-08T08:53:23.041 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:23.041 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.041 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.041 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:23.042 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123432+0000", 2024-05-08T08:53:23.042 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:23.042 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.042 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.042 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:23.042 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123433+0000", 2024-05-08T08:53:23.042 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4099999999999999e-07 2024-05-08T08:53:23.042 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.042 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.042 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:23.042 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123435+0000", 2024-05-08T08:53:23.042 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9379999999999999e-06 2024-05-08T08:53:23.042 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.042 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.043 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:23.043 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123441+0000", 2024-05-08T08:53:23.043 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9290000000000003e-06 2024-05-08T08:53:23.043 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.043 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.043 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:23.043 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123447+0000", 2024-05-08T08:53:23.043 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8279999999999999e-06 2024-05-08T08:53:23.043 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.043 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.043 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:23.043 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.186609+0000", 2024-05-08T08:53:23.043 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.063162067000000002 2024-05-08T08:53:23.043 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.043 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.044 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:23.044 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.186679+0000", 2024-05-08T08:53:23.044 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9721999999999994e-05 2024-05-08T08:53:23.044 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.044 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.044 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:23.044 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.186764+0000", 2024-05-08T08:53:23.044 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.5063999999999997e-05 2024-05-08T08:53:23.044 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:23.044 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:23.044 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:23.044 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.044 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.044 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:22519 3.4 3:200df2c8:::rbd_data.1140100c9703.0000000000000002:14 [read 1390592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:23.045 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.123453+0000", 2024-05-08T08:53:23.045 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 9.8729743160000005, 2024-05-08T08:53:23.045 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.063507447999999994, 2024-05-08T08:53:23.045 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:23.045 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:23.045 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:23.045 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:23.045 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:23.045 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:23.045 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 22519 2024-05-08T08:53:23.045 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.045 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:23.045 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.045 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:23.045 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123453+0000", 2024-05-08T08:53:23.046 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:23.046 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.046 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.046 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:23.046 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123453+0000", 2024-05-08T08:53:23.046 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:23.046 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.046 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.046 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:23.046 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123454+0000", 2024-05-08T08:53:23.046 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5100000000000001e-07 2024-05-08T08:53:23.046 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.046 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.046 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:23.046 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123456+0000", 2024-05-08T08:53:23.047 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0789999999999999e-06 2024-05-08T08:53:23.047 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.047 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.047 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:23.047 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123462+0000", 2024-05-08T08:53:23.047 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1990000000000003e-06 2024-05-08T08:53:23.047 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.047 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.047 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:23.047 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123466+0000", 2024-05-08T08:53:23.047 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4449999999999998e-06 2024-05-08T08:53:23.047 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.047 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.047 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:23.047 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.186793+0000", 2024-05-08T08:53:23.047 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.063326128999999995 2024-05-08T08:53:23.047 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.048 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.048 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:23.048 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.186868+0000", 2024-05-08T08:53:23.048 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5475000000000005e-05 2024-05-08T08:53:23.048 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.048 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.048 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:23.048 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.186961+0000", 2024-05-08T08:53:23.048 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2769999999999994e-05 2024-05-08T08:53:23.048 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:23.048 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:23.048 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:23.048 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.048 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.048 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:22520 3.4 3:200df2c8:::rbd_data.1140100c9703.0000000000000002:14 [read 1399808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:23.048 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.123473+0000", 2024-05-08T08:53:23.048 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 9.8729546779999993, 2024-05-08T08:53:23.049 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.063690254000000002, 2024-05-08T08:53:23.049 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:23.049 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:23.049 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:23.049 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:23.049 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:23.049 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:23.049 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 22520 2024-05-08T08:53:23.049 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.049 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:23.049 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.049 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:23.049 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123473+0000", 2024-05-08T08:53:23.049 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:23.049 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.049 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.050 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:23.050 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123473+0000", 2024-05-08T08:53:23.050 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:23.050 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.050 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.050 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:23.050 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123474+0000", 2024-05-08T08:53:23.050 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0100000000000005e-07 2024-05-08T08:53:23.050 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.050 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.050 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:23.050 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123476+0000", 2024-05-08T08:53:23.050 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0420000000000001e-06 2024-05-08T08:53:23.050 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.050 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.050 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:23.051 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123482+0000", 2024-05-08T08:53:23.051 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2670000000000003e-06 2024-05-08T08:53:23.051 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.051 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.051 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:23.051 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123487+0000", 2024-05-08T08:53:23.051 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6500000000000004e-06 2024-05-08T08:53:23.051 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.051 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.051 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:23.051 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.186990+0000", 2024-05-08T08:53:23.051 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.063503967999999994 2024-05-08T08:53:23.051 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.051 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.051 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:23.051 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.187077+0000", 2024-05-08T08:53:23.052 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.6415000000000005e-05 2024-05-08T08:53:23.052 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.052 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.052 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:23.052 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.187163+0000", 2024-05-08T08:53:23.052 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.6410999999999996e-05 2024-05-08T08:53:23.052 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:23.052 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:23.052 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:23.052 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.052 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.053 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:22521 3.4 3:200df2c8:::rbd_data.1140100c9703.0000000000000002:14 [read 1411072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:23.053 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.123493+0000", 2024-05-08T08:53:23.053 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 9.8729342340000006, 2024-05-08T08:53:23.053 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.063877505000000001, 2024-05-08T08:53:23.053 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:23.053 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:23.053 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:23.053 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:23.053 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:23.053 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:23.053 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 22521 2024-05-08T08:53:23.053 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.053 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:23.053 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.054 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:23.054 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123493+0000", 2024-05-08T08:53:23.054 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:23.054 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.054 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.054 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:23.054 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123493+0000", 2024-05-08T08:53:23.054 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:23.054 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.054 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.054 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:23.054 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123494+0000", 2024-05-08T08:53:23.054 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8700000000000001e-07 2024-05-08T08:53:23.054 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.054 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.055 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:23.055 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123496+0000", 2024-05-08T08:53:23.055 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.973e-06 2024-05-08T08:53:23.055 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.055 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.055 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:23.055 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123502+0000", 2024-05-08T08:53:23.055 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2979999999999997e-06 2024-05-08T08:53:23.055 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.055 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.055 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:23.055 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123507+0000", 2024-05-08T08:53:23.055 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9949999999999996e-06 2024-05-08T08:53:23.055 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.055 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.055 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:23.056 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.187194+0000", 2024-05-08T08:53:23.056 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.063686417999999995 2024-05-08T08:53:23.056 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.056 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.056 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:23.056 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.187279+0000", 2024-05-08T08:53:23.056 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.5771000000000003e-05 2024-05-08T08:53:23.056 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.056 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.056 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:23.056 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.187371+0000", 2024-05-08T08:53:23.056 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1663000000000007e-05 2024-05-08T08:53:23.056 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:23.056 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:23.056 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:23.056 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.057 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.057 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:22522 3.4 3:200df2c8:::rbd_data.1140100c9703.0000000000000002:14 [read 1420288~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:23.057 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.123514+0000", 2024-05-08T08:53:23.057 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 9.8729132830000008, 2024-05-08T08:53:23.057 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.064058437999999995, 2024-05-08T08:53:23.057 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:23.057 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:23.057 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:23.057 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:23.057 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:23.057 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:23.057 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 22522 2024-05-08T08:53:23.057 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.057 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:23.058 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.058 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:23.058 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123514+0000", 2024-05-08T08:53:23.058 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:23.058 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.058 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.058 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:23.058 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123514+0000", 2024-05-08T08:53:23.058 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:23.058 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.058 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.058 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:23.058 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123515+0000", 2024-05-08T08:53:23.058 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5199999999999998e-07 2024-05-08T08:53:23.058 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.059 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.059 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:23.059 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123517+0000", 2024-05-08T08:53:23.059 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1330000000000002e-06 2024-05-08T08:53:23.059 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.059 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.059 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:23.059 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123523+0000", 2024-05-08T08:53:23.059 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0950000000000001e-06 2024-05-08T08:53:23.059 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.059 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.059 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:23.059 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123528+0000", 2024-05-08T08:53:23.059 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5299999999999998e-06 2024-05-08T08:53:23.060 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.060 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.060 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:23.060 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.187401+0000", 2024-05-08T08:53:23.060 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.063873616999999994 2024-05-08T08:53:23.060 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.060 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.060 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:23.060 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.187477+0000", 2024-05-08T08:53:23.060 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5740000000000003e-05 2024-05-08T08:53:23.060 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.060 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.060 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:23.060 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.187573+0000", 2024-05-08T08:53:23.061 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5971000000000007e-05 2024-05-08T08:53:23.061 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:23.061 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:23.061 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:23.061 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.061 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.061 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:22523 3.4 3:200df2c8:::rbd_data.1140100c9703.0000000000000002:14 [read 1429504~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:23.061 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.123533+0000", 2024-05-08T08:53:23.061 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 9.8728942330000002, 2024-05-08T08:53:23.061 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.064248632999999999, 2024-05-08T08:53:23.061 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:23.061 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:23.061 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:23.061 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:23.061 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:23.061 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:23.062 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 22523 2024-05-08T08:53:23.062 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.062 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:23.062 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.062 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:23.062 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123533+0000", 2024-05-08T08:53:23.062 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:23.062 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.062 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.062 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:23.062 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123533+0000", 2024-05-08T08:53:23.062 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:23.062 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.062 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.062 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:23.063 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123534+0000", 2024-05-08T08:53:23.063 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.39e-07 2024-05-08T08:53:23.063 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.063 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.063 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:23.063 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123536+0000", 2024-05-08T08:53:23.063 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.951e-06 2024-05-08T08:53:23.063 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.063 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.063 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:23.063 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123542+0000", 2024-05-08T08:53:23.063 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.801e-06 2024-05-08T08:53:23.063 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.063 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.063 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:23.064 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123546+0000", 2024-05-08T08:53:23.064 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.454e-06 2024-05-08T08:53:23.064 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.064 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.064 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:23.064 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.187616+0000", 2024-05-08T08:53:23.064 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.064069599000000005 2024-05-08T08:53:23.064 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.064 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.064 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:23.064 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.187688+0000", 2024-05-08T08:53:23.064 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2557000000000003e-05 2024-05-08T08:53:23.064 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.064 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.064 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:23.065 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.187782+0000", 2024-05-08T08:53:23.065 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3931999999999994e-05 2024-05-08T08:53:23.065 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:23.065 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:23.065 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:23.065 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.065 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.065 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:22524 3.4 3:200df2c8:::rbd_data.1140100c9703.0000000000000002:14 [read 1438720~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:23.065 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.123552+0000", 2024-05-08T08:53:23.065 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 9.8728759119999996, 2024-05-08T08:53:23.065 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.064451531000000006, 2024-05-08T08:53:23.065 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:23.065 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:23.066 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:23.066 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:23.066 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:23.066 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:23.066 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 22524 2024-05-08T08:53:23.066 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.066 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:23.066 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.066 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:23.066 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123552+0000", 2024-05-08T08:53:23.066 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:23.066 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.066 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.066 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:23.066 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123552+0000", 2024-05-08T08:53:23.067 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:23.067 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.067 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.067 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:23.067 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123552+0000", 2024-05-08T08:53:23.067 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9400000000000001e-07 2024-05-08T08:53:23.067 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.067 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.067 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:23.067 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123554+0000", 2024-05-08T08:53:23.067 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.945e-06 2024-05-08T08:53:23.067 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.067 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.067 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:23.068 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123560+0000", 2024-05-08T08:53:23.068 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6999999999999996e-06 2024-05-08T08:53:23.068 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.068 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.068 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:23.068 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.123564+0000", 2024-05-08T08:53:23.068 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5360000000000003e-06 2024-05-08T08:53:23.068 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.068 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.068 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:23.068 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.187824+0000", 2024-05-08T08:53:23.068 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.064260203000000002 2024-05-08T08:53:23.068 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.068 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.068 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:23.069 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.187896+0000", 2024-05-08T08:53:23.069 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1104000000000002e-05 2024-05-08T08:53:23.069 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.069 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.069 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:23.069 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.188003+0000", 2024-05-08T08:53:23.069 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000107749 2024-05-08T08:53:23.069 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:23.069 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:23.069 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:23.069 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.069 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.069 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:22780 3.4 3:200df2c8:::rbd_data.1140100c9703.0000000000000002:14 [read 1430528~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:23.069 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.130416+0000", 2024-05-08T08:53:23.069 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 9.8660115299999998, 2024-05-08T08:53:23.069 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.061508963, 2024-05-08T08:53:23.070 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:23.070 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:23.070 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:23.070 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:23.070 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:23.070 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:23.070 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 22780 2024-05-08T08:53:23.070 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.070 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:23.070 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.070 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:23.070 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.130416+0000", 2024-05-08T08:53:23.070 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:23.070 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.070 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.070 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:23.071 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.130416+0000", 2024-05-08T08:53:23.071 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:23.071 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.071 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.071 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:23.071 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.130417+0000", 2024-05-08T08:53:23.071 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6300000000000001e-07 2024-05-08T08:53:23.071 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.071 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.071 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:23.071 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.130419+0000", 2024-05-08T08:53:23.071 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0059999999999999e-06 2024-05-08T08:53:23.071 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.071 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.071 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:23.072 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.130425+0000", 2024-05-08T08:53:23.072 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0399999999999998e-06 2024-05-08T08:53:23.072 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.072 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.072 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:23.072 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.130430+0000", 2024-05-08T08:53:23.072 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1850000000000003e-06 2024-05-08T08:53:23.072 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.072 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.072 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:23.072 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.191770+0000", 2024-05-08T08:53:23.072 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.061340266999999997 2024-05-08T08:53:23.072 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.072 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.072 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:23.073 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.191839+0000", 2024-05-08T08:53:23.073 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8520000000000001e-05 2024-05-08T08:53:23.073 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.073 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.073 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:23.073 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.191925+0000", 2024-05-08T08:53:23.073 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.6582000000000006e-05 2024-05-08T08:53:23.073 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:23.074 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:23.074 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:23.074 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.074 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.074 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4949.0:22781 3.4 3:200df2c8:::rbd_data.1140100c9703.0000000000000002:14 [read 1439744~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T08:53:23.074 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:13.130436+0000", 2024-05-08T08:53:23.074 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 9.8659913849999992, 2024-05-08T08:53:23.074 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.061768243, 2024-05-08T08:53:23.074 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:23.074 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:23.074 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:23.074 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:23.074 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4949", 2024-05-08T08:53:23.075 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1840274759", 2024-05-08T08:53:23.075 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 22781 2024-05-08T08:53:23.075 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.075 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:23.075 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.075 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:23.075 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.130436+0000", 2024-05-08T08:53:23.075 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:23.075 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.075 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.075 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:23.075 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.130436+0000", 2024-05-08T08:53:23.075 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:23.075 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.076 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.076 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:23.076 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.130437+0000", 2024-05-08T08:53:23.076 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6100000000000002e-07 2024-05-08T08:53:23.076 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.076 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.076 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:23.076 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.130444+0000", 2024-05-08T08:53:23.076 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.6629999999999993e-06 2024-05-08T08:53:23.076 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.076 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.076 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:23.076 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.130451+0000", 2024-05-08T08:53:23.076 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.007e-06 2024-05-08T08:53:23.076 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.077 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.077 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:23.077 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.130457+0000", 2024-05-08T08:53:23.077 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1719999999999998e-06 2024-05-08T08:53:23.077 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.077 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.077 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:23.077 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.191964+0000", 2024-05-08T08:53:23.077 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.061507305999999998 2024-05-08T08:53:23.077 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.077 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.077 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:23.077 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.192045+0000", 2024-05-08T08:53:23.077 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1347999999999994e-05 2024-05-08T08:53:23.077 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.078 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.078 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:23.078 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:13.192205+0000", 2024-05-08T08:53:23.078 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00015938599999999999 2024-05-08T08:53:23.078 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:23.078 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:23.078 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:23.078 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:23.078 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:23.078 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:23.080 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:23.082+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:23.080 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:53:23.181 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:53:23.184+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:53:23.192 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:23.192 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:53:23.192 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:53:23.192 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:23.200 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:53:23.282 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:23.285+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:23.350 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:23.350 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:53:23.350 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:53:23.351 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:53:23.351 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:23.359 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:53:23.375 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-05-08T08:53:23.383 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:23.386+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:23.475 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:23.475 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:53:23.475 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:53:23.475 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:53:23.475 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.475 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:437 3.17 3:ea93a512:::rbd_data.1140100c9703.0000000000000077:head [set-alloc-hint object_size 2097152 write_size 2097152,write 1170432~1024] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e63)", 2024-05-08T08:53:23.475 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:08.390804+0000", 2024-05-08T08:53:23.475 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 15.085488023, 2024-05-08T08:53:23.476 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0406068690000001, 2024-05-08T08:53:23.476 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:23.476 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:23.476 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:53:23.476 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:23.476 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:23.476 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:23.476 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 437 2024-05-08T08:53:23.476 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.476 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:23.476 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.476 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:23.476 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390804+0000", 2024-05-08T08:53:23.476 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:23.476 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.477 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.477 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:23.477 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390804+0000", 2024-05-08T08:53:23.477 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:23.477 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.477 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.477 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:23.477 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390806+0000", 2024-05-08T08:53:23.477 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.666e-06 2024-05-08T08:53:23.477 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.477 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.477 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:23.477 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390818+0000", 2024-05-08T08:53:23.477 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2187999999999999e-05 2024-05-08T08:53:23.478 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.478 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.478 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:23.478 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390834+0000", 2024-05-08T08:53:23.478 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6191000000000001e-05 2024-05-08T08:53:23.478 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.478 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.478 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:23.478 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390845+0000", 2024-05-08T08:53:23.478 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0831e-05 2024-05-08T08:53:23.478 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.478 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.478 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:23.478 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390922+0000", 2024-05-08T08:53:23.478 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7253e-05 2024-05-08T08:53:23.479 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.479 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.479 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op must wait for map", 2024-05-08T08:53:23.479 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390930+0000", 2024-05-08T08:53:23.479 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8259999999999993e-06 2024-05-08T08:53:23.479 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.479 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.479 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:23.479 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392155+0000", 2024-05-08T08:53:23.479 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0012256649999999999 2024-05-08T08:53:23.479 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.479 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.479 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:53:23.479 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392165+0000", 2024-05-08T08:53:23.479 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5799999999999998e-06 2024-05-08T08:53:23.479 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.480 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.480 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:23.480 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.429596+0000", 2024-05-08T08:53:23.480 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0374313639999999 2024-05-08T08:53:23.480 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.480 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.480 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:23.480 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.429728+0000", 2024-05-08T08:53:23.480 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000131552 2024-05-08T08:53:23.480 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.480 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.480 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 0,1", 2024-05-08T08:53:23.480 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.429860+0000", 2024-05-08T08:53:23.480 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000132137 2024-05-08T08:53:23.480 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.480 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.480 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:53:23.481 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.430811+0000", 2024-05-08T08:53:23.481 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00095040200000000004 2024-05-08T08:53:23.481 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.481 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.481 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:53:23.481 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431126+0000", 2024-05-08T08:53:23.481 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000315006 2024-05-08T08:53:23.481 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.481 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.481 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:53:23.481 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431272+0000", 2024-05-08T08:53:23.481 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000146487 2024-05-08T08:53:23.481 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.481 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.481 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:53:23.481 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431315+0000", 2024-05-08T08:53:23.481 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2697000000000001e-05 2024-05-08T08:53:23.482 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.482 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.482 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:23.482 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431411+0000", 2024-05-08T08:53:23.482 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6024000000000001e-05 2024-05-08T08:53:23.482 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:23.482 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:23.482 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:23.482 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.482 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.482 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:439 3.17 3:ea93a512:::rbd_data.1140100c9703.0000000000000077:head [read 1163264~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-05-08T08:53:23.482 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:08.390871+0000", 2024-05-08T08:53:23.482 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 15.085420512000001, 2024-05-08T08:53:23.482 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0408790590000001, 2024-05-08T08:53:23.483 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:23.483 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:23.483 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:23.483 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:23.483 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:23.483 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:23.483 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 439 2024-05-08T08:53:23.483 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.483 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:23.483 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.483 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:23.483 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390871+0000", 2024-05-08T08:53:23.483 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:23.483 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.483 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.484 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:23.484 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390871+0000", 2024-05-08T08:53:23.484 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:23.484 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.484 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.484 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:23.484 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390872+0000", 2024-05-08T08:53:23.484 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6300000000000005e-07 2024-05-08T08:53:23.484 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.484 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.484 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:23.484 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390875+0000", 2024-05-08T08:53:23.484 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3459999999999998e-06 2024-05-08T08:53:23.484 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.484 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.485 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:23.485 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390891+0000", 2024-05-08T08:53:23.485 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5255e-05 2024-05-08T08:53:23.485 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.485 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.485 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:23.485 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390897+0000", 2024-05-08T08:53:23.485 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2360000000000001e-06 2024-05-08T08:53:23.485 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.485 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.485 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:23.485 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390995+0000", 2024-05-08T08:53:23.485 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8522000000000003e-05 2024-05-08T08:53:23.485 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.485 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.485 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting_for_map not empty", 2024-05-08T08:53:23.486 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.391005+0000", 2024-05-08T08:53:23.486 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1460000000000002e-06 2024-05-08T08:53:23.486 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.486 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.486 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:23.486 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392191+0000", 2024-05-08T08:53:23.486 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0011860270000000001 2024-05-08T08:53:23.486 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.486 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.486 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:53:23.486 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392197+0000", 2024-05-08T08:53:23.486 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4289999999999998e-06 2024-05-08T08:53:23.486 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.486 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.487 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:23.487 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.430596+0000", 2024-05-08T08:53:23.487 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0383989090000001 2024-05-08T08:53:23.487 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.487 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.487 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for rw locks", 2024-05-08T08:53:23.487 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.430676+0000", 2024-05-08T08:53:23.487 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9781000000000001e-05 2024-05-08T08:53:23.487 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.487 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.487 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:23.487 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431460+0000", 2024-05-08T08:53:23.487 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00078427300000000004 2024-05-08T08:53:23.487 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.488 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.488 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:23.488 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431540+0000", 2024-05-08T08:53:23.488 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0224999999999998e-05 2024-05-08T08:53:23.488 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.488 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.488 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:23.488 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431750+0000", 2024-05-08T08:53:23.488 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00021024699999999999 2024-05-08T08:53:23.488 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:23.488 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:23.488 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:23.488 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.489 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.489 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:945 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 4096000~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:23.489 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:53:23.489 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 14.025270945000001, 2024-05-08T08:53:23.489 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086774741000000002, 2024-05-08T08:53:23.489 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:23.489 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:23.489 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:23.489 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:23.489 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:23.489 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:23.489 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 945 2024-05-08T08:53:23.489 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.489 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:23.490 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.490 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:23.490 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:53:23.490 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:23.490 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.490 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.490 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:23.490 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:53:23.490 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:23.490 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.490 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.490 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:23.490 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:53:23.490 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3499999999999996e-07 2024-05-08T08:53:23.491 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.491 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.491 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:23.491 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451024+0000", 2024-05-08T08:53:23.491 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.627e-06 2024-05-08T08:53:23.491 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.491 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.491 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:23.491 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451037+0000", 2024-05-08T08:53:23.491 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2431e-05 2024-05-08T08:53:23.491 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.491 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.491 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:23.491 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451042+0000", 2024-05-08T08:53:23.491 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8409999999999996e-06 2024-05-08T08:53:23.491 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.492 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.492 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:23.492 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537608+0000", 2024-05-08T08:53:23.492 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086565243 2024-05-08T08:53:23.492 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.492 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.492 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:23.492 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537687+0000", 2024-05-08T08:53:23.492 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9519999999999998e-05 2024-05-08T08:53:23.492 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.492 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.492 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:23.492 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537796+0000", 2024-05-08T08:53:23.492 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000108544 2024-05-08T08:53:23.493 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:23.493 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:23.493 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:23.493 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.493 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.493 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:964 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 458752~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:23.493 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451051+0000", 2024-05-08T08:53:23.493 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 14.025240772, 2024-05-08T08:53:23.493 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086990490000000004, 2024-05-08T08:53:23.493 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:23.493 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:23.493 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:23.493 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:23.493 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:23.493 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:23.494 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 964 2024-05-08T08:53:23.494 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.494 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:23.494 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.494 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:23.494 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451051+0000", 2024-05-08T08:53:23.494 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:23.494 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.494 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.494 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:23.494 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451051+0000", 2024-05-08T08:53:23.494 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:23.494 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.494 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.495 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:23.495 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451052+0000", 2024-05-08T08:53:23.495 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7299999999999996e-07 2024-05-08T08:53:23.495 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.495 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.495 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:23.495 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451055+0000", 2024-05-08T08:53:23.495 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.2770000000000001e-06 2024-05-08T08:53:23.495 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.495 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.495 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:23.495 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451061+0000", 2024-05-08T08:53:23.495 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4060000000000002e-06 2024-05-08T08:53:23.495 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.495 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.495 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:23.496 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451066+0000", 2024-05-08T08:53:23.496 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0839999999999999e-06 2024-05-08T08:53:23.496 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.496 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.496 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:23.496 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537839+0000", 2024-05-08T08:53:23.496 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086772962999999995 2024-05-08T08:53:23.496 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.496 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.496 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:23.496 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537934+0000", 2024-05-08T08:53:23.496 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4742999999999997e-05 2024-05-08T08:53:23.496 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.496 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.496 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:23.496 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538042+0000", 2024-05-08T08:53:23.497 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000107444 2024-05-08T08:53:23.497 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:23.497 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:23.497 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:23.497 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.497 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.497 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:965 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 950272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:23.497 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451075+0000", 2024-05-08T08:53:23.497 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 14.02521671, 2024-05-08T08:53:23.497 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087230196999999995, 2024-05-08T08:53:23.497 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:23.497 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:23.497 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:23.497 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:23.497 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:23.498 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:23.498 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 965 2024-05-08T08:53:23.498 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.498 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:23.498 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.498 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:23.498 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451075+0000", 2024-05-08T08:53:23.498 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:23.498 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.498 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.498 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:23.498 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451075+0000", 2024-05-08T08:53:23.498 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:23.498 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.499 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.499 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:23.499 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451076+0000", 2024-05-08T08:53:23.499 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5000000000000002e-07 2024-05-08T08:53:23.499 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.499 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.499 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:23.499 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451078+0000", 2024-05-08T08:53:23.499 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.5639999999999999e-06 2024-05-08T08:53:23.499 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.499 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.499 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:23.499 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451085+0000", 2024-05-08T08:53:23.499 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5640000000000002e-06 2024-05-08T08:53:23.499 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.499 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.500 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:23.500 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451090+0000", 2024-05-08T08:53:23.500 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1699999999999996e-06 2024-05-08T08:53:23.500 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.500 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.500 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:23.500 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538106+0000", 2024-05-08T08:53:23.500 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087015888999999999 2024-05-08T08:53:23.500 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.500 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.500 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:23.500 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538190+0000", 2024-05-08T08:53:23.500 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3961999999999993e-05 2024-05-08T08:53:23.500 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.501 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.501 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:23.501 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538305+0000", 2024-05-08T08:53:23.501 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011529800000000001 2024-05-08T08:53:23.501 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:23.501 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:23.501 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:23.501 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.501 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.501 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:966 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 1441792~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:23.501 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:53:23.501 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 14.025193336999999, 2024-05-08T08:53:23.501 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087446323000000006, 2024-05-08T08:53:23.501 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:23.501 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:23.502 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:23.502 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:23.502 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:23.502 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:23.502 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 966 2024-05-08T08:53:23.502 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.502 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:23.502 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.502 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:23.502 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:53:23.502 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:23.502 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.502 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.503 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:23.503 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:53:23.503 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:23.503 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.503 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.503 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:23.503 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:53:23.503 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3700000000000001e-07 2024-05-08T08:53:23.503 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.503 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.503 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:23.503 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451101+0000", 2024-05-08T08:53:23.503 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2560000000000001e-06 2024-05-08T08:53:23.503 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.504 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.504 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:23.504 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451107+0000", 2024-05-08T08:53:23.504 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1070000000000001e-06 2024-05-08T08:53:23.504 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.504 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.504 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:23.504 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451112+0000", 2024-05-08T08:53:23.504 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1420000000000001e-06 2024-05-08T08:53:23.504 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.504 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.504 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:23.504 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538341+0000", 2024-05-08T08:53:23.504 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087228302999999993 2024-05-08T08:53:23.504 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.505 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.505 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:23.505 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538424+0000", 2024-05-08T08:53:23.505 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3497000000000004e-05 2024-05-08T08:53:23.505 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.505 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.505 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:23.505 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538545+0000", 2024-05-08T08:53:23.505 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000120581 2024-05-08T08:53:23.505 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:23.505 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:23.505 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:23.505 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.505 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.505 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:967 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:23.506 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451120+0000", 2024-05-08T08:53:23.506 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 14.025171862000001, 2024-05-08T08:53:23.506 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087632725999999994, 2024-05-08T08:53:23.506 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:23.506 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:23.506 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:23.506 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:23.506 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:23.506 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:23.506 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 967 2024-05-08T08:53:23.506 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.506 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:23.506 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.506 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:23.507 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451120+0000", 2024-05-08T08:53:23.507 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:23.507 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.507 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.507 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:23.507 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451120+0000", 2024-05-08T08:53:23.507 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:23.507 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.507 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.507 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:23.507 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451121+0000", 2024-05-08T08:53:23.507 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8800000000000002e-07 2024-05-08T08:53:23.507 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.507 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.507 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:23.508 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451130+0000", 2024-05-08T08:53:23.508 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2299999999999997e-06 2024-05-08T08:53:23.508 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.508 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.508 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:23.508 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451137+0000", 2024-05-08T08:53:23.508 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4669999999999998e-06 2024-05-08T08:53:23.508 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.508 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.508 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:23.508 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451143+0000", 2024-05-08T08:53:23.508 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2839999999999997e-06 2024-05-08T08:53:23.508 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.508 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.508 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:23.509 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538575+0000", 2024-05-08T08:53:23.509 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087432789999999996 2024-05-08T08:53:23.509 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.509 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.509 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:23.509 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538648+0000", 2024-05-08T08:53:23.509 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2877e-05 2024-05-08T08:53:23.509 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.509 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.509 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:23.509 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538753+0000", 2024-05-08T08:53:23.509 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010449 2024-05-08T08:53:23.509 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:23.509 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:23.510 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:23.510 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.510 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.510 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:968 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 2424832~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:23.510 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451152+0000", 2024-05-08T08:53:23.510 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 14.025139777, 2024-05-08T08:53:23.510 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087813325999999997, 2024-05-08T08:53:23.510 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:23.510 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:23.510 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:23.510 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:23.510 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:23.510 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:23.510 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 968 2024-05-08T08:53:23.510 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.511 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:23.511 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.511 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:23.511 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451152+0000", 2024-05-08T08:53:23.511 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:23.511 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.511 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.511 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:23.511 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451152+0000", 2024-05-08T08:53:23.511 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:23.511 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.511 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.512 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:23.512 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451153+0000", 2024-05-08T08:53:23.512 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3099999999999998e-07 2024-05-08T08:53:23.512 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.512 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.512 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:23.512 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451155+0000", 2024-05-08T08:53:23.512 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4760000000000001e-06 2024-05-08T08:53:23.512 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.512 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.512 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:23.512 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451162+0000", 2024-05-08T08:53:23.512 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5969999999999997e-06 2024-05-08T08:53:23.512 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.512 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.513 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:23.513 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451167+0000", 2024-05-08T08:53:23.513 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.977e-06 2024-05-08T08:53:23.513 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.513 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.513 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:23.513 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538781+0000", 2024-05-08T08:53:23.513 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087614846999999996 2024-05-08T08:53:23.513 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.513 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.513 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:23.513 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538857+0000", 2024-05-08T08:53:23.513 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5358000000000006e-05 2024-05-08T08:53:23.513 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.513 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.513 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:23.514 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538965+0000", 2024-05-08T08:53:23.514 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010864 2024-05-08T08:53:23.514 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:23.514 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:23.514 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:23.514 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.514 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.514 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:969 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 2916352~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:23.514 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:53:23.514 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 14.025118072, 2024-05-08T08:53:23.514 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087998980000000004, 2024-05-08T08:53:23.514 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:23.514 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:23.514 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:23.515 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:23.515 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:23.515 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:23.515 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 969 2024-05-08T08:53:23.515 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.515 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:23.515 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.515 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:23.515 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:53:23.515 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:23.515 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.515 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.515 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:23.515 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:53:23.515 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:23.516 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.516 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.516 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:23.516 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:53:23.516 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3599999999999999e-07 2024-05-08T08:53:23.516 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.516 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.516 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:23.516 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451177+0000", 2024-05-08T08:53:23.516 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4590000000000001e-06 2024-05-08T08:53:23.516 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.516 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.516 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:23.516 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451184+0000", 2024-05-08T08:53:23.516 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4109999999999999e-06 2024-05-08T08:53:23.516 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.517 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.517 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:23.517 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451191+0000", 2024-05-08T08:53:23.517 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7279999999999998e-06 2024-05-08T08:53:23.517 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.517 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.517 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:23.517 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538997+0000", 2024-05-08T08:53:23.517 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087806170000000003 2024-05-08T08:53:23.517 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.517 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.517 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:23.517 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539069+0000", 2024-05-08T08:53:23.517 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2373999999999995e-05 2024-05-08T08:53:23.517 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.518 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.518 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:23.518 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539173+0000", 2024-05-08T08:53:23.518 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000103402 2024-05-08T08:53:23.518 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:23.518 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:23.518 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:23.518 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.518 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.518 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:970 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 3407872~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:23.518 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451198+0000", 2024-05-08T08:53:23.518 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 14.025093475, 2024-05-08T08:53:23.518 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088183368999999998, 2024-05-08T08:53:23.518 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:23.519 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:23.519 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:23.519 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:23.519 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:23.519 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:23.519 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 970 2024-05-08T08:53:23.519 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.519 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:23.519 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.519 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:23.519 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451198+0000", 2024-05-08T08:53:23.519 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:23.519 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.519 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.519 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:23.520 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451198+0000", 2024-05-08T08:53:23.520 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:23.520 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.520 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.520 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:23.520 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451199+0000", 2024-05-08T08:53:23.520 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7400000000000003e-07 2024-05-08T08:53:23.520 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.520 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.520 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:23.520 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451211+0000", 2024-05-08T08:53:23.520 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2109999999999999e-05 2024-05-08T08:53:23.520 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.520 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.520 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:23.521 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451219+0000", 2024-05-08T08:53:23.521 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4170000000000003e-06 2024-05-08T08:53:23.521 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.521 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.521 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:23.521 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451224+0000", 2024-05-08T08:53:23.521 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3990000000000003e-06 2024-05-08T08:53:23.521 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.521 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.521 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:23.521 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539204+0000", 2024-05-08T08:53:23.521 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087980099000000006 2024-05-08T08:53:23.521 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.522 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.522 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:23.522 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539276+0000", 2024-05-08T08:53:23.522 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1513000000000005e-05 2024-05-08T08:53:23.522 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.522 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.522 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:23.522 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539382+0000", 2024-05-08T08:53:23.522 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010615699999999999 2024-05-08T08:53:23.522 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:23.522 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:23.522 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:23.522 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.523 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.523 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:971 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 3899392~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:23.523 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451233+0000", 2024-05-08T08:53:23.523 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 14.025058767999999, 2024-05-08T08:53:23.523 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088431525999999996, 2024-05-08T08:53:23.523 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:23.523 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:23.523 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:23.523 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:23.523 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:23.523 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:23.523 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 971 2024-05-08T08:53:23.523 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.524 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:23.524 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.524 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:23.524 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451233+0000", 2024-05-08T08:53:23.524 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:23.524 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.524 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.524 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:23.524 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451233+0000", 2024-05-08T08:53:23.524 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:23.524 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.524 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.524 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:23.524 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451234+0000", 2024-05-08T08:53:23.525 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8500000000000001e-07 2024-05-08T08:53:23.525 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.525 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.525 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:23.525 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451237+0000", 2024-05-08T08:53:23.525 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8320000000000002e-06 2024-05-08T08:53:23.525 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.525 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.525 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:23.525 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451243+0000", 2024-05-08T08:53:23.525 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6749999999999996e-06 2024-05-08T08:53:23.525 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.525 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.525 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:23.526 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451249+0000", 2024-05-08T08:53:23.526 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1229999999999997e-06 2024-05-08T08:53:23.526 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.526 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.526 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:23.526 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539412+0000", 2024-05-08T08:53:23.526 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088162668999999999 2024-05-08T08:53:23.526 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.526 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.526 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:23.526 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539505+0000", 2024-05-08T08:53:23.526 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3240999999999997e-05 2024-05-08T08:53:23.526 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.527 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.527 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:23.527 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539665+0000", 2024-05-08T08:53:23.527 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00015930100000000001 2024-05-08T08:53:23.527 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:23.527 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:23.527 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:23.527 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.527 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.527 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:972 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 196608~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:23.527 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451257+0000", 2024-05-08T08:53:23.527 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 14.025034823, 2024-05-08T08:53:23.527 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088719928000000003, 2024-05-08T08:53:23.528 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:23.528 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:23.528 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:23.528 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:23.528 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:23.528 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:23.528 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 972 2024-05-08T08:53:23.528 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.528 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:23.528 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.528 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:23.528 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451257+0000", 2024-05-08T08:53:23.528 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:23.528 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.528 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.528 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:23.529 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451257+0000", 2024-05-08T08:53:23.529 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:23.529 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.529 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.529 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:23.529 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451258+0000", 2024-05-08T08:53:23.529 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7999999999999996e-07 2024-05-08T08:53:23.529 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.529 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.529 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:23.529 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451260+0000", 2024-05-08T08:53:23.529 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4609999999999998e-06 2024-05-08T08:53:23.529 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.529 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.529 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:23.529 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451267+0000", 2024-05-08T08:53:23.530 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.81e-06 2024-05-08T08:53:23.530 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.530 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.530 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:23.530 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451273+0000", 2024-05-08T08:53:23.530 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0859999999999998e-06 2024-05-08T08:53:23.530 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.530 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.530 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:23.530 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539718+0000", 2024-05-08T08:53:23.530 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088445059000000006 2024-05-08T08:53:23.530 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.530 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.530 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:23.531 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539834+0000", 2024-05-08T08:53:23.531 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000116426 2024-05-08T08:53:23.531 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.531 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.531 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:23.531 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539977+0000", 2024-05-08T08:53:23.531 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014260600000000001 2024-05-08T08:53:23.531 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:23.531 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:23.531 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:23.531 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.531 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.531 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:973 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 688128~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:23.531 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:53:23.531 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 14.025012239, 2024-05-08T08:53:23.532 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088918496, 2024-05-08T08:53:23.532 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:23.532 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:23.532 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:23.532 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:23.532 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:23.532 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:23.532 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 973 2024-05-08T08:53:23.532 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.532 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:23.532 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.532 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:23.532 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:53:23.532 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:23.532 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.532 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.532 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:23.533 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:53:23.533 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:23.533 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.533 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.533 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:23.533 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:53:23.533 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9099999999999999e-07 2024-05-08T08:53:23.533 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.533 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.533 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:23.533 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451282+0000", 2024-05-08T08:53:23.533 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1550000000000001e-06 2024-05-08T08:53:23.533 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.533 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.534 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:23.534 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451288+0000", 2024-05-08T08:53:23.534 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8889999999999999e-06 2024-05-08T08:53:23.534 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.534 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.534 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:23.534 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451294+0000", 2024-05-08T08:53:23.534 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0850000000000002e-06 2024-05-08T08:53:23.534 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.534 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.534 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:23.534 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540011+0000", 2024-05-08T08:53:23.534 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088717314000000005 2024-05-08T08:53:23.534 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.534 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.535 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:23.535 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540089+0000", 2024-05-08T08:53:23.535 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7438e-05 2024-05-08T08:53:23.535 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.535 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.535 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:23.535 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540198+0000", 2024-05-08T08:53:23.535 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000109224 2024-05-08T08:53:23.535 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:23.535 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:23.535 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:23.535 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.535 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.535 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:974 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 1179648~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:23.536 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451301+0000", 2024-05-08T08:53:23.536 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 14.024990605999999, 2024-05-08T08:53:23.536 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089105078000000004, 2024-05-08T08:53:23.536 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:23.536 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:23.536 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:23.536 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:23.536 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:23.536 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:23.536 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 974 2024-05-08T08:53:23.536 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.536 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:23.536 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.536 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:23.537 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451301+0000", 2024-05-08T08:53:23.537 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:23.537 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.537 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.537 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:23.537 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451301+0000", 2024-05-08T08:53:23.537 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:23.537 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.537 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.537 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:23.537 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451302+0000", 2024-05-08T08:53:23.537 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7800000000000002e-07 2024-05-08T08:53:23.537 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.537 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.537 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:23.537 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451304+0000", 2024-05-08T08:53:23.538 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.65e-06 2024-05-08T08:53:23.538 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.538 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.538 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:23.538 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451311+0000", 2024-05-08T08:53:23.538 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.472e-06 2024-05-08T08:53:23.538 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.538 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.538 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:23.538 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451316+0000", 2024-05-08T08:53:23.538 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9189999999999998e-06 2024-05-08T08:53:23.538 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.538 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.539 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:23.539 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540234+0000", 2024-05-08T08:53:23.539 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088917752000000003 2024-05-08T08:53:23.539 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.539 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.539 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:23.539 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540308+0000", 2024-05-08T08:53:23.539 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4758000000000005e-05 2024-05-08T08:53:23.539 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.539 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.539 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:23.539 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540406+0000", 2024-05-08T08:53:23.539 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8048999999999996e-05 2024-05-08T08:53:23.539 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:23.540 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:23.540 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:23.540 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.540 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.540 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:975 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 1671168~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:23.540 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:53:23.540 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 14.024948276, 2024-05-08T08:53:23.540 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089294139999999994, 2024-05-08T08:53:23.540 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:23.540 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:23.540 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:23.540 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:23.540 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:23.541 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:23.541 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 975 2024-05-08T08:53:23.541 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.541 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:23.541 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.541 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:23.541 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:53:23.541 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:23.541 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.541 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.541 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:23.541 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:53:23.541 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:23.541 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.541 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.542 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:23.542 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:53:23.542 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1299999999999999e-07 2024-05-08T08:53:23.542 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.542 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.542 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:23.542 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451348+0000", 2024-05-08T08:53:23.542 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.484e-06 2024-05-08T08:53:23.542 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.542 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.542 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:23.542 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451356+0000", 2024-05-08T08:53:23.542 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8499999999999994e-06 2024-05-08T08:53:23.543 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.543 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.543 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:23.543 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451366+0000", 2024-05-08T08:53:23.543 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.9850000000000001e-06 2024-05-08T08:53:23.543 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.543 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.543 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:23.543 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540448+0000", 2024-05-08T08:53:23.543 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089082254 2024-05-08T08:53:23.543 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.543 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.543 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:23.543 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540528+0000", 2024-05-08T08:53:23.544 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0234999999999993e-05 2024-05-08T08:53:23.544 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.544 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.544 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:23.544 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540638+0000", 2024-05-08T08:53:23.544 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010951900000000001 2024-05-08T08:53:23.544 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:23.544 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:23.544 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:23.544 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.544 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.544 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:976 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 2162688~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:23.544 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451375+0000", 2024-05-08T08:53:23.544 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 14.024916529, 2024-05-08T08:53:23.545 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089472745000000006, 2024-05-08T08:53:23.545 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:23.545 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:23.545 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:23.545 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:23.545 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:23.545 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:23.545 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 976 2024-05-08T08:53:23.545 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.545 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:23.545 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.545 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:23.545 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451375+0000", 2024-05-08T08:53:23.545 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:23.545 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.546 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.546 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:23.546 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451375+0000", 2024-05-08T08:53:23.546 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:23.546 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.546 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.546 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:23.546 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451376+0000", 2024-05-08T08:53:23.546 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2399999999999998e-07 2024-05-08T08:53:23.546 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.546 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.546 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:23.546 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451379+0000", 2024-05-08T08:53:23.546 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6359999999999998e-06 2024-05-08T08:53:23.546 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.547 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.547 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:23.547 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451387+0000", 2024-05-08T08:53:23.547 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9929999999999995e-06 2024-05-08T08:53:23.547 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.547 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.547 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:23.547 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451393+0000", 2024-05-08T08:53:23.547 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4699999999999999e-06 2024-05-08T08:53:23.547 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.547 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.547 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:23.547 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540677+0000", 2024-05-08T08:53:23.547 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089283504999999999 2024-05-08T08:53:23.547 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.548 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.548 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:23.548 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540743+0000", 2024-05-08T08:53:23.548 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6303999999999994e-05 2024-05-08T08:53:23.548 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.548 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.548 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:23.548 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540848+0000", 2024-05-08T08:53:23.548 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000105213 2024-05-08T08:53:23.548 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:23.548 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:23.548 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:23.548 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.548 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.548 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:977 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 2654208~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:23.549 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:53:23.549 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 14.024891096999999, 2024-05-08T08:53:23.549 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089698473000000001, 2024-05-08T08:53:23.549 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:23.549 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:23.549 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:23.549 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:23.549 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:23.549 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:23.549 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 977 2024-05-08T08:53:23.549 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.549 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:23.549 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.549 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:23.549 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:53:23.549 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:23.550 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.550 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.550 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:23.550 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:53:23.550 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:23.550 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.550 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.550 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:23.550 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:53:23.550 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2899999999999999e-07 2024-05-08T08:53:23.550 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.550 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.550 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:23.551 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451404+0000", 2024-05-08T08:53:23.551 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6649999999999999e-06 2024-05-08T08:53:23.551 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.551 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.551 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:23.551 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451410+0000", 2024-05-08T08:53:23.551 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3269999999999997e-06 2024-05-08T08:53:23.551 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.551 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.551 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:23.551 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451415+0000", 2024-05-08T08:53:23.551 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9529999999999998e-06 2024-05-08T08:53:23.551 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.551 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.552 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:23.552 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540912+0000", 2024-05-08T08:53:23.552 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089496950000000006 2024-05-08T08:53:23.552 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.552 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.552 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:23.552 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541000+0000", 2024-05-08T08:53:23.552 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7429000000000006e-05 2024-05-08T08:53:23.552 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.552 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.552 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:23.552 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541099+0000", 2024-05-08T08:53:23.552 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.9720000000000001e-05 2024-05-08T08:53:23.552 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:23.553 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:23.553 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:23.553 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.553 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.553 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:978 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 3145728~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:23.553 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:53:23.553 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 14.024869889, 2024-05-08T08:53:23.553 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089900635000000007, 2024-05-08T08:53:23.553 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:23.553 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:23.553 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:23.553 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:23.553 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:23.553 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:23.554 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 978 2024-05-08T08:53:23.554 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.554 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:23.554 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.554 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:23.554 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:53:23.554 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:23.554 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.554 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.554 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:23.554 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:53:23.554 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:23.554 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.555 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.555 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:23.555 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:53:23.555 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9400000000000001e-07 2024-05-08T08:53:23.555 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.555 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.555 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:23.555 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451425+0000", 2024-05-08T08:53:23.555 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2460000000000002e-06 2024-05-08T08:53:23.555 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.555 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.555 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:23.555 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451431+0000", 2024-05-08T08:53:23.555 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2419999999999997e-06 2024-05-08T08:53:23.555 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.556 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.556 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:23.556 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451436+0000", 2024-05-08T08:53:23.556 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.887e-06 2024-05-08T08:53:23.556 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.556 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.556 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:23.556 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541152+0000", 2024-05-08T08:53:23.556 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089716194999999999 2024-05-08T08:53:23.556 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.556 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.556 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:23.556 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541224+0000", 2024-05-08T08:53:23.556 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2416000000000006e-05 2024-05-08T08:53:23.556 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.556 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.557 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:23.557 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541323+0000", 2024-05-08T08:53:23.557 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8255e-05 2024-05-08T08:53:23.557 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:23.557 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:23.557 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:23.557 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.557 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.557 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:979 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 3637248~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:23.557 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451462+0000", 2024-05-08T08:53:23.557 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 14.02482944, 2024-05-08T08:53:23.557 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.090036441999999994, 2024-05-08T08:53:23.557 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:23.557 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:23.558 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:23.558 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:23.558 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:23.558 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:23.558 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 979 2024-05-08T08:53:23.558 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.558 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:23.558 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.558 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:23.558 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451462+0000", 2024-05-08T08:53:23.558 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:23.558 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.558 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.558 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:23.558 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451462+0000", 2024-05-08T08:53:23.559 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:23.559 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.559 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.559 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:23.559 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451463+0000", 2024-05-08T08:53:23.559 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4300000000000002e-07 2024-05-08T08:53:23.559 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.559 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.559 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:23.559 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451471+0000", 2024-05-08T08:53:23.559 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0800000000000006e-06 2024-05-08T08:53:23.559 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.559 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.559 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:23.559 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451481+0000", 2024-05-08T08:53:23.559 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.38e-06 2024-05-08T08:53:23.559 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.560 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.560 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:23.560 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451487+0000", 2024-05-08T08:53:23.560 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4590000000000003e-06 2024-05-08T08:53:23.560 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.560 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.560 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:23.560 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541350+0000", 2024-05-08T08:53:23.560 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089862599000000001 2024-05-08T08:53:23.560 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.560 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.560 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:23.560 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541407+0000", 2024-05-08T08:53:23.560 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7667999999999997e-05 2024-05-08T08:53:23.560 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.560 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.560 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:23.561 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541499+0000", 2024-05-08T08:53:23.561 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1513000000000003e-05 2024-05-08T08:53:23.561 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:23.561 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:23.561 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:23.561 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.561 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.561 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:980 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 4128768~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:23.561 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451496+0000", 2024-05-08T08:53:23.561 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 14.024795684000001, 2024-05-08T08:53:23.561 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.090258985, 2024-05-08T08:53:23.561 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:23.561 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:23.561 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:23.561 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:23.561 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:23.562 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:23.562 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 980 2024-05-08T08:53:23.562 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.562 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:23.562 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.562 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:23.562 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451496+0000", 2024-05-08T08:53:23.562 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:23.562 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.562 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.562 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:23.562 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451496+0000", 2024-05-08T08:53:23.562 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:23.562 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.562 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.562 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:23.563 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451497+0000", 2024-05-08T08:53:23.563 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7599999999999997e-07 2024-05-08T08:53:23.563 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.563 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.563 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:23.563 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451500+0000", 2024-05-08T08:53:23.563 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7930000000000002e-06 2024-05-08T08:53:23.563 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.563 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.563 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:23.563 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451506+0000", 2024-05-08T08:53:23.564 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6980000000000001e-06 2024-05-08T08:53:23.564 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.564 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.564 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:23.564 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451512+0000", 2024-05-08T08:53:23.564 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2490000000000001e-06 2024-05-08T08:53:23.564 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.564 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.564 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:23.564 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541550+0000", 2024-05-08T08:53:23.564 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.090037961999999999 2024-05-08T08:53:23.564 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.564 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.564 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:23.564 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541647+0000", 2024-05-08T08:53:23.565 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6924999999999998e-05 2024-05-08T08:53:23.565 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:23.565 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:23.565 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:23.565 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541755+0000", 2024-05-08T08:53:23.565 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000108682 2024-05-08T08:53:23.565 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:23.565 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:23.565 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:23.565 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:23.565 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:23.565 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:23.567 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:53:23.567 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:23.569+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:23.668 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:23.671+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:23.675 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:53:23.675 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:53:23.675 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:53:23.675 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:53:23.684 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:53:23.769 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:23.772+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:23.830 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:53:23.830 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:53:23.830 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:53:23.830 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:53:23.830 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:53:23.839 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:53:23.883 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:23.886+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:23.947 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:53:23.947 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:53:23.947 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:53:23.947 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:53:23.947 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.947 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4742 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1832960~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:23.947 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.585513+0000", 2024-05-08T08:53:23.947 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 52.363742045999999, 2024-05-08T08:53:23.948 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035588763000000002, 2024-05-08T08:53:23.948 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:23.948 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:23.948 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:23.948 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:23.948 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:23.948 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:23.948 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4742 2024-05-08T08:53:23.948 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.948 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:23.948 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.948 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:23.948 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585513+0000", 2024-05-08T08:53:23.948 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:23.948 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.949 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.949 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:23.949 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585513+0000", 2024-05-08T08:53:23.949 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:23.949 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.949 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.949 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:23.949 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585514+0000", 2024-05-08T08:53:23.949 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6100000000000001e-07 2024-05-08T08:53:23.949 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.949 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.949 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:23.949 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585516+0000", 2024-05-08T08:53:23.949 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.526e-06 2024-05-08T08:53:23.950 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.950 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.950 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:23.950 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585523+0000", 2024-05-08T08:53:23.950 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.562e-06 2024-05-08T08:53:23.950 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.950 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.950 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:23.950 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585528+0000", 2024-05-08T08:53:23.950 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7990000000000001e-06 2024-05-08T08:53:23.950 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.950 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.950 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:23.950 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.620939+0000", 2024-05-08T08:53:23.951 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035411419 2024-05-08T08:53:23.951 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.951 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.951 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:23.951 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621014+0000", 2024-05-08T08:53:23.951 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5017000000000004e-05 2024-05-08T08:53:23.951 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.951 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.951 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:23.951 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621102+0000", 2024-05-08T08:53:23.951 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7979000000000006e-05 2024-05-08T08:53:23.951 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:23.951 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:23.951 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:23.951 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.951 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.952 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4743 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1849344~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:23.952 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.585628+0000", 2024-05-08T08:53:23.952 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 52.363627159000004, 2024-05-08T08:53:23.952 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035659962000000003, 2024-05-08T08:53:23.952 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:23.952 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:23.952 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:23.952 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:23.952 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:23.952 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:23.952 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4743 2024-05-08T08:53:23.952 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.952 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:23.952 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.952 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:23.953 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585628+0000", 2024-05-08T08:53:23.953 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:23.953 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.953 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.953 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:23.953 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585628+0000", 2024-05-08T08:53:23.953 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:23.953 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.953 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.953 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:23.953 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585629+0000", 2024-05-08T08:53:23.953 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2300000000000001e-07 2024-05-08T08:53:23.953 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.953 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.953 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:23.954 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585632+0000", 2024-05-08T08:53:23.954 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1370000000000002e-06 2024-05-08T08:53:23.954 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.954 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.954 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:23.954 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585641+0000", 2024-05-08T08:53:23.954 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5220000000000007e-06 2024-05-08T08:53:23.954 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.954 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.954 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:23.954 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585646+0000", 2024-05-08T08:53:23.955 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.609e-06 2024-05-08T08:53:23.955 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.955 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.955 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:23.955 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621132+0000", 2024-05-08T08:53:23.955 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035485472999999997 2024-05-08T08:53:23.955 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.955 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.955 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:23.955 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621204+0000", 2024-05-08T08:53:23.955 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2259000000000001e-05 2024-05-08T08:53:23.955 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.955 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.956 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:23.956 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621288+0000", 2024-05-08T08:53:23.956 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4338999999999993e-05 2024-05-08T08:53:23.956 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:23.956 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:23.956 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:23.956 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.956 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.956 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4746 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1865728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:23.956 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.585656+0000", 2024-05-08T08:53:23.956 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 52.363599477000001, 2024-05-08T08:53:23.956 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035815909, 2024-05-08T08:53:23.956 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:23.956 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:23.957 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:23.957 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:23.957 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:23.957 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:23.957 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4746 2024-05-08T08:53:23.957 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.957 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:23.957 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.957 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:23.957 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585656+0000", 2024-05-08T08:53:23.957 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:23.957 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.957 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.957 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:23.957 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585656+0000", 2024-05-08T08:53:23.957 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:23.958 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.958 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.958 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:23.958 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585656+0000", 2024-05-08T08:53:23.958 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.08e-07 2024-05-08T08:53:23.958 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.958 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.958 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:23.958 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585658+0000", 2024-05-08T08:53:23.958 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0889999999999998e-06 2024-05-08T08:53:23.958 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.958 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.958 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:23.958 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585664+0000", 2024-05-08T08:53:23.958 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3990000000000003e-06 2024-05-08T08:53:23.958 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.959 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.959 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:23.959 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585668+0000", 2024-05-08T08:53:23.959 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3170000000000003e-06 2024-05-08T08:53:23.959 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.959 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.959 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:23.959 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621316+0000", 2024-05-08T08:53:23.959 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035647908999999998 2024-05-08T08:53:23.959 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.959 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.959 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:23.959 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621389+0000", 2024-05-08T08:53:23.959 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2531000000000001e-05 2024-05-08T08:53:23.959 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.959 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.960 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:23.960 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621472+0000", 2024-05-08T08:53:23.960 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3256000000000004e-05 2024-05-08T08:53:23.960 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:23.960 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:23.960 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:23.960 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.960 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.960 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4748 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1882112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:23.960 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.585697+0000", 2024-05-08T08:53:23.960 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 52.363558030999997, 2024-05-08T08:53:23.960 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035962485000000002, 2024-05-08T08:53:23.960 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:23.960 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:23.960 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:23.961 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:23.961 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:23.961 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:23.961 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4748 2024-05-08T08:53:23.961 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.961 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:23.961 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.961 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:23.961 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585697+0000", 2024-05-08T08:53:23.961 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:23.961 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.961 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.961 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:23.961 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585697+0000", 2024-05-08T08:53:23.961 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:23.961 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.962 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.962 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:23.962 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585698+0000", 2024-05-08T08:53:23.962 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0900000000000001e-07 2024-05-08T08:53:23.962 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.962 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.962 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:23.962 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585701+0000", 2024-05-08T08:53:23.962 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0149999999999999e-06 2024-05-08T08:53:23.962 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.962 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.962 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:23.962 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585712+0000", 2024-05-08T08:53:23.962 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0655e-05 2024-05-08T08:53:23.963 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.963 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.963 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:23.963 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585717+0000", 2024-05-08T08:53:23.963 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3800000000000002e-06 2024-05-08T08:53:23.963 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.963 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.963 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:23.963 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621502+0000", 2024-05-08T08:53:23.963 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035784559000000001 2024-05-08T08:53:23.963 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.963 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.964 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:23.964 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621575+0000", 2024-05-08T08:53:23.964 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3269999999999995e-05 2024-05-08T08:53:23.964 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.964 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.964 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:23.964 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621660+0000", 2024-05-08T08:53:23.964 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4997e-05 2024-05-08T08:53:23.964 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:23.964 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:23.964 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:23.964 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.964 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.965 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4750 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1898496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:23.965 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.585810+0000", 2024-05-08T08:53:23.965 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 52.363445200999998, 2024-05-08T08:53:23.965 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036034928000000001, 2024-05-08T08:53:23.965 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:23.965 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:23.965 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:23.965 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:23.965 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:23.965 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:23.965 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4750 2024-05-08T08:53:23.965 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.965 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:23.965 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.965 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:23.966 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585810+0000", 2024-05-08T08:53:23.966 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:23.966 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.966 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.966 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:23.966 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585810+0000", 2024-05-08T08:53:23.966 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:23.966 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.966 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.966 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:23.966 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585811+0000", 2024-05-08T08:53:23.966 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6499999999999999e-07 2024-05-08T08:53:23.966 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.966 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.966 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:23.967 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585814+0000", 2024-05-08T08:53:23.967 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1709999999999998e-06 2024-05-08T08:53:23.967 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.967 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.967 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:23.967 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585822+0000", 2024-05-08T08:53:23.967 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8909999999999995e-06 2024-05-08T08:53:23.967 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.967 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.967 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:23.967 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585827+0000", 2024-05-08T08:53:23.967 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5289999999999999e-06 2024-05-08T08:53:23.967 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.967 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.967 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:23.967 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621688+0000", 2024-05-08T08:53:23.968 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035861111000000001 2024-05-08T08:53:23.968 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.968 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.968 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:23.968 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621761+0000", 2024-05-08T08:53:23.968 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2211000000000004e-05 2024-05-08T08:53:23.968 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.968 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.968 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:23.968 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621845+0000", 2024-05-08T08:53:23.968 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4450000000000006e-05 2024-05-08T08:53:23.968 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:23.968 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:23.968 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:23.968 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.968 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.969 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4752 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1914880~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:23.969 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.585858+0000", 2024-05-08T08:53:23.969 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 52.363397231999997, 2024-05-08T08:53:23.969 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036180081000000003, 2024-05-08T08:53:23.969 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:23.969 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:23.969 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:23.969 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:23.969 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:23.969 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:23.969 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4752 2024-05-08T08:53:23.969 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.969 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:23.969 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.969 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:23.969 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585858+0000", 2024-05-08T08:53:23.969 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:23.970 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.970 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.970 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:23.970 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585858+0000", 2024-05-08T08:53:23.970 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:23.970 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.970 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.970 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:23.970 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585859+0000", 2024-05-08T08:53:23.970 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.82e-07 2024-05-08T08:53:23.970 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.970 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.970 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:23.970 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585861+0000", 2024-05-08T08:53:23.971 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.695e-06 2024-05-08T08:53:23.971 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.971 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.971 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:23.971 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585869+0000", 2024-05-08T08:53:23.971 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0299999999999994e-06 2024-05-08T08:53:23.971 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.971 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.971 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:23.971 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585874+0000", 2024-05-08T08:53:23.971 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9620000000000001e-06 2024-05-08T08:53:23.971 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.971 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.971 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:23.971 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621878+0000", 2024-05-08T08:53:23.971 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036004043999999999 2024-05-08T08:53:23.971 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.972 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.972 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:23.972 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621954+0000", 2024-05-08T08:53:23.972 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5414000000000007e-05 2024-05-08T08:53:23.972 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.972 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.972 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:23.972 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622038+0000", 2024-05-08T08:53:23.972 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4454000000000002e-05 2024-05-08T08:53:23.972 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:23.972 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:23.972 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:23.972 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.972 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.972 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4753 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1931264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:23.972 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.585947+0000", 2024-05-08T08:53:23.973 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 52.363308584000002, 2024-05-08T08:53:23.973 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036267992999999998, 2024-05-08T08:53:23.973 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:23.973 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:23.973 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:23.973 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:23.973 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:23.973 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:23.973 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4753 2024-05-08T08:53:23.973 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.973 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:23.973 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.973 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:23.973 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585947+0000", 2024-05-08T08:53:23.973 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:23.973 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.974 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.974 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:23.974 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585947+0000", 2024-05-08T08:53:23.974 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:23.974 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.974 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.974 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:23.974 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585948+0000", 2024-05-08T08:53:23.974 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6000000000000002e-07 2024-05-08T08:53:23.974 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.974 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.974 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:23.974 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585952+0000", 2024-05-08T08:53:23.975 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7090000000000001e-06 2024-05-08T08:53:23.975 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.975 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.975 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:23.975 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585963+0000", 2024-05-08T08:53:23.975 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0740000000000001e-05 2024-05-08T08:53:23.975 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.975 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.975 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:23.975 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585970+0000", 2024-05-08T08:53:23.975 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3309999999999998e-06 2024-05-08T08:53:23.975 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.975 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.975 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:23.976 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622065+0000", 2024-05-08T08:53:23.976 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036094676999999999 2024-05-08T08:53:23.976 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.976 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.976 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:23.976 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622132+0000", 2024-05-08T08:53:23.976 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7214999999999999e-05 2024-05-08T08:53:23.976 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.976 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.976 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:23.976 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622215+0000", 2024-05-08T08:53:23.976 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2460999999999995e-05 2024-05-08T08:53:23.976 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:23.976 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:23.976 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:23.976 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.977 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.977 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4756 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1947648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:23.977 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586018+0000", 2024-05-08T08:53:23.977 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 52.36323694, 2024-05-08T08:53:23.977 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036382335000000002, 2024-05-08T08:53:23.977 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:23.977 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:23.977 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:23.977 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:23.977 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:23.977 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:23.977 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4756 2024-05-08T08:53:23.977 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.977 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:23.977 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.977 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:23.978 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586018+0000", 2024-05-08T08:53:23.978 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:23.978 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.978 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.978 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:23.978 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586018+0000", 2024-05-08T08:53:23.978 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:23.978 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.978 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.978 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:23.978 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586019+0000", 2024-05-08T08:53:23.978 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9999999999999997e-07 2024-05-08T08:53:23.978 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.978 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.978 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:23.978 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586023+0000", 2024-05-08T08:53:23.979 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7330000000000001e-06 2024-05-08T08:53:23.979 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.979 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.979 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:23.979 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586031+0000", 2024-05-08T08:53:23.979 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4940000000000003e-06 2024-05-08T08:53:23.979 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.979 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.979 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:23.979 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586038+0000", 2024-05-08T08:53:23.979 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2539999999999998e-06 2024-05-08T08:53:23.979 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.979 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.979 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:23.979 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622243+0000", 2024-05-08T08:53:23.980 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036205009000000003 2024-05-08T08:53:23.980 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.980 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.980 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:23.980 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622313+0000", 2024-05-08T08:53:23.980 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9912000000000004e-05 2024-05-08T08:53:23.980 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.980 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.980 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:23.980 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622401+0000", 2024-05-08T08:53:23.980 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8232999999999994e-05 2024-05-08T08:53:23.980 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:23.980 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:23.980 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:23.980 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.980 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.981 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4758 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1964032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:23.981 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586080+0000", 2024-05-08T08:53:23.981 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 52.363175521000002, 2024-05-08T08:53:23.981 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036547659000000003, 2024-05-08T08:53:23.981 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:23.981 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:23.981 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:23.981 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:23.981 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:23.981 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:23.981 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4758 2024-05-08T08:53:23.981 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.981 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:23.981 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.981 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:23.981 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586080+0000", 2024-05-08T08:53:23.982 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:23.982 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.982 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.982 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:23.982 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586080+0000", 2024-05-08T08:53:23.982 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:23.982 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.982 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.982 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:23.982 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586081+0000", 2024-05-08T08:53:23.982 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5499999999999998e-07 2024-05-08T08:53:23.982 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.982 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.982 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:23.982 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586084+0000", 2024-05-08T08:53:23.982 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9919999999999999e-06 2024-05-08T08:53:23.983 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.983 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.983 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:23.983 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586091+0000", 2024-05-08T08:53:23.983 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5440000000000001e-06 2024-05-08T08:53:23.983 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.983 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.983 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:23.983 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586098+0000", 2024-05-08T08:53:23.983 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8639999999999998e-06 2024-05-08T08:53:23.983 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.983 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.983 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:23.983 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622442+0000", 2024-05-08T08:53:23.983 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036344110999999998 2024-05-08T08:53:23.983 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.984 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.984 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:23.984 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622516+0000", 2024-05-08T08:53:23.984 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4089000000000002e-05 2024-05-08T08:53:23.984 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.984 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.984 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:23.984 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622628+0000", 2024-05-08T08:53:23.984 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00011140400000000001 2024-05-08T08:53:23.984 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:23.984 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:23.984 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:23.984 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.984 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.984 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4760 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1980416~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:23.985 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586178+0000", 2024-05-08T08:53:23.985 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 52.363076909, 2024-05-08T08:53:23.985 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036647526999999999, 2024-05-08T08:53:23.985 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:23.985 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:23.985 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:23.985 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:23.985 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:23.985 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:23.985 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4760 2024-05-08T08:53:23.985 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.985 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:23.985 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.985 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:23.985 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586178+0000", 2024-05-08T08:53:23.985 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:23.986 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.986 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.986 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:23.986 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586178+0000", 2024-05-08T08:53:23.986 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:23.986 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.986 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.986 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:23.986 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586179+0000", 2024-05-08T08:53:23.986 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8599999999999997e-07 2024-05-08T08:53:23.986 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.986 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.986 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:23.986 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586182+0000", 2024-05-08T08:53:23.986 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.174e-06 2024-05-08T08:53:23.987 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.987 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.987 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:23.987 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586191+0000", 2024-05-08T08:53:23.987 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1869999999999997e-06 2024-05-08T08:53:23.987 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.987 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.987 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:23.987 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586197+0000", 2024-05-08T08:53:23.987 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2090000000000002e-06 2024-05-08T08:53:23.987 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.987 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.987 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:23.987 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622657+0000", 2024-05-08T08:53:23.987 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036459793999999997 2024-05-08T08:53:23.987 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.988 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.988 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:23.988 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622725+0000", 2024-05-08T08:53:23.988 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8374999999999995e-05 2024-05-08T08:53:23.988 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.988 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.988 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:23.988 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622826+0000", 2024-05-08T08:53:23.988 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010100200000000001 2024-05-08T08:53:23.988 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:23.988 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:23.988 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:23.988 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.988 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.988 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4762 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1996800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:23.988 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586299+0000", 2024-05-08T08:53:23.989 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 52.362956578000002, 2024-05-08T08:53:23.989 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036720653999999998, 2024-05-08T08:53:23.989 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:23.989 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:23.989 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:23.989 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:23.989 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:23.989 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:23.989 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4762 2024-05-08T08:53:23.989 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.989 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:23.989 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.989 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:23.989 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586299+0000", 2024-05-08T08:53:23.989 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:23.990 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.990 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.990 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:23.990 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586299+0000", 2024-05-08T08:53:23.990 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:23.990 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.990 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.990 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:23.990 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586299+0000", 2024-05-08T08:53:23.990 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.99e-07 2024-05-08T08:53:23.990 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.990 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.990 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:23.990 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586303+0000", 2024-05-08T08:53:23.990 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1949999999999999e-06 2024-05-08T08:53:23.991 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.991 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.991 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:23.991 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586318+0000", 2024-05-08T08:53:23.991 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5401999999999999e-05 2024-05-08T08:53:23.991 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.991 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.991 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:23.991 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586324+0000", 2024-05-08T08:53:23.991 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8119999999999996e-06 2024-05-08T08:53:23.991 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.991 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.991 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:23.991 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622853+0000", 2024-05-08T08:53:23.992 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036529013999999999 2024-05-08T08:53:23.992 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.992 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.992 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:23.992 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622924+0000", 2024-05-08T08:53:23.992 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1654000000000002e-05 2024-05-08T08:53:23.992 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.992 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.992 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:23.992 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623019+0000", 2024-05-08T08:53:23.992 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4977999999999998e-05 2024-05-08T08:53:23.992 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:23.992 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:23.992 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:23.992 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.993 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.993 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4764 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 2013184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:23.993 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586506+0000", 2024-05-08T08:53:23.993 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 52.362749473000001, 2024-05-08T08:53:23.993 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036859467, 2024-05-08T08:53:23.993 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:23.993 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:23.993 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:23.993 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:23.993 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:23.993 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:23.993 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4764 2024-05-08T08:53:23.993 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.993 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:23.993 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.994 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:23.994 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586506+0000", 2024-05-08T08:53:23.994 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:23.994 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.994 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.994 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:23.994 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586506+0000", 2024-05-08T08:53:23.994 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:23.994 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.994 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.994 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:23.994 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586507+0000", 2024-05-08T08:53:23.995 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9100000000000003e-07 2024-05-08T08:53:23.995 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.995 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.995 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:23.995 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586512+0000", 2024-05-08T08:53:23.995 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9470000000000002e-06 2024-05-08T08:53:23.995 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.995 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.995 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:23.995 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586521+0000", 2024-05-08T08:53:23.995 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3379999999999994e-06 2024-05-08T08:53:23.995 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.995 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.995 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:23.995 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586528+0000", 2024-05-08T08:53:23.996 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1289999999999998e-06 2024-05-08T08:53:23.996 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.996 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.996 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:23.996 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623095+0000", 2024-05-08T08:53:23.996 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036566411 2024-05-08T08:53:23.996 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.996 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.996 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:23.996 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623221+0000", 2024-05-08T08:53:23.996 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012622299999999999 2024-05-08T08:53:23.996 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.996 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.996 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:23.996 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623365+0000", 2024-05-08T08:53:23.996 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000144628 2024-05-08T08:53:23.997 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:23.997 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:23.997 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:23.997 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.997 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.997 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4766 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 2029568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:23.997 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586543+0000", 2024-05-08T08:53:23.997 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 52.362711986000001, 2024-05-08T08:53:23.997 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.037033635000000002, 2024-05-08T08:53:23.997 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:23.997 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:23.997 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:23.997 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:23.997 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:23.997 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:23.997 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4766 2024-05-08T08:53:23.998 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.998 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:23.998 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.998 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:23.998 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586543+0000", 2024-05-08T08:53:23.998 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:23.998 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.998 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.998 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:23.998 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586543+0000", 2024-05-08T08:53:23.998 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:23.998 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.998 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.999 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:23.999 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586544+0000", 2024-05-08T08:53:23.999 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3099999999999998e-07 2024-05-08T08:53:23.999 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.999 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.999 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:23.999 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586547+0000", 2024-05-08T08:53:23.999 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.751e-06 2024-05-08T08:53:23.999 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:23.999 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:23.999 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:23.999 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586553+0000", 2024-05-08T08:53:23.999 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2870000000000001e-06 2024-05-08T08:53:24.000 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.000 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.000 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:24.000 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586558+0000", 2024-05-08T08:53:24.000 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5709999999999997e-06 2024-05-08T08:53:24.000 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.000 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.000 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:24.000 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623416+0000", 2024-05-08T08:53:24.000 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036857613999999997 2024-05-08T08:53:24.000 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.000 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.000 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:24.000 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623489+0000", 2024-05-08T08:53:24.000 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3172999999999999e-05 2024-05-08T08:53:24.001 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.001 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.001 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:24.001 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623577+0000", 2024-05-08T08:53:24.001 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7807999999999997e-05 2024-05-08T08:53:24.001 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:24.001 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:24.001 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:24.001 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.001 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.001 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4768 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 2045952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:24.001 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586566+0000", 2024-05-08T08:53:24.001 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 52.362689680000003, 2024-05-08T08:53:24.001 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.03726206, 2024-05-08T08:53:24.002 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:24.002 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:24.002 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:24.002 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:24.002 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:24.002 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:24.002 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4768 2024-05-08T08:53:24.002 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.002 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:24.002 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.002 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:24.002 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586566+0000", 2024-05-08T08:53:24.002 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:24.002 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.002 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.003 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:24.003 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586566+0000", 2024-05-08T08:53:24.003 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:24.003 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.003 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.003 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:24.003 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586566+0000", 2024-05-08T08:53:24.003 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8299999999999998e-07 2024-05-08T08:53:24.003 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.003 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.003 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:24.003 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586568+0000", 2024-05-08T08:53:24.003 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.063e-06 2024-05-08T08:53:24.003 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.003 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.004 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:24.004 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586574+0000", 2024-05-08T08:53:24.004 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8300000000000001e-06 2024-05-08T08:53:24.004 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.004 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.004 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:24.004 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586579+0000", 2024-05-08T08:53:24.004 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7559999999999999e-06 2024-05-08T08:53:24.004 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.004 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.004 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:24.004 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623612+0000", 2024-05-08T08:53:24.004 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.037033041000000003 2024-05-08T08:53:24.004 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.004 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.005 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:24.005 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623699+0000", 2024-05-08T08:53:24.005 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6817000000000006e-05 2024-05-08T08:53:24.005 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.005 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.005 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:24.005 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623828+0000", 2024-05-08T08:53:24.005 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012917000000000001 2024-05-08T08:53:24.005 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:24.005 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:24.005 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:24.005 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.005 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.005 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4770 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 2062336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:24.005 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586585+0000", 2024-05-08T08:53:24.006 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 52.362670497000003, 2024-05-08T08:53:24.006 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.037449891999999999, 2024-05-08T08:53:24.006 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:24.006 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:24.006 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:24.006 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:24.006 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:24.006 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:24.006 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4770 2024-05-08T08:53:24.006 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.006 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:24.006 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.006 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:24.006 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586585+0000", 2024-05-08T08:53:24.006 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:24.007 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.007 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.007 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:24.007 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586585+0000", 2024-05-08T08:53:24.007 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:24.007 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.007 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.007 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:24.007 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586585+0000", 2024-05-08T08:53:24.007 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:53:24.007 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.007 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.007 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:24.007 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586587+0000", 2024-05-08T08:53:24.007 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.181e-06 2024-05-08T08:53:24.007 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.007 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.008 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:24.008 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586592+0000", 2024-05-08T08:53:24.008 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9710000000000003e-06 2024-05-08T08:53:24.008 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.008 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.008 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:24.008 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586598+0000", 2024-05-08T08:53:24.008 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.118e-06 2024-05-08T08:53:24.008 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.008 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.008 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:24.008 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623862+0000", 2024-05-08T08:53:24.008 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.037264711999999998 2024-05-08T08:53:24.008 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.008 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.008 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:24.009 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623941+0000", 2024-05-08T08:53:24.009 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9084000000000004e-05 2024-05-08T08:53:24.009 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.009 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.009 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:24.009 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.624035+0000", 2024-05-08T08:53:24.009 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3456999999999996e-05 2024-05-08T08:53:24.009 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:24.009 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:24.009 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:24.009 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.009 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.009 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4772 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 2078720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:24.009 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586621+0000", 2024-05-08T08:53:24.009 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 52.362634548999999, 2024-05-08T08:53:24.009 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.037650391999999998, 2024-05-08T08:53:24.010 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:24.010 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:24.010 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:24.010 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:24.010 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:24.010 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:24.010 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4772 2024-05-08T08:53:24.010 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.010 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:24.010 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.010 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:24.010 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586621+0000", 2024-05-08T08:53:24.010 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:24.010 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.010 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.010 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:24.011 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586621+0000", 2024-05-08T08:53:24.011 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:24.011 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.011 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.011 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:24.011 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586621+0000", 2024-05-08T08:53:24.011 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:53:24.011 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.011 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.011 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:24.011 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586623+0000", 2024-05-08T08:53:24.011 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9240000000000001e-06 2024-05-08T08:53:24.011 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.011 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.011 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:24.012 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586628+0000", 2024-05-08T08:53:24.012 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6410000000000001e-06 2024-05-08T08:53:24.012 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.012 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.012 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:24.012 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586633+0000", 2024-05-08T08:53:24.012 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.763e-06 2024-05-08T08:53:24.012 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.012 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.012 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:24.012 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.624077+0000", 2024-05-08T08:53:24.012 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.037444709999999999 2024-05-08T08:53:24.012 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.012 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.012 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:24.012 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.624164+0000", 2024-05-08T08:53:24.013 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6769999999999998e-05 2024-05-08T08:53:24.013 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.013 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.013 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:24.013 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.624271+0000", 2024-05-08T08:53:24.013 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010723 2024-05-08T08:53:24.013 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:24.013 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:24.013 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:24.013 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.013 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.013 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4774 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 2095104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:24.013 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586640+0000", 2024-05-08T08:53:24.013 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 52.362615411, 2024-05-08T08:53:24.013 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.037827178000000003, 2024-05-08T08:53:24.013 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:24.014 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:24.014 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:24.014 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:24.014 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:24.014 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:24.014 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4774 2024-05-08T08:53:24.014 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.014 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:24.014 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.014 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:24.014 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586640+0000", 2024-05-08T08:53:24.014 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:24.014 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.014 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.014 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:24.015 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586640+0000", 2024-05-08T08:53:24.015 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:24.015 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.015 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.015 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:24.015 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586640+0000", 2024-05-08T08:53:24.015 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5199999999999998e-07 2024-05-08T08:53:24.015 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.015 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.015 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:24.015 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586643+0000", 2024-05-08T08:53:24.015 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.232e-06 2024-05-08T08:53:24.015 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.015 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.015 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:24.016 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586649+0000", 2024-05-08T08:53:24.016 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9660000000000001e-06 2024-05-08T08:53:24.016 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.016 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.016 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:24.016 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586653+0000", 2024-05-08T08:53:24.016 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7350000000000004e-06 2024-05-08T08:53:24.016 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.016 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.016 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:24.016 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.624303+0000", 2024-05-08T08:53:24.016 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.037649647000000001 2024-05-08T08:53:24.016 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.016 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.016 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:24.016 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.624373+0000", 2024-05-08T08:53:24.017 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0415999999999998e-05 2024-05-08T08:53:24.017 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.017 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.017 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:24.017 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.624467+0000", 2024-05-08T08:53:24.017 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3830000000000001e-05 2024-05-08T08:53:24.017 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:24.017 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:24.017 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:24.017 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.017 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.017 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4621.0:9089 3.1f e33/18 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:head v 33'117, mlcod=31'115)", 2024-05-08T08:53:24.017 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:53:24.017 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 51.429637198999998, 2024-05-08T08:53:24.017 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.111244524, 2024-05-08T08:53:24.017 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:24.018 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:24.018 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:53:24.018 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:24.018 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.018 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:24.018 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:53:24.018 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:24.018 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.018 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.018 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:24.018 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:53:24.018 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:24.018 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.018 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.018 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:24.018 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:53:24.019 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5900000000000001e-07 2024-05-08T08:53:24.019 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.019 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.019 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:24.019 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539600+0000", 2024-05-08T08:53:24.019 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.019981364000000001 2024-05-08T08:53:24.019 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.019 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.019 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:24.019 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539624+0000", 2024-05-08T08:53:24.019 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4043e-05 2024-05-08T08:53:24.019 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.019 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.019 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:24.019 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539635+0000", 2024-05-08T08:53:24.019 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1511e-05 2024-05-08T08:53:24.020 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.020 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.020 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:24.020 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539728+0000", 2024-05-08T08:53:24.020 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2498999999999996e-05 2024-05-08T08:53:24.020 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.020 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.020 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:24.020 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539746+0000", 2024-05-08T08:53:24.020 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8289000000000001e-05 2024-05-08T08:53:24.020 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.020 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.020 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:53:24.020 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.630822+0000", 2024-05-08T08:53:24.020 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.091076283999999993 2024-05-08T08:53:24.020 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.021 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.021 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:24.021 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.630863+0000", 2024-05-08T08:53:24.021 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0374999999999999e-05 2024-05-08T08:53:24.021 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:24.021 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:24.021 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:24.021 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.021 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.021 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4795.0:230 3.1f e40/18 3:f8db1a56:::rbd_data.11402b26385.0000000000000000:head v 40'171, mlcod=39'168)", 2024-05-08T08:53:24.021 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:45.927773+0000", 2024-05-08T08:53:24.021 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 38.021482116000001, 2024-05-08T08:53:24.021 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045892405999999997, 2024-05-08T08:53:24.021 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:24.021 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:24.022 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:53:24.022 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:24.022 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.022 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:24.022 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.927773+0000", 2024-05-08T08:53:24.022 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:24.022 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.022 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.022 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:24.022 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.927773+0000", 2024-05-08T08:53:24.022 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:24.022 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.022 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.022 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:24.022 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.927774+0000", 2024-05-08T08:53:24.023 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0500000000000002e-07 2024-05-08T08:53:24.023 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.023 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.023 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:24.023 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.931581+0000", 2024-05-08T08:53:24.023 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0038071799999999999 2024-05-08T08:53:24.023 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.023 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.023 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:24.023 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.931599+0000", 2024-05-08T08:53:24.023 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7479e-05 2024-05-08T08:53:24.023 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.023 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.023 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:24.023 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.931624+0000", 2024-05-08T08:53:24.023 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5372999999999999e-05 2024-05-08T08:53:24.023 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.024 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.024 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:24.024 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.931698+0000", 2024-05-08T08:53:24.024 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4043999999999998e-05 2024-05-08T08:53:24.024 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.024 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.024 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:24.024 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.931725+0000", 2024-05-08T08:53:24.024 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6883e-05 2024-05-08T08:53:24.024 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.024 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.024 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:53:24.024 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.973623+0000", 2024-05-08T08:53:24.024 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.041898283000000001 2024-05-08T08:53:24.024 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.024 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.025 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:24.025 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.973666+0000", 2024-05-08T08:53:24.025 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2358999999999999e-05 2024-05-08T08:53:24.025 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:24.025 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:24.025 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:24.025 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.025 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.025 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4195.0:4 3.11 3:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-05-08T08:53:24.025 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:49.764654+0000", 2024-05-08T08:53:24.025 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 34.184601657999998, 2024-05-08T08:53:24.025 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.30714275499999999, 2024-05-08T08:53:24.025 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:24.025 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:24.025 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-05-08T08:53:24.026 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:24.026 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4195", 2024-05-08T08:53:24.026 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/59179521", 2024-05-08T08:53:24.026 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4 2024-05-08T08:53:24.026 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.026 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:24.026 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.026 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:24.026 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764654+0000", 2024-05-08T08:53:24.026 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:24.026 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.026 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.026 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:24.027 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764654+0000", 2024-05-08T08:53:24.027 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:24.027 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.027 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.027 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:24.027 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764655+0000", 2024-05-08T08:53:24.027 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.452e-06 2024-05-08T08:53:24.027 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.027 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.027 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:24.027 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764661+0000", 2024-05-08T08:53:24.027 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5369999999999997e-06 2024-05-08T08:53:24.027 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.027 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.027 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:24.027 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764671+0000", 2024-05-08T08:53:24.028 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.028e-05 2024-05-08T08:53:24.028 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.028 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.028 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:24.028 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764680+0000", 2024-05-08T08:53:24.028 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0629999999999995e-06 2024-05-08T08:53:24.028 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.028 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.028 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:24.028 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764753+0000", 2024-05-08T08:53:24.028 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2679e-05 2024-05-08T08:53:24.028 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.028 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.028 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-05-08T08:53:24.028 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764765+0000", 2024-05-08T08:53:24.028 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2415000000000001e-05 2024-05-08T08:53:24.028 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.029 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.029 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:24.029 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:50.071692+0000", 2024-05-08T08:53:24.029 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.306927324 2024-05-08T08:53:24.029 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.029 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.029 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:24.029 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:50.071796+0000", 2024-05-08T08:53:24.029 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104005 2024-05-08T08:53:24.029 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:24.029 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:24.029 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:24.029 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:24.029 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:24.029 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:53:24.031 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:53:24.031 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:24.034+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:24.058 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T08:53:24.132 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:24.135+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:24.178 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:53:24.178 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:53:24.178 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.178 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4416.0:13462 3.1a 3.ac0e573a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-05-08T08:53:24.179 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:23.480993+0000", 2024-05-08T08:53:24.179 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.70103418500000003, 2024-05-08T08:53:24.179 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.701055541, 2024-05-08T08:53:24.179 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:24.179 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:24.179 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "waiting for peered", 2024-05-08T08:53:24.179 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:24.179 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4416", 2024-05-08T08:53:24.179 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:24.179 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 13462 2024-05-08T08:53:24.179 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.179 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:24.179 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.179 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:24.179 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:23.480993+0000", 2024-05-08T08:53:24.180 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:24.180 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.180 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.180 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:24.180 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:23.480993+0000", 2024-05-08T08:53:24.180 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:24.180 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.180 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.180 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:24.180 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:23.480995+0000", 2024-05-08T08:53:24.180 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.815e-06 2024-05-08T08:53:24.180 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.180 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.180 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:24.180 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:23.481001+0000", 2024-05-08T08:53:24.180 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5950000000000004e-06 2024-05-08T08:53:24.181 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.181 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.181 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:24.181 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:23.481012+0000", 2024-05-08T08:53:24.181 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0694e-05 2024-05-08T08:53:24.181 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.181 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.181 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:24.181 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:23.481020+0000", 2024-05-08T08:53:24.181 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6839999999999996e-06 2024-05-08T08:53:24.181 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.181 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.181 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:24.181 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:23.481098+0000", 2024-05-08T08:53:24.181 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8188000000000004e-05 2024-05-08T08:53:24.182 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:24.182 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:24.182 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-05-08T08:53:24.182 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:23.481108+0000", 2024-05-08T08:53:24.182 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8430000000000005e-06 2024-05-08T08:53:24.182 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:24.182 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:24.182 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:24.182 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:24.182 INFO:teuthology.orchestra.run.smithi192.stdout: ], 2024-05-08T08:53:24.182 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 1 2024-05-08T08:53:24.182 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:53:24.185 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:53:24.185 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:53:24.187 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:53:24.204 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:53:24.204 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:53:24.221 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:53:24.221 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:53:24.239 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:53:24.239 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:53:24.257 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:53:24.258 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:53:24.271 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:24.274+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:24.277 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:53:24.277 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:53:24.334 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:24.337+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:24.334 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:24.334 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:24.343 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:53:24.435 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:24.438+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:24.460 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 5, 4] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-05-08T08:53:24.460 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T08:53:24.461 INFO:tasks.thrashosds.thrasher:reset_purged_snaps_last 2024-05-08T08:53:24.461 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 reset_purged_snaps_last 2024-05-08T08:53:24.474 INFO:teuthology.orchestra.run.smithi059.stderr:noscrub is set 2024-05-08T08:53:24.496 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:24.497 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:24.506 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:53:24.535 INFO:teuthology.orchestra.run.smithi196.stdout:truncating image image_client.0-clone19 from 0x5ff0400 (overlap 0x37d1200) to 0x323c200 2024-05-08T08:53:24.548 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:24.551+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:24.622 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 reset_purged_snaps_last 2024-05-08T08:53:24.661 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:24.662 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:24.666 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:24.669+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:24.671 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:53:24.737 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:53:24.740+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:53:24.763 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 reset_purged_snaps_last 2024-05-08T08:53:24.777 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:24.777 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:24.786 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:53:24.847 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:53:24.850+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:53:24.864 INFO:teuthology.orchestra.run.smithi196.stdout:checking clone #18, image image_client.0-clone18 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent19 2024-05-08T08:53:24.899 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:24.899 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:24.908 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:53:24.939 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:24.942+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:24.974 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 reset_purged_snaps_last 2024-05-08T08:53:25.040 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:25.043+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:25.073 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:25.073 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:25.085 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:53:25.170 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:25.173+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:25.182 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 reset_purged_snaps_last 2024-05-08T08:53:25.266 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:25.269+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:25.289 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:25.289 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:25.303 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:53:25.343 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:25.346+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:25.430 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 reset_purged_snaps_last 2024-05-08T08:53:25.444 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:25.447+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:25.501 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:25.501 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:25.514 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:53:25.545 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:25.548+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:25.638 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:25.638 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:25.646 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:25.649+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:25.647 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:53:25.734 INFO:teuthology.orchestra.run.smithi196.stdout:3058 punch from 0x2c5ae00 to 0x2c5e600, (0x3800 bytes) 2024-05-08T08:53:25.747 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:25.750+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:25.757 INFO:teuthology.orchestra.run.smithi196.stdout:3059 trunc from 0x5ff0400 to 0x308e200 2024-05-08T08:53:25.799 INFO:teuthology.orchestra.run.smithi196.stdout:3061 write 0xa38c000 thru 0xa398690 (0xc691 bytes) 2024-05-08T08:53:25.805 INFO:teuthology.orchestra.run.smithi196.stdout:3062 write 0x2032000 thru 0x20397db (0x77dc bytes) 2024-05-08T08:53:25.805 INFO:teuthology.orchestra.run.smithi196.stdout:3063 write 0x3e0e800 thru 0x3e10a77 (0x2278 bytes) 2024-05-08T08:53:25.805 INFO:teuthology.orchestra.run.smithi196.stdout:3064 write 0x9ad6000 thru 0x9adf5ad (0x95ae bytes) 2024-05-08T08:53:25.805 INFO:teuthology.orchestra.run.smithi196.stdout:3066 punch from 0x45bde00 to 0x45c4200, (0x6400 bytes) 2024-05-08T08:53:25.807 INFO:teuthology.orchestra.run.smithi196.stdout:3067 write 0x6525400 thru 0x6527633 (0x2234 bytes) 2024-05-08T08:53:25.807 INFO:teuthology.orchestra.run.smithi196.stdout:3069 write 0x58fb400 thru 0x5901c31 (0x6832 bytes) 2024-05-08T08:53:25.807 INFO:teuthology.orchestra.run.smithi196.stdout:3071 write 0x8e80c00 thru 0x8e83a2c (0x2e2d bytes) 2024-05-08T08:53:25.807 INFO:teuthology.orchestra.run.smithi196.stdout:3072 write 0x4a23400 thru 0x4a26354 (0x2f55 bytes) 2024-05-08T08:53:25.813 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:25.813 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:25.825 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:53:25.831 INFO:teuthology.orchestra.run.smithi196.stdout:3074 write 0xa764800 thru 0xa771ac0 (0xd2c1 bytes) 2024-05-08T08:53:25.832 INFO:teuthology.orchestra.run.smithi196.stdout:3077 write 0x7d11800 thru 0x7d20a0f (0xf210 bytes) 2024-05-08T08:53:25.832 INFO:teuthology.orchestra.run.smithi196.stdout:3080 punch from 0x3831200 to 0x383b200, (0xa000 bytes) 2024-05-08T08:53:25.834 INFO:teuthology.orchestra.run.smithi196.stdout:3082 punch from 0x4e4fe00 to 0x4e56400, (0x6600 bytes) 2024-05-08T08:53:25.835 INFO:teuthology.orchestra.run.smithi196.stdout:3083 read 0x8245000 thru 0x82526f3 (0xd6f4 bytes) 2024-05-08T08:53:25.835 INFO:teuthology.orchestra.run.smithi196.stdout:3085 read 0x231f200 thru 0x232b24d (0xc04e bytes) 2024-05-08T08:53:25.848 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:25.851+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:25.851 INFO:teuthology.orchestra.run.smithi196.stdout:3087 write 0xc672c00 thru 0xc67cd61 (0xa162 bytes) 2024-05-08T08:53:25.852 INFO:teuthology.orchestra.run.smithi196.stdout:3089 read 0xc2c0c00 thru 0xc2c8615 (0x7a16 bytes) 2024-05-08T08:53:25.853 INFO:teuthology.orchestra.run.smithi196.stdout:3091 read 0x3109400 thru 0x310ffa1 (0x6ba2 bytes) 2024-05-08T08:53:25.853 INFO:teuthology.orchestra.run.smithi196.stdout:3096 write 0x7e87000 thru 0x7e89162 (0x2163 bytes) 2024-05-08T08:53:25.853 INFO:teuthology.orchestra.run.smithi196.stdout:3097 write 0x202e800 thru 0x2031d53 (0x3554 bytes) 2024-05-08T08:53:25.853 INFO:teuthology.orchestra.run.smithi196.stdout:3098 read 0x681200 thru 0x6846e2 (0x34e3 bytes) 2024-05-08T08:53:25.873 INFO:teuthology.orchestra.run.smithi196.stdout:3099 write 0xd50ca00 thru 0xd50dd38 (0x1339 bytes) 2024-05-08T08:53:25.880 INFO:teuthology.orchestra.run.smithi196.stdout:3100 write 0xbe54800 thru 0xbe6280e (0xe00f bytes) 2024-05-08T08:53:25.880 INFO:teuthology.orchestra.run.smithi196.stdout:3101 punch from 0x94f5400 to 0x94f8c00, (0x3800 bytes) 2024-05-08T08:53:25.881 INFO:teuthology.orchestra.run.smithi196.stdout:3104 write 0x1d42000 thru 0x1d4f1b5 (0xd1b6 bytes) 2024-05-08T08:53:25.881 INFO:teuthology.orchestra.run.smithi196.stdout:3107 trunc from 0xd50de00 to 0xdaf1400 2024-05-08T08:53:25.902 INFO:teuthology.orchestra.run.smithi196.stdout:3108 read 0xe69c00 thru 0xe76d7f (0xd180 bytes) 2024-05-08T08:53:25.908 INFO:teuthology.orchestra.run.smithi196.stdout:3111 trunc from 0xdaf1400 to 0xa128a00 2024-05-08T08:53:25.926 INFO:teuthology.orchestra.run.smithi196.stdout:3112 write 0x3fdda00 thru 0x3fe5f0d (0x850e bytes) 2024-05-08T08:53:25.949 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:53:25.952+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:53:25.949 INFO:teuthology.orchestra.run.smithi196.stdout:3116 write 0xcfe7000 thru 0xcfeae82 (0x3e83 bytes) 2024-05-08T08:53:25.963 INFO:teuthology.orchestra.run.smithi196.stdout:3119 write 0xd0af200 thru 0xd0b0c47 (0x1a48 bytes) 2024-05-08T08:53:25.964 INFO:teuthology.orchestra.run.smithi196.stdout:3122 punch from 0x74bba00 to 0x74c5e00, (0xa400 bytes) 2024-05-08T08:53:25.967 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:25.967 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:25.969 INFO:teuthology.orchestra.run.smithi196.stdout:3124 write 0x5c5ba00 thru 0x5c6b60d (0xfc0e bytes) 2024-05-08T08:53:25.969 INFO:teuthology.orchestra.run.smithi196.stdout:3125 punch from 0x32d4a00 to 0x32e0800, (0xbe00 bytes) 2024-05-08T08:53:25.970 INFO:teuthology.orchestra.run.smithi196.stdout:3127 write 0x5ea3800 thru 0x5eb0c95 (0xd496 bytes) 2024-05-08T08:53:25.970 INFO:teuthology.orchestra.run.smithi196.stdout:3129 trunc from 0xd0b0e00 to 0xd0bd600 2024-05-08T08:53:25.979 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:53:25.992 INFO:teuthology.orchestra.run.smithi196.stdout:3130 punch from 0xcfe9000 to 0xcff8400, (0xf400 bytes) 2024-05-08T08:53:26.008 INFO:teuthology.orchestra.run.smithi196.stdout:3132 write 0xd832e00 thru 0xd8415c8 (0xe7c9 bytes) 2024-05-08T08:53:26.009 INFO:teuthology.orchestra.run.smithi196.stdout:3135 write 0xbf5f600 thru 0xbf66151 (0x6b52 bytes) 2024-05-08T08:53:26.021 INFO:teuthology.orchestra.run.smithi196.stdout:3136 trunc from 0xd841600 to 0x31f9a00 2024-05-08T08:53:26.050 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:26.053+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:26.079 INFO:teuthology.orchestra.run.smithi196.stdout:3139 write 0x38e4c00 thru 0x38e5a36 (0xe37 bytes) 2024-05-08T08:53:26.080 INFO:teuthology.orchestra.run.smithi196.stdout:3143 punch from 0x1191c00 to 0x1196c00, (0x5000 bytes) 2024-05-08T08:53:26.085 INFO:teuthology.orchestra.run.smithi196.stdout:3145 write 0x1ea2800 thru 0x1eaad0b (0x850c bytes) 2024-05-08T08:53:26.086 INFO:teuthology.orchestra.run.smithi196.stdout:3147 read 0xe5c000 thru 0xe646a7 (0x86a8 bytes) 2024-05-08T08:53:26.119 INFO:teuthology.orchestra.run.smithi196.stdout:3148 write 0xd06c200 thru 0xd0729d8 (0x67d9 bytes) 2024-05-08T08:53:26.120 INFO:teuthology.orchestra.run.smithi196.stdout:3150 write 0xc522600 thru 0xc526c92 (0x4693 bytes) 2024-05-08T08:53:26.138 INFO:teuthology.orchestra.run.smithi196.stdout:3151 write 0xdafae00 thru 0xdaff798 (0x4999 bytes) 2024-05-08T08:53:26.139 INFO:teuthology.orchestra.run.smithi196.stdout:3154 write 0x7c8d000 thru 0x7c9a71c (0xd71d bytes) 2024-05-08T08:53:26.150 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:26.153+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:26.163 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:26.164 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:26.166 INFO:teuthology.orchestra.run.smithi196.stdout:3156 write 0xe6b1a00 thru 0xe6bbf15 (0xa516 bytes) 2024-05-08T08:53:26.166 INFO:teuthology.orchestra.run.smithi196.stdout:3157 write 0x8afca00 thru 0x8b0786e (0xae6f bytes) 2024-05-08T08:53:26.166 INFO:teuthology.orchestra.run.smithi196.stdout:3159 punch from 0x4f17800 to 0x4f26000, (0xe800 bytes) 2024-05-08T08:53:26.172 INFO:teuthology.orchestra.run.smithi196.stdout:3160 trunc from 0xe6bc000 to 0x9925200 2024-05-08T08:53:26.175 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:53:26.207 INFO:teuthology.orchestra.run.smithi196.stdout:3162 trunc from 0x9925200 to 0x2d93800 2024-05-08T08:53:26.238 INFO:teuthology.orchestra.run.smithi196.stdout:3164 write 0x24e3c00 thru 0x24edfe9 (0xa3ea bytes) 2024-05-08T08:53:26.238 INFO:teuthology.orchestra.run.smithi196.stdout:3167 punch from 0x1f44c00 to 0x1f53400, (0xe800 bytes) 2024-05-08T08:53:26.239 INFO:teuthology.orchestra.run.smithi196.stdout:3171 read 0x2664a00 thru 0x26707c2 (0xbdc3 bytes) 2024-05-08T08:53:26.239 INFO:teuthology.orchestra.run.smithi196.stdout:3174 punch from 0x2dd600 to 0x2dda00, (0x400 bytes) 2024-05-08T08:53:26.251 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:26.254+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:26.260 INFO:teuthology.orchestra.run.smithi196.stdout:3176 write 0x3f2aa00 thru 0x3f2ce25 (0x2426 bytes) 2024-05-08T08:53:26.260 INFO:teuthology.orchestra.run.smithi196.stdout:3177 read 0x3a03c00 thru 0x3a0c496 (0x8897 bytes) 2024-05-08T08:53:26.260 INFO:teuthology.orchestra.run.smithi196.stdout:3178 punch from 0x2b16800 to 0x2b1a600, (0x3e00 bytes) 2024-05-08T08:53:26.284 INFO:teuthology.orchestra.run.smithi196.stdout:3180 write 0xa686c00 thru 0xa68e368 (0x7769 bytes) 2024-05-08T08:53:26.285 INFO:teuthology.orchestra.run.smithi196.stdout:3181 read 0x94a4000 thru 0x94a714f (0x3150 bytes) 2024-05-08T08:53:26.286 INFO:teuthology.orchestra.run.smithi196.stdout:3182 clone 21 order 21 su 8192 sc 9 2024-05-08T08:53:26.288 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T08:53:26.317 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:26.317 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:26.326 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:53:26.367 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:53:26.370+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:53:26.418 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:53:26.418 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:53:26.436 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:53:26.436 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:53:26.453 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:53:26.453 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:53:26.454 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:26.456+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:26.471 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:53:26.471 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:53:26.481 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:26.481 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:53:26.481 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:53:26.482 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:53:26.482 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.482 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:437 3.17 3:ea93a512:::rbd_data.1140100c9703.0000000000000077:head [set-alloc-hint object_size 2097152 write_size 2097152,write 1170432~1024] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e63)", 2024-05-08T08:53:26.482 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:08.390804+0000", 2024-05-08T08:53:26.482 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 18.092171768, 2024-05-08T08:53:26.482 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0406068690000001, 2024-05-08T08:53:26.482 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:26.482 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:26.482 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:53:26.482 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:26.482 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:26.482 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:26.482 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 437 2024-05-08T08:53:26.482 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.483 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:26.483 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.483 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:26.483 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390804+0000", 2024-05-08T08:53:26.483 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:26.483 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.483 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.483 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:26.483 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390804+0000", 2024-05-08T08:53:26.483 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:26.483 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.483 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.483 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:26.483 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390806+0000", 2024-05-08T08:53:26.483 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.666e-06 2024-05-08T08:53:26.483 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.484 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.484 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:26.484 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390818+0000", 2024-05-08T08:53:26.484 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2187999999999999e-05 2024-05-08T08:53:26.484 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.484 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.484 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:26.484 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390834+0000", 2024-05-08T08:53:26.484 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6191000000000001e-05 2024-05-08T08:53:26.484 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.484 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.484 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:26.484 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390845+0000", 2024-05-08T08:53:26.484 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0831e-05 2024-05-08T08:53:26.484 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.484 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.484 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:26.485 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390922+0000", 2024-05-08T08:53:26.485 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7253e-05 2024-05-08T08:53:26.485 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.485 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.485 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op must wait for map", 2024-05-08T08:53:26.485 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390930+0000", 2024-05-08T08:53:26.485 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8259999999999993e-06 2024-05-08T08:53:26.485 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.485 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.485 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:26.485 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392155+0000", 2024-05-08T08:53:26.485 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0012256649999999999 2024-05-08T08:53:26.485 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.485 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.485 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:53:26.485 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392165+0000", 2024-05-08T08:53:26.486 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5799999999999998e-06 2024-05-08T08:53:26.486 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.486 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.486 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:26.486 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.429596+0000", 2024-05-08T08:53:26.486 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0374313639999999 2024-05-08T08:53:26.486 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.486 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.486 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:26.486 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.429728+0000", 2024-05-08T08:53:26.486 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000131552 2024-05-08T08:53:26.486 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.486 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.487 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 0,1", 2024-05-08T08:53:26.487 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.429860+0000", 2024-05-08T08:53:26.487 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000132137 2024-05-08T08:53:26.487 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.487 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.487 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:53:26.487 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.430811+0000", 2024-05-08T08:53:26.487 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00095040200000000004 2024-05-08T08:53:26.487 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.487 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.487 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:53:26.487 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431126+0000", 2024-05-08T08:53:26.487 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000315006 2024-05-08T08:53:26.487 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.487 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.488 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:53:26.488 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431272+0000", 2024-05-08T08:53:26.488 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000146487 2024-05-08T08:53:26.488 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.488 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.488 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:53:26.488 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431315+0000", 2024-05-08T08:53:26.488 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2697000000000001e-05 2024-05-08T08:53:26.488 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.488 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.488 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:26.488 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431411+0000", 2024-05-08T08:53:26.488 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6024000000000001e-05 2024-05-08T08:53:26.488 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:26.488 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:26.488 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:26.488 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.489 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.489 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:439 3.17 3:ea93a512:::rbd_data.1140100c9703.0000000000000077:head [read 1163264~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-05-08T08:53:26.489 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:08.390871+0000", 2024-05-08T08:53:26.489 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 18.092104256999999, 2024-05-08T08:53:26.489 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0408790590000001, 2024-05-08T08:53:26.489 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:26.489 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:26.489 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:26.489 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:26.489 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:26.489 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:26.489 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 439 2024-05-08T08:53:26.489 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.489 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:26.490 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.490 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:26.490 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390871+0000", 2024-05-08T08:53:26.490 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:26.490 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.490 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.490 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:26.490 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390871+0000", 2024-05-08T08:53:26.490 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:26.490 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.490 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.490 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:26.490 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390872+0000", 2024-05-08T08:53:26.490 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6300000000000005e-07 2024-05-08T08:53:26.490 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.491 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.491 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:26.491 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390875+0000", 2024-05-08T08:53:26.491 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3459999999999998e-06 2024-05-08T08:53:26.491 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.491 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.491 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:26.491 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390891+0000", 2024-05-08T08:53:26.491 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5255e-05 2024-05-08T08:53:26.491 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.491 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.491 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:26.491 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390897+0000", 2024-05-08T08:53:26.491 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2360000000000001e-06 2024-05-08T08:53:26.491 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.491 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.492 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:26.492 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390995+0000", 2024-05-08T08:53:26.492 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8522000000000003e-05 2024-05-08T08:53:26.492 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.492 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.492 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting_for_map not empty", 2024-05-08T08:53:26.492 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.391005+0000", 2024-05-08T08:53:26.492 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1460000000000002e-06 2024-05-08T08:53:26.492 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.492 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.492 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:26.492 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392191+0000", 2024-05-08T08:53:26.492 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0011860270000000001 2024-05-08T08:53:26.492 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.492 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.492 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:53:26.493 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392197+0000", 2024-05-08T08:53:26.493 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4289999999999998e-06 2024-05-08T08:53:26.493 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.493 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.493 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:26.493 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.430596+0000", 2024-05-08T08:53:26.493 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0383989090000001 2024-05-08T08:53:26.493 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.493 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.493 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for rw locks", 2024-05-08T08:53:26.493 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.430676+0000", 2024-05-08T08:53:26.493 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9781000000000001e-05 2024-05-08T08:53:26.493 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.493 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.493 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:26.494 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431460+0000", 2024-05-08T08:53:26.494 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00078427300000000004 2024-05-08T08:53:26.494 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.494 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.494 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:26.494 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431540+0000", 2024-05-08T08:53:26.494 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0224999999999998e-05 2024-05-08T08:53:26.494 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.494 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.494 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:26.494 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431750+0000", 2024-05-08T08:53:26.494 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00021024699999999999 2024-05-08T08:53:26.494 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:26.495 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:26.495 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:26.495 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.495 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.495 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:945 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 4096000~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:26.495 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:53:26.495 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 17.031954689999999, 2024-05-08T08:53:26.495 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086774741000000002, 2024-05-08T08:53:26.495 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:26.495 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:26.495 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:26.495 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:26.495 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:26.495 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:26.495 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 945 2024-05-08T08:53:26.495 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.496 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:26.496 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.496 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:26.496 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:53:26.496 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:26.496 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.496 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.496 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:26.496 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:53:26.496 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:26.496 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.496 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.496 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:26.496 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:53:26.496 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3499999999999996e-07 2024-05-08T08:53:26.496 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.497 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.497 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:26.497 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451024+0000", 2024-05-08T08:53:26.497 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.627e-06 2024-05-08T08:53:26.497 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.497 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.497 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:26.497 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451037+0000", 2024-05-08T08:53:26.497 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2431e-05 2024-05-08T08:53:26.497 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.497 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.497 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:26.497 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451042+0000", 2024-05-08T08:53:26.497 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8409999999999996e-06 2024-05-08T08:53:26.498 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.498 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.498 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:26.498 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537608+0000", 2024-05-08T08:53:26.498 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086565243 2024-05-08T08:53:26.498 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.498 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.498 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:26.498 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537687+0000", 2024-05-08T08:53:26.498 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9519999999999998e-05 2024-05-08T08:53:26.498 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.498 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.498 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:26.498 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537796+0000", 2024-05-08T08:53:26.498 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000108544 2024-05-08T08:53:26.499 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:26.499 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:26.499 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:26.499 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.499 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.499 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:964 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 458752~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:26.499 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451051+0000", 2024-05-08T08:53:26.499 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 17.031924517, 2024-05-08T08:53:26.499 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086990490000000004, 2024-05-08T08:53:26.499 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:26.499 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:26.499 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:26.499 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:26.499 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:26.499 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:26.499 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 964 2024-05-08T08:53:26.500 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.500 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:26.500 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.500 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:26.500 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451051+0000", 2024-05-08T08:53:26.500 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:26.500 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.500 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.500 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:26.500 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451051+0000", 2024-05-08T08:53:26.500 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:26.500 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.500 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.500 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:26.500 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451052+0000", 2024-05-08T08:53:26.501 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7299999999999996e-07 2024-05-08T08:53:26.501 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.501 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.501 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:26.501 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451055+0000", 2024-05-08T08:53:26.501 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.2770000000000001e-06 2024-05-08T08:53:26.501 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.501 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.501 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:26.501 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451061+0000", 2024-05-08T08:53:26.501 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4060000000000002e-06 2024-05-08T08:53:26.501 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.501 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.501 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:26.501 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451066+0000", 2024-05-08T08:53:26.501 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0839999999999999e-06 2024-05-08T08:53:26.501 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.502 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.502 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:26.502 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537839+0000", 2024-05-08T08:53:26.502 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086772962999999995 2024-05-08T08:53:26.502 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.502 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.502 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:26.502 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537934+0000", 2024-05-08T08:53:26.502 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4742999999999997e-05 2024-05-08T08:53:26.502 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.502 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.502 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:26.502 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538042+0000", 2024-05-08T08:53:26.502 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000107444 2024-05-08T08:53:26.502 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:26.503 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:26.503 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:26.503 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.503 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.503 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:965 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 950272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:26.503 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451075+0000", 2024-05-08T08:53:26.503 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 17.031900454999999, 2024-05-08T08:53:26.503 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087230196999999995, 2024-05-08T08:53:26.503 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:26.503 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:26.503 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:26.503 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:26.503 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:26.503 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:26.503 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 965 2024-05-08T08:53:26.503 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.503 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:26.504 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.504 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:26.504 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451075+0000", 2024-05-08T08:53:26.504 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:26.504 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.504 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.504 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:26.504 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451075+0000", 2024-05-08T08:53:26.504 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:26.504 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.504 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.504 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:26.504 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451076+0000", 2024-05-08T08:53:26.504 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5000000000000002e-07 2024-05-08T08:53:26.504 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.504 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.504 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:26.505 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451078+0000", 2024-05-08T08:53:26.505 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.5639999999999999e-06 2024-05-08T08:53:26.505 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.505 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.505 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:26.505 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451085+0000", 2024-05-08T08:53:26.505 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5640000000000002e-06 2024-05-08T08:53:26.505 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.505 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.505 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:26.505 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451090+0000", 2024-05-08T08:53:26.505 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1699999999999996e-06 2024-05-08T08:53:26.505 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.505 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.505 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:26.505 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538106+0000", 2024-05-08T08:53:26.505 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087015888999999999 2024-05-08T08:53:26.506 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.506 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.506 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:26.506 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538190+0000", 2024-05-08T08:53:26.506 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3961999999999993e-05 2024-05-08T08:53:26.506 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.506 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.506 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:26.506 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538305+0000", 2024-05-08T08:53:26.506 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011529800000000001 2024-05-08T08:53:26.506 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:26.506 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:26.506 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:26.506 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.506 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.506 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:966 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 1441792~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:26.507 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:53:26.507 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 17.031877082000001, 2024-05-08T08:53:26.507 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087446323000000006, 2024-05-08T08:53:26.507 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:26.507 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:26.507 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:26.507 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:26.507 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:26.507 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:26.507 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 966 2024-05-08T08:53:26.507 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.507 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:26.507 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.507 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:26.507 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:53:26.507 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:26.507 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.508 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.508 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:26.508 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:53:26.508 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:26.508 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.508 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.508 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:26.508 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:53:26.508 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3700000000000001e-07 2024-05-08T08:53:26.508 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.508 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.508 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:26.508 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451101+0000", 2024-05-08T08:53:26.508 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2560000000000001e-06 2024-05-08T08:53:26.508 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.508 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.509 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:26.509 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451107+0000", 2024-05-08T08:53:26.509 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1070000000000001e-06 2024-05-08T08:53:26.509 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.509 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.509 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:26.509 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451112+0000", 2024-05-08T08:53:26.509 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1420000000000001e-06 2024-05-08T08:53:26.509 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.509 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.509 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:26.509 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538341+0000", 2024-05-08T08:53:26.509 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087228302999999993 2024-05-08T08:53:26.509 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.509 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.509 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:26.509 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538424+0000", 2024-05-08T08:53:26.510 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3497000000000004e-05 2024-05-08T08:53:26.510 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.510 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.510 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:26.510 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538545+0000", 2024-05-08T08:53:26.510 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000120581 2024-05-08T08:53:26.510 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:26.510 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:26.510 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:26.510 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.510 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.510 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:967 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:26.510 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451120+0000", 2024-05-08T08:53:26.510 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 17.031855607000001, 2024-05-08T08:53:26.511 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087632725999999994, 2024-05-08T08:53:26.511 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:26.511 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:26.511 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:26.511 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:26.511 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:26.511 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:26.511 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 967 2024-05-08T08:53:26.511 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.511 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:26.511 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.511 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:26.511 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451120+0000", 2024-05-08T08:53:26.511 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:26.511 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.511 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.512 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:26.512 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451120+0000", 2024-05-08T08:53:26.512 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:26.512 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.512 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.512 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:26.512 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451121+0000", 2024-05-08T08:53:26.512 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8800000000000002e-07 2024-05-08T08:53:26.512 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.512 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.512 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:26.512 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451130+0000", 2024-05-08T08:53:26.512 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2299999999999997e-06 2024-05-08T08:53:26.512 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.512 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.512 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:26.513 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451137+0000", 2024-05-08T08:53:26.513 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4669999999999998e-06 2024-05-08T08:53:26.513 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.513 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.513 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:26.513 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451143+0000", 2024-05-08T08:53:26.513 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2839999999999997e-06 2024-05-08T08:53:26.513 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.513 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.513 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:26.513 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538575+0000", 2024-05-08T08:53:26.513 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087432789999999996 2024-05-08T08:53:26.513 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.513 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.513 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:26.513 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538648+0000", 2024-05-08T08:53:26.513 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2877e-05 2024-05-08T08:53:26.514 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.514 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.514 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:26.514 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538753+0000", 2024-05-08T08:53:26.514 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010449 2024-05-08T08:53:26.514 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:26.514 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:26.514 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:26.514 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.514 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.514 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:968 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 2424832~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:26.514 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451152+0000", 2024-05-08T08:53:26.514 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 17.031823522, 2024-05-08T08:53:26.514 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087813325999999997, 2024-05-08T08:53:26.514 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:26.515 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:26.515 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:26.515 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:26.515 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:26.515 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:26.515 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 968 2024-05-08T08:53:26.515 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.515 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:26.515 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.515 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:26.515 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451152+0000", 2024-05-08T08:53:26.515 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:26.515 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.515 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.515 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:26.515 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451152+0000", 2024-05-08T08:53:26.516 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:26.516 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.516 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.516 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:26.516 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451153+0000", 2024-05-08T08:53:26.516 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3099999999999998e-07 2024-05-08T08:53:26.516 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.516 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.516 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:26.516 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451155+0000", 2024-05-08T08:53:26.516 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4760000000000001e-06 2024-05-08T08:53:26.516 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.516 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.516 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:26.516 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451162+0000", 2024-05-08T08:53:26.516 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5969999999999997e-06 2024-05-08T08:53:26.517 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.517 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.517 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:26.517 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451167+0000", 2024-05-08T08:53:26.517 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.977e-06 2024-05-08T08:53:26.517 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.517 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.517 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:26.517 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538781+0000", 2024-05-08T08:53:26.517 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087614846999999996 2024-05-08T08:53:26.517 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.517 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.517 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:26.517 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538857+0000", 2024-05-08T08:53:26.517 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5358000000000006e-05 2024-05-08T08:53:26.517 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.517 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.518 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:26.518 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538965+0000", 2024-05-08T08:53:26.518 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010864 2024-05-08T08:53:26.518 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:26.518 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:26.518 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:26.518 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.518 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.518 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:969 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 2916352~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:26.518 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:53:26.518 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 17.031801817000002, 2024-05-08T08:53:26.518 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087998980000000004, 2024-05-08T08:53:26.518 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:26.518 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:26.519 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:26.519 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:26.519 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:26.519 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:26.519 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 969 2024-05-08T08:53:26.519 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.519 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:26.519 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.519 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:26.519 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:53:26.519 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:26.519 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.519 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.519 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:26.519 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:53:26.519 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:26.519 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.520 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.520 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:26.520 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:53:26.520 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3599999999999999e-07 2024-05-08T08:53:26.520 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.520 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.520 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:26.520 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451177+0000", 2024-05-08T08:53:26.520 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4590000000000001e-06 2024-05-08T08:53:26.520 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.520 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.520 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:26.520 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451184+0000", 2024-05-08T08:53:26.520 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4109999999999999e-06 2024-05-08T08:53:26.520 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.521 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.521 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:26.521 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451191+0000", 2024-05-08T08:53:26.521 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7279999999999998e-06 2024-05-08T08:53:26.521 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.521 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.521 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:26.521 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538997+0000", 2024-05-08T08:53:26.521 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087806170000000003 2024-05-08T08:53:26.521 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.521 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.521 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:26.521 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539069+0000", 2024-05-08T08:53:26.521 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2373999999999995e-05 2024-05-08T08:53:26.521 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.521 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.522 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:26.522 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539173+0000", 2024-05-08T08:53:26.522 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000103402 2024-05-08T08:53:26.522 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:26.522 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:26.522 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:26.522 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.522 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.522 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:970 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 3407872~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:26.522 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451198+0000", 2024-05-08T08:53:26.522 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 17.031777219999999, 2024-05-08T08:53:26.522 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088183368999999998, 2024-05-08T08:53:26.522 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:26.523 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:26.523 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:26.523 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:26.523 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:26.523 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:26.523 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 970 2024-05-08T08:53:26.523 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.523 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:26.523 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.523 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:26.523 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451198+0000", 2024-05-08T08:53:26.523 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:26.523 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.523 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.523 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:26.523 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451198+0000", 2024-05-08T08:53:26.523 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:26.524 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.524 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.524 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:26.524 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451199+0000", 2024-05-08T08:53:26.524 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7400000000000003e-07 2024-05-08T08:53:26.524 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.524 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.524 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:26.524 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451211+0000", 2024-05-08T08:53:26.524 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2109999999999999e-05 2024-05-08T08:53:26.524 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.524 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.524 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:26.524 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451219+0000", 2024-05-08T08:53:26.524 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4170000000000003e-06 2024-05-08T08:53:26.524 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.524 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.525 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:26.525 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451224+0000", 2024-05-08T08:53:26.525 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3990000000000003e-06 2024-05-08T08:53:26.525 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.525 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.525 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:26.525 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539204+0000", 2024-05-08T08:53:26.525 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087980099000000006 2024-05-08T08:53:26.525 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.525 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.525 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:26.525 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539276+0000", 2024-05-08T08:53:26.525 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1513000000000005e-05 2024-05-08T08:53:26.525 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.525 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.525 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:26.526 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539382+0000", 2024-05-08T08:53:26.526 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010615699999999999 2024-05-08T08:53:26.526 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:26.526 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:26.526 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:26.526 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.526 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.526 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:971 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 3899392~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:26.526 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451233+0000", 2024-05-08T08:53:26.526 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 17.031742513000001, 2024-05-08T08:53:26.526 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088431525999999996, 2024-05-08T08:53:26.526 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:26.526 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:26.526 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:26.526 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:26.526 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:26.527 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:26.527 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 971 2024-05-08T08:53:26.527 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.527 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:26.527 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.527 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:26.527 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451233+0000", 2024-05-08T08:53:26.527 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:26.527 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.527 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.527 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:26.527 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451233+0000", 2024-05-08T08:53:26.527 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:26.527 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.527 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.527 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:26.527 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451234+0000", 2024-05-08T08:53:26.528 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8500000000000001e-07 2024-05-08T08:53:26.528 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.528 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.528 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:26.528 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451237+0000", 2024-05-08T08:53:26.528 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8320000000000002e-06 2024-05-08T08:53:26.528 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.528 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.528 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:26.528 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451243+0000", 2024-05-08T08:53:26.528 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6749999999999996e-06 2024-05-08T08:53:26.528 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.528 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.528 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:26.528 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451249+0000", 2024-05-08T08:53:26.528 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1229999999999997e-06 2024-05-08T08:53:26.528 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.529 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.529 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:26.529 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539412+0000", 2024-05-08T08:53:26.529 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088162668999999999 2024-05-08T08:53:26.529 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.529 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.529 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:26.529 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539505+0000", 2024-05-08T08:53:26.529 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3240999999999997e-05 2024-05-08T08:53:26.529 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.529 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.529 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:26.529 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539665+0000", 2024-05-08T08:53:26.529 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00015930100000000001 2024-05-08T08:53:26.529 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:26.529 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:26.529 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:26.530 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.530 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.530 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:972 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 196608~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:26.530 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451257+0000", 2024-05-08T08:53:26.530 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 17.031718567999999, 2024-05-08T08:53:26.530 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088719928000000003, 2024-05-08T08:53:26.530 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:26.530 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:26.530 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:26.530 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:26.530 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:26.530 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:26.530 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 972 2024-05-08T08:53:26.530 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.530 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:26.531 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.531 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:26.531 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451257+0000", 2024-05-08T08:53:26.531 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:26.531 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.531 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.531 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:26.531 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451257+0000", 2024-05-08T08:53:26.531 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:26.531 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.531 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.531 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:26.531 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451258+0000", 2024-05-08T08:53:26.531 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7999999999999996e-07 2024-05-08T08:53:26.531 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.532 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.532 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:26.532 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451260+0000", 2024-05-08T08:53:26.532 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4609999999999998e-06 2024-05-08T08:53:26.532 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.532 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.532 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:26.532 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451267+0000", 2024-05-08T08:53:26.532 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.81e-06 2024-05-08T08:53:26.532 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.532 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.532 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:26.532 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451273+0000", 2024-05-08T08:53:26.532 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0859999999999998e-06 2024-05-08T08:53:26.532 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.533 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.533 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:26.533 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539718+0000", 2024-05-08T08:53:26.533 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088445059000000006 2024-05-08T08:53:26.533 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.533 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.533 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:26.533 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539834+0000", 2024-05-08T08:53:26.533 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000116426 2024-05-08T08:53:26.533 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.533 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.533 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:26.533 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539977+0000", 2024-05-08T08:53:26.533 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014260600000000001 2024-05-08T08:53:26.533 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:26.534 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:26.534 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:26.534 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.534 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.534 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:973 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 688128~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:26.534 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:53:26.534 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 17.031695983999999, 2024-05-08T08:53:26.534 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088918496, 2024-05-08T08:53:26.534 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:26.534 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:26.534 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:26.534 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:26.534 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:26.534 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:26.534 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 973 2024-05-08T08:53:26.534 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.535 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:26.535 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.535 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:26.535 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:53:26.535 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:26.535 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.535 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.535 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:26.535 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:53:26.535 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:26.535 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.535 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.535 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:26.535 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:53:26.535 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9099999999999999e-07 2024-05-08T08:53:26.535 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.536 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.536 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:26.536 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451282+0000", 2024-05-08T08:53:26.536 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1550000000000001e-06 2024-05-08T08:53:26.536 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.536 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.536 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:26.536 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451288+0000", 2024-05-08T08:53:26.536 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8889999999999999e-06 2024-05-08T08:53:26.536 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.536 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.536 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:26.536 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451294+0000", 2024-05-08T08:53:26.536 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0850000000000002e-06 2024-05-08T08:53:26.536 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.536 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.537 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:26.537 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540011+0000", 2024-05-08T08:53:26.537 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088717314000000005 2024-05-08T08:53:26.537 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.537 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.537 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:26.537 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540089+0000", 2024-05-08T08:53:26.537 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7438e-05 2024-05-08T08:53:26.537 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.537 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.537 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:26.537 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540198+0000", 2024-05-08T08:53:26.537 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000109224 2024-05-08T08:53:26.537 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:26.537 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:26.537 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:26.537 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.538 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.538 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:974 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 1179648~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:26.538 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451301+0000", 2024-05-08T08:53:26.538 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 17.031674350999999, 2024-05-08T08:53:26.538 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089105078000000004, 2024-05-08T08:53:26.538 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:26.538 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:26.538 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:26.538 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:26.538 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:26.538 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:26.538 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 974 2024-05-08T08:53:26.538 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.538 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:26.538 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.538 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:26.539 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451301+0000", 2024-05-08T08:53:26.539 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:26.539 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.539 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.539 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:26.539 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451301+0000", 2024-05-08T08:53:26.539 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:26.539 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.539 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.539 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:26.539 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451302+0000", 2024-05-08T08:53:26.539 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7800000000000002e-07 2024-05-08T08:53:26.539 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.539 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.539 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:26.539 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451304+0000", 2024-05-08T08:53:26.540 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.65e-06 2024-05-08T08:53:26.540 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.540 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.540 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:26.540 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451311+0000", 2024-05-08T08:53:26.540 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.472e-06 2024-05-08T08:53:26.540 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.540 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.540 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:26.540 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451316+0000", 2024-05-08T08:53:26.540 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9189999999999998e-06 2024-05-08T08:53:26.540 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.540 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.540 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:26.540 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540234+0000", 2024-05-08T08:53:26.540 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088917752000000003 2024-05-08T08:53:26.541 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.541 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.541 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:26.541 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540308+0000", 2024-05-08T08:53:26.541 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4758000000000005e-05 2024-05-08T08:53:26.541 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.541 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.541 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:26.541 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540406+0000", 2024-05-08T08:53:26.541 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8048999999999996e-05 2024-05-08T08:53:26.541 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:26.541 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:26.541 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:26.541 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.541 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.541 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:975 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 1671168~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:26.542 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:53:26.542 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 17.031632021, 2024-05-08T08:53:26.542 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089294139999999994, 2024-05-08T08:53:26.542 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:26.542 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:26.542 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:26.542 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:26.542 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:26.542 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:26.542 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 975 2024-05-08T08:53:26.542 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.542 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:26.542 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.542 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:26.542 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:53:26.543 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:26.543 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.543 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.543 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:26.543 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:53:26.543 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:26.543 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.543 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.543 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:26.543 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:53:26.543 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1299999999999999e-07 2024-05-08T08:53:26.543 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.543 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.543 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:26.543 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451348+0000", 2024-05-08T08:53:26.544 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.484e-06 2024-05-08T08:53:26.544 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.544 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.544 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:26.544 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451356+0000", 2024-05-08T08:53:26.544 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8499999999999994e-06 2024-05-08T08:53:26.544 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.544 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.544 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:26.544 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451366+0000", 2024-05-08T08:53:26.544 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.9850000000000001e-06 2024-05-08T08:53:26.544 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.544 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.544 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:26.544 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540448+0000", 2024-05-08T08:53:26.544 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089082254 2024-05-08T08:53:26.545 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.545 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.545 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:26.545 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540528+0000", 2024-05-08T08:53:26.545 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0234999999999993e-05 2024-05-08T08:53:26.545 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.545 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.545 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:26.545 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540638+0000", 2024-05-08T08:53:26.545 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010951900000000001 2024-05-08T08:53:26.545 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:26.545 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:26.545 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:26.545 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.545 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.545 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:976 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 2162688~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:26.546 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451375+0000", 2024-05-08T08:53:26.546 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 17.031600273999999, 2024-05-08T08:53:26.546 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089472745000000006, 2024-05-08T08:53:26.546 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:26.546 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:26.546 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:26.546 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:26.546 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:26.546 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:26.546 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 976 2024-05-08T08:53:26.546 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.546 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:26.546 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.546 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:26.546 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451375+0000", 2024-05-08T08:53:26.546 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:26.546 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.547 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.547 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:26.547 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451375+0000", 2024-05-08T08:53:26.547 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:26.547 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.547 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.547 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:26.547 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451376+0000", 2024-05-08T08:53:26.547 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2399999999999998e-07 2024-05-08T08:53:26.547 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.547 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.547 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:26.547 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451379+0000", 2024-05-08T08:53:26.547 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6359999999999998e-06 2024-05-08T08:53:26.547 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.547 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.548 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:26.548 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451387+0000", 2024-05-08T08:53:26.548 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9929999999999995e-06 2024-05-08T08:53:26.548 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.548 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.548 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:26.548 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451393+0000", 2024-05-08T08:53:26.548 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4699999999999999e-06 2024-05-08T08:53:26.548 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.548 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.548 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:26.548 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540677+0000", 2024-05-08T08:53:26.548 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089283504999999999 2024-05-08T08:53:26.548 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.548 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.548 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:26.549 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540743+0000", 2024-05-08T08:53:26.549 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6303999999999994e-05 2024-05-08T08:53:26.549 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.549 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.549 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:26.549 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540848+0000", 2024-05-08T08:53:26.549 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000105213 2024-05-08T08:53:26.549 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:26.549 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:26.549 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:26.549 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.549 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.549 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:977 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 2654208~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:26.549 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:53:26.549 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 17.031574842000001, 2024-05-08T08:53:26.549 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089698473000000001, 2024-05-08T08:53:26.549 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:26.550 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:26.550 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:26.550 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:26.550 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:26.550 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:26.550 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 977 2024-05-08T08:53:26.550 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.550 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:26.550 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.550 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:26.550 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:53:26.550 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:26.550 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.550 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.550 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:26.550 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:53:26.551 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:26.551 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.551 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.551 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:26.551 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:53:26.551 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2899999999999999e-07 2024-05-08T08:53:26.551 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.551 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.551 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:26.551 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451404+0000", 2024-05-08T08:53:26.551 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6649999999999999e-06 2024-05-08T08:53:26.551 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.551 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.551 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:26.551 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451410+0000", 2024-05-08T08:53:26.551 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3269999999999997e-06 2024-05-08T08:53:26.552 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.552 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.552 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:26.552 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451415+0000", 2024-05-08T08:53:26.552 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9529999999999998e-06 2024-05-08T08:53:26.552 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.552 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.552 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:26.552 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540912+0000", 2024-05-08T08:53:26.552 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089496950000000006 2024-05-08T08:53:26.552 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.552 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.552 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:26.552 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541000+0000", 2024-05-08T08:53:26.552 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7429000000000006e-05 2024-05-08T08:53:26.552 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.553 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.553 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:26.553 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541099+0000", 2024-05-08T08:53:26.553 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.9720000000000001e-05 2024-05-08T08:53:26.553 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:26.553 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:26.553 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:26.553 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.553 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.553 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:978 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 3145728~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:26.553 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:53:26.553 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 17.031553634000002, 2024-05-08T08:53:26.553 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089900635000000007, 2024-05-08T08:53:26.553 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:26.553 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:26.554 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:26.554 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:26.554 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:26.554 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:26.554 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 978 2024-05-08T08:53:26.554 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.554 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:26.554 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.554 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:26.554 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:53:26.554 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:26.554 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.554 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.554 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:26.554 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:53:26.554 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:26.555 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.555 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.555 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:26.555 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:53:26.555 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9400000000000001e-07 2024-05-08T08:53:26.555 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.555 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.555 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:26.555 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451425+0000", 2024-05-08T08:53:26.555 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2460000000000002e-06 2024-05-08T08:53:26.555 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.555 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.555 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:26.555 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451431+0000", 2024-05-08T08:53:26.555 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2419999999999997e-06 2024-05-08T08:53:26.555 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.556 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.556 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:26.556 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451436+0000", 2024-05-08T08:53:26.556 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.887e-06 2024-05-08T08:53:26.556 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.556 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.556 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:26.556 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541152+0000", 2024-05-08T08:53:26.556 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089716194999999999 2024-05-08T08:53:26.556 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.556 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.556 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:26.556 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541224+0000", 2024-05-08T08:53:26.556 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2416000000000006e-05 2024-05-08T08:53:26.556 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.556 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.557 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:26.557 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541323+0000", 2024-05-08T08:53:26.557 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8255e-05 2024-05-08T08:53:26.557 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:26.557 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:26.557 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:26.557 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.557 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.557 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:979 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 3637248~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:26.557 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451462+0000", 2024-05-08T08:53:26.557 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 17.031513185000001, 2024-05-08T08:53:26.557 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.090036441999999994, 2024-05-08T08:53:26.557 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:26.557 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:26.557 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:26.557 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:26.557 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:26.558 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:26.558 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 979 2024-05-08T08:53:26.558 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.558 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:26.558 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.558 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:26.558 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451462+0000", 2024-05-08T08:53:26.558 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:26.558 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.558 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.558 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:26.558 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451462+0000", 2024-05-08T08:53:26.558 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:26.558 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.558 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.559 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:26.559 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451463+0000", 2024-05-08T08:53:26.559 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4300000000000002e-07 2024-05-08T08:53:26.559 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.559 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.559 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:26.559 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451471+0000", 2024-05-08T08:53:26.559 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0800000000000006e-06 2024-05-08T08:53:26.559 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.559 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.559 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:26.559 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451481+0000", 2024-05-08T08:53:26.559 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.38e-06 2024-05-08T08:53:26.559 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.559 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.559 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:26.560 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451487+0000", 2024-05-08T08:53:26.560 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4590000000000003e-06 2024-05-08T08:53:26.560 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.560 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.560 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:26.560 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541350+0000", 2024-05-08T08:53:26.560 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089862599000000001 2024-05-08T08:53:26.560 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.560 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.560 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:26.560 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541407+0000", 2024-05-08T08:53:26.560 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7667999999999997e-05 2024-05-08T08:53:26.560 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.560 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.560 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:26.561 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541499+0000", 2024-05-08T08:53:26.561 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1513000000000003e-05 2024-05-08T08:53:26.561 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:26.561 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:26.561 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:26.561 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.561 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.561 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:980 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 4128768~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:26.561 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451496+0000", 2024-05-08T08:53:26.561 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 17.031479429000001, 2024-05-08T08:53:26.561 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.090258985, 2024-05-08T08:53:26.561 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:26.561 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:26.561 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:26.561 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:26.561 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:26.562 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:26.562 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 980 2024-05-08T08:53:26.562 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.562 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:26.562 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.562 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:26.562 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451496+0000", 2024-05-08T08:53:26.562 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:26.562 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.562 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.562 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:26.562 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451496+0000", 2024-05-08T08:53:26.562 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:26.563 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.563 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.563 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:26.563 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451497+0000", 2024-05-08T08:53:26.563 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7599999999999997e-07 2024-05-08T08:53:26.563 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.563 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.563 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:26.563 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451500+0000", 2024-05-08T08:53:26.563 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7930000000000002e-06 2024-05-08T08:53:26.563 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.563 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.563 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:26.563 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451506+0000", 2024-05-08T08:53:26.563 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6980000000000001e-06 2024-05-08T08:53:26.563 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.564 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.564 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:26.564 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451512+0000", 2024-05-08T08:53:26.564 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2490000000000001e-06 2024-05-08T08:53:26.564 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.564 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.564 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:26.564 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541550+0000", 2024-05-08T08:53:26.564 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.090037961999999999 2024-05-08T08:53:26.564 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.564 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.564 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:26.564 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541647+0000", 2024-05-08T08:53:26.564 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6924999999999998e-05 2024-05-08T08:53:26.564 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:26.564 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:26.565 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:26.565 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541755+0000", 2024-05-08T08:53:26.565 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000108682 2024-05-08T08:53:26.565 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:26.565 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:26.565 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:26.565 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:26.565 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:26.565 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:26.565 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-05-08T08:53:26.567 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:53:26.567 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:53:26.568 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:53:26.568 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:53:26.569 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:53:26.625 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:26.570+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:26.635 INFO:teuthology.orchestra.run.smithi196.stdout:leaving image image_client.0-clone20 intact 2024-05-08T08:53:26.668 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:26.671+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:26.675 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:53:26.675 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:53:26.676 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:53:26.676 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:53:26.684 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:53:26.769 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:26.772+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:26.835 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:53:26.836 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:53:26.836 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:53:26.836 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:53:26.836 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:53:26.844 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:53:26.886 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:26.889+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:26.970 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:26.973+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:27.003 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:53:27.004 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:53:27.004 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:53:27.004 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:53:27.004 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.004 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4742 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1832960~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:27.004 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.585513+0000", 2024-05-08T08:53:27.004 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 55.420474900000002, 2024-05-08T08:53:27.004 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035588763000000002, 2024-05-08T08:53:27.004 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:27.004 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:27.004 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:27.004 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:27.005 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:27.005 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:27.005 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4742 2024-05-08T08:53:27.005 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.005 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:27.005 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.005 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:27.005 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585513+0000", 2024-05-08T08:53:27.005 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:27.005 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.005 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.005 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:27.005 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585513+0000", 2024-05-08T08:53:27.005 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:27.006 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.006 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.006 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:27.006 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585514+0000", 2024-05-08T08:53:27.006 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6100000000000001e-07 2024-05-08T08:53:27.006 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.006 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.006 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:27.006 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585516+0000", 2024-05-08T08:53:27.006 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.526e-06 2024-05-08T08:53:27.006 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.006 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.006 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:27.006 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585523+0000", 2024-05-08T08:53:27.007 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.562e-06 2024-05-08T08:53:27.007 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.007 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.007 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:27.007 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585528+0000", 2024-05-08T08:53:27.007 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7990000000000001e-06 2024-05-08T08:53:27.007 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.007 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.007 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:27.007 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.620939+0000", 2024-05-08T08:53:27.007 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035411419 2024-05-08T08:53:27.007 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.007 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.007 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:27.007 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621014+0000", 2024-05-08T08:53:27.008 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5017000000000004e-05 2024-05-08T08:53:27.008 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.008 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.008 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:27.008 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621102+0000", 2024-05-08T08:53:27.008 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7979000000000006e-05 2024-05-08T08:53:27.008 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.008 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:27.008 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.008 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.008 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.008 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4743 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1849344~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:27.008 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.585628+0000", 2024-05-08T08:53:27.008 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 55.420360013, 2024-05-08T08:53:27.008 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035659962000000003, 2024-05-08T08:53:27.008 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:27.009 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:27.009 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:27.009 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:27.009 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:27.009 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:27.009 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4743 2024-05-08T08:53:27.009 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.009 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:27.009 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.009 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:27.009 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585628+0000", 2024-05-08T08:53:27.009 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:27.009 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.009 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.010 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:27.010 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585628+0000", 2024-05-08T08:53:27.010 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:27.010 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.010 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.010 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:27.010 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585629+0000", 2024-05-08T08:53:27.010 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2300000000000001e-07 2024-05-08T08:53:27.010 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.010 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.010 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:27.010 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585632+0000", 2024-05-08T08:53:27.010 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1370000000000002e-06 2024-05-08T08:53:27.010 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.010 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.011 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:27.011 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585641+0000", 2024-05-08T08:53:27.011 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5220000000000007e-06 2024-05-08T08:53:27.011 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.011 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.011 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:27.011 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585646+0000", 2024-05-08T08:53:27.011 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.609e-06 2024-05-08T08:53:27.011 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.011 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.011 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:27.011 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621132+0000", 2024-05-08T08:53:27.011 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035485472999999997 2024-05-08T08:53:27.011 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.012 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.012 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:27.012 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621204+0000", 2024-05-08T08:53:27.012 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2259000000000001e-05 2024-05-08T08:53:27.012 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.012 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.012 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:27.012 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621288+0000", 2024-05-08T08:53:27.012 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4338999999999993e-05 2024-05-08T08:53:27.012 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.012 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:27.012 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.012 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.012 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.013 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4746 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1865728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:27.013 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.585656+0000", 2024-05-08T08:53:27.013 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 55.420332330999997, 2024-05-08T08:53:27.013 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035815909, 2024-05-08T08:53:27.013 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:27.013 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:27.013 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:27.013 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:27.013 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:27.013 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:27.013 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4746 2024-05-08T08:53:27.013 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.013 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:27.013 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.014 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:27.014 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585656+0000", 2024-05-08T08:53:27.014 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:27.014 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.014 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.014 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:27.014 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585656+0000", 2024-05-08T08:53:27.014 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:27.014 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.014 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.014 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:27.014 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585656+0000", 2024-05-08T08:53:27.014 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.08e-07 2024-05-08T08:53:27.015 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.015 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.015 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:27.015 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585658+0000", 2024-05-08T08:53:27.015 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0889999999999998e-06 2024-05-08T08:53:27.015 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.015 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.015 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:27.015 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585664+0000", 2024-05-08T08:53:27.015 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3990000000000003e-06 2024-05-08T08:53:27.015 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.015 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.015 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:27.015 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585668+0000", 2024-05-08T08:53:27.015 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3170000000000003e-06 2024-05-08T08:53:27.016 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.016 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.016 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:27.016 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621316+0000", 2024-05-08T08:53:27.016 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035647908999999998 2024-05-08T08:53:27.016 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.016 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.016 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:27.016 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621389+0000", 2024-05-08T08:53:27.016 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2531000000000001e-05 2024-05-08T08:53:27.016 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.016 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.016 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:27.016 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621472+0000", 2024-05-08T08:53:27.016 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3256000000000004e-05 2024-05-08T08:53:27.016 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.017 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:27.017 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.017 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.017 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.017 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4748 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1882112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:27.017 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.585697+0000", 2024-05-08T08:53:27.017 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 55.420290885, 2024-05-08T08:53:27.017 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035962485000000002, 2024-05-08T08:53:27.017 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:27.017 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:27.017 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:27.017 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:27.017 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:27.018 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:27.018 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4748 2024-05-08T08:53:27.018 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.018 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:27.018 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.018 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:27.018 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585697+0000", 2024-05-08T08:53:27.018 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:27.018 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.018 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.018 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:27.018 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585697+0000", 2024-05-08T08:53:27.018 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:27.018 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.019 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.019 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:27.019 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585698+0000", 2024-05-08T08:53:27.019 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0900000000000001e-07 2024-05-08T08:53:27.019 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.019 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.019 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:27.019 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585701+0000", 2024-05-08T08:53:27.019 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0149999999999999e-06 2024-05-08T08:53:27.019 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.019 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.019 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:27.019 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585712+0000", 2024-05-08T08:53:27.019 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0655e-05 2024-05-08T08:53:27.019 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.020 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.020 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:27.020 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585717+0000", 2024-05-08T08:53:27.020 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3800000000000002e-06 2024-05-08T08:53:27.020 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.020 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.020 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:27.020 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621502+0000", 2024-05-08T08:53:27.020 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035784559000000001 2024-05-08T08:53:27.020 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.020 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.020 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:27.020 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621575+0000", 2024-05-08T08:53:27.020 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3269999999999995e-05 2024-05-08T08:53:27.020 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.020 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.021 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:27.021 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621660+0000", 2024-05-08T08:53:27.021 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4997e-05 2024-05-08T08:53:27.021 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.021 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:27.021 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.021 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.021 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.021 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4750 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1898496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:27.021 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.585810+0000", 2024-05-08T08:53:27.021 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 55.420178055000001, 2024-05-08T08:53:27.021 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036034928000000001, 2024-05-08T08:53:27.021 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:27.021 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:27.021 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:27.021 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:27.022 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:27.022 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:27.022 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4750 2024-05-08T08:53:27.022 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.022 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:27.022 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.022 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:27.022 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585810+0000", 2024-05-08T08:53:27.022 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:27.022 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.022 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.022 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:27.022 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585810+0000", 2024-05-08T08:53:27.022 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:27.023 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.023 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.023 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:27.023 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585811+0000", 2024-05-08T08:53:27.023 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6499999999999999e-07 2024-05-08T08:53:27.023 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.023 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.023 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:27.023 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585814+0000", 2024-05-08T08:53:27.023 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1709999999999998e-06 2024-05-08T08:53:27.023 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.023 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.023 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:27.023 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585822+0000", 2024-05-08T08:53:27.023 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8909999999999995e-06 2024-05-08T08:53:27.024 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.024 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.024 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:27.024 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585827+0000", 2024-05-08T08:53:27.024 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5289999999999999e-06 2024-05-08T08:53:27.024 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.024 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.024 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:27.024 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621688+0000", 2024-05-08T08:53:27.024 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.035861111000000001 2024-05-08T08:53:27.024 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.024 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.024 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:27.024 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621761+0000", 2024-05-08T08:53:27.024 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2211000000000004e-05 2024-05-08T08:53:27.025 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.025 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.025 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:27.025 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621845+0000", 2024-05-08T08:53:27.025 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4450000000000006e-05 2024-05-08T08:53:27.025 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.025 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:27.025 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.025 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.025 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.025 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4752 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1914880~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:27.025 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.585858+0000", 2024-05-08T08:53:27.025 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 55.420130086, 2024-05-08T08:53:27.025 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036180081000000003, 2024-05-08T08:53:27.025 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:27.026 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:27.026 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:27.026 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:27.026 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:27.026 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:27.026 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4752 2024-05-08T08:53:27.026 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.026 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:27.026 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.026 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:27.026 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585858+0000", 2024-05-08T08:53:27.026 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:27.026 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.026 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.026 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:27.027 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585858+0000", 2024-05-08T08:53:27.027 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:27.027 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.027 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.027 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:27.027 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585859+0000", 2024-05-08T08:53:27.027 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.82e-07 2024-05-08T08:53:27.027 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.027 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.027 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:27.027 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585861+0000", 2024-05-08T08:53:27.027 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.695e-06 2024-05-08T08:53:27.027 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.027 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.027 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:27.028 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585869+0000", 2024-05-08T08:53:27.028 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0299999999999994e-06 2024-05-08T08:53:27.028 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.028 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.028 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:27.028 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585874+0000", 2024-05-08T08:53:27.028 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9620000000000001e-06 2024-05-08T08:53:27.028 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.028 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.028 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:27.028 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621878+0000", 2024-05-08T08:53:27.028 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036004043999999999 2024-05-08T08:53:27.028 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.028 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.028 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:27.028 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.621954+0000", 2024-05-08T08:53:27.028 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5414000000000007e-05 2024-05-08T08:53:27.029 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.029 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.029 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:27.029 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622038+0000", 2024-05-08T08:53:27.029 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4454000000000002e-05 2024-05-08T08:53:27.029 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.029 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:27.029 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.029 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.029 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.029 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4753 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1931264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:27.029 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.585947+0000", 2024-05-08T08:53:27.029 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 55.420041437999998, 2024-05-08T08:53:27.029 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036267992999999998, 2024-05-08T08:53:27.029 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:27.029 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:27.030 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:27.030 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:27.030 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:27.030 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:27.030 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4753 2024-05-08T08:53:27.030 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.030 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:27.030 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.030 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:27.030 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585947+0000", 2024-05-08T08:53:27.030 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:27.030 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.030 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.030 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:27.030 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585947+0000", 2024-05-08T08:53:27.031 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:27.031 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.031 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.031 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:27.031 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585948+0000", 2024-05-08T08:53:27.031 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6000000000000002e-07 2024-05-08T08:53:27.031 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.031 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.031 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:27.031 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585952+0000", 2024-05-08T08:53:27.031 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7090000000000001e-06 2024-05-08T08:53:27.031 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.031 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.031 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:27.031 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585963+0000", 2024-05-08T08:53:27.031 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0740000000000001e-05 2024-05-08T08:53:27.032 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.032 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.032 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:27.032 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.585970+0000", 2024-05-08T08:53:27.032 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3309999999999998e-06 2024-05-08T08:53:27.032 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.032 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.032 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:27.032 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622065+0000", 2024-05-08T08:53:27.032 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036094676999999999 2024-05-08T08:53:27.032 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.032 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.032 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:27.032 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622132+0000", 2024-05-08T08:53:27.032 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7214999999999999e-05 2024-05-08T08:53:27.032 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.033 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.033 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:27.033 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622215+0000", 2024-05-08T08:53:27.033 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2460999999999995e-05 2024-05-08T08:53:27.033 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.033 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:27.033 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.033 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.033 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.033 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4756 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1947648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:27.033 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586018+0000", 2024-05-08T08:53:27.033 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 55.419969793999996, 2024-05-08T08:53:27.033 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036382335000000002, 2024-05-08T08:53:27.033 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:27.034 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:27.034 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:27.034 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:27.034 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:27.034 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:27.034 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4756 2024-05-08T08:53:27.034 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.034 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:27.034 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.034 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:27.034 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586018+0000", 2024-05-08T08:53:27.034 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:27.034 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.034 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.035 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:27.035 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586018+0000", 2024-05-08T08:53:27.035 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:27.035 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.035 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.035 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:27.035 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586019+0000", 2024-05-08T08:53:27.035 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9999999999999997e-07 2024-05-08T08:53:27.035 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.035 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.035 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:27.035 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586023+0000", 2024-05-08T08:53:27.035 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7330000000000001e-06 2024-05-08T08:53:27.035 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.035 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.035 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:27.036 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586031+0000", 2024-05-08T08:53:27.036 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4940000000000003e-06 2024-05-08T08:53:27.036 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.036 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.036 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:27.036 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586038+0000", 2024-05-08T08:53:27.036 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2539999999999998e-06 2024-05-08T08:53:27.036 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.036 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.036 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:27.036 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622243+0000", 2024-05-08T08:53:27.036 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036205009000000003 2024-05-08T08:53:27.036 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.036 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.036 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:27.036 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622313+0000", 2024-05-08T08:53:27.036 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9912000000000004e-05 2024-05-08T08:53:27.037 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.037 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.037 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:27.037 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622401+0000", 2024-05-08T08:53:27.037 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8232999999999994e-05 2024-05-08T08:53:27.037 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.037 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:27.037 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.037 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.037 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.037 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4758 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1964032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:27.037 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586080+0000", 2024-05-08T08:53:27.037 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 55.419908374999999, 2024-05-08T08:53:27.037 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036547659000000003, 2024-05-08T08:53:27.037 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:27.037 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:27.037 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:27.038 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:27.038 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:27.038 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:27.038 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4758 2024-05-08T08:53:27.038 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.038 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:27.038 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.038 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:27.038 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586080+0000", 2024-05-08T08:53:27.038 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:27.038 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.038 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.038 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:27.038 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586080+0000", 2024-05-08T08:53:27.038 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:27.038 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.039 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.039 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:27.039 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586081+0000", 2024-05-08T08:53:27.039 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5499999999999998e-07 2024-05-08T08:53:27.039 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.039 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.039 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:27.039 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586084+0000", 2024-05-08T08:53:27.039 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9919999999999999e-06 2024-05-08T08:53:27.039 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.039 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.039 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:27.039 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586091+0000", 2024-05-08T08:53:27.039 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5440000000000001e-06 2024-05-08T08:53:27.039 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.039 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.039 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:27.040 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586098+0000", 2024-05-08T08:53:27.040 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8639999999999998e-06 2024-05-08T08:53:27.040 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.040 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.040 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:27.040 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622442+0000", 2024-05-08T08:53:27.040 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036344110999999998 2024-05-08T08:53:27.040 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.040 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.040 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:27.040 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622516+0000", 2024-05-08T08:53:27.040 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4089000000000002e-05 2024-05-08T08:53:27.040 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.040 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.040 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:27.040 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622628+0000", 2024-05-08T08:53:27.041 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00011140400000000001 2024-05-08T08:53:27.041 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.041 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:27.041 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.041 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.041 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.041 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4760 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1980416~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:27.041 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586178+0000", 2024-05-08T08:53:27.041 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 55.419809763000003, 2024-05-08T08:53:27.041 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036647526999999999, 2024-05-08T08:53:27.041 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:27.041 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:27.041 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:27.041 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:27.041 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:27.041 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:27.041 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4760 2024-05-08T08:53:27.042 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.042 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:27.042 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.042 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:27.042 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586178+0000", 2024-05-08T08:53:27.042 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:27.042 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.042 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.042 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:27.042 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586178+0000", 2024-05-08T08:53:27.042 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:27.042 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.042 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.042 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:27.042 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586179+0000", 2024-05-08T08:53:27.042 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8599999999999997e-07 2024-05-08T08:53:27.043 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.043 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.043 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:27.043 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586182+0000", 2024-05-08T08:53:27.043 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.174e-06 2024-05-08T08:53:27.043 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.043 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.043 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:27.043 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586191+0000", 2024-05-08T08:53:27.043 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1869999999999997e-06 2024-05-08T08:53:27.043 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.043 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.043 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:27.043 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586197+0000", 2024-05-08T08:53:27.043 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2090000000000002e-06 2024-05-08T08:53:27.043 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.044 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.044 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:27.044 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622657+0000", 2024-05-08T08:53:27.044 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036459793999999997 2024-05-08T08:53:27.044 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.044 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.044 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:27.044 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622725+0000", 2024-05-08T08:53:27.044 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8374999999999995e-05 2024-05-08T08:53:27.044 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.044 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.044 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:27.044 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622826+0000", 2024-05-08T08:53:27.044 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010100200000000001 2024-05-08T08:53:27.044 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.045 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:27.045 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.045 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.045 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.045 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4762 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 1996800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:27.045 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586299+0000", 2024-05-08T08:53:27.045 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 55.419689431999998, 2024-05-08T08:53:27.045 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036720653999999998, 2024-05-08T08:53:27.045 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:27.045 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:27.045 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:27.045 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:27.045 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:27.045 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:27.045 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4762 2024-05-08T08:53:27.045 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.046 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:27.046 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.046 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:27.046 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586299+0000", 2024-05-08T08:53:27.046 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:27.046 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.046 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.046 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:27.046 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586299+0000", 2024-05-08T08:53:27.046 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:27.046 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.046 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.046 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:27.047 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586299+0000", 2024-05-08T08:53:27.047 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.99e-07 2024-05-08T08:53:27.047 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.047 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.047 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:27.047 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586303+0000", 2024-05-08T08:53:27.047 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1949999999999999e-06 2024-05-08T08:53:27.047 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.047 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.047 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:27.047 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586318+0000", 2024-05-08T08:53:27.047 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5401999999999999e-05 2024-05-08T08:53:27.047 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.047 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.047 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:27.047 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586324+0000", 2024-05-08T08:53:27.048 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8119999999999996e-06 2024-05-08T08:53:27.048 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.048 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.048 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:27.048 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622853+0000", 2024-05-08T08:53:27.048 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036529013999999999 2024-05-08T08:53:27.048 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.048 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.048 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:27.048 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.622924+0000", 2024-05-08T08:53:27.048 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1654000000000002e-05 2024-05-08T08:53:27.048 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.048 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.048 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:27.048 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623019+0000", 2024-05-08T08:53:27.048 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4977999999999998e-05 2024-05-08T08:53:27.049 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.049 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:27.049 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.049 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.049 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.049 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4764 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 2013184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:27.049 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586506+0000", 2024-05-08T08:53:27.049 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 55.419482326999997, 2024-05-08T08:53:27.049 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036859467, 2024-05-08T08:53:27.049 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:27.049 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:27.049 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:27.049 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:27.049 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:27.049 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:27.049 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4764 2024-05-08T08:53:27.049 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.050 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:27.050 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.050 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:27.050 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586506+0000", 2024-05-08T08:53:27.050 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:27.050 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.050 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.050 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:27.050 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586506+0000", 2024-05-08T08:53:27.050 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:27.050 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.050 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.050 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:27.050 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586507+0000", 2024-05-08T08:53:27.050 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9100000000000003e-07 2024-05-08T08:53:27.050 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.050 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.051 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:27.051 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586512+0000", 2024-05-08T08:53:27.051 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9470000000000002e-06 2024-05-08T08:53:27.051 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.051 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.051 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:27.051 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586521+0000", 2024-05-08T08:53:27.051 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3379999999999994e-06 2024-05-08T08:53:27.051 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.051 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.051 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:27.051 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586528+0000", 2024-05-08T08:53:27.051 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1289999999999998e-06 2024-05-08T08:53:27.051 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.051 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.051 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:27.052 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623095+0000", 2024-05-08T08:53:27.052 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036566411 2024-05-08T08:53:27.052 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.052 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.052 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:27.052 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623221+0000", 2024-05-08T08:53:27.052 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012622299999999999 2024-05-08T08:53:27.052 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.052 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.052 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:27.052 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623365+0000", 2024-05-08T08:53:27.052 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000144628 2024-05-08T08:53:27.052 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.052 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:27.052 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.053 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.053 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.053 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4766 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 2029568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:27.053 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586543+0000", 2024-05-08T08:53:27.053 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 55.419444839999997, 2024-05-08T08:53:27.053 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.037033635000000002, 2024-05-08T08:53:27.053 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:27.053 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:27.053 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:27.053 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:27.053 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:27.053 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:27.053 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4766 2024-05-08T08:53:27.053 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.053 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:27.054 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.054 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:27.054 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586543+0000", 2024-05-08T08:53:27.054 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:27.054 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.054 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.054 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:27.054 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586543+0000", 2024-05-08T08:53:27.054 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:27.054 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.054 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.054 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:27.054 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586544+0000", 2024-05-08T08:53:27.055 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3099999999999998e-07 2024-05-08T08:53:27.055 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.055 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.055 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:27.055 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586547+0000", 2024-05-08T08:53:27.055 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.751e-06 2024-05-08T08:53:27.055 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.055 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.055 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:27.055 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586553+0000", 2024-05-08T08:53:27.055 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2870000000000001e-06 2024-05-08T08:53:27.055 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.055 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.055 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:27.056 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586558+0000", 2024-05-08T08:53:27.056 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5709999999999997e-06 2024-05-08T08:53:27.056 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.056 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.056 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:27.056 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623416+0000", 2024-05-08T08:53:27.056 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.036857613999999997 2024-05-08T08:53:27.056 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.056 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.056 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:27.056 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623489+0000", 2024-05-08T08:53:27.056 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3172999999999999e-05 2024-05-08T08:53:27.056 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.056 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.056 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:27.056 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623577+0000", 2024-05-08T08:53:27.057 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7807999999999997e-05 2024-05-08T08:53:27.057 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.057 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:27.057 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.057 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.057 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.057 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4768 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 2045952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:27.057 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586566+0000", 2024-05-08T08:53:27.057 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 55.419422533999999, 2024-05-08T08:53:27.057 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.03726206, 2024-05-08T08:53:27.057 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:27.057 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:27.057 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:27.057 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:27.057 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:27.058 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:27.058 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4768 2024-05-08T08:53:27.058 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.058 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:27.058 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.058 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:27.058 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586566+0000", 2024-05-08T08:53:27.058 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:27.058 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.058 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.058 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:27.058 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586566+0000", 2024-05-08T08:53:27.058 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:27.058 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.059 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.059 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:27.059 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586566+0000", 2024-05-08T08:53:27.059 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8299999999999998e-07 2024-05-08T08:53:27.059 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.059 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.059 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:27.059 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586568+0000", 2024-05-08T08:53:27.059 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.063e-06 2024-05-08T08:53:27.059 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.059 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.059 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:27.059 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586574+0000", 2024-05-08T08:53:27.059 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8300000000000001e-06 2024-05-08T08:53:27.059 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.060 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.060 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:27.060 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586579+0000", 2024-05-08T08:53:27.060 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7559999999999999e-06 2024-05-08T08:53:27.060 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.060 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.060 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:27.060 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623612+0000", 2024-05-08T08:53:27.060 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.037033041000000003 2024-05-08T08:53:27.060 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.060 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.060 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:27.060 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623699+0000", 2024-05-08T08:53:27.060 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6817000000000006e-05 2024-05-08T08:53:27.060 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.060 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.060 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:27.061 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623828+0000", 2024-05-08T08:53:27.061 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012917000000000001 2024-05-08T08:53:27.061 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.061 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:27.061 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.061 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.061 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.061 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4770 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 2062336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:27.061 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586585+0000", 2024-05-08T08:53:27.061 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 55.419403351, 2024-05-08T08:53:27.061 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.037449891999999999, 2024-05-08T08:53:27.061 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:27.061 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:27.061 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:27.061 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:27.061 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:27.061 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:27.062 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4770 2024-05-08T08:53:27.062 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.062 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:27.062 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.062 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:27.062 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586585+0000", 2024-05-08T08:53:27.062 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:27.062 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.062 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.062 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:27.062 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586585+0000", 2024-05-08T08:53:27.062 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:27.062 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.063 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.063 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:27.063 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586585+0000", 2024-05-08T08:53:27.063 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:53:27.063 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.063 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.063 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:27.063 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586587+0000", 2024-05-08T08:53:27.063 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.181e-06 2024-05-08T08:53:27.063 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.063 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.063 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:27.063 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586592+0000", 2024-05-08T08:53:27.063 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9710000000000003e-06 2024-05-08T08:53:27.063 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.064 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.064 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:27.064 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586598+0000", 2024-05-08T08:53:27.064 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.118e-06 2024-05-08T08:53:27.064 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.064 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.064 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:27.064 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623862+0000", 2024-05-08T08:53:27.064 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.037264711999999998 2024-05-08T08:53:27.064 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.064 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.064 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:27.064 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.623941+0000", 2024-05-08T08:53:27.064 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9084000000000004e-05 2024-05-08T08:53:27.064 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.064 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.064 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:27.065 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.624035+0000", 2024-05-08T08:53:27.065 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3456999999999996e-05 2024-05-08T08:53:27.065 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.065 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:27.065 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.065 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.065 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.065 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4772 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 2078720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:27.065 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586621+0000", 2024-05-08T08:53:27.065 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 55.419367403000003, 2024-05-08T08:53:27.065 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.037650391999999998, 2024-05-08T08:53:27.065 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:27.065 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:27.065 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:27.065 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:27.065 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:27.066 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:27.066 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4772 2024-05-08T08:53:27.066 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.066 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:27.066 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.066 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:27.066 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586621+0000", 2024-05-08T08:53:27.066 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:27.066 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.066 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.066 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:27.066 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586621+0000", 2024-05-08T08:53:27.066 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:27.066 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.066 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.066 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:27.067 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586621+0000", 2024-05-08T08:53:27.067 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:53:27.067 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.067 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.067 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:27.067 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586623+0000", 2024-05-08T08:53:27.067 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9240000000000001e-06 2024-05-08T08:53:27.067 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.067 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.067 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:27.067 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586628+0000", 2024-05-08T08:53:27.067 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6410000000000001e-06 2024-05-08T08:53:27.067 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.067 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.067 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:27.067 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586633+0000", 2024-05-08T08:53:27.067 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.763e-06 2024-05-08T08:53:27.068 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.068 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.068 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:27.068 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.624077+0000", 2024-05-08T08:53:27.068 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.037444709999999999 2024-05-08T08:53:27.068 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.068 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.068 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:27.068 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.624164+0000", 2024-05-08T08:53:27.068 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6769999999999998e-05 2024-05-08T08:53:27.068 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.068 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.068 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:27.068 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.624271+0000", 2024-05-08T08:53:27.068 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010723 2024-05-08T08:53:27.069 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.069 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:27.069 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.069 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.069 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.069 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:4774 3.15 3:aad3c5d5:::rbd_data.1140782590a5.0000000000000011:6 [read 2095104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:27.069 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.586640+0000", 2024-05-08T08:53:27.069 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 55.419348264999996, 2024-05-08T08:53:27.069 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.037827178000000003, 2024-05-08T08:53:27.069 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:27.069 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:27.069 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:27.069 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:27.069 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:27.069 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:27.070 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4774 2024-05-08T08:53:27.070 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.070 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:27.070 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.070 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:27.070 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586640+0000", 2024-05-08T08:53:27.070 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:27.070 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.070 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.070 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:27.070 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586640+0000", 2024-05-08T08:53:27.070 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:27.070 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.070 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.071 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:27.071 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586640+0000", 2024-05-08T08:53:27.071 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5199999999999998e-07 2024-05-08T08:53:27.071 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.071 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.071 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:27.071 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586643+0000", 2024-05-08T08:53:27.071 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.232e-06 2024-05-08T08:53:27.071 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.071 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.071 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:27.071 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586649+0000", 2024-05-08T08:53:27.071 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9660000000000001e-06 2024-05-08T08:53:27.071 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.071 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.071 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:27.071 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.586653+0000", 2024-05-08T08:53:27.072 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7350000000000004e-06 2024-05-08T08:53:27.072 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.072 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.072 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:27.072 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.624303+0000", 2024-05-08T08:53:27.072 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.037649647000000001 2024-05-08T08:53:27.072 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.072 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.072 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:27.072 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.624373+0000", 2024-05-08T08:53:27.072 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0415999999999998e-05 2024-05-08T08:53:27.072 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.072 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.072 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:27.072 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.624467+0000", 2024-05-08T08:53:27.072 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3830000000000001e-05 2024-05-08T08:53:27.072 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.073 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:27.073 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.073 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.073 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.073 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4621.0:9089 3.1f e33/18 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:head v 33'117, mlcod=31'115)", 2024-05-08T08:53:27.073 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:53:27.073 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 54.486370053000002, 2024-05-08T08:53:27.073 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.111244524, 2024-05-08T08:53:27.073 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:27.073 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:27.073 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:53:27.073 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:27.073 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.073 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:27.073 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:53:27.073 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:27.074 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.074 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.074 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:27.074 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:53:27.074 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:27.074 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.074 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.074 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:27.074 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:53:27.074 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5900000000000001e-07 2024-05-08T08:53:27.074 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.074 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.074 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:27.074 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539600+0000", 2024-05-08T08:53:27.074 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.019981364000000001 2024-05-08T08:53:27.074 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.075 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.075 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:27.075 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539624+0000", 2024-05-08T08:53:27.075 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4043e-05 2024-05-08T08:53:27.075 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.075 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.075 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:27.075 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539635+0000", 2024-05-08T08:53:27.075 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1511e-05 2024-05-08T08:53:27.075 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.075 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.075 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:27.075 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539728+0000", 2024-05-08T08:53:27.075 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2498999999999996e-05 2024-05-08T08:53:27.075 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.075 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.075 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:27.076 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539746+0000", 2024-05-08T08:53:27.076 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8289000000000001e-05 2024-05-08T08:53:27.076 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.076 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.076 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:53:27.076 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.630822+0000", 2024-05-08T08:53:27.076 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.091076283999999993 2024-05-08T08:53:27.076 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.076 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.076 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:27.076 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.630863+0000", 2024-05-08T08:53:27.076 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0374999999999999e-05 2024-05-08T08:53:27.076 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.076 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:27.076 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.077 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.077 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.077 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4795.0:230 3.1f e40/18 3:f8db1a56:::rbd_data.11402b26385.0000000000000000:head v 40'171, mlcod=39'168)", 2024-05-08T08:53:27.077 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:45.927773+0000", 2024-05-08T08:53:27.077 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.078214969999998, 2024-05-08T08:53:27.077 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045892405999999997, 2024-05-08T08:53:27.077 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:27.077 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:27.077 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:53:27.077 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:27.077 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.077 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:27.077 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.927773+0000", 2024-05-08T08:53:27.077 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:27.077 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.077 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.078 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:27.078 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.927773+0000", 2024-05-08T08:53:27.078 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:27.078 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.078 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.078 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:27.078 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.927774+0000", 2024-05-08T08:53:27.078 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0500000000000002e-07 2024-05-08T08:53:27.078 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.078 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.078 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:27.078 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.931581+0000", 2024-05-08T08:53:27.078 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0038071799999999999 2024-05-08T08:53:27.078 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.078 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.079 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:27.079 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.931599+0000", 2024-05-08T08:53:27.079 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7479e-05 2024-05-08T08:53:27.079 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.079 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.079 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:27.079 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.931624+0000", 2024-05-08T08:53:27.079 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5372999999999999e-05 2024-05-08T08:53:27.079 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.079 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.079 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:27.079 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.931698+0000", 2024-05-08T08:53:27.079 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4043999999999998e-05 2024-05-08T08:53:27.079 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.079 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.080 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:27.080 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.931725+0000", 2024-05-08T08:53:27.080 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6883e-05 2024-05-08T08:53:27.080 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.080 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.080 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:53:27.080 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.973623+0000", 2024-05-08T08:53:27.080 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.041898283000000001 2024-05-08T08:53:27.080 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.080 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.080 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:27.080 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.973666+0000", 2024-05-08T08:53:27.080 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2358999999999999e-05 2024-05-08T08:53:27.080 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.080 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:27.080 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.081 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.081 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.081 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4195.0:4 3.11 3:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-05-08T08:53:27.081 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:49.764654+0000", 2024-05-08T08:53:27.081 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.241334512000002, 2024-05-08T08:53:27.081 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.30714275499999999, 2024-05-08T08:53:27.081 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:27.081 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:27.081 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-05-08T08:53:27.081 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:27.081 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4195", 2024-05-08T08:53:27.081 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/59179521", 2024-05-08T08:53:27.081 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4 2024-05-08T08:53:27.081 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.081 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:27.081 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.081 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:27.082 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764654+0000", 2024-05-08T08:53:27.082 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:27.082 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.082 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.082 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:27.082 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764654+0000", 2024-05-08T08:53:27.082 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:27.082 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.082 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.082 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:27.082 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764655+0000", 2024-05-08T08:53:27.082 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.452e-06 2024-05-08T08:53:27.082 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.082 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.082 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:27.082 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764661+0000", 2024-05-08T08:53:27.083 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5369999999999997e-06 2024-05-08T08:53:27.083 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.083 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.083 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:27.083 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764671+0000", 2024-05-08T08:53:27.083 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.028e-05 2024-05-08T08:53:27.083 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.083 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.083 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:27.083 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764680+0000", 2024-05-08T08:53:27.083 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0629999999999995e-06 2024-05-08T08:53:27.083 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.083 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.083 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:27.083 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764753+0000", 2024-05-08T08:53:27.083 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2679e-05 2024-05-08T08:53:27.084 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.084 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.084 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-05-08T08:53:27.084 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764765+0000", 2024-05-08T08:53:27.084 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2415000000000001e-05 2024-05-08T08:53:27.084 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.084 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.084 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:27.084 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:50.071692+0000", 2024-05-08T08:53:27.084 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.306927324 2024-05-08T08:53:27.084 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.084 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.084 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:27.084 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:50.071796+0000", 2024-05-08T08:53:27.084 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104005 2024-05-08T08:53:27.084 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.084 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:27.085 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.085 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.085 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:27.085 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:53:27.086 INFO:teuthology.orchestra.run.smithi196.stdout:checking clone #19, image image_client.0-clone19 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent20 2024-05-08T08:53:27.087 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:53:27.087 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:27.089+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:27.187 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:27.190+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:27.244 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:53:27.244 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:53:27.244 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:53:27.244 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:53:27.253 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:53:27.298 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:27.301+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:27.387 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:53:27.387 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:53:27.387 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:53:27.387 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:53:27.387 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:53:27.389 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:27.391+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:27.398 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:53:27.490 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:53:27.493+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:53:27.493 INFO:teuthology.orchestra.run.smithi059.stderr:nodeep-scrub is set 2024-05-08T08:53:27.569 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:53:27.569 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:53:27.569 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:53:27.569 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:53:27.569 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.569 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22071 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 765952~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:27.569 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857643+0000", 2024-05-08T08:53:27.569 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 59.713054882999998, 2024-05-08T08:53:27.569 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047106528000000002, 2024-05-08T08:53:27.569 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:27.569 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:27.570 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:27.570 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:27.570 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:27.570 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:27.570 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22071 2024-05-08T08:53:27.570 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.570 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:27.570 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.570 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:27.570 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857643+0000", 2024-05-08T08:53:27.570 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:27.570 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.570 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.570 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:27.571 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857643+0000", 2024-05-08T08:53:27.571 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:27.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.571 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.571 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:27.571 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857644+0000", 2024-05-08T08:53:27.571 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2000000000000001e-07 2024-05-08T08:53:27.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.571 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.571 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:27.571 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857646+0000", 2024-05-08T08:53:27.571 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0059999999999999e-06 2024-05-08T08:53:27.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.572 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.572 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:27.572 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857651+0000", 2024-05-08T08:53:27.572 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.84e-06 2024-05-08T08:53:27.572 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.572 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.572 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:27.572 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857657+0000", 2024-05-08T08:53:27.572 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5620000000000003e-06 2024-05-08T08:53:27.572 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.572 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.572 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:27.572 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904620+0000", 2024-05-08T08:53:27.572 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046962661000000003 2024-05-08T08:53:27.572 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.573 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.573 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:27.573 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904678+0000", 2024-05-08T08:53:27.573 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8656999999999998e-05 2024-05-08T08:53:27.573 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.573 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.573 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:27.573 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904750+0000", 2024-05-08T08:53:27.573 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1482000000000004e-05 2024-05-08T08:53:27.573 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.573 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:27.573 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.573 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.573 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.573 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22073 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 782336~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:27.573 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:53:27.574 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 59.713030875000001, 2024-05-08T08:53:27.574 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047244234000000003, 2024-05-08T08:53:27.574 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:27.574 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:27.574 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:27.574 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:27.574 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:27.574 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:27.574 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22073 2024-05-08T08:53:27.574 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.574 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:27.574 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.574 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:27.574 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:53:27.575 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:27.575 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.575 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.575 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:27.575 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:53:27.575 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:27.575 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.575 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.575 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:27.575 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857668+0000", 2024-05-08T08:53:27.575 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7800000000000002e-07 2024-05-08T08:53:27.575 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.575 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.575 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:27.575 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857670+0000", 2024-05-08T08:53:27.576 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0820000000000001e-06 2024-05-08T08:53:27.576 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.576 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.576 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:27.576 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857678+0000", 2024-05-08T08:53:27.576 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7589999999999997e-06 2024-05-08T08:53:27.576 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.576 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.576 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:27.576 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857691+0000", 2024-05-08T08:53:27.576 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3672999999999999e-05 2024-05-08T08:53:27.576 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.576 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.576 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:27.577 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904780+0000", 2024-05-08T08:53:27.577 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047088616999999999 2024-05-08T08:53:27.577 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.577 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.577 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:27.577 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904840+0000", 2024-05-08T08:53:27.577 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0356999999999998e-05 2024-05-08T08:53:27.577 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.577 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.577 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:27.577 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904912+0000", 2024-05-08T08:53:27.577 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1367999999999998e-05 2024-05-08T08:53:27.577 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.577 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:27.577 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.578 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.578 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.578 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22075 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 798720~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:27.578 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:53:27.578 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 59.713000477999998, 2024-05-08T08:53:27.578 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047384643999999997, 2024-05-08T08:53:27.578 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:27.578 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:27.578 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:27.578 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:27.578 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:27.578 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:27.578 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22075 2024-05-08T08:53:27.578 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.579 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:27.579 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.579 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:27.579 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:53:27.579 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:27.579 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.579 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.579 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:27.579 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:53:27.579 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:27.579 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.579 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.579 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:27.579 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:53:27.579 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7399999999999999e-07 2024-05-08T08:53:27.580 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.580 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.580 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:27.580 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857705+0000", 2024-05-08T08:53:27.580 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1980000000000004e-06 2024-05-08T08:53:27.580 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.580 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.580 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:27.580 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857713+0000", 2024-05-08T08:53:27.580 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7039999999999994e-06 2024-05-08T08:53:27.580 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.580 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.580 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:27.580 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857718+0000", 2024-05-08T08:53:27.580 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.865e-06 2024-05-08T08:53:27.581 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.581 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.581 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:27.581 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904941+0000", 2024-05-08T08:53:27.581 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047223198000000001 2024-05-08T08:53:27.581 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.581 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.581 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:27.581 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904999+0000", 2024-05-08T08:53:27.581 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8254999999999997e-05 2024-05-08T08:53:27.581 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.581 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.581 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:27.581 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905082+0000", 2024-05-08T08:53:27.581 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3049999999999999e-05 2024-05-08T08:53:27.581 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.582 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:27.582 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.582 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.582 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.582 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22077 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 815104~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:27.582 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:53:27.582 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 59.712962466999997, 2024-05-08T08:53:27.582 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047519448999999998, 2024-05-08T08:53:27.582 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:27.582 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:27.582 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:27.582 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:27.582 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:27.583 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:27.583 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22077 2024-05-08T08:53:27.583 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.583 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:27.583 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.583 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:27.583 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:53:27.583 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:27.583 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.583 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.583 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:27.583 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:53:27.583 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:27.583 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.583 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.584 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:27.584 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:53:27.584 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.99e-07 2024-05-08T08:53:27.584 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.584 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.584 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:27.584 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857739+0000", 2024-05-08T08:53:27.584 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3599999999999999e-06 2024-05-08T08:53:27.584 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.584 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.584 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:27.584 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857745+0000", 2024-05-08T08:53:27.584 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5139999999999998e-06 2024-05-08T08:53:27.584 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.584 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.585 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:27.585 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857750+0000", 2024-05-08T08:53:27.585 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8550000000000001e-06 2024-05-08T08:53:27.585 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.585 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.585 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:27.585 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905110+0000", 2024-05-08T08:53:27.585 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047360126000000002 2024-05-08T08:53:27.585 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.585 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.585 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:27.585 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905175+0000", 2024-05-08T08:53:27.585 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4689000000000004e-05 2024-05-08T08:53:27.585 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.585 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.585 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:27.586 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905255+0000", 2024-05-08T08:53:27.586 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0306e-05 2024-05-08T08:53:27.586 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.586 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:27.586 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.586 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.586 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.586 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22079 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 831488~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:27.586 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:53:27.586 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 59.712941919000002, 2024-05-08T08:53:27.586 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.04767499, 2024-05-08T08:53:27.586 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:27.586 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:27.586 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:27.587 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:27.587 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:27.587 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:27.587 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22079 2024-05-08T08:53:27.587 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.587 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:27.587 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.587 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:27.587 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:53:27.587 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:27.587 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.587 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.587 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:27.587 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:53:27.587 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:27.588 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.588 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.588 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:27.588 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857757+0000", 2024-05-08T08:53:27.588 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8299999999999997e-07 2024-05-08T08:53:27.588 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.588 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.588 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:27.588 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857759+0000", 2024-05-08T08:53:27.588 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1390000000000002e-06 2024-05-08T08:53:27.588 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.588 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.588 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:27.588 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857765+0000", 2024-05-08T08:53:27.588 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4609999999999996e-06 2024-05-08T08:53:27.589 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.589 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.589 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:27.589 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857770+0000", 2024-05-08T08:53:27.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6689999999999997e-06 2024-05-08T08:53:27.589 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.589 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.589 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:27.589 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905281+0000", 2024-05-08T08:53:27.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047510668999999998 2024-05-08T08:53:27.589 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.589 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.589 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:27.589 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905344+0000", 2024-05-08T08:53:27.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2797000000000002e-05 2024-05-08T08:53:27.590 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.590 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.590 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:27.590 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905431+0000", 2024-05-08T08:53:27.590 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7771999999999999e-05 2024-05-08T08:53:27.590 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.590 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:27.590 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.590 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.590 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.590 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22082 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 847872~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:27.590 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:53:27.590 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 59.712922227999996, 2024-05-08T08:53:27.590 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047831934999999999, 2024-05-08T08:53:27.591 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:27.591 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:27.591 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:27.591 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:27.591 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:27.591 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:27.591 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22082 2024-05-08T08:53:27.591 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.591 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:27.591 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.591 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:27.591 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:53:27.591 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:27.591 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.592 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.592 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:27.592 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:53:27.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:27.592 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.592 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.592 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:27.592 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:53:27.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:53:27.592 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.592 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.592 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:27.592 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857778+0000", 2024-05-08T08:53:27.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0789999999999999e-06 2024-05-08T08:53:27.592 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.593 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.593 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:27.593 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857784+0000", 2024-05-08T08:53:27.593 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0480000000000004e-06 2024-05-08T08:53:27.593 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.593 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.593 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:27.593 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857790+0000", 2024-05-08T08:53:27.593 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2719999999999997e-06 2024-05-08T08:53:27.593 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.593 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.593 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:27.593 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905458+0000", 2024-05-08T08:53:27.593 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047668470999999997 2024-05-08T08:53:27.593 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.594 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.594 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:27.594 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905521+0000", 2024-05-08T08:53:27.594 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2952000000000003e-05 2024-05-08T08:53:27.594 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.594 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.594 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:27.594 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905608+0000", 2024-05-08T08:53:27.594 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6759000000000001e-05 2024-05-08T08:53:27.594 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.594 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:27.594 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.594 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.594 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.595 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22084 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 864256~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:27.595 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:53:27.595 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 59.712898277999997, 2024-05-08T08:53:27.595 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047977748000000001, 2024-05-08T08:53:27.595 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:27.595 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:27.595 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:27.595 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:27.595 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:27.595 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:27.595 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22084 2024-05-08T08:53:27.595 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.595 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:27.595 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.595 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:27.596 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:53:27.596 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:27.596 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.596 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:27.596 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:53:27.596 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:27.596 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.596 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:27.596 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:53:27.596 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:53:27.596 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.596 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:27.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857802+0000", 2024-05-08T08:53:27.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0779999999999998e-06 2024-05-08T08:53:27.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.597 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:27.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857812+0000", 2024-05-08T08:53:27.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7939999999999997e-06 2024-05-08T08:53:27.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.597 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:27.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857817+0000", 2024-05-08T08:53:27.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8899999999999998e-06 2024-05-08T08:53:27.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:27.598 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905636+0000", 2024-05-08T08:53:27.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047819252 2024-05-08T08:53:27.598 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:27.598 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905700+0000", 2024-05-08T08:53:27.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4016999999999994e-05 2024-05-08T08:53:27.598 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:27.598 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905778+0000", 2024-05-08T08:53:27.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7306999999999997e-05 2024-05-08T08:53:27.598 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.598 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:27.599 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.599 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.599 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.599 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22086 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 880640~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:27.599 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:53:27.599 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 59.712874927999998, 2024-05-08T08:53:27.599 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048119149, 2024-05-08T08:53:27.599 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:27.599 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:27.599 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:27.599 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:27.599 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:27.599 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:27.599 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22086 2024-05-08T08:53:27.600 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.600 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:27.600 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.600 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:27.600 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:53:27.600 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:27.600 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.600 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.600 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:27.600 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:53:27.600 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:27.600 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.600 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.600 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:27.600 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857824+0000", 2024-05-08T08:53:27.601 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4200000000000002e-07 2024-05-08T08:53:27.601 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.601 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:27.601 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857826+0000", 2024-05-08T08:53:27.601 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0590000000000001e-06 2024-05-08T08:53:27.601 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.601 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:27.601 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857832+0000", 2024-05-08T08:53:27.601 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8029999999999999e-06 2024-05-08T08:53:27.601 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.601 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:27.601 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857837+0000", 2024-05-08T08:53:27.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7929999999999997e-06 2024-05-08T08:53:27.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:27.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905803+0000", 2024-05-08T08:53:27.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047965761000000003 2024-05-08T08:53:27.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:27.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905862+0000", 2024-05-08T08:53:27.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9203000000000002e-05 2024-05-08T08:53:27.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:27.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905942+0000", 2024-05-08T08:53:27.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0187999999999999e-05 2024-05-08T08:53:27.603 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.603 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:27.603 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.603 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.603 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22088 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 897024~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:27.603 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:53:27.603 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 59.712854876000002, 2024-05-08T08:53:27.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048256049000000002, 2024-05-08T08:53:27.603 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:27.603 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:27.603 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:27.603 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:27.603 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:27.604 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:27.604 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22088 2024-05-08T08:53:27.604 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.604 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:27.604 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.604 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:27.604 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:53:27.604 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:27.604 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.604 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.604 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:27.604 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:53:27.604 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:27.604 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.604 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.604 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:27.605 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857844+0000", 2024-05-08T08:53:27.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.46e-07 2024-05-08T08:53:27.605 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.605 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.605 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:27.605 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857846+0000", 2024-05-08T08:53:27.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0899999999999999e-06 2024-05-08T08:53:27.605 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.605 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.605 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:27.605 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857855+0000", 2024-05-08T08:53:27.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8219999999999996e-06 2024-05-08T08:53:27.605 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.605 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.605 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:27.605 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857860+0000", 2024-05-08T08:53:27.606 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2989999999999996e-06 2024-05-08T08:53:27.606 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.606 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:27.606 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905966+0000", 2024-05-08T08:53:27.606 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048106542000000002 2024-05-08T08:53:27.606 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.606 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:27.606 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906024+0000", 2024-05-08T08:53:27.606 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7958000000000003e-05 2024-05-08T08:53:27.606 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.606 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:27.607 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906099+0000", 2024-05-08T08:53:27.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4992000000000003e-05 2024-05-08T08:53:27.607 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.607 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:27.607 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.607 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22091 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 913408~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:27.607 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:53:27.607 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 59.712831115999997, 2024-05-08T08:53:27.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048395290000000001, 2024-05-08T08:53:27.607 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:27.607 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:27.607 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:27.607 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:27.607 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:27.608 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:27.608 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22091 2024-05-08T08:53:27.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.608 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:27.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.608 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:27.608 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:53:27.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:27.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.608 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:27.608 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:53:27.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:27.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.609 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:27.609 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:53:27.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4900000000000001e-07 2024-05-08T08:53:27.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.609 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:27.609 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857870+0000", 2024-05-08T08:53:27.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2510000000000001e-06 2024-05-08T08:53:27.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.609 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:27.609 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857877+0000", 2024-05-08T08:53:27.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0809999999999996e-06 2024-05-08T08:53:27.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.610 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:27.610 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857881+0000", 2024-05-08T08:53:27.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.724e-06 2024-05-08T08:53:27.610 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.610 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:27.610 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906127+0000", 2024-05-08T08:53:27.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048245345000000002 2024-05-08T08:53:27.610 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.610 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:27.610 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906190+0000", 2024-05-08T08:53:27.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2762000000000007e-05 2024-05-08T08:53:27.610 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.611 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:27.611 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906262+0000", 2024-05-08T08:53:27.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2778e-05 2024-05-08T08:53:27.611 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.611 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:27.611 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.611 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22093 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 929792~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:27.611 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:53:27.611 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 59.712808752000001, 2024-05-08T08:53:27.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048532875000000003, 2024-05-08T08:53:27.611 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:27.611 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:27.611 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:27.612 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:27.612 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:27.612 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:27.612 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22093 2024-05-08T08:53:27.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.612 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:27.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.612 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:27.612 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:53:27.612 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:27.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.612 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:27.612 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:53:27.612 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:27.613 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.613 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.613 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:27.613 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857890+0000", 2024-05-08T08:53:27.613 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-07 2024-05-08T08:53:27.613 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.613 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.613 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:27.613 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857899+0000", 2024-05-08T08:53:27.613 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6889999999999999e-06 2024-05-08T08:53:27.613 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.613 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.613 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:27.613 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857906+0000", 2024-05-08T08:53:27.613 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6490000000000002e-06 2024-05-08T08:53:27.613 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.614 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.614 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:27.614 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857917+0000", 2024-05-08T08:53:27.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0764999999999999e-05 2024-05-08T08:53:27.614 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.614 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.614 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:27.614 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906287+0000", 2024-05-08T08:53:27.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048370442 2024-05-08T08:53:27.614 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.614 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.614 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:27.614 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906349+0000", 2024-05-08T08:53:27.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1537000000000004e-05 2024-05-08T08:53:27.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:27.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906422+0000", 2024-05-08T08:53:27.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3431999999999998e-05 2024-05-08T08:53:27.615 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.615 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:27.615 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.615 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22095 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 946176~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:27.615 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:53:27.615 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 59.712774048, 2024-05-08T08:53:27.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048657593999999998, 2024-05-08T08:53:27.615 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:27.616 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:27.616 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:27.616 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:27.616 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:27.616 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:27.616 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22095 2024-05-08T08:53:27.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.616 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:27.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:27.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:53:27.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:27.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:27.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:53:27.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:27.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.617 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.617 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:27.617 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857925+0000", 2024-05-08T08:53:27.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3099999999999998e-07 2024-05-08T08:53:27.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.617 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.617 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:27.617 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857927+0000", 2024-05-08T08:53:27.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.34e-06 2024-05-08T08:53:27.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.617 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.617 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:27.617 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857938+0000", 2024-05-08T08:53:27.618 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1426e-05 2024-05-08T08:53:27.618 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.618 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.618 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:27.618 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857943+0000", 2024-05-08T08:53:27.618 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5140000000000003e-06 2024-05-08T08:53:27.618 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.618 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.618 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:27.618 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906448+0000", 2024-05-08T08:53:27.618 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048504641000000001 2024-05-08T08:53:27.618 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.618 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.618 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:27.619 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906510+0000", 2024-05-08T08:53:27.619 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2663000000000006e-05 2024-05-08T08:53:27.619 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.619 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.619 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:27.619 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906582+0000", 2024-05-08T08:53:27.619 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1478999999999998e-05 2024-05-08T08:53:27.619 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.619 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:27.619 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.619 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.619 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.619 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22097 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 962560~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:27.619 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:53:27.619 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 59.712746359, 2024-05-08T08:53:27.620 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048791353000000003, 2024-05-08T08:53:27.620 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:27.620 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:27.620 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:27.620 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:27.620 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:27.620 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:27.620 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22097 2024-05-08T08:53:27.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.620 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:27.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.620 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:27.620 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:53:27.620 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:27.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:27.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:53:27.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:27.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:27.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:53:27.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9200000000000002e-07 2024-05-08T08:53:27.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:27.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857954+0000", 2024-05-08T08:53:27.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1900000000000002e-06 2024-05-08T08:53:27.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.622 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.622 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:27.622 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857961+0000", 2024-05-08T08:53:27.622 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4989999999999999e-06 2024-05-08T08:53:27.622 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.622 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.622 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:27.622 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857971+0000", 2024-05-08T08:53:27.622 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0485999999999999e-05 2024-05-08T08:53:27.622 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.622 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.622 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:27.622 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906607+0000", 2024-05-08T08:53:27.622 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048635109000000003 2024-05-08T08:53:27.623 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.623 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.623 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:27.623 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906667+0000", 2024-05-08T08:53:27.623 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0439000000000002e-05 2024-05-08T08:53:27.623 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.623 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.623 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:27.623 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906743+0000", 2024-05-08T08:53:27.623 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6237999999999998e-05 2024-05-08T08:53:27.623 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.623 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:27.623 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.624 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.624 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22099 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 978944~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:27.624 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:53:27.624 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 59.712720199000003, 2024-05-08T08:53:27.624 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048932055000000002, 2024-05-08T08:53:27.624 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:27.624 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:27.624 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:27.624 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:27.624 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:27.624 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:27.624 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22099 2024-05-08T08:53:27.624 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.624 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:27.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.625 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:27.625 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:53:27.625 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:27.625 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.625 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.625 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:27.625 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:53:27.625 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:27.625 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.625 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.625 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:27.625 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:53:27.625 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.27e-07 2024-05-08T08:53:27.625 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.626 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.626 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:27.626 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857987+0000", 2024-05-08T08:53:27.626 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5890000000000003e-06 2024-05-08T08:53:27.626 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.626 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.626 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:27.626 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857994+0000", 2024-05-08T08:53:27.626 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6619999999999999e-06 2024-05-08T08:53:27.626 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.626 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.626 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:27.626 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857998+0000", 2024-05-08T08:53:27.626 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7110000000000003e-06 2024-05-08T08:53:27.627 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.627 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.627 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:27.627 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906772+0000", 2024-05-08T08:53:27.627 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048773415000000001 2024-05-08T08:53:27.627 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.627 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.627 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:27.627 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906834+0000", 2024-05-08T08:53:27.627 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1752000000000001e-05 2024-05-08T08:53:27.627 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.627 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.627 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:27.627 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906910+0000", 2024-05-08T08:53:27.627 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6499000000000001e-05 2024-05-08T08:53:27.628 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.628 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:27.628 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.628 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.628 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22101 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 995328~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:27.628 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:53:27.628 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 59.712693068, 2024-05-08T08:53:27.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049087364000000001, 2024-05-08T08:53:27.628 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:27.628 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:27.628 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:27.628 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:27.628 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:27.629 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:27.629 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22101 2024-05-08T08:53:27.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.629 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:27.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.629 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:27.629 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:53:27.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:27.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.629 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:27.629 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:53:27.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:27.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.630 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:27.630 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858006+0000", 2024-05-08T08:53:27.630 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1399999999999997e-07 2024-05-08T08:53:27.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.630 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.630 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:27.630 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858008+0000", 2024-05-08T08:53:27.630 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2060000000000001e-06 2024-05-08T08:53:27.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.630 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.630 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:27.630 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858017+0000", 2024-05-08T08:53:27.630 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1579999999999994e-06 2024-05-08T08:53:27.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.631 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.631 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:27.631 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858022+0000", 2024-05-08T08:53:27.631 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4199999999999998e-06 2024-05-08T08:53:27.631 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.631 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.631 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:27.631 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906940+0000", 2024-05-08T08:53:27.631 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048917627999999998 2024-05-08T08:53:27.631 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.631 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.631 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:27.631 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907002+0000", 2024-05-08T08:53:27.631 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2399999999999999e-05 2024-05-08T08:53:27.631 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.632 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.632 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:27.632 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907092+0000", 2024-05-08T08:53:27.632 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0137999999999997e-05 2024-05-08T08:53:27.632 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.632 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:27.632 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.632 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.632 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.632 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22103 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1011712~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:27.632 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:53:27.632 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 59.712668282999999, 2024-05-08T08:53:27.632 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049228542, 2024-05-08T08:53:27.632 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:27.632 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:27.633 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:27.633 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:27.633 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:27.633 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:27.633 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22103 2024-05-08T08:53:27.633 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.633 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:27.633 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.633 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:27.633 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:53:27.633 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:27.633 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.633 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.633 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:27.633 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:53:27.634 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:27.634 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.634 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:27.634 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:53:27.634 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4299999999999999e-07 2024-05-08T08:53:27.634 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.634 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:27.634 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858032+0000", 2024-05-08T08:53:27.634 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0590000000000001e-06 2024-05-08T08:53:27.634 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.634 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:27.635 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858037+0000", 2024-05-08T08:53:27.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.977e-06 2024-05-08T08:53:27.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.635 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.635 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:27.635 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858042+0000", 2024-05-08T08:53:27.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7650000000000001e-06 2024-05-08T08:53:27.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.635 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.635 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:27.635 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907118+0000", 2024-05-08T08:53:27.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049076147000000001 2024-05-08T08:53:27.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.635 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.636 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:27.636 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907180+0000", 2024-05-08T08:53:27.636 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1971999999999996e-05 2024-05-08T08:53:27.636 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.636 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:27.636 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907258+0000", 2024-05-08T08:53:27.636 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8279e-05 2024-05-08T08:53:27.636 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.636 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:27.636 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.636 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.636 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22105 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1028096~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:27.636 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:53:27.637 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 59.712649040000002, 2024-05-08T08:53:27.637 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049394609999999999, 2024-05-08T08:53:27.637 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:27.637 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:27.637 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:27.637 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:27.637 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:27.637 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:27.637 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22105 2024-05-08T08:53:27.637 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.637 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:27.637 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.637 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:27.637 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:53:27.637 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:27.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.638 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.638 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:27.638 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:53:27.638 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:27.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.638 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.638 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:27.638 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858050+0000", 2024-05-08T08:53:27.638 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5999999999999999e-07 2024-05-08T08:53:27.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.638 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.638 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:27.638 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858051+0000", 2024-05-08T08:53:27.638 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8870000000000001e-06 2024-05-08T08:53:27.639 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.639 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.639 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:27.639 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858057+0000", 2024-05-08T08:53:27.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8019999999999997e-06 2024-05-08T08:53:27.639 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.639 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.639 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:27.639 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858062+0000", 2024-05-08T08:53:27.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.561e-06 2024-05-08T08:53:27.639 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.639 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.639 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:27.639 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907289+0000", 2024-05-08T08:53:27.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049226957000000002 2024-05-08T08:53:27.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.640 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.640 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:27.640 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907356+0000", 2024-05-08T08:53:27.640 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6915000000000006e-05 2024-05-08T08:53:27.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.640 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.640 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:27.640 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907444+0000", 2024-05-08T08:53:27.640 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8127999999999994e-05 2024-05-08T08:53:27.640 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.640 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:27.640 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.641 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22107 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1044480~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:27.641 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:53:27.641 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 59.712630699999998, 2024-05-08T08:53:27.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049799942999999999, 2024-05-08T08:53:27.641 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:27.641 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:27.641 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:27.641 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:27.641 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:27.641 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:27.641 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22107 2024-05-08T08:53:27.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.641 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:27.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.641 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:27.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:53:27.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:27.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:27.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:53:27.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:27.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:27.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:53:27.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4799999999999999e-07 2024-05-08T08:53:27.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.643 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:27.643 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858070+0000", 2024-05-08T08:53:27.643 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.012e-06 2024-05-08T08:53:27.643 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.643 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.643 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:27.643 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858077+0000", 2024-05-08T08:53:27.643 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6290000000000001e-06 2024-05-08T08:53:27.643 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.643 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.643 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:27.643 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858083+0000", 2024-05-08T08:53:27.643 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8880000000000002e-06 2024-05-08T08:53:27.643 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.643 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.644 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:27.644 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907671+0000", 2024-05-08T08:53:27.644 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049587191000000003 2024-05-08T08:53:27.644 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.644 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.644 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:27.644 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907763+0000", 2024-05-08T08:53:27.644 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2231000000000005e-05 2024-05-08T08:53:27.644 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.644 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.644 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:27.644 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907867+0000", 2024-05-08T08:53:27.644 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104644 2024-05-08T08:53:27.644 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.644 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:27.645 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.645 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.645 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22294 3.1c 3:3aad0389:::rbd_data.11406886b07b.0000000000000080:head [stat,delete] snapc 7=[7] RETRY=6 ondisk+retry+write+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:27.645 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:53:27.645 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 59.712036851999997, 2024-05-08T08:53:27.645 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051152191, 2024-05-08T08:53:27.645 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:27.645 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:27.645 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:27.645 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:27.645 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:27.645 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:27.645 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22294 2024-05-08T08:53:27.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.645 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:27.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.646 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:27.646 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:53:27.646 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:27.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.646 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:27.646 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:53:27.646 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:27.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.646 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:27.646 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858662+0000", 2024-05-08T08:53:27.646 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4799999999999998e-07 2024-05-08T08:53:27.647 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.647 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.647 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:27.647 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858664+0000", 2024-05-08T08:53:27.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1229999999999998e-06 2024-05-08T08:53:27.647 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.647 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.647 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:27.647 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858670+0000", 2024-05-08T08:53:27.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3099999999999997e-06 2024-05-08T08:53:27.647 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.647 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.647 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:27.647 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858676+0000", 2024-05-08T08:53:27.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9750000000000004e-06 2024-05-08T08:53:27.648 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.648 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.648 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:27.648 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907901+0000", 2024-05-08T08:53:27.648 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049224520000000001 2024-05-08T08:53:27.648 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.648 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.648 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:27.648 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.908085+0000", 2024-05-08T08:53:27.648 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00018377500000000001 2024-05-08T08:53:27.648 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.648 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.648 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:27.648 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.909814+0000", 2024-05-08T08:53:27.648 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00172894 2024-05-08T08:53:27.648 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.649 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:27.649 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.649 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.649 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.649 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4416.0:13461 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-05-08T08:53:27.649 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:22.487456+0000", 2024-05-08T08:53:27.649 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.0832418109999997, 2024-05-08T08:53:27.649 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.99102620799999996, 2024-05-08T08:53:27.649 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:27.649 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:27.649 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:27.649 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:27.649 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4416", 2024-05-08T08:53:27.649 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:27.649 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 13461 2024-05-08T08:53:27.650 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.650 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:27.650 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.650 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:27.650 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487456+0000", 2024-05-08T08:53:27.650 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:27.650 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.650 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.650 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:27.650 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487456+0000", 2024-05-08T08:53:27.650 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:27.650 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.650 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.650 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:27.650 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487458+0000", 2024-05-08T08:53:27.651 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5659999999999999e-06 2024-05-08T08:53:27.651 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.651 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.651 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:27.651 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487463+0000", 2024-05-08T08:53:27.651 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6890000000000003e-06 2024-05-08T08:53:27.651 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.651 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.651 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:27.651 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487472+0000", 2024-05-08T08:53:27.651 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7710000000000001e-06 2024-05-08T08:53:27.651 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.651 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.651 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:27.651 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487478+0000", 2024-05-08T08:53:27.652 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1390000000000003e-06 2024-05-08T08:53:27.652 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.652 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.652 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:27.652 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487520+0000", 2024-05-08T08:53:27.652 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2259999999999999e-05 2024-05-08T08:53:27.652 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.652 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.652 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-05-08T08:53:27.652 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487528+0000", 2024-05-08T08:53:27.652 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7479999999999993e-06 2024-05-08T08:53:27.652 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.652 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.652 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:27.652 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:23.478139+0000", 2024-05-08T08:53:27.653 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.99061165200000001 2024-05-08T08:53:27.653 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.653 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.653 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:27.653 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:23.478301+0000", 2024-05-08T08:53:27.653 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000161602 2024-05-08T08:53:27.653 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:27.653 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:27.653 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:27.653 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:23.478483+0000", 2024-05-08T08:53:27.653 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000181781 2024-05-08T08:53:27.653 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.653 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:27.653 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.653 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:27.653 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:27.654 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:53:27.655 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:53:27.655 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:27.658+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:27.708 INFO:teuthology.orchestra.run.smithi196.stdout:3183 punch from 0x9028200 to 0x902ba00, (0x3800 bytes) 2024-05-08T08:53:27.718 INFO:teuthology.orchestra.run.smithi196.stdout:3184 read 0xa149e00 thru 0xa1540c9 (0xa2ca bytes) 2024-05-08T08:53:27.718 INFO:teuthology.orchestra.run.smithi196.stdout:3185 write 0xa335200 thru 0xa341615 (0xc416 bytes) 2024-05-08T08:53:27.718 INFO:teuthology.orchestra.run.smithi196.stdout:3186 write 0x5757e00 thru 0x575a946 (0x2b47 bytes) 2024-05-08T08:53:27.725 INFO:teuthology.orchestra.run.smithi196.stdout:3187 trunc from 0xa68e400 to 0x4ff0000 2024-05-08T08:53:27.756 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:27.759+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:27.783 INFO:teuthology.orchestra.run.smithi196.stdout:3188 write 0xc95d200 thru 0xc968e19 (0xbc1a bytes) 2024-05-08T08:53:27.784 INFO:teuthology.orchestra.run.smithi196.stdout:3190 read 0xb533000 thru 0xb53c68c (0x968d bytes) 2024-05-08T08:53:27.784 INFO:teuthology.orchestra.run.smithi196.stdout:3192 read 0xa643e00 thru 0xa650502 (0xc703 bytes) 2024-05-08T08:53:27.816 INFO:teuthology.orchestra.run.smithi196.stdout:3193 write 0xe92b600 thru 0xe93453b (0x8f3c bytes) 2024-05-08T08:53:27.817 INFO:teuthology.orchestra.run.smithi196.stdout:3196 read 0x24c4a00 thru 0x24d1cd5 (0xd2d6 bytes) 2024-05-08T08:53:27.817 INFO:teuthology.orchestra.run.smithi196.stdout:3197 read 0x899200 thru 0x8a3540 (0xa341 bytes) 2024-05-08T08:53:27.817 INFO:teuthology.orchestra.run.smithi196.stdout:3200 write 0x6ea7800 thru 0x6ea8cee (0x14ef bytes) 2024-05-08T08:53:27.817 INFO:teuthology.orchestra.run.smithi196.stdout:3201 write 0x22e7200 thru 0x22f1515 (0xa316 bytes) 2024-05-08T08:53:27.817 INFO:teuthology.orchestra.run.smithi196.stdout:3204 read 0x714b400 thru 0x71589e3 (0xd5e4 bytes) 2024-05-08T08:53:27.818 INFO:teuthology.orchestra.run.smithi196.stdout:3205 write 0x583a000 thru 0x5843b4b (0x9b4c bytes) 2024-05-08T08:53:27.818 INFO:teuthology.orchestra.run.smithi196.stdout:3210 read 0x73d4800 thru 0x73d80ff (0x3900 bytes) 2024-05-08T08:53:27.818 INFO:teuthology.orchestra.run.smithi196.stdout:3211 punch from 0x7a8e00 to 0x7aae00, (0x2000 bytes) 2024-05-08T08:53:27.827 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:53:27.827 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:53:27.828 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:53:27.828 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:53:27.840 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:53:27.882 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:27.884+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:27.958 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:27.961+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:28.006 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:53:28.007 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:53:28.007 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:53:28.007 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:53:28.007 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:53:28.017 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:53:28.060 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:28.062+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:28.100 INFO:teuthology.orchestra.run.smithi196.stdout:3212 trunc from 0xe934600 to 0xe512a00 2024-05-08T08:53:28.154 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:53:28.154 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:53:28.154 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:53:28.155 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:53:28.155 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.155 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6690 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1093632~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:28.155 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:53:28.155 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 56.440213518, 2024-05-08T08:53:28.155 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052462754, 2024-05-08T08:53:28.155 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:28.155 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:28.155 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:28.155 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:28.155 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:28.155 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:28.155 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6690 2024-05-08T08:53:28.155 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.156 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:28.156 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.156 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:28.156 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:53:28.156 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:28.156 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.156 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.156 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:28.156 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:53:28.156 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:28.156 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.156 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.156 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:28.156 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715346+0000", 2024-05-08T08:53:28.156 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.84e-07 2024-05-08T08:53:28.157 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.157 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.157 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:28.157 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715348+0000", 2024-05-08T08:53:28.157 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.114e-06 2024-05-08T08:53:28.157 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.157 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.157 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:28.157 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715353+0000", 2024-05-08T08:53:28.157 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5720000000000002e-06 2024-05-08T08:53:28.157 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.157 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.157 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:28.157 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715358+0000", 2024-05-08T08:53:28.157 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7140000000000001e-06 2024-05-08T08:53:28.157 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.158 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.158 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:28.158 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767638+0000", 2024-05-08T08:53:28.158 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052279562000000002 2024-05-08T08:53:28.158 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.158 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.158 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:28.158 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767713+0000", 2024-05-08T08:53:28.158 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5652e-05 2024-05-08T08:53:28.158 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.158 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.158 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:28.158 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767808+0000", 2024-05-08T08:53:28.158 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4755999999999999e-05 2024-05-08T08:53:28.158 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:28.159 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:28.159 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:28.159 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.159 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.159 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6692 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1110016~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:28.159 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:53:28.159 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 56.440194427999998, 2024-05-08T08:53:28.159 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052666495000000001, 2024-05-08T08:53:28.159 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:28.159 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:28.159 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:28.159 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:28.159 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:28.159 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:28.159 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6692 2024-05-08T08:53:28.159 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.160 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:28.160 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.160 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:28.160 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:53:28.160 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:28.160 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.160 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.160 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:28.160 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:53:28.160 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:28.160 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.160 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.160 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:28.160 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:53:28.160 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8599999999999999e-07 2024-05-08T08:53:28.160 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.161 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.161 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:28.161 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715372+0000", 2024-05-08T08:53:28.161 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0990000000000001e-06 2024-05-08T08:53:28.161 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.161 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.161 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:28.161 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715378+0000", 2024-05-08T08:53:28.161 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0680000000000002e-06 2024-05-08T08:53:28.161 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.161 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.161 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:28.161 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715383+0000", 2024-05-08T08:53:28.161 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8740000000000003e-06 2024-05-08T08:53:28.161 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.161 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.162 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:28.162 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767848+0000", 2024-05-08T08:53:28.162 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052464668999999998 2024-05-08T08:53:28.162 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.162 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.162 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:28.162 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767927+0000", 2024-05-08T08:53:28.162 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9228999999999996e-05 2024-05-08T08:53:28.162 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.162 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.162 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:28.162 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768031+0000", 2024-05-08T08:53:28.162 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010417 2024-05-08T08:53:28.162 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:28.162 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:28.163 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:28.163 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.163 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.163 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6694 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1126400~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:28.163 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:53:28.163 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 56.440169531000002, 2024-05-08T08:53:28.163 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052848829, 2024-05-08T08:53:28.163 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:28.163 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:28.163 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:28.163 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:28.163 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:28.163 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:28.163 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6694 2024-05-08T08:53:28.164 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.164 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:28.164 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.164 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:28.164 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:53:28.164 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:28.164 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.164 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.164 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:28.164 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:53:28.164 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:28.164 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.164 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.164 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:28.164 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715390+0000", 2024-05-08T08:53:28.164 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9900000000000001e-07 2024-05-08T08:53:28.165 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.165 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.165 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:28.165 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715392+0000", 2024-05-08T08:53:28.165 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8279999999999999e-06 2024-05-08T08:53:28.165 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.165 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.165 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:28.165 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715398+0000", 2024-05-08T08:53:28.165 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7619999999999998e-06 2024-05-08T08:53:28.165 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.165 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.165 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:28.166 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715403+0000", 2024-05-08T08:53:28.166 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7790000000000003e-06 2024-05-08T08:53:28.166 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.166 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.166 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:28.166 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768067+0000", 2024-05-08T08:53:28.166 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052663785999999997 2024-05-08T08:53:28.166 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.166 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.166 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:28.166 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768142+0000", 2024-05-08T08:53:28.166 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5018999999999994e-05 2024-05-08T08:53:28.166 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.166 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.166 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:28.167 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768238+0000", 2024-05-08T08:53:28.167 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6255999999999995e-05 2024-05-08T08:53:28.167 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:28.167 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:28.167 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:28.167 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.167 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.167 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6696 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1142784~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:28.167 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:53:28.167 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 56.440150023000001, 2024-05-08T08:53:28.167 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.053051057999999998, 2024-05-08T08:53:28.167 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:28.167 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:28.167 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:28.167 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:28.168 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:28.168 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:28.168 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6696 2024-05-08T08:53:28.168 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.168 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:28.168 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.168 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:28.168 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:53:28.168 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:28.168 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.168 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.168 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:28.168 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:53:28.168 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:28.168 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.168 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.169 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:28.169 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:53:28.169 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-05-08T08:53:28.169 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.169 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.169 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:28.169 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715411+0000", 2024-05-08T08:53:28.169 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9630000000000001e-06 2024-05-08T08:53:28.169 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.169 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.169 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:28.169 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715417+0000", 2024-05-08T08:53:28.169 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7250000000000002e-06 2024-05-08T08:53:28.169 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.169 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.170 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:28.170 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715421+0000", 2024-05-08T08:53:28.170 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4089999999999996e-06 2024-05-08T08:53:28.170 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.170 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.170 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:28.170 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768277+0000", 2024-05-08T08:53:28.170 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052855293999999997 2024-05-08T08:53:28.170 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.170 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.170 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:28.170 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768358+0000", 2024-05-08T08:53:28.170 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1720999999999999e-05 2024-05-08T08:53:28.170 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.170 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.171 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:28.171 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768460+0000", 2024-05-08T08:53:28.171 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010154600000000001 2024-05-08T08:53:28.171 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:28.171 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:28.171 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:28.171 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.171 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.171 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6698 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1159168~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:28.171 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:53:28.171 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 56.440131156, 2024-05-08T08:53:28.171 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.053245726, 2024-05-08T08:53:28.171 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:28.171 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:28.171 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:28.172 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:28.172 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:28.172 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:28.172 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6698 2024-05-08T08:53:28.172 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.172 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:28.172 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.172 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:28.172 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:53:28.172 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:28.172 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.172 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.172 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:28.172 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:53:28.172 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:28.172 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.173 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.173 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:28.173 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716936+0000", 2024-05-08T08:53:28.173 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001508665 2024-05-08T08:53:28.173 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.173 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.173 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:28.173 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716941+0000", 2024-05-08T08:53:28.173 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1910000000000002e-06 2024-05-08T08:53:28.173 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.173 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.173 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:28.173 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716950+0000", 2024-05-08T08:53:28.173 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1900000000000001e-06 2024-05-08T08:53:28.173 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.173 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.173 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:28.174 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716958+0000", 2024-05-08T08:53:28.174 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8019999999999992e-06 2024-05-08T08:53:28.174 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.174 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.174 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:28.174 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768499+0000", 2024-05-08T08:53:28.174 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051540926000000001 2024-05-08T08:53:28.174 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.174 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.174 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:28.174 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768574+0000", 2024-05-08T08:53:28.174 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4988999999999996e-05 2024-05-08T08:53:28.174 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.174 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.174 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:28.175 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768673+0000", 2024-05-08T08:53:28.175 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9963000000000005e-05 2024-05-08T08:53:28.175 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:28.175 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:28.175 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:28.175 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.175 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.175 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6706 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1224704~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:28.175 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:53:28.175 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 56.438525849000001, 2024-05-08T08:53:28.175 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052435598, 2024-05-08T08:53:28.175 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:28.175 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:28.175 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:28.175 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:28.175 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:28.176 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:28.176 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6706 2024-05-08T08:53:28.176 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.176 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:28.176 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.176 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:28.176 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:53:28.176 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:28.176 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.176 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.176 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:28.176 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:53:28.176 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:28.176 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.176 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.177 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:28.177 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:53:28.177 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:53:28.177 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.177 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.177 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:28.177 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717035+0000", 2024-05-08T08:53:28.177 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9290000000000001e-06 2024-05-08T08:53:28.177 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.177 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.177 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:28.177 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717041+0000", 2024-05-08T08:53:28.177 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6720000000000001e-06 2024-05-08T08:53:28.177 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.177 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.177 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:28.178 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717046+0000", 2024-05-08T08:53:28.178 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5650000000000003e-06 2024-05-08T08:53:28.178 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.178 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.178 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:28.178 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769311+0000", 2024-05-08T08:53:28.178 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052265645999999999 2024-05-08T08:53:28.178 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.178 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.178 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:28.178 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769379+0000", 2024-05-08T08:53:28.178 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7391000000000006e-05 2024-05-08T08:53:28.178 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.178 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.178 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:28.179 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769469+0000", 2024-05-08T08:53:28.179 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0025999999999995e-05 2024-05-08T08:53:28.179 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:28.179 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:28.179 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:28.179 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.179 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.179 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6708 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1241088~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:28.179 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:53:28.179 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 56.438507745000003, 2024-05-08T08:53:28.179 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052640552, 2024-05-08T08:53:28.179 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:28.179 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:28.179 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:28.179 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:28.179 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:28.180 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:28.180 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6708 2024-05-08T08:53:28.180 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.180 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:28.180 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.180 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:28.180 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:53:28.180 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:28.180 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.180 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.180 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:28.180 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:53:28.180 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:28.180 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.181 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.181 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:28.181 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717052+0000", 2024-05-08T08:53:28.181 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:53:28.181 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.181 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.181 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:28.181 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717053+0000", 2024-05-08T08:53:28.181 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8330000000000001e-06 2024-05-08T08:53:28.181 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.181 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.181 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:28.181 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717058+0000", 2024-05-08T08:53:28.181 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.933e-06 2024-05-08T08:53:28.181 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.181 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.181 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:28.182 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717063+0000", 2024-05-08T08:53:28.182 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2180000000000001e-06 2024-05-08T08:53:28.182 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.182 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.182 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:28.182 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769507+0000", 2024-05-08T08:53:28.182 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052444866999999999 2024-05-08T08:53:28.182 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.182 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.182 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:28.182 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769589+0000", 2024-05-08T08:53:28.182 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2039000000000005e-05 2024-05-08T08:53:28.182 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.182 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.182 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:28.183 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769692+0000", 2024-05-08T08:53:28.183 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102252 2024-05-08T08:53:28.183 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:28.183 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:28.183 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:28.183 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.183 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.183 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6710 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1257472~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:28.183 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:53:28.183 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 56.438489721000003, 2024-05-08T08:53:28.183 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.05283057, 2024-05-08T08:53:28.183 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:28.183 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:28.183 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:28.183 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:28.183 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:28.183 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:28.184 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6710 2024-05-08T08:53:28.184 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.184 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:28.184 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.184 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:28.184 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:53:28.184 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:28.184 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.184 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.184 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:28.184 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:53:28.184 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:28.184 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.184 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.184 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:28.184 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718951+0000", 2024-05-08T08:53:28.185 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0018817809999999999 2024-05-08T08:53:28.185 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.185 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.185 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:28.185 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718956+0000", 2024-05-08T08:53:28.185 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5399999999999997e-06 2024-05-08T08:53:28.185 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.185 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.185 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:28.185 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718964+0000", 2024-05-08T08:53:28.185 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6629999999999999e-06 2024-05-08T08:53:28.185 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.185 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.186 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:28.186 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718972+0000", 2024-05-08T08:53:28.186 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5229999999999998e-06 2024-05-08T08:53:28.186 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.186 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.186 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:28.186 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769729+0000", 2024-05-08T08:53:28.186 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050756959999999997 2024-05-08T08:53:28.186 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.186 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.186 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:28.186 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769803+0000", 2024-05-08T08:53:28.186 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4538999999999999e-05 2024-05-08T08:53:28.186 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.187 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.187 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:28.187 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769900+0000", 2024-05-08T08:53:28.187 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6564000000000006e-05 2024-05-08T08:53:28.187 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:28.187 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:28.187 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:28.187 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.187 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.187 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6726 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1388544~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:28.187 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:53:28.187 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 56.436414949000003, 2024-05-08T08:53:28.187 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052451506000000002, 2024-05-08T08:53:28.187 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:28.187 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:28.188 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:28.188 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:28.188 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:28.188 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:28.188 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6726 2024-05-08T08:53:28.188 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.188 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:28.188 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.188 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:28.188 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:53:28.188 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:28.188 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.188 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.188 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:28.188 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:53:28.189 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:28.189 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.189 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.189 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:28.189 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719200+0000", 2024-05-08T08:53:28.189 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5865999999999997e-05 2024-05-08T08:53:28.189 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.189 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.189 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:28.189 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719204+0000", 2024-05-08T08:53:28.189 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8630000000000001e-06 2024-05-08T08:53:28.189 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.189 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.189 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:28.189 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719215+0000", 2024-05-08T08:53:28.189 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1590000000000001e-05 2024-05-08T08:53:28.190 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.190 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.190 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:28.190 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719220+0000", 2024-05-08T08:53:28.190 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1619999999999999e-06 2024-05-08T08:53:28.190 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.190 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.190 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:28.190 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771425+0000", 2024-05-08T08:53:28.190 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052204249000000001 2024-05-08T08:53:28.190 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.190 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.190 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:28.190 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771498+0000", 2024-05-08T08:53:28.191 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3782999999999995e-05 2024-05-08T08:53:28.191 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.191 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.191 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:28.191 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771595+0000", 2024-05-08T08:53:28.191 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6992999999999998e-05 2024-05-08T08:53:28.191 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:28.191 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:28.191 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:28.191 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.191 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.191 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6728 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1404928~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:28.191 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:53:28.191 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 56.436327544000001, 2024-05-08T08:53:28.191 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052590747, 2024-05-08T08:53:28.191 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:28.191 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:28.192 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:28.192 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:28.192 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:28.192 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:28.192 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6728 2024-05-08T08:53:28.192 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.192 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:28.192 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.192 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:28.192 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:53:28.192 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:28.192 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.192 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.192 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:28.192 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:53:28.192 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:28.193 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.193 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.193 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:28.193 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719232+0000", 2024-05-08T08:53:28.193 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4000000000000001e-07 2024-05-08T08:53:28.193 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.193 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.193 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:28.193 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719234+0000", 2024-05-08T08:53:28.193 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4499999999999998e-06 2024-05-08T08:53:28.193 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.193 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.193 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:28.193 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719240+0000", 2024-05-08T08:53:28.193 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5720000000000002e-06 2024-05-08T08:53:28.193 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.194 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.194 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:28.194 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719245+0000", 2024-05-08T08:53:28.194 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8690000000000003e-06 2024-05-08T08:53:28.194 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.194 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.194 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:28.194 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771636+0000", 2024-05-08T08:53:28.194 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052390979999999997 2024-05-08T08:53:28.194 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.194 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.194 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:28.194 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771717+0000", 2024-05-08T08:53:28.194 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1291999999999993e-05 2024-05-08T08:53:28.194 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.195 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.195 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:28.195 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771822+0000", 2024-05-08T08:53:28.195 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104944 2024-05-08T08:53:28.195 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:28.195 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:28.195 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:28.195 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.195 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.195 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6730 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1421312~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:28.195 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:53:28.195 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 56.436307302000003, 2024-05-08T08:53:28.195 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052770477000000003, 2024-05-08T08:53:28.195 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:28.195 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:28.195 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:28.196 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:28.196 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:28.196 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:28.196 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6730 2024-05-08T08:53:28.196 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.196 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:28.196 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.196 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:28.196 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:53:28.196 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:28.196 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.196 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.196 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:28.196 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:53:28.196 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:28.196 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.197 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.197 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:28.197 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:53:28.197 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9000000000000002e-07 2024-05-08T08:53:28.197 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.197 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.197 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:28.197 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719254+0000", 2024-05-08T08:53:28.197 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4099999999999998e-06 2024-05-08T08:53:28.197 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.197 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.197 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:28.197 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719263+0000", 2024-05-08T08:53:28.197 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4109999999999996e-06 2024-05-08T08:53:28.198 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.198 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.198 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:28.198 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719268+0000", 2024-05-08T08:53:28.198 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2669999999999997e-06 2024-05-08T08:53:28.198 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.198 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.198 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:28.198 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771846+0000", 2024-05-08T08:53:28.198 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052577677000000003 2024-05-08T08:53:28.198 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.198 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.198 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:28.198 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771920+0000", 2024-05-08T08:53:28.198 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4460000000000002e-05 2024-05-08T08:53:28.199 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.199 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.199 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:28.199 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772022+0000", 2024-05-08T08:53:28.199 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010186199999999999 2024-05-08T08:53:28.199 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:28.199 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:28.199 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:28.199 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.199 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.199 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6732 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1437696~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:28.199 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:53:28.199 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 56.436282495, 2024-05-08T08:53:28.199 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052905575000000003, 2024-05-08T08:53:28.199 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:28.200 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:28.200 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:28.200 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:28.200 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:28.200 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:28.200 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6732 2024-05-08T08:53:28.200 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.200 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:28.200 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.200 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:28.200 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:53:28.200 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:28.200 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.200 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.200 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:28.200 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:53:28.201 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:28.201 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.201 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.201 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:28.201 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721392+0000", 2024-05-08T08:53:28.201 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002115287 2024-05-08T08:53:28.201 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.201 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.201 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:28.201 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721396+0000", 2024-05-08T08:53:28.201 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1679999999999997e-06 2024-05-08T08:53:28.201 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.201 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.201 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:28.201 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721404+0000", 2024-05-08T08:53:28.202 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4409999999999993e-06 2024-05-08T08:53:28.202 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.202 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.202 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:28.202 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721410+0000", 2024-05-08T08:53:28.202 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9150000000000001e-06 2024-05-08T08:53:28.202 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.202 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.202 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:28.202 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772061+0000", 2024-05-08T08:53:28.202 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050651068 2024-05-08T08:53:28.202 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.202 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.202 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:28.202 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772127+0000", 2024-05-08T08:53:28.202 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5424000000000002e-05 2024-05-08T08:53:28.203 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.203 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.203 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:28.203 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772182+0000", 2024-05-08T08:53:28.203 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5272000000000002e-05 2024-05-08T08:53:28.203 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:28.203 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:28.203 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:28.203 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.203 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.203 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6749 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1585152~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:28.203 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:53:28.203 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 56.433981725000002, 2024-05-08T08:53:28.203 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052350264000000001, 2024-05-08T08:53:28.203 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:28.203 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:28.204 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:28.204 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:28.204 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:28.204 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:28.204 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6749 2024-05-08T08:53:28.204 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.204 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:28.204 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.204 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:28.204 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:53:28.204 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:28.204 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.204 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.204 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:28.204 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:53:28.204 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:28.204 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.205 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.205 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:28.205 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721578+0000", 2024-05-08T08:53:28.205 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0399999999999996e-07 2024-05-08T08:53:28.205 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.205 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.205 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:28.205 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721580+0000", 2024-05-08T08:53:28.205 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5450000000000002e-06 2024-05-08T08:53:28.205 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.205 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.205 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:28.205 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721587+0000", 2024-05-08T08:53:28.205 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0190000000000001e-06 2024-05-08T08:53:28.205 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.205 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.206 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:28.206 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721593+0000", 2024-05-08T08:53:28.206 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5659999999999997e-06 2024-05-08T08:53:28.206 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.206 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.206 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:28.206 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773759+0000", 2024-05-08T08:53:28.206 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052166553999999997 2024-05-08T08:53:28.206 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.206 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.206 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:28.206 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773833+0000", 2024-05-08T08:53:28.206 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3913999999999997e-05 2024-05-08T08:53:28.206 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.207 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.207 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:28.207 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773927+0000", 2024-05-08T08:53:28.207 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4061999999999995e-05 2024-05-08T08:53:28.207 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:28.207 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:28.207 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:28.207 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.207 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.207 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6751 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1601536~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:28.207 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:53:28.207 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 56.433958445999998, 2024-05-08T08:53:28.207 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052461302000000001, 2024-05-08T08:53:28.207 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:28.207 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:28.207 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:28.208 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:28.208 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:28.208 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:28.208 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6751 2024-05-08T08:53:28.208 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.208 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:28.208 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.208 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:28.208 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:53:28.208 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:28.208 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.208 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.208 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:28.208 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:53:28.208 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:28.209 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.209 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.209 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:28.209 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722939+0000", 2024-05-08T08:53:28.209 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001338601 2024-05-08T08:53:28.209 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.209 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.209 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:28.209 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722944+0000", 2024-05-08T08:53:28.209 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4469999999999999e-06 2024-05-08T08:53:28.209 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.209 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.209 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:28.209 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722955+0000", 2024-05-08T08:53:28.210 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1892999999999999e-05 2024-05-08T08:53:28.210 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.210 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.210 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:28.210 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722963+0000", 2024-05-08T08:53:28.210 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1080000000000004e-06 2024-05-08T08:53:28.210 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.210 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.210 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:28.210 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773964+0000", 2024-05-08T08:53:28.210 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051001022 2024-05-08T08:53:28.210 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.210 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.210 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:28.211 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.774006+0000", 2024-05-08T08:53:28.211 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2729999999999999e-05 2024-05-08T08:53:28.211 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.211 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.211 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:28.211 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.774062+0000", 2024-05-08T08:53:28.211 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5501000000000003e-05 2024-05-08T08:53:28.211 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:28.211 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:28.211 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:28.211 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.211 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.211 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6760 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1683456~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:28.211 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:53:28.211 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 56.432449069999997, 2024-05-08T08:53:28.211 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052571312000000002, 2024-05-08T08:53:28.212 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:28.212 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:28.212 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:28.212 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:28.212 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:28.212 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:28.212 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6760 2024-05-08T08:53:28.212 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.212 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:28.212 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.212 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:28.212 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:53:28.212 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:28.212 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.212 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.213 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:28.213 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:53:28.213 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:28.213 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.213 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.213 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:28.213 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:53:28.213 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2100000000000002e-07 2024-05-08T08:53:28.213 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.213 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.213 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:28.213 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723113+0000", 2024-05-08T08:53:28.213 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3149999999999999e-06 2024-05-08T08:53:28.213 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.213 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.214 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:28.214 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723118+0000", 2024-05-08T08:53:28.214 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8860000000000001e-06 2024-05-08T08:53:28.214 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.214 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.214 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:28.214 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723123+0000", 2024-05-08T08:53:28.214 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6770000000000003e-06 2024-05-08T08:53:28.214 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.214 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.214 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:28.214 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775414+0000", 2024-05-08T08:53:28.214 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052290692999999999 2024-05-08T08:53:28.215 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.215 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.215 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:28.215 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775459+0000", 2024-05-08T08:53:28.215 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5639999999999997e-05 2024-05-08T08:53:28.215 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.215 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.215 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:28.215 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775681+0000", 2024-05-08T08:53:28.215 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00022168000000000001 2024-05-08T08:53:28.215 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:28.215 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:28.215 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:28.215 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.215 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.215 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6761 3.16 3:6bd6d6da:::rbd_data.1140782590a5.0000000000000015:6 [read 1095680~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:28.216 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:53:28.216 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 56.432429849999998, 2024-05-08T08:53:28.216 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052751275, 2024-05-08T08:53:28.216 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:28.216 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:28.216 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:28.216 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:28.216 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:28.216 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:28.216 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6761 2024-05-08T08:53:28.216 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.216 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:28.216 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.216 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:28.217 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:53:28.217 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:28.217 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.217 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.217 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:28.217 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:53:28.217 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:28.217 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.217 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.217 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:28.217 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724939+0000", 2024-05-08T08:53:28.217 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001809817 2024-05-08T08:53:28.217 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.217 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.217 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:28.218 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724943+0000", 2024-05-08T08:53:28.218 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4920000000000004e-06 2024-05-08T08:53:28.218 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.218 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.218 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:28.218 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724956+0000", 2024-05-08T08:53:28.218 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2427e-05 2024-05-08T08:53:28.218 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.218 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.218 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:28.218 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724963+0000", 2024-05-08T08:53:28.218 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2559999999999996e-06 2024-05-08T08:53:28.218 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.218 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.219 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:28.219 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775711+0000", 2024-05-08T08:53:28.219 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050747621 2024-05-08T08:53:28.219 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.219 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.219 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:28.219 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775785+0000", 2024-05-08T08:53:28.219 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4634000000000004e-05 2024-05-08T08:53:28.219 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.219 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.219 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:28.219 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775880+0000", 2024-05-08T08:53:28.219 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5027999999999999e-05 2024-05-08T08:53:28.219 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:28.219 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:28.220 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:28.220 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.220 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.220 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6770 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1765376~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:28.220 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:53:28.220 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 56.430398754999999, 2024-05-08T08:53:28.220 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052339296, 2024-05-08T08:53:28.220 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:28.220 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:28.220 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:28.220 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:28.220 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:28.220 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:28.220 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6770 2024-05-08T08:53:28.220 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.220 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:28.221 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.221 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:28.221 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:53:28.221 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:28.221 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.221 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.221 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:28.221 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:53:28.221 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:28.221 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.221 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.221 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:28.221 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725161+0000", 2024-05-08T08:53:28.221 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-05-08T08:53:28.221 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.222 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.222 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:28.222 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725163+0000", 2024-05-08T08:53:28.222 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4370000000000001e-06 2024-05-08T08:53:28.222 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.222 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.222 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:28.222 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725169+0000", 2024-05-08T08:53:28.222 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8409999999999996e-06 2024-05-08T08:53:28.222 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.222 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.222 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:28.222 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725174+0000", 2024-05-08T08:53:28.222 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7249999999999997e-06 2024-05-08T08:53:28.223 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.223 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.223 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:28.223 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777343+0000", 2024-05-08T08:53:28.223 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052169633999999999 2024-05-08T08:53:28.223 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.223 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.223 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:28.223 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777409+0000", 2024-05-08T08:53:28.223 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6033999999999998e-05 2024-05-08T08:53:28.223 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.223 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.223 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:28.223 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777499+0000", 2024-05-08T08:53:28.223 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0224999999999998e-05 2024-05-08T08:53:28.224 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:28.224 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:28.224 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:28.224 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.224 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.224 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6771 3.16 3:6bd6d6da:::rbd_data.1140782590a5.0000000000000015:6 [read 1177600~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:28.224 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:53:28.224 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 56.430378937999997, 2024-05-08T08:53:28.224 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052438604999999999, 2024-05-08T08:53:28.224 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:28.224 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:28.224 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:28.224 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:28.224 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:28.224 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:28.225 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6771 2024-05-08T08:53:28.225 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.225 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:28.225 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.225 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:28.225 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:53:28.225 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:28.225 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.225 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.225 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:28.225 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:53:28.225 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:28.225 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.225 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.225 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:28.225 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:53:28.226 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:53:28.226 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.226 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.226 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:28.226 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725188+0000", 2024-05-08T08:53:28.226 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1579999999999999e-06 2024-05-08T08:53:28.226 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.226 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.226 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:28.226 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725195+0000", 2024-05-08T08:53:28.226 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8909999999999995e-06 2024-05-08T08:53:28.226 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.226 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.226 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:28.227 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725200+0000", 2024-05-08T08:53:28.227 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9810000000000003e-06 2024-05-08T08:53:28.227 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.227 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.227 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:28.227 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777527+0000", 2024-05-08T08:53:28.227 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052326582000000003 2024-05-08T08:53:28.227 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.227 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.227 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:28.227 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777566+0000", 2024-05-08T08:53:28.227 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9447999999999999e-05 2024-05-08T08:53:28.227 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.227 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.227 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:28.227 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777619+0000", 2024-05-08T08:53:28.228 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2135000000000001e-05 2024-05-08T08:53:28.228 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:28.228 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:28.228 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:28.228 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.228 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.228 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6772 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1781760~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:28.228 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:53:28.228 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 56.430350881000003, 2024-05-08T08:53:28.228 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052601799999999997, 2024-05-08T08:53:28.228 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:28.228 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:28.228 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:28.228 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:28.228 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:28.228 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:28.229 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6772 2024-05-08T08:53:28.229 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.229 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:28.229 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.229 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:28.229 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:53:28.229 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:28.229 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.229 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.229 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:28.229 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:53:28.229 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:28.229 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.229 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.229 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:28.230 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726939+0000", 2024-05-08T08:53:28.230 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001730493 2024-05-08T08:53:28.230 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.230 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.230 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:28.230 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726943+0000", 2024-05-08T08:53:28.230 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7330000000000003e-06 2024-05-08T08:53:28.230 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.230 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.230 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:28.230 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726954+0000", 2024-05-08T08:53:28.230 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1107000000000001e-05 2024-05-08T08:53:28.230 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.230 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.231 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:28.231 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726962+0000", 2024-05-08T08:53:28.231 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.503e-06 2024-05-08T08:53:28.231 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.231 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.231 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:28.231 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777653+0000", 2024-05-08T08:53:28.231 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050691542999999999 2024-05-08T08:53:28.231 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.231 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.231 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:28.231 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777719+0000", 2024-05-08T08:53:28.231 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5228000000000006e-05 2024-05-08T08:53:28.231 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.231 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.232 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:28.232 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777810+0000", 2024-05-08T08:53:28.232 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1193000000000006e-05 2024-05-08T08:53:28.232 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:28.232 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:28.232 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:28.232 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.232 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.232 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6782 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1880064~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:28.232 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:53:28.232 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 56.428382374000002, 2024-05-08T08:53:28.232 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052326492000000002, 2024-05-08T08:53:28.232 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:28.232 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:28.233 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:28.233 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:28.233 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:28.233 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:28.233 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6782 2024-05-08T08:53:28.233 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.233 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:28.233 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.233 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:28.233 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:53:28.233 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:28.233 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.233 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.234 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:28.234 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:53:28.234 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:28.234 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.234 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.234 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:28.234 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728939+0000", 2024-05-08T08:53:28.234 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001762719 2024-05-08T08:53:28.234 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.234 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.234 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:28.234 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728944+0000", 2024-05-08T08:53:28.234 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.746e-06 2024-05-08T08:53:28.234 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.235 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.235 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:28.235 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728955+0000", 2024-05-08T08:53:28.235 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1043000000000001e-05 2024-05-08T08:53:28.235 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.235 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.235 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:28.235 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728963+0000", 2024-05-08T08:53:28.235 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5809999999999999e-06 2024-05-08T08:53:28.235 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.235 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.235 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:28.235 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779320+0000", 2024-05-08T08:53:28.235 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050357290999999998 2024-05-08T08:53:28.235 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.235 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.236 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:28.236 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779401+0000", 2024-05-08T08:53:28.236 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0574000000000005e-05 2024-05-08T08:53:28.236 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:28.236 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:28.236 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:28.236 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779503+0000", 2024-05-08T08:53:28.236 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102538 2024-05-08T08:53:28.236 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:28.236 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:28.236 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:28.236 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:28.236 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:28.236 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:53:28.239 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:28.240+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:28.239 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:53:28.239 INFO:teuthology.orchestra.run.smithi196.stdout:3214 write 0xc5dd000 thru 0xc5de06a (0x106b bytes) 2024-05-08T08:53:28.239 INFO:teuthology.orchestra.run.smithi196.stdout:3218 read 0x234f600 thru 0x235f26e (0xfc6f bytes) 2024-05-08T08:53:28.239 INFO:teuthology.orchestra.run.smithi196.stdout:3219 read 0x6b0800 thru 0x6b1222 (0xa23 bytes) 2024-05-08T08:53:28.240 INFO:teuthology.orchestra.run.smithi196.stdout:3220 punch from 0x4b67000 to 0x4b74200, (0xd200 bytes) 2024-05-08T08:53:28.240 INFO:teuthology.orchestra.run.smithi196.stdout:3221 write 0x975a800 thru 0x9766d84 (0xc585 bytes) 2024-05-08T08:53:28.240 INFO:teuthology.orchestra.run.smithi196.stdout:3225 write 0x9990200 thru 0x9991128 (0xf29 bytes) 2024-05-08T08:53:28.240 INFO:teuthology.orchestra.run.smithi196.stdout:3227 read 0x91bbc00 thru 0x91c5070 (0x9471 bytes) 2024-05-08T08:53:28.240 INFO:teuthology.orchestra.run.smithi196.stdout:3229 clone 22 order 19 su 65536 sc 9 2024-05-08T08:53:28.338 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:28.341+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:28.354 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:28.355 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:53:28.355 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:53:28.355 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:28.363 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:53:28.439 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:53:28.442+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:53:28.473 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:28.473 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:53:28.473 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:53:28.473 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:53:28.473 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:28.481 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:53:28.540 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:28.543+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:28.569 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T08:53:28.610 INFO:teuthology.orchestra.run.smithi196.stdout:leaving image image_client.0-clone21 intact 2024-05-08T08:53:28.641 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:28.641 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:53:28.641 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:53:28.642 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:53:28.642 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.642 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4621.0:9089 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 23339520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:53:28.642 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:53:28.642 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 56.150094432000003, 2024-05-08T08:53:28.642 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.13814907200000001, 2024-05-08T08:53:28.642 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:28.642 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:28.642 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:53:28.642 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:28.642 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4621", 2024-05-08T08:53:28.643 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:28.643 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9089 2024-05-08T08:53:28.643 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.643 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:28.643 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.643 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:28.643 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:53:28.643 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:28.643 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.643 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.643 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:28.643 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:53:28.643 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:28.644 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.644 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.644 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:28.644 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493005+0000", 2024-05-08T08:53:28.644 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1700000000000001e-06 2024-05-08T08:53:28.644 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.644 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.644 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:28.644 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.513963+0000", 2024-05-08T08:53:28.644 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.020957581999999999 2024-05-08T08:53:28.644 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.644 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.645 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:28.645 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.513995+0000", 2024-05-08T08:53:28.645 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1993999999999999e-05 2024-05-08T08:53:28.645 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.645 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.645 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:28.645 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514007+0000", 2024-05-08T08:53:28.645 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1581e-05 2024-05-08T08:53:28.645 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.645 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.645 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:28.645 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514122+0000", 2024-05-08T08:53:28.645 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000115282 2024-05-08T08:53:28.645 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.646 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.646 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op must wait for map", 2024-05-08T08:53:28.646 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514128+0000", 2024-05-08T08:53:28.646 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9950000000000002e-06 2024-05-08T08:53:28.646 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.646 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.646 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:28.646 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.515800+0000", 2024-05-08T08:53:28.646 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001671886 2024-05-08T08:53:28.646 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.646 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.647 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:28.647 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.515893+0000", 2024-05-08T08:53:28.647 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3490000000000001e-05 2024-05-08T08:53:28.647 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.647 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.647 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 3", 2024-05-08T08:53:28.647 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.517647+0000", 2024-05-08T08:53:28.647 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0017535630000000001 2024-05-08T08:53:28.647 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.647 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.647 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:53:28.648 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.627578+0000", 2024-05-08T08:53:28.648 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.109931557 2024-05-08T08:53:28.648 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.648 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.648 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:53:28.648 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631055+0000", 2024-05-08T08:53:28.648 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0034761420000000002 2024-05-08T08:53:28.648 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.648 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.648 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:53:28.648 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631096+0000", 2024-05-08T08:53:28.648 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1059000000000001e-05 2024-05-08T08:53:28.648 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.649 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.649 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:28.649 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631151+0000", 2024-05-08T08:53:28.649 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5770999999999998e-05 2024-05-08T08:53:28.649 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:28.649 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:28.649 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:28.649 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.649 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.649 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1067 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 10985472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:28.649 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485431+0000", 2024-05-08T08:53:28.649 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 52.157665610999999, 2024-05-08T08:53:28.650 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070412460999999996, 2024-05-08T08:53:28.650 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:28.650 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:28.650 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:28.650 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:28.650 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:28.650 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:28.650 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1067 2024-05-08T08:53:28.650 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.650 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:28.650 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.650 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:28.650 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485431+0000", 2024-05-08T08:53:28.651 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:28.651 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.651 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.651 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:28.651 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485431+0000", 2024-05-08T08:53:28.651 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:28.651 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.651 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.651 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:28.651 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485432+0000", 2024-05-08T08:53:28.651 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8299999999999998e-07 2024-05-08T08:53:28.651 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.651 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.651 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:28.652 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485434+0000", 2024-05-08T08:53:28.652 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0849999999999999e-06 2024-05-08T08:53:28.652 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.652 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.652 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:28.652 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485440+0000", 2024-05-08T08:53:28.652 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0020000000000003e-06 2024-05-08T08:53:28.652 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.652 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.652 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:28.652 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485445+0000", 2024-05-08T08:53:28.652 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5180000000000004e-06 2024-05-08T08:53:28.652 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.652 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.653 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:28.653 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555731+0000", 2024-05-08T08:53:28.653 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070286214 2024-05-08T08:53:28.653 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.653 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.653 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:28.653 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555787+0000", 2024-05-08T08:53:28.653 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5992999999999997e-05 2024-05-08T08:53:28.653 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.653 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.653 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:28.653 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555844+0000", 2024-05-08T08:53:28.653 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6266e-05 2024-05-08T08:53:28.653 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:28.654 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:28.654 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:28.654 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.654 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.654 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1071 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11026432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:28.654 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:53:28.654 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 52.157644941000001, 2024-05-08T08:53:28.654 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070580037999999998, 2024-05-08T08:53:28.654 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:28.654 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:28.654 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:28.654 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:28.654 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:28.655 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:28.655 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1071 2024-05-08T08:53:28.655 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.655 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:28.655 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.655 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:28.655 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:53:28.655 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:28.655 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.655 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.655 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:28.655 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:53:28.655 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:28.656 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.656 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.656 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:28.656 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:53:28.656 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2800000000000002e-07 2024-05-08T08:53:28.656 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.656 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.656 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:28.656 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485454+0000", 2024-05-08T08:53:28.656 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2290000000000002e-06 2024-05-08T08:53:28.656 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.656 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.656 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:28.656 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485461+0000", 2024-05-08T08:53:28.657 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9920000000000001e-06 2024-05-08T08:53:28.657 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.657 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.657 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:28.657 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485467+0000", 2024-05-08T08:53:28.657 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3979999999999998e-06 2024-05-08T08:53:28.657 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.657 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.657 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:28.657 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555880+0000", 2024-05-08T08:53:28.657 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070413165 2024-05-08T08:53:28.657 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.657 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.658 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:28.658 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555926+0000", 2024-05-08T08:53:28.658 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6217000000000002e-05 2024-05-08T08:53:28.658 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.658 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.658 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:28.658 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556032+0000", 2024-05-08T08:53:28.658 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000105609 2024-05-08T08:53:28.658 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:28.658 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:28.658 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:28.658 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.658 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.659 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1073 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11067392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:28.659 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:53:28.659 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 52.157623719999997, 2024-05-08T08:53:28.659 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070821404000000004, 2024-05-08T08:53:28.659 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:28.659 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:28.659 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:28.659 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:28.659 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:28.659 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:28.659 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1073 2024-05-08T08:53:28.659 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.660 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:28.660 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.660 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:28.660 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:53:28.660 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:28.660 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.660 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.660 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:28.660 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:53:28.660 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:28.660 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.660 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.661 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:28.661 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:53:28.661 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.08e-07 2024-05-08T08:53:28.661 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.661 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.661 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:28.661 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485476+0000", 2024-05-08T08:53:28.661 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3929999999999998e-06 2024-05-08T08:53:28.661 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.661 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.661 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:28.661 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485485+0000", 2024-05-08T08:53:28.661 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1880000000000008e-06 2024-05-08T08:53:28.661 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.662 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.662 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:28.662 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485494+0000", 2024-05-08T08:53:28.662 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3109999999999995e-06 2024-05-08T08:53:28.662 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.662 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.662 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:28.662 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556082+0000", 2024-05-08T08:53:28.662 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070587426999999994 2024-05-08T08:53:28.662 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.662 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.662 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:28.662 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556177+0000", 2024-05-08T08:53:28.663 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5051999999999997e-05 2024-05-08T08:53:28.663 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.663 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.663 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:28.663 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556294+0000", 2024-05-08T08:53:28.663 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000117625 2024-05-08T08:53:28.663 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:28.663 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:28.663 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:28.663 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.663 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.663 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1075 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11108352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:28.663 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:53:28.663 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 52.157593143, 2024-05-08T08:53:28.664 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071045026999999997, 2024-05-08T08:53:28.664 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:28.664 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:28.664 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:28.664 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:28.664 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:28.664 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:28.664 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1075 2024-05-08T08:53:28.664 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.664 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:28.664 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.664 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:28.664 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:53:28.665 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:28.665 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.665 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.665 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:28.665 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:53:28.665 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:28.665 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.665 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.665 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:28.665 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:53:28.665 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.32e-07 2024-05-08T08:53:28.665 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.665 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.666 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:28.666 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485518+0000", 2024-05-08T08:53:28.666 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3769e-05 2024-05-08T08:53:28.666 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.666 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.666 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:28.666 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485527+0000", 2024-05-08T08:53:28.666 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6779999999999995e-06 2024-05-08T08:53:28.666 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.666 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.666 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:28.666 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485532+0000", 2024-05-08T08:53:28.667 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9830000000000004e-06 2024-05-08T08:53:28.667 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.667 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.667 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:28.667 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556340+0000", 2024-05-08T08:53:28.667 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070807743000000006 2024-05-08T08:53:28.667 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.667 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.667 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:28.667 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556435+0000", 2024-05-08T08:53:28.667 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4732e-05 2024-05-08T08:53:28.667 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.668 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.668 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:28.668 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556549+0000", 2024-05-08T08:53:28.668 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011369 2024-05-08T08:53:28.668 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:28.668 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:28.668 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:28.668 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.668 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.668 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1078 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11149312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:28.668 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:53:28.668 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 52.157552944000003, 2024-05-08T08:53:28.668 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071167276000000002, 2024-05-08T08:53:28.669 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:28.669 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:28.669 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:28.669 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:28.669 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:28.669 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:28.669 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1078 2024-05-08T08:53:28.669 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.669 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:28.669 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.669 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:28.669 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:53:28.669 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:28.670 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.670 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.670 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:28.670 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:53:28.670 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:28.670 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.670 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.670 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:28.670 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:53:28.670 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5600000000000001e-07 2024-05-08T08:53:28.670 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.670 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.670 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:28.671 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485547+0000", 2024-05-08T08:53:28.671 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3590000000000002e-06 2024-05-08T08:53:28.671 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.671 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.671 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:28.671 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485553+0000", 2024-05-08T08:53:28.671 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2770000000000002e-06 2024-05-08T08:53:28.671 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.671 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.671 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:28.671 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485558+0000", 2024-05-08T08:53:28.671 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.95e-06 2024-05-08T08:53:28.671 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.672 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.672 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:28.672 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556582+0000", 2024-05-08T08:53:28.672 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071024444000000006 2024-05-08T08:53:28.672 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.672 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.672 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:28.672 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556637+0000", 2024-05-08T08:53:28.672 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4672000000000001e-05 2024-05-08T08:53:28.672 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.672 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.672 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:28.672 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556711+0000", 2024-05-08T08:53:28.672 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4117999999999997e-05 2024-05-08T08:53:28.673 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:28.673 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:28.673 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:28.673 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.673 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.673 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1080 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11190272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:28.673 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:53:28.673 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 52.157532211000003, 2024-05-08T08:53:28.673 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071350721000000006, 2024-05-08T08:53:28.673 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:28.673 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:28.673 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:28.673 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:28.673 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:28.674 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:28.674 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1080 2024-05-08T08:53:28.674 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.674 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:28.674 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.674 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:28.674 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:53:28.674 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:28.674 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.674 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.674 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:28.674 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:53:28.674 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:28.674 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.675 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.675 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:28.675 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:53:28.675 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.03e-07 2024-05-08T08:53:28.675 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.675 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.675 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:28.675 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485567+0000", 2024-05-08T08:53:28.675 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1090000000000001e-06 2024-05-08T08:53:28.675 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.675 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.675 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:28.675 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485573+0000", 2024-05-08T08:53:28.676 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3049999999999998e-06 2024-05-08T08:53:28.676 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.676 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.676 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:28.676 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485578+0000", 2024-05-08T08:53:28.676 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0130000000000001e-06 2024-05-08T08:53:28.676 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.676 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.676 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:28.676 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556743+0000", 2024-05-08T08:53:28.676 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071164449000000005 2024-05-08T08:53:28.676 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.676 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.676 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:28.676 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556797+0000", 2024-05-08T08:53:28.677 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4589999999999997e-05 2024-05-08T08:53:28.677 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.677 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.677 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:28.677 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556915+0000", 2024-05-08T08:53:28.677 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000117852 2024-05-08T08:53:28.677 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:28.677 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:28.677 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:28.677 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.677 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.677 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1082 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11231232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:28.677 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:53:28.677 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 52.157511968000001, 2024-05-08T08:53:28.678 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071591396000000002, 2024-05-08T08:53:28.678 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:28.678 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:28.678 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:28.678 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:28.678 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:28.678 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:28.678 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1082 2024-05-08T08:53:28.678 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.678 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:28.678 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.678 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:28.678 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:53:28.679 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:28.679 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.679 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.679 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:28.679 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:53:28.679 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:28.679 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.679 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.679 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:28.679 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:53:28.679 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1800000000000001e-07 2024-05-08T08:53:28.679 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.680 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.680 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:28.680 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485587+0000", 2024-05-08T08:53:28.680 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2589999999999999e-06 2024-05-08T08:53:28.680 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.680 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.680 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:28.680 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485594+0000", 2024-05-08T08:53:28.680 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0789999999999997e-06 2024-05-08T08:53:28.680 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.680 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.680 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:28.681 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485598+0000", 2024-05-08T08:53:28.681 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7480000000000001e-06 2024-05-08T08:53:28.681 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.681 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.681 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:28.681 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556971+0000", 2024-05-08T08:53:28.681 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071372774999999999 2024-05-08T08:53:28.681 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.681 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.681 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:28.681 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557069+0000", 2024-05-08T08:53:28.681 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.7732999999999994e-05 2024-05-08T08:53:28.681 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.681 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.681 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:28.682 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557176+0000", 2024-05-08T08:53:28.682 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000107384 2024-05-08T08:53:28.682 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:28.682 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:28.682 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:28.682 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.682 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.682 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1084 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11272192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:28.682 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:53:28.682 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 52.157473328999998, 2024-05-08T08:53:28.682 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071808977999999996, 2024-05-08T08:53:28.682 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:28.683 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:28.683 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:28.683 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:28.683 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:28.683 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:28.683 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1084 2024-05-08T08:53:28.683 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.683 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:28.683 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.683 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:28.683 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:53:28.683 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:28.684 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.684 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.684 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:28.684 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:53:28.684 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:28.684 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.684 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.684 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:28.684 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485624+0000", 2024-05-08T08:53:28.684 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:53:28.684 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.684 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.684 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:28.685 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485626+0000", 2024-05-08T08:53:28.685 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9460000000000001e-06 2024-05-08T08:53:28.685 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.685 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.685 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:28.685 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485633+0000", 2024-05-08T08:53:28.685 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1149999999999996e-06 2024-05-08T08:53:28.685 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.685 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.685 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:28.685 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485638+0000", 2024-05-08T08:53:28.685 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5670000000000002e-06 2024-05-08T08:53:28.685 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.686 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.686 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:28.686 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557226+0000", 2024-05-08T08:53:28.686 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071587267999999996 2024-05-08T08:53:28.686 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.686 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.686 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:28.686 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557321+0000", 2024-05-08T08:53:28.686 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5785000000000005e-05 2024-05-08T08:53:28.686 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.686 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.686 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:28.687 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557432+0000", 2024-05-08T08:53:28.687 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011092800000000001 2024-05-08T08:53:28.687 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:28.687 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:28.687 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:28.687 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.687 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.687 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1086 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11313152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:28.687 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:53:28.687 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 52.157450001999997, 2024-05-08T08:53:28.687 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071933132999999996, 2024-05-08T08:53:28.687 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:28.687 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:28.688 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:28.688 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:28.688 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:28.688 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:28.688 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1086 2024-05-08T08:53:28.688 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.688 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:28.688 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.688 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:28.688 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:53:28.688 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:28.688 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.689 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.689 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:28.689 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:53:28.689 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:28.689 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.689 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.689 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:28.689 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486328+0000", 2024-05-08T08:53:28.689 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00068107900000000002 2024-05-08T08:53:28.689 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.689 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.689 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:28.689 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486331+0000", 2024-05-08T08:53:28.690 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.585e-06 2024-05-08T08:53:28.690 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.690 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.690 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:28.690 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486343+0000", 2024-05-08T08:53:28.690 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1856e-05 2024-05-08T08:53:28.690 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.690 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.690 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:28.690 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486350+0000", 2024-05-08T08:53:28.690 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8600000000000004e-06 2024-05-08T08:53:28.690 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.690 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.691 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:28.691 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557470+0000", 2024-05-08T08:53:28.691 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071120326999999997 2024-05-08T08:53:28.691 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.691 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.691 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:28.691 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557514+0000", 2024-05-08T08:53:28.691 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3909000000000003e-05 2024-05-08T08:53:28.691 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.691 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.691 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:28.691 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557580+0000", 2024-05-08T08:53:28.691 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5517000000000003e-05 2024-05-08T08:53:28.692 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:28.692 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:28.692 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:28.692 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.692 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.692 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1089 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11354112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:28.692 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:53:28.692 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 52.156737130000003, 2024-05-08T08:53:28.692 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071457381, 2024-05-08T08:53:28.692 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:28.692 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:28.692 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:28.692 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:28.693 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:28.693 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:28.693 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1089 2024-05-08T08:53:28.693 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.693 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:28.693 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.693 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:28.693 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:53:28.693 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:28.693 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.693 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.693 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:28.693 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:53:28.693 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:28.693 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.694 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.694 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:28.694 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486851+0000", 2024-05-08T08:53:28.694 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00049102 2024-05-08T08:53:28.694 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.694 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.694 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:28.694 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486854+0000", 2024-05-08T08:53:28.694 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6859999999999999e-06 2024-05-08T08:53:28.694 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.694 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.694 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:28.694 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486864+0000", 2024-05-08T08:53:28.695 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6080000000000002e-06 2024-05-08T08:53:28.695 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.695 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.695 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:28.695 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486870+0000", 2024-05-08T08:53:28.695 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.477e-06 2024-05-08T08:53:28.695 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.695 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.695 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:28.695 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557612+0000", 2024-05-08T08:53:28.695 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070741557999999996 2024-05-08T08:53:28.695 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.695 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.695 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:28.696 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557695+0000", 2024-05-08T08:53:28.696 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3300000000000005e-05 2024-05-08T08:53:28.696 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.696 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.696 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:28.696 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557817+0000", 2024-05-08T08:53:28.696 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012173200000000001 2024-05-08T08:53:28.696 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:28.696 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:28.696 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:28.696 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.696 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.697 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1091 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11395072~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:28.697 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:53:28.697 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 52.156216335000003, 2024-05-08T08:53:28.697 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071184737999999997, 2024-05-08T08:53:28.697 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:28.697 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:28.697 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:28.697 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:28.697 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:28.697 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:28.697 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1091 2024-05-08T08:53:28.697 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.698 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:28.698 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.698 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:28.698 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:53:28.698 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:28.698 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.698 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.698 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:28.698 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:53:28.698 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:28.698 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.698 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.698 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:28.698 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486881+0000", 2024-05-08T08:53:28.699 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0699999999999997e-07 2024-05-08T08:53:28.699 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.699 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.699 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:28.699 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486883+0000", 2024-05-08T08:53:28.699 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.402e-06 2024-05-08T08:53:28.699 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.699 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.699 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:28.699 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486890+0000", 2024-05-08T08:53:28.699 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0480000000000001e-06 2024-05-08T08:53:28.699 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.699 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.700 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:28.700 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486896+0000", 2024-05-08T08:53:28.700 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7640000000000002e-06 2024-05-08T08:53:28.700 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.700 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.700 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:28.700 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557860+0000", 2024-05-08T08:53:28.700 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070963791999999998 2024-05-08T08:53:28.700 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.700 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.700 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:28.700 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557952+0000", 2024-05-08T08:53:28.700 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2082000000000004e-05 2024-05-08T08:53:28.700 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.701 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.701 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:28.701 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558065+0000", 2024-05-08T08:53:28.701 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000113143 2024-05-08T08:53:28.701 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:28.701 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:28.701 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:28.701 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.701 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.701 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1094 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11436032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:28.701 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:53:28.701 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 52.156193166999998, 2024-05-08T08:53:28.701 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071400507000000002, 2024-05-08T08:53:28.701 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:28.702 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:28.702 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:28.702 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:28.702 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:28.702 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:28.702 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1094 2024-05-08T08:53:28.702 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.702 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:28.702 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.702 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:28.702 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:53:28.702 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:28.702 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.703 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.703 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:28.703 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:53:28.703 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:28.703 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.703 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.703 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:28.703 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487940+0000", 2024-05-08T08:53:28.703 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001036868 2024-05-08T08:53:28.703 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.703 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.703 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:28.703 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487944+0000", 2024-05-08T08:53:28.704 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7069999999999998e-06 2024-05-08T08:53:28.704 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.704 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.704 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:28.704 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487951+0000", 2024-05-08T08:53:28.704 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2649999999999999e-06 2024-05-08T08:53:28.704 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.704 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.704 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:28.704 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487961+0000", 2024-05-08T08:53:28.704 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4110000000000002e-06 2024-05-08T08:53:28.704 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.704 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.705 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:28.705 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558117+0000", 2024-05-08T08:53:28.705 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070156662999999994 2024-05-08T08:53:28.705 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.705 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.705 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:28.705 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558214+0000", 2024-05-08T08:53:28.705 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6444e-05 2024-05-08T08:53:28.705 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.705 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.705 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:28.705 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558304+0000", 2024-05-08T08:53:28.705 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0148999999999994e-05 2024-05-08T08:53:28.705 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:28.706 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:28.706 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:28.706 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.706 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.706 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1096 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11476992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:28.706 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:53:28.706 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 52.155126635999999, 2024-05-08T08:53:28.706 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070480074000000004, 2024-05-08T08:53:28.706 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:28.706 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:28.706 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:28.706 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:28.706 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:28.707 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:28.707 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1096 2024-05-08T08:53:28.707 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.707 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:28.707 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.707 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:28.707 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:53:28.707 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:28.707 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.707 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.707 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:28.707 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:53:28.707 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:28.708 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.708 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.708 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:28.708 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:53:28.708 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3700000000000001e-07 2024-05-08T08:53:28.708 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.708 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.708 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:28.708 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487973+0000", 2024-05-08T08:53:28.708 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0870000000000001e-06 2024-05-08T08:53:28.708 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.708 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.709 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:28.709 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487977+0000", 2024-05-08T08:53:28.709 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0600000000000001e-06 2024-05-08T08:53:28.709 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.709 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.709 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:28.709 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487980+0000", 2024-05-08T08:53:28.709 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1089999999999998e-06 2024-05-08T08:53:28.709 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.709 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.709 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:28.709 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558338+0000", 2024-05-08T08:53:28.709 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070358233000000006 2024-05-08T08:53:28.710 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.710 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.710 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:28.710 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558392+0000", 2024-05-08T08:53:28.710 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4537999999999999e-05 2024-05-08T08:53:28.710 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.710 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.710 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:28.710 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558450+0000", 2024-05-08T08:53:28.710 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7710000000000001e-05 2024-05-08T08:53:28.710 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:28.711 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:28.711 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:28.711 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.711 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.711 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1099 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11517952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:28.711 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:53:28.711 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 52.155112760000002, 2024-05-08T08:53:28.711 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070720506000000002, 2024-05-08T08:53:28.711 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:28.711 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:28.711 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:28.711 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:28.711 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:28.712 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:28.712 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1099 2024-05-08T08:53:28.712 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.712 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:28.712 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.712 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:28.712 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:53:28.712 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:28.712 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.712 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.712 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:28.712 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:53:28.712 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:28.712 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.712 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.713 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:28.713 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:53:28.713 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3999999999999998e-07 2024-05-08T08:53:28.713 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.713 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.713 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:28.713 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487986+0000", 2024-05-08T08:53:28.713 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3519999999999999e-06 2024-05-08T08:53:28.713 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.713 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.713 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:28.713 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487989+0000", 2024-05-08T08:53:28.713 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.23e-06 2024-05-08T08:53:28.714 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.714 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.714 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:28.714 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487991+0000", 2024-05-08T08:53:28.714 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6350000000000002e-06 2024-05-08T08:53:28.714 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.714 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.714 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:28.714 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558489+0000", 2024-05-08T08:53:28.714 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070497924000000003 2024-05-08T08:53:28.714 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.714 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.714 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:28.715 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558583+0000", 2024-05-08T08:53:28.715 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3405000000000004e-05 2024-05-08T08:53:28.715 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.715 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.715 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:28.715 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558704+0000", 2024-05-08T08:53:28.715 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012171999999999999 2024-05-08T08:53:28.715 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:28.715 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:28.715 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:28.715 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.715 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.716 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1100 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11558912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:28.716 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:53:28.716 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 52.155099761000002, 2024-05-08T08:53:28.716 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070962266999999996, 2024-05-08T08:53:28.716 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:28.716 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:28.716 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:28.716 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:28.716 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:28.716 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:28.716 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1100 2024-05-08T08:53:28.716 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.716 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:28.717 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.717 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:28.717 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:53:28.717 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:28.717 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.717 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.717 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:28.717 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:53:28.717 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:28.717 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.717 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.717 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:28.717 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488430+0000", 2024-05-08T08:53:28.717 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000432676 2024-05-08T08:53:28.718 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.718 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.718 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:28.718 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488433+0000", 2024-05-08T08:53:28.718 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.45e-06 2024-05-08T08:53:28.718 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.718 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.718 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:28.718 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488442+0000", 2024-05-08T08:53:28.718 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.687e-06 2024-05-08T08:53:28.718 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.718 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.719 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:28.719 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488446+0000", 2024-05-08T08:53:28.719 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4939999999999997e-06 2024-05-08T08:53:28.719 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.719 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.719 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:28.719 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558756+0000", 2024-05-08T08:53:28.719 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070310051999999998 2024-05-08T08:53:28.719 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.719 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.719 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:28.719 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558844+0000", 2024-05-08T08:53:28.720 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7690999999999998e-05 2024-05-08T08:53:28.720 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.720 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.720 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:28.720 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558959+0000", 2024-05-08T08:53:28.720 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000115217 2024-05-08T08:53:28.720 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:28.720 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:28.720 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:28.720 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.720 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.720 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1103 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11599872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:28.720 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:53:28.720 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 52.154642086000003, 2024-05-08T08:53:28.721 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070777100999999995, 2024-05-08T08:53:28.721 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:28.721 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:28.721 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:28.721 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:28.721 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:28.721 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:28.721 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1103 2024-05-08T08:53:28.721 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.721 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:28.721 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.721 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:28.721 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:53:28.722 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:28.722 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.722 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.722 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:28.722 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:53:28.722 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:28.722 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.722 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.722 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:28.722 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488959+0000", 2024-05-08T08:53:28.722 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00050457200000000001 2024-05-08T08:53:28.722 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.723 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.723 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:28.723 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488969+0000", 2024-05-08T08:53:28.723 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.7159999999999998e-06 2024-05-08T08:53:28.723 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.723 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.723 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:28.723 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488981+0000", 2024-05-08T08:53:28.723 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2150000000000001e-05 2024-05-08T08:53:28.723 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.723 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.723 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:28.723 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488988+0000", 2024-05-08T08:53:28.724 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8759999999999999e-06 2024-05-08T08:53:28.724 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.724 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.724 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:28.724 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558992+0000", 2024-05-08T08:53:28.724 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070004358000000003 2024-05-08T08:53:28.724 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.724 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.724 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:28.724 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559093+0000", 2024-05-08T08:53:28.724 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000100299 2024-05-08T08:53:28.724 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.724 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.725 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:28.725 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559232+0000", 2024-05-08T08:53:28.725 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013913 2024-05-08T08:53:28.725 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:28.725 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:28.725 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:28.725 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.725 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.725 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1104 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11640832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:28.725 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:53:28.725 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 52.154097270999998, 2024-05-08T08:53:28.725 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070521947000000001, 2024-05-08T08:53:28.726 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:28.726 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:28.726 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:28.726 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:28.726 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:28.726 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:28.726 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1104 2024-05-08T08:53:28.726 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.726 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:28.726 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.726 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:28.726 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:53:28.726 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:28.727 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.727 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.727 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:28.727 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:53:28.727 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:28.727 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.727 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.727 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:28.727 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489700+0000", 2024-05-08T08:53:28.727 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00070067600000000001 2024-05-08T08:53:28.727 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.727 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.727 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:28.728 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489704+0000", 2024-05-08T08:53:28.728 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1640000000000003e-06 2024-05-08T08:53:28.728 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.728 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.728 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:28.728 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489716+0000", 2024-05-08T08:53:28.728 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1379000000000001e-05 2024-05-08T08:53:28.728 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.728 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.728 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:28.728 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489724+0000", 2024-05-08T08:53:28.728 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8949999999999998e-06 2024-05-08T08:53:28.728 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.729 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.729 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:28.729 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559278+0000", 2024-05-08T08:53:28.729 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.069554151999999994 2024-05-08T08:53:28.729 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.729 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.729 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:28.729 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559387+0000", 2024-05-08T08:53:28.729 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000109258 2024-05-08T08:53:28.729 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.729 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.729 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:28.729 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559521+0000", 2024-05-08T08:53:28.730 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000134423 2024-05-08T08:53:28.730 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:28.730 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:28.730 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:28.730 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.730 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.730 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:8636 3.17 3:e8a0a3e4:::rbd_header.11402b26385:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-05-08T08:53:28.730 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:54.150971+0000", 2024-05-08T08:53:28.730 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 34.492125277, 2024-05-08T08:53:28.730 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.217273233, 2024-05-08T08:53:28.730 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:28.730 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:28.731 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:28.731 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:28.731 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:28.731 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:28.731 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 8636 2024-05-08T08:53:28.731 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.731 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:28.731 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.731 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:28.731 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150971+0000", 2024-05-08T08:53:28.731 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:28.731 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.732 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.732 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:28.732 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150971+0000", 2024-05-08T08:53:28.732 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:28.732 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.732 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.732 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:28.732 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150972+0000", 2024-05-08T08:53:28.732 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.71e-07 2024-05-08T08:53:28.732 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.732 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.732 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:28.733 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150977+0000", 2024-05-08T08:53:28.733 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.152e-06 2024-05-08T08:53:28.733 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.733 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.733 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:28.733 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150990+0000", 2024-05-08T08:53:28.733 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2364999999999999e-05 2024-05-08T08:53:28.733 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.733 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.733 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:28.733 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150998+0000", 2024-05-08T08:53:28.733 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0739999999999993e-06 2024-05-08T08:53:28.734 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.734 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.734 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:28.734 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.151054+0000", 2024-05-08T08:53:28.734 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5949000000000003e-05 2024-05-08T08:53:28.734 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.734 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.734 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:53:28.734 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.151060+0000", 2024-05-08T08:53:28.734 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6120000000000004e-06 2024-05-08T08:53:28.734 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.734 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.735 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:28.735 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.367640+0000", 2024-05-08T08:53:28.735 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2165792739999999 2024-05-08T08:53:28.735 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.735 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.735 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:28.735 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.367839+0000", 2024-05-08T08:53:28.735 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000199185 2024-05-08T08:53:28.735 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.735 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.735 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:28.735 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.368245+0000", 2024-05-08T08:53:28.735 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00040575099999999998 2024-05-08T08:53:28.736 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:28.736 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:28.736 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:28.736 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.736 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.736 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5302.0:1516 3.c 3:319bf789:::rbd_data.1140d3fc54c.0000000000000025:head [stat,write 1892352~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:53:28.736 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:27.822220+0000", 2024-05-08T08:53:28.736 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.82087633999999998, 2024-05-08T08:53:28.736 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.092643744, 2024-05-08T08:53:28.736 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:28.736 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:28.736 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:53:28.737 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:28.737 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5302", 2024-05-08T08:53:28.737 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/359277859", 2024-05-08T08:53:28.737 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1516 2024-05-08T08:53:28.737 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.737 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:28.737 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.737 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:28.737 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822220+0000", 2024-05-08T08:53:28.737 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:28.737 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.737 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.737 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:28.737 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822220+0000", 2024-05-08T08:53:28.738 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:28.738 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.738 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.738 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:28.738 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822221+0000", 2024-05-08T08:53:28.738 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3500000000000002e-07 2024-05-08T08:53:28.738 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.738 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.738 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:28.738 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822228+0000", 2024-05-08T08:53:28.738 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9959999999999996e-06 2024-05-08T08:53:28.738 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.739 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.739 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:28.739 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822234+0000", 2024-05-08T08:53:28.739 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5429999999999999e-06 2024-05-08T08:53:28.739 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.739 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.739 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:28.739 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822238+0000", 2024-05-08T08:53:28.739 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.732e-06 2024-05-08T08:53:28.739 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.739 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.739 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:28.739 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822285+0000", 2024-05-08T08:53:28.740 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6810000000000001e-05 2024-05-08T08:53:28.740 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.740 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.740 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:28.740 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822358+0000", 2024-05-08T08:53:28.740 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3473000000000006e-05 2024-05-08T08:53:28.740 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.740 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.740 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 5", 2024-05-08T08:53:28.740 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822489+0000", 2024-05-08T08:53:28.740 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013011199999999999 2024-05-08T08:53:28.740 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.741 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.741 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:53:28.741 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.825332+0000", 2024-05-08T08:53:28.741 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0028434799999999998 2024-05-08T08:53:28.741 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.741 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.741 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:53:28.741 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.914753+0000", 2024-05-08T08:53:28.741 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089421424999999999 2024-05-08T08:53:28.741 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.741 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.741 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:53:28.741 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.914807+0000", 2024-05-08T08:53:28.741 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3829000000000002e-05 2024-05-08T08:53:28.742 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:28.742 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:28.742 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:28.742 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.914864+0000", 2024-05-08T08:53:28.742 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6909e-05 2024-05-08T08:53:28.742 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:28.742 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:28.742 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:28.742 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:28.742 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:28.742 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:28.743 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:28.644+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:28.744 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:53:28.744 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:53:28.745 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:53:28.745 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:53:28.745 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:53:28.745 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:53:28.746 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:53:28.747 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:28.750+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:28.754 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:53:28.754 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:53:28.774 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:53:28.774 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:53:28.793 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:53:28.793 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:53:28.848 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:28.851+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:28.870 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:28.870 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:28.879 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:53:28.948 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:28.952+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:28.996 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:28.996 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:29.008 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:53:29.049 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:29.052+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:29.119 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:29.119 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:29.128 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:53:29.173 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:29.176+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:29.237 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:29.238 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:29.246 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:53:29.288 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:29.291+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:29.294 INFO:teuthology.orchestra.run.smithi196.stdout:checking clone #20, image image_client.0-clone20 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent21 2024-05-08T08:53:29.352 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:29.355+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:29.433 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:29.433 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:29.444 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:53:29.486 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:53:29.489+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:53:29.513 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-05-08T08:53:29.573 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:29.576+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:29.655 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:29.658+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:29.655 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:29.655 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:29.670 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:53:29.756 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:29.759+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:29.804 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:29.804 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:29.815 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:53:29.856 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:29.859+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:29.958 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:53:29.960+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:53:29.972 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:29.972 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:29.982 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:53:30.059 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:30.061+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:30.097 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:30.097 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:30.106 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:53:30.159 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:30.162+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:30.214 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:30.214 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:30.223 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:53:30.259 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:30.262+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:30.262 INFO:teuthology.orchestra.run.smithi196.stdout:3233 write 0x1a85e00 thru 0x1a8bdfa (0x5ffb bytes) 2024-05-08T08:53:30.264 INFO:teuthology.orchestra.run.smithi196.stdout:3236 read 0xb65d800 thru 0xb66ac98 (0xd499 bytes) 2024-05-08T08:53:30.265 INFO:teuthology.orchestra.run.smithi196.stdout:3237 clone 23 order 22 su 524288 sc 12 2024-05-08T08:53:30.361 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:53:30.364+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:53:30.369 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:30.369 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:30.378 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:53:30.462 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:30.465+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:30.483 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:30.483 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:30.492 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:53:30.518 INFO:teuthology.orchestra.run.smithi059.stderr:noscrub is unset 2024-05-08T08:53:30.562 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:30.565+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:30.599 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:30.600 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:30.608 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:53:30.610 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 5, 4] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-05-08T08:53:30.610 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T08:53:30.610 INFO:tasks.thrashosds.thrasher:inject_pause on osd.1 2024-05-08T08:53:30.610 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-05-08T08:53:30.610 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-05-08T08:53:30.611 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok config set filestore_inject_stall 3 2024-05-08T08:53:30.693 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:30.696+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:30.728 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:30.729 INFO:teuthology.orchestra.run.smithi059.stdout: "success": "" 2024-05-08T08:53:30.729 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:30.755 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:30.755 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:30.765 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:53:30.765 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:53:30.767+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:53:30.805 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T08:53:30.827 INFO:teuthology.orchestra.run.smithi196.stdout:truncating image image_client.0-clone22 from 0xe512a00 (overlap 0xe512a00) to 0x0 2024-05-08T08:53:30.865 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:30.868+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:30.950 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:30.950 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:30.961 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:53:30.966 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:30.968+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:30.975 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:53:30.974+0000 7fb618ffd640 -1 reset not still connected to 0x5636d727cb60 2024-05-08T08:53:31.027 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:53:31.027 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:53:31.067 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:31.070+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:31.069 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:53:31.070 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:53:31.104 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:53:31.104 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:53:31.128 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:53:31.128 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:53:31.135 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:31.135 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:53:31.135 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:53:31.135 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:31.144 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:53:31.146 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:53:31.146 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:53:31.164 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:53:31.164 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:53:31.167 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:31.170+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:31.268 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:31.272+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:31.292 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:31.292 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:53:31.292 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:53:31.293 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:53:31.293 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:31.306 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:53:31.369 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:31.372+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:31.470 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:31.473+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:31.491 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:31.491 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:53:31.491 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:53:31.491 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:53:31.491 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.491 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4621.0:9089 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 23339520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:53:31.492 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:53:31.492 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 58.997401889999999, 2024-05-08T08:53:31.492 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.13814907200000001, 2024-05-08T08:53:31.492 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:31.492 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:31.492 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:53:31.492 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:31.492 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4621", 2024-05-08T08:53:31.492 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:31.492 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9089 2024-05-08T08:53:31.492 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.492 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:31.492 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.492 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:31.493 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:53:31.493 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.493 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.493 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.493 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:31.493 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:53:31.493 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.493 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.493 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.493 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:31.493 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493005+0000", 2024-05-08T08:53:31.493 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1700000000000001e-06 2024-05-08T08:53:31.493 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.493 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.493 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:31.494 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.513963+0000", 2024-05-08T08:53:31.494 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.020957581999999999 2024-05-08T08:53:31.494 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.494 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.494 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:31.494 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.513995+0000", 2024-05-08T08:53:31.494 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1993999999999999e-05 2024-05-08T08:53:31.494 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.494 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.494 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:31.494 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514007+0000", 2024-05-08T08:53:31.494 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1581e-05 2024-05-08T08:53:31.494 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.494 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.495 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:31.495 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514122+0000", 2024-05-08T08:53:31.495 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000115282 2024-05-08T08:53:31.495 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.495 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.495 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op must wait for map", 2024-05-08T08:53:31.495 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514128+0000", 2024-05-08T08:53:31.495 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9950000000000002e-06 2024-05-08T08:53:31.495 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.495 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.495 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:31.495 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.515800+0000", 2024-05-08T08:53:31.495 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001671886 2024-05-08T08:53:31.495 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.495 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.495 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:31.496 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.515893+0000", 2024-05-08T08:53:31.496 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3490000000000001e-05 2024-05-08T08:53:31.496 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.496 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.496 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 3", 2024-05-08T08:53:31.496 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.517647+0000", 2024-05-08T08:53:31.496 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0017535630000000001 2024-05-08T08:53:31.496 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.496 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.496 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:53:31.496 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.627578+0000", 2024-05-08T08:53:31.496 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.109931557 2024-05-08T08:53:31.496 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.496 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.496 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:53:31.497 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631055+0000", 2024-05-08T08:53:31.497 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0034761420000000002 2024-05-08T08:53:31.497 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.497 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.497 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:53:31.497 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631096+0000", 2024-05-08T08:53:31.497 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1059000000000001e-05 2024-05-08T08:53:31.497 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.497 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.497 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:31.497 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631151+0000", 2024-05-08T08:53:31.497 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5770999999999998e-05 2024-05-08T08:53:31.497 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.497 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.497 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.498 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.498 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.498 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1067 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 10985472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:31.498 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485431+0000", 2024-05-08T08:53:31.498 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 55.004973069000002, 2024-05-08T08:53:31.498 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070412460999999996, 2024-05-08T08:53:31.498 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:31.498 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:31.498 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:31.498 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:31.498 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:31.498 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:31.498 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1067 2024-05-08T08:53:31.498 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.499 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:31.499 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.499 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:31.499 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485431+0000", 2024-05-08T08:53:31.499 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.499 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.499 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.499 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:31.499 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485431+0000", 2024-05-08T08:53:31.499 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.499 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.499 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.499 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:31.499 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485432+0000", 2024-05-08T08:53:31.499 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8299999999999998e-07 2024-05-08T08:53:31.500 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.500 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.500 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:31.500 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485434+0000", 2024-05-08T08:53:31.500 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0849999999999999e-06 2024-05-08T08:53:31.500 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.500 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.500 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:31.500 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485440+0000", 2024-05-08T08:53:31.500 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0020000000000003e-06 2024-05-08T08:53:31.500 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.500 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.500 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:31.500 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485445+0000", 2024-05-08T08:53:31.500 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5180000000000004e-06 2024-05-08T08:53:31.500 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.501 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.501 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:31.501 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555731+0000", 2024-05-08T08:53:31.501 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070286214 2024-05-08T08:53:31.501 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.501 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.501 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:31.501 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555787+0000", 2024-05-08T08:53:31.501 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5992999999999997e-05 2024-05-08T08:53:31.501 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.501 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.501 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:31.501 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555844+0000", 2024-05-08T08:53:31.501 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6266e-05 2024-05-08T08:53:31.501 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.501 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.502 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.502 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.502 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.502 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1071 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11026432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:31.502 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:53:31.502 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 55.004952398999997, 2024-05-08T08:53:31.502 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070580037999999998, 2024-05-08T08:53:31.502 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:31.502 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:31.502 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:31.502 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:31.502 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:31.502 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:31.502 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1071 2024-05-08T08:53:31.502 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.503 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:31.503 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.503 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:31.503 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:53:31.503 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.503 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.503 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.503 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:31.503 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:53:31.503 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.503 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.503 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.503 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:31.503 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:53:31.503 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2800000000000002e-07 2024-05-08T08:53:31.503 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.504 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.504 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:31.504 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485454+0000", 2024-05-08T08:53:31.504 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2290000000000002e-06 2024-05-08T08:53:31.504 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.504 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.504 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:31.504 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485461+0000", 2024-05-08T08:53:31.504 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9920000000000001e-06 2024-05-08T08:53:31.504 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.504 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.504 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:31.504 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485467+0000", 2024-05-08T08:53:31.504 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3979999999999998e-06 2024-05-08T08:53:31.505 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.505 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.505 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:31.505 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555880+0000", 2024-05-08T08:53:31.505 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070413165 2024-05-08T08:53:31.505 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.505 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.505 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:31.505 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555926+0000", 2024-05-08T08:53:31.505 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6217000000000002e-05 2024-05-08T08:53:31.505 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.505 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.505 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:31.505 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556032+0000", 2024-05-08T08:53:31.505 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000105609 2024-05-08T08:53:31.506 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.506 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.506 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.506 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.506 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.506 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1073 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11067392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:31.506 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:53:31.506 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 55.004931178, 2024-05-08T08:53:31.506 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070821404000000004, 2024-05-08T08:53:31.506 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:31.506 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:31.506 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:31.506 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:31.506 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:31.506 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:31.507 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1073 2024-05-08T08:53:31.507 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.507 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:31.507 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.507 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:31.507 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:53:31.507 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.507 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.507 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.507 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:31.507 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:53:31.507 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.507 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.507 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.507 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:31.508 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:53:31.508 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.08e-07 2024-05-08T08:53:31.508 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.508 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.508 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:31.508 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485476+0000", 2024-05-08T08:53:31.508 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3929999999999998e-06 2024-05-08T08:53:31.508 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.508 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.508 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:31.508 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485485+0000", 2024-05-08T08:53:31.508 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1880000000000008e-06 2024-05-08T08:53:31.508 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.508 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.508 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:31.509 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485494+0000", 2024-05-08T08:53:31.509 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3109999999999995e-06 2024-05-08T08:53:31.509 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.509 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.509 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:31.509 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556082+0000", 2024-05-08T08:53:31.509 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070587426999999994 2024-05-08T08:53:31.509 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.509 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.509 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:31.509 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556177+0000", 2024-05-08T08:53:31.509 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5051999999999997e-05 2024-05-08T08:53:31.509 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.509 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.509 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:31.510 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556294+0000", 2024-05-08T08:53:31.510 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000117625 2024-05-08T08:53:31.510 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.510 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.510 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.510 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.510 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.510 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1075 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11108352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:31.510 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:53:31.510 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 55.004900601000003, 2024-05-08T08:53:31.510 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071045026999999997, 2024-05-08T08:53:31.510 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:31.510 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:31.510 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:31.510 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:31.511 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:31.511 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:31.511 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1075 2024-05-08T08:53:31.511 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.511 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:31.511 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.511 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:31.511 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:53:31.511 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.511 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.511 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.511 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:31.511 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:53:31.511 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.512 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.512 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.512 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:31.512 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:53:31.512 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.32e-07 2024-05-08T08:53:31.512 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.512 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.512 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:31.512 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485518+0000", 2024-05-08T08:53:31.512 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3769e-05 2024-05-08T08:53:31.512 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.512 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.512 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:31.512 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485527+0000", 2024-05-08T08:53:31.512 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6779999999999995e-06 2024-05-08T08:53:31.513 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.513 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.513 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:31.513 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485532+0000", 2024-05-08T08:53:31.513 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9830000000000004e-06 2024-05-08T08:53:31.513 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.513 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.513 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:31.513 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556340+0000", 2024-05-08T08:53:31.513 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070807743000000006 2024-05-08T08:53:31.513 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.513 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.513 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:31.513 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556435+0000", 2024-05-08T08:53:31.513 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4732e-05 2024-05-08T08:53:31.514 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.514 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.514 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:31.514 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556549+0000", 2024-05-08T08:53:31.514 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011369 2024-05-08T08:53:31.514 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.514 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.514 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.514 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.514 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.514 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1078 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11149312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:31.514 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:53:31.514 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 55.004860401999998, 2024-05-08T08:53:31.514 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071167276000000002, 2024-05-08T08:53:31.514 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:31.515 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:31.515 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:31.515 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:31.515 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:31.515 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:31.515 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1078 2024-05-08T08:53:31.515 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.515 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:31.515 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.515 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:31.515 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:53:31.515 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.515 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.515 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.515 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:31.515 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:53:31.515 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.516 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.516 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.516 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:31.516 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:53:31.516 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5600000000000001e-07 2024-05-08T08:53:31.516 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.516 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.516 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:31.516 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485547+0000", 2024-05-08T08:53:31.516 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3590000000000002e-06 2024-05-08T08:53:31.516 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.516 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.516 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:31.516 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485553+0000", 2024-05-08T08:53:31.516 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2770000000000002e-06 2024-05-08T08:53:31.516 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.516 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.517 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:31.517 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485558+0000", 2024-05-08T08:53:31.517 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.95e-06 2024-05-08T08:53:31.517 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.517 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.517 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:31.517 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556582+0000", 2024-05-08T08:53:31.517 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071024444000000006 2024-05-08T08:53:31.517 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.517 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.517 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:31.517 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556637+0000", 2024-05-08T08:53:31.517 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4672000000000001e-05 2024-05-08T08:53:31.517 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.517 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.518 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:31.518 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556711+0000", 2024-05-08T08:53:31.518 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4117999999999997e-05 2024-05-08T08:53:31.518 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.518 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.518 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.518 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.518 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.518 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1080 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11190272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:31.518 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:53:31.518 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 55.004839668999999, 2024-05-08T08:53:31.518 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071350721000000006, 2024-05-08T08:53:31.518 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:31.518 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:31.518 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:31.519 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:31.519 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:31.519 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:31.519 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1080 2024-05-08T08:53:31.519 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.519 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:31.519 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.519 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:31.519 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:53:31.519 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.519 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.519 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.519 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:31.519 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:53:31.520 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.520 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.520 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.520 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:31.520 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:53:31.520 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.03e-07 2024-05-08T08:53:31.520 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.520 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.520 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:31.520 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485567+0000", 2024-05-08T08:53:31.520 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1090000000000001e-06 2024-05-08T08:53:31.520 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.520 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.520 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:31.520 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485573+0000", 2024-05-08T08:53:31.521 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3049999999999998e-06 2024-05-08T08:53:31.521 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.521 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.521 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:31.521 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485578+0000", 2024-05-08T08:53:31.521 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0130000000000001e-06 2024-05-08T08:53:31.521 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.521 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.521 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:31.521 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556743+0000", 2024-05-08T08:53:31.521 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071164449000000005 2024-05-08T08:53:31.521 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.521 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.521 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:31.521 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556797+0000", 2024-05-08T08:53:31.522 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4589999999999997e-05 2024-05-08T08:53:31.522 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.522 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.522 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:31.522 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556915+0000", 2024-05-08T08:53:31.522 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000117852 2024-05-08T08:53:31.522 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.522 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.522 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.522 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.522 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.522 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1082 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11231232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:31.522 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:53:31.522 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 55.004819425999997, 2024-05-08T08:53:31.523 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071591396000000002, 2024-05-08T08:53:31.523 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:31.523 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:31.523 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:31.523 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:31.523 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:31.523 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:31.523 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1082 2024-05-08T08:53:31.523 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.523 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:31.523 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.523 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:31.523 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:53:31.523 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.523 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.524 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.524 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:31.524 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:53:31.524 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.524 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.524 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.524 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:31.524 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:53:31.524 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1800000000000001e-07 2024-05-08T08:53:31.524 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.524 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.524 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:31.524 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485587+0000", 2024-05-08T08:53:31.524 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2589999999999999e-06 2024-05-08T08:53:31.524 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.525 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.525 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:31.525 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485594+0000", 2024-05-08T08:53:31.525 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0789999999999997e-06 2024-05-08T08:53:31.525 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.525 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.525 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:31.525 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485598+0000", 2024-05-08T08:53:31.525 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7480000000000001e-06 2024-05-08T08:53:31.525 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.525 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.525 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:31.525 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556971+0000", 2024-05-08T08:53:31.525 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071372774999999999 2024-05-08T08:53:31.526 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.526 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.526 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:31.526 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557069+0000", 2024-05-08T08:53:31.526 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.7732999999999994e-05 2024-05-08T08:53:31.526 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.526 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.526 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:31.526 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557176+0000", 2024-05-08T08:53:31.526 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000107384 2024-05-08T08:53:31.526 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.526 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.526 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.526 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.526 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.527 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1084 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11272192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:31.527 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:53:31.527 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 55.004780787000001, 2024-05-08T08:53:31.527 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071808977999999996, 2024-05-08T08:53:31.527 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:31.527 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:31.527 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:31.527 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:31.527 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:31.527 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:31.527 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1084 2024-05-08T08:53:31.527 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.527 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:31.527 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.527 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:31.528 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:53:31.528 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.528 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.528 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.528 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:31.528 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:53:31.528 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.528 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.528 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.528 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:31.528 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485624+0000", 2024-05-08T08:53:31.528 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:53:31.528 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.528 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.528 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:31.528 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485626+0000", 2024-05-08T08:53:31.529 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9460000000000001e-06 2024-05-08T08:53:31.529 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.529 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.529 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:31.529 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485633+0000", 2024-05-08T08:53:31.529 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1149999999999996e-06 2024-05-08T08:53:31.529 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.529 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.529 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:31.529 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485638+0000", 2024-05-08T08:53:31.529 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5670000000000002e-06 2024-05-08T08:53:31.529 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.529 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.529 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:31.529 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557226+0000", 2024-05-08T08:53:31.530 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071587267999999996 2024-05-08T08:53:31.530 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.530 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.530 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:31.530 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557321+0000", 2024-05-08T08:53:31.530 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5785000000000005e-05 2024-05-08T08:53:31.530 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.530 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.530 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:31.530 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557432+0000", 2024-05-08T08:53:31.530 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011092800000000001 2024-05-08T08:53:31.530 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.530 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.530 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.531 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.531 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.531 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1086 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11313152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:31.531 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:53:31.531 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 55.00475746, 2024-05-08T08:53:31.531 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071933132999999996, 2024-05-08T08:53:31.531 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:31.531 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:31.531 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:31.531 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:31.531 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:31.531 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:31.531 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1086 2024-05-08T08:53:31.531 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.531 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:31.532 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.532 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:31.532 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:53:31.532 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.532 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.532 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.532 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:31.532 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:53:31.532 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.532 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.532 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.532 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:31.532 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486328+0000", 2024-05-08T08:53:31.532 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00068107900000000002 2024-05-08T08:53:31.532 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.533 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.533 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:31.533 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486331+0000", 2024-05-08T08:53:31.533 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.585e-06 2024-05-08T08:53:31.533 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.533 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.533 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:31.533 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486343+0000", 2024-05-08T08:53:31.533 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1856e-05 2024-05-08T08:53:31.533 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.533 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.533 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:31.533 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486350+0000", 2024-05-08T08:53:31.533 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8600000000000004e-06 2024-05-08T08:53:31.533 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.534 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.534 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:31.534 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557470+0000", 2024-05-08T08:53:31.534 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071120326999999997 2024-05-08T08:53:31.534 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.534 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.534 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:31.534 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557514+0000", 2024-05-08T08:53:31.534 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3909000000000003e-05 2024-05-08T08:53:31.534 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.534 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.534 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:31.534 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557580+0000", 2024-05-08T08:53:31.535 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5517000000000003e-05 2024-05-08T08:53:31.535 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.535 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.535 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.535 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.535 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.535 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1089 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11354112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:31.535 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:53:31.535 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 55.004044587999999, 2024-05-08T08:53:31.535 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071457381, 2024-05-08T08:53:31.535 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:31.535 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:31.535 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:31.535 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:31.536 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:31.536 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:31.536 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1089 2024-05-08T08:53:31.536 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.536 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:31.536 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.536 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:31.536 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:53:31.536 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.536 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.536 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.536 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:31.536 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:53:31.537 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.537 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.537 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.537 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:31.537 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486851+0000", 2024-05-08T08:53:31.537 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00049102 2024-05-08T08:53:31.537 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.537 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.537 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:31.537 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486854+0000", 2024-05-08T08:53:31.537 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6859999999999999e-06 2024-05-08T08:53:31.537 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.537 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.537 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:31.537 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486864+0000", 2024-05-08T08:53:31.538 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6080000000000002e-06 2024-05-08T08:53:31.538 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.538 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.538 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:31.538 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486870+0000", 2024-05-08T08:53:31.538 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.477e-06 2024-05-08T08:53:31.538 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.538 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.538 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:31.538 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557612+0000", 2024-05-08T08:53:31.538 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070741557999999996 2024-05-08T08:53:31.538 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.538 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.538 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:31.539 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557695+0000", 2024-05-08T08:53:31.539 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3300000000000005e-05 2024-05-08T08:53:31.539 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.539 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.539 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:31.539 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557817+0000", 2024-05-08T08:53:31.539 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012173200000000001 2024-05-08T08:53:31.539 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.539 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.539 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.539 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.539 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.539 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1091 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11395072~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:31.539 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:53:31.539 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 55.003523792999999, 2024-05-08T08:53:31.539 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071184737999999997, 2024-05-08T08:53:31.540 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:31.540 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:31.540 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:31.540 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:31.540 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:31.540 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:31.540 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1091 2024-05-08T08:53:31.540 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.540 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:31.540 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.540 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:31.540 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:53:31.540 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.540 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.540 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.540 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:31.541 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:53:31.541 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.541 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.541 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.541 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:31.541 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486881+0000", 2024-05-08T08:53:31.541 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0699999999999997e-07 2024-05-08T08:53:31.541 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.541 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.541 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:31.541 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486883+0000", 2024-05-08T08:53:31.541 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.402e-06 2024-05-08T08:53:31.541 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.541 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.541 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:31.541 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486890+0000", 2024-05-08T08:53:31.541 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0480000000000001e-06 2024-05-08T08:53:31.542 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.542 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.542 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:31.542 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486896+0000", 2024-05-08T08:53:31.542 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7640000000000002e-06 2024-05-08T08:53:31.542 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.542 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.542 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:31.542 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557860+0000", 2024-05-08T08:53:31.542 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070963791999999998 2024-05-08T08:53:31.542 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.542 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.542 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:31.542 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557952+0000", 2024-05-08T08:53:31.542 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2082000000000004e-05 2024-05-08T08:53:31.543 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.543 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.543 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:31.543 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558065+0000", 2024-05-08T08:53:31.543 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000113143 2024-05-08T08:53:31.543 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.543 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.543 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.543 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.543 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.543 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1094 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11436032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:31.543 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:53:31.543 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 55.003500625000001, 2024-05-08T08:53:31.543 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071400507000000002, 2024-05-08T08:53:31.543 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:31.544 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:31.544 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:31.544 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:31.544 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:31.544 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:31.544 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1094 2024-05-08T08:53:31.544 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.544 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:31.544 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.544 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:31.544 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:53:31.544 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.544 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.544 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.544 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:31.545 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:53:31.545 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.545 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.545 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.545 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:31.545 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487940+0000", 2024-05-08T08:53:31.545 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001036868 2024-05-08T08:53:31.545 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.545 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.545 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:31.545 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487944+0000", 2024-05-08T08:53:31.545 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7069999999999998e-06 2024-05-08T08:53:31.545 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.545 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.545 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:31.545 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487951+0000", 2024-05-08T08:53:31.546 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2649999999999999e-06 2024-05-08T08:53:31.546 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.546 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.546 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:31.546 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487961+0000", 2024-05-08T08:53:31.546 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4110000000000002e-06 2024-05-08T08:53:31.546 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.546 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.546 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:31.546 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558117+0000", 2024-05-08T08:53:31.546 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070156662999999994 2024-05-08T08:53:31.546 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.546 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.546 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:31.546 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558214+0000", 2024-05-08T08:53:31.547 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6444e-05 2024-05-08T08:53:31.547 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.547 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.547 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:31.547 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558304+0000", 2024-05-08T08:53:31.547 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0148999999999994e-05 2024-05-08T08:53:31.547 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.547 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.547 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.547 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.547 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.547 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1096 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11476992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:31.547 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:53:31.547 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 55.002434094000002, 2024-05-08T08:53:31.547 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070480074000000004, 2024-05-08T08:53:31.547 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:31.548 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:31.548 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:31.548 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:31.548 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:31.548 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:31.548 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1096 2024-05-08T08:53:31.548 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.548 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:31.548 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.548 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:31.548 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:53:31.548 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.548 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.548 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.548 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:31.548 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:53:31.549 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.549 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.549 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.549 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:31.549 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:53:31.549 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3700000000000001e-07 2024-05-08T08:53:31.549 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.549 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.549 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:31.549 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487973+0000", 2024-05-08T08:53:31.549 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0870000000000001e-06 2024-05-08T08:53:31.549 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.549 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.549 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:31.549 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487977+0000", 2024-05-08T08:53:31.549 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0600000000000001e-06 2024-05-08T08:53:31.550 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.550 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.550 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:31.550 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487980+0000", 2024-05-08T08:53:31.550 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1089999999999998e-06 2024-05-08T08:53:31.550 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.550 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.550 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:31.550 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558338+0000", 2024-05-08T08:53:31.550 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070358233000000006 2024-05-08T08:53:31.550 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.550 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.550 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:31.550 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558392+0000", 2024-05-08T08:53:31.550 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4537999999999999e-05 2024-05-08T08:53:31.551 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.551 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.551 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:31.551 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558450+0000", 2024-05-08T08:53:31.551 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7710000000000001e-05 2024-05-08T08:53:31.551 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.551 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.551 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.551 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.551 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.551 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1099 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11517952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:31.551 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:53:31.551 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 55.002420217999997, 2024-05-08T08:53:31.551 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070720506000000002, 2024-05-08T08:53:31.551 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:31.552 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:31.552 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:31.552 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:31.552 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:31.552 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:31.552 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1099 2024-05-08T08:53:31.552 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.552 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:31.552 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.552 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:31.552 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:53:31.552 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.552 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.552 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.552 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:31.553 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:53:31.553 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.553 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.553 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.553 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:31.553 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:53:31.553 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3999999999999998e-07 2024-05-08T08:53:31.553 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.553 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.553 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:31.553 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487986+0000", 2024-05-08T08:53:31.553 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3519999999999999e-06 2024-05-08T08:53:31.553 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.553 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.553 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:31.553 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487989+0000", 2024-05-08T08:53:31.553 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.23e-06 2024-05-08T08:53:31.554 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.554 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.554 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:31.554 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487991+0000", 2024-05-08T08:53:31.554 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6350000000000002e-06 2024-05-08T08:53:31.554 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.554 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.554 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:31.554 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558489+0000", 2024-05-08T08:53:31.554 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070497924000000003 2024-05-08T08:53:31.554 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.554 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.554 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:31.554 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558583+0000", 2024-05-08T08:53:31.554 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3405000000000004e-05 2024-05-08T08:53:31.555 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.555 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.555 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:31.555 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558704+0000", 2024-05-08T08:53:31.555 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012171999999999999 2024-05-08T08:53:31.555 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.555 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.555 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.555 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.555 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.555 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1100 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11558912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:31.555 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:53:31.555 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 55.002407218999998, 2024-05-08T08:53:31.555 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070962266999999996, 2024-05-08T08:53:31.555 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:31.556 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:31.556 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:31.556 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:31.556 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:31.556 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:31.556 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1100 2024-05-08T08:53:31.556 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.556 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:31.556 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.556 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:31.556 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:53:31.556 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.556 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.556 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.556 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:31.556 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:53:31.557 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.557 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.557 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.557 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:31.557 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488430+0000", 2024-05-08T08:53:31.557 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000432676 2024-05-08T08:53:31.557 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.557 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.557 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:31.557 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488433+0000", 2024-05-08T08:53:31.557 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.45e-06 2024-05-08T08:53:31.557 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.557 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.557 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:31.557 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488442+0000", 2024-05-08T08:53:31.557 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.687e-06 2024-05-08T08:53:31.557 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.558 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.558 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:31.558 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488446+0000", 2024-05-08T08:53:31.558 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4939999999999997e-06 2024-05-08T08:53:31.558 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.558 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.558 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:31.558 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558756+0000", 2024-05-08T08:53:31.558 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070310051999999998 2024-05-08T08:53:31.558 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.558 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.558 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:31.558 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558844+0000", 2024-05-08T08:53:31.558 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7690999999999998e-05 2024-05-08T08:53:31.558 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.558 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.559 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:31.559 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558959+0000", 2024-05-08T08:53:31.559 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000115217 2024-05-08T08:53:31.559 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.559 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.559 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.559 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.559 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.559 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1103 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11599872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:31.559 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:53:31.559 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 55.001949543999999, 2024-05-08T08:53:31.559 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070777100999999995, 2024-05-08T08:53:31.559 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:31.559 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:31.559 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:31.560 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:31.560 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:31.560 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:31.560 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1103 2024-05-08T08:53:31.560 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.560 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:31.560 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.560 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:31.560 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:53:31.560 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.560 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.560 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.560 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:31.560 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:53:31.560 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.560 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.560 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.561 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:31.561 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488959+0000", 2024-05-08T08:53:31.561 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00050457200000000001 2024-05-08T08:53:31.561 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.561 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.561 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:31.561 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488969+0000", 2024-05-08T08:53:31.561 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.7159999999999998e-06 2024-05-08T08:53:31.561 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.561 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.561 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:31.561 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488981+0000", 2024-05-08T08:53:31.561 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2150000000000001e-05 2024-05-08T08:53:31.561 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.561 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.562 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:31.562 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488988+0000", 2024-05-08T08:53:31.562 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8759999999999999e-06 2024-05-08T08:53:31.562 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.562 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.562 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:31.562 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558992+0000", 2024-05-08T08:53:31.562 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070004358000000003 2024-05-08T08:53:31.562 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.562 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.562 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:31.562 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559093+0000", 2024-05-08T08:53:31.562 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000100299 2024-05-08T08:53:31.562 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.562 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.562 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:31.563 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559232+0000", 2024-05-08T08:53:31.563 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013913 2024-05-08T08:53:31.563 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.563 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.563 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.563 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.563 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.563 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1104 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11640832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:31.563 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:53:31.563 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 55.001404729000001, 2024-05-08T08:53:31.563 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070521947000000001, 2024-05-08T08:53:31.563 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:31.563 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:31.563 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:31.563 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:31.563 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:31.563 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:31.564 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1104 2024-05-08T08:53:31.564 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.564 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:31.564 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.564 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:31.564 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:53:31.564 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.564 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.564 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.564 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:31.564 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:53:31.564 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.564 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.564 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.564 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:31.564 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489700+0000", 2024-05-08T08:53:31.565 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00070067600000000001 2024-05-08T08:53:31.565 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.565 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.565 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:31.565 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489704+0000", 2024-05-08T08:53:31.565 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1640000000000003e-06 2024-05-08T08:53:31.565 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.565 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.565 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:31.565 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489716+0000", 2024-05-08T08:53:31.565 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1379000000000001e-05 2024-05-08T08:53:31.565 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.565 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.565 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:31.565 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489724+0000", 2024-05-08T08:53:31.565 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8949999999999998e-06 2024-05-08T08:53:31.566 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.566 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.566 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:31.566 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559278+0000", 2024-05-08T08:53:31.566 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.069554151999999994 2024-05-08T08:53:31.566 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.566 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.566 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:31.566 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559387+0000", 2024-05-08T08:53:31.566 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000109258 2024-05-08T08:53:31.566 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.566 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.566 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:31.566 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559521+0000", 2024-05-08T08:53:31.566 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000134423 2024-05-08T08:53:31.566 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.567 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.567 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.567 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.567 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.567 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:8636 3.17 3:e8a0a3e4:::rbd_header.11402b26385:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-05-08T08:53:31.567 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:54.150971+0000", 2024-05-08T08:53:31.567 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 37.339432735000003, 2024-05-08T08:53:31.567 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.217273233, 2024-05-08T08:53:31.567 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:31.567 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:31.567 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:31.567 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:31.567 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:31.568 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:31.568 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 8636 2024-05-08T08:53:31.568 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.568 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:31.568 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.568 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:31.568 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150971+0000", 2024-05-08T08:53:31.568 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.568 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.568 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.568 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:31.568 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150971+0000", 2024-05-08T08:53:31.568 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.568 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.568 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.569 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:31.569 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150972+0000", 2024-05-08T08:53:31.569 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.71e-07 2024-05-08T08:53:31.569 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.569 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.569 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:31.569 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150977+0000", 2024-05-08T08:53:31.569 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.152e-06 2024-05-08T08:53:31.569 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.569 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.569 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:31.569 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150990+0000", 2024-05-08T08:53:31.569 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2364999999999999e-05 2024-05-08T08:53:31.569 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.569 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.569 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:31.570 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150998+0000", 2024-05-08T08:53:31.570 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0739999999999993e-06 2024-05-08T08:53:31.570 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.570 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.570 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:31.570 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.151054+0000", 2024-05-08T08:53:31.570 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5949000000000003e-05 2024-05-08T08:53:31.570 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.570 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.570 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:53:31.570 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.151060+0000", 2024-05-08T08:53:31.570 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6120000000000004e-06 2024-05-08T08:53:31.570 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.570 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.570 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:31.571 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.367640+0000", 2024-05-08T08:53:31.571 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2165792739999999 2024-05-08T08:53:31.571 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.571 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.571 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:31.571 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.367839+0000", 2024-05-08T08:53:31.571 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000199185 2024-05-08T08:53:31.571 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.571 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.571 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:31.571 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.368245+0000", 2024-05-08T08:53:31.571 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00040575099999999998 2024-05-08T08:53:31.571 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.571 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.571 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.571 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.572 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.572 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5302.0:1516 3.c 3:319bf789:::rbd_data.1140d3fc54c.0000000000000025:head [stat,write 1892352~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:53:31.572 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:27.822220+0000", 2024-05-08T08:53:31.572 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 3.6681837980000003, 2024-05-08T08:53:31.572 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.092643744, 2024-05-08T08:53:31.572 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:31.572 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:31.572 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:53:31.572 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:31.572 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5302", 2024-05-08T08:53:31.572 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/359277859", 2024-05-08T08:53:31.572 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1516 2024-05-08T08:53:31.572 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.572 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:31.572 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.573 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:31.573 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822220+0000", 2024-05-08T08:53:31.573 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.573 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.573 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.573 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:31.573 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822220+0000", 2024-05-08T08:53:31.573 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.573 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.573 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.573 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:31.573 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822221+0000", 2024-05-08T08:53:31.573 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3500000000000002e-07 2024-05-08T08:53:31.573 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.574 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.574 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:31.574 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822228+0000", 2024-05-08T08:53:31.574 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9959999999999996e-06 2024-05-08T08:53:31.574 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.574 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.574 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:31.574 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822234+0000", 2024-05-08T08:53:31.574 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5429999999999999e-06 2024-05-08T08:53:31.574 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.574 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.574 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:31.574 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822238+0000", 2024-05-08T08:53:31.574 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.732e-06 2024-05-08T08:53:31.575 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.575 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.575 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:31.575 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822285+0000", 2024-05-08T08:53:31.575 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6810000000000001e-05 2024-05-08T08:53:31.575 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.575 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.575 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:31.575 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822358+0000", 2024-05-08T08:53:31.575 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3473000000000006e-05 2024-05-08T08:53:31.575 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.575 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.575 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 5", 2024-05-08T08:53:31.575 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822489+0000", 2024-05-08T08:53:31.575 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013011199999999999 2024-05-08T08:53:31.576 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.576 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.576 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:53:31.576 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.825332+0000", 2024-05-08T08:53:31.576 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0028434799999999998 2024-05-08T08:53:31.576 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.576 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.576 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:53:31.576 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.914753+0000", 2024-05-08T08:53:31.576 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089421424999999999 2024-05-08T08:53:31.576 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.576 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.576 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:53:31.576 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.914807+0000", 2024-05-08T08:53:31.576 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3829000000000002e-05 2024-05-08T08:53:31.576 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.576 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.577 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:31.577 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.914864+0000", 2024-05-08T08:53:31.577 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6909e-05 2024-05-08T08:53:31.577 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.577 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.577 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.577 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.577 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.577 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:31.579 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:53:31.579 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:31.581+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:31.679 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:31.682+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:31.780 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:31.783+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:31.812 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:31.812 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:53:31.812 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.812 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:17758 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:31.812 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.774423+0000", 2024-05-08T08:53:31.812 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.031641158000000003, 2024-05-08T08:53:31.813 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.031663577999999998, 2024-05-08T08:53:31.813 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:31.813 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:31.813 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:31.813 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:31.813 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:31.813 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:31.813 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 17758 2024-05-08T08:53:31.813 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.813 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:31.813 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.813 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:31.813 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.774423+0000", 2024-05-08T08:53:31.813 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.814 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.814 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.814 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:31.814 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.774423+0000", 2024-05-08T08:53:31.814 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.814 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.814 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.814 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:31.814 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.774423+0000", 2024-05-08T08:53:31.814 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5900000000000003e-07 2024-05-08T08:53:31.814 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.814 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.814 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:31.814 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.774426+0000", 2024-05-08T08:53:31.815 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8839999999999998e-06 2024-05-08T08:53:31.815 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.815 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.815 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:31.815 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.774433+0000", 2024-05-08T08:53:31.815 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4500000000000001e-06 2024-05-08T08:53:31.815 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.815 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.815 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:31.815 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.774438+0000", 2024-05-08T08:53:31.815 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3430000000000003e-06 2024-05-08T08:53:31.815 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.815 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.815 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.816 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.816 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.816 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:14819 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:31.816 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.778737+0000", 2024-05-08T08:53:31.816 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.027326680999999999, 2024-05-08T08:53:31.816 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.027419579999999999, 2024-05-08T08:53:31.816 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:31.816 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:31.816 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:31.816 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:31.816 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:31.816 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:31.816 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14819 2024-05-08T08:53:31.816 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.816 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:31.816 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.817 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:31.817 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.778737+0000", 2024-05-08T08:53:31.817 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.817 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.817 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.817 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:31.817 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.778737+0000", 2024-05-08T08:53:31.817 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.817 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.817 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.817 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:31.817 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.778738+0000", 2024-05-08T08:53:31.817 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0139999999999999e-06 2024-05-08T08:53:31.817 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.817 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.818 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:31.818 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.778743+0000", 2024-05-08T08:53:31.818 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5199999999999999e-06 2024-05-08T08:53:31.818 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.818 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.818 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:31.818 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.778751+0000", 2024-05-08T08:53:31.818 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.5169999999999999e-06 2024-05-08T08:53:31.818 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.818 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.818 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:31.818 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.778760+0000", 2024-05-08T08:53:31.818 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4309999999999994e-06 2024-05-08T08:53:31.818 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.819 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.819 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.819 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.819 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.819 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:14855 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:31.819 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.787886+0000", 2024-05-08T08:53:31.819 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.018177549000000001, 2024-05-08T08:53:31.819 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.018331784, 2024-05-08T08:53:31.819 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:31.819 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:31.819 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:31.819 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:31.819 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:31.819 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:31.819 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14855 2024-05-08T08:53:31.820 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.820 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:31.820 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.820 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:31.820 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.787886+0000", 2024-05-08T08:53:31.820 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.820 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.820 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.820 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:31.820 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.787886+0000", 2024-05-08T08:53:31.820 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.820 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.820 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.820 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:31.820 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.787887+0000", 2024-05-08T08:53:31.820 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2399999999999998e-07 2024-05-08T08:53:31.821 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.821 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.821 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:31.821 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.787902+0000", 2024-05-08T08:53:31.821 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.4535000000000001e-05 2024-05-08T08:53:31.821 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.821 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.821 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:31.821 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.787910+0000", 2024-05-08T08:53:31.821 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0290000000000005e-06 2024-05-08T08:53:31.821 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.821 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.821 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:31.821 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.787917+0000", 2024-05-08T08:53:31.821 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4410000000000004e-06 2024-05-08T08:53:31.822 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.822 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.822 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.822 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.822 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.822 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:18240 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:31.822 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.790143+0000", 2024-05-08T08:53:31.822 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.015921009, 2024-05-08T08:53:31.822 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.016135712999999999, 2024-05-08T08:53:31.822 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:31.822 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:31.822 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:31.822 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:31.822 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:31.823 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:31.823 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 18240 2024-05-08T08:53:31.823 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.823 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:31.823 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.823 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:31.823 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.790143+0000", 2024-05-08T08:53:31.823 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.823 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.823 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.823 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:31.823 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.790143+0000", 2024-05-08T08:53:31.823 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.823 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.823 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.823 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:31.824 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.790332+0000", 2024-05-08T08:53:31.824 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00018907799999999999 2024-05-08T08:53:31.824 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.824 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.824 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:31.824 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.790340+0000", 2024-05-08T08:53:31.824 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4379999999999995e-06 2024-05-08T08:53:31.824 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.824 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.824 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:31.824 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.790353+0000", 2024-05-08T08:53:31.824 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2429000000000001e-05 2024-05-08T08:53:31.824 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.824 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.824 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:31.824 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.790360+0000", 2024-05-08T08:53:31.825 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1749999999999999e-06 2024-05-08T08:53:31.825 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.825 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.825 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.825 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.825 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.825 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:17759 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:31.825 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.774445+0000", 2024-05-08T08:53:31.825 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.031618486000000001, 2024-05-08T08:53:31.825 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.031892351999999999, 2024-05-08T08:53:31.825 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:31.825 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:31.825 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:31.825 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:31.825 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:31.826 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:31.826 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 17759 2024-05-08T08:53:31.826 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.826 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:31.826 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.826 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:31.826 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.774445+0000", 2024-05-08T08:53:31.826 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.826 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.826 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.826 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:31.826 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.774445+0000", 2024-05-08T08:53:31.826 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.827 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.827 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.827 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:31.827 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.774446+0000", 2024-05-08T08:53:31.827 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.65e-07 2024-05-08T08:53:31.827 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.827 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.827 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:31.827 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.774449+0000", 2024-05-08T08:53:31.827 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.756e-06 2024-05-08T08:53:31.827 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.827 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.827 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:31.827 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.774456+0000", 2024-05-08T08:53:31.827 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2579999999999998e-06 2024-05-08T08:53:31.828 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.828 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.828 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:31.828 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.774461+0000", 2024-05-08T08:53:31.828 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3290000000000001e-06 2024-05-08T08:53:31.828 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.828 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.828 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.828 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.828 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.828 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:17784 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:31.828 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.776523+0000", 2024-05-08T08:53:31.828 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.029540614, 2024-05-08T08:53:31.828 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.029877956000000001, 2024-05-08T08:53:31.828 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:31.829 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:31.829 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:31.829 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:31.829 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:31.829 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:31.829 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 17784 2024-05-08T08:53:31.829 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.829 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:31.829 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.829 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:31.829 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.776523+0000", 2024-05-08T08:53:31.829 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.829 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.829 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.829 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:31.830 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.776523+0000", 2024-05-08T08:53:31.830 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.830 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.830 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.830 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:31.830 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.776524+0000", 2024-05-08T08:53:31.830 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0400000000000002e-07 2024-05-08T08:53:31.830 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.830 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.830 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:31.830 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.776527+0000", 2024-05-08T08:53:31.830 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0259999999999999e-06 2024-05-08T08:53:31.830 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.830 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.831 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:31.831 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.776535+0000", 2024-05-08T08:53:31.831 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9500000000000001e-06 2024-05-08T08:53:31.831 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.831 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.831 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:31.831 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.776540+0000", 2024-05-08T08:53:31.831 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6790000000000002e-06 2024-05-08T08:53:31.831 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.831 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.831 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.831 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.831 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.831 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:14820 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:31.831 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.778999+0000", 2024-05-08T08:53:31.831 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.027064719000000001, 2024-05-08T08:53:31.832 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.027461739999999998, 2024-05-08T08:53:31.832 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:31.832 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:31.832 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:31.832 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:31.832 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:31.832 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:31.832 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14820 2024-05-08T08:53:31.832 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.832 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:31.832 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.832 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:31.832 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.778999+0000", 2024-05-08T08:53:31.832 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.832 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.833 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.833 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:31.833 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.778999+0000", 2024-05-08T08:53:31.833 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.833 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.833 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.833 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:31.833 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.779000+0000", 2024-05-08T08:53:31.833 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4499999999999996e-07 2024-05-08T08:53:31.833 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.833 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.833 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:31.833 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.779005+0000", 2024-05-08T08:53:31.833 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8840000000000002e-06 2024-05-08T08:53:31.833 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.834 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.834 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:31.834 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.779015+0000", 2024-05-08T08:53:31.834 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0349e-05 2024-05-08T08:53:31.834 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.834 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.834 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:31.834 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.779022+0000", 2024-05-08T08:53:31.834 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9330000000000004e-06 2024-05-08T08:53:31.834 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.834 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.834 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.834 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.834 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.834 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:18209 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:31.835 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.787843+0000", 2024-05-08T08:53:31.835 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.018221125000000001, 2024-05-08T08:53:31.835 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.018676656999999999, 2024-05-08T08:53:31.835 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:31.835 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:31.835 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:31.835 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:31.835 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:31.835 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:31.835 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 18209 2024-05-08T08:53:31.835 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.835 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:31.835 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.835 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:31.835 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.787843+0000", 2024-05-08T08:53:31.836 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.836 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.836 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.836 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:31.836 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.787843+0000", 2024-05-08T08:53:31.836 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.836 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.836 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.836 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:31.836 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788381+0000", 2024-05-08T08:53:31.836 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00053830299999999996 2024-05-08T08:53:31.836 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.836 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.836 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:31.836 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788386+0000", 2024-05-08T08:53:31.836 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.566e-06 2024-05-08T08:53:31.837 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.837 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.837 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:31.837 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788395+0000", 2024-05-08T08:53:31.837 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4539999999999996e-06 2024-05-08T08:53:31.837 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.837 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.837 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:31.837 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788402+0000", 2024-05-08T08:53:31.837 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6630000000000004e-06 2024-05-08T08:53:31.837 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.837 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.837 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.837 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.837 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.838 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:18241 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:31.838 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.790373+0000", 2024-05-08T08:53:31.838 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.015690929999999999, 2024-05-08T08:53:31.838 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.01620591, 2024-05-08T08:53:31.838 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:31.838 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:31.838 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:31.838 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:31.838 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:31.838 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:31.838 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 18241 2024-05-08T08:53:31.838 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.838 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:31.838 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.838 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:31.839 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.790373+0000", 2024-05-08T08:53:31.839 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.839 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.839 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.839 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:31.839 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.790373+0000", 2024-05-08T08:53:31.839 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.839 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.839 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.839 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:31.839 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.790374+0000", 2024-05-08T08:53:31.839 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1099999999999995e-07 2024-05-08T08:53:31.839 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.839 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.840 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:31.840 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.790378+0000", 2024-05-08T08:53:31.840 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9389999999999999e-06 2024-05-08T08:53:31.840 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.840 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.840 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:31.840 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.790385+0000", 2024-05-08T08:53:31.840 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.515e-06 2024-05-08T08:53:31.840 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.840 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.840 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:31.840 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.790391+0000", 2024-05-08T08:53:31.840 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5550000000000002e-06 2024-05-08T08:53:31.840 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.840 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.841 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.841 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.841 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.841 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:17760 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:31.841 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.774478+0000", 2024-05-08T08:53:31.841 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.031586053000000003, 2024-05-08T08:53:31.841 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.032160465999999999, 2024-05-08T08:53:31.841 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:31.841 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:31.841 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:31.841 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:31.841 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:31.841 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:31.841 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 17760 2024-05-08T08:53:31.841 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.842 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:31.842 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.842 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:31.842 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.774478+0000", 2024-05-08T08:53:31.842 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.842 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.842 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.842 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:31.842 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.774478+0000", 2024-05-08T08:53:31.842 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.842 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.842 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.842 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:31.842 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.774479+0000", 2024-05-08T08:53:31.842 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4600000000000004e-07 2024-05-08T08:53:31.843 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.843 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.843 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:31.843 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.774481+0000", 2024-05-08T08:53:31.843 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.898e-06 2024-05-08T08:53:31.843 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.843 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.843 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:31.843 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.774489+0000", 2024-05-08T08:53:31.843 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1169999999999998e-06 2024-05-08T08:53:31.843 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.843 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.843 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:31.843 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.774494+0000", 2024-05-08T08:53:31.844 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6720000000000001e-06 2024-05-08T08:53:31.844 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.844 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.844 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.844 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.844 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.844 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:17810 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:31.844 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.779306+0000", 2024-05-08T08:53:31.844 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.026757748000000001, 2024-05-08T08:53:31.844 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.027390848999999998, 2024-05-08T08:53:31.844 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:31.844 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:31.844 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:31.844 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:31.845 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:31.845 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:31.845 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 17810 2024-05-08T08:53:31.845 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.845 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:31.845 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.845 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:31.845 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.779306+0000", 2024-05-08T08:53:31.845 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.845 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.845 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.845 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:31.845 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.779306+0000", 2024-05-08T08:53:31.845 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.845 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.846 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.846 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:31.846 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.779307+0000", 2024-05-08T08:53:31.846 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5600000000000004e-07 2024-05-08T08:53:31.846 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.846 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.846 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:31.846 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.779311+0000", 2024-05-08T08:53:31.846 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6799999999999999e-06 2024-05-08T08:53:31.846 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.846 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.846 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:31.846 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.779319+0000", 2024-05-08T08:53:31.846 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.5350000000000004e-06 2024-05-08T08:53:31.847 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.847 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.847 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:31.847 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.779326+0000", 2024-05-08T08:53:31.847 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4819999999999999e-06 2024-05-08T08:53:31.847 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.847 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.847 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.847 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.847 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.847 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:14840 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:31.847 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.785104+0000", 2024-05-08T08:53:31.847 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.020960441999999999, 2024-05-08T08:53:31.847 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.021652861999999998, 2024-05-08T08:53:31.847 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:31.848 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:31.848 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:31.848 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:31.848 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:31.848 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:31.848 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14840 2024-05-08T08:53:31.848 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.848 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:31.848 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.848 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:31.848 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.785104+0000", 2024-05-08T08:53:31.848 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.848 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.848 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.848 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:31.849 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.785104+0000", 2024-05-08T08:53:31.849 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.849 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.849 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.849 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:31.849 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.785104+0000", 2024-05-08T08:53:31.849 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0400000000000005e-07 2024-05-08T08:53:31.849 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.849 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.849 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:31.849 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.785109+0000", 2024-05-08T08:53:31.849 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7149999999999997e-06 2024-05-08T08:53:31.849 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.849 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.849 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:31.850 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.785118+0000", 2024-05-08T08:53:31.850 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.8519999999999993e-06 2024-05-08T08:53:31.850 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.850 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.850 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:31.850 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.785124+0000", 2024-05-08T08:53:31.850 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8320000000000002e-06 2024-05-08T08:53:31.850 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.850 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.850 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.850 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.850 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.850 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:18210 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:31.850 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.788414+0000", 2024-05-08T08:53:31.850 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.017650058, 2024-05-08T08:53:31.850 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.018400395999999999, 2024-05-08T08:53:31.851 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:31.851 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:31.851 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:31.851 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:31.851 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:31.851 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:31.851 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 18210 2024-05-08T08:53:31.851 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.851 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:31.851 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.851 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:31.851 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788414+0000", 2024-05-08T08:53:31.851 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.851 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.851 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.852 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:31.852 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788414+0000", 2024-05-08T08:53:31.852 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.852 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.852 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.852 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:31.852 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788414+0000", 2024-05-08T08:53:31.852 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8800000000000002e-07 2024-05-08T08:53:31.852 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.852 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.852 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:31.852 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788418+0000", 2024-05-08T08:53:31.852 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1939999999999998e-06 2024-05-08T08:53:31.852 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.852 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.853 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:31.853 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788424+0000", 2024-05-08T08:53:31.853 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5139999999999998e-06 2024-05-08T08:53:31.853 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.853 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.853 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:31.853 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788430+0000", 2024-05-08T08:53:31.853 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5450000000000003e-06 2024-05-08T08:53:31.853 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.853 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.853 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.853 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.853 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.853 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:18515 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:31.854 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.806167+0000", 2024-05-08T08:53:31.854 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 4294967295.999897, 2024-05-08T08:53:31.854 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00070622600000000001, 2024-05-08T08:53:31.854 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:31.854 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:31.854 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:31.854 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:31.854 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:31.854 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:31.854 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 18515 2024-05-08T08:53:31.854 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.854 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:31.854 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.854 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:31.854 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.806167+0000", 2024-05-08T08:53:31.855 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.855 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.855 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.855 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:31.855 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.806167+0000", 2024-05-08T08:53:31.855 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.855 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.855 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.855 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:31.855 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.806167+0000", 2024-05-08T08:53:31.855 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7899999999999998e-07 2024-05-08T08:53:31.855 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.855 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.855 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:31.855 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.806170+0000", 2024-05-08T08:53:31.856 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.915e-06 2024-05-08T08:53:31.856 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.856 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.856 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:31.856 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.806181+0000", 2024-05-08T08:53:31.856 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1046000000000001e-05 2024-05-08T08:53:31.856 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.856 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.856 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:31.856 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.806190+0000", 2024-05-08T08:53:31.856 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4600000000000003e-06 2024-05-08T08:53:31.856 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.856 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.856 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.856 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.857 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.857 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:14800 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:31.857 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.774500+0000", 2024-05-08T08:53:31.857 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.031563701, 2024-05-08T08:53:31.857 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.032432995999999999, 2024-05-08T08:53:31.857 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:31.857 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:31.857 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:31.857 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:31.857 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:31.857 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:31.857 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14800 2024-05-08T08:53:31.857 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.857 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:31.857 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.857 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:31.858 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.774500+0000", 2024-05-08T08:53:31.858 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.858 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.858 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.858 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:31.858 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.774500+0000", 2024-05-08T08:53:31.858 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.858 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.858 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.858 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:31.858 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.774501+0000", 2024-05-08T08:53:31.858 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3399999999999998e-07 2024-05-08T08:53:31.858 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.858 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.859 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:31.859 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.774505+0000", 2024-05-08T08:53:31.859 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1300000000000003e-06 2024-05-08T08:53:31.859 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.859 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.859 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:31.859 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.774513+0000", 2024-05-08T08:53:31.859 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9000000000000006e-06 2024-05-08T08:53:31.859 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.859 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.859 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:31.859 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.774520+0000", 2024-05-08T08:53:31.859 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5010000000000001e-06 2024-05-08T08:53:31.859 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.860 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.860 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.860 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.860 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.860 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:14809 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:31.860 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.776666+0000", 2024-05-08T08:53:31.860 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.029397526, 2024-05-08T08:53:31.860 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.030332676999999999, 2024-05-08T08:53:31.860 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:31.860 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:31.860 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:31.860 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:31.860 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:31.860 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:31.860 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14809 2024-05-08T08:53:31.860 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.861 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:31.861 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.861 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:31.861 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.776666+0000", 2024-05-08T08:53:31.861 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.861 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.861 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.861 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:31.861 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.776666+0000", 2024-05-08T08:53:31.861 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.861 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.861 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.861 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:31.861 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.776667+0000", 2024-05-08T08:53:31.861 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8800000000000002e-07 2024-05-08T08:53:31.862 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.862 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.862 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:31.862 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.776671+0000", 2024-05-08T08:53:31.862 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2640000000000002e-06 2024-05-08T08:53:31.862 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.862 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.862 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:31.862 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.776682+0000", 2024-05-08T08:53:31.862 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0380000000000001e-05 2024-05-08T08:53:31.862 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.862 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.862 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:31.862 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.776689+0000", 2024-05-08T08:53:31.862 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7519999999999999e-06 2024-05-08T08:53:31.863 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.863 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.863 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.863 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.863 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.863 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:14821 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:31.863 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.779342+0000", 2024-05-08T08:53:31.863 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.026722458000000001, 2024-05-08T08:53:31.863 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.027715386000000002, 2024-05-08T08:53:31.863 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:31.863 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:31.863 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:31.863 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:31.863 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:31.863 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:31.864 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14821 2024-05-08T08:53:31.864 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.864 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:31.864 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.864 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:31.864 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.779342+0000", 2024-05-08T08:53:31.864 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.864 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.864 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.864 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:31.864 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.779342+0000", 2024-05-08T08:53:31.864 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.864 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.864 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.864 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:31.865 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.779342+0000", 2024-05-08T08:53:31.865 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5199999999999997e-07 2024-05-08T08:53:31.865 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.865 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.865 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:31.865 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.779346+0000", 2024-05-08T08:53:31.865 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.478e-06 2024-05-08T08:53:31.865 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.865 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.865 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:31.865 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.779351+0000", 2024-05-08T08:53:31.865 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1209999999999998e-06 2024-05-08T08:53:31.865 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.865 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.866 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:31.866 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.779357+0000", 2024-05-08T08:53:31.866 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.981e-06 2024-05-08T08:53:31.866 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.866 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.866 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.866 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.866 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.866 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:14837 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:31.866 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.782850+0000", 2024-05-08T08:53:31.866 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.023214254, 2024-05-08T08:53:31.866 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.024328392000000001, 2024-05-08T08:53:31.866 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:31.866 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:31.867 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:31.867 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:31.867 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:31.867 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:31.867 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14837 2024-05-08T08:53:31.867 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.867 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:31.867 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.867 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:31.867 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.782850+0000", 2024-05-08T08:53:31.867 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.867 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.867 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.867 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:31.868 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.782850+0000", 2024-05-08T08:53:31.868 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.868 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.868 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.868 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:31.868 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.782851+0000", 2024-05-08T08:53:31.868 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0419999999999999e-06 2024-05-08T08:53:31.868 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.868 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.868 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:31.868 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.782855+0000", 2024-05-08T08:53:31.868 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2289999999999997e-06 2024-05-08T08:53:31.868 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.868 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.868 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:31.869 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.782866+0000", 2024-05-08T08:53:31.869 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0566e-05 2024-05-08T08:53:31.869 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.869 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.869 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:31.869 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.782873+0000", 2024-05-08T08:53:31.869 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1330000000000001e-06 2024-05-08T08:53:31.869 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.869 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.869 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.869 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.869 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.869 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:18211 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:31.869 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.788438+0000", 2024-05-08T08:53:31.869 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.017626422999999999, 2024-05-08T08:53:31.870 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.018812572, 2024-05-08T08:53:31.870 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:31.870 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:31.870 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:31.870 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:31.870 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:31.870 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:31.870 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 18211 2024-05-08T08:53:31.870 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.870 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:31.870 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.870 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:31.870 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788438+0000", 2024-05-08T08:53:31.870 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.871 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.871 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.871 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:31.871 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788438+0000", 2024-05-08T08:53:31.871 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.871 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.871 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.871 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:31.871 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788438+0000", 2024-05-08T08:53:31.871 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9100000000000004e-07 2024-05-08T08:53:31.871 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.871 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.871 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:31.871 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788441+0000", 2024-05-08T08:53:31.871 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.593e-06 2024-05-08T08:53:31.871 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.872 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.872 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:31.872 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788447+0000", 2024-05-08T08:53:31.872 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2759999999999997e-06 2024-05-08T08:53:31.872 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.872 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.872 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:31.872 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788452+0000", 2024-05-08T08:53:31.872 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0259999999999998e-06 2024-05-08T08:53:31.872 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.872 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.872 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.872 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.872 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.872 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:18243 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:31.873 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.790422+0000", 2024-05-08T08:53:31.873 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.015642321000000001, 2024-05-08T08:53:31.873 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.016895384999999999, 2024-05-08T08:53:31.873 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:31.873 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:31.873 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:31.873 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:31.873 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:31.873 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:31.873 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 18243 2024-05-08T08:53:31.873 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.873 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:31.873 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.873 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:31.873 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.790422+0000", 2024-05-08T08:53:31.874 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.874 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.874 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.874 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:31.874 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.790422+0000", 2024-05-08T08:53:31.874 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.874 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.874 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.874 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:31.874 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.790960+0000", 2024-05-08T08:53:31.874 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000538354 2024-05-08T08:53:31.874 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.874 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.874 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:31.875 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.790965+0000", 2024-05-08T08:53:31.875 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7110000000000003e-06 2024-05-08T08:53:31.875 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.875 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.875 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:31.875 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.790974+0000", 2024-05-08T08:53:31.875 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6989999999999998e-06 2024-05-08T08:53:31.875 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.875 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.875 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:31.875 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.790981+0000", 2024-05-08T08:53:31.875 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5610000000000004e-06 2024-05-08T08:53:31.875 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.875 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.875 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.876 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.876 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.876 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:14860 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:31.876 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.806164+0000", 2024-05-08T08:53:31.876 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 4294967295.9998999, 2024-05-08T08:53:31.876 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0012203030000000001, 2024-05-08T08:53:31.876 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:31.876 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:31.876 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:31.876 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:31.876 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:31.876 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:31.876 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14860 2024-05-08T08:53:31.876 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.877 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:31.877 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.877 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:31.877 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.806164+0000", 2024-05-08T08:53:31.877 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.877 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.877 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.877 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:31.877 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.806164+0000", 2024-05-08T08:53:31.877 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.877 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.877 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.877 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:31.877 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.806165+0000", 2024-05-08T08:53:31.877 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4099999999999998e-07 2024-05-08T08:53:31.877 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.878 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.878 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:31.878 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.806170+0000", 2024-05-08T08:53:31.878 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8149999999999996e-06 2024-05-08T08:53:31.878 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.878 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.878 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:31.878 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.806183+0000", 2024-05-08T08:53:31.878 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3106e-05 2024-05-08T08:53:31.878 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.878 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.878 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:31.878 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.806190+0000", 2024-05-08T08:53:31.878 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1940000000000001e-06 2024-05-08T08:53:31.878 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.879 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.879 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.879 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.879 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.879 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:14810 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:31.879 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.776796+0000", 2024-05-08T08:53:31.879 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.029268018999999999, 2024-05-08T08:53:31.879 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.030660957999999999, 2024-05-08T08:53:31.879 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:31.879 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:31.879 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:31.879 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:31.879 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:31.879 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:31.879 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14810 2024-05-08T08:53:31.880 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.880 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:31.880 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.880 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:31.880 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.776796+0000", 2024-05-08T08:53:31.880 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.880 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.880 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.880 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:31.880 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.776796+0000", 2024-05-08T08:53:31.880 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.880 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.880 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.880 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:31.880 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.776797+0000", 2024-05-08T08:53:31.881 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.54e-07 2024-05-08T08:53:31.881 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.881 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.881 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:31.881 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.776801+0000", 2024-05-08T08:53:31.881 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2320000000000003e-06 2024-05-08T08:53:31.881 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.881 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.881 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:31.881 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.776810+0000", 2024-05-08T08:53:31.881 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5449999999999992e-06 2024-05-08T08:53:31.881 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.881 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.881 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:31.881 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.776817+0000", 2024-05-08T08:53:31.882 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5030000000000002e-06 2024-05-08T08:53:31.882 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.882 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.882 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.882 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.882 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.882 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:14822 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:31.882 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.779481+0000", 2024-05-08T08:53:31.882 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.026582813, 2024-05-08T08:53:31.882 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.028040992000000001, 2024-05-08T08:53:31.882 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:31.882 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:31.882 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:31.882 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:31.882 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:31.883 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:31.883 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14822 2024-05-08T08:53:31.883 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.883 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:31.883 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.883 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:31.883 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.779481+0000", 2024-05-08T08:53:31.883 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.883 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.883 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.883 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:31.883 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.779481+0000", 2024-05-08T08:53:31.883 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.883 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.883 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.884 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:31.884 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.779482+0000", 2024-05-08T08:53:31.884 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5499999999999997e-07 2024-05-08T08:53:31.884 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.884 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.884 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:31.884 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.779486+0000", 2024-05-08T08:53:31.884 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4129999999999999e-06 2024-05-08T08:53:31.884 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.884 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.884 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:31.884 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.779495+0000", 2024-05-08T08:53:31.884 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.507e-06 2024-05-08T08:53:31.884 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.885 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.885 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:31.885 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.779501+0000", 2024-05-08T08:53:31.885 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2450000000000003e-06 2024-05-08T08:53:31.885 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.885 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.885 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.885 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.885 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.885 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:14841 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:31.885 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.785391+0000", 2024-05-08T08:53:31.885 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.020672751, 2024-05-08T08:53:31.885 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.022190436000000001, 2024-05-08T08:53:31.885 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:31.885 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:31.885 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:31.886 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:31.886 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:31.886 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:31.886 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14841 2024-05-08T08:53:31.886 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.886 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:31.886 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.886 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:31.886 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.785391+0000", 2024-05-08T08:53:31.886 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.886 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.886 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.886 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:31.886 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.785391+0000", 2024-05-08T08:53:31.886 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.887 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.887 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.887 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:31.887 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.785392+0000", 2024-05-08T08:53:31.887 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.5499999999999997e-07 2024-05-08T08:53:31.887 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.887 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.887 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:31.887 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.785397+0000", 2024-05-08T08:53:31.887 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6319999999999999e-06 2024-05-08T08:53:31.887 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.887 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.887 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:31.887 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.785408+0000", 2024-05-08T08:53:31.887 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1209999999999999e-05 2024-05-08T08:53:31.887 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.888 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.888 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:31.888 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.785415+0000", 2024-05-08T08:53:31.888 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1400000000000002e-06 2024-05-08T08:53:31.888 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.888 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.888 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.888 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.888 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.888 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:18212 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:31.888 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.788481+0000", 2024-05-08T08:53:31.888 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.017583188999999999, 2024-05-08T08:53:31.888 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.019205063000000001, 2024-05-08T08:53:31.888 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:31.888 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:31.889 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:31.889 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:31.889 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:31.889 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:31.889 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 18212 2024-05-08T08:53:31.889 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.889 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:31.889 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.889 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:31.889 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788481+0000", 2024-05-08T08:53:31.889 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.889 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.889 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.889 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:31.889 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788481+0000", 2024-05-08T08:53:31.889 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.890 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.890 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.890 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:31.890 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788481+0000", 2024-05-08T08:53:31.890 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9800000000000004e-07 2024-05-08T08:53:31.890 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.890 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.890 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:31.890 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788485+0000", 2024-05-08T08:53:31.890 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.411e-06 2024-05-08T08:53:31.890 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.890 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.890 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:31.890 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788492+0000", 2024-05-08T08:53:31.890 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4479999999999997e-06 2024-05-08T08:53:31.891 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.891 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.891 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:31.891 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788498+0000", 2024-05-08T08:53:31.891 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4620000000000004e-06 2024-05-08T08:53:31.891 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.891 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.891 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.891 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.891 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.891 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:18244 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:31.891 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.790995+0000", 2024-05-08T08:53:31.891 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.015069157999999999, 2024-05-08T08:53:31.891 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.016815034999999999, 2024-05-08T08:53:31.891 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:31.892 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:31.892 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:31.892 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:31.892 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:31.892 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:31.892 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 18244 2024-05-08T08:53:31.892 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.892 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:31.892 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.892 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:31.892 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.790995+0000", 2024-05-08T08:53:31.892 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.892 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.892 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.892 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:31.893 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.790995+0000", 2024-05-08T08:53:31.893 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.893 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.893 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.893 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:31.893 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.790995+0000", 2024-05-08T08:53:31.893 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7300000000000001e-07 2024-05-08T08:53:31.893 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.893 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.893 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:31.893 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.790998+0000", 2024-05-08T08:53:31.893 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7980000000000001e-06 2024-05-08T08:53:31.893 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.893 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.893 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:31.894 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.791005+0000", 2024-05-08T08:53:31.894 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1049999999999997e-06 2024-05-08T08:53:31.894 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.894 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.894 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:31.894 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.791011+0000", 2024-05-08T08:53:31.894 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6969999999999998e-06 2024-05-08T08:53:31.894 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.894 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.894 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.894 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.894 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.894 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:14811 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:31.894 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.777056+0000", 2024-05-08T08:53:31.894 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.029007508000000001, 2024-05-08T08:53:31.895 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.030845043999999999, 2024-05-08T08:53:31.895 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:31.895 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:31.895 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:31.895 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:31.895 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:31.895 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:31.895 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14811 2024-05-08T08:53:31.895 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.895 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:31.895 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.895 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:31.895 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.777056+0000", 2024-05-08T08:53:31.895 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.895 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.896 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.896 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:31.896 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.777056+0000", 2024-05-08T08:53:31.896 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.896 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.896 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.896 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:31.896 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.777057+0000", 2024-05-08T08:53:31.896 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.06e-07 2024-05-08T08:53:31.896 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.896 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.896 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:31.896 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.777063+0000", 2024-05-08T08:53:31.896 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8760000000000002e-06 2024-05-08T08:53:31.896 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.896 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.897 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:31.897 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.777073+0000", 2024-05-08T08:53:31.897 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0190000000000001e-05 2024-05-08T08:53:31.897 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.897 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.897 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:31.897 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.777081+0000", 2024-05-08T08:53:31.897 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5569999999999998e-06 2024-05-08T08:53:31.897 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.897 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.897 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.897 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.897 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.897 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:14823 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:31.897 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.779705+0000", 2024-05-08T08:53:31.898 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.02635875, 2024-05-08T08:53:31.898 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.028277949, 2024-05-08T08:53:31.898 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:31.898 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:31.898 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:31.898 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:31.898 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:31.898 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:31.898 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14823 2024-05-08T08:53:31.898 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.898 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:31.898 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.898 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:31.898 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.779705+0000", 2024-05-08T08:53:31.898 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.899 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.899 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.899 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:31.899 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.779705+0000", 2024-05-08T08:53:31.899 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.899 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.899 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.899 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:31.899 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.779706+0000", 2024-05-08T08:53:31.899 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7899999999999997e-07 2024-05-08T08:53:31.899 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.899 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.899 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:31.899 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.779711+0000", 2024-05-08T08:53:31.899 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.561e-06 2024-05-08T08:53:31.900 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.900 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.900 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:31.900 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.779722+0000", 2024-05-08T08:53:31.900 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0933e-05 2024-05-08T08:53:31.900 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.900 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.900 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:31.900 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.779729+0000", 2024-05-08T08:53:31.900 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1210000000000001e-06 2024-05-08T08:53:31.900 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.900 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.900 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.900 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.900 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.901 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:14842 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:31.901 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.785430+0000", 2024-05-08T08:53:31.901 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.020634293000000001, 2024-05-08T08:53:31.901 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.022610982000000002, 2024-05-08T08:53:31.901 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:31.901 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:31.901 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:31.901 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:31.901 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:31.901 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:31.901 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14842 2024-05-08T08:53:31.901 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.901 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:31.901 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.901 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:31.901 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.785430+0000", 2024-05-08T08:53:31.902 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.902 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.902 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.902 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:31.902 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.785430+0000", 2024-05-08T08:53:31.902 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.902 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.902 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.902 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:31.902 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.785430+0000", 2024-05-08T08:53:31.902 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6899999999999997e-07 2024-05-08T08:53:31.902 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.902 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.902 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:31.902 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.785433+0000", 2024-05-08T08:53:31.903 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0790000000000001e-06 2024-05-08T08:53:31.903 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.903 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.903 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:31.903 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.785453+0000", 2024-05-08T08:53:31.903 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9908e-05 2024-05-08T08:53:31.903 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.903 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.903 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:31.903 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.785460+0000", 2024-05-08T08:53:31.903 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8589999999999999e-06 2024-05-08T08:53:31.903 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.903 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.903 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.903 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.904 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.904 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:18213 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:31.904 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.788506+0000", 2024-05-08T08:53:31.904 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.017557547, 2024-05-08T08:53:31.904 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.019600801000000001, 2024-05-08T08:53:31.904 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:31.904 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:31.904 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:31.904 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:31.904 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:31.905 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:31.905 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 18213 2024-05-08T08:53:31.905 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.905 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:31.905 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.905 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:31.905 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788506+0000", 2024-05-08T08:53:31.905 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.905 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.905 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.905 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:31.905 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788506+0000", 2024-05-08T08:53:31.905 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.905 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.905 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.906 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:31.906 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788507+0000", 2024-05-08T08:53:31.906 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.15e-07 2024-05-08T08:53:31.906 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.906 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.906 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:31.906 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788509+0000", 2024-05-08T08:53:31.906 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6599999999999999e-06 2024-05-08T08:53:31.906 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.906 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.906 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:31.906 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788516+0000", 2024-05-08T08:53:31.906 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3910000000000003e-06 2024-05-08T08:53:31.906 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.907 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.907 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:31.907 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788521+0000", 2024-05-08T08:53:31.907 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9840000000000001e-06 2024-05-08T08:53:31.907 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.907 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.907 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.907 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.907 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.907 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:18246 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:31.907 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.791021+0000", 2024-05-08T08:53:31.907 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.015043261, 2024-05-08T08:53:31.907 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.017144487, 2024-05-08T08:53:31.907 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:31.908 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:31.908 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:31.908 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:31.908 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:31.908 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:31.908 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 18246 2024-05-08T08:53:31.908 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.908 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:31.908 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.908 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:31.908 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.791021+0000", 2024-05-08T08:53:31.908 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.908 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.908 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.908 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:31.909 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.791021+0000", 2024-05-08T08:53:31.909 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.909 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.909 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.909 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:31.909 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.791021+0000", 2024-05-08T08:53:31.909 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9299999999999998e-07 2024-05-08T08:53:31.909 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.909 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.909 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:31.909 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.791024+0000", 2024-05-08T08:53:31.909 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8609999999999998e-06 2024-05-08T08:53:31.909 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.909 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.910 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:31.910 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.791030+0000", 2024-05-08T08:53:31.910 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2389999999999999e-06 2024-05-08T08:53:31.910 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.910 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.910 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:31.910 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.791036+0000", 2024-05-08T08:53:31.910 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.626e-06 2024-05-08T08:53:31.910 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.910 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.910 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.910 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.910 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.910 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:17787 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:31.911 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.777013+0000", 2024-05-08T08:53:31.911 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.029050660999999998, 2024-05-08T08:53:31.911 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.031213637999999998, 2024-05-08T08:53:31.911 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:31.911 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:31.911 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:31.911 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:31.911 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:31.911 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:31.911 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 17787 2024-05-08T08:53:31.911 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.911 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:31.911 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.911 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:31.911 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.777013+0000", 2024-05-08T08:53:31.912 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.912 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.912 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.912 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:31.912 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.777013+0000", 2024-05-08T08:53:31.912 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.912 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.912 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.912 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:31.912 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.777014+0000", 2024-05-08T08:53:31.912 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2600000000000001e-07 2024-05-08T08:53:31.912 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.912 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.912 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:31.912 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.777018+0000", 2024-05-08T08:53:31.913 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0980000000000004e-06 2024-05-08T08:53:31.913 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.913 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.913 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:31.913 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.777075+0000", 2024-05-08T08:53:31.913 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6931000000000001e-05 2024-05-08T08:53:31.913 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.913 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.913 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:31.913 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.777082+0000", 2024-05-08T08:53:31.913 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4339999999999997e-06 2024-05-08T08:53:31.913 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.913 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.913 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.913 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.914 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.914 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:17881 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:31.914 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.782909+0000", 2024-05-08T08:53:31.914 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.023155426, 2024-05-08T08:53:31.914 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.025377440000000001, 2024-05-08T08:53:31.914 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:31.914 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:31.914 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:31.914 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:31.914 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:31.914 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:31.914 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 17881 2024-05-08T08:53:31.914 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.914 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:31.914 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.915 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:31.915 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.782909+0000", 2024-05-08T08:53:31.915 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.915 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.915 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.915 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:31.915 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.782909+0000", 2024-05-08T08:53:31.915 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.915 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.915 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.915 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:31.915 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.782909+0000", 2024-05-08T08:53:31.915 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8799999999999998e-07 2024-05-08T08:53:31.915 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.915 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.916 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:31.916 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.782911+0000", 2024-05-08T08:53:31.916 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2639999999999998e-06 2024-05-08T08:53:31.916 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.916 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.916 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:31.916 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.782916+0000", 2024-05-08T08:53:31.916 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0599999999999998e-06 2024-05-08T08:53:31.916 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.916 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.916 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:31.916 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.782920+0000", 2024-05-08T08:53:31.916 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0969999999999999e-06 2024-05-08T08:53:31.916 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.916 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.916 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.917 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.917 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.917 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:14843 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:31.917 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.785470+0000", 2024-05-08T08:53:31.917 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.020593917, 2024-05-08T08:53:31.917 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.022873896000000001, 2024-05-08T08:53:31.917 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:31.917 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:31.917 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:31.917 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:31.917 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:31.917 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:31.917 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14843 2024-05-08T08:53:31.917 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.917 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:31.918 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.918 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:31.918 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.785470+0000", 2024-05-08T08:53:31.918 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.918 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.918 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.918 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:31.918 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.785470+0000", 2024-05-08T08:53:31.918 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.918 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.918 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.918 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:31.918 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.785471+0000", 2024-05-08T08:53:31.918 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1799999999999995e-07 2024-05-08T08:53:31.919 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.919 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.919 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:31.919 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.785474+0000", 2024-05-08T08:53:31.919 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3220000000000001e-06 2024-05-08T08:53:31.919 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.919 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.919 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:31.919 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.785482+0000", 2024-05-08T08:53:31.919 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.695e-06 2024-05-08T08:53:31.919 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.919 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.919 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:31.919 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.785487+0000", 2024-05-08T08:53:31.920 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5389999999999998e-06 2024-05-08T08:53:31.920 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.920 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.920 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.920 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.920 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.920 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:18214 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:31.920 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.788528+0000", 2024-05-08T08:53:31.920 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.017536012, 2024-05-08T08:53:31.920 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.019882935000000001, 2024-05-08T08:53:31.920 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:31.920 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:31.920 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:31.920 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:31.920 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:31.921 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:31.921 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 18214 2024-05-08T08:53:31.921 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.921 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:31.921 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.921 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:31.921 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788528+0000", 2024-05-08T08:53:31.921 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.921 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.921 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.921 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:31.921 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788528+0000", 2024-05-08T08:53:31.921 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.921 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.921 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.922 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:31.922 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788528+0000", 2024-05-08T08:53:31.922 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2e-07 2024-05-08T08:53:31.922 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.922 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.922 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:31.922 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788538+0000", 2024-05-08T08:53:31.922 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8120000000000002e-06 2024-05-08T08:53:31.922 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.922 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.922 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:31.922 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788545+0000", 2024-05-08T08:53:31.922 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6780000000000003e-06 2024-05-08T08:53:31.922 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.923 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.923 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:31.923 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788550+0000", 2024-05-08T08:53:31.923 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3319999999999999e-06 2024-05-08T08:53:31.923 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.923 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.923 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.923 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.923 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.923 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:17788 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:31.923 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.777094+0000", 2024-05-08T08:53:31.923 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.028970065999999999, 2024-05-08T08:53:31.923 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.031376915999999998, 2024-05-08T08:53:31.923 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:31.924 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:31.924 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:31.924 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:31.924 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:31.924 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:31.924 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 17788 2024-05-08T08:53:31.924 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.924 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:31.924 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.924 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:31.924 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.777094+0000", 2024-05-08T08:53:31.924 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.924 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.924 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.924 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:31.924 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.777094+0000", 2024-05-08T08:53:31.925 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.925 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.925 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.925 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:31.925 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.777094+0000", 2024-05-08T08:53:31.925 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7899999999999998e-07 2024-05-08T08:53:31.925 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.925 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.925 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:31.925 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.777098+0000", 2024-05-08T08:53:31.925 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8580000000000002e-06 2024-05-08T08:53:31.925 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.925 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.925 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:31.925 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.777107+0000", 2024-05-08T08:53:31.926 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1699999999999997e-06 2024-05-08T08:53:31.926 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.926 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.926 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:31.926 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.777113+0000", 2024-05-08T08:53:31.926 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4119999999999998e-06 2024-05-08T08:53:31.926 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.926 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.926 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.926 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.926 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.926 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:17882 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:31.926 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.782926+0000", 2024-05-08T08:53:31.926 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.023138184999999999, 2024-05-08T08:53:31.926 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.025604353999999999, 2024-05-08T08:53:31.927 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:31.927 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:31.927 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:31.927 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:31.927 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:31.927 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:31.927 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 17882 2024-05-08T08:53:31.927 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.927 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:31.927 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.927 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:31.927 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.782926+0000", 2024-05-08T08:53:31.927 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.927 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.927 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.927 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:31.927 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.782926+0000", 2024-05-08T08:53:31.928 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.928 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.928 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.928 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:31.928 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.782926+0000", 2024-05-08T08:53:31.928 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.46e-07 2024-05-08T08:53:31.928 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.928 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.928 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:31.928 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.782928+0000", 2024-05-08T08:53:31.928 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1900000000000002e-06 2024-05-08T08:53:31.928 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.928 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.928 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:31.928 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.782933+0000", 2024-05-08T08:53:31.929 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9690000000000002e-06 2024-05-08T08:53:31.929 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.929 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.929 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:31.929 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.782938+0000", 2024-05-08T08:53:31.929 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3930000000000001e-06 2024-05-08T08:53:31.929 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.929 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.929 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.929 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.929 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.929 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:14844 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:31.929 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.785495+0000", 2024-05-08T08:53:31.929 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.020568524000000001, 2024-05-08T08:53:31.929 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0230928, 2024-05-08T08:53:31.929 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:31.929 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:31.929 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:31.930 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:31.930 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:31.930 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:31.930 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14844 2024-05-08T08:53:31.930 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.930 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:31.930 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.930 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:31.930 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.785495+0000", 2024-05-08T08:53:31.930 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.930 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.930 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.930 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:31.930 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.785495+0000", 2024-05-08T08:53:31.931 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.931 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.931 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.931 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:31.931 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.785613+0000", 2024-05-08T08:53:31.931 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011719500000000001 2024-05-08T08:53:31.931 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.931 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.931 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:31.931 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.785617+0000", 2024-05-08T08:53:31.931 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2339999999999996e-06 2024-05-08T08:53:31.931 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.931 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.931 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:31.931 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.785626+0000", 2024-05-08T08:53:31.931 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1679999999999993e-06 2024-05-08T08:53:31.932 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.932 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.932 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:31.932 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.785633+0000", 2024-05-08T08:53:31.932 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.686e-06 2024-05-08T08:53:31.932 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.932 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.932 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.932 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.932 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.932 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:18215 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:31.932 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.788558+0000", 2024-05-08T08:53:31.932 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.017506219999999999, 2024-05-08T08:53:31.932 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.020087937, 2024-05-08T08:53:31.932 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:31.932 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:31.932 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:31.933 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:31.933 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:31.933 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:31.933 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 18215 2024-05-08T08:53:31.933 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.933 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:31.933 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.933 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:31.933 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788558+0000", 2024-05-08T08:53:31.933 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.933 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.933 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.933 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:31.933 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788558+0000", 2024-05-08T08:53:31.933 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.934 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.934 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.934 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:31.934 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788558+0000", 2024-05-08T08:53:31.934 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7100000000000002e-07 2024-05-08T08:53:31.934 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.934 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.934 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:31.934 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788561+0000", 2024-05-08T08:53:31.934 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9050000000000001e-06 2024-05-08T08:53:31.934 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.934 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.934 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:31.934 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788569+0000", 2024-05-08T08:53:31.934 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9929999999999995e-06 2024-05-08T08:53:31.935 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.935 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.935 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:31.935 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788575+0000", 2024-05-08T08:53:31.935 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9240000000000004e-06 2024-05-08T08:53:31.935 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.935 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.935 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.935 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.935 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.935 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:17789 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:31.935 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.777123+0000", 2024-05-08T08:53:31.935 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.028940996, 2024-05-08T08:53:31.935 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.031583262000000001, 2024-05-08T08:53:31.936 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:31.936 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:31.936 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:31.936 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:31.936 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:31.936 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:31.936 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 17789 2024-05-08T08:53:31.936 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.936 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:31.936 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.936 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:31.936 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.777123+0000", 2024-05-08T08:53:31.936 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.936 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.937 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.937 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:31.937 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.777123+0000", 2024-05-08T08:53:31.937 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.937 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.937 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.937 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:31.937 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.777123+0000", 2024-05-08T08:53:31.937 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6600000000000002e-07 2024-05-08T08:53:31.937 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.937 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.937 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:31.937 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.777127+0000", 2024-05-08T08:53:31.937 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0800000000000002e-06 2024-05-08T08:53:31.938 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.938 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.938 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:31.938 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.777134+0000", 2024-05-08T08:53:31.938 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0770000000000002e-06 2024-05-08T08:53:31.938 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.938 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.938 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:31.938 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.777139+0000", 2024-05-08T08:53:31.938 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4519999999999996e-06 2024-05-08T08:53:31.938 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.938 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.938 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.938 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.939 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.939 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:17856 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:31.939 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.780420+0000", 2024-05-08T08:53:31.939 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.025643745999999999, 2024-05-08T08:53:31.939 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.028344214999999999, 2024-05-08T08:53:31.939 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:31.939 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:31.939 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:31.939 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:31.939 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:31.939 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:31.939 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 17856 2024-05-08T08:53:31.939 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.939 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:31.939 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.940 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:31.940 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.780420+0000", 2024-05-08T08:53:31.940 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.940 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.940 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.940 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:31.940 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.780420+0000", 2024-05-08T08:53:31.940 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.940 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.940 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.940 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:31.940 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.780421+0000", 2024-05-08T08:53:31.940 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7000000000000003e-07 2024-05-08T08:53:31.940 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.941 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.941 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:31.941 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.780426+0000", 2024-05-08T08:53:31.941 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9080000000000003e-06 2024-05-08T08:53:31.941 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.941 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.941 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:31.941 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.780437+0000", 2024-05-08T08:53:31.941 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1388e-05 2024-05-08T08:53:31.941 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.941 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.941 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:31.941 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.780445+0000", 2024-05-08T08:53:31.941 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7130000000000005e-06 2024-05-08T08:53:31.941 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.942 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.942 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.942 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.942 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.942 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:14845 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:31.942 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.785644+0000", 2024-05-08T08:53:31.942 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.020420420000000002, 2024-05-08T08:53:31.942 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.023204313000000001, 2024-05-08T08:53:31.942 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:31.942 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:31.942 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:31.942 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:31.942 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:31.942 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:31.943 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14845 2024-05-08T08:53:31.943 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.943 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:31.943 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.943 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:31.943 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.785644+0000", 2024-05-08T08:53:31.943 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.943 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.943 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.943 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:31.943 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.785644+0000", 2024-05-08T08:53:31.943 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.943 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.943 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.943 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:31.943 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.785644+0000", 2024-05-08T08:53:31.944 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1999999999999999e-07 2024-05-08T08:53:31.944 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.944 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.944 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:31.944 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.785647+0000", 2024-05-08T08:53:31.944 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.151e-06 2024-05-08T08:53:31.944 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.944 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.944 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:31.944 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.785654+0000", 2024-05-08T08:53:31.944 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1720000000000001e-06 2024-05-08T08:53:31.944 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.944 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.944 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:31.945 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.785663+0000", 2024-05-08T08:53:31.945 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0600000000000008e-06 2024-05-08T08:53:31.945 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.945 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.945 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.945 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.945 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.945 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:18216 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:31.945 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.788583+0000", 2024-05-08T08:53:31.945 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.017481391999999998, 2024-05-08T08:53:31.945 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.020324492, 2024-05-08T08:53:31.945 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:31.945 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:31.945 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:31.945 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:31.945 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:31.946 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:31.946 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 18216 2024-05-08T08:53:31.946 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.946 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:31.946 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.946 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:31.946 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788583+0000", 2024-05-08T08:53:31.946 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.946 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.946 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.946 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:31.946 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788583+0000", 2024-05-08T08:53:31.946 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.946 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.946 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.947 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:31.947 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788583+0000", 2024-05-08T08:53:31.947 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0900000000000002e-07 2024-05-08T08:53:31.947 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.947 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.947 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:31.947 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788586+0000", 2024-05-08T08:53:31.947 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6840000000000001e-06 2024-05-08T08:53:31.947 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.947 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.947 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:31.947 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788593+0000", 2024-05-08T08:53:31.947 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2749999999999998e-06 2024-05-08T08:53:31.947 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.947 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.948 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:31.948 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788598+0000", 2024-05-08T08:53:31.948 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0010000000000001e-06 2024-05-08T08:53:31.948 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.948 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.948 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.948 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.948 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.948 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:14838 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:31.948 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.783022+0000", 2024-05-08T08:53:31.948 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.023042396999999999, 2024-05-08T08:53:31.948 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.025951740000000001, 2024-05-08T08:53:31.948 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:31.948 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:31.948 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:31.948 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:31.948 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:31.949 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:31.949 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14838 2024-05-08T08:53:31.949 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.949 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:31.949 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.949 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:31.949 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.783022+0000", 2024-05-08T08:53:31.949 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.949 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.949 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.949 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:31.949 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.783022+0000", 2024-05-08T08:53:31.949 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.949 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.949 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.949 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:31.949 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.783022+0000", 2024-05-08T08:53:31.950 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8999999999999995e-07 2024-05-08T08:53:31.950 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.950 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.950 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:31.950 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.783027+0000", 2024-05-08T08:53:31.950 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3529999999999997e-06 2024-05-08T08:53:31.950 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.950 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.950 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:31.950 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.783038+0000", 2024-05-08T08:53:31.950 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1569000000000001e-05 2024-05-08T08:53:31.950 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.950 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.950 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:31.951 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.783046+0000", 2024-05-08T08:53:31.951 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1650000000000006e-06 2024-05-08T08:53:31.951 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.951 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.951 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.951 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.951 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.951 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:18217 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:31.951 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.788604+0000", 2024-05-08T08:53:31.951 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.017459498, 2024-05-08T08:53:31.951 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.020426031000000001, 2024-05-08T08:53:31.951 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:31.951 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:31.951 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:31.951 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:31.951 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:31.952 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:31.952 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 18217 2024-05-08T08:53:31.952 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.952 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:31.952 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.952 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:31.952 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788604+0000", 2024-05-08T08:53:31.952 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.952 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.952 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.952 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:31.952 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788604+0000", 2024-05-08T08:53:31.952 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.952 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.952 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.952 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:31.952 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788605+0000", 2024-05-08T08:53:31.953 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2899999999999999e-07 2024-05-08T08:53:31.953 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.953 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.953 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:31.953 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788607+0000", 2024-05-08T08:53:31.953 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.458e-06 2024-05-08T08:53:31.953 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.953 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.953 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:31.953 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788613+0000", 2024-05-08T08:53:31.953 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9689999999999999e-06 2024-05-08T08:53:31.953 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.953 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.953 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:31.953 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788618+0000", 2024-05-08T08:53:31.953 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8890000000000001e-06 2024-05-08T08:53:31.954 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.954 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.954 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.954 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.954 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.954 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:18377 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:31.954 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.795726+0000", 2024-05-08T08:53:31.954 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.010338444, 2024-05-08T08:53:31.954 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.013363595000000001, 2024-05-08T08:53:31.954 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:31.954 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:31.954 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:31.954 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:31.954 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:31.955 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:31.955 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 18377 2024-05-08T08:53:31.955 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.955 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:31.955 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.955 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:31.955 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.795726+0000", 2024-05-08T08:53:31.955 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.955 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.955 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.955 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:31.955 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.795726+0000", 2024-05-08T08:53:31.955 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.955 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.955 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.955 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:31.956 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.795726+0000", 2024-05-08T08:53:31.956 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6399999999999998e-07 2024-05-08T08:53:31.956 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.956 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.956 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:31.956 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.795729+0000", 2024-05-08T08:53:31.956 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7609999999999999e-06 2024-05-08T08:53:31.956 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.956 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.956 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:31.956 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.795735+0000", 2024-05-08T08:53:31.956 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3779999999999998e-06 2024-05-08T08:53:31.956 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.956 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.957 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:31.957 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.795740+0000", 2024-05-08T08:53:31.957 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7779999999999998e-06 2024-05-08T08:53:31.957 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.957 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.957 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.957 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.957 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.957 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:14824 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:31.957 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.781066+0000", 2024-05-08T08:53:31.957 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.024998234000000001, 2024-05-08T08:53:31.957 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.028083799999999999, 2024-05-08T08:53:31.957 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:31.957 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:31.957 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:31.957 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:31.958 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:31.958 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:31.958 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14824 2024-05-08T08:53:31.958 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.958 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:31.958 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.958 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:31.958 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.781066+0000", 2024-05-08T08:53:31.958 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.958 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.958 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.958 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:31.958 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.781066+0000", 2024-05-08T08:53:31.958 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.958 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.958 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.959 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:31.959 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.781067+0000", 2024-05-08T08:53:31.959 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8999999999999995e-07 2024-05-08T08:53:31.959 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.959 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.959 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:31.959 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.781072+0000", 2024-05-08T08:53:31.959 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2889999999999997e-06 2024-05-08T08:53:31.959 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.959 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.959 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:31.959 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.781083+0000", 2024-05-08T08:53:31.959 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0942e-05 2024-05-08T08:53:31.959 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.959 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.960 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:31.960 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.781090+0000", 2024-05-08T08:53:31.960 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.832e-06 2024-05-08T08:53:31.960 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.960 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.960 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.960 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.960 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.960 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:14839 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:31.960 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.783148+0000", 2024-05-08T08:53:31.960 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.022915979, 2024-05-08T08:53:31.960 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.026057724000000001, 2024-05-08T08:53:31.960 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:31.960 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:31.960 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:31.960 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:31.960 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:31.961 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:31.961 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14839 2024-05-08T08:53:31.961 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.961 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:31.961 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.961 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:31.961 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.783148+0000", 2024-05-08T08:53:31.961 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.961 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.961 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.961 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:31.961 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.783148+0000", 2024-05-08T08:53:31.961 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.961 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.961 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.962 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:31.962 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.783149+0000", 2024-05-08T08:53:31.962 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1999999999999998e-07 2024-05-08T08:53:31.962 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.962 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.962 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:31.962 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.783173+0000", 2024-05-08T08:53:31.962 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4170000000000001e-05 2024-05-08T08:53:31.962 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.962 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.962 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:31.962 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.783240+0000", 2024-05-08T08:53:31.962 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7312999999999997e-05 2024-05-08T08:53:31.962 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.962 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.963 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:31.963 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.783272+0000", 2024-05-08T08:53:31.963 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1634999999999998e-05 2024-05-08T08:53:31.963 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.963 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.963 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.963 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.963 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.963 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:18218 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:31.963 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.788625+0000", 2024-05-08T08:53:31.963 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.017438905000000001, 2024-05-08T08:53:31.963 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.020641389999999999, 2024-05-08T08:53:31.963 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:31.963 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:31.963 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:31.963 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:31.963 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:31.964 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:31.964 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 18218 2024-05-08T08:53:31.964 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.964 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:31.964 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.964 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:31.964 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788625+0000", 2024-05-08T08:53:31.964 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.964 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.964 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.964 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:31.964 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788625+0000", 2024-05-08T08:53:31.964 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.964 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.964 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.964 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:31.964 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788625+0000", 2024-05-08T08:53:31.965 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3599999999999999e-07 2024-05-08T08:53:31.965 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.965 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.965 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:31.965 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788628+0000", 2024-05-08T08:53:31.965 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4559999999999999e-06 2024-05-08T08:53:31.965 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.965 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.965 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:31.965 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788634+0000", 2024-05-08T08:53:31.965 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1109999999999996e-06 2024-05-08T08:53:31.965 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.965 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.965 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:31.965 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788639+0000", 2024-05-08T08:53:31.966 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7670000000000003e-06 2024-05-08T08:53:31.966 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.966 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.966 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.966 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.966 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.966 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:18395 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:31.967 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.795749+0000", 2024-05-08T08:53:31.967 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.01031468, 2024-05-08T08:53:31.967 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.013574678999999999, 2024-05-08T08:53:31.967 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:31.967 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:31.967 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:31.967 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:31.967 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:31.967 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:31.967 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 18395 2024-05-08T08:53:31.967 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.967 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:31.967 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.967 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:31.967 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.795749+0000", 2024-05-08T08:53:31.968 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.968 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.968 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.968 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:31.968 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.795749+0000", 2024-05-08T08:53:31.968 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.968 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.968 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.968 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:31.968 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.795750+0000", 2024-05-08T08:53:31.968 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1699999999999999e-07 2024-05-08T08:53:31.968 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.968 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.968 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:31.968 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.795753+0000", 2024-05-08T08:53:31.968 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8619999999999999e-06 2024-05-08T08:53:31.969 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.969 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.969 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:31.969 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.795759+0000", 2024-05-08T08:53:31.969 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7139999999999996e-06 2024-05-08T08:53:31.969 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.969 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.969 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:31.969 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.795765+0000", 2024-05-08T08:53:31.969 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0159999999999997e-06 2024-05-08T08:53:31.969 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.969 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.969 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.969 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.969 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.969 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:14825 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:31.970 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.781102+0000", 2024-05-08T08:53:31.970 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.024961526000000001, 2024-05-08T08:53:31.970 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.02828224, 2024-05-08T08:53:31.970 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:31.970 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:31.970 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:31.970 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:31.970 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:31.970 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:31.970 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14825 2024-05-08T08:53:31.970 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.970 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:31.970 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.970 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:31.970 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.781102+0000", 2024-05-08T08:53:31.970 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.971 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.971 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.971 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:31.971 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.781102+0000", 2024-05-08T08:53:31.971 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.971 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.971 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.971 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:31.971 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.781103+0000", 2024-05-08T08:53:31.971 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0399999999999996e-07 2024-05-08T08:53:31.971 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.971 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.971 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:31.971 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.781106+0000", 2024-05-08T08:53:31.971 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4570000000000001e-06 2024-05-08T08:53:31.972 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.972 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.972 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:31.972 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.781114+0000", 2024-05-08T08:53:31.972 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7970000000000001e-06 2024-05-08T08:53:31.972 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.972 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.972 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:31.972 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.781120+0000", 2024-05-08T08:53:31.972 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.784e-06 2024-05-08T08:53:31.972 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.972 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.972 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.972 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.972 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.973 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:18219 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:31.973 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.788645+0000", 2024-05-08T08:53:31.973 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.017418792999999998, 2024-05-08T08:53:31.973 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.020796391000000001, 2024-05-08T08:53:31.973 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:31.973 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:31.973 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:31.973 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:31.973 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:31.973 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:31.973 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 18219 2024-05-08T08:53:31.973 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.973 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:31.973 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.973 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:31.974 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788645+0000", 2024-05-08T08:53:31.974 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.974 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.974 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.974 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:31.974 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788645+0000", 2024-05-08T08:53:31.974 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.974 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.974 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.974 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:31.974 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788690+0000", 2024-05-08T08:53:31.974 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4878e-05 2024-05-08T08:53:31.974 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.974 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.975 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:31.975 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788694+0000", 2024-05-08T08:53:31.975 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5130000000000001e-06 2024-05-08T08:53:31.975 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.975 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.975 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:31.975 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788701+0000", 2024-05-08T08:53:31.975 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5059999999999998e-06 2024-05-08T08:53:31.975 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.975 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.975 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:31.975 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788707+0000", 2024-05-08T08:53:31.975 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9410000000000004e-06 2024-05-08T08:53:31.975 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.975 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.975 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.976 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.976 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.976 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:14826 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:31.976 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.781128+0000", 2024-05-08T08:53:31.976 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.024935767000000001, 2024-05-08T08:53:31.976 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.028373203, 2024-05-08T08:53:31.976 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:31.976 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:31.976 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:31.976 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:31.976 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:31.976 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:31.976 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14826 2024-05-08T08:53:31.976 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.976 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:31.976 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.977 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:31.977 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.781128+0000", 2024-05-08T08:53:31.977 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.977 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.977 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.977 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:31.977 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.781128+0000", 2024-05-08T08:53:31.977 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.977 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.977 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.977 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:31.977 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.781129+0000", 2024-05-08T08:53:31.977 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8299999999999998e-07 2024-05-08T08:53:31.977 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.977 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.978 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:31.978 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.781131+0000", 2024-05-08T08:53:31.978 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7719999999999999e-06 2024-05-08T08:53:31.978 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.978 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.978 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:31.978 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.781138+0000", 2024-05-08T08:53:31.978 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7259999999999996e-06 2024-05-08T08:53:31.978 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.978 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.978 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:31.978 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.781144+0000", 2024-05-08T08:53:31.978 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5799999999999999e-06 2024-05-08T08:53:31.978 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.978 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.979 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.979 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.979 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.979 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:14846 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:31.979 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.785671+0000", 2024-05-08T08:53:31.979 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.020392787999999998, 2024-05-08T08:53:31.979 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.023887403000000001, 2024-05-08T08:53:31.979 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:31.979 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:31.979 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:31.979 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:31.979 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:31.979 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:31.979 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14846 2024-05-08T08:53:31.979 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.979 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:31.979 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.980 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:31.980 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.785671+0000", 2024-05-08T08:53:31.980 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.980 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.980 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.980 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:31.980 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.785671+0000", 2024-05-08T08:53:31.980 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.980 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.980 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.980 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:31.980 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.786139+0000", 2024-05-08T08:53:31.980 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00046803300000000002 2024-05-08T08:53:31.980 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.980 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.980 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:31.980 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.786144+0000", 2024-05-08T08:53:31.981 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2000000000000002e-06 2024-05-08T08:53:31.981 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.981 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.981 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:31.981 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.786153+0000", 2024-05-08T08:53:31.981 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.9709999999999993e-06 2024-05-08T08:53:31.981 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.981 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.981 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:31.981 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.786162+0000", 2024-05-08T08:53:31.981 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.5629999999999992e-06 2024-05-08T08:53:31.981 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.981 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.981 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.981 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.982 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.982 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:18221 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:31.982 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.788717+0000", 2024-05-08T08:53:31.982 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.017346922000000001, 2024-05-08T08:53:31.982 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.020901217999999999, 2024-05-08T08:53:31.982 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:31.982 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:31.982 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:31.982 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:31.982 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:31.982 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:31.982 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 18221 2024-05-08T08:53:31.982 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.982 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:31.982 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.983 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:31.983 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788717+0000", 2024-05-08T08:53:31.983 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.983 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.983 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.983 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:31.983 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788717+0000", 2024-05-08T08:53:31.983 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.983 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.983 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.983 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:31.983 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788777+0000", 2024-05-08T08:53:31.983 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0058e-05 2024-05-08T08:53:31.983 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.983 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.983 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:31.984 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788781+0000", 2024-05-08T08:53:31.984 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9829999999999998e-06 2024-05-08T08:53:31.984 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.984 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.984 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:31.984 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788793+0000", 2024-05-08T08:53:31.984 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1885999999999999e-05 2024-05-08T08:53:31.984 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.984 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.984 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:31.984 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788799+0000", 2024-05-08T08:53:31.984 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.173e-06 2024-05-08T08:53:31.984 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.984 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.984 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.985 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.985 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.985 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:14827 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:31.985 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.781151+0000", 2024-05-08T08:53:31.985 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.024913243000000002, 2024-05-08T08:53:31.985 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.028528811000000001, 2024-05-08T08:53:31.985 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:31.985 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:31.985 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:31.985 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:31.985 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:31.985 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:31.985 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14827 2024-05-08T08:53:31.985 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.985 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:31.985 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.985 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:31.986 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.781151+0000", 2024-05-08T08:53:31.986 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.986 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.986 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.986 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:31.986 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.781151+0000", 2024-05-08T08:53:31.986 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.986 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.986 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.986 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:31.986 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.781151+0000", 2024-05-08T08:53:31.986 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7800000000000002e-07 2024-05-08T08:53:31.986 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.986 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.986 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:31.986 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.781154+0000", 2024-05-08T08:53:31.987 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.616e-06 2024-05-08T08:53:31.987 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.987 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.987 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:31.987 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.781160+0000", 2024-05-08T08:53:31.987 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6209999999999998e-06 2024-05-08T08:53:31.987 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.987 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.987 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:31.987 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.781165+0000", 2024-05-08T08:53:31.987 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9110000000000001e-06 2024-05-08T08:53:31.987 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.987 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.987 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.987 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.987 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.987 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:14847 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:31.988 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.786180+0000", 2024-05-08T08:53:31.988 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.019883805000000001, 2024-05-08T08:53:31.988 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.023558092999999999, 2024-05-08T08:53:31.988 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:31.988 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:31.988 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:31.988 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:31.988 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:31.988 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:31.988 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14847 2024-05-08T08:53:31.988 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.988 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:31.988 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.988 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:31.988 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.786180+0000", 2024-05-08T08:53:31.988 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.989 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.989 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.989 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:31.989 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.786180+0000", 2024-05-08T08:53:31.989 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.989 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.989 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.989 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:31.989 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.786181+0000", 2024-05-08T08:53:31.989 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.92e-07 2024-05-08T08:53:31.989 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.989 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.989 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:31.989 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.786184+0000", 2024-05-08T08:53:31.989 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.427e-06 2024-05-08T08:53:31.989 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.990 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.990 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:31.990 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.786191+0000", 2024-05-08T08:53:31.990 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1239999999999999e-06 2024-05-08T08:53:31.990 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.990 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.990 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:31.990 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.786197+0000", 2024-05-08T08:53:31.990 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0279999999999997e-06 2024-05-08T08:53:31.990 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.990 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.990 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.990 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.990 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.990 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:18222 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:31.990 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.788809+0000", 2024-05-08T08:53:31.991 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.017255117, 2024-05-08T08:53:31.991 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.020988142000000001, 2024-05-08T08:53:31.991 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:31.991 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:31.991 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:31.991 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:31.991 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:31.991 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:31.991 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 18222 2024-05-08T08:53:31.991 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.991 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:31.991 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.991 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:31.991 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788809+0000", 2024-05-08T08:53:31.991 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.991 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.992 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.992 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:31.992 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788809+0000", 2024-05-08T08:53:31.992 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.992 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.992 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.992 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:31.992 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788865+0000", 2024-05-08T08:53:31.992 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5970000000000001e-05 2024-05-08T08:53:31.992 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.992 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.992 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:31.992 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788869+0000", 2024-05-08T08:53:31.992 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.799e-06 2024-05-08T08:53:31.992 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.992 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.993 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:31.993 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788876+0000", 2024-05-08T08:53:31.993 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3119999999999996e-06 2024-05-08T08:53:31.993 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.993 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.993 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:31.993 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788883+0000", 2024-05-08T08:53:31.993 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6819999999999997e-06 2024-05-08T08:53:31.993 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.993 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.993 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.993 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.993 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.993 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:14828 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:31.993 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.781173+0000", 2024-05-08T08:53:31.994 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.024891291999999999, 2024-05-08T08:53:31.994 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.02868389, 2024-05-08T08:53:31.994 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:31.994 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:31.994 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:31.994 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:31.994 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:31.994 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:31.994 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14828 2024-05-08T08:53:31.994 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.994 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:31.994 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.994 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:31.994 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.781173+0000", 2024-05-08T08:53:31.994 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.995 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.995 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.995 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:31.995 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.781173+0000", 2024-05-08T08:53:31.995 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.995 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.995 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.995 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:31.995 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.781173+0000", 2024-05-08T08:53:31.995 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3900000000000005e-07 2024-05-08T08:53:31.995 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.995 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.995 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:31.995 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.781176+0000", 2024-05-08T08:53:31.995 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7839999999999999e-06 2024-05-08T08:53:31.995 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.995 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.996 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:31.996 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.781183+0000", 2024-05-08T08:53:31.996 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.72e-06 2024-05-08T08:53:31.996 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.996 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.996 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:31.996 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.781189+0000", 2024-05-08T08:53:31.996 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8660000000000002e-06 2024-05-08T08:53:31.996 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.996 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.996 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.996 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.996 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.996 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:14848 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:31.996 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.786206+0000", 2024-05-08T08:53:31.996 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.019857824999999999, 2024-05-08T08:53:31.996 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.023707659999999998, 2024-05-08T08:53:31.997 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:31.997 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:31.997 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:31.997 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:31.997 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:31.997 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:31.997 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14848 2024-05-08T08:53:31.997 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.997 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:31.997 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.997 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:31.997 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.786206+0000", 2024-05-08T08:53:31.997 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.997 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.997 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.997 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:31.997 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.786206+0000", 2024-05-08T08:53:31.998 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:31.998 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.998 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.998 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:31.998 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.786207+0000", 2024-05-08T08:53:31.998 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.08e-07 2024-05-08T08:53:31.998 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.998 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.998 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:31.998 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.786209+0000", 2024-05-08T08:53:31.998 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6769999999999999e-06 2024-05-08T08:53:31.998 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.998 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.998 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:31.999 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.786216+0000", 2024-05-08T08:53:31.999 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4779999999999996e-06 2024-05-08T08:53:31.999 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.999 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.999 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:31.999 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.786221+0000", 2024-05-08T08:53:31.999 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1399999999999999e-06 2024-05-08T08:53:31.999 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.999 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:31.999 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:31.999 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:31.999 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:31.999 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:18223 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:31.999 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.788893+0000", 2024-05-08T08:53:31.999 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.017170642999999999, 2024-05-08T08:53:32.000 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.021084619999999998, 2024-05-08T08:53:32.000 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:32.000 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:32.000 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:32.000 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:32.000 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:32.000 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:32.000 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 18223 2024-05-08T08:53:32.000 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.000 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:32.000 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.000 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:32.000 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788893+0000", 2024-05-08T08:53:32.000 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.000 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.000 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.001 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:32.001 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788893+0000", 2024-05-08T08:53:32.001 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.001 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.001 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.001 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:32.001 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788944+0000", 2024-05-08T08:53:32.001 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0921000000000003e-05 2024-05-08T08:53:32.001 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.001 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.001 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:32.001 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788948+0000", 2024-05-08T08:53:32.001 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7000000000000002e-06 2024-05-08T08:53:32.001 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.001 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.001 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:32.001 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788962+0000", 2024-05-08T08:53:32.002 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.4338000000000001e-05 2024-05-08T08:53:32.002 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.002 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.002 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:32.002 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788968+0000", 2024-05-08T08:53:32.002 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.178e-06 2024-05-08T08:53:32.002 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.002 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:32.002 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.002 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.002 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.002 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:14829 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:32.002 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.781196+0000", 2024-05-08T08:53:32.002 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.024867622999999998, 2024-05-08T08:53:32.003 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.028842478000000001, 2024-05-08T08:53:32.003 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:32.003 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:32.003 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:32.003 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:32.003 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:32.003 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:32.003 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14829 2024-05-08T08:53:32.003 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.003 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:32.003 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.003 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:32.003 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.781196+0000", 2024-05-08T08:53:32.003 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.003 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.003 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.004 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:32.004 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.781196+0000", 2024-05-08T08:53:32.004 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.004 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.004 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.004 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:32.004 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.781197+0000", 2024-05-08T08:53:32.004 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.68e-07 2024-05-08T08:53:32.004 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.004 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.004 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:32.004 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.781200+0000", 2024-05-08T08:53:32.004 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7410000000000001e-06 2024-05-08T08:53:32.004 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.004 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.004 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:32.004 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.781206+0000", 2024-05-08T08:53:32.005 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5740000000000001e-06 2024-05-08T08:53:32.005 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.005 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.005 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:32.005 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.781211+0000", 2024-05-08T08:53:32.005 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.215e-06 2024-05-08T08:53:32.005 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.005 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:32.005 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.005 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.005 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.005 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:14849 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:32.005 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.786228+0000", 2024-05-08T08:53:32.005 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.019835800000000001, 2024-05-08T08:53:32.005 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.023868461000000001, 2024-05-08T08:53:32.006 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:32.006 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:32.006 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:32.006 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:32.006 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:32.006 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:32.006 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14849 2024-05-08T08:53:32.006 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.006 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:32.006 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.006 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:32.006 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.786228+0000", 2024-05-08T08:53:32.006 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.006 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.006 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.006 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:32.007 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.786228+0000", 2024-05-08T08:53:32.007 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.007 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.007 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.007 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:32.007 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.786229+0000", 2024-05-08T08:53:32.007 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0499999999999999e-07 2024-05-08T08:53:32.007 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.007 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.007 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:32.007 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.786232+0000", 2024-05-08T08:53:32.007 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9679999999999998e-06 2024-05-08T08:53:32.007 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.007 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.007 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:32.008 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.786240+0000", 2024-05-08T08:53:32.008 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0700000000000007e-06 2024-05-08T08:53:32.008 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.008 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.008 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:32.008 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.786245+0000", 2024-05-08T08:53:32.008 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3090000000000003e-06 2024-05-08T08:53:32.008 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.008 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:32.008 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.008 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.008 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.008 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:18224 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:32.008 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.788979+0000", 2024-05-08T08:53:32.008 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.017084766000000001, 2024-05-08T08:53:32.008 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.021176077000000001, 2024-05-08T08:53:32.009 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:32.009 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:32.009 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:32.009 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:32.009 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:32.009 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:32.009 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 18224 2024-05-08T08:53:32.009 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.009 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:32.009 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.009 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:32.009 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788979+0000", 2024-05-08T08:53:32.009 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.009 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.009 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.010 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:32.010 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.788979+0000", 2024-05-08T08:53:32.010 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.010 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.010 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.010 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:32.010 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789014+0000", 2024-05-08T08:53:32.010 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4588000000000002e-05 2024-05-08T08:53:32.010 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.010 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.010 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:32.010 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789018+0000", 2024-05-08T08:53:32.010 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7699999999999999e-06 2024-05-08T08:53:32.010 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.010 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.010 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:32.011 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789025+0000", 2024-05-08T08:53:32.011 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.17e-06 2024-05-08T08:53:32.011 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.011 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.011 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:32.011 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789031+0000", 2024-05-08T08:53:32.011 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2190000000000001e-06 2024-05-08T08:53:32.011 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.011 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:32.011 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.011 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.011 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.011 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:17858 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:32.011 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.782110+0000", 2024-05-08T08:53:32.011 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.023953575000000001, 2024-05-08T08:53:32.011 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.028104614999999999, 2024-05-08T08:53:32.012 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:32.012 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:32.012 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:32.012 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:32.012 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:32.012 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:32.012 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 17858 2024-05-08T08:53:32.012 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.012 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:32.012 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.012 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:32.012 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.782110+0000", 2024-05-08T08:53:32.012 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.012 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.012 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.013 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:32.013 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.782110+0000", 2024-05-08T08:53:32.013 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.013 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.013 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.013 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:32.013 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.782111+0000", 2024-05-08T08:53:32.013 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4600000000000004e-07 2024-05-08T08:53:32.013 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.013 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.013 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:32.013 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.782115+0000", 2024-05-08T08:53:32.013 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3919999999999996e-06 2024-05-08T08:53:32.013 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.013 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.014 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:32.014 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.782124+0000", 2024-05-08T08:53:32.014 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3289999999999994e-06 2024-05-08T08:53:32.014 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.014 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.014 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:32.014 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.782131+0000", 2024-05-08T08:53:32.014 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.075e-06 2024-05-08T08:53:32.014 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.014 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:32.014 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.014 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.014 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.014 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:14856 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:32.014 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.789471+0000", 2024-05-08T08:53:32.015 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.01659269, 2024-05-08T08:53:32.015 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.020803041000000001, 2024-05-08T08:53:32.015 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:32.015 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:32.015 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:32.015 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:32.015 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:32.015 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:32.015 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14856 2024-05-08T08:53:32.015 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.015 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:32.015 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.015 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:32.015 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789471+0000", 2024-05-08T08:53:32.015 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.015 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.015 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.016 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:32.016 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789471+0000", 2024-05-08T08:53:32.016 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.016 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.016 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.016 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:32.016 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789472+0000", 2024-05-08T08:53:32.016 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.8199999999999998e-07 2024-05-08T08:53:32.016 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.016 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.016 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:32.016 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789477+0000", 2024-05-08T08:53:32.016 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0989999999999998e-06 2024-05-08T08:53:32.016 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.016 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.016 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:32.017 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789499+0000", 2024-05-08T08:53:32.017 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1458999999999999e-05 2024-05-08T08:53:32.017 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.017 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.017 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:32.017 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789507+0000", 2024-05-08T08:53:32.017 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9279999999999993e-06 2024-05-08T08:53:32.017 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.017 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:32.017 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.017 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.017 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.017 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:18225 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:32.017 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.789041+0000", 2024-05-08T08:53:32.017 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.017023234000000002, 2024-05-08T08:53:32.017 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.021293390999999998, 2024-05-08T08:53:32.017 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:32.018 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:32.018 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:32.018 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:32.018 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:32.018 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:32.018 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 18225 2024-05-08T08:53:32.018 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.018 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:32.018 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.018 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:32.018 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789041+0000", 2024-05-08T08:53:32.018 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.018 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.018 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.018 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:32.019 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789041+0000", 2024-05-08T08:53:32.019 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.019 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.019 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.019 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:32.019 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789610+0000", 2024-05-08T08:53:32.019 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00056890200000000004 2024-05-08T08:53:32.019 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.019 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.019 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:32.019 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789615+0000", 2024-05-08T08:53:32.019 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9989999999999999e-06 2024-05-08T08:53:32.019 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.019 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.019 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:32.019 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789625+0000", 2024-05-08T08:53:32.020 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0858e-05 2024-05-08T08:53:32.020 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.020 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.020 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:32.020 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789633+0000", 2024-05-08T08:53:32.020 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.255e-06 2024-05-08T08:53:32.020 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.020 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:32.020 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.020 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.020 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.020 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:14830 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:32.020 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.782228+0000", 2024-05-08T08:53:32.020 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.023835477000000001, 2024-05-08T08:53:32.021 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.028164684999999998, 2024-05-08T08:53:32.021 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:32.021 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:32.021 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:32.021 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:32.021 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:32.021 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:32.021 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14830 2024-05-08T08:53:32.021 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.021 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:32.021 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.021 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:32.021 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.782228+0000", 2024-05-08T08:53:32.021 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.022 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.022 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.022 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:32.022 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.782228+0000", 2024-05-08T08:53:32.022 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.022 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.022 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.022 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:32.022 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.782229+0000", 2024-05-08T08:53:32.022 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.92e-07 2024-05-08T08:53:32.022 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.022 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.022 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:32.022 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.782239+0000", 2024-05-08T08:53:32.022 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6949999999999995e-06 2024-05-08T08:53:32.022 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.023 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.023 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:32.023 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.782249+0000", 2024-05-08T08:53:32.023 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8230000000000006e-06 2024-05-08T08:53:32.023 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.023 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.023 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:32.023 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.782255+0000", 2024-05-08T08:53:32.023 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5459999999999997e-06 2024-05-08T08:53:32.023 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.023 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:32.023 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.023 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.023 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.023 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:18226 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:32.023 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.789645+0000", 2024-05-08T08:53:32.023 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.016419324999999999, 2024-05-08T08:53:32.024 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.020807170999999999, 2024-05-08T08:53:32.024 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:32.024 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:32.024 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:32.024 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:32.024 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:32.024 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:32.024 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 18226 2024-05-08T08:53:32.024 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.024 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:32.024 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.024 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:32.024 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789645+0000", 2024-05-08T08:53:32.024 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.024 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.025 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.025 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:32.025 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789645+0000", 2024-05-08T08:53:32.025 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.025 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.025 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.025 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:32.025 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789645+0000", 2024-05-08T08:53:32.025 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6499999999999999e-07 2024-05-08T08:53:32.025 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.025 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.025 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:32.025 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789648+0000", 2024-05-08T08:53:32.026 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.2490000000000001e-06 2024-05-08T08:53:32.026 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.026 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.026 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:32.026 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789657+0000", 2024-05-08T08:53:32.026 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.6500000000000002e-06 2024-05-08T08:53:32.026 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.026 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.026 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:32.026 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789663+0000", 2024-05-08T08:53:32.026 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.57e-06 2024-05-08T08:53:32.026 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.026 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:32.026 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.026 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.026 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.027 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:14831 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:32.027 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.782277+0000", 2024-05-08T08:53:32.027 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.023787341, 2024-05-08T08:53:32.027 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.028318014999999998, 2024-05-08T08:53:32.027 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:32.027 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:32.027 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:32.027 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:32.027 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:32.027 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:32.027 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14831 2024-05-08T08:53:32.027 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.027 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:32.027 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.027 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:32.027 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.782277+0000", 2024-05-08T08:53:32.027 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.028 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.028 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.028 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:32.028 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.782277+0000", 2024-05-08T08:53:32.028 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.028 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.028 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.028 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:32.028 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.782277+0000", 2024-05-08T08:53:32.028 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7299999999999996e-07 2024-05-08T08:53:32.028 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.028 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.028 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:32.028 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.782281+0000", 2024-05-08T08:53:32.028 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4630000000000001e-06 2024-05-08T08:53:32.028 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.029 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.029 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:32.029 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.782288+0000", 2024-05-08T08:53:32.029 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5490000000000001e-06 2024-05-08T08:53:32.029 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.029 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.029 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:32.029 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.782299+0000", 2024-05-08T08:53:32.029 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0937e-05 2024-05-08T08:53:32.029 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.029 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:32.029 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.029 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.029 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.029 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:18227 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:32.029 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.789671+0000", 2024-05-08T08:53:32.030 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.016392509999999999, 2024-05-08T08:53:32.030 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.020981983999999999, 2024-05-08T08:53:32.030 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:32.030 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:32.030 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:32.030 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:32.030 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:32.030 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:32.030 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 18227 2024-05-08T08:53:32.030 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.030 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:32.030 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.030 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:32.030 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789671+0000", 2024-05-08T08:53:32.031 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.031 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.031 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.031 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:32.031 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789671+0000", 2024-05-08T08:53:32.031 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.031 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.031 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.031 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:32.031 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789672+0000", 2024-05-08T08:53:32.031 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9000000000000002e-07 2024-05-08T08:53:32.031 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.031 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.031 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:32.031 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789693+0000", 2024-05-08T08:53:32.031 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1498999999999999e-05 2024-05-08T08:53:32.032 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.032 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.032 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:32.032 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789702+0000", 2024-05-08T08:53:32.032 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.8259999999999999e-06 2024-05-08T08:53:32.032 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.032 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.032 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:32.032 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789708+0000", 2024-05-08T08:53:32.032 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0820000000000004e-06 2024-05-08T08:53:32.032 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.032 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:32.032 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.032 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.032 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.032 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:18444 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:32.032 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.795986+0000", 2024-05-08T08:53:32.033 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.010078283, 2024-05-08T08:53:32.033 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.014729209, 2024-05-08T08:53:32.033 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:32.033 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:32.033 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:32.033 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:32.033 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:32.033 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:32.033 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 18444 2024-05-08T08:53:32.033 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.033 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:32.033 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.033 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:32.033 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.795986+0000", 2024-05-08T08:53:32.034 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.034 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.034 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.034 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:32.034 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.795986+0000", 2024-05-08T08:53:32.034 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.034 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.034 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.034 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:32.034 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.795986+0000", 2024-05-08T08:53:32.034 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2600000000000002e-07 2024-05-08T08:53:32.034 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.034 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.034 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:32.035 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.795990+0000", 2024-05-08T08:53:32.035 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3230000000000002e-06 2024-05-08T08:53:32.035 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.035 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.035 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:32.035 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.795997+0000", 2024-05-08T08:53:32.035 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1169999999999998e-06 2024-05-08T08:53:32.035 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.035 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.035 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:32.035 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.796002+0000", 2024-05-08T08:53:32.035 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9590000000000003e-06 2024-05-08T08:53:32.035 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.035 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:32.035 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.035 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.036 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.036 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:14832 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:32.036 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.782315+0000", 2024-05-08T08:53:32.036 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.023748726000000001, 2024-05-08T08:53:32.036 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.028458673, 2024-05-08T08:53:32.036 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:32.036 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:32.036 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:32.036 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:32.036 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:32.036 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:32.036 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14832 2024-05-08T08:53:32.036 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.036 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:32.036 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.036 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:32.036 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.782315+0000", 2024-05-08T08:53:32.037 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.037 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.037 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.037 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:32.037 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.782315+0000", 2024-05-08T08:53:32.037 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.037 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.037 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.037 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:32.037 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.782316+0000", 2024-05-08T08:53:32.037 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2399999999999998e-07 2024-05-08T08:53:32.037 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.037 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.037 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:32.037 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.782319+0000", 2024-05-08T08:53:32.037 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3809999999999999e-06 2024-05-08T08:53:32.037 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.038 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.038 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:32.038 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.782331+0000", 2024-05-08T08:53:32.038 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1783999999999999e-05 2024-05-08T08:53:32.038 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.038 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.038 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:32.038 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.782336+0000", 2024-05-08T08:53:32.038 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4090000000000002e-06 2024-05-08T08:53:32.038 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.038 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:32.038 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.038 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.038 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.038 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:14857 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:32.038 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.789702+0000", 2024-05-08T08:53:32.039 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.016362003, 2024-05-08T08:53:32.039 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.021135656999999999, 2024-05-08T08:53:32.039 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:32.039 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:32.039 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:32.039 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:32.039 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:32.039 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:32.039 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14857 2024-05-08T08:53:32.039 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.039 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:32.039 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.039 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:32.039 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789702+0000", 2024-05-08T08:53:32.039 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.039 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.040 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.040 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:32.040 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789702+0000", 2024-05-08T08:53:32.040 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.040 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.040 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.040 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:32.040 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789703+0000", 2024-05-08T08:53:32.040 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2099999999999996e-07 2024-05-08T08:53:32.040 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.040 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.040 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:32.040 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789707+0000", 2024-05-08T08:53:32.040 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7759999999999997e-06 2024-05-08T08:53:32.040 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.041 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.041 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:32.041 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789716+0000", 2024-05-08T08:53:32.041 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7020000000000007e-06 2024-05-08T08:53:32.041 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.041 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.041 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:32.041 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789723+0000", 2024-05-08T08:53:32.041 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6880000000000002e-06 2024-05-08T08:53:32.041 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.041 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:32.041 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.041 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.041 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.041 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:18446 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:32.041 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.796868+0000", 2024-05-08T08:53:32.042 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0091955170000000003, 2024-05-08T08:53:32.042 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.014027473, 2024-05-08T08:53:32.042 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:32.042 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:32.042 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:32.042 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:32.042 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:32.042 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:32.042 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 18446 2024-05-08T08:53:32.042 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.042 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:32.042 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.042 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:32.042 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.796868+0000", 2024-05-08T08:53:32.042 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.043 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.043 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.043 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:32.043 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.796868+0000", 2024-05-08T08:53:32.043 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.043 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.043 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.043 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:32.043 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.796869+0000", 2024-05-08T08:53:32.043 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2900000000000003e-07 2024-05-08T08:53:32.043 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.043 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.043 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:32.043 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.796874+0000", 2024-05-08T08:53:32.043 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5850000000000001e-06 2024-05-08T08:53:32.044 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.044 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.044 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:32.044 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.796881+0000", 2024-05-08T08:53:32.044 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1849999999999998e-06 2024-05-08T08:53:32.044 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.044 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.044 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:32.044 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.796887+0000", 2024-05-08T08:53:32.044 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7749999999999998e-06 2024-05-08T08:53:32.044 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.044 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:32.044 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.044 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.044 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.044 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:14833 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:32.044 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.782345+0000", 2024-05-08T08:53:32.045 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.023718917999999999, 2024-05-08T08:53:32.045 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.028621864, 2024-05-08T08:53:32.045 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:32.045 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:32.045 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:32.045 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:32.045 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:32.045 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:32.045 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14833 2024-05-08T08:53:32.045 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.045 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:32.045 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.045 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:32.045 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.782345+0000", 2024-05-08T08:53:32.045 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.045 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.046 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.046 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:32.046 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.782345+0000", 2024-05-08T08:53:32.046 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.046 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.046 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.046 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:32.046 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.782345+0000", 2024-05-08T08:53:32.046 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8200000000000001e-07 2024-05-08T08:53:32.046 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.046 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.046 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:32.046 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.782348+0000", 2024-05-08T08:53:32.046 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.897e-06 2024-05-08T08:53:32.046 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.047 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.047 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:32.047 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.782355+0000", 2024-05-08T08:53:32.047 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3180000000000003e-06 2024-05-08T08:53:32.047 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.047 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.047 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:32.047 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.782359+0000", 2024-05-08T08:53:32.047 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8389999999999997e-06 2024-05-08T08:53:32.047 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.047 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:32.047 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.047 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.047 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.047 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:14850 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:32.047 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.786252+0000", 2024-05-08T08:53:32.047 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.019811878000000002, 2024-05-08T08:53:32.048 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.024774032000000001, 2024-05-08T08:53:32.048 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:32.048 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:32.048 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:32.048 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:32.048 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:32.048 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:32.048 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14850 2024-05-08T08:53:32.048 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.048 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:32.048 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.048 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:32.048 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.786252+0000", 2024-05-08T08:53:32.048 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.048 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.049 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.049 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:32.049 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.786252+0000", 2024-05-08T08:53:32.049 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.049 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.049 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.049 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:32.049 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.786960+0000", 2024-05-08T08:53:32.049 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000707802 2024-05-08T08:53:32.049 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.049 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.049 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:32.049 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.786965+0000", 2024-05-08T08:53:32.049 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8910000000000003e-06 2024-05-08T08:53:32.049 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.049 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.050 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:32.050 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.786974+0000", 2024-05-08T08:53:32.050 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3139999999999993e-06 2024-05-08T08:53:32.050 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.050 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.050 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:32.050 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.786981+0000", 2024-05-08T08:53:32.050 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8970000000000002e-06 2024-05-08T08:53:32.050 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.050 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:32.050 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.050 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.050 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.050 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:18228 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:32.051 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.789739+0000", 2024-05-08T08:53:32.051 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.016324614000000001, 2024-05-08T08:53:32.051 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.021348441999999999, 2024-05-08T08:53:32.051 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:32.051 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:32.051 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:32.051 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:32.051 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:32.051 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:32.051 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 18228 2024-05-08T08:53:32.051 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.051 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:32.051 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.051 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:32.051 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789739+0000", 2024-05-08T08:53:32.051 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.052 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.052 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.052 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:32.052 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789739+0000", 2024-05-08T08:53:32.052 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.052 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.052 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.052 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:32.052 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789740+0000", 2024-05-08T08:53:32.052 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0999999999999998e-07 2024-05-08T08:53:32.052 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.052 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.052 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:32.052 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789743+0000", 2024-05-08T08:53:32.052 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.2279999999999998e-06 2024-05-08T08:53:32.053 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.053 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.053 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:32.053 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789752+0000", 2024-05-08T08:53:32.053 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4489999999999999e-06 2024-05-08T08:53:32.053 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.053 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.053 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:32.053 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789758+0000", 2024-05-08T08:53:32.053 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8139999999999997e-06 2024-05-08T08:53:32.053 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.053 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:32.053 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.053 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.053 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.054 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:14858 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:32.054 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.792505+0000", 2024-05-08T08:53:32.054 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.013558733, 2024-05-08T08:53:32.054 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.018641185000000001, 2024-05-08T08:53:32.054 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:32.054 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:32.054 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:32.054 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:32.054 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:32.054 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:32.054 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14858 2024-05-08T08:53:32.054 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.054 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:32.054 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.055 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:32.055 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.792505+0000", 2024-05-08T08:53:32.055 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.055 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.055 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.055 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:32.055 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.792505+0000", 2024-05-08T08:53:32.055 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.055 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.055 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.055 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:32.055 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.792506+0000", 2024-05-08T08:53:32.055 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0999999999999998e-07 2024-05-08T08:53:32.055 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.055 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.056 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:32.056 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.792510+0000", 2024-05-08T08:53:32.056 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9310000000000001e-06 2024-05-08T08:53:32.056 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.056 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.056 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:32.056 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.792518+0000", 2024-05-08T08:53:32.056 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0269999999999995e-06 2024-05-08T08:53:32.056 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.056 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.056 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:32.056 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.792523+0000", 2024-05-08T08:53:32.056 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0220000000000004e-06 2024-05-08T08:53:32.056 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.056 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:32.056 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.057 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.057 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.057 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:18229 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:32.057 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.789767+0000", 2024-05-08T08:53:32.057 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.016297262999999999, 2024-05-08T08:53:32.057 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.021443045000000001, 2024-05-08T08:53:32.057 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:32.057 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:32.057 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:32.057 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:32.057 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:32.057 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:32.057 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 18229 2024-05-08T08:53:32.057 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.057 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:32.058 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.058 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:32.058 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789767+0000", 2024-05-08T08:53:32.058 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.058 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.058 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.058 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:32.058 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789767+0000", 2024-05-08T08:53:32.058 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.058 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.058 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.058 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:32.058 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789767+0000", 2024-05-08T08:53:32.058 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4000000000000001e-07 2024-05-08T08:53:32.058 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.058 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.059 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:32.059 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789770+0000", 2024-05-08T08:53:32.059 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9110000000000002e-06 2024-05-08T08:53:32.059 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.059 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.059 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:32.059 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789778+0000", 2024-05-08T08:53:32.059 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2749999999999998e-06 2024-05-08T08:53:32.059 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.059 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.059 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:32.059 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789789+0000", 2024-05-08T08:53:32.059 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1411e-05 2024-05-08T08:53:32.059 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.059 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:32.059 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.059 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.059 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.060 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:18449 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:32.060 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.802250+0000", 2024-05-08T08:53:32.060 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0038143249999999999, 2024-05-08T08:53:32.060 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.009018847, 2024-05-08T08:53:32.060 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:32.060 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:32.060 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:32.060 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:32.060 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:32.060 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:32.060 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 18449 2024-05-08T08:53:32.060 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.060 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:32.060 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.060 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:32.060 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.802250+0000", 2024-05-08T08:53:32.061 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.061 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.061 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.061 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:32.061 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.802250+0000", 2024-05-08T08:53:32.061 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.061 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.061 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.061 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:32.061 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.802286+0000", 2024-05-08T08:53:32.061 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6106999999999998e-05 2024-05-08T08:53:32.061 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.061 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.061 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:32.061 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.802289+0000", 2024-05-08T08:53:32.061 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1779999999999999e-06 2024-05-08T08:53:32.062 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.062 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.062 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:32.062 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.802295+0000", 2024-05-08T08:53:32.062 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5740000000000003e-06 2024-05-08T08:53:32.062 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.062 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.062 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:32.062 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.802298+0000", 2024-05-08T08:53:32.062 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8750000000000002e-06 2024-05-08T08:53:32.062 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.062 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:32.062 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.062 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.063 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.063 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:14834 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:32.063 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.782368+0000", 2024-05-08T08:53:32.063 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.023696139000000001, 2024-05-08T08:53:32.063 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.028969545999999999, 2024-05-08T08:53:32.063 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:32.063 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:32.063 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "reached pg", 2024-05-08T08:53:32.063 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:32.063 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:32.063 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:32.063 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14834 2024-05-08T08:53:32.063 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.063 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:32.063 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.064 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:32.064 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.782368+0000", 2024-05-08T08:53:32.064 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.064 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.064 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.064 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:32.064 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.782368+0000", 2024-05-08T08:53:32.064 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.064 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.064 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.064 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:32.064 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.782368+0000", 2024-05-08T08:53:32.064 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.96e-07 2024-05-08T08:53:32.064 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.065 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.065 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:32.065 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.782381+0000", 2024-05-08T08:53:32.065 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3200999999999999e-05 2024-05-08T08:53:32.065 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.065 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.065 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:32.065 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.782388+0000", 2024-05-08T08:53:32.065 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6089999999999997e-06 2024-05-08T08:53:32.065 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.065 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.065 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:32.065 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.782395+0000", 2024-05-08T08:53:32.065 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0600000000000002e-06 2024-05-08T08:53:32.065 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.065 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.066 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:32.066 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.811316+0000", 2024-05-08T08:53:32.066 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.028920663999999999 2024-05-08T08:53:32.066 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.066 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:32.066 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.066 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.066 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.066 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:14851 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:32.066 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.786993+0000", 2024-05-08T08:53:32.066 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.019071314999999998, 2024-05-08T08:53:32.066 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.024393405999999999, 2024-05-08T08:53:32.066 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:32.066 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:32.067 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:32.067 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:32.067 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:32.067 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:32.067 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14851 2024-05-08T08:53:32.067 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.067 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:32.067 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.067 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:32.067 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.786993+0000", 2024-05-08T08:53:32.067 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.067 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.067 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.067 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:32.067 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.786993+0000", 2024-05-08T08:53:32.068 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.068 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.068 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.068 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:32.068 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.787719+0000", 2024-05-08T08:53:32.068 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00072622599999999995 2024-05-08T08:53:32.068 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.068 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.068 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:32.068 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.787725+0000", 2024-05-08T08:53:32.068 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6749999999999999e-06 2024-05-08T08:53:32.068 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.068 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.068 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:32.068 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.787741+0000", 2024-05-08T08:53:32.069 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6209999999999999e-05 2024-05-08T08:53:32.069 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.069 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.069 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:32.069 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.787747+0000", 2024-05-08T08:53:32.069 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7349999999999999e-06 2024-05-08T08:53:32.069 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.069 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:32.069 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.069 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.069 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.069 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:18230 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:32.069 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.789798+0000", 2024-05-08T08:53:32.069 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.016266393000000001, 2024-05-08T08:53:32.069 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.02165019, 2024-05-08T08:53:32.069 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:32.070 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:32.070 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:32.070 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:32.070 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:32.070 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:32.070 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 18230 2024-05-08T08:53:32.070 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.070 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:32.070 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.070 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:32.070 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789798+0000", 2024-05-08T08:53:32.070 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.070 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.070 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.071 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:32.071 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789798+0000", 2024-05-08T08:53:32.071 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.071 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.071 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.071 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:32.071 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789798+0000", 2024-05-08T08:53:32.071 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1799999999999995e-07 2024-05-08T08:53:32.071 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.071 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.071 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:32.071 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789801+0000", 2024-05-08T08:53:32.071 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0280000000000001e-06 2024-05-08T08:53:32.071 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.071 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.072 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:32.072 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789808+0000", 2024-05-08T08:53:32.072 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9999999999999999e-06 2024-05-08T08:53:32.072 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.072 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.072 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:32.072 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789814+0000", 2024-05-08T08:53:32.072 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5060000000000003e-06 2024-05-08T08:53:32.072 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.072 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:32.072 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.072 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.072 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.072 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:18339 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:32.072 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.793454+0000", 2024-05-08T08:53:32.072 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.012609947999999999, 2024-05-08T08:53:32.073 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.018054001, 2024-05-08T08:53:32.073 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:32.073 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:32.073 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:32.073 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:32.073 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:32.073 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:32.073 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 18339 2024-05-08T08:53:32.073 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.073 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:32.073 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.073 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:32.073 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.793454+0000", 2024-05-08T08:53:32.073 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.073 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.074 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.074 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:32.074 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.793454+0000", 2024-05-08T08:53:32.074 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.074 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.074 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.074 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:32.074 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.793455+0000", 2024-05-08T08:53:32.074 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4000000000000001e-07 2024-05-08T08:53:32.074 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.074 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.074 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:32.074 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.793506+0000", 2024-05-08T08:53:32.074 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1381999999999997e-05 2024-05-08T08:53:32.074 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.075 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.075 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:32.075 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.793542+0000", 2024-05-08T08:53:32.075 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5571000000000002e-05 2024-05-08T08:53:32.075 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.075 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.075 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:32.075 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.793589+0000", 2024-05-08T08:53:32.075 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7024000000000002e-05 2024-05-08T08:53:32.075 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.075 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:32.075 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.075 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.075 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.075 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:18231 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:32.076 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.789821+0000", 2024-05-08T08:53:32.076 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.016242998000000002, 2024-05-08T08:53:32.076 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.021751184, 2024-05-08T08:53:32.076 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:32.076 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:32.076 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:32.076 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:32.076 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:32.076 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:32.076 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 18231 2024-05-08T08:53:32.076 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.076 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:32.076 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.076 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:32.076 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789821+0000", 2024-05-08T08:53:32.077 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.077 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.077 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.077 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:32.077 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789821+0000", 2024-05-08T08:53:32.077 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.077 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.077 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.077 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:32.077 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789821+0000", 2024-05-08T08:53:32.077 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1100000000000001e-07 2024-05-08T08:53:32.077 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.077 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.077 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:32.077 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789824+0000", 2024-05-08T08:53:32.078 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7750000000000001e-06 2024-05-08T08:53:32.078 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.078 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.078 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:32.078 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789831+0000", 2024-05-08T08:53:32.078 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3629999999999999e-06 2024-05-08T08:53:32.078 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.078 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.078 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:32.078 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789836+0000", 2024-05-08T08:53:32.078 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0760000000000002e-06 2024-05-08T08:53:32.078 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.078 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:32.078 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.078 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.079 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.079 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:18475 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:32.079 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.803991+0000", 2024-05-08T08:53:32.079 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0020734579999999998, 2024-05-08T08:53:32.079 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0076381599999999997, 2024-05-08T08:53:32.079 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:32.079 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:32.079 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:32.079 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:32.079 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:32.079 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:32.079 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 18475 2024-05-08T08:53:32.079 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.079 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:32.079 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.079 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:32.080 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.803991+0000", 2024-05-08T08:53:32.080 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.080 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.080 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.080 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:32.080 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.803991+0000", 2024-05-08T08:53:32.080 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.080 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.080 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.080 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:32.080 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.803991+0000", 2024-05-08T08:53:32.080 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.2099999999999995e-07 2024-05-08T08:53:32.080 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.080 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.080 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:32.081 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.803996+0000", 2024-05-08T08:53:32.081 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6709999999999998e-06 2024-05-08T08:53:32.081 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.081 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.081 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:32.081 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.804008+0000", 2024-05-08T08:53:32.081 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1617000000000001e-05 2024-05-08T08:53:32.081 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.081 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.081 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:32.081 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.804017+0000", 2024-05-08T08:53:32.081 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0589999999999992e-06 2024-05-08T08:53:32.081 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.081 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:32.081 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.082 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.082 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.082 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:18232 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:32.082 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.789842+0000", 2024-05-08T08:53:32.082 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.016221533, 2024-05-08T08:53:32.082 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.021845686, 2024-05-08T08:53:32.082 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:32.082 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:32.082 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:32.082 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:32.082 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:32.082 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:32.082 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 18232 2024-05-08T08:53:32.082 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.083 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:32.083 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.083 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:32.083 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789842+0000", 2024-05-08T08:53:32.083 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.083 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.083 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.083 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:32.083 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789842+0000", 2024-05-08T08:53:32.083 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.083 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.083 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.083 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:32.083 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789843+0000", 2024-05-08T08:53:32.083 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.08e-07 2024-05-08T08:53:32.084 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.084 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.084 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:32.084 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789845+0000", 2024-05-08T08:53:32.084 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6400000000000001e-06 2024-05-08T08:53:32.084 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.084 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.084 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:32.084 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789852+0000", 2024-05-08T08:53:32.084 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5309999999999998e-06 2024-05-08T08:53:32.084 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.084 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.084 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:32.084 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789857+0000", 2024-05-08T08:53:32.084 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2070000000000003e-06 2024-05-08T08:53:32.084 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.085 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:32.085 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.085 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.085 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.085 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:14859 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:32.085 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.804032+0000", 2024-05-08T08:53:32.085 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.002032065, 2024-05-08T08:53:32.085 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0077165610000000003, 2024-05-08T08:53:32.085 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:32.085 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:32.085 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:32.085 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:32.085 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:32.085 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:32.085 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14859 2024-05-08T08:53:32.086 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.086 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:32.086 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.086 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:32.086 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.804032+0000", 2024-05-08T08:53:32.086 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.086 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.086 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.086 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:32.086 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.804032+0000", 2024-05-08T08:53:32.086 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.086 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.086 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.086 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:32.087 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.804033+0000", 2024-05-08T08:53:32.087 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.78e-07 2024-05-08T08:53:32.087 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.087 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.087 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:32.087 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.804037+0000", 2024-05-08T08:53:32.087 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2370000000000003e-06 2024-05-08T08:53:32.087 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.087 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.087 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:32.087 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.804045+0000", 2024-05-08T08:53:32.087 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5410000000000003e-06 2024-05-08T08:53:32.087 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.087 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.087 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:32.088 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.804050+0000", 2024-05-08T08:53:32.088 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3770000000000004e-06 2024-05-08T08:53:32.088 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.088 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:32.088 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.088 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.088 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.088 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:14852 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:32.088 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.787770+0000", 2024-05-08T08:53:32.088 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.018294451999999999, 2024-05-08T08:53:32.088 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.024039337000000001, 2024-05-08T08:53:32.088 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:32.088 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:32.088 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:32.088 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:32.088 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:32.089 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:32.089 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14852 2024-05-08T08:53:32.089 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.089 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:32.089 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.089 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:32.089 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.787770+0000", 2024-05-08T08:53:32.089 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.089 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.089 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.089 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:32.089 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.787770+0000", 2024-05-08T08:53:32.089 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.089 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.089 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.090 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:32.090 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.787770+0000", 2024-05-08T08:53:32.090 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.54e-07 2024-05-08T08:53:32.090 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.090 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.090 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:32.090 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.787774+0000", 2024-05-08T08:53:32.090 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5439999999999999e-06 2024-05-08T08:53:32.090 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.090 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.090 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:32.090 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.787782+0000", 2024-05-08T08:53:32.090 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.5609999999999999e-06 2024-05-08T08:53:32.090 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.091 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.091 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:32.091 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.787789+0000", 2024-05-08T08:53:32.091 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1090000000000003e-06 2024-05-08T08:53:32.091 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.091 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:32.091 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.091 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.091 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.091 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:18233 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:32.091 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.789864+0000", 2024-05-08T08:53:32.091 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.016200071999999999, 2024-05-08T08:53:32.091 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.022005618000000001, 2024-05-08T08:53:32.091 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:32.091 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:32.092 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:32.092 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:32.092 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:32.092 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:32.092 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 18233 2024-05-08T08:53:32.092 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.092 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:32.092 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.092 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:32.092 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789864+0000", 2024-05-08T08:53:32.092 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.092 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.092 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.092 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:32.092 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789864+0000", 2024-05-08T08:53:32.093 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.093 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.093 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.093 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:32.093 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789864+0000", 2024-05-08T08:53:32.093 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2300000000000002e-07 2024-05-08T08:53:32.093 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.093 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.093 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:32.093 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789867+0000", 2024-05-08T08:53:32.093 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7140000000000002e-06 2024-05-08T08:53:32.093 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.093 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.093 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:32.093 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789873+0000", 2024-05-08T08:53:32.094 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0920000000000003e-06 2024-05-08T08:53:32.094 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.094 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.094 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:32.094 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789878+0000", 2024-05-08T08:53:32.094 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0309999999999998e-06 2024-05-08T08:53:32.094 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.094 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:32.094 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.094 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.094 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.094 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:18029 3.0 3.b56c42a0 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:32.094 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.784171+0000", 2024-05-08T08:53:32.094 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.021893102000000001, 2024-05-08T08:53:32.094 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.027758853999999999, 2024-05-08T08:53:32.095 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:32.095 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:32.095 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:32.095 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:32.095 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:32.095 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:32.095 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 18029 2024-05-08T08:53:32.095 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.095 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:32.095 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.095 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:32.095 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.784171+0000", 2024-05-08T08:53:32.095 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.095 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.095 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.095 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:32.096 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.784171+0000", 2024-05-08T08:53:32.096 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.096 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.096 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.096 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:32.096 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.784171+0000", 2024-05-08T08:53:32.096 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.72e-07 2024-05-08T08:53:32.096 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.096 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.096 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:32.096 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.784174+0000", 2024-05-08T08:53:32.096 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2709999999999999e-06 2024-05-08T08:53:32.096 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.096 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.096 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:32.097 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.784180+0000", 2024-05-08T08:53:32.097 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3550000000000001e-06 2024-05-08T08:53:32.097 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.097 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.097 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:32.097 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.784186+0000", 2024-05-08T08:53:32.097 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5939999999999999e-06 2024-05-08T08:53:32.097 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.097 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:32.097 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.097 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.097 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.097 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:18234 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:32.097 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.789885+0000", 2024-05-08T08:53:32.097 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.016179083, 2024-05-08T08:53:32.097 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.022100809999999999, 2024-05-08T08:53:32.097 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:32.098 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:32.098 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:32.098 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:32.098 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:32.098 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:32.098 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 18234 2024-05-08T08:53:32.098 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.098 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:32.098 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.098 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:32.098 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789885+0000", 2024-05-08T08:53:32.098 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.098 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.098 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.098 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:32.099 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789885+0000", 2024-05-08T08:53:32.099 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.099 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.099 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.099 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:32.099 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789885+0000", 2024-05-08T08:53:32.099 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:53:32.099 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.099 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.099 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:32.099 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789888+0000", 2024-05-08T08:53:32.099 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6450000000000001e-06 2024-05-08T08:53:32.099 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.099 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.099 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:32.099 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789896+0000", 2024-05-08T08:53:32.100 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.6939999999999995e-06 2024-05-08T08:53:32.100 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.100 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.100 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:32.100 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789901+0000", 2024-05-08T08:53:32.100 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1959999999999999e-06 2024-05-08T08:53:32.100 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.100 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:32.100 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.100 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.100 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.100 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:18207 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:32.100 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.787798+0000", 2024-05-08T08:53:32.100 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.018265490999999998, 2024-05-08T08:53:32.100 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.024244205000000001, 2024-05-08T08:53:32.100 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:32.100 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:32.101 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:32.101 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:32.101 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:32.101 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:32.101 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 18207 2024-05-08T08:53:32.101 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.101 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:32.101 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.101 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:32.101 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.787798+0000", 2024-05-08T08:53:32.101 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.101 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.101 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.101 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:32.101 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.787798+0000", 2024-05-08T08:53:32.102 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.102 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.102 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.102 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:32.102 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.787799+0000", 2024-05-08T08:53:32.102 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2200000000000004e-07 2024-05-08T08:53:32.102 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.102 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.102 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:32.102 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.787802+0000", 2024-05-08T08:53:32.102 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9950000000000001e-06 2024-05-08T08:53:32.102 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.102 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.102 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:32.102 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.787808+0000", 2024-05-08T08:53:32.103 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3690000000000003e-06 2024-05-08T08:53:32.103 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.103 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.103 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:32.103 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.787814+0000", 2024-05-08T08:53:32.103 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6160000000000001e-06 2024-05-08T08:53:32.103 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.103 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:32.103 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.103 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.103 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.103 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:18235 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:32.103 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.789980+0000", 2024-05-08T08:53:32.103 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.016083957999999999, 2024-05-08T08:53:32.103 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.022123710000000001, 2024-05-08T08:53:32.104 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:32.104 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:32.104 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:32.104 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:32.104 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:32.104 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:32.104 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 18235 2024-05-08T08:53:32.104 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.104 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:32.104 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.104 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:32.104 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789980+0000", 2024-05-08T08:53:32.104 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.104 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.104 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.104 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:32.105 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789980+0000", 2024-05-08T08:53:32.105 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.105 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.105 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.105 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:32.105 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789981+0000", 2024-05-08T08:53:32.105 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.13e-07 2024-05-08T08:53:32.105 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.105 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.105 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:32.105 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789985+0000", 2024-05-08T08:53:32.105 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8820000000000003e-06 2024-05-08T08:53:32.105 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.105 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.105 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:32.106 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.789994+0000", 2024-05-08T08:53:32.106 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.713e-06 2024-05-08T08:53:32.106 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.106 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.106 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:32.106 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.790002+0000", 2024-05-08T08:53:32.106 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4900000000000003e-06 2024-05-08T08:53:32.106 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.106 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:32.106 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.106 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.106 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.106 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:14853 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:32.106 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.787810+0000", 2024-05-08T08:53:32.106 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.018254158999999999, 2024-05-08T08:53:32.107 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.024354126, 2024-05-08T08:53:32.107 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:32.107 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:32.107 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:32.107 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:32.107 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:32.107 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:32.107 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14853 2024-05-08T08:53:32.107 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.107 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:32.107 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.107 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:32.107 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.787810+0000", 2024-05-08T08:53:32.107 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.107 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.108 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.108 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:32.108 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.787810+0000", 2024-05-08T08:53:32.108 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.108 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.108 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.108 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:32.108 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.787811+0000", 2024-05-08T08:53:32.108 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0999999999999998e-07 2024-05-08T08:53:32.108 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.108 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.108 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:32.108 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.787814+0000", 2024-05-08T08:53:32.108 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.2289999999999999e-06 2024-05-08T08:53:32.108 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.109 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.109 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:32.109 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.787821+0000", 2024-05-08T08:53:32.109 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5900000000000002e-06 2024-05-08T08:53:32.109 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.109 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.109 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:32.109 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.787827+0000", 2024-05-08T08:53:32.109 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.378e-06 2024-05-08T08:53:32.109 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.109 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:32.109 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.109 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.109 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.109 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:18236 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:32.109 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.790056+0000", 2024-05-08T08:53:32.110 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.016008379, 2024-05-08T08:53:32.110 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.022169524999999999, 2024-05-08T08:53:32.110 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:32.110 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:32.110 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:32.110 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:32.110 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:32.110 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:32.110 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 18236 2024-05-08T08:53:32.110 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.110 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:32.110 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.110 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:32.110 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.790056+0000", 2024-05-08T08:53:32.110 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.111 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.111 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.111 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:32.111 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.790056+0000", 2024-05-08T08:53:32.111 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.111 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.111 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.111 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:32.111 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.790056+0000", 2024-05-08T08:53:32.111 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7999999999999995e-07 2024-05-08T08:53:32.111 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.111 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.111 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:32.111 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.790060+0000", 2024-05-08T08:53:32.111 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1740000000000002e-06 2024-05-08T08:53:32.112 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.112 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.112 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:32.112 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.790069+0000", 2024-05-08T08:53:32.112 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.8659999999999995e-06 2024-05-08T08:53:32.112 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.112 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.112 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:32.112 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.790076+0000", 2024-05-08T08:53:32.112 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7739999999999999e-06 2024-05-08T08:53:32.112 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.112 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:32.112 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.112 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.112 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.113 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:18208 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:32.113 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.787821+0000", 2024-05-08T08:53:32.113 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.018243013999999998, 2024-05-08T08:53:32.113 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.024465028999999999, 2024-05-08T08:53:32.113 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:32.113 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:32.113 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:32.113 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:32.113 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:32.113 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:32.113 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 18208 2024-05-08T08:53:32.113 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.113 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:32.113 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.113 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:32.114 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.787821+0000", 2024-05-08T08:53:32.114 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.114 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.114 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.114 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:32.114 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.787821+0000", 2024-05-08T08:53:32.114 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.114 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.114 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.114 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:32.114 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.787822+0000", 2024-05-08T08:53:32.114 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0399999999999995e-07 2024-05-08T08:53:32.114 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.114 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.115 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:32.115 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.787824+0000", 2024-05-08T08:53:32.115 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7800000000000001e-06 2024-05-08T08:53:32.115 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.115 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.115 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:32.115 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.787831+0000", 2024-05-08T08:53:32.115 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3670000000000002e-06 2024-05-08T08:53:32.115 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.115 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.115 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:32.115 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.787836+0000", 2024-05-08T08:53:32.115 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0520000000000001e-06 2024-05-08T08:53:32.115 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.115 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:32.116 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.116 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.116 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.116 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:18237 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:32.116 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.790087+0000", 2024-05-08T08:53:32.116 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.015976981000000001, 2024-05-08T08:53:32.116 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.022260210999999998, 2024-05-08T08:53:32.116 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:32.116 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:32.116 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:32.116 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:32.116 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:32.116 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:32.116 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 18237 2024-05-08T08:53:32.116 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.117 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:32.117 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.117 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:32.117 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.790087+0000", 2024-05-08T08:53:32.117 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.117 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.117 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.117 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:32.117 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.790087+0000", 2024-05-08T08:53:32.117 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.117 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.117 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.117 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:32.117 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.790087+0000", 2024-05-08T08:53:32.117 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9999999999999998e-07 2024-05-08T08:53:32.117 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.118 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.118 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:32.118 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.790091+0000", 2024-05-08T08:53:32.118 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1240000000000001e-06 2024-05-08T08:53:32.118 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.118 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.118 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:32.118 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.790097+0000", 2024-05-08T08:53:32.118 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.545e-06 2024-05-08T08:53:32.118 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.118 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.118 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:32.118 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.790102+0000", 2024-05-08T08:53:32.118 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2530000000000004e-06 2024-05-08T08:53:32.118 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.119 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:32.119 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.119 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.119 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.119 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:14854 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:32.119 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.787835+0000", 2024-05-08T08:53:32.119 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.018228919999999999, 2024-05-08T08:53:32.119 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.024573558999999998, 2024-05-08T08:53:32.119 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:32.119 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:32.119 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:32.119 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:32.119 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:32.119 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:32.119 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14854 2024-05-08T08:53:32.120 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.120 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:32.120 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.120 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:32.120 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.787835+0000", 2024-05-08T08:53:32.120 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.120 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.120 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.120 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:32.120 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.787835+0000", 2024-05-08T08:53:32.120 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.120 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.120 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.120 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:32.120 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.787836+0000", 2024-05-08T08:53:32.121 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9999999999999997e-07 2024-05-08T08:53:32.121 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.121 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.121 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:32.121 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.787839+0000", 2024-05-08T08:53:32.121 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7779999999999999e-06 2024-05-08T08:53:32.121 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.121 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.121 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:32.121 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.787845+0000", 2024-05-08T08:53:32.121 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3609999999999997e-06 2024-05-08T08:53:32.121 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.121 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.121 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:32.122 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.787850+0000", 2024-05-08T08:53:32.122 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0159999999999999e-06 2024-05-08T08:53:32.122 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.122 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:32.122 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.122 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.122 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.122 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:18238 3.0 3.6c38bb00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:32.122 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.790110+0000", 2024-05-08T08:53:32.122 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.015954386000000001, 2024-05-08T08:53:32.122 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.022362112999999999, 2024-05-08T08:53:32.122 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:32.122 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:32.122 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:32.122 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:32.123 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:32.123 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:32.123 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 18238 2024-05-08T08:53:32.123 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.123 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:32.123 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.123 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:32.123 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.790110+0000", 2024-05-08T08:53:32.123 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.123 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.123 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.123 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:32.123 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.790110+0000", 2024-05-08T08:53:32.123 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.123 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.124 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.124 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:32.124 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.790110+0000", 2024-05-08T08:53:32.124 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6899999999999997e-07 2024-05-08T08:53:32.124 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.124 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.124 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:32.124 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.790122+0000", 2024-05-08T08:53:32.124 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2278e-05 2024-05-08T08:53:32.124 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.124 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.124 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:32.124 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.790130+0000", 2024-05-08T08:53:32.124 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1099999999999997e-06 2024-05-08T08:53:32.124 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.124 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.124 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:32.125 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.790135+0000", 2024-05-08T08:53:32.125 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.322e-06 2024-05-08T08:53:32.125 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.125 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:32.125 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.125 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.125 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T08:53:32.125 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 101 2024-05-08T08:53:32.125 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:32.127 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:53:32.127 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:32.129+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:32.229 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:32.230+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:32.329 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:32.332+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:32.356 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:32.356 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:53:32.356 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:53:32.357 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:53:32.357 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:32.366 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:53:32.450 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:53:32.453+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:53:32.530 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:32.534+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:32.537 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:32.537 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:53:32.537 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:53:32.537 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:53:32.537 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.538 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:12443 3.0 3:02d0158e:::rbd_data.114071bcf828.0000000000000037:head [stat,delete] snapc 19=[19] RETRY=3 ondisk+retry+write+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:32.538 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.610027+0000", 2024-05-08T08:53:32.538 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.929174944, 2024-05-08T08:53:32.538 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.083586333999999998, 2024-05-08T08:53:32.538 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:32.538 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:32.538 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:32.538 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:32.538 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:32.538 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:32.538 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 12443 2024-05-08T08:53:32.538 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.538 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:32.538 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.539 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:32.539 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610027+0000", 2024-05-08T08:53:32.539 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.539 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.539 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.539 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:32.539 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610027+0000", 2024-05-08T08:53:32.539 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.539 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.539 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.539 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:32.539 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610028+0000", 2024-05-08T08:53:32.539 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.13e-07 2024-05-08T08:53:32.539 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.539 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.539 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:32.540 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610030+0000", 2024-05-08T08:53:32.540 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7669999999999999e-06 2024-05-08T08:53:32.540 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.540 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.540 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:32.540 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610039+0000", 2024-05-08T08:53:32.540 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4959999999999996e-06 2024-05-08T08:53:32.540 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.540 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.540 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:32.540 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610045+0000", 2024-05-08T08:53:32.540 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6779999999999997e-06 2024-05-08T08:53:32.540 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.540 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.540 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:32.540 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.660873+0000", 2024-05-08T08:53:32.540 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.050828748 2024-05-08T08:53:32.541 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.541 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.541 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:32.541 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.661017+0000", 2024-05-08T08:53:32.541 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014391999999999999 2024-05-08T08:53:32.541 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.541 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.541 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:32.541 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.693613+0000", 2024-05-08T08:53:32.541 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.032596111999999997 2024-05-08T08:53:32.541 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.541 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:32.541 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.541 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.541 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.541 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:14301 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [watch reconnect cookie 139837155901536 gen 12] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:32.542 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.645078+0000", 2024-05-08T08:53:32.542 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.89412430600000004, 2024-05-08T08:53:32.542 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088281852999999993, 2024-05-08T08:53:32.542 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:32.542 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:32.542 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:32.542 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:32.542 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:32.542 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:32.542 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14301 2024-05-08T08:53:32.542 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.542 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:32.542 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.542 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:32.542 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645078+0000", 2024-05-08T08:53:32.542 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.543 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.543 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.543 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:32.543 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645078+0000", 2024-05-08T08:53:32.543 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.543 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.543 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.543 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:32.543 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645084+0000", 2024-05-08T08:53:32.543 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4620000000000001e-06 2024-05-08T08:53:32.543 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.543 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.543 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:32.543 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645088+0000", 2024-05-08T08:53:32.543 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8039999999999999e-06 2024-05-08T08:53:32.543 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.544 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.544 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:32.544 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645106+0000", 2024-05-08T08:53:32.544 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.8318999999999999e-05 2024-05-08T08:53:32.544 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.544 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.544 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:32.544 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645113+0000", 2024-05-08T08:53:32.544 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0310000000000001e-06 2024-05-08T08:53:32.544 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.544 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.544 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:32.544 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.670493+0000", 2024-05-08T08:53:32.544 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.025379993 2024-05-08T08:53:32.544 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.544 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.544 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:32.545 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.670571+0000", 2024-05-08T08:53:32.545 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7613000000000004e-05 2024-05-08T08:53:32.545 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.545 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.545 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:32.545 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733360+0000", 2024-05-08T08:53:32.545 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.062788630999999998 2024-05-08T08:53:32.545 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.545 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:32.545 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.545 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.545 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.545 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:14420 3.0 3:06a8948e:::rbd_data.114071bcf828.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:32.545 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.649639+0000", 2024-05-08T08:53:32.545 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.88956315799999996, 2024-05-08T08:53:32.546 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.083851897999999994, 2024-05-08T08:53:32.546 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:32.546 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:32.546 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:53:32.546 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:32.546 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:32.546 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:32.546 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14420 2024-05-08T08:53:32.546 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.546 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:32.546 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.546 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:32.546 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649639+0000", 2024-05-08T08:53:32.546 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.546 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.546 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.546 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:32.547 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649639+0000", 2024-05-08T08:53:32.547 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.547 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.547 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.547 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:32.547 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649640+0000", 2024-05-08T08:53:32.547 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6899999999999997e-07 2024-05-08T08:53:32.547 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.547 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.547 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:32.547 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649895+0000", 2024-05-08T08:53:32.547 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00025531000000000002 2024-05-08T08:53:32.547 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.547 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.547 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:32.547 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649905+0000", 2024-05-08T08:53:32.547 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0198e-05 2024-05-08T08:53:32.548 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.548 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.548 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:32.548 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649911+0000", 2024-05-08T08:53:32.548 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2310000000000001e-06 2024-05-08T08:53:32.548 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.548 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.548 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:32.548 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.684099+0000", 2024-05-08T08:53:32.548 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.034188017000000001 2024-05-08T08:53:32.548 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.548 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.548 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:32.548 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.684172+0000", 2024-05-08T08:53:32.548 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3201999999999995e-05 2024-05-08T08:53:32.548 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.549 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.549 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 2", 2024-05-08T08:53:32.549 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.684418+0000", 2024-05-08T08:53:32.549 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00024559099999999997 2024-05-08T08:53:32.549 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.549 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.549 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:53:32.549 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.705538+0000", 2024-05-08T08:53:32.549 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.021119707000000001 2024-05-08T08:53:32.549 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.549 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.549 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:53:32.549 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733399+0000", 2024-05-08T08:53:32.549 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.027861543999999999 2024-05-08T08:53:32.549 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.549 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.550 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:53:32.550 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733445+0000", 2024-05-08T08:53:32.550 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5587000000000002e-05 2024-05-08T08:53:32.550 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.550 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.550 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:32.550 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733491+0000", 2024-05-08T08:53:32.550 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5841999999999999e-05 2024-05-08T08:53:32.550 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.550 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:32.550 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.550 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.550 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.550 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28547 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889179216] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:32.551 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.355654+0000", 2024-05-08T08:53:32.551 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.18354793, 2024-05-08T08:53:32.551 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078692261999999999, 2024-05-08T08:53:32.551 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:32.551 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:32.551 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:32.551 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:32.551 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:32.551 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:32.551 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28547 2024-05-08T08:53:32.551 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.551 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:32.551 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.551 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:32.551 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355654+0000", 2024-05-08T08:53:32.551 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.552 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.552 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.552 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:32.552 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355654+0000", 2024-05-08T08:53:32.552 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.552 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.552 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.552 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:32.552 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355654+0000", 2024-05-08T08:53:32.552 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3999999999999998e-07 2024-05-08T08:53:32.552 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.552 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.552 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:32.552 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355656+0000", 2024-05-08T08:53:32.552 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.353e-06 2024-05-08T08:53:32.553 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.553 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.553 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:32.553 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355663+0000", 2024-05-08T08:53:32.553 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2259999999999999e-06 2024-05-08T08:53:32.553 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.553 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.553 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:32.553 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355666+0000", 2024-05-08T08:53:32.553 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7930000000000002e-06 2024-05-08T08:53:32.553 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.553 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.553 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:32.553 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434117+0000", 2024-05-08T08:53:32.553 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078451231999999996 2024-05-08T08:53:32.554 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.554 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.554 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:32.554 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434198+0000", 2024-05-08T08:53:32.554 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0879999999999998e-05 2024-05-08T08:53:32.554 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.554 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.554 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:32.554 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434346+0000", 2024-05-08T08:53:32.554 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014853800000000001 2024-05-08T08:53:32.554 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.554 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:32.554 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.554 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.555 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.555 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28548 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889189696] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:32.555 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.355669+0000", 2024-05-08T08:53:32.555 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.18353258, 2024-05-08T08:53:32.555 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078956606999999998, 2024-05-08T08:53:32.555 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:32.555 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:32.555 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:32.555 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:32.555 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:32.555 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:32.555 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28548 2024-05-08T08:53:32.555 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.555 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:32.556 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.556 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:32.556 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355669+0000", 2024-05-08T08:53:32.556 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.556 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.556 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.556 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:32.556 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355669+0000", 2024-05-08T08:53:32.556 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.556 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.556 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.556 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:32.556 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355762+0000", 2024-05-08T08:53:32.556 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3015000000000003e-05 2024-05-08T08:53:32.556 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.557 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.557 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:32.557 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355765+0000", 2024-05-08T08:53:32.557 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.368e-06 2024-05-08T08:53:32.557 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.557 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.557 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:32.557 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355771+0000", 2024-05-08T08:53:32.557 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5989999999999998e-06 2024-05-08T08:53:32.557 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.557 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.557 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:32.557 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355775+0000", 2024-05-08T08:53:32.557 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7280000000000001e-06 2024-05-08T08:53:32.558 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.558 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.558 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:32.558 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434379+0000", 2024-05-08T08:53:32.558 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078603662000000005 2024-05-08T08:53:32.558 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.558 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.558 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:32.558 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434473+0000", 2024-05-08T08:53:32.558 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4104000000000006e-05 2024-05-08T08:53:32.558 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.558 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.558 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:32.558 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434626+0000", 2024-05-08T08:53:32.559 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00015313100000000001 2024-05-08T08:53:32.559 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.559 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:32.559 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.559 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.559 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.559 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28549 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889202400] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:32.559 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.355781+0000", 2024-05-08T08:53:32.559 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.183421006, 2024-05-08T08:53:32.559 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079117946999999994, 2024-05-08T08:53:32.559 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:32.559 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:32.559 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:32.559 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:32.559 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:32.560 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:32.560 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28549 2024-05-08T08:53:32.560 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.560 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:32.560 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.560 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:32.560 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355781+0000", 2024-05-08T08:53:32.560 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.560 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.560 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.560 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:32.560 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355781+0000", 2024-05-08T08:53:32.560 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.560 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.560 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.560 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:32.561 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356162+0000", 2024-05-08T08:53:32.561 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00038099100000000001 2024-05-08T08:53:32.561 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.561 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.561 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:32.561 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356165+0000", 2024-05-08T08:53:32.561 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1240000000000001e-06 2024-05-08T08:53:32.561 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.561 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.561 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:32.561 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356173+0000", 2024-05-08T08:53:32.561 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.6199999999999999e-06 2024-05-08T08:53:32.561 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.561 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.561 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:32.561 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356182+0000", 2024-05-08T08:53:32.562 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7760000000000003e-06 2024-05-08T08:53:32.562 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.562 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.562 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:32.562 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434675+0000", 2024-05-08T08:53:32.562 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078493119 2024-05-08T08:53:32.562 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.562 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.562 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:32.562 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434765+0000", 2024-05-08T08:53:32.562 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0402000000000006e-05 2024-05-08T08:53:32.562 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.562 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.563 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:32.563 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434899+0000", 2024-05-08T08:53:32.563 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013391499999999999 2024-05-08T08:53:32.563 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.563 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:32.563 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.563 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.563 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.563 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28550 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888597280] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:32.563 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:53:32.563 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.18300996, 2024-05-08T08:53:32.563 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078933990999999995, 2024-05-08T08:53:32.563 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:32.563 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:32.563 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:32.563 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:32.563 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:32.564 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:32.564 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28550 2024-05-08T08:53:32.564 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.564 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:32.564 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.564 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:32.564 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:53:32.564 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.564 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.564 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.564 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:32.564 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:53:32.564 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.564 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.564 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.564 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:32.565 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:53:32.565 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.34e-07 2024-05-08T08:53:32.565 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.565 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.565 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:32.565 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356195+0000", 2024-05-08T08:53:32.565 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8710000000000001e-06 2024-05-08T08:53:32.565 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.565 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.565 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:32.565 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356200+0000", 2024-05-08T08:53:32.565 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3000000000000003e-06 2024-05-08T08:53:32.565 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.565 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.565 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:32.565 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356203+0000", 2024-05-08T08:53:32.565 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4039999999999999e-06 2024-05-08T08:53:32.566 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.566 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.566 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:32.566 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434928+0000", 2024-05-08T08:53:32.566 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078725331999999995 2024-05-08T08:53:32.566 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.566 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.566 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:32.566 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435007+0000", 2024-05-08T08:53:32.566 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8415000000000001e-05 2024-05-08T08:53:32.566 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.566 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.566 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:32.566 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435126+0000", 2024-05-08T08:53:32.567 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011923500000000001 2024-05-08T08:53:32.567 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.567 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:32.567 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.567 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.567 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.567 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28551 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888527200] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:32.567 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.356210+0000", 2024-05-08T08:53:32.567 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.182991612, 2024-05-08T08:53:32.567 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079195072000000005, 2024-05-08T08:53:32.567 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:32.567 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:32.567 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:32.567 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:32.567 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:32.567 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:32.567 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28551 2024-05-08T08:53:32.568 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.568 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:32.568 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.568 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:32.568 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356210+0000", 2024-05-08T08:53:32.568 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.568 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.568 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.568 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:32.568 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356210+0000", 2024-05-08T08:53:32.568 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.568 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.568 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.568 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:32.568 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356211+0000", 2024-05-08T08:53:32.569 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4499999999999998e-07 2024-05-08T08:53:32.569 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.569 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.569 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:32.569 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356212+0000", 2024-05-08T08:53:32.569 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6029999999999999e-06 2024-05-08T08:53:32.569 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.569 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.569 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:32.569 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356221+0000", 2024-05-08T08:53:32.569 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4200000000000007e-06 2024-05-08T08:53:32.569 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.569 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.569 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:32.569 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356224+0000", 2024-05-08T08:53:32.569 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.038e-06 2024-05-08T08:53:32.570 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.570 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.570 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:32.570 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435178+0000", 2024-05-08T08:53:32.570 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078954625000000001 2024-05-08T08:53:32.570 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.570 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.570 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:32.570 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435256+0000", 2024-05-08T08:53:32.570 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7337999999999997e-05 2024-05-08T08:53:32.570 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.570 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.570 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:32.570 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435405+0000", 2024-05-08T08:53:32.570 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000149703 2024-05-08T08:53:32.570 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.571 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:32.571 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.571 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.571 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.571 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28552 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889231456] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:32.571 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.356228+0000", 2024-05-08T08:53:32.571 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.182973934, 2024-05-08T08:53:32.571 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079346920000000001, 2024-05-08T08:53:32.571 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:32.571 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:32.571 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:32.571 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:32.571 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:32.571 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:32.571 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28552 2024-05-08T08:53:32.571 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.571 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:32.572 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.572 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:32.572 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356228+0000", 2024-05-08T08:53:32.572 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.572 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.572 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.572 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:32.572 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356228+0000", 2024-05-08T08:53:32.572 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.572 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.572 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.572 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:32.572 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356959+0000", 2024-05-08T08:53:32.572 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00073097099999999996 2024-05-08T08:53:32.572 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.572 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.572 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:32.573 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356967+0000", 2024-05-08T08:53:32.573 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0409999999999998e-06 2024-05-08T08:53:32.573 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.573 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.573 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:32.573 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356979+0000", 2024-05-08T08:53:32.573 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1489e-05 2024-05-08T08:53:32.573 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.573 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.573 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:32.573 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356986+0000", 2024-05-08T08:53:32.573 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3699999999999997e-06 2024-05-08T08:53:32.573 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.573 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.573 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:32.573 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435433+0000", 2024-05-08T08:53:32.573 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078447520000000007 2024-05-08T08:53:32.574 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.574 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.574 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:32.574 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435484+0000", 2024-05-08T08:53:32.574 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0229000000000003e-05 2024-05-08T08:53:32.574 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.574 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.574 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:32.574 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435575+0000", 2024-05-08T08:53:32.574 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1299999999999997e-05 2024-05-08T08:53:32.574 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.574 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:32.574 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.574 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.574 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.575 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28553 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888713648] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:32.575 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:53:32.575 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.18219927599999999, 2024-05-08T08:53:32.575 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078782034000000001, 2024-05-08T08:53:32.575 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:32.575 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:32.575 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:32.575 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:32.575 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:32.575 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:32.575 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28553 2024-05-08T08:53:32.575 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.575 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:32.575 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.575 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:32.576 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:53:32.576 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.576 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.576 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.576 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:32.576 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:53:32.576 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.576 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.576 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.576 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:32.576 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:53:32.576 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6600000000000002e-07 2024-05-08T08:53:32.576 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.576 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.576 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:32.576 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357006+0000", 2024-05-08T08:53:32.577 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7190000000000001e-06 2024-05-08T08:53:32.577 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.577 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.577 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:32.577 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357013+0000", 2024-05-08T08:53:32.577 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.435e-06 2024-05-08T08:53:32.577 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.577 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.577 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:32.577 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357018+0000", 2024-05-08T08:53:32.577 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7260000000000002e-06 2024-05-08T08:53:32.577 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.577 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.577 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:32.577 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435623+0000", 2024-05-08T08:53:32.578 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078604972999999995 2024-05-08T08:53:32.578 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.578 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.578 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:32.578 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435695+0000", 2024-05-08T08:53:32.578 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1959999999999995e-05 2024-05-08T08:53:32.578 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.578 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.578 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:32.578 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435785+0000", 2024-05-08T08:53:32.578 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.9754999999999997e-05 2024-05-08T08:53:32.578 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.578 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:32.579 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.579 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.579 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.579 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28554 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888980272] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:32.579 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357024+0000", 2024-05-08T08:53:32.579 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.18217764, 2024-05-08T08:53:32.579 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078999870999999999, 2024-05-08T08:53:32.579 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:32.579 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:32.579 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:32.579 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:32.579 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:32.579 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:32.579 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28554 2024-05-08T08:53:32.579 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.579 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:32.580 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.580 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:32.580 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357024+0000", 2024-05-08T08:53:32.580 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.580 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.580 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.580 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:32.580 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357024+0000", 2024-05-08T08:53:32.580 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.580 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.580 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.580 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:32.580 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357025+0000", 2024-05-08T08:53:32.580 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:53:32.580 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.580 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.581 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:32.581 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357027+0000", 2024-05-08T08:53:32.581 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2050000000000001e-06 2024-05-08T08:53:32.581 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.581 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.581 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:32.581 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357033+0000", 2024-05-08T08:53:32.581 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5679999999999996e-06 2024-05-08T08:53:32.581 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.581 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.581 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:32.581 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357038+0000", 2024-05-08T08:53:32.581 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2889999999999999e-06 2024-05-08T08:53:32.581 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.581 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.581 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:32.581 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435823+0000", 2024-05-08T08:53:32.582 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078785677999999998 2024-05-08T08:53:32.582 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.582 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.582 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:32.582 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435892+0000", 2024-05-08T08:53:32.582 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8379000000000004e-05 2024-05-08T08:53:32.582 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.582 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.582 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:32.582 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436024+0000", 2024-05-08T08:53:32.582 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013239799999999999 2024-05-08T08:53:32.582 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.582 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:32.582 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.582 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.583 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.583 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28555 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889001104] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:32.583 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:53:32.583 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.182158875, 2024-05-08T08:53:32.583 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079115742000000003, 2024-05-08T08:53:32.583 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:32.583 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:32.583 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:32.583 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:32.583 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:32.583 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:32.583 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28555 2024-05-08T08:53:32.583 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.583 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:32.583 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.583 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:32.583 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:53:32.584 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.584 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.584 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.584 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:32.584 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:53:32.584 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.584 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.584 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.584 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:32.584 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:53:32.584 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9200000000000002e-07 2024-05-08T08:53:32.584 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.584 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.584 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:32.584 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357046+0000", 2024-05-08T08:53:32.584 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1670000000000002e-06 2024-05-08T08:53:32.585 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.585 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.585 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:32.585 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357052+0000", 2024-05-08T08:53:32.585 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.139e-06 2024-05-08T08:53:32.585 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.585 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.585 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:32.585 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357056+0000", 2024-05-08T08:53:32.585 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2440000000000004e-06 2024-05-08T08:53:32.585 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.585 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.585 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:32.585 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436059+0000", 2024-05-08T08:53:32.585 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079003070999999994 2024-05-08T08:53:32.585 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.585 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.586 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:32.586 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436097+0000", 2024-05-08T08:53:32.586 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7784000000000003e-05 2024-05-08T08:53:32.586 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.586 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.586 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:32.586 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436159+0000", 2024-05-08T08:53:32.586 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2045000000000007e-05 2024-05-08T08:53:32.586 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.586 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:32.586 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.586 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.586 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.586 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28556 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889022064] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:32.587 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357065+0000", 2024-05-08T08:53:32.587 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.18213684199999999, 2024-05-08T08:53:32.587 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079327976999999994, 2024-05-08T08:53:32.587 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:32.587 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:32.587 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:32.587 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:32.587 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:32.587 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:32.587 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28556 2024-05-08T08:53:32.587 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.587 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:32.587 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.587 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:32.587 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357065+0000", 2024-05-08T08:53:32.588 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.588 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.588 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.588 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:32.588 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357065+0000", 2024-05-08T08:53:32.588 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.588 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.588 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.588 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:32.588 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358958+0000", 2024-05-08T08:53:32.588 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001892915 2024-05-08T08:53:32.588 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.588 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.588 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:32.588 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358962+0000", 2024-05-08T08:53:32.588 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4599999999999999e-06 2024-05-08T08:53:32.589 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.589 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.589 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:32.589 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358970+0000", 2024-05-08T08:53:32.589 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.6090000000000001e-06 2024-05-08T08:53:32.589 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.589 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.589 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:32.589 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358975+0000", 2024-05-08T08:53:32.589 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7639999999999996e-06 2024-05-08T08:53:32.589 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.589 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.589 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:32.589 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436196+0000", 2024-05-08T08:53:32.589 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.077221564000000006 2024-05-08T08:53:32.590 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.590 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.590 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:32.590 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436269+0000", 2024-05-08T08:53:32.590 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2720999999999997e-05 2024-05-08T08:53:32.590 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.590 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.590 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:32.590 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436393+0000", 2024-05-08T08:53:32.590 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012394399999999999 2024-05-08T08:53:32.590 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.590 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:32.590 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.590 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.590 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.590 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28563 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888855520] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:32.591 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:53:32.591 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.180127443, 2024-05-08T08:53:32.591 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078795566999999997, 2024-05-08T08:53:32.591 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:32.591 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:32.591 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:32.591 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:32.591 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:32.591 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:32.591 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28563 2024-05-08T08:53:32.591 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.591 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:32.591 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.591 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:32.591 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:53:32.591 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.591 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.592 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.592 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:32.592 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:53:32.592 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.592 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.592 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.592 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:32.592 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:53:32.592 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1e-07 2024-05-08T08:53:32.592 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.592 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.592 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:32.592 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359080+0000", 2024-05-08T08:53:32.592 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8940000000000001e-06 2024-05-08T08:53:32.592 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.592 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.592 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:32.593 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359083+0000", 2024-05-08T08:53:32.593 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1949999999999999e-06 2024-05-08T08:53:32.593 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.593 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.593 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:32.593 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359085+0000", 2024-05-08T08:53:32.593 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4710000000000001e-06 2024-05-08T08:53:32.593 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.593 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.593 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:32.593 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437604+0000", 2024-05-08T08:53:32.593 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078518278999999996 2024-05-08T08:53:32.593 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.593 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.593 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:32.593 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437704+0000", 2024-05-08T08:53:32.593 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010010599999999999 2024-05-08T08:53:32.594 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.594 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.594 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:32.594 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437870+0000", 2024-05-08T08:53:32.594 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00016641200000000001 2024-05-08T08:53:32.594 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.594 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:32.594 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.594 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.594 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.594 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28564 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888538944] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:32.594 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.359089+0000", 2024-05-08T08:53:32.594 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.18011322799999999, 2024-05-08T08:53:32.594 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078952675, 2024-05-08T08:53:32.594 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:32.595 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:32.595 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:32.595 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:32.595 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:32.595 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:32.595 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28564 2024-05-08T08:53:32.595 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.595 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:32.595 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.595 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:32.595 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359089+0000", 2024-05-08T08:53:32.595 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.595 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.595 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.595 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:32.596 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359089+0000", 2024-05-08T08:53:32.596 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.596 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.596 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.596 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:32.596 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360959+0000", 2024-05-08T08:53:32.596 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0018707089999999999 2024-05-08T08:53:32.596 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.596 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.596 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:32.596 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360963+0000", 2024-05-08T08:53:32.596 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0680000000000001e-06 2024-05-08T08:53:32.596 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.596 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.596 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:32.596 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360969+0000", 2024-05-08T08:53:32.597 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1920000000000002e-06 2024-05-08T08:53:32.597 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.597 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.597 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:32.597 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360973+0000", 2024-05-08T08:53:32.597 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.031e-06 2024-05-08T08:53:32.597 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.597 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.597 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:32.597 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437916+0000", 2024-05-08T08:53:32.597 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.076942939000000002 2024-05-08T08:53:32.597 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.597 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.597 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:32.597 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437973+0000", 2024-05-08T08:53:32.597 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7358999999999997e-05 2024-05-08T08:53:32.598 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.598 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.598 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:32.598 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.438041+0000", 2024-05-08T08:53:32.598 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8376999999999999e-05 2024-05-08T08:53:32.598 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.598 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:32.598 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.598 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.598 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.598 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28574 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889427600] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:32.598 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:53:32.598 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.178108721, 2024-05-08T08:53:32.598 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078782476000000004, 2024-05-08T08:53:32.599 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:32.599 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:32.599 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:32.599 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:32.599 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:32.599 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:32.599 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28574 2024-05-08T08:53:32.599 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.599 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:32.599 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.599 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:32.599 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:53:32.599 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.599 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.600 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.600 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:32.600 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:53:32.600 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.600 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.600 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.600 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:32.600 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:53:32.600 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1799999999999999e-07 2024-05-08T08:53:32.600 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.600 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.600 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:32.600 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361095+0000", 2024-05-08T08:53:32.600 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3850000000000001e-06 2024-05-08T08:53:32.600 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.600 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.601 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:32.601 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361098+0000", 2024-05-08T08:53:32.601 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7989999999999998e-06 2024-05-08T08:53:32.601 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.601 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.601 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:32.601 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361100+0000", 2024-05-08T08:53:32.601 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4669999999999998e-06 2024-05-08T08:53:32.601 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.601 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.601 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:32.601 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439768+0000", 2024-05-08T08:53:32.601 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078668057999999999 2024-05-08T08:53:32.601 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.601 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.601 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:32.602 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439811+0000", 2024-05-08T08:53:32.602 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2459000000000001e-05 2024-05-08T08:53:32.602 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.602 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.602 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:32.602 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439876+0000", 2024-05-08T08:53:32.602 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5090000000000002e-05 2024-05-08T08:53:32.602 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.602 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:32.602 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.602 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.602 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.602 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28575 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889439216] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:32.602 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361103+0000", 2024-05-08T08:53:32.603 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.178098706, 2024-05-08T08:53:32.603 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078966828000000003, 2024-05-08T08:53:32.603 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:32.603 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:32.603 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:32.603 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:32.603 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:32.603 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:32.603 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28575 2024-05-08T08:53:32.603 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.603 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:32.603 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.603 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:32.603 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361103+0000", 2024-05-08T08:53:32.603 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.604 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.604 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.604 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:32.604 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361103+0000", 2024-05-08T08:53:32.604 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.604 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.604 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.604 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:32.604 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361104+0000", 2024-05-08T08:53:32.604 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1400000000000001e-07 2024-05-08T08:53:32.604 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.604 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.604 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:32.604 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361105+0000", 2024-05-08T08:53:32.604 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5540000000000001e-06 2024-05-08T08:53:32.604 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.605 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.605 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:32.605 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361108+0000", 2024-05-08T08:53:32.605 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.841e-06 2024-05-08T08:53:32.605 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.605 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.605 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:32.605 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361110+0000", 2024-05-08T08:53:32.605 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.497e-06 2024-05-08T08:53:32.605 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.605 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.605 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:32.605 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439916+0000", 2024-05-08T08:53:32.605 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078805372999999998 2024-05-08T08:53:32.605 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.605 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.605 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:32.606 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440000+0000", 2024-05-08T08:53:32.606 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4730999999999999e-05 2024-05-08T08:53:32.606 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.606 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.606 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:32.606 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440070+0000", 2024-05-08T08:53:32.606 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9617999999999997e-05 2024-05-08T08:53:32.606 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.606 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:32.606 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.606 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.606 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.606 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28576 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889451328] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:32.606 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361113+0000", 2024-05-08T08:53:32.607 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.17808852, 2024-05-08T08:53:32.607 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079088862999999995, 2024-05-08T08:53:32.607 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:32.607 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:32.607 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:32.607 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:32.607 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:32.607 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:32.607 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28576 2024-05-08T08:53:32.607 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.607 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:32.607 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.607 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:32.607 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361113+0000", 2024-05-08T08:53:32.608 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.608 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.608 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.608 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:32.608 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361113+0000", 2024-05-08T08:53:32.608 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.608 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.608 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.608 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:32.608 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361114+0000", 2024-05-08T08:53:32.608 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1299999999999999e-07 2024-05-08T08:53:32.608 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.608 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.608 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:32.609 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361115+0000", 2024-05-08T08:53:32.609 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.4300000000000001e-06 2024-05-08T08:53:32.609 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.609 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.609 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:32.609 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361118+0000", 2024-05-08T08:53:32.609 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9000000000000002e-06 2024-05-08T08:53:32.609 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.609 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.609 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:32.609 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361120+0000", 2024-05-08T08:53:32.609 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4739999999999999e-06 2024-05-08T08:53:32.609 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.609 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.609 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:32.610 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440096+0000", 2024-05-08T08:53:32.610 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078975629000000006 2024-05-08T08:53:32.610 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.610 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.610 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:32.610 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440135+0000", 2024-05-08T08:53:32.610 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9202000000000002e-05 2024-05-08T08:53:32.610 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.610 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.610 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:32.610 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440202+0000", 2024-05-08T08:53:32.610 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7014999999999994e-05 2024-05-08T08:53:32.610 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.610 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:32.610 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.611 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.611 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.611 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28577 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889462640] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:32.611 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:53:32.611 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.178078338, 2024-05-08T08:53:32.611 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079194622000000006, 2024-05-08T08:53:32.611 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:32.611 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:32.611 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:32.611 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:32.611 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:32.611 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:32.611 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28577 2024-05-08T08:53:32.611 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.612 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:32.612 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.612 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:32.612 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:53:32.612 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.612 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.612 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.612 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:32.612 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:53:32.612 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.612 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.612 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.612 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:32.612 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:53:32.613 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1400000000000001e-07 2024-05-08T08:53:32.613 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.613 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.613 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:32.613 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361125+0000", 2024-05-08T08:53:32.613 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.372e-06 2024-05-08T08:53:32.613 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.613 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.613 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:32.613 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361128+0000", 2024-05-08T08:53:32.613 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0309999999999999e-06 2024-05-08T08:53:32.613 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.613 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.613 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:32.614 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361131+0000", 2024-05-08T08:53:32.614 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4040000000000002e-06 2024-05-08T08:53:32.614 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.614 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.614 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:32.614 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440225+0000", 2024-05-08T08:53:32.614 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079094600000000001 2024-05-08T08:53:32.614 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.614 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.614 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:32.614 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440260+0000", 2024-05-08T08:53:32.614 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4703000000000003e-05 2024-05-08T08:53:32.614 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.614 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.615 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:32.615 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440318+0000", 2024-05-08T08:53:32.615 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8298000000000003e-05 2024-05-08T08:53:32.615 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.615 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:32.615 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.615 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.615 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.615 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28578 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889474192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:32.615 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361134+0000", 2024-05-08T08:53:32.615 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.17806791299999999, 2024-05-08T08:53:32.615 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079310295000000003, 2024-05-08T08:53:32.615 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:32.615 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:32.615 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:32.616 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:32.616 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:32.616 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:32.616 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28578 2024-05-08T08:53:32.616 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.616 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:32.616 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.616 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:32.616 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361134+0000", 2024-05-08T08:53:32.616 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.616 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.616 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.616 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:32.616 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361134+0000", 2024-05-08T08:53:32.616 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:32.616 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.616 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.617 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:32.617 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.362958+0000", 2024-05-08T08:53:32.617 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0018237380000000001 2024-05-08T08:53:32.617 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.617 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.617 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:32.617 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.362961+0000", 2024-05-08T08:53:32.617 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.123e-06 2024-05-08T08:53:32.617 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.617 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.617 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:32.617 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.397404+0000", 2024-05-08T08:53:32.617 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.034443226 2024-05-08T08:53:32.617 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.617 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.617 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:32.617 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.397439+0000", 2024-05-08T08:53:32.618 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4690000000000002e-05 2024-05-08T08:53:32.618 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.618 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.618 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:32.618 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440340+0000", 2024-05-08T08:53:32.618 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.042901419000000003 2024-05-08T08:53:32.618 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.618 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.618 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:32.618 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440376+0000", 2024-05-08T08:53:32.618 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5444999999999997e-05 2024-05-08T08:53:32.618 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:32.618 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:32.618 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:32.618 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440444+0000", 2024-05-08T08:53:32.618 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8653999999999997e-05 2024-05-08T08:53:32.618 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.619 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:32.619 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.619 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:32.619 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:32.619 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:32.619 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-05-08T08:53:32.620 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:53:32.631 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:32.634+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:32.731 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:32.734+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:32.781 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:32.781 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:53:32.782 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:53:32.782 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:32.793 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:53:32.837 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:32.840+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:32.876 INFO:teuthology.orchestra.run.smithi196.stdout:checking clone #21, image image_client.0-clone21 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent22 2024-05-08T08:53:32.915 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:32.915 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:53:32.915 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:53:32.915 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:53:32.915 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:32.924 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:53:32.966 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:53:32.969+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:53:33.033 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:53:33.036+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:53:33.134 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:33.137+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:33.148 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:33.148 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:53:33.148 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:53:33.148 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:53:33.148 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.148 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:437 3.17 3:ea93a512:::rbd_data.1140100c9703.0000000000000077:head [set-alloc-hint object_size 2097152 write_size 2097152,write 1170432~1024] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e63)", 2024-05-08T08:53:33.149 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:08.390804+0000", 2024-05-08T08:53:33.149 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 24.758703896, 2024-05-08T08:53:33.149 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0406068690000001, 2024-05-08T08:53:33.149 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:33.149 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:33.149 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:53:33.149 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:33.149 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:33.149 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:33.149 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 437 2024-05-08T08:53:33.149 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.149 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:33.149 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.149 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:33.149 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390804+0000", 2024-05-08T08:53:33.150 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:33.150 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.150 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.150 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:33.150 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390804+0000", 2024-05-08T08:53:33.150 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:33.150 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.150 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.150 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:33.150 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390806+0000", 2024-05-08T08:53:33.150 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.666e-06 2024-05-08T08:53:33.150 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.150 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.150 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:33.151 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390818+0000", 2024-05-08T08:53:33.151 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2187999999999999e-05 2024-05-08T08:53:33.151 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.151 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.151 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:33.151 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390834+0000", 2024-05-08T08:53:33.151 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6191000000000001e-05 2024-05-08T08:53:33.151 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.151 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.151 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:33.151 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390845+0000", 2024-05-08T08:53:33.151 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0831e-05 2024-05-08T08:53:33.151 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.151 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.151 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:33.151 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390922+0000", 2024-05-08T08:53:33.152 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7253e-05 2024-05-08T08:53:33.152 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.152 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.152 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op must wait for map", 2024-05-08T08:53:33.152 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390930+0000", 2024-05-08T08:53:33.152 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8259999999999993e-06 2024-05-08T08:53:33.152 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.152 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.152 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:33.152 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392155+0000", 2024-05-08T08:53:33.152 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0012256649999999999 2024-05-08T08:53:33.152 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.152 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.152 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:53:33.152 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392165+0000", 2024-05-08T08:53:33.152 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5799999999999998e-06 2024-05-08T08:53:33.153 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.153 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.153 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:33.153 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.429596+0000", 2024-05-08T08:53:33.153 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0374313639999999 2024-05-08T08:53:33.153 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.153 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.153 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:33.153 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.429728+0000", 2024-05-08T08:53:33.153 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000131552 2024-05-08T08:53:33.153 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.153 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.153 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 0,1", 2024-05-08T08:53:33.153 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.429860+0000", 2024-05-08T08:53:33.154 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000132137 2024-05-08T08:53:33.154 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.154 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.154 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:53:33.154 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.430811+0000", 2024-05-08T08:53:33.154 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00095040200000000004 2024-05-08T08:53:33.154 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.154 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.154 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:53:33.154 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431126+0000", 2024-05-08T08:53:33.154 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000315006 2024-05-08T08:53:33.154 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.154 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.154 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:53:33.154 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431272+0000", 2024-05-08T08:53:33.155 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000146487 2024-05-08T08:53:33.155 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.155 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.155 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:53:33.155 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431315+0000", 2024-05-08T08:53:33.155 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2697000000000001e-05 2024-05-08T08:53:33.155 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.155 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.155 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:33.155 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431411+0000", 2024-05-08T08:53:33.155 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6024000000000001e-05 2024-05-08T08:53:33.155 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:33.155 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:33.155 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:33.155 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.156 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.156 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:439 3.17 3:ea93a512:::rbd_data.1140100c9703.0000000000000077:head [read 1163264~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-05-08T08:53:33.156 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:08.390871+0000", 2024-05-08T08:53:33.156 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 24.758636384999999, 2024-05-08T08:53:33.156 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0408790590000001, 2024-05-08T08:53:33.156 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:33.156 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:33.156 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:33.156 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:33.156 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:33.156 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:33.156 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 439 2024-05-08T08:53:33.156 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.156 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:33.156 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.157 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:33.157 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390871+0000", 2024-05-08T08:53:33.157 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:33.157 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.157 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.157 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:33.157 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390871+0000", 2024-05-08T08:53:33.157 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:33.157 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.157 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.157 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:33.157 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390872+0000", 2024-05-08T08:53:33.157 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6300000000000005e-07 2024-05-08T08:53:33.157 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.157 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.157 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:33.158 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390875+0000", 2024-05-08T08:53:33.158 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3459999999999998e-06 2024-05-08T08:53:33.158 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.158 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.158 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:33.158 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390891+0000", 2024-05-08T08:53:33.158 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5255e-05 2024-05-08T08:53:33.158 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.158 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.158 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:33.158 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390897+0000", 2024-05-08T08:53:33.158 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2360000000000001e-06 2024-05-08T08:53:33.158 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.158 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.158 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:33.159 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390995+0000", 2024-05-08T08:53:33.159 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8522000000000003e-05 2024-05-08T08:53:33.159 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.159 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.159 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting_for_map not empty", 2024-05-08T08:53:33.159 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.391005+0000", 2024-05-08T08:53:33.159 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1460000000000002e-06 2024-05-08T08:53:33.159 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.159 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.159 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:33.159 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392191+0000", 2024-05-08T08:53:33.159 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0011860270000000001 2024-05-08T08:53:33.159 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.159 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.159 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:53:33.160 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392197+0000", 2024-05-08T08:53:33.160 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4289999999999998e-06 2024-05-08T08:53:33.160 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.160 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.160 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:33.160 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.430596+0000", 2024-05-08T08:53:33.160 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0383989090000001 2024-05-08T08:53:33.160 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.160 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.160 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for rw locks", 2024-05-08T08:53:33.160 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.430676+0000", 2024-05-08T08:53:33.160 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9781000000000001e-05 2024-05-08T08:53:33.160 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.160 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.160 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:33.160 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431460+0000", 2024-05-08T08:53:33.161 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00078427300000000004 2024-05-08T08:53:33.161 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.161 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.161 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:33.161 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431540+0000", 2024-05-08T08:53:33.161 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0224999999999998e-05 2024-05-08T08:53:33.161 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.161 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.161 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:33.161 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431750+0000", 2024-05-08T08:53:33.161 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00021024699999999999 2024-05-08T08:53:33.161 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:33.161 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:33.161 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:33.161 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.162 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.162 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:945 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 4096000~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:33.162 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:53:33.162 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 23.698486817999999, 2024-05-08T08:53:33.162 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086774741000000002, 2024-05-08T08:53:33.162 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:33.162 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:33.162 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:33.162 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:33.162 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:33.162 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:33.162 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 945 2024-05-08T08:53:33.162 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.162 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:33.163 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.163 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:33.163 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:53:33.163 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:33.163 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.163 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.163 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:33.163 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:53:33.163 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:33.163 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.163 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.163 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:33.163 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:53:33.163 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3499999999999996e-07 2024-05-08T08:53:33.164 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.164 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.164 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:33.164 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451024+0000", 2024-05-08T08:53:33.164 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.627e-06 2024-05-08T08:53:33.164 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.164 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.164 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:33.164 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451037+0000", 2024-05-08T08:53:33.164 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2431e-05 2024-05-08T08:53:33.164 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.164 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.164 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:33.164 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451042+0000", 2024-05-08T08:53:33.165 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8409999999999996e-06 2024-05-08T08:53:33.165 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.165 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.165 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:33.165 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537608+0000", 2024-05-08T08:53:33.165 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086565243 2024-05-08T08:53:33.165 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.165 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.165 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:33.165 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537687+0000", 2024-05-08T08:53:33.165 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9519999999999998e-05 2024-05-08T08:53:33.165 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.165 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.165 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:33.165 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537796+0000", 2024-05-08T08:53:33.166 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000108544 2024-05-08T08:53:33.166 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:33.166 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:33.166 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:33.166 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.166 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.166 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:964 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 458752~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:33.166 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451051+0000", 2024-05-08T08:53:33.166 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 23.698456645, 2024-05-08T08:53:33.166 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086990490000000004, 2024-05-08T08:53:33.166 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:33.166 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:33.166 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:33.166 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:33.166 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:33.167 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:33.167 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 964 2024-05-08T08:53:33.167 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.167 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:33.167 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.167 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:33.167 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451051+0000", 2024-05-08T08:53:33.167 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:33.167 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.167 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.167 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:33.167 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451051+0000", 2024-05-08T08:53:33.167 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:33.167 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.167 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.168 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:33.168 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451052+0000", 2024-05-08T08:53:33.168 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7299999999999996e-07 2024-05-08T08:53:33.168 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.168 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.168 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:33.168 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451055+0000", 2024-05-08T08:53:33.168 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.2770000000000001e-06 2024-05-08T08:53:33.168 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.168 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.168 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:33.168 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451061+0000", 2024-05-08T08:53:33.168 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4060000000000002e-06 2024-05-08T08:53:33.168 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.168 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.169 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:33.169 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451066+0000", 2024-05-08T08:53:33.169 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0839999999999999e-06 2024-05-08T08:53:33.169 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.169 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.169 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:33.169 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537839+0000", 2024-05-08T08:53:33.169 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086772962999999995 2024-05-08T08:53:33.169 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.169 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.169 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:33.169 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537934+0000", 2024-05-08T08:53:33.169 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4742999999999997e-05 2024-05-08T08:53:33.169 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.170 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.170 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:33.170 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538042+0000", 2024-05-08T08:53:33.170 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000107444 2024-05-08T08:53:33.170 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:33.170 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:33.170 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:33.170 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.170 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.170 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:965 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 950272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:33.170 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451075+0000", 2024-05-08T08:53:33.170 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 23.698432582999999, 2024-05-08T08:53:33.170 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087230196999999995, 2024-05-08T08:53:33.170 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:33.171 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:33.171 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:33.171 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:33.171 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:33.171 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:33.171 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 965 2024-05-08T08:53:33.171 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.171 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:33.171 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.171 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:33.171 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451075+0000", 2024-05-08T08:53:33.171 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:33.171 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.171 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.171 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:33.172 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451075+0000", 2024-05-08T08:53:33.172 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:33.172 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.172 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.172 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:33.172 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451076+0000", 2024-05-08T08:53:33.172 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5000000000000002e-07 2024-05-08T08:53:33.172 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.172 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.172 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:33.172 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451078+0000", 2024-05-08T08:53:33.172 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.5639999999999999e-06 2024-05-08T08:53:33.172 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.172 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.172 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:33.172 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451085+0000", 2024-05-08T08:53:33.172 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5640000000000002e-06 2024-05-08T08:53:33.173 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.173 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.173 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:33.173 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451090+0000", 2024-05-08T08:53:33.173 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1699999999999996e-06 2024-05-08T08:53:33.173 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.173 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.173 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:33.173 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538106+0000", 2024-05-08T08:53:33.173 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087015888999999999 2024-05-08T08:53:33.173 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.173 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.173 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:33.173 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538190+0000", 2024-05-08T08:53:33.173 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3961999999999993e-05 2024-05-08T08:53:33.173 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.173 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.174 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:33.174 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538305+0000", 2024-05-08T08:53:33.174 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011529800000000001 2024-05-08T08:53:33.174 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:33.174 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:33.174 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:33.174 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.174 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.174 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:966 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 1441792~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:33.174 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:53:33.174 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 23.698409210000001, 2024-05-08T08:53:33.174 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087446323000000006, 2024-05-08T08:53:33.174 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:33.175 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:33.175 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:33.175 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:33.175 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:33.175 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:33.175 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 966 2024-05-08T08:53:33.175 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.175 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:33.175 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.175 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:33.175 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:53:33.175 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:33.175 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.175 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.175 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:33.176 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:53:33.176 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:33.176 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.176 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.176 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:33.176 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:53:33.176 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3700000000000001e-07 2024-05-08T08:53:33.176 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.176 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.176 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:33.176 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451101+0000", 2024-05-08T08:53:33.176 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2560000000000001e-06 2024-05-08T08:53:33.176 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.176 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.177 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:33.177 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451107+0000", 2024-05-08T08:53:33.177 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1070000000000001e-06 2024-05-08T08:53:33.177 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.177 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.177 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:33.177 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451112+0000", 2024-05-08T08:53:33.177 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1420000000000001e-06 2024-05-08T08:53:33.177 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.177 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.177 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:33.177 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538341+0000", 2024-05-08T08:53:33.177 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087228302999999993 2024-05-08T08:53:33.177 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.177 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.178 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:33.178 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538424+0000", 2024-05-08T08:53:33.178 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3497000000000004e-05 2024-05-08T08:53:33.178 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.178 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.178 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:33.178 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538545+0000", 2024-05-08T08:53:33.178 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000120581 2024-05-08T08:53:33.178 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:33.178 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:33.178 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:33.178 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.178 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.178 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:967 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:33.178 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451120+0000", 2024-05-08T08:53:33.179 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 23.698387735000001, 2024-05-08T08:53:33.179 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087632725999999994, 2024-05-08T08:53:33.179 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:33.179 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:33.179 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:33.179 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:33.179 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:33.179 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:33.179 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 967 2024-05-08T08:53:33.179 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.179 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:33.179 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.179 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:33.179 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451120+0000", 2024-05-08T08:53:33.180 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:33.180 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.180 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.180 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:33.180 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451120+0000", 2024-05-08T08:53:33.180 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:33.180 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.180 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.180 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:33.180 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451121+0000", 2024-05-08T08:53:33.180 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8800000000000002e-07 2024-05-08T08:53:33.180 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.180 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.180 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:33.180 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451130+0000", 2024-05-08T08:53:33.180 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2299999999999997e-06 2024-05-08T08:53:33.181 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.181 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.181 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:33.181 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451137+0000", 2024-05-08T08:53:33.181 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4669999999999998e-06 2024-05-08T08:53:33.181 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.181 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.181 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:33.181 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451143+0000", 2024-05-08T08:53:33.181 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2839999999999997e-06 2024-05-08T08:53:33.181 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.181 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.181 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:33.181 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538575+0000", 2024-05-08T08:53:33.182 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087432789999999996 2024-05-08T08:53:33.182 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.182 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.182 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:33.182 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538648+0000", 2024-05-08T08:53:33.182 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2877e-05 2024-05-08T08:53:33.182 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.182 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.182 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:33.182 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538753+0000", 2024-05-08T08:53:33.182 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010449 2024-05-08T08:53:33.182 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:33.182 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:33.182 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:33.182 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.183 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.183 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:968 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 2424832~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:33.183 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451152+0000", 2024-05-08T08:53:33.183 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 23.69835565, 2024-05-08T08:53:33.183 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087813325999999997, 2024-05-08T08:53:33.183 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:33.183 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:33.183 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:33.183 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:33.183 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:33.183 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:33.183 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 968 2024-05-08T08:53:33.183 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.183 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:33.183 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.184 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:33.184 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451152+0000", 2024-05-08T08:53:33.184 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:33.184 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.184 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.184 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:33.184 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451152+0000", 2024-05-08T08:53:33.184 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:33.184 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.184 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.184 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:33.184 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451153+0000", 2024-05-08T08:53:33.184 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3099999999999998e-07 2024-05-08T08:53:33.184 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.185 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.185 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:33.185 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451155+0000", 2024-05-08T08:53:33.185 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4760000000000001e-06 2024-05-08T08:53:33.185 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.185 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.185 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:33.185 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451162+0000", 2024-05-08T08:53:33.185 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5969999999999997e-06 2024-05-08T08:53:33.185 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.185 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.185 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:33.185 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451167+0000", 2024-05-08T08:53:33.185 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.977e-06 2024-05-08T08:53:33.185 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.186 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.186 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:33.186 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538781+0000", 2024-05-08T08:53:33.186 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087614846999999996 2024-05-08T08:53:33.186 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.186 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.186 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:33.186 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538857+0000", 2024-05-08T08:53:33.186 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5358000000000006e-05 2024-05-08T08:53:33.186 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.186 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.186 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:33.186 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538965+0000", 2024-05-08T08:53:33.186 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010864 2024-05-08T08:53:33.186 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:33.187 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:33.187 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:33.187 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.187 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.187 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:969 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 2916352~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:33.187 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:53:33.187 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 23.698333945000002, 2024-05-08T08:53:33.187 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087998980000000004, 2024-05-08T08:53:33.187 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:33.187 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:33.187 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:33.187 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:33.187 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:33.187 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:33.188 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 969 2024-05-08T08:53:33.188 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.188 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:33.188 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.188 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:33.188 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:53:33.188 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:33.188 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.188 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.188 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:33.188 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:53:33.188 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:33.188 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.188 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.188 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:33.189 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:53:33.189 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3599999999999999e-07 2024-05-08T08:53:33.189 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.189 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.189 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:33.189 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451177+0000", 2024-05-08T08:53:33.189 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4590000000000001e-06 2024-05-08T08:53:33.189 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.189 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.189 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:33.189 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451184+0000", 2024-05-08T08:53:33.189 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4109999999999999e-06 2024-05-08T08:53:33.189 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.189 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.189 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:33.190 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451191+0000", 2024-05-08T08:53:33.190 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7279999999999998e-06 2024-05-08T08:53:33.190 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.190 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.190 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:33.190 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538997+0000", 2024-05-08T08:53:33.190 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087806170000000003 2024-05-08T08:53:33.190 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.190 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.190 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:33.190 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539069+0000", 2024-05-08T08:53:33.190 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2373999999999995e-05 2024-05-08T08:53:33.190 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.190 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.190 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:33.191 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539173+0000", 2024-05-08T08:53:33.191 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000103402 2024-05-08T08:53:33.191 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:33.191 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:33.191 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:33.191 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.191 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.191 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:970 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 3407872~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:33.191 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451198+0000", 2024-05-08T08:53:33.191 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 23.698309347999999, 2024-05-08T08:53:33.191 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088183368999999998, 2024-05-08T08:53:33.191 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:33.191 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:33.191 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:33.192 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:33.192 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:33.192 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:33.192 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 970 2024-05-08T08:53:33.192 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.192 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:33.192 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.192 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:33.192 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451198+0000", 2024-05-08T08:53:33.192 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:33.192 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.192 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.192 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:33.192 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451198+0000", 2024-05-08T08:53:33.193 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:33.193 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.193 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.193 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:33.193 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451199+0000", 2024-05-08T08:53:33.193 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7400000000000003e-07 2024-05-08T08:53:33.193 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.193 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.193 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:33.193 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451211+0000", 2024-05-08T08:53:33.193 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2109999999999999e-05 2024-05-08T08:53:33.193 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.193 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.193 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:33.193 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451219+0000", 2024-05-08T08:53:33.194 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4170000000000003e-06 2024-05-08T08:53:33.194 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.194 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.194 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:33.194 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451224+0000", 2024-05-08T08:53:33.194 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3990000000000003e-06 2024-05-08T08:53:33.194 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.194 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.194 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:33.194 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539204+0000", 2024-05-08T08:53:33.194 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087980099000000006 2024-05-08T08:53:33.194 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.194 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.194 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:33.195 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539276+0000", 2024-05-08T08:53:33.195 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1513000000000005e-05 2024-05-08T08:53:33.195 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.195 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.195 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:33.195 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539382+0000", 2024-05-08T08:53:33.195 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010615699999999999 2024-05-08T08:53:33.195 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:33.195 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:33.195 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:33.195 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.195 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.195 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:971 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 3899392~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:33.195 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451233+0000", 2024-05-08T08:53:33.195 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 23.698274641000001, 2024-05-08T08:53:33.196 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088431525999999996, 2024-05-08T08:53:33.196 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:33.196 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:33.196 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:33.196 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:33.196 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:33.196 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:33.196 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 971 2024-05-08T08:53:33.196 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.196 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:33.196 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.196 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:33.196 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451233+0000", 2024-05-08T08:53:33.196 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:33.196 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.197 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.197 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:33.197 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451233+0000", 2024-05-08T08:53:33.197 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:33.197 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.197 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.197 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:33.197 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451234+0000", 2024-05-08T08:53:33.197 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8500000000000001e-07 2024-05-08T08:53:33.197 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.197 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.197 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:33.197 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451237+0000", 2024-05-08T08:53:33.197 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8320000000000002e-06 2024-05-08T08:53:33.197 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.197 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.198 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:33.198 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451243+0000", 2024-05-08T08:53:33.198 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6749999999999996e-06 2024-05-08T08:53:33.198 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.198 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.198 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:33.198 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451249+0000", 2024-05-08T08:53:33.198 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1229999999999997e-06 2024-05-08T08:53:33.198 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.198 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.198 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:33.198 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539412+0000", 2024-05-08T08:53:33.198 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088162668999999999 2024-05-08T08:53:33.198 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.199 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.199 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:33.199 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539505+0000", 2024-05-08T08:53:33.199 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3240999999999997e-05 2024-05-08T08:53:33.199 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.199 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.199 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:33.199 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539665+0000", 2024-05-08T08:53:33.199 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00015930100000000001 2024-05-08T08:53:33.199 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:33.199 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:33.199 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:33.199 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.199 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.199 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:972 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 196608~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:33.200 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451257+0000", 2024-05-08T08:53:33.200 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 23.698250695999999, 2024-05-08T08:53:33.200 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088719928000000003, 2024-05-08T08:53:33.200 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:33.200 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:33.200 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:33.200 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:33.200 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:33.200 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:33.200 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 972 2024-05-08T08:53:33.200 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.200 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:33.200 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.200 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:33.201 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451257+0000", 2024-05-08T08:53:33.201 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:33.201 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.201 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.201 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:33.201 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451257+0000", 2024-05-08T08:53:33.201 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:33.201 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.201 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.201 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:33.201 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451258+0000", 2024-05-08T08:53:33.201 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7999999999999996e-07 2024-05-08T08:53:33.201 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.201 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.201 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:33.202 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451260+0000", 2024-05-08T08:53:33.202 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4609999999999998e-06 2024-05-08T08:53:33.202 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.202 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.202 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:33.202 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451267+0000", 2024-05-08T08:53:33.202 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.81e-06 2024-05-08T08:53:33.202 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.202 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.202 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:33.202 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451273+0000", 2024-05-08T08:53:33.202 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0859999999999998e-06 2024-05-08T08:53:33.202 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.202 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.203 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:33.203 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539718+0000", 2024-05-08T08:53:33.203 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088445059000000006 2024-05-08T08:53:33.203 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.203 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.203 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:33.203 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539834+0000", 2024-05-08T08:53:33.203 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000116426 2024-05-08T08:53:33.203 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.203 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.203 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:33.203 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539977+0000", 2024-05-08T08:53:33.203 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014260600000000001 2024-05-08T08:53:33.203 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:33.203 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:33.204 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:33.204 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.204 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.204 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:973 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 688128~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:33.204 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:53:33.204 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 23.698228111999999, 2024-05-08T08:53:33.204 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088918496, 2024-05-08T08:53:33.204 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:33.204 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:33.204 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:33.204 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:33.204 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:33.204 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:33.204 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 973 2024-05-08T08:53:33.204 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.205 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:33.205 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.205 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:33.205 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:53:33.205 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:33.205 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.205 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.205 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:33.205 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:53:33.205 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:33.205 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.205 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.205 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:33.205 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:53:33.205 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9099999999999999e-07 2024-05-08T08:53:33.206 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.206 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.206 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:33.206 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451282+0000", 2024-05-08T08:53:33.206 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1550000000000001e-06 2024-05-08T08:53:33.206 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.206 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.206 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:33.206 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451288+0000", 2024-05-08T08:53:33.206 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8889999999999999e-06 2024-05-08T08:53:33.206 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.206 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.206 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:33.206 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451294+0000", 2024-05-08T08:53:33.206 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0850000000000002e-06 2024-05-08T08:53:33.207 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.207 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.207 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:33.207 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540011+0000", 2024-05-08T08:53:33.207 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088717314000000005 2024-05-08T08:53:33.207 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.207 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.207 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:33.207 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540089+0000", 2024-05-08T08:53:33.207 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7438e-05 2024-05-08T08:53:33.207 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.207 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.207 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:33.207 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540198+0000", 2024-05-08T08:53:33.207 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000109224 2024-05-08T08:53:33.208 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:33.208 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:33.208 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:33.208 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.208 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.208 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:974 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 1179648~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:33.208 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451301+0000", 2024-05-08T08:53:33.208 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 23.698206479, 2024-05-08T08:53:33.208 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089105078000000004, 2024-05-08T08:53:33.208 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:33.208 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:33.208 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:33.208 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:33.208 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:33.209 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:33.209 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 974 2024-05-08T08:53:33.209 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.209 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:33.209 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.209 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:33.209 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451301+0000", 2024-05-08T08:53:33.209 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:33.209 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.209 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.209 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:33.209 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451301+0000", 2024-05-08T08:53:33.209 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:33.209 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.209 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.209 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:33.210 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451302+0000", 2024-05-08T08:53:33.210 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7800000000000002e-07 2024-05-08T08:53:33.210 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.210 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.210 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:33.210 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451304+0000", 2024-05-08T08:53:33.210 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.65e-06 2024-05-08T08:53:33.210 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.210 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.210 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:33.210 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451311+0000", 2024-05-08T08:53:33.210 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.472e-06 2024-05-08T08:53:33.210 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.210 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.210 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:33.211 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451316+0000", 2024-05-08T08:53:33.211 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9189999999999998e-06 2024-05-08T08:53:33.211 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.211 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.211 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:33.211 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540234+0000", 2024-05-08T08:53:33.211 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088917752000000003 2024-05-08T08:53:33.211 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.211 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.211 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:33.211 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540308+0000", 2024-05-08T08:53:33.211 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4758000000000005e-05 2024-05-08T08:53:33.211 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.211 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.212 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:33.212 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540406+0000", 2024-05-08T08:53:33.212 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8048999999999996e-05 2024-05-08T08:53:33.212 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:33.212 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:33.212 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:33.212 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.212 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.212 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:975 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 1671168~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:33.212 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:53:33.212 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 23.698164149, 2024-05-08T08:53:33.212 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089294139999999994, 2024-05-08T08:53:33.212 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:33.212 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:33.212 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:33.213 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:33.213 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:33.213 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:33.213 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 975 2024-05-08T08:53:33.213 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.213 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:33.213 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.213 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:33.213 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:53:33.213 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:33.213 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.213 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.213 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:33.213 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:53:33.213 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:33.214 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.214 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.214 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:33.214 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:53:33.214 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1299999999999999e-07 2024-05-08T08:53:33.214 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.214 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.214 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:33.214 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451348+0000", 2024-05-08T08:53:33.214 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.484e-06 2024-05-08T08:53:33.214 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.214 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.214 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:33.214 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451356+0000", 2024-05-08T08:53:33.215 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8499999999999994e-06 2024-05-08T08:53:33.215 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.215 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.215 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:33.215 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451366+0000", 2024-05-08T08:53:33.215 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.9850000000000001e-06 2024-05-08T08:53:33.215 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.215 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.215 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:33.215 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540448+0000", 2024-05-08T08:53:33.215 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089082254 2024-05-08T08:53:33.215 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.215 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.215 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:33.215 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540528+0000", 2024-05-08T08:53:33.216 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0234999999999993e-05 2024-05-08T08:53:33.216 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.216 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.216 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:33.216 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540638+0000", 2024-05-08T08:53:33.216 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010951900000000001 2024-05-08T08:53:33.216 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:33.216 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:33.216 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:33.216 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.216 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.216 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:976 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 2162688~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:33.216 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451375+0000", 2024-05-08T08:53:33.216 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 23.698132401999999, 2024-05-08T08:53:33.217 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089472745000000006, 2024-05-08T08:53:33.217 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:33.217 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:33.217 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:33.217 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:33.217 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:33.217 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:33.217 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 976 2024-05-08T08:53:33.217 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.217 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:33.217 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.217 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:33.217 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451375+0000", 2024-05-08T08:53:33.217 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:33.217 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.218 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.218 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:33.218 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451375+0000", 2024-05-08T08:53:33.218 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:33.218 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.218 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.218 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:33.218 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451376+0000", 2024-05-08T08:53:33.218 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2399999999999998e-07 2024-05-08T08:53:33.218 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.218 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.218 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:33.218 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451379+0000", 2024-05-08T08:53:33.218 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6359999999999998e-06 2024-05-08T08:53:33.218 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.219 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.219 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:33.219 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451387+0000", 2024-05-08T08:53:33.219 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9929999999999995e-06 2024-05-08T08:53:33.219 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.219 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.219 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:33.219 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451393+0000", 2024-05-08T08:53:33.219 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4699999999999999e-06 2024-05-08T08:53:33.219 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.219 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.219 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:33.219 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540677+0000", 2024-05-08T08:53:33.219 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089283504999999999 2024-05-08T08:53:33.219 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.220 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.220 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:33.220 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540743+0000", 2024-05-08T08:53:33.220 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6303999999999994e-05 2024-05-08T08:53:33.220 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.220 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.220 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:33.220 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540848+0000", 2024-05-08T08:53:33.220 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000105213 2024-05-08T08:53:33.220 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:33.220 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:33.220 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:33.220 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.220 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.220 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:977 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 2654208~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:33.220 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:53:33.221 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 23.698106970000001, 2024-05-08T08:53:33.221 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089698473000000001, 2024-05-08T08:53:33.221 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:33.221 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:33.221 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:33.221 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:33.221 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:33.221 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:33.221 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 977 2024-05-08T08:53:33.221 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.221 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:33.221 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.221 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:33.221 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:53:33.222 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:33.222 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.222 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.222 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:33.222 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:53:33.222 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:33.222 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.222 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.222 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:33.222 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:53:33.222 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2899999999999999e-07 2024-05-08T08:53:33.222 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.222 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.222 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:33.222 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451404+0000", 2024-05-08T08:53:33.223 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6649999999999999e-06 2024-05-08T08:53:33.223 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.223 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.223 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:33.223 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451410+0000", 2024-05-08T08:53:33.223 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3269999999999997e-06 2024-05-08T08:53:33.223 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.223 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.223 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:33.223 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451415+0000", 2024-05-08T08:53:33.223 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9529999999999998e-06 2024-05-08T08:53:33.223 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.223 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.223 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:33.223 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540912+0000", 2024-05-08T08:53:33.224 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089496950000000006 2024-05-08T08:53:33.224 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.224 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.224 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:33.224 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541000+0000", 2024-05-08T08:53:33.224 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7429000000000006e-05 2024-05-08T08:53:33.224 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.224 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.224 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:33.224 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541099+0000", 2024-05-08T08:53:33.224 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.9720000000000001e-05 2024-05-08T08:53:33.224 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:33.224 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:33.224 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:33.224 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.224 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.225 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:978 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 3145728~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:33.225 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:53:33.225 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 23.698085762000002, 2024-05-08T08:53:33.225 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089900635000000007, 2024-05-08T08:53:33.225 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:33.225 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:33.225 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:33.225 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:33.225 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:33.225 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:33.225 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 978 2024-05-08T08:53:33.225 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.225 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:33.225 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.225 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:33.226 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:53:33.226 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:33.226 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.226 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.226 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:33.226 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:53:33.226 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:33.226 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.226 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.226 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:33.226 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:53:33.226 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9400000000000001e-07 2024-05-08T08:53:33.226 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.226 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.227 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:33.227 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451425+0000", 2024-05-08T08:53:33.227 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2460000000000002e-06 2024-05-08T08:53:33.227 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.227 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.227 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:33.227 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451431+0000", 2024-05-08T08:53:33.227 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2419999999999997e-06 2024-05-08T08:53:33.227 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.227 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.227 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:33.227 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451436+0000", 2024-05-08T08:53:33.227 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.887e-06 2024-05-08T08:53:33.227 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.228 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.228 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:33.228 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541152+0000", 2024-05-08T08:53:33.228 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089716194999999999 2024-05-08T08:53:33.228 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.228 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.228 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:33.228 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541224+0000", 2024-05-08T08:53:33.228 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2416000000000006e-05 2024-05-08T08:53:33.228 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.228 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.228 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:33.228 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541323+0000", 2024-05-08T08:53:33.228 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8255e-05 2024-05-08T08:53:33.229 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:33.229 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:33.229 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:33.229 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.229 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.229 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:979 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 3637248~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:33.229 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451462+0000", 2024-05-08T08:53:33.229 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 23.698045313000001, 2024-05-08T08:53:33.229 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.090036441999999994, 2024-05-08T08:53:33.229 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:33.229 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:33.229 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:33.229 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:33.230 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:33.230 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:33.230 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 979 2024-05-08T08:53:33.230 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.230 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:33.230 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.230 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:33.230 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451462+0000", 2024-05-08T08:53:33.230 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:33.230 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.230 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.230 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:33.230 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451462+0000", 2024-05-08T08:53:33.230 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:33.231 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.231 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.231 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:33.231 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451463+0000", 2024-05-08T08:53:33.231 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4300000000000002e-07 2024-05-08T08:53:33.231 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.231 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.231 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:33.231 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451471+0000", 2024-05-08T08:53:33.231 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0800000000000006e-06 2024-05-08T08:53:33.231 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.231 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.231 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:33.231 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451481+0000", 2024-05-08T08:53:33.231 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.38e-06 2024-05-08T08:53:33.232 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.232 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.232 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:33.232 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451487+0000", 2024-05-08T08:53:33.232 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4590000000000003e-06 2024-05-08T08:53:33.232 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.232 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.232 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:33.232 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541350+0000", 2024-05-08T08:53:33.232 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089862599000000001 2024-05-08T08:53:33.232 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.232 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.232 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:33.232 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541407+0000", 2024-05-08T08:53:33.232 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7667999999999997e-05 2024-05-08T08:53:33.232 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.233 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.233 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:33.233 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541499+0000", 2024-05-08T08:53:33.233 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1513000000000003e-05 2024-05-08T08:53:33.233 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:33.233 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:33.233 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:33.233 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.233 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.233 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:980 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 4128768~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:33.233 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451496+0000", 2024-05-08T08:53:33.233 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 23.698011557000001, 2024-05-08T08:53:33.233 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.090258985, 2024-05-08T08:53:33.233 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:33.234 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:33.234 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:33.234 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:33.234 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:33.234 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:33.234 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 980 2024-05-08T08:53:33.234 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.234 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:33.234 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.234 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:33.234 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451496+0000", 2024-05-08T08:53:33.234 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:33.234 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.234 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.234 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:33.235 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451496+0000", 2024-05-08T08:53:33.235 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:33.235 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.235 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.235 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:33.235 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451497+0000", 2024-05-08T08:53:33.235 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7599999999999997e-07 2024-05-08T08:53:33.235 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.235 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.235 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:33.235 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451500+0000", 2024-05-08T08:53:33.235 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7930000000000002e-06 2024-05-08T08:53:33.235 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.235 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.236 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:33.236 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451506+0000", 2024-05-08T08:53:33.236 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6980000000000001e-06 2024-05-08T08:53:33.236 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.236 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.236 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:33.236 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451512+0000", 2024-05-08T08:53:33.236 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2490000000000001e-06 2024-05-08T08:53:33.236 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.236 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.236 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:33.236 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541550+0000", 2024-05-08T08:53:33.236 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.090037961999999999 2024-05-08T08:53:33.236 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.236 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.237 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:33.237 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541647+0000", 2024-05-08T08:53:33.237 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6924999999999998e-05 2024-05-08T08:53:33.237 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:33.237 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:33.237 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:33.237 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541755+0000", 2024-05-08T08:53:33.237 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000108682 2024-05-08T08:53:33.237 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:33.237 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:33.237 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:33.237 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:33.237 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:33.237 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:33.238 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T08:53:33.240 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:53:33.240 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:33.242+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:33.340 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:33.343+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:33.348 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:53:33.348 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:53:33.348 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:33.348 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5371.0:8839 3.13 3.95e3f9f3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:33.348 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:33.351944+0000", 2024-05-08T08:53:33.349 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 2.7427999999999999e-05, 2024-05-08T08:53:33.349 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1303e-05, 2024-05-08T08:53:33.349 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:33.349 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:33.349 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-05-08T08:53:33.349 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:33.349 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5371", 2024-05-08T08:53:33.349 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/1127237996", 2024-05-08T08:53:33.349 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 8839 2024-05-08T08:53:33.349 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:33.349 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:33.349 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:33.349 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:33.349 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:33.351944+0000", 2024-05-08T08:53:33.349 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:33.350 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:33.350 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:33.350 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:33.350 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:33.351944+0000", 2024-05-08T08:53:33.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:33.350 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:33.350 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:33.350 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:33.350 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:33.351944+0000", 2024-05-08T08:53:33.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8599999999999998e-07 2024-05-08T08:53:33.350 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:33.350 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:33.350 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:33.350 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:33.351948+0000", 2024-05-08T08:53:33.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4929999999999998e-06 2024-05-08T08:53:33.351 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:33.351 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:33.351 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:33.351 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:33.351960+0000", 2024-05-08T08:53:33.351 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1868000000000001e-05 2024-05-08T08:53:33.351 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:33.351 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:33.351 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:33.351 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:33.351968+0000", 2024-05-08T08:53:33.351 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0630000000000006e-06 2024-05-08T08:53:33.351 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:33.351 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:33.351 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:33.351 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:33.351 INFO:teuthology.orchestra.run.smithi192.stdout: ], 2024-05-08T08:53:33.351 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 1 2024-05-08T08:53:33.352 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:53:33.357 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:53:33.441 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:33.444+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:33.473 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:53:33.474 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:53:33.507 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:53:33.507 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:53:33.524 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:53:33.524 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:53:33.524 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:53:33.524 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:53:33.524 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:53:33.526 INFO:teuthology.orchestra.run.smithi059.stderr:nodeep-scrub is unset 2024-05-08T08:53:33.535 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:53:33.537 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:53:33.538 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:53:33.542 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:33.545+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:33.572 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:53:33.573 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:53:33.606 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:53:33.607 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:53:33.626 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:53:33.626 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:53:33.643 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:33.646+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:33.692 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:33.692 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:33.702 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:53:33.744 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:33.747+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:33.845 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:33.848+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:33.863 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:33.863 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:33.874 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:53:33.945 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:33.948+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:34.006 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:34.006 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:34.017 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:53:34.058 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:34.061+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:34.148 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:34.151+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:34.201 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:34.201 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:34.212 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:53:34.254 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:34.257+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:34.350 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:34.353+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:34.397 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:34.397 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:34.407 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:53:34.494 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:34.494+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:34.551 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:34.554+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:34.552 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:34.552 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:34.561 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:53:34.614 INFO:teuthology.orchestra.run.smithi196.stdout:3239 write 0xc97ee00 thru 0xc98b5d0 (0xc7d1 bytes) 2024-05-08T08:53:34.626 INFO:teuthology.orchestra.run.smithi196.stdout:3240 trunc from 0xe512a00 to 0x4c1a000 2024-05-08T08:53:34.653 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:34.655+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:34.739 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:34.740 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:34.750 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:53:34.762 INFO:teuthology.orchestra.run.smithi196.stdout:3241 read 0x36fd000 thru 0x3700641 (0x3642 bytes) 2024-05-08T08:53:34.763 INFO:teuthology.orchestra.run.smithi196.stdout:3242 read 0x4426a00 thru 0x44288a6 (0x1ea7 bytes) 2024-05-08T08:53:34.763 INFO:teuthology.orchestra.run.smithi196.stdout:3243 read 0x1b72e00 thru 0x1b7302f (0x230 bytes) 2024-05-08T08:53:34.763 INFO:teuthology.orchestra.run.smithi196.stdout:3245 read 0x19ad400 thru 0x19b796e (0xa56f bytes) 2024-05-08T08:53:34.764 INFO:teuthology.orchestra.run.smithi196.stdout:3249 read 0x3fcee00 thru 0x3fdcc1c (0xde1d bytes) 2024-05-08T08:53:34.764 INFO:teuthology.orchestra.run.smithi196.stdout:3254 write 0x2472c00 thru 0x247e4f3 (0xb8f4 bytes) 2024-05-08T08:53:34.765 INFO:teuthology.orchestra.run.smithi196.stdout:3255 punch from 0x12c7800 to 0x12cbc00, (0x4400 bytes) 2024-05-08T08:53:34.791 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:34.794+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:34.854 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:34.857+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:34.902 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:34.903 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:34.903 INFO:teuthology.orchestra.run.smithi196.stdout:3258 write 0xede2e00 thru 0xede4ddb (0x1fdc bytes) 2024-05-08T08:53:34.912 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:53:34.920 INFO:teuthology.orchestra.run.smithi196.stdout:3260 trunc from 0xede4e00 to 0xdc2600 2024-05-08T08:53:34.958 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:34.960+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:35.056 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:35.059+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:35.132 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:35.132 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:35.144 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:53:35.157 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:35.159+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:35.258 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:35.261+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:35.318 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:35.318 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:35.331 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:53:35.359 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:35.362+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:35.371 INFO:teuthology.orchestra.run.smithi196.stdout:3261 punch from 0x53da00 to 0x53fa00, (0x2000 bytes) 2024-05-08T08:53:35.398 INFO:teuthology.orchestra.run.smithi196.stdout:3263 write 0xd5f6400 thru 0xd600d64 (0xa965 bytes) 2024-05-08T08:53:35.399 INFO:teuthology.orchestra.run.smithi196.stdout:3268 write 0x67ba800 thru 0x67c95b0 (0xedb1 bytes) 2024-05-08T08:53:35.415 INFO:teuthology.orchestra.run.smithi196.stdout:3270 write 0xed35800 thru 0xed4543f (0xfc40 bytes) 2024-05-08T08:53:35.455 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:35.455 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:35.459 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:53:35.462+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:53:35.463 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:53:35.547 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-05-08T08:53:35.560 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:35.563+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:35.617 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:35.617 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:35.619 INFO:teuthology.orchestra.run.smithi196.stdout:3271 punch from 0xaa56c00 to 0xaa60800, (0x9c00 bytes) 2024-05-08T08:53:35.620 INFO:teuthology.orchestra.run.smithi196.stdout:3273 punch from 0x1be1a00 to 0x1beca00, (0xb000 bytes) 2024-05-08T08:53:35.620 INFO:teuthology.orchestra.run.smithi196.stdout:3274 clone 24 order 18 su 65536 sc 6 2024-05-08T08:53:35.629 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:53:35.637 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T08:53:35.662 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:35.665+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:35.738 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 5, 4] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-05-08T08:53:35.738 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T08:53:35.738 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-05-08T08:53:35.738 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-05-08T08:53:35.741 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:35.742 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:35.752 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:53:35.763 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:35.766+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:35.801 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:53:35.801 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:53:35.838 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:53:35.838 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:53:35.864 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:35.867+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:35.865 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:53:35.865 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:53:35.886 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:53:35.886 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:53:35.905 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:53:35.905 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:53:35.907 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:35.907 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:53:35.907 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:53:35.907 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:35.917 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:53:35.923 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:53:35.924 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:53:35.965 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:35.968+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:36.034 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:36.034 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:53:36.034 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:53:36.035 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:53:36.035 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:36.043 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:53:36.067 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:53:36.067 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:36.067 INFO:teuthology.orchestra.run.smithi059.stdout: "epoch": 79, 2024-05-08T08:53:36.067 INFO:teuthology.orchestra.run.smithi059.stdout: "fsid": "38dfae8c-5c5e-4b0f-a264-91f22c6c5554", 2024-05-08T08:53:36.067 INFO:teuthology.orchestra.run.smithi059.stdout: "created": "2024-05-08T08:51:45.571369+0000", 2024-05-08T08:53:36.067 INFO:teuthology.orchestra.run.smithi059.stdout: "modified": "2024-05-08T08:53:33.519336+0000", 2024-05-08T08:53:36.067 INFO:teuthology.orchestra.run.smithi059.stdout: "last_up_change": "2024-05-08T08:51:56.847706+0000", 2024-05-08T08:53:36.067 INFO:teuthology.orchestra.run.smithi059.stdout: "last_in_change": "2024-05-08T08:53:19.443905+0000", 2024-05-08T08:53:36.068 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-05-08T08:53:36.068 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_num": 5799936, 2024-05-08T08:53:36.068 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_set": [ 2024-05-08T08:53:36.068 INFO:teuthology.orchestra.run.smithi059.stdout: "pglog_hardlimit", 2024-05-08T08:53:36.068 INFO:teuthology.orchestra.run.smithi059.stdout: "purged_snapdirs", 2024-05-08T08:53:36.068 INFO:teuthology.orchestra.run.smithi059.stdout: "recovery_deletes", 2024-05-08T08:53:36.068 INFO:teuthology.orchestra.run.smithi059.stdout: "sortbitwise" 2024-05-08T08:53:36.068 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T08:53:36.068 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_version": 4, 2024-05-08T08:53:36.068 INFO:teuthology.orchestra.run.smithi059.stdout: "full_ratio": 0.94999998807907104, 2024-05-08T08:53:36.068 INFO:teuthology.orchestra.run.smithi059.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-05-08T08:53:36.068 INFO:teuthology.orchestra.run.smithi059.stdout: "nearfull_ratio": 0.85000002384185791, 2024-05-08T08:53:36.068 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_snapshot": "", 2024-05-08T08:53:36.068 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_max": 3, 2024-05-08T08:53:36.068 INFO:teuthology.orchestra.run.smithi059.stdout: "max_osd": 6, 2024-05-08T08:53:36.068 INFO:teuthology.orchestra.run.smithi059.stdout: "require_min_compat_client": "luminous", 2024-05-08T08:53:36.069 INFO:teuthology.orchestra.run.smithi059.stdout: "min_compat_client": "luminous", 2024-05-08T08:53:36.069 INFO:teuthology.orchestra.run.smithi059.stdout: "require_osd_release": "squid", 2024-05-08T08:53:36.069 INFO:teuthology.orchestra.run.smithi059.stdout: "allow_crimson": false, 2024-05-08T08:53:36.069 INFO:teuthology.orchestra.run.smithi059.stdout: "pools": [ 2024-05-08T08:53:36.069 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.069 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 1, 2024-05-08T08:53:36.069 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_name": ".mgr", 2024-05-08T08:53:36.069 INFO:teuthology.orchestra.run.smithi059.stdout: "create_time": "2024-05-08T08:51:57.787790+0000", 2024-05-08T08:53:36.069 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T08:53:36.069 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_names": "hashpspool", 2024-05-08T08:53:36.069 INFO:teuthology.orchestra.run.smithi059.stdout: "type": 1, 2024-05-08T08:53:36.069 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 2, 2024-05-08T08:53:36.069 INFO:teuthology.orchestra.run.smithi059.stdout: "min_size": 1, 2024-05-08T08:53:36.070 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_rule": 0, 2024-05-08T08:53:36.070 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_count": 0, 2024-05-08T08:53:36.070 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_target": 0, 2024-05-08T08:53:36.070 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_barrier": 0, 2024-05-08T08:53:36.070 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-08T08:53:36.070 INFO:teuthology.orchestra.run.smithi059.stdout: "object_hash": 2, 2024-05-08T08:53:36.070 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscale_mode": "off", 2024-05-08T08:53:36.070 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num": 1, 2024-05-08T08:53:36.070 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num": 1, 2024-05-08T08:53:36.070 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num_target": 1, 2024-05-08T08:53:36.070 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_target": 1, 2024-05-08T08:53:36.070 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_pending": 1, 2024-05-08T08:53:36.070 INFO:teuthology.orchestra.run.smithi059.stdout: "last_pg_merge_meta": { 2024-05-08T08:53:36.071 INFO:teuthology.orchestra.run.smithi059.stdout: "source_pgid": "0.0", 2024-05-08T08:53:36.071 INFO:teuthology.orchestra.run.smithi059.stdout: "ready_epoch": 0, 2024-05-08T08:53:36.071 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_started": 0, 2024-05-08T08:53:36.071 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_clean": 0, 2024-05-08T08:53:36.071 INFO:teuthology.orchestra.run.smithi059.stdout: "source_version": "0'0", 2024-05-08T08:53:36.071 INFO:teuthology.orchestra.run.smithi059.stdout: "target_version": "0'0" 2024-05-08T08:53:36.071 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.071 INFO:teuthology.orchestra.run.smithi059.stdout: "last_change": "14", 2024-05-08T08:53:36.071 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend": "0", 2024-05-08T08:53:36.071 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-08T08:53:36.071 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_preluminous": "0", 2024-05-08T08:53:36.071 INFO:teuthology.orchestra.run.smithi059.stdout: "auid": 0, 2024-05-08T08:53:36.072 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_mode": "selfmanaged", 2024-05-08T08:53:36.072 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_seq": 0, 2024-05-08T08:53:36.072 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_epoch": 0, 2024-05-08T08:53:36.072 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_snaps": [], 2024-05-08T08:53:36.072 INFO:teuthology.orchestra.run.smithi059.stdout: "removed_snaps": "[]", 2024-05-08T08:53:36.072 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_bytes": 0, 2024-05-08T08:53:36.072 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_objects": 0, 2024-05-08T08:53:36.072 INFO:teuthology.orchestra.run.smithi059.stdout: "tiers": [], 2024-05-08T08:53:36.072 INFO:teuthology.orchestra.run.smithi059.stdout: "tier_of": -1, 2024-05-08T08:53:36.072 INFO:teuthology.orchestra.run.smithi059.stdout: "read_tier": -1, 2024-05-08T08:53:36.072 INFO:teuthology.orchestra.run.smithi059.stdout: "write_tier": -1, 2024-05-08T08:53:36.072 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_mode": "none", 2024-05-08T08:53:36.072 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_bytes": 0, 2024-05-08T08:53:36.073 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_objects": 0, 2024-05-08T08:53:36.073 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-08T08:53:36.073 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-08T08:53:36.073 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-08T08:53:36.073 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_flush_age": 0, 2024-05-08T08:53:36.073 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_evict_age": 0, 2024-05-08T08:53:36.073 INFO:teuthology.orchestra.run.smithi059.stdout: "erasure_code_profile": "", 2024-05-08T08:53:36.073 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_params": { 2024-05-08T08:53:36.073 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "none" 2024-05-08T08:53:36.073 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.073 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_period": 0, 2024-05-08T08:53:36.073 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_count": 0, 2024-05-08T08:53:36.073 INFO:teuthology.orchestra.run.smithi059.stdout: "use_gmt_hitset": true, 2024-05-08T08:53:36.073 INFO:teuthology.orchestra.run.smithi059.stdout: "min_read_recency_for_promote": 0, 2024-05-08T08:53:36.074 INFO:teuthology.orchestra.run.smithi059.stdout: "min_write_recency_for_promote": 0, 2024-05-08T08:53:36.074 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_grade_decay_rate": 0, 2024-05-08T08:53:36.074 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_search_last_n": 0, 2024-05-08T08:53:36.074 INFO:teuthology.orchestra.run.smithi059.stdout: "grade_table": [], 2024-05-08T08:53:36.074 INFO:teuthology.orchestra.run.smithi059.stdout: "stripe_width": 0, 2024-05-08T08:53:36.074 INFO:teuthology.orchestra.run.smithi059.stdout: "expected_num_objects": 0, 2024-05-08T08:53:36.074 INFO:teuthology.orchestra.run.smithi059.stdout: "fast_read": false, 2024-05-08T08:53:36.074 INFO:teuthology.orchestra.run.smithi059.stdout: "options": { 2024-05-08T08:53:36.074 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_max": 32, 2024-05-08T08:53:36.074 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_min": 1 2024-05-08T08:53:36.074 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.074 INFO:teuthology.orchestra.run.smithi059.stdout: "application_metadata": { 2024-05-08T08:53:36.074 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": {} 2024-05-08T08:53:36.075 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.075 INFO:teuthology.orchestra.run.smithi059.stdout: "read_balance": { 2024-05-08T08:53:36.075 INFO:teuthology.orchestra.run.smithi059.stdout: "score_type": "Fair distribution", 2024-05-08T08:53:36.075 INFO:teuthology.orchestra.run.smithi059.stdout: "score_acting": 6.059999942779541, 2024-05-08T08:53:36.075 INFO:teuthology.orchestra.run.smithi059.stdout: "score_stable": 6.059999942779541, 2024-05-08T08:53:36.075 INFO:teuthology.orchestra.run.smithi059.stdout: "optimal_score": 0.33000001311302185, 2024-05-08T08:53:36.075 INFO:teuthology.orchestra.run.smithi059.stdout: "raw_score_acting": 2, 2024-05-08T08:53:36.075 INFO:teuthology.orchestra.run.smithi059.stdout: "raw_score_stable": 2, 2024-05-08T08:53:36.075 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity_weighted": 0.87819164991378784, 2024-05-08T08:53:36.075 INFO:teuthology.orchestra.run.smithi059.stdout: "average_primary_affinity": 0.87999999523162842, 2024-05-08T08:53:36.075 INFO:teuthology.orchestra.run.smithi059.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-05-08T08:53:36.075 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:36.075 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.076 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.076 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 2, 2024-05-08T08:53:36.076 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_name": "rbd", 2024-05-08T08:53:36.076 INFO:teuthology.orchestra.run.smithi059.stdout: "create_time": "2024-05-08T08:52:02.563677+0000", 2024-05-08T08:53:36.076 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T08:53:36.076 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_names": "hashpspool", 2024-05-08T08:53:36.076 INFO:teuthology.orchestra.run.smithi059.stdout: "type": 1, 2024-05-08T08:53:36.076 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 2, 2024-05-08T08:53:36.076 INFO:teuthology.orchestra.run.smithi059.stdout: "min_size": 1, 2024-05-08T08:53:36.076 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_rule": 0, 2024-05-08T08:53:36.076 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_count": 0, 2024-05-08T08:53:36.076 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_target": 0, 2024-05-08T08:53:36.076 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_barrier": 0, 2024-05-08T08:53:36.076 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-08T08:53:36.076 INFO:teuthology.orchestra.run.smithi059.stdout: "object_hash": 2, 2024-05-08T08:53:36.077 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscale_mode": "off", 2024-05-08T08:53:36.077 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num": 8, 2024-05-08T08:53:36.077 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num": 8, 2024-05-08T08:53:36.077 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num_target": 8, 2024-05-08T08:53:36.077 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_target": 8, 2024-05-08T08:53:36.077 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_pending": 8, 2024-05-08T08:53:36.077 INFO:teuthology.orchestra.run.smithi059.stdout: "last_pg_merge_meta": { 2024-05-08T08:53:36.077 INFO:teuthology.orchestra.run.smithi059.stdout: "source_pgid": "0.0", 2024-05-08T08:53:36.077 INFO:teuthology.orchestra.run.smithi059.stdout: "ready_epoch": 0, 2024-05-08T08:53:36.077 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_started": 0, 2024-05-08T08:53:36.077 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_clean": 0, 2024-05-08T08:53:36.077 INFO:teuthology.orchestra.run.smithi059.stdout: "source_version": "0'0", 2024-05-08T08:53:36.077 INFO:teuthology.orchestra.run.smithi059.stdout: "target_version": "0'0" 2024-05-08T08:53:36.077 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.078 INFO:teuthology.orchestra.run.smithi059.stdout: "last_change": "17", 2024-05-08T08:53:36.078 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend": "0", 2024-05-08T08:53:36.078 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-08T08:53:36.078 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_preluminous": "0", 2024-05-08T08:53:36.078 INFO:teuthology.orchestra.run.smithi059.stdout: "auid": 0, 2024-05-08T08:53:36.078 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_mode": "selfmanaged", 2024-05-08T08:53:36.078 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_seq": 0, 2024-05-08T08:53:36.078 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_epoch": 0, 2024-05-08T08:53:36.078 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_snaps": [], 2024-05-08T08:53:36.078 INFO:teuthology.orchestra.run.smithi059.stdout: "removed_snaps": "[]", 2024-05-08T08:53:36.078 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_bytes": 0, 2024-05-08T08:53:36.078 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_objects": 0, 2024-05-08T08:53:36.078 INFO:teuthology.orchestra.run.smithi059.stdout: "tiers": [], 2024-05-08T08:53:36.079 INFO:teuthology.orchestra.run.smithi059.stdout: "tier_of": -1, 2024-05-08T08:53:36.079 INFO:teuthology.orchestra.run.smithi059.stdout: "read_tier": -1, 2024-05-08T08:53:36.079 INFO:teuthology.orchestra.run.smithi059.stdout: "write_tier": -1, 2024-05-08T08:53:36.079 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_mode": "none", 2024-05-08T08:53:36.079 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_bytes": 0, 2024-05-08T08:53:36.079 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_objects": 0, 2024-05-08T08:53:36.079 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-08T08:53:36.079 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-08T08:53:36.079 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-08T08:53:36.079 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_flush_age": 0, 2024-05-08T08:53:36.079 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_evict_age": 0, 2024-05-08T08:53:36.079 INFO:teuthology.orchestra.run.smithi059.stdout: "erasure_code_profile": "", 2024-05-08T08:53:36.079 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_params": { 2024-05-08T08:53:36.079 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "none" 2024-05-08T08:53:36.080 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.080 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_period": 0, 2024-05-08T08:53:36.080 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_count": 0, 2024-05-08T08:53:36.080 INFO:teuthology.orchestra.run.smithi059.stdout: "use_gmt_hitset": true, 2024-05-08T08:53:36.080 INFO:teuthology.orchestra.run.smithi059.stdout: "min_read_recency_for_promote": 0, 2024-05-08T08:53:36.080 INFO:teuthology.orchestra.run.smithi059.stdout: "min_write_recency_for_promote": 0, 2024-05-08T08:53:36.080 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_grade_decay_rate": 0, 2024-05-08T08:53:36.080 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_search_last_n": 0, 2024-05-08T08:53:36.080 INFO:teuthology.orchestra.run.smithi059.stdout: "grade_table": [], 2024-05-08T08:53:36.080 INFO:teuthology.orchestra.run.smithi059.stdout: "stripe_width": 0, 2024-05-08T08:53:36.080 INFO:teuthology.orchestra.run.smithi059.stdout: "expected_num_objects": 0, 2024-05-08T08:53:36.080 INFO:teuthology.orchestra.run.smithi059.stdout: "fast_read": false, 2024-05-08T08:53:36.080 INFO:teuthology.orchestra.run.smithi059.stdout: "options": {}, 2024-05-08T08:53:36.080 INFO:teuthology.orchestra.run.smithi059.stdout: "application_metadata": { 2024-05-08T08:53:36.080 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd": {} 2024-05-08T08:53:36.081 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.081 INFO:teuthology.orchestra.run.smithi059.stdout: "read_balance": { 2024-05-08T08:53:36.081 INFO:teuthology.orchestra.run.smithi059.stdout: "score_type": "Fair distribution", 2024-05-08T08:53:36.081 INFO:teuthology.orchestra.run.smithi059.stdout: "score_acting": 2.25, 2024-05-08T08:53:36.081 INFO:teuthology.orchestra.run.smithi059.stdout: "score_stable": 2.25, 2024-05-08T08:53:36.081 INFO:teuthology.orchestra.run.smithi059.stdout: "optimal_score": 1, 2024-05-08T08:53:36.081 INFO:teuthology.orchestra.run.smithi059.stdout: "raw_score_acting": 2.25, 2024-05-08T08:53:36.081 INFO:teuthology.orchestra.run.smithi059.stdout: "raw_score_stable": 2.25, 2024-05-08T08:53:36.081 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity_weighted": 0.87819164991378784, 2024-05-08T08:53:36.081 INFO:teuthology.orchestra.run.smithi059.stdout: "average_primary_affinity": 0.87999999523162842, 2024-05-08T08:53:36.081 INFO:teuthology.orchestra.run.smithi059.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-05-08T08:53:36.081 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:36.081 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.081 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.082 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 3, 2024-05-08T08:53:36.082 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_name": "pool_client.0", 2024-05-08T08:53:36.082 INFO:teuthology.orchestra.run.smithi059.stdout: "create_time": "2024-05-08T08:52:09.955467+0000", 2024-05-08T08:53:36.082 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 8193, 2024-05-08T08:53:36.082 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-05-08T08:53:36.082 INFO:teuthology.orchestra.run.smithi059.stdout: "type": 1, 2024-05-08T08:53:36.082 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 2, 2024-05-08T08:53:36.082 INFO:teuthology.orchestra.run.smithi059.stdout: "min_size": 1, 2024-05-08T08:53:36.082 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_rule": 0, 2024-05-08T08:53:36.082 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_count": 0, 2024-05-08T08:53:36.082 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_target": 0, 2024-05-08T08:53:36.082 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_barrier": 0, 2024-05-08T08:53:36.082 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-08T08:53:36.082 INFO:teuthology.orchestra.run.smithi059.stdout: "object_hash": 2, 2024-05-08T08:53:36.083 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscale_mode": "off", 2024-05-08T08:53:36.083 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num": 32, 2024-05-08T08:53:36.083 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num": 32, 2024-05-08T08:53:36.083 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num_target": 32, 2024-05-08T08:53:36.083 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_target": 32, 2024-05-08T08:53:36.083 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_pending": 32, 2024-05-08T08:53:36.083 INFO:teuthology.orchestra.run.smithi059.stdout: "last_pg_merge_meta": { 2024-05-08T08:53:36.083 INFO:teuthology.orchestra.run.smithi059.stdout: "source_pgid": "0.0", 2024-05-08T08:53:36.083 INFO:teuthology.orchestra.run.smithi059.stdout: "ready_epoch": 0, 2024-05-08T08:53:36.083 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_started": 0, 2024-05-08T08:53:36.083 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_clean": 0, 2024-05-08T08:53:36.083 INFO:teuthology.orchestra.run.smithi059.stdout: "source_version": "0'0", 2024-05-08T08:53:36.083 INFO:teuthology.orchestra.run.smithi059.stdout: "target_version": "0'0" 2024-05-08T08:53:36.083 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.084 INFO:teuthology.orchestra.run.smithi059.stdout: "last_change": "78", 2024-05-08T08:53:36.084 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend": "0", 2024-05-08T08:53:36.084 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-08T08:53:36.084 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_preluminous": "0", 2024-05-08T08:53:36.084 INFO:teuthology.orchestra.run.smithi059.stdout: "auid": 0, 2024-05-08T08:53:36.084 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_mode": "selfmanaged", 2024-05-08T08:53:36.084 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_seq": 25, 2024-05-08T08:53:36.084 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_epoch": 78, 2024-05-08T08:53:36.084 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_snaps": [], 2024-05-08T08:53:36.084 INFO:teuthology.orchestra.run.smithi059.stdout: "removed_snaps": "[]", 2024-05-08T08:53:36.084 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_bytes": 0, 2024-05-08T08:53:36.084 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_objects": 0, 2024-05-08T08:53:36.084 INFO:teuthology.orchestra.run.smithi059.stdout: "tiers": [], 2024-05-08T08:53:36.084 INFO:teuthology.orchestra.run.smithi059.stdout: "tier_of": -1, 2024-05-08T08:53:36.084 INFO:teuthology.orchestra.run.smithi059.stdout: "read_tier": -1, 2024-05-08T08:53:36.085 INFO:teuthology.orchestra.run.smithi059.stdout: "write_tier": -1, 2024-05-08T08:53:36.085 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_mode": "none", 2024-05-08T08:53:36.085 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_bytes": 0, 2024-05-08T08:53:36.085 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_objects": 0, 2024-05-08T08:53:36.085 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-08T08:53:36.085 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-08T08:53:36.085 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-08T08:53:36.086 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_flush_age": 0, 2024-05-08T08:53:36.086 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_evict_age": 0, 2024-05-08T08:53:36.086 INFO:teuthology.orchestra.run.smithi059.stdout: "erasure_code_profile": "", 2024-05-08T08:53:36.086 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_params": { 2024-05-08T08:53:36.086 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "none" 2024-05-08T08:53:36.086 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.086 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_period": 0, 2024-05-08T08:53:36.086 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_count": 0, 2024-05-08T08:53:36.086 INFO:teuthology.orchestra.run.smithi059.stdout: "use_gmt_hitset": true, 2024-05-08T08:53:36.086 INFO:teuthology.orchestra.run.smithi059.stdout: "min_read_recency_for_promote": 0, 2024-05-08T08:53:36.086 INFO:teuthology.orchestra.run.smithi059.stdout: "min_write_recency_for_promote": 0, 2024-05-08T08:53:36.086 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_grade_decay_rate": 0, 2024-05-08T08:53:36.087 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_search_last_n": 0, 2024-05-08T08:53:36.087 INFO:teuthology.orchestra.run.smithi059.stdout: "grade_table": [], 2024-05-08T08:53:36.087 INFO:teuthology.orchestra.run.smithi059.stdout: "stripe_width": 0, 2024-05-08T08:53:36.087 INFO:teuthology.orchestra.run.smithi059.stdout: "expected_num_objects": 0, 2024-05-08T08:53:36.087 INFO:teuthology.orchestra.run.smithi059.stdout: "fast_read": false, 2024-05-08T08:53:36.087 INFO:teuthology.orchestra.run.smithi059.stdout: "options": {}, 2024-05-08T08:53:36.087 INFO:teuthology.orchestra.run.smithi059.stdout: "application_metadata": { 2024-05-08T08:53:36.087 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd": {} 2024-05-08T08:53:36.087 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.087 INFO:teuthology.orchestra.run.smithi059.stdout: "read_balance": { 2024-05-08T08:53:36.087 INFO:teuthology.orchestra.run.smithi059.stdout: "score_type": "Fair distribution", 2024-05-08T08:53:36.087 INFO:teuthology.orchestra.run.smithi059.stdout: "score_acting": 1.5, 2024-05-08T08:53:36.087 INFO:teuthology.orchestra.run.smithi059.stdout: "score_stable": 1.5, 2024-05-08T08:53:36.087 INFO:teuthology.orchestra.run.smithi059.stdout: "optimal_score": 1, 2024-05-08T08:53:36.087 INFO:teuthology.orchestra.run.smithi059.stdout: "raw_score_acting": 1.5, 2024-05-08T08:53:36.088 INFO:teuthology.orchestra.run.smithi059.stdout: "raw_score_stable": 1.5, 2024-05-08T08:53:36.088 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity_weighted": 0.87819164991378784, 2024-05-08T08:53:36.088 INFO:teuthology.orchestra.run.smithi059.stdout: "average_primary_affinity": 0.87999999523162842, 2024-05-08T08:53:36.088 INFO:teuthology.orchestra.run.smithi059.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-05-08T08:53:36.088 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:36.088 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:36.088 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T08:53:36.088 INFO:teuthology.orchestra.run.smithi059.stdout: "osds": [ 2024-05-08T08:53:36.088 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.088 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 0, 2024-05-08T08:53:36.088 INFO:teuthology.orchestra.run.smithi059.stdout: "uuid": "300801a5-1919-4b23-8c6d-bc7dad61cc4d", 2024-05-08T08:53:36.088 INFO:teuthology.orchestra.run.smithi059.stdout: "up": 1, 2024-05-08T08:53:36.088 INFO:teuthology.orchestra.run.smithi059.stdout: "in": 1, 2024-05-08T08:53:36.089 INFO:teuthology.orchestra.run.smithi059.stdout: "weight": 1, 2024-05-08T08:53:36.089 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity": 1, 2024-05-08T08:53:36.089 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_begin": 0, 2024-05-08T08:53:36.089 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_end": 0, 2024-05-08T08:53:36.089 INFO:teuthology.orchestra.run.smithi059.stdout: "up_from": 11, 2024-05-08T08:53:36.089 INFO:teuthology.orchestra.run.smithi059.stdout: "up_thru": 73, 2024-05-08T08:53:36.089 INFO:teuthology.orchestra.run.smithi059.stdout: "down_at": 0, 2024-05-08T08:53:36.089 INFO:teuthology.orchestra.run.smithi059.stdout: "lost_at": 0, 2024-05-08T08:53:36.089 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrs": { 2024-05-08T08:53:36.089 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:53:36.089 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.089 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:53:36.089 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6800", 2024-05-08T08:53:36.089 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 802160151 2024-05-08T08:53:36.089 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.090 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.090 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:53:36.090 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6801", 2024-05-08T08:53:36.090 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 802160151 2024-05-08T08:53:36.090 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:36.090 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:36.090 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.090 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addrs": { 2024-05-08T08:53:36.090 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:53:36.090 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.090 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:53:36.090 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6802", 2024-05-08T08:53:36.090 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 802160151 2024-05-08T08:53:36.090 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.091 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.091 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:53:36.091 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6803", 2024-05-08T08:53:36.091 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 802160151 2024-05-08T08:53:36.091 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:36.091 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:36.091 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.091 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addrs": { 2024-05-08T08:53:36.091 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:53:36.091 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.091 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:53:36.091 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6806", 2024-05-08T08:53:36.091 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 802160151 2024-05-08T08:53:36.091 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.091 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.092 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:53:36.092 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6807", 2024-05-08T08:53:36.092 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 802160151 2024-05-08T08:53:36.092 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:36.092 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:36.092 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.092 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addrs": { 2024-05-08T08:53:36.092 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:53:36.092 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.092 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:53:36.092 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6804", 2024-05-08T08:53:36.092 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 802160151 2024-05-08T08:53:36.092 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.092 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.092 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:53:36.093 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6805", 2024-05-08T08:53:36.093 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 802160151 2024-05-08T08:53:36.093 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:36.093 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:36.093 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.093 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "172.21.15.59:6801/802160151", 2024-05-08T08:53:36.093 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "172.21.15.59:6803/802160151", 2024-05-08T08:53:36.093 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addr": "172.21.15.59:6807/802160151", 2024-05-08T08:53:36.093 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addr": "172.21.15.59:6805/802160151", 2024-05-08T08:53:36.093 INFO:teuthology.orchestra.run.smithi059.stdout: "state": [ 2024-05-08T08:53:36.093 INFO:teuthology.orchestra.run.smithi059.stdout: "exists", 2024-05-08T08:53:36.093 INFO:teuthology.orchestra.run.smithi059.stdout: "up" 2024-05-08T08:53:36.093 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:36.093 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.093 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.093 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 1, 2024-05-08T08:53:36.094 INFO:teuthology.orchestra.run.smithi059.stdout: "uuid": "0fead0de-6a65-4eba-b400-eea5692702e0", 2024-05-08T08:53:36.094 INFO:teuthology.orchestra.run.smithi059.stdout: "up": 1, 2024-05-08T08:53:36.094 INFO:teuthology.orchestra.run.smithi059.stdout: "in": 1, 2024-05-08T08:53:36.094 INFO:teuthology.orchestra.run.smithi059.stdout: "weight": 1, 2024-05-08T08:53:36.094 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity": 1, 2024-05-08T08:53:36.094 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_begin": 0, 2024-05-08T08:53:36.094 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_end": 0, 2024-05-08T08:53:36.094 INFO:teuthology.orchestra.run.smithi059.stdout: "up_from": 11, 2024-05-08T08:53:36.094 INFO:teuthology.orchestra.run.smithi059.stdout: "up_thru": 73, 2024-05-08T08:53:36.094 INFO:teuthology.orchestra.run.smithi059.stdout: "down_at": 0, 2024-05-08T08:53:36.094 INFO:teuthology.orchestra.run.smithi059.stdout: "lost_at": 0, 2024-05-08T08:53:36.094 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrs": { 2024-05-08T08:53:36.094 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:53:36.095 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.095 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:53:36.095 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6816", 2024-05-08T08:53:36.095 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4116801011 2024-05-08T08:53:36.095 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.095 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.095 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:53:36.095 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6817", 2024-05-08T08:53:36.095 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4116801011 2024-05-08T08:53:36.095 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:36.095 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:36.095 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.095 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addrs": { 2024-05-08T08:53:36.095 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:53:36.095 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.096 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:53:36.096 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6818", 2024-05-08T08:53:36.096 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4116801011 2024-05-08T08:53:36.096 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.096 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.096 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:53:36.096 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6819", 2024-05-08T08:53:36.096 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4116801011 2024-05-08T08:53:36.096 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:36.096 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:36.096 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.096 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addrs": { 2024-05-08T08:53:36.096 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:53:36.096 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.096 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:53:36.096 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6822", 2024-05-08T08:53:36.096 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4116801011 2024-05-08T08:53:36.097 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.097 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.097 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:53:36.097 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6823", 2024-05-08T08:53:36.097 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4116801011 2024-05-08T08:53:36.097 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:36.097 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:36.097 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.097 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addrs": { 2024-05-08T08:53:36.097 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:53:36.097 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.097 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:53:36.097 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6820", 2024-05-08T08:53:36.097 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4116801011 2024-05-08T08:53:36.097 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.097 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.098 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:53:36.098 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6821", 2024-05-08T08:53:36.098 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4116801011 2024-05-08T08:53:36.098 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:36.098 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:36.098 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.098 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "172.21.15.59:6817/4116801011", 2024-05-08T08:53:36.098 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "172.21.15.59:6819/4116801011", 2024-05-08T08:53:36.098 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addr": "172.21.15.59:6823/4116801011", 2024-05-08T08:53:36.098 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addr": "172.21.15.59:6821/4116801011", 2024-05-08T08:53:36.098 INFO:teuthology.orchestra.run.smithi059.stdout: "state": [ 2024-05-08T08:53:36.098 INFO:teuthology.orchestra.run.smithi059.stdout: "exists", 2024-05-08T08:53:36.098 INFO:teuthology.orchestra.run.smithi059.stdout: "up" 2024-05-08T08:53:36.098 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:36.098 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.099 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.099 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 2, 2024-05-08T08:53:36.099 INFO:teuthology.orchestra.run.smithi059.stdout: "uuid": "1a233de6-1b82-4a51-8e96-4cd4abdf16c8", 2024-05-08T08:53:36.099 INFO:teuthology.orchestra.run.smithi059.stdout: "up": 1, 2024-05-08T08:53:36.099 INFO:teuthology.orchestra.run.smithi059.stdout: "in": 1, 2024-05-08T08:53:36.099 INFO:teuthology.orchestra.run.smithi059.stdout: "weight": 1, 2024-05-08T08:53:36.099 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity": 1, 2024-05-08T08:53:36.099 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_begin": 0, 2024-05-08T08:53:36.099 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_end": 0, 2024-05-08T08:53:36.099 INFO:teuthology.orchestra.run.smithi059.stdout: "up_from": 11, 2024-05-08T08:53:36.099 INFO:teuthology.orchestra.run.smithi059.stdout: "up_thru": 71, 2024-05-08T08:53:36.099 INFO:teuthology.orchestra.run.smithi059.stdout: "down_at": 0, 2024-05-08T08:53:36.099 INFO:teuthology.orchestra.run.smithi059.stdout: "lost_at": 0, 2024-05-08T08:53:36.099 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrs": { 2024-05-08T08:53:36.100 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:53:36.100 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.100 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:53:36.100 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6808", 2024-05-08T08:53:36.100 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1547661179 2024-05-08T08:53:36.100 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.100 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.100 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:53:36.100 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6809", 2024-05-08T08:53:36.100 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1547661179 2024-05-08T08:53:36.100 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:36.100 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:36.100 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.100 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addrs": { 2024-05-08T08:53:36.100 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:53:36.101 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.101 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:53:36.101 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6810", 2024-05-08T08:53:36.101 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1547661179 2024-05-08T08:53:36.101 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.101 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.101 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:53:36.101 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6811", 2024-05-08T08:53:36.101 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1547661179 2024-05-08T08:53:36.101 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:36.101 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:36.101 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.101 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addrs": { 2024-05-08T08:53:36.101 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:53:36.101 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.102 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:53:36.102 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6814", 2024-05-08T08:53:36.102 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1547661179 2024-05-08T08:53:36.102 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.102 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.102 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:53:36.102 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6815", 2024-05-08T08:53:36.102 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1547661179 2024-05-08T08:53:36.102 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:36.102 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:36.102 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.102 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addrs": { 2024-05-08T08:53:36.102 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:53:36.102 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.103 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:53:36.103 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6812", 2024-05-08T08:53:36.103 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1547661179 2024-05-08T08:53:36.103 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.103 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.103 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:53:36.103 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6813", 2024-05-08T08:53:36.103 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1547661179 2024-05-08T08:53:36.103 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:36.103 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:36.103 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.103 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "172.21.15.59:6809/1547661179", 2024-05-08T08:53:36.103 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "172.21.15.59:6811/1547661179", 2024-05-08T08:53:36.103 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addr": "172.21.15.59:6815/1547661179", 2024-05-08T08:53:36.103 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addr": "172.21.15.59:6813/1547661179", 2024-05-08T08:53:36.104 INFO:teuthology.orchestra.run.smithi059.stdout: "state": [ 2024-05-08T08:53:36.104 INFO:teuthology.orchestra.run.smithi059.stdout: "exists", 2024-05-08T08:53:36.104 INFO:teuthology.orchestra.run.smithi059.stdout: "up" 2024-05-08T08:53:36.104 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:36.104 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.104 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.104 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 3, 2024-05-08T08:53:36.104 INFO:teuthology.orchestra.run.smithi059.stdout: "uuid": "cc4870b8-dcd9-4e43-978d-8de6c6396abf", 2024-05-08T08:53:36.104 INFO:teuthology.orchestra.run.smithi059.stdout: "up": 1, 2024-05-08T08:53:36.104 INFO:teuthology.orchestra.run.smithi059.stdout: "in": 1, 2024-05-08T08:53:36.104 INFO:teuthology.orchestra.run.smithi059.stdout: "weight": 1, 2024-05-08T08:53:36.104 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity": 1, 2024-05-08T08:53:36.104 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_begin": 0, 2024-05-08T08:53:36.104 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_end": 0, 2024-05-08T08:53:36.105 INFO:teuthology.orchestra.run.smithi059.stdout: "up_from": 11, 2024-05-08T08:53:36.105 INFO:teuthology.orchestra.run.smithi059.stdout: "up_thru": 73, 2024-05-08T08:53:36.105 INFO:teuthology.orchestra.run.smithi059.stdout: "down_at": 0, 2024-05-08T08:53:36.105 INFO:teuthology.orchestra.run.smithi059.stdout: "lost_at": 0, 2024-05-08T08:53:36.105 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrs": { 2024-05-08T08:53:36.105 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:53:36.105 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.105 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:53:36.105 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6800", 2024-05-08T08:53:36.105 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4054400810 2024-05-08T08:53:36.105 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.105 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.105 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:53:36.105 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6801", 2024-05-08T08:53:36.106 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4054400810 2024-05-08T08:53:36.106 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:36.106 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:36.106 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.106 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addrs": { 2024-05-08T08:53:36.106 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:53:36.106 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.106 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:53:36.106 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6802", 2024-05-08T08:53:36.106 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4054400810 2024-05-08T08:53:36.106 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.106 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.106 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:53:36.106 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6803", 2024-05-08T08:53:36.106 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4054400810 2024-05-08T08:53:36.107 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:36.107 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:36.107 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.107 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addrs": { 2024-05-08T08:53:36.107 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:53:36.107 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.107 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:53:36.107 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6806", 2024-05-08T08:53:36.107 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4054400810 2024-05-08T08:53:36.107 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.107 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.107 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:53:36.107 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6807", 2024-05-08T08:53:36.107 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4054400810 2024-05-08T08:53:36.108 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:36.108 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:36.108 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.108 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addrs": { 2024-05-08T08:53:36.108 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:53:36.108 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.108 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:53:36.108 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6804", 2024-05-08T08:53:36.108 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4054400810 2024-05-08T08:53:36.108 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.108 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.108 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:53:36.108 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6805", 2024-05-08T08:53:36.108 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4054400810 2024-05-08T08:53:36.108 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:36.109 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:36.109 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.109 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "172.21.15.192:6801/4054400810", 2024-05-08T08:53:36.109 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "172.21.15.192:6803/4054400810", 2024-05-08T08:53:36.109 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addr": "172.21.15.192:6807/4054400810", 2024-05-08T08:53:36.109 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addr": "172.21.15.192:6805/4054400810", 2024-05-08T08:53:36.109 INFO:teuthology.orchestra.run.smithi059.stdout: "state": [ 2024-05-08T08:53:36.109 INFO:teuthology.orchestra.run.smithi059.stdout: "exists", 2024-05-08T08:53:36.109 INFO:teuthology.orchestra.run.smithi059.stdout: "up" 2024-05-08T08:53:36.109 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:36.109 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.109 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.109 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 4, 2024-05-08T08:53:36.109 INFO:teuthology.orchestra.run.smithi059.stdout: "uuid": "d1d05620-42fb-4107-a2f9-e83542d076ee", 2024-05-08T08:53:36.109 INFO:teuthology.orchestra.run.smithi059.stdout: "up": 1, 2024-05-08T08:53:36.109 INFO:teuthology.orchestra.run.smithi059.stdout: "in": 1, 2024-05-08T08:53:36.109 INFO:teuthology.orchestra.run.smithi059.stdout: "weight": 1, 2024-05-08T08:53:36.110 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity": 0.9892578125, 2024-05-08T08:53:36.110 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_begin": 0, 2024-05-08T08:53:36.110 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_end": 0, 2024-05-08T08:53:36.110 INFO:teuthology.orchestra.run.smithi059.stdout: "up_from": 11, 2024-05-08T08:53:36.110 INFO:teuthology.orchestra.run.smithi059.stdout: "up_thru": 73, 2024-05-08T08:53:36.110 INFO:teuthology.orchestra.run.smithi059.stdout: "down_at": 0, 2024-05-08T08:53:36.110 INFO:teuthology.orchestra.run.smithi059.stdout: "lost_at": 0, 2024-05-08T08:53:36.110 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrs": { 2024-05-08T08:53:36.110 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:53:36.110 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.110 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:53:36.110 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6808", 2024-05-08T08:53:36.110 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 58985565 2024-05-08T08:53:36.110 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.110 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.111 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:53:36.111 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6809", 2024-05-08T08:53:36.111 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 58985565 2024-05-08T08:53:36.111 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:36.111 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:36.111 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.111 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addrs": { 2024-05-08T08:53:36.111 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:53:36.111 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.111 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:53:36.111 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6810", 2024-05-08T08:53:36.111 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 58985565 2024-05-08T08:53:36.111 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.111 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.111 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:53:36.111 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6811", 2024-05-08T08:53:36.112 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 58985565 2024-05-08T08:53:36.112 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:36.112 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:36.112 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.112 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addrs": { 2024-05-08T08:53:36.112 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:53:36.112 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.112 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:53:36.112 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6814", 2024-05-08T08:53:36.112 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 58985565 2024-05-08T08:53:36.112 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.112 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.112 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:53:36.112 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6815", 2024-05-08T08:53:36.112 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 58985565 2024-05-08T08:53:36.113 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:36.113 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:36.113 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.113 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addrs": { 2024-05-08T08:53:36.113 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:53:36.113 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.113 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:53:36.113 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6812", 2024-05-08T08:53:36.113 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 58985565 2024-05-08T08:53:36.113 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.113 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.113 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:53:36.113 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6813", 2024-05-08T08:53:36.113 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 58985565 2024-05-08T08:53:36.113 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:36.113 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:36.113 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.114 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "172.21.15.192:6809/58985565", 2024-05-08T08:53:36.114 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "172.21.15.192:6811/58985565", 2024-05-08T08:53:36.114 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addr": "172.21.15.192:6815/58985565", 2024-05-08T08:53:36.114 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addr": "172.21.15.192:6813/58985565", 2024-05-08T08:53:36.114 INFO:teuthology.orchestra.run.smithi059.stdout: "state": [ 2024-05-08T08:53:36.114 INFO:teuthology.orchestra.run.smithi059.stdout: "exists", 2024-05-08T08:53:36.114 INFO:teuthology.orchestra.run.smithi059.stdout: "up" 2024-05-08T08:53:36.114 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:36.114 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.114 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.114 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 5, 2024-05-08T08:53:36.114 INFO:teuthology.orchestra.run.smithi059.stdout: "uuid": "ab26136c-31c3-4090-9fa2-988dba4f731f", 2024-05-08T08:53:36.114 INFO:teuthology.orchestra.run.smithi059.stdout: "up": 1, 2024-05-08T08:53:36.114 INFO:teuthology.orchestra.run.smithi059.stdout: "in": 1, 2024-05-08T08:53:36.115 INFO:teuthology.orchestra.run.smithi059.stdout: "weight": 1, 2024-05-08T08:53:36.115 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity": 0.2798919677734375, 2024-05-08T08:53:36.115 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_begin": 0, 2024-05-08T08:53:36.115 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_end": 0, 2024-05-08T08:53:36.115 INFO:teuthology.orchestra.run.smithi059.stdout: "up_from": 11, 2024-05-08T08:53:36.115 INFO:teuthology.orchestra.run.smithi059.stdout: "up_thru": 18, 2024-05-08T08:53:36.115 INFO:teuthology.orchestra.run.smithi059.stdout: "down_at": 0, 2024-05-08T08:53:36.115 INFO:teuthology.orchestra.run.smithi059.stdout: "lost_at": 0, 2024-05-08T08:53:36.115 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrs": { 2024-05-08T08:53:36.115 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:53:36.115 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.115 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:53:36.115 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6816", 2024-05-08T08:53:36.115 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1283571573 2024-05-08T08:53:36.115 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.116 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.116 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:53:36.116 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6817", 2024-05-08T08:53:36.116 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1283571573 2024-05-08T08:53:36.116 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:36.116 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:36.116 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.116 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addrs": { 2024-05-08T08:53:36.116 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:53:36.116 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.116 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:53:36.116 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6818", 2024-05-08T08:53:36.116 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1283571573 2024-05-08T08:53:36.116 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.116 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.116 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:53:36.117 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6819", 2024-05-08T08:53:36.117 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1283571573 2024-05-08T08:53:36.117 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:36.117 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:36.117 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.117 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addrs": { 2024-05-08T08:53:36.117 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:53:36.117 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.117 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:53:36.117 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6822", 2024-05-08T08:53:36.117 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1283571573 2024-05-08T08:53:36.117 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.117 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.117 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:53:36.117 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6823", 2024-05-08T08:53:36.117 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1283571573 2024-05-08T08:53:36.118 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:36.118 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:36.118 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.118 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addrs": { 2024-05-08T08:53:36.118 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:53:36.118 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.118 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:53:36.118 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6820", 2024-05-08T08:53:36.118 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1283571573 2024-05-08T08:53:36.118 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.118 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.118 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:53:36.118 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6821", 2024-05-08T08:53:36.118 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1283571573 2024-05-08T08:53:36.118 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:36.119 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:36.119 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.119 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "172.21.15.192:6817/1283571573", 2024-05-08T08:53:36.119 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "172.21.15.192:6819/1283571573", 2024-05-08T08:53:36.119 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addr": "172.21.15.192:6823/1283571573", 2024-05-08T08:53:36.119 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addr": "172.21.15.192:6821/1283571573", 2024-05-08T08:53:36.119 INFO:teuthology.orchestra.run.smithi059.stdout: "state": [ 2024-05-08T08:53:36.119 INFO:teuthology.orchestra.run.smithi059.stdout: "exists", 2024-05-08T08:53:36.119 INFO:teuthology.orchestra.run.smithi059.stdout: "up" 2024-05-08T08:53:36.119 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:36.119 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:36.119 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T08:53:36.119 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_xinfo": [ 2024-05-08T08:53:36.119 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.119 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 0, 2024-05-08T08:53:36.119 INFO:teuthology.orchestra.run.smithi059.stdout: "down_stamp": "0.000000", 2024-05-08T08:53:36.119 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_probability": 0, 2024-05-08T08:53:36.120 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_interval": 0, 2024-05-08T08:53:36.120 INFO:teuthology.orchestra.run.smithi059.stdout: "features": 4540701547738038271, 2024-05-08T08:53:36.120 INFO:teuthology.orchestra.run.smithi059.stdout: "old_weight": 0, 2024-05-08T08:53:36.120 INFO:teuthology.orchestra.run.smithi059.stdout: "last_purged_snaps_scrub": "2024-05-08T08:51:52.396946+0000", 2024-05-08T08:53:36.120 INFO:teuthology.orchestra.run.smithi059.stdout: "dead_epoch": 0 2024-05-08T08:53:36.120 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.120 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.120 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 1, 2024-05-08T08:53:36.120 INFO:teuthology.orchestra.run.smithi059.stdout: "down_stamp": "0.000000", 2024-05-08T08:53:36.120 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_probability": 0, 2024-05-08T08:53:36.120 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_interval": 0, 2024-05-08T08:53:36.120 INFO:teuthology.orchestra.run.smithi059.stdout: "features": 4540701547738038271, 2024-05-08T08:53:36.120 INFO:teuthology.orchestra.run.smithi059.stdout: "old_weight": 0, 2024-05-08T08:53:36.120 INFO:teuthology.orchestra.run.smithi059.stdout: "last_purged_snaps_scrub": "2024-05-08T08:51:52.453160+0000", 2024-05-08T08:53:36.120 INFO:teuthology.orchestra.run.smithi059.stdout: "dead_epoch": 0 2024-05-08T08:53:36.121 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.121 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.121 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 2, 2024-05-08T08:53:36.121 INFO:teuthology.orchestra.run.smithi059.stdout: "down_stamp": "0.000000", 2024-05-08T08:53:36.121 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_probability": 0, 2024-05-08T08:53:36.121 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_interval": 0, 2024-05-08T08:53:36.121 INFO:teuthology.orchestra.run.smithi059.stdout: "features": 4540701547738038271, 2024-05-08T08:53:36.121 INFO:teuthology.orchestra.run.smithi059.stdout: "old_weight": 0, 2024-05-08T08:53:36.121 INFO:teuthology.orchestra.run.smithi059.stdout: "last_purged_snaps_scrub": "2024-05-08T08:51:52.387606+0000", 2024-05-08T08:53:36.121 INFO:teuthology.orchestra.run.smithi059.stdout: "dead_epoch": 0 2024-05-08T08:53:36.121 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.121 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.121 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 3, 2024-05-08T08:53:36.121 INFO:teuthology.orchestra.run.smithi059.stdout: "down_stamp": "0.000000", 2024-05-08T08:53:36.121 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_probability": 0, 2024-05-08T08:53:36.121 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_interval": 0, 2024-05-08T08:53:36.121 INFO:teuthology.orchestra.run.smithi059.stdout: "features": 4540701547738038271, 2024-05-08T08:53:36.122 INFO:teuthology.orchestra.run.smithi059.stdout: "old_weight": 0, 2024-05-08T08:53:36.122 INFO:teuthology.orchestra.run.smithi059.stdout: "last_purged_snaps_scrub": "2024-05-08T08:51:52.360952+0000", 2024-05-08T08:53:36.122 INFO:teuthology.orchestra.run.smithi059.stdout: "dead_epoch": 0 2024-05-08T08:53:36.122 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.122 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.122 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 4, 2024-05-08T08:53:36.122 INFO:teuthology.orchestra.run.smithi059.stdout: "down_stamp": "0.000000", 2024-05-08T08:53:36.122 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_probability": 0, 2024-05-08T08:53:36.122 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_interval": 0, 2024-05-08T08:53:36.122 INFO:teuthology.orchestra.run.smithi059.stdout: "features": 4540701547738038271, 2024-05-08T08:53:36.122 INFO:teuthology.orchestra.run.smithi059.stdout: "old_weight": 0, 2024-05-08T08:53:36.122 INFO:teuthology.orchestra.run.smithi059.stdout: "last_purged_snaps_scrub": "2024-05-08T08:51:52.468681+0000", 2024-05-08T08:53:36.122 INFO:teuthology.orchestra.run.smithi059.stdout: "dead_epoch": 0 2024-05-08T08:53:36.122 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.122 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.123 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 5, 2024-05-08T08:53:36.123 INFO:teuthology.orchestra.run.smithi059.stdout: "down_stamp": "0.000000", 2024-05-08T08:53:36.123 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_probability": 0, 2024-05-08T08:53:36.123 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_interval": 0, 2024-05-08T08:53:36.123 INFO:teuthology.orchestra.run.smithi059.stdout: "features": 4540701547738038271, 2024-05-08T08:53:36.123 INFO:teuthology.orchestra.run.smithi059.stdout: "old_weight": 0, 2024-05-08T08:53:36.123 INFO:teuthology.orchestra.run.smithi059.stdout: "last_purged_snaps_scrub": "2024-05-08T08:51:52.401080+0000", 2024-05-08T08:53:36.123 INFO:teuthology.orchestra.run.smithi059.stdout: "dead_epoch": 0 2024-05-08T08:53:36.123 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:36.123 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T08:53:36.123 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_upmap": [ 2024-05-08T08:53:36.123 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.123 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "3.4", 2024-05-08T08:53:36.123 INFO:teuthology.orchestra.run.smithi059.stdout: "osds": [ 2024-05-08T08:53:36.123 INFO:teuthology.orchestra.run.smithi059.stdout: 1, 2024-05-08T08:53:36.123 INFO:teuthology.orchestra.run.smithi059.stdout: 5 2024-05-08T08:53:36.124 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:36.124 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.124 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.124 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "3.17", 2024-05-08T08:53:36.125 INFO:teuthology.orchestra.run.smithi059.stdout: "osds": [ 2024-05-08T08:53:36.125 INFO:teuthology.orchestra.run.smithi059.stdout: 1, 2024-05-08T08:53:36.125 INFO:teuthology.orchestra.run.smithi059.stdout: 0 2024-05-08T08:53:36.125 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:36.125 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.126 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.126 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "3.1d", 2024-05-08T08:53:36.126 INFO:teuthology.orchestra.run.smithi059.stdout: "osds": [ 2024-05-08T08:53:36.126 INFO:teuthology.orchestra.run.smithi059.stdout: 0, 2024-05-08T08:53:36.126 INFO:teuthology.orchestra.run.smithi059.stdout: 5 2024-05-08T08:53:36.126 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:36.126 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:36.126 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T08:53:36.126 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_upmap_items": [], 2024-05-08T08:53:36.126 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_upmap_primaries": [], 2024-05-08T08:53:36.126 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_temp": [], 2024-05-08T08:53:36.126 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_temp": [], 2024-05-08T08:53:36.126 INFO:teuthology.orchestra.run.smithi059.stdout: "blocklist": {}, 2024-05-08T08:53:36.127 INFO:teuthology.orchestra.run.smithi059.stdout: "range_blocklist": {}, 2024-05-08T08:53:36.127 INFO:teuthology.orchestra.run.smithi059.stdout: "erasure_code_profiles": { 2024-05-08T08:53:36.127 INFO:teuthology.orchestra.run.smithi059.stdout: "default": { 2024-05-08T08:53:36.127 INFO:teuthology.orchestra.run.smithi059.stdout: "crush-failure-domain": "osd", 2024-05-08T08:53:36.127 INFO:teuthology.orchestra.run.smithi059.stdout: "k": "2", 2024-05-08T08:53:36.127 INFO:teuthology.orchestra.run.smithi059.stdout: "m": "1", 2024-05-08T08:53:36.127 INFO:teuthology.orchestra.run.smithi059.stdout: "plugin": "jerasure", 2024-05-08T08:53:36.127 INFO:teuthology.orchestra.run.smithi059.stdout: "technique": "reed_sol_van" 2024-05-08T08:53:36.127 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:36.127 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.127 INFO:teuthology.orchestra.run.smithi059.stdout: "removed_snaps_queue": [], 2024-05-08T08:53:36.127 INFO:teuthology.orchestra.run.smithi059.stdout: "new_removed_snaps": [], 2024-05-08T08:53:36.127 INFO:teuthology.orchestra.run.smithi059.stdout: "new_purged_snaps": [], 2024-05-08T08:53:36.127 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_node_flags": {}, 2024-05-08T08:53:36.127 INFO:teuthology.orchestra.run.smithi059.stdout: "device_class_flags": {}, 2024-05-08T08:53:36.128 INFO:teuthology.orchestra.run.smithi059.stdout: "stretch_mode": { 2024-05-08T08:53:36.128 INFO:teuthology.orchestra.run.smithi059.stdout: "stretch_mode_enabled": false, 2024-05-08T08:53:36.128 INFO:teuthology.orchestra.run.smithi059.stdout: "stretch_bucket_count": 0, 2024-05-08T08:53:36.128 INFO:teuthology.orchestra.run.smithi059.stdout: "degraded_stretch_mode": 0, 2024-05-08T08:53:36.128 INFO:teuthology.orchestra.run.smithi059.stdout: "recovering_stretch_mode": 0, 2024-05-08T08:53:36.128 INFO:teuthology.orchestra.run.smithi059.stdout: "stretch_mode_bucket": 0 2024-05-08T08:53:36.128 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:36.128 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:36.129 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:53:36.070+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:53:36.130 INFO:tasks.thrashosds.thrasher:j is {'epoch': 79, 'fsid': '38dfae8c-5c5e-4b0f-a264-91f22c6c5554', 'created': '2024-05-08T08:51:45.571369+0000', 'modified': '2024-05-08T08:53:33.519336+0000', 'last_up_change': '2024-05-08T08:51:56.847706+0000', 'last_in_change': '2024-05-08T08:53:19.443905+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T08:51:57.787790+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '14', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.8781916499137878, 'average_primary_affinity': 0.8799999952316284, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-05-08T08:52:02.563677+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.8781916499137878, 'average_primary_affinity': 0.8799999952316284, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-05-08T08:52:09.955467+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 25, 'snap_epoch': 78, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.8781916499137878, 'average_primary_affinity': 0.8799999952316284, 'average_primary_affinity_weighted': 0.8799999952316284}}], 'osds': [{'osd': 0, 'uuid': '300801a5-1919-4b23-8c6d-bc7dad61cc4d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 73, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6800', 'nonce': 802160151}, {'type': 'v1', 'addr': '172.21.15.59:6801', 'nonce': 802160151}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6802', 'nonce': 802160151}, {'type': 'v1', 'addr': '172.21.15.59:6803', 'nonce': 802160151}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6806', 'nonce': 802160151}, {'type': 'v1', 'addr': '172.21.15.59:6807', 'nonce': 802160151}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6804', 'nonce': 802160151}, {'type': 'v1', 'addr': '172.21.15.59:6805', 'nonce': 802160151}]}, 'public_addr': '172.21.15.59:6801/802160151', 'cluster_addr': '172.21.15.59:6803/802160151', 'heartbeat_back_addr': '172.21.15.59:6807/802160151', 'heartbeat_front_addr': '172.21.15.59:6805/802160151', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '0fead0de-6a65-4eba-b400-eea5692702e0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 73, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6816', 'nonce': 4116801011}, {'type': 'v1', 'addr': '172.21.15.59:6817', 'nonce': 4116801011}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6818', 'nonce': 4116801011}, {'type': 'v1', 'addr': '172.21.15.59:6819', 'nonce': 4116801011}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6822', 'nonce': 4116801011}, {'type': 'v1', 'addr': '172.21.15.59:6823', 'nonce': 4116801011}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6820', 'nonce': 4116801011}, {'type': 'v1', 'addr': '172.21.15.59:6821', 'nonce': 4116801011}]}, 'public_addr': '172.21.15.59:6817/4116801011', 'cluster_addr': '172.21.15.59:6819/4116801011', 'heartbeat_back_addr': '172.21.15.59:6823/4116801011', 'heartbeat_front_addr': '172.21.15.59:6821/4116801011', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '1a233de6-1b82-4a51-8e96-4cd4abdf16c8', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 71, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6808', 'nonce': 1547661179}, {'type': 'v1', 'addr': '172.21.15.59:6809', 'nonce': 1547661179}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6810', 'nonce': 1547661179}, {'type': 'v1', 'addr': '172.21.15.59:6811', 'nonce': 1547661179}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6814', 'nonce': 1547661179}, {'type': 'v1', 'addr': '172.21.15.59:6815', 'nonce': 1547661179}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6812', 'nonce': 1547661179}, {'type': 'v1', 'addr': '172.21.15.59:6813', 'nonce': 1547661179}]}, 'public_addr': '172.21.15.59:6809/1547661179', 'cluster_addr': '172.21.15.59:6811/1547661179', 'heartbeat_back_addr': '172.21.15.59:6815/1547661179', 'heartbeat_front_addr': '172.21.15.59:6813/1547661179', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'cc4870b8-dcd9-4e43-978d-8de6c6396abf', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 73, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6800', 'nonce': 4054400810}, {'type': 'v1', 'addr': '172.21.15.192:6801', 'nonce': 4054400810}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6802', 'nonce': 4054400810}, {'type': 'v1', 'addr': '172.21.15.192:6803', 'nonce': 4054400810}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6806', 'nonce': 4054400810}, {'type': 'v1', 'addr': '172.21.15.192:6807', 'nonce': 4054400810}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6804', 'nonce': 4054400810}, {'type': 'v1', 'addr': '172.21.15.192:6805', 'nonce': 4054400810}]}, 'public_addr': '172.21.15.192:6801/4054400810', 'cluster_addr': '172.21.15.192:6803/4054400810', 'heartbeat_back_addr': '172.21.15.192:6807/4054400810', 'heartbeat_front_addr': '172.21.15.192:6805/4054400810', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'd1d05620-42fb-4107-a2f9-e83542d076ee', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.9892578125, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 73, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6808', 'nonce': 58985565}, {'type': 'v1', 'addr': '172.21.15.192:6809', 'nonce': 58985565}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6810', 'nonce': 58985565}, {'type': 'v1', 'addr': '172.21.15.192:6811', 'nonce': 58985565}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6814', 'nonce': 58985565}, {'type': 'v1', 'addr': '172.21.15.192:6815', 'nonce': 58985565}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6812', 'nonce': 58985565}, {'type': 'v1', 'addr': '172.21.15.192:6813', 'nonce': 58985565}]}, 'public_addr': '172.21.15.192:6809/58985565', 'cluster_addr': '172.21.15.192:6811/58985565', 'heartbeat_back_addr': '172.21.15.192:6815/58985565', 'heartbeat_front_addr': '172.21.15.192:6813/58985565', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'ab26136c-31c3-4090-9fa2-988dba4f731f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.2798919677734375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 18, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6816', 'nonce': 1283571573}, {'type': 'v1', 'addr': '172.21.15.192:6817', 'nonce': 1283571573}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6818', 'nonce': 1283571573}, {'type': 'v1', 'addr': '172.21.15.192:6819', 'nonce': 1283571573}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6822', 'nonce': 1283571573}, {'type': 'v1', 'addr': '172.21.15.192:6823', 'nonce': 1283571573}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6820', 'nonce': 1283571573}, {'type': 'v1', 'addr': '172.21.15.192:6821', 'nonce': 1283571573}]}, 'public_addr': '172.21.15.192:6817/1283571573', 'cluster_addr': '172.21.15.192:6819/1283571573', 'heartbeat_back_addr': '172.21.15.192:6823/1283571573', 'heartbeat_front_addr': '172.21.15.192:6821/1283571573', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:51:52.396946+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:51:52.453160+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:51:52.387606+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:51:52.360952+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:51:52.468681+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:51:52.401080+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.4', 'osds': [1, 5]}, {'pgid': '3.17', 'osds': [1, 0]}, {'pgid': '3.1d', 'osds': [0, 5]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-05-08T08:53:36.130 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-05-08T08:53:36.155 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:36.155 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:53:36.155 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:53:36.155 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:53:36.155 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.155 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:437 3.17 3:ea93a512:::rbd_data.1140100c9703.0000000000000077:head [set-alloc-hint object_size 2097152 write_size 2097152,write 1170432~1024] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e63)", 2024-05-08T08:53:36.155 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:08.390804+0000", 2024-05-08T08:53:36.156 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 27.76657522, 2024-05-08T08:53:36.156 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0406068690000001, 2024-05-08T08:53:36.156 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:36.156 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:36.156 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:53:36.156 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:36.156 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:36.156 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:36.156 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 437 2024-05-08T08:53:36.156 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.156 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:36.156 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.156 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:36.156 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390804+0000", 2024-05-08T08:53:36.156 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:36.156 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.157 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.157 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:36.157 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390804+0000", 2024-05-08T08:53:36.157 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:36.157 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.157 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.157 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:36.157 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390806+0000", 2024-05-08T08:53:36.157 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.666e-06 2024-05-08T08:53:36.157 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.157 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.157 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:36.157 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390818+0000", 2024-05-08T08:53:36.157 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2187999999999999e-05 2024-05-08T08:53:36.158 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.158 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.158 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:36.158 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390834+0000", 2024-05-08T08:53:36.158 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6191000000000001e-05 2024-05-08T08:53:36.158 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.158 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.158 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:36.158 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390845+0000", 2024-05-08T08:53:36.158 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0831e-05 2024-05-08T08:53:36.158 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.158 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.158 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:36.158 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390922+0000", 2024-05-08T08:53:36.159 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7253e-05 2024-05-08T08:53:36.159 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.159 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.159 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op must wait for map", 2024-05-08T08:53:36.159 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390930+0000", 2024-05-08T08:53:36.159 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8259999999999993e-06 2024-05-08T08:53:36.159 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.159 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.159 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:36.159 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392155+0000", 2024-05-08T08:53:36.159 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0012256649999999999 2024-05-08T08:53:36.159 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.159 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.159 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:53:36.159 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392165+0000", 2024-05-08T08:53:36.159 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5799999999999998e-06 2024-05-08T08:53:36.160 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.160 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.160 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:36.160 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.429596+0000", 2024-05-08T08:53:36.160 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0374313639999999 2024-05-08T08:53:36.160 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.160 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.160 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:36.160 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.429728+0000", 2024-05-08T08:53:36.160 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000131552 2024-05-08T08:53:36.160 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.160 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.160 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 0,1", 2024-05-08T08:53:36.160 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.429860+0000", 2024-05-08T08:53:36.160 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000132137 2024-05-08T08:53:36.161 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.161 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.161 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:53:36.161 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.430811+0000", 2024-05-08T08:53:36.161 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00095040200000000004 2024-05-08T08:53:36.161 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.161 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.161 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:53:36.161 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431126+0000", 2024-05-08T08:53:36.161 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000315006 2024-05-08T08:53:36.161 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.161 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.161 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:53:36.161 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431272+0000", 2024-05-08T08:53:36.161 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000146487 2024-05-08T08:53:36.162 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.162 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.162 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:53:36.162 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431315+0000", 2024-05-08T08:53:36.162 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2697000000000001e-05 2024-05-08T08:53:36.162 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.162 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.162 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:36.162 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431411+0000", 2024-05-08T08:53:36.162 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6024000000000001e-05 2024-05-08T08:53:36.162 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:36.162 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:36.162 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:36.162 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.162 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.163 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:439 3.17 3:ea93a512:::rbd_data.1140100c9703.0000000000000077:head [read 1163264~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-05-08T08:53:36.163 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:08.390871+0000", 2024-05-08T08:53:36.163 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 27.766507708999999, 2024-05-08T08:53:36.163 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0408790590000001, 2024-05-08T08:53:36.163 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:36.163 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:36.163 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:36.163 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:36.163 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:36.163 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:36.163 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 439 2024-05-08T08:53:36.163 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.163 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:36.163 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.163 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:36.164 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390871+0000", 2024-05-08T08:53:36.164 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:36.164 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.164 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.164 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:36.164 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390871+0000", 2024-05-08T08:53:36.164 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:36.164 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.164 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.164 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:36.164 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390872+0000", 2024-05-08T08:53:36.164 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6300000000000005e-07 2024-05-08T08:53:36.164 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.164 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.164 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:36.165 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390875+0000", 2024-05-08T08:53:36.165 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3459999999999998e-06 2024-05-08T08:53:36.165 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.165 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.165 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:36.165 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390891+0000", 2024-05-08T08:53:36.165 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5255e-05 2024-05-08T08:53:36.165 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.165 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.165 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:36.165 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390897+0000", 2024-05-08T08:53:36.165 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2360000000000001e-06 2024-05-08T08:53:36.165 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.165 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.165 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:36.166 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390995+0000", 2024-05-08T08:53:36.166 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8522000000000003e-05 2024-05-08T08:53:36.166 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.166 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.166 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting_for_map not empty", 2024-05-08T08:53:36.166 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.391005+0000", 2024-05-08T08:53:36.166 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1460000000000002e-06 2024-05-08T08:53:36.166 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.166 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.166 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:36.166 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392191+0000", 2024-05-08T08:53:36.166 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0011860270000000001 2024-05-08T08:53:36.166 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.166 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.166 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:53:36.167 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392197+0000", 2024-05-08T08:53:36.167 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4289999999999998e-06 2024-05-08T08:53:36.167 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.167 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.167 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:36.167 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.430596+0000", 2024-05-08T08:53:36.167 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0383989090000001 2024-05-08T08:53:36.167 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.167 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.167 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for rw locks", 2024-05-08T08:53:36.167 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.430676+0000", 2024-05-08T08:53:36.167 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9781000000000001e-05 2024-05-08T08:53:36.167 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.167 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.167 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:36.168 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431460+0000", 2024-05-08T08:53:36.168 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00078427300000000004 2024-05-08T08:53:36.168 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.168 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.168 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:36.168 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431540+0000", 2024-05-08T08:53:36.168 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0224999999999998e-05 2024-05-08T08:53:36.168 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.168 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.168 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:36.168 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431750+0000", 2024-05-08T08:53:36.168 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00021024699999999999 2024-05-08T08:53:36.168 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:36.168 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:36.169 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:36.169 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.169 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.169 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:945 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 4096000~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:36.169 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:53:36.169 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 26.706358141999999, 2024-05-08T08:53:36.169 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086774741000000002, 2024-05-08T08:53:36.169 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:36.169 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:36.169 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:36.169 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:36.169 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:36.169 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:36.169 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 945 2024-05-08T08:53:36.169 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.170 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:36.170 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.170 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:36.170 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:53:36.170 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:36.170 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.170 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.170 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:36.170 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:53:36.170 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:36.170 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.170 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.170 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:36.170 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:53:36.171 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3499999999999996e-07 2024-05-08T08:53:36.171 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.171 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.171 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:36.171 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451024+0000", 2024-05-08T08:53:36.171 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.627e-06 2024-05-08T08:53:36.171 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.171 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.171 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:36.171 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451037+0000", 2024-05-08T08:53:36.171 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2431e-05 2024-05-08T08:53:36.171 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.171 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.171 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:36.171 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451042+0000", 2024-05-08T08:53:36.172 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8409999999999996e-06 2024-05-08T08:53:36.172 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.172 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.172 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:36.172 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537608+0000", 2024-05-08T08:53:36.172 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086565243 2024-05-08T08:53:36.172 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.172 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.172 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:36.172 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537687+0000", 2024-05-08T08:53:36.172 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9519999999999998e-05 2024-05-08T08:53:36.172 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.172 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.172 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:36.173 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537796+0000", 2024-05-08T08:53:36.173 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000108544 2024-05-08T08:53:36.173 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:36.173 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:36.173 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:36.173 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.173 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.173 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:964 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 458752~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:36.173 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451051+0000", 2024-05-08T08:53:36.173 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 26.706327969, 2024-05-08T08:53:36.173 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086990490000000004, 2024-05-08T08:53:36.173 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:36.173 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:36.173 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:36.174 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:36.174 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:36.174 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:36.174 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 964 2024-05-08T08:53:36.174 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.174 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:36.174 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.174 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:36.174 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451051+0000", 2024-05-08T08:53:36.174 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:36.174 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.174 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.174 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:36.175 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451051+0000", 2024-05-08T08:53:36.175 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:36.175 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.175 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.175 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:36.175 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451052+0000", 2024-05-08T08:53:36.175 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7299999999999996e-07 2024-05-08T08:53:36.175 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.175 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.175 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:36.175 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451055+0000", 2024-05-08T08:53:36.175 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.2770000000000001e-06 2024-05-08T08:53:36.175 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.175 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.175 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:36.176 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451061+0000", 2024-05-08T08:53:36.176 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4060000000000002e-06 2024-05-08T08:53:36.176 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.176 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.176 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:36.176 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451066+0000", 2024-05-08T08:53:36.176 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0839999999999999e-06 2024-05-08T08:53:36.176 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.176 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.176 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:36.176 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537839+0000", 2024-05-08T08:53:36.176 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086772962999999995 2024-05-08T08:53:36.176 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.176 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.176 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:36.177 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537934+0000", 2024-05-08T08:53:36.177 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4742999999999997e-05 2024-05-08T08:53:36.177 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.177 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.177 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:36.177 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538042+0000", 2024-05-08T08:53:36.177 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000107444 2024-05-08T08:53:36.177 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:36.177 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:36.177 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:36.177 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.177 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.177 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:965 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 950272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:36.177 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451075+0000", 2024-05-08T08:53:36.177 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 26.706303906999999, 2024-05-08T08:53:36.178 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087230196999999995, 2024-05-08T08:53:36.178 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:36.178 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:36.178 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:36.178 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:36.178 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:36.178 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:36.178 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 965 2024-05-08T08:53:36.178 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.178 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:36.178 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.178 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:36.178 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451075+0000", 2024-05-08T08:53:36.178 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:36.179 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.179 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.179 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:36.179 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451075+0000", 2024-05-08T08:53:36.179 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:36.179 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.179 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.179 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:36.179 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451076+0000", 2024-05-08T08:53:36.179 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5000000000000002e-07 2024-05-08T08:53:36.179 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.179 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.179 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:36.179 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451078+0000", 2024-05-08T08:53:36.179 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.5639999999999999e-06 2024-05-08T08:53:36.180 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.180 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.180 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:36.180 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451085+0000", 2024-05-08T08:53:36.180 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5640000000000002e-06 2024-05-08T08:53:36.180 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.180 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.180 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:36.180 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451090+0000", 2024-05-08T08:53:36.180 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1699999999999996e-06 2024-05-08T08:53:36.180 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.180 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.180 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:36.180 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538106+0000", 2024-05-08T08:53:36.180 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087015888999999999 2024-05-08T08:53:36.180 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.180 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.181 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:36.181 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538190+0000", 2024-05-08T08:53:36.181 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3961999999999993e-05 2024-05-08T08:53:36.181 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.181 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.181 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:36.181 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538305+0000", 2024-05-08T08:53:36.181 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011529800000000001 2024-05-08T08:53:36.181 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:36.181 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:36.181 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:36.181 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.181 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.181 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:966 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 1441792~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:36.182 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:53:36.182 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 26.706280534000001, 2024-05-08T08:53:36.182 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087446323000000006, 2024-05-08T08:53:36.182 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:36.182 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:36.182 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:36.182 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:36.182 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:36.182 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:36.182 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 966 2024-05-08T08:53:36.182 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.182 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:36.182 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.182 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:36.183 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:53:36.183 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:36.183 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.183 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.183 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:36.183 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:53:36.183 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:36.183 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.183 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.183 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:36.183 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:53:36.183 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3700000000000001e-07 2024-05-08T08:53:36.183 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.183 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.183 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:36.184 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451101+0000", 2024-05-08T08:53:36.184 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2560000000000001e-06 2024-05-08T08:53:36.184 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.184 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.184 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:36.184 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451107+0000", 2024-05-08T08:53:36.184 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1070000000000001e-06 2024-05-08T08:53:36.184 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.184 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.184 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:36.184 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451112+0000", 2024-05-08T08:53:36.184 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1420000000000001e-06 2024-05-08T08:53:36.184 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.184 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.184 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:36.185 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538341+0000", 2024-05-08T08:53:36.185 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087228302999999993 2024-05-08T08:53:36.185 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.185 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.185 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:36.185 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538424+0000", 2024-05-08T08:53:36.185 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3497000000000004e-05 2024-05-08T08:53:36.185 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.185 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.185 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:36.185 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538545+0000", 2024-05-08T08:53:36.185 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000120581 2024-05-08T08:53:36.185 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:36.185 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:36.185 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:36.186 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.186 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.186 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:967 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:36.186 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451120+0000", 2024-05-08T08:53:36.186 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 26.706259059000001, 2024-05-08T08:53:36.186 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087632725999999994, 2024-05-08T08:53:36.186 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:36.186 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:36.186 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:36.186 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:36.186 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:36.186 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:36.186 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 967 2024-05-08T08:53:36.186 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.187 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:36.187 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.187 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:36.187 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451120+0000", 2024-05-08T08:53:36.187 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:36.187 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.187 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.187 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:36.187 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451120+0000", 2024-05-08T08:53:36.187 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:36.187 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.187 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.187 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:36.187 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451121+0000", 2024-05-08T08:53:36.188 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8800000000000002e-07 2024-05-08T08:53:36.188 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.188 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.188 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:36.188 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451130+0000", 2024-05-08T08:53:36.188 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2299999999999997e-06 2024-05-08T08:53:36.188 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.188 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.188 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:36.188 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451137+0000", 2024-05-08T08:53:36.188 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4669999999999998e-06 2024-05-08T08:53:36.188 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.188 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.188 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:36.188 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451143+0000", 2024-05-08T08:53:36.188 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2839999999999997e-06 2024-05-08T08:53:36.189 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.189 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.189 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:36.189 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538575+0000", 2024-05-08T08:53:36.189 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087432789999999996 2024-05-08T08:53:36.189 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.189 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.189 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:36.189 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538648+0000", 2024-05-08T08:53:36.189 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2877e-05 2024-05-08T08:53:36.189 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.189 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.189 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:36.189 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538753+0000", 2024-05-08T08:53:36.189 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010449 2024-05-08T08:53:36.189 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:36.190 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:36.190 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:36.190 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.190 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.190 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:968 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 2424832~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:36.190 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451152+0000", 2024-05-08T08:53:36.190 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 26.706226974, 2024-05-08T08:53:36.190 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087813325999999997, 2024-05-08T08:53:36.190 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:36.190 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:36.190 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:36.190 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:36.190 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:36.190 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:36.190 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 968 2024-05-08T08:53:36.191 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.191 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:36.191 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.191 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:36.191 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451152+0000", 2024-05-08T08:53:36.191 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:36.191 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.191 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.191 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:36.191 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451152+0000", 2024-05-08T08:53:36.191 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:36.191 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.191 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.191 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:36.191 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451153+0000", 2024-05-08T08:53:36.191 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3099999999999998e-07 2024-05-08T08:53:36.192 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.192 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.192 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:36.192 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451155+0000", 2024-05-08T08:53:36.192 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4760000000000001e-06 2024-05-08T08:53:36.192 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.192 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.192 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:36.192 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451162+0000", 2024-05-08T08:53:36.192 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5969999999999997e-06 2024-05-08T08:53:36.192 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.192 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.192 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:36.192 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451167+0000", 2024-05-08T08:53:36.192 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.977e-06 2024-05-08T08:53:36.193 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.193 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.193 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:36.193 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538781+0000", 2024-05-08T08:53:36.193 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087614846999999996 2024-05-08T08:53:36.193 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.193 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.193 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:36.193 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538857+0000", 2024-05-08T08:53:36.193 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5358000000000006e-05 2024-05-08T08:53:36.193 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.193 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.193 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:36.193 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538965+0000", 2024-05-08T08:53:36.193 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010864 2024-05-08T08:53:36.193 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:36.194 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:36.194 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:36.194 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.194 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.194 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:969 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 2916352~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:36.194 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:53:36.194 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 26.706205269000002, 2024-05-08T08:53:36.194 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087998980000000004, 2024-05-08T08:53:36.194 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:36.194 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:36.194 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:36.194 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:36.194 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:36.194 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:36.195 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 969 2024-05-08T08:53:36.195 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.195 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:36.195 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.195 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:36.195 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:53:36.195 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:36.195 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.195 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.195 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:36.195 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:53:36.195 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:36.195 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.195 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.195 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:36.196 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:53:36.196 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3599999999999999e-07 2024-05-08T08:53:36.196 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.196 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.196 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:36.196 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451177+0000", 2024-05-08T08:53:36.196 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4590000000000001e-06 2024-05-08T08:53:36.196 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.196 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.196 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:36.196 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451184+0000", 2024-05-08T08:53:36.196 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4109999999999999e-06 2024-05-08T08:53:36.196 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.196 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.196 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:36.196 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451191+0000", 2024-05-08T08:53:36.197 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7279999999999998e-06 2024-05-08T08:53:36.197 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.197 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.197 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:36.197 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538997+0000", 2024-05-08T08:53:36.197 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087806170000000003 2024-05-08T08:53:36.197 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.197 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.197 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:36.197 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539069+0000", 2024-05-08T08:53:36.197 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2373999999999995e-05 2024-05-08T08:53:36.197 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.197 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.197 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:36.197 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539173+0000", 2024-05-08T08:53:36.197 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000103402 2024-05-08T08:53:36.197 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:36.198 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:36.198 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:36.198 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.198 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.198 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:970 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 3407872~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:36.198 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451198+0000", 2024-05-08T08:53:36.198 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 26.706180671999999, 2024-05-08T08:53:36.198 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088183368999999998, 2024-05-08T08:53:36.198 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:36.198 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:36.198 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:36.198 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:36.198 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:36.198 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:36.199 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 970 2024-05-08T08:53:36.199 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.199 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:36.199 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.199 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:36.199 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451198+0000", 2024-05-08T08:53:36.199 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:36.199 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.199 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.199 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:36.199 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451198+0000", 2024-05-08T08:53:36.199 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:36.199 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.200 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.200 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:36.200 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451199+0000", 2024-05-08T08:53:36.200 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7400000000000003e-07 2024-05-08T08:53:36.200 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.200 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.200 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:36.200 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451211+0000", 2024-05-08T08:53:36.200 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2109999999999999e-05 2024-05-08T08:53:36.200 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.200 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.200 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:36.200 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451219+0000", 2024-05-08T08:53:36.200 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4170000000000003e-06 2024-05-08T08:53:36.201 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.201 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.201 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:36.201 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451224+0000", 2024-05-08T08:53:36.201 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3990000000000003e-06 2024-05-08T08:53:36.201 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.201 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.201 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:36.201 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539204+0000", 2024-05-08T08:53:36.201 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087980099000000006 2024-05-08T08:53:36.201 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.201 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.201 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:36.201 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539276+0000", 2024-05-08T08:53:36.201 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1513000000000005e-05 2024-05-08T08:53:36.201 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.202 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.202 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:36.202 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539382+0000", 2024-05-08T08:53:36.202 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010615699999999999 2024-05-08T08:53:36.202 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:36.202 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:36.202 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:36.202 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.202 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.202 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:971 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 3899392~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:36.202 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451233+0000", 2024-05-08T08:53:36.202 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 26.706145965000001, 2024-05-08T08:53:36.202 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088431525999999996, 2024-05-08T08:53:36.202 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:36.202 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:36.203 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:36.203 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:36.203 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:36.203 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:36.203 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 971 2024-05-08T08:53:36.203 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.203 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:36.203 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.203 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:36.203 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451233+0000", 2024-05-08T08:53:36.203 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:36.203 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.203 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.203 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:36.203 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451233+0000", 2024-05-08T08:53:36.203 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:36.204 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.204 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.204 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:36.204 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451234+0000", 2024-05-08T08:53:36.204 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8500000000000001e-07 2024-05-08T08:53:36.204 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.204 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.204 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:36.204 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451237+0000", 2024-05-08T08:53:36.204 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8320000000000002e-06 2024-05-08T08:53:36.204 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.204 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.204 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:36.204 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451243+0000", 2024-05-08T08:53:36.204 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6749999999999996e-06 2024-05-08T08:53:36.205 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.205 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.205 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:36.205 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451249+0000", 2024-05-08T08:53:36.205 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1229999999999997e-06 2024-05-08T08:53:36.205 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.205 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.205 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:36.205 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539412+0000", 2024-05-08T08:53:36.205 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088162668999999999 2024-05-08T08:53:36.205 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.205 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.205 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:36.205 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539505+0000", 2024-05-08T08:53:36.205 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3240999999999997e-05 2024-05-08T08:53:36.206 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.206 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.206 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:36.206 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539665+0000", 2024-05-08T08:53:36.206 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00015930100000000001 2024-05-08T08:53:36.206 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:36.206 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:36.206 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:36.206 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.206 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.206 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:972 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 196608~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:36.206 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451257+0000", 2024-05-08T08:53:36.206 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 26.706122019999999, 2024-05-08T08:53:36.206 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088719928000000003, 2024-05-08T08:53:36.207 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:36.207 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:36.207 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:36.207 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:36.207 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:36.207 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:36.207 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 972 2024-05-08T08:53:36.207 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.207 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:36.207 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.207 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:36.207 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451257+0000", 2024-05-08T08:53:36.207 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:36.207 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.207 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.208 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:36.208 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451257+0000", 2024-05-08T08:53:36.208 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:36.208 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.208 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.208 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:36.208 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451258+0000", 2024-05-08T08:53:36.208 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7999999999999996e-07 2024-05-08T08:53:36.208 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.208 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.208 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:36.208 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451260+0000", 2024-05-08T08:53:36.208 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4609999999999998e-06 2024-05-08T08:53:36.208 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.208 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.209 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:36.209 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451267+0000", 2024-05-08T08:53:36.209 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.81e-06 2024-05-08T08:53:36.209 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.209 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.209 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:36.209 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451273+0000", 2024-05-08T08:53:36.209 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0859999999999998e-06 2024-05-08T08:53:36.209 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.209 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.209 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:36.209 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539718+0000", 2024-05-08T08:53:36.209 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088445059000000006 2024-05-08T08:53:36.209 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.209 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.209 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:36.209 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539834+0000", 2024-05-08T08:53:36.210 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000116426 2024-05-08T08:53:36.210 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.210 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.210 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:36.210 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539977+0000", 2024-05-08T08:53:36.210 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014260600000000001 2024-05-08T08:53:36.210 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:36.210 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:36.210 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:36.210 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.210 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.210 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:973 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 688128~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:36.210 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:53:36.210 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 26.706099435999999, 2024-05-08T08:53:36.210 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088918496, 2024-05-08T08:53:36.211 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:36.211 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:36.211 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:36.211 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:36.211 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:36.211 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:36.211 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 973 2024-05-08T08:53:36.211 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.211 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:36.211 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.211 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:36.211 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:53:36.211 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:36.211 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.211 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.211 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:36.211 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:53:36.212 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:36.212 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.212 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.212 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:36.212 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:53:36.212 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9099999999999999e-07 2024-05-08T08:53:36.212 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.212 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.212 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:36.212 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451282+0000", 2024-05-08T08:53:36.212 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1550000000000001e-06 2024-05-08T08:53:36.212 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.212 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.212 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:36.212 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451288+0000", 2024-05-08T08:53:36.212 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8889999999999999e-06 2024-05-08T08:53:36.212 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.213 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.213 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:36.213 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451294+0000", 2024-05-08T08:53:36.213 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0850000000000002e-06 2024-05-08T08:53:36.213 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.213 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.213 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:36.213 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540011+0000", 2024-05-08T08:53:36.213 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088717314000000005 2024-05-08T08:53:36.213 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.213 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.213 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:36.213 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540089+0000", 2024-05-08T08:53:36.213 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7438e-05 2024-05-08T08:53:36.214 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.214 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.214 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:36.214 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540198+0000", 2024-05-08T08:53:36.214 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000109224 2024-05-08T08:53:36.214 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:36.214 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:36.214 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:36.214 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.214 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.214 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:974 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 1179648~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:36.214 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451301+0000", 2024-05-08T08:53:36.214 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 26.706077802999999, 2024-05-08T08:53:36.214 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089105078000000004, 2024-05-08T08:53:36.215 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:36.215 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:36.215 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:36.215 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:36.215 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:36.215 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:36.215 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 974 2024-05-08T08:53:36.215 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.216 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:36.216 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.216 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:36.216 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451301+0000", 2024-05-08T08:53:36.216 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:36.217 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.217 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.217 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:36.217 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451301+0000", 2024-05-08T08:53:36.217 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:36.217 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.218 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.218 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:36.218 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451302+0000", 2024-05-08T08:53:36.218 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7800000000000002e-07 2024-05-08T08:53:36.219 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.219 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.219 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:36.219 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451304+0000", 2024-05-08T08:53:36.219 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.65e-06 2024-05-08T08:53:36.220 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.220 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.220 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:36.220 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451311+0000", 2024-05-08T08:53:36.221 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.472e-06 2024-05-08T08:53:36.221 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.221 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.221 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:36.222 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451316+0000", 2024-05-08T08:53:36.222 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9189999999999998e-06 2024-05-08T08:53:36.222 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.222 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.223 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:36.223 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540234+0000", 2024-05-08T08:53:36.223 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088917752000000003 2024-05-08T08:53:36.223 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.224 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.224 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:36.224 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540308+0000", 2024-05-08T08:53:36.224 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4758000000000005e-05 2024-05-08T08:53:36.224 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.224 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.224 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:36.225 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540406+0000", 2024-05-08T08:53:36.225 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8048999999999996e-05 2024-05-08T08:53:36.225 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:36.225 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:36.225 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:36.225 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.225 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.225 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:975 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 1671168~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:36.225 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:53:36.225 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 26.706035473, 2024-05-08T08:53:36.225 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089294139999999994, 2024-05-08T08:53:36.225 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:36.225 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:36.225 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:36.225 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:36.226 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:36.226 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:36.226 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 975 2024-05-08T08:53:36.226 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.226 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:36.226 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.226 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:36.226 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:53:36.226 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:36.226 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.226 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.226 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:36.226 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:53:36.226 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:36.227 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.227 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.227 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:36.227 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:53:36.227 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1299999999999999e-07 2024-05-08T08:53:36.227 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.227 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.227 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:36.227 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451348+0000", 2024-05-08T08:53:36.227 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.484e-06 2024-05-08T08:53:36.227 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.227 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.227 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:36.227 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451356+0000", 2024-05-08T08:53:36.227 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8499999999999994e-06 2024-05-08T08:53:36.228 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.228 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.228 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:36.228 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451366+0000", 2024-05-08T08:53:36.228 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.9850000000000001e-06 2024-05-08T08:53:36.228 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.228 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.228 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:36.228 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540448+0000", 2024-05-08T08:53:36.228 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089082254 2024-05-08T08:53:36.228 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.229 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.229 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:36.229 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540528+0000", 2024-05-08T08:53:36.229 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0234999999999993e-05 2024-05-08T08:53:36.229 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.229 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.229 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:36.229 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540638+0000", 2024-05-08T08:53:36.229 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010951900000000001 2024-05-08T08:53:36.229 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:36.229 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:36.229 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:36.229 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.229 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.229 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:976 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 2162688~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:36.230 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451375+0000", 2024-05-08T08:53:36.230 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 26.706003725999999, 2024-05-08T08:53:36.230 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089472745000000006, 2024-05-08T08:53:36.230 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:36.230 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:36.230 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:36.230 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:36.230 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:36.230 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:36.230 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 976 2024-05-08T08:53:36.230 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.230 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:36.230 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.230 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:36.231 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451375+0000", 2024-05-08T08:53:36.231 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:36.231 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.231 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.231 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:36.231 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451375+0000", 2024-05-08T08:53:36.231 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:36.231 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.231 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.231 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:36.231 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451376+0000", 2024-05-08T08:53:36.231 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2399999999999998e-07 2024-05-08T08:53:36.231 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.231 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.231 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:36.232 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451379+0000", 2024-05-08T08:53:36.232 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6359999999999998e-06 2024-05-08T08:53:36.232 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.232 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.232 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:36.232 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451387+0000", 2024-05-08T08:53:36.232 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9929999999999995e-06 2024-05-08T08:53:36.232 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.232 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.232 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:36.232 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451393+0000", 2024-05-08T08:53:36.232 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4699999999999999e-06 2024-05-08T08:53:36.232 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.232 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.232 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:36.232 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540677+0000", 2024-05-08T08:53:36.233 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089283504999999999 2024-05-08T08:53:36.233 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.233 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.233 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:36.233 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540743+0000", 2024-05-08T08:53:36.233 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6303999999999994e-05 2024-05-08T08:53:36.233 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.233 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.233 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:36.233 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540848+0000", 2024-05-08T08:53:36.233 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000105213 2024-05-08T08:53:36.233 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:36.233 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:36.233 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:36.233 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.234 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.234 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:977 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 2654208~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:36.234 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:53:36.234 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 26.705978294000001, 2024-05-08T08:53:36.234 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089698473000000001, 2024-05-08T08:53:36.234 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:36.234 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:36.234 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:36.234 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:36.234 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:36.234 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:36.234 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 977 2024-05-08T08:53:36.234 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.234 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:36.234 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.234 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:36.235 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:53:36.235 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:36.235 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.235 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.235 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:36.235 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:53:36.235 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:36.235 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.235 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.235 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:36.235 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:53:36.235 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2899999999999999e-07 2024-05-08T08:53:36.235 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.235 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.235 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:36.236 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451404+0000", 2024-05-08T08:53:36.236 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6649999999999999e-06 2024-05-08T08:53:36.236 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.236 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.236 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:36.236 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451410+0000", 2024-05-08T08:53:36.236 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3269999999999997e-06 2024-05-08T08:53:36.236 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.236 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.236 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:36.236 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451415+0000", 2024-05-08T08:53:36.236 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9529999999999998e-06 2024-05-08T08:53:36.236 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.236 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.236 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:36.236 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540912+0000", 2024-05-08T08:53:36.237 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089496950000000006 2024-05-08T08:53:36.237 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.237 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.237 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:36.237 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541000+0000", 2024-05-08T08:53:36.237 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7429000000000006e-05 2024-05-08T08:53:36.237 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.237 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.237 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:36.237 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541099+0000", 2024-05-08T08:53:36.237 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.9720000000000001e-05 2024-05-08T08:53:36.237 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:36.237 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:36.237 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:36.237 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.238 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.238 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:978 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 3145728~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:36.238 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:53:36.238 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 26.705957086000002, 2024-05-08T08:53:36.238 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089900635000000007, 2024-05-08T08:53:36.238 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:36.238 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:36.238 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:36.238 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:36.238 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:36.238 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:36.238 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 978 2024-05-08T08:53:36.238 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.238 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:36.238 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.239 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:36.239 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:53:36.239 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:36.239 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.239 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.239 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:36.239 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:53:36.239 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:36.239 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.239 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.239 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:36.239 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:53:36.239 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9400000000000001e-07 2024-05-08T08:53:36.239 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.239 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.240 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:36.240 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451425+0000", 2024-05-08T08:53:36.240 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2460000000000002e-06 2024-05-08T08:53:36.240 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.240 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.240 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:36.240 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451431+0000", 2024-05-08T08:53:36.240 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2419999999999997e-06 2024-05-08T08:53:36.240 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.240 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.240 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:36.240 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451436+0000", 2024-05-08T08:53:36.240 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.887e-06 2024-05-08T08:53:36.240 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.240 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.241 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:36.241 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541152+0000", 2024-05-08T08:53:36.241 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089716194999999999 2024-05-08T08:53:36.241 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.241 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.241 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:36.241 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541224+0000", 2024-05-08T08:53:36.241 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2416000000000006e-05 2024-05-08T08:53:36.241 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.241 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.241 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:36.241 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541323+0000", 2024-05-08T08:53:36.241 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8255e-05 2024-05-08T08:53:36.241 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:36.241 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:36.242 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:36.242 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.242 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.242 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:979 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 3637248~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:36.242 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451462+0000", 2024-05-08T08:53:36.242 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 26.705916637000001, 2024-05-08T08:53:36.242 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.090036441999999994, 2024-05-08T08:53:36.242 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:36.242 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:36.242 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:36.242 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:36.242 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:36.242 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:36.242 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 979 2024-05-08T08:53:36.243 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.243 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:36.243 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.243 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:36.243 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451462+0000", 2024-05-08T08:53:36.243 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:36.243 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.243 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.243 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:36.243 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451462+0000", 2024-05-08T08:53:36.243 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:36.243 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.243 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.243 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:36.243 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451463+0000", 2024-05-08T08:53:36.243 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4300000000000002e-07 2024-05-08T08:53:36.244 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.244 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.244 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:36.244 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451471+0000", 2024-05-08T08:53:36.244 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0800000000000006e-06 2024-05-08T08:53:36.244 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.244 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.244 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:36.244 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451481+0000", 2024-05-08T08:53:36.244 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.38e-06 2024-05-08T08:53:36.244 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.244 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.244 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:36.244 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451487+0000", 2024-05-08T08:53:36.244 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4590000000000003e-06 2024-05-08T08:53:36.245 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.245 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.245 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:36.245 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541350+0000", 2024-05-08T08:53:36.245 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089862599000000001 2024-05-08T08:53:36.245 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.245 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.245 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:36.245 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541407+0000", 2024-05-08T08:53:36.245 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7667999999999997e-05 2024-05-08T08:53:36.245 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.245 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.245 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:36.245 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541499+0000", 2024-05-08T08:53:36.245 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1513000000000003e-05 2024-05-08T08:53:36.245 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:36.246 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:36.246 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:36.246 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.246 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.246 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:980 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 4128768~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:36.246 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451496+0000", 2024-05-08T08:53:36.246 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 26.705882881000001, 2024-05-08T08:53:36.246 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.090258985, 2024-05-08T08:53:36.246 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:36.246 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:36.246 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:36.246 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:36.246 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:36.246 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:36.246 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 980 2024-05-08T08:53:36.247 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.247 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:36.247 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.247 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:36.247 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451496+0000", 2024-05-08T08:53:36.247 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:36.247 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.247 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.247 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:36.247 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451496+0000", 2024-05-08T08:53:36.247 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:36.247 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.247 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.247 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:36.247 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451497+0000", 2024-05-08T08:53:36.248 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7599999999999997e-07 2024-05-08T08:53:36.248 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.248 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.248 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:36.248 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451500+0000", 2024-05-08T08:53:36.248 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7930000000000002e-06 2024-05-08T08:53:36.248 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.248 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.248 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:36.248 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451506+0000", 2024-05-08T08:53:36.248 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6980000000000001e-06 2024-05-08T08:53:36.248 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.248 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.248 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:36.248 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451512+0000", 2024-05-08T08:53:36.249 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2490000000000001e-06 2024-05-08T08:53:36.249 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.249 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.249 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:36.249 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541550+0000", 2024-05-08T08:53:36.249 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.090037961999999999 2024-05-08T08:53:36.249 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.249 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.249 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:36.249 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541647+0000", 2024-05-08T08:53:36.249 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6924999999999998e-05 2024-05-08T08:53:36.249 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:36.249 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:36.249 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:36.250 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541755+0000", 2024-05-08T08:53:36.250 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000108682 2024-05-08T08:53:36.250 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:36.250 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:36.250 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:36.250 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:36.250 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:36.250 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:36.252 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:53:36.252 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:36.255+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:36.352 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:36.355+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:36.369 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:53:36.369 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:53:36.369 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:53:36.369 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:53:36.378 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:53:36.453 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:36.456+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:36.484 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:53:36.484 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:53:36.484 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:53:36.484 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:53:36.484 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:53:36.493 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:53:36.529 INFO:teuthology.orchestra.run.smithi059.stderr:noscrub is set 2024-05-08T08:53:36.586 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:36.589+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:36.605 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:53:36.605 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:53:36.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:53:36.605 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:53:36.605 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.605 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4621.0:9089 3.1f e33/18 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:head v 33'117, mlcod=31'115)", 2024-05-08T08:53:36.605 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:53:36.605 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 64.087532523999997, 2024-05-08T08:53:36.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.111244524, 2024-05-08T08:53:36.606 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:36.606 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:36.606 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:53:36.606 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:36.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.606 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:36.606 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:53:36.606 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:36.606 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.606 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:36.606 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:53:36.606 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:36.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.607 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:36.607 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:53:36.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5900000000000001e-07 2024-05-08T08:53:36.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.607 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:36.607 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539600+0000", 2024-05-08T08:53:36.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.019981364000000001 2024-05-08T08:53:36.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.607 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:36.607 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539624+0000", 2024-05-08T08:53:36.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4043e-05 2024-05-08T08:53:36.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.608 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:36.608 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539635+0000", 2024-05-08T08:53:36.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1511e-05 2024-05-08T08:53:36.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.608 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:36.608 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539728+0000", 2024-05-08T08:53:36.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2498999999999996e-05 2024-05-08T08:53:36.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.608 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:36.608 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539746+0000", 2024-05-08T08:53:36.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8289000000000001e-05 2024-05-08T08:53:36.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.609 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:53:36.609 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.630822+0000", 2024-05-08T08:53:36.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.091076283999999993 2024-05-08T08:53:36.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.609 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:36.609 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.630863+0000", 2024-05-08T08:53:36.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0374999999999999e-05 2024-05-08T08:53:36.609 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:36.610 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:36.610 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:36.610 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.610 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4795.0:230 3.1f e40/18 3:f8db1a56:::rbd_data.11402b26385.0000000000000000:head v 40'171, mlcod=39'168)", 2024-05-08T08:53:36.610 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:45.927773+0000", 2024-05-08T08:53:36.610 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 50.679377441, 2024-05-08T08:53:36.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045892405999999997, 2024-05-08T08:53:36.610 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:36.610 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:36.610 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:53:36.610 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:36.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.611 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:36.611 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.927773+0000", 2024-05-08T08:53:36.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:36.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.611 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:36.611 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.927773+0000", 2024-05-08T08:53:36.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:36.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.611 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:36.611 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.927774+0000", 2024-05-08T08:53:36.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0500000000000002e-07 2024-05-08T08:53:36.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.611 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:36.611 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.931581+0000", 2024-05-08T08:53:36.612 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0038071799999999999 2024-05-08T08:53:36.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.612 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:36.612 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.931599+0000", 2024-05-08T08:53:36.612 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7479e-05 2024-05-08T08:53:36.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.612 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:36.612 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.931624+0000", 2024-05-08T08:53:36.612 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5372999999999999e-05 2024-05-08T08:53:36.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.612 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:36.613 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.931698+0000", 2024-05-08T08:53:36.613 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4043999999999998e-05 2024-05-08T08:53:36.613 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.613 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.613 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:36.613 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.931725+0000", 2024-05-08T08:53:36.613 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6883e-05 2024-05-08T08:53:36.613 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.613 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.613 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:53:36.613 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.973623+0000", 2024-05-08T08:53:36.613 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.041898283000000001 2024-05-08T08:53:36.613 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.613 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.613 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:36.614 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.973666+0000", 2024-05-08T08:53:36.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2358999999999999e-05 2024-05-08T08:53:36.614 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:36.614 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:36.614 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:36.614 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.614 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.614 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4195.0:4 3.11 3:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-05-08T08:53:36.614 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:49.764654+0000", 2024-05-08T08:53:36.614 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 46.842496982999997, 2024-05-08T08:53:36.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.30714275499999999, 2024-05-08T08:53:36.614 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:36.614 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:36.614 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-05-08T08:53:36.614 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:36.615 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4195", 2024-05-08T08:53:36.615 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/59179521", 2024-05-08T08:53:36.615 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4 2024-05-08T08:53:36.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.615 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:36.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:36.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764654+0000", 2024-05-08T08:53:36.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:36.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:36.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764654+0000", 2024-05-08T08:53:36.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:36.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:36.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764655+0000", 2024-05-08T08:53:36.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.452e-06 2024-05-08T08:53:36.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:36.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764661+0000", 2024-05-08T08:53:36.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5369999999999997e-06 2024-05-08T08:53:36.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:36.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764671+0000", 2024-05-08T08:53:36.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.028e-05 2024-05-08T08:53:36.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.617 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.617 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:36.617 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764680+0000", 2024-05-08T08:53:36.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0629999999999995e-06 2024-05-08T08:53:36.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.617 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.617 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:36.617 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764753+0000", 2024-05-08T08:53:36.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2679e-05 2024-05-08T08:53:36.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.617 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.617 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-05-08T08:53:36.617 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764765+0000", 2024-05-08T08:53:36.618 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2415000000000001e-05 2024-05-08T08:53:36.618 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.618 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.618 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:36.618 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:50.071692+0000", 2024-05-08T08:53:36.618 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.306927324 2024-05-08T08:53:36.618 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.618 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.618 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:36.618 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:50.071796+0000", 2024-05-08T08:53:36.618 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104005 2024-05-08T08:53:36.618 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:36.618 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:36.619 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:36.619 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.619 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.619 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22681 3.15 3:a907109d:::rbd_data.1140d3fc54c.000000000000004f:18 [read 1204224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:36.619 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.097653+0000", 2024-05-08T08:53:36.619 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.5094973549999997, 2024-05-08T08:53:36.619 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.041941285000000002, 2024-05-08T08:53:36.619 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:36.619 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:36.619 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:36.619 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:36.619 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:36.619 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:36.619 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22681 2024-05-08T08:53:36.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.620 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:36.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.620 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:36.620 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097653+0000", 2024-05-08T08:53:36.620 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:36.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.620 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:36.620 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097653+0000", 2024-05-08T08:53:36.620 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:36.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.620 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:36.620 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097654+0000", 2024-05-08T08:53:36.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9400000000000001e-07 2024-05-08T08:53:36.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:36.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097655+0000", 2024-05-08T08:53:36.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6220000000000001e-06 2024-05-08T08:53:36.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:36.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097661+0000", 2024-05-08T08:53:36.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5210000000000002e-06 2024-05-08T08:53:36.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.622 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:36.622 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097665+0000", 2024-05-08T08:53:36.622 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3170000000000003e-06 2024-05-08T08:53:36.622 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.622 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.622 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:36.622 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139509+0000", 2024-05-08T08:53:36.622 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.041843475999999998 2024-05-08T08:53:36.622 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.622 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.622 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:36.622 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139554+0000", 2024-05-08T08:53:36.622 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5478000000000001e-05 2024-05-08T08:53:36.622 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.622 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.623 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:36.623 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139595+0000", 2024-05-08T08:53:36.623 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0577000000000001e-05 2024-05-08T08:53:36.623 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:36.623 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:36.623 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:36.623 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.623 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.623 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22682 3.15 3:a907109d:::rbd_data.1140d3fc54c.000000000000004f:18 [read 1269760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:36.623 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.097670+0000", 2024-05-08T08:53:36.623 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.5094806629999997, 2024-05-08T08:53:36.623 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042031394, 2024-05-08T08:53:36.623 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:36.623 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:36.623 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:36.624 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:36.624 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:36.624 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:36.624 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22682 2024-05-08T08:53:36.624 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.624 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:36.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.624 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:36.624 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097670+0000", 2024-05-08T08:53:36.624 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:36.624 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.624 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:36.624 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097670+0000", 2024-05-08T08:53:36.624 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:36.625 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.625 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.625 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:36.625 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097670+0000", 2024-05-08T08:53:36.625 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1800000000000002e-07 2024-05-08T08:53:36.625 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.625 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.625 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:36.625 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097672+0000", 2024-05-08T08:53:36.625 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6929999999999999e-06 2024-05-08T08:53:36.625 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.625 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.625 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:36.625 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097677+0000", 2024-05-08T08:53:36.625 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2480000000000004e-06 2024-05-08T08:53:36.626 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.626 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.626 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:36.626 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097682+0000", 2024-05-08T08:53:36.626 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4290000000000003e-06 2024-05-08T08:53:36.626 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.626 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.626 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:36.626 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139620+0000", 2024-05-08T08:53:36.626 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.041938256 2024-05-08T08:53:36.626 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.626 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.626 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:36.626 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139662+0000", 2024-05-08T08:53:36.626 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1727999999999998e-05 2024-05-08T08:53:36.627 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.627 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.627 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:36.627 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139701+0000", 2024-05-08T08:53:36.627 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9721999999999997e-05 2024-05-08T08:53:36.627 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:36.627 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:36.627 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:36.627 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.627 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.627 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22683 3.15 3:a907109d:::rbd_data.1140d3fc54c.000000000000004f:18 [read 1335296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:36.627 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.097686+0000", 2024-05-08T08:53:36.627 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.5094643090000002, 2024-05-08T08:53:36.627 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042114716000000003, 2024-05-08T08:53:36.627 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:36.628 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:36.628 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:36.628 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:36.628 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:36.628 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:36.628 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22683 2024-05-08T08:53:36.628 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.628 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:36.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.628 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:36.628 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097686+0000", 2024-05-08T08:53:36.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:36.628 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.628 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:36.629 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097686+0000", 2024-05-08T08:53:36.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:36.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.629 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:36.629 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097687+0000", 2024-05-08T08:53:36.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3999999999999997e-07 2024-05-08T08:53:36.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.629 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:36.629 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097688+0000", 2024-05-08T08:53:36.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6789999999999999e-06 2024-05-08T08:53:36.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.629 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:36.630 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097693+0000", 2024-05-08T08:53:36.630 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.045e-06 2024-05-08T08:53:36.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.630 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.630 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:36.630 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097697+0000", 2024-05-08T08:53:36.630 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0260000000000001e-06 2024-05-08T08:53:36.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.630 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.630 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:36.630 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139725+0000", 2024-05-08T08:53:36.630 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042027737000000003 2024-05-08T08:53:36.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.630 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.631 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:36.631 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139764+0000", 2024-05-08T08:53:36.631 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8668999999999999e-05 2024-05-08T08:53:36.631 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.631 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.631 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:36.631 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139801+0000", 2024-05-08T08:53:36.631 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7219999999999999e-05 2024-05-08T08:53:36.631 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:36.631 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:36.631 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:36.631 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.631 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.631 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22684 3.15 3:a907109d:::rbd_data.1140d3fc54c.000000000000004f:18 [read 1400832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:36.631 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.097703+0000", 2024-05-08T08:53:36.632 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.5094479559999998, 2024-05-08T08:53:36.632 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042198929000000003, 2024-05-08T08:53:36.632 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:36.632 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:36.632 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:36.632 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:36.632 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:36.632 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:36.632 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22684 2024-05-08T08:53:36.632 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.632 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:36.632 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.632 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:36.632 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097703+0000", 2024-05-08T08:53:36.632 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:36.633 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.633 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.633 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:36.633 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097703+0000", 2024-05-08T08:53:36.633 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:36.633 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.633 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.633 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:36.633 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097703+0000", 2024-05-08T08:53:36.633 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4400000000000001e-07 2024-05-08T08:53:36.633 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.633 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.633 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:36.633 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097705+0000", 2024-05-08T08:53:36.634 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9199999999999998e-06 2024-05-08T08:53:36.634 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.634 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:36.634 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097711+0000", 2024-05-08T08:53:36.634 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5559999999999998e-06 2024-05-08T08:53:36.634 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.634 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:36.634 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097715+0000", 2024-05-08T08:53:36.634 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3159999999999998e-06 2024-05-08T08:53:36.634 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.634 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:36.635 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139825+0000", 2024-05-08T08:53:36.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042110284999999997 2024-05-08T08:53:36.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.635 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.635 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:36.635 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139864+0000", 2024-05-08T08:53:36.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8980000000000003e-05 2024-05-08T08:53:36.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.635 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.635 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:36.635 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139902+0000", 2024-05-08T08:53:36.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7527999999999997e-05 2024-05-08T08:53:36.635 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:36.635 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:36.635 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:36.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.636 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22685 3.15 3:a907109d:::rbd_data.1140d3fc54c.000000000000004f:18 [read 1466368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:36.636 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.097721+0000", 2024-05-08T08:53:36.636 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.5094292989999998, 2024-05-08T08:53:36.636 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042287791999999998, 2024-05-08T08:53:36.636 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:36.636 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:36.636 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:36.636 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:36.636 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:36.636 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:36.636 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22685 2024-05-08T08:53:36.636 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.636 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:36.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.636 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:36.637 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097721+0000", 2024-05-08T08:53:36.637 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:36.637 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.637 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.637 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:36.637 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097721+0000", 2024-05-08T08:53:36.637 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:36.637 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.637 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.637 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:36.637 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097722+0000", 2024-05-08T08:53:36.637 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2500000000000001e-07 2024-05-08T08:53:36.637 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.637 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.637 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:36.638 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097724+0000", 2024-05-08T08:53:36.638 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.835e-06 2024-05-08T08:53:36.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.638 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.638 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:36.638 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097729+0000", 2024-05-08T08:53:36.638 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5029999999999997e-06 2024-05-08T08:53:36.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.638 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.638 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:36.638 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097733+0000", 2024-05-08T08:53:36.638 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3309999999999997e-06 2024-05-08T08:53:36.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.638 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.638 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:36.639 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139923+0000", 2024-05-08T08:53:36.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042189988999999997 2024-05-08T08:53:36.639 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.639 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.639 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:36.639 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139970+0000", 2024-05-08T08:53:36.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6776000000000001e-05 2024-05-08T08:53:36.639 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.639 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.639 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:36.639 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140009+0000", 2024-05-08T08:53:36.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9032999999999997e-05 2024-05-08T08:53:36.639 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:36.639 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:36.639 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:36.639 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.640 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.640 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22686 3.15 3:a907109d:::rbd_data.1140d3fc54c.000000000000004f:18 [read 1531904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:36.640 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.097739+0000", 2024-05-08T08:53:36.640 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.5094113, 2024-05-08T08:53:36.640 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042371008000000002, 2024-05-08T08:53:36.640 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:36.640 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:36.640 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:36.640 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:36.640 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:36.640 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:36.640 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22686 2024-05-08T08:53:36.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.640 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:36.640 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.640 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:36.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097739+0000", 2024-05-08T08:53:36.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:36.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.641 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:36.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097739+0000", 2024-05-08T08:53:36.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:36.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.641 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:36.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097740+0000", 2024-05-08T08:53:36.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8599999999999999e-07 2024-05-08T08:53:36.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:36.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097742+0000", 2024-05-08T08:53:36.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9410000000000001e-06 2024-05-08T08:53:36.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:36.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097747+0000", 2024-05-08T08:53:36.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2079999999999999e-06 2024-05-08T08:53:36.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:36.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097751+0000", 2024-05-08T08:53:36.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2250000000000002e-06 2024-05-08T08:53:36.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.643 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:36.643 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140032+0000", 2024-05-08T08:53:36.643 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042281244000000003 2024-05-08T08:53:36.643 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.643 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.643 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:36.643 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140074+0000", 2024-05-08T08:53:36.643 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1316999999999998e-05 2024-05-08T08:53:36.643 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.643 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.643 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:36.643 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140110+0000", 2024-05-08T08:53:36.643 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6687000000000003e-05 2024-05-08T08:53:36.643 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:36.643 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:36.644 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:36.644 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.644 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.644 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22716 3.15 3:a88fdee9:::rbd_data.1140d3fc54c.000000000000004d:18 [read 1097728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:36.644 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.098153+0000", 2024-05-08T08:53:36.644 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.5089975930000001, 2024-05-08T08:53:36.644 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042058535000000001, 2024-05-08T08:53:36.644 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:36.644 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:36.644 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:36.644 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:36.644 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:36.644 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:36.644 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22716 2024-05-08T08:53:36.644 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.645 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:36.645 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.645 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:36.645 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098153+0000", 2024-05-08T08:53:36.645 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:36.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.645 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.645 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:36.645 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098153+0000", 2024-05-08T08:53:36.645 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:36.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.645 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.645 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:36.645 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098154+0000", 2024-05-08T08:53:36.645 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.13e-07 2024-05-08T08:53:36.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.646 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:36.646 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098161+0000", 2024-05-08T08:53:36.646 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7360000000000001e-06 2024-05-08T08:53:36.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.646 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:36.646 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098171+0000", 2024-05-08T08:53:36.646 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1039999999999996e-06 2024-05-08T08:53:36.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.646 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:36.646 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098177+0000", 2024-05-08T08:53:36.646 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5760000000000002e-06 2024-05-08T08:53:36.647 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.647 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.647 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:36.647 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140135+0000", 2024-05-08T08:53:36.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.041957842000000002 2024-05-08T08:53:36.647 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.647 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.647 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:36.647 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140174+0000", 2024-05-08T08:53:36.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8902999999999997e-05 2024-05-08T08:53:36.647 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.647 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.647 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:36.647 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140212+0000", 2024-05-08T08:53:36.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7761e-05 2024-05-08T08:53:36.648 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:36.648 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:36.648 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:36.648 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.648 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.648 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22717 3.15 3:a88fdee9:::rbd_data.1140d3fc54c.000000000000004d:18 [read 1163264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:36.648 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.098188+0000", 2024-05-08T08:53:36.648 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.5089628709999996, 2024-05-08T08:53:36.648 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042124907000000003, 2024-05-08T08:53:36.648 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:36.648 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:36.648 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:36.648 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:36.648 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:36.648 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:36.649 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22717 2024-05-08T08:53:36.649 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.649 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:36.649 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.649 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:36.649 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098188+0000", 2024-05-08T08:53:36.649 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:36.649 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.649 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.649 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:36.649 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098188+0000", 2024-05-08T08:53:36.649 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:36.649 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.649 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.649 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:36.650 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098188+0000", 2024-05-08T08:53:36.650 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3799999999999998e-07 2024-05-08T08:53:36.650 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.650 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.650 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:36.650 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098190+0000", 2024-05-08T08:53:36.650 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.114e-06 2024-05-08T08:53:36.650 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.650 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.650 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:36.650 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098197+0000", 2024-05-08T08:53:36.650 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9959999999999996e-06 2024-05-08T08:53:36.650 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.650 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.650 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:36.651 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098202+0000", 2024-05-08T08:53:36.651 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9110000000000001e-06 2024-05-08T08:53:36.651 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.651 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.651 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:36.651 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140235+0000", 2024-05-08T08:53:36.651 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042032913999999998 2024-05-08T08:53:36.651 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.651 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.651 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:36.651 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140275+0000", 2024-05-08T08:53:36.651 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9564999999999998e-05 2024-05-08T08:53:36.651 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.651 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.652 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:36.652 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140313+0000", 2024-05-08T08:53:36.652 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7969000000000002e-05 2024-05-08T08:53:36.652 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:36.652 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:36.652 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:36.652 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.652 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.652 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22718 3.15 3:a88fdee9:::rbd_data.1140d3fc54c.000000000000004d:18 [read 1228800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:36.652 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.098209+0000", 2024-05-08T08:53:36.652 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.5089417249999997, 2024-05-08T08:53:36.652 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.04220409, 2024-05-08T08:53:36.652 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:36.652 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:36.652 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:36.652 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:36.653 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:36.653 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:36.653 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22718 2024-05-08T08:53:36.653 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.653 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:36.653 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.653 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:36.653 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098209+0000", 2024-05-08T08:53:36.653 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:36.653 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.653 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.653 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:36.653 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098209+0000", 2024-05-08T08:53:36.653 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:36.653 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.653 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.654 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:36.654 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098209+0000", 2024-05-08T08:53:36.654 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9299999999999999e-07 2024-05-08T08:53:36.654 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.654 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.654 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:36.654 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098212+0000", 2024-05-08T08:53:36.654 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.21e-06 2024-05-08T08:53:36.654 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.654 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.654 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:36.654 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098219+0000", 2024-05-08T08:53:36.654 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0269999999999998e-06 2024-05-08T08:53:36.654 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.655 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.655 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:36.655 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098223+0000", 2024-05-08T08:53:36.655 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7319999999999998e-06 2024-05-08T08:53:36.655 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.655 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.655 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:36.655 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140338+0000", 2024-05-08T08:53:36.655 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042114390000000002 2024-05-08T08:53:36.655 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.655 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.655 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:36.655 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140376+0000", 2024-05-08T08:53:36.655 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8414999999999997e-05 2024-05-08T08:53:36.655 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.656 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.656 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:36.656 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140413+0000", 2024-05-08T08:53:36.656 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6922999999999999e-05 2024-05-08T08:53:36.656 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:36.656 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:36.656 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:36.656 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.656 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.656 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22719 3.15 3:a88fdee9:::rbd_data.1140d3fc54c.000000000000004d:18 [read 1294336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:36.656 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.098229+0000", 2024-05-08T08:53:36.656 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.5089218740000003, 2024-05-08T08:53:36.656 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042283016, 2024-05-08T08:53:36.656 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:36.656 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:36.656 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:36.657 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:36.657 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:36.657 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:36.657 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22719 2024-05-08T08:53:36.657 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.657 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:36.657 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.657 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:36.657 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098229+0000", 2024-05-08T08:53:36.657 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:36.657 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.657 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.657 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:36.657 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098229+0000", 2024-05-08T08:53:36.657 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:36.657 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.657 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.658 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:36.658 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098229+0000", 2024-05-08T08:53:36.658 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:53:36.658 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.658 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.658 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:36.658 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098231+0000", 2024-05-08T08:53:36.658 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.81e-06 2024-05-08T08:53:36.658 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.658 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.658 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:36.658 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098237+0000", 2024-05-08T08:53:36.658 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3389999999999998e-06 2024-05-08T08:53:36.659 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.659 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.659 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:36.659 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098243+0000", 2024-05-08T08:53:36.659 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.648e-06 2024-05-08T08:53:36.659 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.659 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.659 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:36.659 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140436+0000", 2024-05-08T08:53:36.659 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042192738 2024-05-08T08:53:36.659 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.659 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.659 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:36.659 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140475+0000", 2024-05-08T08:53:36.660 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9279999999999997e-05 2024-05-08T08:53:36.660 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.660 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.660 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:36.660 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140512+0000", 2024-05-08T08:53:36.660 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6847000000000001e-05 2024-05-08T08:53:36.660 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:36.660 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:36.660 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:36.660 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.660 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.660 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22720 3.15 3:a88fdee9:::rbd_data.1140d3fc54c.000000000000004d:18 [read 1359872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:36.660 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.098248+0000", 2024-05-08T08:53:36.660 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.5089023629999998, 2024-05-08T08:53:36.660 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042360800999999997, 2024-05-08T08:53:36.661 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:36.661 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:36.661 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:36.661 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:36.661 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:36.661 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:36.661 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22720 2024-05-08T08:53:36.661 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.661 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:36.661 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.661 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:36.661 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098248+0000", 2024-05-08T08:53:36.661 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:36.661 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.661 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.662 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:36.662 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098248+0000", 2024-05-08T08:53:36.662 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:36.662 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.662 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.662 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:36.662 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098491+0000", 2024-05-08T08:53:36.662 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000242659 2024-05-08T08:53:36.662 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.662 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.662 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:36.662 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098495+0000", 2024-05-08T08:53:36.662 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.799e-06 2024-05-08T08:53:36.662 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.663 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.663 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:36.663 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098505+0000", 2024-05-08T08:53:36.663 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7079999999999992e-06 2024-05-08T08:53:36.663 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.663 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.663 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:36.663 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098511+0000", 2024-05-08T08:53:36.663 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4370000000000004e-06 2024-05-08T08:53:36.663 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.663 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.663 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:36.663 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140533+0000", 2024-05-08T08:53:36.663 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042022371000000003 2024-05-08T08:53:36.664 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.664 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.664 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:36.664 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140571+0000", 2024-05-08T08:53:36.664 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7700000000000002e-05 2024-05-08T08:53:36.664 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.664 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.664 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:36.664 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140609+0000", 2024-05-08T08:53:36.664 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8127000000000003e-05 2024-05-08T08:53:36.664 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:36.664 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:36.664 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:36.665 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.665 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.665 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22721 3.15 3:a88fdee9:::rbd_data.1140d3fc54c.000000000000004d:18 [read 1425408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:36.665 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.098522+0000", 2024-05-08T08:53:36.665 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.5086287389999997, 2024-05-08T08:53:36.665 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042185779999999999, 2024-05-08T08:53:36.665 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:36.665 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:36.665 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:36.665 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:36.665 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:36.665 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:36.665 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22721 2024-05-08T08:53:36.665 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.665 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:36.665 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.666 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:36.666 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098522+0000", 2024-05-08T08:53:36.666 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:36.666 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.666 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.666 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:36.666 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098522+0000", 2024-05-08T08:53:36.666 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:36.666 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.666 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.666 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:36.666 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098708+0000", 2024-05-08T08:53:36.666 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00018629700000000001 2024-05-08T08:53:36.666 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.667 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.667 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:36.667 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098719+0000", 2024-05-08T08:53:36.667 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0992000000000001e-05 2024-05-08T08:53:36.667 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.667 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.667 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:36.667 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098730+0000", 2024-05-08T08:53:36.667 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0431999999999999e-05 2024-05-08T08:53:36.667 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.667 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.667 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:36.667 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098737+0000", 2024-05-08T08:53:36.667 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.063e-06 2024-05-08T08:53:36.667 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.667 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.668 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:36.668 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140631+0000", 2024-05-08T08:53:36.668 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.041894503999999999 2024-05-08T08:53:36.668 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.668 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.668 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:36.668 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140671+0000", 2024-05-08T08:53:36.668 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9780999999999997e-05 2024-05-08T08:53:36.668 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.668 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.668 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:36.668 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140708+0000", 2024-05-08T08:53:36.668 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6711000000000001e-05 2024-05-08T08:53:36.668 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:36.669 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:36.669 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:36.669 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.669 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.669 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22722 3.15 3:a88fdee9:::rbd_data.1140d3fc54c.000000000000004d:18 [read 1490944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:36.669 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.098753+0000", 2024-05-08T08:53:36.669 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.508397735, 2024-05-08T08:53:36.669 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042053944000000003, 2024-05-08T08:53:36.669 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:36.669 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:36.669 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:36.669 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:36.669 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:36.669 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:36.669 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22722 2024-05-08T08:53:36.670 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.670 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:36.670 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.670 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:36.670 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098753+0000", 2024-05-08T08:53:36.670 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:36.670 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.670 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.670 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:36.670 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098753+0000", 2024-05-08T08:53:36.670 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:36.670 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.670 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.670 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:36.671 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098753+0000", 2024-05-08T08:53:36.671 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1399999999999997e-07 2024-05-08T08:53:36.671 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.671 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.671 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:36.671 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098756+0000", 2024-05-08T08:53:36.671 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9780000000000001e-06 2024-05-08T08:53:36.671 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.671 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.671 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:36.671 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098764+0000", 2024-05-08T08:53:36.671 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4340000000000003e-06 2024-05-08T08:53:36.671 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.672 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.672 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:36.672 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098770+0000", 2024-05-08T08:53:36.672 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8080000000000001e-06 2024-05-08T08:53:36.672 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.672 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.672 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:36.672 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140731+0000", 2024-05-08T08:53:36.672 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.041961131999999998 2024-05-08T08:53:36.672 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.672 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.672 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:36.672 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140769+0000", 2024-05-08T08:53:36.672 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.773e-05 2024-05-08T08:53:36.672 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.673 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.673 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:36.673 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140807+0000", 2024-05-08T08:53:36.673 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8347999999999999e-05 2024-05-08T08:53:36.673 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:36.673 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:36.673 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:36.673 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.673 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.673 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22723 3.15 3:a88fdee9:::rbd_data.1140d3fc54c.000000000000004d:18 [read 1556480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:36.673 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.098778+0000", 2024-05-08T08:53:36.673 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.5083729630000002, 2024-05-08T08:53:36.673 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042128577, 2024-05-08T08:53:36.673 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:36.673 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:36.674 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:36.674 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:36.674 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:36.674 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:36.674 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22723 2024-05-08T08:53:36.674 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.674 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:36.674 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.674 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:36.674 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098778+0000", 2024-05-08T08:53:36.674 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:36.674 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.674 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.675 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:36.675 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098778+0000", 2024-05-08T08:53:36.675 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:36.675 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.675 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.675 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:36.675 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098834+0000", 2024-05-08T08:53:36.675 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5788000000000002e-05 2024-05-08T08:53:36.675 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.675 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.675 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:36.675 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098837+0000", 2024-05-08T08:53:36.675 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3510000000000002e-06 2024-05-08T08:53:36.675 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.676 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.676 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:36.676 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098845+0000", 2024-05-08T08:53:36.676 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5490000000000006e-06 2024-05-08T08:53:36.676 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.676 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.676 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:36.676 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098861+0000", 2024-05-08T08:53:36.676 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5233e-05 2024-05-08T08:53:36.676 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.676 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.676 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:36.676 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140829+0000", 2024-05-08T08:53:36.676 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.04196863 2024-05-08T08:53:36.676 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.677 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.677 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:36.677 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140869+0000", 2024-05-08T08:53:36.677 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9461e-05 2024-05-08T08:53:36.677 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.677 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.677 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:36.677 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140906+0000", 2024-05-08T08:53:36.677 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7564999999999997e-05 2024-05-08T08:53:36.677 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:36.677 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:36.677 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:36.677 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.677 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.678 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22732 3.15 3:a907109d:::rbd_data.1140d3fc54c.000000000000004f:18 [read 1097728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:36.678 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.098871+0000", 2024-05-08T08:53:36.678 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.5082794829999999, 2024-05-08T08:53:36.678 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042141517000000003, 2024-05-08T08:53:36.678 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:36.678 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:36.678 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:36.678 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:36.678 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:36.678 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:36.678 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22732 2024-05-08T08:53:36.678 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.678 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:36.679 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.679 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:36.679 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098871+0000", 2024-05-08T08:53:36.679 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:36.679 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.679 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.679 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:36.679 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098871+0000", 2024-05-08T08:53:36.679 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:36.679 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.679 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.679 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:36.679 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.099940+0000", 2024-05-08T08:53:36.679 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0010683279999999999 2024-05-08T08:53:36.679 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.680 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.680 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:36.680 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.099944+0000", 2024-05-08T08:53:36.680 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9339999999999997e-06 2024-05-08T08:53:36.680 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.680 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.680 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:36.680 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.099956+0000", 2024-05-08T08:53:36.680 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1970999999999999e-05 2024-05-08T08:53:36.680 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.680 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.680 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:36.680 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.099968+0000", 2024-05-08T08:53:36.680 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.189e-05 2024-05-08T08:53:36.680 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.681 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.681 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:36.681 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140930+0000", 2024-05-08T08:53:36.681 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.040961806000000003 2024-05-08T08:53:36.681 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.681 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.681 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:36.681 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140974+0000", 2024-05-08T08:53:36.681 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3918999999999997e-05 2024-05-08T08:53:36.681 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.681 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.681 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:36.681 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.141013+0000", 2024-05-08T08:53:36.681 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8668999999999999e-05 2024-05-08T08:53:36.681 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:36.681 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:36.682 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:36.682 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.682 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.682 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22765 3.15 3:a9c637ae:::rbd_data.114071bcf828.000000000000013f:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:36.682 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.100902+0000", 2024-05-08T08:53:36.682 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.5062486880000003, 2024-05-08T08:53:36.682 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042471271999999997, 2024-05-08T08:53:36.682 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:36.682 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:36.682 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:36.682 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:36.682 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:36.682 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:36.682 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22765 2024-05-08T08:53:36.682 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.683 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:36.683 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.683 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:36.683 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.100902+0000", 2024-05-08T08:53:36.683 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:36.683 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.683 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.683 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:36.683 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.100902+0000", 2024-05-08T08:53:36.683 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:36.683 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.683 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.683 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:36.683 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.100902+0000", 2024-05-08T08:53:36.683 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.39e-07 2024-05-08T08:53:36.684 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.684 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.684 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:36.684 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.100904+0000", 2024-05-08T08:53:36.684 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6479999999999999e-06 2024-05-08T08:53:36.684 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.684 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.684 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:36.684 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.100910+0000", 2024-05-08T08:53:36.684 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1399999999999997e-06 2024-05-08T08:53:36.684 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.684 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.684 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:36.684 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.100916+0000", 2024-05-08T08:53:36.685 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6999999999999996e-06 2024-05-08T08:53:36.685 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.685 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.685 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:36.685 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.141733+0000", 2024-05-08T08:53:36.685 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.040817271000000002 2024-05-08T08:53:36.685 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.685 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.685 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:36.685 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.141812+0000", 2024-05-08T08:53:36.685 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9270999999999994e-05 2024-05-08T08:53:36.685 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.685 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.686 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:36.686 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.143373+0000", 2024-05-08T08:53:36.686 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001560903 2024-05-08T08:53:36.686 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:36.686 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:36.686 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:36.686 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.686 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.686 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:23031 3.15 3:ac9fb9cd:::rbd_data.114071bcf828.00000000000000f7:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:36.686 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.113562+0000", 2024-05-08T08:53:36.686 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.4935889280000003, 2024-05-08T08:53:36.686 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042560938999999999, 2024-05-08T08:53:36.686 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:36.686 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:36.686 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:53:36.687 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:36.687 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:36.687 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:36.687 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 23031 2024-05-08T08:53:36.687 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.687 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:36.687 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.687 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:36.687 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.113562+0000", 2024-05-08T08:53:36.687 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:36.687 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.687 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.687 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:36.687 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.113562+0000", 2024-05-08T08:53:36.687 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:36.688 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.688 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.688 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:36.688 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.113938+0000", 2024-05-08T08:53:36.688 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000376401 2024-05-08T08:53:36.688 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.688 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.688 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:36.688 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.113943+0000", 2024-05-08T08:53:36.688 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0529999999999997e-06 2024-05-08T08:53:36.688 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.688 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.688 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:36.688 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.113951+0000", 2024-05-08T08:53:36.688 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8569999999999995e-06 2024-05-08T08:53:36.689 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.689 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.689 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:36.689 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.113959+0000", 2024-05-08T08:53:36.689 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4710000000000001e-06 2024-05-08T08:53:36.689 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.689 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.689 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:36.689 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.153955+0000", 2024-05-08T08:53:36.689 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.039996747999999999 2024-05-08T08:53:36.689 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.689 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.689 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:36.689 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.154014+0000", 2024-05-08T08:53:36.689 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8464999999999997e-05 2024-05-08T08:53:36.690 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.690 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.690 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-05-08T08:53:36.690 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.154148+0000", 2024-05-08T08:53:36.690 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00013428600000000001 2024-05-08T08:53:36.690 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.690 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.690 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:53:36.690 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.155390+0000", 2024-05-08T08:53:36.690 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001241792 2024-05-08T08:53:36.690 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.690 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.690 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:53:36.690 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.156068+0000", 2024-05-08T08:53:36.690 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00067823700000000005 2024-05-08T08:53:36.691 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.691 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.691 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:53:36.691 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.156102+0000", 2024-05-08T08:53:36.691 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3812000000000001e-05 2024-05-08T08:53:36.691 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:36.691 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:36.691 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:36.691 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.156123+0000", 2024-05-08T08:53:36.691 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0817000000000001e-05 2024-05-08T08:53:36.691 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:36.691 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:36.691 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:36.691 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:36.691 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:36.691 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:53:36.692 INFO:teuthology.orchestra.run.smithi196.stdout:flattening image image_client.0-clone23 2024-05-08T08:53:36.693 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:53:36.694 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:53:36.696+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:53:36.793 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:36.796+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:36.801 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:53:36.801 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:53:36.802 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:53:36.802 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:53:36.811 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:53:36.895 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:36.898+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:36.960 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:53:36.960 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:53:36.960 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:53:36.960 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:53:36.960 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:53:36.969 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:53:37.010 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:37.013+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:37.033 INFO:teuthology.orchestra.run.smithi196.stdout:checking clone #22, image image_client.0-clone22 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent23 2024-05-08T08:53:37.090 INFO:teuthology.orchestra.run.smithi196.stdout:3275 write 0xc779200 thru 0xc7875dd (0xe3de bytes) 2024-05-08T08:53:37.091 INFO:teuthology.orchestra.run.smithi196.stdout:3280 write 0xb4f0400 thru 0xb4f66de (0x62df bytes) 2024-05-08T08:53:37.091 INFO:teuthology.orchestra.run.smithi196.stdout:3281 punch from 0xa2e5600 to 0xa2eae00, (0x5800 bytes) 2024-05-08T08:53:37.095 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:37.098+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:37.109 INFO:teuthology.orchestra.run.smithi196.stdout:3285 write 0x6292800 thru 0x629b81d (0x901e bytes) 2024-05-08T08:53:37.109 INFO:teuthology.orchestra.run.smithi196.stdout:3286 read 0x385c600 thru 0x38661ae (0x9baf bytes) 2024-05-08T08:53:37.119 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:53:37.119 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:53:37.119 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:53:37.119 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:53:37.119 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.119 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22071 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 765952~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:37.120 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857643+0000", 2024-05-08T08:53:37.120 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 69.263678065999997, 2024-05-08T08:53:37.120 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047106528000000002, 2024-05-08T08:53:37.120 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:37.120 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:37.120 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:37.120 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:37.120 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:37.120 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:37.120 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22071 2024-05-08T08:53:37.120 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.120 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:37.120 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.120 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:37.121 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857643+0000", 2024-05-08T08:53:37.121 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:37.121 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.121 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.121 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:37.121 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857643+0000", 2024-05-08T08:53:37.121 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:37.121 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.121 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.121 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:37.121 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857644+0000", 2024-05-08T08:53:37.121 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2000000000000001e-07 2024-05-08T08:53:37.121 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.121 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.121 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:37.122 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857646+0000", 2024-05-08T08:53:37.122 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0059999999999999e-06 2024-05-08T08:53:37.122 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.122 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.122 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:37.122 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857651+0000", 2024-05-08T08:53:37.122 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.84e-06 2024-05-08T08:53:37.122 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.122 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.122 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:37.122 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857657+0000", 2024-05-08T08:53:37.122 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5620000000000003e-06 2024-05-08T08:53:37.122 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.122 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.122 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:37.122 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904620+0000", 2024-05-08T08:53:37.123 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046962661000000003 2024-05-08T08:53:37.123 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.123 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.123 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:37.123 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904678+0000", 2024-05-08T08:53:37.123 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8656999999999998e-05 2024-05-08T08:53:37.123 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.123 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.123 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:37.123 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904750+0000", 2024-05-08T08:53:37.123 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1482000000000004e-05 2024-05-08T08:53:37.123 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.123 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:37.123 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.123 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.124 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.124 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22073 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 782336~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:37.124 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:53:37.124 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 69.263654058, 2024-05-08T08:53:37.124 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047244234000000003, 2024-05-08T08:53:37.124 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:37.124 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:37.124 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:37.124 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:37.124 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:37.124 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:37.124 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22073 2024-05-08T08:53:37.124 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.124 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:37.125 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.125 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:37.125 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:53:37.125 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:37.125 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.125 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.125 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:37.125 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:53:37.125 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:37.125 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.125 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.125 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:37.125 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857668+0000", 2024-05-08T08:53:37.125 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7800000000000002e-07 2024-05-08T08:53:37.126 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.126 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.126 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:37.126 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857670+0000", 2024-05-08T08:53:37.126 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0820000000000001e-06 2024-05-08T08:53:37.126 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.126 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.126 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:37.126 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857678+0000", 2024-05-08T08:53:37.126 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7589999999999997e-06 2024-05-08T08:53:37.126 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.126 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.126 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:37.126 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857691+0000", 2024-05-08T08:53:37.126 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3672999999999999e-05 2024-05-08T08:53:37.127 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.127 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.127 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:37.127 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904780+0000", 2024-05-08T08:53:37.127 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047088616999999999 2024-05-08T08:53:37.127 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.127 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.127 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:37.127 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904840+0000", 2024-05-08T08:53:37.127 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0356999999999998e-05 2024-05-08T08:53:37.127 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.127 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.127 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:37.127 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904912+0000", 2024-05-08T08:53:37.128 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1367999999999998e-05 2024-05-08T08:53:37.128 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.128 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:37.128 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.128 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.128 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.128 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22075 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 798720~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:37.128 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:53:37.128 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 69.263623660999997, 2024-05-08T08:53:37.128 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047384643999999997, 2024-05-08T08:53:37.128 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:37.128 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:37.128 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:37.128 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:37.128 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:37.128 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:37.129 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22075 2024-05-08T08:53:37.129 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.129 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:37.129 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.129 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:37.129 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:53:37.129 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:37.129 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.129 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.129 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:37.129 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:53:37.129 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:37.129 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.129 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.129 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:37.129 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:53:37.129 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7399999999999999e-07 2024-05-08T08:53:37.130 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.130 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.130 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:37.130 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857705+0000", 2024-05-08T08:53:37.130 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1980000000000004e-06 2024-05-08T08:53:37.130 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.130 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.130 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:37.130 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857713+0000", 2024-05-08T08:53:37.130 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7039999999999994e-06 2024-05-08T08:53:37.130 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.130 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.130 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:37.130 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857718+0000", 2024-05-08T08:53:37.130 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.865e-06 2024-05-08T08:53:37.131 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.131 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.131 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:37.131 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904941+0000", 2024-05-08T08:53:37.131 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047223198000000001 2024-05-08T08:53:37.131 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.131 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.131 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:37.131 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904999+0000", 2024-05-08T08:53:37.131 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8254999999999997e-05 2024-05-08T08:53:37.131 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.131 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.131 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:37.131 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905082+0000", 2024-05-08T08:53:37.131 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3049999999999999e-05 2024-05-08T08:53:37.132 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.132 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:37.132 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.132 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.132 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.132 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22077 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 815104~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:37.132 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:53:37.132 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 69.263585649999996, 2024-05-08T08:53:37.132 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047519448999999998, 2024-05-08T08:53:37.132 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:37.132 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:37.132 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:37.132 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:37.132 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:37.132 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:37.133 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22077 2024-05-08T08:53:37.133 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.133 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:37.133 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.133 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:37.133 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:53:37.133 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:37.133 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.133 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.133 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:37.133 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:53:37.133 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:37.133 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.133 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.133 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:37.134 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:53:37.134 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.99e-07 2024-05-08T08:53:37.134 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.134 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.134 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:37.134 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857739+0000", 2024-05-08T08:53:37.134 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3599999999999999e-06 2024-05-08T08:53:37.134 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.134 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.134 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:37.134 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857745+0000", 2024-05-08T08:53:37.134 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5139999999999998e-06 2024-05-08T08:53:37.134 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.134 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.135 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:37.135 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857750+0000", 2024-05-08T08:53:37.135 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8550000000000001e-06 2024-05-08T08:53:37.135 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.135 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.135 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:37.135 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905110+0000", 2024-05-08T08:53:37.135 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047360126000000002 2024-05-08T08:53:37.135 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.135 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.135 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:37.135 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905175+0000", 2024-05-08T08:53:37.135 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4689000000000004e-05 2024-05-08T08:53:37.135 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.135 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.135 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:37.136 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905255+0000", 2024-05-08T08:53:37.136 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0306e-05 2024-05-08T08:53:37.136 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.136 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:37.136 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.136 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.136 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.136 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22079 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 831488~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:37.136 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:53:37.136 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 69.263565102000001, 2024-05-08T08:53:37.136 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.04767499, 2024-05-08T08:53:37.136 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:37.136 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:37.136 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:37.136 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:37.136 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:37.137 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:37.137 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22079 2024-05-08T08:53:37.137 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.137 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:37.137 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.137 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:37.137 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:53:37.137 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:37.137 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.137 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.137 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:37.137 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:53:37.137 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:37.137 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.137 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.138 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:37.138 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857757+0000", 2024-05-08T08:53:37.138 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8299999999999997e-07 2024-05-08T08:53:37.138 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.138 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.138 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:37.138 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857759+0000", 2024-05-08T08:53:37.138 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1390000000000002e-06 2024-05-08T08:53:37.138 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.138 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.138 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:37.138 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857765+0000", 2024-05-08T08:53:37.138 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4609999999999996e-06 2024-05-08T08:53:37.138 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.138 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.139 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:37.139 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857770+0000", 2024-05-08T08:53:37.139 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6689999999999997e-06 2024-05-08T08:53:37.139 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.139 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.139 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:37.139 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905281+0000", 2024-05-08T08:53:37.139 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047510668999999998 2024-05-08T08:53:37.139 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.139 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.139 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:37.139 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905344+0000", 2024-05-08T08:53:37.139 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2797000000000002e-05 2024-05-08T08:53:37.139 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.139 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.139 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:37.140 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905431+0000", 2024-05-08T08:53:37.140 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7771999999999999e-05 2024-05-08T08:53:37.140 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.140 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:37.140 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.140 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.140 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.140 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22082 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 847872~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:37.140 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:53:37.140 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 69.263545410999996, 2024-05-08T08:53:37.140 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047831934999999999, 2024-05-08T08:53:37.140 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:37.140 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:37.140 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:37.140 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:37.140 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:37.141 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:37.141 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22082 2024-05-08T08:53:37.141 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.141 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:37.141 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.141 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:37.141 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:53:37.141 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:37.141 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.141 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.141 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:37.141 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:53:37.141 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:37.141 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.141 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.142 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:37.142 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:53:37.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:53:37.142 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.142 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.142 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:37.142 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857778+0000", 2024-05-08T08:53:37.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0789999999999999e-06 2024-05-08T08:53:37.142 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.142 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.142 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:37.142 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857784+0000", 2024-05-08T08:53:37.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0480000000000004e-06 2024-05-08T08:53:37.142 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.142 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.143 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:37.143 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857790+0000", 2024-05-08T08:53:37.143 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2719999999999997e-06 2024-05-08T08:53:37.143 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.143 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.143 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:37.143 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905458+0000", 2024-05-08T08:53:37.143 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047668470999999997 2024-05-08T08:53:37.143 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.143 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.143 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:37.143 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905521+0000", 2024-05-08T08:53:37.143 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2952000000000003e-05 2024-05-08T08:53:37.143 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.144 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.144 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:37.144 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905608+0000", 2024-05-08T08:53:37.144 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6759000000000001e-05 2024-05-08T08:53:37.144 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.144 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:37.144 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.144 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.144 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.144 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22084 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 864256~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:37.144 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:53:37.144 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 69.263521460999996, 2024-05-08T08:53:37.144 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047977748000000001, 2024-05-08T08:53:37.144 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:37.144 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:37.144 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:37.145 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:37.145 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:37.145 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:37.145 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22084 2024-05-08T08:53:37.145 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.145 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:37.145 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.145 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:37.145 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:53:37.145 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:37.145 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.145 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.145 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:37.145 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:53:37.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:37.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.146 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.146 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:37.146 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:53:37.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:53:37.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.146 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.146 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:37.146 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857802+0000", 2024-05-08T08:53:37.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0779999999999998e-06 2024-05-08T08:53:37.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.146 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.146 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:37.147 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857812+0000", 2024-05-08T08:53:37.147 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7939999999999997e-06 2024-05-08T08:53:37.147 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.147 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.149 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:37.149 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857817+0000", 2024-05-08T08:53:37.149 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8899999999999998e-06 2024-05-08T08:53:37.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.149 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:37.150 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905636+0000", 2024-05-08T08:53:37.150 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047819252 2024-05-08T08:53:37.150 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.150 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.150 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:37.150 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905700+0000", 2024-05-08T08:53:37.150 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4016999999999994e-05 2024-05-08T08:53:37.150 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.150 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.150 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:37.150 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905778+0000", 2024-05-08T08:53:37.150 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7306999999999997e-05 2024-05-08T08:53:37.150 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.150 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:37.151 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.151 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.151 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.151 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22086 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 880640~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:37.151 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:53:37.151 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 69.263498111000004, 2024-05-08T08:53:37.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048119149, 2024-05-08T08:53:37.151 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:37.151 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:37.151 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:37.151 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:37.151 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:37.151 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:37.151 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22086 2024-05-08T08:53:37.151 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.152 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:37.152 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.152 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:37.152 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:53:37.152 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:37.152 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.152 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.152 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:37.152 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:53:37.152 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:37.152 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.152 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.152 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:37.153 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857824+0000", 2024-05-08T08:53:37.153 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4200000000000002e-07 2024-05-08T08:53:37.153 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.153 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.153 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:37.153 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857826+0000", 2024-05-08T08:53:37.153 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0590000000000001e-06 2024-05-08T08:53:37.153 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.153 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.153 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:37.153 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857832+0000", 2024-05-08T08:53:37.153 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8029999999999999e-06 2024-05-08T08:53:37.153 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.153 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.153 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:37.154 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857837+0000", 2024-05-08T08:53:37.154 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7929999999999997e-06 2024-05-08T08:53:37.154 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.154 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.154 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:37.154 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905803+0000", 2024-05-08T08:53:37.154 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047965761000000003 2024-05-08T08:53:37.154 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.154 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.154 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:37.154 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905862+0000", 2024-05-08T08:53:37.154 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9203000000000002e-05 2024-05-08T08:53:37.154 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.155 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.155 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:37.155 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905942+0000", 2024-05-08T08:53:37.155 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0187999999999999e-05 2024-05-08T08:53:37.155 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.155 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:37.155 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.155 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.155 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.155 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22088 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 897024~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:37.155 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:53:37.155 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 69.263478058999993, 2024-05-08T08:53:37.155 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048256049000000002, 2024-05-08T08:53:37.155 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:37.155 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:37.156 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:37.156 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:37.156 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:37.156 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:37.156 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22088 2024-05-08T08:53:37.156 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.156 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:37.156 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.156 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:37.156 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:53:37.156 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:37.156 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.156 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.156 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:37.156 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:53:37.157 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:37.157 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.157 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.157 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:37.157 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857844+0000", 2024-05-08T08:53:37.157 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.46e-07 2024-05-08T08:53:37.157 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.157 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.157 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:37.157 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857846+0000", 2024-05-08T08:53:37.157 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0899999999999999e-06 2024-05-08T08:53:37.157 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.157 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.157 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:37.158 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857855+0000", 2024-05-08T08:53:37.158 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8219999999999996e-06 2024-05-08T08:53:37.158 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.158 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.158 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:37.158 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857860+0000", 2024-05-08T08:53:37.158 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2989999999999996e-06 2024-05-08T08:53:37.158 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.158 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.158 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:37.158 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905966+0000", 2024-05-08T08:53:37.158 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048106542000000002 2024-05-08T08:53:37.158 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.158 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.158 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:37.159 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906024+0000", 2024-05-08T08:53:37.159 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7958000000000003e-05 2024-05-08T08:53:37.159 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.159 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.159 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:37.159 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906099+0000", 2024-05-08T08:53:37.159 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4992000000000003e-05 2024-05-08T08:53:37.159 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.159 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:37.159 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.159 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.159 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.159 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22091 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 913408~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:37.159 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:53:37.160 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 69.263454299000003, 2024-05-08T08:53:37.160 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048395290000000001, 2024-05-08T08:53:37.160 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:37.160 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:37.160 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:37.160 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:37.160 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:37.160 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:37.160 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22091 2024-05-08T08:53:37.160 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.160 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:37.160 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.160 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:37.160 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:53:37.160 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:37.161 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.161 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.161 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:37.161 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:53:37.161 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:37.161 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.161 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.161 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:37.161 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:53:37.161 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4900000000000001e-07 2024-05-08T08:53:37.161 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.161 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.161 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:37.161 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857870+0000", 2024-05-08T08:53:37.162 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2510000000000001e-06 2024-05-08T08:53:37.162 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.162 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.162 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:37.162 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857877+0000", 2024-05-08T08:53:37.162 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0809999999999996e-06 2024-05-08T08:53:37.162 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.162 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.162 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:37.162 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857881+0000", 2024-05-08T08:53:37.162 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.724e-06 2024-05-08T08:53:37.162 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.162 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.162 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:37.163 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906127+0000", 2024-05-08T08:53:37.163 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048245345000000002 2024-05-08T08:53:37.163 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.163 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.163 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:37.163 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906190+0000", 2024-05-08T08:53:37.163 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2762000000000007e-05 2024-05-08T08:53:37.163 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.163 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.163 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:37.163 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906262+0000", 2024-05-08T08:53:37.163 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2778e-05 2024-05-08T08:53:37.163 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.164 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:37.164 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.164 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.164 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.164 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22093 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 929792~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:37.164 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:53:37.164 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 69.263431935, 2024-05-08T08:53:37.164 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048532875000000003, 2024-05-08T08:53:37.164 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:37.164 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:37.164 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:37.164 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:37.164 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:37.164 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:37.164 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22093 2024-05-08T08:53:37.165 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.165 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:37.165 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.165 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:37.165 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:53:37.165 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:37.165 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.165 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.165 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:37.165 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:53:37.165 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:37.165 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.165 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.165 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:37.165 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857890+0000", 2024-05-08T08:53:37.166 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-07 2024-05-08T08:53:37.166 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.166 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.166 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:37.166 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857899+0000", 2024-05-08T08:53:37.166 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6889999999999999e-06 2024-05-08T08:53:37.166 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.166 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.166 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:37.166 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857906+0000", 2024-05-08T08:53:37.166 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6490000000000002e-06 2024-05-08T08:53:37.166 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.166 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.166 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:37.166 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857917+0000", 2024-05-08T08:53:37.167 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0764999999999999e-05 2024-05-08T08:53:37.167 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.167 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.167 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:37.167 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906287+0000", 2024-05-08T08:53:37.167 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048370442 2024-05-08T08:53:37.167 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.167 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.167 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:37.167 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906349+0000", 2024-05-08T08:53:37.167 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1537000000000004e-05 2024-05-08T08:53:37.167 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.167 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.167 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:37.167 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906422+0000", 2024-05-08T08:53:37.167 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3431999999999998e-05 2024-05-08T08:53:37.168 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.168 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:37.168 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.168 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.168 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.168 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22095 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 946176~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:37.168 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:53:37.168 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 69.263397230999999, 2024-05-08T08:53:37.168 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048657593999999998, 2024-05-08T08:53:37.168 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:37.168 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:37.168 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:37.168 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:37.168 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:37.168 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:37.168 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22095 2024-05-08T08:53:37.169 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.169 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:37.169 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.169 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:37.169 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:53:37.169 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:37.169 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.169 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.169 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:37.169 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:53:37.169 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:37.169 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.169 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.169 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:37.169 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857925+0000", 2024-05-08T08:53:37.169 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3099999999999998e-07 2024-05-08T08:53:37.170 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.170 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.170 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:37.170 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857927+0000", 2024-05-08T08:53:37.170 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.34e-06 2024-05-08T08:53:37.170 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.170 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.170 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:37.170 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857938+0000", 2024-05-08T08:53:37.170 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1426e-05 2024-05-08T08:53:37.170 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.170 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.170 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:37.170 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857943+0000", 2024-05-08T08:53:37.170 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5140000000000003e-06 2024-05-08T08:53:37.171 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.171 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.171 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:37.171 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906448+0000", 2024-05-08T08:53:37.171 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048504641000000001 2024-05-08T08:53:37.171 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.171 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.171 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:37.171 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906510+0000", 2024-05-08T08:53:37.171 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2663000000000006e-05 2024-05-08T08:53:37.171 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.171 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.171 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:37.171 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906582+0000", 2024-05-08T08:53:37.172 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1478999999999998e-05 2024-05-08T08:53:37.172 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.172 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:37.172 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.172 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.172 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.172 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22097 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 962560~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:37.172 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:53:37.172 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 69.263369542000007, 2024-05-08T08:53:37.172 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048791353000000003, 2024-05-08T08:53:37.172 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:37.172 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:37.172 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:37.172 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:37.172 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:37.173 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:37.173 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22097 2024-05-08T08:53:37.173 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.173 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:37.173 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.173 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:37.173 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:53:37.173 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:37.173 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.173 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.173 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:37.173 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:53:37.173 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:37.173 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.174 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.174 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:37.174 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:53:37.174 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9200000000000002e-07 2024-05-08T08:53:37.174 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.174 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.174 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:37.174 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857954+0000", 2024-05-08T08:53:37.174 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1900000000000002e-06 2024-05-08T08:53:37.175 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.175 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.175 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:37.175 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857961+0000", 2024-05-08T08:53:37.175 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4989999999999999e-06 2024-05-08T08:53:37.175 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.175 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.175 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:37.175 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857971+0000", 2024-05-08T08:53:37.175 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0485999999999999e-05 2024-05-08T08:53:37.175 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.175 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.175 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:37.175 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906607+0000", 2024-05-08T08:53:37.176 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048635109000000003 2024-05-08T08:53:37.176 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.176 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.176 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:37.176 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906667+0000", 2024-05-08T08:53:37.176 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0439000000000002e-05 2024-05-08T08:53:37.176 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.176 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.176 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:37.176 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906743+0000", 2024-05-08T08:53:37.176 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6237999999999998e-05 2024-05-08T08:53:37.176 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.176 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:37.176 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.176 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.176 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.177 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22099 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 978944~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:37.177 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:53:37.177 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 69.263343382000002, 2024-05-08T08:53:37.177 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048932055000000002, 2024-05-08T08:53:37.177 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:37.177 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:37.177 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:37.177 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:37.177 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:37.177 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:37.177 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22099 2024-05-08T08:53:37.177 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.177 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:37.178 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.178 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:37.178 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:53:37.178 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:37.178 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.178 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.178 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:37.178 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:53:37.178 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:37.178 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.178 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.178 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:37.178 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:53:37.179 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.27e-07 2024-05-08T08:53:37.179 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.179 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.179 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:37.179 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857987+0000", 2024-05-08T08:53:37.179 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5890000000000003e-06 2024-05-08T08:53:37.179 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.179 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.179 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:37.179 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857994+0000", 2024-05-08T08:53:37.179 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6619999999999999e-06 2024-05-08T08:53:37.179 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.179 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.179 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:37.179 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857998+0000", 2024-05-08T08:53:37.180 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7110000000000003e-06 2024-05-08T08:53:37.180 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.180 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.180 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:37.180 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906772+0000", 2024-05-08T08:53:37.180 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048773415000000001 2024-05-08T08:53:37.180 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.180 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.180 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:37.180 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906834+0000", 2024-05-08T08:53:37.180 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1752000000000001e-05 2024-05-08T08:53:37.180 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.181 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.181 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:37.181 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906910+0000", 2024-05-08T08:53:37.181 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6499000000000001e-05 2024-05-08T08:53:37.181 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.181 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:37.181 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.181 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.181 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.181 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22101 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 995328~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:37.181 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:53:37.181 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 69.263316251000006, 2024-05-08T08:53:37.181 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049087364000000001, 2024-05-08T08:53:37.181 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:37.181 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:37.182 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:37.182 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:37.182 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:37.182 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:37.182 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22101 2024-05-08T08:53:37.182 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.182 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:37.182 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.182 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:37.182 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:53:37.182 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:37.183 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.183 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.183 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:37.183 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:53:37.183 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:37.183 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.183 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.183 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:37.183 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858006+0000", 2024-05-08T08:53:37.183 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1399999999999997e-07 2024-05-08T08:53:37.183 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.183 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.183 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:37.183 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858008+0000", 2024-05-08T08:53:37.184 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2060000000000001e-06 2024-05-08T08:53:37.184 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.184 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.184 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:37.184 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858017+0000", 2024-05-08T08:53:37.184 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1579999999999994e-06 2024-05-08T08:53:37.184 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.184 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.184 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:37.184 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858022+0000", 2024-05-08T08:53:37.184 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4199999999999998e-06 2024-05-08T08:53:37.184 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.184 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.184 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:37.184 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906940+0000", 2024-05-08T08:53:37.185 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048917627999999998 2024-05-08T08:53:37.185 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.185 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.185 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:37.185 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907002+0000", 2024-05-08T08:53:37.185 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2399999999999999e-05 2024-05-08T08:53:37.185 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.185 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.185 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:37.185 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907092+0000", 2024-05-08T08:53:37.185 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0137999999999997e-05 2024-05-08T08:53:37.185 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.185 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:37.186 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.186 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.186 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.186 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22103 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1011712~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:37.186 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:53:37.186 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 69.263291465999998, 2024-05-08T08:53:37.186 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049228542, 2024-05-08T08:53:37.186 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:37.186 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:37.186 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:37.186 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:37.186 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:37.186 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:37.186 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22103 2024-05-08T08:53:37.187 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.187 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:37.187 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.187 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:37.187 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:53:37.187 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:37.187 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.187 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.187 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:37.187 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:53:37.187 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:37.187 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.187 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.187 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:37.188 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:53:37.188 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4299999999999999e-07 2024-05-08T08:53:37.188 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.188 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.188 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:37.188 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858032+0000", 2024-05-08T08:53:37.188 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0590000000000001e-06 2024-05-08T08:53:37.188 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.188 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.188 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:37.188 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858037+0000", 2024-05-08T08:53:37.188 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.977e-06 2024-05-08T08:53:37.188 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.189 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.189 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:37.189 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858042+0000", 2024-05-08T08:53:37.189 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7650000000000001e-06 2024-05-08T08:53:37.189 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.189 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.189 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:37.189 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907118+0000", 2024-05-08T08:53:37.189 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049076147000000001 2024-05-08T08:53:37.189 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.189 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.189 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:37.189 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907180+0000", 2024-05-08T08:53:37.189 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1971999999999996e-05 2024-05-08T08:53:37.189 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.190 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.190 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:37.190 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907258+0000", 2024-05-08T08:53:37.190 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8279e-05 2024-05-08T08:53:37.190 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.190 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:37.190 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.190 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.190 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.190 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22105 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1028096~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:37.191 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:53:37.191 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 69.263272223000001, 2024-05-08T08:53:37.191 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049394609999999999, 2024-05-08T08:53:37.191 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:37.191 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:37.191 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:37.191 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:37.191 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:37.191 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:37.191 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22105 2024-05-08T08:53:37.191 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.191 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:37.191 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.191 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:37.191 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:53:37.192 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:37.192 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.192 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.192 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:37.192 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:53:37.192 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:37.192 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.192 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.192 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:37.192 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858050+0000", 2024-05-08T08:53:37.192 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5999999999999999e-07 2024-05-08T08:53:37.192 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.192 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.192 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:37.193 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858051+0000", 2024-05-08T08:53:37.193 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8870000000000001e-06 2024-05-08T08:53:37.193 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.193 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.193 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:37.193 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858057+0000", 2024-05-08T08:53:37.193 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8019999999999997e-06 2024-05-08T08:53:37.193 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.193 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.193 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:37.193 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858062+0000", 2024-05-08T08:53:37.193 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.561e-06 2024-05-08T08:53:37.194 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.194 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.194 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:37.194 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907289+0000", 2024-05-08T08:53:37.194 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049226957000000002 2024-05-08T08:53:37.194 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.194 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.194 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:37.194 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907356+0000", 2024-05-08T08:53:37.194 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6915000000000006e-05 2024-05-08T08:53:37.194 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.194 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.194 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:37.194 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907444+0000", 2024-05-08T08:53:37.195 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8127999999999994e-05 2024-05-08T08:53:37.195 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.195 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:37.195 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.195 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.195 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.195 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22107 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1044480~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:37.195 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:53:37.195 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 69.263253883000004, 2024-05-08T08:53:37.195 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049799942999999999, 2024-05-08T08:53:37.195 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:37.195 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:37.195 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:37.196 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:37.196 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:37.196 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:37.196 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22107 2024-05-08T08:53:37.196 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.196 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:37.196 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.196 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:37.196 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:53:37.196 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:37.196 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.196 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.196 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:37.196 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:53:37.196 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:37.197 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.197 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.197 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:37.197 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:53:37.197 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4799999999999999e-07 2024-05-08T08:53:37.197 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.197 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.197 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:37.197 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858070+0000", 2024-05-08T08:53:37.197 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.012e-06 2024-05-08T08:53:37.197 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.197 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.197 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:37.198 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858077+0000", 2024-05-08T08:53:37.198 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6290000000000001e-06 2024-05-08T08:53:37.198 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.198 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.198 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:37.198 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858083+0000", 2024-05-08T08:53:37.198 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8880000000000002e-06 2024-05-08T08:53:37.198 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.198 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.198 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:37.198 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907671+0000", 2024-05-08T08:53:37.198 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049587191000000003 2024-05-08T08:53:37.198 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.199 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.199 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:37.199 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907763+0000", 2024-05-08T08:53:37.199 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2231000000000005e-05 2024-05-08T08:53:37.199 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.199 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.199 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:37.199 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907867+0000", 2024-05-08T08:53:37.199 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104644 2024-05-08T08:53:37.199 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.199 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:37.199 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.199 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.199 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.200 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22294 3.1c 3:3aad0389:::rbd_data.11406886b07b.0000000000000080:head [stat,delete] snapc 7=[7] RETRY=6 ondisk+retry+write+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:37.200 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:53:37.200 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 69.262660034999996, 2024-05-08T08:53:37.200 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051152191, 2024-05-08T08:53:37.200 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:37.200 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:37.200 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:37.200 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:37.200 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:37.200 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:37.200 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22294 2024-05-08T08:53:37.200 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.200 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:37.200 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.201 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:37.201 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:53:37.201 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:37.201 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.201 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.201 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:37.201 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:53:37.201 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:37.201 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.201 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.201 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:37.201 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858662+0000", 2024-05-08T08:53:37.201 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4799999999999998e-07 2024-05-08T08:53:37.202 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.202 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.202 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:37.202 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858664+0000", 2024-05-08T08:53:37.202 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1229999999999998e-06 2024-05-08T08:53:37.202 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.202 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.202 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:37.202 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858670+0000", 2024-05-08T08:53:37.202 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3099999999999997e-06 2024-05-08T08:53:37.202 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.202 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.202 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:37.203 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858676+0000", 2024-05-08T08:53:37.203 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9750000000000004e-06 2024-05-08T08:53:37.203 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.203 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.203 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:37.203 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907901+0000", 2024-05-08T08:53:37.203 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049224520000000001 2024-05-08T08:53:37.203 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.203 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.203 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:37.203 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.908085+0000", 2024-05-08T08:53:37.203 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00018377500000000001 2024-05-08T08:53:37.203 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.203 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.204 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:37.204 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.909814+0000", 2024-05-08T08:53:37.204 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00172894 2024-05-08T08:53:37.204 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.204 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:37.204 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.204 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.204 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.204 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4416.0:13461 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-05-08T08:53:37.204 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:22.487456+0000", 2024-05-08T08:53:37.204 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 14.633864994, 2024-05-08T08:53:37.204 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.99102620799999996, 2024-05-08T08:53:37.204 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:37.204 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:37.204 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:37.205 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:37.205 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4416", 2024-05-08T08:53:37.205 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:37.205 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 13461 2024-05-08T08:53:37.205 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.205 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:37.205 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.205 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:37.205 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487456+0000", 2024-05-08T08:53:37.205 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:37.205 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.205 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.205 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:37.205 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487456+0000", 2024-05-08T08:53:37.206 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:37.206 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.206 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.206 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:37.206 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487458+0000", 2024-05-08T08:53:37.206 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5659999999999999e-06 2024-05-08T08:53:37.206 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.206 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.206 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:37.206 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487463+0000", 2024-05-08T08:53:37.206 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6890000000000003e-06 2024-05-08T08:53:37.206 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.206 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.206 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:37.206 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487472+0000", 2024-05-08T08:53:37.207 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7710000000000001e-06 2024-05-08T08:53:37.207 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.207 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.207 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:37.207 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487478+0000", 2024-05-08T08:53:37.207 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1390000000000003e-06 2024-05-08T08:53:37.207 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.207 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.207 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:37.207 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487520+0000", 2024-05-08T08:53:37.207 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2259999999999999e-05 2024-05-08T08:53:37.207 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.207 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.207 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-05-08T08:53:37.207 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487528+0000", 2024-05-08T08:53:37.208 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7479999999999993e-06 2024-05-08T08:53:37.208 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.208 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.208 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:37.208 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:23.478139+0000", 2024-05-08T08:53:37.208 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.99061165200000001 2024-05-08T08:53:37.208 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.208 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.208 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:37.208 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:23.478301+0000", 2024-05-08T08:53:37.208 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000161602 2024-05-08T08:53:37.208 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.208 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.208 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:37.208 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:23.478483+0000", 2024-05-08T08:53:37.209 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000181781 2024-05-08T08:53:37.209 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.209 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:37.209 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.209 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.209 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:37.209 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:53:37.211 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:53:37.212 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:37.213+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:37.212 INFO:teuthology.orchestra.run.smithi196.stdout:3288 trunc from 0xed45600 to 0x5ab2a00 2024-05-08T08:53:37.212 INFO:teuthology.orchestra.run.smithi196.stdout:3289 write 0x4b4c00 thru 0x4bfb78 (0xaf79 bytes) 2024-05-08T08:53:37.213 INFO:teuthology.orchestra.run.smithi196.stdout:3291 read 0x58cb000 thru 0x58d9d4e (0xed4f bytes) 2024-05-08T08:53:37.213 INFO:teuthology.orchestra.run.smithi196.stdout:3292 write 0xca57000 thru 0xca62197 (0xb198 bytes) 2024-05-08T08:53:37.213 INFO:teuthology.orchestra.run.smithi196.stdout:3293 read 0x35c4200 thru 0x35d0ea6 (0xcca7 bytes) 2024-05-08T08:53:37.213 INFO:teuthology.orchestra.run.smithi196.stdout:3297 read 0x7962a00 thru 0x796f490 (0xca91 bytes) 2024-05-08T08:53:37.213 INFO:teuthology.orchestra.run.smithi196.stdout:3298 punch from 0x7079e00 to 0x7085c00, (0xbe00 bytes) 2024-05-08T08:53:37.213 INFO:teuthology.orchestra.run.smithi196.stdout:3301 read 0x778e200 thru 0x7790b76 (0x2977 bytes) 2024-05-08T08:53:37.213 INFO:teuthology.orchestra.run.smithi196.stdout:3302 read 0x27c00 thru 0x33f53 (0xc354 bytes) 2024-05-08T08:53:37.213 INFO:teuthology.orchestra.run.smithi196.stdout:3303 read 0x9c9ce00 thru 0x9ca4bf9 (0x7dfa bytes) 2024-05-08T08:53:37.213 INFO:teuthology.orchestra.run.smithi196.stdout:3304 trunc from 0xca62200 to 0x20f9400 2024-05-08T08:53:37.251 INFO:teuthology.orchestra.run.smithi196.stdout:3306 trunc from 0x20f9400 to 0xca49c00 2024-05-08T08:53:37.266 INFO:teuthology.orchestra.run.smithi196.stdout:3307 read 0x8c17e00 thru 0x8c233c7 (0xb5c8 bytes) 2024-05-08T08:53:37.277 INFO:teuthology.orchestra.run.smithi196.stdout:3309 write 0xeaf9000 thru 0xeafff90 (0x6f91 bytes) 2024-05-08T08:53:37.278 INFO:teuthology.orchestra.run.smithi196.stdout:3310 punch from 0xb5c1a00 to 0xb5d0e00, (0xf400 bytes) 2024-05-08T08:53:37.284 INFO:teuthology.orchestra.run.smithi196.stdout:3317 trunc from 0xeb00000 to 0xd71da00 2024-05-08T08:53:37.307 INFO:teuthology.orchestra.run.smithi196.stdout:3321 write 0xc2c8000 thru 0xc2cca0a (0x4a0b bytes) 2024-05-08T08:53:37.307 INFO:teuthology.orchestra.run.smithi196.stdout:3323 read 0x656000 thru 0x65b78b (0x578c bytes) 2024-05-08T08:53:37.307 INFO:teuthology.orchestra.run.smithi196.stdout:3324 punch from 0x8804e00 to 0x8814a00, (0xfc00 bytes) 2024-05-08T08:53:37.307 INFO:teuthology.orchestra.run.smithi196.stdout:3325 punch from 0xd5b600 to 0xd63e00, (0x8800 bytes) 2024-05-08T08:53:37.307 INFO:teuthology.orchestra.run.smithi196.stdout:3326 write 0x4b7ae00 thru 0x4b89464 (0xe665 bytes) 2024-05-08T08:53:37.309 INFO:teuthology.orchestra.run.smithi196.stdout:3328 trunc from 0xd71da00 to 0xe776800 2024-05-08T08:53:37.311 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:37.314+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:37.335 INFO:teuthology.orchestra.run.smithi196.stdout:3329 trunc from 0xe776800 to 0xb8db800 2024-05-08T08:53:37.348 INFO:teuthology.orchestra.run.smithi196.stdout:3331 punch from 0xe71000 to 0xe76c00, (0x5c00 bytes) 2024-05-08T08:53:37.350 INFO:teuthology.orchestra.run.smithi196.stdout:3333 read 0x7662400 thru 0x76709c2 (0xe5c3 bytes) 2024-05-08T08:53:37.362 INFO:teuthology.orchestra.run.smithi196.stdout:3336 trunc from 0xb8db800 to 0x1545400 2024-05-08T08:53:37.371 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:53:37.371 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:53:37.371 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:53:37.371 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:53:37.382 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:53:37.386 INFO:teuthology.orchestra.run.smithi196.stdout:3339 read 0x1de600 thru 0x1eba23 (0xd424 bytes) 2024-05-08T08:53:37.394 INFO:teuthology.orchestra.run.smithi196.stdout:3342 read 0xca6800 thru 0xcb43fb (0xdbfc bytes) 2024-05-08T08:53:37.411 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:37.414+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:37.416 INFO:teuthology.orchestra.run.smithi196.stdout:3343 write 0xb6f1a00 thru 0xb6f6fba (0x55bb bytes) 2024-05-08T08:53:37.418 INFO:teuthology.orchestra.run.smithi196.stdout:3346 read 0x7f54e00 thru 0x7f5a637 (0x5838 bytes) 2024-05-08T08:53:37.418 INFO:teuthology.orchestra.run.smithi196.stdout:3350 write 0xabc5a00 thru 0xabd1201 (0xb802 bytes) 2024-05-08T08:53:37.418 INFO:teuthology.orchestra.run.smithi196.stdout:3351 punch from 0x4e24a00 to 0x4e27200, (0x2800 bytes) 2024-05-08T08:53:37.422 INFO:teuthology.orchestra.run.smithi196.stdout:3352 read 0x101c00 thru 0x10dc30 (0xc031 bytes) 2024-05-08T08:53:37.422 INFO:teuthology.orchestra.run.smithi196.stdout:3353 punch from 0x228cc00 to 0x228da00, (0xe00 bytes) 2024-05-08T08:53:37.443 INFO:teuthology.orchestra.run.smithi196.stdout:3354 write 0xbfb1000 thru 0xbfbc2ae (0xb2af bytes) 2024-05-08T08:53:37.446 INFO:teuthology.orchestra.run.smithi196.stdout:3356 trunc from 0xbfbc400 to 0xe6be800 2024-05-08T08:53:37.465 INFO:teuthology.orchestra.run.smithi196.stdout:3357 trunc from 0xe6be800 to 0xd0ec400 2024-05-08T08:53:37.497 INFO:teuthology.orchestra.run.smithi196.stdout:3359 write 0xeddf800 thru 0xede0ef8 (0x16f9 bytes) 2024-05-08T08:53:37.497 INFO:teuthology.orchestra.run.smithi196.stdout:3360 read 0x3c93200 thru 0x3c93b61 (0x962 bytes) 2024-05-08T08:53:37.497 INFO:teuthology.orchestra.run.smithi196.stdout:3361 write 0x3b0fe00 thru 0x3b181f0 (0x83f1 bytes) 2024-05-08T08:53:37.497 INFO:teuthology.orchestra.run.smithi196.stdout:3363 write 0x17b4000 thru 0x17b584b (0x184c bytes) 2024-05-08T08:53:37.497 INFO:teuthology.orchestra.run.smithi196.stdout:3364 write 0x6fc4a00 thru 0x6fcd223 (0x8824 bytes) 2024-05-08T08:53:37.497 INFO:teuthology.orchestra.run.smithi196.stdout:3365 write 0x617b200 thru 0x618201a (0x6e1b bytes) 2024-05-08T08:53:37.498 INFO:teuthology.orchestra.run.smithi196.stdout:3366 punch from 0xe1be000 to 0xe1cdc00, (0xfc00 bytes) 2024-05-08T08:53:37.498 INFO:teuthology.orchestra.run.smithi196.stdout:3371 read 0xe4b0200 thru 0xe4bda6a (0xd86b bytes) 2024-05-08T08:53:37.498 INFO:teuthology.orchestra.run.smithi196.stdout:3372 write 0x956ea00 thru 0x957dedd (0xf4de bytes) 2024-05-08T08:53:37.498 INFO:teuthology.orchestra.run.smithi196.stdout:3373 read 0x20ddc00 thru 0x20e6ea7 (0x92a8 bytes) 2024-05-08T08:53:37.498 INFO:teuthology.orchestra.run.smithi196.stdout:3374 read 0x56b6c00 thru 0x56c49d0 (0xddd1 bytes) 2024-05-08T08:53:37.498 INFO:teuthology.orchestra.run.smithi196.stdout:3375 write 0x6423a00 thru 0x6429077 (0x5678 bytes) 2024-05-08T08:53:37.498 INFO:teuthology.orchestra.run.smithi196.stdout:3376 write 0x6d42800 thru 0x6d4d108 (0xa909 bytes) 2024-05-08T08:53:37.498 INFO:teuthology.orchestra.run.smithi196.stdout:3378 write 0xb774200 thru 0xb774323 (0x124 bytes) 2024-05-08T08:53:37.498 INFO:teuthology.orchestra.run.smithi196.stdout:3379 read 0xabbda00 thru 0xabc9b9e (0xc19f bytes) 2024-05-08T08:53:37.499 INFO:teuthology.orchestra.run.smithi196.stdout:3383 write 0x3d81000 thru 0x3d862f7 (0x52f8 bytes) 2024-05-08T08:53:37.499 INFO:teuthology.orchestra.run.smithi196.stdout:3386 write 0xdfdc000 thru 0xdfea7f0 (0xe7f1 bytes) 2024-05-08T08:53:37.499 INFO:teuthology.orchestra.run.smithi196.stdout:3388 write 0x424d200 thru 0x4255136 (0x7f37 bytes) 2024-05-08T08:53:37.499 INFO:teuthology.orchestra.run.smithi196.stdout:3390 write 0xc1d4a00 thru 0xc1e4432 (0xfa33 bytes) 2024-05-08T08:53:37.499 INFO:teuthology.orchestra.run.smithi196.stdout:3392 write 0x2b8800 thru 0x2c2453 (0x9c54 bytes) 2024-05-08T08:53:37.500 INFO:teuthology.orchestra.run.smithi196.stdout:3393 read 0xa82c400 thru 0xa82d8de (0x14df bytes) 2024-05-08T08:53:37.500 INFO:teuthology.orchestra.run.smithi196.stdout:3395 write 0xe85c800 thru 0xe864a15 (0x8216 bytes) 2024-05-08T08:53:37.500 INFO:teuthology.orchestra.run.smithi196.stdout:3399 write 0xad8ea00 thru 0xad9dc58 (0xf259 bytes) 2024-05-08T08:53:37.500 INFO:teuthology.orchestra.run.smithi196.stdout:3401 read 0x630600 thru 0x63d091 (0xca92 bytes) 2024-05-08T08:53:37.501 INFO:teuthology.orchestra.run.smithi196.stdout:3403 read 0x1f66800 thru 0x1f6ad4e (0x454f bytes) 2024-05-08T08:53:37.501 INFO:teuthology.orchestra.run.smithi196.stdout:3404 write 0xa48800 thru 0xa54791 (0xbf92 bytes) 2024-05-08T08:53:37.501 INFO:teuthology.orchestra.run.smithi196.stdout:3405 read 0x293ca00 thru 0x2944b7c (0x817d bytes) 2024-05-08T08:53:37.502 INFO:teuthology.orchestra.run.smithi196.stdout:3406 punch from 0x68ebc00 to 0x68f7c00, (0xc000 bytes) 2024-05-08T08:53:37.503 INFO:teuthology.orchestra.run.smithi196.stdout:3407 read 0x3f6d000 thru 0x3f7264c (0x564d bytes) 2024-05-08T08:53:37.503 INFO:teuthology.orchestra.run.smithi196.stdout:3408 read 0x4e62400 thru 0x4e64339 (0x1f3a bytes) 2024-05-08T08:53:37.504 INFO:teuthology.orchestra.run.smithi196.stdout:3411 punch from 0x3946600 to 0x3951400, (0xae00 bytes) 2024-05-08T08:53:37.504 INFO:teuthology.orchestra.run.smithi196.stdout:3415 read 0x7082800 thru 0x7083c71 (0x1472 bytes) 2024-05-08T08:53:37.512 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:53:37.515+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:53:37.520 INFO:teuthology.orchestra.run.smithi196.stdout:3416 trunc from 0xede1000 to 0xf60c00 2024-05-08T08:53:37.551 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:53:37.551 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:53:37.551 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:53:37.551 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:53:37.551 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:53:37.561 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:53:37.591 INFO:teuthology.orchestra.run.smithi196.stdout:3417 write 0x548d200 thru 0x548f85a (0x265b bytes) 2024-05-08T08:53:37.591 INFO:teuthology.orchestra.run.smithi196.stdout:3418 read 0x2f10800 thru 0x2f19277 (0x8a78 bytes) 2024-05-08T08:53:37.592 INFO:teuthology.orchestra.run.smithi196.stdout:3419 flatten 2024-05-08T08:53:37.613 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:53:37.616+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:53:37.671 INFO:teuthology.orchestra.run.smithi196.stdout:3421 write 0x919dc00 thru 0x91abda0 (0xe1a1 bytes) 2024-05-08T08:53:37.671 INFO:teuthology.orchestra.run.smithi196.stdout:3423 write 0x6601000 thru 0x660ea87 (0xda88 bytes) 2024-05-08T08:53:37.671 INFO:teuthology.orchestra.run.smithi196.stdout:3424 read 0x2a6b600 thru 0x2a7a70d (0xf10e bytes) 2024-05-08T08:53:37.677 INFO:teuthology.orchestra.run.smithi196.stdout:3425 trunc from 0x91abe00 to 0xd88b000 2024-05-08T08:53:37.686 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:53:37.686 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:53:37.686 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:53:37.686 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:53:37.686 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.686 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6690 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1093632~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:37.686 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:53:37.686 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.972932775999993, 2024-05-08T08:53:37.686 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052462754, 2024-05-08T08:53:37.686 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:37.687 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:37.687 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:37.687 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:37.687 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:37.687 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:37.687 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6690 2024-05-08T08:53:37.687 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.687 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:37.687 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.687 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:37.687 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:53:37.687 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:37.687 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.688 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.688 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:37.688 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:53:37.688 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:37.688 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.688 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.688 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:37.688 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715346+0000", 2024-05-08T08:53:37.688 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.84e-07 2024-05-08T08:53:37.688 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.688 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.688 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:37.688 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715348+0000", 2024-05-08T08:53:37.688 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.114e-06 2024-05-08T08:53:37.689 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.689 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.689 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:37.689 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715353+0000", 2024-05-08T08:53:37.689 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5720000000000002e-06 2024-05-08T08:53:37.689 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.689 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.689 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:37.689 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715358+0000", 2024-05-08T08:53:37.689 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7140000000000001e-06 2024-05-08T08:53:37.689 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.689 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.689 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:37.689 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767638+0000", 2024-05-08T08:53:37.689 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052279562000000002 2024-05-08T08:53:37.690 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.690 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.690 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:37.690 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767713+0000", 2024-05-08T08:53:37.690 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5652e-05 2024-05-08T08:53:37.690 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.690 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.690 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:37.690 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767808+0000", 2024-05-08T08:53:37.690 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4755999999999999e-05 2024-05-08T08:53:37.690 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.690 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:37.690 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.691 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.691 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.691 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6692 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1110016~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:37.691 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:53:37.691 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.972913685999998, 2024-05-08T08:53:37.691 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052666495000000001, 2024-05-08T08:53:37.691 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:37.691 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:37.691 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:37.691 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:37.691 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:37.691 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:37.691 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6692 2024-05-08T08:53:37.691 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.691 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:37.692 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.692 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:37.692 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:53:37.692 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:37.692 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.692 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.692 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:37.692 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:53:37.692 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:37.692 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.692 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.692 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:37.692 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:53:37.692 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8599999999999999e-07 2024-05-08T08:53:37.692 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.693 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.693 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:37.693 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715372+0000", 2024-05-08T08:53:37.693 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0990000000000001e-06 2024-05-08T08:53:37.693 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.693 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.693 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:37.693 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715378+0000", 2024-05-08T08:53:37.693 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0680000000000002e-06 2024-05-08T08:53:37.693 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.693 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.693 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:37.693 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715383+0000", 2024-05-08T08:53:37.693 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8740000000000003e-06 2024-05-08T08:53:37.693 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.693 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.694 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:37.694 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767848+0000", 2024-05-08T08:53:37.694 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052464668999999998 2024-05-08T08:53:37.694 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.694 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.694 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:37.694 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767927+0000", 2024-05-08T08:53:37.694 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9228999999999996e-05 2024-05-08T08:53:37.694 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.694 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.694 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:37.694 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768031+0000", 2024-05-08T08:53:37.694 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010417 2024-05-08T08:53:37.694 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.694 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:37.695 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.695 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.695 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.695 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6694 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1126400~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:37.695 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:53:37.695 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.972888788999995, 2024-05-08T08:53:37.695 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052848829, 2024-05-08T08:53:37.695 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:37.695 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:37.695 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:37.695 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:37.695 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:37.695 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:37.695 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6694 2024-05-08T08:53:37.695 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.695 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:37.696 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.696 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:37.696 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:53:37.696 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:37.696 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.696 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.696 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:37.696 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:53:37.696 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:37.696 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.696 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.696 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:37.696 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715390+0000", 2024-05-08T08:53:37.696 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9900000000000001e-07 2024-05-08T08:53:37.696 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.697 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.697 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:37.697 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715392+0000", 2024-05-08T08:53:37.697 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8279999999999999e-06 2024-05-08T08:53:37.697 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.697 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.697 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:37.697 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715398+0000", 2024-05-08T08:53:37.697 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7619999999999998e-06 2024-05-08T08:53:37.697 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.697 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.697 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:37.697 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715403+0000", 2024-05-08T08:53:37.697 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7790000000000003e-06 2024-05-08T08:53:37.697 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.697 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.697 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:37.698 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768067+0000", 2024-05-08T08:53:37.698 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052663785999999997 2024-05-08T08:53:37.698 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.698 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.698 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:37.698 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768142+0000", 2024-05-08T08:53:37.698 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5018999999999994e-05 2024-05-08T08:53:37.698 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.698 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.698 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:37.698 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768238+0000", 2024-05-08T08:53:37.698 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6255999999999995e-05 2024-05-08T08:53:37.698 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.698 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:37.699 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.699 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.699 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.699 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6696 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1142784~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:37.699 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:53:37.699 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.972869281000001, 2024-05-08T08:53:37.699 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.053051057999999998, 2024-05-08T08:53:37.699 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:37.699 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:37.699 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:37.699 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:37.699 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:37.699 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:37.699 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6696 2024-05-08T08:53:37.699 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.699 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:37.700 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.700 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:37.700 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:53:37.700 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:37.700 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.700 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.700 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:37.700 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:53:37.700 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:37.700 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.700 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.700 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:37.700 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:53:37.700 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-05-08T08:53:37.700 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.700 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.700 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:37.701 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715411+0000", 2024-05-08T08:53:37.701 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9630000000000001e-06 2024-05-08T08:53:37.701 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.701 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.701 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:37.701 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715417+0000", 2024-05-08T08:53:37.701 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7250000000000002e-06 2024-05-08T08:53:37.701 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.701 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.701 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:37.701 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715421+0000", 2024-05-08T08:53:37.701 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4089999999999996e-06 2024-05-08T08:53:37.701 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.701 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.701 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:37.702 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768277+0000", 2024-05-08T08:53:37.702 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052855293999999997 2024-05-08T08:53:37.702 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.702 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.702 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:37.702 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768358+0000", 2024-05-08T08:53:37.702 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1720999999999999e-05 2024-05-08T08:53:37.702 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.702 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.702 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:37.702 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768460+0000", 2024-05-08T08:53:37.702 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010154600000000001 2024-05-08T08:53:37.702 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.702 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:37.703 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.703 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.703 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.703 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6698 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1159168~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:37.703 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:53:37.703 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.972850414000007, 2024-05-08T08:53:37.703 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.053245726, 2024-05-08T08:53:37.703 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:37.703 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:37.703 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:37.703 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:37.703 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:37.703 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:37.703 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6698 2024-05-08T08:53:37.703 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.703 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:37.703 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.704 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:37.704 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:53:37.704 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:37.704 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.704 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.704 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:37.704 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:53:37.704 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:37.704 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.704 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.704 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:37.704 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716936+0000", 2024-05-08T08:53:37.704 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001508665 2024-05-08T08:53:37.704 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.704 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.705 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:37.705 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716941+0000", 2024-05-08T08:53:37.705 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1910000000000002e-06 2024-05-08T08:53:37.705 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.705 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.705 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:37.705 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716950+0000", 2024-05-08T08:53:37.705 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1900000000000001e-06 2024-05-08T08:53:37.705 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.705 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.705 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:37.705 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716958+0000", 2024-05-08T08:53:37.705 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8019999999999992e-06 2024-05-08T08:53:37.705 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.705 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.705 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:37.706 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768499+0000", 2024-05-08T08:53:37.706 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051540926000000001 2024-05-08T08:53:37.706 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.706 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.706 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:37.706 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768574+0000", 2024-05-08T08:53:37.706 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4988999999999996e-05 2024-05-08T08:53:37.706 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.706 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.706 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:37.706 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768673+0000", 2024-05-08T08:53:37.706 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9963000000000005e-05 2024-05-08T08:53:37.706 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.706 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:37.706 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.707 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.707 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.707 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6706 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1224704~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:37.707 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:53:37.707 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.971245107000001, 2024-05-08T08:53:37.707 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052435598, 2024-05-08T08:53:37.707 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:37.707 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:37.707 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:37.707 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:37.707 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:37.707 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:37.707 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6706 2024-05-08T08:53:37.707 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.707 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:37.708 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.708 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:37.708 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:53:37.708 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:37.708 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.708 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.708 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:37.708 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:53:37.708 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:37.708 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.708 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.708 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:37.708 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:53:37.708 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:53:37.708 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.708 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.708 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:37.709 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717035+0000", 2024-05-08T08:53:37.709 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9290000000000001e-06 2024-05-08T08:53:37.709 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.709 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.709 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:37.709 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717041+0000", 2024-05-08T08:53:37.709 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6720000000000001e-06 2024-05-08T08:53:37.709 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.709 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.709 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:37.709 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717046+0000", 2024-05-08T08:53:37.709 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5650000000000003e-06 2024-05-08T08:53:37.709 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.709 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.709 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:37.709 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769311+0000", 2024-05-08T08:53:37.709 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052265645999999999 2024-05-08T08:53:37.710 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.710 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.710 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:37.710 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769379+0000", 2024-05-08T08:53:37.710 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7391000000000006e-05 2024-05-08T08:53:37.710 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.710 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.710 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:37.710 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769469+0000", 2024-05-08T08:53:37.710 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0025999999999995e-05 2024-05-08T08:53:37.710 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.710 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:37.710 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.710 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.710 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.710 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6708 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1241088~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:37.711 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:53:37.711 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.971227002999996, 2024-05-08T08:53:37.711 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052640552, 2024-05-08T08:53:37.711 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:37.711 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:37.711 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:37.711 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:37.711 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:37.711 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:37.711 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6708 2024-05-08T08:53:37.711 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.711 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:37.711 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.711 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:37.711 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:53:37.711 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:37.711 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.712 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.712 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:37.712 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:53:37.712 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:37.712 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.712 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.712 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:37.712 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717052+0000", 2024-05-08T08:53:37.712 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:53:37.712 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.712 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.712 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:37.712 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717053+0000", 2024-05-08T08:53:37.712 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8330000000000001e-06 2024-05-08T08:53:37.712 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.713 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.713 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:37.713 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717058+0000", 2024-05-08T08:53:37.713 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.933e-06 2024-05-08T08:53:37.713 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.713 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.713 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:37.713 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717063+0000", 2024-05-08T08:53:37.713 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2180000000000001e-06 2024-05-08T08:53:37.713 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.713 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.713 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:37.713 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769507+0000", 2024-05-08T08:53:37.713 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052444866999999999 2024-05-08T08:53:37.713 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.713 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.713 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:37.714 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769589+0000", 2024-05-08T08:53:37.714 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2039000000000005e-05 2024-05-08T08:53:37.714 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.714 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.714 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:37.714 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769692+0000", 2024-05-08T08:53:37.714 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102252 2024-05-08T08:53:37.714 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.714 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:37.714 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.714 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.714 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.714 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6710 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1257472~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:37.714 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:53:37.714 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.971208978999996, 2024-05-08T08:53:37.715 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.05283057, 2024-05-08T08:53:37.715 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:37.715 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:37.715 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:37.715 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:37.715 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:37.715 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:37.715 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6710 2024-05-08T08:53:37.715 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.715 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:37.715 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.715 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:37.715 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:53:37.715 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:37.716 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.716 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.716 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:37.716 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:53:37.716 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:37.716 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.716 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.716 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:37.716 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718951+0000", 2024-05-08T08:53:37.716 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0018817809999999999 2024-05-08T08:53:37.716 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.716 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.716 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:37.716 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718956+0000", 2024-05-08T08:53:37.716 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5399999999999997e-06 2024-05-08T08:53:37.716 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.717 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.717 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:37.717 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718964+0000", 2024-05-08T08:53:37.717 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6629999999999999e-06 2024-05-08T08:53:37.717 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.717 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.717 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:37.717 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718972+0000", 2024-05-08T08:53:37.717 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5229999999999998e-06 2024-05-08T08:53:37.717 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.717 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.717 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:37.717 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769729+0000", 2024-05-08T08:53:37.717 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050756959999999997 2024-05-08T08:53:37.717 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.718 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.718 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:37.718 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769803+0000", 2024-05-08T08:53:37.718 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4538999999999999e-05 2024-05-08T08:53:37.718 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.718 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.718 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:37.718 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769900+0000", 2024-05-08T08:53:37.718 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6564000000000006e-05 2024-05-08T08:53:37.718 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.718 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:37.718 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.718 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.718 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.718 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6726 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1388544~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:37.719 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:53:37.719 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.969134206999996, 2024-05-08T08:53:37.719 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052451506000000002, 2024-05-08T08:53:37.719 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:37.719 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:37.719 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:37.719 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:37.719 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:37.719 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:37.719 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6726 2024-05-08T08:53:37.719 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.719 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:37.719 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.719 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:37.719 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:53:37.719 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:37.720 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.720 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.720 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:37.720 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:53:37.720 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:37.720 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.720 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.720 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:37.720 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719200+0000", 2024-05-08T08:53:37.720 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5865999999999997e-05 2024-05-08T08:53:37.720 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.720 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.720 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:37.720 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719204+0000", 2024-05-08T08:53:37.720 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8630000000000001e-06 2024-05-08T08:53:37.720 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.721 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.721 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:37.721 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719215+0000", 2024-05-08T08:53:37.721 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1590000000000001e-05 2024-05-08T08:53:37.721 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.721 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.721 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:37.721 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719220+0000", 2024-05-08T08:53:37.721 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1619999999999999e-06 2024-05-08T08:53:37.721 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.721 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.721 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:37.721 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771425+0000", 2024-05-08T08:53:37.721 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052204249000000001 2024-05-08T08:53:37.722 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.722 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.722 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:37.722 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771498+0000", 2024-05-08T08:53:37.722 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3782999999999995e-05 2024-05-08T08:53:37.722 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.722 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.722 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:37.722 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771595+0000", 2024-05-08T08:53:37.722 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6992999999999998e-05 2024-05-08T08:53:37.722 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.722 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:37.722 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.722 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.722 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.722 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6728 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1404928~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:37.723 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:53:37.723 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.969046801999994, 2024-05-08T08:53:37.723 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052590747, 2024-05-08T08:53:37.723 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:37.723 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:37.723 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:37.723 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:37.723 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:37.723 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:37.723 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6728 2024-05-08T08:53:37.723 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.723 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:37.723 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.723 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:37.723 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:53:37.723 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:37.724 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.724 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.724 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:37.724 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:53:37.724 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:37.724 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.724 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.724 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:37.724 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719232+0000", 2024-05-08T08:53:37.724 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4000000000000001e-07 2024-05-08T08:53:37.724 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.724 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.724 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:37.724 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719234+0000", 2024-05-08T08:53:37.724 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4499999999999998e-06 2024-05-08T08:53:37.724 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.725 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.725 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:37.725 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719240+0000", 2024-05-08T08:53:37.725 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5720000000000002e-06 2024-05-08T08:53:37.725 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.725 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.725 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:37.725 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719245+0000", 2024-05-08T08:53:37.725 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8690000000000003e-06 2024-05-08T08:53:37.725 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.725 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.725 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:37.725 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771636+0000", 2024-05-08T08:53:37.725 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052390979999999997 2024-05-08T08:53:37.725 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.726 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.726 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:37.726 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771717+0000", 2024-05-08T08:53:37.726 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1291999999999993e-05 2024-05-08T08:53:37.726 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.726 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.726 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:37.726 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771822+0000", 2024-05-08T08:53:37.726 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104944 2024-05-08T08:53:37.726 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.726 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:37.726 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.726 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.726 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.726 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6730 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1421312~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:37.727 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:53:37.727 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.969026560000003, 2024-05-08T08:53:37.727 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052770477000000003, 2024-05-08T08:53:37.727 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:37.727 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:37.727 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:37.727 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:37.727 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:37.727 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:37.727 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6730 2024-05-08T08:53:37.727 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.727 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:37.727 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.727 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:37.727 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:53:37.728 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:37.728 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.728 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.728 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:37.728 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:53:37.728 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:37.728 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.728 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.728 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:37.728 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:53:37.728 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9000000000000002e-07 2024-05-08T08:53:37.728 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.728 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.728 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:37.728 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719254+0000", 2024-05-08T08:53:37.729 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4099999999999998e-06 2024-05-08T08:53:37.729 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.729 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.729 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:37.729 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719263+0000", 2024-05-08T08:53:37.729 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4109999999999996e-06 2024-05-08T08:53:37.729 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.729 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.729 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:37.729 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719268+0000", 2024-05-08T08:53:37.729 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2669999999999997e-06 2024-05-08T08:53:37.729 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.729 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.729 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:37.729 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771846+0000", 2024-05-08T08:53:37.730 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052577677000000003 2024-05-08T08:53:37.730 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.730 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.730 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:37.730 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771920+0000", 2024-05-08T08:53:37.730 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4460000000000002e-05 2024-05-08T08:53:37.730 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.730 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.730 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:37.730 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772022+0000", 2024-05-08T08:53:37.730 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010186199999999999 2024-05-08T08:53:37.730 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.730 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:37.730 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.730 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.731 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.731 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6732 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1437696~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:37.731 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:53:37.731 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.969001753000001, 2024-05-08T08:53:37.731 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052905575000000003, 2024-05-08T08:53:37.731 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:37.731 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:37.731 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:37.731 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:37.731 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:37.731 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:37.731 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6732 2024-05-08T08:53:37.731 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.731 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:37.731 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.731 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:37.732 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:53:37.732 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:37.732 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.732 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.732 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:37.732 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:53:37.732 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:37.732 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.732 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.732 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:37.732 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721392+0000", 2024-05-08T08:53:37.732 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002115287 2024-05-08T08:53:37.732 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.732 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.732 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:37.732 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721396+0000", 2024-05-08T08:53:37.733 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1679999999999997e-06 2024-05-08T08:53:37.733 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.733 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.733 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:37.733 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721404+0000", 2024-05-08T08:53:37.733 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4409999999999993e-06 2024-05-08T08:53:37.733 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.733 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.733 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:37.733 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721410+0000", 2024-05-08T08:53:37.733 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9150000000000001e-06 2024-05-08T08:53:37.733 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.733 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.733 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:37.733 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772061+0000", 2024-05-08T08:53:37.733 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050651068 2024-05-08T08:53:37.734 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.734 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.734 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:37.734 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772127+0000", 2024-05-08T08:53:37.734 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5424000000000002e-05 2024-05-08T08:53:37.734 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.734 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.734 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:37.734 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772182+0000", 2024-05-08T08:53:37.734 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5272000000000002e-05 2024-05-08T08:53:37.734 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.734 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:37.734 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.734 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.734 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.735 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6749 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1585152~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:37.735 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:53:37.735 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.966700982999996, 2024-05-08T08:53:37.735 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052350264000000001, 2024-05-08T08:53:37.735 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:37.735 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:37.735 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:37.735 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:37.735 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:37.735 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:37.735 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6749 2024-05-08T08:53:37.735 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.735 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:37.735 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.735 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:37.736 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:53:37.736 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:37.736 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.736 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.736 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:37.736 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:53:37.736 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:37.736 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.736 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.736 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:37.736 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721578+0000", 2024-05-08T08:53:37.736 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0399999999999996e-07 2024-05-08T08:53:37.736 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.736 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.736 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:37.737 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721580+0000", 2024-05-08T08:53:37.737 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5450000000000002e-06 2024-05-08T08:53:37.737 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.737 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.737 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:37.737 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721587+0000", 2024-05-08T08:53:37.737 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0190000000000001e-06 2024-05-08T08:53:37.737 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.737 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.737 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:37.737 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721593+0000", 2024-05-08T08:53:37.737 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5659999999999997e-06 2024-05-08T08:53:37.737 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.737 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.738 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:37.738 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773759+0000", 2024-05-08T08:53:37.738 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052166553999999997 2024-05-08T08:53:37.738 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.738 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.738 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:37.738 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773833+0000", 2024-05-08T08:53:37.738 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3913999999999997e-05 2024-05-08T08:53:37.738 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.738 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.738 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:37.738 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773927+0000", 2024-05-08T08:53:37.738 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4061999999999995e-05 2024-05-08T08:53:37.738 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.738 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:37.739 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.739 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.739 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.739 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6751 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1601536~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:37.739 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:53:37.739 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.966677704000006, 2024-05-08T08:53:37.739 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052461302000000001, 2024-05-08T08:53:37.739 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:37.739 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:37.739 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:37.739 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:37.739 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:37.739 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:37.739 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6751 2024-05-08T08:53:37.739 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.740 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:37.740 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.740 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:37.740 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:53:37.740 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:37.740 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.740 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.740 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:37.740 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:53:37.740 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:37.740 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.740 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.740 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:37.740 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722939+0000", 2024-05-08T08:53:37.741 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001338601 2024-05-08T08:53:37.741 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.741 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.741 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:37.741 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722944+0000", 2024-05-08T08:53:37.741 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4469999999999999e-06 2024-05-08T08:53:37.741 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.741 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.741 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:37.741 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722955+0000", 2024-05-08T08:53:37.741 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1892999999999999e-05 2024-05-08T08:53:37.741 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.741 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.741 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:37.741 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722963+0000", 2024-05-08T08:53:37.742 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1080000000000004e-06 2024-05-08T08:53:37.742 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.742 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.742 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:37.742 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773964+0000", 2024-05-08T08:53:37.742 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051001022 2024-05-08T08:53:37.742 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.742 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.742 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:37.742 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.774006+0000", 2024-05-08T08:53:37.742 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2729999999999999e-05 2024-05-08T08:53:37.742 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.742 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.742 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:37.742 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.774062+0000", 2024-05-08T08:53:37.742 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5501000000000003e-05 2024-05-08T08:53:37.743 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.743 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:37.743 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.743 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.743 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.743 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6760 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1683456~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:37.743 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:53:37.743 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.965168328000004, 2024-05-08T08:53:37.743 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052571312000000002, 2024-05-08T08:53:37.743 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:37.743 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:37.743 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:37.743 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:37.743 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:37.743 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:37.744 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6760 2024-05-08T08:53:37.744 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.744 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:37.744 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.744 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:37.744 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:53:37.744 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:37.744 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.744 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.744 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:37.744 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:53:37.744 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:37.744 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.744 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.744 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:37.744 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:53:37.745 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2100000000000002e-07 2024-05-08T08:53:37.745 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.745 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.745 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:37.745 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723113+0000", 2024-05-08T08:53:37.745 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3149999999999999e-06 2024-05-08T08:53:37.745 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.745 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.745 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:37.745 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723118+0000", 2024-05-08T08:53:37.745 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8860000000000001e-06 2024-05-08T08:53:37.745 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.745 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.745 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:37.745 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723123+0000", 2024-05-08T08:53:37.745 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6770000000000003e-06 2024-05-08T08:53:37.746 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.746 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.746 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:37.746 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775414+0000", 2024-05-08T08:53:37.746 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052290692999999999 2024-05-08T08:53:37.746 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.746 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.746 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:37.746 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775459+0000", 2024-05-08T08:53:37.746 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5639999999999997e-05 2024-05-08T08:53:37.746 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.746 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.746 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:37.746 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775681+0000", 2024-05-08T08:53:37.746 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00022168000000000001 2024-05-08T08:53:37.746 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.747 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:37.747 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.747 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.747 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.747 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6761 3.16 3:6bd6d6da:::rbd_data.1140782590a5.0000000000000015:6 [read 1095680~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:37.747 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:53:37.747 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.965149108000006, 2024-05-08T08:53:37.747 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052751275, 2024-05-08T08:53:37.747 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:37.747 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:37.747 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:37.747 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:37.747 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:37.747 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:37.747 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6761 2024-05-08T08:53:37.748 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.748 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:37.748 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.748 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:37.748 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:53:37.748 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:37.748 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.748 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.748 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:37.748 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:53:37.748 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:37.748 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.748 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.748 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:37.748 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724939+0000", 2024-05-08T08:53:37.749 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001809817 2024-05-08T08:53:37.749 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.749 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.749 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:37.749 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724943+0000", 2024-05-08T08:53:37.749 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4920000000000004e-06 2024-05-08T08:53:37.749 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.749 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.749 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:37.749 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724956+0000", 2024-05-08T08:53:37.749 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2427e-05 2024-05-08T08:53:37.749 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.749 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.749 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:37.749 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724963+0000", 2024-05-08T08:53:37.750 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2559999999999996e-06 2024-05-08T08:53:37.750 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.750 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.750 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:37.750 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775711+0000", 2024-05-08T08:53:37.750 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050747621 2024-05-08T08:53:37.750 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.750 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.750 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:37.750 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775785+0000", 2024-05-08T08:53:37.750 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4634000000000004e-05 2024-05-08T08:53:37.750 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.750 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.750 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:37.750 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775880+0000", 2024-05-08T08:53:37.751 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5027999999999999e-05 2024-05-08T08:53:37.751 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.751 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:37.751 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.751 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.751 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.751 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6770 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1765376~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:37.751 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:53:37.751 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.963118012999999, 2024-05-08T08:53:37.751 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052339296, 2024-05-08T08:53:37.751 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:37.751 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:37.751 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:37.751 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:37.751 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:37.752 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:37.752 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6770 2024-05-08T08:53:37.752 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.752 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:37.752 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.752 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:37.752 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:53:37.752 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:37.752 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.752 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.752 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:37.752 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:53:37.752 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:37.752 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.752 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.752 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:37.753 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725161+0000", 2024-05-08T08:53:37.753 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-05-08T08:53:37.753 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.753 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.753 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:37.753 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725163+0000", 2024-05-08T08:53:37.753 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4370000000000001e-06 2024-05-08T08:53:37.753 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.753 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.753 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:37.753 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725169+0000", 2024-05-08T08:53:37.753 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8409999999999996e-06 2024-05-08T08:53:37.753 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.753 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.753 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:37.754 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725174+0000", 2024-05-08T08:53:37.754 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7249999999999997e-06 2024-05-08T08:53:37.754 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.754 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.754 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:37.754 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777343+0000", 2024-05-08T08:53:37.754 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052169633999999999 2024-05-08T08:53:37.754 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.754 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.754 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:37.754 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777409+0000", 2024-05-08T08:53:37.754 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6033999999999998e-05 2024-05-08T08:53:37.754 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.754 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.754 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:37.755 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777499+0000", 2024-05-08T08:53:37.755 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0224999999999998e-05 2024-05-08T08:53:37.755 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.755 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:37.755 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.755 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.755 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.755 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6771 3.16 3:6bd6d6da:::rbd_data.1140782590a5.0000000000000015:6 [read 1177600~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:37.755 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:53:37.755 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.963098196000004, 2024-05-08T08:53:37.755 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052438604999999999, 2024-05-08T08:53:37.755 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:37.755 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:37.755 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:37.755 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:37.756 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:37.756 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:37.756 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6771 2024-05-08T08:53:37.756 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.756 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:37.756 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.756 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:37.756 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:53:37.756 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:37.756 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.756 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.756 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:37.756 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:53:37.756 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:37.756 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.756 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.757 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:37.757 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:53:37.757 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:53:37.757 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.757 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.757 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:37.757 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725188+0000", 2024-05-08T08:53:37.757 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1579999999999999e-06 2024-05-08T08:53:37.757 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.757 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.757 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:37.757 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725195+0000", 2024-05-08T08:53:37.757 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8909999999999995e-06 2024-05-08T08:53:37.757 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.757 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.757 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:37.758 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725200+0000", 2024-05-08T08:53:37.758 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9810000000000003e-06 2024-05-08T08:53:37.758 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.758 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.758 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:37.758 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777527+0000", 2024-05-08T08:53:37.758 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052326582000000003 2024-05-08T08:53:37.758 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.758 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.758 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:37.758 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777566+0000", 2024-05-08T08:53:37.758 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9447999999999999e-05 2024-05-08T08:53:37.758 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.758 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.758 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:37.759 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777619+0000", 2024-05-08T08:53:37.759 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2135000000000001e-05 2024-05-08T08:53:37.759 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.759 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:37.759 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.759 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.759 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.759 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6772 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1781760~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:37.759 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:53:37.759 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.963070138999996, 2024-05-08T08:53:37.759 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052601799999999997, 2024-05-08T08:53:37.759 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:37.759 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:37.759 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:37.759 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:37.760 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:37.760 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:37.760 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6772 2024-05-08T08:53:37.760 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.760 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:37.760 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.760 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:37.760 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:53:37.760 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:37.760 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.760 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.760 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:37.760 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:53:37.760 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:37.760 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.760 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.761 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:37.761 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726939+0000", 2024-05-08T08:53:37.761 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001730493 2024-05-08T08:53:37.761 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.761 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.761 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:37.761 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726943+0000", 2024-05-08T08:53:37.761 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7330000000000003e-06 2024-05-08T08:53:37.761 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.761 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.761 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:37.761 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726954+0000", 2024-05-08T08:53:37.761 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1107000000000001e-05 2024-05-08T08:53:37.761 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.761 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.762 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:37.762 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726962+0000", 2024-05-08T08:53:37.762 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.503e-06 2024-05-08T08:53:37.762 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.762 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.762 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:37.762 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777653+0000", 2024-05-08T08:53:37.762 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050691542999999999 2024-05-08T08:53:37.762 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.762 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.762 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:37.762 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777719+0000", 2024-05-08T08:53:37.762 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5228000000000006e-05 2024-05-08T08:53:37.762 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.762 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.763 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:37.763 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777810+0000", 2024-05-08T08:53:37.763 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1193000000000006e-05 2024-05-08T08:53:37.763 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.763 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:37.763 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.763 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.763 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.763 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6782 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1880064~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:37.763 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:53:37.763 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.961101631999995, 2024-05-08T08:53:37.763 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052326492000000002, 2024-05-08T08:53:37.763 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:37.763 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:37.763 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:37.763 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:37.764 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:37.764 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:37.764 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6782 2024-05-08T08:53:37.764 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.764 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:37.764 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.764 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:37.764 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:53:37.764 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:37.764 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.764 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.764 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:37.764 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:53:37.764 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:37.764 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.765 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.765 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:37.765 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728939+0000", 2024-05-08T08:53:37.765 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001762719 2024-05-08T08:53:37.765 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.765 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.765 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:37.765 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728944+0000", 2024-05-08T08:53:37.765 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.746e-06 2024-05-08T08:53:37.765 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.765 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.765 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:37.765 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728955+0000", 2024-05-08T08:53:37.765 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1043000000000001e-05 2024-05-08T08:53:37.765 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.766 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.766 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:37.766 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728963+0000", 2024-05-08T08:53:37.766 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5809999999999999e-06 2024-05-08T08:53:37.766 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.766 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.766 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:37.766 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779320+0000", 2024-05-08T08:53:37.766 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050357290999999998 2024-05-08T08:53:37.766 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.766 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.766 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:37.766 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779401+0000", 2024-05-08T08:53:37.766 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0574000000000005e-05 2024-05-08T08:53:37.767 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:37.767 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:37.767 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:37.767 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779503+0000", 2024-05-08T08:53:37.767 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102538 2024-05-08T08:53:37.767 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.767 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:37.767 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.767 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:37.767 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:37.767 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:53:37.769 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:37.771+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:37.770 INFO:teuthology.orchestra.run.smithi196.stdout:3426 write 0xa09ae00 thru 0xa09e67c (0x387d bytes) 2024-05-08T08:53:37.770 INFO:teuthology.orchestra.run.smithi196.stdout:3428 trunc from 0xd88b000 to 0xcbffe00 2024-05-08T08:53:37.770 INFO:teuthology.orchestra.run.smithi196.stdout:3430 write 0x7835200 thru 0x783ac73 (0x5a74 bytes) 2024-05-08T08:53:37.770 INFO:teuthology.orchestra.run.smithi196.stdout:3432 read 0x4f57200 thru 0x4f629a1 (0xb7a2 bytes) 2024-05-08T08:53:37.770 INFO:teuthology.orchestra.run.smithi196.stdout:3433 read 0xb2f9e00 thru 0xb2ff71a (0x591b bytes) 2024-05-08T08:53:37.770 INFO:teuthology.orchestra.run.smithi196.stdout:3436 trunc from 0xcbffe00 to 0xc1db600 2024-05-08T08:53:37.770 INFO:teuthology.orchestra.run.smithi196.stdout:3439 trunc from 0xc1db600 to 0x383d200 2024-05-08T08:53:37.770 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:53:37.772 INFO:teuthology.orchestra.run.smithi196.stdout:3448 read 0x400800 thru 0x40ab4c (0xa34d bytes) 2024-05-08T08:53:37.772 INFO:teuthology.orchestra.run.smithi196.stdout:3451 read 0x21a8a00 thru 0x21b07b7 (0x7db8 bytes) 2024-05-08T08:53:37.772 INFO:teuthology.orchestra.run.smithi196.stdout:3452 read 0x3505400 thru 0x3507dd9 (0x29da bytes) 2024-05-08T08:53:37.785 INFO:teuthology.orchestra.run.smithi196.stdout:3453 write 0x51e1c00 thru 0x51eb2aa (0x96ab bytes) 2024-05-08T08:53:37.796 INFO:teuthology.orchestra.run.smithi196.stdout:3454 trunc from 0x51eb400 to 0xa889a00 2024-05-08T08:53:37.809 INFO:teuthology.orchestra.run.smithi196.stdout:3455 write 0x1c84c00 thru 0x1c94026 (0xf427 bytes) 2024-05-08T08:53:37.814 INFO:teuthology.orchestra.run.smithi196.stdout:3456 trunc from 0xa889a00 to 0xd139000 2024-05-08T08:53:37.830 INFO:teuthology.orchestra.run.smithi196.stdout:3458 write 0x9799200 thru 0x97a651c (0xd31d bytes) 2024-05-08T08:53:37.830 INFO:teuthology.orchestra.run.smithi196.stdout:3459 write 0x1f09a00 thru 0x1f10e77 (0x7478 bytes) 2024-05-08T08:53:37.830 INFO:teuthology.orchestra.run.smithi196.stdout:3461 write 0xbe9600 thru 0xbf2809 (0x920a bytes) 2024-05-08T08:53:37.834 INFO:teuthology.orchestra.run.smithi196.stdout:3463 trunc from 0xd139000 to 0x9816000 2024-05-08T08:53:37.858 INFO:teuthology.orchestra.run.smithi196.stdout:3464 punch from 0x211a800 to 0x2125e00, (0xb600 bytes) 2024-05-08T08:53:37.861 INFO:teuthology.orchestra.run.smithi196.stdout:3465 read 0x21b0600 thru 0x21be4f3 (0xdef4 bytes) 2024-05-08T08:53:37.861 INFO:teuthology.orchestra.run.smithi196.stdout:3466 punch from 0x87bd600 to 0x87c0800, (0x3200 bytes) 2024-05-08T08:53:37.861 INFO:teuthology.orchestra.run.smithi196.stdout:3467 read 0x6a34200 thru 0x6a4060c (0xc40d bytes) 2024-05-08T08:53:37.861 INFO:teuthology.orchestra.run.smithi196.stdout:3469 clone 25 order 20 su 262144 sc 10 2024-05-08T08:53:37.868 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:37.872+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:37.890 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:37.890 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:53:37.890 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:53:37.890 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:37.899 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:53:37.936 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T08:53:37.987 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:53:37.990+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:53:38.009 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:38.009 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:53:38.009 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:53:38.009 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:53:38.009 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:38.019 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:53:38.069 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:53:38.069 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:53:38.070 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:38.073+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:38.088 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:53:38.088 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:53:38.107 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:53:38.107 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:53:38.125 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:53:38.125 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:53:38.143 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:53:38.143 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:53:38.161 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:53:38.161 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:53:38.174 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:38.174+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:38.175 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:38.175 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:38.182 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:53:38.272 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:38.276+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:38.332 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:38.332 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:38.341 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:53:38.373 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:38.377+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:38.450 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:38.451 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:38.459 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:53:38.503 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:38.506+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:38.546 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-05-08T08:53:38.576 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:38.576 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:38.577 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:53:38.579+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:53:38.587 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:53:38.676 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:38.679+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:38.740 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:38.741 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:38.750 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:53:38.778 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:38.780+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:38.879 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:38.881+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:38.904 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:38.904 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:38.914 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:53:38.931 INFO:teuthology.orchestra.run.smithi196.stdout:checking clone #23, image image_client.0-clone23 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent24 2024-05-08T08:53:38.998 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:39.001+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:39.077 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:39.077 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:39.081 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:39.083+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:39.087 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:53:39.181 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:39.184+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:39.202 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:39.202 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:39.212 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:53:39.297 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:39.300+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:39.361 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:39.362 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:39.369 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:53:39.383 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:39.386+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:39.484 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:39.487+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:39.515 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:39.515 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:39.524 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:53:39.541 INFO:teuthology.orchestra.run.smithi059.stderr:nodeep-scrub is set 2024-05-08T08:53:39.587 INFO:teuthology.orchestra.run.smithi196.stdout:3473 read 0x3013c00 thru 0x301c552 (0x8953 bytes) 2024-05-08T08:53:39.588 INFO:teuthology.orchestra.run.smithi196.stdout:3474 write 0x10b6600 thru 0x10b7859 (0x125a bytes) 2024-05-08T08:53:39.591 INFO:teuthology.orchestra.run.smithi196.stdout:3475 write 0x27a0a00 thru 0x27ac60b (0xbc0c bytes) 2024-05-08T08:53:39.594 INFO:teuthology.orchestra.run.smithi196.stdout:3476 write 0x1379600 thru 0x1380d85 (0x7786 bytes) 2024-05-08T08:53:39.594 INFO:teuthology.orchestra.run.smithi196.stdout:3477 write 0x187a400 thru 0x18876c0 (0xd2c1 bytes) 2024-05-08T08:53:39.595 INFO:teuthology.orchestra.run.smithi196.stdout:3478 write 0x7c8dc00 thru 0x7c9a4af (0xc8b0 bytes) 2024-05-08T08:53:39.596 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:53:39.599+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:53:39.596 INFO:teuthology.orchestra.run.smithi196.stdout:3479 write 0x9013000 thru 0x90192f1 (0x62f2 bytes) 2024-05-08T08:53:39.596 INFO:teuthology.orchestra.run.smithi196.stdout:3481 write 0x47a3c00 thru 0x47b0d7f (0xd180 bytes) 2024-05-08T08:53:39.626 INFO:teuthology.orchestra.run.smithi196.stdout:3482 write 0xb4a2400 thru 0xb4a7afb (0x56fc bytes) 2024-05-08T08:53:39.627 INFO:teuthology.orchestra.run.smithi196.stdout:3483 punch from 0x74fb000 to 0x7503000, (0x8000 bytes) 2024-05-08T08:53:39.628 INFO:teuthology.orchestra.run.smithi196.stdout:3484 read 0x22f5a00 thru 0x2302994 (0xcf95 bytes) 2024-05-08T08:53:39.630 INFO:teuthology.orchestra.run.smithi196.stdout:3486 trunc from 0xb4a7c00 to 0xcddfe00 2024-05-08T08:53:39.643 INFO:teuthology.orchestra.run.smithi196.stdout:3489 read 0xb4b7000 thru 0xb4c2fc1 (0xbfc2 bytes) 2024-05-08T08:53:39.664 INFO:teuthology.orchestra.run.smithi196.stdout:3491 write 0xd13ca00 thru 0xd14955f (0xcb60 bytes) 2024-05-08T08:53:39.664 INFO:teuthology.orchestra.run.smithi196.stdout:3493 read 0x517a000 thru 0x517df02 (0x3f03 bytes) 2024-05-08T08:53:39.677 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:39.677 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:39.680 INFO:teuthology.orchestra.run.smithi196.stdout:3495 trunc from 0xd149600 to 0x5c8200 2024-05-08T08:53:39.687 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:53:39.687 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:39.689+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:39.727 INFO:teuthology.orchestra.run.smithi196.stdout:3497 read 0x78e00 thru 0x81e5b (0x905c bytes) 2024-05-08T08:53:39.734 INFO:teuthology.orchestra.run.smithi196.stdout:3500 read 0x151600 thru 0x15d000 (0xba01 bytes) 2024-05-08T08:53:39.734 INFO:teuthology.orchestra.run.smithi196.stdout:3501 read 0x190400 thru 0x1958ca (0x54cb bytes) 2024-05-08T08:53:39.735 INFO:teuthology.orchestra.run.smithi196.stdout:3502 read 0x169200 thru 0x1783e2 (0xf1e3 bytes) 2024-05-08T08:53:39.749 INFO:teuthology.orchestra.run.smithi196.stdout:3503 write 0x3692a00 thru 0x36a06be (0xdcbf bytes) 2024-05-08T08:53:39.756 INFO:teuthology.orchestra.run.smithi196.stdout:3504 trunc from 0x36a0800 to 0x4af7c00 2024-05-08T08:53:39.770 INFO:teuthology.orchestra.run.smithi196.stdout:3505 punch from 0x471b400 to 0x4721000, (0x5c00 bytes) 2024-05-08T08:53:39.829 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:39.831+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:39.829 INFO:teuthology.orchestra.run.smithi196.stdout:3506 write 0x793f000 thru 0x7946274 (0x7275 bytes) 2024-05-08T08:53:39.829 INFO:teuthology.orchestra.run.smithi196.stdout:3507 write 0xd58a800 thru 0xd599c08 (0xf409 bytes) 2024-05-08T08:53:39.829 INFO:teuthology.orchestra.run.smithi196.stdout:3510 write 0xe43a00 thru 0xe4ba2a (0x802b bytes) 2024-05-08T08:53:39.829 INFO:teuthology.orchestra.run.smithi196.stdout:3511 write 0xf61600 thru 0xf6f7cd (0xe1ce bytes) 2024-05-08T08:53:39.830 INFO:teuthology.orchestra.run.smithi196.stdout:3513 write 0x1d11a00 thru 0x1d1e951 (0xcf52 bytes) 2024-05-08T08:53:39.830 INFO:teuthology.orchestra.run.smithi196.stdout:3516 write 0x5c7f400 thru 0x5c8a020 (0xac21 bytes) 2024-05-08T08:53:39.830 INFO:teuthology.orchestra.run.smithi196.stdout:3517 trunc from 0xd599e00 to 0x474aa00 2024-05-08T08:53:39.838 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:39.838 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:39.848 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:53:39.853 INFO:teuthology.orchestra.run.smithi196.stdout:3518 punch from 0x2844200 to 0x284e400, (0xa200 bytes) 2024-05-08T08:53:39.855 INFO:teuthology.orchestra.run.smithi196.stdout:3519 read 0x3103a00 thru 0x310caa9 (0x90aa bytes) 2024-05-08T08:53:39.855 INFO:teuthology.orchestra.run.smithi196.stdout:3521 read 0x2826800 thru 0x282a911 (0x4112 bytes) 2024-05-08T08:53:39.855 INFO:teuthology.orchestra.run.smithi196.stdout:3523 read 0x22a600 thru 0x230c64 (0x6665 bytes) 2024-05-08T08:53:39.855 INFO:teuthology.orchestra.run.smithi196.stdout:3524 read 0x380e600 thru 0x38178e9 (0x92ea bytes) 2024-05-08T08:53:39.855 INFO:teuthology.orchestra.run.smithi196.stdout:3525 read 0x48b600 thru 0x490d8f (0x5790 bytes) 2024-05-08T08:53:39.859 INFO:teuthology.orchestra.run.smithi196.stdout:3526 punch from 0x1d2c800 to 0x1d37800, (0xb000 bytes) 2024-05-08T08:53:39.859 INFO:teuthology.orchestra.run.smithi196.stdout:3527 read 0x3853a00 thru 0x3858d8d (0x538e bytes) 2024-05-08T08:53:39.859 INFO:teuthology.orchestra.run.smithi196.stdout:3528 punch from 0x2f96e00 to 0x2f9a800, (0x3a00 bytes) 2024-05-08T08:53:39.859 INFO:teuthology.orchestra.run.smithi196.stdout:3530 write 0x37e5e00 thru 0x37eda29 (0x7c2a bytes) 2024-05-08T08:53:39.859 INFO:teuthology.orchestra.run.smithi196.stdout:3531 read 0x1213a00 thru 0x122177a (0xdd7b bytes) 2024-05-08T08:53:39.871 INFO:teuthology.orchestra.run.smithi196.stdout:3532 trunc from 0x474aa00 to 0xea08a00 2024-05-08T08:53:39.885 INFO:teuthology.orchestra.run.smithi196.stdout:3533 write 0x5133e00 thru 0x513ce9a (0x909b bytes) 2024-05-08T08:53:39.897 INFO:teuthology.orchestra.run.smithi196.stdout:3538 trunc from 0xea08a00 to 0x99bc200 2024-05-08T08:53:39.915 INFO:teuthology.orchestra.run.smithi196.stdout:3541 read 0x8f6ba00 thru 0x8f77d51 (0xc352 bytes) 2024-05-08T08:53:39.916 INFO:teuthology.orchestra.run.smithi196.stdout:3542 read 0x3a48c00 thru 0x3a588e9 (0xfcea bytes) 2024-05-08T08:53:39.916 INFO:teuthology.orchestra.run.smithi196.stdout:3543 write 0x1a33400 thru 0x1a387cb (0x53cc bytes) 2024-05-08T08:53:39.916 INFO:teuthology.orchestra.run.smithi196.stdout:3544 read 0x118bc00 thru 0x1193bd1 (0x7fd2 bytes) 2024-05-08T08:53:39.930 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:39.932+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:39.932 INFO:teuthology.orchestra.run.smithi196.stdout:3546 write 0xc321600 thru 0xc327370 (0x5d71 bytes) 2024-05-08T08:53:39.940 INFO:teuthology.orchestra.run.smithi196.stdout:3547 punch from 0xbaf6e00 to 0xbb00c00, (0x9e00 bytes) 2024-05-08T08:53:39.958 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:39.958 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:39.961 INFO:teuthology.orchestra.run.smithi196.stdout:3548 write 0xd341800 thru 0xd349031 (0x7832 bytes) 2024-05-08T08:53:39.961 INFO:teuthology.orchestra.run.smithi196.stdout:3549 write 0x3de2200 thru 0x3defe89 (0xdc8a bytes) 2024-05-08T08:53:39.968 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:53:39.977 INFO:teuthology.orchestra.run.smithi196.stdout:3550 write 0xec32800 thru 0xec41ae8 (0xf2e9 bytes) 2024-05-08T08:53:39.984 INFO:teuthology.orchestra.run.smithi196.stdout:3551 punch from 0xce08200 to 0xce0ee00, (0x6c00 bytes) 2024-05-08T08:53:39.993 INFO:teuthology.orchestra.run.smithi196.stdout:3552 trunc from 0xec41c00 to 0x7b01600 2024-05-08T08:53:40.010 INFO:teuthology.orchestra.run.smithi196.stdout:3553 punch from 0x28e0400 to 0x28eb000, (0xac00 bytes) 2024-05-08T08:53:40.012 INFO:teuthology.orchestra.run.smithi196.stdout:3554 read 0x7526000 thru 0x7534a4f (0xea50 bytes) 2024-05-08T08:53:40.013 INFO:teuthology.orchestra.run.smithi196.stdout:3556 write 0x3391200 thru 0x3391b17 (0x918 bytes) 2024-05-08T08:53:40.013 INFO:teuthology.orchestra.run.smithi196.stdout:3558 write 0x1c2aa00 thru 0x1c39f45 (0xf546 bytes) 2024-05-08T08:53:40.013 INFO:teuthology.orchestra.run.smithi196.stdout:3560 write 0x6688400 thru 0x6694816 (0xc417 bytes) 2024-05-08T08:53:40.013 INFO:teuthology.orchestra.run.smithi196.stdout:3561 read 0x54cc400 thru 0x54d0948 (0x4549 bytes) 2024-05-08T08:53:40.013 INFO:teuthology.orchestra.run.smithi196.stdout:3562 write 0x271f200 thru 0x272cafd (0xd8fe bytes) 2024-05-08T08:53:40.013 INFO:teuthology.orchestra.run.smithi196.stdout:3563 punch from 0x17f7200 to 0x17fae00, (0x3c00 bytes) 2024-05-08T08:53:40.013 INFO:teuthology.orchestra.run.smithi196.stdout:3564 write 0x5366c00 thru 0x536a12e (0x352f bytes) 2024-05-08T08:53:40.030 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:53:40.033+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:53:40.041 INFO:teuthology.orchestra.run.smithi196.stdout:3565 write 0xa7c6400 thru 0xa7ccef5 (0x6af6 bytes) 2024-05-08T08:53:40.041 INFO:teuthology.orchestra.run.smithi196.stdout:3566 read 0x8cb4600 thru 0x8cb8acb (0x44cc bytes) 2024-05-08T08:53:40.041 INFO:teuthology.orchestra.run.smithi196.stdout:3569 write 0x526f600 thru 0x5271d18 (0x2719 bytes) 2024-05-08T08:53:40.065 INFO:teuthology.orchestra.run.smithi196.stdout:3570 write 0xbcb3000 thru 0xbcc0dbe (0xddbf bytes) 2024-05-08T08:53:40.066 INFO:teuthology.orchestra.run.smithi196.stdout:3571 write 0x5f5e400 thru 0x5f69152 (0xad53 bytes) 2024-05-08T08:53:40.066 INFO:teuthology.orchestra.run.smithi196.stdout:3572 read 0x9333800 thru 0x9335277 (0x1a78 bytes) 2024-05-08T08:53:40.066 INFO:teuthology.orchestra.run.smithi196.stdout:3573 read 0x3239800 thru 0x3246c9b (0xd49c bytes) 2024-05-08T08:53:40.066 INFO:teuthology.orchestra.run.smithi196.stdout:3575 read 0x720ec00 thru 0x721a53e (0xb93f bytes) 2024-05-08T08:53:40.066 INFO:teuthology.orchestra.run.smithi196.stdout:3577 punch from 0xb691000 to 0xb6a0600, (0xf600 bytes) 2024-05-08T08:53:40.066 INFO:teuthology.orchestra.run.smithi196.stdout:3578 read 0x9378000 thru 0x937f716 (0x7717 bytes) 2024-05-08T08:53:40.066 INFO:teuthology.orchestra.run.smithi196.stdout:3582 punch from 0xb464400 to 0xb46da00, (0x9600 bytes) 2024-05-08T08:53:40.066 INFO:teuthology.orchestra.run.smithi196.stdout:3583 write 0x98f7400 thru 0x99034be (0xc0bf bytes) 2024-05-08T08:53:40.066 INFO:teuthology.orchestra.run.smithi196.stdout:3585 read 0x6a00000 thru 0x6a0c1e6 (0xc1e7 bytes) 2024-05-08T08:53:40.066 INFO:teuthology.orchestra.run.smithi196.stdout:3586 clone 26 order 18 su 16384 sc 12 2024-05-08T08:53:40.080 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:40.080 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:40.090 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:53:40.131 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:40.135+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:40.176 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T08:53:40.195 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:40.195 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:40.205 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:53:40.232 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:53:40.235+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:53:40.301 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:53:40.301 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:53:40.311 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:40.312 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:40.318 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:53:40.318 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:53:40.320 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:53:40.334 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:40.337+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:40.335 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:53:40.335 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:53:40.354 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:53:40.354 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:53:40.371 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:53:40.372 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:53:40.389 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:53:40.390 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:53:40.432 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:40.432 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:53:40.432 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:53:40.432 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:40.434 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:40.438+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:40.440 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:53:40.535 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:40.538+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:40.594 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:40.594 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:53:40.595 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:53:40.595 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:53:40.595 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:40.603 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:53:40.615 INFO:teuthology.orchestra.run.smithi196.stdout:truncating image image_client.0-clone25 from 0xbcc0e00 (overlap 0x5c8200) to 0x94000 2024-05-08T08:53:40.647 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:53:40.650+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:53:40.730 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:40.730 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:53:40.730 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:53:40.730 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:53:40.731 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.731 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4621.0:9089 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 23339520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:53:40.731 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:53:40.731 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 68.238307402999993, 2024-05-08T08:53:40.731 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.13814907200000001, 2024-05-08T08:53:40.731 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:40.731 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:40.731 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:53:40.731 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:40.731 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4621", 2024-05-08T08:53:40.731 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:40.731 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9089 2024-05-08T08:53:40.732 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.732 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:40.732 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.732 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:40.732 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:53:40.732 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:40.732 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.732 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.732 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:40.732 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:53:40.732 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:40.732 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.732 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.732 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:40.732 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493005+0000", 2024-05-08T08:53:40.733 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1700000000000001e-06 2024-05-08T08:53:40.733 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.733 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.733 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:40.733 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.513963+0000", 2024-05-08T08:53:40.733 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.020957581999999999 2024-05-08T08:53:40.733 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.733 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.733 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:40.733 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.513995+0000", 2024-05-08T08:53:40.733 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1993999999999999e-05 2024-05-08T08:53:40.733 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.733 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.733 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:40.734 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514007+0000", 2024-05-08T08:53:40.734 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1581e-05 2024-05-08T08:53:40.734 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.734 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.734 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:40.734 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514122+0000", 2024-05-08T08:53:40.734 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000115282 2024-05-08T08:53:40.734 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.734 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.734 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op must wait for map", 2024-05-08T08:53:40.734 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514128+0000", 2024-05-08T08:53:40.734 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9950000000000002e-06 2024-05-08T08:53:40.734 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.735 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.735 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:40.735 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.515800+0000", 2024-05-08T08:53:40.735 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001671886 2024-05-08T08:53:40.735 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.735 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.735 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:40.735 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.515893+0000", 2024-05-08T08:53:40.735 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3490000000000001e-05 2024-05-08T08:53:40.735 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.735 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.735 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 3", 2024-05-08T08:53:40.735 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.517647+0000", 2024-05-08T08:53:40.735 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0017535630000000001 2024-05-08T08:53:40.736 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.736 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.736 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:53:40.736 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.627578+0000", 2024-05-08T08:53:40.736 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.109931557 2024-05-08T08:53:40.736 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.736 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.736 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:53:40.736 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631055+0000", 2024-05-08T08:53:40.736 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0034761420000000002 2024-05-08T08:53:40.736 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.736 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.736 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:53:40.736 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631096+0000", 2024-05-08T08:53:40.736 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1059000000000001e-05 2024-05-08T08:53:40.736 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.737 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.737 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:40.737 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631151+0000", 2024-05-08T08:53:40.737 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5770999999999998e-05 2024-05-08T08:53:40.737 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:40.737 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:40.737 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:40.737 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.737 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.737 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1071 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11026432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:40.737 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:53:40.737 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 64.245857912000005, 2024-05-08T08:53:40.738 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070580037999999998, 2024-05-08T08:53:40.738 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:40.738 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:40.738 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:40.738 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:40.738 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:40.738 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:40.738 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1071 2024-05-08T08:53:40.738 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.739 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:40.739 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.739 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:40.739 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:53:40.739 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:40.739 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.740 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.740 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:40.740 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:53:40.740 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:40.740 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.740 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.740 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:40.741 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:53:40.741 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2800000000000002e-07 2024-05-08T08:53:40.741 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.741 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.741 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:40.741 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485454+0000", 2024-05-08T08:53:40.742 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2290000000000002e-06 2024-05-08T08:53:40.742 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.742 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.742 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:40.742 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485461+0000", 2024-05-08T08:53:40.742 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9920000000000001e-06 2024-05-08T08:53:40.742 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.742 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.743 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:40.743 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485467+0000", 2024-05-08T08:53:40.743 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3979999999999998e-06 2024-05-08T08:53:40.743 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.743 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.743 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:40.743 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555880+0000", 2024-05-08T08:53:40.743 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070413165 2024-05-08T08:53:40.743 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.744 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.744 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:40.744 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555926+0000", 2024-05-08T08:53:40.744 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6217000000000002e-05 2024-05-08T08:53:40.744 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.744 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.744 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:40.744 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556032+0000", 2024-05-08T08:53:40.745 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000105609 2024-05-08T08:53:40.745 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:40.745 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:40.745 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:40.745 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.745 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.745 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1073 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11067392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:40.745 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:53:40.746 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 64.245836690999994, 2024-05-08T08:53:40.746 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070821404000000004, 2024-05-08T08:53:40.746 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:40.746 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:40.746 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:40.746 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:40.747 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:40.747 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:40.747 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1073 2024-05-08T08:53:40.747 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.747 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:40.747 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.747 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:40.747 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:53:40.747 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:40.747 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.747 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.747 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:40.747 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:53:40.747 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:40.747 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.748 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.748 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:40.748 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:53:40.748 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.08e-07 2024-05-08T08:53:40.748 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.748 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.748 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:40.748 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485476+0000", 2024-05-08T08:53:40.748 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3929999999999998e-06 2024-05-08T08:53:40.748 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.748 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.748 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:40.748 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485485+0000", 2024-05-08T08:53:40.748 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1880000000000008e-06 2024-05-08T08:53:40.749 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.749 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.749 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:40.749 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485494+0000", 2024-05-08T08:53:40.749 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3109999999999995e-06 2024-05-08T08:53:40.749 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.749 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.749 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:40.749 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556082+0000", 2024-05-08T08:53:40.749 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070587426999999994 2024-05-08T08:53:40.749 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.750 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.750 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:40.750 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556177+0000", 2024-05-08T08:53:40.750 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5051999999999997e-05 2024-05-08T08:53:40.750 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.750 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.750 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:40.750 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556294+0000", 2024-05-08T08:53:40.750 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000117625 2024-05-08T08:53:40.750 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:40.750 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:40.750 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:40.750 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.751 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.751 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1075 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11108352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:40.751 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:53:40.751 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 64.245806114000004, 2024-05-08T08:53:40.751 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071045026999999997, 2024-05-08T08:53:40.751 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:40.751 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:40.751 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:40.751 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:40.751 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:40.751 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:40.751 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1075 2024-05-08T08:53:40.751 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.751 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:40.751 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.752 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:40.752 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:53:40.752 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:40.752 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.752 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.752 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:40.752 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:53:40.752 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:40.752 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.752 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.752 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:40.752 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:53:40.752 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.32e-07 2024-05-08T08:53:40.753 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.753 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.753 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:40.753 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485518+0000", 2024-05-08T08:53:40.753 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3769e-05 2024-05-08T08:53:40.753 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.753 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.753 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:40.753 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485527+0000", 2024-05-08T08:53:40.753 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6779999999999995e-06 2024-05-08T08:53:40.753 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.753 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.753 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:40.753 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485532+0000", 2024-05-08T08:53:40.753 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9830000000000004e-06 2024-05-08T08:53:40.754 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.754 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.754 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:40.754 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556340+0000", 2024-05-08T08:53:40.754 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070807743000000006 2024-05-08T08:53:40.754 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.754 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.754 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:40.754 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556435+0000", 2024-05-08T08:53:40.754 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4732e-05 2024-05-08T08:53:40.754 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.754 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.754 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:40.754 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556549+0000", 2024-05-08T08:53:40.754 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011369 2024-05-08T08:53:40.755 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:40.755 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:40.755 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:40.755 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.755 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.755 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1078 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11149312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:40.755 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:53:40.755 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 64.245765915000007, 2024-05-08T08:53:40.755 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071167276000000002, 2024-05-08T08:53:40.755 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:40.755 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:40.755 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:40.755 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:40.755 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:40.755 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:40.756 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1078 2024-05-08T08:53:40.756 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.756 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:40.756 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.756 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:40.756 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:53:40.756 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:40.756 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.756 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.756 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:40.756 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:53:40.756 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:40.756 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.756 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.756 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:40.756 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:53:40.757 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5600000000000001e-07 2024-05-08T08:53:40.757 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.757 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.757 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:40.757 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485547+0000", 2024-05-08T08:53:40.757 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3590000000000002e-06 2024-05-08T08:53:40.757 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.757 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.757 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:40.757 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485553+0000", 2024-05-08T08:53:40.757 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2770000000000002e-06 2024-05-08T08:53:40.757 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.757 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.757 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:40.757 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485558+0000", 2024-05-08T08:53:40.757 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.95e-06 2024-05-08T08:53:40.758 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.758 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.758 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:40.758 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556582+0000", 2024-05-08T08:53:40.758 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071024444000000006 2024-05-08T08:53:40.758 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.758 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.758 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:40.758 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556637+0000", 2024-05-08T08:53:40.758 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4672000000000001e-05 2024-05-08T08:53:40.758 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.758 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.758 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:40.758 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556711+0000", 2024-05-08T08:53:40.759 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4117999999999997e-05 2024-05-08T08:53:40.759 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:40.759 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:40.759 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:40.759 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.759 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.759 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1080 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11190272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:40.759 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:53:40.759 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 64.245745181999993, 2024-05-08T08:53:40.759 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071350721000000006, 2024-05-08T08:53:40.759 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:40.759 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:40.759 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:40.759 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:40.759 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:40.760 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:40.760 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1080 2024-05-08T08:53:40.760 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.760 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:40.760 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.760 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:40.760 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:53:40.760 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:40.760 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.760 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.760 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:40.760 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:53:40.760 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:40.760 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.760 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.761 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:40.761 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:53:40.761 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.03e-07 2024-05-08T08:53:40.761 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.761 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.761 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:40.761 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485567+0000", 2024-05-08T08:53:40.761 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1090000000000001e-06 2024-05-08T08:53:40.761 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.761 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.761 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:40.761 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485573+0000", 2024-05-08T08:53:40.761 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3049999999999998e-06 2024-05-08T08:53:40.761 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.762 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.762 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:40.762 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485578+0000", 2024-05-08T08:53:40.762 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0130000000000001e-06 2024-05-08T08:53:40.762 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.762 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.762 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:40.762 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556743+0000", 2024-05-08T08:53:40.762 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071164449000000005 2024-05-08T08:53:40.762 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.762 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.763 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:40.763 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556797+0000", 2024-05-08T08:53:40.763 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4589999999999997e-05 2024-05-08T08:53:40.763 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.763 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.763 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:40.763 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556915+0000", 2024-05-08T08:53:40.763 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000117852 2024-05-08T08:53:40.763 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:40.763 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:40.763 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:40.763 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.763 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.763 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1082 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11231232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:40.763 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:53:40.764 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 64.245724938999999, 2024-05-08T08:53:40.764 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071591396000000002, 2024-05-08T08:53:40.764 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:40.764 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:40.764 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:40.764 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:40.764 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:40.764 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:40.764 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1082 2024-05-08T08:53:40.764 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.764 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:40.764 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.764 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:40.764 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:53:40.764 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:40.765 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.765 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.765 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:40.765 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:53:40.765 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:40.765 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.765 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.765 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:40.765 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:53:40.765 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1800000000000001e-07 2024-05-08T08:53:40.765 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.765 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.765 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:40.765 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485587+0000", 2024-05-08T08:53:40.765 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2589999999999999e-06 2024-05-08T08:53:40.766 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.766 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.766 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:40.766 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485594+0000", 2024-05-08T08:53:40.766 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0789999999999997e-06 2024-05-08T08:53:40.766 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.766 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.766 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:40.766 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485598+0000", 2024-05-08T08:53:40.766 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7480000000000001e-06 2024-05-08T08:53:40.766 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.766 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.766 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:40.767 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556971+0000", 2024-05-08T08:53:40.767 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071372774999999999 2024-05-08T08:53:40.767 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.767 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.767 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:40.767 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557069+0000", 2024-05-08T08:53:40.767 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.7732999999999994e-05 2024-05-08T08:53:40.767 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.767 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.767 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:40.767 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557176+0000", 2024-05-08T08:53:40.767 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000107384 2024-05-08T08:53:40.767 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:40.767 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:40.767 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:40.768 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.768 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.768 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1084 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11272192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:40.768 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:53:40.768 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 64.245686300000003, 2024-05-08T08:53:40.768 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071808977999999996, 2024-05-08T08:53:40.768 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:40.768 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:40.768 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:40.768 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:40.769 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:40.769 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:40.769 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1084 2024-05-08T08:53:40.769 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.769 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:40.769 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.769 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:40.769 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:53:40.769 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:40.769 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.769 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.769 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:40.769 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:53:40.769 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:40.770 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.770 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.770 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:40.770 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485624+0000", 2024-05-08T08:53:40.770 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:53:40.770 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.770 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.770 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:40.770 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485626+0000", 2024-05-08T08:53:40.770 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9460000000000001e-06 2024-05-08T08:53:40.770 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.770 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.770 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:40.770 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485633+0000", 2024-05-08T08:53:40.771 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1149999999999996e-06 2024-05-08T08:53:40.771 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.771 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.771 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:40.771 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485638+0000", 2024-05-08T08:53:40.771 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5670000000000002e-06 2024-05-08T08:53:40.771 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.771 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.771 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:40.771 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557226+0000", 2024-05-08T08:53:40.771 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071587267999999996 2024-05-08T08:53:40.771 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.771 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.771 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:40.771 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557321+0000", 2024-05-08T08:53:40.772 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5785000000000005e-05 2024-05-08T08:53:40.772 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.772 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.772 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:40.772 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557432+0000", 2024-05-08T08:53:40.772 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011092800000000001 2024-05-08T08:53:40.772 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:40.772 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:40.772 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:40.772 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.772 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.772 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1086 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11313152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:40.772 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:53:40.772 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 64.245662972999995, 2024-05-08T08:53:40.773 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071933132999999996, 2024-05-08T08:53:40.773 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:40.773 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:40.773 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:40.773 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:40.773 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:40.773 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:40.773 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1086 2024-05-08T08:53:40.773 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.773 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:40.773 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.773 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:40.773 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:53:40.773 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:40.773 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.774 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.774 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:40.774 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:53:40.774 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:40.774 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.774 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.774 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:40.774 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486328+0000", 2024-05-08T08:53:40.774 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00068107900000000002 2024-05-08T08:53:40.774 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.774 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.774 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:40.774 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486331+0000", 2024-05-08T08:53:40.775 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.585e-06 2024-05-08T08:53:40.775 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.775 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.775 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:40.775 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486343+0000", 2024-05-08T08:53:40.775 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1856e-05 2024-05-08T08:53:40.775 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.775 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.775 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:40.775 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486350+0000", 2024-05-08T08:53:40.775 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8600000000000004e-06 2024-05-08T08:53:40.775 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.775 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.775 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:40.776 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557470+0000", 2024-05-08T08:53:40.776 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071120326999999997 2024-05-08T08:53:40.776 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.776 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.776 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:40.776 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557514+0000", 2024-05-08T08:53:40.776 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3909000000000003e-05 2024-05-08T08:53:40.776 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.776 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.776 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:40.776 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557580+0000", 2024-05-08T08:53:40.776 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5517000000000003e-05 2024-05-08T08:53:40.776 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:40.777 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:40.777 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:40.777 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.777 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.777 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1089 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11354112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:40.777 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:53:40.777 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 64.244950101000001, 2024-05-08T08:53:40.777 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071457381, 2024-05-08T08:53:40.777 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:40.777 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:40.777 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:40.777 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:40.777 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:40.777 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:40.777 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1089 2024-05-08T08:53:40.778 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.778 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:40.778 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.778 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:40.778 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:53:40.778 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:40.778 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.778 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.778 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:40.778 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:53:40.778 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:40.778 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.778 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.778 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:40.779 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486851+0000", 2024-05-08T08:53:40.779 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00049102 2024-05-08T08:53:40.779 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.779 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.779 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:40.779 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486854+0000", 2024-05-08T08:53:40.779 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6859999999999999e-06 2024-05-08T08:53:40.779 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.779 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.779 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:40.779 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486864+0000", 2024-05-08T08:53:40.779 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6080000000000002e-06 2024-05-08T08:53:40.779 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.779 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.779 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:40.780 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486870+0000", 2024-05-08T08:53:40.780 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.477e-06 2024-05-08T08:53:40.780 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.780 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.780 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:40.780 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557612+0000", 2024-05-08T08:53:40.780 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070741557999999996 2024-05-08T08:53:40.780 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.780 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.780 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:40.780 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557695+0000", 2024-05-08T08:53:40.780 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3300000000000005e-05 2024-05-08T08:53:40.780 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.780 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.781 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:40.781 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557817+0000", 2024-05-08T08:53:40.781 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012173200000000001 2024-05-08T08:53:40.781 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:40.781 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:40.781 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:40.781 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.781 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.781 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1091 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11395072~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:40.781 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:53:40.781 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 64.244429306000001, 2024-05-08T08:53:40.781 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071184737999999997, 2024-05-08T08:53:40.781 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:40.781 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:40.781 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:40.782 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:40.782 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:40.782 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:40.782 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1091 2024-05-08T08:53:40.782 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.782 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:40.782 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.782 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:40.782 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:53:40.782 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:40.782 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.782 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.782 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:40.783 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:53:40.783 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:40.783 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.783 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.783 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:40.783 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486881+0000", 2024-05-08T08:53:40.783 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0699999999999997e-07 2024-05-08T08:53:40.783 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.783 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.783 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:40.783 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486883+0000", 2024-05-08T08:53:40.783 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.402e-06 2024-05-08T08:53:40.783 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.783 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.783 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:40.784 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486890+0000", 2024-05-08T08:53:40.784 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0480000000000001e-06 2024-05-08T08:53:40.784 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.784 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.784 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:40.784 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486896+0000", 2024-05-08T08:53:40.784 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7640000000000002e-06 2024-05-08T08:53:40.784 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.784 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.784 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:40.784 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557860+0000", 2024-05-08T08:53:40.784 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070963791999999998 2024-05-08T08:53:40.784 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.784 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.784 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:40.785 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557952+0000", 2024-05-08T08:53:40.785 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2082000000000004e-05 2024-05-08T08:53:40.785 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.785 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.785 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:40.785 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558065+0000", 2024-05-08T08:53:40.785 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000113143 2024-05-08T08:53:40.785 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:40.785 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:40.785 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:40.785 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.785 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.785 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1094 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11436032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:40.785 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:53:40.786 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 64.244406138000002, 2024-05-08T08:53:40.786 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071400507000000002, 2024-05-08T08:53:40.786 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:40.786 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:40.786 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:40.786 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:40.786 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:40.786 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:40.786 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1094 2024-05-08T08:53:40.786 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.787 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:40.787 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.787 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:40.787 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:53:40.787 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:40.787 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.787 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.787 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:40.787 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:53:40.787 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:40.787 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.787 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.787 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:40.787 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487940+0000", 2024-05-08T08:53:40.788 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001036868 2024-05-08T08:53:40.788 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.788 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.788 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:40.788 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487944+0000", 2024-05-08T08:53:40.788 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7069999999999998e-06 2024-05-08T08:53:40.788 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.788 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.788 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:40.788 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487951+0000", 2024-05-08T08:53:40.788 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2649999999999999e-06 2024-05-08T08:53:40.788 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.788 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.788 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:40.788 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487961+0000", 2024-05-08T08:53:40.789 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4110000000000002e-06 2024-05-08T08:53:40.789 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.789 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.789 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:40.789 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558117+0000", 2024-05-08T08:53:40.789 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070156662999999994 2024-05-08T08:53:40.789 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.789 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.789 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:40.789 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558214+0000", 2024-05-08T08:53:40.789 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6444e-05 2024-05-08T08:53:40.789 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.789 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.789 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:40.790 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558304+0000", 2024-05-08T08:53:40.790 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0148999999999994e-05 2024-05-08T08:53:40.790 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:40.790 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:40.790 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:40.790 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.790 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.790 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1096 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11476992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:40.790 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:53:40.790 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 64.243339606999996, 2024-05-08T08:53:40.790 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070480074000000004, 2024-05-08T08:53:40.790 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:40.791 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:40.791 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:40.791 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:40.791 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:40.791 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:40.791 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1096 2024-05-08T08:53:40.791 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.791 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:40.791 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.791 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:40.791 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:53:40.791 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:40.791 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.791 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.792 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:40.792 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:53:40.792 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:40.792 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.792 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.792 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:40.792 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:53:40.792 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3700000000000001e-07 2024-05-08T08:53:40.792 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.792 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.793 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:40.793 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487973+0000", 2024-05-08T08:53:40.793 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0870000000000001e-06 2024-05-08T08:53:40.793 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.793 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.793 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:40.793 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487977+0000", 2024-05-08T08:53:40.793 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0600000000000001e-06 2024-05-08T08:53:40.793 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.793 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.793 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:40.793 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487980+0000", 2024-05-08T08:53:40.793 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1089999999999998e-06 2024-05-08T08:53:40.793 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.794 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.794 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:40.794 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558338+0000", 2024-05-08T08:53:40.794 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070358233000000006 2024-05-08T08:53:40.794 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.794 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.794 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:40.794 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558392+0000", 2024-05-08T08:53:40.794 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4537999999999999e-05 2024-05-08T08:53:40.794 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.794 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.794 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:40.795 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558450+0000", 2024-05-08T08:53:40.795 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7710000000000001e-05 2024-05-08T08:53:40.795 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:40.795 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:40.795 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:40.795 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.795 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.795 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1099 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11517952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:40.795 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:53:40.795 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 64.243325730999999, 2024-05-08T08:53:40.795 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070720506000000002, 2024-05-08T08:53:40.795 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:40.795 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:40.795 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:40.795 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:40.796 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:40.796 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:40.796 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1099 2024-05-08T08:53:40.796 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.796 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:40.796 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.796 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:40.796 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:53:40.796 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:40.796 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.796 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.796 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:40.796 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:53:40.797 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:40.797 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.797 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.797 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:40.797 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:53:40.797 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3999999999999998e-07 2024-05-08T08:53:40.797 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.797 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.797 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:40.797 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487986+0000", 2024-05-08T08:53:40.797 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3519999999999999e-06 2024-05-08T08:53:40.797 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.798 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.798 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:40.798 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487989+0000", 2024-05-08T08:53:40.798 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.23e-06 2024-05-08T08:53:40.798 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.798 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.798 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:40.799 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487991+0000", 2024-05-08T08:53:40.799 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6350000000000002e-06 2024-05-08T08:53:40.799 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.799 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.799 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:40.799 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558489+0000", 2024-05-08T08:53:40.799 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070497924000000003 2024-05-08T08:53:40.799 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.799 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.799 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:40.799 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558583+0000", 2024-05-08T08:53:40.799 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3405000000000004e-05 2024-05-08T08:53:40.799 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.800 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.800 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:40.800 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558704+0000", 2024-05-08T08:53:40.800 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012171999999999999 2024-05-08T08:53:40.800 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:40.800 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:40.800 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:40.800 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.800 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.800 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1100 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11558912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:40.800 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:53:40.801 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 64.243312732000007, 2024-05-08T08:53:40.801 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070962266999999996, 2024-05-08T08:53:40.801 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:40.801 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:40.801 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:40.801 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:40.801 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:40.801 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:40.801 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1100 2024-05-08T08:53:40.801 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.801 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:40.801 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.802 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:40.802 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:53:40.802 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:40.802 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.802 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.802 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:40.802 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:53:40.802 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:40.802 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.802 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.802 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:40.802 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488430+0000", 2024-05-08T08:53:40.802 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000432676 2024-05-08T08:53:40.802 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.803 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.803 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:40.803 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488433+0000", 2024-05-08T08:53:40.803 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.45e-06 2024-05-08T08:53:40.803 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.803 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.803 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:40.803 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488442+0000", 2024-05-08T08:53:40.803 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.687e-06 2024-05-08T08:53:40.803 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.803 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.803 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:40.803 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488446+0000", 2024-05-08T08:53:40.803 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4939999999999997e-06 2024-05-08T08:53:40.803 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.804 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.804 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:40.804 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558756+0000", 2024-05-08T08:53:40.804 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070310051999999998 2024-05-08T08:53:40.804 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.804 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.804 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:40.804 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558844+0000", 2024-05-08T08:53:40.804 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7690999999999998e-05 2024-05-08T08:53:40.804 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.804 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.804 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:40.804 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558959+0000", 2024-05-08T08:53:40.804 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000115217 2024-05-08T08:53:40.805 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:40.805 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:40.805 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:40.805 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.805 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.805 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1103 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11599872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:40.805 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:53:40.805 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 64.242855057, 2024-05-08T08:53:40.805 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070777100999999995, 2024-05-08T08:53:40.805 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:40.805 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:40.805 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:40.805 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:40.805 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:40.805 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:40.806 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1103 2024-05-08T08:53:40.806 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.806 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:40.806 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.806 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:40.806 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:53:40.806 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:40.806 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.806 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.806 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:40.806 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:53:40.806 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:40.806 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.806 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.807 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:40.807 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488959+0000", 2024-05-08T08:53:40.807 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00050457200000000001 2024-05-08T08:53:40.807 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.807 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.807 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:40.807 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488969+0000", 2024-05-08T08:53:40.807 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.7159999999999998e-06 2024-05-08T08:53:40.807 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.807 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.807 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:40.807 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488981+0000", 2024-05-08T08:53:40.808 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2150000000000001e-05 2024-05-08T08:53:40.808 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.808 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.808 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:40.808 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488988+0000", 2024-05-08T08:53:40.808 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8759999999999999e-06 2024-05-08T08:53:40.808 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.808 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.808 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:40.808 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558992+0000", 2024-05-08T08:53:40.809 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070004358000000003 2024-05-08T08:53:40.809 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.809 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.809 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:40.809 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559093+0000", 2024-05-08T08:53:40.809 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000100299 2024-05-08T08:53:40.809 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.809 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.809 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:40.809 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559232+0000", 2024-05-08T08:53:40.810 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013913 2024-05-08T08:53:40.810 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:40.810 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:40.810 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:40.810 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.810 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.810 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1104 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11640832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:40.810 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:53:40.810 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 64.242310242000002, 2024-05-08T08:53:40.810 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070521947000000001, 2024-05-08T08:53:40.810 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:40.810 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:40.810 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:40.810 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:40.811 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:40.811 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:40.811 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1104 2024-05-08T08:53:40.811 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.811 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:40.811 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.811 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:40.811 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:53:40.811 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:40.811 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.811 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.811 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:40.811 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:53:40.811 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:40.811 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.811 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.812 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:40.812 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489700+0000", 2024-05-08T08:53:40.812 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00070067600000000001 2024-05-08T08:53:40.812 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.812 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.812 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:40.812 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489704+0000", 2024-05-08T08:53:40.812 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1640000000000003e-06 2024-05-08T08:53:40.812 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.812 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.812 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:40.812 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489716+0000", 2024-05-08T08:53:40.812 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1379000000000001e-05 2024-05-08T08:53:40.812 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.812 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.812 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:40.813 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489724+0000", 2024-05-08T08:53:40.813 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8949999999999998e-06 2024-05-08T08:53:40.813 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.813 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.813 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:40.813 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559278+0000", 2024-05-08T08:53:40.813 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.069554151999999994 2024-05-08T08:53:40.813 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.813 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.813 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:40.813 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559387+0000", 2024-05-08T08:53:40.813 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000109258 2024-05-08T08:53:40.813 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.814 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.814 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:40.814 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559521+0000", 2024-05-08T08:53:40.814 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000134423 2024-05-08T08:53:40.814 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:40.814 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:40.814 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:40.814 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.814 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.815 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:8636 3.17 3:e8a0a3e4:::rbd_header.11402b26385:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-05-08T08:53:40.815 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:54.150971+0000", 2024-05-08T08:53:40.815 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 46.580338247999997, 2024-05-08T08:53:40.815 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.217273233, 2024-05-08T08:53:40.815 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:40.815 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:40.815 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:40.815 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:40.815 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:40.815 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:40.815 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 8636 2024-05-08T08:53:40.815 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.815 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:40.816 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.816 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:40.816 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150971+0000", 2024-05-08T08:53:40.816 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:40.816 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.816 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.816 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:40.816 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150971+0000", 2024-05-08T08:53:40.816 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:40.816 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.816 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.816 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:40.816 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150972+0000", 2024-05-08T08:53:40.816 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.71e-07 2024-05-08T08:53:40.816 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.816 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.817 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:40.817 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150977+0000", 2024-05-08T08:53:40.817 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.152e-06 2024-05-08T08:53:40.817 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.817 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.817 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:40.817 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150990+0000", 2024-05-08T08:53:40.817 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2364999999999999e-05 2024-05-08T08:53:40.817 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.817 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.817 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:40.817 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150998+0000", 2024-05-08T08:53:40.817 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0739999999999993e-06 2024-05-08T08:53:40.817 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.817 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.818 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:40.818 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.151054+0000", 2024-05-08T08:53:40.818 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5949000000000003e-05 2024-05-08T08:53:40.818 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.818 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.818 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:53:40.818 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.151060+0000", 2024-05-08T08:53:40.818 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6120000000000004e-06 2024-05-08T08:53:40.818 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.818 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.818 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:40.818 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.367640+0000", 2024-05-08T08:53:40.818 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2165792739999999 2024-05-08T08:53:40.818 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.819 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.819 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:40.819 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.367839+0000", 2024-05-08T08:53:40.819 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000199185 2024-05-08T08:53:40.819 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.819 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.820 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:40.820 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.368245+0000", 2024-05-08T08:53:40.820 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00040575099999999998 2024-05-08T08:53:40.820 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:40.820 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:40.820 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:40.820 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.820 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.820 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5302.0:1516 3.c 3:319bf789:::rbd_data.1140d3fc54c.0000000000000025:head [stat,write 1892352~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:53:40.820 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:27.822220+0000", 2024-05-08T08:53:40.820 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 12.909089311000001, 2024-05-08T08:53:40.820 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.092643744, 2024-05-08T08:53:40.820 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:40.820 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:40.821 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:53:40.821 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:40.821 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5302", 2024-05-08T08:53:40.821 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/359277859", 2024-05-08T08:53:40.821 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1516 2024-05-08T08:53:40.821 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.821 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:40.821 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.821 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:40.821 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822220+0000", 2024-05-08T08:53:40.821 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:40.821 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.821 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.821 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:40.821 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822220+0000", 2024-05-08T08:53:40.822 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:40.822 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.822 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.822 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:40.822 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822221+0000", 2024-05-08T08:53:40.822 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3500000000000002e-07 2024-05-08T08:53:40.822 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.822 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.822 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:40.822 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822228+0000", 2024-05-08T08:53:40.822 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9959999999999996e-06 2024-05-08T08:53:40.822 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.822 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.823 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:40.823 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822234+0000", 2024-05-08T08:53:40.823 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5429999999999999e-06 2024-05-08T08:53:40.823 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.823 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.823 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:40.823 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822238+0000", 2024-05-08T08:53:40.823 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.732e-06 2024-05-08T08:53:40.823 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.823 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.823 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:40.823 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822285+0000", 2024-05-08T08:53:40.823 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6810000000000001e-05 2024-05-08T08:53:40.823 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.823 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.824 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:40.824 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822358+0000", 2024-05-08T08:53:40.825 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3473000000000006e-05 2024-05-08T08:53:40.825 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.825 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.825 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 5", 2024-05-08T08:53:40.825 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822489+0000", 2024-05-08T08:53:40.825 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013011199999999999 2024-05-08T08:53:40.825 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.826 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.826 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:53:40.826 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.825332+0000", 2024-05-08T08:53:40.826 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0028434799999999998 2024-05-08T08:53:40.826 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.826 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.826 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:53:40.826 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.914753+0000", 2024-05-08T08:53:40.826 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089421424999999999 2024-05-08T08:53:40.826 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.826 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.826 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:53:40.826 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.914807+0000", 2024-05-08T08:53:40.826 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3829000000000002e-05 2024-05-08T08:53:40.826 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.827 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.827 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:40.827 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.914864+0000", 2024-05-08T08:53:40.827 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6909e-05 2024-05-08T08:53:40.827 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:40.827 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:40.827 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:40.827 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.827 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.827 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:24567 3.6 3:66af1c40:::rbd_data.1140d3fc54c.000000000000005a:18 [read 2080768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:40.827 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.216598+0000", 2024-05-08T08:53:40.827 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 8.5147112309999997, 2024-05-08T08:53:40.827 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071602647000000005, 2024-05-08T08:53:40.827 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:40.827 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:40.828 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:40.828 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:40.828 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:40.828 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:40.828 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 24567 2024-05-08T08:53:40.828 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.828 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:40.828 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.828 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:40.828 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216598+0000", 2024-05-08T08:53:40.828 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:40.828 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.828 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.828 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:40.828 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216598+0000", 2024-05-08T08:53:40.829 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:40.829 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.829 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.829 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:40.829 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216599+0000", 2024-05-08T08:53:40.829 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8999999999999997e-07 2024-05-08T08:53:40.829 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.829 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.829 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:40.829 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216601+0000", 2024-05-08T08:53:40.829 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9889999999999999e-06 2024-05-08T08:53:40.829 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.830 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.830 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:40.830 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216607+0000", 2024-05-08T08:53:40.830 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3600000000000001e-06 2024-05-08T08:53:40.830 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.830 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.830 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:40.831 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216612+0000", 2024-05-08T08:53:40.831 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.69e-06 2024-05-08T08:53:40.831 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.831 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.831 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:40.831 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.287076+0000", 2024-05-08T08:53:40.831 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070464355000000006 2024-05-08T08:53:40.831 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.831 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.831 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:40.831 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.287151+0000", 2024-05-08T08:53:40.831 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4924000000000003e-05 2024-05-08T08:53:40.831 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:40.831 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:40.831 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:40.831 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.288201+0000", 2024-05-08T08:53:40.832 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0010498389999999999 2024-05-08T08:53:40.832 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:40.832 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:40.832 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:40.832 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:40.832 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:40.832 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:40.834 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:53:40.834 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:40.837+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:40.935 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:40.938+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:40.941 INFO:teuthology.orchestra.run.smithi196.stdout:checking clone #24, image image_client.0-clone24 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent25 2024-05-08T08:53:40.988 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:40.989 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:53:40.989 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:53:40.989 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:40.999 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:53:41.040 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:41.043+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:41.131 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 5, 4] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-05-08T08:53:41.131 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T08:53:41.131 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-05-08T08:53:41.132 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-05-08T08:53:41.176 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:41.179+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:41.182 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:41.182 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:53:41.182 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:53:41.182 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:53:41.182 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:41.194 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:53:41.237 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:41.240+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:41.301 INFO:teuthology.orchestra.run.smithi059.stderr:no valid command found; 10 closest matches: 2024-05-08T08:53:41.301 INFO:teuthology.orchestra.run.smithi059.stderr:0 2024-05-08T08:53:41.301 INFO:teuthology.orchestra.run.smithi059.stderr:1 2024-05-08T08:53:41.302 INFO:teuthology.orchestra.run.smithi059.stderr:2 2024-05-08T08:53:41.302 INFO:teuthology.orchestra.run.smithi059.stderr:abort 2024-05-08T08:53:41.302 INFO:teuthology.orchestra.run.smithi059.stderr:assert 2024-05-08T08:53:41.302 INFO:teuthology.orchestra.run.smithi059.stderr:bench [] [] [] [] 2024-05-08T08:53:41.302 INFO:teuthology.orchestra.run.smithi059.stderr:bluefs debug_inject_read_zeros 2024-05-08T08:53:41.302 INFO:teuthology.orchestra.run.smithi059.stderr:bluefs files list 2024-05-08T08:53:41.302 INFO:teuthology.orchestra.run.smithi059.stderr:bluefs stats 2024-05-08T08:53:41.302 INFO:teuthology.orchestra.run.smithi059.stderr:bluestore allocator dump block 2024-05-08T08:53:41.302 INFO:teuthology.orchestra.run.smithi059.stderr:Error EINVAL: invalid command 2024-05-08T08:53:41.305 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-08T08:53:41.305 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-05-08T08:53:41.305 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-05-08T08:53:41.349 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:53:41.352+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:53:41.355 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:41.356 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:53:41.356 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:53:41.356 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:53:41.356 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.356 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:12443 3.0 3:02d0158e:::rbd_data.114071bcf828.0000000000000037:head [stat,delete] snapc 19=[19] RETRY=3 ondisk+retry+write+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:41.356 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.610027+0000", 2024-05-08T08:53:41.356 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 9.7478092430000007, 2024-05-08T08:53:41.356 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.083586333999999998, 2024-05-08T08:53:41.356 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:41.356 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:41.356 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:41.356 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:41.356 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:41.356 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:41.356 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 12443 2024-05-08T08:53:41.357 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.357 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:41.357 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.357 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:41.357 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610027+0000", 2024-05-08T08:53:41.357 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:41.357 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.357 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.357 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:41.357 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610027+0000", 2024-05-08T08:53:41.357 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:41.357 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.357 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.358 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:41.358 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610028+0000", 2024-05-08T08:53:41.358 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.13e-07 2024-05-08T08:53:41.358 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.358 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.358 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:41.358 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610030+0000", 2024-05-08T08:53:41.358 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7669999999999999e-06 2024-05-08T08:53:41.358 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.358 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.358 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:41.358 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610039+0000", 2024-05-08T08:53:41.358 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4959999999999996e-06 2024-05-08T08:53:41.358 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.358 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.358 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:41.359 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610045+0000", 2024-05-08T08:53:41.359 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6779999999999997e-06 2024-05-08T08:53:41.359 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.359 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.359 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:41.359 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.660873+0000", 2024-05-08T08:53:41.359 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.050828748 2024-05-08T08:53:41.359 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.359 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.359 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:41.359 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.661017+0000", 2024-05-08T08:53:41.359 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014391999999999999 2024-05-08T08:53:41.359 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.359 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.359 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:41.359 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.693613+0000", 2024-05-08T08:53:41.359 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.032596111999999997 2024-05-08T08:53:41.360 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:41.360 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:41.360 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:41.360 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.360 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.360 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:14301 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [watch reconnect cookie 139837155901536 gen 12] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:41.360 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.645078+0000", 2024-05-08T08:53:41.360 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 9.7127586049999994, 2024-05-08T08:53:41.360 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088281852999999993, 2024-05-08T08:53:41.360 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:41.360 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:41.360 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:41.360 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:41.360 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:41.360 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:41.361 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14301 2024-05-08T08:53:41.361 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.361 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:41.361 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.361 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:41.361 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645078+0000", 2024-05-08T08:53:41.361 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:41.361 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.361 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.361 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:41.361 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645078+0000", 2024-05-08T08:53:41.361 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:41.361 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.361 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.362 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:41.362 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645084+0000", 2024-05-08T08:53:41.362 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4620000000000001e-06 2024-05-08T08:53:41.362 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.362 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.362 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:41.362 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645088+0000", 2024-05-08T08:53:41.362 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8039999999999999e-06 2024-05-08T08:53:41.362 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.362 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.362 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:41.362 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645106+0000", 2024-05-08T08:53:41.362 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.8318999999999999e-05 2024-05-08T08:53:41.362 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.362 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.363 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:41.363 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645113+0000", 2024-05-08T08:53:41.363 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0310000000000001e-06 2024-05-08T08:53:41.363 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.363 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.363 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:41.363 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.670493+0000", 2024-05-08T08:53:41.363 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.025379993 2024-05-08T08:53:41.363 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.363 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.363 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:41.363 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.670571+0000", 2024-05-08T08:53:41.363 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7613000000000004e-05 2024-05-08T08:53:41.363 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.363 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.363 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:41.364 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733360+0000", 2024-05-08T08:53:41.364 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.062788630999999998 2024-05-08T08:53:41.364 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:41.364 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:41.364 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:41.364 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.364 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.364 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:14420 3.0 3:06a8948e:::rbd_data.114071bcf828.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:41.364 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.649639+0000", 2024-05-08T08:53:41.364 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 9.7081974570000007, 2024-05-08T08:53:41.364 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.083851897999999994, 2024-05-08T08:53:41.364 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:41.365 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:41.365 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:53:41.365 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:41.365 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:41.365 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:41.365 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14420 2024-05-08T08:53:41.365 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.365 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:41.365 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.365 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:41.365 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649639+0000", 2024-05-08T08:53:41.365 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:41.365 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.365 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.366 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:41.366 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649639+0000", 2024-05-08T08:53:41.366 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:41.366 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.366 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.366 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:41.366 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649640+0000", 2024-05-08T08:53:41.366 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6899999999999997e-07 2024-05-08T08:53:41.366 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.366 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.366 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:41.366 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649895+0000", 2024-05-08T08:53:41.366 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00025531000000000002 2024-05-08T08:53:41.366 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.367 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.367 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:41.367 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649905+0000", 2024-05-08T08:53:41.367 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0198e-05 2024-05-08T08:53:41.367 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.367 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.367 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:41.367 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649911+0000", 2024-05-08T08:53:41.367 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2310000000000001e-06 2024-05-08T08:53:41.367 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.367 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.367 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:41.367 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.684099+0000", 2024-05-08T08:53:41.367 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.034188017000000001 2024-05-08T08:53:41.367 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.367 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.368 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:41.368 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.684172+0000", 2024-05-08T08:53:41.368 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3201999999999995e-05 2024-05-08T08:53:41.368 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.368 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.368 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 2", 2024-05-08T08:53:41.368 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.684418+0000", 2024-05-08T08:53:41.368 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00024559099999999997 2024-05-08T08:53:41.368 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.368 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.368 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:53:41.368 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.705538+0000", 2024-05-08T08:53:41.368 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.021119707000000001 2024-05-08T08:53:41.368 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.369 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.369 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:53:41.369 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733399+0000", 2024-05-08T08:53:41.369 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.027861543999999999 2024-05-08T08:53:41.369 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.369 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.369 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:53:41.369 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733445+0000", 2024-05-08T08:53:41.369 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5587000000000002e-05 2024-05-08T08:53:41.369 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.369 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.369 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:41.369 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733491+0000", 2024-05-08T08:53:41.369 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5841999999999999e-05 2024-05-08T08:53:41.369 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:41.370 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:41.370 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:41.370 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.370 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.370 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28547 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889179216] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:41.370 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.355654+0000", 2024-05-08T08:53:41.370 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 9.0021822290000006, 2024-05-08T08:53:41.370 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078692261999999999, 2024-05-08T08:53:41.370 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:41.370 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:41.370 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:41.370 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:41.370 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:41.370 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:41.370 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28547 2024-05-08T08:53:41.371 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.371 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:41.371 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.371 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:41.371 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355654+0000", 2024-05-08T08:53:41.371 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:41.371 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.371 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.371 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:41.371 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355654+0000", 2024-05-08T08:53:41.371 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:41.371 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.371 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.372 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:41.372 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355654+0000", 2024-05-08T08:53:41.372 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3999999999999998e-07 2024-05-08T08:53:41.372 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.372 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.372 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:41.372 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355656+0000", 2024-05-08T08:53:41.372 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.353e-06 2024-05-08T08:53:41.372 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.372 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.372 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:41.372 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355663+0000", 2024-05-08T08:53:41.372 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2259999999999999e-06 2024-05-08T08:53:41.373 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.373 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.373 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:41.373 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355666+0000", 2024-05-08T08:53:41.373 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7930000000000002e-06 2024-05-08T08:53:41.373 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.373 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.373 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:41.373 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434117+0000", 2024-05-08T08:53:41.373 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078451231999999996 2024-05-08T08:53:41.373 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.373 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.373 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:41.373 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434198+0000", 2024-05-08T08:53:41.374 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0879999999999998e-05 2024-05-08T08:53:41.374 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.374 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.374 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:41.374 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434346+0000", 2024-05-08T08:53:41.374 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014853800000000001 2024-05-08T08:53:41.374 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:41.374 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:41.374 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:41.374 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.374 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.374 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28548 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889189696] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:41.374 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.355669+0000", 2024-05-08T08:53:41.374 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 9.0021668790000007, 2024-05-08T08:53:41.374 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078956606999999998, 2024-05-08T08:53:41.375 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:41.375 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:41.375 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:41.375 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:41.375 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:41.375 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:41.375 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28548 2024-05-08T08:53:41.375 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.375 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:41.375 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.375 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:41.375 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355669+0000", 2024-05-08T08:53:41.375 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:41.375 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.376 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.376 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:41.376 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355669+0000", 2024-05-08T08:53:41.376 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:41.376 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.376 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.376 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:41.376 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355762+0000", 2024-05-08T08:53:41.377 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3015000000000003e-05 2024-05-08T08:53:41.377 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.377 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.378 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:41.378 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355765+0000", 2024-05-08T08:53:41.378 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.368e-06 2024-05-08T08:53:41.378 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.378 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.378 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:41.378 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355771+0000", 2024-05-08T08:53:41.378 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5989999999999998e-06 2024-05-08T08:53:41.378 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.378 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.378 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:41.378 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355775+0000", 2024-05-08T08:53:41.378 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7280000000000001e-06 2024-05-08T08:53:41.378 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.379 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.379 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:41.379 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434379+0000", 2024-05-08T08:53:41.379 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078603662000000005 2024-05-08T08:53:41.379 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.379 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.379 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:41.379 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434473+0000", 2024-05-08T08:53:41.379 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4104000000000006e-05 2024-05-08T08:53:41.379 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.379 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.379 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:41.379 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434626+0000", 2024-05-08T08:53:41.379 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00015313100000000001 2024-05-08T08:53:41.379 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:41.380 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:41.380 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:41.380 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.380 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.380 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28549 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889202400] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:41.380 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.355781+0000", 2024-05-08T08:53:41.380 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 9.0020553050000007, 2024-05-08T08:53:41.380 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079117946999999994, 2024-05-08T08:53:41.380 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:41.380 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:41.380 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:41.380 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:41.380 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:41.380 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:41.380 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28549 2024-05-08T08:53:41.381 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.381 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:41.381 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.381 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:41.381 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355781+0000", 2024-05-08T08:53:41.381 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:41.381 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.381 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.381 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:41.381 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355781+0000", 2024-05-08T08:53:41.381 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:41.381 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.381 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.381 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:41.382 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356162+0000", 2024-05-08T08:53:41.382 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00038099100000000001 2024-05-08T08:53:41.382 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.382 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.382 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:41.382 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356165+0000", 2024-05-08T08:53:41.382 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1240000000000001e-06 2024-05-08T08:53:41.382 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.382 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.382 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:41.382 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356173+0000", 2024-05-08T08:53:41.382 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.6199999999999999e-06 2024-05-08T08:53:41.382 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.382 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.382 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:41.383 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356182+0000", 2024-05-08T08:53:41.383 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7760000000000003e-06 2024-05-08T08:53:41.383 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.383 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.383 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:41.383 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434675+0000", 2024-05-08T08:53:41.383 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078493119 2024-05-08T08:53:41.383 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.383 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.383 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:41.383 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434765+0000", 2024-05-08T08:53:41.383 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0402000000000006e-05 2024-05-08T08:53:41.383 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.383 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.383 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:41.383 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434899+0000", 2024-05-08T08:53:41.384 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013391499999999999 2024-05-08T08:53:41.384 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:41.384 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:41.384 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:41.384 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.384 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.384 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28550 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888597280] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:41.384 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:53:41.384 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 9.0016442590000008, 2024-05-08T08:53:41.384 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078933990999999995, 2024-05-08T08:53:41.384 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:41.384 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:41.384 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:41.384 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:41.384 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:41.384 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:41.385 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28550 2024-05-08T08:53:41.385 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.385 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:41.385 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.385 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:41.385 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:53:41.385 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:41.385 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.385 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.385 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:41.385 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:53:41.385 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:41.385 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.385 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.386 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:41.386 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:53:41.386 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.34e-07 2024-05-08T08:53:41.386 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.386 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.386 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:41.386 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356195+0000", 2024-05-08T08:53:41.386 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8710000000000001e-06 2024-05-08T08:53:41.386 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.386 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.386 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:41.386 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356200+0000", 2024-05-08T08:53:41.386 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3000000000000003e-06 2024-05-08T08:53:41.386 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.386 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.387 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:41.387 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356203+0000", 2024-05-08T08:53:41.387 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4039999999999999e-06 2024-05-08T08:53:41.387 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.387 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.387 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:41.387 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434928+0000", 2024-05-08T08:53:41.387 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078725331999999995 2024-05-08T08:53:41.387 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.387 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.387 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:41.387 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435007+0000", 2024-05-08T08:53:41.387 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8415000000000001e-05 2024-05-08T08:53:41.387 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.387 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.388 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:41.388 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435126+0000", 2024-05-08T08:53:41.388 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011923500000000001 2024-05-08T08:53:41.388 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:41.388 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:41.388 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:41.388 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.388 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.388 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28551 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888527200] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:41.388 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.356210+0000", 2024-05-08T08:53:41.388 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 9.0016259109999996, 2024-05-08T08:53:41.388 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079195072000000005, 2024-05-08T08:53:41.388 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:41.388 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:41.389 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:41.389 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:41.389 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:41.389 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:41.389 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28551 2024-05-08T08:53:41.389 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.389 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:41.389 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.389 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:41.389 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356210+0000", 2024-05-08T08:53:41.389 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:41.389 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.389 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.390 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:41.390 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356210+0000", 2024-05-08T08:53:41.390 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:41.390 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.390 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.390 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:41.390 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356211+0000", 2024-05-08T08:53:41.390 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4499999999999998e-07 2024-05-08T08:53:41.390 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.390 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.390 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:41.391 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356212+0000", 2024-05-08T08:53:41.391 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6029999999999999e-06 2024-05-08T08:53:41.391 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.391 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.391 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:41.391 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356221+0000", 2024-05-08T08:53:41.391 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4200000000000007e-06 2024-05-08T08:53:41.391 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.391 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.391 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:41.391 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356224+0000", 2024-05-08T08:53:41.391 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.038e-06 2024-05-08T08:53:41.392 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.392 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.392 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:41.392 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435178+0000", 2024-05-08T08:53:41.392 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078954625000000001 2024-05-08T08:53:41.392 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.392 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.392 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:41.392 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435256+0000", 2024-05-08T08:53:41.392 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7337999999999997e-05 2024-05-08T08:53:41.392 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.392 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.392 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:41.393 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435405+0000", 2024-05-08T08:53:41.393 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000149703 2024-05-08T08:53:41.393 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:41.393 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:41.393 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:41.393 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.393 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.393 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28552 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889231456] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:41.393 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.356228+0000", 2024-05-08T08:53:41.393 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 9.0016082330000007, 2024-05-08T08:53:41.393 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079346920000000001, 2024-05-08T08:53:41.393 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:41.393 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:41.393 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:41.394 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:41.394 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:41.394 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:41.394 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28552 2024-05-08T08:53:41.394 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.394 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:41.394 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.394 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:41.394 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356228+0000", 2024-05-08T08:53:41.394 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:41.394 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.394 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.394 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:41.394 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356228+0000", 2024-05-08T08:53:41.395 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:41.395 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.395 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.395 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:41.395 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356959+0000", 2024-05-08T08:53:41.395 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00073097099999999996 2024-05-08T08:53:41.395 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.395 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.395 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:41.395 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356967+0000", 2024-05-08T08:53:41.395 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0409999999999998e-06 2024-05-08T08:53:41.395 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.395 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.396 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:41.396 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356979+0000", 2024-05-08T08:53:41.396 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1489e-05 2024-05-08T08:53:41.396 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.396 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.396 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:41.396 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356986+0000", 2024-05-08T08:53:41.396 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3699999999999997e-06 2024-05-08T08:53:41.396 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.396 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.396 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:41.396 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435433+0000", 2024-05-08T08:53:41.396 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078447520000000007 2024-05-08T08:53:41.396 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.396 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.397 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:41.397 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435484+0000", 2024-05-08T08:53:41.397 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0229000000000003e-05 2024-05-08T08:53:41.397 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.397 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.397 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:41.397 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435575+0000", 2024-05-08T08:53:41.397 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1299999999999997e-05 2024-05-08T08:53:41.397 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:41.397 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:41.397 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:41.397 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.398 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.398 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28553 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888713648] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:41.398 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:53:41.398 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 9.0008335749999997, 2024-05-08T08:53:41.398 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078782034000000001, 2024-05-08T08:53:41.398 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:41.398 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:41.398 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:41.398 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:41.398 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:41.398 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:41.398 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28553 2024-05-08T08:53:41.398 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.399 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:41.399 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.399 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:41.399 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:53:41.399 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:41.399 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.399 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.399 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:41.399 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:53:41.399 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:41.399 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.399 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.399 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:41.399 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:53:41.400 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6600000000000002e-07 2024-05-08T08:53:41.400 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.400 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.400 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:41.400 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357006+0000", 2024-05-08T08:53:41.400 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7190000000000001e-06 2024-05-08T08:53:41.400 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.400 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.400 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:41.400 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357013+0000", 2024-05-08T08:53:41.400 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.435e-06 2024-05-08T08:53:41.400 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.401 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.401 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:41.401 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357018+0000", 2024-05-08T08:53:41.401 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7260000000000002e-06 2024-05-08T08:53:41.401 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.401 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.401 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:41.401 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435623+0000", 2024-05-08T08:53:41.401 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078604972999999995 2024-05-08T08:53:41.401 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.401 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.401 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:41.401 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435695+0000", 2024-05-08T08:53:41.402 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1959999999999995e-05 2024-05-08T08:53:41.402 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.402 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.402 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:41.402 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435785+0000", 2024-05-08T08:53:41.402 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.9754999999999997e-05 2024-05-08T08:53:41.402 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:41.402 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:41.402 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:41.402 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.402 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.402 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28554 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888980272] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:41.402 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357024+0000", 2024-05-08T08:53:41.402 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 9.0008119390000001, 2024-05-08T08:53:41.402 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078999870999999999, 2024-05-08T08:53:41.403 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:41.403 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:41.403 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:41.403 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:41.403 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:41.403 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:41.403 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28554 2024-05-08T08:53:41.403 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.403 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:41.403 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.403 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:41.403 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357024+0000", 2024-05-08T08:53:41.403 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:41.404 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.404 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.404 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:41.404 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357024+0000", 2024-05-08T08:53:41.404 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:41.404 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.404 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.404 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:41.404 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357025+0000", 2024-05-08T08:53:41.404 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:53:41.404 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.404 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.404 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:41.405 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357027+0000", 2024-05-08T08:53:41.405 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2050000000000001e-06 2024-05-08T08:53:41.405 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.405 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.405 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:41.405 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357033+0000", 2024-05-08T08:53:41.405 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5679999999999996e-06 2024-05-08T08:53:41.405 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.405 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.405 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:41.405 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357038+0000", 2024-05-08T08:53:41.405 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2889999999999999e-06 2024-05-08T08:53:41.405 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.406 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.406 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:41.406 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435823+0000", 2024-05-08T08:53:41.406 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078785677999999998 2024-05-08T08:53:41.406 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.406 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.406 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:41.406 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435892+0000", 2024-05-08T08:53:41.406 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8379000000000004e-05 2024-05-08T08:53:41.406 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.406 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.406 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:41.406 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436024+0000", 2024-05-08T08:53:41.406 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013239799999999999 2024-05-08T08:53:41.406 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:41.407 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:41.407 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:41.407 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.407 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.407 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28555 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889001104] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:41.407 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:53:41.407 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 9.000793174, 2024-05-08T08:53:41.407 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079115742000000003, 2024-05-08T08:53:41.407 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:41.407 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:41.407 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:41.408 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:41.408 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:41.408 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:41.408 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28555 2024-05-08T08:53:41.408 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.408 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:41.408 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.408 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:41.408 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:53:41.408 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:41.408 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.408 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.408 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:41.409 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:53:41.409 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:41.409 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.409 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.409 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:41.409 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:53:41.409 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9200000000000002e-07 2024-05-08T08:53:41.409 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.409 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.409 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:41.409 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357046+0000", 2024-05-08T08:53:41.410 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1670000000000002e-06 2024-05-08T08:53:41.410 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.410 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.410 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:41.410 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357052+0000", 2024-05-08T08:53:41.410 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.139e-06 2024-05-08T08:53:41.410 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.410 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.410 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:41.410 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357056+0000", 2024-05-08T08:53:41.410 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2440000000000004e-06 2024-05-08T08:53:41.410 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.410 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.410 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:41.410 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436059+0000", 2024-05-08T08:53:41.411 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079003070999999994 2024-05-08T08:53:41.411 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.411 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.411 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:41.411 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436097+0000", 2024-05-08T08:53:41.411 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7784000000000003e-05 2024-05-08T08:53:41.411 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.411 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.411 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:41.411 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436159+0000", 2024-05-08T08:53:41.411 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2045000000000007e-05 2024-05-08T08:53:41.411 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:41.411 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:41.411 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:41.411 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.411 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.412 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28556 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889022064] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:41.412 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357065+0000", 2024-05-08T08:53:41.412 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 9.0007711409999995, 2024-05-08T08:53:41.412 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079327976999999994, 2024-05-08T08:53:41.412 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:41.412 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:41.412 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:41.412 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:41.412 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:41.412 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:41.412 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28556 2024-05-08T08:53:41.412 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.412 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:41.412 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.412 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:41.413 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357065+0000", 2024-05-08T08:53:41.413 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:41.413 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.413 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.413 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:41.413 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357065+0000", 2024-05-08T08:53:41.413 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:41.413 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.413 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.413 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:41.413 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358958+0000", 2024-05-08T08:53:41.413 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001892915 2024-05-08T08:53:41.413 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.413 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.413 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:41.414 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358962+0000", 2024-05-08T08:53:41.414 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4599999999999999e-06 2024-05-08T08:53:41.414 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.414 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.414 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:41.414 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358970+0000", 2024-05-08T08:53:41.414 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.6090000000000001e-06 2024-05-08T08:53:41.414 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.414 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.414 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:41.414 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358975+0000", 2024-05-08T08:53:41.414 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7639999999999996e-06 2024-05-08T08:53:41.414 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.414 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.414 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:41.415 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436196+0000", 2024-05-08T08:53:41.415 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.077221564000000006 2024-05-08T08:53:41.415 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.415 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.415 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:41.415 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436269+0000", 2024-05-08T08:53:41.415 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2720999999999997e-05 2024-05-08T08:53:41.415 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.415 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.415 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:41.415 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436393+0000", 2024-05-08T08:53:41.415 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012394399999999999 2024-05-08T08:53:41.415 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:41.416 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:41.416 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:41.416 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.416 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.416 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28563 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888855520] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:41.416 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:53:41.416 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 8.9987617419999992, 2024-05-08T08:53:41.416 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078795566999999997, 2024-05-08T08:53:41.416 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:41.416 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:41.416 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:41.416 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:41.416 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:41.417 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:41.417 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28563 2024-05-08T08:53:41.417 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.417 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:41.417 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.417 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:41.417 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:53:41.417 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:41.417 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.417 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.417 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:41.417 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:53:41.417 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:41.417 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.417 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.418 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:41.418 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:53:41.418 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1e-07 2024-05-08T08:53:41.418 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.418 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.418 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:41.418 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359080+0000", 2024-05-08T08:53:41.418 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8940000000000001e-06 2024-05-08T08:53:41.418 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.418 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.418 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:41.419 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359083+0000", 2024-05-08T08:53:41.419 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1949999999999999e-06 2024-05-08T08:53:41.419 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.419 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.419 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:41.419 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359085+0000", 2024-05-08T08:53:41.419 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4710000000000001e-06 2024-05-08T08:53:41.419 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.419 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.419 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:41.419 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437604+0000", 2024-05-08T08:53:41.419 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078518278999999996 2024-05-08T08:53:41.419 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.419 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.419 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:41.419 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437704+0000", 2024-05-08T08:53:41.420 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010010599999999999 2024-05-08T08:53:41.420 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.420 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.420 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:41.420 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437870+0000", 2024-05-08T08:53:41.420 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00016641200000000001 2024-05-08T08:53:41.420 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:41.420 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:41.420 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:41.420 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.420 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.420 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28564 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888538944] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:41.420 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.359089+0000", 2024-05-08T08:53:41.420 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 8.9987475270000008, 2024-05-08T08:53:41.420 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078952675, 2024-05-08T08:53:41.421 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:41.421 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:41.421 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:41.421 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:41.421 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:41.421 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:41.421 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28564 2024-05-08T08:53:41.421 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.421 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:41.421 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.421 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:41.421 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359089+0000", 2024-05-08T08:53:41.421 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:41.421 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.422 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.422 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:41.422 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359089+0000", 2024-05-08T08:53:41.422 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:41.422 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.422 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.422 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:41.422 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360959+0000", 2024-05-08T08:53:41.422 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0018707089999999999 2024-05-08T08:53:41.422 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.422 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.422 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:41.422 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360963+0000", 2024-05-08T08:53:41.422 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0680000000000001e-06 2024-05-08T08:53:41.422 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.423 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.423 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:41.423 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360969+0000", 2024-05-08T08:53:41.423 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1920000000000002e-06 2024-05-08T08:53:41.423 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.423 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.423 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:41.423 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360973+0000", 2024-05-08T08:53:41.423 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.031e-06 2024-05-08T08:53:41.423 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.423 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.423 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:41.423 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437916+0000", 2024-05-08T08:53:41.423 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.076942939000000002 2024-05-08T08:53:41.423 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.423 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.424 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:41.424 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437973+0000", 2024-05-08T08:53:41.424 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7358999999999997e-05 2024-05-08T08:53:41.424 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.424 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.424 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:41.424 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.438041+0000", 2024-05-08T08:53:41.424 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8376999999999999e-05 2024-05-08T08:53:41.424 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:41.424 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:41.424 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:41.424 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.424 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.424 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28574 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889427600] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:41.424 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:53:41.424 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 8.9967430200000003, 2024-05-08T08:53:41.425 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078782476000000004, 2024-05-08T08:53:41.425 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:41.425 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:41.425 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:41.425 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:41.425 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:41.425 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:41.425 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28574 2024-05-08T08:53:41.425 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.425 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:41.425 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.425 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:41.425 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:53:41.425 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:41.426 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.426 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.426 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:41.426 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:53:41.426 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:41.426 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.426 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.426 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:41.426 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:53:41.426 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1799999999999999e-07 2024-05-08T08:53:41.426 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.426 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.426 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:41.426 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361095+0000", 2024-05-08T08:53:41.427 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3850000000000001e-06 2024-05-08T08:53:41.427 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.427 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.427 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:41.427 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361098+0000", 2024-05-08T08:53:41.427 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7989999999999998e-06 2024-05-08T08:53:41.427 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.427 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.427 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:41.427 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361100+0000", 2024-05-08T08:53:41.427 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4669999999999998e-06 2024-05-08T08:53:41.427 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.427 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.427 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:41.427 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439768+0000", 2024-05-08T08:53:41.427 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078668057999999999 2024-05-08T08:53:41.428 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.428 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.428 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:41.428 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439811+0000", 2024-05-08T08:53:41.428 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2459000000000001e-05 2024-05-08T08:53:41.428 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.428 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.428 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:41.428 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439876+0000", 2024-05-08T08:53:41.428 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5090000000000002e-05 2024-05-08T08:53:41.428 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:41.428 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:41.428 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:41.428 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.428 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.428 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28575 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889439216] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:41.429 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361103+0000", 2024-05-08T08:53:41.429 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 8.9967330049999994, 2024-05-08T08:53:41.429 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078966828000000003, 2024-05-08T08:53:41.429 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:41.429 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:41.429 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:41.429 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:41.429 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:41.429 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:41.429 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28575 2024-05-08T08:53:41.429 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.429 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:41.429 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.429 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:41.429 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361103+0000", 2024-05-08T08:53:41.430 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:41.430 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.430 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.430 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:41.430 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361103+0000", 2024-05-08T08:53:41.430 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:41.430 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.430 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.430 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:41.430 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361104+0000", 2024-05-08T08:53:41.430 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1400000000000001e-07 2024-05-08T08:53:41.430 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.430 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.430 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:41.431 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361105+0000", 2024-05-08T08:53:41.431 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5540000000000001e-06 2024-05-08T08:53:41.431 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.431 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.431 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:41.431 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361108+0000", 2024-05-08T08:53:41.431 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.841e-06 2024-05-08T08:53:41.431 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.431 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.431 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:41.431 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361110+0000", 2024-05-08T08:53:41.431 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.497e-06 2024-05-08T08:53:41.431 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.431 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.432 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:41.432 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439916+0000", 2024-05-08T08:53:41.432 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078805372999999998 2024-05-08T08:53:41.432 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.432 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.432 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:41.432 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440000+0000", 2024-05-08T08:53:41.432 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4730999999999999e-05 2024-05-08T08:53:41.432 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.432 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.432 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:41.432 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440070+0000", 2024-05-08T08:53:41.432 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9617999999999997e-05 2024-05-08T08:53:41.432 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:41.432 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:41.433 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:41.433 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.433 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.433 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28576 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889451328] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:41.433 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361113+0000", 2024-05-08T08:53:41.433 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 8.9967228190000004, 2024-05-08T08:53:41.433 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079088862999999995, 2024-05-08T08:53:41.433 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:41.433 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:41.433 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:41.433 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:41.433 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:41.433 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:41.433 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28576 2024-05-08T08:53:41.433 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.434 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:41.434 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.434 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:41.434 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361113+0000", 2024-05-08T08:53:41.434 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:41.434 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.434 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.434 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:41.434 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361113+0000", 2024-05-08T08:53:41.434 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:41.434 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.434 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.434 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:41.434 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361114+0000", 2024-05-08T08:53:41.435 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1299999999999999e-07 2024-05-08T08:53:41.435 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.435 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.435 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:41.435 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361115+0000", 2024-05-08T08:53:41.435 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.4300000000000001e-06 2024-05-08T08:53:41.435 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.435 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.435 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:41.435 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361118+0000", 2024-05-08T08:53:41.435 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9000000000000002e-06 2024-05-08T08:53:41.435 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.435 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.436 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:41.436 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361120+0000", 2024-05-08T08:53:41.436 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4739999999999999e-06 2024-05-08T08:53:41.436 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.436 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.436 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:41.436 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440096+0000", 2024-05-08T08:53:41.436 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078975629000000006 2024-05-08T08:53:41.436 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.436 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.436 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:41.436 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440135+0000", 2024-05-08T08:53:41.436 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9202000000000002e-05 2024-05-08T08:53:41.436 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.437 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.437 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:41.437 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440202+0000", 2024-05-08T08:53:41.437 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7014999999999994e-05 2024-05-08T08:53:41.437 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:41.437 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:41.437 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:41.437 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.437 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.437 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28577 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889462640] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:41.437 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:53:41.437 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 8.9967126369999999, 2024-05-08T08:53:41.437 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079194622000000006, 2024-05-08T08:53:41.437 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:41.437 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:41.438 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:41.438 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:41.438 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:41.438 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:41.438 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28577 2024-05-08T08:53:41.438 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.438 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:41.438 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.438 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:41.439 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:53:41.439 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:41.439 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.439 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.439 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:41.439 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:53:41.439 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:41.439 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.439 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.439 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:41.439 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:53:41.440 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1400000000000001e-07 2024-05-08T08:53:41.440 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.440 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.440 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:41.440 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361125+0000", 2024-05-08T08:53:41.440 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.372e-06 2024-05-08T08:53:41.440 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.440 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.440 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:41.440 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361128+0000", 2024-05-08T08:53:41.440 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0309999999999999e-06 2024-05-08T08:53:41.440 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.440 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.440 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:41.440 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361131+0000", 2024-05-08T08:53:41.441 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4040000000000002e-06 2024-05-08T08:53:41.441 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.441 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.441 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:41.441 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440225+0000", 2024-05-08T08:53:41.441 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079094600000000001 2024-05-08T08:53:41.441 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.441 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.441 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:41.441 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440260+0000", 2024-05-08T08:53:41.441 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4703000000000003e-05 2024-05-08T08:53:41.441 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.441 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.441 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:41.441 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440318+0000", 2024-05-08T08:53:41.442 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8298000000000003e-05 2024-05-08T08:53:41.442 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:41.442 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:41.442 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:41.442 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.442 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.442 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28578 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889474192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:41.442 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361134+0000", 2024-05-08T08:53:41.442 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 8.9967022120000006, 2024-05-08T08:53:41.442 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079310295000000003, 2024-05-08T08:53:41.442 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:41.442 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:41.443 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:41.443 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:41.443 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:41.443 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:41.443 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28578 2024-05-08T08:53:41.443 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.443 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:41.443 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.443 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:41.443 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361134+0000", 2024-05-08T08:53:41.443 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:41.444 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.444 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.444 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:41.444 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361134+0000", 2024-05-08T08:53:41.444 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:41.444 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.444 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.444 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:41.444 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.362958+0000", 2024-05-08T08:53:41.444 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0018237380000000001 2024-05-08T08:53:41.444 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.444 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.444 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:41.444 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.362961+0000", 2024-05-08T08:53:41.444 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.123e-06 2024-05-08T08:53:41.445 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.445 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.445 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:41.445 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.397404+0000", 2024-05-08T08:53:41.445 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.034443226 2024-05-08T08:53:41.445 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.445 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.445 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:41.445 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.397439+0000", 2024-05-08T08:53:41.445 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4690000000000002e-05 2024-05-08T08:53:41.445 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.445 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.445 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:41.445 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440340+0000", 2024-05-08T08:53:41.445 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.042901419000000003 2024-05-08T08:53:41.446 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.446 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.446 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:41.446 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440376+0000", 2024-05-08T08:53:41.446 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5444999999999997e-05 2024-05-08T08:53:41.446 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.446 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.446 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:41.446 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440444+0000", 2024-05-08T08:53:41.446 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8653999999999997e-05 2024-05-08T08:53:41.446 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:41.446 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:41.446 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:41.446 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:41.446 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:41.447 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:41.448 INFO:teuthology.orchestra.run.smithi196.stdout:3591 write 0xe77dc00 thru 0xe783943 (0x5d44 bytes) 2024-05-08T08:53:41.448 INFO:teuthology.orchestra.run.smithi196.stdout:3593 trunc from 0xe783a00 to 0xbf15200 2024-05-08T08:53:41.449 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:53:41.450 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-08T08:53:41.450 INFO:teuthology.orchestra.run.smithi059.stderr:no valid command found; 10 closest matches: 2024-05-08T08:53:41.451 INFO:teuthology.orchestra.run.smithi059.stderr:0 2024-05-08T08:53:41.451 INFO:teuthology.orchestra.run.smithi059.stderr:1 2024-05-08T08:53:41.451 INFO:teuthology.orchestra.run.smithi059.stderr:2 2024-05-08T08:53:41.451 INFO:teuthology.orchestra.run.smithi059.stderr:abort 2024-05-08T08:53:41.451 INFO:teuthology.orchestra.run.smithi059.stderr:assert 2024-05-08T08:53:41.451 INFO:teuthology.orchestra.run.smithi059.stderr:bench [] [] [] [] 2024-05-08T08:53:41.451 INFO:teuthology.orchestra.run.smithi059.stderr:bluefs debug_inject_read_zeros 2024-05-08T08:53:41.451 INFO:teuthology.orchestra.run.smithi059.stderr:bluefs files list 2024-05-08T08:53:41.451 INFO:teuthology.orchestra.run.smithi059.stderr:bluefs stats 2024-05-08T08:53:41.451 INFO:teuthology.orchestra.run.smithi059.stderr:bluestore allocator dump block 2024-05-08T08:53:41.451 INFO:teuthology.orchestra.run.smithi059.stderr:Error EINVAL: invalid command 2024-05-08T08:53:41.452 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-05-08T08:53:41.452 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-05-08T08:53:41.452 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:53:41.451+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:53:41.461 INFO:teuthology.orchestra.run.smithi196.stdout:3594 punch from 0x3e81a00 to 0x3e8e000, (0xc600 bytes) 2024-05-08T08:53:41.467 INFO:teuthology.orchestra.run.smithi196.stdout:3595 read 0x4cb8e00 thru 0x4cc5f97 (0xd198 bytes) 2024-05-08T08:53:41.468 INFO:teuthology.orchestra.run.smithi196.stdout:3596 write 0x2f9fe00 thru 0x2fabf69 (0xc16a bytes) 2024-05-08T08:53:41.469 INFO:teuthology.orchestra.run.smithi196.stdout:3597 trunc from 0xbf15200 to 0xd19fe00 2024-05-08T08:53:41.491 INFO:teuthology.orchestra.run.smithi196.stdout:3598 write 0x5894400 thru 0x589f86d (0xb46e bytes) 2024-05-08T08:53:41.497 INFO:teuthology.orchestra.run.smithi196.stdout:3599 trunc from 0xd19fe00 to 0x8585600 2024-05-08T08:53:41.548 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:41.551+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:41.555 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-05-08T08:53:41.574 INFO:teuthology.orchestra.run.smithi196.stdout:3600 write 0x5d35e00 thru 0x5d3618f (0x390 bytes) 2024-05-08T08:53:41.576 INFO:teuthology.orchestra.run.smithi196.stdout:3602 trunc from 0x8585600 to 0x795dc00 2024-05-08T08:53:41.610 INFO:teuthology.orchestra.run.smithi196.stdout:3606 write 0x3297400 thru 0x32a0f76 (0x9b77 bytes) 2024-05-08T08:53:41.611 INFO:teuthology.orchestra.run.smithi196.stdout:3607 read 0x7419800 thru 0x741d634 (0x3e35 bytes) 2024-05-08T08:53:41.611 INFO:teuthology.orchestra.run.smithi196.stdout:3609 write 0x4f91400 thru 0x4f94282 (0x2e83 bytes) 2024-05-08T08:53:41.611 INFO:teuthology.orchestra.run.smithi196.stdout:3610 read 0x2d6800 thru 0x2e1e8c (0xb68d bytes) 2024-05-08T08:53:41.614 INFO:teuthology.orchestra.run.smithi196.stdout:3614 write 0x5081c00 thru 0x508aa30 (0x8e31 bytes) 2024-05-08T08:53:41.615 INFO:teuthology.orchestra.run.smithi196.stdout:3615 read 0x28cdc00 thru 0x28dd767 (0xfb68 bytes) 2024-05-08T08:53:41.616 INFO:teuthology.orchestra.run.smithi196.stdout:3616 write 0x404e400 thru 0x405869e (0xa29f bytes) 2024-05-08T08:53:41.616 INFO:teuthology.orchestra.run.smithi196.stdout:3618 write 0x6948800 thru 0x6951e56 (0x9657 bytes) 2024-05-08T08:53:41.616 INFO:teuthology.orchestra.run.smithi196.stdout:3619 punch from 0x1cacc00 to 0x1cb4600, (0x7a00 bytes) 2024-05-08T08:53:41.621 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:41.621 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:53:41.621 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:53:41.621 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:41.624 INFO:teuthology.orchestra.run.smithi196.stdout:3626 trunc from 0x795dc00 to 0xd7c3e00 2024-05-08T08:53:41.636 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:53:41.650 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:41.653+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:41.654 INFO:teuthology.orchestra.run.smithi196.stdout:3627 read 0x3bb1c00 thru 0x3bc05d4 (0xe9d5 bytes) 2024-05-08T08:53:41.667 INFO:teuthology.orchestra.run.smithi059.stderr:no valid command found; 10 closest matches: 2024-05-08T08:53:41.667 INFO:teuthology.orchestra.run.smithi059.stderr:0 2024-05-08T08:53:41.667 INFO:teuthology.orchestra.run.smithi059.stderr:1 2024-05-08T08:53:41.667 INFO:teuthology.orchestra.run.smithi059.stderr:2 2024-05-08T08:53:41.667 INFO:teuthology.orchestra.run.smithi059.stderr:abort 2024-05-08T08:53:41.667 INFO:teuthology.orchestra.run.smithi059.stderr:assert 2024-05-08T08:53:41.667 INFO:teuthology.orchestra.run.smithi059.stderr:bench [] [] [] [] 2024-05-08T08:53:41.667 INFO:teuthology.orchestra.run.smithi059.stderr:bluefs debug_inject_read_zeros 2024-05-08T08:53:41.667 INFO:teuthology.orchestra.run.smithi059.stderr:bluefs files list 2024-05-08T08:53:41.667 INFO:teuthology.orchestra.run.smithi059.stderr:bluefs stats 2024-05-08T08:53:41.667 INFO:teuthology.orchestra.run.smithi059.stderr:bluestore allocator dump block 2024-05-08T08:53:41.668 INFO:teuthology.orchestra.run.smithi059.stderr:Error EINVAL: invalid command 2024-05-08T08:53:41.671 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-08T08:53:41.671 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-05-08T08:53:41.671 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-05-08T08:53:41.672 INFO:teuthology.orchestra.run.smithi196.stdout:3628 trunc from 0xd7c3e00 to 0x5ff400 2024-05-08T08:53:41.705 INFO:teuthology.orchestra.run.smithi196.stdout:3631 read 0xa0a00 thru 0xa64f1 (0x5af2 bytes) 2024-05-08T08:53:41.709 INFO:teuthology.orchestra.run.smithi196.stdout:3633 trunc from 0x5ff400 to 0x29e5800 2024-05-08T08:53:41.718 INFO:teuthology.orchestra.run.smithi196.stdout:3634 punch from 0xe85a00 to 0xe92a00, (0xd000 bytes) 2024-05-08T08:53:41.743 INFO:teuthology.orchestra.run.smithi196.stdout:3635 write 0xd0a9e00 thru 0xd0b5da9 (0xbfaa bytes) 2024-05-08T08:53:41.747 INFO:teuthology.orchestra.run.smithi196.stdout:3636 trunc from 0xd0b5e00 to 0xb1d8e00 2024-05-08T08:53:41.751 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:41.754+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:41.769 INFO:teuthology.orchestra.run.smithi196.stdout:3637 write 0x6143600 thru 0x6147fd6 (0x49d7 bytes) 2024-05-08T08:53:41.770 INFO:teuthology.orchestra.run.smithi196.stdout:3639 read 0x735e00 thru 0x74334d (0xd54e bytes) 2024-05-08T08:53:41.770 INFO:teuthology.orchestra.run.smithi196.stdout:3641 write 0x91bfa00 thru 0x91cbc9f (0xc2a0 bytes) 2024-05-08T08:53:41.770 INFO:teuthology.orchestra.run.smithi196.stdout:3643 read 0xabd6600 thru 0xabe07a4 (0xa1a5 bytes) 2024-05-08T08:53:41.770 INFO:teuthology.orchestra.run.smithi196.stdout:3645 punch from 0x8198a00 to 0x81a0c00, (0x8200 bytes) 2024-05-08T08:53:41.772 INFO:teuthology.orchestra.run.smithi196.stdout:3646 read 0x1483000 thru 0x148c553 (0x9554 bytes) 2024-05-08T08:53:41.772 INFO:teuthology.orchestra.run.smithi196.stdout:3648 write 0x5f61e00 thru 0x5f662c4 (0x44c5 bytes) 2024-05-08T08:53:41.776 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:41.776 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:53:41.776 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:53:41.776 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:53:41.777 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:41.783 INFO:teuthology.orchestra.run.smithi196.stdout:3649 trunc from 0xb1d8e00 to 0x185d800 2024-05-08T08:53:41.790 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:53:41.822 INFO:teuthology.orchestra.run.smithi196.stdout:3651 write 0x4f7ca00 thru 0x4f8c92d (0xff2e bytes) 2024-05-08T08:53:41.822 INFO:teuthology.orchestra.run.smithi196.stdout:3652 read 0x3903e00 thru 0x3906d8b (0x2f8c bytes) 2024-05-08T08:53:41.822 INFO:teuthology.orchestra.run.smithi196.stdout:3653 punch from 0x2bb800 to 0x2cb600, (0xfe00 bytes) 2024-05-08T08:53:41.828 INFO:teuthology.orchestra.run.smithi196.stdout:3655 write 0x3020200 thru 0x302857c (0x837d bytes) 2024-05-08T08:53:41.828 INFO:teuthology.orchestra.run.smithi196.stdout:3656 read 0x2fffc00 thru 0x3008bd0 (0x8fd1 bytes) 2024-05-08T08:53:41.828 INFO:teuthology.orchestra.run.smithi196.stdout:3657 read 0x4f5ac00 thru 0x4f5fd73 (0x5174 bytes) 2024-05-08T08:53:41.828 INFO:teuthology.orchestra.run.smithi196.stdout:3658 read 0x3b19e00 thru 0x3b1c4a0 (0x26a1 bytes) 2024-05-08T08:53:41.828 INFO:teuthology.orchestra.run.smithi196.stdout:3659 read 0x6be400 thru 0x6c796f (0x9570 bytes) 2024-05-08T08:53:41.828 INFO:teuthology.orchestra.run.smithi196.stdout:3660 read 0x2a07e00 thru 0x2a09bdb (0x1ddc bytes) 2024-05-08T08:53:41.852 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:41.855+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:41.870 INFO:teuthology.orchestra.run.smithi196.stdout:3661 write 0xb3ec200 thru 0xb3eec7c (0x2a7d bytes) 2024-05-08T08:53:41.870 INFO:teuthology.orchestra.run.smithi196.stdout:3662 write 0x2020200 thru 0x2021041 (0xe42 bytes) 2024-05-08T08:53:41.870 INFO:teuthology.orchestra.run.smithi196.stdout:3663 punch from 0xdcd600 to 0xdd0200, (0x2c00 bytes) 2024-05-08T08:53:41.870 INFO:teuthology.orchestra.run.smithi196.stdout:3664 write 0x67db000 thru 0x67e059e (0x559f bytes) 2024-05-08T08:53:41.870 INFO:teuthology.orchestra.run.smithi196.stdout:3665 read 0x6ec0c00 thru 0x6ecd8a6 (0xcca7 bytes) 2024-05-08T08:53:41.870 INFO:teuthology.orchestra.run.smithi196.stdout:3668 read 0x5005200 thru 0x500607b (0xe7c bytes) 2024-05-08T08:53:41.870 INFO:teuthology.orchestra.run.smithi196.stdout:3669 read 0x7c6e000 thru 0x7c7db86 (0xfb87 bytes) 2024-05-08T08:53:41.871 INFO:teuthology.orchestra.run.smithi196.stdout:3672 read 0x90f2600 thru 0x90f9338 (0x6d39 bytes) 2024-05-08T08:53:41.871 INFO:teuthology.orchestra.run.smithi196.stdout:3675 write 0x50a1600 thru 0x50b0529 (0xef2a bytes) 2024-05-08T08:53:41.871 INFO:teuthology.orchestra.run.smithi196.stdout:3678 punch from 0x84fe200 to 0x8508c00, (0xaa00 bytes) 2024-05-08T08:53:41.881 INFO:teuthology.orchestra.run.smithi196.stdout:3680 trunc from 0xb3eee00 to 0x20b4800 2024-05-08T08:53:41.903 INFO:teuthology.orchestra.run.smithi059.stderr:no valid command found; 10 closest matches: 2024-05-08T08:53:41.903 INFO:teuthology.orchestra.run.smithi059.stderr:0 2024-05-08T08:53:41.904 INFO:teuthology.orchestra.run.smithi059.stderr:1 2024-05-08T08:53:41.904 INFO:teuthology.orchestra.run.smithi059.stderr:2 2024-05-08T08:53:41.904 INFO:teuthology.orchestra.run.smithi059.stderr:abort 2024-05-08T08:53:41.904 INFO:teuthology.orchestra.run.smithi059.stderr:assert 2024-05-08T08:53:41.904 INFO:teuthology.orchestra.run.smithi059.stderr:bench [] [] [] [] 2024-05-08T08:53:41.904 INFO:teuthology.orchestra.run.smithi059.stderr:bluefs debug_inject_read_zeros 2024-05-08T08:53:41.904 INFO:teuthology.orchestra.run.smithi059.stderr:bluefs files list 2024-05-08T08:53:41.904 INFO:teuthology.orchestra.run.smithi059.stderr:bluefs stats 2024-05-08T08:53:41.904 INFO:teuthology.orchestra.run.smithi059.stderr:bluestore allocator dump block 2024-05-08T08:53:41.904 INFO:teuthology.orchestra.run.smithi059.stderr:Error EINVAL: invalid command 2024-05-08T08:53:41.907 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-08T08:53:41.907 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-05-08T08:53:41.907 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 'trim stale osdmaps' 2024-05-08T08:53:41.920 INFO:teuthology.orchestra.run.smithi196.stdout:3681 trunc from 0x20b4800 to 0x70d4e00 2024-05-08T08:53:41.929 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:41.929 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:53:41.929 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:53:41.929 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:53:41.929 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.929 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:437 3.17 3:ea93a512:::rbd_data.1140100c9703.0000000000000077:head [set-alloc-hint object_size 2097152 write_size 2097152,write 1170432~1024] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e63)", 2024-05-08T08:53:41.929 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:08.390804+0000", 2024-05-08T08:53:41.929 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 33.540080226000001, 2024-05-08T08:53:41.929 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0406068690000001, 2024-05-08T08:53:41.929 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:41.929 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:41.929 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:53:41.929 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:41.930 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:41.930 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:41.930 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 437 2024-05-08T08:53:41.930 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.930 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:41.930 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.930 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:41.930 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390804+0000", 2024-05-08T08:53:41.930 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:41.930 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.930 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.930 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:41.930 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390804+0000", 2024-05-08T08:53:41.931 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:41.931 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.931 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.931 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:41.931 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390806+0000", 2024-05-08T08:53:41.931 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.666e-06 2024-05-08T08:53:41.931 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.931 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.931 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:41.931 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390818+0000", 2024-05-08T08:53:41.931 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2187999999999999e-05 2024-05-08T08:53:41.931 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.931 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.931 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:41.931 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390834+0000", 2024-05-08T08:53:41.931 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6191000000000001e-05 2024-05-08T08:53:41.931 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.932 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.932 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:41.932 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390845+0000", 2024-05-08T08:53:41.932 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0831e-05 2024-05-08T08:53:41.932 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.932 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.932 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:41.932 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390922+0000", 2024-05-08T08:53:41.932 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7253e-05 2024-05-08T08:53:41.932 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.932 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.932 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op must wait for map", 2024-05-08T08:53:41.932 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390930+0000", 2024-05-08T08:53:41.932 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8259999999999993e-06 2024-05-08T08:53:41.932 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.933 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.933 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:41.933 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392155+0000", 2024-05-08T08:53:41.933 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0012256649999999999 2024-05-08T08:53:41.933 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.933 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.933 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:53:41.933 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392165+0000", 2024-05-08T08:53:41.933 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5799999999999998e-06 2024-05-08T08:53:41.933 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.933 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.933 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:41.933 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.429596+0000", 2024-05-08T08:53:41.933 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0374313639999999 2024-05-08T08:53:41.933 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.933 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.934 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:41.934 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.429728+0000", 2024-05-08T08:53:41.934 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000131552 2024-05-08T08:53:41.934 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.934 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.934 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 0,1", 2024-05-08T08:53:41.934 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.429860+0000", 2024-05-08T08:53:41.934 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000132137 2024-05-08T08:53:41.934 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.934 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.934 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:53:41.934 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.430811+0000", 2024-05-08T08:53:41.934 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00095040200000000004 2024-05-08T08:53:41.934 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.935 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.935 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:53:41.935 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431126+0000", 2024-05-08T08:53:41.935 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000315006 2024-05-08T08:53:41.935 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.935 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.935 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:53:41.935 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431272+0000", 2024-05-08T08:53:41.935 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000146487 2024-05-08T08:53:41.935 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.936 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.936 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:53:41.936 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431315+0000", 2024-05-08T08:53:41.936 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2697000000000001e-05 2024-05-08T08:53:41.936 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.936 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.936 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:41.936 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431411+0000", 2024-05-08T08:53:41.936 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6024000000000001e-05 2024-05-08T08:53:41.936 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:41.936 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:41.936 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:41.937 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.937 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.937 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:439 3.17 3:ea93a512:::rbd_data.1140100c9703.0000000000000077:head [read 1163264~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-05-08T08:53:41.937 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:08.390871+0000", 2024-05-08T08:53:41.937 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 33.540012715000003, 2024-05-08T08:53:41.937 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0408790590000001, 2024-05-08T08:53:41.937 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:41.937 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:41.937 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:41.937 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:41.937 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:41.937 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:41.937 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 439 2024-05-08T08:53:41.937 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.937 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:41.938 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.938 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:41.938 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390871+0000", 2024-05-08T08:53:41.938 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:41.938 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.938 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.938 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:41.938 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390871+0000", 2024-05-08T08:53:41.938 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:41.938 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.938 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.938 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:41.938 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390872+0000", 2024-05-08T08:53:41.938 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6300000000000005e-07 2024-05-08T08:53:41.938 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.938 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.939 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:41.939 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390875+0000", 2024-05-08T08:53:41.939 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3459999999999998e-06 2024-05-08T08:53:41.939 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.939 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.939 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:41.939 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390891+0000", 2024-05-08T08:53:41.939 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5255e-05 2024-05-08T08:53:41.939 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.939 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.939 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:41.939 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390897+0000", 2024-05-08T08:53:41.939 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2360000000000001e-06 2024-05-08T08:53:41.939 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.940 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.940 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:41.940 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390995+0000", 2024-05-08T08:53:41.940 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8522000000000003e-05 2024-05-08T08:53:41.940 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.940 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.940 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting_for_map not empty", 2024-05-08T08:53:41.940 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.391005+0000", 2024-05-08T08:53:41.940 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1460000000000002e-06 2024-05-08T08:53:41.940 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.940 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.940 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:41.940 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392191+0000", 2024-05-08T08:53:41.940 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0011860270000000001 2024-05-08T08:53:41.940 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.940 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.941 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:53:41.941 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392197+0000", 2024-05-08T08:53:41.941 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4289999999999998e-06 2024-05-08T08:53:41.941 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.941 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.941 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:41.941 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.430596+0000", 2024-05-08T08:53:41.941 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0383989090000001 2024-05-08T08:53:41.941 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.941 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.941 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for rw locks", 2024-05-08T08:53:41.941 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.430676+0000", 2024-05-08T08:53:41.941 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9781000000000001e-05 2024-05-08T08:53:41.941 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.941 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.941 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:41.942 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431460+0000", 2024-05-08T08:53:41.942 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00078427300000000004 2024-05-08T08:53:41.942 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.942 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.942 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:41.942 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431540+0000", 2024-05-08T08:53:41.942 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0224999999999998e-05 2024-05-08T08:53:41.942 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.942 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.942 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:41.942 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431750+0000", 2024-05-08T08:53:41.942 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00021024699999999999 2024-05-08T08:53:41.943 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:41.943 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:41.943 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:41.943 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.943 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.943 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:945 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 4096000~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:41.943 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:53:41.943 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 32.479863148, 2024-05-08T08:53:41.943 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086774741000000002, 2024-05-08T08:53:41.943 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:41.943 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:41.943 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:41.943 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:41.943 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:41.943 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:41.944 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 945 2024-05-08T08:53:41.944 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.944 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:41.944 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.944 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:41.944 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:53:41.944 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:41.944 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.944 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.944 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:41.944 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:53:41.944 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:41.944 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.944 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.944 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:41.944 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:53:41.944 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3499999999999996e-07 2024-05-08T08:53:41.945 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.945 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.945 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:41.945 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451024+0000", 2024-05-08T08:53:41.945 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.627e-06 2024-05-08T08:53:41.945 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.945 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.945 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:41.945 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451037+0000", 2024-05-08T08:53:41.945 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2431e-05 2024-05-08T08:53:41.945 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.945 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.945 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:41.945 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451042+0000", 2024-05-08T08:53:41.945 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8409999999999996e-06 2024-05-08T08:53:41.946 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.946 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.946 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:41.946 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537608+0000", 2024-05-08T08:53:41.946 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086565243 2024-05-08T08:53:41.946 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.946 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.946 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:41.946 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537687+0000", 2024-05-08T08:53:41.946 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9519999999999998e-05 2024-05-08T08:53:41.946 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.946 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.946 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:41.946 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537796+0000", 2024-05-08T08:53:41.946 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000108544 2024-05-08T08:53:41.946 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:41.947 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:41.947 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:41.947 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.947 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.947 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:964 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 458752~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:41.947 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451051+0000", 2024-05-08T08:53:41.947 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 32.479832975000001, 2024-05-08T08:53:41.947 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086990490000000004, 2024-05-08T08:53:41.947 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:41.947 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:41.947 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:41.947 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:41.947 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:41.947 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:41.947 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 964 2024-05-08T08:53:41.947 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.948 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:41.948 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.948 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:41.948 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451051+0000", 2024-05-08T08:53:41.948 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:41.948 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.948 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.948 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:41.948 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451051+0000", 2024-05-08T08:53:41.948 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:41.948 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.948 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.948 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:41.948 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451052+0000", 2024-05-08T08:53:41.948 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7299999999999996e-07 2024-05-08T08:53:41.948 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.949 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.949 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:41.949 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451055+0000", 2024-05-08T08:53:41.949 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.2770000000000001e-06 2024-05-08T08:53:41.949 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.949 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.949 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:41.949 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451061+0000", 2024-05-08T08:53:41.949 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4060000000000002e-06 2024-05-08T08:53:41.949 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.949 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.949 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:41.949 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451066+0000", 2024-05-08T08:53:41.949 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0839999999999999e-06 2024-05-08T08:53:41.949 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.950 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.950 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:41.950 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537839+0000", 2024-05-08T08:53:41.950 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086772962999999995 2024-05-08T08:53:41.950 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.950 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.950 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:41.950 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537934+0000", 2024-05-08T08:53:41.950 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4742999999999997e-05 2024-05-08T08:53:41.950 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.950 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.950 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:41.950 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538042+0000", 2024-05-08T08:53:41.950 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000107444 2024-05-08T08:53:41.950 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:41.951 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:41.951 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:41.951 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.951 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.951 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:965 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 950272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:41.951 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451075+0000", 2024-05-08T08:53:41.951 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 32.479808912999999, 2024-05-08T08:53:41.951 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087230196999999995, 2024-05-08T08:53:41.951 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:41.951 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:41.951 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:41.951 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:41.951 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:41.951 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:41.952 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 965 2024-05-08T08:53:41.952 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.952 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:41.952 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.952 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:41.952 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451075+0000", 2024-05-08T08:53:41.952 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:41.952 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.952 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.952 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:41.952 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451075+0000", 2024-05-08T08:53:41.952 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:41.952 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.952 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.952 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:41.952 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451076+0000", 2024-05-08T08:53:41.952 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5000000000000002e-07 2024-05-08T08:53:41.953 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.953 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.953 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:41.953 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451078+0000", 2024-05-08T08:53:41.953 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.5639999999999999e-06 2024-05-08T08:53:41.953 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.953 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.953 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:41.953 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451085+0000", 2024-05-08T08:53:41.953 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5640000000000002e-06 2024-05-08T08:53:41.953 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.953 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.953 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:41.953 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451090+0000", 2024-05-08T08:53:41.953 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1699999999999996e-06 2024-05-08T08:53:41.954 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.954 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.954 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:41.954 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538106+0000", 2024-05-08T08:53:41.954 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087015888999999999 2024-05-08T08:53:41.954 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.954 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.954 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:41.954 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538190+0000", 2024-05-08T08:53:41.954 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3961999999999993e-05 2024-05-08T08:53:41.954 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.954 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.954 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:41.954 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538305+0000", 2024-05-08T08:53:41.955 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011529800000000001 2024-05-08T08:53:41.955 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:41.955 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:41.955 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:41.955 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.955 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.955 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:966 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 1441792~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:41.955 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:53:41.955 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 32.479785540000002, 2024-05-08T08:53:41.955 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087446323000000006, 2024-05-08T08:53:41.955 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:41.955 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:41.955 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:41.955 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:41.956 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:41.956 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:41.956 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 966 2024-05-08T08:53:41.956 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.956 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:41.956 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.956 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:41.956 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:53:41.956 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:41.956 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.956 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.956 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:41.956 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:53:41.956 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:41.957 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.957 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.957 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:41.957 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:53:41.957 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3700000000000001e-07 2024-05-08T08:53:41.957 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.957 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.957 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:41.957 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451101+0000", 2024-05-08T08:53:41.957 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2560000000000001e-06 2024-05-08T08:53:41.957 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.957 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.957 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:41.958 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451107+0000", 2024-05-08T08:53:41.958 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1070000000000001e-06 2024-05-08T08:53:41.958 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.958 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.958 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:41.958 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451112+0000", 2024-05-08T08:53:41.958 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1420000000000001e-06 2024-05-08T08:53:41.958 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.958 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.958 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:41.958 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538341+0000", 2024-05-08T08:53:41.958 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087228302999999993 2024-05-08T08:53:41.958 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.959 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.959 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:41.959 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538424+0000", 2024-05-08T08:53:41.959 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3497000000000004e-05 2024-05-08T08:53:41.959 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.959 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.959 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:41.959 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538545+0000", 2024-05-08T08:53:41.959 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000120581 2024-05-08T08:53:41.959 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:41.959 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:41.959 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:41.959 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.959 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.959 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:967 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:41.959 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451120+0000", 2024-05-08T08:53:41.960 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 32.479764064999998, 2024-05-08T08:53:41.960 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087632725999999994, 2024-05-08T08:53:41.960 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:41.960 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:41.960 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:41.960 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:41.960 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:41.960 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:41.960 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 967 2024-05-08T08:53:41.960 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.960 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:41.960 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.960 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:41.960 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451120+0000", 2024-05-08T08:53:41.960 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:41.961 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.961 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.961 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:41.961 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451120+0000", 2024-05-08T08:53:41.961 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:41.961 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.961 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.961 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:41.961 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451121+0000", 2024-05-08T08:53:41.961 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8800000000000002e-07 2024-05-08T08:53:41.961 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.961 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.961 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:41.961 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451130+0000", 2024-05-08T08:53:41.961 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2299999999999997e-06 2024-05-08T08:53:41.961 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.962 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.962 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:41.962 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451137+0000", 2024-05-08T08:53:41.962 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4669999999999998e-06 2024-05-08T08:53:41.962 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.962 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.962 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:41.962 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451143+0000", 2024-05-08T08:53:41.962 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2839999999999997e-06 2024-05-08T08:53:41.962 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.962 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.962 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:41.962 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538575+0000", 2024-05-08T08:53:41.962 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087432789999999996 2024-05-08T08:53:41.963 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.963 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.963 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:41.963 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538648+0000", 2024-05-08T08:53:41.963 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2877e-05 2024-05-08T08:53:41.963 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.963 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.963 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:41.963 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538753+0000", 2024-05-08T08:53:41.963 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010449 2024-05-08T08:53:41.963 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:41.963 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:41.963 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:41.963 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.963 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.963 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:968 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 2424832~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:41.963 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451152+0000", 2024-05-08T08:53:41.964 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 32.479731979999997, 2024-05-08T08:53:41.964 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087813325999999997, 2024-05-08T08:53:41.964 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:41.964 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:41.964 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:41.964 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:41.964 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:41.964 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:41.964 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 968 2024-05-08T08:53:41.964 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.964 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:41.964 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.964 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:41.964 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451152+0000", 2024-05-08T08:53:41.964 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:41.965 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.965 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.965 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:41.965 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451152+0000", 2024-05-08T08:53:41.965 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:41.965 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.965 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.965 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:41.965 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451153+0000", 2024-05-08T08:53:41.965 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3099999999999998e-07 2024-05-08T08:53:41.965 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.965 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.965 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:41.965 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451155+0000", 2024-05-08T08:53:41.965 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4760000000000001e-06 2024-05-08T08:53:41.966 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.966 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.966 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:41.966 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451162+0000", 2024-05-08T08:53:41.966 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5969999999999997e-06 2024-05-08T08:53:41.966 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.966 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.966 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:41.966 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451167+0000", 2024-05-08T08:53:41.966 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.977e-06 2024-05-08T08:53:41.966 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.966 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.966 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:41.967 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538781+0000", 2024-05-08T08:53:41.967 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087614846999999996 2024-05-08T08:53:41.967 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.967 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.967 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:41.967 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538857+0000", 2024-05-08T08:53:41.967 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5358000000000006e-05 2024-05-08T08:53:41.967 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.967 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.967 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:41.967 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538965+0000", 2024-05-08T08:53:41.967 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010864 2024-05-08T08:53:41.967 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:41.968 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:41.968 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:41.968 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.968 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.968 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:969 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 2916352~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:41.968 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:53:41.968 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 32.479710275000002, 2024-05-08T08:53:41.968 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087998980000000004, 2024-05-08T08:53:41.968 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:41.968 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:41.968 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:41.968 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:41.968 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:41.969 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:41.969 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 969 2024-05-08T08:53:41.969 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.969 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:41.969 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.969 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:41.969 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:53:41.969 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:41.969 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.969 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.969 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:41.969 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:53:41.969 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:41.969 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.969 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.970 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:41.970 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:53:41.970 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3599999999999999e-07 2024-05-08T08:53:41.970 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.970 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.970 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:41.970 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451177+0000", 2024-05-08T08:53:41.970 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4590000000000001e-06 2024-05-08T08:53:41.970 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.970 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.970 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:41.970 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451184+0000", 2024-05-08T08:53:41.970 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4109999999999999e-06 2024-05-08T08:53:41.971 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.971 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.971 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:41.971 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451191+0000", 2024-05-08T08:53:41.971 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7279999999999998e-06 2024-05-08T08:53:41.971 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.971 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.971 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:41.971 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538997+0000", 2024-05-08T08:53:41.971 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087806170000000003 2024-05-08T08:53:41.971 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.971 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.971 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:41.971 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539069+0000", 2024-05-08T08:53:41.971 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2373999999999995e-05 2024-05-08T08:53:41.972 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.972 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.972 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:41.972 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539173+0000", 2024-05-08T08:53:41.972 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000103402 2024-05-08T08:53:41.972 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:41.972 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:41.972 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:41.972 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.972 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.972 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:970 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 3407872~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:41.972 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451198+0000", 2024-05-08T08:53:41.972 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 32.479685678000003, 2024-05-08T08:53:41.972 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088183368999999998, 2024-05-08T08:53:41.973 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:41.973 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:41.973 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:41.973 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:41.973 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:41.973 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:41.973 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 970 2024-05-08T08:53:41.973 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.973 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:41.973 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.973 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:41.973 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451198+0000", 2024-05-08T08:53:41.973 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:41.973 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.973 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.973 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:41.974 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451198+0000", 2024-05-08T08:53:41.974 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:41.974 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.974 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.974 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:41.974 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451199+0000", 2024-05-08T08:53:41.974 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7400000000000003e-07 2024-05-08T08:53:41.974 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.974 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.974 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:41.974 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451211+0000", 2024-05-08T08:53:41.974 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2109999999999999e-05 2024-05-08T08:53:41.974 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.974 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.975 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:41.975 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451219+0000", 2024-05-08T08:53:41.975 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4170000000000003e-06 2024-05-08T08:53:41.975 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.975 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.975 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:41.975 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451224+0000", 2024-05-08T08:53:41.975 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3990000000000003e-06 2024-05-08T08:53:41.975 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.975 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.975 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:41.975 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539204+0000", 2024-05-08T08:53:41.975 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087980099000000006 2024-05-08T08:53:41.975 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.975 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.976 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:41.976 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539276+0000", 2024-05-08T08:53:41.976 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1513000000000005e-05 2024-05-08T08:53:41.976 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.976 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.976 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:41.976 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539382+0000", 2024-05-08T08:53:41.976 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010615699999999999 2024-05-08T08:53:41.976 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:41.976 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:41.976 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:41.976 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.976 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.976 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:971 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 3899392~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:41.976 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451233+0000", 2024-05-08T08:53:41.976 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 32.479650970999998, 2024-05-08T08:53:41.977 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088431525999999996, 2024-05-08T08:53:41.977 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:41.977 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:41.977 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:41.977 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:41.977 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:41.977 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:41.977 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 971 2024-05-08T08:53:41.977 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.977 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:41.977 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.977 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:41.977 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451233+0000", 2024-05-08T08:53:41.977 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:41.977 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.978 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.978 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:41.978 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451233+0000", 2024-05-08T08:53:41.978 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:41.978 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.978 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.978 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:41.978 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451234+0000", 2024-05-08T08:53:41.978 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8500000000000001e-07 2024-05-08T08:53:41.978 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.978 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.978 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:41.978 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451237+0000", 2024-05-08T08:53:41.979 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8320000000000002e-06 2024-05-08T08:53:41.979 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.979 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.979 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:41.979 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451243+0000", 2024-05-08T08:53:41.979 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6749999999999996e-06 2024-05-08T08:53:41.979 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.979 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.979 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:41.979 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451249+0000", 2024-05-08T08:53:41.979 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1229999999999997e-06 2024-05-08T08:53:41.979 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.979 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.980 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:41.980 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539412+0000", 2024-05-08T08:53:41.980 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088162668999999999 2024-05-08T08:53:41.980 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.980 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.980 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:41.980 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539505+0000", 2024-05-08T08:53:41.980 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3240999999999997e-05 2024-05-08T08:53:41.980 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.980 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.980 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:41.980 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539665+0000", 2024-05-08T08:53:41.980 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00015930100000000001 2024-05-08T08:53:41.980 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:41.980 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:41.981 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:41.981 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.981 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.981 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:972 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 196608~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:41.981 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451257+0000", 2024-05-08T08:53:41.981 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 32.479627026000003, 2024-05-08T08:53:41.981 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088719928000000003, 2024-05-08T08:53:41.981 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:41.981 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:41.981 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:41.981 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:41.981 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:41.981 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:41.981 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 972 2024-05-08T08:53:41.982 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.982 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:41.982 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.982 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:41.982 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451257+0000", 2024-05-08T08:53:41.982 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:41.982 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.982 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.982 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:41.982 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451257+0000", 2024-05-08T08:53:41.982 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:41.982 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.982 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.982 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:41.982 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451258+0000", 2024-05-08T08:53:41.983 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7999999999999996e-07 2024-05-08T08:53:41.983 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.983 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.983 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:41.983 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451260+0000", 2024-05-08T08:53:41.983 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4609999999999998e-06 2024-05-08T08:53:41.983 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.983 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.983 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:41.983 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451267+0000", 2024-05-08T08:53:41.983 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.81e-06 2024-05-08T08:53:41.983 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.983 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.983 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:41.983 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451273+0000", 2024-05-08T08:53:41.983 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0859999999999998e-06 2024-05-08T08:53:41.984 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.984 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.984 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:41.984 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539718+0000", 2024-05-08T08:53:41.984 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088445059000000006 2024-05-08T08:53:41.984 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.984 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.984 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:41.984 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539834+0000", 2024-05-08T08:53:41.984 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000116426 2024-05-08T08:53:41.984 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.984 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.984 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:41.985 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539977+0000", 2024-05-08T08:53:41.985 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014260600000000001 2024-05-08T08:53:41.985 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:41.985 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:41.985 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:41.985 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.985 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.985 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:973 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 688128~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:41.985 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:53:41.985 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 32.479604442000003, 2024-05-08T08:53:41.985 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088918496, 2024-05-08T08:53:41.985 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:41.985 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:41.985 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:41.985 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:41.985 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:41.986 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:41.986 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 973 2024-05-08T08:53:41.986 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.986 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:41.986 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.986 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:41.986 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:53:41.986 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:41.986 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.986 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.986 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:41.986 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:53:41.986 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:41.986 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.986 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.986 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:41.987 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:53:41.987 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9099999999999999e-07 2024-05-08T08:53:41.987 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.987 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.987 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:41.987 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451282+0000", 2024-05-08T08:53:41.987 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1550000000000001e-06 2024-05-08T08:53:41.987 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.987 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.987 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:41.987 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451288+0000", 2024-05-08T08:53:41.987 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8889999999999999e-06 2024-05-08T08:53:41.987 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.987 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.988 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:41.988 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451294+0000", 2024-05-08T08:53:41.988 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0850000000000002e-06 2024-05-08T08:53:41.988 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.988 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.988 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:41.988 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540011+0000", 2024-05-08T08:53:41.988 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088717314000000005 2024-05-08T08:53:41.988 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.988 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.989 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:41.989 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540089+0000", 2024-05-08T08:53:41.989 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7438e-05 2024-05-08T08:53:41.989 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.989 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.989 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:41.989 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540198+0000", 2024-05-08T08:53:41.989 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000109224 2024-05-08T08:53:41.989 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:41.989 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:41.989 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:41.989 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.989 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.990 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:974 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 1179648~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:41.990 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451301+0000", 2024-05-08T08:53:41.990 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 32.479582809, 2024-05-08T08:53:41.990 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089105078000000004, 2024-05-08T08:53:41.990 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:41.990 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:41.990 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:41.990 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:41.990 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:41.990 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:41.990 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 974 2024-05-08T08:53:41.990 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.990 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:41.990 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.991 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:41.991 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451301+0000", 2024-05-08T08:53:41.991 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:41.991 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.991 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.991 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:41.991 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451301+0000", 2024-05-08T08:53:41.991 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:41.991 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.991 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.991 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:41.991 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451302+0000", 2024-05-08T08:53:41.991 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7800000000000002e-07 2024-05-08T08:53:41.991 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.991 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.992 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:41.992 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451304+0000", 2024-05-08T08:53:41.992 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.65e-06 2024-05-08T08:53:41.992 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.992 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.992 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:41.992 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451311+0000", 2024-05-08T08:53:41.992 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.472e-06 2024-05-08T08:53:41.992 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.992 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.992 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:41.992 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451316+0000", 2024-05-08T08:53:41.992 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9189999999999998e-06 2024-05-08T08:53:41.992 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.992 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.992 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:41.993 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540234+0000", 2024-05-08T08:53:41.993 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088917752000000003 2024-05-08T08:53:41.993 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.993 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.993 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:41.993 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540308+0000", 2024-05-08T08:53:41.993 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4758000000000005e-05 2024-05-08T08:53:41.993 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.993 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.993 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:41.993 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540406+0000", 2024-05-08T08:53:41.993 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8048999999999996e-05 2024-05-08T08:53:41.993 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:41.993 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:41.994 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:41.994 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.994 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.994 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:975 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 1671168~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:41.994 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:53:41.994 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 32.479540479000001, 2024-05-08T08:53:41.994 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089294139999999994, 2024-05-08T08:53:41.994 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:41.994 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:41.994 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:41.994 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:41.994 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:41.994 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:41.995 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 975 2024-05-08T08:53:41.995 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.995 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:41.995 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.995 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:41.995 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:53:41.995 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:41.995 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.995 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.995 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:41.995 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:53:41.995 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:41.996 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.996 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.996 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:41.996 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:53:41.996 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1299999999999999e-07 2024-05-08T08:53:41.996 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.996 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.996 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:41.996 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451348+0000", 2024-05-08T08:53:41.996 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.484e-06 2024-05-08T08:53:41.996 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.996 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.996 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:41.996 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451356+0000", 2024-05-08T08:53:41.996 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8499999999999994e-06 2024-05-08T08:53:41.997 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.997 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.997 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:41.997 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451366+0000", 2024-05-08T08:53:41.997 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.9850000000000001e-06 2024-05-08T08:53:41.997 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.997 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.997 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:41.997 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540448+0000", 2024-05-08T08:53:41.997 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089082254 2024-05-08T08:53:41.997 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.997 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.997 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:41.997 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540528+0000", 2024-05-08T08:53:41.997 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0234999999999993e-05 2024-05-08T08:53:41.997 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.998 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.998 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:41.998 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540638+0000", 2024-05-08T08:53:41.998 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010951900000000001 2024-05-08T08:53:41.998 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:41.998 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:41.998 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:41.998 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.998 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.998 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:976 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 2162688~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:41.998 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451375+0000", 2024-05-08T08:53:41.998 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 32.479508731999999, 2024-05-08T08:53:41.998 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089472745000000006, 2024-05-08T08:53:41.999 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:41.999 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:41.999 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:41.999 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:41.999 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:41.999 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:41.999 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 976 2024-05-08T08:53:41.999 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:41.999 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:41.999 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:41.999 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:41.999 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451375+0000", 2024-05-08T08:53:41.999 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:42.000 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:42.000 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:42.000 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:42.000 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451375+0000", 2024-05-08T08:53:42.000 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:42.000 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:42.000 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:42.000 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:42.000 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451376+0000", 2024-05-08T08:53:42.000 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2399999999999998e-07 2024-05-08T08:53:42.000 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:42.000 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:42.000 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:42.000 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451379+0000", 2024-05-08T08:53:42.000 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6359999999999998e-06 2024-05-08T08:53:42.000 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:42.001 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:42.001 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:42.001 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451387+0000", 2024-05-08T08:53:42.001 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9929999999999995e-06 2024-05-08T08:53:42.001 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:42.001 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:42.001 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:42.001 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451393+0000", 2024-05-08T08:53:42.001 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4699999999999999e-06 2024-05-08T08:53:42.001 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:42.001 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:42.001 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:42.001 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540677+0000", 2024-05-08T08:53:42.001 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089283504999999999 2024-05-08T08:53:42.001 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:42.001 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:42.001 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:42.002 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540743+0000", 2024-05-08T08:53:42.002 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6303999999999994e-05 2024-05-08T08:53:42.002 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:42.002 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:42.002 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:42.002 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540848+0000", 2024-05-08T08:53:42.002 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000105213 2024-05-08T08:53:42.002 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:42.002 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:42.002 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:42.002 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:42.002 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:42.002 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:977 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 2654208~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:42.002 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:53:42.003 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 32.479483299999998, 2024-05-08T08:53:42.003 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089698473000000001, 2024-05-08T08:53:42.003 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:42.003 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:42.003 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:42.003 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:42.003 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:42.003 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:42.003 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 977 2024-05-08T08:53:42.003 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:42.003 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:42.003 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:42.003 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:42.003 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:53:42.003 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:42.003 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:42.004 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:42.004 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:42.004 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:53:42.004 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:42.004 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:42.004 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:42.004 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:42.004 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:53:42.004 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2899999999999999e-07 2024-05-08T08:53:42.004 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:42.004 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:42.004 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:42.004 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451404+0000", 2024-05-08T08:53:42.004 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6649999999999999e-06 2024-05-08T08:53:42.005 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:42.005 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:42.005 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:42.005 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451410+0000", 2024-05-08T08:53:42.005 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3269999999999997e-06 2024-05-08T08:53:42.005 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:42.005 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:42.005 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:42.005 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451415+0000", 2024-05-08T08:53:42.005 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9529999999999998e-06 2024-05-08T08:53:42.005 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:42.005 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:42.005 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:42.005 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540912+0000", 2024-05-08T08:53:42.005 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089496950000000006 2024-05-08T08:53:42.006 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:42.006 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:42.006 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:42.006 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541000+0000", 2024-05-08T08:53:42.006 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7429000000000006e-05 2024-05-08T08:53:42.006 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:42.006 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:42.006 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:42.006 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541099+0000", 2024-05-08T08:53:42.006 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.9720000000000001e-05 2024-05-08T08:53:42.006 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:42.006 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:42.006 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:42.007 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:42.007 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:42.007 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:978 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 3145728~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:42.007 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:53:42.007 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 32.479462091999999, 2024-05-08T08:53:42.007 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089900635000000007, 2024-05-08T08:53:42.007 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:42.007 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:42.007 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:42.007 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:42.007 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:42.007 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:42.007 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 978 2024-05-08T08:53:42.008 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:42.008 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:42.008 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:42.008 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:42.008 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:53:42.008 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:42.008 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:42.008 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:42.008 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:42.008 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:53:42.008 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:42.008 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:42.008 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:42.009 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:42.009 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:53:42.009 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9400000000000001e-07 2024-05-08T08:53:42.009 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:42.009 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:42.009 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:42.009 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451425+0000", 2024-05-08T08:53:42.009 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2460000000000002e-06 2024-05-08T08:53:42.009 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:42.009 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:42.009 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:42.009 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451431+0000", 2024-05-08T08:53:42.009 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2419999999999997e-06 2024-05-08T08:53:42.009 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:42.010 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:42.010 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:42.010 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451436+0000", 2024-05-08T08:53:42.010 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.887e-06 2024-05-08T08:53:42.010 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:42.010 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:42.010 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:42.010 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541152+0000", 2024-05-08T08:53:42.010 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089716194999999999 2024-05-08T08:53:42.010 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:42.010 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:42.010 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:42.010 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541224+0000", 2024-05-08T08:53:42.011 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2416000000000006e-05 2024-05-08T08:53:42.011 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:42.011 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:42.011 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:42.011 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541323+0000", 2024-05-08T08:53:42.011 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8255e-05 2024-05-08T08:53:42.011 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:42.011 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:42.011 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:42.011 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:42.011 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:42.011 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:979 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 3637248~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:42.011 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451462+0000", 2024-05-08T08:53:42.011 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 32.479421643000002, 2024-05-08T08:53:42.012 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.090036441999999994, 2024-05-08T08:53:42.012 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:42.012 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:42.012 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:42.012 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:42.012 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:42.012 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:42.012 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 979 2024-05-08T08:53:42.012 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:42.012 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:42.012 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:42.012 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:42.013 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451462+0000", 2024-05-08T08:53:42.013 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:42.013 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:42.013 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:42.013 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:42.013 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451462+0000", 2024-05-08T08:53:42.013 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:42.013 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:42.013 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:42.013 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:42.013 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451463+0000", 2024-05-08T08:53:42.013 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4300000000000002e-07 2024-05-08T08:53:42.013 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:42.013 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:42.013 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:42.014 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451471+0000", 2024-05-08T08:53:42.014 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0800000000000006e-06 2024-05-08T08:53:42.014 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:42.014 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:42.014 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:42.014 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451481+0000", 2024-05-08T08:53:42.014 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.38e-06 2024-05-08T08:53:42.014 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:42.014 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:42.014 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:42.014 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451487+0000", 2024-05-08T08:53:42.014 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4590000000000003e-06 2024-05-08T08:53:42.014 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:42.015 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:42.015 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:42.015 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541350+0000", 2024-05-08T08:53:42.015 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089862599000000001 2024-05-08T08:53:42.015 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:42.015 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:42.015 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:42.015 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541407+0000", 2024-05-08T08:53:42.015 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7667999999999997e-05 2024-05-08T08:53:42.015 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:42.015 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:42.015 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:42.015 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541499+0000", 2024-05-08T08:53:42.015 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1513000000000003e-05 2024-05-08T08:53:42.016 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:42.016 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:42.016 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:42.016 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:42.016 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:42.016 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:980 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 4128768~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:42.016 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451496+0000", 2024-05-08T08:53:42.016 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 32.479387887000001, 2024-05-08T08:53:42.016 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.090258985, 2024-05-08T08:53:42.016 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:42.016 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:42.016 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:42.016 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:42.016 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:42.016 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:42.017 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 980 2024-05-08T08:53:42.017 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:42.017 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:42.017 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:42.017 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:42.017 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451496+0000", 2024-05-08T08:53:42.017 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:42.017 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:42.017 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:42.017 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:42.017 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451496+0000", 2024-05-08T08:53:42.017 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:42.017 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:42.017 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:42.018 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:42.018 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451497+0000", 2024-05-08T08:53:42.018 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7599999999999997e-07 2024-05-08T08:53:42.018 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:42.018 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:42.018 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:42.018 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451500+0000", 2024-05-08T08:53:42.018 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7930000000000002e-06 2024-05-08T08:53:42.018 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:42.018 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:42.018 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:42.018 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451506+0000", 2024-05-08T08:53:42.019 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6980000000000001e-06 2024-05-08T08:53:42.019 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:42.019 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:42.019 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:42.019 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451512+0000", 2024-05-08T08:53:42.019 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2490000000000001e-06 2024-05-08T08:53:42.019 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:42.019 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:42.019 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:42.019 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541550+0000", 2024-05-08T08:53:42.019 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.090037961999999999 2024-05-08T08:53:42.020 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:42.020 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:42.020 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:42.020 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541647+0000", 2024-05-08T08:53:42.020 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6924999999999998e-05 2024-05-08T08:53:42.020 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:42.020 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:42.020 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:42.020 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541755+0000", 2024-05-08T08:53:42.020 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000108682 2024-05-08T08:53:42.020 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:42.020 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:42.020 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:42.020 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:42.020 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:42.021 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:42.023 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:53:42.025 INFO:teuthology.orchestra.run.smithi196.stdout:3684 write 0x757a00 thru 0x75eee7 (0x74e8 bytes) 2024-05-08T08:53:42.025 INFO:teuthology.orchestra.run.smithi196.stdout:3685 write 0x41e5a00 thru 0x41ecb29 (0x712a bytes) 2024-05-08T08:53:42.025 INFO:teuthology.orchestra.run.smithi196.stdout:3688 punch from 0x1b32a00 to 0x1b39800, (0x6e00 bytes) 2024-05-08T08:53:42.026 INFO:teuthology.orchestra.run.smithi196.stdout:3691 punch from 0x1544800 to 0x1546a00, (0x2200 bytes) 2024-05-08T08:53:42.026 INFO:teuthology.orchestra.run.smithi196.stdout:3692 read 0x1989400 thru 0x198dc62 (0x4863 bytes) 2024-05-08T08:53:42.026 INFO:teuthology.orchestra.run.smithi196.stdout:3696 read 0x637ac00 thru 0x6382c78 (0x8079 bytes) 2024-05-08T08:53:42.026 INFO:teuthology.orchestra.run.smithi196.stdout:3697 read 0x26af800 thru 0x26b338c (0x3b8d bytes) 2024-05-08T08:53:42.026 INFO:teuthology.orchestra.run.smithi196.stdout:3699 read 0x70b6400 thru 0x70c5764 (0xf365 bytes) 2024-05-08T08:53:42.026 INFO:teuthology.orchestra.run.smithi196.stdout:3701 read 0xfe3200 thru 0xfef467 (0xc268 bytes) 2024-05-08T08:53:42.026 INFO:teuthology.orchestra.run.smithi196.stdout:3703 read 0x6184800 thru 0x618bd4b (0x754c bytes) 2024-05-08T08:53:42.026 INFO:teuthology.orchestra.run.smithi196.stdout:3706 read 0x3887a00 thru 0x388c97b (0x4f7c bytes) 2024-05-08T08:53:42.026 INFO:teuthology.orchestra.run.smithi196.stdout:3707 write 0x787f000 thru 0x788eb87 (0xfb88 bytes) 2024-05-08T08:53:42.026 INFO:teuthology.orchestra.run.smithi196.stdout:3709 read 0x6e5d800 thru 0x6e6a414 (0xcc15 bytes) 2024-05-08T08:53:42.026 INFO:teuthology.orchestra.run.smithi196.stdout:3710 write 0x4fe2c00 thru 0x4fe3c74 (0x1075 bytes) 2024-05-08T08:53:42.026 INFO:teuthology.orchestra.run.smithi196.stdout:3711 write 0x5e38000 thru 0x5e44bf4 (0xcbf5 bytes) 2024-05-08T08:53:42.026 INFO:teuthology.orchestra.run.smithi196.stdout:3715 punch from 0x4d98000 to 0x4d9de00, (0x5e00 bytes) 2024-05-08T08:53:42.027 INFO:teuthology.orchestra.run.smithi196.stdout:3716 write 0x88cbc00 thru 0x88ce1d3 (0x25d4 bytes) 2024-05-08T08:53:42.027 INFO:teuthology.orchestra.run.smithi196.stdout:3719 trunc from 0x88ce200 to 0xa114600 2024-05-08T08:53:42.027 INFO:teuthology.orchestra.run.smithi196.stdout:3720 read 0xe6ec00 thru 0xe74f9b (0x639c bytes) 2024-05-08T08:53:42.027 INFO:teuthology.orchestra.run.smithi196.stdout:3721 read 0x159aa00 thru 0x15a8773 (0xdd74 bytes) 2024-05-08T08:53:42.064 INFO:teuthology.orchestra.run.smithi196.stdout:3723 write 0xc9d2400 thru 0xc9e0381 (0xdf82 bytes) 2024-05-08T08:53:42.064 INFO:teuthology.orchestra.run.smithi196.stdout:3724 punch from 0x816f000 to 0x8171600, (0x2600 bytes) 2024-05-08T08:53:42.095 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:42.026+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:42.096 INFO:teuthology.orchestra.run.smithi196.stdout:3725 write 0xac2a000 thru 0xac36e34 (0xce35 bytes) 2024-05-08T08:53:42.102 INFO:teuthology.orchestra.run.smithi196.stdout:3728 trunc from 0xc9e0400 to 0x7e56a00 2024-05-08T08:53:42.122 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:42.125+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:42.129 INFO:teuthology.orchestra.run.smithi196.stdout:3729 write 0x57a9200 thru 0x57b38cd (0xa6ce bytes) 2024-05-08T08:53:42.129 INFO:teuthology.orchestra.run.smithi196.stdout:3730 clone 27 order 25 su 1048576 sc 11 2024-05-08T08:53:42.144 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:53:42.144 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:53:42.144 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:53:42.144 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:53:42.153 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:53:42.168 INFO:teuthology.orchestra.run.smithi059.stderr:no valid command found; 10 closest matches: 2024-05-08T08:53:42.168 INFO:teuthology.orchestra.run.smithi059.stderr:0 2024-05-08T08:53:42.168 INFO:teuthology.orchestra.run.smithi059.stderr:1 2024-05-08T08:53:42.168 INFO:teuthology.orchestra.run.smithi059.stderr:2 2024-05-08T08:53:42.169 INFO:teuthology.orchestra.run.smithi059.stderr:abort 2024-05-08T08:53:42.169 INFO:teuthology.orchestra.run.smithi059.stderr:assert 2024-05-08T08:53:42.169 INFO:teuthology.orchestra.run.smithi059.stderr:bench [] [] [] [] 2024-05-08T08:53:42.169 INFO:teuthology.orchestra.run.smithi059.stderr:bluefs debug_inject_read_zeros 2024-05-08T08:53:42.169 INFO:teuthology.orchestra.run.smithi059.stderr:bluefs files list 2024-05-08T08:53:42.169 INFO:teuthology.orchestra.run.smithi059.stderr:bluefs stats 2024-05-08T08:53:42.169 INFO:teuthology.orchestra.run.smithi059.stderr:bluestore allocator dump block 2024-05-08T08:53:42.169 INFO:teuthology.orchestra.run.smithi059.stderr:Error EINVAL: invalid command 2024-05-08T08:53:42.171 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-08T08:53:42.171 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-05-08T08:53:42.172 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 'trim stale osdmaps' 2024-05-08T08:53:42.237 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:42.240+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:42.298 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:53:42.298 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:53:42.298 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:53:42.298 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:53:42.298 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:53:42.307 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:53:42.348 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:42.351+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:42.349 INFO:teuthology.orchestra.run.smithi059.stderr:no valid command found; 10 closest matches: 2024-05-08T08:53:42.349 INFO:teuthology.orchestra.run.smithi059.stderr:0 2024-05-08T08:53:42.349 INFO:teuthology.orchestra.run.smithi059.stderr:1 2024-05-08T08:53:42.349 INFO:teuthology.orchestra.run.smithi059.stderr:2 2024-05-08T08:53:42.349 INFO:teuthology.orchestra.run.smithi059.stderr:abort 2024-05-08T08:53:42.350 INFO:teuthology.orchestra.run.smithi059.stderr:assert 2024-05-08T08:53:42.350 INFO:teuthology.orchestra.run.smithi059.stderr:bench [] [] [] [] 2024-05-08T08:53:42.350 INFO:teuthology.orchestra.run.smithi059.stderr:bluefs debug_inject_read_zeros 2024-05-08T08:53:42.350 INFO:teuthology.orchestra.run.smithi059.stderr:bluefs files list 2024-05-08T08:53:42.350 INFO:teuthology.orchestra.run.smithi059.stderr:bluefs stats 2024-05-08T08:53:42.350 INFO:teuthology.orchestra.run.smithi059.stderr:bluestore allocator dump block 2024-05-08T08:53:42.350 INFO:teuthology.orchestra.run.smithi059.stderr:Error EINVAL: invalid command 2024-05-08T08:53:42.352 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-08T08:53:42.352 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-05-08T08:53:42.401 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T08:53:42.425 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:42.428+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:42.455 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:53:42.455 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:53:42.455 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:53:42.455 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:53:42.455 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.455 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4621.0:9089 3.1f e33/18 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:head v 33'117, mlcod=31'115)", 2024-05-08T08:53:42.456 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:53:42.456 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 69.937631968000005, 2024-05-08T08:53:42.456 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.111244524, 2024-05-08T08:53:42.456 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:42.456 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:42.456 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:53:42.456 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:42.456 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.456 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:42.456 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:53:42.456 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:42.456 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.456 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.456 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:42.456 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:53:42.457 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:42.457 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.457 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.457 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:42.457 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:53:42.457 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5900000000000001e-07 2024-05-08T08:53:42.457 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.457 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.457 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:42.457 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539600+0000", 2024-05-08T08:53:42.457 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.019981364000000001 2024-05-08T08:53:42.457 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.457 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.457 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:42.458 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539624+0000", 2024-05-08T08:53:42.458 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4043e-05 2024-05-08T08:53:42.458 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.458 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.458 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:42.458 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539635+0000", 2024-05-08T08:53:42.458 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1511e-05 2024-05-08T08:53:42.458 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.458 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.458 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:42.458 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539728+0000", 2024-05-08T08:53:42.458 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2498999999999996e-05 2024-05-08T08:53:42.458 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.458 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.458 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:42.459 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539746+0000", 2024-05-08T08:53:42.459 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8289000000000001e-05 2024-05-08T08:53:42.459 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.459 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.459 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:53:42.459 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.630822+0000", 2024-05-08T08:53:42.459 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.091076283999999993 2024-05-08T08:53:42.459 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.459 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.459 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:42.459 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.630863+0000", 2024-05-08T08:53:42.459 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0374999999999999e-05 2024-05-08T08:53:42.459 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:42.460 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:42.460 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:42.460 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.460 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.460 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4795.0:230 3.1f e40/18 3:f8db1a56:::rbd_data.11402b26385.0000000000000000:head v 40'171, mlcod=39'168)", 2024-05-08T08:53:42.460 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:45.927773+0000", 2024-05-08T08:53:42.460 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 56.529476885000001, 2024-05-08T08:53:42.460 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045892405999999997, 2024-05-08T08:53:42.460 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:42.460 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:42.460 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:53:42.460 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:42.461 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.461 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:42.461 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.927773+0000", 2024-05-08T08:53:42.461 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:42.461 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.461 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.461 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:42.461 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.927773+0000", 2024-05-08T08:53:42.461 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:42.461 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.462 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.462 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:42.462 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.927774+0000", 2024-05-08T08:53:42.462 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0500000000000002e-07 2024-05-08T08:53:42.462 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.462 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.462 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:42.462 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.931581+0000", 2024-05-08T08:53:42.462 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0038071799999999999 2024-05-08T08:53:42.462 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.462 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.463 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:42.463 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.931599+0000", 2024-05-08T08:53:42.463 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7479e-05 2024-05-08T08:53:42.463 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.463 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.463 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:42.463 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.931624+0000", 2024-05-08T08:53:42.463 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5372999999999999e-05 2024-05-08T08:53:42.463 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.463 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.463 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:42.463 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.931698+0000", 2024-05-08T08:53:42.463 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4043999999999998e-05 2024-05-08T08:53:42.464 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.464 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.465 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:42.465 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.931725+0000", 2024-05-08T08:53:42.465 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6883e-05 2024-05-08T08:53:42.466 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.466 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.466 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:53:42.466 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.973623+0000", 2024-05-08T08:53:42.466 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.041898283000000001 2024-05-08T08:53:42.466 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.466 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.466 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:42.467 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.973666+0000", 2024-05-08T08:53:42.467 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2358999999999999e-05 2024-05-08T08:53:42.467 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:42.467 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:42.467 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:42.467 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.467 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.467 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4195.0:4 3.11 3:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-05-08T08:53:42.467 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:49.764654+0000", 2024-05-08T08:53:42.467 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 52.692596426999998, 2024-05-08T08:53:42.467 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.30714275499999999, 2024-05-08T08:53:42.467 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:42.467 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:42.468 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-05-08T08:53:42.468 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:42.468 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4195", 2024-05-08T08:53:42.468 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/59179521", 2024-05-08T08:53:42.468 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4 2024-05-08T08:53:42.468 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.468 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:42.468 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.468 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:42.468 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764654+0000", 2024-05-08T08:53:42.468 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:42.468 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.468 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.468 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:42.468 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764654+0000", 2024-05-08T08:53:42.469 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:42.469 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.469 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.469 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:42.469 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764655+0000", 2024-05-08T08:53:42.469 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.452e-06 2024-05-08T08:53:42.469 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.469 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.469 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:42.469 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764661+0000", 2024-05-08T08:53:42.469 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5369999999999997e-06 2024-05-08T08:53:42.469 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.469 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.469 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:42.469 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764671+0000", 2024-05-08T08:53:42.469 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.028e-05 2024-05-08T08:53:42.469 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.470 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.470 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:42.470 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764680+0000", 2024-05-08T08:53:42.470 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0629999999999995e-06 2024-05-08T08:53:42.470 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.470 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.470 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:42.470 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764753+0000", 2024-05-08T08:53:42.470 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2679e-05 2024-05-08T08:53:42.470 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.470 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.470 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-05-08T08:53:42.470 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764765+0000", 2024-05-08T08:53:42.470 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2415000000000001e-05 2024-05-08T08:53:42.471 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.471 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.471 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:42.471 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:50.071692+0000", 2024-05-08T08:53:42.471 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.306927324 2024-05-08T08:53:42.471 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.471 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.471 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:42.471 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:50.071796+0000", 2024-05-08T08:53:42.471 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104005 2024-05-08T08:53:42.471 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:42.471 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:42.471 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:42.471 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.471 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.472 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22681 3.15 3:a907109d:::rbd_data.1140d3fc54c.000000000000004f:18 [read 1204224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:42.472 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.097653+0000", 2024-05-08T08:53:42.472 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 10.359596799, 2024-05-08T08:53:42.472 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.041941285000000002, 2024-05-08T08:53:42.472 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:42.472 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:42.472 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:42.472 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:42.472 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:42.472 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:42.472 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22681 2024-05-08T08:53:42.472 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.473 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:42.473 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.473 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:42.473 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097653+0000", 2024-05-08T08:53:42.473 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:42.473 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.473 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.473 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:42.473 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097653+0000", 2024-05-08T08:53:42.473 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:42.473 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.473 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.474 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:42.474 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097654+0000", 2024-05-08T08:53:42.474 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9400000000000001e-07 2024-05-08T08:53:42.474 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.474 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.474 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:42.474 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097655+0000", 2024-05-08T08:53:42.474 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6220000000000001e-06 2024-05-08T08:53:42.474 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.474 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.474 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:42.474 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097661+0000", 2024-05-08T08:53:42.474 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5210000000000002e-06 2024-05-08T08:53:42.475 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.475 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.475 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:42.475 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097665+0000", 2024-05-08T08:53:42.475 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3170000000000003e-06 2024-05-08T08:53:42.475 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.475 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.475 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:42.475 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139509+0000", 2024-05-08T08:53:42.475 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.041843475999999998 2024-05-08T08:53:42.475 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.475 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.475 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:42.475 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139554+0000", 2024-05-08T08:53:42.475 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5478000000000001e-05 2024-05-08T08:53:42.476 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.476 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.476 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:42.476 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139595+0000", 2024-05-08T08:53:42.476 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0577000000000001e-05 2024-05-08T08:53:42.476 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:42.476 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:42.476 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:42.476 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.476 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.476 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22682 3.15 3:a907109d:::rbd_data.1140d3fc54c.000000000000004f:18 [read 1269760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:42.476 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.097670+0000", 2024-05-08T08:53:42.476 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 10.359580106999999, 2024-05-08T08:53:42.476 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042031394, 2024-05-08T08:53:42.477 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:42.477 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:42.477 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:42.477 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:42.477 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:42.477 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:42.477 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22682 2024-05-08T08:53:42.477 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.477 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:42.477 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.477 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:42.477 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097670+0000", 2024-05-08T08:53:42.477 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:42.477 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.477 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.477 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:42.478 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097670+0000", 2024-05-08T08:53:42.478 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:42.478 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.478 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.478 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:42.478 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097670+0000", 2024-05-08T08:53:42.478 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1800000000000002e-07 2024-05-08T08:53:42.478 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.478 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.478 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:42.478 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097672+0000", 2024-05-08T08:53:42.478 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6929999999999999e-06 2024-05-08T08:53:42.478 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.478 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.479 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:42.479 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097677+0000", 2024-05-08T08:53:42.479 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2480000000000004e-06 2024-05-08T08:53:42.479 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.479 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.479 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:42.479 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097682+0000", 2024-05-08T08:53:42.479 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4290000000000003e-06 2024-05-08T08:53:42.479 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.479 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.479 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:42.479 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139620+0000", 2024-05-08T08:53:42.479 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.041938256 2024-05-08T08:53:42.479 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.479 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.479 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:42.480 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139662+0000", 2024-05-08T08:53:42.480 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1727999999999998e-05 2024-05-08T08:53:42.480 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.480 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.480 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:42.480 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139701+0000", 2024-05-08T08:53:42.480 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9721999999999997e-05 2024-05-08T08:53:42.480 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:42.480 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:42.480 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:42.480 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.480 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.480 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22683 3.15 3:a907109d:::rbd_data.1140d3fc54c.000000000000004f:18 [read 1335296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:42.480 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.097686+0000", 2024-05-08T08:53:42.480 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 10.359563753, 2024-05-08T08:53:42.481 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042114716000000003, 2024-05-08T08:53:42.481 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:42.481 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:42.481 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:42.481 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:42.481 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:42.481 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:42.481 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22683 2024-05-08T08:53:42.481 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.481 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:42.481 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.481 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:42.481 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097686+0000", 2024-05-08T08:53:42.481 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:42.481 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.481 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.482 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:42.482 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097686+0000", 2024-05-08T08:53:42.482 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:42.482 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.482 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.482 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:42.482 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097687+0000", 2024-05-08T08:53:42.482 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3999999999999997e-07 2024-05-08T08:53:42.482 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.482 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.483 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:42.483 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097688+0000", 2024-05-08T08:53:42.483 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6789999999999999e-06 2024-05-08T08:53:42.483 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.483 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.483 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:42.483 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097693+0000", 2024-05-08T08:53:42.483 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.045e-06 2024-05-08T08:53:42.483 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.483 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.483 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:42.483 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097697+0000", 2024-05-08T08:53:42.483 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0260000000000001e-06 2024-05-08T08:53:42.484 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.484 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.484 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:42.484 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139725+0000", 2024-05-08T08:53:42.484 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042027737000000003 2024-05-08T08:53:42.484 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.484 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.485 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:42.485 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139764+0000", 2024-05-08T08:53:42.485 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8668999999999999e-05 2024-05-08T08:53:42.485 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.485 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.485 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:42.485 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139801+0000", 2024-05-08T08:53:42.485 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7219999999999999e-05 2024-05-08T08:53:42.486 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:42.486 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:42.486 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:42.486 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.486 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.486 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22684 3.15 3:a907109d:::rbd_data.1140d3fc54c.000000000000004f:18 [read 1400832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:42.486 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.097703+0000", 2024-05-08T08:53:42.486 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 10.3595474, 2024-05-08T08:53:42.486 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042198929000000003, 2024-05-08T08:53:42.486 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:42.486 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:42.486 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:42.486 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:42.487 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:42.487 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:42.487 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22684 2024-05-08T08:53:42.487 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.487 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:42.487 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.487 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:42.487 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097703+0000", 2024-05-08T08:53:42.487 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:42.487 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.487 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.487 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:42.487 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097703+0000", 2024-05-08T08:53:42.487 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:42.487 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.488 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.488 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:42.488 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097703+0000", 2024-05-08T08:53:42.488 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4400000000000001e-07 2024-05-08T08:53:42.488 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.488 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.488 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:42.488 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097705+0000", 2024-05-08T08:53:42.488 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9199999999999998e-06 2024-05-08T08:53:42.488 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.488 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.488 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:42.488 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097711+0000", 2024-05-08T08:53:42.488 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5559999999999998e-06 2024-05-08T08:53:42.488 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.488 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.489 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:42.489 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097715+0000", 2024-05-08T08:53:42.489 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3159999999999998e-06 2024-05-08T08:53:42.489 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.489 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.489 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:42.489 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139825+0000", 2024-05-08T08:53:42.489 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042110284999999997 2024-05-08T08:53:42.489 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.489 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.489 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:42.489 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139864+0000", 2024-05-08T08:53:42.489 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8980000000000003e-05 2024-05-08T08:53:42.489 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.489 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.490 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:42.490 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139902+0000", 2024-05-08T08:53:42.490 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7527999999999997e-05 2024-05-08T08:53:42.490 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:42.490 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:42.490 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:42.490 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.490 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.490 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22685 3.15 3:a907109d:::rbd_data.1140d3fc54c.000000000000004f:18 [read 1466368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:42.490 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.097721+0000", 2024-05-08T08:53:42.490 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 10.359528743, 2024-05-08T08:53:42.490 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042287791999999998, 2024-05-08T08:53:42.490 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:42.490 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:42.491 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:42.491 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:42.491 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:42.491 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:42.491 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22685 2024-05-08T08:53:42.491 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.491 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:42.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.492 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:42.492 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097721+0000", 2024-05-08T08:53:42.492 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:42.492 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.492 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.492 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:42.493 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097721+0000", 2024-05-08T08:53:42.493 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:42.493 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.493 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.493 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:42.493 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097722+0000", 2024-05-08T08:53:42.494 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2500000000000001e-07 2024-05-08T08:53:42.494 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.494 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.494 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:42.494 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097724+0000", 2024-05-08T08:53:42.494 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.835e-06 2024-05-08T08:53:42.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.495 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:42.495 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097729+0000", 2024-05-08T08:53:42.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5029999999999997e-06 2024-05-08T08:53:42.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.495 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:42.496 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097733+0000", 2024-05-08T08:53:42.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3309999999999997e-06 2024-05-08T08:53:42.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.496 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:42.496 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139923+0000", 2024-05-08T08:53:42.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042189988999999997 2024-05-08T08:53:42.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.496 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:42.496 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139970+0000", 2024-05-08T08:53:42.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6776000000000001e-05 2024-05-08T08:53:42.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.496 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:42.497 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140009+0000", 2024-05-08T08:53:42.497 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9032999999999997e-05 2024-05-08T08:53:42.497 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:42.497 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:42.497 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:42.497 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.497 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.497 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22686 3.15 3:a907109d:::rbd_data.1140d3fc54c.000000000000004f:18 [read 1531904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:42.497 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.097739+0000", 2024-05-08T08:53:42.497 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 10.359510744, 2024-05-08T08:53:42.497 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042371008000000002, 2024-05-08T08:53:42.497 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:42.497 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:42.497 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:42.497 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:42.497 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:42.498 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:42.498 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22686 2024-05-08T08:53:42.498 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.498 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:42.498 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.498 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:42.498 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097739+0000", 2024-05-08T08:53:42.498 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:42.498 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.498 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.498 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:42.498 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097739+0000", 2024-05-08T08:53:42.499 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:42.499 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.499 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.499 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:42.499 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097740+0000", 2024-05-08T08:53:42.499 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8599999999999999e-07 2024-05-08T08:53:42.499 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.499 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.499 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:42.499 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097742+0000", 2024-05-08T08:53:42.499 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9410000000000001e-06 2024-05-08T08:53:42.499 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.499 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.499 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:42.499 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097747+0000", 2024-05-08T08:53:42.499 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2079999999999999e-06 2024-05-08T08:53:42.500 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.500 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.500 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:42.500 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097751+0000", 2024-05-08T08:53:42.500 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2250000000000002e-06 2024-05-08T08:53:42.500 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.500 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.500 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:42.500 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140032+0000", 2024-05-08T08:53:42.500 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042281244000000003 2024-05-08T08:53:42.500 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.500 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.500 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:42.500 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140074+0000", 2024-05-08T08:53:42.500 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1316999999999998e-05 2024-05-08T08:53:42.500 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.500 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.501 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:42.501 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140110+0000", 2024-05-08T08:53:42.501 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6687000000000003e-05 2024-05-08T08:53:42.501 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:42.501 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:42.501 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:42.501 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.501 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.501 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22716 3.15 3:a88fdee9:::rbd_data.1140d3fc54c.000000000000004d:18 [read 1097728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:42.501 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.098153+0000", 2024-05-08T08:53:42.501 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 10.359097037, 2024-05-08T08:53:42.501 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042058535000000001, 2024-05-08T08:53:42.501 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:42.501 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:42.501 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:42.501 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:42.501 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:42.502 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:42.502 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22716 2024-05-08T08:53:42.502 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.502 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:42.502 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.502 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:42.502 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098153+0000", 2024-05-08T08:53:42.502 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:42.502 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.502 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.502 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:42.502 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098153+0000", 2024-05-08T08:53:42.502 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:42.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.503 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:42.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098154+0000", 2024-05-08T08:53:42.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.13e-07 2024-05-08T08:53:42.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.503 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:42.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098161+0000", 2024-05-08T08:53:42.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7360000000000001e-06 2024-05-08T08:53:42.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.504 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:42.504 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098171+0000", 2024-05-08T08:53:42.504 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1039999999999996e-06 2024-05-08T08:53:42.504 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.504 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.504 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:42.504 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098177+0000", 2024-05-08T08:53:42.504 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5760000000000002e-06 2024-05-08T08:53:42.504 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.504 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.504 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:42.504 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140135+0000", 2024-05-08T08:53:42.504 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.041957842000000002 2024-05-08T08:53:42.504 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.504 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.505 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:42.505 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140174+0000", 2024-05-08T08:53:42.505 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8902999999999997e-05 2024-05-08T08:53:42.505 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.505 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.505 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:42.505 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140212+0000", 2024-05-08T08:53:42.505 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7761e-05 2024-05-08T08:53:42.505 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:42.505 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:42.505 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:42.505 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.505 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.505 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22717 3.15 3:a88fdee9:::rbd_data.1140d3fc54c.000000000000004d:18 [read 1163264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:42.505 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.098188+0000", 2024-05-08T08:53:42.505 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 10.359062314999999, 2024-05-08T08:53:42.506 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042124907000000003, 2024-05-08T08:53:42.506 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:42.506 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:42.506 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:42.506 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:42.506 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:42.506 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:42.506 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22717 2024-05-08T08:53:42.506 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.506 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:42.506 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.506 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:42.506 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098188+0000", 2024-05-08T08:53:42.506 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:42.507 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.507 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.507 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:42.507 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098188+0000", 2024-05-08T08:53:42.507 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:42.507 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.507 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.507 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:42.507 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098188+0000", 2024-05-08T08:53:42.507 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3799999999999998e-07 2024-05-08T08:53:42.507 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.507 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.507 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:42.507 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098190+0000", 2024-05-08T08:53:42.507 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.114e-06 2024-05-08T08:53:42.507 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.507 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.508 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:42.508 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098197+0000", 2024-05-08T08:53:42.508 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9959999999999996e-06 2024-05-08T08:53:42.508 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.508 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.508 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:42.508 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098202+0000", 2024-05-08T08:53:42.508 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9110000000000001e-06 2024-05-08T08:53:42.508 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.508 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.508 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:42.508 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140235+0000", 2024-05-08T08:53:42.508 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042032913999999998 2024-05-08T08:53:42.508 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.508 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.508 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:42.509 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140275+0000", 2024-05-08T08:53:42.509 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9564999999999998e-05 2024-05-08T08:53:42.509 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.509 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.509 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:42.509 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140313+0000", 2024-05-08T08:53:42.509 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7969000000000002e-05 2024-05-08T08:53:42.509 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:42.509 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:42.509 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:42.509 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.509 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.509 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22718 3.15 3:a88fdee9:::rbd_data.1140d3fc54c.000000000000004d:18 [read 1228800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:42.509 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.098209+0000", 2024-05-08T08:53:42.509 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 10.359041168999999, 2024-05-08T08:53:42.509 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.04220409, 2024-05-08T08:53:42.510 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:42.510 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:42.510 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:42.510 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:42.510 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:42.510 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:42.510 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22718 2024-05-08T08:53:42.510 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.510 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:42.510 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.510 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:42.510 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098209+0000", 2024-05-08T08:53:42.510 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:42.510 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.511 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.511 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:42.511 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098209+0000", 2024-05-08T08:53:42.511 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:42.511 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.511 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.511 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:42.511 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098209+0000", 2024-05-08T08:53:42.511 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9299999999999999e-07 2024-05-08T08:53:42.511 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.511 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.511 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:42.512 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098212+0000", 2024-05-08T08:53:42.512 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.21e-06 2024-05-08T08:53:42.512 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.512 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.512 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:42.512 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098219+0000", 2024-05-08T08:53:42.512 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0269999999999998e-06 2024-05-08T08:53:42.512 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.512 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.512 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:42.512 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098223+0000", 2024-05-08T08:53:42.512 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7319999999999998e-06 2024-05-08T08:53:42.512 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.512 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.512 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:42.512 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140338+0000", 2024-05-08T08:53:42.512 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042114390000000002 2024-05-08T08:53:42.513 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.513 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.513 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:42.513 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140376+0000", 2024-05-08T08:53:42.513 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8414999999999997e-05 2024-05-08T08:53:42.513 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.513 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.513 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:42.513 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140413+0000", 2024-05-08T08:53:42.513 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6922999999999999e-05 2024-05-08T08:53:42.513 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:42.513 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:42.513 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:42.513 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.513 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.513 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22719 3.15 3:a88fdee9:::rbd_data.1140d3fc54c.000000000000004d:18 [read 1294336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:42.513 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.098229+0000", 2024-05-08T08:53:42.514 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 10.359021318, 2024-05-08T08:53:42.514 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042283016, 2024-05-08T08:53:42.514 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:42.514 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:42.514 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:42.514 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:42.514 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:42.514 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:42.514 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22719 2024-05-08T08:53:42.514 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.514 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:42.514 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.514 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:42.514 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098229+0000", 2024-05-08T08:53:42.514 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:42.515 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.515 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.515 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:42.515 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098229+0000", 2024-05-08T08:53:42.515 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:42.515 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.515 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.515 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:42.515 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098229+0000", 2024-05-08T08:53:42.515 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:53:42.515 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.515 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.515 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:42.515 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098231+0000", 2024-05-08T08:53:42.516 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.81e-06 2024-05-08T08:53:42.516 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.516 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.516 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:42.516 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098237+0000", 2024-05-08T08:53:42.516 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3389999999999998e-06 2024-05-08T08:53:42.516 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.516 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.516 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:42.516 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098243+0000", 2024-05-08T08:53:42.516 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.648e-06 2024-05-08T08:53:42.516 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.516 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.516 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:42.516 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140436+0000", 2024-05-08T08:53:42.516 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042192738 2024-05-08T08:53:42.517 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.517 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.517 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:42.517 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140475+0000", 2024-05-08T08:53:42.517 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9279999999999997e-05 2024-05-08T08:53:42.517 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.517 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.517 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:42.517 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140512+0000", 2024-05-08T08:53:42.517 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6847000000000001e-05 2024-05-08T08:53:42.517 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:42.517 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:42.517 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:42.517 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.517 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.518 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22720 3.15 3:a88fdee9:::rbd_data.1140d3fc54c.000000000000004d:18 [read 1359872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:42.518 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.098248+0000", 2024-05-08T08:53:42.518 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 10.359001807, 2024-05-08T08:53:42.518 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042360800999999997, 2024-05-08T08:53:42.518 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:42.518 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:42.518 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:42.518 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:42.518 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:42.518 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:42.518 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22720 2024-05-08T08:53:42.518 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.518 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:42.518 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.519 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:42.519 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098248+0000", 2024-05-08T08:53:42.519 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:42.519 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.519 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.519 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:42.519 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098248+0000", 2024-05-08T08:53:42.519 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:42.519 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.519 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.519 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:42.519 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098491+0000", 2024-05-08T08:53:42.519 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000242659 2024-05-08T08:53:42.519 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.519 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.519 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:42.519 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098495+0000", 2024-05-08T08:53:42.520 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.799e-06 2024-05-08T08:53:42.520 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.520 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.520 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:42.520 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098505+0000", 2024-05-08T08:53:42.520 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7079999999999992e-06 2024-05-08T08:53:42.520 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.520 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.520 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:42.520 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098511+0000", 2024-05-08T08:53:42.520 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4370000000000004e-06 2024-05-08T08:53:42.520 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.520 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.520 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:42.520 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140533+0000", 2024-05-08T08:53:42.520 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042022371000000003 2024-05-08T08:53:42.521 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.521 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.521 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:42.521 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140571+0000", 2024-05-08T08:53:42.521 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7700000000000002e-05 2024-05-08T08:53:42.521 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.521 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.521 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:42.521 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140609+0000", 2024-05-08T08:53:42.521 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8127000000000003e-05 2024-05-08T08:53:42.521 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:42.521 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:42.522 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:42.522 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.522 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.522 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22721 3.15 3:a88fdee9:::rbd_data.1140d3fc54c.000000000000004d:18 [read 1425408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:42.522 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.098522+0000", 2024-05-08T08:53:42.522 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 10.358728183, 2024-05-08T08:53:42.522 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042185779999999999, 2024-05-08T08:53:42.522 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:42.522 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:42.522 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:42.522 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:42.522 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:42.522 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:42.522 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22721 2024-05-08T08:53:42.523 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.523 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:42.523 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.523 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:42.523 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098522+0000", 2024-05-08T08:53:42.523 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:42.523 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.523 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.523 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:42.523 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098522+0000", 2024-05-08T08:53:42.523 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:42.523 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.523 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.523 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:42.523 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098708+0000", 2024-05-08T08:53:42.523 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00018629700000000001 2024-05-08T08:53:42.524 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.524 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.524 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:42.524 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098719+0000", 2024-05-08T08:53:42.524 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0992000000000001e-05 2024-05-08T08:53:42.524 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.524 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.524 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:42.524 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098730+0000", 2024-05-08T08:53:42.524 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0431999999999999e-05 2024-05-08T08:53:42.524 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.524 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.524 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:42.524 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098737+0000", 2024-05-08T08:53:42.524 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.063e-06 2024-05-08T08:53:42.524 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.524 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.525 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:42.525 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140631+0000", 2024-05-08T08:53:42.525 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.041894503999999999 2024-05-08T08:53:42.525 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.525 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.525 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:42.525 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140671+0000", 2024-05-08T08:53:42.525 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9780999999999997e-05 2024-05-08T08:53:42.525 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.525 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.525 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:42.525 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140708+0000", 2024-05-08T08:53:42.525 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6711000000000001e-05 2024-05-08T08:53:42.525 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:42.525 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:42.526 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:42.526 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.526 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.526 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22722 3.15 3:a88fdee9:::rbd_data.1140d3fc54c.000000000000004d:18 [read 1490944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:42.526 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.098753+0000", 2024-05-08T08:53:42.526 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 10.358497179, 2024-05-08T08:53:42.526 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042053944000000003, 2024-05-08T08:53:42.526 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:42.526 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:42.526 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:42.526 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:42.526 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:42.526 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:42.527 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22722 2024-05-08T08:53:42.527 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.527 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:42.527 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.527 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:42.527 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098753+0000", 2024-05-08T08:53:42.527 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:42.527 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.527 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.527 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:42.527 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098753+0000", 2024-05-08T08:53:42.527 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:42.527 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.527 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.527 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:42.527 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098753+0000", 2024-05-08T08:53:42.528 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1399999999999997e-07 2024-05-08T08:53:42.528 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.528 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.528 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:42.528 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098756+0000", 2024-05-08T08:53:42.528 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9780000000000001e-06 2024-05-08T08:53:42.528 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.528 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.528 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:42.528 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098764+0000", 2024-05-08T08:53:42.528 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4340000000000003e-06 2024-05-08T08:53:42.528 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.528 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.528 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:42.528 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098770+0000", 2024-05-08T08:53:42.528 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8080000000000001e-06 2024-05-08T08:53:42.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.529 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.529 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:42.529 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140731+0000", 2024-05-08T08:53:42.529 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.041961131999999998 2024-05-08T08:53:42.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.529 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.529 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:42.529 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140769+0000", 2024-05-08T08:53:42.529 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.773e-05 2024-05-08T08:53:42.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.529 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.529 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:42.529 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140807+0000", 2024-05-08T08:53:42.529 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8347999999999999e-05 2024-05-08T08:53:42.529 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:42.530 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:42.530 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:42.530 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.530 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.530 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22723 3.15 3:a88fdee9:::rbd_data.1140d3fc54c.000000000000004d:18 [read 1556480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:42.530 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.098778+0000", 2024-05-08T08:53:42.530 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 10.358472407000001, 2024-05-08T08:53:42.530 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042128577, 2024-05-08T08:53:42.530 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:42.530 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:42.530 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:42.530 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:42.530 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:42.530 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:42.530 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22723 2024-05-08T08:53:42.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.531 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:42.531 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.531 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:42.531 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098778+0000", 2024-05-08T08:53:42.531 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:42.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.531 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.531 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:42.531 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098778+0000", 2024-05-08T08:53:42.531 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:42.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.531 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.531 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:42.531 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098834+0000", 2024-05-08T08:53:42.531 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5788000000000002e-05 2024-05-08T08:53:42.532 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.532 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.532 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:42.532 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098837+0000", 2024-05-08T08:53:42.532 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3510000000000002e-06 2024-05-08T08:53:42.532 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.533 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.533 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:42.533 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098845+0000", 2024-05-08T08:53:42.533 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5490000000000006e-06 2024-05-08T08:53:42.533 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.533 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.534 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:42.534 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098861+0000", 2024-05-08T08:53:42.534 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5233e-05 2024-05-08T08:53:42.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.534 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.534 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:42.535 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140829+0000", 2024-05-08T08:53:42.535 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.04196863 2024-05-08T08:53:42.535 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.535 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.535 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:42.535 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140869+0000", 2024-05-08T08:53:42.536 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9461e-05 2024-05-08T08:53:42.536 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.536 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.536 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:42.536 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140906+0000", 2024-05-08T08:53:42.536 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7564999999999997e-05 2024-05-08T08:53:42.536 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:42.536 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:42.536 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:42.536 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.536 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.536 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22732 3.15 3:a907109d:::rbd_data.1140d3fc54c.000000000000004f:18 [read 1097728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:42.536 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.098871+0000", 2024-05-08T08:53:42.537 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 10.358378927, 2024-05-08T08:53:42.537 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042141517000000003, 2024-05-08T08:53:42.537 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:42.537 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:42.537 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:42.537 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:42.537 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:42.537 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:42.537 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22732 2024-05-08T08:53:42.537 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.537 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:42.537 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.537 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:42.537 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098871+0000", 2024-05-08T08:53:42.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:42.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.538 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:42.538 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098871+0000", 2024-05-08T08:53:42.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:42.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.538 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:42.538 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.099940+0000", 2024-05-08T08:53:42.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0010683279999999999 2024-05-08T08:53:42.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.538 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:42.539 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.099944+0000", 2024-05-08T08:53:42.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9339999999999997e-06 2024-05-08T08:53:42.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.539 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.539 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:42.539 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.099956+0000", 2024-05-08T08:53:42.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1970999999999999e-05 2024-05-08T08:53:42.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.539 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.539 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:42.539 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.099968+0000", 2024-05-08T08:53:42.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.189e-05 2024-05-08T08:53:42.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.539 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.539 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:42.539 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140930+0000", 2024-05-08T08:53:42.540 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.040961806000000003 2024-05-08T08:53:42.540 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.540 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.540 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:42.540 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140974+0000", 2024-05-08T08:53:42.540 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3918999999999997e-05 2024-05-08T08:53:42.540 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.540 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.540 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:42.540 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.141013+0000", 2024-05-08T08:53:42.540 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8668999999999999e-05 2024-05-08T08:53:42.540 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:42.540 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:42.540 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:42.540 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.540 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.540 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22765 3.15 3:a9c637ae:::rbd_data.114071bcf828.000000000000013f:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:42.541 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.100902+0000", 2024-05-08T08:53:42.541 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 10.356348132000001, 2024-05-08T08:53:42.541 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042471271999999997, 2024-05-08T08:53:42.541 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:42.541 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:42.541 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:42.541 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:42.541 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:42.541 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:42.541 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22765 2024-05-08T08:53:42.541 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.541 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:42.541 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.541 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:42.541 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.100902+0000", 2024-05-08T08:53:42.541 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:42.541 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.542 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.542 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:42.542 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.100902+0000", 2024-05-08T08:53:42.542 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:42.542 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.542 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.542 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:42.542 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.100902+0000", 2024-05-08T08:53:42.542 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.39e-07 2024-05-08T08:53:42.542 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.542 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.542 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:42.542 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.100904+0000", 2024-05-08T08:53:42.542 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6479999999999999e-06 2024-05-08T08:53:42.542 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.543 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:42.543 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.100910+0000", 2024-05-08T08:53:42.543 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1399999999999997e-06 2024-05-08T08:53:42.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.543 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:42.543 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.100916+0000", 2024-05-08T08:53:42.543 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6999999999999996e-06 2024-05-08T08:53:42.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.543 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:42.543 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.141733+0000", 2024-05-08T08:53:42.543 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.040817271000000002 2024-05-08T08:53:42.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.544 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:42.544 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.141812+0000", 2024-05-08T08:53:42.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9270999999999994e-05 2024-05-08T08:53:42.544 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.544 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.544 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:42.544 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.143373+0000", 2024-05-08T08:53:42.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001560903 2024-05-08T08:53:42.544 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:42.544 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:42.544 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:42.544 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.544 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.544 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:23031 3.15 3:ac9fb9cd:::rbd_data.114071bcf828.00000000000000f7:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:42.544 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.113562+0000", 2024-05-08T08:53:42.544 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 10.343688372000001, 2024-05-08T08:53:42.545 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042560938999999999, 2024-05-08T08:53:42.545 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:42.545 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:42.545 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:53:42.545 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:42.545 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:42.545 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:42.545 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 23031 2024-05-08T08:53:42.545 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.545 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:42.545 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.545 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:42.545 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.113562+0000", 2024-05-08T08:53:42.545 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:42.545 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.545 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.546 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:42.546 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.113562+0000", 2024-05-08T08:53:42.546 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:42.546 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.546 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.546 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:42.546 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.113938+0000", 2024-05-08T08:53:42.546 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000376401 2024-05-08T08:53:42.546 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.546 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.546 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:42.546 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.113943+0000", 2024-05-08T08:53:42.546 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0529999999999997e-06 2024-05-08T08:53:42.546 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.547 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:42.547 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.113951+0000", 2024-05-08T08:53:42.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8569999999999995e-06 2024-05-08T08:53:42.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.547 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:42.547 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.113959+0000", 2024-05-08T08:53:42.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4710000000000001e-06 2024-05-08T08:53:42.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.547 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:42.547 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.153955+0000", 2024-05-08T08:53:42.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.039996747999999999 2024-05-08T08:53:42.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.547 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:42.547 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.154014+0000", 2024-05-08T08:53:42.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8464999999999997e-05 2024-05-08T08:53:42.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-05-08T08:53:42.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.154148+0000", 2024-05-08T08:53:42.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00013428600000000001 2024-05-08T08:53:42.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:53:42.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.155390+0000", 2024-05-08T08:53:42.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001241792 2024-05-08T08:53:42.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:53:42.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.156068+0000", 2024-05-08T08:53:42.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00067823700000000005 2024-05-08T08:53:42.549 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.549 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.549 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:53:42.549 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.156102+0000", 2024-05-08T08:53:42.549 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3812000000000001e-05 2024-05-08T08:53:42.549 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:42.549 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:42.549 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:42.549 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.156123+0000", 2024-05-08T08:53:42.549 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0817000000000001e-05 2024-05-08T08:53:42.549 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:42.549 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:42.549 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:42.549 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:42.549 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:42.549 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:53:42.550 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:53:42.551 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:53:42.551 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:53:42.552 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:53:42.552 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:53:42.552 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:42.553+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:42.553 INFO:teuthology.orchestra.run.smithi059.stderr:noscrub is unset 2024-05-08T08:53:42.565 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:53:42.565 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:53:42.583 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:53:42.583 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:53:42.602 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:53:42.603 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:53:42.622 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:53:42.622 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:53:42.624 INFO:teuthology.orchestra.run.smithi196.stdout:truncating image image_client.0-clone26 from 0x7e56a00 (overlap 0x5ff400) to 0x0 2024-05-08T08:53:42.679 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:53:42.682+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:53:42.715 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:42.715 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:42.724 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:53:42.767 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:42.770+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:42.829 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:42.830 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:42.839 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:53:42.854 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:42.856+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:42.953 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:42.956+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:42.987 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:42.987 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:42.996 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:53:43.081 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:43.084+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:43.143 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:43.143 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:43.152 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:53:43.155 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:43.158+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:43.168 INFO:teuthology.orchestra.run.smithi196.stdout:checking clone #25, image image_client.0-clone25 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent26 2024-05-08T08:53:43.231 INFO:teuthology.orchestra.run.smithi196.stdout:3732 punch from 0x2b2cc00 to 0x2b3b400, (0xe800 bytes) 2024-05-08T08:53:43.244 INFO:teuthology.orchestra.run.smithi196.stdout:3733 punch from 0x7622600 to 0x7631200, (0xec00 bytes) 2024-05-08T08:53:43.249 INFO:teuthology.orchestra.run.smithi196.stdout:3734 read 0x5cf0800 thru 0x5cf4f64 (0x4765 bytes) 2024-05-08T08:53:43.249 INFO:teuthology.orchestra.run.smithi196.stdout:3739 write 0x282da00 thru 0x2835271 (0x7872 bytes) 2024-05-08T08:53:43.249 INFO:teuthology.orchestra.run.smithi196.stdout:3742 write 0x5d1a600 thru 0x5d25eb5 (0xb8b6 bytes) 2024-05-08T08:53:43.250 INFO:teuthology.orchestra.run.smithi196.stdout:3744 write 0x776f800 thru 0x777b0de (0xb8df bytes) 2024-05-08T08:53:43.250 INFO:teuthology.orchestra.run.smithi196.stdout:3745 clone 28 order 22 su 65536 sc 11 2024-05-08T08:53:43.256 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:43.259+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:43.300 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:43.301 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:43.309 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:53:43.356 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:43.359+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:43.414 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:43.414 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:43.423 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:53:43.464 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:43.467+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:43.543 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:43.543 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:43.554 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:53:43.598 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:53:43.601+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:53:43.637 INFO:teuthology.orchestra.run.smithi196.stdout:truncating image image_client.0-clone27 from 0x7e56a00 (overlap 0x7e56a00) to 0x586fc00 2024-05-08T08:53:43.659 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:43.662+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:43.687 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:43.687 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:43.699 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:53:43.783 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:53:43.786+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:53:43.851 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:43.851 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:43.860 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:53:43.902 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:43.904+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:43.961 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:43.964+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:44.016 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:44.016 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:44.023 INFO:teuthology.orchestra.run.smithi196.stdout:checking clone #26, image image_client.0-clone26 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent27 2024-05-08T08:53:44.025 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:53:44.069 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:44.072+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:44.088 INFO:teuthology.orchestra.run.smithi196.stdout:3747 trunc from 0x7e56a00 to 0xdd23000 2024-05-08T08:53:44.101 INFO:teuthology.orchestra.run.smithi196.stdout:3748 read 0x9d4e400 thru 0x9d5a6f2 (0xc2f3 bytes) 2024-05-08T08:53:44.102 INFO:teuthology.orchestra.run.smithi196.stdout:3749 write 0xd342000 thru 0xd34b71a (0x971b bytes) 2024-05-08T08:53:44.102 INFO:teuthology.orchestra.run.smithi196.stdout:3754 read 0x26f800 thru 0x27c0a0 (0xc8a1 bytes) 2024-05-08T08:53:44.103 INFO:teuthology.orchestra.run.smithi196.stdout:3755 read 0x4c76c00 thru 0x4c83d78 (0xd179 bytes) 2024-05-08T08:53:44.104 INFO:teuthology.orchestra.run.smithi196.stdout:3759 read 0x9c7e00 thru 0x9d5d54 (0xdf55 bytes) 2024-05-08T08:53:44.105 INFO:teuthology.orchestra.run.smithi196.stdout:3761 write 0xcc65200 thru 0xcc68ae5 (0x38e6 bytes) 2024-05-08T08:53:44.105 INFO:teuthology.orchestra.run.smithi196.stdout:3762 write 0xd6f1000 thru 0xd6fcd4e (0xbd4f bytes) 2024-05-08T08:53:44.113 INFO:teuthology.orchestra.run.smithi196.stdout:3764 trunc from 0xdd23000 to 0x7172c00 2024-05-08T08:53:44.150 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:44.150 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:44.164 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:53:44.164 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:44.166+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:44.185 INFO:teuthology.orchestra.run.smithi196.stdout:3765 trunc from 0x7172c00 to 0xe3e1800 2024-05-08T08:53:44.193 INFO:teuthology.orchestra.run.smithi196.stdout:3766 write 0x7c2ce00 thru 0x7c306cb (0x38cc bytes) 2024-05-08T08:53:44.195 INFO:teuthology.orchestra.run.smithi196.stdout:3768 write 0x7efae00 thru 0x7efff5b (0x515c bytes) 2024-05-08T08:53:44.195 INFO:teuthology.orchestra.run.smithi196.stdout:3769 read 0xa12c200 thru 0xa133e89 (0x7c8a bytes) 2024-05-08T08:53:44.195 INFO:teuthology.orchestra.run.smithi196.stdout:3770 read 0x8d30c00 thru 0x8d3ba56 (0xae57 bytes) 2024-05-08T08:53:44.195 INFO:teuthology.orchestra.run.smithi196.stdout:3771 read 0x8594e00 thru 0x8598679 (0x387a bytes) 2024-05-08T08:53:44.195 INFO:teuthology.orchestra.run.smithi196.stdout:3772 write 0x78cf200 thru 0x78d1931 (0x2732 bytes) 2024-05-08T08:53:44.195 INFO:teuthology.orchestra.run.smithi196.stdout:3773 write 0xad30600 thru 0xad392a5 (0x8ca6 bytes) 2024-05-08T08:53:44.196 INFO:teuthology.orchestra.run.smithi196.stdout:3774 punch from 0xdceaa00 to 0xdcf7800, (0xce00 bytes) 2024-05-08T08:53:44.196 INFO:teuthology.orchestra.run.smithi196.stdout:3775 punch from 0x2981400 to 0x298c600, (0xb200 bytes) 2024-05-08T08:53:44.211 INFO:teuthology.orchestra.run.smithi196.stdout:3777 trunc from 0xe3e1800 to 0x22ba800 2024-05-08T08:53:44.264 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:44.267+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:44.311 INFO:teuthology.orchestra.run.smithi196.stdout:3778 write 0xccb7c00 thru 0xccb873b (0xb3c bytes) 2024-05-08T08:53:44.311 INFO:teuthology.orchestra.run.smithi196.stdout:3779 read 0x466a000 thru 0x4670aa7 (0x6aa8 bytes) 2024-05-08T08:53:44.326 INFO:teuthology.orchestra.run.smithi196.stdout:3781 write 0xec6be00 thru 0xec721dd (0x63de bytes) 2024-05-08T08:53:44.326 INFO:teuthology.orchestra.run.smithi196.stdout:3782 read 0xc4a5c00 thru 0xc4a7aef (0x1ef0 bytes) 2024-05-08T08:53:44.332 INFO:teuthology.orchestra.run.smithi196.stdout:3783 trunc from 0xec72200 to 0xa570e00 2024-05-08T08:53:44.342 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:44.342 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:44.352 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:53:44.352 INFO:teuthology.orchestra.run.smithi196.stdout:3784 write 0x1a99400 thru 0x1a9e49b (0x509c bytes) 2024-05-08T08:53:44.353 INFO:teuthology.orchestra.run.smithi196.stdout:3786 read 0x28d8800 thru 0x28e5f21 (0xd722 bytes) 2024-05-08T08:53:44.354 INFO:teuthology.orchestra.run.smithi196.stdout:3788 write 0x81a5c00 thru 0x81b3abb (0xdebc bytes) 2024-05-08T08:53:44.354 INFO:teuthology.orchestra.run.smithi196.stdout:3789 punch from 0xfd6800 to 0xfd8600, (0x1e00 bytes) 2024-05-08T08:53:44.355 INFO:teuthology.orchestra.run.smithi196.stdout:3791 read 0x5158200 thru 0x5162454 (0xa255 bytes) 2024-05-08T08:53:44.364 INFO:teuthology.orchestra.run.smithi196.stdout:3792 trunc from 0xa570e00 to 0x2db2c00 2024-05-08T08:53:44.378 INFO:teuthology.orchestra.run.smithi196.stdout:3796 read 0x3ff200 thru 0x40e4be (0xf2bf bytes) 2024-05-08T08:53:44.397 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:44.400+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:44.397 INFO:teuthology.orchestra.run.smithi196.stdout:3798 write 0xaab7800 thru 0xaabb79f (0x3fa0 bytes) 2024-05-08T08:53:44.399 INFO:teuthology.orchestra.run.smithi196.stdout:3799 write 0x3256600 thru 0x325e9e6 (0x83e7 bytes) 2024-05-08T08:53:44.399 INFO:teuthology.orchestra.run.smithi196.stdout:3800 write 0x6583c00 thru 0x6591f74 (0xe375 bytes) 2024-05-08T08:53:44.399 INFO:teuthology.orchestra.run.smithi196.stdout:3802 write 0x24b8a00 thru 0x24ba388 (0x1989 bytes) 2024-05-08T08:53:44.399 INFO:teuthology.orchestra.run.smithi196.stdout:3803 write 0x9c73a00 thru 0x9c75f97 (0x2598 bytes) 2024-05-08T08:53:44.399 INFO:teuthology.orchestra.run.smithi196.stdout:3805 punch from 0x84e9e00 to 0x84f1e00, (0x8000 bytes) 2024-05-08T08:53:44.419 INFO:teuthology.orchestra.run.smithi196.stdout:3806 write 0xe505600 thru 0xe505d4f (0x750 bytes) 2024-05-08T08:53:44.423 INFO:teuthology.orchestra.run.smithi196.stdout:3808 trunc from 0xe505e00 to 0xbe58600 2024-05-08T08:53:44.438 INFO:teuthology.orchestra.run.smithi196.stdout:3812 read 0x8c00a00 thru 0x8c018be (0xebf bytes) 2024-05-08T08:53:44.445 INFO:teuthology.orchestra.run.smithi196.stdout:3815 read 0x278f600 thru 0x278fe89 (0x88a bytes) 2024-05-08T08:53:44.446 INFO:teuthology.orchestra.run.smithi196.stdout:3819 punch from 0x6b29400 to 0x6b2c800, (0x3400 bytes) 2024-05-08T08:53:44.448 INFO:teuthology.orchestra.run.smithi196.stdout:3821 read 0x2d42c00 thru 0x2d4e4f8 (0xb8f9 bytes) 2024-05-08T08:53:44.448 INFO:teuthology.orchestra.run.smithi196.stdout:3822 read 0x4db8e00 thru 0x4dbcdc4 (0x3fc5 bytes) 2024-05-08T08:53:44.448 INFO:teuthology.orchestra.run.smithi196.stdout:3825 punch from 0x3147800 to 0x3147c00, (0x400 bytes) 2024-05-08T08:53:44.448 INFO:teuthology.orchestra.run.smithi196.stdout:3826 read 0x830aa00 thru 0x831252e (0x7b2f bytes) 2024-05-08T08:53:44.449 INFO:teuthology.orchestra.run.smithi196.stdout:3827 read 0x82ec200 thru 0x82fa79d (0xe59e bytes) 2024-05-08T08:53:44.449 INFO:teuthology.orchestra.run.smithi196.stdout:3828 punch from 0x7900200 to 0x790b000, (0xae00 bytes) 2024-05-08T08:53:44.449 INFO:teuthology.orchestra.run.smithi196.stdout:3829 read 0x6874200 thru 0x687deee (0x9cef bytes) 2024-05-08T08:53:44.449 INFO:teuthology.orchestra.run.smithi196.stdout:3833 write 0x4a6ba00 thru 0x4a6d2f7 (0x18f8 bytes) 2024-05-08T08:53:44.449 INFO:teuthology.orchestra.run.smithi196.stdout:3834 read 0x374cc00 thru 0x3754d1c (0x811d bytes) 2024-05-08T08:53:44.449 INFO:teuthology.orchestra.run.smithi196.stdout:3836 write 0x7c15600 thru 0x7c20f5f (0xb960 bytes) 2024-05-08T08:53:44.450 INFO:teuthology.orchestra.run.smithi196.stdout:3837 write 0x666ca00 thru 0x6674574 (0x7b75 bytes) 2024-05-08T08:53:44.451 INFO:teuthology.orchestra.run.smithi196.stdout:3838 trunc from 0xbe58600 to 0xd874200 2024-05-08T08:53:44.465 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:44.467+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:44.466 INFO:teuthology.orchestra.run.smithi196.stdout:3839 write 0x1c10800 thru 0x1c1a58d (0x9d8e bytes) 2024-05-08T08:53:44.471 INFO:teuthology.orchestra.run.smithi196.stdout:3841 trunc from 0xd874200 to 0xaa47c00 2024-05-08T08:53:44.472 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:44.472 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:44.482 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:53:44.485 INFO:teuthology.orchestra.run.smithi196.stdout:3844 read 0x600ec00 thru 0x601c996 (0xdd97 bytes) 2024-05-08T08:53:44.506 INFO:teuthology.orchestra.run.smithi196.stdout:3846 write 0xe1ab200 thru 0xe1af526 (0x4327 bytes) 2024-05-08T08:53:44.513 INFO:teuthology.orchestra.run.smithi196.stdout:3847 read 0xadb3600 thru 0xadb53a0 (0x1da1 bytes) 2024-05-08T08:53:44.513 INFO:teuthology.orchestra.run.smithi196.stdout:3849 read 0x78e6200 thru 0x78f2c60 (0xca61 bytes) 2024-05-08T08:53:44.513 INFO:teuthology.orchestra.run.smithi196.stdout:3850 read 0xaa76c00 thru 0xaa775ca (0x9cb bytes) 2024-05-08T08:53:44.514 INFO:teuthology.orchestra.run.smithi196.stdout:3851 read 0x8f79200 thru 0x8f7bca5 (0x2aa6 bytes) 2024-05-08T08:53:44.514 INFO:teuthology.orchestra.run.smithi196.stdout:3854 write 0x84ec00 thru 0x85b0e6 (0xc4e7 bytes) 2024-05-08T08:53:44.514 INFO:teuthology.orchestra.run.smithi196.stdout:3855 punch from 0x1d5d000 to 0x1d64e00, (0x7e00 bytes) 2024-05-08T08:53:44.514 INFO:teuthology.orchestra.run.smithi196.stdout:3857 read 0x7990e00 thru 0x7991f0c (0x110d bytes) 2024-05-08T08:53:44.515 INFO:teuthology.orchestra.run.smithi196.stdout:3859 read 0xb708000 thru 0xb70dae6 (0x5ae7 bytes) 2024-05-08T08:53:44.515 INFO:teuthology.orchestra.run.smithi196.stdout:3864 write 0x9898400 thru 0x9898e51 (0xa52 bytes) 2024-05-08T08:53:44.515 INFO:teuthology.orchestra.run.smithi196.stdout:3865 punch from 0x26c1800 to 0x26ca000, (0x8800 bytes) 2024-05-08T08:53:44.518 INFO:teuthology.orchestra.run.smithi196.stdout:3870 trunc from 0xe1af600 to 0xb39a400 2024-05-08T08:53:44.539 INFO:teuthology.orchestra.run.smithi196.stdout:3873 trunc from 0xb39a400 to 0x4a3b600 2024-05-08T08:53:44.558 INFO:teuthology.orchestra.run.smithi196.stdout:3874 read 0x3ffde00 thru 0x400d5aa (0xf7ab bytes) 2024-05-08T08:53:44.559 INFO:teuthology.orchestra.run.smithi196.stdout:3877 read 0x3603800 thru 0x36137aa (0xffab bytes) 2024-05-08T08:53:44.559 INFO:teuthology.orchestra.run.smithi196.stdout:3885 punch from 0x275600 to 0x27a200, (0x4c00 bytes) 2024-05-08T08:53:44.560 INFO:teuthology.orchestra.run.smithi196.stdout:3886 read 0x2d25000 thru 0x2d2625b (0x125c bytes) 2024-05-08T08:53:44.561 INFO:teuthology.orchestra.run.smithi196.stdout:3890 trunc from 0x4a3b600 to 0x4877800 2024-05-08T08:53:44.564 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-05-08T08:53:44.566 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:44.568+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:44.571 INFO:teuthology.orchestra.run.smithi196.stdout:3892 punch from 0x2d3f800 to 0x2d4e600, (0xee00 bytes) 2024-05-08T08:53:44.574 INFO:teuthology.orchestra.run.smithi196.stdout:3894 read 0x158c000 thru 0x1595b73 (0x9b74 bytes) 2024-05-08T08:53:44.589 INFO:teuthology.orchestra.run.smithi196.stdout:3896 write 0x87de000 thru 0x87e5e4a (0x7e4b bytes) 2024-05-08T08:53:44.598 INFO:teuthology.orchestra.run.smithi196.stdout:3897 trunc from 0x87e6000 to 0x13e2e00 2024-05-08T08:53:44.621 INFO:teuthology.orchestra.run.smithi196.stdout:3901 read 0xc84600 thru 0xc8f741 (0xb142 bytes) 2024-05-08T08:53:44.637 INFO:teuthology.orchestra.run.smithi196.stdout:3902 write 0x5485000 thru 0x5492681 (0xd682 bytes) 2024-05-08T08:53:44.639 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T08:53:44.648 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:44.648 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:44.660 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:53:44.662 INFO:teuthology.orchestra.run.smithi196.stdout:3903 write 0xe2c8600 thru 0xe2d28f1 (0xa2f2 bytes) 2024-05-08T08:53:44.663 INFO:teuthology.orchestra.run.smithi196.stdout:3904 read 0x145ea00 thru 0x14669cd (0x7fce bytes) 2024-05-08T08:53:44.663 INFO:teuthology.orchestra.run.smithi196.stdout:3906 punch from 0x4d7b800 to 0x4d89200, (0xda00 bytes) 2024-05-08T08:53:44.680 INFO:teuthology.orchestra.run.smithi196.stdout:3907 trunc from 0xe2d2a00 to 0x112c800 2024-05-08T08:53:44.702 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:44.705+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:44.715 INFO:teuthology.orchestra.run.smithi196.stdout:3908 write 0x2fcbe00 thru 0x2fdb4de (0xf6df bytes) 2024-05-08T08:53:44.716 INFO:teuthology.orchestra.run.smithi196.stdout:3910 trunc from 0x2fdb600 to 0x35fd000 2024-05-08T08:53:44.744 INFO:teuthology.orchestra.run.smithi196.stdout:3913 write 0x491d000 thru 0x4921bec (0x4bed bytes) 2024-05-08T08:53:44.760 INFO:teuthology.orchestra.run.smithi196.stdout:3915 write 0x65cc800 thru 0x65dc5cb (0xfdcc bytes) 2024-05-08T08:53:44.760 INFO:teuthology.orchestra.run.smithi196.stdout:3917 read 0x5bda00 thru 0x5c8091 (0xa692 bytes) 2024-05-08T08:53:44.761 INFO:teuthology.orchestra.run.smithi196.stdout:3919 read 0x23f5200 thru 0x2403665 (0xe466 bytes) 2024-05-08T08:53:44.761 INFO:teuthology.orchestra.run.smithi196.stdout:3923 read 0x2a44800 thru 0x2a47874 (0x3075 bytes) 2024-05-08T08:53:44.768 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:53:44.771+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:53:44.780 INFO:teuthology.orchestra.run.smithi196.stdout:3926 write 0xb004000 thru 0xb00b8bc (0x78bd bytes) 2024-05-08T08:53:44.786 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:53:44.787 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:53:44.796 INFO:teuthology.orchestra.run.smithi196.stdout:3927 write 0xb52d400 thru 0xb534c29 (0x782a bytes) 2024-05-08T08:53:44.796 INFO:teuthology.orchestra.run.smithi196.stdout:3931 read 0x71dfc00 thru 0x71ed315 (0xd716 bytes) 2024-05-08T08:53:44.796 INFO:teuthology.orchestra.run.smithi196.stdout:3932 read 0x2b08a00 thru 0x2b0cf21 (0x4522 bytes) 2024-05-08T08:53:44.796 INFO:teuthology.orchestra.run.smithi196.stdout:3933 write 0x6eb3c00 thru 0x6ec0a01 (0xce02 bytes) 2024-05-08T08:53:44.797 INFO:teuthology.orchestra.run.smithi196.stdout:3935 write 0x531cc00 thru 0x5323e8d (0x728e bytes) 2024-05-08T08:53:44.797 INFO:teuthology.orchestra.run.smithi196.stdout:3936 write 0x2409e00 thru 0x24193f1 (0xf5f2 bytes) 2024-05-08T08:53:44.797 INFO:teuthology.orchestra.run.smithi196.stdout:3937 write 0x76c4600 thru 0x76cee28 (0xa829 bytes) 2024-05-08T08:53:44.797 INFO:teuthology.orchestra.run.smithi196.stdout:3941 read 0x647de00 thru 0x6482ec8 (0x50c9 bytes) 2024-05-08T08:53:44.798 INFO:teuthology.orchestra.run.smithi196.stdout:3942 write 0x1481000 thru 0x148eb64 (0xdb65 bytes) 2024-05-08T08:53:44.798 INFO:teuthology.orchestra.run.smithi196.stdout:3944 read 0xa963e00 thru 0xa96b8a9 (0x7aaa bytes) 2024-05-08T08:53:44.799 INFO:teuthology.orchestra.run.smithi196.stdout:3945 write 0x8313c00 thru 0x83179f0 (0x3df1 bytes) 2024-05-08T08:53:44.799 INFO:teuthology.orchestra.run.smithi196.stdout:3946 write 0x1a17a00 thru 0x1a1f911 (0x7f12 bytes) 2024-05-08T08:53:44.799 INFO:teuthology.orchestra.run.smithi196.stdout:3947 write 0xa69200 thru 0xa728dd (0x96de bytes) 2024-05-08T08:53:44.799 INFO:teuthology.orchestra.run.smithi196.stdout:3950 write 0xdf7000 thru 0xdfdda0 (0x6da1 bytes) 2024-05-08T08:53:44.800 INFO:teuthology.orchestra.run.smithi196.stdout:3953 read 0x3ca4800 thru 0x3ca480f (0x10 bytes) 2024-05-08T08:53:44.800 INFO:teuthology.orchestra.run.smithi196.stdout:3954 read 0xa016a00 thru 0xa01a2f2 (0x38f3 bytes) 2024-05-08T08:53:44.800 INFO:teuthology.orchestra.run.smithi196.stdout:3955 punch from 0x244c00 to 0x252000, (0xd400 bytes) 2024-05-08T08:53:44.809 INFO:teuthology.orchestra.run.smithi196.stdout:3958 trunc from 0xb534e00 to 0x4e26200 2024-05-08T08:53:44.812 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:53:44.812 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:53:44.835 INFO:teuthology.orchestra.run.smithi196.stdout:3959 punch from 0x3092600 to 0x3099200, (0x6c00 bytes) 2024-05-08T08:53:44.836 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:53:44.836 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:53:44.837 INFO:teuthology.orchestra.run.smithi196.stdout:3960 read 0x5d3a00 thru 0x5dfb95 (0xc196 bytes) 2024-05-08T08:53:44.838 INFO:teuthology.orchestra.run.smithi196.stdout:3962 punch from 0x38e2600 to 0x38e6000, (0x3a00 bytes) 2024-05-08T08:53:44.838 INFO:teuthology.orchestra.run.smithi196.stdout:3963 punch from 0x3b8f600 to 0x3b9ea00, (0xf400 bytes) 2024-05-08T08:53:44.838 INFO:teuthology.orchestra.run.smithi196.stdout:3964 read 0x232c800 thru 0x2338298 (0xba99 bytes) 2024-05-08T08:53:44.838 INFO:teuthology.orchestra.run.smithi196.stdout:3967 write 0x3674c00 thru 0x367d53a (0x893b bytes) 2024-05-08T08:53:44.839 INFO:teuthology.orchestra.run.smithi196.stdout:3968 read 0x45e5400 thru 0x45f2938 (0xd539 bytes) 2024-05-08T08:53:44.841 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:44.841 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:53:44.841 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:53:44.841 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:53:44.841 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.841 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:437 3.17 3:ea93a512:::rbd_data.1140100c9703.0000000000000077:head [set-alloc-hint object_size 2097152 write_size 2097152,write 1170432~1024] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e63)", 2024-05-08T08:53:44.841 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:08.390804+0000", 2024-05-08T08:53:44.842 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 36.452156584000001, 2024-05-08T08:53:44.842 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0406068690000001, 2024-05-08T08:53:44.842 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:44.842 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:44.842 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:53:44.842 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:44.842 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:44.842 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:44.842 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 437 2024-05-08T08:53:44.842 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.842 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:44.842 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.842 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:44.842 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390804+0000", 2024-05-08T08:53:44.843 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:44.843 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.843 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.843 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:44.843 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390804+0000", 2024-05-08T08:53:44.843 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:44.843 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.843 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.843 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:44.843 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390806+0000", 2024-05-08T08:53:44.843 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.666e-06 2024-05-08T08:53:44.843 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.843 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.843 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:44.843 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390818+0000", 2024-05-08T08:53:44.843 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2187999999999999e-05 2024-05-08T08:53:44.844 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.844 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.844 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:44.844 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390834+0000", 2024-05-08T08:53:44.844 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6191000000000001e-05 2024-05-08T08:53:44.844 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.844 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.844 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:44.844 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390845+0000", 2024-05-08T08:53:44.844 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0831e-05 2024-05-08T08:53:44.844 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.844 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.844 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:44.844 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390922+0000", 2024-05-08T08:53:44.844 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7253e-05 2024-05-08T08:53:44.844 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.845 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.845 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op must wait for map", 2024-05-08T08:53:44.845 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390930+0000", 2024-05-08T08:53:44.845 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8259999999999993e-06 2024-05-08T08:53:44.845 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.845 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.845 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:44.845 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392155+0000", 2024-05-08T08:53:44.845 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0012256649999999999 2024-05-08T08:53:44.845 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.845 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.845 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:53:44.845 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392165+0000", 2024-05-08T08:53:44.845 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5799999999999998e-06 2024-05-08T08:53:44.845 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.845 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.846 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:44.846 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.429596+0000", 2024-05-08T08:53:44.846 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0374313639999999 2024-05-08T08:53:44.846 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.846 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.846 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:44.846 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.429728+0000", 2024-05-08T08:53:44.846 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000131552 2024-05-08T08:53:44.846 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.846 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.846 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 0,1", 2024-05-08T08:53:44.846 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.429860+0000", 2024-05-08T08:53:44.846 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000132137 2024-05-08T08:53:44.846 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.846 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.847 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:53:44.847 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.430811+0000", 2024-05-08T08:53:44.847 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00095040200000000004 2024-05-08T08:53:44.847 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.847 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.847 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:53:44.847 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431126+0000", 2024-05-08T08:53:44.847 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000315006 2024-05-08T08:53:44.847 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.847 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.847 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:53:44.847 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431272+0000", 2024-05-08T08:53:44.847 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000146487 2024-05-08T08:53:44.847 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.847 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.848 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:53:44.848 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431315+0000", 2024-05-08T08:53:44.848 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2697000000000001e-05 2024-05-08T08:53:44.848 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.848 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.848 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:44.848 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431411+0000", 2024-05-08T08:53:44.848 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6024000000000001e-05 2024-05-08T08:53:44.848 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:44.848 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:44.848 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:44.848 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.848 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.848 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:439 3.17 3:ea93a512:::rbd_data.1140100c9703.0000000000000077:head [read 1163264~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-05-08T08:53:44.848 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:08.390871+0000", 2024-05-08T08:53:44.848 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 36.452089073000003, 2024-05-08T08:53:44.849 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0408790590000001, 2024-05-08T08:53:44.849 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:44.849 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:44.849 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:44.849 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:44.849 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:44.849 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:44.849 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 439 2024-05-08T08:53:44.849 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.849 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:44.849 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.849 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:44.849 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390871+0000", 2024-05-08T08:53:44.849 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:44.849 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.849 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.849 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:44.850 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390871+0000", 2024-05-08T08:53:44.850 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:44.850 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.850 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.850 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:44.850 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390872+0000", 2024-05-08T08:53:44.850 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6300000000000005e-07 2024-05-08T08:53:44.850 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.850 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.850 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:44.850 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390875+0000", 2024-05-08T08:53:44.850 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3459999999999998e-06 2024-05-08T08:53:44.850 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.850 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.850 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:44.851 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390891+0000", 2024-05-08T08:53:44.851 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5255e-05 2024-05-08T08:53:44.851 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.851 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.851 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:44.851 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390897+0000", 2024-05-08T08:53:44.851 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2360000000000001e-06 2024-05-08T08:53:44.851 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.851 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.851 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:44.851 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390995+0000", 2024-05-08T08:53:44.851 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8522000000000003e-05 2024-05-08T08:53:44.851 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.851 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.851 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting_for_map not empty", 2024-05-08T08:53:44.851 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.391005+0000", 2024-05-08T08:53:44.852 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1460000000000002e-06 2024-05-08T08:53:44.852 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.852 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.852 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:44.852 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392191+0000", 2024-05-08T08:53:44.852 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0011860270000000001 2024-05-08T08:53:44.852 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.852 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.852 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:53:44.852 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392197+0000", 2024-05-08T08:53:44.852 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4289999999999998e-06 2024-05-08T08:53:44.852 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.852 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.852 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:44.852 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.430596+0000", 2024-05-08T08:53:44.852 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0383989090000001 2024-05-08T08:53:44.853 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.853 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.853 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for rw locks", 2024-05-08T08:53:44.853 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.430676+0000", 2024-05-08T08:53:44.853 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9781000000000001e-05 2024-05-08T08:53:44.853 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.853 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.853 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:44.853 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431460+0000", 2024-05-08T08:53:44.853 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00078427300000000004 2024-05-08T08:53:44.853 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.853 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.853 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:44.853 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431540+0000", 2024-05-08T08:53:44.853 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0224999999999998e-05 2024-05-08T08:53:44.853 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.854 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.854 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:44.854 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431750+0000", 2024-05-08T08:53:44.854 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00021024699999999999 2024-05-08T08:53:44.854 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:44.854 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:44.854 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:44.854 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.854 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.854 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:945 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 4096000~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:44.854 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:53:44.854 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 35.391939506, 2024-05-08T08:53:44.854 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086774741000000002, 2024-05-08T08:53:44.854 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:44.854 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:44.855 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:44.855 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:44.855 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:44.855 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:44.855 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 945 2024-05-08T08:53:44.855 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.855 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:44.855 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.855 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:44.855 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:53:44.855 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:44.855 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.855 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.855 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:44.855 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:53:44.855 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:44.856 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.856 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.856 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:44.856 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:53:44.856 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3499999999999996e-07 2024-05-08T08:53:44.856 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.856 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.856 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:44.856 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451024+0000", 2024-05-08T08:53:44.856 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.627e-06 2024-05-08T08:53:44.856 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.856 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.856 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:44.856 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451037+0000", 2024-05-08T08:53:44.856 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2431e-05 2024-05-08T08:53:44.856 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.857 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.857 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:44.857 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451042+0000", 2024-05-08T08:53:44.857 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8409999999999996e-06 2024-05-08T08:53:44.857 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.857 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.857 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:44.857 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537608+0000", 2024-05-08T08:53:44.857 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086565243 2024-05-08T08:53:44.857 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.857 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.857 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:44.857 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537687+0000", 2024-05-08T08:53:44.857 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9519999999999998e-05 2024-05-08T08:53:44.857 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.857 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.857 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:44.858 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537796+0000", 2024-05-08T08:53:44.858 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000108544 2024-05-08T08:53:44.858 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:44.858 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:44.858 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:44.858 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.858 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.858 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:964 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 458752~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:44.858 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451051+0000", 2024-05-08T08:53:44.858 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 35.391909333000001, 2024-05-08T08:53:44.858 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086990490000000004, 2024-05-08T08:53:44.858 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:44.858 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:44.858 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:44.859 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:44.859 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:44.859 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:44.859 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 964 2024-05-08T08:53:44.859 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.859 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:44.859 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.859 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:44.859 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451051+0000", 2024-05-08T08:53:44.859 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:44.859 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.859 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.859 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:44.859 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451051+0000", 2024-05-08T08:53:44.859 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:44.860 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.860 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.860 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:44.860 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451052+0000", 2024-05-08T08:53:44.860 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7299999999999996e-07 2024-05-08T08:53:44.860 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.860 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.860 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:44.860 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451055+0000", 2024-05-08T08:53:44.860 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.2770000000000001e-06 2024-05-08T08:53:44.860 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.860 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.860 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:44.860 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451061+0000", 2024-05-08T08:53:44.860 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4060000000000002e-06 2024-05-08T08:53:44.860 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.861 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.861 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:44.861 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451066+0000", 2024-05-08T08:53:44.861 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0839999999999999e-06 2024-05-08T08:53:44.861 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.861 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.861 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:44.861 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537839+0000", 2024-05-08T08:53:44.861 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086772962999999995 2024-05-08T08:53:44.861 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.861 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.861 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:44.861 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537934+0000", 2024-05-08T08:53:44.861 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4742999999999997e-05 2024-05-08T08:53:44.861 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.861 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.862 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:44.862 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538042+0000", 2024-05-08T08:53:44.862 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000107444 2024-05-08T08:53:44.862 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:44.862 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:44.862 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:44.862 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.862 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.862 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:965 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 950272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:44.862 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451075+0000", 2024-05-08T08:53:44.862 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 35.391885271, 2024-05-08T08:53:44.862 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087230196999999995, 2024-05-08T08:53:44.862 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:44.862 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:44.863 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:44.863 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:44.863 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:44.863 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:44.863 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 965 2024-05-08T08:53:44.863 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.863 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:44.863 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.863 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:44.863 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451075+0000", 2024-05-08T08:53:44.863 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:44.863 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.863 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.863 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:44.863 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451075+0000", 2024-05-08T08:53:44.864 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:44.864 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.864 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.864 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:44.864 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451076+0000", 2024-05-08T08:53:44.864 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5000000000000002e-07 2024-05-08T08:53:44.864 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.864 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.864 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:44.864 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451078+0000", 2024-05-08T08:53:44.864 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.5639999999999999e-06 2024-05-08T08:53:44.864 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.864 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.864 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:44.864 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451085+0000", 2024-05-08T08:53:44.865 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5640000000000002e-06 2024-05-08T08:53:44.865 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.865 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.865 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:44.865 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451090+0000", 2024-05-08T08:53:44.865 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1699999999999996e-06 2024-05-08T08:53:44.865 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.865 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.865 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:44.865 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538106+0000", 2024-05-08T08:53:44.865 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087015888999999999 2024-05-08T08:53:44.865 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.865 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.865 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:44.865 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538190+0000", 2024-05-08T08:53:44.865 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3961999999999993e-05 2024-05-08T08:53:44.866 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.866 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.866 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:44.866 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538305+0000", 2024-05-08T08:53:44.866 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011529800000000001 2024-05-08T08:53:44.866 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:44.866 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:44.866 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:44.866 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.866 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.866 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:966 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 1441792~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:44.866 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:53:44.866 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 35.391861898000002, 2024-05-08T08:53:44.866 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087446323000000006, 2024-05-08T08:53:44.867 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:44.867 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:44.867 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:44.867 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:44.867 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:44.867 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:44.867 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 966 2024-05-08T08:53:44.867 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.867 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:44.867 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.867 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:44.867 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:53:44.867 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:44.867 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.867 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.867 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:44.867 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:53:44.868 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:44.868 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.868 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.868 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:44.868 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:53:44.868 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3700000000000001e-07 2024-05-08T08:53:44.868 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.868 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.868 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:44.868 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451101+0000", 2024-05-08T08:53:44.868 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2560000000000001e-06 2024-05-08T08:53:44.868 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.868 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.868 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:44.868 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451107+0000", 2024-05-08T08:53:44.869 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1070000000000001e-06 2024-05-08T08:53:44.869 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.869 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.869 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:44.869 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451112+0000", 2024-05-08T08:53:44.869 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1420000000000001e-06 2024-05-08T08:53:44.869 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.869 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.869 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:44.869 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538341+0000", 2024-05-08T08:53:44.869 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087228302999999993 2024-05-08T08:53:44.869 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.869 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.869 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:44.869 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538424+0000", 2024-05-08T08:53:44.870 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3497000000000004e-05 2024-05-08T08:53:44.870 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.870 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.870 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:44.870 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538545+0000", 2024-05-08T08:53:44.870 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000120581 2024-05-08T08:53:44.870 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:44.870 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:44.870 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:44.870 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.870 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.870 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:967 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:44.870 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451120+0000", 2024-05-08T08:53:44.870 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 35.391840422999998, 2024-05-08T08:53:44.871 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087632725999999994, 2024-05-08T08:53:44.871 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:44.871 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:44.871 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:44.871 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:44.871 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:44.871 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:44.871 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 967 2024-05-08T08:53:44.871 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.871 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:44.871 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.871 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:44.871 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451120+0000", 2024-05-08T08:53:44.871 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:44.871 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.872 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.872 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:44.872 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451120+0000", 2024-05-08T08:53:44.872 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:44.872 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.872 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.872 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:44.872 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451121+0000", 2024-05-08T08:53:44.872 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8800000000000002e-07 2024-05-08T08:53:44.872 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.872 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.872 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:44.872 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451130+0000", 2024-05-08T08:53:44.872 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2299999999999997e-06 2024-05-08T08:53:44.873 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.873 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.873 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:44.873 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451137+0000", 2024-05-08T08:53:44.873 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4669999999999998e-06 2024-05-08T08:53:44.873 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.873 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.873 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:44.873 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451143+0000", 2024-05-08T08:53:44.873 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2839999999999997e-06 2024-05-08T08:53:44.873 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.873 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.873 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:44.873 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538575+0000", 2024-05-08T08:53:44.873 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087432789999999996 2024-05-08T08:53:44.873 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.874 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.874 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:44.874 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538648+0000", 2024-05-08T08:53:44.874 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2877e-05 2024-05-08T08:53:44.874 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.874 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.874 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:44.874 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538753+0000", 2024-05-08T08:53:44.874 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010449 2024-05-08T08:53:44.874 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:44.874 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:44.874 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:44.874 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.874 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.874 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:968 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 2424832~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:44.875 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451152+0000", 2024-05-08T08:53:44.875 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 35.391808337999997, 2024-05-08T08:53:44.875 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087813325999999997, 2024-05-08T08:53:44.875 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:44.875 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:44.875 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:44.875 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:44.875 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:44.875 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:44.875 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 968 2024-05-08T08:53:44.875 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.875 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:44.875 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.875 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:44.875 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451152+0000", 2024-05-08T08:53:44.875 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:44.876 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.876 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.876 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:44.876 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451152+0000", 2024-05-08T08:53:44.876 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:44.876 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.876 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.876 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:44.876 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451153+0000", 2024-05-08T08:53:44.876 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3099999999999998e-07 2024-05-08T08:53:44.876 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.876 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.876 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:44.876 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451155+0000", 2024-05-08T08:53:44.876 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4760000000000001e-06 2024-05-08T08:53:44.877 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.877 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.877 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:44.877 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451162+0000", 2024-05-08T08:53:44.877 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5969999999999997e-06 2024-05-08T08:53:44.877 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.877 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.877 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:44.877 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451167+0000", 2024-05-08T08:53:44.877 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.977e-06 2024-05-08T08:53:44.877 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.877 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.877 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:44.877 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538781+0000", 2024-05-08T08:53:44.878 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087614846999999996 2024-05-08T08:53:44.878 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.878 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.878 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:44.878 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538857+0000", 2024-05-08T08:53:44.878 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5358000000000006e-05 2024-05-08T08:53:44.878 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.878 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.878 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:44.878 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538965+0000", 2024-05-08T08:53:44.878 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010864 2024-05-08T08:53:44.878 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:44.878 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:44.878 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:44.878 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.879 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.879 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:969 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 2916352~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:44.879 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:53:44.879 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 35.391786633000002, 2024-05-08T08:53:44.879 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087998980000000004, 2024-05-08T08:53:44.879 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:44.879 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:44.879 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:44.879 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:44.879 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:44.879 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:44.879 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 969 2024-05-08T08:53:44.879 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.879 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:44.879 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.880 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:44.880 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:53:44.880 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:44.880 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.880 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.880 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:44.880 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:53:44.880 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:44.880 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.880 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.880 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:44.880 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:53:44.880 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3599999999999999e-07 2024-05-08T08:53:44.880 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.880 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.881 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:44.881 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451177+0000", 2024-05-08T08:53:44.881 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4590000000000001e-06 2024-05-08T08:53:44.881 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.881 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.881 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:44.881 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451184+0000", 2024-05-08T08:53:44.881 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4109999999999999e-06 2024-05-08T08:53:44.881 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.881 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.881 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:44.881 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451191+0000", 2024-05-08T08:53:44.881 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7279999999999998e-06 2024-05-08T08:53:44.881 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.881 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.882 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:44.882 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538997+0000", 2024-05-08T08:53:44.882 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087806170000000003 2024-05-08T08:53:44.882 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.882 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.882 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:44.882 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539069+0000", 2024-05-08T08:53:44.882 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2373999999999995e-05 2024-05-08T08:53:44.882 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.882 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.882 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:44.882 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539173+0000", 2024-05-08T08:53:44.882 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000103402 2024-05-08T08:53:44.883 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:44.883 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:44.883 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:44.883 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.883 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.883 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:970 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 3407872~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:44.883 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451198+0000", 2024-05-08T08:53:44.883 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 35.391762036000003, 2024-05-08T08:53:44.883 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088183368999999998, 2024-05-08T08:53:44.883 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:44.883 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:44.883 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:44.883 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:44.883 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:44.883 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:44.884 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 970 2024-05-08T08:53:44.884 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.884 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:44.884 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.884 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:44.884 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451198+0000", 2024-05-08T08:53:44.884 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:44.884 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.884 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.884 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:44.884 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451198+0000", 2024-05-08T08:53:44.884 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:44.884 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.884 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.884 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:44.885 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451199+0000", 2024-05-08T08:53:44.885 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7400000000000003e-07 2024-05-08T08:53:44.885 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.885 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.885 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:44.885 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451211+0000", 2024-05-08T08:53:44.885 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2109999999999999e-05 2024-05-08T08:53:44.885 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.885 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.885 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:44.885 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451219+0000", 2024-05-08T08:53:44.885 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4170000000000003e-06 2024-05-08T08:53:44.885 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.885 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.886 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:44.886 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451224+0000", 2024-05-08T08:53:44.886 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3990000000000003e-06 2024-05-08T08:53:44.886 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.886 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.886 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:44.886 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539204+0000", 2024-05-08T08:53:44.886 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087980099000000006 2024-05-08T08:53:44.886 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.886 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.886 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:44.886 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539276+0000", 2024-05-08T08:53:44.886 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1513000000000005e-05 2024-05-08T08:53:44.886 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.886 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.887 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:44.887 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539382+0000", 2024-05-08T08:53:44.887 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010615699999999999 2024-05-08T08:53:44.887 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:44.887 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:44.887 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:44.887 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.887 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.887 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:971 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 3899392~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:44.887 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451233+0000", 2024-05-08T08:53:44.887 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 35.391727328999998, 2024-05-08T08:53:44.887 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088431525999999996, 2024-05-08T08:53:44.887 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:44.887 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:44.887 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:44.888 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:44.888 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:44.888 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:44.888 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 971 2024-05-08T08:53:44.888 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.888 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:44.888 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.888 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:44.888 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451233+0000", 2024-05-08T08:53:44.888 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:44.888 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.888 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.888 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:44.888 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451233+0000", 2024-05-08T08:53:44.888 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:44.888 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.889 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.889 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:44.889 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451234+0000", 2024-05-08T08:53:44.889 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8500000000000001e-07 2024-05-08T08:53:44.889 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.889 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.889 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:44.889 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451237+0000", 2024-05-08T08:53:44.889 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8320000000000002e-06 2024-05-08T08:53:44.889 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.889 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.889 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:44.889 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451243+0000", 2024-05-08T08:53:44.889 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6749999999999996e-06 2024-05-08T08:53:44.889 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.889 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.890 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:44.890 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451249+0000", 2024-05-08T08:53:44.890 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1229999999999997e-06 2024-05-08T08:53:44.890 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.890 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.890 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:44.890 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539412+0000", 2024-05-08T08:53:44.890 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088162668999999999 2024-05-08T08:53:44.890 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.890 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.890 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:44.890 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539505+0000", 2024-05-08T08:53:44.890 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3240999999999997e-05 2024-05-08T08:53:44.890 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.890 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.891 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:44.891 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539665+0000", 2024-05-08T08:53:44.891 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00015930100000000001 2024-05-08T08:53:44.891 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:44.891 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:44.891 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:44.891 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.891 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.891 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:972 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 196608~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:44.891 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451257+0000", 2024-05-08T08:53:44.891 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 35.391703384000003, 2024-05-08T08:53:44.891 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088719928000000003, 2024-05-08T08:53:44.891 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:44.891 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:44.891 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:44.892 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:44.892 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:44.892 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:44.892 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 972 2024-05-08T08:53:44.892 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.892 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:44.892 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.892 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:44.892 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451257+0000", 2024-05-08T08:53:44.892 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:44.892 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.892 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.892 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:44.892 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451257+0000", 2024-05-08T08:53:44.892 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:44.893 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.893 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.893 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:44.893 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451258+0000", 2024-05-08T08:53:44.893 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7999999999999996e-07 2024-05-08T08:53:44.893 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.893 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.893 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:44.893 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451260+0000", 2024-05-08T08:53:44.893 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4609999999999998e-06 2024-05-08T08:53:44.893 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.893 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.893 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:44.893 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451267+0000", 2024-05-08T08:53:44.894 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.81e-06 2024-05-08T08:53:44.894 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.894 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.894 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:44.894 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451273+0000", 2024-05-08T08:53:44.894 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0859999999999998e-06 2024-05-08T08:53:44.894 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.894 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.894 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:44.894 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539718+0000", 2024-05-08T08:53:44.894 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088445059000000006 2024-05-08T08:53:44.894 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.894 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.894 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:44.895 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539834+0000", 2024-05-08T08:53:44.895 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000116426 2024-05-08T08:53:44.895 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.895 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.895 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:44.895 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539977+0000", 2024-05-08T08:53:44.895 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014260600000000001 2024-05-08T08:53:44.895 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:44.895 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:44.895 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:44.895 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.895 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.895 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:973 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 688128~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:44.895 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:53:44.895 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 35.391680800000003, 2024-05-08T08:53:44.895 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088918496, 2024-05-08T08:53:44.896 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:44.896 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:44.896 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:44.896 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:44.896 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:44.896 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:44.896 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 973 2024-05-08T08:53:44.896 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.896 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:44.896 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.896 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:44.896 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:53:44.896 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:44.896 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.896 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.896 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:44.897 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:53:44.897 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:44.897 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.897 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.897 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:44.897 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:53:44.897 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9099999999999999e-07 2024-05-08T08:53:44.897 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.897 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.897 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:44.897 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451282+0000", 2024-05-08T08:53:44.897 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1550000000000001e-06 2024-05-08T08:53:44.897 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.897 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.897 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:44.897 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451288+0000", 2024-05-08T08:53:44.898 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8889999999999999e-06 2024-05-08T08:53:44.898 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.898 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.898 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:44.898 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451294+0000", 2024-05-08T08:53:44.898 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0850000000000002e-06 2024-05-08T08:53:44.898 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.898 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.898 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:44.898 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540011+0000", 2024-05-08T08:53:44.898 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088717314000000005 2024-05-08T08:53:44.898 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.898 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.898 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:44.898 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540089+0000", 2024-05-08T08:53:44.899 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7438e-05 2024-05-08T08:53:44.899 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.899 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.899 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:44.899 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540198+0000", 2024-05-08T08:53:44.899 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000109224 2024-05-08T08:53:44.899 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:44.899 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:44.899 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:44.899 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.899 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.899 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:974 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 1179648~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:44.899 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451301+0000", 2024-05-08T08:53:44.899 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 35.391659167, 2024-05-08T08:53:44.899 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089105078000000004, 2024-05-08T08:53:44.900 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:44.900 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:44.900 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:44.900 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:44.900 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:44.900 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:44.900 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 974 2024-05-08T08:53:44.900 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.900 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:44.900 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.900 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:44.900 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451301+0000", 2024-05-08T08:53:44.900 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:44.900 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.901 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.901 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:44.901 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451301+0000", 2024-05-08T08:53:44.901 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:44.901 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.901 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.901 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:44.901 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451302+0000", 2024-05-08T08:53:44.901 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7800000000000002e-07 2024-05-08T08:53:44.901 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.901 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.901 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:44.901 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451304+0000", 2024-05-08T08:53:44.901 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.65e-06 2024-05-08T08:53:44.901 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.901 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.902 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:44.902 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451311+0000", 2024-05-08T08:53:44.902 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.472e-06 2024-05-08T08:53:44.902 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.902 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.902 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:44.902 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451316+0000", 2024-05-08T08:53:44.902 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9189999999999998e-06 2024-05-08T08:53:44.902 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.902 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.902 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:44.902 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540234+0000", 2024-05-08T08:53:44.902 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088917752000000003 2024-05-08T08:53:44.902 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.903 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.903 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:44.903 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540308+0000", 2024-05-08T08:53:44.903 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4758000000000005e-05 2024-05-08T08:53:44.903 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.903 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.903 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:44.903 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540406+0000", 2024-05-08T08:53:44.903 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8048999999999996e-05 2024-05-08T08:53:44.903 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:44.903 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:44.903 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:44.903 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.903 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.903 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:975 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 1671168~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:44.904 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:53:44.904 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 35.391616837000001, 2024-05-08T08:53:44.904 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089294139999999994, 2024-05-08T08:53:44.904 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:44.904 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:44.904 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:44.904 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:44.904 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:44.904 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:44.904 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 975 2024-05-08T08:53:44.904 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.904 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:44.904 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.904 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:44.904 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:53:44.904 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:44.905 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.905 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.905 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:44.905 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:53:44.905 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:44.905 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.905 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.905 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:44.905 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:53:44.905 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1299999999999999e-07 2024-05-08T08:53:44.905 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.905 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.905 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:44.906 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451348+0000", 2024-05-08T08:53:44.906 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.484e-06 2024-05-08T08:53:44.906 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.906 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.906 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:44.906 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451356+0000", 2024-05-08T08:53:44.906 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8499999999999994e-06 2024-05-08T08:53:44.906 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.906 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.906 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:44.906 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451366+0000", 2024-05-08T08:53:44.906 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.9850000000000001e-06 2024-05-08T08:53:44.906 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.906 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.907 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:44.907 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540448+0000", 2024-05-08T08:53:44.907 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089082254 2024-05-08T08:53:44.907 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.907 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.907 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:44.907 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540528+0000", 2024-05-08T08:53:44.907 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0234999999999993e-05 2024-05-08T08:53:44.907 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.907 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.907 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:44.907 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540638+0000", 2024-05-08T08:53:44.907 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010951900000000001 2024-05-08T08:53:44.907 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:44.907 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:44.908 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:44.908 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.908 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.908 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:976 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 2162688~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:44.908 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451375+0000", 2024-05-08T08:53:44.908 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 35.39158509, 2024-05-08T08:53:44.908 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089472745000000006, 2024-05-08T08:53:44.908 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:44.908 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:44.908 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:44.908 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:44.908 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:44.908 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:44.908 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 976 2024-05-08T08:53:44.909 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.909 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:44.909 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.909 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:44.909 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451375+0000", 2024-05-08T08:53:44.909 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:44.909 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.909 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.909 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:44.909 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451375+0000", 2024-05-08T08:53:44.909 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:44.909 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.909 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.909 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:44.910 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451376+0000", 2024-05-08T08:53:44.910 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2399999999999998e-07 2024-05-08T08:53:44.910 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.910 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.910 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:44.910 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451379+0000", 2024-05-08T08:53:44.910 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6359999999999998e-06 2024-05-08T08:53:44.910 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.910 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.910 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:44.910 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451387+0000", 2024-05-08T08:53:44.910 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9929999999999995e-06 2024-05-08T08:53:44.910 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.910 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.910 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:44.911 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451393+0000", 2024-05-08T08:53:44.911 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4699999999999999e-06 2024-05-08T08:53:44.911 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.911 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.911 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:44.911 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540677+0000", 2024-05-08T08:53:44.911 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089283504999999999 2024-05-08T08:53:44.911 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.911 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.911 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:44.911 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540743+0000", 2024-05-08T08:53:44.911 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6303999999999994e-05 2024-05-08T08:53:44.911 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.911 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.911 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:44.912 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540848+0000", 2024-05-08T08:53:44.912 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000105213 2024-05-08T08:53:44.912 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:44.912 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:44.912 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:44.912 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.912 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.912 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:977 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 2654208~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:44.912 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:53:44.912 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 35.391559657999998, 2024-05-08T08:53:44.912 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089698473000000001, 2024-05-08T08:53:44.912 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:44.912 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:44.912 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:44.912 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:44.912 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:44.913 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:44.913 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 977 2024-05-08T08:53:44.913 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.913 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:44.913 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.913 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:44.913 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:53:44.913 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:44.913 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.913 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.913 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:44.913 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:53:44.913 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:44.913 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.914 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.914 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:44.914 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:53:44.914 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2899999999999999e-07 2024-05-08T08:53:44.914 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.914 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.914 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:44.914 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451404+0000", 2024-05-08T08:53:44.914 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6649999999999999e-06 2024-05-08T08:53:44.914 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.914 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.914 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:44.914 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451410+0000", 2024-05-08T08:53:44.914 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3269999999999997e-06 2024-05-08T08:53:44.914 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.915 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.915 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:44.915 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451415+0000", 2024-05-08T08:53:44.915 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9529999999999998e-06 2024-05-08T08:53:44.915 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.915 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.915 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:44.915 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540912+0000", 2024-05-08T08:53:44.915 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089496950000000006 2024-05-08T08:53:44.915 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.915 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.915 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:44.915 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541000+0000", 2024-05-08T08:53:44.915 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7429000000000006e-05 2024-05-08T08:53:44.915 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.916 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.916 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:44.916 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541099+0000", 2024-05-08T08:53:44.916 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.9720000000000001e-05 2024-05-08T08:53:44.916 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:44.916 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:44.916 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:44.916 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.916 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.916 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:978 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 3145728~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:44.916 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:53:44.916 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 35.391538449999999, 2024-05-08T08:53:44.916 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089900635000000007, 2024-05-08T08:53:44.916 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:44.916 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:44.916 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:44.917 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:44.917 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:44.917 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:44.917 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 978 2024-05-08T08:53:44.917 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.917 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:44.917 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.917 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:44.917 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:53:44.917 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:44.917 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.917 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.917 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:44.917 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:53:44.918 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:44.918 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.918 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.918 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:44.918 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:53:44.918 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9400000000000001e-07 2024-05-08T08:53:44.918 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.918 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.918 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:44.918 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451425+0000", 2024-05-08T08:53:44.918 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2460000000000002e-06 2024-05-08T08:53:44.918 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.918 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.918 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:44.919 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451431+0000", 2024-05-08T08:53:44.919 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2419999999999997e-06 2024-05-08T08:53:44.919 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.919 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.919 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:44.919 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451436+0000", 2024-05-08T08:53:44.919 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.887e-06 2024-05-08T08:53:44.919 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.919 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.919 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:44.919 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541152+0000", 2024-05-08T08:53:44.919 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089716194999999999 2024-05-08T08:53:44.919 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.919 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.919 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:44.920 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541224+0000", 2024-05-08T08:53:44.920 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2416000000000006e-05 2024-05-08T08:53:44.920 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.920 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.920 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:44.920 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541323+0000", 2024-05-08T08:53:44.920 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8255e-05 2024-05-08T08:53:44.920 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:44.920 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:44.920 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:44.920 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.920 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.920 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:979 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 3637248~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:44.920 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451462+0000", 2024-05-08T08:53:44.920 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 35.391498001000002, 2024-05-08T08:53:44.921 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.090036441999999994, 2024-05-08T08:53:44.921 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:44.921 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:44.921 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:44.921 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:44.921 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:44.921 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:44.921 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 979 2024-05-08T08:53:44.921 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.921 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:44.921 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.921 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:44.921 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451462+0000", 2024-05-08T08:53:44.921 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:44.921 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.922 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.922 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:44.922 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451462+0000", 2024-05-08T08:53:44.922 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:44.922 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.922 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.922 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:44.922 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451463+0000", 2024-05-08T08:53:44.922 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4300000000000002e-07 2024-05-08T08:53:44.922 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.922 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.922 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:44.922 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451471+0000", 2024-05-08T08:53:44.922 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0800000000000006e-06 2024-05-08T08:53:44.923 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.923 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.923 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:44.923 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451481+0000", 2024-05-08T08:53:44.923 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.38e-06 2024-05-08T08:53:44.923 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.923 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.923 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:44.923 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451487+0000", 2024-05-08T08:53:44.923 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4590000000000003e-06 2024-05-08T08:53:44.923 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.923 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.923 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:44.923 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541350+0000", 2024-05-08T08:53:44.923 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089862599000000001 2024-05-08T08:53:44.923 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.924 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.924 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:44.924 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541407+0000", 2024-05-08T08:53:44.924 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7667999999999997e-05 2024-05-08T08:53:44.924 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.924 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.924 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:44.924 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541499+0000", 2024-05-08T08:53:44.924 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1513000000000003e-05 2024-05-08T08:53:44.924 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:44.924 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:44.924 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:44.924 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.924 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.924 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:980 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 4128768~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:44.925 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451496+0000", 2024-05-08T08:53:44.925 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 35.391464245000002, 2024-05-08T08:53:44.925 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.090258985, 2024-05-08T08:53:44.925 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:44.925 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:44.925 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:44.925 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:44.925 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:44.925 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:44.925 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 980 2024-05-08T08:53:44.925 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.925 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:44.925 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.925 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:44.925 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451496+0000", 2024-05-08T08:53:44.926 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:44.926 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.926 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.926 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:44.926 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451496+0000", 2024-05-08T08:53:44.926 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:44.926 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.926 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.926 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:44.926 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451497+0000", 2024-05-08T08:53:44.926 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7599999999999997e-07 2024-05-08T08:53:44.926 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.926 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.926 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:44.926 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451500+0000", 2024-05-08T08:53:44.927 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7930000000000002e-06 2024-05-08T08:53:44.927 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.927 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.927 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:44.927 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451506+0000", 2024-05-08T08:53:44.927 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6980000000000001e-06 2024-05-08T08:53:44.927 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.927 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.927 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:44.927 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451512+0000", 2024-05-08T08:53:44.927 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2490000000000001e-06 2024-05-08T08:53:44.927 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.927 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.927 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:44.927 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541550+0000", 2024-05-08T08:53:44.928 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.090037961999999999 2024-05-08T08:53:44.928 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.928 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.928 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:44.928 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541647+0000", 2024-05-08T08:53:44.928 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6924999999999998e-05 2024-05-08T08:53:44.928 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:44.928 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:44.928 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:44.928 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541755+0000", 2024-05-08T08:53:44.928 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000108682 2024-05-08T08:53:44.928 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:44.928 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:44.928 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:44.929 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:44.929 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:44.929 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:44.931 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:53:44.931 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:53:44.932 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:53:44.932 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:53:44.933 INFO:teuthology.orchestra.run.smithi196.stdout:3969 trunc from 0x4e26200 to 0x7d13c00 2024-05-08T08:53:44.933 INFO:teuthology.orchestra.run.smithi196.stdout:3970 write 0x6a1ae00 thru 0x6a1b42c (0x62d bytes) 2024-05-08T08:53:44.933 INFO:teuthology.orchestra.run.smithi196.stdout:3971 punch from 0x25f1800 to 0x25f7600, (0x5e00 bytes) 2024-05-08T08:53:44.933 INFO:teuthology.orchestra.run.smithi196.stdout:3972 write 0xa99be00 thru 0xa9a0ebe (0x50bf bytes) 2024-05-08T08:53:44.933 INFO:teuthology.orchestra.run.smithi196.stdout:3973 write 0xbb60000 thru 0xbb65d43 (0x5d44 bytes) 2024-05-08T08:53:44.933 INFO:teuthology.orchestra.run.smithi196.stdout:3974 read 0x603ba00 thru 0x6049ac2 (0xe0c3 bytes) 2024-05-08T08:53:44.933 INFO:teuthology.orchestra.run.smithi196.stdout:3975 trunc from 0xbb65e00 to 0xd99b000 2024-05-08T08:53:44.933 INFO:teuthology.orchestra.run.smithi196.stdout:3981 punch from 0x595ac00 to 0x5968800, (0xdc00 bytes) 2024-05-08T08:53:44.933 INFO:teuthology.orchestra.run.smithi196.stdout:3982 read 0xce17e00 thru 0xce18b3c (0xd3d bytes) 2024-05-08T08:53:44.933 INFO:teuthology.orchestra.run.smithi196.stdout:3984 write 0xe304a00 thru 0xe30941f (0x4a20 bytes) 2024-05-08T08:53:44.933 INFO:teuthology.orchestra.run.smithi196.stdout:3986 trunc from 0xe309600 to 0x735fa00 2024-05-08T08:53:44.933 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:53:44.934 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:53:44.934 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:53:44.935 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:44.933+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:44.946 INFO:teuthology.orchestra.run.smithi196.stdout:3987 punch from 0x2134e00 to 0x2137e00, (0x3000 bytes) 2024-05-08T08:53:44.949 INFO:teuthology.orchestra.run.smithi196.stdout:3988 flatten 2024-05-08T08:53:44.961 INFO:teuthology.orchestra.run.smithi196.stdout:3989 read 0x5e2200 thru 0x5eda7b (0xb87c bytes) 2024-05-08T08:53:44.962 INFO:teuthology.orchestra.run.smithi196.stdout:3990 write 0x2299800 thru 0x229e2d4 (0x4ad5 bytes) 2024-05-08T08:53:44.975 INFO:teuthology.orchestra.run.smithi196.stdout:3993 write 0x89f4a00 thru 0x8a03015 (0xe616 bytes) 2024-05-08T08:53:44.977 INFO:teuthology.orchestra.run.smithi196.stdout:3995 read 0x8111a00 thru 0x811f7ee (0xddef bytes) 2024-05-08T08:53:44.977 INFO:teuthology.orchestra.run.smithi196.stdout:3996 read 0x6b8bc00 thru 0x6b91fda (0x63db bytes) 2024-05-08T08:53:44.977 INFO:teuthology.orchestra.run.smithi196.stdout:3997 punch from 0x6f7c800 to 0x6f80400, (0x3c00 bytes) 2024-05-08T08:53:44.978 INFO:teuthology.orchestra.run.smithi196.stdout:3998 read 0x4a35600 thru 0x4a396a4 (0x40a5 bytes) 2024-05-08T08:53:44.988 INFO:teuthology.orchestra.run.smithi196.stdout:4000 trunc from 0x8a03200 to 0xd9d0600 2024-05-08T08:53:44.998 INFO:teuthology.orchestra.run.smithi196.stdout:4004 write 0x2973e00 thru 0x297d715 (0x9916 bytes) 2024-05-08T08:53:44.999 INFO:teuthology.orchestra.run.smithi196.stdout:4005 write 0x600de00 thru 0x601551b (0x771c bytes) 2024-05-08T08:53:45.001 INFO:teuthology.orchestra.run.smithi196.stdout:4006 trunc from 0xd9d0600 to 0xc453a00 2024-05-08T08:53:45.031 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:45.034+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:45.032 INFO:teuthology.orchestra.run.smithi196.stdout:4010 trunc from 0xc453a00 to 0x47bbe00 2024-05-08T08:53:45.055 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:53:45.055 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:53:45.055 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:53:45.055 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:53:45.064 INFO:teuthology.orchestra.run.smithi196.stdout:4011 write 0x5856800 thru 0x585a3b6 (0x3bb7 bytes) 2024-05-08T08:53:45.065 INFO:teuthology.orchestra.run.smithi196.stdout:4016 write 0x4a2b600 thru 0x4a32ba2 (0x75a3 bytes) 2024-05-08T08:53:45.066 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:53:45.066 INFO:teuthology.orchestra.run.smithi196.stdout:4017 read 0x3052200 thru 0x305336a (0x116b bytes) 2024-05-08T08:53:45.067 INFO:teuthology.orchestra.run.smithi196.stdout:4018 punch from 0x37cc400 to 0x37d5e00, (0x9a00 bytes) 2024-05-08T08:53:45.067 INFO:teuthology.orchestra.run.smithi196.stdout:4020 write 0x3f8f600 thru 0x3f9da0d (0xe40e bytes) 2024-05-08T08:53:45.071 INFO:teuthology.orchestra.run.smithi196.stdout:4021 trunc from 0x585a400 to 0x1beec00 2024-05-08T08:53:45.120 INFO:teuthology.orchestra.run.smithi196.stdout:4023 write 0x8fa6600 thru 0x8fb5b54 (0xf555 bytes) 2024-05-08T08:53:45.120 INFO:teuthology.orchestra.run.smithi196.stdout:4024 read 0x1bad200 thru 0x1bb55b7 (0x83b8 bytes) 2024-05-08T08:53:45.120 INFO:teuthology.orchestra.run.smithi196.stdout:4025 punch from 0x609fc00 to 0x60a4600, (0x4a00 bytes) 2024-05-08T08:53:45.121 INFO:teuthology.orchestra.run.smithi196.stdout:4026 write 0x16b8800 thru 0x16bbfb4 (0x37b5 bytes) 2024-05-08T08:53:45.126 INFO:teuthology.orchestra.run.smithi196.stdout:4027 trunc from 0x8fb5c00 to 0x4b12e00 2024-05-08T08:53:45.131 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:45.134+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:45.141 INFO:teuthology.orchestra.run.smithi196.stdout:4028 read 0xd6b200 thru 0xd6d912 (0x2713 bytes) 2024-05-08T08:53:45.142 INFO:teuthology.orchestra.run.smithi196.stdout:4029 write 0x1afa200 thru 0x1b086cd (0xe4ce bytes) 2024-05-08T08:53:45.143 INFO:teuthology.orchestra.run.smithi196.stdout:4030 write 0x4622200 thru 0x462576f (0x3570 bytes) 2024-05-08T08:53:45.160 INFO:teuthology.orchestra.run.smithi196.stdout:4032 write 0x89be400 thru 0x89c9bca (0xb7cb bytes) 2024-05-08T08:53:45.161 INFO:teuthology.orchestra.run.smithi196.stdout:4034 write 0x971800 thru 0x97a815 (0x9016 bytes) 2024-05-08T08:53:45.162 INFO:teuthology.orchestra.run.smithi196.stdout:4035 read 0x4c3b600 thru 0x4c44be1 (0x95e2 bytes) 2024-05-08T08:53:45.162 INFO:teuthology.orchestra.run.smithi196.stdout:4036 write 0x79f8e00 thru 0x79ff2a0 (0x64a1 bytes) 2024-05-08T08:53:45.162 INFO:teuthology.orchestra.run.smithi196.stdout:4037 punch from 0x4903e00 to 0x4905600, (0x1800 bytes) 2024-05-08T08:53:45.163 INFO:teuthology.orchestra.run.smithi196.stdout:4038 read 0x6bc7000 thru 0x6bc8126 (0x1127 bytes) 2024-05-08T08:53:45.163 INFO:teuthology.orchestra.run.smithi196.stdout:4041 write 0x1d1ee00 thru 0x1d29ab3 (0xacb4 bytes) 2024-05-08T08:53:45.163 INFO:teuthology.orchestra.run.smithi196.stdout:4042 read 0x5434c00 thru 0x543a67f (0x5a80 bytes) 2024-05-08T08:53:45.164 INFO:teuthology.orchestra.run.smithi196.stdout:4043 write 0x573a00 thru 0x578f80 (0x5581 bytes) 2024-05-08T08:53:45.164 INFO:teuthology.orchestra.run.smithi196.stdout:4046 punch from 0x7d3c000 to 0x7d3f000, (0x3000 bytes) 2024-05-08T08:53:45.164 INFO:teuthology.orchestra.run.smithi196.stdout:4048 write 0x1b73600 thru 0x1b76da7 (0x37a8 bytes) 2024-05-08T08:53:45.167 INFO:teuthology.orchestra.run.smithi196.stdout:4049 write 0x5ce5c00 thru 0x5cef457 (0x9858 bytes) 2024-05-08T08:53:45.167 INFO:teuthology.orchestra.run.smithi196.stdout:4052 write 0x440f200 thru 0x441d99d (0xe79e bytes) 2024-05-08T08:53:45.167 INFO:teuthology.orchestra.run.smithi196.stdout:4055 write 0x15fd600 thru 0x160465c (0x705d bytes) 2024-05-08T08:53:45.190 INFO:teuthology.orchestra.run.smithi196.stdout:4057 write 0xdc6dc00 thru 0xdc727f0 (0x4bf1 bytes) 2024-05-08T08:53:45.192 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:53:45.192 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:53:45.192 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:53:45.192 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:53:45.192 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:53:45.202 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:53:45.206 INFO:teuthology.orchestra.run.smithi196.stdout:4058 write 0xed48a00 thru 0xed57348 (0xe949 bytes) 2024-05-08T08:53:45.221 INFO:teuthology.orchestra.run.smithi196.stdout:4059 trunc from 0xed57400 to 0x2473000 2024-05-08T08:53:45.232 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:45.235+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:45.250 INFO:teuthology.orchestra.run.smithi196.stdout:4060 read 0x17c7400 thru 0x17c823e (0xe3f bytes) 2024-05-08T08:53:45.267 INFO:teuthology.orchestra.run.smithi196.stdout:4061 write 0x7375000 thru 0x7375e07 (0xe08 bytes) 2024-05-08T08:53:45.268 INFO:teuthology.orchestra.run.smithi196.stdout:4062 read 0xface00 thru 0xfb602f (0x9230 bytes) 2024-05-08T08:53:45.273 INFO:teuthology.orchestra.run.smithi196.stdout:4063 punch from 0x1136a00 to 0x1138400, (0x1a00 bytes) 2024-05-08T08:53:45.274 INFO:teuthology.orchestra.run.smithi196.stdout:4065 write 0x52aaa00 thru 0x52aca07 (0x2008 bytes) 2024-05-08T08:53:45.274 INFO:teuthology.orchestra.run.smithi196.stdout:4066 punch from 0x71afa00 to 0x71b0a00, (0x1000 bytes) 2024-05-08T08:53:45.274 INFO:teuthology.orchestra.run.smithi196.stdout:4068 read 0x6694a00 thru 0x669783a (0x2e3b bytes) 2024-05-08T08:53:45.274 INFO:teuthology.orchestra.run.smithi196.stdout:4069 write 0x98de00 thru 0x993587 (0x5788 bytes) 2024-05-08T08:53:45.275 INFO:teuthology.orchestra.run.smithi196.stdout:4070 read 0x4569600 thru 0x4573177 (0x9b78 bytes) 2024-05-08T08:53:45.278 INFO:teuthology.orchestra.run.smithi196.stdout:4073 trunc from 0x7376000 to 0x4843000 2024-05-08T08:53:45.297 INFO:teuthology.orchestra.run.smithi196.stdout:4074 punch from 0x6d1400 to 0x6d3e00, (0x2a00 bytes) 2024-05-08T08:53:45.300 INFO:teuthology.orchestra.run.smithi196.stdout:4076 punch from 0x1c08400 to 0x1c0a600, (0x2200 bytes) 2024-05-08T08:53:45.300 INFO:teuthology.orchestra.run.smithi196.stdout:4079 punch from 0x30c0800 to 0x30d0200, (0xfa00 bytes) 2024-05-08T08:53:45.319 INFO:teuthology.orchestra.run.smithi196.stdout:4080 write 0x9709600 thru 0x9718b71 (0xf572 bytes) 2024-05-08T08:53:45.319 INFO:teuthology.orchestra.run.smithi196.stdout:4082 write 0x351aa00 thru 0x3529fa4 (0xf5a5 bytes) 2024-05-08T08:53:45.320 INFO:teuthology.orchestra.run.smithi196.stdout:4084 write 0x1667c00 thru 0x16742ba (0xc6bb bytes) 2024-05-08T08:53:45.330 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:53:45.330 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:53:45.330 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:53:45.330 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:53:45.330 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.330 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4621.0:9089 3.1f e33/18 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:head v 33'117, mlcod=31'115)", 2024-05-08T08:53:45.330 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:53:45.330 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 72.811799686000001, 2024-05-08T08:53:45.330 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.111244524, 2024-05-08T08:53:45.331 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:45.331 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:45.331 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:53:45.331 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:45.331 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.331 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:45.331 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:53:45.331 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:45.331 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.331 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.331 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:45.331 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:53:45.331 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:45.331 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.331 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.332 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:45.332 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:53:45.332 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5900000000000001e-07 2024-05-08T08:53:45.332 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.332 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.332 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:45.332 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539600+0000", 2024-05-08T08:53:45.332 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.019981364000000001 2024-05-08T08:53:45.332 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.332 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.332 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:45.332 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539624+0000", 2024-05-08T08:53:45.332 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4043e-05 2024-05-08T08:53:45.332 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.332 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.332 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:45.333 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539635+0000", 2024-05-08T08:53:45.333 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1511e-05 2024-05-08T08:53:45.333 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.333 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.333 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:45.333 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539728+0000", 2024-05-08T08:53:45.333 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2498999999999996e-05 2024-05-08T08:53:45.333 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.333 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.333 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:45.333 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539746+0000", 2024-05-08T08:53:45.333 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8289000000000001e-05 2024-05-08T08:53:45.333 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.333 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.333 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:53:45.333 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.630822+0000", 2024-05-08T08:53:45.334 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.091076283999999993 2024-05-08T08:53:45.334 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.334 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.334 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:45.334 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.630863+0000", 2024-05-08T08:53:45.334 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0374999999999999e-05 2024-05-08T08:53:45.334 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.334 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:45.334 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.334 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.334 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.334 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4795.0:230 3.1f e40/18 3:f8db1a56:::rbd_data.11402b26385.0000000000000000:head v 40'171, mlcod=39'168)", 2024-05-08T08:53:45.334 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:45.927773+0000", 2024-05-08T08:53:45.334 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 59.403644602999996, 2024-05-08T08:53:45.335 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045892405999999997, 2024-05-08T08:53:45.335 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:45.335 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:45.335 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:53:45.335 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:45.335 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.335 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:45.335 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.927773+0000", 2024-05-08T08:53:45.335 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:45.335 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.335 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.335 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:45.335 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.927773+0000", 2024-05-08T08:53:45.335 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:45.335 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.336 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.336 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:45.336 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.927774+0000", 2024-05-08T08:53:45.336 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0500000000000002e-07 2024-05-08T08:53:45.336 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.336 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.336 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:45.336 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.931581+0000", 2024-05-08T08:53:45.336 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0038071799999999999 2024-05-08T08:53:45.336 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.336 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.336 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:45.336 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.931599+0000", 2024-05-08T08:53:45.336 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7479e-05 2024-05-08T08:53:45.336 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.337 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.337 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:45.337 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.931624+0000", 2024-05-08T08:53:45.337 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5372999999999999e-05 2024-05-08T08:53:45.337 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.337 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.337 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:45.337 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.931698+0000", 2024-05-08T08:53:45.337 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4043999999999998e-05 2024-05-08T08:53:45.337 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.337 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.337 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:45.337 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.931725+0000", 2024-05-08T08:53:45.337 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6883e-05 2024-05-08T08:53:45.338 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.338 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.338 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:53:45.338 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.973623+0000", 2024-05-08T08:53:45.338 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.041898283000000001 2024-05-08T08:53:45.338 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.338 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.338 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:45.338 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.973666+0000", 2024-05-08T08:53:45.338 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2358999999999999e-05 2024-05-08T08:53:45.338 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.338 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:45.338 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.338 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.339 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.339 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4195.0:4 3.11 3:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-05-08T08:53:45.339 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:49.764654+0000", 2024-05-08T08:53:45.339 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 55.566764145, 2024-05-08T08:53:45.339 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.30714275499999999, 2024-05-08T08:53:45.339 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:45.339 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:45.339 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-05-08T08:53:45.339 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:45.339 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4195", 2024-05-08T08:53:45.339 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/59179521", 2024-05-08T08:53:45.339 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4 2024-05-08T08:53:45.340 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.340 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:45.340 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.340 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:45.340 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764654+0000", 2024-05-08T08:53:45.340 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:45.340 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.340 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.340 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:45.340 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764654+0000", 2024-05-08T08:53:45.340 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:45.340 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.340 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.340 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:45.340 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764655+0000", 2024-05-08T08:53:45.341 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.452e-06 2024-05-08T08:53:45.341 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.341 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.341 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:45.341 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764661+0000", 2024-05-08T08:53:45.341 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5369999999999997e-06 2024-05-08T08:53:45.341 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.341 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.341 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:45.341 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764671+0000", 2024-05-08T08:53:45.341 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.028e-05 2024-05-08T08:53:45.341 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.341 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.341 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:45.341 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764680+0000", 2024-05-08T08:53:45.341 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0629999999999995e-06 2024-05-08T08:53:45.342 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.342 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.342 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:45.342 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764753+0000", 2024-05-08T08:53:45.342 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2679e-05 2024-05-08T08:53:45.342 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.342 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.342 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-05-08T08:53:45.342 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764765+0000", 2024-05-08T08:53:45.342 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2415000000000001e-05 2024-05-08T08:53:45.342 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.342 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.342 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:45.342 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:50.071692+0000", 2024-05-08T08:53:45.343 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.306927324 2024-05-08T08:53:45.343 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.343 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.343 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:45.343 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:50.071796+0000", 2024-05-08T08:53:45.343 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104005 2024-05-08T08:53:45.343 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.343 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:45.343 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.343 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.343 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.343 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22681 3.15 3:a907109d:::rbd_data.1140d3fc54c.000000000000004f:18 [read 1204224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:45.343 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.097653+0000", 2024-05-08T08:53:45.344 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 13.233764517000001, 2024-05-08T08:53:45.344 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.041941285000000002, 2024-05-08T08:53:45.344 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:45.344 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:45.344 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:45.344 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:45.344 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:45.344 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:45.344 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22681 2024-05-08T08:53:45.344 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.344 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:45.344 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.344 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:45.344 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097653+0000", 2024-05-08T08:53:45.344 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:45.345 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.345 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:45.345 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097653+0000", 2024-05-08T08:53:45.345 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:45.345 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.345 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:45.345 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097654+0000", 2024-05-08T08:53:45.345 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9400000000000001e-07 2024-05-08T08:53:45.345 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.345 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:45.345 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097655+0000", 2024-05-08T08:53:45.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6220000000000001e-06 2024-05-08T08:53:45.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.346 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:45.346 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097661+0000", 2024-05-08T08:53:45.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5210000000000002e-06 2024-05-08T08:53:45.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.346 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:45.346 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097665+0000", 2024-05-08T08:53:45.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3170000000000003e-06 2024-05-08T08:53:45.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.346 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:45.346 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139509+0000", 2024-05-08T08:53:45.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.041843475999999998 2024-05-08T08:53:45.347 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.347 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.347 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:45.347 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139554+0000", 2024-05-08T08:53:45.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5478000000000001e-05 2024-05-08T08:53:45.347 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.347 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.347 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:45.347 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139595+0000", 2024-05-08T08:53:45.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0577000000000001e-05 2024-05-08T08:53:45.347 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.347 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:45.347 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.347 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.347 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.347 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22682 3.15 3:a907109d:::rbd_data.1140d3fc54c.000000000000004f:18 [read 1269760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:45.348 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.097670+0000", 2024-05-08T08:53:45.348 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 13.233747825, 2024-05-08T08:53:45.348 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042031394, 2024-05-08T08:53:45.348 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:45.348 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:45.348 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:45.348 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:45.348 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:45.348 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:45.348 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22682 2024-05-08T08:53:45.348 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.348 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:45.348 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.348 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:45.348 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097670+0000", 2024-05-08T08:53:45.348 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:45.349 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.349 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.349 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:45.349 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097670+0000", 2024-05-08T08:53:45.349 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:45.349 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.349 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.349 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:45.349 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097670+0000", 2024-05-08T08:53:45.349 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1800000000000002e-07 2024-05-08T08:53:45.349 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.349 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.349 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:45.349 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097672+0000", 2024-05-08T08:53:45.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6929999999999999e-06 2024-05-08T08:53:45.350 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.350 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.350 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:45.350 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097677+0000", 2024-05-08T08:53:45.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2480000000000004e-06 2024-05-08T08:53:45.350 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.350 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.350 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:45.350 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097682+0000", 2024-05-08T08:53:45.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4290000000000003e-06 2024-05-08T08:53:45.350 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.350 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.350 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:45.351 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139620+0000", 2024-05-08T08:53:45.351 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.041938256 2024-05-08T08:53:45.351 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.351 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.351 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:45.351 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139662+0000", 2024-05-08T08:53:45.351 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1727999999999998e-05 2024-05-08T08:53:45.351 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.351 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.351 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:45.351 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139701+0000", 2024-05-08T08:53:45.351 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9721999999999997e-05 2024-05-08T08:53:45.351 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.351 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:45.351 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.351 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.351 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.352 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22683 3.15 3:a907109d:::rbd_data.1140d3fc54c.000000000000004f:18 [read 1335296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:45.352 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.097686+0000", 2024-05-08T08:53:45.352 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 13.233731471, 2024-05-08T08:53:45.352 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042114716000000003, 2024-05-08T08:53:45.352 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:45.352 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:45.352 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:45.352 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:45.352 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:45.352 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:45.352 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22683 2024-05-08T08:53:45.352 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.352 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:45.352 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.352 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:45.352 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097686+0000", 2024-05-08T08:53:45.353 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:45.353 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.353 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.353 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:45.353 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097686+0000", 2024-05-08T08:53:45.353 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:45.353 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.353 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.353 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:45.353 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097687+0000", 2024-05-08T08:53:45.353 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3999999999999997e-07 2024-05-08T08:53:45.353 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.353 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.353 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:45.354 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097688+0000", 2024-05-08T08:53:45.354 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6789999999999999e-06 2024-05-08T08:53:45.354 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.354 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.354 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:45.354 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097693+0000", 2024-05-08T08:53:45.354 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.045e-06 2024-05-08T08:53:45.354 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.354 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.354 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:45.354 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097697+0000", 2024-05-08T08:53:45.354 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0260000000000001e-06 2024-05-08T08:53:45.354 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.354 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.354 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:45.355 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139725+0000", 2024-05-08T08:53:45.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042027737000000003 2024-05-08T08:53:45.355 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.355 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.355 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:45.355 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139764+0000", 2024-05-08T08:53:45.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8668999999999999e-05 2024-05-08T08:53:45.355 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.355 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.355 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:45.355 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139801+0000", 2024-05-08T08:53:45.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7219999999999999e-05 2024-05-08T08:53:45.355 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.355 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:45.355 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.356 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.356 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.356 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22684 3.15 3:a907109d:::rbd_data.1140d3fc54c.000000000000004f:18 [read 1400832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:45.356 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.097703+0000", 2024-05-08T08:53:45.356 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 13.233715117999999, 2024-05-08T08:53:45.356 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042198929000000003, 2024-05-08T08:53:45.356 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:45.356 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:45.356 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:45.356 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:45.356 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:45.356 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:45.356 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22684 2024-05-08T08:53:45.356 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.356 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:45.356 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.357 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:45.357 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097703+0000", 2024-05-08T08:53:45.357 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:45.357 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.357 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.357 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:45.357 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097703+0000", 2024-05-08T08:53:45.357 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:45.357 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.357 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.357 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:45.357 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097703+0000", 2024-05-08T08:53:45.357 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4400000000000001e-07 2024-05-08T08:53:45.357 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.358 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.358 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:45.358 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097705+0000", 2024-05-08T08:53:45.358 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9199999999999998e-06 2024-05-08T08:53:45.358 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.358 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.358 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:45.358 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097711+0000", 2024-05-08T08:53:45.358 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5559999999999998e-06 2024-05-08T08:53:45.358 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.358 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.358 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:45.358 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097715+0000", 2024-05-08T08:53:45.358 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3159999999999998e-06 2024-05-08T08:53:45.358 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.359 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.359 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:45.359 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139825+0000", 2024-05-08T08:53:45.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042110284999999997 2024-05-08T08:53:45.359 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.359 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.359 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:45.359 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139864+0000", 2024-05-08T08:53:45.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8980000000000003e-05 2024-05-08T08:53:45.359 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.359 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.359 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:45.359 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139902+0000", 2024-05-08T08:53:45.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7527999999999997e-05 2024-05-08T08:53:45.359 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.359 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:45.360 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.360 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.360 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.360 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22685 3.15 3:a907109d:::rbd_data.1140d3fc54c.000000000000004f:18 [read 1466368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:45.360 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.097721+0000", 2024-05-08T08:53:45.360 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 13.233696460999999, 2024-05-08T08:53:45.360 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042287791999999998, 2024-05-08T08:53:45.360 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:45.360 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:45.360 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:45.360 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:45.360 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:45.360 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:45.360 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22685 2024-05-08T08:53:45.360 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.361 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:45.361 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.361 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:45.361 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097721+0000", 2024-05-08T08:53:45.361 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:45.361 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.361 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.361 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:45.361 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097721+0000", 2024-05-08T08:53:45.361 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:45.361 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.361 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.361 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:45.361 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097722+0000", 2024-05-08T08:53:45.361 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2500000000000001e-07 2024-05-08T08:53:45.362 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.362 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.362 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:45.362 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097724+0000", 2024-05-08T08:53:45.362 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.835e-06 2024-05-08T08:53:45.362 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.362 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.362 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:45.362 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097729+0000", 2024-05-08T08:53:45.362 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5029999999999997e-06 2024-05-08T08:53:45.362 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.362 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.362 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:45.362 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097733+0000", 2024-05-08T08:53:45.363 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3309999999999997e-06 2024-05-08T08:53:45.363 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.363 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.363 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:45.363 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139923+0000", 2024-05-08T08:53:45.363 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042189988999999997 2024-05-08T08:53:45.363 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.363 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.363 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:45.363 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139970+0000", 2024-05-08T08:53:45.363 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6776000000000001e-05 2024-05-08T08:53:45.363 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.363 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.363 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:45.363 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140009+0000", 2024-05-08T08:53:45.364 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9032999999999997e-05 2024-05-08T08:53:45.364 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.364 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:45.364 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.364 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.364 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.364 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22686 3.15 3:a907109d:::rbd_data.1140d3fc54c.000000000000004f:18 [read 1531904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:45.364 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.097739+0000", 2024-05-08T08:53:45.364 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 13.233678462, 2024-05-08T08:53:45.364 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042371008000000002, 2024-05-08T08:53:45.364 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:45.364 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:45.364 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:45.364 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:45.364 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:45.364 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:45.365 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22686 2024-05-08T08:53:45.365 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.365 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:45.365 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.365 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:45.365 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097739+0000", 2024-05-08T08:53:45.365 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:45.365 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.365 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.365 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:45.365 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097739+0000", 2024-05-08T08:53:45.365 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:45.365 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.365 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.365 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:45.365 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097740+0000", 2024-05-08T08:53:45.366 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8599999999999999e-07 2024-05-08T08:53:45.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.366 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.366 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:45.366 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097742+0000", 2024-05-08T08:53:45.366 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9410000000000001e-06 2024-05-08T08:53:45.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.366 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.366 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:45.366 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097747+0000", 2024-05-08T08:53:45.366 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2079999999999999e-06 2024-05-08T08:53:45.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.366 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.366 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:45.367 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097751+0000", 2024-05-08T08:53:45.367 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2250000000000002e-06 2024-05-08T08:53:45.367 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.367 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.367 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:45.367 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140032+0000", 2024-05-08T08:53:45.367 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042281244000000003 2024-05-08T08:53:45.367 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.367 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.367 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:45.367 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140074+0000", 2024-05-08T08:53:45.367 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1316999999999998e-05 2024-05-08T08:53:45.367 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.367 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.367 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:45.367 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140110+0000", 2024-05-08T08:53:45.367 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6687000000000003e-05 2024-05-08T08:53:45.368 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.368 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:45.368 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.368 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.368 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.368 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22716 3.15 3:a88fdee9:::rbd_data.1140d3fc54c.000000000000004d:18 [read 1097728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:45.368 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.098153+0000", 2024-05-08T08:53:45.368 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 13.233264755, 2024-05-08T08:53:45.368 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042058535000000001, 2024-05-08T08:53:45.368 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:45.368 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:45.368 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:45.368 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:45.368 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:45.368 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:45.368 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22716 2024-05-08T08:53:45.369 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.369 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:45.369 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.369 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:45.369 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098153+0000", 2024-05-08T08:53:45.369 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:45.369 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.369 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.369 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:45.369 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098153+0000", 2024-05-08T08:53:45.369 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:45.369 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.369 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.369 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:45.369 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098154+0000", 2024-05-08T08:53:45.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.13e-07 2024-05-08T08:53:45.370 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.370 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.370 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:45.370 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098161+0000", 2024-05-08T08:53:45.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7360000000000001e-06 2024-05-08T08:53:45.370 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.370 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.370 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:45.370 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098171+0000", 2024-05-08T08:53:45.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1039999999999996e-06 2024-05-08T08:53:45.370 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.370 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.370 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:45.371 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098177+0000", 2024-05-08T08:53:45.371 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5760000000000002e-06 2024-05-08T08:53:45.371 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.371 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.371 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:45.371 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140135+0000", 2024-05-08T08:53:45.371 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.041957842000000002 2024-05-08T08:53:45.371 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.371 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.371 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:45.371 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140174+0000", 2024-05-08T08:53:45.371 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8902999999999997e-05 2024-05-08T08:53:45.371 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.371 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.372 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:45.372 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140212+0000", 2024-05-08T08:53:45.372 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7761e-05 2024-05-08T08:53:45.372 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.372 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:45.372 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.372 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.372 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.372 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22717 3.15 3:a88fdee9:::rbd_data.1140d3fc54c.000000000000004d:18 [read 1163264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:45.372 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.098188+0000", 2024-05-08T08:53:45.372 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 13.233230033, 2024-05-08T08:53:45.372 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042124907000000003, 2024-05-08T08:53:45.372 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:45.372 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:45.372 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:45.373 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:45.373 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:45.373 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:45.373 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22717 2024-05-08T08:53:45.373 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.373 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:45.373 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.373 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:45.373 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098188+0000", 2024-05-08T08:53:45.373 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:45.373 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.373 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.373 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:45.373 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098188+0000", 2024-05-08T08:53:45.373 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:45.373 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.374 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.374 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:45.374 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098188+0000", 2024-05-08T08:53:45.374 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3799999999999998e-07 2024-05-08T08:53:45.374 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.374 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.374 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:45.374 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098190+0000", 2024-05-08T08:53:45.374 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.114e-06 2024-05-08T08:53:45.374 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.374 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.374 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:45.374 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098197+0000", 2024-05-08T08:53:45.374 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9959999999999996e-06 2024-05-08T08:53:45.375 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.375 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.375 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:45.375 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098202+0000", 2024-05-08T08:53:45.375 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9110000000000001e-06 2024-05-08T08:53:45.375 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.375 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.375 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:45.375 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140235+0000", 2024-05-08T08:53:45.375 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042032913999999998 2024-05-08T08:53:45.375 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.375 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.375 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:45.375 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140275+0000", 2024-05-08T08:53:45.375 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9564999999999998e-05 2024-05-08T08:53:45.375 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.376 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.376 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:45.376 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140313+0000", 2024-05-08T08:53:45.376 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7969000000000002e-05 2024-05-08T08:53:45.376 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.376 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:45.376 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.376 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.376 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.376 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22718 3.15 3:a88fdee9:::rbd_data.1140d3fc54c.000000000000004d:18 [read 1228800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:45.376 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.098209+0000", 2024-05-08T08:53:45.376 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 13.233208887, 2024-05-08T08:53:45.376 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.04220409, 2024-05-08T08:53:45.376 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:45.376 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:45.377 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:45.377 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:45.377 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:45.377 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:45.377 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22718 2024-05-08T08:53:45.377 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.377 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:45.377 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.377 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:45.377 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098209+0000", 2024-05-08T08:53:45.377 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:45.377 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.377 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.377 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:45.377 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098209+0000", 2024-05-08T08:53:45.378 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:45.378 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.378 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.378 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:45.378 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098209+0000", 2024-05-08T08:53:45.378 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9299999999999999e-07 2024-05-08T08:53:45.378 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.378 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.378 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:45.378 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098212+0000", 2024-05-08T08:53:45.378 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.21e-06 2024-05-08T08:53:45.378 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.378 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.378 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:45.378 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098219+0000", 2024-05-08T08:53:45.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0269999999999998e-06 2024-05-08T08:53:45.379 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.379 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.379 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:45.379 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098223+0000", 2024-05-08T08:53:45.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7319999999999998e-06 2024-05-08T08:53:45.379 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.379 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.379 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:45.379 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140338+0000", 2024-05-08T08:53:45.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042114390000000002 2024-05-08T08:53:45.379 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.379 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.379 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:45.379 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140376+0000", 2024-05-08T08:53:45.380 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8414999999999997e-05 2024-05-08T08:53:45.380 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.380 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.380 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:45.380 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140413+0000", 2024-05-08T08:53:45.380 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6922999999999999e-05 2024-05-08T08:53:45.380 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.380 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:45.380 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.380 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.380 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.380 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22719 3.15 3:a88fdee9:::rbd_data.1140d3fc54c.000000000000004d:18 [read 1294336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:45.380 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.098229+0000", 2024-05-08T08:53:45.380 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 13.233189036000001, 2024-05-08T08:53:45.380 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042283016, 2024-05-08T08:53:45.380 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:45.381 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:45.381 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:45.381 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:45.381 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:45.381 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:45.381 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22719 2024-05-08T08:53:45.381 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.381 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:45.381 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.381 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:45.381 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098229+0000", 2024-05-08T08:53:45.381 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:45.381 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.381 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.382 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:45.382 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098229+0000", 2024-05-08T08:53:45.382 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:45.382 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.382 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.382 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:45.382 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098229+0000", 2024-05-08T08:53:45.382 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:53:45.382 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.382 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.382 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:45.382 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098231+0000", 2024-05-08T08:53:45.382 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.81e-06 2024-05-08T08:53:45.383 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.383 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.383 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:45.383 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098237+0000", 2024-05-08T08:53:45.383 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3389999999999998e-06 2024-05-08T08:53:45.383 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.383 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.383 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:45.383 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098243+0000", 2024-05-08T08:53:45.383 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.648e-06 2024-05-08T08:53:45.383 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.383 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.383 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:45.383 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140436+0000", 2024-05-08T08:53:45.384 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042192738 2024-05-08T08:53:45.384 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.384 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:45.384 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140475+0000", 2024-05-08T08:53:45.384 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9279999999999997e-05 2024-05-08T08:53:45.384 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.384 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:45.384 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140512+0000", 2024-05-08T08:53:45.384 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6847000000000001e-05 2024-05-08T08:53:45.384 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.384 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:45.384 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.384 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.385 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.385 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22720 3.15 3:a88fdee9:::rbd_data.1140d3fc54c.000000000000004d:18 [read 1359872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:45.385 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.098248+0000", 2024-05-08T08:53:45.385 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 13.233169524999999, 2024-05-08T08:53:45.385 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042360800999999997, 2024-05-08T08:53:45.385 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:45.385 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:45.385 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:45.385 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:45.385 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:45.385 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:45.385 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22720 2024-05-08T08:53:45.385 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.385 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:45.385 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.386 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:45.386 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098248+0000", 2024-05-08T08:53:45.386 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:45.386 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.386 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.386 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:45.386 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098248+0000", 2024-05-08T08:53:45.386 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:45.386 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.386 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.386 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:45.386 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098491+0000", 2024-05-08T08:53:45.386 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000242659 2024-05-08T08:53:45.386 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.387 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.387 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:45.387 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098495+0000", 2024-05-08T08:53:45.387 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.799e-06 2024-05-08T08:53:45.387 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.387 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.387 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:45.387 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098505+0000", 2024-05-08T08:53:45.387 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7079999999999992e-06 2024-05-08T08:53:45.387 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.387 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.387 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:45.387 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098511+0000", 2024-05-08T08:53:45.387 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4370000000000004e-06 2024-05-08T08:53:45.388 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.388 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.388 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:45.388 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140533+0000", 2024-05-08T08:53:45.388 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042022371000000003 2024-05-08T08:53:45.388 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.388 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.388 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:45.388 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140571+0000", 2024-05-08T08:53:45.388 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7700000000000002e-05 2024-05-08T08:53:45.388 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.388 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.388 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:45.388 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140609+0000", 2024-05-08T08:53:45.389 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8127000000000003e-05 2024-05-08T08:53:45.389 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.389 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:45.389 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.389 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.389 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.389 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22721 3.15 3:a88fdee9:::rbd_data.1140d3fc54c.000000000000004d:18 [read 1425408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:45.389 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.098522+0000", 2024-05-08T08:53:45.389 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 13.232895900999999, 2024-05-08T08:53:45.389 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042185779999999999, 2024-05-08T08:53:45.389 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:45.389 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:45.389 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:45.389 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:45.389 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:45.390 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:45.390 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22721 2024-05-08T08:53:45.390 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.390 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:45.390 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.390 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:45.390 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098522+0000", 2024-05-08T08:53:45.390 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:45.390 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.390 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.390 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:45.390 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098522+0000", 2024-05-08T08:53:45.390 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:45.390 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.390 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.391 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:45.391 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098708+0000", 2024-05-08T08:53:45.391 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00018629700000000001 2024-05-08T08:53:45.391 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.391 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.391 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:45.391 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098719+0000", 2024-05-08T08:53:45.391 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0992000000000001e-05 2024-05-08T08:53:45.391 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.391 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.391 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:45.391 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098730+0000", 2024-05-08T08:53:45.391 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0431999999999999e-05 2024-05-08T08:53:45.391 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.391 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.392 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:45.392 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098737+0000", 2024-05-08T08:53:45.392 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.063e-06 2024-05-08T08:53:45.392 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.392 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.392 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:45.392 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140631+0000", 2024-05-08T08:53:45.392 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.041894503999999999 2024-05-08T08:53:45.392 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.392 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.392 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:45.392 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140671+0000", 2024-05-08T08:53:45.392 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9780999999999997e-05 2024-05-08T08:53:45.393 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.393 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.393 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:45.393 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140708+0000", 2024-05-08T08:53:45.393 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6711000000000001e-05 2024-05-08T08:53:45.393 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.393 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:45.393 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.393 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.393 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.393 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22722 3.15 3:a88fdee9:::rbd_data.1140d3fc54c.000000000000004d:18 [read 1490944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:45.393 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.098753+0000", 2024-05-08T08:53:45.393 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 13.232664896999999, 2024-05-08T08:53:45.393 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042053944000000003, 2024-05-08T08:53:45.393 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:45.394 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:45.394 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:45.394 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:45.394 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:45.394 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:45.394 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22722 2024-05-08T08:53:45.394 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.394 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:45.394 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.394 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:45.394 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098753+0000", 2024-05-08T08:53:45.394 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:45.394 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.394 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.394 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:45.395 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098753+0000", 2024-05-08T08:53:45.395 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:45.395 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.395 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.395 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:45.395 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098753+0000", 2024-05-08T08:53:45.395 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1399999999999997e-07 2024-05-08T08:53:45.395 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.395 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.395 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:45.395 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098756+0000", 2024-05-08T08:53:45.395 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9780000000000001e-06 2024-05-08T08:53:45.395 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.395 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.395 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:45.396 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098764+0000", 2024-05-08T08:53:45.396 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4340000000000003e-06 2024-05-08T08:53:45.396 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.396 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.396 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:45.396 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098770+0000", 2024-05-08T08:53:45.396 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8080000000000001e-06 2024-05-08T08:53:45.396 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.396 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.396 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:45.396 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140731+0000", 2024-05-08T08:53:45.396 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.041961131999999998 2024-05-08T08:53:45.396 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.396 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.396 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:45.397 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140769+0000", 2024-05-08T08:53:45.397 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.773e-05 2024-05-08T08:53:45.397 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.397 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.397 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:45.397 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140807+0000", 2024-05-08T08:53:45.397 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8347999999999999e-05 2024-05-08T08:53:45.397 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.397 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:45.397 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.397 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.397 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.397 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22723 3.15 3:a88fdee9:::rbd_data.1140d3fc54c.000000000000004d:18 [read 1556480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:45.397 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.098778+0000", 2024-05-08T08:53:45.397 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 13.232640125, 2024-05-08T08:53:45.398 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042128577, 2024-05-08T08:53:45.398 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:45.398 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:45.398 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:45.398 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:45.398 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:45.398 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:45.398 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22723 2024-05-08T08:53:45.398 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.398 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:45.398 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.398 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:45.398 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098778+0000", 2024-05-08T08:53:45.398 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:45.399 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.399 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.399 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:45.399 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098778+0000", 2024-05-08T08:53:45.399 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:45.399 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.399 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.399 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:45.399 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098834+0000", 2024-05-08T08:53:45.399 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5788000000000002e-05 2024-05-08T08:53:45.399 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.399 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.399 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:45.400 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098837+0000", 2024-05-08T08:53:45.400 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3510000000000002e-06 2024-05-08T08:53:45.400 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.400 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.400 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:45.400 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098845+0000", 2024-05-08T08:53:45.400 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5490000000000006e-06 2024-05-08T08:53:45.400 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.400 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.400 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:45.400 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098861+0000", 2024-05-08T08:53:45.400 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5233e-05 2024-05-08T08:53:45.400 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.400 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.400 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:45.401 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140829+0000", 2024-05-08T08:53:45.401 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.04196863 2024-05-08T08:53:45.401 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.401 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.401 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:45.401 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140869+0000", 2024-05-08T08:53:45.401 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9461e-05 2024-05-08T08:53:45.401 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.401 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.401 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:45.401 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140906+0000", 2024-05-08T08:53:45.401 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7564999999999997e-05 2024-05-08T08:53:45.401 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.401 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:45.402 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.402 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.402 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.402 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22732 3.15 3:a907109d:::rbd_data.1140d3fc54c.000000000000004f:18 [read 1097728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:45.402 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.098871+0000", 2024-05-08T08:53:45.402 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 13.232546644999999, 2024-05-08T08:53:45.402 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042141517000000003, 2024-05-08T08:53:45.402 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:45.402 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:45.402 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:45.402 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:45.402 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:45.402 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:45.402 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22732 2024-05-08T08:53:45.403 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.403 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:45.403 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.403 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:45.403 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098871+0000", 2024-05-08T08:53:45.403 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:45.403 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.403 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.403 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:45.403 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098871+0000", 2024-05-08T08:53:45.403 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:45.403 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.403 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.403 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:45.404 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.099940+0000", 2024-05-08T08:53:45.404 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0010683279999999999 2024-05-08T08:53:45.404 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.404 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.404 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:45.404 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.099944+0000", 2024-05-08T08:53:45.404 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9339999999999997e-06 2024-05-08T08:53:45.404 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.404 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.404 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:45.404 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.099956+0000", 2024-05-08T08:53:45.404 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1970999999999999e-05 2024-05-08T08:53:45.404 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.404 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.405 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:45.405 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.099968+0000", 2024-05-08T08:53:45.405 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.189e-05 2024-05-08T08:53:45.405 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.405 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.405 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:45.405 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140930+0000", 2024-05-08T08:53:45.405 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.040961806000000003 2024-05-08T08:53:45.405 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.405 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.405 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:45.405 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140974+0000", 2024-05-08T08:53:45.405 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3918999999999997e-05 2024-05-08T08:53:45.405 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.405 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.406 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:45.406 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.141013+0000", 2024-05-08T08:53:45.406 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8668999999999999e-05 2024-05-08T08:53:45.406 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.406 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:45.406 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.406 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.406 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.406 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22765 3.15 3:a9c637ae:::rbd_data.114071bcf828.000000000000013f:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:45.406 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.100902+0000", 2024-05-08T08:53:45.406 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 13.23051585, 2024-05-08T08:53:45.406 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042471271999999997, 2024-05-08T08:53:45.406 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:45.406 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:45.409 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:45.409 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:45.410 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:45.410 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:45.410 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22765 2024-05-08T08:53:45.410 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.410 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:45.410 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.410 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:45.410 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.100902+0000", 2024-05-08T08:53:45.410 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:45.410 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.410 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.410 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:45.410 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.100902+0000", 2024-05-08T08:53:45.410 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:45.411 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.411 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.411 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:45.411 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.100902+0000", 2024-05-08T08:53:45.411 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.39e-07 2024-05-08T08:53:45.411 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.411 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.411 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:45.411 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.100904+0000", 2024-05-08T08:53:45.411 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6479999999999999e-06 2024-05-08T08:53:45.411 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.412 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.412 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:45.412 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.100910+0000", 2024-05-08T08:53:45.412 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1399999999999997e-06 2024-05-08T08:53:45.412 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.412 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.412 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:45.412 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.100916+0000", 2024-05-08T08:53:45.412 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6999999999999996e-06 2024-05-08T08:53:45.412 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.412 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.412 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:45.412 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.141733+0000", 2024-05-08T08:53:45.412 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.040817271000000002 2024-05-08T08:53:45.413 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.413 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.413 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:45.413 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.141812+0000", 2024-05-08T08:53:45.413 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9270999999999994e-05 2024-05-08T08:53:45.413 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.413 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.413 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:45.413 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.143373+0000", 2024-05-08T08:53:45.413 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001560903 2024-05-08T08:53:45.413 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.413 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:45.413 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.413 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.413 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.413 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:23031 3.15 3:ac9fb9cd:::rbd_data.114071bcf828.00000000000000f7:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:45.414 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.113562+0000", 2024-05-08T08:53:45.414 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 13.21785609, 2024-05-08T08:53:45.414 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042560938999999999, 2024-05-08T08:53:45.414 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:45.414 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:45.414 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:53:45.414 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:45.414 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:45.414 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:45.414 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 23031 2024-05-08T08:53:45.414 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.414 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:45.414 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.414 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:45.415 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.113562+0000", 2024-05-08T08:53:45.415 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:45.415 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.415 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.415 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:45.415 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.113562+0000", 2024-05-08T08:53:45.415 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:45.415 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.415 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.415 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:45.415 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.113938+0000", 2024-05-08T08:53:45.415 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000376401 2024-05-08T08:53:45.415 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.415 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.416 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:45.416 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.113943+0000", 2024-05-08T08:53:45.416 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0529999999999997e-06 2024-05-08T08:53:45.416 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.416 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.416 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:45.416 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.113951+0000", 2024-05-08T08:53:45.416 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8569999999999995e-06 2024-05-08T08:53:45.416 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.416 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.416 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:45.416 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.113959+0000", 2024-05-08T08:53:45.416 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4710000000000001e-06 2024-05-08T08:53:45.417 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.417 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.417 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:45.417 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.153955+0000", 2024-05-08T08:53:45.417 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.039996747999999999 2024-05-08T08:53:45.417 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.417 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.417 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:45.417 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.154014+0000", 2024-05-08T08:53:45.417 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8464999999999997e-05 2024-05-08T08:53:45.417 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.417 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.417 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-05-08T08:53:45.417 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.154148+0000", 2024-05-08T08:53:45.417 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00013428600000000001 2024-05-08T08:53:45.418 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.418 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.418 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:53:45.418 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.155390+0000", 2024-05-08T08:53:45.418 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001241792 2024-05-08T08:53:45.418 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.418 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.418 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:53:45.418 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.156068+0000", 2024-05-08T08:53:45.418 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00067823700000000005 2024-05-08T08:53:45.418 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.418 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.418 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:53:45.418 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.156102+0000", 2024-05-08T08:53:45.419 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3812000000000001e-05 2024-05-08T08:53:45.419 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.419 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.419 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:45.419 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.156123+0000", 2024-05-08T08:53:45.419 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0817000000000001e-05 2024-05-08T08:53:45.419 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.419 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:45.419 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.419 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.419 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:45.419 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:53:45.423 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:53:45.424 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:45.424+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:45.425 INFO:teuthology.orchestra.run.smithi196.stdout:4085 write 0x97eb800 thru 0x97f3293 (0x7a94 bytes) 2024-05-08T08:53:45.425 INFO:teuthology.orchestra.run.smithi196.stdout:4086 read 0x4499800 thru 0x44a2f0f (0x9710 bytes) 2024-05-08T08:53:45.425 INFO:teuthology.orchestra.run.smithi196.stdout:4087 read 0x4f30800 thru 0x4f30f30 (0x731 bytes) 2024-05-08T08:53:45.425 INFO:teuthology.orchestra.run.smithi196.stdout:4089 trunc from 0x97f3400 to 0xc177400 2024-05-08T08:53:45.425 INFO:teuthology.orchestra.run.smithi196.stdout:4090 write 0x1509400 thru 0x15167b6 (0xd3b7 bytes) 2024-05-08T08:53:45.425 INFO:teuthology.orchestra.run.smithi196.stdout:4091 write 0x7ffc400 thru 0x8006bd0 (0xa7d1 bytes) 2024-05-08T08:53:45.425 INFO:teuthology.orchestra.run.smithi196.stdout:4092 punch from 0xbe5ea00 to 0xbe6ca00, (0xe000 bytes) 2024-05-08T08:53:45.425 INFO:teuthology.orchestra.run.smithi196.stdout:4095 trunc from 0xc177400 to 0xde28400 2024-05-08T08:53:45.425 INFO:teuthology.orchestra.run.smithi196.stdout:4097 write 0x3205c00 thru 0x320c4c0 (0x68c1 bytes) 2024-05-08T08:53:45.425 INFO:teuthology.orchestra.run.smithi196.stdout:4099 punch from 0x43d1a00 to 0x43dd600, (0xbc00 bytes) 2024-05-08T08:53:45.426 INFO:teuthology.orchestra.run.smithi196.stdout:4101 write 0x1b1e800 thru 0x1b28481 (0x9c82 bytes) 2024-05-08T08:53:45.426 INFO:teuthology.orchestra.run.smithi196.stdout:4102 write 0x6e12c00 thru 0x6e14738 (0x1b39 bytes) 2024-05-08T08:53:45.426 INFO:teuthology.orchestra.run.smithi196.stdout:4106 trunc from 0xde28400 to 0xa4e5000 2024-05-08T08:53:45.426 INFO:teuthology.orchestra.run.smithi196.stdout:4108 read 0x48c8e00 thru 0x48d88a4 (0xfaa5 bytes) 2024-05-08T08:53:45.426 INFO:teuthology.orchestra.run.smithi196.stdout:4109 write 0x8e7a200 thru 0x8e894f4 (0xf2f5 bytes) 2024-05-08T08:53:45.426 INFO:teuthology.orchestra.run.smithi196.stdout:4110 punch from 0x5e0e000 to 0x5e12400, (0x4400 bytes) 2024-05-08T08:53:45.426 INFO:teuthology.orchestra.run.smithi196.stdout:4111 read 0x1401a00 thru 0x140b4eb (0x9aec bytes) 2024-05-08T08:53:45.438 INFO:teuthology.orchestra.run.smithi196.stdout:4112 write 0xc69d200 thru 0xc6a1e9f (0x4ca0 bytes) 2024-05-08T08:53:45.445 INFO:teuthology.orchestra.run.smithi196.stdout:4113 read 0xacb9800 thru 0xacbbf7a (0x277b bytes) 2024-05-08T08:53:45.445 INFO:teuthology.orchestra.run.smithi196.stdout:4114 read 0x92af600 thru 0x92b3e6b (0x486c bytes) 2024-05-08T08:53:45.445 INFO:teuthology.orchestra.run.smithi196.stdout:4118 read 0x2322e00 thru 0x232b8f9 (0x8afa bytes) 2024-05-08T08:53:45.446 INFO:teuthology.orchestra.run.smithi196.stdout:4119 read 0x2888800 thru 0x289728a (0xea8b bytes) 2024-05-08T08:53:45.447 INFO:teuthology.orchestra.run.smithi196.stdout:4120 write 0xa935e00 thru 0xa939adf (0x3ce0 bytes) 2024-05-08T08:53:45.447 INFO:teuthology.orchestra.run.smithi196.stdout:4121 read 0x5059800 thru 0x50600c5 (0x68c6 bytes) 2024-05-08T08:53:45.447 INFO:teuthology.orchestra.run.smithi196.stdout:4127 punch from 0xc1c5e00 to 0xc1d1e00, (0xc000 bytes) 2024-05-08T08:53:45.447 INFO:teuthology.orchestra.run.smithi196.stdout:4129 read 0x403ac00 thru 0x403b20e (0x60f bytes) 2024-05-08T08:53:45.455 INFO:teuthology.orchestra.run.smithi196.stdout:4131 trunc from 0xc6a2000 to 0x5c9c800 2024-05-08T08:53:45.481 INFO:teuthology.orchestra.run.smithi196.stdout:4132 read 0x262f200 thru 0x2638daa (0x9bab bytes) 2024-05-08T08:53:45.481 INFO:teuthology.orchestra.run.smithi196.stdout:4133 punch from 0x36bbc00 to 0x36c0800, (0x4c00 bytes) 2024-05-08T08:53:45.481 INFO:teuthology.orchestra.run.smithi196.stdout:4134 punch from 0x3f0f400 to 0x3f12800, (0x3400 bytes) 2024-05-08T08:53:45.485 INFO:teuthology.orchestra.run.smithi196.stdout:4137 trunc from 0x5c9c800 to 0x2d76000 2024-05-08T08:53:45.508 INFO:teuthology.orchestra.run.smithi196.stdout:4138 trunc from 0x2d76000 to 0xc48800 2024-05-08T08:53:45.522 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:53:45.525+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:53:45.529 INFO:teuthology.orchestra.run.smithi196.stdout:4140 read 0x26ec00 thru 0x2737a6 (0x4ba7 bytes) 2024-05-08T08:53:45.534 INFO:teuthology.orchestra.run.smithi196.stdout:4141 read 0xb48e00 thru 0xb54072 (0xb273 bytes) 2024-05-08T08:53:45.548 INFO:teuthology.orchestra.run.smithi196.stdout:4142 write 0x772a400 thru 0x7733efe (0x9aff bytes) 2024-05-08T08:53:45.554 INFO:teuthology.orchestra.run.smithi196.stdout:4143 read 0x3bf7800 thru 0x3bfb320 (0x3b21 bytes) 2024-05-08T08:53:45.555 INFO:teuthology.orchestra.run.smithi196.stdout:4144 trunc from 0x7734000 to 0x69a0c00 2024-05-08T08:53:45.558 INFO:teuthology.orchestra.run.smithi059.stderr:nodeep-scrub is unset 2024-05-08T08:53:45.574 INFO:teuthology.orchestra.run.smithi196.stdout:4146 read 0x14d7200 thru 0x14e6095 (0xee96 bytes) 2024-05-08T08:53:45.581 INFO:teuthology.orchestra.run.smithi196.stdout:4147 punch from 0x5b9a200 to 0x5ba8e00, (0xec00 bytes) 2024-05-08T08:53:45.587 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:53:45.587 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:53:45.587 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:53:45.587 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:53:45.596 INFO:teuthology.orchestra.run.smithi196.stdout:4148 write 0xa65b000 thru 0xa661f09 (0x6f0a bytes) 2024-05-08T08:53:45.597 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:53:45.603 INFO:teuthology.orchestra.run.smithi196.stdout:4149 write 0x1784800 thru 0x1790c99 (0xc49a bytes) 2024-05-08T08:53:45.604 INFO:teuthology.orchestra.run.smithi196.stdout:4152 punch from 0x15e5600 to 0x15e8400, (0x2e00 bytes) 2024-05-08T08:53:45.617 INFO:teuthology.orchestra.run.smithi196.stdout:4153 trunc from 0xa662000 to 0xf40e00 2024-05-08T08:53:45.623 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:53:45.626+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:53:45.665 INFO:teuthology.orchestra.run.smithi196.stdout:4154 write 0x6293800 thru 0x629dc45 (0xa446 bytes) 2024-05-08T08:53:45.666 INFO:teuthology.orchestra.run.smithi196.stdout:4158 read 0x2d96600 thru 0x2d975c2 (0xfc3 bytes) 2024-05-08T08:53:45.666 INFO:teuthology.orchestra.run.smithi196.stdout:4159 write 0x1145000 thru 0x1145753 (0x754 bytes) 2024-05-08T08:53:45.666 INFO:teuthology.orchestra.run.smithi196.stdout:4160 read 0x291a000 thru 0x2923384 (0x9385 bytes) 2024-05-08T08:53:45.667 INFO:teuthology.orchestra.run.smithi196.stdout:4161 read 0x5524a00 thru 0x55267bc (0x1dbd bytes) 2024-05-08T08:53:45.667 INFO:teuthology.orchestra.run.smithi196.stdout:4166 punch from 0xf6c200 to 0xf79800, (0xd600 bytes) 2024-05-08T08:53:45.688 INFO:teuthology.orchestra.run.smithi196.stdout:4167 write 0xdb65a00 thru 0xdb66901 (0xf02 bytes) 2024-05-08T08:53:45.689 INFO:teuthology.orchestra.run.smithi196.stdout:4170 clone 29 order 22 su 1048576 sc 14 2024-05-08T08:53:45.713 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:53:45.713 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:53:45.713 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:53:45.713 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:53:45.713 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:53:45.722 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:53:45.725 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:45.726+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:45.825 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:45.827+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:45.829 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:53:45.829 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:53:45.829 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:53:45.829 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:53:45.829 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.829 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22071 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 765952~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:45.829 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857643+0000", 2024-05-08T08:53:45.830 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 77.973935994000001, 2024-05-08T08:53:45.830 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047106528000000002, 2024-05-08T08:53:45.830 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:45.830 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:45.830 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:45.830 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:45.830 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:45.830 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:45.830 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22071 2024-05-08T08:53:45.830 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.830 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:45.830 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.830 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:45.830 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857643+0000", 2024-05-08T08:53:45.831 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:45.831 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.831 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.831 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:45.831 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857643+0000", 2024-05-08T08:53:45.831 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:45.831 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.831 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.831 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:45.831 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857644+0000", 2024-05-08T08:53:45.831 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2000000000000001e-07 2024-05-08T08:53:45.831 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.831 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.831 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:45.831 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857646+0000", 2024-05-08T08:53:45.831 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0059999999999999e-06 2024-05-08T08:53:45.832 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.832 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.832 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:45.832 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857651+0000", 2024-05-08T08:53:45.832 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.84e-06 2024-05-08T08:53:45.832 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.832 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.832 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:45.832 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857657+0000", 2024-05-08T08:53:45.832 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5620000000000003e-06 2024-05-08T08:53:45.832 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.832 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.832 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:45.832 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904620+0000", 2024-05-08T08:53:45.832 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046962661000000003 2024-05-08T08:53:45.832 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.833 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.833 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:45.833 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904678+0000", 2024-05-08T08:53:45.833 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8656999999999998e-05 2024-05-08T08:53:45.833 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.833 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.833 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:45.833 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904750+0000", 2024-05-08T08:53:45.833 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1482000000000004e-05 2024-05-08T08:53:45.833 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.833 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:45.833 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.833 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.833 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.833 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22073 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 782336~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:45.834 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:53:45.834 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 77.973911986000005, 2024-05-08T08:53:45.834 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047244234000000003, 2024-05-08T08:53:45.834 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:45.834 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:45.834 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:45.834 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:45.834 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:45.834 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:45.834 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22073 2024-05-08T08:53:45.834 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.834 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:45.834 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.834 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:45.835 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:53:45.835 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:45.835 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.835 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.835 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:45.835 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:53:45.835 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:45.835 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.835 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.835 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:45.835 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857668+0000", 2024-05-08T08:53:45.835 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7800000000000002e-07 2024-05-08T08:53:45.835 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.835 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.835 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:45.836 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857670+0000", 2024-05-08T08:53:45.836 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0820000000000001e-06 2024-05-08T08:53:45.836 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.836 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.836 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:45.836 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857678+0000", 2024-05-08T08:53:45.836 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7589999999999997e-06 2024-05-08T08:53:45.836 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.836 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.836 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:45.836 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857691+0000", 2024-05-08T08:53:45.836 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3672999999999999e-05 2024-05-08T08:53:45.836 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.836 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.836 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:45.837 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904780+0000", 2024-05-08T08:53:45.837 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047088616999999999 2024-05-08T08:53:45.837 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.837 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.837 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:45.837 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904840+0000", 2024-05-08T08:53:45.837 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0356999999999998e-05 2024-05-08T08:53:45.837 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.837 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.837 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:45.837 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904912+0000", 2024-05-08T08:53:45.837 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1367999999999998e-05 2024-05-08T08:53:45.837 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.837 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:45.837 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.837 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.838 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.838 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22075 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 798720~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:45.838 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:53:45.838 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 77.973881589000001, 2024-05-08T08:53:45.838 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047384643999999997, 2024-05-08T08:53:45.838 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:45.838 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:45.838 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:45.838 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:45.838 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:45.838 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:45.838 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22075 2024-05-08T08:53:45.838 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.838 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:45.838 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.839 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:45.839 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:53:45.839 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:45.839 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.839 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.839 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:45.839 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:53:45.839 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:45.839 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.839 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.839 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:45.839 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:53:45.839 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7399999999999999e-07 2024-05-08T08:53:45.839 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.839 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.839 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:45.840 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857705+0000", 2024-05-08T08:53:45.840 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1980000000000004e-06 2024-05-08T08:53:45.840 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.840 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.840 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:45.840 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857713+0000", 2024-05-08T08:53:45.840 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7039999999999994e-06 2024-05-08T08:53:45.840 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.840 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.840 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:45.840 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857718+0000", 2024-05-08T08:53:45.840 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.865e-06 2024-05-08T08:53:45.840 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.840 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.840 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:45.840 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904941+0000", 2024-05-08T08:53:45.840 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047223198000000001 2024-05-08T08:53:45.841 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.841 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.841 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:45.841 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904999+0000", 2024-05-08T08:53:45.841 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8254999999999997e-05 2024-05-08T08:53:45.841 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.841 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.841 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:45.841 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905082+0000", 2024-05-08T08:53:45.841 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3049999999999999e-05 2024-05-08T08:53:45.841 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.841 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:45.841 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.841 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.841 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.841 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22077 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 815104~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:45.842 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:53:45.842 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 77.973843578, 2024-05-08T08:53:45.842 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047519448999999998, 2024-05-08T08:53:45.842 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:45.842 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:45.842 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:45.842 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:45.842 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:45.842 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:45.842 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22077 2024-05-08T08:53:45.842 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.842 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:45.842 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.842 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:45.842 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:53:45.842 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:45.843 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.843 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.843 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:45.843 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:53:45.843 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:45.843 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.843 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.843 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:45.843 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:53:45.843 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.99e-07 2024-05-08T08:53:45.843 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.843 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.843 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:45.843 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857739+0000", 2024-05-08T08:53:45.843 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3599999999999999e-06 2024-05-08T08:53:45.844 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.844 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.844 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:45.844 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857745+0000", 2024-05-08T08:53:45.844 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5139999999999998e-06 2024-05-08T08:53:45.844 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.844 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.844 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:45.844 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857750+0000", 2024-05-08T08:53:45.844 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8550000000000001e-06 2024-05-08T08:53:45.844 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.844 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.844 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:45.844 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905110+0000", 2024-05-08T08:53:45.844 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047360126000000002 2024-05-08T08:53:45.844 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.844 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.845 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:45.845 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905175+0000", 2024-05-08T08:53:45.845 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4689000000000004e-05 2024-05-08T08:53:45.845 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.845 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.845 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:45.845 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905255+0000", 2024-05-08T08:53:45.845 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0306e-05 2024-05-08T08:53:45.845 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.845 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:45.845 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.845 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.845 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.845 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22079 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 831488~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:45.845 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:53:45.846 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 77.973823030000005, 2024-05-08T08:53:45.846 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.04767499, 2024-05-08T08:53:45.846 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:45.846 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:45.846 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:45.846 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:45.846 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:45.846 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:45.846 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22079 2024-05-08T08:53:45.846 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.846 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:45.846 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.846 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:45.846 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:53:45.846 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:45.847 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.847 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.847 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:45.847 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:53:45.847 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:45.847 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.847 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.847 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:45.847 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857757+0000", 2024-05-08T08:53:45.847 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8299999999999997e-07 2024-05-08T08:53:45.847 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.847 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.847 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:45.847 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857759+0000", 2024-05-08T08:53:45.847 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1390000000000002e-06 2024-05-08T08:53:45.847 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.848 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.848 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:45.848 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857765+0000", 2024-05-08T08:53:45.848 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4609999999999996e-06 2024-05-08T08:53:45.848 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.848 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.848 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:45.848 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857770+0000", 2024-05-08T08:53:45.848 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6689999999999997e-06 2024-05-08T08:53:45.848 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.848 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.848 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:45.848 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905281+0000", 2024-05-08T08:53:45.848 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047510668999999998 2024-05-08T08:53:45.848 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.849 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.849 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:45.849 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905344+0000", 2024-05-08T08:53:45.849 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2797000000000002e-05 2024-05-08T08:53:45.849 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.849 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.849 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:45.849 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905431+0000", 2024-05-08T08:53:45.849 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7771999999999999e-05 2024-05-08T08:53:45.849 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.849 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:45.849 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.849 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.849 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.849 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22082 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 847872~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:45.849 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:53:45.849 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 77.973803339, 2024-05-08T08:53:45.850 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047831934999999999, 2024-05-08T08:53:45.850 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:45.850 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:45.850 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:45.850 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:45.850 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:45.850 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:45.850 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22082 2024-05-08T08:53:45.850 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.850 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:45.850 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.850 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:45.850 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:53:45.850 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:45.850 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.851 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.851 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:45.851 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:53:45.851 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:45.851 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.851 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.851 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:45.851 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:53:45.851 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:53:45.851 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.851 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.851 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:45.851 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857778+0000", 2024-05-08T08:53:45.851 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0789999999999999e-06 2024-05-08T08:53:45.851 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.851 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.851 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:45.852 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857784+0000", 2024-05-08T08:53:45.852 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0480000000000004e-06 2024-05-08T08:53:45.852 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.852 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.852 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:45.852 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857790+0000", 2024-05-08T08:53:45.852 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2719999999999997e-06 2024-05-08T08:53:45.852 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.852 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.852 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:45.852 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905458+0000", 2024-05-08T08:53:45.852 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047668470999999997 2024-05-08T08:53:45.852 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.852 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.852 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:45.852 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905521+0000", 2024-05-08T08:53:45.853 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2952000000000003e-05 2024-05-08T08:53:45.853 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.853 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.853 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:45.853 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905608+0000", 2024-05-08T08:53:45.853 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6759000000000001e-05 2024-05-08T08:53:45.853 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.853 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:45.853 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.853 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.853 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.853 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22084 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 864256~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:45.853 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:53:45.853 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 77.973779389000001, 2024-05-08T08:53:45.853 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047977748000000001, 2024-05-08T08:53:45.853 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:45.854 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:45.854 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:45.854 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:45.854 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:45.854 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:45.854 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22084 2024-05-08T08:53:45.854 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.854 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:45.854 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.854 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:45.854 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:53:45.854 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:45.854 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.854 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.855 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:45.855 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:53:45.855 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:45.855 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.855 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.855 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:45.855 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:53:45.855 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:53:45.855 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.855 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.855 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:45.855 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857802+0000", 2024-05-08T08:53:45.855 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0779999999999998e-06 2024-05-08T08:53:45.855 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.856 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.856 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:45.856 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857812+0000", 2024-05-08T08:53:45.856 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7939999999999997e-06 2024-05-08T08:53:45.856 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.856 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.856 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:45.856 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857817+0000", 2024-05-08T08:53:45.856 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8899999999999998e-06 2024-05-08T08:53:45.856 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.856 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.856 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:45.856 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905636+0000", 2024-05-08T08:53:45.856 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047819252 2024-05-08T08:53:45.857 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.857 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.857 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:45.857 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905700+0000", 2024-05-08T08:53:45.857 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4016999999999994e-05 2024-05-08T08:53:45.857 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.857 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.857 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:45.857 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905778+0000", 2024-05-08T08:53:45.857 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7306999999999997e-05 2024-05-08T08:53:45.857 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.857 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:45.857 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.857 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.857 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.857 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22086 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 880640~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:45.858 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:53:45.858 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 77.973756038999994, 2024-05-08T08:53:45.858 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048119149, 2024-05-08T08:53:45.858 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:45.858 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:45.858 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:45.858 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:45.858 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:45.858 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:45.858 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22086 2024-05-08T08:53:45.858 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.858 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:45.858 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.858 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:45.859 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:53:45.859 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:45.859 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.859 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.859 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:45.859 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:53:45.859 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:45.859 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.859 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.859 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:45.859 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857824+0000", 2024-05-08T08:53:45.859 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4200000000000002e-07 2024-05-08T08:53:45.859 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.859 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.859 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:45.859 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857826+0000", 2024-05-08T08:53:45.859 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0590000000000001e-06 2024-05-08T08:53:45.860 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.860 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.860 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:45.860 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857832+0000", 2024-05-08T08:53:45.860 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8029999999999999e-06 2024-05-08T08:53:45.860 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.860 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.860 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:45.860 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857837+0000", 2024-05-08T08:53:45.860 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7929999999999997e-06 2024-05-08T08:53:45.860 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.860 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.860 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:45.860 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905803+0000", 2024-05-08T08:53:45.860 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047965761000000003 2024-05-08T08:53:45.860 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.861 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.861 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:45.861 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905862+0000", 2024-05-08T08:53:45.861 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9203000000000002e-05 2024-05-08T08:53:45.861 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.861 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.861 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:45.861 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905942+0000", 2024-05-08T08:53:45.861 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0187999999999999e-05 2024-05-08T08:53:45.861 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.861 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:45.861 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.861 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.861 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.861 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22088 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 897024~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:45.861 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:53:45.861 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 77.973735986999998, 2024-05-08T08:53:45.862 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048256049000000002, 2024-05-08T08:53:45.862 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:45.862 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:45.862 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:45.862 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:45.862 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:45.862 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:45.862 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22088 2024-05-08T08:53:45.862 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.862 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:45.862 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.862 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:45.862 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:53:45.862 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:45.862 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.862 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.863 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:45.863 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:53:45.863 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:45.863 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.863 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.863 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:45.863 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857844+0000", 2024-05-08T08:53:45.863 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.46e-07 2024-05-08T08:53:45.863 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.863 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.863 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:45.863 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857846+0000", 2024-05-08T08:53:45.863 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0899999999999999e-06 2024-05-08T08:53:45.863 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.863 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.863 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:45.864 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857855+0000", 2024-05-08T08:53:45.864 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8219999999999996e-06 2024-05-08T08:53:45.864 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.864 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.864 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:45.864 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857860+0000", 2024-05-08T08:53:45.864 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2989999999999996e-06 2024-05-08T08:53:45.864 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.864 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.864 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:45.864 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905966+0000", 2024-05-08T08:53:45.864 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048106542000000002 2024-05-08T08:53:45.864 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.864 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.864 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:45.864 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906024+0000", 2024-05-08T08:53:45.865 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7958000000000003e-05 2024-05-08T08:53:45.865 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.865 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.865 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:45.865 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906099+0000", 2024-05-08T08:53:45.865 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4992000000000003e-05 2024-05-08T08:53:45.865 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.865 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:45.865 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.865 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.865 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.865 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22091 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 913408~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:45.865 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:53:45.865 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 77.973712226999993, 2024-05-08T08:53:45.865 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048395290000000001, 2024-05-08T08:53:45.865 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:45.865 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:45.866 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:45.866 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:45.866 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:45.866 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:45.866 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22091 2024-05-08T08:53:45.866 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.866 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:45.866 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.866 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:45.866 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:53:45.866 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:45.866 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.866 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.866 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:45.867 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:53:45.867 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:45.867 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.867 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.867 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:45.867 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:53:45.867 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4900000000000001e-07 2024-05-08T08:53:45.867 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.867 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.867 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:45.867 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857870+0000", 2024-05-08T08:53:45.867 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2510000000000001e-06 2024-05-08T08:53:45.867 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.867 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.867 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:45.868 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857877+0000", 2024-05-08T08:53:45.868 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0809999999999996e-06 2024-05-08T08:53:45.868 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.868 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.868 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:45.868 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857881+0000", 2024-05-08T08:53:45.868 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.724e-06 2024-05-08T08:53:45.868 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.868 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.868 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:45.868 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906127+0000", 2024-05-08T08:53:45.868 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048245345000000002 2024-05-08T08:53:45.868 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.868 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.868 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:45.868 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906190+0000", 2024-05-08T08:53:45.869 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2762000000000007e-05 2024-05-08T08:53:45.869 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.869 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.869 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:45.869 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906262+0000", 2024-05-08T08:53:45.869 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2778e-05 2024-05-08T08:53:45.869 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.869 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:45.869 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.869 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.869 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.869 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22093 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 929792~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:45.869 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:53:45.869 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 77.973689863000004, 2024-05-08T08:53:45.870 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048532875000000003, 2024-05-08T08:53:45.870 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:45.870 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:45.870 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:45.870 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:45.870 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:45.870 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:45.870 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22093 2024-05-08T08:53:45.870 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.870 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:45.870 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.870 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:45.870 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:53:45.870 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:45.870 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.871 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.871 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:45.871 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:53:45.871 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:45.871 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.871 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.871 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:45.871 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857890+0000", 2024-05-08T08:53:45.871 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-07 2024-05-08T08:53:45.871 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.871 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.871 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:45.871 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857899+0000", 2024-05-08T08:53:45.871 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6889999999999999e-06 2024-05-08T08:53:45.872 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.872 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.872 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:45.872 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857906+0000", 2024-05-08T08:53:45.872 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6490000000000002e-06 2024-05-08T08:53:45.872 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.872 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.872 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:45.872 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857917+0000", 2024-05-08T08:53:45.872 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0764999999999999e-05 2024-05-08T08:53:45.872 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.872 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.872 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:45.872 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906287+0000", 2024-05-08T08:53:45.872 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048370442 2024-05-08T08:53:45.872 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.873 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.873 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:45.873 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906349+0000", 2024-05-08T08:53:45.873 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1537000000000004e-05 2024-05-08T08:53:45.873 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.873 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.873 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:45.873 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906422+0000", 2024-05-08T08:53:45.873 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3431999999999998e-05 2024-05-08T08:53:45.873 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.873 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:45.873 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.873 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.873 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.873 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22095 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 946176~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:45.874 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:53:45.874 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 77.973655159000003, 2024-05-08T08:53:45.874 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048657593999999998, 2024-05-08T08:53:45.874 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:45.874 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:45.874 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:45.874 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:45.874 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:45.874 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:45.874 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22095 2024-05-08T08:53:45.874 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.874 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:45.874 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.874 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:45.874 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:53:45.875 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:45.875 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.875 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.875 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:45.875 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:53:45.875 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:45.875 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.875 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.875 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:45.875 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857925+0000", 2024-05-08T08:53:45.875 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3099999999999998e-07 2024-05-08T08:53:45.875 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.875 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.875 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:45.875 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857927+0000", 2024-05-08T08:53:45.876 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.34e-06 2024-05-08T08:53:45.876 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.876 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.876 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:45.876 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857938+0000", 2024-05-08T08:53:45.876 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1426e-05 2024-05-08T08:53:45.876 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.876 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.876 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:45.876 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857943+0000", 2024-05-08T08:53:45.876 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5140000000000003e-06 2024-05-08T08:53:45.876 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.876 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.876 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:45.876 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906448+0000", 2024-05-08T08:53:45.876 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048504641000000001 2024-05-08T08:53:45.877 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.877 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.877 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:45.877 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906510+0000", 2024-05-08T08:53:45.877 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2663000000000006e-05 2024-05-08T08:53:45.877 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.877 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.877 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:45.877 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906582+0000", 2024-05-08T08:53:45.877 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1478999999999998e-05 2024-05-08T08:53:45.877 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.877 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:45.877 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.877 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.877 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.878 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22097 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 962560~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:45.878 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:53:45.878 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 77.973627469999997, 2024-05-08T08:53:45.878 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048791353000000003, 2024-05-08T08:53:45.878 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:45.878 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:45.878 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:45.878 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:45.878 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:45.878 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:45.878 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22097 2024-05-08T08:53:45.878 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.878 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:45.878 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.879 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:45.879 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:53:45.879 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:45.879 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.879 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.879 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:45.879 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:53:45.879 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:45.879 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.879 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.879 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:45.879 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:53:45.879 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9200000000000002e-07 2024-05-08T08:53:45.879 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.879 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.880 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:45.880 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857954+0000", 2024-05-08T08:53:45.880 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1900000000000002e-06 2024-05-08T08:53:45.880 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.880 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.880 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:45.880 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857961+0000", 2024-05-08T08:53:45.880 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4989999999999999e-06 2024-05-08T08:53:45.880 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.880 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.880 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:45.880 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857971+0000", 2024-05-08T08:53:45.880 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0485999999999999e-05 2024-05-08T08:53:45.880 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.880 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.880 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:45.881 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906607+0000", 2024-05-08T08:53:45.881 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048635109000000003 2024-05-08T08:53:45.881 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.881 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.881 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:45.881 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906667+0000", 2024-05-08T08:53:45.881 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0439000000000002e-05 2024-05-08T08:53:45.881 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.881 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.881 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:45.881 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906743+0000", 2024-05-08T08:53:45.881 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6237999999999998e-05 2024-05-08T08:53:45.881 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.881 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:45.881 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.881 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.882 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.882 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22099 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 978944~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:45.882 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:53:45.882 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 77.973601310000006, 2024-05-08T08:53:45.882 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048932055000000002, 2024-05-08T08:53:45.882 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:45.882 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:45.882 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:45.882 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:45.882 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:45.882 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:45.882 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22099 2024-05-08T08:53:45.882 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.882 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:45.882 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.882 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:45.883 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:53:45.883 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:45.883 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.883 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.883 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:45.883 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:53:45.883 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:45.883 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.883 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.883 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:45.883 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:53:45.883 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.27e-07 2024-05-08T08:53:45.883 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.883 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.883 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:45.884 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857987+0000", 2024-05-08T08:53:45.884 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5890000000000003e-06 2024-05-08T08:53:45.884 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.884 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.884 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:45.884 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857994+0000", 2024-05-08T08:53:45.884 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6619999999999999e-06 2024-05-08T08:53:45.884 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.884 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.884 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:45.884 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857998+0000", 2024-05-08T08:53:45.884 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7110000000000003e-06 2024-05-08T08:53:45.884 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.884 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.884 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:45.885 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906772+0000", 2024-05-08T08:53:45.885 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048773415000000001 2024-05-08T08:53:45.885 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.885 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.885 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:45.885 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906834+0000", 2024-05-08T08:53:45.885 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1752000000000001e-05 2024-05-08T08:53:45.885 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.885 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.885 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:45.885 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906910+0000", 2024-05-08T08:53:45.885 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6499000000000001e-05 2024-05-08T08:53:45.885 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.885 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:45.885 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.885 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.885 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.886 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22101 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 995328~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:45.886 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:53:45.886 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 77.973574178999996, 2024-05-08T08:53:45.886 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049087364000000001, 2024-05-08T08:53:45.886 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:45.886 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:45.886 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:45.886 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:45.886 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:45.886 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:45.886 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22101 2024-05-08T08:53:45.886 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.886 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:45.887 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.887 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:45.887 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:53:45.887 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:45.887 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.887 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.887 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:45.887 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:53:45.887 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:45.887 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.887 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.887 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:45.887 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858006+0000", 2024-05-08T08:53:45.887 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1399999999999997e-07 2024-05-08T08:53:45.887 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.887 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.888 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:45.888 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858008+0000", 2024-05-08T08:53:45.888 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2060000000000001e-06 2024-05-08T08:53:45.888 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.888 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.888 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:45.888 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858017+0000", 2024-05-08T08:53:45.888 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1579999999999994e-06 2024-05-08T08:53:45.888 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.888 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.888 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:45.888 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858022+0000", 2024-05-08T08:53:45.888 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4199999999999998e-06 2024-05-08T08:53:45.888 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.888 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.888 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:45.888 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906940+0000", 2024-05-08T08:53:45.889 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048917627999999998 2024-05-08T08:53:45.889 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.889 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.889 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:45.889 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907002+0000", 2024-05-08T08:53:45.889 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2399999999999999e-05 2024-05-08T08:53:45.889 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.889 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.889 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:45.889 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907092+0000", 2024-05-08T08:53:45.889 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0137999999999997e-05 2024-05-08T08:53:45.889 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.889 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:45.889 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.889 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.889 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.889 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22103 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1011712~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:45.890 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:53:45.890 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 77.973549394000003, 2024-05-08T08:53:45.890 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049228542, 2024-05-08T08:53:45.890 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:45.890 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:45.890 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:45.890 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:45.890 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:45.890 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:45.890 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22103 2024-05-08T08:53:45.890 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.890 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:45.890 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.890 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:45.890 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:53:45.890 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:45.891 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.891 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.891 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:45.891 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:53:45.891 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:45.891 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.891 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.891 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:45.891 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:53:45.891 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4299999999999999e-07 2024-05-08T08:53:45.891 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.891 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.891 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:45.891 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858032+0000", 2024-05-08T08:53:45.891 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0590000000000001e-06 2024-05-08T08:53:45.891 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.892 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.892 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:45.892 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858037+0000", 2024-05-08T08:53:45.892 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.977e-06 2024-05-08T08:53:45.892 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.892 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.892 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:45.892 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858042+0000", 2024-05-08T08:53:45.892 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7650000000000001e-06 2024-05-08T08:53:45.892 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.892 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.892 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:45.892 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907118+0000", 2024-05-08T08:53:45.892 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049076147000000001 2024-05-08T08:53:45.892 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.892 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.893 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:45.893 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907180+0000", 2024-05-08T08:53:45.893 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1971999999999996e-05 2024-05-08T08:53:45.893 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.893 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.893 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:45.893 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907258+0000", 2024-05-08T08:53:45.893 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8279e-05 2024-05-08T08:53:45.893 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.893 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:45.893 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.893 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.893 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.893 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22105 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1028096~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:45.893 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:53:45.893 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 77.973530151000006, 2024-05-08T08:53:45.893 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049394609999999999, 2024-05-08T08:53:45.894 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:45.894 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:45.894 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:45.894 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:45.894 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:45.894 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:45.894 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22105 2024-05-08T08:53:45.894 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.894 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:45.894 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.894 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:45.894 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:53:45.894 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:45.894 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.894 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.895 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:45.895 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:53:45.895 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:45.895 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.895 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.895 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:45.895 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858050+0000", 2024-05-08T08:53:45.895 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5999999999999999e-07 2024-05-08T08:53:45.895 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.895 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.895 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:45.895 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858051+0000", 2024-05-08T08:53:45.895 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8870000000000001e-06 2024-05-08T08:53:45.895 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.895 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.895 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:45.896 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858057+0000", 2024-05-08T08:53:45.896 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8019999999999997e-06 2024-05-08T08:53:45.896 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.896 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.896 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:45.896 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858062+0000", 2024-05-08T08:53:45.896 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.561e-06 2024-05-08T08:53:45.896 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.896 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.896 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:45.896 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907289+0000", 2024-05-08T08:53:45.896 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049226957000000002 2024-05-08T08:53:45.896 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.896 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.896 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:45.896 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907356+0000", 2024-05-08T08:53:45.896 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6915000000000006e-05 2024-05-08T08:53:45.897 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.897 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.897 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:45.897 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907444+0000", 2024-05-08T08:53:45.897 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8127999999999994e-05 2024-05-08T08:53:45.897 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.897 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:45.897 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.897 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.897 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.897 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22107 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1044480~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:45.897 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:53:45.897 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 77.973511810999995, 2024-05-08T08:53:45.897 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049799942999999999, 2024-05-08T08:53:45.897 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:45.897 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:45.898 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:45.898 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:45.898 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:45.898 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:45.898 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22107 2024-05-08T08:53:45.898 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.898 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:45.898 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.898 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:45.898 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:53:45.898 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:45.898 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.898 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.898 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:45.898 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:53:45.898 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:45.899 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.899 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.899 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:45.899 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:53:45.899 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4799999999999999e-07 2024-05-08T08:53:45.899 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.899 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.899 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:45.899 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858070+0000", 2024-05-08T08:53:45.899 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.012e-06 2024-05-08T08:53:45.899 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.899 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.899 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:45.899 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858077+0000", 2024-05-08T08:53:45.899 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6290000000000001e-06 2024-05-08T08:53:45.899 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.900 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.900 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:45.900 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858083+0000", 2024-05-08T08:53:45.900 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8880000000000002e-06 2024-05-08T08:53:45.900 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.900 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.900 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:45.900 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907671+0000", 2024-05-08T08:53:45.900 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049587191000000003 2024-05-08T08:53:45.900 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.900 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.900 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:45.900 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907763+0000", 2024-05-08T08:53:45.901 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2231000000000005e-05 2024-05-08T08:53:45.901 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.901 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.901 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:45.901 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907867+0000", 2024-05-08T08:53:45.901 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104644 2024-05-08T08:53:45.901 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.901 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:45.901 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.901 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.901 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.901 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22294 3.1c 3:3aad0389:::rbd_data.11406886b07b.0000000000000080:head [stat,delete] snapc 7=[7] RETRY=6 ondisk+retry+write+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:45.901 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:53:45.901 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 77.972917963, 2024-05-08T08:53:45.902 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051152191, 2024-05-08T08:53:45.902 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:45.902 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:45.902 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:45.902 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:45.902 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:45.902 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:45.902 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22294 2024-05-08T08:53:45.902 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.902 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:45.902 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.902 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:45.902 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:53:45.902 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:45.902 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.902 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.903 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:45.903 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:53:45.903 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:45.903 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.903 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.903 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:45.903 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858662+0000", 2024-05-08T08:53:45.903 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4799999999999998e-07 2024-05-08T08:53:45.903 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.903 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.903 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:45.903 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858664+0000", 2024-05-08T08:53:45.903 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1229999999999998e-06 2024-05-08T08:53:45.903 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.903 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.903 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:45.904 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858670+0000", 2024-05-08T08:53:45.904 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3099999999999997e-06 2024-05-08T08:53:45.904 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.904 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.904 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:45.904 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858676+0000", 2024-05-08T08:53:45.904 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9750000000000004e-06 2024-05-08T08:53:45.904 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.904 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.904 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:45.904 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907901+0000", 2024-05-08T08:53:45.904 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049224520000000001 2024-05-08T08:53:45.904 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.904 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.905 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:45.905 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.908085+0000", 2024-05-08T08:53:45.905 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00018377500000000001 2024-05-08T08:53:45.905 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.905 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.905 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:45.905 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.909814+0000", 2024-05-08T08:53:45.905 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00172894 2024-05-08T08:53:45.905 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.905 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:45.905 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.905 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.905 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.905 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4416.0:13461 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-05-08T08:53:45.906 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:22.487456+0000", 2024-05-08T08:53:45.906 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.344122922, 2024-05-08T08:53:45.906 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.99102620799999996, 2024-05-08T08:53:45.906 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:45.906 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:45.906 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:45.906 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:45.906 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4416", 2024-05-08T08:53:45.906 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:45.906 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 13461 2024-05-08T08:53:45.906 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.906 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:45.906 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.906 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:45.906 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487456+0000", 2024-05-08T08:53:45.907 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:45.907 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.907 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.907 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:45.907 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487456+0000", 2024-05-08T08:53:45.907 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:45.907 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.907 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.907 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:45.907 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487458+0000", 2024-05-08T08:53:45.907 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5659999999999999e-06 2024-05-08T08:53:45.907 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.907 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.907 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:45.907 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487463+0000", 2024-05-08T08:53:45.907 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6890000000000003e-06 2024-05-08T08:53:45.908 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.908 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.908 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:45.908 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487472+0000", 2024-05-08T08:53:45.908 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7710000000000001e-06 2024-05-08T08:53:45.908 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.908 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.908 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:45.908 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487478+0000", 2024-05-08T08:53:45.908 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1390000000000003e-06 2024-05-08T08:53:45.908 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.908 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.908 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:45.908 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487520+0000", 2024-05-08T08:53:45.908 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2259999999999999e-05 2024-05-08T08:53:45.908 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.909 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.909 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-05-08T08:53:45.909 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487528+0000", 2024-05-08T08:53:45.909 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7479999999999993e-06 2024-05-08T08:53:45.909 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.909 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.909 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:45.909 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:23.478139+0000", 2024-05-08T08:53:45.909 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.99061165200000001 2024-05-08T08:53:45.909 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.909 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.909 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:45.909 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:23.478301+0000", 2024-05-08T08:53:45.909 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000161602 2024-05-08T08:53:45.909 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:45.909 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:45.910 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:45.910 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:23.478483+0000", 2024-05-08T08:53:45.910 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000181781 2024-05-08T08:53:45.910 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.910 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:45.910 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.910 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:45.910 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:45.910 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:53:45.911 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:53:45.926 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:45.929+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:46.028 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:46.030+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:46.057 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:53:46.057 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:53:46.057 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:53:46.057 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:53:46.066 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:53:46.128 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:46.132+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:46.212 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:53:46.213 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:53:46.213 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:53:46.213 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:53:46.213 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:53:46.221 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:53:46.229 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:53:46.233+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:53:46.327 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:53:46.327 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:53:46.327 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:53:46.327 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:53:46.328 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.328 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6690 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1093632~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:46.328 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:53:46.328 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 74.614188505000001, 2024-05-08T08:53:46.328 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052462754, 2024-05-08T08:53:46.328 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:46.328 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:46.328 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:46.328 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:46.328 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:46.328 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:46.328 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6690 2024-05-08T08:53:46.328 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.328 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:46.328 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.329 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:46.329 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:53:46.329 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:46.329 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.329 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.329 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:46.329 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:53:46.329 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:46.329 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.329 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.329 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:46.329 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715346+0000", 2024-05-08T08:53:46.329 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.84e-07 2024-05-08T08:53:46.329 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.329 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.329 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:46.330 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715348+0000", 2024-05-08T08:53:46.330 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.114e-06 2024-05-08T08:53:46.330 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.330 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.330 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:46.330 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715353+0000", 2024-05-08T08:53:46.330 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5720000000000002e-06 2024-05-08T08:53:46.331 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.331 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.331 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:46.331 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715358+0000", 2024-05-08T08:53:46.331 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7140000000000001e-06 2024-05-08T08:53:46.331 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.331 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.331 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:46.331 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767638+0000", 2024-05-08T08:53:46.331 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052279562000000002 2024-05-08T08:53:46.331 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.331 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.331 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:46.332 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767713+0000", 2024-05-08T08:53:46.332 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5652e-05 2024-05-08T08:53:46.332 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.332 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.332 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:46.332 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767808+0000", 2024-05-08T08:53:46.332 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4755999999999999e-05 2024-05-08T08:53:46.332 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:46.332 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:46.332 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:46.332 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.332 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.333 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6692 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1110016~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:46.333 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:53:46.333 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 74.614169415000006, 2024-05-08T08:53:46.333 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052666495000000001, 2024-05-08T08:53:46.333 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:46.333 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:46.333 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:46.333 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:46.333 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:46.333 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:46.333 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6692 2024-05-08T08:53:46.333 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.334 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:46.334 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.334 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:46.334 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:53:46.334 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:46.334 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.334 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.334 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:46.334 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:53:46.334 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:46.334 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.335 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.335 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:46.335 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:53:46.335 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8599999999999999e-07 2024-05-08T08:53:46.335 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.335 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.335 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:46.335 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715372+0000", 2024-05-08T08:53:46.335 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0990000000000001e-06 2024-05-08T08:53:46.335 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.335 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.335 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:46.335 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715378+0000", 2024-05-08T08:53:46.336 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0680000000000002e-06 2024-05-08T08:53:46.336 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.336 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.336 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:46.336 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715383+0000", 2024-05-08T08:53:46.336 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8740000000000003e-06 2024-05-08T08:53:46.336 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.336 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.336 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:46.336 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767848+0000", 2024-05-08T08:53:46.336 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052464668999999998 2024-05-08T08:53:46.336 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.336 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.337 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:46.337 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767927+0000", 2024-05-08T08:53:46.337 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9228999999999996e-05 2024-05-08T08:53:46.337 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.337 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.337 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:46.337 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768031+0000", 2024-05-08T08:53:46.337 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010417 2024-05-08T08:53:46.337 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:46.337 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:46.337 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:46.337 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.337 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.338 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6694 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1126400~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:46.338 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:53:46.338 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 74.614144518000003, 2024-05-08T08:53:46.338 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052848829, 2024-05-08T08:53:46.338 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:46.338 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:46.338 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:46.338 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:46.338 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:46.338 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:46.338 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6694 2024-05-08T08:53:46.338 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.339 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:46.339 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.339 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:46.339 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:53:46.339 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:46.339 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.339 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.339 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:46.339 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:53:46.339 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:46.339 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.340 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.340 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:46.340 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715390+0000", 2024-05-08T08:53:46.340 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9900000000000001e-07 2024-05-08T08:53:46.340 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.340 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.340 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:46.340 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715392+0000", 2024-05-08T08:53:46.340 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8279999999999999e-06 2024-05-08T08:53:46.340 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.340 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.340 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:46.340 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715398+0000", 2024-05-08T08:53:46.341 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7619999999999998e-06 2024-05-08T08:53:46.341 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.341 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.341 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:46.341 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715403+0000", 2024-05-08T08:53:46.341 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7790000000000003e-06 2024-05-08T08:53:46.341 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.341 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.341 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:46.341 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768067+0000", 2024-05-08T08:53:46.341 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052663785999999997 2024-05-08T08:53:46.341 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.341 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.342 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:46.342 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768142+0000", 2024-05-08T08:53:46.342 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5018999999999994e-05 2024-05-08T08:53:46.342 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.342 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.342 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:46.342 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768238+0000", 2024-05-08T08:53:46.342 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6255999999999995e-05 2024-05-08T08:53:46.342 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:46.342 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:46.342 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:46.342 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.343 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.343 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6696 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1142784~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:46.343 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:53:46.343 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 74.614125009999995, 2024-05-08T08:53:46.343 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.053051057999999998, 2024-05-08T08:53:46.343 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:46.343 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:46.343 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:46.343 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:46.343 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:46.343 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:46.343 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6696 2024-05-08T08:53:46.343 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.344 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:46.344 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.344 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:46.344 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:53:46.344 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:46.344 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.344 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.344 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:46.344 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:53:46.344 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:46.344 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.344 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.345 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:46.345 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:53:46.345 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-05-08T08:53:46.345 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.345 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:46.345 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715411+0000", 2024-05-08T08:53:46.345 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9630000000000001e-06 2024-05-08T08:53:46.345 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.345 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:46.345 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715417+0000", 2024-05-08T08:53:46.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7250000000000002e-06 2024-05-08T08:53:46.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.346 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:46.346 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715421+0000", 2024-05-08T08:53:46.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4089999999999996e-06 2024-05-08T08:53:46.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.346 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:46.346 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768277+0000", 2024-05-08T08:53:46.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052855293999999997 2024-05-08T08:53:46.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.347 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.347 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:46.347 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768358+0000", 2024-05-08T08:53:46.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1720999999999999e-05 2024-05-08T08:53:46.347 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.347 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.347 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:46.347 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768460+0000", 2024-05-08T08:53:46.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010154600000000001 2024-05-08T08:53:46.347 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:46.347 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:46.347 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:46.347 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.348 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.348 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6698 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1159168~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:46.348 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:53:46.348 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 74.614106143000001, 2024-05-08T08:53:46.348 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.053245726, 2024-05-08T08:53:46.348 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:46.348 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:46.348 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:46.348 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:46.348 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:46.348 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:46.348 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6698 2024-05-08T08:53:46.348 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.348 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:46.349 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.349 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:46.349 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:53:46.349 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:46.349 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.349 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.349 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:46.349 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:53:46.349 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:46.349 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.349 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.349 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:46.349 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716936+0000", 2024-05-08T08:53:46.349 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001508665 2024-05-08T08:53:46.350 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.350 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.350 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:46.350 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716941+0000", 2024-05-08T08:53:46.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1910000000000002e-06 2024-05-08T08:53:46.350 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.350 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.350 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:46.350 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716950+0000", 2024-05-08T08:53:46.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1900000000000001e-06 2024-05-08T08:53:46.350 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.350 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.350 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:46.351 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716958+0000", 2024-05-08T08:53:46.351 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8019999999999992e-06 2024-05-08T08:53:46.351 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.351 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.351 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:46.351 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768499+0000", 2024-05-08T08:53:46.351 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051540926000000001 2024-05-08T08:53:46.351 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.351 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.351 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:46.352 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768574+0000", 2024-05-08T08:53:46.352 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4988999999999996e-05 2024-05-08T08:53:46.352 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.352 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.352 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:46.352 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768673+0000", 2024-05-08T08:53:46.352 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9963000000000005e-05 2024-05-08T08:53:46.352 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:46.352 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:46.352 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:46.352 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.352 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.353 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6706 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1224704~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:46.353 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:53:46.353 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 74.612500835999995, 2024-05-08T08:53:46.353 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052435598, 2024-05-08T08:53:46.353 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:46.353 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:46.353 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:46.353 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:46.353 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:46.353 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:46.353 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6706 2024-05-08T08:53:46.353 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.353 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:46.354 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.354 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:46.354 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:53:46.354 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:46.354 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.354 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.354 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:46.354 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:53:46.354 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:46.354 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.354 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.355 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:46.355 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:53:46.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:53:46.355 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.355 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.355 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:46.355 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717035+0000", 2024-05-08T08:53:46.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9290000000000001e-06 2024-05-08T08:53:46.355 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.355 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.355 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:46.355 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717041+0000", 2024-05-08T08:53:46.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6720000000000001e-06 2024-05-08T08:53:46.356 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.356 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.356 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:46.356 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717046+0000", 2024-05-08T08:53:46.356 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5650000000000003e-06 2024-05-08T08:53:46.356 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.356 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.356 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:46.356 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769311+0000", 2024-05-08T08:53:46.356 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052265645999999999 2024-05-08T08:53:46.356 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.356 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.357 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:46.357 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769379+0000", 2024-05-08T08:53:46.357 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7391000000000006e-05 2024-05-08T08:53:46.357 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.357 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.357 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:46.357 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769469+0000", 2024-05-08T08:53:46.357 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0025999999999995e-05 2024-05-08T08:53:46.357 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:46.357 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:46.357 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:46.357 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.357 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.358 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6708 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1241088~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:46.358 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:53:46.358 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 74.612482732000004, 2024-05-08T08:53:46.358 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052640552, 2024-05-08T08:53:46.358 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:46.358 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:46.358 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:46.358 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:46.358 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:46.358 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:46.358 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6708 2024-05-08T08:53:46.358 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.358 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:46.358 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.359 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:46.359 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:53:46.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:46.359 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.359 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.359 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:46.359 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:53:46.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:46.359 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.359 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.359 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:46.359 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717052+0000", 2024-05-08T08:53:46.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:53:46.360 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.360 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.360 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:46.360 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717053+0000", 2024-05-08T08:53:46.360 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8330000000000001e-06 2024-05-08T08:53:46.360 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.360 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.360 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:46.360 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717058+0000", 2024-05-08T08:53:46.360 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.933e-06 2024-05-08T08:53:46.360 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.360 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.361 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:46.361 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717063+0000", 2024-05-08T08:53:46.361 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2180000000000001e-06 2024-05-08T08:53:46.361 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.361 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.361 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:46.361 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769507+0000", 2024-05-08T08:53:46.361 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052444866999999999 2024-05-08T08:53:46.361 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.361 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.361 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:46.361 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769589+0000", 2024-05-08T08:53:46.361 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2039000000000005e-05 2024-05-08T08:53:46.361 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.362 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.362 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:46.362 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769692+0000", 2024-05-08T08:53:46.362 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102252 2024-05-08T08:53:46.362 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:46.362 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:46.362 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:46.362 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.362 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.362 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6710 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1257472~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:46.362 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:53:46.362 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 74.612464708000005, 2024-05-08T08:53:46.362 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.05283057, 2024-05-08T08:53:46.362 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:46.363 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:46.363 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:46.363 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:46.363 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:46.363 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:46.363 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6710 2024-05-08T08:53:46.363 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.363 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:46.363 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.363 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:46.363 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:53:46.363 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:46.363 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.363 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.364 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:46.364 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:53:46.364 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:46.364 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.364 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.364 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:46.364 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718951+0000", 2024-05-08T08:53:46.364 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0018817809999999999 2024-05-08T08:53:46.364 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.364 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.364 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:46.364 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718956+0000", 2024-05-08T08:53:46.364 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5399999999999997e-06 2024-05-08T08:53:46.365 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.365 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.365 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:46.365 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718964+0000", 2024-05-08T08:53:46.365 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6629999999999999e-06 2024-05-08T08:53:46.365 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.365 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.365 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:46.365 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718972+0000", 2024-05-08T08:53:46.365 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5229999999999998e-06 2024-05-08T08:53:46.365 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.365 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.365 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:46.366 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769729+0000", 2024-05-08T08:53:46.366 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050756959999999997 2024-05-08T08:53:46.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.366 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.366 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:46.366 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769803+0000", 2024-05-08T08:53:46.366 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4538999999999999e-05 2024-05-08T08:53:46.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.366 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.366 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:46.366 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769900+0000", 2024-05-08T08:53:46.366 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6564000000000006e-05 2024-05-08T08:53:46.366 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:46.367 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:46.367 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:46.367 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.367 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.367 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6726 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1388544~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:46.367 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:53:46.367 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 74.610389936000004, 2024-05-08T08:53:46.367 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052451506000000002, 2024-05-08T08:53:46.367 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:46.367 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:46.367 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:46.367 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:46.367 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:46.367 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:46.368 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6726 2024-05-08T08:53:46.368 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.368 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:46.368 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.368 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:46.368 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:53:46.368 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:46.368 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.368 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.368 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:46.368 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:53:46.368 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:46.368 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.368 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.369 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:46.369 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719200+0000", 2024-05-08T08:53:46.369 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5865999999999997e-05 2024-05-08T08:53:46.369 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.369 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.369 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:46.369 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719204+0000", 2024-05-08T08:53:46.369 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8630000000000001e-06 2024-05-08T08:53:46.369 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.369 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.369 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:46.369 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719215+0000", 2024-05-08T08:53:46.369 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1590000000000001e-05 2024-05-08T08:53:46.369 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.370 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.370 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:46.370 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719220+0000", 2024-05-08T08:53:46.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1619999999999999e-06 2024-05-08T08:53:46.370 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.370 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.370 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:46.370 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771425+0000", 2024-05-08T08:53:46.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052204249000000001 2024-05-08T08:53:46.370 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.370 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.370 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:46.370 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771498+0000", 2024-05-08T08:53:46.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3782999999999995e-05 2024-05-08T08:53:46.371 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.371 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.371 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:46.371 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771595+0000", 2024-05-08T08:53:46.371 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6992999999999998e-05 2024-05-08T08:53:46.371 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:46.371 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:46.371 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:46.371 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.371 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.371 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6728 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1404928~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:46.371 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:53:46.371 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 74.610302531000002, 2024-05-08T08:53:46.371 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052590747, 2024-05-08T08:53:46.371 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:46.372 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:46.372 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:46.372 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:46.372 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:46.372 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:46.372 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6728 2024-05-08T08:53:46.372 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.372 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:46.372 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.372 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:46.372 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:53:46.372 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:46.372 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.372 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.373 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:46.373 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:53:46.373 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:46.373 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.373 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.373 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:46.373 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719232+0000", 2024-05-08T08:53:46.373 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4000000000000001e-07 2024-05-08T08:53:46.373 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.373 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.373 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:46.373 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719234+0000", 2024-05-08T08:53:46.374 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4499999999999998e-06 2024-05-08T08:53:46.374 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.374 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.374 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:46.374 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719240+0000", 2024-05-08T08:53:46.374 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5720000000000002e-06 2024-05-08T08:53:46.374 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.374 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.374 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:46.374 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719245+0000", 2024-05-08T08:53:46.374 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8690000000000003e-06 2024-05-08T08:53:46.374 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.374 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.375 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:46.375 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771636+0000", 2024-05-08T08:53:46.375 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052390979999999997 2024-05-08T08:53:46.375 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.375 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.375 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:46.375 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771717+0000", 2024-05-08T08:53:46.375 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1291999999999993e-05 2024-05-08T08:53:46.375 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.375 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.375 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:46.375 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771822+0000", 2024-05-08T08:53:46.375 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104944 2024-05-08T08:53:46.376 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:46.376 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:46.376 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:46.376 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.376 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.376 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6730 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1421312~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:46.376 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:53:46.376 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 74.610282288999997, 2024-05-08T08:53:46.376 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052770477000000003, 2024-05-08T08:53:46.376 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:46.376 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:46.376 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:46.377 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:46.377 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:46.377 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:46.377 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6730 2024-05-08T08:53:46.377 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.377 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:46.377 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.377 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:46.377 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:53:46.377 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:46.377 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.377 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.377 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:46.377 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:53:46.378 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:46.378 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.378 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.378 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:46.378 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:53:46.378 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9000000000000002e-07 2024-05-08T08:53:46.378 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.378 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.378 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:46.378 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719254+0000", 2024-05-08T08:53:46.378 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4099999999999998e-06 2024-05-08T08:53:46.378 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.378 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.379 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:46.379 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719263+0000", 2024-05-08T08:53:46.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4109999999999996e-06 2024-05-08T08:53:46.379 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.379 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.379 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:46.379 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719268+0000", 2024-05-08T08:53:46.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2669999999999997e-06 2024-05-08T08:53:46.379 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.379 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.379 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:46.379 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771846+0000", 2024-05-08T08:53:46.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052577677000000003 2024-05-08T08:53:46.379 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.380 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.380 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:46.380 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771920+0000", 2024-05-08T08:53:46.380 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4460000000000002e-05 2024-05-08T08:53:46.380 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.380 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.380 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:46.380 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772022+0000", 2024-05-08T08:53:46.380 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010186199999999999 2024-05-08T08:53:46.380 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:46.380 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:46.380 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:46.380 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.380 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.380 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6732 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1437696~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:46.381 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:53:46.381 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 74.610257481999994, 2024-05-08T08:53:46.381 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052905575000000003, 2024-05-08T08:53:46.381 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:46.381 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:46.381 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:46.381 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:46.381 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:46.381 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:46.381 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6732 2024-05-08T08:53:46.381 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.381 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:46.381 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.381 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:46.382 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:53:46.382 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:46.382 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.382 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.382 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:46.382 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:53:46.382 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:46.382 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.382 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.382 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:46.382 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721392+0000", 2024-05-08T08:53:46.382 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002115287 2024-05-08T08:53:46.382 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.382 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.383 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:46.383 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721396+0000", 2024-05-08T08:53:46.383 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1679999999999997e-06 2024-05-08T08:53:46.383 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.383 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.383 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:46.383 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721404+0000", 2024-05-08T08:53:46.383 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4409999999999993e-06 2024-05-08T08:53:46.383 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.383 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.383 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:46.383 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721410+0000", 2024-05-08T08:53:46.383 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9150000000000001e-06 2024-05-08T08:53:46.383 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.384 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:46.384 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772061+0000", 2024-05-08T08:53:46.384 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050651068 2024-05-08T08:53:46.384 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.384 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:46.384 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772127+0000", 2024-05-08T08:53:46.384 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5424000000000002e-05 2024-05-08T08:53:46.384 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.384 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:46.384 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772182+0000", 2024-05-08T08:53:46.384 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5272000000000002e-05 2024-05-08T08:53:46.385 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:46.385 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:46.385 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:46.385 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.385 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.385 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6749 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1585152~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:46.385 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:53:46.385 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 74.607956712000004, 2024-05-08T08:53:46.385 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052350264000000001, 2024-05-08T08:53:46.385 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:46.385 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:46.385 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:46.386 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:46.386 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:46.386 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:46.386 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6749 2024-05-08T08:53:46.386 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.386 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:46.386 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.386 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:46.386 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:53:46.386 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:46.386 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.386 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.387 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:46.387 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:53:46.387 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:46.387 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.387 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.387 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:46.387 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721578+0000", 2024-05-08T08:53:46.387 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0399999999999996e-07 2024-05-08T08:53:46.387 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.387 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.387 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:46.387 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721580+0000", 2024-05-08T08:53:46.387 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5450000000000002e-06 2024-05-08T08:53:46.387 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.388 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.388 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:46.388 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721587+0000", 2024-05-08T08:53:46.388 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0190000000000001e-06 2024-05-08T08:53:46.388 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.388 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.388 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:46.388 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721593+0000", 2024-05-08T08:53:46.388 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5659999999999997e-06 2024-05-08T08:53:46.388 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.388 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.388 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:46.388 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773759+0000", 2024-05-08T08:53:46.389 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052166553999999997 2024-05-08T08:53:46.389 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.389 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.389 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:46.389 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773833+0000", 2024-05-08T08:53:46.389 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3913999999999997e-05 2024-05-08T08:53:46.389 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.389 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.389 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:46.389 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773927+0000", 2024-05-08T08:53:46.389 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4061999999999995e-05 2024-05-08T08:53:46.389 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:46.390 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:46.390 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:46.390 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.390 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.390 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6751 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1601536~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:46.390 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:53:46.390 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 74.607933432999999, 2024-05-08T08:53:46.390 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052461302000000001, 2024-05-08T08:53:46.390 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:46.390 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:46.390 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:46.390 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:46.391 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:46.391 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:46.391 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6751 2024-05-08T08:53:46.391 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.391 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:46.391 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.391 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:46.391 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:53:46.391 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:46.391 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.391 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.391 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:46.392 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:53:46.392 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:46.392 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.392 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.392 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:46.392 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722939+0000", 2024-05-08T08:53:46.392 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001338601 2024-05-08T08:53:46.392 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.392 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.392 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:46.392 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722944+0000", 2024-05-08T08:53:46.392 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4469999999999999e-06 2024-05-08T08:53:46.392 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.393 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.393 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:46.393 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722955+0000", 2024-05-08T08:53:46.393 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1892999999999999e-05 2024-05-08T08:53:46.393 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.393 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.393 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:46.393 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722963+0000", 2024-05-08T08:53:46.393 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1080000000000004e-06 2024-05-08T08:53:46.393 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.393 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.393 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:46.393 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773964+0000", 2024-05-08T08:53:46.394 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051001022 2024-05-08T08:53:46.394 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.394 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.394 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:46.394 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.774006+0000", 2024-05-08T08:53:46.394 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2729999999999999e-05 2024-05-08T08:53:46.394 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.394 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.394 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:46.394 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.774062+0000", 2024-05-08T08:53:46.394 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5501000000000003e-05 2024-05-08T08:53:46.394 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:46.395 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:46.395 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:46.395 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.395 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.395 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6760 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1683456~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:46.395 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:53:46.395 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 74.606424056999998, 2024-05-08T08:53:46.395 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052571312000000002, 2024-05-08T08:53:46.395 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:46.395 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:46.395 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:46.395 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:46.395 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:46.396 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:46.396 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6760 2024-05-08T08:53:46.396 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.396 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:46.396 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.396 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:46.396 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:53:46.396 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:46.396 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.396 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.396 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:46.396 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:53:46.396 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:46.397 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.397 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.397 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:46.397 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:53:46.397 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2100000000000002e-07 2024-05-08T08:53:46.397 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.397 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.397 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:46.397 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723113+0000", 2024-05-08T08:53:46.397 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3149999999999999e-06 2024-05-08T08:53:46.397 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.397 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.397 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:46.398 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723118+0000", 2024-05-08T08:53:46.398 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8860000000000001e-06 2024-05-08T08:53:46.398 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.398 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.398 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:46.398 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723123+0000", 2024-05-08T08:53:46.398 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6770000000000003e-06 2024-05-08T08:53:46.398 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.398 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.398 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:46.398 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775414+0000", 2024-05-08T08:53:46.398 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052290692999999999 2024-05-08T08:53:46.398 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.399 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.399 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:46.399 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775459+0000", 2024-05-08T08:53:46.399 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5639999999999997e-05 2024-05-08T08:53:46.399 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.399 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.399 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:46.399 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775681+0000", 2024-05-08T08:53:46.399 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00022168000000000001 2024-05-08T08:53:46.399 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:46.399 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:46.399 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:46.399 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.399 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.400 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6761 3.16 3:6bd6d6da:::rbd_data.1140782590a5.0000000000000015:6 [read 1095680~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:46.400 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:53:46.400 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 74.606404836999999, 2024-05-08T08:53:46.400 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052751275, 2024-05-08T08:53:46.400 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:46.400 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:46.400 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:46.400 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:46.400 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:46.400 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:46.400 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6761 2024-05-08T08:53:46.400 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.400 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:46.400 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.401 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:46.401 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:53:46.401 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:46.401 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.401 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.401 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:46.401 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:53:46.401 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:46.401 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.401 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.401 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:46.401 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724939+0000", 2024-05-08T08:53:46.401 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001809817 2024-05-08T08:53:46.401 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.402 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.402 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:46.402 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724943+0000", 2024-05-08T08:53:46.402 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4920000000000004e-06 2024-05-08T08:53:46.402 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.402 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.402 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:46.402 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724956+0000", 2024-05-08T08:53:46.402 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2427e-05 2024-05-08T08:53:46.402 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.402 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.402 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:46.403 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724963+0000", 2024-05-08T08:53:46.403 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2559999999999996e-06 2024-05-08T08:53:46.403 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.403 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.403 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:46.403 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775711+0000", 2024-05-08T08:53:46.403 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050747621 2024-05-08T08:53:46.403 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.403 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.403 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:46.403 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775785+0000", 2024-05-08T08:53:46.403 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4634000000000004e-05 2024-05-08T08:53:46.403 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.403 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.404 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:46.404 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775880+0000", 2024-05-08T08:53:46.404 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5027999999999999e-05 2024-05-08T08:53:46.404 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:46.404 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:46.404 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:46.404 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.404 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.404 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6770 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1765376~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:46.404 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:53:46.404 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 74.604373742000007, 2024-05-08T08:53:46.404 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052339296, 2024-05-08T08:53:46.404 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:46.404 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:46.405 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:46.405 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:46.405 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:46.405 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:46.405 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6770 2024-05-08T08:53:46.405 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.405 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:46.405 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.405 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:46.405 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:53:46.405 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:46.405 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.405 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.406 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:46.406 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:53:46.406 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:46.406 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.406 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.406 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:46.406 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725161+0000", 2024-05-08T08:53:46.406 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-05-08T08:53:46.406 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.406 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.406 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:46.406 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725163+0000", 2024-05-08T08:53:46.406 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4370000000000001e-06 2024-05-08T08:53:46.407 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.407 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.407 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:46.407 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725169+0000", 2024-05-08T08:53:46.407 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8409999999999996e-06 2024-05-08T08:53:46.407 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.407 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.407 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:46.407 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725174+0000", 2024-05-08T08:53:46.407 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7249999999999997e-06 2024-05-08T08:53:46.407 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.407 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.408 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:46.408 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777343+0000", 2024-05-08T08:53:46.408 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052169633999999999 2024-05-08T08:53:46.408 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.408 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.408 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:46.408 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777409+0000", 2024-05-08T08:53:46.408 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6033999999999998e-05 2024-05-08T08:53:46.408 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.408 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.408 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:46.408 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777499+0000", 2024-05-08T08:53:46.409 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0224999999999998e-05 2024-05-08T08:53:46.409 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:46.409 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:46.409 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:46.409 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.409 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.409 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6771 3.16 3:6bd6d6da:::rbd_data.1140782590a5.0000000000000015:6 [read 1177600~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:46.409 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:53:46.409 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 74.604353924999998, 2024-05-08T08:53:46.409 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052438604999999999, 2024-05-08T08:53:46.409 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:46.409 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:46.409 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:46.410 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:46.410 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:46.410 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:46.410 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6771 2024-05-08T08:53:46.410 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.410 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:46.410 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.410 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:46.410 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:53:46.410 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:46.410 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.410 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.410 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:46.411 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:53:46.411 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:46.411 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.411 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.411 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:46.411 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:53:46.411 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:53:46.411 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.411 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.411 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:46.411 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725188+0000", 2024-05-08T08:53:46.411 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1579999999999999e-06 2024-05-08T08:53:46.411 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.412 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.412 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:46.412 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725195+0000", 2024-05-08T08:53:46.412 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8909999999999995e-06 2024-05-08T08:53:46.412 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.412 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.412 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:46.412 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725200+0000", 2024-05-08T08:53:46.412 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9810000000000003e-06 2024-05-08T08:53:46.412 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.412 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.412 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:46.412 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777527+0000", 2024-05-08T08:53:46.413 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052326582000000003 2024-05-08T08:53:46.413 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.413 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.413 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:46.413 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777566+0000", 2024-05-08T08:53:46.413 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9447999999999999e-05 2024-05-08T08:53:46.413 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.413 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.413 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:46.413 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777619+0000", 2024-05-08T08:53:46.413 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2135000000000001e-05 2024-05-08T08:53:46.413 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:46.413 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:46.413 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:46.414 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.414 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.414 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6772 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1781760~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:46.414 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:53:46.414 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 74.604325868000004, 2024-05-08T08:53:46.414 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052601799999999997, 2024-05-08T08:53:46.414 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:46.414 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:46.414 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:46.414 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:46.414 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:46.415 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:46.415 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6772 2024-05-08T08:53:46.415 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.415 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:46.415 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.415 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:46.415 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:53:46.415 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:46.415 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.415 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.415 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:46.415 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:53:46.415 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:46.416 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.416 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.416 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:46.416 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726939+0000", 2024-05-08T08:53:46.416 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001730493 2024-05-08T08:53:46.416 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.416 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.416 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:46.416 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726943+0000", 2024-05-08T08:53:46.416 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7330000000000003e-06 2024-05-08T08:53:46.416 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.416 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.416 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:46.417 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726954+0000", 2024-05-08T08:53:46.417 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1107000000000001e-05 2024-05-08T08:53:46.417 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.417 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.417 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:46.417 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726962+0000", 2024-05-08T08:53:46.417 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.503e-06 2024-05-08T08:53:46.417 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.417 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.417 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:46.417 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777653+0000", 2024-05-08T08:53:46.417 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050691542999999999 2024-05-08T08:53:46.417 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.418 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.418 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:46.418 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777719+0000", 2024-05-08T08:53:46.418 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5228000000000006e-05 2024-05-08T08:53:46.418 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.418 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.418 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:46.418 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777810+0000", 2024-05-08T08:53:46.418 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1193000000000006e-05 2024-05-08T08:53:46.418 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:46.418 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:46.418 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:46.418 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.419 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.419 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6782 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1880064~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:46.419 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:53:46.419 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 74.602357361000003, 2024-05-08T08:53:46.419 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052326492000000002, 2024-05-08T08:53:46.419 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:46.419 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:46.419 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:46.419 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:46.419 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:46.419 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:46.419 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6782 2024-05-08T08:53:46.419 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.419 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:46.420 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.420 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:46.420 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:53:46.420 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:46.420 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.420 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.420 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:46.420 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:53:46.420 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:46.420 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.420 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.420 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:46.420 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728939+0000", 2024-05-08T08:53:46.420 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001762719 2024-05-08T08:53:46.421 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.421 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.421 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:46.421 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728944+0000", 2024-05-08T08:53:46.421 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.746e-06 2024-05-08T08:53:46.421 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.421 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.421 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:46.421 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728955+0000", 2024-05-08T08:53:46.421 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1043000000000001e-05 2024-05-08T08:53:46.421 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.421 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.421 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:46.421 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728963+0000", 2024-05-08T08:53:46.422 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5809999999999999e-06 2024-05-08T08:53:46.422 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.422 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.422 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:46.422 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779320+0000", 2024-05-08T08:53:46.422 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050357290999999998 2024-05-08T08:53:46.422 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.422 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.422 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:46.422 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779401+0000", 2024-05-08T08:53:46.422 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0574000000000005e-05 2024-05-08T08:53:46.422 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:46.422 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:46.422 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:46.423 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779503+0000", 2024-05-08T08:53:46.423 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102538 2024-05-08T08:53:46.423 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:46.423 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:46.423 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:46.423 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:46.423 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:46.423 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:53:46.425 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:46.426+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:46.425 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:53:46.525 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:46.528+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:46.547 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:46.548 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:53:46.548 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:53:46.548 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:46.557 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:53:46.626 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:46.628+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:46.707 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:46.707 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:53:46.707 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:53:46.707 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:53:46.707 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:46.716 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:53:46.727 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:46.730+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:46.828 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:46.828 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:53:46.829 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:53:46.829 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:53:46.829 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.829 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4621.0:9089 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 23339520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:53:46.829 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:53:46.829 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 74.337408418999999, 2024-05-08T08:53:46.829 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.13814907200000001, 2024-05-08T08:53:46.829 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:46.829 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:46.829 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:53:46.829 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:46.829 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4621", 2024-05-08T08:53:46.829 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:46.829 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9089 2024-05-08T08:53:46.830 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.830 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:46.830 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.830 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:46.830 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:53:46.830 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:46.830 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.830 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.830 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:46.830 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:53:46.830 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:46.830 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.830 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.830 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:46.831 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493005+0000", 2024-05-08T08:53:46.831 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1700000000000001e-06 2024-05-08T08:53:46.831 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.831 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.831 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:46.831 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.513963+0000", 2024-05-08T08:53:46.831 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.020957581999999999 2024-05-08T08:53:46.831 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.831 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.831 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:46.831 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.513995+0000", 2024-05-08T08:53:46.831 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1993999999999999e-05 2024-05-08T08:53:46.831 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.831 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.831 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:46.832 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514007+0000", 2024-05-08T08:53:46.832 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1581e-05 2024-05-08T08:53:46.832 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.832 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.832 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:46.832 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514122+0000", 2024-05-08T08:53:46.832 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000115282 2024-05-08T08:53:46.832 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.832 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.832 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op must wait for map", 2024-05-08T08:53:46.832 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514128+0000", 2024-05-08T08:53:46.832 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9950000000000002e-06 2024-05-08T08:53:46.832 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.832 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.832 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:46.833 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.515800+0000", 2024-05-08T08:53:46.833 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001671886 2024-05-08T08:53:46.833 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.833 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.833 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:46.833 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.515893+0000", 2024-05-08T08:53:46.833 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3490000000000001e-05 2024-05-08T08:53:46.833 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.833 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.833 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 3", 2024-05-08T08:53:46.833 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.517647+0000", 2024-05-08T08:53:46.833 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0017535630000000001 2024-05-08T08:53:46.833 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.833 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.833 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:53:46.833 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.627578+0000", 2024-05-08T08:53:46.833 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.109931557 2024-05-08T08:53:46.834 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.834 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.834 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:53:46.834 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631055+0000", 2024-05-08T08:53:46.834 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0034761420000000002 2024-05-08T08:53:46.834 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.834 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.834 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:53:46.834 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631096+0000", 2024-05-08T08:53:46.834 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1059000000000001e-05 2024-05-08T08:53:46.834 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.834 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.834 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:46.834 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631151+0000", 2024-05-08T08:53:46.834 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5770999999999998e-05 2024-05-08T08:53:46.835 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:46.835 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:46.835 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:46.835 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.835 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.835 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1071 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11026432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:46.835 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:53:46.835 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 70.344958927999997, 2024-05-08T08:53:46.835 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070580037999999998, 2024-05-08T08:53:46.835 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:46.835 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:46.835 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:46.835 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:46.835 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:46.836 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:46.836 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1071 2024-05-08T08:53:46.836 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.836 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:46.836 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.836 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:46.836 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:53:46.836 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:46.836 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.836 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.836 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:46.836 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:53:46.836 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:46.836 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.837 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.837 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:46.837 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:53:46.837 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2800000000000002e-07 2024-05-08T08:53:46.837 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.837 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.837 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:46.837 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485454+0000", 2024-05-08T08:53:46.837 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2290000000000002e-06 2024-05-08T08:53:46.837 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.837 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.837 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:46.837 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485461+0000", 2024-05-08T08:53:46.837 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9920000000000001e-06 2024-05-08T08:53:46.838 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.838 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.838 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:46.838 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485467+0000", 2024-05-08T08:53:46.838 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3979999999999998e-06 2024-05-08T08:53:46.838 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.838 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.838 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:46.838 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555880+0000", 2024-05-08T08:53:46.838 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070413165 2024-05-08T08:53:46.838 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.838 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.838 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:46.838 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555926+0000", 2024-05-08T08:53:46.839 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6217000000000002e-05 2024-05-08T08:53:46.839 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.839 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.839 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:46.839 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556032+0000", 2024-05-08T08:53:46.839 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000105609 2024-05-08T08:53:46.839 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:46.839 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:46.839 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:46.839 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.839 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.839 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1073 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11067392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:46.839 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:53:46.839 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 70.344937707, 2024-05-08T08:53:46.839 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070821404000000004, 2024-05-08T08:53:46.839 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:46.840 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:46.840 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:46.840 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:46.840 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:46.840 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:46.840 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1073 2024-05-08T08:53:46.840 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.840 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:46.840 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.840 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:46.840 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:53:46.840 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:46.840 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.840 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.840 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:46.841 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:53:46.841 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:46.841 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.841 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.841 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:46.841 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:53:46.841 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.08e-07 2024-05-08T08:53:46.841 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.841 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.841 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:46.841 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485476+0000", 2024-05-08T08:53:46.841 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3929999999999998e-06 2024-05-08T08:53:46.841 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.841 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.841 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:46.842 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485485+0000", 2024-05-08T08:53:46.842 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1880000000000008e-06 2024-05-08T08:53:46.842 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.842 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.842 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:46.842 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485494+0000", 2024-05-08T08:53:46.842 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3109999999999995e-06 2024-05-08T08:53:46.842 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.842 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.842 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:46.842 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556082+0000", 2024-05-08T08:53:46.842 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070587426999999994 2024-05-08T08:53:46.842 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.843 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.843 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:46.843 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556177+0000", 2024-05-08T08:53:46.843 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5051999999999997e-05 2024-05-08T08:53:46.843 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.843 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.843 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:46.843 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556294+0000", 2024-05-08T08:53:46.843 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000117625 2024-05-08T08:53:46.843 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:46.843 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:46.843 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:46.843 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.843 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.844 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1075 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11108352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:46.844 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:53:46.844 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 70.344907129999996, 2024-05-08T08:53:46.844 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071045026999999997, 2024-05-08T08:53:46.844 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:46.844 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:46.844 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:46.844 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:46.844 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:46.844 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:46.844 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1075 2024-05-08T08:53:46.844 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.844 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:46.844 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.844 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:46.844 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:53:46.845 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:46.845 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.845 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.845 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:46.845 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:53:46.845 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:46.845 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.845 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.845 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:46.845 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:53:46.845 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.32e-07 2024-05-08T08:53:46.845 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.845 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.845 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:46.845 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485518+0000", 2024-05-08T08:53:46.845 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3769e-05 2024-05-08T08:53:46.845 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.846 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.846 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:46.846 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485527+0000", 2024-05-08T08:53:46.846 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6779999999999995e-06 2024-05-08T08:53:46.846 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.846 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.846 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:46.846 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485532+0000", 2024-05-08T08:53:46.846 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9830000000000004e-06 2024-05-08T08:53:46.846 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.846 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.846 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:46.846 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556340+0000", 2024-05-08T08:53:46.846 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070807743000000006 2024-05-08T08:53:46.847 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.847 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.847 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:46.847 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556435+0000", 2024-05-08T08:53:46.847 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4732e-05 2024-05-08T08:53:46.847 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.847 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.847 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:46.847 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556549+0000", 2024-05-08T08:53:46.847 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011369 2024-05-08T08:53:46.847 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:46.847 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:46.847 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:46.847 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.847 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.848 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1078 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11149312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:46.848 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:53:46.848 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 70.344866930999999, 2024-05-08T08:53:46.848 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071167276000000002, 2024-05-08T08:53:46.848 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:46.848 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:46.848 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:46.848 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:46.848 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:46.848 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:46.848 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1078 2024-05-08T08:53:46.848 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.848 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:46.848 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.848 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:46.848 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:53:46.849 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:46.849 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.849 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.849 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:46.849 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:53:46.849 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:46.849 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.849 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.849 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:46.849 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:53:46.849 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5600000000000001e-07 2024-05-08T08:53:46.849 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.849 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.849 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:46.849 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485547+0000", 2024-05-08T08:53:46.849 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3590000000000002e-06 2024-05-08T08:53:46.850 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.850 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.850 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:46.850 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485553+0000", 2024-05-08T08:53:46.850 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2770000000000002e-06 2024-05-08T08:53:46.850 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.850 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.850 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:46.850 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485558+0000", 2024-05-08T08:53:46.850 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.95e-06 2024-05-08T08:53:46.850 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.850 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.850 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:46.850 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556582+0000", 2024-05-08T08:53:46.851 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071024444000000006 2024-05-08T08:53:46.851 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.851 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.851 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:46.851 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556637+0000", 2024-05-08T08:53:46.851 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4672000000000001e-05 2024-05-08T08:53:46.851 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.851 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.851 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:46.851 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556711+0000", 2024-05-08T08:53:46.851 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4117999999999997e-05 2024-05-08T08:53:46.851 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:46.851 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:46.852 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:46.852 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.852 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.852 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1080 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11190272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:46.852 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:53:46.852 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 70.344846197999999, 2024-05-08T08:53:46.852 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071350721000000006, 2024-05-08T08:53:46.852 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:46.852 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:46.852 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:46.852 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:46.852 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:46.852 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:46.853 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1080 2024-05-08T08:53:46.853 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.853 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:46.853 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.853 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:46.853 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:53:46.853 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:46.853 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.853 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.853 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:46.853 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:53:46.853 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:46.853 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.853 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.853 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:46.853 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:53:46.854 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.03e-07 2024-05-08T08:53:46.854 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.854 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.854 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:46.854 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485567+0000", 2024-05-08T08:53:46.854 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1090000000000001e-06 2024-05-08T08:53:46.854 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.854 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.854 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:46.854 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485573+0000", 2024-05-08T08:53:46.854 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3049999999999998e-06 2024-05-08T08:53:46.854 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.854 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.854 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:46.854 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485578+0000", 2024-05-08T08:53:46.855 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0130000000000001e-06 2024-05-08T08:53:46.855 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.855 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.855 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:46.855 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556743+0000", 2024-05-08T08:53:46.855 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071164449000000005 2024-05-08T08:53:46.855 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.855 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.855 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:46.855 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556797+0000", 2024-05-08T08:53:46.855 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4589999999999997e-05 2024-05-08T08:53:46.855 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.855 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.855 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:46.855 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556915+0000", 2024-05-08T08:53:46.855 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000117852 2024-05-08T08:53:46.856 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:46.856 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:46.856 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:46.856 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.856 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.856 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1082 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11231232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:46.856 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:53:46.856 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 70.344825955000005, 2024-05-08T08:53:46.856 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071591396000000002, 2024-05-08T08:53:46.856 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:46.856 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:46.856 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:46.856 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:46.856 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:46.856 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:46.856 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1082 2024-05-08T08:53:46.856 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.857 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:46.857 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.857 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:46.857 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:53:46.857 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:46.857 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.857 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.857 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:46.857 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:53:46.857 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:46.857 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.857 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.857 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:46.857 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:53:46.857 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1800000000000001e-07 2024-05-08T08:53:46.857 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.858 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.858 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:46.858 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485587+0000", 2024-05-08T08:53:46.858 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2589999999999999e-06 2024-05-08T08:53:46.858 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.858 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.858 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:46.858 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485594+0000", 2024-05-08T08:53:46.858 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0789999999999997e-06 2024-05-08T08:53:46.858 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.858 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.858 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:46.858 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485598+0000", 2024-05-08T08:53:46.858 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7480000000000001e-06 2024-05-08T08:53:46.858 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.858 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.859 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:46.859 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556971+0000", 2024-05-08T08:53:46.859 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071372774999999999 2024-05-08T08:53:46.859 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.859 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.859 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:46.859 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557069+0000", 2024-05-08T08:53:46.859 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.7732999999999994e-05 2024-05-08T08:53:46.859 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.859 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.859 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:46.859 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557176+0000", 2024-05-08T08:53:46.859 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000107384 2024-05-08T08:53:46.859 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:46.859 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:46.859 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:46.860 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.860 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.860 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1084 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11272192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:46.860 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:53:46.860 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 70.344787315999994, 2024-05-08T08:53:46.860 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071808977999999996, 2024-05-08T08:53:46.860 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:46.860 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:46.860 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:46.860 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:46.860 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:46.860 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:46.860 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1084 2024-05-08T08:53:46.860 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.860 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:46.860 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.860 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:46.861 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:53:46.861 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:46.861 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.861 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.861 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:46.861 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:53:46.861 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:46.861 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.861 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.861 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:46.861 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485624+0000", 2024-05-08T08:53:46.861 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:53:46.861 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.861 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.861 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:46.861 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485626+0000", 2024-05-08T08:53:46.861 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9460000000000001e-06 2024-05-08T08:53:46.862 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.862 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.862 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:46.862 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485633+0000", 2024-05-08T08:53:46.862 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1149999999999996e-06 2024-05-08T08:53:46.862 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.862 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.862 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:46.862 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485638+0000", 2024-05-08T08:53:46.862 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5670000000000002e-06 2024-05-08T08:53:46.862 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.862 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.862 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:46.862 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557226+0000", 2024-05-08T08:53:46.862 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071587267999999996 2024-05-08T08:53:46.862 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.863 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.863 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:46.863 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557321+0000", 2024-05-08T08:53:46.863 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5785000000000005e-05 2024-05-08T08:53:46.863 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.863 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.863 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:46.863 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557432+0000", 2024-05-08T08:53:46.863 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011092800000000001 2024-05-08T08:53:46.863 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:46.863 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:46.863 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:46.863 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.863 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.863 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1086 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11313152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:46.864 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:53:46.864 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 70.344763989, 2024-05-08T08:53:46.864 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071933132999999996, 2024-05-08T08:53:46.864 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:46.864 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:46.864 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:46.864 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:46.864 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:46.864 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:46.864 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1086 2024-05-08T08:53:46.864 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.864 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:46.864 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.864 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:46.864 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:53:46.864 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:46.865 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.865 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.865 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:46.865 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:53:46.865 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:46.865 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.865 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.865 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:46.865 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486328+0000", 2024-05-08T08:53:46.865 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00068107900000000002 2024-05-08T08:53:46.865 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.865 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.865 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:46.865 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486331+0000", 2024-05-08T08:53:46.865 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.585e-06 2024-05-08T08:53:46.865 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.866 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.866 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:46.866 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486343+0000", 2024-05-08T08:53:46.866 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1856e-05 2024-05-08T08:53:46.866 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.866 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.866 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:46.866 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486350+0000", 2024-05-08T08:53:46.866 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8600000000000004e-06 2024-05-08T08:53:46.866 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.866 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.866 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:46.866 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557470+0000", 2024-05-08T08:53:46.866 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071120326999999997 2024-05-08T08:53:46.866 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.867 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.867 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:46.867 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557514+0000", 2024-05-08T08:53:46.867 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3909000000000003e-05 2024-05-08T08:53:46.867 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.867 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.867 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:46.867 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557580+0000", 2024-05-08T08:53:46.867 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5517000000000003e-05 2024-05-08T08:53:46.867 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:46.867 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:46.867 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:46.867 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.867 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.867 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1089 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11354112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:46.868 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:53:46.868 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 70.344051117000006, 2024-05-08T08:53:46.868 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071457381, 2024-05-08T08:53:46.868 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:46.868 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:46.868 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:46.868 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:46.868 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:46.868 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:46.868 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1089 2024-05-08T08:53:46.868 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.868 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:46.868 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.868 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:46.869 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:53:46.869 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:46.869 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.869 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.869 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:46.869 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:53:46.869 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:46.869 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.869 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.869 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:46.869 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486851+0000", 2024-05-08T08:53:46.869 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00049102 2024-05-08T08:53:46.869 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.869 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.869 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:46.869 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486854+0000", 2024-05-08T08:53:46.869 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6859999999999999e-06 2024-05-08T08:53:46.870 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.870 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.870 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:46.870 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486864+0000", 2024-05-08T08:53:46.870 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6080000000000002e-06 2024-05-08T08:53:46.870 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.870 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.870 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:46.870 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486870+0000", 2024-05-08T08:53:46.870 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.477e-06 2024-05-08T08:53:46.870 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.870 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.870 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:46.871 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557612+0000", 2024-05-08T08:53:46.871 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070741557999999996 2024-05-08T08:53:46.871 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.871 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.871 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:46.871 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557695+0000", 2024-05-08T08:53:46.871 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3300000000000005e-05 2024-05-08T08:53:46.871 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.871 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.871 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:46.871 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557817+0000", 2024-05-08T08:53:46.871 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012173200000000001 2024-05-08T08:53:46.871 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:46.871 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:46.871 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:46.871 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.872 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.872 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1091 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11395072~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:46.872 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:53:46.872 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 70.343530322000007, 2024-05-08T08:53:46.872 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071184737999999997, 2024-05-08T08:53:46.872 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:46.872 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:46.872 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:46.872 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:46.872 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:46.872 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:46.872 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1091 2024-05-08T08:53:46.872 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.872 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:46.872 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.872 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:46.872 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:53:46.873 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:46.873 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.873 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.873 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:46.873 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:53:46.873 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:46.873 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.873 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.873 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:46.873 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486881+0000", 2024-05-08T08:53:46.873 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0699999999999997e-07 2024-05-08T08:53:46.873 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.873 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.874 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:46.874 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486883+0000", 2024-05-08T08:53:46.874 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.402e-06 2024-05-08T08:53:46.874 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.874 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.874 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:46.874 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486890+0000", 2024-05-08T08:53:46.874 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0480000000000001e-06 2024-05-08T08:53:46.874 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.874 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.874 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:46.874 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486896+0000", 2024-05-08T08:53:46.874 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7640000000000002e-06 2024-05-08T08:53:46.874 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.875 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.875 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:46.875 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557860+0000", 2024-05-08T08:53:46.875 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070963791999999998 2024-05-08T08:53:46.875 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.875 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.875 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:46.875 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557952+0000", 2024-05-08T08:53:46.875 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2082000000000004e-05 2024-05-08T08:53:46.875 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.875 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.875 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:46.875 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558065+0000", 2024-05-08T08:53:46.875 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000113143 2024-05-08T08:53:46.875 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:46.875 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:46.876 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:46.876 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.876 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.876 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1094 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11436032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:46.876 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:53:46.876 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 70.343507153999994, 2024-05-08T08:53:46.876 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071400507000000002, 2024-05-08T08:53:46.876 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:46.876 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:46.876 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:46.876 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:46.876 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:46.876 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:46.876 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1094 2024-05-08T08:53:46.876 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.876 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:46.876 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.877 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:46.877 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:53:46.877 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:46.877 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.877 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.877 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:46.877 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:53:46.877 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:46.877 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.877 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.877 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:46.877 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487940+0000", 2024-05-08T08:53:46.877 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001036868 2024-05-08T08:53:46.877 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.877 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.877 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:46.877 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487944+0000", 2024-05-08T08:53:46.878 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7069999999999998e-06 2024-05-08T08:53:46.878 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.878 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.878 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:46.878 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487951+0000", 2024-05-08T08:53:46.878 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2649999999999999e-06 2024-05-08T08:53:46.878 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.878 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.878 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:46.878 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487961+0000", 2024-05-08T08:53:46.878 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4110000000000002e-06 2024-05-08T08:53:46.878 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.878 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.878 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:46.879 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558117+0000", 2024-05-08T08:53:46.879 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070156662999999994 2024-05-08T08:53:46.879 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.879 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.879 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:46.879 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558214+0000", 2024-05-08T08:53:46.879 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6444e-05 2024-05-08T08:53:46.879 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.879 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.879 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:46.879 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558304+0000", 2024-05-08T08:53:46.879 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0148999999999994e-05 2024-05-08T08:53:46.879 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:46.879 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:46.879 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:46.880 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.880 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.880 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1096 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11476992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:46.880 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:53:46.880 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 70.342440623000002, 2024-05-08T08:53:46.880 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070480074000000004, 2024-05-08T08:53:46.880 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:46.880 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:46.880 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:46.880 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:46.880 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:46.880 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:46.880 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1096 2024-05-08T08:53:46.880 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.880 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:46.880 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.881 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:46.881 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:53:46.881 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:46.881 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.881 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.881 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:46.881 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:53:46.881 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:46.881 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.881 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.881 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:46.881 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:53:46.881 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3700000000000001e-07 2024-05-08T08:53:46.881 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.881 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.882 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:46.882 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487973+0000", 2024-05-08T08:53:46.882 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0870000000000001e-06 2024-05-08T08:53:46.882 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.882 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.882 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:46.882 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487977+0000", 2024-05-08T08:53:46.882 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0600000000000001e-06 2024-05-08T08:53:46.882 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.882 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.882 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:46.882 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487980+0000", 2024-05-08T08:53:46.882 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1089999999999998e-06 2024-05-08T08:53:46.882 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.882 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.883 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:46.883 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558338+0000", 2024-05-08T08:53:46.883 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070358233000000006 2024-05-08T08:53:46.883 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.883 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.883 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:46.883 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558392+0000", 2024-05-08T08:53:46.883 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4537999999999999e-05 2024-05-08T08:53:46.883 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.883 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.883 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:46.883 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558450+0000", 2024-05-08T08:53:46.883 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7710000000000001e-05 2024-05-08T08:53:46.883 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:46.884 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:46.884 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:46.884 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.884 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.884 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1099 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11517952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:46.884 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:53:46.884 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 70.342426747000005, 2024-05-08T08:53:46.884 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070720506000000002, 2024-05-08T08:53:46.885 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:46.885 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:46.885 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:46.885 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:46.885 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:46.885 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:46.886 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1099 2024-05-08T08:53:46.886 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.886 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:46.886 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.886 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:46.886 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:53:46.887 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:46.887 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.887 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.887 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:46.887 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:53:46.887 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:46.887 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.887 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.888 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:46.888 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:53:46.888 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3999999999999998e-07 2024-05-08T08:53:46.888 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.888 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.888 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:46.888 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487986+0000", 2024-05-08T08:53:46.889 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3519999999999999e-06 2024-05-08T08:53:46.889 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.889 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.889 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:46.889 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487989+0000", 2024-05-08T08:53:46.889 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.23e-06 2024-05-08T08:53:46.889 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.890 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.890 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:46.890 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487991+0000", 2024-05-08T08:53:46.890 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6350000000000002e-06 2024-05-08T08:53:46.890 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.890 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.890 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:46.891 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558489+0000", 2024-05-08T08:53:46.891 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070497924000000003 2024-05-08T08:53:46.891 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.891 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.891 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:46.891 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558583+0000", 2024-05-08T08:53:46.891 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3405000000000004e-05 2024-05-08T08:53:46.891 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.892 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.892 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:46.892 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558704+0000", 2024-05-08T08:53:46.892 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012171999999999999 2024-05-08T08:53:46.892 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:46.892 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:46.892 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:46.893 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.893 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.893 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1100 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11558912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:46.893 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:53:46.893 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 70.342413747999998, 2024-05-08T08:53:46.893 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070962266999999996, 2024-05-08T08:53:46.893 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:46.894 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:46.894 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:46.894 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:46.894 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:46.894 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:46.894 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1100 2024-05-08T08:53:46.894 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.895 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:46.895 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.895 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:46.895 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:53:46.895 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:46.895 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.895 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.895 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:46.896 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:53:46.896 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:46.896 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.896 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.896 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:46.896 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488430+0000", 2024-05-08T08:53:46.896 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000432676 2024-05-08T08:53:46.897 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.897 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.897 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:46.897 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488433+0000", 2024-05-08T08:53:46.897 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.45e-06 2024-05-08T08:53:46.897 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.897 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.897 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:46.898 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488442+0000", 2024-05-08T08:53:46.898 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.687e-06 2024-05-08T08:53:46.898 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.898 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.898 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:46.898 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488446+0000", 2024-05-08T08:53:46.899 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4939999999999997e-06 2024-05-08T08:53:46.899 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.899 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.899 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:46.899 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558756+0000", 2024-05-08T08:53:46.899 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070310051999999998 2024-05-08T08:53:46.899 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.899 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.900 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:46.900 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558844+0000", 2024-05-08T08:53:46.900 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7690999999999998e-05 2024-05-08T08:53:46.900 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.900 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.900 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:46.900 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558959+0000", 2024-05-08T08:53:46.901 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000115217 2024-05-08T08:53:46.901 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:46.901 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:46.901 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:46.901 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.901 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.901 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1103 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11599872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:46.901 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:53:46.901 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 70.341956073000006, 2024-05-08T08:53:46.901 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070777100999999995, 2024-05-08T08:53:46.901 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:46.902 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:46.902 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:46.902 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:46.902 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:46.902 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:46.902 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1103 2024-05-08T08:53:46.902 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.902 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:46.902 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.903 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:46.903 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:53:46.903 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:46.903 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.903 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.903 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:46.903 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:53:46.903 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:46.904 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.904 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.904 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:46.904 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488959+0000", 2024-05-08T08:53:46.904 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00050457200000000001 2024-05-08T08:53:46.905 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.905 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.905 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:46.905 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488969+0000", 2024-05-08T08:53:46.905 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.7159999999999998e-06 2024-05-08T08:53:46.905 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.905 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.905 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:46.905 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488981+0000", 2024-05-08T08:53:46.905 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2150000000000001e-05 2024-05-08T08:53:46.905 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.905 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.905 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:46.906 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488988+0000", 2024-05-08T08:53:46.906 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8759999999999999e-06 2024-05-08T08:53:46.906 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.906 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.906 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:46.906 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558992+0000", 2024-05-08T08:53:46.906 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070004358000000003 2024-05-08T08:53:46.906 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.906 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.906 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:46.906 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559093+0000", 2024-05-08T08:53:46.906 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000100299 2024-05-08T08:53:46.906 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.907 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.907 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:46.907 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559232+0000", 2024-05-08T08:53:46.907 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013913 2024-05-08T08:53:46.907 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:46.907 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:46.907 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:46.907 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.907 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.907 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1104 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11640832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:46.907 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:53:46.907 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 70.341411257999994, 2024-05-08T08:53:46.907 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070521947000000001, 2024-05-08T08:53:46.907 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:46.907 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:46.908 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:46.908 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:46.908 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:46.908 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:46.908 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1104 2024-05-08T08:53:46.908 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.908 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:46.908 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.908 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:46.908 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:53:46.908 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:46.908 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.908 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.908 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:46.908 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:53:46.908 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:46.908 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.909 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.909 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:46.909 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489700+0000", 2024-05-08T08:53:46.909 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00070067600000000001 2024-05-08T08:53:46.909 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.909 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.909 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:46.909 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489704+0000", 2024-05-08T08:53:46.909 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1640000000000003e-06 2024-05-08T08:53:46.909 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.909 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.909 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:46.909 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489716+0000", 2024-05-08T08:53:46.909 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1379000000000001e-05 2024-05-08T08:53:46.910 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.910 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.910 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:46.910 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489724+0000", 2024-05-08T08:53:46.910 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8949999999999998e-06 2024-05-08T08:53:46.910 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.910 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.910 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:46.910 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559278+0000", 2024-05-08T08:53:46.910 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.069554151999999994 2024-05-08T08:53:46.910 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.911 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.911 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:46.911 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559387+0000", 2024-05-08T08:53:46.911 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000109258 2024-05-08T08:53:46.911 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.911 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.911 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:46.911 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559521+0000", 2024-05-08T08:53:46.911 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000134423 2024-05-08T08:53:46.911 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:46.911 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:46.911 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:46.911 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.911 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.911 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:8636 3.17 3:e8a0a3e4:::rbd_header.11402b26385:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-05-08T08:53:46.912 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:54.150971+0000", 2024-05-08T08:53:46.912 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 52.679439264000003, 2024-05-08T08:53:46.912 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.217273233, 2024-05-08T08:53:46.912 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:46.912 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:46.912 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:46.912 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:46.912 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:46.912 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:46.912 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 8636 2024-05-08T08:53:46.912 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.912 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:46.912 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.912 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:46.913 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150971+0000", 2024-05-08T08:53:46.913 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:46.913 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.913 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.913 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:46.913 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150971+0000", 2024-05-08T08:53:46.913 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:46.913 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.913 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.913 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:46.913 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150972+0000", 2024-05-08T08:53:46.913 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.71e-07 2024-05-08T08:53:46.913 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.913 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.913 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:46.914 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150977+0000", 2024-05-08T08:53:46.914 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.152e-06 2024-05-08T08:53:46.914 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.914 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.914 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:46.914 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150990+0000", 2024-05-08T08:53:46.914 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2364999999999999e-05 2024-05-08T08:53:46.914 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.914 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.914 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:46.914 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150998+0000", 2024-05-08T08:53:46.915 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0739999999999993e-06 2024-05-08T08:53:46.915 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.915 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.915 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:46.915 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.151054+0000", 2024-05-08T08:53:46.915 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5949000000000003e-05 2024-05-08T08:53:46.915 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.915 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.915 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:53:46.915 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.151060+0000", 2024-05-08T08:53:46.915 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6120000000000004e-06 2024-05-08T08:53:46.915 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.915 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.915 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:46.915 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.367640+0000", 2024-05-08T08:53:46.915 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2165792739999999 2024-05-08T08:53:46.916 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.916 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.916 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:46.916 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.367839+0000", 2024-05-08T08:53:46.916 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000199185 2024-05-08T08:53:46.916 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.916 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.916 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:46.916 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.368245+0000", 2024-05-08T08:53:46.916 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00040575099999999998 2024-05-08T08:53:46.916 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:46.916 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:46.916 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:46.916 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.917 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.917 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5302.0:1516 3.c 3:319bf789:::rbd_data.1140d3fc54c.0000000000000025:head [stat,write 1892352~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:53:46.917 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:27.822220+0000", 2024-05-08T08:53:46.917 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 19.008190327000001, 2024-05-08T08:53:46.917 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.092643744, 2024-05-08T08:53:46.917 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:46.917 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:46.917 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:53:46.917 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:46.917 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5302", 2024-05-08T08:53:46.917 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/359277859", 2024-05-08T08:53:46.917 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1516 2024-05-08T08:53:46.917 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.917 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:46.917 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.918 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:46.918 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822220+0000", 2024-05-08T08:53:46.918 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:46.918 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.918 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.918 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:46.918 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822220+0000", 2024-05-08T08:53:46.918 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:46.918 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.918 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.918 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:46.918 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822221+0000", 2024-05-08T08:53:46.918 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3500000000000002e-07 2024-05-08T08:53:46.919 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.919 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.919 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:46.919 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822228+0000", 2024-05-08T08:53:46.919 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9959999999999996e-06 2024-05-08T08:53:46.919 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.919 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.919 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:46.919 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822234+0000", 2024-05-08T08:53:46.919 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5429999999999999e-06 2024-05-08T08:53:46.919 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.919 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.919 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:46.919 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822238+0000", 2024-05-08T08:53:46.919 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.732e-06 2024-05-08T08:53:46.920 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.920 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.920 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:46.920 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822285+0000", 2024-05-08T08:53:46.920 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6810000000000001e-05 2024-05-08T08:53:46.920 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.920 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.920 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:46.920 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822358+0000", 2024-05-08T08:53:46.920 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3473000000000006e-05 2024-05-08T08:53:46.920 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.920 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.920 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 5", 2024-05-08T08:53:46.920 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822489+0000", 2024-05-08T08:53:46.920 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013011199999999999 2024-05-08T08:53:46.921 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.921 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.921 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:53:46.921 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.825332+0000", 2024-05-08T08:53:46.921 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0028434799999999998 2024-05-08T08:53:46.921 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.921 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.921 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:53:46.921 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.914753+0000", 2024-05-08T08:53:46.921 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089421424999999999 2024-05-08T08:53:46.921 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.921 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.921 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:53:46.921 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.914807+0000", 2024-05-08T08:53:46.921 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3829000000000002e-05 2024-05-08T08:53:46.922 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.922 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.922 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:46.922 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.914864+0000", 2024-05-08T08:53:46.922 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6909e-05 2024-05-08T08:53:46.922 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:46.923 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:46.923 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:46.923 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.923 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.923 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:24567 3.6 3:66af1c40:::rbd_data.1140d3fc54c.000000000000005a:18 [read 2080768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:46.923 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.216598+0000", 2024-05-08T08:53:46.924 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 14.613812247, 2024-05-08T08:53:46.924 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071602647000000005, 2024-05-08T08:53:46.924 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:46.924 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:46.924 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:46.924 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:46.925 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:46.925 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:46.925 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 24567 2024-05-08T08:53:46.925 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.925 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:46.925 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.926 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:46.926 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216598+0000", 2024-05-08T08:53:46.926 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:46.926 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.926 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.926 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:46.926 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216598+0000", 2024-05-08T08:53:46.926 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:46.926 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.926 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.926 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:46.926 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216599+0000", 2024-05-08T08:53:46.927 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8999999999999997e-07 2024-05-08T08:53:46.927 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.927 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.927 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:46.927 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216601+0000", 2024-05-08T08:53:46.927 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9889999999999999e-06 2024-05-08T08:53:46.927 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.927 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.927 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:46.927 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216607+0000", 2024-05-08T08:53:46.927 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3600000000000001e-06 2024-05-08T08:53:46.927 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.927 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.927 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:46.928 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216612+0000", 2024-05-08T08:53:46.928 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.69e-06 2024-05-08T08:53:46.928 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.928 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.928 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:46.928 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.287076+0000", 2024-05-08T08:53:46.928 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070464355000000006 2024-05-08T08:53:46.928 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.928 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.928 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:46.928 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.287151+0000", 2024-05-08T08:53:46.928 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4924000000000003e-05 2024-05-08T08:53:46.928 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:46.928 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:46.928 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:46.928 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.288201+0000", 2024-05-08T08:53:46.928 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0010498389999999999 2024-05-08T08:53:46.929 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:46.929 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:46.929 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:46.929 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:46.929 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:46.929 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:46.930 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:46.831+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:46.931 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:53:46.931 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:46.933+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:46.936 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T08:53:47.030 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:47.033+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:47.038 INFO:teuthology.orchestra.run.smithi196.stdout:checking clone #27, image image_client.0-clone27 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent28 2024-05-08T08:53:47.103 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:47.103 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:53:47.103 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:47.103 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5590.0:294 3.4 3:27afa2bf:::rbd_data.1140cd7686bd.000000000000000a:head [sparse-read 1314816~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-05-08T08:53:47.103 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:47.106237+0000", 2024-05-08T08:53:47.103 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.00016908799999999999, 2024-05-08T08:53:47.103 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00018902, 2024-05-08T08:53:47.104 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:47.104 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:47.104 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:47.104 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:47.104 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5590", 2024-05-08T08:53:47.104 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/1866919379", 2024-05-08T08:53:47.105 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 294 2024-05-08T08:53:47.105 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:47.105 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:47.105 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:47.105 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:47.105 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:47.106237+0000", 2024-05-08T08:53:47.105 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:47.105 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:47.106 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:47.106 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:47.106 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:47.106237+0000", 2024-05-08T08:53:47.106 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:47.106 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:47.106 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:47.106 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:47.107 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:47.106238+0000", 2024-05-08T08:53:47.107 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1100000000000001e-07 2024-05-08T08:53:47.107 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:47.107 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:47.107 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:47.107 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:47.106240+0000", 2024-05-08T08:53:47.107 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3120000000000001e-06 2024-05-08T08:53:47.108 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:47.108 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:47.108 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:47.108 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:47.106245+0000", 2024-05-08T08:53:47.108 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4790000000000004e-06 2024-05-08T08:53:47.108 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:47.108 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:47.109 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:47.109 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:47.106250+0000", 2024-05-08T08:53:47.109 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1760000000000003e-06 2024-05-08T08:53:47.109 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:47.109 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:47.109 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:47.109 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:47.106335+0000", 2024-05-08T08:53:47.109 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.5123999999999993e-05 2024-05-08T08:53:47.110 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:47.110 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:47.110 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:47.110 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:47.106375+0000", 2024-05-08T08:53:47.110 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9839000000000003e-05 2024-05-08T08:53:47.110 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:47.111 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:47.111 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:47.111 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:47.111 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T08:53:47.111 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 1 2024-05-08T08:53:47.111 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:47.116 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:53:47.119 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:53:47.120 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:53:47.131 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:47.133+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:47.142 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:53:47.142 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:53:47.164 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:53:47.164 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:53:47.188 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:53:47.189 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:53:47.210 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:53:47.211 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:53:47.231 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:53:47.232 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:53:47.232 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:47.235+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:47.248 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:47.248 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:47.258 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:53:47.342 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:47.345+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:47.352 INFO:teuthology.orchestra.run.smithi196.stdout:4173 trunc from 0xdb66a00 to 0xba15600 2024-05-08T08:53:47.353 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 5, 4] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-05-08T08:53:47.353 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T08:53:47.353 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-05-08T08:53:47.353 INFO:tasks.thrashosds.thrasher:Setting osd 0 primary_affinity to 0.000000 2024-05-08T08:53:47.353 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 0 2024-05-08T08:53:47.397 INFO:teuthology.orchestra.run.smithi196.stdout:4174 trunc from 0xba15600 to 0x878400 2024-05-08T08:53:47.421 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:47.421 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:47.435 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:47.436+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:47.436 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:53:47.483 INFO:teuthology.orchestra.run.smithi196.stdout:4176 write 0xc283800 thru 0xc290adf (0xd2e0 bytes) 2024-05-08T08:53:47.483 INFO:teuthology.orchestra.run.smithi196.stdout:4182 write 0xb4ace00 thru 0xb4b7edb (0xb0dc bytes) 2024-05-08T08:53:47.483 INFO:teuthology.orchestra.run.smithi196.stdout:4185 read 0x314b200 thru 0x3154833 (0x9634 bytes) 2024-05-08T08:53:47.487 INFO:teuthology.orchestra.run.smithi196.stdout:4189 trunc from 0xc290c00 to 0x8bbb200 2024-05-08T08:53:47.513 INFO:teuthology.orchestra.run.smithi196.stdout:4191 read 0x84a7a00 thru 0x84b0f33 (0x9534 bytes) 2024-05-08T08:53:47.513 INFO:teuthology.orchestra.run.smithi196.stdout:4192 read 0x6e37c00 thru 0x6e3a7c3 (0x2bc4 bytes) 2024-05-08T08:53:47.518 INFO:teuthology.orchestra.run.smithi196.stdout:4194 trunc from 0x8bbb200 to 0xdd85a00 2024-05-08T08:53:47.534 INFO:teuthology.orchestra.run.smithi196.stdout:4195 write 0x3618200 thru 0x3622603 (0xa404 bytes) 2024-05-08T08:53:47.534 INFO:teuthology.orchestra.run.smithi196.stdout:4196 punch from 0x2a1fe00 to 0x2a20200, (0x400 bytes) 2024-05-08T08:53:47.534 INFO:teuthology.orchestra.run.smithi196.stdout:4197 punch from 0xaeade00 to 0xaeaf600, (0x1800 bytes) 2024-05-08T08:53:47.534 INFO:teuthology.orchestra.run.smithi196.stdout:4198 read 0x4e2ba00 thru 0x4e2c7e1 (0xde2 bytes) 2024-05-08T08:53:47.534 INFO:teuthology.orchestra.run.smithi196.stdout:truncating to largest ever: 0xee33600 2024-05-08T08:53:47.535 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:47.537+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:47.535 INFO:teuthology.orchestra.run.smithi196.stdout:4201 trunc from 0xdd85a00 to 0xee33600 2024-05-08T08:53:47.547 INFO:teuthology.orchestra.run.smithi196.stdout:4202 punch from 0x9596a00 to 0x95a3400, (0xca00 bytes) 2024-05-08T08:53:47.550 INFO:teuthology.orchestra.run.smithi196.stdout:4203 read 0xad82e00 thru 0xad8d40a (0xa60b bytes) 2024-05-08T08:53:47.550 INFO:teuthology.orchestra.run.smithi196.stdout:4205 write 0x5033a00 thru 0x5041bff (0xe200 bytes) 2024-05-08T08:53:47.550 INFO:teuthology.orchestra.run.smithi196.stdout:4206 read 0xd962e00 thru 0xd96ace1 (0x7ee2 bytes) 2024-05-08T08:53:47.550 INFO:teuthology.orchestra.run.smithi196.stdout:4207 read 0xbfae000 thru 0xbfbbdea (0xddeb bytes) 2024-05-08T08:53:47.550 INFO:teuthology.orchestra.run.smithi196.stdout:4208 write 0xa46ca00 thru 0xa46dc2d (0x122e bytes) 2024-05-08T08:53:47.550 INFO:teuthology.orchestra.run.smithi196.stdout:4209 punch from 0xcedee00 to 0xcee7e00, (0x9000 bytes) 2024-05-08T08:53:47.551 INFO:teuthology.orchestra.run.smithi196.stdout:4212 write 0x68b5e00 thru 0x68bb0f1 (0x52f2 bytes) 2024-05-08T08:53:47.551 INFO:teuthology.orchestra.run.smithi196.stdout:4213 read 0x6ff5c00 thru 0x6ff5dad (0x1ae bytes) 2024-05-08T08:53:47.551 INFO:teuthology.orchestra.run.smithi196.stdout:4215 punch from 0xa2f800 to 0xa3c200, (0xca00 bytes) 2024-05-08T08:53:47.551 INFO:teuthology.orchestra.run.smithi196.stdout:4216 write 0x36c1400 thru 0x36d0ed4 (0xfad5 bytes) 2024-05-08T08:53:47.553 INFO:teuthology.orchestra.run.smithi196.stdout:4218 read 0xbf5e600 thru 0xbf69bc2 (0xb5c3 bytes) 2024-05-08T08:53:47.567 INFO:teuthology.orchestra.run.smithi196.stdout:4220 trunc from 0xee33600 to 0x3187400 2024-05-08T08:53:47.574 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-05-08T08:53:47.593 INFO:teuthology.orchestra.run.smithi196.stdout:4221 read 0x178d600 thru 0x1790f2c (0x392d bytes) 2024-05-08T08:53:47.600 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:47.600 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:47.610 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:53:47.613 INFO:teuthology.orchestra.run.smithi196.stdout:4223 trunc from 0x3187400 to 0xddd2200 2024-05-08T08:53:47.622 INFO:teuthology.orchestra.run.smithi196.stdout:4225 punch from 0x1558400 to 0x1563400, (0xb000 bytes) 2024-05-08T08:53:47.625 INFO:teuthology.orchestra.run.smithi196.stdout:4230 read 0x33a6800 thru 0x33b3c19 (0xd41a bytes) 2024-05-08T08:53:47.625 INFO:teuthology.orchestra.run.smithi196.stdout:4231 write 0x4276600 thru 0x4280975 (0xa376 bytes) 2024-05-08T08:53:47.625 INFO:teuthology.orchestra.run.smithi196.stdout:4232 read 0x20a3a00 thru 0x20a4681 (0xc82 bytes) 2024-05-08T08:53:47.625 INFO:teuthology.orchestra.run.smithi196.stdout:4233 punch from 0x5394600 to 0x5396400, (0x1e00 bytes) 2024-05-08T08:53:47.626 INFO:teuthology.orchestra.run.smithi196.stdout:4236 read 0x60bc400 thru 0x60c2679 (0x627a bytes) 2024-05-08T08:53:47.626 INFO:teuthology.orchestra.run.smithi196.stdout:4237 write 0x8cdc200 thru 0x8ce257d (0x637e bytes) 2024-05-08T08:53:47.626 INFO:teuthology.orchestra.run.smithi196.stdout:4239 read 0x7566c00 thru 0x75675e5 (0x9e6 bytes) 2024-05-08T08:53:47.626 INFO:teuthology.orchestra.run.smithi196.stdout:4240 punch from 0x93d6c00 to 0x93e0800, (0x9c00 bytes) 2024-05-08T08:53:47.626 INFO:teuthology.orchestra.run.smithi196.stdout:4246 write 0xbbac000 thru 0xbbb990e (0xd90f bytes) 2024-05-08T08:53:47.634 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:47.636+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:47.642 INFO:teuthology.orchestra.run.smithi196.stdout:4247 write 0xed4b200 thru 0xed52cbc (0x7abd bytes) 2024-05-08T08:53:47.642 INFO:teuthology.orchestra.run.smithi196.stdout:4251 punch from 0x17b1800 to 0x17bbe00, (0xa600 bytes) 2024-05-08T08:53:47.644 INFO:teuthology.orchestra.run.smithi196.stdout:4252 write 0xaba7000 thru 0xabb4d76 (0xdd77 bytes) 2024-05-08T08:53:47.644 INFO:teuthology.orchestra.run.smithi196.stdout:4253 read 0xb0a0600 thru 0xb0a65c8 (0x5fc9 bytes) 2024-05-08T08:53:47.644 INFO:teuthology.orchestra.run.smithi196.stdout:4254 read 0x3218e00 thru 0x3228a50 (0xfc51 bytes) 2024-05-08T08:53:47.645 INFO:teuthology.orchestra.run.smithi196.stdout:4255 read 0xcd99000 thru 0xcd9ba4c (0x2a4d bytes) 2024-05-08T08:53:47.645 INFO:teuthology.orchestra.run.smithi196.stdout:4256 read 0xc53a200 thru 0xc53b068 (0xe69 bytes) 2024-05-08T08:53:47.645 INFO:teuthology.orchestra.run.smithi196.stdout:4258 read 0x2354e00 thru 0x235bdc6 (0x6fc7 bytes) 2024-05-08T08:53:47.645 INFO:teuthology.orchestra.run.smithi196.stdout:4260 read 0x8331000 thru 0x83325df (0x15e0 bytes) 2024-05-08T08:53:47.661 INFO:teuthology.orchestra.run.smithi196.stdout:4264 trunc from 0xed52e00 to 0x1485800 2024-05-08T08:53:47.693 INFO:teuthology.orchestra.run.smithi196.stdout:4265 punch from 0x3b3000 to 0x3b8200, (0x5200 bytes) 2024-05-08T08:53:47.696 INFO:teuthology.orchestra.run.smithi196.stdout:4267 punch from 0x1e0200 to 0x1eda00, (0xd800 bytes) 2024-05-08T08:53:47.699 INFO:teuthology.orchestra.run.smithi196.stdout:4270 trunc from 0x1485800 to 0x4648800 2024-05-08T08:53:47.709 INFO:teuthology.orchestra.run.smithi196.stdout:4271 read 0x2dcc400 thru 0x2dd3f24 (0x7b25 bytes) 2024-05-08T08:53:47.711 INFO:teuthology.orchestra.run.smithi196.stdout:4274 trunc from 0x4648800 to 0x49cfc00 2024-05-08T08:53:47.721 INFO:teuthology.orchestra.run.smithi196.stdout:4275 read 0x4fac00 thru 0x50822e (0xd62f bytes) 2024-05-08T08:53:47.722 INFO:teuthology.orchestra.run.smithi196.stdout:4276 write 0x172a200 thru 0x172c6e3 (0x24e4 bytes) 2024-05-08T08:53:47.726 INFO:teuthology.orchestra.run.smithi196.stdout:4277 trunc from 0x49cfc00 to 0x7e0fc00 2024-05-08T08:53:47.736 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:53:47.739+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:53:47.738 INFO:teuthology.orchestra.run.smithi196.stdout:4279 trunc from 0x7e0fc00 to 0x9117c00 2024-05-08T08:53:47.747 INFO:teuthology.orchestra.run.smithi196.stdout:4280 write 0x84ad000 thru 0x84ad155 (0x156 bytes) 2024-05-08T08:53:47.747 INFO:teuthology.orchestra.run.smithi196.stdout:4281 punch from 0x2834e00 to 0x2841a00, (0xcc00 bytes) 2024-05-08T08:53:47.752 INFO:teuthology.orchestra.run.smithi196.stdout:4284 trunc from 0x9117c00 to 0xc01c800 2024-05-08T08:53:47.761 INFO:teuthology.orchestra.run.smithi196.stdout:4285 read 0x46eee00 thru 0x46fe3b8 (0xf5b9 bytes) 2024-05-08T08:53:47.775 INFO:teuthology.orchestra.run.smithi196.stdout:4287 trunc from 0xc01c800 to 0xdaba00 2024-05-08T08:53:47.780 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:47.781 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:47.792 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:53:47.815 INFO:teuthology.orchestra.run.smithi196.stdout:4288 trunc from 0xdaba00 to 0xdea5400 2024-05-08T08:53:47.825 INFO:teuthology.orchestra.run.smithi196.stdout:4289 write 0x6c17e00 thru 0x6c1bcf0 (0x3ef1 bytes) 2024-05-08T08:53:47.826 INFO:teuthology.orchestra.run.smithi196.stdout:4290 read 0x3a38600 thru 0x3a42420 (0x9e21 bytes) 2024-05-08T08:53:47.826 INFO:teuthology.orchestra.run.smithi196.stdout:4291 write 0x4a5a200 thru 0x4a67765 (0xd566 bytes) 2024-05-08T08:53:47.837 INFO:teuthology.orchestra.run.smithi196.stdout:4292 trunc from 0xdea5400 to 0x534a200 2024-05-08T08:53:47.872 INFO:teuthology.orchestra.run.smithi196.stdout:4294 write 0xc5d1200 thru 0xc5d79f3 (0x67f4 bytes) 2024-05-08T08:53:47.876 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:47.879+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:47.877 INFO:teuthology.orchestra.run.smithi196.stdout:4295 punch from 0x9b37000 to 0x9b3dc00, (0x6c00 bytes) 2024-05-08T08:53:47.879 INFO:teuthology.orchestra.run.smithi196.stdout:4297 write 0xb9dae00 thru 0xb9e3309 (0x850a bytes) 2024-05-08T08:53:47.879 INFO:teuthology.orchestra.run.smithi196.stdout:4298 read 0x4263000 thru 0x4271c32 (0xec33 bytes) 2024-05-08T08:53:47.880 INFO:teuthology.orchestra.run.smithi196.stdout:4299 read 0x2250e00 thru 0x2254d09 (0x3f0a bytes) 2024-05-08T08:53:47.881 INFO:teuthology.orchestra.run.smithi196.stdout:4302 read 0xbcb6400 thru 0xbcc3f00 (0xdb01 bytes) 2024-05-08T08:53:47.881 INFO:teuthology.orchestra.run.smithi196.stdout:4303 read 0xac75a00 thru 0xac844ca (0xeacb bytes) 2024-05-08T08:53:47.881 INFO:teuthology.orchestra.run.smithi196.stdout:4304 read 0x7ccb200 thru 0x7cdb042 (0xfe43 bytes) 2024-05-08T08:53:47.881 INFO:teuthology.orchestra.run.smithi196.stdout:4305 write 0xb926a00 thru 0xb92b490 (0x4a91 bytes) 2024-05-08T08:53:47.881 INFO:teuthology.orchestra.run.smithi196.stdout:4306 punch from 0x3470400 to 0x347e600, (0xe200 bytes) 2024-05-08T08:53:47.882 INFO:teuthology.orchestra.run.smithi196.stdout:4308 write 0x6af5400 thru 0x6b0530a (0xff0b bytes) 2024-05-08T08:53:47.882 INFO:teuthology.orchestra.run.smithi196.stdout:4311 write 0x98d2400 thru 0x98e0eaf (0xeab0 bytes) 2024-05-08T08:53:47.882 INFO:teuthology.orchestra.run.smithi196.stdout:4312 write 0x63d5600 thru 0x63dd8ba (0x82bb bytes) 2024-05-08T08:53:47.891 INFO:teuthology.orchestra.run.smithi196.stdout:4313 trunc from 0xc5d7a00 to 0x462de00 2024-05-08T08:53:47.938 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:47.941+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:47.938 INFO:teuthology.orchestra.run.smithi196.stdout:4314 write 0xa5dea00 thru 0xa5e15dc (0x2bdd bytes) 2024-05-08T08:53:47.939 INFO:teuthology.orchestra.run.smithi196.stdout:4316 punch from 0x3382400 to 0x3392000, (0xfc00 bytes) 2024-05-08T08:53:47.940 INFO:teuthology.orchestra.run.smithi196.stdout:4317 read 0x413d200 thru 0x4148701 (0xb502 bytes) 2024-05-08T08:53:47.952 INFO:teuthology.orchestra.run.smithi196.stdout:4319 trunc from 0xa5e1600 to 0x6bc000 2024-05-08T08:53:47.953 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:47.953 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:47.963 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:53:47.985 INFO:teuthology.orchestra.run.smithi196.stdout:4322 punch from 0x2c6c00 to 0x2cba00, (0x4e00 bytes) 2024-05-08T08:53:48.010 INFO:teuthology.orchestra.run.smithi196.stdout:4323 write 0xb8ac600 thru 0xb8b4d11 (0x8712 bytes) 2024-05-08T08:53:48.029 INFO:teuthology.orchestra.run.smithi196.stdout:4324 write 0xda77c00 thru 0xda7dcb0 (0x60b1 bytes) 2024-05-08T08:53:48.033 INFO:teuthology.orchestra.run.smithi196.stdout:4325 trunc from 0xda7de00 to 0xb281800 2024-05-08T08:53:48.039 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:48.042+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:48.054 INFO:teuthology.orchestra.run.smithi196.stdout:4326 read 0xb211c00 thru 0xb21a36d (0x876e bytes) 2024-05-08T08:53:48.056 INFO:teuthology.orchestra.run.smithi196.stdout:4328 write 0x784a000 thru 0x7852492 (0x8493 bytes) 2024-05-08T08:53:48.056 INFO:teuthology.orchestra.run.smithi196.stdout:4330 punch from 0x1158800 to 0x115ea00, (0x6200 bytes) 2024-05-08T08:53:48.057 INFO:teuthology.orchestra.run.smithi196.stdout:4331 read 0x3974c00 thru 0x3977726 (0x2b27 bytes) 2024-05-08T08:53:48.058 INFO:teuthology.orchestra.run.smithi196.stdout:4332 punch from 0xaa34a00 to 0xaa39600, (0x4c00 bytes) 2024-05-08T08:53:48.058 INFO:teuthology.orchestra.run.smithi196.stdout:4333 write 0x6acb400 thru 0x6acb48c (0x8d bytes) 2024-05-08T08:53:48.058 INFO:teuthology.orchestra.run.smithi196.stdout:4335 punch from 0x6c38400 to 0x6c42000, (0x9c00 bytes) 2024-05-08T08:53:48.060 INFO:teuthology.orchestra.run.smithi196.stdout:4336 punch from 0x84a4a00 to 0x84b0200, (0xb800 bytes) 2024-05-08T08:53:48.075 INFO:teuthology.orchestra.run.smithi196.stdout:4338 write 0xd161c00 thru 0xd16312d (0x152e bytes) 2024-05-08T08:53:48.076 INFO:teuthology.orchestra.run.smithi196.stdout:4340 read 0x75e0000 thru 0x75e1a2a (0x1a2b bytes) 2024-05-08T08:53:48.077 INFO:teuthology.orchestra.run.smithi196.stdout:4341 read 0x776cc00 thru 0x7771021 (0x4422 bytes) 2024-05-08T08:53:48.081 INFO:teuthology.orchestra.run.smithi196.stdout:4345 punch from 0x4b36400 to 0x4b3c200, (0x5e00 bytes) 2024-05-08T08:53:48.081 INFO:teuthology.orchestra.run.smithi196.stdout:4346 write 0x129e800 thru 0x12ab529 (0xcd2a bytes) 2024-05-08T08:53:48.081 INFO:teuthology.orchestra.run.smithi196.stdout:4348 read 0xb3da00 thru 0xb49e73 (0xc474 bytes) 2024-05-08T08:53:48.081 INFO:teuthology.orchestra.run.smithi196.stdout:4349 write 0x8222800 thru 0x8231855 (0xf056 bytes) 2024-05-08T08:53:48.083 INFO:teuthology.orchestra.run.smithi196.stdout:4352 trunc from 0xd163200 to 0x97bea00 2024-05-08T08:53:48.088 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:48.088 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:48.100 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:53:48.113 INFO:teuthology.orchestra.run.smithi196.stdout:4354 trunc from 0x97bea00 to 0x77dc600 2024-05-08T08:53:48.137 INFO:teuthology.orchestra.run.smithi196.stdout:4356 punch from 0x3e9de00 to 0x3ea2c00, (0x4e00 bytes) 2024-05-08T08:53:48.142 INFO:teuthology.orchestra.run.smithi196.stdout:4358 read 0x507d400 thru 0x5082d89 (0x598a bytes) 2024-05-08T08:53:48.142 INFO:teuthology.orchestra.run.smithi196.stdout:4360 read 0x6d1fe00 thru 0x6d26a67 (0x6c68 bytes) 2024-05-08T08:53:48.142 INFO:teuthology.orchestra.run.smithi196.stdout:4362 punch from 0x413a000 to 0x4145600, (0xb600 bytes) 2024-05-08T08:53:48.143 INFO:teuthology.orchestra.run.smithi196.stdout:4363 read 0xcce00 thru 0xd6052 (0x9253 bytes) 2024-05-08T08:53:48.143 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:48.146+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:48.143 INFO:teuthology.orchestra.run.smithi196.stdout:4364 punch from 0x2a05c00 to 0x2a06800, (0xc00 bytes) 2024-05-08T08:53:48.143 INFO:teuthology.orchestra.run.smithi196.stdout:4369 read 0x5bef800 thru 0x5bf5c3c (0x643d bytes) 2024-05-08T08:53:48.149 INFO:teuthology.orchestra.run.smithi196.stdout:4372 trunc from 0x77dc600 to 0x2548800 2024-05-08T08:53:48.201 INFO:teuthology.orchestra.run.smithi196.stdout:4373 write 0x896bc00 thru 0x896e359 (0x275a bytes) 2024-05-08T08:53:48.216 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:48.216 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:48.225 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:53:48.227 INFO:teuthology.orchestra.run.smithi196.stdout:4374 write 0xda3ae00 thru 0xda47d21 (0xcf22 bytes) 2024-05-08T08:53:48.227 INFO:teuthology.orchestra.run.smithi196.stdout:4375 write 0x9281c00 thru 0x9284719 (0x2b1a bytes) 2024-05-08T08:53:48.227 INFO:teuthology.orchestra.run.smithi196.stdout:4376 write 0xd346e00 thru 0xd34e7a8 (0x79a9 bytes) 2024-05-08T08:53:48.227 INFO:teuthology.orchestra.run.smithi196.stdout:4378 trunc from 0xda47e00 to 0xdd7c200 2024-05-08T08:53:48.249 INFO:teuthology.orchestra.run.smithi196.stdout:4379 read 0x25e0c00 thru 0x25e3da7 (0x31a8 bytes) 2024-05-08T08:53:48.249 INFO:teuthology.orchestra.run.smithi196.stdout:4380 read 0x3deea00 thru 0x3def2b8 (0x8b9 bytes) 2024-05-08T08:53:48.257 INFO:teuthology.orchestra.run.smithi196.stdout:4382 write 0xe75a200 thru 0xe767dd5 (0xdbd6 bytes) 2024-05-08T08:53:48.258 INFO:teuthology.orchestra.run.smithi196.stdout:4384 punch from 0x4726200 to 0x4729800, (0x3600 bytes) 2024-05-08T08:53:48.266 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:48.269+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:48.276 INFO:teuthology.orchestra.run.smithi196.stdout:4385 trunc from 0xe767e00 to 0x507400 2024-05-08T08:53:48.297 INFO:teuthology.orchestra.run.smithi196.stdout:4387 punch from 0x2b0000 to 0x2b8e00, (0x8e00 bytes) 2024-05-08T08:53:48.316 INFO:teuthology.orchestra.run.smithi196.stdout:4389 write 0x1019a00 thru 0x10209ca (0x6fcb bytes) 2024-05-08T08:53:48.339 INFO:teuthology.orchestra.run.smithi196.stdout:4390 write 0x599bc00 thru 0x599e2eb (0x26ec bytes) 2024-05-08T08:53:48.339 INFO:teuthology.orchestra.run.smithi196.stdout:4392 read 0x3ee1400 thru 0x3ee19b7 (0x5b8 bytes) 2024-05-08T08:53:48.342 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:48.345+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:48.366 INFO:teuthology.orchestra.run.smithi196.stdout:4393 write 0xc3f9e00 thru 0xc3ff3da (0x55db bytes) 2024-05-08T08:53:48.366 INFO:teuthology.orchestra.run.smithi196.stdout:4394 read 0xb969200 thru 0xb96b11d (0x1f1e bytes) 2024-05-08T08:53:48.366 INFO:teuthology.orchestra.run.smithi196.stdout:4397 punch from 0x207ce00 to 0x2086200, (0x9400 bytes) 2024-05-08T08:53:48.366 INFO:teuthology.orchestra.run.smithi196.stdout:4398 punch from 0x4e40200 to 0x4e42000, (0x1e00 bytes) 2024-05-08T08:53:48.366 INFO:teuthology.orchestra.run.smithi196.stdout:4399 write 0xba83800 thru 0xba85c35 (0x2436 bytes) 2024-05-08T08:53:48.366 INFO:teuthology.orchestra.run.smithi196.stdout:4400 clone 30 order 23 su 65536 sc 7 2024-05-08T08:53:48.385 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:48.385 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:48.394 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:53:48.443 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:53:48.447+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:53:48.541 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:48.541 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:48.544 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:48.548+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:48.549 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:53:48.570 INFO:teuthology.orchestra.run.smithi059.stderr:noscrub is set 2024-05-08T08:53:48.573 INFO:teuthology.orchestra.run.smithi059.stderr:set osd.0 primary-affinity to 0 (802) 2024-05-08T08:53:48.645 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:48.648+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:48.658 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:48.658 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:48.667 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:53:48.746 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:48.750+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:48.772 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:48.772 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:48.781 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:53:48.847 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:48.850+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:48.887 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:48.887 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:48.896 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:53:48.948 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:48.952+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:49.041 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:49.041 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:49.048 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:49.052+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:49.049 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:53:49.149 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:49.153+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:49.153 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:49.153 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:49.162 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:53:49.245 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T08:53:49.250 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:49.253+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:49.286 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:49.286 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:49.296 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:53:49.376 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:53:49.377 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:53:49.377 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:49.379+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:49.394 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:53:49.395 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:53:49.411 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:53:49.412 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:53:49.430 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:53:49.430 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:53:49.452 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:49.455+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:49.465 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:53:49.465 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:53:49.465 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:49.466 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:53:49.466 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:53:49.466 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:53:49.466 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:49.475 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:53:49.484 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:53:49.485 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:53:49.553 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:49.556+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:49.599 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:49.599 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:53:49.599 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:53:49.599 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:53:49.599 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.600 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4621.0:9089 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 23339520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:53:49.600 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:53:49.600 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 77.108396438, 2024-05-08T08:53:49.600 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.13814907200000001, 2024-05-08T08:53:49.600 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:49.600 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:49.600 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:53:49.600 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:49.600 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4621", 2024-05-08T08:53:49.600 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:49.600 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9089 2024-05-08T08:53:49.600 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.600 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:49.600 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.600 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:49.601 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:53:49.601 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:49.601 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.601 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.601 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:49.601 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:53:49.601 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:49.601 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.601 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.601 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:49.601 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493005+0000", 2024-05-08T08:53:49.601 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1700000000000001e-06 2024-05-08T08:53:49.601 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.601 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.602 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:49.602 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.513963+0000", 2024-05-08T08:53:49.602 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.020957581999999999 2024-05-08T08:53:49.602 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.602 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.602 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:49.602 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.513995+0000", 2024-05-08T08:53:49.602 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1993999999999999e-05 2024-05-08T08:53:49.602 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.602 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.602 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:49.602 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514007+0000", 2024-05-08T08:53:49.602 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1581e-05 2024-05-08T08:53:49.603 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.603 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.603 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:49.603 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514122+0000", 2024-05-08T08:53:49.603 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000115282 2024-05-08T08:53:49.603 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.603 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.603 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op must wait for map", 2024-05-08T08:53:49.603 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514128+0000", 2024-05-08T08:53:49.603 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9950000000000002e-06 2024-05-08T08:53:49.603 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.603 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.603 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:49.603 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.515800+0000", 2024-05-08T08:53:49.603 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001671886 2024-05-08T08:53:49.604 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.604 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.604 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:49.604 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.515893+0000", 2024-05-08T08:53:49.604 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3490000000000001e-05 2024-05-08T08:53:49.604 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.604 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.604 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 3", 2024-05-08T08:53:49.604 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.517647+0000", 2024-05-08T08:53:49.604 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0017535630000000001 2024-05-08T08:53:49.604 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.604 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.604 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:53:49.604 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.627578+0000", 2024-05-08T08:53:49.604 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.109931557 2024-05-08T08:53:49.605 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.605 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.605 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:53:49.605 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631055+0000", 2024-05-08T08:53:49.605 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0034761420000000002 2024-05-08T08:53:49.605 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.605 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.605 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:53:49.605 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631096+0000", 2024-05-08T08:53:49.605 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1059000000000001e-05 2024-05-08T08:53:49.605 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.605 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.605 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:49.605 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631151+0000", 2024-05-08T08:53:49.605 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5770999999999998e-05 2024-05-08T08:53:49.605 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:49.606 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:49.606 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:49.606 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.606 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.606 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1071 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11026432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:49.606 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:53:49.606 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 73.115946946999998, 2024-05-08T08:53:49.606 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070580037999999998, 2024-05-08T08:53:49.606 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:49.606 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:49.606 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:49.606 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:49.606 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:49.606 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:49.607 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1071 2024-05-08T08:53:49.607 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.607 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:49.607 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.607 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:49.607 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:53:49.607 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:49.607 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.607 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.607 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:49.607 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:53:49.607 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:49.607 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.607 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.607 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:49.607 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:53:49.608 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2800000000000002e-07 2024-05-08T08:53:49.608 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.608 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.608 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:49.608 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485454+0000", 2024-05-08T08:53:49.608 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2290000000000002e-06 2024-05-08T08:53:49.608 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.608 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.608 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:49.608 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485461+0000", 2024-05-08T08:53:49.608 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9920000000000001e-06 2024-05-08T08:53:49.608 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.608 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.608 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:49.608 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485467+0000", 2024-05-08T08:53:49.609 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3979999999999998e-06 2024-05-08T08:53:49.609 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.609 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.609 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:49.609 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555880+0000", 2024-05-08T08:53:49.609 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070413165 2024-05-08T08:53:49.609 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.609 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.609 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:49.609 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555926+0000", 2024-05-08T08:53:49.609 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6217000000000002e-05 2024-05-08T08:53:49.609 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.609 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.609 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:49.609 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556032+0000", 2024-05-08T08:53:49.610 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000105609 2024-05-08T08:53:49.610 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:49.610 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:49.610 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:49.610 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.610 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.610 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1073 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11067392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:49.610 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:53:49.610 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 73.115925726, 2024-05-08T08:53:49.610 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070821404000000004, 2024-05-08T08:53:49.610 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:49.610 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:49.610 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:49.610 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:49.611 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:49.611 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:49.611 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1073 2024-05-08T08:53:49.611 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.611 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:49.611 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.611 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:49.611 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:53:49.611 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:49.611 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.611 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.611 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:49.611 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:53:49.611 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:49.611 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.612 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.612 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:49.612 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:53:49.612 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.08e-07 2024-05-08T08:53:49.612 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.612 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.612 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:49.612 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485476+0000", 2024-05-08T08:53:49.612 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3929999999999998e-06 2024-05-08T08:53:49.612 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.612 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.612 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:49.612 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485485+0000", 2024-05-08T08:53:49.612 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1880000000000008e-06 2024-05-08T08:53:49.613 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.613 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.613 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:49.613 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485494+0000", 2024-05-08T08:53:49.613 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3109999999999995e-06 2024-05-08T08:53:49.613 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.613 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.613 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:49.613 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556082+0000", 2024-05-08T08:53:49.613 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070587426999999994 2024-05-08T08:53:49.613 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.613 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.613 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:49.613 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556177+0000", 2024-05-08T08:53:49.613 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5051999999999997e-05 2024-05-08T08:53:49.614 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.614 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.614 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:49.614 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556294+0000", 2024-05-08T08:53:49.614 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000117625 2024-05-08T08:53:49.614 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:49.614 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:49.614 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:49.614 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.614 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.614 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1075 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11108352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:49.614 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:53:49.614 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 73.115895148999996, 2024-05-08T08:53:49.614 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071045026999999997, 2024-05-08T08:53:49.615 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:49.615 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:49.615 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:49.615 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:49.615 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:49.615 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:49.615 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1075 2024-05-08T08:53:49.615 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.615 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:49.615 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.615 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:49.615 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:53:49.615 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:49.615 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.615 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.616 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:49.616 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:53:49.616 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:49.616 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.616 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.616 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:49.616 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:53:49.616 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.32e-07 2024-05-08T08:53:49.616 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.616 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.616 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:49.616 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485518+0000", 2024-05-08T08:53:49.616 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3769e-05 2024-05-08T08:53:49.616 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.616 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.617 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:49.617 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485527+0000", 2024-05-08T08:53:49.617 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6779999999999995e-06 2024-05-08T08:53:49.617 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.617 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.617 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:49.617 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485532+0000", 2024-05-08T08:53:49.617 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9830000000000004e-06 2024-05-08T08:53:49.617 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.617 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.617 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:49.617 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556340+0000", 2024-05-08T08:53:49.617 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070807743000000006 2024-05-08T08:53:49.617 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.617 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.618 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:49.618 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556435+0000", 2024-05-08T08:53:49.618 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4732e-05 2024-05-08T08:53:49.618 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.618 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.618 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:49.618 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556549+0000", 2024-05-08T08:53:49.618 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011369 2024-05-08T08:53:49.618 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:49.618 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:49.618 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:49.618 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.618 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.618 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1078 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11149312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:49.619 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:53:49.619 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 73.115854949999999, 2024-05-08T08:53:49.619 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071167276000000002, 2024-05-08T08:53:49.619 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:49.619 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:49.619 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:49.619 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:49.619 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:49.619 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:49.619 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1078 2024-05-08T08:53:49.619 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.619 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:49.619 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.619 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:49.620 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:53:49.620 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:49.620 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.620 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.620 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:49.620 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:53:49.620 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:49.620 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.620 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.620 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:49.620 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:53:49.620 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5600000000000001e-07 2024-05-08T08:53:49.620 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.620 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.620 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:49.621 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485547+0000", 2024-05-08T08:53:49.621 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3590000000000002e-06 2024-05-08T08:53:49.621 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.621 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.621 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:49.621 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485553+0000", 2024-05-08T08:53:49.621 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2770000000000002e-06 2024-05-08T08:53:49.621 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.621 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.621 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:49.621 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485558+0000", 2024-05-08T08:53:49.621 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.95e-06 2024-05-08T08:53:49.621 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.621 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.621 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:49.622 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556582+0000", 2024-05-08T08:53:49.622 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071024444000000006 2024-05-08T08:53:49.622 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.622 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.622 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:49.622 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556637+0000", 2024-05-08T08:53:49.622 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4672000000000001e-05 2024-05-08T08:53:49.622 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.622 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.622 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:49.622 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556711+0000", 2024-05-08T08:53:49.622 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4117999999999997e-05 2024-05-08T08:53:49.622 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:49.622 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:49.623 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:49.623 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.623 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.623 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1080 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11190272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:49.623 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:53:49.623 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 73.115834217, 2024-05-08T08:53:49.623 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071350721000000006, 2024-05-08T08:53:49.623 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:49.623 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:49.623 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:49.623 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:49.623 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:49.623 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:49.623 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1080 2024-05-08T08:53:49.625 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.625 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:49.625 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.625 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:49.625 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:53:49.625 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:49.625 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.625 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.626 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:49.626 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:53:49.626 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:49.626 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.626 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.626 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:49.626 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:53:49.626 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.03e-07 2024-05-08T08:53:49.626 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.626 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.626 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:49.626 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485567+0000", 2024-05-08T08:53:49.626 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1090000000000001e-06 2024-05-08T08:53:49.626 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.627 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.627 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:49.627 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485573+0000", 2024-05-08T08:53:49.627 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3049999999999998e-06 2024-05-08T08:53:49.627 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.627 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.627 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:49.627 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485578+0000", 2024-05-08T08:53:49.627 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0130000000000001e-06 2024-05-08T08:53:49.627 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.627 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.627 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:49.627 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556743+0000", 2024-05-08T08:53:49.627 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071164449000000005 2024-05-08T08:53:49.627 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.628 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.628 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:49.628 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556797+0000", 2024-05-08T08:53:49.628 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4589999999999997e-05 2024-05-08T08:53:49.628 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.628 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.628 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:49.628 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556915+0000", 2024-05-08T08:53:49.628 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000117852 2024-05-08T08:53:49.628 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:49.628 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:49.628 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:49.628 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.628 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.628 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1082 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11231232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:49.629 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:53:49.629 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 73.115813974000005, 2024-05-08T08:53:49.629 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071591396000000002, 2024-05-08T08:53:49.629 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:49.629 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:49.629 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:49.629 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:49.629 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:49.629 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:49.629 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1082 2024-05-08T08:53:49.629 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.629 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:49.629 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.629 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:49.629 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:53:49.629 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:49.630 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.630 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.630 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:49.630 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:53:49.630 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:49.630 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.630 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.630 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:49.630 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:53:49.630 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1800000000000001e-07 2024-05-08T08:53:49.630 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.630 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.630 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:49.630 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485587+0000", 2024-05-08T08:53:49.631 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2589999999999999e-06 2024-05-08T08:53:49.631 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.631 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.631 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:49.631 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485594+0000", 2024-05-08T08:53:49.631 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0789999999999997e-06 2024-05-08T08:53:49.631 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.631 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.631 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:49.631 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485598+0000", 2024-05-08T08:53:49.631 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7480000000000001e-06 2024-05-08T08:53:49.631 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.631 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.631 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:49.631 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556971+0000", 2024-05-08T08:53:49.632 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071372774999999999 2024-05-08T08:53:49.632 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.632 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.632 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:49.632 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557069+0000", 2024-05-08T08:53:49.632 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.7732999999999994e-05 2024-05-08T08:53:49.632 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.632 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.632 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:49.632 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557176+0000", 2024-05-08T08:53:49.632 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000107384 2024-05-08T08:53:49.632 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:49.632 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:49.632 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:49.632 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.633 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.633 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1084 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11272192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:49.633 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:53:49.633 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 73.115775334999995, 2024-05-08T08:53:49.633 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071808977999999996, 2024-05-08T08:53:49.633 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:49.633 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:49.633 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:49.633 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:49.633 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:49.633 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:49.633 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1084 2024-05-08T08:53:49.633 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.633 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:49.633 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.633 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:49.634 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:53:49.634 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:49.634 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.634 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.634 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:49.634 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:53:49.634 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:49.634 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.634 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.634 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:49.634 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485624+0000", 2024-05-08T08:53:49.634 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:53:49.634 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.634 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.635 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:49.635 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485626+0000", 2024-05-08T08:53:49.635 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9460000000000001e-06 2024-05-08T08:53:49.635 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.635 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.635 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:49.635 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485633+0000", 2024-05-08T08:53:49.635 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1149999999999996e-06 2024-05-08T08:53:49.635 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.635 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.635 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:49.635 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485638+0000", 2024-05-08T08:53:49.635 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5670000000000002e-06 2024-05-08T08:53:49.635 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.635 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.635 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:49.636 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557226+0000", 2024-05-08T08:53:49.636 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071587267999999996 2024-05-08T08:53:49.636 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.636 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.636 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:49.636 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557321+0000", 2024-05-08T08:53:49.636 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5785000000000005e-05 2024-05-08T08:53:49.636 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.636 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.636 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:49.636 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557432+0000", 2024-05-08T08:53:49.636 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011092800000000001 2024-05-08T08:53:49.636 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:49.636 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:49.636 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:49.636 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.637 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.637 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1086 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11313152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:49.637 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:53:49.637 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 73.115752008000001, 2024-05-08T08:53:49.637 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071933132999999996, 2024-05-08T08:53:49.637 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:49.637 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:49.637 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:49.637 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:49.637 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:49.637 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:49.637 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1086 2024-05-08T08:53:49.637 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.637 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:49.637 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.638 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:49.638 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:53:49.638 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:49.638 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.638 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.638 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:49.638 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:53:49.638 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:49.638 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.638 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.638 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:49.638 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486328+0000", 2024-05-08T08:53:49.638 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00068107900000000002 2024-05-08T08:53:49.638 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.638 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.639 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:49.639 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486331+0000", 2024-05-08T08:53:49.639 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.585e-06 2024-05-08T08:53:49.639 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.639 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.639 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:49.639 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486343+0000", 2024-05-08T08:53:49.639 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1856e-05 2024-05-08T08:53:49.639 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.639 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.639 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:49.639 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486350+0000", 2024-05-08T08:53:49.639 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8600000000000004e-06 2024-05-08T08:53:49.639 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.639 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.640 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:49.640 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557470+0000", 2024-05-08T08:53:49.640 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071120326999999997 2024-05-08T08:53:49.640 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.640 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.640 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:49.640 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557514+0000", 2024-05-08T08:53:49.640 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3909000000000003e-05 2024-05-08T08:53:49.640 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.640 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.640 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:49.640 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557580+0000", 2024-05-08T08:53:49.640 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5517000000000003e-05 2024-05-08T08:53:49.640 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:49.640 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:49.640 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:49.641 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.641 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.641 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1089 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11354112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:49.641 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:53:49.641 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 73.115039135999993, 2024-05-08T08:53:49.641 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071457381, 2024-05-08T08:53:49.641 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:49.641 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:49.641 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:49.641 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:49.641 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:49.641 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:49.641 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1089 2024-05-08T08:53:49.641 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.641 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:49.642 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.642 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:49.642 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:53:49.642 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:49.642 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.642 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.642 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:49.642 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:53:49.642 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:49.642 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.642 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.642 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:49.642 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486851+0000", 2024-05-08T08:53:49.642 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00049102 2024-05-08T08:53:49.643 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.643 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.643 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:49.643 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486854+0000", 2024-05-08T08:53:49.643 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6859999999999999e-06 2024-05-08T08:53:49.643 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.643 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.643 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:49.643 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486864+0000", 2024-05-08T08:53:49.643 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6080000000000002e-06 2024-05-08T08:53:49.643 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.643 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.643 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:49.643 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486870+0000", 2024-05-08T08:53:49.643 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.477e-06 2024-05-08T08:53:49.644 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.644 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.644 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:49.644 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557612+0000", 2024-05-08T08:53:49.644 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070741557999999996 2024-05-08T08:53:49.644 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.644 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.644 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:49.644 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557695+0000", 2024-05-08T08:53:49.644 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3300000000000005e-05 2024-05-08T08:53:49.644 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.644 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.644 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:49.644 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557817+0000", 2024-05-08T08:53:49.645 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012173200000000001 2024-05-08T08:53:49.645 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:49.645 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:49.645 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:49.645 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.645 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.645 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1091 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11395072~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:49.645 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:53:49.645 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 73.114518340999993, 2024-05-08T08:53:49.645 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071184737999999997, 2024-05-08T08:53:49.645 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:49.645 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:49.645 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:49.645 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:49.645 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:49.645 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:49.646 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1091 2024-05-08T08:53:49.646 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.646 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:49.646 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.646 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:49.646 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:53:49.646 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:49.646 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.646 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.646 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:49.646 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:53:49.646 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:49.646 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.646 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.646 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:49.647 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486881+0000", 2024-05-08T08:53:49.647 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0699999999999997e-07 2024-05-08T08:53:49.647 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.647 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.647 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:49.647 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486883+0000", 2024-05-08T08:53:49.647 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.402e-06 2024-05-08T08:53:49.647 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.647 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.647 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:49.647 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486890+0000", 2024-05-08T08:53:49.647 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0480000000000001e-06 2024-05-08T08:53:49.647 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.647 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.647 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:49.648 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486896+0000", 2024-05-08T08:53:49.648 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7640000000000002e-06 2024-05-08T08:53:49.648 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.648 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.648 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:49.648 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557860+0000", 2024-05-08T08:53:49.648 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070963791999999998 2024-05-08T08:53:49.648 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.648 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.648 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:49.648 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557952+0000", 2024-05-08T08:53:49.648 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2082000000000004e-05 2024-05-08T08:53:49.648 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.648 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.648 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:49.648 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558065+0000", 2024-05-08T08:53:49.648 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000113143 2024-05-08T08:53:49.649 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:49.649 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:49.649 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:49.649 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.649 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.649 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1094 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11436032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:49.649 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:53:49.649 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 73.114495172999995, 2024-05-08T08:53:49.649 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071400507000000002, 2024-05-08T08:53:49.649 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:49.649 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:49.649 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:49.649 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:49.649 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:49.649 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:49.649 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1094 2024-05-08T08:53:49.650 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.650 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:49.650 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.650 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:49.650 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:53:49.650 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:49.650 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.650 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.650 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:49.650 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:53:49.650 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:49.650 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.650 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.650 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:49.650 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487940+0000", 2024-05-08T08:53:49.650 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001036868 2024-05-08T08:53:49.651 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.651 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.651 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:49.651 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487944+0000", 2024-05-08T08:53:49.651 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7069999999999998e-06 2024-05-08T08:53:49.651 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.651 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.651 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:49.651 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487951+0000", 2024-05-08T08:53:49.651 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2649999999999999e-06 2024-05-08T08:53:49.651 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.651 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.651 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:49.651 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487961+0000", 2024-05-08T08:53:49.651 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4110000000000002e-06 2024-05-08T08:53:49.651 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.651 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.652 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:49.652 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558117+0000", 2024-05-08T08:53:49.652 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070156662999999994 2024-05-08T08:53:49.652 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.652 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.652 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:49.652 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558214+0000", 2024-05-08T08:53:49.652 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6444e-05 2024-05-08T08:53:49.652 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.652 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.652 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:49.652 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558304+0000", 2024-05-08T08:53:49.652 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0148999999999994e-05 2024-05-08T08:53:49.652 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:49.652 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:49.653 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:49.653 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.653 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.653 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1096 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11476992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:49.653 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:53:49.653 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 73.113428642000002, 2024-05-08T08:53:49.653 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070480074000000004, 2024-05-08T08:53:49.653 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:49.653 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:49.653 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:49.653 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:49.653 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:49.653 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:49.653 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1096 2024-05-08T08:53:49.653 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.653 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:49.653 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.654 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:49.654 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:53:49.654 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:49.654 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.654 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.654 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:49.654 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:53:49.654 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:49.654 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.654 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.654 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:49.654 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:53:49.655 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3700000000000001e-07 2024-05-08T08:53:49.655 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.655 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.655 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:49.655 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487973+0000", 2024-05-08T08:53:49.655 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0870000000000001e-06 2024-05-08T08:53:49.655 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.655 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.655 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:49.655 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487977+0000", 2024-05-08T08:53:49.655 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0600000000000001e-06 2024-05-08T08:53:49.655 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.655 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.655 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:49.655 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487980+0000", 2024-05-08T08:53:49.656 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1089999999999998e-06 2024-05-08T08:53:49.656 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.656 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.656 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:49.656 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558338+0000", 2024-05-08T08:53:49.656 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070358233000000006 2024-05-08T08:53:49.656 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.656 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.656 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:49.656 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558392+0000", 2024-05-08T08:53:49.656 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4537999999999999e-05 2024-05-08T08:53:49.656 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.656 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.656 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:49.656 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558450+0000", 2024-05-08T08:53:49.656 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7710000000000001e-05 2024-05-08T08:53:49.656 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:49.657 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:49.657 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:49.657 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.657 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.657 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1099 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11517952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:49.657 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:53:49.657 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 73.113414766000005, 2024-05-08T08:53:49.657 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070720506000000002, 2024-05-08T08:53:49.657 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:49.657 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:49.657 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:49.658 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:49.658 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:49.658 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:49.658 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1099 2024-05-08T08:53:49.658 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.658 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:49.658 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.658 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:49.658 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:53:49.658 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:49.658 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.658 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.658 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:49.658 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:53:49.658 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:49.659 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.659 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.659 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:49.659 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:53:49.659 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3999999999999998e-07 2024-05-08T08:53:49.659 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.659 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.659 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:49.659 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487986+0000", 2024-05-08T08:53:49.659 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3519999999999999e-06 2024-05-08T08:53:49.659 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.659 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.659 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:49.659 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487989+0000", 2024-05-08T08:53:49.659 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.23e-06 2024-05-08T08:53:49.659 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.660 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.660 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:49.660 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487991+0000", 2024-05-08T08:53:49.660 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6350000000000002e-06 2024-05-08T08:53:49.660 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.660 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.660 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:49.660 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558489+0000", 2024-05-08T08:53:49.660 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070497924000000003 2024-05-08T08:53:49.660 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.660 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.660 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:49.660 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558583+0000", 2024-05-08T08:53:49.660 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3405000000000004e-05 2024-05-08T08:53:49.660 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.660 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.661 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:49.661 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558704+0000", 2024-05-08T08:53:49.661 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012171999999999999 2024-05-08T08:53:49.661 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:49.661 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:49.661 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:49.661 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.661 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.661 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1100 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11558912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:49.661 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:53:49.661 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 73.113401766999999, 2024-05-08T08:53:49.661 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070962266999999996, 2024-05-08T08:53:49.661 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:49.661 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:49.661 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:49.661 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:49.662 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:49.662 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:49.662 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1100 2024-05-08T08:53:49.662 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.662 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:49.662 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.662 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:49.662 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:53:49.662 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:49.662 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.662 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.662 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:49.662 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:53:49.662 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:49.662 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.662 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.663 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:49.663 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488430+0000", 2024-05-08T08:53:49.663 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000432676 2024-05-08T08:53:49.663 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.663 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.663 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:49.663 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488433+0000", 2024-05-08T08:53:49.663 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.45e-06 2024-05-08T08:53:49.663 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.663 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.663 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:49.663 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488442+0000", 2024-05-08T08:53:49.663 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.687e-06 2024-05-08T08:53:49.663 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.663 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.663 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:49.664 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488446+0000", 2024-05-08T08:53:49.664 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4939999999999997e-06 2024-05-08T08:53:49.664 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.664 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.664 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:49.664 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558756+0000", 2024-05-08T08:53:49.664 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070310051999999998 2024-05-08T08:53:49.664 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.664 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.664 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:49.664 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558844+0000", 2024-05-08T08:53:49.664 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7690999999999998e-05 2024-05-08T08:53:49.664 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.664 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.664 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:49.664 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558959+0000", 2024-05-08T08:53:49.665 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000115217 2024-05-08T08:53:49.665 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:49.665 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:49.665 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:49.665 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.665 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.665 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1103 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11599872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:49.665 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:53:49.665 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 73.112944092000006, 2024-05-08T08:53:49.665 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070777100999999995, 2024-05-08T08:53:49.665 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:49.665 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:49.665 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:49.665 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:49.665 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:49.666 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:49.666 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1103 2024-05-08T08:53:49.666 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.666 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:49.666 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.666 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:49.666 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:53:49.666 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:49.666 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.666 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.666 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:49.666 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:53:49.666 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:49.666 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.666 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.666 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:49.667 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488959+0000", 2024-05-08T08:53:49.667 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00050457200000000001 2024-05-08T08:53:49.667 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.667 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.667 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:49.667 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488969+0000", 2024-05-08T08:53:49.667 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.7159999999999998e-06 2024-05-08T08:53:49.667 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.667 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.667 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:49.667 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488981+0000", 2024-05-08T08:53:49.667 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2150000000000001e-05 2024-05-08T08:53:49.667 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.667 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.668 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:49.668 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488988+0000", 2024-05-08T08:53:49.668 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8759999999999999e-06 2024-05-08T08:53:49.668 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.668 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.668 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:49.668 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558992+0000", 2024-05-08T08:53:49.668 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070004358000000003 2024-05-08T08:53:49.668 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.668 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.668 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:49.669 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559093+0000", 2024-05-08T08:53:49.669 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000100299 2024-05-08T08:53:49.669 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.669 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.669 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:49.669 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559232+0000", 2024-05-08T08:53:49.669 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013913 2024-05-08T08:53:49.669 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:49.669 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:49.669 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:49.669 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.669 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.669 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1104 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11640832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:49.669 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:53:49.669 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 73.112399276999994, 2024-05-08T08:53:49.669 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070521947000000001, 2024-05-08T08:53:49.670 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:49.670 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:49.670 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:49.670 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:49.670 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:49.670 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:49.670 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1104 2024-05-08T08:53:49.670 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.670 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:49.670 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.670 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:49.670 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:53:49.670 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:49.670 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.670 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.670 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:49.671 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:53:49.671 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:49.671 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.671 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.671 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:49.671 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489700+0000", 2024-05-08T08:53:49.671 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00070067600000000001 2024-05-08T08:53:49.671 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.671 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.671 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:49.671 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489704+0000", 2024-05-08T08:53:49.671 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1640000000000003e-06 2024-05-08T08:53:49.671 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.671 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.671 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:49.671 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489716+0000", 2024-05-08T08:53:49.672 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1379000000000001e-05 2024-05-08T08:53:49.672 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.672 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.672 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:49.672 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489724+0000", 2024-05-08T08:53:49.672 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8949999999999998e-06 2024-05-08T08:53:49.672 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.672 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.672 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:49.672 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559278+0000", 2024-05-08T08:53:49.672 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.069554151999999994 2024-05-08T08:53:49.672 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.672 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.672 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:49.672 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559387+0000", 2024-05-08T08:53:49.672 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000109258 2024-05-08T08:53:49.673 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.673 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.673 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:49.673 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559521+0000", 2024-05-08T08:53:49.673 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000134423 2024-05-08T08:53:49.673 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:49.673 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:49.673 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:49.673 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.673 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.673 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:8636 3.17 3:e8a0a3e4:::rbd_header.11402b26385:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-05-08T08:53:49.673 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:54.150971+0000", 2024-05-08T08:53:49.673 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 55.450427283000003, 2024-05-08T08:53:49.673 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.217273233, 2024-05-08T08:53:49.673 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:49.673 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:49.673 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:49.674 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:49.674 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:49.674 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:49.674 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 8636 2024-05-08T08:53:49.674 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.674 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:49.674 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.674 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:49.674 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150971+0000", 2024-05-08T08:53:49.674 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:49.674 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.674 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.674 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:49.674 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150971+0000", 2024-05-08T08:53:49.674 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:49.675 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.675 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.675 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:49.675 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150972+0000", 2024-05-08T08:53:49.675 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.71e-07 2024-05-08T08:53:49.675 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.675 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.675 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:49.675 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150977+0000", 2024-05-08T08:53:49.675 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.152e-06 2024-05-08T08:53:49.675 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.675 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.675 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:49.675 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150990+0000", 2024-05-08T08:53:49.675 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2364999999999999e-05 2024-05-08T08:53:49.676 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.676 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.676 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:49.676 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150998+0000", 2024-05-08T08:53:49.676 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0739999999999993e-06 2024-05-08T08:53:49.676 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.676 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.676 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:49.676 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.151054+0000", 2024-05-08T08:53:49.676 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5949000000000003e-05 2024-05-08T08:53:49.676 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.676 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.676 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:53:49.677 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.151060+0000", 2024-05-08T08:53:49.677 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6120000000000004e-06 2024-05-08T08:53:49.677 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.677 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.677 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:49.677 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.367640+0000", 2024-05-08T08:53:49.677 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2165792739999999 2024-05-08T08:53:49.677 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.677 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.677 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:49.677 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.367839+0000", 2024-05-08T08:53:49.677 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000199185 2024-05-08T08:53:49.677 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.677 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.678 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:49.678 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.368245+0000", 2024-05-08T08:53:49.678 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00040575099999999998 2024-05-08T08:53:49.678 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:49.678 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:49.678 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:49.678 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.678 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.678 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5302.0:1516 3.c 3:319bf789:::rbd_data.1140d3fc54c.0000000000000025:head [stat,write 1892352~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:53:49.678 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:27.822220+0000", 2024-05-08T08:53:49.678 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 21.779178345999998, 2024-05-08T08:53:49.679 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.092643744, 2024-05-08T08:53:49.679 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:49.679 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:49.679 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:53:49.679 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:49.679 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5302", 2024-05-08T08:53:49.679 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/359277859", 2024-05-08T08:53:49.679 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1516 2024-05-08T08:53:49.679 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.679 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:49.679 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.679 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:49.679 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822220+0000", 2024-05-08T08:53:49.680 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:49.680 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.680 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.680 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:49.680 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822220+0000", 2024-05-08T08:53:49.680 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:49.680 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.680 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.680 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:49.680 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822221+0000", 2024-05-08T08:53:49.680 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3500000000000002e-07 2024-05-08T08:53:49.680 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.680 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.681 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:49.681 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822228+0000", 2024-05-08T08:53:49.681 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9959999999999996e-06 2024-05-08T08:53:49.681 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.681 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.681 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:49.681 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822234+0000", 2024-05-08T08:53:49.681 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5429999999999999e-06 2024-05-08T08:53:49.681 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.681 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.681 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:49.681 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822238+0000", 2024-05-08T08:53:49.681 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.732e-06 2024-05-08T08:53:49.681 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.681 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.681 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:49.682 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822285+0000", 2024-05-08T08:53:49.682 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6810000000000001e-05 2024-05-08T08:53:49.682 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.682 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.682 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:49.682 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822358+0000", 2024-05-08T08:53:49.682 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3473000000000006e-05 2024-05-08T08:53:49.682 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.682 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.682 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 5", 2024-05-08T08:53:49.682 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822489+0000", 2024-05-08T08:53:49.682 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013011199999999999 2024-05-08T08:53:49.682 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.682 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.683 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:53:49.683 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.825332+0000", 2024-05-08T08:53:49.683 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0028434799999999998 2024-05-08T08:53:49.683 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.683 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.683 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:53:49.683 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.914753+0000", 2024-05-08T08:53:49.683 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089421424999999999 2024-05-08T08:53:49.683 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.683 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.683 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:53:49.683 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.914807+0000", 2024-05-08T08:53:49.684 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3829000000000002e-05 2024-05-08T08:53:49.684 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.684 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.684 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:49.684 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.914864+0000", 2024-05-08T08:53:49.684 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6909e-05 2024-05-08T08:53:49.684 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:49.684 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:49.684 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:49.684 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.684 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.684 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:24567 3.6 3:66af1c40:::rbd_data.1140d3fc54c.000000000000005a:18 [read 2080768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:49.685 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.216598+0000", 2024-05-08T08:53:49.685 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 17.384800265999999, 2024-05-08T08:53:49.685 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071602647000000005, 2024-05-08T08:53:49.685 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:49.685 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:49.685 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:49.685 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:49.685 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:49.685 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:49.685 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 24567 2024-05-08T08:53:49.685 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.685 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:49.685 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.685 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:49.686 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216598+0000", 2024-05-08T08:53:49.686 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:49.686 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.686 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.686 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:49.686 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216598+0000", 2024-05-08T08:53:49.686 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:49.686 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.686 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.686 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:49.686 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216599+0000", 2024-05-08T08:53:49.686 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8999999999999997e-07 2024-05-08T08:53:49.686 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.686 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.686 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:49.687 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216601+0000", 2024-05-08T08:53:49.687 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9889999999999999e-06 2024-05-08T08:53:49.687 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.687 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.687 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:49.687 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216607+0000", 2024-05-08T08:53:49.687 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3600000000000001e-06 2024-05-08T08:53:49.687 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.687 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.687 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:49.687 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216612+0000", 2024-05-08T08:53:49.687 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.69e-06 2024-05-08T08:53:49.687 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.687 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.687 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:49.688 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.287076+0000", 2024-05-08T08:53:49.688 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070464355000000006 2024-05-08T08:53:49.688 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.688 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.688 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:49.688 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.287151+0000", 2024-05-08T08:53:49.688 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4924000000000003e-05 2024-05-08T08:53:49.688 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:49.688 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:49.688 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:49.688 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.288201+0000", 2024-05-08T08:53:49.689 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0010498389999999999 2024-05-08T08:53:49.689 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:49.689 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:49.689 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:49.689 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:49.689 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:49.689 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:49.690 INFO:teuthology.orchestra.run.smithi196.stdout:leaving image image_client.0-clone29 intact 2024-05-08T08:53:49.691 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:53:49.691 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:49.693+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:49.791 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:49.793+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:49.802 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:49.803 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:53:49.803 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:53:49.803 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:49.811 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:53:49.891 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:49.894+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:49.923 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:49.923 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:53:49.923 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:53:49.923 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:53:49.923 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:49.933 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:53:49.992 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:53:49.994+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:53:49.993 INFO:teuthology.orchestra.run.smithi196.stdout:checking clone #28, image image_client.0-clone28 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent29 2024-05-08T08:53:50.045 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:50.045 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:53:50.045 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:53:50.045 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:53:50.046 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.046 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:12443 3.0 3:02d0158e:::rbd_data.114071bcf828.0000000000000037:head [stat,delete] snapc 19=[19] RETRY=3 ondisk+retry+write+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:50.046 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.610027+0000", 2024-05-08T08:53:50.046 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 18.437536207000001, 2024-05-08T08:53:50.046 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.083586333999999998, 2024-05-08T08:53:50.046 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:50.046 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:50.046 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:50.046 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:50.046 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:50.046 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:50.046 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 12443 2024-05-08T08:53:50.046 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.046 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:50.047 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.047 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:50.047 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610027+0000", 2024-05-08T08:53:50.047 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:50.047 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.047 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.047 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:50.047 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610027+0000", 2024-05-08T08:53:50.047 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:50.047 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.047 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.047 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:50.047 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610028+0000", 2024-05-08T08:53:50.047 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.13e-07 2024-05-08T08:53:50.047 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.047 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.047 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:50.048 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610030+0000", 2024-05-08T08:53:50.048 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7669999999999999e-06 2024-05-08T08:53:50.048 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.048 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.048 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:50.048 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610039+0000", 2024-05-08T08:53:50.048 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4959999999999996e-06 2024-05-08T08:53:50.048 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.048 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.048 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:50.048 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610045+0000", 2024-05-08T08:53:50.048 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6779999999999997e-06 2024-05-08T08:53:50.048 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.048 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.048 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:50.048 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.660873+0000", 2024-05-08T08:53:50.049 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.050828748 2024-05-08T08:53:50.049 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.049 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.049 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:50.049 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.661017+0000", 2024-05-08T08:53:50.049 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014391999999999999 2024-05-08T08:53:50.049 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.049 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.049 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:50.049 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.693613+0000", 2024-05-08T08:53:50.049 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.032596111999999997 2024-05-08T08:53:50.049 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.049 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:50.049 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.049 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.049 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.049 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:14301 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [watch reconnect cookie 139837155901536 gen 12] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:50.050 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.645078+0000", 2024-05-08T08:53:50.050 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 18.402485569, 2024-05-08T08:53:50.050 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088281852999999993, 2024-05-08T08:53:50.050 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:50.050 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:50.050 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:50.050 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:50.050 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:50.050 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:50.050 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14301 2024-05-08T08:53:50.050 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.050 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:50.050 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.050 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:50.050 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645078+0000", 2024-05-08T08:53:50.051 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:50.051 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.051 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.051 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:50.051 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645078+0000", 2024-05-08T08:53:50.051 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:50.051 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.051 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.051 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:50.051 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645084+0000", 2024-05-08T08:53:50.051 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4620000000000001e-06 2024-05-08T08:53:50.051 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.051 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.051 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:50.051 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645088+0000", 2024-05-08T08:53:50.052 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8039999999999999e-06 2024-05-08T08:53:50.052 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.052 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.052 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:50.052 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645106+0000", 2024-05-08T08:53:50.052 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.8318999999999999e-05 2024-05-08T08:53:50.052 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.052 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.052 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:50.052 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645113+0000", 2024-05-08T08:53:50.052 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0310000000000001e-06 2024-05-08T08:53:50.052 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.052 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.052 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:50.052 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.670493+0000", 2024-05-08T08:53:50.052 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.025379993 2024-05-08T08:53:50.053 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.053 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.053 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:50.053 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.670571+0000", 2024-05-08T08:53:50.053 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7613000000000004e-05 2024-05-08T08:53:50.053 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.053 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.053 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:50.053 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733360+0000", 2024-05-08T08:53:50.053 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.062788630999999998 2024-05-08T08:53:50.053 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.053 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:50.053 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.053 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.053 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.054 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:14420 3.0 3:06a8948e:::rbd_data.114071bcf828.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:50.054 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.649639+0000", 2024-05-08T08:53:50.054 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 18.397924420999999, 2024-05-08T08:53:50.054 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.083851897999999994, 2024-05-08T08:53:50.054 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:50.054 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:50.054 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:53:50.054 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:50.054 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:50.054 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:50.054 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14420 2024-05-08T08:53:50.054 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.054 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:50.054 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.055 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:50.055 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649639+0000", 2024-05-08T08:53:50.055 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:50.055 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.055 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.055 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:50.055 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649639+0000", 2024-05-08T08:53:50.055 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:50.055 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.055 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.055 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:50.055 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649640+0000", 2024-05-08T08:53:50.055 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6899999999999997e-07 2024-05-08T08:53:50.055 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.055 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.055 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:50.056 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649895+0000", 2024-05-08T08:53:50.056 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00025531000000000002 2024-05-08T08:53:50.056 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.056 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.056 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:50.056 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649905+0000", 2024-05-08T08:53:50.056 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0198e-05 2024-05-08T08:53:50.056 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.056 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.056 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:50.056 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649911+0000", 2024-05-08T08:53:50.056 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2310000000000001e-06 2024-05-08T08:53:50.056 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.056 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.056 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:50.056 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.684099+0000", 2024-05-08T08:53:50.056 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.034188017000000001 2024-05-08T08:53:50.057 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.057 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.057 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:50.057 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.684172+0000", 2024-05-08T08:53:50.057 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3201999999999995e-05 2024-05-08T08:53:50.057 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.057 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.057 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 2", 2024-05-08T08:53:50.057 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.684418+0000", 2024-05-08T08:53:50.057 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00024559099999999997 2024-05-08T08:53:50.057 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.057 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.057 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:53:50.057 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.705538+0000", 2024-05-08T08:53:50.057 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.021119707000000001 2024-05-08T08:53:50.058 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.058 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.058 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:53:50.058 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733399+0000", 2024-05-08T08:53:50.058 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.027861543999999999 2024-05-08T08:53:50.058 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.058 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.058 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:53:50.058 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733445+0000", 2024-05-08T08:53:50.058 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5587000000000002e-05 2024-05-08T08:53:50.058 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.058 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.058 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:50.058 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733491+0000", 2024-05-08T08:53:50.059 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5841999999999999e-05 2024-05-08T08:53:50.059 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.059 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:50.059 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.059 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.059 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.059 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28547 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889179216] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:50.059 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.355654+0000", 2024-05-08T08:53:50.059 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 17.691909193000001, 2024-05-08T08:53:50.059 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078692261999999999, 2024-05-08T08:53:50.059 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:50.059 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:50.059 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:50.059 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:50.059 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:50.059 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:50.060 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28547 2024-05-08T08:53:50.060 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.060 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:50.060 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.060 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:50.060 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355654+0000", 2024-05-08T08:53:50.060 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:50.060 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.060 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.060 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:50.060 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355654+0000", 2024-05-08T08:53:50.060 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:50.060 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.060 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.060 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:50.061 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355654+0000", 2024-05-08T08:53:50.061 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3999999999999998e-07 2024-05-08T08:53:50.061 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.061 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.061 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:50.061 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355656+0000", 2024-05-08T08:53:50.061 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.353e-06 2024-05-08T08:53:50.061 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.061 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.061 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:50.061 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355663+0000", 2024-05-08T08:53:50.061 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2259999999999999e-06 2024-05-08T08:53:50.061 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.061 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.062 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:50.062 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355666+0000", 2024-05-08T08:53:50.062 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7930000000000002e-06 2024-05-08T08:53:50.062 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.062 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.062 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:50.062 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434117+0000", 2024-05-08T08:53:50.062 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078451231999999996 2024-05-08T08:53:50.062 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.062 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.062 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:50.062 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434198+0000", 2024-05-08T08:53:50.062 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0879999999999998e-05 2024-05-08T08:53:50.062 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.063 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.063 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:50.063 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434346+0000", 2024-05-08T08:53:50.063 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014853800000000001 2024-05-08T08:53:50.063 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.063 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:50.063 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.063 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.063 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.063 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28548 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889189696] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:50.063 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.355669+0000", 2024-05-08T08:53:50.063 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 17.691893842999999, 2024-05-08T08:53:50.063 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078956606999999998, 2024-05-08T08:53:50.063 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:50.063 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:50.063 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:50.064 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:50.064 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:50.064 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:50.064 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28548 2024-05-08T08:53:50.064 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.064 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:50.064 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.064 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:50.064 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355669+0000", 2024-05-08T08:53:50.064 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:50.064 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.064 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.064 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:50.065 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355669+0000", 2024-05-08T08:53:50.065 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:50.065 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.065 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.065 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:50.065 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355762+0000", 2024-05-08T08:53:50.065 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3015000000000003e-05 2024-05-08T08:53:50.065 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.065 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.065 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:50.065 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355765+0000", 2024-05-08T08:53:50.065 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.368e-06 2024-05-08T08:53:50.065 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.065 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.065 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:50.066 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355771+0000", 2024-05-08T08:53:50.066 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5989999999999998e-06 2024-05-08T08:53:50.066 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.066 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.066 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:50.066 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355775+0000", 2024-05-08T08:53:50.066 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7280000000000001e-06 2024-05-08T08:53:50.066 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.066 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.066 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:50.066 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434379+0000", 2024-05-08T08:53:50.066 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078603662000000005 2024-05-08T08:53:50.066 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.066 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.066 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:50.066 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434473+0000", 2024-05-08T08:53:50.067 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4104000000000006e-05 2024-05-08T08:53:50.067 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.067 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.067 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:50.067 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434626+0000", 2024-05-08T08:53:50.067 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00015313100000000001 2024-05-08T08:53:50.067 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.067 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:50.067 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.067 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.067 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.067 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28549 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889202400] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:50.067 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.355781+0000", 2024-05-08T08:53:50.067 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 17.691782269000001, 2024-05-08T08:53:50.067 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079117946999999994, 2024-05-08T08:53:50.067 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:50.068 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:50.068 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:50.068 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:50.068 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:50.068 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:50.068 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28549 2024-05-08T08:53:50.068 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.068 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:50.068 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.068 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:50.068 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355781+0000", 2024-05-08T08:53:50.068 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:50.068 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.068 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.068 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:50.068 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355781+0000", 2024-05-08T08:53:50.068 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:50.069 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.069 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.069 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:50.069 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356162+0000", 2024-05-08T08:53:50.069 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00038099100000000001 2024-05-08T08:53:50.069 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.069 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.069 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:50.069 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356165+0000", 2024-05-08T08:53:50.069 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1240000000000001e-06 2024-05-08T08:53:50.069 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.069 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.069 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:50.069 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356173+0000", 2024-05-08T08:53:50.069 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.6199999999999999e-06 2024-05-08T08:53:50.069 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.070 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.070 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:50.070 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356182+0000", 2024-05-08T08:53:50.070 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7760000000000003e-06 2024-05-08T08:53:50.070 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.070 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.070 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:50.070 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434675+0000", 2024-05-08T08:53:50.070 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078493119 2024-05-08T08:53:50.070 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.070 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.070 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:50.070 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434765+0000", 2024-05-08T08:53:50.070 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0402000000000006e-05 2024-05-08T08:53:50.070 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.070 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.071 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:50.071 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434899+0000", 2024-05-08T08:53:50.071 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013391499999999999 2024-05-08T08:53:50.071 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.071 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:50.071 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.071 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.071 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.071 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28550 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888597280] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:50.071 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:53:50.071 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 17.691371223000001, 2024-05-08T08:53:50.071 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078933990999999995, 2024-05-08T08:53:50.071 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:50.071 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:50.071 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:50.071 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:50.072 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:50.072 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:50.072 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28550 2024-05-08T08:53:50.072 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.072 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:50.072 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.072 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:50.072 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:53:50.072 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:50.072 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.072 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.072 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:50.072 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:53:50.072 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:50.072 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.072 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.073 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:50.073 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:53:50.073 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.34e-07 2024-05-08T08:53:50.073 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.073 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.073 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:50.073 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356195+0000", 2024-05-08T08:53:50.073 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8710000000000001e-06 2024-05-08T08:53:50.073 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.073 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.073 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:50.073 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356200+0000", 2024-05-08T08:53:50.073 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3000000000000003e-06 2024-05-08T08:53:50.073 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.073 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.073 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:50.073 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356203+0000", 2024-05-08T08:53:50.074 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4039999999999999e-06 2024-05-08T08:53:50.074 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.074 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.074 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:50.074 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434928+0000", 2024-05-08T08:53:50.074 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078725331999999995 2024-05-08T08:53:50.074 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.074 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.074 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:50.074 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435007+0000", 2024-05-08T08:53:50.074 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8415000000000001e-05 2024-05-08T08:53:50.074 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.074 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.074 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:50.074 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435126+0000", 2024-05-08T08:53:50.075 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011923500000000001 2024-05-08T08:53:50.075 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.075 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:50.075 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.075 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.075 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.075 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28551 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888527200] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:50.075 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.356210+0000", 2024-05-08T08:53:50.075 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 17.691352875, 2024-05-08T08:53:50.075 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079195072000000005, 2024-05-08T08:53:50.075 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:50.075 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:50.075 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:50.075 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:50.075 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:50.075 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:50.076 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28551 2024-05-08T08:53:50.076 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.076 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:50.076 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.076 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:50.076 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356210+0000", 2024-05-08T08:53:50.076 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:50.076 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.076 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.076 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:50.076 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356210+0000", 2024-05-08T08:53:50.076 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:50.076 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.076 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.076 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:50.076 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356211+0000", 2024-05-08T08:53:50.077 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4499999999999998e-07 2024-05-08T08:53:50.077 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.077 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.077 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:50.077 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356212+0000", 2024-05-08T08:53:50.077 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6029999999999999e-06 2024-05-08T08:53:50.077 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.077 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.077 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:50.077 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356221+0000", 2024-05-08T08:53:50.077 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4200000000000007e-06 2024-05-08T08:53:50.077 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.077 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.077 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:50.077 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356224+0000", 2024-05-08T08:53:50.078 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.038e-06 2024-05-08T08:53:50.078 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.078 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.078 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:50.078 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435178+0000", 2024-05-08T08:53:50.078 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078954625000000001 2024-05-08T08:53:50.078 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.078 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.078 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:50.078 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435256+0000", 2024-05-08T08:53:50.078 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7337999999999997e-05 2024-05-08T08:53:50.078 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.078 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.078 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:50.079 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435405+0000", 2024-05-08T08:53:50.079 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000149703 2024-05-08T08:53:50.079 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.079 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:50.079 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.079 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.079 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.079 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28552 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889231456] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:50.079 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.356228+0000", 2024-05-08T08:53:50.079 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 17.691335197000001, 2024-05-08T08:53:50.079 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079346920000000001, 2024-05-08T08:53:50.079 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:50.079 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:50.079 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:50.079 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:50.080 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:50.080 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:50.080 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28552 2024-05-08T08:53:50.080 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.080 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:50.080 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.080 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:50.080 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356228+0000", 2024-05-08T08:53:50.080 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:50.080 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.080 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.080 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:50.080 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356228+0000", 2024-05-08T08:53:50.080 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:50.080 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.080 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.081 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:50.081 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356959+0000", 2024-05-08T08:53:50.081 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00073097099999999996 2024-05-08T08:53:50.081 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.081 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.081 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:50.081 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356967+0000", 2024-05-08T08:53:50.081 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0409999999999998e-06 2024-05-08T08:53:50.081 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.081 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.081 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:50.081 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356979+0000", 2024-05-08T08:53:50.081 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1489e-05 2024-05-08T08:53:50.081 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.081 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.082 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:50.082 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356986+0000", 2024-05-08T08:53:50.082 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3699999999999997e-06 2024-05-08T08:53:50.082 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.082 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.082 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:50.082 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435433+0000", 2024-05-08T08:53:50.082 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078447520000000007 2024-05-08T08:53:50.082 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.082 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.082 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:50.082 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435484+0000", 2024-05-08T08:53:50.082 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0229000000000003e-05 2024-05-08T08:53:50.082 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.082 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.082 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:50.083 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435575+0000", 2024-05-08T08:53:50.083 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1299999999999997e-05 2024-05-08T08:53:50.083 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.083 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:50.083 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.083 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.083 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.083 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28553 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888713648] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:50.083 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:53:50.083 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 17.690560539, 2024-05-08T08:53:50.083 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078782034000000001, 2024-05-08T08:53:50.083 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:50.083 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:50.084 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:50.084 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:50.084 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:50.084 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:50.084 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28553 2024-05-08T08:53:50.084 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.084 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:50.084 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.084 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:50.084 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:53:50.084 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:50.084 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.084 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.084 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:50.085 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:53:50.085 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:50.085 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.085 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.085 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:50.085 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:53:50.085 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6600000000000002e-07 2024-05-08T08:53:50.085 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.085 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.085 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:50.085 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357006+0000", 2024-05-08T08:53:50.085 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7190000000000001e-06 2024-05-08T08:53:50.085 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.085 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.085 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:50.085 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357013+0000", 2024-05-08T08:53:50.086 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.435e-06 2024-05-08T08:53:50.086 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.086 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.086 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:50.086 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357018+0000", 2024-05-08T08:53:50.086 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7260000000000002e-06 2024-05-08T08:53:50.086 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.086 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.086 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:50.086 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435623+0000", 2024-05-08T08:53:50.086 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078604972999999995 2024-05-08T08:53:50.086 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.086 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.086 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:50.086 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435695+0000", 2024-05-08T08:53:50.087 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1959999999999995e-05 2024-05-08T08:53:50.087 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.087 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.087 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:50.087 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435785+0000", 2024-05-08T08:53:50.087 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.9754999999999997e-05 2024-05-08T08:53:50.087 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.087 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:50.087 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.087 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.087 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.087 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28554 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888980272] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:50.087 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357024+0000", 2024-05-08T08:53:50.087 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 17.690538903, 2024-05-08T08:53:50.087 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078999870999999999, 2024-05-08T08:53:50.087 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:50.088 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:50.088 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:50.088 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:50.088 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:50.088 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:50.088 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28554 2024-05-08T08:53:50.088 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.088 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:50.088 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.088 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:50.088 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357024+0000", 2024-05-08T08:53:50.088 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:50.088 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.088 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.088 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:50.088 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357024+0000", 2024-05-08T08:53:50.088 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:50.089 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.089 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.089 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:50.089 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357025+0000", 2024-05-08T08:53:50.089 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:53:50.089 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.089 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.089 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:50.089 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357027+0000", 2024-05-08T08:53:50.089 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2050000000000001e-06 2024-05-08T08:53:50.089 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.089 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.089 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:50.089 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357033+0000", 2024-05-08T08:53:50.089 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5679999999999996e-06 2024-05-08T08:53:50.089 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.090 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.090 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:50.090 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357038+0000", 2024-05-08T08:53:50.090 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2889999999999999e-06 2024-05-08T08:53:50.090 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.090 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.090 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:50.090 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435823+0000", 2024-05-08T08:53:50.090 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078785677999999998 2024-05-08T08:53:50.090 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.090 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.090 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:50.090 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435892+0000", 2024-05-08T08:53:50.090 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8379000000000004e-05 2024-05-08T08:53:50.090 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.091 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.091 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:50.091 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436024+0000", 2024-05-08T08:53:50.091 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013239799999999999 2024-05-08T08:53:50.091 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.091 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:50.091 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.091 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.091 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.091 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28555 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889001104] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:50.091 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:53:50.091 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 17.690520138, 2024-05-08T08:53:50.091 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079115742000000003, 2024-05-08T08:53:50.091 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:50.091 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:50.092 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:50.092 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:50.092 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:50.092 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:50.092 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28555 2024-05-08T08:53:50.092 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.092 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:50.092 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.092 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:50.092 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:53:50.092 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:50.092 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.092 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.092 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:50.092 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:53:50.092 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:50.093 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.093 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.093 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:50.093 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:53:50.093 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9200000000000002e-07 2024-05-08T08:53:50.093 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.093 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.093 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:50.093 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357046+0000", 2024-05-08T08:53:50.093 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1670000000000002e-06 2024-05-08T08:53:50.093 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.093 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.093 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:50.093 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357052+0000", 2024-05-08T08:53:50.093 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.139e-06 2024-05-08T08:53:50.093 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.093 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.094 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:50.094 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357056+0000", 2024-05-08T08:53:50.094 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2440000000000004e-06 2024-05-08T08:53:50.094 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.094 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.094 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:50.094 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436059+0000", 2024-05-08T08:53:50.094 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079003070999999994 2024-05-08T08:53:50.094 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.094 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.094 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:50.094 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436097+0000", 2024-05-08T08:53:50.094 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7784000000000003e-05 2024-05-08T08:53:50.095 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.095 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.095 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:50.095 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436159+0000", 2024-05-08T08:53:50.095 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2045000000000007e-05 2024-05-08T08:53:50.095 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.095 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:50.095 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.095 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.095 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.095 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28556 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889022064] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:50.095 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357065+0000", 2024-05-08T08:53:50.095 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 17.690498105, 2024-05-08T08:53:50.095 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079327976999999994, 2024-05-08T08:53:50.095 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:50.096 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:50.096 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:50.096 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:50.096 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:50.096 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:50.096 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28556 2024-05-08T08:53:50.096 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.096 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:50.096 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.096 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:50.096 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357065+0000", 2024-05-08T08:53:50.096 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:50.096 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.096 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.096 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:50.097 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357065+0000", 2024-05-08T08:53:50.097 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:50.097 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.097 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.097 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:50.097 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358958+0000", 2024-05-08T08:53:50.097 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001892915 2024-05-08T08:53:50.097 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.097 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.097 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:50.097 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358962+0000", 2024-05-08T08:53:50.097 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4599999999999999e-06 2024-05-08T08:53:50.097 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.097 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.097 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:50.098 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358970+0000", 2024-05-08T08:53:50.098 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.6090000000000001e-06 2024-05-08T08:53:50.098 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.098 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.098 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:50.098 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358975+0000", 2024-05-08T08:53:50.098 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7639999999999996e-06 2024-05-08T08:53:50.098 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.098 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.098 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:50.098 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436196+0000", 2024-05-08T08:53:50.098 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.077221564000000006 2024-05-08T08:53:50.098 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.098 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.098 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:50.098 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436269+0000", 2024-05-08T08:53:50.099 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2720999999999997e-05 2024-05-08T08:53:50.099 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.099 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.099 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:50.099 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436393+0000", 2024-05-08T08:53:50.099 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012394399999999999 2024-05-08T08:53:50.099 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.099 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:50.099 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.099 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.099 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.099 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28563 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888855520] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:50.099 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:53:50.099 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 17.688488706000001, 2024-05-08T08:53:50.099 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078795566999999997, 2024-05-08T08:53:50.099 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:50.100 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:50.100 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:50.100 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:50.100 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:50.100 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:50.100 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28563 2024-05-08T08:53:50.100 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.100 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:50.100 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.100 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:50.100 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:53:50.100 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:50.100 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.100 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.100 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:50.100 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:53:50.101 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:50.101 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.101 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.101 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:50.101 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:53:50.101 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1e-07 2024-05-08T08:53:50.101 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.101 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.101 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:50.101 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359080+0000", 2024-05-08T08:53:50.101 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8940000000000001e-06 2024-05-08T08:53:50.101 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.101 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.101 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:50.101 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359083+0000", 2024-05-08T08:53:50.101 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1949999999999999e-06 2024-05-08T08:53:50.101 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.102 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.102 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:50.102 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359085+0000", 2024-05-08T08:53:50.102 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4710000000000001e-06 2024-05-08T08:53:50.102 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.102 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.102 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:50.102 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437604+0000", 2024-05-08T08:53:50.102 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078518278999999996 2024-05-08T08:53:50.102 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.102 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.102 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:50.102 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437704+0000", 2024-05-08T08:53:50.102 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010010599999999999 2024-05-08T08:53:50.102 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.103 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.103 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:50.103 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437870+0000", 2024-05-08T08:53:50.103 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00016641200000000001 2024-05-08T08:53:50.103 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.103 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:50.103 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.103 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.103 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.103 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28564 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888538944] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:50.103 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.359089+0000", 2024-05-08T08:53:50.103 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 17.688474491000001, 2024-05-08T08:53:50.103 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078952675, 2024-05-08T08:53:50.103 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:50.103 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:50.103 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:50.104 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:50.104 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:50.104 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:50.104 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28564 2024-05-08T08:53:50.104 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.104 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:50.104 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.104 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:50.104 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359089+0000", 2024-05-08T08:53:50.104 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:50.104 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.104 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.104 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:50.104 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359089+0000", 2024-05-08T08:53:50.105 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:50.105 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.105 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.105 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:50.105 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360959+0000", 2024-05-08T08:53:50.105 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0018707089999999999 2024-05-08T08:53:50.105 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.105 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.105 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:50.105 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360963+0000", 2024-05-08T08:53:50.105 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0680000000000001e-06 2024-05-08T08:53:50.105 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.105 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.105 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:50.105 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360969+0000", 2024-05-08T08:53:50.105 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1920000000000002e-06 2024-05-08T08:53:50.106 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.106 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.106 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:50.106 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360973+0000", 2024-05-08T08:53:50.106 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.031e-06 2024-05-08T08:53:50.106 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.106 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.106 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:50.106 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437916+0000", 2024-05-08T08:53:50.106 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.076942939000000002 2024-05-08T08:53:50.106 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.106 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.106 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:50.106 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437973+0000", 2024-05-08T08:53:50.106 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7358999999999997e-05 2024-05-08T08:53:50.107 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.107 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.107 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:50.107 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.438041+0000", 2024-05-08T08:53:50.107 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8376999999999999e-05 2024-05-08T08:53:50.107 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.107 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:50.107 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.107 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.107 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.107 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28574 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889427600] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:50.107 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:53:50.107 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 17.686469983999999, 2024-05-08T08:53:50.107 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078782476000000004, 2024-05-08T08:53:50.107 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:50.107 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:50.107 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:50.108 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:50.108 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:50.108 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:50.108 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28574 2024-05-08T08:53:50.108 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.108 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:50.108 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.108 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:50.108 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:53:50.108 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:50.108 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.108 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.108 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:50.108 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:53:50.108 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:50.108 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.108 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.109 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:50.109 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:53:50.109 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1799999999999999e-07 2024-05-08T08:53:50.109 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.109 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.109 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:50.109 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361095+0000", 2024-05-08T08:53:50.109 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3850000000000001e-06 2024-05-08T08:53:50.109 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.109 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.109 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:50.109 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361098+0000", 2024-05-08T08:53:50.109 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7989999999999998e-06 2024-05-08T08:53:50.109 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.109 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.109 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:50.110 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361100+0000", 2024-05-08T08:53:50.110 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4669999999999998e-06 2024-05-08T08:53:50.110 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.110 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.110 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:50.110 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439768+0000", 2024-05-08T08:53:50.110 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078668057999999999 2024-05-08T08:53:50.110 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.110 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.110 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:50.110 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439811+0000", 2024-05-08T08:53:50.110 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2459000000000001e-05 2024-05-08T08:53:50.110 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.110 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.110 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:50.111 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439876+0000", 2024-05-08T08:53:50.111 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5090000000000002e-05 2024-05-08T08:53:50.111 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.111 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:50.111 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.111 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.111 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.111 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28575 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889439216] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:50.111 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361103+0000", 2024-05-08T08:53:50.111 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 17.686459969000001, 2024-05-08T08:53:50.111 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078966828000000003, 2024-05-08T08:53:50.111 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:50.111 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:50.111 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:50.111 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:50.111 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:50.111 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:50.112 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28575 2024-05-08T08:53:50.112 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.112 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:50.112 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.112 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:50.112 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361103+0000", 2024-05-08T08:53:50.112 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:50.112 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.112 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.112 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:50.112 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361103+0000", 2024-05-08T08:53:50.112 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:50.112 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.112 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.112 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:50.113 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361104+0000", 2024-05-08T08:53:50.113 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1400000000000001e-07 2024-05-08T08:53:50.113 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.113 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.113 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:50.113 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361105+0000", 2024-05-08T08:53:50.113 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5540000000000001e-06 2024-05-08T08:53:50.113 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.113 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.113 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:50.113 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361108+0000", 2024-05-08T08:53:50.113 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.841e-06 2024-05-08T08:53:50.114 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.114 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.114 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:50.114 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361110+0000", 2024-05-08T08:53:50.114 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.497e-06 2024-05-08T08:53:50.114 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.114 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.114 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:50.114 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439916+0000", 2024-05-08T08:53:50.114 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078805372999999998 2024-05-08T08:53:50.114 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.114 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.115 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:50.115 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440000+0000", 2024-05-08T08:53:50.115 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4730999999999999e-05 2024-05-08T08:53:50.115 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.115 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.115 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:50.115 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440070+0000", 2024-05-08T08:53:50.115 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9617999999999997e-05 2024-05-08T08:53:50.115 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.115 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:50.115 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.115 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.115 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.115 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28576 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889451328] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:50.116 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361113+0000", 2024-05-08T08:53:50.116 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 17.686449783, 2024-05-08T08:53:50.116 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079088862999999995, 2024-05-08T08:53:50.116 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:50.116 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:50.116 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:50.116 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:50.116 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:50.116 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:50.116 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28576 2024-05-08T08:53:50.116 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.116 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:50.116 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.116 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:50.117 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361113+0000", 2024-05-08T08:53:50.117 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:50.117 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.117 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.117 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:50.117 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361113+0000", 2024-05-08T08:53:50.117 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:50.117 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.117 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.117 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:50.117 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361114+0000", 2024-05-08T08:53:50.117 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1299999999999999e-07 2024-05-08T08:53:50.117 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.117 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.117 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:50.117 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361115+0000", 2024-05-08T08:53:50.117 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.4300000000000001e-06 2024-05-08T08:53:50.118 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.118 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.118 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:50.118 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361118+0000", 2024-05-08T08:53:50.118 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9000000000000002e-06 2024-05-08T08:53:50.118 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.118 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.118 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:50.118 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361120+0000", 2024-05-08T08:53:50.118 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4739999999999999e-06 2024-05-08T08:53:50.118 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.118 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.118 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:50.118 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440096+0000", 2024-05-08T08:53:50.118 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078975629000000006 2024-05-08T08:53:50.118 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.119 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.119 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:50.119 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440135+0000", 2024-05-08T08:53:50.119 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9202000000000002e-05 2024-05-08T08:53:50.119 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.119 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.119 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:50.119 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440202+0000", 2024-05-08T08:53:50.119 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7014999999999994e-05 2024-05-08T08:53:50.119 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.119 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:50.119 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.119 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.119 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.119 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28577 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889462640] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:50.120 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:53:50.120 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 17.686439601, 2024-05-08T08:53:50.120 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079194622000000006, 2024-05-08T08:53:50.120 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:50.120 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:50.120 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:50.120 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:50.120 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:50.120 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:50.120 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28577 2024-05-08T08:53:50.120 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.120 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:50.120 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.120 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:50.120 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:53:50.120 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:50.120 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.121 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.121 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:50.121 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:53:50.121 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:50.121 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.121 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.121 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:50.121 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:53:50.121 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1400000000000001e-07 2024-05-08T08:53:50.121 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.121 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.121 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:50.121 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361125+0000", 2024-05-08T08:53:50.121 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.372e-06 2024-05-08T08:53:50.121 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.121 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.121 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:50.122 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361128+0000", 2024-05-08T08:53:50.122 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0309999999999999e-06 2024-05-08T08:53:50.122 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.122 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.122 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:50.122 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361131+0000", 2024-05-08T08:53:50.122 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4040000000000002e-06 2024-05-08T08:53:50.122 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.122 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.122 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:50.122 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440225+0000", 2024-05-08T08:53:50.122 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079094600000000001 2024-05-08T08:53:50.122 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.122 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.123 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:50.123 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440260+0000", 2024-05-08T08:53:50.123 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4703000000000003e-05 2024-05-08T08:53:50.123 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.123 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.123 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:50.123 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440318+0000", 2024-05-08T08:53:50.123 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8298000000000003e-05 2024-05-08T08:53:50.123 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.123 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:50.123 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.123 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.123 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.123 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28578 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889474192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:50.123 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361134+0000", 2024-05-08T08:53:50.123 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 17.686429176000001, 2024-05-08T08:53:50.124 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079310295000000003, 2024-05-08T08:53:50.124 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:50.124 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:50.124 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:50.124 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:50.124 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:50.124 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:50.124 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28578 2024-05-08T08:53:50.124 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.124 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:50.124 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.124 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:50.124 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361134+0000", 2024-05-08T08:53:50.125 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:50.125 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.125 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.125 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:50.125 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361134+0000", 2024-05-08T08:53:50.125 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:50.125 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.125 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.125 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:50.125 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.362958+0000", 2024-05-08T08:53:50.125 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0018237380000000001 2024-05-08T08:53:50.125 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.125 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.125 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:50.126 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.362961+0000", 2024-05-08T08:53:50.126 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.123e-06 2024-05-08T08:53:50.126 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.126 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.126 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:50.126 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.397404+0000", 2024-05-08T08:53:50.126 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.034443226 2024-05-08T08:53:50.126 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.126 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.126 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:50.126 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.397439+0000", 2024-05-08T08:53:50.126 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4690000000000002e-05 2024-05-08T08:53:50.126 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.127 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.127 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:50.127 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440340+0000", 2024-05-08T08:53:50.127 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.042901419000000003 2024-05-08T08:53:50.127 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.127 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.127 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:50.127 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440376+0000", 2024-05-08T08:53:50.127 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5444999999999997e-05 2024-05-08T08:53:50.127 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.127 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.127 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:50.127 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440444+0000", 2024-05-08T08:53:50.127 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8653999999999997e-05 2024-05-08T08:53:50.127 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.128 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:50.128 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.128 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.128 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:50.128 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:50.129 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:53:50.130 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:53:50.131+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:53:50.229 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:50.232+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:50.295 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:50.295 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:53:50.295 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:53:50.295 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:50.305 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:53:50.331 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:50.333+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:50.431 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:53:50.433+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:53:50.432 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:50.432 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:53:50.432 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:53:50.432 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:53:50.432 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:50.442 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:53:50.532 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:50.535+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:50.589 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-05-08T08:53:50.598 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:50.598 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:53:50.598 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:53:50.598 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:53:50.598 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.598 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:437 3.17 3:ea93a512:::rbd_data.1140100c9703.0000000000000077:head [set-alloc-hint object_size 2097152 write_size 2097152,write 1170432~1024] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e63)", 2024-05-08T08:53:50.598 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:08.390804+0000", 2024-05-08T08:53:50.598 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 42.209223455, 2024-05-08T08:53:50.598 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0406068690000001, 2024-05-08T08:53:50.598 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:50.598 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:50.598 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:53:50.599 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:50.599 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:50.599 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:50.599 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 437 2024-05-08T08:53:50.599 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.599 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:50.599 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.599 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:50.599 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390804+0000", 2024-05-08T08:53:50.600 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:50.600 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.600 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.600 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:50.600 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390804+0000", 2024-05-08T08:53:50.600 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:50.600 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.600 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.600 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:50.600 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390806+0000", 2024-05-08T08:53:50.600 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.666e-06 2024-05-08T08:53:50.600 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.600 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.600 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:50.601 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390818+0000", 2024-05-08T08:53:50.601 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2187999999999999e-05 2024-05-08T08:53:50.601 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.601 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.601 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:50.601 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390834+0000", 2024-05-08T08:53:50.601 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6191000000000001e-05 2024-05-08T08:53:50.601 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.601 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.601 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:50.601 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390845+0000", 2024-05-08T08:53:50.601 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0831e-05 2024-05-08T08:53:50.601 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.601 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.601 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:50.602 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390922+0000", 2024-05-08T08:53:50.602 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7253e-05 2024-05-08T08:53:50.602 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.602 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.602 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op must wait for map", 2024-05-08T08:53:50.602 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390930+0000", 2024-05-08T08:53:50.602 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8259999999999993e-06 2024-05-08T08:53:50.602 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.602 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.602 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:50.602 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392155+0000", 2024-05-08T08:53:50.602 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0012256649999999999 2024-05-08T08:53:50.602 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.602 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.603 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:53:50.603 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392165+0000", 2024-05-08T08:53:50.603 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5799999999999998e-06 2024-05-08T08:53:50.603 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.603 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.603 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:50.603 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.429596+0000", 2024-05-08T08:53:50.603 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0374313639999999 2024-05-08T08:53:50.603 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.603 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.603 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:50.603 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.429728+0000", 2024-05-08T08:53:50.603 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000131552 2024-05-08T08:53:50.603 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.604 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.604 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 0,1", 2024-05-08T08:53:50.604 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.429860+0000", 2024-05-08T08:53:50.604 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000132137 2024-05-08T08:53:50.604 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.604 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.604 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:53:50.604 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.430811+0000", 2024-05-08T08:53:50.604 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00095040200000000004 2024-05-08T08:53:50.604 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.604 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.604 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:53:50.604 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431126+0000", 2024-05-08T08:53:50.604 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000315006 2024-05-08T08:53:50.604 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.605 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.605 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:53:50.605 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431272+0000", 2024-05-08T08:53:50.605 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000146487 2024-05-08T08:53:50.605 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.605 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.605 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:53:50.605 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431315+0000", 2024-05-08T08:53:50.605 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2697000000000001e-05 2024-05-08T08:53:50.605 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.605 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.605 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:50.605 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431411+0000", 2024-05-08T08:53:50.605 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6024000000000001e-05 2024-05-08T08:53:50.605 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.605 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:50.606 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.606 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.606 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.606 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:439 3.17 3:ea93a512:::rbd_data.1140100c9703.0000000000000077:head [read 1163264~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-05-08T08:53:50.606 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:08.390871+0000", 2024-05-08T08:53:50.606 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 42.209155944000003, 2024-05-08T08:53:50.606 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0408790590000001, 2024-05-08T08:53:50.606 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:50.606 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:50.606 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:50.606 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:50.606 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:50.606 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:50.606 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 439 2024-05-08T08:53:50.606 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.607 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:50.607 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.607 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:50.607 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390871+0000", 2024-05-08T08:53:50.607 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:50.607 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.607 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.607 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:50.607 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390871+0000", 2024-05-08T08:53:50.607 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:50.607 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.607 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.607 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:50.607 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390872+0000", 2024-05-08T08:53:50.608 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6300000000000005e-07 2024-05-08T08:53:50.608 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.608 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.608 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:50.608 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390875+0000", 2024-05-08T08:53:50.608 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3459999999999998e-06 2024-05-08T08:53:50.608 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.608 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.608 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:50.608 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390891+0000", 2024-05-08T08:53:50.608 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5255e-05 2024-05-08T08:53:50.608 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.608 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.608 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:50.609 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390897+0000", 2024-05-08T08:53:50.609 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2360000000000001e-06 2024-05-08T08:53:50.609 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.609 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.609 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:50.609 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390995+0000", 2024-05-08T08:53:50.609 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8522000000000003e-05 2024-05-08T08:53:50.609 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.609 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.609 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting_for_map not empty", 2024-05-08T08:53:50.609 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.391005+0000", 2024-05-08T08:53:50.609 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1460000000000002e-06 2024-05-08T08:53:50.609 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.609 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.609 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:50.609 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392191+0000", 2024-05-08T08:53:50.610 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0011860270000000001 2024-05-08T08:53:50.610 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.610 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.610 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:53:50.610 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392197+0000", 2024-05-08T08:53:50.610 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4289999999999998e-06 2024-05-08T08:53:50.610 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.610 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.610 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:50.610 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.430596+0000", 2024-05-08T08:53:50.610 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0383989090000001 2024-05-08T08:53:50.610 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.610 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.610 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for rw locks", 2024-05-08T08:53:50.611 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.430676+0000", 2024-05-08T08:53:50.611 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9781000000000001e-05 2024-05-08T08:53:50.611 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.611 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.611 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:50.611 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431460+0000", 2024-05-08T08:53:50.611 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00078427300000000004 2024-05-08T08:53:50.611 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.611 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.611 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:50.612 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431540+0000", 2024-05-08T08:53:50.612 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0224999999999998e-05 2024-05-08T08:53:50.612 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.612 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.612 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:50.612 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431750+0000", 2024-05-08T08:53:50.612 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00021024699999999999 2024-05-08T08:53:50.612 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.612 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:50.612 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.612 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.612 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.612 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:945 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 4096000~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:50.612 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:53:50.612 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 41.149006376999999, 2024-05-08T08:53:50.613 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086774741000000002, 2024-05-08T08:53:50.613 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:50.613 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:50.613 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:50.613 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:50.613 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:50.613 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:50.613 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 945 2024-05-08T08:53:50.613 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.613 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:50.613 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.614 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:50.614 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:53:50.614 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:50.614 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.614 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.614 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:50.614 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:53:50.614 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:50.614 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.614 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.614 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:50.614 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:53:50.614 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3499999999999996e-07 2024-05-08T08:53:50.614 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.615 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.615 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:50.615 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451024+0000", 2024-05-08T08:53:50.615 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.627e-06 2024-05-08T08:53:50.615 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.615 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.615 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:50.615 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451037+0000", 2024-05-08T08:53:50.615 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2431e-05 2024-05-08T08:53:50.615 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.615 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.615 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:50.615 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451042+0000", 2024-05-08T08:53:50.615 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8409999999999996e-06 2024-05-08T08:53:50.616 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.616 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.616 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:50.616 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537608+0000", 2024-05-08T08:53:50.616 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086565243 2024-05-08T08:53:50.616 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.616 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.616 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:50.616 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537687+0000", 2024-05-08T08:53:50.616 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9519999999999998e-05 2024-05-08T08:53:50.616 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.616 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.616 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:50.616 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537796+0000", 2024-05-08T08:53:50.616 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000108544 2024-05-08T08:53:50.617 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.617 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:50.617 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.617 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.617 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.617 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:964 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 458752~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:50.617 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451051+0000", 2024-05-08T08:53:50.617 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 41.148976204, 2024-05-08T08:53:50.617 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086990490000000004, 2024-05-08T08:53:50.617 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:50.618 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:50.618 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:50.618 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:50.618 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:50.618 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:50.618 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 964 2024-05-08T08:53:50.618 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.618 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:50.618 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.618 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:50.618 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451051+0000", 2024-05-08T08:53:50.618 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:50.618 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.618 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.619 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:50.619 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451051+0000", 2024-05-08T08:53:50.619 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:50.619 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.619 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.619 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:50.619 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451052+0000", 2024-05-08T08:53:50.619 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7299999999999996e-07 2024-05-08T08:53:50.619 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.619 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.619 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:50.619 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451055+0000", 2024-05-08T08:53:50.619 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.2770000000000001e-06 2024-05-08T08:53:50.619 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.619 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.619 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:50.620 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451061+0000", 2024-05-08T08:53:50.620 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4060000000000002e-06 2024-05-08T08:53:50.620 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.620 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.620 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:50.620 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451066+0000", 2024-05-08T08:53:50.620 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0839999999999999e-06 2024-05-08T08:53:50.620 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.620 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.620 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:50.620 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537839+0000", 2024-05-08T08:53:50.620 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086772962999999995 2024-05-08T08:53:50.620 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.620 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.620 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:50.621 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537934+0000", 2024-05-08T08:53:50.621 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4742999999999997e-05 2024-05-08T08:53:50.621 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.621 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.621 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:50.621 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538042+0000", 2024-05-08T08:53:50.621 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000107444 2024-05-08T08:53:50.621 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.621 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:50.621 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.621 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.621 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.621 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:965 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 950272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:50.621 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451075+0000", 2024-05-08T08:53:50.621 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 41.148952141999999, 2024-05-08T08:53:50.622 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087230196999999995, 2024-05-08T08:53:50.622 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:50.622 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:50.622 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:50.622 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:50.622 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:50.622 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:50.622 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 965 2024-05-08T08:53:50.622 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.622 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:50.622 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.622 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:50.622 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451075+0000", 2024-05-08T08:53:50.622 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:50.622 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.623 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.623 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:50.623 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451075+0000", 2024-05-08T08:53:50.623 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:50.623 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.623 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.623 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:50.623 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451076+0000", 2024-05-08T08:53:50.623 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5000000000000002e-07 2024-05-08T08:53:50.623 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.623 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.623 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:50.623 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451078+0000", 2024-05-08T08:53:50.623 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.5639999999999999e-06 2024-05-08T08:53:50.623 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.624 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.624 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:50.624 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451085+0000", 2024-05-08T08:53:50.624 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5640000000000002e-06 2024-05-08T08:53:50.624 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.624 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.624 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:50.624 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451090+0000", 2024-05-08T08:53:50.624 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1699999999999996e-06 2024-05-08T08:53:50.624 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.624 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.624 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:50.624 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538106+0000", 2024-05-08T08:53:50.624 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087015888999999999 2024-05-08T08:53:50.624 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.625 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.625 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:50.625 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538190+0000", 2024-05-08T08:53:50.625 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3961999999999993e-05 2024-05-08T08:53:50.625 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.625 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.625 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:50.625 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538305+0000", 2024-05-08T08:53:50.625 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011529800000000001 2024-05-08T08:53:50.625 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.625 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:50.625 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.625 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.625 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.626 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:966 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 1441792~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:50.626 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:53:50.626 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 41.148928769000001, 2024-05-08T08:53:50.626 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087446323000000006, 2024-05-08T08:53:50.626 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:50.626 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:50.626 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:50.626 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:50.626 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:50.626 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:50.626 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 966 2024-05-08T08:53:50.626 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.626 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:50.626 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.627 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:50.627 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:53:50.627 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:50.627 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.627 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.627 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:50.627 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:53:50.627 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:50.627 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.627 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.627 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:50.627 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:53:50.627 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3700000000000001e-07 2024-05-08T08:53:50.627 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.627 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.628 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:50.628 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451101+0000", 2024-05-08T08:53:50.628 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2560000000000001e-06 2024-05-08T08:53:50.628 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.628 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.628 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:50.628 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451107+0000", 2024-05-08T08:53:50.628 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1070000000000001e-06 2024-05-08T08:53:50.628 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.628 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.628 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:50.628 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451112+0000", 2024-05-08T08:53:50.628 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1420000000000001e-06 2024-05-08T08:53:50.628 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.628 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.628 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:50.629 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538341+0000", 2024-05-08T08:53:50.629 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087228302999999993 2024-05-08T08:53:50.629 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.629 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.629 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:50.629 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538424+0000", 2024-05-08T08:53:50.629 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3497000000000004e-05 2024-05-08T08:53:50.629 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.629 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.629 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:50.629 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538545+0000", 2024-05-08T08:53:50.629 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000120581 2024-05-08T08:53:50.629 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.629 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:50.629 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.629 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.630 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.630 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:967 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:50.630 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451120+0000", 2024-05-08T08:53:50.630 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 41.148907293999997, 2024-05-08T08:53:50.630 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087632725999999994, 2024-05-08T08:53:50.630 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:50.630 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:50.630 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:50.630 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:50.630 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:50.630 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:50.630 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 967 2024-05-08T08:53:50.630 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.630 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:50.631 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.631 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:50.631 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451120+0000", 2024-05-08T08:53:50.631 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:50.631 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.631 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.631 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:50.631 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451120+0000", 2024-05-08T08:53:50.631 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:50.631 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.631 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.631 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:50.631 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451121+0000", 2024-05-08T08:53:50.631 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8800000000000002e-07 2024-05-08T08:53:50.631 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.632 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.632 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:50.632 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451130+0000", 2024-05-08T08:53:50.632 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2299999999999997e-06 2024-05-08T08:53:50.632 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.632 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.632 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:50.632 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451137+0000", 2024-05-08T08:53:50.632 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4669999999999998e-06 2024-05-08T08:53:50.632 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.632 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.632 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:50.632 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451143+0000", 2024-05-08T08:53:50.632 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2839999999999997e-06 2024-05-08T08:53:50.632 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.633 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.633 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:50.633 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538575+0000", 2024-05-08T08:53:50.633 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087432789999999996 2024-05-08T08:53:50.633 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.633 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.633 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:50.633 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538648+0000", 2024-05-08T08:53:50.633 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2877e-05 2024-05-08T08:53:50.633 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.633 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.633 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:50.633 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538753+0000", 2024-05-08T08:53:50.633 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010449 2024-05-08T08:53:50.633 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.634 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:50.634 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.634 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.634 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.634 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:968 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 2424832~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:50.634 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451152+0000", 2024-05-08T08:53:50.634 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 41.148875209000003, 2024-05-08T08:53:50.634 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087813325999999997, 2024-05-08T08:53:50.634 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:50.634 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:50.634 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:50.634 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:50.634 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:50.634 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:50.634 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 968 2024-05-08T08:53:50.635 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.635 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:50.635 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.635 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:50.635 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451152+0000", 2024-05-08T08:53:50.635 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:50.635 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.635 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.635 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:50.635 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451152+0000", 2024-05-08T08:53:50.635 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:50.635 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.635 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.635 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:50.635 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451153+0000", 2024-05-08T08:53:50.635 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3099999999999998e-07 2024-05-08T08:53:50.636 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.636 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.636 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:50.636 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451155+0000", 2024-05-08T08:53:50.636 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4760000000000001e-06 2024-05-08T08:53:50.636 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.636 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.636 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:50.636 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451162+0000", 2024-05-08T08:53:50.636 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5969999999999997e-06 2024-05-08T08:53:50.636 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.636 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.636 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:50.636 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451167+0000", 2024-05-08T08:53:50.636 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.977e-06 2024-05-08T08:53:50.636 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.637 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.637 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:50.637 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538781+0000", 2024-05-08T08:53:50.637 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087614846999999996 2024-05-08T08:53:50.637 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.637 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.637 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:50.637 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538857+0000", 2024-05-08T08:53:50.637 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5358000000000006e-05 2024-05-08T08:53:50.637 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.637 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.637 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:50.637 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538965+0000", 2024-05-08T08:53:50.637 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010864 2024-05-08T08:53:50.637 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.637 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:50.638 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.638 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.638 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.638 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:969 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 2916352~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:50.638 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:53:50.638 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 41.148853504000002, 2024-05-08T08:53:50.638 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087998980000000004, 2024-05-08T08:53:50.638 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:50.638 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:50.638 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:50.638 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:50.638 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:50.638 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:50.638 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 969 2024-05-08T08:53:50.639 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.639 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:50.639 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.639 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:50.639 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:53:50.639 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:50.639 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.639 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.639 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:50.639 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:53:50.639 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:50.639 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.639 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.639 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:50.640 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:53:50.640 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3599999999999999e-07 2024-05-08T08:53:50.640 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.640 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.640 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:50.640 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451177+0000", 2024-05-08T08:53:50.640 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4590000000000001e-06 2024-05-08T08:53:50.640 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.640 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.640 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:50.640 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451184+0000", 2024-05-08T08:53:50.640 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4109999999999999e-06 2024-05-08T08:53:50.640 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.640 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.640 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:50.640 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451191+0000", 2024-05-08T08:53:50.641 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7279999999999998e-06 2024-05-08T08:53:50.641 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.641 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.641 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:50.641 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538997+0000", 2024-05-08T08:53:50.641 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087806170000000003 2024-05-08T08:53:50.641 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.641 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.641 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:50.641 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539069+0000", 2024-05-08T08:53:50.641 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2373999999999995e-05 2024-05-08T08:53:50.641 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.641 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.641 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:50.641 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539173+0000", 2024-05-08T08:53:50.641 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000103402 2024-05-08T08:53:50.642 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.642 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:50.642 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.642 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.642 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.642 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:970 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 3407872~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:50.642 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451198+0000", 2024-05-08T08:53:50.642 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 41.148828907000002, 2024-05-08T08:53:50.642 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088183368999999998, 2024-05-08T08:53:50.642 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:50.642 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:50.642 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:50.642 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:50.642 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:50.643 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:50.643 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 970 2024-05-08T08:53:50.643 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.643 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:50.643 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.643 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:50.643 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451198+0000", 2024-05-08T08:53:50.643 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:50.643 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.643 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.643 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:50.643 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451198+0000", 2024-05-08T08:53:50.643 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:50.643 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.643 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.644 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:50.644 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451199+0000", 2024-05-08T08:53:50.644 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7400000000000003e-07 2024-05-08T08:53:50.644 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.644 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.644 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:50.644 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451211+0000", 2024-05-08T08:53:50.644 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2109999999999999e-05 2024-05-08T08:53:50.644 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.644 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.644 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:50.644 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451219+0000", 2024-05-08T08:53:50.644 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4170000000000003e-06 2024-05-08T08:53:50.644 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.645 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.645 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:50.645 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451224+0000", 2024-05-08T08:53:50.645 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3990000000000003e-06 2024-05-08T08:53:50.645 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.645 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.645 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:50.645 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539204+0000", 2024-05-08T08:53:50.645 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087980099000000006 2024-05-08T08:53:50.645 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.645 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.645 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:50.645 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539276+0000", 2024-05-08T08:53:50.645 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1513000000000005e-05 2024-05-08T08:53:50.645 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.645 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.646 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:50.646 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539382+0000", 2024-05-08T08:53:50.646 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010615699999999999 2024-05-08T08:53:50.646 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.646 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:50.646 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.646 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.646 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.646 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:971 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 3899392~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:50.646 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451233+0000", 2024-05-08T08:53:50.646 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 41.148794199999998, 2024-05-08T08:53:50.646 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088431525999999996, 2024-05-08T08:53:50.646 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:50.646 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:50.647 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:50.647 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:50.647 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:50.647 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:50.647 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 971 2024-05-08T08:53:50.647 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.647 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:50.647 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.647 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:50.647 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451233+0000", 2024-05-08T08:53:50.647 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:50.647 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.647 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.647 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:50.647 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451233+0000", 2024-05-08T08:53:50.648 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:50.648 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.648 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.648 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:50.648 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451234+0000", 2024-05-08T08:53:50.648 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8500000000000001e-07 2024-05-08T08:53:50.648 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.648 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.648 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:50.648 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451237+0000", 2024-05-08T08:53:50.648 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8320000000000002e-06 2024-05-08T08:53:50.648 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.648 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.648 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:50.648 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451243+0000", 2024-05-08T08:53:50.648 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6749999999999996e-06 2024-05-08T08:53:50.649 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.649 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.649 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:50.649 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451249+0000", 2024-05-08T08:53:50.649 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1229999999999997e-06 2024-05-08T08:53:50.649 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.649 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.649 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:50.649 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539412+0000", 2024-05-08T08:53:50.649 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088162668999999999 2024-05-08T08:53:50.649 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.649 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.649 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:50.649 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539505+0000", 2024-05-08T08:53:50.649 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3240999999999997e-05 2024-05-08T08:53:50.649 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.650 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.650 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:50.650 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539665+0000", 2024-05-08T08:53:50.650 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00015930100000000001 2024-05-08T08:53:50.650 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.650 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:50.650 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.650 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.650 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.650 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:972 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 196608~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:50.650 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451257+0000", 2024-05-08T08:53:50.650 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 41.148770255000002, 2024-05-08T08:53:50.650 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088719928000000003, 2024-05-08T08:53:50.650 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:50.650 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:50.651 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:50.651 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:50.651 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:50.651 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:50.651 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 972 2024-05-08T08:53:50.651 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.651 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:50.651 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.651 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:50.651 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451257+0000", 2024-05-08T08:53:50.651 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:50.651 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.651 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.651 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:50.652 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451257+0000", 2024-05-08T08:53:50.652 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:50.652 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.652 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.652 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:50.652 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451258+0000", 2024-05-08T08:53:50.652 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7999999999999996e-07 2024-05-08T08:53:50.652 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.652 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.652 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:50.652 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451260+0000", 2024-05-08T08:53:50.652 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4609999999999998e-06 2024-05-08T08:53:50.652 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.652 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.652 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:50.653 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451267+0000", 2024-05-08T08:53:50.653 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.81e-06 2024-05-08T08:53:50.653 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.653 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.653 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:50.653 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451273+0000", 2024-05-08T08:53:50.653 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0859999999999998e-06 2024-05-08T08:53:50.653 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.653 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.653 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:50.653 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539718+0000", 2024-05-08T08:53:50.653 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088445059000000006 2024-05-08T08:53:50.653 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.653 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.653 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:50.654 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539834+0000", 2024-05-08T08:53:50.654 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000116426 2024-05-08T08:53:50.654 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.654 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.654 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:50.654 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539977+0000", 2024-05-08T08:53:50.654 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014260600000000001 2024-05-08T08:53:50.654 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.654 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:50.654 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.654 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.654 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.654 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:973 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 688128~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:50.654 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:53:50.655 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 41.148747671000002, 2024-05-08T08:53:50.655 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088918496, 2024-05-08T08:53:50.655 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:50.655 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:50.655 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:50.655 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:50.655 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:50.655 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:50.655 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 973 2024-05-08T08:53:50.655 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.655 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:50.655 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.655 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:50.655 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:53:50.655 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:50.656 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.656 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.656 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:50.656 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:53:50.656 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:50.656 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.656 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.656 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:50.656 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:53:50.656 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9099999999999999e-07 2024-05-08T08:53:50.656 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.656 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.656 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:50.656 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451282+0000", 2024-05-08T08:53:50.657 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1550000000000001e-06 2024-05-08T08:53:50.657 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.657 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.657 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:50.657 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451288+0000", 2024-05-08T08:53:50.657 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8889999999999999e-06 2024-05-08T08:53:50.657 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.657 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.657 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:50.657 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451294+0000", 2024-05-08T08:53:50.657 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0850000000000002e-06 2024-05-08T08:53:50.657 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.657 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.657 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:50.658 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540011+0000", 2024-05-08T08:53:50.658 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088717314000000005 2024-05-08T08:53:50.658 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.658 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.658 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:50.658 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540089+0000", 2024-05-08T08:53:50.658 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7438e-05 2024-05-08T08:53:50.658 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.658 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.658 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:50.658 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540198+0000", 2024-05-08T08:53:50.658 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000109224 2024-05-08T08:53:50.658 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.658 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:50.658 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.659 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.659 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.659 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:974 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 1179648~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:50.659 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451301+0000", 2024-05-08T08:53:50.659 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 41.148726038, 2024-05-08T08:53:50.659 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089105078000000004, 2024-05-08T08:53:50.659 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:50.659 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:50.659 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:50.659 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:50.659 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:50.659 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:50.659 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 974 2024-05-08T08:53:50.659 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.659 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:50.659 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.660 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:50.660 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451301+0000", 2024-05-08T08:53:50.660 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:50.660 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.660 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.660 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:50.660 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451301+0000", 2024-05-08T08:53:50.660 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:50.660 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.660 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.660 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:50.660 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451302+0000", 2024-05-08T08:53:50.660 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7800000000000002e-07 2024-05-08T08:53:50.660 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.660 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.661 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:50.661 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451304+0000", 2024-05-08T08:53:50.661 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.65e-06 2024-05-08T08:53:50.661 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.661 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.661 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:50.661 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451311+0000", 2024-05-08T08:53:50.661 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.472e-06 2024-05-08T08:53:50.661 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.661 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.661 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:50.661 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451316+0000", 2024-05-08T08:53:50.661 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9189999999999998e-06 2024-05-08T08:53:50.661 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.661 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.662 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:50.662 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540234+0000", 2024-05-08T08:53:50.662 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088917752000000003 2024-05-08T08:53:50.662 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.662 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.662 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:50.662 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540308+0000", 2024-05-08T08:53:50.662 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4758000000000005e-05 2024-05-08T08:53:50.662 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.662 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.662 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:50.662 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540406+0000", 2024-05-08T08:53:50.662 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8048999999999996e-05 2024-05-08T08:53:50.662 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.662 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:50.663 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.663 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.663 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.663 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:975 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 1671168~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:50.663 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:53:50.663 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 41.148683708, 2024-05-08T08:53:50.663 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089294139999999994, 2024-05-08T08:53:50.663 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:50.663 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:50.663 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:50.663 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:50.663 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:50.663 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:50.663 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 975 2024-05-08T08:53:50.663 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.663 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:50.664 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.664 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:50.664 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:53:50.664 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:50.664 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.664 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.664 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:50.664 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:53:50.664 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:50.664 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.664 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.664 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:50.664 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:53:50.664 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1299999999999999e-07 2024-05-08T08:53:50.664 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.665 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.665 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:50.665 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451348+0000", 2024-05-08T08:53:50.665 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.484e-06 2024-05-08T08:53:50.665 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.665 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.665 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:50.665 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451356+0000", 2024-05-08T08:53:50.665 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8499999999999994e-06 2024-05-08T08:53:50.665 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.665 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.665 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:50.665 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451366+0000", 2024-05-08T08:53:50.665 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.9850000000000001e-06 2024-05-08T08:53:50.665 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.666 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.666 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:50.666 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540448+0000", 2024-05-08T08:53:50.666 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089082254 2024-05-08T08:53:50.666 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.666 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.666 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:50.666 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540528+0000", 2024-05-08T08:53:50.666 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0234999999999993e-05 2024-05-08T08:53:50.666 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.666 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.666 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:50.666 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540638+0000", 2024-05-08T08:53:50.666 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010951900000000001 2024-05-08T08:53:50.666 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.667 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:50.667 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.667 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.667 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.667 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:976 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 2162688~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:50.667 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451375+0000", 2024-05-08T08:53:50.667 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 41.148651960999999, 2024-05-08T08:53:50.667 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089472745000000006, 2024-05-08T08:53:50.667 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:50.667 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:50.667 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:50.667 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:50.667 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:50.667 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:50.667 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 976 2024-05-08T08:53:50.668 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.668 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:50.668 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.668 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:50.668 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451375+0000", 2024-05-08T08:53:50.668 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:50.668 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.668 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.668 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:50.668 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451375+0000", 2024-05-08T08:53:50.668 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:50.668 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.668 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.668 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:50.668 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451376+0000", 2024-05-08T08:53:50.668 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2399999999999998e-07 2024-05-08T08:53:50.669 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.669 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.669 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:50.669 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451379+0000", 2024-05-08T08:53:50.669 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6359999999999998e-06 2024-05-08T08:53:50.669 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.669 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.669 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:50.669 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451387+0000", 2024-05-08T08:53:50.669 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9929999999999995e-06 2024-05-08T08:53:50.669 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.669 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.669 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:50.669 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451393+0000", 2024-05-08T08:53:50.669 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4699999999999999e-06 2024-05-08T08:53:50.670 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.670 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.670 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:50.670 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540677+0000", 2024-05-08T08:53:50.670 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089283504999999999 2024-05-08T08:53:50.670 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.670 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.670 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:50.670 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540743+0000", 2024-05-08T08:53:50.670 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6303999999999994e-05 2024-05-08T08:53:50.670 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.670 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.670 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:50.670 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540848+0000", 2024-05-08T08:53:50.670 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000105213 2024-05-08T08:53:50.671 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.671 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:50.671 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.671 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.671 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.671 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:977 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 2654208~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:50.671 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:53:50.671 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 41.148626528999998, 2024-05-08T08:53:50.671 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089698473000000001, 2024-05-08T08:53:50.671 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:50.671 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:50.671 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:50.671 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:50.671 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:50.671 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:50.672 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 977 2024-05-08T08:53:50.672 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.672 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:50.672 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.672 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:50.672 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:53:50.672 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:50.672 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.672 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.672 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:50.672 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:53:50.672 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:50.672 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.672 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.672 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:50.672 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:53:50.673 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2899999999999999e-07 2024-05-08T08:53:50.673 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.673 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.673 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:50.673 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451404+0000", 2024-05-08T08:53:50.673 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6649999999999999e-06 2024-05-08T08:53:50.673 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.673 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.673 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:50.673 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451410+0000", 2024-05-08T08:53:50.673 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3269999999999997e-06 2024-05-08T08:53:50.673 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.673 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.673 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:50.673 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451415+0000", 2024-05-08T08:53:50.674 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9529999999999998e-06 2024-05-08T08:53:50.674 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.674 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.674 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:50.674 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540912+0000", 2024-05-08T08:53:50.674 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089496950000000006 2024-05-08T08:53:50.674 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.674 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.674 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:50.674 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541000+0000", 2024-05-08T08:53:50.674 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7429000000000006e-05 2024-05-08T08:53:50.674 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.674 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.674 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:50.674 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541099+0000", 2024-05-08T08:53:50.675 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.9720000000000001e-05 2024-05-08T08:53:50.675 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.675 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:50.675 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.675 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.675 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.675 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:978 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 3145728~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:50.675 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:53:50.675 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 41.148605320999998, 2024-05-08T08:53:50.675 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089900635000000007, 2024-05-08T08:53:50.675 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:50.675 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:50.675 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:50.675 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:50.675 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:50.675 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:50.676 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 978 2024-05-08T08:53:50.676 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.676 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:50.676 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.676 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:50.676 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:53:50.676 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:50.676 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.676 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.676 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:50.676 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:53:50.676 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:50.676 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.676 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.676 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:50.676 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:53:50.677 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9400000000000001e-07 2024-05-08T08:53:50.677 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.677 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.677 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:50.677 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451425+0000", 2024-05-08T08:53:50.677 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2460000000000002e-06 2024-05-08T08:53:50.677 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.677 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.677 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:50.677 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451431+0000", 2024-05-08T08:53:50.677 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2419999999999997e-06 2024-05-08T08:53:50.677 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.677 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.677 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:50.677 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451436+0000", 2024-05-08T08:53:50.678 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.887e-06 2024-05-08T08:53:50.678 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.678 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.678 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:50.678 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541152+0000", 2024-05-08T08:53:50.678 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089716194999999999 2024-05-08T08:53:50.678 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.678 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.678 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:50.678 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541224+0000", 2024-05-08T08:53:50.678 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2416000000000006e-05 2024-05-08T08:53:50.678 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.678 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.678 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:50.679 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541323+0000", 2024-05-08T08:53:50.679 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8255e-05 2024-05-08T08:53:50.679 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.679 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:50.679 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.679 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.679 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.679 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:979 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 3637248~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:50.679 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451462+0000", 2024-05-08T08:53:50.679 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 41.148564872000001, 2024-05-08T08:53:50.679 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.090036441999999994, 2024-05-08T08:53:50.679 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:50.679 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:50.679 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:50.679 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:50.680 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:50.680 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:50.680 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 979 2024-05-08T08:53:50.680 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.680 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:50.680 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.680 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:50.680 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451462+0000", 2024-05-08T08:53:50.680 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:50.680 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.680 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.680 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:50.680 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451462+0000", 2024-05-08T08:53:50.680 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:50.680 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.680 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.681 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:50.681 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451463+0000", 2024-05-08T08:53:50.681 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4300000000000002e-07 2024-05-08T08:53:50.681 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.681 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.681 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:50.681 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451471+0000", 2024-05-08T08:53:50.681 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0800000000000006e-06 2024-05-08T08:53:50.681 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.681 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.681 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:50.681 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451481+0000", 2024-05-08T08:53:50.681 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.38e-06 2024-05-08T08:53:50.681 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.681 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.682 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:50.682 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451487+0000", 2024-05-08T08:53:50.682 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4590000000000003e-06 2024-05-08T08:53:50.682 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.682 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.682 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:50.682 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541350+0000", 2024-05-08T08:53:50.682 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089862599000000001 2024-05-08T08:53:50.682 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.682 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.682 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:50.682 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541407+0000", 2024-05-08T08:53:50.682 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7667999999999997e-05 2024-05-08T08:53:50.682 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.682 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.683 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:50.683 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541499+0000", 2024-05-08T08:53:50.683 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1513000000000003e-05 2024-05-08T08:53:50.683 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.683 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:50.683 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.683 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.683 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.683 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:980 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 4128768~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:50.683 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451496+0000", 2024-05-08T08:53:50.683 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 41.148531116000001, 2024-05-08T08:53:50.683 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.090258985, 2024-05-08T08:53:50.683 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:50.683 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:50.683 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:50.683 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:50.684 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:50.684 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:50.684 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 980 2024-05-08T08:53:50.684 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.684 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:50.684 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.684 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:50.684 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451496+0000", 2024-05-08T08:53:50.684 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:50.684 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.684 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.684 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:50.684 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451496+0000", 2024-05-08T08:53:50.684 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:50.684 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.684 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.684 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:50.685 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451497+0000", 2024-05-08T08:53:50.685 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7599999999999997e-07 2024-05-08T08:53:50.685 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.685 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.685 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:50.685 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451500+0000", 2024-05-08T08:53:50.685 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7930000000000002e-06 2024-05-08T08:53:50.685 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.685 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.685 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:50.685 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451506+0000", 2024-05-08T08:53:50.685 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6980000000000001e-06 2024-05-08T08:53:50.685 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.685 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.685 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:50.686 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451512+0000", 2024-05-08T08:53:50.686 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2490000000000001e-06 2024-05-08T08:53:50.686 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.686 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.686 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:50.686 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541550+0000", 2024-05-08T08:53:50.686 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.090037961999999999 2024-05-08T08:53:50.686 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.686 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.686 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:50.686 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541647+0000", 2024-05-08T08:53:50.686 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6924999999999998e-05 2024-05-08T08:53:50.686 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:50.686 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:50.687 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:50.687 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541755+0000", 2024-05-08T08:53:50.687 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000108682 2024-05-08T08:53:50.687 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.687 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:50.687 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.687 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:50.687 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:50.687 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:50.688 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:50.691+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:50.689 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:53:50.690 INFO:teuthology.orchestra.run.smithi196.stdout:4401 read 0x17a5600 thru 0x17b2a82 (0xd483 bytes) 2024-05-08T08:53:50.690 INFO:teuthology.orchestra.run.smithi196.stdout:4402 punch from 0xbc63600 to 0xbc67800, (0x4200 bytes) 2024-05-08T08:53:50.690 INFO:teuthology.orchestra.run.smithi196.stdout:4404 write 0xbc93a00 thru 0xbc98b2d (0x512e bytes) 2024-05-08T08:53:50.690 INFO:teuthology.orchestra.run.smithi196.stdout:4406 punch from 0x6d5e200 to 0x6d60200, (0x2000 bytes) 2024-05-08T08:53:50.690 INFO:teuthology.orchestra.run.smithi196.stdout:4407 punch from 0x1d12400 to 0x1d1ca00, (0xa600 bytes) 2024-05-08T08:53:50.690 INFO:teuthology.orchestra.run.smithi196.stdout:4408 write 0x78ac00 thru 0x78e073 (0x3474 bytes) 2024-05-08T08:53:50.690 INFO:teuthology.orchestra.run.smithi196.stdout:4410 read 0x9f90800 thru 0x9f9570d (0x4f0e bytes) 2024-05-08T08:53:50.705 INFO:teuthology.orchestra.run.smithi196.stdout:4412 write 0xd944c00 thru 0xd951337 (0xc738 bytes) 2024-05-08T08:53:50.706 INFO:teuthology.orchestra.run.smithi196.stdout:4413 read 0xc5fba00 thru 0xc5ff9cd (0x3fce bytes) 2024-05-08T08:53:50.706 INFO:teuthology.orchestra.run.smithi196.stdout:4414 read 0xc3f9600 thru 0xc4040d0 (0xaad1 bytes) 2024-05-08T08:53:50.707 INFO:teuthology.orchestra.run.smithi196.stdout:4415 read 0xc8aac00 thru 0xc8b9105 (0xe506 bytes) 2024-05-08T08:53:50.707 INFO:teuthology.orchestra.run.smithi196.stdout:4416 read 0xb0f9c00 thru 0xb100144 (0x6545 bytes) 2024-05-08T08:53:50.708 INFO:teuthology.orchestra.run.smithi196.stdout:4420 read 0x5658000 thru 0x56620ba (0xa0bb bytes) 2024-05-08T08:53:50.708 INFO:teuthology.orchestra.run.smithi196.stdout:4423 read 0x79e0600 thru 0x79edae6 (0xd4e7 bytes) 2024-05-08T08:53:50.708 INFO:teuthology.orchestra.run.smithi196.stdout:4427 write 0x6f4ae00 thru 0x6f51197 (0x6398 bytes) 2024-05-08T08:53:50.708 INFO:teuthology.orchestra.run.smithi196.stdout:4429 write 0x68c7800 thru 0x68d211a (0xa91b bytes) 2024-05-08T08:53:50.720 INFO:teuthology.orchestra.run.smithi196.stdout:4431 trunc from 0xd951400 to 0x392de00 2024-05-08T08:53:50.762 INFO:teuthology.orchestra.run.smithi196.stdout:4434 write 0x29b400 thru 0x2a8189 (0xcd8a bytes) 2024-05-08T08:53:50.764 INFO:teuthology.orchestra.run.smithi196.stdout:4436 read 0x13c9e00 thru 0x13d51dd (0xb3de bytes) 2024-05-08T08:53:50.789 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:50.792+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:50.805 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:53:50.805 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:53:50.806 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:53:50.806 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:53:50.810 INFO:teuthology.orchestra.run.smithi196.stdout:4438 write 0xe19bc00 thru 0xe1ab747 (0xfb48 bytes) 2024-05-08T08:53:50.811 INFO:teuthology.orchestra.run.smithi196.stdout:4439 write 0xcf83a00 thru 0xcf937ab (0xfdac bytes) 2024-05-08T08:53:50.811 INFO:teuthology.orchestra.run.smithi196.stdout:4440 write 0x2ac6a00 thru 0x2ac98dc (0x2edd bytes) 2024-05-08T08:53:50.815 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:53:50.816 INFO:teuthology.orchestra.run.smithi196.stdout:4441 write 0xd170800 thru 0xd17759b (0x6d9c bytes) 2024-05-08T08:53:50.831 INFO:teuthology.orchestra.run.smithi196.stdout:4443 trunc from 0xe1ab800 to 0xf53e00 2024-05-08T08:53:50.892 INFO:teuthology.orchestra.run.smithi196.stdout:4444 read 0xcffe00 thru 0xd07d56 (0x7f57 bytes) 2024-05-08T08:53:50.902 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:50.905+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:50.909 INFO:teuthology.orchestra.run.smithi196.stdout:4445 write 0x7a35c00 thru 0x7a404de (0xa8df bytes) 2024-05-08T08:53:50.910 INFO:teuthology.orchestra.run.smithi196.stdout:4447 punch from 0x57fce00 to 0x5807a00, (0xac00 bytes) 2024-05-08T08:53:50.911 INFO:teuthology.orchestra.run.smithi196.stdout:4448 punch from 0x52ca000 to 0x52d4800, (0xa800 bytes) 2024-05-08T08:53:50.911 INFO:teuthology.orchestra.run.smithi196.stdout:4449 read 0x95a000 thru 0x965102 (0xb103 bytes) 2024-05-08T08:53:50.911 INFO:teuthology.orchestra.run.smithi196.stdout:4450 read 0x910800 thru 0x919b23 (0x9324 bytes) 2024-05-08T08:53:50.912 INFO:teuthology.orchestra.run.smithi196.stdout:4451 read 0x4584a00 thru 0x459129e (0xc89f bytes) 2024-05-08T08:53:50.912 INFO:teuthology.orchestra.run.smithi196.stdout:4452 read 0x2e5f000 thru 0x2e6d882 (0xe883 bytes) 2024-05-08T08:53:50.913 INFO:teuthology.orchestra.run.smithi196.stdout:4453 read 0x5260200 thru 0x526c859 (0xc65a bytes) 2024-05-08T08:53:50.913 INFO:teuthology.orchestra.run.smithi196.stdout:4454 read 0x6ec5c00 thru 0x6ed437f (0xe780 bytes) 2024-05-08T08:53:50.930 INFO:teuthology.orchestra.run.smithi196.stdout:4455 write 0xe6d3c00 thru 0xe6d9b2e (0x5f2f bytes) 2024-05-08T08:53:50.936 INFO:teuthology.orchestra.run.smithi196.stdout:4457 read 0x813e200 thru 0x814d305 (0xf106 bytes) 2024-05-08T08:53:50.937 INFO:teuthology.orchestra.run.smithi196.stdout:4458 write 0xe4c2a00 thru 0xe4c6cd6 (0x42d7 bytes) 2024-05-08T08:53:50.937 INFO:teuthology.orchestra.run.smithi196.stdout:4460 write 0x52da000 thru 0x52e1fbf (0x7fc0 bytes) 2024-05-08T08:53:50.937 INFO:teuthology.orchestra.run.smithi196.stdout:4461 read 0x2e2ec00 thru 0x2e32917 (0x3d18 bytes) 2024-05-08T08:53:50.954 INFO:teuthology.orchestra.run.smithi196.stdout:4462 trunc from 0xe6d9c00 to 0x1c0e00 2024-05-08T08:53:50.977 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:53:50.977 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:53:50.977 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:53:50.977 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:53:50.978 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:53:50.984 INFO:teuthology.orchestra.run.smithi196.stdout:4466 punch from 0x1aa600 to 0x1b7200, (0xcc00 bytes) 2024-05-08T08:53:50.987 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:53:50.990 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:53:50.992+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:53:51.007 INFO:teuthology.orchestra.run.smithi196.stdout:4467 write 0x1bc0000 thru 0x1bc69f3 (0x69f4 bytes) 2024-05-08T08:53:51.034 INFO:teuthology.orchestra.run.smithi196.stdout:4468 write 0xe721600 thru 0xe726fcb (0x59cc bytes) 2024-05-08T08:53:51.035 INFO:teuthology.orchestra.run.smithi196.stdout:4469 read 0x73fe400 thru 0x740b7a4 (0xd3a5 bytes) 2024-05-08T08:53:51.035 INFO:teuthology.orchestra.run.smithi196.stdout:4471 punch from 0xd7abe00 to 0xd7b5e00, (0xa000 bytes) 2024-05-08T08:53:51.040 INFO:teuthology.orchestra.run.smithi196.stdout:4472 clone 31 order 22 su 1048576 sc 6 2024-05-08T08:53:51.091 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:51.094+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:51.136 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:53:51.136 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:53:51.137 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:53:51.137 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:53:51.137 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.137 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4621.0:9089 3.1f e33/18 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:head v 33'117, mlcod=31'115)", 2024-05-08T08:53:51.137 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:53:51.137 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 78.619048389, 2024-05-08T08:53:51.137 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.111244524, 2024-05-08T08:53:51.137 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:51.137 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:51.137 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:53:51.137 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:51.137 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.137 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:51.137 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:53:51.137 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:51.138 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.138 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.138 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:51.138 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:53:51.138 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:51.138 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.138 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.138 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:51.138 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:53:51.138 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5900000000000001e-07 2024-05-08T08:53:51.138 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.138 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.138 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:51.138 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539600+0000", 2024-05-08T08:53:51.139 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.019981364000000001 2024-05-08T08:53:51.139 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.139 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.139 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:51.139 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539624+0000", 2024-05-08T08:53:51.139 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4043e-05 2024-05-08T08:53:51.139 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.139 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.139 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:51.139 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539635+0000", 2024-05-08T08:53:51.139 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1511e-05 2024-05-08T08:53:51.139 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.139 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.139 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:51.140 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539728+0000", 2024-05-08T08:53:51.140 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2498999999999996e-05 2024-05-08T08:53:51.140 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.140 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.140 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:51.140 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539746+0000", 2024-05-08T08:53:51.140 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8289000000000001e-05 2024-05-08T08:53:51.140 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.140 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.140 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:53:51.140 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.630822+0000", 2024-05-08T08:53:51.140 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.091076283999999993 2024-05-08T08:53:51.140 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.140 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.140 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:51.140 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.630863+0000", 2024-05-08T08:53:51.141 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0374999999999999e-05 2024-05-08T08:53:51.141 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.141 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:51.141 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.141 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.141 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.141 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4795.0:230 3.1f e40/18 3:f8db1a56:::rbd_data.11402b26385.0000000000000000:head v 40'171, mlcod=39'168)", 2024-05-08T08:53:51.141 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:45.927773+0000", 2024-05-08T08:53:51.141 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.210893306000003, 2024-05-08T08:53:51.141 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045892405999999997, 2024-05-08T08:53:51.141 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:51.141 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:51.141 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:53:51.141 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:51.141 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.142 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:51.142 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.927773+0000", 2024-05-08T08:53:51.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:51.142 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.142 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.142 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:51.142 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.927773+0000", 2024-05-08T08:53:51.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:51.142 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.142 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.142 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:51.142 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.927774+0000", 2024-05-08T08:53:51.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0500000000000002e-07 2024-05-08T08:53:51.142 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.143 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.143 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:51.143 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.931581+0000", 2024-05-08T08:53:51.143 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0038071799999999999 2024-05-08T08:53:51.143 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.143 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.143 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:51.143 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.931599+0000", 2024-05-08T08:53:51.143 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7479e-05 2024-05-08T08:53:51.143 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.143 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.143 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:51.143 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.931624+0000", 2024-05-08T08:53:51.143 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5372999999999999e-05 2024-05-08T08:53:51.143 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.143 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.144 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:51.144 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.931698+0000", 2024-05-08T08:53:51.144 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4043999999999998e-05 2024-05-08T08:53:51.144 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.144 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.144 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:51.144 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.931725+0000", 2024-05-08T08:53:51.144 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6883e-05 2024-05-08T08:53:51.144 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.144 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.144 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:53:51.144 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.973623+0000", 2024-05-08T08:53:51.144 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.041898283000000001 2024-05-08T08:53:51.144 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.144 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.145 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:51.145 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.973666+0000", 2024-05-08T08:53:51.145 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2358999999999999e-05 2024-05-08T08:53:51.145 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.145 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:51.145 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.145 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.145 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.145 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4195.0:4 3.11 3:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-05-08T08:53:51.145 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:49.764654+0000", 2024-05-08T08:53:51.145 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 61.374012848, 2024-05-08T08:53:51.145 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.30714275499999999, 2024-05-08T08:53:51.145 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:51.145 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:51.145 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-05-08T08:53:51.146 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:51.146 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4195", 2024-05-08T08:53:51.146 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/59179521", 2024-05-08T08:53:51.146 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4 2024-05-08T08:53:51.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.146 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:51.146 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.146 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:51.146 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764654+0000", 2024-05-08T08:53:51.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:51.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.146 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.146 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:51.146 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764654+0000", 2024-05-08T08:53:51.147 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:51.147 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.147 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.147 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:51.147 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764655+0000", 2024-05-08T08:53:51.147 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.452e-06 2024-05-08T08:53:51.147 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.147 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.147 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:51.147 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764661+0000", 2024-05-08T08:53:51.147 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5369999999999997e-06 2024-05-08T08:53:51.147 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.147 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.147 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:51.147 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764671+0000", 2024-05-08T08:53:51.148 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.028e-05 2024-05-08T08:53:51.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.148 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.148 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:51.148 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764680+0000", 2024-05-08T08:53:51.148 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0629999999999995e-06 2024-05-08T08:53:51.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.148 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.148 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:51.148 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764753+0000", 2024-05-08T08:53:51.148 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2679e-05 2024-05-08T08:53:51.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.148 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.148 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-05-08T08:53:51.148 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764765+0000", 2024-05-08T08:53:51.148 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2415000000000001e-05 2024-05-08T08:53:51.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.149 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:51.149 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:50.071692+0000", 2024-05-08T08:53:51.149 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.306927324 2024-05-08T08:53:51.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.149 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:51.149 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:50.071796+0000", 2024-05-08T08:53:51.149 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104005 2024-05-08T08:53:51.149 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.149 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:51.149 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.149 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22681 3.15 3:a907109d:::rbd_data.1140d3fc54c.000000000000004f:18 [read 1204224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:51.150 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.097653+0000", 2024-05-08T08:53:51.150 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 19.04101322, 2024-05-08T08:53:51.150 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.041941285000000002, 2024-05-08T08:53:51.150 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:51.150 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:51.150 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:51.150 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:51.150 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:51.150 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:51.150 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22681 2024-05-08T08:53:51.150 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.150 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:51.150 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.150 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:51.151 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097653+0000", 2024-05-08T08:53:51.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:51.151 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.151 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.151 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:51.151 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097653+0000", 2024-05-08T08:53:51.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:51.151 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.151 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.151 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:51.151 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097654+0000", 2024-05-08T08:53:51.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9400000000000001e-07 2024-05-08T08:53:51.151 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.151 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.151 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:51.152 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097655+0000", 2024-05-08T08:53:51.152 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6220000000000001e-06 2024-05-08T08:53:51.152 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.152 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.152 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:51.152 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097661+0000", 2024-05-08T08:53:51.152 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5210000000000002e-06 2024-05-08T08:53:51.152 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.152 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.152 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:51.152 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097665+0000", 2024-05-08T08:53:51.152 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3170000000000003e-06 2024-05-08T08:53:51.152 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.152 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.152 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:51.152 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139509+0000", 2024-05-08T08:53:51.153 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.041843475999999998 2024-05-08T08:53:51.153 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.153 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.153 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:51.153 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139554+0000", 2024-05-08T08:53:51.153 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5478000000000001e-05 2024-05-08T08:53:51.153 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.153 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.153 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:51.153 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139595+0000", 2024-05-08T08:53:51.153 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0577000000000001e-05 2024-05-08T08:53:51.153 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.153 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:51.153 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.153 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.153 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.154 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22682 3.15 3:a907109d:::rbd_data.1140d3fc54c.000000000000004f:18 [read 1269760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:51.154 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.097670+0000", 2024-05-08T08:53:51.154 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 19.040996528000001, 2024-05-08T08:53:51.154 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042031394, 2024-05-08T08:53:51.154 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:51.154 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:51.154 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:51.154 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:51.154 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:51.154 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:51.154 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22682 2024-05-08T08:53:51.154 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.154 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:51.155 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.155 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:51.155 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097670+0000", 2024-05-08T08:53:51.155 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:51.155 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.155 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.155 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:51.155 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097670+0000", 2024-05-08T08:53:51.155 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:51.155 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.155 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.155 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:51.155 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097670+0000", 2024-05-08T08:53:51.155 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1800000000000002e-07 2024-05-08T08:53:51.155 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.156 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.156 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:51.156 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097672+0000", 2024-05-08T08:53:51.156 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6929999999999999e-06 2024-05-08T08:53:51.156 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.156 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.156 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:51.156 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097677+0000", 2024-05-08T08:53:51.156 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2480000000000004e-06 2024-05-08T08:53:51.156 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.156 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.156 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:51.156 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097682+0000", 2024-05-08T08:53:51.156 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4290000000000003e-06 2024-05-08T08:53:51.156 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.157 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.157 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:51.157 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139620+0000", 2024-05-08T08:53:51.157 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.041938256 2024-05-08T08:53:51.157 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.157 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.157 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:51.157 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139662+0000", 2024-05-08T08:53:51.157 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1727999999999998e-05 2024-05-08T08:53:51.157 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.157 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.157 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:51.157 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139701+0000", 2024-05-08T08:53:51.157 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9721999999999997e-05 2024-05-08T08:53:51.157 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.158 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:51.158 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.158 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.158 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.158 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22683 3.15 3:a907109d:::rbd_data.1140d3fc54c.000000000000004f:18 [read 1335296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:51.158 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.097686+0000", 2024-05-08T08:53:51.158 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 19.040980174000001, 2024-05-08T08:53:51.158 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042114716000000003, 2024-05-08T08:53:51.158 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:51.158 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:51.158 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:51.158 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:51.158 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:51.158 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:51.158 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22683 2024-05-08T08:53:51.159 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.159 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:51.159 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.159 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:51.159 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097686+0000", 2024-05-08T08:53:51.159 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:51.159 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.159 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.159 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:51.159 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097686+0000", 2024-05-08T08:53:51.159 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:51.159 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.159 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.159 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:51.159 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097687+0000", 2024-05-08T08:53:51.159 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3999999999999997e-07 2024-05-08T08:53:51.160 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.160 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.160 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:51.160 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097688+0000", 2024-05-08T08:53:51.160 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6789999999999999e-06 2024-05-08T08:53:51.160 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.160 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.160 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:51.160 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097693+0000", 2024-05-08T08:53:51.160 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.045e-06 2024-05-08T08:53:51.160 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.160 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.160 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:51.160 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097697+0000", 2024-05-08T08:53:51.160 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0260000000000001e-06 2024-05-08T08:53:51.161 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.161 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.161 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:51.161 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139725+0000", 2024-05-08T08:53:51.161 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042027737000000003 2024-05-08T08:53:51.161 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.161 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.161 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:51.161 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139764+0000", 2024-05-08T08:53:51.161 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8668999999999999e-05 2024-05-08T08:53:51.161 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.161 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.161 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:51.161 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139801+0000", 2024-05-08T08:53:51.161 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7219999999999999e-05 2024-05-08T08:53:51.162 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.162 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:51.162 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.162 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.162 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.162 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22684 3.15 3:a907109d:::rbd_data.1140d3fc54c.000000000000004f:18 [read 1400832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:51.162 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.097703+0000", 2024-05-08T08:53:51.162 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 19.040963820999998, 2024-05-08T08:53:51.162 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042198929000000003, 2024-05-08T08:53:51.162 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:51.162 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:51.162 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:51.162 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:51.162 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:51.163 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:51.163 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22684 2024-05-08T08:53:51.163 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.163 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:51.163 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.163 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:51.163 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097703+0000", 2024-05-08T08:53:51.163 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:51.163 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.163 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.163 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:51.163 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097703+0000", 2024-05-08T08:53:51.163 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:51.163 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.163 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.164 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:51.164 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097703+0000", 2024-05-08T08:53:51.164 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4400000000000001e-07 2024-05-08T08:53:51.164 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.164 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.164 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:51.164 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097705+0000", 2024-05-08T08:53:51.164 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9199999999999998e-06 2024-05-08T08:53:51.164 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.164 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.164 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:51.164 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097711+0000", 2024-05-08T08:53:51.164 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5559999999999998e-06 2024-05-08T08:53:51.164 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.165 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.165 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:51.165 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097715+0000", 2024-05-08T08:53:51.165 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3159999999999998e-06 2024-05-08T08:53:51.165 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.165 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.165 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:51.165 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139825+0000", 2024-05-08T08:53:51.165 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042110284999999997 2024-05-08T08:53:51.165 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.165 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.165 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:51.165 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139864+0000", 2024-05-08T08:53:51.165 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8980000000000003e-05 2024-05-08T08:53:51.166 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.166 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.166 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:51.166 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139902+0000", 2024-05-08T08:53:51.166 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7527999999999997e-05 2024-05-08T08:53:51.166 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.166 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:51.166 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.166 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.166 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.166 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22685 3.15 3:a907109d:::rbd_data.1140d3fc54c.000000000000004f:18 [read 1466368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:51.166 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.097721+0000", 2024-05-08T08:53:51.166 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 19.040945164, 2024-05-08T08:53:51.166 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042287791999999998, 2024-05-08T08:53:51.166 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:51.167 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:51.167 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:51.167 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:51.167 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:51.167 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:51.167 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22685 2024-05-08T08:53:51.167 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.167 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:51.167 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.167 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:51.167 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097721+0000", 2024-05-08T08:53:51.167 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:51.167 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.167 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.167 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:51.168 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097721+0000", 2024-05-08T08:53:51.168 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:51.168 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.168 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.168 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:51.168 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097722+0000", 2024-05-08T08:53:51.168 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2500000000000001e-07 2024-05-08T08:53:51.168 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.168 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.168 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:51.168 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097724+0000", 2024-05-08T08:53:51.168 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.835e-06 2024-05-08T08:53:51.168 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.168 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.168 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:51.169 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097729+0000", 2024-05-08T08:53:51.169 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5029999999999997e-06 2024-05-08T08:53:51.169 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.169 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.169 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:51.169 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097733+0000", 2024-05-08T08:53:51.169 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3309999999999997e-06 2024-05-08T08:53:51.169 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.169 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.169 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:51.169 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139923+0000", 2024-05-08T08:53:51.169 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042189988999999997 2024-05-08T08:53:51.169 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.169 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.169 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:51.170 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139970+0000", 2024-05-08T08:53:51.170 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6776000000000001e-05 2024-05-08T08:53:51.170 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.170 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.170 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:51.170 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140009+0000", 2024-05-08T08:53:51.170 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9032999999999997e-05 2024-05-08T08:53:51.170 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.170 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:51.170 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.170 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.170 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.170 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22686 3.15 3:a907109d:::rbd_data.1140d3fc54c.000000000000004f:18 [read 1531904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:51.170 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.097739+0000", 2024-05-08T08:53:51.170 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 19.040927164999999, 2024-05-08T08:53:51.171 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042371008000000002, 2024-05-08T08:53:51.171 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:51.171 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:51.171 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:51.171 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:51.171 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:51.171 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:51.171 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22686 2024-05-08T08:53:51.171 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.171 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:51.171 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.171 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:51.171 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097739+0000", 2024-05-08T08:53:51.171 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:51.171 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.172 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.172 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:51.172 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097739+0000", 2024-05-08T08:53:51.172 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:51.172 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.172 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.172 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:51.172 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097740+0000", 2024-05-08T08:53:51.172 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8599999999999999e-07 2024-05-08T08:53:51.172 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.172 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.172 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:51.172 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097742+0000", 2024-05-08T08:53:51.172 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9410000000000001e-06 2024-05-08T08:53:51.172 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.173 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.173 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:51.173 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097747+0000", 2024-05-08T08:53:51.173 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2079999999999999e-06 2024-05-08T08:53:51.173 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.173 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.173 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:51.173 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097751+0000", 2024-05-08T08:53:51.173 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2250000000000002e-06 2024-05-08T08:53:51.173 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.173 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.173 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:51.173 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140032+0000", 2024-05-08T08:53:51.173 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042281244000000003 2024-05-08T08:53:51.174 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.174 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.174 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:51.174 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140074+0000", 2024-05-08T08:53:51.174 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1316999999999998e-05 2024-05-08T08:53:51.174 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.174 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.174 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:51.174 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140110+0000", 2024-05-08T08:53:51.174 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6687000000000003e-05 2024-05-08T08:53:51.174 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.174 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:51.174 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.174 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.175 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.175 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22716 3.15 3:a88fdee9:::rbd_data.1140d3fc54c.000000000000004d:18 [read 1097728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:51.175 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.098153+0000", 2024-05-08T08:53:51.175 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 19.040513458, 2024-05-08T08:53:51.175 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042058535000000001, 2024-05-08T08:53:51.175 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:51.175 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:51.175 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:51.175 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:51.175 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:51.175 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:51.175 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22716 2024-05-08T08:53:51.175 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.175 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:51.175 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.176 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:51.176 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098153+0000", 2024-05-08T08:53:51.176 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:51.176 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.176 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.176 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:51.176 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098153+0000", 2024-05-08T08:53:51.176 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:51.176 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.176 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.176 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:51.176 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098154+0000", 2024-05-08T08:53:51.176 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.13e-07 2024-05-08T08:53:51.176 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.177 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.177 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:51.177 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098161+0000", 2024-05-08T08:53:51.177 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7360000000000001e-06 2024-05-08T08:53:51.177 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.177 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.177 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:51.177 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098171+0000", 2024-05-08T08:53:51.177 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1039999999999996e-06 2024-05-08T08:53:51.177 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.177 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.177 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:51.177 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098177+0000", 2024-05-08T08:53:51.177 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5760000000000002e-06 2024-05-08T08:53:51.177 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.178 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.178 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:51.178 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140135+0000", 2024-05-08T08:53:51.178 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.041957842000000002 2024-05-08T08:53:51.178 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.178 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.178 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:51.178 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140174+0000", 2024-05-08T08:53:51.178 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8902999999999997e-05 2024-05-08T08:53:51.178 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.178 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.178 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:51.178 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140212+0000", 2024-05-08T08:53:51.179 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7761e-05 2024-05-08T08:53:51.179 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.179 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:51.179 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.179 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.179 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.179 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22717 3.15 3:a88fdee9:::rbd_data.1140d3fc54c.000000000000004d:18 [read 1163264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:51.179 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.098188+0000", 2024-05-08T08:53:51.179 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 19.040478736000001, 2024-05-08T08:53:51.179 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042124907000000003, 2024-05-08T08:53:51.179 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:51.179 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:51.179 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:51.179 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:51.179 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:51.180 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:51.180 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22717 2024-05-08T08:53:51.180 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.180 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:51.180 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.180 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:51.180 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098188+0000", 2024-05-08T08:53:51.180 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:51.180 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.180 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.180 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:51.180 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098188+0000", 2024-05-08T08:53:51.180 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:51.180 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.180 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.181 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:51.181 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098188+0000", 2024-05-08T08:53:51.181 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3799999999999998e-07 2024-05-08T08:53:51.181 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.181 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.181 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:51.181 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098190+0000", 2024-05-08T08:53:51.181 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.114e-06 2024-05-08T08:53:51.181 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.181 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.181 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:51.181 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098197+0000", 2024-05-08T08:53:51.181 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9959999999999996e-06 2024-05-08T08:53:51.181 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.181 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.181 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:51.182 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098202+0000", 2024-05-08T08:53:51.182 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9110000000000001e-06 2024-05-08T08:53:51.182 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.182 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.182 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:51.182 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140235+0000", 2024-05-08T08:53:51.182 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042032913999999998 2024-05-08T08:53:51.182 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.182 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.182 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:51.182 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140275+0000", 2024-05-08T08:53:51.182 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9564999999999998e-05 2024-05-08T08:53:51.182 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.182 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.183 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:51.183 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140313+0000", 2024-05-08T08:53:51.183 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7969000000000002e-05 2024-05-08T08:53:51.183 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.183 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:51.183 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.183 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.183 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.183 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22718 3.15 3:a88fdee9:::rbd_data.1140d3fc54c.000000000000004d:18 [read 1228800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:51.183 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.098209+0000", 2024-05-08T08:53:51.183 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 19.040457589999999, 2024-05-08T08:53:51.183 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.04220409, 2024-05-08T08:53:51.183 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:51.183 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:51.183 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:51.183 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:51.184 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:51.184 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:51.184 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22718 2024-05-08T08:53:51.184 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.184 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:51.184 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.184 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:51.184 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098209+0000", 2024-05-08T08:53:51.184 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:51.184 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.184 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.184 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:51.184 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098209+0000", 2024-05-08T08:53:51.184 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:51.184 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.184 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.185 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:51.185 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098209+0000", 2024-05-08T08:53:51.185 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9299999999999999e-07 2024-05-08T08:53:51.185 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.185 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.185 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:51.185 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098212+0000", 2024-05-08T08:53:51.185 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.21e-06 2024-05-08T08:53:51.185 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.185 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.185 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:51.185 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098219+0000", 2024-05-08T08:53:51.185 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0269999999999998e-06 2024-05-08T08:53:51.185 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.185 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.185 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:51.186 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098223+0000", 2024-05-08T08:53:51.186 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7319999999999998e-06 2024-05-08T08:53:51.186 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.186 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.186 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:51.186 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140338+0000", 2024-05-08T08:53:51.186 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042114390000000002 2024-05-08T08:53:51.186 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.186 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.186 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:51.186 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140376+0000", 2024-05-08T08:53:51.186 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8414999999999997e-05 2024-05-08T08:53:51.186 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.187 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.187 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:51.187 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140413+0000", 2024-05-08T08:53:51.187 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6922999999999999e-05 2024-05-08T08:53:51.187 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.187 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:51.187 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.187 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.187 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.187 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22719 3.15 3:a88fdee9:::rbd_data.1140d3fc54c.000000000000004d:18 [read 1294336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:51.187 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.098229+0000", 2024-05-08T08:53:51.187 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 19.040437739000001, 2024-05-08T08:53:51.187 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042283016, 2024-05-08T08:53:51.187 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:51.187 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:51.187 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:51.188 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:51.188 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:51.188 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:51.188 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22719 2024-05-08T08:53:51.188 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.188 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:51.188 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.188 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:51.188 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098229+0000", 2024-05-08T08:53:51.188 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:51.188 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.188 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.188 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:51.188 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098229+0000", 2024-05-08T08:53:51.188 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:51.189 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.189 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.189 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:51.189 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098229+0000", 2024-05-08T08:53:51.189 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:53:51.189 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.189 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.189 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:51.189 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098231+0000", 2024-05-08T08:53:51.189 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.81e-06 2024-05-08T08:53:51.189 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.189 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.189 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:51.189 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098237+0000", 2024-05-08T08:53:51.190 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3389999999999998e-06 2024-05-08T08:53:51.190 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.190 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.190 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:51.190 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098243+0000", 2024-05-08T08:53:51.190 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.648e-06 2024-05-08T08:53:51.190 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.190 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.190 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:51.190 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140436+0000", 2024-05-08T08:53:51.190 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042192738 2024-05-08T08:53:51.190 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.190 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.190 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:51.191 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140475+0000", 2024-05-08T08:53:51.191 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9279999999999997e-05 2024-05-08T08:53:51.191 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.191 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.191 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:51.191 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140512+0000", 2024-05-08T08:53:51.191 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6847000000000001e-05 2024-05-08T08:53:51.191 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.191 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:51.191 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.191 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.191 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.191 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22720 3.15 3:a88fdee9:::rbd_data.1140d3fc54c.000000000000004d:18 [read 1359872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:51.191 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.098248+0000", 2024-05-08T08:53:51.191 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 19.040418228, 2024-05-08T08:53:51.192 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042360800999999997, 2024-05-08T08:53:51.192 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:51.192 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:51.192 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:51.192 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:51.192 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:51.192 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:51.192 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22720 2024-05-08T08:53:51.192 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.192 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:51.192 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.192 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:51.192 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098248+0000", 2024-05-08T08:53:51.192 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:51.193 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.193 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.193 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:51.193 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098248+0000", 2024-05-08T08:53:51.193 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:51.193 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.193 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.193 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:51.193 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098491+0000", 2024-05-08T08:53:51.193 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000242659 2024-05-08T08:53:51.193 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.193 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.193 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:51.193 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098495+0000", 2024-05-08T08:53:51.194 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.799e-06 2024-05-08T08:53:51.194 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.194 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.194 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:51.194 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098505+0000", 2024-05-08T08:53:51.194 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7079999999999992e-06 2024-05-08T08:53:51.194 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.194 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.194 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:51.194 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098511+0000", 2024-05-08T08:53:51.194 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4370000000000004e-06 2024-05-08T08:53:51.194 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.194 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.194 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:51.195 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140533+0000", 2024-05-08T08:53:51.195 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042022371000000003 2024-05-08T08:53:51.195 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.195 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.195 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:51.195 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140571+0000", 2024-05-08T08:53:51.195 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7700000000000002e-05 2024-05-08T08:53:51.195 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.195 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.195 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:51.195 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140609+0000", 2024-05-08T08:53:51.195 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8127000000000003e-05 2024-05-08T08:53:51.195 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.195 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:51.196 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.196 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.196 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.196 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22721 3.15 3:a88fdee9:::rbd_data.1140d3fc54c.000000000000004d:18 [read 1425408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:51.196 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.098522+0000", 2024-05-08T08:53:51.196 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 19.040144604000002, 2024-05-08T08:53:51.196 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042185779999999999, 2024-05-08T08:53:51.196 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:51.196 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:51.196 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:51.196 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:51.196 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:51.196 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:51.196 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22721 2024-05-08T08:53:51.196 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.197 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:51.197 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.197 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:51.197 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098522+0000", 2024-05-08T08:53:51.197 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:51.197 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.197 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.197 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:51.197 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098522+0000", 2024-05-08T08:53:51.197 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:51.197 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.197 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.197 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:51.197 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098708+0000", 2024-05-08T08:53:51.197 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00018629700000000001 2024-05-08T08:53:51.197 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.198 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.198 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:51.198 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098719+0000", 2024-05-08T08:53:51.198 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0992000000000001e-05 2024-05-08T08:53:51.198 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.198 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.198 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:51.198 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098730+0000", 2024-05-08T08:53:51.198 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0431999999999999e-05 2024-05-08T08:53:51.198 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.198 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.198 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:51.198 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098737+0000", 2024-05-08T08:53:51.198 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.063e-06 2024-05-08T08:53:51.199 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.199 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.199 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:51.199 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140631+0000", 2024-05-08T08:53:51.199 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.041894503999999999 2024-05-08T08:53:51.199 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.199 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.199 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:51.199 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140671+0000", 2024-05-08T08:53:51.199 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9780999999999997e-05 2024-05-08T08:53:51.199 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.199 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.199 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:51.199 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140708+0000", 2024-05-08T08:53:51.199 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6711000000000001e-05 2024-05-08T08:53:51.200 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.200 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:51.200 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.200 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.200 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.200 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22722 3.15 3:a88fdee9:::rbd_data.1140d3fc54c.000000000000004d:18 [read 1490944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:51.200 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.098753+0000", 2024-05-08T08:53:51.200 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 19.039913599999998, 2024-05-08T08:53:51.200 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042053944000000003, 2024-05-08T08:53:51.200 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:51.200 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:51.200 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:51.200 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:51.200 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:51.201 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:51.201 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22722 2024-05-08T08:53:51.201 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.201 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:51.201 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.201 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:51.201 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098753+0000", 2024-05-08T08:53:51.201 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:51.201 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.201 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.201 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:51.201 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098753+0000", 2024-05-08T08:53:51.201 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:51.201 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.201 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.202 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:51.202 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098753+0000", 2024-05-08T08:53:51.202 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1399999999999997e-07 2024-05-08T08:53:51.202 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.202 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.202 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:51.202 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098756+0000", 2024-05-08T08:53:51.202 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9780000000000001e-06 2024-05-08T08:53:51.202 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.202 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.202 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:51.202 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098764+0000", 2024-05-08T08:53:51.202 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4340000000000003e-06 2024-05-08T08:53:51.202 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.203 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.203 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:51.203 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098770+0000", 2024-05-08T08:53:51.203 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8080000000000001e-06 2024-05-08T08:53:51.203 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.203 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.203 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:51.203 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140731+0000", 2024-05-08T08:53:51.203 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.041961131999999998 2024-05-08T08:53:51.203 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.203 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.203 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:51.203 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140769+0000", 2024-05-08T08:53:51.204 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.773e-05 2024-05-08T08:53:51.204 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.204 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.204 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:51.204 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140807+0000", 2024-05-08T08:53:51.204 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8347999999999999e-05 2024-05-08T08:53:51.204 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.204 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:51.204 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.204 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.204 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.204 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22723 3.15 3:a88fdee9:::rbd_data.1140d3fc54c.000000000000004d:18 [read 1556480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:51.204 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.098778+0000", 2024-05-08T08:53:51.204 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 19.039888827999999, 2024-05-08T08:53:51.204 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042128577, 2024-05-08T08:53:51.205 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:51.205 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:51.205 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:51.205 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:51.205 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:51.205 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:51.205 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22723 2024-05-08T08:53:51.205 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.205 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:51.205 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.205 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:51.205 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098778+0000", 2024-05-08T08:53:51.205 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:51.205 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.205 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.205 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:51.205 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098778+0000", 2024-05-08T08:53:51.206 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:51.206 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.206 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.206 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:51.206 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098834+0000", 2024-05-08T08:53:51.206 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5788000000000002e-05 2024-05-08T08:53:51.206 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.206 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.206 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:51.206 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098837+0000", 2024-05-08T08:53:51.206 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3510000000000002e-06 2024-05-08T08:53:51.206 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.206 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.206 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:51.207 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098845+0000", 2024-05-08T08:53:51.207 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5490000000000006e-06 2024-05-08T08:53:51.207 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.207 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.207 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:51.207 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098861+0000", 2024-05-08T08:53:51.207 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5233e-05 2024-05-08T08:53:51.207 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.207 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.207 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:51.207 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140829+0000", 2024-05-08T08:53:51.207 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.04196863 2024-05-08T08:53:51.207 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.207 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.207 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:51.207 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140869+0000", 2024-05-08T08:53:51.207 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9461e-05 2024-05-08T08:53:51.208 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.208 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.208 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:51.208 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140906+0000", 2024-05-08T08:53:51.208 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7564999999999997e-05 2024-05-08T08:53:51.208 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.208 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:51.208 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.208 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.208 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.208 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22732 3.15 3:a907109d:::rbd_data.1140d3fc54c.000000000000004f:18 [read 1097728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:51.208 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.098871+0000", 2024-05-08T08:53:51.208 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 19.039795347999998, 2024-05-08T08:53:51.208 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042141517000000003, 2024-05-08T08:53:51.208 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:51.208 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:51.208 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:51.209 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:51.209 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:51.209 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:51.209 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22732 2024-05-08T08:53:51.209 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.209 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:51.209 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.209 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:51.209 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098871+0000", 2024-05-08T08:53:51.209 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:51.209 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.209 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.209 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:51.209 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098871+0000", 2024-05-08T08:53:51.209 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:51.209 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.210 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.210 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:51.210 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.099940+0000", 2024-05-08T08:53:51.210 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0010683279999999999 2024-05-08T08:53:51.210 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.210 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.210 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:51.210 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.099944+0000", 2024-05-08T08:53:51.210 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9339999999999997e-06 2024-05-08T08:53:51.210 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.210 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.210 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:51.210 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.099956+0000", 2024-05-08T08:53:51.210 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1970999999999999e-05 2024-05-08T08:53:51.210 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.211 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.211 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:51.211 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.099968+0000", 2024-05-08T08:53:51.211 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.189e-05 2024-05-08T08:53:51.211 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.211 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.211 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:51.211 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140930+0000", 2024-05-08T08:53:51.211 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.040961806000000003 2024-05-08T08:53:51.211 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.211 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.211 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:51.211 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140974+0000", 2024-05-08T08:53:51.211 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3918999999999997e-05 2024-05-08T08:53:51.211 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.212 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.212 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:51.212 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.141013+0000", 2024-05-08T08:53:51.212 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8668999999999999e-05 2024-05-08T08:53:51.212 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.212 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:51.212 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.212 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.212 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.212 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22765 3.15 3:a9c637ae:::rbd_data.114071bcf828.000000000000013f:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:51.212 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.100902+0000", 2024-05-08T08:53:51.212 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 19.037764552999999, 2024-05-08T08:53:51.212 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042471271999999997, 2024-05-08T08:53:51.212 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:51.212 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:51.213 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:51.213 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:51.213 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:51.213 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:51.213 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22765 2024-05-08T08:53:51.213 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.213 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:51.213 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.213 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:51.213 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.100902+0000", 2024-05-08T08:53:51.213 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:51.213 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.213 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.213 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:51.213 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.100902+0000", 2024-05-08T08:53:51.214 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:51.214 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.214 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.214 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:51.214 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.100902+0000", 2024-05-08T08:53:51.214 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.39e-07 2024-05-08T08:53:51.214 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.214 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.214 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:51.214 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.100904+0000", 2024-05-08T08:53:51.214 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6479999999999999e-06 2024-05-08T08:53:51.214 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.214 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.214 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:51.214 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.100910+0000", 2024-05-08T08:53:51.215 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1399999999999997e-06 2024-05-08T08:53:51.215 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.215 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.215 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:51.215 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.100916+0000", 2024-05-08T08:53:51.215 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6999999999999996e-06 2024-05-08T08:53:51.215 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.215 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.215 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:51.215 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.141733+0000", 2024-05-08T08:53:51.215 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.040817271000000002 2024-05-08T08:53:51.215 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.215 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.215 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:51.216 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.141812+0000", 2024-05-08T08:53:51.216 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9270999999999994e-05 2024-05-08T08:53:51.216 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.216 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.216 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:51.216 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.143373+0000", 2024-05-08T08:53:51.216 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001560903 2024-05-08T08:53:51.216 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.216 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:51.216 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.216 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.216 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.216 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:23031 3.15 3:ac9fb9cd:::rbd_data.114071bcf828.00000000000000f7:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:51.216 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.113562+0000", 2024-05-08T08:53:51.216 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 19.025104793000001, 2024-05-08T08:53:51.217 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042560938999999999, 2024-05-08T08:53:51.217 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:51.217 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:51.217 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:53:51.217 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:51.217 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:51.217 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:51.217 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 23031 2024-05-08T08:53:51.217 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.217 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:51.217 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.217 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:51.217 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.113562+0000", 2024-05-08T08:53:51.217 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:51.217 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.218 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.218 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:51.218 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.113562+0000", 2024-05-08T08:53:51.218 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:51.218 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.218 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.218 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:51.218 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.113938+0000", 2024-05-08T08:53:51.218 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000376401 2024-05-08T08:53:51.218 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.218 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.218 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:51.218 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.113943+0000", 2024-05-08T08:53:51.218 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0529999999999997e-06 2024-05-08T08:53:51.219 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.219 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.219 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:51.219 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.113951+0000", 2024-05-08T08:53:51.219 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8569999999999995e-06 2024-05-08T08:53:51.219 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.219 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.219 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:51.219 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.113959+0000", 2024-05-08T08:53:51.219 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4710000000000001e-06 2024-05-08T08:53:51.219 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.219 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.219 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:51.219 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.153955+0000", 2024-05-08T08:53:51.220 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.039996747999999999 2024-05-08T08:53:51.220 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.220 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.220 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:51.220 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.154014+0000", 2024-05-08T08:53:51.220 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8464999999999997e-05 2024-05-08T08:53:51.220 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.220 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.220 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-05-08T08:53:51.220 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.154148+0000", 2024-05-08T08:53:51.220 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00013428600000000001 2024-05-08T08:53:51.220 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.221 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.221 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:53:51.221 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.155390+0000", 2024-05-08T08:53:51.221 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001241792 2024-05-08T08:53:51.221 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.221 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.221 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:53:51.221 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.156068+0000", 2024-05-08T08:53:51.221 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00067823700000000005 2024-05-08T08:53:51.221 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.221 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.221 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:53:51.221 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.156102+0000", 2024-05-08T08:53:51.221 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3812000000000001e-05 2024-05-08T08:53:51.221 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.221 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.222 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:51.222 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.156123+0000", 2024-05-08T08:53:51.222 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0817000000000001e-05 2024-05-08T08:53:51.222 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.222 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:51.222 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.222 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.222 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:51.222 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:53:51.224 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:53:51.224 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:51.226+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:51.325 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:51.328+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:51.329 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:53:51.329 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:53:51.329 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:53:51.329 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:53:51.338 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:53:51.426 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:51.429+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:51.484 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:53:51.484 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:53:51.484 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:53:51.484 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:53:51.484 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:53:51.493 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:53:51.496 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T08:53:51.536 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:51.539+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:51.579 INFO:teuthology.orchestra.run.smithi059.stderr:nodeep-scrub is set 2024-05-08T08:53:51.606 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:53:51.606 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:53:51.606 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:53:51.607 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:53:51.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.607 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22071 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 765952~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:51.607 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857643+0000", 2024-05-08T08:53:51.607 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 83.749793952999994, 2024-05-08T08:53:51.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047106528000000002, 2024-05-08T08:53:51.607 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:51.608 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:51.608 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:51.608 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:51.608 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:51.608 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:51.608 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22071 2024-05-08T08:53:51.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.609 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:51.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.609 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:51.609 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857643+0000", 2024-05-08T08:53:51.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:51.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.609 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:51.610 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857643+0000", 2024-05-08T08:53:51.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:51.610 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.610 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:51.610 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857644+0000", 2024-05-08T08:53:51.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2000000000000001e-07 2024-05-08T08:53:51.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.611 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:51.611 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857646+0000", 2024-05-08T08:53:51.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0059999999999999e-06 2024-05-08T08:53:51.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.611 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:51.612 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857651+0000", 2024-05-08T08:53:51.612 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.84e-06 2024-05-08T08:53:51.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.612 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:51.612 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857657+0000", 2024-05-08T08:53:51.612 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5620000000000003e-06 2024-05-08T08:53:51.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.613 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:51.613 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904620+0000", 2024-05-08T08:53:51.613 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046962661000000003 2024-05-08T08:53:51.613 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.613 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.613 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:51.613 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904678+0000", 2024-05-08T08:53:51.613 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8656999999999998e-05 2024-05-08T08:53:51.614 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.614 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.614 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:51.614 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904750+0000", 2024-05-08T08:53:51.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1482000000000004e-05 2024-05-08T08:53:51.614 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.614 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:51.615 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.615 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22073 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 782336~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:51.615 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:53:51.615 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 83.749769944999997, 2024-05-08T08:53:51.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047244234000000003, 2024-05-08T08:53:51.615 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:51.615 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:51.616 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:51.616 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:51.616 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:51.616 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:51.616 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22073 2024-05-08T08:53:51.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.616 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:51.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.617 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:51.617 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:53:51.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:51.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.617 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.617 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:51.617 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:53:51.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:51.618 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.618 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.618 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:51.618 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857668+0000", 2024-05-08T08:53:51.618 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7800000000000002e-07 2024-05-08T08:53:51.618 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.618 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.618 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:51.619 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857670+0000", 2024-05-08T08:53:51.619 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0820000000000001e-06 2024-05-08T08:53:51.619 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.619 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.619 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:51.619 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857678+0000", 2024-05-08T08:53:51.619 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7589999999999997e-06 2024-05-08T08:53:51.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.620 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:51.620 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857691+0000", 2024-05-08T08:53:51.620 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3672999999999999e-05 2024-05-08T08:53:51.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:51.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904780+0000", 2024-05-08T08:53:51.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047088616999999999 2024-05-08T08:53:51.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:51.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904840+0000", 2024-05-08T08:53:51.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0356999999999998e-05 2024-05-08T08:53:51.622 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.622 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.622 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:51.622 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904912+0000", 2024-05-08T08:53:51.622 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1367999999999998e-05 2024-05-08T08:53:51.622 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.622 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:51.623 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.623 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.623 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.623 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22075 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 798720~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:51.623 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:53:51.623 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 83.749739547999994, 2024-05-08T08:53:51.624 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047384643999999997, 2024-05-08T08:53:51.624 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:51.624 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:51.624 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:51.624 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:51.624 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:51.624 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:51.624 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22075 2024-05-08T08:53:51.625 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.625 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:51.625 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.625 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:51.625 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:53:51.625 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:51.625 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.626 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.626 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:51.626 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:53:51.626 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:51.626 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.626 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.626 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:51.627 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:53:51.627 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7399999999999999e-07 2024-05-08T08:53:51.627 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.627 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.627 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:51.627 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857705+0000", 2024-05-08T08:53:51.627 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1980000000000004e-06 2024-05-08T08:53:51.628 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.628 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:51.628 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857713+0000", 2024-05-08T08:53:51.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7039999999999994e-06 2024-05-08T08:53:51.628 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.628 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:51.629 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857718+0000", 2024-05-08T08:53:51.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.865e-06 2024-05-08T08:53:51.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.629 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:51.629 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904941+0000", 2024-05-08T08:53:51.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047223198000000001 2024-05-08T08:53:51.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.630 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.630 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:51.630 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904999+0000", 2024-05-08T08:53:51.630 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8254999999999997e-05 2024-05-08T08:53:51.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.630 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.630 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:51.630 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905082+0000", 2024-05-08T08:53:51.631 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3049999999999999e-05 2024-05-08T08:53:51.631 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.631 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:51.631 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.631 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.631 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.631 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22077 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 815104~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:51.631 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:53:51.632 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 83.749701536999993, 2024-05-08T08:53:51.632 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047519448999999998, 2024-05-08T08:53:51.632 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:51.632 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:51.632 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:51.632 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:51.632 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:51.632 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:51.633 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22077 2024-05-08T08:53:51.633 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.633 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:51.633 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.633 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:51.633 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:53:51.633 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:51.633 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.634 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:51.634 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:53:51.634 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:51.634 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.634 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:51.635 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:53:51.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.99e-07 2024-05-08T08:53:51.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.635 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.635 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:51.635 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857739+0000", 2024-05-08T08:53:51.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3599999999999999e-06 2024-05-08T08:53:51.636 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.636 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:51.636 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857745+0000", 2024-05-08T08:53:51.636 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5139999999999998e-06 2024-05-08T08:53:51.636 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.637 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:51.637 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857750+0000", 2024-05-08T08:53:51.637 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8550000000000001e-06 2024-05-08T08:53:51.637 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.637 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.637 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:51.637 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905110+0000", 2024-05-08T08:53:51.637 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047360126000000002 2024-05-08T08:53:51.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.638 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.638 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:51.638 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905175+0000", 2024-05-08T08:53:51.638 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4689000000000004e-05 2024-05-08T08:53:51.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.638 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.639 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:51.639 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905255+0000", 2024-05-08T08:53:51.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0306e-05 2024-05-08T08:53:51.639 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.639 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:51.639 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.639 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.640 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.640 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22079 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 831488~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:51.640 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:53:51.640 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 83.749680988999998, 2024-05-08T08:53:51.640 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.04767499, 2024-05-08T08:53:51.640 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:51.640 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:51.641 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:51.641 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:51.641 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:51.641 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:51.641 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22079 2024-05-08T08:53:51.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.641 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:51.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:51.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:53:51.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:51.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:51.643 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:53:51.643 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:51.643 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.643 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.643 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:51.643 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857757+0000", 2024-05-08T08:53:51.643 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8299999999999997e-07 2024-05-08T08:53:51.643 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.644 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.644 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:51.644 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857759+0000", 2024-05-08T08:53:51.644 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1390000000000002e-06 2024-05-08T08:53:51.644 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.644 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.644 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:51.644 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857765+0000", 2024-05-08T08:53:51.644 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4609999999999996e-06 2024-05-08T08:53:51.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.645 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.645 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:51.645 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857770+0000", 2024-05-08T08:53:51.645 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6689999999999997e-06 2024-05-08T08:53:51.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.645 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.646 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:51.646 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905281+0000", 2024-05-08T08:53:51.646 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047510668999999998 2024-05-08T08:53:51.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.646 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:51.646 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905344+0000", 2024-05-08T08:53:51.646 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2797000000000002e-05 2024-05-08T08:53:51.647 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.647 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.647 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:51.647 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905431+0000", 2024-05-08T08:53:51.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7771999999999999e-05 2024-05-08T08:53:51.647 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.647 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:51.648 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.648 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.648 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.648 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22082 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 847872~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:51.648 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:53:51.648 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 83.749661298000007, 2024-05-08T08:53:51.648 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047831934999999999, 2024-05-08T08:53:51.649 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:51.649 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:51.649 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:51.649 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:51.649 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:51.649 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:51.649 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22082 2024-05-08T08:53:51.649 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.649 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:51.650 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.650 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:51.650 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:53:51.650 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:51.650 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.650 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.650 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:51.650 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:53:51.650 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:51.650 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.650 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.651 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:51.651 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:53:51.651 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:53:51.651 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.651 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.651 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:51.651 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857778+0000", 2024-05-08T08:53:51.651 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0789999999999999e-06 2024-05-08T08:53:51.651 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.651 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.651 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:51.651 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857784+0000", 2024-05-08T08:53:51.651 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0480000000000004e-06 2024-05-08T08:53:51.651 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.651 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.652 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:51.652 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857790+0000", 2024-05-08T08:53:51.652 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2719999999999997e-06 2024-05-08T08:53:51.652 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.652 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.652 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:51.652 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905458+0000", 2024-05-08T08:53:51.652 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047668470999999997 2024-05-08T08:53:51.652 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.652 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.652 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:51.652 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905521+0000", 2024-05-08T08:53:51.652 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2952000000000003e-05 2024-05-08T08:53:51.652 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.652 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.652 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:51.653 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905608+0000", 2024-05-08T08:53:51.653 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6759000000000001e-05 2024-05-08T08:53:51.653 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.653 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:51.653 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.653 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.653 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.653 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22084 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 864256~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:51.653 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:53:51.653 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 83.749637347999993, 2024-05-08T08:53:51.653 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047977748000000001, 2024-05-08T08:53:51.653 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:51.653 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:51.653 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:51.653 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:51.653 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:51.654 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:51.654 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22084 2024-05-08T08:53:51.654 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.654 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:51.654 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.654 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:51.654 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:53:51.654 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:51.654 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.654 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.654 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:51.654 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:53:51.654 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:51.655 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.655 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.655 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:51.655 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:53:51.655 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:53:51.655 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.655 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.655 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:51.655 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857802+0000", 2024-05-08T08:53:51.655 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0779999999999998e-06 2024-05-08T08:53:51.655 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.655 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.655 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:51.655 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857812+0000", 2024-05-08T08:53:51.656 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7939999999999997e-06 2024-05-08T08:53:51.656 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.656 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.656 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:51.656 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857817+0000", 2024-05-08T08:53:51.656 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8899999999999998e-06 2024-05-08T08:53:51.656 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.656 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.656 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:51.656 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905636+0000", 2024-05-08T08:53:51.656 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047819252 2024-05-08T08:53:51.656 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.656 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.656 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:51.656 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905700+0000", 2024-05-08T08:53:51.657 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4016999999999994e-05 2024-05-08T08:53:51.657 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.657 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.657 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:51.657 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905778+0000", 2024-05-08T08:53:51.657 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7306999999999997e-05 2024-05-08T08:53:51.657 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.657 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:51.657 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.657 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.657 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.657 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22086 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 880640~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:51.657 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:53:51.657 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 83.749613998000001, 2024-05-08T08:53:51.657 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048119149, 2024-05-08T08:53:51.657 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:51.658 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:51.658 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:51.658 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:51.658 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:51.658 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:51.658 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22086 2024-05-08T08:53:51.658 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.658 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:51.658 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.658 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:51.658 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:53:51.658 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:51.658 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.658 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.659 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:51.659 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:53:51.659 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:51.659 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.659 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.659 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:51.659 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857824+0000", 2024-05-08T08:53:51.659 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4200000000000002e-07 2024-05-08T08:53:51.659 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.659 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.659 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:51.659 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857826+0000", 2024-05-08T08:53:51.659 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0590000000000001e-06 2024-05-08T08:53:51.659 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.659 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.660 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:51.660 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857832+0000", 2024-05-08T08:53:51.660 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8029999999999999e-06 2024-05-08T08:53:51.660 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.660 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.660 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:51.660 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857837+0000", 2024-05-08T08:53:51.660 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7929999999999997e-06 2024-05-08T08:53:51.660 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.660 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.660 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:51.660 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905803+0000", 2024-05-08T08:53:51.660 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047965761000000003 2024-05-08T08:53:51.660 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.660 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.660 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:51.661 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905862+0000", 2024-05-08T08:53:51.661 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9203000000000002e-05 2024-05-08T08:53:51.661 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.661 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.661 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:51.661 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905942+0000", 2024-05-08T08:53:51.661 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0187999999999999e-05 2024-05-08T08:53:51.661 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.661 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:51.661 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.661 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.661 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.661 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22088 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 897024~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:51.661 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:53:51.661 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 83.749593946000005, 2024-05-08T08:53:51.661 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048256049000000002, 2024-05-08T08:53:51.662 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:51.662 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:51.662 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:51.662 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:51.662 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:51.662 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:51.662 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22088 2024-05-08T08:53:51.662 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.662 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:51.662 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.662 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:51.662 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:53:51.662 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:51.662 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.663 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.663 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:51.663 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:53:51.663 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:51.663 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.663 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.663 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:51.663 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857844+0000", 2024-05-08T08:53:51.663 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.46e-07 2024-05-08T08:53:51.663 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.663 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.663 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:51.663 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857846+0000", 2024-05-08T08:53:51.663 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0899999999999999e-06 2024-05-08T08:53:51.663 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.663 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.663 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:51.664 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857855+0000", 2024-05-08T08:53:51.664 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8219999999999996e-06 2024-05-08T08:53:51.664 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.664 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.664 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:51.664 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857860+0000", 2024-05-08T08:53:51.664 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2989999999999996e-06 2024-05-08T08:53:51.664 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.664 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.664 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:51.664 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905966+0000", 2024-05-08T08:53:51.664 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048106542000000002 2024-05-08T08:53:51.664 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.664 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.664 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:51.664 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906024+0000", 2024-05-08T08:53:51.664 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7958000000000003e-05 2024-05-08T08:53:51.665 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.665 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.665 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:51.665 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906099+0000", 2024-05-08T08:53:51.665 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4992000000000003e-05 2024-05-08T08:53:51.665 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.665 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:51.665 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.665 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.665 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.665 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22091 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 913408~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:51.665 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:53:51.665 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 83.749570186, 2024-05-08T08:53:51.665 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048395290000000001, 2024-05-08T08:53:51.665 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:51.666 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:51.666 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:51.666 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:51.666 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:51.666 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:51.666 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22091 2024-05-08T08:53:51.666 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.666 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:51.666 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.666 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:51.666 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:53:51.666 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:51.666 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.666 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.667 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:51.667 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:53:51.667 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:51.667 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.667 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.667 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:51.667 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:53:51.667 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4900000000000001e-07 2024-05-08T08:53:51.667 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.667 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.667 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:51.667 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857870+0000", 2024-05-08T08:53:51.667 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2510000000000001e-06 2024-05-08T08:53:51.667 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.668 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.668 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:51.668 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857877+0000", 2024-05-08T08:53:51.668 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0809999999999996e-06 2024-05-08T08:53:51.668 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.668 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.668 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:51.668 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857881+0000", 2024-05-08T08:53:51.668 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.724e-06 2024-05-08T08:53:51.668 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.668 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.668 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:51.668 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906127+0000", 2024-05-08T08:53:51.668 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048245345000000002 2024-05-08T08:53:51.668 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.668 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.669 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:51.669 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906190+0000", 2024-05-08T08:53:51.669 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2762000000000007e-05 2024-05-08T08:53:51.669 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.669 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.669 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:51.669 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906262+0000", 2024-05-08T08:53:51.669 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2778e-05 2024-05-08T08:53:51.669 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.669 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:51.669 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.669 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.669 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.669 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22093 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 929792~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:51.669 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:53:51.669 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 83.749547821999997, 2024-05-08T08:53:51.670 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048532875000000003, 2024-05-08T08:53:51.670 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:51.670 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:51.670 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:51.670 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:51.670 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:51.670 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:51.670 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22093 2024-05-08T08:53:51.670 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.670 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:51.670 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.670 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:51.670 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:53:51.670 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:51.671 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.671 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.671 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:51.671 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:53:51.671 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:51.671 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.671 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.671 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:51.671 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857890+0000", 2024-05-08T08:53:51.671 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-07 2024-05-08T08:53:51.671 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.671 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.671 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:51.671 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857899+0000", 2024-05-08T08:53:51.671 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6889999999999999e-06 2024-05-08T08:53:51.671 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.672 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.672 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:51.672 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857906+0000", 2024-05-08T08:53:51.672 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6490000000000002e-06 2024-05-08T08:53:51.672 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.672 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.672 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:51.672 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857917+0000", 2024-05-08T08:53:51.672 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0764999999999999e-05 2024-05-08T08:53:51.672 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.672 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.672 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:51.672 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906287+0000", 2024-05-08T08:53:51.672 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048370442 2024-05-08T08:53:51.672 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.672 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.673 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:51.673 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906349+0000", 2024-05-08T08:53:51.673 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1537000000000004e-05 2024-05-08T08:53:51.673 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.673 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.673 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:51.673 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906422+0000", 2024-05-08T08:53:51.673 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3431999999999998e-05 2024-05-08T08:53:51.673 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.673 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:51.673 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.673 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.673 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.673 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22095 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 946176~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:51.673 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:53:51.674 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 83.749513117999996, 2024-05-08T08:53:51.674 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048657593999999998, 2024-05-08T08:53:51.674 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:51.674 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:51.674 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:51.674 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:51.674 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:51.674 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:51.674 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22095 2024-05-08T08:53:51.674 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.674 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:51.674 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.674 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:51.674 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:53:51.675 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:51.675 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.675 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.675 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:51.675 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:53:51.675 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:51.675 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.675 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.675 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:51.675 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857925+0000", 2024-05-08T08:53:51.675 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3099999999999998e-07 2024-05-08T08:53:51.675 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.675 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.675 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:51.675 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857927+0000", 2024-05-08T08:53:51.675 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.34e-06 2024-05-08T08:53:51.676 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.676 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.676 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:51.676 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857938+0000", 2024-05-08T08:53:51.676 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1426e-05 2024-05-08T08:53:51.676 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.676 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.676 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:51.676 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857943+0000", 2024-05-08T08:53:51.676 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5140000000000003e-06 2024-05-08T08:53:51.676 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.676 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.676 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:51.676 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906448+0000", 2024-05-08T08:53:51.677 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048504641000000001 2024-05-08T08:53:51.677 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.677 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.677 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:51.677 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906510+0000", 2024-05-08T08:53:51.677 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2663000000000006e-05 2024-05-08T08:53:51.677 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.677 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.677 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:51.677 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906582+0000", 2024-05-08T08:53:51.677 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1478999999999998e-05 2024-05-08T08:53:51.677 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.677 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:51.677 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.677 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.678 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.678 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22097 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 962560~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:51.678 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:53:51.678 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 83.749485429000003, 2024-05-08T08:53:51.678 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048791353000000003, 2024-05-08T08:53:51.678 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:51.678 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:51.678 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:51.678 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:51.678 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:51.678 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:51.678 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22097 2024-05-08T08:53:51.678 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.678 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:51.678 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.679 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:51.679 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:53:51.679 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:51.679 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.679 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.679 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:51.679 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:53:51.679 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:51.679 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.679 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.679 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:51.679 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:53:51.679 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9200000000000002e-07 2024-05-08T08:53:51.679 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.680 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.680 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:51.680 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857954+0000", 2024-05-08T08:53:51.680 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1900000000000002e-06 2024-05-08T08:53:51.680 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.680 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.680 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:51.680 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857961+0000", 2024-05-08T08:53:51.680 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4989999999999999e-06 2024-05-08T08:53:51.680 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.680 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.680 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:51.680 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857971+0000", 2024-05-08T08:53:51.680 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0485999999999999e-05 2024-05-08T08:53:51.680 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.681 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.681 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:51.681 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906607+0000", 2024-05-08T08:53:51.681 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048635109000000003 2024-05-08T08:53:51.681 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.681 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.681 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:51.681 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906667+0000", 2024-05-08T08:53:51.681 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0439000000000002e-05 2024-05-08T08:53:51.681 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.681 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.681 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:51.681 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906743+0000", 2024-05-08T08:53:51.681 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6237999999999998e-05 2024-05-08T08:53:51.681 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.682 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:51.682 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.682 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.682 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.682 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22099 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 978944~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:51.682 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:53:51.682 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 83.749459268999999, 2024-05-08T08:53:51.682 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048932055000000002, 2024-05-08T08:53:51.682 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:51.682 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:51.682 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:51.682 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:51.682 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:51.682 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:51.683 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22099 2024-05-08T08:53:51.683 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.683 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:51.683 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.683 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:51.683 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:53:51.683 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:51.683 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.683 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.683 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:51.683 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:53:51.683 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:51.683 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.683 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.683 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:51.684 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:53:51.684 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.27e-07 2024-05-08T08:53:51.684 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.684 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.684 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:51.684 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857987+0000", 2024-05-08T08:53:51.684 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5890000000000003e-06 2024-05-08T08:53:51.684 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.684 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.684 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:51.684 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857994+0000", 2024-05-08T08:53:51.684 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6619999999999999e-06 2024-05-08T08:53:51.684 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.684 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.684 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:51.685 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857998+0000", 2024-05-08T08:53:51.685 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7110000000000003e-06 2024-05-08T08:53:51.685 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.685 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.685 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:51.685 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906772+0000", 2024-05-08T08:53:51.685 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048773415000000001 2024-05-08T08:53:51.685 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.685 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.685 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:51.685 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906834+0000", 2024-05-08T08:53:51.685 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1752000000000001e-05 2024-05-08T08:53:51.686 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.686 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.686 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:51.686 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906910+0000", 2024-05-08T08:53:51.686 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6499000000000001e-05 2024-05-08T08:53:51.686 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.686 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:51.686 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.686 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.686 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.686 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22101 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 995328~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:51.686 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:53:51.686 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 83.749432138000003, 2024-05-08T08:53:51.686 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049087364000000001, 2024-05-08T08:53:51.686 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:51.687 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:51.687 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:51.687 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:51.687 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:51.687 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:51.687 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22101 2024-05-08T08:53:51.687 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.687 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:51.687 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.687 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:51.687 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:53:51.687 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:51.687 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.687 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.687 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:51.688 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:53:51.688 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:51.688 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.688 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.688 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:51.688 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858006+0000", 2024-05-08T08:53:51.688 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1399999999999997e-07 2024-05-08T08:53:51.688 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.688 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.688 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:51.688 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858008+0000", 2024-05-08T08:53:51.688 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2060000000000001e-06 2024-05-08T08:53:51.688 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.688 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.688 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:51.689 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858017+0000", 2024-05-08T08:53:51.689 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1579999999999994e-06 2024-05-08T08:53:51.689 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.689 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.689 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:51.689 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858022+0000", 2024-05-08T08:53:51.689 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4199999999999998e-06 2024-05-08T08:53:51.689 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.689 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.689 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:51.689 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906940+0000", 2024-05-08T08:53:51.689 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048917627999999998 2024-05-08T08:53:51.689 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.689 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.689 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:51.689 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907002+0000", 2024-05-08T08:53:51.690 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2399999999999999e-05 2024-05-08T08:53:51.690 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.690 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.690 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:51.690 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907092+0000", 2024-05-08T08:53:51.690 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0137999999999997e-05 2024-05-08T08:53:51.690 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.690 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:51.690 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.690 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.690 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.690 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22103 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1011712~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:51.690 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:53:51.690 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 83.749407352999995, 2024-05-08T08:53:51.690 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049228542, 2024-05-08T08:53:51.691 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:51.691 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:51.691 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:51.691 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:51.691 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:51.691 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:51.691 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22103 2024-05-08T08:53:51.691 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.691 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:51.691 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.691 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:51.691 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:53:51.691 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:51.691 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.692 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.692 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:51.692 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:53:51.692 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:51.692 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.692 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.692 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:51.692 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:53:51.692 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4299999999999999e-07 2024-05-08T08:53:51.692 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.692 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.692 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:51.692 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858032+0000", 2024-05-08T08:53:51.693 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0590000000000001e-06 2024-05-08T08:53:51.693 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.693 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.693 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:51.693 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858037+0000", 2024-05-08T08:53:51.693 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.977e-06 2024-05-08T08:53:51.693 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.693 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.693 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:51.693 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858042+0000", 2024-05-08T08:53:51.693 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7650000000000001e-06 2024-05-08T08:53:51.693 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.693 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.693 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:51.694 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907118+0000", 2024-05-08T08:53:51.694 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049076147000000001 2024-05-08T08:53:51.694 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.694 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.694 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:51.694 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907180+0000", 2024-05-08T08:53:51.694 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1971999999999996e-05 2024-05-08T08:53:51.694 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.694 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.694 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:51.694 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907258+0000", 2024-05-08T08:53:51.694 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8279e-05 2024-05-08T08:53:51.694 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.695 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:51.695 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.695 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.695 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.695 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22105 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1028096~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:51.695 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:53:51.695 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 83.749388109999998, 2024-05-08T08:53:51.695 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049394609999999999, 2024-05-08T08:53:51.695 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:51.695 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:51.695 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:51.695 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:51.695 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:51.695 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:51.696 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22105 2024-05-08T08:53:51.696 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.696 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:51.696 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.696 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:51.696 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:53:51.696 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:51.696 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.696 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.696 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:51.696 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:53:51.696 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:51.696 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.696 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.696 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:51.697 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858050+0000", 2024-05-08T08:53:51.697 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5999999999999999e-07 2024-05-08T08:53:51.697 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.697 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.697 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:51.697 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858051+0000", 2024-05-08T08:53:51.697 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8870000000000001e-06 2024-05-08T08:53:51.697 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.697 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.697 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:51.697 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858057+0000", 2024-05-08T08:53:51.697 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8019999999999997e-06 2024-05-08T08:53:51.697 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.697 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.697 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:51.698 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858062+0000", 2024-05-08T08:53:51.698 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.561e-06 2024-05-08T08:53:51.698 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.698 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.698 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:51.698 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907289+0000", 2024-05-08T08:53:51.698 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049226957000000002 2024-05-08T08:53:51.698 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.698 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.698 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:51.698 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907356+0000", 2024-05-08T08:53:51.698 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6915000000000006e-05 2024-05-08T08:53:51.698 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.698 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.699 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:51.699 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907444+0000", 2024-05-08T08:53:51.699 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8127999999999994e-05 2024-05-08T08:53:51.699 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.699 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:51.699 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.699 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.699 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.699 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22107 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1044480~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:51.699 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:53:51.699 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 83.749369770000001, 2024-05-08T08:53:51.699 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049799942999999999, 2024-05-08T08:53:51.699 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:51.700 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:51.700 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:51.700 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:51.700 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:51.700 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:51.700 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22107 2024-05-08T08:53:51.700 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.700 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:51.700 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.700 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:51.700 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:53:51.700 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:51.700 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.700 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.700 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:51.701 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:53:51.701 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:51.701 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.701 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.701 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:51.701 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:53:51.701 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4799999999999999e-07 2024-05-08T08:53:51.701 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.701 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.701 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:51.701 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858070+0000", 2024-05-08T08:53:51.701 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.012e-06 2024-05-08T08:53:51.701 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.701 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.702 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:51.702 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858077+0000", 2024-05-08T08:53:51.702 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6290000000000001e-06 2024-05-08T08:53:51.702 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.702 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.702 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:51.702 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858083+0000", 2024-05-08T08:53:51.702 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8880000000000002e-06 2024-05-08T08:53:51.702 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.702 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.702 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:51.702 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907671+0000", 2024-05-08T08:53:51.702 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049587191000000003 2024-05-08T08:53:51.703 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.703 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.703 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:51.703 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907763+0000", 2024-05-08T08:53:51.703 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2231000000000005e-05 2024-05-08T08:53:51.703 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.703 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.703 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:51.703 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907867+0000", 2024-05-08T08:53:51.703 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104644 2024-05-08T08:53:51.703 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.703 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:51.703 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.703 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.703 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.703 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22294 3.1c 3:3aad0389:::rbd_data.11406886b07b.0000000000000080:head [stat,delete] snapc 7=[7] RETRY=6 ondisk+retry+write+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:51.704 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:53:51.704 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 83.748775921999993, 2024-05-08T08:53:51.704 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051152191, 2024-05-08T08:53:51.704 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:51.704 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:51.704 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:51.704 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:51.704 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:51.704 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:51.704 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22294 2024-05-08T08:53:51.704 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.704 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:51.704 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.704 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:51.704 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:53:51.705 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:51.705 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.705 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.705 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:51.705 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:53:51.705 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:51.705 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.705 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.705 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:51.705 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858662+0000", 2024-05-08T08:53:51.705 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4799999999999998e-07 2024-05-08T08:53:51.705 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.705 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.705 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:51.705 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858664+0000", 2024-05-08T08:53:51.706 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1229999999999998e-06 2024-05-08T08:53:51.706 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.706 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.706 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:51.706 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858670+0000", 2024-05-08T08:53:51.706 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3099999999999997e-06 2024-05-08T08:53:51.706 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.706 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.706 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:51.706 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858676+0000", 2024-05-08T08:53:51.706 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9750000000000004e-06 2024-05-08T08:53:51.706 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.706 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.706 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:51.707 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907901+0000", 2024-05-08T08:53:51.707 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049224520000000001 2024-05-08T08:53:51.707 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.707 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.707 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:51.707 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.908085+0000", 2024-05-08T08:53:51.707 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00018377500000000001 2024-05-08T08:53:51.707 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.707 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.707 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:51.707 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.909814+0000", 2024-05-08T08:53:51.707 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00172894 2024-05-08T08:53:51.707 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.707 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:51.708 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.708 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.708 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.708 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4416.0:13461 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-05-08T08:53:51.708 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:22.487456+0000", 2024-05-08T08:53:51.708 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 29.119980881, 2024-05-08T08:53:51.708 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.99102620799999996, 2024-05-08T08:53:51.708 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:51.708 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:51.708 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:51.708 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:51.708 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4416", 2024-05-08T08:53:51.708 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:51.708 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 13461 2024-05-08T08:53:51.708 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.708 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:51.708 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.709 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:51.709 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487456+0000", 2024-05-08T08:53:51.709 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:51.709 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.709 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.709 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:51.709 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487456+0000", 2024-05-08T08:53:51.709 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:51.709 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.709 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.709 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:51.709 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487458+0000", 2024-05-08T08:53:51.709 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5659999999999999e-06 2024-05-08T08:53:51.709 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.709 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.710 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:51.710 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487463+0000", 2024-05-08T08:53:51.710 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6890000000000003e-06 2024-05-08T08:53:51.710 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.710 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.710 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:51.710 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487472+0000", 2024-05-08T08:53:51.710 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7710000000000001e-06 2024-05-08T08:53:51.710 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.710 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.710 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:51.710 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487478+0000", 2024-05-08T08:53:51.710 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1390000000000003e-06 2024-05-08T08:53:51.710 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.710 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.710 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:51.711 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487520+0000", 2024-05-08T08:53:51.711 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2259999999999999e-05 2024-05-08T08:53:51.711 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.711 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.711 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-05-08T08:53:51.711 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487528+0000", 2024-05-08T08:53:51.711 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7479999999999993e-06 2024-05-08T08:53:51.711 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.711 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.711 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:51.711 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:23.478139+0000", 2024-05-08T08:53:51.711 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.99061165200000001 2024-05-08T08:53:51.711 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.711 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.711 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:51.712 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:23.478301+0000", 2024-05-08T08:53:51.712 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000161602 2024-05-08T08:53:51.712 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:51.712 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:51.712 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:51.712 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:23.478483+0000", 2024-05-08T08:53:51.712 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000181781 2024-05-08T08:53:51.712 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.712 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:51.712 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.712 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:51.712 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:51.712 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:53:51.713 INFO:teuthology.orchestra.run.smithi196.stdout:flattening image image_client.0-clone30 2024-05-08T08:53:51.713 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:53:51.714 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:53:51.714 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:53:51.715 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:53:51.715 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:53:51.716 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:53:51.716 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:53:51.716 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:53:51.716 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:53:51.717 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:53:51.717 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:53:51.717 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:51.715+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:51.722 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:53:51.723 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:53:51.815 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:51.817+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:51.865 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:51.865 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:51.874 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:53:51.916 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:51.918+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:51.982 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:51.983 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:51.992 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:53:52.017 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:52.019+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:52.055 INFO:teuthology.orchestra.run.smithi196.stdout:checking clone #29, image image_client.0-clone29 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent30 2024-05-08T08:53:52.098 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:52.099 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:52.107 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:53:52.151 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:52.153+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:52.219 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:52.222+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:52.226 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:52.226 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:52.235 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:53:52.320 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:52.323+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:52.390 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:52.390 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:52.399 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:53:52.420 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:52.423+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:52.509 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:52.509 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:52.518 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:53:52.560 INFO:teuthology.orchestra.run.smithi196.stdout:4473 write 0x5e34e00 thru 0x5e35d84 (0xf85 bytes) 2024-05-08T08:53:52.561 INFO:teuthology.orchestra.run.smithi196.stdout:4474 read 0xa954600 thru 0xa95c954 (0x8355 bytes) 2024-05-08T08:53:52.561 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:53:52.563+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:53:52.562 INFO:teuthology.orchestra.run.smithi196.stdout:4475 clone 32 order 21 su 1048576 sc 8 2024-05-08T08:53:52.622 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:52.625+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:52.634 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:52.634 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:52.644 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:53:52.660 INFO:teuthology.orchestra.run.smithi196.stdout:truncating image image_client.0-clone31 from 0xe727000 (overlap 0xe727000) to 0x0 2024-05-08T08:53:52.723 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:52.725+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:52.810 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:52.810 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:52.820 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:53:52.823 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:52.825+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:52.924 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:52.926+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:52.933 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:52.934 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:52.943 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:53:53.000 INFO:teuthology.orchestra.run.smithi196.stdout:checking clone #30, image image_client.0-clone30 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent31 2024-05-08T08:53:53.024 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:53.026+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:53.054 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:53.054 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:53.063 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:53:53.148 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:53:53.150+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:53:53.227 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:53:53.229+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:53:53.228 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:53.228 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:53.237 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:53:53.328 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:53:53.330+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:53:53.424 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:53.425 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:53.429 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:53.432+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:53.435 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:53:53.530 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:53.533+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:53.542 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:53.542 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:53.551 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:53:53.590 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 5, 4] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-05-08T08:53:53.591 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T08:53:53.591 INFO:tasks.thrashosds.thrasher:Removing osd 5, in_osds are: [0, 1, 2, 3, 5, 4] 2024-05-08T08:53:53.591 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 5 2024-05-08T08:53:53.596 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-05-08T08:53:53.636 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:53.639+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:53.679 INFO:teuthology.orchestra.run.smithi196.stdout:4476 trunc from 0xe727000 to 0x589d800 2024-05-08T08:53:53.700 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:53.700 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:53.710 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:53:53.714 INFO:teuthology.orchestra.run.smithi196.stdout:4479 punch from 0x3ac5000 to 0x3ad1e00, (0xce00 bytes) 2024-05-08T08:53:53.732 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:53.735+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:53.733 INFO:teuthology.orchestra.run.smithi196.stdout:4480 write 0xdfe9400 thru 0xdfed3dd (0x3fde bytes) 2024-05-08T08:53:53.734 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T08:53:53.740 INFO:teuthology.orchestra.run.smithi196.stdout:4482 punch from 0x6270c00 to 0x627ae00, (0xa200 bytes) 2024-05-08T08:53:53.757 INFO:teuthology.orchestra.run.smithi196.stdout:4484 write 0xeb73c00 thru 0xeb74d6e (0x116f bytes) 2024-05-08T08:53:53.763 INFO:teuthology.orchestra.run.smithi196.stdout:4486 write 0x6523600 thru 0x652d42b (0x9e2c bytes) 2024-05-08T08:53:53.768 INFO:teuthology.orchestra.run.smithi196.stdout:4488 trunc from 0xeb74e00 to 0xb042600 2024-05-08T08:53:53.796 INFO:teuthology.orchestra.run.smithi196.stdout:4490 read 0xa8b2a00 thru 0xa8b461d (0x1c1e bytes) 2024-05-08T08:53:53.813 INFO:teuthology.orchestra.run.smithi196.stdout:4491 trunc from 0xb042600 to 0x2990000 2024-05-08T08:53:53.828 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:53.828 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:53.833 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:53.836+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:53.838 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:53:53.849 INFO:teuthology.orchestra.run.smithi196.stdout:4492 punch from 0x24c0000 to 0x24c0600, (0x600 bytes) 2024-05-08T08:53:53.872 INFO:teuthology.orchestra.run.smithi196.stdout:4493 write 0xe16ce00 thru 0xe17c867 (0xfa68 bytes) 2024-05-08T08:53:53.872 INFO:teuthology.orchestra.run.smithi196.stdout:4494 write 0x270e600 thru 0x271b88d (0xd28e bytes) 2024-05-08T08:53:53.874 INFO:teuthology.orchestra.run.smithi196.stdout:4498 read 0x7d30600 thru 0x7d3efb4 (0xe9b5 bytes) 2024-05-08T08:53:53.875 INFO:teuthology.orchestra.run.smithi196.stdout:4499 punch from 0x5c48600 to 0x5c51e00, (0x9800 bytes) 2024-05-08T08:53:53.875 INFO:teuthology.orchestra.run.smithi196.stdout:4500 write 0xbf0c200 thru 0xbf1a688 (0xe489 bytes) 2024-05-08T08:53:53.875 INFO:teuthology.orchestra.run.smithi196.stdout:4501 punch from 0x9d0c200 to 0x9d0ce00, (0xc00 bytes) 2024-05-08T08:53:53.875 INFO:teuthology.orchestra.run.smithi196.stdout:4502 read 0x7617c00 thru 0x7626564 (0xe965 bytes) 2024-05-08T08:53:53.875 INFO:teuthology.orchestra.run.smithi196.stdout:4504 write 0x7794a00 thru 0x77a25a7 (0xdba8 bytes) 2024-05-08T08:53:53.875 INFO:teuthology.orchestra.run.smithi196.stdout:4505 read 0x9927600 thru 0x992cb40 (0x5541 bytes) 2024-05-08T08:53:53.875 INFO:teuthology.orchestra.run.smithi196.stdout:4511 write 0xe1de00 thru 0xe2d666 (0xf867 bytes) 2024-05-08T08:53:53.883 INFO:teuthology.orchestra.run.smithi196.stdout:4513 trunc from 0xe17ca00 to 0x7330800 2024-05-08T08:53:53.893 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:53:53.893 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:53:53.916 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:53:53.916 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:53:53.918 INFO:teuthology.orchestra.run.smithi196.stdout:4514 read 0x27bf800 thru 0x27c58ff (0x6100 bytes) 2024-05-08T08:53:53.919 INFO:teuthology.orchestra.run.smithi196.stdout:4516 read 0x18b2a00 thru 0x18b3830 (0xe31 bytes) 2024-05-08T08:53:53.920 INFO:teuthology.orchestra.run.smithi196.stdout:4520 read 0x4873e00 thru 0x487a289 (0x648a bytes) 2024-05-08T08:53:53.934 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:53:53.937+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:53:53.936 INFO:teuthology.orchestra.run.smithi196.stdout:4522 write 0xacb5200 thru 0xacbbbdf (0x69e0 bytes) 2024-05-08T08:53:53.937 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:53:53.937 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:53:53.953 INFO:teuthology.orchestra.run.smithi196.stdout:4523 write 0xe9dfe00 thru 0xe9e502b (0x522c bytes) 2024-05-08T08:53:53.954 INFO:teuthology.orchestra.run.smithi196.stdout:4527 write 0xd1da600 thru 0xd1e03ba (0x5dbb bytes) 2024-05-08T08:53:53.954 INFO:teuthology.orchestra.run.smithi196.stdout:4528 punch from 0x4c62800 to 0x4c67c00, (0x5400 bytes) 2024-05-08T08:53:53.954 INFO:teuthology.orchestra.run.smithi196.stdout:4529 read 0x5cc2600 thru 0x5cd10d8 (0xead9 bytes) 2024-05-08T08:53:53.955 INFO:teuthology.orchestra.run.smithi196.stdout:4532 punch from 0xcd76000 to 0xcd7d400, (0x7400 bytes) 2024-05-08T08:53:53.955 INFO:teuthology.orchestra.run.smithi196.stdout:4535 punch from 0x6198a00 to 0x61a2600, (0x9c00 bytes) 2024-05-08T08:53:53.955 INFO:teuthology.orchestra.run.smithi196.stdout:4537 write 0x7d0d200 thru 0x7d103b1 (0x31b2 bytes) 2024-05-08T08:53:53.959 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:53:53.960 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:53:53.960 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:53.960 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:53.966 INFO:teuthology.orchestra.run.smithi196.stdout:4538 trunc from 0xe9e5200 to 0x5298400 2024-05-08T08:53:53.970 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:53:53.983 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:53:53.983 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:53:53.991 INFO:teuthology.orchestra.run.smithi196.stdout:4539 read 0x4d89e00 thru 0x4d96b30 (0xcd31 bytes) 2024-05-08T08:53:53.992 INFO:teuthology.orchestra.run.smithi196.stdout:4542 read 0x2d4f800 thru 0x2d5cf05 (0xd706 bytes) 2024-05-08T08:53:53.997 INFO:teuthology.orchestra.run.smithi196.stdout:4547 read 0x4994400 thru 0x4998c5f (0x4860 bytes) 2024-05-08T08:53:53.997 INFO:teuthology.orchestra.run.smithi196.stdout:4551 write 0x2a5bc00 thru 0x2a5ebc5 (0x2fc6 bytes) 2024-05-08T08:53:53.998 INFO:teuthology.orchestra.run.smithi196.stdout:4552 write 0x12e1800 thru 0x12e2057 (0x858 bytes) 2024-05-08T08:53:53.998 INFO:teuthology.orchestra.run.smithi196.stdout:4553 punch from 0x17ae00 to 0x18a600, (0xf800 bytes) 2024-05-08T08:53:53.998 INFO:teuthology.orchestra.run.smithi196.stdout:4554 read 0x349a400 thru 0x349a618 (0x219 bytes) 2024-05-08T08:53:53.998 INFO:teuthology.orchestra.run.smithi196.stdout:4556 read 0x4daaa00 thru 0x4db4751 (0x9d52 bytes) 2024-05-08T08:53:53.998 INFO:teuthology.orchestra.run.smithi196.stdout:4559 clone 33 order 24 su 524288 sc 9 2024-05-08T08:53:54.003 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:53:54.003 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:53:54.004 INFO:tasks.ceph.mon.a.smithi059.stderr:2024-05-08T08:53:54.007+0000 7fc9cc811640 -1 mon.a@0(leader).osd e91 definitely_dead 0 2024-05-08T08:53:54.057 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:53:54.060+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:53:54.080 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:53:54.080 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:53:54.080 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:53:54.080 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:53:54.080 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:53:54.089 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:53:54.136 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:54.138+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:54.195 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:53:54.196 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:53:54.196 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:53:54.196 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:53:54.196 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.196 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22071 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 765952~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:54.196 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857643+0000", 2024-05-08T08:53:54.196 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 86.340081663999996, 2024-05-08T08:53:54.196 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047106528000000002, 2024-05-08T08:53:54.196 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:54.196 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:54.196 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:54.196 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:54.196 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:54.196 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:54.197 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22071 2024-05-08T08:53:54.197 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.197 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:54.197 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.197 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:54.197 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857643+0000", 2024-05-08T08:53:54.197 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:54.197 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.197 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.197 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:54.197 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857643+0000", 2024-05-08T08:53:54.197 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:54.197 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.197 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.197 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:54.197 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857644+0000", 2024-05-08T08:53:54.198 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2000000000000001e-07 2024-05-08T08:53:54.198 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.198 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.198 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:54.198 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857646+0000", 2024-05-08T08:53:54.198 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0059999999999999e-06 2024-05-08T08:53:54.198 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.198 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.198 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:54.198 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857651+0000", 2024-05-08T08:53:54.198 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.84e-06 2024-05-08T08:53:54.198 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.199 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.199 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:54.199 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857657+0000", 2024-05-08T08:53:54.199 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5620000000000003e-06 2024-05-08T08:53:54.199 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.199 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.199 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:54.199 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904620+0000", 2024-05-08T08:53:54.199 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046962661000000003 2024-05-08T08:53:54.199 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.199 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.199 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:54.199 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904678+0000", 2024-05-08T08:53:54.199 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8656999999999998e-05 2024-05-08T08:53:54.199 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.200 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.200 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:54.200 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904750+0000", 2024-05-08T08:53:54.200 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1482000000000004e-05 2024-05-08T08:53:54.200 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.200 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:54.200 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.200 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.200 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.200 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22073 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 782336~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:54.200 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:53:54.200 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 86.340057655999999, 2024-05-08T08:53:54.200 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047244234000000003, 2024-05-08T08:53:54.200 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:54.200 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:54.201 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:54.201 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:54.201 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:54.201 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:54.201 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22073 2024-05-08T08:53:54.201 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.201 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:54.201 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.201 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:54.201 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:53:54.201 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:54.201 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.201 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.201 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:54.201 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:53:54.202 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:54.202 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.202 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.202 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:54.202 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857668+0000", 2024-05-08T08:53:54.202 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7800000000000002e-07 2024-05-08T08:53:54.202 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.202 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.202 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:54.202 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857670+0000", 2024-05-08T08:53:54.202 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0820000000000001e-06 2024-05-08T08:53:54.202 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.202 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.202 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:54.202 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857678+0000", 2024-05-08T08:53:54.203 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7589999999999997e-06 2024-05-08T08:53:54.203 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.203 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.203 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:54.203 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857691+0000", 2024-05-08T08:53:54.203 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3672999999999999e-05 2024-05-08T08:53:54.203 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.203 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.203 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:54.203 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904780+0000", 2024-05-08T08:53:54.203 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047088616999999999 2024-05-08T08:53:54.203 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.203 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.203 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:54.204 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904840+0000", 2024-05-08T08:53:54.204 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0356999999999998e-05 2024-05-08T08:53:54.204 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.204 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.204 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:54.204 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904912+0000", 2024-05-08T08:53:54.204 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1367999999999998e-05 2024-05-08T08:53:54.204 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.204 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:54.204 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.204 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.204 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.204 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22075 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 798720~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:54.205 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:53:54.205 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 86.340027258999996, 2024-05-08T08:53:54.205 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047384643999999997, 2024-05-08T08:53:54.205 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:54.205 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:54.205 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:54.205 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:54.205 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:54.205 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:54.205 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22075 2024-05-08T08:53:54.205 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.205 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:54.205 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.206 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:54.206 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:53:54.206 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:54.206 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.206 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.206 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:54.206 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:53:54.206 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:54.206 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.206 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.206 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:54.206 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:53:54.206 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7399999999999999e-07 2024-05-08T08:53:54.206 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.206 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.207 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:54.207 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857705+0000", 2024-05-08T08:53:54.207 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1980000000000004e-06 2024-05-08T08:53:54.207 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.207 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.207 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:54.207 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857713+0000", 2024-05-08T08:53:54.207 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7039999999999994e-06 2024-05-08T08:53:54.207 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.207 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.207 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:54.207 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857718+0000", 2024-05-08T08:53:54.207 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.865e-06 2024-05-08T08:53:54.207 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.207 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.207 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:54.207 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904941+0000", 2024-05-08T08:53:54.208 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047223198000000001 2024-05-08T08:53:54.208 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.208 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.208 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:54.208 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904999+0000", 2024-05-08T08:53:54.208 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8254999999999997e-05 2024-05-08T08:53:54.208 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.208 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.208 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:54.208 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905082+0000", 2024-05-08T08:53:54.208 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3049999999999999e-05 2024-05-08T08:53:54.208 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.208 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:54.208 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.208 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.208 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.209 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22077 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 815104~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:54.209 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:53:54.209 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 86.339989247999995, 2024-05-08T08:53:54.209 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047519448999999998, 2024-05-08T08:53:54.209 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:54.209 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:54.209 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:54.209 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:54.209 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:54.209 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:54.209 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22077 2024-05-08T08:53:54.209 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.209 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:54.209 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.209 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:54.210 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:53:54.210 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:54.210 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.210 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.210 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:54.210 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:53:54.210 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:54.210 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.210 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.210 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:54.210 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:53:54.210 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.99e-07 2024-05-08T08:53:54.210 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.210 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.210 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:54.210 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857739+0000", 2024-05-08T08:53:54.211 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3599999999999999e-06 2024-05-08T08:53:54.211 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.211 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.211 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:54.211 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857745+0000", 2024-05-08T08:53:54.211 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5139999999999998e-06 2024-05-08T08:53:54.211 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.211 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.211 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:54.211 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857750+0000", 2024-05-08T08:53:54.211 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8550000000000001e-06 2024-05-08T08:53:54.211 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.211 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.211 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:54.211 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905110+0000", 2024-05-08T08:53:54.211 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047360126000000002 2024-05-08T08:53:54.211 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.212 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.212 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:54.212 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905175+0000", 2024-05-08T08:53:54.212 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4689000000000004e-05 2024-05-08T08:53:54.212 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.212 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.212 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:54.212 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905255+0000", 2024-05-08T08:53:54.212 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0306e-05 2024-05-08T08:53:54.212 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.212 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:54.212 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.212 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.212 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.212 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22079 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 831488~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:54.212 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:53:54.213 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 86.3399687, 2024-05-08T08:53:54.213 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.04767499, 2024-05-08T08:53:54.213 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:54.213 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:54.213 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:54.213 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:54.213 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:54.213 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:54.213 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22079 2024-05-08T08:53:54.213 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.213 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:54.213 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.213 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:54.213 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:53:54.213 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:54.213 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.213 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.214 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:54.214 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:53:54.214 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:54.214 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.214 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.214 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:54.214 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857757+0000", 2024-05-08T08:53:54.214 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8299999999999997e-07 2024-05-08T08:53:54.214 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.214 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.214 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:54.214 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857759+0000", 2024-05-08T08:53:54.214 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1390000000000002e-06 2024-05-08T08:53:54.215 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.215 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.215 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:54.215 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857765+0000", 2024-05-08T08:53:54.215 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4609999999999996e-06 2024-05-08T08:53:54.215 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.215 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.215 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:54.215 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857770+0000", 2024-05-08T08:53:54.215 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6689999999999997e-06 2024-05-08T08:53:54.215 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.215 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.215 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:54.215 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905281+0000", 2024-05-08T08:53:54.215 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047510668999999998 2024-05-08T08:53:54.215 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.216 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.216 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:54.216 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905344+0000", 2024-05-08T08:53:54.216 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2797000000000002e-05 2024-05-08T08:53:54.216 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.216 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.216 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:54.216 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905431+0000", 2024-05-08T08:53:54.216 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7771999999999999e-05 2024-05-08T08:53:54.216 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.216 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:54.216 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.216 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.216 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.216 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22082 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 847872~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:54.216 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:53:54.217 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 86.339949008999994, 2024-05-08T08:53:54.217 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047831934999999999, 2024-05-08T08:53:54.217 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:54.217 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:54.217 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:54.217 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:54.217 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:54.217 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:54.217 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22082 2024-05-08T08:53:54.217 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.217 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:54.217 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.217 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:54.217 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:53:54.217 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:54.217 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.217 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.218 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:54.218 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:53:54.218 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:54.218 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.218 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.218 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:54.218 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:53:54.218 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:53:54.218 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.218 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.218 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:54.218 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857778+0000", 2024-05-08T08:53:54.218 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0789999999999999e-06 2024-05-08T08:53:54.218 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.218 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.219 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:54.219 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857784+0000", 2024-05-08T08:53:54.219 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0480000000000004e-06 2024-05-08T08:53:54.219 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.219 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.219 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:54.219 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857790+0000", 2024-05-08T08:53:54.219 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2719999999999997e-06 2024-05-08T08:53:54.219 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.219 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.219 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:54.219 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905458+0000", 2024-05-08T08:53:54.219 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047668470999999997 2024-05-08T08:53:54.219 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.219 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.219 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:54.219 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905521+0000", 2024-05-08T08:53:54.220 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2952000000000003e-05 2024-05-08T08:53:54.220 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.220 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.220 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:54.220 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905608+0000", 2024-05-08T08:53:54.220 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6759000000000001e-05 2024-05-08T08:53:54.220 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.220 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:54.220 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.220 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.220 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.220 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22084 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 864256~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:54.220 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:53:54.220 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 86.339925058999995, 2024-05-08T08:53:54.220 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047977748000000001, 2024-05-08T08:53:54.220 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:54.221 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:54.221 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:54.221 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:54.221 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:54.221 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:54.221 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22084 2024-05-08T08:53:54.221 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.221 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:54.221 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.221 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:54.221 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:53:54.221 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:54.221 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.221 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.221 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:54.221 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:53:54.222 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:54.222 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.222 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.222 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:54.222 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:53:54.222 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:53:54.222 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.222 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.222 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:54.222 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857802+0000", 2024-05-08T08:53:54.222 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0779999999999998e-06 2024-05-08T08:53:54.222 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.222 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.222 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:54.222 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857812+0000", 2024-05-08T08:53:54.222 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7939999999999997e-06 2024-05-08T08:53:54.223 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.223 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.223 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:54.223 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857817+0000", 2024-05-08T08:53:54.223 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8899999999999998e-06 2024-05-08T08:53:54.223 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.223 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.223 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:54.223 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905636+0000", 2024-05-08T08:53:54.223 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047819252 2024-05-08T08:53:54.223 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.223 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.223 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:54.223 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905700+0000", 2024-05-08T08:53:54.223 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4016999999999994e-05 2024-05-08T08:53:54.224 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.224 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.224 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:54.224 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905778+0000", 2024-05-08T08:53:54.224 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7306999999999997e-05 2024-05-08T08:53:54.224 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.224 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:54.224 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.224 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.224 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.224 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22086 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 880640~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:54.224 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:53:54.224 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 86.339901709000003, 2024-05-08T08:53:54.224 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048119149, 2024-05-08T08:53:54.224 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:54.225 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:54.225 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:54.225 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:54.225 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:54.225 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:54.225 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22086 2024-05-08T08:53:54.225 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.225 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:54.225 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.225 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:54.225 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:53:54.225 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:54.225 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.225 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.225 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:54.225 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:53:54.226 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:54.226 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.226 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.226 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:54.226 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857824+0000", 2024-05-08T08:53:54.226 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4200000000000002e-07 2024-05-08T08:53:54.226 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.226 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.226 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:54.226 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857826+0000", 2024-05-08T08:53:54.226 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0590000000000001e-06 2024-05-08T08:53:54.226 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.226 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.226 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:54.226 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857832+0000", 2024-05-08T08:53:54.226 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8029999999999999e-06 2024-05-08T08:53:54.227 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.227 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.227 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:54.227 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857837+0000", 2024-05-08T08:53:54.227 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7929999999999997e-06 2024-05-08T08:53:54.227 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.227 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.227 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:54.227 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905803+0000", 2024-05-08T08:53:54.227 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047965761000000003 2024-05-08T08:53:54.227 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.227 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.227 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:54.227 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905862+0000", 2024-05-08T08:53:54.227 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9203000000000002e-05 2024-05-08T08:53:54.227 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.228 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.228 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:54.228 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905942+0000", 2024-05-08T08:53:54.228 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0187999999999999e-05 2024-05-08T08:53:54.228 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.228 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:54.228 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.228 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.228 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.228 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22088 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 897024~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:54.228 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:53:54.229 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 86.339881657000006, 2024-05-08T08:53:54.229 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048256049000000002, 2024-05-08T08:53:54.229 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:54.229 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:54.229 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:54.229 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:54.229 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:54.229 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:54.229 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22088 2024-05-08T08:53:54.229 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.229 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:54.229 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.229 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:54.230 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:53:54.230 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:54.230 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.230 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.230 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:54.230 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:53:54.230 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:54.230 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.230 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.230 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:54.230 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857844+0000", 2024-05-08T08:53:54.230 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.46e-07 2024-05-08T08:53:54.231 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.231 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.231 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:54.231 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857846+0000", 2024-05-08T08:53:54.231 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0899999999999999e-06 2024-05-08T08:53:54.231 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.231 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.231 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:54.231 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857855+0000", 2024-05-08T08:53:54.231 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8219999999999996e-06 2024-05-08T08:53:54.231 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.231 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.231 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:54.231 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857860+0000", 2024-05-08T08:53:54.231 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2989999999999996e-06 2024-05-08T08:53:54.231 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.232 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.232 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:54.232 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905966+0000", 2024-05-08T08:53:54.232 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048106542000000002 2024-05-08T08:53:54.232 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.232 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.232 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:54.232 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906024+0000", 2024-05-08T08:53:54.232 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7958000000000003e-05 2024-05-08T08:53:54.232 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.232 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.232 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:54.232 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906099+0000", 2024-05-08T08:53:54.232 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4992000000000003e-05 2024-05-08T08:53:54.232 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.232 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:54.232 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.233 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.233 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.233 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22091 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 913408~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:54.233 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:53:54.233 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 86.339857897000002, 2024-05-08T08:53:54.233 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048395290000000001, 2024-05-08T08:53:54.233 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:54.233 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:54.233 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:54.233 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:54.233 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:54.233 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:54.233 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22091 2024-05-08T08:53:54.233 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.233 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:54.233 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.234 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:54.234 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:53:54.234 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:54.234 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.234 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.234 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:54.234 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:53:54.234 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:54.234 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.234 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.234 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:54.234 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:53:54.234 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4900000000000001e-07 2024-05-08T08:53:54.234 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.235 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.235 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:54.235 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857870+0000", 2024-05-08T08:53:54.235 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2510000000000001e-06 2024-05-08T08:53:54.235 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.235 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.235 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:54.235 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857877+0000", 2024-05-08T08:53:54.235 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0809999999999996e-06 2024-05-08T08:53:54.235 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.235 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.235 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:54.235 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857881+0000", 2024-05-08T08:53:54.235 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.724e-06 2024-05-08T08:53:54.235 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.236 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.236 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:54.236 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906127+0000", 2024-05-08T08:53:54.236 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048245345000000002 2024-05-08T08:53:54.236 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.236 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.236 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:54.236 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906190+0000", 2024-05-08T08:53:54.236 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2762000000000007e-05 2024-05-08T08:53:54.236 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.236 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.236 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:54.236 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906262+0000", 2024-05-08T08:53:54.236 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2778e-05 2024-05-08T08:53:54.236 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.236 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:54.236 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.237 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.237 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.237 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22093 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 929792~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:54.237 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:53:54.237 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 86.339835532999999, 2024-05-08T08:53:54.237 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048532875000000003, 2024-05-08T08:53:54.237 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:54.237 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:54.237 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:54.237 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:54.237 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:54.237 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:54.237 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22093 2024-05-08T08:53:54.237 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.237 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:54.237 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.237 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:54.238 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:53:54.238 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:54.238 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.238 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.238 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:54.238 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:53:54.238 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:54.238 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.238 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.238 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:54.238 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857890+0000", 2024-05-08T08:53:54.238 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-07 2024-05-08T08:53:54.238 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.238 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.238 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:54.239 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857899+0000", 2024-05-08T08:53:54.239 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6889999999999999e-06 2024-05-08T08:53:54.239 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.239 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.239 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:54.239 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857906+0000", 2024-05-08T08:53:54.239 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6490000000000002e-06 2024-05-08T08:53:54.239 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.239 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.239 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:54.239 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857917+0000", 2024-05-08T08:53:54.239 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0764999999999999e-05 2024-05-08T08:53:54.239 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.239 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.239 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:54.239 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906287+0000", 2024-05-08T08:53:54.239 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048370442 2024-05-08T08:53:54.240 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.240 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.240 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:54.240 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906349+0000", 2024-05-08T08:53:54.240 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1537000000000004e-05 2024-05-08T08:53:54.240 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.240 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.240 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:54.240 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906422+0000", 2024-05-08T08:53:54.240 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3431999999999998e-05 2024-05-08T08:53:54.240 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.240 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:54.240 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.240 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.240 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.241 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22095 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 946176~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:54.241 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:53:54.241 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 86.339800828999998, 2024-05-08T08:53:54.241 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048657593999999998, 2024-05-08T08:53:54.241 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:54.241 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:54.241 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:54.241 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:54.241 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:54.241 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:54.241 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22095 2024-05-08T08:53:54.241 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.241 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:54.241 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.242 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:54.242 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:53:54.242 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:54.242 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.242 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.242 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:54.242 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:53:54.242 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:54.242 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.242 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.242 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:54.242 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857925+0000", 2024-05-08T08:53:54.242 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3099999999999998e-07 2024-05-08T08:53:54.242 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.243 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.243 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:54.243 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857927+0000", 2024-05-08T08:53:54.243 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.34e-06 2024-05-08T08:53:54.243 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.243 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.243 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:54.243 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857938+0000", 2024-05-08T08:53:54.243 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1426e-05 2024-05-08T08:53:54.243 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.243 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.243 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:54.243 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857943+0000", 2024-05-08T08:53:54.244 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5140000000000003e-06 2024-05-08T08:53:54.244 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.244 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.244 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:54.244 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906448+0000", 2024-05-08T08:53:54.244 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048504641000000001 2024-05-08T08:53:54.244 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.244 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.244 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:54.244 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906510+0000", 2024-05-08T08:53:54.244 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2663000000000006e-05 2024-05-08T08:53:54.244 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.244 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.244 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:54.244 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906582+0000", 2024-05-08T08:53:54.245 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1478999999999998e-05 2024-05-08T08:53:54.245 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.245 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:54.245 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.245 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.245 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.245 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22097 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 962560~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:54.245 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:53:54.245 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 86.339773140000005, 2024-05-08T08:53:54.245 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048791353000000003, 2024-05-08T08:53:54.245 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:54.245 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:54.245 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:54.245 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:54.246 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:54.246 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:54.246 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22097 2024-05-08T08:53:54.246 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.246 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:54.246 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.246 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:54.246 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:53:54.246 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:54.246 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.246 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.246 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:54.246 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:53:54.247 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:54.247 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.247 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.247 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:54.247 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:53:54.247 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9200000000000002e-07 2024-05-08T08:53:54.247 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.247 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.247 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:54.247 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857954+0000", 2024-05-08T08:53:54.247 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1900000000000002e-06 2024-05-08T08:53:54.247 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.247 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.248 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:54.248 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857961+0000", 2024-05-08T08:53:54.248 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4989999999999999e-06 2024-05-08T08:53:54.248 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.248 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.248 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:54.248 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857971+0000", 2024-05-08T08:53:54.248 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0485999999999999e-05 2024-05-08T08:53:54.248 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.248 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.248 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:54.248 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906607+0000", 2024-05-08T08:53:54.248 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048635109000000003 2024-05-08T08:53:54.248 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.249 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.249 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:54.249 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906667+0000", 2024-05-08T08:53:54.249 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0439000000000002e-05 2024-05-08T08:53:54.249 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.249 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.249 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:54.249 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906743+0000", 2024-05-08T08:53:54.249 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6237999999999998e-05 2024-05-08T08:53:54.249 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.249 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:54.249 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.249 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.249 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.249 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22099 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 978944~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:54.250 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:53:54.250 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 86.339746980000001, 2024-05-08T08:53:54.250 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048932055000000002, 2024-05-08T08:53:54.250 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:54.250 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:54.250 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:54.250 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:54.250 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:54.250 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:54.250 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22099 2024-05-08T08:53:54.250 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.250 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:54.250 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.250 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:54.250 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:53:54.251 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:54.251 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.251 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.251 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:54.251 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:53:54.251 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:54.251 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.251 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.251 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:54.251 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:53:54.251 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.27e-07 2024-05-08T08:53:54.251 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.251 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.251 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:54.251 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857987+0000", 2024-05-08T08:53:54.252 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5890000000000003e-06 2024-05-08T08:53:54.252 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.252 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.252 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:54.252 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857994+0000", 2024-05-08T08:53:54.252 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6619999999999999e-06 2024-05-08T08:53:54.252 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.252 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.252 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:54.252 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857998+0000", 2024-05-08T08:53:54.252 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7110000000000003e-06 2024-05-08T08:53:54.252 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.252 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.252 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:54.252 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906772+0000", 2024-05-08T08:53:54.253 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048773415000000001 2024-05-08T08:53:54.253 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.253 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.253 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:54.253 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906834+0000", 2024-05-08T08:53:54.253 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1752000000000001e-05 2024-05-08T08:53:54.253 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.253 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.253 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:54.253 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906910+0000", 2024-05-08T08:53:54.253 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6499000000000001e-05 2024-05-08T08:53:54.253 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.253 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:54.253 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.253 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.253 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.254 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22101 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 995328~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:54.254 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:53:54.254 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 86.339719849000005, 2024-05-08T08:53:54.254 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049087364000000001, 2024-05-08T08:53:54.254 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:54.254 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:54.254 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:54.254 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:54.254 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:54.254 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:54.254 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22101 2024-05-08T08:53:54.254 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.254 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:54.255 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.255 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:54.255 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:53:54.255 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:54.255 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.255 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.255 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:54.255 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:53:54.255 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:54.255 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.255 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.255 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:54.255 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858006+0000", 2024-05-08T08:53:54.255 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1399999999999997e-07 2024-05-08T08:53:54.255 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.255 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.255 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:54.256 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858008+0000", 2024-05-08T08:53:54.256 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2060000000000001e-06 2024-05-08T08:53:54.256 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.256 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.256 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:54.256 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858017+0000", 2024-05-08T08:53:54.256 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1579999999999994e-06 2024-05-08T08:53:54.256 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.256 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.256 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:54.256 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858022+0000", 2024-05-08T08:53:54.256 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4199999999999998e-06 2024-05-08T08:53:54.256 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.256 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.257 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:54.257 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906940+0000", 2024-05-08T08:53:54.257 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048917627999999998 2024-05-08T08:53:54.257 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.257 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.257 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:54.257 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907002+0000", 2024-05-08T08:53:54.257 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2399999999999999e-05 2024-05-08T08:53:54.257 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.257 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.257 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:54.257 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907092+0000", 2024-05-08T08:53:54.257 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0137999999999997e-05 2024-05-08T08:53:54.257 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.257 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:54.257 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.257 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.258 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.258 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22103 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1011712~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:54.258 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:53:54.258 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 86.339695063999997, 2024-05-08T08:53:54.258 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049228542, 2024-05-08T08:53:54.258 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:54.258 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:54.258 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:54.258 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:54.258 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:54.258 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:54.258 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22103 2024-05-08T08:53:54.258 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.258 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:54.258 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.259 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:54.259 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:53:54.259 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:54.259 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.259 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.259 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:54.259 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:53:54.259 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:54.259 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.259 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.259 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:54.259 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:53:54.259 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4299999999999999e-07 2024-05-08T08:53:54.259 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.259 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.259 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:54.259 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858032+0000", 2024-05-08T08:53:54.260 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0590000000000001e-06 2024-05-08T08:53:54.260 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.260 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.260 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:54.260 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858037+0000", 2024-05-08T08:53:54.260 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.977e-06 2024-05-08T08:53:54.260 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.260 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.260 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:54.260 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858042+0000", 2024-05-08T08:53:54.260 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7650000000000001e-06 2024-05-08T08:53:54.260 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.260 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.260 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:54.260 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907118+0000", 2024-05-08T08:53:54.260 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049076147000000001 2024-05-08T08:53:54.261 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.261 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.261 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:54.261 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907180+0000", 2024-05-08T08:53:54.261 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1971999999999996e-05 2024-05-08T08:53:54.261 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.261 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.261 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:54.261 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907258+0000", 2024-05-08T08:53:54.261 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8279e-05 2024-05-08T08:53:54.261 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.261 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:54.261 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.261 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.261 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.261 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22105 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1028096~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:54.262 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:53:54.262 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 86.339675821, 2024-05-08T08:53:54.262 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049394609999999999, 2024-05-08T08:53:54.262 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:54.262 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:54.262 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:54.262 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:54.262 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:54.262 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:54.262 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22105 2024-05-08T08:53:54.262 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.262 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:54.262 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.262 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:54.263 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:53:54.263 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:54.263 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.263 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.263 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:54.263 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:53:54.263 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:54.263 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.263 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.263 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:54.263 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858050+0000", 2024-05-08T08:53:54.263 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5999999999999999e-07 2024-05-08T08:53:54.263 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.263 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.263 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:54.263 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858051+0000", 2024-05-08T08:53:54.264 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8870000000000001e-06 2024-05-08T08:53:54.264 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.264 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.264 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:54.264 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858057+0000", 2024-05-08T08:53:54.264 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8019999999999997e-06 2024-05-08T08:53:54.264 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.264 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.264 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:54.264 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858062+0000", 2024-05-08T08:53:54.264 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.561e-06 2024-05-08T08:53:54.264 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.264 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.264 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:54.264 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907289+0000", 2024-05-08T08:53:54.265 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049226957000000002 2024-05-08T08:53:54.265 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.265 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.265 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:54.265 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907356+0000", 2024-05-08T08:53:54.265 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6915000000000006e-05 2024-05-08T08:53:54.265 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.265 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.265 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:54.265 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907444+0000", 2024-05-08T08:53:54.265 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8127999999999994e-05 2024-05-08T08:53:54.265 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.265 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:54.265 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.265 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.265 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.265 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22107 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1044480~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:54.266 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:53:54.266 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 86.339657481000003, 2024-05-08T08:53:54.266 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049799942999999999, 2024-05-08T08:53:54.266 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:54.266 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:54.266 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:54.266 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:54.266 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:54.266 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:54.266 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22107 2024-05-08T08:53:54.266 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.266 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:54.266 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.266 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:54.266 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:53:54.267 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:54.267 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.267 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.267 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:54.267 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:53:54.267 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:54.267 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.267 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.267 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:54.267 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:53:54.267 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4799999999999999e-07 2024-05-08T08:53:54.267 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.267 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.267 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:54.267 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858070+0000", 2024-05-08T08:53:54.267 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.012e-06 2024-05-08T08:53:54.268 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.268 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.268 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:54.268 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858077+0000", 2024-05-08T08:53:54.268 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6290000000000001e-06 2024-05-08T08:53:54.268 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.268 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.268 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:54.268 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858083+0000", 2024-05-08T08:53:54.268 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8880000000000002e-06 2024-05-08T08:53:54.268 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.268 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.268 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:54.268 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907671+0000", 2024-05-08T08:53:54.268 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049587191000000003 2024-05-08T08:53:54.269 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.269 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.269 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:54.269 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907763+0000", 2024-05-08T08:53:54.269 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2231000000000005e-05 2024-05-08T08:53:54.269 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.269 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.269 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:54.269 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907867+0000", 2024-05-08T08:53:54.269 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104644 2024-05-08T08:53:54.269 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.269 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:54.269 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.269 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.269 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.270 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22294 3.1c 3:3aad0389:::rbd_data.11406886b07b.0000000000000080:head [stat,delete] snapc 7=[7] RETRY=6 ondisk+retry+write+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:53:54.270 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:53:54.270 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 86.339063632999995, 2024-05-08T08:53:54.270 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051152191, 2024-05-08T08:53:54.270 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:54.270 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:54.270 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:54.270 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:54.270 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:53:54.270 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:53:54.270 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22294 2024-05-08T08:53:54.270 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.270 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:54.270 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.270 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:54.271 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:53:54.271 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:54.271 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.271 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.271 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:54.271 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:53:54.271 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:54.271 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.271 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.271 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:54.271 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858662+0000", 2024-05-08T08:53:54.271 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4799999999999998e-07 2024-05-08T08:53:54.271 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.271 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.271 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:54.271 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858664+0000", 2024-05-08T08:53:54.271 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1229999999999998e-06 2024-05-08T08:53:54.271 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.272 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.272 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:54.272 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858670+0000", 2024-05-08T08:53:54.272 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3099999999999997e-06 2024-05-08T08:53:54.272 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.272 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.272 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:54.272 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858676+0000", 2024-05-08T08:53:54.272 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9750000000000004e-06 2024-05-08T08:53:54.272 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.272 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.272 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:54.272 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907901+0000", 2024-05-08T08:53:54.272 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049224520000000001 2024-05-08T08:53:54.272 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.272 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.273 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:54.273 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.908085+0000", 2024-05-08T08:53:54.273 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00018377500000000001 2024-05-08T08:53:54.273 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.273 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.273 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:54.273 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.909814+0000", 2024-05-08T08:53:54.273 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00172894 2024-05-08T08:53:54.273 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.273 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:54.273 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.273 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.273 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.273 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4416.0:13461 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-05-08T08:53:54.273 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:22.487456+0000", 2024-05-08T08:53:54.273 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 31.710268591999998, 2024-05-08T08:53:54.273 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.99102620799999996, 2024-05-08T08:53:54.274 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:54.274 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:54.274 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:54.274 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:54.274 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4416", 2024-05-08T08:53:54.274 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:54.274 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 13461 2024-05-08T08:53:54.274 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.274 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:54.274 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.274 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:54.274 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487456+0000", 2024-05-08T08:53:54.274 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:54.274 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.275 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.275 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:54.275 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487456+0000", 2024-05-08T08:53:54.275 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:54.275 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.275 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.275 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:54.275 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487458+0000", 2024-05-08T08:53:54.275 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5659999999999999e-06 2024-05-08T08:53:54.275 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.275 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.275 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:54.275 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487463+0000", 2024-05-08T08:53:54.275 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6890000000000003e-06 2024-05-08T08:53:54.275 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.276 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.276 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:54.276 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487472+0000", 2024-05-08T08:53:54.276 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7710000000000001e-06 2024-05-08T08:53:54.276 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.276 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.276 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:54.276 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487478+0000", 2024-05-08T08:53:54.276 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1390000000000003e-06 2024-05-08T08:53:54.276 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.276 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.276 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:54.276 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487520+0000", 2024-05-08T08:53:54.276 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2259999999999999e-05 2024-05-08T08:53:54.277 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.277 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.277 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-05-08T08:53:54.277 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487528+0000", 2024-05-08T08:53:54.277 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7479999999999993e-06 2024-05-08T08:53:54.277 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.277 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.277 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:54.277 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:23.478139+0000", 2024-05-08T08:53:54.277 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.99061165200000001 2024-05-08T08:53:54.277 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.277 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.277 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:54.277 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:23.478301+0000", 2024-05-08T08:53:54.277 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000161602 2024-05-08T08:53:54.277 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.278 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.278 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:54.278 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:23.478483+0000", 2024-05-08T08:53:54.278 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000181781 2024-05-08T08:53:54.278 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.278 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:54.278 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.278 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.278 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:54.278 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:53:54.280 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:53:54.280 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:54.282+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:54.380 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:54.383+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:54.425 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:53:54.426 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:53:54.426 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:53:54.426 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:53:54.434 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:53:54.481 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:53:54.484+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:53:54.579 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:53:54.580 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:53:54.580 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:53:54.580 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:53:54.580 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:53:54.582 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:54.585+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:54.588 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:53:54.593 INFO:teuthology.orchestra.run.smithi059.stderr:noscrub is unset 2024-05-08T08:53:54.597 INFO:teuthology.orchestra.run.smithi059.stderr:marked out osd.5. 2024-05-08T08:53:54.683 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:54.686+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:54.697 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:53:54.697 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:53:54.697 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:53:54.697 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:53:54.697 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.697 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6690 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1093632~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:54.697 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:53:54.697 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 82.984056764000002, 2024-05-08T08:53:54.697 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052462754, 2024-05-08T08:53:54.697 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:54.698 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:54.698 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:54.698 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:54.698 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:54.698 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:54.698 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6690 2024-05-08T08:53:54.698 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.698 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:54.698 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.698 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:54.698 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:53:54.698 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:54.698 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.698 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.699 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:54.699 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:53:54.699 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:54.699 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.699 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.699 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:54.699 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715346+0000", 2024-05-08T08:53:54.699 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.84e-07 2024-05-08T08:53:54.699 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.699 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.699 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:54.699 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715348+0000", 2024-05-08T08:53:54.699 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.114e-06 2024-05-08T08:53:54.699 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.699 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.700 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:54.700 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715353+0000", 2024-05-08T08:53:54.700 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5720000000000002e-06 2024-05-08T08:53:54.700 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.700 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.700 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:54.700 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715358+0000", 2024-05-08T08:53:54.700 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7140000000000001e-06 2024-05-08T08:53:54.700 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.700 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.700 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:54.700 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767638+0000", 2024-05-08T08:53:54.700 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052279562000000002 2024-05-08T08:53:54.700 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.700 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.701 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:54.701 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767713+0000", 2024-05-08T08:53:54.701 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5652e-05 2024-05-08T08:53:54.701 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.701 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.701 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:54.701 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767808+0000", 2024-05-08T08:53:54.701 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4755999999999999e-05 2024-05-08T08:53:54.701 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.701 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:54.701 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.701 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.701 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.701 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6692 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1110016~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:54.701 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:53:54.702 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 82.984037674000007, 2024-05-08T08:53:54.702 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052666495000000001, 2024-05-08T08:53:54.702 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:54.702 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:54.702 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:54.702 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:54.702 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:54.702 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:54.702 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6692 2024-05-08T08:53:54.702 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.702 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:54.702 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.702 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:54.702 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:53:54.703 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:54.703 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.703 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.703 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:54.703 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:53:54.703 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:54.703 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.703 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.703 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:54.703 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:53:54.703 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8599999999999999e-07 2024-05-08T08:53:54.703 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.703 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.703 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:54.703 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715372+0000", 2024-05-08T08:53:54.704 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0990000000000001e-06 2024-05-08T08:53:54.704 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.704 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.704 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:54.704 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715378+0000", 2024-05-08T08:53:54.704 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0680000000000002e-06 2024-05-08T08:53:54.704 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.704 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.704 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:54.704 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715383+0000", 2024-05-08T08:53:54.704 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8740000000000003e-06 2024-05-08T08:53:54.704 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.704 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.704 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:54.704 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767848+0000", 2024-05-08T08:53:54.705 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052464668999999998 2024-05-08T08:53:54.705 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.705 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.705 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:54.705 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767927+0000", 2024-05-08T08:53:54.705 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9228999999999996e-05 2024-05-08T08:53:54.705 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.705 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.705 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:54.705 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768031+0000", 2024-05-08T08:53:54.705 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010417 2024-05-08T08:53:54.705 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.705 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:54.705 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.706 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.706 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.706 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6694 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1126400~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:54.706 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:53:54.706 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 82.984012777000004, 2024-05-08T08:53:54.706 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052848829, 2024-05-08T08:53:54.706 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:54.706 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:54.706 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:54.706 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:54.706 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:54.706 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:54.706 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6694 2024-05-08T08:53:54.706 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.706 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:54.707 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.707 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:54.707 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:53:54.707 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:54.707 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.707 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.707 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:54.707 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:53:54.707 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:54.707 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.707 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.707 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:54.707 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715390+0000", 2024-05-08T08:53:54.708 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9900000000000001e-07 2024-05-08T08:53:54.708 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.708 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.708 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:54.708 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715392+0000", 2024-05-08T08:53:54.708 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8279999999999999e-06 2024-05-08T08:53:54.708 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.708 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.708 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:54.708 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715398+0000", 2024-05-08T08:53:54.708 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7619999999999998e-06 2024-05-08T08:53:54.708 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.708 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.708 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:54.708 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715403+0000", 2024-05-08T08:53:54.708 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7790000000000003e-06 2024-05-08T08:53:54.709 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.709 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.709 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:54.709 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768067+0000", 2024-05-08T08:53:54.709 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052663785999999997 2024-05-08T08:53:54.709 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.709 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.709 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:54.709 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768142+0000", 2024-05-08T08:53:54.709 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5018999999999994e-05 2024-05-08T08:53:54.709 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.709 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.709 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:54.709 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768238+0000", 2024-05-08T08:53:54.709 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6255999999999995e-05 2024-05-08T08:53:54.710 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.710 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:54.710 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.710 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.710 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.710 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6696 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1142784~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:54.710 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:53:54.710 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 82.983993268999996, 2024-05-08T08:53:54.710 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.053051057999999998, 2024-05-08T08:53:54.710 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:54.710 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:54.710 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:54.710 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:54.710 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:54.711 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:54.711 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6696 2024-05-08T08:53:54.711 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.711 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:54.711 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.711 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:54.711 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:53:54.711 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:54.711 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.711 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.711 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:54.711 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:53:54.711 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:54.711 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.711 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.712 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:54.712 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:53:54.712 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-05-08T08:53:54.712 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.712 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.712 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:54.712 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715411+0000", 2024-05-08T08:53:54.712 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9630000000000001e-06 2024-05-08T08:53:54.712 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.712 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.712 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:54.712 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715417+0000", 2024-05-08T08:53:54.712 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7250000000000002e-06 2024-05-08T08:53:54.712 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.712 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.713 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:54.713 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715421+0000", 2024-05-08T08:53:54.713 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4089999999999996e-06 2024-05-08T08:53:54.713 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.713 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.713 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:54.713 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768277+0000", 2024-05-08T08:53:54.713 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052855293999999997 2024-05-08T08:53:54.713 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.713 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.713 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:54.713 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768358+0000", 2024-05-08T08:53:54.713 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1720999999999999e-05 2024-05-08T08:53:54.713 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.713 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.714 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:54.714 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768460+0000", 2024-05-08T08:53:54.714 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010154600000000001 2024-05-08T08:53:54.714 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.714 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:54.714 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.714 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.714 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.714 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6698 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1159168~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:54.714 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:53:54.714 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 82.983974402000001, 2024-05-08T08:53:54.714 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.053245726, 2024-05-08T08:53:54.714 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:54.714 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:54.715 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:54.715 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:54.715 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:54.715 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:54.715 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6698 2024-05-08T08:53:54.715 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.715 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:54.715 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.715 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:54.715 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:53:54.715 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:54.715 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.715 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.715 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:54.716 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:53:54.716 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:54.716 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.716 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.716 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:54.716 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716936+0000", 2024-05-08T08:53:54.716 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001508665 2024-05-08T08:53:54.716 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.716 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.716 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:54.716 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716941+0000", 2024-05-08T08:53:54.716 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1910000000000002e-06 2024-05-08T08:53:54.716 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.716 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.716 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:54.717 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716950+0000", 2024-05-08T08:53:54.717 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1900000000000001e-06 2024-05-08T08:53:54.717 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.717 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.717 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:54.717 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716958+0000", 2024-05-08T08:53:54.717 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8019999999999992e-06 2024-05-08T08:53:54.717 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.717 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.717 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:54.717 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768499+0000", 2024-05-08T08:53:54.717 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051540926000000001 2024-05-08T08:53:54.717 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.717 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.717 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:54.718 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768574+0000", 2024-05-08T08:53:54.718 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4988999999999996e-05 2024-05-08T08:53:54.718 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.718 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.718 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:54.718 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768673+0000", 2024-05-08T08:53:54.718 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9963000000000005e-05 2024-05-08T08:53:54.718 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.718 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:54.718 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.718 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.718 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.718 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6706 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1224704~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:54.718 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:53:54.718 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 82.982369094999996, 2024-05-08T08:53:54.719 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052435598, 2024-05-08T08:53:54.719 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:54.719 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:54.719 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:54.719 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:54.719 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:54.719 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:54.719 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6706 2024-05-08T08:53:54.719 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.719 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:54.719 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.719 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:54.719 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:53:54.719 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:54.719 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.720 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.720 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:54.720 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:53:54.720 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:54.720 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.720 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.720 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:54.720 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:53:54.720 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:53:54.720 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.720 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.720 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:54.720 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717035+0000", 2024-05-08T08:53:54.720 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9290000000000001e-06 2024-05-08T08:53:54.720 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.721 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.721 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:54.721 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717041+0000", 2024-05-08T08:53:54.721 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6720000000000001e-06 2024-05-08T08:53:54.721 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.721 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.721 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:54.721 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717046+0000", 2024-05-08T08:53:54.721 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5650000000000003e-06 2024-05-08T08:53:54.721 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.721 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.721 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:54.721 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769311+0000", 2024-05-08T08:53:54.721 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052265645999999999 2024-05-08T08:53:54.721 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.722 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.722 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:54.722 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769379+0000", 2024-05-08T08:53:54.722 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7391000000000006e-05 2024-05-08T08:53:54.722 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.722 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.722 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:54.722 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769469+0000", 2024-05-08T08:53:54.722 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0025999999999995e-05 2024-05-08T08:53:54.722 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.722 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:54.722 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.722 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.722 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.722 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6708 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1241088~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:54.723 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:53:54.723 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 82.982350991000004, 2024-05-08T08:53:54.723 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052640552, 2024-05-08T08:53:54.723 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:54.723 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:54.723 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:54.723 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:54.723 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:54.723 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:54.723 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6708 2024-05-08T08:53:54.723 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.723 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:54.723 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.723 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:54.724 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:53:54.724 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:54.724 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.724 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.724 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:54.724 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:53:54.724 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:54.724 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.724 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.724 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:54.724 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717052+0000", 2024-05-08T08:53:54.724 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:53:54.724 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.724 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.724 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:54.725 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717053+0000", 2024-05-08T08:53:54.725 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8330000000000001e-06 2024-05-08T08:53:54.725 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.725 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.725 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:54.725 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717058+0000", 2024-05-08T08:53:54.725 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.933e-06 2024-05-08T08:53:54.725 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.725 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.725 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:54.725 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717063+0000", 2024-05-08T08:53:54.725 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2180000000000001e-06 2024-05-08T08:53:54.725 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.725 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.725 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:54.726 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769507+0000", 2024-05-08T08:53:54.726 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052444866999999999 2024-05-08T08:53:54.726 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.726 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.726 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:54.726 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769589+0000", 2024-05-08T08:53:54.726 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2039000000000005e-05 2024-05-08T08:53:54.726 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.726 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.726 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:54.730 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769692+0000", 2024-05-08T08:53:54.730 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102252 2024-05-08T08:53:54.730 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.730 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:54.730 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.730 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.730 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.730 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6710 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1257472~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:54.730 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:53:54.730 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 82.982332967000005, 2024-05-08T08:53:54.731 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.05283057, 2024-05-08T08:53:54.731 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:54.731 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:54.731 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:54.731 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:54.731 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:54.731 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:54.731 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6710 2024-05-08T08:53:54.731 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.731 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:54.731 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.731 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:54.731 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:53:54.731 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:54.732 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.732 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.732 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:54.732 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:53:54.732 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:54.732 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.732 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.732 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:54.732 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718951+0000", 2024-05-08T08:53:54.732 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0018817809999999999 2024-05-08T08:53:54.732 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.732 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.732 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:54.732 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718956+0000", 2024-05-08T08:53:54.733 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5399999999999997e-06 2024-05-08T08:53:54.733 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.733 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.733 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:54.733 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718964+0000", 2024-05-08T08:53:54.733 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6629999999999999e-06 2024-05-08T08:53:54.733 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.733 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.733 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:54.733 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718972+0000", 2024-05-08T08:53:54.733 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5229999999999998e-06 2024-05-08T08:53:54.733 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.733 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.733 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:54.733 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769729+0000", 2024-05-08T08:53:54.734 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050756959999999997 2024-05-08T08:53:54.734 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.734 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.734 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:54.734 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769803+0000", 2024-05-08T08:53:54.734 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4538999999999999e-05 2024-05-08T08:53:54.734 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.734 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.734 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:54.734 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769900+0000", 2024-05-08T08:53:54.734 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6564000000000006e-05 2024-05-08T08:53:54.734 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.734 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:54.734 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.735 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.735 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.735 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6726 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1388544~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:54.735 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:53:54.735 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 82.980258195000005, 2024-05-08T08:53:54.735 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052451506000000002, 2024-05-08T08:53:54.735 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:54.735 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:54.735 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:54.735 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:54.735 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:54.735 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:54.735 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6726 2024-05-08T08:53:54.735 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.735 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:54.736 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.736 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:54.736 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:53:54.736 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:54.736 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.736 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.736 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:54.736 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:53:54.736 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:54.736 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.736 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.736 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:54.736 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719200+0000", 2024-05-08T08:53:54.736 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5865999999999997e-05 2024-05-08T08:53:54.737 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.737 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.737 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:54.737 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719204+0000", 2024-05-08T08:53:54.737 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8630000000000001e-06 2024-05-08T08:53:54.737 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.737 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.737 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:54.737 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719215+0000", 2024-05-08T08:53:54.737 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1590000000000001e-05 2024-05-08T08:53:54.737 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.737 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.737 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:54.737 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719220+0000", 2024-05-08T08:53:54.737 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1619999999999999e-06 2024-05-08T08:53:54.738 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.738 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.738 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:54.738 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771425+0000", 2024-05-08T08:53:54.738 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052204249000000001 2024-05-08T08:53:54.738 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.738 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.738 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:54.738 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771498+0000", 2024-05-08T08:53:54.738 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3782999999999995e-05 2024-05-08T08:53:54.738 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.738 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.738 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:54.738 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771595+0000", 2024-05-08T08:53:54.738 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6992999999999998e-05 2024-05-08T08:53:54.739 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.739 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:54.739 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.739 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.739 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.739 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6728 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1404928~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:54.739 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:53:54.739 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 82.980170790000003, 2024-05-08T08:53:54.739 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052590747, 2024-05-08T08:53:54.739 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:54.739 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:54.739 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:54.739 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:54.739 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:54.739 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:54.740 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6728 2024-05-08T08:53:54.740 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.740 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:54.740 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.740 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:54.740 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:53:54.740 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:54.740 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.740 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.740 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:54.740 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:53:54.740 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:54.740 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.740 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.740 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:54.740 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719232+0000", 2024-05-08T08:53:54.741 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4000000000000001e-07 2024-05-08T08:53:54.741 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.741 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.741 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:54.741 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719234+0000", 2024-05-08T08:53:54.741 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4499999999999998e-06 2024-05-08T08:53:54.741 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.741 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.741 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:54.741 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719240+0000", 2024-05-08T08:53:54.741 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5720000000000002e-06 2024-05-08T08:53:54.741 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.741 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.741 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:54.741 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719245+0000", 2024-05-08T08:53:54.742 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8690000000000003e-06 2024-05-08T08:53:54.742 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.742 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.742 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:54.742 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771636+0000", 2024-05-08T08:53:54.742 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052390979999999997 2024-05-08T08:53:54.742 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.742 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.742 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:54.742 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771717+0000", 2024-05-08T08:53:54.742 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1291999999999993e-05 2024-05-08T08:53:54.742 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.742 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.742 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:54.743 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771822+0000", 2024-05-08T08:53:54.743 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104944 2024-05-08T08:53:54.743 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.743 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:54.743 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.743 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.743 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.743 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6730 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1421312~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:54.743 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:53:54.743 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 82.980150547999997, 2024-05-08T08:53:54.743 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052770477000000003, 2024-05-08T08:53:54.743 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:54.743 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:54.743 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:54.743 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:54.743 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:54.744 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:54.744 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6730 2024-05-08T08:53:54.744 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.744 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:54.744 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.744 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:54.744 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:53:54.744 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:54.744 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.744 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.744 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:54.744 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:53:54.744 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:54.744 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.744 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.744 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:54.745 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:53:54.745 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9000000000000002e-07 2024-05-08T08:53:54.745 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.745 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.745 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:54.745 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719254+0000", 2024-05-08T08:53:54.745 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4099999999999998e-06 2024-05-08T08:53:54.745 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.745 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.745 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:54.745 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719263+0000", 2024-05-08T08:53:54.745 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4109999999999996e-06 2024-05-08T08:53:54.745 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.745 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.745 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:54.746 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719268+0000", 2024-05-08T08:53:54.746 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2669999999999997e-06 2024-05-08T08:53:54.746 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.746 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.746 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:54.746 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771846+0000", 2024-05-08T08:53:54.746 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052577677000000003 2024-05-08T08:53:54.746 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.746 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.746 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:54.746 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771920+0000", 2024-05-08T08:53:54.746 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4460000000000002e-05 2024-05-08T08:53:54.746 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.747 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.747 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:54.747 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772022+0000", 2024-05-08T08:53:54.747 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010186199999999999 2024-05-08T08:53:54.747 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.747 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:54.747 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.747 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.747 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.747 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6732 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1437696~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:54.747 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:53:54.747 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 82.980125740999995, 2024-05-08T08:53:54.747 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052905575000000003, 2024-05-08T08:53:54.747 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:54.747 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:54.748 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:54.748 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:54.748 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:54.748 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:54.748 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6732 2024-05-08T08:53:54.748 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.748 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:54.748 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.748 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:54.748 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:53:54.748 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:54.748 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.748 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.748 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:54.748 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:53:54.749 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:54.749 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.749 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.749 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:54.749 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721392+0000", 2024-05-08T08:53:54.749 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002115287 2024-05-08T08:53:54.749 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.749 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.749 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:54.749 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721396+0000", 2024-05-08T08:53:54.749 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1679999999999997e-06 2024-05-08T08:53:54.749 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.749 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.749 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:54.750 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721404+0000", 2024-05-08T08:53:54.750 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4409999999999993e-06 2024-05-08T08:53:54.750 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.750 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.750 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:54.750 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721410+0000", 2024-05-08T08:53:54.750 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9150000000000001e-06 2024-05-08T08:53:54.750 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.750 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.750 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:54.750 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772061+0000", 2024-05-08T08:53:54.750 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050651068 2024-05-08T08:53:54.750 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.750 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.751 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:54.751 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772127+0000", 2024-05-08T08:53:54.751 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5424000000000002e-05 2024-05-08T08:53:54.751 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.751 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.751 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:54.751 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772182+0000", 2024-05-08T08:53:54.751 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5272000000000002e-05 2024-05-08T08:53:54.751 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.751 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:54.751 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.751 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.751 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.751 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6749 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1585152~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:54.751 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:53:54.752 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 82.977824971000004, 2024-05-08T08:53:54.752 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052350264000000001, 2024-05-08T08:53:54.752 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:54.752 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:54.752 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:54.752 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:54.752 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:54.752 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:54.752 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6749 2024-05-08T08:53:54.752 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.752 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:54.752 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.752 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:54.752 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:53:54.752 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:54.753 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.753 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.753 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:54.753 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:53:54.753 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:54.753 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.753 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.753 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:54.753 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721578+0000", 2024-05-08T08:53:54.753 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0399999999999996e-07 2024-05-08T08:53:54.753 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.753 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.753 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:54.753 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721580+0000", 2024-05-08T08:53:54.753 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5450000000000002e-06 2024-05-08T08:53:54.754 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.754 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.754 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:54.754 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721587+0000", 2024-05-08T08:53:54.754 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0190000000000001e-06 2024-05-08T08:53:54.754 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.754 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.754 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:54.754 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721593+0000", 2024-05-08T08:53:54.754 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5659999999999997e-06 2024-05-08T08:53:54.754 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.754 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.754 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:54.755 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773759+0000", 2024-05-08T08:53:54.755 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052166553999999997 2024-05-08T08:53:54.755 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.755 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.755 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:54.755 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773833+0000", 2024-05-08T08:53:54.755 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3913999999999997e-05 2024-05-08T08:53:54.755 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.755 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.755 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:54.755 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773927+0000", 2024-05-08T08:53:54.755 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4061999999999995e-05 2024-05-08T08:53:54.755 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.755 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:54.755 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.755 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.756 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.756 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6751 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1601536~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:54.756 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:53:54.756 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 82.977801692, 2024-05-08T08:53:54.756 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052461302000000001, 2024-05-08T08:53:54.756 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:54.756 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:54.756 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:54.756 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:54.756 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:54.756 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:54.756 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6751 2024-05-08T08:53:54.756 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.756 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:54.756 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.756 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:54.756 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:53:54.757 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:54.757 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.757 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.757 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:54.757 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:53:54.757 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:54.757 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.757 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.757 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:54.757 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722939+0000", 2024-05-08T08:53:54.757 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001338601 2024-05-08T08:53:54.757 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.757 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.757 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:54.757 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722944+0000", 2024-05-08T08:53:54.757 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4469999999999999e-06 2024-05-08T08:53:54.758 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.758 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.758 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:54.758 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722955+0000", 2024-05-08T08:53:54.758 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1892999999999999e-05 2024-05-08T08:53:54.758 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.758 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.758 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:54.758 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722963+0000", 2024-05-08T08:53:54.758 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1080000000000004e-06 2024-05-08T08:53:54.758 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.758 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.758 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:54.758 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773964+0000", 2024-05-08T08:53:54.758 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051001022 2024-05-08T08:53:54.759 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.759 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.759 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:54.759 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.774006+0000", 2024-05-08T08:53:54.759 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2729999999999999e-05 2024-05-08T08:53:54.759 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.759 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.759 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:54.759 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.774062+0000", 2024-05-08T08:53:54.759 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5501000000000003e-05 2024-05-08T08:53:54.759 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.759 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:54.759 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.759 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.759 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.759 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6760 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1683456~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:54.759 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:53:54.760 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 82.976292315999999, 2024-05-08T08:53:54.760 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052571312000000002, 2024-05-08T08:53:54.760 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:54.760 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:54.760 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:54.760 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:54.760 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:54.760 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:54.760 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6760 2024-05-08T08:53:54.760 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.760 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:54.760 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.760 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:54.760 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:53:54.761 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:54.761 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.761 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.761 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:54.761 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:53:54.761 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:54.761 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.761 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.761 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:54.761 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:53:54.761 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2100000000000002e-07 2024-05-08T08:53:54.761 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.761 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.761 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:54.761 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723113+0000", 2024-05-08T08:53:54.761 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3149999999999999e-06 2024-05-08T08:53:54.761 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.762 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.762 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:54.762 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723118+0000", 2024-05-08T08:53:54.762 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8860000000000001e-06 2024-05-08T08:53:54.762 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.762 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.762 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:54.762 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723123+0000", 2024-05-08T08:53:54.762 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6770000000000003e-06 2024-05-08T08:53:54.762 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.762 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.762 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:54.762 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775414+0000", 2024-05-08T08:53:54.762 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052290692999999999 2024-05-08T08:53:54.762 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.763 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.763 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:54.763 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775459+0000", 2024-05-08T08:53:54.763 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5639999999999997e-05 2024-05-08T08:53:54.763 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.763 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.763 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:54.763 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775681+0000", 2024-05-08T08:53:54.763 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00022168000000000001 2024-05-08T08:53:54.763 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.763 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:54.763 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.763 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.763 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.763 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6761 3.16 3:6bd6d6da:::rbd_data.1140782590a5.0000000000000015:6 [read 1095680~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:54.763 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:53:54.764 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 82.976273096, 2024-05-08T08:53:54.764 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052751275, 2024-05-08T08:53:54.764 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:54.764 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:54.764 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:54.764 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:54.764 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:54.764 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:54.764 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6761 2024-05-08T08:53:54.764 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.764 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:54.764 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.764 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:54.764 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:53:54.764 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:54.764 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.765 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.765 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:54.765 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:53:54.765 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:54.765 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.765 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.765 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:54.765 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724939+0000", 2024-05-08T08:53:54.765 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001809817 2024-05-08T08:53:54.765 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.765 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.765 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:54.765 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724943+0000", 2024-05-08T08:53:54.765 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4920000000000004e-06 2024-05-08T08:53:54.765 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.765 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.765 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:54.766 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724956+0000", 2024-05-08T08:53:54.766 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2427e-05 2024-05-08T08:53:54.766 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.766 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.766 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:54.766 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724963+0000", 2024-05-08T08:53:54.766 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2559999999999996e-06 2024-05-08T08:53:54.766 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.766 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.766 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:54.766 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775711+0000", 2024-05-08T08:53:54.766 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050747621 2024-05-08T08:53:54.766 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.766 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.766 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:54.767 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775785+0000", 2024-05-08T08:53:54.767 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4634000000000004e-05 2024-05-08T08:53:54.767 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.767 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.767 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:54.767 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775880+0000", 2024-05-08T08:53:54.767 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5027999999999999e-05 2024-05-08T08:53:54.767 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.767 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:54.767 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.767 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.767 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.767 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6770 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1765376~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:54.767 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:53:54.767 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 82.974242000999993, 2024-05-08T08:53:54.767 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052339296, 2024-05-08T08:53:54.767 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:54.768 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:54.768 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:54.768 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:54.768 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:54.768 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:54.768 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6770 2024-05-08T08:53:54.768 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.768 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:54.768 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.768 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:54.768 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:53:54.768 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:54.768 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.768 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.769 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:54.769 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:53:54.769 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:54.769 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.769 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.769 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:54.769 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725161+0000", 2024-05-08T08:53:54.769 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-05-08T08:53:54.769 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.769 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.769 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:54.769 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725163+0000", 2024-05-08T08:53:54.769 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4370000000000001e-06 2024-05-08T08:53:54.769 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.769 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.769 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:54.769 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725169+0000", 2024-05-08T08:53:54.770 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8409999999999996e-06 2024-05-08T08:53:54.770 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.770 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.770 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:54.770 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725174+0000", 2024-05-08T08:53:54.770 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7249999999999997e-06 2024-05-08T08:53:54.770 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.770 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.770 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:54.770 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777343+0000", 2024-05-08T08:53:54.770 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052169633999999999 2024-05-08T08:53:54.770 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.770 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.770 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:54.770 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777409+0000", 2024-05-08T08:53:54.770 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6033999999999998e-05 2024-05-08T08:53:54.771 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.771 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.771 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:54.771 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777499+0000", 2024-05-08T08:53:54.771 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0224999999999998e-05 2024-05-08T08:53:54.771 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.771 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:54.771 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.771 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.771 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.771 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6771 3.16 3:6bd6d6da:::rbd_data.1140782590a5.0000000000000015:6 [read 1177600~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:54.771 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:53:54.771 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 82.974222183999998, 2024-05-08T08:53:54.771 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052438604999999999, 2024-05-08T08:53:54.771 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:54.771 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:54.771 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:54.772 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:54.772 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:54.772 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:54.772 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6771 2024-05-08T08:53:54.772 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.772 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:54.772 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.772 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:54.772 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:53:54.772 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:54.772 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.772 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.772 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:54.772 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:53:54.773 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:54.773 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.773 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.773 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:54.773 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:53:54.773 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:53:54.773 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.773 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.773 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:54.773 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725188+0000", 2024-05-08T08:53:54.773 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1579999999999999e-06 2024-05-08T08:53:54.773 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.773 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.773 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:54.773 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725195+0000", 2024-05-08T08:53:54.773 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8909999999999995e-06 2024-05-08T08:53:54.774 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.774 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.774 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:54.774 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725200+0000", 2024-05-08T08:53:54.774 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9810000000000003e-06 2024-05-08T08:53:54.774 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.774 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.774 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:54.774 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777527+0000", 2024-05-08T08:53:54.774 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052326582000000003 2024-05-08T08:53:54.774 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.774 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.774 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:54.774 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777566+0000", 2024-05-08T08:53:54.774 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9447999999999999e-05 2024-05-08T08:53:54.775 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.775 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.775 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:54.775 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777619+0000", 2024-05-08T08:53:54.775 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2135000000000001e-05 2024-05-08T08:53:54.775 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.775 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:54.775 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.775 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.775 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.775 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6772 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1781760~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:54.775 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:53:54.775 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 82.974194127000004, 2024-05-08T08:53:54.775 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052601799999999997, 2024-05-08T08:53:54.775 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:54.776 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:54.776 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:54.776 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:54.776 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:54.776 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:54.776 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6772 2024-05-08T08:53:54.776 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.776 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:54.776 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.776 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:54.776 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:53:54.776 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:54.776 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.776 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.776 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:54.776 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:53:54.776 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:54.776 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.777 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.777 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:54.777 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726939+0000", 2024-05-08T08:53:54.777 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001730493 2024-05-08T08:53:54.777 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.777 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.777 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:54.777 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726943+0000", 2024-05-08T08:53:54.777 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7330000000000003e-06 2024-05-08T08:53:54.777 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.777 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.777 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:54.777 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726954+0000", 2024-05-08T08:53:54.777 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1107000000000001e-05 2024-05-08T08:53:54.777 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.778 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.778 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:54.778 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726962+0000", 2024-05-08T08:53:54.778 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.503e-06 2024-05-08T08:53:54.778 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.778 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.778 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:54.778 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777653+0000", 2024-05-08T08:53:54.778 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050691542999999999 2024-05-08T08:53:54.778 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.778 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.778 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:54.778 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777719+0000", 2024-05-08T08:53:54.778 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5228000000000006e-05 2024-05-08T08:53:54.778 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.779 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.779 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:54.779 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777810+0000", 2024-05-08T08:53:54.779 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1193000000000006e-05 2024-05-08T08:53:54.779 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.779 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:54.779 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.779 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.779 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.779 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6782 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1880064~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:53:54.779 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:53:54.779 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 82.972225620000003, 2024-05-08T08:53:54.779 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052326492000000002, 2024-05-08T08:53:54.779 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:54.779 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:54.779 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:54.780 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:54.780 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:53:54.780 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:54.780 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6782 2024-05-08T08:53:54.780 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.780 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:54.780 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.780 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:54.780 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:53:54.780 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:54.780 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.780 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.780 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:54.780 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:53:54.780 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:54.780 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.780 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.781 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:54.781 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728939+0000", 2024-05-08T08:53:54.781 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001762719 2024-05-08T08:53:54.781 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.781 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.781 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:54.781 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728944+0000", 2024-05-08T08:53:54.781 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.746e-06 2024-05-08T08:53:54.781 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.781 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.781 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:54.781 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728955+0000", 2024-05-08T08:53:54.781 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1043000000000001e-05 2024-05-08T08:53:54.781 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.781 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.781 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:54.781 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728963+0000", 2024-05-08T08:53:54.782 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5809999999999999e-06 2024-05-08T08:53:54.782 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.782 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.782 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:54.782 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779320+0000", 2024-05-08T08:53:54.782 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050357290999999998 2024-05-08T08:53:54.782 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.782 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.782 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:54.782 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779401+0000", 2024-05-08T08:53:54.782 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0574000000000005e-05 2024-05-08T08:53:54.782 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:54.782 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:54.782 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:54.782 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779503+0000", 2024-05-08T08:53:54.783 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102538 2024-05-08T08:53:54.783 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.783 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:54.783 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.783 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:54.783 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:54.783 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:53:54.785 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:54.787+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:54.785 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:53:54.885 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:54.888+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:54.905 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:54.905 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:53:54.905 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:53:54.906 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:54.915 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:53:54.986 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:54.989+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:55.025 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:55.025 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:53:55.025 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:53:55.025 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:53:55.025 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:55.034 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:53:55.118 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:55.121+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:55.186 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:55.186 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:53:55.186 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:53:55.186 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:53:55.186 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.186 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4621.0:9089 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 23339520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:53:55.186 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:53:55.186 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 82.694756084999995, 2024-05-08T08:53:55.186 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.13814907200000001, 2024-05-08T08:53:55.186 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:55.186 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:55.187 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:53:55.187 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:55.187 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4621", 2024-05-08T08:53:55.187 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:53:55.187 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9089 2024-05-08T08:53:55.187 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.187 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:55.187 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.187 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:55.187 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:53:55.187 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:55.187 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.187 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.187 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:55.187 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:53:55.188 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:55.188 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.188 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.188 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:55.188 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493005+0000", 2024-05-08T08:53:55.188 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1700000000000001e-06 2024-05-08T08:53:55.188 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.188 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.188 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:55.188 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.513963+0000", 2024-05-08T08:53:55.188 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.020957581999999999 2024-05-08T08:53:55.188 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.188 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.188 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:55.189 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.513995+0000", 2024-05-08T08:53:55.189 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1993999999999999e-05 2024-05-08T08:53:55.189 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.189 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.189 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:55.189 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514007+0000", 2024-05-08T08:53:55.189 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1581e-05 2024-05-08T08:53:55.189 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.189 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.189 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:55.189 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514122+0000", 2024-05-08T08:53:55.189 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000115282 2024-05-08T08:53:55.189 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.189 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.190 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op must wait for map", 2024-05-08T08:53:55.190 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514128+0000", 2024-05-08T08:53:55.190 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9950000000000002e-06 2024-05-08T08:53:55.190 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.190 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.190 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:55.190 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.515800+0000", 2024-05-08T08:53:55.190 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001671886 2024-05-08T08:53:55.190 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.190 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.190 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:55.190 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.515893+0000", 2024-05-08T08:53:55.190 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3490000000000001e-05 2024-05-08T08:53:55.190 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.191 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.191 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 3", 2024-05-08T08:53:55.191 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.517647+0000", 2024-05-08T08:53:55.191 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0017535630000000001 2024-05-08T08:53:55.191 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.191 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.191 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:53:55.191 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.627578+0000", 2024-05-08T08:53:55.191 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.109931557 2024-05-08T08:53:55.191 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.191 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.191 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:53:55.191 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631055+0000", 2024-05-08T08:53:55.191 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0034761420000000002 2024-05-08T08:53:55.192 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.192 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.192 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:53:55.192 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631096+0000", 2024-05-08T08:53:55.192 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1059000000000001e-05 2024-05-08T08:53:55.192 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.192 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.192 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:55.192 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631151+0000", 2024-05-08T08:53:55.192 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5770999999999998e-05 2024-05-08T08:53:55.192 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.192 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:55.192 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.192 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.192 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.192 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1071 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11026432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:55.192 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:53:55.193 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 78.702306594000007, 2024-05-08T08:53:55.193 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070580037999999998, 2024-05-08T08:53:55.193 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:55.193 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:55.193 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:55.193 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:55.193 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:55.193 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:55.193 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1071 2024-05-08T08:53:55.193 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.193 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:55.193 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.193 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:55.194 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:53:55.194 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:55.194 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.194 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.194 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:55.194 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:53:55.194 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:55.194 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.194 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.194 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:55.194 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:53:55.194 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2800000000000002e-07 2024-05-08T08:53:55.194 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.194 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.194 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:55.195 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485454+0000", 2024-05-08T08:53:55.195 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2290000000000002e-06 2024-05-08T08:53:55.195 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.195 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.195 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:55.195 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485461+0000", 2024-05-08T08:53:55.195 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9920000000000001e-06 2024-05-08T08:53:55.195 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.195 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.195 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:55.195 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485467+0000", 2024-05-08T08:53:55.195 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3979999999999998e-06 2024-05-08T08:53:55.195 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.195 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.196 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:55.196 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555880+0000", 2024-05-08T08:53:55.196 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070413165 2024-05-08T08:53:55.196 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.196 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.196 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:55.196 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555926+0000", 2024-05-08T08:53:55.196 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6217000000000002e-05 2024-05-08T08:53:55.196 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.196 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.196 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:55.196 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556032+0000", 2024-05-08T08:53:55.196 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000105609 2024-05-08T08:53:55.196 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.196 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:55.197 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.197 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.197 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.197 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1073 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11067392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:55.197 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:53:55.197 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 78.702285372999995, 2024-05-08T08:53:55.197 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070821404000000004, 2024-05-08T08:53:55.197 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:55.197 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:55.197 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:55.197 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:55.197 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:55.197 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:55.197 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1073 2024-05-08T08:53:55.197 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.197 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:55.198 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.198 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:55.198 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:53:55.198 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:55.198 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.198 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.198 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:55.198 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:53:55.198 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:55.198 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.198 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.198 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:55.198 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:53:55.199 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.08e-07 2024-05-08T08:53:55.199 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.199 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.199 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:55.199 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485476+0000", 2024-05-08T08:53:55.199 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3929999999999998e-06 2024-05-08T08:53:55.199 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.199 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.199 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:55.199 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485485+0000", 2024-05-08T08:53:55.199 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1880000000000008e-06 2024-05-08T08:53:55.199 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.199 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.199 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:55.200 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485494+0000", 2024-05-08T08:53:55.200 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3109999999999995e-06 2024-05-08T08:53:55.200 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.200 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.200 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:55.200 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556082+0000", 2024-05-08T08:53:55.200 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070587426999999994 2024-05-08T08:53:55.200 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.200 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.200 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:55.200 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556177+0000", 2024-05-08T08:53:55.200 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5051999999999997e-05 2024-05-08T08:53:55.200 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.200 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.200 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:55.200 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556294+0000", 2024-05-08T08:53:55.201 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000117625 2024-05-08T08:53:55.201 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.201 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:55.201 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.201 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.201 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.201 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1075 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11108352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:55.201 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:53:55.201 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 78.702254796000005, 2024-05-08T08:53:55.201 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071045026999999997, 2024-05-08T08:53:55.201 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:55.201 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:55.201 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:55.201 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:55.202 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:55.202 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:55.202 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1075 2024-05-08T08:53:55.202 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.202 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:55.202 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.202 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:55.202 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:53:55.202 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:55.202 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.202 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.202 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:55.202 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:53:55.202 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:55.202 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.203 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.203 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:55.203 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:53:55.203 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.32e-07 2024-05-08T08:53:55.203 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.203 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.203 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:55.203 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485518+0000", 2024-05-08T08:53:55.203 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3769e-05 2024-05-08T08:53:55.203 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.203 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.203 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:55.203 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485527+0000", 2024-05-08T08:53:55.203 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6779999999999995e-06 2024-05-08T08:53:55.203 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.204 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.204 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:55.204 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485532+0000", 2024-05-08T08:53:55.204 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9830000000000004e-06 2024-05-08T08:53:55.204 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.204 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.204 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:55.204 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556340+0000", 2024-05-08T08:53:55.204 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070807743000000006 2024-05-08T08:53:55.204 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.204 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.204 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:55.204 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556435+0000", 2024-05-08T08:53:55.204 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4732e-05 2024-05-08T08:53:55.205 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.205 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.205 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:55.205 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556549+0000", 2024-05-08T08:53:55.205 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011369 2024-05-08T08:53:55.205 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.205 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:55.205 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.205 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.205 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.205 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1078 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11149312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:55.205 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:53:55.205 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 78.702214596999994, 2024-05-08T08:53:55.205 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071167276000000002, 2024-05-08T08:53:55.205 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:55.205 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:55.206 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:55.206 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:55.206 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:55.206 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:55.206 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1078 2024-05-08T08:53:55.206 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.206 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:55.206 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.206 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:55.206 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:53:55.206 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:55.206 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.206 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.206 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:55.207 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:53:55.207 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:55.207 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.207 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.207 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:55.207 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:53:55.207 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5600000000000001e-07 2024-05-08T08:53:55.207 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.207 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.207 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:55.207 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485547+0000", 2024-05-08T08:53:55.207 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3590000000000002e-06 2024-05-08T08:53:55.207 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.207 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.207 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:55.207 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485553+0000", 2024-05-08T08:53:55.208 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2770000000000002e-06 2024-05-08T08:53:55.208 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.208 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.208 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:55.208 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485558+0000", 2024-05-08T08:53:55.208 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.95e-06 2024-05-08T08:53:55.208 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.208 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.208 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:55.208 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556582+0000", 2024-05-08T08:53:55.208 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071024444000000006 2024-05-08T08:53:55.208 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.208 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.208 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:55.208 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556637+0000", 2024-05-08T08:53:55.209 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4672000000000001e-05 2024-05-08T08:53:55.209 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.209 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.209 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:55.209 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556711+0000", 2024-05-08T08:53:55.209 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4117999999999997e-05 2024-05-08T08:53:55.209 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.209 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:55.209 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.209 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.209 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.209 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1080 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11190272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:55.209 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:53:55.209 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 78.702193863999995, 2024-05-08T08:53:55.209 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071350721000000006, 2024-05-08T08:53:55.209 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:55.210 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:55.210 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:55.210 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:55.210 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:55.210 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:55.210 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1080 2024-05-08T08:53:55.210 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.210 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:55.210 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.210 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:55.210 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:53:55.210 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:55.210 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.210 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.210 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:55.211 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:53:55.211 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:55.211 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.211 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.211 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:55.211 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:53:55.211 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.03e-07 2024-05-08T08:53:55.211 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.211 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.211 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:55.211 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485567+0000", 2024-05-08T08:53:55.211 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1090000000000001e-06 2024-05-08T08:53:55.211 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.211 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.211 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:55.211 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485573+0000", 2024-05-08T08:53:55.212 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3049999999999998e-06 2024-05-08T08:53:55.212 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.212 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.212 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:55.212 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485578+0000", 2024-05-08T08:53:55.212 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0130000000000001e-06 2024-05-08T08:53:55.212 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.212 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.212 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:55.212 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556743+0000", 2024-05-08T08:53:55.212 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071164449000000005 2024-05-08T08:53:55.212 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.212 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.212 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:55.212 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556797+0000", 2024-05-08T08:53:55.212 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4589999999999997e-05 2024-05-08T08:53:55.213 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.213 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.213 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:55.213 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556915+0000", 2024-05-08T08:53:55.213 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000117852 2024-05-08T08:53:55.213 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.213 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:55.213 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.213 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.213 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.213 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1082 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11231232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:55.213 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:53:55.213 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 78.702173621, 2024-05-08T08:53:55.213 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071591396000000002, 2024-05-08T08:53:55.213 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:55.214 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:55.214 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:55.214 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:55.214 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:55.214 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:55.214 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1082 2024-05-08T08:53:55.214 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.214 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:55.214 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.214 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:55.214 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:53:55.214 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:55.214 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.214 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.215 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:55.215 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:53:55.215 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:55.215 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.215 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.215 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:55.215 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:53:55.215 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1800000000000001e-07 2024-05-08T08:53:55.215 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.215 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.215 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:55.215 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485587+0000", 2024-05-08T08:53:55.215 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2589999999999999e-06 2024-05-08T08:53:55.215 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.215 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.215 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:55.216 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485594+0000", 2024-05-08T08:53:55.216 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0789999999999997e-06 2024-05-08T08:53:55.216 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.216 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.216 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:55.216 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485598+0000", 2024-05-08T08:53:55.216 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7480000000000001e-06 2024-05-08T08:53:55.216 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.216 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.216 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:55.216 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556971+0000", 2024-05-08T08:53:55.216 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071372774999999999 2024-05-08T08:53:55.216 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.216 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.216 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:55.217 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557069+0000", 2024-05-08T08:53:55.217 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.7732999999999994e-05 2024-05-08T08:53:55.217 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.217 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.217 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:55.217 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557176+0000", 2024-05-08T08:53:55.217 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000107384 2024-05-08T08:53:55.217 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.217 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:55.217 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.217 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.217 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.217 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1084 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11272192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:55.217 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:53:55.217 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 78.702134982000004, 2024-05-08T08:53:55.218 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071808977999999996, 2024-05-08T08:53:55.218 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:55.218 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:55.218 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:55.218 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:55.218 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:55.218 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:55.218 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1084 2024-05-08T08:53:55.218 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.218 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:55.218 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.218 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:55.218 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:53:55.218 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:55.218 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.219 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.219 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:55.219 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:53:55.219 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:55.219 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.219 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.219 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:55.219 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485624+0000", 2024-05-08T08:53:55.219 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:53:55.219 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.219 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.219 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:55.219 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485626+0000", 2024-05-08T08:53:55.219 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9460000000000001e-06 2024-05-08T08:53:55.219 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.219 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.220 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:55.220 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485633+0000", 2024-05-08T08:53:55.220 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1149999999999996e-06 2024-05-08T08:53:55.220 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.220 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.220 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:55.220 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485638+0000", 2024-05-08T08:53:55.220 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5670000000000002e-06 2024-05-08T08:53:55.220 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.220 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.220 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:55.220 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557226+0000", 2024-05-08T08:53:55.220 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071587267999999996 2024-05-08T08:53:55.220 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.220 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.220 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:55.220 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557321+0000", 2024-05-08T08:53:55.221 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5785000000000005e-05 2024-05-08T08:53:55.221 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.221 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.221 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:55.221 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557432+0000", 2024-05-08T08:53:55.221 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011092800000000001 2024-05-08T08:53:55.221 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.221 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:55.221 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.221 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.221 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.221 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1086 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11313152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:55.221 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:53:55.221 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 78.702111654999996, 2024-05-08T08:53:55.221 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071933132999999996, 2024-05-08T08:53:55.221 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:55.222 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:55.222 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:55.222 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:55.222 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:55.222 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:55.222 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1086 2024-05-08T08:53:55.222 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.222 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:55.222 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.222 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:55.222 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:53:55.222 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:55.222 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.222 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.222 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:55.223 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:53:55.223 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:55.223 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.223 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.223 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:55.223 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486328+0000", 2024-05-08T08:53:55.223 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00068107900000000002 2024-05-08T08:53:55.223 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.223 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.223 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:55.223 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486331+0000", 2024-05-08T08:53:55.223 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.585e-06 2024-05-08T08:53:55.223 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.223 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.223 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:55.224 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486343+0000", 2024-05-08T08:53:55.224 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1856e-05 2024-05-08T08:53:55.224 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.224 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.224 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:55.224 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486350+0000", 2024-05-08T08:53:55.224 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8600000000000004e-06 2024-05-08T08:53:55.224 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.224 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.224 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:55.224 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557470+0000", 2024-05-08T08:53:55.224 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071120326999999997 2024-05-08T08:53:55.224 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.224 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.224 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:55.225 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557514+0000", 2024-05-08T08:53:55.225 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3909000000000003e-05 2024-05-08T08:53:55.225 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.225 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.225 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:55.225 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557580+0000", 2024-05-08T08:53:55.225 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5517000000000003e-05 2024-05-08T08:53:55.225 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.225 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:55.225 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.225 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.225 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.225 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1089 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11354112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:55.225 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:53:55.226 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 78.701398783000002, 2024-05-08T08:53:55.226 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071457381, 2024-05-08T08:53:55.226 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:55.226 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:55.226 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:55.226 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:55.226 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:55.226 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:55.226 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1089 2024-05-08T08:53:55.226 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.226 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:55.226 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.226 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:55.226 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:53:55.227 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:55.227 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.227 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.227 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:55.227 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:53:55.227 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:55.227 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.227 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.227 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:55.227 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486851+0000", 2024-05-08T08:53:55.227 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00049102 2024-05-08T08:53:55.227 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.227 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.227 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:55.228 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486854+0000", 2024-05-08T08:53:55.228 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6859999999999999e-06 2024-05-08T08:53:55.228 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.228 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.228 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:55.228 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486864+0000", 2024-05-08T08:53:55.228 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6080000000000002e-06 2024-05-08T08:53:55.228 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.228 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.228 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:55.228 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486870+0000", 2024-05-08T08:53:55.228 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.477e-06 2024-05-08T08:53:55.228 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.228 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.228 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:55.229 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557612+0000", 2024-05-08T08:53:55.229 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070741557999999996 2024-05-08T08:53:55.229 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.229 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.229 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:55.229 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557695+0000", 2024-05-08T08:53:55.229 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3300000000000005e-05 2024-05-08T08:53:55.229 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.229 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.229 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:55.229 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557817+0000", 2024-05-08T08:53:55.229 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012173200000000001 2024-05-08T08:53:55.229 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.229 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:55.229 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.230 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.230 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.230 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1091 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11395072~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:55.230 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:53:55.230 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 78.700877988000002, 2024-05-08T08:53:55.230 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071184737999999997, 2024-05-08T08:53:55.230 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:55.230 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:55.230 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:55.230 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:55.230 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:55.230 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:55.230 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1091 2024-05-08T08:53:55.230 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.231 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:55.231 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.231 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:55.231 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:53:55.231 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:55.231 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.231 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.231 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:55.231 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:53:55.231 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:55.231 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.231 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.231 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:55.231 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486881+0000", 2024-05-08T08:53:55.232 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0699999999999997e-07 2024-05-08T08:53:55.232 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.232 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.232 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:55.232 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486883+0000", 2024-05-08T08:53:55.232 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.402e-06 2024-05-08T08:53:55.232 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.232 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.232 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:55.232 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486890+0000", 2024-05-08T08:53:55.232 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0480000000000001e-06 2024-05-08T08:53:55.232 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.232 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.232 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:55.232 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486896+0000", 2024-05-08T08:53:55.233 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7640000000000002e-06 2024-05-08T08:53:55.233 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.233 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.233 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:55.233 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557860+0000", 2024-05-08T08:53:55.233 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070963791999999998 2024-05-08T08:53:55.233 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.233 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.233 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:55.233 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557952+0000", 2024-05-08T08:53:55.233 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2082000000000004e-05 2024-05-08T08:53:55.233 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.233 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.233 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:55.233 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558065+0000", 2024-05-08T08:53:55.234 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000113143 2024-05-08T08:53:55.234 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.234 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:55.234 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.234 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.234 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.234 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1094 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11436032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:55.234 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:53:55.234 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 78.700854820000004, 2024-05-08T08:53:55.234 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071400507000000002, 2024-05-08T08:53:55.234 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:55.234 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:55.234 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:55.235 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:55.235 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:55.235 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:55.235 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1094 2024-05-08T08:53:55.235 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.235 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:55.235 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.235 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:55.235 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:53:55.235 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:55.235 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.235 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.235 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:55.235 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:53:55.235 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:55.236 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.236 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.236 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:55.236 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487940+0000", 2024-05-08T08:53:55.236 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001036868 2024-05-08T08:53:55.236 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.236 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.236 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:55.236 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487944+0000", 2024-05-08T08:53:55.236 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7069999999999998e-06 2024-05-08T08:53:55.236 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.236 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.236 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:55.236 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487951+0000", 2024-05-08T08:53:55.236 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2649999999999999e-06 2024-05-08T08:53:55.236 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.237 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.237 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:55.237 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487961+0000", 2024-05-08T08:53:55.237 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4110000000000002e-06 2024-05-08T08:53:55.237 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.237 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.237 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:55.237 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558117+0000", 2024-05-08T08:53:55.237 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070156662999999994 2024-05-08T08:53:55.237 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.237 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.237 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:55.237 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558214+0000", 2024-05-08T08:53:55.237 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6444e-05 2024-05-08T08:53:55.238 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.238 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.238 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:55.238 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558304+0000", 2024-05-08T08:53:55.238 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0148999999999994e-05 2024-05-08T08:53:55.238 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.238 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:55.238 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.238 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.238 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.238 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1096 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11476992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:55.238 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:53:55.238 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 78.699788288999997, 2024-05-08T08:53:55.239 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070480074000000004, 2024-05-08T08:53:55.239 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:55.239 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:55.239 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:55.239 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:55.239 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:55.239 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:55.239 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1096 2024-05-08T08:53:55.239 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.239 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:55.239 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.239 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:55.239 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:53:55.239 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:55.239 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.240 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.240 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:55.240 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:53:55.240 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:55.240 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.240 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.240 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:55.240 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:53:55.240 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3700000000000001e-07 2024-05-08T08:53:55.240 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.240 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.240 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:55.240 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487973+0000", 2024-05-08T08:53:55.240 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0870000000000001e-06 2024-05-08T08:53:55.240 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.240 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.241 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:55.241 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487977+0000", 2024-05-08T08:53:55.241 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0600000000000001e-06 2024-05-08T08:53:55.241 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.241 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.241 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:55.241 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487980+0000", 2024-05-08T08:53:55.241 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1089999999999998e-06 2024-05-08T08:53:55.241 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.241 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.241 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:55.241 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558338+0000", 2024-05-08T08:53:55.241 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070358233000000006 2024-05-08T08:53:55.241 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.242 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.242 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:55.242 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558392+0000", 2024-05-08T08:53:55.242 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4537999999999999e-05 2024-05-08T08:53:55.242 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.242 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.242 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:55.242 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558450+0000", 2024-05-08T08:53:55.242 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7710000000000001e-05 2024-05-08T08:53:55.242 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.242 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:55.242 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.242 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.242 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.242 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1099 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11517952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:55.243 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:53:55.243 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 78.699774413, 2024-05-08T08:53:55.243 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070720506000000002, 2024-05-08T08:53:55.243 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:55.243 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:55.243 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:55.243 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:55.243 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:55.243 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:55.243 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1099 2024-05-08T08:53:55.243 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.243 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:55.243 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.243 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:55.244 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:53:55.244 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:55.244 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.244 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.244 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:55.244 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:53:55.244 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:55.244 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.244 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.244 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:55.244 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:53:55.244 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3999999999999998e-07 2024-05-08T08:53:55.244 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.244 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.244 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:55.244 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487986+0000", 2024-05-08T08:53:55.245 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3519999999999999e-06 2024-05-08T08:53:55.245 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.245 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.245 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:55.245 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487989+0000", 2024-05-08T08:53:55.245 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.23e-06 2024-05-08T08:53:55.245 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.245 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.245 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:55.245 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487991+0000", 2024-05-08T08:53:55.245 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6350000000000002e-06 2024-05-08T08:53:55.245 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.245 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.245 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:55.245 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558489+0000", 2024-05-08T08:53:55.246 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070497924000000003 2024-05-08T08:53:55.246 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.246 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.246 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:55.246 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558583+0000", 2024-05-08T08:53:55.246 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3405000000000004e-05 2024-05-08T08:53:55.246 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.246 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.246 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:55.246 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558704+0000", 2024-05-08T08:53:55.246 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012171999999999999 2024-05-08T08:53:55.246 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.246 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:55.246 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.247 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.247 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.247 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1100 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11558912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:55.247 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:53:55.247 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 78.699761413999994, 2024-05-08T08:53:55.247 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070962266999999996, 2024-05-08T08:53:55.247 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:55.247 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:55.247 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:55.247 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:55.247 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:55.247 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:55.247 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1100 2024-05-08T08:53:55.247 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.247 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:55.247 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.248 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:55.248 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:53:55.248 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:55.248 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.248 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.248 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:55.248 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:53:55.248 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:55.248 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.248 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.248 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:55.248 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488430+0000", 2024-05-08T08:53:55.248 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000432676 2024-05-08T08:53:55.248 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.248 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.248 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:55.249 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488433+0000", 2024-05-08T08:53:55.249 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.45e-06 2024-05-08T08:53:55.249 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.249 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.249 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:55.249 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488442+0000", 2024-05-08T08:53:55.249 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.687e-06 2024-05-08T08:53:55.249 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.249 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.249 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:55.249 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488446+0000", 2024-05-08T08:53:55.249 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4939999999999997e-06 2024-05-08T08:53:55.249 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.249 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.249 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:55.249 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558756+0000", 2024-05-08T08:53:55.250 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070310051999999998 2024-05-08T08:53:55.250 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.250 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.250 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:55.250 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558844+0000", 2024-05-08T08:53:55.250 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7690999999999998e-05 2024-05-08T08:53:55.250 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.250 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.250 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:55.250 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558959+0000", 2024-05-08T08:53:55.250 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000115217 2024-05-08T08:53:55.250 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.250 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:55.250 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.251 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.251 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.251 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1103 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11599872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:55.251 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:53:55.251 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 78.699303739000001, 2024-05-08T08:53:55.251 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070777100999999995, 2024-05-08T08:53:55.251 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:55.251 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:55.251 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:55.251 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:55.251 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:55.251 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:55.251 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1103 2024-05-08T08:53:55.251 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.251 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:55.252 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.252 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:55.252 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:53:55.252 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:55.252 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.252 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.252 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:55.252 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:53:55.252 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:55.252 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.252 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.252 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:55.252 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488959+0000", 2024-05-08T08:53:55.252 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00050457200000000001 2024-05-08T08:53:55.252 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.253 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.253 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:55.253 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488969+0000", 2024-05-08T08:53:55.253 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.7159999999999998e-06 2024-05-08T08:53:55.253 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.253 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.253 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:55.253 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488981+0000", 2024-05-08T08:53:55.253 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2150000000000001e-05 2024-05-08T08:53:55.253 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.253 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.253 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:55.253 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488988+0000", 2024-05-08T08:53:55.253 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8759999999999999e-06 2024-05-08T08:53:55.253 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.254 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.254 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:55.254 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558992+0000", 2024-05-08T08:53:55.254 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070004358000000003 2024-05-08T08:53:55.254 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.254 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.254 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:55.254 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559093+0000", 2024-05-08T08:53:55.254 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000100299 2024-05-08T08:53:55.254 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.254 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.254 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:55.254 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559232+0000", 2024-05-08T08:53:55.254 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013913 2024-05-08T08:53:55.255 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.255 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:55.255 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.255 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.255 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.255 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1104 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11640832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:53:55.255 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:53:55.255 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 78.698758924000003, 2024-05-08T08:53:55.255 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070521947000000001, 2024-05-08T08:53:55.255 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:55.255 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:55.255 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:55.255 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:55.255 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:53:55.255 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:53:55.256 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1104 2024-05-08T08:53:55.256 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.256 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:55.256 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.256 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:55.256 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:53:55.256 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:55.256 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.256 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.256 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:55.256 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:53:55.256 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:55.256 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.256 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.257 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:55.257 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489700+0000", 2024-05-08T08:53:55.257 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00070067600000000001 2024-05-08T08:53:55.257 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.257 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.257 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:55.257 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489704+0000", 2024-05-08T08:53:55.257 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1640000000000003e-06 2024-05-08T08:53:55.257 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.257 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.257 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:55.257 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489716+0000", 2024-05-08T08:53:55.257 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1379000000000001e-05 2024-05-08T08:53:55.257 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.257 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.258 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:55.258 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489724+0000", 2024-05-08T08:53:55.258 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8949999999999998e-06 2024-05-08T08:53:55.258 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.258 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.258 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:55.258 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559278+0000", 2024-05-08T08:53:55.258 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.069554151999999994 2024-05-08T08:53:55.258 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.258 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.258 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:55.258 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559387+0000", 2024-05-08T08:53:55.258 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000109258 2024-05-08T08:53:55.258 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.259 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.259 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:55.259 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559521+0000", 2024-05-08T08:53:55.259 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000134423 2024-05-08T08:53:55.259 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.259 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:55.259 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.259 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.259 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.259 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:8636 3.17 3:e8a0a3e4:::rbd_header.11402b26385:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-05-08T08:53:55.259 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:54.150971+0000", 2024-05-08T08:53:55.259 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 61.036786929999998, 2024-05-08T08:53:55.259 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.217273233, 2024-05-08T08:53:55.259 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:55.259 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:55.260 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:55.260 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:55.260 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:53:55.260 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:55.260 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 8636 2024-05-08T08:53:55.260 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.260 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:55.260 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.260 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:55.260 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150971+0000", 2024-05-08T08:53:55.260 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:55.260 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.260 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.260 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:55.260 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150971+0000", 2024-05-08T08:53:55.261 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:55.261 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.261 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.261 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:55.261 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150972+0000", 2024-05-08T08:53:55.261 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.71e-07 2024-05-08T08:53:55.261 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.261 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.261 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:55.261 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150977+0000", 2024-05-08T08:53:55.261 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.152e-06 2024-05-08T08:53:55.261 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.261 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.261 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:55.261 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150990+0000", 2024-05-08T08:53:55.262 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2364999999999999e-05 2024-05-08T08:53:55.262 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.262 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.262 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:55.262 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150998+0000", 2024-05-08T08:53:55.262 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0739999999999993e-06 2024-05-08T08:53:55.262 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.262 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.262 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:55.262 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.151054+0000", 2024-05-08T08:53:55.262 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5949000000000003e-05 2024-05-08T08:53:55.262 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.262 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.262 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:53:55.263 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.151060+0000", 2024-05-08T08:53:55.263 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6120000000000004e-06 2024-05-08T08:53:55.263 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.263 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.263 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:55.263 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.367640+0000", 2024-05-08T08:53:55.263 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2165792739999999 2024-05-08T08:53:55.263 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.263 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.263 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:55.263 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.367839+0000", 2024-05-08T08:53:55.263 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000199185 2024-05-08T08:53:55.263 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.263 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.263 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:55.263 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.368245+0000", 2024-05-08T08:53:55.264 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00040575099999999998 2024-05-08T08:53:55.264 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.264 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:55.264 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.264 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.264 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.264 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5302.0:1516 3.c 3:319bf789:::rbd_data.1140d3fc54c.0000000000000025:head [stat,write 1892352~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:53:55.264 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:27.822220+0000", 2024-05-08T08:53:55.264 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 27.365537993, 2024-05-08T08:53:55.264 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.092643744, 2024-05-08T08:53:55.264 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:55.264 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:55.264 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:53:55.264 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:55.264 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5302", 2024-05-08T08:53:55.265 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/359277859", 2024-05-08T08:53:55.265 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1516 2024-05-08T08:53:55.265 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.265 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:55.265 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.265 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:55.265 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822220+0000", 2024-05-08T08:53:55.265 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:55.265 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.265 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.265 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:55.265 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822220+0000", 2024-05-08T08:53:55.265 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:55.265 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.265 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.266 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:55.266 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822221+0000", 2024-05-08T08:53:55.266 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3500000000000002e-07 2024-05-08T08:53:55.266 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.266 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.266 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:55.266 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822228+0000", 2024-05-08T08:53:55.266 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9959999999999996e-06 2024-05-08T08:53:55.266 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.266 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.266 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:55.266 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822234+0000", 2024-05-08T08:53:55.266 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5429999999999999e-06 2024-05-08T08:53:55.266 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.267 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.267 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:55.267 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822238+0000", 2024-05-08T08:53:55.267 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.732e-06 2024-05-08T08:53:55.267 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.267 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.267 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:55.267 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822285+0000", 2024-05-08T08:53:55.267 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6810000000000001e-05 2024-05-08T08:53:55.267 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.267 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.267 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:55.267 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822358+0000", 2024-05-08T08:53:55.267 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3473000000000006e-05 2024-05-08T08:53:55.268 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.268 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.268 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 5", 2024-05-08T08:53:55.268 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822489+0000", 2024-05-08T08:53:55.268 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013011199999999999 2024-05-08T08:53:55.268 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.268 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.268 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:53:55.268 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.825332+0000", 2024-05-08T08:53:55.268 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0028434799999999998 2024-05-08T08:53:55.268 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.268 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.268 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:53:55.268 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.914753+0000", 2024-05-08T08:53:55.268 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089421424999999999 2024-05-08T08:53:55.269 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.269 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.269 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:53:55.269 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.914807+0000", 2024-05-08T08:53:55.269 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3829000000000002e-05 2024-05-08T08:53:55.269 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.269 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.269 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:55.269 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.914864+0000", 2024-05-08T08:53:55.269 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6909e-05 2024-05-08T08:53:55.269 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.269 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:55.269 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.269 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.269 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.269 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:24567 3.6 3:66af1c40:::rbd_data.1140d3fc54c.000000000000005a:18 [read 2080768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:55.270 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.216598+0000", 2024-05-08T08:53:55.270 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 22.971159913000001, 2024-05-08T08:53:55.270 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071602647000000005, 2024-05-08T08:53:55.270 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:55.270 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:55.270 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:55.270 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:55.270 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:55.270 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:55.270 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 24567 2024-05-08T08:53:55.270 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.270 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:55.270 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.270 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:55.270 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216598+0000", 2024-05-08T08:53:55.270 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:55.271 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.271 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.271 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:55.271 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216598+0000", 2024-05-08T08:53:55.271 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:55.271 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.271 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.271 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:55.271 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216599+0000", 2024-05-08T08:53:55.271 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8999999999999997e-07 2024-05-08T08:53:55.271 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.271 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.271 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:55.271 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216601+0000", 2024-05-08T08:53:55.271 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9889999999999999e-06 2024-05-08T08:53:55.271 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.271 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.272 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:55.272 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216607+0000", 2024-05-08T08:53:55.272 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3600000000000001e-06 2024-05-08T08:53:55.272 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.272 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.272 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:55.272 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216612+0000", 2024-05-08T08:53:55.272 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.69e-06 2024-05-08T08:53:55.272 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.272 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.272 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:55.272 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.287076+0000", 2024-05-08T08:53:55.272 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070464355000000006 2024-05-08T08:53:55.272 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.272 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.273 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:55.273 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.287151+0000", 2024-05-08T08:53:55.273 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4924000000000003e-05 2024-05-08T08:53:55.273 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.273 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.273 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:55.273 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.288201+0000", 2024-05-08T08:53:55.273 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0010498389999999999 2024-05-08T08:53:55.273 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.273 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:55.273 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.273 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.273 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:55.273 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:55.275 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:53:55.275 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:55.277+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:55.375 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:55.378+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:55.429 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:55.429 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:53:55.429 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:53:55.429 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:55.437 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:53:55.476 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:55.479+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:55.546 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:55.547 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:53:55.547 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:53:55.547 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:53:55.547 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:55.555 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:53:55.577 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:55.580+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:55.667 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:55.667 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:53:55.667 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:53:55.667 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:53:55.667 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.667 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:12443 3.0 3:02d0158e:::rbd_data.114071bcf828.0000000000000037:head [stat,delete] snapc 19=[19] RETRY=3 ondisk+retry+write+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:55.668 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.610027+0000", 2024-05-08T08:53:55.668 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 24.059080959999999, 2024-05-08T08:53:55.668 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.083586333999999998, 2024-05-08T08:53:55.668 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:55.668 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:55.668 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:55.668 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:55.668 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:55.668 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:55.668 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 12443 2024-05-08T08:53:55.668 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.668 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:55.668 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.668 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:55.669 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610027+0000", 2024-05-08T08:53:55.669 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:55.669 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.669 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.669 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:55.669 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610027+0000", 2024-05-08T08:53:55.669 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:55.669 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.669 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.669 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:55.669 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610028+0000", 2024-05-08T08:53:55.669 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.13e-07 2024-05-08T08:53:55.669 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.670 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.670 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:55.670 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610030+0000", 2024-05-08T08:53:55.670 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7669999999999999e-06 2024-05-08T08:53:55.670 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.670 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.670 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:55.670 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610039+0000", 2024-05-08T08:53:55.670 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4959999999999996e-06 2024-05-08T08:53:55.670 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.670 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.670 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:55.670 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610045+0000", 2024-05-08T08:53:55.670 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6779999999999997e-06 2024-05-08T08:53:55.671 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.671 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.671 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:55.671 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.660873+0000", 2024-05-08T08:53:55.671 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.050828748 2024-05-08T08:53:55.671 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.671 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.671 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:55.671 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.661017+0000", 2024-05-08T08:53:55.671 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014391999999999999 2024-05-08T08:53:55.671 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.671 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.671 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:55.672 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.693613+0000", 2024-05-08T08:53:55.672 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.032596111999999997 2024-05-08T08:53:55.672 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.672 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:55.672 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.672 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.672 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.672 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:14301 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [watch reconnect cookie 139837155901536 gen 12] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:55.672 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.645078+0000", 2024-05-08T08:53:55.672 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 24.024030322000002, 2024-05-08T08:53:55.672 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088281852999999993, 2024-05-08T08:53:55.672 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:55.672 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:55.672 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:55.672 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:55.673 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:55.673 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:55.673 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14301 2024-05-08T08:53:55.673 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.673 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:55.673 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.673 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:55.673 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645078+0000", 2024-05-08T08:53:55.673 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:55.673 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.673 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.673 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:55.673 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645078+0000", 2024-05-08T08:53:55.673 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:55.674 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.674 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.674 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:55.674 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645084+0000", 2024-05-08T08:53:55.674 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4620000000000001e-06 2024-05-08T08:53:55.674 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.674 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.674 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:55.674 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645088+0000", 2024-05-08T08:53:55.674 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8039999999999999e-06 2024-05-08T08:53:55.674 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.674 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.674 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:55.674 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645106+0000", 2024-05-08T08:53:55.675 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.8318999999999999e-05 2024-05-08T08:53:55.675 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.675 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.675 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:55.675 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645113+0000", 2024-05-08T08:53:55.675 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0310000000000001e-06 2024-05-08T08:53:55.675 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.675 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.675 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:55.675 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.670493+0000", 2024-05-08T08:53:55.675 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.025379993 2024-05-08T08:53:55.675 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.675 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.675 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:55.676 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.670571+0000", 2024-05-08T08:53:55.676 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7613000000000004e-05 2024-05-08T08:53:55.676 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.676 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.676 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:55.676 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733360+0000", 2024-05-08T08:53:55.676 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.062788630999999998 2024-05-08T08:53:55.676 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.676 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:55.676 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.676 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.676 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.676 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:14420 3.0 3:06a8948e:::rbd_data.114071bcf828.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:55.676 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.649639+0000", 2024-05-08T08:53:55.677 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 24.019469174000001, 2024-05-08T08:53:55.677 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.083851897999999994, 2024-05-08T08:53:55.677 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:55.677 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:55.677 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:53:55.677 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:55.677 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:55.677 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:55.677 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14420 2024-05-08T08:53:55.677 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.677 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:55.677 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.677 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:55.677 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649639+0000", 2024-05-08T08:53:55.677 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:55.678 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.678 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.678 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:55.678 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649639+0000", 2024-05-08T08:53:55.678 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:55.678 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.678 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.678 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:55.678 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649640+0000", 2024-05-08T08:53:55.678 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6899999999999997e-07 2024-05-08T08:53:55.678 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.678 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.678 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:55.678 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649895+0000", 2024-05-08T08:53:55.679 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00025531000000000002 2024-05-08T08:53:55.679 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.679 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.679 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:55.679 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649905+0000", 2024-05-08T08:53:55.679 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0198e-05 2024-05-08T08:53:55.679 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.679 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.679 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:55.679 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649911+0000", 2024-05-08T08:53:55.679 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2310000000000001e-06 2024-05-08T08:53:55.679 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.679 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.679 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:55.679 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.684099+0000", 2024-05-08T08:53:55.680 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.034188017000000001 2024-05-08T08:53:55.680 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.680 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.680 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:55.680 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.684172+0000", 2024-05-08T08:53:55.680 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3201999999999995e-05 2024-05-08T08:53:55.680 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.680 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.680 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 2", 2024-05-08T08:53:55.680 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.684418+0000", 2024-05-08T08:53:55.680 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00024559099999999997 2024-05-08T08:53:55.680 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.680 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.680 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:53:55.680 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.705538+0000", 2024-05-08T08:53:55.681 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.021119707000000001 2024-05-08T08:53:55.681 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.681 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.681 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:53:55.681 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733399+0000", 2024-05-08T08:53:55.681 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.027861543999999999 2024-05-08T08:53:55.681 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.681 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.681 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:53:55.681 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733445+0000", 2024-05-08T08:53:55.681 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5587000000000002e-05 2024-05-08T08:53:55.681 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.681 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.681 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:55.682 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733491+0000", 2024-05-08T08:53:55.682 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5841999999999999e-05 2024-05-08T08:53:55.682 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.682 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:55.682 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.682 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.682 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.682 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28547 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889179216] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:55.682 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.355654+0000", 2024-05-08T08:53:55.682 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 23.313453945999999, 2024-05-08T08:53:55.682 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078692261999999999, 2024-05-08T08:53:55.682 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:55.682 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:55.682 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:55.683 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:55.683 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:55.683 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:55.683 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28547 2024-05-08T08:53:55.683 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.683 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:55.683 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.683 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:55.683 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355654+0000", 2024-05-08T08:53:55.683 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:55.683 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.683 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.683 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:55.683 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355654+0000", 2024-05-08T08:53:55.683 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:55.684 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.684 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.684 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:55.684 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355654+0000", 2024-05-08T08:53:55.684 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3999999999999998e-07 2024-05-08T08:53:55.684 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.684 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.684 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:55.684 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355656+0000", 2024-05-08T08:53:55.684 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.353e-06 2024-05-08T08:53:55.684 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.684 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.684 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:55.684 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355663+0000", 2024-05-08T08:53:55.685 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2259999999999999e-06 2024-05-08T08:53:55.685 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.685 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.685 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:55.685 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355666+0000", 2024-05-08T08:53:55.685 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7930000000000002e-06 2024-05-08T08:53:55.685 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.685 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.685 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:55.685 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434117+0000", 2024-05-08T08:53:55.685 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078451231999999996 2024-05-08T08:53:55.685 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.685 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.685 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:55.686 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434198+0000", 2024-05-08T08:53:55.686 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0879999999999998e-05 2024-05-08T08:53:55.686 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.686 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.686 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:55.686 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434346+0000", 2024-05-08T08:53:55.686 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014853800000000001 2024-05-08T08:53:55.686 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.686 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:55.686 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.686 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.686 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.686 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28548 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889189696] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:55.686 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.355669+0000", 2024-05-08T08:53:55.687 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 23.313438596000001, 2024-05-08T08:53:55.687 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078956606999999998, 2024-05-08T08:53:55.687 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:55.687 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:55.687 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:55.687 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:55.687 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:55.687 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:55.687 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28548 2024-05-08T08:53:55.687 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.687 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:55.687 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.687 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:55.688 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355669+0000", 2024-05-08T08:53:55.688 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:55.688 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.688 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.688 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:55.688 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355669+0000", 2024-05-08T08:53:55.688 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:55.688 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.688 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.688 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:55.688 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355762+0000", 2024-05-08T08:53:55.688 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3015000000000003e-05 2024-05-08T08:53:55.688 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.688 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.689 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:55.689 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355765+0000", 2024-05-08T08:53:55.689 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.368e-06 2024-05-08T08:53:55.689 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.689 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.689 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:55.689 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355771+0000", 2024-05-08T08:53:55.689 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5989999999999998e-06 2024-05-08T08:53:55.689 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.689 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.689 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:55.689 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355775+0000", 2024-05-08T08:53:55.689 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7280000000000001e-06 2024-05-08T08:53:55.689 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.689 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.690 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:55.690 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434379+0000", 2024-05-08T08:53:55.690 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078603662000000005 2024-05-08T08:53:55.690 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.690 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.690 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:55.690 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434473+0000", 2024-05-08T08:53:55.690 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4104000000000006e-05 2024-05-08T08:53:55.690 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.690 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.690 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:55.690 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434626+0000", 2024-05-08T08:53:55.690 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00015313100000000001 2024-05-08T08:53:55.690 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.691 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:55.691 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.691 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.691 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.691 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28549 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889202400] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:55.691 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.355781+0000", 2024-05-08T08:53:55.691 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 23.313327021999999, 2024-05-08T08:53:55.691 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079117946999999994, 2024-05-08T08:53:55.691 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:55.691 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:55.691 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:55.691 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:55.691 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:55.691 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:55.691 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28549 2024-05-08T08:53:55.692 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.692 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:55.692 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.692 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:55.692 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355781+0000", 2024-05-08T08:53:55.692 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:55.692 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.692 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.692 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:55.692 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355781+0000", 2024-05-08T08:53:55.692 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:55.692 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.692 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.693 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:55.693 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356162+0000", 2024-05-08T08:53:55.693 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00038099100000000001 2024-05-08T08:53:55.693 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.693 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.693 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:55.693 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356165+0000", 2024-05-08T08:53:55.693 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1240000000000001e-06 2024-05-08T08:53:55.693 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.693 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.693 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:55.693 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356173+0000", 2024-05-08T08:53:55.693 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.6199999999999999e-06 2024-05-08T08:53:55.693 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.694 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.694 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:55.694 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356182+0000", 2024-05-08T08:53:55.694 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7760000000000003e-06 2024-05-08T08:53:55.694 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.694 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.694 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:55.694 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434675+0000", 2024-05-08T08:53:55.694 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078493119 2024-05-08T08:53:55.694 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.694 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.694 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:55.694 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434765+0000", 2024-05-08T08:53:55.695 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0402000000000006e-05 2024-05-08T08:53:55.695 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.695 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.695 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:55.695 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434899+0000", 2024-05-08T08:53:55.695 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013391499999999999 2024-05-08T08:53:55.695 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.695 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:55.695 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.695 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.695 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.695 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28550 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888597280] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:55.695 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:53:55.695 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 23.312915975999999, 2024-05-08T08:53:55.695 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078933990999999995, 2024-05-08T08:53:55.696 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:55.696 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:55.696 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:55.696 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:55.696 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:55.696 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:55.696 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28550 2024-05-08T08:53:55.696 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.696 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:55.696 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.696 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:55.696 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:53:55.696 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:55.696 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.696 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.696 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:55.696 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:53:55.697 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:55.697 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.697 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.697 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:55.697 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:53:55.697 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.34e-07 2024-05-08T08:53:55.697 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.697 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.697 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:55.697 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356195+0000", 2024-05-08T08:53:55.697 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8710000000000001e-06 2024-05-08T08:53:55.697 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.697 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.697 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:55.697 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356200+0000", 2024-05-08T08:53:55.698 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3000000000000003e-06 2024-05-08T08:53:55.698 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.698 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.698 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:55.698 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356203+0000", 2024-05-08T08:53:55.698 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4039999999999999e-06 2024-05-08T08:53:55.698 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.698 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.698 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:55.698 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434928+0000", 2024-05-08T08:53:55.698 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078725331999999995 2024-05-08T08:53:55.698 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.698 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.698 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:55.698 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435007+0000", 2024-05-08T08:53:55.699 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8415000000000001e-05 2024-05-08T08:53:55.699 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.699 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.699 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:55.699 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435126+0000", 2024-05-08T08:53:55.699 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011923500000000001 2024-05-08T08:53:55.699 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.699 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:55.699 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.699 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.699 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.699 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28551 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888527200] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:55.699 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.356210+0000", 2024-05-08T08:53:55.700 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 23.312897628000002, 2024-05-08T08:53:55.700 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079195072000000005, 2024-05-08T08:53:55.700 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:55.700 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:55.700 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:55.700 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:55.700 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:55.700 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:55.700 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28551 2024-05-08T08:53:55.700 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.700 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:55.700 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.700 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:55.700 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356210+0000", 2024-05-08T08:53:55.700 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:55.701 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.701 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.701 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:55.701 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356210+0000", 2024-05-08T08:53:55.701 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:55.701 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.701 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.701 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:55.701 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356211+0000", 2024-05-08T08:53:55.701 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4499999999999998e-07 2024-05-08T08:53:55.701 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.701 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.701 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:55.701 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356212+0000", 2024-05-08T08:53:55.701 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6029999999999999e-06 2024-05-08T08:53:55.701 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.702 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.702 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:55.702 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356221+0000", 2024-05-08T08:53:55.702 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4200000000000007e-06 2024-05-08T08:53:55.702 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.702 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.702 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:55.702 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356224+0000", 2024-05-08T08:53:55.702 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.038e-06 2024-05-08T08:53:55.702 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.702 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.702 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:55.702 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435178+0000", 2024-05-08T08:53:55.702 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078954625000000001 2024-05-08T08:53:55.702 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.703 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.703 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:55.703 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435256+0000", 2024-05-08T08:53:55.703 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7337999999999997e-05 2024-05-08T08:53:55.703 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.703 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.703 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:55.703 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435405+0000", 2024-05-08T08:53:55.703 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000149703 2024-05-08T08:53:55.703 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.703 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:55.703 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.703 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.703 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.703 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28552 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889231456] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:55.704 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.356228+0000", 2024-05-08T08:53:55.704 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 23.312879949999999, 2024-05-08T08:53:55.704 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079346920000000001, 2024-05-08T08:53:55.704 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:55.704 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:55.704 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:55.704 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:55.704 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:55.704 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:55.704 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28552 2024-05-08T08:53:55.704 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.704 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:55.704 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.704 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:55.704 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356228+0000", 2024-05-08T08:53:55.705 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:55.705 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.705 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.705 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:55.705 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356228+0000", 2024-05-08T08:53:55.705 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:55.705 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.705 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.705 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:55.705 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356959+0000", 2024-05-08T08:53:55.705 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00073097099999999996 2024-05-08T08:53:55.705 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.705 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.705 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:55.705 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356967+0000", 2024-05-08T08:53:55.706 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0409999999999998e-06 2024-05-08T08:53:55.706 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.706 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.706 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:55.706 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356979+0000", 2024-05-08T08:53:55.706 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1489e-05 2024-05-08T08:53:55.706 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.706 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.706 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:55.706 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356986+0000", 2024-05-08T08:53:55.706 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3699999999999997e-06 2024-05-08T08:53:55.706 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.706 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.706 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:55.707 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435433+0000", 2024-05-08T08:53:55.707 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078447520000000007 2024-05-08T08:53:55.707 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.707 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.707 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:55.707 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435484+0000", 2024-05-08T08:53:55.707 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0229000000000003e-05 2024-05-08T08:53:55.707 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.707 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.707 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:55.707 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435575+0000", 2024-05-08T08:53:55.707 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1299999999999997e-05 2024-05-08T08:53:55.707 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.707 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:55.708 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.708 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.708 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.708 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28553 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888713648] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:55.708 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:53:55.708 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 23.312105291999998, 2024-05-08T08:53:55.708 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078782034000000001, 2024-05-08T08:53:55.708 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:55.708 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:55.708 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:55.708 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:55.708 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:55.708 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:55.709 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28553 2024-05-08T08:53:55.709 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.709 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:55.709 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.709 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:55.709 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:53:55.709 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:55.709 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.709 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.709 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:55.709 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:53:55.709 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:55.710 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.710 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.710 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:55.710 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:53:55.710 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6600000000000002e-07 2024-05-08T08:53:55.710 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.710 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.710 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:55.710 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357006+0000", 2024-05-08T08:53:55.710 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7190000000000001e-06 2024-05-08T08:53:55.710 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.710 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.710 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:55.710 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357013+0000", 2024-05-08T08:53:55.711 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.435e-06 2024-05-08T08:53:55.711 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.711 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.711 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:55.711 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357018+0000", 2024-05-08T08:53:55.711 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7260000000000002e-06 2024-05-08T08:53:55.711 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.711 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.711 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:55.711 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435623+0000", 2024-05-08T08:53:55.711 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078604972999999995 2024-05-08T08:53:55.711 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.711 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.711 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:55.711 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435695+0000", 2024-05-08T08:53:55.711 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1959999999999995e-05 2024-05-08T08:53:55.712 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.712 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.712 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:55.712 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435785+0000", 2024-05-08T08:53:55.712 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.9754999999999997e-05 2024-05-08T08:53:55.712 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.712 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:55.712 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.712 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.712 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.712 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28554 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888980272] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:55.712 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357024+0000", 2024-05-08T08:53:55.712 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 23.312083655999999, 2024-05-08T08:53:55.712 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078999870999999999, 2024-05-08T08:53:55.712 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:55.712 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:55.713 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:55.713 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:55.713 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:55.713 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:55.713 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28554 2024-05-08T08:53:55.713 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.713 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:55.713 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.713 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:55.713 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357024+0000", 2024-05-08T08:53:55.713 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:55.713 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.713 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.714 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:55.714 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357024+0000", 2024-05-08T08:53:55.714 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:55.714 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.714 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.714 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:55.714 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357025+0000", 2024-05-08T08:53:55.714 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:53:55.714 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.714 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.714 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:55.714 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357027+0000", 2024-05-08T08:53:55.714 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2050000000000001e-06 2024-05-08T08:53:55.715 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.715 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.715 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:55.715 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357033+0000", 2024-05-08T08:53:55.715 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5679999999999996e-06 2024-05-08T08:53:55.715 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.715 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.715 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:55.715 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357038+0000", 2024-05-08T08:53:55.715 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2889999999999999e-06 2024-05-08T08:53:55.715 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.715 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.715 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:55.715 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435823+0000", 2024-05-08T08:53:55.716 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078785677999999998 2024-05-08T08:53:55.716 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.716 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.716 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:55.716 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435892+0000", 2024-05-08T08:53:55.716 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8379000000000004e-05 2024-05-08T08:53:55.716 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.716 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.716 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:55.716 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436024+0000", 2024-05-08T08:53:55.716 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013239799999999999 2024-05-08T08:53:55.716 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.716 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:55.716 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.716 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.717 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.717 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28555 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889001104] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:55.717 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:53:55.717 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 23.312064890999999, 2024-05-08T08:53:55.717 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079115742000000003, 2024-05-08T08:53:55.717 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:55.717 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:55.717 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:55.717 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:55.717 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:55.717 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:55.717 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28555 2024-05-08T08:53:55.717 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.717 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:55.717 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.718 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:55.718 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:53:55.718 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:55.718 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.718 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.718 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:55.718 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:53:55.718 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:55.718 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.718 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.718 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:55.718 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:53:55.718 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9200000000000002e-07 2024-05-08T08:53:55.718 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.719 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.719 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:55.719 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357046+0000", 2024-05-08T08:53:55.719 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1670000000000002e-06 2024-05-08T08:53:55.719 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.719 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.719 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:55.719 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357052+0000", 2024-05-08T08:53:55.719 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.139e-06 2024-05-08T08:53:55.719 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.719 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.719 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:55.719 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357056+0000", 2024-05-08T08:53:55.719 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2440000000000004e-06 2024-05-08T08:53:55.719 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.720 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.720 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:55.720 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436059+0000", 2024-05-08T08:53:55.720 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079003070999999994 2024-05-08T08:53:55.720 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.720 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.720 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:55.720 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436097+0000", 2024-05-08T08:53:55.720 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7784000000000003e-05 2024-05-08T08:53:55.720 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.720 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.720 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:55.720 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436159+0000", 2024-05-08T08:53:55.720 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2045000000000007e-05 2024-05-08T08:53:55.720 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.721 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:55.721 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.721 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.721 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.721 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28556 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889022064] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:55.721 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357065+0000", 2024-05-08T08:53:55.721 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 23.312042858000002, 2024-05-08T08:53:55.721 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079327976999999994, 2024-05-08T08:53:55.721 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:55.721 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:55.721 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:55.721 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:55.721 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:55.722 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:55.722 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28556 2024-05-08T08:53:55.722 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.722 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:55.722 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.722 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:55.722 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357065+0000", 2024-05-08T08:53:55.722 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:55.722 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.722 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.722 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:55.722 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357065+0000", 2024-05-08T08:53:55.722 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:55.722 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.723 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.723 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:55.723 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358958+0000", 2024-05-08T08:53:55.723 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001892915 2024-05-08T08:53:55.723 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.723 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.723 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:55.723 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358962+0000", 2024-05-08T08:53:55.723 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4599999999999999e-06 2024-05-08T08:53:55.723 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.723 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.723 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:55.723 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358970+0000", 2024-05-08T08:53:55.723 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.6090000000000001e-06 2024-05-08T08:53:55.724 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.724 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.724 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:55.724 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358975+0000", 2024-05-08T08:53:55.724 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7639999999999996e-06 2024-05-08T08:53:55.724 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.724 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.724 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:55.724 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436196+0000", 2024-05-08T08:53:55.724 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.077221564000000006 2024-05-08T08:53:55.724 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.724 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.724 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:55.724 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436269+0000", 2024-05-08T08:53:55.725 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2720999999999997e-05 2024-05-08T08:53:55.725 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.725 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.725 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:55.725 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436393+0000", 2024-05-08T08:53:55.725 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012394399999999999 2024-05-08T08:53:55.725 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.725 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:55.725 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.725 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.725 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.725 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28563 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888855520] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:55.725 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:53:55.725 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 23.310033459, 2024-05-08T08:53:55.726 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078795566999999997, 2024-05-08T08:53:55.726 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:55.726 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:55.726 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:55.726 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:55.726 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:55.726 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:55.726 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28563 2024-05-08T08:53:55.726 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.726 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:55.726 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.726 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:55.726 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:53:55.726 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:55.726 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.727 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.727 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:55.727 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:53:55.727 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:55.727 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.727 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.727 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:55.727 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:53:55.727 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1e-07 2024-05-08T08:53:55.727 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.727 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.727 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:55.727 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359080+0000", 2024-05-08T08:53:55.727 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8940000000000001e-06 2024-05-08T08:53:55.728 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.728 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.728 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:55.728 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359083+0000", 2024-05-08T08:53:55.728 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1949999999999999e-06 2024-05-08T08:53:55.728 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.728 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.728 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:55.728 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359085+0000", 2024-05-08T08:53:55.728 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4710000000000001e-06 2024-05-08T08:53:55.728 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.728 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.728 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:55.728 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437604+0000", 2024-05-08T08:53:55.728 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078518278999999996 2024-05-08T08:53:55.729 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.729 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.729 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:55.729 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437704+0000", 2024-05-08T08:53:55.729 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010010599999999999 2024-05-08T08:53:55.729 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.729 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.729 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:55.729 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437870+0000", 2024-05-08T08:53:55.729 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00016641200000000001 2024-05-08T08:53:55.729 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.729 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:55.729 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.729 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.729 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.730 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28564 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888538944] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:55.730 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.359089+0000", 2024-05-08T08:53:55.730 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 23.310019243999999, 2024-05-08T08:53:55.730 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078952675, 2024-05-08T08:53:55.730 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:55.730 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:55.730 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:55.730 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:55.730 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:55.730 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:55.730 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28564 2024-05-08T08:53:55.730 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.730 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:55.730 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.731 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:55.731 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359089+0000", 2024-05-08T08:53:55.731 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:55.731 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.731 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.731 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:55.731 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359089+0000", 2024-05-08T08:53:55.731 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:55.731 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.731 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.731 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:55.731 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360959+0000", 2024-05-08T08:53:55.731 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0018707089999999999 2024-05-08T08:53:55.731 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.732 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.732 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:55.732 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360963+0000", 2024-05-08T08:53:55.732 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0680000000000001e-06 2024-05-08T08:53:55.732 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.732 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.732 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:55.732 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360969+0000", 2024-05-08T08:53:55.732 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1920000000000002e-06 2024-05-08T08:53:55.732 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.732 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.732 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:55.732 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360973+0000", 2024-05-08T08:53:55.732 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.031e-06 2024-05-08T08:53:55.732 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.732 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.733 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:55.733 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437916+0000", 2024-05-08T08:53:55.733 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.076942939000000002 2024-05-08T08:53:55.733 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.733 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.733 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:55.733 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437973+0000", 2024-05-08T08:53:55.733 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7358999999999997e-05 2024-05-08T08:53:55.733 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.733 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.733 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:55.733 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.438041+0000", 2024-05-08T08:53:55.733 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8376999999999999e-05 2024-05-08T08:53:55.733 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.734 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:55.734 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.734 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.734 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.734 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28574 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889427600] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:55.734 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:53:55.734 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 23.308014737000001, 2024-05-08T08:53:55.734 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078782476000000004, 2024-05-08T08:53:55.734 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:55.734 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:55.734 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:55.734 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:55.734 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:55.734 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:55.735 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28574 2024-05-08T08:53:55.735 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.735 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:55.735 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.735 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:55.735 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:53:55.735 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:55.735 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.735 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.735 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:55.735 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:53:55.735 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:55.735 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.735 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.736 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:55.736 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:53:55.736 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1799999999999999e-07 2024-05-08T08:53:55.736 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.736 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.736 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:55.736 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361095+0000", 2024-05-08T08:53:55.736 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3850000000000001e-06 2024-05-08T08:53:55.736 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.736 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.736 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:55.736 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361098+0000", 2024-05-08T08:53:55.736 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7989999999999998e-06 2024-05-08T08:53:55.736 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.737 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.737 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:55.737 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361100+0000", 2024-05-08T08:53:55.737 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4669999999999998e-06 2024-05-08T08:53:55.737 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.737 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.737 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:55.737 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439768+0000", 2024-05-08T08:53:55.737 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078668057999999999 2024-05-08T08:53:55.737 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.737 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.737 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:55.737 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439811+0000", 2024-05-08T08:53:55.737 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2459000000000001e-05 2024-05-08T08:53:55.737 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.738 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.738 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:55.738 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439876+0000", 2024-05-08T08:53:55.738 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5090000000000002e-05 2024-05-08T08:53:55.738 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.738 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:55.738 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.738 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.738 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.738 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28575 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889439216] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:55.738 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361103+0000", 2024-05-08T08:53:55.738 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 23.308004722, 2024-05-08T08:53:55.738 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078966828000000003, 2024-05-08T08:53:55.738 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:55.738 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:55.739 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:55.739 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:55.739 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:55.739 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:55.739 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28575 2024-05-08T08:53:55.739 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.739 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:55.739 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.739 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:55.739 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361103+0000", 2024-05-08T08:53:55.739 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:55.739 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.739 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.739 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:55.740 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361103+0000", 2024-05-08T08:53:55.740 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:55.740 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.740 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.740 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:55.740 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361104+0000", 2024-05-08T08:53:55.740 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1400000000000001e-07 2024-05-08T08:53:55.740 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.740 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.740 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:55.740 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361105+0000", 2024-05-08T08:53:55.740 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5540000000000001e-06 2024-05-08T08:53:55.740 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.740 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.740 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:55.741 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361108+0000", 2024-05-08T08:53:55.741 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.841e-06 2024-05-08T08:53:55.741 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.741 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.741 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:55.741 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361110+0000", 2024-05-08T08:53:55.741 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.497e-06 2024-05-08T08:53:55.741 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.741 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.741 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:55.741 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439916+0000", 2024-05-08T08:53:55.741 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078805372999999998 2024-05-08T08:53:55.741 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.741 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.741 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:55.742 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440000+0000", 2024-05-08T08:53:55.742 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4730999999999999e-05 2024-05-08T08:53:55.742 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.742 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.742 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:55.742 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440070+0000", 2024-05-08T08:53:55.742 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9617999999999997e-05 2024-05-08T08:53:55.742 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.742 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:55.742 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.742 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.742 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.742 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28576 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889451328] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:55.742 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361113+0000", 2024-05-08T08:53:55.743 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 23.307994535999999, 2024-05-08T08:53:55.743 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079088862999999995, 2024-05-08T08:53:55.743 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:55.743 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:55.743 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:55.743 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:55.743 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:55.743 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:55.743 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28576 2024-05-08T08:53:55.743 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.743 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:55.743 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.743 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:55.743 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361113+0000", 2024-05-08T08:53:55.744 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:55.744 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.744 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.744 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:55.744 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361113+0000", 2024-05-08T08:53:55.744 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:55.744 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.744 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.744 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:55.744 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361114+0000", 2024-05-08T08:53:55.744 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1299999999999999e-07 2024-05-08T08:53:55.744 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.744 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.744 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:55.744 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361115+0000", 2024-05-08T08:53:55.745 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.4300000000000001e-06 2024-05-08T08:53:55.745 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.745 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.745 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:55.745 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361118+0000", 2024-05-08T08:53:55.745 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9000000000000002e-06 2024-05-08T08:53:55.745 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.745 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.745 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:55.745 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361120+0000", 2024-05-08T08:53:55.745 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4739999999999999e-06 2024-05-08T08:53:55.745 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.745 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.745 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:55.745 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440096+0000", 2024-05-08T08:53:55.746 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078975629000000006 2024-05-08T08:53:55.746 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.746 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.746 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:55.746 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440135+0000", 2024-05-08T08:53:55.746 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9202000000000002e-05 2024-05-08T08:53:55.746 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.746 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.746 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:55.746 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440202+0000", 2024-05-08T08:53:55.746 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7014999999999994e-05 2024-05-08T08:53:55.746 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.746 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:55.746 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.747 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.747 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.747 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28577 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889462640] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:55.747 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:53:55.747 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 23.307984353999998, 2024-05-08T08:53:55.747 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079194622000000006, 2024-05-08T08:53:55.747 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:55.747 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:55.747 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:55.747 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:55.747 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:55.747 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:55.747 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28577 2024-05-08T08:53:55.747 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.747 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:55.748 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.748 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:55.748 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:53:55.748 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:55.748 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.748 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.748 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:55.748 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:53:55.748 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:55.748 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.748 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.748 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:55.748 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:53:55.748 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1400000000000001e-07 2024-05-08T08:53:55.749 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.749 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.749 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:55.749 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361125+0000", 2024-05-08T08:53:55.749 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.372e-06 2024-05-08T08:53:55.749 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.749 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.749 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:55.749 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361128+0000", 2024-05-08T08:53:55.749 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0309999999999999e-06 2024-05-08T08:53:55.749 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.749 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.749 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:55.749 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361131+0000", 2024-05-08T08:53:55.749 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4040000000000002e-06 2024-05-08T08:53:55.750 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.750 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.750 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:55.750 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440225+0000", 2024-05-08T08:53:55.750 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079094600000000001 2024-05-08T08:53:55.750 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.750 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.750 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:55.750 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440260+0000", 2024-05-08T08:53:55.750 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4703000000000003e-05 2024-05-08T08:53:55.750 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.750 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.750 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:55.750 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440318+0000", 2024-05-08T08:53:55.751 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8298000000000003e-05 2024-05-08T08:53:55.751 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.751 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:55.751 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.751 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.751 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.751 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28578 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889474192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:55.751 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361134+0000", 2024-05-08T08:53:55.751 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 23.307973928999999, 2024-05-08T08:53:55.751 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079310295000000003, 2024-05-08T08:53:55.751 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:55.751 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:55.751 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:55.751 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:55.751 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:55.752 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:55.752 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28578 2024-05-08T08:53:55.752 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.752 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:55.752 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.752 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:55.752 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361134+0000", 2024-05-08T08:53:55.752 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:55.752 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.752 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.752 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:55.752 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361134+0000", 2024-05-08T08:53:55.752 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:55.752 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.752 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.753 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:55.753 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.362958+0000", 2024-05-08T08:53:55.753 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0018237380000000001 2024-05-08T08:53:55.753 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.753 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.753 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:55.753 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.362961+0000", 2024-05-08T08:53:55.753 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.123e-06 2024-05-08T08:53:55.753 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.753 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.753 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:55.753 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.397404+0000", 2024-05-08T08:53:55.753 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.034443226 2024-05-08T08:53:55.753 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.753 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.754 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:55.754 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.397439+0000", 2024-05-08T08:53:55.754 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4690000000000002e-05 2024-05-08T08:53:55.754 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.754 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.754 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:55.754 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440340+0000", 2024-05-08T08:53:55.754 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.042901419000000003 2024-05-08T08:53:55.754 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.754 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.754 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:55.754 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440376+0000", 2024-05-08T08:53:55.754 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5444999999999997e-05 2024-05-08T08:53:55.754 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:55.754 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:55.755 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:55.755 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440444+0000", 2024-05-08T08:53:55.755 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8653999999999997e-05 2024-05-08T08:53:55.755 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.755 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:55.755 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.755 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:55.755 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:55.755 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:55.757 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:53:55.757 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:53:55.759+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:53:55.857 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:55.860+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:55.909 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:55.909 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:53:55.909 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:53:55.909 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:55.917 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:53:55.959 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:55.961+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:56.016 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T08:53:56.060 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:56.063+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:56.067 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:56.068 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:53:56.068 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:53:56.068 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:53:56.068 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:56.077 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:53:56.142 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:53:56.142 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:53:56.162 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:53:56.162 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:53:56.163 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:56.165+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:56.177 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:53:56.178 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:53:56.195 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:53:56.195 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:53:56.213 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:53:56.214 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:53:56.228 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:56.228 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:56.232 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:53:56.232 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:53:56.237 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:53:56.284 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:56.287+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:56.355 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:56.355 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:56.364 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:56.367+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:56.370 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:53:56.465 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:56.468+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:56.496 INFO:teuthology.orchestra.run.smithi196.stdout:truncating image image_client.0-clone32 from 0x5298400 (overlap 0x2990000) to 0x14c8000 2024-05-08T08:53:56.505 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:56.505 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:56.517 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:53:56.565 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:56.568+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:56.614 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-05-08T08:53:56.666 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:56.669+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:56.686 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:56.687 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:56.697 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:53:56.781 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:56.784+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:56.851 INFO:teuthology.orchestra.run.smithi196.stdout:checking clone #31, image image_client.0-clone31 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent32 2024-05-08T08:53:56.858 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:56.858 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:56.867 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:56.870+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:56.868 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:53:56.919 INFO:teuthology.orchestra.run.smithi196.stdout:4561 write 0x762400 thru 0x771265 (0xee66 bytes) 2024-05-08T08:53:56.920 INFO:teuthology.orchestra.run.smithi196.stdout:4563 read 0x3cb8e00 thru 0x3cba87b (0x1a7c bytes) 2024-05-08T08:53:56.961 INFO:teuthology.orchestra.run.smithi196.stdout:4564 write 0x8ebcc00 thru 0x8ebed3e (0x213f bytes) 2024-05-08T08:53:56.962 INFO:teuthology.orchestra.run.smithi196.stdout:4568 write 0x7d60a00 thru 0x7d64738 (0x3d39 bytes) 2024-05-08T08:53:56.962 INFO:teuthology.orchestra.run.smithi196.stdout:4571 write 0x41ac600 thru 0x41b0e0f (0x4810 bytes) 2024-05-08T08:53:56.963 INFO:teuthology.orchestra.run.smithi196.stdout:4573 read 0x6ece600 thru 0x6eda8dc (0xc2dd bytes) 2024-05-08T08:53:56.963 INFO:teuthology.orchestra.run.smithi196.stdout:4582 clone 34 order 23 su 1048576 sc 4 2024-05-08T08:53:56.968 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:56.971+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:57.030 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:57.030 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:57.041 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:53:57.069 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:53:57.071+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:53:57.170 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:57.173+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:57.190 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:57.190 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:57.199 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:53:57.284 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:57.287+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:57.357 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:57.358 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:57.368 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:53:57.372 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:57.375+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:57.473 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:53:57.476+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:53:57.516 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:57.516 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:57.525 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:53:57.574 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:57.577+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:57.606 INFO:teuthology.orchestra.run.smithi059.stderr:nodeep-scrub is unset 2024-05-08T08:53:57.641 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:57.642 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:57.651 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:53:57.675 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:57.678+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:57.775 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:57.778+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:57.784 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:57.785 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:57.794 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:53:57.876 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:53:57.879+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:53:57.917 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:57.917 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:57.926 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:53:57.977 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:57.980+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:58.076 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:58.076 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:58.077 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:53:58.080+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:53:58.084 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:53:58.178 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:58.181+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:58.234 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:58.234 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:58.243 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:53:58.243 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T08:53:58.279 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:58.282+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:58.380 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:53:58.383+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:53:58.396 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:53:58.396 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:53:58.405 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:53:58.410 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:53:58.411 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:53:58.431 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:53:58.432 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:53:58.448 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:53:58.449 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:53:58.466 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:53:58.466 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:53:58.481 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:53:58.484+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:53:58.483 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:53:58.483 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:53:58.503 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:53:58.503 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:53:58.524 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:58.524 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:53:58.524 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:53:58.524 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:53:58.524 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.524 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:12443 3.0 3:02d0158e:::rbd_data.114071bcf828.0000000000000037:head [stat,delete] snapc 19=[19] RETRY=3 ondisk+retry+write+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:58.524 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.610027+0000", 2024-05-08T08:53:58.524 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 26.916072399000001, 2024-05-08T08:53:58.524 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.083586333999999998, 2024-05-08T08:53:58.524 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:58.524 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:58.525 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:58.525 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:58.525 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:58.525 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:58.525 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 12443 2024-05-08T08:53:58.525 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.525 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:58.525 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.525 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:58.525 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610027+0000", 2024-05-08T08:53:58.525 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:58.525 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.525 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.525 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:58.525 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610027+0000", 2024-05-08T08:53:58.526 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:58.526 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.526 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.526 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:58.526 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610028+0000", 2024-05-08T08:53:58.526 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.13e-07 2024-05-08T08:53:58.526 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.526 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.526 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:58.526 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610030+0000", 2024-05-08T08:53:58.526 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7669999999999999e-06 2024-05-08T08:53:58.526 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.526 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.526 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:58.527 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610039+0000", 2024-05-08T08:53:58.527 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4959999999999996e-06 2024-05-08T08:53:58.527 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.527 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.527 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:58.527 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610045+0000", 2024-05-08T08:53:58.527 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6779999999999997e-06 2024-05-08T08:53:58.527 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.527 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.527 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:58.527 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.660873+0000", 2024-05-08T08:53:58.527 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.050828748 2024-05-08T08:53:58.527 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.527 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.527 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:58.528 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.661017+0000", 2024-05-08T08:53:58.528 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014391999999999999 2024-05-08T08:53:58.528 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.528 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.528 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:58.528 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.693613+0000", 2024-05-08T08:53:58.528 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.032596111999999997 2024-05-08T08:53:58.528 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:58.528 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:58.528 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:58.528 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.528 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.528 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:14301 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [watch reconnect cookie 139837155901536 gen 12] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:58.528 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.645078+0000", 2024-05-08T08:53:58.528 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 26.881021761, 2024-05-08T08:53:58.529 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088281852999999993, 2024-05-08T08:53:58.529 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:58.529 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:58.529 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:58.529 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:58.529 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:58.529 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:58.529 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14301 2024-05-08T08:53:58.529 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.529 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:58.529 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.529 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:58.529 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645078+0000", 2024-05-08T08:53:58.529 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:58.529 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.530 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.530 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:58.530 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645078+0000", 2024-05-08T08:53:58.530 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:58.530 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.530 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.530 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:58.530 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645084+0000", 2024-05-08T08:53:58.530 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4620000000000001e-06 2024-05-08T08:53:58.530 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.530 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.530 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:58.530 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645088+0000", 2024-05-08T08:53:58.530 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8039999999999999e-06 2024-05-08T08:53:58.530 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.531 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.531 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:58.531 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645106+0000", 2024-05-08T08:53:58.531 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.8318999999999999e-05 2024-05-08T08:53:58.531 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.531 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.531 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:58.531 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645113+0000", 2024-05-08T08:53:58.531 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0310000000000001e-06 2024-05-08T08:53:58.531 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.531 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.531 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:58.531 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.670493+0000", 2024-05-08T08:53:58.532 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.025379993 2024-05-08T08:53:58.532 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.532 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.532 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:58.532 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.670571+0000", 2024-05-08T08:53:58.532 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7613000000000004e-05 2024-05-08T08:53:58.532 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.532 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.532 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:58.532 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733360+0000", 2024-05-08T08:53:58.532 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.062788630999999998 2024-05-08T08:53:58.532 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:58.532 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:58.532 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:58.532 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.533 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.533 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:14420 3.0 3:06a8948e:::rbd_data.114071bcf828.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:58.533 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.649639+0000", 2024-05-08T08:53:58.533 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 26.876460612999999, 2024-05-08T08:53:58.533 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.083851897999999994, 2024-05-08T08:53:58.533 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:58.533 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:58.533 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:53:58.533 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:58.533 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:58.533 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:58.533 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14420 2024-05-08T08:53:58.533 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.533 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:58.533 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.534 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:58.534 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649639+0000", 2024-05-08T08:53:58.534 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:58.534 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.534 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.534 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:58.534 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649639+0000", 2024-05-08T08:53:58.534 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:58.534 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.534 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.534 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:58.534 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649640+0000", 2024-05-08T08:53:58.534 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6899999999999997e-07 2024-05-08T08:53:58.534 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.534 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.535 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:58.535 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649895+0000", 2024-05-08T08:53:58.535 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00025531000000000002 2024-05-08T08:53:58.535 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.535 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.535 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:58.535 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649905+0000", 2024-05-08T08:53:58.535 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0198e-05 2024-05-08T08:53:58.535 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.535 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.535 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:58.535 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649911+0000", 2024-05-08T08:53:58.535 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2310000000000001e-06 2024-05-08T08:53:58.535 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.536 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.536 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:58.536 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.684099+0000", 2024-05-08T08:53:58.536 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.034188017000000001 2024-05-08T08:53:58.536 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.536 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.536 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:58.536 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.684172+0000", 2024-05-08T08:53:58.536 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3201999999999995e-05 2024-05-08T08:53:58.536 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.536 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.536 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 2", 2024-05-08T08:53:58.536 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.684418+0000", 2024-05-08T08:53:58.536 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00024559099999999997 2024-05-08T08:53:58.536 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.537 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.537 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:53:58.537 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.705538+0000", 2024-05-08T08:53:58.537 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.021119707000000001 2024-05-08T08:53:58.537 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.537 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.537 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:53:58.537 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733399+0000", 2024-05-08T08:53:58.537 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.027861543999999999 2024-05-08T08:53:58.537 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.537 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.537 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:53:58.537 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733445+0000", 2024-05-08T08:53:58.537 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5587000000000002e-05 2024-05-08T08:53:58.537 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.538 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.538 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:58.538 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733491+0000", 2024-05-08T08:53:58.538 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5841999999999999e-05 2024-05-08T08:53:58.538 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:58.538 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:58.538 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:58.538 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.538 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.538 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28547 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889179216] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:58.538 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.355654+0000", 2024-05-08T08:53:58.538 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 26.170445385000001, 2024-05-08T08:53:58.538 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078692261999999999, 2024-05-08T08:53:58.538 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:58.539 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:58.539 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:58.539 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:58.539 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:58.539 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:58.539 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28547 2024-05-08T08:53:58.539 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.539 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:58.539 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.539 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:58.539 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355654+0000", 2024-05-08T08:53:58.539 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:58.539 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.539 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.539 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:58.540 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355654+0000", 2024-05-08T08:53:58.540 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:58.540 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.540 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.540 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:58.540 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355654+0000", 2024-05-08T08:53:58.540 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3999999999999998e-07 2024-05-08T08:53:58.540 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.540 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.540 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:58.540 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355656+0000", 2024-05-08T08:53:58.540 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.353e-06 2024-05-08T08:53:58.540 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.540 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.540 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:58.540 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355663+0000", 2024-05-08T08:53:58.541 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2259999999999999e-06 2024-05-08T08:53:58.541 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.541 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.541 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:58.541 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355666+0000", 2024-05-08T08:53:58.541 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7930000000000002e-06 2024-05-08T08:53:58.541 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.541 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.541 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:58.541 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434117+0000", 2024-05-08T08:53:58.541 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078451231999999996 2024-05-08T08:53:58.541 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.541 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.541 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:58.542 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434198+0000", 2024-05-08T08:53:58.542 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0879999999999998e-05 2024-05-08T08:53:58.542 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.542 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.542 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:58.542 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434346+0000", 2024-05-08T08:53:58.542 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014853800000000001 2024-05-08T08:53:58.542 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:58.542 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:58.542 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:58.542 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.542 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.542 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28548 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889189696] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:58.542 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.355669+0000", 2024-05-08T08:53:58.543 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 26.170430034999999, 2024-05-08T08:53:58.543 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078956606999999998, 2024-05-08T08:53:58.543 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:58.543 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:58.543 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:58.543 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:58.543 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:58.543 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:58.543 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28548 2024-05-08T08:53:58.543 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.543 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:58.543 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.543 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:58.543 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355669+0000", 2024-05-08T08:53:58.543 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:58.544 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.544 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.544 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:58.544 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355669+0000", 2024-05-08T08:53:58.544 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:58.544 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.544 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.544 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:58.544 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355762+0000", 2024-05-08T08:53:58.544 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3015000000000003e-05 2024-05-08T08:53:58.544 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.544 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.544 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:58.544 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355765+0000", 2024-05-08T08:53:58.544 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.368e-06 2024-05-08T08:53:58.544 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.545 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.545 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:58.545 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355771+0000", 2024-05-08T08:53:58.545 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5989999999999998e-06 2024-05-08T08:53:58.545 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.545 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.545 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:58.545 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355775+0000", 2024-05-08T08:53:58.545 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7280000000000001e-06 2024-05-08T08:53:58.545 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.545 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.545 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:58.545 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434379+0000", 2024-05-08T08:53:58.545 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078603662000000005 2024-05-08T08:53:58.545 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.545 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.546 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:58.546 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434473+0000", 2024-05-08T08:53:58.546 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4104000000000006e-05 2024-05-08T08:53:58.546 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.546 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.546 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:58.546 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434626+0000", 2024-05-08T08:53:58.546 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00015313100000000001 2024-05-08T08:53:58.546 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:58.546 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:58.546 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:58.546 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.546 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.546 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28549 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889202400] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:58.546 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.355781+0000", 2024-05-08T08:53:58.547 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 26.170318461000001, 2024-05-08T08:53:58.547 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079117946999999994, 2024-05-08T08:53:58.547 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:58.547 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:58.547 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:58.547 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:58.547 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:58.547 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:58.547 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28549 2024-05-08T08:53:58.547 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.547 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:58.547 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.547 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:58.547 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355781+0000", 2024-05-08T08:53:58.547 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:58.547 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.547 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.548 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:58.548 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355781+0000", 2024-05-08T08:53:58.548 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:58.548 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.548 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.548 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:58.548 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356162+0000", 2024-05-08T08:53:58.548 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00038099100000000001 2024-05-08T08:53:58.548 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.548 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.548 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:58.548 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356165+0000", 2024-05-08T08:53:58.548 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1240000000000001e-06 2024-05-08T08:53:58.548 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.548 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.548 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:58.549 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356173+0000", 2024-05-08T08:53:58.549 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.6199999999999999e-06 2024-05-08T08:53:58.549 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.549 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.549 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:58.549 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356182+0000", 2024-05-08T08:53:58.549 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7760000000000003e-06 2024-05-08T08:53:58.549 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.549 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.549 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:58.549 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434675+0000", 2024-05-08T08:53:58.549 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078493119 2024-05-08T08:53:58.549 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.549 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.549 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:58.549 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434765+0000", 2024-05-08T08:53:58.550 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0402000000000006e-05 2024-05-08T08:53:58.550 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.550 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.550 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:58.550 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434899+0000", 2024-05-08T08:53:58.550 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013391499999999999 2024-05-08T08:53:58.550 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:58.550 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:58.550 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:58.550 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.550 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.550 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28550 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888597280] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:58.550 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:53:58.550 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 26.169907415000001, 2024-05-08T08:53:58.550 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078933990999999995, 2024-05-08T08:53:58.551 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:58.551 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:58.551 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:58.551 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:58.551 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:58.551 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:58.551 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28550 2024-05-08T08:53:58.551 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.551 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:58.551 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.551 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:58.551 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:53:58.551 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:58.551 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.551 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.551 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:58.551 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:53:58.552 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:58.552 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.552 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.552 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:58.552 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:53:58.552 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.34e-07 2024-05-08T08:53:58.552 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.552 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.552 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:58.552 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356195+0000", 2024-05-08T08:53:58.552 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8710000000000001e-06 2024-05-08T08:53:58.552 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.552 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.552 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:58.552 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356200+0000", 2024-05-08T08:53:58.552 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3000000000000003e-06 2024-05-08T08:53:58.553 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.553 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.553 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:58.553 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356203+0000", 2024-05-08T08:53:58.553 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4039999999999999e-06 2024-05-08T08:53:58.553 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.553 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.553 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:58.553 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434928+0000", 2024-05-08T08:53:58.553 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078725331999999995 2024-05-08T08:53:58.553 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.553 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.553 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:58.553 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435007+0000", 2024-05-08T08:53:58.553 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8415000000000001e-05 2024-05-08T08:53:58.553 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.553 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.554 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:58.554 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435126+0000", 2024-05-08T08:53:58.554 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011923500000000001 2024-05-08T08:53:58.554 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:58.554 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:58.554 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:58.554 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.554 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.554 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28551 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888527200] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:58.554 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.356210+0000", 2024-05-08T08:53:58.554 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 26.169889067, 2024-05-08T08:53:58.554 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079195072000000005, 2024-05-08T08:53:58.554 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:58.554 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:58.554 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:58.555 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:58.555 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:58.555 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:58.555 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28551 2024-05-08T08:53:58.555 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.555 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:58.555 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.555 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:58.555 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356210+0000", 2024-05-08T08:53:58.555 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:58.555 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.555 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.555 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:58.555 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356210+0000", 2024-05-08T08:53:58.555 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:58.555 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.555 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.556 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:58.556 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356211+0000", 2024-05-08T08:53:58.556 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4499999999999998e-07 2024-05-08T08:53:58.556 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.556 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.556 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:58.556 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356212+0000", 2024-05-08T08:53:58.556 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6029999999999999e-06 2024-05-08T08:53:58.556 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.556 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.556 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:58.556 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356221+0000", 2024-05-08T08:53:58.556 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4200000000000007e-06 2024-05-08T08:53:58.556 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.556 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.556 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:58.556 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356224+0000", 2024-05-08T08:53:58.557 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.038e-06 2024-05-08T08:53:58.557 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.557 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.557 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:58.557 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435178+0000", 2024-05-08T08:53:58.557 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078954625000000001 2024-05-08T08:53:58.557 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.557 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.557 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:58.557 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435256+0000", 2024-05-08T08:53:58.557 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7337999999999997e-05 2024-05-08T08:53:58.557 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.557 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.557 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:58.557 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435405+0000", 2024-05-08T08:53:58.557 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000149703 2024-05-08T08:53:58.558 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:58.558 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:58.558 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:58.558 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.558 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.558 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28552 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889231456] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:58.558 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.356228+0000", 2024-05-08T08:53:58.558 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 26.169871389000001, 2024-05-08T08:53:58.558 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079346920000000001, 2024-05-08T08:53:58.558 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:58.558 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:58.558 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:58.558 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:58.558 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:58.558 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:58.559 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28552 2024-05-08T08:53:58.559 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.559 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:58.559 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.559 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:58.559 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356228+0000", 2024-05-08T08:53:58.559 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:58.559 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.559 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.559 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:58.559 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356228+0000", 2024-05-08T08:53:58.559 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:58.559 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.559 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.559 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:58.559 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356959+0000", 2024-05-08T08:53:58.560 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00073097099999999996 2024-05-08T08:53:58.560 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.560 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.560 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:58.560 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356967+0000", 2024-05-08T08:53:58.560 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0409999999999998e-06 2024-05-08T08:53:58.560 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.560 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.560 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:58.560 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356979+0000", 2024-05-08T08:53:58.560 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1489e-05 2024-05-08T08:53:58.560 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.560 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.560 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:58.560 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356986+0000", 2024-05-08T08:53:58.560 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3699999999999997e-06 2024-05-08T08:53:58.560 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.561 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.561 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:58.561 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435433+0000", 2024-05-08T08:53:58.561 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078447520000000007 2024-05-08T08:53:58.561 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.561 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.561 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:58.561 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435484+0000", 2024-05-08T08:53:58.561 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0229000000000003e-05 2024-05-08T08:53:58.561 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.561 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.561 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:58.561 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435575+0000", 2024-05-08T08:53:58.561 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1299999999999997e-05 2024-05-08T08:53:58.561 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:58.561 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:58.561 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:58.562 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.562 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.562 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28553 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888713648] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:58.562 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:53:58.562 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 26.169096731, 2024-05-08T08:53:58.562 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078782034000000001, 2024-05-08T08:53:58.562 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:58.562 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:58.562 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:58.562 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:58.562 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:58.562 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:58.562 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28553 2024-05-08T08:53:58.562 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.562 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:58.563 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.563 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:58.563 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:53:58.563 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:58.563 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.563 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.563 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:58.563 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:53:58.563 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:58.563 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.563 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.563 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:58.563 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:53:58.563 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6600000000000002e-07 2024-05-08T08:53:58.564 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.564 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.564 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:58.564 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357006+0000", 2024-05-08T08:53:58.564 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7190000000000001e-06 2024-05-08T08:53:58.564 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.564 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.564 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:58.564 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357013+0000", 2024-05-08T08:53:58.564 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.435e-06 2024-05-08T08:53:58.564 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.564 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.564 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:58.564 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357018+0000", 2024-05-08T08:53:58.564 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7260000000000002e-06 2024-05-08T08:53:58.565 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.565 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.565 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:58.565 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435623+0000", 2024-05-08T08:53:58.565 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078604972999999995 2024-05-08T08:53:58.565 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.565 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.565 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:58.565 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435695+0000", 2024-05-08T08:53:58.565 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1959999999999995e-05 2024-05-08T08:53:58.565 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.565 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.565 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:58.565 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435785+0000", 2024-05-08T08:53:58.565 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.9754999999999997e-05 2024-05-08T08:53:58.565 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:58.566 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:58.566 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:58.566 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.566 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.566 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28554 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888980272] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:58.566 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357024+0000", 2024-05-08T08:53:58.566 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 26.169075095, 2024-05-08T08:53:58.566 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078999870999999999, 2024-05-08T08:53:58.566 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:58.566 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:58.566 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:58.566 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:58.566 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:58.566 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:58.566 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28554 2024-05-08T08:53:58.567 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.567 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:58.567 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.567 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:58.567 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357024+0000", 2024-05-08T08:53:58.567 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:58.567 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.567 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.567 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:58.567 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357024+0000", 2024-05-08T08:53:58.567 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:58.567 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.567 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.568 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:58.568 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357025+0000", 2024-05-08T08:53:58.568 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:53:58.568 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.568 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.568 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:58.568 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357027+0000", 2024-05-08T08:53:58.568 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2050000000000001e-06 2024-05-08T08:53:58.568 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.568 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.568 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:58.568 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357033+0000", 2024-05-08T08:53:58.568 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5679999999999996e-06 2024-05-08T08:53:58.568 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.568 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.569 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:58.569 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357038+0000", 2024-05-08T08:53:58.569 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2889999999999999e-06 2024-05-08T08:53:58.569 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.569 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.569 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:58.569 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435823+0000", 2024-05-08T08:53:58.569 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078785677999999998 2024-05-08T08:53:58.569 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.569 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.569 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:58.569 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435892+0000", 2024-05-08T08:53:58.569 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8379000000000004e-05 2024-05-08T08:53:58.569 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.569 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.569 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:58.570 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436024+0000", 2024-05-08T08:53:58.570 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013239799999999999 2024-05-08T08:53:58.570 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:58.570 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:58.570 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:58.570 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.570 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.570 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28555 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889001104] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:58.570 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:53:58.570 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 26.16905633, 2024-05-08T08:53:58.570 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079115742000000003, 2024-05-08T08:53:58.570 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:58.570 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:58.570 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:58.570 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:58.571 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:58.571 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:58.571 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28555 2024-05-08T08:53:58.571 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.571 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:58.571 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.571 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:58.571 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:53:58.571 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:58.571 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.571 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.571 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:58.571 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:53:58.571 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:58.571 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.572 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.572 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:58.572 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:53:58.572 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9200000000000002e-07 2024-05-08T08:53:58.572 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.572 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.572 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:58.572 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357046+0000", 2024-05-08T08:53:58.572 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1670000000000002e-06 2024-05-08T08:53:58.572 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.572 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.572 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:58.572 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357052+0000", 2024-05-08T08:53:58.572 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.139e-06 2024-05-08T08:53:58.573 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.573 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.573 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:58.573 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357056+0000", 2024-05-08T08:53:58.573 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2440000000000004e-06 2024-05-08T08:53:58.573 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.573 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.573 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:58.573 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436059+0000", 2024-05-08T08:53:58.573 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079003070999999994 2024-05-08T08:53:58.573 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.573 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.573 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:58.573 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436097+0000", 2024-05-08T08:53:58.573 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7784000000000003e-05 2024-05-08T08:53:58.573 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.574 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.574 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:58.574 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436159+0000", 2024-05-08T08:53:58.574 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2045000000000007e-05 2024-05-08T08:53:58.574 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:58.574 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:58.574 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:58.574 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.574 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.574 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28556 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889022064] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:58.574 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357065+0000", 2024-05-08T08:53:58.574 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 26.169034297, 2024-05-08T08:53:58.574 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079327976999999994, 2024-05-08T08:53:58.574 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:58.575 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:58.575 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:58.575 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:58.575 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:58.575 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:58.575 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28556 2024-05-08T08:53:58.575 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.575 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:58.575 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.575 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:58.575 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357065+0000", 2024-05-08T08:53:58.575 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:58.575 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.575 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.575 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:58.576 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357065+0000", 2024-05-08T08:53:58.576 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:58.576 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.576 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.576 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:58.576 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358958+0000", 2024-05-08T08:53:58.576 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001892915 2024-05-08T08:53:58.576 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.576 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.576 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:58.576 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358962+0000", 2024-05-08T08:53:58.576 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4599999999999999e-06 2024-05-08T08:53:58.576 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.576 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.577 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:58.577 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358970+0000", 2024-05-08T08:53:58.577 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.6090000000000001e-06 2024-05-08T08:53:58.577 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.577 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.577 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:58.577 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358975+0000", 2024-05-08T08:53:58.577 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7639999999999996e-06 2024-05-08T08:53:58.577 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.577 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.577 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:58.577 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436196+0000", 2024-05-08T08:53:58.577 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.077221564000000006 2024-05-08T08:53:58.577 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.577 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.577 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:58.578 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436269+0000", 2024-05-08T08:53:58.578 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2720999999999997e-05 2024-05-08T08:53:58.578 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.578 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.578 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:58.578 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436393+0000", 2024-05-08T08:53:58.578 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012394399999999999 2024-05-08T08:53:58.578 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:58.578 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:58.578 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:58.578 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.578 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.578 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28563 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888855520] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:58.579 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:53:58.579 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 26.167024898000001, 2024-05-08T08:53:58.579 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078795566999999997, 2024-05-08T08:53:58.579 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:58.579 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:58.579 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:58.579 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:58.579 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:58.579 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:58.579 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28563 2024-05-08T08:53:58.579 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.579 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:58.579 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.579 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:58.579 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:53:58.580 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:58.580 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.580 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.580 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:58.580 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:53:58.580 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:58.580 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.580 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.580 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:58.580 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:53:58.580 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1e-07 2024-05-08T08:53:58.580 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.580 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.580 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:58.581 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359080+0000", 2024-05-08T08:53:58.581 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8940000000000001e-06 2024-05-08T08:53:58.581 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.581 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.581 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:58.581 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359083+0000", 2024-05-08T08:53:58.581 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1949999999999999e-06 2024-05-08T08:53:58.581 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.581 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.581 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:58.581 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359085+0000", 2024-05-08T08:53:58.581 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4710000000000001e-06 2024-05-08T08:53:58.581 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.581 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.581 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:58.582 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437604+0000", 2024-05-08T08:53:58.582 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078518278999999996 2024-05-08T08:53:58.582 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.582 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.582 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:58.582 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437704+0000", 2024-05-08T08:53:58.582 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010010599999999999 2024-05-08T08:53:58.582 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.582 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.582 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:58.582 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437870+0000", 2024-05-08T08:53:58.582 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00016641200000000001 2024-05-08T08:53:58.582 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:58.582 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:58.582 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:58.583 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.583 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.583 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28564 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888538944] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:58.583 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.359089+0000", 2024-05-08T08:53:58.583 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 26.167010683000001, 2024-05-08T08:53:58.583 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078952675, 2024-05-08T08:53:58.583 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:58.583 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:58.583 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:58.583 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:58.583 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:58.583 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:58.583 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28564 2024-05-08T08:53:58.583 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.583 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:58.584 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.584 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:58.584 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359089+0000", 2024-05-08T08:53:58.584 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:58.584 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.584 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.584 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:58.584 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359089+0000", 2024-05-08T08:53:58.584 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:58.584 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.584 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.584 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:58.584 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360959+0000", 2024-05-08T08:53:58.584 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0018707089999999999 2024-05-08T08:53:58.584 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.585 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.585 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:58.585 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360963+0000", 2024-05-08T08:53:58.585 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0680000000000001e-06 2024-05-08T08:53:58.585 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.585 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.585 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:58.585 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360969+0000", 2024-05-08T08:53:58.585 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1920000000000002e-06 2024-05-08T08:53:58.585 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.585 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.585 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:58.585 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360973+0000", 2024-05-08T08:53:58.585 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.031e-06 2024-05-08T08:53:58.585 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.586 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.586 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:58.586 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437916+0000", 2024-05-08T08:53:58.586 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.076942939000000002 2024-05-08T08:53:58.586 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.586 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.586 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:58.586 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437973+0000", 2024-05-08T08:53:58.586 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7358999999999997e-05 2024-05-08T08:53:58.586 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.586 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.586 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:58.586 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.438041+0000", 2024-05-08T08:53:58.586 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8376999999999999e-05 2024-05-08T08:53:58.586 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:58.587 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:58.587 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:58.587 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.587 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.587 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28574 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889427600] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:58.587 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:53:58.587 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 26.165006175999999, 2024-05-08T08:53:58.587 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078782476000000004, 2024-05-08T08:53:58.587 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:58.587 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:58.587 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:58.587 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:58.587 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:58.587 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:58.587 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28574 2024-05-08T08:53:58.588 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.588 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:58.588 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.588 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:58.588 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:53:58.588 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:58.588 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.588 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.588 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:58.588 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:53:58.588 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:58.588 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.588 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.588 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:58.588 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:53:58.589 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1799999999999999e-07 2024-05-08T08:53:58.589 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.589 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.589 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:58.589 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361095+0000", 2024-05-08T08:53:58.589 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3850000000000001e-06 2024-05-08T08:53:58.589 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.589 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.589 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:58.589 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361098+0000", 2024-05-08T08:53:58.589 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7989999999999998e-06 2024-05-08T08:53:58.589 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.589 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.589 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:58.590 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361100+0000", 2024-05-08T08:53:58.590 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4669999999999998e-06 2024-05-08T08:53:58.590 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.590 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.590 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:58.590 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439768+0000", 2024-05-08T08:53:58.590 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078668057999999999 2024-05-08T08:53:58.590 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.590 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.590 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:58.590 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439811+0000", 2024-05-08T08:53:58.590 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2459000000000001e-05 2024-05-08T08:53:58.590 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.590 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.591 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:58.591 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439876+0000", 2024-05-08T08:53:58.591 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5090000000000002e-05 2024-05-08T08:53:58.591 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:58.591 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:58.591 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:58.591 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.591 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.591 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28575 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889439216] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:58.591 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361103+0000", 2024-05-08T08:53:58.591 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 26.164996161000001, 2024-05-08T08:53:58.591 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078966828000000003, 2024-05-08T08:53:58.591 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:58.591 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:58.592 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:58.592 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:58.592 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:58.592 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:58.592 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28575 2024-05-08T08:53:58.592 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.592 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:58.592 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.592 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:58.592 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361103+0000", 2024-05-08T08:53:58.592 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:58.592 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.592 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.592 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:58.592 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361103+0000", 2024-05-08T08:53:58.593 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:58.593 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.593 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.593 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:58.593 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361104+0000", 2024-05-08T08:53:58.593 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1400000000000001e-07 2024-05-08T08:53:58.593 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.593 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.593 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:58.593 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361105+0000", 2024-05-08T08:53:58.593 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5540000000000001e-06 2024-05-08T08:53:58.593 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.593 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.593 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:58.593 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361108+0000", 2024-05-08T08:53:58.593 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.841e-06 2024-05-08T08:53:58.594 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.594 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.594 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:58.594 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361110+0000", 2024-05-08T08:53:58.594 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.497e-06 2024-05-08T08:53:58.594 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.594 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.594 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:58.594 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439916+0000", 2024-05-08T08:53:58.594 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078805372999999998 2024-05-08T08:53:58.594 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.594 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.594 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:58.594 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440000+0000", 2024-05-08T08:53:58.595 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4730999999999999e-05 2024-05-08T08:53:58.595 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.595 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.595 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:58.595 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440070+0000", 2024-05-08T08:53:58.595 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9617999999999997e-05 2024-05-08T08:53:58.595 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:58.595 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:58.595 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:58.595 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.595 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.595 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28576 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889451328] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:58.595 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361113+0000", 2024-05-08T08:53:58.595 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 26.164985975, 2024-05-08T08:53:58.595 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079088862999999995, 2024-05-08T08:53:58.596 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:58.596 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:58.596 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:58.596 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:58.596 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:58.596 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:58.596 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28576 2024-05-08T08:53:58.596 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.596 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:58.596 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.596 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:58.596 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361113+0000", 2024-05-08T08:53:58.596 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:58.596 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.596 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.597 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:58.597 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361113+0000", 2024-05-08T08:53:58.597 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:58.597 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.597 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.597 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:58.597 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361114+0000", 2024-05-08T08:53:58.597 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1299999999999999e-07 2024-05-08T08:53:58.597 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.597 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.597 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:58.597 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361115+0000", 2024-05-08T08:53:58.597 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.4300000000000001e-06 2024-05-08T08:53:58.597 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.597 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.597 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:58.598 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361118+0000", 2024-05-08T08:53:58.598 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9000000000000002e-06 2024-05-08T08:53:58.598 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.598 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.598 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:58.598 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361120+0000", 2024-05-08T08:53:58.598 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4739999999999999e-06 2024-05-08T08:53:58.598 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.598 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.598 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:58.598 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440096+0000", 2024-05-08T08:53:58.598 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078975629000000006 2024-05-08T08:53:58.598 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.598 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.598 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:58.599 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440135+0000", 2024-05-08T08:53:58.599 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9202000000000002e-05 2024-05-08T08:53:58.599 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.599 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.599 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:58.599 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440202+0000", 2024-05-08T08:53:58.599 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7014999999999994e-05 2024-05-08T08:53:58.599 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:58.599 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:58.599 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:58.599 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.599 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.599 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28577 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889462640] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:58.599 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:53:58.600 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 26.164975793, 2024-05-08T08:53:58.600 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079194622000000006, 2024-05-08T08:53:58.600 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:58.600 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:58.600 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:58.600 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:58.600 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:58.600 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:58.600 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28577 2024-05-08T08:53:58.600 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.600 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:58.600 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.600 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:58.600 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:53:58.600 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:58.601 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.601 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.601 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:58.601 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:53:58.601 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:58.601 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.601 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.601 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:58.601 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:53:58.601 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1400000000000001e-07 2024-05-08T08:53:58.601 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.601 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.601 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:58.601 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361125+0000", 2024-05-08T08:53:58.601 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.372e-06 2024-05-08T08:53:58.602 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.602 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.602 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:58.602 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361128+0000", 2024-05-08T08:53:58.602 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0309999999999999e-06 2024-05-08T08:53:58.602 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.602 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.602 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:58.602 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361131+0000", 2024-05-08T08:53:58.602 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4040000000000002e-06 2024-05-08T08:53:58.602 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.602 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.602 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:58.602 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440225+0000", 2024-05-08T08:53:58.603 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079094600000000001 2024-05-08T08:53:58.603 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.603 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.603 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:58.603 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440260+0000", 2024-05-08T08:53:58.603 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4703000000000003e-05 2024-05-08T08:53:58.603 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.603 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.603 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:58.603 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440318+0000", 2024-05-08T08:53:58.603 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8298000000000003e-05 2024-05-08T08:53:58.603 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:58.603 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:58.603 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:58.603 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.603 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.604 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28578 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889474192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:58.604 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361134+0000", 2024-05-08T08:53:58.604 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 26.164965368000001, 2024-05-08T08:53:58.604 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079310295000000003, 2024-05-08T08:53:58.604 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:58.604 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:58.604 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:58.604 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:58.604 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:53:58.604 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:58.604 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28578 2024-05-08T08:53:58.604 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.604 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:58.604 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.604 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:58.604 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361134+0000", 2024-05-08T08:53:58.605 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:58.605 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.605 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.605 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:58.605 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361134+0000", 2024-05-08T08:53:58.605 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:58.605 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.605 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.605 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:58.605 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.362958+0000", 2024-05-08T08:53:58.605 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0018237380000000001 2024-05-08T08:53:58.605 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.605 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.605 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:58.605 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.362961+0000", 2024-05-08T08:53:58.605 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.123e-06 2024-05-08T08:53:58.606 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.606 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.606 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:58.606 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.397404+0000", 2024-05-08T08:53:58.606 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.034443226 2024-05-08T08:53:58.606 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.606 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.606 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:58.606 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.397439+0000", 2024-05-08T08:53:58.606 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4690000000000002e-05 2024-05-08T08:53:58.606 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.606 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.606 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:58.606 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440340+0000", 2024-05-08T08:53:58.607 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.042901419000000003 2024-05-08T08:53:58.607 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.607 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.607 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:58.607 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440376+0000", 2024-05-08T08:53:58.607 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5444999999999997e-05 2024-05-08T08:53:58.607 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:58.607 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:58.607 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:58.607 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440444+0000", 2024-05-08T08:53:58.607 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8653999999999997e-05 2024-05-08T08:53:58.607 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:58.607 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:58.607 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:58.607 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:58.607 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:58.608 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:58.609 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:53:58.610 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:58.612+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:58.710 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:58.712+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:58.760 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:58.760 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:53:58.760 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:53:58.760 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:58.768 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:53:58.810 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:58.813+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:58.911 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:58.914+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:58.917 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:58.917 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:53:58.917 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:53:58.917 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:53:58.917 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:58.926 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:53:59.013 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:59.016+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:59.037 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:53:59.037 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:53:59.038 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:53:59.038 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:53:59.038 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.038 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:437 3.17 3:ea93a512:::rbd_data.1140100c9703.0000000000000077:head [set-alloc-hint object_size 2097152 write_size 2097152,write 1170432~1024] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e63)", 2024-05-08T08:53:59.038 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:08.390804+0000", 2024-05-08T08:53:59.038 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 50.648698920999998, 2024-05-08T08:53:59.038 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0406068690000001, 2024-05-08T08:53:59.038 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:59.038 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:59.038 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:53:59.038 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:59.038 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:59.039 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:59.039 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 437 2024-05-08T08:53:59.039 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.039 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:59.039 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.039 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:59.039 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390804+0000", 2024-05-08T08:53:59.039 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:59.039 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.039 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.039 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:59.039 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390804+0000", 2024-05-08T08:53:59.039 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:59.039 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.039 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.040 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:59.040 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390806+0000", 2024-05-08T08:53:59.040 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.666e-06 2024-05-08T08:53:59.040 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.040 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.040 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:59.040 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390818+0000", 2024-05-08T08:53:59.040 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2187999999999999e-05 2024-05-08T08:53:59.040 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.040 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.040 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:59.040 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390834+0000", 2024-05-08T08:53:59.040 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6191000000000001e-05 2024-05-08T08:53:59.040 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.041 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.041 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:59.041 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390845+0000", 2024-05-08T08:53:59.041 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0831e-05 2024-05-08T08:53:59.041 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.041 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.041 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:59.041 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390922+0000", 2024-05-08T08:53:59.041 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7253e-05 2024-05-08T08:53:59.041 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.041 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.041 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op must wait for map", 2024-05-08T08:53:59.041 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390930+0000", 2024-05-08T08:53:59.041 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8259999999999993e-06 2024-05-08T08:53:59.041 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.042 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.042 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:59.042 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392155+0000", 2024-05-08T08:53:59.042 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0012256649999999999 2024-05-08T08:53:59.042 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.042 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.042 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:53:59.042 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392165+0000", 2024-05-08T08:53:59.042 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5799999999999998e-06 2024-05-08T08:53:59.042 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.042 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.042 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:59.042 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.429596+0000", 2024-05-08T08:53:59.043 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0374313639999999 2024-05-08T08:53:59.043 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.043 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.043 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:59.043 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.429728+0000", 2024-05-08T08:53:59.043 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000131552 2024-05-08T08:53:59.043 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.043 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.043 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 0,1", 2024-05-08T08:53:59.043 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.429860+0000", 2024-05-08T08:53:59.044 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000132137 2024-05-08T08:53:59.044 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.044 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.044 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:53:59.044 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.430811+0000", 2024-05-08T08:53:59.044 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00095040200000000004 2024-05-08T08:53:59.044 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.044 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.044 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:53:59.044 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431126+0000", 2024-05-08T08:53:59.044 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000315006 2024-05-08T08:53:59.044 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.044 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.044 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:53:59.045 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431272+0000", 2024-05-08T08:53:59.045 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000146487 2024-05-08T08:53:59.045 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.045 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.045 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:53:59.045 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431315+0000", 2024-05-08T08:53:59.045 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2697000000000001e-05 2024-05-08T08:53:59.045 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.045 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.045 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:59.045 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431411+0000", 2024-05-08T08:53:59.045 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6024000000000001e-05 2024-05-08T08:53:59.046 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:59.046 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:59.046 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:59.046 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.046 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.046 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:439 3.17 3:ea93a512:::rbd_data.1140100c9703.0000000000000077:head [read 1163264~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-05-08T08:53:59.046 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:08.390871+0000", 2024-05-08T08:53:59.046 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 50.64863141, 2024-05-08T08:53:59.046 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0408790590000001, 2024-05-08T08:53:59.046 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:59.046 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:59.047 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:59.047 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:59.047 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:59.047 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:59.047 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 439 2024-05-08T08:53:59.047 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.047 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:59.047 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.047 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:59.048 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390871+0000", 2024-05-08T08:53:59.048 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:59.048 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.048 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.048 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:59.048 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390871+0000", 2024-05-08T08:53:59.048 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:59.048 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.048 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.048 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:59.048 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390872+0000", 2024-05-08T08:53:59.048 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6300000000000005e-07 2024-05-08T08:53:59.048 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.048 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.048 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:59.049 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390875+0000", 2024-05-08T08:53:59.049 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3459999999999998e-06 2024-05-08T08:53:59.049 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.049 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.049 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:59.049 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390891+0000", 2024-05-08T08:53:59.049 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5255e-05 2024-05-08T08:53:59.049 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.049 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.049 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:59.049 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390897+0000", 2024-05-08T08:53:59.049 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2360000000000001e-06 2024-05-08T08:53:59.049 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.049 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.050 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:59.050 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390995+0000", 2024-05-08T08:53:59.050 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8522000000000003e-05 2024-05-08T08:53:59.050 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.050 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.050 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting_for_map not empty", 2024-05-08T08:53:59.050 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.391005+0000", 2024-05-08T08:53:59.050 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1460000000000002e-06 2024-05-08T08:53:59.050 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.050 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.051 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:59.051 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392191+0000", 2024-05-08T08:53:59.051 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0011860270000000001 2024-05-08T08:53:59.051 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.051 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.051 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:53:59.051 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392197+0000", 2024-05-08T08:53:59.051 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4289999999999998e-06 2024-05-08T08:53:59.051 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.051 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.051 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:59.051 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.430596+0000", 2024-05-08T08:53:59.051 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0383989090000001 2024-05-08T08:53:59.052 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.052 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.052 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for rw locks", 2024-05-08T08:53:59.052 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.430676+0000", 2024-05-08T08:53:59.052 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9781000000000001e-05 2024-05-08T08:53:59.052 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.052 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.052 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:59.052 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431460+0000", 2024-05-08T08:53:59.052 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00078427300000000004 2024-05-08T08:53:59.052 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.053 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.053 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:59.053 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431540+0000", 2024-05-08T08:53:59.053 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0224999999999998e-05 2024-05-08T08:53:59.053 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.053 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.053 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:59.053 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431750+0000", 2024-05-08T08:53:59.053 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00021024699999999999 2024-05-08T08:53:59.053 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:59.053 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:59.053 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:59.053 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.053 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.053 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:945 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 4096000~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:59.053 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:53:59.054 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 49.588481842999997, 2024-05-08T08:53:59.054 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086774741000000002, 2024-05-08T08:53:59.054 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:59.054 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:59.054 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:59.054 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:59.054 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:59.054 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:59.054 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 945 2024-05-08T08:53:59.054 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.054 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:59.054 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.054 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:59.054 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:53:59.055 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:59.055 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.055 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.055 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:59.055 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:53:59.055 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:59.055 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.055 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.055 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:59.055 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:53:59.055 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3499999999999996e-07 2024-05-08T08:53:59.055 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.055 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.055 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:59.056 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451024+0000", 2024-05-08T08:53:59.056 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.627e-06 2024-05-08T08:53:59.056 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.056 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.056 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:59.056 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451037+0000", 2024-05-08T08:53:59.056 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2431e-05 2024-05-08T08:53:59.056 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.056 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.056 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:59.056 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451042+0000", 2024-05-08T08:53:59.056 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8409999999999996e-06 2024-05-08T08:53:59.056 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.056 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.056 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:59.057 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537608+0000", 2024-05-08T08:53:59.057 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086565243 2024-05-08T08:53:59.057 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.057 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.057 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:59.057 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537687+0000", 2024-05-08T08:53:59.057 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9519999999999998e-05 2024-05-08T08:53:59.057 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.057 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.057 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:59.057 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537796+0000", 2024-05-08T08:53:59.057 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000108544 2024-05-08T08:53:59.057 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:59.058 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:59.058 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:59.058 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.058 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.058 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:964 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 458752~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:59.058 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451051+0000", 2024-05-08T08:53:59.058 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 49.588451669999998, 2024-05-08T08:53:59.058 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086990490000000004, 2024-05-08T08:53:59.058 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:59.059 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:59.059 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:59.059 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:59.059 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:59.059 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:59.059 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 964 2024-05-08T08:53:59.059 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.059 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:59.059 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.059 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:59.059 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451051+0000", 2024-05-08T08:53:59.059 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:59.059 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.059 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.059 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:59.060 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451051+0000", 2024-05-08T08:53:59.060 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:59.060 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.060 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.060 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:59.060 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451052+0000", 2024-05-08T08:53:59.060 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7299999999999996e-07 2024-05-08T08:53:59.060 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.060 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.060 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:59.060 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451055+0000", 2024-05-08T08:53:59.060 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.2770000000000001e-06 2024-05-08T08:53:59.060 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.060 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.061 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:59.061 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451061+0000", 2024-05-08T08:53:59.061 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4060000000000002e-06 2024-05-08T08:53:59.061 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.061 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.061 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:59.061 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451066+0000", 2024-05-08T08:53:59.061 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0839999999999999e-06 2024-05-08T08:53:59.061 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.061 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.061 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:59.061 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537839+0000", 2024-05-08T08:53:59.061 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086772962999999995 2024-05-08T08:53:59.061 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.061 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.062 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:59.062 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537934+0000", 2024-05-08T08:53:59.062 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4742999999999997e-05 2024-05-08T08:53:59.062 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.062 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.062 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:59.062 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538042+0000", 2024-05-08T08:53:59.062 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000107444 2024-05-08T08:53:59.062 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:59.062 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:59.062 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:59.062 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.062 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.063 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:965 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 950272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:59.063 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451075+0000", 2024-05-08T08:53:59.063 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 49.588427608000003, 2024-05-08T08:53:59.063 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087230196999999995, 2024-05-08T08:53:59.063 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:59.063 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:59.063 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:59.063 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:59.063 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:59.063 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:59.063 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 965 2024-05-08T08:53:59.063 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.063 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:59.063 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.063 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:59.064 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451075+0000", 2024-05-08T08:53:59.064 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:59.064 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.064 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.064 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:59.064 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451075+0000", 2024-05-08T08:53:59.064 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:59.064 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.064 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.064 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:59.064 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451076+0000", 2024-05-08T08:53:59.064 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5000000000000002e-07 2024-05-08T08:53:59.064 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.064 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.064 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:59.065 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451078+0000", 2024-05-08T08:53:59.065 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.5639999999999999e-06 2024-05-08T08:53:59.065 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.065 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.065 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:59.065 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451085+0000", 2024-05-08T08:53:59.065 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5640000000000002e-06 2024-05-08T08:53:59.065 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.065 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.065 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:59.065 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451090+0000", 2024-05-08T08:53:59.065 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1699999999999996e-06 2024-05-08T08:53:59.065 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.065 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.065 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:59.066 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538106+0000", 2024-05-08T08:53:59.066 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087015888999999999 2024-05-08T08:53:59.066 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.066 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.066 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:59.066 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538190+0000", 2024-05-08T08:53:59.066 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3961999999999993e-05 2024-05-08T08:53:59.066 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.066 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.066 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:59.066 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538305+0000", 2024-05-08T08:53:59.066 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011529800000000001 2024-05-08T08:53:59.066 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:59.067 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:59.067 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:59.067 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.067 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.067 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:966 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 1441792~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:59.067 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:53:59.068 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 49.588404234999999, 2024-05-08T08:53:59.068 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087446323000000006, 2024-05-08T08:53:59.068 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:59.068 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:59.068 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:59.068 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:59.068 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:59.069 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:59.069 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 966 2024-05-08T08:53:59.069 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.069 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:59.069 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.069 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:59.069 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:53:59.069 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:59.069 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.069 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.069 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:59.069 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:53:59.069 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:59.069 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.070 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.070 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:59.070 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:53:59.070 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3700000000000001e-07 2024-05-08T08:53:59.070 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.070 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.070 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:59.070 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451101+0000", 2024-05-08T08:53:59.070 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2560000000000001e-06 2024-05-08T08:53:59.070 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.070 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.070 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:59.070 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451107+0000", 2024-05-08T08:53:59.070 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1070000000000001e-06 2024-05-08T08:53:59.071 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.071 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.071 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:59.071 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451112+0000", 2024-05-08T08:53:59.071 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1420000000000001e-06 2024-05-08T08:53:59.071 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.071 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.071 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:59.071 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538341+0000", 2024-05-08T08:53:59.071 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087228302999999993 2024-05-08T08:53:59.071 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.071 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.071 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:59.071 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538424+0000", 2024-05-08T08:53:59.071 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3497000000000004e-05 2024-05-08T08:53:59.071 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.072 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.072 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:59.072 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538545+0000", 2024-05-08T08:53:59.072 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000120581 2024-05-08T08:53:59.072 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:59.072 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:59.072 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:59.072 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.072 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.072 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:967 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:59.072 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451120+0000", 2024-05-08T08:53:59.072 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 49.588382760000002, 2024-05-08T08:53:59.072 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087632725999999994, 2024-05-08T08:53:59.072 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:59.073 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:59.073 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:59.073 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:59.073 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:59.073 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:59.073 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 967 2024-05-08T08:53:59.073 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.073 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:59.074 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.074 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:59.074 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451120+0000", 2024-05-08T08:53:59.074 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:59.074 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.075 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.075 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:59.075 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451120+0000", 2024-05-08T08:53:59.075 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:59.075 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.075 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.075 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:59.075 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451121+0000", 2024-05-08T08:53:59.075 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8800000000000002e-07 2024-05-08T08:53:59.075 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.075 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.075 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:59.075 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451130+0000", 2024-05-08T08:53:59.076 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2299999999999997e-06 2024-05-08T08:53:59.076 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.076 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.076 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:59.076 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451137+0000", 2024-05-08T08:53:59.076 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4669999999999998e-06 2024-05-08T08:53:59.076 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.076 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.076 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:59.076 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451143+0000", 2024-05-08T08:53:59.076 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2839999999999997e-06 2024-05-08T08:53:59.076 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.076 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.077 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:59.077 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538575+0000", 2024-05-08T08:53:59.077 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087432789999999996 2024-05-08T08:53:59.077 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.077 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.077 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:59.077 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538648+0000", 2024-05-08T08:53:59.077 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2877e-05 2024-05-08T08:53:59.077 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.077 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.077 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:59.078 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538753+0000", 2024-05-08T08:53:59.078 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010449 2024-05-08T08:53:59.078 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:59.078 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:59.078 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:59.078 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.078 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.078 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:968 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 2424832~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:59.078 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451152+0000", 2024-05-08T08:53:59.078 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 49.588350675000001, 2024-05-08T08:53:59.078 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087813325999999997, 2024-05-08T08:53:59.078 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:59.078 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:59.078 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:59.079 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:59.079 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:59.079 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:59.079 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 968 2024-05-08T08:53:59.079 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.079 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:59.079 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.079 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:59.080 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451152+0000", 2024-05-08T08:53:59.080 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:59.080 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.080 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.080 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:59.080 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451152+0000", 2024-05-08T08:53:59.081 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:59.081 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.081 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.081 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:59.081 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451153+0000", 2024-05-08T08:53:59.082 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3099999999999998e-07 2024-05-08T08:53:59.082 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.082 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.082 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:59.082 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451155+0000", 2024-05-08T08:53:59.082 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4760000000000001e-06 2024-05-08T08:53:59.082 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.082 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.082 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:59.083 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451162+0000", 2024-05-08T08:53:59.083 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5969999999999997e-06 2024-05-08T08:53:59.083 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.083 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.083 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:59.083 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451167+0000", 2024-05-08T08:53:59.083 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.977e-06 2024-05-08T08:53:59.083 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.083 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.083 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:59.083 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538781+0000", 2024-05-08T08:53:59.083 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087614846999999996 2024-05-08T08:53:59.083 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.083 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.083 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:59.084 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538857+0000", 2024-05-08T08:53:59.084 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5358000000000006e-05 2024-05-08T08:53:59.084 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.084 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.084 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:59.084 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538965+0000", 2024-05-08T08:53:59.084 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010864 2024-05-08T08:53:59.084 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:59.084 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:59.084 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:59.084 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.084 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.084 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:969 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 2916352~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:59.084 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:53:59.085 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 49.588328969999999, 2024-05-08T08:53:59.085 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087998980000000004, 2024-05-08T08:53:59.085 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:59.085 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:59.085 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:59.085 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:59.085 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:59.085 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:59.085 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 969 2024-05-08T08:53:59.086 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.086 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:59.086 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.086 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:59.086 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:53:59.086 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:59.086 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.086 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.087 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:59.087 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:53:59.087 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:59.087 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.087 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.087 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:59.087 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:53:59.087 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3599999999999999e-07 2024-05-08T08:53:59.087 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.087 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.087 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:59.087 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451177+0000", 2024-05-08T08:53:59.087 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4590000000000001e-06 2024-05-08T08:53:59.087 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.088 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.088 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:59.088 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451184+0000", 2024-05-08T08:53:59.088 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4109999999999999e-06 2024-05-08T08:53:59.088 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.088 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.088 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:59.088 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451191+0000", 2024-05-08T08:53:59.088 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7279999999999998e-06 2024-05-08T08:53:59.088 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.089 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.089 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:59.089 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538997+0000", 2024-05-08T08:53:59.089 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087806170000000003 2024-05-08T08:53:59.089 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.089 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.089 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:59.089 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539069+0000", 2024-05-08T08:53:59.089 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2373999999999995e-05 2024-05-08T08:53:59.089 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.089 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.089 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:59.090 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539173+0000", 2024-05-08T08:53:59.090 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000103402 2024-05-08T08:53:59.090 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:59.090 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:59.090 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:59.090 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.090 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.090 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:970 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 3407872~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:59.090 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451198+0000", 2024-05-08T08:53:59.090 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 49.588304373, 2024-05-08T08:53:59.090 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088183368999999998, 2024-05-08T08:53:59.090 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:59.090 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:59.090 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:59.091 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:59.091 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:59.091 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:59.091 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 970 2024-05-08T08:53:59.091 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.091 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:59.091 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.091 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:59.091 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451198+0000", 2024-05-08T08:53:59.091 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:59.091 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.091 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.091 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:59.091 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451198+0000", 2024-05-08T08:53:59.092 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:59.092 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.092 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.092 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:59.092 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451199+0000", 2024-05-08T08:53:59.092 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7400000000000003e-07 2024-05-08T08:53:59.092 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.092 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.092 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:59.093 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451211+0000", 2024-05-08T08:53:59.093 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2109999999999999e-05 2024-05-08T08:53:59.093 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.093 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.093 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:59.093 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451219+0000", 2024-05-08T08:53:59.093 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4170000000000003e-06 2024-05-08T08:53:59.093 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.093 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.093 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:59.093 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451224+0000", 2024-05-08T08:53:59.093 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3990000000000003e-06 2024-05-08T08:53:59.093 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.093 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.093 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:59.094 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539204+0000", 2024-05-08T08:53:59.094 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087980099000000006 2024-05-08T08:53:59.094 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.094 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.094 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:59.094 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539276+0000", 2024-05-08T08:53:59.094 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1513000000000005e-05 2024-05-08T08:53:59.094 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.094 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.094 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:59.094 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539382+0000", 2024-05-08T08:53:59.095 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010615699999999999 2024-05-08T08:53:59.095 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:59.095 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:59.095 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:59.095 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.095 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.095 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:971 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 3899392~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:59.095 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451233+0000", 2024-05-08T08:53:59.095 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 49.588269666000002, 2024-05-08T08:53:59.095 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088431525999999996, 2024-05-08T08:53:59.095 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:59.095 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:59.095 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:59.095 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:59.096 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:59.096 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:59.096 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 971 2024-05-08T08:53:59.096 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.096 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:59.096 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.096 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:59.096 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451233+0000", 2024-05-08T08:53:59.096 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:59.096 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.096 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.096 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:59.096 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451233+0000", 2024-05-08T08:53:59.096 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:59.096 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.097 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.097 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:59.097 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451234+0000", 2024-05-08T08:53:59.097 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8500000000000001e-07 2024-05-08T08:53:59.097 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.097 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.097 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:59.097 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451237+0000", 2024-05-08T08:53:59.097 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8320000000000002e-06 2024-05-08T08:53:59.097 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.097 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.097 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:59.098 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451243+0000", 2024-05-08T08:53:59.098 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6749999999999996e-06 2024-05-08T08:53:59.098 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.098 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.098 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:59.098 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451249+0000", 2024-05-08T08:53:59.098 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1229999999999997e-06 2024-05-08T08:53:59.098 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.098 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.098 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:59.099 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539412+0000", 2024-05-08T08:53:59.099 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088162668999999999 2024-05-08T08:53:59.099 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.099 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.099 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:59.099 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539505+0000", 2024-05-08T08:53:59.099 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3240999999999997e-05 2024-05-08T08:53:59.099 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.099 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.099 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:59.099 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539665+0000", 2024-05-08T08:53:59.099 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00015930100000000001 2024-05-08T08:53:59.099 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:59.099 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:59.099 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:59.100 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.100 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.100 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:972 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 196608~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:59.100 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451257+0000", 2024-05-08T08:53:59.100 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 49.588245721, 2024-05-08T08:53:59.100 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088719928000000003, 2024-05-08T08:53:59.100 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:59.101 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:59.101 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:59.101 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:59.101 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:59.101 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:59.101 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 972 2024-05-08T08:53:59.101 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.101 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:59.101 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.101 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:59.101 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451257+0000", 2024-05-08T08:53:59.102 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:59.102 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.102 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.102 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:59.102 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451257+0000", 2024-05-08T08:53:59.102 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:59.102 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.102 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.102 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:59.102 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451258+0000", 2024-05-08T08:53:59.102 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7999999999999996e-07 2024-05-08T08:53:59.102 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.103 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.103 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:59.103 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451260+0000", 2024-05-08T08:53:59.103 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4609999999999998e-06 2024-05-08T08:53:59.103 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.104 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.104 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:59.104 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451267+0000", 2024-05-08T08:53:59.104 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.81e-06 2024-05-08T08:53:59.104 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.104 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.104 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:59.104 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451273+0000", 2024-05-08T08:53:59.104 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0859999999999998e-06 2024-05-08T08:53:59.104 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.104 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.104 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:59.104 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539718+0000", 2024-05-08T08:53:59.105 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088445059000000006 2024-05-08T08:53:59.105 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.105 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.105 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:59.105 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539834+0000", 2024-05-08T08:53:59.105 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000116426 2024-05-08T08:53:59.105 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.105 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.105 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:59.105 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539977+0000", 2024-05-08T08:53:59.105 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014260600000000001 2024-05-08T08:53:59.105 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:59.105 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:59.105 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:59.105 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.106 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.106 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:973 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 688128~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:59.106 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:53:59.106 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 49.588223137, 2024-05-08T08:53:59.106 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088918496, 2024-05-08T08:53:59.106 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:59.106 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:59.106 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:59.106 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:59.106 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:59.107 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:59.107 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 973 2024-05-08T08:53:59.107 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.107 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:59.107 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.107 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:59.107 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:53:59.107 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:59.107 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.107 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.107 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:59.107 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:53:59.107 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:59.107 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.107 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.108 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:59.108 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:53:59.108 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9099999999999999e-07 2024-05-08T08:53:59.108 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.108 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.108 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:59.108 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451282+0000", 2024-05-08T08:53:59.108 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1550000000000001e-06 2024-05-08T08:53:59.108 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.108 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.108 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:59.108 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451288+0000", 2024-05-08T08:53:59.108 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8889999999999999e-06 2024-05-08T08:53:59.108 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.109 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.109 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:59.109 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451294+0000", 2024-05-08T08:53:59.109 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0850000000000002e-06 2024-05-08T08:53:59.109 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.109 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.109 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:59.109 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540011+0000", 2024-05-08T08:53:59.109 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088717314000000005 2024-05-08T08:53:59.109 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.109 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.109 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:59.109 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540089+0000", 2024-05-08T08:53:59.109 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7438e-05 2024-05-08T08:53:59.109 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.109 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.110 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:59.110 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540198+0000", 2024-05-08T08:53:59.110 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000109224 2024-05-08T08:53:59.110 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:59.110 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:59.110 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:59.110 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.111 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.111 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:974 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 1179648~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:59.111 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451301+0000", 2024-05-08T08:53:59.111 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 49.588201503999997, 2024-05-08T08:53:59.111 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089105078000000004, 2024-05-08T08:53:59.112 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:59.112 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:59.112 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:59.112 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:59.112 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:59.112 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:59.112 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 974 2024-05-08T08:53:59.112 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.112 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:59.112 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.112 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:59.112 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451301+0000", 2024-05-08T08:53:59.113 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:59.113 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.113 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.113 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:59.113 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451301+0000", 2024-05-08T08:53:59.113 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:59.113 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.113 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.113 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:59.113 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451302+0000", 2024-05-08T08:53:59.113 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7800000000000002e-07 2024-05-08T08:53:59.114 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.114 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.114 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:59.114 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451304+0000", 2024-05-08T08:53:59.114 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.65e-06 2024-05-08T08:53:59.114 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.114 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.114 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:59.115 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451311+0000", 2024-05-08T08:53:59.115 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.472e-06 2024-05-08T08:53:59.115 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.115 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.115 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:59.115 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451316+0000", 2024-05-08T08:53:59.115 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9189999999999998e-06 2024-05-08T08:53:59.115 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.115 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.115 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:59.115 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540234+0000", 2024-05-08T08:53:59.115 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088917752000000003 2024-05-08T08:53:59.115 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.115 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.115 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:59.116 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540308+0000", 2024-05-08T08:53:59.116 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4758000000000005e-05 2024-05-08T08:53:59.116 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.116 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.116 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:59.116 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540406+0000", 2024-05-08T08:53:59.116 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8048999999999996e-05 2024-05-08T08:53:59.116 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:59.116 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:59.116 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:59.116 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.116 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.116 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:975 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 1671168~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:59.117 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:53:59.117 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 49.588159173999998, 2024-05-08T08:53:59.117 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089294139999999994, 2024-05-08T08:53:59.117 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:59.117 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:59.117 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:59.117 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:59.117 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:59.117 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:59.117 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 975 2024-05-08T08:53:59.117 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.117 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:59.117 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.117 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:59.118 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:53:59.118 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:59.118 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.118 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.118 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:59.118 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:53:59.118 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:59.118 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.118 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.118 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:59.118 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:53:59.118 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1299999999999999e-07 2024-05-08T08:53:59.118 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.119 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.119 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:59.119 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451348+0000", 2024-05-08T08:53:59.119 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.484e-06 2024-05-08T08:53:59.119 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.119 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.119 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:59.119 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451356+0000", 2024-05-08T08:53:59.119 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8499999999999994e-06 2024-05-08T08:53:59.119 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.119 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.119 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:59.119 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451366+0000", 2024-05-08T08:53:59.119 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.9850000000000001e-06 2024-05-08T08:53:59.119 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.120 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.120 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:59.120 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540448+0000", 2024-05-08T08:53:59.120 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089082254 2024-05-08T08:53:59.120 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.120 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.120 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:59.120 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540528+0000", 2024-05-08T08:53:59.120 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0234999999999993e-05 2024-05-08T08:53:59.120 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.120 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.120 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:59.120 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540638+0000", 2024-05-08T08:53:59.120 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010951900000000001 2024-05-08T08:53:59.120 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:59.121 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:59.121 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:59.121 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.121 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.121 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:976 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 2162688~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:59.121 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451375+0000", 2024-05-08T08:53:59.121 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 49.588127427000003, 2024-05-08T08:53:59.121 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089472745000000006, 2024-05-08T08:53:59.121 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:59.121 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:59.121 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:59.121 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:59.121 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:59.121 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:59.121 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 976 2024-05-08T08:53:59.122 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.122 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:59.122 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.122 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:59.122 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451375+0000", 2024-05-08T08:53:59.122 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:59.122 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.122 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.122 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:59.122 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451375+0000", 2024-05-08T08:53:59.122 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:59.123 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.123 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.123 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:59.123 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451376+0000", 2024-05-08T08:53:59.123 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2399999999999998e-07 2024-05-08T08:53:59.123 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.123 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.123 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:59.123 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451379+0000", 2024-05-08T08:53:59.123 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6359999999999998e-06 2024-05-08T08:53:59.123 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.123 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.123 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:59.124 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451387+0000", 2024-05-08T08:53:59.124 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9929999999999995e-06 2024-05-08T08:53:59.124 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.124 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.124 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:59.124 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451393+0000", 2024-05-08T08:53:59.124 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4699999999999999e-06 2024-05-08T08:53:59.124 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.124 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.124 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:59.124 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540677+0000", 2024-05-08T08:53:59.125 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089283504999999999 2024-05-08T08:53:59.125 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.125 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.125 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:59.125 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540743+0000", 2024-05-08T08:53:59.125 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6303999999999994e-05 2024-05-08T08:53:59.125 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.125 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.125 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:59.126 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540848+0000", 2024-05-08T08:53:59.126 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000105213 2024-05-08T08:53:59.126 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:59.126 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:59.126 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:59.126 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.126 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.126 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:977 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 2654208~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:59.126 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:53:59.126 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 49.588101995000002, 2024-05-08T08:53:59.126 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089698473000000001, 2024-05-08T08:53:59.126 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:59.126 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:59.126 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:59.127 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:59.127 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:59.127 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:59.127 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 977 2024-05-08T08:53:59.127 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.127 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:59.127 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.127 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:59.127 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:53:59.127 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:59.127 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.127 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.127 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:59.128 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:53:59.128 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:59.128 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.128 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.128 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:59.128 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:53:59.129 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2899999999999999e-07 2024-05-08T08:53:59.129 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.129 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.129 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:59.129 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451404+0000", 2024-05-08T08:53:59.129 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6649999999999999e-06 2024-05-08T08:53:59.129 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.129 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.129 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:59.129 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451410+0000", 2024-05-08T08:53:59.129 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3269999999999997e-06 2024-05-08T08:53:59.130 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.130 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.130 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:59.130 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451415+0000", 2024-05-08T08:53:59.130 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9529999999999998e-06 2024-05-08T08:53:59.130 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.130 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.130 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:59.130 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540912+0000", 2024-05-08T08:53:59.130 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089496950000000006 2024-05-08T08:53:59.130 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.130 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.131 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:59.131 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541000+0000", 2024-05-08T08:53:59.131 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7429000000000006e-05 2024-05-08T08:53:59.131 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.131 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.131 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:59.131 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541099+0000", 2024-05-08T08:53:59.131 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.9720000000000001e-05 2024-05-08T08:53:59.131 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:59.131 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:59.132 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:59.132 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.132 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.132 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:978 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 3145728~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:59.132 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:53:59.132 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 49.588080787000003, 2024-05-08T08:53:59.132 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089900635000000007, 2024-05-08T08:53:59.132 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:59.132 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:59.132 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:59.132 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:59.132 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:59.132 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:59.132 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 978 2024-05-08T08:53:59.133 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.133 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:59.133 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.133 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:59.133 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:53:59.133 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:59.133 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.133 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.133 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:59.133 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:53:59.133 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:59.133 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.133 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.133 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:59.133 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:53:59.134 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9400000000000001e-07 2024-05-08T08:53:59.134 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.134 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.134 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:59.134 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451425+0000", 2024-05-08T08:53:59.134 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2460000000000002e-06 2024-05-08T08:53:59.134 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.134 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.134 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:59.134 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451431+0000", 2024-05-08T08:53:59.135 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2419999999999997e-06 2024-05-08T08:53:59.135 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.135 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.135 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:59.135 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451436+0000", 2024-05-08T08:53:59.135 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.887e-06 2024-05-08T08:53:59.135 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.135 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.135 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:59.135 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541152+0000", 2024-05-08T08:53:59.135 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089716194999999999 2024-05-08T08:53:59.136 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.136 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.136 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:59.136 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541224+0000", 2024-05-08T08:53:59.136 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2416000000000006e-05 2024-05-08T08:53:59.136 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.136 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.136 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:59.136 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541323+0000", 2024-05-08T08:53:59.136 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8255e-05 2024-05-08T08:53:59.136 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:59.136 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:59.136 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:59.137 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.137 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.137 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:979 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 3637248~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:59.137 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451462+0000", 2024-05-08T08:53:59.137 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 49.588040337999999, 2024-05-08T08:53:59.137 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.090036441999999994, 2024-05-08T08:53:59.137 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:59.137 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:59.137 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:59.137 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:59.137 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:59.137 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:59.137 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 979 2024-05-08T08:53:59.137 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.138 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:59.138 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.138 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:59.138 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451462+0000", 2024-05-08T08:53:59.138 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:59.138 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.138 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.138 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:59.139 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451462+0000", 2024-05-08T08:53:59.139 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:59.139 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.139 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.139 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:59.139 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451463+0000", 2024-05-08T08:53:59.139 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4300000000000002e-07 2024-05-08T08:53:59.139 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.139 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.139 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:59.139 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451471+0000", 2024-05-08T08:53:59.139 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0800000000000006e-06 2024-05-08T08:53:59.139 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.139 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.140 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:59.140 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451481+0000", 2024-05-08T08:53:59.140 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.38e-06 2024-05-08T08:53:59.140 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.140 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.140 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:59.140 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451487+0000", 2024-05-08T08:53:59.140 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4590000000000003e-06 2024-05-08T08:53:59.140 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.140 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.140 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:59.140 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541350+0000", 2024-05-08T08:53:59.140 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089862599000000001 2024-05-08T08:53:59.140 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.140 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.141 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:59.141 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541407+0000", 2024-05-08T08:53:59.141 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7667999999999997e-05 2024-05-08T08:53:59.141 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.141 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.141 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:59.141 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541499+0000", 2024-05-08T08:53:59.141 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1513000000000003e-05 2024-05-08T08:53:59.141 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:59.141 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:59.141 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:59.141 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.141 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.141 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:980 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 4128768~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:53:59.142 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451496+0000", 2024-05-08T08:53:59.142 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 49.588006581999998, 2024-05-08T08:53:59.142 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.090258985, 2024-05-08T08:53:59.142 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:53:59.142 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:53:59.142 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:53:59.142 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:53:59.142 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:53:59.142 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:53:59.142 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 980 2024-05-08T08:53:59.142 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.142 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:53:59.142 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.142 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:53:59.142 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451496+0000", 2024-05-08T08:53:59.143 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:59.143 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.143 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.143 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:53:59.143 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451496+0000", 2024-05-08T08:53:59.143 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:53:59.143 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.143 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.143 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:53:59.143 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451497+0000", 2024-05-08T08:53:59.143 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7599999999999997e-07 2024-05-08T08:53:59.143 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.143 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.143 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:53:59.143 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451500+0000", 2024-05-08T08:53:59.144 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7930000000000002e-06 2024-05-08T08:53:59.144 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.144 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.144 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:53:59.144 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451506+0000", 2024-05-08T08:53:59.144 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6980000000000001e-06 2024-05-08T08:53:59.144 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.144 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.144 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:53:59.144 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451512+0000", 2024-05-08T08:53:59.144 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2490000000000001e-06 2024-05-08T08:53:59.144 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.145 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.145 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:53:59.145 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541550+0000", 2024-05-08T08:53:59.145 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.090037961999999999 2024-05-08T08:53:59.145 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.145 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.145 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:53:59.145 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541647+0000", 2024-05-08T08:53:59.146 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6924999999999998e-05 2024-05-08T08:53:59.146 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:53:59.146 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:53:59.146 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:53:59.146 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541755+0000", 2024-05-08T08:53:59.146 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000108682 2024-05-08T08:53:59.146 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:59.146 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:59.146 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:59.146 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:53:59.146 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:53:59.147 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:53:59.148 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:53:59.149 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:59.151+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:59.248 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:59.251+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:59.255 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:53:59.255 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:53:59.255 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.255 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4416.0:20531 3.16 3.68b74dd6 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-05-08T08:53:59.256 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:58.673786+0000", 2024-05-08T08:53:59.256 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.58535375899999997, 2024-05-08T08:53:59.256 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.58537501000000003, 2024-05-08T08:53:59.256 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:59.256 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:59.256 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "waiting for peered", 2024-05-08T08:53:59.256 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:59.256 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4416", 2024-05-08T08:53:59.256 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:53:59.256 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20531 2024-05-08T08:53:59.256 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.256 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:59.256 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.256 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:59.257 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673786+0000", 2024-05-08T08:53:59.257 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:59.257 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.257 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.257 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:59.257 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673786+0000", 2024-05-08T08:53:59.257 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:59.257 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.257 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.257 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:59.257 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673787+0000", 2024-05-08T08:53:59.257 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9800000000000003e-07 2024-05-08T08:53:59.257 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.257 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.257 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:59.257 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673791+0000", 2024-05-08T08:53:59.258 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.104e-06 2024-05-08T08:53:59.258 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.258 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.258 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:59.258 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673801+0000", 2024-05-08T08:53:59.258 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9960000000000005e-06 2024-05-08T08:53:59.258 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.258 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.258 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:59.258 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673806+0000", 2024-05-08T08:53:59.258 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2320000000000001e-06 2024-05-08T08:53:59.258 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.258 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.258 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:59.259 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673860+0000", 2024-05-08T08:53:59.259 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3468999999999999e-05 2024-05-08T08:53:59.259 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.259 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.259 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-05-08T08:53:59.259 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673866+0000", 2024-05-08T08:53:59.259 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3500000000000002e-06 2024-05-08T08:53:59.259 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:59.259 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:59.259 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:59.259 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:59.259 INFO:teuthology.orchestra.run.smithi192.stdout: ], 2024-05-08T08:53:59.259 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 1 2024-05-08T08:53:59.259 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:53:59.265 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:53:59.350 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:59.352+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:59.370 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:53:59.370 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:53:59.370 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:53:59.370 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:53:59.370 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:53:59.378 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:53:59.451 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:53:59.454+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:53:59.526 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:53:59.527 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:53:59.527 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:53:59.527 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:53:59.527 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.527 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4621.0:9089 3.1f e33/18 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:head v 33'117, mlcod=31'115)", 2024-05-08T08:53:59.527 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:53:59.527 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 87.009076444000002, 2024-05-08T08:53:59.527 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.111244524, 2024-05-08T08:53:59.527 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:59.527 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:59.527 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:53:59.527 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:59.528 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.528 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:59.528 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:53:59.528 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:59.528 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.528 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.528 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:59.528 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:53:59.528 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:59.528 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.528 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.528 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:59.528 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:53:59.528 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5900000000000001e-07 2024-05-08T08:53:59.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.529 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.529 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:59.529 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539600+0000", 2024-05-08T08:53:59.529 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.019981364000000001 2024-05-08T08:53:59.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.529 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.529 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:59.529 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539624+0000", 2024-05-08T08:53:59.529 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4043e-05 2024-05-08T08:53:59.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.529 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.529 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:59.529 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539635+0000", 2024-05-08T08:53:59.529 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1511e-05 2024-05-08T08:53:59.530 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.530 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.530 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:59.530 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539728+0000", 2024-05-08T08:53:59.530 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2498999999999996e-05 2024-05-08T08:53:59.530 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.530 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.530 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:59.530 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539746+0000", 2024-05-08T08:53:59.530 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8289000000000001e-05 2024-05-08T08:53:59.530 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.531 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.531 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:53:59.531 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.630822+0000", 2024-05-08T08:53:59.531 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.091076283999999993 2024-05-08T08:53:59.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.531 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.531 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:59.531 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.630863+0000", 2024-05-08T08:53:59.531 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0374999999999999e-05 2024-05-08T08:53:59.531 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:59.531 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:59.531 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:59.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.531 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.531 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4795.0:230 3.1f e40/18 3:f8db1a56:::rbd_data.11402b26385.0000000000000000:head v 40'171, mlcod=39'168)", 2024-05-08T08:53:59.532 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:45.927773+0000", 2024-05-08T08:53:59.532 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 73.600921361000005, 2024-05-08T08:53:59.532 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045892405999999997, 2024-05-08T08:53:59.532 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:59.532 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:59.532 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:53:59.532 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:59.532 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.532 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:59.532 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.927773+0000", 2024-05-08T08:53:59.532 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:59.532 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.532 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.532 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:59.532 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.927773+0000", 2024-05-08T08:53:59.533 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:59.533 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.533 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.533 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:59.533 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.927774+0000", 2024-05-08T08:53:59.533 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0500000000000002e-07 2024-05-08T08:53:59.533 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.533 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.533 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:59.533 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.931581+0000", 2024-05-08T08:53:59.533 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0038071799999999999 2024-05-08T08:53:59.533 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.533 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.533 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:59.533 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.931599+0000", 2024-05-08T08:53:59.534 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7479e-05 2024-05-08T08:53:59.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.534 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.534 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:59.534 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.931624+0000", 2024-05-08T08:53:59.534 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5372999999999999e-05 2024-05-08T08:53:59.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.534 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.534 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:59.534 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.931698+0000", 2024-05-08T08:53:59.534 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4043999999999998e-05 2024-05-08T08:53:59.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.534 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.535 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:59.535 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.931725+0000", 2024-05-08T08:53:59.535 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6883e-05 2024-05-08T08:53:59.535 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.535 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.535 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:53:59.535 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.973623+0000", 2024-05-08T08:53:59.535 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.041898283000000001 2024-05-08T08:53:59.535 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.535 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.535 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:59.535 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.973666+0000", 2024-05-08T08:53:59.535 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2358999999999999e-05 2024-05-08T08:53:59.535 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:59.535 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:59.535 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:59.536 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.536 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.536 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4195.0:4 3.11 3:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-05-08T08:53:59.536 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:49.764654+0000", 2024-05-08T08:53:59.536 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 69.764040902999994, 2024-05-08T08:53:59.536 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.30714275499999999, 2024-05-08T08:53:59.536 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:59.536 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:59.536 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-05-08T08:53:59.536 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:59.536 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4195", 2024-05-08T08:53:59.536 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/59179521", 2024-05-08T08:53:59.536 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4 2024-05-08T08:53:59.536 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.536 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:59.536 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.537 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:59.537 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764654+0000", 2024-05-08T08:53:59.537 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:59.537 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.537 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.537 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:59.537 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764654+0000", 2024-05-08T08:53:59.537 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:59.537 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.537 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.537 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:59.537 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764655+0000", 2024-05-08T08:53:59.537 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.452e-06 2024-05-08T08:53:59.537 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.537 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.538 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:59.538 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764661+0000", 2024-05-08T08:53:59.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5369999999999997e-06 2024-05-08T08:53:59.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.538 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:59.538 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764671+0000", 2024-05-08T08:53:59.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.028e-05 2024-05-08T08:53:59.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.538 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:59.538 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764680+0000", 2024-05-08T08:53:59.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0629999999999995e-06 2024-05-08T08:53:59.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.539 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:59.539 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764753+0000", 2024-05-08T08:53:59.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2679e-05 2024-05-08T08:53:59.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.539 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.539 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-05-08T08:53:59.539 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764765+0000", 2024-05-08T08:53:59.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2415000000000001e-05 2024-05-08T08:53:59.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.539 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.539 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:59.539 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:50.071692+0000", 2024-05-08T08:53:59.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.306927324 2024-05-08T08:53:59.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.540 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.540 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:59.540 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:50.071796+0000", 2024-05-08T08:53:59.540 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104005 2024-05-08T08:53:59.540 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:59.540 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:59.540 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:59.540 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.540 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.540 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22681 3.15 3:a907109d:::rbd_data.1140d3fc54c.000000000000004f:18 [read 1204224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:59.540 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.097653+0000", 2024-05-08T08:53:59.540 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 27.431041274999998, 2024-05-08T08:53:59.540 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.041941285000000002, 2024-05-08T08:53:59.540 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:59.540 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:59.540 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:59.541 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:59.541 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:59.541 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:59.541 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22681 2024-05-08T08:53:59.541 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.541 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:59.541 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.541 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:59.541 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097653+0000", 2024-05-08T08:53:59.541 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:59.541 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.541 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.541 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:59.541 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097653+0000", 2024-05-08T08:53:59.541 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:59.542 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.542 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.542 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:59.542 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097654+0000", 2024-05-08T08:53:59.542 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9400000000000001e-07 2024-05-08T08:53:59.542 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.542 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.542 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:59.542 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097655+0000", 2024-05-08T08:53:59.542 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6220000000000001e-06 2024-05-08T08:53:59.542 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.542 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.542 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:59.543 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097661+0000", 2024-05-08T08:53:59.543 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5210000000000002e-06 2024-05-08T08:53:59.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.543 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:59.543 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097665+0000", 2024-05-08T08:53:59.543 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3170000000000003e-06 2024-05-08T08:53:59.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.543 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:59.543 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139509+0000", 2024-05-08T08:53:59.543 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.041843475999999998 2024-05-08T08:53:59.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.544 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:59.544 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139554+0000", 2024-05-08T08:53:59.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5478000000000001e-05 2024-05-08T08:53:59.544 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.544 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.544 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:59.544 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139595+0000", 2024-05-08T08:53:59.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0577000000000001e-05 2024-05-08T08:53:59.544 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:59.544 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:59.544 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:59.544 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.544 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.544 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22682 3.15 3:a907109d:::rbd_data.1140d3fc54c.000000000000004f:18 [read 1269760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:59.545 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.097670+0000", 2024-05-08T08:53:59.545 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 27.431024582999999, 2024-05-08T08:53:59.545 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042031394, 2024-05-08T08:53:59.545 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:59.545 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:59.545 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:59.545 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:59.545 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:59.545 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:59.545 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22682 2024-05-08T08:53:59.545 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.545 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:59.545 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.545 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:59.545 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097670+0000", 2024-05-08T08:53:59.545 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:59.546 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.546 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.546 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:59.546 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097670+0000", 2024-05-08T08:53:59.546 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:59.546 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.546 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.546 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:59.546 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097670+0000", 2024-05-08T08:53:59.546 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1800000000000002e-07 2024-05-08T08:53:59.546 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.546 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.546 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:59.547 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097672+0000", 2024-05-08T08:53:59.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6929999999999999e-06 2024-05-08T08:53:59.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.547 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:59.547 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097677+0000", 2024-05-08T08:53:59.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2480000000000004e-06 2024-05-08T08:53:59.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.547 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:59.547 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097682+0000", 2024-05-08T08:53:59.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4290000000000003e-06 2024-05-08T08:53:59.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.547 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:59.547 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139620+0000", 2024-05-08T08:53:59.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.041938256 2024-05-08T08:53:59.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:59.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139662+0000", 2024-05-08T08:53:59.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1727999999999998e-05 2024-05-08T08:53:59.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:59.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139701+0000", 2024-05-08T08:53:59.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9721999999999997e-05 2024-05-08T08:53:59.548 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:59.548 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:59.548 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:59.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.549 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.549 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22683 3.15 3:a907109d:::rbd_data.1140d3fc54c.000000000000004f:18 [read 1335296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:59.549 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.097686+0000", 2024-05-08T08:53:59.549 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 27.431008229, 2024-05-08T08:53:59.549 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042114716000000003, 2024-05-08T08:53:59.549 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:59.549 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:59.549 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:59.549 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:59.549 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:59.549 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:59.549 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22683 2024-05-08T08:53:59.549 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.549 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:59.549 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.550 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:59.550 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097686+0000", 2024-05-08T08:53:59.550 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:59.550 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.550 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.550 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:59.550 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097686+0000", 2024-05-08T08:53:59.550 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:59.550 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.550 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.550 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:59.550 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097687+0000", 2024-05-08T08:53:59.550 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3999999999999997e-07 2024-05-08T08:53:59.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.551 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:59.551 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097688+0000", 2024-05-08T08:53:59.551 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6789999999999999e-06 2024-05-08T08:53:59.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.551 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:59.551 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097693+0000", 2024-05-08T08:53:59.551 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.045e-06 2024-05-08T08:53:59.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.551 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:59.551 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097697+0000", 2024-05-08T08:53:59.552 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0260000000000001e-06 2024-05-08T08:53:59.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.552 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.552 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:59.552 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139725+0000", 2024-05-08T08:53:59.552 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042027737000000003 2024-05-08T08:53:59.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.552 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.552 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:59.552 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139764+0000", 2024-05-08T08:53:59.552 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8668999999999999e-05 2024-05-08T08:53:59.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.552 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.552 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:59.552 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139801+0000", 2024-05-08T08:53:59.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7219999999999999e-05 2024-05-08T08:53:59.553 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:59.553 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:59.553 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:59.553 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.553 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.553 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22684 3.15 3:a907109d:::rbd_data.1140d3fc54c.000000000000004f:18 [read 1400832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:59.553 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.097703+0000", 2024-05-08T08:53:59.553 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 27.430991876, 2024-05-08T08:53:59.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042198929000000003, 2024-05-08T08:53:59.553 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:59.553 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:59.553 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:59.553 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:59.553 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:59.553 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:59.554 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22684 2024-05-08T08:53:59.554 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.554 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:59.554 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.554 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:59.554 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097703+0000", 2024-05-08T08:53:59.554 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:59.554 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.554 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.554 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:59.554 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097703+0000", 2024-05-08T08:53:59.554 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:59.554 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.554 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.554 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:59.555 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097703+0000", 2024-05-08T08:53:59.555 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4400000000000001e-07 2024-05-08T08:53:59.555 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.555 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.555 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:59.555 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097705+0000", 2024-05-08T08:53:59.555 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9199999999999998e-06 2024-05-08T08:53:59.555 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.555 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.555 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:59.555 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097711+0000", 2024-05-08T08:53:59.555 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5559999999999998e-06 2024-05-08T08:53:59.555 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.555 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.555 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:59.555 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097715+0000", 2024-05-08T08:53:59.555 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3159999999999998e-06 2024-05-08T08:53:59.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.556 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.556 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:59.556 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139825+0000", 2024-05-08T08:53:59.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042110284999999997 2024-05-08T08:53:59.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.556 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.556 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:59.556 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139864+0000", 2024-05-08T08:53:59.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8980000000000003e-05 2024-05-08T08:53:59.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.556 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.556 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:59.556 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139902+0000", 2024-05-08T08:53:59.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7527999999999997e-05 2024-05-08T08:53:59.556 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:59.557 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:59.557 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:59.557 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.557 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.557 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22685 3.15 3:a907109d:::rbd_data.1140d3fc54c.000000000000004f:18 [read 1466368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:59.557 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.097721+0000", 2024-05-08T08:53:59.557 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 27.430973218999998, 2024-05-08T08:53:59.557 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042287791999999998, 2024-05-08T08:53:59.557 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:59.557 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:59.557 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:59.557 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:59.557 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:59.557 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:59.557 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22685 2024-05-08T08:53:59.558 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.558 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:59.558 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.558 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:59.558 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097721+0000", 2024-05-08T08:53:59.558 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:59.558 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.558 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.558 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:59.558 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097721+0000", 2024-05-08T08:53:59.558 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:59.558 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.558 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.558 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:59.559 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097722+0000", 2024-05-08T08:53:59.559 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2500000000000001e-07 2024-05-08T08:53:59.559 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.559 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.559 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:59.559 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097724+0000", 2024-05-08T08:53:59.559 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.835e-06 2024-05-08T08:53:59.559 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.559 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.559 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:59.559 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097729+0000", 2024-05-08T08:53:59.559 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5029999999999997e-06 2024-05-08T08:53:59.559 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.559 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.559 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:59.559 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097733+0000", 2024-05-08T08:53:59.560 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3309999999999997e-06 2024-05-08T08:53:59.560 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.560 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.560 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:59.560 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139923+0000", 2024-05-08T08:53:59.560 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042189988999999997 2024-05-08T08:53:59.560 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.560 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.560 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:59.560 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139970+0000", 2024-05-08T08:53:59.560 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6776000000000001e-05 2024-05-08T08:53:59.560 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.560 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.560 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:59.561 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140009+0000", 2024-05-08T08:53:59.561 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9032999999999997e-05 2024-05-08T08:53:59.561 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:59.561 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:59.561 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:59.561 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.561 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.561 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22686 3.15 3:a907109d:::rbd_data.1140d3fc54c.000000000000004f:18 [read 1531904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:59.561 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.097739+0000", 2024-05-08T08:53:59.561 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 27.430955220000001, 2024-05-08T08:53:59.561 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042371008000000002, 2024-05-08T08:53:59.561 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:59.561 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:59.561 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:59.561 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:59.562 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:59.562 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:59.562 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22686 2024-05-08T08:53:59.562 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.562 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:59.562 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.562 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:59.562 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097739+0000", 2024-05-08T08:53:59.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:59.562 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.562 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.562 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:59.562 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097739+0000", 2024-05-08T08:53:59.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:59.562 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.563 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.563 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:59.563 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097740+0000", 2024-05-08T08:53:59.563 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8599999999999999e-07 2024-05-08T08:53:59.563 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.563 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.563 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:59.563 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097742+0000", 2024-05-08T08:53:59.563 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9410000000000001e-06 2024-05-08T08:53:59.563 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.563 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.563 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:59.564 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097747+0000", 2024-05-08T08:53:59.564 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2079999999999999e-06 2024-05-08T08:53:59.564 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.564 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.564 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:59.564 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097751+0000", 2024-05-08T08:53:59.564 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2250000000000002e-06 2024-05-08T08:53:59.564 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.564 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.564 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:59.564 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140032+0000", 2024-05-08T08:53:59.565 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042281244000000003 2024-05-08T08:53:59.565 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.565 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.565 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:59.565 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140074+0000", 2024-05-08T08:53:59.565 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1316999999999998e-05 2024-05-08T08:53:59.565 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.565 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.565 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:59.565 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140110+0000", 2024-05-08T08:53:59.565 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6687000000000003e-05 2024-05-08T08:53:59.565 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:59.565 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:59.565 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:59.566 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.566 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.566 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22716 3.15 3:a88fdee9:::rbd_data.1140d3fc54c.000000000000004d:18 [read 1097728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:59.566 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.098153+0000", 2024-05-08T08:53:59.566 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 27.430541513000001, 2024-05-08T08:53:59.566 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042058535000000001, 2024-05-08T08:53:59.566 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:59.566 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:59.566 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:59.566 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:59.566 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:59.566 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:59.566 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22716 2024-05-08T08:53:59.567 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.567 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:59.567 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.567 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:59.567 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098153+0000", 2024-05-08T08:53:59.567 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:59.567 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.567 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.567 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:59.568 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098153+0000", 2024-05-08T08:53:59.568 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:59.568 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.568 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.568 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:59.568 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098154+0000", 2024-05-08T08:53:59.568 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.13e-07 2024-05-08T08:53:59.568 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.568 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.568 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:59.568 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098161+0000", 2024-05-08T08:53:59.568 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7360000000000001e-06 2024-05-08T08:53:59.569 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.569 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.569 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:59.569 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098171+0000", 2024-05-08T08:53:59.569 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1039999999999996e-06 2024-05-08T08:53:59.569 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.569 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.569 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:59.569 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098177+0000", 2024-05-08T08:53:59.569 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5760000000000002e-06 2024-05-08T08:53:59.569 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.569 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.569 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:59.569 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140135+0000", 2024-05-08T08:53:59.570 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.041957842000000002 2024-05-08T08:53:59.570 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.570 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.570 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:59.570 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140174+0000", 2024-05-08T08:53:59.570 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8902999999999997e-05 2024-05-08T08:53:59.570 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.570 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.570 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:59.570 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140212+0000", 2024-05-08T08:53:59.570 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7761e-05 2024-05-08T08:53:59.570 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:59.570 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:59.570 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:59.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.571 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.571 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22717 3.15 3:a88fdee9:::rbd_data.1140d3fc54c.000000000000004d:18 [read 1163264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:59.571 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.098188+0000", 2024-05-08T08:53:59.571 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 27.430506790999999, 2024-05-08T08:53:59.571 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042124907000000003, 2024-05-08T08:53:59.571 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:59.571 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:59.571 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:59.571 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:59.571 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:59.571 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:59.571 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22717 2024-05-08T08:53:59.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.571 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:59.572 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.572 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:59.572 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098188+0000", 2024-05-08T08:53:59.572 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:59.572 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.572 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.572 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:59.572 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098188+0000", 2024-05-08T08:53:59.572 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:59.572 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.572 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.572 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:59.572 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098188+0000", 2024-05-08T08:53:59.572 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3799999999999998e-07 2024-05-08T08:53:59.572 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.573 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.573 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:59.573 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098190+0000", 2024-05-08T08:53:59.573 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.114e-06 2024-05-08T08:53:59.573 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.573 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.573 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:59.573 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098197+0000", 2024-05-08T08:53:59.573 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9959999999999996e-06 2024-05-08T08:53:59.573 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.573 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.573 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:59.573 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098202+0000", 2024-05-08T08:53:59.573 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9110000000000001e-06 2024-05-08T08:53:59.573 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.574 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.574 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:59.574 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140235+0000", 2024-05-08T08:53:59.574 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042032913999999998 2024-05-08T08:53:59.574 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.574 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.574 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:59.574 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140275+0000", 2024-05-08T08:53:59.574 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9564999999999998e-05 2024-05-08T08:53:59.574 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.574 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.574 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:59.574 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140313+0000", 2024-05-08T08:53:59.575 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7969000000000002e-05 2024-05-08T08:53:59.575 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:59.575 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:59.575 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:59.575 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.575 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.575 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22718 3.15 3:a88fdee9:::rbd_data.1140d3fc54c.000000000000004d:18 [read 1228800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:59.575 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.098209+0000", 2024-05-08T08:53:59.575 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 27.430485645000001, 2024-05-08T08:53:59.575 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.04220409, 2024-05-08T08:53:59.575 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:59.575 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:59.575 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:59.575 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:59.575 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:59.576 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:59.576 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22718 2024-05-08T08:53:59.576 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.576 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:59.576 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.576 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:59.576 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098209+0000", 2024-05-08T08:53:59.576 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:59.576 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.576 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.576 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:59.576 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098209+0000", 2024-05-08T08:53:59.576 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:59.576 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.577 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.577 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:59.577 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098209+0000", 2024-05-08T08:53:59.577 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9299999999999999e-07 2024-05-08T08:53:59.577 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.577 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.577 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:59.577 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098212+0000", 2024-05-08T08:53:59.577 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.21e-06 2024-05-08T08:53:59.577 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.577 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.578 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:59.578 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098219+0000", 2024-05-08T08:53:59.578 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0269999999999998e-06 2024-05-08T08:53:59.578 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.578 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.578 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:59.578 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098223+0000", 2024-05-08T08:53:59.578 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7319999999999998e-06 2024-05-08T08:53:59.578 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.578 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.578 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:59.578 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140338+0000", 2024-05-08T08:53:59.578 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042114390000000002 2024-05-08T08:53:59.578 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.579 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.579 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:59.579 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140376+0000", 2024-05-08T08:53:59.579 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8414999999999997e-05 2024-05-08T08:53:59.579 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.579 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.579 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:59.579 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140413+0000", 2024-05-08T08:53:59.579 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6922999999999999e-05 2024-05-08T08:53:59.579 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:59.579 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:59.579 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:59.579 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.579 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.579 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22719 3.15 3:a88fdee9:::rbd_data.1140d3fc54c.000000000000004d:18 [read 1294336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:59.580 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.098229+0000", 2024-05-08T08:53:59.580 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 27.430465794, 2024-05-08T08:53:59.580 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042283016, 2024-05-08T08:53:59.580 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:59.580 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:59.580 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:59.580 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:59.580 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:59.580 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:59.580 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22719 2024-05-08T08:53:59.580 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.580 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:59.580 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.580 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:59.580 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098229+0000", 2024-05-08T08:53:59.581 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:59.581 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.581 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.581 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:59.581 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098229+0000", 2024-05-08T08:53:59.581 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:59.581 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.581 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.581 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:59.581 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098229+0000", 2024-05-08T08:53:59.581 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:53:59.581 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.581 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.581 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:59.582 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098231+0000", 2024-05-08T08:53:59.582 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.81e-06 2024-05-08T08:53:59.582 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.582 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.582 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:59.582 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098237+0000", 2024-05-08T08:53:59.582 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3389999999999998e-06 2024-05-08T08:53:59.582 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.582 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.582 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:59.582 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098243+0000", 2024-05-08T08:53:59.582 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.648e-06 2024-05-08T08:53:59.582 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.583 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.583 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:59.583 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140436+0000", 2024-05-08T08:53:59.583 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042192738 2024-05-08T08:53:59.583 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.583 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.583 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:59.583 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140475+0000", 2024-05-08T08:53:59.583 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9279999999999997e-05 2024-05-08T08:53:59.583 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.583 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.583 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:59.583 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140512+0000", 2024-05-08T08:53:59.583 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6847000000000001e-05 2024-05-08T08:53:59.583 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:59.584 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:59.584 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:59.584 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.584 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.584 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22720 3.15 3:a88fdee9:::rbd_data.1140d3fc54c.000000000000004d:18 [read 1359872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:59.584 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.098248+0000", 2024-05-08T08:53:59.584 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 27.430446282999998, 2024-05-08T08:53:59.584 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042360800999999997, 2024-05-08T08:53:59.584 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:59.584 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:59.584 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:59.584 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:59.584 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:59.584 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:59.584 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22720 2024-05-08T08:53:59.585 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.585 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:59.585 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.585 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:59.585 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098248+0000", 2024-05-08T08:53:59.585 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:59.585 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.585 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.585 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:59.585 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098248+0000", 2024-05-08T08:53:59.585 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:59.585 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.585 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.585 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:59.585 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098491+0000", 2024-05-08T08:53:59.586 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000242659 2024-05-08T08:53:59.586 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.586 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.586 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:59.586 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098495+0000", 2024-05-08T08:53:59.586 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.799e-06 2024-05-08T08:53:59.586 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.586 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.586 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:59.586 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098505+0000", 2024-05-08T08:53:59.586 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7079999999999992e-06 2024-05-08T08:53:59.586 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.586 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.586 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:59.587 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098511+0000", 2024-05-08T08:53:59.587 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4370000000000004e-06 2024-05-08T08:53:59.587 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.587 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.587 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:59.587 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140533+0000", 2024-05-08T08:53:59.587 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042022371000000003 2024-05-08T08:53:59.587 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.587 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.587 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:59.587 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140571+0000", 2024-05-08T08:53:59.587 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7700000000000002e-05 2024-05-08T08:53:59.587 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.587 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.587 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:59.588 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140609+0000", 2024-05-08T08:53:59.588 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8127000000000003e-05 2024-05-08T08:53:59.588 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:59.588 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:59.588 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:59.588 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.588 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.588 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22721 3.15 3:a88fdee9:::rbd_data.1140d3fc54c.000000000000004d:18 [read 1425408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:59.588 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.098522+0000", 2024-05-08T08:53:59.588 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 27.430172659, 2024-05-08T08:53:59.588 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042185779999999999, 2024-05-08T08:53:59.588 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:59.588 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:59.588 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:59.588 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:59.589 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:59.589 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:59.589 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22721 2024-05-08T08:53:59.589 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.589 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:59.589 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.589 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:59.589 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098522+0000", 2024-05-08T08:53:59.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:59.589 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.589 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.589 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:59.589 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098522+0000", 2024-05-08T08:53:59.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:59.589 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.590 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.590 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:59.590 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098708+0000", 2024-05-08T08:53:59.590 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00018629700000000001 2024-05-08T08:53:59.590 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.590 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.590 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:59.590 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098719+0000", 2024-05-08T08:53:59.590 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0992000000000001e-05 2024-05-08T08:53:59.590 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.590 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.590 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:59.590 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098730+0000", 2024-05-08T08:53:59.590 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0431999999999999e-05 2024-05-08T08:53:59.591 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.591 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.591 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:59.591 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098737+0000", 2024-05-08T08:53:59.591 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.063e-06 2024-05-08T08:53:59.591 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.591 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.591 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:59.591 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140631+0000", 2024-05-08T08:53:59.591 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.041894503999999999 2024-05-08T08:53:59.591 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.591 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.591 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:59.591 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140671+0000", 2024-05-08T08:53:59.591 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9780999999999997e-05 2024-05-08T08:53:59.592 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.592 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.592 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:59.592 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140708+0000", 2024-05-08T08:53:59.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6711000000000001e-05 2024-05-08T08:53:59.592 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:59.592 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:59.592 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:59.592 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.592 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.592 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22722 3.15 3:a88fdee9:::rbd_data.1140d3fc54c.000000000000004d:18 [read 1490944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:59.592 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.098753+0000", 2024-05-08T08:53:59.592 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 27.429941655, 2024-05-08T08:53:59.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042053944000000003, 2024-05-08T08:53:59.592 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:59.592 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:59.593 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:59.593 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:59.593 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:59.593 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:59.593 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22722 2024-05-08T08:53:59.593 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.593 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:59.593 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.593 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:59.593 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098753+0000", 2024-05-08T08:53:59.593 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:59.594 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.594 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.594 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:59.594 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098753+0000", 2024-05-08T08:53:59.594 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:59.594 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.594 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.594 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:59.594 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098753+0000", 2024-05-08T08:53:59.595 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1399999999999997e-07 2024-05-08T08:53:59.595 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.595 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.595 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:59.595 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098756+0000", 2024-05-08T08:53:59.595 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9780000000000001e-06 2024-05-08T08:53:59.595 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.595 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.595 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:59.595 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098764+0000", 2024-05-08T08:53:59.595 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4340000000000003e-06 2024-05-08T08:53:59.595 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.595 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.596 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:59.596 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098770+0000", 2024-05-08T08:53:59.596 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8080000000000001e-06 2024-05-08T08:53:59.596 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.596 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:59.596 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140731+0000", 2024-05-08T08:53:59.596 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.041961131999999998 2024-05-08T08:53:59.596 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.596 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:59.596 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140769+0000", 2024-05-08T08:53:59.596 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.773e-05 2024-05-08T08:53:59.596 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.596 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:59.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140807+0000", 2024-05-08T08:53:59.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8347999999999999e-05 2024-05-08T08:53:59.597 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:59.597 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:59.597 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:59.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.597 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22723 3.15 3:a88fdee9:::rbd_data.1140d3fc54c.000000000000004d:18 [read 1556480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:59.597 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.098778+0000", 2024-05-08T08:53:59.597 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 27.429916883000001, 2024-05-08T08:53:59.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042128577, 2024-05-08T08:53:59.597 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:59.597 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:59.598 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:59.598 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:59.598 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:59.598 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:59.598 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22723 2024-05-08T08:53:59.598 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.598 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:59.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:59.598 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098778+0000", 2024-05-08T08:53:59.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:59.599 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.599 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.599 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:59.599 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098778+0000", 2024-05-08T08:53:59.599 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:59.599 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.599 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.599 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:59.599 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098834+0000", 2024-05-08T08:53:59.599 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5788000000000002e-05 2024-05-08T08:53:59.599 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.600 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.600 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:59.600 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098837+0000", 2024-05-08T08:53:59.600 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3510000000000002e-06 2024-05-08T08:53:59.600 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.600 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.600 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:59.600 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098845+0000", 2024-05-08T08:53:59.600 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5490000000000006e-06 2024-05-08T08:53:59.600 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.600 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.600 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:59.600 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098861+0000", 2024-05-08T08:53:59.601 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5233e-05 2024-05-08T08:53:59.601 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.601 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:59.601 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140829+0000", 2024-05-08T08:53:59.601 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.04196863 2024-05-08T08:53:59.601 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.601 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:59.601 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140869+0000", 2024-05-08T08:53:59.601 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9461e-05 2024-05-08T08:53:59.601 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.601 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:59.601 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140906+0000", 2024-05-08T08:53:59.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7564999999999997e-05 2024-05-08T08:53:59.602 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:59.602 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:59.602 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:59.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.602 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22732 3.15 3:a907109d:::rbd_data.1140d3fc54c.000000000000004f:18 [read 1097728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:59.602 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.098871+0000", 2024-05-08T08:53:59.602 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 27.429823403, 2024-05-08T08:53:59.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042141517000000003, 2024-05-08T08:53:59.602 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:59.602 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:59.602 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:59.602 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:59.602 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:59.603 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:59.603 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22732 2024-05-08T08:53:59.603 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.603 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:59.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.603 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:59.603 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098871+0000", 2024-05-08T08:53:59.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:59.603 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.603 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:59.603 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098871+0000", 2024-05-08T08:53:59.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:59.603 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.603 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:59.603 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.099940+0000", 2024-05-08T08:53:59.604 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0010683279999999999 2024-05-08T08:53:59.604 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.604 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.604 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:59.604 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.099944+0000", 2024-05-08T08:53:59.604 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9339999999999997e-06 2024-05-08T08:53:59.604 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.604 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.604 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:59.604 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.099956+0000", 2024-05-08T08:53:59.604 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1970999999999999e-05 2024-05-08T08:53:59.604 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.604 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.604 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:59.604 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.099968+0000", 2024-05-08T08:53:59.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.189e-05 2024-05-08T08:53:59.605 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.605 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.605 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:59.605 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140930+0000", 2024-05-08T08:53:59.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.040961806000000003 2024-05-08T08:53:59.605 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.605 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.605 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:59.605 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140974+0000", 2024-05-08T08:53:59.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3918999999999997e-05 2024-05-08T08:53:59.605 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.605 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.605 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:59.605 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.141013+0000", 2024-05-08T08:53:59.606 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8668999999999999e-05 2024-05-08T08:53:59.606 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:59.606 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:59.606 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:59.606 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.606 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22765 3.15 3:a9c637ae:::rbd_data.114071bcf828.000000000000013f:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:59.606 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.100902+0000", 2024-05-08T08:53:59.606 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 27.427792608000001, 2024-05-08T08:53:59.606 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042471271999999997, 2024-05-08T08:53:59.606 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:59.606 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:59.606 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:53:59.607 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:59.607 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:59.607 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:59.607 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22765 2024-05-08T08:53:59.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.607 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:59.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.607 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:59.607 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.100902+0000", 2024-05-08T08:53:59.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:59.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.607 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:59.607 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.100902+0000", 2024-05-08T08:53:59.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:59.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.608 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:59.608 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.100902+0000", 2024-05-08T08:53:59.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.39e-07 2024-05-08T08:53:59.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.608 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:59.608 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.100904+0000", 2024-05-08T08:53:59.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6479999999999999e-06 2024-05-08T08:53:59.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.608 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:59.608 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.100910+0000", 2024-05-08T08:53:59.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1399999999999997e-06 2024-05-08T08:53:59.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.609 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:59.609 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.100916+0000", 2024-05-08T08:53:59.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6999999999999996e-06 2024-05-08T08:53:59.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.609 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:59.609 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.141733+0000", 2024-05-08T08:53:59.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.040817271000000002 2024-05-08T08:53:59.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.609 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:59.609 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.141812+0000", 2024-05-08T08:53:59.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9270999999999994e-05 2024-05-08T08:53:59.610 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.610 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:59.610 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.143373+0000", 2024-05-08T08:53:59.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001560903 2024-05-08T08:53:59.610 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:59.610 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:59.610 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:59.610 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.610 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:23031 3.15 3:ac9fb9cd:::rbd_data.114071bcf828.00000000000000f7:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:53:59.610 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.113562+0000", 2024-05-08T08:53:59.610 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 27.415132847999999, 2024-05-08T08:53:59.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042560938999999999, 2024-05-08T08:53:59.611 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:53:59.611 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:53:59.611 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:53:59.611 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:53:59.611 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:53:59.611 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:53:59.611 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 23031 2024-05-08T08:53:59.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.611 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:53:59.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.611 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:53:59.611 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.113562+0000", 2024-05-08T08:53:59.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:59.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.612 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:53:59.612 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.113562+0000", 2024-05-08T08:53:59.612 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:53:59.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.612 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:53:59.612 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.113938+0000", 2024-05-08T08:53:59.612 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000376401 2024-05-08T08:53:59.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.612 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:53:59.612 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.113943+0000", 2024-05-08T08:53:59.612 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0529999999999997e-06 2024-05-08T08:53:59.613 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.613 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.613 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:53:59.613 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.113951+0000", 2024-05-08T08:53:59.613 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8569999999999995e-06 2024-05-08T08:53:59.613 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.613 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.613 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:53:59.613 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.113959+0000", 2024-05-08T08:53:59.613 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4710000000000001e-06 2024-05-08T08:53:59.613 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.613 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.613 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:53:59.613 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.153955+0000", 2024-05-08T08:53:59.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.039996747999999999 2024-05-08T08:53:59.614 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.614 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.614 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:53:59.614 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.154014+0000", 2024-05-08T08:53:59.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8464999999999997e-05 2024-05-08T08:53:59.614 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.614 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.614 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-05-08T08:53:59.614 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.154148+0000", 2024-05-08T08:53:59.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00013428600000000001 2024-05-08T08:53:59.614 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.614 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.614 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:53:59.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.155390+0000", 2024-05-08T08:53:59.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001241792 2024-05-08T08:53:59.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:53:59.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.156068+0000", 2024-05-08T08:53:59.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00067823700000000005 2024-05-08T08:53:59.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:53:59.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.156102+0000", 2024-05-08T08:53:59.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3812000000000001e-05 2024-05-08T08:53:59.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:53:59.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:53:59.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:53:59.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.156123+0000", 2024-05-08T08:53:59.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0817000000000001e-05 2024-05-08T08:53:59.616 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:59.616 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:59.616 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:59.616 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:53:59.616 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:53:59.616 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:53:59.617 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4] out_osds: [5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-05-08T08:53:59.617 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T08:53:59.617 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-05-08T08:53:59.617 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-05-08T08:53:59.619 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:53:59.619 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:53:59.620+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:53:59.622 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-05-08T08:53:59.644 INFO:teuthology.orchestra.run.smithi196.stdout:truncating image image_client.0-clone33 from 0x8ebee00 (overlap 0x5298400) to 0x0 2024-05-08T08:53:59.718 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:53:59.720+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:53:59.768 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:53:59.768 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:53:59.768 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:53:59.768 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:53:59.777 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:53:59.796 INFO:teuthology.orchestra.run.smithi059.stderr:no valid command found; 10 closest matches: 2024-05-08T08:53:59.796 INFO:teuthology.orchestra.run.smithi059.stderr:0 2024-05-08T08:53:59.796 INFO:teuthology.orchestra.run.smithi059.stderr:1 2024-05-08T08:53:59.796 INFO:teuthology.orchestra.run.smithi059.stderr:2 2024-05-08T08:53:59.796 INFO:teuthology.orchestra.run.smithi059.stderr:abort 2024-05-08T08:53:59.796 INFO:teuthology.orchestra.run.smithi059.stderr:assert 2024-05-08T08:53:59.796 INFO:teuthology.orchestra.run.smithi059.stderr:bench [] [] [] [] 2024-05-08T08:53:59.796 INFO:teuthology.orchestra.run.smithi059.stderr:bluefs debug_inject_read_zeros 2024-05-08T08:53:59.796 INFO:teuthology.orchestra.run.smithi059.stderr:bluefs files list 2024-05-08T08:53:59.796 INFO:teuthology.orchestra.run.smithi059.stderr:bluefs stats 2024-05-08T08:53:59.796 INFO:teuthology.orchestra.run.smithi059.stderr:bluestore allocator dump block 2024-05-08T08:53:59.797 INFO:teuthology.orchestra.run.smithi059.stderr:Error EINVAL: invalid command 2024-05-08T08:53:59.799 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-08T08:53:59.799 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-05-08T08:53:59.800 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-05-08T08:53:59.846 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:53:59.849+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:53:59.920 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:53:59.923+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:53:59.927 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:53:59.927 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:53:59.927 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:53:59.927 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:53:59.927 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:53:59.936 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:53:59.942 INFO:teuthology.orchestra.run.smithi059.stderr:no valid command found; 10 closest matches: 2024-05-08T08:53:59.943 INFO:teuthology.orchestra.run.smithi059.stderr:0 2024-05-08T08:53:59.943 INFO:teuthology.orchestra.run.smithi059.stderr:1 2024-05-08T08:53:59.943 INFO:teuthology.orchestra.run.smithi059.stderr:2 2024-05-08T08:53:59.943 INFO:teuthology.orchestra.run.smithi059.stderr:abort 2024-05-08T08:53:59.943 INFO:teuthology.orchestra.run.smithi059.stderr:assert 2024-05-08T08:53:59.943 INFO:teuthology.orchestra.run.smithi059.stderr:bench [] [] [] [] 2024-05-08T08:53:59.943 INFO:teuthology.orchestra.run.smithi059.stderr:bluefs debug_inject_read_zeros 2024-05-08T08:53:59.943 INFO:teuthology.orchestra.run.smithi059.stderr:bluefs files list 2024-05-08T08:53:59.943 INFO:teuthology.orchestra.run.smithi059.stderr:bluefs stats 2024-05-08T08:53:59.943 INFO:teuthology.orchestra.run.smithi059.stderr:bluestore allocator dump block 2024-05-08T08:53:59.943 INFO:teuthology.orchestra.run.smithi059.stderr:Error EINVAL: invalid command 2024-05-08T08:53:59.945 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-08T08:53:59.946 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-05-08T08:53:59.947 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-05-08T08:54:00.031 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:00.034+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:00.089 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:00.089 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:54:00.089 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:54:00.089 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:54:00.089 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.090 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22071 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 765952~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:00.090 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857643+0000", 2024-05-08T08:54:00.090 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 92.233661651999995, 2024-05-08T08:54:00.090 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047106528000000002, 2024-05-08T08:54:00.090 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:00.090 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:00.090 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:00.090 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:00.090 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:00.090 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:00.090 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22071 2024-05-08T08:54:00.090 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.090 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:00.091 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.091 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:00.091 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857643+0000", 2024-05-08T08:54:00.091 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:00.091 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.091 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.091 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:00.091 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857643+0000", 2024-05-08T08:54:00.091 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:00.091 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.091 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.091 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:00.091 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857644+0000", 2024-05-08T08:54:00.092 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2000000000000001e-07 2024-05-08T08:54:00.092 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.092 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.092 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:00.092 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857646+0000", 2024-05-08T08:54:00.092 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0059999999999999e-06 2024-05-08T08:54:00.092 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.092 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.092 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:00.092 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857651+0000", 2024-05-08T08:54:00.092 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.84e-06 2024-05-08T08:54:00.092 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.092 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.092 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:00.093 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857657+0000", 2024-05-08T08:54:00.093 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5620000000000003e-06 2024-05-08T08:54:00.093 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.093 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.093 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:00.093 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904620+0000", 2024-05-08T08:54:00.093 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046962661000000003 2024-05-08T08:54:00.093 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.093 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.093 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:00.093 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904678+0000", 2024-05-08T08:54:00.093 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8656999999999998e-05 2024-05-08T08:54:00.093 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.093 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.094 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:00.094 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904750+0000", 2024-05-08T08:54:00.094 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1482000000000004e-05 2024-05-08T08:54:00.094 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.094 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:00.094 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.094 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.094 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.094 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22073 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 782336~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:00.094 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:54:00.094 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 92.233637643999998, 2024-05-08T08:54:00.094 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047244234000000003, 2024-05-08T08:54:00.094 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:00.094 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:00.095 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:00.095 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:00.095 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:00.095 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:00.095 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22073 2024-05-08T08:54:00.095 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.095 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:00.095 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.095 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:00.095 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:54:00.095 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:00.095 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.095 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.095 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:00.096 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:54:00.096 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:00.096 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.096 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.096 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:00.096 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857668+0000", 2024-05-08T08:54:00.096 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7800000000000002e-07 2024-05-08T08:54:00.096 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.096 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.096 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:00.096 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857670+0000", 2024-05-08T08:54:00.096 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0820000000000001e-06 2024-05-08T08:54:00.096 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.096 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.097 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:00.097 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857678+0000", 2024-05-08T08:54:00.097 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7589999999999997e-06 2024-05-08T08:54:00.097 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.097 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.097 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:00.097 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857691+0000", 2024-05-08T08:54:00.097 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3672999999999999e-05 2024-05-08T08:54:00.097 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.097 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.097 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:00.097 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904780+0000", 2024-05-08T08:54:00.097 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047088616999999999 2024-05-08T08:54:00.097 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.097 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.098 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:00.098 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904840+0000", 2024-05-08T08:54:00.098 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0356999999999998e-05 2024-05-08T08:54:00.098 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.098 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.098 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:00.098 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904912+0000", 2024-05-08T08:54:00.098 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1367999999999998e-05 2024-05-08T08:54:00.098 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.098 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:00.098 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.098 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.098 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.098 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22075 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 798720~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:00.099 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:54:00.099 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 92.233607246999995, 2024-05-08T08:54:00.099 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047384643999999997, 2024-05-08T08:54:00.099 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:00.099 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:00.099 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:00.099 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:00.099 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:00.099 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:00.099 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22075 2024-05-08T08:54:00.099 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.099 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:00.099 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.099 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:00.099 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:54:00.100 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:00.100 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.100 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.100 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:00.100 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:54:00.100 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:00.100 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.100 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.100 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:00.100 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:54:00.100 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7399999999999999e-07 2024-05-08T08:54:00.100 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.100 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.100 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:00.101 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857705+0000", 2024-05-08T08:54:00.101 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1980000000000004e-06 2024-05-08T08:54:00.101 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.101 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.101 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:00.101 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857713+0000", 2024-05-08T08:54:00.101 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7039999999999994e-06 2024-05-08T08:54:00.101 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.101 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.101 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:00.101 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857718+0000", 2024-05-08T08:54:00.101 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.865e-06 2024-05-08T08:54:00.101 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.101 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.102 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:00.102 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904941+0000", 2024-05-08T08:54:00.102 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047223198000000001 2024-05-08T08:54:00.102 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.102 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.102 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:00.102 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904999+0000", 2024-05-08T08:54:00.102 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8254999999999997e-05 2024-05-08T08:54:00.102 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.102 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.102 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:00.102 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905082+0000", 2024-05-08T08:54:00.102 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3049999999999999e-05 2024-05-08T08:54:00.103 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.103 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:00.103 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.103 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.103 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.103 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22077 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 815104~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:00.103 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:54:00.103 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 92.233569235999994, 2024-05-08T08:54:00.103 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047519448999999998, 2024-05-08T08:54:00.103 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:00.103 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:00.103 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:00.103 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:00.103 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:00.104 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:00.104 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22077 2024-05-08T08:54:00.104 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.104 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:00.104 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.104 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:00.104 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:54:00.104 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:00.104 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.104 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.104 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:00.104 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:54:00.104 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:00.104 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.104 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.105 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:00.105 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:54:00.105 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.99e-07 2024-05-08T08:54:00.105 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.105 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.105 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:00.105 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857739+0000", 2024-05-08T08:54:00.105 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3599999999999999e-06 2024-05-08T08:54:00.105 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.105 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.105 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:00.105 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857745+0000", 2024-05-08T08:54:00.105 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5139999999999998e-06 2024-05-08T08:54:00.106 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.106 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.106 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:00.106 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857750+0000", 2024-05-08T08:54:00.106 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8550000000000001e-06 2024-05-08T08:54:00.106 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.106 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.106 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:00.106 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905110+0000", 2024-05-08T08:54:00.106 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047360126000000002 2024-05-08T08:54:00.106 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.106 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.106 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:00.107 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905175+0000", 2024-05-08T08:54:00.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4689000000000004e-05 2024-05-08T08:54:00.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.107 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:00.107 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905255+0000", 2024-05-08T08:54:00.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0306e-05 2024-05-08T08:54:00.107 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.107 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:00.107 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.107 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22079 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 831488~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:00.107 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:54:00.107 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 92.233548687999999, 2024-05-08T08:54:00.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.04767499, 2024-05-08T08:54:00.108 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:00.108 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:00.108 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:00.108 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:00.108 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:00.108 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:00.108 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22079 2024-05-08T08:54:00.108 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.108 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:00.108 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.108 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:00.108 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:54:00.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:00.108 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.108 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.109 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:00.109 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:54:00.109 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:00.109 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.109 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.109 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:00.109 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857757+0000", 2024-05-08T08:54:00.109 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8299999999999997e-07 2024-05-08T08:54:00.109 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.109 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.109 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:00.109 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857759+0000", 2024-05-08T08:54:00.109 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1390000000000002e-06 2024-05-08T08:54:00.110 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.110 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.110 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:00.110 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857765+0000", 2024-05-08T08:54:00.110 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4609999999999996e-06 2024-05-08T08:54:00.110 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.110 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.110 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:00.110 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857770+0000", 2024-05-08T08:54:00.110 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6689999999999997e-06 2024-05-08T08:54:00.110 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.110 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.110 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:00.111 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905281+0000", 2024-05-08T08:54:00.111 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047510668999999998 2024-05-08T08:54:00.111 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.111 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.111 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:00.111 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905344+0000", 2024-05-08T08:54:00.111 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2797000000000002e-05 2024-05-08T08:54:00.111 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.111 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.111 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:00.111 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905431+0000", 2024-05-08T08:54:00.111 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7771999999999999e-05 2024-05-08T08:54:00.111 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.112 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:00.112 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.112 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.112 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.112 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22082 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 847872~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:00.112 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:54:00.112 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 92.233528996999993, 2024-05-08T08:54:00.112 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047831934999999999, 2024-05-08T08:54:00.112 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:00.112 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:00.112 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:00.112 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:00.112 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:00.112 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:00.113 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22082 2024-05-08T08:54:00.113 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.113 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:00.113 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.113 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:00.113 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:54:00.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:00.113 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.113 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.113 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:00.113 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:54:00.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:00.113 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.113 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.114 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:00.114 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:54:00.114 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:54:00.114 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.114 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.114 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:00.114 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857778+0000", 2024-05-08T08:54:00.114 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0789999999999999e-06 2024-05-08T08:54:00.114 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.114 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.114 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:00.114 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857784+0000", 2024-05-08T08:54:00.114 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0480000000000004e-06 2024-05-08T08:54:00.114 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.115 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.115 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:00.115 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857790+0000", 2024-05-08T08:54:00.115 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2719999999999997e-06 2024-05-08T08:54:00.115 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.115 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.115 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:00.115 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905458+0000", 2024-05-08T08:54:00.115 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047668470999999997 2024-05-08T08:54:00.115 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.115 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.115 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:00.115 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905521+0000", 2024-05-08T08:54:00.115 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2952000000000003e-05 2024-05-08T08:54:00.116 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.116 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.116 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:00.116 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905608+0000", 2024-05-08T08:54:00.116 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6759000000000001e-05 2024-05-08T08:54:00.116 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.116 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:00.116 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.116 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.116 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.116 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22084 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 864256~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:00.116 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:54:00.116 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 92.233505046999994, 2024-05-08T08:54:00.116 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047977748000000001, 2024-05-08T08:54:00.117 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:00.117 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:00.117 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:00.117 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:00.117 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:00.117 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:00.117 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22084 2024-05-08T08:54:00.117 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.117 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:00.117 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.117 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:00.117 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:54:00.117 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:00.117 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.118 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.118 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:00.118 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:54:00.118 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:00.118 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.118 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.118 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:00.118 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:54:00.118 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:54:00.118 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.118 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.118 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:00.118 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857802+0000", 2024-05-08T08:54:00.118 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0779999999999998e-06 2024-05-08T08:54:00.119 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.119 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.119 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:00.119 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857812+0000", 2024-05-08T08:54:00.119 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7939999999999997e-06 2024-05-08T08:54:00.119 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.119 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.119 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:00.119 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857817+0000", 2024-05-08T08:54:00.119 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8899999999999998e-06 2024-05-08T08:54:00.119 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.119 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.119 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:00.119 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905636+0000", 2024-05-08T08:54:00.120 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047819252 2024-05-08T08:54:00.120 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.120 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.120 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:00.120 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905700+0000", 2024-05-08T08:54:00.120 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4016999999999994e-05 2024-05-08T08:54:00.120 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.120 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.120 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:00.120 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905778+0000", 2024-05-08T08:54:00.120 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7306999999999997e-05 2024-05-08T08:54:00.120 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.120 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:00.120 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.121 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.121 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.121 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22086 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 880640~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:00.121 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:54:00.121 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 92.233481697000002, 2024-05-08T08:54:00.121 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048119149, 2024-05-08T08:54:00.121 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:00.121 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:00.121 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:00.121 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:00.121 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:00.121 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:00.121 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22086 2024-05-08T08:54:00.121 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.122 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:00.122 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.122 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:00.122 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:54:00.122 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:00.122 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.122 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.122 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:00.122 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:54:00.122 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:00.122 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.122 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.122 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:00.122 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857824+0000", 2024-05-08T08:54:00.123 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4200000000000002e-07 2024-05-08T08:54:00.123 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.123 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.123 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:00.123 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857826+0000", 2024-05-08T08:54:00.123 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0590000000000001e-06 2024-05-08T08:54:00.123 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.123 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.123 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:00.123 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857832+0000", 2024-05-08T08:54:00.123 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8029999999999999e-06 2024-05-08T08:54:00.123 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.123 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.123 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:00.124 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857837+0000", 2024-05-08T08:54:00.124 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7929999999999997e-06 2024-05-08T08:54:00.124 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.124 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.124 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:00.124 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905803+0000", 2024-05-08T08:54:00.124 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047965761000000003 2024-05-08T08:54:00.124 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.124 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.124 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:00.124 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905862+0000", 2024-05-08T08:54:00.124 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9203000000000002e-05 2024-05-08T08:54:00.124 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.124 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.125 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:00.125 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905942+0000", 2024-05-08T08:54:00.125 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0187999999999999e-05 2024-05-08T08:54:00.125 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.125 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:00.125 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.125 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.125 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.125 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22088 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 897024~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:00.125 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:54:00.125 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 92.233461645000006, 2024-05-08T08:54:00.125 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048256049000000002, 2024-05-08T08:54:00.125 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:00.126 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:00.126 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:00.126 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:00.126 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:00.126 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:00.126 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22088 2024-05-08T08:54:00.126 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.126 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:00.126 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.126 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:00.126 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:54:00.126 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:00.126 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.127 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.127 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:00.127 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:54:00.127 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:00.127 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.127 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.127 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:00.127 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857844+0000", 2024-05-08T08:54:00.127 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.46e-07 2024-05-08T08:54:00.127 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.127 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.127 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:00.127 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857846+0000", 2024-05-08T08:54:00.127 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0899999999999999e-06 2024-05-08T08:54:00.127 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.128 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.128 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:00.128 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857855+0000", 2024-05-08T08:54:00.128 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8219999999999996e-06 2024-05-08T08:54:00.128 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.128 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.128 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:00.128 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857860+0000", 2024-05-08T08:54:00.128 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2989999999999996e-06 2024-05-08T08:54:00.128 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.128 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.128 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:00.128 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905966+0000", 2024-05-08T08:54:00.128 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048106542000000002 2024-05-08T08:54:00.129 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.129 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.129 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:00.129 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906024+0000", 2024-05-08T08:54:00.129 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7958000000000003e-05 2024-05-08T08:54:00.129 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.129 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.129 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:00.129 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906099+0000", 2024-05-08T08:54:00.129 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4992000000000003e-05 2024-05-08T08:54:00.129 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.129 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:00.129 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.129 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.130 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.130 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22091 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 913408~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:00.130 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:54:00.130 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 92.233437885000001, 2024-05-08T08:54:00.130 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048395290000000001, 2024-05-08T08:54:00.130 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:00.130 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:00.130 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:00.130 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:00.130 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:00.130 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:00.130 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22091 2024-05-08T08:54:00.130 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.130 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:00.131 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.131 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:00.131 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:54:00.131 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:00.131 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.131 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.131 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:00.131 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:54:00.131 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:00.131 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.131 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.131 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:00.131 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:54:00.132 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4900000000000001e-07 2024-05-08T08:54:00.132 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.132 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.132 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:00.132 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857870+0000", 2024-05-08T08:54:00.132 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2510000000000001e-06 2024-05-08T08:54:00.132 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.132 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.132 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:00.132 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857877+0000", 2024-05-08T08:54:00.132 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0809999999999996e-06 2024-05-08T08:54:00.132 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.132 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.132 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:00.132 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857881+0000", 2024-05-08T08:54:00.133 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.724e-06 2024-05-08T08:54:00.133 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.133 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.133 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:00.133 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906127+0000", 2024-05-08T08:54:00.133 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048245345000000002 2024-05-08T08:54:00.133 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.133 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.133 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:00.133 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906190+0000", 2024-05-08T08:54:00.133 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2762000000000007e-05 2024-05-08T08:54:00.133 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.133 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.133 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:00.133 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906262+0000", 2024-05-08T08:54:00.134 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2778e-05 2024-05-08T08:54:00.134 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.134 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:00.134 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.134 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.134 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.134 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22093 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 929792~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:00.134 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:54:00.134 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 92.233415520999998, 2024-05-08T08:54:00.134 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048532875000000003, 2024-05-08T08:54:00.134 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:00.134 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:00.134 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:00.135 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:00.135 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:00.135 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:00.135 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22093 2024-05-08T08:54:00.135 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.135 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:00.135 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.135 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:00.135 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:54:00.135 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:00.135 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.135 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.135 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:00.135 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:54:00.136 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:00.136 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.136 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.136 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:00.136 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857890+0000", 2024-05-08T08:54:00.136 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-07 2024-05-08T08:54:00.136 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.136 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.136 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:00.136 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857899+0000", 2024-05-08T08:54:00.136 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6889999999999999e-06 2024-05-08T08:54:00.136 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.136 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.136 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:00.137 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857906+0000", 2024-05-08T08:54:00.137 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6490000000000002e-06 2024-05-08T08:54:00.137 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.137 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.137 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:00.137 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857917+0000", 2024-05-08T08:54:00.137 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0764999999999999e-05 2024-05-08T08:54:00.137 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.137 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.137 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:00.137 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906287+0000", 2024-05-08T08:54:00.137 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048370442 2024-05-08T08:54:00.137 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.137 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.137 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:00.138 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906349+0000", 2024-05-08T08:54:00.138 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1537000000000004e-05 2024-05-08T08:54:00.138 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.138 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.138 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:00.138 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906422+0000", 2024-05-08T08:54:00.138 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3431999999999998e-05 2024-05-08T08:54:00.138 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.138 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:00.138 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.138 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.138 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.138 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22095 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 946176~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:00.139 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:54:00.139 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 92.233380816999997, 2024-05-08T08:54:00.139 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048657593999999998, 2024-05-08T08:54:00.139 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:00.139 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:00.139 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:00.139 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:00.139 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:00.139 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:00.139 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22095 2024-05-08T08:54:00.139 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.139 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:00.139 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.139 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:00.139 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:54:00.140 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:00.140 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.140 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.140 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:00.140 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:54:00.140 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:00.140 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.140 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.140 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:00.140 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857925+0000", 2024-05-08T08:54:00.140 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3099999999999998e-07 2024-05-08T08:54:00.140 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.140 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.140 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:00.141 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857927+0000", 2024-05-08T08:54:00.141 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.34e-06 2024-05-08T08:54:00.141 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.141 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.141 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:00.141 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857938+0000", 2024-05-08T08:54:00.141 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1426e-05 2024-05-08T08:54:00.141 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.141 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.141 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:00.141 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857943+0000", 2024-05-08T08:54:00.141 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5140000000000003e-06 2024-05-08T08:54:00.141 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.141 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.142 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:00.142 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906448+0000", 2024-05-08T08:54:00.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048504641000000001 2024-05-08T08:54:00.142 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.142 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.142 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:00.142 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906510+0000", 2024-05-08T08:54:00.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2663000000000006e-05 2024-05-08T08:54:00.142 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.142 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.142 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:00.142 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906582+0000", 2024-05-08T08:54:00.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1478999999999998e-05 2024-05-08T08:54:00.142 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.143 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:00.143 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.143 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.143 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.143 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22097 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 962560~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:00.143 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:54:00.143 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 92.233353128000005, 2024-05-08T08:54:00.143 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048791353000000003, 2024-05-08T08:54:00.143 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:00.143 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:00.143 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:00.143 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:00.143 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:00.143 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:00.144 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22097 2024-05-08T08:54:00.144 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.144 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:00.144 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.144 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:00.144 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:54:00.144 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:00.144 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.144 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.144 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:00.144 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:54:00.144 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:00.144 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.144 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.144 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:00.145 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:54:00.145 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9200000000000002e-07 2024-05-08T08:54:00.145 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.145 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.145 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:00.145 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857954+0000", 2024-05-08T08:54:00.145 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1900000000000002e-06 2024-05-08T08:54:00.145 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.145 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.145 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:00.145 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857961+0000", 2024-05-08T08:54:00.145 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4989999999999999e-06 2024-05-08T08:54:00.145 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.145 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.145 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:00.146 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857971+0000", 2024-05-08T08:54:00.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0485999999999999e-05 2024-05-08T08:54:00.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.146 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.146 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:00.146 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906607+0000", 2024-05-08T08:54:00.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048635109000000003 2024-05-08T08:54:00.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.146 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.146 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:00.146 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906667+0000", 2024-05-08T08:54:00.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0439000000000002e-05 2024-05-08T08:54:00.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.147 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.147 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:00.147 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906743+0000", 2024-05-08T08:54:00.147 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6237999999999998e-05 2024-05-08T08:54:00.147 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.147 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:00.147 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.147 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.147 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.147 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22099 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 978944~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:00.147 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:54:00.147 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 92.233326968, 2024-05-08T08:54:00.147 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048932055000000002, 2024-05-08T08:54:00.147 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:00.147 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:00.148 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:00.148 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:00.148 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:00.148 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:00.148 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22099 2024-05-08T08:54:00.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.148 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:00.148 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.148 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:00.148 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:54:00.148 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:00.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.148 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.148 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:00.149 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:54:00.149 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:00.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.149 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:00.149 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:54:00.149 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.27e-07 2024-05-08T08:54:00.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.149 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:00.149 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857987+0000", 2024-05-08T08:54:00.149 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5890000000000003e-06 2024-05-08T08:54:00.150 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.150 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.150 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:00.150 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857994+0000", 2024-05-08T08:54:00.150 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6619999999999999e-06 2024-05-08T08:54:00.150 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.150 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.150 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:00.150 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857998+0000", 2024-05-08T08:54:00.150 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7110000000000003e-06 2024-05-08T08:54:00.150 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.150 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.150 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:00.150 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906772+0000", 2024-05-08T08:54:00.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048773415000000001 2024-05-08T08:54:00.151 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.151 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.151 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:00.151 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906834+0000", 2024-05-08T08:54:00.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1752000000000001e-05 2024-05-08T08:54:00.151 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.151 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.151 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:00.151 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906910+0000", 2024-05-08T08:54:00.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6499000000000001e-05 2024-05-08T08:54:00.151 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.151 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:00.151 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.151 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.151 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.152 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22101 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 995328~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:00.152 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:54:00.152 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 92.233299837000004, 2024-05-08T08:54:00.152 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049087364000000001, 2024-05-08T08:54:00.152 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:00.152 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:00.152 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:00.152 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:00.152 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:00.152 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:00.152 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22101 2024-05-08T08:54:00.152 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.152 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:00.152 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.152 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:00.153 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:54:00.153 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:00.153 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.153 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.153 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:00.153 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:54:00.153 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:00.153 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.153 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.153 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:00.153 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858006+0000", 2024-05-08T08:54:00.153 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1399999999999997e-07 2024-05-08T08:54:00.153 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.153 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.153 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:00.153 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858008+0000", 2024-05-08T08:54:00.153 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2060000000000001e-06 2024-05-08T08:54:00.154 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.154 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.154 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:00.154 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858017+0000", 2024-05-08T08:54:00.154 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1579999999999994e-06 2024-05-08T08:54:00.154 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.154 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.154 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:00.154 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858022+0000", 2024-05-08T08:54:00.154 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4199999999999998e-06 2024-05-08T08:54:00.154 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.154 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.154 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:00.154 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906940+0000", 2024-05-08T08:54:00.155 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048917627999999998 2024-05-08T08:54:00.155 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.155 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.155 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:00.155 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907002+0000", 2024-05-08T08:54:00.155 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2399999999999999e-05 2024-05-08T08:54:00.155 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.155 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.155 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:00.155 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907092+0000", 2024-05-08T08:54:00.155 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0137999999999997e-05 2024-05-08T08:54:00.155 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.155 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:00.155 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.156 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.156 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.156 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22103 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1011712~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:00.156 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:54:00.156 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 92.233275051999996, 2024-05-08T08:54:00.156 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049228542, 2024-05-08T08:54:00.156 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:00.156 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:00.156 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:00.156 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:00.156 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:00.156 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:00.156 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22103 2024-05-08T08:54:00.156 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.156 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:00.156 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.157 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:00.157 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:54:00.157 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:00.157 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.157 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.157 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:00.157 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:54:00.157 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:00.157 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.157 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.157 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:00.157 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:54:00.157 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4299999999999999e-07 2024-05-08T08:54:00.157 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.158 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.158 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:00.158 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858032+0000", 2024-05-08T08:54:00.158 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0590000000000001e-06 2024-05-08T08:54:00.158 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.158 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.158 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:00.158 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858037+0000", 2024-05-08T08:54:00.158 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.977e-06 2024-05-08T08:54:00.158 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.158 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.158 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:00.159 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858042+0000", 2024-05-08T08:54:00.159 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7650000000000001e-06 2024-05-08T08:54:00.159 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.159 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.159 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:00.159 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907118+0000", 2024-05-08T08:54:00.159 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049076147000000001 2024-05-08T08:54:00.159 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.159 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.159 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:00.159 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907180+0000", 2024-05-08T08:54:00.159 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1971999999999996e-05 2024-05-08T08:54:00.159 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.160 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.160 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:00.160 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907258+0000", 2024-05-08T08:54:00.160 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8279e-05 2024-05-08T08:54:00.160 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.160 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:00.160 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.160 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.160 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.160 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22105 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1028096~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:00.160 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:54:00.160 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 92.233255808999999, 2024-05-08T08:54:00.160 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049394609999999999, 2024-05-08T08:54:00.160 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:00.161 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:00.161 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:00.161 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:00.161 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:00.161 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:00.161 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22105 2024-05-08T08:54:00.161 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.161 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:00.161 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.161 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:00.161 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:54:00.161 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:00.161 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.161 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.161 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:00.162 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:54:00.162 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:00.162 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.162 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.162 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:00.162 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858050+0000", 2024-05-08T08:54:00.162 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5999999999999999e-07 2024-05-08T08:54:00.162 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.162 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.162 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:00.162 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858051+0000", 2024-05-08T08:54:00.162 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8870000000000001e-06 2024-05-08T08:54:00.162 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.163 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.163 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:00.163 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858057+0000", 2024-05-08T08:54:00.163 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8019999999999997e-06 2024-05-08T08:54:00.163 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.163 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.163 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:00.163 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858062+0000", 2024-05-08T08:54:00.163 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.561e-06 2024-05-08T08:54:00.163 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.163 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.163 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:00.163 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907289+0000", 2024-05-08T08:54:00.163 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049226957000000002 2024-05-08T08:54:00.163 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.163 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.164 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:00.164 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907356+0000", 2024-05-08T08:54:00.164 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6915000000000006e-05 2024-05-08T08:54:00.164 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.164 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.164 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:00.164 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907444+0000", 2024-05-08T08:54:00.164 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8127999999999994e-05 2024-05-08T08:54:00.164 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.164 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:00.164 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.164 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.164 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.164 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22107 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1044480~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:00.165 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:54:00.165 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 92.233237469000002, 2024-05-08T08:54:00.165 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049799942999999999, 2024-05-08T08:54:00.165 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:00.165 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:00.165 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:00.165 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:00.165 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:00.165 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:00.165 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22107 2024-05-08T08:54:00.165 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.165 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:00.165 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.166 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:00.166 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:54:00.166 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:00.166 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.166 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.166 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:00.166 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:54:00.166 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:00.166 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.166 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.166 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:00.166 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:54:00.166 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4799999999999999e-07 2024-05-08T08:54:00.167 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.167 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.167 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:00.167 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858070+0000", 2024-05-08T08:54:00.167 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.012e-06 2024-05-08T08:54:00.167 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.167 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.167 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:00.167 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858077+0000", 2024-05-08T08:54:00.167 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6290000000000001e-06 2024-05-08T08:54:00.167 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.167 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.167 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:00.168 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858083+0000", 2024-05-08T08:54:00.168 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8880000000000002e-06 2024-05-08T08:54:00.168 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.168 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.168 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:00.168 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907671+0000", 2024-05-08T08:54:00.168 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049587191000000003 2024-05-08T08:54:00.168 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.168 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.168 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:00.168 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907763+0000", 2024-05-08T08:54:00.168 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2231000000000005e-05 2024-05-08T08:54:00.168 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.168 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.168 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:00.169 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907867+0000", 2024-05-08T08:54:00.169 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104644 2024-05-08T08:54:00.169 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.169 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:00.169 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.169 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.169 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.169 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22294 3.1c 3:3aad0389:::rbd_data.11406886b07b.0000000000000080:head [stat,delete] snapc 7=[7] RETRY=6 ondisk+retry+write+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:00.169 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:54:00.169 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 92.232643620999994, 2024-05-08T08:54:00.169 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051152191, 2024-05-08T08:54:00.169 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:00.169 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:00.169 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:00.170 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:00.170 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:00.170 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:00.170 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22294 2024-05-08T08:54:00.170 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.170 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:00.170 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.170 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:00.170 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:54:00.170 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:00.170 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.170 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.170 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:00.170 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:54:00.171 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:00.171 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.171 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.171 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:00.171 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858662+0000", 2024-05-08T08:54:00.171 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4799999999999998e-07 2024-05-08T08:54:00.171 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.171 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.171 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:00.171 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858664+0000", 2024-05-08T08:54:00.171 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1229999999999998e-06 2024-05-08T08:54:00.171 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.171 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.171 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:00.171 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858670+0000", 2024-05-08T08:54:00.172 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3099999999999997e-06 2024-05-08T08:54:00.172 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.172 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.172 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:00.172 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858676+0000", 2024-05-08T08:54:00.172 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9750000000000004e-06 2024-05-08T08:54:00.172 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.172 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.172 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:00.172 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907901+0000", 2024-05-08T08:54:00.172 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049224520000000001 2024-05-08T08:54:00.172 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.172 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.172 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:00.172 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.908085+0000", 2024-05-08T08:54:00.173 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00018377500000000001 2024-05-08T08:54:00.173 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.173 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.173 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:00.173 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.909814+0000", 2024-05-08T08:54:00.173 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00172894 2024-05-08T08:54:00.173 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.173 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:00.173 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.173 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.173 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.173 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4416.0:13461 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-05-08T08:54:00.173 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:22.487456+0000", 2024-05-08T08:54:00.173 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.603848579999998, 2024-05-08T08:54:00.173 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.99102620799999996, 2024-05-08T08:54:00.174 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:00.174 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:00.174 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:00.174 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:00.174 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4416", 2024-05-08T08:54:00.174 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:54:00.174 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 13461 2024-05-08T08:54:00.174 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.174 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:00.174 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.174 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:00.174 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487456+0000", 2024-05-08T08:54:00.174 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:00.174 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.174 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.175 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:00.175 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487456+0000", 2024-05-08T08:54:00.175 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:00.175 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.175 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.175 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:00.175 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487458+0000", 2024-05-08T08:54:00.175 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5659999999999999e-06 2024-05-08T08:54:00.175 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.175 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.175 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:00.175 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487463+0000", 2024-05-08T08:54:00.175 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6890000000000003e-06 2024-05-08T08:54:00.175 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.175 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.176 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:00.176 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487472+0000", 2024-05-08T08:54:00.176 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7710000000000001e-06 2024-05-08T08:54:00.176 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.176 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.176 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:00.176 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487478+0000", 2024-05-08T08:54:00.176 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1390000000000003e-06 2024-05-08T08:54:00.176 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.176 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.176 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:00.176 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487520+0000", 2024-05-08T08:54:00.176 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2259999999999999e-05 2024-05-08T08:54:00.177 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.177 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.177 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-05-08T08:54:00.177 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487528+0000", 2024-05-08T08:54:00.177 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7479999999999993e-06 2024-05-08T08:54:00.177 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.177 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.177 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:00.177 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:23.478139+0000", 2024-05-08T08:54:00.177 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.99061165200000001 2024-05-08T08:54:00.177 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.177 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.177 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:00.177 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:23.478301+0000", 2024-05-08T08:54:00.177 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000161602 2024-05-08T08:54:00.178 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.178 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.178 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:00.178 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:23.478483+0000", 2024-05-08T08:54:00.178 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000181781 2024-05-08T08:54:00.178 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.178 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:00.178 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.178 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.178 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:00.178 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:00.179 INFO:teuthology.orchestra.run.smithi196.stdout:checking clone #32, image image_client.0-clone32 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent33 2024-05-08T08:54:00.181 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-08T08:54:00.181 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:54:00.181 INFO:teuthology.orchestra.run.smithi059.stderr:no valid command found; 10 closest matches: 2024-05-08T08:54:00.182 INFO:teuthology.orchestra.run.smithi059.stderr:0 2024-05-08T08:54:00.182 INFO:teuthology.orchestra.run.smithi059.stderr:1 2024-05-08T08:54:00.182 INFO:teuthology.orchestra.run.smithi059.stderr:2 2024-05-08T08:54:00.182 INFO:teuthology.orchestra.run.smithi059.stderr:abort 2024-05-08T08:54:00.182 INFO:teuthology.orchestra.run.smithi059.stderr:assert 2024-05-08T08:54:00.182 INFO:teuthology.orchestra.run.smithi059.stderr:bench [] [] [] [] 2024-05-08T08:54:00.182 INFO:teuthology.orchestra.run.smithi059.stderr:bluefs debug_inject_read_zeros 2024-05-08T08:54:00.182 INFO:teuthology.orchestra.run.smithi059.stderr:bluefs files list 2024-05-08T08:54:00.182 INFO:teuthology.orchestra.run.smithi059.stderr:bluefs stats 2024-05-08T08:54:00.182 INFO:teuthology.orchestra.run.smithi059.stderr:bluestore allocator dump block 2024-05-08T08:54:00.182 INFO:teuthology.orchestra.run.smithi059.stderr:Error EINVAL: invalid command 2024-05-08T08:54:00.182 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-05-08T08:54:00.183 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-05-08T08:54:00.183 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:00.182+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:00.281 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:00.284+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:00.295 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:00.295 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:54:00.295 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:54:00.295 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:00.304 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:54:00.322 INFO:teuthology.orchestra.run.smithi196.stdout:4587 trunc from 0x8ebee00 to 0x76cf600 2024-05-08T08:54:00.345 INFO:teuthology.orchestra.run.smithi196.stdout:4588 read 0x210b200 thru 0x210db09 (0x290a bytes) 2024-05-08T08:54:00.346 INFO:teuthology.orchestra.run.smithi196.stdout:4591 read 0x5b6dc00 thru 0x5b767d4 (0x8bd5 bytes) 2024-05-08T08:54:00.347 INFO:teuthology.orchestra.run.smithi196.stdout:4593 write 0x24f5a00 thru 0x24f808f (0x2690 bytes) 2024-05-08T08:54:00.348 INFO:teuthology.orchestra.run.smithi196.stdout:4594 punch from 0x4b6a600 to 0x4b6f600, (0x5000 bytes) 2024-05-08T08:54:00.352 INFO:teuthology.orchestra.run.smithi196.stdout:4595 trunc from 0x76cf600 to 0x6942e00 2024-05-08T08:54:00.367 INFO:teuthology.orchestra.run.smithi059.stderr:no valid command found; 10 closest matches: 2024-05-08T08:54:00.367 INFO:teuthology.orchestra.run.smithi059.stderr:0 2024-05-08T08:54:00.367 INFO:teuthology.orchestra.run.smithi059.stderr:1 2024-05-08T08:54:00.367 INFO:teuthology.orchestra.run.smithi059.stderr:2 2024-05-08T08:54:00.367 INFO:teuthology.orchestra.run.smithi059.stderr:abort 2024-05-08T08:54:00.367 INFO:teuthology.orchestra.run.smithi059.stderr:assert 2024-05-08T08:54:00.367 INFO:teuthology.orchestra.run.smithi059.stderr:bench [] [] [] [] 2024-05-08T08:54:00.367 INFO:teuthology.orchestra.run.smithi059.stderr:bluefs debug_inject_read_zeros 2024-05-08T08:54:00.367 INFO:teuthology.orchestra.run.smithi059.stderr:bluefs files list 2024-05-08T08:54:00.367 INFO:teuthology.orchestra.run.smithi059.stderr:bluefs stats 2024-05-08T08:54:00.367 INFO:teuthology.orchestra.run.smithi059.stderr:bluestore allocator dump block 2024-05-08T08:54:00.367 INFO:teuthology.orchestra.run.smithi059.stderr:Error EINVAL: invalid command 2024-05-08T08:54:00.370 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-08T08:54:00.370 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-05-08T08:54:00.370 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 'trim stale osdmaps' 2024-05-08T08:54:00.371 INFO:teuthology.orchestra.run.smithi196.stdout:4596 write 0x13ef800 thru 0x13fa7e8 (0xafe9 bytes) 2024-05-08T08:54:00.371 INFO:teuthology.orchestra.run.smithi196.stdout:4597 read 0xbab600 thru 0xbac178 (0xb79 bytes) 2024-05-08T08:54:00.372 INFO:teuthology.orchestra.run.smithi196.stdout:4600 read 0x1d25600 thru 0x1d2d603 (0x8004 bytes) 2024-05-08T08:54:00.414 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:00.417+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:00.457 INFO:teuthology.orchestra.run.smithi196.stdout:4601 write 0xacf0600 thru 0xad00059 (0xfa5a bytes) 2024-05-08T08:54:00.457 INFO:teuthology.orchestra.run.smithi196.stdout:4602 write 0x2072600 thru 0x207ff2c (0xd92d bytes) 2024-05-08T08:54:00.458 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:00.458 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:54:00.458 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:54:00.458 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:54:00.458 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:00.467 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:54:00.470 INFO:teuthology.orchestra.run.smithi196.stdout:4604 trunc from 0xad00200 to 0xa24800 2024-05-08T08:54:00.483 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:00.486+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:00.515 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T08:54:00.518 INFO:teuthology.orchestra.run.smithi059.stderr:no valid command found; 10 closest matches: 2024-05-08T08:54:00.519 INFO:teuthology.orchestra.run.smithi059.stderr:0 2024-05-08T08:54:00.519 INFO:teuthology.orchestra.run.smithi059.stderr:1 2024-05-08T08:54:00.519 INFO:teuthology.orchestra.run.smithi059.stderr:2 2024-05-08T08:54:00.519 INFO:teuthology.orchestra.run.smithi059.stderr:abort 2024-05-08T08:54:00.519 INFO:teuthology.orchestra.run.smithi059.stderr:assert 2024-05-08T08:54:00.519 INFO:teuthology.orchestra.run.smithi059.stderr:bench [] [] [] [] 2024-05-08T08:54:00.519 INFO:teuthology.orchestra.run.smithi059.stderr:bluefs debug_inject_read_zeros 2024-05-08T08:54:00.519 INFO:teuthology.orchestra.run.smithi059.stderr:bluefs files list 2024-05-08T08:54:00.519 INFO:teuthology.orchestra.run.smithi059.stderr:bluefs stats 2024-05-08T08:54:00.519 INFO:teuthology.orchestra.run.smithi059.stderr:bluestore allocator dump block 2024-05-08T08:54:00.519 INFO:teuthology.orchestra.run.smithi059.stderr:Error EINVAL: invalid command 2024-05-08T08:54:00.522 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-08T08:54:00.522 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-05-08T08:54:00.543 INFO:teuthology.orchestra.run.smithi196.stdout:4606 clone 35 order 21 su 131072 sc 2 2024-05-08T08:54:00.584 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:00.587+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:00.585 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:00.586 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:54:00.586 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:54:00.586 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:54:00.586 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.586 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6690 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1093632~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:00.586 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:54:00.586 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 88.872626479000004, 2024-05-08T08:54:00.586 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052462754, 2024-05-08T08:54:00.586 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:00.586 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:00.586 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:00.586 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:00.586 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:00.587 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:00.587 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6690 2024-05-08T08:54:00.587 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.587 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:00.587 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.587 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:00.587 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:54:00.587 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:00.587 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.587 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.587 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:00.587 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:54:00.587 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:00.587 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.587 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.588 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:00.588 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715346+0000", 2024-05-08T08:54:00.588 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.84e-07 2024-05-08T08:54:00.588 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.588 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.588 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:00.588 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715348+0000", 2024-05-08T08:54:00.588 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.114e-06 2024-05-08T08:54:00.588 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.588 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.588 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:00.588 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715353+0000", 2024-05-08T08:54:00.588 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5720000000000002e-06 2024-05-08T08:54:00.588 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.589 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.589 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:00.589 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715358+0000", 2024-05-08T08:54:00.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7140000000000001e-06 2024-05-08T08:54:00.589 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.589 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.589 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:00.589 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767638+0000", 2024-05-08T08:54:00.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052279562000000002 2024-05-08T08:54:00.589 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.589 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.589 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:00.589 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767713+0000", 2024-05-08T08:54:00.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5652e-05 2024-05-08T08:54:00.590 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.590 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.590 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:00.590 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767808+0000", 2024-05-08T08:54:00.590 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4755999999999999e-05 2024-05-08T08:54:00.590 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.590 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:00.590 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.590 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.590 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.590 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6692 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1110016~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:00.590 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:54:00.590 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 88.872607388999995, 2024-05-08T08:54:00.590 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052666495000000001, 2024-05-08T08:54:00.591 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:00.591 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:00.591 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:00.591 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:00.591 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:00.591 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:00.591 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6692 2024-05-08T08:54:00.591 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.591 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:00.591 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.591 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:00.591 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:54:00.591 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:00.591 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.592 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.592 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:00.592 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:54:00.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:00.592 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.592 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.592 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:00.592 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:54:00.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8599999999999999e-07 2024-05-08T08:54:00.592 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.592 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.592 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:00.592 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715372+0000", 2024-05-08T08:54:00.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0990000000000001e-06 2024-05-08T08:54:00.592 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.593 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.593 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:00.593 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715378+0000", 2024-05-08T08:54:00.593 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0680000000000002e-06 2024-05-08T08:54:00.593 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.593 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.593 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:00.593 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715383+0000", 2024-05-08T08:54:00.593 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8740000000000003e-06 2024-05-08T08:54:00.593 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.593 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.593 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:00.593 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767848+0000", 2024-05-08T08:54:00.593 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052464668999999998 2024-05-08T08:54:00.593 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.594 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.594 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:00.594 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767927+0000", 2024-05-08T08:54:00.594 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9228999999999996e-05 2024-05-08T08:54:00.594 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.594 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.594 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:00.594 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768031+0000", 2024-05-08T08:54:00.594 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010417 2024-05-08T08:54:00.594 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.594 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:00.594 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.594 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.595 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.595 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6694 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1126400~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:00.595 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:54:00.595 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 88.872582492000006, 2024-05-08T08:54:00.595 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052848829, 2024-05-08T08:54:00.595 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:00.595 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:00.595 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:00.595 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:00.595 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:00.595 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:00.595 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6694 2024-05-08T08:54:00.595 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.595 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:00.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.596 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:00.596 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:54:00.596 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:00.596 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.596 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:00.596 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:54:00.596 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:00.596 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.596 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:00.596 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715390+0000", 2024-05-08T08:54:00.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9900000000000001e-07 2024-05-08T08:54:00.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.597 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:00.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715392+0000", 2024-05-08T08:54:00.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8279999999999999e-06 2024-05-08T08:54:00.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.597 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:00.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715398+0000", 2024-05-08T08:54:00.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7619999999999998e-06 2024-05-08T08:54:00.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.597 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:00.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715403+0000", 2024-05-08T08:54:00.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7790000000000003e-06 2024-05-08T08:54:00.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:00.598 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768067+0000", 2024-05-08T08:54:00.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052663785999999997 2024-05-08T08:54:00.598 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:00.598 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768142+0000", 2024-05-08T08:54:00.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5018999999999994e-05 2024-05-08T08:54:00.598 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.599 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:00.599 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768238+0000", 2024-05-08T08:54:00.599 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6255999999999995e-05 2024-05-08T08:54:00.599 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.599 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:00.599 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.599 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.599 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.599 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6696 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1142784~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:00.599 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:54:00.599 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 88.872562983999998, 2024-05-08T08:54:00.600 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.053051057999999998, 2024-05-08T08:54:00.600 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:00.600 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:00.600 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:00.600 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:00.600 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:00.600 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:00.600 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6696 2024-05-08T08:54:00.600 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.600 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:00.600 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.600 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:00.600 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:54:00.600 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:00.600 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.600 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.601 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:00.601 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:54:00.601 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:00.601 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.601 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:00.601 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:54:00.601 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-05-08T08:54:00.601 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.601 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:00.601 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715411+0000", 2024-05-08T08:54:00.601 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9630000000000001e-06 2024-05-08T08:54:00.601 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:00.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715417+0000", 2024-05-08T08:54:00.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7250000000000002e-06 2024-05-08T08:54:00.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:00.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715421+0000", 2024-05-08T08:54:00.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4089999999999996e-06 2024-05-08T08:54:00.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:00.603 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768277+0000", 2024-05-08T08:54:00.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052855293999999997 2024-05-08T08:54:00.603 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.603 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:00.603 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768358+0000", 2024-05-08T08:54:00.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1720999999999999e-05 2024-05-08T08:54:00.603 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.603 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:00.603 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768460+0000", 2024-05-08T08:54:00.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010154600000000001 2024-05-08T08:54:00.603 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.603 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:00.603 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.604 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.604 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.604 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6698 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1159168~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:00.604 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:54:00.604 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 88.872544117000004, 2024-05-08T08:54:00.604 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.053245726, 2024-05-08T08:54:00.604 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:00.604 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:00.604 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:00.604 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:00.604 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:00.604 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:00.604 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6698 2024-05-08T08:54:00.604 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.604 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:00.605 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.605 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:00.605 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:54:00.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:00.605 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.605 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.605 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:00.605 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:54:00.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:00.605 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.605 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.605 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:00.605 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716936+0000", 2024-05-08T08:54:00.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001508665 2024-05-08T08:54:00.606 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.606 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:00.606 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716941+0000", 2024-05-08T08:54:00.606 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1910000000000002e-06 2024-05-08T08:54:00.606 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.606 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:00.606 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716950+0000", 2024-05-08T08:54:00.606 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1900000000000001e-06 2024-05-08T08:54:00.606 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.606 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:00.607 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716958+0000", 2024-05-08T08:54:00.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8019999999999992e-06 2024-05-08T08:54:00.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.607 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:00.607 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768499+0000", 2024-05-08T08:54:00.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051540926000000001 2024-05-08T08:54:00.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.607 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:00.607 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768574+0000", 2024-05-08T08:54:00.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4988999999999996e-05 2024-05-08T08:54:00.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.608 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:00.608 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768673+0000", 2024-05-08T08:54:00.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9963000000000005e-05 2024-05-08T08:54:00.608 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.608 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:00.608 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.608 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6706 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1224704~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:00.608 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:54:00.608 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 88.870938809999998, 2024-05-08T08:54:00.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052435598, 2024-05-08T08:54:00.608 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:00.608 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:00.608 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:00.609 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:00.609 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:00.609 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:00.609 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6706 2024-05-08T08:54:00.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.609 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:00.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.609 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:00.609 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:54:00.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:00.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.609 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:00.609 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:54:00.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:00.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.610 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:00.610 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:54:00.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:54:00.610 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.610 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:00.610 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717035+0000", 2024-05-08T08:54:00.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9290000000000001e-06 2024-05-08T08:54:00.610 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.610 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:00.610 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717041+0000", 2024-05-08T08:54:00.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6720000000000001e-06 2024-05-08T08:54:00.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.611 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:00.611 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717046+0000", 2024-05-08T08:54:00.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5650000000000003e-06 2024-05-08T08:54:00.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.611 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:00.611 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769311+0000", 2024-05-08T08:54:00.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052265645999999999 2024-05-08T08:54:00.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.611 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:00.611 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769379+0000", 2024-05-08T08:54:00.612 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7391000000000006e-05 2024-05-08T08:54:00.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.612 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:00.612 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769469+0000", 2024-05-08T08:54:00.612 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0025999999999995e-05 2024-05-08T08:54:00.612 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.612 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:00.612 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.612 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6708 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1241088~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:00.613 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:54:00.613 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 88.870920706000007, 2024-05-08T08:54:00.613 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052640552, 2024-05-08T08:54:00.613 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:00.613 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:00.613 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:00.613 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:00.613 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:00.613 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:00.613 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6708 2024-05-08T08:54:00.613 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.613 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:00.613 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.613 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:00.614 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:54:00.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:00.614 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.614 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.614 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:00.614 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:54:00.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:00.614 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.614 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.614 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:00.614 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717052+0000", 2024-05-08T08:54:00.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:54:00.614 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:00.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717053+0000", 2024-05-08T08:54:00.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8330000000000001e-06 2024-05-08T08:54:00.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:00.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717058+0000", 2024-05-08T08:54:00.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.933e-06 2024-05-08T08:54:00.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:00.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717063+0000", 2024-05-08T08:54:00.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2180000000000001e-06 2024-05-08T08:54:00.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:00.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769507+0000", 2024-05-08T08:54:00.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052444866999999999 2024-05-08T08:54:00.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:00.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769589+0000", 2024-05-08T08:54:00.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2039000000000005e-05 2024-05-08T08:54:00.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:00.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769692+0000", 2024-05-08T08:54:00.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102252 2024-05-08T08:54:00.616 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.617 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:00.617 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.617 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.617 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6710 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1257472~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:00.617 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:54:00.617 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 88.870902681999993, 2024-05-08T08:54:00.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.05283057, 2024-05-08T08:54:00.617 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:00.617 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:00.617 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:00.617 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:00.617 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:00.617 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:00.617 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6710 2024-05-08T08:54:00.618 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.618 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:00.618 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.618 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:00.618 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:54:00.618 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:00.618 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.618 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.618 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:00.618 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:54:00.618 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:00.618 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.618 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.619 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:00.619 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718951+0000", 2024-05-08T08:54:00.619 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0018817809999999999 2024-05-08T08:54:00.619 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.619 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.619 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:00.619 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718956+0000", 2024-05-08T08:54:00.619 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5399999999999997e-06 2024-05-08T08:54:00.619 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.619 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.619 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:00.619 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718964+0000", 2024-05-08T08:54:00.619 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6629999999999999e-06 2024-05-08T08:54:00.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.620 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:00.620 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718972+0000", 2024-05-08T08:54:00.620 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5229999999999998e-06 2024-05-08T08:54:00.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.620 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:00.620 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769729+0000", 2024-05-08T08:54:00.620 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050756959999999997 2024-05-08T08:54:00.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.620 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:00.620 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769803+0000", 2024-05-08T08:54:00.620 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4538999999999999e-05 2024-05-08T08:54:00.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:00.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769900+0000", 2024-05-08T08:54:00.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6564000000000006e-05 2024-05-08T08:54:00.621 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.621 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:00.621 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.621 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6726 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1388544~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:00.621 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:54:00.621 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 88.868827909999993, 2024-05-08T08:54:00.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052451506000000002, 2024-05-08T08:54:00.621 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:00.622 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:00.622 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:00.622 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:00.622 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:00.622 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:00.622 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6726 2024-05-08T08:54:00.622 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.622 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:00.622 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.622 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:00.622 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:54:00.622 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:00.622 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.623 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.623 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:00.623 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:54:00.623 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:00.623 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.623 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.623 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:00.623 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719200+0000", 2024-05-08T08:54:00.623 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5865999999999997e-05 2024-05-08T08:54:00.623 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.623 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.623 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:00.623 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719204+0000", 2024-05-08T08:54:00.623 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8630000000000001e-06 2024-05-08T08:54:00.623 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.624 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:00.624 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719215+0000", 2024-05-08T08:54:00.624 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1590000000000001e-05 2024-05-08T08:54:00.624 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.624 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:00.624 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719220+0000", 2024-05-08T08:54:00.624 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1619999999999999e-06 2024-05-08T08:54:00.624 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.624 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:00.624 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771425+0000", 2024-05-08T08:54:00.624 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052204249000000001 2024-05-08T08:54:00.624 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.625 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:00.625 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771498+0000", 2024-05-08T08:54:00.625 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3782999999999995e-05 2024-05-08T08:54:00.625 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.625 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.625 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:00.625 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771595+0000", 2024-05-08T08:54:00.625 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6992999999999998e-05 2024-05-08T08:54:00.625 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.625 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:00.625 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.625 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.625 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.625 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6728 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1404928~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:00.625 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:54:00.625 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 88.868740505000005, 2024-05-08T08:54:00.626 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052590747, 2024-05-08T08:54:00.626 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:00.626 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:00.626 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:00.626 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:00.626 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:00.626 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:00.626 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6728 2024-05-08T08:54:00.626 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.626 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:00.626 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.626 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:00.626 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:54:00.626 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:00.627 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.627 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.627 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:00.627 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:54:00.627 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:00.627 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.627 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.627 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:00.627 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719232+0000", 2024-05-08T08:54:00.627 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4000000000000001e-07 2024-05-08T08:54:00.627 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.627 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.627 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:00.627 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719234+0000", 2024-05-08T08:54:00.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4499999999999998e-06 2024-05-08T08:54:00.628 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.628 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:00.628 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719240+0000", 2024-05-08T08:54:00.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5720000000000002e-06 2024-05-08T08:54:00.628 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.628 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:00.628 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719245+0000", 2024-05-08T08:54:00.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8690000000000003e-06 2024-05-08T08:54:00.628 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.628 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:00.628 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771636+0000", 2024-05-08T08:54:00.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052390979999999997 2024-05-08T08:54:00.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.629 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:00.629 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771717+0000", 2024-05-08T08:54:00.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1291999999999993e-05 2024-05-08T08:54:00.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.629 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:00.629 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771822+0000", 2024-05-08T08:54:00.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104944 2024-05-08T08:54:00.629 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.629 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:00.629 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.630 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.630 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6730 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1421312~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:00.630 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:54:00.630 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 88.868720263, 2024-05-08T08:54:00.630 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052770477000000003, 2024-05-08T08:54:00.630 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:00.630 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:00.630 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:00.630 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:00.630 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:00.630 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:00.630 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6730 2024-05-08T08:54:00.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.631 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:00.631 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.631 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:00.631 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:54:00.631 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:00.631 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.631 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.631 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:00.631 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:54:00.631 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:00.631 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.631 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.631 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:00.632 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:54:00.632 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9000000000000002e-07 2024-05-08T08:54:00.632 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.632 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.632 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:00.632 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719254+0000", 2024-05-08T08:54:00.632 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4099999999999998e-06 2024-05-08T08:54:00.632 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.632 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.632 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:00.632 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719263+0000", 2024-05-08T08:54:00.632 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4109999999999996e-06 2024-05-08T08:54:00.632 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.632 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.632 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:00.633 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719268+0000", 2024-05-08T08:54:00.633 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2669999999999997e-06 2024-05-08T08:54:00.633 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.633 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.633 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:00.633 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771846+0000", 2024-05-08T08:54:00.633 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052577677000000003 2024-05-08T08:54:00.633 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.633 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.633 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:00.633 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771920+0000", 2024-05-08T08:54:00.633 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4460000000000002e-05 2024-05-08T08:54:00.633 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.633 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.633 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:00.634 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772022+0000", 2024-05-08T08:54:00.634 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010186199999999999 2024-05-08T08:54:00.634 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.634 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:00.634 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.634 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.634 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6732 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1437696~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:00.634 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:54:00.634 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 88.868695455999998, 2024-05-08T08:54:00.634 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052905575000000003, 2024-05-08T08:54:00.634 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:00.634 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:00.634 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:00.635 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:00.635 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:00.635 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:00.635 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6732 2024-05-08T08:54:00.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.635 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:00.635 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.635 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:00.635 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:54:00.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:00.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.635 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.635 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:00.635 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:54:00.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:00.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.636 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:00.636 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721392+0000", 2024-05-08T08:54:00.636 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002115287 2024-05-08T08:54:00.636 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.636 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:00.636 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721396+0000", 2024-05-08T08:54:00.636 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1679999999999997e-06 2024-05-08T08:54:00.636 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.636 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:00.636 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721404+0000", 2024-05-08T08:54:00.636 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4409999999999993e-06 2024-05-08T08:54:00.637 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.637 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.637 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:00.637 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721410+0000", 2024-05-08T08:54:00.637 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9150000000000001e-06 2024-05-08T08:54:00.637 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.637 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.637 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:00.637 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772061+0000", 2024-05-08T08:54:00.637 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050651068 2024-05-08T08:54:00.637 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.637 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.637 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:00.637 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772127+0000", 2024-05-08T08:54:00.637 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5424000000000002e-05 2024-05-08T08:54:00.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.638 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.638 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:00.638 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772182+0000", 2024-05-08T08:54:00.638 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5272000000000002e-05 2024-05-08T08:54:00.638 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.638 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:00.638 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.638 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.638 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6749 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1585152~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:00.638 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:54:00.638 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 88.866394686000007, 2024-05-08T08:54:00.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052350264000000001, 2024-05-08T08:54:00.639 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:00.639 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:00.639 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:00.639 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:00.639 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:00.639 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:00.639 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6749 2024-05-08T08:54:00.639 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.639 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:00.639 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.639 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:00.639 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:54:00.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:00.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.640 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.640 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:00.640 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:54:00.640 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:00.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.640 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.640 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:00.640 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721578+0000", 2024-05-08T08:54:00.640 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0399999999999996e-07 2024-05-08T08:54:00.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.640 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.640 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:00.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721580+0000", 2024-05-08T08:54:00.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5450000000000002e-06 2024-05-08T08:54:00.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.641 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:00.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721587+0000", 2024-05-08T08:54:00.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0190000000000001e-06 2024-05-08T08:54:00.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.641 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:00.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721593+0000", 2024-05-08T08:54:00.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5659999999999997e-06 2024-05-08T08:54:00.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:00.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773759+0000", 2024-05-08T08:54:00.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052166553999999997 2024-05-08T08:54:00.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:00.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773833+0000", 2024-05-08T08:54:00.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3913999999999997e-05 2024-05-08T08:54:00.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:00.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773927+0000", 2024-05-08T08:54:00.643 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4061999999999995e-05 2024-05-08T08:54:00.643 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.643 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:00.643 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.643 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.643 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.643 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6751 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1601536~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:00.643 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:54:00.643 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 88.866371407000003, 2024-05-08T08:54:00.643 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052461302000000001, 2024-05-08T08:54:00.643 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:00.644 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:00.644 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:00.644 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:00.644 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:00.644 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:00.644 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6751 2024-05-08T08:54:00.644 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.644 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:00.644 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.644 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:00.644 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:54:00.644 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:00.644 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.644 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.645 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:00.645 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:54:00.645 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:00.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.645 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.645 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:00.645 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722939+0000", 2024-05-08T08:54:00.645 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001338601 2024-05-08T08:54:00.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.645 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.645 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:00.645 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722944+0000", 2024-05-08T08:54:00.645 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4469999999999999e-06 2024-05-08T08:54:00.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.645 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.645 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:00.646 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722955+0000", 2024-05-08T08:54:00.646 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1892999999999999e-05 2024-05-08T08:54:00.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.646 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:00.646 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722963+0000", 2024-05-08T08:54:00.646 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1080000000000004e-06 2024-05-08T08:54:00.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.646 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:00.646 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773964+0000", 2024-05-08T08:54:00.646 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051001022 2024-05-08T08:54:00.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.647 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:00.647 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.774006+0000", 2024-05-08T08:54:00.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2729999999999999e-05 2024-05-08T08:54:00.647 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.647 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.647 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:00.647 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.774062+0000", 2024-05-08T08:54:00.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5501000000000003e-05 2024-05-08T08:54:00.647 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.647 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:00.647 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.647 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.647 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.647 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6760 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1683456~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:00.648 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:54:00.648 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 88.864862031000001, 2024-05-08T08:54:00.648 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052571312000000002, 2024-05-08T08:54:00.648 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:00.648 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:00.648 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:00.648 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:00.648 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:00.648 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:00.648 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6760 2024-05-08T08:54:00.648 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.648 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:00.648 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.649 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:00.649 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:54:00.649 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:00.649 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.649 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.649 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:00.649 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:54:00.649 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:00.649 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.649 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.649 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:00.649 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:54:00.649 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2100000000000002e-07 2024-05-08T08:54:00.649 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.649 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.649 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:00.650 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723113+0000", 2024-05-08T08:54:00.650 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3149999999999999e-06 2024-05-08T08:54:00.650 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.650 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.650 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:00.650 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723118+0000", 2024-05-08T08:54:00.650 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8860000000000001e-06 2024-05-08T08:54:00.650 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.650 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.650 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:00.650 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723123+0000", 2024-05-08T08:54:00.650 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6770000000000003e-06 2024-05-08T08:54:00.651 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.651 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.651 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:00.651 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775414+0000", 2024-05-08T08:54:00.651 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052290692999999999 2024-05-08T08:54:00.651 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.651 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.651 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:00.651 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775459+0000", 2024-05-08T08:54:00.651 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5639999999999997e-05 2024-05-08T08:54:00.651 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.651 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.651 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:00.651 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775681+0000", 2024-05-08T08:54:00.652 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00022168000000000001 2024-05-08T08:54:00.652 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.652 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:00.652 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.652 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.652 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.652 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6761 3.16 3:6bd6d6da:::rbd_data.1140782590a5.0000000000000015:6 [read 1095680~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:00.652 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:54:00.652 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 88.864842811000003, 2024-05-08T08:54:00.652 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052751275, 2024-05-08T08:54:00.652 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:00.652 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:00.652 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:00.652 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:00.652 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:00.653 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:00.653 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6761 2024-05-08T08:54:00.653 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.653 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:00.653 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.653 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:00.653 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:54:00.653 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:00.653 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.653 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.653 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:00.653 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:54:00.653 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:00.653 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.653 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.654 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:00.654 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724939+0000", 2024-05-08T08:54:00.654 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001809817 2024-05-08T08:54:00.654 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.654 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.654 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:00.654 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724943+0000", 2024-05-08T08:54:00.654 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4920000000000004e-06 2024-05-08T08:54:00.654 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.654 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.654 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:00.654 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724956+0000", 2024-05-08T08:54:00.654 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2427e-05 2024-05-08T08:54:00.655 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.655 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.655 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:00.655 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724963+0000", 2024-05-08T08:54:00.655 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2559999999999996e-06 2024-05-08T08:54:00.655 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.655 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.655 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:00.655 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775711+0000", 2024-05-08T08:54:00.655 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050747621 2024-05-08T08:54:00.655 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.655 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.655 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:00.655 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775785+0000", 2024-05-08T08:54:00.656 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4634000000000004e-05 2024-05-08T08:54:00.656 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.656 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.656 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:00.656 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775880+0000", 2024-05-08T08:54:00.656 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5027999999999999e-05 2024-05-08T08:54:00.656 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.656 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:00.657 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.657 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.657 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.657 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6770 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1765376~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:00.657 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:54:00.657 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 88.862811715999996, 2024-05-08T08:54:00.657 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052339296, 2024-05-08T08:54:00.657 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:00.657 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:00.657 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:00.657 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:00.657 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:00.657 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:00.657 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6770 2024-05-08T08:54:00.658 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.658 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:00.658 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.658 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:00.658 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:54:00.658 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:00.658 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.658 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.658 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:00.658 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:54:00.658 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:00.658 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.658 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.658 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:00.659 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725161+0000", 2024-05-08T08:54:00.659 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-05-08T08:54:00.659 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.659 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.659 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:00.659 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725163+0000", 2024-05-08T08:54:00.662 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4370000000000001e-06 2024-05-08T08:54:00.662 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.662 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.662 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:00.662 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725169+0000", 2024-05-08T08:54:00.662 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8409999999999996e-06 2024-05-08T08:54:00.662 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.662 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.662 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:00.663 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725174+0000", 2024-05-08T08:54:00.663 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7249999999999997e-06 2024-05-08T08:54:00.663 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.663 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.663 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:00.663 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777343+0000", 2024-05-08T08:54:00.663 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052169633999999999 2024-05-08T08:54:00.663 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.663 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.663 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:00.663 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777409+0000", 2024-05-08T08:54:00.663 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6033999999999998e-05 2024-05-08T08:54:00.664 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.664 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.664 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:00.664 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777499+0000", 2024-05-08T08:54:00.664 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0224999999999998e-05 2024-05-08T08:54:00.664 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.664 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:00.664 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.664 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.664 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.664 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6771 3.16 3:6bd6d6da:::rbd_data.1140782590a5.0000000000000015:6 [read 1177600~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:00.664 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:54:00.664 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 88.862791899000001, 2024-05-08T08:54:00.664 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052438604999999999, 2024-05-08T08:54:00.664 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:00.665 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:00.665 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:00.665 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:00.665 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:00.665 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:00.665 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6771 2024-05-08T08:54:00.665 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.665 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:00.665 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.665 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:00.665 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:54:00.665 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:00.665 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.665 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.665 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:00.665 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:54:00.665 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:00.666 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.666 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.666 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:00.666 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:54:00.666 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:54:00.666 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.666 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.666 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:00.666 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725188+0000", 2024-05-08T08:54:00.666 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1579999999999999e-06 2024-05-08T08:54:00.666 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.666 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.666 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:00.666 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725195+0000", 2024-05-08T08:54:00.667 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8909999999999995e-06 2024-05-08T08:54:00.667 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.667 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.667 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:00.667 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725200+0000", 2024-05-08T08:54:00.667 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9810000000000003e-06 2024-05-08T08:54:00.667 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.667 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.667 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:00.667 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777527+0000", 2024-05-08T08:54:00.667 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052326582000000003 2024-05-08T08:54:00.667 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.667 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.667 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:00.667 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777566+0000", 2024-05-08T08:54:00.667 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9447999999999999e-05 2024-05-08T08:54:00.667 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.668 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.668 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:00.668 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777619+0000", 2024-05-08T08:54:00.668 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2135000000000001e-05 2024-05-08T08:54:00.668 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.668 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:00.668 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.668 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.668 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.668 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6772 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1781760~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:00.668 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:54:00.669 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 88.862763842000007, 2024-05-08T08:54:00.669 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052601799999999997, 2024-05-08T08:54:00.669 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:00.669 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:00.669 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:00.669 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:00.669 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:00.669 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:00.669 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6772 2024-05-08T08:54:00.669 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.669 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:00.669 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.669 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:00.669 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:54:00.669 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:00.670 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.670 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.670 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:00.670 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:54:00.670 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:00.670 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.670 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.670 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:00.670 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726939+0000", 2024-05-08T08:54:00.670 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001730493 2024-05-08T08:54:00.670 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.670 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.670 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:00.670 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726943+0000", 2024-05-08T08:54:00.670 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7330000000000003e-06 2024-05-08T08:54:00.670 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.671 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.671 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:00.671 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726954+0000", 2024-05-08T08:54:00.671 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1107000000000001e-05 2024-05-08T08:54:00.671 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.671 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.671 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:00.671 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726962+0000", 2024-05-08T08:54:00.671 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.503e-06 2024-05-08T08:54:00.671 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.671 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.671 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:00.671 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777653+0000", 2024-05-08T08:54:00.671 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050691542999999999 2024-05-08T08:54:00.671 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.671 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.672 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:00.672 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777719+0000", 2024-05-08T08:54:00.672 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5228000000000006e-05 2024-05-08T08:54:00.672 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.672 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.672 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:00.672 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777810+0000", 2024-05-08T08:54:00.672 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1193000000000006e-05 2024-05-08T08:54:00.672 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.672 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:00.672 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.672 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.672 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.672 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6782 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1880064~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:00.672 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:54:00.673 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 88.860795335000006, 2024-05-08T08:54:00.673 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052326492000000002, 2024-05-08T08:54:00.673 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:00.673 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:00.673 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:00.673 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:00.673 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:00.673 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:00.673 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6782 2024-05-08T08:54:00.673 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.673 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:00.674 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.674 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:00.674 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:54:00.674 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:00.674 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.674 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.674 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:00.674 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:54:00.674 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:00.674 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.674 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.674 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:00.674 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728939+0000", 2024-05-08T08:54:00.674 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001762719 2024-05-08T08:54:00.675 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.675 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.675 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:00.675 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728944+0000", 2024-05-08T08:54:00.675 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.746e-06 2024-05-08T08:54:00.675 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.675 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.675 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:00.675 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728955+0000", 2024-05-08T08:54:00.675 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1043000000000001e-05 2024-05-08T08:54:00.675 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.675 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.675 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:00.675 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728963+0000", 2024-05-08T08:54:00.675 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5809999999999999e-06 2024-05-08T08:54:00.676 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.676 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.676 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:00.676 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779320+0000", 2024-05-08T08:54:00.676 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050357290999999998 2024-05-08T08:54:00.676 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.676 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.676 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:00.676 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779401+0000", 2024-05-08T08:54:00.676 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0574000000000005e-05 2024-05-08T08:54:00.676 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:00.676 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:00.676 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:00.676 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779503+0000", 2024-05-08T08:54:00.676 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102538 2024-05-08T08:54:00.676 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.677 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:00.677 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.677 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:00.677 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:00.677 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:00.678 INFO:teuthology.orchestra.run.smithi059.stderr:noscrub is set 2024-05-08T08:54:00.678 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:54:00.678 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:54:00.680 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:54:00.680 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:54:00.681 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:54:00.681 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:54:00.681 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:54:00.700 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:54:00.700 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:54:00.701 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:00.703+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:00.719 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:54:00.719 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:54:00.736 INFO:teuthology.orchestra.run.smithi196.stdout:truncating image image_client.0-clone34 from 0xa24800 (overlap 0xa24800) to 0x0 2024-05-08T08:54:00.737 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:54:00.737 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:54:00.786 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:00.789+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:00.795 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:00.795 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:00.804 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:54:00.887 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:00.890+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:00.915 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:00.915 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:00.924 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:54:01.009 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:01.012+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:01.074 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:01.074 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:01.083 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:54:01.124 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:01.127+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:01.190 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:01.193+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:01.233 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:01.234 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:01.243 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:54:01.275 INFO:teuthology.orchestra.run.smithi196.stdout:checking clone #33, image image_client.0-clone33 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent34 2024-05-08T08:54:01.291 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:01.294+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:01.330 INFO:teuthology.orchestra.run.smithi196.stdout:4607 read 0x5e7e00 thru 0x5edcb5 (0x5eb6 bytes) 2024-05-08T08:54:01.331 INFO:teuthology.orchestra.run.smithi196.stdout:truncating to largest ever: 0xee36200 2024-05-08T08:54:01.349 INFO:teuthology.orchestra.run.smithi196.stdout:4609 trunc from 0xa24800 to 0xee36200 2024-05-08T08:54:01.352 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:01.352 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:01.361 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:54:01.363 INFO:teuthology.orchestra.run.smithi196.stdout:4610 write 0x49bb400 thru 0x49c008b (0x4c8c bytes) 2024-05-08T08:54:01.364 INFO:teuthology.orchestra.run.smithi196.stdout:4612 punch from 0x68b2c00 to 0x68bd000, (0xa400 bytes) 2024-05-08T08:54:01.369 INFO:teuthology.orchestra.run.smithi196.stdout:4613 write 0xc51200 thru 0xc537cb (0x25cc bytes) 2024-05-08T08:54:01.369 INFO:teuthology.orchestra.run.smithi196.stdout:4617 read 0x9b8d800 thru 0x9b95000 (0x7801 bytes) 2024-05-08T08:54:01.370 INFO:teuthology.orchestra.run.smithi196.stdout:4619 write 0xc46b600 thru 0xc47aa09 (0xf40a bytes) 2024-05-08T08:54:01.370 INFO:teuthology.orchestra.run.smithi196.stdout:4621 write 0x23b9400 thru 0x23c7623 (0xe224 bytes) 2024-05-08T08:54:01.371 INFO:teuthology.orchestra.run.smithi196.stdout:4623 trunc from 0xee36200 to 0xddec800 2024-05-08T08:54:01.402 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:01.405+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:01.402 INFO:teuthology.orchestra.run.smithi196.stdout:4628 trunc from 0xddec800 to 0x2ba9e00 2024-05-08T08:54:01.417 INFO:teuthology.orchestra.run.smithi196.stdout:4629 read 0x1cf000 thru 0x1d6e73 (0x7e74 bytes) 2024-05-08T08:54:01.437 INFO:teuthology.orchestra.run.smithi196.stdout:4630 write 0x93e7400 thru 0x93f4b29 (0xd72a bytes) 2024-05-08T08:54:01.437 INFO:teuthology.orchestra.run.smithi196.stdout:4631 punch from 0x7b54000 to 0x7b5c200, (0x8200 bytes) 2024-05-08T08:54:01.438 INFO:teuthology.orchestra.run.smithi196.stdout:4632 read 0x2239c00 thru 0x2239efb (0x2fc bytes) 2024-05-08T08:54:01.439 INFO:teuthology.orchestra.run.smithi196.stdout:4634 write 0x86a4000 thru 0x86b0966 (0xc967 bytes) 2024-05-08T08:54:01.439 INFO:teuthology.orchestra.run.smithi196.stdout:4640 write 0x2094600 thru 0x20961e9 (0x1bea bytes) 2024-05-08T08:54:01.439 INFO:teuthology.orchestra.run.smithi196.stdout:4641 punch from 0x8306a00 to 0x830ea00, (0x8000 bytes) 2024-05-08T08:54:01.440 INFO:teuthology.orchestra.run.smithi196.stdout:4642 write 0x4e19200 thru 0x4e25226 (0xc027 bytes) 2024-05-08T08:54:01.440 INFO:teuthology.orchestra.run.smithi196.stdout:4643 write 0x8c30c00 thru 0x8c3694b (0x5d4c bytes) 2024-05-08T08:54:01.460 INFO:teuthology.orchestra.run.smithi196.stdout:4644 write 0x97eec00 thru 0x97f3c8b (0x508c bytes) 2024-05-08T08:54:01.465 INFO:teuthology.orchestra.run.smithi196.stdout:4645 trunc from 0x97f3e00 to 0x560ec00 2024-05-08T08:54:01.482 INFO:teuthology.orchestra.run.smithi196.stdout:4648 read 0x21efa00 thru 0x21f3a7d (0x407e bytes) 2024-05-08T08:54:01.489 INFO:teuthology.orchestra.run.smithi196.stdout:4649 trunc from 0x560ec00 to 0x9b80600 2024-05-08T08:54:01.493 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:01.496+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:01.504 INFO:teuthology.orchestra.run.smithi196.stdout:4650 trunc from 0x9b80600 to 0x4130400 2024-05-08T08:54:01.516 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:01.516 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:01.521 INFO:teuthology.orchestra.run.smithi196.stdout:4651 read 0x2433400 thru 0x2440669 (0xd26a bytes) 2024-05-08T08:54:01.525 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:54:01.528 INFO:teuthology.orchestra.run.smithi196.stdout:4652 punch from 0x145000 to 0x146600, (0x1600 bytes) 2024-05-08T08:54:01.529 INFO:teuthology.orchestra.run.smithi196.stdout:4654 punch from 0xc78400 to 0xc7c600, (0x4200 bytes) 2024-05-08T08:54:01.529 INFO:teuthology.orchestra.run.smithi196.stdout:4656 punch from 0x23c7a00 to 0x23d4e00, (0xd400 bytes) 2024-05-08T08:54:01.529 INFO:teuthology.orchestra.run.smithi196.stdout:4657 read 0x2152400 thru 0x2158032 (0x5c33 bytes) 2024-05-08T08:54:01.532 INFO:teuthology.orchestra.run.smithi196.stdout:4663 read 0x3a72e00 thru 0x3a7fdd1 (0xcfd2 bytes) 2024-05-08T08:54:01.532 INFO:teuthology.orchestra.run.smithi196.stdout:4669 read 0x2ae1600 thru 0x2aed33f (0xbd40 bytes) 2024-05-08T08:54:01.532 INFO:teuthology.orchestra.run.smithi196.stdout:4671 punch from 0x3d7fe00 to 0x3d80600, (0x800 bytes) 2024-05-08T08:54:01.532 INFO:teuthology.orchestra.run.smithi196.stdout:4673 read 0x29fdc00 thru 0x29fe11a (0x51b bytes) 2024-05-08T08:54:01.558 INFO:teuthology.orchestra.run.smithi196.stdout:4675 write 0xa381600 thru 0xa384c08 (0x3609 bytes) 2024-05-08T08:54:01.559 INFO:teuthology.orchestra.run.smithi196.stdout:4678 punch from 0x7cbbe00 to 0x7cc4200, (0x8400 bytes) 2024-05-08T08:54:01.560 INFO:teuthology.orchestra.run.smithi196.stdout:4681 read 0x414b600 thru 0x414d09b (0x1a9c bytes) 2024-05-08T08:54:01.560 INFO:teuthology.orchestra.run.smithi196.stdout:4682 write 0x8671000 thru 0x8673b26 (0x2b27 bytes) 2024-05-08T08:54:01.560 INFO:teuthology.orchestra.run.smithi196.stdout:4683 punch from 0x9090e00 to 0x9097600, (0x6800 bytes) 2024-05-08T08:54:01.560 INFO:teuthology.orchestra.run.smithi196.stdout:4684 read 0x944a400 thru 0x94519b0 (0x75b1 bytes) 2024-05-08T08:54:01.565 INFO:teuthology.orchestra.run.smithi196.stdout:4685 trunc from 0xa384e00 to 0x61c2e00 2024-05-08T08:54:01.581 INFO:teuthology.orchestra.run.smithi196.stdout:4688 punch from 0x20fa200 to 0x2108000, (0xde00 bytes) 2024-05-08T08:54:01.583 INFO:teuthology.orchestra.run.smithi196.stdout:4689 punch from 0x11bfa00 to 0x11c2600, (0x2c00 bytes) 2024-05-08T08:54:01.583 INFO:teuthology.orchestra.run.smithi196.stdout:4690 punch from 0x2f0ee00 to 0x2f17e00, (0x9000 bytes) 2024-05-08T08:54:01.583 INFO:teuthology.orchestra.run.smithi196.stdout:4694 punch from 0x5957e00 to 0x5963000, (0xb200 bytes) 2024-05-08T08:54:01.583 INFO:teuthology.orchestra.run.smithi196.stdout:4696 write 0x2ebb200 thru 0x2ebcdf6 (0x1bf7 bytes) 2024-05-08T08:54:01.583 INFO:teuthology.orchestra.run.smithi196.stdout:4698 write 0x30aba00 thru 0x30b132d (0x592e bytes) 2024-05-08T08:54:01.583 INFO:teuthology.orchestra.run.smithi196.stdout:4700 write 0x3b3fe00 thru 0x3b3ff84 (0x185 bytes) 2024-05-08T08:54:01.607 INFO:teuthology.orchestra.run.smithi196.stdout:4702 write 0x694d400 thru 0x6957e32 (0xaa33 bytes) 2024-05-08T08:54:01.610 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:01.613+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:01.623 INFO:teuthology.orchestra.run.smithi196.stdout:4703 trunc from 0x6958000 to 0x7ef800 2024-05-08T08:54:01.667 INFO:teuthology.orchestra.run.smithi196.stdout:4704 read 0x529c00 thru 0x530252 (0x6653 bytes) 2024-05-08T08:54:01.675 INFO:teuthology.orchestra.run.smithi196.stdout:4705 read 0x4f800 thru 0x5af2b (0xb72c bytes) 2024-05-08T08:54:01.675 INFO:teuthology.orchestra.run.smithi196.stdout:4707 punch from 0x5b6800 to 0x5b8a00, (0x2200 bytes) 2024-05-08T08:54:01.675 INFO:teuthology.orchestra.run.smithi196.stdout:4708 read 0x42e400 thru 0x437a2e (0x962f bytes) 2024-05-08T08:54:01.686 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:01.687 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:01.693 INFO:teuthology.orchestra.run.smithi196.stdout:4709 write 0x29aee00 thru 0x29b3534 (0x4735 bytes) 2024-05-08T08:54:01.694 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:01.697+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:01.697 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:54:01.723 INFO:teuthology.orchestra.run.smithi196.stdout:4710 write 0xe28f000 thru 0xe28ff64 (0xf65 bytes) 2024-05-08T08:54:01.724 INFO:teuthology.orchestra.run.smithi196.stdout:4711 read 0xad31200 thru 0xad36711 (0x5512 bytes) 2024-05-08T08:54:01.724 INFO:teuthology.orchestra.run.smithi196.stdout:4714 write 0xd5800 thru 0xd96f1 (0x3ef2 bytes) 2024-05-08T08:54:01.730 INFO:teuthology.orchestra.run.smithi196.stdout:4715 punch from 0x29c8000 to 0x29cc600, (0x4600 bytes) 2024-05-08T08:54:01.730 INFO:teuthology.orchestra.run.smithi196.stdout:4718 punch from 0x52d4a00 to 0x52db000, (0x6600 bytes) 2024-05-08T08:54:01.730 INFO:teuthology.orchestra.run.smithi196.stdout:4722 read 0x1cc2e00 thru 0x1ccf4be (0xc6bf bytes) 2024-05-08T08:54:01.730 INFO:teuthology.orchestra.run.smithi196.stdout:4723 write 0xcda4200 thru 0xcda9cab (0x5aac bytes) 2024-05-08T08:54:01.730 INFO:teuthology.orchestra.run.smithi196.stdout:4726 write 0x34b2800 thru 0x34c05c6 (0xddc7 bytes) 2024-05-08T08:54:01.755 INFO:teuthology.orchestra.run.smithi196.stdout:4728 write 0xeb0d400 thru 0xeb163fd (0x8ffe bytes) 2024-05-08T08:54:01.757 INFO:teuthology.orchestra.run.smithi196.stdout:4729 trunc from 0xeb16400 to 0xd5c7400 2024-05-08T08:54:01.783 INFO:teuthology.orchestra.run.smithi196.stdout:4730 read 0x1e36800 thru 0x1e446c9 (0xdeca bytes) 2024-05-08T08:54:01.795 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:01.798+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:01.804 INFO:teuthology.orchestra.run.smithi196.stdout:4735 trunc from 0xd5c7400 to 0x16b2400 2024-05-08T08:54:01.839 INFO:teuthology.orchestra.run.smithi196.stdout:4736 write 0xa4e0000 thru 0xa4ec375 (0xc376 bytes) 2024-05-08T08:54:01.840 INFO:teuthology.orchestra.run.smithi196.stdout:4738 read 0x54dc200 thru 0x54e9e09 (0xdc0a bytes) 2024-05-08T08:54:01.853 INFO:teuthology.orchestra.run.smithi196.stdout:4739 write 0xa8bf800 thru 0xa8cb22e (0xba2f bytes) 2024-05-08T08:54:01.853 INFO:teuthology.orchestra.run.smithi196.stdout:4740 clone 36 order 21 su 32768 sc 5 2024-05-08T08:54:01.855 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:01.855 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:01.866 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:54:01.896 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:01.899+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:01.975 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:01.975 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:01.984 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:54:01.997 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:02.000+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:02.090 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:02.090 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:02.098 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:02.101+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:02.099 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:54:02.199 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:02.202+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:02.244 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:02.244 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:02.252 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:54:02.300 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:02.303+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:02.357 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:02.357 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:02.366 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:54:02.410 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:02.413+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:02.471 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:02.471 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:02.479 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:54:02.503 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:02.505+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:02.604 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:02.607+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:02.623 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:02.623 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:02.632 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:54:02.681 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-05-08T08:54:02.716 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:02.719+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:02.747 INFO:teuthology.orchestra.run.smithi196.stdout:truncating image image_client.0-clone35 from 0xa8cb400 (overlap 0x7ef800) to 0x3f7c00 2024-05-08T08:54:02.751 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T08:54:02.789 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:02.789 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:02.799 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:54:02.840 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:02.843+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:02.885 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:54:02.885 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:54:02.904 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:54:02.904 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:54:02.910 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:02.913+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:02.934 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:54:02.934 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:54:02.949 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:02.949 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:02.951 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:54:02.951 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:54:02.958 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:54:02.969 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:54:02.969 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:54:02.986 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:54:02.986 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:54:03.012 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:03.014+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:03.109 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:03.109 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:54:03.109 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:54:03.109 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:54:03.109 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:03.112 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:03.115+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:03.115 INFO:teuthology.orchestra.run.smithi196.stdout:checking clone #34, image image_client.0-clone34 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent35 2024-05-08T08:54:03.118 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:54:03.172 INFO:teuthology.orchestra.run.smithi196.stdout:4741 read 0x91ee400 thru 0x91f82da (0x9edb bytes) 2024-05-08T08:54:03.172 INFO:teuthology.orchestra.run.smithi196.stdout:4742 read 0x2497e00 thru 0x249d22f (0x5430 bytes) 2024-05-08T08:54:03.172 INFO:teuthology.orchestra.run.smithi196.stdout:4753 write 0x9ea9600 thru 0x9ead8a5 (0x42a6 bytes) 2024-05-08T08:54:03.173 INFO:teuthology.orchestra.run.smithi196.stdout:4754 clone 37 order 21 su 131072 sc 9 2024-05-08T08:54:03.213 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:03.216+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:03.266 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:03.266 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:54:03.266 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:54:03.267 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:54:03.267 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.267 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6690 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1093632~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:03.267 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:54:03.267 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 91.553208613999999, 2024-05-08T08:54:03.267 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052462754, 2024-05-08T08:54:03.267 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:03.267 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:03.267 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:03.267 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:03.267 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:03.267 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:03.267 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6690 2024-05-08T08:54:03.268 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.268 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:03.268 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.268 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:03.268 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:54:03.268 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:03.268 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.268 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.268 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:03.268 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:54:03.268 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:03.268 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.268 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.268 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:03.268 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715346+0000", 2024-05-08T08:54:03.269 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.84e-07 2024-05-08T08:54:03.269 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.269 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.269 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:03.269 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715348+0000", 2024-05-08T08:54:03.269 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.114e-06 2024-05-08T08:54:03.269 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.269 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.269 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:03.269 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715353+0000", 2024-05-08T08:54:03.269 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5720000000000002e-06 2024-05-08T08:54:03.269 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.269 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.269 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:03.269 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715358+0000", 2024-05-08T08:54:03.269 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7140000000000001e-06 2024-05-08T08:54:03.270 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.270 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.270 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:03.270 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767638+0000", 2024-05-08T08:54:03.270 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052279562000000002 2024-05-08T08:54:03.270 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.270 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.270 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:03.270 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767713+0000", 2024-05-08T08:54:03.270 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5652e-05 2024-05-08T08:54:03.270 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.270 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.270 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:03.270 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767808+0000", 2024-05-08T08:54:03.271 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4755999999999999e-05 2024-05-08T08:54:03.271 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:03.271 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:03.271 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:03.271 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.271 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.271 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6692 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1110016~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:03.271 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:54:03.271 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 91.553189524000004, 2024-05-08T08:54:03.271 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052666495000000001, 2024-05-08T08:54:03.271 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:03.271 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:03.271 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:03.271 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:03.271 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:03.271 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:03.272 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6692 2024-05-08T08:54:03.272 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.272 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:03.272 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.272 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:03.272 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:54:03.272 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:03.272 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.272 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.272 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:03.272 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:54:03.272 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:03.272 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.272 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.272 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:03.272 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:54:03.273 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8599999999999999e-07 2024-05-08T08:54:03.273 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.273 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.273 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:03.273 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715372+0000", 2024-05-08T08:54:03.273 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0990000000000001e-06 2024-05-08T08:54:03.273 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.273 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.273 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:03.273 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715378+0000", 2024-05-08T08:54:03.273 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0680000000000002e-06 2024-05-08T08:54:03.273 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.273 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.273 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:03.273 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715383+0000", 2024-05-08T08:54:03.273 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8740000000000003e-06 2024-05-08T08:54:03.274 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.274 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.274 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:03.274 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767848+0000", 2024-05-08T08:54:03.274 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052464668999999998 2024-05-08T08:54:03.274 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.274 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.274 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:03.274 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767927+0000", 2024-05-08T08:54:03.274 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9228999999999996e-05 2024-05-08T08:54:03.274 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.274 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.274 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:03.274 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768031+0000", 2024-05-08T08:54:03.274 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010417 2024-05-08T08:54:03.274 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:03.275 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:03.275 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:03.275 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.275 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.275 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6694 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1126400~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:03.275 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:54:03.275 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 91.553164627000001, 2024-05-08T08:54:03.275 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052848829, 2024-05-08T08:54:03.275 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:03.275 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:03.275 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:03.275 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:03.275 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:03.275 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:03.275 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6694 2024-05-08T08:54:03.275 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.275 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:03.276 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.276 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:03.276 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:54:03.276 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:03.276 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.276 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.276 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:03.276 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:54:03.276 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:03.276 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.276 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.276 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:03.276 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715390+0000", 2024-05-08T08:54:03.276 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9900000000000001e-07 2024-05-08T08:54:03.276 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.276 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.276 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:03.277 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715392+0000", 2024-05-08T08:54:03.277 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8279999999999999e-06 2024-05-08T08:54:03.277 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.277 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.277 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:03.277 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715398+0000", 2024-05-08T08:54:03.277 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7619999999999998e-06 2024-05-08T08:54:03.277 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.277 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.277 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:03.277 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715403+0000", 2024-05-08T08:54:03.278 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7790000000000003e-06 2024-05-08T08:54:03.278 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.278 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.278 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:03.278 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768067+0000", 2024-05-08T08:54:03.278 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052663785999999997 2024-05-08T08:54:03.278 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.278 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.278 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:03.278 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768142+0000", 2024-05-08T08:54:03.278 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5018999999999994e-05 2024-05-08T08:54:03.278 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.278 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.278 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:03.279 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768238+0000", 2024-05-08T08:54:03.279 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6255999999999995e-05 2024-05-08T08:54:03.279 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:03.279 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:03.279 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:03.279 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.279 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.279 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6696 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1142784~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:03.279 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:54:03.279 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 91.553145119000007, 2024-05-08T08:54:03.279 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.053051057999999998, 2024-05-08T08:54:03.279 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:03.279 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:03.279 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:03.279 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:03.279 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:03.280 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:03.280 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6696 2024-05-08T08:54:03.280 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.280 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:03.280 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.280 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:03.280 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:54:03.280 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:03.280 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.280 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.280 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:03.280 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:54:03.280 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:03.280 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.280 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.280 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:03.281 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:54:03.281 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-05-08T08:54:03.281 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.281 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.281 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:03.281 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715411+0000", 2024-05-08T08:54:03.281 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9630000000000001e-06 2024-05-08T08:54:03.281 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.281 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.281 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:03.281 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715417+0000", 2024-05-08T08:54:03.281 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7250000000000002e-06 2024-05-08T08:54:03.281 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.281 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.281 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:03.281 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715421+0000", 2024-05-08T08:54:03.282 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4089999999999996e-06 2024-05-08T08:54:03.282 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.282 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.282 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:03.282 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768277+0000", 2024-05-08T08:54:03.282 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052855293999999997 2024-05-08T08:54:03.282 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.282 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.282 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:03.282 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768358+0000", 2024-05-08T08:54:03.282 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1720999999999999e-05 2024-05-08T08:54:03.282 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.282 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.282 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:03.282 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768460+0000", 2024-05-08T08:54:03.282 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010154600000000001 2024-05-08T08:54:03.283 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:03.283 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:03.283 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:03.283 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.283 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.283 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6698 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1159168~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:03.283 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:54:03.283 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 91.553126251999998, 2024-05-08T08:54:03.283 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.053245726, 2024-05-08T08:54:03.283 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:03.283 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:03.283 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:03.283 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:03.283 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:03.283 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:03.283 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6698 2024-05-08T08:54:03.284 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.284 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:03.284 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.284 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:03.284 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:54:03.284 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:03.284 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.284 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.284 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:03.284 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:54:03.284 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:03.284 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.284 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.284 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:03.284 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716936+0000", 2024-05-08T08:54:03.285 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001508665 2024-05-08T08:54:03.285 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.285 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.285 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:03.285 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716941+0000", 2024-05-08T08:54:03.285 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1910000000000002e-06 2024-05-08T08:54:03.285 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.285 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.285 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:03.285 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716950+0000", 2024-05-08T08:54:03.285 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1900000000000001e-06 2024-05-08T08:54:03.285 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.285 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.285 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:03.285 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716958+0000", 2024-05-08T08:54:03.286 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8019999999999992e-06 2024-05-08T08:54:03.286 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.286 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.286 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:03.286 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768499+0000", 2024-05-08T08:54:03.286 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051540926000000001 2024-05-08T08:54:03.286 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.286 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.286 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:03.286 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768574+0000", 2024-05-08T08:54:03.286 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4988999999999996e-05 2024-05-08T08:54:03.286 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.286 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.286 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:03.287 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768673+0000", 2024-05-08T08:54:03.287 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9963000000000005e-05 2024-05-08T08:54:03.287 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:03.287 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:03.287 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:03.287 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.287 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.287 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6706 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1224704~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:03.287 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:54:03.287 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 91.551520944999993, 2024-05-08T08:54:03.287 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052435598, 2024-05-08T08:54:03.287 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:03.287 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:03.287 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:03.287 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:03.288 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:03.288 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:03.288 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6706 2024-05-08T08:54:03.288 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.288 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:03.288 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.288 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:03.288 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:54:03.288 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:03.288 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.288 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.288 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:03.288 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:54:03.288 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:03.288 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.288 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.289 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:03.289 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:54:03.289 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:54:03.289 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.289 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.289 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:03.289 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717035+0000", 2024-05-08T08:54:03.289 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9290000000000001e-06 2024-05-08T08:54:03.289 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.289 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.289 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:03.289 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717041+0000", 2024-05-08T08:54:03.289 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6720000000000001e-06 2024-05-08T08:54:03.289 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.289 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.290 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:03.290 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717046+0000", 2024-05-08T08:54:03.290 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5650000000000003e-06 2024-05-08T08:54:03.290 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.290 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.290 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:03.290 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769311+0000", 2024-05-08T08:54:03.290 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052265645999999999 2024-05-08T08:54:03.290 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.290 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.290 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:03.290 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769379+0000", 2024-05-08T08:54:03.290 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7391000000000006e-05 2024-05-08T08:54:03.290 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.291 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.291 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:03.291 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769469+0000", 2024-05-08T08:54:03.291 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0025999999999995e-05 2024-05-08T08:54:03.291 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:03.291 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:03.291 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:03.291 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.291 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.291 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6708 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1241088~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:03.291 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:54:03.291 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 91.551502841000001, 2024-05-08T08:54:03.291 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052640552, 2024-05-08T08:54:03.291 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:03.291 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:03.292 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:03.292 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:03.292 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:03.292 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:03.292 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6708 2024-05-08T08:54:03.292 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.292 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:03.292 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.292 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:03.292 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:54:03.292 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:03.292 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.292 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.292 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:03.292 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:54:03.293 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:03.293 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.293 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.293 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:03.293 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717052+0000", 2024-05-08T08:54:03.293 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:54:03.293 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.293 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.293 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:03.293 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717053+0000", 2024-05-08T08:54:03.293 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8330000000000001e-06 2024-05-08T08:54:03.293 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.293 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.293 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:03.293 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717058+0000", 2024-05-08T08:54:03.294 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.933e-06 2024-05-08T08:54:03.294 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.294 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.294 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:03.294 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717063+0000", 2024-05-08T08:54:03.294 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2180000000000001e-06 2024-05-08T08:54:03.294 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.294 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.294 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:03.294 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769507+0000", 2024-05-08T08:54:03.294 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052444866999999999 2024-05-08T08:54:03.294 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.294 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.294 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:03.295 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769589+0000", 2024-05-08T08:54:03.295 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2039000000000005e-05 2024-05-08T08:54:03.295 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.295 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.295 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:03.295 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769692+0000", 2024-05-08T08:54:03.295 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102252 2024-05-08T08:54:03.295 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:03.295 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:03.295 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:03.295 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.295 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.295 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6710 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1257472~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:03.295 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:54:03.295 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 91.551484817000002, 2024-05-08T08:54:03.296 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.05283057, 2024-05-08T08:54:03.296 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:03.296 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:03.296 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:03.296 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:03.296 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:03.296 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:03.296 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6710 2024-05-08T08:54:03.296 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.296 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:03.296 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.296 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:03.296 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:54:03.296 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:03.296 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.296 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.296 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:03.297 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:54:03.297 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:03.297 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.297 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.297 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:03.297 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718951+0000", 2024-05-08T08:54:03.297 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0018817809999999999 2024-05-08T08:54:03.297 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.297 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.297 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:03.297 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718956+0000", 2024-05-08T08:54:03.297 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5399999999999997e-06 2024-05-08T08:54:03.297 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.297 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.297 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:03.298 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718964+0000", 2024-05-08T08:54:03.298 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6629999999999999e-06 2024-05-08T08:54:03.298 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.298 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.298 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:03.298 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718972+0000", 2024-05-08T08:54:03.298 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5229999999999998e-06 2024-05-08T08:54:03.298 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.298 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.298 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:03.298 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769729+0000", 2024-05-08T08:54:03.298 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050756959999999997 2024-05-08T08:54:03.298 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.298 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.298 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:03.299 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769803+0000", 2024-05-08T08:54:03.299 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4538999999999999e-05 2024-05-08T08:54:03.299 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.299 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.299 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:03.299 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769900+0000", 2024-05-08T08:54:03.299 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6564000000000006e-05 2024-05-08T08:54:03.299 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:03.299 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:03.299 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:03.299 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.299 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.299 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6726 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1388544~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:03.299 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:54:03.300 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 91.549410045000002, 2024-05-08T08:54:03.300 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052451506000000002, 2024-05-08T08:54:03.300 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:03.300 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:03.300 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:03.300 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:03.300 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:03.300 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:03.300 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6726 2024-05-08T08:54:03.300 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.300 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:03.300 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.300 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:03.300 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:54:03.300 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:03.301 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.301 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.301 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:03.301 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:54:03.301 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:03.301 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.301 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.301 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:03.301 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719200+0000", 2024-05-08T08:54:03.301 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5865999999999997e-05 2024-05-08T08:54:03.301 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.301 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.301 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:03.301 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719204+0000", 2024-05-08T08:54:03.301 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8630000000000001e-06 2024-05-08T08:54:03.302 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.302 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.302 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:03.302 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719215+0000", 2024-05-08T08:54:03.302 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1590000000000001e-05 2024-05-08T08:54:03.302 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.302 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.302 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:03.302 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719220+0000", 2024-05-08T08:54:03.302 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1619999999999999e-06 2024-05-08T08:54:03.302 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.302 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.302 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:03.302 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771425+0000", 2024-05-08T08:54:03.303 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052204249000000001 2024-05-08T08:54:03.303 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.303 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.303 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:03.303 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771498+0000", 2024-05-08T08:54:03.303 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3782999999999995e-05 2024-05-08T08:54:03.303 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.303 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.303 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:03.303 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771595+0000", 2024-05-08T08:54:03.303 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6992999999999998e-05 2024-05-08T08:54:03.303 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:03.303 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:03.303 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:03.303 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.303 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.304 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6728 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1404928~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:03.304 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:54:03.304 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 91.54932264, 2024-05-08T08:54:03.304 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052590747, 2024-05-08T08:54:03.304 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:03.304 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:03.304 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:03.304 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:03.304 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:03.304 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:03.304 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6728 2024-05-08T08:54:03.304 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.304 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:03.304 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.304 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:03.304 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:54:03.305 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:03.305 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.305 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.305 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:03.305 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:54:03.305 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:03.305 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.305 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.305 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:03.305 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719232+0000", 2024-05-08T08:54:03.305 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4000000000000001e-07 2024-05-08T08:54:03.305 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.305 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.305 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:03.305 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719234+0000", 2024-05-08T08:54:03.305 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4499999999999998e-06 2024-05-08T08:54:03.306 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.306 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.306 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:03.306 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719240+0000", 2024-05-08T08:54:03.306 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5720000000000002e-06 2024-05-08T08:54:03.306 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.306 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.306 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:03.306 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719245+0000", 2024-05-08T08:54:03.306 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8690000000000003e-06 2024-05-08T08:54:03.306 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.306 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.306 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:03.306 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771636+0000", 2024-05-08T08:54:03.306 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052390979999999997 2024-05-08T08:54:03.307 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.307 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.307 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:03.307 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771717+0000", 2024-05-08T08:54:03.307 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1291999999999993e-05 2024-05-08T08:54:03.307 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.307 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.307 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:03.307 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771822+0000", 2024-05-08T08:54:03.307 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104944 2024-05-08T08:54:03.307 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:03.307 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:03.307 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:03.307 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.307 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.308 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6730 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1421312~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:03.308 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:54:03.308 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 91.549302397999995, 2024-05-08T08:54:03.308 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052770477000000003, 2024-05-08T08:54:03.308 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:03.308 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:03.308 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:03.308 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:03.308 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:03.308 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:03.308 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6730 2024-05-08T08:54:03.308 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.308 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:03.308 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.308 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:03.309 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:54:03.309 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:03.309 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.309 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.309 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:03.309 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:54:03.309 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:03.309 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.309 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.309 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:03.309 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:54:03.309 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9000000000000002e-07 2024-05-08T08:54:03.309 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.309 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.310 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:03.310 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719254+0000", 2024-05-08T08:54:03.310 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4099999999999998e-06 2024-05-08T08:54:03.310 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.310 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.310 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:03.310 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719263+0000", 2024-05-08T08:54:03.310 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4109999999999996e-06 2024-05-08T08:54:03.310 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.310 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.310 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:03.310 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719268+0000", 2024-05-08T08:54:03.310 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2669999999999997e-06 2024-05-08T08:54:03.310 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.310 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.311 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:03.311 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771846+0000", 2024-05-08T08:54:03.311 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052577677000000003 2024-05-08T08:54:03.311 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.311 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.311 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:03.311 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771920+0000", 2024-05-08T08:54:03.311 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4460000000000002e-05 2024-05-08T08:54:03.311 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.311 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.311 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:03.311 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772022+0000", 2024-05-08T08:54:03.311 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010186199999999999 2024-05-08T08:54:03.311 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:03.311 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:03.312 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:03.312 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.312 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.312 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6732 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1437696~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:03.312 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:54:03.312 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 91.549277591000006, 2024-05-08T08:54:03.312 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052905575000000003, 2024-05-08T08:54:03.312 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:03.312 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:03.312 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:03.312 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:03.312 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:03.312 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:03.312 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6732 2024-05-08T08:54:03.313 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.313 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:03.313 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.313 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:03.313 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:54:03.313 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:03.313 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.313 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.313 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:03.313 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:54:03.313 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:03.313 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.313 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.313 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:03.313 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721392+0000", 2024-05-08T08:54:03.314 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002115287 2024-05-08T08:54:03.314 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.314 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.314 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:03.314 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721396+0000", 2024-05-08T08:54:03.314 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1679999999999997e-06 2024-05-08T08:54:03.314 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.314 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.314 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:03.314 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721404+0000", 2024-05-08T08:54:03.314 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4409999999999993e-06 2024-05-08T08:54:03.314 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.314 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.314 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:03.315 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721410+0000", 2024-05-08T08:54:03.315 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9150000000000001e-06 2024-05-08T08:54:03.315 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.315 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.315 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:03.315 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772061+0000", 2024-05-08T08:54:03.315 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050651068 2024-05-08T08:54:03.315 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.315 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.315 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:03.315 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772127+0000", 2024-05-08T08:54:03.315 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5424000000000002e-05 2024-05-08T08:54:03.315 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.315 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.315 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:03.315 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772182+0000", 2024-05-08T08:54:03.316 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5272000000000002e-05 2024-05-08T08:54:03.316 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:03.316 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:03.316 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:03.316 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.316 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.316 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6749 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1585152~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:03.316 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:54:03.316 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 91.546976821000001, 2024-05-08T08:54:03.316 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052350264000000001, 2024-05-08T08:54:03.316 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:03.316 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:03.316 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:03.316 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:03.316 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:03.317 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:03.317 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6749 2024-05-08T08:54:03.317 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.317 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:03.317 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.317 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:03.317 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:54:03.317 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:03.317 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.317 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.317 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:03.317 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:54:03.317 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:03.317 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.317 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.317 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:03.318 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721578+0000", 2024-05-08T08:54:03.318 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0399999999999996e-07 2024-05-08T08:54:03.318 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.318 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.318 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:03.318 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721580+0000", 2024-05-08T08:54:03.318 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5450000000000002e-06 2024-05-08T08:54:03.318 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.318 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.318 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:03.318 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721587+0000", 2024-05-08T08:54:03.318 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0190000000000001e-06 2024-05-08T08:54:03.318 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.319 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.319 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:03.319 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721593+0000", 2024-05-08T08:54:03.319 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5659999999999997e-06 2024-05-08T08:54:03.319 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.319 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.319 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:03.319 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773759+0000", 2024-05-08T08:54:03.319 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052166553999999997 2024-05-08T08:54:03.319 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.319 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.319 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:03.319 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773833+0000", 2024-05-08T08:54:03.319 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3913999999999997e-05 2024-05-08T08:54:03.319 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.320 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.320 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:03.320 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773927+0000", 2024-05-08T08:54:03.320 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4061999999999995e-05 2024-05-08T08:54:03.320 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:03.320 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:03.320 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:03.320 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.320 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.320 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6751 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1601536~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:03.320 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:54:03.320 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 91.546953541999997, 2024-05-08T08:54:03.320 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052461302000000001, 2024-05-08T08:54:03.320 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:03.321 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:03.321 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:03.321 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:03.321 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:03.321 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:03.321 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6751 2024-05-08T08:54:03.321 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.321 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:03.321 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.321 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:03.321 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:54:03.321 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:03.321 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.321 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.321 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:03.322 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:54:03.322 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:03.322 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.322 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.322 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:03.322 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722939+0000", 2024-05-08T08:54:03.322 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001338601 2024-05-08T08:54:03.322 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.322 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.322 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:03.322 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722944+0000", 2024-05-08T08:54:03.322 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4469999999999999e-06 2024-05-08T08:54:03.322 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.322 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.323 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:03.323 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722955+0000", 2024-05-08T08:54:03.323 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1892999999999999e-05 2024-05-08T08:54:03.323 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.323 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.323 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:03.323 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722963+0000", 2024-05-08T08:54:03.323 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1080000000000004e-06 2024-05-08T08:54:03.323 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.323 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.323 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:03.323 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773964+0000", 2024-05-08T08:54:03.323 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051001022 2024-05-08T08:54:03.323 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.323 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.324 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:03.324 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.774006+0000", 2024-05-08T08:54:03.324 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2729999999999999e-05 2024-05-08T08:54:03.324 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.324 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.324 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:03.324 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.774062+0000", 2024-05-08T08:54:03.324 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5501000000000003e-05 2024-05-08T08:54:03.324 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:03.324 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:03.324 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:03.324 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.324 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.324 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6760 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1683456~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:03.324 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:54:03.325 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 91.545444165999996, 2024-05-08T08:54:03.325 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052571312000000002, 2024-05-08T08:54:03.325 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:03.325 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:03.325 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:03.325 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:03.325 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:03.325 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:03.325 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6760 2024-05-08T08:54:03.325 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.325 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:03.325 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.325 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:03.325 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:54:03.326 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:03.326 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.326 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.326 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:03.326 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:54:03.326 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:03.326 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.326 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.326 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:03.326 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:54:03.326 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2100000000000002e-07 2024-05-08T08:54:03.326 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.326 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.326 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:03.326 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723113+0000", 2024-05-08T08:54:03.327 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3149999999999999e-06 2024-05-08T08:54:03.327 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.327 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.327 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:03.327 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723118+0000", 2024-05-08T08:54:03.327 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8860000000000001e-06 2024-05-08T08:54:03.327 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.327 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.327 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:03.327 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723123+0000", 2024-05-08T08:54:03.327 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6770000000000003e-06 2024-05-08T08:54:03.327 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.327 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.327 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:03.328 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775414+0000", 2024-05-08T08:54:03.328 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052290692999999999 2024-05-08T08:54:03.328 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.328 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.328 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:03.328 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775459+0000", 2024-05-08T08:54:03.328 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5639999999999997e-05 2024-05-08T08:54:03.328 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.328 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.328 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:03.328 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775681+0000", 2024-05-08T08:54:03.328 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00022168000000000001 2024-05-08T08:54:03.328 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:03.328 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:03.328 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:03.328 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.329 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.329 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6761 3.16 3:6bd6d6da:::rbd_data.1140782590a5.0000000000000015:6 [read 1095680~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:03.329 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:54:03.329 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 91.545424945999997, 2024-05-08T08:54:03.329 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052751275, 2024-05-08T08:54:03.329 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:03.329 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:03.329 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:03.329 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:03.329 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:03.329 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:03.329 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6761 2024-05-08T08:54:03.329 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.329 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:03.329 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.330 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:03.330 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:54:03.330 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:03.330 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.330 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.330 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:03.330 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:54:03.330 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:03.330 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.330 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.330 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:03.330 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724939+0000", 2024-05-08T08:54:03.330 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001809817 2024-05-08T08:54:03.330 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.330 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.331 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:03.331 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724943+0000", 2024-05-08T08:54:03.331 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4920000000000004e-06 2024-05-08T08:54:03.331 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.331 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.331 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:03.331 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724956+0000", 2024-05-08T08:54:03.331 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2427e-05 2024-05-08T08:54:03.331 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.331 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.331 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:03.331 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724963+0000", 2024-05-08T08:54:03.331 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2559999999999996e-06 2024-05-08T08:54:03.331 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.332 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.332 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:03.332 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775711+0000", 2024-05-08T08:54:03.332 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050747621 2024-05-08T08:54:03.332 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.332 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.332 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:03.332 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775785+0000", 2024-05-08T08:54:03.332 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4634000000000004e-05 2024-05-08T08:54:03.332 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.332 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.332 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:03.332 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775880+0000", 2024-05-08T08:54:03.332 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5027999999999999e-05 2024-05-08T08:54:03.332 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:03.332 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:03.333 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:03.333 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.333 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.333 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6770 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1765376~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:03.333 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:54:03.333 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 91.543393851000005, 2024-05-08T08:54:03.333 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052339296, 2024-05-08T08:54:03.333 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:03.333 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:03.333 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:03.333 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:03.333 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:03.333 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:03.333 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6770 2024-05-08T08:54:03.333 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.333 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:03.334 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.334 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:03.334 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:54:03.334 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:03.334 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.334 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.334 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:03.334 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:54:03.334 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:03.334 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.334 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.334 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:03.334 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725161+0000", 2024-05-08T08:54:03.334 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-05-08T08:54:03.335 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.335 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.335 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:03.335 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725163+0000", 2024-05-08T08:54:03.335 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4370000000000001e-06 2024-05-08T08:54:03.335 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.335 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.335 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:03.335 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725169+0000", 2024-05-08T08:54:03.335 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8409999999999996e-06 2024-05-08T08:54:03.335 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.335 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.335 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:03.335 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725174+0000", 2024-05-08T08:54:03.336 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7249999999999997e-06 2024-05-08T08:54:03.336 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.336 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.336 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:03.336 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777343+0000", 2024-05-08T08:54:03.336 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052169633999999999 2024-05-08T08:54:03.336 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.336 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.336 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:03.336 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777409+0000", 2024-05-08T08:54:03.336 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6033999999999998e-05 2024-05-08T08:54:03.336 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.336 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.336 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:03.336 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777499+0000", 2024-05-08T08:54:03.336 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0224999999999998e-05 2024-05-08T08:54:03.337 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:03.337 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:03.337 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:03.337 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.337 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.337 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6771 3.16 3:6bd6d6da:::rbd_data.1140782590a5.0000000000000015:6 [read 1177600~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:03.337 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:54:03.337 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 91.543374033999996, 2024-05-08T08:54:03.337 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052438604999999999, 2024-05-08T08:54:03.337 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:03.337 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:03.337 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:03.337 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:03.337 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:03.337 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:03.338 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6771 2024-05-08T08:54:03.338 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.338 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:03.338 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.338 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:03.338 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:54:03.338 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:03.338 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.338 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.338 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:03.338 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:54:03.338 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:03.338 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.338 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.339 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:03.339 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:54:03.339 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:54:03.339 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.339 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.339 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:03.339 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725188+0000", 2024-05-08T08:54:03.339 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1579999999999999e-06 2024-05-08T08:54:03.339 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.339 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.339 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:03.339 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725195+0000", 2024-05-08T08:54:03.339 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8909999999999995e-06 2024-05-08T08:54:03.339 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.339 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.339 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:03.340 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725200+0000", 2024-05-08T08:54:03.340 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9810000000000003e-06 2024-05-08T08:54:03.340 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.340 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.340 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:03.340 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777527+0000", 2024-05-08T08:54:03.340 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052326582000000003 2024-05-08T08:54:03.340 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.340 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.340 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:03.340 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777566+0000", 2024-05-08T08:54:03.340 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9447999999999999e-05 2024-05-08T08:54:03.340 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.340 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.340 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:03.341 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777619+0000", 2024-05-08T08:54:03.341 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2135000000000001e-05 2024-05-08T08:54:03.341 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:03.341 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:03.341 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:03.341 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.341 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.341 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6772 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1781760~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:03.341 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:54:03.341 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 91.543345977000001, 2024-05-08T08:54:03.341 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052601799999999997, 2024-05-08T08:54:03.341 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:03.341 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:03.341 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:03.341 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:03.341 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:03.341 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:03.342 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6772 2024-05-08T08:54:03.342 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.342 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:03.342 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.342 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:03.342 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:54:03.342 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:03.342 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.342 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.342 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:03.342 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:54:03.342 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:03.342 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.342 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.342 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:03.343 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726939+0000", 2024-05-08T08:54:03.343 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001730493 2024-05-08T08:54:03.343 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.343 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.343 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:03.343 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726943+0000", 2024-05-08T08:54:03.343 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7330000000000003e-06 2024-05-08T08:54:03.343 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.343 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.343 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:03.343 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726954+0000", 2024-05-08T08:54:03.343 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1107000000000001e-05 2024-05-08T08:54:03.343 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.343 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.343 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:03.343 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726962+0000", 2024-05-08T08:54:03.344 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.503e-06 2024-05-08T08:54:03.344 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.344 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.344 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:03.344 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777653+0000", 2024-05-08T08:54:03.344 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050691542999999999 2024-05-08T08:54:03.344 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.344 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.344 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:03.344 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777719+0000", 2024-05-08T08:54:03.344 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5228000000000006e-05 2024-05-08T08:54:03.344 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.344 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.344 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:03.344 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777810+0000", 2024-05-08T08:54:03.345 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1193000000000006e-05 2024-05-08T08:54:03.345 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:03.345 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:03.345 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:03.345 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.345 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6782 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1880064~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:03.345 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:54:03.345 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 91.54137747, 2024-05-08T08:54:03.345 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052326492000000002, 2024-05-08T08:54:03.345 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:03.345 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:03.345 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:03.345 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:03.345 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:03.346 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:03.346 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6782 2024-05-08T08:54:03.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.346 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:03.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.346 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:03.346 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:54:03.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:03.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.346 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:03.346 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:54:03.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:03.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.347 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.347 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:03.347 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728939+0000", 2024-05-08T08:54:03.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001762719 2024-05-08T08:54:03.347 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.347 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.347 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:03.347 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728944+0000", 2024-05-08T08:54:03.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.746e-06 2024-05-08T08:54:03.347 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.347 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.347 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:03.347 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728955+0000", 2024-05-08T08:54:03.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1043000000000001e-05 2024-05-08T08:54:03.348 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.348 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.348 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:03.348 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728963+0000", 2024-05-08T08:54:03.348 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5809999999999999e-06 2024-05-08T08:54:03.348 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.348 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.348 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:03.348 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779320+0000", 2024-05-08T08:54:03.348 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050357290999999998 2024-05-08T08:54:03.348 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.348 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.348 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:03.348 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779401+0000", 2024-05-08T08:54:03.348 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0574000000000005e-05 2024-05-08T08:54:03.348 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:03.349 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:03.349 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:03.349 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779503+0000", 2024-05-08T08:54:03.349 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102538 2024-05-08T08:54:03.349 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:03.349 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:03.349 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:03.349 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:03.349 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:03.349 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:03.351 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:03.353+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:03.351 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:54:03.451 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:03.454+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:03.462 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:03.462 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:54:03.462 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:54:03.462 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:03.471 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:54:03.555 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:03.558+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:03.621 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:03.621 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:54:03.621 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:54:03.621 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:54:03.621 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:03.629 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:54:03.653 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:03.655+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:03.653 INFO:teuthology.orchestra.run.smithi059.stderr:nodeep-scrub is set 2024-05-08T08:54:03.753 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:03.756+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:03.765 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:03.765 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:54:03.765 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:54:03.765 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:54:03.765 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.766 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4621.0:9089 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 23339520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:54:03.766 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:54:03.766 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 91.274577386000004, 2024-05-08T08:54:03.766 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.13814907200000001, 2024-05-08T08:54:03.766 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:03.766 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:03.766 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:54:03.766 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:03.766 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4621", 2024-05-08T08:54:03.766 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:03.766 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9089 2024-05-08T08:54:03.766 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.766 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:03.766 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.767 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:03.767 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:54:03.767 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:03.767 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.767 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.767 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:03.767 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:54:03.767 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:03.767 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.767 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.767 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:03.767 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493005+0000", 2024-05-08T08:54:03.767 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1700000000000001e-06 2024-05-08T08:54:03.768 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.768 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.768 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:03.768 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.513963+0000", 2024-05-08T08:54:03.768 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.020957581999999999 2024-05-08T08:54:03.768 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.768 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.768 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:03.768 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.513995+0000", 2024-05-08T08:54:03.768 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1993999999999999e-05 2024-05-08T08:54:03.768 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.768 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.768 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:03.768 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514007+0000", 2024-05-08T08:54:03.768 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1581e-05 2024-05-08T08:54:03.769 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.769 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.769 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:03.769 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514122+0000", 2024-05-08T08:54:03.769 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000115282 2024-05-08T08:54:03.769 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.769 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.769 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op must wait for map", 2024-05-08T08:54:03.769 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514128+0000", 2024-05-08T08:54:03.769 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9950000000000002e-06 2024-05-08T08:54:03.769 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.769 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.769 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:03.769 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.515800+0000", 2024-05-08T08:54:03.770 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001671886 2024-05-08T08:54:03.770 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.770 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.770 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:03.770 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.515893+0000", 2024-05-08T08:54:03.770 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3490000000000001e-05 2024-05-08T08:54:03.770 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.770 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.770 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 3", 2024-05-08T08:54:03.770 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.517647+0000", 2024-05-08T08:54:03.770 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0017535630000000001 2024-05-08T08:54:03.770 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.770 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.770 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:54:03.771 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.627578+0000", 2024-05-08T08:54:03.771 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.109931557 2024-05-08T08:54:03.771 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.771 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.771 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:54:03.771 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631055+0000", 2024-05-08T08:54:03.771 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0034761420000000002 2024-05-08T08:54:03.771 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.771 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.771 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:54:03.771 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631096+0000", 2024-05-08T08:54:03.771 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1059000000000001e-05 2024-05-08T08:54:03.771 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.771 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.772 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:03.772 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631151+0000", 2024-05-08T08:54:03.772 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5770999999999998e-05 2024-05-08T08:54:03.772 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:03.772 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:03.772 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:03.772 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.772 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.772 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1071 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11026432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:03.772 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:54:03.772 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 87.282127895000002, 2024-05-08T08:54:03.772 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070580037999999998, 2024-05-08T08:54:03.772 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:03.772 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:03.773 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:03.773 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:03.773 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:03.773 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:03.773 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1071 2024-05-08T08:54:03.773 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.773 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:03.773 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.773 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:03.773 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:54:03.773 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:03.773 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.773 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.773 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:03.773 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:54:03.774 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:03.774 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.774 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.774 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:03.774 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:54:03.774 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2800000000000002e-07 2024-05-08T08:54:03.774 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.774 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.774 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:03.774 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485454+0000", 2024-05-08T08:54:03.774 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2290000000000002e-06 2024-05-08T08:54:03.774 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.774 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.775 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:03.775 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485461+0000", 2024-05-08T08:54:03.775 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9920000000000001e-06 2024-05-08T08:54:03.775 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.775 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.775 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:03.775 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485467+0000", 2024-05-08T08:54:03.775 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3979999999999998e-06 2024-05-08T08:54:03.775 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.775 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.775 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:03.775 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555880+0000", 2024-05-08T08:54:03.775 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070413165 2024-05-08T08:54:03.775 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.775 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.775 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:03.775 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555926+0000", 2024-05-08T08:54:03.776 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6217000000000002e-05 2024-05-08T08:54:03.776 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.776 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.776 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:03.776 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556032+0000", 2024-05-08T08:54:03.776 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000105609 2024-05-08T08:54:03.776 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:03.776 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:03.776 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:03.776 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.776 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.776 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1073 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11067392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:03.776 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:54:03.776 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 87.282106674000005, 2024-05-08T08:54:03.776 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070821404000000004, 2024-05-08T08:54:03.776 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:03.777 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:03.777 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:03.777 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:03.777 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:03.777 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:03.777 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1073 2024-05-08T08:54:03.777 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.777 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:03.777 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.777 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:03.777 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:54:03.777 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:03.777 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.777 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.777 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:03.777 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:54:03.777 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:03.778 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.778 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.778 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:03.778 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:54:03.778 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.08e-07 2024-05-08T08:54:03.778 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.778 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.778 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:03.778 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485476+0000", 2024-05-08T08:54:03.778 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3929999999999998e-06 2024-05-08T08:54:03.778 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.778 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.778 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:03.778 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485485+0000", 2024-05-08T08:54:03.779 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1880000000000008e-06 2024-05-08T08:54:03.779 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.779 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.779 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:03.779 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485494+0000", 2024-05-08T08:54:03.779 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3109999999999995e-06 2024-05-08T08:54:03.779 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.779 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.779 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:03.779 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556082+0000", 2024-05-08T08:54:03.779 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070587426999999994 2024-05-08T08:54:03.779 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.779 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.779 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:03.779 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556177+0000", 2024-05-08T08:54:03.780 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5051999999999997e-05 2024-05-08T08:54:03.780 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.780 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.780 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:03.780 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556294+0000", 2024-05-08T08:54:03.780 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000117625 2024-05-08T08:54:03.780 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:03.780 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:03.780 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:03.780 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.780 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.780 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1075 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11108352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:03.780 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:54:03.780 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 87.282076097000001, 2024-05-08T08:54:03.780 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071045026999999997, 2024-05-08T08:54:03.780 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:03.781 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:03.781 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:03.781 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:03.781 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:03.781 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:03.781 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1075 2024-05-08T08:54:03.781 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.781 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:03.781 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.781 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:03.781 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:54:03.781 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:03.781 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.781 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.782 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:03.782 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:54:03.782 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:03.782 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.782 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.782 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:03.782 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:54:03.782 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.32e-07 2024-05-08T08:54:03.782 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.782 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.782 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:03.782 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485518+0000", 2024-05-08T08:54:03.782 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3769e-05 2024-05-08T08:54:03.782 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.783 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.783 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:03.783 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485527+0000", 2024-05-08T08:54:03.783 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6779999999999995e-06 2024-05-08T08:54:03.783 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.783 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.783 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:03.783 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485532+0000", 2024-05-08T08:54:03.783 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9830000000000004e-06 2024-05-08T08:54:03.783 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.783 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.783 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:03.783 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556340+0000", 2024-05-08T08:54:03.783 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070807743000000006 2024-05-08T08:54:03.784 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.784 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.784 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:03.784 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556435+0000", 2024-05-08T08:54:03.784 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4732e-05 2024-05-08T08:54:03.784 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.784 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.784 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:03.784 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556549+0000", 2024-05-08T08:54:03.784 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011369 2024-05-08T08:54:03.784 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:03.784 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:03.784 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:03.784 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.784 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.785 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1078 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11149312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:03.785 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:54:03.785 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 87.282035898000004, 2024-05-08T08:54:03.785 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071167276000000002, 2024-05-08T08:54:03.785 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:03.785 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:03.785 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:03.785 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:03.785 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:03.785 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:03.785 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1078 2024-05-08T08:54:03.785 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.785 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:03.785 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.785 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:03.786 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:54:03.786 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:03.786 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.786 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.786 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:03.786 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:54:03.786 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:03.786 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.786 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.786 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:03.786 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:54:03.786 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5600000000000001e-07 2024-05-08T08:54:03.786 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.786 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.787 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:03.787 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485547+0000", 2024-05-08T08:54:03.787 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3590000000000002e-06 2024-05-08T08:54:03.787 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.787 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.787 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:03.787 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485553+0000", 2024-05-08T08:54:03.787 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2770000000000002e-06 2024-05-08T08:54:03.787 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.787 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.787 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:03.787 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485558+0000", 2024-05-08T08:54:03.787 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.95e-06 2024-05-08T08:54:03.787 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.787 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.788 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:03.788 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556582+0000", 2024-05-08T08:54:03.788 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071024444000000006 2024-05-08T08:54:03.788 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.788 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.788 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:03.788 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556637+0000", 2024-05-08T08:54:03.788 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4672000000000001e-05 2024-05-08T08:54:03.788 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.788 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.788 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:03.788 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556711+0000", 2024-05-08T08:54:03.788 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4117999999999997e-05 2024-05-08T08:54:03.788 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:03.788 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:03.789 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:03.789 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.789 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.789 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1080 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11190272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:03.789 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:54:03.789 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 87.282015165000004, 2024-05-08T08:54:03.789 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071350721000000006, 2024-05-08T08:54:03.789 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:03.789 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:03.789 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:03.789 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:03.789 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:03.789 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:03.789 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1080 2024-05-08T08:54:03.789 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.790 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:03.790 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.790 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:03.790 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:54:03.790 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:03.790 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.790 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.790 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:03.790 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:54:03.790 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:03.790 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.790 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.790 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:03.790 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:54:03.790 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.03e-07 2024-05-08T08:54:03.791 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.791 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.791 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:03.791 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485567+0000", 2024-05-08T08:54:03.791 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1090000000000001e-06 2024-05-08T08:54:03.791 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.791 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.791 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:03.791 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485573+0000", 2024-05-08T08:54:03.791 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3049999999999998e-06 2024-05-08T08:54:03.791 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.791 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.791 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:03.791 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485578+0000", 2024-05-08T08:54:03.792 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0130000000000001e-06 2024-05-08T08:54:03.792 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.792 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.792 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:03.792 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556743+0000", 2024-05-08T08:54:03.792 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071164449000000005 2024-05-08T08:54:03.792 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.792 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.792 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:03.792 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556797+0000", 2024-05-08T08:54:03.792 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4589999999999997e-05 2024-05-08T08:54:03.792 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.792 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.792 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:03.792 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556915+0000", 2024-05-08T08:54:03.793 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000117852 2024-05-08T08:54:03.793 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:03.793 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:03.793 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:03.793 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.793 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.793 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1082 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11231232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:03.793 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:54:03.793 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 87.281994921999996, 2024-05-08T08:54:03.793 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071591396000000002, 2024-05-08T08:54:03.793 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:03.793 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:03.793 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:03.793 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:03.793 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:03.794 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:03.794 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1082 2024-05-08T08:54:03.794 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.794 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:03.794 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.794 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:03.794 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:54:03.794 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:03.794 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.794 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.794 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:03.794 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:54:03.794 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:03.795 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.795 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.795 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:03.795 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:54:03.795 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1800000000000001e-07 2024-05-08T08:54:03.795 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.795 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.795 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:03.795 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485587+0000", 2024-05-08T08:54:03.795 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2589999999999999e-06 2024-05-08T08:54:03.795 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.795 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.795 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:03.795 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485594+0000", 2024-05-08T08:54:03.795 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0789999999999997e-06 2024-05-08T08:54:03.796 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.796 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.796 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:03.796 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485598+0000", 2024-05-08T08:54:03.796 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7480000000000001e-06 2024-05-08T08:54:03.796 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.796 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.796 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:03.796 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556971+0000", 2024-05-08T08:54:03.796 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071372774999999999 2024-05-08T08:54:03.796 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.796 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.796 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:03.796 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557069+0000", 2024-05-08T08:54:03.797 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.7732999999999994e-05 2024-05-08T08:54:03.797 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.797 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.797 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:03.797 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557176+0000", 2024-05-08T08:54:03.797 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000107384 2024-05-08T08:54:03.797 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:03.797 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:03.797 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:03.797 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.797 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.797 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1084 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11272192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:03.797 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:54:03.797 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 87.281956283, 2024-05-08T08:54:03.797 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071808977999999996, 2024-05-08T08:54:03.797 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:03.797 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:03.798 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:03.798 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:03.798 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:03.798 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:03.798 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1084 2024-05-08T08:54:03.798 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.798 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:03.798 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.798 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:03.798 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:54:03.798 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:03.798 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.798 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.798 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:03.799 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:54:03.799 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:03.799 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.799 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.799 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:03.799 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485624+0000", 2024-05-08T08:54:03.799 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:54:03.799 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.799 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.799 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:03.799 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485626+0000", 2024-05-08T08:54:03.799 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9460000000000001e-06 2024-05-08T08:54:03.799 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.799 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.799 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:03.800 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485633+0000", 2024-05-08T08:54:03.800 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1149999999999996e-06 2024-05-08T08:54:03.800 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.800 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.800 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:03.800 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485638+0000", 2024-05-08T08:54:03.800 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5670000000000002e-06 2024-05-08T08:54:03.800 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.800 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.800 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:03.800 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557226+0000", 2024-05-08T08:54:03.800 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071587267999999996 2024-05-08T08:54:03.800 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.800 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.800 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:03.800 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557321+0000", 2024-05-08T08:54:03.801 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5785000000000005e-05 2024-05-08T08:54:03.801 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.801 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.801 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:03.801 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557432+0000", 2024-05-08T08:54:03.801 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011092800000000001 2024-05-08T08:54:03.801 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:03.801 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:03.801 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:03.801 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.801 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.801 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1086 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11313152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:03.801 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:54:03.801 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 87.281932956000006, 2024-05-08T08:54:03.801 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071933132999999996, 2024-05-08T08:54:03.801 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:03.802 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:03.802 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:03.802 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:03.802 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:03.802 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:03.802 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1086 2024-05-08T08:54:03.802 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.802 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:03.802 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.802 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:03.802 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:54:03.802 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:03.802 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.803 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.803 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:03.803 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:54:03.803 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:03.803 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.803 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.803 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:03.803 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486328+0000", 2024-05-08T08:54:03.803 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00068107900000000002 2024-05-08T08:54:03.803 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.803 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.803 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:03.803 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486331+0000", 2024-05-08T08:54:03.803 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.585e-06 2024-05-08T08:54:03.803 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.804 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.804 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:03.804 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486343+0000", 2024-05-08T08:54:03.804 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1856e-05 2024-05-08T08:54:03.804 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.804 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.804 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:03.804 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486350+0000", 2024-05-08T08:54:03.804 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8600000000000004e-06 2024-05-08T08:54:03.804 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.804 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.804 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:03.804 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557470+0000", 2024-05-08T08:54:03.804 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071120326999999997 2024-05-08T08:54:03.804 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.805 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.805 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:03.805 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557514+0000", 2024-05-08T08:54:03.805 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3909000000000003e-05 2024-05-08T08:54:03.805 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.805 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.805 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:03.805 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557580+0000", 2024-05-08T08:54:03.805 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5517000000000003e-05 2024-05-08T08:54:03.805 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:03.805 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:03.805 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:03.805 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.805 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.805 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1089 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11354112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:03.806 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:54:03.806 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 87.281220083999997, 2024-05-08T08:54:03.806 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071457381, 2024-05-08T08:54:03.806 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:03.806 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:03.806 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:03.806 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:03.806 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:03.806 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:03.806 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1089 2024-05-08T08:54:03.806 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.806 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:03.806 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.806 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:03.807 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:54:03.807 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:03.807 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.807 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.807 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:03.807 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:54:03.807 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:03.807 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.807 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.807 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:03.807 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486851+0000", 2024-05-08T08:54:03.807 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00049102 2024-05-08T08:54:03.807 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.807 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.808 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:03.808 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486854+0000", 2024-05-08T08:54:03.808 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6859999999999999e-06 2024-05-08T08:54:03.808 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.808 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.808 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:03.808 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486864+0000", 2024-05-08T08:54:03.808 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6080000000000002e-06 2024-05-08T08:54:03.808 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.808 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.808 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:03.808 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486870+0000", 2024-05-08T08:54:03.808 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.477e-06 2024-05-08T08:54:03.808 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.808 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.809 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:03.809 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557612+0000", 2024-05-08T08:54:03.809 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070741557999999996 2024-05-08T08:54:03.809 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.809 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.809 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:03.809 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557695+0000", 2024-05-08T08:54:03.809 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3300000000000005e-05 2024-05-08T08:54:03.809 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.809 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.809 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:03.809 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557817+0000", 2024-05-08T08:54:03.809 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012173200000000001 2024-05-08T08:54:03.809 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:03.809 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:03.810 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:03.810 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.810 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.810 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1091 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11395072~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:03.810 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:54:03.810 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 87.280699288999998, 2024-05-08T08:54:03.810 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071184737999999997, 2024-05-08T08:54:03.810 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:03.810 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:03.810 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:03.810 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:03.810 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:03.810 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:03.810 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1091 2024-05-08T08:54:03.811 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.811 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:03.811 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.811 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:03.811 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:54:03.811 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:03.811 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.811 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.811 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:03.811 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:54:03.811 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:03.811 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.811 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.811 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:03.811 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486881+0000", 2024-05-08T08:54:03.811 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0699999999999997e-07 2024-05-08T08:54:03.811 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.812 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.812 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:03.812 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486883+0000", 2024-05-08T08:54:03.812 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.402e-06 2024-05-08T08:54:03.812 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.812 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.812 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:03.812 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486890+0000", 2024-05-08T08:54:03.812 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0480000000000001e-06 2024-05-08T08:54:03.812 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.812 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.812 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:03.812 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486896+0000", 2024-05-08T08:54:03.812 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7640000000000002e-06 2024-05-08T08:54:03.812 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.813 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.813 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:03.813 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557860+0000", 2024-05-08T08:54:03.813 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070963791999999998 2024-05-08T08:54:03.813 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.813 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.813 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:03.813 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557952+0000", 2024-05-08T08:54:03.813 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2082000000000004e-05 2024-05-08T08:54:03.813 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.813 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.813 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:03.813 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558065+0000", 2024-05-08T08:54:03.813 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000113143 2024-05-08T08:54:03.813 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:03.813 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:03.814 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:03.814 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.814 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.814 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1094 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11436032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:03.814 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:54:03.814 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 87.280676120999999, 2024-05-08T08:54:03.814 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071400507000000002, 2024-05-08T08:54:03.814 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:03.814 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:03.814 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:03.814 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:03.814 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:03.814 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:03.814 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1094 2024-05-08T08:54:03.814 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.815 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:03.815 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.815 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:03.815 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:54:03.815 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:03.815 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.815 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.815 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:03.815 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:54:03.815 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:03.815 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.815 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.815 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:03.815 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487940+0000", 2024-05-08T08:54:03.816 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001036868 2024-05-08T08:54:03.816 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.816 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.816 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:03.816 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487944+0000", 2024-05-08T08:54:03.816 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7069999999999998e-06 2024-05-08T08:54:03.816 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.816 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.816 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:03.816 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487951+0000", 2024-05-08T08:54:03.816 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2649999999999999e-06 2024-05-08T08:54:03.816 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.816 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.816 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:03.816 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487961+0000", 2024-05-08T08:54:03.817 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4110000000000002e-06 2024-05-08T08:54:03.817 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.817 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.817 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:03.817 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558117+0000", 2024-05-08T08:54:03.817 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070156662999999994 2024-05-08T08:54:03.817 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.817 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.817 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:03.817 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558214+0000", 2024-05-08T08:54:03.817 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6444e-05 2024-05-08T08:54:03.817 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.817 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.817 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:03.817 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558304+0000", 2024-05-08T08:54:03.817 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0148999999999994e-05 2024-05-08T08:54:03.818 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:03.818 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:03.818 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:03.818 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.818 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.818 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1096 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11476992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:03.818 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:54:03.818 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 87.279609590000007, 2024-05-08T08:54:03.818 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070480074000000004, 2024-05-08T08:54:03.818 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:03.818 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:03.818 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:03.818 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:03.818 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:03.819 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:03.819 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1096 2024-05-08T08:54:03.819 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.819 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:03.819 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.819 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:03.819 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:54:03.819 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:03.819 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.819 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.819 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:03.819 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:54:03.819 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:03.819 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.820 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.820 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:03.820 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:54:03.820 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3700000000000001e-07 2024-05-08T08:54:03.820 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.820 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.820 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:03.820 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487973+0000", 2024-05-08T08:54:03.820 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0870000000000001e-06 2024-05-08T08:54:03.820 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.820 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.820 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:03.820 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487977+0000", 2024-05-08T08:54:03.820 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0600000000000001e-06 2024-05-08T08:54:03.820 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.821 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.821 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:03.821 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487980+0000", 2024-05-08T08:54:03.821 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1089999999999998e-06 2024-05-08T08:54:03.821 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.821 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.821 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:03.821 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558338+0000", 2024-05-08T08:54:03.821 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070358233000000006 2024-05-08T08:54:03.821 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.821 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.821 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:03.821 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558392+0000", 2024-05-08T08:54:03.821 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4537999999999999e-05 2024-05-08T08:54:03.821 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.821 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.822 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:03.822 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558450+0000", 2024-05-08T08:54:03.822 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7710000000000001e-05 2024-05-08T08:54:03.822 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:03.822 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:03.822 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:03.822 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.822 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.822 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1099 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11517952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:03.822 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:54:03.822 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 87.279595713999996, 2024-05-08T08:54:03.822 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070720506000000002, 2024-05-08T08:54:03.822 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:03.822 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:03.823 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:03.823 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:03.823 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:03.823 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:03.823 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1099 2024-05-08T08:54:03.823 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.823 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:03.823 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.823 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:03.823 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:54:03.823 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:03.823 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.823 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.823 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:03.823 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:54:03.823 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:03.824 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.824 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.824 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:03.824 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:54:03.824 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3999999999999998e-07 2024-05-08T08:54:03.824 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.824 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.824 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:03.824 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487986+0000", 2024-05-08T08:54:03.824 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3519999999999999e-06 2024-05-08T08:54:03.824 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.824 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.824 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:03.824 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487989+0000", 2024-05-08T08:54:03.824 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.23e-06 2024-05-08T08:54:03.825 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.825 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.825 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:03.825 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487991+0000", 2024-05-08T08:54:03.825 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6350000000000002e-06 2024-05-08T08:54:03.825 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.825 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.825 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:03.825 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558489+0000", 2024-05-08T08:54:03.825 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070497924000000003 2024-05-08T08:54:03.825 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.825 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.825 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:03.825 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558583+0000", 2024-05-08T08:54:03.825 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3405000000000004e-05 2024-05-08T08:54:03.825 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.826 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.826 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:03.826 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558704+0000", 2024-05-08T08:54:03.826 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012171999999999999 2024-05-08T08:54:03.826 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:03.826 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:03.826 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:03.826 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.826 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.826 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1100 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11558912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:03.826 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:54:03.826 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 87.279582715000004, 2024-05-08T08:54:03.826 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070962266999999996, 2024-05-08T08:54:03.826 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:03.827 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:03.827 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:03.827 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:03.827 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:03.827 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:03.827 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1100 2024-05-08T08:54:03.827 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.827 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:03.827 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.827 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:03.827 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:54:03.827 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:03.827 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.827 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.827 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:03.827 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:54:03.828 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:03.828 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.828 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.828 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:03.828 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488430+0000", 2024-05-08T08:54:03.828 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000432676 2024-05-08T08:54:03.828 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.828 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.828 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:03.828 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488433+0000", 2024-05-08T08:54:03.828 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.45e-06 2024-05-08T08:54:03.828 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.828 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.828 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:03.828 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488442+0000", 2024-05-08T08:54:03.829 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.687e-06 2024-05-08T08:54:03.829 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.829 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.829 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:03.829 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488446+0000", 2024-05-08T08:54:03.829 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4939999999999997e-06 2024-05-08T08:54:03.829 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.829 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.829 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:03.829 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558756+0000", 2024-05-08T08:54:03.829 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070310051999999998 2024-05-08T08:54:03.829 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.829 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.829 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:03.829 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558844+0000", 2024-05-08T08:54:03.830 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7690999999999998e-05 2024-05-08T08:54:03.830 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.830 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.830 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:03.830 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558959+0000", 2024-05-08T08:54:03.830 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000115217 2024-05-08T08:54:03.830 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:03.830 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:03.830 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:03.830 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.830 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.830 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1103 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11599872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:03.830 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:54:03.830 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 87.279125039999997, 2024-05-08T08:54:03.831 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070777100999999995, 2024-05-08T08:54:03.831 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:03.831 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:03.831 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:03.831 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:03.831 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:03.831 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:03.831 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1103 2024-05-08T08:54:03.831 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.831 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:03.831 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.831 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:03.831 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:54:03.831 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:03.831 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.831 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.832 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:03.832 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:54:03.832 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:03.832 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.832 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.832 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:03.832 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488959+0000", 2024-05-08T08:54:03.832 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00050457200000000001 2024-05-08T08:54:03.832 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.832 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.832 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:03.832 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488969+0000", 2024-05-08T08:54:03.832 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.7159999999999998e-06 2024-05-08T08:54:03.832 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.832 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.833 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:03.833 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488981+0000", 2024-05-08T08:54:03.833 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2150000000000001e-05 2024-05-08T08:54:03.833 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.833 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.833 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:03.833 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488988+0000", 2024-05-08T08:54:03.833 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8759999999999999e-06 2024-05-08T08:54:03.833 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.833 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.833 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:03.833 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558992+0000", 2024-05-08T08:54:03.833 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070004358000000003 2024-05-08T08:54:03.833 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.833 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.834 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:03.834 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559093+0000", 2024-05-08T08:54:03.834 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000100299 2024-05-08T08:54:03.834 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.834 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.834 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:03.834 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559232+0000", 2024-05-08T08:54:03.834 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013913 2024-05-08T08:54:03.834 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:03.834 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:03.834 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:03.834 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.834 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.834 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1104 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11640832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:03.834 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:54:03.835 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 87.278580224999999, 2024-05-08T08:54:03.835 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070521947000000001, 2024-05-08T08:54:03.835 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:03.835 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:03.835 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:03.835 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:03.835 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:03.835 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:03.835 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1104 2024-05-08T08:54:03.835 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.835 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:03.835 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.835 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:03.835 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:54:03.835 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:03.836 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.836 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.836 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:03.836 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:54:03.836 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:03.836 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.836 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.836 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:03.836 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489700+0000", 2024-05-08T08:54:03.836 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00070067600000000001 2024-05-08T08:54:03.836 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.836 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.836 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:03.836 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489704+0000", 2024-05-08T08:54:03.836 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1640000000000003e-06 2024-05-08T08:54:03.836 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.837 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.837 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:03.837 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489716+0000", 2024-05-08T08:54:03.837 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1379000000000001e-05 2024-05-08T08:54:03.837 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.837 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.837 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:03.837 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489724+0000", 2024-05-08T08:54:03.837 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8949999999999998e-06 2024-05-08T08:54:03.837 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.837 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.837 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:03.837 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559278+0000", 2024-05-08T08:54:03.837 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.069554151999999994 2024-05-08T08:54:03.837 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.837 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.838 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:03.838 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559387+0000", 2024-05-08T08:54:03.838 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000109258 2024-05-08T08:54:03.838 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.838 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.838 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:03.838 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559521+0000", 2024-05-08T08:54:03.838 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000134423 2024-05-08T08:54:03.838 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:03.838 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:03.838 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:03.838 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.838 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.839 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:8636 3.17 3:e8a0a3e4:::rbd_header.11402b26385:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-05-08T08:54:03.839 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:54.150971+0000", 2024-05-08T08:54:03.839 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 69.616608231000001, 2024-05-08T08:54:03.839 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.217273233, 2024-05-08T08:54:03.839 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:03.839 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:03.839 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:03.839 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:03.839 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:54:03.839 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:54:03.839 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 8636 2024-05-08T08:54:03.839 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.839 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:03.839 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.839 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:03.839 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150971+0000", 2024-05-08T08:54:03.839 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:03.840 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.840 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.840 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:03.840 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150971+0000", 2024-05-08T08:54:03.840 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:03.840 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.840 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.840 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:03.840 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150972+0000", 2024-05-08T08:54:03.840 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.71e-07 2024-05-08T08:54:03.840 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.840 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.840 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:03.840 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150977+0000", 2024-05-08T08:54:03.840 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.152e-06 2024-05-08T08:54:03.841 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.841 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.841 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:03.841 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150990+0000", 2024-05-08T08:54:03.841 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2364999999999999e-05 2024-05-08T08:54:03.841 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.841 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.841 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:03.841 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150998+0000", 2024-05-08T08:54:03.841 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0739999999999993e-06 2024-05-08T08:54:03.841 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.841 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.841 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:03.841 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.151054+0000", 2024-05-08T08:54:03.841 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5949000000000003e-05 2024-05-08T08:54:03.841 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.842 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.842 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:54:03.842 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.151060+0000", 2024-05-08T08:54:03.842 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6120000000000004e-06 2024-05-08T08:54:03.842 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.842 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.842 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:03.842 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.367640+0000", 2024-05-08T08:54:03.842 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2165792739999999 2024-05-08T08:54:03.842 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.842 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.842 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:03.842 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.367839+0000", 2024-05-08T08:54:03.842 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000199185 2024-05-08T08:54:03.842 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.843 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.843 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:03.843 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.368245+0000", 2024-05-08T08:54:03.843 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00040575099999999998 2024-05-08T08:54:03.843 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:03.843 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:03.843 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:03.843 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.843 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.843 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5302.0:1516 3.c 3:319bf789:::rbd_data.1140d3fc54c.0000000000000025:head [stat,write 1892352~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:54:03.843 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:27.822220+0000", 2024-05-08T08:54:03.843 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 35.945359293999999, 2024-05-08T08:54:03.843 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.092643744, 2024-05-08T08:54:03.843 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:03.843 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:03.843 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:54:03.844 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:03.844 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5302", 2024-05-08T08:54:03.844 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/359277859", 2024-05-08T08:54:03.844 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1516 2024-05-08T08:54:03.844 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.844 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:03.844 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.844 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:03.844 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822220+0000", 2024-05-08T08:54:03.844 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:03.844 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.844 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.844 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:03.844 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822220+0000", 2024-05-08T08:54:03.844 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:03.844 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.845 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.845 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:03.845 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822221+0000", 2024-05-08T08:54:03.845 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3500000000000002e-07 2024-05-08T08:54:03.845 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.845 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.845 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:03.845 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822228+0000", 2024-05-08T08:54:03.845 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9959999999999996e-06 2024-05-08T08:54:03.845 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.845 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.845 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:03.845 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822234+0000", 2024-05-08T08:54:03.845 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5429999999999999e-06 2024-05-08T08:54:03.845 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.846 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.846 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:03.846 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822238+0000", 2024-05-08T08:54:03.846 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.732e-06 2024-05-08T08:54:03.846 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.846 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.846 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:03.846 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822285+0000", 2024-05-08T08:54:03.846 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6810000000000001e-05 2024-05-08T08:54:03.846 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.846 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.846 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:03.846 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822358+0000", 2024-05-08T08:54:03.846 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3473000000000006e-05 2024-05-08T08:54:03.846 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.847 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.847 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 5", 2024-05-08T08:54:03.847 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822489+0000", 2024-05-08T08:54:03.847 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013011199999999999 2024-05-08T08:54:03.847 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.847 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.847 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:54:03.847 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.825332+0000", 2024-05-08T08:54:03.847 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0028434799999999998 2024-05-08T08:54:03.847 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.847 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.847 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:54:03.847 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.914753+0000", 2024-05-08T08:54:03.847 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089421424999999999 2024-05-08T08:54:03.847 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.848 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.848 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:54:03.848 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.914807+0000", 2024-05-08T08:54:03.848 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3829000000000002e-05 2024-05-08T08:54:03.848 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.848 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.848 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:03.848 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.914864+0000", 2024-05-08T08:54:03.848 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6909e-05 2024-05-08T08:54:03.848 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:03.848 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:03.848 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:03.848 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.848 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.848 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:24567 3.6 3:66af1c40:::rbd_data.1140d3fc54c.000000000000005a:18 [read 2080768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:03.848 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.216598+0000", 2024-05-08T08:54:03.848 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 31.550981214, 2024-05-08T08:54:03.849 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071602647000000005, 2024-05-08T08:54:03.849 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:03.849 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:03.849 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:03.849 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:03.849 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:03.849 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:03.849 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 24567 2024-05-08T08:54:03.849 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.849 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:03.849 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.849 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:03.849 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216598+0000", 2024-05-08T08:54:03.849 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:03.849 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.849 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.849 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:03.850 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216598+0000", 2024-05-08T08:54:03.850 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:03.850 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.850 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.850 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:03.850 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216599+0000", 2024-05-08T08:54:03.850 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8999999999999997e-07 2024-05-08T08:54:03.850 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.850 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.850 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:03.850 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216601+0000", 2024-05-08T08:54:03.850 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9889999999999999e-06 2024-05-08T08:54:03.850 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.850 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.851 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:03.851 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216607+0000", 2024-05-08T08:54:03.851 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3600000000000001e-06 2024-05-08T08:54:03.851 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.851 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.851 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:03.851 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216612+0000", 2024-05-08T08:54:03.851 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.69e-06 2024-05-08T08:54:03.851 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.851 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.851 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:03.851 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.287076+0000", 2024-05-08T08:54:03.851 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070464355000000006 2024-05-08T08:54:03.851 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.851 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.851 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:03.852 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.287151+0000", 2024-05-08T08:54:03.852 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4924000000000003e-05 2024-05-08T08:54:03.852 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:03.852 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:03.852 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:03.852 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.288201+0000", 2024-05-08T08:54:03.852 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0010498389999999999 2024-05-08T08:54:03.852 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:03.852 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:03.852 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:03.852 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:03.852 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:03.852 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:03.853 INFO:teuthology.orchestra.run.smithi196.stdout:leaving image image_client.0-clone36 intact 2024-05-08T08:54:03.854 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:54:03.855 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:03.858+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:03.956 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:03.959+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:04.006 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:04.006 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:54:04.006 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:54:04.006 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:04.015 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:54:04.059 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:04.062+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:04.127 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:04.127 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:54:04.127 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:54:04.127 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:54:04.127 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:04.137 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:54:04.158 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:04.161+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:04.260 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:04.262+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:04.290 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:04.290 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:54:04.290 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:54:04.290 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:54:04.290 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.290 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:12443 3.0 3:02d0158e:::rbd_data.114071bcf828.0000000000000037:head [stat,delete] snapc 19=[19] RETRY=3 ondisk+retry+write+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:04.290 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.610027+0000", 2024-05-08T08:54:04.290 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 32.682222809000002, 2024-05-08T08:54:04.291 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.083586333999999998, 2024-05-08T08:54:04.291 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:04.291 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:04.291 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:04.291 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:04.291 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:04.291 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:04.291 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 12443 2024-05-08T08:54:04.291 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.291 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:04.291 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.291 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:04.291 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610027+0000", 2024-05-08T08:54:04.291 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:04.292 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.292 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.292 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:04.292 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610027+0000", 2024-05-08T08:54:04.292 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:04.292 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.292 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.292 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:04.292 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610028+0000", 2024-05-08T08:54:04.292 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.13e-07 2024-05-08T08:54:04.292 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.292 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.292 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:04.292 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610030+0000", 2024-05-08T08:54:04.292 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7669999999999999e-06 2024-05-08T08:54:04.293 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.293 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.293 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:04.293 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610039+0000", 2024-05-08T08:54:04.293 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4959999999999996e-06 2024-05-08T08:54:04.293 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.293 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.293 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:04.293 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610045+0000", 2024-05-08T08:54:04.293 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6779999999999997e-06 2024-05-08T08:54:04.293 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.293 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.293 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:04.293 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.660873+0000", 2024-05-08T08:54:04.293 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.050828748 2024-05-08T08:54:04.294 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.294 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.294 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:04.294 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.661017+0000", 2024-05-08T08:54:04.294 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014391999999999999 2024-05-08T08:54:04.294 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.294 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.294 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:04.294 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.693613+0000", 2024-05-08T08:54:04.294 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.032596111999999997 2024-05-08T08:54:04.294 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.294 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:04.294 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.294 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.294 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.295 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:14301 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [watch reconnect cookie 139837155901536 gen 12] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:04.295 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.645078+0000", 2024-05-08T08:54:04.295 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 32.647172171000001, 2024-05-08T08:54:04.295 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088281852999999993, 2024-05-08T08:54:04.295 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:04.295 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:04.295 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:04.295 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:04.295 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:04.295 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:04.295 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14301 2024-05-08T08:54:04.295 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.295 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:04.295 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.296 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:04.296 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645078+0000", 2024-05-08T08:54:04.296 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:04.296 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.296 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.296 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:04.296 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645078+0000", 2024-05-08T08:54:04.296 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:04.296 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.296 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.296 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:04.296 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645084+0000", 2024-05-08T08:54:04.296 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4620000000000001e-06 2024-05-08T08:54:04.296 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.297 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.297 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:04.297 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645088+0000", 2024-05-08T08:54:04.297 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8039999999999999e-06 2024-05-08T08:54:04.297 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.297 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.297 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:04.297 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645106+0000", 2024-05-08T08:54:04.297 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.8318999999999999e-05 2024-05-08T08:54:04.297 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.297 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.297 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:04.297 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645113+0000", 2024-05-08T08:54:04.297 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0310000000000001e-06 2024-05-08T08:54:04.298 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.298 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.298 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:04.298 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.670493+0000", 2024-05-08T08:54:04.298 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.025379993 2024-05-08T08:54:04.298 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.298 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.298 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:04.298 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.670571+0000", 2024-05-08T08:54:04.298 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7613000000000004e-05 2024-05-08T08:54:04.298 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.298 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.298 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:04.299 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733360+0000", 2024-05-08T08:54:04.299 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.062788630999999998 2024-05-08T08:54:04.299 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.299 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:04.299 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.299 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.299 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.299 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:14420 3.0 3:06a8948e:::rbd_data.114071bcf828.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:04.299 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.649639+0000", 2024-05-08T08:54:04.299 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 32.642611023000001, 2024-05-08T08:54:04.299 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.083851897999999994, 2024-05-08T08:54:04.299 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:04.299 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:04.299 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:54:04.299 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:04.300 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:04.300 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:04.300 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14420 2024-05-08T08:54:04.300 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.300 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:04.300 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.300 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:04.300 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649639+0000", 2024-05-08T08:54:04.300 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:04.300 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.300 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.300 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:04.300 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649639+0000", 2024-05-08T08:54:04.300 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:04.300 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.301 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.301 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:04.301 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649640+0000", 2024-05-08T08:54:04.301 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6899999999999997e-07 2024-05-08T08:54:04.301 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.301 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.301 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:04.301 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649895+0000", 2024-05-08T08:54:04.301 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00025531000000000002 2024-05-08T08:54:04.301 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.301 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.301 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:04.301 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649905+0000", 2024-05-08T08:54:04.301 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0198e-05 2024-05-08T08:54:04.301 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.302 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.302 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:04.302 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649911+0000", 2024-05-08T08:54:04.302 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2310000000000001e-06 2024-05-08T08:54:04.302 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.302 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.302 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:04.302 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.684099+0000", 2024-05-08T08:54:04.302 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.034188017000000001 2024-05-08T08:54:04.302 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.302 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.302 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:04.302 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.684172+0000", 2024-05-08T08:54:04.302 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3201999999999995e-05 2024-05-08T08:54:04.302 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.303 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.303 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 2", 2024-05-08T08:54:04.303 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.684418+0000", 2024-05-08T08:54:04.303 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00024559099999999997 2024-05-08T08:54:04.303 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.303 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.303 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:54:04.303 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.705538+0000", 2024-05-08T08:54:04.303 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.021119707000000001 2024-05-08T08:54:04.303 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.303 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.303 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:54:04.303 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733399+0000", 2024-05-08T08:54:04.303 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.027861543999999999 2024-05-08T08:54:04.303 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.304 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.304 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:54:04.304 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733445+0000", 2024-05-08T08:54:04.304 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5587000000000002e-05 2024-05-08T08:54:04.304 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.304 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.304 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:04.304 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733491+0000", 2024-05-08T08:54:04.304 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5841999999999999e-05 2024-05-08T08:54:04.304 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.304 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:04.304 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.304 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.304 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.304 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28547 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889179216] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:04.304 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.355654+0000", 2024-05-08T08:54:04.305 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 31.936595794999999, 2024-05-08T08:54:04.305 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078692261999999999, 2024-05-08T08:54:04.305 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:04.305 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:04.305 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:04.305 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:04.305 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:04.305 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:04.305 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28547 2024-05-08T08:54:04.305 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.305 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:04.305 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.305 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:04.305 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355654+0000", 2024-05-08T08:54:04.305 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:04.305 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.306 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.306 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:04.306 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355654+0000", 2024-05-08T08:54:04.306 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:04.306 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.306 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.306 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:04.306 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355654+0000", 2024-05-08T08:54:04.306 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3999999999999998e-07 2024-05-08T08:54:04.306 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.306 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.306 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:04.306 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355656+0000", 2024-05-08T08:54:04.306 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.353e-06 2024-05-08T08:54:04.306 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.307 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.307 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:04.307 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355663+0000", 2024-05-08T08:54:04.307 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2259999999999999e-06 2024-05-08T08:54:04.307 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.307 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.307 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:04.307 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355666+0000", 2024-05-08T08:54:04.307 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7930000000000002e-06 2024-05-08T08:54:04.307 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.307 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.307 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:04.307 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434117+0000", 2024-05-08T08:54:04.307 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078451231999999996 2024-05-08T08:54:04.308 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.308 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.308 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:04.308 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434198+0000", 2024-05-08T08:54:04.308 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0879999999999998e-05 2024-05-08T08:54:04.308 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.308 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.308 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:04.308 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434346+0000", 2024-05-08T08:54:04.308 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014853800000000001 2024-05-08T08:54:04.308 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.308 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:04.308 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.308 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.309 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.309 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28548 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889189696] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:04.309 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.355669+0000", 2024-05-08T08:54:04.309 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 31.936580445000001, 2024-05-08T08:54:04.309 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078956606999999998, 2024-05-08T08:54:04.309 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:04.309 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:04.309 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:04.309 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:04.309 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:04.309 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:04.309 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28548 2024-05-08T08:54:04.309 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.309 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:04.309 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.310 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:04.310 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355669+0000", 2024-05-08T08:54:04.310 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:04.310 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.310 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.310 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:04.310 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355669+0000", 2024-05-08T08:54:04.310 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:04.310 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.310 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.310 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:04.310 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355762+0000", 2024-05-08T08:54:04.310 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3015000000000003e-05 2024-05-08T08:54:04.310 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.311 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.311 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:04.311 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355765+0000", 2024-05-08T08:54:04.311 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.368e-06 2024-05-08T08:54:04.311 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.311 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.311 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:04.311 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355771+0000", 2024-05-08T08:54:04.311 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5989999999999998e-06 2024-05-08T08:54:04.311 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.311 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.311 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:04.311 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355775+0000", 2024-05-08T08:54:04.311 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7280000000000001e-06 2024-05-08T08:54:04.311 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.312 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.312 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:04.312 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434379+0000", 2024-05-08T08:54:04.312 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078603662000000005 2024-05-08T08:54:04.312 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.312 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.312 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:04.312 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434473+0000", 2024-05-08T08:54:04.312 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4104000000000006e-05 2024-05-08T08:54:04.312 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.312 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.312 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:04.312 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434626+0000", 2024-05-08T08:54:04.312 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00015313100000000001 2024-05-08T08:54:04.313 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.313 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:04.313 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.313 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.313 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.313 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28549 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889202400] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:04.313 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.355781+0000", 2024-05-08T08:54:04.313 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 31.936468870999999, 2024-05-08T08:54:04.313 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079117946999999994, 2024-05-08T08:54:04.313 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:04.313 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:04.313 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:04.313 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:04.313 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:04.313 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:04.313 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28549 2024-05-08T08:54:04.314 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.314 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:04.314 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.314 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:04.314 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355781+0000", 2024-05-08T08:54:04.314 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:04.314 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.314 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.314 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:04.314 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355781+0000", 2024-05-08T08:54:04.314 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:04.314 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.314 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.314 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:04.315 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356162+0000", 2024-05-08T08:54:04.315 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00038099100000000001 2024-05-08T08:54:04.315 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.315 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.315 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:04.315 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356165+0000", 2024-05-08T08:54:04.315 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1240000000000001e-06 2024-05-08T08:54:04.315 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.315 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.315 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:04.315 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356173+0000", 2024-05-08T08:54:04.315 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.6199999999999999e-06 2024-05-08T08:54:04.315 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.315 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.316 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:04.316 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356182+0000", 2024-05-08T08:54:04.316 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7760000000000003e-06 2024-05-08T08:54:04.316 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.316 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.316 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:04.316 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434675+0000", 2024-05-08T08:54:04.316 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078493119 2024-05-08T08:54:04.316 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.316 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.316 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:04.316 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434765+0000", 2024-05-08T08:54:04.316 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0402000000000006e-05 2024-05-08T08:54:04.316 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.316 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.316 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:04.316 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434899+0000", 2024-05-08T08:54:04.317 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013391499999999999 2024-05-08T08:54:04.317 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.317 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:04.317 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.317 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.317 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.317 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28550 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888597280] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:04.317 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:54:04.317 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 31.936057824999999, 2024-05-08T08:54:04.317 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078933990999999995, 2024-05-08T08:54:04.317 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:04.317 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:04.317 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:04.317 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:04.317 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:04.317 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:04.318 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28550 2024-05-08T08:54:04.318 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.318 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:04.318 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.318 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:04.318 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:54:04.318 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:04.318 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.318 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.318 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:04.318 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:54:04.318 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:04.318 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.318 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.319 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:04.319 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:54:04.319 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.34e-07 2024-05-08T08:54:04.319 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.319 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.319 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:04.319 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356195+0000", 2024-05-08T08:54:04.319 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8710000000000001e-06 2024-05-08T08:54:04.319 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.319 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.319 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:04.319 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356200+0000", 2024-05-08T08:54:04.319 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3000000000000003e-06 2024-05-08T08:54:04.319 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.319 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.319 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:04.320 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356203+0000", 2024-05-08T08:54:04.320 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4039999999999999e-06 2024-05-08T08:54:04.320 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.320 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.320 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:04.320 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434928+0000", 2024-05-08T08:54:04.320 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078725331999999995 2024-05-08T08:54:04.320 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.320 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.320 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:04.320 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435007+0000", 2024-05-08T08:54:04.320 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8415000000000001e-05 2024-05-08T08:54:04.320 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.321 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.321 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:04.321 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435126+0000", 2024-05-08T08:54:04.321 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011923500000000001 2024-05-08T08:54:04.321 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.321 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:04.321 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.321 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.321 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.321 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28551 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888527200] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:04.321 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.356210+0000", 2024-05-08T08:54:04.321 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 31.936039477000001, 2024-05-08T08:54:04.321 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079195072000000005, 2024-05-08T08:54:04.321 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:04.321 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:04.322 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:04.322 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:04.322 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:04.322 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:04.322 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28551 2024-05-08T08:54:04.322 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.322 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:04.322 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.322 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:04.322 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356210+0000", 2024-05-08T08:54:04.322 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:04.322 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.322 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.323 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:04.323 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356210+0000", 2024-05-08T08:54:04.323 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:04.323 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.323 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.323 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:04.323 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356211+0000", 2024-05-08T08:54:04.323 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4499999999999998e-07 2024-05-08T08:54:04.323 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.323 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.323 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:04.323 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356212+0000", 2024-05-08T08:54:04.323 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6029999999999999e-06 2024-05-08T08:54:04.323 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.323 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.323 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:04.323 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356221+0000", 2024-05-08T08:54:04.324 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4200000000000007e-06 2024-05-08T08:54:04.324 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.324 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.324 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:04.324 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356224+0000", 2024-05-08T08:54:04.324 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.038e-06 2024-05-08T08:54:04.324 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.324 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.324 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:04.324 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435178+0000", 2024-05-08T08:54:04.324 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078954625000000001 2024-05-08T08:54:04.324 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.324 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.324 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:04.324 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435256+0000", 2024-05-08T08:54:04.324 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7337999999999997e-05 2024-05-08T08:54:04.325 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.325 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.325 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:04.325 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435405+0000", 2024-05-08T08:54:04.325 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000149703 2024-05-08T08:54:04.325 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.325 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:04.325 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.325 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.325 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.325 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28552 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889231456] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:04.325 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.356228+0000", 2024-05-08T08:54:04.325 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 31.936021798999999, 2024-05-08T08:54:04.325 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079346920000000001, 2024-05-08T08:54:04.325 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:04.325 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:04.325 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:04.326 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:04.326 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:04.326 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:04.326 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28552 2024-05-08T08:54:04.326 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.326 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:04.326 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.326 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:04.326 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356228+0000", 2024-05-08T08:54:04.326 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:04.326 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.326 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.326 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:04.326 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356228+0000", 2024-05-08T08:54:04.327 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:04.327 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.327 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.327 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:04.327 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356959+0000", 2024-05-08T08:54:04.327 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00073097099999999996 2024-05-08T08:54:04.327 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.327 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.327 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:04.327 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356967+0000", 2024-05-08T08:54:04.327 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0409999999999998e-06 2024-05-08T08:54:04.327 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.327 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.327 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:04.327 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356979+0000", 2024-05-08T08:54:04.327 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1489e-05 2024-05-08T08:54:04.327 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.328 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.328 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:04.328 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356986+0000", 2024-05-08T08:54:04.328 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3699999999999997e-06 2024-05-08T08:54:04.328 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.328 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.328 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:04.328 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435433+0000", 2024-05-08T08:54:04.328 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078447520000000007 2024-05-08T08:54:04.328 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.328 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.328 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:04.328 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435484+0000", 2024-05-08T08:54:04.328 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0229000000000003e-05 2024-05-08T08:54:04.328 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.328 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.329 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:04.329 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435575+0000", 2024-05-08T08:54:04.329 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1299999999999997e-05 2024-05-08T08:54:04.329 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.329 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:04.329 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.329 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.329 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.329 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28553 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888713648] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:04.329 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:54:04.329 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 31.935247141000001, 2024-05-08T08:54:04.329 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078782034000000001, 2024-05-08T08:54:04.329 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:04.329 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:04.330 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:04.330 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:04.330 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:04.330 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:04.330 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28553 2024-05-08T08:54:04.330 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.330 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:04.330 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.330 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:04.330 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:54:04.330 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:04.330 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.330 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.330 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:04.331 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:54:04.331 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:04.331 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.331 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.331 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:04.331 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:54:04.331 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6600000000000002e-07 2024-05-08T08:54:04.331 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.331 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.331 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:04.331 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357006+0000", 2024-05-08T08:54:04.331 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7190000000000001e-06 2024-05-08T08:54:04.331 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.331 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.331 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:04.332 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357013+0000", 2024-05-08T08:54:04.332 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.435e-06 2024-05-08T08:54:04.332 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.332 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.332 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:04.332 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357018+0000", 2024-05-08T08:54:04.332 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7260000000000002e-06 2024-05-08T08:54:04.332 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.332 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.332 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:04.332 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435623+0000", 2024-05-08T08:54:04.332 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078604972999999995 2024-05-08T08:54:04.332 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.332 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.332 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:04.332 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435695+0000", 2024-05-08T08:54:04.333 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1959999999999995e-05 2024-05-08T08:54:04.333 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.333 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.333 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:04.333 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435785+0000", 2024-05-08T08:54:04.333 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.9754999999999997e-05 2024-05-08T08:54:04.333 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.333 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:04.333 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.333 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.333 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.333 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28554 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888980272] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:04.333 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357024+0000", 2024-05-08T08:54:04.333 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 31.935225504999998, 2024-05-08T08:54:04.333 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078999870999999999, 2024-05-08T08:54:04.334 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:04.334 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:04.334 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:04.334 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:04.334 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:04.334 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:04.334 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28554 2024-05-08T08:54:04.334 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.334 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:04.334 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.334 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:04.334 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357024+0000", 2024-05-08T08:54:04.334 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:04.334 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.334 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.335 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:04.335 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357024+0000", 2024-05-08T08:54:04.335 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:04.335 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.335 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.335 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:04.335 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357025+0000", 2024-05-08T08:54:04.335 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:54:04.335 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.335 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.335 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:04.335 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357027+0000", 2024-05-08T08:54:04.335 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2050000000000001e-06 2024-05-08T08:54:04.335 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.335 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.336 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:04.336 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357033+0000", 2024-05-08T08:54:04.336 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5679999999999996e-06 2024-05-08T08:54:04.336 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.336 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.336 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:04.336 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357038+0000", 2024-05-08T08:54:04.336 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2889999999999999e-06 2024-05-08T08:54:04.336 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.336 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.336 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:04.336 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435823+0000", 2024-05-08T08:54:04.336 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078785677999999998 2024-05-08T08:54:04.336 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.336 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.336 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:04.337 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435892+0000", 2024-05-08T08:54:04.337 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8379000000000004e-05 2024-05-08T08:54:04.337 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.337 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.337 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:04.337 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436024+0000", 2024-05-08T08:54:04.337 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013239799999999999 2024-05-08T08:54:04.337 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.337 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:04.337 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.337 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.337 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.337 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28555 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889001104] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:04.337 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:54:04.337 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 31.935206740000002, 2024-05-08T08:54:04.337 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079115742000000003, 2024-05-08T08:54:04.338 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:04.338 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:04.338 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:04.338 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:04.338 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:04.338 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:04.338 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28555 2024-05-08T08:54:04.338 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.338 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:04.338 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.338 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:04.338 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:54:04.338 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:04.338 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.339 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.339 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:04.339 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:54:04.339 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:04.339 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.339 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.339 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:04.339 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:54:04.339 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9200000000000002e-07 2024-05-08T08:54:04.339 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.339 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.339 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:04.339 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357046+0000", 2024-05-08T08:54:04.339 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1670000000000002e-06 2024-05-08T08:54:04.339 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.339 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.339 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:04.340 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357052+0000", 2024-05-08T08:54:04.340 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.139e-06 2024-05-08T08:54:04.340 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.340 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.340 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:04.340 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357056+0000", 2024-05-08T08:54:04.340 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2440000000000004e-06 2024-05-08T08:54:04.340 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.340 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.340 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:04.340 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436059+0000", 2024-05-08T08:54:04.340 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079003070999999994 2024-05-08T08:54:04.340 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.340 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.340 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:04.340 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436097+0000", 2024-05-08T08:54:04.341 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7784000000000003e-05 2024-05-08T08:54:04.341 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.341 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.341 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:04.341 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436159+0000", 2024-05-08T08:54:04.341 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2045000000000007e-05 2024-05-08T08:54:04.341 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.341 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:04.341 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.341 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.341 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.341 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28556 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889022064] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:04.341 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357065+0000", 2024-05-08T08:54:04.341 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 31.935184707000001, 2024-05-08T08:54:04.342 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079327976999999994, 2024-05-08T08:54:04.342 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:04.342 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:04.342 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:04.342 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:04.342 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:04.342 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:04.342 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28556 2024-05-08T08:54:04.342 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.342 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:04.342 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.342 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:04.342 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357065+0000", 2024-05-08T08:54:04.342 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:04.342 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.343 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.343 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:04.343 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357065+0000", 2024-05-08T08:54:04.343 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:04.343 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.343 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.343 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:04.343 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358958+0000", 2024-05-08T08:54:04.343 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001892915 2024-05-08T08:54:04.343 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.343 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.343 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:04.343 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358962+0000", 2024-05-08T08:54:04.343 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4599999999999999e-06 2024-05-08T08:54:04.343 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.344 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.344 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:04.344 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358970+0000", 2024-05-08T08:54:04.344 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.6090000000000001e-06 2024-05-08T08:54:04.344 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.344 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.344 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:04.344 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358975+0000", 2024-05-08T08:54:04.344 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7639999999999996e-06 2024-05-08T08:54:04.344 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.344 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.344 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:04.344 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436196+0000", 2024-05-08T08:54:04.344 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.077221564000000006 2024-05-08T08:54:04.344 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.344 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.344 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:04.345 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436269+0000", 2024-05-08T08:54:04.345 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2720999999999997e-05 2024-05-08T08:54:04.345 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.349 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.349 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:04.349 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436393+0000", 2024-05-08T08:54:04.349 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012394399999999999 2024-05-08T08:54:04.349 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.350 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:04.350 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.350 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.350 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.350 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28563 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888855520] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:04.350 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:54:04.350 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 31.933175307999999, 2024-05-08T08:54:04.351 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078795566999999997, 2024-05-08T08:54:04.351 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:04.351 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:04.351 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:04.351 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:04.351 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:04.351 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:04.351 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28563 2024-05-08T08:54:04.352 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.352 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:04.352 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.352 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:04.352 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:54:04.352 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:04.352 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.352 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.352 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:04.352 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:54:04.352 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:04.352 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.353 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.353 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:04.353 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:54:04.353 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1e-07 2024-05-08T08:54:04.353 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.353 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.353 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:04.353 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359080+0000", 2024-05-08T08:54:04.353 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8940000000000001e-06 2024-05-08T08:54:04.353 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.353 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.353 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:04.353 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359083+0000", 2024-05-08T08:54:04.353 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1949999999999999e-06 2024-05-08T08:54:04.354 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.354 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.354 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:04.354 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359085+0000", 2024-05-08T08:54:04.354 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4710000000000001e-06 2024-05-08T08:54:04.354 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.354 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.354 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:04.354 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437604+0000", 2024-05-08T08:54:04.354 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078518278999999996 2024-05-08T08:54:04.354 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.354 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.354 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:04.354 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437704+0000", 2024-05-08T08:54:04.355 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010010599999999999 2024-05-08T08:54:04.355 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.355 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.355 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:04.355 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437870+0000", 2024-05-08T08:54:04.355 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00016641200000000001 2024-05-08T08:54:04.355 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.355 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:04.355 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.355 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.355 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.355 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28564 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888538944] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:04.355 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.359089+0000", 2024-05-08T08:54:04.355 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 31.933161092999999, 2024-05-08T08:54:04.355 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078952675, 2024-05-08T08:54:04.356 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:04.356 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:04.356 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:04.356 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:04.356 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:04.356 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:04.356 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28564 2024-05-08T08:54:04.356 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.356 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:04.356 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.356 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:04.356 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359089+0000", 2024-05-08T08:54:04.356 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:04.356 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.356 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.356 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:04.357 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359089+0000", 2024-05-08T08:54:04.357 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:04.357 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.357 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.357 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:04.357 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360959+0000", 2024-05-08T08:54:04.357 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0018707089999999999 2024-05-08T08:54:04.357 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.357 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.357 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:04.358 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360963+0000", 2024-05-08T08:54:04.358 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0680000000000001e-06 2024-05-08T08:54:04.358 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.358 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.358 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:04.358 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360969+0000", 2024-05-08T08:54:04.358 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1920000000000002e-06 2024-05-08T08:54:04.358 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.358 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.358 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:04.358 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360973+0000", 2024-05-08T08:54:04.358 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.031e-06 2024-05-08T08:54:04.358 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.358 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.358 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:04.359 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437916+0000", 2024-05-08T08:54:04.359 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.076942939000000002 2024-05-08T08:54:04.359 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.359 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.359 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:04.359 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437973+0000", 2024-05-08T08:54:04.359 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7358999999999997e-05 2024-05-08T08:54:04.359 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.359 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.359 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:04.359 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.438041+0000", 2024-05-08T08:54:04.359 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8376999999999999e-05 2024-05-08T08:54:04.359 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.359 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:04.359 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.360 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.360 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.360 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28574 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889427600] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:04.360 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:54:04.360 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 31.931156586, 2024-05-08T08:54:04.360 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078782476000000004, 2024-05-08T08:54:04.360 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:04.360 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:04.360 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:04.360 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:04.360 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:04.360 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:04.360 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28574 2024-05-08T08:54:04.361 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.361 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:04.361 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.361 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:04.361 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:54:04.361 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:04.361 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.361 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.361 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:04.361 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:54:04.361 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:04.361 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.361 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.361 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:04.361 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:54:04.361 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1799999999999999e-07 2024-05-08T08:54:04.362 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.362 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.362 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:04.362 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361095+0000", 2024-05-08T08:54:04.362 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3850000000000001e-06 2024-05-08T08:54:04.362 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.362 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.362 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:04.362 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361098+0000", 2024-05-08T08:54:04.362 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7989999999999998e-06 2024-05-08T08:54:04.362 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.362 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.362 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:04.362 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361100+0000", 2024-05-08T08:54:04.362 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4669999999999998e-06 2024-05-08T08:54:04.363 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.363 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.363 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:04.363 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439768+0000", 2024-05-08T08:54:04.363 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078668057999999999 2024-05-08T08:54:04.363 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.363 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.363 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:04.363 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439811+0000", 2024-05-08T08:54:04.363 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2459000000000001e-05 2024-05-08T08:54:04.363 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.363 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.363 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:04.363 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439876+0000", 2024-05-08T08:54:04.363 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5090000000000002e-05 2024-05-08T08:54:04.364 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.364 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:04.364 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.364 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.364 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.364 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28575 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889439216] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:04.364 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361103+0000", 2024-05-08T08:54:04.364 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 31.931146570999999, 2024-05-08T08:54:04.364 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078966828000000003, 2024-05-08T08:54:04.364 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:04.364 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:04.364 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:04.364 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:04.364 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:04.364 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:04.364 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28575 2024-05-08T08:54:04.364 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.365 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:04.365 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.365 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:04.365 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361103+0000", 2024-05-08T08:54:04.365 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:04.365 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.365 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.365 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:04.365 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361103+0000", 2024-05-08T08:54:04.365 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:04.365 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.365 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.365 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:04.365 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361104+0000", 2024-05-08T08:54:04.365 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1400000000000001e-07 2024-05-08T08:54:04.365 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.366 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.366 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:04.366 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361105+0000", 2024-05-08T08:54:04.366 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5540000000000001e-06 2024-05-08T08:54:04.366 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.366 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.366 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:04.367 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361108+0000", 2024-05-08T08:54:04.367 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.841e-06 2024-05-08T08:54:04.367 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.367 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.367 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:04.367 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361110+0000", 2024-05-08T08:54:04.367 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.497e-06 2024-05-08T08:54:04.367 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.367 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.367 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:04.367 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439916+0000", 2024-05-08T08:54:04.367 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078805372999999998 2024-05-08T08:54:04.367 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.367 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.367 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:04.367 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440000+0000", 2024-05-08T08:54:04.368 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4730999999999999e-05 2024-05-08T08:54:04.368 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.368 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.368 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:04.368 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440070+0000", 2024-05-08T08:54:04.368 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9617999999999997e-05 2024-05-08T08:54:04.368 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.368 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:04.368 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.368 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.368 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.368 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28576 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889451328] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:04.368 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361113+0000", 2024-05-08T08:54:04.368 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 31.931136384999999, 2024-05-08T08:54:04.368 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079088862999999995, 2024-05-08T08:54:04.368 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:04.369 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:04.369 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:04.369 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:04.369 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:04.369 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:04.369 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28576 2024-05-08T08:54:04.369 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.369 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:04.369 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.369 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:04.369 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361113+0000", 2024-05-08T08:54:04.369 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:04.369 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.369 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.369 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:04.369 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361113+0000", 2024-05-08T08:54:04.370 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:04.370 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.370 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.370 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:04.370 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361114+0000", 2024-05-08T08:54:04.370 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1299999999999999e-07 2024-05-08T08:54:04.370 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.370 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.370 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:04.370 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361115+0000", 2024-05-08T08:54:04.370 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.4300000000000001e-06 2024-05-08T08:54:04.370 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.370 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.370 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:04.371 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361118+0000", 2024-05-08T08:54:04.371 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9000000000000002e-06 2024-05-08T08:54:04.371 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.371 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.371 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:04.371 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361120+0000", 2024-05-08T08:54:04.371 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4739999999999999e-06 2024-05-08T08:54:04.371 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.371 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.371 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:04.371 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440096+0000", 2024-05-08T08:54:04.371 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078975629000000006 2024-05-08T08:54:04.371 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.371 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.371 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:04.371 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440135+0000", 2024-05-08T08:54:04.372 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9202000000000002e-05 2024-05-08T08:54:04.372 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.372 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.372 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:04.372 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440202+0000", 2024-05-08T08:54:04.372 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7014999999999994e-05 2024-05-08T08:54:04.372 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.372 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:04.372 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.372 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.372 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.372 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28577 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889462640] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:04.372 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:54:04.372 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 31.931126203000002, 2024-05-08T08:54:04.372 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079194622000000006, 2024-05-08T08:54:04.373 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:04.373 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:04.373 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:04.373 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:04.373 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:04.373 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:04.373 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28577 2024-05-08T08:54:04.373 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.373 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:04.373 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.373 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:04.373 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:54:04.373 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:04.373 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.373 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.374 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:04.374 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:54:04.374 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:04.374 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.374 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.374 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:04.374 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:54:04.374 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1400000000000001e-07 2024-05-08T08:54:04.374 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.374 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.374 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:04.374 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361125+0000", 2024-05-08T08:54:04.374 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.372e-06 2024-05-08T08:54:04.374 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.374 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.375 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:04.375 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361128+0000", 2024-05-08T08:54:04.375 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0309999999999999e-06 2024-05-08T08:54:04.375 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.375 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.375 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:04.375 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361131+0000", 2024-05-08T08:54:04.375 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4040000000000002e-06 2024-05-08T08:54:04.375 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.375 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.375 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:04.375 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440225+0000", 2024-05-08T08:54:04.375 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079094600000000001 2024-05-08T08:54:04.375 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.375 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.375 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:04.376 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440260+0000", 2024-05-08T08:54:04.376 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4703000000000003e-05 2024-05-08T08:54:04.376 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.376 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.376 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:04.376 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440318+0000", 2024-05-08T08:54:04.376 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8298000000000003e-05 2024-05-08T08:54:04.376 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.376 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:04.376 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.376 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.376 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.376 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28578 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889474192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:04.376 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361134+0000", 2024-05-08T08:54:04.376 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 31.931115777999999, 2024-05-08T08:54:04.376 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079310295000000003, 2024-05-08T08:54:04.376 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:04.377 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:04.377 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:04.377 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:04.377 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:04.377 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:04.377 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28578 2024-05-08T08:54:04.377 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.377 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:04.377 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.377 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:04.377 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361134+0000", 2024-05-08T08:54:04.377 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:04.377 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.377 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.377 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:04.377 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361134+0000", 2024-05-08T08:54:04.377 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:04.378 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.378 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.378 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:04.378 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.362958+0000", 2024-05-08T08:54:04.378 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0018237380000000001 2024-05-08T08:54:04.378 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.378 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.378 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:04.378 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.362961+0000", 2024-05-08T08:54:04.378 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.123e-06 2024-05-08T08:54:04.378 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.378 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.378 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:04.378 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.397404+0000", 2024-05-08T08:54:04.378 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.034443226 2024-05-08T08:54:04.379 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.379 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.379 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:04.379 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.397439+0000", 2024-05-08T08:54:04.379 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4690000000000002e-05 2024-05-08T08:54:04.379 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.379 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.379 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:04.379 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440340+0000", 2024-05-08T08:54:04.379 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.042901419000000003 2024-05-08T08:54:04.379 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.379 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.379 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:04.379 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440376+0000", 2024-05-08T08:54:04.380 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5444999999999997e-05 2024-05-08T08:54:04.380 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.380 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.380 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:04.380 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440444+0000", 2024-05-08T08:54:04.380 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8653999999999997e-05 2024-05-08T08:54:04.380 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.380 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:04.380 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.380 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.380 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:04.380 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:04.382 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:54:04.382 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:04.384+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:04.476 INFO:teuthology.orchestra.run.smithi196.stdout:checking clone #35, image image_client.0-clone35 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent36 2024-05-08T08:54:04.482 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:04.485+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:04.542 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:04.542 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:54:04.542 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:54:04.542 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:04.548 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:54:04.562 INFO:teuthology.orchestra.run.smithi196.stdout:4757 punch from 0x6f90400 to 0x6f9f400, (0xf000 bytes) 2024-05-08T08:54:04.573 INFO:teuthology.orchestra.run.smithi196.stdout:4760 write 0x5451c00 thru 0x5455db1 (0x41b2 bytes) 2024-05-08T08:54:04.573 INFO:teuthology.orchestra.run.smithi196.stdout:4761 write 0x292d000 thru 0x2933c8b (0x6c8c bytes) 2024-05-08T08:54:04.574 INFO:teuthology.orchestra.run.smithi196.stdout:4764 trunc from 0xa8cb400 to 0x9fe1e00 2024-05-08T08:54:04.583 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:04.586+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:04.623 INFO:teuthology.orchestra.run.smithi196.stdout:4766 read 0x43bd600 thru 0x43c4ee0 (0x78e1 bytes) 2024-05-08T08:54:04.624 INFO:teuthology.orchestra.run.smithi196.stdout:4768 write 0x640ca00 thru 0x64179cf (0xafd0 bytes) 2024-05-08T08:54:04.624 INFO:teuthology.orchestra.run.smithi196.stdout:4769 read 0x9ea7800 thru 0x9eb7392 (0xfb93 bytes) 2024-05-08T08:54:04.624 INFO:teuthology.orchestra.run.smithi196.stdout:4770 read 0x99e4000 thru 0x99f368e (0xf68f bytes) 2024-05-08T08:54:04.625 INFO:teuthology.orchestra.run.smithi196.stdout:4771 write 0x992a400 thru 0x99360e4 (0xbce5 bytes) 2024-05-08T08:54:04.636 INFO:teuthology.orchestra.run.smithi196.stdout:4772 trunc from 0x9fe1e00 to 0xaa3e00 2024-05-08T08:54:04.683 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:04.686+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:04.705 INFO:teuthology.orchestra.run.smithi196.stdout:4773 read 0x3f1800 thru 0x3f5f91 (0x4792 bytes) 2024-05-08T08:54:04.716 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:04.716 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:54:04.716 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:54:04.717 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:54:04.717 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:04.720 INFO:teuthology.orchestra.run.smithi196.stdout:4775 write 0x57e3600 thru 0x57e7fd4 (0x49d5 bytes) 2024-05-08T08:54:04.726 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:54:04.739 INFO:teuthology.orchestra.run.smithi196.stdout:4778 write 0x8f16000 thru 0x8f17fc4 (0x1fc5 bytes) 2024-05-08T08:54:04.739 INFO:teuthology.orchestra.run.smithi196.stdout:4779 read 0x1a55600 thru 0x1a64613 (0xf014 bytes) 2024-05-08T08:54:04.758 INFO:teuthology.orchestra.run.smithi196.stdout:4780 write 0xd945800 thru 0xd94eed7 (0x96d8 bytes) 2024-05-08T08:54:04.758 INFO:teuthology.orchestra.run.smithi196.stdout:4781 read 0x10e4600 thru 0x10e8ed4 (0x48d5 bytes) 2024-05-08T08:54:04.772 INFO:teuthology.orchestra.run.smithi196.stdout:4782 trunc from 0xd94f000 to 0x2018000 2024-05-08T08:54:04.790 INFO:teuthology.orchestra.run.smithi196.stdout:4784 read 0x1a42800 thru 0x1a4e37c (0xbb7d bytes) 2024-05-08T08:54:04.791 INFO:teuthology.orchestra.run.smithi196.stdout:4787 clone 38 order 21 su 131072 sc 13 2024-05-08T08:54:04.811 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:04.814+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:04.879 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:04.880 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:54:04.880 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:54:04.880 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:54:04.880 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.880 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:437 3.17 3:ea93a512:::rbd_data.1140100c9703.0000000000000077:head [set-alloc-hint object_size 2097152 write_size 2097152,write 1170432~1024] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e63)", 2024-05-08T08:54:04.880 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:08.390804+0000", 2024-05-08T08:54:04.880 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 56.490709695, 2024-05-08T08:54:04.880 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0406068690000001, 2024-05-08T08:54:04.880 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:04.880 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:04.880 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:54:04.880 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:04.880 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:04.881 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:04.881 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 437 2024-05-08T08:54:04.881 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.881 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:04.881 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.881 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:04.881 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390804+0000", 2024-05-08T08:54:04.881 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:04.881 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.881 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.881 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:04.881 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390804+0000", 2024-05-08T08:54:04.881 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:04.881 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.882 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.882 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:04.882 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390806+0000", 2024-05-08T08:54:04.882 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.666e-06 2024-05-08T08:54:04.882 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.882 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.882 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:04.882 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390818+0000", 2024-05-08T08:54:04.882 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2187999999999999e-05 2024-05-08T08:54:04.882 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.882 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.882 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:04.882 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390834+0000", 2024-05-08T08:54:04.882 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6191000000000001e-05 2024-05-08T08:54:04.883 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.883 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.883 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:04.883 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390845+0000", 2024-05-08T08:54:04.883 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0831e-05 2024-05-08T08:54:04.883 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.883 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.883 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:04.883 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390922+0000", 2024-05-08T08:54:04.883 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7253e-05 2024-05-08T08:54:04.883 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.883 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.883 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op must wait for map", 2024-05-08T08:54:04.883 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390930+0000", 2024-05-08T08:54:04.884 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8259999999999993e-06 2024-05-08T08:54:04.884 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.884 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.884 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:04.884 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392155+0000", 2024-05-08T08:54:04.884 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0012256649999999999 2024-05-08T08:54:04.884 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.884 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.884 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:54:04.884 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392165+0000", 2024-05-08T08:54:04.884 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5799999999999998e-06 2024-05-08T08:54:04.884 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.884 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.884 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:04.884 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.429596+0000", 2024-05-08T08:54:04.885 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0374313639999999 2024-05-08T08:54:04.885 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.885 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.885 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:04.885 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.429728+0000", 2024-05-08T08:54:04.885 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000131552 2024-05-08T08:54:04.885 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.885 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.885 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 0,1", 2024-05-08T08:54:04.885 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.429860+0000", 2024-05-08T08:54:04.885 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000132137 2024-05-08T08:54:04.885 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.885 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.885 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:54:04.886 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.430811+0000", 2024-05-08T08:54:04.886 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00095040200000000004 2024-05-08T08:54:04.886 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.886 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.886 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:54:04.886 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431126+0000", 2024-05-08T08:54:04.886 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000315006 2024-05-08T08:54:04.886 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.886 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.886 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:54:04.886 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431272+0000", 2024-05-08T08:54:04.886 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000146487 2024-05-08T08:54:04.886 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.886 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.886 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:54:04.887 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431315+0000", 2024-05-08T08:54:04.887 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2697000000000001e-05 2024-05-08T08:54:04.887 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.887 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.887 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:04.887 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431411+0000", 2024-05-08T08:54:04.887 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6024000000000001e-05 2024-05-08T08:54:04.887 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.887 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:04.887 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.887 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.887 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.887 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:439 3.17 3:ea93a512:::rbd_data.1140100c9703.0000000000000077:head [read 1163264~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-05-08T08:54:04.887 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:08.390871+0000", 2024-05-08T08:54:04.887 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 56.490642184000002, 2024-05-08T08:54:04.888 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0408790590000001, 2024-05-08T08:54:04.888 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:04.888 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:04.888 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:04.888 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:04.888 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:04.888 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:04.888 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 439 2024-05-08T08:54:04.888 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.888 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:04.888 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.888 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:04.888 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390871+0000", 2024-05-08T08:54:04.888 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:04.889 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.889 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.889 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:04.889 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390871+0000", 2024-05-08T08:54:04.889 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:04.889 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.889 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.889 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:04.889 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390872+0000", 2024-05-08T08:54:04.889 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6300000000000005e-07 2024-05-08T08:54:04.889 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.889 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.889 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:04.889 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390875+0000", 2024-05-08T08:54:04.889 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3459999999999998e-06 2024-05-08T08:54:04.890 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.890 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.890 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:04.890 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390891+0000", 2024-05-08T08:54:04.890 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5255e-05 2024-05-08T08:54:04.890 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.890 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.890 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:04.890 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390897+0000", 2024-05-08T08:54:04.890 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2360000000000001e-06 2024-05-08T08:54:04.890 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.890 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.890 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:04.890 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390995+0000", 2024-05-08T08:54:04.891 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8522000000000003e-05 2024-05-08T08:54:04.891 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.891 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.891 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting_for_map not empty", 2024-05-08T08:54:04.891 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.391005+0000", 2024-05-08T08:54:04.891 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1460000000000002e-06 2024-05-08T08:54:04.891 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.891 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.891 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:04.891 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392191+0000", 2024-05-08T08:54:04.891 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0011860270000000001 2024-05-08T08:54:04.891 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.891 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.891 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:54:04.891 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392197+0000", 2024-05-08T08:54:04.891 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4289999999999998e-06 2024-05-08T08:54:04.892 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.892 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.892 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:04.892 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.430596+0000", 2024-05-08T08:54:04.892 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0383989090000001 2024-05-08T08:54:04.892 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.892 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.892 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for rw locks", 2024-05-08T08:54:04.892 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.430676+0000", 2024-05-08T08:54:04.892 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9781000000000001e-05 2024-05-08T08:54:04.892 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.892 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.892 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:04.892 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431460+0000", 2024-05-08T08:54:04.893 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00078427300000000004 2024-05-08T08:54:04.893 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.893 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.893 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:04.893 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431540+0000", 2024-05-08T08:54:04.893 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0224999999999998e-05 2024-05-08T08:54:04.893 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.893 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.893 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:04.893 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431750+0000", 2024-05-08T08:54:04.893 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00021024699999999999 2024-05-08T08:54:04.893 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.893 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:04.893 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.893 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.894 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.894 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:945 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 4096000~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:04.894 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:54:04.894 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 55.430492616999999, 2024-05-08T08:54:04.894 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086774741000000002, 2024-05-08T08:54:04.894 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:04.894 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:04.894 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:04.894 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:04.894 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:04.894 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:04.894 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 945 2024-05-08T08:54:04.894 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.895 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:04.895 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.895 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:04.895 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:54:04.895 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:04.895 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.895 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.895 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:04.895 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:54:04.895 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:04.895 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.895 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.895 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:04.895 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:54:04.895 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3499999999999996e-07 2024-05-08T08:54:04.896 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.896 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.896 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:04.896 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451024+0000", 2024-05-08T08:54:04.896 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.627e-06 2024-05-08T08:54:04.896 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.896 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.896 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:04.896 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451037+0000", 2024-05-08T08:54:04.896 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2431e-05 2024-05-08T08:54:04.896 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.896 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.896 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:04.896 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451042+0000", 2024-05-08T08:54:04.897 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8409999999999996e-06 2024-05-08T08:54:04.897 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.897 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.897 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:04.897 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537608+0000", 2024-05-08T08:54:04.897 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086565243 2024-05-08T08:54:04.897 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.897 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.897 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:04.897 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537687+0000", 2024-05-08T08:54:04.897 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9519999999999998e-05 2024-05-08T08:54:04.897 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.897 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.897 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:04.897 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537796+0000", 2024-05-08T08:54:04.898 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000108544 2024-05-08T08:54:04.898 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.898 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:04.898 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.898 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.898 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.898 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:964 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 458752~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:04.898 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451051+0000", 2024-05-08T08:54:04.898 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 55.430462444, 2024-05-08T08:54:04.898 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086990490000000004, 2024-05-08T08:54:04.898 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:04.898 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:04.898 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:04.899 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:04.899 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:04.899 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:04.899 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 964 2024-05-08T08:54:04.899 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.899 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:04.899 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.899 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:04.899 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451051+0000", 2024-05-08T08:54:04.899 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:04.899 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.899 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.899 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:04.899 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451051+0000", 2024-05-08T08:54:04.899 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:04.900 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.900 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.900 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:04.900 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451052+0000", 2024-05-08T08:54:04.900 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7299999999999996e-07 2024-05-08T08:54:04.900 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.900 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.900 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:04.900 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451055+0000", 2024-05-08T08:54:04.900 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.2770000000000001e-06 2024-05-08T08:54:04.900 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.900 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.900 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:04.900 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451061+0000", 2024-05-08T08:54:04.900 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4060000000000002e-06 2024-05-08T08:54:04.901 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.901 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.901 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:04.901 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451066+0000", 2024-05-08T08:54:04.901 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0839999999999999e-06 2024-05-08T08:54:04.901 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.901 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.901 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:04.901 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537839+0000", 2024-05-08T08:54:04.901 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086772962999999995 2024-05-08T08:54:04.901 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.901 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.901 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:04.901 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537934+0000", 2024-05-08T08:54:04.901 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4742999999999997e-05 2024-05-08T08:54:04.902 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.902 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.902 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:04.902 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538042+0000", 2024-05-08T08:54:04.902 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000107444 2024-05-08T08:54:04.902 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.902 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:04.902 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.902 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.902 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.902 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:965 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 950272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:04.902 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451075+0000", 2024-05-08T08:54:04.902 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 55.430438381999998, 2024-05-08T08:54:04.902 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087230196999999995, 2024-05-08T08:54:04.903 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:04.903 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:04.903 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:04.903 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:04.903 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:04.903 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:04.903 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 965 2024-05-08T08:54:04.903 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.903 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:04.903 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.903 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:04.903 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451075+0000", 2024-05-08T08:54:04.903 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:04.903 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.903 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.904 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:04.904 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451075+0000", 2024-05-08T08:54:04.904 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:04.904 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.904 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.904 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:04.904 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451076+0000", 2024-05-08T08:54:04.904 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5000000000000002e-07 2024-05-08T08:54:04.904 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.904 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.904 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:04.904 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451078+0000", 2024-05-08T08:54:04.904 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.5639999999999999e-06 2024-05-08T08:54:04.904 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.904 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.905 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:04.905 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451085+0000", 2024-05-08T08:54:04.905 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5640000000000002e-06 2024-05-08T08:54:04.905 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.905 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.905 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:04.905 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451090+0000", 2024-05-08T08:54:04.905 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1699999999999996e-06 2024-05-08T08:54:04.905 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.905 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.905 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:04.905 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538106+0000", 2024-05-08T08:54:04.905 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087015888999999999 2024-05-08T08:54:04.905 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.906 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.906 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:04.906 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538190+0000", 2024-05-08T08:54:04.906 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3961999999999993e-05 2024-05-08T08:54:04.906 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.906 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.906 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:04.906 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538305+0000", 2024-05-08T08:54:04.906 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011529800000000001 2024-05-08T08:54:04.906 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.906 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:04.906 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.906 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.906 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.907 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:966 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 1441792~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:04.907 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:54:04.907 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 55.430415009000001, 2024-05-08T08:54:04.907 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087446323000000006, 2024-05-08T08:54:04.907 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:04.907 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:04.907 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:04.907 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:04.907 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:04.907 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:04.907 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 966 2024-05-08T08:54:04.907 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.907 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:04.907 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.907 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:04.908 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:54:04.908 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:04.908 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.908 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.908 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:04.908 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:54:04.908 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:04.908 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.908 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.908 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:04.908 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:54:04.908 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3700000000000001e-07 2024-05-08T08:54:04.908 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.908 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.909 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:04.909 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451101+0000", 2024-05-08T08:54:04.909 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2560000000000001e-06 2024-05-08T08:54:04.909 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.909 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.909 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:04.909 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451107+0000", 2024-05-08T08:54:04.909 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1070000000000001e-06 2024-05-08T08:54:04.909 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.909 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.909 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:04.909 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451112+0000", 2024-05-08T08:54:04.909 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1420000000000001e-06 2024-05-08T08:54:04.909 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.909 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.910 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:04.910 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538341+0000", 2024-05-08T08:54:04.910 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087228302999999993 2024-05-08T08:54:04.910 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.910 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.910 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:04.910 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538424+0000", 2024-05-08T08:54:04.910 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3497000000000004e-05 2024-05-08T08:54:04.910 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.910 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.910 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:04.910 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538545+0000", 2024-05-08T08:54:04.910 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000120581 2024-05-08T08:54:04.910 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.911 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:04.911 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.911 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.911 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.911 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:967 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:04.911 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451120+0000", 2024-05-08T08:54:04.911 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 55.430393533999997, 2024-05-08T08:54:04.911 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087632725999999994, 2024-05-08T08:54:04.911 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:04.911 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:04.911 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:04.911 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:04.911 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:04.911 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:04.912 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 967 2024-05-08T08:54:04.912 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.912 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:04.912 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.912 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:04.912 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451120+0000", 2024-05-08T08:54:04.912 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:04.912 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.912 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.912 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:04.912 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451120+0000", 2024-05-08T08:54:04.912 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:04.912 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.912 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.912 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:04.913 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451121+0000", 2024-05-08T08:54:04.913 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8800000000000002e-07 2024-05-08T08:54:04.913 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.913 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.913 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:04.913 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451130+0000", 2024-05-08T08:54:04.913 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2299999999999997e-06 2024-05-08T08:54:04.913 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.913 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.913 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:04.913 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451137+0000", 2024-05-08T08:54:04.913 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4669999999999998e-06 2024-05-08T08:54:04.913 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.913 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.913 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:04.914 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451143+0000", 2024-05-08T08:54:04.914 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2839999999999997e-06 2024-05-08T08:54:04.914 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.914 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.914 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:04.914 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538575+0000", 2024-05-08T08:54:04.914 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087432789999999996 2024-05-08T08:54:04.914 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.914 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.914 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:04.914 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538648+0000", 2024-05-08T08:54:04.914 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2877e-05 2024-05-08T08:54:04.914 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.914 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.915 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:04.915 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538753+0000", 2024-05-08T08:54:04.915 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010449 2024-05-08T08:54:04.915 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.915 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:04.915 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.915 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.915 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.915 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:968 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 2424832~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:04.915 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451152+0000", 2024-05-08T08:54:04.915 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 55.430361449000003, 2024-05-08T08:54:04.915 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087813325999999997, 2024-05-08T08:54:04.915 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:04.915 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:04.915 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:04.916 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:04.916 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:04.916 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:04.916 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 968 2024-05-08T08:54:04.916 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.916 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:04.916 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.916 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:04.916 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451152+0000", 2024-05-08T08:54:04.916 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:04.916 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.916 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.916 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:04.916 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451152+0000", 2024-05-08T08:54:04.917 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:04.917 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.917 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.917 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:04.917 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451153+0000", 2024-05-08T08:54:04.917 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3099999999999998e-07 2024-05-08T08:54:04.917 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.917 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.917 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:04.917 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451155+0000", 2024-05-08T08:54:04.917 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4760000000000001e-06 2024-05-08T08:54:04.917 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.917 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.917 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:04.917 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451162+0000", 2024-05-08T08:54:04.918 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5969999999999997e-06 2024-05-08T08:54:04.918 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.918 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.918 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:04.918 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451167+0000", 2024-05-08T08:54:04.918 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.977e-06 2024-05-08T08:54:04.918 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.918 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.918 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:04.918 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538781+0000", 2024-05-08T08:54:04.918 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087614846999999996 2024-05-08T08:54:04.918 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.918 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.918 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:04.919 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538857+0000", 2024-05-08T08:54:04.919 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5358000000000006e-05 2024-05-08T08:54:04.919 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.919 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.919 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:04.919 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538965+0000", 2024-05-08T08:54:04.919 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010864 2024-05-08T08:54:04.919 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.919 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:04.919 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.919 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.919 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.919 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:969 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 2916352~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:04.919 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:54:04.919 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 55.430339744000001, 2024-05-08T08:54:04.920 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087998980000000004, 2024-05-08T08:54:04.920 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:04.920 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:04.920 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:04.920 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:04.920 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:04.920 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:04.920 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 969 2024-05-08T08:54:04.920 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.920 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:04.920 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.920 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:04.920 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:54:04.920 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:04.920 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.921 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.921 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:04.921 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:54:04.921 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:04.921 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.921 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.921 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:04.921 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:54:04.921 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3599999999999999e-07 2024-05-08T08:54:04.921 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.921 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.921 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:04.921 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451177+0000", 2024-05-08T08:54:04.921 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4590000000000001e-06 2024-05-08T08:54:04.922 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.922 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.922 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:04.922 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451184+0000", 2024-05-08T08:54:04.922 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4109999999999999e-06 2024-05-08T08:54:04.922 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.922 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.922 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:04.922 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451191+0000", 2024-05-08T08:54:04.922 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7279999999999998e-06 2024-05-08T08:54:04.922 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.922 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.922 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:04.922 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538997+0000", 2024-05-08T08:54:04.922 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087806170000000003 2024-05-08T08:54:04.923 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.923 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.923 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:04.923 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539069+0000", 2024-05-08T08:54:04.923 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2373999999999995e-05 2024-05-08T08:54:04.923 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.923 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.923 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:04.923 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539173+0000", 2024-05-08T08:54:04.923 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000103402 2024-05-08T08:54:04.923 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.923 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:04.923 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.923 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.923 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.924 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:970 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 3407872~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:04.924 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451198+0000", 2024-05-08T08:54:04.924 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 55.430315147000002, 2024-05-08T08:54:04.924 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088183368999999998, 2024-05-08T08:54:04.924 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:04.924 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:04.924 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:04.924 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:04.924 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:04.924 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:04.924 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 970 2024-05-08T08:54:04.924 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.924 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:04.924 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.925 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:04.925 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451198+0000", 2024-05-08T08:54:04.925 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:04.925 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.925 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.925 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:04.925 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451198+0000", 2024-05-08T08:54:04.925 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:04.925 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.925 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.925 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:04.925 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451199+0000", 2024-05-08T08:54:04.925 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7400000000000003e-07 2024-05-08T08:54:04.925 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.925 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.925 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:04.926 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451211+0000", 2024-05-08T08:54:04.926 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2109999999999999e-05 2024-05-08T08:54:04.926 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.926 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.926 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:04.926 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451219+0000", 2024-05-08T08:54:04.926 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4170000000000003e-06 2024-05-08T08:54:04.926 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.926 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.926 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:04.926 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451224+0000", 2024-05-08T08:54:04.926 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3990000000000003e-06 2024-05-08T08:54:04.926 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.926 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.927 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:04.927 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539204+0000", 2024-05-08T08:54:04.927 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087980099000000006 2024-05-08T08:54:04.927 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.927 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.927 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:04.927 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539276+0000", 2024-05-08T08:54:04.927 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1513000000000005e-05 2024-05-08T08:54:04.927 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.927 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.927 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:04.927 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539382+0000", 2024-05-08T08:54:04.927 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010615699999999999 2024-05-08T08:54:04.927 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.927 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:04.928 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.928 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.928 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.928 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:971 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 3899392~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:04.928 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451233+0000", 2024-05-08T08:54:04.928 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 55.430280439999997, 2024-05-08T08:54:04.928 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088431525999999996, 2024-05-08T08:54:04.928 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:04.928 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:04.928 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:04.928 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:04.928 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:04.928 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:04.928 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 971 2024-05-08T08:54:04.928 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.929 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:04.929 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.929 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:04.929 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451233+0000", 2024-05-08T08:54:04.929 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:04.929 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.929 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.929 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:04.929 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451233+0000", 2024-05-08T08:54:04.929 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:04.929 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.929 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.929 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:04.929 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451234+0000", 2024-05-08T08:54:04.930 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8500000000000001e-07 2024-05-08T08:54:04.930 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.930 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.930 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:04.930 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451237+0000", 2024-05-08T08:54:04.930 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8320000000000002e-06 2024-05-08T08:54:04.930 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.930 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.930 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:04.930 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451243+0000", 2024-05-08T08:54:04.930 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6749999999999996e-06 2024-05-08T08:54:04.930 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.930 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.930 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:04.930 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451249+0000", 2024-05-08T08:54:04.931 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1229999999999997e-06 2024-05-08T08:54:04.931 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.931 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.931 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:04.931 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539412+0000", 2024-05-08T08:54:04.931 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088162668999999999 2024-05-08T08:54:04.931 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.931 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.931 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:04.931 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539505+0000", 2024-05-08T08:54:04.931 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3240999999999997e-05 2024-05-08T08:54:04.931 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.931 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.931 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:04.931 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539665+0000", 2024-05-08T08:54:04.931 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00015930100000000001 2024-05-08T08:54:04.932 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.932 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:04.932 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.932 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.932 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.932 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:972 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 196608~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:04.932 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451257+0000", 2024-05-08T08:54:04.932 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 55.430256495000002, 2024-05-08T08:54:04.932 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088719928000000003, 2024-05-08T08:54:04.932 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:04.932 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:04.932 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:04.932 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:04.932 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:04.933 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:04.933 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 972 2024-05-08T08:54:04.933 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.933 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:04.933 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.933 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:04.933 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451257+0000", 2024-05-08T08:54:04.933 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:04.933 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.933 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.933 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:04.933 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451257+0000", 2024-05-08T08:54:04.933 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:04.933 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.934 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.934 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:04.934 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451258+0000", 2024-05-08T08:54:04.934 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7999999999999996e-07 2024-05-08T08:54:04.934 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.934 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.934 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:04.934 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451260+0000", 2024-05-08T08:54:04.934 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4609999999999998e-06 2024-05-08T08:54:04.934 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.934 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.934 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:04.934 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451267+0000", 2024-05-08T08:54:04.934 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.81e-06 2024-05-08T08:54:04.935 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.935 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.935 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:04.935 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451273+0000", 2024-05-08T08:54:04.935 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0859999999999998e-06 2024-05-08T08:54:04.935 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.935 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.935 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:04.935 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539718+0000", 2024-05-08T08:54:04.935 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088445059000000006 2024-05-08T08:54:04.935 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.935 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.935 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:04.935 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539834+0000", 2024-05-08T08:54:04.935 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000116426 2024-05-08T08:54:04.936 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.936 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.936 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:04.936 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539977+0000", 2024-05-08T08:54:04.936 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014260600000000001 2024-05-08T08:54:04.936 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.936 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:04.936 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.936 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.936 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.936 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:973 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 688128~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:04.936 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:54:04.936 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 55.430233911000002, 2024-05-08T08:54:04.936 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088918496, 2024-05-08T08:54:04.936 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:04.936 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:04.937 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:04.937 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:04.937 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:04.937 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:04.937 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 973 2024-05-08T08:54:04.937 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.937 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:04.937 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.937 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:04.937 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:54:04.937 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:04.937 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.937 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.937 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:04.937 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:54:04.938 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:04.938 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.938 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.938 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:04.938 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:54:04.938 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9099999999999999e-07 2024-05-08T08:54:04.938 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.938 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.938 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:04.938 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451282+0000", 2024-05-08T08:54:04.938 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1550000000000001e-06 2024-05-08T08:54:04.938 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.938 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.939 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:04.939 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451288+0000", 2024-05-08T08:54:04.939 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8889999999999999e-06 2024-05-08T08:54:04.939 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.939 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.939 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:04.939 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451294+0000", 2024-05-08T08:54:04.939 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0850000000000002e-06 2024-05-08T08:54:04.939 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.939 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.939 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:04.939 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540011+0000", 2024-05-08T08:54:04.939 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088717314000000005 2024-05-08T08:54:04.939 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.939 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.940 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:04.940 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540089+0000", 2024-05-08T08:54:04.940 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7438e-05 2024-05-08T08:54:04.940 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.940 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.940 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:04.940 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540198+0000", 2024-05-08T08:54:04.940 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000109224 2024-05-08T08:54:04.940 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.940 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:04.940 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.940 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.940 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.940 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:974 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 1179648~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:04.940 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451301+0000", 2024-05-08T08:54:04.941 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 55.430212277999999, 2024-05-08T08:54:04.941 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089105078000000004, 2024-05-08T08:54:04.941 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:04.941 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:04.941 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:04.941 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:04.941 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:04.941 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:04.941 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 974 2024-05-08T08:54:04.941 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.941 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:04.941 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.941 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:04.941 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451301+0000", 2024-05-08T08:54:04.941 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:04.941 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.941 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.942 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:04.942 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451301+0000", 2024-05-08T08:54:04.942 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:04.942 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.942 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.942 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:04.942 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451302+0000", 2024-05-08T08:54:04.942 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7800000000000002e-07 2024-05-08T08:54:04.942 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.942 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.942 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:04.942 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451304+0000", 2024-05-08T08:54:04.942 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.65e-06 2024-05-08T08:54:04.942 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.943 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.943 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:04.943 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451311+0000", 2024-05-08T08:54:04.943 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.472e-06 2024-05-08T08:54:04.943 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.943 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.943 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:04.943 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451316+0000", 2024-05-08T08:54:04.943 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9189999999999998e-06 2024-05-08T08:54:04.943 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.943 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.943 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:04.943 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540234+0000", 2024-05-08T08:54:04.943 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088917752000000003 2024-05-08T08:54:04.943 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.944 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.944 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:04.944 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540308+0000", 2024-05-08T08:54:04.944 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4758000000000005e-05 2024-05-08T08:54:04.944 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.944 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.944 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:04.944 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540406+0000", 2024-05-08T08:54:04.944 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8048999999999996e-05 2024-05-08T08:54:04.944 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.944 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:04.944 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.944 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.944 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.944 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:975 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 1671168~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:04.945 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:54:04.945 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 55.430169948, 2024-05-08T08:54:04.945 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089294139999999994, 2024-05-08T08:54:04.945 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:04.945 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:04.945 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:04.945 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:04.945 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:04.945 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:04.945 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 975 2024-05-08T08:54:04.945 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.945 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:04.945 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.945 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:04.945 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:54:04.945 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:04.946 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.946 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.946 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:04.946 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:54:04.946 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:04.946 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.946 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.946 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:04.946 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:54:04.946 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1299999999999999e-07 2024-05-08T08:54:04.946 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.946 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.946 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:04.946 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451348+0000", 2024-05-08T08:54:04.947 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.484e-06 2024-05-08T08:54:04.947 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.947 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.947 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:04.947 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451356+0000", 2024-05-08T08:54:04.947 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8499999999999994e-06 2024-05-08T08:54:04.947 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.947 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.947 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:04.947 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451366+0000", 2024-05-08T08:54:04.947 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.9850000000000001e-06 2024-05-08T08:54:04.947 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.947 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.947 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:04.947 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540448+0000", 2024-05-08T08:54:04.948 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089082254 2024-05-08T08:54:04.948 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.948 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.948 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:04.948 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540528+0000", 2024-05-08T08:54:04.948 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0234999999999993e-05 2024-05-08T08:54:04.948 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.948 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.948 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:04.948 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540638+0000", 2024-05-08T08:54:04.948 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010951900000000001 2024-05-08T08:54:04.948 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.948 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:04.948 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.949 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.949 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.949 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:976 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 2162688~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:04.949 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451375+0000", 2024-05-08T08:54:04.949 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 55.430138200999998, 2024-05-08T08:54:04.949 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089472745000000006, 2024-05-08T08:54:04.949 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:04.949 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:04.949 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:04.949 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:04.949 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:04.949 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:04.949 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 976 2024-05-08T08:54:04.949 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.949 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:04.949 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.950 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:04.950 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451375+0000", 2024-05-08T08:54:04.950 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:04.950 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.950 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.950 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:04.950 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451375+0000", 2024-05-08T08:54:04.950 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:04.950 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.950 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.950 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:04.950 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451376+0000", 2024-05-08T08:54:04.950 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2399999999999998e-07 2024-05-08T08:54:04.950 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.951 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.951 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:04.951 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451379+0000", 2024-05-08T08:54:04.951 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6359999999999998e-06 2024-05-08T08:54:04.951 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.951 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.951 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:04.951 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451387+0000", 2024-05-08T08:54:04.951 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9929999999999995e-06 2024-05-08T08:54:04.951 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.951 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.951 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:04.951 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451393+0000", 2024-05-08T08:54:04.951 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4699999999999999e-06 2024-05-08T08:54:04.951 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.951 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.952 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:04.952 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540677+0000", 2024-05-08T08:54:04.952 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089283504999999999 2024-05-08T08:54:04.952 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.952 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.952 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:04.952 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540743+0000", 2024-05-08T08:54:04.952 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6303999999999994e-05 2024-05-08T08:54:04.952 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.952 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.952 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:04.952 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540848+0000", 2024-05-08T08:54:04.952 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000105213 2024-05-08T08:54:04.952 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.952 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:04.952 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.953 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.953 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.953 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:977 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 2654208~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:04.953 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:54:04.953 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 55.430112768999997, 2024-05-08T08:54:04.953 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089698473000000001, 2024-05-08T08:54:04.953 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:04.953 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:04.953 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:04.953 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:04.953 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:04.953 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:04.953 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 977 2024-05-08T08:54:04.953 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.953 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:04.953 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.954 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:04.954 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:54:04.954 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:04.954 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.954 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.954 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:04.954 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:54:04.954 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:04.954 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.954 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.954 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:04.954 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:54:04.954 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2899999999999999e-07 2024-05-08T08:54:04.954 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.955 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.955 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:04.955 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451404+0000", 2024-05-08T08:54:04.955 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6649999999999999e-06 2024-05-08T08:54:04.955 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.955 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.955 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:04.955 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451410+0000", 2024-05-08T08:54:04.955 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3269999999999997e-06 2024-05-08T08:54:04.955 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.955 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.955 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:04.955 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451415+0000", 2024-05-08T08:54:04.955 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9529999999999998e-06 2024-05-08T08:54:04.956 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.956 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.956 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:04.956 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540912+0000", 2024-05-08T08:54:04.956 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089496950000000006 2024-05-08T08:54:04.956 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.956 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.956 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:04.956 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541000+0000", 2024-05-08T08:54:04.956 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7429000000000006e-05 2024-05-08T08:54:04.956 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.956 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.956 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:04.956 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541099+0000", 2024-05-08T08:54:04.956 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.9720000000000001e-05 2024-05-08T08:54:04.957 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.957 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:04.957 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.957 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.957 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.957 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:978 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 3145728~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:04.957 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:54:04.957 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 55.430091560999998, 2024-05-08T08:54:04.957 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089900635000000007, 2024-05-08T08:54:04.957 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:04.957 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:04.957 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:04.957 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:04.957 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:04.957 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:04.958 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 978 2024-05-08T08:54:04.958 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.958 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:04.958 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.958 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:04.958 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:54:04.958 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:04.958 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.958 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.958 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:04.958 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:54:04.958 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:04.958 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.958 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.958 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:04.959 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:54:04.959 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9400000000000001e-07 2024-05-08T08:54:04.959 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.959 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.959 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:04.959 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451425+0000", 2024-05-08T08:54:04.959 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2460000000000002e-06 2024-05-08T08:54:04.959 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.959 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.959 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:04.959 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451431+0000", 2024-05-08T08:54:04.959 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2419999999999997e-06 2024-05-08T08:54:04.959 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.959 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.960 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:04.960 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451436+0000", 2024-05-08T08:54:04.960 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.887e-06 2024-05-08T08:54:04.960 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.960 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.960 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:04.960 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541152+0000", 2024-05-08T08:54:04.960 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089716194999999999 2024-05-08T08:54:04.960 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.960 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.960 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:04.960 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541224+0000", 2024-05-08T08:54:04.960 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2416000000000006e-05 2024-05-08T08:54:04.960 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.961 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.961 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:04.961 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541323+0000", 2024-05-08T08:54:04.961 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8255e-05 2024-05-08T08:54:04.961 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.961 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:04.961 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.961 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.961 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.961 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:979 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 3637248~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:04.961 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451462+0000", 2024-05-08T08:54:04.961 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 55.430051112000001, 2024-05-08T08:54:04.961 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.090036441999999994, 2024-05-08T08:54:04.961 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:04.962 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:04.962 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:04.962 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:04.962 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:04.962 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:04.962 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 979 2024-05-08T08:54:04.962 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.962 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:04.962 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.962 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:04.962 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451462+0000", 2024-05-08T08:54:04.962 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:04.962 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.962 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.963 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:04.963 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451462+0000", 2024-05-08T08:54:04.963 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:04.963 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.963 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.963 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:04.963 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451463+0000", 2024-05-08T08:54:04.963 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4300000000000002e-07 2024-05-08T08:54:04.963 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.963 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.963 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:04.963 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451471+0000", 2024-05-08T08:54:04.963 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0800000000000006e-06 2024-05-08T08:54:04.963 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.964 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.964 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:04.964 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451481+0000", 2024-05-08T08:54:04.964 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.38e-06 2024-05-08T08:54:04.964 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.964 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.964 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:04.964 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451487+0000", 2024-05-08T08:54:04.964 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4590000000000003e-06 2024-05-08T08:54:04.964 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.964 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.964 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:04.964 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541350+0000", 2024-05-08T08:54:04.964 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089862599000000001 2024-05-08T08:54:04.964 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.964 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.965 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:04.965 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541407+0000", 2024-05-08T08:54:04.965 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7667999999999997e-05 2024-05-08T08:54:04.965 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.965 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.965 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:04.965 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541499+0000", 2024-05-08T08:54:04.965 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1513000000000003e-05 2024-05-08T08:54:04.965 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.965 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:04.965 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.965 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.965 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.965 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:980 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 4128768~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:04.965 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451496+0000", 2024-05-08T08:54:04.966 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 55.430017356, 2024-05-08T08:54:04.966 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.090258985, 2024-05-08T08:54:04.966 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:04.966 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:04.966 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:04.966 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:04.966 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:04.966 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:04.966 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 980 2024-05-08T08:54:04.966 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.966 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:04.966 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.966 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:04.966 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451496+0000", 2024-05-08T08:54:04.966 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:04.967 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.967 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.967 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:04.967 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451496+0000", 2024-05-08T08:54:04.967 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:04.967 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.967 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.967 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:04.967 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451497+0000", 2024-05-08T08:54:04.967 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7599999999999997e-07 2024-05-08T08:54:04.967 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.967 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.967 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:04.967 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451500+0000", 2024-05-08T08:54:04.967 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7930000000000002e-06 2024-05-08T08:54:04.968 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.968 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.968 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:04.968 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451506+0000", 2024-05-08T08:54:04.968 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6980000000000001e-06 2024-05-08T08:54:04.968 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.968 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.968 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:04.968 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451512+0000", 2024-05-08T08:54:04.968 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2490000000000001e-06 2024-05-08T08:54:04.968 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.968 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.968 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:04.968 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541550+0000", 2024-05-08T08:54:04.968 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.090037961999999999 2024-05-08T08:54:04.969 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.969 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.969 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:04.969 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541647+0000", 2024-05-08T08:54:04.969 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6924999999999998e-05 2024-05-08T08:54:04.969 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:04.969 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:04.969 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:04.969 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541755+0000", 2024-05-08T08:54:04.969 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000108682 2024-05-08T08:54:04.969 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.969 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:04.969 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.969 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:04.969 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:04.970 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:04.971 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:54:04.972 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:04.974+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:04.998 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T08:54:05.071 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:05.074+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:05.086 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:05.086 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:54:05.086 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:54:05.086 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:05.095 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:54:05.139 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:54:05.139 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:54:05.156 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:54:05.156 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:54:05.171 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:05.174+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:05.173 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:54:05.173 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:54:05.190 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:54:05.191 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:54:05.201 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:05.201 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:05.208 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:54:05.208 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:54:05.211 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:54:05.225 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:54:05.225 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:54:05.272 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:05.275+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:05.316 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:05.316 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:05.324 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:54:05.408 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:05.411+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:05.471 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:05.471 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:05.473 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:05.476+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:05.480 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:54:05.523 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4] out_osds: [5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-05-08T08:54:05.523 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T08:54:05.523 INFO:tasks.thrashosds.thrasher:inject_pause on osd.4 2024-05-08T08:54:05.523 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-05-08T08:54:05.523 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-05-08T08:54:05.523 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok config set filestore_inject_stall 3 2024-05-08T08:54:05.575 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:05.578+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:05.628 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:05.628 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:05.639 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:54:05.667 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-05-08T08:54:05.674 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:05.674 INFO:teuthology.orchestra.run.smithi192.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-05-08T08:54:05.674 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:05.711 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:05.714+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:05.755 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:05.755 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:05.764 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:54:05.795 INFO:teuthology.orchestra.run.smithi196.stdout:truncating image image_client.0-clone37 from 0x2018000 (overlap 0xaa3e00) to 0x110600 2024-05-08T08:54:05.809 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:05.812+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:05.877 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:05.877 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:05.878 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:05.880+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:05.887 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:54:05.979 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:05.982+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:06.032 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:06.032 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:06.040 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:54:06.081 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:06.084+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:06.181 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:06.184+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:06.189 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:06.189 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:06.198 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:54:06.282 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:06.285+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:06.312 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:06.312 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:06.322 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:54:06.383 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:06.386+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:06.440 INFO:teuthology.orchestra.run.smithi196.stdout:checking clone #36, image image_client.0-clone36 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent37 2024-05-08T08:54:06.475 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:06.475 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:06.484 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:54:06.485 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:06.487+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:06.585 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:06.588+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:06.665 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:06.666 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:06.674 INFO:teuthology.orchestra.run.smithi059.stderr:noscrub is unset 2024-05-08T08:54:06.678 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:54:06.690 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:06.693+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:06.787 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:06.790+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:06.812 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:06.813 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:06.822 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:54:06.888 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:06.890+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:06.933 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:06.933 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:06.942 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:54:06.989 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:06.992+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:07.016 INFO:teuthology.orchestra.run.smithi196.stdout:4789 write 0x147a00 thru 0x1492ae (0x18af bytes) 2024-05-08T08:54:07.042 INFO:teuthology.orchestra.run.smithi196.stdout:4790 write 0x490cc00 thru 0x4918378 (0xb779 bytes) 2024-05-08T08:54:07.043 INFO:teuthology.orchestra.run.smithi196.stdout:4791 clone 39 order 19 su 2048 sc 4 2024-05-08T08:54:07.054 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:07.054 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:07.063 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:54:07.104 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:07.107+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:07.191 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:07.193+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:07.213 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:07.213 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:07.222 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:54:07.237 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T08:54:07.291 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:07.294+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:07.333 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:07.333 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:07.343 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:54:07.392 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:07.395+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:07.394 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:54:07.394 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:54:07.412 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:54:07.413 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:54:07.429 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:54:07.430 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:54:07.449 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:54:07.449 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:54:07.456 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:07.456 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:54:07.457 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:54:07.457 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:54:07.457 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.457 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:437 3.17 3:ea93a512:::rbd_data.1140100c9703.0000000000000077:head [set-alloc-hint object_size 2097152 write_size 2097152,write 1170432~1024] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e63)", 2024-05-08T08:54:07.457 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:08.390804+0000", 2024-05-08T08:54:07.457 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 59.067170582000003, 2024-05-08T08:54:07.457 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0406068690000001, 2024-05-08T08:54:07.457 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:07.457 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:07.457 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:54:07.457 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:07.457 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:07.457 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:07.457 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 437 2024-05-08T08:54:07.457 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.457 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:07.458 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.458 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:07.458 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390804+0000", 2024-05-08T08:54:07.458 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:07.458 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.458 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.458 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:07.458 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390804+0000", 2024-05-08T08:54:07.458 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:07.458 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.458 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.458 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:07.458 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390806+0000", 2024-05-08T08:54:07.459 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.666e-06 2024-05-08T08:54:07.459 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.459 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.459 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:07.459 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390818+0000", 2024-05-08T08:54:07.459 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2187999999999999e-05 2024-05-08T08:54:07.459 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.459 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.459 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:07.459 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390834+0000", 2024-05-08T08:54:07.459 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6191000000000001e-05 2024-05-08T08:54:07.459 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.459 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.459 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:07.459 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390845+0000", 2024-05-08T08:54:07.459 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0831e-05 2024-05-08T08:54:07.459 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.460 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.460 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:07.460 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390922+0000", 2024-05-08T08:54:07.460 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7253e-05 2024-05-08T08:54:07.460 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.460 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.460 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op must wait for map", 2024-05-08T08:54:07.460 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390930+0000", 2024-05-08T08:54:07.460 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8259999999999993e-06 2024-05-08T08:54:07.460 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.460 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.460 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:07.460 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392155+0000", 2024-05-08T08:54:07.460 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0012256649999999999 2024-05-08T08:54:07.460 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.461 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.461 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:54:07.461 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392165+0000", 2024-05-08T08:54:07.461 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5799999999999998e-06 2024-05-08T08:54:07.461 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.461 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.461 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:07.461 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.429596+0000", 2024-05-08T08:54:07.461 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0374313639999999 2024-05-08T08:54:07.461 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.461 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.461 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:07.461 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.429728+0000", 2024-05-08T08:54:07.461 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000131552 2024-05-08T08:54:07.462 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.462 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.462 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 0,1", 2024-05-08T08:54:07.462 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.429860+0000", 2024-05-08T08:54:07.462 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000132137 2024-05-08T08:54:07.462 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.462 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.462 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:54:07.462 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.430811+0000", 2024-05-08T08:54:07.462 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00095040200000000004 2024-05-08T08:54:07.462 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.462 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.462 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:54:07.462 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431126+0000", 2024-05-08T08:54:07.463 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000315006 2024-05-08T08:54:07.463 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.463 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.463 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:54:07.463 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431272+0000", 2024-05-08T08:54:07.463 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000146487 2024-05-08T08:54:07.463 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.463 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.463 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:54:07.463 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431315+0000", 2024-05-08T08:54:07.463 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2697000000000001e-05 2024-05-08T08:54:07.463 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.463 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.463 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:07.463 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431411+0000", 2024-05-08T08:54:07.464 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6024000000000001e-05 2024-05-08T08:54:07.464 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:07.464 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:07.464 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:07.464 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.464 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.464 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:439 3.17 3:ea93a512:::rbd_data.1140100c9703.0000000000000077:head [read 1163264~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-05-08T08:54:07.464 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:08.390871+0000", 2024-05-08T08:54:07.464 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 59.067103070999998, 2024-05-08T08:54:07.464 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0408790590000001, 2024-05-08T08:54:07.464 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:07.464 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:07.464 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:07.464 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:07.464 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:07.465 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:07.465 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 439 2024-05-08T08:54:07.465 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.465 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:07.465 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.465 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:07.465 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390871+0000", 2024-05-08T08:54:07.465 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:07.465 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.465 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.465 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:07.465 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390871+0000", 2024-05-08T08:54:07.466 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:07.466 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.466 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.466 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:07.466 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390872+0000", 2024-05-08T08:54:07.466 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6300000000000005e-07 2024-05-08T08:54:07.466 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.466 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.466 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:07.466 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390875+0000", 2024-05-08T08:54:07.466 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3459999999999998e-06 2024-05-08T08:54:07.466 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.466 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.466 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:07.467 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390891+0000", 2024-05-08T08:54:07.467 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5255e-05 2024-05-08T08:54:07.467 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.467 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.467 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:07.467 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390897+0000", 2024-05-08T08:54:07.467 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2360000000000001e-06 2024-05-08T08:54:07.467 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.467 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.467 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:07.467 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390995+0000", 2024-05-08T08:54:07.467 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8522000000000003e-05 2024-05-08T08:54:07.467 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.467 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.468 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting_for_map not empty", 2024-05-08T08:54:07.468 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.391005+0000", 2024-05-08T08:54:07.468 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1460000000000002e-06 2024-05-08T08:54:07.468 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.468 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.468 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:07.468 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392191+0000", 2024-05-08T08:54:07.468 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0011860270000000001 2024-05-08T08:54:07.468 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.468 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.468 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:54:07.468 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392197+0000", 2024-05-08T08:54:07.468 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4289999999999998e-06 2024-05-08T08:54:07.469 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.469 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.469 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:07.469 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.430596+0000", 2024-05-08T08:54:07.469 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0383989090000001 2024-05-08T08:54:07.469 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.469 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.469 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for rw locks", 2024-05-08T08:54:07.469 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.430676+0000", 2024-05-08T08:54:07.469 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9781000000000001e-05 2024-05-08T08:54:07.469 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.469 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.469 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:07.469 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431460+0000", 2024-05-08T08:54:07.469 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00078427300000000004 2024-05-08T08:54:07.470 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.470 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.470 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:07.470 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431540+0000", 2024-05-08T08:54:07.470 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0224999999999998e-05 2024-05-08T08:54:07.470 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.470 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.470 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:07.470 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431750+0000", 2024-05-08T08:54:07.470 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00021024699999999999 2024-05-08T08:54:07.470 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:07.470 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:07.470 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:07.470 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.470 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.470 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:945 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 4096000~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:07.471 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:54:07.471 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 58.006953504000002, 2024-05-08T08:54:07.471 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086774741000000002, 2024-05-08T08:54:07.471 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:07.471 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:07.471 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:07.471 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:07.471 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:07.471 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:07.471 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 945 2024-05-08T08:54:07.471 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.471 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:07.471 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.471 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:07.472 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:54:07.472 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:07.472 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.472 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.472 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:07.472 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:54:07.472 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:07.472 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.472 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.472 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:07.472 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:54:07.472 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3499999999999996e-07 2024-05-08T08:54:07.472 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.472 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.472 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:07.472 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451024+0000", 2024-05-08T08:54:07.473 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.627e-06 2024-05-08T08:54:07.473 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.473 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.473 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:07.473 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451037+0000", 2024-05-08T08:54:07.473 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2431e-05 2024-05-08T08:54:07.473 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.473 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.473 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:07.473 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451042+0000", 2024-05-08T08:54:07.473 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8409999999999996e-06 2024-05-08T08:54:07.473 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.473 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.473 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:07.473 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537608+0000", 2024-05-08T08:54:07.473 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086565243 2024-05-08T08:54:07.474 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.474 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.474 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:07.474 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537687+0000", 2024-05-08T08:54:07.474 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9519999999999998e-05 2024-05-08T08:54:07.474 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.474 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.474 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:07.474 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537796+0000", 2024-05-08T08:54:07.474 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000108544 2024-05-08T08:54:07.474 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:07.474 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:07.474 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:07.474 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.475 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.475 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:964 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 458752~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:07.475 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451051+0000", 2024-05-08T08:54:07.475 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 58.006923331000003, 2024-05-08T08:54:07.475 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086990490000000004, 2024-05-08T08:54:07.475 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:07.475 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:07.475 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:07.475 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:07.475 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:07.475 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:07.475 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 964 2024-05-08T08:54:07.475 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.475 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:07.475 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.475 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:07.475 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451051+0000", 2024-05-08T08:54:07.476 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:07.476 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.476 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.476 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:07.476 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451051+0000", 2024-05-08T08:54:07.476 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:07.476 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.476 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.476 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:07.476 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451052+0000", 2024-05-08T08:54:07.476 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7299999999999996e-07 2024-05-08T08:54:07.476 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.476 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.476 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:07.476 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451055+0000", 2024-05-08T08:54:07.477 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.2770000000000001e-06 2024-05-08T08:54:07.477 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.477 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.477 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:07.477 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451061+0000", 2024-05-08T08:54:07.477 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4060000000000002e-06 2024-05-08T08:54:07.477 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.477 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.477 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:07.477 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451066+0000", 2024-05-08T08:54:07.477 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0839999999999999e-06 2024-05-08T08:54:07.477 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.477 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.477 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:07.478 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537839+0000", 2024-05-08T08:54:07.478 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086772962999999995 2024-05-08T08:54:07.478 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.478 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.478 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:07.478 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537934+0000", 2024-05-08T08:54:07.478 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4742999999999997e-05 2024-05-08T08:54:07.478 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.478 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.478 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:07.478 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538042+0000", 2024-05-08T08:54:07.479 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000107444 2024-05-08T08:54:07.479 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:07.479 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:07.479 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:07.479 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.479 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.479 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:965 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 950272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:07.479 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451075+0000", 2024-05-08T08:54:07.479 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 58.006899269000002, 2024-05-08T08:54:07.479 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087230196999999995, 2024-05-08T08:54:07.479 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:07.479 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:07.479 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:07.479 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:07.479 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:07.480 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:07.480 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 965 2024-05-08T08:54:07.480 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.480 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:07.480 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.480 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:07.480 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451075+0000", 2024-05-08T08:54:07.480 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:07.480 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.480 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.480 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:07.480 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451075+0000", 2024-05-08T08:54:07.480 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:07.480 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.480 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.480 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:07.480 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451076+0000", 2024-05-08T08:54:07.481 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5000000000000002e-07 2024-05-08T08:54:07.481 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.481 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.481 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:07.481 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451078+0000", 2024-05-08T08:54:07.481 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.5639999999999999e-06 2024-05-08T08:54:07.481 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.481 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.481 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:07.481 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451085+0000", 2024-05-08T08:54:07.481 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5640000000000002e-06 2024-05-08T08:54:07.481 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.481 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.481 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:07.481 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451090+0000", 2024-05-08T08:54:07.482 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1699999999999996e-06 2024-05-08T08:54:07.482 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.482 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.482 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:07.482 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538106+0000", 2024-05-08T08:54:07.482 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087015888999999999 2024-05-08T08:54:07.482 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.482 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.482 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:07.482 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538190+0000", 2024-05-08T08:54:07.482 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3961999999999993e-05 2024-05-08T08:54:07.482 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.482 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.482 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:07.482 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538305+0000", 2024-05-08T08:54:07.483 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011529800000000001 2024-05-08T08:54:07.483 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:07.483 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:07.483 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:07.483 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.483 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.483 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:966 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 1441792~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:07.483 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:54:07.483 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 58.006875895999997, 2024-05-08T08:54:07.483 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087446323000000006, 2024-05-08T08:54:07.483 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:07.483 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:07.483 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:07.483 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:07.483 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:07.483 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:07.484 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 966 2024-05-08T08:54:07.484 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.484 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:07.484 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.484 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:07.484 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:54:07.484 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:07.484 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.484 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.484 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:07.484 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:54:07.484 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:07.484 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.484 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.484 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:07.484 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:54:07.485 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3700000000000001e-07 2024-05-08T08:54:07.485 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.485 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.485 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:07.485 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451101+0000", 2024-05-08T08:54:07.485 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2560000000000001e-06 2024-05-08T08:54:07.485 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.485 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.485 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:07.485 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451107+0000", 2024-05-08T08:54:07.485 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1070000000000001e-06 2024-05-08T08:54:07.485 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.485 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.485 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:07.485 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451112+0000", 2024-05-08T08:54:07.485 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1420000000000001e-06 2024-05-08T08:54:07.486 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.486 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.486 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:07.486 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538341+0000", 2024-05-08T08:54:07.486 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087228302999999993 2024-05-08T08:54:07.486 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.486 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.486 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:07.486 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538424+0000", 2024-05-08T08:54:07.486 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3497000000000004e-05 2024-05-08T08:54:07.486 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.486 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.486 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:07.487 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538545+0000", 2024-05-08T08:54:07.487 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000120581 2024-05-08T08:54:07.487 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:07.487 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:07.487 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:07.487 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.487 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.487 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:967 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:07.487 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451120+0000", 2024-05-08T08:54:07.487 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 58.006854421, 2024-05-08T08:54:07.487 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087632725999999994, 2024-05-08T08:54:07.487 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:07.487 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:07.487 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:07.487 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:07.488 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:07.488 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:07.488 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 967 2024-05-08T08:54:07.488 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.488 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:07.488 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.488 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:07.488 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451120+0000", 2024-05-08T08:54:07.488 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:07.488 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.488 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.488 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:07.488 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451120+0000", 2024-05-08T08:54:07.488 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:07.488 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.489 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.489 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:07.489 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451121+0000", 2024-05-08T08:54:07.489 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8800000000000002e-07 2024-05-08T08:54:07.489 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.489 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.489 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:07.489 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451130+0000", 2024-05-08T08:54:07.489 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2299999999999997e-06 2024-05-08T08:54:07.489 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.489 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.490 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:07.490 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451137+0000", 2024-05-08T08:54:07.490 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4669999999999998e-06 2024-05-08T08:54:07.490 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.490 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.490 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:07.490 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451143+0000", 2024-05-08T08:54:07.490 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2839999999999997e-06 2024-05-08T08:54:07.490 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.490 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.491 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:07.491 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538575+0000", 2024-05-08T08:54:07.491 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087432789999999996 2024-05-08T08:54:07.491 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.491 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.491 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:07.491 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538648+0000", 2024-05-08T08:54:07.492 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2877e-05 2024-05-08T08:54:07.492 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.492 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.492 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:07.492 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538753+0000", 2024-05-08T08:54:07.492 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010449 2024-05-08T08:54:07.492 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:07.492 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:07.492 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:07.492 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.492 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.492 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:968 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 2424832~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:07.492 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451152+0000", 2024-05-08T08:54:07.492 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 58.006822335999999, 2024-05-08T08:54:07.493 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087813325999999997, 2024-05-08T08:54:07.493 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:07.493 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:07.493 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:07.493 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:07.493 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:07.493 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:07.493 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 968 2024-05-08T08:54:07.493 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.493 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:07.493 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.493 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:07.493 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451152+0000", 2024-05-08T08:54:07.493 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:07.493 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.493 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.494 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:07.494 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451152+0000", 2024-05-08T08:54:07.494 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:07.494 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.494 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.494 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:07.494 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451153+0000", 2024-05-08T08:54:07.494 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3099999999999998e-07 2024-05-08T08:54:07.494 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.494 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.494 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:07.494 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451155+0000", 2024-05-08T08:54:07.494 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4760000000000001e-06 2024-05-08T08:54:07.494 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.494 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.495 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:07.495 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451162+0000", 2024-05-08T08:54:07.495 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5969999999999997e-06 2024-05-08T08:54:07.495 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.495 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.495 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:07.495 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451167+0000", 2024-05-08T08:54:07.495 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.977e-06 2024-05-08T08:54:07.495 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.495 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.495 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:07.495 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538781+0000", 2024-05-08T08:54:07.495 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087614846999999996 2024-05-08T08:54:07.495 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.495 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.495 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:07.496 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538857+0000", 2024-05-08T08:54:07.496 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5358000000000006e-05 2024-05-08T08:54:07.496 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.496 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.496 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:07.496 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538965+0000", 2024-05-08T08:54:07.496 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010864 2024-05-08T08:54:07.496 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:07.496 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:07.496 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:07.496 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.496 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.496 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:969 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 2916352~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:07.496 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:54:07.497 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 58.006800630999997, 2024-05-08T08:54:07.497 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087998980000000004, 2024-05-08T08:54:07.497 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:07.497 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:07.497 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:07.497 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:07.497 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:07.497 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:07.497 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 969 2024-05-08T08:54:07.497 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.497 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:07.498 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.498 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:07.498 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:54:07.498 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:07.498 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.498 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.498 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:07.498 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:54:07.498 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:07.498 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.498 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.498 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:07.498 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:54:07.499 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3599999999999999e-07 2024-05-08T08:54:07.499 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.499 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.499 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:07.499 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451177+0000", 2024-05-08T08:54:07.499 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4590000000000001e-06 2024-05-08T08:54:07.499 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.499 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.499 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:07.499 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451184+0000", 2024-05-08T08:54:07.500 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4109999999999999e-06 2024-05-08T08:54:07.500 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.500 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.500 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:07.500 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451191+0000", 2024-05-08T08:54:07.500 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7279999999999998e-06 2024-05-08T08:54:07.500 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.500 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.500 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:07.500 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538997+0000", 2024-05-08T08:54:07.500 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087806170000000003 2024-05-08T08:54:07.500 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.500 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.500 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:07.500 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539069+0000", 2024-05-08T08:54:07.500 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2373999999999995e-05 2024-05-08T08:54:07.501 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.501 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.501 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:07.501 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539173+0000", 2024-05-08T08:54:07.501 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000103402 2024-05-08T08:54:07.501 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:07.501 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:07.501 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:07.501 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.501 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.501 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:970 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 3407872~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:07.501 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451198+0000", 2024-05-08T08:54:07.501 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 58.006776033999998, 2024-05-08T08:54:07.501 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088183368999999998, 2024-05-08T08:54:07.501 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:07.502 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:07.502 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:07.502 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:07.502 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:07.502 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:07.502 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 970 2024-05-08T08:54:07.502 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.502 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:07.502 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.502 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:07.502 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451198+0000", 2024-05-08T08:54:07.502 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:07.502 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.502 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.503 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:07.503 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451198+0000", 2024-05-08T08:54:07.503 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:07.503 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.503 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.503 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:07.503 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451199+0000", 2024-05-08T08:54:07.503 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7400000000000003e-07 2024-05-08T08:54:07.503 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.503 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.503 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:07.503 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451211+0000", 2024-05-08T08:54:07.503 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2109999999999999e-05 2024-05-08T08:54:07.503 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.503 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.503 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:07.504 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451219+0000", 2024-05-08T08:54:07.504 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4170000000000003e-06 2024-05-08T08:54:07.504 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.504 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.504 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:07.504 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451224+0000", 2024-05-08T08:54:07.504 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3990000000000003e-06 2024-05-08T08:54:07.504 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.504 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.504 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:07.504 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539204+0000", 2024-05-08T08:54:07.504 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087980099000000006 2024-05-08T08:54:07.504 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.504 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.504 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:07.505 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539276+0000", 2024-05-08T08:54:07.505 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1513000000000005e-05 2024-05-08T08:54:07.505 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.505 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.505 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:07.505 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539382+0000", 2024-05-08T08:54:07.505 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010615699999999999 2024-05-08T08:54:07.505 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:07.505 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:07.505 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:07.505 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.505 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.505 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:971 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 3899392~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:07.505 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451233+0000", 2024-05-08T08:54:07.505 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 58.006741327, 2024-05-08T08:54:07.506 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088431525999999996, 2024-05-08T08:54:07.506 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:07.506 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:07.506 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:07.506 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:07.506 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:07.506 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:07.506 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 971 2024-05-08T08:54:07.506 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.506 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:07.506 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.506 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:07.506 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451233+0000", 2024-05-08T08:54:07.506 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:07.506 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.506 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.507 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:07.507 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451233+0000", 2024-05-08T08:54:07.507 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:07.507 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.507 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.507 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:07.507 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451234+0000", 2024-05-08T08:54:07.507 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8500000000000001e-07 2024-05-08T08:54:07.507 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.507 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.507 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:07.507 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451237+0000", 2024-05-08T08:54:07.507 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8320000000000002e-06 2024-05-08T08:54:07.507 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.507 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.508 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:07.508 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451243+0000", 2024-05-08T08:54:07.508 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6749999999999996e-06 2024-05-08T08:54:07.508 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.508 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.508 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:07.508 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451249+0000", 2024-05-08T08:54:07.508 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1229999999999997e-06 2024-05-08T08:54:07.508 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.508 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.508 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:07.508 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539412+0000", 2024-05-08T08:54:07.508 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088162668999999999 2024-05-08T08:54:07.509 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.509 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.509 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:07.509 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539505+0000", 2024-05-08T08:54:07.509 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3240999999999997e-05 2024-05-08T08:54:07.509 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.509 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.509 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:07.509 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539665+0000", 2024-05-08T08:54:07.509 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00015930100000000001 2024-05-08T08:54:07.509 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:07.509 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:07.509 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:07.510 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.510 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.510 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:972 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 196608~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:07.510 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451257+0000", 2024-05-08T08:54:07.510 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 58.006717381999998, 2024-05-08T08:54:07.510 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088719928000000003, 2024-05-08T08:54:07.510 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:07.510 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:07.510 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:07.510 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:07.510 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:07.510 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:07.510 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 972 2024-05-08T08:54:07.511 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.511 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:07.511 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.511 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:07.511 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451257+0000", 2024-05-08T08:54:07.511 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:07.511 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.511 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.511 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:07.511 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451257+0000", 2024-05-08T08:54:07.511 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:07.511 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.511 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.511 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:07.511 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451258+0000", 2024-05-08T08:54:07.511 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7999999999999996e-07 2024-05-08T08:54:07.512 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.512 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.512 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:07.512 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451260+0000", 2024-05-08T08:54:07.512 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4609999999999998e-06 2024-05-08T08:54:07.512 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.512 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.512 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:07.512 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451267+0000", 2024-05-08T08:54:07.512 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.81e-06 2024-05-08T08:54:07.512 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.512 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.512 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:07.512 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451273+0000", 2024-05-08T08:54:07.512 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0859999999999998e-06 2024-05-08T08:54:07.512 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.513 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.513 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:07.513 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539718+0000", 2024-05-08T08:54:07.513 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088445059000000006 2024-05-08T08:54:07.513 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.513 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.513 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:07.513 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539834+0000", 2024-05-08T08:54:07.513 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000116426 2024-05-08T08:54:07.513 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.513 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.513 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:07.513 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539977+0000", 2024-05-08T08:54:07.513 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014260600000000001 2024-05-08T08:54:07.513 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:07.513 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:07.514 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:07.514 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.514 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.514 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:973 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 688128~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:07.514 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:54:07.514 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 58.006694797999998, 2024-05-08T08:54:07.514 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088918496, 2024-05-08T08:54:07.514 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:07.514 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:07.514 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:07.514 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:07.514 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:07.515 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:07.515 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 973 2024-05-08T08:54:07.515 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.515 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:07.515 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.515 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:07.515 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:54:07.515 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:07.515 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.515 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.515 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:07.515 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:54:07.515 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:07.515 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.515 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.516 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:07.516 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:54:07.516 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9099999999999999e-07 2024-05-08T08:54:07.516 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.516 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.516 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:07.516 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451282+0000", 2024-05-08T08:54:07.516 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1550000000000001e-06 2024-05-08T08:54:07.516 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.516 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.516 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:07.516 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451288+0000", 2024-05-08T08:54:07.516 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8889999999999999e-06 2024-05-08T08:54:07.516 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.516 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.517 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:07.517 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451294+0000", 2024-05-08T08:54:07.517 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0850000000000002e-06 2024-05-08T08:54:07.517 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.517 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.517 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:07.517 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540011+0000", 2024-05-08T08:54:07.517 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088717314000000005 2024-05-08T08:54:07.517 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.517 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.517 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:07.517 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540089+0000", 2024-05-08T08:54:07.517 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7438e-05 2024-05-08T08:54:07.517 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.517 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.517 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:07.518 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540198+0000", 2024-05-08T08:54:07.518 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000109224 2024-05-08T08:54:07.518 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:07.518 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:07.518 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:07.518 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.518 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.518 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:974 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 1179648~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:07.518 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451301+0000", 2024-05-08T08:54:07.518 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 58.006673165000002, 2024-05-08T08:54:07.518 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089105078000000004, 2024-05-08T08:54:07.518 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:07.518 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:07.518 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:07.519 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:07.519 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:07.519 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:07.519 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 974 2024-05-08T08:54:07.519 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.519 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:07.519 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.519 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:07.519 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451301+0000", 2024-05-08T08:54:07.519 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:07.519 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.519 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.519 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:07.519 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451301+0000", 2024-05-08T08:54:07.520 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:07.520 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.520 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.520 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:07.520 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451302+0000", 2024-05-08T08:54:07.520 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7800000000000002e-07 2024-05-08T08:54:07.520 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.520 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.520 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:07.520 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451304+0000", 2024-05-08T08:54:07.520 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.65e-06 2024-05-08T08:54:07.520 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.521 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.521 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:07.521 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451311+0000", 2024-05-08T08:54:07.521 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.472e-06 2024-05-08T08:54:07.521 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.521 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.521 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:07.521 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451316+0000", 2024-05-08T08:54:07.521 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9189999999999998e-06 2024-05-08T08:54:07.521 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.521 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.521 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:07.521 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540234+0000", 2024-05-08T08:54:07.521 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088917752000000003 2024-05-08T08:54:07.521 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.521 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.521 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:07.522 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540308+0000", 2024-05-08T08:54:07.522 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4758000000000005e-05 2024-05-08T08:54:07.522 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.522 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.522 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:07.522 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540406+0000", 2024-05-08T08:54:07.522 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8048999999999996e-05 2024-05-08T08:54:07.522 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:07.522 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:07.522 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:07.522 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.522 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.523 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:975 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 1671168~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:07.523 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:54:07.523 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 58.006630835000003, 2024-05-08T08:54:07.523 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089294139999999994, 2024-05-08T08:54:07.523 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:07.523 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:07.523 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:07.523 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:07.523 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:07.523 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:07.523 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 975 2024-05-08T08:54:07.523 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.523 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:07.523 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.524 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:07.524 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:54:07.524 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:07.524 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.524 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.524 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:07.524 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:54:07.524 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:07.524 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.524 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.524 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:07.524 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:54:07.524 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1299999999999999e-07 2024-05-08T08:54:07.524 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.524 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.525 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:07.525 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451348+0000", 2024-05-08T08:54:07.525 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.484e-06 2024-05-08T08:54:07.525 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.525 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.525 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:07.525 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451356+0000", 2024-05-08T08:54:07.525 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8499999999999994e-06 2024-05-08T08:54:07.525 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.525 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.525 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:07.525 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451366+0000", 2024-05-08T08:54:07.525 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.9850000000000001e-06 2024-05-08T08:54:07.525 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.525 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.526 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:07.526 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540448+0000", 2024-05-08T08:54:07.526 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089082254 2024-05-08T08:54:07.526 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.526 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.526 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:07.526 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540528+0000", 2024-05-08T08:54:07.526 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0234999999999993e-05 2024-05-08T08:54:07.526 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.526 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.526 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:07.526 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540638+0000", 2024-05-08T08:54:07.526 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010951900000000001 2024-05-08T08:54:07.526 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:07.526 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:07.526 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:07.527 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.527 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.527 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:976 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 2162688~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:07.527 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451375+0000", 2024-05-08T08:54:07.527 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 58.006599088000002, 2024-05-08T08:54:07.527 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089472745000000006, 2024-05-08T08:54:07.527 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:07.527 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:07.527 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:07.527 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:07.527 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:07.527 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:07.527 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 976 2024-05-08T08:54:07.527 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.527 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:07.528 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.528 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:07.528 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451375+0000", 2024-05-08T08:54:07.528 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:07.528 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.528 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.528 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:07.528 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451375+0000", 2024-05-08T08:54:07.528 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:07.528 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.528 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.528 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:07.528 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451376+0000", 2024-05-08T08:54:07.528 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2399999999999998e-07 2024-05-08T08:54:07.528 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.529 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.529 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:07.529 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451379+0000", 2024-05-08T08:54:07.529 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6359999999999998e-06 2024-05-08T08:54:07.529 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.529 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.529 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:07.529 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451387+0000", 2024-05-08T08:54:07.529 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9929999999999995e-06 2024-05-08T08:54:07.529 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.529 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.529 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:07.529 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451393+0000", 2024-05-08T08:54:07.529 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4699999999999999e-06 2024-05-08T08:54:07.529 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.530 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.530 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:07.530 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540677+0000", 2024-05-08T08:54:07.530 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089283504999999999 2024-05-08T08:54:07.530 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.530 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.530 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:07.530 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540743+0000", 2024-05-08T08:54:07.530 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6303999999999994e-05 2024-05-08T08:54:07.530 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.530 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.531 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:07.531 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540848+0000", 2024-05-08T08:54:07.531 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000105213 2024-05-08T08:54:07.531 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:07.531 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:07.531 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:07.531 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.531 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.531 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:977 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 2654208~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:07.531 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:54:07.531 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 58.006573656, 2024-05-08T08:54:07.531 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089698473000000001, 2024-05-08T08:54:07.531 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:07.531 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:07.531 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:07.531 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:07.532 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:07.532 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:07.532 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 977 2024-05-08T08:54:07.532 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.532 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:07.532 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.532 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:07.532 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:54:07.532 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:07.532 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.532 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.532 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:07.532 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:54:07.532 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:07.532 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.532 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.533 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:07.533 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:54:07.533 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2899999999999999e-07 2024-05-08T08:54:07.533 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.533 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.533 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:07.533 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451404+0000", 2024-05-08T08:54:07.533 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6649999999999999e-06 2024-05-08T08:54:07.533 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.533 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.533 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:07.533 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451410+0000", 2024-05-08T08:54:07.533 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3269999999999997e-06 2024-05-08T08:54:07.533 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.533 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.533 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:07.534 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451415+0000", 2024-05-08T08:54:07.534 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9529999999999998e-06 2024-05-08T08:54:07.534 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.534 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.534 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:07.534 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540912+0000", 2024-05-08T08:54:07.534 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089496950000000006 2024-05-08T08:54:07.534 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.534 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.534 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:07.534 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541000+0000", 2024-05-08T08:54:07.534 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7429000000000006e-05 2024-05-08T08:54:07.534 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.534 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.534 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:07.534 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541099+0000", 2024-05-08T08:54:07.535 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.9720000000000001e-05 2024-05-08T08:54:07.535 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:07.535 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:07.535 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:07.535 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.535 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.535 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:978 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 3145728~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:07.535 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:54:07.535 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 58.006552448000001, 2024-05-08T08:54:07.535 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089900635000000007, 2024-05-08T08:54:07.535 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:07.535 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:07.535 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:07.535 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:07.535 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:07.535 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:07.536 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 978 2024-05-08T08:54:07.536 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.536 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:07.536 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.536 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:07.536 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:54:07.536 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:07.536 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.536 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.536 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:07.536 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:54:07.536 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:07.536 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.536 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.536 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:07.536 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:54:07.537 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9400000000000001e-07 2024-05-08T08:54:07.537 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.537 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.537 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:07.537 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451425+0000", 2024-05-08T08:54:07.537 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2460000000000002e-06 2024-05-08T08:54:07.537 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.537 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.537 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:07.537 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451431+0000", 2024-05-08T08:54:07.537 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2419999999999997e-06 2024-05-08T08:54:07.537 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.537 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.537 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:07.537 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451436+0000", 2024-05-08T08:54:07.537 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.887e-06 2024-05-08T08:54:07.538 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.538 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.538 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:07.538 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541152+0000", 2024-05-08T08:54:07.538 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089716194999999999 2024-05-08T08:54:07.538 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.538 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.538 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:07.538 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541224+0000", 2024-05-08T08:54:07.538 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2416000000000006e-05 2024-05-08T08:54:07.538 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.538 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.538 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:07.538 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541323+0000", 2024-05-08T08:54:07.539 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8255e-05 2024-05-08T08:54:07.539 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:07.539 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:07.539 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:07.539 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.539 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.539 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:979 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 3637248~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:07.539 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451462+0000", 2024-05-08T08:54:07.539 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 58.006511998999997, 2024-05-08T08:54:07.539 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.090036441999999994, 2024-05-08T08:54:07.539 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:07.539 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:07.539 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:07.539 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:07.539 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:07.540 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:07.540 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 979 2024-05-08T08:54:07.540 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.540 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:07.540 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.540 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:07.540 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451462+0000", 2024-05-08T08:54:07.540 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:07.540 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.540 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.540 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:07.540 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451462+0000", 2024-05-08T08:54:07.540 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:07.541 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.541 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.541 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:07.541 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451463+0000", 2024-05-08T08:54:07.541 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4300000000000002e-07 2024-05-08T08:54:07.541 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.541 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.541 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:07.541 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451471+0000", 2024-05-08T08:54:07.541 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0800000000000006e-06 2024-05-08T08:54:07.541 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.541 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.541 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:07.542 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451481+0000", 2024-05-08T08:54:07.542 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.38e-06 2024-05-08T08:54:07.542 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.542 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.542 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:07.542 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451487+0000", 2024-05-08T08:54:07.542 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4590000000000003e-06 2024-05-08T08:54:07.542 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.542 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.542 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:07.542 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541350+0000", 2024-05-08T08:54:07.542 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089862599000000001 2024-05-08T08:54:07.542 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.542 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.543 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:07.543 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541407+0000", 2024-05-08T08:54:07.543 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7667999999999997e-05 2024-05-08T08:54:07.543 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.543 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.543 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:07.543 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541499+0000", 2024-05-08T08:54:07.543 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1513000000000003e-05 2024-05-08T08:54:07.543 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:07.543 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:07.543 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:07.543 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.543 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.543 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:980 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 4128768~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:07.544 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451496+0000", 2024-05-08T08:54:07.544 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 58.006478242999997, 2024-05-08T08:54:07.544 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.090258985, 2024-05-08T08:54:07.544 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:07.544 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:07.544 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:07.544 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:07.544 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:07.544 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:07.544 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 980 2024-05-08T08:54:07.544 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.544 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:07.544 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.544 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:07.544 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451496+0000", 2024-05-08T08:54:07.544 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:07.545 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.545 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.545 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:07.545 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451496+0000", 2024-05-08T08:54:07.545 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:07.545 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.545 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.545 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:07.545 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451497+0000", 2024-05-08T08:54:07.545 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7599999999999997e-07 2024-05-08T08:54:07.545 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.545 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.545 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:07.545 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451500+0000", 2024-05-08T08:54:07.546 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7930000000000002e-06 2024-05-08T08:54:07.546 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.546 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.546 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:07.546 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451506+0000", 2024-05-08T08:54:07.546 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6980000000000001e-06 2024-05-08T08:54:07.546 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.546 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.546 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:07.546 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451512+0000", 2024-05-08T08:54:07.546 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2490000000000001e-06 2024-05-08T08:54:07.546 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.546 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.546 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:07.546 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541550+0000", 2024-05-08T08:54:07.546 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.090037961999999999 2024-05-08T08:54:07.547 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.547 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.547 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:07.547 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541647+0000", 2024-05-08T08:54:07.547 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6924999999999998e-05 2024-05-08T08:54:07.547 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:07.547 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:07.547 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:07.547 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541755+0000", 2024-05-08T08:54:07.547 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000108682 2024-05-08T08:54:07.547 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:07.547 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:07.547 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:07.547 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:07.547 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:07.547 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:07.549 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:54:07.549 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:54:07.550 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:54:07.550 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:54:07.551 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:54:07.551 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:07.551+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:07.650 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:07.653+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:07.658 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:07.658 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:54:07.658 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:54:07.659 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:07.667 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:54:07.751 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:07.754+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:07.804 INFO:teuthology.orchestra.run.smithi196.stdout:truncating image image_client.0-clone38 from 0x4918400 (overlap 0x2018000) to 0x100c000 2024-05-08T08:54:07.825 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:07.825 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:54:07.825 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:54:07.825 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:54:07.825 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:07.838 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:54:07.852 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:07.854+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:07.952 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:07.955+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:07.989 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:07.989 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:54:07.989 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:54:07.989 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:54:07.989 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:07.989 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4621.0:9089 3.1f e33/18 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:head v 33'117, mlcod=31'115)", 2024-05-08T08:54:07.989 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:54:07.989 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 95.471179340000006, 2024-05-08T08:54:07.989 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.111244524, 2024-05-08T08:54:07.989 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:07.989 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:07.990 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:54:07.990 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:07.990 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:07.990 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:07.990 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:54:07.990 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:07.990 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:07.990 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:07.990 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:07.990 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:54:07.990 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:07.990 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:07.990 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:07.990 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:07.990 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:54:07.991 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5900000000000001e-07 2024-05-08T08:54:07.991 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:07.991 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:07.991 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:07.991 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539600+0000", 2024-05-08T08:54:07.991 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.019981364000000001 2024-05-08T08:54:07.991 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:07.991 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:07.991 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:07.991 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539624+0000", 2024-05-08T08:54:07.991 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4043e-05 2024-05-08T08:54:07.991 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:07.991 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:07.991 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:07.992 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539635+0000", 2024-05-08T08:54:07.992 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1511e-05 2024-05-08T08:54:07.992 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:07.992 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:07.992 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:07.992 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539728+0000", 2024-05-08T08:54:07.992 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2498999999999996e-05 2024-05-08T08:54:07.992 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:07.992 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:07.992 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:07.992 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539746+0000", 2024-05-08T08:54:07.992 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8289000000000001e-05 2024-05-08T08:54:07.992 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:07.992 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:07.992 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:54:07.993 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.630822+0000", 2024-05-08T08:54:07.993 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.091076283999999993 2024-05-08T08:54:07.993 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:07.993 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:07.993 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:07.993 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.630863+0000", 2024-05-08T08:54:07.993 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0374999999999999e-05 2024-05-08T08:54:07.993 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:07.993 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:07.993 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:07.993 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:07.993 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:07.993 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4795.0:230 3.1f e40/18 3:f8db1a56:::rbd_data.11402b26385.0000000000000000:head v 40'171, mlcod=39'168)", 2024-05-08T08:54:07.993 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:45.927773+0000", 2024-05-08T08:54:07.993 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 82.063024256999995, 2024-05-08T08:54:07.994 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045892405999999997, 2024-05-08T08:54:07.994 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:07.994 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:07.994 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:54:07.994 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:07.994 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:07.994 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:07.994 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.927773+0000", 2024-05-08T08:54:07.994 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:07.994 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:07.994 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:07.994 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:07.994 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.927773+0000", 2024-05-08T08:54:07.994 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:07.995 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:07.995 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:07.995 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:07.995 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.927774+0000", 2024-05-08T08:54:07.995 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0500000000000002e-07 2024-05-08T08:54:07.995 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:07.995 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:07.995 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:07.995 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.931581+0000", 2024-05-08T08:54:07.995 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0038071799999999999 2024-05-08T08:54:07.995 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:07.995 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:07.995 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:07.995 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.931599+0000", 2024-05-08T08:54:07.995 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7479e-05 2024-05-08T08:54:07.996 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:07.996 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:07.996 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:07.996 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.931624+0000", 2024-05-08T08:54:07.996 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5372999999999999e-05 2024-05-08T08:54:07.996 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:07.996 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:07.996 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:07.996 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.931698+0000", 2024-05-08T08:54:07.996 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4043999999999998e-05 2024-05-08T08:54:07.996 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:07.996 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:07.996 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:07.996 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.931725+0000", 2024-05-08T08:54:07.996 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6883e-05 2024-05-08T08:54:07.997 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:07.997 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:07.997 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:54:07.997 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.973623+0000", 2024-05-08T08:54:07.997 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.041898283000000001 2024-05-08T08:54:07.997 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:07.997 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:07.997 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:07.997 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.973666+0000", 2024-05-08T08:54:07.997 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2358999999999999e-05 2024-05-08T08:54:07.997 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:07.997 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:07.997 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:07.997 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:07.997 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:07.998 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4195.0:4 3.11 3:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-05-08T08:54:07.998 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:49.764654+0000", 2024-05-08T08:54:07.998 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 78.226143798999999, 2024-05-08T08:54:07.998 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.30714275499999999, 2024-05-08T08:54:07.998 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:07.998 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:07.998 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-05-08T08:54:07.998 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:07.998 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4195", 2024-05-08T08:54:07.998 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/59179521", 2024-05-08T08:54:07.998 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4 2024-05-08T08:54:07.998 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:07.998 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:07.998 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:07.999 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:07.999 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764654+0000", 2024-05-08T08:54:07.999 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:07.999 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:07.999 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:07.999 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:07.999 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764654+0000", 2024-05-08T08:54:07.999 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:07.999 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:07.999 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:07.999 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:07.999 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764655+0000", 2024-05-08T08:54:07.999 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.452e-06 2024-05-08T08:54:07.999 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:07.999 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.000 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:08.000 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764661+0000", 2024-05-08T08:54:08.000 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5369999999999997e-06 2024-05-08T08:54:08.000 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.000 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.000 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:08.000 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764671+0000", 2024-05-08T08:54:08.000 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.028e-05 2024-05-08T08:54:08.000 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.000 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.000 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:08.000 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764680+0000", 2024-05-08T08:54:08.000 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0629999999999995e-06 2024-05-08T08:54:08.000 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.000 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.001 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:08.001 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764753+0000", 2024-05-08T08:54:08.001 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2679e-05 2024-05-08T08:54:08.001 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.001 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.001 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-05-08T08:54:08.001 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764765+0000", 2024-05-08T08:54:08.001 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2415000000000001e-05 2024-05-08T08:54:08.001 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.001 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.001 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:08.001 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:50.071692+0000", 2024-05-08T08:54:08.001 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.306927324 2024-05-08T08:54:08.001 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.001 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.001 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:08.002 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:50.071796+0000", 2024-05-08T08:54:08.002 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104005 2024-05-08T08:54:08.002 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:08.002 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:08.002 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:08.002 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.002 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.002 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22683 3.15 3:a907109d:::rbd_data.1140d3fc54c.000000000000004f:18 [read 1335296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:08.002 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.097686+0000", 2024-05-08T08:54:08.002 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 35.893111124999997, 2024-05-08T08:54:08.002 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042114716000000003, 2024-05-08T08:54:08.002 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:08.002 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:08.002 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:08.003 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:08.003 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:54:08.003 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:08.003 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22683 2024-05-08T08:54:08.003 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.003 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:08.003 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.003 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:08.003 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097686+0000", 2024-05-08T08:54:08.003 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:08.003 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.003 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.003 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:08.003 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097686+0000", 2024-05-08T08:54:08.003 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:08.003 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.004 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.004 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:08.004 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097687+0000", 2024-05-08T08:54:08.004 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3999999999999997e-07 2024-05-08T08:54:08.004 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.004 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.004 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:08.004 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097688+0000", 2024-05-08T08:54:08.004 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6789999999999999e-06 2024-05-08T08:54:08.004 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.004 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.004 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:08.004 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097693+0000", 2024-05-08T08:54:08.004 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.045e-06 2024-05-08T08:54:08.004 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.005 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.005 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:08.005 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097697+0000", 2024-05-08T08:54:08.005 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0260000000000001e-06 2024-05-08T08:54:08.005 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.005 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.005 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:08.005 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139725+0000", 2024-05-08T08:54:08.005 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042027737000000003 2024-05-08T08:54:08.005 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.005 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.005 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:08.005 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139764+0000", 2024-05-08T08:54:08.005 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8668999999999999e-05 2024-05-08T08:54:08.005 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.006 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.006 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:08.006 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139801+0000", 2024-05-08T08:54:08.006 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7219999999999999e-05 2024-05-08T08:54:08.006 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:08.006 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:08.006 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:08.006 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.006 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.006 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22684 3.15 3:a907109d:::rbd_data.1140d3fc54c.000000000000004f:18 [read 1400832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:08.006 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.097703+0000", 2024-05-08T08:54:08.006 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 35.893094771999998, 2024-05-08T08:54:08.006 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042198929000000003, 2024-05-08T08:54:08.006 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:08.007 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:08.007 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:08.007 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:08.007 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:54:08.007 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:08.007 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22684 2024-05-08T08:54:08.007 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.007 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:08.007 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.007 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:08.007 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097703+0000", 2024-05-08T08:54:08.007 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:08.007 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.007 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.007 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:08.007 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097703+0000", 2024-05-08T08:54:08.008 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:08.008 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.008 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.008 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:08.008 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097703+0000", 2024-05-08T08:54:08.008 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4400000000000001e-07 2024-05-08T08:54:08.008 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.008 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.008 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:08.008 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097705+0000", 2024-05-08T08:54:08.008 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9199999999999998e-06 2024-05-08T08:54:08.008 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.008 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.008 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:08.008 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097711+0000", 2024-05-08T08:54:08.008 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5559999999999998e-06 2024-05-08T08:54:08.008 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.009 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.009 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:08.009 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097715+0000", 2024-05-08T08:54:08.009 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3159999999999998e-06 2024-05-08T08:54:08.009 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.009 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.009 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:08.009 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139825+0000", 2024-05-08T08:54:08.009 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042110284999999997 2024-05-08T08:54:08.009 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.009 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.009 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:08.009 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139864+0000", 2024-05-08T08:54:08.009 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8980000000000003e-05 2024-05-08T08:54:08.009 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.009 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.009 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:08.010 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139902+0000", 2024-05-08T08:54:08.010 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7527999999999997e-05 2024-05-08T08:54:08.010 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:08.010 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:08.010 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:08.010 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.010 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.010 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22685 3.15 3:a907109d:::rbd_data.1140d3fc54c.000000000000004f:18 [read 1466368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:08.010 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.097721+0000", 2024-05-08T08:54:08.010 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 35.893076115, 2024-05-08T08:54:08.010 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042287791999999998, 2024-05-08T08:54:08.010 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:08.010 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:08.010 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:08.010 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:08.010 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:54:08.011 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:08.011 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22685 2024-05-08T08:54:08.011 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.011 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:08.011 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.011 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:08.011 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097721+0000", 2024-05-08T08:54:08.011 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:08.011 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.011 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.011 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:08.011 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097721+0000", 2024-05-08T08:54:08.011 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:08.011 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.011 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.011 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:08.011 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097722+0000", 2024-05-08T08:54:08.012 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2500000000000001e-07 2024-05-08T08:54:08.012 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.012 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.012 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:08.012 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097724+0000", 2024-05-08T08:54:08.012 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.835e-06 2024-05-08T08:54:08.012 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.012 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.012 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:08.012 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097729+0000", 2024-05-08T08:54:08.012 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5029999999999997e-06 2024-05-08T08:54:08.012 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.012 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.012 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:08.012 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097733+0000", 2024-05-08T08:54:08.013 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3309999999999997e-06 2024-05-08T08:54:08.013 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.013 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.013 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:08.013 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139923+0000", 2024-05-08T08:54:08.013 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042189988999999997 2024-05-08T08:54:08.013 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.013 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.013 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:08.013 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139970+0000", 2024-05-08T08:54:08.013 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6776000000000001e-05 2024-05-08T08:54:08.013 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.013 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.013 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:08.013 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140009+0000", 2024-05-08T08:54:08.013 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9032999999999997e-05 2024-05-08T08:54:08.013 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:08.014 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:08.014 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:08.014 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.014 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.014 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22686 3.15 3:a907109d:::rbd_data.1140d3fc54c.000000000000004f:18 [read 1531904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:08.014 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.097739+0000", 2024-05-08T08:54:08.014 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 35.893058115999999, 2024-05-08T08:54:08.014 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042371008000000002, 2024-05-08T08:54:08.014 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:08.014 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:08.014 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:08.014 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:08.014 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:54:08.014 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:08.014 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22686 2024-05-08T08:54:08.015 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.015 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:08.015 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.015 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:08.015 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097739+0000", 2024-05-08T08:54:08.015 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:08.015 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.015 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.015 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:08.015 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097739+0000", 2024-05-08T08:54:08.015 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:08.015 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.015 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.015 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:08.015 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097740+0000", 2024-05-08T08:54:08.015 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8599999999999999e-07 2024-05-08T08:54:08.016 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.016 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.016 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:08.016 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097742+0000", 2024-05-08T08:54:08.016 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9410000000000001e-06 2024-05-08T08:54:08.016 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.016 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.016 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:08.016 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097747+0000", 2024-05-08T08:54:08.016 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2079999999999999e-06 2024-05-08T08:54:08.016 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.016 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.016 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:08.016 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097751+0000", 2024-05-08T08:54:08.016 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2250000000000002e-06 2024-05-08T08:54:08.017 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.017 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.017 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:08.017 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140032+0000", 2024-05-08T08:54:08.017 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042281244000000003 2024-05-08T08:54:08.017 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.017 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.017 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:08.017 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140074+0000", 2024-05-08T08:54:08.017 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1316999999999998e-05 2024-05-08T08:54:08.017 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.017 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.017 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:08.017 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140110+0000", 2024-05-08T08:54:08.017 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6687000000000003e-05 2024-05-08T08:54:08.018 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:08.018 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:08.018 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:08.018 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.018 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.018 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22716 3.15 3:a88fdee9:::rbd_data.1140d3fc54c.000000000000004d:18 [read 1097728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:08.018 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.098153+0000", 2024-05-08T08:54:08.018 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 35.892644408999999, 2024-05-08T08:54:08.018 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042058535000000001, 2024-05-08T08:54:08.018 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:08.018 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:08.018 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:08.018 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:08.018 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:54:08.018 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:08.018 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22716 2024-05-08T08:54:08.019 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.019 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:08.019 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.019 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:08.019 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098153+0000", 2024-05-08T08:54:08.019 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:08.019 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.019 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.019 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:08.019 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098153+0000", 2024-05-08T08:54:08.019 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:08.019 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.019 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.019 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:08.020 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098154+0000", 2024-05-08T08:54:08.020 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.13e-07 2024-05-08T08:54:08.020 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.020 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.020 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:08.020 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098161+0000", 2024-05-08T08:54:08.020 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7360000000000001e-06 2024-05-08T08:54:08.020 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.020 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.020 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:08.020 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098171+0000", 2024-05-08T08:54:08.020 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1039999999999996e-06 2024-05-08T08:54:08.020 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.020 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.020 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:08.021 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098177+0000", 2024-05-08T08:54:08.021 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5760000000000002e-06 2024-05-08T08:54:08.021 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.021 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.021 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:08.021 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140135+0000", 2024-05-08T08:54:08.021 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.041957842000000002 2024-05-08T08:54:08.021 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.021 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.021 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:08.021 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140174+0000", 2024-05-08T08:54:08.021 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8902999999999997e-05 2024-05-08T08:54:08.021 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.021 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.021 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:08.021 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140212+0000", 2024-05-08T08:54:08.021 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7761e-05 2024-05-08T08:54:08.022 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:08.022 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:08.022 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:08.022 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.022 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.022 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22717 3.15 3:a88fdee9:::rbd_data.1140d3fc54c.000000000000004d:18 [read 1163264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:08.022 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.098188+0000", 2024-05-08T08:54:08.022 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 35.892609686999997, 2024-05-08T08:54:08.022 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042124907000000003, 2024-05-08T08:54:08.022 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:08.022 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:08.022 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:08.022 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:08.022 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:54:08.022 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:08.023 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22717 2024-05-08T08:54:08.023 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.023 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:08.023 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.023 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:08.023 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098188+0000", 2024-05-08T08:54:08.023 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:08.023 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.023 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.023 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:08.023 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098188+0000", 2024-05-08T08:54:08.023 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:08.023 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.023 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.024 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:08.024 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098188+0000", 2024-05-08T08:54:08.024 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3799999999999998e-07 2024-05-08T08:54:08.024 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.024 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.024 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:08.024 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098190+0000", 2024-05-08T08:54:08.024 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.114e-06 2024-05-08T08:54:08.024 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.024 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.024 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:08.024 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098197+0000", 2024-05-08T08:54:08.024 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9959999999999996e-06 2024-05-08T08:54:08.024 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.024 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.025 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:08.025 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098202+0000", 2024-05-08T08:54:08.025 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9110000000000001e-06 2024-05-08T08:54:08.025 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.025 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.025 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:08.025 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140235+0000", 2024-05-08T08:54:08.025 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042032913999999998 2024-05-08T08:54:08.025 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.025 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.025 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:08.025 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140275+0000", 2024-05-08T08:54:08.025 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9564999999999998e-05 2024-05-08T08:54:08.025 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.025 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.025 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:08.026 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140313+0000", 2024-05-08T08:54:08.026 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7969000000000002e-05 2024-05-08T08:54:08.026 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:08.026 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:08.026 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:08.026 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.026 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.026 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22718 3.15 3:a88fdee9:::rbd_data.1140d3fc54c.000000000000004d:18 [read 1228800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:08.026 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.098209+0000", 2024-05-08T08:54:08.026 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 35.892588541000002, 2024-05-08T08:54:08.026 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.04220409, 2024-05-08T08:54:08.026 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:08.026 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:08.026 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:08.026 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:08.027 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:54:08.027 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:08.027 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22718 2024-05-08T08:54:08.027 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.027 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:08.027 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.027 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:08.027 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098209+0000", 2024-05-08T08:54:08.027 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:08.027 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.027 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.027 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:08.027 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098209+0000", 2024-05-08T08:54:08.027 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:08.027 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.028 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.028 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:08.028 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098209+0000", 2024-05-08T08:54:08.028 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9299999999999999e-07 2024-05-08T08:54:08.028 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.028 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.028 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:08.028 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098212+0000", 2024-05-08T08:54:08.028 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.21e-06 2024-05-08T08:54:08.028 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.028 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.028 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:08.028 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098219+0000", 2024-05-08T08:54:08.028 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0269999999999998e-06 2024-05-08T08:54:08.028 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.029 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.029 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:08.029 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098223+0000", 2024-05-08T08:54:08.029 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7319999999999998e-06 2024-05-08T08:54:08.029 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.029 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.029 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:08.029 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140338+0000", 2024-05-08T08:54:08.029 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042114390000000002 2024-05-08T08:54:08.029 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.029 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.029 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:08.029 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140376+0000", 2024-05-08T08:54:08.029 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8414999999999997e-05 2024-05-08T08:54:08.029 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.029 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.030 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:08.030 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140413+0000", 2024-05-08T08:54:08.030 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6922999999999999e-05 2024-05-08T08:54:08.030 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:08.030 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:08.030 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:08.030 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.030 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.030 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22719 3.15 3:a88fdee9:::rbd_data.1140d3fc54c.000000000000004d:18 [read 1294336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:08.030 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.098229+0000", 2024-05-08T08:54:08.030 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 35.892568689999997, 2024-05-08T08:54:08.030 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042283016, 2024-05-08T08:54:08.030 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:08.030 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:08.031 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:08.031 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:08.031 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:54:08.031 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:08.031 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22719 2024-05-08T08:54:08.031 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.031 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:08.031 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.031 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:08.031 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098229+0000", 2024-05-08T08:54:08.031 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:08.031 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.031 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.031 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:08.031 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098229+0000", 2024-05-08T08:54:08.031 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:08.032 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.032 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.032 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:08.032 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098229+0000", 2024-05-08T08:54:08.032 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:54:08.032 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.032 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.032 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:08.032 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098231+0000", 2024-05-08T08:54:08.032 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.81e-06 2024-05-08T08:54:08.032 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.032 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.032 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:08.032 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098237+0000", 2024-05-08T08:54:08.032 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3389999999999998e-06 2024-05-08T08:54:08.032 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.032 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.033 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:08.033 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098243+0000", 2024-05-08T08:54:08.033 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.648e-06 2024-05-08T08:54:08.033 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.033 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.033 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:08.033 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140436+0000", 2024-05-08T08:54:08.033 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042192738 2024-05-08T08:54:08.033 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.033 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.033 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:08.033 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140475+0000", 2024-05-08T08:54:08.033 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9279999999999997e-05 2024-05-08T08:54:08.033 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.033 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.034 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:08.034 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140512+0000", 2024-05-08T08:54:08.034 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6847000000000001e-05 2024-05-08T08:54:08.034 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:08.034 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:08.034 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:08.034 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.034 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.034 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22720 3.15 3:a88fdee9:::rbd_data.1140d3fc54c.000000000000004d:18 [read 1359872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:08.034 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.098248+0000", 2024-05-08T08:54:08.034 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 35.892549179, 2024-05-08T08:54:08.034 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042360800999999997, 2024-05-08T08:54:08.034 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:08.034 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:08.034 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:08.034 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:08.035 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:54:08.035 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:08.035 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22720 2024-05-08T08:54:08.035 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.035 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:08.035 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.035 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:08.035 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098248+0000", 2024-05-08T08:54:08.035 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:08.035 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.035 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.035 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:08.035 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098248+0000", 2024-05-08T08:54:08.035 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:08.035 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.035 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.036 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:08.036 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098491+0000", 2024-05-08T08:54:08.036 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000242659 2024-05-08T08:54:08.036 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.036 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.036 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:08.036 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098495+0000", 2024-05-08T08:54:08.036 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.799e-06 2024-05-08T08:54:08.036 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.036 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.036 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:08.036 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098505+0000", 2024-05-08T08:54:08.036 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7079999999999992e-06 2024-05-08T08:54:08.036 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.036 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.036 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:08.037 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098511+0000", 2024-05-08T08:54:08.037 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4370000000000004e-06 2024-05-08T08:54:08.037 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.037 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.037 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:08.037 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140533+0000", 2024-05-08T08:54:08.037 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042022371000000003 2024-05-08T08:54:08.037 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.037 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.037 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:08.037 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140571+0000", 2024-05-08T08:54:08.037 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7700000000000002e-05 2024-05-08T08:54:08.037 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.037 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.037 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:08.037 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140609+0000", 2024-05-08T08:54:08.038 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8127000000000003e-05 2024-05-08T08:54:08.038 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:08.038 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:08.038 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:08.038 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.038 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.038 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22721 3.15 3:a88fdee9:::rbd_data.1140d3fc54c.000000000000004d:18 [read 1425408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:08.038 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.098522+0000", 2024-05-08T08:54:08.038 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 35.892275554999998, 2024-05-08T08:54:08.038 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042185779999999999, 2024-05-08T08:54:08.038 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:08.038 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:08.038 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:08.038 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:08.038 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:54:08.039 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:08.039 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22721 2024-05-08T08:54:08.039 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.039 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:08.039 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.039 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:08.039 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098522+0000", 2024-05-08T08:54:08.039 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:08.039 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.039 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.039 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:08.039 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098522+0000", 2024-05-08T08:54:08.039 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:08.039 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.039 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.039 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:08.040 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098708+0000", 2024-05-08T08:54:08.040 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00018629700000000001 2024-05-08T08:54:08.040 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.040 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.040 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:08.040 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098719+0000", 2024-05-08T08:54:08.040 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0992000000000001e-05 2024-05-08T08:54:08.040 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.040 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.040 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:08.040 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098730+0000", 2024-05-08T08:54:08.040 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0431999999999999e-05 2024-05-08T08:54:08.040 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.040 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.041 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:08.041 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098737+0000", 2024-05-08T08:54:08.041 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.063e-06 2024-05-08T08:54:08.041 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.041 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.041 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:08.041 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140631+0000", 2024-05-08T08:54:08.041 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.041894503999999999 2024-05-08T08:54:08.041 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.041 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.041 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:08.041 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140671+0000", 2024-05-08T08:54:08.041 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9780999999999997e-05 2024-05-08T08:54:08.041 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.041 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.042 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:08.042 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140708+0000", 2024-05-08T08:54:08.042 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6711000000000001e-05 2024-05-08T08:54:08.042 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:08.042 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:08.042 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:08.042 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.042 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.042 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22722 3.15 3:a88fdee9:::rbd_data.1140d3fc54c.000000000000004d:18 [read 1490944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:08.042 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.098753+0000", 2024-05-08T08:54:08.042 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 35.892044550999998, 2024-05-08T08:54:08.042 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042053944000000003, 2024-05-08T08:54:08.042 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:08.042 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:08.042 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:08.042 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:08.043 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:54:08.043 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:08.043 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22722 2024-05-08T08:54:08.043 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.043 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:08.043 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.043 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:08.043 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098753+0000", 2024-05-08T08:54:08.043 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:08.043 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.043 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.043 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:08.043 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098753+0000", 2024-05-08T08:54:08.043 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:08.043 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.043 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.044 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:08.044 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098753+0000", 2024-05-08T08:54:08.044 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1399999999999997e-07 2024-05-08T08:54:08.044 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.044 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.044 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:08.044 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098756+0000", 2024-05-08T08:54:08.044 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9780000000000001e-06 2024-05-08T08:54:08.044 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.044 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.044 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:08.044 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098764+0000", 2024-05-08T08:54:08.044 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4340000000000003e-06 2024-05-08T08:54:08.044 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.044 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.045 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:08.045 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098770+0000", 2024-05-08T08:54:08.045 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8080000000000001e-06 2024-05-08T08:54:08.045 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.045 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.045 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:08.045 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140731+0000", 2024-05-08T08:54:08.045 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.041961131999999998 2024-05-08T08:54:08.045 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.045 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.045 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:08.045 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140769+0000", 2024-05-08T08:54:08.045 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.773e-05 2024-05-08T08:54:08.045 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.045 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.046 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:08.046 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140807+0000", 2024-05-08T08:54:08.046 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8347999999999999e-05 2024-05-08T08:54:08.046 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:08.046 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:08.046 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:08.046 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.046 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.046 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22723 3.15 3:a88fdee9:::rbd_data.1140d3fc54c.000000000000004d:18 [read 1556480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:08.046 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.098778+0000", 2024-05-08T08:54:08.046 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 35.892019779000002, 2024-05-08T08:54:08.046 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042128577, 2024-05-08T08:54:08.046 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:08.046 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:08.046 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:08.047 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:08.047 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:54:08.047 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:08.047 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22723 2024-05-08T08:54:08.047 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.047 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:08.047 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.047 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:08.047 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098778+0000", 2024-05-08T08:54:08.047 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:08.047 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.047 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.047 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:08.047 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098778+0000", 2024-05-08T08:54:08.047 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:08.048 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.048 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.048 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:08.048 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098834+0000", 2024-05-08T08:54:08.048 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5788000000000002e-05 2024-05-08T08:54:08.048 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.048 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.048 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:08.048 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098837+0000", 2024-05-08T08:54:08.048 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3510000000000002e-06 2024-05-08T08:54:08.048 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.048 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.048 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:08.048 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098845+0000", 2024-05-08T08:54:08.048 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5490000000000006e-06 2024-05-08T08:54:08.048 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.049 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.049 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:08.049 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098861+0000", 2024-05-08T08:54:08.049 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5233e-05 2024-05-08T08:54:08.049 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.049 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.049 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:08.049 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140829+0000", 2024-05-08T08:54:08.049 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.04196863 2024-05-08T08:54:08.049 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.049 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.049 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:08.049 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140869+0000", 2024-05-08T08:54:08.049 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9461e-05 2024-05-08T08:54:08.049 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.050 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.050 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:08.050 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140906+0000", 2024-05-08T08:54:08.050 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7564999999999997e-05 2024-05-08T08:54:08.050 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:08.050 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:08.050 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:08.050 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.050 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.050 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22732 3.15 3:a907109d:::rbd_data.1140d3fc54c.000000000000004f:18 [read 1097728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:08.050 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.098871+0000", 2024-05-08T08:54:08.050 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 35.891926298999998, 2024-05-08T08:54:08.050 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042141517000000003, 2024-05-08T08:54:08.050 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:08.050 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:08.051 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:08.051 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:08.051 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:54:08.051 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:08.051 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22732 2024-05-08T08:54:08.051 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.051 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:08.051 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.051 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:08.051 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098871+0000", 2024-05-08T08:54:08.051 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:08.051 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.051 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.051 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:08.051 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098871+0000", 2024-05-08T08:54:08.051 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:08.052 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.052 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.052 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:08.052 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.099940+0000", 2024-05-08T08:54:08.052 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0010683279999999999 2024-05-08T08:54:08.052 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.052 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.052 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:08.052 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.099944+0000", 2024-05-08T08:54:08.052 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9339999999999997e-06 2024-05-08T08:54:08.052 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.052 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.052 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:08.052 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.099956+0000", 2024-05-08T08:54:08.052 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1970999999999999e-05 2024-05-08T08:54:08.053 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.053 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.053 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:08.053 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.099968+0000", 2024-05-08T08:54:08.053 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.189e-05 2024-05-08T08:54:08.053 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.053 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.053 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:08.053 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140930+0000", 2024-05-08T08:54:08.053 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.040961806000000003 2024-05-08T08:54:08.053 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.053 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.053 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:08.053 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140974+0000", 2024-05-08T08:54:08.053 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3918999999999997e-05 2024-05-08T08:54:08.053 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.054 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.054 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:08.054 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.141013+0000", 2024-05-08T08:54:08.054 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8668999999999999e-05 2024-05-08T08:54:08.054 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:08.054 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:08.054 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:08.054 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.054 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.054 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22765 3.15 3:a9c637ae:::rbd_data.114071bcf828.000000000000013f:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:08.054 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.100902+0000", 2024-05-08T08:54:08.054 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 35.889895504000002, 2024-05-08T08:54:08.054 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042471271999999997, 2024-05-08T08:54:08.054 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:08.055 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:08.055 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:08.055 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:08.055 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:54:08.055 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:08.055 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22765 2024-05-08T08:54:08.055 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.055 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:08.055 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.055 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:08.055 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.100902+0000", 2024-05-08T08:54:08.055 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:08.055 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.055 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.055 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:08.056 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.100902+0000", 2024-05-08T08:54:08.056 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:08.056 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.056 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.056 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:08.056 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.100902+0000", 2024-05-08T08:54:08.056 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.39e-07 2024-05-08T08:54:08.056 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.056 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.056 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:08.056 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.100904+0000", 2024-05-08T08:54:08.056 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6479999999999999e-06 2024-05-08T08:54:08.056 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.056 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.056 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:08.057 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.100910+0000", 2024-05-08T08:54:08.057 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1399999999999997e-06 2024-05-08T08:54:08.057 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.057 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.057 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:08.057 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.100916+0000", 2024-05-08T08:54:08.057 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6999999999999996e-06 2024-05-08T08:54:08.057 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.057 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.057 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:08.057 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.141733+0000", 2024-05-08T08:54:08.057 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.040817271000000002 2024-05-08T08:54:08.057 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.057 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.057 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:08.058 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.141812+0000", 2024-05-08T08:54:08.058 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9270999999999994e-05 2024-05-08T08:54:08.058 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.058 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.058 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:08.058 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.143373+0000", 2024-05-08T08:54:08.058 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001560903 2024-05-08T08:54:08.058 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:08.058 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:08.058 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:08.058 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.058 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.058 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:23031 3.15 3:ac9fb9cd:::rbd_data.114071bcf828.00000000000000f7:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:08.058 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.113562+0000", 2024-05-08T08:54:08.059 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 35.877235744000004, 2024-05-08T08:54:08.059 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042560938999999999, 2024-05-08T08:54:08.059 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:08.059 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:08.059 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:54:08.059 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:08.059 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:54:08.059 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:08.059 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 23031 2024-05-08T08:54:08.059 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.059 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:08.059 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.059 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:08.059 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.113562+0000", 2024-05-08T08:54:08.059 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:08.060 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.060 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.060 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:08.060 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.113562+0000", 2024-05-08T08:54:08.060 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:08.060 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.060 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.060 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:08.060 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.113938+0000", 2024-05-08T08:54:08.060 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000376401 2024-05-08T08:54:08.060 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.060 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.060 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:08.060 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.113943+0000", 2024-05-08T08:54:08.060 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0529999999999997e-06 2024-05-08T08:54:08.060 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.061 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.061 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:08.061 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.113951+0000", 2024-05-08T08:54:08.061 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8569999999999995e-06 2024-05-08T08:54:08.061 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.061 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.061 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:08.061 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.113959+0000", 2024-05-08T08:54:08.061 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4710000000000001e-06 2024-05-08T08:54:08.061 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.061 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.061 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:08.061 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.153955+0000", 2024-05-08T08:54:08.061 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.039996747999999999 2024-05-08T08:54:08.061 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.062 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.062 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:08.062 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.154014+0000", 2024-05-08T08:54:08.062 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8464999999999997e-05 2024-05-08T08:54:08.062 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.062 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.062 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-05-08T08:54:08.062 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.154148+0000", 2024-05-08T08:54:08.062 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00013428600000000001 2024-05-08T08:54:08.062 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.062 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.062 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:54:08.062 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.155390+0000", 2024-05-08T08:54:08.062 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001241792 2024-05-08T08:54:08.062 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.063 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.063 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:54:08.063 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.156068+0000", 2024-05-08T08:54:08.063 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00067823700000000005 2024-05-08T08:54:08.063 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.063 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.063 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:54:08.063 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.156102+0000", 2024-05-08T08:54:08.063 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3812000000000001e-05 2024-05-08T08:54:08.063 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.063 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.063 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:08.063 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.156123+0000", 2024-05-08T08:54:08.063 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0817000000000001e-05 2024-05-08T08:54:08.064 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:08.064 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:08.064 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:08.064 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.064 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.064 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4416.0:20531 3.16 3:6bb2ed16:::rbd_object_map.1140bc4a55c7:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-05-08T08:54:08.064 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:58.673786+0000", 2024-05-08T08:54:08.064 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.3170113949999998, 2024-05-08T08:54:08.064 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.94684119899999997, 2024-05-08T08:54:08.064 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:08.064 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:08.064 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:54:08.064 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:08.064 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4416", 2024-05-08T08:54:08.064 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:54:08.065 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20531 2024-05-08T08:54:08.065 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.065 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:08.065 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.065 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:08.065 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673786+0000", 2024-05-08T08:54:08.065 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:08.065 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.065 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.065 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:08.065 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673786+0000", 2024-05-08T08:54:08.065 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:08.065 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.065 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.065 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:08.066 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673787+0000", 2024-05-08T08:54:08.066 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9800000000000003e-07 2024-05-08T08:54:08.066 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.066 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.066 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:08.066 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673791+0000", 2024-05-08T08:54:08.066 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.104e-06 2024-05-08T08:54:08.066 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.066 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.066 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:08.066 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673801+0000", 2024-05-08T08:54:08.066 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9960000000000005e-06 2024-05-08T08:54:08.066 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.066 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.067 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:08.067 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673806+0000", 2024-05-08T08:54:08.067 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2320000000000001e-06 2024-05-08T08:54:08.067 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.067 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.067 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:08.067 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673860+0000", 2024-05-08T08:54:08.067 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3468999999999999e-05 2024-05-08T08:54:08.067 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.067 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.067 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-05-08T08:54:08.067 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673866+0000", 2024-05-08T08:54:08.067 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3500000000000002e-06 2024-05-08T08:54:08.067 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.068 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.068 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:08.068 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.616700+0000", 2024-05-08T08:54:08.068 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.94283390700000003 2024-05-08T08:54:08.068 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.068 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.068 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:08.068 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.616941+0000", 2024-05-08T08:54:08.068 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00024080400000000001 2024-05-08T08:54:08.068 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.068 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.068 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-05-08T08:54:08.068 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.617072+0000", 2024-05-08T08:54:08.068 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000131688 2024-05-08T08:54:08.068 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.069 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.069 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:54:08.069 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.619571+0000", 2024-05-08T08:54:08.069 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0024990149999999998 2024-05-08T08:54:08.069 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.069 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.069 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:54:08.069 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.620568+0000", 2024-05-08T08:54:08.069 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00099711000000000005 2024-05-08T08:54:08.069 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.069 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.069 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:54:08.069 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.620602+0000", 2024-05-08T08:54:08.069 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.324e-05 2024-05-08T08:54:08.070 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.070 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.070 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:08.070 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.620627+0000", 2024-05-08T08:54:08.070 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5585999999999999e-05 2024-05-08T08:54:08.070 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:08.070 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:08.070 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:08.070 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.070 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.070 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5423.0:429 3.f 3:f1d5704a:::rbd_data.1140cf33de9.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-05-08T08:54:08.070 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:59.659591+0000", 2024-05-08T08:54:08.070 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 8.3312069100000006, 2024-05-08T08:54:08.070 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.044109585, 2024-05-08T08:54:08.070 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:08.071 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:08.071 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:54:08.071 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:08.071 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5423", 2024-05-08T08:54:08.071 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/306969889", 2024-05-08T08:54:08.071 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 429 2024-05-08T08:54:08.071 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.071 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:08.071 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.071 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:08.071 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.659591+0000", 2024-05-08T08:54:08.071 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:08.071 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.071 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.071 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:08.072 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.659591+0000", 2024-05-08T08:54:08.072 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:08.072 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.072 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.072 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:08.072 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.659591+0000", 2024-05-08T08:54:08.072 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7299999999999996e-07 2024-05-08T08:54:08.072 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.072 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.072 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:08.072 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.663640+0000", 2024-05-08T08:54:08.072 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0040485529999999999 2024-05-08T08:54:08.072 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.072 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.072 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:08.072 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.663655+0000", 2024-05-08T08:54:08.072 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5065000000000001e-05 2024-05-08T08:54:08.072 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.073 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.073 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:08.073 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.663661+0000", 2024-05-08T08:54:08.073 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4269999999999996e-06 2024-05-08T08:54:08.073 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.073 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.073 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:08.073 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.663714+0000", 2024-05-08T08:54:08.073 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2992999999999999e-05 2024-05-08T08:54:08.073 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.073 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.073 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:08.073 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.663798+0000", 2024-05-08T08:54:08.073 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3992999999999994e-05 2024-05-08T08:54:08.073 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.073 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.073 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-05-08T08:54:08.074 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.664125+0000", 2024-05-08T08:54:08.074 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00032663899999999999 2024-05-08T08:54:08.074 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.074 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.074 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:54:08.074 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.687754+0000", 2024-05-08T08:54:08.074 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.023629075999999999 2024-05-08T08:54:08.074 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.074 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.074 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:54:08.074 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.703569+0000", 2024-05-08T08:54:08.074 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.015814814999999999 2024-05-08T08:54:08.074 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.074 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.074 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:54:08.075 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.703641+0000", 2024-05-08T08:54:08.075 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2484000000000006e-05 2024-05-08T08:54:08.075 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.075 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.075 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:08.075 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.703700+0000", 2024-05-08T08:54:08.075 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8967e-05 2024-05-08T08:54:08.075 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:08.075 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:08.075 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:08.075 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:08.075 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:08.075 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:08.077 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:54:08.077 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:08.079+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:08.178 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:08.180+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:08.237 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:08.237 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:54:08.237 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:54:08.237 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:08.246 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:54:08.278 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:08.281+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:08.354 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:08.354 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:54:08.354 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:54:08.354 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:54:08.354 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:08.363 INFO:teuthology.orchestra.run.smithi196.stdout:checking clone #37, image image_client.0-clone37 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent38 2024-05-08T08:54:08.363 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:54:08.407 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:08.410+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:08.432 INFO:teuthology.orchestra.run.smithi196.stdout:4795 read 0x2950200 thru 0x2955ade (0x58df bytes) 2024-05-08T08:54:08.433 INFO:teuthology.orchestra.run.smithi196.stdout:4797 write 0x311d600 thru 0x3120813 (0x3214 bytes) 2024-05-08T08:54:08.442 INFO:teuthology.orchestra.run.smithi196.stdout:4802 trunc from 0x4918400 to 0xbb92600 2024-05-08T08:54:08.475 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:08.475 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:54:08.475 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:54:08.475 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:54:08.475 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.475 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22073 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 782336~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:08.475 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:54:08.476 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 100.619511735, 2024-05-08T08:54:08.476 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047244234000000003, 2024-05-08T08:54:08.476 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:08.476 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:08.476 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:08.476 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:08.476 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:08.476 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:08.476 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22073 2024-05-08T08:54:08.476 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.476 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:08.476 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.476 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:08.476 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:54:08.476 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:08.477 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.477 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.477 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:08.477 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:54:08.477 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:08.477 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.477 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.477 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:08.477 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857668+0000", 2024-05-08T08:54:08.477 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7800000000000002e-07 2024-05-08T08:54:08.477 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.477 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.477 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:08.477 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857670+0000", 2024-05-08T08:54:08.477 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0820000000000001e-06 2024-05-08T08:54:08.477 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.478 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.478 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:08.478 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857678+0000", 2024-05-08T08:54:08.478 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7589999999999997e-06 2024-05-08T08:54:08.478 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.478 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.478 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:08.478 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857691+0000", 2024-05-08T08:54:08.478 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3672999999999999e-05 2024-05-08T08:54:08.478 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.478 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.478 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:08.478 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904780+0000", 2024-05-08T08:54:08.478 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047088616999999999 2024-05-08T08:54:08.478 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.478 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.479 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:08.479 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904840+0000", 2024-05-08T08:54:08.479 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0356999999999998e-05 2024-05-08T08:54:08.479 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.479 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.479 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:08.479 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904912+0000", 2024-05-08T08:54:08.479 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1367999999999998e-05 2024-05-08T08:54:08.479 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:08.479 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:08.479 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:08.479 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.479 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.479 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22075 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 798720~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:08.479 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:54:08.480 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 100.619481338, 2024-05-08T08:54:08.480 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047384643999999997, 2024-05-08T08:54:08.480 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:08.480 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:08.480 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:08.480 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:08.480 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:08.480 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:08.480 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22075 2024-05-08T08:54:08.480 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.480 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:08.480 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.480 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:08.480 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:54:08.480 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:08.480 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.480 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.481 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:08.481 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:54:08.481 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:08.481 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.481 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.481 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:08.481 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:54:08.481 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7399999999999999e-07 2024-05-08T08:54:08.481 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.481 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.481 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:08.481 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857705+0000", 2024-05-08T08:54:08.481 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1980000000000004e-06 2024-05-08T08:54:08.481 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.481 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.481 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:08.482 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857713+0000", 2024-05-08T08:54:08.482 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7039999999999994e-06 2024-05-08T08:54:08.482 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.482 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.482 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:08.482 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857718+0000", 2024-05-08T08:54:08.482 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.865e-06 2024-05-08T08:54:08.482 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.482 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.482 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:08.482 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904941+0000", 2024-05-08T08:54:08.482 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047223198000000001 2024-05-08T08:54:08.482 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.482 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.482 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:08.482 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904999+0000", 2024-05-08T08:54:08.483 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8254999999999997e-05 2024-05-08T08:54:08.483 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.483 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.483 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:08.483 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905082+0000", 2024-05-08T08:54:08.483 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3049999999999999e-05 2024-05-08T08:54:08.483 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:08.483 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:08.483 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:08.483 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.483 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.483 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22077 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 815104~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:08.483 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:54:08.483 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 100.619443327, 2024-05-08T08:54:08.484 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047519448999999998, 2024-05-08T08:54:08.484 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:08.484 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:08.484 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:08.484 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:08.484 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:08.484 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:08.484 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22077 2024-05-08T08:54:08.484 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.484 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:08.484 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.484 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:08.484 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:54:08.484 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:08.484 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.484 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.485 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:08.485 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:54:08.485 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:08.485 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.485 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.485 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:08.485 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:54:08.485 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.99e-07 2024-05-08T08:54:08.485 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.485 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.485 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:08.485 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857739+0000", 2024-05-08T08:54:08.485 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3599999999999999e-06 2024-05-08T08:54:08.485 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.485 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.485 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:08.485 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857745+0000", 2024-05-08T08:54:08.486 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5139999999999998e-06 2024-05-08T08:54:08.486 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.486 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.486 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:08.486 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857750+0000", 2024-05-08T08:54:08.486 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8550000000000001e-06 2024-05-08T08:54:08.486 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.486 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.486 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:08.486 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905110+0000", 2024-05-08T08:54:08.486 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047360126000000002 2024-05-08T08:54:08.486 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.486 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.486 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:08.486 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905175+0000", 2024-05-08T08:54:08.487 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4689000000000004e-05 2024-05-08T08:54:08.487 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.487 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.487 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:08.487 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905255+0000", 2024-05-08T08:54:08.487 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0306e-05 2024-05-08T08:54:08.487 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:08.487 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:08.487 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:08.487 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.487 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.487 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22079 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 831488~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:08.487 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:54:08.487 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 100.619422779, 2024-05-08T08:54:08.487 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.04767499, 2024-05-08T08:54:08.487 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:08.488 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:08.488 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:08.488 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:08.488 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:08.488 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:08.488 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22079 2024-05-08T08:54:08.488 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.488 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:08.488 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.488 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:08.488 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:54:08.488 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:08.488 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.488 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.488 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:08.488 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:54:08.489 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:08.489 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.489 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.489 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:08.489 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857757+0000", 2024-05-08T08:54:08.489 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8299999999999997e-07 2024-05-08T08:54:08.489 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.489 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.489 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:08.489 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857759+0000", 2024-05-08T08:54:08.489 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1390000000000002e-06 2024-05-08T08:54:08.489 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.489 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.489 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:08.489 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857765+0000", 2024-05-08T08:54:08.489 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4609999999999996e-06 2024-05-08T08:54:08.489 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.490 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.490 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:08.490 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857770+0000", 2024-05-08T08:54:08.490 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6689999999999997e-06 2024-05-08T08:54:08.490 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.490 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.490 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:08.490 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905281+0000", 2024-05-08T08:54:08.490 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047510668999999998 2024-05-08T08:54:08.490 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.490 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.490 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:08.490 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905344+0000", 2024-05-08T08:54:08.490 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2797000000000002e-05 2024-05-08T08:54:08.490 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:08.491 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905431+0000", 2024-05-08T08:54:08.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7771999999999999e-05 2024-05-08T08:54:08.491 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:08.491 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:08.491 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:08.491 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.491 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22082 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 847872~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:08.491 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:54:08.491 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 100.619403088, 2024-05-08T08:54:08.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047831934999999999, 2024-05-08T08:54:08.491 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:08.491 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:08.491 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:08.492 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:08.492 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:08.492 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:08.492 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22082 2024-05-08T08:54:08.492 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.492 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:08.492 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.492 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:08.492 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:54:08.492 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:08.492 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.492 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.492 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:08.492 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:54:08.493 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:08.493 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.493 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.493 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:08.493 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:54:08.493 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:54:08.493 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.493 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.493 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:08.493 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857778+0000", 2024-05-08T08:54:08.493 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0789999999999999e-06 2024-05-08T08:54:08.493 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.493 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.493 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:08.493 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857784+0000", 2024-05-08T08:54:08.493 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0480000000000004e-06 2024-05-08T08:54:08.494 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.494 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.494 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:08.494 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857790+0000", 2024-05-08T08:54:08.494 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2719999999999997e-06 2024-05-08T08:54:08.494 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.494 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.494 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:08.494 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905458+0000", 2024-05-08T08:54:08.494 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047668470999999997 2024-05-08T08:54:08.494 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.494 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.494 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:08.494 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905521+0000", 2024-05-08T08:54:08.494 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2952000000000003e-05 2024-05-08T08:54:08.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.495 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:08.495 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905608+0000", 2024-05-08T08:54:08.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6759000000000001e-05 2024-05-08T08:54:08.495 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:08.495 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:08.495 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:08.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.495 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22084 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 864256~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:08.495 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:54:08.495 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 100.619379138, 2024-05-08T08:54:08.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047977748000000001, 2024-05-08T08:54:08.495 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:08.496 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:08.496 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:08.496 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:08.496 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:08.496 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:08.496 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22084 2024-05-08T08:54:08.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.496 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:08.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.496 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:08.496 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:54:08.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:08.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.496 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:08.496 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:54:08.497 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:08.497 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.497 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.497 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:08.497 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:54:08.497 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:54:08.497 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.497 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.497 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:08.497 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857802+0000", 2024-05-08T08:54:08.497 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0779999999999998e-06 2024-05-08T08:54:08.497 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.497 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.497 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:08.498 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857812+0000", 2024-05-08T08:54:08.498 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7939999999999997e-06 2024-05-08T08:54:08.498 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.498 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.498 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:08.498 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857817+0000", 2024-05-08T08:54:08.498 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8899999999999998e-06 2024-05-08T08:54:08.498 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.498 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.498 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:08.498 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905636+0000", 2024-05-08T08:54:08.498 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047819252 2024-05-08T08:54:08.498 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.498 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.498 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:08.499 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905700+0000", 2024-05-08T08:54:08.499 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4016999999999994e-05 2024-05-08T08:54:08.499 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.499 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.499 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:08.499 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905778+0000", 2024-05-08T08:54:08.499 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7306999999999997e-05 2024-05-08T08:54:08.499 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:08.499 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:08.499 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:08.499 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.499 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.499 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22086 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 880640~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:08.499 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:54:08.499 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 100.61935578800001, 2024-05-08T08:54:08.500 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048119149, 2024-05-08T08:54:08.500 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:08.500 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:08.500 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:08.500 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:08.500 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:08.500 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:08.500 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22086 2024-05-08T08:54:08.500 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.500 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:08.500 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.500 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:08.500 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:54:08.500 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:08.501 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.501 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.501 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:08.501 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:54:08.501 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:08.501 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.501 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.501 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:08.501 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857824+0000", 2024-05-08T08:54:08.501 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4200000000000002e-07 2024-05-08T08:54:08.501 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.501 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.501 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:08.501 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857826+0000", 2024-05-08T08:54:08.501 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0590000000000001e-06 2024-05-08T08:54:08.502 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.502 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.502 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:08.502 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857832+0000", 2024-05-08T08:54:08.502 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8029999999999999e-06 2024-05-08T08:54:08.502 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.502 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.502 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:08.502 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857837+0000", 2024-05-08T08:54:08.502 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7929999999999997e-06 2024-05-08T08:54:08.502 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.502 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.502 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:08.502 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905803+0000", 2024-05-08T08:54:08.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047965761000000003 2024-05-08T08:54:08.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.503 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:08.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905862+0000", 2024-05-08T08:54:08.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9203000000000002e-05 2024-05-08T08:54:08.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.503 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:08.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905942+0000", 2024-05-08T08:54:08.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0187999999999999e-05 2024-05-08T08:54:08.503 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:08.503 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:08.503 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:08.504 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.504 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.504 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22088 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 897024~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:08.504 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:54:08.504 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 100.619335736, 2024-05-08T08:54:08.504 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048256049000000002, 2024-05-08T08:54:08.504 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:08.504 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:08.504 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:08.504 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:08.504 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:08.504 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:08.504 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22088 2024-05-08T08:54:08.504 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.504 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:08.504 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.505 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:08.505 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:54:08.505 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:08.505 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.505 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.505 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:08.505 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:54:08.505 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:08.505 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.505 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.505 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:08.505 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857844+0000", 2024-05-08T08:54:08.505 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.46e-07 2024-05-08T08:54:08.505 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.505 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.506 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:08.506 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857846+0000", 2024-05-08T08:54:08.506 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0899999999999999e-06 2024-05-08T08:54:08.506 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.506 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.506 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:08.506 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857855+0000", 2024-05-08T08:54:08.506 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8219999999999996e-06 2024-05-08T08:54:08.506 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.506 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.506 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:08.506 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857860+0000", 2024-05-08T08:54:08.506 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2989999999999996e-06 2024-05-08T08:54:08.506 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.506 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.507 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:08.507 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905966+0000", 2024-05-08T08:54:08.507 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048106542000000002 2024-05-08T08:54:08.507 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.507 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.507 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:08.507 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906024+0000", 2024-05-08T08:54:08.507 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7958000000000003e-05 2024-05-08T08:54:08.507 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.507 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.507 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:08.507 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906099+0000", 2024-05-08T08:54:08.507 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4992000000000003e-05 2024-05-08T08:54:08.507 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:08.507 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:08.508 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:08.508 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.508 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.508 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22091 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 913408~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:08.508 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:54:08.508 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 100.61931197600001, 2024-05-08T08:54:08.508 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048395290000000001, 2024-05-08T08:54:08.508 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:08.508 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:08.508 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:08.508 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:08.508 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:08.508 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:08.508 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22091 2024-05-08T08:54:08.509 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.509 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:08.509 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.509 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:08.509 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:54:08.509 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:08.509 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.509 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.509 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:08.509 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:54:08.509 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:08.509 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.509 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.509 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:08.509 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:54:08.509 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4900000000000001e-07 2024-05-08T08:54:08.510 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.510 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.510 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:08.510 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857870+0000", 2024-05-08T08:54:08.510 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2510000000000001e-06 2024-05-08T08:54:08.510 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.510 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.510 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:08.510 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857877+0000", 2024-05-08T08:54:08.510 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0809999999999996e-06 2024-05-08T08:54:08.510 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.510 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.510 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:08.510 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857881+0000", 2024-05-08T08:54:08.510 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.724e-06 2024-05-08T08:54:08.511 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.511 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.511 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:08.511 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906127+0000", 2024-05-08T08:54:08.511 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048245345000000002 2024-05-08T08:54:08.511 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.511 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.511 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:08.511 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906190+0000", 2024-05-08T08:54:08.511 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2762000000000007e-05 2024-05-08T08:54:08.511 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.511 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.511 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:08.511 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906262+0000", 2024-05-08T08:54:08.511 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2778e-05 2024-05-08T08:54:08.512 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:08.512 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:08.512 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:08.512 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.512 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.512 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22093 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 929792~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:08.512 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:54:08.512 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 100.619289612, 2024-05-08T08:54:08.512 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048532875000000003, 2024-05-08T08:54:08.512 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:08.512 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:08.512 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:08.512 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:08.512 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:08.512 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:08.513 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22093 2024-05-08T08:54:08.513 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.513 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:08.513 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.513 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:08.513 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:54:08.513 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:08.513 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.513 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.513 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:08.513 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:54:08.513 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:08.513 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.513 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.514 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:08.514 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857890+0000", 2024-05-08T08:54:08.514 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-07 2024-05-08T08:54:08.514 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.514 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.514 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:08.514 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857899+0000", 2024-05-08T08:54:08.514 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6889999999999999e-06 2024-05-08T08:54:08.514 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.514 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.514 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:08.514 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857906+0000", 2024-05-08T08:54:08.514 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6490000000000002e-06 2024-05-08T08:54:08.514 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.515 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.515 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:08.515 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857917+0000", 2024-05-08T08:54:08.515 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0764999999999999e-05 2024-05-08T08:54:08.515 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.515 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.515 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:08.515 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906287+0000", 2024-05-08T08:54:08.515 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048370442 2024-05-08T08:54:08.515 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.515 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.515 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:08.515 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906349+0000", 2024-05-08T08:54:08.515 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1537000000000004e-05 2024-05-08T08:54:08.515 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.516 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.516 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:08.516 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906422+0000", 2024-05-08T08:54:08.516 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3431999999999998e-05 2024-05-08T08:54:08.516 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:08.516 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:08.516 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:08.516 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.516 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.516 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22095 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 946176~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:08.516 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:54:08.516 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 100.619254908, 2024-05-08T08:54:08.516 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048657593999999998, 2024-05-08T08:54:08.516 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:08.516 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:08.517 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:08.517 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:08.517 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:08.517 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:08.517 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22095 2024-05-08T08:54:08.517 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.517 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:08.517 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.517 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:08.517 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:54:08.517 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:08.517 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.517 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.517 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:08.517 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:54:08.518 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:08.518 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.518 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.518 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:08.518 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857925+0000", 2024-05-08T08:54:08.518 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3099999999999998e-07 2024-05-08T08:54:08.518 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.518 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.518 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:08.518 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857927+0000", 2024-05-08T08:54:08.518 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.34e-06 2024-05-08T08:54:08.518 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.518 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.518 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:08.518 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857938+0000", 2024-05-08T08:54:08.519 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1426e-05 2024-05-08T08:54:08.519 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.519 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.519 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:08.519 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857943+0000", 2024-05-08T08:54:08.519 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5140000000000003e-06 2024-05-08T08:54:08.519 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.519 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.519 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:08.519 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906448+0000", 2024-05-08T08:54:08.519 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048504641000000001 2024-05-08T08:54:08.519 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.519 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.519 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:08.519 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906510+0000", 2024-05-08T08:54:08.520 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2663000000000006e-05 2024-05-08T08:54:08.520 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.520 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.520 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:08.520 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906582+0000", 2024-05-08T08:54:08.520 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1478999999999998e-05 2024-05-08T08:54:08.520 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:08.520 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:08.520 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:08.520 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.520 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.520 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22097 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 962560~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:08.520 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:54:08.520 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 100.619227219, 2024-05-08T08:54:08.520 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048791353000000003, 2024-05-08T08:54:08.520 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:08.521 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:08.521 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:08.521 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:08.521 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:08.521 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:08.521 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22097 2024-05-08T08:54:08.521 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.521 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:08.521 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.521 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:08.521 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:54:08.521 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:08.521 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.521 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.521 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:08.522 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:54:08.522 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:08.522 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.522 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.522 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:08.522 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:54:08.522 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9200000000000002e-07 2024-05-08T08:54:08.522 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.522 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.522 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:08.522 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857954+0000", 2024-05-08T08:54:08.522 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1900000000000002e-06 2024-05-08T08:54:08.522 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.522 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.523 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:08.523 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857961+0000", 2024-05-08T08:54:08.523 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4989999999999999e-06 2024-05-08T08:54:08.523 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.523 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.523 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:08.523 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857971+0000", 2024-05-08T08:54:08.523 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0485999999999999e-05 2024-05-08T08:54:08.523 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.523 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.523 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:08.523 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906607+0000", 2024-05-08T08:54:08.523 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048635109000000003 2024-05-08T08:54:08.523 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.523 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.524 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:08.524 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906667+0000", 2024-05-08T08:54:08.524 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0439000000000002e-05 2024-05-08T08:54:08.524 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.524 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.524 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:08.524 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906743+0000", 2024-05-08T08:54:08.524 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6237999999999998e-05 2024-05-08T08:54:08.524 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:08.524 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:08.524 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:08.524 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.524 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.524 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22099 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 978944~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:08.524 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:54:08.525 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 100.61920105900001, 2024-05-08T08:54:08.525 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048932055000000002, 2024-05-08T08:54:08.525 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:08.525 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:08.525 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:08.525 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:08.525 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:08.525 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:08.525 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22099 2024-05-08T08:54:08.525 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.525 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:08.525 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.525 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:08.525 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:54:08.525 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:08.526 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.526 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.526 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:08.526 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:54:08.526 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:08.526 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.526 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.526 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:08.526 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:54:08.526 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.27e-07 2024-05-08T08:54:08.526 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.526 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.526 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:08.526 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857987+0000", 2024-05-08T08:54:08.527 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5890000000000003e-06 2024-05-08T08:54:08.527 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.527 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.527 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:08.527 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857994+0000", 2024-05-08T08:54:08.527 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6619999999999999e-06 2024-05-08T08:54:08.527 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.527 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.527 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:08.527 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857998+0000", 2024-05-08T08:54:08.527 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7110000000000003e-06 2024-05-08T08:54:08.527 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.527 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.527 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:08.528 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906772+0000", 2024-05-08T08:54:08.528 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048773415000000001 2024-05-08T08:54:08.528 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.528 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.528 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:08.528 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906834+0000", 2024-05-08T08:54:08.528 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1752000000000001e-05 2024-05-08T08:54:08.528 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.528 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.528 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:08.528 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906910+0000", 2024-05-08T08:54:08.528 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6499000000000001e-05 2024-05-08T08:54:08.528 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:08.528 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:08.528 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:08.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.529 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.529 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22101 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 995328~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:08.529 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:54:08.529 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 100.619173928, 2024-05-08T08:54:08.529 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049087364000000001, 2024-05-08T08:54:08.529 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:08.529 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:08.529 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:08.529 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:08.529 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:08.529 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:08.529 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22101 2024-05-08T08:54:08.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.529 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:08.530 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.530 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:08.530 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:54:08.530 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:08.530 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.530 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.530 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:08.530 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:54:08.530 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:08.530 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.530 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.530 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:08.530 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858006+0000", 2024-05-08T08:54:08.530 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1399999999999997e-07 2024-05-08T08:54:08.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.531 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.531 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:08.531 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858008+0000", 2024-05-08T08:54:08.531 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2060000000000001e-06 2024-05-08T08:54:08.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.531 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.531 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:08.531 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858017+0000", 2024-05-08T08:54:08.531 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1579999999999994e-06 2024-05-08T08:54:08.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.531 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.531 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:08.531 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858022+0000", 2024-05-08T08:54:08.531 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4199999999999998e-06 2024-05-08T08:54:08.532 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.532 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.532 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:08.532 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906940+0000", 2024-05-08T08:54:08.532 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048917627999999998 2024-05-08T08:54:08.532 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.532 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.532 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:08.532 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907002+0000", 2024-05-08T08:54:08.532 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2399999999999999e-05 2024-05-08T08:54:08.532 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.532 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.532 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:08.532 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907092+0000", 2024-05-08T08:54:08.532 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0137999999999997e-05 2024-05-08T08:54:08.533 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:08.533 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:08.533 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:08.533 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.533 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.533 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22103 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1011712~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:08.533 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:54:08.533 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 100.619149143, 2024-05-08T08:54:08.533 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049228542, 2024-05-08T08:54:08.533 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:08.533 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:08.533 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:08.533 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:08.533 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:08.533 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:08.533 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22103 2024-05-08T08:54:08.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.534 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:08.534 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.534 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:08.534 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:54:08.534 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:08.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.534 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.534 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:08.534 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:54:08.534 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:08.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.534 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.534 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:08.535 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:54:08.535 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4299999999999999e-07 2024-05-08T08:54:08.535 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.535 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.535 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:08.535 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858032+0000", 2024-05-08T08:54:08.535 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0590000000000001e-06 2024-05-08T08:54:08.535 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.535 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.535 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:08.535 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858037+0000", 2024-05-08T08:54:08.535 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.977e-06 2024-05-08T08:54:08.535 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.535 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.535 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:08.536 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858042+0000", 2024-05-08T08:54:08.536 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7650000000000001e-06 2024-05-08T08:54:08.536 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.536 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.536 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:08.536 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907118+0000", 2024-05-08T08:54:08.536 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049076147000000001 2024-05-08T08:54:08.536 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.536 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.536 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:08.536 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907180+0000", 2024-05-08T08:54:08.536 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1971999999999996e-05 2024-05-08T08:54:08.536 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.536 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.536 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:08.537 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907258+0000", 2024-05-08T08:54:08.537 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8279e-05 2024-05-08T08:54:08.537 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:08.537 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:08.537 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:08.537 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.537 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.537 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22105 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1028096~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:08.537 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:54:08.537 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 100.6191299, 2024-05-08T08:54:08.537 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049394609999999999, 2024-05-08T08:54:08.537 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:08.537 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:08.537 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:08.537 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:08.538 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:08.538 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:08.538 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22105 2024-05-08T08:54:08.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.538 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:08.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.538 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:08.538 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:54:08.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:08.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.538 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:08.538 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:54:08.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:08.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.539 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.539 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:08.539 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858050+0000", 2024-05-08T08:54:08.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5999999999999999e-07 2024-05-08T08:54:08.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.539 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.539 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:08.539 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858051+0000", 2024-05-08T08:54:08.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8870000000000001e-06 2024-05-08T08:54:08.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.539 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.539 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:08.539 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858057+0000", 2024-05-08T08:54:08.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8019999999999997e-06 2024-05-08T08:54:08.540 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.540 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.540 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:08.540 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858062+0000", 2024-05-08T08:54:08.540 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.561e-06 2024-05-08T08:54:08.540 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.540 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.540 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:08.540 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907289+0000", 2024-05-08T08:54:08.540 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049226957000000002 2024-05-08T08:54:08.540 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.540 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.540 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:08.540 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907356+0000", 2024-05-08T08:54:08.540 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6915000000000006e-05 2024-05-08T08:54:08.541 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.541 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.541 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:08.541 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907444+0000", 2024-05-08T08:54:08.541 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8127999999999994e-05 2024-05-08T08:54:08.541 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:08.541 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:08.541 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:08.541 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.541 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.541 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22107 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1044480~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:08.541 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:54:08.541 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 100.61911155999999, 2024-05-08T08:54:08.541 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049799942999999999, 2024-05-08T08:54:08.542 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:08.542 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:08.542 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:08.542 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:08.542 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:08.542 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:08.542 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22107 2024-05-08T08:54:08.542 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.542 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:08.542 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.542 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:08.542 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:54:08.542 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:08.542 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.542 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.543 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:08.543 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:54:08.543 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:08.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.543 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:08.543 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:54:08.543 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4799999999999999e-07 2024-05-08T08:54:08.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.543 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:08.543 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858070+0000", 2024-05-08T08:54:08.543 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.012e-06 2024-05-08T08:54:08.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.544 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:08.544 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858077+0000", 2024-05-08T08:54:08.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6290000000000001e-06 2024-05-08T08:54:08.544 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.544 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.544 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:08.544 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858083+0000", 2024-05-08T08:54:08.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8880000000000002e-06 2024-05-08T08:54:08.544 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.544 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.544 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:08.544 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907671+0000", 2024-05-08T08:54:08.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049587191000000003 2024-05-08T08:54:08.544 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.544 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.545 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:08.545 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907763+0000", 2024-05-08T08:54:08.545 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2231000000000005e-05 2024-05-08T08:54:08.545 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.545 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.545 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:08.545 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907867+0000", 2024-05-08T08:54:08.545 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104644 2024-05-08T08:54:08.545 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:08.545 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:08.545 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:08.545 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.545 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.545 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22294 3.1c 3:3aad0389:::rbd_data.11406886b07b.0000000000000080:head [stat,delete] snapc 7=[7] RETRY=6 ondisk+retry+write+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:08.545 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:54:08.546 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 100.618517712, 2024-05-08T08:54:08.546 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051152191, 2024-05-08T08:54:08.546 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:08.546 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:08.546 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:08.546 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:08.546 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:08.546 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:08.546 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22294 2024-05-08T08:54:08.546 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.546 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:08.546 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.546 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:08.546 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:54:08.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:08.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.547 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:08.547 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:54:08.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:08.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.547 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:08.547 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858662+0000", 2024-05-08T08:54:08.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4799999999999998e-07 2024-05-08T08:54:08.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.547 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:08.547 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858664+0000", 2024-05-08T08:54:08.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1229999999999998e-06 2024-05-08T08:54:08.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:08.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858670+0000", 2024-05-08T08:54:08.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3099999999999997e-06 2024-05-08T08:54:08.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:08.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858676+0000", 2024-05-08T08:54:08.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9750000000000004e-06 2024-05-08T08:54:08.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:08.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907901+0000", 2024-05-08T08:54:08.549 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049224520000000001 2024-05-08T08:54:08.549 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.549 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.549 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:08.549 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.908085+0000", 2024-05-08T08:54:08.549 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00018377500000000001 2024-05-08T08:54:08.549 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.549 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.549 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:08.549 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.909814+0000", 2024-05-08T08:54:08.549 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00172894 2024-05-08T08:54:08.549 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:08.549 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:08.549 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:08.549 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.550 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.550 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4416.0:13461 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-05-08T08:54:08.550 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:22.487456+0000", 2024-05-08T08:54:08.550 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 45.989722671000003, 2024-05-08T08:54:08.550 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.99102620799999996, 2024-05-08T08:54:08.550 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:08.550 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:08.550 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:08.550 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:08.550 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4416", 2024-05-08T08:54:08.550 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:54:08.550 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 13461 2024-05-08T08:54:08.550 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.550 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:08.550 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.551 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:08.551 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487456+0000", 2024-05-08T08:54:08.551 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:08.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.551 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:08.551 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487456+0000", 2024-05-08T08:54:08.551 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:08.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.551 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:08.551 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487458+0000", 2024-05-08T08:54:08.551 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5659999999999999e-06 2024-05-08T08:54:08.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.552 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:08.552 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487463+0000", 2024-05-08T08:54:08.552 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6890000000000003e-06 2024-05-08T08:54:08.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.552 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.552 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:08.552 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487472+0000", 2024-05-08T08:54:08.552 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7710000000000001e-06 2024-05-08T08:54:08.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.552 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.552 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:08.552 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487478+0000", 2024-05-08T08:54:08.552 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1390000000000003e-06 2024-05-08T08:54:08.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.552 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.553 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:08.553 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487520+0000", 2024-05-08T08:54:08.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2259999999999999e-05 2024-05-08T08:54:08.553 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.553 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.553 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-05-08T08:54:08.553 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487528+0000", 2024-05-08T08:54:08.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7479999999999993e-06 2024-05-08T08:54:08.553 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.553 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.553 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:08.553 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:23.478139+0000", 2024-05-08T08:54:08.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.99061165200000001 2024-05-08T08:54:08.553 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.553 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.553 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:08.553 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:23.478301+0000", 2024-05-08T08:54:08.554 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000161602 2024-05-08T08:54:08.554 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.554 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.554 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:08.554 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:23.478483+0000", 2024-05-08T08:54:08.554 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000181781 2024-05-08T08:54:08.554 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:08.554 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:08.554 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:08.554 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.554 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.554 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5752.0:404 3.1a 3:5f5e8104:::rbd_data.1140bc4a55c7.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 5175296~45056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-05-08T08:54:08.554 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:00.390863+0000", 2024-05-08T08:54:08.554 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 8.0863164760000004, 2024-05-08T08:54:08.554 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.055189174000000001, 2024-05-08T08:54:08.555 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:08.555 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:08.555 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:54:08.555 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:08.555 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5752", 2024-05-08T08:54:08.555 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/3534419815", 2024-05-08T08:54:08.555 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 404 2024-05-08T08:54:08.555 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.555 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:08.555 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.555 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:08.555 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.390863+0000", 2024-05-08T08:54:08.555 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:08.555 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.556 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.556 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:08.556 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.390863+0000", 2024-05-08T08:54:08.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:08.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.556 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.556 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:08.556 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.390864+0000", 2024-05-08T08:54:08.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.471e-06 2024-05-08T08:54:08.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.556 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.556 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:08.556 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397475+0000", 2024-05-08T08:54:08.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0066109109999999997 2024-05-08T08:54:08.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.556 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.557 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:08.557 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397498+0000", 2024-05-08T08:54:08.557 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2674e-05 2024-05-08T08:54:08.557 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.557 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.557 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:08.557 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397508+0000", 2024-05-08T08:54:08.557 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9019999999999994e-06 2024-05-08T08:54:08.557 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.557 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.557 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:08.557 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397609+0000", 2024-05-08T08:54:08.557 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010121899999999999 2024-05-08T08:54:08.557 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.557 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.557 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:08.558 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397711+0000", 2024-05-08T08:54:08.558 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102489 2024-05-08T08:54:08.558 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.558 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.558 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-05-08T08:54:08.558 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397999+0000", 2024-05-08T08:54:08.558 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00028733799999999999 2024-05-08T08:54:08.558 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.558 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.558 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:54:08.558 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.432093+0000", 2024-05-08T08:54:08.558 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.034094865000000002 2024-05-08T08:54:08.558 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.558 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.559 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:54:08.559 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.445975+0000", 2024-05-08T08:54:08.559 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013881187 2024-05-08T08:54:08.559 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.559 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.559 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:54:08.559 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.446013+0000", 2024-05-08T08:54:08.559 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8155000000000003e-05 2024-05-08T08:54:08.559 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:08.559 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:08.559 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:08.559 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.446052+0000", 2024-05-08T08:54:08.559 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8962999999999999e-05 2024-05-08T08:54:08.559 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:08.559 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:08.560 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:08.560 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:08.560 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:08.560 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:08.562 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:54:08.562 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:08.564+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:08.563 INFO:teuthology.orchestra.run.smithi196.stdout:4804 write 0xea05c00 thru 0xea0e91e (0x8d1f bytes) 2024-05-08T08:54:08.563 INFO:teuthology.orchestra.run.smithi196.stdout:4806 punch from 0xcf40800 to 0xcf4da00, (0xd200 bytes) 2024-05-08T08:54:08.563 INFO:teuthology.orchestra.run.smithi196.stdout:4807 write 0xd807e00 thru 0xd80ea07 (0x6c08 bytes) 2024-05-08T08:54:08.563 INFO:teuthology.orchestra.run.smithi196.stdout:4809 write 0x62ffc00 thru 0x630b631 (0xba32 bytes) 2024-05-08T08:54:08.563 INFO:teuthology.orchestra.run.smithi196.stdout:4810 trunc from 0xea0ea00 to 0x543ea00 2024-05-08T08:54:08.563 INFO:teuthology.orchestra.run.smithi196.stdout:4812 read 0x3122000 thru 0x312d8a5 (0xb8a6 bytes) 2024-05-08T08:54:08.563 INFO:teuthology.orchestra.run.smithi196.stdout:4815 punch from 0x2ad9e00 to 0x2ae1600, (0x7800 bytes) 2024-05-08T08:54:08.563 INFO:teuthology.orchestra.run.smithi196.stdout:4818 read 0x3a16e00 thru 0x3a1e404 (0x7605 bytes) 2024-05-08T08:54:08.564 INFO:teuthology.orchestra.run.smithi196.stdout:4821 read 0x35bd600 thru 0x35c803b (0xaa3c bytes) 2024-05-08T08:54:08.564 INFO:teuthology.orchestra.run.smithi196.stdout:4822 write 0x3c8fc00 thru 0x3c9a38a (0xa78b bytes) 2024-05-08T08:54:08.564 INFO:teuthology.orchestra.run.smithi196.stdout:4823 trunc from 0x543ea00 to 0x2523c00 2024-05-08T08:54:08.630 INFO:teuthology.orchestra.run.smithi196.stdout:4824 write 0x7d75a00 thru 0x7d7ffb5 (0xa5b6 bytes) 2024-05-08T08:54:08.631 INFO:teuthology.orchestra.run.smithi196.stdout:4829 write 0x52fb400 thru 0x52fdbc2 (0x27c3 bytes) 2024-05-08T08:54:08.631 INFO:teuthology.orchestra.run.smithi196.stdout:4830 read 0x1356600 thru 0x135dad4 (0x74d5 bytes) 2024-05-08T08:54:08.631 INFO:teuthology.orchestra.run.smithi196.stdout:4831 read 0x4ddd000 thru 0x4de46fe (0x76ff bytes) 2024-05-08T08:54:08.631 INFO:teuthology.orchestra.run.smithi196.stdout:4832 read 0x7a76800 thru 0x7a79faf (0x37b0 bytes) 2024-05-08T08:54:08.631 INFO:teuthology.orchestra.run.smithi196.stdout:4836 punch from 0x29d9a00 to 0x29da600, (0xc00 bytes) 2024-05-08T08:54:08.632 INFO:teuthology.orchestra.run.smithi196.stdout:4838 write 0x4cbb400 thru 0x4cbc591 (0x1192 bytes) 2024-05-08T08:54:08.632 INFO:teuthology.orchestra.run.smithi196.stdout:4841 punch from 0xdc2c00 to 0xdd2200, (0xf600 bytes) 2024-05-08T08:54:08.639 INFO:teuthology.orchestra.run.smithi196.stdout:4842 read 0x5112000 thru 0x511cc28 (0xac29 bytes) 2024-05-08T08:54:08.662 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:08.665+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:08.690 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-05-08T08:54:08.701 INFO:teuthology.orchestra.run.smithi196.stdout:4844 write 0x7f11800 thru 0x7f1aa9e (0x929f bytes) 2024-05-08T08:54:08.708 INFO:teuthology.orchestra.run.smithi196.stdout:4845 trunc from 0x7f1ac00 to 0x72cf200 2024-05-08T08:54:08.738 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:08.738 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:54:08.738 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:54:08.739 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:08.748 INFO:teuthology.orchestra.run.smithi196.stdout:4846 write 0x36c200 thru 0x36e125 (0x1f26 bytes) 2024-05-08T08:54:08.749 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:54:08.792 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:08.793+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:08.830 INFO:teuthology.orchestra.run.smithi196.stdout:4847 write 0xcfc7c00 thru 0xcfd09fe (0x8dff bytes) 2024-05-08T08:54:08.838 INFO:teuthology.orchestra.run.smithi196.stdout:4848 trunc from 0xcfd0a00 to 0xbee2200 2024-05-08T08:54:08.863 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:08.866+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:08.876 INFO:teuthology.orchestra.run.smithi196.stdout:4849 write 0x6585c00 thru 0x6592289 (0xc68a bytes) 2024-05-08T08:54:08.891 INFO:teuthology.orchestra.run.smithi196.stdout:4853 trunc from 0xbee2200 to 0x352a200 2024-05-08T08:54:08.916 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:08.916 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:54:08.916 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:54:08.916 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:54:08.916 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:08.926 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:54:08.941 INFO:teuthology.orchestra.run.smithi196.stdout:4857 read 0x1847000 thru 0x184b5e3 (0x45e4 bytes) 2024-05-08T08:54:08.953 INFO:teuthology.orchestra.run.smithi196.stdout:4859 write 0x3c4e000 thru 0x3c54f3d (0x6f3e bytes) 2024-05-08T08:54:08.961 INFO:teuthology.orchestra.run.smithi196.stdout:4860 trunc from 0x3c55000 to 0x5813a00 2024-05-08T08:54:08.969 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:08.972+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:08.982 INFO:teuthology.orchestra.run.smithi196.stdout:4862 read 0x34c2600 thru 0x34c37ff (0x1200 bytes) 2024-05-08T08:54:08.996 INFO:teuthology.orchestra.run.smithi196.stdout:4863 trunc from 0x5813a00 to 0xc180a00 2024-05-08T08:54:09.022 INFO:teuthology.orchestra.run.smithi196.stdout:4864 write 0xbabd200 thru 0xbac2c4b (0x5a4c bytes) 2024-05-08T08:54:09.024 INFO:teuthology.orchestra.run.smithi196.stdout:4865 write 0x26d5800 thru 0x26db043 (0x5844 bytes) 2024-05-08T08:54:09.026 INFO:teuthology.orchestra.run.smithi196.stdout:4866 flatten 2024-05-08T08:54:09.050 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:09.050 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:54:09.050 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:54:09.050 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:54:09.051 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.051 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6690 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1093632~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:09.051 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:54:09.051 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 97.336358525999998, 2024-05-08T08:54:09.051 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052462754, 2024-05-08T08:54:09.051 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:09.051 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:09.051 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:09.051 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:09.051 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:09.051 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:09.051 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6690 2024-05-08T08:54:09.051 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.051 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:09.051 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.052 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:09.052 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:54:09.052 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:09.052 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.052 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.052 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:09.052 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:54:09.052 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:09.052 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.052 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.052 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:09.052 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715346+0000", 2024-05-08T08:54:09.052 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.84e-07 2024-05-08T08:54:09.052 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.052 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.053 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:09.053 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715348+0000", 2024-05-08T08:54:09.053 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.114e-06 2024-05-08T08:54:09.053 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.053 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.053 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:09.053 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715353+0000", 2024-05-08T08:54:09.053 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5720000000000002e-06 2024-05-08T08:54:09.053 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.053 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.053 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:09.053 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715358+0000", 2024-05-08T08:54:09.053 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7140000000000001e-06 2024-05-08T08:54:09.053 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.053 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.053 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:09.053 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767638+0000", 2024-05-08T08:54:09.054 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052279562000000002 2024-05-08T08:54:09.054 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.054 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.054 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:09.054 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767713+0000", 2024-05-08T08:54:09.054 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5652e-05 2024-05-08T08:54:09.054 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.054 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.054 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:09.054 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767808+0000", 2024-05-08T08:54:09.054 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4755999999999999e-05 2024-05-08T08:54:09.054 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:09.054 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:09.054 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:09.054 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.055 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.055 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6692 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1110016~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:09.055 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:54:09.055 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 97.336339436000003, 2024-05-08T08:54:09.055 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052666495000000001, 2024-05-08T08:54:09.055 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:09.055 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:09.055 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:09.055 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:09.055 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:09.055 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:09.055 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6692 2024-05-08T08:54:09.055 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.055 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:09.055 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.056 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:09.056 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:54:09.056 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:09.056 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.056 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.056 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:09.056 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:54:09.056 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:09.056 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.056 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.056 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:09.056 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:54:09.056 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8599999999999999e-07 2024-05-08T08:54:09.056 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.056 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.056 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:09.057 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715372+0000", 2024-05-08T08:54:09.057 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0990000000000001e-06 2024-05-08T08:54:09.057 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.057 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.057 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:09.057 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715378+0000", 2024-05-08T08:54:09.057 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0680000000000002e-06 2024-05-08T08:54:09.057 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.057 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.057 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:09.057 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715383+0000", 2024-05-08T08:54:09.057 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8740000000000003e-06 2024-05-08T08:54:09.057 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.057 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.057 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:09.057 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767848+0000", 2024-05-08T08:54:09.057 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052464668999999998 2024-05-08T08:54:09.058 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.058 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.058 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:09.058 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767927+0000", 2024-05-08T08:54:09.058 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9228999999999996e-05 2024-05-08T08:54:09.058 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.058 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.058 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:09.058 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768031+0000", 2024-05-08T08:54:09.058 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010417 2024-05-08T08:54:09.058 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:09.058 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:09.058 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:09.058 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.058 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.058 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6694 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1126400~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:09.059 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:54:09.059 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 97.336314539, 2024-05-08T08:54:09.059 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052848829, 2024-05-08T08:54:09.059 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:09.059 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:09.059 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:09.059 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:09.059 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:09.059 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:09.059 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6694 2024-05-08T08:54:09.059 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.059 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:09.059 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.059 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:09.059 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:54:09.059 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:09.059 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.060 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.060 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:09.060 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:54:09.060 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:09.060 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.060 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.060 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:09.060 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715390+0000", 2024-05-08T08:54:09.060 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9900000000000001e-07 2024-05-08T08:54:09.060 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.060 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.060 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:09.060 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715392+0000", 2024-05-08T08:54:09.060 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8279999999999999e-06 2024-05-08T08:54:09.060 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.060 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.060 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:09.061 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715398+0000", 2024-05-08T08:54:09.061 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7619999999999998e-06 2024-05-08T08:54:09.061 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.061 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.061 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:09.061 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715403+0000", 2024-05-08T08:54:09.061 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7790000000000003e-06 2024-05-08T08:54:09.061 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.061 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.061 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:09.061 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768067+0000", 2024-05-08T08:54:09.061 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052663785999999997 2024-05-08T08:54:09.061 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.061 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.061 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:09.061 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768142+0000", 2024-05-08T08:54:09.061 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5018999999999994e-05 2024-05-08T08:54:09.062 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.062 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.062 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:09.062 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768238+0000", 2024-05-08T08:54:09.062 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6255999999999995e-05 2024-05-08T08:54:09.062 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:09.062 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:09.062 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:09.062 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.062 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.062 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6696 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1142784~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:09.062 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:54:09.062 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 97.336295031000006, 2024-05-08T08:54:09.062 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.053051057999999998, 2024-05-08T08:54:09.063 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:09.063 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:09.063 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:09.063 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:09.063 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:09.063 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:09.063 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6696 2024-05-08T08:54:09.063 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.063 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:09.063 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.063 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:09.063 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:54:09.063 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:09.063 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.064 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.064 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:09.064 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:54:09.064 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:09.064 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.064 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.064 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:09.064 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:54:09.064 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-05-08T08:54:09.064 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.064 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.064 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:09.064 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715411+0000", 2024-05-08T08:54:09.064 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9630000000000001e-06 2024-05-08T08:54:09.064 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.065 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.065 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:09.065 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715417+0000", 2024-05-08T08:54:09.065 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7250000000000002e-06 2024-05-08T08:54:09.065 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.065 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.065 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:09.065 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715421+0000", 2024-05-08T08:54:09.065 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4089999999999996e-06 2024-05-08T08:54:09.065 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.065 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.065 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:09.065 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768277+0000", 2024-05-08T08:54:09.065 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052855293999999997 2024-05-08T08:54:09.066 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.066 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.066 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:09.066 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768358+0000", 2024-05-08T08:54:09.066 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1720999999999999e-05 2024-05-08T08:54:09.066 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.066 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.066 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:09.066 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768460+0000", 2024-05-08T08:54:09.066 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010154600000000001 2024-05-08T08:54:09.066 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:09.066 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:09.066 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:09.066 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.066 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.067 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6698 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1159168~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:09.067 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:54:09.067 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 97.336276163999997, 2024-05-08T08:54:09.067 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.053245726, 2024-05-08T08:54:09.067 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:09.067 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:09.067 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:09.067 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:09.067 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:09.067 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:09.067 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6698 2024-05-08T08:54:09.067 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.067 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:09.067 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.067 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:09.068 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:54:09.068 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:09.068 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.068 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.068 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:09.068 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:54:09.068 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:09.068 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.068 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.068 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:09.068 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716936+0000", 2024-05-08T08:54:09.068 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001508665 2024-05-08T08:54:09.068 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.068 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.068 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:09.069 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716941+0000", 2024-05-08T08:54:09.069 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1910000000000002e-06 2024-05-08T08:54:09.069 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.069 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.069 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:09.069 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716950+0000", 2024-05-08T08:54:09.069 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1900000000000001e-06 2024-05-08T08:54:09.069 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.069 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.069 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:09.069 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716958+0000", 2024-05-08T08:54:09.069 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8019999999999992e-06 2024-05-08T08:54:09.069 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.069 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.070 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:09.070 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768499+0000", 2024-05-08T08:54:09.070 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051540926000000001 2024-05-08T08:54:09.070 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.070 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.070 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:09.070 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768574+0000", 2024-05-08T08:54:09.070 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4988999999999996e-05 2024-05-08T08:54:09.070 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.070 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.070 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:09.070 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768673+0000", 2024-05-08T08:54:09.070 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9963000000000005e-05 2024-05-08T08:54:09.070 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:09.070 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:09.071 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:09.071 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.071 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.071 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6706 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1224704~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:09.071 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:54:09.071 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 97.334670857000006, 2024-05-08T08:54:09.071 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052435598, 2024-05-08T08:54:09.071 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:09.071 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:09.071 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:09.071 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:09.071 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:09.071 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:09.071 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6706 2024-05-08T08:54:09.071 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.072 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:09.072 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.072 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:09.072 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:54:09.072 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:09.072 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.072 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.072 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:09.072 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:54:09.072 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:09.072 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.072 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.072 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:09.072 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:54:09.072 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:54:09.073 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.073 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.073 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:09.073 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717035+0000", 2024-05-08T08:54:09.073 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9290000000000001e-06 2024-05-08T08:54:09.073 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.073 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.073 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:09.073 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717041+0000", 2024-05-08T08:54:09.073 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6720000000000001e-06 2024-05-08T08:54:09.073 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.073 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.073 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:09.073 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717046+0000", 2024-05-08T08:54:09.073 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5650000000000003e-06 2024-05-08T08:54:09.073 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.073 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.074 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:09.074 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769311+0000", 2024-05-08T08:54:09.074 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052265645999999999 2024-05-08T08:54:09.074 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.074 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.074 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:09.074 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769379+0000", 2024-05-08T08:54:09.074 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7391000000000006e-05 2024-05-08T08:54:09.074 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.074 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.074 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:09.074 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769469+0000", 2024-05-08T08:54:09.074 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0025999999999995e-05 2024-05-08T08:54:09.074 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:09.074 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:09.074 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:09.075 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.075 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.075 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6708 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1241088~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:09.075 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:54:09.075 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 97.334652753, 2024-05-08T08:54:09.075 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052640552, 2024-05-08T08:54:09.075 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:09.075 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:09.075 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:09.075 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:09.075 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:09.075 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:09.075 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6708 2024-05-08T08:54:09.075 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.075 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:09.075 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.076 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:09.076 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:54:09.076 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:09.076 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.076 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.076 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:09.076 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:54:09.076 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:09.076 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.076 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.076 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:09.076 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717052+0000", 2024-05-08T08:54:09.076 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:54:09.076 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.076 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.077 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:09.077 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717053+0000", 2024-05-08T08:54:09.077 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8330000000000001e-06 2024-05-08T08:54:09.077 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.077 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.077 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:09.077 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717058+0000", 2024-05-08T08:54:09.077 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.933e-06 2024-05-08T08:54:09.077 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.077 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.077 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:09.077 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717063+0000", 2024-05-08T08:54:09.077 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2180000000000001e-06 2024-05-08T08:54:09.077 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.077 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.077 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:09.078 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769507+0000", 2024-05-08T08:54:09.078 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052444866999999999 2024-05-08T08:54:09.078 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.078 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.078 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:09.078 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769589+0000", 2024-05-08T08:54:09.078 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2039000000000005e-05 2024-05-08T08:54:09.078 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.078 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.078 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:09.078 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769692+0000", 2024-05-08T08:54:09.078 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102252 2024-05-08T08:54:09.078 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:09.078 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:09.078 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:09.078 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.079 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.079 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6710 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1257472~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:09.079 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:54:09.079 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 97.334634729000001, 2024-05-08T08:54:09.079 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.05283057, 2024-05-08T08:54:09.079 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:09.079 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:09.079 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:09.079 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:09.079 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:09.079 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:09.079 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6710 2024-05-08T08:54:09.079 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.079 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:09.079 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.079 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:09.080 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:54:09.080 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:09.080 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.080 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.080 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:09.080 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:54:09.080 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:09.080 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.080 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.080 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:09.080 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718951+0000", 2024-05-08T08:54:09.080 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0018817809999999999 2024-05-08T08:54:09.080 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.080 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.080 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:09.080 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718956+0000", 2024-05-08T08:54:09.080 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5399999999999997e-06 2024-05-08T08:54:09.081 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.081 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.081 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:09.081 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718964+0000", 2024-05-08T08:54:09.081 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6629999999999999e-06 2024-05-08T08:54:09.081 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.081 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.081 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:09.081 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718972+0000", 2024-05-08T08:54:09.081 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5229999999999998e-06 2024-05-08T08:54:09.081 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.081 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.081 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:09.081 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769729+0000", 2024-05-08T08:54:09.081 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050756959999999997 2024-05-08T08:54:09.081 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.082 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.082 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:09.082 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769803+0000", 2024-05-08T08:54:09.082 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4538999999999999e-05 2024-05-08T08:54:09.082 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.082 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.082 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:09.082 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769900+0000", 2024-05-08T08:54:09.082 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6564000000000006e-05 2024-05-08T08:54:09.082 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:09.082 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:09.082 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:09.082 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.082 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.082 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6726 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1388544~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:09.083 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:54:09.083 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 97.332559957000001, 2024-05-08T08:54:09.083 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052451506000000002, 2024-05-08T08:54:09.083 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:09.083 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:09.083 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:09.083 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:09.083 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:09.083 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:09.083 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6726 2024-05-08T08:54:09.083 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.083 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:09.083 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.083 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:09.083 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:54:09.083 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:09.083 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.084 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.084 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:09.084 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:54:09.084 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:09.084 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.084 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.084 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:09.084 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719200+0000", 2024-05-08T08:54:09.084 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5865999999999997e-05 2024-05-08T08:54:09.084 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.084 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.084 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:09.084 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719204+0000", 2024-05-08T08:54:09.084 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8630000000000001e-06 2024-05-08T08:54:09.084 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.085 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.085 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:09.085 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719215+0000", 2024-05-08T08:54:09.085 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1590000000000001e-05 2024-05-08T08:54:09.085 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.085 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.085 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:09.085 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719220+0000", 2024-05-08T08:54:09.085 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1619999999999999e-06 2024-05-08T08:54:09.085 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.085 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.085 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:09.085 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771425+0000", 2024-05-08T08:54:09.085 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052204249000000001 2024-05-08T08:54:09.085 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.086 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.086 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:09.086 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771498+0000", 2024-05-08T08:54:09.086 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3782999999999995e-05 2024-05-08T08:54:09.086 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.086 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.086 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:09.086 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771595+0000", 2024-05-08T08:54:09.086 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6992999999999998e-05 2024-05-08T08:54:09.086 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:09.086 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:09.086 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:09.086 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.086 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.087 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6728 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1404928~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:09.087 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:54:09.087 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 97.332472551999999, 2024-05-08T08:54:09.087 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052590747, 2024-05-08T08:54:09.087 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:09.087 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:09.087 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:09.087 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:09.087 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:09.087 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:09.087 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6728 2024-05-08T08:54:09.087 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.087 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:09.087 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.087 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:09.088 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:54:09.088 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:09.088 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.088 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.088 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:09.088 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:54:09.088 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:09.088 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.088 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.088 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:09.088 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719232+0000", 2024-05-08T08:54:09.088 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4000000000000001e-07 2024-05-08T08:54:09.088 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.088 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.088 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:09.088 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719234+0000", 2024-05-08T08:54:09.088 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4499999999999998e-06 2024-05-08T08:54:09.089 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.089 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.089 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:09.089 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719240+0000", 2024-05-08T08:54:09.089 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5720000000000002e-06 2024-05-08T08:54:09.089 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.089 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.089 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:09.089 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719245+0000", 2024-05-08T08:54:09.089 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8690000000000003e-06 2024-05-08T08:54:09.089 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.089 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.089 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:09.089 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771636+0000", 2024-05-08T08:54:09.089 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052390979999999997 2024-05-08T08:54:09.090 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.090 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.090 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:09.090 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771717+0000", 2024-05-08T08:54:09.090 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1291999999999993e-05 2024-05-08T08:54:09.090 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.090 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.090 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:09.090 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771822+0000", 2024-05-08T08:54:09.090 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104944 2024-05-08T08:54:09.090 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:09.090 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:09.090 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:09.090 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.090 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.090 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6730 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1421312~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:09.091 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:54:09.091 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 97.332452309999994, 2024-05-08T08:54:09.091 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052770477000000003, 2024-05-08T08:54:09.091 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:09.091 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:09.091 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:09.091 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:09.091 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:09.091 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:09.091 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6730 2024-05-08T08:54:09.091 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.091 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:09.091 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.091 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:09.091 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:54:09.092 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:09.092 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.092 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.092 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:09.092 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:54:09.092 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:09.092 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.092 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.092 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:09.092 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:54:09.092 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9000000000000002e-07 2024-05-08T08:54:09.092 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.092 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.092 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:09.092 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719254+0000", 2024-05-08T08:54:09.093 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4099999999999998e-06 2024-05-08T08:54:09.093 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.093 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.093 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:09.093 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719263+0000", 2024-05-08T08:54:09.093 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4109999999999996e-06 2024-05-08T08:54:09.093 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.093 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.093 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:09.093 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719268+0000", 2024-05-08T08:54:09.093 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2669999999999997e-06 2024-05-08T08:54:09.093 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.093 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.093 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:09.093 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771846+0000", 2024-05-08T08:54:09.093 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052577677000000003 2024-05-08T08:54:09.094 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.094 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.094 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:09.094 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771920+0000", 2024-05-08T08:54:09.094 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4460000000000002e-05 2024-05-08T08:54:09.094 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.094 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.094 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:09.094 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772022+0000", 2024-05-08T08:54:09.094 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010186199999999999 2024-05-08T08:54:09.094 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:09.094 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:09.094 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:09.094 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.094 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.094 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6732 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1437696~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:09.095 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:54:09.095 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 97.332427503000005, 2024-05-08T08:54:09.095 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052905575000000003, 2024-05-08T08:54:09.095 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:09.095 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:09.095 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:09.095 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:09.095 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:09.095 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:09.095 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6732 2024-05-08T08:54:09.095 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.095 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:09.095 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.095 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:09.095 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:54:09.095 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:09.096 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.096 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.096 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:09.096 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:54:09.096 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:09.096 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.096 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.096 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:09.096 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721392+0000", 2024-05-08T08:54:09.096 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002115287 2024-05-08T08:54:09.096 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.096 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.096 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:09.096 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721396+0000", 2024-05-08T08:54:09.096 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1679999999999997e-06 2024-05-08T08:54:09.097 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.097 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.097 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:09.097 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721404+0000", 2024-05-08T08:54:09.097 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4409999999999993e-06 2024-05-08T08:54:09.097 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.097 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.097 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:09.097 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721410+0000", 2024-05-08T08:54:09.097 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9150000000000001e-06 2024-05-08T08:54:09.097 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.097 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.097 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:09.097 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772061+0000", 2024-05-08T08:54:09.097 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050651068 2024-05-08T08:54:09.097 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.098 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.098 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:09.098 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772127+0000", 2024-05-08T08:54:09.098 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5424000000000002e-05 2024-05-08T08:54:09.098 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.098 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.098 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:09.098 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772182+0000", 2024-05-08T08:54:09.098 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5272000000000002e-05 2024-05-08T08:54:09.098 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:09.098 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:09.098 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:09.098 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.098 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.098 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6749 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1585152~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:09.099 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:54:09.099 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 97.330126733, 2024-05-08T08:54:09.099 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052350264000000001, 2024-05-08T08:54:09.099 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:09.099 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:09.099 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:09.099 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:09.099 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:09.099 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:09.099 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6749 2024-05-08T08:54:09.099 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.099 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:09.099 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.099 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:09.099 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:54:09.099 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:09.100 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.100 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.100 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:09.100 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:54:09.100 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:09.100 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.100 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.100 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:09.100 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721578+0000", 2024-05-08T08:54:09.100 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0399999999999996e-07 2024-05-08T08:54:09.100 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.100 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.100 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:09.100 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721580+0000", 2024-05-08T08:54:09.101 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5450000000000002e-06 2024-05-08T08:54:09.101 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.101 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.101 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:09.101 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721587+0000", 2024-05-08T08:54:09.101 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0190000000000001e-06 2024-05-08T08:54:09.101 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.101 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.101 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:09.101 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721593+0000", 2024-05-08T08:54:09.101 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5659999999999997e-06 2024-05-08T08:54:09.101 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.101 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.101 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:09.101 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773759+0000", 2024-05-08T08:54:09.102 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052166553999999997 2024-05-08T08:54:09.102 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.102 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.102 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:09.102 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773833+0000", 2024-05-08T08:54:09.102 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3913999999999997e-05 2024-05-08T08:54:09.102 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.102 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.102 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:09.102 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773927+0000", 2024-05-08T08:54:09.102 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4061999999999995e-05 2024-05-08T08:54:09.102 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:09.102 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:09.102 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:09.102 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.103 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.103 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6751 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1601536~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:09.103 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:54:09.103 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 97.330103453999996, 2024-05-08T08:54:09.103 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052461302000000001, 2024-05-08T08:54:09.103 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:09.103 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:09.103 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:09.103 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:09.103 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:09.103 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:09.103 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6751 2024-05-08T08:54:09.103 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.103 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:09.103 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.104 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:09.104 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:54:09.104 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:09.104 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.104 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.104 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:09.104 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:54:09.104 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:09.104 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.104 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.104 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:09.104 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722939+0000", 2024-05-08T08:54:09.104 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001338601 2024-05-08T08:54:09.104 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.104 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.104 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:09.105 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722944+0000", 2024-05-08T08:54:09.105 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4469999999999999e-06 2024-05-08T08:54:09.105 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.105 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.105 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:09.105 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722955+0000", 2024-05-08T08:54:09.105 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1892999999999999e-05 2024-05-08T08:54:09.105 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.105 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.105 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:09.105 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722963+0000", 2024-05-08T08:54:09.105 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1080000000000004e-06 2024-05-08T08:54:09.105 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.105 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.106 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:09.106 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773964+0000", 2024-05-08T08:54:09.106 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051001022 2024-05-08T08:54:09.106 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.106 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.106 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:09.106 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.774006+0000", 2024-05-08T08:54:09.106 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2729999999999999e-05 2024-05-08T08:54:09.106 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.106 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.106 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:09.106 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.774062+0000", 2024-05-08T08:54:09.106 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5501000000000003e-05 2024-05-08T08:54:09.106 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:09.106 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:09.107 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:09.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.107 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6760 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1683456~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:09.107 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:54:09.107 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 97.328594077999995, 2024-05-08T08:54:09.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052571312000000002, 2024-05-08T08:54:09.107 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:09.107 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:09.107 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:09.107 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:09.107 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:09.107 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:09.107 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6760 2024-05-08T08:54:09.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.108 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:09.108 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.108 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:09.108 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:54:09.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:09.108 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.108 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.108 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:09.108 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:54:09.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:09.108 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.108 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.108 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:09.108 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:54:09.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2100000000000002e-07 2024-05-08T08:54:09.108 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.109 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.109 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:09.109 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723113+0000", 2024-05-08T08:54:09.109 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3149999999999999e-06 2024-05-08T08:54:09.109 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.109 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.109 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:09.109 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723118+0000", 2024-05-08T08:54:09.109 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8860000000000001e-06 2024-05-08T08:54:09.109 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.109 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.109 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:09.109 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723123+0000", 2024-05-08T08:54:09.109 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6770000000000003e-06 2024-05-08T08:54:09.109 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.109 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.109 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:09.110 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775414+0000", 2024-05-08T08:54:09.110 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052290692999999999 2024-05-08T08:54:09.110 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.110 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.110 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:09.110 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775459+0000", 2024-05-08T08:54:09.110 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5639999999999997e-05 2024-05-08T08:54:09.110 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.110 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.110 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:09.110 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775681+0000", 2024-05-08T08:54:09.110 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00022168000000000001 2024-05-08T08:54:09.110 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:09.110 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:09.110 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:09.111 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.111 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.111 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6761 3.16 3:6bd6d6da:::rbd_data.1140782590a5.0000000000000015:6 [read 1095680~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:09.111 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:54:09.111 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 97.328574857999996, 2024-05-08T08:54:09.111 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052751275, 2024-05-08T08:54:09.111 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:09.111 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:09.111 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:09.111 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:09.111 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:09.111 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:09.111 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6761 2024-05-08T08:54:09.111 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.112 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:09.112 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.112 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:09.112 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:54:09.112 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:09.112 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.112 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.112 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:09.112 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:54:09.112 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:09.112 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.112 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.112 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:09.112 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724939+0000", 2024-05-08T08:54:09.112 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001809817 2024-05-08T08:54:09.112 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.112 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.113 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:09.113 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724943+0000", 2024-05-08T08:54:09.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4920000000000004e-06 2024-05-08T08:54:09.113 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.113 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.113 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:09.113 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724956+0000", 2024-05-08T08:54:09.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2427e-05 2024-05-08T08:54:09.113 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.113 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.113 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:09.113 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724963+0000", 2024-05-08T08:54:09.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2559999999999996e-06 2024-05-08T08:54:09.113 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.114 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.114 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:09.114 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775711+0000", 2024-05-08T08:54:09.114 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050747621 2024-05-08T08:54:09.114 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.114 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.114 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:09.114 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775785+0000", 2024-05-08T08:54:09.114 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4634000000000004e-05 2024-05-08T08:54:09.114 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.114 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.114 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:09.114 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775880+0000", 2024-05-08T08:54:09.114 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5027999999999999e-05 2024-05-08T08:54:09.114 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:09.115 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:09.115 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:09.115 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.115 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.115 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6770 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1765376~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:09.115 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:54:09.115 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 97.326543763000004, 2024-05-08T08:54:09.115 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052339296, 2024-05-08T08:54:09.115 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:09.115 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:09.115 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:09.115 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:09.115 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:09.115 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:09.115 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6770 2024-05-08T08:54:09.115 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.116 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:09.116 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.116 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:09.116 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:54:09.116 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:09.116 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.116 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.116 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:09.116 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:54:09.116 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:09.116 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.116 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.116 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:09.116 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725161+0000", 2024-05-08T08:54:09.116 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-05-08T08:54:09.116 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.117 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.117 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:09.117 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725163+0000", 2024-05-08T08:54:09.117 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4370000000000001e-06 2024-05-08T08:54:09.117 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.117 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.117 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:09.117 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725169+0000", 2024-05-08T08:54:09.117 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8409999999999996e-06 2024-05-08T08:54:09.117 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.117 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.117 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:09.117 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725174+0000", 2024-05-08T08:54:09.117 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7249999999999997e-06 2024-05-08T08:54:09.117 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.118 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.118 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:09.118 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777343+0000", 2024-05-08T08:54:09.118 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052169633999999999 2024-05-08T08:54:09.118 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.118 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.118 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:09.118 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777409+0000", 2024-05-08T08:54:09.118 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6033999999999998e-05 2024-05-08T08:54:09.118 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.118 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.118 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:09.118 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777499+0000", 2024-05-08T08:54:09.118 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0224999999999998e-05 2024-05-08T08:54:09.118 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:09.118 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:09.119 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:09.119 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.119 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.119 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6771 3.16 3:6bd6d6da:::rbd_data.1140782590a5.0000000000000015:6 [read 1177600~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:09.119 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:54:09.119 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 97.326523945999995, 2024-05-08T08:54:09.119 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052438604999999999, 2024-05-08T08:54:09.119 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:09.119 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:09.119 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:09.119 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:09.119 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:09.119 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:09.119 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6771 2024-05-08T08:54:09.119 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.119 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:09.120 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.120 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:09.120 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:54:09.120 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:09.120 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.120 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.120 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:09.120 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:54:09.120 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:09.120 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.120 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.120 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:09.120 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:54:09.120 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:54:09.120 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.120 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.121 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:09.121 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725188+0000", 2024-05-08T08:54:09.121 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1579999999999999e-06 2024-05-08T08:54:09.121 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.121 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.121 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:09.121 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725195+0000", 2024-05-08T08:54:09.121 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8909999999999995e-06 2024-05-08T08:54:09.121 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.121 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.121 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:09.121 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725200+0000", 2024-05-08T08:54:09.121 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9810000000000003e-06 2024-05-08T08:54:09.121 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.121 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.122 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:09.122 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777527+0000", 2024-05-08T08:54:09.122 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052326582000000003 2024-05-08T08:54:09.122 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.122 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.122 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:09.122 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777566+0000", 2024-05-08T08:54:09.122 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9447999999999999e-05 2024-05-08T08:54:09.122 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.122 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.122 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:09.122 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777619+0000", 2024-05-08T08:54:09.122 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2135000000000001e-05 2024-05-08T08:54:09.122 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:09.122 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:09.122 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:09.122 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.123 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.123 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6772 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1781760~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:09.123 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:54:09.123 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 97.326495889, 2024-05-08T08:54:09.123 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052601799999999997, 2024-05-08T08:54:09.123 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:09.123 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:09.123 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:09.123 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:09.123 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:09.123 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:09.123 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6772 2024-05-08T08:54:09.123 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.123 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:09.123 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.124 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:09.124 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:54:09.124 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:09.124 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.124 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.124 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:09.124 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:54:09.124 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:09.124 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.124 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.124 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:09.124 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726939+0000", 2024-05-08T08:54:09.124 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001730493 2024-05-08T08:54:09.124 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.124 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.125 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:09.125 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726943+0000", 2024-05-08T08:54:09.125 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7330000000000003e-06 2024-05-08T08:54:09.125 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.125 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.125 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:09.125 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726954+0000", 2024-05-08T08:54:09.125 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1107000000000001e-05 2024-05-08T08:54:09.125 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.125 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.125 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:09.125 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726962+0000", 2024-05-08T08:54:09.125 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.503e-06 2024-05-08T08:54:09.125 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.125 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.125 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:09.125 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777653+0000", 2024-05-08T08:54:09.126 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050691542999999999 2024-05-08T08:54:09.126 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.126 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.126 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:09.126 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777719+0000", 2024-05-08T08:54:09.126 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5228000000000006e-05 2024-05-08T08:54:09.126 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.126 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.126 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:09.126 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777810+0000", 2024-05-08T08:54:09.126 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1193000000000006e-05 2024-05-08T08:54:09.126 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:09.126 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:09.126 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:09.126 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.127 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.127 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6782 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1880064~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:09.127 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:54:09.127 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 97.324527381999999, 2024-05-08T08:54:09.127 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052326492000000002, 2024-05-08T08:54:09.127 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:09.127 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:09.127 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:09.127 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:09.127 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:09.127 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:09.127 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6782 2024-05-08T08:54:09.127 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.127 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:09.127 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.128 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:09.128 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:54:09.128 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:09.128 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.128 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.128 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:09.128 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:54:09.128 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:09.128 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.128 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.128 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:09.128 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728939+0000", 2024-05-08T08:54:09.128 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001762719 2024-05-08T08:54:09.128 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.128 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.129 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:09.129 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728944+0000", 2024-05-08T08:54:09.129 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.746e-06 2024-05-08T08:54:09.129 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.129 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.129 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:09.129 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728955+0000", 2024-05-08T08:54:09.129 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1043000000000001e-05 2024-05-08T08:54:09.129 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.129 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.129 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:09.129 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728963+0000", 2024-05-08T08:54:09.129 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5809999999999999e-06 2024-05-08T08:54:09.129 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.129 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.129 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:09.130 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779320+0000", 2024-05-08T08:54:09.130 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050357290999999998 2024-05-08T08:54:09.130 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.130 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.130 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:09.130 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779401+0000", 2024-05-08T08:54:09.130 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0574000000000005e-05 2024-05-08T08:54:09.130 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:09.130 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:09.130 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:09.130 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779503+0000", 2024-05-08T08:54:09.130 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102538 2024-05-08T08:54:09.130 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:09.130 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:09.130 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:09.130 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:09.131 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:09.131 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:09.132 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:54:09.133 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:09.134+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:09.233 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:09.235+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:09.281 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:09.281 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:54:09.281 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:54:09.282 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:09.293 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:54:09.333 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:09.336+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:09.415 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:09.415 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:54:09.415 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:54:09.415 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:54:09.415 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:09.424 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:54:09.434 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:09.437+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:09.471 INFO:teuthology.orchestra.run.smithi196.stdout:4867 read 0xb01a400 thru 0xb024cc1 (0xa8c2 bytes) 2024-05-08T08:54:09.472 INFO:teuthology.orchestra.run.smithi196.stdout:4870 write 0xa249a00 thru 0xa24f4bd (0x5abe bytes) 2024-05-08T08:54:09.474 INFO:teuthology.orchestra.run.smithi196.stdout:4872 trunc from 0xc180a00 to 0xa5a1000 2024-05-08T08:54:09.497 INFO:teuthology.orchestra.run.smithi196.stdout:4875 read 0x8918200 thru 0x891d67b (0x547c bytes) 2024-05-08T08:54:09.498 INFO:teuthology.orchestra.run.smithi196.stdout:4877 write 0x78af800 thru 0x78b6e54 (0x7655 bytes) 2024-05-08T08:54:09.504 INFO:teuthology.orchestra.run.smithi196.stdout:4878 trunc from 0xa5a1000 to 0x55e4200 2024-05-08T08:54:09.535 INFO:teuthology.orchestra.run.smithi196.stdout:4879 read 0x547800 thru 0x55492b (0xd12c bytes) 2024-05-08T08:54:09.535 INFO:teuthology.orchestra.run.smithi196.stdout:4880 read 0x50a3a00 thru 0x50a5240 (0x1841 bytes) 2024-05-08T08:54:09.535 INFO:teuthology.orchestra.run.smithi196.stdout:4882 write 0x2cb6200 thru 0x2cc436d (0xe16e bytes) 2024-05-08T08:54:09.535 INFO:teuthology.orchestra.run.smithi196.stdout:4888 read 0x55a5c00 thru 0x55b0688 (0xaa89 bytes) 2024-05-08T08:54:09.535 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:09.538+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:09.536 INFO:teuthology.orchestra.run.smithi196.stdout:4889 trunc from 0x55e4200 to 0x6a74200 2024-05-08T08:54:09.551 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:09.551 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:54:09.552 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:54:09.552 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:54:09.552 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.552 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4621.0:9089 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 23339520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:54:09.552 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:54:09.552 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 97.060262644999995, 2024-05-08T08:54:09.552 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.13814907200000001, 2024-05-08T08:54:09.552 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:09.552 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:09.552 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:54:09.552 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:09.552 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4621", 2024-05-08T08:54:09.552 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:09.552 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9089 2024-05-08T08:54:09.552 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.553 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:09.553 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.553 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:09.553 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:54:09.553 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:09.553 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.553 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.553 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:09.553 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:54:09.553 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:09.553 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.553 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.553 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:09.553 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493005+0000", 2024-05-08T08:54:09.553 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1700000000000001e-06 2024-05-08T08:54:09.554 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.554 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.554 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:09.554 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.513963+0000", 2024-05-08T08:54:09.554 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.020957581999999999 2024-05-08T08:54:09.554 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.554 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.554 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:09.554 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.513995+0000", 2024-05-08T08:54:09.554 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1993999999999999e-05 2024-05-08T08:54:09.554 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.554 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.554 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:09.554 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514007+0000", 2024-05-08T08:54:09.555 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1581e-05 2024-05-08T08:54:09.555 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.555 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.555 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:09.555 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514122+0000", 2024-05-08T08:54:09.555 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000115282 2024-05-08T08:54:09.555 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.555 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.555 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op must wait for map", 2024-05-08T08:54:09.555 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514128+0000", 2024-05-08T08:54:09.555 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9950000000000002e-06 2024-05-08T08:54:09.555 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.555 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.555 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:09.556 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.515800+0000", 2024-05-08T08:54:09.556 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001671886 2024-05-08T08:54:09.556 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.556 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.556 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:09.556 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.515893+0000", 2024-05-08T08:54:09.556 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3490000000000001e-05 2024-05-08T08:54:09.556 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.556 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.556 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 3", 2024-05-08T08:54:09.556 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.517647+0000", 2024-05-08T08:54:09.556 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0017535630000000001 2024-05-08T08:54:09.556 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.556 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.556 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:54:09.557 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.627578+0000", 2024-05-08T08:54:09.557 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.109931557 2024-05-08T08:54:09.557 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.557 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.557 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:54:09.557 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631055+0000", 2024-05-08T08:54:09.557 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0034761420000000002 2024-05-08T08:54:09.557 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.557 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.557 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:54:09.557 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631096+0000", 2024-05-08T08:54:09.557 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1059000000000001e-05 2024-05-08T08:54:09.557 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.557 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.558 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:09.558 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631151+0000", 2024-05-08T08:54:09.558 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5770999999999998e-05 2024-05-08T08:54:09.558 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:09.558 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:09.558 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:09.558 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.558 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.558 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1071 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11026432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:09.558 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:54:09.558 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 93.067813154000007, 2024-05-08T08:54:09.558 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070580037999999998, 2024-05-08T08:54:09.558 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:09.558 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:09.559 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:09.559 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:09.559 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:09.559 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:09.559 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1071 2024-05-08T08:54:09.559 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.559 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:09.559 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.559 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:09.559 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:54:09.559 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:09.559 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.559 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.559 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:09.559 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:54:09.560 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:09.560 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.560 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.560 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:09.560 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:54:09.560 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2800000000000002e-07 2024-05-08T08:54:09.560 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.560 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.560 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:09.560 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485454+0000", 2024-05-08T08:54:09.560 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2290000000000002e-06 2024-05-08T08:54:09.560 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.560 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.560 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:09.560 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485461+0000", 2024-05-08T08:54:09.561 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9920000000000001e-06 2024-05-08T08:54:09.561 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.561 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.561 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:09.561 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485467+0000", 2024-05-08T08:54:09.561 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3979999999999998e-06 2024-05-08T08:54:09.561 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.561 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.561 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:09.561 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555880+0000", 2024-05-08T08:54:09.561 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070413165 2024-05-08T08:54:09.561 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.561 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.561 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:09.561 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555926+0000", 2024-05-08T08:54:09.562 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6217000000000002e-05 2024-05-08T08:54:09.562 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.562 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.562 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:09.562 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556032+0000", 2024-05-08T08:54:09.562 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000105609 2024-05-08T08:54:09.562 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:09.562 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:09.562 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:09.562 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.562 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.562 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1073 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11067392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:09.562 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:54:09.562 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 93.067791932999995, 2024-05-08T08:54:09.563 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070821404000000004, 2024-05-08T08:54:09.563 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:09.563 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:09.563 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:09.563 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:09.563 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:09.563 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:09.563 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1073 2024-05-08T08:54:09.563 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.563 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:09.563 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.563 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:09.563 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:54:09.563 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:09.563 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.564 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.564 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:09.564 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:54:09.564 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:09.564 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.564 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.564 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:09.564 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:54:09.564 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.08e-07 2024-05-08T08:54:09.564 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.564 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.564 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:09.564 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485476+0000", 2024-05-08T08:54:09.564 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3929999999999998e-06 2024-05-08T08:54:09.565 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.565 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.565 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:09.565 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485485+0000", 2024-05-08T08:54:09.565 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1880000000000008e-06 2024-05-08T08:54:09.565 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.565 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.565 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:09.565 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485494+0000", 2024-05-08T08:54:09.565 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3109999999999995e-06 2024-05-08T08:54:09.565 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.565 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.565 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:09.565 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556082+0000", 2024-05-08T08:54:09.566 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070587426999999994 2024-05-08T08:54:09.566 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.566 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.566 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:09.566 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556177+0000", 2024-05-08T08:54:09.566 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5051999999999997e-05 2024-05-08T08:54:09.566 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.566 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.566 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:09.566 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556294+0000", 2024-05-08T08:54:09.566 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000117625 2024-05-08T08:54:09.566 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:09.566 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:09.567 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:09.567 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.567 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.567 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1075 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11108352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:09.567 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:54:09.567 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 93.067761356000005, 2024-05-08T08:54:09.567 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071045026999999997, 2024-05-08T08:54:09.567 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:09.567 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:09.567 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:09.567 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:09.567 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:09.567 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:09.567 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1075 2024-05-08T08:54:09.568 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.568 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:09.568 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.568 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:09.568 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:54:09.568 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:09.568 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.568 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.568 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:09.568 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:54:09.568 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:09.568 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.568 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.568 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:09.568 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:54:09.569 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.32e-07 2024-05-08T08:54:09.569 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.569 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.569 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:09.569 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485518+0000", 2024-05-08T08:54:09.569 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3769e-05 2024-05-08T08:54:09.569 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.569 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.569 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:09.569 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485527+0000", 2024-05-08T08:54:09.569 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6779999999999995e-06 2024-05-08T08:54:09.569 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.569 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.569 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:09.570 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485532+0000", 2024-05-08T08:54:09.570 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9830000000000004e-06 2024-05-08T08:54:09.570 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.570 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.570 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:09.570 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556340+0000", 2024-05-08T08:54:09.570 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070807743000000006 2024-05-08T08:54:09.570 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.570 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.570 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:09.570 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556435+0000", 2024-05-08T08:54:09.570 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4732e-05 2024-05-08T08:54:09.570 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.570 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.571 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:09.571 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556549+0000", 2024-05-08T08:54:09.571 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011369 2024-05-08T08:54:09.571 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:09.571 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:09.571 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:09.571 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.571 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.571 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1078 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11149312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:09.571 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:54:09.571 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 93.067721156999994, 2024-05-08T08:54:09.571 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071167276000000002, 2024-05-08T08:54:09.571 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:09.571 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:09.572 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:09.572 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:09.572 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:09.572 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:09.572 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1078 2024-05-08T08:54:09.572 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.572 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:09.572 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.572 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:09.572 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:54:09.572 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:09.572 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.572 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.572 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:09.573 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:54:09.573 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:09.573 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.573 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.573 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:09.573 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:54:09.573 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5600000000000001e-07 2024-05-08T08:54:09.573 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.573 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.573 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:09.573 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485547+0000", 2024-05-08T08:54:09.573 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3590000000000002e-06 2024-05-08T08:54:09.573 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.573 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.574 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:09.574 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485553+0000", 2024-05-08T08:54:09.574 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2770000000000002e-06 2024-05-08T08:54:09.574 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.574 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.574 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:09.574 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485558+0000", 2024-05-08T08:54:09.574 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.95e-06 2024-05-08T08:54:09.574 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.574 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.574 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:09.574 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556582+0000", 2024-05-08T08:54:09.574 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071024444000000006 2024-05-08T08:54:09.574 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.575 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.575 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:09.575 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556637+0000", 2024-05-08T08:54:09.575 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4672000000000001e-05 2024-05-08T08:54:09.575 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.575 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.575 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:09.575 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556711+0000", 2024-05-08T08:54:09.575 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4117999999999997e-05 2024-05-08T08:54:09.575 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:09.575 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:09.575 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:09.575 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.575 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.576 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1080 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11190272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:09.576 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:54:09.576 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 93.067700423999995, 2024-05-08T08:54:09.576 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071350721000000006, 2024-05-08T08:54:09.576 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:09.576 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:09.576 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:09.576 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:09.576 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:09.576 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:09.576 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1080 2024-05-08T08:54:09.576 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.576 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:09.576 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.576 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:09.577 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:54:09.577 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:09.577 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.577 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.577 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:09.577 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:54:09.577 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:09.577 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.577 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.577 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:09.577 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:54:09.577 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.03e-07 2024-05-08T08:54:09.577 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.577 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.578 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:09.578 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485567+0000", 2024-05-08T08:54:09.578 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1090000000000001e-06 2024-05-08T08:54:09.578 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.578 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.578 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:09.578 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485573+0000", 2024-05-08T08:54:09.578 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3049999999999998e-06 2024-05-08T08:54:09.578 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.578 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.578 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:09.578 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485578+0000", 2024-05-08T08:54:09.578 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0130000000000001e-06 2024-05-08T08:54:09.579 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.579 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.579 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:09.579 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556743+0000", 2024-05-08T08:54:09.579 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071164449000000005 2024-05-08T08:54:09.579 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.579 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.579 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:09.579 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556797+0000", 2024-05-08T08:54:09.579 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4589999999999997e-05 2024-05-08T08:54:09.579 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.579 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.579 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:09.579 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556915+0000", 2024-05-08T08:54:09.579 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000117852 2024-05-08T08:54:09.580 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:09.580 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:09.580 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:09.580 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.580 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.580 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1082 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11231232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:09.580 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:54:09.580 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 93.067680181, 2024-05-08T08:54:09.580 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071591396000000002, 2024-05-08T08:54:09.580 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:09.580 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:09.580 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:09.580 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:09.580 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:09.580 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:09.581 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1082 2024-05-08T08:54:09.581 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.581 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:09.581 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.581 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:09.581 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:54:09.581 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:09.581 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.581 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.581 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:09.581 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:54:09.581 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:09.581 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.581 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.581 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:09.582 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:54:09.582 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1800000000000001e-07 2024-05-08T08:54:09.582 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.582 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.582 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:09.582 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485587+0000", 2024-05-08T08:54:09.582 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2589999999999999e-06 2024-05-08T08:54:09.582 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.582 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.582 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:09.582 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485594+0000", 2024-05-08T08:54:09.582 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0789999999999997e-06 2024-05-08T08:54:09.582 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.582 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.582 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:09.583 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485598+0000", 2024-05-08T08:54:09.583 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7480000000000001e-06 2024-05-08T08:54:09.583 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.583 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.583 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:09.583 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556971+0000", 2024-05-08T08:54:09.583 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071372774999999999 2024-05-08T08:54:09.583 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.583 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.583 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:09.583 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557069+0000", 2024-05-08T08:54:09.583 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.7732999999999994e-05 2024-05-08T08:54:09.583 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.583 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.583 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:09.584 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557176+0000", 2024-05-08T08:54:09.584 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000107384 2024-05-08T08:54:09.584 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:09.584 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:09.584 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:09.584 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.584 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.584 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1084 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11272192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:09.584 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:54:09.584 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 93.067641542000004, 2024-05-08T08:54:09.584 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071808977999999996, 2024-05-08T08:54:09.584 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:09.584 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:09.584 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:09.584 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:09.584 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:09.585 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:09.585 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1084 2024-05-08T08:54:09.585 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.585 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:09.585 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.585 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:09.585 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:54:09.585 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:09.585 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.585 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.585 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:09.585 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:54:09.585 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:09.585 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.585 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.585 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:09.585 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485624+0000", 2024-05-08T08:54:09.586 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:54:09.586 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.586 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.586 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:09.586 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485626+0000", 2024-05-08T08:54:09.586 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9460000000000001e-06 2024-05-08T08:54:09.586 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.586 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.586 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:09.586 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485633+0000", 2024-05-08T08:54:09.586 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1149999999999996e-06 2024-05-08T08:54:09.586 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.586 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.586 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:09.587 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485638+0000", 2024-05-08T08:54:09.587 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5670000000000002e-06 2024-05-08T08:54:09.587 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.587 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.587 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:09.587 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557226+0000", 2024-05-08T08:54:09.587 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071587267999999996 2024-05-08T08:54:09.587 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.587 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.587 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:09.587 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557321+0000", 2024-05-08T08:54:09.587 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5785000000000005e-05 2024-05-08T08:54:09.587 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.587 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.587 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:09.587 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557432+0000", 2024-05-08T08:54:09.588 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011092800000000001 2024-05-08T08:54:09.588 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:09.588 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:09.588 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:09.588 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.588 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.588 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1086 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11313152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:09.588 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:54:09.588 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 93.067618214999996, 2024-05-08T08:54:09.588 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071933132999999996, 2024-05-08T08:54:09.588 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:09.588 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:09.588 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:09.588 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:09.589 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:09.589 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:09.589 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1086 2024-05-08T08:54:09.589 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.589 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:09.589 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.589 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:09.589 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:54:09.589 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:09.589 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.589 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.589 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:09.589 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:54:09.589 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:09.589 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.590 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.590 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:09.590 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486328+0000", 2024-05-08T08:54:09.590 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00068107900000000002 2024-05-08T08:54:09.590 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.590 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.590 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:09.590 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486331+0000", 2024-05-08T08:54:09.590 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.585e-06 2024-05-08T08:54:09.590 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.590 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.590 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:09.590 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486343+0000", 2024-05-08T08:54:09.590 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1856e-05 2024-05-08T08:54:09.591 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.591 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.591 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:09.591 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486350+0000", 2024-05-08T08:54:09.591 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8600000000000004e-06 2024-05-08T08:54:09.591 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.591 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.591 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:09.591 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557470+0000", 2024-05-08T08:54:09.591 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071120326999999997 2024-05-08T08:54:09.591 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.591 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.591 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:09.591 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557514+0000", 2024-05-08T08:54:09.592 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3909000000000003e-05 2024-05-08T08:54:09.592 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.592 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.592 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:09.592 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557580+0000", 2024-05-08T08:54:09.592 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5517000000000003e-05 2024-05-08T08:54:09.592 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:09.592 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:09.592 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:09.592 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.592 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.592 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1089 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11354112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:09.592 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:54:09.592 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 93.066905343000002, 2024-05-08T08:54:09.593 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071457381, 2024-05-08T08:54:09.593 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:09.593 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:09.593 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:09.593 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:09.593 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:09.593 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:09.593 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1089 2024-05-08T08:54:09.593 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.593 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:09.593 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.593 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:09.593 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:54:09.593 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:09.593 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.594 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.594 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:09.594 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:54:09.594 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:09.594 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.594 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.594 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:09.594 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486851+0000", 2024-05-08T08:54:09.594 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00049102 2024-05-08T08:54:09.594 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.594 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.594 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:09.594 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486854+0000", 2024-05-08T08:54:09.594 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6859999999999999e-06 2024-05-08T08:54:09.595 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.595 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.595 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:09.595 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486864+0000", 2024-05-08T08:54:09.595 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6080000000000002e-06 2024-05-08T08:54:09.595 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.595 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.595 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:09.595 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486870+0000", 2024-05-08T08:54:09.595 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.477e-06 2024-05-08T08:54:09.595 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.595 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.595 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:09.595 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557612+0000", 2024-05-08T08:54:09.595 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070741557999999996 2024-05-08T08:54:09.596 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.596 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.596 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:09.596 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557695+0000", 2024-05-08T08:54:09.596 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3300000000000005e-05 2024-05-08T08:54:09.596 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.596 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.596 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:09.596 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557817+0000", 2024-05-08T08:54:09.596 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012173200000000001 2024-05-08T08:54:09.596 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:09.596 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:09.596 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:09.596 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.596 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.597 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1091 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11395072~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:09.597 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:54:09.597 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 93.066384548000002, 2024-05-08T08:54:09.597 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071184737999999997, 2024-05-08T08:54:09.597 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:09.597 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:09.597 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:09.597 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:09.597 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:09.597 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:09.597 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1091 2024-05-08T08:54:09.597 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.597 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:09.597 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.597 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:09.597 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:54:09.598 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:09.598 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.598 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.598 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:09.598 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:54:09.598 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:09.598 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.598 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.598 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:09.598 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486881+0000", 2024-05-08T08:54:09.602 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0699999999999997e-07 2024-05-08T08:54:09.602 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.602 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.602 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:09.603 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486883+0000", 2024-05-08T08:54:09.603 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.402e-06 2024-05-08T08:54:09.603 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.603 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.603 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:09.603 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486890+0000", 2024-05-08T08:54:09.603 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0480000000000001e-06 2024-05-08T08:54:09.603 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.603 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.603 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:09.603 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486896+0000", 2024-05-08T08:54:09.603 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7640000000000002e-06 2024-05-08T08:54:09.603 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.604 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.604 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:09.604 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557860+0000", 2024-05-08T08:54:09.604 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070963791999999998 2024-05-08T08:54:09.604 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.604 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.604 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:09.604 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557952+0000", 2024-05-08T08:54:09.604 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2082000000000004e-05 2024-05-08T08:54:09.604 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.604 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.604 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:09.604 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558065+0000", 2024-05-08T08:54:09.604 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000113143 2024-05-08T08:54:09.605 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:09.605 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:09.605 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:09.605 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.605 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.605 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1094 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11436032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:09.605 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:54:09.605 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 93.066361380000004, 2024-05-08T08:54:09.605 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071400507000000002, 2024-05-08T08:54:09.605 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:09.605 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:09.605 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:09.605 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:09.605 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:09.605 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:09.606 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1094 2024-05-08T08:54:09.606 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.606 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:09.606 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.606 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:09.606 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:54:09.606 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:09.606 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.606 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.606 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:09.606 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:54:09.606 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:09.606 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.606 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.607 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:09.607 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487940+0000", 2024-05-08T08:54:09.607 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001036868 2024-05-08T08:54:09.607 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.607 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.607 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:09.607 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487944+0000", 2024-05-08T08:54:09.607 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7069999999999998e-06 2024-05-08T08:54:09.607 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.607 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.607 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:09.607 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487951+0000", 2024-05-08T08:54:09.607 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2649999999999999e-06 2024-05-08T08:54:09.607 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.608 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.608 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:09.608 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487961+0000", 2024-05-08T08:54:09.608 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4110000000000002e-06 2024-05-08T08:54:09.608 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.608 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.608 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:09.608 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558117+0000", 2024-05-08T08:54:09.608 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070156662999999994 2024-05-08T08:54:09.608 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.608 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.608 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:09.608 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558214+0000", 2024-05-08T08:54:09.608 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6444e-05 2024-05-08T08:54:09.609 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.609 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.609 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:09.609 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558304+0000", 2024-05-08T08:54:09.609 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0148999999999994e-05 2024-05-08T08:54:09.609 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:09.609 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:09.609 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:09.609 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.609 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.609 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1096 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11476992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:09.609 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:54:09.609 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 93.065294848999997, 2024-05-08T08:54:09.609 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070480074000000004, 2024-05-08T08:54:09.610 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:09.610 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:09.610 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:09.610 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:09.610 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:09.610 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:09.610 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1096 2024-05-08T08:54:09.610 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.610 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:09.610 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.610 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:09.610 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:54:09.610 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:09.610 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.611 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.611 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:09.611 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:54:09.611 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:09.611 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.611 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.611 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:09.611 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:54:09.611 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3700000000000001e-07 2024-05-08T08:54:09.611 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.611 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.611 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:09.611 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487973+0000", 2024-05-08T08:54:09.611 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0870000000000001e-06 2024-05-08T08:54:09.612 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.612 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.612 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:09.612 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487977+0000", 2024-05-08T08:54:09.612 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0600000000000001e-06 2024-05-08T08:54:09.612 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.612 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.612 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:09.612 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487980+0000", 2024-05-08T08:54:09.612 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1089999999999998e-06 2024-05-08T08:54:09.612 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.612 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.612 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:09.612 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558338+0000", 2024-05-08T08:54:09.613 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070358233000000006 2024-05-08T08:54:09.613 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.613 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.613 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:09.613 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558392+0000", 2024-05-08T08:54:09.613 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4537999999999999e-05 2024-05-08T08:54:09.613 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.613 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.613 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:09.613 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558450+0000", 2024-05-08T08:54:09.613 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7710000000000001e-05 2024-05-08T08:54:09.613 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:09.613 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:09.613 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:09.614 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.614 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.614 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1099 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11517952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:09.614 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:54:09.614 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 93.065280973, 2024-05-08T08:54:09.614 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070720506000000002, 2024-05-08T08:54:09.614 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:09.614 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:09.614 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:09.614 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:09.614 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:09.614 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:09.614 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1099 2024-05-08T08:54:09.614 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.614 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:09.615 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.615 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:09.615 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:54:09.615 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:09.615 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.615 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.615 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:09.615 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:54:09.615 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:09.615 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.615 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.615 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:09.615 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:54:09.615 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3999999999999998e-07 2024-05-08T08:54:09.615 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.616 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.616 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:09.616 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487986+0000", 2024-05-08T08:54:09.616 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3519999999999999e-06 2024-05-08T08:54:09.616 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.616 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.616 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:09.616 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487989+0000", 2024-05-08T08:54:09.616 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.23e-06 2024-05-08T08:54:09.616 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.616 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.616 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:09.616 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487991+0000", 2024-05-08T08:54:09.616 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6350000000000002e-06 2024-05-08T08:54:09.616 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.617 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.617 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:09.617 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558489+0000", 2024-05-08T08:54:09.617 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070497924000000003 2024-05-08T08:54:09.617 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.617 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.617 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:09.617 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558583+0000", 2024-05-08T08:54:09.617 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3405000000000004e-05 2024-05-08T08:54:09.617 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.617 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.617 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:09.617 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558704+0000", 2024-05-08T08:54:09.617 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012171999999999999 2024-05-08T08:54:09.617 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:09.618 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:09.618 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:09.618 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.618 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.618 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1100 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11558912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:09.618 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:54:09.618 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 93.065267973999994, 2024-05-08T08:54:09.618 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070962266999999996, 2024-05-08T08:54:09.618 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:09.618 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:09.618 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:09.618 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:09.618 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:09.618 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:09.619 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1100 2024-05-08T08:54:09.619 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.619 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:09.619 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.619 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:09.619 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:54:09.619 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:09.619 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.619 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.619 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:09.619 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:54:09.619 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:09.619 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.619 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.619 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:09.619 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488430+0000", 2024-05-08T08:54:09.620 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000432676 2024-05-08T08:54:09.620 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.620 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.620 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:09.620 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488433+0000", 2024-05-08T08:54:09.620 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.45e-06 2024-05-08T08:54:09.620 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.620 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.620 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:09.620 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488442+0000", 2024-05-08T08:54:09.620 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.687e-06 2024-05-08T08:54:09.620 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.620 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.620 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:09.620 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488446+0000", 2024-05-08T08:54:09.621 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4939999999999997e-06 2024-05-08T08:54:09.621 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.621 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.621 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:09.621 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558756+0000", 2024-05-08T08:54:09.621 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070310051999999998 2024-05-08T08:54:09.621 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.621 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.621 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:09.621 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558844+0000", 2024-05-08T08:54:09.621 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7690999999999998e-05 2024-05-08T08:54:09.621 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.621 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.621 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:09.621 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558959+0000", 2024-05-08T08:54:09.621 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000115217 2024-05-08T08:54:09.621 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:09.622 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:09.622 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:09.622 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.622 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.622 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1103 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11599872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:09.622 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:54:09.622 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 93.064810299000001, 2024-05-08T08:54:09.622 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070777100999999995, 2024-05-08T08:54:09.622 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:09.622 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:09.622 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:09.622 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:09.622 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:09.622 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:09.623 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1103 2024-05-08T08:54:09.623 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.623 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:09.623 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.623 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:09.623 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:54:09.623 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:09.623 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.623 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.623 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:09.623 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:54:09.623 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:09.623 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.623 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.623 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:09.624 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488959+0000", 2024-05-08T08:54:09.624 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00050457200000000001 2024-05-08T08:54:09.624 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.624 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.624 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:09.624 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488969+0000", 2024-05-08T08:54:09.624 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.7159999999999998e-06 2024-05-08T08:54:09.624 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.624 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.624 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:09.624 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488981+0000", 2024-05-08T08:54:09.624 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2150000000000001e-05 2024-05-08T08:54:09.624 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.624 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.625 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:09.625 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488988+0000", 2024-05-08T08:54:09.625 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8759999999999999e-06 2024-05-08T08:54:09.625 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.625 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.625 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:09.625 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558992+0000", 2024-05-08T08:54:09.625 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070004358000000003 2024-05-08T08:54:09.625 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.625 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.625 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:09.625 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559093+0000", 2024-05-08T08:54:09.625 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000100299 2024-05-08T08:54:09.625 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.626 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.626 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:09.626 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559232+0000", 2024-05-08T08:54:09.626 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013913 2024-05-08T08:54:09.626 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:09.626 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:09.626 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:09.626 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.626 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.626 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1104 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11640832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:09.626 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:54:09.626 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 93.064265484000003, 2024-05-08T08:54:09.626 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070521947000000001, 2024-05-08T08:54:09.626 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:09.627 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:09.627 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:09.627 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:09.627 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:09.627 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:09.627 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1104 2024-05-08T08:54:09.627 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.627 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:09.627 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.627 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:09.627 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:54:09.627 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:09.627 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.627 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.628 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:09.628 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:54:09.628 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:09.628 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.628 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.628 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:09.628 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489700+0000", 2024-05-08T08:54:09.628 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00070067600000000001 2024-05-08T08:54:09.628 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.628 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.628 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:09.628 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489704+0000", 2024-05-08T08:54:09.628 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1640000000000003e-06 2024-05-08T08:54:09.628 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.628 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.629 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:09.629 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489716+0000", 2024-05-08T08:54:09.629 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1379000000000001e-05 2024-05-08T08:54:09.629 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.629 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.629 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:09.629 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489724+0000", 2024-05-08T08:54:09.629 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8949999999999998e-06 2024-05-08T08:54:09.629 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.629 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.629 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:09.629 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559278+0000", 2024-05-08T08:54:09.629 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.069554151999999994 2024-05-08T08:54:09.629 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.629 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.630 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:09.630 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559387+0000", 2024-05-08T08:54:09.630 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000109258 2024-05-08T08:54:09.630 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.630 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.630 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:09.630 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559521+0000", 2024-05-08T08:54:09.630 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000134423 2024-05-08T08:54:09.630 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:09.630 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:09.630 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:09.630 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.630 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.630 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:8636 3.17 3:e8a0a3e4:::rbd_header.11402b26385:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-05-08T08:54:09.631 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:54.150971+0000", 2024-05-08T08:54:09.631 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 75.402293490000005, 2024-05-08T08:54:09.631 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.217273233, 2024-05-08T08:54:09.631 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:09.631 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:09.631 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:09.631 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:09.631 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:54:09.631 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:54:09.631 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 8636 2024-05-08T08:54:09.631 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.631 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:09.631 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.631 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:09.631 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150971+0000", 2024-05-08T08:54:09.632 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:09.632 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.632 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.632 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:09.632 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150971+0000", 2024-05-08T08:54:09.632 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:09.632 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.632 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.632 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:09.632 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150972+0000", 2024-05-08T08:54:09.632 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.71e-07 2024-05-08T08:54:09.632 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.632 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.632 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:09.632 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150977+0000", 2024-05-08T08:54:09.633 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.152e-06 2024-05-08T08:54:09.633 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.633 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.633 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:09.633 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150990+0000", 2024-05-08T08:54:09.633 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2364999999999999e-05 2024-05-08T08:54:09.633 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.633 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.633 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:09.633 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150998+0000", 2024-05-08T08:54:09.633 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0739999999999993e-06 2024-05-08T08:54:09.633 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.633 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.633 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:09.633 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.151054+0000", 2024-05-08T08:54:09.634 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5949000000000003e-05 2024-05-08T08:54:09.634 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.634 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.634 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:54:09.634 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.151060+0000", 2024-05-08T08:54:09.634 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6120000000000004e-06 2024-05-08T08:54:09.634 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.634 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.634 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:09.634 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.367640+0000", 2024-05-08T08:54:09.634 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2165792739999999 2024-05-08T08:54:09.634 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.634 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.634 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:09.635 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.367839+0000", 2024-05-08T08:54:09.635 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000199185 2024-05-08T08:54:09.635 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.635 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.635 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:09.635 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.368245+0000", 2024-05-08T08:54:09.635 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00040575099999999998 2024-05-08T08:54:09.635 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:09.635 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:09.635 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:09.635 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.635 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.635 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5302.0:1516 3.c 3:319bf789:::rbd_data.1140d3fc54c.0000000000000025:head [stat,write 1892352~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:54:09.635 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:27.822220+0000", 2024-05-08T08:54:09.636 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 41.731044552999997, 2024-05-08T08:54:09.636 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.092643744, 2024-05-08T08:54:09.636 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:09.636 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:09.636 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:54:09.636 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:09.636 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5302", 2024-05-08T08:54:09.636 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/359277859", 2024-05-08T08:54:09.636 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1516 2024-05-08T08:54:09.636 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.636 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:09.636 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.636 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:09.636 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822220+0000", 2024-05-08T08:54:09.636 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:09.637 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.637 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.637 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:09.637 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822220+0000", 2024-05-08T08:54:09.637 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:09.637 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.637 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.637 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:09.637 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822221+0000", 2024-05-08T08:54:09.637 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3500000000000002e-07 2024-05-08T08:54:09.637 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.637 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.637 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:09.637 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822228+0000", 2024-05-08T08:54:09.637 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9959999999999996e-06 2024-05-08T08:54:09.638 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.638 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.638 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:09.638 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822234+0000", 2024-05-08T08:54:09.638 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5429999999999999e-06 2024-05-08T08:54:09.638 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.638 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.638 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:09.638 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822238+0000", 2024-05-08T08:54:09.638 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.732e-06 2024-05-08T08:54:09.638 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.638 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.638 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:09.638 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822285+0000", 2024-05-08T08:54:09.638 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6810000000000001e-05 2024-05-08T08:54:09.639 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.639 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.639 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:09.639 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822358+0000", 2024-05-08T08:54:09.639 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3473000000000006e-05 2024-05-08T08:54:09.639 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.639 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.639 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 5", 2024-05-08T08:54:09.639 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822489+0000", 2024-05-08T08:54:09.639 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013011199999999999 2024-05-08T08:54:09.639 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.639 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.639 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:54:09.639 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.825332+0000", 2024-05-08T08:54:09.639 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0028434799999999998 2024-05-08T08:54:09.639 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.640 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.640 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:54:09.640 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.914753+0000", 2024-05-08T08:54:09.640 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089421424999999999 2024-05-08T08:54:09.640 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.640 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.640 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:54:09.640 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.914807+0000", 2024-05-08T08:54:09.640 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3829000000000002e-05 2024-05-08T08:54:09.640 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.640 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.640 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:09.640 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.914864+0000", 2024-05-08T08:54:09.640 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6909e-05 2024-05-08T08:54:09.640 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:09.641 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:09.641 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:09.641 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.641 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.641 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:24567 3.6 3:66af1c40:::rbd_data.1140d3fc54c.000000000000005a:18 [read 2080768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:09.641 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.216598+0000", 2024-05-08T08:54:09.641 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 37.336666473000001, 2024-05-08T08:54:09.641 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071602647000000005, 2024-05-08T08:54:09.641 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:09.641 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:09.641 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:09.641 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:09.641 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:09.641 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:09.641 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 24567 2024-05-08T08:54:09.641 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.642 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:09.642 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.642 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:09.642 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216598+0000", 2024-05-08T08:54:09.642 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:09.642 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.642 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.642 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:09.642 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216598+0000", 2024-05-08T08:54:09.642 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:09.642 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.642 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.642 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:09.642 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216599+0000", 2024-05-08T08:54:09.643 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8999999999999997e-07 2024-05-08T08:54:09.643 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.643 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.643 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:09.643 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216601+0000", 2024-05-08T08:54:09.643 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9889999999999999e-06 2024-05-08T08:54:09.643 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.643 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.643 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:09.643 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216607+0000", 2024-05-08T08:54:09.643 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3600000000000001e-06 2024-05-08T08:54:09.643 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.643 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.643 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:09.643 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216612+0000", 2024-05-08T08:54:09.644 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.69e-06 2024-05-08T08:54:09.644 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.644 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.644 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:09.644 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.287076+0000", 2024-05-08T08:54:09.644 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070464355000000006 2024-05-08T08:54:09.644 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.644 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.644 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:09.644 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.287151+0000", 2024-05-08T08:54:09.644 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4924000000000003e-05 2024-05-08T08:54:09.644 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:09.644 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:09.644 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:09.645 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.288201+0000", 2024-05-08T08:54:09.645 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0010498389999999999 2024-05-08T08:54:09.645 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:09.645 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:09.645 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:09.645 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:09.645 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:09.645 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:09.645 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T08:54:09.648 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:54:09.648 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:09.649+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:09.648 INFO:teuthology.orchestra.run.smithi196.stdout:4892 read 0x66e1800 thru 0x66e4f32 (0x3733 bytes) 2024-05-08T08:54:09.649 INFO:teuthology.orchestra.run.smithi196.stdout:4893 trunc from 0x6a74200 to 0x1901a00 2024-05-08T08:54:09.649 INFO:teuthology.orchestra.run.smithi196.stdout:4894 write 0x45e5400 thru 0x45e5b8b (0x78c bytes) 2024-05-08T08:54:09.649 INFO:teuthology.orchestra.run.smithi196.stdout:4900 write 0x5208e00 thru 0x52172fc (0xe4fd bytes) 2024-05-08T08:54:09.649 INFO:teuthology.orchestra.run.smithi196.stdout:4904 write 0xb9a7400 thru 0xb9a793c (0x53d bytes) 2024-05-08T08:54:09.649 INFO:teuthology.orchestra.run.smithi196.stdout:4907 punch from 0x6b6a00 to 0x6ba800, (0x3e00 bytes) 2024-05-08T08:54:09.649 INFO:teuthology.orchestra.run.smithi196.stdout:4910 write 0xfdda00 thru 0xfe4e84 (0x7485 bytes) 2024-05-08T08:54:09.650 INFO:teuthology.orchestra.run.smithi196.stdout:4915 write 0x8a36e00 thru 0x8a40737 (0x9938 bytes) 2024-05-08T08:54:09.650 INFO:teuthology.orchestra.run.smithi196.stdout:4918 write 0x762f200 thru 0x763253f (0x3340 bytes) 2024-05-08T08:54:09.650 INFO:teuthology.orchestra.run.smithi196.stdout:4919 read 0x3904c00 thru 0x391477c (0xfb7d bytes) 2024-05-08T08:54:09.650 INFO:teuthology.orchestra.run.smithi196.stdout:4920 write 0x86a4400 thru 0x86b3108 (0xed09 bytes) 2024-05-08T08:54:09.681 INFO:teuthology.orchestra.run.smithi196.stdout:4921 write 0xbb60a00 thru 0xbb6b2e3 (0xa8e4 bytes) 2024-05-08T08:54:09.681 INFO:teuthology.orchestra.run.smithi196.stdout:4922 punch from 0x59f2a00 to 0x59f7000, (0x4600 bytes) 2024-05-08T08:54:09.688 INFO:teuthology.orchestra.run.smithi059.stderr:nodeep-scrub is unset 2024-05-08T08:54:09.695 INFO:teuthology.orchestra.run.smithi196.stdout:4927 trunc from 0xbb6b400 to 0x4cf6e00 2024-05-08T08:54:09.732 INFO:teuthology.orchestra.run.smithi196.stdout:4928 write 0x1010400 thru 0x1010a64 (0x665 bytes) 2024-05-08T08:54:09.738 INFO:teuthology.orchestra.run.smithi196.stdout:4929 read 0x41db000 thru 0x41e52c2 (0xa2c3 bytes) 2024-05-08T08:54:09.738 INFO:teuthology.orchestra.run.smithi196.stdout:4931 punch from 0x3ef0000 to 0x3efcc00, (0xcc00 bytes) 2024-05-08T08:54:09.744 INFO:teuthology.orchestra.run.smithi196.stdout:4932 trunc from 0x4cf6e00 to 0x96f5600 2024-05-08T08:54:09.748 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:09.751+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:09.760 INFO:teuthology.orchestra.run.smithi196.stdout:4933 read 0x88a200 thru 0x898339 (0xe13a bytes) 2024-05-08T08:54:09.768 INFO:teuthology.orchestra.run.smithi196.stdout:4934 read 0x4e4600 thru 0x4f0d3c (0xc73d bytes) 2024-05-08T08:54:09.779 INFO:teuthology.orchestra.run.smithi196.stdout:4935 write 0xa047e00 thru 0xa048af0 (0xcf1 bytes) 2024-05-08T08:54:09.795 INFO:teuthology.orchestra.run.smithi196.stdout:4936 trunc from 0xa048c00 to 0x1127800 2024-05-08T08:54:09.823 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:09.823 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:54:09.823 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:54:09.823 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:09.833 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:54:09.838 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:54:09.839 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:54:09.848 INFO:teuthology.orchestra.run.smithi196.stdout:4937 write 0xa664400 thru 0xa66f24a (0xae4b bytes) 2024-05-08T08:54:09.849 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:09.851+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:09.859 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:54:09.859 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:54:09.860 INFO:teuthology.orchestra.run.smithi196.stdout:4939 trunc from 0xa66f400 to 0xe81fa00 2024-05-08T08:54:09.881 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:54:09.881 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:54:09.882 INFO:teuthology.orchestra.run.smithi196.stdout:4940 read 0x73a2c00 thru 0x73a5a64 (0x2e65 bytes) 2024-05-08T08:54:09.901 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:54:09.901 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:54:09.901 INFO:teuthology.orchestra.run.smithi196.stdout:4941 trunc from 0xe81fa00 to 0x4cb7200 2024-05-08T08:54:09.928 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:54:09.928 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:54:09.930 INFO:teuthology.orchestra.run.smithi196.stdout:4943 trunc from 0x4cb7200 to 0xbb93a00 2024-05-08T08:54:09.940 INFO:teuthology.orchestra.run.smithi196.stdout:4947 read 0x722be00 thru 0x723182e (0x5a2f bytes) 2024-05-08T08:54:09.950 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:09.952+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:09.951 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:54:09.951 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:54:09.963 INFO:teuthology.orchestra.run.smithi196.stdout:4951 write 0xdc6de00 thru 0xdc7bdde (0xdfdf bytes) 2024-05-08T08:54:09.963 INFO:teuthology.orchestra.run.smithi196.stdout:4952 read 0x353be00 thru 0x354987d (0xda7e bytes) 2024-05-08T08:54:09.964 INFO:teuthology.orchestra.run.smithi196.stdout:4953 write 0x47bec00 thru 0x47ca938 (0xbd39 bytes) 2024-05-08T08:54:09.964 INFO:teuthology.orchestra.run.smithi196.stdout:4955 read 0xc55ec00 thru 0xc5637a2 (0x4ba3 bytes) 2024-05-08T08:54:09.964 INFO:teuthology.orchestra.run.smithi196.stdout:4956 punch from 0xa0d8a00 to 0xa0df600, (0x6c00 bytes) 2024-05-08T08:54:09.964 INFO:teuthology.orchestra.run.smithi196.stdout:4957 punch from 0xc702e00 to 0xc707200, (0x4400 bytes) 2024-05-08T08:54:09.964 INFO:teuthology.orchestra.run.smithi196.stdout:4958 write 0x259ca00 thru 0x25a57a9 (0x8daa bytes) 2024-05-08T08:54:09.965 INFO:teuthology.orchestra.run.smithi196.stdout:4963 trunc from 0xdc7be00 to 0xe5eea00 2024-05-08T08:54:09.967 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:09.967 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:09.979 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:54:10.016 INFO:teuthology.orchestra.run.smithi196.stdout:4965 trunc from 0xe5eea00 to 0x1f53200 2024-05-08T08:54:10.055 INFO:teuthology.orchestra.run.smithi196.stdout:4966 trunc from 0x1f53200 to 0xcac0200 2024-05-08T08:54:10.063 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:10.066+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:10.071 INFO:teuthology.orchestra.run.smithi196.stdout:4970 read 0x4764400 thru 0x4767af0 (0x36f1 bytes) 2024-05-08T08:54:10.084 INFO:teuthology.orchestra.run.smithi196.stdout:4974 trunc from 0xcac0200 to 0x1beb400 2024-05-08T08:54:10.121 INFO:teuthology.orchestra.run.smithi196.stdout:4975 write 0xaa26e00 thru 0xaa359e4 (0xebe5 bytes) 2024-05-08T08:54:10.127 INFO:teuthology.orchestra.run.smithi196.stdout:4977 write 0x2b4ae00 thru 0x2b5289e (0x7a9f bytes) 2024-05-08T08:54:10.143 INFO:teuthology.orchestra.run.smithi196.stdout:4978 write 0xb2cae00 thru 0xb2cf882 (0x4a83 bytes) 2024-05-08T08:54:10.148 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:10.148 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:10.151 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:10.154+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:10.157 INFO:teuthology.orchestra.run.smithi196.stdout:4979 trunc from 0xb2cfa00 to 0x5ae8400 2024-05-08T08:54:10.158 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:54:10.201 INFO:teuthology.orchestra.run.smithi196.stdout:4980 trunc from 0x5ae8400 to 0xbdc400 2024-05-08T08:54:10.226 INFO:teuthology.orchestra.run.smithi196.stdout:4982 punch from 0xa2a000 to 0xa38600, (0xe600 bytes) 2024-05-08T08:54:10.228 INFO:teuthology.orchestra.run.smithi196.stdout:4986 read 0x522400 thru 0x528730 (0x6331 bytes) 2024-05-08T08:54:10.229 INFO:teuthology.orchestra.run.smithi196.stdout:4989 read 0xb21a00 thru 0xb26b7d (0x517e bytes) 2024-05-08T08:54:10.249 INFO:teuthology.orchestra.run.smithi196.stdout:4990 write 0x24a9a00 thru 0x24b8cf7 (0xf2f8 bytes) 2024-05-08T08:54:10.253 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:10.255+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:10.279 INFO:teuthology.orchestra.run.smithi196.stdout:4992 write 0xaf15a00 thru 0xaf1ddf2 (0x83f3 bytes) 2024-05-08T08:54:10.299 INFO:teuthology.orchestra.run.smithi196.stdout:4993 write 0xcc72800 thru 0xcc80bba (0xe3bb bytes) 2024-05-08T08:54:10.300 INFO:teuthology.orchestra.run.smithi196.stdout:4995 read 0xacfe00 thru 0xadea53 (0xec54 bytes) 2024-05-08T08:54:10.301 INFO:teuthology.orchestra.run.smithi196.stdout:4998 trunc from 0xcc80c00 to 0xc746800 2024-05-08T08:54:10.324 INFO:teuthology.orchestra.run.smithi196.stdout:5002 read 0x29dfc00 thru 0x29e6ec8 (0x72c9 bytes) 2024-05-08T08:54:10.329 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:10.329 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:10.336 INFO:teuthology.orchestra.run.smithi196.stdout:5004 trunc from 0xc746800 to 0x24f9000 2024-05-08T08:54:10.339 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:54:10.354 INFO:teuthology.orchestra.run.smithi196.stdout:5006 read 0x1275000 thru 0x127a646 (0x5647 bytes) 2024-05-08T08:54:10.356 INFO:teuthology.orchestra.run.smithi196.stdout:5007 read 0x922800 thru 0x927d83 (0x5584 bytes) 2024-05-08T08:54:10.378 INFO:teuthology.orchestra.run.smithi196.stdout:5008 write 0xd734800 thru 0xd7385fb (0x3dfc bytes) 2024-05-08T08:54:10.379 INFO:teuthology.orchestra.run.smithi196.stdout:5009 write 0x9b11800 thru 0x9b1af45 (0x9746 bytes) 2024-05-08T08:54:10.379 INFO:teuthology.orchestra.run.smithi196.stdout:5010 write 0x2071400 thru 0x2080bc1 (0xf7c2 bytes) 2024-05-08T08:54:10.379 INFO:teuthology.orchestra.run.smithi196.stdout:5013 write 0xb98ec00 thru 0xb993372 (0x4773 bytes) 2024-05-08T08:54:10.379 INFO:teuthology.orchestra.run.smithi196.stdout:5014 read 0x204de00 thru 0x2057a3d (0x9c3e bytes) 2024-05-08T08:54:10.379 INFO:teuthology.orchestra.run.smithi196.stdout:5017 write 0xb4cc200 thru 0xb4d1cf1 (0x5af2 bytes) 2024-05-08T08:54:10.379 INFO:teuthology.orchestra.run.smithi196.stdout:5018 punch from 0x6bb2200 to 0x6bc0800, (0xe600 bytes) 2024-05-08T08:54:10.380 INFO:teuthology.orchestra.run.smithi196.stdout:5019 punch from 0xe35000 to 0xe39000, (0x4000 bytes) 2024-05-08T08:54:10.383 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:10.386+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:10.389 INFO:teuthology.orchestra.run.smithi196.stdout:5020 trunc from 0xd738600 to 0x58e3400 2024-05-08T08:54:10.439 INFO:teuthology.orchestra.run.smithi196.stdout:5021 punch from 0x4512c00 to 0x4517400, (0x4800 bytes) 2024-05-08T08:54:10.454 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:10.457+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:10.464 INFO:teuthology.orchestra.run.smithi196.stdout:5024 write 0xb5cba00 thru 0xb5d77f6 (0xbdf7 bytes) 2024-05-08T08:54:10.464 INFO:teuthology.orchestra.run.smithi196.stdout:5026 punch from 0x7871000 to 0x7874a00, (0x3a00 bytes) 2024-05-08T08:54:10.465 INFO:teuthology.orchestra.run.smithi196.stdout:5027 trunc from 0xb5d7800 to 0xa191000 2024-05-08T08:54:10.472 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:10.472 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:10.483 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:54:10.498 INFO:teuthology.orchestra.run.smithi196.stdout:5030 punch from 0x414fa00 to 0x4156000, (0x6600 bytes) 2024-05-08T08:54:10.508 INFO:teuthology.orchestra.run.smithi196.stdout:5032 trunc from 0xa191000 to 0x2056200 2024-05-08T08:54:10.541 INFO:teuthology.orchestra.run.smithi196.stdout:5033 read 0x1446c00 thru 0x144df5a (0x735b bytes) 2024-05-08T08:54:10.543 INFO:teuthology.orchestra.run.smithi196.stdout:5034 trunc from 0x2056200 to 0x2e45200 2024-05-08T08:54:10.556 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:10.559+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:10.578 INFO:teuthology.orchestra.run.smithi196.stdout:5035 write 0x5bd8000 thru 0x5bdf633 (0x7634 bytes) 2024-05-08T08:54:10.578 INFO:teuthology.orchestra.run.smithi196.stdout:5037 read 0x40ad000 thru 0x40bc7a8 (0xf7a9 bytes) 2024-05-08T08:54:10.581 INFO:teuthology.orchestra.run.smithi196.stdout:5042 trunc from 0x5bdf800 to 0x3146a00 2024-05-08T08:54:10.610 INFO:teuthology.orchestra.run.smithi196.stdout:5047 trunc from 0x3146a00 to 0x5ba9e00 2024-05-08T08:54:10.627 INFO:teuthology.orchestra.run.smithi196.stdout:5048 write 0x800600 thru 0x80e3ed (0xddee bytes) 2024-05-08T08:54:10.635 INFO:teuthology.orchestra.run.smithi196.stdout:5050 trunc from 0x5ba9e00 to 0xcfcfc00 2024-05-08T08:54:10.648 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:10.648 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:10.656 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:10.659+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:10.657 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:54:10.659 INFO:teuthology.orchestra.run.smithi196.stdout:5051 trunc from 0xcfcfc00 to 0xc6b0400 2024-05-08T08:54:10.669 INFO:teuthology.orchestra.run.smithi196.stdout:5052 read 0x6071200 thru 0x6072373 (0x1174 bytes) 2024-05-08T08:54:10.670 INFO:teuthology.orchestra.run.smithi196.stdout:5053 read 0xa82fa00 thru 0xa838d90 (0x9391 bytes) 2024-05-08T08:54:10.670 INFO:teuthology.orchestra.run.smithi196.stdout:5054 write 0x9b42800 thru 0x9b47726 (0x4f27 bytes) 2024-05-08T08:54:10.675 INFO:teuthology.orchestra.run.smithi196.stdout:5055 trunc from 0xc6b0400 to 0xc496800 2024-05-08T08:54:10.685 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4] out_osds: [5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-05-08T08:54:10.685 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T08:54:10.686 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T08:54:10.704 INFO:teuthology.orchestra.run.smithi196.stdout:5057 write 0xb91ac00 thru 0xb924976 (0x9d77 bytes) 2024-05-08T08:54:10.714 INFO:teuthology.orchestra.run.smithi196.stdout:5058 trunc from 0xc496800 to 0x369ec00 2024-05-08T08:54:10.757 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:10.760+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:10.771 INFO:teuthology.orchestra.run.smithi196.stdout:5061 write 0x5107000 thru 0x51117a8 (0xa7a9 bytes) 2024-05-08T08:54:10.771 INFO:teuthology.orchestra.run.smithi196.stdout:5063 read 0x31a6a00 thru 0x31a9c16 (0x3217 bytes) 2024-05-08T08:54:10.771 INFO:teuthology.orchestra.run.smithi196.stdout:5065 read 0x352e00 thru 0x3546ed (0x18ee bytes) 2024-05-08T08:54:10.771 INFO:teuthology.orchestra.run.smithi196.stdout:5066 write 0x2298a00 thru 0x22a6469 (0xda6a bytes) 2024-05-08T08:54:10.772 INFO:teuthology.orchestra.run.smithi196.stdout:5067 read 0x310ae00 thru 0x31173d5 (0xc5d6 bytes) 2024-05-08T08:54:10.772 INFO:teuthology.orchestra.run.smithi196.stdout:5068 trunc from 0x5111800 to 0x5da2600 2024-05-08T08:54:10.773 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:10.773 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:10.785 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:54:10.830 INFO:teuthology.orchestra.run.smithi196.stdout:5069 write 0xcb0be00 thru 0xcb0e6b1 (0x28b2 bytes) 2024-05-08T08:54:10.830 INFO:teuthology.orchestra.run.smithi196.stdout:5070 read 0x99b000 thru 0x9a3814 (0x8815 bytes) 2024-05-08T08:54:10.831 INFO:teuthology.orchestra.run.smithi196.stdout:5072 punch from 0x5ac7000 to 0x5ad0000, (0x9000 bytes) 2024-05-08T08:54:10.835 INFO:teuthology.orchestra.run.smithi196.stdout:5073 trunc from 0xcb0e800 to 0x955dc00 2024-05-08T08:54:10.858 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:10.861+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:10.865 INFO:teuthology.orchestra.run.smithi196.stdout:5075 write 0x6288600 thru 0x628d05a (0x4a5b bytes) 2024-05-08T08:54:10.866 INFO:teuthology.orchestra.run.smithi196.stdout:5076 punch from 0x4e69000 to 0x4e6e400, (0x5400 bytes) 2024-05-08T08:54:10.871 INFO:teuthology.orchestra.run.smithi196.stdout:5077 read 0x491b600 thru 0x4928867 (0xd268 bytes) 2024-05-08T08:54:10.896 INFO:teuthology.orchestra.run.smithi196.stdout:5079 write 0xec3e200 thru 0xec47d58 (0x9b59 bytes) 2024-05-08T08:54:10.896 INFO:teuthology.orchestra.run.smithi196.stdout:5082 punch from 0x3bd6a00 to 0x3bda600, (0x3c00 bytes) 2024-05-08T08:54:10.897 INFO:teuthology.orchestra.run.smithi196.stdout:5084 punch from 0x4969e00 to 0x496be00, (0x2000 bytes) 2024-05-08T08:54:10.897 INFO:teuthology.orchestra.run.smithi196.stdout:5085 write 0x3eae400 thru 0x3eae472 (0x73 bytes) 2024-05-08T08:54:10.897 INFO:teuthology.orchestra.run.smithi196.stdout:5086 read 0x7ed6e00 thru 0x7edc2f9 (0x54fa bytes) 2024-05-08T08:54:10.897 INFO:teuthology.orchestra.run.smithi196.stdout:5088 read 0x6f0c200 thru 0x6f1af72 (0xed73 bytes) 2024-05-08T08:54:10.897 INFO:teuthology.orchestra.run.smithi196.stdout:5089 write 0xad09600 thru 0xad1583c (0xc23d bytes) 2024-05-08T08:54:10.897 INFO:teuthology.orchestra.run.smithi196.stdout:5090 write 0x1f31e00 thru 0x1f3c065 (0xa266 bytes) 2024-05-08T08:54:10.897 INFO:teuthology.orchestra.run.smithi196.stdout:5091 trunc from 0xec47e00 to 0xe540e00 2024-05-08T08:54:10.899 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:10.899 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:10.910 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:54:10.959 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:10.962+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:10.961 INFO:teuthology.orchestra.run.smithi196.stdout:5093 trunc from 0xe540e00 to 0x2845200 2024-05-08T08:54:10.999 INFO:teuthology.orchestra.run.smithi196.stdout:5094 write 0x716800 thru 0x718ab6 (0x22b7 bytes) 2024-05-08T08:54:11.025 INFO:teuthology.orchestra.run.smithi196.stdout:5096 write 0x6c83000 thru 0x6c8b5e7 (0x85e8 bytes) 2024-05-08T08:54:11.025 INFO:teuthology.orchestra.run.smithi196.stdout:5097 read 0x5717000 thru 0x571d3dd (0x63de bytes) 2024-05-08T08:54:11.025 INFO:teuthology.orchestra.run.smithi196.stdout:5098 read 0x60aaa00 thru 0x60acc12 (0x2213 bytes) 2024-05-08T08:54:11.025 INFO:teuthology.orchestra.run.smithi196.stdout:5101 punch from 0x5dfa400 to 0x5e03e00, (0x9a00 bytes) 2024-05-08T08:54:11.026 INFO:teuthology.orchestra.run.smithi196.stdout:5102 write 0xda2200 thru 0xdab8c5 (0x96c6 bytes) 2024-05-08T08:54:11.026 INFO:teuthology.orchestra.run.smithi196.stdout:5103 read 0x5d30c00 thru 0x5d3ec37 (0xe038 bytes) 2024-05-08T08:54:11.026 INFO:teuthology.orchestra.run.smithi196.stdout:5106 write 0x2071e00 thru 0x2080682 (0xe883 bytes) 2024-05-08T08:54:11.040 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:54:11.040 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-05-08T08:54:11.042 INFO:teuthology.orchestra.run.smithi196.stdout:5107 punch from 0x5345600 to 0x5355200, (0xfc00 bytes) 2024-05-08T08:54:11.042 INFO:teuthology.orchestra.run.smithi196.stdout:5108 punch from 0x56f1200 to 0x56f7400, (0x6200 bytes) 2024-05-08T08:54:11.055 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":165,"stamp":"2024-05-08T08:54:09.762499+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1104981687,"num_objects":2779,"num_object_clones":430,"num_object_copies":5558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2779,"num_whiteouts":1619,"num_read":423929,"num_read_kb":1324947,"num_write":17966,"num_write_kb":410141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2942,"num_bytes_recovered":182360369,"num_keys_recovered":458,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16472,"ondisk_log_size":16472,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":471859200,"kb_used":344144,"kb_used_data":77204,"kb_used_omap":7,"kb_used_meta":266872,"kb_avail":471515056,"statfs":{"total":483183820800,"available":482831417344,"internally_reserved":0,"allocated":79056896,"data_stored":621724526,"data_compressed":31232996,"data_compressed_allocated":60354560,"data_compressed_original":611525120,"omap_allocated":7973,"internal_metadata":273277147},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":16759527,"num_objects":32,"num_object_clones":10,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":6676,"num_read_kb":33887,"num_write":217,"num_write_kb":9373,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.092544"},"pg_stats":[{"pgid":"3.1f","version":"98'555","reported_seq":18999,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-08T08:54:07.684797+0000","last_change":"2024-05-08T08:53:49.581255+0000","last_active":"2024-05-08T08:54:07.684797+0000","last_peered":"2024-05-08T08:54:07.684797+0000","last_clean":"2024-05-08T08:54:07.684797+0000","last_became_active":"2024-05-08T08:53:49.580563+0000","last_became_peered":"2024-05-08T08:53:49.580563+0000","last_unstale":"2024-05-08T08:54:07.684797+0000","last_undegraded":"2024-05-08T08:54:07.684797+0000","last_fullsized":"2024-05-08T08:54:07.684797+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":555,"log_dups_size":0,"ondisk_log_size":555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:27:21.049294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000691565,"stat_sum":{"num_bytes":61477223,"num_objects":87,"num_object_clones":17,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":44,"num_read":18294,"num_read_kb":77977,"num_write":799,"num_write_kb":32181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"98'477","reported_seq":6062,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-08T08:54:07.680534+0000","last_change":"2024-05-08T08:52:13.944947+0000","last_active":"2024-05-08T08:54:07.680534+0000","last_peered":"2024-05-08T08:54:07.680534+0000","last_clean":"2024-05-08T08:54:07.680534+0000","last_became_active":"2024-05-08T08:52:11.930844+0000","last_became_peered":"2024-05-08T08:52:11.930844+0000","last_unstale":"2024-05-08T08:54:07.680534+0000","last_undegraded":"2024-05-08T08:54:07.680534+0000","last_fullsized":"2024-05-08T08:54:07.680534+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":477,"log_dups_size":0,"ondisk_log_size":477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:53:39.466113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040424900000000003,"stat_sum":{"num_bytes":19188864,"num_objects":90,"num_object_clones":12,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":60,"num_read":6873,"num_read_kb":29927,"num_write":434,"num_write_kb":11813,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"103'504","reported_seq":2265,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-08T08:54:07.689459+0000","last_change":"2024-05-08T08:53:59.618675+0000","last_active":"2024-05-08T08:54:07.689459+0000","last_peered":"2024-05-08T08:54:07.689459+0000","last_clean":"2024-05-08T08:54:07.689459+0000","last_became_active":"2024-05-08T08:53:59.618085+0000","last_became_peered":"2024-05-08T08:53:59.618085+0000","last_unstale":"2024-05-08T08:54:07.689459+0000","last_undegraded":"2024-05-08T08:54:07.689459+0000","last_fullsized":"2024-05-08T08:54:07.689459+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":504,"log_dups_size":0,"ondisk_log_size":504,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:37:02.883846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072111799999999998,"stat_sum":{"num_bytes":13656334,"num_objects":78,"num_object_clones":11,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":44,"num_read":6100,"num_read_kb":21825,"num_write":699,"num_write_kb":6576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":141,"num_bytes_recovered":5594929,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"101'488","reported_seq":14377,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-08T08:54:07.749860+0000","last_change":"2024-05-08T08:53:58.622642+0000","last_active":"2024-05-08T08:54:07.749860+0000","last_peered":"2024-05-08T08:54:07.749860+0000","last_clean":"2024-05-08T08:54:07.749860+0000","last_became_active":"2024-05-08T08:53:58.621483+0000","last_became_peered":"2024-05-08T08:53:58.621483+0000","last_unstale":"2024-05-08T08:54:07.749860+0000","last_undegraded":"2024-05-08T08:54:07.749860+0000","last_fullsized":"2024-05-08T08:54:07.749860+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":488,"log_dups_size":0,"ondisk_log_size":488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:02:01.584307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094844199999999997,"stat_sum":{"num_bytes":54401589,"num_objects":90,"num_object_clones":11,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":53,"num_read":14349,"num_read_kb":75203,"num_write":539,"num_write_kb":25652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":287,"num_bytes_recovered":49590415,"num_keys_recovered":166,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"103'469","reported_seq":24225,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-08T08:54:07.710813+0000","last_change":"2024-05-08T08:53:49.585842+0000","last_active":"2024-05-08T08:54:07.710813+0000","last_peered":"2024-05-08T08:54:07.710813+0000","last_clean":"2024-05-08T08:54:07.710813+0000","last_became_active":"2024-05-08T08:53:49.583097+0000","last_became_peered":"2024-05-08T08:53:49.583097+0000","last_unstale":"2024-05-08T08:54:07.710813+0000","last_undegraded":"2024-05-08T08:54:07.710813+0000","last_fullsized":"2024-05-08T08:54:07.710813+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":469,"log_dups_size":0,"ondisk_log_size":469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:39:15.522869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000685079,"stat_sum":{"num_bytes":46999808,"num_objects":98,"num_object_clones":13,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":62,"num_read":23894,"num_read_kb":39296,"num_write":546,"num_write_kb":22541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":129,"num_bytes_recovered":3090520,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"101'566","reported_seq":17130,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-08T08:54:07.753867+0000","last_change":"2024-05-08T08:53:24.485198+0000","last_active":"2024-05-08T08:54:07.753867+0000","last_peered":"2024-05-08T08:54:07.753867+0000","last_clean":"2024-05-08T08:54:07.753867+0000","last_became_active":"2024-05-08T08:53:24.484124+0000","last_became_peered":"2024-05-08T08:53:24.484124+0000","last_unstale":"2024-05-08T08:54:07.753867+0000","last_undegraded":"2024-05-08T08:54:07.753867+0000","last_fullsized":"2024-05-08T08:54:07.753867+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":566,"log_dups_size":0,"ondisk_log_size":566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:24:56.148821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066525600000000003,"stat_sum":{"num_bytes":25621884,"num_objects":91,"num_object_clones":15,"num_object_copies":182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91,"num_whiteouts":47,"num_read":18714,"num_read_kb":51039,"num_write":697,"num_write_kb":18239,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":130,"num_bytes_recovered":8549042,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"101'315","reported_seq":9394,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-08T08:54:07.680682+0000","last_change":"2024-05-08T08:53:24.484713+0000","last_active":"2024-05-08T08:54:07.680682+0000","last_peered":"2024-05-08T08:54:07.680682+0000","last_clean":"2024-05-08T08:54:07.680682+0000","last_became_active":"2024-05-08T08:53:24.483403+0000","last_became_peered":"2024-05-08T08:53:24.483403+0000","last_unstale":"2024-05-08T08:54:07.680682+0000","last_undegraded":"2024-05-08T08:54:07.680682+0000","last_fullsized":"2024-05-08T08:54:07.680682+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":315,"log_dups_size":0,"ondisk_log_size":315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:06:04.800475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082481000000000004,"stat_sum":{"num_bytes":19215480,"num_objects":86,"num_object_clones":12,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":55,"num_read":8589,"num_read_kb":41299,"num_write":279,"num_write_kb":15800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":8814624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"101'428","reported_seq":16466,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-08T08:54:07.680836+0000","last_change":"2024-05-08T08:52:13.945184+0000","last_active":"2024-05-08T08:54:07.680836+0000","last_peered":"2024-05-08T08:54:07.680836+0000","last_clean":"2024-05-08T08:54:07.680836+0000","last_became_active":"2024-05-08T08:52:11.927366+0000","last_became_peered":"2024-05-08T08:52:11.927366+0000","last_unstale":"2024-05-08T08:54:07.680836+0000","last_undegraded":"2024-05-08T08:54:07.680836+0000","last_fullsized":"2024-05-08T08:54:07.680836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":428,"log_dups_size":0,"ondisk_log_size":428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:58:13.739129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057208900000000002,"stat_sum":{"num_bytes":79243480,"num_objects":99,"num_object_clones":19,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":57,"num_read":16558,"num_read_kb":42176,"num_write":477,"num_write_kb":22117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"101'678","reported_seq":15479,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-08T08:54:07.681451+0000","last_change":"2024-05-08T08:53:11.359646+0000","last_active":"2024-05-08T08:54:07.681451+0000","last_peered":"2024-05-08T08:54:07.681451+0000","last_clean":"2024-05-08T08:54:07.681451+0000","last_became_active":"2024-05-08T08:53:11.358391+0000","last_became_peered":"2024-05-08T08:53:11.358391+0000","last_unstale":"2024-05-08T08:54:07.681451+0000","last_undegraded":"2024-05-08T08:54:07.681451+0000","last_fullsized":"2024-05-08T08:54:07.681451+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":678,"log_dups_size":0,"ondisk_log_size":678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:59:28.634622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000866371,"stat_sum":{"num_bytes":55753874,"num_objects":76,"num_object_clones":12,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":39,"num_read":17383,"num_read_kb":67037,"num_write":819,"num_write_kb":15445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":97,"num_bytes_recovered":2508994,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"102'472","reported_seq":17511,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-08T08:54:07.684889+0000","last_change":"2024-05-08T08:53:59.617614+0000","last_active":"2024-05-08T08:54:07.684889+0000","last_peered":"2024-05-08T08:54:07.684889+0000","last_clean":"2024-05-08T08:54:07.684889+0000","last_became_active":"2024-05-08T08:53:59.615917+0000","last_became_peered":"2024-05-08T08:53:59.615917+0000","last_unstale":"2024-05-08T08:54:07.684889+0000","last_undegraded":"2024-05-08T08:54:07.684889+0000","last_fullsized":"2024-05-08T08:54:07.684889+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":472,"log_dups_size":0,"ondisk_log_size":472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:16:11.372622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084652399999999997,"stat_sum":{"num_bytes":37906141,"num_objects":76,"num_object_clones":11,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":39,"num_read":17947,"num_read_kb":69916,"num_write":588,"num_write_kb":23760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":118,"num_bytes_recovered":21349055,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"100'448","reported_seq":14840,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-08T08:54:07.747113+0000","last_change":"2024-05-08T08:52:13.944999+0000","last_active":"2024-05-08T08:54:07.747113+0000","last_peered":"2024-05-08T08:54:07.747113+0000","last_clean":"2024-05-08T08:54:07.747113+0000","last_became_active":"2024-05-08T08:52:11.931695+0000","last_became_peered":"2024-05-08T08:52:11.931695+0000","last_unstale":"2024-05-08T08:54:07.747113+0000","last_undegraded":"2024-05-08T08:54:07.747113+0000","last_fullsized":"2024-05-08T08:54:07.747113+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":448,"log_dups_size":0,"ondisk_log_size":448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:39:09.230921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063902800000000004,"stat_sum":{"num_bytes":24120414,"num_objects":80,"num_object_clones":11,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":49,"num_read":15626,"num_read_kb":40513,"num_write":495,"num_write_kb":6308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"100'548","reported_seq":11181,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-08T08:54:07.684805+0000","last_change":"2024-05-08T08:53:23.481964+0000","last_active":"2024-05-08T08:54:07.684805+0000","last_peered":"2024-05-08T08:54:07.684805+0000","last_clean":"2024-05-08T08:54:07.684805+0000","last_became_active":"2024-05-08T08:53:23.481362+0000","last_became_peered":"2024-05-08T08:53:23.481362+0000","last_unstale":"2024-05-08T08:54:07.684805+0000","last_undegraded":"2024-05-08T08:54:07.684805+0000","last_fullsized":"2024-05-08T08:54:07.684805+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":548,"log_dups_size":0,"ondisk_log_size":548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:28:00.765823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000430331,"stat_sum":{"num_bytes":34726003,"num_objects":80,"num_object_clones":16,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":43,"num_read":9629,"num_read_kb":26966,"num_write":520,"num_write_kb":10085,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":3612768,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":193,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-08T08:54:07.684920+0000","last_change":"2024-05-08T08:53:20.362387+0000","last_active":"2024-05-08T08:54:07.684920+0000","last_peered":"2024-05-08T08:54:07.684920+0000","last_clean":"2024-05-08T08:54:07.684920+0000","last_became_active":"2024-05-08T08:53:20.362113+0000","last_became_peered":"2024-05-08T08:53:20.362113+0000","last_unstale":"2024-05-08T08:54:07.684920+0000","last_undegraded":"2024-05-08T08:54:07.684920+0000","last_fullsized":"2024-05-08T08:54:07.684920+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:28:25.204125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"94'373","reported_seq":18297,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-08T08:54:07.764040+0000","last_change":"2024-05-08T08:53:49.580812+0000","last_active":"2024-05-08T08:54:07.764040+0000","last_peered":"2024-05-08T08:54:07.764040+0000","last_clean":"2024-05-08T08:54:07.764040+0000","last_became_active":"2024-05-08T08:53:49.579888+0000","last_became_peered":"2024-05-08T08:53:49.579888+0000","last_unstale":"2024-05-08T08:54:07.764040+0000","last_undegraded":"2024-05-08T08:54:07.764040+0000","last_fullsized":"2024-05-08T08:54:07.764040+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":373,"log_dups_size":0,"ondisk_log_size":373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:57:19.387824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088867899999999997,"stat_sum":{"num_bytes":40280281,"num_objects":79,"num_object_clones":9,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":47,"num_read":19746,"num_read_kb":80299,"num_write":352,"num_write_kb":15670,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":184,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-08T08:54:07.681396+0000","last_change":"2024-05-08T08:52:03.896159+0000","last_active":"2024-05-08T08:54:07.681396+0000","last_peered":"2024-05-08T08:54:07.681396+0000","last_clean":"2024-05-08T08:54:07.681396+0000","last_became_active":"2024-05-08T08:52:03.895797+0000","last_became_peered":"2024-05-08T08:52:03.895797+0000","last_unstale":"2024-05-08T08:54:07.681396+0000","last_undegraded":"2024-05-08T08:54:07.681396+0000","last_fullsized":"2024-05-08T08:54:07.681396+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:34:33.513525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"100'354","reported_seq":5683,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-08T08:54:07.680825+0000","last_change":"2024-05-08T08:52:13.944951+0000","last_active":"2024-05-08T08:54:07.680825+0000","last_peered":"2024-05-08T08:54:07.680825+0000","last_clean":"2024-05-08T08:54:07.680825+0000","last_became_active":"2024-05-08T08:52:11.926905+0000","last_became_peered":"2024-05-08T08:52:11.926905+0000","last_unstale":"2024-05-08T08:54:07.680825+0000","last_undegraded":"2024-05-08T08:54:07.680825+0000","last_fullsized":"2024-05-08T08:54:07.680825+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":354,"log_dups_size":0,"ondisk_log_size":354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:40:07.804343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044527,"stat_sum":{"num_bytes":13993121,"num_objects":90,"num_object_clones":17,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":50,"num_read":6042,"num_read_kb":29651,"num_write":295,"num_write_kb":10940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":183,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-08T08:54:07.684923+0000","last_change":"2024-05-08T08:52:04.144764+0000","last_active":"2024-05-08T08:54:07.684923+0000","last_peered":"2024-05-08T08:54:07.684923+0000","last_clean":"2024-05-08T08:54:07.684923+0000","last_became_active":"2024-05-08T08:52:04.144444+0000","last_became_peered":"2024-05-08T08:52:04.144444+0000","last_unstale":"2024-05-08T08:54:07.684923+0000","last_undegraded":"2024-05-08T08:54:07.684923+0000","last_fullsized":"2024-05-08T08:54:07.684923+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:58:31.498922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"99'576","reported_seq":24572,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-08T08:54:07.681258+0000","last_change":"2024-05-08T08:53:58.619407+0000","last_active":"2024-05-08T08:54:07.681258+0000","last_peered":"2024-05-08T08:54:07.681258+0000","last_clean":"2024-05-08T08:54:07.681258+0000","last_became_active":"2024-05-08T08:53:58.618388+0000","last_became_peered":"2024-05-08T08:53:58.618388+0000","last_unstale":"2024-05-08T08:54:07.681258+0000","last_undegraded":"2024-05-08T08:54:07.681258+0000","last_fullsized":"2024-05-08T08:54:07.681258+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":576,"log_dups_size":0,"ondisk_log_size":576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:46:57.991543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000669351,"stat_sum":{"num_bytes":46779392,"num_objects":109,"num_object_clones":19,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":62,"num_read":20877,"num_read_kb":45306,"num_write":398,"num_write_kb":10714,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":215,"num_bytes_recovered":10925568,"num_keys_recovered":44,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":184,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-08T08:54:07.681260+0000","last_change":"2024-05-08T08:52:04.144543+0000","last_active":"2024-05-08T08:54:07.681260+0000","last_peered":"2024-05-08T08:54:07.681260+0000","last_clean":"2024-05-08T08:54:07.681260+0000","last_became_active":"2024-05-08T08:52:04.144242+0000","last_became_peered":"2024-05-08T08:52:04.144242+0000","last_unstale":"2024-05-08T08:54:07.681260+0000","last_undegraded":"2024-05-08T08:54:07.681260+0000","last_fullsized":"2024-05-08T08:54:07.681260+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:57:36.844633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"101'778","reported_seq":12074,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-08T08:54:07.681004+0000","last_change":"2024-05-08T08:53:58.616694+0000","last_active":"2024-05-08T08:54:07.681004+0000","last_peered":"2024-05-08T08:54:07.681004+0000","last_clean":"2024-05-08T08:54:07.681004+0000","last_became_active":"2024-05-08T08:53:58.615614+0000","last_became_peered":"2024-05-08T08:53:58.615614+0000","last_unstale":"2024-05-08T08:54:07.681004+0000","last_undegraded":"2024-05-08T08:54:07.681004+0000","last_fullsized":"2024-05-08T08:54:07.681004+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":778,"log_dups_size":0,"ondisk_log_size":778,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:29:05.148859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064078099999999999,"stat_sum":{"num_bytes":16686331,"num_objects":102,"num_object_clones":15,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":66,"num_read":13662,"num_read_kb":29310,"num_write":1006,"num_write_kb":9257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":142,"num_bytes_recovered":4609275,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"101'447","reported_seq":8586,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-08T08:54:07.719738+0000","last_change":"2024-05-08T08:53:24.481238+0000","last_active":"2024-05-08T08:54:07.719738+0000","last_peered":"2024-05-08T08:54:07.719738+0000","last_clean":"2024-05-08T08:54:07.719738+0000","last_became_active":"2024-05-08T08:53:24.480423+0000","last_became_peered":"2024-05-08T08:53:24.480423+0000","last_unstale":"2024-05-08T08:54:07.719738+0000","last_undegraded":"2024-05-08T08:54:07.719738+0000","last_fullsized":"2024-05-08T08:54:07.719738+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:05:58.712373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047037799999999997,"stat_sum":{"num_bytes":29133134,"num_objects":74,"num_object_clones":13,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":43,"num_read":9058,"num_read_kb":27843,"num_write":521,"num_write_kb":5251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":101,"num_bytes_recovered":5689443,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":164,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-08T08:54:07.680780+0000","last_change":"2024-05-08T08:53:55.365435+0000","last_active":"2024-05-08T08:54:07.680780+0000","last_peered":"2024-05-08T08:54:07.680780+0000","last_clean":"2024-05-08T08:54:07.680780+0000","last_became_active":"2024-05-08T08:53:55.365059+0000","last_became_peered":"2024-05-08T08:53:55.365059+0000","last_unstale":"2024-05-08T08:54:07.680780+0000","last_undegraded":"2024-05-08T08:54:07.680780+0000","last_fullsized":"2024-05-08T08:54:07.680780+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:15:39.278668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"101'426","reported_seq":8459,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-08T08:54:07.681126+0000","last_change":"2024-05-08T08:52:13.940504+0000","last_active":"2024-05-08T08:54:07.681126+0000","last_peered":"2024-05-08T08:54:07.681126+0000","last_clean":"2024-05-08T08:54:07.681126+0000","last_became_active":"2024-05-08T08:52:11.928301+0000","last_became_peered":"2024-05-08T08:52:11.928301+0000","last_unstale":"2024-05-08T08:54:07.681126+0000","last_undegraded":"2024-05-08T08:54:07.681126+0000","last_fullsized":"2024-05-08T08:54:07.681126+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":426,"log_dups_size":0,"ondisk_log_size":426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:41:04.242411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046631800000000002,"stat_sum":{"num_bytes":44329575,"num_objects":88,"num_object_clones":16,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":50,"num_read":5119,"num_read_kb":21014,"num_write":441,"num_write_kb":13418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":185,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-08T08:54:07.680537+0000","last_change":"2024-05-08T08:52:03.891768+0000","last_active":"2024-05-08T08:54:07.680537+0000","last_peered":"2024-05-08T08:54:07.680537+0000","last_clean":"2024-05-08T08:54:07.680537+0000","last_became_active":"2024-05-08T08:52:03.891374+0000","last_became_peered":"2024-05-08T08:52:03.891374+0000","last_unstale":"2024-05-08T08:54:07.680537+0000","last_undegraded":"2024-05-08T08:54:07.680537+0000","last_fullsized":"2024-05-08T08:54:07.680537+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:18:10.240492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"101'772","reported_seq":23543,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-08T08:54:07.740555+0000","last_change":"2024-05-08T08:53:49.582654+0000","last_active":"2024-05-08T08:54:07.740555+0000","last_peered":"2024-05-08T08:54:07.740555+0000","last_clean":"2024-05-08T08:54:07.740555+0000","last_became_active":"2024-05-08T08:53:49.581797+0000","last_became_peered":"2024-05-08T08:53:49.581797+0000","last_unstale":"2024-05-08T08:54:07.740555+0000","last_undegraded":"2024-05-08T08:54:07.740555+0000","last_fullsized":"2024-05-08T08:54:07.740555+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":772,"log_dups_size":0,"ondisk_log_size":772,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:58:33.518509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046440499999999998,"stat_sum":{"num_bytes":44461900,"num_objects":101,"num_object_clones":16,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":54,"num_read":26123,"num_read_kb":40722,"num_write":497,"num_write_kb":8416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":135,"num_bytes_recovered":3549534,"num_keys_recovered":46,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":184,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-08T08:54:07.684579+0000","last_change":"2024-05-08T08:52:03.891551+0000","last_active":"2024-05-08T08:54:07.684579+0000","last_peered":"2024-05-08T08:54:07.684579+0000","last_clean":"2024-05-08T08:54:07.684579+0000","last_became_active":"2024-05-08T08:52:03.891296+0000","last_became_peered":"2024-05-08T08:52:03.891296+0000","last_unstale":"2024-05-08T08:54:07.684579+0000","last_undegraded":"2024-05-08T08:54:07.684579+0000","last_fullsized":"2024-05-08T08:54:07.684579+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:54.806343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"100'698","reported_seq":10140,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-08T08:54:07.680707+0000","last_change":"2024-05-08T08:53:58.616835+0000","last_active":"2024-05-08T08:54:07.680707+0000","last_peered":"2024-05-08T08:54:07.680707+0000","last_clean":"2024-05-08T08:54:07.680707+0000","last_became_active":"2024-05-08T08:53:58.616032+0000","last_became_peered":"2024-05-08T08:53:58.616032+0000","last_unstale":"2024-05-08T08:54:07.680707+0000","last_undegraded":"2024-05-08T08:54:07.680707+0000","last_fullsized":"2024-05-08T08:54:07.680707+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":698,"log_dups_size":0,"ondisk_log_size":698,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:00:06.621575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062135599999999999,"stat_sum":{"num_bytes":39801281,"num_objects":101,"num_object_clones":18,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":57,"num_read":11284,"num_read_kb":33491,"num_write":977,"num_write_kb":11001,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":125,"num_bytes_recovered":4950942,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":164,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-08T08:54:07.680667+0000","last_change":"2024-05-08T08:53:55.362409+0000","last_active":"2024-05-08T08:54:07.680667+0000","last_peered":"2024-05-08T08:54:07.680667+0000","last_clean":"2024-05-08T08:54:07.680667+0000","last_became_active":"2024-05-08T08:53:55.362055+0000","last_became_peered":"2024-05-08T08:53:55.362055+0000","last_unstale":"2024-05-08T08:54:07.680667+0000","last_undegraded":"2024-05-08T08:54:07.680667+0000","last_fullsized":"2024-05-08T08:54:07.680667+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:34:23.719119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":340,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-08T08:54:07.684530+0000","last_change":"2024-05-08T08:51:58.861899+0000","last_active":"2024-05-08T08:54:07.684530+0000","last_peered":"2024-05-08T08:54:07.684530+0000","last_clean":"2024-05-08T08:54:07.684530+0000","last_became_active":"2024-05-08T08:51:58.861596+0000","last_became_peered":"2024-05-08T08:51:58.861596+0000","last_unstale":"2024-05-08T08:54:07.684530+0000","last_undegraded":"2024-05-08T08:54:07.684530+0000","last_fullsized":"2024-05-08T08:54:07.684530+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:51:57.849666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:51:57.849666+0000","last_clean_scrub_stamp":"2024-05-08T08:51:57.849666+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:53:21.739950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"100'257","reported_seq":7382,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-08T08:54:07.680995+0000","last_change":"2024-05-08T08:52:13.945362+0000","last_active":"2024-05-08T08:54:07.680995+0000","last_peered":"2024-05-08T08:54:07.680995+0000","last_clean":"2024-05-08T08:54:07.680995+0000","last_became_active":"2024-05-08T08:52:11.926975+0000","last_became_peered":"2024-05-08T08:52:11.926975+0000","last_unstale":"2024-05-08T08:54:07.680995+0000","last_undegraded":"2024-05-08T08:54:07.680995+0000","last_fullsized":"2024-05-08T08:54:07.680995+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":257,"log_dups_size":0,"ondisk_log_size":257,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:54:22.517027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068218599999999999,"stat_sum":{"num_bytes":29473888,"num_objects":82,"num_object_clones":15,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":48,"num_read":7010,"num_read_kb":18435,"num_write":221,"num_write_kb":6932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"99'357","reported_seq":13508,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-08T08:54:07.685819+0000","last_change":"2024-05-08T08:53:23.479481+0000","last_active":"2024-05-08T08:54:07.685819+0000","last_peered":"2024-05-08T08:54:07.685819+0000","last_clean":"2024-05-08T08:54:07.685819+0000","last_became_active":"2024-05-08T08:53:23.478564+0000","last_became_peered":"2024-05-08T08:53:23.478564+0000","last_unstale":"2024-05-08T08:54:07.685819+0000","last_undegraded":"2024-05-08T08:54:07.685819+0000","last_fullsized":"2024-05-08T08:54:07.685819+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":357,"log_dups_size":0,"ondisk_log_size":357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:24:43.863288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00030754000000000003,"stat_sum":{"num_bytes":34703919,"num_objects":87,"num_object_clones":11,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":50,"num_read":13623,"num_read_kb":59857,"num_write":299,"num_write_kb":16239,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":131,"num_bytes_recovered":12377088,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"101'1022","reported_seq":10532,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-08T08:54:07.684714+0000","last_change":"2024-05-08T08:53:24.481272+0000","last_active":"2024-05-08T08:54:07.684714+0000","last_peered":"2024-05-08T08:54:07.684714+0000","last_clean":"2024-05-08T08:54:07.684714+0000","last_became_active":"2024-05-08T08:53:24.480359+0000","last_became_peered":"2024-05-08T08:53:24.480359+0000","last_unstale":"2024-05-08T08:54:07.684714+0000","last_undegraded":"2024-05-08T08:54:07.684714+0000","last_fullsized":"2024-05-08T08:54:07.684714+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":1022,"log_dups_size":0,"ondisk_log_size":1022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:45:47.843488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070350100000000004,"stat_sum":{"num_bytes":20702063,"num_objects":80,"num_object_clones":12,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":50,"num_read":14827,"num_read_kb":26766,"num_write":1254,"num_write_kb":5325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":2175343,"num_keys_recovered":61,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"102'468","reported_seq":9828,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-08T08:54:07.681175+0000","last_change":"2024-05-08T08:52:13.940600+0000","last_active":"2024-05-08T08:54:07.681175+0000","last_peered":"2024-05-08T08:54:07.681175+0000","last_clean":"2024-05-08T08:54:07.681175+0000","last_became_active":"2024-05-08T08:52:11.927196+0000","last_became_peered":"2024-05-08T08:52:11.927196+0000","last_unstale":"2024-05-08T08:54:07.681175+0000","last_undegraded":"2024-05-08T08:54:07.681175+0000","last_fullsized":"2024-05-08T08:54:07.681175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":468,"log_dups_size":0,"ondisk_log_size":468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:56:25.159328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048296599999999998,"stat_sum":{"num_bytes":37807717,"num_objects":91,"num_object_clones":11,"num_object_copies":182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91,"num_whiteouts":56,"num_read":10295,"num_read_kb":33366,"num_write":395,"num_write_kb":7236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"98'248","reported_seq":12793,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-08T08:54:07.684727+0000","last_change":"2024-05-08T08:53:58.616502+0000","last_active":"2024-05-08T08:54:07.684727+0000","last_peered":"2024-05-08T08:54:07.684727+0000","last_clean":"2024-05-08T08:54:07.684727+0000","last_became_active":"2024-05-08T08:53:58.615095+0000","last_became_peered":"2024-05-08T08:53:58.615095+0000","last_unstale":"2024-05-08T08:54:07.684727+0000","last_undegraded":"2024-05-08T08:54:07.684727+0000","last_fullsized":"2024-05-08T08:54:07.684727+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":248,"log_dups_size":0,"ondisk_log_size":248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:03:46.740095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048793000000000003,"stat_sum":{"num_bytes":14430907,"num_objects":80,"num_object_clones":10,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":49,"num_read":12459,"num_read_kb":35553,"num_write":210,"num_write_kb":6495,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":107,"num_bytes_recovered":2139323,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"98'365","reported_seq":6449,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-08T08:54:07.684856+0000","last_change":"2024-05-08T08:52:13.945141+0000","last_active":"2024-05-08T08:54:07.684856+0000","last_peered":"2024-05-08T08:54:07.684856+0000","last_clean":"2024-05-08T08:54:07.684856+0000","last_became_active":"2024-05-08T08:52:11.932594+0000","last_became_peered":"2024-05-08T08:52:11.932594+0000","last_unstale":"2024-05-08T08:54:07.684856+0000","last_undegraded":"2024-05-08T08:54:07.684856+0000","last_fullsized":"2024-05-08T08:54:07.684856+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":365,"log_dups_size":0,"ondisk_log_size":365,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:55:44.906413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074388199999999996,"stat_sum":{"num_bytes":16919093,"num_objects":78,"num_object_clones":9,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":51,"num_read":7186,"num_read_kb":31175,"num_write":315,"num_write_kb":9263,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"98'426","reported_seq":4988,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-08T08:54:07.754582+0000","last_change":"2024-05-08T08:53:58.622564+0000","last_active":"2024-05-08T08:54:07.754582+0000","last_peered":"2024-05-08T08:54:07.754582+0000","last_clean":"2024-05-08T08:54:07.754582+0000","last_became_active":"2024-05-08T08:53:58.621584+0000","last_became_peered":"2024-05-08T08:53:58.621584+0000","last_unstale":"2024-05-08T08:54:07.754582+0000","last_undegraded":"2024-05-08T08:54:07.754582+0000","last_fullsized":"2024-05-08T08:54:07.754582+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":426,"log_dups_size":0,"ondisk_log_size":426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:15:42.459412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085079600000000002,"stat_sum":{"num_bytes":21652544,"num_objects":60,"num_object_clones":12,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":34,"num_read":11229,"num_read_kb":39584,"num_write":485,"num_write_kb":11139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":188,"num_bytes_recovered":11952192,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"98'366","reported_seq":6912,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-08T08:54:07.684947+0000","last_change":"2024-05-08T08:53:58.616115+0000","last_active":"2024-05-08T08:54:07.684947+0000","last_peered":"2024-05-08T08:54:07.684947+0000","last_clean":"2024-05-08T08:54:07.684947+0000","last_became_active":"2024-05-08T08:53:58.615064+0000","last_became_peered":"2024-05-08T08:53:58.615064+0000","last_unstale":"2024-05-08T08:54:07.684947+0000","last_undegraded":"2024-05-08T08:54:07.684947+0000","last_fullsized":"2024-05-08T08:54:07.684947+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":366,"log_dups_size":0,"ondisk_log_size":366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:02:48.851489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010326370000000001,"stat_sum":{"num_bytes":24051938,"num_objects":73,"num_object_clones":8,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":43,"num_read":7748,"num_read_kb":33777,"num_write":347,"num_write_kb":9075,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":107,"num_bytes_recovered":4068050,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"100'308","reported_seq":7391,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-08T08:54:07.680945+0000","last_change":"2024-05-08T08:53:58.619593+0000","last_active":"2024-05-08T08:54:07.680945+0000","last_peered":"2024-05-08T08:54:07.680945+0000","last_clean":"2024-05-08T08:54:07.680945+0000","last_became_active":"2024-05-08T08:53:58.618787+0000","last_became_peered":"2024-05-08T08:53:58.618787+0000","last_unstale":"2024-05-08T08:54:07.680945+0000","last_undegraded":"2024-05-08T08:54:07.680945+0000","last_fullsized":"2024-05-08T08:54:07.680945+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":308,"log_dups_size":0,"ondisk_log_size":308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:31:59.632233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053593,"stat_sum":{"num_bytes":32025169,"num_objects":81,"num_object_clones":13,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":51,"num_read":7199,"num_read_kb":38631,"num_write":298,"num_write_kb":13563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":112,"num_bytes_recovered":7552081,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"101'490","reported_seq":9474,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-08T08:54:07.684762+0000","last_change":"2024-05-08T08:53:24.480561+0000","last_active":"2024-05-08T08:54:07.684762+0000","last_peered":"2024-05-08T08:54:07.684762+0000","last_clean":"2024-05-08T08:54:07.684762+0000","last_became_active":"2024-05-08T08:53:24.479891+0000","last_became_peered":"2024-05-08T08:53:24.479891+0000","last_unstale":"2024-05-08T08:54:07.684762+0000","last_undegraded":"2024-05-08T08:54:07.684762+0000","last_fullsized":"2024-05-08T08:54:07.684762+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":490,"log_dups_size":0,"ondisk_log_size":490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:57:48.379726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041230699999999999,"stat_sum":{"num_bytes":31224357,"num_objects":91,"num_object_clones":16,"num_object_copies":182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91,"num_whiteouts":57,"num_read":10656,"num_read_kb":29292,"num_write":402,"num_write_kb":8912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":131,"num_bytes_recovered":2604032,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"103'1262","reported_seq":21096,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-08T08:54:07.688052+0000","last_change":"2024-05-08T08:53:49.581481+0000","last_active":"2024-05-08T08:54:07.688052+0000","last_peered":"2024-05-08T08:54:07.688052+0000","last_clean":"2024-05-08T08:54:07.688052+0000","last_became_active":"2024-05-08T08:53:49.580557+0000","last_became_peered":"2024-05-08T08:53:49.580557+0000","last_unstale":"2024-05-08T08:54:07.688052+0000","last_undegraded":"2024-05-08T08:54:07.688052+0000","last_fullsized":"2024-05-08T08:54:07.688052+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":1262,"log_dups_size":0,"ondisk_log_size":1262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:53:51.176889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059220300000000002,"stat_sum":{"num_bytes":41633984,"num_objects":99,"num_object_clones":10,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":56,"num_read":26568,"num_read_kb":48525,"num_write":1809,"num_write_kb":9553,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"102'414","reported_seq":9220,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-08T08:54:07.727277+0000","last_change":"2024-05-08T08:53:58.616558+0000","last_active":"2024-05-08T08:54:07.727277+0000","last_peered":"2024-05-08T08:54:07.727277+0000","last_clean":"2024-05-08T08:54:07.727277+0000","last_became_active":"2024-05-08T08:53:58.615947+0000","last_became_peered":"2024-05-08T08:53:58.615947+0000","last_unstale":"2024-05-08T08:54:07.727277+0000","last_undegraded":"2024-05-08T08:54:07.727277+0000","last_fullsized":"2024-05-08T08:54:07.727277+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":414,"log_dups_size":0,"ondisk_log_size":414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:11:03.092284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059972400000000005,"stat_sum":{"num_bytes":51989631,"num_objects":100,"num_object_clones":20,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":53,"num_read":9146,"num_read_kb":39076,"num_write":393,"num_write_kb":8809,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":130,"num_bytes_recovered":6657151,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1104391319,"num_objects":2777,"num_object_clones":430,"num_object_copies":5554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2777,"num_whiteouts":1619,"num_read":423813,"num_read_kb":1324847,"num_write":17807,"num_write_kb":407725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2942,"num_bytes_recovered":182360369,"num_keys_recovered":458,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":84680704,"data_stored":717492801,"data_compressed":36250577,"data_compressed_allocated":70287360,"data_compressed_original":708741632,"omap_allocated":0,"internal_metadata":0},"log_size":16355,"ondisk_log_size":16355,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":11046,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":0,"seq":47244640284,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640287,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48924,"kb_used_data":13584,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94322916,"statfs":{"total":96636764160,"available":96586665984,"internally_reserved":0,"allocated":13910016,"data_stored":112111693,"data_compressed":5939331,"data_compressed_allocated":11169792,"data_compressed_original":110738944,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640286,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":93132,"kb_used_data":17536,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94278708,"statfs":{"total":96636764160,"available":96541396992,"internally_reserved":0,"allocated":17956864,"data_stored":134302789,"data_compressed":6909895,"data_compressed_allocated":12992512,"data_compressed_original":131399168,"omap_allocated":1593,"internal_metadata":77396423},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640285,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38280,"kb_used_data":11256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94333560,"statfs":{"total":96636764160,"available":96597565440,"internally_reserved":0,"allocated":11526144,"data_stored":85023184,"data_compressed":4931880,"data_compressed_allocated":8761344,"data_compressed_original":83598336,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640284,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72252,"kb_used_data":18864,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94299588,"statfs":{"total":96636764160,"available":96562778112,"internally_reserved":0,"allocated":19316736,"data_stored":172249252,"data_compressed":7644479,"data_compressed_allocated":15917056,"data_compressed_original":170486784,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640287,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":91556,"kb_used_data":15964,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94280284,"statfs":{"total":96636764160,"available":96543010816,"internally_reserved":0,"allocated":16347136,"data_stored":118037608,"data_compressed":5807411,"data_compressed_allocated":11513856,"data_compressed_original":115301888,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5523,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5523,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":15106048,"data_stored":117006517,"data_compressed":5801969,"data_compressed_allocated":11481088,"data_compressed_original":114777600,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":18132992,"data_stored":171815838,"data_compressed":7648833,"data_compressed_allocated":15929344,"data_compressed_original":170548224,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":10321920,"data_stored":84527262,"data_compressed":4931880,"data_compressed_allocated":8761344,"data_compressed_original":83598336,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16703488,"data_stored":133228790,"data_compressed":6908645,"data_compressed_allocated":12963840,"data_compressed_original":130838016,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12709888,"data_stored":111615815,"data_compressed":5939331,"data_compressed_allocated":11169792,"data_compressed_original":110738944,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":11706368,"data_stored":99298579,"data_compressed":5019919,"data_compressed_allocated":9981952,"data_compressed_original":98240512,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T08:54:11.060 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:11.063+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:11.074 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:11.074 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:11.076 INFO:teuthology.orchestra.run.smithi196.stdout:5109 write 0xc263c00 thru 0xc26d61e (0x9a1f bytes) 2024-05-08T08:54:11.077 INFO:teuthology.orchestra.run.smithi196.stdout:5110 write 0xb9d7000 thru 0xb9ddf76 (0x6f77 bytes) 2024-05-08T08:54:11.077 INFO:teuthology.orchestra.run.smithi196.stdout:5112 write 0x5291c00 thru 0x529674f (0x4b50 bytes) 2024-05-08T08:54:11.078 INFO:teuthology.orchestra.run.smithi196.stdout:5113 read 0x12d8c00 thru 0x12d9154 (0x555 bytes) 2024-05-08T08:54:11.078 INFO:teuthology.orchestra.run.smithi196.stdout:5116 read 0x3837000 thru 0x3844bdd (0xdbde bytes) 2024-05-08T08:54:11.078 INFO:teuthology.orchestra.run.smithi196.stdout:5118 punch from 0x2662600 to 0x2666400, (0x3e00 bytes) 2024-05-08T08:54:11.078 INFO:teuthology.orchestra.run.smithi196.stdout:5122 write 0xa0fc00 thru 0xa17f22 (0x8323 bytes) 2024-05-08T08:54:11.081 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:54:11.088 INFO:teuthology.orchestra.run.smithi196.stdout:5123 read 0xa345e00 thru 0xa3553a0 (0xf5a1 bytes) 2024-05-08T08:54:11.088 INFO:teuthology.orchestra.run.smithi196.stdout:5125 read 0x5c3c200 thru 0x5c44e1b (0x8c1c bytes) 2024-05-08T08:54:11.089 INFO:teuthology.orchestra.run.smithi196.stdout:5127 write 0x4936000 thru 0x493b393 (0x5394 bytes) 2024-05-08T08:54:11.089 INFO:teuthology.orchestra.run.smithi196.stdout:5128 read 0x42daa00 thru 0x42e0bd8 (0x61d9 bytes) 2024-05-08T08:54:11.091 INFO:teuthology.orchestra.run.smithi196.stdout:5129 trunc from 0xc26d800 to 0xe2fde00 2024-05-08T08:54:11.117 INFO:teuthology.orchestra.run.smithi196.stdout:5131 punch from 0xe092c00 to 0xe093a00, (0xe00 bytes) 2024-05-08T08:54:11.118 INFO:teuthology.orchestra.run.smithi196.stdout:5132 read 0x996fe00 thru 0x997e93e (0xeb3f bytes) 2024-05-08T08:54:11.118 INFO:teuthology.orchestra.run.smithi196.stdout:5133 read 0x9809600 thru 0x980f775 (0x6176 bytes) 2024-05-08T08:54:11.118 INFO:teuthology.orchestra.run.smithi196.stdout:5134 write 0x5b0a200 thru 0x5b0b08b (0xe8c bytes) 2024-05-08T08:54:11.118 INFO:teuthology.orchestra.run.smithi196.stdout:5136 punch from 0x5fed600 to 0x5ff2800, (0x5200 bytes) 2024-05-08T08:54:11.119 INFO:teuthology.orchestra.run.smithi196.stdout:5137 read 0x70bb400 thru 0x70bdab4 (0x26b5 bytes) 2024-05-08T08:54:11.119 INFO:teuthology.orchestra.run.smithi196.stdout:5138 write 0x697d400 thru 0x6982175 (0x4d76 bytes) 2024-05-08T08:54:11.119 INFO:teuthology.orchestra.run.smithi196.stdout:5139 write 0xc390e00 thru 0xc39227c (0x147d bytes) 2024-05-08T08:54:11.126 INFO:teuthology.orchestra.run.smithi196.stdout:5141 trunc from 0xe2fde00 to 0x898aa00 2024-05-08T08:54:11.156 INFO:teuthology.orchestra.run.smithi196.stdout:5142 punch from 0x8336a00 to 0x833ac00, (0x4200 bytes) 2024-05-08T08:54:11.166 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:11.169+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:11.173 INFO:teuthology.orchestra.run.smithi196.stdout:5144 write 0xdb8ce00 thru 0xdb90c9e (0x3e9f bytes) 2024-05-08T08:54:11.181 INFO:teuthology.orchestra.run.smithi196.stdout:5147 write 0xacf8400 thru 0xad05a0c (0xd60d bytes) 2024-05-08T08:54:11.181 INFO:teuthology.orchestra.run.smithi196.stdout:5149 punch from 0x9fbba00 to 0x9fc1200, (0x5800 bytes) 2024-05-08T08:54:11.182 INFO:teuthology.orchestra.run.smithi196.stdout:5152 trunc from 0xdb90e00 to 0xd2ca800 2024-05-08T08:54:11.227 INFO:teuthology.orchestra.run.smithi196.stdout:5153 write 0xea0a000 thru 0xea12d81 (0x8d82 bytes) 2024-05-08T08:54:11.228 INFO:teuthology.orchestra.run.smithi196.stdout:5155 write 0x7007c00 thru 0x700fc49 (0x804a bytes) 2024-05-08T08:54:11.228 INFO:teuthology.orchestra.run.smithi196.stdout:5157 read 0xe445c00 thru 0xe44afd6 (0x53d7 bytes) 2024-05-08T08:54:11.232 INFO:teuthology.orchestra.run.smithi196.stdout:5159 write 0xe4c1000 thru 0xe4c1e89 (0xe8a bytes) 2024-05-08T08:54:11.232 INFO:teuthology.orchestra.run.smithi196.stdout:5161 punch from 0xe28ba00 to 0xe294000, (0x8600 bytes) 2024-05-08T08:54:11.241 INFO:teuthology.orchestra.run.smithi196.stdout:5163 trunc from 0xea12e00 to 0x72a3e00 2024-05-08T08:54:11.244 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:11.244 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:11.254 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:54:11.272 INFO:teuthology.orchestra.run.smithi196.stdout:5165 read 0x4fbe200 thru 0x4fcaa43 (0xc844 bytes) 2024-05-08T08:54:11.288 INFO:teuthology.orchestra.run.smithi196.stdout:5168 write 0xc1f1800 thru 0xc1fd878 (0xc079 bytes) 2024-05-08T08:54:11.289 INFO:teuthology.orchestra.run.smithi196.stdout:5170 write 0x508e200 thru 0x5094280 (0x6081 bytes) 2024-05-08T08:54:11.289 INFO:teuthology.orchestra.run.smithi196.stdout:5173 write 0x12f3000 thru 0x12fc176 (0x9177 bytes) 2024-05-08T08:54:11.295 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:11.298+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:11.318 INFO:teuthology.orchestra.run.smithi196.stdout:5174 write 0xd72de00 thru 0xd7315af (0x37b0 bytes) 2024-05-08T08:54:11.325 INFO:teuthology.orchestra.run.smithi196.stdout:5179 read 0x5458400 thru 0x54664d3 (0xe0d4 bytes) 2024-05-08T08:54:11.325 INFO:teuthology.orchestra.run.smithi196.stdout:5181 write 0x5ea6200 thru 0x5eb5a59 (0xf85a bytes) 2024-05-08T08:54:11.326 INFO:teuthology.orchestra.run.smithi196.stdout:5182 write 0xa90c000 thru 0xa9133c0 (0x73c1 bytes) 2024-05-08T08:54:11.327 INFO:teuthology.orchestra.run.smithi196.stdout:5183 trunc from 0xd731600 to 0xe243400 2024-05-08T08:54:11.354 INFO:teuthology.orchestra.run.smithi196.stdout:5184 read 0xd8f7800 thru 0xd90726a (0xfa6b bytes) 2024-05-08T08:54:11.362 INFO:teuthology.orchestra.run.smithi196.stdout:5187 read 0x946fe00 thru 0x947e020 (0xe221 bytes) 2024-05-08T08:54:11.362 INFO:teuthology.orchestra.run.smithi196.stdout:5188 read 0x5664c00 thru 0x56737d9 (0xebda bytes) 2024-05-08T08:54:11.362 INFO:teuthology.orchestra.run.smithi196.stdout:5189 read 0x1e49600 thru 0x1e57bc3 (0xe5c4 bytes) 2024-05-08T08:54:11.363 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:11.365+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:11.363 INFO:teuthology.orchestra.run.smithi196.stdout:5190 read 0x8f4b400 thru 0x8f51d89 (0x698a bytes) 2024-05-08T08:54:11.364 INFO:teuthology.orchestra.run.smithi196.stdout:5191 read 0x8f76200 thru 0x8f7c567 (0x6368 bytes) 2024-05-08T08:54:11.364 INFO:teuthology.orchestra.run.smithi196.stdout:5192 punch from 0x62bcc00 to 0x62c6c00, (0xa000 bytes) 2024-05-08T08:54:11.364 INFO:teuthology.orchestra.run.smithi196.stdout:5197 punch from 0x68c6000 to 0x68c6800, (0x800 bytes) 2024-05-08T08:54:11.364 INFO:teuthology.orchestra.run.smithi196.stdout:5198 write 0x9cbaa00 thru 0x9cc3c41 (0x9242 bytes) 2024-05-08T08:54:11.364 INFO:teuthology.orchestra.run.smithi196.stdout:5199 write 0xe66600 thru 0xe6be57 (0x5858 bytes) 2024-05-08T08:54:11.364 INFO:teuthology.orchestra.run.smithi196.stdout:5200 read 0x373a00 thru 0x378fbc (0x55bd bytes) 2024-05-08T08:54:11.364 INFO:teuthology.orchestra.run.smithi196.stdout:5202 write 0x59fb400 thru 0x5a063da (0xafdb bytes) 2024-05-08T08:54:11.365 INFO:teuthology.orchestra.run.smithi196.stdout:5203 write 0x4d80e00 thru 0x4d909e1 (0xfbe2 bytes) 2024-05-08T08:54:11.365 INFO:teuthology.orchestra.run.smithi196.stdout:5205 punch from 0x3ee000 to 0x3f9e00, (0xbe00 bytes) 2024-05-08T08:54:11.374 INFO:teuthology.orchestra.run.smithi196.stdout:5206 trunc from 0xe243400 to 0x73f3800 2024-05-08T08:54:11.413 INFO:teuthology.orchestra.run.smithi196.stdout:5208 punch from 0xc58800 to 0xc59600, (0xe00 bytes) 2024-05-08T08:54:11.414 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:11.414 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:11.415 INFO:teuthology.orchestra.run.smithi196.stdout:5209 read 0x1887e00 thru 0x1894c49 (0xce4a bytes) 2024-05-08T08:54:11.415 INFO:teuthology.orchestra.run.smithi196.stdout:5210 punch from 0x4964400 to 0x4966c00, (0x2800 bytes) 2024-05-08T08:54:11.425 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:54:11.432 INFO:teuthology.orchestra.run.smithi196.stdout:5211 write 0xd74d600 thru 0xd750299 (0x2c9a bytes) 2024-05-08T08:54:11.437 INFO:teuthology.orchestra.run.smithi196.stdout:5212 trunc from 0xd750400 to 0xa7e7c00 2024-05-08T08:54:11.463 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:11.466+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:11.467 INFO:teuthology.orchestra.run.smithi196.stdout:5216 write 0x7635000 thru 0x7636c1b (0x1c1c bytes) 2024-05-08T08:54:11.468 INFO:teuthology.orchestra.run.smithi196.stdout:5219 read 0x6dda200 thru 0x6ddad4d (0xb4e bytes) 2024-05-08T08:54:11.489 INFO:teuthology.orchestra.run.smithi196.stdout:5223 write 0xb9da600 thru 0xb9e9a89 (0xf48a bytes) 2024-05-08T08:54:11.491 INFO:teuthology.orchestra.run.smithi196.stdout:5224 write 0x565d800 thru 0x56604e4 (0x2ce5 bytes) 2024-05-08T08:54:11.491 INFO:teuthology.orchestra.run.smithi196.stdout:5225 punch from 0x1882e00 to 0x1890400, (0xd600 bytes) 2024-05-08T08:54:11.495 INFO:teuthology.orchestra.run.smithi196.stdout:5227 read 0x25d7400 thru 0x25e5103 (0xdd04 bytes) 2024-05-08T08:54:11.501 INFO:teuthology.orchestra.run.smithi196.stdout:5229 trunc from 0xb9e9c00 to 0x304fe00 2024-05-08T08:54:11.549 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:11.550 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:11.559 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:54:11.564 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:11.567+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:11.567 INFO:teuthology.orchestra.run.smithi196.stdout:5231 write 0x7aaa000 thru 0x7aac6d1 (0x26d2 bytes) 2024-05-08T08:54:11.567 INFO:teuthology.orchestra.run.smithi196.stdout:5234 punch from 0x98c200 to 0x99be00, (0xfc00 bytes) 2024-05-08T08:54:11.576 INFO:teuthology.orchestra.run.smithi196.stdout:5237 trunc from 0x7aac800 to 0xdc5a600 2024-05-08T08:54:11.586 INFO:teuthology.orchestra.run.smithi196.stdout:5239 punch from 0x3d67600 to 0x3d76a00, (0xf400 bytes) 2024-05-08T08:54:11.587 INFO:teuthology.orchestra.run.smithi196.stdout:5240 write 0x7b05000 thru 0x7b07730 (0x2731 bytes) 2024-05-08T08:54:11.588 INFO:teuthology.orchestra.run.smithi196.stdout:5241 read 0x1fa7000 thru 0x1fa7984 (0x985 bytes) 2024-05-08T08:54:11.588 INFO:teuthology.orchestra.run.smithi196.stdout:5242 punch from 0x92b3a00 to 0x92b5400, (0x1a00 bytes) 2024-05-08T08:54:11.588 INFO:teuthology.orchestra.run.smithi196.stdout:5243 read 0x6d62200 thru 0x6d6693b (0x473c bytes) 2024-05-08T08:54:11.600 INFO:teuthology.orchestra.run.smithi196.stdout:5246 trunc from 0xdc5a600 to 0x43b9000 2024-05-08T08:54:11.625 INFO:teuthology.orchestra.run.smithi196.stdout:5247 trunc from 0x43b9000 to 0xb39e00 2024-05-08T08:54:11.663 INFO:teuthology.orchestra.run.smithi196.stdout:5251 read 0x8c8c00 thru 0x8d5e2c (0xd22d bytes) 2024-05-08T08:54:11.666 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:11.668+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:11.669 INFO:teuthology.orchestra.run.smithi196.stdout:5254 read 0x569c00 thru 0x570cfd (0x70fe bytes) 2024-05-08T08:54:11.675 INFO:teuthology.orchestra.run.smithi196.stdout:5256 trunc from 0xb39e00 to 0x5cf4600 2024-05-08T08:54:11.678 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:11.678 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:11.685 INFO:teuthology.orchestra.run.smithi196.stdout:5258 read 0x5025800 thru 0x5031143 (0xb944 bytes) 2024-05-08T08:54:11.686 INFO:teuthology.orchestra.run.smithi196.stdout:5259 read 0x564e000 thru 0x564fbd5 (0x1bd6 bytes) 2024-05-08T08:54:11.688 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:54:11.705 INFO:teuthology.orchestra.run.smithi196.stdout:5263 write 0xcff3200 thru 0xcff557a (0x237b bytes) 2024-05-08T08:54:11.709 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-05-08T08:54:11.718 INFO:teuthology.orchestra.run.smithi196.stdout:5264 trunc from 0xcff5600 to 0x254ec00 2024-05-08T08:54:11.762 INFO:teuthology.orchestra.run.smithi196.stdout:5266 write 0x644f800 thru 0x645902e (0x982f bytes) 2024-05-08T08:54:11.763 INFO:teuthology.orchestra.run.smithi196.stdout:5273 punch from 0x5376a00 to 0x5378c00, (0x2200 bytes) 2024-05-08T08:54:11.767 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:11.770+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:11.785 INFO:teuthology.orchestra.run.smithi196.stdout:5278 write 0xd27fe00 thru 0xd287f8c (0x818d bytes) 2024-05-08T08:54:11.786 INFO:teuthology.orchestra.run.smithi196.stdout:5279 read 0x8a7d800 thru 0x8a841af (0x69b0 bytes) 2024-05-08T08:54:11.786 INFO:teuthology.orchestra.run.smithi196.stdout:5281 trunc from 0xd288000 to 0xd28f000 2024-05-08T08:54:11.807 INFO:teuthology.orchestra.run.smithi196.stdout:5282 write 0x4925e00 thru 0x4934354 (0xe555 bytes) 2024-05-08T08:54:11.808 INFO:teuthology.orchestra.run.smithi196.stdout:5284 trunc from 0xd28f000 to 0xd53a400 2024-05-08T08:54:11.826 INFO:teuthology.orchestra.run.smithi196.stdout:5285 punch from 0x72e7400 to 0x72f3c00, (0xc800 bytes) 2024-05-08T08:54:11.838 INFO:teuthology.orchestra.run.smithi196.stdout:5287 trunc from 0xd53a400 to 0x3d6d200 2024-05-08T08:54:11.853 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:11.853 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:11.862 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:54:11.865 INFO:teuthology.orchestra.run.smithi196.stdout:5288 write 0x2aaa000 thru 0x2ab4b24 (0xab25 bytes) 2024-05-08T08:54:11.868 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:11.871+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:11.885 INFO:teuthology.orchestra.run.smithi196.stdout:5290 write 0x643b000 thru 0x644932c (0xe32d bytes) 2024-05-08T08:54:11.885 INFO:teuthology.orchestra.run.smithi196.stdout:5292 punch from 0x3314c00 to 0x3319200, (0x4600 bytes) 2024-05-08T08:54:11.910 INFO:teuthology.orchestra.run.smithi196.stdout:5293 write 0xc002000 thru 0xc00544d (0x344e bytes) 2024-05-08T08:54:11.911 INFO:teuthology.orchestra.run.smithi196.stdout:5294 trunc from 0xc005600 to 0xb436e00 2024-05-08T08:54:11.933 INFO:teuthology.orchestra.run.smithi196.stdout:5296 read 0x4504e00 thru 0x450e7d2 (0x99d3 bytes) 2024-05-08T08:54:11.935 INFO:teuthology.orchestra.run.smithi196.stdout:5297 punch from 0x3e9f800 to 0x3ea5400, (0x5c00 bytes) 2024-05-08T08:54:11.935 INFO:teuthology.orchestra.run.smithi196.stdout:5298 write 0x6c73e00 thru 0x6c81e87 (0xe088 bytes) 2024-05-08T08:54:11.935 INFO:teuthology.orchestra.run.smithi196.stdout:5299 write 0x62b8400 thru 0x62b884d (0x44e bytes) 2024-05-08T08:54:11.936 INFO:teuthology.orchestra.run.smithi196.stdout:5300 punch from 0x75c8a00 to 0x75d1600, (0x8c00 bytes) 2024-05-08T08:54:11.937 INFO:teuthology.orchestra.run.smithi196.stdout:5301 trunc from 0xb436e00 to 0xc5a9c00 2024-05-08T08:54:11.965 INFO:teuthology.orchestra.run.smithi196.stdout:5302 write 0x856f200 thru 0x85753f2 (0x61f3 bytes) 2024-05-08T08:54:11.966 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T08:54:11.967 INFO:teuthology.orchestra.run.smithi196.stdout:5303 read 0x532200 thru 0x53a9ac (0x87ad bytes) 2024-05-08T08:54:11.967 INFO:teuthology.orchestra.run.smithi196.stdout:5304 read 0x9575600 thru 0x95756c6 (0xc7 bytes) 2024-05-08T08:54:11.968 INFO:teuthology.orchestra.run.smithi196.stdout:5306 punch from 0x3548800 to 0x354c400, (0x3c00 bytes) 2024-05-08T08:54:11.969 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:11.971+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:11.979 INFO:teuthology.orchestra.run.smithi196.stdout:5307 trunc from 0xc5a9c00 to 0x2730a00 2024-05-08T08:54:11.984 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:11.985 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:11.996 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:54:12.020 INFO:teuthology.orchestra.run.smithi196.stdout:5308 read 0x25fe600 thru 0x25fe98d (0x38e bytes) 2024-05-08T08:54:12.047 INFO:teuthology.orchestra.run.smithi196.stdout:5309 write 0xd6dc600 thru 0xd6eb3be (0xedbf bytes) 2024-05-08T08:54:12.058 INFO:teuthology.orchestra.run.smithi196.stdout:5310 trunc from 0xd6eb400 to 0x5d5ec00 2024-05-08T08:54:12.070 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:12.073+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:12.091 INFO:teuthology.orchestra.run.smithi196.stdout:5314 punch from 0x179d200 to 0x17aca00, (0xf800 bytes) 2024-05-08T08:54:12.093 INFO:teuthology.orchestra.run.smithi196.stdout:5315 read 0x4319e00 thru 0x43253c8 (0xb5c9 bytes) 2024-05-08T08:54:12.093 INFO:teuthology.orchestra.run.smithi196.stdout:5316 punch from 0x42c5800 to 0x42d2400, (0xcc00 bytes) 2024-05-08T08:54:12.093 INFO:teuthology.orchestra.run.smithi196.stdout:5317 read 0x33eac00 thru 0x33f436b (0x976c bytes) 2024-05-08T08:54:12.119 INFO:teuthology.orchestra.run.smithi196.stdout:5318 write 0xb011c00 thru 0xb02010a (0xe50b bytes) 2024-05-08T08:54:12.119 INFO:teuthology.orchestra.run.smithi196.stdout:5319 punch from 0x3b8ae00 to 0x3b93a00, (0x8c00 bytes) 2024-05-08T08:54:12.120 INFO:teuthology.orchestra.run.smithi196.stdout:5323 punch from 0xa196800 to 0xa1a4000, (0xd800 bytes) 2024-05-08T08:54:12.120 INFO:teuthology.orchestra.run.smithi196.stdout:5329 read 0x6d6f400 thru 0x6d75d22 (0x6923 bytes) 2024-05-08T08:54:12.120 INFO:teuthology.orchestra.run.smithi196.stdout:5332 write 0xa17f200 thru 0xa18b6a4 (0xc4a5 bytes) 2024-05-08T08:54:12.140 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:54:12.140 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:54:12.145 INFO:teuthology.orchestra.run.smithi196.stdout:5335 write 0xe1c7000 thru 0xe1ce862 (0x7863 bytes) 2024-05-08T08:54:12.146 INFO:teuthology.orchestra.run.smithi196.stdout:5338 punch from 0xca0a400 to 0xca16a00, (0xc600 bytes) 2024-05-08T08:54:12.147 INFO:teuthology.orchestra.run.smithi196.stdout:5341 punch from 0x1637200 to 0x1641e00, (0xac00 bytes) 2024-05-08T08:54:12.147 INFO:teuthology.orchestra.run.smithi196.stdout:5343 read 0x4e2bc00 thru 0x4e34da1 (0x91a2 bytes) 2024-05-08T08:54:12.147 INFO:teuthology.orchestra.run.smithi196.stdout:5344 write 0xc763200 thru 0xc76e424 (0xb225 bytes) 2024-05-08T08:54:12.147 INFO:teuthology.orchestra.run.smithi196.stdout:5345 punch from 0x4033000 to 0x403e200, (0xb200 bytes) 2024-05-08T08:54:12.148 INFO:teuthology.orchestra.run.smithi196.stdout:5346 punch from 0xb91e000 to 0xb927000, (0x9000 bytes) 2024-05-08T08:54:12.148 INFO:teuthology.orchestra.run.smithi196.stdout:5349 read 0x492e00 thru 0x494b5e (0x1d5f bytes) 2024-05-08T08:54:12.148 INFO:teuthology.orchestra.run.smithi196.stdout:5350 write 0xc0fe000 thru 0xc0fe015 (0x16 bytes) 2024-05-08T08:54:12.148 INFO:teuthology.orchestra.run.smithi196.stdout:5354 read 0xc3b9200 thru 0xc3c5048 (0xbe49 bytes) 2024-05-08T08:54:12.162 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:12.162 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:54:12.162 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:54:12.162 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:12.162 INFO:teuthology.orchestra.run.smithi196.stdout:5356 trunc from 0xe1cea00 to 0x18be400 2024-05-08T08:54:12.170 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:54:12.171 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:54:12.171 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:12.173+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:12.174 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:54:12.199 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:54:12.199 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:54:12.224 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:54:12.224 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:54:12.238 INFO:teuthology.orchestra.run.smithi196.stdout:5358 write 0xe1e1800 thru 0xe1e6e18 (0x5619 bytes) 2024-05-08T08:54:12.239 INFO:teuthology.orchestra.run.smithi196.stdout:5359 read 0x4e45000 thru 0x4e518ba (0xc8bb bytes) 2024-05-08T08:54:12.240 INFO:teuthology.orchestra.run.smithi196.stdout:5360 punch from 0xc3d6800 to 0xc3d9e00, (0x3600 bytes) 2024-05-08T08:54:12.240 INFO:teuthology.orchestra.run.smithi196.stdout:5361 write 0x3890600 thru 0x389e987 (0xe388 bytes) 2024-05-08T08:54:12.240 INFO:teuthology.orchestra.run.smithi196.stdout:5363 read 0x24d0800 thru 0x24d6efc (0x66fd bytes) 2024-05-08T08:54:12.240 INFO:teuthology.orchestra.run.smithi196.stdout:5365 punch from 0xa671800 to 0xa672a00, (0x1200 bytes) 2024-05-08T08:54:12.241 INFO:teuthology.orchestra.run.smithi196.stdout:5367 read 0xcaa6800 thru 0xcaaeb07 (0x8308 bytes) 2024-05-08T08:54:12.241 INFO:teuthology.orchestra.run.smithi196.stdout:5370 write 0x97afa00 thru 0x97bd2be (0xd8bf bytes) 2024-05-08T08:54:12.259 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:54:12.259 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:54:12.272 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:12.274+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:12.277 INFO:teuthology.orchestra.run.smithi196.stdout:5372 write 0xebc3200 thru 0xebd0df3 (0xdbf4 bytes) 2024-05-08T08:54:12.278 INFO:teuthology.orchestra.run.smithi196.stdout:5373 read 0xc48e00 thru 0xc4a057 (0x1258 bytes) 2024-05-08T08:54:12.278 INFO:teuthology.orchestra.run.smithi196.stdout:5374 punch from 0x57f1000 to 0x57fe400, (0xd400 bytes) 2024-05-08T08:54:12.282 INFO:teuthology.orchestra.run.smithi196.stdout:5375 punch from 0x9bbce00 to 0x9bc5400, (0x8600 bytes) 2024-05-08T08:54:12.283 INFO:teuthology.orchestra.run.smithi196.stdout:5377 write 0x4b3f000 thru 0x4b49c3d (0xac3e bytes) 2024-05-08T08:54:12.283 INFO:teuthology.orchestra.run.smithi196.stdout:5378 write 0x5d00000 thru 0x5d0ec6b (0xec6c bytes) 2024-05-08T08:54:12.283 INFO:teuthology.orchestra.run.smithi196.stdout:5380 read 0xce72e00 thru 0xce807f8 (0xd9f9 bytes) 2024-05-08T08:54:12.285 INFO:teuthology.orchestra.run.smithi196.stdout:5384 trunc from 0xebd0e00 to 0xc7b2800 2024-05-08T08:54:12.294 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:54:12.295 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:54:12.330 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:12.330 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:54:12.330 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:54:12.330 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:54:12.330 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:12.333 INFO:teuthology.orchestra.run.smithi196.stdout:5386 write 0x9796800 thru 0x97a367b (0xce7c bytes) 2024-05-08T08:54:12.334 INFO:teuthology.orchestra.run.smithi196.stdout:5392 write 0xc3c5400 thru 0xc3c55d4 (0x1d5 bytes) 2024-05-08T08:54:12.334 INFO:teuthology.orchestra.run.smithi196.stdout:5393 write 0x5f73200 thru 0x5f75b7f (0x2980 bytes) 2024-05-08T08:54:12.334 INFO:teuthology.orchestra.run.smithi196.stdout:5394 write 0x279da00 thru 0x279eb09 (0x110a bytes) 2024-05-08T08:54:12.337 INFO:teuthology.orchestra.run.smithi196.stdout:5395 trunc from 0xc7b2800 to 0x9fab600 2024-05-08T08:54:12.340 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:54:12.363 INFO:teuthology.orchestra.run.smithi196.stdout:5396 read 0x6f70200 thru 0x6f70cf3 (0xaf4 bytes) 2024-05-08T08:54:12.365 INFO:teuthology.orchestra.run.smithi196.stdout:5399 write 0x99a4c00 thru 0x99aa7c6 (0x5bc7 bytes) 2024-05-08T08:54:12.365 INFO:teuthology.orchestra.run.smithi196.stdout:5400 write 0x6161600 thru 0x616fdea (0xe7eb bytes) 2024-05-08T08:54:12.365 INFO:teuthology.orchestra.run.smithi196.stdout:5403 read 0x4de400 thru 0x4eacc5 (0xc8c6 bytes) 2024-05-08T08:54:12.365 INFO:teuthology.orchestra.run.smithi196.stdout:5405 write 0x187d000 thru 0x18844cc (0x74cd bytes) 2024-05-08T08:54:12.365 INFO:teuthology.orchestra.run.smithi196.stdout:5406 read 0x1acb200 thru 0x1ad7b4a (0xc94b bytes) 2024-05-08T08:54:12.365 INFO:teuthology.orchestra.run.smithi196.stdout:5407 write 0x6ef9000 thru 0x6ef9fe5 (0xfe6 bytes) 2024-05-08T08:54:12.365 INFO:teuthology.orchestra.run.smithi196.stdout:5408 read 0x3d05c00 thru 0x3d093a8 (0x37a9 bytes) 2024-05-08T08:54:12.372 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:12.375+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:12.393 INFO:teuthology.orchestra.run.smithi196.stdout:5409 write 0xa82e800 thru 0xa8335aa (0x4dab bytes) 2024-05-08T08:54:12.414 INFO:teuthology.orchestra.run.smithi196.stdout:5411 write 0xd190400 thru 0xd194375 (0x3f76 bytes) 2024-05-08T08:54:12.415 INFO:teuthology.orchestra.run.smithi196.stdout:5414 read 0xbede00 thru 0xbf1156 (0x3357 bytes) 2024-05-08T08:54:12.416 INFO:teuthology.orchestra.run.smithi196.stdout:5415 write 0x9fdd600 thru 0x9fe6085 (0x8a86 bytes) 2024-05-08T08:54:12.416 INFO:teuthology.orchestra.run.smithi196.stdout:5416 write 0x3c66c00 thru 0x3c74dff (0xe200 bytes) 2024-05-08T08:54:12.416 INFO:teuthology.orchestra.run.smithi196.stdout:5419 clone 40 order 23 su 262144 sc 11 2024-05-08T08:54:12.473 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:12.476+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:12.504 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:12.504 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:54:12.504 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:54:12.504 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:54:12.504 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.504 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4621.0:9089 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 23339520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:54:12.504 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:54:12.504 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 100.00951112200001, 2024-05-08T08:54:12.504 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.13814907200000001, 2024-05-08T08:54:12.504 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:12.505 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:12.505 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:54:12.505 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:12.505 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4621", 2024-05-08T08:54:12.505 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:12.505 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9089 2024-05-08T08:54:12.505 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.505 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:12.505 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.505 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:12.505 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:54:12.505 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:12.505 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.505 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.505 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:12.506 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:54:12.506 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:12.506 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.506 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.506 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:12.506 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493005+0000", 2024-05-08T08:54:12.506 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1700000000000001e-06 2024-05-08T08:54:12.506 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.506 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.506 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:12.506 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.513963+0000", 2024-05-08T08:54:12.506 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.020957581999999999 2024-05-08T08:54:12.506 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.506 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.507 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:12.507 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.513995+0000", 2024-05-08T08:54:12.507 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1993999999999999e-05 2024-05-08T08:54:12.507 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.507 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.507 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:12.507 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514007+0000", 2024-05-08T08:54:12.507 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1581e-05 2024-05-08T08:54:12.507 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.507 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.507 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:12.507 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514122+0000", 2024-05-08T08:54:12.507 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000115282 2024-05-08T08:54:12.507 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.507 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.508 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op must wait for map", 2024-05-08T08:54:12.508 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514128+0000", 2024-05-08T08:54:12.508 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9950000000000002e-06 2024-05-08T08:54:12.508 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.508 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.508 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:12.508 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.515800+0000", 2024-05-08T08:54:12.508 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001671886 2024-05-08T08:54:12.508 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.508 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.508 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:12.508 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.515893+0000", 2024-05-08T08:54:12.508 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3490000000000001e-05 2024-05-08T08:54:12.508 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.509 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.509 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 3", 2024-05-08T08:54:12.509 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.517647+0000", 2024-05-08T08:54:12.509 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0017535630000000001 2024-05-08T08:54:12.509 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.509 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.509 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:54:12.509 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.627578+0000", 2024-05-08T08:54:12.509 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.109931557 2024-05-08T08:54:12.509 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.509 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.509 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:54:12.509 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631055+0000", 2024-05-08T08:54:12.509 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0034761420000000002 2024-05-08T08:54:12.510 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.510 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.510 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:54:12.510 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631096+0000", 2024-05-08T08:54:12.510 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1059000000000001e-05 2024-05-08T08:54:12.510 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.510 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.510 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:12.510 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631151+0000", 2024-05-08T08:54:12.510 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5770999999999998e-05 2024-05-08T08:54:12.510 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:12.510 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:12.510 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:12.510 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.510 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.510 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1071 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11026432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:12.511 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:54:12.511 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 96.017061631000004, 2024-05-08T08:54:12.511 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070580037999999998, 2024-05-08T08:54:12.511 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:12.511 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:12.511 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:12.511 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:12.511 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:12.511 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:12.511 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1071 2024-05-08T08:54:12.511 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.511 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:12.511 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.511 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:12.511 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:54:12.511 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:12.511 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.512 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.512 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:12.512 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:54:12.512 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:12.512 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.512 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.512 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:12.512 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:54:12.512 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2800000000000002e-07 2024-05-08T08:54:12.512 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.512 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.512 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:12.512 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485454+0000", 2024-05-08T08:54:12.512 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2290000000000002e-06 2024-05-08T08:54:12.512 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.512 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.512 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:12.513 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485461+0000", 2024-05-08T08:54:12.513 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9920000000000001e-06 2024-05-08T08:54:12.513 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.513 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.513 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:12.513 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485467+0000", 2024-05-08T08:54:12.513 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3979999999999998e-06 2024-05-08T08:54:12.513 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.513 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.513 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:12.513 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555880+0000", 2024-05-08T08:54:12.513 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070413165 2024-05-08T08:54:12.513 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.513 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.513 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:12.513 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555926+0000", 2024-05-08T08:54:12.514 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6217000000000002e-05 2024-05-08T08:54:12.514 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.514 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.514 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:12.514 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556032+0000", 2024-05-08T08:54:12.514 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000105609 2024-05-08T08:54:12.514 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:12.514 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:12.514 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:12.514 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.514 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.514 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1073 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11067392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:12.514 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:54:12.514 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 96.017040410000007, 2024-05-08T08:54:12.514 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070821404000000004, 2024-05-08T08:54:12.515 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:12.515 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:12.515 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:12.515 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:12.515 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:12.515 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:12.515 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1073 2024-05-08T08:54:12.515 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.515 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:12.515 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.515 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:12.515 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:54:12.515 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:12.515 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.516 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.516 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:12.516 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:54:12.516 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:12.516 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.516 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.516 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:12.516 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:54:12.516 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.08e-07 2024-05-08T08:54:12.516 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.516 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.516 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:12.516 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485476+0000", 2024-05-08T08:54:12.516 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3929999999999998e-06 2024-05-08T08:54:12.517 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.517 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.517 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:12.517 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485485+0000", 2024-05-08T08:54:12.517 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1880000000000008e-06 2024-05-08T08:54:12.517 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.517 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.517 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:12.517 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485494+0000", 2024-05-08T08:54:12.517 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3109999999999995e-06 2024-05-08T08:54:12.517 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.518 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.518 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:12.518 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556082+0000", 2024-05-08T08:54:12.518 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070587426999999994 2024-05-08T08:54:12.518 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.518 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.518 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:12.518 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556177+0000", 2024-05-08T08:54:12.518 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5051999999999997e-05 2024-05-08T08:54:12.518 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.518 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.518 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:12.518 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556294+0000", 2024-05-08T08:54:12.518 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000117625 2024-05-08T08:54:12.519 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:12.519 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:12.519 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:12.519 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.519 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.519 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1075 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11108352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:12.519 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:54:12.519 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 96.017009833000003, 2024-05-08T08:54:12.519 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071045026999999997, 2024-05-08T08:54:12.519 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:12.519 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:12.519 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:12.520 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:12.520 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:12.520 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:12.520 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1075 2024-05-08T08:54:12.520 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.520 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:12.520 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.520 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:12.520 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:54:12.520 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:12.520 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.520 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.520 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:12.520 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:54:12.521 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:12.521 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.521 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.521 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:12.521 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:54:12.521 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.32e-07 2024-05-08T08:54:12.521 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.521 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.521 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:12.521 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485518+0000", 2024-05-08T08:54:12.521 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3769e-05 2024-05-08T08:54:12.521 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.522 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.522 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:12.522 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485527+0000", 2024-05-08T08:54:12.522 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6779999999999995e-06 2024-05-08T08:54:12.522 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.522 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.522 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:12.522 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485532+0000", 2024-05-08T08:54:12.522 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9830000000000004e-06 2024-05-08T08:54:12.522 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.522 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.522 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:12.523 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556340+0000", 2024-05-08T08:54:12.523 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070807743000000006 2024-05-08T08:54:12.523 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.523 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.523 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:12.523 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556435+0000", 2024-05-08T08:54:12.523 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4732e-05 2024-05-08T08:54:12.523 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.523 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.523 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:12.523 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556549+0000", 2024-05-08T08:54:12.523 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011369 2024-05-08T08:54:12.524 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:12.524 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:12.524 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:12.524 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.524 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.524 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1078 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11149312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:12.524 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:54:12.524 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 96.016969634000006, 2024-05-08T08:54:12.524 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071167276000000002, 2024-05-08T08:54:12.524 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:12.524 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:12.524 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:12.525 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:12.525 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:12.525 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:12.525 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1078 2024-05-08T08:54:12.525 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.525 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:12.525 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.525 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:12.525 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:54:12.525 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:12.525 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.525 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.525 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:12.526 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:54:12.526 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:12.526 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.526 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.526 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:12.526 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:54:12.526 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5600000000000001e-07 2024-05-08T08:54:12.526 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.526 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.526 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:12.526 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485547+0000", 2024-05-08T08:54:12.526 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3590000000000002e-06 2024-05-08T08:54:12.527 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.527 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.527 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:12.527 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485553+0000", 2024-05-08T08:54:12.527 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2770000000000002e-06 2024-05-08T08:54:12.527 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.527 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.527 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:12.527 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485558+0000", 2024-05-08T08:54:12.527 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.95e-06 2024-05-08T08:54:12.527 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.527 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.528 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:12.528 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556582+0000", 2024-05-08T08:54:12.528 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071024444000000006 2024-05-08T08:54:12.528 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.528 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.528 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:12.528 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556637+0000", 2024-05-08T08:54:12.528 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4672000000000001e-05 2024-05-08T08:54:12.528 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.528 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.528 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:12.528 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556711+0000", 2024-05-08T08:54:12.529 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4117999999999997e-05 2024-05-08T08:54:12.529 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:12.529 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:12.529 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:12.529 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.529 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.529 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1080 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11190272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:12.529 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:54:12.529 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 96.016948901000006, 2024-05-08T08:54:12.529 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071350721000000006, 2024-05-08T08:54:12.529 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:12.529 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:12.529 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:12.530 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:12.530 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:12.530 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:12.530 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1080 2024-05-08T08:54:12.530 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.530 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:12.530 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.530 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:12.530 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:54:12.530 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:12.530 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.530 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.531 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:12.531 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:54:12.531 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:12.531 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.531 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.531 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:12.531 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:54:12.531 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.03e-07 2024-05-08T08:54:12.531 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.531 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.531 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:12.531 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485567+0000", 2024-05-08T08:54:12.531 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1090000000000001e-06 2024-05-08T08:54:12.532 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.532 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.532 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:12.532 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485573+0000", 2024-05-08T08:54:12.532 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3049999999999998e-06 2024-05-08T08:54:12.532 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.532 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.532 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:12.532 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485578+0000", 2024-05-08T08:54:12.532 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0130000000000001e-06 2024-05-08T08:54:12.532 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.533 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.533 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:12.533 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556743+0000", 2024-05-08T08:54:12.533 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071164449000000005 2024-05-08T08:54:12.533 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.533 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.533 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:12.533 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556797+0000", 2024-05-08T08:54:12.533 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4589999999999997e-05 2024-05-08T08:54:12.533 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.533 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.533 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:12.533 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556915+0000", 2024-05-08T08:54:12.533 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000117852 2024-05-08T08:54:12.534 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:12.534 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:12.534 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:12.534 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.534 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.534 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1082 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11231232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:12.534 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:54:12.534 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 96.016928657999998, 2024-05-08T08:54:12.534 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071591396000000002, 2024-05-08T08:54:12.534 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:12.534 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:12.534 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:12.534 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:12.535 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:12.535 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:12.535 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1082 2024-05-08T08:54:12.535 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.535 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:12.535 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.535 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:12.535 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:54:12.535 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:12.535 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.535 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.535 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:12.535 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:54:12.535 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:12.535 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.536 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.536 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:12.536 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:54:12.536 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1800000000000001e-07 2024-05-08T08:54:12.536 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.536 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.536 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:12.536 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485587+0000", 2024-05-08T08:54:12.536 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2589999999999999e-06 2024-05-08T08:54:12.536 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.536 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.536 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:12.536 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485594+0000", 2024-05-08T08:54:12.536 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0789999999999997e-06 2024-05-08T08:54:12.536 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.536 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.536 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:12.537 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485598+0000", 2024-05-08T08:54:12.537 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7480000000000001e-06 2024-05-08T08:54:12.537 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.537 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.537 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:12.537 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556971+0000", 2024-05-08T08:54:12.537 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071372774999999999 2024-05-08T08:54:12.537 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.537 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.537 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:12.537 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557069+0000", 2024-05-08T08:54:12.537 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.7732999999999994e-05 2024-05-08T08:54:12.537 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.537 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.537 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:12.537 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557176+0000", 2024-05-08T08:54:12.538 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000107384 2024-05-08T08:54:12.538 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:12.538 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:12.538 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:12.538 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.538 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.538 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1084 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11272192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:12.538 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:54:12.538 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 96.016890019000002, 2024-05-08T08:54:12.538 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071808977999999996, 2024-05-08T08:54:12.538 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:12.538 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:12.539 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:12.539 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:12.539 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:12.539 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:12.539 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1084 2024-05-08T08:54:12.539 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.539 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:12.539 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.539 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:12.539 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:54:12.539 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:12.539 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.539 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.539 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:12.539 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:54:12.539 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:12.540 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.540 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.540 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:12.540 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485624+0000", 2024-05-08T08:54:12.540 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:54:12.540 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.540 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.540 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:12.540 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485626+0000", 2024-05-08T08:54:12.540 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9460000000000001e-06 2024-05-08T08:54:12.540 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.540 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.540 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:12.540 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485633+0000", 2024-05-08T08:54:12.540 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1149999999999996e-06 2024-05-08T08:54:12.540 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.541 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.541 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:12.541 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485638+0000", 2024-05-08T08:54:12.541 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5670000000000002e-06 2024-05-08T08:54:12.541 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.541 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.541 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:12.541 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557226+0000", 2024-05-08T08:54:12.541 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071587267999999996 2024-05-08T08:54:12.541 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.541 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.541 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:12.541 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557321+0000", 2024-05-08T08:54:12.541 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5785000000000005e-05 2024-05-08T08:54:12.541 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.541 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.542 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:12.542 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557432+0000", 2024-05-08T08:54:12.542 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011092800000000001 2024-05-08T08:54:12.542 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:12.542 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:12.542 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:12.542 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.542 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.542 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1086 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11313152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:12.542 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:54:12.542 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 96.016866691999994, 2024-05-08T08:54:12.542 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071933132999999996, 2024-05-08T08:54:12.543 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:12.543 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:12.543 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:12.543 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:12.543 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:12.543 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:12.543 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1086 2024-05-08T08:54:12.543 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.543 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:12.543 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.543 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:12.543 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:54:12.543 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:12.543 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.544 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.544 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:12.544 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:54:12.544 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:12.544 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.544 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.544 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:12.544 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486328+0000", 2024-05-08T08:54:12.544 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00068107900000000002 2024-05-08T08:54:12.544 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.544 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.544 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:12.544 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486331+0000", 2024-05-08T08:54:12.544 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.585e-06 2024-05-08T08:54:12.545 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.545 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.545 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:12.545 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486343+0000", 2024-05-08T08:54:12.545 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1856e-05 2024-05-08T08:54:12.545 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.545 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.545 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:12.545 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486350+0000", 2024-05-08T08:54:12.545 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8600000000000004e-06 2024-05-08T08:54:12.545 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.545 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.545 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:12.545 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557470+0000", 2024-05-08T08:54:12.545 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071120326999999997 2024-05-08T08:54:12.546 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.546 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.546 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:12.546 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557514+0000", 2024-05-08T08:54:12.546 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3909000000000003e-05 2024-05-08T08:54:12.546 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.546 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.546 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:12.546 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557580+0000", 2024-05-08T08:54:12.546 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5517000000000003e-05 2024-05-08T08:54:12.546 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:12.546 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:12.546 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:12.546 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.547 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.547 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1089 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11354112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:12.547 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:54:12.547 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 96.01615382, 2024-05-08T08:54:12.547 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071457381, 2024-05-08T08:54:12.547 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:12.547 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:12.547 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:12.547 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:12.547 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:12.547 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:12.547 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1089 2024-05-08T08:54:12.547 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.547 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:12.547 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.548 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:12.548 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:54:12.548 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:12.548 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.548 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.548 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:12.548 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:54:12.548 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:12.548 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.548 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.548 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:12.548 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486851+0000", 2024-05-08T08:54:12.548 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00049102 2024-05-08T08:54:12.548 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.549 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.549 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:12.549 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486854+0000", 2024-05-08T08:54:12.549 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6859999999999999e-06 2024-05-08T08:54:12.549 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.549 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.549 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:12.549 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486864+0000", 2024-05-08T08:54:12.549 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6080000000000002e-06 2024-05-08T08:54:12.549 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.549 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.549 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:12.549 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486870+0000", 2024-05-08T08:54:12.550 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.477e-06 2024-05-08T08:54:12.550 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.550 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.550 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:12.550 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557612+0000", 2024-05-08T08:54:12.550 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070741557999999996 2024-05-08T08:54:12.550 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.550 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.550 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:12.550 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557695+0000", 2024-05-08T08:54:12.550 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3300000000000005e-05 2024-05-08T08:54:12.550 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.551 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.551 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:12.551 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557817+0000", 2024-05-08T08:54:12.551 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012173200000000001 2024-05-08T08:54:12.551 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:12.551 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:12.551 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:12.551 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.551 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.551 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1091 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11395072~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:12.551 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:54:12.551 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 96.015633025, 2024-05-08T08:54:12.551 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071184737999999997, 2024-05-08T08:54:12.552 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:12.552 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:12.552 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:12.552 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:12.552 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:12.552 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:12.552 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1091 2024-05-08T08:54:12.552 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.552 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:12.552 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.552 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:12.552 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:54:12.553 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:12.553 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.553 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.553 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:12.553 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:54:12.553 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:12.553 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.553 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.553 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:12.553 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486881+0000", 2024-05-08T08:54:12.553 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0699999999999997e-07 2024-05-08T08:54:12.553 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.554 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.554 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:12.554 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486883+0000", 2024-05-08T08:54:12.554 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.402e-06 2024-05-08T08:54:12.554 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.554 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.554 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:12.554 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486890+0000", 2024-05-08T08:54:12.554 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0480000000000001e-06 2024-05-08T08:54:12.554 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.554 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.554 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:12.555 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486896+0000", 2024-05-08T08:54:12.555 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7640000000000002e-06 2024-05-08T08:54:12.555 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.555 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.555 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:12.555 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557860+0000", 2024-05-08T08:54:12.555 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070963791999999998 2024-05-08T08:54:12.555 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.555 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.555 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:12.555 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557952+0000", 2024-05-08T08:54:12.555 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2082000000000004e-05 2024-05-08T08:54:12.556 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.556 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.556 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:12.556 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558065+0000", 2024-05-08T08:54:12.556 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000113143 2024-05-08T08:54:12.556 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:12.556 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:12.556 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:12.556 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.556 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.556 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1094 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11436032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:12.556 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:54:12.556 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 96.015609857000001, 2024-05-08T08:54:12.557 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071400507000000002, 2024-05-08T08:54:12.557 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:12.557 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:12.557 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:12.557 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:12.557 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:12.557 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:12.557 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1094 2024-05-08T08:54:12.557 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.557 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:12.557 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.557 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:12.557 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:54:12.558 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:12.558 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.558 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.558 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:12.558 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:54:12.558 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:12.558 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.558 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.558 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:12.558 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487940+0000", 2024-05-08T08:54:12.558 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001036868 2024-05-08T08:54:12.558 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.558 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.559 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:12.559 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487944+0000", 2024-05-08T08:54:12.559 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7069999999999998e-06 2024-05-08T08:54:12.559 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.559 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.559 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:12.559 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487951+0000", 2024-05-08T08:54:12.559 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2649999999999999e-06 2024-05-08T08:54:12.559 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.559 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.559 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:12.559 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487961+0000", 2024-05-08T08:54:12.560 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4110000000000002e-06 2024-05-08T08:54:12.560 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.560 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.560 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:12.560 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558117+0000", 2024-05-08T08:54:12.560 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070156662999999994 2024-05-08T08:54:12.560 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.560 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.560 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:12.560 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558214+0000", 2024-05-08T08:54:12.560 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6444e-05 2024-05-08T08:54:12.560 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.561 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.561 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:12.561 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558304+0000", 2024-05-08T08:54:12.561 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0148999999999994e-05 2024-05-08T08:54:12.561 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:12.561 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:12.561 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:12.561 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.561 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.561 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1096 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11476992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:12.561 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:54:12.561 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 96.014543325999995, 2024-05-08T08:54:12.562 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070480074000000004, 2024-05-08T08:54:12.562 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:12.562 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:12.562 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:12.562 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:12.562 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:12.562 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:12.562 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1096 2024-05-08T08:54:12.562 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.562 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:12.562 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.562 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:12.563 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:54:12.563 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:12.563 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.563 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.563 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:12.563 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:54:12.563 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:12.563 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.563 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.563 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:12.563 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:54:12.563 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3700000000000001e-07 2024-05-08T08:54:12.563 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.564 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.564 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:12.564 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487973+0000", 2024-05-08T08:54:12.564 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0870000000000001e-06 2024-05-08T08:54:12.564 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.564 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.564 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:12.564 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487977+0000", 2024-05-08T08:54:12.564 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0600000000000001e-06 2024-05-08T08:54:12.564 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.564 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.564 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:12.564 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487980+0000", 2024-05-08T08:54:12.565 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1089999999999998e-06 2024-05-08T08:54:12.565 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.565 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.565 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:12.565 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558338+0000", 2024-05-08T08:54:12.565 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070358233000000006 2024-05-08T08:54:12.565 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.565 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.565 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:12.565 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558392+0000", 2024-05-08T08:54:12.565 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4537999999999999e-05 2024-05-08T08:54:12.565 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.565 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.566 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:12.566 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558450+0000", 2024-05-08T08:54:12.566 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7710000000000001e-05 2024-05-08T08:54:12.566 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:12.566 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:12.566 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:12.566 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.566 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.566 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1099 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11517952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:12.566 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:54:12.566 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 96.014529449999998, 2024-05-08T08:54:12.566 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070720506000000002, 2024-05-08T08:54:12.566 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:12.567 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:12.567 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:12.567 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:12.567 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:12.567 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:12.567 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1099 2024-05-08T08:54:12.567 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.567 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:12.567 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.567 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:12.567 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:54:12.567 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:12.567 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.567 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.567 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:12.567 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:54:12.567 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:12.568 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.568 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.568 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:12.568 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:54:12.568 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3999999999999998e-07 2024-05-08T08:54:12.568 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.568 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.568 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:12.568 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487986+0000", 2024-05-08T08:54:12.568 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3519999999999999e-06 2024-05-08T08:54:12.568 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.568 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.568 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:12.568 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487989+0000", 2024-05-08T08:54:12.568 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.23e-06 2024-05-08T08:54:12.568 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.569 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.569 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:12.569 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487991+0000", 2024-05-08T08:54:12.569 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6350000000000002e-06 2024-05-08T08:54:12.569 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.569 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.569 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:12.569 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558489+0000", 2024-05-08T08:54:12.569 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070497924000000003 2024-05-08T08:54:12.569 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.569 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.569 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:12.569 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558583+0000", 2024-05-08T08:54:12.569 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3405000000000004e-05 2024-05-08T08:54:12.569 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.570 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.570 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:12.570 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558704+0000", 2024-05-08T08:54:12.570 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012171999999999999 2024-05-08T08:54:12.570 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:12.570 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:12.570 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:12.570 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.570 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.570 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1100 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11558912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:12.570 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:54:12.570 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 96.014516451000006, 2024-05-08T08:54:12.570 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070962266999999996, 2024-05-08T08:54:12.570 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:12.570 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:12.571 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:12.571 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:12.571 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:12.571 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:12.571 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1100 2024-05-08T08:54:12.571 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.571 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:12.571 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.571 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:12.571 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:54:12.571 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:12.571 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.571 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.571 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:12.571 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:54:12.572 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:12.572 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.572 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.572 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:12.572 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488430+0000", 2024-05-08T08:54:12.572 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000432676 2024-05-08T08:54:12.572 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.572 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.572 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:12.572 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488433+0000", 2024-05-08T08:54:12.572 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.45e-06 2024-05-08T08:54:12.572 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.572 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.572 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:12.572 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488442+0000", 2024-05-08T08:54:12.572 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.687e-06 2024-05-08T08:54:12.573 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.573 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.573 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:12.573 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488446+0000", 2024-05-08T08:54:12.573 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4939999999999997e-06 2024-05-08T08:54:12.573 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.573 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.573 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:12.573 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558756+0000", 2024-05-08T08:54:12.573 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070310051999999998 2024-05-08T08:54:12.573 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.573 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.573 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:12.573 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558844+0000", 2024-05-08T08:54:12.573 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7690999999999998e-05 2024-05-08T08:54:12.574 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.574 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.574 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:12.574 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558959+0000", 2024-05-08T08:54:12.574 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000115217 2024-05-08T08:54:12.574 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:12.574 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:12.574 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:12.574 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.574 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.574 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1103 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11599872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:12.574 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:54:12.574 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 96.014058775999999, 2024-05-08T08:54:12.574 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070777100999999995, 2024-05-08T08:54:12.575 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:12.575 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:12.575 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:12.575 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:12.575 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:12.575 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:12.575 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1103 2024-05-08T08:54:12.575 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.575 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:12.575 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.575 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:12.575 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:54:12.576 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:12.576 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.576 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.576 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:12.576 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:54:12.576 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:12.576 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.576 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.576 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:12.576 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488959+0000", 2024-05-08T08:54:12.576 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00050457200000000001 2024-05-08T08:54:12.576 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.576 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.577 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:12.577 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488969+0000", 2024-05-08T08:54:12.577 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.7159999999999998e-06 2024-05-08T08:54:12.577 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.577 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.577 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:12.577 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488981+0000", 2024-05-08T08:54:12.577 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2150000000000001e-05 2024-05-08T08:54:12.577 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.577 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.577 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:12.577 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488988+0000", 2024-05-08T08:54:12.577 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8759999999999999e-06 2024-05-08T08:54:12.578 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.578 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.578 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:12.578 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558992+0000", 2024-05-08T08:54:12.578 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070004358000000003 2024-05-08T08:54:12.578 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.578 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.578 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:12.578 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559093+0000", 2024-05-08T08:54:12.578 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000100299 2024-05-08T08:54:12.578 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.578 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.578 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:12.579 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559232+0000", 2024-05-08T08:54:12.579 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013913 2024-05-08T08:54:12.579 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:12.579 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:12.579 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:12.579 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.579 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.579 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1104 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11640832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:12.579 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:54:12.579 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 96.013513961000001, 2024-05-08T08:54:12.579 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070521947000000001, 2024-05-08T08:54:12.579 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:12.580 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:12.580 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:12.580 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:12.580 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:12.580 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:12.580 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1104 2024-05-08T08:54:12.580 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.580 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:12.580 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.580 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:12.580 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:54:12.580 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:12.581 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.581 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.581 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:12.581 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:54:12.581 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:12.581 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.581 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.581 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:12.581 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489700+0000", 2024-05-08T08:54:12.581 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00070067600000000001 2024-05-08T08:54:12.581 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.581 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.582 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:12.582 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489704+0000", 2024-05-08T08:54:12.582 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1640000000000003e-06 2024-05-08T08:54:12.582 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.582 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.582 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:12.582 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489716+0000", 2024-05-08T08:54:12.582 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1379000000000001e-05 2024-05-08T08:54:12.582 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.582 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.582 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:12.582 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489724+0000", 2024-05-08T08:54:12.583 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8949999999999998e-06 2024-05-08T08:54:12.583 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.583 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.583 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:12.583 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559278+0000", 2024-05-08T08:54:12.583 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.069554151999999994 2024-05-08T08:54:12.583 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.583 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.583 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:12.583 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559387+0000", 2024-05-08T08:54:12.583 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000109258 2024-05-08T08:54:12.583 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.583 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.583 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:12.584 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559521+0000", 2024-05-08T08:54:12.584 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000134423 2024-05-08T08:54:12.584 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:12.584 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:12.584 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:12.584 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.584 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.584 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:8636 3.17 3:e8a0a3e4:::rbd_header.11402b26385:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-05-08T08:54:12.584 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:54.150971+0000", 2024-05-08T08:54:12.584 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 78.351541967000003, 2024-05-08T08:54:12.584 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.217273233, 2024-05-08T08:54:12.585 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:12.585 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:12.585 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:12.585 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:12.585 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:54:12.585 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:54:12.585 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 8636 2024-05-08T08:54:12.585 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.585 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:12.585 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.585 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:12.585 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150971+0000", 2024-05-08T08:54:12.585 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:12.586 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.586 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.586 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:12.586 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150971+0000", 2024-05-08T08:54:12.586 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:12.586 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.586 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.586 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:12.586 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150972+0000", 2024-05-08T08:54:12.586 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.71e-07 2024-05-08T08:54:12.586 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.586 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.587 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:12.587 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150977+0000", 2024-05-08T08:54:12.587 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.152e-06 2024-05-08T08:54:12.587 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.587 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.587 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:12.587 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150990+0000", 2024-05-08T08:54:12.587 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2364999999999999e-05 2024-05-08T08:54:12.587 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.587 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.587 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:12.587 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150998+0000", 2024-05-08T08:54:12.587 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0739999999999993e-06 2024-05-08T08:54:12.588 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.588 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.588 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:12.588 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.151054+0000", 2024-05-08T08:54:12.588 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5949000000000003e-05 2024-05-08T08:54:12.588 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.588 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.588 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:54:12.588 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.151060+0000", 2024-05-08T08:54:12.588 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6120000000000004e-06 2024-05-08T08:54:12.588 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.588 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.588 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:12.589 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.367640+0000", 2024-05-08T08:54:12.589 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2165792739999999 2024-05-08T08:54:12.589 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.589 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.589 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:12.589 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.367839+0000", 2024-05-08T08:54:12.589 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000199185 2024-05-08T08:54:12.589 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.589 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.589 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:12.589 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.368245+0000", 2024-05-08T08:54:12.589 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00040575099999999998 2024-05-08T08:54:12.590 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:12.590 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:12.590 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:12.590 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.590 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.590 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5302.0:1516 3.c 3:319bf789:::rbd_data.1140d3fc54c.0000000000000025:head [stat,write 1892352~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:54:12.590 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:27.822220+0000", 2024-05-08T08:54:12.590 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 44.680293030000001, 2024-05-08T08:54:12.590 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.092643744, 2024-05-08T08:54:12.590 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:12.590 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:12.591 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:54:12.591 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:12.591 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5302", 2024-05-08T08:54:12.591 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/359277859", 2024-05-08T08:54:12.591 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1516 2024-05-08T08:54:12.591 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.591 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:12.591 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.591 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:12.591 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822220+0000", 2024-05-08T08:54:12.591 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:12.591 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.591 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.591 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:12.592 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822220+0000", 2024-05-08T08:54:12.592 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:12.592 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.592 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.592 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:12.592 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822221+0000", 2024-05-08T08:54:12.592 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3500000000000002e-07 2024-05-08T08:54:12.592 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.592 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.592 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:12.592 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822228+0000", 2024-05-08T08:54:12.592 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9959999999999996e-06 2024-05-08T08:54:12.592 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.593 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.593 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:12.593 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822234+0000", 2024-05-08T08:54:12.593 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5429999999999999e-06 2024-05-08T08:54:12.593 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.593 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.593 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:12.593 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822238+0000", 2024-05-08T08:54:12.593 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.732e-06 2024-05-08T08:54:12.593 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.593 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.593 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:12.593 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822285+0000", 2024-05-08T08:54:12.594 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6810000000000001e-05 2024-05-08T08:54:12.594 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.594 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.594 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:12.594 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822358+0000", 2024-05-08T08:54:12.594 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3473000000000006e-05 2024-05-08T08:54:12.594 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.594 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.594 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 5", 2024-05-08T08:54:12.594 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822489+0000", 2024-05-08T08:54:12.594 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013011199999999999 2024-05-08T08:54:12.594 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.594 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.594 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:54:12.595 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.825332+0000", 2024-05-08T08:54:12.595 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0028434799999999998 2024-05-08T08:54:12.595 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.595 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.595 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:54:12.595 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.914753+0000", 2024-05-08T08:54:12.595 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089421424999999999 2024-05-08T08:54:12.595 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.595 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.595 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:54:12.595 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.914807+0000", 2024-05-08T08:54:12.595 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3829000000000002e-05 2024-05-08T08:54:12.595 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.595 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.595 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:12.596 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.914864+0000", 2024-05-08T08:54:12.596 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6909e-05 2024-05-08T08:54:12.596 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:12.596 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:12.596 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:12.596 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.596 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.596 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:24567 3.6 3:66af1c40:::rbd_data.1140d3fc54c.000000000000005a:18 [read 2080768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:12.596 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.216598+0000", 2024-05-08T08:54:12.596 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 40.285914949999999, 2024-05-08T08:54:12.596 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071602647000000005, 2024-05-08T08:54:12.596 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:12.596 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:12.596 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:12.596 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:12.597 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:12.597 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:12.597 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 24567 2024-05-08T08:54:12.597 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.597 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:12.597 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.597 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:12.597 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216598+0000", 2024-05-08T08:54:12.597 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:12.597 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.597 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.597 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:12.597 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216598+0000", 2024-05-08T08:54:12.597 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:12.597 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.597 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.598 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:12.598 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216599+0000", 2024-05-08T08:54:12.598 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8999999999999997e-07 2024-05-08T08:54:12.598 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.598 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.598 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:12.598 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216601+0000", 2024-05-08T08:54:12.598 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9889999999999999e-06 2024-05-08T08:54:12.598 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.598 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.598 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:12.598 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216607+0000", 2024-05-08T08:54:12.598 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3600000000000001e-06 2024-05-08T08:54:12.598 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.598 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.599 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:12.599 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216612+0000", 2024-05-08T08:54:12.599 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.69e-06 2024-05-08T08:54:12.599 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.599 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.599 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:12.599 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.287076+0000", 2024-05-08T08:54:12.599 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070464355000000006 2024-05-08T08:54:12.599 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.599 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.599 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:12.599 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.287151+0000", 2024-05-08T08:54:12.599 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4924000000000003e-05 2024-05-08T08:54:12.599 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.599 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.599 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:12.600 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.288201+0000", 2024-05-08T08:54:12.600 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0010498389999999999 2024-05-08T08:54:12.600 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:12.600 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:12.600 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:12.600 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:12.600 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:12.600 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:12.602 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:54:12.602 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:12.604+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:12.702 INFO:teuthology.orchestra.run.smithi059.stderr:noscrub is set 2024-05-08T08:54:12.702 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:12.705+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:12.717 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:12.717 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:54:12.717 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:54:12.717 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:12.727 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:54:12.803 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:12.806+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:12.877 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:12.877 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:54:12.877 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:54:12.877 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:54:12.877 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:12.886 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:54:12.929 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:12.932+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:12.998 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:12.998 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:54:12.998 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:54:12.998 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:54:12.998 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.998 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:12443 3.0 3:02d0158e:::rbd_data.114071bcf828.0000000000000037:head [stat,delete] snapc 19=[19] RETRY=3 ondisk+retry+write+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:12.998 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.610027+0000", 2024-05-08T08:54:12.998 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 41.390096847999999, 2024-05-08T08:54:12.999 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.083586333999999998, 2024-05-08T08:54:12.999 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:12.999 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:12.999 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:12.999 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:12.999 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:12.999 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:12.999 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 12443 2024-05-08T08:54:12.999 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:12.999 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:12.999 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:12.999 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:12.999 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610027+0000", 2024-05-08T08:54:12.999 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:13.000 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.000 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.000 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:13.000 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610027+0000", 2024-05-08T08:54:13.000 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:13.000 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.000 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.000 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:13.000 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610028+0000", 2024-05-08T08:54:13.000 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.13e-07 2024-05-08T08:54:13.000 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.001 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.001 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:13.001 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610030+0000", 2024-05-08T08:54:13.001 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7669999999999999e-06 2024-05-08T08:54:13.001 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.001 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.001 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:13.001 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610039+0000", 2024-05-08T08:54:13.001 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4959999999999996e-06 2024-05-08T08:54:13.001 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.001 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.001 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:13.001 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610045+0000", 2024-05-08T08:54:13.002 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6779999999999997e-06 2024-05-08T08:54:13.002 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.002 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.002 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:13.002 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.660873+0000", 2024-05-08T08:54:13.002 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.050828748 2024-05-08T08:54:13.002 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.002 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.002 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:13.002 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.661017+0000", 2024-05-08T08:54:13.002 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014391999999999999 2024-05-08T08:54:13.002 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.003 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.003 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:13.003 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.693613+0000", 2024-05-08T08:54:13.003 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.032596111999999997 2024-05-08T08:54:13.003 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.003 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:13.003 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.003 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.003 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.003 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:14301 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [watch reconnect cookie 139837155901536 gen 12] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:13.003 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.645078+0000", 2024-05-08T08:54:13.003 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 41.355046209999998, 2024-05-08T08:54:13.003 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088281852999999993, 2024-05-08T08:54:13.003 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:13.004 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:13.004 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:13.004 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:13.004 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:13.004 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:13.004 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14301 2024-05-08T08:54:13.004 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.004 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:13.004 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.004 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:13.004 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645078+0000", 2024-05-08T08:54:13.004 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:13.004 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.005 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.005 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:13.005 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645078+0000", 2024-05-08T08:54:13.005 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:13.005 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.005 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.005 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:13.005 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645084+0000", 2024-05-08T08:54:13.005 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4620000000000001e-06 2024-05-08T08:54:13.005 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.005 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.005 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:13.006 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645088+0000", 2024-05-08T08:54:13.006 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8039999999999999e-06 2024-05-08T08:54:13.006 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.006 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.006 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:13.006 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645106+0000", 2024-05-08T08:54:13.006 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.8318999999999999e-05 2024-05-08T08:54:13.006 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.006 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.006 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:13.006 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645113+0000", 2024-05-08T08:54:13.006 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0310000000000001e-06 2024-05-08T08:54:13.007 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.007 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.007 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:13.007 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.670493+0000", 2024-05-08T08:54:13.007 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.025379993 2024-05-08T08:54:13.007 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.007 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.007 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:13.007 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.670571+0000", 2024-05-08T08:54:13.007 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7613000000000004e-05 2024-05-08T08:54:13.007 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.007 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.007 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:13.008 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733360+0000", 2024-05-08T08:54:13.008 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.062788630999999998 2024-05-08T08:54:13.008 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.008 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:13.008 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.008 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.008 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.008 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:14420 3.0 3:06a8948e:::rbd_data.114071bcf828.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:13.008 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.649639+0000", 2024-05-08T08:54:13.008 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 41.350485061999997, 2024-05-08T08:54:13.008 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.083851897999999994, 2024-05-08T08:54:13.009 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:13.009 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:13.009 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:54:13.009 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:13.009 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:13.009 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:13.009 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14420 2024-05-08T08:54:13.009 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.009 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:13.009 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.009 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:13.009 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649639+0000", 2024-05-08T08:54:13.009 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:13.010 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.010 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.010 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:13.010 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649639+0000", 2024-05-08T08:54:13.010 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:13.010 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.010 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.010 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:13.010 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649640+0000", 2024-05-08T08:54:13.010 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6899999999999997e-07 2024-05-08T08:54:13.010 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.010 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.011 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:13.011 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649895+0000", 2024-05-08T08:54:13.011 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00025531000000000002 2024-05-08T08:54:13.011 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.011 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.011 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:13.011 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649905+0000", 2024-05-08T08:54:13.011 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0198e-05 2024-05-08T08:54:13.011 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.011 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.011 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:13.011 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649911+0000", 2024-05-08T08:54:13.011 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2310000000000001e-06 2024-05-08T08:54:13.011 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.012 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.012 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:13.012 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.684099+0000", 2024-05-08T08:54:13.012 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.034188017000000001 2024-05-08T08:54:13.012 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.012 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.012 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:13.012 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.684172+0000", 2024-05-08T08:54:13.012 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3201999999999995e-05 2024-05-08T08:54:13.012 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.012 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.012 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 2", 2024-05-08T08:54:13.013 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.684418+0000", 2024-05-08T08:54:13.013 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00024559099999999997 2024-05-08T08:54:13.013 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.013 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.013 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:54:13.013 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.705538+0000", 2024-05-08T08:54:13.013 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.021119707000000001 2024-05-08T08:54:13.013 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.013 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.013 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:54:13.013 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733399+0000", 2024-05-08T08:54:13.013 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.027861543999999999 2024-05-08T08:54:13.013 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.014 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.014 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:54:13.014 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733445+0000", 2024-05-08T08:54:13.014 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5587000000000002e-05 2024-05-08T08:54:13.014 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.014 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.014 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:13.014 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733491+0000", 2024-05-08T08:54:13.014 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5841999999999999e-05 2024-05-08T08:54:13.014 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.014 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:13.014 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.015 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.015 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.015 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28547 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889179216] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:13.015 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.355654+0000", 2024-05-08T08:54:13.015 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 40.644469833999999, 2024-05-08T08:54:13.015 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078692261999999999, 2024-05-08T08:54:13.015 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:13.015 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:13.015 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:13.015 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:13.016 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:13.016 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:13.016 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28547 2024-05-08T08:54:13.016 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.016 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:13.016 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.016 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:13.016 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355654+0000", 2024-05-08T08:54:13.016 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:13.016 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.016 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.016 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:13.017 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355654+0000", 2024-05-08T08:54:13.017 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:13.017 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.017 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.017 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:13.017 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355654+0000", 2024-05-08T08:54:13.017 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3999999999999998e-07 2024-05-08T08:54:13.017 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.017 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.017 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:13.017 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355656+0000", 2024-05-08T08:54:13.017 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.353e-06 2024-05-08T08:54:13.018 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.018 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.018 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:13.018 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355663+0000", 2024-05-08T08:54:13.018 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2259999999999999e-06 2024-05-08T08:54:13.018 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.018 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.018 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:13.018 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355666+0000", 2024-05-08T08:54:13.018 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7930000000000002e-06 2024-05-08T08:54:13.018 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.018 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.018 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:13.019 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434117+0000", 2024-05-08T08:54:13.019 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078451231999999996 2024-05-08T08:54:13.019 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.019 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.019 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:13.019 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434198+0000", 2024-05-08T08:54:13.019 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0879999999999998e-05 2024-05-08T08:54:13.019 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.019 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.019 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:13.019 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434346+0000", 2024-05-08T08:54:13.019 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014853800000000001 2024-05-08T08:54:13.019 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.019 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:13.019 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.019 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.020 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.020 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28548 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889189696] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:13.020 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.355669+0000", 2024-05-08T08:54:13.020 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 40.644454484000001, 2024-05-08T08:54:13.020 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078956606999999998, 2024-05-08T08:54:13.020 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:13.020 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:13.020 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:13.020 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:13.020 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:13.020 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:13.020 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28548 2024-05-08T08:54:13.020 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.020 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:13.020 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.020 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:13.020 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355669+0000", 2024-05-08T08:54:13.021 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:13.021 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.021 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.021 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:13.021 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355669+0000", 2024-05-08T08:54:13.021 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:13.021 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.021 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.021 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:13.021 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355762+0000", 2024-05-08T08:54:13.021 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3015000000000003e-05 2024-05-08T08:54:13.021 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.021 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.021 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:13.021 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355765+0000", 2024-05-08T08:54:13.021 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.368e-06 2024-05-08T08:54:13.022 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.022 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.022 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:13.022 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355771+0000", 2024-05-08T08:54:13.022 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5989999999999998e-06 2024-05-08T08:54:13.022 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.022 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.022 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:13.022 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355775+0000", 2024-05-08T08:54:13.022 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7280000000000001e-06 2024-05-08T08:54:13.022 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.022 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.022 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:13.022 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434379+0000", 2024-05-08T08:54:13.022 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078603662000000005 2024-05-08T08:54:13.023 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.023 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.023 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:13.023 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434473+0000", 2024-05-08T08:54:13.023 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4104000000000006e-05 2024-05-08T08:54:13.023 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.023 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.023 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:13.023 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434626+0000", 2024-05-08T08:54:13.023 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00015313100000000001 2024-05-08T08:54:13.023 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.023 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:13.023 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.023 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.023 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.023 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28549 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889202400] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:13.024 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.355781+0000", 2024-05-08T08:54:13.024 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 40.644342909999999, 2024-05-08T08:54:13.024 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079117946999999994, 2024-05-08T08:54:13.024 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:13.024 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:13.024 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:13.024 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:13.024 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:13.024 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:13.024 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28549 2024-05-08T08:54:13.024 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.024 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:13.024 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.024 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:13.024 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355781+0000", 2024-05-08T08:54:13.025 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:13.025 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.025 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.025 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:13.025 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355781+0000", 2024-05-08T08:54:13.025 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:13.025 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.025 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.025 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:13.025 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356162+0000", 2024-05-08T08:54:13.025 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00038099100000000001 2024-05-08T08:54:13.025 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.025 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.026 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:13.026 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356165+0000", 2024-05-08T08:54:13.026 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1240000000000001e-06 2024-05-08T08:54:13.026 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.026 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.026 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:13.026 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356173+0000", 2024-05-08T08:54:13.026 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.6199999999999999e-06 2024-05-08T08:54:13.026 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.026 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.026 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:13.027 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356182+0000", 2024-05-08T08:54:13.027 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7760000000000003e-06 2024-05-08T08:54:13.027 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.027 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.027 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:13.027 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434675+0000", 2024-05-08T08:54:13.027 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078493119 2024-05-08T08:54:13.027 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.027 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.027 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:13.027 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434765+0000", 2024-05-08T08:54:13.027 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0402000000000006e-05 2024-05-08T08:54:13.027 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.028 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.028 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:13.028 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434899+0000", 2024-05-08T08:54:13.028 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013391499999999999 2024-05-08T08:54:13.028 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.028 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:13.028 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.028 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.028 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.028 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28550 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888597280] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:13.028 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:54:13.028 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 40.643931864000002, 2024-05-08T08:54:13.029 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078933990999999995, 2024-05-08T08:54:13.029 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:13.029 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:13.029 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:13.029 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:13.029 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:13.029 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:13.029 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28550 2024-05-08T08:54:13.029 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.029 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:13.029 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.029 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:13.029 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:54:13.030 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:13.030 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.030 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.030 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:13.030 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:54:13.030 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:13.030 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.030 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.030 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:13.030 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:54:13.030 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.34e-07 2024-05-08T08:54:13.031 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.031 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.031 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:13.031 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356195+0000", 2024-05-08T08:54:13.031 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8710000000000001e-06 2024-05-08T08:54:13.031 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.031 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.031 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:13.031 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356200+0000", 2024-05-08T08:54:13.031 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3000000000000003e-06 2024-05-08T08:54:13.031 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.032 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.032 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:13.032 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356203+0000", 2024-05-08T08:54:13.032 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4039999999999999e-06 2024-05-08T08:54:13.032 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.032 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.032 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:13.032 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434928+0000", 2024-05-08T08:54:13.032 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078725331999999995 2024-05-08T08:54:13.032 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.032 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.032 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:13.033 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435007+0000", 2024-05-08T08:54:13.033 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8415000000000001e-05 2024-05-08T08:54:13.033 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.033 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.033 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:13.033 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435126+0000", 2024-05-08T08:54:13.033 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011923500000000001 2024-05-08T08:54:13.033 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.033 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:13.033 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.033 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.033 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.033 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28551 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888527200] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:13.033 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.356210+0000", 2024-05-08T08:54:13.034 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 40.643913515999998, 2024-05-08T08:54:13.034 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079195072000000005, 2024-05-08T08:54:13.034 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:13.034 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:13.034 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:13.034 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:13.034 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:13.034 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:13.034 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28551 2024-05-08T08:54:13.034 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.034 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:13.034 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.034 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:13.034 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356210+0000", 2024-05-08T08:54:13.035 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:13.035 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.035 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.035 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:13.035 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356210+0000", 2024-05-08T08:54:13.035 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:13.035 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.035 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.035 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:13.035 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356211+0000", 2024-05-08T08:54:13.035 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4499999999999998e-07 2024-05-08T08:54:13.035 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.035 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.035 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:13.035 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356212+0000", 2024-05-08T08:54:13.036 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6029999999999999e-06 2024-05-08T08:54:13.036 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.036 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.036 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:13.036 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356221+0000", 2024-05-08T08:54:13.036 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4200000000000007e-06 2024-05-08T08:54:13.036 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.036 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.036 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:13.036 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356224+0000", 2024-05-08T08:54:13.036 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.038e-06 2024-05-08T08:54:13.036 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.036 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.037 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:13.037 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435178+0000", 2024-05-08T08:54:13.037 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078954625000000001 2024-05-08T08:54:13.037 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.037 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.037 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:13.037 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435256+0000", 2024-05-08T08:54:13.037 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7337999999999997e-05 2024-05-08T08:54:13.037 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.037 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.037 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:13.037 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435405+0000", 2024-05-08T08:54:13.038 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000149703 2024-05-08T08:54:13.038 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.038 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:13.038 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.038 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.038 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.038 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28552 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889231456] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:13.038 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.356228+0000", 2024-05-08T08:54:13.038 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 40.643895837999999, 2024-05-08T08:54:13.038 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079346920000000001, 2024-05-08T08:54:13.038 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:13.038 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:13.039 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:13.039 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:13.039 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:13.039 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:13.039 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28552 2024-05-08T08:54:13.039 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.039 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:13.039 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.039 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:13.039 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356228+0000", 2024-05-08T08:54:13.039 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:13.040 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.040 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.040 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:13.040 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356228+0000", 2024-05-08T08:54:13.040 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:13.040 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.040 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.040 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:13.040 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356959+0000", 2024-05-08T08:54:13.040 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00073097099999999996 2024-05-08T08:54:13.040 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.040 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.041 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:13.041 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356967+0000", 2024-05-08T08:54:13.041 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0409999999999998e-06 2024-05-08T08:54:13.041 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.041 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.041 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:13.041 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356979+0000", 2024-05-08T08:54:13.041 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1489e-05 2024-05-08T08:54:13.041 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.041 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.042 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:13.042 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356986+0000", 2024-05-08T08:54:13.042 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3699999999999997e-06 2024-05-08T08:54:13.042 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.042 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.042 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:13.042 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435433+0000", 2024-05-08T08:54:13.042 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078447520000000007 2024-05-08T08:54:13.042 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.042 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.042 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:13.042 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435484+0000", 2024-05-08T08:54:13.043 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0229000000000003e-05 2024-05-08T08:54:13.043 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.043 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.043 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:13.043 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435575+0000", 2024-05-08T08:54:13.043 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1299999999999997e-05 2024-05-08T08:54:13.043 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.043 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:13.043 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.043 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.043 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.044 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28553 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888713648] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:13.044 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:54:13.044 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 40.643121180000001, 2024-05-08T08:54:13.044 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078782034000000001, 2024-05-08T08:54:13.044 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:13.044 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:13.044 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:13.044 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:13.044 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:13.044 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:13.044 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28553 2024-05-08T08:54:13.044 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.044 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:13.045 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.045 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:13.045 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:54:13.045 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:13.045 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.045 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.045 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:13.045 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:54:13.045 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:13.045 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.045 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.045 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:13.045 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:54:13.046 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6600000000000002e-07 2024-05-08T08:54:13.046 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.046 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.046 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:13.046 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357006+0000", 2024-05-08T08:54:13.046 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7190000000000001e-06 2024-05-08T08:54:13.046 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.046 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.046 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:13.046 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357013+0000", 2024-05-08T08:54:13.046 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.435e-06 2024-05-08T08:54:13.046 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.046 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.046 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:13.047 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357018+0000", 2024-05-08T08:54:13.047 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7260000000000002e-06 2024-05-08T08:54:13.047 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.047 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.047 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:13.047 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435623+0000", 2024-05-08T08:54:13.047 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078604972999999995 2024-05-08T08:54:13.047 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.047 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.047 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:13.047 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435695+0000", 2024-05-08T08:54:13.047 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1959999999999995e-05 2024-05-08T08:54:13.047 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.047 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.048 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:13.048 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435785+0000", 2024-05-08T08:54:13.048 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.9754999999999997e-05 2024-05-08T08:54:13.048 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.048 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:13.048 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.048 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.048 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.048 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28554 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888980272] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:13.048 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357024+0000", 2024-05-08T08:54:13.048 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 40.643099544000002, 2024-05-08T08:54:13.048 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078999870999999999, 2024-05-08T08:54:13.048 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:13.048 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:13.048 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:13.049 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:13.049 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:13.049 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:13.049 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28554 2024-05-08T08:54:13.049 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.049 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:13.049 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.049 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:13.049 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357024+0000", 2024-05-08T08:54:13.049 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:13.049 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.049 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.049 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:13.049 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357024+0000", 2024-05-08T08:54:13.050 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:13.050 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.050 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.050 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:13.050 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357025+0000", 2024-05-08T08:54:13.050 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:54:13.050 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.050 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.050 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:13.050 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357027+0000", 2024-05-08T08:54:13.050 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2050000000000001e-06 2024-05-08T08:54:13.050 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.050 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.050 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:13.050 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357033+0000", 2024-05-08T08:54:13.050 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5679999999999996e-06 2024-05-08T08:54:13.051 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.051 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.051 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:13.051 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357038+0000", 2024-05-08T08:54:13.051 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2889999999999999e-06 2024-05-08T08:54:13.051 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.051 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.051 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:13.051 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435823+0000", 2024-05-08T08:54:13.051 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078785677999999998 2024-05-08T08:54:13.051 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.051 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.051 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:13.051 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435892+0000", 2024-05-08T08:54:13.051 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8379000000000004e-05 2024-05-08T08:54:13.052 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.052 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.052 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:13.052 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436024+0000", 2024-05-08T08:54:13.052 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013239799999999999 2024-05-08T08:54:13.052 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.052 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:13.052 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.052 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.052 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.052 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28555 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889001104] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:13.052 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:54:13.052 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 40.643080779000002, 2024-05-08T08:54:13.052 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079115742000000003, 2024-05-08T08:54:13.052 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:13.053 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:13.053 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:13.053 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:13.053 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:13.053 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:13.053 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28555 2024-05-08T08:54:13.053 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.053 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:13.053 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.053 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:13.053 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:54:13.053 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:13.053 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.053 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.053 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:13.053 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:54:13.054 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:13.054 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.054 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.054 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:13.054 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:54:13.054 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9200000000000002e-07 2024-05-08T08:54:13.054 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.054 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.054 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:13.054 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357046+0000", 2024-05-08T08:54:13.054 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1670000000000002e-06 2024-05-08T08:54:13.054 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.054 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.054 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:13.055 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357052+0000", 2024-05-08T08:54:13.055 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.139e-06 2024-05-08T08:54:13.055 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.055 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.055 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:13.055 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357056+0000", 2024-05-08T08:54:13.055 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2440000000000004e-06 2024-05-08T08:54:13.055 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.055 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.055 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:13.055 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436059+0000", 2024-05-08T08:54:13.055 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079003070999999994 2024-05-08T08:54:13.055 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.055 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.056 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:13.056 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436097+0000", 2024-05-08T08:54:13.056 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7784000000000003e-05 2024-05-08T08:54:13.056 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.056 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.056 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:13.056 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436159+0000", 2024-05-08T08:54:13.056 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2045000000000007e-05 2024-05-08T08:54:13.056 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.056 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:13.056 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.056 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.057 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.057 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28556 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889022064] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:13.057 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357065+0000", 2024-05-08T08:54:13.057 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 40.643058746000001, 2024-05-08T08:54:13.057 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079327976999999994, 2024-05-08T08:54:13.057 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:13.057 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:13.057 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:13.057 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:13.057 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:13.057 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:13.058 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28556 2024-05-08T08:54:13.058 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.058 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:13.058 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.058 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:13.058 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357065+0000", 2024-05-08T08:54:13.058 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:13.058 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.058 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.058 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:13.059 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357065+0000", 2024-05-08T08:54:13.059 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:13.059 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.059 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.059 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:13.059 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358958+0000", 2024-05-08T08:54:13.059 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001892915 2024-05-08T08:54:13.059 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.059 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.059 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:13.059 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358962+0000", 2024-05-08T08:54:13.059 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4599999999999999e-06 2024-05-08T08:54:13.059 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.060 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.060 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:13.060 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358970+0000", 2024-05-08T08:54:13.060 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.6090000000000001e-06 2024-05-08T08:54:13.060 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.060 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.060 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:13.060 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358975+0000", 2024-05-08T08:54:13.060 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7639999999999996e-06 2024-05-08T08:54:13.060 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.060 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.061 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:13.061 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436196+0000", 2024-05-08T08:54:13.061 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.077221564000000006 2024-05-08T08:54:13.061 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.061 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.061 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:13.061 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436269+0000", 2024-05-08T08:54:13.061 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2720999999999997e-05 2024-05-08T08:54:13.061 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.061 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.061 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:13.062 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436393+0000", 2024-05-08T08:54:13.062 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012394399999999999 2024-05-08T08:54:13.062 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.062 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:13.062 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.062 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.062 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.062 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28563 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888855520] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:13.062 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:54:13.062 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 40.641049346999999, 2024-05-08T08:54:13.062 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078795566999999997, 2024-05-08T08:54:13.063 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:13.063 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:13.063 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:13.063 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:13.063 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:13.063 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:13.063 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28563 2024-05-08T08:54:13.063 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.063 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:13.063 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.063 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:13.063 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:54:13.064 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:13.064 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.064 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.064 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:13.064 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:54:13.064 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:13.064 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.064 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.064 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:13.064 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:54:13.064 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1e-07 2024-05-08T08:54:13.065 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.065 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.065 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:13.065 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359080+0000", 2024-05-08T08:54:13.065 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8940000000000001e-06 2024-05-08T08:54:13.065 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.065 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.065 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:13.065 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359083+0000", 2024-05-08T08:54:13.065 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1949999999999999e-06 2024-05-08T08:54:13.065 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.066 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.066 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:13.066 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359085+0000", 2024-05-08T08:54:13.066 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4710000000000001e-06 2024-05-08T08:54:13.066 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.066 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.066 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:13.066 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437604+0000", 2024-05-08T08:54:13.066 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078518278999999996 2024-05-08T08:54:13.066 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.066 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.066 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:13.067 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437704+0000", 2024-05-08T08:54:13.067 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010010599999999999 2024-05-08T08:54:13.067 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.067 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.067 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:13.067 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437870+0000", 2024-05-08T08:54:13.067 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00016641200000000001 2024-05-08T08:54:13.067 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.067 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:13.067 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.067 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.068 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.068 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28564 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888538944] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:13.068 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.359089+0000", 2024-05-08T08:54:13.068 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 40.641035131999999, 2024-05-08T08:54:13.068 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078952675, 2024-05-08T08:54:13.068 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:13.068 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:13.068 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:13.068 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:13.068 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:13.068 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:13.068 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28564 2024-05-08T08:54:13.068 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.069 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:13.069 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.069 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:13.069 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359089+0000", 2024-05-08T08:54:13.069 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:13.069 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.069 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.069 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:13.069 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359089+0000", 2024-05-08T08:54:13.069 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:13.069 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.069 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.069 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:13.070 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360959+0000", 2024-05-08T08:54:13.070 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0018707089999999999 2024-05-08T08:54:13.070 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.070 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.070 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:13.070 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360963+0000", 2024-05-08T08:54:13.070 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0680000000000001e-06 2024-05-08T08:54:13.070 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.070 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.070 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:13.071 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360969+0000", 2024-05-08T08:54:13.071 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1920000000000002e-06 2024-05-08T08:54:13.071 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.071 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.071 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:13.071 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360973+0000", 2024-05-08T08:54:13.071 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.031e-06 2024-05-08T08:54:13.071 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.071 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.071 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:13.071 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437916+0000", 2024-05-08T08:54:13.071 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.076942939000000002 2024-05-08T08:54:13.071 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.072 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.072 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:13.072 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437973+0000", 2024-05-08T08:54:13.072 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7358999999999997e-05 2024-05-08T08:54:13.072 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.072 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.072 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:13.072 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.438041+0000", 2024-05-08T08:54:13.072 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8376999999999999e-05 2024-05-08T08:54:13.072 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.073 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:13.073 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.073 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.073 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.073 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28574 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889427600] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:13.073 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:54:13.073 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 40.639030624999997, 2024-05-08T08:54:13.073 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078782476000000004, 2024-05-08T08:54:13.073 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:13.073 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:13.074 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:13.074 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:13.074 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:13.074 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:13.074 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28574 2024-05-08T08:54:13.074 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.074 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:13.074 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.074 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:13.074 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:54:13.074 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:13.074 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.074 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.075 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:13.075 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:54:13.075 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:13.075 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.075 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.075 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:13.075 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:54:13.075 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1799999999999999e-07 2024-05-08T08:54:13.075 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.075 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.075 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:13.075 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361095+0000", 2024-05-08T08:54:13.075 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3850000000000001e-06 2024-05-08T08:54:13.076 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.076 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.076 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:13.076 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361098+0000", 2024-05-08T08:54:13.076 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7989999999999998e-06 2024-05-08T08:54:13.076 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.076 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.076 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:13.076 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361100+0000", 2024-05-08T08:54:13.076 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4669999999999998e-06 2024-05-08T08:54:13.076 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.076 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.076 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:13.076 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439768+0000", 2024-05-08T08:54:13.076 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078668057999999999 2024-05-08T08:54:13.077 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.077 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.077 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:13.077 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439811+0000", 2024-05-08T08:54:13.077 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2459000000000001e-05 2024-05-08T08:54:13.077 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.077 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.077 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:13.077 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439876+0000", 2024-05-08T08:54:13.077 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5090000000000002e-05 2024-05-08T08:54:13.077 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.077 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:13.077 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.077 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.077 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.078 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28575 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889439216] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:13.078 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361103+0000", 2024-05-08T08:54:13.078 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 40.639020610000003, 2024-05-08T08:54:13.078 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078966828000000003, 2024-05-08T08:54:13.078 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:13.078 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:13.078 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:13.078 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:13.078 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:13.078 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:13.078 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28575 2024-05-08T08:54:13.078 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.079 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:13.079 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.079 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:13.079 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361103+0000", 2024-05-08T08:54:13.079 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:13.079 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.079 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.079 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:13.079 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361103+0000", 2024-05-08T08:54:13.079 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:13.079 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.079 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.079 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:13.079 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361104+0000", 2024-05-08T08:54:13.080 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1400000000000001e-07 2024-05-08T08:54:13.080 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.080 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.080 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:13.080 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361105+0000", 2024-05-08T08:54:13.080 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5540000000000001e-06 2024-05-08T08:54:13.080 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.080 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.080 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:13.080 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361108+0000", 2024-05-08T08:54:13.080 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.841e-06 2024-05-08T08:54:13.080 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.080 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.081 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:13.081 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361110+0000", 2024-05-08T08:54:13.081 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.497e-06 2024-05-08T08:54:13.081 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.081 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.081 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:13.081 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439916+0000", 2024-05-08T08:54:13.081 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078805372999999998 2024-05-08T08:54:13.081 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.081 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.081 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:13.081 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440000+0000", 2024-05-08T08:54:13.081 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4730999999999999e-05 2024-05-08T08:54:13.081 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.081 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.082 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:13.082 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440070+0000", 2024-05-08T08:54:13.082 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9617999999999997e-05 2024-05-08T08:54:13.082 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.082 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:13.082 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.082 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.082 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.082 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28576 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889451328] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:13.082 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361113+0000", 2024-05-08T08:54:13.082 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 40.639010423999999, 2024-05-08T08:54:13.082 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079088862999999995, 2024-05-08T08:54:13.082 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:13.083 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:13.083 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:13.083 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:13.083 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:13.083 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:13.083 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28576 2024-05-08T08:54:13.083 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.083 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:13.083 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.083 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:13.083 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361113+0000", 2024-05-08T08:54:13.083 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:13.083 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.083 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.083 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:13.083 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361113+0000", 2024-05-08T08:54:13.084 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:13.084 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.084 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.084 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:13.084 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361114+0000", 2024-05-08T08:54:13.084 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1299999999999999e-07 2024-05-08T08:54:13.084 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.084 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.084 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:13.084 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361115+0000", 2024-05-08T08:54:13.084 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.4300000000000001e-06 2024-05-08T08:54:13.084 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.084 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.085 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:13.085 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361118+0000", 2024-05-08T08:54:13.085 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9000000000000002e-06 2024-05-08T08:54:13.085 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.085 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.085 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:13.085 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361120+0000", 2024-05-08T08:54:13.085 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4739999999999999e-06 2024-05-08T08:54:13.085 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.085 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.085 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:13.085 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440096+0000", 2024-05-08T08:54:13.085 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078975629000000006 2024-05-08T08:54:13.085 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.085 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.086 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:13.086 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440135+0000", 2024-05-08T08:54:13.086 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9202000000000002e-05 2024-05-08T08:54:13.086 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.086 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.086 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:13.086 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440202+0000", 2024-05-08T08:54:13.086 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7014999999999994e-05 2024-05-08T08:54:13.086 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.086 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:13.086 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.086 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.086 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.087 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28577 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889462640] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:13.087 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:54:13.087 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 40.639000242000002, 2024-05-08T08:54:13.087 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079194622000000006, 2024-05-08T08:54:13.087 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:13.087 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:13.087 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:13.087 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:13.087 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:13.087 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:13.087 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28577 2024-05-08T08:54:13.087 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.087 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:13.087 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.087 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:13.087 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:54:13.088 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:13.088 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.088 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.088 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:13.088 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:54:13.088 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:13.088 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.088 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.088 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:13.088 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:54:13.088 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1400000000000001e-07 2024-05-08T08:54:13.089 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.089 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.089 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:13.089 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361125+0000", 2024-05-08T08:54:13.089 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.372e-06 2024-05-08T08:54:13.089 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.089 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.089 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:13.089 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361128+0000", 2024-05-08T08:54:13.089 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0309999999999999e-06 2024-05-08T08:54:13.089 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.089 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.090 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:13.090 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361131+0000", 2024-05-08T08:54:13.090 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4040000000000002e-06 2024-05-08T08:54:13.090 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.090 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.090 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:13.090 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440225+0000", 2024-05-08T08:54:13.090 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079094600000000001 2024-05-08T08:54:13.090 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.090 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.090 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:13.091 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440260+0000", 2024-05-08T08:54:13.091 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4703000000000003e-05 2024-05-08T08:54:13.091 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.091 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.091 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:13.091 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440318+0000", 2024-05-08T08:54:13.091 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8298000000000003e-05 2024-05-08T08:54:13.091 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.091 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:13.091 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.091 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.091 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.092 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28578 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889474192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:13.092 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361134+0000", 2024-05-08T08:54:13.092 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 40.638989817000002, 2024-05-08T08:54:13.092 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079310295000000003, 2024-05-08T08:54:13.092 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:13.092 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:13.092 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:13.092 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:13.092 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:13.092 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:13.092 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28578 2024-05-08T08:54:13.092 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.092 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:13.093 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.093 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:13.093 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361134+0000", 2024-05-08T08:54:13.093 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:13.093 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.093 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.093 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:13.093 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361134+0000", 2024-05-08T08:54:13.093 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:13.093 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.093 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.093 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:13.093 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.362958+0000", 2024-05-08T08:54:13.094 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0018237380000000001 2024-05-08T08:54:13.094 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.094 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.094 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:13.094 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.362961+0000", 2024-05-08T08:54:13.094 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.123e-06 2024-05-08T08:54:13.094 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.094 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.094 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:13.094 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.397404+0000", 2024-05-08T08:54:13.094 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.034443226 2024-05-08T08:54:13.095 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.095 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.095 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:13.095 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.397439+0000", 2024-05-08T08:54:13.095 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4690000000000002e-05 2024-05-08T08:54:13.095 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.095 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.095 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:13.095 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440340+0000", 2024-05-08T08:54:13.095 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.042901419000000003 2024-05-08T08:54:13.095 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.095 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.096 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:13.096 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440376+0000", 2024-05-08T08:54:13.096 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5444999999999997e-05 2024-05-08T08:54:13.096 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.096 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.096 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:13.096 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440444+0000", 2024-05-08T08:54:13.096 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8653999999999997e-05 2024-05-08T08:54:13.096 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.096 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:13.096 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.096 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.096 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:13.097 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:13.098 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:54:13.099 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:13.100+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:13.199 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:13.202+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:13.251 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:13.251 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:54:13.251 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:54:13.251 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:13.256 INFO:teuthology.orchestra.run.smithi196.stdout:checking clone #38, image image_client.0-clone38 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent39 2024-05-08T08:54:13.260 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:54:13.300 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:13.303+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:13.372 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:13.372 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:54:13.372 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:54:13.372 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:54:13.372 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:13.381 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:54:13.390 INFO:teuthology.orchestra.run.smithi196.stdout:5422 write 0xe632400 thru 0xe6399d0 (0x75d1 bytes) 2024-05-08T08:54:13.401 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:13.404+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:13.405 INFO:teuthology.orchestra.run.smithi196.stdout:5425 trunc from 0xe639a00 to 0x2161000 2024-05-08T08:54:13.471 INFO:teuthology.orchestra.run.smithi196.stdout:5426 read 0xf9ce00 thru 0xfa5f77 (0x9178 bytes) 2024-05-08T08:54:13.473 INFO:teuthology.orchestra.run.smithi196.stdout:5427 read 0x1e2ca00 thru 0x1e3917f (0xc780 bytes) 2024-05-08T08:54:13.489 INFO:teuthology.orchestra.run.smithi196.stdout:5430 write 0x4195200 thru 0x4195ba3 (0x9a4 bytes) 2024-05-08T08:54:13.491 INFO:teuthology.orchestra.run.smithi196.stdout:skipping zero length punch hole 2024-05-08T08:54:13.491 INFO:teuthology.orchestra.run.smithi196.stdout:5432 read 0xe74600 thru 0xe7f819 (0xb21a bytes) 2024-05-08T08:54:13.501 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:13.504+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:13.510 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:13.510 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:54:13.510 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:54:13.510 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:54:13.510 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.510 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:437 3.17 3:ea93a512:::rbd_data.1140100c9703.0000000000000077:head [set-alloc-hint object_size 2097152 write_size 2097152,write 1170432~1024] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e63)", 2024-05-08T08:54:13.510 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:08.390804+0000", 2024-05-08T08:54:13.510 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 65.121257712000002, 2024-05-08T08:54:13.511 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0406068690000001, 2024-05-08T08:54:13.511 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:13.511 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:13.511 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:54:13.511 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:13.511 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:13.511 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:13.511 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 437 2024-05-08T08:54:13.511 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.511 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:13.511 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.512 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:13.512 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390804+0000", 2024-05-08T08:54:13.512 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:13.512 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.512 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.512 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:13.512 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390804+0000", 2024-05-08T08:54:13.512 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:13.512 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.512 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.512 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:13.512 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390806+0000", 2024-05-08T08:54:13.513 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.666e-06 2024-05-08T08:54:13.513 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.513 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.513 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:13.513 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390818+0000", 2024-05-08T08:54:13.513 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2187999999999999e-05 2024-05-08T08:54:13.513 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.513 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.513 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:13.513 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390834+0000", 2024-05-08T08:54:13.513 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6191000000000001e-05 2024-05-08T08:54:13.513 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.514 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.514 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:13.514 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390845+0000", 2024-05-08T08:54:13.514 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0831e-05 2024-05-08T08:54:13.514 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.514 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.514 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:13.514 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390922+0000", 2024-05-08T08:54:13.514 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7253e-05 2024-05-08T08:54:13.514 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.514 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.515 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op must wait for map", 2024-05-08T08:54:13.515 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390930+0000", 2024-05-08T08:54:13.515 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8259999999999993e-06 2024-05-08T08:54:13.515 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.515 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.515 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:13.515 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392155+0000", 2024-05-08T08:54:13.515 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0012256649999999999 2024-05-08T08:54:13.515 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.515 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.515 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:54:13.515 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392165+0000", 2024-05-08T08:54:13.515 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5799999999999998e-06 2024-05-08T08:54:13.515 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.516 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.516 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:13.516 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.429596+0000", 2024-05-08T08:54:13.516 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0374313639999999 2024-05-08T08:54:13.516 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.516 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.516 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:13.516 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.429728+0000", 2024-05-08T08:54:13.516 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000131552 2024-05-08T08:54:13.516 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.516 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.516 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 0,1", 2024-05-08T08:54:13.516 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.429860+0000", 2024-05-08T08:54:13.517 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000132137 2024-05-08T08:54:13.517 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.517 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.517 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:54:13.517 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.430811+0000", 2024-05-08T08:54:13.517 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00095040200000000004 2024-05-08T08:54:13.517 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.517 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.517 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:54:13.517 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431126+0000", 2024-05-08T08:54:13.517 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000315006 2024-05-08T08:54:13.517 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.517 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.517 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:54:13.517 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431272+0000", 2024-05-08T08:54:13.517 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000146487 2024-05-08T08:54:13.518 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.518 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.518 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:54:13.518 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431315+0000", 2024-05-08T08:54:13.518 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2697000000000001e-05 2024-05-08T08:54:13.518 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.518 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.518 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:13.518 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431411+0000", 2024-05-08T08:54:13.518 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6024000000000001e-05 2024-05-08T08:54:13.518 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.518 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:13.518 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.518 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.519 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.519 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:439 3.17 3:ea93a512:::rbd_data.1140100c9703.0000000000000077:head [read 1163264~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-05-08T08:54:13.519 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:08.390871+0000", 2024-05-08T08:54:13.519 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 65.121190201000005, 2024-05-08T08:54:13.519 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0408790590000001, 2024-05-08T08:54:13.519 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:13.519 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:13.519 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:13.519 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:13.519 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:13.519 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:13.519 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 439 2024-05-08T08:54:13.519 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.519 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:13.519 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.520 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:13.520 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390871+0000", 2024-05-08T08:54:13.520 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:13.520 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.520 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.520 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:13.520 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390871+0000", 2024-05-08T08:54:13.520 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:13.520 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.520 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.520 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:13.520 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390872+0000", 2024-05-08T08:54:13.520 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6300000000000005e-07 2024-05-08T08:54:13.520 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.520 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.520 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:13.521 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390875+0000", 2024-05-08T08:54:13.521 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3459999999999998e-06 2024-05-08T08:54:13.521 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.521 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.521 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:13.521 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390891+0000", 2024-05-08T08:54:13.521 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5255e-05 2024-05-08T08:54:13.521 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.521 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.521 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:13.521 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390897+0000", 2024-05-08T08:54:13.521 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2360000000000001e-06 2024-05-08T08:54:13.521 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.521 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.521 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:13.521 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390995+0000", 2024-05-08T08:54:13.522 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8522000000000003e-05 2024-05-08T08:54:13.522 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.522 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.522 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting_for_map not empty", 2024-05-08T08:54:13.522 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.391005+0000", 2024-05-08T08:54:13.522 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1460000000000002e-06 2024-05-08T08:54:13.522 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.522 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.522 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:13.522 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392191+0000", 2024-05-08T08:54:13.522 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0011860270000000001 2024-05-08T08:54:13.522 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.522 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.523 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:54:13.523 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392197+0000", 2024-05-08T08:54:13.523 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4289999999999998e-06 2024-05-08T08:54:13.523 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.523 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.523 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:13.523 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.430596+0000", 2024-05-08T08:54:13.523 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0383989090000001 2024-05-08T08:54:13.523 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.523 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.523 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for rw locks", 2024-05-08T08:54:13.523 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.430676+0000", 2024-05-08T08:54:13.523 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9781000000000001e-05 2024-05-08T08:54:13.523 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.523 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.524 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:13.524 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431460+0000", 2024-05-08T08:54:13.524 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00078427300000000004 2024-05-08T08:54:13.524 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.524 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.524 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:13.524 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431540+0000", 2024-05-08T08:54:13.524 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0224999999999998e-05 2024-05-08T08:54:13.524 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.524 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.524 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:13.524 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431750+0000", 2024-05-08T08:54:13.524 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00021024699999999999 2024-05-08T08:54:13.525 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.525 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:13.525 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.525 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.525 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.525 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:945 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 4096000~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:13.525 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:54:13.525 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 64.061040633999994, 2024-05-08T08:54:13.525 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086774741000000002, 2024-05-08T08:54:13.525 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:13.525 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:13.525 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:13.525 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:13.525 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:13.526 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:13.526 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 945 2024-05-08T08:54:13.526 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.526 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:13.526 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.526 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:13.526 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:54:13.526 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:13.526 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.526 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.526 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:13.526 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:54:13.527 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:13.527 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.527 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.527 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:13.527 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:54:13.527 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3499999999999996e-07 2024-05-08T08:54:13.527 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.527 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.527 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:13.527 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451024+0000", 2024-05-08T08:54:13.527 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.627e-06 2024-05-08T08:54:13.527 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.528 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.528 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:13.528 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451037+0000", 2024-05-08T08:54:13.528 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2431e-05 2024-05-08T08:54:13.528 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.528 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.528 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:13.528 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451042+0000", 2024-05-08T08:54:13.528 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8409999999999996e-06 2024-05-08T08:54:13.528 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.528 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.528 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:13.528 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537608+0000", 2024-05-08T08:54:13.528 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086565243 2024-05-08T08:54:13.529 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.529 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.529 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:13.529 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537687+0000", 2024-05-08T08:54:13.529 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9519999999999998e-05 2024-05-08T08:54:13.529 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.529 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.529 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:13.529 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537796+0000", 2024-05-08T08:54:13.529 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000108544 2024-05-08T08:54:13.529 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.529 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:13.529 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.529 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.530 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.530 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:964 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 458752~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:13.530 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451051+0000", 2024-05-08T08:54:13.530 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 64.061010460999995, 2024-05-08T08:54:13.530 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086990490000000004, 2024-05-08T08:54:13.530 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:13.530 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:13.530 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:13.530 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:13.530 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:13.530 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:13.530 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 964 2024-05-08T08:54:13.530 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.530 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:13.531 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.531 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:13.531 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451051+0000", 2024-05-08T08:54:13.531 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:13.531 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.531 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.531 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:13.531 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451051+0000", 2024-05-08T08:54:13.531 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:13.531 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.531 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.531 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:13.531 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451052+0000", 2024-05-08T08:54:13.532 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7299999999999996e-07 2024-05-08T08:54:13.532 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.532 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.532 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:13.532 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451055+0000", 2024-05-08T08:54:13.532 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.2770000000000001e-06 2024-05-08T08:54:13.532 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.532 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.532 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:13.532 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451061+0000", 2024-05-08T08:54:13.532 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4060000000000002e-06 2024-05-08T08:54:13.532 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.533 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.533 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:13.533 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451066+0000", 2024-05-08T08:54:13.533 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0839999999999999e-06 2024-05-08T08:54:13.533 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.533 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.533 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:13.533 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537839+0000", 2024-05-08T08:54:13.533 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086772962999999995 2024-05-08T08:54:13.533 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.533 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.534 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:13.534 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537934+0000", 2024-05-08T08:54:13.534 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4742999999999997e-05 2024-05-08T08:54:13.534 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.534 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.534 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:13.534 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538042+0000", 2024-05-08T08:54:13.534 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000107444 2024-05-08T08:54:13.534 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.534 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:13.534 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.534 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.535 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.535 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:965 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 950272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:13.535 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451075+0000", 2024-05-08T08:54:13.535 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 64.060986399000001, 2024-05-08T08:54:13.535 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087230196999999995, 2024-05-08T08:54:13.535 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:13.535 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:13.535 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:13.535 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:13.535 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:13.535 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:13.536 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 965 2024-05-08T08:54:13.536 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.536 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:13.536 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.536 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:13.536 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451075+0000", 2024-05-08T08:54:13.536 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:13.536 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.536 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.536 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:13.536 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451075+0000", 2024-05-08T08:54:13.537 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:13.537 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.537 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.537 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:13.537 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451076+0000", 2024-05-08T08:54:13.537 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5000000000000002e-07 2024-05-08T08:54:13.537 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.537 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.537 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:13.537 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451078+0000", 2024-05-08T08:54:13.537 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.5639999999999999e-06 2024-05-08T08:54:13.537 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.538 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.538 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:13.538 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451085+0000", 2024-05-08T08:54:13.538 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5640000000000002e-06 2024-05-08T08:54:13.538 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.538 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.538 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:13.538 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451090+0000", 2024-05-08T08:54:13.538 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1699999999999996e-06 2024-05-08T08:54:13.538 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.538 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.539 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:13.539 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538106+0000", 2024-05-08T08:54:13.539 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087015888999999999 2024-05-08T08:54:13.539 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.539 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.539 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:13.539 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538190+0000", 2024-05-08T08:54:13.539 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3961999999999993e-05 2024-05-08T08:54:13.539 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.539 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.539 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:13.540 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538305+0000", 2024-05-08T08:54:13.540 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011529800000000001 2024-05-08T08:54:13.540 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.540 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:13.540 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.540 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.540 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.540 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:966 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 1441792~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:13.540 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:54:13.540 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 64.060963025999996, 2024-05-08T08:54:13.540 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087446323000000006, 2024-05-08T08:54:13.541 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:13.541 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:13.541 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:13.541 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:13.541 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:13.541 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:13.541 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 966 2024-05-08T08:54:13.541 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.541 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:13.541 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.541 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:13.541 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:54:13.541 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:13.542 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.542 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.542 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:13.542 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:54:13.542 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:13.542 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.542 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.542 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:13.542 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:54:13.542 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3700000000000001e-07 2024-05-08T08:54:13.543 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.543 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.543 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:13.543 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451101+0000", 2024-05-08T08:54:13.543 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2560000000000001e-06 2024-05-08T08:54:13.543 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.543 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.543 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:13.543 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451107+0000", 2024-05-08T08:54:13.543 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1070000000000001e-06 2024-05-08T08:54:13.543 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.543 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.544 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:13.544 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451112+0000", 2024-05-08T08:54:13.544 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1420000000000001e-06 2024-05-08T08:54:13.544 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.544 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.544 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:13.544 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538341+0000", 2024-05-08T08:54:13.544 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087228302999999993 2024-05-08T08:54:13.544 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.544 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.544 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:13.544 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538424+0000", 2024-05-08T08:54:13.544 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3497000000000004e-05 2024-05-08T08:54:13.545 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.545 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.545 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:13.545 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538545+0000", 2024-05-08T08:54:13.545 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000120581 2024-05-08T08:54:13.545 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.545 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:13.545 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.545 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.545 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.545 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:967 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:13.545 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451120+0000", 2024-05-08T08:54:13.545 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 64.060941550999999, 2024-05-08T08:54:13.546 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087632725999999994, 2024-05-08T08:54:13.546 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:13.546 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:13.546 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:13.546 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:13.546 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:13.546 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:13.546 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 967 2024-05-08T08:54:13.546 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.546 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:13.546 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.546 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:13.546 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451120+0000", 2024-05-08T08:54:13.546 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:13.547 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.547 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.547 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:13.547 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451120+0000", 2024-05-08T08:54:13.547 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:13.547 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.547 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.547 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:13.547 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451121+0000", 2024-05-08T08:54:13.547 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8800000000000002e-07 2024-05-08T08:54:13.547 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.547 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.547 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:13.547 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451130+0000", 2024-05-08T08:54:13.547 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2299999999999997e-06 2024-05-08T08:54:13.547 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.548 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.548 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:13.548 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451137+0000", 2024-05-08T08:54:13.548 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4669999999999998e-06 2024-05-08T08:54:13.548 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.548 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.548 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:13.548 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451143+0000", 2024-05-08T08:54:13.548 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2839999999999997e-06 2024-05-08T08:54:13.548 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.548 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.548 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:13.548 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538575+0000", 2024-05-08T08:54:13.548 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087432789999999996 2024-05-08T08:54:13.548 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.549 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.549 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:13.549 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538648+0000", 2024-05-08T08:54:13.549 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2877e-05 2024-05-08T08:54:13.549 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.549 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.549 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:13.549 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538753+0000", 2024-05-08T08:54:13.549 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010449 2024-05-08T08:54:13.549 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.549 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:13.549 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.549 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.549 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.549 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:968 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 2424832~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:13.550 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451152+0000", 2024-05-08T08:54:13.550 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 64.060909465999998, 2024-05-08T08:54:13.550 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087813325999999997, 2024-05-08T08:54:13.550 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:13.550 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:13.550 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:13.550 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:13.550 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:13.550 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:13.550 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 968 2024-05-08T08:54:13.550 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.550 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:13.550 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.550 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:13.550 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451152+0000", 2024-05-08T08:54:13.551 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:13.551 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.551 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.551 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:13.551 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451152+0000", 2024-05-08T08:54:13.551 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:13.551 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.551 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.551 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:13.551 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451153+0000", 2024-05-08T08:54:13.551 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3099999999999998e-07 2024-05-08T08:54:13.551 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.551 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.551 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:13.551 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451155+0000", 2024-05-08T08:54:13.552 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4760000000000001e-06 2024-05-08T08:54:13.552 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.552 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.552 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:13.552 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451162+0000", 2024-05-08T08:54:13.552 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5969999999999997e-06 2024-05-08T08:54:13.552 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.552 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.552 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:13.552 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451167+0000", 2024-05-08T08:54:13.552 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.977e-06 2024-05-08T08:54:13.552 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.552 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.552 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:13.552 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538781+0000", 2024-05-08T08:54:13.553 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087614846999999996 2024-05-08T08:54:13.553 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.553 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.553 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:13.553 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538857+0000", 2024-05-08T08:54:13.553 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5358000000000006e-05 2024-05-08T08:54:13.553 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.553 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.553 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:13.553 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538965+0000", 2024-05-08T08:54:13.553 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010864 2024-05-08T08:54:13.553 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.553 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:13.553 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.554 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.554 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.554 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:969 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 2916352~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:13.554 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:54:13.554 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 64.060887761000004, 2024-05-08T08:54:13.554 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087998980000000004, 2024-05-08T08:54:13.554 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:13.554 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:13.554 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:13.554 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:13.554 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:13.554 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:13.554 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 969 2024-05-08T08:54:13.555 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.555 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:13.555 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.555 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:13.555 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:54:13.555 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:13.555 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.555 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.555 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:13.555 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:54:13.555 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:13.555 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.556 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.556 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:13.556 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:54:13.556 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3599999999999999e-07 2024-05-08T08:54:13.556 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.556 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.556 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:13.556 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451177+0000", 2024-05-08T08:54:13.556 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4590000000000001e-06 2024-05-08T08:54:13.556 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.556 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.556 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:13.557 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451184+0000", 2024-05-08T08:54:13.557 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4109999999999999e-06 2024-05-08T08:54:13.557 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.557 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.557 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:13.557 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451191+0000", 2024-05-08T08:54:13.557 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7279999999999998e-06 2024-05-08T08:54:13.557 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.557 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.557 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:13.557 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538997+0000", 2024-05-08T08:54:13.557 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087806170000000003 2024-05-08T08:54:13.558 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.558 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.558 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:13.558 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539069+0000", 2024-05-08T08:54:13.558 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2373999999999995e-05 2024-05-08T08:54:13.558 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.558 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.558 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:13.558 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539173+0000", 2024-05-08T08:54:13.558 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000103402 2024-05-08T08:54:13.558 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.558 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:13.559 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.559 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.559 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.559 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:970 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 3407872~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:13.559 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451198+0000", 2024-05-08T08:54:13.559 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 64.060863163999997, 2024-05-08T08:54:13.559 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088183368999999998, 2024-05-08T08:54:13.559 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:13.559 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:13.559 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:13.559 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:13.559 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:13.560 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:13.560 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 970 2024-05-08T08:54:13.560 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.560 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:13.560 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.560 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:13.560 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451198+0000", 2024-05-08T08:54:13.560 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:13.560 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.560 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.561 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:13.561 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451198+0000", 2024-05-08T08:54:13.561 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:13.561 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.561 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.561 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:13.561 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451199+0000", 2024-05-08T08:54:13.561 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7400000000000003e-07 2024-05-08T08:54:13.561 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.561 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.561 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:13.562 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451211+0000", 2024-05-08T08:54:13.562 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2109999999999999e-05 2024-05-08T08:54:13.562 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.562 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.562 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:13.562 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451219+0000", 2024-05-08T08:54:13.562 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4170000000000003e-06 2024-05-08T08:54:13.562 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.562 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.562 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:13.563 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451224+0000", 2024-05-08T08:54:13.563 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3990000000000003e-06 2024-05-08T08:54:13.563 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.563 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.563 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:13.563 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539204+0000", 2024-05-08T08:54:13.563 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087980099000000006 2024-05-08T08:54:13.563 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.563 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.563 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:13.563 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539276+0000", 2024-05-08T08:54:13.563 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1513000000000005e-05 2024-05-08T08:54:13.564 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.564 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.564 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:13.564 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539382+0000", 2024-05-08T08:54:13.564 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010615699999999999 2024-05-08T08:54:13.564 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.564 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:13.564 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.564 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.564 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.564 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:971 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 3899392~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:13.565 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451233+0000", 2024-05-08T08:54:13.565 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 64.060828457, 2024-05-08T08:54:13.565 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088431525999999996, 2024-05-08T08:54:13.565 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:13.565 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:13.565 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:13.565 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:13.565 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:13.565 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:13.565 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 971 2024-05-08T08:54:13.565 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.565 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:13.565 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.566 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:13.566 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451233+0000", 2024-05-08T08:54:13.566 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:13.566 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.566 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.566 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:13.566 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451233+0000", 2024-05-08T08:54:13.566 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:13.566 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.566 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.566 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:13.566 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451234+0000", 2024-05-08T08:54:13.567 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8500000000000001e-07 2024-05-08T08:54:13.567 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.567 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.567 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:13.567 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451237+0000", 2024-05-08T08:54:13.567 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8320000000000002e-06 2024-05-08T08:54:13.567 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.567 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.567 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:13.567 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451243+0000", 2024-05-08T08:54:13.567 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6749999999999996e-06 2024-05-08T08:54:13.567 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.567 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.568 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:13.568 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451249+0000", 2024-05-08T08:54:13.568 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1229999999999997e-06 2024-05-08T08:54:13.568 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.568 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.568 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:13.568 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539412+0000", 2024-05-08T08:54:13.568 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088162668999999999 2024-05-08T08:54:13.568 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.568 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.568 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:13.568 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539505+0000", 2024-05-08T08:54:13.569 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3240999999999997e-05 2024-05-08T08:54:13.569 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.569 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.569 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:13.569 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539665+0000", 2024-05-08T08:54:13.569 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00015930100000000001 2024-05-08T08:54:13.569 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.569 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:13.569 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.569 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.569 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.569 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:972 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 196608~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:13.570 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451257+0000", 2024-05-08T08:54:13.570 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 64.060804512000004, 2024-05-08T08:54:13.570 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088719928000000003, 2024-05-08T08:54:13.570 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:13.570 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:13.570 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:13.570 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:13.570 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:13.570 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:13.571 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 972 2024-05-08T08:54:13.571 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.571 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:13.571 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.571 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:13.571 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451257+0000", 2024-05-08T08:54:13.571 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:13.571 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.571 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.571 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:13.571 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451257+0000", 2024-05-08T08:54:13.571 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:13.572 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.572 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.572 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:13.572 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451258+0000", 2024-05-08T08:54:13.572 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7999999999999996e-07 2024-05-08T08:54:13.572 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.572 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.572 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:13.572 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451260+0000", 2024-05-08T08:54:13.572 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4609999999999998e-06 2024-05-08T08:54:13.572 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.572 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.573 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:13.573 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451267+0000", 2024-05-08T08:54:13.573 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.81e-06 2024-05-08T08:54:13.573 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.573 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.573 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:13.573 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451273+0000", 2024-05-08T08:54:13.573 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0859999999999998e-06 2024-05-08T08:54:13.573 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.573 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.573 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:13.573 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539718+0000", 2024-05-08T08:54:13.573 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088445059000000006 2024-05-08T08:54:13.574 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.574 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.574 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:13.574 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539834+0000", 2024-05-08T08:54:13.574 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000116426 2024-05-08T08:54:13.574 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.574 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.574 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:13.574 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539977+0000", 2024-05-08T08:54:13.574 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014260600000000001 2024-05-08T08:54:13.574 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.574 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:13.575 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.575 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.575 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.575 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:973 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 688128~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:13.575 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:54:13.575 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 64.060781927999997, 2024-05-08T08:54:13.575 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088918496, 2024-05-08T08:54:13.575 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:13.575 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:13.575 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:13.575 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:13.575 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:13.575 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:13.576 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 973 2024-05-08T08:54:13.576 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.576 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:13.576 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.576 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:13.576 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:54:13.576 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:13.576 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.576 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.576 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:13.576 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:54:13.576 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:13.576 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.576 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.576 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:13.577 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:54:13.577 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9099999999999999e-07 2024-05-08T08:54:13.577 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.577 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.577 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:13.577 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451282+0000", 2024-05-08T08:54:13.577 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1550000000000001e-06 2024-05-08T08:54:13.577 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.577 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.577 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:13.577 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451288+0000", 2024-05-08T08:54:13.577 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8889999999999999e-06 2024-05-08T08:54:13.577 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.577 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.578 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:13.578 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451294+0000", 2024-05-08T08:54:13.578 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0850000000000002e-06 2024-05-08T08:54:13.578 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.578 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.578 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:13.578 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540011+0000", 2024-05-08T08:54:13.578 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088717314000000005 2024-05-08T08:54:13.578 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.578 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.578 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:13.578 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540089+0000", 2024-05-08T08:54:13.578 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7438e-05 2024-05-08T08:54:13.579 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.579 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.579 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:13.579 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540198+0000", 2024-05-08T08:54:13.579 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000109224 2024-05-08T08:54:13.579 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.579 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:13.579 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.579 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.579 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.579 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:974 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 1179648~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:13.579 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451301+0000", 2024-05-08T08:54:13.579 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 64.060760294999994, 2024-05-08T08:54:13.580 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089105078000000004, 2024-05-08T08:54:13.580 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:13.580 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:13.580 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:13.580 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:13.580 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:13.580 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:13.580 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 974 2024-05-08T08:54:13.580 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.580 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:13.580 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.581 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:13.581 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451301+0000", 2024-05-08T08:54:13.581 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:13.581 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.581 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.581 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:13.581 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451301+0000", 2024-05-08T08:54:13.581 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:13.581 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.581 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.581 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:13.581 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451302+0000", 2024-05-08T08:54:13.582 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7800000000000002e-07 2024-05-08T08:54:13.582 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.582 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.582 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:13.582 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451304+0000", 2024-05-08T08:54:13.582 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.65e-06 2024-05-08T08:54:13.582 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.582 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.582 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:13.582 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451311+0000", 2024-05-08T08:54:13.582 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.472e-06 2024-05-08T08:54:13.582 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.583 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.583 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:13.583 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451316+0000", 2024-05-08T08:54:13.583 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9189999999999998e-06 2024-05-08T08:54:13.583 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.583 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.583 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:13.583 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540234+0000", 2024-05-08T08:54:13.583 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088917752000000003 2024-05-08T08:54:13.583 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.584 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.584 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:13.584 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540308+0000", 2024-05-08T08:54:13.584 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4758000000000005e-05 2024-05-08T08:54:13.584 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.584 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.584 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:13.584 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540406+0000", 2024-05-08T08:54:13.584 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8048999999999996e-05 2024-05-08T08:54:13.584 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.585 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:13.585 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.585 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.585 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.585 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:975 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 1671168~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:13.585 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:54:13.585 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 64.060717964999995, 2024-05-08T08:54:13.585 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089294139999999994, 2024-05-08T08:54:13.585 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:13.585 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:13.585 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:13.585 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:13.585 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:13.586 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:13.586 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 975 2024-05-08T08:54:13.586 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.586 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:13.586 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.586 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:13.586 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:54:13.586 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:13.586 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.586 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.587 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:13.587 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:54:13.587 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:13.587 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.587 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.587 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:13.587 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:54:13.587 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1299999999999999e-07 2024-05-08T08:54:13.587 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.587 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.588 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:13.588 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451348+0000", 2024-05-08T08:54:13.588 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.484e-06 2024-05-08T08:54:13.588 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.588 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.588 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:13.588 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451356+0000", 2024-05-08T08:54:13.588 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8499999999999994e-06 2024-05-08T08:54:13.588 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.588 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.588 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:13.588 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451366+0000", 2024-05-08T08:54:13.589 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.9850000000000001e-06 2024-05-08T08:54:13.589 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.589 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.589 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:13.589 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540448+0000", 2024-05-08T08:54:13.589 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089082254 2024-05-08T08:54:13.589 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.589 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.589 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:13.589 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540528+0000", 2024-05-08T08:54:13.589 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0234999999999993e-05 2024-05-08T08:54:13.589 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.589 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.590 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:13.590 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540638+0000", 2024-05-08T08:54:13.590 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010951900000000001 2024-05-08T08:54:13.590 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.590 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:13.590 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.590 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.590 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.590 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:976 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 2162688~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:13.590 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451375+0000", 2024-05-08T08:54:13.590 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 64.060686218000001, 2024-05-08T08:54:13.590 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089472745000000006, 2024-05-08T08:54:13.590 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:13.590 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:13.590 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:13.591 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:13.591 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:13.591 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:13.591 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 976 2024-05-08T08:54:13.591 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.591 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:13.591 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.591 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:13.591 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451375+0000", 2024-05-08T08:54:13.591 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:13.591 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.591 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.591 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:13.592 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451375+0000", 2024-05-08T08:54:13.592 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:13.592 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.592 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.592 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:13.592 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451376+0000", 2024-05-08T08:54:13.592 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2399999999999998e-07 2024-05-08T08:54:13.592 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.592 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.592 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:13.592 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451379+0000", 2024-05-08T08:54:13.592 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6359999999999998e-06 2024-05-08T08:54:13.592 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.593 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.593 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:13.593 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451387+0000", 2024-05-08T08:54:13.593 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9929999999999995e-06 2024-05-08T08:54:13.593 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.593 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.593 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:13.593 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451393+0000", 2024-05-08T08:54:13.593 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4699999999999999e-06 2024-05-08T08:54:13.593 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.593 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.593 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:13.594 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540677+0000", 2024-05-08T08:54:13.594 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089283504999999999 2024-05-08T08:54:13.594 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.594 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.594 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:13.594 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540743+0000", 2024-05-08T08:54:13.594 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6303999999999994e-05 2024-05-08T08:54:13.594 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.594 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.594 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:13.594 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540848+0000", 2024-05-08T08:54:13.594 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000105213 2024-05-08T08:54:13.595 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.595 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:13.595 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.595 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.595 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.595 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:977 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 2654208~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:13.595 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:54:13.595 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 64.060660786, 2024-05-08T08:54:13.595 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089698473000000001, 2024-05-08T08:54:13.595 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:13.595 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:13.596 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:13.596 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:13.596 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:13.596 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:13.596 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 977 2024-05-08T08:54:13.596 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.596 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:13.596 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.596 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:13.596 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:54:13.596 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:13.596 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.596 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.597 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:13.597 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:54:13.597 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:13.597 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.597 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.597 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:13.597 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:54:13.597 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2899999999999999e-07 2024-05-08T08:54:13.597 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.597 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.597 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:13.597 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451404+0000", 2024-05-08T08:54:13.597 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6649999999999999e-06 2024-05-08T08:54:13.598 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.598 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.598 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:13.598 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451410+0000", 2024-05-08T08:54:13.598 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3269999999999997e-06 2024-05-08T08:54:13.598 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.598 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.598 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:13.598 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451415+0000", 2024-05-08T08:54:13.598 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9529999999999998e-06 2024-05-08T08:54:13.598 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.599 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.599 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:13.599 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540912+0000", 2024-05-08T08:54:13.599 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089496950000000006 2024-05-08T08:54:13.599 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.599 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.599 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:13.599 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541000+0000", 2024-05-08T08:54:13.599 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7429000000000006e-05 2024-05-08T08:54:13.599 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.599 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.599 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:13.599 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541099+0000", 2024-05-08T08:54:13.600 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.9720000000000001e-05 2024-05-08T08:54:13.600 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.600 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:13.600 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.600 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.600 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.600 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:978 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 3145728~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:13.600 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:54:13.600 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 64.060639578000007, 2024-05-08T08:54:13.600 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089900635000000007, 2024-05-08T08:54:13.600 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:13.601 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:13.601 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:13.601 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:13.601 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:13.601 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:13.601 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 978 2024-05-08T08:54:13.601 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.601 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:13.601 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.601 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:13.601 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:54:13.601 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:13.601 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.601 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.602 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:13.602 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:54:13.602 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:13.602 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.602 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.602 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:13.602 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:54:13.602 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9400000000000001e-07 2024-05-08T08:54:13.602 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.602 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.602 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:13.603 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451425+0000", 2024-05-08T08:54:13.603 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2460000000000002e-06 2024-05-08T08:54:13.603 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.603 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.603 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:13.603 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451431+0000", 2024-05-08T08:54:13.603 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2419999999999997e-06 2024-05-08T08:54:13.603 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.603 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.603 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:13.603 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451436+0000", 2024-05-08T08:54:13.603 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.887e-06 2024-05-08T08:54:13.603 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.604 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.604 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:13.604 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541152+0000", 2024-05-08T08:54:13.604 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089716194999999999 2024-05-08T08:54:13.604 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.604 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.604 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:13.604 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541224+0000", 2024-05-08T08:54:13.604 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2416000000000006e-05 2024-05-08T08:54:13.604 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.604 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.605 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:13.605 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541323+0000", 2024-05-08T08:54:13.605 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8255e-05 2024-05-08T08:54:13.605 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.605 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:13.605 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.605 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.605 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.605 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:979 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 3637248~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:13.605 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451462+0000", 2024-05-08T08:54:13.605 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 64.060599128999996, 2024-05-08T08:54:13.605 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.090036441999999994, 2024-05-08T08:54:13.605 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:13.605 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:13.605 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:13.605 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:13.606 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:13.606 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:13.606 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 979 2024-05-08T08:54:13.606 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.606 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:13.606 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.606 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:13.606 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451462+0000", 2024-05-08T08:54:13.606 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:13.606 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.606 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.606 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:13.606 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451462+0000", 2024-05-08T08:54:13.607 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:13.607 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.607 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.607 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:13.607 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451463+0000", 2024-05-08T08:54:13.607 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4300000000000002e-07 2024-05-08T08:54:13.607 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.607 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.607 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:13.607 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451471+0000", 2024-05-08T08:54:13.607 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0800000000000006e-06 2024-05-08T08:54:13.607 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.607 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.607 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:13.607 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451481+0000", 2024-05-08T08:54:13.607 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.38e-06 2024-05-08T08:54:13.608 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.608 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.608 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:13.608 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451487+0000", 2024-05-08T08:54:13.608 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4590000000000003e-06 2024-05-08T08:54:13.608 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.608 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.608 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:13.608 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541350+0000", 2024-05-08T08:54:13.608 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089862599000000001 2024-05-08T08:54:13.608 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.608 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.608 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:13.608 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541407+0000", 2024-05-08T08:54:13.608 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7667999999999997e-05 2024-05-08T08:54:13.608 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.608 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.609 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:13.609 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541499+0000", 2024-05-08T08:54:13.609 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1513000000000003e-05 2024-05-08T08:54:13.609 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.609 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:13.609 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.609 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.609 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.609 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:980 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 4128768~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:13.609 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451496+0000", 2024-05-08T08:54:13.609 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 64.060565373000003, 2024-05-08T08:54:13.609 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.090258985, 2024-05-08T08:54:13.609 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:13.609 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:13.609 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:13.609 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:13.610 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:13.610 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:13.610 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 980 2024-05-08T08:54:13.610 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.610 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:13.610 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.610 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:13.610 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451496+0000", 2024-05-08T08:54:13.610 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:13.610 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.610 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.610 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:13.610 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451496+0000", 2024-05-08T08:54:13.610 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:13.610 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.610 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.611 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:13.611 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451497+0000", 2024-05-08T08:54:13.611 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7599999999999997e-07 2024-05-08T08:54:13.611 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.611 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.611 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:13.611 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451500+0000", 2024-05-08T08:54:13.611 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7930000000000002e-06 2024-05-08T08:54:13.611 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.611 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.611 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:13.611 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451506+0000", 2024-05-08T08:54:13.611 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6980000000000001e-06 2024-05-08T08:54:13.611 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.611 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.612 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:13.612 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451512+0000", 2024-05-08T08:54:13.612 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2490000000000001e-06 2024-05-08T08:54:13.612 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.612 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.612 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:13.612 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541550+0000", 2024-05-08T08:54:13.612 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.090037961999999999 2024-05-08T08:54:13.612 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.612 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.612 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:13.612 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541647+0000", 2024-05-08T08:54:13.612 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6924999999999998e-05 2024-05-08T08:54:13.612 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:13.612 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:13.613 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:13.613 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541755+0000", 2024-05-08T08:54:13.613 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000108682 2024-05-08T08:54:13.613 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.613 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:13.613 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.613 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:13.613 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:13.613 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:13.615 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:13.617+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:13.616 INFO:teuthology.orchestra.run.smithi196.stdout:5436 write 0x8c31400 thru 0x8c399ab (0x85ac bytes) 2024-05-08T08:54:13.616 INFO:teuthology.orchestra.run.smithi196.stdout:5437 trunc from 0x8c39a00 to 0x2818e00 2024-05-08T08:54:13.616 INFO:teuthology.orchestra.run.smithi196.stdout:5438 write 0x75e3800 thru 0x75e525d (0x1a5e bytes) 2024-05-08T08:54:13.616 INFO:teuthology.orchestra.run.smithi196.stdout:5439 read 0x2066800 thru 0x2071c01 (0xb402 bytes) 2024-05-08T08:54:13.616 INFO:teuthology.orchestra.run.smithi196.stdout:5440 trunc from 0x75e5400 to 0x676400 2024-05-08T08:54:13.616 INFO:teuthology.orchestra.run.smithi196.stdout:5442 write 0x5bad200 thru 0x5bb9212 (0xc013 bytes) 2024-05-08T08:54:13.617 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:54:13.622 INFO:teuthology.orchestra.run.smithi196.stdout:5443 trunc from 0x5bb9400 to 0xe08f200 2024-05-08T08:54:13.636 INFO:teuthology.orchestra.run.smithi196.stdout:5444 write 0x69c7400 thru 0x69d6ad3 (0xf6d4 bytes) 2024-05-08T08:54:13.637 INFO:teuthology.orchestra.run.smithi196.stdout:5446 read 0x2a63600 thru 0x2a71068 (0xda69 bytes) 2024-05-08T08:54:13.638 INFO:teuthology.orchestra.run.smithi196.stdout:5448 write 0xa740000 thru 0xa745df8 (0x5df9 bytes) 2024-05-08T08:54:13.638 INFO:teuthology.orchestra.run.smithi196.stdout:5452 write 0x6898000 thru 0x68a6299 (0xe29a bytes) 2024-05-08T08:54:13.638 INFO:teuthology.orchestra.run.smithi196.stdout:5453 write 0x4929200 thru 0x4934b3b (0xb93c bytes) 2024-05-08T08:54:13.639 INFO:teuthology.orchestra.run.smithi196.stdout:5454 write 0xbb40c00 thru 0xbb4e5f6 (0xd9f7 bytes) 2024-05-08T08:54:13.639 INFO:teuthology.orchestra.run.smithi196.stdout:5455 write 0xdc30000 thru 0xdc3d24b (0xd24c bytes) 2024-05-08T08:54:13.639 INFO:teuthology.orchestra.run.smithi196.stdout:5457 punch from 0xb9ccc00 to 0xb9d5800, (0x8c00 bytes) 2024-05-08T08:54:13.639 INFO:teuthology.orchestra.run.smithi196.stdout:5458 punch from 0x1460800 to 0x1466800, (0x6000 bytes) 2024-05-08T08:54:13.661 INFO:teuthology.orchestra.run.smithi196.stdout:5462 write 0xe876800 thru 0xe881b3d (0xb33e bytes) 2024-05-08T08:54:13.663 INFO:teuthology.orchestra.run.smithi196.stdout:5464 write 0x6b0e00 thru 0x6b78b5 (0x6ab6 bytes) 2024-05-08T08:54:13.663 INFO:teuthology.orchestra.run.smithi196.stdout:5466 read 0x8c6c00 thru 0x8d5e24 (0xf225 bytes) 2024-05-08T08:54:13.664 INFO:teuthology.orchestra.run.smithi196.stdout:5467 write 0x9a2ba00 thru 0x9a2e89e (0x2e9f bytes) 2024-05-08T08:54:13.665 INFO:teuthology.orchestra.run.smithi196.stdout:5469 read 0x8b0da00 thru 0x8b1bb48 (0xe149 bytes) 2024-05-08T08:54:13.665 INFO:teuthology.orchestra.run.smithi196.stdout:5471 write 0xbfc0c00 thru 0xbfc2753 (0x1b54 bytes) 2024-05-08T08:54:13.665 INFO:teuthology.orchestra.run.smithi196.stdout:5472 write 0x7f3a600 thru 0x7f3bae8 (0x14e9 bytes) 2024-05-08T08:54:13.665 INFO:teuthology.orchestra.run.smithi196.stdout:5474 write 0x128d000 thru 0x12983eb (0xb3ec bytes) 2024-05-08T08:54:13.665 INFO:teuthology.orchestra.run.smithi196.stdout:5477 write 0xa8b7600 thru 0xa8baa63 (0x3464 bytes) 2024-05-08T08:54:13.666 INFO:teuthology.orchestra.run.smithi196.stdout:5481 clone 41 order 22 su 65536 sc 9 2024-05-08T08:54:13.715 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:13.718+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:13.737 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:13.737 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:54:13.737 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:54:13.737 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:13.746 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:54:13.771 INFO:teuthology.orchestra.run.smithi196.stdout:leaving image image_client.0-clone40 intact 2024-05-08T08:54:13.816 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:13.819+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:13.852 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:13.853 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:54:13.853 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:54:13.853 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:54:13.853 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:13.861 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:54:13.917 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:13.920+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:13.969 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:13.969 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:54:13.970 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:54:13.970 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:54:13.970 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:13.970 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4621.0:9089 3.1f e33/18 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:head v 33'117, mlcod=31'115)", 2024-05-08T08:54:13.970 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:54:13.970 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 101.45177855, 2024-05-08T08:54:13.970 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.111244524, 2024-05-08T08:54:13.970 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:13.970 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:13.970 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:54:13.970 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:13.970 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:13.971 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:13.971 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:54:13.971 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:13.971 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:13.971 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:13.971 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:13.971 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:54:13.971 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:13.971 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:13.971 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:13.971 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:13.971 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:54:13.972 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5900000000000001e-07 2024-05-08T08:54:13.972 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:13.972 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:13.972 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:13.972 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539600+0000", 2024-05-08T08:54:13.972 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.019981364000000001 2024-05-08T08:54:13.972 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:13.972 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:13.972 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:13.972 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539624+0000", 2024-05-08T08:54:13.972 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4043e-05 2024-05-08T08:54:13.972 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:13.973 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:13.973 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:13.973 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539635+0000", 2024-05-08T08:54:13.973 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1511e-05 2024-05-08T08:54:13.973 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:13.973 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:13.973 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:13.973 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539728+0000", 2024-05-08T08:54:13.973 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2498999999999996e-05 2024-05-08T08:54:13.974 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:13.974 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:13.974 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:13.974 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539746+0000", 2024-05-08T08:54:13.974 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8289000000000001e-05 2024-05-08T08:54:13.974 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:13.974 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:13.974 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:54:13.974 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.630822+0000", 2024-05-08T08:54:13.975 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.091076283999999993 2024-05-08T08:54:13.975 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:13.975 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:13.975 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:13.975 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.630863+0000", 2024-05-08T08:54:13.975 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0374999999999999e-05 2024-05-08T08:54:13.975 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:13.975 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:13.975 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:13.976 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:13.976 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:13.976 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4795.0:230 3.1f e40/18 3:f8db1a56:::rbd_data.11402b26385.0000000000000000:head v 40'171, mlcod=39'168)", 2024-05-08T08:54:13.976 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:45.927773+0000", 2024-05-08T08:54:13.976 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 88.043623467000003, 2024-05-08T08:54:13.976 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045892405999999997, 2024-05-08T08:54:13.976 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:13.976 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:13.976 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:54:13.977 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:13.977 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:13.977 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:13.977 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.927773+0000", 2024-05-08T08:54:13.977 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:13.977 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:13.977 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:13.977 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:13.977 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.927773+0000", 2024-05-08T08:54:13.977 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:13.978 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:13.978 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:13.978 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:13.978 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.927774+0000", 2024-05-08T08:54:13.978 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0500000000000002e-07 2024-05-08T08:54:13.978 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:13.978 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:13.978 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:13.978 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.931581+0000", 2024-05-08T08:54:13.978 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0038071799999999999 2024-05-08T08:54:13.978 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:13.979 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:13.979 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:13.979 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.931599+0000", 2024-05-08T08:54:13.979 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7479e-05 2024-05-08T08:54:13.979 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:13.979 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:13.979 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:13.979 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.931624+0000", 2024-05-08T08:54:13.979 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5372999999999999e-05 2024-05-08T08:54:13.979 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:13.979 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:13.979 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:13.980 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.931698+0000", 2024-05-08T08:54:13.980 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4043999999999998e-05 2024-05-08T08:54:13.980 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:13.980 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:13.980 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:13.980 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.931725+0000", 2024-05-08T08:54:13.980 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6883e-05 2024-05-08T08:54:13.980 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:13.980 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:13.980 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:54:13.980 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.973623+0000", 2024-05-08T08:54:13.980 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.041898283000000001 2024-05-08T08:54:13.980 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:13.980 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:13.980 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:13.980 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.973666+0000", 2024-05-08T08:54:13.981 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2358999999999999e-05 2024-05-08T08:54:13.981 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:13.981 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:13.981 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:13.981 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:13.981 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:13.981 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4195.0:4 3.11 3:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-05-08T08:54:13.981 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:49.764654+0000", 2024-05-08T08:54:13.981 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 84.206743008999993, 2024-05-08T08:54:13.981 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.30714275499999999, 2024-05-08T08:54:13.981 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:13.981 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:13.981 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-05-08T08:54:13.981 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:13.981 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4195", 2024-05-08T08:54:13.982 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/59179521", 2024-05-08T08:54:13.982 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4 2024-05-08T08:54:13.982 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:13.982 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:13.982 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:13.982 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:13.982 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764654+0000", 2024-05-08T08:54:13.982 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:13.982 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:13.982 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:13.982 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:13.982 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764654+0000", 2024-05-08T08:54:13.982 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:13.982 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:13.982 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:13.983 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:13.983 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764655+0000", 2024-05-08T08:54:13.983 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.452e-06 2024-05-08T08:54:13.983 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:13.983 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:13.983 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:13.983 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764661+0000", 2024-05-08T08:54:13.983 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5369999999999997e-06 2024-05-08T08:54:13.983 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:13.983 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:13.983 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:13.983 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764671+0000", 2024-05-08T08:54:13.983 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.028e-05 2024-05-08T08:54:13.983 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:13.984 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:13.984 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:13.984 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764680+0000", 2024-05-08T08:54:13.984 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0629999999999995e-06 2024-05-08T08:54:13.984 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:13.984 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:13.984 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:13.984 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764753+0000", 2024-05-08T08:54:13.984 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2679e-05 2024-05-08T08:54:13.984 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:13.984 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:13.984 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-05-08T08:54:13.984 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764765+0000", 2024-05-08T08:54:13.984 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2415000000000001e-05 2024-05-08T08:54:13.985 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:13.985 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:13.985 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:13.985 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:50.071692+0000", 2024-05-08T08:54:13.985 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.306927324 2024-05-08T08:54:13.985 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:13.985 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:13.985 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:13.985 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:50.071796+0000", 2024-05-08T08:54:13.985 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104005 2024-05-08T08:54:13.985 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:13.985 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:13.985 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:13.985 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:13.985 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:13.986 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22683 3.15 3:a907109d:::rbd_data.1140d3fc54c.000000000000004f:18 [read 1335296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:13.986 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.097686+0000", 2024-05-08T08:54:13.986 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.873710334999998, 2024-05-08T08:54:13.986 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042114716000000003, 2024-05-08T08:54:13.986 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:13.986 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:13.986 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:13.986 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:13.986 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:54:13.986 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:13.986 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22683 2024-05-08T08:54:13.986 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:13.986 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:13.986 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:13.987 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:13.987 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097686+0000", 2024-05-08T08:54:13.987 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:13.987 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:13.987 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:13.987 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:13.987 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097686+0000", 2024-05-08T08:54:13.987 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:13.987 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:13.987 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:13.987 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:13.987 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097687+0000", 2024-05-08T08:54:13.987 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3999999999999997e-07 2024-05-08T08:54:13.987 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:13.988 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:13.988 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:13.988 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097688+0000", 2024-05-08T08:54:13.988 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6789999999999999e-06 2024-05-08T08:54:13.988 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:13.988 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:13.988 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:13.988 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097693+0000", 2024-05-08T08:54:13.988 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.045e-06 2024-05-08T08:54:13.988 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:13.988 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:13.988 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:13.988 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097697+0000", 2024-05-08T08:54:13.988 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0260000000000001e-06 2024-05-08T08:54:13.989 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:13.989 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:13.989 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:13.989 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139725+0000", 2024-05-08T08:54:13.989 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042027737000000003 2024-05-08T08:54:13.989 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:13.989 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:13.989 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:13.989 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139764+0000", 2024-05-08T08:54:13.989 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8668999999999999e-05 2024-05-08T08:54:13.989 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:13.989 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:13.989 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:13.990 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139801+0000", 2024-05-08T08:54:13.990 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7219999999999999e-05 2024-05-08T08:54:13.990 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:13.990 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:13.990 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:13.990 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:13.990 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:13.990 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22684 3.15 3:a907109d:::rbd_data.1140d3fc54c.000000000000004f:18 [read 1400832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:13.990 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.097703+0000", 2024-05-08T08:54:13.990 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.873693981999999, 2024-05-08T08:54:13.990 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042198929000000003, 2024-05-08T08:54:13.990 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:13.991 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:13.991 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:13.991 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:13.991 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:54:13.991 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:13.991 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22684 2024-05-08T08:54:13.991 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:13.991 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:13.991 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:13.991 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:13.991 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097703+0000", 2024-05-08T08:54:13.991 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:13.992 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:13.992 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:13.992 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:13.992 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097703+0000", 2024-05-08T08:54:13.992 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:13.992 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:13.992 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:13.992 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:13.992 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097703+0000", 2024-05-08T08:54:13.992 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4400000000000001e-07 2024-05-08T08:54:13.992 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:13.992 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:13.992 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:13.993 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097705+0000", 2024-05-08T08:54:13.993 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9199999999999998e-06 2024-05-08T08:54:13.993 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:13.993 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:13.993 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:13.993 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097711+0000", 2024-05-08T08:54:13.993 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5559999999999998e-06 2024-05-08T08:54:13.993 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:13.993 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:13.993 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:13.993 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097715+0000", 2024-05-08T08:54:13.993 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3159999999999998e-06 2024-05-08T08:54:13.994 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:13.994 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:13.994 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:13.994 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139825+0000", 2024-05-08T08:54:13.994 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042110284999999997 2024-05-08T08:54:13.994 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:13.994 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:13.994 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:13.994 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139864+0000", 2024-05-08T08:54:13.994 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8980000000000003e-05 2024-05-08T08:54:13.994 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:13.995 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:13.995 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:13.995 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139902+0000", 2024-05-08T08:54:13.995 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7527999999999997e-05 2024-05-08T08:54:13.995 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:13.995 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:13.995 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:13.995 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:13.995 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:13.995 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22685 3.15 3:a907109d:::rbd_data.1140d3fc54c.000000000000004f:18 [read 1466368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:13.995 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.097721+0000", 2024-05-08T08:54:13.996 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.873675325000001, 2024-05-08T08:54:13.996 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042287791999999998, 2024-05-08T08:54:13.996 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:13.996 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:13.996 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:13.996 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:13.996 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:54:13.996 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:13.996 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22685 2024-05-08T08:54:13.996 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:13.996 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:13.996 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:13.996 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:13.997 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097721+0000", 2024-05-08T08:54:13.997 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:13.997 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:13.997 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:13.997 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:13.997 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097721+0000", 2024-05-08T08:54:13.997 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:13.997 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:13.997 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:13.997 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:13.997 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097722+0000", 2024-05-08T08:54:13.997 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2500000000000001e-07 2024-05-08T08:54:13.998 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:13.998 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:13.998 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:13.998 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097724+0000", 2024-05-08T08:54:13.998 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.835e-06 2024-05-08T08:54:13.998 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:13.998 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:13.998 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:13.998 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097729+0000", 2024-05-08T08:54:13.998 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5029999999999997e-06 2024-05-08T08:54:13.999 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:13.999 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:13.999 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:13.999 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097733+0000", 2024-05-08T08:54:13.999 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3309999999999997e-06 2024-05-08T08:54:13.999 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:13.999 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:13.999 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:14.000 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139923+0000", 2024-05-08T08:54:14.000 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042189988999999997 2024-05-08T08:54:14.000 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.000 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.000 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:14.000 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139970+0000", 2024-05-08T08:54:14.000 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6776000000000001e-05 2024-05-08T08:54:14.000 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.000 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.000 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:14.001 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140009+0000", 2024-05-08T08:54:14.001 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9032999999999997e-05 2024-05-08T08:54:14.001 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:14.001 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:14.001 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:14.001 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.001 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.001 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22686 3.15 3:a907109d:::rbd_data.1140d3fc54c.000000000000004f:18 [read 1531904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:14.001 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.097739+0000", 2024-05-08T08:54:14.002 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.873657326, 2024-05-08T08:54:14.002 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042371008000000002, 2024-05-08T08:54:14.002 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:14.002 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:14.002 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:14.002 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:14.002 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:54:14.002 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:14.003 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22686 2024-05-08T08:54:14.003 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.003 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:14.003 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.003 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:14.003 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097739+0000", 2024-05-08T08:54:14.003 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:14.003 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.003 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.003 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:14.003 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097739+0000", 2024-05-08T08:54:14.004 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:14.004 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.004 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.004 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:14.004 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097740+0000", 2024-05-08T08:54:14.004 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8599999999999999e-07 2024-05-08T08:54:14.004 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.004 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.004 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:14.004 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097742+0000", 2024-05-08T08:54:14.004 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9410000000000001e-06 2024-05-08T08:54:14.005 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.005 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.005 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:14.005 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097747+0000", 2024-05-08T08:54:14.005 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2079999999999999e-06 2024-05-08T08:54:14.005 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.005 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.005 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:14.005 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097751+0000", 2024-05-08T08:54:14.005 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2250000000000002e-06 2024-05-08T08:54:14.005 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.005 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.006 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:14.006 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140032+0000", 2024-05-08T08:54:14.006 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042281244000000003 2024-05-08T08:54:14.006 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.006 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.006 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:14.006 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140074+0000", 2024-05-08T08:54:14.006 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1316999999999998e-05 2024-05-08T08:54:14.006 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.006 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.006 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:14.006 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140110+0000", 2024-05-08T08:54:14.006 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6687000000000003e-05 2024-05-08T08:54:14.006 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:14.006 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:14.007 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:14.007 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.007 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.007 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22716 3.15 3:a88fdee9:::rbd_data.1140d3fc54c.000000000000004d:18 [read 1097728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:14.007 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.098153+0000", 2024-05-08T08:54:14.007 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.873243619, 2024-05-08T08:54:14.007 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042058535000000001, 2024-05-08T08:54:14.007 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:14.007 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:14.007 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:14.007 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:14.007 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:54:14.007 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:14.007 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22716 2024-05-08T08:54:14.007 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.008 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:14.008 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.008 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:14.008 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098153+0000", 2024-05-08T08:54:14.008 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:14.008 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.008 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.008 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:14.008 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098153+0000", 2024-05-08T08:54:14.008 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:14.008 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.008 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.008 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:14.008 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098154+0000", 2024-05-08T08:54:14.008 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.13e-07 2024-05-08T08:54:14.009 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.009 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.009 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:14.009 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098161+0000", 2024-05-08T08:54:14.009 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7360000000000001e-06 2024-05-08T08:54:14.009 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.009 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.009 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:14.009 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098171+0000", 2024-05-08T08:54:14.009 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1039999999999996e-06 2024-05-08T08:54:14.009 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.009 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.009 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:14.009 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098177+0000", 2024-05-08T08:54:14.009 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5760000000000002e-06 2024-05-08T08:54:14.009 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.010 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.010 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:14.010 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140135+0000", 2024-05-08T08:54:14.010 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.041957842000000002 2024-05-08T08:54:14.010 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.010 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.010 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:14.010 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140174+0000", 2024-05-08T08:54:14.010 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8902999999999997e-05 2024-05-08T08:54:14.010 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.010 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.010 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:14.010 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140212+0000", 2024-05-08T08:54:14.010 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7761e-05 2024-05-08T08:54:14.010 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:14.011 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:14.011 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:14.011 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.011 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.011 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22717 3.15 3:a88fdee9:::rbd_data.1140d3fc54c.000000000000004d:18 [read 1163264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:14.011 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.098188+0000", 2024-05-08T08:54:14.011 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.873208896999998, 2024-05-08T08:54:14.011 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042124907000000003, 2024-05-08T08:54:14.011 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:14.011 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:14.011 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:14.011 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:14.011 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:54:14.011 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:14.011 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22717 2024-05-08T08:54:14.011 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.012 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:14.012 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.012 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:14.012 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098188+0000", 2024-05-08T08:54:14.012 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:14.012 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.012 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.012 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:14.012 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098188+0000", 2024-05-08T08:54:14.012 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:14.012 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.012 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.012 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:14.012 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098188+0000", 2024-05-08T08:54:14.012 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3799999999999998e-07 2024-05-08T08:54:14.012 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.013 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.013 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:14.013 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098190+0000", 2024-05-08T08:54:14.013 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.114e-06 2024-05-08T08:54:14.013 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.013 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.013 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:14.013 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098197+0000", 2024-05-08T08:54:14.013 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9959999999999996e-06 2024-05-08T08:54:14.013 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.013 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.013 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:14.013 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098202+0000", 2024-05-08T08:54:14.014 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9110000000000001e-06 2024-05-08T08:54:14.014 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.014 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.014 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:14.014 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140235+0000", 2024-05-08T08:54:14.014 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042032913999999998 2024-05-08T08:54:14.014 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.014 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.014 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:14.014 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140275+0000", 2024-05-08T08:54:14.014 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9564999999999998e-05 2024-05-08T08:54:14.014 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.014 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.014 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:14.014 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140313+0000", 2024-05-08T08:54:14.015 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7969000000000002e-05 2024-05-08T08:54:14.015 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:14.015 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:14.015 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:14.015 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.015 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.015 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22718 3.15 3:a88fdee9:::rbd_data.1140d3fc54c.000000000000004d:18 [read 1228800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:14.015 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.098209+0000", 2024-05-08T08:54:14.015 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.873187751000003, 2024-05-08T08:54:14.015 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.04220409, 2024-05-08T08:54:14.015 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:14.015 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:14.016 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:14.016 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:14.016 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:54:14.016 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:14.016 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22718 2024-05-08T08:54:14.016 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.016 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:14.016 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.016 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:14.016 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098209+0000", 2024-05-08T08:54:14.016 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:14.016 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.017 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.017 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:14.017 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098209+0000", 2024-05-08T08:54:14.017 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:14.017 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.017 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.017 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:14.017 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098209+0000", 2024-05-08T08:54:14.017 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9299999999999999e-07 2024-05-08T08:54:14.017 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.017 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.017 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:14.018 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098212+0000", 2024-05-08T08:54:14.018 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.21e-06 2024-05-08T08:54:14.018 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.018 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.018 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:14.018 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098219+0000", 2024-05-08T08:54:14.018 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0269999999999998e-06 2024-05-08T08:54:14.018 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.018 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.018 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:14.018 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098223+0000", 2024-05-08T08:54:14.019 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7319999999999998e-06 2024-05-08T08:54:14.019 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.019 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.019 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:14.019 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140338+0000", 2024-05-08T08:54:14.019 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042114390000000002 2024-05-08T08:54:14.019 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.019 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.019 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:14.019 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140376+0000", 2024-05-08T08:54:14.019 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8414999999999997e-05 2024-05-08T08:54:14.019 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.020 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.020 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:14.020 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140413+0000", 2024-05-08T08:54:14.020 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6922999999999999e-05 2024-05-08T08:54:14.020 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:14.020 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:14.020 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:14.020 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.020 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.020 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22719 3.15 3:a88fdee9:::rbd_data.1140d3fc54c.000000000000004d:18 [read 1294336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:14.020 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.098229+0000", 2024-05-08T08:54:14.020 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.873167899999999, 2024-05-08T08:54:14.021 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042283016, 2024-05-08T08:54:14.021 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:14.021 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:14.021 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:14.021 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:14.021 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:54:14.021 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:14.021 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22719 2024-05-08T08:54:14.021 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.021 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:14.021 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.021 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:14.022 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098229+0000", 2024-05-08T08:54:14.022 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:14.022 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.022 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.022 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:14.022 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098229+0000", 2024-05-08T08:54:14.022 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:14.022 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.022 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.022 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:14.023 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098229+0000", 2024-05-08T08:54:14.023 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:54:14.023 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.023 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.023 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:14.023 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098231+0000", 2024-05-08T08:54:14.023 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.81e-06 2024-05-08T08:54:14.023 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.023 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.023 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:14.023 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098237+0000", 2024-05-08T08:54:14.024 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3389999999999998e-06 2024-05-08T08:54:14.024 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.024 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.024 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:14.024 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098243+0000", 2024-05-08T08:54:14.024 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.648e-06 2024-05-08T08:54:14.024 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.024 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.024 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:14.024 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140436+0000", 2024-05-08T08:54:14.024 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042192738 2024-05-08T08:54:14.024 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.024 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.025 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:14.025 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140475+0000", 2024-05-08T08:54:14.025 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9279999999999997e-05 2024-05-08T08:54:14.025 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.025 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.025 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:14.025 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140512+0000", 2024-05-08T08:54:14.025 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6847000000000001e-05 2024-05-08T08:54:14.025 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:14.025 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:14.025 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:14.025 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.026 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.026 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22720 3.15 3:a88fdee9:::rbd_data.1140d3fc54c.000000000000004d:18 [read 1359872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:14.026 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.098248+0000", 2024-05-08T08:54:14.026 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.873148389000001, 2024-05-08T08:54:14.026 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042360800999999997, 2024-05-08T08:54:14.026 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:14.026 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:14.026 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:14.026 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:14.026 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:54:14.026 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:14.026 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22720 2024-05-08T08:54:14.026 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.026 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:14.027 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.027 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:14.027 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098248+0000", 2024-05-08T08:54:14.027 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:14.027 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.027 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.027 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:14.027 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098248+0000", 2024-05-08T08:54:14.027 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:14.027 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.028 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.028 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:14.028 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098491+0000", 2024-05-08T08:54:14.028 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000242659 2024-05-08T08:54:14.028 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.028 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.028 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:14.028 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098495+0000", 2024-05-08T08:54:14.028 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.799e-06 2024-05-08T08:54:14.028 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.029 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.029 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:14.029 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098505+0000", 2024-05-08T08:54:14.029 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7079999999999992e-06 2024-05-08T08:54:14.029 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.029 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.029 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:14.029 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098511+0000", 2024-05-08T08:54:14.029 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4370000000000004e-06 2024-05-08T08:54:14.029 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.030 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.030 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:14.030 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140533+0000", 2024-05-08T08:54:14.030 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042022371000000003 2024-05-08T08:54:14.030 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.030 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.030 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:14.030 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140571+0000", 2024-05-08T08:54:14.031 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7700000000000002e-05 2024-05-08T08:54:14.031 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.031 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.031 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:14.031 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140609+0000", 2024-05-08T08:54:14.031 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8127000000000003e-05 2024-05-08T08:54:14.031 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:14.031 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:14.031 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:14.032 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.032 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.032 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22721 3.15 3:a88fdee9:::rbd_data.1140d3fc54c.000000000000004d:18 [read 1425408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:14.032 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.098522+0000", 2024-05-08T08:54:14.032 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.872874764999999, 2024-05-08T08:54:14.032 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042185779999999999, 2024-05-08T08:54:14.032 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:14.032 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:14.032 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:14.032 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:14.033 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:54:14.033 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:14.033 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22721 2024-05-08T08:54:14.033 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.033 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:14.033 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.033 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:14.033 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098522+0000", 2024-05-08T08:54:14.033 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:14.033 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.033 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.033 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:14.034 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098522+0000", 2024-05-08T08:54:14.034 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:14.034 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.034 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.034 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:14.034 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098708+0000", 2024-05-08T08:54:14.034 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00018629700000000001 2024-05-08T08:54:14.034 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.034 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.034 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:14.034 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098719+0000", 2024-05-08T08:54:14.035 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0992000000000001e-05 2024-05-08T08:54:14.035 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.035 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.035 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:14.035 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098730+0000", 2024-05-08T08:54:14.035 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0431999999999999e-05 2024-05-08T08:54:14.035 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.035 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.035 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:14.035 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098737+0000", 2024-05-08T08:54:14.035 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.063e-06 2024-05-08T08:54:14.035 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.035 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.035 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:14.035 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140631+0000", 2024-05-08T08:54:14.036 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.041894503999999999 2024-05-08T08:54:14.036 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.036 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.036 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:14.036 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140671+0000", 2024-05-08T08:54:14.036 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9780999999999997e-05 2024-05-08T08:54:14.036 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.036 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.036 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:14.036 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140708+0000", 2024-05-08T08:54:14.036 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6711000000000001e-05 2024-05-08T08:54:14.036 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:14.036 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:14.036 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:14.036 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.037 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.037 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22722 3.15 3:a88fdee9:::rbd_data.1140d3fc54c.000000000000004d:18 [read 1490944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:14.037 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.098753+0000", 2024-05-08T08:54:14.037 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.872643760999999, 2024-05-08T08:54:14.037 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042053944000000003, 2024-05-08T08:54:14.037 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:14.037 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:14.037 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:14.037 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:14.037 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:54:14.037 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:14.037 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22722 2024-05-08T08:54:14.037 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.037 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:14.037 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.038 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:14.038 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098753+0000", 2024-05-08T08:54:14.038 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:14.038 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.038 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.038 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:14.038 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098753+0000", 2024-05-08T08:54:14.038 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:14.038 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.038 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.038 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:14.038 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098753+0000", 2024-05-08T08:54:14.038 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1399999999999997e-07 2024-05-08T08:54:14.038 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.038 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.039 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:14.039 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098756+0000", 2024-05-08T08:54:14.039 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9780000000000001e-06 2024-05-08T08:54:14.039 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.039 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.039 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:14.039 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098764+0000", 2024-05-08T08:54:14.039 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4340000000000003e-06 2024-05-08T08:54:14.039 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.039 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.039 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:14.039 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098770+0000", 2024-05-08T08:54:14.039 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8080000000000001e-06 2024-05-08T08:54:14.039 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.039 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.039 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:14.040 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140731+0000", 2024-05-08T08:54:14.040 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.041961131999999998 2024-05-08T08:54:14.040 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.040 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.040 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:14.040 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140769+0000", 2024-05-08T08:54:14.040 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.773e-05 2024-05-08T08:54:14.040 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.040 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.040 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:14.040 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140807+0000", 2024-05-08T08:54:14.040 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8347999999999999e-05 2024-05-08T08:54:14.040 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:14.040 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:14.040 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:14.041 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.041 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.041 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22723 3.15 3:a88fdee9:::rbd_data.1140d3fc54c.000000000000004d:18 [read 1556480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:14.041 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.098778+0000", 2024-05-08T08:54:14.041 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.872618989000003, 2024-05-08T08:54:14.041 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042128577, 2024-05-08T08:54:14.041 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:14.041 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:14.041 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:14.041 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:14.041 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:54:14.041 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:14.041 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22723 2024-05-08T08:54:14.041 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.042 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:14.042 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.042 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:14.042 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098778+0000", 2024-05-08T08:54:14.042 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:14.042 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.042 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.042 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:14.042 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098778+0000", 2024-05-08T08:54:14.042 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:14.042 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.042 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.042 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:14.042 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098834+0000", 2024-05-08T08:54:14.042 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5788000000000002e-05 2024-05-08T08:54:14.043 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.043 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.043 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:14.043 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098837+0000", 2024-05-08T08:54:14.043 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3510000000000002e-06 2024-05-08T08:54:14.043 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.043 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.043 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:14.043 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098845+0000", 2024-05-08T08:54:14.043 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5490000000000006e-06 2024-05-08T08:54:14.043 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.043 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.043 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:14.043 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098861+0000", 2024-05-08T08:54:14.043 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5233e-05 2024-05-08T08:54:14.044 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.044 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.044 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:14.044 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140829+0000", 2024-05-08T08:54:14.044 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.04196863 2024-05-08T08:54:14.044 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.044 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.044 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:14.044 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140869+0000", 2024-05-08T08:54:14.044 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9461e-05 2024-05-08T08:54:14.044 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.044 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.044 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:14.044 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140906+0000", 2024-05-08T08:54:14.045 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7564999999999997e-05 2024-05-08T08:54:14.045 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:14.045 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:14.045 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:14.045 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.045 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.045 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22732 3.15 3:a907109d:::rbd_data.1140d3fc54c.000000000000004f:18 [read 1097728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:14.045 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.098871+0000", 2024-05-08T08:54:14.045 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.872525508999999, 2024-05-08T08:54:14.045 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042141517000000003, 2024-05-08T08:54:14.045 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:14.045 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:14.045 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:14.046 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:14.046 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:54:14.046 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:14.046 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22732 2024-05-08T08:54:14.046 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.046 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:14.046 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.046 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:14.046 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098871+0000", 2024-05-08T08:54:14.046 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:14.046 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.046 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.047 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:14.047 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098871+0000", 2024-05-08T08:54:14.047 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:14.047 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.047 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.047 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:14.047 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.099940+0000", 2024-05-08T08:54:14.047 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0010683279999999999 2024-05-08T08:54:14.047 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.047 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.047 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:14.047 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.099944+0000", 2024-05-08T08:54:14.047 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9339999999999997e-06 2024-05-08T08:54:14.048 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.048 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.048 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:14.048 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.099956+0000", 2024-05-08T08:54:14.048 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1970999999999999e-05 2024-05-08T08:54:14.048 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.048 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.048 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:14.048 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.099968+0000", 2024-05-08T08:54:14.048 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.189e-05 2024-05-08T08:54:14.048 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.048 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.049 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:14.049 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140930+0000", 2024-05-08T08:54:14.049 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.040961806000000003 2024-05-08T08:54:14.049 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.049 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.049 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:14.049 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140974+0000", 2024-05-08T08:54:14.049 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3918999999999997e-05 2024-05-08T08:54:14.049 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.049 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.049 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:14.049 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.141013+0000", 2024-05-08T08:54:14.050 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8668999999999999e-05 2024-05-08T08:54:14.050 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:14.050 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:14.050 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:14.050 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.050 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.050 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22765 3.15 3:a9c637ae:::rbd_data.114071bcf828.000000000000013f:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:14.050 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.100902+0000", 2024-05-08T08:54:14.050 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.870494714000003, 2024-05-08T08:54:14.050 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042471271999999997, 2024-05-08T08:54:14.050 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:14.050 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:14.051 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:14.051 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:14.051 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:54:14.051 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:14.051 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22765 2024-05-08T08:54:14.051 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.051 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:14.051 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.051 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:14.051 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.100902+0000", 2024-05-08T08:54:14.051 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:14.051 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.052 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.052 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:14.052 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.100902+0000", 2024-05-08T08:54:14.052 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:14.052 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.052 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.052 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:14.052 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.100902+0000", 2024-05-08T08:54:14.052 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.39e-07 2024-05-08T08:54:14.052 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.052 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.052 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:14.052 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.100904+0000", 2024-05-08T08:54:14.052 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6479999999999999e-06 2024-05-08T08:54:14.053 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.053 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.053 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:14.053 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.100910+0000", 2024-05-08T08:54:14.053 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1399999999999997e-06 2024-05-08T08:54:14.053 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.053 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.053 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:14.053 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.100916+0000", 2024-05-08T08:54:14.053 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6999999999999996e-06 2024-05-08T08:54:14.053 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.053 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.054 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:14.054 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.141733+0000", 2024-05-08T08:54:14.054 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.040817271000000002 2024-05-08T08:54:14.054 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.054 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.054 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:14.054 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.141812+0000", 2024-05-08T08:54:14.054 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9270999999999994e-05 2024-05-08T08:54:14.054 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.054 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.054 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:14.055 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.143373+0000", 2024-05-08T08:54:14.055 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001560903 2024-05-08T08:54:14.055 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:14.055 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:14.055 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:14.055 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.055 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.055 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:23031 3.15 3:ac9fb9cd:::rbd_data.114071bcf828.00000000000000f7:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:14.055 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.113562+0000", 2024-05-08T08:54:14.055 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.857834953999998, 2024-05-08T08:54:14.055 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042560938999999999, 2024-05-08T08:54:14.056 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:14.056 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:14.056 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:54:14.056 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:14.056 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:54:14.056 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:14.056 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 23031 2024-05-08T08:54:14.056 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.056 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:14.056 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.056 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:14.056 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.113562+0000", 2024-05-08T08:54:14.057 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:14.057 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.057 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.057 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:14.057 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.113562+0000", 2024-05-08T08:54:14.057 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:14.057 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.057 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.057 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:14.058 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.113938+0000", 2024-05-08T08:54:14.058 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000376401 2024-05-08T08:54:14.058 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.058 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.058 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:14.058 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.113943+0000", 2024-05-08T08:54:14.058 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0529999999999997e-06 2024-05-08T08:54:14.058 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.058 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.058 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:14.059 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.113951+0000", 2024-05-08T08:54:14.059 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8569999999999995e-06 2024-05-08T08:54:14.059 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.059 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.059 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:14.059 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.113959+0000", 2024-05-08T08:54:14.059 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4710000000000001e-06 2024-05-08T08:54:14.059 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.059 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.060 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:14.060 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.153955+0000", 2024-05-08T08:54:14.060 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.039996747999999999 2024-05-08T08:54:14.060 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.060 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.060 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:14.060 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.154014+0000", 2024-05-08T08:54:14.060 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8464999999999997e-05 2024-05-08T08:54:14.060 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.060 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.061 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-05-08T08:54:14.061 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.154148+0000", 2024-05-08T08:54:14.061 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00013428600000000001 2024-05-08T08:54:14.061 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.061 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.061 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:54:14.061 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.155390+0000", 2024-05-08T08:54:14.061 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001241792 2024-05-08T08:54:14.061 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.061 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.061 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:54:14.061 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.156068+0000", 2024-05-08T08:54:14.062 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00067823700000000005 2024-05-08T08:54:14.062 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.062 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.062 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:54:14.062 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.156102+0000", 2024-05-08T08:54:14.062 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3812000000000001e-05 2024-05-08T08:54:14.062 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.062 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.062 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:14.062 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.156123+0000", 2024-05-08T08:54:14.062 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0817000000000001e-05 2024-05-08T08:54:14.063 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:14.063 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:14.063 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:14.063 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.063 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.063 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4416.0:20531 3.16 3:6bb2ed16:::rbd_object_map.1140bc4a55c7:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-05-08T08:54:14.063 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:58.673786+0000", 2024-05-08T08:54:14.063 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 15.297610604999999, 2024-05-08T08:54:14.063 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.94684119899999997, 2024-05-08T08:54:14.063 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:14.063 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:14.063 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:54:14.064 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:14.064 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4416", 2024-05-08T08:54:14.064 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:54:14.064 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20531 2024-05-08T08:54:14.064 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.064 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:14.064 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.064 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:14.064 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673786+0000", 2024-05-08T08:54:14.064 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:14.064 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.064 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.064 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:14.065 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673786+0000", 2024-05-08T08:54:14.065 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:14.065 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.065 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.065 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:14.065 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673787+0000", 2024-05-08T08:54:14.065 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9800000000000003e-07 2024-05-08T08:54:14.065 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.065 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.065 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:14.065 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673791+0000", 2024-05-08T08:54:14.065 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.104e-06 2024-05-08T08:54:14.065 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.065 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.065 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:14.066 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673801+0000", 2024-05-08T08:54:14.066 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9960000000000005e-06 2024-05-08T08:54:14.066 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.066 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.066 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:14.066 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673806+0000", 2024-05-08T08:54:14.066 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2320000000000001e-06 2024-05-08T08:54:14.066 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.066 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.066 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:14.066 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673860+0000", 2024-05-08T08:54:14.066 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3468999999999999e-05 2024-05-08T08:54:14.066 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.066 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.066 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-05-08T08:54:14.067 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673866+0000", 2024-05-08T08:54:14.067 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3500000000000002e-06 2024-05-08T08:54:14.067 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.067 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.067 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:14.067 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.616700+0000", 2024-05-08T08:54:14.067 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.94283390700000003 2024-05-08T08:54:14.067 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.067 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.067 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:14.067 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.616941+0000", 2024-05-08T08:54:14.067 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00024080400000000001 2024-05-08T08:54:14.067 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.067 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.067 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-05-08T08:54:14.067 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.617072+0000", 2024-05-08T08:54:14.068 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000131688 2024-05-08T08:54:14.068 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.068 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.068 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:54:14.068 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.619571+0000", 2024-05-08T08:54:14.068 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0024990149999999998 2024-05-08T08:54:14.068 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.068 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.068 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:54:14.068 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.620568+0000", 2024-05-08T08:54:14.068 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00099711000000000005 2024-05-08T08:54:14.068 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.068 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.068 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:54:14.068 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.620602+0000", 2024-05-08T08:54:14.068 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.324e-05 2024-05-08T08:54:14.069 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.069 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.069 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:14.069 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.620627+0000", 2024-05-08T08:54:14.069 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5585999999999999e-05 2024-05-08T08:54:14.069 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:14.069 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:14.069 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:14.069 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.069 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.069 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5423.0:429 3.f 3:f1d5704a:::rbd_data.1140cf33de9.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-05-08T08:54:14.069 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:59.659591+0000", 2024-05-08T08:54:14.069 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 14.31180612, 2024-05-08T08:54:14.069 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.044109585, 2024-05-08T08:54:14.069 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:14.070 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:14.070 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:54:14.070 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:14.070 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5423", 2024-05-08T08:54:14.070 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/306969889", 2024-05-08T08:54:14.070 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 429 2024-05-08T08:54:14.070 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.070 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:14.070 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.070 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:14.070 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.659591+0000", 2024-05-08T08:54:14.070 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:14.070 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.070 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.070 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:14.071 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.659591+0000", 2024-05-08T08:54:14.071 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:14.071 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.071 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.071 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:14.071 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.659591+0000", 2024-05-08T08:54:14.071 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7299999999999996e-07 2024-05-08T08:54:14.071 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.071 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.071 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:14.071 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.663640+0000", 2024-05-08T08:54:14.071 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0040485529999999999 2024-05-08T08:54:14.071 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.071 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.071 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:14.071 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.663655+0000", 2024-05-08T08:54:14.072 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5065000000000001e-05 2024-05-08T08:54:14.072 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.072 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.072 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:14.072 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.663661+0000", 2024-05-08T08:54:14.072 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4269999999999996e-06 2024-05-08T08:54:14.072 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.072 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.072 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:14.072 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.663714+0000", 2024-05-08T08:54:14.072 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2992999999999999e-05 2024-05-08T08:54:14.073 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.073 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.073 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:14.073 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.663798+0000", 2024-05-08T08:54:14.073 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3992999999999994e-05 2024-05-08T08:54:14.073 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.073 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.073 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-05-08T08:54:14.073 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.664125+0000", 2024-05-08T08:54:14.073 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00032663899999999999 2024-05-08T08:54:14.073 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.074 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.074 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:54:14.074 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.687754+0000", 2024-05-08T08:54:14.074 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.023629075999999999 2024-05-08T08:54:14.074 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.074 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.074 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:54:14.074 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.703569+0000", 2024-05-08T08:54:14.074 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.015814814999999999 2024-05-08T08:54:14.074 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.075 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.075 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:54:14.075 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.703641+0000", 2024-05-08T08:54:14.075 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2484000000000006e-05 2024-05-08T08:54:14.075 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.075 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.075 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:14.075 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.703700+0000", 2024-05-08T08:54:14.075 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8967e-05 2024-05-08T08:54:14.075 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:14.076 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:14.076 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:14.076 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:14.076 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:14.076 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:14.077 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:14.080+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:14.078 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:54:14.157 INFO:teuthology.orchestra.run.smithi196.stdout:checking clone #39, image image_client.0-clone39 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent40 2024-05-08T08:54:14.178 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:14.181+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:14.190 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:14.190 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:54:14.190 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:54:14.191 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:14.199 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:54:14.279 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:14.282+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:14.309 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T08:54:14.370 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:14.370 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:54:14.370 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:54:14.370 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:54:14.370 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:14.379 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:14.382+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:14.382 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:54:14.464 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:54:14.464 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:54:14.481 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:14.483+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:14.486 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:54:14.487 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:54:14.509 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:54:14.510 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:54:14.515 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:14.515 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:54:14.515 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:54:14.515 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:54:14.515 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.515 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22073 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 782336~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:14.515 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:54:14.515 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 106.65919354499999, 2024-05-08T08:54:14.515 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047244234000000003, 2024-05-08T08:54:14.516 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:14.516 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:14.516 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:14.516 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:14.516 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:14.516 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:14.516 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22073 2024-05-08T08:54:14.516 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.516 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:14.516 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.516 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:14.516 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:54:14.517 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:14.517 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.517 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.517 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:14.517 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:54:14.517 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:14.517 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.517 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.517 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:14.517 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857668+0000", 2024-05-08T08:54:14.517 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7800000000000002e-07 2024-05-08T08:54:14.517 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.517 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.517 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:14.518 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857670+0000", 2024-05-08T08:54:14.518 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0820000000000001e-06 2024-05-08T08:54:14.518 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.518 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.518 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:14.518 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857678+0000", 2024-05-08T08:54:14.518 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7589999999999997e-06 2024-05-08T08:54:14.518 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.518 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.518 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:14.518 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857691+0000", 2024-05-08T08:54:14.518 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3672999999999999e-05 2024-05-08T08:54:14.518 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.519 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.519 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:14.519 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904780+0000", 2024-05-08T08:54:14.519 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047088616999999999 2024-05-08T08:54:14.519 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.519 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.519 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:14.519 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904840+0000", 2024-05-08T08:54:14.519 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0356999999999998e-05 2024-05-08T08:54:14.519 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.519 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.519 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:14.519 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904912+0000", 2024-05-08T08:54:14.519 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1367999999999998e-05 2024-05-08T08:54:14.520 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:14.520 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:14.520 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:14.520 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.520 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.520 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22075 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 798720~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:14.520 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:54:14.520 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 106.659163148, 2024-05-08T08:54:14.520 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047384643999999997, 2024-05-08T08:54:14.520 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:14.520 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:14.520 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:14.520 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:14.520 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:14.521 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:14.521 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22075 2024-05-08T08:54:14.521 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.521 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:14.521 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.521 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:14.521 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:54:14.521 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:14.521 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.521 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.521 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:14.521 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:54:14.522 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:14.522 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.522 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.522 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:14.522 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:54:14.522 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7399999999999999e-07 2024-05-08T08:54:14.522 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.522 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.522 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:14.522 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857705+0000", 2024-05-08T08:54:14.522 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1980000000000004e-06 2024-05-08T08:54:14.522 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.523 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.523 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:14.523 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857713+0000", 2024-05-08T08:54:14.523 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7039999999999994e-06 2024-05-08T08:54:14.523 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.523 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.523 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:14.523 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857718+0000", 2024-05-08T08:54:14.523 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.865e-06 2024-05-08T08:54:14.523 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.523 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.523 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:14.523 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904941+0000", 2024-05-08T08:54:14.524 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047223198000000001 2024-05-08T08:54:14.524 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.524 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.524 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:14.524 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904999+0000", 2024-05-08T08:54:14.524 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8254999999999997e-05 2024-05-08T08:54:14.524 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.524 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.524 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:14.524 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905082+0000", 2024-05-08T08:54:14.524 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3049999999999999e-05 2024-05-08T08:54:14.524 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:14.524 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:14.525 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:14.525 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.525 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.525 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22077 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 815104~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:14.525 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:54:14.525 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 106.659125137, 2024-05-08T08:54:14.525 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047519448999999998, 2024-05-08T08:54:14.525 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:14.525 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:14.525 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:14.525 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:14.525 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:14.526 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:14.526 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22077 2024-05-08T08:54:14.526 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.526 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:14.526 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.526 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:14.526 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:54:14.526 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:14.526 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.526 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.526 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:14.526 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:54:14.526 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:14.527 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.527 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.527 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:14.527 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:54:14.527 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.99e-07 2024-05-08T08:54:14.527 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.527 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.527 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:14.527 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857739+0000", 2024-05-08T08:54:14.527 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3599999999999999e-06 2024-05-08T08:54:14.527 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.527 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.527 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:14.528 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857745+0000", 2024-05-08T08:54:14.528 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5139999999999998e-06 2024-05-08T08:54:14.528 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.528 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.528 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:14.528 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857750+0000", 2024-05-08T08:54:14.528 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8550000000000001e-06 2024-05-08T08:54:14.528 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.528 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.528 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:14.528 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905110+0000", 2024-05-08T08:54:14.528 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047360126000000002 2024-05-08T08:54:14.528 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.529 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.529 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:14.529 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905175+0000", 2024-05-08T08:54:14.529 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4689000000000004e-05 2024-05-08T08:54:14.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.529 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.529 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:14.529 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905255+0000", 2024-05-08T08:54:14.529 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0306e-05 2024-05-08T08:54:14.529 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:14.529 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:14.529 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:14.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.530 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.530 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22079 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 831488~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:14.530 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:54:14.530 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 106.65910458899999, 2024-05-08T08:54:14.530 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.04767499, 2024-05-08T08:54:14.530 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:14.530 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:14.530 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:14.530 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:14.530 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:14.530 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:14.530 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22079 2024-05-08T08:54:14.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.531 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:14.531 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.531 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:14.531 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:54:14.531 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:14.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.531 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.531 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:14.531 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:54:14.531 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:14.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.532 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.532 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:14.532 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857757+0000", 2024-05-08T08:54:14.532 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8299999999999997e-07 2024-05-08T08:54:14.532 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.532 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.532 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:14.532 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857759+0000", 2024-05-08T08:54:14.532 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1390000000000002e-06 2024-05-08T08:54:14.532 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.532 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.532 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:14.532 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857765+0000", 2024-05-08T08:54:14.533 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4609999999999996e-06 2024-05-08T08:54:14.533 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.533 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.533 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:14.533 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857770+0000", 2024-05-08T08:54:14.533 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6689999999999997e-06 2024-05-08T08:54:14.533 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.533 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.533 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:14.533 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905281+0000", 2024-05-08T08:54:14.533 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047510668999999998 2024-05-08T08:54:14.533 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.533 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.534 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:14.534 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905344+0000", 2024-05-08T08:54:14.534 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2797000000000002e-05 2024-05-08T08:54:14.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.534 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.534 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:14.534 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905431+0000", 2024-05-08T08:54:14.534 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7771999999999999e-05 2024-05-08T08:54:14.534 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:14.534 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:14.534 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:14.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.535 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.535 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22082 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 847872~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:14.535 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:54:14.535 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 106.659084898, 2024-05-08T08:54:14.535 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047831934999999999, 2024-05-08T08:54:14.535 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:14.535 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:14.535 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:14.535 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:14.535 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:14.535 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:14.535 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22082 2024-05-08T08:54:14.535 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.536 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:14.536 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.536 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:14.536 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:54:14.536 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:14.536 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.536 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.536 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:14.536 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:54:14.536 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:14.536 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.536 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.536 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:14.537 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:54:14.537 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:54:14.537 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.537 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.537 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:14.537 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857778+0000", 2024-05-08T08:54:14.537 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0789999999999999e-06 2024-05-08T08:54:14.537 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.537 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.537 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:14.537 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857784+0000", 2024-05-08T08:54:14.537 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0480000000000004e-06 2024-05-08T08:54:14.537 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.537 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.538 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:14.538 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857790+0000", 2024-05-08T08:54:14.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2719999999999997e-06 2024-05-08T08:54:14.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.538 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:14.538 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905458+0000", 2024-05-08T08:54:14.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047668470999999997 2024-05-08T08:54:14.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.538 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:14.538 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905521+0000", 2024-05-08T08:54:14.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2952000000000003e-05 2024-05-08T08:54:14.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.539 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.539 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:14.539 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905608+0000", 2024-05-08T08:54:14.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6759000000000001e-05 2024-05-08T08:54:14.539 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:14.539 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:14.539 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:14.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.539 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.539 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22084 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 864256~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:14.539 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:54:14.540 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 106.659060948, 2024-05-08T08:54:14.540 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047977748000000001, 2024-05-08T08:54:14.540 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:14.540 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:14.540 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:14.540 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:14.540 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:14.540 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:14.540 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22084 2024-05-08T08:54:14.540 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.540 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:14.540 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.540 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:14.540 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:54:14.541 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:14.541 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.541 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.541 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:14.541 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:54:14.541 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:14.541 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.541 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.541 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:14.541 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:54:14.541 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:54:14.541 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.541 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.542 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:14.542 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857802+0000", 2024-05-08T08:54:14.542 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0779999999999998e-06 2024-05-08T08:54:14.542 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.542 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.542 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:14.542 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857812+0000", 2024-05-08T08:54:14.542 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7939999999999997e-06 2024-05-08T08:54:14.542 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.542 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.542 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:14.542 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857817+0000", 2024-05-08T08:54:14.542 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8899999999999998e-06 2024-05-08T08:54:14.542 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.545 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:14.545 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905636+0000", 2024-05-08T08:54:14.545 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047819252 2024-05-08T08:54:14.545 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.545 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.545 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:14.545 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905700+0000", 2024-05-08T08:54:14.545 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4016999999999994e-05 2024-05-08T08:54:14.545 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.545 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.545 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:14.546 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905778+0000", 2024-05-08T08:54:14.546 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7306999999999997e-05 2024-05-08T08:54:14.546 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:14.546 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:14.546 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:14.546 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.546 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.546 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22086 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 880640~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:14.546 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:54:14.546 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 106.659037598, 2024-05-08T08:54:14.546 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048119149, 2024-05-08T08:54:14.546 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:14.546 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:14.547 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:14.547 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:14.547 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:14.547 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:14.547 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22086 2024-05-08T08:54:14.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.547 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:14.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.547 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:14.547 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:54:14.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:14.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:14.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:54:14.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:14.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:14.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857824+0000", 2024-05-08T08:54:14.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4200000000000002e-07 2024-05-08T08:54:14.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:14.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857826+0000", 2024-05-08T08:54:14.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0590000000000001e-06 2024-05-08T08:54:14.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.549 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:14.549 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857832+0000", 2024-05-08T08:54:14.549 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8029999999999999e-06 2024-05-08T08:54:14.549 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.549 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.549 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:14.549 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857837+0000", 2024-05-08T08:54:14.549 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7929999999999997e-06 2024-05-08T08:54:14.549 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.549 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.549 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:14.549 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905803+0000", 2024-05-08T08:54:14.550 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047965761000000003 2024-05-08T08:54:14.550 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.550 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.550 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:14.550 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905862+0000", 2024-05-08T08:54:14.550 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9203000000000002e-05 2024-05-08T08:54:14.550 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.550 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.550 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:14.550 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905942+0000", 2024-05-08T08:54:14.550 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0187999999999999e-05 2024-05-08T08:54:14.550 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:14.550 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:14.551 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:14.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.551 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22088 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 897024~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:14.551 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:54:14.551 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 106.659017546, 2024-05-08T08:54:14.551 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048256049000000002, 2024-05-08T08:54:14.551 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:14.551 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:14.551 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:14.551 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:14.551 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:14.551 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:14.551 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22088 2024-05-08T08:54:14.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.552 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:14.552 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.552 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:14.552 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:54:14.552 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:14.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.552 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.552 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:14.552 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:54:14.552 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:14.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.552 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.552 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:14.552 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857844+0000", 2024-05-08T08:54:14.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.46e-07 2024-05-08T08:54:14.553 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.553 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.553 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:14.553 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857846+0000", 2024-05-08T08:54:14.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0899999999999999e-06 2024-05-08T08:54:14.553 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.553 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.553 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:14.553 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857855+0000", 2024-05-08T08:54:14.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8219999999999996e-06 2024-05-08T08:54:14.553 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.553 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.554 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:14.554 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857860+0000", 2024-05-08T08:54:14.554 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2989999999999996e-06 2024-05-08T08:54:14.554 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.554 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.554 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:14.554 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905966+0000", 2024-05-08T08:54:14.554 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048106542000000002 2024-05-08T08:54:14.554 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.554 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.554 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:14.555 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906024+0000", 2024-05-08T08:54:14.555 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7958000000000003e-05 2024-05-08T08:54:14.555 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.555 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.555 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:14.555 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906099+0000", 2024-05-08T08:54:14.555 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4992000000000003e-05 2024-05-08T08:54:14.555 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:14.555 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:14.555 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:14.555 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.556 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.556 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22091 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 913408~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:14.556 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:54:14.556 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 106.658993786, 2024-05-08T08:54:14.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048395290000000001, 2024-05-08T08:54:14.556 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:14.556 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:14.556 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:14.556 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:14.556 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:14.556 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:14.556 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22091 2024-05-08T08:54:14.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.557 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:14.557 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.557 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:14.557 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:54:14.557 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:14.557 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.557 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.557 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:14.557 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:54:14.557 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:14.557 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.557 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.557 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:14.557 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:54:14.558 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4900000000000001e-07 2024-05-08T08:54:14.558 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.558 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.558 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:14.558 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857870+0000", 2024-05-08T08:54:14.558 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2510000000000001e-06 2024-05-08T08:54:14.558 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.558 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.558 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:14.558 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857877+0000", 2024-05-08T08:54:14.558 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0809999999999996e-06 2024-05-08T08:54:14.558 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.558 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.558 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:14.559 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857881+0000", 2024-05-08T08:54:14.559 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.724e-06 2024-05-08T08:54:14.559 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.559 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.559 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:14.559 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906127+0000", 2024-05-08T08:54:14.559 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048245345000000002 2024-05-08T08:54:14.559 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.559 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.559 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:14.559 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906190+0000", 2024-05-08T08:54:14.559 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2762000000000007e-05 2024-05-08T08:54:14.559 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.559 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.560 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:14.560 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906262+0000", 2024-05-08T08:54:14.560 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2778e-05 2024-05-08T08:54:14.560 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:14.560 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:14.560 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:14.560 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.560 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.560 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22093 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 929792~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:14.560 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:54:14.560 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 106.65897142199999, 2024-05-08T08:54:14.560 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048532875000000003, 2024-05-08T08:54:14.560 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:14.560 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:14.561 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:14.561 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:14.561 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:14.561 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:14.561 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22093 2024-05-08T08:54:14.561 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.561 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:14.561 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.561 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:14.561 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:54:14.561 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:14.561 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.561 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.561 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:14.562 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:54:14.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:14.562 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.562 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.562 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:14.562 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857890+0000", 2024-05-08T08:54:14.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-07 2024-05-08T08:54:14.562 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.562 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.562 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:14.562 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857899+0000", 2024-05-08T08:54:14.563 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6889999999999999e-06 2024-05-08T08:54:14.563 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.563 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.563 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:14.563 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857906+0000", 2024-05-08T08:54:14.563 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6490000000000002e-06 2024-05-08T08:54:14.563 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.563 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.563 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:14.563 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857917+0000", 2024-05-08T08:54:14.563 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0764999999999999e-05 2024-05-08T08:54:14.563 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.564 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.564 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:14.564 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906287+0000", 2024-05-08T08:54:14.564 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048370442 2024-05-08T08:54:14.564 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.564 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.564 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:14.564 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906349+0000", 2024-05-08T08:54:14.564 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1537000000000004e-05 2024-05-08T08:54:14.564 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.564 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.564 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:14.565 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906422+0000", 2024-05-08T08:54:14.565 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3431999999999998e-05 2024-05-08T08:54:14.565 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:14.565 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:14.565 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:14.565 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.565 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.565 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22095 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 946176~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:14.565 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:54:14.565 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 106.65893671800001, 2024-05-08T08:54:14.565 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048657593999999998, 2024-05-08T08:54:14.566 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:14.566 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:14.566 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:14.566 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:14.566 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:14.566 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:14.566 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22095 2024-05-08T08:54:14.566 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.566 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:14.566 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.566 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:14.566 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:54:14.566 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:14.567 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.567 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.567 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:14.567 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:54:14.567 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:14.567 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.567 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.567 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:14.567 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857925+0000", 2024-05-08T08:54:14.567 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3099999999999998e-07 2024-05-08T08:54:14.567 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.567 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.567 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:14.568 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857927+0000", 2024-05-08T08:54:14.568 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.34e-06 2024-05-08T08:54:14.568 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.568 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.568 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:14.568 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857938+0000", 2024-05-08T08:54:14.568 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1426e-05 2024-05-08T08:54:14.568 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.568 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.568 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:14.568 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857943+0000", 2024-05-08T08:54:14.568 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5140000000000003e-06 2024-05-08T08:54:14.568 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.568 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.569 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:14.569 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906448+0000", 2024-05-08T08:54:14.569 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048504641000000001 2024-05-08T08:54:14.569 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.569 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.569 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:14.569 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906510+0000", 2024-05-08T08:54:14.569 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2663000000000006e-05 2024-05-08T08:54:14.569 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.569 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.569 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:14.569 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906582+0000", 2024-05-08T08:54:14.569 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1478999999999998e-05 2024-05-08T08:54:14.569 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:14.570 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:14.570 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:14.570 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.570 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.570 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22097 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 962560~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:14.570 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:54:14.570 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 106.658909029, 2024-05-08T08:54:14.570 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048791353000000003, 2024-05-08T08:54:14.570 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:14.570 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:14.570 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:14.570 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:14.570 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:14.570 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:14.571 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22097 2024-05-08T08:54:14.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.571 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:14.571 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.571 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:14.571 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:54:14.571 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:14.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.571 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.571 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:14.571 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:54:14.571 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:14.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.571 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.571 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:14.572 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:54:14.572 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9200000000000002e-07 2024-05-08T08:54:14.572 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.572 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.572 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:14.572 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857954+0000", 2024-05-08T08:54:14.572 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1900000000000002e-06 2024-05-08T08:54:14.572 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.572 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.572 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:14.572 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857961+0000", 2024-05-08T08:54:14.572 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4989999999999999e-06 2024-05-08T08:54:14.572 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.572 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.573 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:14.573 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857971+0000", 2024-05-08T08:54:14.573 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0485999999999999e-05 2024-05-08T08:54:14.573 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.573 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.573 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:14.573 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906607+0000", 2024-05-08T08:54:14.573 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048635109000000003 2024-05-08T08:54:14.573 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.573 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.573 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:14.573 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906667+0000", 2024-05-08T08:54:14.573 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0439000000000002e-05 2024-05-08T08:54:14.573 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.573 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.574 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:14.574 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906743+0000", 2024-05-08T08:54:14.574 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6237999999999998e-05 2024-05-08T08:54:14.574 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:14.574 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:14.574 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:14.574 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.574 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.574 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22099 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 978944~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:14.574 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:54:14.574 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 106.658882869, 2024-05-08T08:54:14.574 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048932055000000002, 2024-05-08T08:54:14.574 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:14.575 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:14.575 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:14.575 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:14.575 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:14.575 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:14.575 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22099 2024-05-08T08:54:14.575 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.575 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:14.575 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.575 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:14.575 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:54:14.575 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:14.575 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.576 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.576 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:14.576 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:54:14.576 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:14.576 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.576 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.576 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:14.576 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:54:14.576 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.27e-07 2024-05-08T08:54:14.576 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.576 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.576 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:14.576 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857987+0000", 2024-05-08T08:54:14.577 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5890000000000003e-06 2024-05-08T08:54:14.577 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.577 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.577 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:14.577 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857994+0000", 2024-05-08T08:54:14.577 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6619999999999999e-06 2024-05-08T08:54:14.577 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.577 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.577 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:14.577 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857998+0000", 2024-05-08T08:54:14.577 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7110000000000003e-06 2024-05-08T08:54:14.577 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.577 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.577 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:14.578 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906772+0000", 2024-05-08T08:54:14.578 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048773415000000001 2024-05-08T08:54:14.578 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.578 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.578 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:14.578 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906834+0000", 2024-05-08T08:54:14.578 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1752000000000001e-05 2024-05-08T08:54:14.578 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.578 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.578 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:14.578 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906910+0000", 2024-05-08T08:54:14.578 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6499000000000001e-05 2024-05-08T08:54:14.579 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:14.579 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:14.579 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:14.579 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.579 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.579 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22101 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 995328~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:14.579 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:54:14.579 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 106.658855738, 2024-05-08T08:54:14.579 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049087364000000001, 2024-05-08T08:54:14.579 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:14.579 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:14.580 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:14.580 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:14.580 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:14.580 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:14.580 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22101 2024-05-08T08:54:14.580 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.580 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:14.580 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.580 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:14.580 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:54:14.580 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:14.580 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.580 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.581 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:14.581 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:54:14.581 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:14.581 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.581 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.581 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:14.581 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858006+0000", 2024-05-08T08:54:14.581 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1399999999999997e-07 2024-05-08T08:54:14.581 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.581 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.581 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:14.581 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858008+0000", 2024-05-08T08:54:14.581 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2060000000000001e-06 2024-05-08T08:54:14.582 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.582 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.582 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:14.582 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858017+0000", 2024-05-08T08:54:14.582 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1579999999999994e-06 2024-05-08T08:54:14.582 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.582 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.582 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:14.582 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858022+0000", 2024-05-08T08:54:14.582 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4199999999999998e-06 2024-05-08T08:54:14.582 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.582 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.582 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:14.583 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906940+0000", 2024-05-08T08:54:14.583 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048917627999999998 2024-05-08T08:54:14.583 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.583 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.583 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:14.583 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907002+0000", 2024-05-08T08:54:14.583 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2399999999999999e-05 2024-05-08T08:54:14.583 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.583 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.583 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:14.583 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907092+0000", 2024-05-08T08:54:14.583 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0137999999999997e-05 2024-05-08T08:54:14.584 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:14.584 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:14.584 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:14.584 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.584 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.584 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22103 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1011712~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:14.584 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:54:14.584 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 106.65883095300001, 2024-05-08T08:54:14.584 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049228542, 2024-05-08T08:54:14.584 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:14.584 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:14.584 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:14.584 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:14.585 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:14.585 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:14.585 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22103 2024-05-08T08:54:14.585 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.585 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:14.585 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.585 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:14.585 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:54:14.585 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:14.585 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.585 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.585 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:14.586 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:54:14.586 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:14.586 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.586 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.586 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:14.586 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:54:14.586 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4299999999999999e-07 2024-05-08T08:54:14.586 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.586 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.586 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:14.586 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858032+0000", 2024-05-08T08:54:14.586 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0590000000000001e-06 2024-05-08T08:54:14.586 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.587 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.587 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:14.587 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858037+0000", 2024-05-08T08:54:14.587 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.977e-06 2024-05-08T08:54:14.587 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.587 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.587 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:14.587 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858042+0000", 2024-05-08T08:54:14.587 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7650000000000001e-06 2024-05-08T08:54:14.587 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.587 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.587 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:14.588 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907118+0000", 2024-05-08T08:54:14.588 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049076147000000001 2024-05-08T08:54:14.588 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.588 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.588 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:14.588 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907180+0000", 2024-05-08T08:54:14.588 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1971999999999996e-05 2024-05-08T08:54:14.588 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.588 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.588 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:14.588 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907258+0000", 2024-05-08T08:54:14.588 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8279e-05 2024-05-08T08:54:14.589 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:14.589 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:14.589 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:14.589 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.589 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.589 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22105 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1028096~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:14.589 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:54:14.589 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 106.65881170999999, 2024-05-08T08:54:14.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049394609999999999, 2024-05-08T08:54:14.589 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:14.589 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:14.589 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:14.589 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:14.589 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:14.590 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:14.590 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22105 2024-05-08T08:54:14.590 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.590 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:14.590 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.590 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:14.590 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:54:14.590 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:14.590 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.590 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.590 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:14.591 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:54:14.591 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:14.591 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.591 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.591 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:14.591 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858050+0000", 2024-05-08T08:54:14.591 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5999999999999999e-07 2024-05-08T08:54:14.591 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.591 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.591 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:14.591 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858051+0000", 2024-05-08T08:54:14.591 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8870000000000001e-06 2024-05-08T08:54:14.591 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.591 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.592 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:14.592 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858057+0000", 2024-05-08T08:54:14.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8019999999999997e-06 2024-05-08T08:54:14.592 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.592 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.592 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:14.592 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858062+0000", 2024-05-08T08:54:14.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.561e-06 2024-05-08T08:54:14.592 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.592 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.592 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:14.592 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907289+0000", 2024-05-08T08:54:14.593 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049226957000000002 2024-05-08T08:54:14.593 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.593 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.593 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:14.593 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907356+0000", 2024-05-08T08:54:14.593 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6915000000000006e-05 2024-05-08T08:54:14.593 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.593 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.593 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:14.593 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907444+0000", 2024-05-08T08:54:14.593 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8127999999999994e-05 2024-05-08T08:54:14.593 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:14.594 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:14.594 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:14.594 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.594 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.594 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22107 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1044480~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:14.594 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:54:14.594 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 106.65879337, 2024-05-08T08:54:14.594 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049799942999999999, 2024-05-08T08:54:14.594 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:14.594 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:14.594 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:14.594 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:14.595 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:14.595 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:14.595 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22107 2024-05-08T08:54:14.595 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.595 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:14.595 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.595 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:14.595 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:54:14.595 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:14.595 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.595 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.595 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:14.595 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:54:14.596 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:14.596 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.596 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:14.596 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:54:14.596 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4799999999999999e-07 2024-05-08T08:54:14.596 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.596 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:14.596 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858070+0000", 2024-05-08T08:54:14.596 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.012e-06 2024-05-08T08:54:14.596 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.596 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:14.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858077+0000", 2024-05-08T08:54:14.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6290000000000001e-06 2024-05-08T08:54:14.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.597 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:14.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858083+0000", 2024-05-08T08:54:14.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8880000000000002e-06 2024-05-08T08:54:14.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.597 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:14.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907671+0000", 2024-05-08T08:54:14.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049587191000000003 2024-05-08T08:54:14.598 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:14.598 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907763+0000", 2024-05-08T08:54:14.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2231000000000005e-05 2024-05-08T08:54:14.598 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:14.598 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907867+0000", 2024-05-08T08:54:14.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104644 2024-05-08T08:54:14.598 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:14.599 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:14.599 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:14.599 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.599 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.599 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22294 3.1c 3:3aad0389:::rbd_data.11406886b07b.0000000000000080:head [stat,delete] snapc 7=[7] RETRY=6 ondisk+retry+write+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:14.599 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:54:14.599 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 106.658199522, 2024-05-08T08:54:14.599 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051152191, 2024-05-08T08:54:14.599 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:14.599 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:14.599 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:14.599 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:14.600 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:14.600 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:14.600 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22294 2024-05-08T08:54:14.600 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.600 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:14.600 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.600 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:14.600 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:54:14.600 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:14.600 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.600 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.600 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:14.600 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:54:14.600 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:14.601 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.601 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:14.601 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858662+0000", 2024-05-08T08:54:14.601 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4799999999999998e-07 2024-05-08T08:54:14.601 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.601 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:14.601 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858664+0000", 2024-05-08T08:54:14.601 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1229999999999998e-06 2024-05-08T08:54:14.601 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.601 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:14.601 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858670+0000", 2024-05-08T08:54:14.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3099999999999997e-06 2024-05-08T08:54:14.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:14.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858676+0000", 2024-05-08T08:54:14.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9750000000000004e-06 2024-05-08T08:54:14.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:14.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907901+0000", 2024-05-08T08:54:14.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049224520000000001 2024-05-08T08:54:14.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.603 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:14.603 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.908085+0000", 2024-05-08T08:54:14.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00018377500000000001 2024-05-08T08:54:14.603 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.603 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:14.603 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.909814+0000", 2024-05-08T08:54:14.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00172894 2024-05-08T08:54:14.603 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:14.603 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:14.603 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:14.603 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.604 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.604 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4416.0:13461 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-05-08T08:54:14.604 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:22.487456+0000", 2024-05-08T08:54:14.604 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 52.029404481, 2024-05-08T08:54:14.604 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.99102620799999996, 2024-05-08T08:54:14.604 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:14.604 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:14.604 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:14.604 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:14.604 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4416", 2024-05-08T08:54:14.604 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:54:14.604 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 13461 2024-05-08T08:54:14.604 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.605 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:14.605 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.605 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:14.605 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487456+0000", 2024-05-08T08:54:14.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:14.605 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.605 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.605 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:14.605 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487456+0000", 2024-05-08T08:54:14.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:14.605 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.605 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.605 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:14.605 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487458+0000", 2024-05-08T08:54:14.606 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5659999999999999e-06 2024-05-08T08:54:14.606 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.606 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:14.606 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487463+0000", 2024-05-08T08:54:14.606 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6890000000000003e-06 2024-05-08T08:54:14.606 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.606 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:14.606 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487472+0000", 2024-05-08T08:54:14.606 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7710000000000001e-06 2024-05-08T08:54:14.606 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.607 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:14.607 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487478+0000", 2024-05-08T08:54:14.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1390000000000003e-06 2024-05-08T08:54:14.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.607 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:14.607 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487520+0000", 2024-05-08T08:54:14.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2259999999999999e-05 2024-05-08T08:54:14.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.607 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-05-08T08:54:14.608 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487528+0000", 2024-05-08T08:54:14.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7479999999999993e-06 2024-05-08T08:54:14.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.608 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:14.608 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:23.478139+0000", 2024-05-08T08:54:14.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.99061165200000001 2024-05-08T08:54:14.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.608 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:14.608 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:23.478301+0000", 2024-05-08T08:54:14.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000161602 2024-05-08T08:54:14.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.609 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:14.609 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:23.478483+0000", 2024-05-08T08:54:14.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000181781 2024-05-08T08:54:14.609 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:14.609 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:14.609 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:14.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.609 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5752.0:404 3.1a 3:5f5e8104:::rbd_data.1140bc4a55c7.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 5175296~45056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-05-08T08:54:14.609 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:00.390863+0000", 2024-05-08T08:54:14.610 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 14.125998286, 2024-05-08T08:54:14.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.055189174000000001, 2024-05-08T08:54:14.610 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:14.610 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:14.610 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:54:14.610 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:14.610 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5752", 2024-05-08T08:54:14.610 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/3534419815", 2024-05-08T08:54:14.610 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 404 2024-05-08T08:54:14.610 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.611 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:14.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.611 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:14.611 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.390863+0000", 2024-05-08T08:54:14.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:14.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.611 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:14.611 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.390863+0000", 2024-05-08T08:54:14.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:14.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.611 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:14.612 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.390864+0000", 2024-05-08T08:54:14.612 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.471e-06 2024-05-08T08:54:14.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.612 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:14.612 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397475+0000", 2024-05-08T08:54:14.612 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0066109109999999997 2024-05-08T08:54:14.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.612 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:14.612 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397498+0000", 2024-05-08T08:54:14.612 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2674e-05 2024-05-08T08:54:14.613 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.613 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.613 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:14.613 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397508+0000", 2024-05-08T08:54:14.613 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9019999999999994e-06 2024-05-08T08:54:14.613 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.613 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.613 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:14.613 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397609+0000", 2024-05-08T08:54:14.613 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010121899999999999 2024-05-08T08:54:14.613 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.613 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.614 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:14.614 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397711+0000", 2024-05-08T08:54:14.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102489 2024-05-08T08:54:14.614 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.614 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.614 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-05-08T08:54:14.614 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397999+0000", 2024-05-08T08:54:14.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00028733799999999999 2024-05-08T08:54:14.614 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.614 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.614 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:54:14.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.432093+0000", 2024-05-08T08:54:14.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.034094865000000002 2024-05-08T08:54:14.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:54:14.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.445975+0000", 2024-05-08T08:54:14.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013881187 2024-05-08T08:54:14.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:54:14.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.446013+0000", 2024-05-08T08:54:14.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8155000000000003e-05 2024-05-08T08:54:14.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:14.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:14.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:14.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.446052+0000", 2024-05-08T08:54:14.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8962999999999999e-05 2024-05-08T08:54:14.616 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:14.616 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:14.616 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:14.616 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:14.616 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:14.616 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:14.618 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:54:14.618 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:54:14.619 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:54:14.619 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:54:14.619 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:54:14.620 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:54:14.621 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:54:14.621 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:14.620+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:14.716 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-05-08T08:54:14.719 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:14.722+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:14.729 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:14.730 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:14.738 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:54:14.819 INFO:teuthology.orchestra.run.smithi196.stdout:5482 read 0x4659200 thru 0x4661786 (0x8587 bytes) 2024-05-08T08:54:14.821 INFO:teuthology.orchestra.run.smithi196.stdout:5483 write 0x681e600 thru 0x682406a (0x5a6b bytes) 2024-05-08T08:54:14.822 INFO:teuthology.orchestra.run.smithi196.stdout:5484 write 0xd786800 thru 0xd793516 (0xcd17 bytes) 2024-05-08T08:54:14.823 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:14.826+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:14.833 INFO:teuthology.orchestra.run.smithi196.stdout:5485 trunc from 0xe881c00 to 0x547b600 2024-05-08T08:54:14.898 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:14.898 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:14.907 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:54:14.908 INFO:teuthology.orchestra.run.smithi196.stdout:5486 trunc from 0x547b600 to 0xabcfa00 2024-05-08T08:54:14.942 INFO:teuthology.orchestra.run.smithi196.stdout:5487 write 0xd10ea00 thru 0xd11beac (0xd4ad bytes) 2024-05-08T08:54:14.942 INFO:teuthology.orchestra.run.smithi196.stdout:5488 read 0xa934600 thru 0xa936a7f (0x2480 bytes) 2024-05-08T08:54:14.942 INFO:teuthology.orchestra.run.smithi196.stdout:5493 read 0x740a00 thru 0x741491 (0xa92 bytes) 2024-05-08T08:54:14.948 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:14.951+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:14.952 INFO:teuthology.orchestra.run.smithi196.stdout:5494 trunc from 0xd11c000 to 0x4452000 2024-05-08T08:54:15.021 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:15.023+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:15.035 INFO:teuthology.orchestra.run.smithi196.stdout:5495 write 0x4ebc200 thru 0x4ebd9e0 (0x17e1 bytes) 2024-05-08T08:54:15.061 INFO:teuthology.orchestra.run.smithi196.stdout:5496 write 0xad98000 thru 0xad9e45d (0x645e bytes) 2024-05-08T08:54:15.061 INFO:teuthology.orchestra.run.smithi196.stdout:5497 punch from 0x2ff4200 to 0x2ffae00, (0x6c00 bytes) 2024-05-08T08:54:15.062 INFO:teuthology.orchestra.run.smithi196.stdout:5499 punch from 0x2e21200 to 0x2e29000, (0x7e00 bytes) 2024-05-08T08:54:15.062 INFO:teuthology.orchestra.run.smithi196.stdout:5502 write 0x8a05200 thru 0x8a14a08 (0xf809 bytes) 2024-05-08T08:54:15.062 INFO:teuthology.orchestra.run.smithi196.stdout:5504 flatten 2024-05-08T08:54:15.074 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:15.074 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:15.088 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:54:15.119 INFO:teuthology.orchestra.run.smithi196.stdout:5505 trunc from 0xad9e600 to 0x687f800 2024-05-08T08:54:15.122 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:15.125+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:15.139 INFO:teuthology.orchestra.run.smithi196.stdout:5508 read 0x60bc400 thru 0x60c368e (0x728f bytes) 2024-05-08T08:54:15.155 INFO:teuthology.orchestra.run.smithi196.stdout:5509 trunc from 0x687f800 to 0xe42ae00 2024-05-08T08:54:15.164 INFO:teuthology.orchestra.run.smithi196.stdout:5511 read 0xaa89e00 thru 0xaa8a868 (0xa69 bytes) 2024-05-08T08:54:15.171 INFO:teuthology.orchestra.run.smithi196.stdout:5512 write 0x4375600 thru 0x437688f (0x1290 bytes) 2024-05-08T08:54:15.171 INFO:teuthology.orchestra.run.smithi196.stdout:5513 read 0x1770000 thru 0x17724dc (0x24dd bytes) 2024-05-08T08:54:15.171 INFO:teuthology.orchestra.run.smithi196.stdout:5514 read 0x2f98000 thru 0x2f9890f (0x910 bytes) 2024-05-08T08:54:15.172 INFO:teuthology.orchestra.run.smithi196.stdout:5519 trunc from 0xe42ae00 to 0xe945c00 2024-05-08T08:54:15.184 INFO:teuthology.orchestra.run.smithi196.stdout:5520 punch from 0x3d5b600 to 0x3d5e000, (0x2a00 bytes) 2024-05-08T08:54:15.186 INFO:teuthology.orchestra.run.smithi196.stdout:5521 read 0x7f07200 thru 0x7f07306 (0x107 bytes) 2024-05-08T08:54:15.186 INFO:teuthology.orchestra.run.smithi196.stdout:5522 read 0x96cbe00 thru 0x96cdd90 (0x1f91 bytes) 2024-05-08T08:54:15.186 INFO:teuthology.orchestra.run.smithi196.stdout:5523 read 0xd03ca00 thru 0xd04c606 (0xfc07 bytes) 2024-05-08T08:54:15.194 INFO:teuthology.orchestra.run.smithi196.stdout:5524 trunc from 0xe945c00 to 0x7ec0800 2024-05-08T08:54:15.206 INFO:teuthology.orchestra.run.smithi196.stdout:5525 read 0x6a0000 thru 0x6a4a0f (0x4a10 bytes) 2024-05-08T08:54:15.213 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:15.213 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:15.215 INFO:teuthology.orchestra.run.smithi196.stdout:5526 trunc from 0x7ec0800 to 0x6f9f400 2024-05-08T08:54:15.223 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:54:15.224 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:15.226+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:15.226 INFO:teuthology.orchestra.run.smithi196.stdout:5527 punch from 0xd2a400 to 0xd2f600, (0x5200 bytes) 2024-05-08T08:54:15.248 INFO:teuthology.orchestra.run.smithi196.stdout:5528 write 0x8292000 thru 0x829cc4b (0xac4c bytes) 2024-05-08T08:54:15.258 INFO:teuthology.orchestra.run.smithi196.stdout:5530 trunc from 0x829ce00 to 0x592ae00 2024-05-08T08:54:15.278 INFO:teuthology.orchestra.run.smithi196.stdout:5532 write 0x3b8c00 thru 0x3c0bec (0x7fed bytes) 2024-05-08T08:54:15.309 INFO:teuthology.orchestra.run.smithi196.stdout:5534 write 0xa9d0000 thru 0xa9d2acf (0x2ad0 bytes) 2024-05-08T08:54:15.323 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:15.326+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:15.333 INFO:teuthology.orchestra.run.smithi196.stdout:5537 write 0xb66f000 thru 0xb679db2 (0xadb3 bytes) 2024-05-08T08:54:15.340 INFO:teuthology.orchestra.run.smithi196.stdout:5539 read 0x5888600 thru 0x588a4e2 (0x1ee3 bytes) 2024-05-08T08:54:15.340 INFO:teuthology.orchestra.run.smithi196.stdout:5540 read 0x5570200 thru 0x5574270 (0x4071 bytes) 2024-05-08T08:54:15.341 INFO:teuthology.orchestra.run.smithi196.stdout:5541 read 0x2b78c00 thru 0x2b877c8 (0xebc9 bytes) 2024-05-08T08:54:15.341 INFO:teuthology.orchestra.run.smithi196.stdout:5544 write 0xa8a7400 thru 0xa8a8ea9 (0x1aaa bytes) 2024-05-08T08:54:15.342 INFO:teuthology.orchestra.run.smithi196.stdout:5545 write 0xf6e000 thru 0xf72a31 (0x4a32 bytes) 2024-05-08T08:54:15.345 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:15.345 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:15.356 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:54:15.367 INFO:teuthology.orchestra.run.smithi196.stdout:5546 write 0xc9cac00 thru 0xc9d5787 (0xab88 bytes) 2024-05-08T08:54:15.382 INFO:teuthology.orchestra.run.smithi196.stdout:5548 write 0xe4f6000 thru 0xe502ea8 (0xcea9 bytes) 2024-05-08T08:54:15.383 INFO:teuthology.orchestra.run.smithi196.stdout:5555 read 0xc4d0000 thru 0xc4de231 (0xe232 bytes) 2024-05-08T08:54:15.383 INFO:teuthology.orchestra.run.smithi196.stdout:5557 punch from 0xd529e00 to 0xd534c00, (0xae00 bytes) 2024-05-08T08:54:15.387 INFO:teuthology.orchestra.run.smithi196.stdout:5558 read 0xb8a2a00 thru 0xb8adfc4 (0xb5c5 bytes) 2024-05-08T08:54:15.388 INFO:teuthology.orchestra.run.smithi196.stdout:5559 write 0x4305400 thru 0x430d8d9 (0x84da bytes) 2024-05-08T08:54:15.388 INFO:teuthology.orchestra.run.smithi196.stdout:5561 write 0x6663c00 thru 0x6672dbc (0xf1bd bytes) 2024-05-08T08:54:15.388 INFO:teuthology.orchestra.run.smithi196.stdout:5565 clone 42 order 21 su 1048576 sc 14 2024-05-08T08:54:15.440 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:15.443+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:15.506 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:15.506 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:15.515 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:54:15.556 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:15.559+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:15.626 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:15.629+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:15.665 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:15.666 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:15.674 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:54:15.708 INFO:teuthology.orchestra.run.smithi059.stderr:nodeep-scrub is set 2024-05-08T08:54:15.727 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:15.730+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:15.794 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:15.794 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:15.803 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:54:15.828 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:15.831+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:15.912 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:15.912 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:15.921 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:54:15.965 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:15.968+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:16.030 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:16.032+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:16.035 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:16.035 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:16.045 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:54:16.058 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4] out_osds: [5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-05-08T08:54:16.058 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T08:54:16.058 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T08:54:16.131 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:16.134+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:16.149 INFO:teuthology.orchestra.run.smithi196.stdout:checking clone #40, image image_client.0-clone40 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent41 2024-05-08T08:54:16.197 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:16.198 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:16.207 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:54:16.231 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:16.234+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:16.333 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:16.335+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:16.369 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:16.369 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:16.378 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:54:16.419 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:54:16.419 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-05-08T08:54:16.431 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":171,"stamp":"2024-05-08T08:54:15.763312+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1153096210,"num_objects":2904,"num_object_clones":438,"num_object_copies":5808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2904,"num_whiteouts":1633,"num_read":458532,"num_read_kb":1411257,"num_write":20966,"num_write_kb":423632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2942,"num_bytes_recovered":182360369,"num_keys_recovered":458,"num_objects_omap":43,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18886,"ondisk_log_size":18886,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":471859200,"kb_used":338804,"kb_used_data":80204,"kb_used_omap":7,"kb_used_meta":258552,"kb_avail":471520396,"statfs":{"total":483183820800,"available":482836885504,"internally_reserved":0,"allocated":82128896,"data_stored":641316662,"data_compressed":32607555,"data_compressed_allocated":62259200,"data_compressed_original":630414848,"omap_allocated":7973,"internal_metadata":264757467},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":48114523,"num_objects":125,"num_object_clones":8,"num_object_copies":250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":125,"num_whiteouts":14,"num_read":34603,"num_read_kb":86310,"num_write":3000,"num_write_kb":13491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000812"},"pg_stats":[{"pgid":"3.1f","version":"104'626","reported_seq":19129,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-08T08:54:12.712111+0000","last_change":"2024-05-08T08:53:49.581255+0000","last_active":"2024-05-08T08:54:12.712111+0000","last_peered":"2024-05-08T08:54:12.712111+0000","last_clean":"2024-05-08T08:54:12.712111+0000","last_became_active":"2024-05-08T08:53:49.580563+0000","last_became_peered":"2024-05-08T08:53:49.580563+0000","last_unstale":"2024-05-08T08:54:12.712111+0000","last_undegraded":"2024-05-08T08:54:12.712111+0000","last_fullsized":"2024-05-08T08:54:12.712111+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":626,"log_dups_size":0,"ondisk_log_size":626,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:32:38.412178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000691565,"stat_sum":{"num_bytes":64664935,"num_objects":94,"num_object_clones":17,"num_object_copies":188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":44,"num_read":18350,"num_read_kb":78433,"num_write":873,"num_write_kb":32844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"105'777","reported_seq":10953,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-08T08:54:12.765492+0000","last_change":"2024-05-08T08:52:13.944947+0000","last_active":"2024-05-08T08:54:12.765492+0000","last_peered":"2024-05-08T08:54:12.765492+0000","last_clean":"2024-05-08T08:54:12.765492+0000","last_became_active":"2024-05-08T08:52:11.930844+0000","last_became_peered":"2024-05-08T08:52:11.930844+0000","last_unstale":"2024-05-08T08:54:12.765492+0000","last_undegraded":"2024-05-08T08:54:12.765492+0000","last_fullsized":"2024-05-08T08:54:12.765492+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":777,"log_dups_size":0,"ondisk_log_size":777,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:32:01.380277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040424900000000003,"stat_sum":{"num_bytes":20532352,"num_objects":94,"num_object_clones":12,"num_object_copies":188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":60,"num_read":12657,"num_read_kb":34912,"num_write":599,"num_write_kb":12172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"105'556","reported_seq":2393,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-08T08:54:12.753176+0000","last_change":"2024-05-08T08:53:59.618675+0000","last_active":"2024-05-08T08:54:12.753176+0000","last_peered":"2024-05-08T08:54:12.753176+0000","last_clean":"2024-05-08T08:54:12.753176+0000","last_became_active":"2024-05-08T08:53:59.618085+0000","last_became_peered":"2024-05-08T08:53:59.618085+0000","last_unstale":"2024-05-08T08:54:12.753176+0000","last_undegraded":"2024-05-08T08:54:12.753176+0000","last_fullsized":"2024-05-08T08:54:12.753176+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":556,"log_dups_size":0,"ondisk_log_size":556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:49:24.185887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072111799999999998,"stat_sum":{"num_bytes":14415646,"num_objects":83,"num_object_clones":11,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":46,"num_read":6210,"num_read_kb":22235,"num_write":755,"num_write_kb":6764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":141,"num_bytes_recovered":5594929,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"105'522","reported_seq":16603,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-08T08:54:12.731500+0000","last_change":"2024-05-08T08:53:58.622642+0000","last_active":"2024-05-08T08:54:12.731500+0000","last_peered":"2024-05-08T08:54:12.731500+0000","last_clean":"2024-05-08T08:54:12.731500+0000","last_became_active":"2024-05-08T08:53:58.621483+0000","last_became_peered":"2024-05-08T08:53:58.621483+0000","last_unstale":"2024-05-08T08:54:12.731500+0000","last_undegraded":"2024-05-08T08:54:12.731500+0000","last_fullsized":"2024-05-08T08:54:12.731500+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":522,"log_dups_size":0,"ondisk_log_size":522,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:52:39.648052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094844199999999997,"stat_sum":{"num_bytes":55879221,"num_objects":95,"num_object_clones":11,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":54,"num_read":16542,"num_read_kb":88441,"num_write":574,"num_write_kb":26287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":287,"num_bytes_recovered":49590415,"num_keys_recovered":166,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"104'511","reported_seq":24411,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-08T08:54:12.721152+0000","last_change":"2024-05-08T08:53:49.585842+0000","last_active":"2024-05-08T08:54:12.721152+0000","last_peered":"2024-05-08T08:54:12.721152+0000","last_clean":"2024-05-08T08:54:12.721152+0000","last_became_active":"2024-05-08T08:53:49.583097+0000","last_became_peered":"2024-05-08T08:53:49.583097+0000","last_unstale":"2024-05-08T08:54:12.721152+0000","last_undegraded":"2024-05-08T08:54:12.721152+0000","last_fullsized":"2024-05-08T08:54:12.721152+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:03:53.915325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000685079,"stat_sum":{"num_bytes":47790336,"num_objects":100,"num_object_clones":13,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":62,"num_read":24116,"num_read_kb":39847,"num_write":586,"num_write_kb":22732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":129,"num_bytes_recovered":3090520,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"105'601","reported_seq":19364,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-08T08:54:12.761795+0000","last_change":"2024-05-08T08:53:24.485198+0000","last_active":"2024-05-08T08:54:12.761795+0000","last_peered":"2024-05-08T08:54:12.761795+0000","last_clean":"2024-05-08T08:54:12.761795+0000","last_became_active":"2024-05-08T08:53:24.484124+0000","last_became_peered":"2024-05-08T08:53:24.484124+0000","last_unstale":"2024-05-08T08:54:12.761795+0000","last_undegraded":"2024-05-08T08:54:12.761795+0000","last_fullsized":"2024-05-08T08:54:12.761795+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":601,"log_dups_size":0,"ondisk_log_size":601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:55:48.349495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066525600000000003,"stat_sum":{"num_bytes":27794812,"num_objects":96,"num_object_clones":16,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":47,"num_read":20937,"num_read_kb":54912,"num_write":731,"num_write_kb":18868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":130,"num_bytes_recovered":8549042,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"104'344","reported_seq":9587,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-08T08:54:12.704813+0000","last_change":"2024-05-08T08:53:24.484713+0000","last_active":"2024-05-08T08:54:12.704813+0000","last_peered":"2024-05-08T08:54:12.704813+0000","last_clean":"2024-05-08T08:54:12.704813+0000","last_became_active":"2024-05-08T08:53:24.483403+0000","last_became_peered":"2024-05-08T08:53:24.483403+0000","last_unstale":"2024-05-08T08:54:12.704813+0000","last_undegraded":"2024-05-08T08:54:12.704813+0000","last_fullsized":"2024-05-08T08:54:12.704813+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":344,"log_dups_size":0,"ondisk_log_size":344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:01:07.321189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082481000000000004,"stat_sum":{"num_bytes":20318328,"num_objects":90,"num_object_clones":12,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":56,"num_read":8755,"num_read_kb":41829,"num_write":307,"num_write_kb":15945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":8814624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"104'463","reported_seq":16995,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-08T08:54:12.708929+0000","last_change":"2024-05-08T08:52:13.945184+0000","last_active":"2024-05-08T08:54:12.708929+0000","last_peered":"2024-05-08T08:54:12.708929+0000","last_clean":"2024-05-08T08:54:12.708929+0000","last_became_active":"2024-05-08T08:52:11.927366+0000","last_became_peered":"2024-05-08T08:52:11.927366+0000","last_unstale":"2024-05-08T08:54:12.708929+0000","last_undegraded":"2024-05-08T08:54:12.708929+0000","last_fullsized":"2024-05-08T08:54:12.708929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:33:01.754556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057208900000000002,"stat_sum":{"num_bytes":80472280,"num_objects":100,"num_object_clones":19,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":57,"num_read":17053,"num_read_kb":43815,"num_write":512,"num_write_kb":22239,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"104'722","reported_seq":17771,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-08T08:54:12.704926+0000","last_change":"2024-05-08T08:53:11.359646+0000","last_active":"2024-05-08T08:54:12.704926+0000","last_peered":"2024-05-08T08:54:12.704926+0000","last_clean":"2024-05-08T08:54:12.704926+0000","last_became_active":"2024-05-08T08:53:11.358391+0000","last_became_peered":"2024-05-08T08:53:11.358391+0000","last_unstale":"2024-05-08T08:54:12.704926+0000","last_undegraded":"2024-05-08T08:54:12.704926+0000","last_fullsized":"2024-05-08T08:54:12.704926+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":722,"log_dups_size":0,"ondisk_log_size":722,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:09:54.184834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000866371,"stat_sum":{"num_bytes":58717330,"num_objects":82,"num_object_clones":13,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":39,"num_read":19629,"num_read_kb":70375,"num_write":863,"num_write_kb":16290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":97,"num_bytes_recovered":2508994,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"105'1327","reported_seq":19542,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-08T08:54:12.712352+0000","last_change":"2024-05-08T08:53:59.617614+0000","last_active":"2024-05-08T08:54:12.712352+0000","last_peered":"2024-05-08T08:54:12.712352+0000","last_clean":"2024-05-08T08:54:12.712352+0000","last_became_active":"2024-05-08T08:53:59.615917+0000","last_became_peered":"2024-05-08T08:53:59.615917+0000","last_unstale":"2024-05-08T08:54:12.712352+0000","last_undegraded":"2024-05-08T08:54:12.712352+0000","last_fullsized":"2024-05-08T08:54:12.712352+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":1327,"log_dups_size":0,"ondisk_log_size":1327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:28:51.272442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084652399999999997,"stat_sum":{"num_bytes":40021352,"num_objects":82,"num_object_clones":12,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":41,"num_read":23668,"num_read_kb":74308,"num_write":2146,"num_write_kb":25802,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":118,"num_bytes_recovered":21349055,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"104'467","reported_seq":14919,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-08T08:54:12.712008+0000","last_change":"2024-05-08T08:52:13.944999+0000","last_active":"2024-05-08T08:54:12.712008+0000","last_peered":"2024-05-08T08:54:12.712008+0000","last_clean":"2024-05-08T08:54:12.712008+0000","last_became_active":"2024-05-08T08:52:11.931695+0000","last_became_peered":"2024-05-08T08:52:11.931695+0000","last_unstale":"2024-05-08T08:54:12.712008+0000","last_undegraded":"2024-05-08T08:54:12.712008+0000","last_fullsized":"2024-05-08T08:54:12.712008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":467,"log_dups_size":0,"ondisk_log_size":467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:12:24.947593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063902800000000004,"stat_sum":{"num_bytes":25074782,"num_objects":83,"num_object_clones":11,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":50,"num_read":15778,"num_read_kb":40841,"num_write":513,"num_write_kb":6413,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"104'579","reported_seq":11224,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-08T08:54:12.712072+0000","last_change":"2024-05-08T08:53:23.481964+0000","last_active":"2024-05-08T08:54:12.712072+0000","last_peered":"2024-05-08T08:54:12.712072+0000","last_clean":"2024-05-08T08:54:12.712072+0000","last_became_active":"2024-05-08T08:53:23.481362+0000","last_became_peered":"2024-05-08T08:53:23.481362+0000","last_unstale":"2024-05-08T08:54:12.712072+0000","last_undegraded":"2024-05-08T08:54:12.712072+0000","last_fullsized":"2024-05-08T08:54:12.712072+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":579,"log_dups_size":0,"ondisk_log_size":579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:16:19.912147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000430331,"stat_sum":{"num_bytes":35524723,"num_objects":82,"num_object_clones":16,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":43,"num_read":9638,"num_read_kb":27118,"num_write":551,"num_write_kb":10214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":3612768,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":198,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-08T08:54:12.712271+0000","last_change":"2024-05-08T08:53:20.362387+0000","last_active":"2024-05-08T08:54:12.712271+0000","last_peered":"2024-05-08T08:54:12.712271+0000","last_clean":"2024-05-08T08:54:12.712271+0000","last_became_active":"2024-05-08T08:53:20.362113+0000","last_became_peered":"2024-05-08T08:53:20.362113+0000","last_unstale":"2024-05-08T08:54:12.712271+0000","last_undegraded":"2024-05-08T08:54:12.712271+0000","last_fullsized":"2024-05-08T08:54:12.712271+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:28:25.204125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"104'413","reported_seq":18426,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-08T08:54:12.705511+0000","last_change":"2024-05-08T08:53:49.580812+0000","last_active":"2024-05-08T08:54:12.705511+0000","last_peered":"2024-05-08T08:54:12.705511+0000","last_clean":"2024-05-08T08:54:12.705511+0000","last_became_active":"2024-05-08T08:53:49.579888+0000","last_became_peered":"2024-05-08T08:53:49.579888+0000","last_unstale":"2024-05-08T08:54:12.705511+0000","last_undegraded":"2024-05-08T08:54:12.705511+0000","last_fullsized":"2024-05-08T08:54:12.705511+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":413,"log_dups_size":0,"ondisk_log_size":413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:59:55.976697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088867899999999997,"stat_sum":{"num_bytes":40662233,"num_objects":80,"num_object_clones":9,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":47,"num_read":19927,"num_read_kb":80884,"num_write":392,"num_write_kb":15830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":189,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-08T08:54:12.705532+0000","last_change":"2024-05-08T08:52:03.896159+0000","last_active":"2024-05-08T08:54:12.705532+0000","last_peered":"2024-05-08T08:54:12.705532+0000","last_clean":"2024-05-08T08:54:12.705532+0000","last_became_active":"2024-05-08T08:52:03.895797+0000","last_became_peered":"2024-05-08T08:52:03.895797+0000","last_unstale":"2024-05-08T08:54:12.705532+0000","last_undegraded":"2024-05-08T08:54:12.705532+0000","last_fullsized":"2024-05-08T08:54:12.705532+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:34:33.513525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"104'407","reported_seq":5906,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-08T08:54:12.709160+0000","last_change":"2024-05-08T08:52:13.944951+0000","last_active":"2024-05-08T08:54:12.709160+0000","last_peered":"2024-05-08T08:54:12.709160+0000","last_clean":"2024-05-08T08:54:12.709160+0000","last_became_active":"2024-05-08T08:52:11.926905+0000","last_became_peered":"2024-05-08T08:52:11.926905+0000","last_unstale":"2024-05-08T08:54:12.709160+0000","last_undegraded":"2024-05-08T08:54:12.709160+0000","last_fullsized":"2024-05-08T08:54:12.709160+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:34:47.364151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044527,"stat_sum":{"num_bytes":15654049,"num_objects":95,"num_object_clones":17,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":50,"num_read":6208,"num_read_kb":30301,"num_write":348,"num_write_kb":11190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":188,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-08T08:54:12.712218+0000","last_change":"2024-05-08T08:52:04.144764+0000","last_active":"2024-05-08T08:54:12.712218+0000","last_peered":"2024-05-08T08:54:12.712218+0000","last_clean":"2024-05-08T08:54:12.712218+0000","last_became_active":"2024-05-08T08:52:04.144444+0000","last_became_peered":"2024-05-08T08:52:04.144444+0000","last_unstale":"2024-05-08T08:54:12.712218+0000","last_undegraded":"2024-05-08T08:54:12.712218+0000","last_fullsized":"2024-05-08T08:54:12.712218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:58:31.498922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"104'627","reported_seq":24691,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-08T08:54:12.705559+0000","last_change":"2024-05-08T08:53:58.619407+0000","last_active":"2024-05-08T08:54:12.705559+0000","last_peered":"2024-05-08T08:54:12.705559+0000","last_clean":"2024-05-08T08:54:12.705559+0000","last_became_active":"2024-05-08T08:53:58.618388+0000","last_became_peered":"2024-05-08T08:53:58.618388+0000","last_unstale":"2024-05-08T08:54:12.705559+0000","last_undegraded":"2024-05-08T08:54:12.705559+0000","last_fullsized":"2024-05-08T08:54:12.705559+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":627,"log_dups_size":0,"ondisk_log_size":627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:43:53.043612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000669351,"stat_sum":{"num_bytes":49267216,"num_objects":114,"num_object_clones":19,"num_object_copies":228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":114,"num_whiteouts":62,"num_read":20948,"num_read_kb":45941,"num_write":452,"num_write_kb":11277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":215,"num_bytes_recovered":10925568,"num_keys_recovered":44,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":189,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-08T08:54:12.705603+0000","last_change":"2024-05-08T08:52:04.144543+0000","last_active":"2024-05-08T08:54:12.705603+0000","last_peered":"2024-05-08T08:54:12.705603+0000","last_clean":"2024-05-08T08:54:12.705603+0000","last_became_active":"2024-05-08T08:52:04.144242+0000","last_became_peered":"2024-05-08T08:52:04.144242+0000","last_unstale":"2024-05-08T08:54:12.705603+0000","last_undegraded":"2024-05-08T08:54:12.705603+0000","last_fullsized":"2024-05-08T08:54:12.705603+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:57:36.844633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"104'821","reported_seq":13256,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-08T08:54:12.709133+0000","last_change":"2024-05-08T08:53:58.616694+0000","last_active":"2024-05-08T08:54:12.709133+0000","last_peered":"2024-05-08T08:54:12.709133+0000","last_clean":"2024-05-08T08:54:12.709133+0000","last_became_active":"2024-05-08T08:53:58.615614+0000","last_became_peered":"2024-05-08T08:53:58.615614+0000","last_unstale":"2024-05-08T08:54:12.709133+0000","last_undegraded":"2024-05-08T08:54:12.709133+0000","last_fullsized":"2024-05-08T08:54:12.709133+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":821,"log_dups_size":0,"ondisk_log_size":821,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:08.618308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064078099999999999,"stat_sum":{"num_bytes":16686331,"num_objects":102,"num_object_clones":15,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":66,"num_read":14796,"num_read_kb":32084,"num_write":1049,"num_write_kb":9451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":142,"num_bytes_recovered":4609275,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"104'476","reported_seq":10054,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-08T08:54:12.721111+0000","last_change":"2024-05-08T08:53:24.481238+0000","last_active":"2024-05-08T08:54:12.721111+0000","last_peered":"2024-05-08T08:54:12.721111+0000","last_clean":"2024-05-08T08:54:12.721111+0000","last_became_active":"2024-05-08T08:53:24.480423+0000","last_became_peered":"2024-05-08T08:53:24.480423+0000","last_unstale":"2024-05-08T08:54:12.721111+0000","last_undegraded":"2024-05-08T08:54:12.721111+0000","last_fullsized":"2024-05-08T08:54:12.721111+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":476,"log_dups_size":0,"ondisk_log_size":476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:32:46.179481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047037799999999997,"stat_sum":{"num_bytes":30593870,"num_objects":78,"num_object_clones":13,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":44,"num_read":10598,"num_read_kb":30313,"num_write":548,"num_write_kb":5702,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":101,"num_bytes_recovered":5689443,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":168,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-08T08:54:12.705272+0000","last_change":"2024-05-08T08:53:55.365435+0000","last_active":"2024-05-08T08:54:12.705272+0000","last_peered":"2024-05-08T08:54:12.705272+0000","last_clean":"2024-05-08T08:54:12.705272+0000","last_became_active":"2024-05-08T08:53:55.365059+0000","last_became_peered":"2024-05-08T08:53:55.365059+0000","last_unstale":"2024-05-08T08:54:12.705272+0000","last_undegraded":"2024-05-08T08:54:12.705272+0000","last_fullsized":"2024-05-08T08:54:12.705272+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:15:39.278668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"104'474","reported_seq":10532,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-08T08:54:12.705323+0000","last_change":"2024-05-08T08:52:13.940504+0000","last_active":"2024-05-08T08:54:12.705323+0000","last_peered":"2024-05-08T08:54:12.705323+0000","last_clean":"2024-05-08T08:54:12.705323+0000","last_became_active":"2024-05-08T08:52:11.928301+0000","last_became_peered":"2024-05-08T08:52:11.928301+0000","last_unstale":"2024-05-08T08:54:12.705323+0000","last_undegraded":"2024-05-08T08:54:12.705323+0000","last_fullsized":"2024-05-08T08:54:12.705323+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":474,"log_dups_size":0,"ondisk_log_size":474,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:11:00.464865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046631800000000002,"stat_sum":{"num_bytes":45686375,"num_objects":93,"num_object_clones":16,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":51,"num_read":7142,"num_read_kb":32984,"num_write":490,"num_write_kb":14123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":189,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-08T08:54:12.708948+0000","last_change":"2024-05-08T08:52:03.891768+0000","last_active":"2024-05-08T08:54:12.708948+0000","last_peered":"2024-05-08T08:54:12.708948+0000","last_clean":"2024-05-08T08:54:12.708948+0000","last_became_active":"2024-05-08T08:52:03.891374+0000","last_became_peered":"2024-05-08T08:52:03.891374+0000","last_unstale":"2024-05-08T08:54:12.708948+0000","last_undegraded":"2024-05-08T08:54:12.708948+0000","last_fullsized":"2024-05-08T08:54:12.708948+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:18:10.240492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"104'833","reported_seq":24406,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-08T08:54:12.721126+0000","last_change":"2024-05-08T08:53:49.582654+0000","last_active":"2024-05-08T08:54:12.721126+0000","last_peered":"2024-05-08T08:54:12.721126+0000","last_clean":"2024-05-08T08:54:12.721126+0000","last_became_active":"2024-05-08T08:53:49.581797+0000","last_became_peered":"2024-05-08T08:53:49.581797+0000","last_unstale":"2024-05-08T08:54:12.721126+0000","last_undegraded":"2024-05-08T08:54:12.721126+0000","last_fullsized":"2024-05-08T08:54:12.721126+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":833,"log_dups_size":0,"ondisk_log_size":833,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:36:06.182385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046440499999999998,"stat_sum":{"num_bytes":45467468,"num_objects":104,"num_object_clones":16,"num_object_copies":208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":54,"num_read":27113,"num_read_kb":43142,"num_write":558,"num_write_kb":8683,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":135,"num_bytes_recovered":3549534,"num_keys_recovered":46,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":188,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-08T08:54:12.711714+0000","last_change":"2024-05-08T08:52:03.891551+0000","last_active":"2024-05-08T08:54:12.711714+0000","last_peered":"2024-05-08T08:54:12.711714+0000","last_clean":"2024-05-08T08:54:12.711714+0000","last_became_active":"2024-05-08T08:52:03.891296+0000","last_became_peered":"2024-05-08T08:52:03.891296+0000","last_unstale":"2024-05-08T08:54:12.711714+0000","last_undegraded":"2024-05-08T08:54:12.711714+0000","last_fullsized":"2024-05-08T08:54:12.711714+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:54.806343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"105'762","reported_seq":11679,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-08T08:54:12.705127+0000","last_change":"2024-05-08T08:53:58.616835+0000","last_active":"2024-05-08T08:54:12.705127+0000","last_peered":"2024-05-08T08:54:12.705127+0000","last_clean":"2024-05-08T08:54:12.705127+0000","last_became_active":"2024-05-08T08:53:58.616032+0000","last_became_peered":"2024-05-08T08:53:58.616032+0000","last_unstale":"2024-05-08T08:54:12.705127+0000","last_undegraded":"2024-05-08T08:54:12.705127+0000","last_fullsized":"2024-05-08T08:54:12.705127+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":762,"log_dups_size":0,"ondisk_log_size":762,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:51:20.034794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062135599999999999,"stat_sum":{"num_bytes":40747500,"num_objects":104,"num_object_clones":18,"num_object_copies":208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":57,"num_read":12766,"num_read_kb":44578,"num_write":1043,"num_write_kb":11773,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":125,"num_bytes_recovered":4950942,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":168,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-08T08:54:12.709006+0000","last_change":"2024-05-08T08:53:55.362409+0000","last_active":"2024-05-08T08:54:12.709006+0000","last_peered":"2024-05-08T08:54:12.709006+0000","last_clean":"2024-05-08T08:54:12.709006+0000","last_became_active":"2024-05-08T08:53:55.362055+0000","last_became_peered":"2024-05-08T08:53:55.362055+0000","last_unstale":"2024-05-08T08:54:12.709006+0000","last_undegraded":"2024-05-08T08:54:12.709006+0000","last_fullsized":"2024-05-08T08:54:12.709006+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:34:23.719119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":344,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-08T08:54:12.711723+0000","last_change":"2024-05-08T08:51:58.861899+0000","last_active":"2024-05-08T08:54:12.711723+0000","last_peered":"2024-05-08T08:54:12.711723+0000","last_clean":"2024-05-08T08:54:12.711723+0000","last_became_active":"2024-05-08T08:51:58.861596+0000","last_became_peered":"2024-05-08T08:51:58.861596+0000","last_unstale":"2024-05-08T08:54:12.711723+0000","last_undegraded":"2024-05-08T08:54:12.711723+0000","last_fullsized":"2024-05-08T08:54:12.711723+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:51:57.849666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:51:57.849666+0000","last_clean_scrub_stamp":"2024-05-08T08:51:57.849666+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:53:21.739950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"104'301","reported_seq":8411,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-08T08:54:12.709297+0000","last_change":"2024-05-08T08:52:13.945362+0000","last_active":"2024-05-08T08:54:12.709297+0000","last_peered":"2024-05-08T08:54:12.709297+0000","last_clean":"2024-05-08T08:54:12.709297+0000","last_became_active":"2024-05-08T08:52:11.926975+0000","last_became_peered":"2024-05-08T08:52:11.926975+0000","last_unstale":"2024-05-08T08:54:12.709297+0000","last_undegraded":"2024-05-08T08:54:12.709297+0000","last_fullsized":"2024-05-08T08:54:12.709297+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":301,"log_dups_size":0,"ondisk_log_size":301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:59:31.931674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068218599999999999,"stat_sum":{"num_bytes":31656032,"num_objects":88,"num_object_clones":15,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":48,"num_read":7991,"num_read_kb":28894,"num_write":267,"num_write_kb":7624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"104'392","reported_seq":13622,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-08T08:54:12.721323+0000","last_change":"2024-05-08T08:53:23.479481+0000","last_active":"2024-05-08T08:54:12.721323+0000","last_peered":"2024-05-08T08:54:12.721323+0000","last_clean":"2024-05-08T08:54:12.721323+0000","last_became_active":"2024-05-08T08:53:23.478564+0000","last_became_peered":"2024-05-08T08:53:23.478564+0000","last_unstale":"2024-05-08T08:54:12.721323+0000","last_undegraded":"2024-05-08T08:54:12.721323+0000","last_fullsized":"2024-05-08T08:54:12.721323+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":392,"log_dups_size":0,"ondisk_log_size":392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:22:31.826500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00030754000000000003,"stat_sum":{"num_bytes":36214831,"num_objects":91,"num_object_clones":11,"num_object_copies":182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91,"num_whiteouts":51,"num_read":13699,"num_read_kb":60565,"num_write":333,"num_write_kb":16367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":131,"num_bytes_recovered":12377088,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"105'1075","reported_seq":10620,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-08T08:54:12.712035+0000","last_change":"2024-05-08T08:53:24.481272+0000","last_active":"2024-05-08T08:54:12.712035+0000","last_peered":"2024-05-08T08:54:12.712035+0000","last_clean":"2024-05-08T08:54:12.712035+0000","last_became_active":"2024-05-08T08:53:24.480359+0000","last_became_peered":"2024-05-08T08:53:24.480359+0000","last_unstale":"2024-05-08T08:54:12.712035+0000","last_undegraded":"2024-05-08T08:54:12.712035+0000","last_fullsized":"2024-05-08T08:54:12.712035+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":1075,"log_dups_size":0,"ondisk_log_size":1075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:58:16.911602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070350100000000004,"stat_sum":{"num_bytes":21900143,"num_objects":84,"num_object_clones":12,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":50,"num_read":14862,"num_read_kb":27055,"num_write":1307,"num_write_kb":5588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":2175343,"num_keys_recovered":61,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"104'495","reported_seq":10010,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-08T08:54:12.705645+0000","last_change":"2024-05-08T08:52:13.940600+0000","last_active":"2024-05-08T08:54:12.705645+0000","last_peered":"2024-05-08T08:54:12.705645+0000","last_clean":"2024-05-08T08:54:12.705645+0000","last_became_active":"2024-05-08T08:52:11.927196+0000","last_became_peered":"2024-05-08T08:52:11.927196+0000","last_unstale":"2024-05-08T08:54:12.705645+0000","last_undegraded":"2024-05-08T08:54:12.705645+0000","last_fullsized":"2024-05-08T08:54:12.705645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":495,"log_dups_size":0,"ondisk_log_size":495,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:36:49.551899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048296599999999998,"stat_sum":{"num_bytes":39118437,"num_objects":95,"num_object_clones":12,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":59,"num_read":10453,"num_read_kb":33758,"num_write":419,"num_write_kb":7319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"104'295","reported_seq":12889,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-08T08:54:12.712062+0000","last_change":"2024-05-08T08:53:58.616502+0000","last_active":"2024-05-08T08:54:12.712062+0000","last_peered":"2024-05-08T08:54:12.712062+0000","last_clean":"2024-05-08T08:54:12.712062+0000","last_became_active":"2024-05-08T08:53:58.615095+0000","last_became_peered":"2024-05-08T08:53:58.615095+0000","last_unstale":"2024-05-08T08:54:12.712062+0000","last_undegraded":"2024-05-08T08:54:12.712062+0000","last_fullsized":"2024-05-08T08:54:12.712062+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":295,"log_dups_size":0,"ondisk_log_size":295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:02:08.918189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048793000000000003,"stat_sum":{"num_bytes":15228603,"num_objects":82,"num_object_clones":10,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":49,"num_read":12504,"num_read_kb":36091,"num_write":257,"num_write_kb":6693,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":107,"num_bytes_recovered":2139323,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"104'379","reported_seq":7680,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-08T08:54:12.712162+0000","last_change":"2024-05-08T08:52:13.945141+0000","last_active":"2024-05-08T08:54:12.712162+0000","last_peered":"2024-05-08T08:54:12.712162+0000","last_clean":"2024-05-08T08:54:12.712162+0000","last_became_active":"2024-05-08T08:52:11.932594+0000","last_became_peered":"2024-05-08T08:52:11.932594+0000","last_unstale":"2024-05-08T08:54:12.712162+0000","last_undegraded":"2024-05-08T08:54:12.712162+0000","last_fullsized":"2024-05-08T08:54:12.712162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":379,"log_dups_size":0,"ondisk_log_size":379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:15:14.334869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074388199999999996,"stat_sum":{"num_bytes":18975285,"num_objects":83,"num_object_clones":10,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":51,"num_read":8530,"num_read_kb":31864,"num_write":328,"num_write_kb":9690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"104'460","reported_seq":7619,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-08T08:54:12.721128+0000","last_change":"2024-05-08T08:53:58.622564+0000","last_active":"2024-05-08T08:54:12.721128+0000","last_peered":"2024-05-08T08:54:12.721128+0000","last_clean":"2024-05-08T08:54:12.721128+0000","last_became_active":"2024-05-08T08:53:58.621584+0000","last_became_peered":"2024-05-08T08:53:58.621584+0000","last_unstale":"2024-05-08T08:54:12.721128+0000","last_undegraded":"2024-05-08T08:54:12.721128+0000","last_fullsized":"2024-05-08T08:54:12.721128+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":460,"log_dups_size":0,"ondisk_log_size":460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:16:16.868378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085079600000000002,"stat_sum":{"num_bytes":27483200,"num_objects":70,"num_object_clones":14,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":34,"num_read":13923,"num_read_kb":41940,"num_write":519,"num_write_kb":11968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":188,"num_bytes_recovered":11952192,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"104'410","reported_seq":7000,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-08T08:54:12.712199+0000","last_change":"2024-05-08T08:53:58.616115+0000","last_active":"2024-05-08T08:54:12.712199+0000","last_peered":"2024-05-08T08:54:12.712199+0000","last_clean":"2024-05-08T08:54:12.712199+0000","last_became_active":"2024-05-08T08:53:58.615064+0000","last_became_peered":"2024-05-08T08:53:58.615064+0000","last_unstale":"2024-05-08T08:54:12.712199+0000","last_undegraded":"2024-05-08T08:54:12.712199+0000","last_fullsized":"2024-05-08T08:54:12.712199+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":410,"log_dups_size":0,"ondisk_log_size":410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:01:07.258043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010326370000000001,"stat_sum":{"num_bytes":24172770,"num_objects":75,"num_object_clones":8,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":44,"num_read":7790,"num_read_kb":34149,"num_write":390,"num_write_kb":9252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":107,"num_bytes_recovered":4068050,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"104'342","reported_seq":7451,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-08T08:54:12.704812+0000","last_change":"2024-05-08T08:53:58.619593+0000","last_active":"2024-05-08T08:54:12.704812+0000","last_peered":"2024-05-08T08:54:12.704812+0000","last_clean":"2024-05-08T08:54:12.704812+0000","last_became_active":"2024-05-08T08:53:58.618787+0000","last_became_peered":"2024-05-08T08:53:58.618787+0000","last_unstale":"2024-05-08T08:54:12.704812+0000","last_undegraded":"2024-05-08T08:54:12.704812+0000","last_fullsized":"2024-05-08T08:54:12.704812+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:45:03.074014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053593,"stat_sum":{"num_bytes":32900689,"num_objects":83,"num_object_clones":13,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":51,"num_read":7220,"num_read_kb":38823,"num_write":332,"num_write_kb":13709,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":112,"num_bytes_recovered":7552081,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"104'534","reported_seq":9556,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-08T08:54:12.711787+0000","last_change":"2024-05-08T08:53:24.480561+0000","last_active":"2024-05-08T08:54:12.711787+0000","last_peered":"2024-05-08T08:54:12.711787+0000","last_clean":"2024-05-08T08:54:12.711787+0000","last_became_active":"2024-05-08T08:53:24.479891+0000","last_became_peered":"2024-05-08T08:53:24.479891+0000","last_unstale":"2024-05-08T08:54:12.711787+0000","last_undegraded":"2024-05-08T08:54:12.711787+0000","last_fullsized":"2024-05-08T08:54:12.711787+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":534,"log_dups_size":0,"ondisk_log_size":534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:54:56.888481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041230699999999999,"stat_sum":{"num_bytes":31926821,"num_objects":94,"num_object_clones":16,"num_object_copies":188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":57,"num_read":10690,"num_read_kb":29463,"num_write":446,"num_write_kb":9107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":131,"num_bytes_recovered":2604032,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"105'1333","reported_seq":23084,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-08T08:54:12.737215+0000","last_change":"2024-05-08T08:53:49.581481+0000","last_active":"2024-05-08T08:54:12.737215+0000","last_peered":"2024-05-08T08:54:12.737215+0000","last_clean":"2024-05-08T08:54:12.737215+0000","last_became_active":"2024-05-08T08:53:49.580557+0000","last_became_peered":"2024-05-08T08:53:49.580557+0000","last_unstale":"2024-05-08T08:54:12.737215+0000","last_undegraded":"2024-05-08T08:54:12.737215+0000","last_fullsized":"2024-05-08T08:54:12.737215+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":1333,"log_dups_size":0,"ondisk_log_size":1333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:27:42.933986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059220300000000002,"stat_sum":{"num_bytes":44968261,"num_objects":106,"num_object_clones":11,"num_object_copies":212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":106,"num_whiteouts":56,"num_read":28580,"num_read_kb":51589,"num_write":1895,"num_write_kb":10342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"104'445","reported_seq":9356,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-08T08:54:12.711886+0000","last_change":"2024-05-08T08:53:58.616558+0000","last_active":"2024-05-08T08:54:12.711886+0000","last_peered":"2024-05-08T08:54:12.711886+0000","last_clean":"2024-05-08T08:54:12.711886+0000","last_became_active":"2024-05-08T08:53:58.615947+0000","last_became_peered":"2024-05-08T08:53:58.615947+0000","last_unstale":"2024-05-08T08:54:12.711886+0000","last_undegraded":"2024-05-08T08:54:12.711886+0000","last_fullsized":"2024-05-08T08:54:12.711886+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:51:57.345071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059972400000000005,"stat_sum":{"num_bytes":51989631,"num_objects":100,"num_object_clones":20,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":53,"num_read":9343,"num_read_kb":39673,"num_write":424,"num_write_kb":8958,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":130,"num_bytes_recovered":6657151,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1152505842,"num_objects":2902,"num_object_clones":438,"num_object_copies":5804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2902,"num_whiteouts":1633,"num_read":458416,"num_read_kb":1411157,"num_write":20807,"num_write_kb":421216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2942,"num_bytes_recovered":182360369,"num_keys_recovered":458,"num_objects_omap":43,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":88211456,"data_stored":737510647,"data_compressed":37840063,"data_compressed_allocated":72417280,"data_compressed_original":727885312,"omap_allocated":0,"internal_metadata":0},"log_size":18769,"ondisk_log_size":18769,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":11046,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":0,"seq":47244640285,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640288,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49656,"kb_used_data":14320,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94322184,"statfs":{"total":96636764160,"available":96585916416,"internally_reserved":0,"allocated":14663680,"data_stored":115856131,"data_compressed":6327792,"data_compressed_allocated":11616256,"data_compressed_original":114284032,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640287,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81128,"kb_used_data":18016,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94290712,"statfs":{"total":96636764160,"available":96553689088,"internally_reserved":0,"allocated":18448384,"data_stored":137609553,"data_compressed":7099205,"data_compressed_allocated":13283328,"data_compressed_original":134585344,"omap_allocated":1593,"internal_metadata":64616903},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640286,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42932,"kb_used_data":11756,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94328908,"statfs":{"total":96636764160,"available":96592801792,"internally_reserved":0,"allocated":12038144,"data_stored":87450218,"data_compressed":5128368,"data_compressed_allocated":9060352,"data_compressed_original":85902336,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640285,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73128,"kb_used_data":19744,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94298712,"statfs":{"total":96636764160,"available":96561881088,"internally_reserved":0,"allocated":20217856,"data_stored":179049453,"data_compressed":8120705,"data_compressed_allocated":16564224,"data_compressed_original":177138688,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640288,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":91960,"kb_used_data":16368,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94279880,"statfs":{"total":96636764160,"available":96542597120,"internally_reserved":0,"allocated":16760832,"data_stored":121351307,"data_compressed":5931485,"data_compressed_allocated":11735040,"data_compressed_original":118504448,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5523,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5523,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":15654912,"data_stored":120421690,"data_compressed":5990756,"data_compressed_allocated":11767808,"data_compressed_original":118029312,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":19247104,"data_stored":178773108,"data_compressed":8193436,"data_compressed_allocated":16646144,"data_compressed_original":177261568,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":10809344,"data_stored":86943918,"data_compressed":5128368,"data_compressed_allocated":9060352,"data_compressed_original":85902336,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":17354752,"data_stored":136723505,"data_compressed":7179792,"data_compressed_allocated":13344768,"data_compressed_original":134167552,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":13438976,"data_stored":115349847,"data_compressed":6327792,"data_compressed_allocated":11616256,"data_compressed_original":114284032,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":11706368,"data_stored":99298579,"data_compressed":5019919,"data_compressed_allocated":9981952,"data_compressed_original":98240512,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T08:54:16.472 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:16.475+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:16.503 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:16.503 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:16.513 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:54:16.535 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:16.537+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:16.622 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T08:54:16.629 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:16.629 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:16.638 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:54:16.666 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:16.669+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:16.736 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:16.739+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:16.745 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:16.745 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:16.747 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:54:16.748 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:54:16.754 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:54:16.765 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:54:16.765 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:54:16.782 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:54:16.783 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:54:16.798 INFO:teuthology.orchestra.run.smithi196.stdout:5567 write 0xb3d2800 thru 0xb3d2bc6 (0x3c7 bytes) 2024-05-08T08:54:16.799 INFO:teuthology.orchestra.run.smithi196.stdout:5570 read 0x60a0a00 thru 0x60aeb37 (0xe138 bytes) 2024-05-08T08:54:16.799 INFO:teuthology.orchestra.run.smithi196.stdout:5571 read 0x1ffc000 thru 0x200987a (0xd87b bytes) 2024-05-08T08:54:16.800 INFO:teuthology.orchestra.run.smithi196.stdout:5572 write 0xd6fd800 thru 0xd70c131 (0xe932 bytes) 2024-05-08T08:54:16.800 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:54:16.801 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:54:16.801 INFO:teuthology.orchestra.run.smithi196.stdout:5574 write 0xa07600 thru 0xa0e3b0 (0x6db1 bytes) 2024-05-08T08:54:16.802 INFO:teuthology.orchestra.run.smithi196.stdout:5575 write 0x82f8800 thru 0x82fe31b (0x5b1c bytes) 2024-05-08T08:54:16.803 INFO:teuthology.orchestra.run.smithi196.stdout:5576 punch from 0xbd7c800 to 0xbd80600, (0x3e00 bytes) 2024-05-08T08:54:16.821 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:54:16.821 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:54:16.825 INFO:teuthology.orchestra.run.smithi196.stdout:5577 write 0x3a53600 thru 0x3a5708a (0x3a8b bytes) 2024-05-08T08:54:16.826 INFO:teuthology.orchestra.run.smithi196.stdout:5578 write 0x7588e00 thru 0x75986b6 (0xf8b7 bytes) 2024-05-08T08:54:16.826 INFO:teuthology.orchestra.run.smithi196.stdout:5585 punch from 0xd63d800 to 0xd63f000, (0x1800 bytes) 2024-05-08T08:54:16.837 INFO:teuthology.orchestra.run.smithi196.stdout:5586 write 0xc320e00 thru 0xc321eb5 (0x10b6 bytes) 2024-05-08T08:54:16.838 INFO:teuthology.orchestra.run.smithi196.stdout:5587 read 0xacc00 thru 0xb6c46 (0xa047 bytes) 2024-05-08T08:54:16.839 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:16.842+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:16.842 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:54:16.842 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:54:16.844 INFO:teuthology.orchestra.run.smithi196.stdout:5592 trunc from 0xe503000 to 0xb128e00 2024-05-08T08:54:16.885 INFO:teuthology.orchestra.run.smithi196.stdout:5593 read 0x1fa6400 thru 0x1fad444 (0x7045 bytes) 2024-05-08T08:54:16.887 INFO:teuthology.orchestra.run.smithi196.stdout:5595 read 0x5886400 thru 0x58869d4 (0x5d5 bytes) 2024-05-08T08:54:16.887 INFO:teuthology.orchestra.run.smithi196.stdout:5596 punch from 0xb0d7c00 to 0xb0e0800, (0x8c00 bytes) 2024-05-08T08:54:16.887 INFO:teuthology.orchestra.run.smithi196.stdout:5598 clone 43 order 19 su 65536 sc 2 2024-05-08T08:54:16.915 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:16.915 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:54:16.915 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:54:16.915 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:16.924 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:54:16.938 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:16.941+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:17.039 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:17.042+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:17.070 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:17.070 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:54:17.070 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:54:17.070 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:54:17.070 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:17.078 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:54:17.141 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:17.143+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:17.186 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:17.186 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:54:17.186 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:54:17.186 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:54:17.186 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.186 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22073 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 782336~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:17.186 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:54:17.186 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 109.330482164, 2024-05-08T08:54:17.186 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047244234000000003, 2024-05-08T08:54:17.187 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:17.187 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:17.187 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:17.187 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:17.187 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:17.187 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:17.187 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22073 2024-05-08T08:54:17.187 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.187 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:17.187 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.187 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:17.187 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:54:17.187 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:17.187 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.187 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.188 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:17.188 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:54:17.188 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:17.188 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.188 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.188 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:17.188 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857668+0000", 2024-05-08T08:54:17.188 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7800000000000002e-07 2024-05-08T08:54:17.188 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.188 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.188 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:17.188 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857670+0000", 2024-05-08T08:54:17.188 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0820000000000001e-06 2024-05-08T08:54:17.188 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.188 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.189 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:17.189 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857678+0000", 2024-05-08T08:54:17.189 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7589999999999997e-06 2024-05-08T08:54:17.189 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.189 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.189 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:17.189 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857691+0000", 2024-05-08T08:54:17.189 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3672999999999999e-05 2024-05-08T08:54:17.189 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.189 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.189 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:17.189 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904780+0000", 2024-05-08T08:54:17.189 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047088616999999999 2024-05-08T08:54:17.189 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.189 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.189 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:17.190 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904840+0000", 2024-05-08T08:54:17.190 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0356999999999998e-05 2024-05-08T08:54:17.190 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.190 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.190 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:17.190 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904912+0000", 2024-05-08T08:54:17.190 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1367999999999998e-05 2024-05-08T08:54:17.190 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.190 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:17.190 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.190 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.190 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.190 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22075 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 798720~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:17.191 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:54:17.191 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 109.330451767, 2024-05-08T08:54:17.191 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047384643999999997, 2024-05-08T08:54:17.191 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:17.191 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:17.191 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:17.191 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:17.191 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:17.191 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:17.191 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22075 2024-05-08T08:54:17.191 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.191 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:17.191 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.191 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:17.192 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:54:17.192 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:17.192 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.192 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.192 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:17.192 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:54:17.192 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:17.192 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.192 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.192 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:17.192 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:54:17.192 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7399999999999999e-07 2024-05-08T08:54:17.192 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.192 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.192 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:17.193 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857705+0000", 2024-05-08T08:54:17.193 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1980000000000004e-06 2024-05-08T08:54:17.193 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.193 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.193 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:17.193 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857713+0000", 2024-05-08T08:54:17.193 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7039999999999994e-06 2024-05-08T08:54:17.193 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.193 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.193 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:17.193 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857718+0000", 2024-05-08T08:54:17.193 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.865e-06 2024-05-08T08:54:17.193 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.193 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.193 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:17.193 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904941+0000", 2024-05-08T08:54:17.193 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047223198000000001 2024-05-08T08:54:17.194 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.194 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.194 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:17.194 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904999+0000", 2024-05-08T08:54:17.194 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8254999999999997e-05 2024-05-08T08:54:17.194 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.194 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.194 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:17.194 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905082+0000", 2024-05-08T08:54:17.194 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3049999999999999e-05 2024-05-08T08:54:17.194 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.194 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:17.194 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.194 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.195 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.195 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22077 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 815104~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:17.195 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:54:17.195 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 109.330413756, 2024-05-08T08:54:17.195 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047519448999999998, 2024-05-08T08:54:17.195 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:17.195 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:17.195 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:17.195 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:17.195 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:17.195 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:17.195 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22077 2024-05-08T08:54:17.195 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.195 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:17.195 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.195 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:17.196 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:54:17.196 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:17.196 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.196 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.196 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:17.196 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:54:17.196 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:17.196 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.196 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.196 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:17.196 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:54:17.196 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.99e-07 2024-05-08T08:54:17.196 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.196 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.196 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:17.196 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857739+0000", 2024-05-08T08:54:17.197 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3599999999999999e-06 2024-05-08T08:54:17.197 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.197 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.197 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:17.197 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857745+0000", 2024-05-08T08:54:17.197 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5139999999999998e-06 2024-05-08T08:54:17.197 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.197 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.197 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:17.197 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857750+0000", 2024-05-08T08:54:17.197 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8550000000000001e-06 2024-05-08T08:54:17.197 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.197 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.197 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:17.197 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905110+0000", 2024-05-08T08:54:17.197 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047360126000000002 2024-05-08T08:54:17.198 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.198 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.198 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:17.198 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905175+0000", 2024-05-08T08:54:17.198 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4689000000000004e-05 2024-05-08T08:54:17.198 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.198 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.198 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:17.198 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905255+0000", 2024-05-08T08:54:17.198 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0306e-05 2024-05-08T08:54:17.198 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.198 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:17.198 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.198 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.199 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.199 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22079 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 831488~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:17.199 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:54:17.199 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 109.330393208, 2024-05-08T08:54:17.199 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.04767499, 2024-05-08T08:54:17.199 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:17.199 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:17.199 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:17.199 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:17.199 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:17.199 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:17.199 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22079 2024-05-08T08:54:17.199 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.200 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:17.200 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.200 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:17.200 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:54:17.200 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:17.200 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.200 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.200 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:17.200 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:54:17.200 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:17.200 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.200 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.200 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:17.200 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857757+0000", 2024-05-08T08:54:17.201 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8299999999999997e-07 2024-05-08T08:54:17.201 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.201 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.201 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:17.201 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857759+0000", 2024-05-08T08:54:17.201 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1390000000000002e-06 2024-05-08T08:54:17.201 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.201 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.201 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:17.201 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857765+0000", 2024-05-08T08:54:17.201 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4609999999999996e-06 2024-05-08T08:54:17.201 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.201 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.201 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:17.201 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857770+0000", 2024-05-08T08:54:17.202 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6689999999999997e-06 2024-05-08T08:54:17.202 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.202 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.202 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:17.202 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905281+0000", 2024-05-08T08:54:17.202 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047510668999999998 2024-05-08T08:54:17.202 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.202 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.202 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:17.202 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905344+0000", 2024-05-08T08:54:17.202 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2797000000000002e-05 2024-05-08T08:54:17.202 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.202 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.202 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:17.203 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905431+0000", 2024-05-08T08:54:17.203 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7771999999999999e-05 2024-05-08T08:54:17.203 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.203 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:17.203 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.203 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.203 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.203 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22082 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 847872~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:17.203 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:54:17.203 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 109.330373517, 2024-05-08T08:54:17.203 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047831934999999999, 2024-05-08T08:54:17.203 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:17.203 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:17.203 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:17.204 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:17.204 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:17.204 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:17.204 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22082 2024-05-08T08:54:17.204 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.204 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:17.204 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.204 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:17.204 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:54:17.204 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:17.204 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.204 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.204 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:17.204 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:54:17.204 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:17.204 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.205 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.205 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:17.205 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:54:17.205 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:54:17.205 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.205 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.205 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:17.205 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857778+0000", 2024-05-08T08:54:17.205 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0789999999999999e-06 2024-05-08T08:54:17.205 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.205 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.205 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:17.205 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857784+0000", 2024-05-08T08:54:17.205 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0480000000000004e-06 2024-05-08T08:54:17.206 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.206 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.206 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:17.206 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857790+0000", 2024-05-08T08:54:17.206 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2719999999999997e-06 2024-05-08T08:54:17.206 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.206 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.206 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:17.206 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905458+0000", 2024-05-08T08:54:17.206 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047668470999999997 2024-05-08T08:54:17.206 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.206 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.206 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:17.206 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905521+0000", 2024-05-08T08:54:17.207 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2952000000000003e-05 2024-05-08T08:54:17.207 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.207 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.207 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:17.207 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905608+0000", 2024-05-08T08:54:17.207 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6759000000000001e-05 2024-05-08T08:54:17.207 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.207 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:17.207 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.207 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.207 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.207 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22084 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 864256~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:17.207 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:54:17.207 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 109.330349567, 2024-05-08T08:54:17.207 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047977748000000001, 2024-05-08T08:54:17.208 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:17.208 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:17.208 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:17.208 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:17.208 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:17.208 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:17.208 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22084 2024-05-08T08:54:17.208 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.208 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:17.208 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.208 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:17.208 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:54:17.208 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:17.208 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.208 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.208 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:17.209 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:54:17.209 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:17.209 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.209 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.209 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:17.209 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:54:17.209 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:54:17.209 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.209 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.209 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:17.209 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857802+0000", 2024-05-08T08:54:17.209 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0779999999999998e-06 2024-05-08T08:54:17.209 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.209 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.209 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:17.209 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857812+0000", 2024-05-08T08:54:17.209 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7939999999999997e-06 2024-05-08T08:54:17.210 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.210 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.210 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:17.210 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857817+0000", 2024-05-08T08:54:17.210 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8899999999999998e-06 2024-05-08T08:54:17.210 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.210 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.210 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:17.210 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905636+0000", 2024-05-08T08:54:17.210 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047819252 2024-05-08T08:54:17.210 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.210 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.210 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:17.210 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905700+0000", 2024-05-08T08:54:17.210 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4016999999999994e-05 2024-05-08T08:54:17.211 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.211 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.211 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:17.211 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905778+0000", 2024-05-08T08:54:17.211 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7306999999999997e-05 2024-05-08T08:54:17.211 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.211 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:17.211 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.211 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.211 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.211 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22086 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 880640~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:17.211 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:54:17.211 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 109.33032621700001, 2024-05-08T08:54:17.211 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048119149, 2024-05-08T08:54:17.211 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:17.211 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:17.212 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:17.212 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:17.212 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:17.212 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:17.212 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22086 2024-05-08T08:54:17.212 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.212 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:17.212 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.212 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:17.212 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:54:17.212 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:17.212 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.212 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.212 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:17.213 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:54:17.213 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:17.213 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.213 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.213 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:17.213 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857824+0000", 2024-05-08T08:54:17.213 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4200000000000002e-07 2024-05-08T08:54:17.213 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.213 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.213 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:17.213 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857826+0000", 2024-05-08T08:54:17.213 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0590000000000001e-06 2024-05-08T08:54:17.213 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.213 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.213 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:17.214 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857832+0000", 2024-05-08T08:54:17.214 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8029999999999999e-06 2024-05-08T08:54:17.214 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.214 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.214 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:17.214 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857837+0000", 2024-05-08T08:54:17.214 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7929999999999997e-06 2024-05-08T08:54:17.214 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.214 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.214 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:17.214 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905803+0000", 2024-05-08T08:54:17.214 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047965761000000003 2024-05-08T08:54:17.214 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.214 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.215 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:17.215 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905862+0000", 2024-05-08T08:54:17.215 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9203000000000002e-05 2024-05-08T08:54:17.215 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.215 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.215 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:17.215 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905942+0000", 2024-05-08T08:54:17.215 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0187999999999999e-05 2024-05-08T08:54:17.215 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.215 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:17.215 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.215 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.215 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.215 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22088 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 897024~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:17.215 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:54:17.215 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 109.330306165, 2024-05-08T08:54:17.216 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048256049000000002, 2024-05-08T08:54:17.216 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:17.216 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:17.216 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:17.216 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:17.216 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:17.216 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:17.216 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22088 2024-05-08T08:54:17.216 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.216 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:17.216 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.216 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:17.216 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:54:17.216 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:17.216 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.216 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.217 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:17.217 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:54:17.217 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:17.217 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.217 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.217 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:17.217 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857844+0000", 2024-05-08T08:54:17.217 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.46e-07 2024-05-08T08:54:17.217 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.217 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.217 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:17.217 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857846+0000", 2024-05-08T08:54:17.217 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0899999999999999e-06 2024-05-08T08:54:17.217 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.217 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.217 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:17.217 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857855+0000", 2024-05-08T08:54:17.218 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8219999999999996e-06 2024-05-08T08:54:17.218 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.218 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.218 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:17.218 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857860+0000", 2024-05-08T08:54:17.218 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2989999999999996e-06 2024-05-08T08:54:17.218 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.218 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.218 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:17.218 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905966+0000", 2024-05-08T08:54:17.218 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048106542000000002 2024-05-08T08:54:17.218 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.218 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.218 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:17.218 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906024+0000", 2024-05-08T08:54:17.219 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7958000000000003e-05 2024-05-08T08:54:17.219 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.219 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.219 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:17.219 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906099+0000", 2024-05-08T08:54:17.219 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4992000000000003e-05 2024-05-08T08:54:17.219 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.219 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:17.219 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.219 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.219 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.219 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22091 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 913408~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:17.219 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:54:17.219 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 109.33028240500001, 2024-05-08T08:54:17.219 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048395290000000001, 2024-05-08T08:54:17.219 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:17.220 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:17.220 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:17.220 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:17.220 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:17.220 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:17.220 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22091 2024-05-08T08:54:17.220 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.220 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:17.220 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.220 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:17.220 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:54:17.220 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:17.220 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.220 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.220 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:17.220 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:54:17.220 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:17.221 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.221 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.221 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:17.221 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:54:17.221 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4900000000000001e-07 2024-05-08T08:54:17.221 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.221 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.221 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:17.221 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857870+0000", 2024-05-08T08:54:17.221 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2510000000000001e-06 2024-05-08T08:54:17.221 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.221 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.221 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:17.221 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857877+0000", 2024-05-08T08:54:17.221 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0809999999999996e-06 2024-05-08T08:54:17.221 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.222 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.222 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:17.222 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857881+0000", 2024-05-08T08:54:17.222 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.724e-06 2024-05-08T08:54:17.222 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.222 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.222 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:17.222 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906127+0000", 2024-05-08T08:54:17.222 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048245345000000002 2024-05-08T08:54:17.222 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.222 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.222 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:17.222 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906190+0000", 2024-05-08T08:54:17.222 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2762000000000007e-05 2024-05-08T08:54:17.222 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.222 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.223 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:17.223 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906262+0000", 2024-05-08T08:54:17.223 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2778e-05 2024-05-08T08:54:17.223 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.223 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:17.223 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.223 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.223 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.223 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22093 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 929792~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:17.223 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:54:17.223 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 109.330260041, 2024-05-08T08:54:17.223 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048532875000000003, 2024-05-08T08:54:17.223 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:17.223 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:17.223 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:17.223 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:17.224 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:17.224 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:17.224 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22093 2024-05-08T08:54:17.224 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.224 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:17.224 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.224 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:17.224 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:54:17.224 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:17.224 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.224 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.224 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:17.224 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:54:17.224 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:17.224 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.224 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.225 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:17.225 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857890+0000", 2024-05-08T08:54:17.225 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-07 2024-05-08T08:54:17.225 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.225 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.225 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:17.225 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857899+0000", 2024-05-08T08:54:17.225 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6889999999999999e-06 2024-05-08T08:54:17.225 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.225 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.225 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:17.225 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857906+0000", 2024-05-08T08:54:17.225 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6490000000000002e-06 2024-05-08T08:54:17.225 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.225 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.225 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:17.225 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857917+0000", 2024-05-08T08:54:17.226 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0764999999999999e-05 2024-05-08T08:54:17.226 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.226 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.226 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:17.226 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906287+0000", 2024-05-08T08:54:17.226 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048370442 2024-05-08T08:54:17.226 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.226 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.226 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:17.226 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906349+0000", 2024-05-08T08:54:17.226 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1537000000000004e-05 2024-05-08T08:54:17.226 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.226 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.226 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:17.226 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906422+0000", 2024-05-08T08:54:17.226 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3431999999999998e-05 2024-05-08T08:54:17.227 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.227 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:17.227 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.227 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.227 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.227 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22095 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 946176~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:17.227 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:54:17.227 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 109.330225337, 2024-05-08T08:54:17.227 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048657593999999998, 2024-05-08T08:54:17.227 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:17.227 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:17.227 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:17.227 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:17.227 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:17.227 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:17.227 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22095 2024-05-08T08:54:17.228 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.228 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:17.228 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.228 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:17.228 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:54:17.228 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:17.228 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.228 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.228 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:17.228 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:54:17.228 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:17.228 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.228 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.228 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:17.228 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857925+0000", 2024-05-08T08:54:17.228 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3099999999999998e-07 2024-05-08T08:54:17.228 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.229 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.229 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:17.229 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857927+0000", 2024-05-08T08:54:17.229 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.34e-06 2024-05-08T08:54:17.229 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.229 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.229 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:17.229 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857938+0000", 2024-05-08T08:54:17.229 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1426e-05 2024-05-08T08:54:17.229 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.229 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.229 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:17.229 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857943+0000", 2024-05-08T08:54:17.229 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5140000000000003e-06 2024-05-08T08:54:17.229 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.230 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.230 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:17.230 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906448+0000", 2024-05-08T08:54:17.230 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048504641000000001 2024-05-08T08:54:17.230 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.230 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.230 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:17.230 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906510+0000", 2024-05-08T08:54:17.230 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2663000000000006e-05 2024-05-08T08:54:17.230 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.230 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.230 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:17.230 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906582+0000", 2024-05-08T08:54:17.230 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1478999999999998e-05 2024-05-08T08:54:17.230 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.231 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:17.231 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.231 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.231 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.231 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22097 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 962560~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:17.231 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:54:17.231 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 109.330197648, 2024-05-08T08:54:17.231 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048791353000000003, 2024-05-08T08:54:17.231 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:17.231 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:17.231 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:17.231 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:17.231 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:17.231 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:17.231 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22097 2024-05-08T08:54:17.231 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.232 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:17.232 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.232 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:17.232 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:54:17.232 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:17.232 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.232 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.232 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:17.232 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:54:17.232 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:17.232 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.232 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.232 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:17.232 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:54:17.232 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9200000000000002e-07 2024-05-08T08:54:17.232 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.232 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.233 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:17.233 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857954+0000", 2024-05-08T08:54:17.233 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1900000000000002e-06 2024-05-08T08:54:17.233 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.233 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.233 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:17.233 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857961+0000", 2024-05-08T08:54:17.233 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4989999999999999e-06 2024-05-08T08:54:17.233 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.233 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.233 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:17.233 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857971+0000", 2024-05-08T08:54:17.233 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0485999999999999e-05 2024-05-08T08:54:17.233 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.233 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.234 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:17.234 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906607+0000", 2024-05-08T08:54:17.234 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048635109000000003 2024-05-08T08:54:17.234 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.234 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.234 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:17.234 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906667+0000", 2024-05-08T08:54:17.234 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0439000000000002e-05 2024-05-08T08:54:17.234 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.234 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.234 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:17.234 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906743+0000", 2024-05-08T08:54:17.234 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6237999999999998e-05 2024-05-08T08:54:17.234 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.235 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:17.235 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.235 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.235 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.235 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22099 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 978944~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:17.235 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:54:17.235 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 109.330171488, 2024-05-08T08:54:17.235 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048932055000000002, 2024-05-08T08:54:17.235 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:17.235 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:17.235 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:17.235 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:17.235 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:17.235 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:17.235 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22099 2024-05-08T08:54:17.235 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.236 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:17.236 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.236 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:17.236 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:54:17.236 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:17.236 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.236 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.236 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:17.236 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:54:17.236 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:17.236 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.236 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.236 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:17.236 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:54:17.236 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.27e-07 2024-05-08T08:54:17.237 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.237 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.237 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:17.237 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857987+0000", 2024-05-08T08:54:17.237 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5890000000000003e-06 2024-05-08T08:54:17.237 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.237 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.237 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:17.237 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857994+0000", 2024-05-08T08:54:17.237 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6619999999999999e-06 2024-05-08T08:54:17.237 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.237 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.237 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:17.237 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857998+0000", 2024-05-08T08:54:17.237 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7110000000000003e-06 2024-05-08T08:54:17.238 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.238 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.238 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:17.238 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906772+0000", 2024-05-08T08:54:17.238 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048773415000000001 2024-05-08T08:54:17.238 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.238 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.238 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:17.238 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906834+0000", 2024-05-08T08:54:17.238 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1752000000000001e-05 2024-05-08T08:54:17.238 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.238 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.238 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:17.238 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906910+0000", 2024-05-08T08:54:17.239 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6499000000000001e-05 2024-05-08T08:54:17.239 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.239 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:17.239 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.239 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.239 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.239 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22101 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 995328~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:17.239 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:54:17.239 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 109.33014435699999, 2024-05-08T08:54:17.239 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049087364000000001, 2024-05-08T08:54:17.240 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:17.240 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:17.240 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:17.240 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:17.240 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:17.240 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:17.240 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22101 2024-05-08T08:54:17.240 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.240 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:17.240 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.240 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:17.240 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:54:17.240 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:17.240 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.240 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.240 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:17.241 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:54:17.241 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:17.241 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.241 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.241 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:17.241 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858006+0000", 2024-05-08T08:54:17.241 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1399999999999997e-07 2024-05-08T08:54:17.241 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.241 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.241 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:17.241 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858008+0000", 2024-05-08T08:54:17.241 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2060000000000001e-06 2024-05-08T08:54:17.241 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.241 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.241 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:17.242 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858017+0000", 2024-05-08T08:54:17.242 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1579999999999994e-06 2024-05-08T08:54:17.242 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.242 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.242 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:17.242 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858022+0000", 2024-05-08T08:54:17.242 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4199999999999998e-06 2024-05-08T08:54:17.242 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.242 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.242 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:17.242 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906940+0000", 2024-05-08T08:54:17.242 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048917627999999998 2024-05-08T08:54:17.242 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.242 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.242 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:17.242 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907002+0000", 2024-05-08T08:54:17.243 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2399999999999999e-05 2024-05-08T08:54:17.243 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.243 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.243 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:17.243 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907092+0000", 2024-05-08T08:54:17.243 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0137999999999997e-05 2024-05-08T08:54:17.243 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.243 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:17.243 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.243 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.243 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.243 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22103 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1011712~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:17.243 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:54:17.243 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 109.330119572, 2024-05-08T08:54:17.243 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049228542, 2024-05-08T08:54:17.244 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:17.244 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:17.244 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:17.244 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:17.244 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:17.244 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:17.244 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22103 2024-05-08T08:54:17.244 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.244 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:17.244 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.244 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:17.244 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:54:17.244 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:17.244 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.244 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.244 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:17.245 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:54:17.245 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:17.245 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.245 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.245 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:17.245 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:54:17.245 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4299999999999999e-07 2024-05-08T08:54:17.245 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.245 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.245 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:17.245 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858032+0000", 2024-05-08T08:54:17.245 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0590000000000001e-06 2024-05-08T08:54:17.245 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.245 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.245 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:17.245 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858037+0000", 2024-05-08T08:54:17.246 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.977e-06 2024-05-08T08:54:17.246 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.246 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.246 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:17.246 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858042+0000", 2024-05-08T08:54:17.246 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7650000000000001e-06 2024-05-08T08:54:17.246 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.246 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.246 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:17.246 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907118+0000", 2024-05-08T08:54:17.246 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049076147000000001 2024-05-08T08:54:17.246 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.246 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.246 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:17.246 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907180+0000", 2024-05-08T08:54:17.247 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1971999999999996e-05 2024-05-08T08:54:17.247 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.247 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.247 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:17.247 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907258+0000", 2024-05-08T08:54:17.247 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8279e-05 2024-05-08T08:54:17.247 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.247 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:17.247 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.247 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.247 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.247 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22105 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1028096~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:17.247 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:54:17.247 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 109.330100329, 2024-05-08T08:54:17.247 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049394609999999999, 2024-05-08T08:54:17.248 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:17.248 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:17.248 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:17.248 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:17.248 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:17.248 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:17.248 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22105 2024-05-08T08:54:17.248 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.248 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:17.248 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.248 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:17.248 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:54:17.248 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:17.249 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.249 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.249 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:17.249 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:54:17.249 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:17.249 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.249 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.249 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:17.249 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858050+0000", 2024-05-08T08:54:17.249 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5999999999999999e-07 2024-05-08T08:54:17.249 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.249 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.249 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:17.249 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858051+0000", 2024-05-08T08:54:17.250 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8870000000000001e-06 2024-05-08T08:54:17.250 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.250 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.250 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:17.250 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858057+0000", 2024-05-08T08:54:17.250 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8019999999999997e-06 2024-05-08T08:54:17.250 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.250 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.250 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:17.250 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858062+0000", 2024-05-08T08:54:17.250 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.561e-06 2024-05-08T08:54:17.250 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.250 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.250 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:17.251 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907289+0000", 2024-05-08T08:54:17.251 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049226957000000002 2024-05-08T08:54:17.251 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.251 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.251 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:17.251 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907356+0000", 2024-05-08T08:54:17.251 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6915000000000006e-05 2024-05-08T08:54:17.251 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.251 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.251 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:17.251 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907444+0000", 2024-05-08T08:54:17.251 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8127999999999994e-05 2024-05-08T08:54:17.251 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.251 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:17.251 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.251 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.252 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.252 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22107 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1044480~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:17.252 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:54:17.252 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 109.33008198900001, 2024-05-08T08:54:17.252 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049799942999999999, 2024-05-08T08:54:17.252 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:17.252 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:17.252 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:17.252 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:17.252 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:17.252 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:17.252 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22107 2024-05-08T08:54:17.252 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.252 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:17.253 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.253 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:17.253 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:54:17.253 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:17.253 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.253 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.253 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:17.253 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:54:17.253 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:17.253 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.253 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.253 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:17.253 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:54:17.253 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4799999999999999e-07 2024-05-08T08:54:17.253 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.254 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.254 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:17.254 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858070+0000", 2024-05-08T08:54:17.254 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.012e-06 2024-05-08T08:54:17.254 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.254 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.254 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:17.254 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858077+0000", 2024-05-08T08:54:17.254 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6290000000000001e-06 2024-05-08T08:54:17.254 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.254 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.254 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:17.254 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858083+0000", 2024-05-08T08:54:17.254 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8880000000000002e-06 2024-05-08T08:54:17.255 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.255 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.255 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:17.255 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907671+0000", 2024-05-08T08:54:17.255 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049587191000000003 2024-05-08T08:54:17.255 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.255 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.255 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:17.255 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907763+0000", 2024-05-08T08:54:17.255 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2231000000000005e-05 2024-05-08T08:54:17.255 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.255 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.255 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:17.255 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907867+0000", 2024-05-08T08:54:17.255 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104644 2024-05-08T08:54:17.255 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.256 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:17.256 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.256 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.256 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.256 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22294 3.1c 3:3aad0389:::rbd_data.11406886b07b.0000000000000080:head [stat,delete] snapc 7=[7] RETRY=6 ondisk+retry+write+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:17.256 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:54:17.256 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 109.329488141, 2024-05-08T08:54:17.256 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051152191, 2024-05-08T08:54:17.256 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:17.256 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:17.256 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:17.256 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:17.256 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:17.256 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:17.256 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22294 2024-05-08T08:54:17.256 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.256 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:17.257 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.257 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:17.257 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:54:17.257 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:17.257 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.257 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.257 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:17.257 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:54:17.257 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:17.257 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.257 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.257 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:17.257 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858662+0000", 2024-05-08T08:54:17.257 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4799999999999998e-07 2024-05-08T08:54:17.257 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.258 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.258 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:17.258 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858664+0000", 2024-05-08T08:54:17.258 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1229999999999998e-06 2024-05-08T08:54:17.258 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.258 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.258 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:17.258 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858670+0000", 2024-05-08T08:54:17.258 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3099999999999997e-06 2024-05-08T08:54:17.258 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.258 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.258 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:17.258 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858676+0000", 2024-05-08T08:54:17.259 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9750000000000004e-06 2024-05-08T08:54:17.259 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.259 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.259 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:17.259 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907901+0000", 2024-05-08T08:54:17.259 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049224520000000001 2024-05-08T08:54:17.259 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.259 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.259 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:17.259 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.908085+0000", 2024-05-08T08:54:17.259 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00018377500000000001 2024-05-08T08:54:17.259 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.259 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.259 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:17.259 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.909814+0000", 2024-05-08T08:54:17.259 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00172894 2024-05-08T08:54:17.260 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.260 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:17.260 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.260 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.260 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.260 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4416.0:13461 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-05-08T08:54:17.260 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:22.487456+0000", 2024-05-08T08:54:17.260 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 54.700693100000002, 2024-05-08T08:54:17.260 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.99102620799999996, 2024-05-08T08:54:17.260 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:17.260 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:17.260 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:17.260 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:17.260 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4416", 2024-05-08T08:54:17.260 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:54:17.260 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 13461 2024-05-08T08:54:17.261 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.261 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:17.261 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.261 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:17.261 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487456+0000", 2024-05-08T08:54:17.261 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:17.261 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.261 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.261 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:17.261 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487456+0000", 2024-05-08T08:54:17.261 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:17.261 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.261 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.261 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:17.261 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487458+0000", 2024-05-08T08:54:17.261 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5659999999999999e-06 2024-05-08T08:54:17.261 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.262 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.262 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:17.262 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487463+0000", 2024-05-08T08:54:17.262 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6890000000000003e-06 2024-05-08T08:54:17.262 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.262 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.262 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:17.262 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487472+0000", 2024-05-08T08:54:17.262 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7710000000000001e-06 2024-05-08T08:54:17.262 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.262 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.262 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:17.262 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487478+0000", 2024-05-08T08:54:17.262 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1390000000000003e-06 2024-05-08T08:54:17.263 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.263 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.263 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:17.263 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487520+0000", 2024-05-08T08:54:17.263 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2259999999999999e-05 2024-05-08T08:54:17.263 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.263 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.263 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-05-08T08:54:17.263 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487528+0000", 2024-05-08T08:54:17.263 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7479999999999993e-06 2024-05-08T08:54:17.263 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.263 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.263 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:17.263 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:23.478139+0000", 2024-05-08T08:54:17.263 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.99061165200000001 2024-05-08T08:54:17.263 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.264 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.264 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:17.264 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:23.478301+0000", 2024-05-08T08:54:17.264 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000161602 2024-05-08T08:54:17.264 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.264 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.264 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:17.264 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:23.478483+0000", 2024-05-08T08:54:17.264 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000181781 2024-05-08T08:54:17.264 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.264 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:17.264 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.264 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.264 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.264 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5752.0:404 3.1a 3:5f5e8104:::rbd_data.1140bc4a55c7.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 5175296~45056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-05-08T08:54:17.264 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:00.390863+0000", 2024-05-08T08:54:17.265 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 16.797286905, 2024-05-08T08:54:17.265 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.055189174000000001, 2024-05-08T08:54:17.265 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:17.265 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:17.265 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:54:17.265 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:17.265 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5752", 2024-05-08T08:54:17.265 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/3534419815", 2024-05-08T08:54:17.265 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 404 2024-05-08T08:54:17.265 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.265 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:17.265 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.265 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:17.265 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.390863+0000", 2024-05-08T08:54:17.265 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:17.265 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.266 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.266 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:17.266 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.390863+0000", 2024-05-08T08:54:17.266 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:17.266 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.266 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.266 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:17.266 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.390864+0000", 2024-05-08T08:54:17.266 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.471e-06 2024-05-08T08:54:17.266 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.266 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.266 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:17.266 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397475+0000", 2024-05-08T08:54:17.266 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0066109109999999997 2024-05-08T08:54:17.267 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.267 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.267 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:17.267 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397498+0000", 2024-05-08T08:54:17.267 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2674e-05 2024-05-08T08:54:17.267 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.267 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.267 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:17.267 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397508+0000", 2024-05-08T08:54:17.267 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9019999999999994e-06 2024-05-08T08:54:17.267 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.267 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.267 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:17.267 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397609+0000", 2024-05-08T08:54:17.267 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010121899999999999 2024-05-08T08:54:17.267 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.268 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.268 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:17.268 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397711+0000", 2024-05-08T08:54:17.268 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102489 2024-05-08T08:54:17.268 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.268 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.268 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-05-08T08:54:17.268 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397999+0000", 2024-05-08T08:54:17.268 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00028733799999999999 2024-05-08T08:54:17.268 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.268 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.268 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:54:17.268 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.432093+0000", 2024-05-08T08:54:17.268 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.034094865000000002 2024-05-08T08:54:17.268 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.269 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.269 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:54:17.269 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.445975+0000", 2024-05-08T08:54:17.269 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013881187 2024-05-08T08:54:17.269 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.269 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.269 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:54:17.269 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.446013+0000", 2024-05-08T08:54:17.269 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8155000000000003e-05 2024-05-08T08:54:17.269 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.269 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.269 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:17.269 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.446052+0000", 2024-05-08T08:54:17.269 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8962999999999999e-05 2024-05-08T08:54:17.270 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.270 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:17.270 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.270 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.270 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:17.270 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:17.271 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:54:17.272 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:17.274+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:17.371 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:17.374+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:17.377 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:17.377 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:54:17.377 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:54:17.377 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:17.386 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:54:17.473 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:17.476+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:17.490 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:17.490 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:54:17.490 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:54:17.490 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:54:17.491 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:17.499 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:54:17.574 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:17.576+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:17.606 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:17.606 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:54:17.606 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:54:17.606 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:54:17.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.606 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6690 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1093632~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:17.607 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:54:17.607 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 105.893061062, 2024-05-08T08:54:17.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052462754, 2024-05-08T08:54:17.607 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:17.607 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:17.607 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:17.607 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:17.607 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:17.607 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:17.607 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6690 2024-05-08T08:54:17.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.607 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:17.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.607 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:17.607 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:54:17.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:17.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.608 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:17.608 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:54:17.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:17.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.608 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:17.608 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715346+0000", 2024-05-08T08:54:17.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.84e-07 2024-05-08T08:54:17.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.608 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:17.608 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715348+0000", 2024-05-08T08:54:17.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.114e-06 2024-05-08T08:54:17.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.609 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:17.609 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715353+0000", 2024-05-08T08:54:17.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5720000000000002e-06 2024-05-08T08:54:17.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.609 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:17.609 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715358+0000", 2024-05-08T08:54:17.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7140000000000001e-06 2024-05-08T08:54:17.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.610 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:17.610 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767638+0000", 2024-05-08T08:54:17.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052279562000000002 2024-05-08T08:54:17.610 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.610 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:17.610 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767713+0000", 2024-05-08T08:54:17.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5652e-05 2024-05-08T08:54:17.610 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.610 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:17.610 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767808+0000", 2024-05-08T08:54:17.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4755999999999999e-05 2024-05-08T08:54:17.611 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.611 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:17.611 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.611 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6692 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1110016~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:17.611 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:54:17.611 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 105.89304197200001, 2024-05-08T08:54:17.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052666495000000001, 2024-05-08T08:54:17.611 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:17.611 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:17.611 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:17.611 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:17.611 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:17.612 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:17.612 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6692 2024-05-08T08:54:17.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.612 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:17.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.612 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:17.612 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:54:17.612 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:17.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.612 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:17.612 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:54:17.612 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:17.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.612 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:17.613 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:54:17.613 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8599999999999999e-07 2024-05-08T08:54:17.613 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.613 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.613 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:17.613 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715372+0000", 2024-05-08T08:54:17.613 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0990000000000001e-06 2024-05-08T08:54:17.613 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.613 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.613 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:17.613 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715378+0000", 2024-05-08T08:54:17.613 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0680000000000002e-06 2024-05-08T08:54:17.613 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.613 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.614 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:17.614 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715383+0000", 2024-05-08T08:54:17.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8740000000000003e-06 2024-05-08T08:54:17.614 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.614 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.614 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:17.614 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767848+0000", 2024-05-08T08:54:17.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052464668999999998 2024-05-08T08:54:17.614 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.614 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.614 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:17.614 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767927+0000", 2024-05-08T08:54:17.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9228999999999996e-05 2024-05-08T08:54:17.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:17.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768031+0000", 2024-05-08T08:54:17.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010417 2024-05-08T08:54:17.615 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.615 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:17.615 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.615 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6694 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1126400~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:17.615 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:54:17.615 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 105.893017075, 2024-05-08T08:54:17.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052848829, 2024-05-08T08:54:17.615 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:17.616 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:17.616 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:17.616 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:17.616 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:17.616 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:17.616 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6694 2024-05-08T08:54:17.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.616 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:17.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:17.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:54:17.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:17.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:17.617 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:54:17.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:17.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.617 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.617 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:17.617 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715390+0000", 2024-05-08T08:54:17.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9900000000000001e-07 2024-05-08T08:54:17.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.617 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.617 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:17.617 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715392+0000", 2024-05-08T08:54:17.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8279999999999999e-06 2024-05-08T08:54:17.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.617 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.617 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:17.617 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715398+0000", 2024-05-08T08:54:17.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7619999999999998e-06 2024-05-08T08:54:17.618 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.618 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.618 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:17.618 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715403+0000", 2024-05-08T08:54:17.618 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7790000000000003e-06 2024-05-08T08:54:17.618 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.618 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.618 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:17.618 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768067+0000", 2024-05-08T08:54:17.618 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052663785999999997 2024-05-08T08:54:17.618 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.618 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.618 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:17.618 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768142+0000", 2024-05-08T08:54:17.618 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5018999999999994e-05 2024-05-08T08:54:17.619 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.619 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.619 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:17.619 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768238+0000", 2024-05-08T08:54:17.619 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6255999999999995e-05 2024-05-08T08:54:17.619 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.619 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:17.619 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.619 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.619 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.619 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6696 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1142784~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:17.619 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:54:17.619 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 105.89299756699999, 2024-05-08T08:54:17.619 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.053051057999999998, 2024-05-08T08:54:17.619 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:17.619 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:17.620 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:17.620 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:17.620 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:17.620 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:17.620 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6696 2024-05-08T08:54:17.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.620 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:17.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.620 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:17.620 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:54:17.620 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:17.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.620 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:17.620 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:54:17.620 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:17.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:17.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:54:17.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-05-08T08:54:17.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:17.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715411+0000", 2024-05-08T08:54:17.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9630000000000001e-06 2024-05-08T08:54:17.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:17.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715417+0000", 2024-05-08T08:54:17.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7250000000000002e-06 2024-05-08T08:54:17.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.622 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.622 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:17.622 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715421+0000", 2024-05-08T08:54:17.622 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4089999999999996e-06 2024-05-08T08:54:17.622 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.622 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.622 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:17.622 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768277+0000", 2024-05-08T08:54:17.622 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052855293999999997 2024-05-08T08:54:17.622 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.622 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.622 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:17.622 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768358+0000", 2024-05-08T08:54:17.622 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1720999999999999e-05 2024-05-08T08:54:17.622 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.623 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.623 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:17.623 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768460+0000", 2024-05-08T08:54:17.623 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010154600000000001 2024-05-08T08:54:17.623 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.623 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:17.623 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.623 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.623 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.623 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6698 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1159168~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:17.623 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:54:17.623 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 105.8929787, 2024-05-08T08:54:17.623 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.053245726, 2024-05-08T08:54:17.623 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:17.623 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:17.624 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:17.624 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:17.624 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:17.624 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:17.624 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6698 2024-05-08T08:54:17.624 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.624 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:17.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.624 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:17.624 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:54:17.624 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:17.624 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.624 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:17.624 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:54:17.624 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:17.624 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.625 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.625 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:17.625 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716936+0000", 2024-05-08T08:54:17.625 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001508665 2024-05-08T08:54:17.625 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.625 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.625 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:17.625 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716941+0000", 2024-05-08T08:54:17.625 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1910000000000002e-06 2024-05-08T08:54:17.625 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.625 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.625 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:17.625 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716950+0000", 2024-05-08T08:54:17.625 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1900000000000001e-06 2024-05-08T08:54:17.626 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.626 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.626 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:17.626 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716958+0000", 2024-05-08T08:54:17.626 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8019999999999992e-06 2024-05-08T08:54:17.626 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.626 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.626 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:17.626 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768499+0000", 2024-05-08T08:54:17.626 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051540926000000001 2024-05-08T08:54:17.626 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.626 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.626 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:17.626 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768574+0000", 2024-05-08T08:54:17.626 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4988999999999996e-05 2024-05-08T08:54:17.627 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.627 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.627 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:17.627 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768673+0000", 2024-05-08T08:54:17.627 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9963000000000005e-05 2024-05-08T08:54:17.627 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.627 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:17.627 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.627 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.627 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.627 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6706 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1224704~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:17.627 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:54:17.627 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 105.89137339299999, 2024-05-08T08:54:17.627 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052435598, 2024-05-08T08:54:17.627 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:17.627 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:17.628 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:17.628 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:17.628 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:17.628 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:17.628 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6706 2024-05-08T08:54:17.628 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.628 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:17.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.628 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:17.628 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:54:17.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:17.628 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.628 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:17.628 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:54:17.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:17.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.629 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:17.629 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:54:17.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:54:17.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.629 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:17.629 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717035+0000", 2024-05-08T08:54:17.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9290000000000001e-06 2024-05-08T08:54:17.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.629 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:17.629 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717041+0000", 2024-05-08T08:54:17.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6720000000000001e-06 2024-05-08T08:54:17.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.630 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.630 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:17.630 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717046+0000", 2024-05-08T08:54:17.630 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5650000000000003e-06 2024-05-08T08:54:17.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.630 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.630 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:17.630 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769311+0000", 2024-05-08T08:54:17.630 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052265645999999999 2024-05-08T08:54:17.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.630 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.630 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:17.630 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769379+0000", 2024-05-08T08:54:17.631 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7391000000000006e-05 2024-05-08T08:54:17.631 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.631 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.631 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:17.631 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769469+0000", 2024-05-08T08:54:17.631 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0025999999999995e-05 2024-05-08T08:54:17.631 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.631 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:17.631 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.631 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.631 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.631 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6708 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1241088~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:17.631 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:54:17.631 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 105.891355289, 2024-05-08T08:54:17.631 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052640552, 2024-05-08T08:54:17.632 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:17.632 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:17.632 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:17.632 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:17.632 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:17.632 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:17.632 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6708 2024-05-08T08:54:17.632 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.632 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:17.632 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.632 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:17.632 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:54:17.632 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:17.632 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.633 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.633 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:17.633 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:54:17.633 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:17.633 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.633 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.633 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:17.633 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717052+0000", 2024-05-08T08:54:17.633 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:54:17.633 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.633 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.633 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:17.633 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717053+0000", 2024-05-08T08:54:17.633 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8330000000000001e-06 2024-05-08T08:54:17.634 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.634 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:17.634 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717058+0000", 2024-05-08T08:54:17.634 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.933e-06 2024-05-08T08:54:17.634 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.634 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:17.634 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717063+0000", 2024-05-08T08:54:17.634 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2180000000000001e-06 2024-05-08T08:54:17.634 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.635 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:17.635 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769507+0000", 2024-05-08T08:54:17.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052444866999999999 2024-05-08T08:54:17.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.635 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.635 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:17.635 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769589+0000", 2024-05-08T08:54:17.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2039000000000005e-05 2024-05-08T08:54:17.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.635 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.635 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:17.635 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769692+0000", 2024-05-08T08:54:17.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102252 2024-05-08T08:54:17.636 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.636 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:17.636 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.636 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.636 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6710 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1257472~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:17.636 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:54:17.636 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 105.891337265, 2024-05-08T08:54:17.636 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.05283057, 2024-05-08T08:54:17.636 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:17.636 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:17.636 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:17.637 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:17.637 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:17.637 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:17.637 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6710 2024-05-08T08:54:17.637 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.637 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:17.637 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.637 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:17.637 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:54:17.637 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:17.637 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.637 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.638 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:17.638 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:54:17.638 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:17.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.638 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.638 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:17.638 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718951+0000", 2024-05-08T08:54:17.638 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0018817809999999999 2024-05-08T08:54:17.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.638 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.638 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:17.638 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718956+0000", 2024-05-08T08:54:17.638 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5399999999999997e-06 2024-05-08T08:54:17.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.639 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.639 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:17.639 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718964+0000", 2024-05-08T08:54:17.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6629999999999999e-06 2024-05-08T08:54:17.639 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.639 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.639 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:17.639 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718972+0000", 2024-05-08T08:54:17.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5229999999999998e-06 2024-05-08T08:54:17.639 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.639 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.639 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:17.639 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769729+0000", 2024-05-08T08:54:17.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050756959999999997 2024-05-08T08:54:17.639 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.640 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.640 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:17.640 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769803+0000", 2024-05-08T08:54:17.640 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4538999999999999e-05 2024-05-08T08:54:17.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.640 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.640 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:17.640 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769900+0000", 2024-05-08T08:54:17.640 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6564000000000006e-05 2024-05-08T08:54:17.640 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.640 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:17.640 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.640 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.640 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6726 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1388544~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:17.641 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:54:17.641 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 105.889262493, 2024-05-08T08:54:17.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052451506000000002, 2024-05-08T08:54:17.641 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:17.641 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:17.641 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:17.641 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:17.641 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:17.641 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:17.641 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6726 2024-05-08T08:54:17.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.641 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:17.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.641 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:17.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:54:17.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:17.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:17.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:54:17.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:17.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:17.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719200+0000", 2024-05-08T08:54:17.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5865999999999997e-05 2024-05-08T08:54:17.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:17.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719204+0000", 2024-05-08T08:54:17.643 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8630000000000001e-06 2024-05-08T08:54:17.643 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.643 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.643 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:17.643 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719215+0000", 2024-05-08T08:54:17.643 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1590000000000001e-05 2024-05-08T08:54:17.643 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.643 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.643 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:17.643 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719220+0000", 2024-05-08T08:54:17.643 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1619999999999999e-06 2024-05-08T08:54:17.643 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.643 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.643 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:17.644 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771425+0000", 2024-05-08T08:54:17.644 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052204249000000001 2024-05-08T08:54:17.644 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.644 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.644 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:17.644 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771498+0000", 2024-05-08T08:54:17.644 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3782999999999995e-05 2024-05-08T08:54:17.644 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.644 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.644 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:17.644 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771595+0000", 2024-05-08T08:54:17.644 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6992999999999998e-05 2024-05-08T08:54:17.644 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.645 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:17.645 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.645 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.645 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6728 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1404928~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:17.645 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:54:17.645 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 105.889175088, 2024-05-08T08:54:17.645 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052590747, 2024-05-08T08:54:17.645 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:17.645 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:17.645 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:17.645 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:17.645 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:17.645 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:17.645 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6728 2024-05-08T08:54:17.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.646 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:17.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.646 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:17.646 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:54:17.646 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:17.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.646 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:17.646 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:54:17.646 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:17.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.646 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:17.647 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719232+0000", 2024-05-08T08:54:17.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4000000000000001e-07 2024-05-08T08:54:17.647 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.647 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.647 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:17.647 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719234+0000", 2024-05-08T08:54:17.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4499999999999998e-06 2024-05-08T08:54:17.647 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.647 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.647 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:17.647 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719240+0000", 2024-05-08T08:54:17.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5720000000000002e-06 2024-05-08T08:54:17.647 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.647 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.647 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:17.648 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719245+0000", 2024-05-08T08:54:17.648 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8690000000000003e-06 2024-05-08T08:54:17.648 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.648 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.648 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:17.648 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771636+0000", 2024-05-08T08:54:17.648 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052390979999999997 2024-05-08T08:54:17.648 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.648 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.648 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:17.648 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771717+0000", 2024-05-08T08:54:17.648 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1291999999999993e-05 2024-05-08T08:54:17.648 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.648 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.648 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:17.649 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771822+0000", 2024-05-08T08:54:17.649 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104944 2024-05-08T08:54:17.649 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.649 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:17.649 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.649 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.649 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.649 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6730 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1421312~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:17.649 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:54:17.649 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 105.889154846, 2024-05-08T08:54:17.649 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052770477000000003, 2024-05-08T08:54:17.649 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:17.649 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:17.650 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:17.650 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:17.650 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:17.650 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:17.650 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6730 2024-05-08T08:54:17.650 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.650 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:17.650 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.650 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:17.650 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:54:17.650 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:17.650 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.650 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.650 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:17.651 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:54:17.651 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:17.651 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.651 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.651 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:17.651 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:54:17.651 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9000000000000002e-07 2024-05-08T08:54:17.651 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.651 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.651 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:17.651 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719254+0000", 2024-05-08T08:54:17.651 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4099999999999998e-06 2024-05-08T08:54:17.651 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.651 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.652 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:17.652 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719263+0000", 2024-05-08T08:54:17.652 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4109999999999996e-06 2024-05-08T08:54:17.652 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.652 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.652 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:17.652 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719268+0000", 2024-05-08T08:54:17.652 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2669999999999997e-06 2024-05-08T08:54:17.652 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.652 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.652 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:17.652 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771846+0000", 2024-05-08T08:54:17.652 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052577677000000003 2024-05-08T08:54:17.652 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.652 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.652 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:17.653 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771920+0000", 2024-05-08T08:54:17.653 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4460000000000002e-05 2024-05-08T08:54:17.653 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.653 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.653 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:17.653 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772022+0000", 2024-05-08T08:54:17.653 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010186199999999999 2024-05-08T08:54:17.653 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.653 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:17.653 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.653 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.653 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.653 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6732 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1437696~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:17.654 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:54:17.654 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 105.88913003899999, 2024-05-08T08:54:17.654 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052905575000000003, 2024-05-08T08:54:17.654 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:17.654 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:17.654 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:17.654 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:17.654 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:17.654 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:17.654 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6732 2024-05-08T08:54:17.654 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.654 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:17.654 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.654 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:17.655 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:54:17.655 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:17.655 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.655 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.655 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:17.655 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:54:17.655 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:17.655 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.655 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.655 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:17.655 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721392+0000", 2024-05-08T08:54:17.655 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002115287 2024-05-08T08:54:17.655 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.655 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.655 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:17.655 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721396+0000", 2024-05-08T08:54:17.656 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1679999999999997e-06 2024-05-08T08:54:17.656 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.656 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.656 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:17.656 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721404+0000", 2024-05-08T08:54:17.656 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4409999999999993e-06 2024-05-08T08:54:17.656 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.656 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.656 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:17.656 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721410+0000", 2024-05-08T08:54:17.656 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9150000000000001e-06 2024-05-08T08:54:17.656 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.656 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.656 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:17.656 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772061+0000", 2024-05-08T08:54:17.656 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050651068 2024-05-08T08:54:17.657 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.657 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.657 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:17.657 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772127+0000", 2024-05-08T08:54:17.657 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5424000000000002e-05 2024-05-08T08:54:17.657 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.657 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.657 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:17.657 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772182+0000", 2024-05-08T08:54:17.657 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5272000000000002e-05 2024-05-08T08:54:17.657 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.657 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:17.657 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.657 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.657 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.657 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6749 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1585152~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:17.657 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:54:17.658 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 105.886829269, 2024-05-08T08:54:17.658 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052350264000000001, 2024-05-08T08:54:17.658 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:17.658 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:17.658 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:17.658 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:17.658 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:17.658 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:17.658 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6749 2024-05-08T08:54:17.658 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.658 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:17.658 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.658 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:17.658 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:54:17.659 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:17.659 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.659 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.659 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:17.659 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:54:17.659 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:17.659 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.659 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.659 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:17.659 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721578+0000", 2024-05-08T08:54:17.659 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0399999999999996e-07 2024-05-08T08:54:17.659 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.659 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.659 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:17.659 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721580+0000", 2024-05-08T08:54:17.660 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5450000000000002e-06 2024-05-08T08:54:17.660 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.660 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.660 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:17.660 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721587+0000", 2024-05-08T08:54:17.660 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0190000000000001e-06 2024-05-08T08:54:17.660 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.660 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.660 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:17.660 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721593+0000", 2024-05-08T08:54:17.660 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5659999999999997e-06 2024-05-08T08:54:17.660 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.660 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.661 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:17.661 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773759+0000", 2024-05-08T08:54:17.661 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052166553999999997 2024-05-08T08:54:17.661 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.661 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.661 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:17.661 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773833+0000", 2024-05-08T08:54:17.661 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3913999999999997e-05 2024-05-08T08:54:17.661 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.661 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.661 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:17.661 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773927+0000", 2024-05-08T08:54:17.661 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4061999999999995e-05 2024-05-08T08:54:17.661 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.661 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:17.662 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.662 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.662 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.662 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6751 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1601536~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:17.662 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:54:17.662 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 105.88680599, 2024-05-08T08:54:17.662 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052461302000000001, 2024-05-08T08:54:17.662 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:17.662 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:17.662 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:17.662 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:17.662 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:17.662 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:17.662 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6751 2024-05-08T08:54:17.663 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.663 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:17.663 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.663 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:17.663 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:54:17.663 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:17.663 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.663 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.663 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:17.663 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:54:17.663 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:17.663 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.663 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.664 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:17.664 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722939+0000", 2024-05-08T08:54:17.664 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001338601 2024-05-08T08:54:17.664 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.664 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.664 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:17.664 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722944+0000", 2024-05-08T08:54:17.664 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4469999999999999e-06 2024-05-08T08:54:17.664 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.664 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.664 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:17.664 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722955+0000", 2024-05-08T08:54:17.664 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1892999999999999e-05 2024-05-08T08:54:17.664 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.665 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.665 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:17.665 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722963+0000", 2024-05-08T08:54:17.665 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1080000000000004e-06 2024-05-08T08:54:17.665 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.665 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.665 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:17.665 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773964+0000", 2024-05-08T08:54:17.665 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051001022 2024-05-08T08:54:17.665 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.665 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.665 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:17.665 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.774006+0000", 2024-05-08T08:54:17.665 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2729999999999999e-05 2024-05-08T08:54:17.666 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.666 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.666 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:17.666 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.774062+0000", 2024-05-08T08:54:17.666 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5501000000000003e-05 2024-05-08T08:54:17.666 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.666 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:17.666 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.666 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.666 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.666 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6760 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1683456~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:17.666 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:54:17.666 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 105.885296614, 2024-05-08T08:54:17.667 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052571312000000002, 2024-05-08T08:54:17.667 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:17.667 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:17.667 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:17.667 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:17.667 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:17.667 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:17.667 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6760 2024-05-08T08:54:17.667 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.667 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:17.667 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.667 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:17.667 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:54:17.667 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:17.668 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.668 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.668 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:17.668 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:54:17.668 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:17.668 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.668 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.668 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:17.668 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:54:17.668 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2100000000000002e-07 2024-05-08T08:54:17.668 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.668 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.668 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:17.668 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723113+0000", 2024-05-08T08:54:17.668 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3149999999999999e-06 2024-05-08T08:54:17.669 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.669 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.669 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:17.669 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723118+0000", 2024-05-08T08:54:17.669 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8860000000000001e-06 2024-05-08T08:54:17.669 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.669 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.669 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:17.669 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723123+0000", 2024-05-08T08:54:17.669 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6770000000000003e-06 2024-05-08T08:54:17.669 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.669 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.669 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:17.669 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775414+0000", 2024-05-08T08:54:17.670 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052290692999999999 2024-05-08T08:54:17.670 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.670 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.670 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:17.670 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775459+0000", 2024-05-08T08:54:17.670 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5639999999999997e-05 2024-05-08T08:54:17.670 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.670 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.670 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:17.670 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775681+0000", 2024-05-08T08:54:17.670 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00022168000000000001 2024-05-08T08:54:17.670 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.670 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:17.671 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.671 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.671 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.671 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6761 3.16 3:6bd6d6da:::rbd_data.1140782590a5.0000000000000015:6 [read 1095680~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:17.671 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:54:17.671 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 105.885277394, 2024-05-08T08:54:17.671 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052751275, 2024-05-08T08:54:17.671 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:17.671 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:17.671 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:17.671 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:17.671 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:17.671 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:17.671 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6761 2024-05-08T08:54:17.671 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.672 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:17.672 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.672 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:17.672 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:54:17.672 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:17.672 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.672 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.672 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:17.672 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:54:17.672 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:17.672 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.672 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.672 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:17.672 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724939+0000", 2024-05-08T08:54:17.673 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001809817 2024-05-08T08:54:17.673 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.673 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.673 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:17.673 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724943+0000", 2024-05-08T08:54:17.673 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4920000000000004e-06 2024-05-08T08:54:17.673 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.673 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.673 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:17.673 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724956+0000", 2024-05-08T08:54:17.673 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2427e-05 2024-05-08T08:54:17.673 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.673 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.673 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:17.674 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724963+0000", 2024-05-08T08:54:17.674 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2559999999999996e-06 2024-05-08T08:54:17.674 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.674 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.674 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:17.674 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775711+0000", 2024-05-08T08:54:17.674 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050747621 2024-05-08T08:54:17.674 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.674 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.674 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:17.674 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775785+0000", 2024-05-08T08:54:17.674 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4634000000000004e-05 2024-05-08T08:54:17.674 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.675 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.675 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:17.675 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775880+0000", 2024-05-08T08:54:17.675 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5027999999999999e-05 2024-05-08T08:54:17.675 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.675 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:17.675 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.675 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.675 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.675 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6770 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1765376~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:17.675 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:54:17.675 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 105.88324629900001, 2024-05-08T08:54:17.675 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052339296, 2024-05-08T08:54:17.675 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:17.676 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:17.676 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:17.676 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:17.676 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:17.676 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:17.676 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6770 2024-05-08T08:54:17.676 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.676 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:17.676 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.676 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:17.676 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:54:17.676 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:17.676 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.676 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.676 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:17.677 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:54:17.677 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:17.677 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.677 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.677 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:17.677 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725161+0000", 2024-05-08T08:54:17.677 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-05-08T08:54:17.677 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.677 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.677 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:17.677 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725163+0000", 2024-05-08T08:54:17.677 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4370000000000001e-06 2024-05-08T08:54:17.677 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.677 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.678 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:17.678 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725169+0000", 2024-05-08T08:54:17.678 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8409999999999996e-06 2024-05-08T08:54:17.678 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.678 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.678 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:17.678 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725174+0000", 2024-05-08T08:54:17.678 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7249999999999997e-06 2024-05-08T08:54:17.678 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.678 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.678 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:17.678 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777343+0000", 2024-05-08T08:54:17.678 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052169633999999999 2024-05-08T08:54:17.678 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.679 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.679 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:17.679 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777409+0000", 2024-05-08T08:54:17.679 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6033999999999998e-05 2024-05-08T08:54:17.679 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.679 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.679 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:17.679 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777499+0000", 2024-05-08T08:54:17.679 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0224999999999998e-05 2024-05-08T08:54:17.679 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.679 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:17.679 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.679 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.679 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.679 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6771 3.16 3:6bd6d6da:::rbd_data.1140782590a5.0000000000000015:6 [read 1177600~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:17.680 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:54:17.680 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 105.883226482, 2024-05-08T08:54:17.680 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052438604999999999, 2024-05-08T08:54:17.680 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:17.680 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:17.680 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:17.680 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:17.680 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:17.680 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:17.680 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6771 2024-05-08T08:54:17.680 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.680 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:17.680 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.680 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:17.680 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:54:17.681 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:17.681 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.681 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.681 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:17.681 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:54:17.681 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:17.681 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.681 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.681 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:17.681 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:54:17.681 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:54:17.681 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.681 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.681 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:17.681 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725188+0000", 2024-05-08T08:54:17.682 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1579999999999999e-06 2024-05-08T08:54:17.682 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.682 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.682 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:17.682 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725195+0000", 2024-05-08T08:54:17.682 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8909999999999995e-06 2024-05-08T08:54:17.682 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.682 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.682 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:17.682 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725200+0000", 2024-05-08T08:54:17.682 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9810000000000003e-06 2024-05-08T08:54:17.682 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.682 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.682 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:17.682 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777527+0000", 2024-05-08T08:54:17.683 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052326582000000003 2024-05-08T08:54:17.683 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.683 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.683 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:17.683 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777566+0000", 2024-05-08T08:54:17.683 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9447999999999999e-05 2024-05-08T08:54:17.683 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.683 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.683 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:17.683 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777619+0000", 2024-05-08T08:54:17.683 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2135000000000001e-05 2024-05-08T08:54:17.683 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.683 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:17.683 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.683 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.683 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.684 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6772 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1781760~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:17.684 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:54:17.684 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 105.883198425, 2024-05-08T08:54:17.684 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052601799999999997, 2024-05-08T08:54:17.684 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:17.684 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:17.684 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:17.684 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:17.684 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:17.684 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:17.684 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6772 2024-05-08T08:54:17.684 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.684 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:17.684 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.684 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:17.685 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:54:17.685 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:17.685 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.685 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.685 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:17.685 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:54:17.685 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:17.685 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.685 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.685 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:17.685 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726939+0000", 2024-05-08T08:54:17.685 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001730493 2024-05-08T08:54:17.685 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.685 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.685 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:17.686 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726943+0000", 2024-05-08T08:54:17.686 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7330000000000003e-06 2024-05-08T08:54:17.686 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.686 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.686 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:17.686 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726954+0000", 2024-05-08T08:54:17.686 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1107000000000001e-05 2024-05-08T08:54:17.686 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.686 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.686 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:17.686 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726962+0000", 2024-05-08T08:54:17.686 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.503e-06 2024-05-08T08:54:17.686 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.687 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.687 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:17.687 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777653+0000", 2024-05-08T08:54:17.687 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050691542999999999 2024-05-08T08:54:17.687 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.687 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.687 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:17.687 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777719+0000", 2024-05-08T08:54:17.687 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5228000000000006e-05 2024-05-08T08:54:17.687 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.687 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.687 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:17.687 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777810+0000", 2024-05-08T08:54:17.687 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1193000000000006e-05 2024-05-08T08:54:17.688 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.688 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:17.688 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.688 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.688 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.688 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6782 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1880064~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:17.688 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:54:17.688 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 105.881229918, 2024-05-08T08:54:17.688 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052326492000000002, 2024-05-08T08:54:17.688 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:17.688 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:17.688 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:17.688 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:17.688 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:17.689 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:17.689 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6782 2024-05-08T08:54:17.689 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.689 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:17.689 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.689 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:17.689 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:54:17.689 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:17.689 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.689 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.689 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:17.689 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:54:17.689 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:17.689 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.690 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.690 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:17.690 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728939+0000", 2024-05-08T08:54:17.690 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001762719 2024-05-08T08:54:17.690 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.690 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.690 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:17.690 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728944+0000", 2024-05-08T08:54:17.690 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.746e-06 2024-05-08T08:54:17.690 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.690 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.690 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:17.690 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728955+0000", 2024-05-08T08:54:17.690 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1043000000000001e-05 2024-05-08T08:54:17.690 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.690 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.691 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:17.691 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728963+0000", 2024-05-08T08:54:17.691 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5809999999999999e-06 2024-05-08T08:54:17.691 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.691 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.691 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:17.691 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779320+0000", 2024-05-08T08:54:17.691 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050357290999999998 2024-05-08T08:54:17.691 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.691 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.691 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:17.691 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779401+0000", 2024-05-08T08:54:17.691 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0574000000000005e-05 2024-05-08T08:54:17.691 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:17.691 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:17.691 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:17.692 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779503+0000", 2024-05-08T08:54:17.692 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102538 2024-05-08T08:54:17.692 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.692 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:17.692 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.692 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:17.692 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:17.692 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:17.694 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:17.696+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:17.694 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:54:17.724 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-05-08T08:54:17.794 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:17.797+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:17.796 INFO:teuthology.orchestra.run.smithi196.stdout:flattening image image_client.0-clone42 2024-05-08T08:54:17.838 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:17.838 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:54:17.838 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:17.838 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(client.5663.0:1251 3.1b e108/88 3:d97f084b:::rbd_object_map.11407ea372d5.000000000000002d:head v 108'526, mlcod=108'524)", 2024-05-08T08:54:17.838 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:54:17.840809+0000", 2024-05-08T08:54:17.838 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.00059824100000000001, 2024-05-08T08:54:17.838 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00062559300000000003, 2024-05-08T08:54:17.838 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:17.838 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:17.838 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:17.839 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:17.839 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:17.839 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:17.839 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:54:17.840809+0000", 2024-05-08T08:54:17.839 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:17.839 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:17.839 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:17.839 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:17.839 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:54:17.840809+0000", 2024-05-08T08:54:17.839 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:17.839 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:17.839 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:17.839 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:17.839 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:54:17.840811+0000", 2024-05-08T08:54:17.839 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.085e-06 2024-05-08T08:54:17.839 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:17.840 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:17.840 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:17.840 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:54:17.840819+0000", 2024-05-08T08:54:17.840 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7150000000000004e-06 2024-05-08T08:54:17.840 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:17.840 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:17.840 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:17.840 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:54:17.840835+0000", 2024-05-08T08:54:17.840 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5571000000000001e-05 2024-05-08T08:54:17.840 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:17.840 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:17.840 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:17.840 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:54:17.840846+0000", 2024-05-08T08:54:17.840 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0652999999999999e-05 2024-05-08T08:54:17.840 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:17.840 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:17.840 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:17.841 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:54:17.840916+0000", 2024-05-08T08:54:17.841 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0114e-05 2024-05-08T08:54:17.841 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:17.841 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:17.841 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:17.841 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:54:17.840943+0000", 2024-05-08T08:54:17.841 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7483000000000001e-05 2024-05-08T08:54:17.841 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:17.841 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:17.841 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:17.841 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:17.841 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T08:54:17.841 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 1 2024-05-08T08:54:17.841 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:17.851 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:54:17.895 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:17.898+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:17.996 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:17.999+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:18.002 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:18.002 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:54:18.002 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:54:18.002 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:54:18.002 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:18.011 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:54:18.097 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:18.099+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:18.166 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:18.166 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:54:18.166 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:54:18.166 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:54:18.166 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.166 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4621.0:9089 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 23339520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:54:18.166 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:54:18.166 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 105.674613835, 2024-05-08T08:54:18.166 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.13814907200000001, 2024-05-08T08:54:18.166 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:18.166 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:18.167 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:54:18.167 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:18.167 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4621", 2024-05-08T08:54:18.167 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:18.167 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9089 2024-05-08T08:54:18.167 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.167 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:18.167 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.167 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:18.167 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:54:18.167 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:18.167 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.167 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.167 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:18.168 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:54:18.168 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:18.168 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.168 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.168 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:18.168 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493005+0000", 2024-05-08T08:54:18.168 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1700000000000001e-06 2024-05-08T08:54:18.168 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.168 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.168 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:18.168 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.513963+0000", 2024-05-08T08:54:18.168 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.020957581999999999 2024-05-08T08:54:18.168 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.168 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.168 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:18.168 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.513995+0000", 2024-05-08T08:54:18.169 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1993999999999999e-05 2024-05-08T08:54:18.169 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.169 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.169 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:18.169 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514007+0000", 2024-05-08T08:54:18.169 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1581e-05 2024-05-08T08:54:18.169 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.169 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.169 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:18.169 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514122+0000", 2024-05-08T08:54:18.169 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000115282 2024-05-08T08:54:18.169 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.169 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.169 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op must wait for map", 2024-05-08T08:54:18.170 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514128+0000", 2024-05-08T08:54:18.170 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9950000000000002e-06 2024-05-08T08:54:18.170 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.170 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.170 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:18.170 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.515800+0000", 2024-05-08T08:54:18.170 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001671886 2024-05-08T08:54:18.170 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.170 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.170 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:18.170 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.515893+0000", 2024-05-08T08:54:18.170 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3490000000000001e-05 2024-05-08T08:54:18.170 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.170 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.170 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 3", 2024-05-08T08:54:18.171 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.517647+0000", 2024-05-08T08:54:18.171 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0017535630000000001 2024-05-08T08:54:18.171 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.171 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.171 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:54:18.171 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.627578+0000", 2024-05-08T08:54:18.171 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.109931557 2024-05-08T08:54:18.171 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.171 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.171 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:54:18.171 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631055+0000", 2024-05-08T08:54:18.171 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0034761420000000002 2024-05-08T08:54:18.171 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.171 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.171 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:54:18.172 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631096+0000", 2024-05-08T08:54:18.172 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1059000000000001e-05 2024-05-08T08:54:18.172 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.172 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.172 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:18.172 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631151+0000", 2024-05-08T08:54:18.172 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5770999999999998e-05 2024-05-08T08:54:18.172 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.172 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:18.172 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.172 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.172 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.172 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1071 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11026432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:18.172 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:54:18.172 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 101.682164344, 2024-05-08T08:54:18.172 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070580037999999998, 2024-05-08T08:54:18.172 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:18.173 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:18.173 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:18.173 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:18.173 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:18.173 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:18.173 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1071 2024-05-08T08:54:18.173 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.173 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:18.173 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.173 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:18.173 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:54:18.173 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:18.173 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.173 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.174 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:18.174 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:54:18.174 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:18.174 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.174 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.174 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:18.174 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:54:18.174 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2800000000000002e-07 2024-05-08T08:54:18.174 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.174 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.174 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:18.174 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485454+0000", 2024-05-08T08:54:18.174 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2290000000000002e-06 2024-05-08T08:54:18.174 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.175 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.175 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:18.175 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485461+0000", 2024-05-08T08:54:18.175 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9920000000000001e-06 2024-05-08T08:54:18.175 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.175 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.175 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:18.175 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485467+0000", 2024-05-08T08:54:18.175 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3979999999999998e-06 2024-05-08T08:54:18.175 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.175 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.175 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:18.175 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555880+0000", 2024-05-08T08:54:18.175 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070413165 2024-05-08T08:54:18.175 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.175 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.176 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:18.176 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555926+0000", 2024-05-08T08:54:18.176 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6217000000000002e-05 2024-05-08T08:54:18.176 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.176 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.176 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:18.176 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556032+0000", 2024-05-08T08:54:18.176 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000105609 2024-05-08T08:54:18.176 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.176 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:18.176 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.176 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.176 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.176 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1073 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11067392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:18.176 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:54:18.176 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 101.682143123, 2024-05-08T08:54:18.176 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070821404000000004, 2024-05-08T08:54:18.177 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:18.177 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:18.177 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:18.177 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:18.177 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:18.177 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:18.177 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1073 2024-05-08T08:54:18.177 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.177 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:18.177 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.177 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:18.177 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:54:18.177 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:18.177 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.177 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.177 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:18.178 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:54:18.178 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:18.178 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.178 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.178 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:18.178 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:54:18.178 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.08e-07 2024-05-08T08:54:18.178 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.178 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.178 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:18.178 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485476+0000", 2024-05-08T08:54:18.178 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3929999999999998e-06 2024-05-08T08:54:18.178 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.178 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.179 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:18.179 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485485+0000", 2024-05-08T08:54:18.179 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1880000000000008e-06 2024-05-08T08:54:18.179 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.179 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.179 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:18.179 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485494+0000", 2024-05-08T08:54:18.179 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3109999999999995e-06 2024-05-08T08:54:18.179 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.179 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.179 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:18.179 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556082+0000", 2024-05-08T08:54:18.179 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070587426999999994 2024-05-08T08:54:18.179 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.179 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.180 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:18.180 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556177+0000", 2024-05-08T08:54:18.180 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5051999999999997e-05 2024-05-08T08:54:18.180 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.180 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.180 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:18.180 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556294+0000", 2024-05-08T08:54:18.180 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000117625 2024-05-08T08:54:18.180 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.180 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:18.180 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.180 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.180 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.180 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1075 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11108352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:18.180 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:54:18.180 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 101.682112546, 2024-05-08T08:54:18.180 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071045026999999997, 2024-05-08T08:54:18.181 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:18.181 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:18.181 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:18.181 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:18.181 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:18.181 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:18.181 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1075 2024-05-08T08:54:18.181 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.181 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:18.181 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.181 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:18.181 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:54:18.181 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:18.181 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.181 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.181 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:18.182 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:54:18.182 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:18.182 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.182 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.182 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:18.182 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:54:18.182 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.32e-07 2024-05-08T08:54:18.182 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.182 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.182 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:18.182 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485518+0000", 2024-05-08T08:54:18.182 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3769e-05 2024-05-08T08:54:18.182 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.182 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.182 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:18.183 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485527+0000", 2024-05-08T08:54:18.183 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6779999999999995e-06 2024-05-08T08:54:18.183 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.183 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.183 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:18.183 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485532+0000", 2024-05-08T08:54:18.183 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9830000000000004e-06 2024-05-08T08:54:18.183 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.183 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.183 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:18.183 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556340+0000", 2024-05-08T08:54:18.183 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070807743000000006 2024-05-08T08:54:18.183 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.183 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.183 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:18.183 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556435+0000", 2024-05-08T08:54:18.183 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4732e-05 2024-05-08T08:54:18.184 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.184 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.184 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:18.184 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556549+0000", 2024-05-08T08:54:18.184 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011369 2024-05-08T08:54:18.184 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.184 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:18.184 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.184 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.184 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.184 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1078 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11149312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:18.184 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:54:18.184 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 101.682072347, 2024-05-08T08:54:18.184 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071167276000000002, 2024-05-08T08:54:18.184 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:18.185 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:18.185 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:18.185 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:18.185 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:18.185 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:18.185 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1078 2024-05-08T08:54:18.185 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.185 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:18.185 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.185 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:18.185 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:54:18.185 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:18.185 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.185 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.185 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:18.185 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:54:18.186 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:18.186 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.186 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.186 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:18.186 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:54:18.186 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5600000000000001e-07 2024-05-08T08:54:18.186 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.186 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.186 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:18.186 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485547+0000", 2024-05-08T08:54:18.186 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3590000000000002e-06 2024-05-08T08:54:18.186 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.186 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.186 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:18.186 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485553+0000", 2024-05-08T08:54:18.186 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2770000000000002e-06 2024-05-08T08:54:18.187 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.187 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.187 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:18.187 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485558+0000", 2024-05-08T08:54:18.187 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.95e-06 2024-05-08T08:54:18.187 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.187 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.187 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:18.187 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556582+0000", 2024-05-08T08:54:18.187 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071024444000000006 2024-05-08T08:54:18.187 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.187 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.187 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:18.187 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556637+0000", 2024-05-08T08:54:18.187 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4672000000000001e-05 2024-05-08T08:54:18.187 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.188 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.188 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:18.188 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556711+0000", 2024-05-08T08:54:18.188 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4117999999999997e-05 2024-05-08T08:54:18.188 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.188 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:18.188 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.188 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.188 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.188 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1080 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11190272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:18.188 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:54:18.188 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 101.682051614, 2024-05-08T08:54:18.188 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071350721000000006, 2024-05-08T08:54:18.188 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:18.188 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:18.188 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:18.188 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:18.189 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:18.189 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:18.189 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1080 2024-05-08T08:54:18.189 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.189 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:18.189 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.189 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:18.189 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:54:18.189 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:18.189 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.189 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.189 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:18.189 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:54:18.189 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:18.189 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.189 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.189 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:18.190 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:54:18.190 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.03e-07 2024-05-08T08:54:18.190 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.190 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.190 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:18.190 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485567+0000", 2024-05-08T08:54:18.190 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1090000000000001e-06 2024-05-08T08:54:18.190 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.190 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.190 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:18.190 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485573+0000", 2024-05-08T08:54:18.190 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3049999999999998e-06 2024-05-08T08:54:18.190 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.190 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.190 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:18.191 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485578+0000", 2024-05-08T08:54:18.191 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0130000000000001e-06 2024-05-08T08:54:18.191 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.191 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.191 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:18.191 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556743+0000", 2024-05-08T08:54:18.191 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071164449000000005 2024-05-08T08:54:18.191 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.191 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.191 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:18.191 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556797+0000", 2024-05-08T08:54:18.191 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4589999999999997e-05 2024-05-08T08:54:18.191 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.191 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.191 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:18.191 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556915+0000", 2024-05-08T08:54:18.192 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000117852 2024-05-08T08:54:18.192 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.192 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:18.192 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.192 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.192 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.192 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1082 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11231232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:18.192 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:54:18.192 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 101.68203137099999, 2024-05-08T08:54:18.192 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071591396000000002, 2024-05-08T08:54:18.192 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:18.192 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:18.192 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:18.192 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:18.192 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:18.193 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:18.193 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1082 2024-05-08T08:54:18.193 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.193 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:18.193 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.193 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:18.193 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:54:18.193 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:18.193 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.193 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.193 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:18.193 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:54:18.193 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:18.193 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.193 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.193 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:18.194 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:54:18.194 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1800000000000001e-07 2024-05-08T08:54:18.194 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.194 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.194 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:18.194 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485587+0000", 2024-05-08T08:54:18.194 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2589999999999999e-06 2024-05-08T08:54:18.194 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.194 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.194 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:18.194 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485594+0000", 2024-05-08T08:54:18.194 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0789999999999997e-06 2024-05-08T08:54:18.194 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.194 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.195 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:18.195 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485598+0000", 2024-05-08T08:54:18.195 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7480000000000001e-06 2024-05-08T08:54:18.195 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.195 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.195 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:18.195 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556971+0000", 2024-05-08T08:54:18.195 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071372774999999999 2024-05-08T08:54:18.195 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.195 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.195 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:18.195 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557069+0000", 2024-05-08T08:54:18.195 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.7732999999999994e-05 2024-05-08T08:54:18.195 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.196 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.196 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:18.196 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557176+0000", 2024-05-08T08:54:18.196 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000107384 2024-05-08T08:54:18.196 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.196 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:18.196 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.196 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.196 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.196 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1084 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11272192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:18.196 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:54:18.196 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 101.681992732, 2024-05-08T08:54:18.196 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071808977999999996, 2024-05-08T08:54:18.196 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:18.196 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:18.196 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:18.197 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:18.197 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:18.197 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:18.197 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1084 2024-05-08T08:54:18.197 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.197 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:18.197 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.197 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:18.197 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:54:18.197 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:18.197 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.197 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.197 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:18.197 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:54:18.197 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:18.197 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.198 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.198 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:18.198 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485624+0000", 2024-05-08T08:54:18.198 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:54:18.198 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.198 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.198 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:18.198 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485626+0000", 2024-05-08T08:54:18.198 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9460000000000001e-06 2024-05-08T08:54:18.198 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.198 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.198 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:18.198 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485633+0000", 2024-05-08T08:54:18.198 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1149999999999996e-06 2024-05-08T08:54:18.199 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.199 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.199 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:18.199 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485638+0000", 2024-05-08T08:54:18.199 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5670000000000002e-06 2024-05-08T08:54:18.199 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.199 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.199 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:18.199 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557226+0000", 2024-05-08T08:54:18.199 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071587267999999996 2024-05-08T08:54:18.199 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.199 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.199 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:18.199 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557321+0000", 2024-05-08T08:54:18.199 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5785000000000005e-05 2024-05-08T08:54:18.199 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.200 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.200 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:18.200 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557432+0000", 2024-05-08T08:54:18.200 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011092800000000001 2024-05-08T08:54:18.200 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.200 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:18.200 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.200 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.200 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.200 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1086 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11313152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:18.200 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:54:18.200 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 101.681969405, 2024-05-08T08:54:18.200 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071933132999999996, 2024-05-08T08:54:18.200 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:18.200 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:18.200 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:18.200 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:18.201 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:18.201 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:18.201 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1086 2024-05-08T08:54:18.201 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.201 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:18.201 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.201 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:18.201 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:54:18.201 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:18.201 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.201 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.202 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:18.202 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:54:18.202 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:18.202 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.202 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.202 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:18.202 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486328+0000", 2024-05-08T08:54:18.202 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00068107900000000002 2024-05-08T08:54:18.202 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.202 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.202 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:18.202 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486331+0000", 2024-05-08T08:54:18.202 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.585e-06 2024-05-08T08:54:18.202 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.203 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.203 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:18.203 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486343+0000", 2024-05-08T08:54:18.203 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1856e-05 2024-05-08T08:54:18.203 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.203 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.203 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:18.203 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486350+0000", 2024-05-08T08:54:18.203 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8600000000000004e-06 2024-05-08T08:54:18.203 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.203 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.203 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:18.203 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557470+0000", 2024-05-08T08:54:18.203 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071120326999999997 2024-05-08T08:54:18.203 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.203 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.204 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:18.204 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557514+0000", 2024-05-08T08:54:18.204 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3909000000000003e-05 2024-05-08T08:54:18.204 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.204 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.204 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:18.204 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557580+0000", 2024-05-08T08:54:18.204 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5517000000000003e-05 2024-05-08T08:54:18.204 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.204 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:18.204 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.204 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.204 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.205 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1089 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11354112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:18.205 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:54:18.205 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 101.681256533, 2024-05-08T08:54:18.205 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071457381, 2024-05-08T08:54:18.205 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:18.205 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:18.205 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:18.205 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:18.205 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:18.205 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:18.205 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1089 2024-05-08T08:54:18.205 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.205 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:18.205 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.206 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:18.206 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:54:18.206 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:18.206 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.206 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.206 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:18.206 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:54:18.206 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:18.206 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.206 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.206 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:18.206 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486851+0000", 2024-05-08T08:54:18.206 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00049102 2024-05-08T08:54:18.206 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.206 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.207 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:18.207 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486854+0000", 2024-05-08T08:54:18.207 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6859999999999999e-06 2024-05-08T08:54:18.207 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.207 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.207 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:18.207 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486864+0000", 2024-05-08T08:54:18.207 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6080000000000002e-06 2024-05-08T08:54:18.207 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.207 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.207 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:18.207 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486870+0000", 2024-05-08T08:54:18.207 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.477e-06 2024-05-08T08:54:18.207 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.207 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.208 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:18.208 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557612+0000", 2024-05-08T08:54:18.208 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070741557999999996 2024-05-08T08:54:18.208 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.208 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.208 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:18.208 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557695+0000", 2024-05-08T08:54:18.208 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3300000000000005e-05 2024-05-08T08:54:18.208 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.208 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.208 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:18.208 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557817+0000", 2024-05-08T08:54:18.208 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012173200000000001 2024-05-08T08:54:18.208 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.208 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:18.209 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.209 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.209 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.209 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1091 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11395072~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:18.209 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:54:18.209 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 101.680735738, 2024-05-08T08:54:18.209 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071184737999999997, 2024-05-08T08:54:18.209 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:18.209 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:18.209 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:18.209 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:18.209 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:18.209 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:18.209 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1091 2024-05-08T08:54:18.209 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.209 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:18.210 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.210 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:18.210 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:54:18.210 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:18.210 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.210 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.210 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:18.210 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:54:18.210 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:18.210 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.210 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.210 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:18.210 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486881+0000", 2024-05-08T08:54:18.210 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0699999999999997e-07 2024-05-08T08:54:18.211 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.211 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.211 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:18.211 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486883+0000", 2024-05-08T08:54:18.211 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.402e-06 2024-05-08T08:54:18.211 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.211 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.211 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:18.211 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486890+0000", 2024-05-08T08:54:18.211 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0480000000000001e-06 2024-05-08T08:54:18.211 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.211 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.211 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:18.211 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486896+0000", 2024-05-08T08:54:18.211 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7640000000000002e-06 2024-05-08T08:54:18.211 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.211 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.212 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:18.212 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557860+0000", 2024-05-08T08:54:18.212 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070963791999999998 2024-05-08T08:54:18.212 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.212 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.212 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:18.212 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557952+0000", 2024-05-08T08:54:18.212 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2082000000000004e-05 2024-05-08T08:54:18.212 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.212 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.212 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:18.212 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558065+0000", 2024-05-08T08:54:18.212 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000113143 2024-05-08T08:54:18.212 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.212 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:18.213 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.213 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.213 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.213 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1094 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11436032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:18.213 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:54:18.213 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 101.68071257, 2024-05-08T08:54:18.213 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071400507000000002, 2024-05-08T08:54:18.213 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:18.213 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:18.213 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:18.213 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:18.213 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:18.213 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:18.213 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1094 2024-05-08T08:54:18.213 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.213 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:18.213 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.214 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:18.214 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:54:18.214 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:18.214 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.214 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.214 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:18.214 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:54:18.214 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:18.214 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.214 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.214 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:18.214 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487940+0000", 2024-05-08T08:54:18.214 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001036868 2024-05-08T08:54:18.214 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.215 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.215 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:18.215 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487944+0000", 2024-05-08T08:54:18.215 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7069999999999998e-06 2024-05-08T08:54:18.215 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.215 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.215 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:18.215 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487951+0000", 2024-05-08T08:54:18.215 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2649999999999999e-06 2024-05-08T08:54:18.215 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.215 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.215 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:18.215 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487961+0000", 2024-05-08T08:54:18.215 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4110000000000002e-06 2024-05-08T08:54:18.215 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.216 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.216 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:18.216 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558117+0000", 2024-05-08T08:54:18.216 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070156662999999994 2024-05-08T08:54:18.216 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.216 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.216 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:18.216 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558214+0000", 2024-05-08T08:54:18.216 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6444e-05 2024-05-08T08:54:18.216 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.216 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.216 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:18.216 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558304+0000", 2024-05-08T08:54:18.216 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0148999999999994e-05 2024-05-08T08:54:18.216 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.216 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:18.217 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.217 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.217 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.217 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1096 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11476992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:18.217 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:54:18.217 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 101.679646039, 2024-05-08T08:54:18.217 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070480074000000004, 2024-05-08T08:54:18.217 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:18.217 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:18.217 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:18.217 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:18.217 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:18.217 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:18.217 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1096 2024-05-08T08:54:18.217 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.217 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:18.218 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.218 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:18.218 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:54:18.218 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:18.218 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.218 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.218 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:18.218 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:54:18.218 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:18.218 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.218 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.218 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:18.218 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:54:18.218 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3700000000000001e-07 2024-05-08T08:54:18.219 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.219 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.219 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:18.219 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487973+0000", 2024-05-08T08:54:18.219 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0870000000000001e-06 2024-05-08T08:54:18.219 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.219 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.219 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:18.219 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487977+0000", 2024-05-08T08:54:18.219 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0600000000000001e-06 2024-05-08T08:54:18.219 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.219 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.219 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:18.219 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487980+0000", 2024-05-08T08:54:18.219 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1089999999999998e-06 2024-05-08T08:54:18.219 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.219 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.220 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:18.220 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558338+0000", 2024-05-08T08:54:18.220 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070358233000000006 2024-05-08T08:54:18.220 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.220 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.220 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:18.220 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558392+0000", 2024-05-08T08:54:18.220 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4537999999999999e-05 2024-05-08T08:54:18.220 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.220 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.220 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:18.220 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558450+0000", 2024-05-08T08:54:18.220 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7710000000000001e-05 2024-05-08T08:54:18.220 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.220 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:18.220 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.221 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.221 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.221 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1099 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11517952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:18.221 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:54:18.221 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 101.67963216299999, 2024-05-08T08:54:18.221 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070720506000000002, 2024-05-08T08:54:18.221 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:18.221 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:18.221 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:18.221 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:18.221 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:18.221 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:18.221 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1099 2024-05-08T08:54:18.221 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.221 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:18.221 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.221 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:18.222 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:54:18.222 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:18.222 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.222 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.222 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:18.222 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:54:18.222 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:18.222 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.222 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.222 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:18.222 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:54:18.222 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3999999999999998e-07 2024-05-08T08:54:18.222 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.222 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.222 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:18.222 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487986+0000", 2024-05-08T08:54:18.223 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3519999999999999e-06 2024-05-08T08:54:18.223 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.223 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.223 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:18.223 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487989+0000", 2024-05-08T08:54:18.223 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.23e-06 2024-05-08T08:54:18.223 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.223 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.223 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:18.223 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487991+0000", 2024-05-08T08:54:18.223 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6350000000000002e-06 2024-05-08T08:54:18.223 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.223 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.223 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:18.223 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558489+0000", 2024-05-08T08:54:18.224 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070497924000000003 2024-05-08T08:54:18.224 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.224 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.224 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:18.224 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558583+0000", 2024-05-08T08:54:18.224 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3405000000000004e-05 2024-05-08T08:54:18.224 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.224 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.224 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:18.224 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558704+0000", 2024-05-08T08:54:18.224 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012171999999999999 2024-05-08T08:54:18.224 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.224 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:18.224 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.224 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.225 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.225 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1100 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11558912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:18.225 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:54:18.225 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 101.679619164, 2024-05-08T08:54:18.225 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070962266999999996, 2024-05-08T08:54:18.225 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:18.225 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:18.225 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:18.225 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:18.225 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:18.225 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:18.225 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1100 2024-05-08T08:54:18.225 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.225 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:18.225 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.225 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:18.226 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:54:18.226 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:18.226 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.226 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.226 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:18.226 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:54:18.226 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:18.226 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.226 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.226 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:18.226 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488430+0000", 2024-05-08T08:54:18.226 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000432676 2024-05-08T08:54:18.226 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.226 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.226 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:18.227 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488433+0000", 2024-05-08T08:54:18.227 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.45e-06 2024-05-08T08:54:18.227 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.227 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.227 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:18.227 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488442+0000", 2024-05-08T08:54:18.227 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.687e-06 2024-05-08T08:54:18.227 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.227 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.227 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:18.227 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488446+0000", 2024-05-08T08:54:18.227 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4939999999999997e-06 2024-05-08T08:54:18.227 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.227 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.227 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:18.227 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558756+0000", 2024-05-08T08:54:18.227 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070310051999999998 2024-05-08T08:54:18.228 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.228 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.228 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:18.228 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558844+0000", 2024-05-08T08:54:18.228 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7690999999999998e-05 2024-05-08T08:54:18.228 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.228 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.228 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:18.228 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558959+0000", 2024-05-08T08:54:18.228 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000115217 2024-05-08T08:54:18.228 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.228 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:18.228 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.228 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.228 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.228 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1103 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11599872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:18.228 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:54:18.229 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 101.67916148899999, 2024-05-08T08:54:18.229 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070777100999999995, 2024-05-08T08:54:18.229 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:18.229 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:18.229 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:18.229 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:18.229 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:18.229 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:18.229 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1103 2024-05-08T08:54:18.229 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.229 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:18.229 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.229 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:18.229 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:54:18.229 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:18.229 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.229 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.230 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:18.230 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:54:18.230 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:18.230 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.230 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.230 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:18.230 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488959+0000", 2024-05-08T08:54:18.230 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00050457200000000001 2024-05-08T08:54:18.230 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.230 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.230 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:18.230 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488969+0000", 2024-05-08T08:54:18.230 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.7159999999999998e-06 2024-05-08T08:54:18.230 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.230 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.231 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:18.231 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488981+0000", 2024-05-08T08:54:18.231 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2150000000000001e-05 2024-05-08T08:54:18.231 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.231 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.231 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:18.231 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488988+0000", 2024-05-08T08:54:18.231 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8759999999999999e-06 2024-05-08T08:54:18.231 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.231 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.231 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:18.231 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558992+0000", 2024-05-08T08:54:18.231 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070004358000000003 2024-05-08T08:54:18.231 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.231 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.231 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:18.232 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559093+0000", 2024-05-08T08:54:18.232 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000100299 2024-05-08T08:54:18.232 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.232 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.232 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:18.232 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559232+0000", 2024-05-08T08:54:18.232 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013913 2024-05-08T08:54:18.232 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.232 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:18.232 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.232 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.232 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.232 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1104 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11640832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:18.232 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:54:18.232 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 101.678616674, 2024-05-08T08:54:18.232 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070521947000000001, 2024-05-08T08:54:18.233 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:18.233 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:18.233 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:18.233 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:18.233 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:18.233 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:18.233 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1104 2024-05-08T08:54:18.233 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.233 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:18.233 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.233 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:18.233 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:54:18.233 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:18.233 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.233 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.233 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:18.234 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:54:18.234 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:18.234 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.234 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.234 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:18.234 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489700+0000", 2024-05-08T08:54:18.234 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00070067600000000001 2024-05-08T08:54:18.234 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.234 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.234 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:18.234 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489704+0000", 2024-05-08T08:54:18.234 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1640000000000003e-06 2024-05-08T08:54:18.234 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.235 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.235 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:18.235 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489716+0000", 2024-05-08T08:54:18.235 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1379000000000001e-05 2024-05-08T08:54:18.235 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.235 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.235 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:18.235 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489724+0000", 2024-05-08T08:54:18.235 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8949999999999998e-06 2024-05-08T08:54:18.235 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.235 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.235 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:18.235 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559278+0000", 2024-05-08T08:54:18.235 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.069554151999999994 2024-05-08T08:54:18.235 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.235 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.236 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:18.236 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559387+0000", 2024-05-08T08:54:18.236 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000109258 2024-05-08T08:54:18.236 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.236 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.236 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:18.236 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559521+0000", 2024-05-08T08:54:18.236 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000134423 2024-05-08T08:54:18.236 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.236 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:18.236 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.236 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.236 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.236 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:8636 3.17 3:e8a0a3e4:::rbd_header.11402b26385:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-05-08T08:54:18.236 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:54.150971+0000", 2024-05-08T08:54:18.236 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 84.016644679999999, 2024-05-08T08:54:18.237 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.217273233, 2024-05-08T08:54:18.237 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:18.237 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:18.237 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:18.237 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:18.237 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:54:18.237 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:54:18.237 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 8636 2024-05-08T08:54:18.237 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.237 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:18.237 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.237 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:18.237 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150971+0000", 2024-05-08T08:54:18.237 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:18.237 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.237 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.238 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:18.238 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150971+0000", 2024-05-08T08:54:18.238 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:18.238 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.238 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.238 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:18.238 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150972+0000", 2024-05-08T08:54:18.238 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.71e-07 2024-05-08T08:54:18.238 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.238 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.238 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:18.238 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150977+0000", 2024-05-08T08:54:18.238 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.152e-06 2024-05-08T08:54:18.238 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.238 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.239 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:18.239 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150990+0000", 2024-05-08T08:54:18.239 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2364999999999999e-05 2024-05-08T08:54:18.239 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.239 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.239 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:18.239 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150998+0000", 2024-05-08T08:54:18.239 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0739999999999993e-06 2024-05-08T08:54:18.239 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.239 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.239 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:18.239 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.151054+0000", 2024-05-08T08:54:18.239 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5949000000000003e-05 2024-05-08T08:54:18.239 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.239 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.239 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:54:18.239 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.151060+0000", 2024-05-08T08:54:18.240 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6120000000000004e-06 2024-05-08T08:54:18.240 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.240 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.240 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:18.240 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.367640+0000", 2024-05-08T08:54:18.240 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2165792739999999 2024-05-08T08:54:18.240 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.240 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.240 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:18.240 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.367839+0000", 2024-05-08T08:54:18.240 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000199185 2024-05-08T08:54:18.240 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.240 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.240 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:18.240 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.368245+0000", 2024-05-08T08:54:18.240 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00040575099999999998 2024-05-08T08:54:18.240 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.241 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:18.241 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.241 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.241 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.241 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5302.0:1516 3.c 3:319bf789:::rbd_data.1140d3fc54c.0000000000000025:head [stat,write 1892352~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:54:18.241 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:27.822220+0000", 2024-05-08T08:54:18.241 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 50.345395742999997, 2024-05-08T08:54:18.241 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.092643744, 2024-05-08T08:54:18.241 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:18.241 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:18.241 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:54:18.241 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:18.241 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5302", 2024-05-08T08:54:18.241 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/359277859", 2024-05-08T08:54:18.241 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1516 2024-05-08T08:54:18.241 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.242 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:18.242 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.242 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:18.242 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822220+0000", 2024-05-08T08:54:18.242 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:18.242 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.242 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.242 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:18.242 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822220+0000", 2024-05-08T08:54:18.242 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:18.242 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.242 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.242 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:18.242 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822221+0000", 2024-05-08T08:54:18.242 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3500000000000002e-07 2024-05-08T08:54:18.243 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.243 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.243 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:18.243 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822228+0000", 2024-05-08T08:54:18.243 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9959999999999996e-06 2024-05-08T08:54:18.243 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.243 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.243 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:18.243 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822234+0000", 2024-05-08T08:54:18.243 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5429999999999999e-06 2024-05-08T08:54:18.243 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.243 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.243 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:18.243 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822238+0000", 2024-05-08T08:54:18.243 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.732e-06 2024-05-08T08:54:18.244 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.244 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.244 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:18.244 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822285+0000", 2024-05-08T08:54:18.244 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6810000000000001e-05 2024-05-08T08:54:18.244 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.244 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.244 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:18.244 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822358+0000", 2024-05-08T08:54:18.244 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3473000000000006e-05 2024-05-08T08:54:18.244 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.244 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.244 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 5", 2024-05-08T08:54:18.244 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822489+0000", 2024-05-08T08:54:18.244 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013011199999999999 2024-05-08T08:54:18.244 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.245 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.245 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:54:18.245 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.825332+0000", 2024-05-08T08:54:18.245 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0028434799999999998 2024-05-08T08:54:18.245 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.245 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.245 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:54:18.245 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.914753+0000", 2024-05-08T08:54:18.245 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089421424999999999 2024-05-08T08:54:18.245 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.245 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.245 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:54:18.245 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.914807+0000", 2024-05-08T08:54:18.245 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3829000000000002e-05 2024-05-08T08:54:18.245 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.245 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.246 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:18.246 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.914864+0000", 2024-05-08T08:54:18.246 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6909e-05 2024-05-08T08:54:18.246 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.246 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:18.246 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.246 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.246 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.246 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:24567 3.6 3:66af1c40:::rbd_data.1140d3fc54c.000000000000005a:18 [read 2080768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:18.246 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.216598+0000", 2024-05-08T08:54:18.246 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 45.951017663000002, 2024-05-08T08:54:18.246 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071602647000000005, 2024-05-08T08:54:18.246 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:18.246 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:18.247 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:18.247 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:18.247 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:18.247 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:18.247 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 24567 2024-05-08T08:54:18.247 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.247 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:18.247 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.247 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:18.247 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216598+0000", 2024-05-08T08:54:18.247 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:18.247 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.247 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.247 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:18.247 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216598+0000", 2024-05-08T08:54:18.247 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:18.247 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.248 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.248 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:18.248 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216599+0000", 2024-05-08T08:54:18.248 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8999999999999997e-07 2024-05-08T08:54:18.248 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.248 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.248 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:18.248 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216601+0000", 2024-05-08T08:54:18.248 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9889999999999999e-06 2024-05-08T08:54:18.248 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.248 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.248 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:18.248 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216607+0000", 2024-05-08T08:54:18.248 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3600000000000001e-06 2024-05-08T08:54:18.248 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.248 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.248 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:18.249 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216612+0000", 2024-05-08T08:54:18.249 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.69e-06 2024-05-08T08:54:18.249 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.249 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.249 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:18.249 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.287076+0000", 2024-05-08T08:54:18.249 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070464355000000006 2024-05-08T08:54:18.249 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.249 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.249 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:18.249 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.287151+0000", 2024-05-08T08:54:18.249 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4924000000000003e-05 2024-05-08T08:54:18.249 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.249 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.249 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:18.249 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.288201+0000", 2024-05-08T08:54:18.249 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0010498389999999999 2024-05-08T08:54:18.250 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.250 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:18.250 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.250 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.250 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:18.250 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:18.251 INFO:teuthology.orchestra.run.smithi196.stdout:checking clone #41, image image_client.0-clone41 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent42 2024-05-08T08:54:18.251 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:18.254+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:18.252 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:54:18.352 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:18.354+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:18.375 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:18.375 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:54:18.375 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:54:18.375 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:18.385 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:54:18.453 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:18.456+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:18.510 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:18.511 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:54:18.511 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:54:18.511 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:54:18.511 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:18.520 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:54:18.554 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:18.557+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:18.655 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:18.658+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:18.692 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:18.692 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:54:18.692 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:54:18.692 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:54:18.692 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.692 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:12443 3.0 3:02d0158e:::rbd_data.114071bcf828.0000000000000037:head [stat,delete] snapc 19=[19] RETRY=3 ondisk+retry+write+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:18.693 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.610027+0000", 2024-05-08T08:54:18.693 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 47.084130256999998, 2024-05-08T08:54:18.693 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.083586333999999998, 2024-05-08T08:54:18.693 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:18.693 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:18.693 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:18.693 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:18.693 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:18.693 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:18.693 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 12443 2024-05-08T08:54:18.693 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.693 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:18.693 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.693 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:18.694 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610027+0000", 2024-05-08T08:54:18.694 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:18.694 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.694 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.694 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:18.694 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610027+0000", 2024-05-08T08:54:18.694 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:18.694 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.694 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.694 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:18.694 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610028+0000", 2024-05-08T08:54:18.694 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.13e-07 2024-05-08T08:54:18.694 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.695 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.695 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:18.695 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610030+0000", 2024-05-08T08:54:18.695 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7669999999999999e-06 2024-05-08T08:54:18.695 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.695 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.695 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:18.695 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610039+0000", 2024-05-08T08:54:18.695 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4959999999999996e-06 2024-05-08T08:54:18.695 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.695 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.695 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:18.695 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610045+0000", 2024-05-08T08:54:18.695 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6779999999999997e-06 2024-05-08T08:54:18.696 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.696 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.696 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:18.696 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.660873+0000", 2024-05-08T08:54:18.696 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.050828748 2024-05-08T08:54:18.696 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.696 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.696 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:18.696 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.661017+0000", 2024-05-08T08:54:18.696 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014391999999999999 2024-05-08T08:54:18.696 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.696 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.696 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:18.696 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.693613+0000", 2024-05-08T08:54:18.697 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.032596111999999997 2024-05-08T08:54:18.697 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.697 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:18.697 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.697 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.697 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.697 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:14301 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [watch reconnect cookie 139837155901536 gen 12] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:18.697 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.645078+0000", 2024-05-08T08:54:18.697 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 47.049079618999997, 2024-05-08T08:54:18.697 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088281852999999993, 2024-05-08T08:54:18.697 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:18.697 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:18.697 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:18.697 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:18.697 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:18.698 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:18.698 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14301 2024-05-08T08:54:18.698 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.698 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:18.698 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.698 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:18.698 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645078+0000", 2024-05-08T08:54:18.698 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:18.698 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.698 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.698 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:18.698 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645078+0000", 2024-05-08T08:54:18.698 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:18.698 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.699 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.699 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:18.699 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645084+0000", 2024-05-08T08:54:18.699 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4620000000000001e-06 2024-05-08T08:54:18.699 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.699 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.699 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:18.699 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645088+0000", 2024-05-08T08:54:18.699 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8039999999999999e-06 2024-05-08T08:54:18.699 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.699 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.699 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:18.699 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645106+0000", 2024-05-08T08:54:18.699 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.8318999999999999e-05 2024-05-08T08:54:18.699 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.700 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.700 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:18.700 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645113+0000", 2024-05-08T08:54:18.700 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0310000000000001e-06 2024-05-08T08:54:18.700 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.700 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.700 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:18.700 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.670493+0000", 2024-05-08T08:54:18.700 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.025379993 2024-05-08T08:54:18.700 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.700 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.700 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:18.700 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.670571+0000", 2024-05-08T08:54:18.700 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7613000000000004e-05 2024-05-08T08:54:18.700 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.701 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.701 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:18.701 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733360+0000", 2024-05-08T08:54:18.701 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.062788630999999998 2024-05-08T08:54:18.701 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.701 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:18.701 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.701 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.701 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.701 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:14420 3.0 3:06a8948e:::rbd_data.114071bcf828.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:18.701 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.649639+0000", 2024-05-08T08:54:18.701 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 47.044518471000004, 2024-05-08T08:54:18.701 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.083851897999999994, 2024-05-08T08:54:18.702 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:18.702 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:18.702 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:54:18.702 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:18.702 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:18.702 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:18.702 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14420 2024-05-08T08:54:18.702 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.702 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:18.702 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.702 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:18.702 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649639+0000", 2024-05-08T08:54:18.702 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:18.702 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.703 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.703 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:18.703 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649639+0000", 2024-05-08T08:54:18.703 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:18.703 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.703 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.703 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:18.703 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649640+0000", 2024-05-08T08:54:18.703 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6899999999999997e-07 2024-05-08T08:54:18.704 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.704 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.704 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:18.704 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649895+0000", 2024-05-08T08:54:18.704 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00025531000000000002 2024-05-08T08:54:18.704 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.704 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.704 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:18.704 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649905+0000", 2024-05-08T08:54:18.704 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0198e-05 2024-05-08T08:54:18.704 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.704 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.704 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:18.704 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649911+0000", 2024-05-08T08:54:18.705 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2310000000000001e-06 2024-05-08T08:54:18.705 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.705 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.705 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:18.705 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.684099+0000", 2024-05-08T08:54:18.705 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.034188017000000001 2024-05-08T08:54:18.705 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.705 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.705 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:18.705 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.684172+0000", 2024-05-08T08:54:18.705 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3201999999999995e-05 2024-05-08T08:54:18.705 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.705 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.705 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 2", 2024-05-08T08:54:18.706 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.684418+0000", 2024-05-08T08:54:18.706 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00024559099999999997 2024-05-08T08:54:18.706 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.706 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.706 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:54:18.706 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.705538+0000", 2024-05-08T08:54:18.706 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.021119707000000001 2024-05-08T08:54:18.706 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.706 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.706 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:54:18.706 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733399+0000", 2024-05-08T08:54:18.706 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.027861543999999999 2024-05-08T08:54:18.706 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.707 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.707 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:54:18.707 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733445+0000", 2024-05-08T08:54:18.707 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5587000000000002e-05 2024-05-08T08:54:18.707 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.707 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.707 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:18.707 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733491+0000", 2024-05-08T08:54:18.707 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5841999999999999e-05 2024-05-08T08:54:18.707 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.707 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:18.707 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.707 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.707 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.707 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28547 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889179216] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:18.708 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.355654+0000", 2024-05-08T08:54:18.708 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 46.338503242999998, 2024-05-08T08:54:18.708 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078692261999999999, 2024-05-08T08:54:18.708 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:18.708 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:18.708 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:18.708 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:18.708 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:18.708 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:18.708 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28547 2024-05-08T08:54:18.708 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.708 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:18.708 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.708 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:18.708 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355654+0000", 2024-05-08T08:54:18.708 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:18.708 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.709 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.709 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:18.709 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355654+0000", 2024-05-08T08:54:18.709 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:18.709 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.709 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.709 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:18.709 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355654+0000", 2024-05-08T08:54:18.709 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3999999999999998e-07 2024-05-08T08:54:18.709 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.709 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.709 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:18.709 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355656+0000", 2024-05-08T08:54:18.709 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.353e-06 2024-05-08T08:54:18.709 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.709 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.710 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:18.710 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355663+0000", 2024-05-08T08:54:18.710 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2259999999999999e-06 2024-05-08T08:54:18.710 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.710 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.710 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:18.710 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355666+0000", 2024-05-08T08:54:18.710 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7930000000000002e-06 2024-05-08T08:54:18.710 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.710 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.710 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:18.710 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434117+0000", 2024-05-08T08:54:18.710 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078451231999999996 2024-05-08T08:54:18.710 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.710 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.710 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:18.711 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434198+0000", 2024-05-08T08:54:18.711 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0879999999999998e-05 2024-05-08T08:54:18.711 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.711 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.711 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:18.711 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434346+0000", 2024-05-08T08:54:18.711 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014853800000000001 2024-05-08T08:54:18.711 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.711 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:18.711 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.711 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.711 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.711 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28548 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889189696] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:18.711 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.355669+0000", 2024-05-08T08:54:18.711 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 46.338487893, 2024-05-08T08:54:18.711 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078956606999999998, 2024-05-08T08:54:18.712 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:18.712 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:18.712 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:18.712 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:18.712 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:18.712 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:18.712 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28548 2024-05-08T08:54:18.712 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.712 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:18.712 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.712 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:18.712 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355669+0000", 2024-05-08T08:54:18.712 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:18.712 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.712 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.712 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:18.712 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355669+0000", 2024-05-08T08:54:18.713 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:18.713 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.713 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.713 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:18.713 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355762+0000", 2024-05-08T08:54:18.713 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3015000000000003e-05 2024-05-08T08:54:18.713 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.713 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.713 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:18.713 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355765+0000", 2024-05-08T08:54:18.713 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.368e-06 2024-05-08T08:54:18.713 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.713 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.713 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:18.713 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355771+0000", 2024-05-08T08:54:18.713 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5989999999999998e-06 2024-05-08T08:54:18.714 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.714 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.714 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:18.714 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355775+0000", 2024-05-08T08:54:18.714 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7280000000000001e-06 2024-05-08T08:54:18.714 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.714 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.714 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:18.714 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434379+0000", 2024-05-08T08:54:18.714 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078603662000000005 2024-05-08T08:54:18.714 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.714 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.714 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:18.714 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434473+0000", 2024-05-08T08:54:18.714 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4104000000000006e-05 2024-05-08T08:54:18.715 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.715 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.715 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:18.715 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434626+0000", 2024-05-08T08:54:18.715 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00015313100000000001 2024-05-08T08:54:18.715 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.715 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:18.715 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.715 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.715 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.715 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28549 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889202400] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:18.715 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.355781+0000", 2024-05-08T08:54:18.715 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 46.338376318999998, 2024-05-08T08:54:18.715 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079117946999999994, 2024-05-08T08:54:18.715 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:18.716 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:18.716 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:18.716 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:18.716 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:18.716 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:18.716 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28549 2024-05-08T08:54:18.716 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.716 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:18.716 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.716 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:18.716 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355781+0000", 2024-05-08T08:54:18.716 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:18.716 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.716 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.717 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:18.717 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355781+0000", 2024-05-08T08:54:18.717 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:18.717 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.717 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.717 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:18.717 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356162+0000", 2024-05-08T08:54:18.717 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00038099100000000001 2024-05-08T08:54:18.717 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.717 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.717 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:18.717 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356165+0000", 2024-05-08T08:54:18.717 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1240000000000001e-06 2024-05-08T08:54:18.717 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.717 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.718 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:18.718 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356173+0000", 2024-05-08T08:54:18.718 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.6199999999999999e-06 2024-05-08T08:54:18.718 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.718 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.718 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:18.718 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356182+0000", 2024-05-08T08:54:18.718 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7760000000000003e-06 2024-05-08T08:54:18.718 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.718 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.718 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:18.718 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434675+0000", 2024-05-08T08:54:18.718 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078493119 2024-05-08T08:54:18.718 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.719 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.719 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:18.719 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434765+0000", 2024-05-08T08:54:18.719 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0402000000000006e-05 2024-05-08T08:54:18.719 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.719 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.719 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:18.719 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434899+0000", 2024-05-08T08:54:18.719 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013391499999999999 2024-05-08T08:54:18.719 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.719 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:18.719 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.719 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.719 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.719 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28550 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888597280] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:18.720 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:54:18.720 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 46.337965273000002, 2024-05-08T08:54:18.720 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078933990999999995, 2024-05-08T08:54:18.720 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:18.720 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:18.720 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:18.720 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:18.720 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:18.720 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:18.720 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28550 2024-05-08T08:54:18.720 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.720 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:18.720 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.720 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:18.720 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:54:18.721 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:18.721 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.721 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.721 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:18.721 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:54:18.721 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:18.721 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.721 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.721 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:18.721 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:54:18.721 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.34e-07 2024-05-08T08:54:18.721 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.721 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.722 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:18.722 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356195+0000", 2024-05-08T08:54:18.722 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8710000000000001e-06 2024-05-08T08:54:18.722 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.722 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.722 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:18.722 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356200+0000", 2024-05-08T08:54:18.722 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3000000000000003e-06 2024-05-08T08:54:18.722 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.722 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.722 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:18.722 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356203+0000", 2024-05-08T08:54:18.722 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4039999999999999e-06 2024-05-08T08:54:18.722 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.723 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.723 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:18.723 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434928+0000", 2024-05-08T08:54:18.723 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078725331999999995 2024-05-08T08:54:18.723 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.723 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.723 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:18.723 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435007+0000", 2024-05-08T08:54:18.723 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8415000000000001e-05 2024-05-08T08:54:18.723 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.723 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.723 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:18.723 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435126+0000", 2024-05-08T08:54:18.723 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011923500000000001 2024-05-08T08:54:18.723 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.724 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:18.724 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.724 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.724 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.724 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28551 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888527200] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:18.724 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.356210+0000", 2024-05-08T08:54:18.724 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 46.337946924999997, 2024-05-08T08:54:18.724 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079195072000000005, 2024-05-08T08:54:18.724 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:18.724 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:18.724 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:18.724 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:18.724 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:18.725 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:18.725 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28551 2024-05-08T08:54:18.725 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.725 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:18.725 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.725 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:18.725 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356210+0000", 2024-05-08T08:54:18.725 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:18.725 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.725 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.725 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:18.725 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356210+0000", 2024-05-08T08:54:18.725 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:18.725 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.725 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.725 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:18.726 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356211+0000", 2024-05-08T08:54:18.726 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4499999999999998e-07 2024-05-08T08:54:18.726 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.726 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.726 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:18.726 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356212+0000", 2024-05-08T08:54:18.726 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6029999999999999e-06 2024-05-08T08:54:18.726 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.726 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.726 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:18.726 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356221+0000", 2024-05-08T08:54:18.726 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4200000000000007e-06 2024-05-08T08:54:18.726 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.726 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.727 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:18.727 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356224+0000", 2024-05-08T08:54:18.727 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.038e-06 2024-05-08T08:54:18.727 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.727 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.727 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:18.727 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435178+0000", 2024-05-08T08:54:18.727 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078954625000000001 2024-05-08T08:54:18.727 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.727 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.727 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:18.727 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435256+0000", 2024-05-08T08:54:18.727 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7337999999999997e-05 2024-05-08T08:54:18.727 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.727 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.728 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:18.728 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435405+0000", 2024-05-08T08:54:18.728 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000149703 2024-05-08T08:54:18.728 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.728 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:18.728 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.728 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.728 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.728 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28552 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889231456] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:18.728 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.356228+0000", 2024-05-08T08:54:18.728 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 46.337929246999998, 2024-05-08T08:54:18.728 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079346920000000001, 2024-05-08T08:54:18.728 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:18.728 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:18.728 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:18.728 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:18.729 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:18.729 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:18.729 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28552 2024-05-08T08:54:18.729 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.729 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:18.729 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.729 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:18.729 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356228+0000", 2024-05-08T08:54:18.729 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:18.729 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.729 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.729 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:18.729 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356228+0000", 2024-05-08T08:54:18.729 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:18.729 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.729 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.730 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:18.730 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356959+0000", 2024-05-08T08:54:18.730 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00073097099999999996 2024-05-08T08:54:18.730 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.730 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.730 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:18.730 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356967+0000", 2024-05-08T08:54:18.730 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0409999999999998e-06 2024-05-08T08:54:18.730 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.730 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.730 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:18.730 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356979+0000", 2024-05-08T08:54:18.730 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1489e-05 2024-05-08T08:54:18.730 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.731 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.731 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:18.731 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356986+0000", 2024-05-08T08:54:18.731 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3699999999999997e-06 2024-05-08T08:54:18.731 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.731 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.731 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:18.731 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435433+0000", 2024-05-08T08:54:18.731 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078447520000000007 2024-05-08T08:54:18.731 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.731 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.731 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:18.731 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435484+0000", 2024-05-08T08:54:18.731 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0229000000000003e-05 2024-05-08T08:54:18.731 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.732 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.732 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:18.732 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435575+0000", 2024-05-08T08:54:18.732 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1299999999999997e-05 2024-05-08T08:54:18.732 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.732 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:18.732 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.732 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.732 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.732 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28553 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888713648] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:18.732 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:54:18.732 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 46.337154589000001, 2024-05-08T08:54:18.732 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078782034000000001, 2024-05-08T08:54:18.732 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:18.733 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:18.733 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:18.733 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:18.733 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:18.733 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:18.733 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28553 2024-05-08T08:54:18.733 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.733 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:18.733 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.733 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:18.733 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:54:18.733 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:18.733 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.733 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.733 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:18.733 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:54:18.734 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:18.734 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.734 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.734 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:18.734 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:54:18.734 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6600000000000002e-07 2024-05-08T08:54:18.734 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.734 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.734 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:18.734 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357006+0000", 2024-05-08T08:54:18.734 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7190000000000001e-06 2024-05-08T08:54:18.734 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.734 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.734 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:18.735 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357013+0000", 2024-05-08T08:54:18.735 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.435e-06 2024-05-08T08:54:18.735 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.735 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.735 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:18.735 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357018+0000", 2024-05-08T08:54:18.735 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7260000000000002e-06 2024-05-08T08:54:18.735 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.735 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.735 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:18.735 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435623+0000", 2024-05-08T08:54:18.735 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078604972999999995 2024-05-08T08:54:18.735 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.735 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.735 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:18.735 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435695+0000", 2024-05-08T08:54:18.736 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1959999999999995e-05 2024-05-08T08:54:18.736 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.736 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.736 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:18.736 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435785+0000", 2024-05-08T08:54:18.736 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.9754999999999997e-05 2024-05-08T08:54:18.736 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.736 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:18.736 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.736 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.736 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.736 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28554 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888980272] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:18.736 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357024+0000", 2024-05-08T08:54:18.736 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 46.337132953000001, 2024-05-08T08:54:18.737 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078999870999999999, 2024-05-08T08:54:18.737 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:18.737 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:18.737 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:18.737 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:18.737 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:18.737 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:18.737 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28554 2024-05-08T08:54:18.737 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.737 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:18.737 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.737 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:18.737 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357024+0000", 2024-05-08T08:54:18.737 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:18.737 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.737 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.738 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:18.738 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357024+0000", 2024-05-08T08:54:18.738 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:18.738 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.738 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.738 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:18.738 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357025+0000", 2024-05-08T08:54:18.738 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:54:18.738 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.738 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.738 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:18.738 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357027+0000", 2024-05-08T08:54:18.738 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2050000000000001e-06 2024-05-08T08:54:18.738 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.739 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.739 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:18.739 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357033+0000", 2024-05-08T08:54:18.739 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5679999999999996e-06 2024-05-08T08:54:18.739 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.739 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.739 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:18.739 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357038+0000", 2024-05-08T08:54:18.739 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2889999999999999e-06 2024-05-08T08:54:18.739 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.739 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.739 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:18.739 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435823+0000", 2024-05-08T08:54:18.739 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078785677999999998 2024-05-08T08:54:18.739 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.739 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.739 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:18.740 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435892+0000", 2024-05-08T08:54:18.740 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8379000000000004e-05 2024-05-08T08:54:18.740 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.740 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.740 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:18.740 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436024+0000", 2024-05-08T08:54:18.740 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013239799999999999 2024-05-08T08:54:18.740 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.740 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:18.740 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.740 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.740 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.740 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28555 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889001104] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:18.740 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:54:18.740 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 46.337114188000001, 2024-05-08T08:54:18.740 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079115742000000003, 2024-05-08T08:54:18.741 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:18.741 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:18.741 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:18.741 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:18.741 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:18.741 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:18.741 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28555 2024-05-08T08:54:18.741 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.741 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:18.741 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.741 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:18.741 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:54:18.741 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:18.741 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.741 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.742 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:18.742 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:54:18.742 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:18.742 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.742 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.742 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:18.742 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:54:18.742 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9200000000000002e-07 2024-05-08T08:54:18.742 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.742 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.742 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:18.742 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357046+0000", 2024-05-08T08:54:18.742 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1670000000000002e-06 2024-05-08T08:54:18.742 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.742 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.743 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:18.743 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357052+0000", 2024-05-08T08:54:18.743 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.139e-06 2024-05-08T08:54:18.743 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.743 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.743 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:18.743 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357056+0000", 2024-05-08T08:54:18.743 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2440000000000004e-06 2024-05-08T08:54:18.743 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.743 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.743 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:18.743 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436059+0000", 2024-05-08T08:54:18.743 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079003070999999994 2024-05-08T08:54:18.743 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.743 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.744 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:18.744 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436097+0000", 2024-05-08T08:54:18.744 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7784000000000003e-05 2024-05-08T08:54:18.744 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.744 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.744 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:18.744 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436159+0000", 2024-05-08T08:54:18.744 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2045000000000007e-05 2024-05-08T08:54:18.744 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.744 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:18.744 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.744 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.744 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.744 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28556 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889022064] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:18.745 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357065+0000", 2024-05-08T08:54:18.745 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 46.337092155000001, 2024-05-08T08:54:18.745 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079327976999999994, 2024-05-08T08:54:18.745 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:18.745 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:18.745 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:18.745 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:18.745 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:18.745 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:18.745 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28556 2024-05-08T08:54:18.745 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.745 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:18.745 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.745 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:18.746 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357065+0000", 2024-05-08T08:54:18.746 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:18.746 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.746 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.746 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:18.746 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357065+0000", 2024-05-08T08:54:18.746 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:18.746 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.746 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.746 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:18.746 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358958+0000", 2024-05-08T08:54:18.746 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001892915 2024-05-08T08:54:18.746 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.746 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.746 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:18.747 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358962+0000", 2024-05-08T08:54:18.747 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4599999999999999e-06 2024-05-08T08:54:18.747 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.747 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.747 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:18.747 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358970+0000", 2024-05-08T08:54:18.747 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.6090000000000001e-06 2024-05-08T08:54:18.747 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.747 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.747 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:18.747 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358975+0000", 2024-05-08T08:54:18.747 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7639999999999996e-06 2024-05-08T08:54:18.747 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.747 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.747 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:18.748 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436196+0000", 2024-05-08T08:54:18.748 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.077221564000000006 2024-05-08T08:54:18.748 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.748 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.748 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:18.748 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436269+0000", 2024-05-08T08:54:18.748 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2720999999999997e-05 2024-05-08T08:54:18.748 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.748 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.748 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:18.748 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436393+0000", 2024-05-08T08:54:18.748 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012394399999999999 2024-05-08T08:54:18.748 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.748 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:18.748 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.749 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.749 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.749 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28563 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888855520] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:18.749 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:54:18.749 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 46.335082755999998, 2024-05-08T08:54:18.749 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078795566999999997, 2024-05-08T08:54:18.749 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:18.749 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:18.749 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:18.749 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:18.749 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:18.749 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:18.749 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28563 2024-05-08T08:54:18.749 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.750 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:18.750 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.750 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:18.750 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:54:18.750 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:18.750 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.750 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.750 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:18.750 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:54:18.750 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:18.750 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.750 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.750 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:18.750 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:54:18.751 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1e-07 2024-05-08T08:54:18.751 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.751 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.751 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:18.751 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359080+0000", 2024-05-08T08:54:18.751 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8940000000000001e-06 2024-05-08T08:54:18.751 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.751 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.751 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:18.751 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359083+0000", 2024-05-08T08:54:18.751 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1949999999999999e-06 2024-05-08T08:54:18.751 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.751 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.751 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:18.751 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359085+0000", 2024-05-08T08:54:18.752 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4710000000000001e-06 2024-05-08T08:54:18.752 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.752 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.752 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:18.752 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437604+0000", 2024-05-08T08:54:18.752 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078518278999999996 2024-05-08T08:54:18.752 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.752 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.752 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:18.752 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437704+0000", 2024-05-08T08:54:18.752 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010010599999999999 2024-05-08T08:54:18.752 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.752 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.752 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:18.752 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437870+0000", 2024-05-08T08:54:18.753 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00016641200000000001 2024-05-08T08:54:18.753 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.753 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:18.753 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.753 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.753 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.753 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28564 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888538944] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:18.753 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.359089+0000", 2024-05-08T08:54:18.753 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 46.335068540999998, 2024-05-08T08:54:18.753 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078952675, 2024-05-08T08:54:18.753 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:18.753 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:18.753 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:18.753 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:18.753 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:18.754 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:18.754 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28564 2024-05-08T08:54:18.754 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.754 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:18.754 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.754 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:18.754 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359089+0000", 2024-05-08T08:54:18.754 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:18.754 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.754 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.754 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:18.754 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359089+0000", 2024-05-08T08:54:18.754 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:18.754 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.755 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.755 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:18.755 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360959+0000", 2024-05-08T08:54:18.755 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0018707089999999999 2024-05-08T08:54:18.755 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.755 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.755 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:18.755 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360963+0000", 2024-05-08T08:54:18.755 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0680000000000001e-06 2024-05-08T08:54:18.755 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.755 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.755 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:18.755 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360969+0000", 2024-05-08T08:54:18.755 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1920000000000002e-06 2024-05-08T08:54:18.756 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.756 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.756 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:18.756 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360973+0000", 2024-05-08T08:54:18.756 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.031e-06 2024-05-08T08:54:18.756 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.756 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.756 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:18.756 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437916+0000", 2024-05-08T08:54:18.756 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.076942939000000002 2024-05-08T08:54:18.756 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.756 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.756 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:18.756 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437973+0000", 2024-05-08T08:54:18.756 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7358999999999997e-05 2024-05-08T08:54:18.757 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.757 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.757 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:18.757 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.438041+0000", 2024-05-08T08:54:18.757 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8376999999999999e-05 2024-05-08T08:54:18.757 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.757 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:18.757 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.757 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.757 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.757 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28574 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889427600] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:18.757 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:54:18.757 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 46.333064034000003, 2024-05-08T08:54:18.757 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078782476000000004, 2024-05-08T08:54:18.757 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:18.758 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:18.758 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:18.758 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:18.758 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:18.758 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:18.758 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28574 2024-05-08T08:54:18.758 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.758 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:18.758 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.758 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:18.758 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:54:18.758 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:18.758 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.759 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.759 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:18.759 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:54:18.759 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:18.759 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.759 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.759 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:18.759 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:54:18.759 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1799999999999999e-07 2024-05-08T08:54:18.759 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.759 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.759 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:18.759 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361095+0000", 2024-05-08T08:54:18.759 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3850000000000001e-06 2024-05-08T08:54:18.759 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.760 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.760 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:18.760 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361098+0000", 2024-05-08T08:54:18.760 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7989999999999998e-06 2024-05-08T08:54:18.760 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.760 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.760 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:18.760 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361100+0000", 2024-05-08T08:54:18.760 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4669999999999998e-06 2024-05-08T08:54:18.760 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.760 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.760 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:18.760 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439768+0000", 2024-05-08T08:54:18.760 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078668057999999999 2024-05-08T08:54:18.760 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.761 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.761 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:18.761 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439811+0000", 2024-05-08T08:54:18.761 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2459000000000001e-05 2024-05-08T08:54:18.761 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.761 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.761 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:18.761 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439876+0000", 2024-05-08T08:54:18.761 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5090000000000002e-05 2024-05-08T08:54:18.761 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.761 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:18.761 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.761 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.761 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.761 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28575 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889439216] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:18.762 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361103+0000", 2024-05-08T08:54:18.762 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 46.333054019000002, 2024-05-08T08:54:18.762 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078966828000000003, 2024-05-08T08:54:18.762 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:18.762 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:18.762 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:18.762 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:18.762 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:18.762 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:18.762 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28575 2024-05-08T08:54:18.762 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.762 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:18.762 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.762 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:18.762 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361103+0000", 2024-05-08T08:54:18.763 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:18.763 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.763 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.763 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:18.763 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361103+0000", 2024-05-08T08:54:18.763 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:18.763 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.763 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.763 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:18.763 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361104+0000", 2024-05-08T08:54:18.763 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1400000000000001e-07 2024-05-08T08:54:18.763 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.763 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.763 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:18.763 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361105+0000", 2024-05-08T08:54:18.764 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5540000000000001e-06 2024-05-08T08:54:18.764 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.764 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.764 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:18.764 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361108+0000", 2024-05-08T08:54:18.764 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.841e-06 2024-05-08T08:54:18.764 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.764 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.764 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:18.764 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361110+0000", 2024-05-08T08:54:18.764 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.497e-06 2024-05-08T08:54:18.764 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.764 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.765 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:18.765 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439916+0000", 2024-05-08T08:54:18.765 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078805372999999998 2024-05-08T08:54:18.765 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.765 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.765 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:18.765 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440000+0000", 2024-05-08T08:54:18.765 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4730999999999999e-05 2024-05-08T08:54:18.765 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.765 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.765 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:18.765 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440070+0000", 2024-05-08T08:54:18.765 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9617999999999997e-05 2024-05-08T08:54:18.765 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.766 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:18.766 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.766 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.766 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.766 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28576 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889451328] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:18.766 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361113+0000", 2024-05-08T08:54:18.766 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 46.333043832999998, 2024-05-08T08:54:18.766 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079088862999999995, 2024-05-08T08:54:18.766 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:18.766 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:18.766 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:18.766 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:18.766 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:18.767 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:18.767 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28576 2024-05-08T08:54:18.767 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.767 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:18.767 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.767 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:18.767 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361113+0000", 2024-05-08T08:54:18.767 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:18.767 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.767 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.767 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:18.767 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361113+0000", 2024-05-08T08:54:18.767 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:18.768 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.768 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.768 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:18.768 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361114+0000", 2024-05-08T08:54:18.768 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1299999999999999e-07 2024-05-08T08:54:18.768 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.768 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.768 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:18.768 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361115+0000", 2024-05-08T08:54:18.768 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.4300000000000001e-06 2024-05-08T08:54:18.768 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.768 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.768 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:18.769 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361118+0000", 2024-05-08T08:54:18.769 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9000000000000002e-06 2024-05-08T08:54:18.769 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.769 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.769 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:18.769 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361120+0000", 2024-05-08T08:54:18.769 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4739999999999999e-06 2024-05-08T08:54:18.769 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.769 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.769 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:18.769 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440096+0000", 2024-05-08T08:54:18.769 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078975629000000006 2024-05-08T08:54:18.769 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.769 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.770 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:18.770 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440135+0000", 2024-05-08T08:54:18.770 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9202000000000002e-05 2024-05-08T08:54:18.770 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.770 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.770 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:18.770 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440202+0000", 2024-05-08T08:54:18.770 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7014999999999994e-05 2024-05-08T08:54:18.770 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.770 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:18.770 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.770 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.770 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.770 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28577 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889462640] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:18.770 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:54:18.771 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 46.333033651000001, 2024-05-08T08:54:18.771 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079194622000000006, 2024-05-08T08:54:18.771 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:18.771 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:18.771 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:18.771 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:18.771 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:18.771 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:18.771 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28577 2024-05-08T08:54:18.771 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.771 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:18.771 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.771 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:18.771 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:54:18.771 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:18.772 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.772 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.772 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:18.772 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:54:18.772 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:18.772 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.772 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.772 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:18.772 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:54:18.772 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1400000000000001e-07 2024-05-08T08:54:18.772 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.772 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.772 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:18.772 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361125+0000", 2024-05-08T08:54:18.773 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.372e-06 2024-05-08T08:54:18.773 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.773 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.773 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:18.773 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361128+0000", 2024-05-08T08:54:18.773 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0309999999999999e-06 2024-05-08T08:54:18.773 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.773 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.773 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:18.773 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361131+0000", 2024-05-08T08:54:18.773 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4040000000000002e-06 2024-05-08T08:54:18.773 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.773 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.773 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:18.773 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440225+0000", 2024-05-08T08:54:18.774 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079094600000000001 2024-05-08T08:54:18.774 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.774 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.774 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:18.774 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440260+0000", 2024-05-08T08:54:18.774 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4703000000000003e-05 2024-05-08T08:54:18.774 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.774 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.774 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:18.774 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440318+0000", 2024-05-08T08:54:18.774 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8298000000000003e-05 2024-05-08T08:54:18.774 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.774 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:18.774 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.775 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.775 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.775 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28578 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889474192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:18.775 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361134+0000", 2024-05-08T08:54:18.775 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 46.333023226000002, 2024-05-08T08:54:18.775 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079310295000000003, 2024-05-08T08:54:18.775 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:18.775 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:18.775 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:18.775 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:18.775 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:18.775 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:18.775 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28578 2024-05-08T08:54:18.775 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.776 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:18.776 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.776 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:18.776 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361134+0000", 2024-05-08T08:54:18.776 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:18.776 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.776 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.776 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:18.776 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361134+0000", 2024-05-08T08:54:18.776 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:18.776 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.776 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.776 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:18.776 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.362958+0000", 2024-05-08T08:54:18.776 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0018237380000000001 2024-05-08T08:54:18.777 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.777 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.777 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:18.777 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.362961+0000", 2024-05-08T08:54:18.777 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.123e-06 2024-05-08T08:54:18.777 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.777 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.777 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:18.777 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.397404+0000", 2024-05-08T08:54:18.777 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.034443226 2024-05-08T08:54:18.777 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.777 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.777 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:18.777 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.397439+0000", 2024-05-08T08:54:18.777 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4690000000000002e-05 2024-05-08T08:54:18.778 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.778 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.778 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:18.778 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440340+0000", 2024-05-08T08:54:18.778 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.042901419000000003 2024-05-08T08:54:18.778 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.778 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.778 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:18.778 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440376+0000", 2024-05-08T08:54:18.778 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5444999999999997e-05 2024-05-08T08:54:18.778 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:18.778 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:18.778 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:18.778 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440444+0000", 2024-05-08T08:54:18.779 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8653999999999997e-05 2024-05-08T08:54:18.779 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.779 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:18.779 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.779 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:18.779 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:18.779 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:18.781 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:54:18.782 INFO:teuthology.orchestra.run.smithi059.stderr:noscrub is unset 2024-05-08T08:54:18.782 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:18.783+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:18.855 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T08:54:18.907 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:18.910+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:18.911 INFO:teuthology.orchestra.run.smithi196.stdout:5600 write 0x917e800 thru 0x9182125 (0x3926 bytes) 2024-05-08T08:54:18.913 INFO:teuthology.orchestra.run.smithi196.stdout:5601 write 0x9e3b200 thru 0x9e44231 (0x9032 bytes) 2024-05-08T08:54:18.918 INFO:teuthology.orchestra.run.smithi196.stdout:5602 trunc from 0xb128e00 to 0xe6dc600 2024-05-08T08:54:18.934 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:18.934 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:54:18.934 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:54:18.934 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:18.941 INFO:teuthology.orchestra.run.smithi196.stdout:5603 write 0x1d0f400 thru 0x1d1be38 (0xca39 bytes) 2024-05-08T08:54:18.942 INFO:teuthology.orchestra.run.smithi196.stdout:5604 write 0x6d72a00 thru 0x6d73fb3 (0x15b4 bytes) 2024-05-08T08:54:18.944 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:54:18.951 INFO:teuthology.orchestra.run.smithi196.stdout:5605 trunc from 0xe6dc600 to 0x66f5e00 2024-05-08T08:54:18.985 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:18.988+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:18.994 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:54:18.994 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:54:18.999 INFO:teuthology.orchestra.run.smithi196.stdout:5609 write 0xe8bde00 thru 0xe8bfc73 (0x1e74 bytes) 2024-05-08T08:54:18.999 INFO:teuthology.orchestra.run.smithi196.stdout:5610 punch from 0xd37d200 to 0xd380400, (0x3200 bytes) 2024-05-08T08:54:19.009 INFO:teuthology.orchestra.run.smithi196.stdout:5612 trunc from 0xe8bfe00 to 0x746a000 2024-05-08T08:54:19.016 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:54:19.016 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:54:19.024 INFO:teuthology.orchestra.run.smithi196.stdout:5613 write 0x2bd5600 thru 0x2be24c0 (0xcec1 bytes) 2024-05-08T08:54:19.034 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:54:19.034 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:54:19.049 INFO:teuthology.orchestra.run.smithi196.stdout:5614 write 0xbecfe00 thru 0xbedba02 (0xbc03 bytes) 2024-05-08T08:54:19.053 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:54:19.053 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:54:19.064 INFO:teuthology.orchestra.run.smithi196.stdout:5615 trunc from 0xbedbc00 to 0x28b200 2024-05-08T08:54:19.082 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:19.085+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:19.087 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:54:19.087 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:54:19.093 INFO:teuthology.orchestra.run.smithi196.stdout:5616 punch from 0xe9c00 to 0xeda00, (0x3e00 bytes) 2024-05-08T08:54:19.095 INFO:teuthology.orchestra.run.smithi196.stdout:5617 clone 44 order 24 su 2097152 sc 5 2024-05-08T08:54:19.106 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:54:19.107 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:54:19.114 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:19.114 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:19.124 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:54:19.208 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:19.211+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:19.274 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:19.274 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:19.283 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:54:19.284 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:19.287+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:19.385 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:19.387+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:19.388 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:19.388 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:19.397 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:54:19.485 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:19.488+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:19.502 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:19.502 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:19.511 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:54:19.595 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:19.598+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:19.656 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:19.657 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:19.665 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:54:19.706 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:19.709+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:19.788 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:19.791+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:19.802 INFO:teuthology.orchestra.run.smithi196.stdout:truncating image image_client.0-clone43 from 0x28b200 (overlap 0x28b200) to 0x1c7c00 2024-05-08T08:54:19.823 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:19.823 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:19.832 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:54:19.889 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:19.891+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:19.942 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:19.942 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:19.951 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:54:19.994 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:19.997+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:20.062 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:20.062 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:20.071 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:54:20.102 INFO:teuthology.orchestra.run.smithi196.stdout:checking clone #42, image image_client.0-clone42 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent43 2024-05-08T08:54:20.112 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:20.115+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:20.191 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:20.194+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:20.227 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:20.227 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:20.236 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:54:20.291 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:20.294+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:20.345 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:20.345 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:20.354 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:54:20.396 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:20.399+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:20.493 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:20.496+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:20.503 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:20.503 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:20.512 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:54:20.594 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:20.597+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:20.620 INFO:teuthology.orchestra.run.smithi196.stdout:5618 write 0x97d7800 thru 0x97de021 (0x6822 bytes) 2024-05-08T08:54:20.621 INFO:teuthology.orchestra.run.smithi196.stdout:5622 write 0x494fe00 thru 0x4955c61 (0x5e62 bytes) 2024-05-08T08:54:20.621 INFO:teuthology.orchestra.run.smithi196.stdout:5623 read 0x6713e00 thru 0x671c6e9 (0x88ea bytes) 2024-05-08T08:54:20.621 INFO:teuthology.orchestra.run.smithi196.stdout:5624 read 0x48ff800 thru 0x49015e3 (0x1de4 bytes) 2024-05-08T08:54:20.621 INFO:teuthology.orchestra.run.smithi196.stdout:5626 write 0x30bb200 thru 0x30c06d4 (0x54d5 bytes) 2024-05-08T08:54:20.621 INFO:teuthology.orchestra.run.smithi196.stdout:5627 read 0x3da9600 thru 0x3dafa80 (0x6481 bytes) 2024-05-08T08:54:20.621 INFO:teuthology.orchestra.run.smithi196.stdout:5628 write 0x80bf600 thru 0x80c898e (0x938f bytes) 2024-05-08T08:54:20.622 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:20.622 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:20.627 INFO:teuthology.orchestra.run.smithi196.stdout:5629 trunc from 0x97de200 to 0x3d0fe00 2024-05-08T08:54:20.632 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:54:20.685 INFO:teuthology.orchestra.run.smithi196.stdout:5630 punch from 0x171b200 to 0x1722600, (0x7400 bytes) 2024-05-08T08:54:20.693 INFO:teuthology.orchestra.run.smithi196.stdout:5634 punch from 0x11fa400 to 0x1201400, (0x7000 bytes) 2024-05-08T08:54:20.694 INFO:teuthology.orchestra.run.smithi196.stdout:5636 punch from 0xe86a00 to 0xe95a00, (0xf000 bytes) 2024-05-08T08:54:20.694 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:20.697+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:20.709 INFO:teuthology.orchestra.run.smithi196.stdout:5637 write 0x7b8f600 thru 0x7b8ff05 (0x906 bytes) 2024-05-08T08:54:20.790 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-05-08T08:54:20.792 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:20.793 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:20.793 INFO:teuthology.orchestra.run.smithi196.stdout:5638 punch from 0x5572600 to 0x5579800, (0x7200 bytes) 2024-05-08T08:54:20.793 INFO:teuthology.orchestra.run.smithi196.stdout:5640 read 0x27e2600 thru 0x27ebd39 (0x973a bytes) 2024-05-08T08:54:20.793 INFO:teuthology.orchestra.run.smithi196.stdout:5647 trunc from 0x7b90000 to 0xddb4600 2024-05-08T08:54:20.793 INFO:teuthology.orchestra.run.smithi196.stdout:5650 write 0xdc63600 thru 0xdc65706 (0x2107 bytes) 2024-05-08T08:54:20.793 INFO:teuthology.orchestra.run.smithi196.stdout:5652 write 0x9e38800 thru 0x9e3af3a (0x273b bytes) 2024-05-08T08:54:20.793 INFO:teuthology.orchestra.run.smithi196.stdout:5653 trunc from 0xddb4600 to 0x4197c00 2024-05-08T08:54:20.793 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:54:20.797 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:20.800+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:20.800 INFO:teuthology.orchestra.run.smithi196.stdout:5655 write 0x67d2e00 thru 0x67d4b40 (0x1d41 bytes) 2024-05-08T08:54:20.813 INFO:teuthology.orchestra.run.smithi196.stdout:5656 trunc from 0x67d4c00 to 0xb23c200 2024-05-08T08:54:20.826 INFO:teuthology.orchestra.run.smithi196.stdout:5657 write 0xa8a6200 thru 0xa8ae993 (0x8794 bytes) 2024-05-08T08:54:20.849 INFO:teuthology.orchestra.run.smithi196.stdout:5658 write 0xebfa800 thru 0xec00c3f (0x6440 bytes) 2024-05-08T08:54:20.872 INFO:teuthology.orchestra.run.smithi196.stdout:5659 trunc from 0xec00e00 to 0x1083600 2024-05-08T08:54:20.896 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:20.899+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:20.921 INFO:teuthology.orchestra.run.smithi196.stdout:5669 write 0x1f7c200 thru 0x1f82622 (0x6423 bytes) 2024-05-08T08:54:20.923 INFO:teuthology.orchestra.run.smithi196.stdout:5670 punch from 0x1636a00 to 0x1637600, (0xc00 bytes) 2024-05-08T08:54:20.923 INFO:teuthology.orchestra.run.smithi196.stdout:5671 punch from 0x8c8a00 to 0x8d8800, (0xfe00 bytes) 2024-05-08T08:54:20.926 INFO:teuthology.orchestra.run.smithi196.stdout:5674 trunc from 0x1f82800 to 0x421c400 2024-05-08T08:54:20.934 INFO:teuthology.orchestra.run.smithi196.stdout:5675 punch from 0x367a600 to 0x3682c00, (0x8600 bytes) 2024-05-08T08:54:20.940 INFO:teuthology.orchestra.run.smithi196.stdout:5677 read 0x1620e00 thru 0x16291c5 (0x83c6 bytes) 2024-05-08T08:54:20.945 INFO:teuthology.orchestra.run.smithi196.stdout:5678 trunc from 0x421c400 to 0x806c200 2024-05-08T08:54:20.950 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:20.950 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:20.959 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:54:20.971 INFO:teuthology.orchestra.run.smithi196.stdout:5679 write 0x9f37000 thru 0x9f427b5 (0xb7b6 bytes) 2024-05-08T08:54:20.971 INFO:teuthology.orchestra.run.smithi196.stdout:5680 clone 45 order 24 su 2097152 sc 13 2024-05-08T08:54:21.000 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:21.003+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:21.098 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:21.101+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:21.109 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:21.109 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:21.118 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:54:21.118 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T08:54:21.203 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:21.206+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:21.271 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:21.271 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:21.280 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:54:21.285 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:54:21.286 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:54:21.300 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:21.303+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:21.304 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:54:21.304 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:54:21.321 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:54:21.322 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:54:21.339 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:54:21.339 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:54:21.357 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:54:21.357 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:54:21.375 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:54:21.375 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:54:21.399 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:21.399 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:54:21.399 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:54:21.399 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:54:21.399 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.399 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:12443 3.0 3:02d0158e:::rbd_data.114071bcf828.0000000000000037:head [stat,delete] snapc 19=[19] RETRY=3 ondisk+retry+write+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:21.399 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.610027+0000", 2024-05-08T08:54:21.399 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 49.790815891000001, 2024-05-08T08:54:21.399 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.083586333999999998, 2024-05-08T08:54:21.400 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:21.400 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:21.400 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:21.400 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:21.400 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:21.400 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:21.400 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 12443 2024-05-08T08:54:21.400 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.400 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:21.400 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.400 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:21.400 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610027+0000", 2024-05-08T08:54:21.401 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:21.401 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.401 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.401 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:21.401 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610027+0000", 2024-05-08T08:54:21.401 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:21.401 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.401 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.401 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:21.401 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610028+0000", 2024-05-08T08:54:21.401 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.13e-07 2024-05-08T08:54:21.402 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.402 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.402 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:21.402 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610030+0000", 2024-05-08T08:54:21.402 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7669999999999999e-06 2024-05-08T08:54:21.402 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.402 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.402 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:21.402 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610039+0000", 2024-05-08T08:54:21.402 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4959999999999996e-06 2024-05-08T08:54:21.402 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.403 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.403 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:21.403 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610045+0000", 2024-05-08T08:54:21.403 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6779999999999997e-06 2024-05-08T08:54:21.403 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.403 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.403 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:21.403 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.660873+0000", 2024-05-08T08:54:21.403 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.050828748 2024-05-08T08:54:21.403 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.403 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.403 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:21.404 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.661017+0000", 2024-05-08T08:54:21.404 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014391999999999999 2024-05-08T08:54:21.404 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.404 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.404 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:21.404 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.693613+0000", 2024-05-08T08:54:21.404 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.032596111999999997 2024-05-08T08:54:21.404 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:21.404 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:21.404 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:21.404 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.404 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.404 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:14301 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [watch reconnect cookie 139837155901536 gen 12] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:21.405 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.645078+0000", 2024-05-08T08:54:21.405 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 49.755765253, 2024-05-08T08:54:21.405 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088281852999999993, 2024-05-08T08:54:21.405 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:21.405 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:21.405 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:21.405 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:21.405 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:21.405 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:21.405 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14301 2024-05-08T08:54:21.405 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.405 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:21.405 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.406 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:21.406 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645078+0000", 2024-05-08T08:54:21.406 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:21.406 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.406 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.406 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:21.406 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645078+0000", 2024-05-08T08:54:21.406 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:21.406 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.406 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.406 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:21.406 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645084+0000", 2024-05-08T08:54:21.407 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4620000000000001e-06 2024-05-08T08:54:21.407 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.407 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.407 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:21.407 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645088+0000", 2024-05-08T08:54:21.407 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8039999999999999e-06 2024-05-08T08:54:21.407 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.407 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.407 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:21.407 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645106+0000", 2024-05-08T08:54:21.407 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.8318999999999999e-05 2024-05-08T08:54:21.407 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.408 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.408 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:21.408 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645113+0000", 2024-05-08T08:54:21.408 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0310000000000001e-06 2024-05-08T08:54:21.408 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.408 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.408 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:21.408 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.670493+0000", 2024-05-08T08:54:21.408 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.025379993 2024-05-08T08:54:21.408 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.408 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.408 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:21.408 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.670571+0000", 2024-05-08T08:54:21.409 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7613000000000004e-05 2024-05-08T08:54:21.409 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.409 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.409 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:21.409 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733360+0000", 2024-05-08T08:54:21.409 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.062788630999999998 2024-05-08T08:54:21.409 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:21.409 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:21.409 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:21.409 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.409 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.409 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:14420 3.0 3:06a8948e:::rbd_data.114071bcf828.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:21.410 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.649639+0000", 2024-05-08T08:54:21.410 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 49.751204104999999, 2024-05-08T08:54:21.410 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.083851897999999994, 2024-05-08T08:54:21.410 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:21.410 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:21.410 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:54:21.410 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:21.410 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:21.410 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:21.410 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14420 2024-05-08T08:54:21.410 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.410 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:21.411 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.411 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:21.411 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649639+0000", 2024-05-08T08:54:21.411 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:21.411 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.411 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.411 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:21.411 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649639+0000", 2024-05-08T08:54:21.411 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:21.411 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.411 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.411 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:21.412 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649640+0000", 2024-05-08T08:54:21.412 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6899999999999997e-07 2024-05-08T08:54:21.412 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.412 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.412 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:21.412 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649895+0000", 2024-05-08T08:54:21.412 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00025531000000000002 2024-05-08T08:54:21.412 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.412 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.412 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:21.412 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649905+0000", 2024-05-08T08:54:21.412 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0198e-05 2024-05-08T08:54:21.412 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.413 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.413 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:21.413 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649911+0000", 2024-05-08T08:54:21.413 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2310000000000001e-06 2024-05-08T08:54:21.413 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.413 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.413 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:21.413 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.684099+0000", 2024-05-08T08:54:21.413 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.034188017000000001 2024-05-08T08:54:21.413 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.413 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.413 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:21.413 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.684172+0000", 2024-05-08T08:54:21.414 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3201999999999995e-05 2024-05-08T08:54:21.414 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.414 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.414 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 2", 2024-05-08T08:54:21.414 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.684418+0000", 2024-05-08T08:54:21.414 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00024559099999999997 2024-05-08T08:54:21.414 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.414 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.414 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:54:21.414 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.705538+0000", 2024-05-08T08:54:21.414 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.021119707000000001 2024-05-08T08:54:21.415 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.415 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.415 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:54:21.415 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733399+0000", 2024-05-08T08:54:21.415 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.027861543999999999 2024-05-08T08:54:21.415 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.415 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.415 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:54:21.415 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733445+0000", 2024-05-08T08:54:21.415 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5587000000000002e-05 2024-05-08T08:54:21.415 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.415 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.416 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:21.416 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733491+0000", 2024-05-08T08:54:21.416 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5841999999999999e-05 2024-05-08T08:54:21.416 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:21.416 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:21.416 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:21.416 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.416 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.416 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28547 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889179216] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:21.416 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.355654+0000", 2024-05-08T08:54:21.416 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 49.045188877000001, 2024-05-08T08:54:21.416 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078692261999999999, 2024-05-08T08:54:21.416 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:21.417 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:21.417 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:21.417 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:21.417 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:21.417 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:21.417 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28547 2024-05-08T08:54:21.417 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.417 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:21.417 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.417 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:21.417 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355654+0000", 2024-05-08T08:54:21.417 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:21.417 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.418 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.418 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:21.418 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355654+0000", 2024-05-08T08:54:21.418 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:21.418 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.418 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.418 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:21.418 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355654+0000", 2024-05-08T08:54:21.418 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3999999999999998e-07 2024-05-08T08:54:21.418 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.418 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.419 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:21.419 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355656+0000", 2024-05-08T08:54:21.419 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.353e-06 2024-05-08T08:54:21.419 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.419 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.419 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:21.419 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355663+0000", 2024-05-08T08:54:21.419 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2259999999999999e-06 2024-05-08T08:54:21.419 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.419 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.419 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:21.419 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355666+0000", 2024-05-08T08:54:21.419 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7930000000000002e-06 2024-05-08T08:54:21.419 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.420 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.420 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:21.420 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434117+0000", 2024-05-08T08:54:21.420 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078451231999999996 2024-05-08T08:54:21.420 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.420 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.420 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:21.420 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434198+0000", 2024-05-08T08:54:21.420 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0879999999999998e-05 2024-05-08T08:54:21.420 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.420 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.420 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:21.420 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434346+0000", 2024-05-08T08:54:21.420 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014853800000000001 2024-05-08T08:54:21.421 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:21.421 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:21.421 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:21.421 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.421 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.421 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28548 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889189696] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:21.421 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.355669+0000", 2024-05-08T08:54:21.421 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 49.045173527000003, 2024-05-08T08:54:21.421 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078956606999999998, 2024-05-08T08:54:21.421 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:21.421 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:21.421 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:21.422 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:21.422 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:21.422 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:21.422 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28548 2024-05-08T08:54:21.422 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.422 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:21.422 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.422 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:21.422 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355669+0000", 2024-05-08T08:54:21.423 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:21.423 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.423 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.423 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:21.423 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355669+0000", 2024-05-08T08:54:21.423 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:21.423 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.423 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.423 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:21.423 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355762+0000", 2024-05-08T08:54:21.423 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3015000000000003e-05 2024-05-08T08:54:21.423 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.424 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.424 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:21.424 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355765+0000", 2024-05-08T08:54:21.424 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.368e-06 2024-05-08T08:54:21.424 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.424 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.424 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:21.424 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355771+0000", 2024-05-08T08:54:21.424 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5989999999999998e-06 2024-05-08T08:54:21.424 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.424 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.425 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:21.425 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355775+0000", 2024-05-08T08:54:21.425 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7280000000000001e-06 2024-05-08T08:54:21.425 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.425 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.425 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:21.425 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434379+0000", 2024-05-08T08:54:21.425 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078603662000000005 2024-05-08T08:54:21.425 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.425 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.425 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:21.425 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434473+0000", 2024-05-08T08:54:21.425 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4104000000000006e-05 2024-05-08T08:54:21.426 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.426 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.426 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:21.426 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434626+0000", 2024-05-08T08:54:21.426 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00015313100000000001 2024-05-08T08:54:21.426 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:21.426 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:21.426 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:21.426 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.426 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.426 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28549 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889202400] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:21.426 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.355781+0000", 2024-05-08T08:54:21.427 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 49.045061953000001, 2024-05-08T08:54:21.427 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079117946999999994, 2024-05-08T08:54:21.427 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:21.427 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:21.427 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:21.427 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:21.427 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:21.427 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:21.427 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28549 2024-05-08T08:54:21.427 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.427 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:21.427 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.427 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:21.427 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355781+0000", 2024-05-08T08:54:21.428 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:21.428 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.428 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.428 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:21.428 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355781+0000", 2024-05-08T08:54:21.428 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:21.428 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.428 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.428 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:21.428 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356162+0000", 2024-05-08T08:54:21.428 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00038099100000000001 2024-05-08T08:54:21.428 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.428 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.428 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:21.429 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356165+0000", 2024-05-08T08:54:21.429 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1240000000000001e-06 2024-05-08T08:54:21.429 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.429 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.429 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:21.429 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356173+0000", 2024-05-08T08:54:21.429 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.6199999999999999e-06 2024-05-08T08:54:21.429 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.429 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.429 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:21.429 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356182+0000", 2024-05-08T08:54:21.429 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7760000000000003e-06 2024-05-08T08:54:21.430 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.430 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.430 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:21.430 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434675+0000", 2024-05-08T08:54:21.430 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078493119 2024-05-08T08:54:21.430 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.430 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.430 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:21.430 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434765+0000", 2024-05-08T08:54:21.430 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0402000000000006e-05 2024-05-08T08:54:21.430 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.430 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.431 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:21.431 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434899+0000", 2024-05-08T08:54:21.431 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013391499999999999 2024-05-08T08:54:21.431 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:21.431 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:21.431 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:21.431 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.431 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.431 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28550 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888597280] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:21.431 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:54:21.431 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 49.044650906999998, 2024-05-08T08:54:21.431 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078933990999999995, 2024-05-08T08:54:21.432 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:21.432 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:21.432 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:21.432 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:21.432 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:21.432 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:21.432 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28550 2024-05-08T08:54:21.432 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.432 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:21.433 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.433 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:21.433 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:54:21.433 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:21.433 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.433 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.433 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:21.433 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:54:21.433 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:21.433 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.434 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.434 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:21.434 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:54:21.434 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.34e-07 2024-05-08T08:54:21.434 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.434 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.434 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:21.434 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356195+0000", 2024-05-08T08:54:21.434 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8710000000000001e-06 2024-05-08T08:54:21.434 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.434 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.435 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:21.435 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356200+0000", 2024-05-08T08:54:21.435 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3000000000000003e-06 2024-05-08T08:54:21.435 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.435 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.435 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:21.435 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356203+0000", 2024-05-08T08:54:21.435 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4039999999999999e-06 2024-05-08T08:54:21.435 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.435 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.435 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:21.436 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434928+0000", 2024-05-08T08:54:21.436 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078725331999999995 2024-05-08T08:54:21.436 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.436 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.436 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:21.436 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435007+0000", 2024-05-08T08:54:21.436 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8415000000000001e-05 2024-05-08T08:54:21.436 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.436 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.436 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:21.436 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435126+0000", 2024-05-08T08:54:21.437 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011923500000000001 2024-05-08T08:54:21.437 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:21.437 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:21.437 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:21.437 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.437 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.437 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28551 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888527200] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:21.437 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.356210+0000", 2024-05-08T08:54:21.437 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 49.044632559, 2024-05-08T08:54:21.437 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079195072000000005, 2024-05-08T08:54:21.437 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:21.437 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:21.438 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:21.438 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:21.438 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:21.438 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:21.438 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28551 2024-05-08T08:54:21.438 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.438 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:21.438 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.438 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:21.438 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356210+0000", 2024-05-08T08:54:21.438 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:21.439 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.439 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.439 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:21.439 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356210+0000", 2024-05-08T08:54:21.439 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:21.439 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.439 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.439 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:21.439 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356211+0000", 2024-05-08T08:54:21.439 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4499999999999998e-07 2024-05-08T08:54:21.440 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.440 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.440 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:21.440 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356212+0000", 2024-05-08T08:54:21.440 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6029999999999999e-06 2024-05-08T08:54:21.440 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.440 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.440 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:21.440 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356221+0000", 2024-05-08T08:54:21.440 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4200000000000007e-06 2024-05-08T08:54:21.441 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.441 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.441 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:21.441 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356224+0000", 2024-05-08T08:54:21.441 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.038e-06 2024-05-08T08:54:21.441 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.441 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.441 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:21.441 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435178+0000", 2024-05-08T08:54:21.441 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078954625000000001 2024-05-08T08:54:21.441 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.441 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.441 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:21.442 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435256+0000", 2024-05-08T08:54:21.442 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7337999999999997e-05 2024-05-08T08:54:21.442 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.442 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.442 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:21.442 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435405+0000", 2024-05-08T08:54:21.442 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000149703 2024-05-08T08:54:21.442 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:21.442 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:21.442 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:21.442 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.442 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.443 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28552 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889231456] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:21.443 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.356228+0000", 2024-05-08T08:54:21.443 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 49.044614881000001, 2024-05-08T08:54:21.443 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079346920000000001, 2024-05-08T08:54:21.443 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:21.443 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:21.443 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:21.443 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:21.443 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:21.443 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:21.443 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28552 2024-05-08T08:54:21.443 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.443 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:21.444 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.444 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:21.444 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356228+0000", 2024-05-08T08:54:21.444 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:21.444 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.444 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.444 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:21.444 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356228+0000", 2024-05-08T08:54:21.444 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:21.444 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.445 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.445 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:21.445 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356959+0000", 2024-05-08T08:54:21.445 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00073097099999999996 2024-05-08T08:54:21.445 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.445 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.445 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:21.445 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356967+0000", 2024-05-08T08:54:21.445 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0409999999999998e-06 2024-05-08T08:54:21.445 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.445 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.445 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:21.445 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356979+0000", 2024-05-08T08:54:21.445 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1489e-05 2024-05-08T08:54:21.446 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.446 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.446 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:21.446 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356986+0000", 2024-05-08T08:54:21.446 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3699999999999997e-06 2024-05-08T08:54:21.446 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.446 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.446 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:21.446 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435433+0000", 2024-05-08T08:54:21.446 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078447520000000007 2024-05-08T08:54:21.446 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.446 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.447 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:21.447 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435484+0000", 2024-05-08T08:54:21.447 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0229000000000003e-05 2024-05-08T08:54:21.447 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.447 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.447 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:21.447 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435575+0000", 2024-05-08T08:54:21.447 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1299999999999997e-05 2024-05-08T08:54:21.447 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:21.447 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:21.447 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:21.447 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.447 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.448 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28553 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888713648] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:21.448 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:54:21.448 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 49.043840222999997, 2024-05-08T08:54:21.448 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078782034000000001, 2024-05-08T08:54:21.448 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:21.448 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:21.448 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:21.448 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:21.448 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:21.448 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:21.448 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28553 2024-05-08T08:54:21.448 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.448 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:21.449 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.449 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:21.449 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:54:21.449 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:21.449 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.449 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.449 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:21.449 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:54:21.449 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:21.449 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.449 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.449 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:21.449 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:54:21.450 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6600000000000002e-07 2024-05-08T08:54:21.450 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.450 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.450 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:21.450 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357006+0000", 2024-05-08T08:54:21.450 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7190000000000001e-06 2024-05-08T08:54:21.450 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.450 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.450 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:21.450 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357013+0000", 2024-05-08T08:54:21.450 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.435e-06 2024-05-08T08:54:21.451 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.451 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.451 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:21.451 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357018+0000", 2024-05-08T08:54:21.451 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7260000000000002e-06 2024-05-08T08:54:21.451 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.451 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.451 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:21.451 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435623+0000", 2024-05-08T08:54:21.451 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078604972999999995 2024-05-08T08:54:21.451 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.451 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.451 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:21.452 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435695+0000", 2024-05-08T08:54:21.452 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1959999999999995e-05 2024-05-08T08:54:21.452 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.452 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.452 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:21.452 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435785+0000", 2024-05-08T08:54:21.452 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.9754999999999997e-05 2024-05-08T08:54:21.452 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:21.452 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:21.452 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:21.452 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.452 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.452 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28554 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888980272] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:21.452 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357024+0000", 2024-05-08T08:54:21.453 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 49.043818586999997, 2024-05-08T08:54:21.453 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078999870999999999, 2024-05-08T08:54:21.453 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:21.453 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:21.453 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:21.453 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:21.453 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:21.453 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:21.453 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28554 2024-05-08T08:54:21.453 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.453 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:21.453 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.453 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:21.454 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357024+0000", 2024-05-08T08:54:21.454 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:21.454 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.454 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.454 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:21.454 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357024+0000", 2024-05-08T08:54:21.454 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:21.454 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.454 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.454 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:21.454 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357025+0000", 2024-05-08T08:54:21.454 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:54:21.455 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.455 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.455 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:21.455 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357027+0000", 2024-05-08T08:54:21.455 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2050000000000001e-06 2024-05-08T08:54:21.455 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.455 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.455 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:21.455 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357033+0000", 2024-05-08T08:54:21.455 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5679999999999996e-06 2024-05-08T08:54:21.455 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.456 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.456 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:21.456 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357038+0000", 2024-05-08T08:54:21.456 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2889999999999999e-06 2024-05-08T08:54:21.456 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.456 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.456 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:21.456 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435823+0000", 2024-05-08T08:54:21.456 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078785677999999998 2024-05-08T08:54:21.456 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.456 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.456 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:21.456 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435892+0000", 2024-05-08T08:54:21.457 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8379000000000004e-05 2024-05-08T08:54:21.457 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.457 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.457 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:21.457 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436024+0000", 2024-05-08T08:54:21.457 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013239799999999999 2024-05-08T08:54:21.457 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:21.457 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:21.457 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:21.457 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.457 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.457 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28555 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889001104] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:21.458 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:54:21.458 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 49.043799821999997, 2024-05-08T08:54:21.458 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079115742000000003, 2024-05-08T08:54:21.458 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:21.458 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:21.458 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:21.458 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:21.458 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:21.458 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:21.458 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28555 2024-05-08T08:54:21.458 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.458 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:21.459 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.459 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:21.459 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:54:21.459 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:21.459 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.459 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.459 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:21.459 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:54:21.459 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:21.459 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.459 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.459 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:21.459 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:54:21.460 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9200000000000002e-07 2024-05-08T08:54:21.460 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.460 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.460 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:21.460 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357046+0000", 2024-05-08T08:54:21.460 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1670000000000002e-06 2024-05-08T08:54:21.460 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.460 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.460 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:21.460 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357052+0000", 2024-05-08T08:54:21.460 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.139e-06 2024-05-08T08:54:21.460 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.460 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.460 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:21.461 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357056+0000", 2024-05-08T08:54:21.461 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2440000000000004e-06 2024-05-08T08:54:21.461 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.461 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.461 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:21.461 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436059+0000", 2024-05-08T08:54:21.461 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079003070999999994 2024-05-08T08:54:21.461 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.461 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.461 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:21.461 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436097+0000", 2024-05-08T08:54:21.461 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7784000000000003e-05 2024-05-08T08:54:21.461 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.462 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.462 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:21.462 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436159+0000", 2024-05-08T08:54:21.462 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2045000000000007e-05 2024-05-08T08:54:21.462 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:21.462 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:21.462 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:21.462 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.462 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.462 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28556 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889022064] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:21.462 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357065+0000", 2024-05-08T08:54:21.462 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 49.043777789000004, 2024-05-08T08:54:21.463 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079327976999999994, 2024-05-08T08:54:21.463 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:21.463 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:21.463 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:21.463 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:21.463 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:21.463 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:21.463 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28556 2024-05-08T08:54:21.463 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.463 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:21.463 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.463 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:21.463 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357065+0000", 2024-05-08T08:54:21.463 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:21.464 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.464 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.464 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:21.464 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357065+0000", 2024-05-08T08:54:21.464 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:21.464 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.464 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.464 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:21.464 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358958+0000", 2024-05-08T08:54:21.464 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001892915 2024-05-08T08:54:21.464 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.464 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.464 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:21.465 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358962+0000", 2024-05-08T08:54:21.465 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4599999999999999e-06 2024-05-08T08:54:21.465 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.465 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.465 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:21.465 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358970+0000", 2024-05-08T08:54:21.465 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.6090000000000001e-06 2024-05-08T08:54:21.465 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.465 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.465 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:21.465 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358975+0000", 2024-05-08T08:54:21.465 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7639999999999996e-06 2024-05-08T08:54:21.466 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.466 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.466 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:21.466 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436196+0000", 2024-05-08T08:54:21.466 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.077221564000000006 2024-05-08T08:54:21.466 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.466 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.466 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:21.466 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436269+0000", 2024-05-08T08:54:21.466 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2720999999999997e-05 2024-05-08T08:54:21.466 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.466 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.467 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:21.467 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436393+0000", 2024-05-08T08:54:21.467 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012394399999999999 2024-05-08T08:54:21.467 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:21.467 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:21.467 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:21.467 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.467 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.467 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28563 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888855520] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:21.467 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:54:21.467 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 49.041768390000001, 2024-05-08T08:54:21.467 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078795566999999997, 2024-05-08T08:54:21.468 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:21.468 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:21.468 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:21.468 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:21.468 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:21.468 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:21.468 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28563 2024-05-08T08:54:21.468 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.468 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:21.468 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.469 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:21.469 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:54:21.469 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:21.469 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.469 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.469 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:21.469 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:54:21.469 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:21.469 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.469 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.469 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:21.470 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:54:21.470 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1e-07 2024-05-08T08:54:21.470 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.470 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.470 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:21.470 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359080+0000", 2024-05-08T08:54:21.470 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8940000000000001e-06 2024-05-08T08:54:21.470 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.470 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.470 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:21.470 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359083+0000", 2024-05-08T08:54:21.470 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1949999999999999e-06 2024-05-08T08:54:21.471 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.471 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.471 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:21.471 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359085+0000", 2024-05-08T08:54:21.471 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4710000000000001e-06 2024-05-08T08:54:21.471 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.471 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.471 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:21.471 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437604+0000", 2024-05-08T08:54:21.471 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078518278999999996 2024-05-08T08:54:21.471 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.472 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.472 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:21.472 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437704+0000", 2024-05-08T08:54:21.472 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010010599999999999 2024-05-08T08:54:21.472 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.472 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.472 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:21.472 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437870+0000", 2024-05-08T08:54:21.472 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00016641200000000001 2024-05-08T08:54:21.472 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:21.472 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:21.473 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:21.473 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.473 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.473 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28564 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888538944] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:21.473 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.359089+0000", 2024-05-08T08:54:21.473 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 49.041754175000001, 2024-05-08T08:54:21.473 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078952675, 2024-05-08T08:54:21.473 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:21.473 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:21.473 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:21.473 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:21.474 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:21.474 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:21.474 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28564 2024-05-08T08:54:21.474 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.474 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:21.474 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.474 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:21.474 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359089+0000", 2024-05-08T08:54:21.474 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:21.474 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.475 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.475 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:21.475 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359089+0000", 2024-05-08T08:54:21.475 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:21.475 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.475 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.475 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:21.475 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360959+0000", 2024-05-08T08:54:21.475 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0018707089999999999 2024-05-08T08:54:21.475 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.475 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.475 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:21.476 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360963+0000", 2024-05-08T08:54:21.476 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0680000000000001e-06 2024-05-08T08:54:21.476 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.476 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.476 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:21.476 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360969+0000", 2024-05-08T08:54:21.476 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1920000000000002e-06 2024-05-08T08:54:21.476 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.476 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.476 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:21.476 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360973+0000", 2024-05-08T08:54:21.476 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.031e-06 2024-05-08T08:54:21.477 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.477 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.477 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:21.477 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437916+0000", 2024-05-08T08:54:21.477 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.076942939000000002 2024-05-08T08:54:21.477 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.477 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.477 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:21.477 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437973+0000", 2024-05-08T08:54:21.477 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7358999999999997e-05 2024-05-08T08:54:21.477 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.477 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.477 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:21.477 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.438041+0000", 2024-05-08T08:54:21.478 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8376999999999999e-05 2024-05-08T08:54:21.478 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:21.478 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:21.478 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:21.478 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.478 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.478 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28574 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889427600] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:21.478 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:54:21.478 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 49.039749667999999, 2024-05-08T08:54:21.478 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078782476000000004, 2024-05-08T08:54:21.478 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:21.478 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:21.479 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:21.479 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:21.479 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:21.479 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:21.479 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28574 2024-05-08T08:54:21.479 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.479 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:21.479 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.479 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:21.479 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:54:21.479 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:21.479 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.480 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.480 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:21.480 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:54:21.480 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:21.480 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.480 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.480 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:21.480 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:54:21.480 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1799999999999999e-07 2024-05-08T08:54:21.480 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.480 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.480 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:21.480 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361095+0000", 2024-05-08T08:54:21.481 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3850000000000001e-06 2024-05-08T08:54:21.481 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.481 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.481 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:21.481 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361098+0000", 2024-05-08T08:54:21.481 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7989999999999998e-06 2024-05-08T08:54:21.481 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.481 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.481 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:21.481 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361100+0000", 2024-05-08T08:54:21.481 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4669999999999998e-06 2024-05-08T08:54:21.482 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.482 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.482 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:21.482 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439768+0000", 2024-05-08T08:54:21.482 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078668057999999999 2024-05-08T08:54:21.482 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.482 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.482 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:21.482 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439811+0000", 2024-05-08T08:54:21.482 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2459000000000001e-05 2024-05-08T08:54:21.482 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.483 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.483 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:21.483 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439876+0000", 2024-05-08T08:54:21.483 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5090000000000002e-05 2024-05-08T08:54:21.483 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:21.483 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:21.483 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:21.483 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.483 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.483 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28575 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889439216] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:21.483 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361103+0000", 2024-05-08T08:54:21.483 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 49.039739652999998, 2024-05-08T08:54:21.484 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078966828000000003, 2024-05-08T08:54:21.484 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:21.484 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:21.484 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:21.484 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:21.484 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:21.484 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:21.484 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28575 2024-05-08T08:54:21.484 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.484 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:21.484 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.485 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:21.485 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361103+0000", 2024-05-08T08:54:21.485 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:21.485 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.485 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.485 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:21.485 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361103+0000", 2024-05-08T08:54:21.485 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:21.485 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.485 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.485 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:21.486 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361104+0000", 2024-05-08T08:54:21.486 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1400000000000001e-07 2024-05-08T08:54:21.486 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.486 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.486 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:21.486 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361105+0000", 2024-05-08T08:54:21.486 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5540000000000001e-06 2024-05-08T08:54:21.486 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.486 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.486 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:21.486 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361108+0000", 2024-05-08T08:54:21.486 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.841e-06 2024-05-08T08:54:21.487 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.487 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.487 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:21.487 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361110+0000", 2024-05-08T08:54:21.487 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.497e-06 2024-05-08T08:54:21.487 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.487 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.487 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:21.487 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439916+0000", 2024-05-08T08:54:21.487 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078805372999999998 2024-05-08T08:54:21.487 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.487 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.488 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:21.488 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440000+0000", 2024-05-08T08:54:21.488 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4730999999999999e-05 2024-05-08T08:54:21.488 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.488 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.488 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:21.488 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440070+0000", 2024-05-08T08:54:21.488 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9617999999999997e-05 2024-05-08T08:54:21.488 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:21.488 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:21.488 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:21.488 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.489 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.489 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28576 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889451328] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:21.489 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361113+0000", 2024-05-08T08:54:21.489 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 49.039729467000001, 2024-05-08T08:54:21.489 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079088862999999995, 2024-05-08T08:54:21.489 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:21.489 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:21.489 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:21.489 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:21.489 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:21.489 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:21.489 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28576 2024-05-08T08:54:21.490 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.490 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:21.490 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.490 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:21.490 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361113+0000", 2024-05-08T08:54:21.490 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:21.490 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.490 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.490 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:21.490 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361113+0000", 2024-05-08T08:54:21.490 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:21.490 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.491 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.491 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:21.491 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361114+0000", 2024-05-08T08:54:21.491 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1299999999999999e-07 2024-05-08T08:54:21.491 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.491 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.491 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:21.491 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361115+0000", 2024-05-08T08:54:21.491 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.4300000000000001e-06 2024-05-08T08:54:21.491 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.491 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.492 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:21.492 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361118+0000", 2024-05-08T08:54:21.492 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9000000000000002e-06 2024-05-08T08:54:21.492 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.492 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.492 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:21.492 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361120+0000", 2024-05-08T08:54:21.492 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4739999999999999e-06 2024-05-08T08:54:21.492 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.492 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.492 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:21.492 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440096+0000", 2024-05-08T08:54:21.493 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078975629000000006 2024-05-08T08:54:21.493 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.493 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.493 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:21.493 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440135+0000", 2024-05-08T08:54:21.493 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9202000000000002e-05 2024-05-08T08:54:21.493 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.493 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.493 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:21.493 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440202+0000", 2024-05-08T08:54:21.493 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7014999999999994e-05 2024-05-08T08:54:21.494 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:21.494 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:21.494 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:21.494 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.494 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.494 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28577 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889462640] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:21.494 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:54:21.494 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 49.039719284999997, 2024-05-08T08:54:21.494 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079194622000000006, 2024-05-08T08:54:21.494 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:21.494 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:21.495 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:21.495 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:21.495 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:21.495 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:21.495 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28577 2024-05-08T08:54:21.495 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.495 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:21.495 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.495 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:21.495 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:54:21.495 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:21.495 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.495 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.496 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:21.496 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:54:21.496 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:21.496 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.496 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.496 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:21.496 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:54:21.496 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1400000000000001e-07 2024-05-08T08:54:21.496 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.496 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.496 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:21.497 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361125+0000", 2024-05-08T08:54:21.497 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.372e-06 2024-05-08T08:54:21.497 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.497 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.497 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:21.497 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361128+0000", 2024-05-08T08:54:21.497 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0309999999999999e-06 2024-05-08T08:54:21.497 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.497 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.497 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:21.497 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361131+0000", 2024-05-08T08:54:21.497 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4040000000000002e-06 2024-05-08T08:54:21.497 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.498 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.498 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:21.498 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440225+0000", 2024-05-08T08:54:21.498 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079094600000000001 2024-05-08T08:54:21.498 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.498 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.498 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:21.498 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440260+0000", 2024-05-08T08:54:21.498 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4703000000000003e-05 2024-05-08T08:54:21.498 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.498 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.499 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:21.499 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440318+0000", 2024-05-08T08:54:21.499 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8298000000000003e-05 2024-05-08T08:54:21.499 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:21.499 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:21.499 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:21.499 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.499 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.499 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28578 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889474192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:21.499 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361134+0000", 2024-05-08T08:54:21.499 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 49.039708859999998, 2024-05-08T08:54:21.499 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079310295000000003, 2024-05-08T08:54:21.499 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:21.500 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:21.500 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:21.500 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:21.500 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:21.500 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:21.500 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28578 2024-05-08T08:54:21.500 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.500 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:21.500 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.500 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:21.500 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361134+0000", 2024-05-08T08:54:21.500 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:21.501 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.501 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.501 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:21.501 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361134+0000", 2024-05-08T08:54:21.501 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:21.501 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.501 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.501 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:21.501 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.362958+0000", 2024-05-08T08:54:21.501 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0018237380000000001 2024-05-08T08:54:21.501 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.501 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.502 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:21.502 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.362961+0000", 2024-05-08T08:54:21.502 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.123e-06 2024-05-08T08:54:21.502 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.502 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.502 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:21.502 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.397404+0000", 2024-05-08T08:54:21.502 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.034443226 2024-05-08T08:54:21.502 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.502 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.502 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:21.502 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.397439+0000", 2024-05-08T08:54:21.503 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4690000000000002e-05 2024-05-08T08:54:21.503 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.503 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.503 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:21.503 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440340+0000", 2024-05-08T08:54:21.503 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.042901419000000003 2024-05-08T08:54:21.503 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.503 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.503 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:21.503 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440376+0000", 2024-05-08T08:54:21.503 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5444999999999997e-05 2024-05-08T08:54:21.503 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.504 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.504 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:21.504 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440444+0000", 2024-05-08T08:54:21.504 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8653999999999997e-05 2024-05-08T08:54:21.504 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:21.504 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:21.504 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:21.504 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:21.504 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:21.504 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:21.505 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4] out_osds: [5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-05-08T08:54:21.505 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T08:54:21.505 INFO:tasks.thrashosds.thrasher:inject_pause on osd.1 2024-05-08T08:54:21.505 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-05-08T08:54:21.505 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-05-08T08:54:21.505 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok config set filestore_inject_stall 3 2024-05-08T08:54:21.507 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:54:21.507 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:21.508+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:21.607 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:21.610+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:21.660 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:21.660 INFO:teuthology.orchestra.run.smithi059.stdout: "success": "" 2024-05-08T08:54:21.660 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:21.660 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:21.661 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:54:21.661 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:54:21.661 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:21.671 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:54:21.712 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:21.715+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:21.735 INFO:teuthology.orchestra.run.smithi059.stderr:nodeep-scrub is unset 2024-05-08T08:54:21.808 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:21.811+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:21.822 INFO:teuthology.orchestra.run.smithi196.stdout:truncating image image_client.0-clone44 from 0x9f42800 (overlap 0x28b200) to 0x208e00 2024-05-08T08:54:21.825 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:21.825 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:54:21.825 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:54:21.825 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:54:21.825 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:21.835 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:54:21.910 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:21.912+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:21.988 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:21.988 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:54:21.988 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:54:21.988 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:54:21.988 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.988 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:437 3.17 3:ea93a512:::rbd_data.1140100c9703.0000000000000077:head [set-alloc-hint object_size 2097152 write_size 2097152,write 1170432~1024] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e63)", 2024-05-08T08:54:21.988 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:08.390804+0000", 2024-05-08T08:54:21.988 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 73.599118705999999, 2024-05-08T08:54:21.989 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0406068690000001, 2024-05-08T08:54:21.989 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:21.989 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:21.989 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:54:21.989 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:21.989 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:21.989 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:21.989 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 437 2024-05-08T08:54:21.989 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.989 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:21.989 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.989 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:21.989 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390804+0000", 2024-05-08T08:54:21.989 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:21.990 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.990 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.990 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:21.990 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390804+0000", 2024-05-08T08:54:21.990 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:21.990 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.990 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.990 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:21.990 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390806+0000", 2024-05-08T08:54:21.990 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.666e-06 2024-05-08T08:54:21.990 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.990 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.990 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:21.991 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390818+0000", 2024-05-08T08:54:21.991 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2187999999999999e-05 2024-05-08T08:54:21.991 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.991 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.991 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:21.991 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390834+0000", 2024-05-08T08:54:21.991 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6191000000000001e-05 2024-05-08T08:54:21.991 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.991 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.991 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:21.991 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390845+0000", 2024-05-08T08:54:21.991 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0831e-05 2024-05-08T08:54:21.991 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.991 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.991 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:21.992 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390922+0000", 2024-05-08T08:54:21.992 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7253e-05 2024-05-08T08:54:21.992 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.992 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.992 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op must wait for map", 2024-05-08T08:54:21.992 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390930+0000", 2024-05-08T08:54:21.992 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8259999999999993e-06 2024-05-08T08:54:21.992 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.992 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.992 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:21.992 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392155+0000", 2024-05-08T08:54:21.992 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0012256649999999999 2024-05-08T08:54:21.992 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.992 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.993 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:54:21.993 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392165+0000", 2024-05-08T08:54:21.993 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5799999999999998e-06 2024-05-08T08:54:21.993 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.993 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.993 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:21.993 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.429596+0000", 2024-05-08T08:54:21.993 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0374313639999999 2024-05-08T08:54:21.993 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.993 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.993 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:21.993 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.429728+0000", 2024-05-08T08:54:21.993 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000131552 2024-05-08T08:54:21.993 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.993 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.993 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 0,1", 2024-05-08T08:54:21.994 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.429860+0000", 2024-05-08T08:54:21.994 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000132137 2024-05-08T08:54:21.994 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.994 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.994 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:54:21.994 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.430811+0000", 2024-05-08T08:54:21.994 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00095040200000000004 2024-05-08T08:54:21.994 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.994 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.994 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:54:21.994 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431126+0000", 2024-05-08T08:54:21.994 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000315006 2024-05-08T08:54:21.994 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.995 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.995 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:54:21.995 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431272+0000", 2024-05-08T08:54:21.995 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000146487 2024-05-08T08:54:21.995 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.995 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.995 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:54:21.995 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431315+0000", 2024-05-08T08:54:21.995 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2697000000000001e-05 2024-05-08T08:54:21.995 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.995 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.995 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:21.995 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431411+0000", 2024-05-08T08:54:21.995 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6024000000000001e-05 2024-05-08T08:54:21.995 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:21.996 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:21.996 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:21.996 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.996 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.996 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:439 3.17 3:ea93a512:::rbd_data.1140100c9703.0000000000000077:head [read 1163264~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-05-08T08:54:21.996 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:08.390871+0000", 2024-05-08T08:54:21.996 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 73.599051195000001, 2024-05-08T08:54:21.996 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0408790590000001, 2024-05-08T08:54:21.996 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:21.996 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:21.996 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:21.996 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:21.996 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:21.996 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:21.997 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 439 2024-05-08T08:54:21.997 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.997 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:21.997 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.997 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:21.997 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390871+0000", 2024-05-08T08:54:21.997 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:21.997 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.997 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.997 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:21.997 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390871+0000", 2024-05-08T08:54:21.997 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:21.997 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.997 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.998 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:21.998 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390872+0000", 2024-05-08T08:54:21.998 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6300000000000005e-07 2024-05-08T08:54:21.998 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.998 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.998 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:21.998 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390875+0000", 2024-05-08T08:54:21.998 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3459999999999998e-06 2024-05-08T08:54:21.998 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.998 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.998 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:21.998 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390891+0000", 2024-05-08T08:54:21.998 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5255e-05 2024-05-08T08:54:21.998 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.999 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.999 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:21.999 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390897+0000", 2024-05-08T08:54:21.999 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2360000000000001e-06 2024-05-08T08:54:21.999 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.999 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.999 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:21.999 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390995+0000", 2024-05-08T08:54:21.999 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8522000000000003e-05 2024-05-08T08:54:21.999 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:21.999 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:21.999 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting_for_map not empty", 2024-05-08T08:54:21.999 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.391005+0000", 2024-05-08T08:54:21.999 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1460000000000002e-06 2024-05-08T08:54:22.000 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.000 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.000 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:22.000 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392191+0000", 2024-05-08T08:54:22.000 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0011860270000000001 2024-05-08T08:54:22.000 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.000 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.000 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:54:22.000 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392197+0000", 2024-05-08T08:54:22.000 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4289999999999998e-06 2024-05-08T08:54:22.000 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.000 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.000 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:22.001 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.430596+0000", 2024-05-08T08:54:22.001 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0383989090000001 2024-05-08T08:54:22.001 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.001 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.001 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for rw locks", 2024-05-08T08:54:22.001 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.430676+0000", 2024-05-08T08:54:22.001 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9781000000000001e-05 2024-05-08T08:54:22.001 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.001 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.001 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:22.001 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431460+0000", 2024-05-08T08:54:22.001 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00078427300000000004 2024-05-08T08:54:22.001 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.001 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.002 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:22.002 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431540+0000", 2024-05-08T08:54:22.002 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0224999999999998e-05 2024-05-08T08:54:22.002 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.002 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.002 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:22.002 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431750+0000", 2024-05-08T08:54:22.002 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00021024699999999999 2024-05-08T08:54:22.002 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:22.002 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:22.003 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:22.003 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.003 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.003 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:945 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 4096000~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:22.003 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:54:22.003 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 72.538901628000005, 2024-05-08T08:54:22.003 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086774741000000002, 2024-05-08T08:54:22.003 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:22.003 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:22.003 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:22.003 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:22.003 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:22.003 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:22.003 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 945 2024-05-08T08:54:22.004 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.004 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:22.004 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.004 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:22.004 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:54:22.004 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:22.004 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.004 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.004 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:22.004 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:54:22.004 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:22.004 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.004 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.005 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:22.005 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:54:22.005 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3499999999999996e-07 2024-05-08T08:54:22.005 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.005 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.005 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:22.005 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451024+0000", 2024-05-08T08:54:22.005 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.627e-06 2024-05-08T08:54:22.005 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.005 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.005 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:22.005 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451037+0000", 2024-05-08T08:54:22.006 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2431e-05 2024-05-08T08:54:22.006 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.006 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.006 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:22.006 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451042+0000", 2024-05-08T08:54:22.006 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8409999999999996e-06 2024-05-08T08:54:22.006 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.006 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.006 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:22.006 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537608+0000", 2024-05-08T08:54:22.006 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086565243 2024-05-08T08:54:22.006 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.006 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.007 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:22.007 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537687+0000", 2024-05-08T08:54:22.007 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9519999999999998e-05 2024-05-08T08:54:22.007 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.007 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.007 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:22.007 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537796+0000", 2024-05-08T08:54:22.007 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000108544 2024-05-08T08:54:22.007 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:22.007 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:22.007 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:22.007 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.007 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.007 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:964 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 458752~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:22.008 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451051+0000", 2024-05-08T08:54:22.008 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 72.538871455000006, 2024-05-08T08:54:22.008 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086990490000000004, 2024-05-08T08:54:22.008 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:22.008 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:22.008 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:22.008 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:22.008 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:22.008 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:22.008 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 964 2024-05-08T08:54:22.008 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.008 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:22.009 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.009 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:22.009 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451051+0000", 2024-05-08T08:54:22.009 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:22.009 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.009 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.009 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:22.009 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451051+0000", 2024-05-08T08:54:22.009 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:22.009 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.009 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.009 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:22.009 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451052+0000", 2024-05-08T08:54:22.009 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7299999999999996e-07 2024-05-08T08:54:22.009 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.009 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.010 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:22.010 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451055+0000", 2024-05-08T08:54:22.010 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.2770000000000001e-06 2024-05-08T08:54:22.010 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.010 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.010 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:22.010 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451061+0000", 2024-05-08T08:54:22.010 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4060000000000002e-06 2024-05-08T08:54:22.010 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.010 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.010 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:22.010 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451066+0000", 2024-05-08T08:54:22.010 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0839999999999999e-06 2024-05-08T08:54:22.011 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.011 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.011 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:22.011 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537839+0000", 2024-05-08T08:54:22.011 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086772962999999995 2024-05-08T08:54:22.011 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.011 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.011 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:22.011 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537934+0000", 2024-05-08T08:54:22.011 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4742999999999997e-05 2024-05-08T08:54:22.011 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.011 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.011 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:22.011 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538042+0000", 2024-05-08T08:54:22.011 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000107444 2024-05-08T08:54:22.011 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:22.012 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:22.012 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:22.012 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.012 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.012 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:965 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 950272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:22.012 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451075+0000", 2024-05-08T08:54:22.012 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 72.538847392999998, 2024-05-08T08:54:22.012 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087230196999999995, 2024-05-08T08:54:22.012 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:22.012 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:22.012 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:22.012 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:22.012 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:22.012 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:22.013 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 965 2024-05-08T08:54:22.013 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.013 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:22.013 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.013 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:22.013 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451075+0000", 2024-05-08T08:54:22.013 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:22.013 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.013 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.013 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:22.013 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451075+0000", 2024-05-08T08:54:22.013 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:22.013 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.014 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.014 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:22.014 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451076+0000", 2024-05-08T08:54:22.014 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5000000000000002e-07 2024-05-08T08:54:22.014 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.014 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.014 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:22.014 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451078+0000", 2024-05-08T08:54:22.014 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.5639999999999999e-06 2024-05-08T08:54:22.014 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.014 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.014 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:22.014 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451085+0000", 2024-05-08T08:54:22.015 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5640000000000002e-06 2024-05-08T08:54:22.015 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.015 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.015 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:22.015 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451090+0000", 2024-05-08T08:54:22.015 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1699999999999996e-06 2024-05-08T08:54:22.015 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.015 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.015 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:22.015 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538106+0000", 2024-05-08T08:54:22.015 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087015888999999999 2024-05-08T08:54:22.015 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.015 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.015 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:22.016 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538190+0000", 2024-05-08T08:54:22.016 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3961999999999993e-05 2024-05-08T08:54:22.016 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.016 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.016 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:22.016 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538305+0000", 2024-05-08T08:54:22.016 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011529800000000001 2024-05-08T08:54:22.016 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:22.016 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:22.016 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:22.016 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.016 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.016 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:966 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 1441792~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:22.016 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:54:22.016 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 72.538824020000007, 2024-05-08T08:54:22.017 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087446323000000006, 2024-05-08T08:54:22.017 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:22.017 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:22.017 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:22.017 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:22.017 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:22.017 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:22.017 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 966 2024-05-08T08:54:22.017 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.017 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:22.017 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.017 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:22.017 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:54:22.017 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:22.017 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.018 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.018 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:22.018 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:54:22.018 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:22.018 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.018 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.018 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:22.018 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:54:22.018 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3700000000000001e-07 2024-05-08T08:54:22.018 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.018 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.018 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:22.018 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451101+0000", 2024-05-08T08:54:22.019 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2560000000000001e-06 2024-05-08T08:54:22.019 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.019 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.019 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:22.019 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451107+0000", 2024-05-08T08:54:22.019 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1070000000000001e-06 2024-05-08T08:54:22.019 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.019 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.019 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:22.019 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451112+0000", 2024-05-08T08:54:22.019 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1420000000000001e-06 2024-05-08T08:54:22.019 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.019 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.019 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:22.020 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538341+0000", 2024-05-08T08:54:22.020 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087228302999999993 2024-05-08T08:54:22.020 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.020 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.020 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:22.020 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538424+0000", 2024-05-08T08:54:22.020 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3497000000000004e-05 2024-05-08T08:54:22.020 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.020 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.020 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:22.020 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538545+0000", 2024-05-08T08:54:22.020 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000120581 2024-05-08T08:54:22.020 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:22.020 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:22.020 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:22.020 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.021 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.021 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:967 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:22.021 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451120+0000", 2024-05-08T08:54:22.021 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 72.538802544999996, 2024-05-08T08:54:22.021 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087632725999999994, 2024-05-08T08:54:22.021 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:22.021 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:22.021 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:22.021 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:22.021 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:22.021 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:22.021 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 967 2024-05-08T08:54:22.021 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.021 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:22.022 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.022 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:22.022 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451120+0000", 2024-05-08T08:54:22.022 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:22.022 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.022 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.022 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:22.022 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451120+0000", 2024-05-08T08:54:22.022 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:22.022 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.022 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.022 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:22.022 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451121+0000", 2024-05-08T08:54:22.023 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8800000000000002e-07 2024-05-08T08:54:22.023 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.023 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.023 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:22.023 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451130+0000", 2024-05-08T08:54:22.023 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2299999999999997e-06 2024-05-08T08:54:22.023 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.023 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.023 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:22.023 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451137+0000", 2024-05-08T08:54:22.023 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4669999999999998e-06 2024-05-08T08:54:22.023 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.023 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.023 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:22.024 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451143+0000", 2024-05-08T08:54:22.024 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2839999999999997e-06 2024-05-08T08:54:22.024 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.024 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.024 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:22.024 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538575+0000", 2024-05-08T08:54:22.024 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087432789999999996 2024-05-08T08:54:22.024 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.024 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.024 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:22.024 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538648+0000", 2024-05-08T08:54:22.024 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2877e-05 2024-05-08T08:54:22.024 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.024 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.025 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:22.025 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538753+0000", 2024-05-08T08:54:22.025 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010449 2024-05-08T08:54:22.025 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:22.025 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:22.025 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:22.025 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.025 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.025 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:968 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 2424832~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:22.026 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451152+0000", 2024-05-08T08:54:22.026 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 72.538770459999995, 2024-05-08T08:54:22.026 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087813325999999997, 2024-05-08T08:54:22.026 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:22.026 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:22.026 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:22.026 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:22.026 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:22.026 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:22.026 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 968 2024-05-08T08:54:22.026 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.027 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:22.027 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.027 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:22.027 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451152+0000", 2024-05-08T08:54:22.027 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:22.027 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.027 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.027 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:22.027 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451152+0000", 2024-05-08T08:54:22.027 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:22.027 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.027 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.027 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:22.027 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451153+0000", 2024-05-08T08:54:22.028 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3099999999999998e-07 2024-05-08T08:54:22.028 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.028 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.028 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:22.028 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451155+0000", 2024-05-08T08:54:22.028 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4760000000000001e-06 2024-05-08T08:54:22.028 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.028 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.028 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:22.028 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451162+0000", 2024-05-08T08:54:22.028 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5969999999999997e-06 2024-05-08T08:54:22.028 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.028 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.028 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:22.028 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451167+0000", 2024-05-08T08:54:22.028 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.977e-06 2024-05-08T08:54:22.028 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.029 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.029 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:22.029 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538781+0000", 2024-05-08T08:54:22.029 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087614846999999996 2024-05-08T08:54:22.029 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.029 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.029 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:22.029 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538857+0000", 2024-05-08T08:54:22.029 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5358000000000006e-05 2024-05-08T08:54:22.029 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.029 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.029 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:22.029 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538965+0000", 2024-05-08T08:54:22.029 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010864 2024-05-08T08:54:22.030 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:22.030 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:22.030 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:22.030 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.030 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.030 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:969 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 2916352~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:22.030 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:54:22.030 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 72.538748755, 2024-05-08T08:54:22.030 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087998980000000004, 2024-05-08T08:54:22.030 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:22.030 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:22.030 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:22.030 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:22.030 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:22.031 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:22.031 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 969 2024-05-08T08:54:22.031 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.031 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:22.031 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.031 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:22.031 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:54:22.031 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:22.031 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.031 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.031 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:22.031 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:54:22.031 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:22.031 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.032 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.032 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:22.032 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:54:22.032 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3599999999999999e-07 2024-05-08T08:54:22.032 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.032 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.032 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:22.032 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451177+0000", 2024-05-08T08:54:22.032 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4590000000000001e-06 2024-05-08T08:54:22.032 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.032 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.032 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:22.032 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451184+0000", 2024-05-08T08:54:22.032 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4109999999999999e-06 2024-05-08T08:54:22.032 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.032 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.033 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:22.033 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451191+0000", 2024-05-08T08:54:22.033 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7279999999999998e-06 2024-05-08T08:54:22.033 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.033 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.033 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:22.033 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538997+0000", 2024-05-08T08:54:22.033 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087806170000000003 2024-05-08T08:54:22.033 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.033 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.033 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:22.033 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539069+0000", 2024-05-08T08:54:22.033 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2373999999999995e-05 2024-05-08T08:54:22.033 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.033 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.033 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:22.034 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539173+0000", 2024-05-08T08:54:22.034 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000103402 2024-05-08T08:54:22.034 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:22.034 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:22.034 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:22.034 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.034 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.034 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:970 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 3407872~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:22.034 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451198+0000", 2024-05-08T08:54:22.034 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 72.538724157999994, 2024-05-08T08:54:22.034 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088183368999999998, 2024-05-08T08:54:22.034 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:22.034 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:22.035 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:22.035 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:22.035 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:22.035 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:22.035 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 970 2024-05-08T08:54:22.035 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.035 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:22.035 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.035 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:22.035 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451198+0000", 2024-05-08T08:54:22.035 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:22.035 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.035 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.035 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:22.035 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451198+0000", 2024-05-08T08:54:22.036 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:22.036 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.036 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.036 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:22.036 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451199+0000", 2024-05-08T08:54:22.036 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7400000000000003e-07 2024-05-08T08:54:22.036 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.036 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.036 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:22.036 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451211+0000", 2024-05-08T08:54:22.036 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2109999999999999e-05 2024-05-08T08:54:22.036 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.037 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.037 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:22.037 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451219+0000", 2024-05-08T08:54:22.037 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4170000000000003e-06 2024-05-08T08:54:22.037 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.037 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.037 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:22.037 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451224+0000", 2024-05-08T08:54:22.037 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3990000000000003e-06 2024-05-08T08:54:22.037 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.037 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.037 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:22.037 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539204+0000", 2024-05-08T08:54:22.037 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087980099000000006 2024-05-08T08:54:22.037 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.037 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.038 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:22.038 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539276+0000", 2024-05-08T08:54:22.038 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1513000000000005e-05 2024-05-08T08:54:22.038 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.038 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.038 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:22.038 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539382+0000", 2024-05-08T08:54:22.038 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010615699999999999 2024-05-08T08:54:22.038 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:22.038 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:22.038 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:22.038 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.038 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.038 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:971 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 3899392~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:22.038 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451233+0000", 2024-05-08T08:54:22.039 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 72.538689450999996, 2024-05-08T08:54:22.039 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088431525999999996, 2024-05-08T08:54:22.039 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:22.039 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:22.039 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:22.039 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:22.039 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:22.039 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:22.039 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 971 2024-05-08T08:54:22.039 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.039 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:22.039 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.039 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:22.039 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451233+0000", 2024-05-08T08:54:22.039 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:22.040 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.040 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.040 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:22.040 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451233+0000", 2024-05-08T08:54:22.040 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:22.040 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.040 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.040 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:22.040 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451234+0000", 2024-05-08T08:54:22.040 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8500000000000001e-07 2024-05-08T08:54:22.040 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.040 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.040 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:22.040 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451237+0000", 2024-05-08T08:54:22.040 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8320000000000002e-06 2024-05-08T08:54:22.040 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.041 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.041 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:22.041 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451243+0000", 2024-05-08T08:54:22.041 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6749999999999996e-06 2024-05-08T08:54:22.041 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.041 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.041 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:22.041 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451249+0000", 2024-05-08T08:54:22.041 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1229999999999997e-06 2024-05-08T08:54:22.041 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.041 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.041 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:22.041 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539412+0000", 2024-05-08T08:54:22.041 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088162668999999999 2024-05-08T08:54:22.041 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.041 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.042 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:22.042 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539505+0000", 2024-05-08T08:54:22.042 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3240999999999997e-05 2024-05-08T08:54:22.042 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.042 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.042 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:22.042 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539665+0000", 2024-05-08T08:54:22.042 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00015930100000000001 2024-05-08T08:54:22.042 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:22.042 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:22.042 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:22.042 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.042 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.043 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:972 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 196608~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:22.043 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451257+0000", 2024-05-08T08:54:22.043 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 72.538665506000001, 2024-05-08T08:54:22.043 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088719928000000003, 2024-05-08T08:54:22.043 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:22.043 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:22.043 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:22.043 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:22.043 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:22.043 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:22.043 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 972 2024-05-08T08:54:22.043 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.043 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:22.044 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.044 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:22.044 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451257+0000", 2024-05-08T08:54:22.044 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:22.044 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.044 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.044 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:22.044 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451257+0000", 2024-05-08T08:54:22.044 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:22.044 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.044 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.044 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:22.044 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451258+0000", 2024-05-08T08:54:22.044 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7999999999999996e-07 2024-05-08T08:54:22.045 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.045 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.045 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:22.045 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451260+0000", 2024-05-08T08:54:22.045 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4609999999999998e-06 2024-05-08T08:54:22.045 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.045 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.045 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:22.045 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451267+0000", 2024-05-08T08:54:22.045 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.81e-06 2024-05-08T08:54:22.045 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.045 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.045 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:22.045 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451273+0000", 2024-05-08T08:54:22.045 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0859999999999998e-06 2024-05-08T08:54:22.046 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.046 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.046 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:22.046 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539718+0000", 2024-05-08T08:54:22.046 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088445059000000006 2024-05-08T08:54:22.046 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.046 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.046 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:22.046 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539834+0000", 2024-05-08T08:54:22.046 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000116426 2024-05-08T08:54:22.046 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.046 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.046 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:22.046 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539977+0000", 2024-05-08T08:54:22.047 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014260600000000001 2024-05-08T08:54:22.047 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:22.047 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:22.047 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:22.047 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.047 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.047 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:973 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 688128~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:22.047 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:54:22.047 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 72.538642921999994, 2024-05-08T08:54:22.047 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088918496, 2024-05-08T08:54:22.047 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:22.047 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:22.047 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:22.048 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:22.048 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:22.048 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:22.048 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 973 2024-05-08T08:54:22.048 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.048 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:22.048 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.048 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:22.048 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:54:22.048 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:22.048 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.048 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.048 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:22.049 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:54:22.049 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:22.049 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.049 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.049 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:22.049 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:54:22.049 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9099999999999999e-07 2024-05-08T08:54:22.049 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.049 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.049 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:22.049 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451282+0000", 2024-05-08T08:54:22.049 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1550000000000001e-06 2024-05-08T08:54:22.049 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.049 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.049 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:22.050 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451288+0000", 2024-05-08T08:54:22.050 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8889999999999999e-06 2024-05-08T08:54:22.050 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.050 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.050 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:22.050 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451294+0000", 2024-05-08T08:54:22.050 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0850000000000002e-06 2024-05-08T08:54:22.050 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.050 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.050 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:22.050 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540011+0000", 2024-05-08T08:54:22.050 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088717314000000005 2024-05-08T08:54:22.050 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.050 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.051 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:22.051 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540089+0000", 2024-05-08T08:54:22.051 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7438e-05 2024-05-08T08:54:22.051 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.051 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.051 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:22.051 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540198+0000", 2024-05-08T08:54:22.051 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000109224 2024-05-08T08:54:22.051 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:22.051 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:22.051 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:22.051 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.051 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.051 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:974 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 1179648~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:22.051 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451301+0000", 2024-05-08T08:54:22.052 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 72.538621289000005, 2024-05-08T08:54:22.052 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089105078000000004, 2024-05-08T08:54:22.052 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:22.052 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:22.052 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:22.052 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:22.052 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:22.052 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:22.052 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 974 2024-05-08T08:54:22.052 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.052 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:22.052 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.052 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:22.052 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451301+0000", 2024-05-08T08:54:22.052 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:22.053 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.053 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.053 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:22.053 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451301+0000", 2024-05-08T08:54:22.053 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:22.053 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.053 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.053 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:22.053 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451302+0000", 2024-05-08T08:54:22.053 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7800000000000002e-07 2024-05-08T08:54:22.053 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.053 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.053 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:22.053 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451304+0000", 2024-05-08T08:54:22.054 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.65e-06 2024-05-08T08:54:22.054 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.054 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.054 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:22.054 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451311+0000", 2024-05-08T08:54:22.054 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.472e-06 2024-05-08T08:54:22.054 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.054 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.054 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:22.054 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451316+0000", 2024-05-08T08:54:22.054 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9189999999999998e-06 2024-05-08T08:54:22.054 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.054 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.054 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:22.055 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540234+0000", 2024-05-08T08:54:22.055 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088917752000000003 2024-05-08T08:54:22.055 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.055 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.055 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:22.055 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540308+0000", 2024-05-08T08:54:22.055 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4758000000000005e-05 2024-05-08T08:54:22.055 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.055 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.055 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:22.055 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540406+0000", 2024-05-08T08:54:22.055 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8048999999999996e-05 2024-05-08T08:54:22.055 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:22.055 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:22.056 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:22.056 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.056 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.056 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:975 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 1671168~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:22.056 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:54:22.056 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 72.538578959000006, 2024-05-08T08:54:22.056 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089294139999999994, 2024-05-08T08:54:22.056 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:22.056 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:22.056 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:22.056 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:22.056 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:22.056 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:22.056 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 975 2024-05-08T08:54:22.057 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.057 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:22.057 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.057 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:22.057 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:54:22.057 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:22.057 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.057 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.057 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:22.057 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:54:22.057 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:22.057 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.057 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.057 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:22.057 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:54:22.058 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1299999999999999e-07 2024-05-08T08:54:22.058 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.058 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.058 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:22.058 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451348+0000", 2024-05-08T08:54:22.058 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.484e-06 2024-05-08T08:54:22.058 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.058 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.058 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:22.058 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451356+0000", 2024-05-08T08:54:22.058 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8499999999999994e-06 2024-05-08T08:54:22.058 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.058 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.058 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:22.059 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451366+0000", 2024-05-08T08:54:22.059 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.9850000000000001e-06 2024-05-08T08:54:22.059 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.059 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.059 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:22.059 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540448+0000", 2024-05-08T08:54:22.059 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089082254 2024-05-08T08:54:22.059 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.059 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.059 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:22.059 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540528+0000", 2024-05-08T08:54:22.059 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0234999999999993e-05 2024-05-08T08:54:22.059 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.059 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.060 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:22.060 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540638+0000", 2024-05-08T08:54:22.060 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010951900000000001 2024-05-08T08:54:22.060 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:22.060 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:22.060 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:22.060 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.060 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.060 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:976 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 2162688~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:22.060 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451375+0000", 2024-05-08T08:54:22.060 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 72.538547211999997, 2024-05-08T08:54:22.060 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089472745000000006, 2024-05-08T08:54:22.060 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:22.060 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:22.060 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:22.061 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:22.061 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:22.061 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:22.061 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 976 2024-05-08T08:54:22.061 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.061 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:22.061 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.061 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:22.061 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451375+0000", 2024-05-08T08:54:22.061 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:22.061 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.061 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.061 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:22.061 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451375+0000", 2024-05-08T08:54:22.062 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:22.062 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.062 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.062 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:22.062 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451376+0000", 2024-05-08T08:54:22.062 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2399999999999998e-07 2024-05-08T08:54:22.062 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.062 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.062 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:22.062 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451379+0000", 2024-05-08T08:54:22.062 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6359999999999998e-06 2024-05-08T08:54:22.062 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.062 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.062 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:22.063 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451387+0000", 2024-05-08T08:54:22.063 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9929999999999995e-06 2024-05-08T08:54:22.063 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.063 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.063 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:22.063 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451393+0000", 2024-05-08T08:54:22.063 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4699999999999999e-06 2024-05-08T08:54:22.063 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.063 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.063 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:22.063 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540677+0000", 2024-05-08T08:54:22.063 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089283504999999999 2024-05-08T08:54:22.063 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.063 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.064 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:22.064 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540743+0000", 2024-05-08T08:54:22.064 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6303999999999994e-05 2024-05-08T08:54:22.064 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.064 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.064 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:22.064 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540848+0000", 2024-05-08T08:54:22.064 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000105213 2024-05-08T08:54:22.064 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:22.064 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:22.064 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:22.064 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.064 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.064 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:977 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 2654208~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:22.065 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:54:22.065 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 72.538521779999996, 2024-05-08T08:54:22.065 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089698473000000001, 2024-05-08T08:54:22.065 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:22.065 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:22.065 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:22.065 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:22.065 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:22.065 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:22.065 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 977 2024-05-08T08:54:22.065 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.065 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:22.065 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.065 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:22.065 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:54:22.066 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:22.066 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.066 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.066 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:22.066 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:54:22.066 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:22.066 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.066 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.066 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:22.066 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:54:22.066 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2899999999999999e-07 2024-05-08T08:54:22.066 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.066 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.066 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:22.067 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451404+0000", 2024-05-08T08:54:22.067 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6649999999999999e-06 2024-05-08T08:54:22.067 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.067 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.067 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:22.067 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451410+0000", 2024-05-08T08:54:22.067 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3269999999999997e-06 2024-05-08T08:54:22.067 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.067 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.067 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:22.067 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451415+0000", 2024-05-08T08:54:22.067 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9529999999999998e-06 2024-05-08T08:54:22.067 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.067 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.067 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:22.067 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540912+0000", 2024-05-08T08:54:22.068 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089496950000000006 2024-05-08T08:54:22.068 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.068 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.068 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:22.068 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541000+0000", 2024-05-08T08:54:22.068 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7429000000000006e-05 2024-05-08T08:54:22.068 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.068 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.068 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:22.068 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541099+0000", 2024-05-08T08:54:22.068 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.9720000000000001e-05 2024-05-08T08:54:22.068 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:22.068 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:22.068 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:22.068 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.068 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.069 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:978 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 3145728~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:22.069 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:54:22.069 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 72.538500572000004, 2024-05-08T08:54:22.069 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089900635000000007, 2024-05-08T08:54:22.069 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:22.069 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:22.069 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:22.069 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:22.069 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:22.069 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:22.069 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 978 2024-05-08T08:54:22.069 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.069 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:22.069 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.069 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:22.070 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:54:22.070 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:22.070 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.070 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.070 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:22.070 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:54:22.070 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:22.070 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.070 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.070 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:22.070 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:54:22.070 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9400000000000001e-07 2024-05-08T08:54:22.070 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.070 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.070 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:22.071 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451425+0000", 2024-05-08T08:54:22.071 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2460000000000002e-06 2024-05-08T08:54:22.071 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.071 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.071 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:22.071 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451431+0000", 2024-05-08T08:54:22.071 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2419999999999997e-06 2024-05-08T08:54:22.071 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.071 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.071 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:22.071 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451436+0000", 2024-05-08T08:54:22.071 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.887e-06 2024-05-08T08:54:22.071 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.071 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.071 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:22.071 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541152+0000", 2024-05-08T08:54:22.072 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089716194999999999 2024-05-08T08:54:22.072 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.072 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.072 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:22.072 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541224+0000", 2024-05-08T08:54:22.072 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2416000000000006e-05 2024-05-08T08:54:22.072 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.072 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.072 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:22.072 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541323+0000", 2024-05-08T08:54:22.072 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8255e-05 2024-05-08T08:54:22.072 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:22.072 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:22.072 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:22.072 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.073 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.073 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:979 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 3637248~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:22.073 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451462+0000", 2024-05-08T08:54:22.073 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 72.538460122999993, 2024-05-08T08:54:22.073 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.090036441999999994, 2024-05-08T08:54:22.073 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:22.073 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:22.073 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:22.073 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:22.073 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:22.073 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:22.073 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 979 2024-05-08T08:54:22.073 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.073 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:22.073 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.074 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:22.074 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451462+0000", 2024-05-08T08:54:22.074 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:22.074 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.074 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.074 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:22.074 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451462+0000", 2024-05-08T08:54:22.074 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:22.074 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.074 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.074 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:22.074 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451463+0000", 2024-05-08T08:54:22.074 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4300000000000002e-07 2024-05-08T08:54:22.075 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.075 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.075 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:22.075 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451471+0000", 2024-05-08T08:54:22.075 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0800000000000006e-06 2024-05-08T08:54:22.075 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.075 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.075 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:22.075 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451481+0000", 2024-05-08T08:54:22.075 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.38e-06 2024-05-08T08:54:22.075 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.075 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.075 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:22.075 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451487+0000", 2024-05-08T08:54:22.076 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4590000000000003e-06 2024-05-08T08:54:22.076 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.076 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.076 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:22.076 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541350+0000", 2024-05-08T08:54:22.076 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089862599000000001 2024-05-08T08:54:22.076 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.076 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.076 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:22.076 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541407+0000", 2024-05-08T08:54:22.076 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7667999999999997e-05 2024-05-08T08:54:22.076 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.076 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.077 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:22.077 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541499+0000", 2024-05-08T08:54:22.077 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1513000000000003e-05 2024-05-08T08:54:22.077 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:22.077 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:22.077 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:22.077 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.077 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.077 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:980 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 4128768~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:22.077 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451496+0000", 2024-05-08T08:54:22.077 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 72.538426367, 2024-05-08T08:54:22.077 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.090258985, 2024-05-08T08:54:22.077 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:22.077 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:22.078 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:22.078 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:22.078 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:22.078 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:22.078 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 980 2024-05-08T08:54:22.078 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.078 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:22.078 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.078 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:22.078 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451496+0000", 2024-05-08T08:54:22.078 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:22.078 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.078 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.079 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:22.079 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451496+0000", 2024-05-08T08:54:22.079 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:22.079 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.079 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.079 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:22.079 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451497+0000", 2024-05-08T08:54:22.079 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7599999999999997e-07 2024-05-08T08:54:22.079 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.079 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.079 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:22.079 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451500+0000", 2024-05-08T08:54:22.079 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7930000000000002e-06 2024-05-08T08:54:22.079 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.079 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.079 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:22.080 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451506+0000", 2024-05-08T08:54:22.080 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6980000000000001e-06 2024-05-08T08:54:22.080 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.080 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.080 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:22.080 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451512+0000", 2024-05-08T08:54:22.080 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2490000000000001e-06 2024-05-08T08:54:22.080 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.080 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.080 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:22.080 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541550+0000", 2024-05-08T08:54:22.080 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.090037961999999999 2024-05-08T08:54:22.080 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.080 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.080 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:22.081 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541647+0000", 2024-05-08T08:54:22.081 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6924999999999998e-05 2024-05-08T08:54:22.081 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:22.081 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:22.081 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:22.081 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541755+0000", 2024-05-08T08:54:22.081 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000108682 2024-05-08T08:54:22.081 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:22.081 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:22.081 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:22.081 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:22.081 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:22.081 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:22.083 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:54:22.083 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:22.085+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:22.170 INFO:teuthology.orchestra.run.smithi196.stdout:checking clone #43, image image_client.0-clone43 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent44 2024-05-08T08:54:22.183 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:22.186+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:22.194 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:22.194 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:54:22.195 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:54:22.195 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:22.203 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:54:22.238 INFO:teuthology.orchestra.run.smithi196.stdout:5681 read 0x126da00 thru 0x126db96 (0x197 bytes) 2024-05-08T08:54:22.239 INFO:teuthology.orchestra.run.smithi196.stdout:5682 read 0x3a35000 thru 0x3a40caf (0xbcb0 bytes) 2024-05-08T08:54:22.240 INFO:teuthology.orchestra.run.smithi196.stdout:5683 punch from 0x3c58c00 to 0x3c66a00, (0xde00 bytes) 2024-05-08T08:54:22.257 INFO:teuthology.orchestra.run.smithi196.stdout:5685 trunc from 0x9f42800 to 0x3853e00 2024-05-08T08:54:22.284 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:22.287+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:22.305 INFO:teuthology.orchestra.run.smithi196.stdout:5688 flatten 2024-05-08T08:54:22.317 INFO:teuthology.orchestra.run.smithi196.stdout:5690 read 0x2c1400 thru 0x2c887a (0x747b bytes) 2024-05-08T08:54:22.321 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:22.321 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:54:22.321 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:54:22.321 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:54:22.321 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:22.330 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:54:22.343 INFO:teuthology.orchestra.run.smithi196.stdout:5693 write 0xea26c00 thru 0xea2f8a3 (0x8ca4 bytes) 2024-05-08T08:54:22.344 INFO:teuthology.orchestra.run.smithi196.stdout:5694 punch from 0xdecce00 to 0xdecf400, (0x2600 bytes) 2024-05-08T08:54:22.344 INFO:teuthology.orchestra.run.smithi196.stdout:5695 read 0x674a800 thru 0x6756d1e (0xc51f bytes) 2024-05-08T08:54:22.345 INFO:teuthology.orchestra.run.smithi196.stdout:5696 punch from 0x6874000 to 0x6875600, (0x1600 bytes) 2024-05-08T08:54:22.345 INFO:teuthology.orchestra.run.smithi196.stdout:5697 punch from 0xdfd7600 to 0xdfe5600, (0xe000 bytes) 2024-05-08T08:54:22.345 INFO:teuthology.orchestra.run.smithi196.stdout:5699 punch from 0x7673600 to 0x7673800, (0x200 bytes) 2024-05-08T08:54:22.345 INFO:teuthology.orchestra.run.smithi196.stdout:5700 read 0xa1b4a00 thru 0xa1bb30e (0x690f bytes) 2024-05-08T08:54:22.345 INFO:teuthology.orchestra.run.smithi196.stdout:5703 write 0xdbad600 thru 0xdbbba40 (0xe441 bytes) 2024-05-08T08:54:22.346 INFO:teuthology.orchestra.run.smithi196.stdout:5704 trunc from 0xea2fa00 to 0xdc9fe00 2024-05-08T08:54:22.378 INFO:teuthology.orchestra.run.smithi196.stdout:5706 trunc from 0xdc9fe00 to 0x410da00 2024-05-08T08:54:22.385 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:22.388+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:22.403 INFO:teuthology.orchestra.run.smithi196.stdout:5707 read 0x3313000 thru 0x3319fad (0x6fae bytes) 2024-05-08T08:54:22.404 INFO:teuthology.orchestra.run.smithi196.stdout:5709 punch from 0x3e91c00 to 0x3e9b600, (0x9a00 bytes) 2024-05-08T08:54:22.415 INFO:teuthology.orchestra.run.smithi196.stdout:5715 write 0x59c2000 thru 0x59c37a3 (0x17a4 bytes) 2024-05-08T08:54:22.416 INFO:teuthology.orchestra.run.smithi196.stdout:5717 write 0x761a00 thru 0x763811 (0x1e12 bytes) 2024-05-08T08:54:22.416 INFO:teuthology.orchestra.run.smithi196.stdout:5718 punch from 0x5509000 to 0x550dc00, (0x4c00 bytes) 2024-05-08T08:54:22.417 INFO:teuthology.orchestra.run.smithi196.stdout:5721 read 0x2900000 thru 0x290851e (0x851f bytes) 2024-05-08T08:54:22.417 INFO:teuthology.orchestra.run.smithi196.stdout:5722 punch from 0x527b000 to 0x5280e00, (0x5e00 bytes) 2024-05-08T08:54:22.417 INFO:teuthology.orchestra.run.smithi196.stdout:5724 write 0x220d800 thru 0x22196bc (0xbebd bytes) 2024-05-08T08:54:22.418 INFO:teuthology.orchestra.run.smithi196.stdout:5725 write 0x38dcc00 thru 0x38e6559 (0x995a bytes) 2024-05-08T08:54:22.442 INFO:teuthology.orchestra.run.smithi196.stdout:5726 write 0xc6b3a00 thru 0xc6ba548 (0x6b49 bytes) 2024-05-08T08:54:22.443 INFO:teuthology.orchestra.run.smithi196.stdout:5727 write 0x3de7200 thru 0x3df124a (0xa04b bytes) 2024-05-08T08:54:22.444 INFO:teuthology.orchestra.run.smithi196.stdout:5731 write 0x4bf8200 thru 0x4c04f5c (0xcd5d bytes) 2024-05-08T08:54:22.451 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:22.451 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:54:22.451 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:54:22.451 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:54:22.451 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.451 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4621.0:9089 3.1f e33/18 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:head v 33'117, mlcod=31'115)", 2024-05-08T08:54:22.451 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:54:22.451 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 109.933334898, 2024-05-08T08:54:22.451 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.111244524, 2024-05-08T08:54:22.452 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:22.452 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:22.452 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:54:22.452 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:22.452 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.452 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:22.452 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:54:22.452 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:22.452 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.452 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.452 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:22.452 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:54:22.452 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:22.452 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.452 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.453 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:22.453 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:54:22.453 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5900000000000001e-07 2024-05-08T08:54:22.453 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.453 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.453 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:22.453 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539600+0000", 2024-05-08T08:54:22.453 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.019981364000000001 2024-05-08T08:54:22.453 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.453 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.453 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:22.453 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539624+0000", 2024-05-08T08:54:22.453 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4043e-05 2024-05-08T08:54:22.453 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.454 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.454 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:22.454 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539635+0000", 2024-05-08T08:54:22.454 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1511e-05 2024-05-08T08:54:22.454 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.454 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.454 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:22.454 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539728+0000", 2024-05-08T08:54:22.454 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2498999999999996e-05 2024-05-08T08:54:22.454 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.454 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.454 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:22.454 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539746+0000", 2024-05-08T08:54:22.455 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8289000000000001e-05 2024-05-08T08:54:22.455 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.455 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.455 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:54:22.455 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.630822+0000", 2024-05-08T08:54:22.455 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.091076283999999993 2024-05-08T08:54:22.455 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.455 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.455 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:22.455 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.630863+0000", 2024-05-08T08:54:22.455 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0374999999999999e-05 2024-05-08T08:54:22.455 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:22.455 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:22.455 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:22.456 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.456 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.456 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4795.0:230 3.1f e40/18 3:f8db1a56:::rbd_data.11402b26385.0000000000000000:head v 40'171, mlcod=39'168)", 2024-05-08T08:54:22.456 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:45.927773+0000", 2024-05-08T08:54:22.456 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 96.525179815000001, 2024-05-08T08:54:22.456 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045892405999999997, 2024-05-08T08:54:22.456 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:22.456 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:22.456 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:54:22.456 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:22.456 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.456 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:22.456 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.927773+0000", 2024-05-08T08:54:22.456 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:22.456 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.457 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.457 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:22.457 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.927773+0000", 2024-05-08T08:54:22.457 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:22.457 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.457 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.457 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:22.457 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.927774+0000", 2024-05-08T08:54:22.457 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0500000000000002e-07 2024-05-08T08:54:22.457 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.457 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.457 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:22.457 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.931581+0000", 2024-05-08T08:54:22.457 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0038071799999999999 2024-05-08T08:54:22.457 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.458 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.458 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:22.458 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.931599+0000", 2024-05-08T08:54:22.458 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7479e-05 2024-05-08T08:54:22.458 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.458 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.458 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:22.458 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.931624+0000", 2024-05-08T08:54:22.458 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5372999999999999e-05 2024-05-08T08:54:22.458 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.458 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.458 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:22.458 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.931698+0000", 2024-05-08T08:54:22.458 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4043999999999998e-05 2024-05-08T08:54:22.459 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.459 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.459 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:22.459 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.931725+0000", 2024-05-08T08:54:22.459 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6883e-05 2024-05-08T08:54:22.459 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.459 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.459 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:54:22.459 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.973623+0000", 2024-05-08T08:54:22.459 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.041898283000000001 2024-05-08T08:54:22.459 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.459 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.459 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:22.459 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:45.973666+0000", 2024-05-08T08:54:22.460 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2358999999999999e-05 2024-05-08T08:54:22.460 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:22.460 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:22.460 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:22.460 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.460 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.460 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4195.0:4 3.11 3:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-05-08T08:54:22.460 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:49.764654+0000", 2024-05-08T08:54:22.460 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 92.688299357000005, 2024-05-08T08:54:22.460 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.30714275499999999, 2024-05-08T08:54:22.460 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:22.460 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:22.460 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-05-08T08:54:22.460 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:22.460 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4195", 2024-05-08T08:54:22.460 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/59179521", 2024-05-08T08:54:22.460 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4 2024-05-08T08:54:22.461 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.461 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:22.461 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.461 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:22.461 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764654+0000", 2024-05-08T08:54:22.461 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:22.461 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.461 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.461 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:22.461 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764654+0000", 2024-05-08T08:54:22.461 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:22.461 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.461 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.461 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:22.461 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764655+0000", 2024-05-08T08:54:22.461 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.452e-06 2024-05-08T08:54:22.462 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.462 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.462 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:22.462 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764661+0000", 2024-05-08T08:54:22.462 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5369999999999997e-06 2024-05-08T08:54:22.462 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.462 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.462 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:22.462 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764671+0000", 2024-05-08T08:54:22.462 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.028e-05 2024-05-08T08:54:22.462 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.462 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.462 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:22.462 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764680+0000", 2024-05-08T08:54:22.463 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0629999999999995e-06 2024-05-08T08:54:22.463 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.463 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.463 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:22.463 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764753+0000", 2024-05-08T08:54:22.463 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2679e-05 2024-05-08T08:54:22.463 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.463 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.463 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-05-08T08:54:22.463 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764765+0000", 2024-05-08T08:54:22.463 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2415000000000001e-05 2024-05-08T08:54:22.463 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.463 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.463 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:22.463 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:50.071692+0000", 2024-05-08T08:54:22.463 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.306927324 2024-05-08T08:54:22.464 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.464 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.464 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:22.464 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:50.071796+0000", 2024-05-08T08:54:22.464 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104005 2024-05-08T08:54:22.464 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:22.464 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:22.464 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:22.464 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.464 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.464 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22683 3.15 3:a907109d:::rbd_data.1140d3fc54c.000000000000004f:18 [read 1335296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:22.464 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.097686+0000", 2024-05-08T08:54:22.464 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 50.355266683000004, 2024-05-08T08:54:22.465 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042114716000000003, 2024-05-08T08:54:22.465 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:22.465 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:22.465 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:22.465 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:22.465 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:54:22.465 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:22.465 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22683 2024-05-08T08:54:22.465 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.465 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:22.465 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.465 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:22.465 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097686+0000", 2024-05-08T08:54:22.465 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:22.466 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.466 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.466 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:22.466 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097686+0000", 2024-05-08T08:54:22.466 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:22.466 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.466 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.466 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:22.466 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097687+0000", 2024-05-08T08:54:22.466 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3999999999999997e-07 2024-05-08T08:54:22.466 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.466 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.466 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:22.466 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097688+0000", 2024-05-08T08:54:22.466 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6789999999999999e-06 2024-05-08T08:54:22.466 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.467 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.467 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:22.467 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097693+0000", 2024-05-08T08:54:22.467 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.045e-06 2024-05-08T08:54:22.467 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.467 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.467 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:22.467 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097697+0000", 2024-05-08T08:54:22.467 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0260000000000001e-06 2024-05-08T08:54:22.467 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.467 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.467 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:22.467 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139725+0000", 2024-05-08T08:54:22.467 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042027737000000003 2024-05-08T08:54:22.468 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.468 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.468 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:22.468 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139764+0000", 2024-05-08T08:54:22.468 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8668999999999999e-05 2024-05-08T08:54:22.469 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.469 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.469 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:22.469 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139801+0000", 2024-05-08T08:54:22.469 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7219999999999999e-05 2024-05-08T08:54:22.469 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:22.469 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:22.469 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:22.469 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.469 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.469 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22684 3.15 3:a907109d:::rbd_data.1140d3fc54c.000000000000004f:18 [read 1400832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:22.469 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.097703+0000", 2024-05-08T08:54:22.469 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 50.355250329999997, 2024-05-08T08:54:22.469 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042198929000000003, 2024-05-08T08:54:22.469 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:22.469 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:22.470 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:22.470 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:22.470 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:54:22.470 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:22.470 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22684 2024-05-08T08:54:22.470 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.470 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:22.470 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.470 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:22.470 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097703+0000", 2024-05-08T08:54:22.470 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:22.470 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.470 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.470 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:22.470 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097703+0000", 2024-05-08T08:54:22.470 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:22.471 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.471 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.471 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:22.471 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097703+0000", 2024-05-08T08:54:22.471 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4400000000000001e-07 2024-05-08T08:54:22.471 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.471 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.471 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:22.471 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097705+0000", 2024-05-08T08:54:22.471 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9199999999999998e-06 2024-05-08T08:54:22.471 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.471 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.471 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:22.471 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097711+0000", 2024-05-08T08:54:22.471 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5559999999999998e-06 2024-05-08T08:54:22.471 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.472 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.472 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:22.472 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097715+0000", 2024-05-08T08:54:22.472 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3159999999999998e-06 2024-05-08T08:54:22.472 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.472 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.472 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:22.472 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139825+0000", 2024-05-08T08:54:22.472 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042110284999999997 2024-05-08T08:54:22.472 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.472 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.472 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:22.472 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139864+0000", 2024-05-08T08:54:22.472 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8980000000000003e-05 2024-05-08T08:54:22.472 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.472 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.473 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:22.473 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139902+0000", 2024-05-08T08:54:22.473 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7527999999999997e-05 2024-05-08T08:54:22.473 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:22.473 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:22.473 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:22.473 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.473 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.473 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22685 3.15 3:a907109d:::rbd_data.1140d3fc54c.000000000000004f:18 [read 1466368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:22.473 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.097721+0000", 2024-05-08T08:54:22.473 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 50.355231672999999, 2024-05-08T08:54:22.473 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042287791999999998, 2024-05-08T08:54:22.473 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:22.473 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:22.473 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:22.473 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:22.474 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:54:22.474 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:22.474 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22685 2024-05-08T08:54:22.474 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.474 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:22.474 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.474 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:22.474 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097721+0000", 2024-05-08T08:54:22.474 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:22.474 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.474 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.474 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:22.474 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097721+0000", 2024-05-08T08:54:22.474 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:22.474 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.475 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.475 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:22.475 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097722+0000", 2024-05-08T08:54:22.475 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2500000000000001e-07 2024-05-08T08:54:22.475 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.475 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.475 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:22.475 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097724+0000", 2024-05-08T08:54:22.475 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.835e-06 2024-05-08T08:54:22.475 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.475 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.475 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:22.475 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097729+0000", 2024-05-08T08:54:22.475 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5029999999999997e-06 2024-05-08T08:54:22.475 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.475 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.476 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:22.476 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097733+0000", 2024-05-08T08:54:22.476 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3309999999999997e-06 2024-05-08T08:54:22.476 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.476 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.476 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:22.476 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139923+0000", 2024-05-08T08:54:22.476 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042189988999999997 2024-05-08T08:54:22.476 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.476 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.476 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:22.476 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.139970+0000", 2024-05-08T08:54:22.476 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6776000000000001e-05 2024-05-08T08:54:22.476 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.476 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.476 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:22.476 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140009+0000", 2024-05-08T08:54:22.477 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9032999999999997e-05 2024-05-08T08:54:22.477 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:22.477 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:22.477 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:22.477 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.477 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.477 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22686 3.15 3:a907109d:::rbd_data.1140d3fc54c.000000000000004f:18 [read 1531904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:22.477 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.097739+0000", 2024-05-08T08:54:22.477 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 50.355213673999998, 2024-05-08T08:54:22.477 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042371008000000002, 2024-05-08T08:54:22.477 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:22.477 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:22.477 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:22.477 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:22.477 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:54:22.478 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:22.478 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22686 2024-05-08T08:54:22.478 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.478 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:22.478 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.478 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:22.478 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097739+0000", 2024-05-08T08:54:22.478 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:22.478 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.478 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.478 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:22.478 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097739+0000", 2024-05-08T08:54:22.478 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:22.478 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.478 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.479 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:22.479 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097740+0000", 2024-05-08T08:54:22.479 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8599999999999999e-07 2024-05-08T08:54:22.479 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.479 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.479 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:22.479 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097742+0000", 2024-05-08T08:54:22.479 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9410000000000001e-06 2024-05-08T08:54:22.479 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.479 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.479 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:22.479 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097747+0000", 2024-05-08T08:54:22.479 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2079999999999999e-06 2024-05-08T08:54:22.479 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.479 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.479 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:22.480 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.097751+0000", 2024-05-08T08:54:22.480 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2250000000000002e-06 2024-05-08T08:54:22.480 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.480 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.480 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:22.480 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140032+0000", 2024-05-08T08:54:22.480 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042281244000000003 2024-05-08T08:54:22.480 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.480 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.480 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:22.480 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140074+0000", 2024-05-08T08:54:22.480 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1316999999999998e-05 2024-05-08T08:54:22.480 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.480 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.480 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:22.481 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140110+0000", 2024-05-08T08:54:22.481 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6687000000000003e-05 2024-05-08T08:54:22.481 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:22.481 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:22.481 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:22.481 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.481 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.481 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22716 3.15 3:a88fdee9:::rbd_data.1140d3fc54c.000000000000004d:18 [read 1097728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:22.481 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.098153+0000", 2024-05-08T08:54:22.481 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 50.354799966999998, 2024-05-08T08:54:22.481 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042058535000000001, 2024-05-08T08:54:22.481 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:22.481 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:22.481 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:22.481 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:22.482 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:54:22.482 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:22.482 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22716 2024-05-08T08:54:22.482 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.482 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:22.482 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.482 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:22.482 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098153+0000", 2024-05-08T08:54:22.482 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:22.482 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.482 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.482 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:22.482 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098153+0000", 2024-05-08T08:54:22.482 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:22.483 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.483 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.483 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:22.483 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098154+0000", 2024-05-08T08:54:22.483 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.13e-07 2024-05-08T08:54:22.483 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.483 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.483 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:22.483 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098161+0000", 2024-05-08T08:54:22.483 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7360000000000001e-06 2024-05-08T08:54:22.483 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.483 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.483 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:22.483 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098171+0000", 2024-05-08T08:54:22.483 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1039999999999996e-06 2024-05-08T08:54:22.483 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.483 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.484 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:22.484 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098177+0000", 2024-05-08T08:54:22.484 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5760000000000002e-06 2024-05-08T08:54:22.484 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.484 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.484 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:22.484 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140135+0000", 2024-05-08T08:54:22.484 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.041957842000000002 2024-05-08T08:54:22.484 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.484 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.484 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:22.484 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140174+0000", 2024-05-08T08:54:22.484 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8902999999999997e-05 2024-05-08T08:54:22.484 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.485 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.485 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:22.485 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140212+0000", 2024-05-08T08:54:22.485 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7761e-05 2024-05-08T08:54:22.485 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:22.485 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:22.485 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:22.485 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.485 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.485 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22717 3.15 3:a88fdee9:::rbd_data.1140d3fc54c.000000000000004d:18 [read 1163264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:22.485 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.098188+0000", 2024-05-08T08:54:22.485 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 50.354765245000003, 2024-05-08T08:54:22.485 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042124907000000003, 2024-05-08T08:54:22.485 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:22.485 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:22.485 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:22.485 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:22.486 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:54:22.486 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:22.486 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22717 2024-05-08T08:54:22.486 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.486 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:22.486 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.486 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:22.486 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098188+0000", 2024-05-08T08:54:22.486 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:22.486 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.486 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.486 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:22.486 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098188+0000", 2024-05-08T08:54:22.486 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:22.487 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.487 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.487 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:22.487 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098188+0000", 2024-05-08T08:54:22.487 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3799999999999998e-07 2024-05-08T08:54:22.487 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.487 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.487 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:22.487 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098190+0000", 2024-05-08T08:54:22.487 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.114e-06 2024-05-08T08:54:22.487 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.487 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.487 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:22.487 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098197+0000", 2024-05-08T08:54:22.487 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9959999999999996e-06 2024-05-08T08:54:22.487 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.488 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.488 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:22.488 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098202+0000", 2024-05-08T08:54:22.488 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9110000000000001e-06 2024-05-08T08:54:22.488 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.488 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.488 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:22.488 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140235+0000", 2024-05-08T08:54:22.488 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042032913999999998 2024-05-08T08:54:22.488 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.488 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.488 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:22.488 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140275+0000", 2024-05-08T08:54:22.488 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9564999999999998e-05 2024-05-08T08:54:22.488 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.488 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.488 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:22.489 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140313+0000", 2024-05-08T08:54:22.489 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7969000000000002e-05 2024-05-08T08:54:22.489 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:22.489 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:22.489 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:22.489 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.489 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.489 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22718 3.15 3:a88fdee9:::rbd_data.1140d3fc54c.000000000000004d:18 [read 1228800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:22.489 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.098209+0000", 2024-05-08T08:54:22.489 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 50.354744099000001, 2024-05-08T08:54:22.489 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.04220409, 2024-05-08T08:54:22.489 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:22.489 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:22.489 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:22.489 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:22.489 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:54:22.490 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:22.490 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22718 2024-05-08T08:54:22.490 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.490 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:22.490 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.490 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:22.490 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098209+0000", 2024-05-08T08:54:22.490 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:22.490 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.490 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.490 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:22.490 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098209+0000", 2024-05-08T08:54:22.490 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:22.490 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.490 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:22.491 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098209+0000", 2024-05-08T08:54:22.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9299999999999999e-07 2024-05-08T08:54:22.491 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:22.491 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098212+0000", 2024-05-08T08:54:22.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.21e-06 2024-05-08T08:54:22.491 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:22.491 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098219+0000", 2024-05-08T08:54:22.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0269999999999998e-06 2024-05-08T08:54:22.491 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.492 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.492 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:22.492 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098223+0000", 2024-05-08T08:54:22.492 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7319999999999998e-06 2024-05-08T08:54:22.492 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.492 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.492 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:22.492 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140338+0000", 2024-05-08T08:54:22.492 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042114390000000002 2024-05-08T08:54:22.492 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.492 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.492 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:22.492 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140376+0000", 2024-05-08T08:54:22.492 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8414999999999997e-05 2024-05-08T08:54:22.493 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.493 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.493 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:22.493 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140413+0000", 2024-05-08T08:54:22.493 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6922999999999999e-05 2024-05-08T08:54:22.493 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:22.493 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:22.493 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:22.493 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.493 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.493 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22719 3.15 3:a88fdee9:::rbd_data.1140d3fc54c.000000000000004d:18 [read 1294336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:22.493 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.098229+0000", 2024-05-08T08:54:22.493 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 50.354724247999997, 2024-05-08T08:54:22.493 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042283016, 2024-05-08T08:54:22.493 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:22.493 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:22.494 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:22.494 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:22.494 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:54:22.494 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:22.494 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22719 2024-05-08T08:54:22.494 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.494 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:22.494 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.494 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:22.494 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098229+0000", 2024-05-08T08:54:22.494 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:22.494 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.494 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.494 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:22.495 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098229+0000", 2024-05-08T08:54:22.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:22.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.495 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:22.495 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098229+0000", 2024-05-08T08:54:22.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:54:22.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.495 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:22.495 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098231+0000", 2024-05-08T08:54:22.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.81e-06 2024-05-08T08:54:22.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.495 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:22.495 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098237+0000", 2024-05-08T08:54:22.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3389999999999998e-06 2024-05-08T08:54:22.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.496 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:22.496 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098243+0000", 2024-05-08T08:54:22.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.648e-06 2024-05-08T08:54:22.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.496 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:22.496 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140436+0000", 2024-05-08T08:54:22.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042192738 2024-05-08T08:54:22.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.496 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:22.497 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140475+0000", 2024-05-08T08:54:22.497 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9279999999999997e-05 2024-05-08T08:54:22.497 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.497 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.497 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:22.497 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140512+0000", 2024-05-08T08:54:22.497 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6847000000000001e-05 2024-05-08T08:54:22.497 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:22.497 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:22.497 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:22.497 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.497 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.497 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22720 3.15 3:a88fdee9:::rbd_data.1140d3fc54c.000000000000004d:18 [read 1359872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:22.497 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.098248+0000", 2024-05-08T08:54:22.497 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 50.354704736999999, 2024-05-08T08:54:22.497 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042360800999999997, 2024-05-08T08:54:22.497 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:22.498 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:22.498 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:22.498 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:22.498 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:54:22.498 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:22.498 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22720 2024-05-08T08:54:22.498 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.498 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:22.498 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.498 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:22.498 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098248+0000", 2024-05-08T08:54:22.498 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:22.499 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.499 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.499 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:22.499 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098248+0000", 2024-05-08T08:54:22.499 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:22.499 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.499 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.499 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:22.499 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098491+0000", 2024-05-08T08:54:22.499 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000242659 2024-05-08T08:54:22.499 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.499 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.499 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:22.499 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098495+0000", 2024-05-08T08:54:22.499 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.799e-06 2024-05-08T08:54:22.499 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.500 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.500 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:22.500 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098505+0000", 2024-05-08T08:54:22.500 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7079999999999992e-06 2024-05-08T08:54:22.500 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.500 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.500 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:22.500 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098511+0000", 2024-05-08T08:54:22.500 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4370000000000004e-06 2024-05-08T08:54:22.500 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.500 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.500 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:22.500 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140533+0000", 2024-05-08T08:54:22.500 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042022371000000003 2024-05-08T08:54:22.500 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.500 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.501 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:22.501 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140571+0000", 2024-05-08T08:54:22.501 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7700000000000002e-05 2024-05-08T08:54:22.501 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.501 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.501 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:22.501 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140609+0000", 2024-05-08T08:54:22.501 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8127000000000003e-05 2024-05-08T08:54:22.501 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:22.501 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:22.501 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:22.501 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.501 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.501 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22721 3.15 3:a88fdee9:::rbd_data.1140d3fc54c.000000000000004d:18 [read 1425408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:22.501 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.098522+0000", 2024-05-08T08:54:22.502 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 50.354431112999997, 2024-05-08T08:54:22.502 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042185779999999999, 2024-05-08T08:54:22.502 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:22.502 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:22.502 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:22.502 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:22.502 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:54:22.502 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:22.502 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22721 2024-05-08T08:54:22.502 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.502 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:22.502 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.502 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:22.502 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098522+0000", 2024-05-08T08:54:22.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:22.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.503 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:22.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098522+0000", 2024-05-08T08:54:22.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:22.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.503 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:22.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098708+0000", 2024-05-08T08:54:22.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00018629700000000001 2024-05-08T08:54:22.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.503 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:22.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098719+0000", 2024-05-08T08:54:22.504 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0992000000000001e-05 2024-05-08T08:54:22.504 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.504 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.504 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:22.504 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098730+0000", 2024-05-08T08:54:22.504 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0431999999999999e-05 2024-05-08T08:54:22.504 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.504 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.504 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:22.504 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098737+0000", 2024-05-08T08:54:22.504 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.063e-06 2024-05-08T08:54:22.504 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.504 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.504 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:22.504 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140631+0000", 2024-05-08T08:54:22.505 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.041894503999999999 2024-05-08T08:54:22.505 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.505 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.505 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:22.505 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140671+0000", 2024-05-08T08:54:22.505 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9780999999999997e-05 2024-05-08T08:54:22.505 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.505 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.505 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:22.505 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140708+0000", 2024-05-08T08:54:22.505 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6711000000000001e-05 2024-05-08T08:54:22.505 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:22.505 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:22.505 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:22.506 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.506 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.506 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22722 3.15 3:a88fdee9:::rbd_data.1140d3fc54c.000000000000004d:18 [read 1490944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:22.506 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.098753+0000", 2024-05-08T08:54:22.506 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 50.354200108999997, 2024-05-08T08:54:22.506 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042053944000000003, 2024-05-08T08:54:22.506 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:22.506 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:22.506 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:22.506 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:22.506 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:54:22.506 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:22.506 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22722 2024-05-08T08:54:22.507 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.507 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:22.507 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.507 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:22.507 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098753+0000", 2024-05-08T08:54:22.507 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:22.507 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.507 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.507 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:22.507 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098753+0000", 2024-05-08T08:54:22.507 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:22.507 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.507 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.507 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:22.508 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098753+0000", 2024-05-08T08:54:22.508 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1399999999999997e-07 2024-05-08T08:54:22.508 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.508 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.508 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:22.508 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098756+0000", 2024-05-08T08:54:22.508 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9780000000000001e-06 2024-05-08T08:54:22.508 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.508 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.508 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:22.508 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098764+0000", 2024-05-08T08:54:22.508 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4340000000000003e-06 2024-05-08T08:54:22.508 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.508 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.508 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:22.509 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098770+0000", 2024-05-08T08:54:22.509 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8080000000000001e-06 2024-05-08T08:54:22.509 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.509 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.509 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:22.509 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140731+0000", 2024-05-08T08:54:22.509 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.041961131999999998 2024-05-08T08:54:22.509 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.509 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.509 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:22.509 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140769+0000", 2024-05-08T08:54:22.509 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.773e-05 2024-05-08T08:54:22.509 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.509 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.510 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:22.510 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140807+0000", 2024-05-08T08:54:22.510 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8347999999999999e-05 2024-05-08T08:54:22.510 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:22.510 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:22.510 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:22.510 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.510 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.510 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22723 3.15 3:a88fdee9:::rbd_data.1140d3fc54c.000000000000004d:18 [read 1556480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:22.510 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.098778+0000", 2024-05-08T08:54:22.510 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 50.354175337000001, 2024-05-08T08:54:22.510 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042128577, 2024-05-08T08:54:22.510 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:22.510 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:22.510 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:22.511 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:22.511 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:54:22.511 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:22.511 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22723 2024-05-08T08:54:22.511 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.511 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:22.511 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.511 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:22.511 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098778+0000", 2024-05-08T08:54:22.511 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:22.511 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.511 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.511 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:22.511 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098778+0000", 2024-05-08T08:54:22.512 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:22.512 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.512 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.512 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:22.512 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098834+0000", 2024-05-08T08:54:22.512 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5788000000000002e-05 2024-05-08T08:54:22.512 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.512 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.512 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:22.512 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098837+0000", 2024-05-08T08:54:22.512 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3510000000000002e-06 2024-05-08T08:54:22.512 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.512 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.512 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:22.513 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098845+0000", 2024-05-08T08:54:22.513 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5490000000000006e-06 2024-05-08T08:54:22.513 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.513 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.513 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:22.513 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098861+0000", 2024-05-08T08:54:22.513 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5233e-05 2024-05-08T08:54:22.513 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.513 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.513 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:22.513 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140829+0000", 2024-05-08T08:54:22.513 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.04196863 2024-05-08T08:54:22.513 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.513 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.513 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:22.514 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140869+0000", 2024-05-08T08:54:22.514 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9461e-05 2024-05-08T08:54:22.514 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.514 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.514 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:22.514 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140906+0000", 2024-05-08T08:54:22.514 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7564999999999997e-05 2024-05-08T08:54:22.514 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:22.514 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:22.514 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:22.514 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.514 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.514 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22732 3.15 3:a907109d:::rbd_data.1140d3fc54c.000000000000004f:18 [read 1097728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:22.514 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.098871+0000", 2024-05-08T08:54:22.515 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 50.354081856999997, 2024-05-08T08:54:22.515 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042141517000000003, 2024-05-08T08:54:22.515 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:22.515 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:22.515 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:22.515 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:22.515 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:54:22.515 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:22.515 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22732 2024-05-08T08:54:22.515 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.515 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:22.515 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.515 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:22.515 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098871+0000", 2024-05-08T08:54:22.515 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:22.515 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.516 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.516 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:22.516 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.098871+0000", 2024-05-08T08:54:22.516 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:22.516 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.516 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.516 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:22.516 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.099940+0000", 2024-05-08T08:54:22.516 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0010683279999999999 2024-05-08T08:54:22.516 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.516 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.516 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:22.516 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.099944+0000", 2024-05-08T08:54:22.516 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9339999999999997e-06 2024-05-08T08:54:22.516 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.517 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.517 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:22.517 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.099956+0000", 2024-05-08T08:54:22.517 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1970999999999999e-05 2024-05-08T08:54:22.517 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.517 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.517 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:22.517 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.099968+0000", 2024-05-08T08:54:22.517 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.189e-05 2024-05-08T08:54:22.517 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.517 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.517 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:22.517 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140930+0000", 2024-05-08T08:54:22.517 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.040961806000000003 2024-05-08T08:54:22.518 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.518 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.518 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:22.518 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.140974+0000", 2024-05-08T08:54:22.518 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3918999999999997e-05 2024-05-08T08:54:22.518 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.518 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.518 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:22.518 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.141013+0000", 2024-05-08T08:54:22.518 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8668999999999999e-05 2024-05-08T08:54:22.518 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:22.518 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:22.518 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:22.518 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.519 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.519 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:22765 3.15 3:a9c637ae:::rbd_data.114071bcf828.000000000000013f:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:22.519 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.100902+0000", 2024-05-08T08:54:22.519 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 50.352051062000001, 2024-05-08T08:54:22.519 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042471271999999997, 2024-05-08T08:54:22.519 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:22.519 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:22.519 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:22.519 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:22.519 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:54:22.519 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:22.519 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22765 2024-05-08T08:54:22.519 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.519 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:22.519 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.520 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:22.520 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.100902+0000", 2024-05-08T08:54:22.520 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:22.520 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.520 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.520 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:22.520 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.100902+0000", 2024-05-08T08:54:22.520 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:22.520 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.520 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.520 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:22.520 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.100902+0000", 2024-05-08T08:54:22.520 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.39e-07 2024-05-08T08:54:22.520 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.520 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.520 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:22.521 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.100904+0000", 2024-05-08T08:54:22.521 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6479999999999999e-06 2024-05-08T08:54:22.521 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.521 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.521 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:22.521 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.100910+0000", 2024-05-08T08:54:22.521 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1399999999999997e-06 2024-05-08T08:54:22.521 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.521 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.521 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:22.521 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.100916+0000", 2024-05-08T08:54:22.521 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6999999999999996e-06 2024-05-08T08:54:22.521 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.521 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.522 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:22.522 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.141733+0000", 2024-05-08T08:54:22.522 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.040817271000000002 2024-05-08T08:54:22.522 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.522 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.522 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:22.522 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.141812+0000", 2024-05-08T08:54:22.522 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9270999999999994e-05 2024-05-08T08:54:22.522 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.522 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.522 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:22.522 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.143373+0000", 2024-05-08T08:54:22.522 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001560903 2024-05-08T08:54:22.522 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:22.522 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:22.523 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:22.523 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.523 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.523 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5093.0:23031 3.15 3:ac9fb9cd:::rbd_data.114071bcf828.00000000000000f7:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:22.523 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:32.113562+0000", 2024-05-08T08:54:22.523 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 50.339391302000003, 2024-05-08T08:54:22.523 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042560938999999999, 2024-05-08T08:54:22.523 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:22.523 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:22.523 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:54:22.523 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:22.523 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5093", 2024-05-08T08:54:22.523 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:22.523 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 23031 2024-05-08T08:54:22.523 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.524 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:22.524 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.524 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:22.524 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.113562+0000", 2024-05-08T08:54:22.524 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:22.524 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.524 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.524 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:22.524 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.113562+0000", 2024-05-08T08:54:22.524 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:22.524 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.524 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.524 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:22.524 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.113938+0000", 2024-05-08T08:54:22.524 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000376401 2024-05-08T08:54:22.525 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.525 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.525 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:22.525 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.113943+0000", 2024-05-08T08:54:22.525 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0529999999999997e-06 2024-05-08T08:54:22.525 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.525 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.525 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:22.525 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.113951+0000", 2024-05-08T08:54:22.525 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8569999999999995e-06 2024-05-08T08:54:22.525 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.525 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.525 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:22.525 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.113959+0000", 2024-05-08T08:54:22.526 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4710000000000001e-06 2024-05-08T08:54:22.526 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.526 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.526 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:22.526 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.153955+0000", 2024-05-08T08:54:22.526 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.039996747999999999 2024-05-08T08:54:22.526 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.526 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.526 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:22.526 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.154014+0000", 2024-05-08T08:54:22.526 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8464999999999997e-05 2024-05-08T08:54:22.526 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.526 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.526 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-05-08T08:54:22.526 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.154148+0000", 2024-05-08T08:54:22.527 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00013428600000000001 2024-05-08T08:54:22.527 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.527 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.527 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:54:22.527 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.155390+0000", 2024-05-08T08:54:22.527 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001241792 2024-05-08T08:54:22.527 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.527 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.527 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:54:22.527 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.156068+0000", 2024-05-08T08:54:22.527 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00067823700000000005 2024-05-08T08:54:22.527 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.527 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.527 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:54:22.527 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.156102+0000", 2024-05-08T08:54:22.527 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3812000000000001e-05 2024-05-08T08:54:22.527 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.528 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.528 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:22.528 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:32.156123+0000", 2024-05-08T08:54:22.528 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0817000000000001e-05 2024-05-08T08:54:22.528 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:22.528 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:22.528 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:22.528 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.528 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.528 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4416.0:20531 3.16 3:6bb2ed16:::rbd_object_map.1140bc4a55c7:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-05-08T08:54:22.528 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:58.673786+0000", 2024-05-08T08:54:22.528 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.779166953000001, 2024-05-08T08:54:22.528 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.94684119899999997, 2024-05-08T08:54:22.528 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:22.528 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:22.528 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:54:22.528 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:22.529 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4416", 2024-05-08T08:54:22.529 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:54:22.529 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20531 2024-05-08T08:54:22.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.529 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:22.529 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.529 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:22.529 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673786+0000", 2024-05-08T08:54:22.529 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:22.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.529 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.529 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:22.529 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673786+0000", 2024-05-08T08:54:22.529 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:22.530 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.530 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.530 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:22.530 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673787+0000", 2024-05-08T08:54:22.530 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9800000000000003e-07 2024-05-08T08:54:22.530 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.530 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.530 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:22.530 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673791+0000", 2024-05-08T08:54:22.530 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.104e-06 2024-05-08T08:54:22.530 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.530 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.530 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:22.530 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673801+0000", 2024-05-08T08:54:22.531 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9960000000000005e-06 2024-05-08T08:54:22.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.531 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.531 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:22.531 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673806+0000", 2024-05-08T08:54:22.531 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2320000000000001e-06 2024-05-08T08:54:22.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.531 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.531 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:22.531 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673860+0000", 2024-05-08T08:54:22.531 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3468999999999999e-05 2024-05-08T08:54:22.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.531 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.531 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-05-08T08:54:22.532 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673866+0000", 2024-05-08T08:54:22.532 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3500000000000002e-06 2024-05-08T08:54:22.532 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.532 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.532 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:22.532 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.616700+0000", 2024-05-08T08:54:22.532 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.94283390700000003 2024-05-08T08:54:22.532 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.532 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.532 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:22.532 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.616941+0000", 2024-05-08T08:54:22.532 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00024080400000000001 2024-05-08T08:54:22.532 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.532 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.533 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-05-08T08:54:22.533 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.617072+0000", 2024-05-08T08:54:22.533 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000131688 2024-05-08T08:54:22.533 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.533 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.533 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:54:22.533 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.619571+0000", 2024-05-08T08:54:22.533 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0024990149999999998 2024-05-08T08:54:22.533 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.533 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.533 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:54:22.533 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.620568+0000", 2024-05-08T08:54:22.533 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00099711000000000005 2024-05-08T08:54:22.533 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.533 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.533 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:54:22.534 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.620602+0000", 2024-05-08T08:54:22.534 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.324e-05 2024-05-08T08:54:22.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.534 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.534 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:22.534 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.620627+0000", 2024-05-08T08:54:22.534 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5585999999999999e-05 2024-05-08T08:54:22.534 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:22.534 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:22.534 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:22.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.534 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.534 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5423.0:429 3.f 3:f1d5704a:::rbd_data.1140cf33de9.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-05-08T08:54:22.534 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:59.659591+0000", 2024-05-08T08:54:22.534 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 22.793362468000002, 2024-05-08T08:54:22.535 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.044109585, 2024-05-08T08:54:22.535 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:22.535 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:22.535 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:54:22.535 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:22.535 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5423", 2024-05-08T08:54:22.535 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/306969889", 2024-05-08T08:54:22.535 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 429 2024-05-08T08:54:22.535 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.535 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:22.535 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.535 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:22.535 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.659591+0000", 2024-05-08T08:54:22.535 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:22.536 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.536 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.536 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:22.536 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.659591+0000", 2024-05-08T08:54:22.536 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:22.536 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.536 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.536 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:22.536 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.659591+0000", 2024-05-08T08:54:22.536 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7299999999999996e-07 2024-05-08T08:54:22.536 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.536 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.536 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:22.537 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.663640+0000", 2024-05-08T08:54:22.537 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0040485529999999999 2024-05-08T08:54:22.537 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.537 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.537 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:22.537 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.663655+0000", 2024-05-08T08:54:22.537 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5065000000000001e-05 2024-05-08T08:54:22.537 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.537 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.537 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:22.537 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.663661+0000", 2024-05-08T08:54:22.537 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4269999999999996e-06 2024-05-08T08:54:22.537 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.537 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.537 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:22.537 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.663714+0000", 2024-05-08T08:54:22.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2992999999999999e-05 2024-05-08T08:54:22.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.538 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:22.538 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.663798+0000", 2024-05-08T08:54:22.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3992999999999994e-05 2024-05-08T08:54:22.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.538 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-05-08T08:54:22.538 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.664125+0000", 2024-05-08T08:54:22.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00032663899999999999 2024-05-08T08:54:22.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.538 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:54:22.538 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.687754+0000", 2024-05-08T08:54:22.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.023629075999999999 2024-05-08T08:54:22.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.539 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.539 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:54:22.539 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.703569+0000", 2024-05-08T08:54:22.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.015814814999999999 2024-05-08T08:54:22.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.539 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.539 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:54:22.539 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.703641+0000", 2024-05-08T08:54:22.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2484000000000006e-05 2024-05-08T08:54:22.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.539 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.539 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:22.539 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.703700+0000", 2024-05-08T08:54:22.540 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8967e-05 2024-05-08T08:54:22.540 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:22.540 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:22.540 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:22.540 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:22.540 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:22.540 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:22.542 INFO:teuthology.orchestra.run.smithi196.stdout:5732 trunc from 0xc6ba600 to 0x7917400 2024-05-08T08:54:22.542 INFO:teuthology.orchestra.run.smithi196.stdout:5734 read 0x3477400 thru 0x347f1a7 (0x7da8 bytes) 2024-05-08T08:54:22.542 INFO:teuthology.orchestra.run.smithi196.stdout:skipping zero length punch hole 2024-05-08T08:54:22.542 INFO:teuthology.orchestra.run.smithi196.stdout:5737 read 0x3ff2800 thru 0x3ff350a (0xd0b bytes) 2024-05-08T08:54:22.542 INFO:teuthology.orchestra.run.smithi196.stdout:5738 punch from 0x65e6600 to 0x65f1a00, (0xb400 bytes) 2024-05-08T08:54:22.542 INFO:teuthology.orchestra.run.smithi196.stdout:5741 clone 46 order 23 su 262144 sc 3 2024-05-08T08:54:22.543 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:54:22.543 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:22.544+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:22.643 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:22.645+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:22.649 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:22.649 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:54:22.649 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:54:22.649 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:22.658 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:54:22.744 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:22.747+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:22.811 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:22.811 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:54:22.811 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:54:22.811 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:54:22.812 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:22.820 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:54:22.846 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:22.848+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:22.947 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:22.950+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:22.969 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:22.969 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:54:22.969 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:54:22.969 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:54:22.969 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.969 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22073 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 782336~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:22.969 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:54:22.969 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 115.113508321, 2024-05-08T08:54:22.969 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047244234000000003, 2024-05-08T08:54:22.969 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:22.969 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:22.970 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:22.970 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:22.970 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:22.970 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:22.970 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22073 2024-05-08T08:54:22.970 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.970 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:22.970 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.970 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:22.970 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:54:22.970 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:22.970 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.970 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.970 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:22.970 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:54:22.971 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:22.971 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.971 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.971 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:22.971 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857668+0000", 2024-05-08T08:54:22.971 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7800000000000002e-07 2024-05-08T08:54:22.971 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.971 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.971 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:22.971 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857670+0000", 2024-05-08T08:54:22.971 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0820000000000001e-06 2024-05-08T08:54:22.971 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.971 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.971 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:22.971 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857678+0000", 2024-05-08T08:54:22.971 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7589999999999997e-06 2024-05-08T08:54:22.971 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.972 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.972 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:22.972 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857691+0000", 2024-05-08T08:54:22.972 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3672999999999999e-05 2024-05-08T08:54:22.972 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.972 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.972 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:22.972 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904780+0000", 2024-05-08T08:54:22.972 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047088616999999999 2024-05-08T08:54:22.972 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.972 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.972 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:22.972 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904840+0000", 2024-05-08T08:54:22.972 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0356999999999998e-05 2024-05-08T08:54:22.972 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.972 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.973 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:22.973 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904912+0000", 2024-05-08T08:54:22.973 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1367999999999998e-05 2024-05-08T08:54:22.973 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:22.973 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:22.973 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:22.973 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.973 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.973 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22075 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 798720~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:22.973 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:54:22.973 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 115.11347792399999, 2024-05-08T08:54:22.973 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047384643999999997, 2024-05-08T08:54:22.973 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:22.973 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:22.973 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:22.973 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:22.974 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:22.974 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:22.974 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22075 2024-05-08T08:54:22.974 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.974 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:22.974 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.974 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:22.974 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:54:22.974 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:22.974 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.974 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.974 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:22.974 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:54:22.974 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:22.974 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.975 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.975 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:22.975 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:54:22.975 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7399999999999999e-07 2024-05-08T08:54:22.975 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.975 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.975 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:22.975 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857705+0000", 2024-05-08T08:54:22.975 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1980000000000004e-06 2024-05-08T08:54:22.975 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.975 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.975 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:22.975 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857713+0000", 2024-05-08T08:54:22.975 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7039999999999994e-06 2024-05-08T08:54:22.975 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.975 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.976 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:22.976 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857718+0000", 2024-05-08T08:54:22.976 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.865e-06 2024-05-08T08:54:22.976 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.976 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.976 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:22.976 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904941+0000", 2024-05-08T08:54:22.976 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047223198000000001 2024-05-08T08:54:22.976 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.976 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.976 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:22.976 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904999+0000", 2024-05-08T08:54:22.976 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8254999999999997e-05 2024-05-08T08:54:22.976 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.976 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.976 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:22.976 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905082+0000", 2024-05-08T08:54:22.977 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3049999999999999e-05 2024-05-08T08:54:22.977 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:22.977 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:22.977 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:22.977 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.977 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.977 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22077 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 815104~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:22.977 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:54:22.977 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 115.11343991299999, 2024-05-08T08:54:22.977 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047519448999999998, 2024-05-08T08:54:22.977 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:22.977 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:22.977 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:22.977 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:22.977 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:22.977 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:22.978 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22077 2024-05-08T08:54:22.978 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.978 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:22.978 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.978 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:22.978 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:54:22.978 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:22.978 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.978 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.978 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:22.978 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:54:22.978 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:22.978 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.978 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.979 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:22.979 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:54:22.979 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.99e-07 2024-05-08T08:54:22.979 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.979 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.979 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:22.979 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857739+0000", 2024-05-08T08:54:22.979 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3599999999999999e-06 2024-05-08T08:54:22.979 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.979 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.979 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:22.979 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857745+0000", 2024-05-08T08:54:22.979 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5139999999999998e-06 2024-05-08T08:54:22.979 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.979 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.980 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:22.980 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857750+0000", 2024-05-08T08:54:22.980 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8550000000000001e-06 2024-05-08T08:54:22.980 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.980 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.980 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:22.980 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905110+0000", 2024-05-08T08:54:22.980 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047360126000000002 2024-05-08T08:54:22.980 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.980 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.980 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:22.980 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905175+0000", 2024-05-08T08:54:22.980 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4689000000000004e-05 2024-05-08T08:54:22.980 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.980 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.980 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:22.980 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905255+0000", 2024-05-08T08:54:22.981 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0306e-05 2024-05-08T08:54:22.981 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:22.981 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:22.981 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:22.981 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.981 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.981 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22079 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 831488~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:22.981 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:54:22.981 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 115.113419365, 2024-05-08T08:54:22.981 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.04767499, 2024-05-08T08:54:22.981 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:22.981 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:22.981 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:22.981 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:22.981 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:22.981 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:22.982 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22079 2024-05-08T08:54:22.982 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.982 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:22.982 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.982 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:22.982 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:54:22.982 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:22.982 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.982 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.982 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:22.982 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:54:22.982 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:22.982 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.982 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.982 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:22.982 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857757+0000", 2024-05-08T08:54:22.982 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8299999999999997e-07 2024-05-08T08:54:22.983 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.983 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.983 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:22.983 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857759+0000", 2024-05-08T08:54:22.983 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1390000000000002e-06 2024-05-08T08:54:22.983 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.983 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.983 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:22.983 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857765+0000", 2024-05-08T08:54:22.983 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4609999999999996e-06 2024-05-08T08:54:22.983 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.983 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.983 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:22.983 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857770+0000", 2024-05-08T08:54:22.983 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6689999999999997e-06 2024-05-08T08:54:22.983 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.984 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.984 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:22.984 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905281+0000", 2024-05-08T08:54:22.984 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047510668999999998 2024-05-08T08:54:22.984 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.984 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.984 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:22.984 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905344+0000", 2024-05-08T08:54:22.984 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2797000000000002e-05 2024-05-08T08:54:22.984 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.984 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.984 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:22.984 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905431+0000", 2024-05-08T08:54:22.984 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7771999999999999e-05 2024-05-08T08:54:22.984 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:22.984 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:22.984 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:22.985 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.985 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.985 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22082 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 847872~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:22.985 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:54:22.985 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 115.11339967399999, 2024-05-08T08:54:22.985 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047831934999999999, 2024-05-08T08:54:22.985 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:22.985 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:22.985 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:22.985 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:22.985 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:22.985 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:22.985 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22082 2024-05-08T08:54:22.985 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.986 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:22.986 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.986 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:22.986 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:54:22.986 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:22.986 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.986 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.986 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:22.986 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:54:22.986 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:22.986 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.986 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.986 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:22.986 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:54:22.986 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:54:22.986 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.987 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.987 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:22.987 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857778+0000", 2024-05-08T08:54:22.987 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0789999999999999e-06 2024-05-08T08:54:22.987 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.987 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.987 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:22.987 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857784+0000", 2024-05-08T08:54:22.987 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0480000000000004e-06 2024-05-08T08:54:22.987 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.987 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.987 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:22.987 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857790+0000", 2024-05-08T08:54:22.987 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2719999999999997e-06 2024-05-08T08:54:22.987 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.987 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.987 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:22.988 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905458+0000", 2024-05-08T08:54:22.988 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047668470999999997 2024-05-08T08:54:22.988 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.988 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.988 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:22.988 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905521+0000", 2024-05-08T08:54:22.988 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2952000000000003e-05 2024-05-08T08:54:22.988 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.988 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.988 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:22.988 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905608+0000", 2024-05-08T08:54:22.988 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6759000000000001e-05 2024-05-08T08:54:22.988 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:22.988 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:22.988 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:22.988 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.989 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.989 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22084 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 864256~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:22.989 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:54:22.989 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 115.11337572399999, 2024-05-08T08:54:22.989 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047977748000000001, 2024-05-08T08:54:22.989 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:22.989 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:22.989 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:22.989 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:22.989 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:22.989 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:22.989 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22084 2024-05-08T08:54:22.989 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.989 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:22.989 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.990 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:22.990 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:54:22.990 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:22.990 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.990 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.990 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:22.990 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:54:22.990 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:22.990 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.990 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.990 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:22.990 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:54:22.990 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:54:22.990 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.990 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.991 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:22.991 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857802+0000", 2024-05-08T08:54:22.991 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0779999999999998e-06 2024-05-08T08:54:22.991 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.991 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.991 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:22.991 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857812+0000", 2024-05-08T08:54:22.991 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7939999999999997e-06 2024-05-08T08:54:22.991 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.991 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.991 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:22.991 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857817+0000", 2024-05-08T08:54:22.991 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8899999999999998e-06 2024-05-08T08:54:22.991 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.991 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.991 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:22.992 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905636+0000", 2024-05-08T08:54:22.992 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047819252 2024-05-08T08:54:22.992 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.992 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.992 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:22.992 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905700+0000", 2024-05-08T08:54:22.992 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4016999999999994e-05 2024-05-08T08:54:22.992 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.992 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.992 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:22.992 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905778+0000", 2024-05-08T08:54:22.992 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7306999999999997e-05 2024-05-08T08:54:22.992 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:22.992 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:22.992 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:22.993 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.993 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.993 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22086 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 880640~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:22.993 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:54:22.993 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 115.113352374, 2024-05-08T08:54:22.993 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048119149, 2024-05-08T08:54:22.993 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:22.993 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:22.993 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:22.993 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:22.993 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:22.993 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:22.993 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22086 2024-05-08T08:54:22.993 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.993 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:22.994 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.994 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:22.994 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:54:22.994 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:22.994 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.994 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.994 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:22.994 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:54:22.994 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:22.994 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.994 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.994 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:22.994 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857824+0000", 2024-05-08T08:54:22.994 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4200000000000002e-07 2024-05-08T08:54:22.995 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.995 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.995 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:22.995 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857826+0000", 2024-05-08T08:54:22.995 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0590000000000001e-06 2024-05-08T08:54:22.995 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.995 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.995 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:22.995 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857832+0000", 2024-05-08T08:54:22.995 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8029999999999999e-06 2024-05-08T08:54:22.995 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.995 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.995 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:22.995 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857837+0000", 2024-05-08T08:54:22.995 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7929999999999997e-06 2024-05-08T08:54:22.996 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.996 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.996 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:22.996 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905803+0000", 2024-05-08T08:54:22.996 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047965761000000003 2024-05-08T08:54:22.996 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.996 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.996 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:22.996 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905862+0000", 2024-05-08T08:54:22.996 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9203000000000002e-05 2024-05-08T08:54:22.996 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.996 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.996 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:22.996 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905942+0000", 2024-05-08T08:54:22.996 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0187999999999999e-05 2024-05-08T08:54:22.997 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:22.997 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:22.997 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:22.997 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.997 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.997 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22088 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 897024~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:22.997 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:54:22.997 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 115.11333232200001, 2024-05-08T08:54:22.997 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048256049000000002, 2024-05-08T08:54:22.997 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:22.997 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:22.997 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:22.997 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:22.997 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:22.997 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:22.997 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22088 2024-05-08T08:54:22.998 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.998 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:22.998 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.998 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:22.998 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:54:22.998 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:22.998 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.998 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.998 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:22.998 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:54:22.998 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:22.998 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.998 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.998 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:22.999 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857844+0000", 2024-05-08T08:54:22.999 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.46e-07 2024-05-08T08:54:22.999 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.999 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.999 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:22.999 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857846+0000", 2024-05-08T08:54:22.999 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0899999999999999e-06 2024-05-08T08:54:22.999 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.999 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.999 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:22.999 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857855+0000", 2024-05-08T08:54:22.999 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8219999999999996e-06 2024-05-08T08:54:22.999 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:22.999 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:22.999 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:22.999 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857860+0000", 2024-05-08T08:54:23.000 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2989999999999996e-06 2024-05-08T08:54:23.000 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.000 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.000 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:23.000 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905966+0000", 2024-05-08T08:54:23.000 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048106542000000002 2024-05-08T08:54:23.000 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.000 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.000 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:23.000 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906024+0000", 2024-05-08T08:54:23.000 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7958000000000003e-05 2024-05-08T08:54:23.000 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.000 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.000 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:23.001 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906099+0000", 2024-05-08T08:54:23.001 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4992000000000003e-05 2024-05-08T08:54:23.001 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:23.001 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:23.001 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:23.001 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.001 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.001 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22091 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 913408~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:23.001 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:54:23.001 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 115.113308562, 2024-05-08T08:54:23.001 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048395290000000001, 2024-05-08T08:54:23.001 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:23.001 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:23.001 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:23.001 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:23.002 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:23.002 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:23.002 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22091 2024-05-08T08:54:23.002 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.002 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:23.002 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.002 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:23.002 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:54:23.002 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:23.002 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.002 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.002 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:23.002 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:54:23.002 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:23.002 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.003 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.003 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:23.003 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:54:23.003 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4900000000000001e-07 2024-05-08T08:54:23.003 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.003 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.003 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:23.003 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857870+0000", 2024-05-08T08:54:23.003 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2510000000000001e-06 2024-05-08T08:54:23.003 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.003 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.003 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:23.003 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857877+0000", 2024-05-08T08:54:23.003 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0809999999999996e-06 2024-05-08T08:54:23.003 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.004 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.004 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:23.004 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857881+0000", 2024-05-08T08:54:23.004 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.724e-06 2024-05-08T08:54:23.004 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.004 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.004 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:23.004 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906127+0000", 2024-05-08T08:54:23.004 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048245345000000002 2024-05-08T08:54:23.004 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.004 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.004 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:23.004 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906190+0000", 2024-05-08T08:54:23.004 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2762000000000007e-05 2024-05-08T08:54:23.004 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.005 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.005 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:23.005 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906262+0000", 2024-05-08T08:54:23.005 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2778e-05 2024-05-08T08:54:23.005 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:23.005 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:23.005 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:23.005 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.005 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.005 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22093 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 929792~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:23.005 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:54:23.005 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 115.113286198, 2024-05-08T08:54:23.005 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048532875000000003, 2024-05-08T08:54:23.005 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:23.005 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:23.005 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:23.006 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:23.006 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:23.006 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:23.006 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22093 2024-05-08T08:54:23.006 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.006 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:23.006 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.006 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:23.006 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:54:23.006 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:23.006 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.006 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.006 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:23.006 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:54:23.007 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:23.007 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.007 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.007 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:23.007 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857890+0000", 2024-05-08T08:54:23.007 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-07 2024-05-08T08:54:23.007 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.007 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.007 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:23.007 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857899+0000", 2024-05-08T08:54:23.007 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6889999999999999e-06 2024-05-08T08:54:23.007 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.007 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.007 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:23.007 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857906+0000", 2024-05-08T08:54:23.007 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6490000000000002e-06 2024-05-08T08:54:23.008 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.008 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.008 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:23.008 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857917+0000", 2024-05-08T08:54:23.008 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0764999999999999e-05 2024-05-08T08:54:23.008 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.008 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.008 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:23.008 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906287+0000", 2024-05-08T08:54:23.008 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048370442 2024-05-08T08:54:23.008 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.008 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.008 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:23.008 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906349+0000", 2024-05-08T08:54:23.008 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1537000000000004e-05 2024-05-08T08:54:23.009 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.009 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.009 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:23.009 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906422+0000", 2024-05-08T08:54:23.009 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3431999999999998e-05 2024-05-08T08:54:23.009 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:23.009 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:23.009 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:23.009 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.009 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.009 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22095 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 946176~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:23.009 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:54:23.009 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 115.113251494, 2024-05-08T08:54:23.009 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048657593999999998, 2024-05-08T08:54:23.009 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:23.010 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:23.010 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:23.010 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:23.010 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:23.010 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:23.010 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22095 2024-05-08T08:54:23.010 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.010 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:23.010 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.010 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:23.010 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:54:23.010 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:23.010 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.010 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.011 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:23.011 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:54:23.011 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:23.011 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.011 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.011 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:23.011 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857925+0000", 2024-05-08T08:54:23.011 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3099999999999998e-07 2024-05-08T08:54:23.011 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.011 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.011 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:23.011 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857927+0000", 2024-05-08T08:54:23.011 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.34e-06 2024-05-08T08:54:23.011 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.011 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.012 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:23.012 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857938+0000", 2024-05-08T08:54:23.012 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1426e-05 2024-05-08T08:54:23.012 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.012 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.012 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:23.012 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857943+0000", 2024-05-08T08:54:23.012 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5140000000000003e-06 2024-05-08T08:54:23.012 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.012 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.012 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:23.012 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906448+0000", 2024-05-08T08:54:23.012 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048504641000000001 2024-05-08T08:54:23.012 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.012 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.012 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:23.012 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906510+0000", 2024-05-08T08:54:23.013 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2663000000000006e-05 2024-05-08T08:54:23.013 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.013 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.013 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:23.013 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906582+0000", 2024-05-08T08:54:23.013 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1478999999999998e-05 2024-05-08T08:54:23.013 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:23.013 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:23.013 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:23.013 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.013 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.013 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22097 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 962560~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:23.013 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:54:23.013 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 115.113223805, 2024-05-08T08:54:23.013 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048791353000000003, 2024-05-08T08:54:23.013 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:23.014 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:23.014 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:23.014 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:23.014 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:23.014 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:23.014 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22097 2024-05-08T08:54:23.014 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.014 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:23.014 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.014 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:23.014 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:54:23.014 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:23.014 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.014 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.014 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:23.015 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:54:23.015 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:23.015 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.015 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.015 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:23.015 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:54:23.015 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9200000000000002e-07 2024-05-08T08:54:23.015 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.015 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.015 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:23.015 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857954+0000", 2024-05-08T08:54:23.015 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1900000000000002e-06 2024-05-08T08:54:23.015 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.015 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.015 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:23.015 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857961+0000", 2024-05-08T08:54:23.016 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4989999999999999e-06 2024-05-08T08:54:23.016 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.016 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.016 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:23.016 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857971+0000", 2024-05-08T08:54:23.016 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0485999999999999e-05 2024-05-08T08:54:23.016 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.016 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.016 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:23.016 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906607+0000", 2024-05-08T08:54:23.016 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048635109000000003 2024-05-08T08:54:23.016 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.016 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.016 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:23.017 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906667+0000", 2024-05-08T08:54:23.017 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0439000000000002e-05 2024-05-08T08:54:23.017 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.017 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.017 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:23.017 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906743+0000", 2024-05-08T08:54:23.017 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6237999999999998e-05 2024-05-08T08:54:23.017 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:23.017 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:23.017 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:23.017 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.017 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.017 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22099 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 978944~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:23.017 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:54:23.017 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 115.113197645, 2024-05-08T08:54:23.017 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048932055000000002, 2024-05-08T08:54:23.018 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:23.018 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:23.018 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:23.018 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:23.018 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:23.018 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:23.018 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22099 2024-05-08T08:54:23.018 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.018 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:23.018 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.018 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:23.018 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:54:23.018 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:23.018 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.019 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.019 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:23.019 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:54:23.019 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:23.019 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.019 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.019 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:23.019 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:54:23.019 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.27e-07 2024-05-08T08:54:23.019 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.019 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.019 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:23.019 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857987+0000", 2024-05-08T08:54:23.019 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5890000000000003e-06 2024-05-08T08:54:23.019 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.020 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.020 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:23.020 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857994+0000", 2024-05-08T08:54:23.020 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6619999999999999e-06 2024-05-08T08:54:23.020 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.020 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.020 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:23.020 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857998+0000", 2024-05-08T08:54:23.020 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7110000000000003e-06 2024-05-08T08:54:23.020 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.020 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.020 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:23.020 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906772+0000", 2024-05-08T08:54:23.020 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048773415000000001 2024-05-08T08:54:23.020 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.021 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.021 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:23.021 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906834+0000", 2024-05-08T08:54:23.021 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1752000000000001e-05 2024-05-08T08:54:23.021 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.021 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.021 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:23.021 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906910+0000", 2024-05-08T08:54:23.021 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6499000000000001e-05 2024-05-08T08:54:23.021 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:23.021 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:23.021 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:23.021 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.021 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.021 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22101 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 995328~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:23.021 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:54:23.022 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 115.113170514, 2024-05-08T08:54:23.022 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049087364000000001, 2024-05-08T08:54:23.022 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:23.022 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:23.022 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:23.022 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:23.022 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:23.022 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:23.022 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22101 2024-05-08T08:54:23.022 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.022 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:23.022 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.022 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:23.022 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:54:23.023 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:23.023 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.023 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.023 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:23.023 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:54:23.023 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:23.023 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.023 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.023 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:23.023 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858006+0000", 2024-05-08T08:54:23.023 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1399999999999997e-07 2024-05-08T08:54:23.023 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.023 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.023 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:23.023 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858008+0000", 2024-05-08T08:54:23.024 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2060000000000001e-06 2024-05-08T08:54:23.024 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.024 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.024 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:23.024 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858017+0000", 2024-05-08T08:54:23.024 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1579999999999994e-06 2024-05-08T08:54:23.024 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.024 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.024 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:23.024 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858022+0000", 2024-05-08T08:54:23.024 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4199999999999998e-06 2024-05-08T08:54:23.024 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.024 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.024 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:23.024 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906940+0000", 2024-05-08T08:54:23.025 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048917627999999998 2024-05-08T08:54:23.025 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.025 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.025 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:23.025 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907002+0000", 2024-05-08T08:54:23.025 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2399999999999999e-05 2024-05-08T08:54:23.025 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.025 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.025 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:23.025 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907092+0000", 2024-05-08T08:54:23.025 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0137999999999997e-05 2024-05-08T08:54:23.025 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:23.025 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:23.025 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:23.025 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.025 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.026 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22103 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1011712~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:23.026 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:54:23.026 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 115.113145729, 2024-05-08T08:54:23.026 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049228542, 2024-05-08T08:54:23.026 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:23.026 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:23.026 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:23.026 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:23.026 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:23.026 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:23.026 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22103 2024-05-08T08:54:23.026 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.026 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:23.026 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.026 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:23.027 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:54:23.027 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:23.027 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.027 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.027 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:23.027 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:54:23.027 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:23.027 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.027 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.027 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:23.027 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:54:23.027 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4299999999999999e-07 2024-05-08T08:54:23.027 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.027 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.027 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:23.028 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858032+0000", 2024-05-08T08:54:23.028 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0590000000000001e-06 2024-05-08T08:54:23.028 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.028 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.028 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:23.028 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858037+0000", 2024-05-08T08:54:23.028 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.977e-06 2024-05-08T08:54:23.028 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.028 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.028 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:23.028 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858042+0000", 2024-05-08T08:54:23.028 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7650000000000001e-06 2024-05-08T08:54:23.028 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.028 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.028 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:23.028 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907118+0000", 2024-05-08T08:54:23.029 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049076147000000001 2024-05-08T08:54:23.029 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.029 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.029 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:23.029 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907180+0000", 2024-05-08T08:54:23.029 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1971999999999996e-05 2024-05-08T08:54:23.029 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.029 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.029 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:23.029 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907258+0000", 2024-05-08T08:54:23.029 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8279e-05 2024-05-08T08:54:23.029 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:23.029 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:23.029 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:23.029 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.030 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.030 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22105 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1028096~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:23.030 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:54:23.030 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 115.113126486, 2024-05-08T08:54:23.030 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049394609999999999, 2024-05-08T08:54:23.030 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:23.030 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:23.030 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:23.030 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:23.030 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:23.030 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:23.030 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22105 2024-05-08T08:54:23.030 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.030 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:23.030 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.031 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:23.031 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:54:23.031 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:23.031 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.031 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.031 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:23.031 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:54:23.031 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:23.031 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.031 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.031 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:23.031 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858050+0000", 2024-05-08T08:54:23.031 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5999999999999999e-07 2024-05-08T08:54:23.031 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.031 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.032 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:23.032 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858051+0000", 2024-05-08T08:54:23.032 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8870000000000001e-06 2024-05-08T08:54:23.032 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.032 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.032 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:23.032 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858057+0000", 2024-05-08T08:54:23.032 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8019999999999997e-06 2024-05-08T08:54:23.032 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.032 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.032 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:23.032 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858062+0000", 2024-05-08T08:54:23.032 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.561e-06 2024-05-08T08:54:23.032 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.032 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.033 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:23.033 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907289+0000", 2024-05-08T08:54:23.033 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049226957000000002 2024-05-08T08:54:23.033 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.033 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.033 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:23.033 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907356+0000", 2024-05-08T08:54:23.033 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6915000000000006e-05 2024-05-08T08:54:23.033 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.033 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.033 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:23.033 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907444+0000", 2024-05-08T08:54:23.033 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8127999999999994e-05 2024-05-08T08:54:23.033 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:23.033 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:23.034 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:23.034 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.034 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.034 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22107 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1044480~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:23.034 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:54:23.034 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 115.113108146, 2024-05-08T08:54:23.034 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049799942999999999, 2024-05-08T08:54:23.034 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:23.034 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:23.034 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:23.034 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:23.034 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:23.034 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:23.034 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22107 2024-05-08T08:54:23.034 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.035 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:23.035 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.035 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:23.035 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:54:23.035 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:23.035 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.035 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.035 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:23.035 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:54:23.035 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:23.035 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.035 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.035 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:23.035 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:54:23.035 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4799999999999999e-07 2024-05-08T08:54:23.036 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.036 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.036 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:23.036 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858070+0000", 2024-05-08T08:54:23.036 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.012e-06 2024-05-08T08:54:23.036 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.036 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.036 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:23.036 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858077+0000", 2024-05-08T08:54:23.036 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6290000000000001e-06 2024-05-08T08:54:23.036 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.036 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.036 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:23.036 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858083+0000", 2024-05-08T08:54:23.036 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8880000000000002e-06 2024-05-08T08:54:23.037 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.037 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.037 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:23.037 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907671+0000", 2024-05-08T08:54:23.037 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049587191000000003 2024-05-08T08:54:23.037 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.037 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.037 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:23.037 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907763+0000", 2024-05-08T08:54:23.037 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2231000000000005e-05 2024-05-08T08:54:23.037 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.037 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.037 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:23.037 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907867+0000", 2024-05-08T08:54:23.037 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104644 2024-05-08T08:54:23.038 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:23.038 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:23.038 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:23.038 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.038 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.038 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22294 3.1c 3:3aad0389:::rbd_data.11406886b07b.0000000000000080:head [stat,delete] snapc 7=[7] RETRY=6 ondisk+retry+write+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:23.038 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:54:23.038 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 115.11251429799999, 2024-05-08T08:54:23.038 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051152191, 2024-05-08T08:54:23.038 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:23.038 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:23.038 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:23.038 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:23.038 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:23.038 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:23.039 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22294 2024-05-08T08:54:23.039 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.039 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:23.039 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.039 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:23.039 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:54:23.039 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:23.039 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.039 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.039 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:23.039 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:54:23.039 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:23.039 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.039 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.039 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:23.039 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858662+0000", 2024-05-08T08:54:23.040 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4799999999999998e-07 2024-05-08T08:54:23.040 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.040 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.040 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:23.040 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858664+0000", 2024-05-08T08:54:23.040 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1229999999999998e-06 2024-05-08T08:54:23.040 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.040 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.040 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:23.040 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858670+0000", 2024-05-08T08:54:23.040 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3099999999999997e-06 2024-05-08T08:54:23.040 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.040 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.040 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:23.040 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858676+0000", 2024-05-08T08:54:23.041 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9750000000000004e-06 2024-05-08T08:54:23.041 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.041 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.041 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:23.041 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907901+0000", 2024-05-08T08:54:23.041 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049224520000000001 2024-05-08T08:54:23.041 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.041 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.041 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:23.041 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.908085+0000", 2024-05-08T08:54:23.041 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00018377500000000001 2024-05-08T08:54:23.041 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.041 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.041 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:23.041 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.909814+0000", 2024-05-08T08:54:23.041 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00172894 2024-05-08T08:54:23.042 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:23.042 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:23.042 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:23.042 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.042 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.042 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4416.0:13461 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-05-08T08:54:23.042 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:22.487456+0000", 2024-05-08T08:54:23.042 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 60.483719256999997, 2024-05-08T08:54:23.042 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.99102620799999996, 2024-05-08T08:54:23.042 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:23.042 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:23.042 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:23.042 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:23.042 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4416", 2024-05-08T08:54:23.042 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:54:23.043 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 13461 2024-05-08T08:54:23.043 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.043 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:23.043 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.043 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:23.043 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487456+0000", 2024-05-08T08:54:23.043 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:23.043 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.043 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.043 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:23.043 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487456+0000", 2024-05-08T08:54:23.043 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:23.043 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.043 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.043 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:23.044 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487458+0000", 2024-05-08T08:54:23.044 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5659999999999999e-06 2024-05-08T08:54:23.044 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.044 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.044 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:23.044 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487463+0000", 2024-05-08T08:54:23.044 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6890000000000003e-06 2024-05-08T08:54:23.044 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.044 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.044 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:23.044 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487472+0000", 2024-05-08T08:54:23.044 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7710000000000001e-06 2024-05-08T08:54:23.044 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.044 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.044 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:23.044 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487478+0000", 2024-05-08T08:54:23.045 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1390000000000003e-06 2024-05-08T08:54:23.045 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.045 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.045 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:23.045 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487520+0000", 2024-05-08T08:54:23.045 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2259999999999999e-05 2024-05-08T08:54:23.045 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.045 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.045 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-05-08T08:54:23.045 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487528+0000", 2024-05-08T08:54:23.045 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7479999999999993e-06 2024-05-08T08:54:23.045 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.045 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.045 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:23.045 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:23.478139+0000", 2024-05-08T08:54:23.046 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.99061165200000001 2024-05-08T08:54:23.046 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.046 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.046 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:23.046 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:23.478301+0000", 2024-05-08T08:54:23.046 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000161602 2024-05-08T08:54:23.046 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.046 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.046 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:23.046 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:23.478483+0000", 2024-05-08T08:54:23.046 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000181781 2024-05-08T08:54:23.046 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:23.046 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:23.046 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:23.046 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.047 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.047 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5752.0:404 3.1a 3:5f5e8104:::rbd_data.1140bc4a55c7.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 5175296~45056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-05-08T08:54:23.047 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:00.390863+0000", 2024-05-08T08:54:23.047 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 22.580313061999998, 2024-05-08T08:54:23.047 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.055189174000000001, 2024-05-08T08:54:23.047 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:23.047 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:23.047 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:54:23.047 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:23.047 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5752", 2024-05-08T08:54:23.047 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/3534419815", 2024-05-08T08:54:23.047 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 404 2024-05-08T08:54:23.047 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.047 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:23.047 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.048 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:23.048 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.390863+0000", 2024-05-08T08:54:23.048 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:23.048 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.048 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.048 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:23.048 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.390863+0000", 2024-05-08T08:54:23.048 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:23.048 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.048 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.048 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:23.048 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.390864+0000", 2024-05-08T08:54:23.048 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.471e-06 2024-05-08T08:54:23.048 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.048 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.048 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:23.049 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397475+0000", 2024-05-08T08:54:23.049 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0066109109999999997 2024-05-08T08:54:23.049 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.049 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.049 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:23.049 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397498+0000", 2024-05-08T08:54:23.049 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2674e-05 2024-05-08T08:54:23.049 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.049 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.049 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:23.049 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397508+0000", 2024-05-08T08:54:23.049 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9019999999999994e-06 2024-05-08T08:54:23.049 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.049 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.049 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:23.049 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397609+0000", 2024-05-08T08:54:23.050 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010121899999999999 2024-05-08T08:54:23.050 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.050 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.050 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:23.050 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397711+0000", 2024-05-08T08:54:23.050 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102489 2024-05-08T08:54:23.050 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.050 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.050 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-05-08T08:54:23.050 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397999+0000", 2024-05-08T08:54:23.050 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00028733799999999999 2024-05-08T08:54:23.050 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.050 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.050 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:54:23.051 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.432093+0000", 2024-05-08T08:54:23.051 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.034094865000000002 2024-05-08T08:54:23.051 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.051 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.051 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:54:23.051 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.445975+0000", 2024-05-08T08:54:23.051 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013881187 2024-05-08T08:54:23.051 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.051 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.051 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:54:23.051 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.446013+0000", 2024-05-08T08:54:23.051 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8155000000000003e-05 2024-05-08T08:54:23.051 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.051 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.051 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:23.052 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.446052+0000", 2024-05-08T08:54:23.052 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8962999999999999e-05 2024-05-08T08:54:23.052 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:23.052 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:23.052 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:23.052 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:23.052 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:23.052 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:23.053 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:23.056+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:23.054 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:54:23.138 INFO:teuthology.orchestra.run.smithi196.stdout:checking clone #44, image image_client.0-clone44 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent45 2024-05-08T08:54:23.153 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:23.156+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:23.165 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:23.165 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:54:23.165 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:54:23.165 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:23.174 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:54:23.221 INFO:teuthology.orchestra.run.smithi196.stdout:5742 trunc from 0x7917400 to 0x9edb000 2024-05-08T08:54:23.239 INFO:teuthology.orchestra.run.smithi196.stdout:5744 trunc from 0x9edb000 to 0xa6dac00 2024-05-08T08:54:23.259 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:23.262+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:23.264 INFO:teuthology.orchestra.run.smithi196.stdout:5747 write 0xe981800 thru 0xe991217 (0xfa18 bytes) 2024-05-08T08:54:23.265 INFO:teuthology.orchestra.run.smithi196.stdout:5749 read 0x40bae00 thru 0x40c52bb (0xa4bc bytes) 2024-05-08T08:54:23.266 INFO:teuthology.orchestra.run.smithi196.stdout:5750 read 0x336d400 thru 0x336fc63 (0x2864 bytes) 2024-05-08T08:54:23.266 INFO:teuthology.orchestra.run.smithi196.stdout:5751 punch from 0x22f3800 to 0x2300c00, (0xd400 bytes) 2024-05-08T08:54:23.267 INFO:teuthology.orchestra.run.smithi196.stdout:5753 read 0x650de00 thru 0x650e33e (0x53f bytes) 2024-05-08T08:54:23.268 INFO:teuthology.orchestra.run.smithi196.stdout:5755 write 0x5b4d400 thru 0x5b5abf0 (0xd7f1 bytes) 2024-05-08T08:54:23.269 INFO:teuthology.orchestra.run.smithi196.stdout:5756 write 0x6ebf400 thru 0x6ebf6aa (0x2ab bytes) 2024-05-08T08:54:23.269 INFO:teuthology.orchestra.run.smithi196.stdout:5758 read 0x1956000 thru 0x195c93a (0x693b bytes) 2024-05-08T08:54:23.270 INFO:teuthology.orchestra.run.smithi196.stdout:5759 read 0x5442400 thru 0x544b15d (0x8d5e bytes) 2024-05-08T08:54:23.270 INFO:teuthology.orchestra.run.smithi196.stdout:5761 read 0x56bf000 thru 0x56c6a40 (0x7a41 bytes) 2024-05-08T08:54:23.271 INFO:teuthology.orchestra.run.smithi196.stdout:5762 read 0x47b6000 thru 0x47b8f92 (0x2f93 bytes) 2024-05-08T08:54:23.271 INFO:teuthology.orchestra.run.smithi196.stdout:5763 write 0x7ec7800 thru 0x7ed4c1b (0xd41c bytes) 2024-05-08T08:54:23.271 INFO:teuthology.orchestra.run.smithi196.stdout:5764 write 0x242ea00 thru 0x242fa9f (0x10a0 bytes) 2024-05-08T08:54:23.272 INFO:teuthology.orchestra.run.smithi196.stdout:5766 read 0x4424800 thru 0x442836e (0x3b6f bytes) 2024-05-08T08:54:23.272 INFO:teuthology.orchestra.run.smithi196.stdout:5767 write 0x5827000 thru 0x5828b4b (0x1b4c bytes) 2024-05-08T08:54:23.288 INFO:teuthology.orchestra.run.smithi196.stdout:5768 trunc from 0xe991400 to 0x1d78000 2024-05-08T08:54:23.317 INFO:teuthology.orchestra.run.smithi196.stdout:5769 read 0x10e1600 thru 0x10e628a (0x4c8b bytes) 2024-05-08T08:54:23.328 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:23.328 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:54:23.328 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:54:23.328 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:54:23.329 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:23.336 INFO:teuthology.orchestra.run.smithi196.stdout:5770 write 0x88b2a00 thru 0x88bcc8b (0xa28c bytes) 2024-05-08T08:54:23.338 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:54:23.353 INFO:teuthology.orchestra.run.smithi196.stdout:5774 write 0x9bf4800 thru 0x9c01348 (0xcb49 bytes) 2024-05-08T08:54:23.353 INFO:teuthology.orchestra.run.smithi196.stdout:5776 punch from 0x371de00 to 0x3727e00, (0xa000 bytes) 2024-05-08T08:54:23.353 INFO:teuthology.orchestra.run.smithi196.stdout:5777 read 0x145a000 thru 0x1463b44 (0x9b45 bytes) 2024-05-08T08:54:23.354 INFO:teuthology.orchestra.run.smithi196.stdout:5778 write 0x519ba00 thru 0x51a8bfa (0xd1fb bytes) 2024-05-08T08:54:23.354 INFO:teuthology.orchestra.run.smithi196.stdout:5780 write 0x9a97000 thru 0x9aa27d1 (0xb7d2 bytes) 2024-05-08T08:54:23.355 INFO:teuthology.orchestra.run.smithi196.stdout:5782 trunc from 0x9c01400 to 0x9eb8a00 2024-05-08T08:54:23.376 INFO:teuthology.orchestra.run.smithi196.stdout:5783 write 0x4ffc400 thru 0x5008053 (0xbc54 bytes) 2024-05-08T08:54:23.377 INFO:teuthology.orchestra.run.smithi196.stdout:5784 write 0x21e1000 thru 0x21e5db3 (0x4db4 bytes) 2024-05-08T08:54:23.377 INFO:teuthology.orchestra.run.smithi196.stdout:5786 read 0x38d7e00 thru 0x38d8ae4 (0xce5 bytes) 2024-05-08T08:54:23.377 INFO:teuthology.orchestra.run.smithi196.stdout:5787 write 0x41b9200 thru 0x41c0fe7 (0x7de8 bytes) 2024-05-08T08:54:23.378 INFO:teuthology.orchestra.run.smithi196.stdout:5789 trunc from 0x9eb8a00 to 0x9764800 2024-05-08T08:54:23.379 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:23.382+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:23.387 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T08:54:23.409 INFO:teuthology.orchestra.run.smithi196.stdout:5791 trunc from 0x9764800 to 0xd5ea00 2024-05-08T08:54:23.457 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:23.460+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:23.457 INFO:teuthology.orchestra.run.smithi196.stdout:5793 clone 47 order 19 su 262144 sc 8 2024-05-08T08:54:23.492 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:23.492 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:54:23.492 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:54:23.492 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:54:23.492 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.492 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6690 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1093632~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:23.492 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:54:23.492 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 111.778583086, 2024-05-08T08:54:23.492 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052462754, 2024-05-08T08:54:23.492 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:23.492 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:23.492 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:23.493 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:23.493 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:23.493 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:23.493 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6690 2024-05-08T08:54:23.493 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.493 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:23.493 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.493 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:23.493 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:54:23.493 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:23.493 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.493 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.493 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:23.493 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:54:23.494 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:23.494 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.494 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.494 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:23.494 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715346+0000", 2024-05-08T08:54:23.494 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.84e-07 2024-05-08T08:54:23.494 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.494 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.494 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:23.494 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715348+0000", 2024-05-08T08:54:23.494 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.114e-06 2024-05-08T08:54:23.494 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.494 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.494 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:23.495 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715353+0000", 2024-05-08T08:54:23.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5720000000000002e-06 2024-05-08T08:54:23.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.495 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:23.495 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715358+0000", 2024-05-08T08:54:23.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7140000000000001e-06 2024-05-08T08:54:23.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.495 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:23.495 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767638+0000", 2024-05-08T08:54:23.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052279562000000002 2024-05-08T08:54:23.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.495 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:23.496 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767713+0000", 2024-05-08T08:54:23.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5652e-05 2024-05-08T08:54:23.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.496 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:23.496 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767808+0000", 2024-05-08T08:54:23.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4755999999999999e-05 2024-05-08T08:54:23.496 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:23.496 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:23.496 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:23.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.496 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6692 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1110016~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:23.496 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:54:23.496 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 111.778563996, 2024-05-08T08:54:23.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052666495000000001, 2024-05-08T08:54:23.497 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:23.497 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:23.497 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:23.497 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:23.497 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:23.497 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:23.497 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6692 2024-05-08T08:54:23.497 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.497 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:23.497 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.497 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:23.497 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:54:23.497 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:23.497 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.497 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.497 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:23.498 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:54:23.498 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:23.498 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.498 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.498 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:23.498 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:54:23.498 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8599999999999999e-07 2024-05-08T08:54:23.498 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.498 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.498 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:23.498 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715372+0000", 2024-05-08T08:54:23.498 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0990000000000001e-06 2024-05-08T08:54:23.498 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.498 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.498 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:23.499 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715378+0000", 2024-05-08T08:54:23.499 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0680000000000002e-06 2024-05-08T08:54:23.499 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.499 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.499 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:23.499 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715383+0000", 2024-05-08T08:54:23.499 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8740000000000003e-06 2024-05-08T08:54:23.499 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.499 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.499 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:23.499 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767848+0000", 2024-05-08T08:54:23.499 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052464668999999998 2024-05-08T08:54:23.499 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.499 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.499 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:23.500 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767927+0000", 2024-05-08T08:54:23.500 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9228999999999996e-05 2024-05-08T08:54:23.500 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.500 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.500 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:23.500 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768031+0000", 2024-05-08T08:54:23.500 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010417 2024-05-08T08:54:23.500 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:23.500 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:23.500 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:23.500 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.500 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.500 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6694 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1126400~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:23.500 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:54:23.500 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 111.778539099, 2024-05-08T08:54:23.501 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052848829, 2024-05-08T08:54:23.501 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:23.501 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:23.501 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:23.501 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:23.501 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:23.501 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:23.501 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6694 2024-05-08T08:54:23.501 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.501 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:23.501 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.501 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:23.501 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:54:23.501 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:23.501 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.501 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.501 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:23.502 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:54:23.502 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:23.502 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.502 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.502 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:23.502 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715390+0000", 2024-05-08T08:54:23.502 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9900000000000001e-07 2024-05-08T08:54:23.502 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.502 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.502 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:23.502 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715392+0000", 2024-05-08T08:54:23.502 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8279999999999999e-06 2024-05-08T08:54:23.502 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.502 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.502 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:23.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715398+0000", 2024-05-08T08:54:23.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7619999999999998e-06 2024-05-08T08:54:23.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.503 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:23.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715403+0000", 2024-05-08T08:54:23.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7790000000000003e-06 2024-05-08T08:54:23.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.503 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:23.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768067+0000", 2024-05-08T08:54:23.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052663785999999997 2024-05-08T08:54:23.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.503 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:23.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768142+0000", 2024-05-08T08:54:23.504 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5018999999999994e-05 2024-05-08T08:54:23.504 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.504 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.504 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:23.504 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768238+0000", 2024-05-08T08:54:23.504 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6255999999999995e-05 2024-05-08T08:54:23.504 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:23.504 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:23.504 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:23.504 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.504 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.504 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6696 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1142784~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:23.504 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:54:23.504 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 111.77851959100001, 2024-05-08T08:54:23.504 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.053051057999999998, 2024-05-08T08:54:23.505 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:23.505 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:23.505 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:23.505 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:23.505 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:23.505 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:23.505 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6696 2024-05-08T08:54:23.505 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.505 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:23.505 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.505 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:23.505 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:54:23.505 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:23.505 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.505 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.506 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:23.506 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:54:23.506 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:23.506 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.506 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.506 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:23.506 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:54:23.506 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-05-08T08:54:23.506 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.506 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.506 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:23.506 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715411+0000", 2024-05-08T08:54:23.506 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9630000000000001e-06 2024-05-08T08:54:23.506 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.506 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.507 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:23.507 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715417+0000", 2024-05-08T08:54:23.507 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7250000000000002e-06 2024-05-08T08:54:23.507 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.507 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.507 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:23.507 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715421+0000", 2024-05-08T08:54:23.507 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4089999999999996e-06 2024-05-08T08:54:23.507 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.507 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.507 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:23.507 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768277+0000", 2024-05-08T08:54:23.507 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052855293999999997 2024-05-08T08:54:23.507 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.507 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.508 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:23.508 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768358+0000", 2024-05-08T08:54:23.508 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1720999999999999e-05 2024-05-08T08:54:23.508 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.508 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.508 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:23.508 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768460+0000", 2024-05-08T08:54:23.508 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010154600000000001 2024-05-08T08:54:23.508 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:23.508 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:23.508 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:23.508 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.508 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.508 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6698 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1159168~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:23.508 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:54:23.508 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 111.778500724, 2024-05-08T08:54:23.509 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.053245726, 2024-05-08T08:54:23.509 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:23.509 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:23.509 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:23.509 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:23.509 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:23.509 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:23.509 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6698 2024-05-08T08:54:23.509 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.509 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:23.509 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.509 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:23.509 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:54:23.509 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:23.509 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.509 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.510 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:23.510 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:54:23.510 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:23.510 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.510 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.510 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:23.510 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716936+0000", 2024-05-08T08:54:23.510 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001508665 2024-05-08T08:54:23.510 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.510 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.510 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:23.510 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716941+0000", 2024-05-08T08:54:23.510 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1910000000000002e-06 2024-05-08T08:54:23.510 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.510 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.510 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:23.511 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716950+0000", 2024-05-08T08:54:23.511 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1900000000000001e-06 2024-05-08T08:54:23.511 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.511 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.511 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:23.511 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716958+0000", 2024-05-08T08:54:23.511 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8019999999999992e-06 2024-05-08T08:54:23.511 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.511 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.511 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:23.511 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768499+0000", 2024-05-08T08:54:23.511 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051540926000000001 2024-05-08T08:54:23.511 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.511 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.511 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:23.511 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768574+0000", 2024-05-08T08:54:23.511 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4988999999999996e-05 2024-05-08T08:54:23.512 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.512 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.512 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:23.512 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768673+0000", 2024-05-08T08:54:23.512 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9963000000000005e-05 2024-05-08T08:54:23.512 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:23.512 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:23.512 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:23.512 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.512 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.512 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6706 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1224704~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:23.512 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:54:23.512 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 111.77689541700001, 2024-05-08T08:54:23.512 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052435598, 2024-05-08T08:54:23.512 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:23.512 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:23.512 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:23.513 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:23.513 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:23.513 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:23.513 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6706 2024-05-08T08:54:23.513 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.513 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:23.513 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.513 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:23.513 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:54:23.513 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:23.513 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.513 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.513 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:23.513 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:54:23.513 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:23.513 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.514 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.514 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:23.514 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:54:23.514 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:54:23.514 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.514 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.514 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:23.514 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717035+0000", 2024-05-08T08:54:23.514 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9290000000000001e-06 2024-05-08T08:54:23.514 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.514 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.514 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:23.514 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717041+0000", 2024-05-08T08:54:23.514 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6720000000000001e-06 2024-05-08T08:54:23.515 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.515 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.515 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:23.515 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717046+0000", 2024-05-08T08:54:23.515 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5650000000000003e-06 2024-05-08T08:54:23.515 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.515 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.515 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:23.515 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769311+0000", 2024-05-08T08:54:23.515 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052265645999999999 2024-05-08T08:54:23.515 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.515 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.515 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:23.515 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769379+0000", 2024-05-08T08:54:23.515 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7391000000000006e-05 2024-05-08T08:54:23.515 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.515 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.516 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:23.516 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769469+0000", 2024-05-08T08:54:23.516 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0025999999999995e-05 2024-05-08T08:54:23.516 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:23.516 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:23.516 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:23.516 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.516 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.516 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6708 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1241088~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:23.516 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:54:23.516 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 111.776877313, 2024-05-08T08:54:23.516 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052640552, 2024-05-08T08:54:23.516 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:23.516 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:23.516 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:23.516 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:23.517 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:23.517 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:23.517 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6708 2024-05-08T08:54:23.517 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.517 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:23.517 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.517 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:23.517 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:54:23.517 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:23.517 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.517 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.517 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:23.517 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:54:23.517 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:23.517 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.517 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.518 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:23.518 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717052+0000", 2024-05-08T08:54:23.518 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:54:23.518 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.518 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.518 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:23.518 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717053+0000", 2024-05-08T08:54:23.518 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8330000000000001e-06 2024-05-08T08:54:23.518 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.518 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.518 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:23.518 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717058+0000", 2024-05-08T08:54:23.518 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.933e-06 2024-05-08T08:54:23.518 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.519 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.519 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:23.519 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717063+0000", 2024-05-08T08:54:23.519 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2180000000000001e-06 2024-05-08T08:54:23.519 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.519 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.519 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:23.519 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769507+0000", 2024-05-08T08:54:23.519 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052444866999999999 2024-05-08T08:54:23.519 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.519 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.519 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:23.519 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769589+0000", 2024-05-08T08:54:23.519 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2039000000000005e-05 2024-05-08T08:54:23.519 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.519 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.520 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:23.520 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769692+0000", 2024-05-08T08:54:23.520 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102252 2024-05-08T08:54:23.520 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:23.520 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:23.520 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:23.520 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.520 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.520 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6710 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1257472~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:23.520 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:54:23.520 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 111.776859289, 2024-05-08T08:54:23.520 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.05283057, 2024-05-08T08:54:23.520 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:23.520 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:23.520 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:23.520 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:23.521 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:23.521 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:23.521 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6710 2024-05-08T08:54:23.521 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.521 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:23.521 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.521 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:23.521 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:54:23.521 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:23.521 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.521 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.521 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:23.521 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:54:23.521 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:23.521 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.522 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.522 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:23.522 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718951+0000", 2024-05-08T08:54:23.522 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0018817809999999999 2024-05-08T08:54:23.522 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.522 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.522 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:23.522 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718956+0000", 2024-05-08T08:54:23.522 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5399999999999997e-06 2024-05-08T08:54:23.522 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.522 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.522 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:23.522 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718964+0000", 2024-05-08T08:54:23.522 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6629999999999999e-06 2024-05-08T08:54:23.522 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.523 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.523 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:23.523 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718972+0000", 2024-05-08T08:54:23.523 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5229999999999998e-06 2024-05-08T08:54:23.523 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.523 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.523 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:23.523 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769729+0000", 2024-05-08T08:54:23.523 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050756959999999997 2024-05-08T08:54:23.523 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.523 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.523 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:23.523 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769803+0000", 2024-05-08T08:54:23.523 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4538999999999999e-05 2024-05-08T08:54:23.523 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.523 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.524 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:23.524 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769900+0000", 2024-05-08T08:54:23.524 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6564000000000006e-05 2024-05-08T08:54:23.524 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:23.524 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:23.524 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:23.524 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.524 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.524 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6726 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1388544~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:23.524 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:54:23.524 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 111.774784517, 2024-05-08T08:54:23.524 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052451506000000002, 2024-05-08T08:54:23.524 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:23.524 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:23.524 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:23.524 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:23.525 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:23.525 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:23.525 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6726 2024-05-08T08:54:23.525 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.525 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:23.525 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.525 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:23.525 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:54:23.525 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:23.525 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.525 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.525 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:23.525 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:54:23.525 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:23.525 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.526 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.526 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:23.526 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719200+0000", 2024-05-08T08:54:23.526 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5865999999999997e-05 2024-05-08T08:54:23.526 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.526 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.526 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:23.526 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719204+0000", 2024-05-08T08:54:23.526 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8630000000000001e-06 2024-05-08T08:54:23.526 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.526 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.526 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:23.526 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719215+0000", 2024-05-08T08:54:23.526 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1590000000000001e-05 2024-05-08T08:54:23.526 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.527 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.527 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:23.527 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719220+0000", 2024-05-08T08:54:23.527 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1619999999999999e-06 2024-05-08T08:54:23.527 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.527 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.527 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:23.527 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771425+0000", 2024-05-08T08:54:23.527 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052204249000000001 2024-05-08T08:54:23.527 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.527 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.527 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:23.527 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771498+0000", 2024-05-08T08:54:23.527 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3782999999999995e-05 2024-05-08T08:54:23.527 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.527 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.528 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:23.528 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771595+0000", 2024-05-08T08:54:23.528 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6992999999999998e-05 2024-05-08T08:54:23.528 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:23.528 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:23.528 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:23.528 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.528 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.528 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6728 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1404928~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:23.528 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:54:23.528 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 111.774697112, 2024-05-08T08:54:23.528 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052590747, 2024-05-08T08:54:23.528 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:23.528 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:23.528 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:23.528 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:23.529 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:23.529 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:23.529 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6728 2024-05-08T08:54:23.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.529 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:23.529 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.529 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:23.529 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:54:23.529 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:23.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.529 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.529 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:23.529 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:54:23.529 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:23.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.530 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.530 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:23.530 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719232+0000", 2024-05-08T08:54:23.530 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4000000000000001e-07 2024-05-08T08:54:23.530 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.530 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.530 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:23.530 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719234+0000", 2024-05-08T08:54:23.530 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4499999999999998e-06 2024-05-08T08:54:23.530 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.530 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.530 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:23.530 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719240+0000", 2024-05-08T08:54:23.530 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5720000000000002e-06 2024-05-08T08:54:23.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.531 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.531 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:23.531 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719245+0000", 2024-05-08T08:54:23.531 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8690000000000003e-06 2024-05-08T08:54:23.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.531 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.531 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:23.531 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771636+0000", 2024-05-08T08:54:23.531 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052390979999999997 2024-05-08T08:54:23.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.531 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.531 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:23.531 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771717+0000", 2024-05-08T08:54:23.531 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1291999999999993e-05 2024-05-08T08:54:23.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.532 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.532 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:23.532 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771822+0000", 2024-05-08T08:54:23.532 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104944 2024-05-08T08:54:23.532 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:23.532 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:23.532 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:23.532 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.532 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.532 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6730 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1421312~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:23.532 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:54:23.532 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 111.77467686999999, 2024-05-08T08:54:23.532 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052770477000000003, 2024-05-08T08:54:23.532 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:23.532 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:23.533 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:23.533 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:23.533 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:23.533 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:23.533 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6730 2024-05-08T08:54:23.533 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.533 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:23.533 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.533 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:23.533 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:54:23.533 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:23.533 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.533 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.533 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:23.533 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:54:23.534 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:23.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.534 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.534 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:23.534 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:54:23.534 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9000000000000002e-07 2024-05-08T08:54:23.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.534 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.534 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:23.534 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719254+0000", 2024-05-08T08:54:23.534 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4099999999999998e-06 2024-05-08T08:54:23.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.534 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.534 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:23.534 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719263+0000", 2024-05-08T08:54:23.535 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4109999999999996e-06 2024-05-08T08:54:23.535 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.535 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.535 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:23.535 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719268+0000", 2024-05-08T08:54:23.535 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2669999999999997e-06 2024-05-08T08:54:23.535 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.535 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.535 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:23.535 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771846+0000", 2024-05-08T08:54:23.535 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052577677000000003 2024-05-08T08:54:23.535 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.535 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.535 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:23.535 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771920+0000", 2024-05-08T08:54:23.536 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4460000000000002e-05 2024-05-08T08:54:23.536 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.536 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.536 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:23.536 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772022+0000", 2024-05-08T08:54:23.536 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010186199999999999 2024-05-08T08:54:23.536 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:23.536 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:23.536 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:23.536 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.536 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.536 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6732 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1437696~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:23.536 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:54:23.536 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 111.774652063, 2024-05-08T08:54:23.536 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052905575000000003, 2024-05-08T08:54:23.536 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:23.537 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:23.537 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:23.537 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:23.537 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:23.537 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:23.537 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6732 2024-05-08T08:54:23.537 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.537 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:23.537 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.537 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:23.537 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:54:23.537 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:23.537 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.537 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.537 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:23.537 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:54:23.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:23.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.538 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:23.538 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721392+0000", 2024-05-08T08:54:23.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002115287 2024-05-08T08:54:23.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.538 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:23.538 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721396+0000", 2024-05-08T08:54:23.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1679999999999997e-06 2024-05-08T08:54:23.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.538 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:23.538 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721404+0000", 2024-05-08T08:54:23.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4409999999999993e-06 2024-05-08T08:54:23.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.539 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.539 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:23.539 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721410+0000", 2024-05-08T08:54:23.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9150000000000001e-06 2024-05-08T08:54:23.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.539 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.539 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:23.539 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772061+0000", 2024-05-08T08:54:23.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050651068 2024-05-08T08:54:23.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.539 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.539 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:23.539 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772127+0000", 2024-05-08T08:54:23.540 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5424000000000002e-05 2024-05-08T08:54:23.540 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.540 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.540 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:23.540 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772182+0000", 2024-05-08T08:54:23.540 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5272000000000002e-05 2024-05-08T08:54:23.540 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:23.540 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:23.540 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:23.540 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.540 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.540 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6749 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1585152~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:23.540 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:54:23.540 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 111.772351293, 2024-05-08T08:54:23.540 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052350264000000001, 2024-05-08T08:54:23.540 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:23.541 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:23.541 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:23.541 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:23.541 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:23.541 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:23.541 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6749 2024-05-08T08:54:23.541 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.541 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:23.541 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.541 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:23.541 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:54:23.541 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:23.541 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.541 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.541 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:23.541 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:54:23.542 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:23.542 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.542 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.542 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:23.542 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721578+0000", 2024-05-08T08:54:23.542 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0399999999999996e-07 2024-05-08T08:54:23.542 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.542 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.542 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:23.542 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721580+0000", 2024-05-08T08:54:23.542 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5450000000000002e-06 2024-05-08T08:54:23.542 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.542 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.542 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:23.543 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721587+0000", 2024-05-08T08:54:23.543 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0190000000000001e-06 2024-05-08T08:54:23.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.543 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:23.543 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721593+0000", 2024-05-08T08:54:23.543 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5659999999999997e-06 2024-05-08T08:54:23.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.543 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:23.543 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773759+0000", 2024-05-08T08:54:23.543 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052166553999999997 2024-05-08T08:54:23.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.543 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:23.544 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773833+0000", 2024-05-08T08:54:23.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3913999999999997e-05 2024-05-08T08:54:23.544 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.544 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.544 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:23.544 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773927+0000", 2024-05-08T08:54:23.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4061999999999995e-05 2024-05-08T08:54:23.544 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:23.544 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:23.544 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:23.544 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.544 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.544 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6751 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1601536~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:23.544 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:54:23.544 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 111.772328014, 2024-05-08T08:54:23.545 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052461302000000001, 2024-05-08T08:54:23.545 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:23.545 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:23.545 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:23.545 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:23.545 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:23.545 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:23.545 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6751 2024-05-08T08:54:23.545 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.545 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:23.545 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.545 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:23.545 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:54:23.545 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:23.545 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.546 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.546 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:23.546 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:54:23.546 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:23.546 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.546 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.546 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:23.546 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722939+0000", 2024-05-08T08:54:23.546 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001338601 2024-05-08T08:54:23.546 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.546 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.546 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:23.546 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722944+0000", 2024-05-08T08:54:23.546 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4469999999999999e-06 2024-05-08T08:54:23.546 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.547 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:23.547 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722955+0000", 2024-05-08T08:54:23.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1892999999999999e-05 2024-05-08T08:54:23.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.547 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:23.547 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722963+0000", 2024-05-08T08:54:23.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1080000000000004e-06 2024-05-08T08:54:23.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.547 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:23.547 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773964+0000", 2024-05-08T08:54:23.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051001022 2024-05-08T08:54:23.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:23.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.774006+0000", 2024-05-08T08:54:23.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2729999999999999e-05 2024-05-08T08:54:23.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:23.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.774062+0000", 2024-05-08T08:54:23.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5501000000000003e-05 2024-05-08T08:54:23.548 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:23.548 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:23.548 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:23.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.548 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6760 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1683456~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:23.548 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:54:23.549 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 111.77081863799999, 2024-05-08T08:54:23.549 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052571312000000002, 2024-05-08T08:54:23.549 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:23.549 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:23.549 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:23.549 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:23.549 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:23.549 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:23.549 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6760 2024-05-08T08:54:23.549 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.549 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:23.549 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.549 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:23.549 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:54:23.549 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:23.550 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.550 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.550 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:23.550 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:54:23.550 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:23.550 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.550 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.550 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:23.550 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:54:23.550 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2100000000000002e-07 2024-05-08T08:54:23.550 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.550 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.550 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:23.550 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723113+0000", 2024-05-08T08:54:23.550 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3149999999999999e-06 2024-05-08T08:54:23.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.551 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:23.551 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723118+0000", 2024-05-08T08:54:23.551 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8860000000000001e-06 2024-05-08T08:54:23.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.551 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:23.551 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723123+0000", 2024-05-08T08:54:23.551 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6770000000000003e-06 2024-05-08T08:54:23.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.551 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:23.551 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775414+0000", 2024-05-08T08:54:23.551 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052290692999999999 2024-05-08T08:54:23.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.552 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.552 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:23.552 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775459+0000", 2024-05-08T08:54:23.552 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5639999999999997e-05 2024-05-08T08:54:23.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.552 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.552 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:23.552 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775681+0000", 2024-05-08T08:54:23.552 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00022168000000000001 2024-05-08T08:54:23.552 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:23.552 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:23.552 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:23.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.552 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.552 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6761 3.16 3:6bd6d6da:::rbd_data.1140782590a5.0000000000000015:6 [read 1095680~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:23.553 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:54:23.553 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 111.770799418, 2024-05-08T08:54:23.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052751275, 2024-05-08T08:54:23.553 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:23.553 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:23.553 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:23.553 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:23.553 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:23.553 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:23.553 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6761 2024-05-08T08:54:23.553 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.553 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:23.553 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.553 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:23.553 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:54:23.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:23.554 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.554 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.554 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:23.554 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:54:23.554 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:23.554 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.554 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.554 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:23.554 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724939+0000", 2024-05-08T08:54:23.554 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001809817 2024-05-08T08:54:23.554 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.554 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.554 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:23.554 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724943+0000", 2024-05-08T08:54:23.555 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4920000000000004e-06 2024-05-08T08:54:23.555 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.555 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.555 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:23.555 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724956+0000", 2024-05-08T08:54:23.555 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2427e-05 2024-05-08T08:54:23.555 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.555 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.555 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:23.555 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724963+0000", 2024-05-08T08:54:23.555 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2559999999999996e-06 2024-05-08T08:54:23.555 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.555 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.556 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:23.556 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775711+0000", 2024-05-08T08:54:23.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050747621 2024-05-08T08:54:23.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.556 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.556 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:23.556 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775785+0000", 2024-05-08T08:54:23.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4634000000000004e-05 2024-05-08T08:54:23.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.556 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.556 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:23.556 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775880+0000", 2024-05-08T08:54:23.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5027999999999999e-05 2024-05-08T08:54:23.556 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:23.556 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:23.556 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:23.557 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.557 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.557 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6770 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1765376~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:23.557 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:54:23.557 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 111.768768323, 2024-05-08T08:54:23.557 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052339296, 2024-05-08T08:54:23.557 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:23.557 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:23.557 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:23.557 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:23.557 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:23.557 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:23.557 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6770 2024-05-08T08:54:23.557 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.558 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:23.558 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.558 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:23.558 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:54:23.558 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:23.558 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.558 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.558 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:23.558 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:54:23.558 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:23.558 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.558 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.558 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:23.558 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725161+0000", 2024-05-08T08:54:23.558 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-05-08T08:54:23.559 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.559 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.559 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:23.559 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725163+0000", 2024-05-08T08:54:23.559 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4370000000000001e-06 2024-05-08T08:54:23.559 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.559 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.559 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:23.559 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725169+0000", 2024-05-08T08:54:23.559 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8409999999999996e-06 2024-05-08T08:54:23.559 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.559 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.559 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:23.559 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725174+0000", 2024-05-08T08:54:23.559 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7249999999999997e-06 2024-05-08T08:54:23.560 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.560 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.560 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:23.560 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777343+0000", 2024-05-08T08:54:23.560 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052169633999999999 2024-05-08T08:54:23.560 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.560 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.560 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:23.560 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777409+0000", 2024-05-08T08:54:23.560 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6033999999999998e-05 2024-05-08T08:54:23.560 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.560 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.560 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:23.560 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777499+0000", 2024-05-08T08:54:23.560 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0224999999999998e-05 2024-05-08T08:54:23.560 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:23.561 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:23.561 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:23.561 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.561 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.561 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6771 3.16 3:6bd6d6da:::rbd_data.1140782590a5.0000000000000015:6 [read 1177600~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:23.561 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:54:23.561 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 111.76874850599999, 2024-05-08T08:54:23.561 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052438604999999999, 2024-05-08T08:54:23.561 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:23.561 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:23.561 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:23.561 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:23.561 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:23.561 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:23.561 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6771 2024-05-08T08:54:23.562 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.562 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:23.562 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.562 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:23.562 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:54:23.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:23.562 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.562 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.562 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:23.562 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:54:23.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:23.562 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.562 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.562 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:23.562 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:54:23.563 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:54:23.563 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.563 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.563 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:23.563 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725188+0000", 2024-05-08T08:54:23.563 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1579999999999999e-06 2024-05-08T08:54:23.563 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.563 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.563 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:23.563 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725195+0000", 2024-05-08T08:54:23.563 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8909999999999995e-06 2024-05-08T08:54:23.563 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.563 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.563 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:23.563 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725200+0000", 2024-05-08T08:54:23.563 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9810000000000003e-06 2024-05-08T08:54:23.564 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.564 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.564 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:23.564 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777527+0000", 2024-05-08T08:54:23.564 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052326582000000003 2024-05-08T08:54:23.564 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.564 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.564 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:23.564 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777566+0000", 2024-05-08T08:54:23.564 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9447999999999999e-05 2024-05-08T08:54:23.564 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.564 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.564 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:23.564 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777619+0000", 2024-05-08T08:54:23.564 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2135000000000001e-05 2024-05-08T08:54:23.565 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:23.565 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:23.565 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:23.565 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.565 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.565 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6772 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1781760~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:23.565 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:54:23.565 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 111.768720449, 2024-05-08T08:54:23.565 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052601799999999997, 2024-05-08T08:54:23.565 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:23.565 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:23.565 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:23.565 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:23.565 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:23.565 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:23.566 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6772 2024-05-08T08:54:23.566 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.566 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:23.566 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.566 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:23.566 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:54:23.566 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:23.566 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.566 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.566 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:23.566 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:54:23.566 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:23.566 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.566 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.566 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:23.567 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726939+0000", 2024-05-08T08:54:23.567 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001730493 2024-05-08T08:54:23.567 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.567 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.567 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:23.567 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726943+0000", 2024-05-08T08:54:23.567 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7330000000000003e-06 2024-05-08T08:54:23.567 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.567 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.567 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:23.567 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726954+0000", 2024-05-08T08:54:23.567 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1107000000000001e-05 2024-05-08T08:54:23.567 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.567 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.567 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:23.567 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726962+0000", 2024-05-08T08:54:23.568 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.503e-06 2024-05-08T08:54:23.568 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.568 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.568 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:23.568 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777653+0000", 2024-05-08T08:54:23.568 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050691542999999999 2024-05-08T08:54:23.568 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.568 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.568 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:23.568 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777719+0000", 2024-05-08T08:54:23.568 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5228000000000006e-05 2024-05-08T08:54:23.568 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.568 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.568 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:23.568 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777810+0000", 2024-05-08T08:54:23.569 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1193000000000006e-05 2024-05-08T08:54:23.569 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:23.569 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:23.569 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:23.569 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.569 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.569 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6782 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1880064~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:23.569 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:54:23.569 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 111.766751942, 2024-05-08T08:54:23.569 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052326492000000002, 2024-05-08T08:54:23.569 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:23.569 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:23.569 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:23.569 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:23.569 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:23.569 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:23.570 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6782 2024-05-08T08:54:23.570 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.570 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:23.570 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.570 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:23.570 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:54:23.570 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:23.570 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.570 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.570 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:23.570 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:54:23.570 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:23.570 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.570 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.570 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:23.571 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728939+0000", 2024-05-08T08:54:23.571 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001762719 2024-05-08T08:54:23.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.571 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.571 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:23.571 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728944+0000", 2024-05-08T08:54:23.571 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.746e-06 2024-05-08T08:54:23.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.571 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.571 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:23.571 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728955+0000", 2024-05-08T08:54:23.571 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1043000000000001e-05 2024-05-08T08:54:23.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.571 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.571 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:23.572 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728963+0000", 2024-05-08T08:54:23.572 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5809999999999999e-06 2024-05-08T08:54:23.572 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.572 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.572 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:23.572 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779320+0000", 2024-05-08T08:54:23.572 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050357290999999998 2024-05-08T08:54:23.572 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.572 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.572 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:23.572 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779401+0000", 2024-05-08T08:54:23.572 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0574000000000005e-05 2024-05-08T08:54:23.572 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:23.572 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:23.572 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:23.572 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779503+0000", 2024-05-08T08:54:23.573 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102538 2024-05-08T08:54:23.573 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:23.573 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:23.573 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:23.573 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:23.573 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:23.573 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:23.574 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:54:23.574 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:54:23.575 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:54:23.575 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:54:23.576 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:23.577+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:23.576 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:54:23.576 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:54:23.577 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:54:23.577 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:54:23.577 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:54:23.593 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:54:23.594 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:54:23.611 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:54:23.611 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:54:23.675 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:23.678+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:23.688 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:23.688 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:23.696 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:54:23.751 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-05-08T08:54:23.781 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:23.784+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:23.829 INFO:teuthology.orchestra.run.smithi196.stdout:truncating image image_client.0-clone46 from 0xd5ea00 (overlap 0xd5ea00) to 0x95bc00 2024-05-08T08:54:23.858 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:23.858 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:23.868 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:54:23.877 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:23.880+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:23.978 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:23.981+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:23.982 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:23.982 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:23.992 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:54:24.079 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:24.082+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:24.104 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:24.105 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:24.114 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:54:24.120 INFO:teuthology.orchestra.run.smithi196.stdout:checking clone #45, image image_client.0-clone45 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent46 2024-05-08T08:54:24.198 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:24.201+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:24.266 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:24.266 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:24.275 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:54:24.280 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:24.283+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:24.381 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:24.385+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:24.425 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:24.425 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:24.434 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:54:24.477 INFO:teuthology.orchestra.run.smithi196.stdout:5794 read 0x3f9600 thru 0x40144d (0x7e4e bytes) 2024-05-08T08:54:24.479 INFO:teuthology.orchestra.run.smithi196.stdout:5795 trunc from 0xd5ea00 to 0x252e600 2024-05-08T08:54:24.482 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:24.485+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:24.498 INFO:teuthology.orchestra.run.smithi196.stdout:5796 trunc from 0x252e600 to 0x67eae00 2024-05-08T08:54:24.508 INFO:teuthology.orchestra.run.smithi196.stdout:5797 read 0x5805600 thru 0x5810bf6 (0xb5f7 bytes) 2024-05-08T08:54:24.508 INFO:teuthology.orchestra.run.smithi196.stdout:5799 read 0x7a2200 thru 0x7a4022 (0x1e23 bytes) 2024-05-08T08:54:24.513 INFO:teuthology.orchestra.run.smithi196.stdout:5801 punch from 0x524dc00 to 0x5255800, (0x7c00 bytes) 2024-05-08T08:54:24.513 INFO:teuthology.orchestra.run.smithi196.stdout:5802 punch from 0xce6400 to 0xcebe00, (0x5a00 bytes) 2024-05-08T08:54:24.516 INFO:teuthology.orchestra.run.smithi196.stdout:5803 trunc from 0x67eae00 to 0x4918600 2024-05-08T08:54:24.534 INFO:teuthology.orchestra.run.smithi196.stdout:5804 read 0x277e400 thru 0x27847f7 (0x63f8 bytes) 2024-05-08T08:54:24.540 INFO:teuthology.orchestra.run.smithi196.stdout:5805 trunc from 0x4918600 to 0x9485c00 2024-05-08T08:54:24.547 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:24.547 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:24.548 INFO:teuthology.orchestra.run.smithi196.stdout:5808 punch from 0x5ea2000 to 0x5ea9600, (0x7600 bytes) 2024-05-08T08:54:24.551 INFO:teuthology.orchestra.run.smithi196.stdout:5809 write 0x6c19000 thru 0x6c1bed9 (0x2eda bytes) 2024-05-08T08:54:24.551 INFO:teuthology.orchestra.run.smithi196.stdout:5810 read 0x88eba00 thru 0x88f8110 (0xc711 bytes) 2024-05-08T08:54:24.551 INFO:teuthology.orchestra.run.smithi196.stdout:5815 punch from 0xab0800 to 0xaba800, (0xa000 bytes) 2024-05-08T08:54:24.551 INFO:teuthology.orchestra.run.smithi196.stdout:5816 punch from 0x8624600 to 0x8624e00, (0x800 bytes) 2024-05-08T08:54:24.552 INFO:teuthology.orchestra.run.smithi196.stdout:5817 read 0x14e7e00 thru 0x14f4cf9 (0xcefa bytes) 2024-05-08T08:54:24.552 INFO:teuthology.orchestra.run.smithi196.stdout:5818 read 0x91ee800 thru 0x91f60cb (0x78cc bytes) 2024-05-08T08:54:24.552 INFO:teuthology.orchestra.run.smithi196.stdout:5820 read 0x6b2dc00 thru 0x6b37b64 (0x9f65 bytes) 2024-05-08T08:54:24.557 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:54:24.561 INFO:teuthology.orchestra.run.smithi196.stdout:5821 trunc from 0x9485c00 to 0x16dd000 2024-05-08T08:54:24.576 INFO:teuthology.orchestra.run.smithi196.stdout:5823 read 0x29fc00 thru 0x2a2681 (0x2a82 bytes) 2024-05-08T08:54:24.577 INFO:teuthology.orchestra.run.smithi196.stdout:5825 punch from 0x15c00 to 0x1b000, (0x5400 bytes) 2024-05-08T08:54:24.597 INFO:teuthology.orchestra.run.smithi196.stdout:5828 write 0x29daa00 thru 0x29dbcd3 (0x12d4 bytes) 2024-05-08T08:54:24.597 INFO:teuthology.orchestra.run.smithi196.stdout:5830 read 0x65b000 thru 0x66314e (0x814f bytes) 2024-05-08T08:54:24.599 INFO:teuthology.orchestra.run.smithi196.stdout:5831 trunc from 0x29dbe00 to 0x1571200 2024-05-08T08:54:24.601 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:24.604+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:24.616 INFO:teuthology.orchestra.run.smithi196.stdout:5833 punch from 0xca4a00 to 0xcac000, (0x7600 bytes) 2024-05-08T08:54:24.635 INFO:teuthology.orchestra.run.smithi196.stdout:5834 write 0x7144600 thru 0x7149f8e (0x598f bytes) 2024-05-08T08:54:24.636 INFO:teuthology.orchestra.run.smithi196.stdout:5838 clone 48 order 19 su 4096 sc 11 2024-05-08T08:54:24.673 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:24.673 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:24.681 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:54:24.684 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:24.687+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:24.750 INFO:teuthology.orchestra.run.smithi059.stderr:noscrub is set 2024-05-08T08:54:24.806 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:24.809+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:24.828 INFO:teuthology.orchestra.run.smithi196.stdout:truncating image image_client.0-clone47 from 0x714a000 (overlap 0xd5ea00) to 0x805800 2024-05-08T08:54:24.839 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:24.839 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:24.850 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:54:24.894 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:24.897+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:24.960 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:24.960 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:24.969 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:54:25.011 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:25.014+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:25.088 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:25.090+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:25.120 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:25.120 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:25.130 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:54:25.156 INFO:teuthology.orchestra.run.smithi196.stdout:checking clone #46, image image_client.0-clone46 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent47 2024-05-08T08:54:25.190 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:25.193+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:25.240 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:25.240 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:25.245 INFO:teuthology.orchestra.run.smithi196.stdout:5840 read 0x28dac00 thru 0x28dee0c (0x420d bytes) 2024-05-08T08:54:25.249 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:54:25.252 INFO:teuthology.orchestra.run.smithi196.stdout:5844 trunc from 0x714a000 to 0xbae4200 2024-05-08T08:54:25.263 INFO:teuthology.orchestra.run.smithi196.stdout:5845 read 0x60c2600 thru 0x60c6303 (0x3d04 bytes) 2024-05-08T08:54:25.264 INFO:teuthology.orchestra.run.smithi196.stdout:5846 write 0x21f6a00 thru 0x21fba24 (0x5025 bytes) 2024-05-08T08:54:25.265 INFO:teuthology.orchestra.run.smithi196.stdout:5847 write 0x2f0b400 thru 0x2f1590b (0xa50c bytes) 2024-05-08T08:54:25.265 INFO:teuthology.orchestra.run.smithi196.stdout:5850 punch from 0x3549a00 to 0x3554800, (0xae00 bytes) 2024-05-08T08:54:25.279 INFO:teuthology.orchestra.run.smithi196.stdout:5851 trunc from 0xbae4200 to 0x5e81200 2024-05-08T08:54:25.291 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:25.293+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:25.326 INFO:teuthology.orchestra.run.smithi196.stdout:5852 write 0x2503a00 thru 0x250d970 (0x9f71 bytes) 2024-05-08T08:54:25.361 INFO:teuthology.orchestra.run.smithi196.stdout:5856 write 0x91c6c00 thru 0x91cdf40 (0x7341 bytes) 2024-05-08T08:54:25.387 INFO:teuthology.orchestra.run.smithi196.stdout:5857 write 0x938b200 thru 0x939799d (0xc79e bytes) 2024-05-08T08:54:25.388 INFO:teuthology.orchestra.run.smithi196.stdout:5858 read 0x37a3000 thru 0x37a3b96 (0xb97 bytes) 2024-05-08T08:54:25.388 INFO:teuthology.orchestra.run.smithi196.stdout:5860 read 0x90e400 thru 0x919122 (0xad23 bytes) 2024-05-08T08:54:25.391 INFO:teuthology.orchestra.run.smithi196.stdout:5861 read 0x6e45a00 thru 0x6e511be (0xb7bf bytes) 2024-05-08T08:54:25.391 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:25.394+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:25.409 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:25.410 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:25.411 INFO:teuthology.orchestra.run.smithi196.stdout:5862 write 0xbdb2000 thru 0xbdbd629 (0xb62a bytes) 2024-05-08T08:54:25.412 INFO:teuthology.orchestra.run.smithi196.stdout:5866 write 0x424ac00 thru 0x424eb70 (0x3f71 bytes) 2024-05-08T08:54:25.412 INFO:teuthology.orchestra.run.smithi196.stdout:5867 punch from 0x6b21a00 to 0x6b23400, (0x1a00 bytes) 2024-05-08T08:54:25.414 INFO:teuthology.orchestra.run.smithi196.stdout:5869 read 0x5e5c400 thru 0x5e6a4fc (0xe0fd bytes) 2024-05-08T08:54:25.417 INFO:teuthology.orchestra.run.smithi196.stdout:5870 read 0x1676a00 thru 0x1686400 (0xfa01 bytes) 2024-05-08T08:54:25.417 INFO:teuthology.orchestra.run.smithi196.stdout:5871 write 0x1dc3800 thru 0x1dc3c69 (0x46a bytes) 2024-05-08T08:54:25.417 INFO:teuthology.orchestra.run.smithi196.stdout:5872 punch from 0x2d4c00 to 0x2e4200, (0xf600 bytes) 2024-05-08T08:54:25.420 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:54:25.437 INFO:teuthology.orchestra.run.smithi196.stdout:5873 punch from 0x8e49c00 to 0x8e4be00, (0x2200 bytes) 2024-05-08T08:54:25.438 INFO:teuthology.orchestra.run.smithi196.stdout:5875 write 0x2afc400 thru 0x2afc40c (0xd bytes) 2024-05-08T08:54:25.439 INFO:teuthology.orchestra.run.smithi196.stdout:5877 write 0xb5fe400 thru 0xb602e79 (0x4a7a bytes) 2024-05-08T08:54:25.439 INFO:teuthology.orchestra.run.smithi196.stdout:5878 read 0x64fac00 thru 0x64fb05f (0x460 bytes) 2024-05-08T08:54:25.442 INFO:teuthology.orchestra.run.smithi196.stdout:5879 trunc from 0xbdbd800 to 0x96f5800 2024-05-08T08:54:25.504 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:25.507+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:25.536 INFO:teuthology.orchestra.run.smithi196.stdout:5880 write 0x1b02400 thru 0x1b0808e (0x5c8f bytes) 2024-05-08T08:54:25.537 INFO:teuthology.orchestra.run.smithi196.stdout:5882 punch from 0x72b8400 to 0x72bf200, (0x6e00 bytes) 2024-05-08T08:54:25.540 INFO:teuthology.orchestra.run.smithi196.stdout:5883 write 0x5797600 thru 0x579eadd (0x74de bytes) 2024-05-08T08:54:25.540 INFO:teuthology.orchestra.run.smithi196.stdout:5886 write 0x8b5ae00 thru 0x8b635a9 (0x87aa bytes) 2024-05-08T08:54:25.540 INFO:teuthology.orchestra.run.smithi196.stdout:5890 read 0x82d8000 thru 0x82e26e3 (0xa6e4 bytes) 2024-05-08T08:54:25.543 INFO:teuthology.orchestra.run.smithi196.stdout:5891 trunc from 0x96f5800 to 0xbab3200 2024-05-08T08:54:25.578 INFO:teuthology.orchestra.run.smithi196.stdout:5892 write 0x796b600 thru 0x7974af2 (0x94f3 bytes) 2024-05-08T08:54:25.585 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:25.585 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:25.589 INFO:teuthology.orchestra.run.smithi196.stdout:5894 trunc from 0xbab3200 to 0x8796600 2024-05-08T08:54:25.593 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:25.595+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:25.595 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:54:25.623 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T08:54:25.635 INFO:teuthology.orchestra.run.smithi196.stdout:5896 write 0x224000 thru 0x232784 (0xe785 bytes) 2024-05-08T08:54:25.658 INFO:teuthology.orchestra.run.smithi196.stdout:5897 write 0xb64d400 thru 0xb64df04 (0xb05 bytes) 2024-05-08T08:54:25.677 INFO:teuthology.orchestra.run.smithi196.stdout:5898 write 0xe963400 thru 0xe965896 (0x2497 bytes) 2024-05-08T08:54:25.681 INFO:teuthology.orchestra.run.smithi196.stdout:5899 trunc from 0xe965a00 to 0xbe18c00 2024-05-08T08:54:25.694 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:25.696+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:25.707 INFO:teuthology.orchestra.run.smithi196.stdout:5900 trunc from 0xbe18c00 to 0xae63a00 2024-05-08T08:54:25.730 INFO:teuthology.orchestra.run.smithi196.stdout:5902 write 0x16f1600 thru 0x16fef30 (0xd931 bytes) 2024-05-08T08:54:25.731 INFO:teuthology.orchestra.run.smithi196.stdout:5904 read 0x75ace00 thru 0x75b0c05 (0x3e06 bytes) 2024-05-08T08:54:25.731 INFO:teuthology.orchestra.run.smithi196.stdout:5910 read 0x58f2e00 thru 0x58fc885 (0x9a86 bytes) 2024-05-08T08:54:25.731 INFO:teuthology.orchestra.run.smithi196.stdout:5913 read 0x969a600 thru 0x969b17b (0xb7c bytes) 2024-05-08T08:54:25.731 INFO:teuthology.orchestra.run.smithi196.stdout:5914 trunc from 0xae63a00 to 0xaec1600 2024-05-08T08:54:25.752 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:25.752 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:25.756 INFO:teuthology.orchestra.run.smithi196.stdout:5917 read 0x5a47400 thru 0x5a492e2 (0x1ee3 bytes) 2024-05-08T08:54:25.756 INFO:teuthology.orchestra.run.smithi196.stdout:5920 write 0x74b1400 thru 0x74bc27f (0xae80 bytes) 2024-05-08T08:54:25.762 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:54:25.766 INFO:teuthology.orchestra.run.smithi196.stdout:5921 trunc from 0xaec1600 to 0x35a0c00 2024-05-08T08:54:25.777 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:54:25.778 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:54:25.795 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:25.798+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:25.804 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:54:25.804 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:54:25.827 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:54:25.828 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:54:25.836 INFO:teuthology.orchestra.run.smithi196.stdout:5922 flatten 2024-05-08T08:54:25.858 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:54:25.858 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:54:25.880 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:54:25.881 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:54:25.896 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:25.899+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:25.902 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:25.902 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:25.909 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:54:25.909 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:54:25.915 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:54:25.976 INFO:teuthology.orchestra.run.smithi196.stdout:5923 punch from 0x1a67000 to 0x1a6ee00, (0x7e00 bytes) 2024-05-08T08:54:25.976 INFO:teuthology.orchestra.run.smithi196.stdout:5925 read 0x1d5a800 thru 0x1d61e35 (0x7636 bytes) 2024-05-08T08:54:25.991 INFO:teuthology.orchestra.run.smithi196.stdout:5926 write 0x3b8e400 thru 0x3b94981 (0x6582 bytes) 2024-05-08T08:54:25.998 INFO:teuthology.orchestra.run.smithi196.stdout:5928 punch from 0x2714a00 to 0x2716400, (0x1a00 bytes) 2024-05-08T08:54:25.999 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:26.002+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:26.025 INFO:teuthology.orchestra.run.smithi196.stdout:5929 write 0x9441200 thru 0x944a98d (0x978e bytes) 2024-05-08T08:54:26.025 INFO:teuthology.orchestra.run.smithi196.stdout:5930 punch from 0x7f0c00 to 0x7ffc00, (0xf000 bytes) 2024-05-08T08:54:26.026 INFO:teuthology.orchestra.run.smithi196.stdout:5931 read 0x196be00 thru 0x19721b2 (0x63b3 bytes) 2024-05-08T08:54:26.062 INFO:teuthology.orchestra.run.smithi196.stdout:5932 write 0x9de9c00 thru 0x9df6b66 (0xcf67 bytes) 2024-05-08T08:54:26.063 INFO:teuthology.orchestra.run.smithi196.stdout:5933 trunc from 0x9df6c00 to 0x979a400 2024-05-08T08:54:26.069 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:26.069 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:54:26.070 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:54:26.070 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:54:26.070 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:26.080 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:54:26.106 INFO:teuthology.orchestra.run.smithi196.stdout:5935 trunc from 0x979a400 to 0x109b800 2024-05-08T08:54:26.121 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:26.124+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:26.178 INFO:teuthology.orchestra.run.smithi196.stdout:5936 write 0x5dd0e00 thru 0x5dd7442 (0x6643 bytes) 2024-05-08T08:54:26.199 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:26.202+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:26.213 INFO:teuthology.orchestra.run.smithi196.stdout:5938 write 0xe62e800 thru 0xe635ed8 (0x76d9 bytes) 2024-05-08T08:54:26.215 INFO:teuthology.orchestra.run.smithi196.stdout:5939 trunc from 0xe636000 to 0xe2a0200 2024-05-08T08:54:26.235 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:26.235 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:54:26.235 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:54:26.235 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:54:26.236 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.236 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6690 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1093632~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:26.236 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:54:26.236 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 114.52214094, 2024-05-08T08:54:26.236 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052462754, 2024-05-08T08:54:26.236 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:26.236 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:26.236 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:26.236 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:26.236 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:26.236 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:26.236 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6690 2024-05-08T08:54:26.236 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.237 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:26.237 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.237 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:26.237 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:54:26.237 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:26.237 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.237 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.237 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:26.237 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:54:26.237 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:26.237 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.237 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.237 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:26.237 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715346+0000", 2024-05-08T08:54:26.237 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.84e-07 2024-05-08T08:54:26.238 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.238 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.238 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:26.238 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715348+0000", 2024-05-08T08:54:26.238 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.114e-06 2024-05-08T08:54:26.238 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.238 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.238 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:26.238 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715353+0000", 2024-05-08T08:54:26.238 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5720000000000002e-06 2024-05-08T08:54:26.238 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.238 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.238 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:26.238 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715358+0000", 2024-05-08T08:54:26.238 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7140000000000001e-06 2024-05-08T08:54:26.239 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.239 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.239 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:26.239 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767638+0000", 2024-05-08T08:54:26.239 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052279562000000002 2024-05-08T08:54:26.239 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.239 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.239 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:26.239 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767713+0000", 2024-05-08T08:54:26.239 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5652e-05 2024-05-08T08:54:26.239 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.239 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.239 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:26.239 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767808+0000", 2024-05-08T08:54:26.240 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4755999999999999e-05 2024-05-08T08:54:26.240 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:26.240 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:26.240 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:26.240 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.240 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.240 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6692 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1110016~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:26.240 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:54:26.240 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 114.52212185, 2024-05-08T08:54:26.240 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052666495000000001, 2024-05-08T08:54:26.240 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:26.240 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:26.240 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:26.240 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:26.240 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:26.241 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:26.241 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6692 2024-05-08T08:54:26.241 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.241 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:26.241 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.241 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:26.241 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:54:26.241 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:26.241 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.241 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.241 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:26.241 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:54:26.241 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:26.241 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.241 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.242 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:26.242 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:54:26.242 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8599999999999999e-07 2024-05-08T08:54:26.242 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.242 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.242 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:26.242 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715372+0000", 2024-05-08T08:54:26.242 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0990000000000001e-06 2024-05-08T08:54:26.242 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.242 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.242 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:26.242 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715378+0000", 2024-05-08T08:54:26.242 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0680000000000002e-06 2024-05-08T08:54:26.242 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.243 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.243 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:26.243 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715383+0000", 2024-05-08T08:54:26.243 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8740000000000003e-06 2024-05-08T08:54:26.243 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.243 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.243 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:26.243 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767848+0000", 2024-05-08T08:54:26.243 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052464668999999998 2024-05-08T08:54:26.243 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.243 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.243 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:26.243 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767927+0000", 2024-05-08T08:54:26.243 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9228999999999996e-05 2024-05-08T08:54:26.243 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.244 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.244 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:26.244 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768031+0000", 2024-05-08T08:54:26.244 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010417 2024-05-08T08:54:26.244 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:26.244 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:26.244 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:26.244 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.244 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.244 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6694 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1126400~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:26.244 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:54:26.244 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 114.522096953, 2024-05-08T08:54:26.244 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052848829, 2024-05-08T08:54:26.244 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:26.244 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:26.245 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:26.245 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:26.245 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:26.245 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:26.245 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6694 2024-05-08T08:54:26.245 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.245 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:26.245 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.245 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:26.245 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:54:26.245 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:26.245 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.245 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.245 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:26.245 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:54:26.246 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:26.246 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.246 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.246 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:26.246 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715390+0000", 2024-05-08T08:54:26.246 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9900000000000001e-07 2024-05-08T08:54:26.246 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.246 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.246 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:26.246 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715392+0000", 2024-05-08T08:54:26.246 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8279999999999999e-06 2024-05-08T08:54:26.246 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.246 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.246 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:26.247 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715398+0000", 2024-05-08T08:54:26.247 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7619999999999998e-06 2024-05-08T08:54:26.247 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.247 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.247 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:26.247 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715403+0000", 2024-05-08T08:54:26.247 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7790000000000003e-06 2024-05-08T08:54:26.247 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.247 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.247 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:26.247 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768067+0000", 2024-05-08T08:54:26.247 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052663785999999997 2024-05-08T08:54:26.247 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.247 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.247 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:26.248 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768142+0000", 2024-05-08T08:54:26.248 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5018999999999994e-05 2024-05-08T08:54:26.248 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.248 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.248 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:26.248 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768238+0000", 2024-05-08T08:54:26.248 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6255999999999995e-05 2024-05-08T08:54:26.248 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:26.248 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:26.248 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:26.248 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.248 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.248 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6696 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1142784~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:26.248 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:54:26.248 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 114.52207744499999, 2024-05-08T08:54:26.249 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.053051057999999998, 2024-05-08T08:54:26.249 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:26.249 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:26.249 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:26.249 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:26.249 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:26.249 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:26.249 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6696 2024-05-08T08:54:26.249 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.249 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:26.249 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.249 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:26.249 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:54:26.249 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:26.250 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.250 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.250 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:26.250 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:54:26.250 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:26.250 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.250 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.250 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:26.250 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:54:26.250 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-05-08T08:54:26.250 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.250 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.250 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:26.250 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715411+0000", 2024-05-08T08:54:26.250 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9630000000000001e-06 2024-05-08T08:54:26.251 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.251 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.251 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:26.251 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715417+0000", 2024-05-08T08:54:26.251 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7250000000000002e-06 2024-05-08T08:54:26.251 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.251 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.251 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:26.251 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715421+0000", 2024-05-08T08:54:26.251 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4089999999999996e-06 2024-05-08T08:54:26.251 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.251 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.251 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:26.251 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768277+0000", 2024-05-08T08:54:26.251 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052855293999999997 2024-05-08T08:54:26.252 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.252 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.252 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:26.252 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768358+0000", 2024-05-08T08:54:26.252 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1720999999999999e-05 2024-05-08T08:54:26.252 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.252 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.252 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:26.252 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768460+0000", 2024-05-08T08:54:26.252 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010154600000000001 2024-05-08T08:54:26.252 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:26.252 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:26.252 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:26.252 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.253 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.253 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6698 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1159168~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:26.253 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:54:26.253 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 114.522058578, 2024-05-08T08:54:26.253 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.053245726, 2024-05-08T08:54:26.253 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:26.253 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:26.253 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:26.253 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:26.253 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:26.253 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:26.253 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6698 2024-05-08T08:54:26.253 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.253 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:26.253 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.254 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:26.254 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:54:26.254 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:26.254 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.254 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.254 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:26.254 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:54:26.254 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:26.254 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.254 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.254 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:26.254 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716936+0000", 2024-05-08T08:54:26.254 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001508665 2024-05-08T08:54:26.254 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.254 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.255 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:26.255 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716941+0000", 2024-05-08T08:54:26.255 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1910000000000002e-06 2024-05-08T08:54:26.255 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.255 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.255 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:26.255 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716950+0000", 2024-05-08T08:54:26.255 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1900000000000001e-06 2024-05-08T08:54:26.255 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.255 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.255 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:26.255 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716958+0000", 2024-05-08T08:54:26.255 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8019999999999992e-06 2024-05-08T08:54:26.255 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.255 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.256 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:26.256 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768499+0000", 2024-05-08T08:54:26.256 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051540926000000001 2024-05-08T08:54:26.256 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.256 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.256 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:26.256 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768574+0000", 2024-05-08T08:54:26.256 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4988999999999996e-05 2024-05-08T08:54:26.256 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.256 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.256 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:26.256 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768673+0000", 2024-05-08T08:54:26.256 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9963000000000005e-05 2024-05-08T08:54:26.256 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:26.256 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:26.256 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:26.257 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.257 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.257 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6706 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1224704~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:26.257 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:54:26.257 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 114.52045327099999, 2024-05-08T08:54:26.257 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052435598, 2024-05-08T08:54:26.257 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:26.257 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:26.257 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:26.257 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:26.257 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:26.257 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:26.257 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6706 2024-05-08T08:54:26.257 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.257 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:26.258 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.258 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:26.258 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:54:26.258 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:26.258 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.258 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.258 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:26.258 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:54:26.258 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:26.258 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.258 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.258 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:26.259 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:54:26.259 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:54:26.259 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.259 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.259 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:26.259 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717035+0000", 2024-05-08T08:54:26.259 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9290000000000001e-06 2024-05-08T08:54:26.259 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.259 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.259 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:26.259 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717041+0000", 2024-05-08T08:54:26.259 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6720000000000001e-06 2024-05-08T08:54:26.259 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.259 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.259 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:26.260 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717046+0000", 2024-05-08T08:54:26.260 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5650000000000003e-06 2024-05-08T08:54:26.260 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.260 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.260 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:26.260 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769311+0000", 2024-05-08T08:54:26.260 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052265645999999999 2024-05-08T08:54:26.260 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.260 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.260 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:26.260 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769379+0000", 2024-05-08T08:54:26.260 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7391000000000006e-05 2024-05-08T08:54:26.260 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.260 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.260 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:26.261 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769469+0000", 2024-05-08T08:54:26.261 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0025999999999995e-05 2024-05-08T08:54:26.261 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:26.261 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:26.261 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:26.261 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.261 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.261 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6708 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1241088~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:26.261 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:54:26.261 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 114.520435167, 2024-05-08T08:54:26.261 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052640552, 2024-05-08T08:54:26.261 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:26.261 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:26.261 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:26.261 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:26.261 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:26.262 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:26.262 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6708 2024-05-08T08:54:26.262 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.262 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:26.262 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.262 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:26.262 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:54:26.262 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:26.262 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.262 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.262 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:26.262 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:54:26.262 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:26.263 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.263 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.263 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:26.263 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717052+0000", 2024-05-08T08:54:26.263 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:54:26.263 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.263 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.263 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:26.263 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717053+0000", 2024-05-08T08:54:26.263 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8330000000000001e-06 2024-05-08T08:54:26.263 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.263 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.263 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:26.263 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717058+0000", 2024-05-08T08:54:26.263 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.933e-06 2024-05-08T08:54:26.263 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.264 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.264 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:26.264 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717063+0000", 2024-05-08T08:54:26.264 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2180000000000001e-06 2024-05-08T08:54:26.264 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.264 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.264 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:26.264 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769507+0000", 2024-05-08T08:54:26.264 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052444866999999999 2024-05-08T08:54:26.264 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.264 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.264 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:26.264 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769589+0000", 2024-05-08T08:54:26.264 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2039000000000005e-05 2024-05-08T08:54:26.264 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.265 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.265 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:26.265 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769692+0000", 2024-05-08T08:54:26.265 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102252 2024-05-08T08:54:26.265 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:26.265 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:26.265 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:26.265 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.265 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.265 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6710 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1257472~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:26.265 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:54:26.265 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 114.520417143, 2024-05-08T08:54:26.265 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.05283057, 2024-05-08T08:54:26.265 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:26.265 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:26.266 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:26.266 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:26.266 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:26.266 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:26.266 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6710 2024-05-08T08:54:26.266 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.266 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:26.266 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.266 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:26.266 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:54:26.266 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:26.266 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.266 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.266 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:26.266 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:54:26.267 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:26.267 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.267 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.267 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:26.267 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718951+0000", 2024-05-08T08:54:26.267 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0018817809999999999 2024-05-08T08:54:26.267 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.267 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.267 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:26.267 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718956+0000", 2024-05-08T08:54:26.267 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5399999999999997e-06 2024-05-08T08:54:26.267 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.267 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.267 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:26.268 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718964+0000", 2024-05-08T08:54:26.268 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6629999999999999e-06 2024-05-08T08:54:26.268 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.268 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.268 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:26.268 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718972+0000", 2024-05-08T08:54:26.268 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5229999999999998e-06 2024-05-08T08:54:26.268 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.268 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.268 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:26.268 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769729+0000", 2024-05-08T08:54:26.268 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050756959999999997 2024-05-08T08:54:26.268 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.268 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.268 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:26.269 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769803+0000", 2024-05-08T08:54:26.269 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4538999999999999e-05 2024-05-08T08:54:26.269 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.269 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.269 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:26.269 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769900+0000", 2024-05-08T08:54:26.269 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6564000000000006e-05 2024-05-08T08:54:26.269 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:26.269 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:26.269 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:26.269 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.269 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.269 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6726 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1388544~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:26.269 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:54:26.269 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 114.518342371, 2024-05-08T08:54:26.270 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052451506000000002, 2024-05-08T08:54:26.270 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:26.270 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:26.270 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:26.270 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:26.270 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:26.270 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:26.270 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6726 2024-05-08T08:54:26.270 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.270 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:26.270 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.270 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:26.270 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:54:26.270 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:26.270 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.271 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.271 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:26.271 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:54:26.271 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:26.271 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.271 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.271 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:26.271 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719200+0000", 2024-05-08T08:54:26.271 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5865999999999997e-05 2024-05-08T08:54:26.271 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.271 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.271 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:26.271 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719204+0000", 2024-05-08T08:54:26.271 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8630000000000001e-06 2024-05-08T08:54:26.271 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.272 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.272 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:26.272 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719215+0000", 2024-05-08T08:54:26.272 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1590000000000001e-05 2024-05-08T08:54:26.272 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.272 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.272 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:26.272 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719220+0000", 2024-05-08T08:54:26.272 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1619999999999999e-06 2024-05-08T08:54:26.272 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.272 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.272 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:26.272 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771425+0000", 2024-05-08T08:54:26.272 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052204249000000001 2024-05-08T08:54:26.272 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.273 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.273 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:26.273 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771498+0000", 2024-05-08T08:54:26.273 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3782999999999995e-05 2024-05-08T08:54:26.273 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.273 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.273 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:26.273 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771595+0000", 2024-05-08T08:54:26.273 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6992999999999998e-05 2024-05-08T08:54:26.273 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:26.273 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:26.273 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:26.273 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.273 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.273 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6728 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1404928~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:26.274 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:54:26.274 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 114.518254966, 2024-05-08T08:54:26.274 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052590747, 2024-05-08T08:54:26.274 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:26.274 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:26.274 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:26.274 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:26.274 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:26.274 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:26.274 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6728 2024-05-08T08:54:26.274 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.274 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:26.274 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.274 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:26.275 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:54:26.275 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:26.275 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.275 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.275 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:26.275 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:54:26.275 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:26.275 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.275 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.275 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:26.275 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719232+0000", 2024-05-08T08:54:26.275 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4000000000000001e-07 2024-05-08T08:54:26.275 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.275 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.276 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:26.276 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719234+0000", 2024-05-08T08:54:26.276 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4499999999999998e-06 2024-05-08T08:54:26.276 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.276 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.276 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:26.276 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719240+0000", 2024-05-08T08:54:26.276 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5720000000000002e-06 2024-05-08T08:54:26.276 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.276 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.276 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:26.276 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719245+0000", 2024-05-08T08:54:26.276 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8690000000000003e-06 2024-05-08T08:54:26.276 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.276 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.277 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:26.277 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771636+0000", 2024-05-08T08:54:26.277 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052390979999999997 2024-05-08T08:54:26.277 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.277 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.277 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:26.277 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771717+0000", 2024-05-08T08:54:26.277 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1291999999999993e-05 2024-05-08T08:54:26.277 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.277 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.277 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:26.277 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771822+0000", 2024-05-08T08:54:26.277 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104944 2024-05-08T08:54:26.277 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:26.278 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:26.278 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:26.278 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.278 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.278 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6730 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1421312~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:26.278 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:54:26.278 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 114.518234724, 2024-05-08T08:54:26.278 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052770477000000003, 2024-05-08T08:54:26.278 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:26.278 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:26.278 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:26.278 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:26.278 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:26.278 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:26.279 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6730 2024-05-08T08:54:26.279 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.279 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:26.279 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.279 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:26.279 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:54:26.279 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:26.279 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.279 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.279 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:26.279 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:54:26.279 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:26.279 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.279 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.279 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:26.280 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:54:26.280 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9000000000000002e-07 2024-05-08T08:54:26.280 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.280 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.280 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:26.280 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719254+0000", 2024-05-08T08:54:26.280 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4099999999999998e-06 2024-05-08T08:54:26.280 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.280 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.280 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:26.280 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719263+0000", 2024-05-08T08:54:26.280 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4109999999999996e-06 2024-05-08T08:54:26.280 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.280 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.280 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:26.281 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719268+0000", 2024-05-08T08:54:26.281 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2669999999999997e-06 2024-05-08T08:54:26.281 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.281 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.281 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:26.281 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771846+0000", 2024-05-08T08:54:26.281 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052577677000000003 2024-05-08T08:54:26.281 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.281 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.281 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:26.281 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771920+0000", 2024-05-08T08:54:26.281 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4460000000000002e-05 2024-05-08T08:54:26.281 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.281 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.281 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:26.282 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772022+0000", 2024-05-08T08:54:26.282 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010186199999999999 2024-05-08T08:54:26.282 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:26.282 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:26.282 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:26.282 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.282 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.282 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6732 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1437696~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:26.282 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:54:26.282 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 114.51820991699999, 2024-05-08T08:54:26.282 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052905575000000003, 2024-05-08T08:54:26.282 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:26.282 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:26.282 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:26.282 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:26.283 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:26.283 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:26.283 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6732 2024-05-08T08:54:26.283 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.283 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:26.283 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.283 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:26.283 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:54:26.283 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:26.283 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.283 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.283 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:26.283 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:54:26.283 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:26.283 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.283 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.284 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:26.284 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721392+0000", 2024-05-08T08:54:26.284 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002115287 2024-05-08T08:54:26.284 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.284 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.284 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:26.284 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721396+0000", 2024-05-08T08:54:26.284 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1679999999999997e-06 2024-05-08T08:54:26.284 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.284 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.284 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:26.284 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721404+0000", 2024-05-08T08:54:26.284 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4409999999999993e-06 2024-05-08T08:54:26.284 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.284 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.284 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:26.285 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721410+0000", 2024-05-08T08:54:26.285 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9150000000000001e-06 2024-05-08T08:54:26.285 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.285 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.285 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:26.285 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772061+0000", 2024-05-08T08:54:26.285 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050651068 2024-05-08T08:54:26.285 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.285 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.285 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:26.285 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772127+0000", 2024-05-08T08:54:26.285 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5424000000000002e-05 2024-05-08T08:54:26.285 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.285 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.285 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:26.285 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772182+0000", 2024-05-08T08:54:26.286 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5272000000000002e-05 2024-05-08T08:54:26.286 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:26.286 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:26.286 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:26.286 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.286 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.286 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6749 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1585152~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:26.286 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:54:26.286 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 114.515909147, 2024-05-08T08:54:26.286 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052350264000000001, 2024-05-08T08:54:26.286 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:26.286 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:26.286 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:26.286 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:26.287 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:26.287 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:26.287 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6749 2024-05-08T08:54:26.287 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.287 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:26.287 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.287 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:26.287 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:54:26.287 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:26.287 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.287 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.287 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:26.287 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:54:26.287 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:26.287 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.287 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.288 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:26.288 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721578+0000", 2024-05-08T08:54:26.288 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0399999999999996e-07 2024-05-08T08:54:26.288 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.288 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.288 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:26.288 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721580+0000", 2024-05-08T08:54:26.288 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5450000000000002e-06 2024-05-08T08:54:26.288 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.288 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.288 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:26.288 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721587+0000", 2024-05-08T08:54:26.288 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0190000000000001e-06 2024-05-08T08:54:26.289 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.289 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.289 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:26.289 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721593+0000", 2024-05-08T08:54:26.289 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5659999999999997e-06 2024-05-08T08:54:26.289 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.289 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.289 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:26.289 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773759+0000", 2024-05-08T08:54:26.289 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052166553999999997 2024-05-08T08:54:26.289 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.289 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.289 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:26.289 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773833+0000", 2024-05-08T08:54:26.290 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3913999999999997e-05 2024-05-08T08:54:26.290 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.290 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.290 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:26.290 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773927+0000", 2024-05-08T08:54:26.290 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4061999999999995e-05 2024-05-08T08:54:26.290 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:26.290 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:26.290 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:26.290 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.290 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.290 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6751 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1601536~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:26.290 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:54:26.290 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 114.515885868, 2024-05-08T08:54:26.291 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052461302000000001, 2024-05-08T08:54:26.291 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:26.291 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:26.291 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:26.291 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:26.291 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:26.291 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:26.291 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6751 2024-05-08T08:54:26.291 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.291 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:26.291 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.291 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:26.291 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:54:26.291 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:26.292 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.292 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.292 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:26.292 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:54:26.292 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:26.292 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.292 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.292 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:26.292 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722939+0000", 2024-05-08T08:54:26.292 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001338601 2024-05-08T08:54:26.292 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.292 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.292 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:26.292 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722944+0000", 2024-05-08T08:54:26.292 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4469999999999999e-06 2024-05-08T08:54:26.293 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.293 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.293 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:26.293 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722955+0000", 2024-05-08T08:54:26.293 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1892999999999999e-05 2024-05-08T08:54:26.293 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.293 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.293 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:26.293 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722963+0000", 2024-05-08T08:54:26.293 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1080000000000004e-06 2024-05-08T08:54:26.293 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.293 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.293 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:26.293 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773964+0000", 2024-05-08T08:54:26.294 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051001022 2024-05-08T08:54:26.294 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.294 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.294 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:26.294 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.774006+0000", 2024-05-08T08:54:26.294 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2729999999999999e-05 2024-05-08T08:54:26.294 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.294 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.294 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:26.294 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.774062+0000", 2024-05-08T08:54:26.294 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5501000000000003e-05 2024-05-08T08:54:26.294 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:26.294 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:26.295 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:26.295 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.295 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.295 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6760 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1683456~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:26.295 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:54:26.295 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 114.514376492, 2024-05-08T08:54:26.295 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052571312000000002, 2024-05-08T08:54:26.295 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:26.295 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:26.295 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:26.295 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:26.295 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:26.295 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:26.295 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6760 2024-05-08T08:54:26.295 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.295 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:26.296 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.296 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:26.296 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:54:26.296 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:26.296 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.296 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.296 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:26.296 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:54:26.296 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:26.296 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.296 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.296 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:26.296 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:54:26.296 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2100000000000002e-07 2024-05-08T08:54:26.296 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.297 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.297 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:26.297 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723113+0000", 2024-05-08T08:54:26.297 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3149999999999999e-06 2024-05-08T08:54:26.297 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.297 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.297 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:26.297 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723118+0000", 2024-05-08T08:54:26.297 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8860000000000001e-06 2024-05-08T08:54:26.297 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.297 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.297 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:26.297 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723123+0000", 2024-05-08T08:54:26.297 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6770000000000003e-06 2024-05-08T08:54:26.297 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.298 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.298 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:26.298 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775414+0000", 2024-05-08T08:54:26.298 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052290692999999999 2024-05-08T08:54:26.298 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.298 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.298 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:26.298 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775459+0000", 2024-05-08T08:54:26.298 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5639999999999997e-05 2024-05-08T08:54:26.298 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.298 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.298 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:26.298 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775681+0000", 2024-05-08T08:54:26.298 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00022168000000000001 2024-05-08T08:54:26.299 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:26.299 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:26.299 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:26.299 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.299 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.299 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6761 3.16 3:6bd6d6da:::rbd_data.1140782590a5.0000000000000015:6 [read 1095680~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:26.299 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:54:26.299 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 114.514357272, 2024-05-08T08:54:26.299 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052751275, 2024-05-08T08:54:26.299 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:26.299 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:26.299 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:26.299 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:26.299 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:26.299 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:26.299 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6761 2024-05-08T08:54:26.300 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.300 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:26.300 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.300 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:26.300 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:54:26.300 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:26.300 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.300 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.300 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:26.300 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:54:26.300 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:26.300 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.300 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.300 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:26.300 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724939+0000", 2024-05-08T08:54:26.301 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001809817 2024-05-08T08:54:26.301 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.301 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.301 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:26.301 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724943+0000", 2024-05-08T08:54:26.301 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4920000000000004e-06 2024-05-08T08:54:26.301 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.301 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.301 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:26.301 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724956+0000", 2024-05-08T08:54:26.301 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2427e-05 2024-05-08T08:54:26.301 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.301 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.301 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:26.302 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724963+0000", 2024-05-08T08:54:26.302 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2559999999999996e-06 2024-05-08T08:54:26.302 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.302 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.302 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:26.302 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775711+0000", 2024-05-08T08:54:26.302 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050747621 2024-05-08T08:54:26.302 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.302 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.302 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:26.302 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775785+0000", 2024-05-08T08:54:26.302 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4634000000000004e-05 2024-05-08T08:54:26.302 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.302 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.303 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:26.303 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775880+0000", 2024-05-08T08:54:26.303 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5027999999999999e-05 2024-05-08T08:54:26.303 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:26.303 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:26.303 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:26.303 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.303 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.303 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6770 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1765376~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:26.303 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:54:26.303 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 114.51232617700001, 2024-05-08T08:54:26.303 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052339296, 2024-05-08T08:54:26.303 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:26.303 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:26.304 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:26.304 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:26.304 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:26.304 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:26.304 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6770 2024-05-08T08:54:26.304 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.304 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:26.304 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.304 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:26.304 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:54:26.304 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:26.304 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.304 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.304 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:26.304 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:54:26.305 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:26.305 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.305 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.305 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:26.305 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725161+0000", 2024-05-08T08:54:26.305 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-05-08T08:54:26.305 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.305 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.305 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:26.305 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725163+0000", 2024-05-08T08:54:26.305 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4370000000000001e-06 2024-05-08T08:54:26.305 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.305 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.305 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:26.306 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725169+0000", 2024-05-08T08:54:26.306 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8409999999999996e-06 2024-05-08T08:54:26.306 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.306 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.306 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:26.306 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725174+0000", 2024-05-08T08:54:26.306 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7249999999999997e-06 2024-05-08T08:54:26.306 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.306 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.306 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:26.306 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777343+0000", 2024-05-08T08:54:26.306 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052169633999999999 2024-05-08T08:54:26.306 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.306 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.306 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:26.307 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777409+0000", 2024-05-08T08:54:26.307 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6033999999999998e-05 2024-05-08T08:54:26.307 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.307 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.307 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:26.307 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777499+0000", 2024-05-08T08:54:26.307 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0224999999999998e-05 2024-05-08T08:54:26.307 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:26.307 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:26.307 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:26.307 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.307 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.307 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6771 3.16 3:6bd6d6da:::rbd_data.1140782590a5.0000000000000015:6 [read 1177600~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:26.307 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:54:26.308 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 114.51230636, 2024-05-08T08:54:26.308 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052438604999999999, 2024-05-08T08:54:26.308 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:26.308 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:26.308 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:26.308 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:26.308 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:26.308 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:26.308 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6771 2024-05-08T08:54:26.308 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.308 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:26.308 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.308 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:26.308 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:54:26.308 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:26.309 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.309 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.309 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:26.309 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:54:26.309 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:26.309 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.309 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.309 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:26.309 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:54:26.309 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:54:26.309 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.309 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.309 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:26.309 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725188+0000", 2024-05-08T08:54:26.309 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1579999999999999e-06 2024-05-08T08:54:26.309 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.310 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.310 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:26.310 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725195+0000", 2024-05-08T08:54:26.310 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8909999999999995e-06 2024-05-08T08:54:26.310 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.310 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.310 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:26.310 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725200+0000", 2024-05-08T08:54:26.310 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9810000000000003e-06 2024-05-08T08:54:26.310 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.310 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.310 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:26.310 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777527+0000", 2024-05-08T08:54:26.310 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052326582000000003 2024-05-08T08:54:26.310 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.311 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.311 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:26.311 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777566+0000", 2024-05-08T08:54:26.311 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9447999999999999e-05 2024-05-08T08:54:26.311 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.311 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.311 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:26.311 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777619+0000", 2024-05-08T08:54:26.311 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2135000000000001e-05 2024-05-08T08:54:26.311 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:26.311 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:26.311 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:26.311 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.311 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.311 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6772 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1781760~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:26.312 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:54:26.312 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 114.512278303, 2024-05-08T08:54:26.312 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052601799999999997, 2024-05-08T08:54:26.312 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:26.312 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:26.312 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:26.312 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:26.312 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:26.312 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:26.312 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6772 2024-05-08T08:54:26.312 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.312 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:26.312 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.312 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:26.312 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:54:26.312 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:26.313 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.313 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.313 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:26.313 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:54:26.313 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:26.313 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.313 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.313 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:26.313 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726939+0000", 2024-05-08T08:54:26.313 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001730493 2024-05-08T08:54:26.313 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.313 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.313 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:26.313 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726943+0000", 2024-05-08T08:54:26.313 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7330000000000003e-06 2024-05-08T08:54:26.313 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.314 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.314 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:26.314 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726954+0000", 2024-05-08T08:54:26.314 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1107000000000001e-05 2024-05-08T08:54:26.314 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.314 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.314 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:26.314 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726962+0000", 2024-05-08T08:54:26.314 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.503e-06 2024-05-08T08:54:26.314 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.314 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.314 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:26.314 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777653+0000", 2024-05-08T08:54:26.315 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050691542999999999 2024-05-08T08:54:26.315 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.315 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.315 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:26.315 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777719+0000", 2024-05-08T08:54:26.315 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5228000000000006e-05 2024-05-08T08:54:26.315 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.315 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.315 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:26.315 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777810+0000", 2024-05-08T08:54:26.315 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1193000000000006e-05 2024-05-08T08:54:26.315 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:26.315 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:26.315 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:26.315 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.315 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.316 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6782 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1880064~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:26.316 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:54:26.316 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 114.510309796, 2024-05-08T08:54:26.316 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052326492000000002, 2024-05-08T08:54:26.316 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:26.316 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:26.316 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:26.316 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:26.316 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:26.316 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:26.316 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6782 2024-05-08T08:54:26.316 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.316 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:26.316 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.316 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:26.316 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:54:26.317 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:26.317 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.317 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.317 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:26.317 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:54:26.317 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:26.317 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.317 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.317 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:26.317 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728939+0000", 2024-05-08T08:54:26.317 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001762719 2024-05-08T08:54:26.317 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.317 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.317 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:26.317 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728944+0000", 2024-05-08T08:54:26.317 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.746e-06 2024-05-08T08:54:26.318 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.318 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.318 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:26.318 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728955+0000", 2024-05-08T08:54:26.318 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1043000000000001e-05 2024-05-08T08:54:26.318 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.318 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.318 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:26.318 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728963+0000", 2024-05-08T08:54:26.318 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5809999999999999e-06 2024-05-08T08:54:26.318 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.318 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.318 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:26.318 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779320+0000", 2024-05-08T08:54:26.318 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050357290999999998 2024-05-08T08:54:26.319 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.319 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.319 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:26.319 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779401+0000", 2024-05-08T08:54:26.319 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0574000000000005e-05 2024-05-08T08:54:26.319 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:26.319 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:26.319 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:26.319 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779503+0000", 2024-05-08T08:54:26.319 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102538 2024-05-08T08:54:26.319 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:26.319 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:26.319 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:26.319 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:26.319 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:26.319 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:26.321 INFO:teuthology.orchestra.run.smithi196.stdout:5940 write 0xe75c800 thru 0xe763007 (0x6808 bytes) 2024-05-08T08:54:26.321 INFO:teuthology.orchestra.run.smithi196.stdout:5944 trunc from 0xe763200 to 0xe4e5c00 2024-05-08T08:54:26.321 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:54:26.322 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:26.323+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:26.324 INFO:teuthology.orchestra.run.smithi196.stdout:5945 trunc from 0xe4e5c00 to 0x6e0600 2024-05-08T08:54:26.378 INFO:teuthology.orchestra.run.smithi196.stdout:5947 write 0x567c800 thru 0x567dc24 (0x1425 bytes) 2024-05-08T08:54:26.379 INFO:teuthology.orchestra.run.smithi196.stdout:5948 read 0x3cd5200 thru 0x3ce391a (0xe71b bytes) 2024-05-08T08:54:26.379 INFO:teuthology.orchestra.run.smithi196.stdout:5949 read 0x1690e00 thru 0x169ddbc (0xcfbd bytes) 2024-05-08T08:54:26.379 INFO:teuthology.orchestra.run.smithi196.stdout:5950 write 0x1a8b200 thru 0x1a94f43 (0x9d44 bytes) 2024-05-08T08:54:26.379 INFO:teuthology.orchestra.run.smithi196.stdout:5952 read 0x4aedc00 thru 0x4af5fac (0x83ad bytes) 2024-05-08T08:54:26.415 INFO:teuthology.orchestra.run.smithi196.stdout:5953 write 0xd800400 thru 0xd800c61 (0x862 bytes) 2024-05-08T08:54:26.417 INFO:teuthology.orchestra.run.smithi196.stdout:5954 read 0xbe1f200 thru 0xbe2788d (0x868e bytes) 2024-05-08T08:54:26.417 INFO:teuthology.orchestra.run.smithi196.stdout:5955 read 0x2a1d800 thru 0x2a2aacb (0xd2cc bytes) 2024-05-08T08:54:26.422 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:26.424+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:26.426 INFO:teuthology.orchestra.run.smithi196.stdout:5957 trunc from 0xd800e00 to 0x618fa00 2024-05-08T08:54:26.446 INFO:teuthology.orchestra.run.smithi196.stdout:5958 read 0x2587600 thru 0x2587e7b (0x87c bytes) 2024-05-08T08:54:26.447 INFO:teuthology.orchestra.run.smithi196.stdout:5959 punch from 0x1920200 to 0x1921800, (0x1600 bytes) 2024-05-08T08:54:26.447 INFO:teuthology.orchestra.run.smithi196.stdout:5962 read 0x335be00 thru 0x336253e (0x673f bytes) 2024-05-08T08:54:26.447 INFO:teuthology.orchestra.run.smithi196.stdout:5963 punch from 0x1fc6e00 to 0x1fcb400, (0x4600 bytes) 2024-05-08T08:54:26.447 INFO:teuthology.orchestra.run.smithi196.stdout:5964 punch from 0x1874200 to 0x187e200, (0xa000 bytes) 2024-05-08T08:54:26.453 INFO:teuthology.orchestra.run.smithi196.stdout:5965 trunc from 0x618fa00 to 0xfc1000 2024-05-08T08:54:26.458 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:26.458 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:54:26.458 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:54:26.458 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:26.470 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:54:26.520 INFO:teuthology.orchestra.run.smithi196.stdout:5967 write 0xe540800 thru 0xe5446a5 (0x3ea6 bytes) 2024-05-08T08:54:26.522 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:26.525+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:26.529 INFO:teuthology.orchestra.run.smithi196.stdout:5968 trunc from 0xe544800 to 0x7f20c00 2024-05-08T08:54:26.561 INFO:teuthology.orchestra.run.smithi196.stdout:5970 trunc from 0x7f20c00 to 0x6406000 2024-05-08T08:54:26.573 INFO:teuthology.orchestra.run.smithi196.stdout:5973 read 0x2737600 thru 0x273d935 (0x6336 bytes) 2024-05-08T08:54:26.582 INFO:teuthology.orchestra.run.smithi196.stdout:5974 trunc from 0x6406000 to 0x41bc200 2024-05-08T08:54:26.592 INFO:teuthology.orchestra.run.smithi196.stdout:5975 read 0x71de00 thru 0x71eb86 (0xd87 bytes) 2024-05-08T08:54:26.594 INFO:teuthology.orchestra.run.smithi196.stdout:5977 clone 49 order 25 su 262144 sc 13 2024-05-08T08:54:26.598 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:26.598 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:54:26.598 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:54:26.598 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:54:26.598 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:26.608 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:54:26.623 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:26.627+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:26.670 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4] out_osds: [5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-05-08T08:54:26.670 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T08:54:26.671 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-05-08T08:54:26.671 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-05-08T08:54:26.724 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:26.727+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:26.764 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:26.764 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:54:26.764 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:54:26.764 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:54:26.764 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.764 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4621.0:9089 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 23339520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:54:26.764 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:54:26.764 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 114.27263105199999, 2024-05-08T08:54:26.764 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.13814907200000001, 2024-05-08T08:54:26.764 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:26.764 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:26.765 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:54:26.765 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:26.765 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4621", 2024-05-08T08:54:26.765 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:26.765 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9089 2024-05-08T08:54:26.765 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.765 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:26.765 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.765 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:26.765 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:54:26.765 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:26.765 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.765 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.765 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:26.765 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:54:26.766 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:26.766 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.766 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.766 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:26.766 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493005+0000", 2024-05-08T08:54:26.766 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1700000000000001e-06 2024-05-08T08:54:26.766 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.766 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.766 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:26.766 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.513963+0000", 2024-05-08T08:54:26.766 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.020957581999999999 2024-05-08T08:54:26.766 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.766 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.766 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:26.767 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.513995+0000", 2024-05-08T08:54:26.767 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1993999999999999e-05 2024-05-08T08:54:26.767 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.767 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.767 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:26.767 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514007+0000", 2024-05-08T08:54:26.767 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1581e-05 2024-05-08T08:54:26.767 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.767 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.767 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:26.767 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514122+0000", 2024-05-08T08:54:26.767 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000115282 2024-05-08T08:54:26.767 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.767 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.767 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op must wait for map", 2024-05-08T08:54:26.767 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514128+0000", 2024-05-08T08:54:26.767 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9950000000000002e-06 2024-05-08T08:54:26.768 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.768 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.768 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:26.768 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.515800+0000", 2024-05-08T08:54:26.768 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001671886 2024-05-08T08:54:26.768 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.768 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.768 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:26.768 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.515893+0000", 2024-05-08T08:54:26.768 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3490000000000001e-05 2024-05-08T08:54:26.768 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.768 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.768 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 3", 2024-05-08T08:54:26.768 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.517647+0000", 2024-05-08T08:54:26.768 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0017535630000000001 2024-05-08T08:54:26.768 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.769 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.769 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:54:26.769 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.627578+0000", 2024-05-08T08:54:26.769 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.109931557 2024-05-08T08:54:26.769 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.769 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.769 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:54:26.769 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631055+0000", 2024-05-08T08:54:26.769 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0034761420000000002 2024-05-08T08:54:26.769 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.769 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.769 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:54:26.769 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631096+0000", 2024-05-08T08:54:26.769 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1059000000000001e-05 2024-05-08T08:54:26.769 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.770 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.770 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:26.770 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631151+0000", 2024-05-08T08:54:26.770 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5770999999999998e-05 2024-05-08T08:54:26.770 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:26.770 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:26.770 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:26.770 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.770 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.770 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1071 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11026432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:26.770 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:54:26.770 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 110.28018156100001, 2024-05-08T08:54:26.770 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070580037999999998, 2024-05-08T08:54:26.770 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:26.771 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:26.771 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:26.771 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:26.771 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:26.771 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:26.771 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1071 2024-05-08T08:54:26.771 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.771 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:26.771 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.771 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:26.771 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:54:26.771 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:26.771 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.771 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.771 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:26.771 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:54:26.772 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:26.772 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.772 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.772 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:26.772 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:54:26.772 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2800000000000002e-07 2024-05-08T08:54:26.772 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.772 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.772 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:26.772 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485454+0000", 2024-05-08T08:54:26.772 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2290000000000002e-06 2024-05-08T08:54:26.772 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.772 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.772 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:26.772 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485461+0000", 2024-05-08T08:54:26.772 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9920000000000001e-06 2024-05-08T08:54:26.773 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.773 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.773 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:26.773 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485467+0000", 2024-05-08T08:54:26.773 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3979999999999998e-06 2024-05-08T08:54:26.773 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.773 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.773 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:26.773 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555880+0000", 2024-05-08T08:54:26.773 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070413165 2024-05-08T08:54:26.773 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.773 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.773 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:26.773 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555926+0000", 2024-05-08T08:54:26.773 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6217000000000002e-05 2024-05-08T08:54:26.774 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.774 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.774 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:26.774 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556032+0000", 2024-05-08T08:54:26.774 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000105609 2024-05-08T08:54:26.774 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:26.774 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:26.774 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:26.774 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.774 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.774 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1073 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11067392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:26.774 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:54:26.774 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 110.28016033999999, 2024-05-08T08:54:26.774 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070821404000000004, 2024-05-08T08:54:26.774 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:26.775 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:26.775 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:26.775 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:26.775 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:26.775 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:26.775 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1073 2024-05-08T08:54:26.775 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.775 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:26.775 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.775 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:26.775 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:54:26.775 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:26.775 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.775 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.775 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:26.776 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:54:26.776 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:26.776 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.776 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.776 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:26.776 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:54:26.776 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.08e-07 2024-05-08T08:54:26.776 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.776 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.776 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:26.776 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485476+0000", 2024-05-08T08:54:26.776 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3929999999999998e-06 2024-05-08T08:54:26.776 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.776 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.776 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:26.777 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485485+0000", 2024-05-08T08:54:26.777 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1880000000000008e-06 2024-05-08T08:54:26.777 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.777 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.777 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:26.777 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485494+0000", 2024-05-08T08:54:26.777 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3109999999999995e-06 2024-05-08T08:54:26.777 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.777 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.777 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:26.777 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556082+0000", 2024-05-08T08:54:26.777 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070587426999999994 2024-05-08T08:54:26.777 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.777 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.777 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:26.778 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556177+0000", 2024-05-08T08:54:26.778 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5051999999999997e-05 2024-05-08T08:54:26.778 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.778 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.778 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:26.778 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556294+0000", 2024-05-08T08:54:26.778 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000117625 2024-05-08T08:54:26.778 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:26.778 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:26.778 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:26.778 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.778 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.778 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1075 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11108352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:26.778 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:54:26.778 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 110.28012976300001, 2024-05-08T08:54:26.779 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071045026999999997, 2024-05-08T08:54:26.779 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:26.779 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:26.779 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:26.779 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:26.779 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:26.779 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:26.779 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1075 2024-05-08T08:54:26.779 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.779 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:26.779 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.779 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:26.779 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:54:26.779 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:26.779 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.780 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.780 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:26.780 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:54:26.780 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:26.780 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.780 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.780 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:26.780 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:54:26.780 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.32e-07 2024-05-08T08:54:26.780 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.780 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.780 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:26.780 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485518+0000", 2024-05-08T08:54:26.780 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3769e-05 2024-05-08T08:54:26.781 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.781 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.781 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:26.781 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485527+0000", 2024-05-08T08:54:26.781 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6779999999999995e-06 2024-05-08T08:54:26.781 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.781 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.781 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:26.781 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485532+0000", 2024-05-08T08:54:26.781 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9830000000000004e-06 2024-05-08T08:54:26.781 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.781 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.781 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:26.781 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556340+0000", 2024-05-08T08:54:26.781 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070807743000000006 2024-05-08T08:54:26.782 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.782 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.782 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:26.782 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556435+0000", 2024-05-08T08:54:26.782 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4732e-05 2024-05-08T08:54:26.782 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.782 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.782 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:26.782 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556549+0000", 2024-05-08T08:54:26.782 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011369 2024-05-08T08:54:26.782 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:26.782 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:26.782 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:26.782 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.783 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.783 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1078 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11149312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:26.783 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:54:26.783 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 110.28008956399999, 2024-05-08T08:54:26.783 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071167276000000002, 2024-05-08T08:54:26.783 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:26.783 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:26.783 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:26.783 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:26.783 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:26.783 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:26.783 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1078 2024-05-08T08:54:26.783 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.783 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:26.783 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.783 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:26.784 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:54:26.784 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:26.784 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.784 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.784 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:26.784 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:54:26.784 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:26.784 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.784 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.784 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:26.784 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:54:26.784 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5600000000000001e-07 2024-05-08T08:54:26.784 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.784 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.784 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:26.784 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485547+0000", 2024-05-08T08:54:26.785 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3590000000000002e-06 2024-05-08T08:54:26.785 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.785 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.785 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:26.785 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485553+0000", 2024-05-08T08:54:26.785 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2770000000000002e-06 2024-05-08T08:54:26.785 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.785 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.785 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:26.785 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485558+0000", 2024-05-08T08:54:26.785 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.95e-06 2024-05-08T08:54:26.785 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.785 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.785 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:26.786 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556582+0000", 2024-05-08T08:54:26.786 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071024444000000006 2024-05-08T08:54:26.786 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.786 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.786 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:26.786 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556637+0000", 2024-05-08T08:54:26.786 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4672000000000001e-05 2024-05-08T08:54:26.786 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.786 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.786 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:26.786 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556711+0000", 2024-05-08T08:54:26.786 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4117999999999997e-05 2024-05-08T08:54:26.786 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:26.786 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:26.786 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:26.787 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.787 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.787 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1080 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11190272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:26.787 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:54:26.787 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 110.28006883099999, 2024-05-08T08:54:26.787 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071350721000000006, 2024-05-08T08:54:26.787 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:26.787 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:26.787 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:26.787 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:26.787 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:26.787 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:26.787 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1080 2024-05-08T08:54:26.787 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.788 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:26.788 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.788 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:26.788 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:54:26.788 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:26.788 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.788 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.788 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:26.788 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:54:26.788 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:26.788 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.788 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.788 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:26.788 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:54:26.788 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.03e-07 2024-05-08T08:54:26.788 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.788 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.789 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:26.789 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485567+0000", 2024-05-08T08:54:26.789 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1090000000000001e-06 2024-05-08T08:54:26.789 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.789 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.789 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:26.789 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485573+0000", 2024-05-08T08:54:26.789 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3049999999999998e-06 2024-05-08T08:54:26.789 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.789 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.789 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:26.789 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485578+0000", 2024-05-08T08:54:26.789 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0130000000000001e-06 2024-05-08T08:54:26.789 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.790 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.790 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:26.790 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556743+0000", 2024-05-08T08:54:26.790 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071164449000000005 2024-05-08T08:54:26.790 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.790 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.790 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:26.790 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556797+0000", 2024-05-08T08:54:26.790 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4589999999999997e-05 2024-05-08T08:54:26.790 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.790 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.790 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:26.790 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556915+0000", 2024-05-08T08:54:26.790 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000117852 2024-05-08T08:54:26.791 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:26.791 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:26.791 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:26.791 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.791 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.791 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1082 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11231232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:26.791 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:54:26.791 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 110.280048588, 2024-05-08T08:54:26.791 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071591396000000002, 2024-05-08T08:54:26.791 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:26.791 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:26.791 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:26.791 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:26.791 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:26.792 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:26.792 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1082 2024-05-08T08:54:26.792 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.792 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:26.792 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.792 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:26.792 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:54:26.792 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:26.792 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.792 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.792 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:26.792 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:54:26.792 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:26.792 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.792 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.793 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:26.793 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:54:26.793 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1800000000000001e-07 2024-05-08T08:54:26.793 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.793 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.793 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:26.793 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485587+0000", 2024-05-08T08:54:26.793 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2589999999999999e-06 2024-05-08T08:54:26.793 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.793 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.793 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:26.793 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485594+0000", 2024-05-08T08:54:26.793 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0789999999999997e-06 2024-05-08T08:54:26.793 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.793 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.794 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:26.794 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485598+0000", 2024-05-08T08:54:26.794 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7480000000000001e-06 2024-05-08T08:54:26.794 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.794 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.794 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:26.794 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556971+0000", 2024-05-08T08:54:26.794 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071372774999999999 2024-05-08T08:54:26.794 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.794 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.794 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:26.794 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557069+0000", 2024-05-08T08:54:26.794 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.7732999999999994e-05 2024-05-08T08:54:26.794 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.795 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.795 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:26.795 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557176+0000", 2024-05-08T08:54:26.795 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000107384 2024-05-08T08:54:26.795 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:26.795 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:26.795 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:26.795 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.795 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.795 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1084 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11272192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:26.795 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:54:26.795 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 110.280009949, 2024-05-08T08:54:26.795 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071808977999999996, 2024-05-08T08:54:26.795 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:26.795 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:26.796 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:26.796 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:26.796 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:26.796 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:26.796 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1084 2024-05-08T08:54:26.796 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.796 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:26.796 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.796 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:26.796 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:54:26.796 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:26.796 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.796 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.796 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:26.796 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:54:26.796 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:26.797 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.797 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.797 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:26.797 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485624+0000", 2024-05-08T08:54:26.797 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:54:26.797 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.797 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.797 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:26.797 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485626+0000", 2024-05-08T08:54:26.797 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9460000000000001e-06 2024-05-08T08:54:26.797 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.797 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.797 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:26.797 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485633+0000", 2024-05-08T08:54:26.797 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1149999999999996e-06 2024-05-08T08:54:26.798 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.798 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.798 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:26.798 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485638+0000", 2024-05-08T08:54:26.798 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5670000000000002e-06 2024-05-08T08:54:26.798 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.798 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.798 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:26.798 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557226+0000", 2024-05-08T08:54:26.798 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071587267999999996 2024-05-08T08:54:26.798 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.798 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.798 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:26.798 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557321+0000", 2024-05-08T08:54:26.798 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5785000000000005e-05 2024-05-08T08:54:26.799 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.799 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.799 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:26.799 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557432+0000", 2024-05-08T08:54:26.799 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011092800000000001 2024-05-08T08:54:26.799 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:26.799 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:26.799 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:26.799 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.799 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.799 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1086 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11313152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:26.799 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:54:26.799 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 110.279986622, 2024-05-08T08:54:26.799 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071933132999999996, 2024-05-08T08:54:26.799 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:26.800 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:26.800 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:26.800 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:26.800 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:26.800 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:26.800 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1086 2024-05-08T08:54:26.800 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.800 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:26.800 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.800 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:26.800 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:54:26.800 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:26.800 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.800 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.800 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:26.801 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:54:26.801 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:26.801 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.801 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.801 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:26.801 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486328+0000", 2024-05-08T08:54:26.801 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00068107900000000002 2024-05-08T08:54:26.801 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.801 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.801 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:26.801 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486331+0000", 2024-05-08T08:54:26.801 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.585e-06 2024-05-08T08:54:26.801 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.801 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.801 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:26.802 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486343+0000", 2024-05-08T08:54:26.802 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1856e-05 2024-05-08T08:54:26.802 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.802 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.802 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:26.802 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486350+0000", 2024-05-08T08:54:26.802 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8600000000000004e-06 2024-05-08T08:54:26.802 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.802 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.802 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:26.802 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557470+0000", 2024-05-08T08:54:26.802 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071120326999999997 2024-05-08T08:54:26.802 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.802 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.803 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:26.803 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557514+0000", 2024-05-08T08:54:26.803 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3909000000000003e-05 2024-05-08T08:54:26.803 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.803 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.803 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:26.803 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557580+0000", 2024-05-08T08:54:26.803 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5517000000000003e-05 2024-05-08T08:54:26.803 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:26.803 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:26.803 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:26.803 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.803 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.803 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1089 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11354112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:26.803 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:54:26.804 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 110.27927375, 2024-05-08T08:54:26.804 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071457381, 2024-05-08T08:54:26.804 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:26.804 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:26.804 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:26.804 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:26.804 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:26.804 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:26.804 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1089 2024-05-08T08:54:26.804 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.804 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:26.804 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.804 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:26.804 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:54:26.804 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:26.805 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.805 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.805 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:26.805 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:54:26.805 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:26.805 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.805 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.805 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:26.805 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486851+0000", 2024-05-08T08:54:26.805 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00049102 2024-05-08T08:54:26.805 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.805 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.805 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:26.805 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486854+0000", 2024-05-08T08:54:26.805 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6859999999999999e-06 2024-05-08T08:54:26.805 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.806 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.806 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:26.806 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486864+0000", 2024-05-08T08:54:26.806 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6080000000000002e-06 2024-05-08T08:54:26.806 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.806 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.806 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:26.806 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486870+0000", 2024-05-08T08:54:26.806 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.477e-06 2024-05-08T08:54:26.806 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.806 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.806 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:26.806 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557612+0000", 2024-05-08T08:54:26.806 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070741557999999996 2024-05-08T08:54:26.807 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.807 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.807 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:26.807 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557695+0000", 2024-05-08T08:54:26.807 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3300000000000005e-05 2024-05-08T08:54:26.807 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.807 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.807 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:26.807 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557817+0000", 2024-05-08T08:54:26.807 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012173200000000001 2024-05-08T08:54:26.807 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:26.807 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:26.807 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:26.807 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.807 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.808 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1091 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11395072~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:26.808 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:54:26.808 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 110.278752955, 2024-05-08T08:54:26.808 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071184737999999997, 2024-05-08T08:54:26.808 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:26.808 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:26.808 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:26.808 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:26.808 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:26.808 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:26.808 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1091 2024-05-08T08:54:26.808 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.808 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:26.808 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.808 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:26.809 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:54:26.809 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:26.809 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.809 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.809 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:26.809 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:54:26.809 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:26.809 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.809 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.809 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:26.809 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486881+0000", 2024-05-08T08:54:26.809 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0699999999999997e-07 2024-05-08T08:54:26.809 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.809 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.810 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:26.810 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486883+0000", 2024-05-08T08:54:26.810 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.402e-06 2024-05-08T08:54:26.810 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.810 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.810 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:26.810 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486890+0000", 2024-05-08T08:54:26.810 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0480000000000001e-06 2024-05-08T08:54:26.810 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.810 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.810 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:26.810 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486896+0000", 2024-05-08T08:54:26.810 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7640000000000002e-06 2024-05-08T08:54:26.810 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.810 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.811 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:26.811 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557860+0000", 2024-05-08T08:54:26.811 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070963791999999998 2024-05-08T08:54:26.811 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.811 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.811 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:26.811 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557952+0000", 2024-05-08T08:54:26.811 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2082000000000004e-05 2024-05-08T08:54:26.811 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.811 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.811 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:26.811 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558065+0000", 2024-05-08T08:54:26.811 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000113143 2024-05-08T08:54:26.811 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:26.811 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:26.812 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:26.812 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.812 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.812 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1094 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11436032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:26.812 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:54:26.812 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 110.278729787, 2024-05-08T08:54:26.812 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071400507000000002, 2024-05-08T08:54:26.812 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:26.812 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:26.812 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:26.812 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:26.812 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:26.812 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:26.812 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1094 2024-05-08T08:54:26.812 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.813 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:26.813 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.813 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:26.813 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:54:26.813 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:26.813 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.813 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.813 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:26.813 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:54:26.813 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:26.813 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.813 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.813 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:26.813 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487940+0000", 2024-05-08T08:54:26.813 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001036868 2024-05-08T08:54:26.814 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.814 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.814 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:26.814 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487944+0000", 2024-05-08T08:54:26.814 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7069999999999998e-06 2024-05-08T08:54:26.814 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.814 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.814 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:26.814 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487951+0000", 2024-05-08T08:54:26.814 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2649999999999999e-06 2024-05-08T08:54:26.814 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.814 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.814 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:26.814 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487961+0000", 2024-05-08T08:54:26.814 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4110000000000002e-06 2024-05-08T08:54:26.815 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.815 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.815 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:26.815 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558117+0000", 2024-05-08T08:54:26.815 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070156662999999994 2024-05-08T08:54:26.815 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.815 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.815 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:26.815 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558214+0000", 2024-05-08T08:54:26.815 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6444e-05 2024-05-08T08:54:26.815 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.815 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.815 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:26.815 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558304+0000", 2024-05-08T08:54:26.815 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0148999999999994e-05 2024-05-08T08:54:26.816 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:26.816 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:26.816 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:26.816 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.816 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.816 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1096 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11476992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:26.816 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:54:26.816 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 110.277663256, 2024-05-08T08:54:26.816 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070480074000000004, 2024-05-08T08:54:26.816 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:26.816 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:26.816 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:26.816 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:26.816 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:26.817 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:26.817 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1096 2024-05-08T08:54:26.817 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.817 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:26.817 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.817 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:26.817 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:54:26.817 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:26.817 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.817 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.817 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:26.817 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:54:26.817 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:26.817 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.817 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.818 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:26.818 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:54:26.818 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3700000000000001e-07 2024-05-08T08:54:26.818 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.818 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.818 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:26.818 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487973+0000", 2024-05-08T08:54:26.818 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0870000000000001e-06 2024-05-08T08:54:26.818 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.818 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.818 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:26.818 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487977+0000", 2024-05-08T08:54:26.818 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0600000000000001e-06 2024-05-08T08:54:26.818 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.819 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.819 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:26.819 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487980+0000", 2024-05-08T08:54:26.819 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1089999999999998e-06 2024-05-08T08:54:26.819 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.819 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.819 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:26.819 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558338+0000", 2024-05-08T08:54:26.819 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070358233000000006 2024-05-08T08:54:26.819 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.819 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.819 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:26.819 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558392+0000", 2024-05-08T08:54:26.819 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4537999999999999e-05 2024-05-08T08:54:26.819 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.819 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.820 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:26.820 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558450+0000", 2024-05-08T08:54:26.820 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7710000000000001e-05 2024-05-08T08:54:26.820 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:26.820 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:26.820 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:26.820 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.820 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.820 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1099 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11517952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:26.820 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:54:26.820 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 110.27764938, 2024-05-08T08:54:26.820 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070720506000000002, 2024-05-08T08:54:26.820 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:26.820 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:26.821 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:26.821 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:26.821 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:26.821 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:26.821 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1099 2024-05-08T08:54:26.821 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.821 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:26.821 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.821 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:26.821 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:54:26.821 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:26.821 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.821 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.821 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:26.822 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:54:26.822 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:26.822 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.822 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.822 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:26.822 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:54:26.822 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3999999999999998e-07 2024-05-08T08:54:26.822 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.822 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.822 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:26.822 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487986+0000", 2024-05-08T08:54:26.822 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3519999999999999e-06 2024-05-08T08:54:26.822 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.822 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.822 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:26.823 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487989+0000", 2024-05-08T08:54:26.823 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.23e-06 2024-05-08T08:54:26.823 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.823 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.823 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:26.823 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487991+0000", 2024-05-08T08:54:26.823 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6350000000000002e-06 2024-05-08T08:54:26.823 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.823 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.823 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:26.823 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558489+0000", 2024-05-08T08:54:26.823 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070497924000000003 2024-05-08T08:54:26.823 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.823 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.823 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:26.824 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558583+0000", 2024-05-08T08:54:26.824 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3405000000000004e-05 2024-05-08T08:54:26.824 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.824 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.824 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:26.824 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558704+0000", 2024-05-08T08:54:26.824 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012171999999999999 2024-05-08T08:54:26.824 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:26.824 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:26.824 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:26.824 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.824 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.824 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1100 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11558912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:26.824 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:54:26.824 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 110.27763638099999, 2024-05-08T08:54:26.825 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070962266999999996, 2024-05-08T08:54:26.825 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:26.825 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:26.825 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:26.825 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:26.825 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:26.825 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:26.825 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1100 2024-05-08T08:54:26.825 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.825 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:26.825 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.825 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:26.825 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:54:26.825 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:26.826 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.826 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.826 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:26.826 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:54:26.826 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:26.826 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.826 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.826 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:26.826 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488430+0000", 2024-05-08T08:54:26.826 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000432676 2024-05-08T08:54:26.826 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.826 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.826 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:26.826 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488433+0000", 2024-05-08T08:54:26.826 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.45e-06 2024-05-08T08:54:26.826 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.827 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.827 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:26.827 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488442+0000", 2024-05-08T08:54:26.827 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.687e-06 2024-05-08T08:54:26.827 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.827 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.827 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:26.827 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488446+0000", 2024-05-08T08:54:26.827 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4939999999999997e-06 2024-05-08T08:54:26.829 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.829 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.829 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:26.829 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558756+0000", 2024-05-08T08:54:26.829 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070310051999999998 2024-05-08T08:54:26.829 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.829 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.829 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:26.829 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558844+0000", 2024-05-08T08:54:26.829 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7690999999999998e-05 2024-05-08T08:54:26.829 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.829 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.830 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:26.830 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558959+0000", 2024-05-08T08:54:26.830 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000115217 2024-05-08T08:54:26.830 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:26.830 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:26.830 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:26.830 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.830 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.830 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1103 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11599872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:26.830 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:54:26.830 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 110.277178706, 2024-05-08T08:54:26.830 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070777100999999995, 2024-05-08T08:54:26.830 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:26.831 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:26.831 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:26.831 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:26.831 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:26.831 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:26.831 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1103 2024-05-08T08:54:26.831 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.831 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:26.831 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.831 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:26.831 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:54:26.831 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:26.831 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.831 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.831 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:26.832 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:54:26.832 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:26.832 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.832 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.832 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:26.832 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488959+0000", 2024-05-08T08:54:26.832 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00050457200000000001 2024-05-08T08:54:26.832 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.832 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.832 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:26.832 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488969+0000", 2024-05-08T08:54:26.832 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.7159999999999998e-06 2024-05-08T08:54:26.832 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.832 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.832 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:26.833 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488981+0000", 2024-05-08T08:54:26.833 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2150000000000001e-05 2024-05-08T08:54:26.833 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.833 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.833 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:26.833 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488988+0000", 2024-05-08T08:54:26.833 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8759999999999999e-06 2024-05-08T08:54:26.833 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.833 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.833 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:26.833 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558992+0000", 2024-05-08T08:54:26.833 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070004358000000003 2024-05-08T08:54:26.833 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.833 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.834 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:26.834 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559093+0000", 2024-05-08T08:54:26.834 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000100299 2024-05-08T08:54:26.834 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.834 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.834 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:26.834 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559232+0000", 2024-05-08T08:54:26.834 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013913 2024-05-08T08:54:26.834 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:26.834 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:26.834 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:26.834 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.834 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.834 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1104 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11640832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:26.834 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:54:26.835 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 110.276633891, 2024-05-08T08:54:26.835 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070521947000000001, 2024-05-08T08:54:26.835 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:26.835 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:26.835 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:26.835 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:26.835 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:26.835 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:26.835 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1104 2024-05-08T08:54:26.835 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.835 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:26.835 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.835 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:26.835 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:54:26.835 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:26.836 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.836 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.836 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:26.836 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:54:26.836 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:26.836 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.836 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.836 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:26.836 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489700+0000", 2024-05-08T08:54:26.836 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00070067600000000001 2024-05-08T08:54:26.836 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.836 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.836 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:26.836 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489704+0000", 2024-05-08T08:54:26.836 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1640000000000003e-06 2024-05-08T08:54:26.837 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.837 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.837 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:26.837 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489716+0000", 2024-05-08T08:54:26.837 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1379000000000001e-05 2024-05-08T08:54:26.837 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.837 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.837 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:26.837 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489724+0000", 2024-05-08T08:54:26.837 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8949999999999998e-06 2024-05-08T08:54:26.837 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.837 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.837 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:26.837 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559278+0000", 2024-05-08T08:54:26.838 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.069554151999999994 2024-05-08T08:54:26.838 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.838 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.838 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:26.838 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559387+0000", 2024-05-08T08:54:26.838 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000109258 2024-05-08T08:54:26.838 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.838 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.838 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:26.838 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559521+0000", 2024-05-08T08:54:26.838 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000134423 2024-05-08T08:54:26.838 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:26.838 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:26.838 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:26.838 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.839 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.839 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:8636 3.17 3:e8a0a3e4:::rbd_header.11402b26385:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-05-08T08:54:26.839 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:54.150971+0000", 2024-05-08T08:54:26.839 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 92.614661897000005, 2024-05-08T08:54:26.839 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.217273233, 2024-05-08T08:54:26.839 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:26.839 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:26.839 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:26.839 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:26.839 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:54:26.839 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:54:26.839 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 8636 2024-05-08T08:54:26.839 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.839 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:26.839 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.840 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:26.840 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150971+0000", 2024-05-08T08:54:26.840 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:26.840 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.840 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.840 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:26.840 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150971+0000", 2024-05-08T08:54:26.840 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:26.840 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.840 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.840 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:26.840 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150972+0000", 2024-05-08T08:54:26.840 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.71e-07 2024-05-08T08:54:26.840 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.840 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.840 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:26.841 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150977+0000", 2024-05-08T08:54:26.841 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.152e-06 2024-05-08T08:54:26.841 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.841 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.841 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:26.841 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150990+0000", 2024-05-08T08:54:26.841 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2364999999999999e-05 2024-05-08T08:54:26.841 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.841 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.841 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:26.841 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150998+0000", 2024-05-08T08:54:26.841 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0739999999999993e-06 2024-05-08T08:54:26.841 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.841 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.842 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:26.842 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.151054+0000", 2024-05-08T08:54:26.842 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5949000000000003e-05 2024-05-08T08:54:26.842 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.842 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.842 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:54:26.842 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.151060+0000", 2024-05-08T08:54:26.842 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6120000000000004e-06 2024-05-08T08:54:26.842 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.842 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.842 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:26.842 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.367640+0000", 2024-05-08T08:54:26.842 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2165792739999999 2024-05-08T08:54:26.842 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.842 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.842 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:26.843 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.367839+0000", 2024-05-08T08:54:26.843 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000199185 2024-05-08T08:54:26.843 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.843 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.843 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:26.843 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.368245+0000", 2024-05-08T08:54:26.843 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00040575099999999998 2024-05-08T08:54:26.843 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:26.843 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:26.843 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:26.843 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.843 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.843 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5302.0:1516 3.c 3:319bf789:::rbd_data.1140d3fc54c.0000000000000025:head [stat,write 1892352~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:54:26.843 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:27.822220+0000", 2024-05-08T08:54:26.843 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 58.943412960000003, 2024-05-08T08:54:26.844 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.092643744, 2024-05-08T08:54:26.844 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:26.844 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:26.844 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:54:26.844 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:26.844 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5302", 2024-05-08T08:54:26.844 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/359277859", 2024-05-08T08:54:26.844 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1516 2024-05-08T08:54:26.844 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.844 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:26.844 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.844 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:26.844 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822220+0000", 2024-05-08T08:54:26.844 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:26.844 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.845 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.845 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:26.845 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822220+0000", 2024-05-08T08:54:26.845 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:26.845 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.845 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.845 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:26.845 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822221+0000", 2024-05-08T08:54:26.845 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3500000000000002e-07 2024-05-08T08:54:26.845 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.845 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.845 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:26.845 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822228+0000", 2024-05-08T08:54:26.845 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9959999999999996e-06 2024-05-08T08:54:26.846 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.846 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.846 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:26.846 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822234+0000", 2024-05-08T08:54:26.846 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5429999999999999e-06 2024-05-08T08:54:26.846 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.846 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.846 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:26.846 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822238+0000", 2024-05-08T08:54:26.846 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.732e-06 2024-05-08T08:54:26.846 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.846 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.846 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:26.846 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822285+0000", 2024-05-08T08:54:26.846 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6810000000000001e-05 2024-05-08T08:54:26.847 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.847 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.847 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:26.847 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822358+0000", 2024-05-08T08:54:26.847 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3473000000000006e-05 2024-05-08T08:54:26.847 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.847 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.847 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 5", 2024-05-08T08:54:26.847 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822489+0000", 2024-05-08T08:54:26.847 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013011199999999999 2024-05-08T08:54:26.847 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.847 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.847 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:54:26.847 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.825332+0000", 2024-05-08T08:54:26.847 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0028434799999999998 2024-05-08T08:54:26.848 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.848 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.848 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:54:26.848 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.914753+0000", 2024-05-08T08:54:26.848 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089421424999999999 2024-05-08T08:54:26.848 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.848 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.848 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:54:26.848 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.914807+0000", 2024-05-08T08:54:26.848 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3829000000000002e-05 2024-05-08T08:54:26.848 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.848 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.848 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:26.848 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.914864+0000", 2024-05-08T08:54:26.848 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6909e-05 2024-05-08T08:54:26.849 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:26.849 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:26.849 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:26.849 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.849 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.849 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:24567 3.6 3:66af1c40:::rbd_data.1140d3fc54c.000000000000005a:18 [read 2080768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:26.849 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.216598+0000", 2024-05-08T08:54:26.849 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 54.549034880000001, 2024-05-08T08:54:26.849 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071602647000000005, 2024-05-08T08:54:26.849 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:26.849 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:26.849 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:26.849 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:26.849 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:26.849 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:26.850 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 24567 2024-05-08T08:54:26.850 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.850 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:26.850 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.850 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:26.850 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216598+0000", 2024-05-08T08:54:26.850 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:26.850 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.850 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.850 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:26.850 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216598+0000", 2024-05-08T08:54:26.850 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:26.850 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.850 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.850 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:26.851 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216599+0000", 2024-05-08T08:54:26.851 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8999999999999997e-07 2024-05-08T08:54:26.851 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.851 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.851 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:26.851 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216601+0000", 2024-05-08T08:54:26.851 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9889999999999999e-06 2024-05-08T08:54:26.851 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.851 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.851 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:26.851 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216607+0000", 2024-05-08T08:54:26.851 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3600000000000001e-06 2024-05-08T08:54:26.851 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.851 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.852 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:26.852 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216612+0000", 2024-05-08T08:54:26.852 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.69e-06 2024-05-08T08:54:26.852 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.852 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.852 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:26.852 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.287076+0000", 2024-05-08T08:54:26.852 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070464355000000006 2024-05-08T08:54:26.852 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.852 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.852 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:26.852 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.287151+0000", 2024-05-08T08:54:26.852 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4924000000000003e-05 2024-05-08T08:54:26.852 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:26.852 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:26.853 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:26.853 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.288201+0000", 2024-05-08T08:54:26.853 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0010498389999999999 2024-05-08T08:54:26.853 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:26.853 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:26.853 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:26.853 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:26.853 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:26.853 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:26.853 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-05-08T08:54:26.855 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:54:26.856 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:26.858+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:26.956 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:26.959+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:27.012 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:27.012 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:54:27.012 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:54:27.012 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:27.018 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:54:27.018 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:27.018 INFO:teuthology.orchestra.run.smithi059.stdout: "epoch": 115, 2024-05-08T08:54:27.018 INFO:teuthology.orchestra.run.smithi059.stdout: "fsid": "38dfae8c-5c5e-4b0f-a264-91f22c6c5554", 2024-05-08T08:54:27.018 INFO:teuthology.orchestra.run.smithi059.stdout: "created": "2024-05-08T08:51:45.571369+0000", 2024-05-08T08:54:27.018 INFO:teuthology.orchestra.run.smithi059.stdout: "modified": "2024-05-08T08:54:26.754375+0000", 2024-05-08T08:54:27.018 INFO:teuthology.orchestra.run.smithi059.stdout: "last_up_change": "2024-05-08T08:51:56.847706+0000", 2024-05-08T08:54:27.018 INFO:teuthology.orchestra.run.smithi059.stdout: "last_in_change": "2024-05-08T08:53:54.592551+0000", 2024-05-08T08:54:27.018 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-05-08T08:54:27.018 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_num": 5801984, 2024-05-08T08:54:27.018 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_set": [ 2024-05-08T08:54:27.018 INFO:teuthology.orchestra.run.smithi059.stdout: "noscrub", 2024-05-08T08:54:27.018 INFO:teuthology.orchestra.run.smithi059.stdout: "pglog_hardlimit", 2024-05-08T08:54:27.019 INFO:teuthology.orchestra.run.smithi059.stdout: "purged_snapdirs", 2024-05-08T08:54:27.019 INFO:teuthology.orchestra.run.smithi059.stdout: "recovery_deletes", 2024-05-08T08:54:27.019 INFO:teuthology.orchestra.run.smithi059.stdout: "sortbitwise" 2024-05-08T08:54:27.019 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T08:54:27.019 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_version": 4, 2024-05-08T08:54:27.019 INFO:teuthology.orchestra.run.smithi059.stdout: "full_ratio": 0.94999998807907104, 2024-05-08T08:54:27.019 INFO:teuthology.orchestra.run.smithi059.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-05-08T08:54:27.019 INFO:teuthology.orchestra.run.smithi059.stdout: "nearfull_ratio": 0.85000002384185791, 2024-05-08T08:54:27.019 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_snapshot": "", 2024-05-08T08:54:27.019 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_max": 3, 2024-05-08T08:54:27.019 INFO:teuthology.orchestra.run.smithi059.stdout: "max_osd": 6, 2024-05-08T08:54:27.019 INFO:teuthology.orchestra.run.smithi059.stdout: "require_min_compat_client": "luminous", 2024-05-08T08:54:27.019 INFO:teuthology.orchestra.run.smithi059.stdout: "min_compat_client": "luminous", 2024-05-08T08:54:27.019 INFO:teuthology.orchestra.run.smithi059.stdout: "require_osd_release": "squid", 2024-05-08T08:54:27.020 INFO:teuthology.orchestra.run.smithi059.stdout: "allow_crimson": false, 2024-05-08T08:54:27.020 INFO:teuthology.orchestra.run.smithi059.stdout: "pools": [ 2024-05-08T08:54:27.020 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.020 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 1, 2024-05-08T08:54:27.020 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_name": ".mgr", 2024-05-08T08:54:27.020 INFO:teuthology.orchestra.run.smithi059.stdout: "create_time": "2024-05-08T08:51:57.787790+0000", 2024-05-08T08:54:27.020 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T08:54:27.020 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_names": "hashpspool", 2024-05-08T08:54:27.020 INFO:teuthology.orchestra.run.smithi059.stdout: "type": 1, 2024-05-08T08:54:27.020 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 2, 2024-05-08T08:54:27.020 INFO:teuthology.orchestra.run.smithi059.stdout: "min_size": 1, 2024-05-08T08:54:27.020 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_rule": 0, 2024-05-08T08:54:27.020 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_count": 0, 2024-05-08T08:54:27.020 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_target": 0, 2024-05-08T08:54:27.021 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_barrier": 0, 2024-05-08T08:54:27.021 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-08T08:54:27.021 INFO:teuthology.orchestra.run.smithi059.stdout: "object_hash": 2, 2024-05-08T08:54:27.021 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscale_mode": "off", 2024-05-08T08:54:27.021 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num": 1, 2024-05-08T08:54:27.021 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num": 1, 2024-05-08T08:54:27.021 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num_target": 1, 2024-05-08T08:54:27.021 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_target": 1, 2024-05-08T08:54:27.021 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_pending": 1, 2024-05-08T08:54:27.021 INFO:teuthology.orchestra.run.smithi059.stdout: "last_pg_merge_meta": { 2024-05-08T08:54:27.021 INFO:teuthology.orchestra.run.smithi059.stdout: "source_pgid": "0.0", 2024-05-08T08:54:27.021 INFO:teuthology.orchestra.run.smithi059.stdout: "ready_epoch": 0, 2024-05-08T08:54:27.021 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_started": 0, 2024-05-08T08:54:27.021 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_clean": 0, 2024-05-08T08:54:27.022 INFO:teuthology.orchestra.run.smithi059.stdout: "source_version": "0'0", 2024-05-08T08:54:27.022 INFO:teuthology.orchestra.run.smithi059.stdout: "target_version": "0'0" 2024-05-08T08:54:27.022 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.022 INFO:teuthology.orchestra.run.smithi059.stdout: "last_change": "14", 2024-05-08T08:54:27.022 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend": "0", 2024-05-08T08:54:27.022 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-08T08:54:27.022 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_preluminous": "0", 2024-05-08T08:54:27.022 INFO:teuthology.orchestra.run.smithi059.stdout: "auid": 0, 2024-05-08T08:54:27.022 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_mode": "selfmanaged", 2024-05-08T08:54:27.022 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_seq": 0, 2024-05-08T08:54:27.022 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_epoch": 0, 2024-05-08T08:54:27.022 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_snaps": [], 2024-05-08T08:54:27.022 INFO:teuthology.orchestra.run.smithi059.stdout: "removed_snaps": "[]", 2024-05-08T08:54:27.022 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_bytes": 0, 2024-05-08T08:54:27.022 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_objects": 0, 2024-05-08T08:54:27.023 INFO:teuthology.orchestra.run.smithi059.stdout: "tiers": [], 2024-05-08T08:54:27.023 INFO:teuthology.orchestra.run.smithi059.stdout: "tier_of": -1, 2024-05-08T08:54:27.023 INFO:teuthology.orchestra.run.smithi059.stdout: "read_tier": -1, 2024-05-08T08:54:27.023 INFO:teuthology.orchestra.run.smithi059.stdout: "write_tier": -1, 2024-05-08T08:54:27.023 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_mode": "none", 2024-05-08T08:54:27.023 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_bytes": 0, 2024-05-08T08:54:27.023 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_objects": 0, 2024-05-08T08:54:27.023 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-08T08:54:27.023 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-08T08:54:27.023 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-08T08:54:27.023 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_flush_age": 0, 2024-05-08T08:54:27.023 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_evict_age": 0, 2024-05-08T08:54:27.023 INFO:teuthology.orchestra.run.smithi059.stdout: "erasure_code_profile": "", 2024-05-08T08:54:27.024 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_params": { 2024-05-08T08:54:27.024 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "none" 2024-05-08T08:54:27.024 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.024 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_period": 0, 2024-05-08T08:54:27.024 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_count": 0, 2024-05-08T08:54:27.024 INFO:teuthology.orchestra.run.smithi059.stdout: "use_gmt_hitset": true, 2024-05-08T08:54:27.024 INFO:teuthology.orchestra.run.smithi059.stdout: "min_read_recency_for_promote": 0, 2024-05-08T08:54:27.024 INFO:teuthology.orchestra.run.smithi059.stdout: "min_write_recency_for_promote": 0, 2024-05-08T08:54:27.024 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_grade_decay_rate": 0, 2024-05-08T08:54:27.024 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_search_last_n": 0, 2024-05-08T08:54:27.024 INFO:teuthology.orchestra.run.smithi059.stdout: "grade_table": [], 2024-05-08T08:54:27.024 INFO:teuthology.orchestra.run.smithi059.stdout: "stripe_width": 0, 2024-05-08T08:54:27.024 INFO:teuthology.orchestra.run.smithi059.stdout: "expected_num_objects": 0, 2024-05-08T08:54:27.024 INFO:teuthology.orchestra.run.smithi059.stdout: "fast_read": false, 2024-05-08T08:54:27.024 INFO:teuthology.orchestra.run.smithi059.stdout: "options": { 2024-05-08T08:54:27.025 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_max": 32, 2024-05-08T08:54:27.025 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_min": 1 2024-05-08T08:54:27.025 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.025 INFO:teuthology.orchestra.run.smithi059.stdout: "application_metadata": { 2024-05-08T08:54:27.025 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": {} 2024-05-08T08:54:27.025 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.025 INFO:teuthology.orchestra.run.smithi059.stdout: "read_balance": { 2024-05-08T08:54:27.025 INFO:teuthology.orchestra.run.smithi059.stdout: "score_type": "Fair distribution", 2024-05-08T08:54:27.025 INFO:teuthology.orchestra.run.smithi059.stdout: "score_acting": 5, 2024-05-08T08:54:27.025 INFO:teuthology.orchestra.run.smithi059.stdout: "score_stable": 5, 2024-05-08T08:54:27.025 INFO:teuthology.orchestra.run.smithi059.stdout: "optimal_score": 0.40000000596046448, 2024-05-08T08:54:27.025 INFO:teuthology.orchestra.run.smithi059.stdout: "raw_score_acting": 2, 2024-05-08T08:54:27.025 INFO:teuthology.orchestra.run.smithi059.stdout: "raw_score_stable": 2, 2024-05-08T08:54:27.025 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity_weighted": 0.71152502298355103, 2024-05-08T08:54:27.026 INFO:teuthology.orchestra.run.smithi059.stdout: "average_primary_affinity": 0.85000002384185791, 2024-05-08T08:54:27.026 INFO:teuthology.orchestra.run.smithi059.stdout: "average_primary_affinity_weighted": 0.70999997854232788 2024-05-08T08:54:27.026 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.026 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.026 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.026 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 2, 2024-05-08T08:54:27.026 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_name": "rbd", 2024-05-08T08:54:27.026 INFO:teuthology.orchestra.run.smithi059.stdout: "create_time": "2024-05-08T08:52:02.563677+0000", 2024-05-08T08:54:27.026 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T08:54:27.026 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_names": "hashpspool", 2024-05-08T08:54:27.026 INFO:teuthology.orchestra.run.smithi059.stdout: "type": 1, 2024-05-08T08:54:27.026 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 2, 2024-05-08T08:54:27.026 INFO:teuthology.orchestra.run.smithi059.stdout: "min_size": 1, 2024-05-08T08:54:27.026 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_rule": 0, 2024-05-08T08:54:27.027 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_count": 0, 2024-05-08T08:54:27.027 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_target": 0, 2024-05-08T08:54:27.027 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_barrier": 0, 2024-05-08T08:54:27.027 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-08T08:54:27.027 INFO:teuthology.orchestra.run.smithi059.stdout: "object_hash": 2, 2024-05-08T08:54:27.027 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscale_mode": "off", 2024-05-08T08:54:27.027 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num": 8, 2024-05-08T08:54:27.027 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num": 8, 2024-05-08T08:54:27.027 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num_target": 8, 2024-05-08T08:54:27.027 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_target": 8, 2024-05-08T08:54:27.027 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_pending": 8, 2024-05-08T08:54:27.027 INFO:teuthology.orchestra.run.smithi059.stdout: "last_pg_merge_meta": { 2024-05-08T08:54:27.027 INFO:teuthology.orchestra.run.smithi059.stdout: "source_pgid": "0.0", 2024-05-08T08:54:27.027 INFO:teuthology.orchestra.run.smithi059.stdout: "ready_epoch": 0, 2024-05-08T08:54:27.028 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_started": 0, 2024-05-08T08:54:27.028 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_clean": 0, 2024-05-08T08:54:27.028 INFO:teuthology.orchestra.run.smithi059.stdout: "source_version": "0'0", 2024-05-08T08:54:27.028 INFO:teuthology.orchestra.run.smithi059.stdout: "target_version": "0'0" 2024-05-08T08:54:27.028 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.028 INFO:teuthology.orchestra.run.smithi059.stdout: "last_change": "17", 2024-05-08T08:54:27.028 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend": "0", 2024-05-08T08:54:27.028 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-08T08:54:27.028 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_preluminous": "0", 2024-05-08T08:54:27.028 INFO:teuthology.orchestra.run.smithi059.stdout: "auid": 0, 2024-05-08T08:54:27.028 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_mode": "selfmanaged", 2024-05-08T08:54:27.028 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_seq": 0, 2024-05-08T08:54:27.028 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_epoch": 0, 2024-05-08T08:54:27.028 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_snaps": [], 2024-05-08T08:54:27.028 INFO:teuthology.orchestra.run.smithi059.stdout: "removed_snaps": "[]", 2024-05-08T08:54:27.029 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_bytes": 0, 2024-05-08T08:54:27.029 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_objects": 0, 2024-05-08T08:54:27.029 INFO:teuthology.orchestra.run.smithi059.stdout: "tiers": [], 2024-05-08T08:54:27.029 INFO:teuthology.orchestra.run.smithi059.stdout: "tier_of": -1, 2024-05-08T08:54:27.029 INFO:teuthology.orchestra.run.smithi059.stdout: "read_tier": -1, 2024-05-08T08:54:27.029 INFO:teuthology.orchestra.run.smithi059.stdout: "write_tier": -1, 2024-05-08T08:54:27.029 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_mode": "none", 2024-05-08T08:54:27.029 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_bytes": 0, 2024-05-08T08:54:27.029 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_objects": 0, 2024-05-08T08:54:27.029 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-08T08:54:27.029 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-08T08:54:27.029 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-08T08:54:27.029 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_flush_age": 0, 2024-05-08T08:54:27.029 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_evict_age": 0, 2024-05-08T08:54:27.029 INFO:teuthology.orchestra.run.smithi059.stdout: "erasure_code_profile": "", 2024-05-08T08:54:27.030 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_params": { 2024-05-08T08:54:27.030 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "none" 2024-05-08T08:54:27.030 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.030 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_period": 0, 2024-05-08T08:54:27.030 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_count": 0, 2024-05-08T08:54:27.030 INFO:teuthology.orchestra.run.smithi059.stdout: "use_gmt_hitset": true, 2024-05-08T08:54:27.030 INFO:teuthology.orchestra.run.smithi059.stdout: "min_read_recency_for_promote": 0, 2024-05-08T08:54:27.030 INFO:teuthology.orchestra.run.smithi059.stdout: "min_write_recency_for_promote": 0, 2024-05-08T08:54:27.030 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_grade_decay_rate": 0, 2024-05-08T08:54:27.030 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_search_last_n": 0, 2024-05-08T08:54:27.030 INFO:teuthology.orchestra.run.smithi059.stdout: "grade_table": [], 2024-05-08T08:54:27.030 INFO:teuthology.orchestra.run.smithi059.stdout: "stripe_width": 0, 2024-05-08T08:54:27.030 INFO:teuthology.orchestra.run.smithi059.stdout: "expected_num_objects": 0, 2024-05-08T08:54:27.030 INFO:teuthology.orchestra.run.smithi059.stdout: "fast_read": false, 2024-05-08T08:54:27.031 INFO:teuthology.orchestra.run.smithi059.stdout: "options": {}, 2024-05-08T08:54:27.031 INFO:teuthology.orchestra.run.smithi059.stdout: "application_metadata": { 2024-05-08T08:54:27.031 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd": {} 2024-05-08T08:54:27.031 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.031 INFO:teuthology.orchestra.run.smithi059.stdout: "read_balance": { 2024-05-08T08:54:27.031 INFO:teuthology.orchestra.run.smithi059.stdout: "score_type": "Fair distribution", 2024-05-08T08:54:27.031 INFO:teuthology.orchestra.run.smithi059.stdout: "score_acting": 1.8799999952316284, 2024-05-08T08:54:27.031 INFO:teuthology.orchestra.run.smithi059.stdout: "score_stable": 1.8799999952316284, 2024-05-08T08:54:27.031 INFO:teuthology.orchestra.run.smithi059.stdout: "optimal_score": 1, 2024-05-08T08:54:27.031 INFO:teuthology.orchestra.run.smithi059.stdout: "raw_score_acting": 1.8799999952316284, 2024-05-08T08:54:27.031 INFO:teuthology.orchestra.run.smithi059.stdout: "raw_score_stable": 1.8799999952316284, 2024-05-08T08:54:27.031 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity_weighted": 0.71152502298355103, 2024-05-08T08:54:27.031 INFO:teuthology.orchestra.run.smithi059.stdout: "average_primary_affinity": 0.85000002384185791, 2024-05-08T08:54:27.031 INFO:teuthology.orchestra.run.smithi059.stdout: "average_primary_affinity_weighted": 0.70999997854232788 2024-05-08T08:54:27.031 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.032 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.032 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.032 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 3, 2024-05-08T08:54:27.032 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_name": "pool_client.0", 2024-05-08T08:54:27.032 INFO:teuthology.orchestra.run.smithi059.stdout: "create_time": "2024-05-08T08:52:09.955467+0000", 2024-05-08T08:54:27.032 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 8193, 2024-05-08T08:54:27.032 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-05-08T08:54:27.032 INFO:teuthology.orchestra.run.smithi059.stdout: "type": 1, 2024-05-08T08:54:27.032 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 2, 2024-05-08T08:54:27.032 INFO:teuthology.orchestra.run.smithi059.stdout: "min_size": 1, 2024-05-08T08:54:27.032 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_rule": 0, 2024-05-08T08:54:27.032 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_count": 0, 2024-05-08T08:54:27.032 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_target": 0, 2024-05-08T08:54:27.032 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_barrier": 0, 2024-05-08T08:54:27.032 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-08T08:54:27.032 INFO:teuthology.orchestra.run.smithi059.stdout: "object_hash": 2, 2024-05-08T08:54:27.033 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscale_mode": "off", 2024-05-08T08:54:27.033 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num": 32, 2024-05-08T08:54:27.033 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num": 32, 2024-05-08T08:54:27.033 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num_target": 32, 2024-05-08T08:54:27.033 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_target": 32, 2024-05-08T08:54:27.033 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_pending": 32, 2024-05-08T08:54:27.033 INFO:teuthology.orchestra.run.smithi059.stdout: "last_pg_merge_meta": { 2024-05-08T08:54:27.033 INFO:teuthology.orchestra.run.smithi059.stdout: "source_pgid": "0.0", 2024-05-08T08:54:27.033 INFO:teuthology.orchestra.run.smithi059.stdout: "ready_epoch": 0, 2024-05-08T08:54:27.033 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_started": 0, 2024-05-08T08:54:27.033 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_clean": 0, 2024-05-08T08:54:27.033 INFO:teuthology.orchestra.run.smithi059.stdout: "source_version": "0'0", 2024-05-08T08:54:27.033 INFO:teuthology.orchestra.run.smithi059.stdout: "target_version": "0'0" 2024-05-08T08:54:27.033 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.033 INFO:teuthology.orchestra.run.smithi059.stdout: "last_change": "115", 2024-05-08T08:54:27.033 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend": "0", 2024-05-08T08:54:27.034 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-08T08:54:27.034 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_preluminous": "0", 2024-05-08T08:54:27.034 INFO:teuthology.orchestra.run.smithi059.stdout: "auid": 0, 2024-05-08T08:54:27.034 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_mode": "selfmanaged", 2024-05-08T08:54:27.034 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_seq": 51, 2024-05-08T08:54:27.034 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_epoch": 115, 2024-05-08T08:54:27.034 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_snaps": [], 2024-05-08T08:54:27.034 INFO:teuthology.orchestra.run.smithi059.stdout: "removed_snaps": "[]", 2024-05-08T08:54:27.034 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_bytes": 0, 2024-05-08T08:54:27.034 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_objects": 0, 2024-05-08T08:54:27.034 INFO:teuthology.orchestra.run.smithi059.stdout: "tiers": [], 2024-05-08T08:54:27.034 INFO:teuthology.orchestra.run.smithi059.stdout: "tier_of": -1, 2024-05-08T08:54:27.034 INFO:teuthology.orchestra.run.smithi059.stdout: "read_tier": -1, 2024-05-08T08:54:27.035 INFO:teuthology.orchestra.run.smithi059.stdout: "write_tier": -1, 2024-05-08T08:54:27.035 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_mode": "none", 2024-05-08T08:54:27.035 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_bytes": 0, 2024-05-08T08:54:27.035 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_objects": 0, 2024-05-08T08:54:27.035 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-08T08:54:27.035 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-08T08:54:27.035 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-08T08:54:27.035 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_flush_age": 0, 2024-05-08T08:54:27.035 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_evict_age": 0, 2024-05-08T08:54:27.035 INFO:teuthology.orchestra.run.smithi059.stdout: "erasure_code_profile": "", 2024-05-08T08:54:27.035 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_params": { 2024-05-08T08:54:27.035 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "none" 2024-05-08T08:54:27.035 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.035 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_period": 0, 2024-05-08T08:54:27.036 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_count": 0, 2024-05-08T08:54:27.036 INFO:teuthology.orchestra.run.smithi059.stdout: "use_gmt_hitset": true, 2024-05-08T08:54:27.036 INFO:teuthology.orchestra.run.smithi059.stdout: "min_read_recency_for_promote": 0, 2024-05-08T08:54:27.036 INFO:teuthology.orchestra.run.smithi059.stdout: "min_write_recency_for_promote": 0, 2024-05-08T08:54:27.036 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_grade_decay_rate": 0, 2024-05-08T08:54:27.036 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_search_last_n": 0, 2024-05-08T08:54:27.036 INFO:teuthology.orchestra.run.smithi059.stdout: "grade_table": [], 2024-05-08T08:54:27.036 INFO:teuthology.orchestra.run.smithi059.stdout: "stripe_width": 0, 2024-05-08T08:54:27.036 INFO:teuthology.orchestra.run.smithi059.stdout: "expected_num_objects": 0, 2024-05-08T08:54:27.036 INFO:teuthology.orchestra.run.smithi059.stdout: "fast_read": false, 2024-05-08T08:54:27.036 INFO:teuthology.orchestra.run.smithi059.stdout: "options": {}, 2024-05-08T08:54:27.036 INFO:teuthology.orchestra.run.smithi059.stdout: "application_metadata": { 2024-05-08T08:54:27.036 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd": {} 2024-05-08T08:54:27.037 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.037 INFO:teuthology.orchestra.run.smithi059.stdout: "read_balance": { 2024-05-08T08:54:27.037 INFO:teuthology.orchestra.run.smithi059.stdout: "score_type": "Fair distribution", 2024-05-08T08:54:27.037 INFO:teuthology.orchestra.run.smithi059.stdout: "score_acting": 1.7200000286102295, 2024-05-08T08:54:27.037 INFO:teuthology.orchestra.run.smithi059.stdout: "score_stable": 1.7200000286102295, 2024-05-08T08:54:27.037 INFO:teuthology.orchestra.run.smithi059.stdout: "optimal_score": 1, 2024-05-08T08:54:27.037 INFO:teuthology.orchestra.run.smithi059.stdout: "raw_score_acting": 1.7200000286102295, 2024-05-08T08:54:27.037 INFO:teuthology.orchestra.run.smithi059.stdout: "raw_score_stable": 1.7200000286102295, 2024-05-08T08:54:27.037 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity_weighted": 0.71152502298355103, 2024-05-08T08:54:27.037 INFO:teuthology.orchestra.run.smithi059.stdout: "average_primary_affinity": 0.85000002384185791, 2024-05-08T08:54:27.037 INFO:teuthology.orchestra.run.smithi059.stdout: "average_primary_affinity_weighted": 0.70999997854232788 2024-05-08T08:54:27.037 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.037 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.037 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T08:54:27.037 INFO:teuthology.orchestra.run.smithi059.stdout: "osds": [ 2024-05-08T08:54:27.038 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.038 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 0, 2024-05-08T08:54:27.038 INFO:teuthology.orchestra.run.smithi059.stdout: "uuid": "300801a5-1919-4b23-8c6d-bc7dad61cc4d", 2024-05-08T08:54:27.038 INFO:teuthology.orchestra.run.smithi059.stdout: "up": 1, 2024-05-08T08:54:27.038 INFO:teuthology.orchestra.run.smithi059.stdout: "in": 1, 2024-05-08T08:54:27.038 INFO:teuthology.orchestra.run.smithi059.stdout: "weight": 1, 2024-05-08T08:54:27.038 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity": 0, 2024-05-08T08:54:27.038 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_begin": 0, 2024-05-08T08:54:27.038 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_end": 0, 2024-05-08T08:54:27.038 INFO:teuthology.orchestra.run.smithi059.stdout: "up_from": 11, 2024-05-08T08:54:27.038 INFO:teuthology.orchestra.run.smithi059.stdout: "up_thru": 95, 2024-05-08T08:54:27.038 INFO:teuthology.orchestra.run.smithi059.stdout: "down_at": 0, 2024-05-08T08:54:27.038 INFO:teuthology.orchestra.run.smithi059.stdout: "lost_at": 0, 2024-05-08T08:54:27.039 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrs": { 2024-05-08T08:54:27.039 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:54:27.039 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.039 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:54:27.039 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6800", 2024-05-08T08:54:27.039 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 802160151 2024-05-08T08:54:27.039 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.039 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.039 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:54:27.039 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6801", 2024-05-08T08:54:27.039 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 802160151 2024-05-08T08:54:27.039 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.039 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:27.039 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.039 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addrs": { 2024-05-08T08:54:27.040 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:54:27.040 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.040 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:54:27.040 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6802", 2024-05-08T08:54:27.040 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 802160151 2024-05-08T08:54:27.040 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.040 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.040 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:54:27.040 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6803", 2024-05-08T08:54:27.040 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 802160151 2024-05-08T08:54:27.040 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.040 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:27.040 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.040 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addrs": { 2024-05-08T08:54:27.040 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:54:27.041 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.041 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:54:27.041 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6806", 2024-05-08T08:54:27.041 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 802160151 2024-05-08T08:54:27.041 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.041 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.041 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:54:27.041 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6807", 2024-05-08T08:54:27.041 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 802160151 2024-05-08T08:54:27.041 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.041 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:27.041 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.041 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addrs": { 2024-05-08T08:54:27.041 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:54:27.041 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.042 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:54:27.042 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6804", 2024-05-08T08:54:27.042 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 802160151 2024-05-08T08:54:27.042 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.042 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.042 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:54:27.042 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6805", 2024-05-08T08:54:27.042 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 802160151 2024-05-08T08:54:27.042 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.042 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:27.042 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.042 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "172.21.15.59:6801/802160151", 2024-05-08T08:54:27.042 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "172.21.15.59:6803/802160151", 2024-05-08T08:54:27.042 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addr": "172.21.15.59:6807/802160151", 2024-05-08T08:54:27.043 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addr": "172.21.15.59:6805/802160151", 2024-05-08T08:54:27.043 INFO:teuthology.orchestra.run.smithi059.stdout: "state": [ 2024-05-08T08:54:27.043 INFO:teuthology.orchestra.run.smithi059.stdout: "exists", 2024-05-08T08:54:27.043 INFO:teuthology.orchestra.run.smithi059.stdout: "up" 2024-05-08T08:54:27.043 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:27.043 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.043 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.043 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 1, 2024-05-08T08:54:27.043 INFO:teuthology.orchestra.run.smithi059.stdout: "uuid": "0fead0de-6a65-4eba-b400-eea5692702e0", 2024-05-08T08:54:27.043 INFO:teuthology.orchestra.run.smithi059.stdout: "up": 1, 2024-05-08T08:54:27.043 INFO:teuthology.orchestra.run.smithi059.stdout: "in": 1, 2024-05-08T08:54:27.043 INFO:teuthology.orchestra.run.smithi059.stdout: "weight": 1, 2024-05-08T08:54:27.043 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity": 1, 2024-05-08T08:54:27.043 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_begin": 0, 2024-05-08T08:54:27.043 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_end": 0, 2024-05-08T08:54:27.044 INFO:teuthology.orchestra.run.smithi059.stdout: "up_from": 11, 2024-05-08T08:54:27.044 INFO:teuthology.orchestra.run.smithi059.stdout: "up_thru": 95, 2024-05-08T08:54:27.044 INFO:teuthology.orchestra.run.smithi059.stdout: "down_at": 0, 2024-05-08T08:54:27.044 INFO:teuthology.orchestra.run.smithi059.stdout: "lost_at": 0, 2024-05-08T08:54:27.044 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrs": { 2024-05-08T08:54:27.044 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:54:27.044 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.044 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:54:27.044 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6816", 2024-05-08T08:54:27.044 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4116801011 2024-05-08T08:54:27.044 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.044 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.044 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:54:27.044 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6817", 2024-05-08T08:54:27.044 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4116801011 2024-05-08T08:54:27.045 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.045 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:27.045 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.045 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addrs": { 2024-05-08T08:54:27.045 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:54:27.045 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.045 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:54:27.045 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6818", 2024-05-08T08:54:27.045 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4116801011 2024-05-08T08:54:27.045 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.045 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.045 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:54:27.045 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6819", 2024-05-08T08:54:27.045 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4116801011 2024-05-08T08:54:27.045 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.046 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:27.046 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.046 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addrs": { 2024-05-08T08:54:27.046 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:54:27.046 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.046 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:54:27.046 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6822", 2024-05-08T08:54:27.046 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4116801011 2024-05-08T08:54:27.046 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.046 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.046 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:54:27.046 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6823", 2024-05-08T08:54:27.046 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4116801011 2024-05-08T08:54:27.046 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.047 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:27.047 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.047 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addrs": { 2024-05-08T08:54:27.047 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:54:27.047 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.047 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:54:27.047 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6820", 2024-05-08T08:54:27.047 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4116801011 2024-05-08T08:54:27.047 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.047 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.047 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:54:27.047 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6821", 2024-05-08T08:54:27.047 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4116801011 2024-05-08T08:54:27.047 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.048 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:27.048 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.048 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "172.21.15.59:6817/4116801011", 2024-05-08T08:54:27.048 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "172.21.15.59:6819/4116801011", 2024-05-08T08:54:27.048 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addr": "172.21.15.59:6823/4116801011", 2024-05-08T08:54:27.048 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addr": "172.21.15.59:6821/4116801011", 2024-05-08T08:54:27.048 INFO:teuthology.orchestra.run.smithi059.stdout: "state": [ 2024-05-08T08:54:27.048 INFO:teuthology.orchestra.run.smithi059.stdout: "exists", 2024-05-08T08:54:27.048 INFO:teuthology.orchestra.run.smithi059.stdout: "up" 2024-05-08T08:54:27.048 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:27.048 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.048 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.048 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 2, 2024-05-08T08:54:27.048 INFO:teuthology.orchestra.run.smithi059.stdout: "uuid": "1a233de6-1b82-4a51-8e96-4cd4abdf16c8", 2024-05-08T08:54:27.048 INFO:teuthology.orchestra.run.smithi059.stdout: "up": 1, 2024-05-08T08:54:27.049 INFO:teuthology.orchestra.run.smithi059.stdout: "in": 1, 2024-05-08T08:54:27.049 INFO:teuthology.orchestra.run.smithi059.stdout: "weight": 1, 2024-05-08T08:54:27.049 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity": 1, 2024-05-08T08:54:27.049 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_begin": 0, 2024-05-08T08:54:27.049 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_end": 0, 2024-05-08T08:54:27.049 INFO:teuthology.orchestra.run.smithi059.stdout: "up_from": 11, 2024-05-08T08:54:27.049 INFO:teuthology.orchestra.run.smithi059.stdout: "up_thru": 95, 2024-05-08T08:54:27.049 INFO:teuthology.orchestra.run.smithi059.stdout: "down_at": 0, 2024-05-08T08:54:27.049 INFO:teuthology.orchestra.run.smithi059.stdout: "lost_at": 0, 2024-05-08T08:54:27.049 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrs": { 2024-05-08T08:54:27.049 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:54:27.049 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.049 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:54:27.049 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6808", 2024-05-08T08:54:27.049 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1547661179 2024-05-08T08:54:27.049 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.050 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.050 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:54:27.050 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6809", 2024-05-08T08:54:27.050 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1547661179 2024-05-08T08:54:27.050 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.050 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:27.050 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.050 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addrs": { 2024-05-08T08:54:27.050 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:54:27.050 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.050 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:54:27.050 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6810", 2024-05-08T08:54:27.050 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1547661179 2024-05-08T08:54:27.051 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.051 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.051 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:54:27.051 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6811", 2024-05-08T08:54:27.051 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1547661179 2024-05-08T08:54:27.051 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.051 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:27.051 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.051 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addrs": { 2024-05-08T08:54:27.051 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:54:27.051 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.051 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:54:27.051 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6814", 2024-05-08T08:54:27.051 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1547661179 2024-05-08T08:54:27.051 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.052 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.052 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:54:27.052 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6815", 2024-05-08T08:54:27.052 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1547661179 2024-05-08T08:54:27.052 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.052 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:27.052 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.052 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addrs": { 2024-05-08T08:54:27.052 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:54:27.052 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.052 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:54:27.052 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6812", 2024-05-08T08:54:27.052 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1547661179 2024-05-08T08:54:27.052 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.052 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.053 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:54:27.053 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6813", 2024-05-08T08:54:27.053 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1547661179 2024-05-08T08:54:27.053 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.053 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:27.053 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.053 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "172.21.15.59:6809/1547661179", 2024-05-08T08:54:27.053 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "172.21.15.59:6811/1547661179", 2024-05-08T08:54:27.053 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addr": "172.21.15.59:6815/1547661179", 2024-05-08T08:54:27.053 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addr": "172.21.15.59:6813/1547661179", 2024-05-08T08:54:27.053 INFO:teuthology.orchestra.run.smithi059.stdout: "state": [ 2024-05-08T08:54:27.053 INFO:teuthology.orchestra.run.smithi059.stdout: "exists", 2024-05-08T08:54:27.053 INFO:teuthology.orchestra.run.smithi059.stdout: "up" 2024-05-08T08:54:27.053 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:27.054 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.054 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.054 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 3, 2024-05-08T08:54:27.054 INFO:teuthology.orchestra.run.smithi059.stdout: "uuid": "cc4870b8-dcd9-4e43-978d-8de6c6396abf", 2024-05-08T08:54:27.054 INFO:teuthology.orchestra.run.smithi059.stdout: "up": 1, 2024-05-08T08:54:27.054 INFO:teuthology.orchestra.run.smithi059.stdout: "in": 1, 2024-05-08T08:54:27.054 INFO:teuthology.orchestra.run.smithi059.stdout: "weight": 1, 2024-05-08T08:54:27.054 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity": 1, 2024-05-08T08:54:27.054 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_begin": 0, 2024-05-08T08:54:27.054 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_end": 0, 2024-05-08T08:54:27.054 INFO:teuthology.orchestra.run.smithi059.stdout: "up_from": 11, 2024-05-08T08:54:27.054 INFO:teuthology.orchestra.run.smithi059.stdout: "up_thru": 96, 2024-05-08T08:54:27.054 INFO:teuthology.orchestra.run.smithi059.stdout: "down_at": 0, 2024-05-08T08:54:27.054 INFO:teuthology.orchestra.run.smithi059.stdout: "lost_at": 0, 2024-05-08T08:54:27.055 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrs": { 2024-05-08T08:54:27.055 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:54:27.055 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.055 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:54:27.055 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6800", 2024-05-08T08:54:27.055 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4054400810 2024-05-08T08:54:27.055 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.055 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.055 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:54:27.055 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6801", 2024-05-08T08:54:27.055 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4054400810 2024-05-08T08:54:27.055 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.055 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:27.055 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.055 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addrs": { 2024-05-08T08:54:27.056 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:54:27.056 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.056 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:54:27.056 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6802", 2024-05-08T08:54:27.056 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4054400810 2024-05-08T08:54:27.056 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.056 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.056 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:54:27.056 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6803", 2024-05-08T08:54:27.056 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4054400810 2024-05-08T08:54:27.056 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.056 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:27.056 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.056 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addrs": { 2024-05-08T08:54:27.056 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:54:27.057 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.057 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:54:27.057 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6806", 2024-05-08T08:54:27.057 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4054400810 2024-05-08T08:54:27.057 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.057 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.057 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:54:27.057 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6807", 2024-05-08T08:54:27.057 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4054400810 2024-05-08T08:54:27.057 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.057 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:27.057 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.057 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addrs": { 2024-05-08T08:54:27.057 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:54:27.058 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.058 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:54:27.058 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6804", 2024-05-08T08:54:27.058 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4054400810 2024-05-08T08:54:27.058 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.058 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.058 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:54:27.058 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6805", 2024-05-08T08:54:27.058 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4054400810 2024-05-08T08:54:27.058 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.058 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:27.058 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.058 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "172.21.15.192:6801/4054400810", 2024-05-08T08:54:27.058 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "172.21.15.192:6803/4054400810", 2024-05-08T08:54:27.059 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addr": "172.21.15.192:6807/4054400810", 2024-05-08T08:54:27.059 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addr": "172.21.15.192:6805/4054400810", 2024-05-08T08:54:27.059 INFO:teuthology.orchestra.run.smithi059.stdout: "state": [ 2024-05-08T08:54:27.059 INFO:teuthology.orchestra.run.smithi059.stdout: "exists", 2024-05-08T08:54:27.059 INFO:teuthology.orchestra.run.smithi059.stdout: "up" 2024-05-08T08:54:27.059 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:27.059 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.059 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.059 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 4, 2024-05-08T08:54:27.059 INFO:teuthology.orchestra.run.smithi059.stdout: "uuid": "d1d05620-42fb-4107-a2f9-e83542d076ee", 2024-05-08T08:54:27.059 INFO:teuthology.orchestra.run.smithi059.stdout: "up": 1, 2024-05-08T08:54:27.059 INFO:teuthology.orchestra.run.smithi059.stdout: "in": 1, 2024-05-08T08:54:27.059 INFO:teuthology.orchestra.run.smithi059.stdout: "weight": 1, 2024-05-08T08:54:27.059 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity": 0.9892578125, 2024-05-08T08:54:27.059 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_begin": 0, 2024-05-08T08:54:27.060 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_end": 0, 2024-05-08T08:54:27.060 INFO:teuthology.orchestra.run.smithi059.stdout: "up_from": 11, 2024-05-08T08:54:27.060 INFO:teuthology.orchestra.run.smithi059.stdout: "up_thru": 96, 2024-05-08T08:54:27.060 INFO:teuthology.orchestra.run.smithi059.stdout: "down_at": 0, 2024-05-08T08:54:27.060 INFO:teuthology.orchestra.run.smithi059.stdout: "lost_at": 0, 2024-05-08T08:54:27.060 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrs": { 2024-05-08T08:54:27.060 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:54:27.060 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.060 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:54:27.060 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6808", 2024-05-08T08:54:27.060 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 58985565 2024-05-08T08:54:27.060 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.060 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.060 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:54:27.061 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6809", 2024-05-08T08:54:27.061 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 58985565 2024-05-08T08:54:27.061 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.061 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:27.061 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.061 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addrs": { 2024-05-08T08:54:27.061 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:54:27.061 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.061 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:54:27.061 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6810", 2024-05-08T08:54:27.061 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 58985565 2024-05-08T08:54:27.061 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.061 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.061 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:54:27.061 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6811", 2024-05-08T08:54:27.062 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 58985565 2024-05-08T08:54:27.062 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.062 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:27.062 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.062 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addrs": { 2024-05-08T08:54:27.062 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:54:27.062 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.062 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:54:27.062 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6814", 2024-05-08T08:54:27.062 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 58985565 2024-05-08T08:54:27.062 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.062 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.062 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:54:27.062 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6815", 2024-05-08T08:54:27.063 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 58985565 2024-05-08T08:54:27.063 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.063 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:27.063 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.063 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addrs": { 2024-05-08T08:54:27.063 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:54:27.063 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.063 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:54:27.063 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6812", 2024-05-08T08:54:27.063 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 58985565 2024-05-08T08:54:27.063 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.063 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.063 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:54:27.063 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6813", 2024-05-08T08:54:27.064 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 58985565 2024-05-08T08:54:27.064 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.064 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:27.064 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.064 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "172.21.15.192:6809/58985565", 2024-05-08T08:54:27.064 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "172.21.15.192:6811/58985565", 2024-05-08T08:54:27.064 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addr": "172.21.15.192:6815/58985565", 2024-05-08T08:54:27.064 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addr": "172.21.15.192:6813/58985565", 2024-05-08T08:54:27.064 INFO:teuthology.orchestra.run.smithi059.stdout: "state": [ 2024-05-08T08:54:27.064 INFO:teuthology.orchestra.run.smithi059.stdout: "exists", 2024-05-08T08:54:27.064 INFO:teuthology.orchestra.run.smithi059.stdout: "up" 2024-05-08T08:54:27.064 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:27.064 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.064 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.064 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 5, 2024-05-08T08:54:27.065 INFO:teuthology.orchestra.run.smithi059.stdout: "uuid": "ab26136c-31c3-4090-9fa2-988dba4f731f", 2024-05-08T08:54:27.065 INFO:teuthology.orchestra.run.smithi059.stdout: "up": 1, 2024-05-08T08:54:27.065 INFO:teuthology.orchestra.run.smithi059.stdout: "in": 0, 2024-05-08T08:54:27.065 INFO:teuthology.orchestra.run.smithi059.stdout: "weight": 0, 2024-05-08T08:54:27.065 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity": 0.2798919677734375, 2024-05-08T08:54:27.065 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_begin": 0, 2024-05-08T08:54:27.065 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_end": 0, 2024-05-08T08:54:27.065 INFO:teuthology.orchestra.run.smithi059.stdout: "up_from": 11, 2024-05-08T08:54:27.065 INFO:teuthology.orchestra.run.smithi059.stdout: "up_thru": 88, 2024-05-08T08:54:27.065 INFO:teuthology.orchestra.run.smithi059.stdout: "down_at": 0, 2024-05-08T08:54:27.065 INFO:teuthology.orchestra.run.smithi059.stdout: "lost_at": 0, 2024-05-08T08:54:27.065 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrs": { 2024-05-08T08:54:27.065 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:54:27.065 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.065 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:54:27.066 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6816", 2024-05-08T08:54:27.066 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1283571573 2024-05-08T08:54:27.066 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.066 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.066 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:54:27.066 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6817", 2024-05-08T08:54:27.066 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1283571573 2024-05-08T08:54:27.066 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.066 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:27.066 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.066 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addrs": { 2024-05-08T08:54:27.066 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:54:27.066 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.066 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:54:27.066 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6818", 2024-05-08T08:54:27.067 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1283571573 2024-05-08T08:54:27.067 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.067 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.067 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:54:27.067 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6819", 2024-05-08T08:54:27.067 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1283571573 2024-05-08T08:54:27.067 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.067 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:27.067 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.067 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addrs": { 2024-05-08T08:54:27.067 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:54:27.067 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.067 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:54:27.067 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6822", 2024-05-08T08:54:27.067 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1283571573 2024-05-08T08:54:27.068 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.068 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.068 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:54:27.068 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6823", 2024-05-08T08:54:27.068 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1283571573 2024-05-08T08:54:27.068 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.068 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:27.068 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.068 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addrs": { 2024-05-08T08:54:27.068 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:54:27.068 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.068 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:54:27.068 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6820", 2024-05-08T08:54:27.068 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1283571573 2024-05-08T08:54:27.069 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.069 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.069 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:54:27.069 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6821", 2024-05-08T08:54:27.069 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1283571573 2024-05-08T08:54:27.069 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.069 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:27.069 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.069 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "172.21.15.192:6817/1283571573", 2024-05-08T08:54:27.069 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "172.21.15.192:6819/1283571573", 2024-05-08T08:54:27.069 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addr": "172.21.15.192:6823/1283571573", 2024-05-08T08:54:27.069 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addr": "172.21.15.192:6821/1283571573", 2024-05-08T08:54:27.069 INFO:teuthology.orchestra.run.smithi059.stdout: "state": [ 2024-05-08T08:54:27.069 INFO:teuthology.orchestra.run.smithi059.stdout: "exists", 2024-05-08T08:54:27.070 INFO:teuthology.orchestra.run.smithi059.stdout: "up" 2024-05-08T08:54:27.070 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:27.070 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.070 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T08:54:27.070 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_xinfo": [ 2024-05-08T08:54:27.070 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.070 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 0, 2024-05-08T08:54:27.070 INFO:teuthology.orchestra.run.smithi059.stdout: "down_stamp": "0.000000", 2024-05-08T08:54:27.070 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_probability": 0, 2024-05-08T08:54:27.070 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_interval": 0, 2024-05-08T08:54:27.070 INFO:teuthology.orchestra.run.smithi059.stdout: "features": 4540701547738038271, 2024-05-08T08:54:27.070 INFO:teuthology.orchestra.run.smithi059.stdout: "old_weight": 0, 2024-05-08T08:54:27.070 INFO:teuthology.orchestra.run.smithi059.stdout: "last_purged_snaps_scrub": "2024-05-08T08:51:52.396946+0000", 2024-05-08T08:54:27.070 INFO:teuthology.orchestra.run.smithi059.stdout: "dead_epoch": 0 2024-05-08T08:54:27.071 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.071 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.071 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 1, 2024-05-08T08:54:27.071 INFO:teuthology.orchestra.run.smithi059.stdout: "down_stamp": "0.000000", 2024-05-08T08:54:27.071 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_probability": 0, 2024-05-08T08:54:27.071 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_interval": 0, 2024-05-08T08:54:27.071 INFO:teuthology.orchestra.run.smithi059.stdout: "features": 4540701547738038271, 2024-05-08T08:54:27.071 INFO:teuthology.orchestra.run.smithi059.stdout: "old_weight": 0, 2024-05-08T08:54:27.071 INFO:teuthology.orchestra.run.smithi059.stdout: "last_purged_snaps_scrub": "2024-05-08T08:51:52.453160+0000", 2024-05-08T08:54:27.071 INFO:teuthology.orchestra.run.smithi059.stdout: "dead_epoch": 0 2024-05-08T08:54:27.071 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.071 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.071 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 2, 2024-05-08T08:54:27.071 INFO:teuthology.orchestra.run.smithi059.stdout: "down_stamp": "0.000000", 2024-05-08T08:54:27.071 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_probability": 0, 2024-05-08T08:54:27.072 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_interval": 0, 2024-05-08T08:54:27.072 INFO:teuthology.orchestra.run.smithi059.stdout: "features": 4540701547738038271, 2024-05-08T08:54:27.072 INFO:teuthology.orchestra.run.smithi059.stdout: "old_weight": 0, 2024-05-08T08:54:27.072 INFO:teuthology.orchestra.run.smithi059.stdout: "last_purged_snaps_scrub": "2024-05-08T08:51:52.387606+0000", 2024-05-08T08:54:27.072 INFO:teuthology.orchestra.run.smithi059.stdout: "dead_epoch": 0 2024-05-08T08:54:27.072 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.072 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.072 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 3, 2024-05-08T08:54:27.072 INFO:teuthology.orchestra.run.smithi059.stdout: "down_stamp": "0.000000", 2024-05-08T08:54:27.072 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_probability": 0, 2024-05-08T08:54:27.072 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_interval": 0, 2024-05-08T08:54:27.072 INFO:teuthology.orchestra.run.smithi059.stdout: "features": 4540701547738038271, 2024-05-08T08:54:27.072 INFO:teuthology.orchestra.run.smithi059.stdout: "old_weight": 0, 2024-05-08T08:54:27.072 INFO:teuthology.orchestra.run.smithi059.stdout: "last_purged_snaps_scrub": "2024-05-08T08:51:52.360952+0000", 2024-05-08T08:54:27.072 INFO:teuthology.orchestra.run.smithi059.stdout: "dead_epoch": 0 2024-05-08T08:54:27.073 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.073 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.073 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 4, 2024-05-08T08:54:27.073 INFO:teuthology.orchestra.run.smithi059.stdout: "down_stamp": "0.000000", 2024-05-08T08:54:27.073 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_probability": 0, 2024-05-08T08:54:27.073 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_interval": 0, 2024-05-08T08:54:27.073 INFO:teuthology.orchestra.run.smithi059.stdout: "features": 4540701547738038271, 2024-05-08T08:54:27.073 INFO:teuthology.orchestra.run.smithi059.stdout: "old_weight": 0, 2024-05-08T08:54:27.073 INFO:teuthology.orchestra.run.smithi059.stdout: "last_purged_snaps_scrub": "2024-05-08T08:51:52.468681+0000", 2024-05-08T08:54:27.073 INFO:teuthology.orchestra.run.smithi059.stdout: "dead_epoch": 0 2024-05-08T08:54:27.073 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.073 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.073 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 5, 2024-05-08T08:54:27.073 INFO:teuthology.orchestra.run.smithi059.stdout: "down_stamp": "0.000000", 2024-05-08T08:54:27.074 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_probability": 0, 2024-05-08T08:54:27.074 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_interval": 0, 2024-05-08T08:54:27.074 INFO:teuthology.orchestra.run.smithi059.stdout: "features": 4540701547738038271, 2024-05-08T08:54:27.074 INFO:teuthology.orchestra.run.smithi059.stdout: "old_weight": 65536, 2024-05-08T08:54:27.074 INFO:teuthology.orchestra.run.smithi059.stdout: "last_purged_snaps_scrub": "2024-05-08T08:51:52.401080+0000", 2024-05-08T08:54:27.074 INFO:teuthology.orchestra.run.smithi059.stdout: "dead_epoch": 0 2024-05-08T08:54:27.075 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.075 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T08:54:27.075 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_upmap": [ 2024-05-08T08:54:27.075 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.075 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "3.4", 2024-05-08T08:54:27.075 INFO:teuthology.orchestra.run.smithi059.stdout: "osds": [ 2024-05-08T08:54:27.075 INFO:teuthology.orchestra.run.smithi059.stdout: 1, 2024-05-08T08:54:27.075 INFO:teuthology.orchestra.run.smithi059.stdout: 5 2024-05-08T08:54:27.075 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:27.075 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.075 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.075 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "3.17", 2024-05-08T08:54:27.075 INFO:teuthology.orchestra.run.smithi059.stdout: "osds": [ 2024-05-08T08:54:27.075 INFO:teuthology.orchestra.run.smithi059.stdout: 1, 2024-05-08T08:54:27.075 INFO:teuthology.orchestra.run.smithi059.stdout: 0 2024-05-08T08:54:27.076 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:27.076 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.076 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.076 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "3.1d", 2024-05-08T08:54:27.076 INFO:teuthology.orchestra.run.smithi059.stdout: "osds": [ 2024-05-08T08:54:27.076 INFO:teuthology.orchestra.run.smithi059.stdout: 0, 2024-05-08T08:54:27.076 INFO:teuthology.orchestra.run.smithi059.stdout: 5 2024-05-08T08:54:27.076 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:27.076 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.076 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T08:54:27.076 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_upmap_items": [], 2024-05-08T08:54:27.076 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_upmap_primaries": [], 2024-05-08T08:54:27.076 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_temp": [], 2024-05-08T08:54:27.076 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_temp": [], 2024-05-08T08:54:27.076 INFO:teuthology.orchestra.run.smithi059.stdout: "blocklist": {}, 2024-05-08T08:54:27.077 INFO:teuthology.orchestra.run.smithi059.stdout: "range_blocklist": {}, 2024-05-08T08:54:27.077 INFO:teuthology.orchestra.run.smithi059.stdout: "erasure_code_profiles": { 2024-05-08T08:54:27.077 INFO:teuthology.orchestra.run.smithi059.stdout: "default": { 2024-05-08T08:54:27.077 INFO:teuthology.orchestra.run.smithi059.stdout: "crush-failure-domain": "osd", 2024-05-08T08:54:27.077 INFO:teuthology.orchestra.run.smithi059.stdout: "k": "2", 2024-05-08T08:54:27.077 INFO:teuthology.orchestra.run.smithi059.stdout: "m": "1", 2024-05-08T08:54:27.077 INFO:teuthology.orchestra.run.smithi059.stdout: "plugin": "jerasure", 2024-05-08T08:54:27.077 INFO:teuthology.orchestra.run.smithi059.stdout: "technique": "reed_sol_van" 2024-05-08T08:54:27.077 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.077 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.077 INFO:teuthology.orchestra.run.smithi059.stdout: "removed_snaps_queue": [], 2024-05-08T08:54:27.077 INFO:teuthology.orchestra.run.smithi059.stdout: "new_removed_snaps": [], 2024-05-08T08:54:27.077 INFO:teuthology.orchestra.run.smithi059.stdout: "new_purged_snaps": [], 2024-05-08T08:54:27.077 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_node_flags": {}, 2024-05-08T08:54:27.078 INFO:teuthology.orchestra.run.smithi059.stdout: "device_class_flags": {}, 2024-05-08T08:54:27.078 INFO:teuthology.orchestra.run.smithi059.stdout: "stretch_mode": { 2024-05-08T08:54:27.078 INFO:teuthology.orchestra.run.smithi059.stdout: "stretch_mode_enabled": false, 2024-05-08T08:54:27.078 INFO:teuthology.orchestra.run.smithi059.stdout: "stretch_bucket_count": 0, 2024-05-08T08:54:27.078 INFO:teuthology.orchestra.run.smithi059.stdout: "degraded_stretch_mode": 0, 2024-05-08T08:54:27.078 INFO:teuthology.orchestra.run.smithi059.stdout: "recovering_stretch_mode": 0, 2024-05-08T08:54:27.078 INFO:teuthology.orchestra.run.smithi059.stdout: "stretch_mode_bucket": 0 2024-05-08T08:54:27.078 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.078 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:27.080 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:54:27.081 INFO:tasks.thrashosds.thrasher:j is {'epoch': 115, 'fsid': '38dfae8c-5c5e-4b0f-a264-91f22c6c5554', 'created': '2024-05-08T08:51:45.571369+0000', 'modified': '2024-05-08T08:54:26.754375+0000', 'last_up_change': '2024-05-08T08:51:56.847706+0000', 'last_in_change': '2024-05-08T08:53:54.592551+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T08:51:57.787790+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '14', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 5, 'score_stable': 5, 'optimal_score': 0.4000000059604645, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.711525022983551, 'average_primary_affinity': 0.8500000238418579, 'average_primary_affinity_weighted': 0.7099999785423279}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-05-08T08:52:02.563677+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.711525022983551, 'average_primary_affinity': 0.8500000238418579, 'average_primary_affinity_weighted': 0.7099999785423279}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-05-08T08:52:09.955467+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '115', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 51, 'snap_epoch': 115, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.7200000286102295, 'score_stable': 1.7200000286102295, 'optimal_score': 1, 'raw_score_acting': 1.7200000286102295, 'raw_score_stable': 1.7200000286102295, 'primary_affinity_weighted': 0.711525022983551, 'average_primary_affinity': 0.8500000238418579, 'average_primary_affinity_weighted': 0.7099999785423279}}], 'osds': [{'osd': 0, 'uuid': '300801a5-1919-4b23-8c6d-bc7dad61cc4d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 95, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6800', 'nonce': 802160151}, {'type': 'v1', 'addr': '172.21.15.59:6801', 'nonce': 802160151}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6802', 'nonce': 802160151}, {'type': 'v1', 'addr': '172.21.15.59:6803', 'nonce': 802160151}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6806', 'nonce': 802160151}, {'type': 'v1', 'addr': '172.21.15.59:6807', 'nonce': 802160151}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6804', 'nonce': 802160151}, {'type': 'v1', 'addr': '172.21.15.59:6805', 'nonce': 802160151}]}, 'public_addr': '172.21.15.59:6801/802160151', 'cluster_addr': '172.21.15.59:6803/802160151', 'heartbeat_back_addr': '172.21.15.59:6807/802160151', 'heartbeat_front_addr': '172.21.15.59:6805/802160151', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '0fead0de-6a65-4eba-b400-eea5692702e0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 95, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6816', 'nonce': 4116801011}, {'type': 'v1', 'addr': '172.21.15.59:6817', 'nonce': 4116801011}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6818', 'nonce': 4116801011}, {'type': 'v1', 'addr': '172.21.15.59:6819', 'nonce': 4116801011}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6822', 'nonce': 4116801011}, {'type': 'v1', 'addr': '172.21.15.59:6823', 'nonce': 4116801011}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6820', 'nonce': 4116801011}, {'type': 'v1', 'addr': '172.21.15.59:6821', 'nonce': 4116801011}]}, 'public_addr': '172.21.15.59:6817/4116801011', 'cluster_addr': '172.21.15.59:6819/4116801011', 'heartbeat_back_addr': '172.21.15.59:6823/4116801011', 'heartbeat_front_addr': '172.21.15.59:6821/4116801011', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '1a233de6-1b82-4a51-8e96-4cd4abdf16c8', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 95, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6808', 'nonce': 1547661179}, {'type': 'v1', 'addr': '172.21.15.59:6809', 'nonce': 1547661179}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6810', 'nonce': 1547661179}, {'type': 'v1', 'addr': '172.21.15.59:6811', 'nonce': 1547661179}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6814', 'nonce': 1547661179}, {'type': 'v1', 'addr': '172.21.15.59:6815', 'nonce': 1547661179}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6812', 'nonce': 1547661179}, {'type': 'v1', 'addr': '172.21.15.59:6813', 'nonce': 1547661179}]}, 'public_addr': '172.21.15.59:6809/1547661179', 'cluster_addr': '172.21.15.59:6811/1547661179', 'heartbeat_back_addr': '172.21.15.59:6815/1547661179', 'heartbeat_front_addr': '172.21.15.59:6813/1547661179', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'cc4870b8-dcd9-4e43-978d-8de6c6396abf', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 96, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6800', 'nonce': 4054400810}, {'type': 'v1', 'addr': '172.21.15.192:6801', 'nonce': 4054400810}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6802', 'nonce': 4054400810}, {'type': 'v1', 'addr': '172.21.15.192:6803', 'nonce': 4054400810}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6806', 'nonce': 4054400810}, {'type': 'v1', 'addr': '172.21.15.192:6807', 'nonce': 4054400810}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6804', 'nonce': 4054400810}, {'type': 'v1', 'addr': '172.21.15.192:6805', 'nonce': 4054400810}]}, 'public_addr': '172.21.15.192:6801/4054400810', 'cluster_addr': '172.21.15.192:6803/4054400810', 'heartbeat_back_addr': '172.21.15.192:6807/4054400810', 'heartbeat_front_addr': '172.21.15.192:6805/4054400810', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'd1d05620-42fb-4107-a2f9-e83542d076ee', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.9892578125, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 96, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6808', 'nonce': 58985565}, {'type': 'v1', 'addr': '172.21.15.192:6809', 'nonce': 58985565}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6810', 'nonce': 58985565}, {'type': 'v1', 'addr': '172.21.15.192:6811', 'nonce': 58985565}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6814', 'nonce': 58985565}, {'type': 'v1', 'addr': '172.21.15.192:6815', 'nonce': 58985565}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6812', 'nonce': 58985565}, {'type': 'v1', 'addr': '172.21.15.192:6813', 'nonce': 58985565}]}, 'public_addr': '172.21.15.192:6809/58985565', 'cluster_addr': '172.21.15.192:6811/58985565', 'heartbeat_back_addr': '172.21.15.192:6815/58985565', 'heartbeat_front_addr': '172.21.15.192:6813/58985565', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'ab26136c-31c3-4090-9fa2-988dba4f731f', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 0.2798919677734375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 88, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6816', 'nonce': 1283571573}, {'type': 'v1', 'addr': '172.21.15.192:6817', 'nonce': 1283571573}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6818', 'nonce': 1283571573}, {'type': 'v1', 'addr': '172.21.15.192:6819', 'nonce': 1283571573}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6822', 'nonce': 1283571573}, {'type': 'v1', 'addr': '172.21.15.192:6823', 'nonce': 1283571573}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6820', 'nonce': 1283571573}, {'type': 'v1', 'addr': '172.21.15.192:6821', 'nonce': 1283571573}]}, 'public_addr': '172.21.15.192:6817/1283571573', 'cluster_addr': '172.21.15.192:6819/1283571573', 'heartbeat_back_addr': '172.21.15.192:6823/1283571573', 'heartbeat_front_addr': '172.21.15.192:6821/1283571573', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:51:52.396946+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:51:52.453160+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:51:52.387606+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:51:52.360952+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:51:52.468681+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-05-08T08:51:52.401080+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.4', 'osds': [1, 5]}, {'pgid': '3.17', 'osds': [1, 0]}, {'pgid': '3.1d', 'osds': [0, 5]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-05-08T08:54:27.082 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T08:54:27.082 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:27.082+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:27.142 INFO:teuthology.orchestra.run.smithi196.stdout:checking clone #47, image image_client.0-clone47 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent48 2024-05-08T08:54:27.180 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:27.183+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:27.238 INFO:teuthology.orchestra.run.smithi196.stdout:5978 punch from 0x1d29e00 to 0x1d38a00, (0xec00 bytes) 2024-05-08T08:54:27.239 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:27.239 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:54:27.239 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:54:27.239 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:54:27.239 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:27.250 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:54:27.250 INFO:teuthology.orchestra.run.smithi196.stdout:5981 trunc from 0x41bc200 to 0x54ab800 2024-05-08T08:54:27.260 INFO:teuthology.orchestra.run.smithi196.stdout:5983 read 0x1954800 thru 0x1957564 (0x2d65 bytes) 2024-05-08T08:54:27.278 INFO:teuthology.orchestra.run.smithi196.stdout:5987 write 0xacbb200 thru 0xacbce8a (0x1c8b bytes) 2024-05-08T08:54:27.281 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:27.284+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:27.285 INFO:teuthology.orchestra.run.smithi196.stdout:5990 read 0x31b5400 thru 0x31bd9dc (0x85dd bytes) 2024-05-08T08:54:27.285 INFO:teuthology.orchestra.run.smithi196.stdout:5991 trunc from 0xacbd000 to 0xa8d9c00 2024-05-08T08:54:27.338 INFO:teuthology.orchestra.run.smithi196.stdout:5992 write 0x2c9f000 thru 0x2c9f24e (0x24f bytes) 2024-05-08T08:54:27.339 INFO:teuthology.orchestra.run.smithi196.stdout:5993 write 0x818c600 thru 0x8195816 (0x9217 bytes) 2024-05-08T08:54:27.343 INFO:teuthology.orchestra.run.smithi196.stdout:5996 write 0x8878a00 thru 0x888243e (0x9a3f bytes) 2024-05-08T08:54:27.350 INFO:teuthology.orchestra.run.smithi196.stdout:5997 trunc from 0xa8d9c00 to 0x538e200 2024-05-08T08:54:27.382 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:27.384+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:27.422 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:27.422 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:54:27.422 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:54:27.422 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:54:27.423 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.423 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:12443 3.0 3:02d0158e:::rbd_data.114071bcf828.0000000000000037:head [stat,delete] snapc 19=[19] RETRY=3 ondisk+retry+write+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:27.423 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.610027+0000", 2024-05-08T08:54:27.423 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 55.814392533000003, 2024-05-08T08:54:27.423 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.083586333999999998, 2024-05-08T08:54:27.423 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:27.423 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:27.423 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:27.423 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:27.423 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:27.423 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:27.423 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 12443 2024-05-08T08:54:27.423 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.423 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:27.424 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.424 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:27.424 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610027+0000", 2024-05-08T08:54:27.424 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:27.424 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.424 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.424 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:27.424 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610027+0000", 2024-05-08T08:54:27.424 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:27.424 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.424 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.424 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:27.424 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610028+0000", 2024-05-08T08:54:27.424 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.13e-07 2024-05-08T08:54:27.425 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.425 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.425 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:27.425 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610030+0000", 2024-05-08T08:54:27.425 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7669999999999999e-06 2024-05-08T08:54:27.425 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.425 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.425 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:27.425 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610039+0000", 2024-05-08T08:54:27.425 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4959999999999996e-06 2024-05-08T08:54:27.425 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.425 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.425 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:27.425 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610045+0000", 2024-05-08T08:54:27.425 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6779999999999997e-06 2024-05-08T08:54:27.425 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.426 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.426 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:27.426 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.660873+0000", 2024-05-08T08:54:27.426 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.050828748 2024-05-08T08:54:27.426 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.426 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.426 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:27.426 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.661017+0000", 2024-05-08T08:54:27.426 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014391999999999999 2024-05-08T08:54:27.426 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.426 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.426 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:27.426 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.693613+0000", 2024-05-08T08:54:27.426 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.032596111999999997 2024-05-08T08:54:27.427 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.427 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:27.427 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.427 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.427 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.427 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:14301 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [watch reconnect cookie 139837155901536 gen 12] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:27.427 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.645078+0000", 2024-05-08T08:54:27.427 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 55.779341895000002, 2024-05-08T08:54:27.427 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088281852999999993, 2024-05-08T08:54:27.427 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:27.427 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:27.427 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:27.427 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:27.427 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:27.427 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:27.427 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14301 2024-05-08T08:54:27.428 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.428 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:27.428 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.428 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:27.428 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645078+0000", 2024-05-08T08:54:27.428 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:27.428 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.428 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.428 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:27.428 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645078+0000", 2024-05-08T08:54:27.428 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:27.428 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.428 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.428 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:27.429 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645084+0000", 2024-05-08T08:54:27.429 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4620000000000001e-06 2024-05-08T08:54:27.429 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.429 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.429 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:27.429 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645088+0000", 2024-05-08T08:54:27.429 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8039999999999999e-06 2024-05-08T08:54:27.429 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.429 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.429 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:27.429 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645106+0000", 2024-05-08T08:54:27.429 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.8318999999999999e-05 2024-05-08T08:54:27.429 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.429 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.429 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:27.430 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645113+0000", 2024-05-08T08:54:27.430 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0310000000000001e-06 2024-05-08T08:54:27.430 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.430 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.430 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:27.430 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.670493+0000", 2024-05-08T08:54:27.430 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.025379993 2024-05-08T08:54:27.430 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.430 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.430 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:27.430 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.670571+0000", 2024-05-08T08:54:27.430 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7613000000000004e-05 2024-05-08T08:54:27.430 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.430 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.430 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:27.431 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733360+0000", 2024-05-08T08:54:27.431 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.062788630999999998 2024-05-08T08:54:27.431 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.431 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:27.431 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.431 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.431 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.431 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:14420 3.0 3:06a8948e:::rbd_data.114071bcf828.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:27.431 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.649639+0000", 2024-05-08T08:54:27.431 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 55.774780747000001, 2024-05-08T08:54:27.431 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.083851897999999994, 2024-05-08T08:54:27.431 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:27.431 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:27.431 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:54:27.432 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:27.432 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:27.432 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:27.432 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14420 2024-05-08T08:54:27.432 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.432 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:27.432 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.432 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:27.432 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649639+0000", 2024-05-08T08:54:27.432 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:27.432 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.432 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.432 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:27.433 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649639+0000", 2024-05-08T08:54:27.433 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:27.433 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.433 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.433 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:27.433 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649640+0000", 2024-05-08T08:54:27.433 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6899999999999997e-07 2024-05-08T08:54:27.433 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.433 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.433 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:27.433 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649895+0000", 2024-05-08T08:54:27.433 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00025531000000000002 2024-05-08T08:54:27.433 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.434 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.434 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:27.434 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649905+0000", 2024-05-08T08:54:27.434 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0198e-05 2024-05-08T08:54:27.434 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.434 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.434 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:27.434 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649911+0000", 2024-05-08T08:54:27.434 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2310000000000001e-06 2024-05-08T08:54:27.434 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.434 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.434 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:27.435 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.684099+0000", 2024-05-08T08:54:27.435 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.034188017000000001 2024-05-08T08:54:27.435 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.435 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.435 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:27.435 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.684172+0000", 2024-05-08T08:54:27.435 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3201999999999995e-05 2024-05-08T08:54:27.435 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.435 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.435 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 2", 2024-05-08T08:54:27.435 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.684418+0000", 2024-05-08T08:54:27.436 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00024559099999999997 2024-05-08T08:54:27.436 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.436 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.436 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:54:27.436 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.705538+0000", 2024-05-08T08:54:27.436 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.021119707000000001 2024-05-08T08:54:27.436 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.436 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.436 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:54:27.436 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733399+0000", 2024-05-08T08:54:27.436 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.027861543999999999 2024-05-08T08:54:27.436 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.436 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.437 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:54:27.437 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733445+0000", 2024-05-08T08:54:27.437 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5587000000000002e-05 2024-05-08T08:54:27.437 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.437 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.437 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:27.437 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733491+0000", 2024-05-08T08:54:27.437 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5841999999999999e-05 2024-05-08T08:54:27.437 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.437 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:27.437 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.437 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.437 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.438 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28547 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889179216] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:27.438 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.355654+0000", 2024-05-08T08:54:27.438 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 55.068765519000003, 2024-05-08T08:54:27.438 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078692261999999999, 2024-05-08T08:54:27.438 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:27.438 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:27.438 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:27.438 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:27.438 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:27.438 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:27.438 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28547 2024-05-08T08:54:27.438 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.438 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:27.439 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.439 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:27.439 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355654+0000", 2024-05-08T08:54:27.439 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:27.439 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.439 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.439 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:27.439 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355654+0000", 2024-05-08T08:54:27.439 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:27.439 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.439 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.439 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:27.439 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355654+0000", 2024-05-08T08:54:27.440 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3999999999999998e-07 2024-05-08T08:54:27.440 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.440 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.440 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:27.440 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355656+0000", 2024-05-08T08:54:27.440 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.353e-06 2024-05-08T08:54:27.440 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.440 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.440 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:27.440 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355663+0000", 2024-05-08T08:54:27.440 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2259999999999999e-06 2024-05-08T08:54:27.440 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.440 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.440 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:27.441 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355666+0000", 2024-05-08T08:54:27.441 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7930000000000002e-06 2024-05-08T08:54:27.441 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.441 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.441 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:27.441 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434117+0000", 2024-05-08T08:54:27.441 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078451231999999996 2024-05-08T08:54:27.441 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.441 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.441 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:27.441 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434198+0000", 2024-05-08T08:54:27.441 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0879999999999998e-05 2024-05-08T08:54:27.441 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.441 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.442 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:27.442 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434346+0000", 2024-05-08T08:54:27.442 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014853800000000001 2024-05-08T08:54:27.442 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.442 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:27.442 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.442 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.442 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.442 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28548 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889189696] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:27.442 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.355669+0000", 2024-05-08T08:54:27.442 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 55.068750168999998, 2024-05-08T08:54:27.442 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078956606999999998, 2024-05-08T08:54:27.442 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:27.442 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:27.443 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:27.443 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:27.443 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:27.443 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:27.443 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28548 2024-05-08T08:54:27.443 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.443 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:27.443 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.443 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:27.443 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355669+0000", 2024-05-08T08:54:27.443 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:27.443 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.445 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.445 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:27.445 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355669+0000", 2024-05-08T08:54:27.445 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:27.445 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.445 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.445 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:27.445 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355762+0000", 2024-05-08T08:54:27.445 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3015000000000003e-05 2024-05-08T08:54:27.445 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.446 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.446 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:27.446 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355765+0000", 2024-05-08T08:54:27.446 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.368e-06 2024-05-08T08:54:27.446 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.446 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.446 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:27.446 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355771+0000", 2024-05-08T08:54:27.446 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5989999999999998e-06 2024-05-08T08:54:27.446 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.446 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.446 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:27.446 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355775+0000", 2024-05-08T08:54:27.447 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7280000000000001e-06 2024-05-08T08:54:27.447 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.447 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.447 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:27.447 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434379+0000", 2024-05-08T08:54:27.447 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078603662000000005 2024-05-08T08:54:27.447 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.447 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.447 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:27.447 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434473+0000", 2024-05-08T08:54:27.447 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4104000000000006e-05 2024-05-08T08:54:27.447 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.447 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.447 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:27.447 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434626+0000", 2024-05-08T08:54:27.447 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00015313100000000001 2024-05-08T08:54:27.448 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.448 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:27.448 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.448 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.448 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.448 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28549 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889202400] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:27.448 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.355781+0000", 2024-05-08T08:54:27.448 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 55.068638595000003, 2024-05-08T08:54:27.448 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079117946999999994, 2024-05-08T08:54:27.448 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:27.448 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:27.448 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:27.448 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:27.448 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:27.448 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:27.448 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28549 2024-05-08T08:54:27.448 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.449 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:27.449 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.449 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:27.449 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355781+0000", 2024-05-08T08:54:27.449 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:27.449 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.449 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.449 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:27.449 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355781+0000", 2024-05-08T08:54:27.449 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:27.449 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.449 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.449 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:27.449 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356162+0000", 2024-05-08T08:54:27.449 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00038099100000000001 2024-05-08T08:54:27.449 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.450 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.450 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:27.450 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356165+0000", 2024-05-08T08:54:27.450 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1240000000000001e-06 2024-05-08T08:54:27.450 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.450 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.450 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:27.450 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356173+0000", 2024-05-08T08:54:27.450 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.6199999999999999e-06 2024-05-08T08:54:27.450 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.450 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.450 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:27.450 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356182+0000", 2024-05-08T08:54:27.450 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7760000000000003e-06 2024-05-08T08:54:27.450 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.451 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.451 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:27.451 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434675+0000", 2024-05-08T08:54:27.451 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078493119 2024-05-08T08:54:27.451 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.451 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.451 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:27.451 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434765+0000", 2024-05-08T08:54:27.451 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0402000000000006e-05 2024-05-08T08:54:27.451 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.451 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.451 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:27.451 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434899+0000", 2024-05-08T08:54:27.451 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013391499999999999 2024-05-08T08:54:27.452 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.452 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:27.452 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.452 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.452 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.452 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28550 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888597280] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:27.452 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:54:27.452 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 55.068227548999999, 2024-05-08T08:54:27.452 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078933990999999995, 2024-05-08T08:54:27.452 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:27.452 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:27.452 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:27.452 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:27.452 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:27.452 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:27.452 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28550 2024-05-08T08:54:27.453 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.453 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:27.453 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.453 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:27.453 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:54:27.453 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:27.453 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.453 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.453 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:27.453 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:54:27.453 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:27.453 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.453 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.453 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:27.453 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:54:27.453 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.34e-07 2024-05-08T08:54:27.454 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.454 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.454 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:27.454 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356195+0000", 2024-05-08T08:54:27.454 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8710000000000001e-06 2024-05-08T08:54:27.454 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.454 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.454 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:27.454 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356200+0000", 2024-05-08T08:54:27.454 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3000000000000003e-06 2024-05-08T08:54:27.454 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.454 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.454 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:27.454 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356203+0000", 2024-05-08T08:54:27.454 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4039999999999999e-06 2024-05-08T08:54:27.455 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.455 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.455 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:27.455 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434928+0000", 2024-05-08T08:54:27.455 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078725331999999995 2024-05-08T08:54:27.455 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.455 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.455 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:27.455 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435007+0000", 2024-05-08T08:54:27.455 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8415000000000001e-05 2024-05-08T08:54:27.455 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.455 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.455 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:27.455 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435126+0000", 2024-05-08T08:54:27.455 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011923500000000001 2024-05-08T08:54:27.455 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.456 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:27.456 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.456 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.456 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.456 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28551 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888527200] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:27.456 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.356210+0000", 2024-05-08T08:54:27.456 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 55.068209201000002, 2024-05-08T08:54:27.456 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079195072000000005, 2024-05-08T08:54:27.456 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:27.456 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:27.456 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:27.456 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:27.456 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:27.456 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:27.456 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28551 2024-05-08T08:54:27.456 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.456 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:27.457 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.457 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:27.457 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356210+0000", 2024-05-08T08:54:27.457 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:27.457 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.457 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.457 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:27.457 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356210+0000", 2024-05-08T08:54:27.457 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:27.457 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.457 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.457 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:27.457 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356211+0000", 2024-05-08T08:54:27.457 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4499999999999998e-07 2024-05-08T08:54:27.457 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.457 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.458 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:27.458 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356212+0000", 2024-05-08T08:54:27.458 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6029999999999999e-06 2024-05-08T08:54:27.458 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.458 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.458 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:27.458 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356221+0000", 2024-05-08T08:54:27.458 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4200000000000007e-06 2024-05-08T08:54:27.458 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.458 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.458 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:27.458 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356224+0000", 2024-05-08T08:54:27.458 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.038e-06 2024-05-08T08:54:27.458 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.459 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.459 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:27.459 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435178+0000", 2024-05-08T08:54:27.459 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078954625000000001 2024-05-08T08:54:27.459 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.459 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.459 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:27.459 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435256+0000", 2024-05-08T08:54:27.459 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7337999999999997e-05 2024-05-08T08:54:27.459 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.459 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.459 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:27.459 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435405+0000", 2024-05-08T08:54:27.460 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000149703 2024-05-08T08:54:27.460 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.460 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:27.460 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.460 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.460 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.460 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28552 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889231456] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:27.460 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.356228+0000", 2024-05-08T08:54:27.460 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 55.068191523000003, 2024-05-08T08:54:27.460 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079346920000000001, 2024-05-08T08:54:27.460 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:27.460 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:27.461 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:27.461 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:27.461 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:27.461 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:27.461 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28552 2024-05-08T08:54:27.461 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.461 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:27.461 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.461 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:27.461 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356228+0000", 2024-05-08T08:54:27.461 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:27.461 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.461 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.461 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:27.461 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356228+0000", 2024-05-08T08:54:27.461 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:27.461 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.462 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.462 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:27.462 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356959+0000", 2024-05-08T08:54:27.462 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00073097099999999996 2024-05-08T08:54:27.462 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.462 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.462 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:27.462 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356967+0000", 2024-05-08T08:54:27.462 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0409999999999998e-06 2024-05-08T08:54:27.462 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.462 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.462 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:27.462 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356979+0000", 2024-05-08T08:54:27.463 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1489e-05 2024-05-08T08:54:27.463 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.463 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.463 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:27.463 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356986+0000", 2024-05-08T08:54:27.463 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3699999999999997e-06 2024-05-08T08:54:27.463 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.463 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.463 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:27.463 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435433+0000", 2024-05-08T08:54:27.463 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078447520000000007 2024-05-08T08:54:27.463 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.463 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.463 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:27.463 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435484+0000", 2024-05-08T08:54:27.463 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0229000000000003e-05 2024-05-08T08:54:27.463 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.464 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.464 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:27.464 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435575+0000", 2024-05-08T08:54:27.464 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1299999999999997e-05 2024-05-08T08:54:27.464 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.464 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:27.464 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.464 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.464 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.464 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28553 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888713648] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:27.464 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:54:27.464 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 55.067416864999998, 2024-05-08T08:54:27.464 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078782034000000001, 2024-05-08T08:54:27.464 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:27.464 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:27.465 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:27.465 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:27.465 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:27.465 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:27.465 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28553 2024-05-08T08:54:27.465 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.465 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:27.465 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.465 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:27.465 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:54:27.465 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:27.465 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.465 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.465 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:27.465 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:54:27.466 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:27.466 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.466 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.466 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:27.466 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:54:27.466 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6600000000000002e-07 2024-05-08T08:54:27.466 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.466 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.466 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:27.466 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357006+0000", 2024-05-08T08:54:27.466 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7190000000000001e-06 2024-05-08T08:54:27.466 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.466 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.466 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:27.466 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357013+0000", 2024-05-08T08:54:27.467 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.435e-06 2024-05-08T08:54:27.467 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.467 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.467 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:27.467 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357018+0000", 2024-05-08T08:54:27.467 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7260000000000002e-06 2024-05-08T08:54:27.467 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.467 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.467 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:27.467 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435623+0000", 2024-05-08T08:54:27.467 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078604972999999995 2024-05-08T08:54:27.467 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.467 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.467 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:27.467 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435695+0000", 2024-05-08T08:54:27.468 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1959999999999995e-05 2024-05-08T08:54:27.468 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.468 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.468 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:27.468 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435785+0000", 2024-05-08T08:54:27.468 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.9754999999999997e-05 2024-05-08T08:54:27.468 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.468 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:27.468 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.468 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.468 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.468 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28554 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888980272] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:27.468 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357024+0000", 2024-05-08T08:54:27.468 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 55.067395228999999, 2024-05-08T08:54:27.468 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078999870999999999, 2024-05-08T08:54:27.468 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:27.468 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:27.469 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:27.469 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:27.469 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:27.469 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:27.469 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28554 2024-05-08T08:54:27.469 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.469 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:27.469 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.469 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:27.469 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357024+0000", 2024-05-08T08:54:27.469 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:27.469 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.469 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.469 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:27.469 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357024+0000", 2024-05-08T08:54:27.469 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:27.470 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.470 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.470 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:27.470 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357025+0000", 2024-05-08T08:54:27.470 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:54:27.470 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.470 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.470 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:27.470 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357027+0000", 2024-05-08T08:54:27.470 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2050000000000001e-06 2024-05-08T08:54:27.470 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.470 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.470 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:27.470 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357033+0000", 2024-05-08T08:54:27.471 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5679999999999996e-06 2024-05-08T08:54:27.471 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.471 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.471 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:27.471 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357038+0000", 2024-05-08T08:54:27.471 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2889999999999999e-06 2024-05-08T08:54:27.471 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.471 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.471 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:27.471 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435823+0000", 2024-05-08T08:54:27.471 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078785677999999998 2024-05-08T08:54:27.471 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.471 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.471 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:27.471 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435892+0000", 2024-05-08T08:54:27.471 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8379000000000004e-05 2024-05-08T08:54:27.472 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.472 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.472 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:27.472 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436024+0000", 2024-05-08T08:54:27.472 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013239799999999999 2024-05-08T08:54:27.472 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.472 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:27.472 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.472 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.472 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.472 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28555 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889001104] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:27.472 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:54:27.472 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 55.067376463999999, 2024-05-08T08:54:27.472 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079115742000000003, 2024-05-08T08:54:27.472 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:27.472 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:27.473 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:27.473 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:27.473 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:27.473 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:27.473 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28555 2024-05-08T08:54:27.473 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.473 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:27.473 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.473 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:27.473 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:54:27.473 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:27.473 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.473 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.473 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:27.473 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:54:27.473 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:27.474 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.474 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.474 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:27.474 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:54:27.474 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9200000000000002e-07 2024-05-08T08:54:27.474 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.474 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.474 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:27.474 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357046+0000", 2024-05-08T08:54:27.474 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1670000000000002e-06 2024-05-08T08:54:27.474 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.474 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.474 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:27.474 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357052+0000", 2024-05-08T08:54:27.475 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.139e-06 2024-05-08T08:54:27.475 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.475 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.475 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:27.475 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357056+0000", 2024-05-08T08:54:27.475 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2440000000000004e-06 2024-05-08T08:54:27.475 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.475 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.475 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:27.475 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436059+0000", 2024-05-08T08:54:27.476 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079003070999999994 2024-05-08T08:54:27.476 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.476 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.476 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:27.476 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436097+0000", 2024-05-08T08:54:27.476 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7784000000000003e-05 2024-05-08T08:54:27.476 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.476 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.476 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:27.476 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436159+0000", 2024-05-08T08:54:27.476 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2045000000000007e-05 2024-05-08T08:54:27.476 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.476 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:27.476 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.477 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.477 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.477 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28556 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889022064] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:27.477 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357065+0000", 2024-05-08T08:54:27.477 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 55.067354430999998, 2024-05-08T08:54:27.477 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079327976999999994, 2024-05-08T08:54:27.477 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:27.477 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:27.477 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:27.477 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:27.477 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:27.477 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:27.477 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28556 2024-05-08T08:54:27.477 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.477 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:27.477 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.478 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:27.478 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357065+0000", 2024-05-08T08:54:27.478 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:27.478 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.478 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.478 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:27.478 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357065+0000", 2024-05-08T08:54:27.478 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:27.478 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.478 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.478 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:27.478 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358958+0000", 2024-05-08T08:54:27.478 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001892915 2024-05-08T08:54:27.478 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.478 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.479 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:27.479 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358962+0000", 2024-05-08T08:54:27.479 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4599999999999999e-06 2024-05-08T08:54:27.479 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.479 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.479 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:27.479 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358970+0000", 2024-05-08T08:54:27.479 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.6090000000000001e-06 2024-05-08T08:54:27.479 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.479 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.479 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:27.479 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358975+0000", 2024-05-08T08:54:27.479 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7639999999999996e-06 2024-05-08T08:54:27.479 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.479 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.480 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:27.480 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436196+0000", 2024-05-08T08:54:27.480 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.077221564000000006 2024-05-08T08:54:27.480 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.480 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.480 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:27.480 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436269+0000", 2024-05-08T08:54:27.480 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2720999999999997e-05 2024-05-08T08:54:27.480 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.480 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.480 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:27.480 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436393+0000", 2024-05-08T08:54:27.480 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012394399999999999 2024-05-08T08:54:27.480 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.480 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:27.481 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.481 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.481 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.481 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28563 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888855520] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:27.481 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:54:27.481 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 55.065345032000003, 2024-05-08T08:54:27.481 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078795566999999997, 2024-05-08T08:54:27.481 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:27.481 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:27.481 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:27.481 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:27.481 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:27.481 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:27.481 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28563 2024-05-08T08:54:27.481 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.482 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:27.482 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.482 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:27.482 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:54:27.482 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:27.482 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.482 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.482 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:27.482 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:54:27.482 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:27.482 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.482 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.482 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:27.483 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:54:27.483 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1e-07 2024-05-08T08:54:27.483 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.483 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.483 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:27.483 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359080+0000", 2024-05-08T08:54:27.483 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8940000000000001e-06 2024-05-08T08:54:27.483 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.483 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.483 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:27.483 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359083+0000", 2024-05-08T08:54:27.483 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1949999999999999e-06 2024-05-08T08:54:27.483 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.483 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.483 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:27.483 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359085+0000", 2024-05-08T08:54:27.484 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4710000000000001e-06 2024-05-08T08:54:27.484 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.484 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.484 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:27.484 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437604+0000", 2024-05-08T08:54:27.484 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078518278999999996 2024-05-08T08:54:27.484 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.484 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.484 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:27.484 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437704+0000", 2024-05-08T08:54:27.484 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010010599999999999 2024-05-08T08:54:27.484 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.484 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.484 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:27.484 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437870+0000", 2024-05-08T08:54:27.484 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00016641200000000001 2024-05-08T08:54:27.485 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.485 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:27.485 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.485 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.485 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.485 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28564 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888538944] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:27.485 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.359089+0000", 2024-05-08T08:54:27.485 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 55.065330817000003, 2024-05-08T08:54:27.485 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078952675, 2024-05-08T08:54:27.485 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:27.485 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:27.485 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:27.485 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:27.485 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:27.485 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:27.486 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28564 2024-05-08T08:54:27.486 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.486 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:27.486 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.486 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:27.486 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359089+0000", 2024-05-08T08:54:27.486 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:27.486 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.486 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.486 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:27.486 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359089+0000", 2024-05-08T08:54:27.486 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:27.486 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.486 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.487 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:27.487 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360959+0000", 2024-05-08T08:54:27.487 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0018707089999999999 2024-05-08T08:54:27.487 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.487 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.487 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:27.487 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360963+0000", 2024-05-08T08:54:27.487 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0680000000000001e-06 2024-05-08T08:54:27.487 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.487 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.487 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:27.487 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360969+0000", 2024-05-08T08:54:27.487 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1920000000000002e-06 2024-05-08T08:54:27.487 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.487 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.488 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:27.488 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360973+0000", 2024-05-08T08:54:27.488 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.031e-06 2024-05-08T08:54:27.488 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.488 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.488 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:27.488 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437916+0000", 2024-05-08T08:54:27.488 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.076942939000000002 2024-05-08T08:54:27.488 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.488 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.488 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:27.488 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437973+0000", 2024-05-08T08:54:27.488 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7358999999999997e-05 2024-05-08T08:54:27.488 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.488 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.489 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:27.489 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.438041+0000", 2024-05-08T08:54:27.489 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8376999999999999e-05 2024-05-08T08:54:27.489 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.489 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:27.489 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.489 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.489 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.489 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28574 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889427600] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:27.489 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:54:27.489 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 55.063326310000001, 2024-05-08T08:54:27.489 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078782476000000004, 2024-05-08T08:54:27.489 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:27.489 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:27.489 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:27.490 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:27.490 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:27.490 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:27.490 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28574 2024-05-08T08:54:27.490 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.490 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:27.490 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.490 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:27.490 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:54:27.490 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:27.490 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.490 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.490 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:27.490 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:54:27.491 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:27.491 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.491 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.491 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:27.491 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:54:27.491 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1799999999999999e-07 2024-05-08T08:54:27.491 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.491 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.491 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:27.491 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361095+0000", 2024-05-08T08:54:27.491 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3850000000000001e-06 2024-05-08T08:54:27.491 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.491 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.491 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:27.491 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361098+0000", 2024-05-08T08:54:27.492 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7989999999999998e-06 2024-05-08T08:54:27.492 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.492 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.492 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:27.492 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361100+0000", 2024-05-08T08:54:27.492 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4669999999999998e-06 2024-05-08T08:54:27.492 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.492 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.492 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:27.492 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439768+0000", 2024-05-08T08:54:27.492 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078668057999999999 2024-05-08T08:54:27.492 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.492 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.492 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:27.493 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439811+0000", 2024-05-08T08:54:27.493 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2459000000000001e-05 2024-05-08T08:54:27.493 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.493 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.493 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:27.493 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439876+0000", 2024-05-08T08:54:27.493 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5090000000000002e-05 2024-05-08T08:54:27.493 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.493 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:27.493 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.493 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.493 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.493 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28575 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889439216] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:27.493 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361103+0000", 2024-05-08T08:54:27.494 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 55.063316295, 2024-05-08T08:54:27.494 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078966828000000003, 2024-05-08T08:54:27.494 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:27.494 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:27.494 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:27.494 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:27.494 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:27.494 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:27.494 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28575 2024-05-08T08:54:27.494 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.494 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:27.494 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.494 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:27.494 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361103+0000", 2024-05-08T08:54:27.495 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:27.495 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.495 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.495 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:27.495 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361103+0000", 2024-05-08T08:54:27.495 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:27.495 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.495 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.495 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:27.495 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361104+0000", 2024-05-08T08:54:27.495 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1400000000000001e-07 2024-05-08T08:54:27.495 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.495 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.495 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:27.495 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361105+0000", 2024-05-08T08:54:27.496 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5540000000000001e-06 2024-05-08T08:54:27.496 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.496 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.496 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:27.496 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361108+0000", 2024-05-08T08:54:27.496 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.841e-06 2024-05-08T08:54:27.496 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.496 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.496 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:27.496 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361110+0000", 2024-05-08T08:54:27.496 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.497e-06 2024-05-08T08:54:27.496 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.496 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.496 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:27.496 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439916+0000", 2024-05-08T08:54:27.496 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078805372999999998 2024-05-08T08:54:27.497 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.497 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.497 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:27.497 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440000+0000", 2024-05-08T08:54:27.497 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4730999999999999e-05 2024-05-08T08:54:27.497 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.497 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.497 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:27.497 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440070+0000", 2024-05-08T08:54:27.497 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9617999999999997e-05 2024-05-08T08:54:27.497 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.497 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:27.497 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.497 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.497 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.498 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28576 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889451328] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:27.498 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361113+0000", 2024-05-08T08:54:27.498 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 55.063306109000003, 2024-05-08T08:54:27.498 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079088862999999995, 2024-05-08T08:54:27.498 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:27.498 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:27.498 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:27.498 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:27.498 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:27.498 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:27.498 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28576 2024-05-08T08:54:27.498 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.498 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:27.498 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.499 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:27.499 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361113+0000", 2024-05-08T08:54:27.499 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:27.499 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.499 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.499 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:27.499 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361113+0000", 2024-05-08T08:54:27.499 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:27.499 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.499 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.499 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:27.499 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361114+0000", 2024-05-08T08:54:27.499 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1299999999999999e-07 2024-05-08T08:54:27.499 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.499 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.500 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:27.500 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361115+0000", 2024-05-08T08:54:27.500 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.4300000000000001e-06 2024-05-08T08:54:27.500 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.500 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.500 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:27.500 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361118+0000", 2024-05-08T08:54:27.500 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9000000000000002e-06 2024-05-08T08:54:27.500 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.500 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.500 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:27.500 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361120+0000", 2024-05-08T08:54:27.500 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4739999999999999e-06 2024-05-08T08:54:27.500 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.501 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.501 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:27.501 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440096+0000", 2024-05-08T08:54:27.501 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078975629000000006 2024-05-08T08:54:27.501 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.501 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.501 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:27.501 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440135+0000", 2024-05-08T08:54:27.501 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9202000000000002e-05 2024-05-08T08:54:27.501 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.501 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.501 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:27.501 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440202+0000", 2024-05-08T08:54:27.501 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7014999999999994e-05 2024-05-08T08:54:27.501 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.502 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:27.502 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.502 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.502 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.502 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28577 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889462640] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:27.502 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:54:27.502 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 55.063295926999999, 2024-05-08T08:54:27.502 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079194622000000006, 2024-05-08T08:54:27.502 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:27.502 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:27.502 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:27.502 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:27.502 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:27.502 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:27.503 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28577 2024-05-08T08:54:27.503 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.503 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:27.503 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.503 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:27.503 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:54:27.503 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:27.503 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.503 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.503 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:27.503 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:54:27.503 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:27.503 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.503 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.504 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:27.504 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:54:27.504 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1400000000000001e-07 2024-05-08T08:54:27.504 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.504 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.504 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:27.504 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361125+0000", 2024-05-08T08:54:27.504 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.372e-06 2024-05-08T08:54:27.504 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.504 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.504 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:27.504 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361128+0000", 2024-05-08T08:54:27.504 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0309999999999999e-06 2024-05-08T08:54:27.504 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.505 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.505 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:27.505 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361131+0000", 2024-05-08T08:54:27.505 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4040000000000002e-06 2024-05-08T08:54:27.505 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.505 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.505 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:27.505 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440225+0000", 2024-05-08T08:54:27.505 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079094600000000001 2024-05-08T08:54:27.505 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.505 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.505 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:27.505 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440260+0000", 2024-05-08T08:54:27.505 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4703000000000003e-05 2024-05-08T08:54:27.505 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.506 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.506 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:27.506 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440318+0000", 2024-05-08T08:54:27.506 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8298000000000003e-05 2024-05-08T08:54:27.506 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.506 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:27.506 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.506 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.506 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.506 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28578 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889474192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:27.506 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361134+0000", 2024-05-08T08:54:27.506 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 55.063285501999999, 2024-05-08T08:54:27.506 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079310295000000003, 2024-05-08T08:54:27.506 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:27.507 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:27.507 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:27.507 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:27.507 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:27.507 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:27.507 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28578 2024-05-08T08:54:27.507 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.507 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:27.507 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.507 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:27.507 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361134+0000", 2024-05-08T08:54:27.507 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:27.507 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.507 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.507 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:27.507 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361134+0000", 2024-05-08T08:54:27.508 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:27.508 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.508 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.508 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:27.508 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.362958+0000", 2024-05-08T08:54:27.508 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0018237380000000001 2024-05-08T08:54:27.508 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.508 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.508 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:27.508 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.362961+0000", 2024-05-08T08:54:27.508 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.123e-06 2024-05-08T08:54:27.508 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.508 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.508 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:27.508 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.397404+0000", 2024-05-08T08:54:27.509 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.034443226 2024-05-08T08:54:27.509 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.509 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.509 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:27.509 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.397439+0000", 2024-05-08T08:54:27.509 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4690000000000002e-05 2024-05-08T08:54:27.509 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.509 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.509 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:27.509 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440340+0000", 2024-05-08T08:54:27.509 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.042901419000000003 2024-05-08T08:54:27.509 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.509 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.509 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:27.509 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440376+0000", 2024-05-08T08:54:27.510 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5444999999999997e-05 2024-05-08T08:54:27.510 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.510 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.510 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:27.510 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440444+0000", 2024-05-08T08:54:27.510 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8653999999999997e-05 2024-05-08T08:54:27.510 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.510 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:27.510 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.510 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.510 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:27.510 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:27.512 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:54:27.513 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:54:27.514 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-05-08T08:54:27.515 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":184,"stamp":"2024-05-08T08:54:26.761176+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1473289280,"num_objects":3039,"num_object_clones":455,"num_object_copies":6078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3039,"num_whiteouts":1639,"num_read":480227,"num_read_kb":1541162,"num_write":21992,"num_write_kb":438118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2942,"num_bytes_recovered":182360369,"num_keys_recovered":458,"num_objects_omap":48,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19787,"ondisk_log_size":19787,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":471859200,"kb_used":379308,"kb_used_data":85924,"kb_used_omap":7,"kb_used_meta":293240,"kb_avail":471479892,"statfs":{"total":483183820800,"available":482795409408,"internally_reserved":0,"allocated":87986176,"data_stored":669039312,"data_compressed":36468417,"data_compressed_allocated":66207744,"data_compressed_original":657014272,"omap_allocated":7973,"internal_metadata":300277979},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":169075963,"num_objects":59,"num_object_clones":17,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":6,"num_read":9682,"num_read_kb":66012,"num_write":492,"num_write_kb":7419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.997035"},"pg_stats":[{"pgid":"3.1f","version":"111'637","reported_seq":19641,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T08:54:22.753288+0000","last_change":"2024-05-08T08:53:49.581255+0000","last_active":"2024-05-08T08:54:22.753288+0000","last_peered":"2024-05-08T08:54:22.753288+0000","last_clean":"2024-05-08T08:54:22.753288+0000","last_became_active":"2024-05-08T08:53:49.580563+0000","last_became_peered":"2024-05-08T08:53:49.580563+0000","last_unstale":"2024-05-08T08:54:22.753288+0000","last_undegraded":"2024-05-08T08:54:22.753288+0000","last_fullsized":"2024-05-08T08:54:22.753288+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":637,"log_dups_size":0,"ondisk_log_size":637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:57:47.374332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000691565,"stat_sum":{"num_bytes":72880487,"num_objects":97,"num_object_clones":17,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":44,"num_read":18835,"num_read_kb":84946,"num_write":886,"num_write_kb":33068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"109'790","reported_seq":11156,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T08:54:22.755288+0000","last_change":"2024-05-08T08:52:13.944947+0000","last_active":"2024-05-08T08:54:22.755288+0000","last_peered":"2024-05-08T08:54:22.755288+0000","last_clean":"2024-05-08T08:54:22.755288+0000","last_became_active":"2024-05-08T08:52:11.930844+0000","last_became_peered":"2024-05-08T08:52:11.930844+0000","last_unstale":"2024-05-08T08:54:22.755288+0000","last_undegraded":"2024-05-08T08:54:22.755288+0000","last_fullsized":"2024-05-08T08:54:22.755288+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":790,"log_dups_size":0,"ondisk_log_size":790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:27:11.168615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040424900000000003,"stat_sum":{"num_bytes":20532352,"num_objects":94,"num_object_clones":12,"num_object_copies":188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":60,"num_read":13076,"num_read_kb":36578,"num_write":602,"num_write_kb":12173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"111'570","reported_seq":3216,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T08:54:22.747744+0000","last_change":"2024-05-08T08:53:59.618675+0000","last_active":"2024-05-08T08:54:22.747744+0000","last_peered":"2024-05-08T08:54:22.747744+0000","last_clean":"2024-05-08T08:54:22.747744+0000","last_became_active":"2024-05-08T08:53:59.618085+0000","last_became_peered":"2024-05-08T08:53:59.618085+0000","last_unstale":"2024-05-08T08:54:22.747744+0000","last_undegraded":"2024-05-08T08:54:22.747744+0000","last_fullsized":"2024-05-08T08:54:22.747744+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":570,"log_dups_size":0,"ondisk_log_size":570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:35:38.387281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072111799999999998,"stat_sum":{"num_bytes":33258270,"num_objects":90,"num_object_clones":12,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":46,"num_read":7013,"num_read_kb":29360,"num_write":769,"num_write_kb":7129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":141,"num_bytes_recovered":5594929,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"111'547","reported_seq":17826,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T08:54:22.747794+0000","last_change":"2024-05-08T08:53:58.622642+0000","last_active":"2024-05-08T08:54:22.747794+0000","last_peered":"2024-05-08T08:54:22.747794+0000","last_clean":"2024-05-08T08:54:22.747794+0000","last_became_active":"2024-05-08T08:53:58.621483+0000","last_became_peered":"2024-05-08T08:53:58.621483+0000","last_unstale":"2024-05-08T08:54:22.747794+0000","last_undegraded":"2024-05-08T08:54:22.747794+0000","last_fullsized":"2024-05-08T08:54:22.747794+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":547,"log_dups_size":0,"ondisk_log_size":547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:17:57.894022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094844199999999997,"stat_sum":{"num_bytes":69379637,"num_objects":104,"num_object_clones":13,"num_object_copies":208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":54,"num_read":17742,"num_read_kb":100687,"num_write":602,"num_write_kb":27789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":287,"num_bytes_recovered":49590415,"num_keys_recovered":166,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"108'527","reported_seq":24855,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T08:54:22.747681+0000","last_change":"2024-05-08T08:53:49.585842+0000","last_active":"2024-05-08T08:54:22.747681+0000","last_peered":"2024-05-08T08:54:22.747681+0000","last_clean":"2024-05-08T08:54:22.747681+0000","last_became_active":"2024-05-08T08:53:49.583097+0000","last_became_peered":"2024-05-08T08:53:49.583097+0000","last_unstale":"2024-05-08T08:54:22.747681+0000","last_undegraded":"2024-05-08T08:54:22.747681+0000","last_fullsized":"2024-05-08T08:54:22.747681+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":527,"log_dups_size":0,"ondisk_log_size":527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:58:31.314038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000685079,"stat_sum":{"num_bytes":51853626,"num_objects":103,"num_object_clones":13,"num_object_copies":206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103,"num_whiteouts":62,"num_read":24621,"num_read_kb":43190,"num_write":605,"num_write_kb":22765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":129,"num_bytes_recovered":3090520,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"111'679","reported_seq":20649,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T08:54:22.764029+0000","last_change":"2024-05-08T08:53:24.485198+0000","last_active":"2024-05-08T08:54:22.764029+0000","last_peered":"2024-05-08T08:54:22.764029+0000","last_clean":"2024-05-08T08:54:22.764029+0000","last_became_active":"2024-05-08T08:53:24.484124+0000","last_became_peered":"2024-05-08T08:53:24.484124+0000","last_unstale":"2024-05-08T08:54:22.764029+0000","last_undegraded":"2024-05-08T08:54:22.764029+0000","last_fullsized":"2024-05-08T08:54:22.764029+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":679,"log_dups_size":0,"ondisk_log_size":679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:47:55.984203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066525600000000003,"stat_sum":{"num_bytes":28843438,"num_objects":99,"num_object_clones":16,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":47,"num_read":23069,"num_read_kb":58145,"num_write":772,"num_write_kb":18926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":130,"num_bytes_recovered":8549042,"num_keys_recovered":12,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"110'348","reported_seq":9833,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T08:54:22.748550+0000","last_change":"2024-05-08T08:53:24.484713+0000","last_active":"2024-05-08T08:54:22.748550+0000","last_peered":"2024-05-08T08:54:22.748550+0000","last_clean":"2024-05-08T08:54:22.748550+0000","last_became_active":"2024-05-08T08:53:24.483403+0000","last_became_peered":"2024-05-08T08:53:24.483403+0000","last_unstale":"2024-05-08T08:54:22.748550+0000","last_undegraded":"2024-05-08T08:54:22.748550+0000","last_fullsized":"2024-05-08T08:54:22.748550+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:10:12.800749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082481000000000004,"stat_sum":{"num_bytes":24053880,"num_objects":91,"num_object_clones":12,"num_object_copies":182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91,"num_whiteouts":56,"num_read":8982,"num_read_kb":44851,"num_write":311,"num_write_kb":15973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":8814624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"109'532","reported_seq":18423,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T08:54:22.755147+0000","last_change":"2024-05-08T08:52:13.945184+0000","last_active":"2024-05-08T08:54:22.755147+0000","last_peered":"2024-05-08T08:54:22.755147+0000","last_clean":"2024-05-08T08:54:22.755147+0000","last_became_active":"2024-05-08T08:52:11.927366+0000","last_became_peered":"2024-05-08T08:52:11.927366+0000","last_unstale":"2024-05-08T08:54:22.755147+0000","last_undegraded":"2024-05-08T08:54:22.755147+0000","last_fullsized":"2024-05-08T08:54:22.755147+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:27:34.845912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057208900000000002,"stat_sum":{"num_bytes":85617896,"num_objects":105,"num_object_clones":19,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":105,"num_whiteouts":57,"num_read":19130,"num_read_kb":51248,"num_write":557,"num_write_kb":23181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"106'723","reported_seq":18021,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T08:54:22.748359+0000","last_change":"2024-05-08T08:53:11.359646+0000","last_active":"2024-05-08T08:54:22.748359+0000","last_peered":"2024-05-08T08:54:22.748359+0000","last_clean":"2024-05-08T08:54:22.748359+0000","last_became_active":"2024-05-08T08:53:11.358391+0000","last_became_peered":"2024-05-08T08:53:11.358391+0000","last_unstale":"2024-05-08T08:54:22.748359+0000","last_undegraded":"2024-05-08T08:54:22.748359+0000","last_fullsized":"2024-05-08T08:54:22.748359+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":723,"log_dups_size":0,"ondisk_log_size":723,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:18:11.723552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000866371,"stat_sum":{"num_bytes":61572242,"num_objects":83,"num_object_clones":13,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":39,"num_read":19858,"num_read_kb":74082,"num_write":864,"num_write_kb":16297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":97,"num_bytes_recovered":2508994,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"108'1334","reported_seq":19828,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T08:54:22.753248+0000","last_change":"2024-05-08T08:53:59.617614+0000","last_active":"2024-05-08T08:54:22.753248+0000","last_peered":"2024-05-08T08:54:22.753248+0000","last_clean":"2024-05-08T08:54:22.753248+0000","last_became_active":"2024-05-08T08:53:59.615917+0000","last_became_peered":"2024-05-08T08:53:59.615917+0000","last_unstale":"2024-05-08T08:54:22.753248+0000","last_undegraded":"2024-05-08T08:54:22.753248+0000","last_fullsized":"2024-05-08T08:54:22.753248+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":1334,"log_dups_size":0,"ondisk_log_size":1334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:05:31.412892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084652399999999997,"stat_sum":{"num_bytes":45283688,"num_objects":85,"num_object_clones":13,"num_object_copies":170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":41,"num_read":23932,"num_read_kb":79349,"num_write":2156,"num_write_kb":26565,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":118,"num_bytes_recovered":21349055,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"111'554","reported_seq":15689,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T08:54:22.753082+0000","last_change":"2024-05-08T08:52:13.944999+0000","last_active":"2024-05-08T08:54:22.753082+0000","last_peered":"2024-05-08T08:54:22.753082+0000","last_clean":"2024-05-08T08:54:22.753082+0000","last_became_active":"2024-05-08T08:52:11.931695+0000","last_became_peered":"2024-05-08T08:52:11.931695+0000","last_unstale":"2024-05-08T08:54:22.753082+0000","last_undegraded":"2024-05-08T08:54:22.753082+0000","last_fullsized":"2024-05-08T08:54:22.753082+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":554,"log_dups_size":0,"ondisk_log_size":554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:35:44.051052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063902800000000004,"stat_sum":{"num_bytes":43369653,"num_objects":90,"num_object_clones":12,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":50,"num_read":16894,"num_read_kb":50280,"num_write":664,"num_write_kb":8162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"111'598","reported_seq":11782,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T08:54:22.753072+0000","last_change":"2024-05-08T08:53:23.481964+0000","last_active":"2024-05-08T08:54:22.753072+0000","last_peered":"2024-05-08T08:54:22.753072+0000","last_clean":"2024-05-08T08:54:22.753072+0000","last_became_active":"2024-05-08T08:53:23.481362+0000","last_became_peered":"2024-05-08T08:53:23.481362+0000","last_unstale":"2024-05-08T08:54:22.753072+0000","last_undegraded":"2024-05-08T08:54:22.753072+0000","last_fullsized":"2024-05-08T08:54:22.753072+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":598,"log_dups_size":0,"ondisk_log_size":598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:57:34.221498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000430331,"stat_sum":{"num_bytes":67317875,"num_objects":90,"num_object_clones":17,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":44,"num_read":10165,"num_read_kb":32507,"num_write":574,"num_write_kb":10875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":3612768,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":212,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T08:54:22.753347+0000","last_change":"2024-05-08T08:53:20.362387+0000","last_active":"2024-05-08T08:54:22.753347+0000","last_peered":"2024-05-08T08:54:22.753347+0000","last_clean":"2024-05-08T08:54:22.753347+0000","last_became_active":"2024-05-08T08:53:20.362113+0000","last_became_peered":"2024-05-08T08:53:20.362113+0000","last_unstale":"2024-05-08T08:54:22.753347+0000","last_undegraded":"2024-05-08T08:54:22.753347+0000","last_fullsized":"2024-05-08T08:54:22.753347+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:28:25.204125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"111'431","reported_seq":18984,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T08:54:22.748571+0000","last_change":"2024-05-08T08:53:49.580812+0000","last_active":"2024-05-08T08:54:22.748571+0000","last_peered":"2024-05-08T08:54:22.748571+0000","last_clean":"2024-05-08T08:54:22.748571+0000","last_became_active":"2024-05-08T08:53:49.579888+0000","last_became_peered":"2024-05-08T08:53:49.579888+0000","last_unstale":"2024-05-08T08:54:22.748571+0000","last_undegraded":"2024-05-08T08:54:22.748571+0000","last_fullsized":"2024-05-08T08:54:22.748571+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:23:52.377493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088867899999999997,"stat_sum":{"num_bytes":55876825,"num_objects":85,"num_object_clones":10,"num_object_copies":170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":47,"num_read":20504,"num_read_kb":84513,"num_write":411,"num_write_kb":16385,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":203,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T08:54:22.748573+0000","last_change":"2024-05-08T08:52:03.896159+0000","last_active":"2024-05-08T08:54:22.748573+0000","last_peered":"2024-05-08T08:54:22.748573+0000","last_clean":"2024-05-08T08:54:22.748573+0000","last_became_active":"2024-05-08T08:52:03.895797+0000","last_became_peered":"2024-05-08T08:52:03.895797+0000","last_unstale":"2024-05-08T08:54:22.748573+0000","last_undegraded":"2024-05-08T08:54:22.748573+0000","last_fullsized":"2024-05-08T08:54:22.748573+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:34:33.513525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"110'418","reported_seq":6284,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T08:54:22.755362+0000","last_change":"2024-05-08T08:52:13.944951+0000","last_active":"2024-05-08T08:54:22.755362+0000","last_peered":"2024-05-08T08:54:22.755362+0000","last_clean":"2024-05-08T08:54:22.755362+0000","last_became_active":"2024-05-08T08:52:11.926905+0000","last_became_peered":"2024-05-08T08:52:11.926905+0000","last_unstale":"2024-05-08T08:54:22.755362+0000","last_undegraded":"2024-05-08T08:54:22.755362+0000","last_fullsized":"2024-05-08T08:54:22.755362+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":418,"log_dups_size":0,"ondisk_log_size":418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:28:34.010821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044527,"stat_sum":{"num_bytes":18755761,"num_objects":98,"num_object_clones":17,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":50,"num_read":6564,"num_read_kb":34229,"num_write":361,"num_write_kb":11482,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":202,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T08:54:22.753191+0000","last_change":"2024-05-08T08:52:04.144764+0000","last_active":"2024-05-08T08:54:22.753191+0000","last_peered":"2024-05-08T08:54:22.753191+0000","last_clean":"2024-05-08T08:54:22.753191+0000","last_became_active":"2024-05-08T08:52:04.144444+0000","last_became_peered":"2024-05-08T08:52:04.144444+0000","last_unstale":"2024-05-08T08:54:22.753191+0000","last_undegraded":"2024-05-08T08:54:22.753191+0000","last_fullsized":"2024-05-08T08:54:22.753191+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:58:31.498922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"112'672","reported_seq":24923,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T08:54:22.750580+0000","last_change":"2024-05-08T08:53:58.619407+0000","last_active":"2024-05-08T08:54:22.750580+0000","last_peered":"2024-05-08T08:54:22.750580+0000","last_clean":"2024-05-08T08:54:22.750580+0000","last_became_active":"2024-05-08T08:53:58.618388+0000","last_became_peered":"2024-05-08T08:53:58.618388+0000","last_unstale":"2024-05-08T08:54:22.750580+0000","last_undegraded":"2024-05-08T08:54:22.750580+0000","last_fullsized":"2024-05-08T08:54:22.750580+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":672,"log_dups_size":0,"ondisk_log_size":672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:11:16.685169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000669351,"stat_sum":{"num_bytes":49627761,"num_objects":118,"num_object_clones":19,"num_object_copies":236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":118,"num_whiteouts":62,"num_read":21278,"num_read_kb":48010,"num_write":517,"num_write_kb":11434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":215,"num_bytes_recovered":10925568,"num_keys_recovered":44,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":203,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T08:54:22.748564+0000","last_change":"2024-05-08T08:52:04.144543+0000","last_active":"2024-05-08T08:54:22.748564+0000","last_peered":"2024-05-08T08:54:22.748564+0000","last_clean":"2024-05-08T08:54:22.748564+0000","last_became_active":"2024-05-08T08:52:04.144242+0000","last_became_peered":"2024-05-08T08:52:04.144242+0000","last_unstale":"2024-05-08T08:54:22.748564+0000","last_undegraded":"2024-05-08T08:54:22.748564+0000","last_fullsized":"2024-05-08T08:54:22.748564+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:57:36.844633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"109'832","reported_seq":13376,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T08:54:22.755308+0000","last_change":"2024-05-08T08:53:58.616694+0000","last_active":"2024-05-08T08:54:22.755308+0000","last_peered":"2024-05-08T08:54:22.755308+0000","last_clean":"2024-05-08T08:54:22.755308+0000","last_became_active":"2024-05-08T08:53:58.615614+0000","last_became_peered":"2024-05-08T08:53:58.615614+0000","last_unstale":"2024-05-08T08:54:22.755308+0000","last_undegraded":"2024-05-08T08:54:22.755308+0000","last_fullsized":"2024-05-08T08:54:22.755308+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":832,"log_dups_size":0,"ondisk_log_size":832,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:53:30.727616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064078099999999999,"stat_sum":{"num_bytes":22984955,"num_objects":104,"num_object_clones":15,"num_object_copies":208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":66,"num_read":14890,"num_read_kb":32244,"num_write":1060,"num_write_kb":9548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":142,"num_bytes_recovered":4609275,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"111'492","reported_seq":10681,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T08:54:22.747812+0000","last_change":"2024-05-08T08:53:24.481238+0000","last_active":"2024-05-08T08:54:22.747812+0000","last_peered":"2024-05-08T08:54:22.747812+0000","last_clean":"2024-05-08T08:54:22.747812+0000","last_became_active":"2024-05-08T08:53:24.480423+0000","last_became_peered":"2024-05-08T08:53:24.480423+0000","last_unstale":"2024-05-08T08:54:22.747812+0000","last_undegraded":"2024-05-08T08:54:22.747812+0000","last_fullsized":"2024-05-08T08:54:22.747812+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":492,"log_dups_size":0,"ondisk_log_size":492,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:12:53.281706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047037799999999997,"stat_sum":{"num_bytes":39198542,"num_objects":81,"num_object_clones":13,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":44,"num_read":11250,"num_read_kb":32978,"num_write":569,"num_write_kb":6409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":101,"num_bytes_recovered":5689443,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":182,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T08:54:22.748706+0000","last_change":"2024-05-08T08:53:55.365435+0000","last_active":"2024-05-08T08:54:22.748706+0000","last_peered":"2024-05-08T08:54:22.748706+0000","last_clean":"2024-05-08T08:54:22.748706+0000","last_became_active":"2024-05-08T08:53:55.365059+0000","last_became_peered":"2024-05-08T08:53:55.365059+0000","last_unstale":"2024-05-08T08:54:22.748706+0000","last_undegraded":"2024-05-08T08:54:22.748706+0000","last_fullsized":"2024-05-08T08:54:22.748706+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:15:39.278668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"109'482","reported_seq":10941,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T08:54:22.748442+0000","last_change":"2024-05-08T08:52:13.940504+0000","last_active":"2024-05-08T08:54:22.748442+0000","last_peered":"2024-05-08T08:54:22.748442+0000","last_clean":"2024-05-08T08:54:22.748442+0000","last_became_active":"2024-05-08T08:52:11.928301+0000","last_became_peered":"2024-05-08T08:52:11.928301+0000","last_unstale":"2024-05-08T08:54:22.748442+0000","last_undegraded":"2024-05-08T08:54:22.748442+0000","last_fullsized":"2024-05-08T08:54:22.748442+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":482,"log_dups_size":0,"ondisk_log_size":482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:02:42.129618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046631800000000002,"stat_sum":{"num_bytes":50064999,"num_objects":98,"num_object_clones":17,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":51,"num_read":7527,"num_read_kb":36742,"num_write":502,"num_write_kb":14712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":203,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T08:54:22.755130+0000","last_change":"2024-05-08T08:52:03.891768+0000","last_active":"2024-05-08T08:54:22.755130+0000","last_peered":"2024-05-08T08:54:22.755130+0000","last_clean":"2024-05-08T08:54:22.755130+0000","last_became_active":"2024-05-08T08:52:03.891374+0000","last_became_peered":"2024-05-08T08:52:03.891374+0000","last_unstale":"2024-05-08T08:54:22.755130+0000","last_undegraded":"2024-05-08T08:54:22.755130+0000","last_fullsized":"2024-05-08T08:54:22.755130+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:18:10.240492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"111'894","reported_seq":24912,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T08:54:22.747547+0000","last_change":"2024-05-08T08:53:49.582654+0000","last_active":"2024-05-08T08:54:22.747547+0000","last_peered":"2024-05-08T08:54:22.747547+0000","last_clean":"2024-05-08T08:54:22.747547+0000","last_became_active":"2024-05-08T08:53:49.581797+0000","last_became_peered":"2024-05-08T08:53:49.581797+0000","last_unstale":"2024-05-08T08:54:22.747547+0000","last_undegraded":"2024-05-08T08:54:22.747547+0000","last_fullsized":"2024-05-08T08:54:22.747547+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:34:48.759888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046440499999999998,"stat_sum":{"num_bytes":67691375,"num_objects":110,"num_object_clones":17,"num_object_copies":220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":110,"num_whiteouts":54,"num_read":27879,"num_read_kb":46680,"num_write":649,"num_write_kb":8923,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":135,"num_bytes_recovered":3549534,"num_keys_recovered":46,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":202,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T08:54:22.752857+0000","last_change":"2024-05-08T08:52:03.891551+0000","last_active":"2024-05-08T08:54:22.752857+0000","last_peered":"2024-05-08T08:54:22.752857+0000","last_clean":"2024-05-08T08:54:22.752857+0000","last_became_active":"2024-05-08T08:52:03.891296+0000","last_became_peered":"2024-05-08T08:52:03.891296+0000","last_unstale":"2024-05-08T08:54:22.752857+0000","last_undegraded":"2024-05-08T08:54:22.752857+0000","last_fullsized":"2024-05-08T08:54:22.752857+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:54.806343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"112'818","reported_seq":12171,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T08:54:22.748693+0000","last_change":"2024-05-08T08:53:58.616835+0000","last_active":"2024-05-08T08:54:22.748693+0000","last_peered":"2024-05-08T08:54:22.748693+0000","last_clean":"2024-05-08T08:54:22.748693+0000","last_became_active":"2024-05-08T08:53:58.616032+0000","last_became_peered":"2024-05-08T08:53:58.616032+0000","last_unstale":"2024-05-08T08:54:22.748693+0000","last_undegraded":"2024-05-08T08:54:22.748693+0000","last_fullsized":"2024-05-08T08:54:22.748693+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":818,"log_dups_size":0,"ondisk_log_size":818,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:41:03.855070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062135599999999999,"stat_sum":{"num_bytes":56057874,"num_objects":109,"num_object_clones":19,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":57,"num_read":13406,"num_read_kb":51060,"num_write":1138,"num_write_kb":13545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":125,"num_bytes_recovered":4950942,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":182,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T08:54:22.755171+0000","last_change":"2024-05-08T08:53:55.362409+0000","last_active":"2024-05-08T08:54:22.755171+0000","last_peered":"2024-05-08T08:54:22.755171+0000","last_clean":"2024-05-08T08:54:22.755171+0000","last_became_active":"2024-05-08T08:53:55.362055+0000","last_became_peered":"2024-05-08T08:53:55.362055+0000","last_unstale":"2024-05-08T08:54:22.755171+0000","last_undegraded":"2024-05-08T08:54:22.755171+0000","last_fullsized":"2024-05-08T08:54:22.755171+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:34:23.719119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":358,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T08:54:22.752870+0000","last_change":"2024-05-08T08:51:58.861899+0000","last_active":"2024-05-08T08:54:22.752870+0000","last_peered":"2024-05-08T08:54:22.752870+0000","last_clean":"2024-05-08T08:54:22.752870+0000","last_became_active":"2024-05-08T08:51:58.861596+0000","last_became_peered":"2024-05-08T08:51:58.861596+0000","last_unstale":"2024-05-08T08:54:22.752870+0000","last_undegraded":"2024-05-08T08:54:22.752870+0000","last_fullsized":"2024-05-08T08:54:22.752870+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:51:57.849666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:51:57.849666+0000","last_clean_scrub_stamp":"2024-05-08T08:51:57.849666+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:53:21.739950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"111'313","reported_seq":8922,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T08:54:22.755352+0000","last_change":"2024-05-08T08:52:13.945362+0000","last_active":"2024-05-08T08:54:22.755352+0000","last_peered":"2024-05-08T08:54:22.755352+0000","last_clean":"2024-05-08T08:54:22.755352+0000","last_became_active":"2024-05-08T08:52:11.926975+0000","last_became_peered":"2024-05-08T08:52:11.926975+0000","last_unstale":"2024-05-08T08:54:22.755352+0000","last_undegraded":"2024-05-08T08:54:22.755352+0000","last_fullsized":"2024-05-08T08:54:22.755352+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:37:15.277712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068218599999999999,"stat_sum":{"num_bytes":45654112,"num_objects":91,"num_object_clones":15,"num_object_copies":182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91,"num_whiteouts":48,"num_read":8476,"num_read_kb":33622,"num_write":282,"num_write_kb":8394,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"111'402","reported_seq":13955,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T08:54:22.747700+0000","last_change":"2024-05-08T08:53:23.479481+0000","last_active":"2024-05-08T08:54:22.747700+0000","last_peered":"2024-05-08T08:54:22.747700+0000","last_clean":"2024-05-08T08:54:22.747700+0000","last_became_active":"2024-05-08T08:53:23.478564+0000","last_became_peered":"2024-05-08T08:53:23.478564+0000","last_unstale":"2024-05-08T08:54:22.747700+0000","last_undegraded":"2024-05-08T08:54:22.747700+0000","last_fullsized":"2024-05-08T08:54:22.747700+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":402,"log_dups_size":0,"ondisk_log_size":402,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:09:29.559654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00030754000000000003,"stat_sum":{"num_bytes":41793071,"num_objects":95,"num_object_clones":12,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":52,"num_read":14003,"num_read_kb":62054,"num_write":342,"num_write_kb":16518,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":131,"num_bytes_recovered":12377088,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"111'1095","reported_seq":11173,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T08:54:22.753183+0000","last_change":"2024-05-08T08:53:24.481272+0000","last_active":"2024-05-08T08:54:22.753183+0000","last_peered":"2024-05-08T08:54:22.753183+0000","last_clean":"2024-05-08T08:54:22.753183+0000","last_became_active":"2024-05-08T08:53:24.480359+0000","last_became_peered":"2024-05-08T08:53:24.480359+0000","last_unstale":"2024-05-08T08:54:22.753183+0000","last_undegraded":"2024-05-08T08:54:22.753183+0000","last_fullsized":"2024-05-08T08:54:22.753183+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":1095,"log_dups_size":0,"ondisk_log_size":1095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:48:29.240956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070350100000000004,"stat_sum":{"num_bytes":26339183,"num_objects":86,"num_object_clones":12,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":50,"num_read":15386,"num_read_kb":29092,"num_write":1331,"num_write_kb":6282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":2175343,"num_keys_recovered":61,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"111'504","reported_seq":10510,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T08:54:22.748221+0000","last_change":"2024-05-08T08:52:13.940600+0000","last_active":"2024-05-08T08:54:22.748221+0000","last_peered":"2024-05-08T08:54:22.748221+0000","last_clean":"2024-05-08T08:54:22.748221+0000","last_became_active":"2024-05-08T08:52:11.927196+0000","last_became_peered":"2024-05-08T08:52:11.927196+0000","last_unstale":"2024-05-08T08:54:22.748221+0000","last_undegraded":"2024-05-08T08:54:22.748221+0000","last_fullsized":"2024-05-08T08:54:22.748221+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":504,"log_dups_size":0,"ondisk_log_size":504,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:23:56.059330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048296599999999998,"stat_sum":{"num_bytes":43172965,"num_objects":99,"num_object_clones":13,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":60,"num_read":10932,"num_read_kb":36092,"num_write":427,"num_write_kb":7390,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"111'301","reported_seq":13089,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T08:54:22.752996+0000","last_change":"2024-05-08T08:53:58.616502+0000","last_active":"2024-05-08T08:54:22.752996+0000","last_peered":"2024-05-08T08:54:22.752996+0000","last_clean":"2024-05-08T08:54:22.752996+0000","last_became_active":"2024-05-08T08:53:58.615095+0000","last_became_peered":"2024-05-08T08:53:58.615095+0000","last_unstale":"2024-05-08T08:54:22.752996+0000","last_undegraded":"2024-05-08T08:54:22.752996+0000","last_fullsized":"2024-05-08T08:54:22.752996+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":301,"log_dups_size":0,"ondisk_log_size":301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:43:57.306476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048793000000000003,"stat_sum":{"num_bytes":20995771,"num_objects":84,"num_object_clones":10,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":49,"num_read":12679,"num_read_kb":38365,"num_write":261,"num_write_kb":6745,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":107,"num_bytes_recovered":2139323,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"111'394","reported_seq":8222,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T08:54:22.753118+0000","last_change":"2024-05-08T08:52:13.945141+0000","last_active":"2024-05-08T08:54:22.753118+0000","last_peered":"2024-05-08T08:54:22.753118+0000","last_clean":"2024-05-08T08:54:22.753118+0000","last_became_active":"2024-05-08T08:52:11.932594+0000","last_became_peered":"2024-05-08T08:52:11.932594+0000","last_unstale":"2024-05-08T08:54:22.753118+0000","last_undegraded":"2024-05-08T08:54:22.753118+0000","last_fullsized":"2024-05-08T08:54:22.753118+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":394,"log_dups_size":0,"ondisk_log_size":394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:31:13.011913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074388199999999996,"stat_sum":{"num_bytes":29495349,"num_objects":86,"num_object_clones":10,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":51,"num_read":9093,"num_read_kb":33752,"num_write":343,"num_write_kb":9957,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"112'496","reported_seq":8393,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T08:54:22.764295+0000","last_change":"2024-05-08T08:53:58.622564+0000","last_active":"2024-05-08T08:54:22.764295+0000","last_peered":"2024-05-08T08:54:22.764295+0000","last_clean":"2024-05-08T08:54:22.764295+0000","last_became_active":"2024-05-08T08:53:58.621584+0000","last_became_peered":"2024-05-08T08:53:58.621584+0000","last_unstale":"2024-05-08T08:54:22.764295+0000","last_undegraded":"2024-05-08T08:54:22.764295+0000","last_fullsized":"2024-05-08T08:54:22.764295+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":496,"log_dups_size":0,"ondisk_log_size":496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:30:33.210141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085079600000000002,"stat_sum":{"num_bytes":39832656,"num_objects":75,"num_object_clones":14,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":34,"num_read":15038,"num_read_kb":46151,"num_write":545,"num_write_kb":12165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":188,"num_bytes_recovered":11952192,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"111'466","reported_seq":7494,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T08:54:22.753296+0000","last_change":"2024-05-08T08:53:58.616115+0000","last_active":"2024-05-08T08:54:22.753296+0000","last_peered":"2024-05-08T08:54:22.753296+0000","last_clean":"2024-05-08T08:54:22.753296+0000","last_became_active":"2024-05-08T08:53:58.615064+0000","last_became_peered":"2024-05-08T08:53:58.615064+0000","last_unstale":"2024-05-08T08:54:22.753296+0000","last_undegraded":"2024-05-08T08:54:22.753296+0000","last_fullsized":"2024-05-08T08:54:22.753296+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":466,"log_dups_size":0,"ondisk_log_size":466,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:04:06.014042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010326370000000001,"stat_sum":{"num_bytes":39948549,"num_objects":81,"num_object_clones":9,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":45,"num_read":8408,"num_read_kb":37914,"num_write":473,"num_write_kb":9513,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":107,"num_bytes_recovered":4068050,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"111'350","reported_seq":7537,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T08:54:22.748076+0000","last_change":"2024-05-08T08:53:58.619593+0000","last_active":"2024-05-08T08:54:22.748076+0000","last_peered":"2024-05-08T08:54:22.748076+0000","last_clean":"2024-05-08T08:54:22.748076+0000","last_became_active":"2024-05-08T08:53:58.618787+0000","last_became_peered":"2024-05-08T08:53:58.618787+0000","last_unstale":"2024-05-08T08:54:22.748076+0000","last_undegraded":"2024-05-08T08:54:22.748076+0000","last_fullsized":"2024-05-08T08:54:22.748076+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":350,"log_dups_size":0,"ondisk_log_size":350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:47:52.512010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053593,"stat_sum":{"num_bytes":33387105,"num_objects":86,"num_object_clones":14,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":51,"num_read":7287,"num_read_kb":39789,"num_write":340,"num_write_kb":13798,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":112,"num_bytes_recovered":7552081,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"111'607","reported_seq":10797,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T08:54:22.753129+0000","last_change":"2024-05-08T08:53:24.480561+0000","last_active":"2024-05-08T08:54:22.753129+0000","last_peered":"2024-05-08T08:54:22.753129+0000","last_clean":"2024-05-08T08:54:22.753129+0000","last_became_active":"2024-05-08T08:53:24.479891+0000","last_became_peered":"2024-05-08T08:53:24.479891+0000","last_unstale":"2024-05-08T08:54:22.753129+0000","last_undegraded":"2024-05-08T08:54:22.753129+0000","last_fullsized":"2024-05-08T08:54:22.753129+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":607,"log_dups_size":0,"ondisk_log_size":607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:51:24.578505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041230699999999999,"stat_sum":{"num_bytes":70820389,"num_objects":104,"num_object_clones":18,"num_object_copies":208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":59,"num_read":12414,"num_read_kb":35133,"num_write":494,"num_write_kb":9578,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":131,"num_bytes_recovered":2604032,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"111'1354","reported_seq":23857,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T08:54:22.753079+0000","last_change":"2024-05-08T08:53:49.581481+0000","last_active":"2024-05-08T08:54:22.753079+0000","last_peered":"2024-05-08T08:54:22.753079+0000","last_clean":"2024-05-08T08:54:22.753079+0000","last_became_active":"2024-05-08T08:53:49.580557+0000","last_became_peered":"2024-05-08T08:53:49.580557+0000","last_unstale":"2024-05-08T08:54:22.753079+0000","last_undegraded":"2024-05-08T08:54:22.753079+0000","last_fullsized":"2024-05-08T08:54:22.753079+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":1354,"log_dups_size":0,"ondisk_log_size":1354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:22:38.300053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059220300000000002,"stat_sum":{"num_bytes":65048937,"num_objects":114,"num_object_clones":12,"num_object_copies":228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":114,"num_whiteouts":56,"num_read":29350,"num_read_kb":56834,"num_write":1918,"num_write_kb":10978,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"110'510","reported_seq":9984,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T08:54:22.753000+0000","last_change":"2024-05-08T08:53:58.616558+0000","last_active":"2024-05-08T08:54:22.753000+0000","last_peered":"2024-05-08T08:54:22.753000+0000","last_clean":"2024-05-08T08:54:22.753000+0000","last_became_active":"2024-05-08T08:53:58.615947+0000","last_became_peered":"2024-05-08T08:53:58.615947+0000","last_unstale":"2024-05-08T08:54:22.753000+0000","last_undegraded":"2024-05-08T08:54:22.753000+0000","last_fullsized":"2024-05-08T08:54:22.753000+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":510,"log_dups_size":0,"ondisk_log_size":510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:27:49.478660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059972400000000005,"stat_sum":{"num_bytes":51989689,"num_objects":102,"num_object_clones":20,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":53,"num_read":10430,"num_read_kb":40585,"num_write":508,"num_write_kb":9043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":130,"num_bytes_recovered":6657151,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1472698912,"num_objects":3037,"num_object_clones":455,"num_object_copies":6074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3037,"num_whiteouts":1639,"num_read":480111,"num_read_kb":1541062,"num_write":21833,"num_write_kb":435702,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2942,"num_bytes_recovered":182360369,"num_keys_recovered":458,"num_objects_omap":48,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":93782016,"data_stored":768042323,"data_compressed":41956287,"data_compressed_allocated":76791808,"data_compressed_original":757655040,"omap_allocated":0,"internal_metadata":0},"log_size":19670,"ondisk_log_size":19670,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":11046,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":0,"seq":47244640287,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640290,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72920,"kb_used_data":15376,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94298920,"statfs":{"total":96636764160,"available":96562094080,"internally_reserved":0,"allocated":15745024,"data_stored":120347362,"data_compressed":7061952,"data_compressed_allocated":12357632,"data_compressed_original":118580736,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640289,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86612,"kb_used_data":19272,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94285228,"statfs":{"total":96636764160,"available":96548073472,"internally_reserved":0,"allocated":19734528,"data_stored":144971797,"data_compressed":7955053,"data_compressed_allocated":14168064,"data_compressed_original":141705216,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640288,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43752,"kb_used_data":12552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94328088,"statfs":{"total":96636764160,"available":96591962112,"internally_reserved":0,"allocated":12853248,"data_stored":90052181,"data_compressed":5545013,"data_compressed_allocated":9596928,"data_compressed_original":88355840,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640287,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78664,"kb_used_data":21116,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94293176,"statfs":{"total":96636764160,"available":96556212224,"internally_reserved":0,"allocated":21622784,"data_stored":185843235,"data_compressed":9092791,"data_compressed_allocated":17469440,"data_compressed_original":183613440,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640290,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":97360,"kb_used_data":17608,"kb_used_omap":1,"kb_used_meta":79742,"kb_avail":94274480,"statfs":{"total":96636764160,"available":96537067520,"internally_reserved":0,"allocated":18030592,"data_stored":127824737,"data_compressed":6813608,"data_compressed_allocated":12615680,"data_compressed_original":124759040,"omap_allocated":1595,"internal_metadata":81656261},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5523,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5523,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16695296,"data_stored":126741077,"data_compressed":6821522,"data_compressed_allocated":12599296,"data_compressed_original":124222464,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20623360,"data_stored":186956125,"data_compressed":9300070,"data_compressed_allocated":17760256,"data_compressed_original":185260032,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":11608064,"data_stored":89630569,"data_compressed":5593345,"data_compressed_allocated":9654272,"data_compressed_original":88470528,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":18718720,"data_stored":145612079,"data_compressed":8159479,"data_compressed_allocated":14438400,"data_compressed_original":142880768,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":14430208,"data_stored":119803894,"data_compressed":7061952,"data_compressed_allocated":12357632,"data_compressed_original":118580736,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":11706368,"data_stored":99298579,"data_compressed":5019919,"data_compressed_allocated":9981952,"data_compressed_original":98240512,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T08:54:27.517 INFO:tasks.thrashosds.thrasher:Setting 3.7 to [4, 3] 2024-05-08T08:54:27.517 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.7', '4', '3'] 2024-05-08T08:54:27.517 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.7 4 3 2024-05-08T08:54:27.517 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:27.514+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:27.533 INFO:teuthology.orchestra.run.smithi196.stdout:checking clone #48, image image_client.0-clone48 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent49 2024-05-08T08:54:27.612 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:27.615+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:27.705 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:27.705 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:54:27.705 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.706 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.6223.0:4410 3.5 3.ae36a85 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-05-08T08:54:27.706 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:54:27.708573+0000", 2024-05-08T08:54:27.706 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 7.1038000000000007e-05, 2024-05-08T08:54:27.706 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.9178999999999994e-05, 2024-05-08T08:54:27.706 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:27.706 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:27.706 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "reached pg", 2024-05-08T08:54:27.706 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:27.706 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.6223", 2024-05-08T08:54:27.706 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2934866360", 2024-05-08T08:54:27.706 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 4410 2024-05-08T08:54:27.706 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.706 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:27.706 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.706 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:27.707 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:54:27.708573+0000", 2024-05-08T08:54:27.707 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:27.707 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.707 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.707 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:27.707 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:54:27.708573+0000", 2024-05-08T08:54:27.707 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:27.707 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.707 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.707 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:27.707 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:54:27.708574+0000", 2024-05-08T08:54:27.707 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1699999999999997e-07 2024-05-08T08:54:27.707 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.707 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.707 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:27.708 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:54:27.708578+0000", 2024-05-08T08:54:27.708 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2930000000000002e-06 2024-05-08T08:54:27.708 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.708 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.708 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:27.708 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:54:27.708589+0000", 2024-05-08T08:54:27.708 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0366e-05 2024-05-08T08:54:27.708 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.708 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.708 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:27.708 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:54:27.708596+0000", 2024-05-08T08:54:27.708 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1980000000000004e-06 2024-05-08T08:54:27.708 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.708 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.708 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:27.709 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:54:27.708649+0000", 2024-05-08T08:54:27.709 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3232000000000001e-05 2024-05-08T08:54:27.709 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.709 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:27.709 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.709 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.709 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T08:54:27.709 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 1 2024-05-08T08:54:27.709 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:27.713 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:27.716+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:27.718 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:54:27.762 INFO:teuthology.orchestra.run.smithi059.stderr:nodeep-scrub is set 2024-05-08T08:54:27.814 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:27.817+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:27.847 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:27.848 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:54:27.848 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:54:27.848 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:54:27.848 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:27.857 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:54:27.915 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:27.919+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:27.924 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T08:54:27.980 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:27.981 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:54:27.981 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:54:27.981 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:54:27.981 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.981 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:437 3.17 3:ea93a512:::rbd_data.1140100c9703.0000000000000077:head [set-alloc-hint object_size 2097152 write_size 2097152,write 1170432~1024] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e63)", 2024-05-08T08:54:27.981 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:08.390804+0000", 2024-05-08T08:54:27.981 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 79.591810206999995, 2024-05-08T08:54:27.981 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0406068690000001, 2024-05-08T08:54:27.981 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:27.981 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:27.981 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:54:27.981 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:27.982 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:27.982 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:27.982 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 437 2024-05-08T08:54:27.982 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.982 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:27.982 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.982 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:27.982 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390804+0000", 2024-05-08T08:54:27.982 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:27.982 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.982 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.982 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:27.982 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390804+0000", 2024-05-08T08:54:27.982 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:27.983 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.983 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.983 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:27.983 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390806+0000", 2024-05-08T08:54:27.983 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.666e-06 2024-05-08T08:54:27.983 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.983 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.983 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:27.983 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390818+0000", 2024-05-08T08:54:27.983 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2187999999999999e-05 2024-05-08T08:54:27.983 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.983 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.983 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:27.983 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390834+0000", 2024-05-08T08:54:27.983 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6191000000000001e-05 2024-05-08T08:54:27.983 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.984 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.984 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:27.984 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390845+0000", 2024-05-08T08:54:27.984 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0831e-05 2024-05-08T08:54:27.984 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.984 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.984 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:27.984 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390922+0000", 2024-05-08T08:54:27.984 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7253e-05 2024-05-08T08:54:27.984 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.984 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.984 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op must wait for map", 2024-05-08T08:54:27.984 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390930+0000", 2024-05-08T08:54:27.984 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8259999999999993e-06 2024-05-08T08:54:27.984 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.985 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.985 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:27.985 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392155+0000", 2024-05-08T08:54:27.985 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0012256649999999999 2024-05-08T08:54:27.985 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.985 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.985 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:54:27.985 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392165+0000", 2024-05-08T08:54:27.985 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5799999999999998e-06 2024-05-08T08:54:27.985 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.985 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.985 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:27.985 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.429596+0000", 2024-05-08T08:54:27.985 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0374313639999999 2024-05-08T08:54:27.985 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.986 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.986 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:27.986 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.429728+0000", 2024-05-08T08:54:27.986 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000131552 2024-05-08T08:54:27.986 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.986 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.986 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 0,1", 2024-05-08T08:54:27.986 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.429860+0000", 2024-05-08T08:54:27.986 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000132137 2024-05-08T08:54:27.986 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.986 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.986 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:54:27.986 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.430811+0000", 2024-05-08T08:54:27.986 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00095040200000000004 2024-05-08T08:54:27.987 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.987 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.987 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:54:27.987 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431126+0000", 2024-05-08T08:54:27.987 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000315006 2024-05-08T08:54:27.987 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.987 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.987 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:54:27.987 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431272+0000", 2024-05-08T08:54:27.987 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000146487 2024-05-08T08:54:27.987 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.987 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.987 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:54:27.987 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431315+0000", 2024-05-08T08:54:27.987 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2697000000000001e-05 2024-05-08T08:54:27.988 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.988 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.988 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:27.988 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431411+0000", 2024-05-08T08:54:27.988 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6024000000000001e-05 2024-05-08T08:54:27.988 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.988 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:27.988 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.988 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.988 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.988 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:439 3.17 3:ea93a512:::rbd_data.1140100c9703.0000000000000077:head [read 1163264~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-05-08T08:54:27.988 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:08.390871+0000", 2024-05-08T08:54:27.988 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 79.591742695999997, 2024-05-08T08:54:27.988 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0408790590000001, 2024-05-08T08:54:27.988 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:27.989 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:27.989 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:27.989 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:27.989 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:27.989 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:27.989 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 439 2024-05-08T08:54:27.989 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.989 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:27.989 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.989 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:27.989 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390871+0000", 2024-05-08T08:54:27.989 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:27.989 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.989 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.990 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:27.990 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390871+0000", 2024-05-08T08:54:27.990 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:27.990 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.990 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.990 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:27.990 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390872+0000", 2024-05-08T08:54:27.990 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6300000000000005e-07 2024-05-08T08:54:27.990 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.990 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.990 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:27.990 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390875+0000", 2024-05-08T08:54:27.990 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3459999999999998e-06 2024-05-08T08:54:27.990 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.990 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.991 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:27.991 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390891+0000", 2024-05-08T08:54:27.991 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5255e-05 2024-05-08T08:54:27.991 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.991 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.991 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:27.991 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390897+0000", 2024-05-08T08:54:27.991 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2360000000000001e-06 2024-05-08T08:54:27.991 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.991 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.991 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:27.991 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390995+0000", 2024-05-08T08:54:27.991 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8522000000000003e-05 2024-05-08T08:54:27.991 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.991 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.991 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting_for_map not empty", 2024-05-08T08:54:27.992 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.391005+0000", 2024-05-08T08:54:27.992 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1460000000000002e-06 2024-05-08T08:54:27.992 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.992 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.992 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:27.992 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392191+0000", 2024-05-08T08:54:27.992 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0011860270000000001 2024-05-08T08:54:27.992 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.992 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.992 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:54:27.992 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392197+0000", 2024-05-08T08:54:27.992 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4289999999999998e-06 2024-05-08T08:54:27.992 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.992 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.992 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:27.992 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.430596+0000", 2024-05-08T08:54:27.993 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0383989090000001 2024-05-08T08:54:27.993 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.993 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.993 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for rw locks", 2024-05-08T08:54:27.993 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.430676+0000", 2024-05-08T08:54:27.993 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9781000000000001e-05 2024-05-08T08:54:27.993 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.993 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.993 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:27.993 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431460+0000", 2024-05-08T08:54:27.993 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00078427300000000004 2024-05-08T08:54:27.993 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.993 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.993 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:27.993 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431540+0000", 2024-05-08T08:54:27.993 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0224999999999998e-05 2024-05-08T08:54:27.994 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.994 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.994 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:27.994 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431750+0000", 2024-05-08T08:54:27.994 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00021024699999999999 2024-05-08T08:54:27.994 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.994 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:27.994 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.994 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.994 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.994 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:945 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 4096000~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:27.994 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:54:27.994 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 78.531593129000001, 2024-05-08T08:54:27.994 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086774741000000002, 2024-05-08T08:54:27.995 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:27.995 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:27.995 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:27.995 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:27.995 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:27.995 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:27.995 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 945 2024-05-08T08:54:27.995 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.995 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:27.995 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.995 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:27.995 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:54:27.995 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:27.995 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.995 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.995 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:27.995 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:54:27.996 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:27.996 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.996 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.996 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:27.996 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:54:27.996 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3499999999999996e-07 2024-05-08T08:54:27.996 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.996 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.996 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:27.996 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451024+0000", 2024-05-08T08:54:27.996 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.627e-06 2024-05-08T08:54:27.996 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.996 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.996 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:27.997 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451037+0000", 2024-05-08T08:54:27.997 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2431e-05 2024-05-08T08:54:27.997 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.997 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.997 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:27.997 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451042+0000", 2024-05-08T08:54:27.997 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8409999999999996e-06 2024-05-08T08:54:27.997 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.997 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.997 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:27.997 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537608+0000", 2024-05-08T08:54:27.997 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086565243 2024-05-08T08:54:27.997 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.997 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.997 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:27.997 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537687+0000", 2024-05-08T08:54:27.997 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9519999999999998e-05 2024-05-08T08:54:27.998 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.998 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.998 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:27.998 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537796+0000", 2024-05-08T08:54:27.998 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000108544 2024-05-08T08:54:27.998 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.998 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:27.998 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:27.998 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.998 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:27.998 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:964 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 458752~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:27.998 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451051+0000", 2024-05-08T08:54:27.998 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 78.531562956000002, 2024-05-08T08:54:27.998 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086990490000000004, 2024-05-08T08:54:27.999 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:27.999 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:27.999 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:27.999 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:27.999 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:27.999 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:27.999 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 964 2024-05-08T08:54:27.999 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:27.999 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:27.999 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.000 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:28.000 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451051+0000", 2024-05-08T08:54:28.000 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:28.000 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.000 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.000 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:28.000 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451051+0000", 2024-05-08T08:54:28.000 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:28.000 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.000 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.000 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:28.000 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451052+0000", 2024-05-08T08:54:28.000 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7299999999999996e-07 2024-05-08T08:54:28.000 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.000 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.001 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:28.001 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451055+0000", 2024-05-08T08:54:28.001 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.2770000000000001e-06 2024-05-08T08:54:28.001 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.001 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.001 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:28.001 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451061+0000", 2024-05-08T08:54:28.001 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4060000000000002e-06 2024-05-08T08:54:28.001 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.001 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.001 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:28.001 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451066+0000", 2024-05-08T08:54:28.001 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0839999999999999e-06 2024-05-08T08:54:28.001 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.001 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.002 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:28.002 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537839+0000", 2024-05-08T08:54:28.002 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086772962999999995 2024-05-08T08:54:28.002 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.002 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.002 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:28.002 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537934+0000", 2024-05-08T08:54:28.002 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4742999999999997e-05 2024-05-08T08:54:28.002 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.002 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.002 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:28.002 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538042+0000", 2024-05-08T08:54:28.002 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000107444 2024-05-08T08:54:28.002 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:28.003 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:28.003 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:28.003 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.003 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.003 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:965 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 950272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:28.003 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451075+0000", 2024-05-08T08:54:28.003 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 78.531538893999993, 2024-05-08T08:54:28.003 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087230196999999995, 2024-05-08T08:54:28.003 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:28.003 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:28.003 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:28.003 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:28.003 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:28.003 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:28.003 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 965 2024-05-08T08:54:28.004 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.004 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:28.004 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.004 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:28.004 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451075+0000", 2024-05-08T08:54:28.004 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:28.004 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.004 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.004 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:28.004 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451075+0000", 2024-05-08T08:54:28.004 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:28.004 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.004 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.004 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:28.004 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451076+0000", 2024-05-08T08:54:28.005 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5000000000000002e-07 2024-05-08T08:54:28.005 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.005 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.005 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:28.005 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451078+0000", 2024-05-08T08:54:28.005 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.5639999999999999e-06 2024-05-08T08:54:28.005 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.005 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.005 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:28.005 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451085+0000", 2024-05-08T08:54:28.005 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5640000000000002e-06 2024-05-08T08:54:28.005 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.005 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.005 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:28.005 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451090+0000", 2024-05-08T08:54:28.006 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1699999999999996e-06 2024-05-08T08:54:28.006 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.006 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.006 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:28.006 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538106+0000", 2024-05-08T08:54:28.006 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087015888999999999 2024-05-08T08:54:28.006 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.006 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.006 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:28.006 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538190+0000", 2024-05-08T08:54:28.006 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3961999999999993e-05 2024-05-08T08:54:28.006 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.006 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.006 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:28.007 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538305+0000", 2024-05-08T08:54:28.007 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011529800000000001 2024-05-08T08:54:28.007 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:28.007 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:28.007 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:28.007 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.007 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.007 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:966 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 1441792~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:28.007 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:54:28.007 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 78.531515521000003, 2024-05-08T08:54:28.007 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087446323000000006, 2024-05-08T08:54:28.007 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:28.007 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:28.007 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:28.007 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:28.008 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:28.008 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:28.008 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 966 2024-05-08T08:54:28.008 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.008 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:28.008 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.008 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:28.008 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:54:28.008 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:28.008 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.008 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.008 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:28.008 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:54:28.008 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:28.009 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.009 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.009 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:28.009 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:54:28.009 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3700000000000001e-07 2024-05-08T08:54:28.009 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.009 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.009 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:28.009 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451101+0000", 2024-05-08T08:54:28.009 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2560000000000001e-06 2024-05-08T08:54:28.009 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.009 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.009 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:28.009 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451107+0000", 2024-05-08T08:54:28.009 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1070000000000001e-06 2024-05-08T08:54:28.010 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.010 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.010 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:28.010 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451112+0000", 2024-05-08T08:54:28.010 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1420000000000001e-06 2024-05-08T08:54:28.010 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.010 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.010 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:28.010 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538341+0000", 2024-05-08T08:54:28.010 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087228302999999993 2024-05-08T08:54:28.010 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.010 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.010 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:28.010 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538424+0000", 2024-05-08T08:54:28.011 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3497000000000004e-05 2024-05-08T08:54:28.011 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.011 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.011 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:28.011 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538545+0000", 2024-05-08T08:54:28.011 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000120581 2024-05-08T08:54:28.011 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:28.011 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:28.011 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:28.011 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.011 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.011 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:967 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:28.011 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451120+0000", 2024-05-08T08:54:28.011 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 78.531494046000006, 2024-05-08T08:54:28.011 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087632725999999994, 2024-05-08T08:54:28.011 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:28.012 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:28.012 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:28.012 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:28.012 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:28.012 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:28.012 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 967 2024-05-08T08:54:28.012 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.012 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:28.012 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.012 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:28.012 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451120+0000", 2024-05-08T08:54:28.012 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:28.012 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.012 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.012 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:28.013 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451120+0000", 2024-05-08T08:54:28.013 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:28.013 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.013 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.013 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:28.013 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451121+0000", 2024-05-08T08:54:28.013 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8800000000000002e-07 2024-05-08T08:54:28.013 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.013 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.013 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:28.013 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451130+0000", 2024-05-08T08:54:28.013 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2299999999999997e-06 2024-05-08T08:54:28.013 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.013 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.013 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:28.013 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451137+0000", 2024-05-08T08:54:28.014 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4669999999999998e-06 2024-05-08T08:54:28.014 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.014 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.014 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:28.014 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451143+0000", 2024-05-08T08:54:28.014 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2839999999999997e-06 2024-05-08T08:54:28.014 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.014 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.014 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:28.014 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538575+0000", 2024-05-08T08:54:28.014 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087432789999999996 2024-05-08T08:54:28.014 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.014 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.015 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:28.015 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538648+0000", 2024-05-08T08:54:28.015 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2877e-05 2024-05-08T08:54:28.015 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.015 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.015 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:28.015 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538753+0000", 2024-05-08T08:54:28.015 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010449 2024-05-08T08:54:28.015 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:28.015 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:28.015 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:28.015 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.015 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.015 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:968 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 2424832~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:28.016 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451152+0000", 2024-05-08T08:54:28.016 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 78.531461961000005, 2024-05-08T08:54:28.016 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087813325999999997, 2024-05-08T08:54:28.016 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:28.016 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:28.016 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:28.016 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:28.016 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:28.016 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:28.016 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 968 2024-05-08T08:54:28.016 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.016 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:28.016 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.016 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:28.016 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451152+0000", 2024-05-08T08:54:28.017 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:28.017 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.017 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.017 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:28.017 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451152+0000", 2024-05-08T08:54:28.017 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:28.017 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.017 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.017 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:28.017 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451153+0000", 2024-05-08T08:54:28.017 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3099999999999998e-07 2024-05-08T08:54:28.017 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.017 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.017 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:28.018 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451155+0000", 2024-05-08T08:54:28.018 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4760000000000001e-06 2024-05-08T08:54:28.018 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.018 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.018 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:28.018 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451162+0000", 2024-05-08T08:54:28.018 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5969999999999997e-06 2024-05-08T08:54:28.018 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.018 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.018 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:28.018 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451167+0000", 2024-05-08T08:54:28.018 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.977e-06 2024-05-08T08:54:28.018 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.018 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.019 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:28.019 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538781+0000", 2024-05-08T08:54:28.019 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087614846999999996 2024-05-08T08:54:28.019 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.019 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.019 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:28.019 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538857+0000", 2024-05-08T08:54:28.019 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5358000000000006e-05 2024-05-08T08:54:28.019 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.019 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.019 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:28.019 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538965+0000", 2024-05-08T08:54:28.019 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010864 2024-05-08T08:54:28.019 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:28.019 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:28.019 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:28.020 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.020 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.020 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:969 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 2916352~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:28.020 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:54:28.020 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 78.531440255999996, 2024-05-08T08:54:28.020 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087998980000000004, 2024-05-08T08:54:28.020 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:28.020 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:28.020 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:28.020 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:28.020 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:28.020 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:28.020 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 969 2024-05-08T08:54:28.020 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.021 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:28.021 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.021 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:28.021 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:54:28.021 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:28.021 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.021 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.021 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:28.021 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:54:28.021 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:28.021 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.021 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.021 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:28.021 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:54:28.022 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3599999999999999e-07 2024-05-08T08:54:28.022 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.022 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.022 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:28.022 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451177+0000", 2024-05-08T08:54:28.022 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4590000000000001e-06 2024-05-08T08:54:28.022 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.022 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.022 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:28.022 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451184+0000", 2024-05-08T08:54:28.022 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4109999999999999e-06 2024-05-08T08:54:28.022 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.022 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.022 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:28.022 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451191+0000", 2024-05-08T08:54:28.023 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7279999999999998e-06 2024-05-08T08:54:28.023 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.023 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.023 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:28.023 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538997+0000", 2024-05-08T08:54:28.023 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087806170000000003 2024-05-08T08:54:28.023 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.023 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.023 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:28.023 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539069+0000", 2024-05-08T08:54:28.023 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2373999999999995e-05 2024-05-08T08:54:28.023 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.023 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.024 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:28.024 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539173+0000", 2024-05-08T08:54:28.024 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000103402 2024-05-08T08:54:28.024 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:28.024 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:28.024 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:28.024 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.024 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.024 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:970 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 3407872~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:28.024 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451198+0000", 2024-05-08T08:54:28.024 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 78.531415659000004, 2024-05-08T08:54:28.024 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088183368999999998, 2024-05-08T08:54:28.024 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:28.024 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:28.024 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:28.024 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:28.025 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:28.025 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:28.025 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 970 2024-05-08T08:54:28.025 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.025 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:28.025 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.025 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:28.025 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451198+0000", 2024-05-08T08:54:28.025 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:28.025 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.025 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.025 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:28.025 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451198+0000", 2024-05-08T08:54:28.025 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:28.025 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.025 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.026 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:28.026 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451199+0000", 2024-05-08T08:54:28.026 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7400000000000003e-07 2024-05-08T08:54:28.026 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.026 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.026 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:28.026 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451211+0000", 2024-05-08T08:54:28.026 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2109999999999999e-05 2024-05-08T08:54:28.026 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.026 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.026 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:28.026 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451219+0000", 2024-05-08T08:54:28.026 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4170000000000003e-06 2024-05-08T08:54:28.026 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.026 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.027 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:28.027 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451224+0000", 2024-05-08T08:54:28.027 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3990000000000003e-06 2024-05-08T08:54:28.027 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.027 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.027 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:28.027 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539204+0000", 2024-05-08T08:54:28.027 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087980099000000006 2024-05-08T08:54:28.027 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.027 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.027 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:28.027 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539276+0000", 2024-05-08T08:54:28.027 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1513000000000005e-05 2024-05-08T08:54:28.027 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.028 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.028 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:28.028 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539382+0000", 2024-05-08T08:54:28.028 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010615699999999999 2024-05-08T08:54:28.028 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:28.028 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:28.028 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:28.028 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.028 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.028 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:971 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 3899392~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:28.028 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451233+0000", 2024-05-08T08:54:28.028 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 78.531380952000006, 2024-05-08T08:54:28.028 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088431525999999996, 2024-05-08T08:54:28.028 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:28.029 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:28.029 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:28.029 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:28.029 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:28.029 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:28.029 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 971 2024-05-08T08:54:28.029 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.029 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:28.029 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.029 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:28.029 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451233+0000", 2024-05-08T08:54:28.029 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:28.029 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.029 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.029 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:28.030 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451233+0000", 2024-05-08T08:54:28.030 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:28.030 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.030 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.030 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:28.030 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451234+0000", 2024-05-08T08:54:28.030 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8500000000000001e-07 2024-05-08T08:54:28.030 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.030 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.030 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:28.030 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451237+0000", 2024-05-08T08:54:28.030 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8320000000000002e-06 2024-05-08T08:54:28.030 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.030 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.031 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:28.031 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451243+0000", 2024-05-08T08:54:28.031 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6749999999999996e-06 2024-05-08T08:54:28.031 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.031 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.031 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:28.031 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451249+0000", 2024-05-08T08:54:28.031 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1229999999999997e-06 2024-05-08T08:54:28.031 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.031 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.031 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:28.031 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539412+0000", 2024-05-08T08:54:28.031 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088162668999999999 2024-05-08T08:54:28.031 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.031 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.032 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:28.032 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539505+0000", 2024-05-08T08:54:28.032 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3240999999999997e-05 2024-05-08T08:54:28.032 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.032 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.032 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:28.032 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539665+0000", 2024-05-08T08:54:28.032 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00015930100000000001 2024-05-08T08:54:28.032 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:28.032 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:28.032 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:28.032 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.032 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.032 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:972 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 196608~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:28.032 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451257+0000", 2024-05-08T08:54:28.032 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 78.531357006999997, 2024-05-08T08:54:28.033 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088719928000000003, 2024-05-08T08:54:28.033 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:28.033 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:28.033 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:28.033 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:28.033 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:28.033 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:28.033 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 972 2024-05-08T08:54:28.033 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.033 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:28.033 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.033 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:28.033 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451257+0000", 2024-05-08T08:54:28.033 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:28.034 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.034 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.034 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:28.034 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451257+0000", 2024-05-08T08:54:28.034 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:28.034 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.034 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.034 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:28.034 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451258+0000", 2024-05-08T08:54:28.034 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7999999999999996e-07 2024-05-08T08:54:28.034 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.034 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.034 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:28.035 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451260+0000", 2024-05-08T08:54:28.035 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4609999999999998e-06 2024-05-08T08:54:28.035 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.035 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.035 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:28.035 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451267+0000", 2024-05-08T08:54:28.035 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.81e-06 2024-05-08T08:54:28.035 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.035 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.035 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:28.035 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451273+0000", 2024-05-08T08:54:28.035 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0859999999999998e-06 2024-05-08T08:54:28.035 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.035 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.035 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:28.035 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539718+0000", 2024-05-08T08:54:28.036 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088445059000000006 2024-05-08T08:54:28.036 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.036 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.036 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:28.036 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539834+0000", 2024-05-08T08:54:28.036 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000116426 2024-05-08T08:54:28.036 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.036 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.036 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:28.036 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539977+0000", 2024-05-08T08:54:28.036 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014260600000000001 2024-05-08T08:54:28.036 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:28.036 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:28.036 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:28.036 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.037 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.037 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:973 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 688128~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:28.037 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:54:28.037 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 78.531334423000004, 2024-05-08T08:54:28.037 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088918496, 2024-05-08T08:54:28.037 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:28.037 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:28.037 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:28.037 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:28.037 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:28.037 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:28.037 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 973 2024-05-08T08:54:28.037 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.037 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:28.037 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.037 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:28.038 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:54:28.038 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:28.038 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.038 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.038 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:28.038 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:54:28.038 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:28.038 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.038 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.038 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:28.038 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:54:28.038 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9099999999999999e-07 2024-05-08T08:54:28.038 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.038 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.039 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:28.039 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451282+0000", 2024-05-08T08:54:28.039 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1550000000000001e-06 2024-05-08T08:54:28.039 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.039 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.039 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:28.039 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451288+0000", 2024-05-08T08:54:28.039 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8889999999999999e-06 2024-05-08T08:54:28.039 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.039 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.039 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:28.039 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451294+0000", 2024-05-08T08:54:28.039 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0850000000000002e-06 2024-05-08T08:54:28.039 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.039 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.040 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:28.040 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540011+0000", 2024-05-08T08:54:28.040 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088717314000000005 2024-05-08T08:54:28.040 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.040 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.040 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:28.040 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540089+0000", 2024-05-08T08:54:28.040 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7438e-05 2024-05-08T08:54:28.040 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.040 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.040 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:28.040 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540198+0000", 2024-05-08T08:54:28.040 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000109224 2024-05-08T08:54:28.040 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:28.040 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:28.041 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:28.041 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.041 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.041 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:974 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 1179648~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:28.041 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451301+0000", 2024-05-08T08:54:28.041 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 78.531312790000001, 2024-05-08T08:54:28.041 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089105078000000004, 2024-05-08T08:54:28.041 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:28.041 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:28.041 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:28.041 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:28.041 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:28.041 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:28.041 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 974 2024-05-08T08:54:28.042 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.042 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:28.042 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.042 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:28.042 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451301+0000", 2024-05-08T08:54:28.042 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:28.042 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.042 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.042 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:28.042 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451301+0000", 2024-05-08T08:54:28.042 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:28.042 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.042 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.042 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:28.042 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451302+0000", 2024-05-08T08:54:28.042 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7800000000000002e-07 2024-05-08T08:54:28.043 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.043 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.043 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:28.043 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451304+0000", 2024-05-08T08:54:28.043 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.65e-06 2024-05-08T08:54:28.043 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.043 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.043 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:28.043 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451311+0000", 2024-05-08T08:54:28.043 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.472e-06 2024-05-08T08:54:28.043 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.043 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.043 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:28.043 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451316+0000", 2024-05-08T08:54:28.043 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9189999999999998e-06 2024-05-08T08:54:28.044 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.044 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.044 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:28.044 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540234+0000", 2024-05-08T08:54:28.044 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088917752000000003 2024-05-08T08:54:28.044 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.044 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.044 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:28.044 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540308+0000", 2024-05-08T08:54:28.044 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4758000000000005e-05 2024-05-08T08:54:28.044 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.044 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.044 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:28.044 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540406+0000", 2024-05-08T08:54:28.044 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8048999999999996e-05 2024-05-08T08:54:28.044 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:28.045 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:28.045 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:28.045 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.045 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.045 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:975 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 1671168~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:28.045 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:54:28.045 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 78.531270460000002, 2024-05-08T08:54:28.045 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089294139999999994, 2024-05-08T08:54:28.045 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:28.045 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:28.045 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:28.045 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:28.045 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:28.045 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:28.045 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 975 2024-05-08T08:54:28.046 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.046 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:28.046 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.046 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:28.046 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:54:28.046 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:28.046 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.046 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.046 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:28.046 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:54:28.046 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:28.046 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.046 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.046 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:28.047 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:54:28.047 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1299999999999999e-07 2024-05-08T08:54:28.047 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.047 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.047 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:28.047 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451348+0000", 2024-05-08T08:54:28.047 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.484e-06 2024-05-08T08:54:28.047 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.047 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.047 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:28.047 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451356+0000", 2024-05-08T08:54:28.047 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8499999999999994e-06 2024-05-08T08:54:28.047 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.047 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.048 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:28.048 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451366+0000", 2024-05-08T08:54:28.048 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.9850000000000001e-06 2024-05-08T08:54:28.048 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.048 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.048 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:28.048 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540448+0000", 2024-05-08T08:54:28.048 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089082254 2024-05-08T08:54:28.048 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.048 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.048 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:28.048 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540528+0000", 2024-05-08T08:54:28.048 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0234999999999993e-05 2024-05-08T08:54:28.048 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.048 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.049 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:28.049 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540638+0000", 2024-05-08T08:54:28.049 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010951900000000001 2024-05-08T08:54:28.049 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:28.049 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:28.049 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:28.049 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.049 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.049 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:976 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 2162688~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:28.049 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451375+0000", 2024-05-08T08:54:28.049 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 78.531238712999993, 2024-05-08T08:54:28.049 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089472745000000006, 2024-05-08T08:54:28.049 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:28.049 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:28.049 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:28.050 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:28.050 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:28.050 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:28.050 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 976 2024-05-08T08:54:28.050 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.050 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:28.050 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.050 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:28.050 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451375+0000", 2024-05-08T08:54:28.050 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:28.050 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.050 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.050 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:28.050 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451375+0000", 2024-05-08T08:54:28.050 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:28.050 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.051 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.051 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:28.051 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451376+0000", 2024-05-08T08:54:28.051 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2399999999999998e-07 2024-05-08T08:54:28.051 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.051 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.051 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:28.051 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451379+0000", 2024-05-08T08:54:28.051 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6359999999999998e-06 2024-05-08T08:54:28.051 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.051 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.051 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:28.051 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451387+0000", 2024-05-08T08:54:28.051 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9929999999999995e-06 2024-05-08T08:54:28.051 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.052 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.052 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:28.052 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451393+0000", 2024-05-08T08:54:28.052 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4699999999999999e-06 2024-05-08T08:54:28.052 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.052 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.052 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:28.052 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540677+0000", 2024-05-08T08:54:28.052 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089283504999999999 2024-05-08T08:54:28.052 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.052 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.052 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:28.052 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540743+0000", 2024-05-08T08:54:28.052 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6303999999999994e-05 2024-05-08T08:54:28.052 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.052 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.053 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:28.053 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540848+0000", 2024-05-08T08:54:28.053 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000105213 2024-05-08T08:54:28.053 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:28.053 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:28.053 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:28.053 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.053 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.053 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:977 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 2654208~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:28.053 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:54:28.053 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 78.531213281000007, 2024-05-08T08:54:28.053 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089698473000000001, 2024-05-08T08:54:28.053 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:28.053 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:28.053 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:28.054 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:28.054 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:28.054 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:28.054 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 977 2024-05-08T08:54:28.054 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.054 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:28.054 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.054 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:28.054 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:54:28.054 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:28.054 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.054 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.054 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:28.054 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:54:28.055 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:28.055 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.055 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.057 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:28.057 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:54:28.057 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2899999999999999e-07 2024-05-08T08:54:28.057 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.057 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.057 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:28.057 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451404+0000", 2024-05-08T08:54:28.058 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6649999999999999e-06 2024-05-08T08:54:28.058 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.058 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.058 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:28.058 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451410+0000", 2024-05-08T08:54:28.058 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3269999999999997e-06 2024-05-08T08:54:28.058 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.058 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.058 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:28.058 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451415+0000", 2024-05-08T08:54:28.058 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9529999999999998e-06 2024-05-08T08:54:28.058 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.058 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.058 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:28.059 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540912+0000", 2024-05-08T08:54:28.059 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089496950000000006 2024-05-08T08:54:28.059 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.059 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.059 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:28.059 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541000+0000", 2024-05-08T08:54:28.059 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7429000000000006e-05 2024-05-08T08:54:28.059 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.059 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.059 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:28.059 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541099+0000", 2024-05-08T08:54:28.059 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.9720000000000001e-05 2024-05-08T08:54:28.059 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:28.059 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:28.059 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:28.060 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.060 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.060 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:978 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 3145728~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:28.060 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:54:28.060 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 78.531192073, 2024-05-08T08:54:28.060 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089900635000000007, 2024-05-08T08:54:28.060 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:28.060 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:28.060 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:28.060 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:28.060 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:28.060 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:28.060 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 978 2024-05-08T08:54:28.060 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.060 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:28.061 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.061 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:28.061 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:54:28.061 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:28.061 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.061 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.061 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:28.061 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:54:28.061 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:28.061 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.061 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.061 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:28.061 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:54:28.061 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9400000000000001e-07 2024-05-08T08:54:28.061 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.061 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.062 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:28.062 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451425+0000", 2024-05-08T08:54:28.062 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2460000000000002e-06 2024-05-08T08:54:28.062 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.062 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.062 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:28.062 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451431+0000", 2024-05-08T08:54:28.062 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2419999999999997e-06 2024-05-08T08:54:28.062 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.062 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.062 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:28.062 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451436+0000", 2024-05-08T08:54:28.062 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.887e-06 2024-05-08T08:54:28.062 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.062 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.063 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:28.063 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541152+0000", 2024-05-08T08:54:28.063 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089716194999999999 2024-05-08T08:54:28.063 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.063 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.063 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:28.063 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541224+0000", 2024-05-08T08:54:28.063 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2416000000000006e-05 2024-05-08T08:54:28.063 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.063 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.063 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:28.063 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541323+0000", 2024-05-08T08:54:28.063 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8255e-05 2024-05-08T08:54:28.063 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:28.064 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:28.064 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:28.064 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.064 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.064 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:979 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 3637248~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:28.064 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451462+0000", 2024-05-08T08:54:28.064 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 78.531151624000003, 2024-05-08T08:54:28.064 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.090036441999999994, 2024-05-08T08:54:28.064 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:28.064 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:28.064 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:28.064 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:28.064 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:28.064 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:28.064 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 979 2024-05-08T08:54:28.064 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.065 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:28.065 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.065 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:28.065 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451462+0000", 2024-05-08T08:54:28.065 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:28.065 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.065 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.065 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:28.065 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451462+0000", 2024-05-08T08:54:28.065 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:28.065 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.065 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.065 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:28.065 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451463+0000", 2024-05-08T08:54:28.065 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4300000000000002e-07 2024-05-08T08:54:28.065 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.066 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.066 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:28.066 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451471+0000", 2024-05-08T08:54:28.066 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0800000000000006e-06 2024-05-08T08:54:28.066 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.066 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.066 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:28.066 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451481+0000", 2024-05-08T08:54:28.066 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.38e-06 2024-05-08T08:54:28.066 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.066 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.066 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:28.066 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451487+0000", 2024-05-08T08:54:28.066 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4590000000000003e-06 2024-05-08T08:54:28.066 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.066 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.067 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:28.067 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541350+0000", 2024-05-08T08:54:28.067 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089862599000000001 2024-05-08T08:54:28.067 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.067 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.067 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:28.067 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541407+0000", 2024-05-08T08:54:28.067 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7667999999999997e-05 2024-05-08T08:54:28.067 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.067 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.067 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:28.067 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541499+0000", 2024-05-08T08:54:28.067 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1513000000000003e-05 2024-05-08T08:54:28.067 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:28.067 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:28.067 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:28.068 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.068 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.068 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:980 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 4128768~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:28.068 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451496+0000", 2024-05-08T08:54:28.068 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 78.531117867999995, 2024-05-08T08:54:28.068 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.090258985, 2024-05-08T08:54:28.068 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:28.068 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:28.068 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:28.068 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:28.068 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:28.068 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:28.068 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 980 2024-05-08T08:54:28.068 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.068 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:28.068 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.069 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:28.069 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451496+0000", 2024-05-08T08:54:28.069 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:28.069 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.069 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.069 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:28.069 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451496+0000", 2024-05-08T08:54:28.069 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:28.069 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.069 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.069 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:28.069 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451497+0000", 2024-05-08T08:54:28.069 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7599999999999997e-07 2024-05-08T08:54:28.069 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.069 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.069 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:28.070 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451500+0000", 2024-05-08T08:54:28.070 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7930000000000002e-06 2024-05-08T08:54:28.070 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.070 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.070 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:28.070 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451506+0000", 2024-05-08T08:54:28.070 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6980000000000001e-06 2024-05-08T08:54:28.070 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.070 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.070 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:28.070 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451512+0000", 2024-05-08T08:54:28.070 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2490000000000001e-06 2024-05-08T08:54:28.070 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.070 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.071 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:28.071 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541550+0000", 2024-05-08T08:54:28.071 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.090037961999999999 2024-05-08T08:54:28.071 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.071 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.071 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:28.071 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541647+0000", 2024-05-08T08:54:28.071 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6924999999999998e-05 2024-05-08T08:54:28.071 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:28.071 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:28.071 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:28.071 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541755+0000", 2024-05-08T08:54:28.071 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000108682 2024-05-08T08:54:28.071 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:28.071 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:28.072 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:28.072 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:28.072 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:28.072 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:28.073 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:54:28.074 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:54:28.074 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:54:28.074 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:28.076+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:28.076 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:54:28.076 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:54:28.093 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:54:28.093 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:54:28.110 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:54:28.110 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:54:28.128 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:54:28.128 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:54:28.145 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:54:28.145 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:54:28.181 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:28.181 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:28.190 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:54:28.197 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:28.200+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:28.275 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:28.278+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:28.294 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:28.294 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:28.303 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:54:28.376 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:28.379+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:28.407 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:28.407 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:28.416 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:54:28.476 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:28.479+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:28.562 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:28.562 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:28.571 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:54:28.577 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:28.580+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:28.676 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:28.676 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:28.678 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:28.681+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:28.685 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:54:28.770 INFO:teuthology.orchestra.run.smithi059.stderr:set 3.7 pg_upmap mapping to [4,3] 2024-05-08T08:54:28.779 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:28.782+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:28.836 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:28.836 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:28.846 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:54:28.880 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:28.883+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:28.981 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:28.984+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:28.993 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:28.993 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:29.002 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:54:29.086 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:29.089+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:29.148 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:29.148 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:29.156 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:54:29.183 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:29.186+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:29.284 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:29.287+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:29.301 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:29.302 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:29.310 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:54:29.384 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:29.387+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:29.427 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:29.427 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:29.436 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:54:29.520 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:29.523+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:29.586 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:29.588+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:29.586 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:29.586 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:29.595 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:54:29.686 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:29.690+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:29.745 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:29.745 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:29.754 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:54:29.779 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-05-08T08:54:29.787 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:29.790+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:29.875 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:29.875 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:29.885 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:54:29.887 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:29.890+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:29.988 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:29.991+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:29.999 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:30.000 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:30.009 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:54:30.088 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:30.091+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:30.119 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:30.119 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:30.128 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:54:30.157 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T08:54:30.190 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:30.193+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:30.280 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:30.280 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:30.290 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:54:30.291 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:30.293+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:30.295 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:54:30.295 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:54:30.313 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:54:30.313 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:54:30.330 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:54:30.331 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:54:30.349 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:54:30.349 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:54:30.366 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:54:30.367 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:54:30.385 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:54:30.385 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:54:30.392 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:30.395+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:30.400 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:30.401 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:54:30.401 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:54:30.401 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:54:30.401 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:30.409 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:54:30.493 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:30.496+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:30.560 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:30.560 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:54:30.560 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:54:30.561 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:54:30.561 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.561 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:437 3.17 3:ea93a512:::rbd_data.1140100c9703.0000000000000077:head [set-alloc-hint object_size 2097152 write_size 2097152,write 1170432~1024] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e63)", 2024-05-08T08:54:30.561 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:08.390804+0000", 2024-05-08T08:54:30.561 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 82.171611748999993, 2024-05-08T08:54:30.561 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0406068690000001, 2024-05-08T08:54:30.561 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:30.561 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:30.561 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:54:30.561 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:30.561 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:30.561 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:30.561 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 437 2024-05-08T08:54:30.561 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.561 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:30.562 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.562 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:30.562 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390804+0000", 2024-05-08T08:54:30.562 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:30.562 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.562 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.562 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:30.562 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390804+0000", 2024-05-08T08:54:30.562 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:30.562 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.562 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.562 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:30.562 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390806+0000", 2024-05-08T08:54:30.562 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.666e-06 2024-05-08T08:54:30.562 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.562 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.563 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:30.563 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390818+0000", 2024-05-08T08:54:30.563 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2187999999999999e-05 2024-05-08T08:54:30.563 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.563 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.563 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:30.563 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390834+0000", 2024-05-08T08:54:30.563 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6191000000000001e-05 2024-05-08T08:54:30.563 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.563 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.563 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:30.563 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390845+0000", 2024-05-08T08:54:30.563 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0831e-05 2024-05-08T08:54:30.563 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.563 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.563 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:30.564 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390922+0000", 2024-05-08T08:54:30.564 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7253e-05 2024-05-08T08:54:30.564 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.564 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.564 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op must wait for map", 2024-05-08T08:54:30.564 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390930+0000", 2024-05-08T08:54:30.564 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8259999999999993e-06 2024-05-08T08:54:30.564 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.564 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.564 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:30.564 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392155+0000", 2024-05-08T08:54:30.564 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0012256649999999999 2024-05-08T08:54:30.564 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.564 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.564 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:54:30.565 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392165+0000", 2024-05-08T08:54:30.565 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5799999999999998e-06 2024-05-08T08:54:30.565 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.565 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.565 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:30.565 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.429596+0000", 2024-05-08T08:54:30.565 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0374313639999999 2024-05-08T08:54:30.565 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.565 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.565 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:30.565 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.429728+0000", 2024-05-08T08:54:30.565 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000131552 2024-05-08T08:54:30.565 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.565 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.565 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 0,1", 2024-05-08T08:54:30.565 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.429860+0000", 2024-05-08T08:54:30.565 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000132137 2024-05-08T08:54:30.566 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.566 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.566 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:54:30.566 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.430811+0000", 2024-05-08T08:54:30.566 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00095040200000000004 2024-05-08T08:54:30.566 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.566 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.566 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:54:30.566 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431126+0000", 2024-05-08T08:54:30.566 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000315006 2024-05-08T08:54:30.566 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.566 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.566 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:54:30.566 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431272+0000", 2024-05-08T08:54:30.566 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000146487 2024-05-08T08:54:30.567 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.567 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.567 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:54:30.567 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431315+0000", 2024-05-08T08:54:30.567 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2697000000000001e-05 2024-05-08T08:54:30.567 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.567 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.567 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:30.567 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431411+0000", 2024-05-08T08:54:30.567 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6024000000000001e-05 2024-05-08T08:54:30.567 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:30.567 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:30.567 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:30.567 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.567 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.567 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:439 3.17 3:ea93a512:::rbd_data.1140100c9703.0000000000000077:head [read 1163264~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-05-08T08:54:30.567 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:08.390871+0000", 2024-05-08T08:54:30.568 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 82.171544237999996, 2024-05-08T08:54:30.568 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0408790590000001, 2024-05-08T08:54:30.568 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:30.568 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:30.568 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:30.568 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:30.568 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:30.568 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:30.568 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 439 2024-05-08T08:54:30.568 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.568 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:30.568 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.568 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:30.568 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390871+0000", 2024-05-08T08:54:30.568 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:30.568 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.569 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.569 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:30.569 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390871+0000", 2024-05-08T08:54:30.569 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:30.569 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.569 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.569 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:30.569 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390872+0000", 2024-05-08T08:54:30.569 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6300000000000005e-07 2024-05-08T08:54:30.569 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.569 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.569 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:30.569 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390875+0000", 2024-05-08T08:54:30.569 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3459999999999998e-06 2024-05-08T08:54:30.569 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.569 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.570 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:30.570 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390891+0000", 2024-05-08T08:54:30.570 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5255e-05 2024-05-08T08:54:30.570 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.570 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.570 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:30.570 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390897+0000", 2024-05-08T08:54:30.570 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2360000000000001e-06 2024-05-08T08:54:30.570 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.570 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.570 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:30.570 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390995+0000", 2024-05-08T08:54:30.570 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8522000000000003e-05 2024-05-08T08:54:30.570 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.570 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.570 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting_for_map not empty", 2024-05-08T08:54:30.571 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.391005+0000", 2024-05-08T08:54:30.571 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1460000000000002e-06 2024-05-08T08:54:30.571 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.571 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.571 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:30.571 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392191+0000", 2024-05-08T08:54:30.571 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0011860270000000001 2024-05-08T08:54:30.571 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.571 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.571 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:54:30.571 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392197+0000", 2024-05-08T08:54:30.571 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4289999999999998e-06 2024-05-08T08:54:30.572 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.572 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.572 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:30.572 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.430596+0000", 2024-05-08T08:54:30.572 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0383989090000001 2024-05-08T08:54:30.572 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.572 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.572 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for rw locks", 2024-05-08T08:54:30.572 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.430676+0000", 2024-05-08T08:54:30.572 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9781000000000001e-05 2024-05-08T08:54:30.572 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.572 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.572 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:30.572 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431460+0000", 2024-05-08T08:54:30.572 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00078427300000000004 2024-05-08T08:54:30.572 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.573 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.573 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:30.573 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431540+0000", 2024-05-08T08:54:30.573 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0224999999999998e-05 2024-05-08T08:54:30.573 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.573 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.573 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:30.573 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431750+0000", 2024-05-08T08:54:30.573 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00021024699999999999 2024-05-08T08:54:30.573 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:30.573 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:30.573 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:30.573 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.573 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.574 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:945 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 4096000~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:30.574 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:54:30.574 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 81.111394670999999, 2024-05-08T08:54:30.574 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086774741000000002, 2024-05-08T08:54:30.574 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:30.574 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:30.574 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:30.574 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:30.574 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:30.574 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:30.574 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 945 2024-05-08T08:54:30.574 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.574 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:30.574 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.575 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:30.575 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:54:30.575 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:30.575 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.575 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.575 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:30.575 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:54:30.575 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:30.575 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.575 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.575 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:30.575 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:54:30.575 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3499999999999996e-07 2024-05-08T08:54:30.575 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.575 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.576 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:30.576 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451024+0000", 2024-05-08T08:54:30.576 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.627e-06 2024-05-08T08:54:30.576 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.576 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.576 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:30.576 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451037+0000", 2024-05-08T08:54:30.576 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2431e-05 2024-05-08T08:54:30.576 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.576 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.576 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:30.576 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451042+0000", 2024-05-08T08:54:30.576 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8409999999999996e-06 2024-05-08T08:54:30.576 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.576 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.576 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:30.576 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537608+0000", 2024-05-08T08:54:30.577 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086565243 2024-05-08T08:54:30.577 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.577 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.577 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:30.577 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537687+0000", 2024-05-08T08:54:30.577 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9519999999999998e-05 2024-05-08T08:54:30.577 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.577 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.577 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:30.577 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537796+0000", 2024-05-08T08:54:30.577 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000108544 2024-05-08T08:54:30.577 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:30.577 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:30.577 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:30.577 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.577 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.577 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:964 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 458752~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:30.578 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451051+0000", 2024-05-08T08:54:30.578 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 81.111364498, 2024-05-08T08:54:30.578 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086990490000000004, 2024-05-08T08:54:30.578 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:30.578 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:30.578 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:30.578 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:30.578 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:30.578 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:30.578 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 964 2024-05-08T08:54:30.578 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.578 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:30.578 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.578 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:30.578 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451051+0000", 2024-05-08T08:54:30.579 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:30.579 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.579 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.579 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:30.579 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451051+0000", 2024-05-08T08:54:30.579 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:30.579 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.579 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.579 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:30.579 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451052+0000", 2024-05-08T08:54:30.579 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7299999999999996e-07 2024-05-08T08:54:30.579 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.579 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.579 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:30.579 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451055+0000", 2024-05-08T08:54:30.580 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.2770000000000001e-06 2024-05-08T08:54:30.580 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.580 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.580 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:30.580 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451061+0000", 2024-05-08T08:54:30.580 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4060000000000002e-06 2024-05-08T08:54:30.580 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.580 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.580 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:30.580 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451066+0000", 2024-05-08T08:54:30.580 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0839999999999999e-06 2024-05-08T08:54:30.580 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.580 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.580 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:30.580 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537839+0000", 2024-05-08T08:54:30.580 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086772962999999995 2024-05-08T08:54:30.580 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.581 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.581 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:30.581 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537934+0000", 2024-05-08T08:54:30.581 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4742999999999997e-05 2024-05-08T08:54:30.581 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.581 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.581 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:30.581 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538042+0000", 2024-05-08T08:54:30.581 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000107444 2024-05-08T08:54:30.581 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:30.581 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:30.581 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:30.581 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.581 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.581 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:965 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 950272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:30.581 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451075+0000", 2024-05-08T08:54:30.582 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 81.111340436000006, 2024-05-08T08:54:30.582 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087230196999999995, 2024-05-08T08:54:30.582 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:30.582 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:30.582 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:30.582 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:30.582 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:30.582 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:30.582 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 965 2024-05-08T08:54:30.582 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.582 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:30.582 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.582 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:30.582 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451075+0000", 2024-05-08T08:54:30.583 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:30.583 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.583 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.583 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:30.583 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451075+0000", 2024-05-08T08:54:30.583 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:30.583 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.583 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.583 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:30.583 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451076+0000", 2024-05-08T08:54:30.583 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5000000000000002e-07 2024-05-08T08:54:30.583 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.583 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.583 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:30.583 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451078+0000", 2024-05-08T08:54:30.583 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.5639999999999999e-06 2024-05-08T08:54:30.583 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.584 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.584 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:30.584 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451085+0000", 2024-05-08T08:54:30.584 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5640000000000002e-06 2024-05-08T08:54:30.584 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.584 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.584 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:30.584 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451090+0000", 2024-05-08T08:54:30.584 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1699999999999996e-06 2024-05-08T08:54:30.584 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.584 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.584 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:30.584 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538106+0000", 2024-05-08T08:54:30.584 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087015888999999999 2024-05-08T08:54:30.584 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.584 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.585 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:30.585 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538190+0000", 2024-05-08T08:54:30.585 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3961999999999993e-05 2024-05-08T08:54:30.585 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.585 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.585 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:30.585 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538305+0000", 2024-05-08T08:54:30.585 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011529800000000001 2024-05-08T08:54:30.585 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:30.585 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:30.585 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:30.585 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.585 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.585 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:966 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 1441792~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:30.585 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:54:30.586 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 81.111317063000001, 2024-05-08T08:54:30.586 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087446323000000006, 2024-05-08T08:54:30.586 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:30.586 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:30.586 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:30.586 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:30.586 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:30.586 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:30.586 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 966 2024-05-08T08:54:30.586 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.586 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:30.586 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.586 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:30.586 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:54:30.586 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:30.586 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.587 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.587 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:30.587 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:54:30.587 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:30.587 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.587 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.587 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:30.587 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:54:30.587 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3700000000000001e-07 2024-05-08T08:54:30.587 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.587 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.587 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:30.587 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451101+0000", 2024-05-08T08:54:30.587 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2560000000000001e-06 2024-05-08T08:54:30.587 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.587 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.588 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:30.588 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451107+0000", 2024-05-08T08:54:30.588 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1070000000000001e-06 2024-05-08T08:54:30.588 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.588 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.588 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:30.588 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451112+0000", 2024-05-08T08:54:30.588 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1420000000000001e-06 2024-05-08T08:54:30.588 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.588 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.588 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:30.588 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538341+0000", 2024-05-08T08:54:30.588 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087228302999999993 2024-05-08T08:54:30.588 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.588 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.588 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:30.588 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538424+0000", 2024-05-08T08:54:30.589 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3497000000000004e-05 2024-05-08T08:54:30.589 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.589 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.589 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:30.589 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538545+0000", 2024-05-08T08:54:30.589 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000120581 2024-05-08T08:54:30.589 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:30.589 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:30.589 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:30.589 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.589 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.589 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:967 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:30.589 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451120+0000", 2024-05-08T08:54:30.589 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 81.111295588000004, 2024-05-08T08:54:30.589 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087632725999999994, 2024-05-08T08:54:30.589 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:30.590 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:30.590 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:30.590 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:30.590 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:30.590 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:30.590 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 967 2024-05-08T08:54:30.590 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.590 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:30.590 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.590 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:30.590 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451120+0000", 2024-05-08T08:54:30.590 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:30.590 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.590 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.590 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:30.590 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451120+0000", 2024-05-08T08:54:30.591 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:30.591 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.591 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.591 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:30.591 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451121+0000", 2024-05-08T08:54:30.591 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8800000000000002e-07 2024-05-08T08:54:30.591 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.591 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.591 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:30.591 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451130+0000", 2024-05-08T08:54:30.591 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2299999999999997e-06 2024-05-08T08:54:30.591 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.591 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.591 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:30.591 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451137+0000", 2024-05-08T08:54:30.592 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4669999999999998e-06 2024-05-08T08:54:30.592 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.592 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.592 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:30.592 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451143+0000", 2024-05-08T08:54:30.592 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2839999999999997e-06 2024-05-08T08:54:30.592 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.592 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.592 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:30.592 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538575+0000", 2024-05-08T08:54:30.592 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087432789999999996 2024-05-08T08:54:30.592 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.592 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.592 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:30.592 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538648+0000", 2024-05-08T08:54:30.593 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2877e-05 2024-05-08T08:54:30.593 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.593 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.593 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:30.593 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538753+0000", 2024-05-08T08:54:30.593 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010449 2024-05-08T08:54:30.593 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:30.593 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:30.593 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:30.593 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.593 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.593 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:968 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 2424832~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:30.593 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451152+0000", 2024-05-08T08:54:30.593 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 81.111263503000004, 2024-05-08T08:54:30.593 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087813325999999997, 2024-05-08T08:54:30.594 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:30.594 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:30.594 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:30.594 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:30.594 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:30.594 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:30.594 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 968 2024-05-08T08:54:30.594 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.594 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:30.594 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.594 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:30.594 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451152+0000", 2024-05-08T08:54:30.594 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:30.594 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.594 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.595 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:30.595 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451152+0000", 2024-05-08T08:54:30.595 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:30.595 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.595 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.595 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:30.595 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451153+0000", 2024-05-08T08:54:30.595 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3099999999999998e-07 2024-05-08T08:54:30.595 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.595 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.595 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:30.595 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451155+0000", 2024-05-08T08:54:30.595 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4760000000000001e-06 2024-05-08T08:54:30.595 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.595 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.595 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:30.595 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451162+0000", 2024-05-08T08:54:30.596 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5969999999999997e-06 2024-05-08T08:54:30.596 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.596 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.596 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:30.596 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451167+0000", 2024-05-08T08:54:30.596 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.977e-06 2024-05-08T08:54:30.596 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.596 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.596 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:30.596 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538781+0000", 2024-05-08T08:54:30.596 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087614846999999996 2024-05-08T08:54:30.596 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.596 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.596 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:30.596 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538857+0000", 2024-05-08T08:54:30.596 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5358000000000006e-05 2024-05-08T08:54:30.597 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.597 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.597 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:30.597 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538965+0000", 2024-05-08T08:54:30.597 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010864 2024-05-08T08:54:30.597 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:30.597 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:30.597 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:30.597 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.597 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.597 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:969 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 2916352~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:30.597 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:54:30.597 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 81.111241797999995, 2024-05-08T08:54:30.597 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087998980000000004, 2024-05-08T08:54:30.597 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:30.597 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:30.598 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:30.598 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:30.598 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:30.598 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:30.598 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 969 2024-05-08T08:54:30.598 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.598 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:30.598 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.598 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:30.598 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:54:30.598 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:30.598 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.598 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.598 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:30.599 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:54:30.599 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:30.599 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.599 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.599 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:30.599 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:54:30.599 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3599999999999999e-07 2024-05-08T08:54:30.599 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.599 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.599 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:30.599 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451177+0000", 2024-05-08T08:54:30.599 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4590000000000001e-06 2024-05-08T08:54:30.599 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.600 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.600 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:30.600 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451184+0000", 2024-05-08T08:54:30.600 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4109999999999999e-06 2024-05-08T08:54:30.600 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.600 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.600 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:30.600 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451191+0000", 2024-05-08T08:54:30.600 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7279999999999998e-06 2024-05-08T08:54:30.600 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.600 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.600 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:30.600 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538997+0000", 2024-05-08T08:54:30.600 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087806170000000003 2024-05-08T08:54:30.600 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.601 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.601 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:30.601 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539069+0000", 2024-05-08T08:54:30.601 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2373999999999995e-05 2024-05-08T08:54:30.601 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.601 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.601 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:30.601 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539173+0000", 2024-05-08T08:54:30.601 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000103402 2024-05-08T08:54:30.601 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:30.601 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:30.601 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:30.601 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.601 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.601 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:970 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 3407872~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:30.601 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451198+0000", 2024-05-08T08:54:30.601 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 81.111217201000002, 2024-05-08T08:54:30.602 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088183368999999998, 2024-05-08T08:54:30.602 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:30.602 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:30.602 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:30.602 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:30.602 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:30.602 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:30.602 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 970 2024-05-08T08:54:30.602 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.602 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:30.602 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.602 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:30.602 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451198+0000", 2024-05-08T08:54:30.602 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:30.603 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.603 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.603 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:30.603 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451198+0000", 2024-05-08T08:54:30.603 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:30.603 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.603 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.603 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:30.603 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451199+0000", 2024-05-08T08:54:30.603 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7400000000000003e-07 2024-05-08T08:54:30.603 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.603 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.603 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:30.603 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451211+0000", 2024-05-08T08:54:30.603 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2109999999999999e-05 2024-05-08T08:54:30.604 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.604 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.604 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:30.604 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451219+0000", 2024-05-08T08:54:30.604 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4170000000000003e-06 2024-05-08T08:54:30.604 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.604 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.604 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:30.604 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451224+0000", 2024-05-08T08:54:30.604 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3990000000000003e-06 2024-05-08T08:54:30.604 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.604 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.604 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:30.604 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539204+0000", 2024-05-08T08:54:30.604 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087980099000000006 2024-05-08T08:54:30.604 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.605 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.605 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:30.605 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539276+0000", 2024-05-08T08:54:30.605 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1513000000000005e-05 2024-05-08T08:54:30.605 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.605 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.605 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:30.605 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539382+0000", 2024-05-08T08:54:30.605 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010615699999999999 2024-05-08T08:54:30.605 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:30.605 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:30.605 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:30.605 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.605 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.605 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:971 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 3899392~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:30.606 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451233+0000", 2024-05-08T08:54:30.606 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 81.111182494000005, 2024-05-08T08:54:30.606 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088431525999999996, 2024-05-08T08:54:30.606 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:30.606 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:30.606 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:30.606 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:30.606 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:30.606 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:30.606 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 971 2024-05-08T08:54:30.606 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.606 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:30.606 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.606 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:30.606 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451233+0000", 2024-05-08T08:54:30.606 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:30.607 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.607 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.607 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:30.607 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451233+0000", 2024-05-08T08:54:30.607 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:30.607 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.607 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.607 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:30.607 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451234+0000", 2024-05-08T08:54:30.607 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8500000000000001e-07 2024-05-08T08:54:30.607 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.607 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.607 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:30.607 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451237+0000", 2024-05-08T08:54:30.607 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8320000000000002e-06 2024-05-08T08:54:30.607 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.608 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.608 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:30.608 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451243+0000", 2024-05-08T08:54:30.608 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6749999999999996e-06 2024-05-08T08:54:30.608 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.608 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.608 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:30.608 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451249+0000", 2024-05-08T08:54:30.608 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1229999999999997e-06 2024-05-08T08:54:30.608 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.608 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.608 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:30.608 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539412+0000", 2024-05-08T08:54:30.608 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088162668999999999 2024-05-08T08:54:30.608 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.608 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.609 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:30.609 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539505+0000", 2024-05-08T08:54:30.609 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3240999999999997e-05 2024-05-08T08:54:30.609 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.609 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.609 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:30.609 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539665+0000", 2024-05-08T08:54:30.609 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00015930100000000001 2024-05-08T08:54:30.609 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:30.609 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:30.609 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:30.609 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.609 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.609 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:972 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 196608~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:30.609 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451257+0000", 2024-05-08T08:54:30.609 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 81.111158548999995, 2024-05-08T08:54:30.609 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088719928000000003, 2024-05-08T08:54:30.609 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:30.610 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:30.610 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:30.610 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:30.610 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:30.610 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:30.610 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 972 2024-05-08T08:54:30.610 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.610 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:30.610 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.610 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:30.610 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451257+0000", 2024-05-08T08:54:30.610 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:30.610 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.610 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.611 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:30.611 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451257+0000", 2024-05-08T08:54:30.611 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:30.611 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.611 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.611 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:30.611 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451258+0000", 2024-05-08T08:54:30.611 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7999999999999996e-07 2024-05-08T08:54:30.611 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.611 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.611 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:30.611 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451260+0000", 2024-05-08T08:54:30.611 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4609999999999998e-06 2024-05-08T08:54:30.611 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.612 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.612 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:30.612 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451267+0000", 2024-05-08T08:54:30.612 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.81e-06 2024-05-08T08:54:30.612 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.612 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.612 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:30.612 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451273+0000", 2024-05-08T08:54:30.612 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0859999999999998e-06 2024-05-08T08:54:30.612 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.612 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.612 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:30.612 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539718+0000", 2024-05-08T08:54:30.612 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088445059000000006 2024-05-08T08:54:30.612 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.613 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.613 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:30.613 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539834+0000", 2024-05-08T08:54:30.613 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000116426 2024-05-08T08:54:30.613 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.613 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.613 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:30.613 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539977+0000", 2024-05-08T08:54:30.613 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014260600000000001 2024-05-08T08:54:30.613 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:30.613 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:30.613 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:30.613 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.613 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.613 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:973 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 688128~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:30.613 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:54:30.614 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 81.111135965000003, 2024-05-08T08:54:30.614 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088918496, 2024-05-08T08:54:30.614 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:30.614 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:30.614 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:30.614 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:30.614 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:30.614 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:30.614 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 973 2024-05-08T08:54:30.614 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.614 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:30.615 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.615 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:30.615 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:54:30.615 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:30.615 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.615 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.615 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:30.615 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:54:30.615 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:30.615 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.615 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.615 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:30.615 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:54:30.615 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9099999999999999e-07 2024-05-08T08:54:30.616 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.616 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.616 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:30.616 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451282+0000", 2024-05-08T08:54:30.616 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1550000000000001e-06 2024-05-08T08:54:30.616 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.616 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.616 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:30.616 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451288+0000", 2024-05-08T08:54:30.616 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8889999999999999e-06 2024-05-08T08:54:30.616 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.616 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.616 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:30.617 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451294+0000", 2024-05-08T08:54:30.617 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0850000000000002e-06 2024-05-08T08:54:30.617 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.617 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.617 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:30.617 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540011+0000", 2024-05-08T08:54:30.617 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088717314000000005 2024-05-08T08:54:30.617 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.617 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.617 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:30.617 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540089+0000", 2024-05-08T08:54:30.617 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7438e-05 2024-05-08T08:54:30.617 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.617 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.617 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:30.617 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540198+0000", 2024-05-08T08:54:30.618 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000109224 2024-05-08T08:54:30.618 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:30.618 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:30.618 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:30.618 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.618 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.618 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:974 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 1179648~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:30.618 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451301+0000", 2024-05-08T08:54:30.618 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 81.111114332, 2024-05-08T08:54:30.618 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089105078000000004, 2024-05-08T08:54:30.618 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:30.618 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:30.618 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:30.618 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:30.619 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:30.619 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:30.619 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 974 2024-05-08T08:54:30.619 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.619 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:30.619 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.619 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:30.619 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451301+0000", 2024-05-08T08:54:30.619 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:30.619 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.619 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.619 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:30.620 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451301+0000", 2024-05-08T08:54:30.620 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:30.620 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.620 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.620 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:30.620 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451302+0000", 2024-05-08T08:54:30.620 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7800000000000002e-07 2024-05-08T08:54:30.620 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.620 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.620 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:30.620 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451304+0000", 2024-05-08T08:54:30.620 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.65e-06 2024-05-08T08:54:30.620 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.620 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.620 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:30.621 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451311+0000", 2024-05-08T08:54:30.621 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.472e-06 2024-05-08T08:54:30.621 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.621 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.621 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:30.621 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451316+0000", 2024-05-08T08:54:30.621 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9189999999999998e-06 2024-05-08T08:54:30.621 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.621 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.621 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:30.621 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540234+0000", 2024-05-08T08:54:30.621 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088917752000000003 2024-05-08T08:54:30.621 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.622 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.622 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:30.622 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540308+0000", 2024-05-08T08:54:30.622 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4758000000000005e-05 2024-05-08T08:54:30.622 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.622 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.622 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:30.622 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540406+0000", 2024-05-08T08:54:30.622 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8048999999999996e-05 2024-05-08T08:54:30.622 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:30.622 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:30.622 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:30.622 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.622 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.623 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:975 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 1671168~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:30.623 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:54:30.623 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 81.111072002, 2024-05-08T08:54:30.623 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089294139999999994, 2024-05-08T08:54:30.623 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:30.623 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:30.623 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:30.623 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:30.623 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:30.623 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:30.623 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 975 2024-05-08T08:54:30.623 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.623 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:30.623 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.623 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:30.624 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:54:30.624 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:30.624 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.624 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.624 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:30.624 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:54:30.624 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:30.624 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.624 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.624 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:30.624 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:54:30.624 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1299999999999999e-07 2024-05-08T08:54:30.624 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.624 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.624 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:30.625 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451348+0000", 2024-05-08T08:54:30.625 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.484e-06 2024-05-08T08:54:30.625 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.625 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.625 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:30.625 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451356+0000", 2024-05-08T08:54:30.625 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8499999999999994e-06 2024-05-08T08:54:30.625 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.625 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.625 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:30.625 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451366+0000", 2024-05-08T08:54:30.625 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.9850000000000001e-06 2024-05-08T08:54:30.625 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.625 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.626 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:30.626 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540448+0000", 2024-05-08T08:54:30.626 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089082254 2024-05-08T08:54:30.626 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.626 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.626 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:30.626 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540528+0000", 2024-05-08T08:54:30.626 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0234999999999993e-05 2024-05-08T08:54:30.626 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.627 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.627 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:30.627 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540638+0000", 2024-05-08T08:54:30.627 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010951900000000001 2024-05-08T08:54:30.627 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:30.627 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:30.627 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:30.627 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.627 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.627 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:976 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 2162688~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:30.627 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451375+0000", 2024-05-08T08:54:30.627 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 81.111040255000006, 2024-05-08T08:54:30.627 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089472745000000006, 2024-05-08T08:54:30.627 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:30.628 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:30.628 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:30.628 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:30.628 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:30.628 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:30.628 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 976 2024-05-08T08:54:30.628 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.628 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:30.628 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.628 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:30.628 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451375+0000", 2024-05-08T08:54:30.628 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:30.628 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.629 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.629 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:30.629 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451375+0000", 2024-05-08T08:54:30.629 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:30.629 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.629 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.629 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:30.629 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451376+0000", 2024-05-08T08:54:30.629 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2399999999999998e-07 2024-05-08T08:54:30.629 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.629 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.629 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:30.629 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451379+0000", 2024-05-08T08:54:30.629 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6359999999999998e-06 2024-05-08T08:54:30.629 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.629 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.630 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:30.630 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451387+0000", 2024-05-08T08:54:30.630 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9929999999999995e-06 2024-05-08T08:54:30.630 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.630 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.630 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:30.630 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451393+0000", 2024-05-08T08:54:30.630 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4699999999999999e-06 2024-05-08T08:54:30.630 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.630 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.630 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:30.630 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540677+0000", 2024-05-08T08:54:30.631 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089283504999999999 2024-05-08T08:54:30.631 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.631 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.631 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:30.631 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540743+0000", 2024-05-08T08:54:30.631 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6303999999999994e-05 2024-05-08T08:54:30.631 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.631 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.631 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:30.631 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540848+0000", 2024-05-08T08:54:30.631 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000105213 2024-05-08T08:54:30.631 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:30.631 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:30.631 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:30.631 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.632 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.632 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:977 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 2654208~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:30.632 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:54:30.632 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 81.111014823000005, 2024-05-08T08:54:30.632 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089698473000000001, 2024-05-08T08:54:30.632 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:30.632 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:30.632 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:30.632 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:30.632 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:30.632 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:30.632 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 977 2024-05-08T08:54:30.632 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.632 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:30.632 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.632 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:30.633 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:54:30.633 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:30.633 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.633 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.633 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:30.633 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:54:30.633 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:30.633 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.633 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.633 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:30.633 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:54:30.633 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2899999999999999e-07 2024-05-08T08:54:30.633 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.633 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.633 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:30.633 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451404+0000", 2024-05-08T08:54:30.634 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6649999999999999e-06 2024-05-08T08:54:30.634 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.634 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.634 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:30.634 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451410+0000", 2024-05-08T08:54:30.634 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3269999999999997e-06 2024-05-08T08:54:30.634 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.634 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.634 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:30.634 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451415+0000", 2024-05-08T08:54:30.634 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9529999999999998e-06 2024-05-08T08:54:30.634 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.634 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.634 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:30.634 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540912+0000", 2024-05-08T08:54:30.635 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089496950000000006 2024-05-08T08:54:30.635 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.635 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.635 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:30.635 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541000+0000", 2024-05-08T08:54:30.635 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7429000000000006e-05 2024-05-08T08:54:30.635 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.635 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.635 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:30.635 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541099+0000", 2024-05-08T08:54:30.635 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.9720000000000001e-05 2024-05-08T08:54:30.635 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:30.635 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:30.635 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:30.635 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.635 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.636 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:978 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 3145728~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:30.636 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:54:30.636 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 81.110993614999998, 2024-05-08T08:54:30.636 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089900635000000007, 2024-05-08T08:54:30.636 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:30.636 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:30.636 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:30.636 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:30.636 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:30.636 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:30.636 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 978 2024-05-08T08:54:30.636 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.637 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:30.637 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.637 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:30.637 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:54:30.637 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:30.637 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.637 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.637 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:30.637 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:54:30.637 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:30.637 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.637 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.637 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:30.637 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:54:30.638 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9400000000000001e-07 2024-05-08T08:54:30.638 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.638 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.638 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:30.638 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451425+0000", 2024-05-08T08:54:30.638 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2460000000000002e-06 2024-05-08T08:54:30.638 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.638 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.638 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:30.638 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451431+0000", 2024-05-08T08:54:30.638 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2419999999999997e-06 2024-05-08T08:54:30.638 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.638 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.638 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:30.638 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451436+0000", 2024-05-08T08:54:30.639 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.887e-06 2024-05-08T08:54:30.639 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.639 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.639 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:30.639 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541152+0000", 2024-05-08T08:54:30.639 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089716194999999999 2024-05-08T08:54:30.639 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.639 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.639 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:30.639 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541224+0000", 2024-05-08T08:54:30.639 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2416000000000006e-05 2024-05-08T08:54:30.639 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.639 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.639 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:30.639 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541323+0000", 2024-05-08T08:54:30.640 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8255e-05 2024-05-08T08:54:30.640 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:30.640 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:30.640 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:30.640 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.640 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.640 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:979 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 3637248~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:30.640 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451462+0000", 2024-05-08T08:54:30.640 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 81.110953166000002, 2024-05-08T08:54:30.640 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.090036441999999994, 2024-05-08T08:54:30.640 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:30.640 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:30.640 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:30.640 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:30.640 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:30.641 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:30.641 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 979 2024-05-08T08:54:30.641 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.641 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:30.641 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.641 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:30.641 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451462+0000", 2024-05-08T08:54:30.641 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:30.641 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.641 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.641 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:30.641 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451462+0000", 2024-05-08T08:54:30.641 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:30.641 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.641 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.642 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:30.642 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451463+0000", 2024-05-08T08:54:30.642 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4300000000000002e-07 2024-05-08T08:54:30.642 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.642 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.642 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:30.642 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451471+0000", 2024-05-08T08:54:30.642 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0800000000000006e-06 2024-05-08T08:54:30.642 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.642 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.642 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:30.643 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451481+0000", 2024-05-08T08:54:30.643 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.38e-06 2024-05-08T08:54:30.643 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.643 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.643 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:30.643 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451487+0000", 2024-05-08T08:54:30.643 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4590000000000003e-06 2024-05-08T08:54:30.643 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.643 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.643 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:30.643 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541350+0000", 2024-05-08T08:54:30.643 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089862599000000001 2024-05-08T08:54:30.643 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.644 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.644 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:30.644 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541407+0000", 2024-05-08T08:54:30.644 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7667999999999997e-05 2024-05-08T08:54:30.644 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.644 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.644 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:30.644 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541499+0000", 2024-05-08T08:54:30.644 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1513000000000003e-05 2024-05-08T08:54:30.644 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:30.644 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:30.644 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:30.644 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.644 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.644 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:980 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 4128768~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:30.644 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451496+0000", 2024-05-08T08:54:30.645 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 81.110919409999994, 2024-05-08T08:54:30.645 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.090258985, 2024-05-08T08:54:30.645 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:30.645 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:30.645 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:30.645 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:30.645 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:30.645 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:30.645 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 980 2024-05-08T08:54:30.645 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.645 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:30.645 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.645 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:30.645 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451496+0000", 2024-05-08T08:54:30.645 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:30.646 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.646 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.646 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:30.646 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451496+0000", 2024-05-08T08:54:30.646 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:30.646 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.646 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.646 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:30.646 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451497+0000", 2024-05-08T08:54:30.646 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7599999999999997e-07 2024-05-08T08:54:30.647 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.647 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.647 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:30.647 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451500+0000", 2024-05-08T08:54:30.647 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7930000000000002e-06 2024-05-08T08:54:30.647 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.647 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.648 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:30.648 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451506+0000", 2024-05-08T08:54:30.648 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6980000000000001e-06 2024-05-08T08:54:30.648 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.648 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.648 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:30.648 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451512+0000", 2024-05-08T08:54:30.649 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2490000000000001e-06 2024-05-08T08:54:30.649 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.649 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.649 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:30.649 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541550+0000", 2024-05-08T08:54:30.649 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.090037961999999999 2024-05-08T08:54:30.649 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.650 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.650 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:30.650 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541647+0000", 2024-05-08T08:54:30.650 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6924999999999998e-05 2024-05-08T08:54:30.650 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:30.650 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:30.650 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:30.650 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541755+0000", 2024-05-08T08:54:30.650 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000108682 2024-05-08T08:54:30.650 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:30.650 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:30.650 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:30.651 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:30.651 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:30.651 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:30.652 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:54:30.653 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:30.655+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:30.753 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:30.756+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:30.771 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:30.771 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:54:30.771 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:54:30.771 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:30.774 INFO:teuthology.orchestra.run.smithi059.stderr:noscrub is unset 2024-05-08T08:54:30.782 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:54:30.854 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:30.856+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:30.940 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:30.940 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:54:30.940 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:54:30.940 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:54:30.940 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:30.949 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:54:30.955 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:30.957+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:31.055 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:31.058+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:31.072 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:31.072 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:54:31.072 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:54:31.072 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:54:31.072 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.072 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4621.0:9089 3.1f e33/18 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:head v 33'117, mlcod=31'115)", 2024-05-08T08:54:31.072 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:54:31.072 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 118.554060402, 2024-05-08T08:54:31.072 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.111244524, 2024-05-08T08:54:31.072 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:31.072 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:31.072 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:54:31.073 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:31.073 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.073 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:31.073 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:54:31.073 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:31.073 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.073 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.073 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:31.073 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:54:31.073 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:31.073 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.073 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.073 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:31.073 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:54:31.074 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5900000000000001e-07 2024-05-08T08:54:31.074 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.074 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.074 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:31.074 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539600+0000", 2024-05-08T08:54:31.074 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.019981364000000001 2024-05-08T08:54:31.074 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.074 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.074 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:31.074 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539624+0000", 2024-05-08T08:54:31.074 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4043e-05 2024-05-08T08:54:31.074 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.074 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.074 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:31.075 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539635+0000", 2024-05-08T08:54:31.075 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1511e-05 2024-05-08T08:54:31.075 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.075 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.075 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:31.075 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539728+0000", 2024-05-08T08:54:31.075 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2498999999999996e-05 2024-05-08T08:54:31.075 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.075 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.075 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:31.075 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539746+0000", 2024-05-08T08:54:31.075 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8289000000000001e-05 2024-05-08T08:54:31.075 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.075 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.075 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:54:31.076 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.630822+0000", 2024-05-08T08:54:31.076 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.091076283999999993 2024-05-08T08:54:31.076 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.076 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.076 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:31.076 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.630863+0000", 2024-05-08T08:54:31.076 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0374999999999999e-05 2024-05-08T08:54:31.076 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.076 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:31.076 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.076 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.076 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.076 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4195.0:4 3.11 3:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-05-08T08:54:31.076 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:49.764654+0000", 2024-05-08T08:54:31.076 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 101.309024861, 2024-05-08T08:54:31.077 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.30714275499999999, 2024-05-08T08:54:31.077 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:31.077 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:31.077 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-05-08T08:54:31.077 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:31.077 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4195", 2024-05-08T08:54:31.077 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/59179521", 2024-05-08T08:54:31.077 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4 2024-05-08T08:54:31.077 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.077 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:31.077 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.077 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:31.077 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764654+0000", 2024-05-08T08:54:31.077 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:31.077 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.078 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.078 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:31.078 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764654+0000", 2024-05-08T08:54:31.078 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:31.078 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.078 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.078 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:31.078 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764655+0000", 2024-05-08T08:54:31.078 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.452e-06 2024-05-08T08:54:31.078 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.078 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.078 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:31.078 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764661+0000", 2024-05-08T08:54:31.078 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5369999999999997e-06 2024-05-08T08:54:31.079 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.079 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.079 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:31.079 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764671+0000", 2024-05-08T08:54:31.079 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.028e-05 2024-05-08T08:54:31.079 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.079 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.079 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:31.079 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764680+0000", 2024-05-08T08:54:31.079 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0629999999999995e-06 2024-05-08T08:54:31.079 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.079 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.079 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:31.079 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764753+0000", 2024-05-08T08:54:31.080 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2679e-05 2024-05-08T08:54:31.080 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.080 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.080 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-05-08T08:54:31.080 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764765+0000", 2024-05-08T08:54:31.080 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2415000000000001e-05 2024-05-08T08:54:31.080 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.080 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.080 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:31.080 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:50.071692+0000", 2024-05-08T08:54:31.080 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.306927324 2024-05-08T08:54:31.080 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.080 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.080 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:31.080 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:50.071796+0000", 2024-05-08T08:54:31.081 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104005 2024-05-08T08:54:31.081 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.081 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:31.081 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.081 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.081 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.081 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4416.0:20531 3.16 3:6bb2ed16:::rbd_object_map.1140bc4a55c7:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-05-08T08:54:31.081 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:58.673786+0000", 2024-05-08T08:54:31.081 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 32.399892457, 2024-05-08T08:54:31.081 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.94684119899999997, 2024-05-08T08:54:31.081 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:31.081 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:31.081 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:54:31.081 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:31.081 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4416", 2024-05-08T08:54:31.082 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:54:31.082 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20531 2024-05-08T08:54:31.082 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.082 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:31.082 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.082 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:31.082 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673786+0000", 2024-05-08T08:54:31.082 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:31.082 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.082 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.082 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:31.082 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673786+0000", 2024-05-08T08:54:31.082 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:31.082 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.082 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.083 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:31.083 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673787+0000", 2024-05-08T08:54:31.083 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9800000000000003e-07 2024-05-08T08:54:31.083 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.083 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.083 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:31.083 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673791+0000", 2024-05-08T08:54:31.083 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.104e-06 2024-05-08T08:54:31.083 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.083 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.083 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:31.083 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673801+0000", 2024-05-08T08:54:31.083 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9960000000000005e-06 2024-05-08T08:54:31.083 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.083 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.083 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:31.084 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673806+0000", 2024-05-08T08:54:31.084 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2320000000000001e-06 2024-05-08T08:54:31.084 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.084 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.084 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:31.084 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673860+0000", 2024-05-08T08:54:31.084 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3468999999999999e-05 2024-05-08T08:54:31.084 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.084 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.084 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-05-08T08:54:31.084 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673866+0000", 2024-05-08T08:54:31.084 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3500000000000002e-06 2024-05-08T08:54:31.084 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.084 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.084 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:31.084 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.616700+0000", 2024-05-08T08:54:31.085 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.94283390700000003 2024-05-08T08:54:31.085 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.085 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.085 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:31.085 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.616941+0000", 2024-05-08T08:54:31.085 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00024080400000000001 2024-05-08T08:54:31.085 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.085 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.085 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-05-08T08:54:31.085 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.617072+0000", 2024-05-08T08:54:31.085 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000131688 2024-05-08T08:54:31.085 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.085 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.085 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:54:31.086 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.619571+0000", 2024-05-08T08:54:31.086 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0024990149999999998 2024-05-08T08:54:31.086 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.086 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.086 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:54:31.086 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.620568+0000", 2024-05-08T08:54:31.086 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00099711000000000005 2024-05-08T08:54:31.086 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.086 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.086 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:54:31.086 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.620602+0000", 2024-05-08T08:54:31.086 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.324e-05 2024-05-08T08:54:31.086 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.086 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.086 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:31.087 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.620627+0000", 2024-05-08T08:54:31.087 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5585999999999999e-05 2024-05-08T08:54:31.087 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.087 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:31.087 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.087 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.087 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.087 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5490 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4227072~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:31.087 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.881809+0000", 2024-05-08T08:54:31.087 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.1918699449999997, 2024-05-08T08:54:31.087 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047460447000000003, 2024-05-08T08:54:31.087 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:31.087 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:31.087 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:31.087 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:31.088 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:31.088 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:31.088 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5490 2024-05-08T08:54:31.088 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.088 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:31.088 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.088 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:31.088 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881809+0000", 2024-05-08T08:54:31.088 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:31.088 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.088 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.088 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:31.088 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881809+0000", 2024-05-08T08:54:31.088 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:31.088 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.089 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.089 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:31.089 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881809+0000", 2024-05-08T08:54:31.089 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7899999999999999e-07 2024-05-08T08:54:31.089 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.089 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.089 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:31.089 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881811+0000", 2024-05-08T08:54:31.089 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0339999999999999e-06 2024-05-08T08:54:31.089 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.089 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.089 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:31.089 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881818+0000", 2024-05-08T08:54:31.089 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5760000000000002e-06 2024-05-08T08:54:31.089 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.090 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.090 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:31.090 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881823+0000", 2024-05-08T08:54:31.090 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4160000000000003e-06 2024-05-08T08:54:31.090 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.090 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.090 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:31.090 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929157+0000", 2024-05-08T08:54:31.090 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047334029999999999 2024-05-08T08:54:31.090 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.090 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.090 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:31.090 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929211+0000", 2024-05-08T08:54:31.090 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3560999999999998e-05 2024-05-08T08:54:31.090 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.091 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.091 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:31.091 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929269+0000", 2024-05-08T08:54:31.091 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8350999999999997e-05 2024-05-08T08:54:31.091 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.091 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:31.091 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.091 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.091 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.091 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5492 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4272128~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:31.091 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.881830+0000", 2024-05-08T08:54:31.091 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.1918486509999999, 2024-05-08T08:54:31.091 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047614006, 2024-05-08T08:54:31.091 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:31.091 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:31.092 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:31.092 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:31.092 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:31.092 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:31.092 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5492 2024-05-08T08:54:31.092 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.092 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:31.092 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.092 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:31.092 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881830+0000", 2024-05-08T08:54:31.092 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:31.092 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.092 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.092 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:31.092 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881830+0000", 2024-05-08T08:54:31.093 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:31.093 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.093 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.093 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:31.093 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881830+0000", 2024-05-08T08:54:31.093 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4099999999999999e-07 2024-05-08T08:54:31.093 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.093 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.093 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:31.093 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881832+0000", 2024-05-08T08:54:31.093 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0370000000000001e-06 2024-05-08T08:54:31.093 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.093 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.093 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:31.093 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881839+0000", 2024-05-08T08:54:31.094 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3289999999999999e-06 2024-05-08T08:54:31.094 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.094 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.094 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:31.094 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881844+0000", 2024-05-08T08:54:31.094 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1170000000000003e-06 2024-05-08T08:54:31.094 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.094 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.094 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:31.094 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929296+0000", 2024-05-08T08:54:31.094 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047452418000000003 2024-05-08T08:54:31.094 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.094 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.094 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:31.094 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929360+0000", 2024-05-08T08:54:31.095 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4197999999999998e-05 2024-05-08T08:54:31.095 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.095 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.095 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:31.095 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929444+0000", 2024-05-08T08:54:31.095 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3466000000000004e-05 2024-05-08T08:54:31.095 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.095 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:31.095 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.095 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.095 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.095 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5494 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4317184~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:31.095 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.881850+0000", 2024-05-08T08:54:31.095 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.1918289020000001, 2024-05-08T08:54:31.095 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047723887, 2024-05-08T08:54:31.096 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:31.096 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:31.096 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:31.096 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:31.096 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:31.096 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:31.096 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5494 2024-05-08T08:54:31.096 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.096 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:31.096 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.096 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:31.096 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881850+0000", 2024-05-08T08:54:31.096 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:31.096 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.096 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.096 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:31.097 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881850+0000", 2024-05-08T08:54:31.097 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:31.097 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.097 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.097 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:31.097 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881850+0000", 2024-05-08T08:54:31.097 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8500000000000002e-07 2024-05-08T08:54:31.097 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.097 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.097 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:31.097 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881856+0000", 2024-05-08T08:54:31.097 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9510000000000003e-06 2024-05-08T08:54:31.097 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.097 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.097 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:31.098 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881862+0000", 2024-05-08T08:54:31.098 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3049999999999998e-06 2024-05-08T08:54:31.098 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.098 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.098 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:31.098 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881868+0000", 2024-05-08T08:54:31.098 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8340000000000004e-06 2024-05-08T08:54:31.098 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.098 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.098 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:31.098 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929475+0000", 2024-05-08T08:54:31.098 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047606796 2024-05-08T08:54:31.098 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.098 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.099 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:31.099 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929518+0000", 2024-05-08T08:54:31.099 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3387000000000003e-05 2024-05-08T08:54:31.099 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.099 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.099 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:31.099 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929574+0000", 2024-05-08T08:54:31.099 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5129e-05 2024-05-08T08:54:31.099 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.099 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:31.099 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.099 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.099 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.099 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5495 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4362240~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:31.100 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.881875+0000", 2024-05-08T08:54:31.100 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.1918040349999997, 2024-05-08T08:54:31.100 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047847967999999998, 2024-05-08T08:54:31.100 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:31.100 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:31.100 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:31.100 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:31.100 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:31.100 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:31.100 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5495 2024-05-08T08:54:31.100 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.100 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:31.100 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.100 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:31.100 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881875+0000", 2024-05-08T08:54:31.101 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:31.101 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.101 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.101 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:31.101 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881875+0000", 2024-05-08T08:54:31.101 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:31.101 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.101 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.101 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:31.101 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881875+0000", 2024-05-08T08:54:31.101 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-07 2024-05-08T08:54:31.101 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.101 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.101 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:31.101 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881877+0000", 2024-05-08T08:54:31.102 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9659999999999999e-06 2024-05-08T08:54:31.102 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.102 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.102 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:31.102 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881883+0000", 2024-05-08T08:54:31.102 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7649999999999999e-06 2024-05-08T08:54:31.102 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.102 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.102 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:31.102 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881887+0000", 2024-05-08T08:54:31.102 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6689999999999997e-06 2024-05-08T08:54:31.102 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.102 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.102 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:31.103 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929600+0000", 2024-05-08T08:54:31.103 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047712746 2024-05-08T08:54:31.103 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.103 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.103 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:31.103 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929659+0000", 2024-05-08T08:54:31.103 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8936e-05 2024-05-08T08:54:31.103 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.103 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.103 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:31.103 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929723+0000", 2024-05-08T08:54:31.103 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3524999999999999e-05 2024-05-08T08:54:31.103 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.103 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:31.103 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.104 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.104 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.104 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5496 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4407296~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:31.104 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.881894+0000", 2024-05-08T08:54:31.104 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.1917841710000001, 2024-05-08T08:54:31.104 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047998015999999998, 2024-05-08T08:54:31.104 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:31.104 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:31.104 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:31.104 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:31.104 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:31.104 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:31.104 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5496 2024-05-08T08:54:31.104 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.104 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:31.105 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.105 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:31.105 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881894+0000", 2024-05-08T08:54:31.105 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:31.105 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.105 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.105 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:31.105 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881894+0000", 2024-05-08T08:54:31.105 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:31.105 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.105 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.105 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:31.105 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881895+0000", 2024-05-08T08:54:31.105 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6699999999999999e-07 2024-05-08T08:54:31.105 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.106 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.106 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:31.106 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881897+0000", 2024-05-08T08:54:31.106 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0229999999999999e-06 2024-05-08T08:54:31.106 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.106 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.106 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:31.106 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881903+0000", 2024-05-08T08:54:31.106 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9850000000000003e-06 2024-05-08T08:54:31.106 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.106 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.106 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:31.106 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881908+0000", 2024-05-08T08:54:31.106 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9019999999999998e-06 2024-05-08T08:54:31.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.107 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:31.107 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929751+0000", 2024-05-08T08:54:31.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047843550999999998 2024-05-08T08:54:31.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.107 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:31.107 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929823+0000", 2024-05-08T08:54:31.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1172000000000002e-05 2024-05-08T08:54:31.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.107 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:31.107 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929892+0000", 2024-05-08T08:54:31.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9915999999999999e-05 2024-05-08T08:54:31.108 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.108 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:31.108 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.108 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.108 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.108 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5498 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4452352~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:31.108 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.881914+0000", 2024-05-08T08:54:31.108 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.1917644129999996, 2024-05-08T08:54:31.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048203553000000003, 2024-05-08T08:54:31.108 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:31.108 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:31.108 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:31.108 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:31.108 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:31.108 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:31.109 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5498 2024-05-08T08:54:31.109 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.109 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:31.109 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.109 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:31.109 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881914+0000", 2024-05-08T08:54:31.109 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:31.109 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.109 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.109 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:31.109 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881914+0000", 2024-05-08T08:54:31.109 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:31.109 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.109 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.109 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:31.109 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881915+0000", 2024-05-08T08:54:31.110 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8500000000000002e-07 2024-05-08T08:54:31.110 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.110 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.110 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:31.110 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881917+0000", 2024-05-08T08:54:31.110 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.181e-06 2024-05-08T08:54:31.110 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.110 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.110 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:31.110 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881924+0000", 2024-05-08T08:54:31.110 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9619999999999996e-06 2024-05-08T08:54:31.110 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.110 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.110 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:31.111 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881929+0000", 2024-05-08T08:54:31.111 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1399999999999999e-06 2024-05-08T08:54:31.111 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.111 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.111 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:31.111 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929952+0000", 2024-05-08T08:54:31.111 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048023134000000002 2024-05-08T08:54:31.111 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.111 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.111 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:31.111 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.930029+0000", 2024-05-08T08:54:31.111 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7490000000000005e-05 2024-05-08T08:54:31.111 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.111 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.112 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:31.112 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.930118+0000", 2024-05-08T08:54:31.112 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8261000000000001e-05 2024-05-08T08:54:31.112 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.112 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:31.112 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.112 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.112 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.112 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5563 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4321280~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:31.112 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.884201+0000", 2024-05-08T08:54:31.112 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.1894771630000003, 2024-05-08T08:54:31.112 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047379969000000001, 2024-05-08T08:54:31.112 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:31.112 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:31.112 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:31.113 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:31.113 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:31.113 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:31.113 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5563 2024-05-08T08:54:31.113 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.113 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:31.113 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.113 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:31.113 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884201+0000", 2024-05-08T08:54:31.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:31.113 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.113 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.113 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:31.113 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884201+0000", 2024-05-08T08:54:31.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:31.113 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.114 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.114 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:31.114 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884202+0000", 2024-05-08T08:54:31.114 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8000000000000002e-07 2024-05-08T08:54:31.114 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.114 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.114 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:31.114 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884208+0000", 2024-05-08T08:54:31.114 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.674e-06 2024-05-08T08:54:31.114 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.114 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.114 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:31.114 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884215+0000", 2024-05-08T08:54:31.114 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0389999999999999e-06 2024-05-08T08:54:31.114 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.115 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.115 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:31.115 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884220+0000", 2024-05-08T08:54:31.115 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4950000000000002e-06 2024-05-08T08:54:31.115 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.115 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.115 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:31.115 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931417+0000", 2024-05-08T08:54:31.115 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047196674000000001 2024-05-08T08:54:31.115 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.115 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.115 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:31.115 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931493+0000", 2024-05-08T08:54:31.115 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5950000000000003e-05 2024-05-08T08:54:31.115 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.115 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.116 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:31.116 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931581+0000", 2024-05-08T08:54:31.116 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8857000000000007e-05 2024-05-08T08:54:31.116 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.116 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:31.116 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.116 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.116 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.116 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5564 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4366336~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:31.116 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.884225+0000", 2024-05-08T08:54:31.116 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.1894536230000003, 2024-05-08T08:54:31.116 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047555989999999999, 2024-05-08T08:54:31.116 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:31.116 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:31.116 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:31.117 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:31.117 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:31.117 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:31.117 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5564 2024-05-08T08:54:31.117 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.117 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:31.117 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.117 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:31.117 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884225+0000", 2024-05-08T08:54:31.117 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:31.117 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.117 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.117 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:31.117 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884225+0000", 2024-05-08T08:54:31.117 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:31.118 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.118 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.118 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:31.118 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884225+0000", 2024-05-08T08:54:31.118 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2499999999999995e-07 2024-05-08T08:54:31.118 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.118 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.118 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:31.118 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884227+0000", 2024-05-08T08:54:31.118 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5829999999999999e-06 2024-05-08T08:54:31.118 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.118 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.118 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:31.118 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884232+0000", 2024-05-08T08:54:31.118 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1159999999999998e-06 2024-05-08T08:54:31.119 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.119 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.119 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:31.119 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884236+0000", 2024-05-08T08:54:31.119 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2490000000000003e-06 2024-05-08T08:54:31.119 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.119 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.119 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:31.119 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931617+0000", 2024-05-08T08:54:31.119 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047380310000000002 2024-05-08T08:54:31.119 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.119 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.119 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:31.119 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931689+0000", 2024-05-08T08:54:31.119 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2358000000000001e-05 2024-05-08T08:54:31.119 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.120 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.120 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:31.120 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931781+0000", 2024-05-08T08:54:31.120 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1848999999999994e-05 2024-05-08T08:54:31.120 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.120 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:31.120 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.120 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.120 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.120 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5565 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4411392~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:31.120 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.884251+0000", 2024-05-08T08:54:31.120 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.18942747, 2024-05-08T08:54:31.120 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047743671000000001, 2024-05-08T08:54:31.120 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:31.120 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:31.121 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:31.121 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:31.121 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:31.121 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:31.121 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5565 2024-05-08T08:54:31.121 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.121 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:31.121 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.121 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:31.121 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884251+0000", 2024-05-08T08:54:31.121 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:31.121 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.121 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.121 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:31.121 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884251+0000", 2024-05-08T08:54:31.121 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:31.122 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.122 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.122 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:31.122 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884252+0000", 2024-05-08T08:54:31.122 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3200000000000005e-07 2024-05-08T08:54:31.122 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.122 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.122 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:31.122 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884253+0000", 2024-05-08T08:54:31.122 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6980000000000001e-06 2024-05-08T08:54:31.122 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.122 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.122 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:31.122 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884259+0000", 2024-05-08T08:54:31.123 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3759999999999999e-06 2024-05-08T08:54:31.123 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.123 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.123 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:31.123 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884263+0000", 2024-05-08T08:54:31.123 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7280000000000003e-06 2024-05-08T08:54:31.123 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.123 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.123 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:31.123 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931818+0000", 2024-05-08T08:54:31.123 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047554368999999999 2024-05-08T08:54:31.123 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.123 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.123 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:31.124 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931897+0000", 2024-05-08T08:54:31.124 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8821999999999999e-05 2024-05-08T08:54:31.124 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.124 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.124 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:31.124 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931995+0000", 2024-05-08T08:54:31.124 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8146000000000005e-05 2024-05-08T08:54:31.124 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.124 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:31.124 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.124 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.124 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.124 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5600 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 3948544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:31.124 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.884619+0000", 2024-05-08T08:54:31.124 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.1890592150000003, 2024-05-08T08:54:31.125 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047574067999999997, 2024-05-08T08:54:31.125 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:31.125 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:31.125 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:31.125 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:31.125 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:31.125 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:31.125 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5600 2024-05-08T08:54:31.125 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.125 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:31.125 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.125 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:31.125 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884619+0000", 2024-05-08T08:54:31.125 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:31.125 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.126 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.126 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:31.126 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884619+0000", 2024-05-08T08:54:31.126 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:31.126 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.126 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.126 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:31.126 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884620+0000", 2024-05-08T08:54:31.126 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.53e-07 2024-05-08T08:54:31.126 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.126 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.126 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:31.126 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884622+0000", 2024-05-08T08:54:31.127 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.796e-06 2024-05-08T08:54:31.127 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.127 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.127 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:31.127 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884627+0000", 2024-05-08T08:54:31.127 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2850000000000002e-06 2024-05-08T08:54:31.127 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.127 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.127 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:31.127 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884631+0000", 2024-05-08T08:54:31.127 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2899999999999996e-06 2024-05-08T08:54:31.127 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.127 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.127 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:31.128 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932024+0000", 2024-05-08T08:54:31.128 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047392680999999999 2024-05-08T08:54:31.128 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.128 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.128 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:31.128 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932102+0000", 2024-05-08T08:54:31.128 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7806999999999996e-05 2024-05-08T08:54:31.128 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.128 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.128 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:31.128 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932193+0000", 2024-05-08T08:54:31.128 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1855999999999996e-05 2024-05-08T08:54:31.128 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.128 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:31.128 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.128 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.129 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.129 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5603 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 3993600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:31.129 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.884680+0000", 2024-05-08T08:54:31.129 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.1889986060000002, 2024-05-08T08:54:31.129 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047745034999999998, 2024-05-08T08:54:31.129 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:31.129 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:31.129 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:31.129 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:31.129 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:31.129 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:31.129 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5603 2024-05-08T08:54:31.129 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.129 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:31.130 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.130 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:31.130 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884680+0000", 2024-05-08T08:54:31.130 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:31.130 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.130 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.130 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:31.130 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884680+0000", 2024-05-08T08:54:31.130 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:31.130 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.130 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.130 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:31.130 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884680+0000", 2024-05-08T08:54:31.130 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4400000000000001e-07 2024-05-08T08:54:31.130 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.131 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.131 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:31.131 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884682+0000", 2024-05-08T08:54:31.131 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.514e-06 2024-05-08T08:54:31.131 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.131 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.131 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:31.131 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884687+0000", 2024-05-08T08:54:31.131 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0359999999999997e-06 2024-05-08T08:54:31.131 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.131 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.131 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:31.131 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884692+0000", 2024-05-08T08:54:31.131 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5990000000000004e-06 2024-05-08T08:54:31.131 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.132 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.132 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:31.132 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932235+0000", 2024-05-08T08:54:31.132 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047543635000000001 2024-05-08T08:54:31.132 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.132 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.132 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:31.132 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932323+0000", 2024-05-08T08:54:31.132 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7311000000000005e-05 2024-05-08T08:54:31.132 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.132 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.132 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:31.132 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932425+0000", 2024-05-08T08:54:31.132 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102496 2024-05-08T08:54:31.132 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.132 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:31.133 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.133 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.133 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.133 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5606 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4038656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:31.133 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.884729+0000", 2024-05-08T08:54:31.133 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.1889493179999997, 2024-05-08T08:54:31.133 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047916488, 2024-05-08T08:54:31.133 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:31.133 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:31.133 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:31.133 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:31.133 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:31.133 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:31.133 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5606 2024-05-08T08:54:31.133 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.133 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:31.134 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.134 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:31.134 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884729+0000", 2024-05-08T08:54:31.134 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:31.134 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.134 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.134 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:31.134 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884729+0000", 2024-05-08T08:54:31.134 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:31.134 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.134 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.134 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:31.134 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.885940+0000", 2024-05-08T08:54:31.134 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001210509 2024-05-08T08:54:31.135 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.135 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.135 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:31.135 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.885945+0000", 2024-05-08T08:54:31.135 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7389999999999999e-06 2024-05-08T08:54:31.135 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.135 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.135 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:31.135 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.885955+0000", 2024-05-08T08:54:31.135 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0813e-05 2024-05-08T08:54:31.135 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.135 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.135 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:31.135 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.885962+0000", 2024-05-08T08:54:31.135 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1160000000000001e-06 2024-05-08T08:54:31.136 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.136 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.136 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:31.136 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932465+0000", 2024-05-08T08:54:31.136 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046502612999999998 2024-05-08T08:54:31.136 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.136 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.136 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:31.136 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932547+0000", 2024-05-08T08:54:31.136 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2025000000000002e-05 2024-05-08T08:54:31.136 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.136 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.136 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:31.136 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932646+0000", 2024-05-08T08:54:31.136 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8672999999999995e-05 2024-05-08T08:54:31.136 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.137 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:31.137 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.137 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.137 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.137 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5635 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4235264~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:31.137 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.886082+0000", 2024-05-08T08:54:31.137 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.1875961989999997, 2024-05-08T08:54:31.137 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047438284999999997, 2024-05-08T08:54:31.137 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:31.137 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:31.137 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:31.137 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:31.137 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:31.137 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:31.137 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5635 2024-05-08T08:54:31.137 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.138 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:31.138 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.138 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:31.138 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886082+0000", 2024-05-08T08:54:31.138 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:31.138 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.138 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.138 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:31.138 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886082+0000", 2024-05-08T08:54:31.138 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:31.138 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.138 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.138 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:31.138 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886083+0000", 2024-05-08T08:54:31.139 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8999999999999996e-07 2024-05-08T08:54:31.139 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.139 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.139 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:31.139 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886086+0000", 2024-05-08T08:54:31.139 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7e-06 2024-05-08T08:54:31.139 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.139 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.139 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:31.139 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886093+0000", 2024-05-08T08:54:31.139 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5320000000000001e-06 2024-05-08T08:54:31.139 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.139 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.139 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:31.139 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886102+0000", 2024-05-08T08:54:31.139 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8070000000000006e-06 2024-05-08T08:54:31.140 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.140 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.140 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:31.140 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933357+0000", 2024-05-08T08:54:31.140 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047255093999999997 2024-05-08T08:54:31.140 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.140 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.140 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:31.140 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933433+0000", 2024-05-08T08:54:31.140 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5796000000000004e-05 2024-05-08T08:54:31.140 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.140 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.140 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:31.140 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933521+0000", 2024-05-08T08:54:31.140 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7766e-05 2024-05-08T08:54:31.140 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.141 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:31.141 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.141 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.141 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.141 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5636 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4280320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:31.141 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.886111+0000", 2024-05-08T08:54:31.141 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.1875675809999997, 2024-05-08T08:54:31.141 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047663644999999998, 2024-05-08T08:54:31.141 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:31.141 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:31.141 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:31.141 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:31.141 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:31.141 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:31.142 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5636 2024-05-08T08:54:31.142 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.142 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:31.142 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.142 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:31.142 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886111+0000", 2024-05-08T08:54:31.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:31.142 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.142 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.142 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:31.142 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886111+0000", 2024-05-08T08:54:31.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:31.142 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.142 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.143 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:31.143 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886111+0000", 2024-05-08T08:54:31.143 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5200000000000002e-07 2024-05-08T08:54:31.143 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.143 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.143 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:31.143 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886114+0000", 2024-05-08T08:54:31.143 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1710000000000001e-06 2024-05-08T08:54:31.143 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.143 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.143 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:31.143 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886120+0000", 2024-05-08T08:54:31.143 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.404e-06 2024-05-08T08:54:31.143 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.143 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.143 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:31.144 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886125+0000", 2024-05-08T08:54:31.144 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2730000000000002e-06 2024-05-08T08:54:31.144 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.144 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.144 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:31.144 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933570+0000", 2024-05-08T08:54:31.144 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047444989999999999 2024-05-08T08:54:31.144 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.144 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.144 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:31.144 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933659+0000", 2024-05-08T08:54:31.144 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8912999999999994e-05 2024-05-08T08:54:31.144 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.144 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.144 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:31.145 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933775+0000", 2024-05-08T08:54:31.145 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000115442 2024-05-08T08:54:31.145 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.145 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:31.145 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.145 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.145 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.145 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5637 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4325376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:31.145 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.886132+0000", 2024-05-08T08:54:31.145 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.1875465429999998, 2024-05-08T08:54:31.145 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047852219000000001, 2024-05-08T08:54:31.145 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:31.145 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:31.145 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:31.145 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:31.146 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:31.146 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:31.146 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5637 2024-05-08T08:54:31.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.146 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:31.146 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.146 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:31.146 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886132+0000", 2024-05-08T08:54:31.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:31.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.146 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.146 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:31.146 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886132+0000", 2024-05-08T08:54:31.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:31.147 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.147 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.147 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:31.147 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886132+0000", 2024-05-08T08:54:31.147 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7300000000000002e-07 2024-05-08T08:54:31.147 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.147 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.147 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:31.147 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886139+0000", 2024-05-08T08:54:31.147 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6109999999999999e-06 2024-05-08T08:54:31.147 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.147 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.147 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:31.147 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886146+0000", 2024-05-08T08:54:31.147 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6279999999999999e-06 2024-05-08T08:54:31.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.148 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.148 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:31.148 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886151+0000", 2024-05-08T08:54:31.148 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1769999999999997e-06 2024-05-08T08:54:31.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.148 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.148 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:31.148 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933812+0000", 2024-05-08T08:54:31.148 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047660912999999999 2024-05-08T08:54:31.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.148 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.148 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:31.148 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933888+0000", 2024-05-08T08:54:31.148 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5865000000000006e-05 2024-05-08T08:54:31.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.149 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:31.149 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933984+0000", 2024-05-08T08:54:31.149 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6651999999999996e-05 2024-05-08T08:54:31.149 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.149 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:31.149 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.149 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5638 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4370432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:31.149 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.886157+0000", 2024-05-08T08:54:31.149 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.1875212790000003, 2024-05-08T08:54:31.149 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048077597, 2024-05-08T08:54:31.149 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:31.149 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:31.149 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:31.150 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:31.150 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:31.150 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:31.150 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5638 2024-05-08T08:54:31.150 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.150 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:31.150 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.150 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:31.150 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886157+0000", 2024-05-08T08:54:31.150 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:31.150 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.150 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.150 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:31.150 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886157+0000", 2024-05-08T08:54:31.150 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:31.151 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.151 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.151 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:31.151 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886158+0000", 2024-05-08T08:54:31.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1100000000000001e-07 2024-05-08T08:54:31.151 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.151 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.151 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:31.151 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886160+0000", 2024-05-08T08:54:31.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9800000000000001e-06 2024-05-08T08:54:31.151 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.151 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.151 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:31.151 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886166+0000", 2024-05-08T08:54:31.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.314e-06 2024-05-08T08:54:31.151 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.152 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.152 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:31.152 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886171+0000", 2024-05-08T08:54:31.152 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7700000000000001e-06 2024-05-08T08:54:31.152 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.152 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.152 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:31.152 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.934025+0000", 2024-05-08T08:54:31.152 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047854509000000003 2024-05-08T08:54:31.152 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.152 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.152 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:31.152 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.934121+0000", 2024-05-08T08:54:31.152 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5688999999999998e-05 2024-05-08T08:54:31.152 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.153 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.153 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:31.153 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.934235+0000", 2024-05-08T08:54:31.153 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000113924 2024-05-08T08:54:31.153 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.153 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:31.153 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.153 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.153 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.153 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5639 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4415488~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:31.153 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.886177+0000", 2024-05-08T08:54:31.153 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.1875015290000004, 2024-05-08T08:54:31.153 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048249541, 2024-05-08T08:54:31.153 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:31.153 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:31.153 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:31.154 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:31.154 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:31.154 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:31.154 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5639 2024-05-08T08:54:31.154 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.154 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:31.154 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.154 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:31.154 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886177+0000", 2024-05-08T08:54:31.154 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:31.154 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.154 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.154 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:31.154 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886177+0000", 2024-05-08T08:54:31.154 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:31.154 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.155 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.155 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:31.155 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886177+0000", 2024-05-08T08:54:31.155 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.65e-07 2024-05-08T08:54:31.155 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.155 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.155 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:31.155 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886179+0000", 2024-05-08T08:54:31.155 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9530000000000002e-06 2024-05-08T08:54:31.155 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.155 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.155 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:31.155 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886186+0000", 2024-05-08T08:54:31.155 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3419999999999996e-06 2024-05-08T08:54:31.155 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.156 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.156 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:31.156 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886192+0000", 2024-05-08T08:54:31.156 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0560000000000001e-06 2024-05-08T08:54:31.156 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.156 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.156 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:31.156 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.934258+0000", 2024-05-08T08:54:31.156 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048066367999999998 2024-05-08T08:54:31.156 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.156 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.156 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:31.156 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.934338+0000", 2024-05-08T08:54:31.156 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9616000000000005e-05 2024-05-08T08:54:31.156 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.156 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.157 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:31.157 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.934427+0000", 2024-05-08T08:54:31.157 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8840999999999999e-05 2024-05-08T08:54:31.157 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.157 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:31.157 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.157 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.157 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:31.157 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:31.158 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:31.161+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:31.159 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:54:31.259 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:31.261+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:31.312 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:31.312 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:54:31.312 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:54:31.312 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:31.322 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:54:31.360 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:31.362+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:31.431 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:31.432 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:54:31.432 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:54:31.432 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:54:31.432 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:31.440 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:54:31.461 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:31.464+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:31.548 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:31.549 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:54:31.549 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:54:31.549 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:54:31.549 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.549 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22073 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 782336~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:31.549 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:54:31.549 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 123.69268445900001, 2024-05-08T08:54:31.549 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047244234000000003, 2024-05-08T08:54:31.549 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:31.549 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:31.549 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:31.550 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:31.550 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:31.550 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:31.550 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22073 2024-05-08T08:54:31.550 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.550 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:31.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.551 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:31.551 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:54:31.551 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:31.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.551 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:31.551 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:54:31.551 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:31.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.551 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:31.551 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857668+0000", 2024-05-08T08:54:31.552 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7800000000000002e-07 2024-05-08T08:54:31.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.552 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.552 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:31.552 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857670+0000", 2024-05-08T08:54:31.552 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0820000000000001e-06 2024-05-08T08:54:31.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.552 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.552 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:31.552 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857678+0000", 2024-05-08T08:54:31.552 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7589999999999997e-06 2024-05-08T08:54:31.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.552 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.552 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:31.552 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857691+0000", 2024-05-08T08:54:31.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3672999999999999e-05 2024-05-08T08:54:31.553 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.553 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.553 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:31.553 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904780+0000", 2024-05-08T08:54:31.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047088616999999999 2024-05-08T08:54:31.553 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.553 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.553 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:31.553 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904840+0000", 2024-05-08T08:54:31.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0356999999999998e-05 2024-05-08T08:54:31.553 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.553 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.553 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:31.553 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904912+0000", 2024-05-08T08:54:31.554 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1367999999999998e-05 2024-05-08T08:54:31.554 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.554 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:31.554 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.554 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.554 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.554 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22075 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 798720~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:31.554 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:54:31.554 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 123.692654062, 2024-05-08T08:54:31.554 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047384643999999997, 2024-05-08T08:54:31.554 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:31.554 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:31.554 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:31.554 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:31.555 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:31.555 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:31.555 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22075 2024-05-08T08:54:31.555 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.555 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:31.555 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.555 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:31.555 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:54:31.555 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:31.555 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.555 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.555 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:31.555 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:54:31.555 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:31.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.556 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.556 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:31.556 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:54:31.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7399999999999999e-07 2024-05-08T08:54:31.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.556 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.556 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:31.556 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857705+0000", 2024-05-08T08:54:31.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1980000000000004e-06 2024-05-08T08:54:31.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.556 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.556 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:31.556 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857713+0000", 2024-05-08T08:54:31.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7039999999999994e-06 2024-05-08T08:54:31.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.557 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.557 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:31.557 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857718+0000", 2024-05-08T08:54:31.557 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.865e-06 2024-05-08T08:54:31.557 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.557 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.557 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:31.558 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904941+0000", 2024-05-08T08:54:31.558 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047223198000000001 2024-05-08T08:54:31.558 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.558 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.558 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:31.558 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904999+0000", 2024-05-08T08:54:31.558 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8254999999999997e-05 2024-05-08T08:54:31.558 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.558 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.558 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:31.559 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905082+0000", 2024-05-08T08:54:31.559 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3049999999999999e-05 2024-05-08T08:54:31.559 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.559 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:31.559 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.559 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.559 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.559 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22077 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 815104~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:31.559 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:54:31.559 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 123.692616051, 2024-05-08T08:54:31.559 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047519448999999998, 2024-05-08T08:54:31.559 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:31.559 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:31.559 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:31.559 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:31.560 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:31.560 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:31.560 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22077 2024-05-08T08:54:31.560 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.560 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:31.560 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.560 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:31.560 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:54:31.560 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:31.560 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.560 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.560 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:31.560 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:54:31.560 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:31.561 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.561 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.561 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:31.561 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:54:31.561 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.99e-07 2024-05-08T08:54:31.561 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.561 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.561 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:31.561 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857739+0000", 2024-05-08T08:54:31.561 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3599999999999999e-06 2024-05-08T08:54:31.561 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.562 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.562 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:31.562 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857745+0000", 2024-05-08T08:54:31.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5139999999999998e-06 2024-05-08T08:54:31.562 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.562 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.562 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:31.562 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857750+0000", 2024-05-08T08:54:31.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8550000000000001e-06 2024-05-08T08:54:31.562 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.562 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.562 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:31.563 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905110+0000", 2024-05-08T08:54:31.563 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047360126000000002 2024-05-08T08:54:31.563 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.563 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.563 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:31.563 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905175+0000", 2024-05-08T08:54:31.563 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4689000000000004e-05 2024-05-08T08:54:31.563 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.563 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.563 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:31.563 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905255+0000", 2024-05-08T08:54:31.563 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0306e-05 2024-05-08T08:54:31.563 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.564 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:31.564 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.564 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.564 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.564 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22079 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 831488~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:31.564 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:54:31.564 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 123.69259550300001, 2024-05-08T08:54:31.564 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.04767499, 2024-05-08T08:54:31.564 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:31.564 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:31.564 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:31.564 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:31.565 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:31.565 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:31.565 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22079 2024-05-08T08:54:31.565 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.565 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:31.565 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.565 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:31.565 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:54:31.565 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:31.565 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.565 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.565 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:31.566 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:54:31.566 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:31.566 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.566 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.566 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:31.566 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857757+0000", 2024-05-08T08:54:31.566 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8299999999999997e-07 2024-05-08T08:54:31.566 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.566 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.566 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:31.566 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857759+0000", 2024-05-08T08:54:31.566 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1390000000000002e-06 2024-05-08T08:54:31.566 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.566 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.567 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:31.567 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857765+0000", 2024-05-08T08:54:31.567 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4609999999999996e-06 2024-05-08T08:54:31.567 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.567 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.567 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:31.567 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857770+0000", 2024-05-08T08:54:31.567 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6689999999999997e-06 2024-05-08T08:54:31.567 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.567 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.567 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:31.567 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905281+0000", 2024-05-08T08:54:31.567 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047510668999999998 2024-05-08T08:54:31.567 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.568 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.568 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:31.568 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905344+0000", 2024-05-08T08:54:31.568 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2797000000000002e-05 2024-05-08T08:54:31.568 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.568 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.568 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:31.568 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905431+0000", 2024-05-08T08:54:31.568 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7771999999999999e-05 2024-05-08T08:54:31.568 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.568 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:31.568 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.568 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.568 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.569 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22082 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 847872~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:31.569 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:54:31.569 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 123.692575812, 2024-05-08T08:54:31.569 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047831934999999999, 2024-05-08T08:54:31.569 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:31.569 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:31.569 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:31.569 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:31.569 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:31.569 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:31.569 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22082 2024-05-08T08:54:31.570 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.570 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:31.570 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.570 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:31.570 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:54:31.570 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:31.570 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.570 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.570 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:31.570 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:54:31.570 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:31.570 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.570 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.570 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:31.570 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:54:31.571 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:54:31.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.571 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.571 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:31.571 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857778+0000", 2024-05-08T08:54:31.571 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0789999999999999e-06 2024-05-08T08:54:31.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.571 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.571 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:31.571 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857784+0000", 2024-05-08T08:54:31.571 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0480000000000004e-06 2024-05-08T08:54:31.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.571 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.571 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:31.571 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857790+0000", 2024-05-08T08:54:31.571 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2719999999999997e-06 2024-05-08T08:54:31.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.572 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.572 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:31.572 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905458+0000", 2024-05-08T08:54:31.572 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047668470999999997 2024-05-08T08:54:31.572 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.572 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.572 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:31.572 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905521+0000", 2024-05-08T08:54:31.572 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2952000000000003e-05 2024-05-08T08:54:31.572 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.572 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.572 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:31.572 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905608+0000", 2024-05-08T08:54:31.572 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6759000000000001e-05 2024-05-08T08:54:31.572 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.572 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:31.573 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.573 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.573 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.573 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22084 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 864256~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:31.573 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:54:31.573 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 123.692551862, 2024-05-08T08:54:31.573 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047977748000000001, 2024-05-08T08:54:31.573 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:31.573 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:31.573 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:31.573 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:31.573 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:31.573 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:31.573 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22084 2024-05-08T08:54:31.573 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.573 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:31.574 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.574 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:31.574 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:54:31.574 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:31.574 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.574 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.574 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:31.574 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:54:31.574 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:31.574 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.574 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.574 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:31.574 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:54:31.574 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:54:31.575 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.575 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.575 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:31.575 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857802+0000", 2024-05-08T08:54:31.575 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0779999999999998e-06 2024-05-08T08:54:31.575 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.575 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.575 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:31.575 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857812+0000", 2024-05-08T08:54:31.575 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7939999999999997e-06 2024-05-08T08:54:31.575 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.575 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.575 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:31.575 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857817+0000", 2024-05-08T08:54:31.575 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8899999999999998e-06 2024-05-08T08:54:31.575 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.576 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.576 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:31.576 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905636+0000", 2024-05-08T08:54:31.576 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047819252 2024-05-08T08:54:31.576 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.576 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.576 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:31.576 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905700+0000", 2024-05-08T08:54:31.576 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4016999999999994e-05 2024-05-08T08:54:31.576 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.576 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.576 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:31.576 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905778+0000", 2024-05-08T08:54:31.576 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7306999999999997e-05 2024-05-08T08:54:31.576 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.576 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:31.577 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.577 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.577 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.577 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22086 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 880640~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:31.577 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:54:31.577 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 123.692528512, 2024-05-08T08:54:31.577 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048119149, 2024-05-08T08:54:31.577 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:31.577 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:31.577 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:31.577 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:31.577 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:31.577 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:31.577 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22086 2024-05-08T08:54:31.577 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.578 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:31.578 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.578 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:31.578 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:54:31.578 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:31.578 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.578 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.578 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:31.578 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:54:31.578 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:31.578 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.578 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.578 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:31.578 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857824+0000", 2024-05-08T08:54:31.579 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4200000000000002e-07 2024-05-08T08:54:31.579 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.579 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.579 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:31.579 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857826+0000", 2024-05-08T08:54:31.579 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0590000000000001e-06 2024-05-08T08:54:31.579 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.579 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.579 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:31.579 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857832+0000", 2024-05-08T08:54:31.579 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8029999999999999e-06 2024-05-08T08:54:31.580 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.580 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.580 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:31.580 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857837+0000", 2024-05-08T08:54:31.580 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7929999999999997e-06 2024-05-08T08:54:31.580 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.580 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.580 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:31.580 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905803+0000", 2024-05-08T08:54:31.580 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047965761000000003 2024-05-08T08:54:31.580 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.580 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.580 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:31.580 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905862+0000", 2024-05-08T08:54:31.580 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9203000000000002e-05 2024-05-08T08:54:31.580 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.581 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.581 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:31.581 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905942+0000", 2024-05-08T08:54:31.581 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0187999999999999e-05 2024-05-08T08:54:31.581 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.581 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:31.581 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.581 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.581 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.581 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22088 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 897024~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:31.581 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:54:31.581 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 123.69250846, 2024-05-08T08:54:31.581 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048256049000000002, 2024-05-08T08:54:31.581 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:31.581 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:31.581 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:31.581 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:31.582 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:31.582 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:31.582 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22088 2024-05-08T08:54:31.582 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.582 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:31.582 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.582 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:31.582 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:54:31.582 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:31.582 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.582 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.582 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:31.582 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:54:31.582 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:31.583 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.583 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.583 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:31.583 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857844+0000", 2024-05-08T08:54:31.583 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.46e-07 2024-05-08T08:54:31.583 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.583 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.583 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:31.583 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857846+0000", 2024-05-08T08:54:31.583 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0899999999999999e-06 2024-05-08T08:54:31.583 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.583 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.583 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:31.583 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857855+0000", 2024-05-08T08:54:31.583 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8219999999999996e-06 2024-05-08T08:54:31.584 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.584 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.584 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:31.584 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857860+0000", 2024-05-08T08:54:31.584 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2989999999999996e-06 2024-05-08T08:54:31.584 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.584 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.584 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:31.584 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905966+0000", 2024-05-08T08:54:31.584 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048106542000000002 2024-05-08T08:54:31.584 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.584 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.584 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:31.584 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906024+0000", 2024-05-08T08:54:31.584 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7958000000000003e-05 2024-05-08T08:54:31.584 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.584 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.585 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:31.585 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906099+0000", 2024-05-08T08:54:31.585 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4992000000000003e-05 2024-05-08T08:54:31.585 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.585 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:31.585 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.585 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.585 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.585 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22091 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 913408~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:31.585 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:54:31.586 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 123.69248469999999, 2024-05-08T08:54:31.586 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048395290000000001, 2024-05-08T08:54:31.586 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:31.586 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:31.586 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:31.586 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:31.586 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:31.586 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:31.586 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22091 2024-05-08T08:54:31.586 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.586 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:31.586 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.586 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:31.586 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:54:31.587 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:31.587 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.587 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.587 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:31.587 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:54:31.587 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:31.587 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.587 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.587 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:31.587 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:54:31.587 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4900000000000001e-07 2024-05-08T08:54:31.587 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.587 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.587 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:31.587 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857870+0000", 2024-05-08T08:54:31.587 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2510000000000001e-06 2024-05-08T08:54:31.587 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.587 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.588 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:31.588 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857877+0000", 2024-05-08T08:54:31.588 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0809999999999996e-06 2024-05-08T08:54:31.588 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.588 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.588 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:31.588 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857881+0000", 2024-05-08T08:54:31.588 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.724e-06 2024-05-08T08:54:31.588 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.588 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.588 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:31.588 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906127+0000", 2024-05-08T08:54:31.588 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048245345000000002 2024-05-08T08:54:31.588 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.588 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.588 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:31.588 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906190+0000", 2024-05-08T08:54:31.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2762000000000007e-05 2024-05-08T08:54:31.589 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.589 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.589 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:31.589 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906262+0000", 2024-05-08T08:54:31.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2778e-05 2024-05-08T08:54:31.589 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.589 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:31.589 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.589 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.589 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.589 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22093 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 929792~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:31.589 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:54:31.589 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 123.69246233600001, 2024-05-08T08:54:31.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048532875000000003, 2024-05-08T08:54:31.590 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:31.590 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:31.590 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:31.590 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:31.590 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:31.590 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:31.590 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22093 2024-05-08T08:54:31.591 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.591 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:31.591 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.591 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:31.591 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:54:31.591 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:31.591 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.591 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.591 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:31.591 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:54:31.591 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:31.591 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.591 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.591 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:31.591 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857890+0000", 2024-05-08T08:54:31.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-07 2024-05-08T08:54:31.592 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.592 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.592 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:31.592 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857899+0000", 2024-05-08T08:54:31.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6889999999999999e-06 2024-05-08T08:54:31.592 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.592 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.592 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:31.592 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857906+0000", 2024-05-08T08:54:31.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6490000000000002e-06 2024-05-08T08:54:31.593 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.593 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.593 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:31.593 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857917+0000", 2024-05-08T08:54:31.593 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0764999999999999e-05 2024-05-08T08:54:31.593 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.593 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.593 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:31.593 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906287+0000", 2024-05-08T08:54:31.593 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048370442 2024-05-08T08:54:31.593 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.593 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.593 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:31.593 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906349+0000", 2024-05-08T08:54:31.593 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1537000000000004e-05 2024-05-08T08:54:31.594 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.594 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.594 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:31.594 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906422+0000", 2024-05-08T08:54:31.594 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3431999999999998e-05 2024-05-08T08:54:31.594 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.594 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:31.594 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.594 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.594 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.594 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22095 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 946176~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:31.595 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:54:31.595 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 123.692427632, 2024-05-08T08:54:31.595 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048657593999999998, 2024-05-08T08:54:31.595 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:31.595 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:31.595 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:31.595 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:31.595 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:31.595 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:31.595 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22095 2024-05-08T08:54:31.596 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.596 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:31.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.596 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:31.596 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:54:31.596 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:31.596 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.596 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:31.596 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:54:31.596 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:31.596 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.596 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:31.596 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857925+0000", 2024-05-08T08:54:31.596 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3099999999999998e-07 2024-05-08T08:54:31.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.597 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:31.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857927+0000", 2024-05-08T08:54:31.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.34e-06 2024-05-08T08:54:31.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.597 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:31.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857938+0000", 2024-05-08T08:54:31.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1426e-05 2024-05-08T08:54:31.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:31.598 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857943+0000", 2024-05-08T08:54:31.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5140000000000003e-06 2024-05-08T08:54:31.598 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:31.598 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906448+0000", 2024-05-08T08:54:31.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048504641000000001 2024-05-08T08:54:31.598 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:31.598 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906510+0000", 2024-05-08T08:54:31.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2663000000000006e-05 2024-05-08T08:54:31.598 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.599 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.599 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:31.599 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906582+0000", 2024-05-08T08:54:31.599 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1478999999999998e-05 2024-05-08T08:54:31.599 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.599 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:31.599 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.599 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.599 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.599 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22097 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 962560~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:31.600 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:54:31.600 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 123.692399943, 2024-05-08T08:54:31.600 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048791353000000003, 2024-05-08T08:54:31.600 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:31.600 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:31.600 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:31.600 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:31.600 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:31.600 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:31.600 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22097 2024-05-08T08:54:31.601 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.601 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:31.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.601 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:31.601 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:54:31.601 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:31.601 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.601 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:31.601 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:54:31.601 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:31.601 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.601 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:31.601 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:54:31.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9200000000000002e-07 2024-05-08T08:54:31.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:31.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857954+0000", 2024-05-08T08:54:31.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1900000000000002e-06 2024-05-08T08:54:31.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:31.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857961+0000", 2024-05-08T08:54:31.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4989999999999999e-06 2024-05-08T08:54:31.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:31.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857971+0000", 2024-05-08T08:54:31.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0485999999999999e-05 2024-05-08T08:54:31.603 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.603 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:31.603 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906607+0000", 2024-05-08T08:54:31.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048635109000000003 2024-05-08T08:54:31.603 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.603 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:31.603 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906667+0000", 2024-05-08T08:54:31.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0439000000000002e-05 2024-05-08T08:54:31.603 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.604 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:31.604 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906743+0000", 2024-05-08T08:54:31.604 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6237999999999998e-05 2024-05-08T08:54:31.604 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.604 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:31.604 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.604 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.604 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.604 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22099 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 978944~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:31.604 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:54:31.604 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 123.69237378299999, 2024-05-08T08:54:31.604 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048932055000000002, 2024-05-08T08:54:31.604 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:31.604 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:31.604 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:31.605 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:31.605 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:31.605 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:31.605 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22099 2024-05-08T08:54:31.605 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.605 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:31.605 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.605 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:31.605 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:54:31.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:31.605 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.605 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.605 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:31.605 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:54:31.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:31.605 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.605 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.606 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:31.606 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:54:31.606 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.27e-07 2024-05-08T08:54:31.606 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.606 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:31.606 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857987+0000", 2024-05-08T08:54:31.606 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5890000000000003e-06 2024-05-08T08:54:31.606 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.606 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:31.606 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857994+0000", 2024-05-08T08:54:31.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6619999999999999e-06 2024-05-08T08:54:31.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.607 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:31.607 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857998+0000", 2024-05-08T08:54:31.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7110000000000003e-06 2024-05-08T08:54:31.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.607 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:31.607 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906772+0000", 2024-05-08T08:54:31.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048773415000000001 2024-05-08T08:54:31.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.607 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:31.607 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906834+0000", 2024-05-08T08:54:31.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1752000000000001e-05 2024-05-08T08:54:31.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.608 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:31.608 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906910+0000", 2024-05-08T08:54:31.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6499000000000001e-05 2024-05-08T08:54:31.608 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.608 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:31.608 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.608 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22101 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 995328~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:31.608 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:54:31.608 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 123.692346652, 2024-05-08T08:54:31.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049087364000000001, 2024-05-08T08:54:31.608 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:31.608 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:31.609 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:31.609 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:31.609 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:31.609 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:31.609 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22101 2024-05-08T08:54:31.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.609 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:31.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.609 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:31.609 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:54:31.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:31.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.609 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:31.609 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:54:31.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:31.610 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.610 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:31.610 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858006+0000", 2024-05-08T08:54:31.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1399999999999997e-07 2024-05-08T08:54:31.610 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.610 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:31.610 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858008+0000", 2024-05-08T08:54:31.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2060000000000001e-06 2024-05-08T08:54:31.610 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.611 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:31.611 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858017+0000", 2024-05-08T08:54:31.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1579999999999994e-06 2024-05-08T08:54:31.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.611 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:31.611 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858022+0000", 2024-05-08T08:54:31.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4199999999999998e-06 2024-05-08T08:54:31.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.611 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:31.611 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906940+0000", 2024-05-08T08:54:31.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048917627999999998 2024-05-08T08:54:31.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.612 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:31.612 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907002+0000", 2024-05-08T08:54:31.612 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2399999999999999e-05 2024-05-08T08:54:31.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.612 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:31.612 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907092+0000", 2024-05-08T08:54:31.612 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0137999999999997e-05 2024-05-08T08:54:31.612 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.612 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:31.612 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.612 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22103 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1011712~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:31.612 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:54:31.612 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 123.692321867, 2024-05-08T08:54:31.613 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049228542, 2024-05-08T08:54:31.613 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:31.613 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:31.613 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:31.613 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:31.613 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:31.613 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:31.613 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22103 2024-05-08T08:54:31.613 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.613 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:31.613 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.613 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:31.613 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:54:31.613 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:31.613 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.613 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.614 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:31.614 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:54:31.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:31.614 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.614 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.614 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:31.614 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:54:31.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4299999999999999e-07 2024-05-08T08:54:31.614 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.614 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.614 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:31.614 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858032+0000", 2024-05-08T08:54:31.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0590000000000001e-06 2024-05-08T08:54:31.614 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.614 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.614 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:31.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858037+0000", 2024-05-08T08:54:31.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.977e-06 2024-05-08T08:54:31.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:31.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858042+0000", 2024-05-08T08:54:31.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7650000000000001e-06 2024-05-08T08:54:31.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:31.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907118+0000", 2024-05-08T08:54:31.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049076147000000001 2024-05-08T08:54:31.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:31.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907180+0000", 2024-05-08T08:54:31.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1971999999999996e-05 2024-05-08T08:54:31.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:31.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907258+0000", 2024-05-08T08:54:31.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8279e-05 2024-05-08T08:54:31.616 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.616 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:31.616 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.616 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22105 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1028096~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:31.616 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:54:31.616 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 123.69230262400001, 2024-05-08T08:54:31.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049394609999999999, 2024-05-08T08:54:31.616 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:31.616 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:31.617 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:31.617 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:31.617 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:31.617 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:31.617 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22105 2024-05-08T08:54:31.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.617 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:31.617 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.617 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:31.617 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:54:31.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:31.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.617 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.617 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:31.617 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:54:31.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:31.618 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.618 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.618 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:31.618 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858050+0000", 2024-05-08T08:54:31.618 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5999999999999999e-07 2024-05-08T08:54:31.618 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.618 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.618 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:31.618 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858051+0000", 2024-05-08T08:54:31.618 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8870000000000001e-06 2024-05-08T08:54:31.618 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.618 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.618 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:31.618 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858057+0000", 2024-05-08T08:54:31.618 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8019999999999997e-06 2024-05-08T08:54:31.619 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.619 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.619 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:31.619 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858062+0000", 2024-05-08T08:54:31.619 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.561e-06 2024-05-08T08:54:31.619 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.619 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.619 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:31.619 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907289+0000", 2024-05-08T08:54:31.619 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049226957000000002 2024-05-08T08:54:31.619 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.619 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.619 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:31.619 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907356+0000", 2024-05-08T08:54:31.620 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6915000000000006e-05 2024-05-08T08:54:31.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.620 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:31.620 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907444+0000", 2024-05-08T08:54:31.620 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8127999999999994e-05 2024-05-08T08:54:31.620 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.620 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:31.620 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.621 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22107 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1044480~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:31.621 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:54:31.621 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 123.692284284, 2024-05-08T08:54:31.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049799942999999999, 2024-05-08T08:54:31.621 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:31.621 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:31.621 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:31.621 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:31.621 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:31.622 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:31.622 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22107 2024-05-08T08:54:31.622 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.622 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:31.622 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.622 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:31.622 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:54:31.622 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:31.622 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.622 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.622 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:31.622 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:54:31.622 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:31.622 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.623 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.623 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:31.623 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:54:31.623 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4799999999999999e-07 2024-05-08T08:54:31.623 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.623 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.623 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:31.623 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858070+0000", 2024-05-08T08:54:31.623 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.012e-06 2024-05-08T08:54:31.623 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.623 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.623 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:31.623 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858077+0000", 2024-05-08T08:54:31.623 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6290000000000001e-06 2024-05-08T08:54:31.623 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.624 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:31.624 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858083+0000", 2024-05-08T08:54:31.624 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8880000000000002e-06 2024-05-08T08:54:31.624 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.624 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:31.624 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907671+0000", 2024-05-08T08:54:31.624 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049587191000000003 2024-05-08T08:54:31.624 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.624 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:31.624 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907763+0000", 2024-05-08T08:54:31.624 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2231000000000005e-05 2024-05-08T08:54:31.625 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.625 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.625 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:31.625 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907867+0000", 2024-05-08T08:54:31.625 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104644 2024-05-08T08:54:31.625 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.625 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:31.625 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.625 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.625 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.625 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22294 3.1c 3:3aad0389:::rbd_data.11406886b07b.0000000000000080:head [stat,delete] snapc 7=[7] RETRY=6 ondisk+retry+write+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:31.625 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:54:31.625 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 123.691690436, 2024-05-08T08:54:31.626 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051152191, 2024-05-08T08:54:31.626 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:31.626 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:31.626 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:31.626 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:31.626 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:31.626 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:31.626 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22294 2024-05-08T08:54:31.626 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.626 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:31.627 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.627 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:31.627 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:54:31.627 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:31.627 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.627 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.627 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:31.627 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:54:31.627 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:31.627 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.627 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.627 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:31.627 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858662+0000", 2024-05-08T08:54:31.627 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4799999999999998e-07 2024-05-08T08:54:31.627 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.628 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:31.628 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858664+0000", 2024-05-08T08:54:31.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1229999999999998e-06 2024-05-08T08:54:31.628 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.628 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:31.628 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858670+0000", 2024-05-08T08:54:31.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3099999999999997e-06 2024-05-08T08:54:31.628 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.628 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:31.628 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858676+0000", 2024-05-08T08:54:31.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9750000000000004e-06 2024-05-08T08:54:31.628 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.629 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:31.629 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907901+0000", 2024-05-08T08:54:31.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049224520000000001 2024-05-08T08:54:31.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.629 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:31.629 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.908085+0000", 2024-05-08T08:54:31.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00018377500000000001 2024-05-08T08:54:31.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.629 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:31.629 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.909814+0000", 2024-05-08T08:54:31.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00172894 2024-05-08T08:54:31.629 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.629 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:31.629 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.630 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.630 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4416.0:13461 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-05-08T08:54:31.630 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:22.487456+0000", 2024-05-08T08:54:31.630 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 69.062895394999998, 2024-05-08T08:54:31.630 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.99102620799999996, 2024-05-08T08:54:31.630 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:31.630 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:31.630 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:31.630 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:31.630 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4416", 2024-05-08T08:54:31.630 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:54:31.630 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 13461 2024-05-08T08:54:31.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.630 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:31.631 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.631 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:31.631 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487456+0000", 2024-05-08T08:54:31.631 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:31.631 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.631 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.631 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:31.631 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487456+0000", 2024-05-08T08:54:31.631 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:31.631 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.631 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.631 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:31.631 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487458+0000", 2024-05-08T08:54:31.631 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5659999999999999e-06 2024-05-08T08:54:31.631 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.631 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.632 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:31.632 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487463+0000", 2024-05-08T08:54:31.632 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6890000000000003e-06 2024-05-08T08:54:31.632 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.632 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.632 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:31.632 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487472+0000", 2024-05-08T08:54:31.632 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7710000000000001e-06 2024-05-08T08:54:31.632 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.632 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.632 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:31.633 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487478+0000", 2024-05-08T08:54:31.633 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1390000000000003e-06 2024-05-08T08:54:31.633 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.633 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.633 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:31.633 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487520+0000", 2024-05-08T08:54:31.633 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2259999999999999e-05 2024-05-08T08:54:31.633 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.633 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.633 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-05-08T08:54:31.633 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487528+0000", 2024-05-08T08:54:31.633 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7479999999999993e-06 2024-05-08T08:54:31.633 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.633 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.633 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:31.633 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:23.478139+0000", 2024-05-08T08:54:31.634 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.99061165200000001 2024-05-08T08:54:31.634 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.634 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:31.634 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:23.478301+0000", 2024-05-08T08:54:31.634 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000161602 2024-05-08T08:54:31.634 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.634 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:31.634 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:23.478483+0000", 2024-05-08T08:54:31.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000181781 2024-05-08T08:54:31.635 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.635 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:31.635 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.635 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.635 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5752.0:404 3.1a 3:5f5e8104:::rbd_data.1140bc4a55c7.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 5175296~45056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-05-08T08:54:31.635 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:00.390863+0000", 2024-05-08T08:54:31.635 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 31.159489199999999, 2024-05-08T08:54:31.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.055189174000000001, 2024-05-08T08:54:31.635 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:31.635 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:31.635 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:54:31.635 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:31.635 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5752", 2024-05-08T08:54:31.635 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/3534419815", 2024-05-08T08:54:31.636 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 404 2024-05-08T08:54:31.636 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.636 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:31.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.636 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:31.636 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.390863+0000", 2024-05-08T08:54:31.636 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:31.636 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.636 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:31.636 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.390863+0000", 2024-05-08T08:54:31.636 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:31.636 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.636 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:31.637 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.390864+0000", 2024-05-08T08:54:31.637 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.471e-06 2024-05-08T08:54:31.637 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.637 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.637 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:31.637 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397475+0000", 2024-05-08T08:54:31.637 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0066109109999999997 2024-05-08T08:54:31.637 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.637 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.637 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:31.637 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397498+0000", 2024-05-08T08:54:31.637 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2674e-05 2024-05-08T08:54:31.637 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.637 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.637 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:31.637 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397508+0000", 2024-05-08T08:54:31.638 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9019999999999994e-06 2024-05-08T08:54:31.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.638 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.638 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:31.638 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397609+0000", 2024-05-08T08:54:31.638 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010121899999999999 2024-05-08T08:54:31.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.638 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.638 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:31.638 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397711+0000", 2024-05-08T08:54:31.638 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102489 2024-05-08T08:54:31.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.638 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.638 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-05-08T08:54:31.638 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397999+0000", 2024-05-08T08:54:31.638 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00028733799999999999 2024-05-08T08:54:31.639 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.639 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.639 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:54:31.639 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.432093+0000", 2024-05-08T08:54:31.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.034094865000000002 2024-05-08T08:54:31.639 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.639 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.639 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:54:31.639 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.445975+0000", 2024-05-08T08:54:31.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013881187 2024-05-08T08:54:31.639 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.639 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.639 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:54:31.639 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.446013+0000", 2024-05-08T08:54:31.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8155000000000003e-05 2024-05-08T08:54:31.639 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:31.639 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:31.640 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:31.640 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.446052+0000", 2024-05-08T08:54:31.640 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8962999999999999e-05 2024-05-08T08:54:31.640 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.640 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:31.640 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.640 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:31.640 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:31.640 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:31.641 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:54:31.642 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:31.645+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:31.742 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:31.745+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:31.788 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:31.788 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:54:31.788 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:54:31.788 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:31.799 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:54:31.843 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:31.846+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:31.944 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:31.946+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:31.953 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:31.953 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:54:31.953 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:54:31.953 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:54:31.953 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:31.962 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:54:32.045 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:32.048+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:32.108 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:32.109 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:54:32.109 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:54:32.109 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:54:32.109 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.109 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6690 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1093632~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:32.109 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:54:32.109 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 120.39537585799999, 2024-05-08T08:54:32.109 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052462754, 2024-05-08T08:54:32.109 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:32.109 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:32.109 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:32.110 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:32.110 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:32.110 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:32.110 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6690 2024-05-08T08:54:32.110 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.110 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:32.110 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.110 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:32.110 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:54:32.110 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:32.110 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.110 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.110 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:32.110 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:54:32.111 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:32.111 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.111 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.111 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:32.111 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715346+0000", 2024-05-08T08:54:32.111 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.84e-07 2024-05-08T08:54:32.111 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.111 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.111 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:32.111 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715348+0000", 2024-05-08T08:54:32.111 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.114e-06 2024-05-08T08:54:32.111 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.111 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.111 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:32.111 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715353+0000", 2024-05-08T08:54:32.111 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5720000000000002e-06 2024-05-08T08:54:32.112 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.112 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.112 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:32.112 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715358+0000", 2024-05-08T08:54:32.112 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7140000000000001e-06 2024-05-08T08:54:32.112 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.112 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.112 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:32.112 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767638+0000", 2024-05-08T08:54:32.112 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052279562000000002 2024-05-08T08:54:32.112 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.112 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.112 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:32.112 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767713+0000", 2024-05-08T08:54:32.112 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5652e-05 2024-05-08T08:54:32.113 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.113 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.113 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:32.113 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767808+0000", 2024-05-08T08:54:32.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4755999999999999e-05 2024-05-08T08:54:32.113 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:32.113 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:32.113 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:32.113 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.113 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.113 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6692 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1110016~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:32.113 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:54:32.113 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 120.395356768, 2024-05-08T08:54:32.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052666495000000001, 2024-05-08T08:54:32.114 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:32.114 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:32.114 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:32.114 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:32.114 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:32.114 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:32.114 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6692 2024-05-08T08:54:32.114 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.114 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:32.114 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.114 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:32.114 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:54:32.114 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:32.114 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.114 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.115 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:32.115 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:54:32.115 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:32.115 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.115 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.115 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:32.115 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:54:32.115 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8599999999999999e-07 2024-05-08T08:54:32.115 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.115 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.115 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:32.115 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715372+0000", 2024-05-08T08:54:32.115 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0990000000000001e-06 2024-05-08T08:54:32.115 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.115 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.115 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:32.116 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715378+0000", 2024-05-08T08:54:32.116 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0680000000000002e-06 2024-05-08T08:54:32.116 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.116 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.116 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:32.116 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715383+0000", 2024-05-08T08:54:32.116 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8740000000000003e-06 2024-05-08T08:54:32.116 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.116 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.116 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:32.116 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767848+0000", 2024-05-08T08:54:32.116 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052464668999999998 2024-05-08T08:54:32.116 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.116 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.116 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:32.117 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767927+0000", 2024-05-08T08:54:32.117 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9228999999999996e-05 2024-05-08T08:54:32.117 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.117 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.117 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:32.117 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768031+0000", 2024-05-08T08:54:32.117 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010417 2024-05-08T08:54:32.117 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:32.117 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:32.117 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:32.117 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.117 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.117 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6694 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1126400~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:32.117 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:54:32.117 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 120.395331871, 2024-05-08T08:54:32.117 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052848829, 2024-05-08T08:54:32.118 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:32.118 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:32.118 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:32.118 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:32.118 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:32.118 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:32.118 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6694 2024-05-08T08:54:32.118 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.118 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:32.118 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.118 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:32.119 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:54:32.119 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:32.119 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.119 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.119 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:32.119 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:54:32.119 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:32.119 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.119 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.119 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:32.119 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715390+0000", 2024-05-08T08:54:32.120 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9900000000000001e-07 2024-05-08T08:54:32.120 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.120 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.120 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:32.120 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715392+0000", 2024-05-08T08:54:32.120 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8279999999999999e-06 2024-05-08T08:54:32.120 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.120 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.120 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:32.120 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715398+0000", 2024-05-08T08:54:32.120 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7619999999999998e-06 2024-05-08T08:54:32.120 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.120 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.120 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:32.120 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715403+0000", 2024-05-08T08:54:32.121 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7790000000000003e-06 2024-05-08T08:54:32.121 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.121 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.121 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:32.121 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768067+0000", 2024-05-08T08:54:32.121 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052663785999999997 2024-05-08T08:54:32.121 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.121 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.122 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:32.122 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768142+0000", 2024-05-08T08:54:32.122 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5018999999999994e-05 2024-05-08T08:54:32.122 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.122 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.122 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:32.122 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768238+0000", 2024-05-08T08:54:32.122 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6255999999999995e-05 2024-05-08T08:54:32.122 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:32.122 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:32.122 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:32.122 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.122 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.123 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6696 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1142784~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:32.123 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:54:32.123 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 120.395312363, 2024-05-08T08:54:32.123 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.053051057999999998, 2024-05-08T08:54:32.123 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:32.123 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:32.123 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:32.123 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:32.123 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:32.123 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:32.123 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6696 2024-05-08T08:54:32.123 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.123 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:32.123 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.123 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:32.124 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:54:32.124 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:32.124 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.124 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.124 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:32.124 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:54:32.124 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:32.124 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.124 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.124 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:32.124 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:54:32.124 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-05-08T08:54:32.124 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.124 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.124 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:32.125 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715411+0000", 2024-05-08T08:54:32.125 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9630000000000001e-06 2024-05-08T08:54:32.125 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.125 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.125 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:32.125 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715417+0000", 2024-05-08T08:54:32.125 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7250000000000002e-06 2024-05-08T08:54:32.125 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.125 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.125 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:32.125 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715421+0000", 2024-05-08T08:54:32.125 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4089999999999996e-06 2024-05-08T08:54:32.125 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.126 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.126 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:32.126 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768277+0000", 2024-05-08T08:54:32.126 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052855293999999997 2024-05-08T08:54:32.126 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.126 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.126 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:32.126 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768358+0000", 2024-05-08T08:54:32.126 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1720999999999999e-05 2024-05-08T08:54:32.126 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.126 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.127 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:32.127 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768460+0000", 2024-05-08T08:54:32.127 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010154600000000001 2024-05-08T08:54:32.127 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:32.127 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:32.127 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:32.127 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.127 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.127 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6698 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1159168~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:32.127 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:54:32.127 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 120.39529349599999, 2024-05-08T08:54:32.127 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.053245726, 2024-05-08T08:54:32.127 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:32.127 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:32.127 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:32.128 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:32.128 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:32.128 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:32.128 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6698 2024-05-08T08:54:32.128 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.128 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:32.128 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.128 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:32.128 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:54:32.128 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:32.128 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.128 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.128 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:32.128 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:54:32.128 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:32.129 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.129 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.129 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:32.129 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716936+0000", 2024-05-08T08:54:32.129 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001508665 2024-05-08T08:54:32.129 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.129 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.129 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:32.129 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716941+0000", 2024-05-08T08:54:32.129 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1910000000000002e-06 2024-05-08T08:54:32.129 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.130 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.130 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:32.130 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716950+0000", 2024-05-08T08:54:32.130 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1900000000000001e-06 2024-05-08T08:54:32.130 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.130 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.130 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:32.130 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716958+0000", 2024-05-08T08:54:32.130 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8019999999999992e-06 2024-05-08T08:54:32.130 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.130 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.130 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:32.130 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768499+0000", 2024-05-08T08:54:32.130 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051540926000000001 2024-05-08T08:54:32.130 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.131 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.131 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:32.131 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768574+0000", 2024-05-08T08:54:32.131 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4988999999999996e-05 2024-05-08T08:54:32.131 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.131 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.131 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:32.131 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768673+0000", 2024-05-08T08:54:32.131 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9963000000000005e-05 2024-05-08T08:54:32.131 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:32.131 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:32.131 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:32.131 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.131 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.132 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6706 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1224704~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:32.132 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:54:32.132 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 120.393688189, 2024-05-08T08:54:32.132 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052435598, 2024-05-08T08:54:32.132 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:32.132 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:32.132 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:32.132 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:32.132 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:32.132 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:32.132 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6706 2024-05-08T08:54:32.133 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.133 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:32.133 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.133 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:32.133 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:54:32.133 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:32.133 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.133 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.133 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:32.133 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:54:32.133 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:32.133 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.133 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.133 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:32.133 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:54:32.133 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:54:32.134 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.134 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.134 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:32.134 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717035+0000", 2024-05-08T08:54:32.134 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9290000000000001e-06 2024-05-08T08:54:32.134 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.134 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.134 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:32.134 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717041+0000", 2024-05-08T08:54:32.134 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6720000000000001e-06 2024-05-08T08:54:32.134 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.134 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.134 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:32.134 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717046+0000", 2024-05-08T08:54:32.134 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5650000000000003e-06 2024-05-08T08:54:32.135 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.135 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.135 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:32.135 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769311+0000", 2024-05-08T08:54:32.135 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052265645999999999 2024-05-08T08:54:32.135 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.135 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.135 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:32.135 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769379+0000", 2024-05-08T08:54:32.135 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7391000000000006e-05 2024-05-08T08:54:32.135 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.135 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.135 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:32.135 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769469+0000", 2024-05-08T08:54:32.135 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0025999999999995e-05 2024-05-08T08:54:32.135 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:32.136 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:32.136 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:32.136 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.136 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.136 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6708 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1241088~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:32.136 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:54:32.136 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 120.393670085, 2024-05-08T08:54:32.136 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052640552, 2024-05-08T08:54:32.136 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:32.136 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:32.136 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:32.136 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:32.136 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:32.136 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:32.136 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6708 2024-05-08T08:54:32.137 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.137 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:32.137 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.137 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:32.137 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:54:32.137 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:32.137 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.137 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.137 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:32.137 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:54:32.137 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:32.137 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.137 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.137 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:32.137 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717052+0000", 2024-05-08T08:54:32.138 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:54:32.138 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.138 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.138 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:32.138 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717053+0000", 2024-05-08T08:54:32.138 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8330000000000001e-06 2024-05-08T08:54:32.138 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.138 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.138 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:32.138 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717058+0000", 2024-05-08T08:54:32.138 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.933e-06 2024-05-08T08:54:32.139 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.139 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.139 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:32.139 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717063+0000", 2024-05-08T08:54:32.139 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2180000000000001e-06 2024-05-08T08:54:32.139 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.139 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.139 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:32.139 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769507+0000", 2024-05-08T08:54:32.139 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052444866999999999 2024-05-08T08:54:32.139 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.140 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.140 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:32.140 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769589+0000", 2024-05-08T08:54:32.140 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2039000000000005e-05 2024-05-08T08:54:32.140 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.140 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.140 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:32.140 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769692+0000", 2024-05-08T08:54:32.140 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102252 2024-05-08T08:54:32.140 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:32.140 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:32.140 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:32.140 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.140 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.140 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6710 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1257472~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:32.141 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:54:32.141 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 120.393652061, 2024-05-08T08:54:32.141 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.05283057, 2024-05-08T08:54:32.141 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:32.141 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:32.141 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:32.141 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:32.141 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:32.141 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:32.141 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6710 2024-05-08T08:54:32.141 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.141 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:32.141 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.141 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:32.141 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:54:32.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:32.142 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.142 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.142 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:32.142 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:54:32.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:32.142 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.142 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.142 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:32.142 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718951+0000", 2024-05-08T08:54:32.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0018817809999999999 2024-05-08T08:54:32.142 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.142 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.142 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:32.142 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718956+0000", 2024-05-08T08:54:32.143 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5399999999999997e-06 2024-05-08T08:54:32.143 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.143 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.143 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:32.143 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718964+0000", 2024-05-08T08:54:32.143 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6629999999999999e-06 2024-05-08T08:54:32.143 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.143 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.143 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:32.143 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718972+0000", 2024-05-08T08:54:32.143 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5229999999999998e-06 2024-05-08T08:54:32.143 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.143 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.143 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:32.143 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769729+0000", 2024-05-08T08:54:32.143 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050756959999999997 2024-05-08T08:54:32.144 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.144 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.144 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:32.144 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769803+0000", 2024-05-08T08:54:32.144 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4538999999999999e-05 2024-05-08T08:54:32.144 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.144 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.144 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:32.144 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769900+0000", 2024-05-08T08:54:32.144 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6564000000000006e-05 2024-05-08T08:54:32.144 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:32.144 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:32.144 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:32.144 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.144 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.145 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6726 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1388544~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:32.145 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:54:32.145 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 120.391577289, 2024-05-08T08:54:32.145 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052451506000000002, 2024-05-08T08:54:32.145 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:32.145 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:32.145 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:32.145 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:32.145 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:32.145 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:32.145 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6726 2024-05-08T08:54:32.145 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.145 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:32.145 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.145 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:32.146 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:54:32.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:32.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.146 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.146 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:32.146 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:54:32.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:32.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.146 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.147 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:32.147 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719200+0000", 2024-05-08T08:54:32.147 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5865999999999997e-05 2024-05-08T08:54:32.147 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.147 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.147 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:32.147 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719204+0000", 2024-05-08T08:54:32.147 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8630000000000001e-06 2024-05-08T08:54:32.147 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.147 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.147 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:32.147 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719215+0000", 2024-05-08T08:54:32.147 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1590000000000001e-05 2024-05-08T08:54:32.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.148 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.148 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:32.148 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719220+0000", 2024-05-08T08:54:32.148 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1619999999999999e-06 2024-05-08T08:54:32.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.148 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.148 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:32.148 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771425+0000", 2024-05-08T08:54:32.148 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052204249000000001 2024-05-08T08:54:32.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.148 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.148 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:32.148 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771498+0000", 2024-05-08T08:54:32.148 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3782999999999995e-05 2024-05-08T08:54:32.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.148 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.149 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:32.149 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771595+0000", 2024-05-08T08:54:32.149 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6992999999999998e-05 2024-05-08T08:54:32.149 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:32.149 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:32.149 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:32.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.149 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6728 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1404928~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:32.149 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:54:32.149 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 120.39148988399999, 2024-05-08T08:54:32.149 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052590747, 2024-05-08T08:54:32.149 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:32.149 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:32.149 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:32.149 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:32.150 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:32.150 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:32.150 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6728 2024-05-08T08:54:32.150 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.150 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:32.150 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.150 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:32.150 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:54:32.150 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:32.150 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.151 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.151 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:32.151 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:54:32.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:32.151 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.151 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.151 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:32.151 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719232+0000", 2024-05-08T08:54:32.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4000000000000001e-07 2024-05-08T08:54:32.151 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.151 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.151 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:32.152 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719234+0000", 2024-05-08T08:54:32.152 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4499999999999998e-06 2024-05-08T08:54:32.152 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.152 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.152 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:32.152 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719240+0000", 2024-05-08T08:54:32.152 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5720000000000002e-06 2024-05-08T08:54:32.152 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.152 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.152 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:32.152 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719245+0000", 2024-05-08T08:54:32.152 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8690000000000003e-06 2024-05-08T08:54:32.152 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.152 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.152 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:32.152 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771636+0000", 2024-05-08T08:54:32.152 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052390979999999997 2024-05-08T08:54:32.153 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.153 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.153 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:32.153 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771717+0000", 2024-05-08T08:54:32.153 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1291999999999993e-05 2024-05-08T08:54:32.153 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.153 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.153 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:32.153 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771822+0000", 2024-05-08T08:54:32.153 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104944 2024-05-08T08:54:32.153 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:32.153 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:32.153 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:32.153 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.154 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.154 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6730 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1421312~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:32.154 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:54:32.154 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 120.391469642, 2024-05-08T08:54:32.154 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052770477000000003, 2024-05-08T08:54:32.154 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:32.154 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:32.154 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:32.154 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:32.154 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:32.154 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:32.154 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6730 2024-05-08T08:54:32.154 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.154 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:32.154 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.154 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:32.155 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:54:32.155 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:32.155 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.155 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.155 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:32.155 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:54:32.155 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:32.155 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.155 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.155 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:32.155 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:54:32.155 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9000000000000002e-07 2024-05-08T08:54:32.155 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.155 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.155 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:32.156 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719254+0000", 2024-05-08T08:54:32.156 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4099999999999998e-06 2024-05-08T08:54:32.156 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.156 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.156 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:32.156 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719263+0000", 2024-05-08T08:54:32.156 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4109999999999996e-06 2024-05-08T08:54:32.156 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.156 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.156 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:32.156 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719268+0000", 2024-05-08T08:54:32.156 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2669999999999997e-06 2024-05-08T08:54:32.156 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.156 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.156 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:32.156 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771846+0000", 2024-05-08T08:54:32.157 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052577677000000003 2024-05-08T08:54:32.157 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.157 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.157 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:32.157 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771920+0000", 2024-05-08T08:54:32.157 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4460000000000002e-05 2024-05-08T08:54:32.157 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.157 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.157 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:32.157 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772022+0000", 2024-05-08T08:54:32.157 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010186199999999999 2024-05-08T08:54:32.157 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:32.157 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:32.157 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:32.157 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.157 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.157 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6732 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1437696~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:32.157 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:54:32.158 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 120.391444835, 2024-05-08T08:54:32.158 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052905575000000003, 2024-05-08T08:54:32.158 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:32.158 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:32.158 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:32.158 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:32.158 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:32.158 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:32.158 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6732 2024-05-08T08:54:32.158 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.158 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:32.158 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.158 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:32.158 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:54:32.158 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:32.159 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.159 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.159 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:32.159 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:54:32.159 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:32.159 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.159 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.159 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:32.159 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721392+0000", 2024-05-08T08:54:32.159 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002115287 2024-05-08T08:54:32.159 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.159 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.159 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:32.159 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721396+0000", 2024-05-08T08:54:32.159 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1679999999999997e-06 2024-05-08T08:54:32.160 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.160 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.160 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:32.160 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721404+0000", 2024-05-08T08:54:32.160 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4409999999999993e-06 2024-05-08T08:54:32.160 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.160 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.160 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:32.160 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721410+0000", 2024-05-08T08:54:32.160 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9150000000000001e-06 2024-05-08T08:54:32.160 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.160 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.161 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:32.161 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772061+0000", 2024-05-08T08:54:32.161 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050651068 2024-05-08T08:54:32.161 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.161 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.161 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:32.161 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772127+0000", 2024-05-08T08:54:32.161 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5424000000000002e-05 2024-05-08T08:54:32.161 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.161 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.161 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:32.161 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772182+0000", 2024-05-08T08:54:32.161 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5272000000000002e-05 2024-05-08T08:54:32.161 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:32.161 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:32.161 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:32.162 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.162 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.162 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6749 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1585152~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:32.162 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:54:32.162 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 120.389144065, 2024-05-08T08:54:32.162 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052350264000000001, 2024-05-08T08:54:32.162 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:32.162 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:32.162 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:32.162 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:32.162 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:32.162 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:32.162 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6749 2024-05-08T08:54:32.162 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.162 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:32.162 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.162 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:32.163 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:54:32.163 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:32.163 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.163 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.163 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:32.163 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:54:32.163 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:32.163 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.163 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.163 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:32.163 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721578+0000", 2024-05-08T08:54:32.163 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0399999999999996e-07 2024-05-08T08:54:32.163 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.163 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.163 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:32.164 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721580+0000", 2024-05-08T08:54:32.164 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5450000000000002e-06 2024-05-08T08:54:32.164 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.164 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.164 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:32.164 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721587+0000", 2024-05-08T08:54:32.164 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0190000000000001e-06 2024-05-08T08:54:32.164 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.164 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.164 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:32.164 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721593+0000", 2024-05-08T08:54:32.164 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5659999999999997e-06 2024-05-08T08:54:32.164 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.164 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.164 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:32.164 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773759+0000", 2024-05-08T08:54:32.165 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052166553999999997 2024-05-08T08:54:32.165 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.165 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.165 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:32.165 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773833+0000", 2024-05-08T08:54:32.165 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3913999999999997e-05 2024-05-08T08:54:32.165 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.165 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.165 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:32.165 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773927+0000", 2024-05-08T08:54:32.165 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4061999999999995e-05 2024-05-08T08:54:32.165 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:32.165 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:32.165 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:32.165 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.165 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.166 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6751 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1601536~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:32.166 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:54:32.166 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 120.38912078600001, 2024-05-08T08:54:32.166 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052461302000000001, 2024-05-08T08:54:32.166 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:32.166 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:32.166 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:32.166 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:32.166 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:32.166 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:32.166 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6751 2024-05-08T08:54:32.166 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.166 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:32.166 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.166 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:32.166 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:54:32.167 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:32.167 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.167 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.167 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:32.167 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:54:32.167 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:32.167 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.167 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.167 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:32.167 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722939+0000", 2024-05-08T08:54:32.167 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001338601 2024-05-08T08:54:32.167 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.167 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.167 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:32.167 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722944+0000", 2024-05-08T08:54:32.167 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4469999999999999e-06 2024-05-08T08:54:32.168 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.168 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.168 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:32.168 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722955+0000", 2024-05-08T08:54:32.168 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1892999999999999e-05 2024-05-08T08:54:32.168 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.168 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.168 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:32.168 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722963+0000", 2024-05-08T08:54:32.168 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1080000000000004e-06 2024-05-08T08:54:32.168 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.168 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.168 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:32.168 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773964+0000", 2024-05-08T08:54:32.168 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051001022 2024-05-08T08:54:32.168 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.169 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.169 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:32.169 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.774006+0000", 2024-05-08T08:54:32.169 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2729999999999999e-05 2024-05-08T08:54:32.169 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.169 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.169 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:32.169 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.774062+0000", 2024-05-08T08:54:32.169 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5501000000000003e-05 2024-05-08T08:54:32.169 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:32.169 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:32.169 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:32.169 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.169 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.169 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6760 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1683456~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:32.169 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:54:32.170 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 120.38761141000001, 2024-05-08T08:54:32.170 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052571312000000002, 2024-05-08T08:54:32.170 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:32.170 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:32.170 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:32.170 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:32.170 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:32.170 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:32.170 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6760 2024-05-08T08:54:32.170 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.170 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:32.170 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.170 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:32.170 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:54:32.170 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:32.170 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.171 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.171 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:32.171 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:54:32.171 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:32.171 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.171 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.171 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:32.171 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:54:32.171 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2100000000000002e-07 2024-05-08T08:54:32.171 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.171 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.171 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:32.171 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723113+0000", 2024-05-08T08:54:32.171 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3149999999999999e-06 2024-05-08T08:54:32.171 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.171 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.172 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:32.172 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723118+0000", 2024-05-08T08:54:32.172 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8860000000000001e-06 2024-05-08T08:54:32.172 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.172 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.172 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:32.172 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723123+0000", 2024-05-08T08:54:32.172 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6770000000000003e-06 2024-05-08T08:54:32.172 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.172 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.172 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:32.172 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775414+0000", 2024-05-08T08:54:32.172 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052290692999999999 2024-05-08T08:54:32.172 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.172 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.173 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:32.173 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775459+0000", 2024-05-08T08:54:32.173 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5639999999999997e-05 2024-05-08T08:54:32.173 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.173 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.173 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:32.173 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775681+0000", 2024-05-08T08:54:32.173 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00022168000000000001 2024-05-08T08:54:32.173 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:32.173 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:32.173 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:32.173 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.173 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.173 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6761 3.16 3:6bd6d6da:::rbd_data.1140782590a5.0000000000000015:6 [read 1095680~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:32.173 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:54:32.173 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 120.38759219000001, 2024-05-08T08:54:32.173 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052751275, 2024-05-08T08:54:32.173 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:32.174 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:32.174 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:32.174 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:32.174 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:32.174 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:32.174 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6761 2024-05-08T08:54:32.174 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.174 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:32.174 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.174 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:32.174 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:54:32.174 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:32.174 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.174 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.174 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:32.174 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:54:32.175 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:32.175 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.175 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.175 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:32.175 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724939+0000", 2024-05-08T08:54:32.175 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001809817 2024-05-08T08:54:32.175 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.175 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.175 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:32.175 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724943+0000", 2024-05-08T08:54:32.175 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4920000000000004e-06 2024-05-08T08:54:32.175 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.175 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.175 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:32.175 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724956+0000", 2024-05-08T08:54:32.175 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2427e-05 2024-05-08T08:54:32.175 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.176 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.176 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:32.176 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724963+0000", 2024-05-08T08:54:32.176 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2559999999999996e-06 2024-05-08T08:54:32.176 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.176 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.176 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:32.176 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775711+0000", 2024-05-08T08:54:32.176 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050747621 2024-05-08T08:54:32.176 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.176 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.176 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:32.176 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775785+0000", 2024-05-08T08:54:32.176 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4634000000000004e-05 2024-05-08T08:54:32.176 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.177 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.177 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:32.177 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775880+0000", 2024-05-08T08:54:32.177 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5027999999999999e-05 2024-05-08T08:54:32.177 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:32.177 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:32.177 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:32.177 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.177 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.177 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6770 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1765376~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:32.177 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:54:32.177 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 120.385561095, 2024-05-08T08:54:32.177 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052339296, 2024-05-08T08:54:32.177 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:32.177 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:32.177 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:32.177 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:32.178 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:32.178 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:32.178 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6770 2024-05-08T08:54:32.178 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.178 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:32.178 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.178 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:32.178 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:54:32.178 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:32.178 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.178 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.178 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:32.178 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:54:32.178 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:32.178 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.178 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.179 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:32.179 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725161+0000", 2024-05-08T08:54:32.179 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-05-08T08:54:32.179 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.179 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.179 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:32.179 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725163+0000", 2024-05-08T08:54:32.179 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4370000000000001e-06 2024-05-08T08:54:32.179 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.179 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.179 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:32.179 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725169+0000", 2024-05-08T08:54:32.179 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8409999999999996e-06 2024-05-08T08:54:32.179 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.179 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.179 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:32.179 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725174+0000", 2024-05-08T08:54:32.180 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7249999999999997e-06 2024-05-08T08:54:32.180 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.180 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.180 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:32.180 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777343+0000", 2024-05-08T08:54:32.180 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052169633999999999 2024-05-08T08:54:32.180 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.180 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.180 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:32.180 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777409+0000", 2024-05-08T08:54:32.180 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6033999999999998e-05 2024-05-08T08:54:32.180 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.180 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.180 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:32.180 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777499+0000", 2024-05-08T08:54:32.180 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0224999999999998e-05 2024-05-08T08:54:32.180 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:32.181 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:32.181 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:32.181 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.181 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.181 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6771 3.16 3:6bd6d6da:::rbd_data.1140782590a5.0000000000000015:6 [read 1177600~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:32.181 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:54:32.181 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 120.38554127800001, 2024-05-08T08:54:32.181 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052438604999999999, 2024-05-08T08:54:32.181 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:32.181 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:32.181 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:32.181 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:32.181 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:32.181 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:32.181 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6771 2024-05-08T08:54:32.181 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.182 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:32.182 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.182 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:32.182 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:54:32.182 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:32.182 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.182 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.182 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:32.182 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:54:32.182 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:32.182 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.182 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.182 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:32.182 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:54:32.182 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:54:32.182 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.183 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.183 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:32.183 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725188+0000", 2024-05-08T08:54:32.183 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1579999999999999e-06 2024-05-08T08:54:32.183 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.183 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.183 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:32.183 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725195+0000", 2024-05-08T08:54:32.183 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8909999999999995e-06 2024-05-08T08:54:32.183 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.183 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.183 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:32.183 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725200+0000", 2024-05-08T08:54:32.183 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9810000000000003e-06 2024-05-08T08:54:32.183 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.183 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.183 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:32.184 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777527+0000", 2024-05-08T08:54:32.184 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052326582000000003 2024-05-08T08:54:32.184 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.184 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.184 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:32.184 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777566+0000", 2024-05-08T08:54:32.184 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9447999999999999e-05 2024-05-08T08:54:32.184 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.184 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.184 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:32.184 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777619+0000", 2024-05-08T08:54:32.184 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2135000000000001e-05 2024-05-08T08:54:32.184 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:32.184 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:32.184 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:32.184 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.184 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.185 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6772 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1781760~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:32.185 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:54:32.185 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 120.385513221, 2024-05-08T08:54:32.185 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052601799999999997, 2024-05-08T08:54:32.185 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:32.185 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:32.185 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:32.185 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:32.185 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:32.185 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:32.185 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6772 2024-05-08T08:54:32.185 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.185 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:32.185 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.185 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:32.185 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:54:32.186 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:32.186 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.186 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.186 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:32.186 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:54:32.186 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:32.186 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.186 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.186 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:32.186 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726939+0000", 2024-05-08T08:54:32.186 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001730493 2024-05-08T08:54:32.186 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.186 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.186 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:32.186 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726943+0000", 2024-05-08T08:54:32.186 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7330000000000003e-06 2024-05-08T08:54:32.186 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.187 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.187 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:32.187 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726954+0000", 2024-05-08T08:54:32.187 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1107000000000001e-05 2024-05-08T08:54:32.187 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.187 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.187 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:32.187 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726962+0000", 2024-05-08T08:54:32.187 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.503e-06 2024-05-08T08:54:32.187 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.187 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.187 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:32.187 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777653+0000", 2024-05-08T08:54:32.187 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050691542999999999 2024-05-08T08:54:32.187 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.187 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.187 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:32.188 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777719+0000", 2024-05-08T08:54:32.188 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5228000000000006e-05 2024-05-08T08:54:32.188 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.188 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.188 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:32.188 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777810+0000", 2024-05-08T08:54:32.188 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1193000000000006e-05 2024-05-08T08:54:32.188 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:32.188 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:32.188 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:32.188 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.188 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.188 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6782 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1880064~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:32.188 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:54:32.188 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 120.383544714, 2024-05-08T08:54:32.188 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052326492000000002, 2024-05-08T08:54:32.188 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:32.189 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:32.189 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:32.189 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:32.189 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:32.189 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:32.189 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6782 2024-05-08T08:54:32.189 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.189 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:32.189 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.189 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:32.189 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:54:32.189 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:32.189 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.189 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.189 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:32.189 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:54:32.189 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:32.190 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.190 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.190 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:32.190 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728939+0000", 2024-05-08T08:54:32.190 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001762719 2024-05-08T08:54:32.190 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.190 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.190 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:32.190 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728944+0000", 2024-05-08T08:54:32.190 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.746e-06 2024-05-08T08:54:32.190 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.190 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.190 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:32.190 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728955+0000", 2024-05-08T08:54:32.190 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1043000000000001e-05 2024-05-08T08:54:32.190 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.190 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.191 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:32.191 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728963+0000", 2024-05-08T08:54:32.191 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5809999999999999e-06 2024-05-08T08:54:32.191 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.191 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.191 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:32.191 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779320+0000", 2024-05-08T08:54:32.191 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050357290999999998 2024-05-08T08:54:32.191 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.191 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.191 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:32.191 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779401+0000", 2024-05-08T08:54:32.191 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0574000000000005e-05 2024-05-08T08:54:32.191 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:32.191 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:32.191 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:32.191 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779503+0000", 2024-05-08T08:54:32.192 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102538 2024-05-08T08:54:32.192 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:32.192 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:32.192 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:32.192 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:32.192 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:32.192 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:32.193 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:54:32.194 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:32.196+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:32.294 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:32.296+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:32.310 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:32.310 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:54:32.310 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:54:32.310 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:32.319 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:54:32.395 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:32.397+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:32.396 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T08:54:32.470 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:32.470 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:54:32.471 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:54:32.471 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:54:32.471 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:32.480 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:54:32.496 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:32.499+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:32.530 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:54:32.531 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:54:32.549 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:54:32.549 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:54:32.567 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:54:32.567 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:54:32.585 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:54:32.585 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:54:32.593 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:32.593 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:32.596 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:32.599+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:32.602 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:54:32.603 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:54:32.603 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:54:32.620 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:54:32.621 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:54:32.698 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:32.701+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:32.713 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:32.713 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:32.721 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:54:32.794 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-05-08T08:54:32.838 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:32.841+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:32.839 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:32.839 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:32.849 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:54:32.933 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:32.936+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:32.999 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:33.002+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:33.002 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:33.003 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:33.012 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:54:33.101 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:33.103+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:33.162 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:33.162 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:33.170 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:54:33.214 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:33.217+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:33.280 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:33.280 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:33.288 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:54:33.329 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:33.332+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:33.404 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:33.406+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:33.438 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:33.438 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:33.447 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:54:33.505 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:33.508+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:33.555 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:33.555 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:33.564 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:54:33.606 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:33.609+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:33.667 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:33.668 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:33.676 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:54:33.720 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:33.722+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:33.781 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:33.782 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:33.790 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4] out_osds: [5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-05-08T08:54:33.790 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-05-08T08:54:33.793 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:54:33.807 INFO:teuthology.orchestra.run.smithi059.stderr:nodeep-scrub is unset 2024-05-08T08:54:33.807 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:33.810+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:33.908 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:33.911+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:33.948 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:33.949 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:33.958 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:54:34.009 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:34.010+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:34.062 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:34.063 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:34.071 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:54:34.112 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:34.115+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:34.210 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:34.212+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:34.217 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:34.217 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:34.226 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:54:34.311 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:34.313+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:34.371 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:34.372 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:34.380 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:54:34.421 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:34.423+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:34.513 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:34.516+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:34.525 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:34.526 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:34.534 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:54:34.614 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:34.617+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:34.633 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T08:54:34.639 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:34.639 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:34.648 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:54:34.715 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:34.718+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:34.764 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:54:34.765 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:54:34.783 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:54:34.783 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:54:34.790 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:34.790 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:54:34.791 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:54:34.791 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:34.799 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:54:34.800 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:54:34.800 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:54:34.802 INFO:teuthology.orchestra.run.smithi059.stderr:reweighted osd.0 to 1 (10000) 2024-05-08T08:54:34.816 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:34.818+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:34.824 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-05-08T08:54:34.825 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:54:34.826 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:54:34.845 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:54:34.845 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:54:34.864 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:54:34.864 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:54:34.919 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:34.922+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:34.926 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:34.926 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:54:34.927 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:54:34.927 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:54:34.927 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:34.937 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:54:35.018 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:35.020+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:35.090 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:35.090 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:54:35.091 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:54:35.091 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:54:35.091 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.091 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4621.0:9089 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 23339520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:54:35.091 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:54:35.091 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 122.598379059, 2024-05-08T08:54:35.091 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.13814907200000001, 2024-05-08T08:54:35.091 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:35.091 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:35.091 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:54:35.091 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:35.091 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4621", 2024-05-08T08:54:35.091 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:35.091 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9089 2024-05-08T08:54:35.091 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.092 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:35.092 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.092 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:35.092 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:54:35.092 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:35.092 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.092 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.092 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:35.092 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:54:35.092 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:35.092 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.092 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.092 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:35.092 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493005+0000", 2024-05-08T08:54:35.092 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1700000000000001e-06 2024-05-08T08:54:35.093 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.093 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.093 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:35.093 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.513963+0000", 2024-05-08T08:54:35.093 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.020957581999999999 2024-05-08T08:54:35.093 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.093 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.093 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:35.093 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.513995+0000", 2024-05-08T08:54:35.093 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1993999999999999e-05 2024-05-08T08:54:35.093 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.093 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.093 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:35.093 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514007+0000", 2024-05-08T08:54:35.093 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1581e-05 2024-05-08T08:54:35.094 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.094 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.094 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:35.094 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514122+0000", 2024-05-08T08:54:35.094 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000115282 2024-05-08T08:54:35.094 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.094 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.094 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op must wait for map", 2024-05-08T08:54:35.094 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514128+0000", 2024-05-08T08:54:35.094 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9950000000000002e-06 2024-05-08T08:54:35.094 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.094 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.094 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:35.094 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.515800+0000", 2024-05-08T08:54:35.094 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001671886 2024-05-08T08:54:35.095 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.095 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.095 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:35.095 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.515893+0000", 2024-05-08T08:54:35.095 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3490000000000001e-05 2024-05-08T08:54:35.095 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.095 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.095 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 3", 2024-05-08T08:54:35.095 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.517647+0000", 2024-05-08T08:54:35.095 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0017535630000000001 2024-05-08T08:54:35.095 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.095 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.095 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:54:35.095 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.627578+0000", 2024-05-08T08:54:35.095 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.109931557 2024-05-08T08:54:35.096 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.096 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.096 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:54:35.096 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631055+0000", 2024-05-08T08:54:35.096 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0034761420000000002 2024-05-08T08:54:35.096 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.096 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.096 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:54:35.096 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631096+0000", 2024-05-08T08:54:35.096 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1059000000000001e-05 2024-05-08T08:54:35.096 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.096 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.096 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:35.096 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631151+0000", 2024-05-08T08:54:35.096 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5770999999999998e-05 2024-05-08T08:54:35.096 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.097 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:35.097 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.097 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.097 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.097 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1071 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11026432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:35.097 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:54:35.097 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 118.60592956799999, 2024-05-08T08:54:35.097 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070580037999999998, 2024-05-08T08:54:35.097 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:35.097 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:35.097 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:35.097 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:35.097 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:35.097 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:35.098 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1071 2024-05-08T08:54:35.098 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.098 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:35.098 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.098 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:35.098 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:54:35.098 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:35.098 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.098 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.098 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:35.098 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:54:35.098 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:35.098 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.098 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.099 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:35.099 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:54:35.099 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2800000000000002e-07 2024-05-08T08:54:35.099 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.099 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.099 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:35.099 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485454+0000", 2024-05-08T08:54:35.099 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2290000000000002e-06 2024-05-08T08:54:35.099 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.099 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.099 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:35.099 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485461+0000", 2024-05-08T08:54:35.099 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9920000000000001e-06 2024-05-08T08:54:35.099 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.099 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.100 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:35.100 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485467+0000", 2024-05-08T08:54:35.100 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3979999999999998e-06 2024-05-08T08:54:35.100 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.100 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.100 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:35.100 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555880+0000", 2024-05-08T08:54:35.100 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070413165 2024-05-08T08:54:35.100 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.100 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.100 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:35.100 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555926+0000", 2024-05-08T08:54:35.100 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6217000000000002e-05 2024-05-08T08:54:35.100 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.100 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.100 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:35.101 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556032+0000", 2024-05-08T08:54:35.101 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000105609 2024-05-08T08:54:35.101 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.101 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:35.101 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.101 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.101 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.101 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1073 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11067392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:35.101 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:54:35.101 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 118.605908347, 2024-05-08T08:54:35.101 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070821404000000004, 2024-05-08T08:54:35.101 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:35.101 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:35.101 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:35.101 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:35.101 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:35.102 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:35.102 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1073 2024-05-08T08:54:35.102 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.102 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:35.102 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.102 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:35.102 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:54:35.102 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:35.102 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.102 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.102 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:35.102 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:54:35.102 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:35.102 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.103 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.103 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:35.103 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:54:35.103 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.08e-07 2024-05-08T08:54:35.103 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.103 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.103 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:35.103 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485476+0000", 2024-05-08T08:54:35.103 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3929999999999998e-06 2024-05-08T08:54:35.103 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.103 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.103 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:35.103 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485485+0000", 2024-05-08T08:54:35.103 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1880000000000008e-06 2024-05-08T08:54:35.103 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.104 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.104 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:35.104 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485494+0000", 2024-05-08T08:54:35.104 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3109999999999995e-06 2024-05-08T08:54:35.104 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.104 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.104 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:35.104 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556082+0000", 2024-05-08T08:54:35.104 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070587426999999994 2024-05-08T08:54:35.104 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.104 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.104 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:35.104 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556177+0000", 2024-05-08T08:54:35.104 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5051999999999997e-05 2024-05-08T08:54:35.104 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.105 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.105 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:35.105 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556294+0000", 2024-05-08T08:54:35.105 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000117625 2024-05-08T08:54:35.105 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.105 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:35.105 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.105 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.105 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.105 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1075 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11108352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:35.105 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:54:35.105 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 118.60587777000001, 2024-05-08T08:54:35.105 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071045026999999997, 2024-05-08T08:54:35.105 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:35.105 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:35.105 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:35.106 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:35.106 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:35.106 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:35.106 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1075 2024-05-08T08:54:35.106 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.106 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:35.106 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.106 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:35.106 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:54:35.106 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:35.106 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.106 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.106 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:35.106 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:54:35.106 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:35.107 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.107 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.107 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:35.107 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:54:35.107 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.32e-07 2024-05-08T08:54:35.107 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.107 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.107 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:35.107 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485518+0000", 2024-05-08T08:54:35.107 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3769e-05 2024-05-08T08:54:35.107 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.107 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.107 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:35.107 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485527+0000", 2024-05-08T08:54:35.107 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6779999999999995e-06 2024-05-08T08:54:35.107 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.108 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.108 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:35.108 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485532+0000", 2024-05-08T08:54:35.108 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9830000000000004e-06 2024-05-08T08:54:35.108 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.108 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.108 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:35.108 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556340+0000", 2024-05-08T08:54:35.108 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070807743000000006 2024-05-08T08:54:35.108 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.108 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.108 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:35.108 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556435+0000", 2024-05-08T08:54:35.108 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4732e-05 2024-05-08T08:54:35.108 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.108 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.109 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:35.109 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556549+0000", 2024-05-08T08:54:35.109 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011369 2024-05-08T08:54:35.109 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.109 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:35.109 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.109 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.109 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.109 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1078 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11149312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:35.109 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:54:35.109 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 118.605837571, 2024-05-08T08:54:35.109 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071167276000000002, 2024-05-08T08:54:35.109 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:35.109 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:35.109 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:35.109 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:35.109 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:35.110 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:35.110 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1078 2024-05-08T08:54:35.110 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.110 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:35.110 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.110 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:35.110 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:54:35.110 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:35.110 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.110 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.110 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:35.110 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:54:35.110 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:35.110 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.110 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.110 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:35.111 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:54:35.111 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5600000000000001e-07 2024-05-08T08:54:35.111 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.111 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.111 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:35.111 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485547+0000", 2024-05-08T08:54:35.111 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3590000000000002e-06 2024-05-08T08:54:35.111 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.111 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.111 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:35.111 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485553+0000", 2024-05-08T08:54:35.111 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2770000000000002e-06 2024-05-08T08:54:35.111 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.111 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.111 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:35.112 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485558+0000", 2024-05-08T08:54:35.112 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.95e-06 2024-05-08T08:54:35.112 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.112 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.112 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:35.112 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556582+0000", 2024-05-08T08:54:35.112 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071024444000000006 2024-05-08T08:54:35.112 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.112 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.112 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:35.112 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556637+0000", 2024-05-08T08:54:35.112 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4672000000000001e-05 2024-05-08T08:54:35.112 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.112 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.112 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:35.112 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556711+0000", 2024-05-08T08:54:35.112 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4117999999999997e-05 2024-05-08T08:54:35.113 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.113 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:35.113 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.113 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.113 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.113 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1080 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11190272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:35.113 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:54:35.113 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 118.605816838, 2024-05-08T08:54:35.113 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071350721000000006, 2024-05-08T08:54:35.113 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:35.113 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:35.113 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:35.113 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:35.113 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:35.113 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:35.113 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1080 2024-05-08T08:54:35.113 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.114 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:35.114 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.114 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:35.114 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:54:35.114 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:35.114 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.114 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.114 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:35.114 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:54:35.114 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:35.114 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.114 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.114 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:35.114 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:54:35.115 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.03e-07 2024-05-08T08:54:35.115 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.115 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.115 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:35.115 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485567+0000", 2024-05-08T08:54:35.115 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1090000000000001e-06 2024-05-08T08:54:35.115 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.115 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.115 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:35.115 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485573+0000", 2024-05-08T08:54:35.115 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3049999999999998e-06 2024-05-08T08:54:35.115 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.115 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.115 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:35.115 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485578+0000", 2024-05-08T08:54:35.115 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0130000000000001e-06 2024-05-08T08:54:35.116 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.116 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.116 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:35.116 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556743+0000", 2024-05-08T08:54:35.116 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071164449000000005 2024-05-08T08:54:35.116 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.116 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.116 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:35.116 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556797+0000", 2024-05-08T08:54:35.116 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4589999999999997e-05 2024-05-08T08:54:35.116 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.116 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.116 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:35.116 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556915+0000", 2024-05-08T08:54:35.117 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000117852 2024-05-08T08:54:35.117 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.117 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:35.117 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.117 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.117 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.117 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1082 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11231232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:35.117 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:54:35.117 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 118.605796595, 2024-05-08T08:54:35.117 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071591396000000002, 2024-05-08T08:54:35.117 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:35.117 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:35.117 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:35.117 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:35.117 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:35.118 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:35.118 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1082 2024-05-08T08:54:35.118 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.118 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:35.118 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.118 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:35.118 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:54:35.118 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:35.118 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.118 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.118 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:35.118 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:54:35.118 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:35.118 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.118 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.119 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:35.119 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:54:35.119 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1800000000000001e-07 2024-05-08T08:54:35.119 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.119 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.119 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:35.119 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485587+0000", 2024-05-08T08:54:35.119 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2589999999999999e-06 2024-05-08T08:54:35.119 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.119 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.119 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:35.119 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485594+0000", 2024-05-08T08:54:35.119 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0789999999999997e-06 2024-05-08T08:54:35.119 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.119 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.119 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:35.120 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485598+0000", 2024-05-08T08:54:35.120 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7480000000000001e-06 2024-05-08T08:54:35.120 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.120 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.120 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:35.120 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556971+0000", 2024-05-08T08:54:35.120 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071372774999999999 2024-05-08T08:54:35.120 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.120 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.120 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:35.120 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557069+0000", 2024-05-08T08:54:35.120 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.7732999999999994e-05 2024-05-08T08:54:35.120 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.120 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.120 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:35.120 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557176+0000", 2024-05-08T08:54:35.121 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000107384 2024-05-08T08:54:35.121 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.121 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:35.121 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.121 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.121 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.121 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1084 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11272192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:35.121 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:54:35.121 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 118.60575795600001, 2024-05-08T08:54:35.121 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071808977999999996, 2024-05-08T08:54:35.121 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:35.121 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:35.121 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:35.121 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:35.121 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:35.122 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:35.122 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1084 2024-05-08T08:54:35.122 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.122 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:35.122 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.122 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:35.122 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:54:35.122 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:35.122 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.122 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.122 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:35.122 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:54:35.122 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:35.122 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.123 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.123 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:35.123 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485624+0000", 2024-05-08T08:54:35.123 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:54:35.123 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.123 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.123 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:35.123 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485626+0000", 2024-05-08T08:54:35.123 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9460000000000001e-06 2024-05-08T08:54:35.123 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.123 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.123 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:35.123 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485633+0000", 2024-05-08T08:54:35.123 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1149999999999996e-06 2024-05-08T08:54:35.123 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.124 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.124 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:35.124 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485638+0000", 2024-05-08T08:54:35.124 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5670000000000002e-06 2024-05-08T08:54:35.124 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.124 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.124 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:35.124 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557226+0000", 2024-05-08T08:54:35.124 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071587267999999996 2024-05-08T08:54:35.124 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.124 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.124 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:35.124 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557321+0000", 2024-05-08T08:54:35.124 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5785000000000005e-05 2024-05-08T08:54:35.124 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.124 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.125 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:35.125 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557432+0000", 2024-05-08T08:54:35.125 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011092800000000001 2024-05-08T08:54:35.125 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.125 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:35.125 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.125 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.125 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.125 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1086 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11313152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:35.125 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:54:35.125 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 118.605734629, 2024-05-08T08:54:35.125 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071933132999999996, 2024-05-08T08:54:35.125 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:35.125 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:35.125 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:35.125 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:35.125 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:35.126 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:35.126 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1086 2024-05-08T08:54:35.126 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.126 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:35.126 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.126 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:35.126 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:54:35.126 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:35.126 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.126 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.126 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:35.126 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:54:35.126 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:35.126 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.127 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.127 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:35.127 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486328+0000", 2024-05-08T08:54:35.127 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00068107900000000002 2024-05-08T08:54:35.127 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.127 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.127 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:35.127 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486331+0000", 2024-05-08T08:54:35.127 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.585e-06 2024-05-08T08:54:35.127 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.127 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.127 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:35.127 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486343+0000", 2024-05-08T08:54:35.127 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1856e-05 2024-05-08T08:54:35.127 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.128 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.128 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:35.128 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486350+0000", 2024-05-08T08:54:35.128 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8600000000000004e-06 2024-05-08T08:54:35.128 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.128 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.128 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:35.128 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557470+0000", 2024-05-08T08:54:35.128 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071120326999999997 2024-05-08T08:54:35.128 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.128 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.128 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:35.128 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557514+0000", 2024-05-08T08:54:35.128 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3909000000000003e-05 2024-05-08T08:54:35.128 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.128 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.129 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:35.129 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557580+0000", 2024-05-08T08:54:35.129 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5517000000000003e-05 2024-05-08T08:54:35.129 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.129 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:35.129 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.129 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.129 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.129 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1089 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11354112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:35.129 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:54:35.129 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 118.605021757, 2024-05-08T08:54:35.129 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071457381, 2024-05-08T08:54:35.129 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:35.129 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:35.130 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:35.130 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:35.130 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:35.130 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:35.130 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1089 2024-05-08T08:54:35.130 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.130 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:35.130 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.130 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:35.130 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:54:35.130 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:35.130 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.130 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.130 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:35.130 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:54:35.131 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:35.131 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.131 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.131 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:35.131 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486851+0000", 2024-05-08T08:54:35.131 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00049102 2024-05-08T08:54:35.131 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.131 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.131 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:35.131 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486854+0000", 2024-05-08T08:54:35.131 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6859999999999999e-06 2024-05-08T08:54:35.131 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.131 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.132 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:35.132 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486864+0000", 2024-05-08T08:54:35.132 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6080000000000002e-06 2024-05-08T08:54:35.132 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.132 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.132 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:35.132 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486870+0000", 2024-05-08T08:54:35.132 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.477e-06 2024-05-08T08:54:35.132 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.132 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.132 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:35.132 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557612+0000", 2024-05-08T08:54:35.132 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070741557999999996 2024-05-08T08:54:35.132 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.132 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.133 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:35.133 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557695+0000", 2024-05-08T08:54:35.133 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3300000000000005e-05 2024-05-08T08:54:35.133 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.133 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.133 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:35.133 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557817+0000", 2024-05-08T08:54:35.133 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012173200000000001 2024-05-08T08:54:35.133 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.133 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:35.133 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.133 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.133 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.133 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1091 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11395072~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:35.134 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:54:35.134 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 118.604500962, 2024-05-08T08:54:35.134 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071184737999999997, 2024-05-08T08:54:35.134 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:35.134 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:35.134 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:35.134 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:35.134 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:35.134 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:35.134 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1091 2024-05-08T08:54:35.134 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.134 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:35.134 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.134 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:35.135 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:54:35.135 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:35.135 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.135 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.135 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:35.135 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:54:35.135 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:35.135 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.135 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.135 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:35.135 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486881+0000", 2024-05-08T08:54:35.135 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0699999999999997e-07 2024-05-08T08:54:35.135 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.135 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.135 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:35.135 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486883+0000", 2024-05-08T08:54:35.136 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.402e-06 2024-05-08T08:54:35.136 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.136 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.136 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:35.136 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486890+0000", 2024-05-08T08:54:35.136 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0480000000000001e-06 2024-05-08T08:54:35.136 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.136 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.136 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:35.136 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486896+0000", 2024-05-08T08:54:35.136 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7640000000000002e-06 2024-05-08T08:54:35.136 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.136 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.136 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:35.136 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557860+0000", 2024-05-08T08:54:35.137 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070963791999999998 2024-05-08T08:54:35.137 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.137 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.137 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:35.137 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557952+0000", 2024-05-08T08:54:35.137 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2082000000000004e-05 2024-05-08T08:54:35.137 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.137 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.137 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:35.137 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558065+0000", 2024-05-08T08:54:35.137 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000113143 2024-05-08T08:54:35.137 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.137 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:35.137 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.137 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.137 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.138 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1094 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11436032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:35.138 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:54:35.138 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 118.604477794, 2024-05-08T08:54:35.138 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071400507000000002, 2024-05-08T08:54:35.138 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:35.138 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:35.138 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:35.138 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:35.138 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:35.138 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:35.138 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1094 2024-05-08T08:54:35.138 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.138 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:35.138 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.138 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:35.138 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:54:35.139 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:35.139 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.139 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.139 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:35.139 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:54:35.139 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:35.139 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.139 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.139 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:35.139 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487940+0000", 2024-05-08T08:54:35.139 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001036868 2024-05-08T08:54:35.139 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.139 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.139 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:35.139 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487944+0000", 2024-05-08T08:54:35.139 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7069999999999998e-06 2024-05-08T08:54:35.140 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.140 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.140 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:35.140 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487951+0000", 2024-05-08T08:54:35.140 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2649999999999999e-06 2024-05-08T08:54:35.140 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.140 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.140 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:35.140 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487961+0000", 2024-05-08T08:54:35.140 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4110000000000002e-06 2024-05-08T08:54:35.140 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.140 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.140 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:35.140 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558117+0000", 2024-05-08T08:54:35.140 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070156662999999994 2024-05-08T08:54:35.140 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.140 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.140 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:35.141 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558214+0000", 2024-05-08T08:54:35.141 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6444e-05 2024-05-08T08:54:35.141 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.141 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.141 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:35.141 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558304+0000", 2024-05-08T08:54:35.141 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0148999999999994e-05 2024-05-08T08:54:35.141 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.141 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:35.141 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.141 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.141 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.141 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1096 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11476992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:35.141 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:54:35.141 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 118.603411263, 2024-05-08T08:54:35.141 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070480074000000004, 2024-05-08T08:54:35.142 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:35.142 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:35.142 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:35.142 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:35.142 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:35.142 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:35.142 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1096 2024-05-08T08:54:35.142 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.142 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:35.142 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.142 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:35.142 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:54:35.142 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:35.142 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.142 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.143 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:35.143 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:54:35.143 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:35.143 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.143 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.143 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:35.143 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:54:35.143 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3700000000000001e-07 2024-05-08T08:54:35.143 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.143 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.143 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:35.143 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487973+0000", 2024-05-08T08:54:35.143 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0870000000000001e-06 2024-05-08T08:54:35.143 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.143 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.144 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:35.144 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487977+0000", 2024-05-08T08:54:35.144 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0600000000000001e-06 2024-05-08T08:54:35.144 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.144 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.144 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:35.144 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487980+0000", 2024-05-08T08:54:35.144 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1089999999999998e-06 2024-05-08T08:54:35.144 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.144 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.144 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:35.144 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558338+0000", 2024-05-08T08:54:35.144 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070358233000000006 2024-05-08T08:54:35.144 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.144 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.144 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:35.144 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558392+0000", 2024-05-08T08:54:35.145 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4537999999999999e-05 2024-05-08T08:54:35.145 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.145 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.145 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:35.145 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558450+0000", 2024-05-08T08:54:35.145 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7710000000000001e-05 2024-05-08T08:54:35.145 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.145 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:35.145 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.145 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.145 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.145 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1099 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11517952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:35.145 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:54:35.145 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 118.603397387, 2024-05-08T08:54:35.145 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070720506000000002, 2024-05-08T08:54:35.145 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:35.145 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:35.146 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:35.146 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:35.146 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:35.146 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:35.146 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1099 2024-05-08T08:54:35.146 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.146 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:35.146 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.146 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:35.146 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:54:35.146 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:35.146 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.146 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.146 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:35.146 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:54:35.147 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:35.147 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.147 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.147 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:35.147 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:54:35.147 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3999999999999998e-07 2024-05-08T08:54:35.147 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.147 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.147 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:35.147 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487986+0000", 2024-05-08T08:54:35.147 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3519999999999999e-06 2024-05-08T08:54:35.147 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.147 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.147 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:35.147 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487989+0000", 2024-05-08T08:54:35.147 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.23e-06 2024-05-08T08:54:35.148 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.148 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.148 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:35.148 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487991+0000", 2024-05-08T08:54:35.148 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6350000000000002e-06 2024-05-08T08:54:35.148 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.148 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.148 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:35.148 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558489+0000", 2024-05-08T08:54:35.148 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070497924000000003 2024-05-08T08:54:35.148 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.148 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.148 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:35.148 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558583+0000", 2024-05-08T08:54:35.148 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3405000000000004e-05 2024-05-08T08:54:35.148 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.148 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.149 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:35.149 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558704+0000", 2024-05-08T08:54:35.149 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012171999999999999 2024-05-08T08:54:35.149 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.149 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:35.149 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.149 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.149 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.149 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1100 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11558912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:35.149 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:54:35.149 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 118.60338438799999, 2024-05-08T08:54:35.149 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070962266999999996, 2024-05-08T08:54:35.149 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:35.149 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:35.150 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:35.150 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:35.150 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:35.150 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:35.150 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1100 2024-05-08T08:54:35.150 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.150 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:35.150 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.150 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:35.150 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:54:35.150 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:35.150 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.150 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.150 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:35.150 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:54:35.151 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:35.151 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.151 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.151 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:35.151 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488430+0000", 2024-05-08T08:54:35.151 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000432676 2024-05-08T08:54:35.151 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.151 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.151 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:35.151 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488433+0000", 2024-05-08T08:54:35.151 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.45e-06 2024-05-08T08:54:35.151 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.151 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.151 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:35.151 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488442+0000", 2024-05-08T08:54:35.151 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.687e-06 2024-05-08T08:54:35.152 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.152 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.152 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:35.152 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488446+0000", 2024-05-08T08:54:35.152 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4939999999999997e-06 2024-05-08T08:54:35.152 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.152 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.152 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:35.152 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558756+0000", 2024-05-08T08:54:35.152 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070310051999999998 2024-05-08T08:54:35.152 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.152 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.152 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:35.152 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558844+0000", 2024-05-08T08:54:35.152 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7690999999999998e-05 2024-05-08T08:54:35.152 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.153 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.153 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:35.153 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558959+0000", 2024-05-08T08:54:35.153 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000115217 2024-05-08T08:54:35.153 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.153 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:35.153 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.153 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.153 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.153 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1103 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11599872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:35.153 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:54:35.153 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 118.602926713, 2024-05-08T08:54:35.153 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070777100999999995, 2024-05-08T08:54:35.153 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:35.153 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:35.153 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:35.154 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:35.154 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:35.154 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:35.154 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1103 2024-05-08T08:54:35.154 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.154 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:35.154 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.154 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:35.154 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:54:35.154 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:35.154 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.154 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.154 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:35.154 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:54:35.154 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:35.154 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.155 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.155 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:35.155 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488959+0000", 2024-05-08T08:54:35.155 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00050457200000000001 2024-05-08T08:54:35.155 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.155 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.155 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:35.155 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488969+0000", 2024-05-08T08:54:35.155 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.7159999999999998e-06 2024-05-08T08:54:35.155 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.155 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.155 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:35.155 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488981+0000", 2024-05-08T08:54:35.155 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2150000000000001e-05 2024-05-08T08:54:35.155 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.155 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.156 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:35.156 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488988+0000", 2024-05-08T08:54:35.156 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8759999999999999e-06 2024-05-08T08:54:35.156 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.156 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.156 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:35.156 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558992+0000", 2024-05-08T08:54:35.156 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070004358000000003 2024-05-08T08:54:35.156 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.156 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.156 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:35.156 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559093+0000", 2024-05-08T08:54:35.156 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000100299 2024-05-08T08:54:35.156 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.156 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.156 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:35.157 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559232+0000", 2024-05-08T08:54:35.157 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013913 2024-05-08T08:54:35.157 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.157 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:35.157 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.157 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.157 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.157 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1104 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11640832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:35.157 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:54:35.157 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 118.602381898, 2024-05-08T08:54:35.157 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070521947000000001, 2024-05-08T08:54:35.157 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:35.157 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:35.157 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:35.157 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:35.158 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:35.158 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:35.158 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1104 2024-05-08T08:54:35.158 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.158 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:35.158 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.158 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:35.158 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:54:35.158 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:35.158 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.158 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.158 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:35.158 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:54:35.158 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:35.158 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.159 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.159 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:35.159 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489700+0000", 2024-05-08T08:54:35.159 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00070067600000000001 2024-05-08T08:54:35.159 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.159 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.159 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:35.159 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489704+0000", 2024-05-08T08:54:35.159 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1640000000000003e-06 2024-05-08T08:54:35.159 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.159 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.159 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:35.159 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489716+0000", 2024-05-08T08:54:35.159 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1379000000000001e-05 2024-05-08T08:54:35.159 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.160 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.160 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:35.160 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489724+0000", 2024-05-08T08:54:35.160 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8949999999999998e-06 2024-05-08T08:54:35.160 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.160 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.160 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:35.160 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559278+0000", 2024-05-08T08:54:35.160 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.069554151999999994 2024-05-08T08:54:35.160 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.160 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.160 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:35.160 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559387+0000", 2024-05-08T08:54:35.160 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000109258 2024-05-08T08:54:35.160 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.161 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.161 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:35.161 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559521+0000", 2024-05-08T08:54:35.161 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000134423 2024-05-08T08:54:35.161 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.161 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:35.161 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.161 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.161 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.161 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:8636 3.17 3:e8a0a3e4:::rbd_header.11402b26385:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-05-08T08:54:35.161 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:54.150971+0000", 2024-05-08T08:54:35.161 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 100.94040990400001, 2024-05-08T08:54:35.161 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.217273233, 2024-05-08T08:54:35.161 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:35.161 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:35.161 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:35.161 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:35.162 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:54:35.162 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:54:35.162 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 8636 2024-05-08T08:54:35.162 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.162 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:35.162 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.162 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:35.162 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150971+0000", 2024-05-08T08:54:35.162 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:35.162 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.162 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.162 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:35.162 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150971+0000", 2024-05-08T08:54:35.162 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:35.162 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.162 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.163 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:35.163 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150972+0000", 2024-05-08T08:54:35.163 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.71e-07 2024-05-08T08:54:35.163 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.163 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.163 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:35.163 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150977+0000", 2024-05-08T08:54:35.163 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.152e-06 2024-05-08T08:54:35.163 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.163 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.163 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:35.163 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150990+0000", 2024-05-08T08:54:35.163 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2364999999999999e-05 2024-05-08T08:54:35.163 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.163 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.163 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:35.163 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150998+0000", 2024-05-08T08:54:35.164 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0739999999999993e-06 2024-05-08T08:54:35.164 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.164 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.164 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:35.164 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.151054+0000", 2024-05-08T08:54:35.164 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5949000000000003e-05 2024-05-08T08:54:35.164 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.164 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.164 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:54:35.164 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.151060+0000", 2024-05-08T08:54:35.164 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6120000000000004e-06 2024-05-08T08:54:35.164 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.164 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.164 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:35.164 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.367640+0000", 2024-05-08T08:54:35.164 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2165792739999999 2024-05-08T08:54:35.164 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.165 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.165 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:35.165 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.367839+0000", 2024-05-08T08:54:35.165 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000199185 2024-05-08T08:54:35.165 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.165 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.165 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:35.165 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.368245+0000", 2024-05-08T08:54:35.165 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00040575099999999998 2024-05-08T08:54:35.165 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.165 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:35.165 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.165 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.165 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.165 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5302.0:1516 3.c 3:319bf789:::rbd_data.1140d3fc54c.0000000000000025:head [stat,write 1892352~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:54:35.165 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:27.822220+0000", 2024-05-08T08:54:35.165 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 67.269160967000005, 2024-05-08T08:54:35.166 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.092643744, 2024-05-08T08:54:35.166 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:35.166 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:35.166 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:54:35.166 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:35.166 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5302", 2024-05-08T08:54:35.166 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/359277859", 2024-05-08T08:54:35.166 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1516 2024-05-08T08:54:35.166 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.166 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:35.166 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.166 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:35.166 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822220+0000", 2024-05-08T08:54:35.166 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:35.166 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.166 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.167 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:35.167 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822220+0000", 2024-05-08T08:54:35.167 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:35.167 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.167 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.167 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:35.167 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822221+0000", 2024-05-08T08:54:35.167 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3500000000000002e-07 2024-05-08T08:54:35.167 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.167 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.167 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:35.167 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822228+0000", 2024-05-08T08:54:35.167 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9959999999999996e-06 2024-05-08T08:54:35.167 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.167 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.167 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:35.168 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822234+0000", 2024-05-08T08:54:35.168 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5429999999999999e-06 2024-05-08T08:54:35.168 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.168 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.168 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:35.168 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822238+0000", 2024-05-08T08:54:35.168 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.732e-06 2024-05-08T08:54:35.168 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.168 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.168 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:35.168 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822285+0000", 2024-05-08T08:54:35.168 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6810000000000001e-05 2024-05-08T08:54:35.168 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.168 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.168 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:35.168 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822358+0000", 2024-05-08T08:54:35.169 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3473000000000006e-05 2024-05-08T08:54:35.169 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.169 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.169 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 5", 2024-05-08T08:54:35.169 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822489+0000", 2024-05-08T08:54:35.169 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013011199999999999 2024-05-08T08:54:35.169 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.169 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.169 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:54:35.169 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.825332+0000", 2024-05-08T08:54:35.169 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0028434799999999998 2024-05-08T08:54:35.169 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.169 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.169 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:54:35.169 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.914753+0000", 2024-05-08T08:54:35.169 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089421424999999999 2024-05-08T08:54:35.170 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.170 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.170 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:54:35.170 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.914807+0000", 2024-05-08T08:54:35.170 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3829000000000002e-05 2024-05-08T08:54:35.170 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.170 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.170 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:35.170 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.914864+0000", 2024-05-08T08:54:35.170 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6909e-05 2024-05-08T08:54:35.170 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.170 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:35.170 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.170 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.170 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.171 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:24567 3.6 3:66af1c40:::rbd_data.1140d3fc54c.000000000000005a:18 [read 2080768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:35.171 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.216598+0000", 2024-05-08T08:54:35.171 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 62.874782887000002, 2024-05-08T08:54:35.171 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071602647000000005, 2024-05-08T08:54:35.171 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:35.171 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:35.171 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:35.171 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:35.171 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:35.171 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:35.171 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 24567 2024-05-08T08:54:35.171 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.171 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:35.171 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.171 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:35.171 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216598+0000", 2024-05-08T08:54:35.171 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:35.172 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.172 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.172 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:35.172 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216598+0000", 2024-05-08T08:54:35.172 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:35.172 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.172 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.172 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:35.172 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216599+0000", 2024-05-08T08:54:35.172 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8999999999999997e-07 2024-05-08T08:54:35.172 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.172 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.172 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:35.172 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216601+0000", 2024-05-08T08:54:35.172 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9889999999999999e-06 2024-05-08T08:54:35.172 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.172 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.173 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:35.173 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216607+0000", 2024-05-08T08:54:35.173 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3600000000000001e-06 2024-05-08T08:54:35.173 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.173 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.173 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:35.173 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216612+0000", 2024-05-08T08:54:35.173 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.69e-06 2024-05-08T08:54:35.173 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.173 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.173 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:35.173 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.287076+0000", 2024-05-08T08:54:35.173 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070464355000000006 2024-05-08T08:54:35.173 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.173 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.173 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:35.174 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.287151+0000", 2024-05-08T08:54:35.174 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4924000000000003e-05 2024-05-08T08:54:35.174 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.174 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.174 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:35.174 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.288201+0000", 2024-05-08T08:54:35.174 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0010498389999999999 2024-05-08T08:54:35.174 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.174 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:35.174 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.174 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.174 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:35.174 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:35.176 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:54:35.176 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:35.178+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:35.276 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:35.278+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:35.326 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:35.326 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:54:35.326 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:54:35.326 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:35.334 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:54:35.378 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:35.381+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:35.443 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:35.444 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:54:35.444 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:54:35.444 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:54:35.444 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:35.452 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:54:35.493 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:35.496+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:35.579 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:35.581+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:35.604 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:35.605 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:54:35.605 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:54:35.605 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:54:35.605 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.605 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:12443 3.0 3:02d0158e:::rbd_data.114071bcf828.0000000000000037:head [stat,delete] snapc 19=[19] RETRY=3 ondisk+retry+write+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:35.605 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.610027+0000", 2024-05-08T08:54:35.605 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 63.996343119999999, 2024-05-08T08:54:35.605 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.083586333999999998, 2024-05-08T08:54:35.605 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:35.605 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:35.605 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:35.605 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:35.605 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:35.606 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:35.606 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 12443 2024-05-08T08:54:35.606 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.606 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:35.606 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.606 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:35.606 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610027+0000", 2024-05-08T08:54:35.606 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:35.606 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.606 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.606 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:35.606 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610027+0000", 2024-05-08T08:54:35.606 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:35.606 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.607 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.607 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:35.607 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610028+0000", 2024-05-08T08:54:35.607 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.13e-07 2024-05-08T08:54:35.607 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.607 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.607 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:35.607 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610030+0000", 2024-05-08T08:54:35.607 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7669999999999999e-06 2024-05-08T08:54:35.607 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.607 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.607 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:35.607 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610039+0000", 2024-05-08T08:54:35.607 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4959999999999996e-06 2024-05-08T08:54:35.607 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.608 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.608 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:35.608 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610045+0000", 2024-05-08T08:54:35.608 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6779999999999997e-06 2024-05-08T08:54:35.608 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.608 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.608 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:35.608 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.660873+0000", 2024-05-08T08:54:35.608 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.050828748 2024-05-08T08:54:35.608 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.608 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.608 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:35.608 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.661017+0000", 2024-05-08T08:54:35.608 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014391999999999999 2024-05-08T08:54:35.608 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.609 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.609 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:35.609 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.693613+0000", 2024-05-08T08:54:35.609 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.032596111999999997 2024-05-08T08:54:35.609 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.609 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:35.609 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.609 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.609 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.609 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:14301 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [watch reconnect cookie 139837155901536 gen 12] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:35.609 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.645078+0000", 2024-05-08T08:54:35.609 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 63.961292481999998, 2024-05-08T08:54:35.609 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088281852999999993, 2024-05-08T08:54:35.609 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:35.609 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:35.610 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:35.610 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:35.610 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:35.610 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:35.610 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14301 2024-05-08T08:54:35.610 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.610 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:35.610 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.610 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:35.610 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645078+0000", 2024-05-08T08:54:35.610 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:35.610 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.610 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.610 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:35.610 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645078+0000", 2024-05-08T08:54:35.611 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:35.611 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.611 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.611 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:35.611 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645084+0000", 2024-05-08T08:54:35.611 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4620000000000001e-06 2024-05-08T08:54:35.611 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.611 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.611 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:35.611 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645088+0000", 2024-05-08T08:54:35.611 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8039999999999999e-06 2024-05-08T08:54:35.611 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.611 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.611 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:35.611 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645106+0000", 2024-05-08T08:54:35.611 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.8318999999999999e-05 2024-05-08T08:54:35.611 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.612 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.612 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:35.612 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645113+0000", 2024-05-08T08:54:35.612 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0310000000000001e-06 2024-05-08T08:54:35.612 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.612 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.612 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:35.612 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.670493+0000", 2024-05-08T08:54:35.612 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.025379993 2024-05-08T08:54:35.612 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.612 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.612 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:35.612 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.670571+0000", 2024-05-08T08:54:35.612 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7613000000000004e-05 2024-05-08T08:54:35.612 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.613 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.613 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:35.613 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733360+0000", 2024-05-08T08:54:35.613 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.062788630999999998 2024-05-08T08:54:35.613 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.613 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:35.613 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.613 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.613 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.613 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:14420 3.0 3:06a8948e:::rbd_data.114071bcf828.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:35.613 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.649639+0000", 2024-05-08T08:54:35.613 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 63.956731333999997, 2024-05-08T08:54:35.613 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.083851897999999994, 2024-05-08T08:54:35.613 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:35.613 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:35.613 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:54:35.614 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:35.614 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:35.614 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:35.614 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14420 2024-05-08T08:54:35.614 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.614 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:35.614 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.614 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:35.614 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649639+0000", 2024-05-08T08:54:35.614 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:35.614 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.614 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.614 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:35.614 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649639+0000", 2024-05-08T08:54:35.614 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:35.614 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.615 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.615 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:35.615 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649640+0000", 2024-05-08T08:54:35.615 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6899999999999997e-07 2024-05-08T08:54:35.615 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.615 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.615 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:35.615 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649895+0000", 2024-05-08T08:54:35.615 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00025531000000000002 2024-05-08T08:54:35.615 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.615 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.615 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:35.615 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649905+0000", 2024-05-08T08:54:35.615 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0198e-05 2024-05-08T08:54:35.615 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.615 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.616 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:35.616 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649911+0000", 2024-05-08T08:54:35.616 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2310000000000001e-06 2024-05-08T08:54:35.616 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.616 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.616 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:35.616 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.684099+0000", 2024-05-08T08:54:35.616 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.034188017000000001 2024-05-08T08:54:35.616 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.616 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.616 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:35.616 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.684172+0000", 2024-05-08T08:54:35.616 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3201999999999995e-05 2024-05-08T08:54:35.616 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.616 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.616 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 2", 2024-05-08T08:54:35.617 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.684418+0000", 2024-05-08T08:54:35.617 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00024559099999999997 2024-05-08T08:54:35.617 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.617 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.617 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:54:35.617 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.705538+0000", 2024-05-08T08:54:35.617 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.021119707000000001 2024-05-08T08:54:35.617 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.617 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.617 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:54:35.617 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733399+0000", 2024-05-08T08:54:35.617 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.027861543999999999 2024-05-08T08:54:35.617 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.617 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.617 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:54:35.617 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733445+0000", 2024-05-08T08:54:35.618 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5587000000000002e-05 2024-05-08T08:54:35.618 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.618 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.618 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:35.618 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733491+0000", 2024-05-08T08:54:35.618 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5841999999999999e-05 2024-05-08T08:54:35.618 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.618 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:35.618 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.618 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.618 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.618 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28547 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889179216] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:35.618 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.355654+0000", 2024-05-08T08:54:35.618 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 63.250716105999999, 2024-05-08T08:54:35.619 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078692261999999999, 2024-05-08T08:54:35.619 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:35.619 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:35.619 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:35.619 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:35.619 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:35.619 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:35.619 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28547 2024-05-08T08:54:35.619 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.619 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:35.619 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.619 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:35.619 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355654+0000", 2024-05-08T08:54:35.619 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:35.619 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.619 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.620 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:35.620 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355654+0000", 2024-05-08T08:54:35.620 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:35.620 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.620 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.620 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:35.620 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355654+0000", 2024-05-08T08:54:35.620 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3999999999999998e-07 2024-05-08T08:54:35.620 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.620 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.620 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:35.620 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355656+0000", 2024-05-08T08:54:35.620 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.353e-06 2024-05-08T08:54:35.620 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.620 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.620 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:35.620 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355663+0000", 2024-05-08T08:54:35.621 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2259999999999999e-06 2024-05-08T08:54:35.621 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.621 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.621 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:35.621 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355666+0000", 2024-05-08T08:54:35.621 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7930000000000002e-06 2024-05-08T08:54:35.621 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.621 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.621 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:35.621 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434117+0000", 2024-05-08T08:54:35.621 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078451231999999996 2024-05-08T08:54:35.621 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.621 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.621 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:35.621 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434198+0000", 2024-05-08T08:54:35.621 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0879999999999998e-05 2024-05-08T08:54:35.622 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.622 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.622 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:35.622 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434346+0000", 2024-05-08T08:54:35.622 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014853800000000001 2024-05-08T08:54:35.622 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.622 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:35.622 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.622 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.622 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.622 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28548 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889189696] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:35.622 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.355669+0000", 2024-05-08T08:54:35.622 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 63.250700756000001, 2024-05-08T08:54:35.622 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078956606999999998, 2024-05-08T08:54:35.622 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:35.622 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:35.623 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:35.623 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:35.623 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:35.623 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:35.623 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28548 2024-05-08T08:54:35.623 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.623 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:35.623 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.623 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:35.623 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355669+0000", 2024-05-08T08:54:35.623 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:35.623 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.623 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.623 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:35.623 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355669+0000", 2024-05-08T08:54:35.623 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:35.624 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.624 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.624 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:35.624 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355762+0000", 2024-05-08T08:54:35.624 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3015000000000003e-05 2024-05-08T08:54:35.624 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.624 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.624 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:35.624 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355765+0000", 2024-05-08T08:54:35.624 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.368e-06 2024-05-08T08:54:35.624 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.624 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.624 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:35.624 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355771+0000", 2024-05-08T08:54:35.624 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5989999999999998e-06 2024-05-08T08:54:35.624 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.624 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.625 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:35.625 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355775+0000", 2024-05-08T08:54:35.625 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7280000000000001e-06 2024-05-08T08:54:35.625 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.625 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.625 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:35.625 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434379+0000", 2024-05-08T08:54:35.625 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078603662000000005 2024-05-08T08:54:35.625 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.625 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.625 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:35.625 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434473+0000", 2024-05-08T08:54:35.625 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4104000000000006e-05 2024-05-08T08:54:35.625 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.625 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.626 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:35.626 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434626+0000", 2024-05-08T08:54:35.626 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00015313100000000001 2024-05-08T08:54:35.626 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.626 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:35.626 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.626 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.626 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.626 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28549 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889202400] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:35.626 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.355781+0000", 2024-05-08T08:54:35.626 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 63.250589181999999, 2024-05-08T08:54:35.626 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079117946999999994, 2024-05-08T08:54:35.626 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:35.626 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:35.627 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:35.627 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:35.627 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:35.627 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:35.627 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28549 2024-05-08T08:54:35.627 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.627 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:35.627 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.627 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:35.627 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355781+0000", 2024-05-08T08:54:35.627 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:35.627 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.627 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.627 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:35.627 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355781+0000", 2024-05-08T08:54:35.627 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:35.627 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.628 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.628 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:35.628 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356162+0000", 2024-05-08T08:54:35.628 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00038099100000000001 2024-05-08T08:54:35.628 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.628 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.628 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:35.628 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356165+0000", 2024-05-08T08:54:35.628 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1240000000000001e-06 2024-05-08T08:54:35.628 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.628 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.628 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:35.628 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356173+0000", 2024-05-08T08:54:35.628 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.6199999999999999e-06 2024-05-08T08:54:35.628 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.628 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.629 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:35.629 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356182+0000", 2024-05-08T08:54:35.629 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7760000000000003e-06 2024-05-08T08:54:35.629 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.629 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.629 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:35.629 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434675+0000", 2024-05-08T08:54:35.629 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078493119 2024-05-08T08:54:35.629 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.629 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.629 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:35.629 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434765+0000", 2024-05-08T08:54:35.629 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0402000000000006e-05 2024-05-08T08:54:35.629 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.629 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.629 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:35.630 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434899+0000", 2024-05-08T08:54:35.630 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013391499999999999 2024-05-08T08:54:35.630 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.630 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:35.630 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.630 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.630 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.630 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28550 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888597280] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:35.630 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:54:35.630 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 63.250178136000002, 2024-05-08T08:54:35.630 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078933990999999995, 2024-05-08T08:54:35.630 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:35.630 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:35.630 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:35.631 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:35.631 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:35.631 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:35.631 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28550 2024-05-08T08:54:35.631 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.631 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:35.631 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.631 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:35.631 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:54:35.631 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:35.631 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.631 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.631 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:35.631 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:54:35.631 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:35.631 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.632 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.632 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:35.632 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:54:35.632 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.34e-07 2024-05-08T08:54:35.632 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.632 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.632 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:35.632 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356195+0000", 2024-05-08T08:54:35.632 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8710000000000001e-06 2024-05-08T08:54:35.632 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.632 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.632 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:35.632 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356200+0000", 2024-05-08T08:54:35.632 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3000000000000003e-06 2024-05-08T08:54:35.632 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.632 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.633 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:35.633 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356203+0000", 2024-05-08T08:54:35.633 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4039999999999999e-06 2024-05-08T08:54:35.633 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.633 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.633 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:35.633 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434928+0000", 2024-05-08T08:54:35.633 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078725331999999995 2024-05-08T08:54:35.633 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.633 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.633 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:35.633 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435007+0000", 2024-05-08T08:54:35.633 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8415000000000001e-05 2024-05-08T08:54:35.633 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.633 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.633 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:35.634 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435126+0000", 2024-05-08T08:54:35.634 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011923500000000001 2024-05-08T08:54:35.634 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.634 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:35.634 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.634 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.634 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.634 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28551 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888527200] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:35.634 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.356210+0000", 2024-05-08T08:54:35.634 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 63.250159787999998, 2024-05-08T08:54:35.634 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079195072000000005, 2024-05-08T08:54:35.634 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:35.634 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:35.634 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:35.634 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:35.634 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:35.635 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:35.635 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28551 2024-05-08T08:54:35.635 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.635 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:35.635 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.635 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:35.635 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356210+0000", 2024-05-08T08:54:35.635 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:35.635 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.635 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.635 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:35.635 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356210+0000", 2024-05-08T08:54:35.635 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:35.635 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.635 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.636 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:35.636 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356211+0000", 2024-05-08T08:54:35.636 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4499999999999998e-07 2024-05-08T08:54:35.636 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.636 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.636 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:35.636 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356212+0000", 2024-05-08T08:54:35.636 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6029999999999999e-06 2024-05-08T08:54:35.636 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.636 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.636 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:35.636 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356221+0000", 2024-05-08T08:54:35.636 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4200000000000007e-06 2024-05-08T08:54:35.636 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.636 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.636 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:35.637 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356224+0000", 2024-05-08T08:54:35.637 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.038e-06 2024-05-08T08:54:35.637 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.637 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.637 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:35.637 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435178+0000", 2024-05-08T08:54:35.637 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078954625000000001 2024-05-08T08:54:35.637 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.637 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.637 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:35.637 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435256+0000", 2024-05-08T08:54:35.637 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7337999999999997e-05 2024-05-08T08:54:35.637 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.637 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.637 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:35.637 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435405+0000", 2024-05-08T08:54:35.638 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000149703 2024-05-08T08:54:35.638 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.638 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:35.638 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.638 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.638 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.638 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28552 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889231456] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:35.638 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.356228+0000", 2024-05-08T08:54:35.638 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 63.250142109999999, 2024-05-08T08:54:35.638 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079346920000000001, 2024-05-08T08:54:35.638 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:35.638 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:35.638 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:35.638 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:35.638 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:35.639 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:35.639 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28552 2024-05-08T08:54:35.639 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.639 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:35.639 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.639 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:35.639 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356228+0000", 2024-05-08T08:54:35.639 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:35.639 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.639 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.639 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:35.639 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356228+0000", 2024-05-08T08:54:35.639 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:35.639 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.639 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.639 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:35.639 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356959+0000", 2024-05-08T08:54:35.640 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00073097099999999996 2024-05-08T08:54:35.640 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.640 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.640 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:35.640 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356967+0000", 2024-05-08T08:54:35.640 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0409999999999998e-06 2024-05-08T08:54:35.640 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.640 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.640 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:35.640 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356979+0000", 2024-05-08T08:54:35.640 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1489e-05 2024-05-08T08:54:35.640 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.640 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.640 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:35.640 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356986+0000", 2024-05-08T08:54:35.640 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3699999999999997e-06 2024-05-08T08:54:35.641 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.641 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.641 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:35.641 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435433+0000", 2024-05-08T08:54:35.641 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078447520000000007 2024-05-08T08:54:35.641 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.641 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.641 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:35.641 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435484+0000", 2024-05-08T08:54:35.641 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0229000000000003e-05 2024-05-08T08:54:35.641 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.641 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.641 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:35.641 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435575+0000", 2024-05-08T08:54:35.641 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1299999999999997e-05 2024-05-08T08:54:35.641 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.641 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:35.642 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.642 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.642 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.642 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28553 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888713648] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:35.642 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:54:35.642 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 63.249367452000001, 2024-05-08T08:54:35.642 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078782034000000001, 2024-05-08T08:54:35.642 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:35.642 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:35.642 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:35.642 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:35.642 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:35.642 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:35.642 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28553 2024-05-08T08:54:35.642 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.642 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:35.643 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.643 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:35.643 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:54:35.643 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:35.643 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.643 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.643 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:35.643 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:54:35.643 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:35.643 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.643 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.643 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:35.643 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:54:35.643 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6600000000000002e-07 2024-05-08T08:54:35.644 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.644 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.644 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:35.644 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357006+0000", 2024-05-08T08:54:35.644 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7190000000000001e-06 2024-05-08T08:54:35.644 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.644 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.644 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:35.644 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357013+0000", 2024-05-08T08:54:35.644 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.435e-06 2024-05-08T08:54:35.644 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.644 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.644 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:35.644 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357018+0000", 2024-05-08T08:54:35.644 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7260000000000002e-06 2024-05-08T08:54:35.644 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.645 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.645 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:35.645 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435623+0000", 2024-05-08T08:54:35.645 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078604972999999995 2024-05-08T08:54:35.645 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.645 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.645 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:35.645 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435695+0000", 2024-05-08T08:54:35.645 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1959999999999995e-05 2024-05-08T08:54:35.645 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.645 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.645 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:35.645 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435785+0000", 2024-05-08T08:54:35.645 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.9754999999999997e-05 2024-05-08T08:54:35.645 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.646 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:35.646 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.646 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.646 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.646 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28554 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888980272] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:35.646 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357024+0000", 2024-05-08T08:54:35.646 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 63.249345816000002, 2024-05-08T08:54:35.646 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078999870999999999, 2024-05-08T08:54:35.646 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:35.646 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:35.646 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:35.646 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:35.646 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:35.646 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:35.646 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28554 2024-05-08T08:54:35.646 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.647 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:35.647 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.647 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:35.647 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357024+0000", 2024-05-08T08:54:35.647 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:35.647 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.647 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.647 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:35.647 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357024+0000", 2024-05-08T08:54:35.647 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:35.647 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.647 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.647 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:35.647 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357025+0000", 2024-05-08T08:54:35.647 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:54:35.647 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.648 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.648 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:35.648 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357027+0000", 2024-05-08T08:54:35.648 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2050000000000001e-06 2024-05-08T08:54:35.648 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.648 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.648 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:35.648 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357033+0000", 2024-05-08T08:54:35.648 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5679999999999996e-06 2024-05-08T08:54:35.648 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.648 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.648 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:35.648 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357038+0000", 2024-05-08T08:54:35.648 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2889999999999999e-06 2024-05-08T08:54:35.648 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.648 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.649 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:35.649 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435823+0000", 2024-05-08T08:54:35.649 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078785677999999998 2024-05-08T08:54:35.649 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.649 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.649 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:35.649 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435892+0000", 2024-05-08T08:54:35.649 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8379000000000004e-05 2024-05-08T08:54:35.649 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.649 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.649 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:35.649 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436024+0000", 2024-05-08T08:54:35.649 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013239799999999999 2024-05-08T08:54:35.649 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.649 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:35.650 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.650 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.650 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.650 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28555 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889001104] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:35.650 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:54:35.650 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 63.249327051000002, 2024-05-08T08:54:35.650 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079115742000000003, 2024-05-08T08:54:35.650 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:35.650 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:35.650 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:35.650 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:35.650 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:35.650 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:35.650 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28555 2024-05-08T08:54:35.650 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.651 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:35.651 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.651 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:35.651 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:54:35.651 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:35.651 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.651 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.651 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:35.651 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:54:35.651 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:35.651 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.651 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.651 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:35.651 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:54:35.651 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9200000000000002e-07 2024-05-08T08:54:35.651 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.652 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.652 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:35.652 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357046+0000", 2024-05-08T08:54:35.652 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1670000000000002e-06 2024-05-08T08:54:35.652 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.652 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.652 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:35.652 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357052+0000", 2024-05-08T08:54:35.652 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.139e-06 2024-05-08T08:54:35.652 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.652 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.652 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:35.652 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357056+0000", 2024-05-08T08:54:35.652 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2440000000000004e-06 2024-05-08T08:54:35.652 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.652 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.653 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:35.653 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436059+0000", 2024-05-08T08:54:35.653 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079003070999999994 2024-05-08T08:54:35.653 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.653 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.653 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:35.653 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436097+0000", 2024-05-08T08:54:35.653 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7784000000000003e-05 2024-05-08T08:54:35.653 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.653 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.653 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:35.653 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436159+0000", 2024-05-08T08:54:35.653 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2045000000000007e-05 2024-05-08T08:54:35.653 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.653 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:35.653 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.654 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.654 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.654 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28556 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889022064] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:35.654 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357065+0000", 2024-05-08T08:54:35.654 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 63.249305018000001, 2024-05-08T08:54:35.654 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079327976999999994, 2024-05-08T08:54:35.654 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:35.654 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:35.654 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:35.654 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:35.654 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:35.654 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:35.654 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28556 2024-05-08T08:54:35.654 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.654 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:35.655 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.655 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:35.655 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357065+0000", 2024-05-08T08:54:35.655 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:35.655 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.655 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.655 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:35.655 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357065+0000", 2024-05-08T08:54:35.655 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:35.655 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.655 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.655 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:35.655 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358958+0000", 2024-05-08T08:54:35.655 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001892915 2024-05-08T08:54:35.655 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.656 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.656 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:35.656 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358962+0000", 2024-05-08T08:54:35.656 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4599999999999999e-06 2024-05-08T08:54:35.656 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.656 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.656 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:35.656 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358970+0000", 2024-05-08T08:54:35.656 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.6090000000000001e-06 2024-05-08T08:54:35.656 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.656 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.656 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:35.656 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358975+0000", 2024-05-08T08:54:35.656 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7639999999999996e-06 2024-05-08T08:54:35.656 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.656 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.656 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:35.657 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436196+0000", 2024-05-08T08:54:35.657 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.077221564000000006 2024-05-08T08:54:35.657 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.657 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.657 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:35.657 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436269+0000", 2024-05-08T08:54:35.657 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2720999999999997e-05 2024-05-08T08:54:35.657 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.657 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.657 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:35.657 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436393+0000", 2024-05-08T08:54:35.657 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012394399999999999 2024-05-08T08:54:35.657 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.657 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:35.657 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.657 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.658 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.658 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28563 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888855520] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:35.658 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:54:35.658 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 63.247295618999999, 2024-05-08T08:54:35.658 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078795566999999997, 2024-05-08T08:54:35.658 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:35.658 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:35.658 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:35.658 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:35.658 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:35.658 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:35.658 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28563 2024-05-08T08:54:35.658 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.658 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:35.658 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.659 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:35.659 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:54:35.659 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:35.659 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.659 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.659 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:35.659 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:54:35.659 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:35.659 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.659 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.659 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:35.659 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:54:35.659 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1e-07 2024-05-08T08:54:35.659 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.659 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.659 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:35.659 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359080+0000", 2024-05-08T08:54:35.660 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8940000000000001e-06 2024-05-08T08:54:35.660 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.660 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.660 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:35.660 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359083+0000", 2024-05-08T08:54:35.660 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1949999999999999e-06 2024-05-08T08:54:35.660 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.660 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.660 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:35.660 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359085+0000", 2024-05-08T08:54:35.660 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4710000000000001e-06 2024-05-08T08:54:35.660 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.660 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.660 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:35.661 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437604+0000", 2024-05-08T08:54:35.661 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078518278999999996 2024-05-08T08:54:35.661 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.661 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.661 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:35.661 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437704+0000", 2024-05-08T08:54:35.661 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010010599999999999 2024-05-08T08:54:35.661 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.661 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.661 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:35.661 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437870+0000", 2024-05-08T08:54:35.661 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00016641200000000001 2024-05-08T08:54:35.661 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.661 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:35.661 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.662 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.662 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.662 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28564 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888538944] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:35.662 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.359089+0000", 2024-05-08T08:54:35.662 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 63.247281403999999, 2024-05-08T08:54:35.662 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078952675, 2024-05-08T08:54:35.662 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:35.662 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:35.662 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:35.662 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:35.662 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:35.662 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:35.662 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28564 2024-05-08T08:54:35.662 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.662 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:35.662 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.663 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:35.663 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359089+0000", 2024-05-08T08:54:35.663 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:35.663 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.663 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.663 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:35.663 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359089+0000", 2024-05-08T08:54:35.663 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:35.663 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.663 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.663 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:35.663 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360959+0000", 2024-05-08T08:54:35.663 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0018707089999999999 2024-05-08T08:54:35.663 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.663 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.663 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:35.663 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360963+0000", 2024-05-08T08:54:35.664 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0680000000000001e-06 2024-05-08T08:54:35.664 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.664 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.664 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:35.664 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360969+0000", 2024-05-08T08:54:35.664 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1920000000000002e-06 2024-05-08T08:54:35.664 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.664 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.664 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:35.664 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360973+0000", 2024-05-08T08:54:35.664 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.031e-06 2024-05-08T08:54:35.664 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.664 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.664 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:35.664 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437916+0000", 2024-05-08T08:54:35.665 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.076942939000000002 2024-05-08T08:54:35.665 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.665 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.665 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:35.665 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437973+0000", 2024-05-08T08:54:35.665 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7358999999999997e-05 2024-05-08T08:54:35.665 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.665 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.665 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:35.665 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.438041+0000", 2024-05-08T08:54:35.665 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8376999999999999e-05 2024-05-08T08:54:35.665 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.665 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:35.665 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.665 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.665 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.666 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28574 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889427600] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:35.666 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:54:35.666 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 63.245276896999997, 2024-05-08T08:54:35.666 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078782476000000004, 2024-05-08T08:54:35.666 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:35.666 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:35.666 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:35.666 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:35.666 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:35.666 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:35.666 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28574 2024-05-08T08:54:35.666 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.666 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:35.666 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.666 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:35.666 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:54:35.667 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:35.667 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.667 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.667 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:35.667 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:54:35.667 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:35.667 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.667 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.667 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:35.667 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:54:35.667 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1799999999999999e-07 2024-05-08T08:54:35.667 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.667 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.667 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:35.667 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361095+0000", 2024-05-08T08:54:35.668 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3850000000000001e-06 2024-05-08T08:54:35.668 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.668 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.668 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:35.668 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361098+0000", 2024-05-08T08:54:35.668 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7989999999999998e-06 2024-05-08T08:54:35.668 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.668 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.668 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:35.668 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361100+0000", 2024-05-08T08:54:35.668 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4669999999999998e-06 2024-05-08T08:54:35.668 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.668 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.668 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:35.668 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439768+0000", 2024-05-08T08:54:35.668 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078668057999999999 2024-05-08T08:54:35.669 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.669 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.669 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:35.669 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439811+0000", 2024-05-08T08:54:35.669 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2459000000000001e-05 2024-05-08T08:54:35.669 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.669 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.669 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:35.669 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439876+0000", 2024-05-08T08:54:35.669 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5090000000000002e-05 2024-05-08T08:54:35.669 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.669 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:35.669 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.669 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.669 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.670 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28575 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889439216] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:35.670 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361103+0000", 2024-05-08T08:54:35.670 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 63.245266882000003, 2024-05-08T08:54:35.670 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078966828000000003, 2024-05-08T08:54:35.670 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:35.670 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:35.670 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:35.670 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:35.670 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:35.670 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:35.670 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28575 2024-05-08T08:54:35.670 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.670 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:35.670 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.670 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:35.671 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361103+0000", 2024-05-08T08:54:35.671 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:35.671 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.671 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.671 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:35.671 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361103+0000", 2024-05-08T08:54:35.671 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:35.671 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.671 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.671 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:35.671 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361104+0000", 2024-05-08T08:54:35.671 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1400000000000001e-07 2024-05-08T08:54:35.671 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.671 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.671 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:35.671 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361105+0000", 2024-05-08T08:54:35.672 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5540000000000001e-06 2024-05-08T08:54:35.672 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.672 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.672 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:35.672 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361108+0000", 2024-05-08T08:54:35.672 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.841e-06 2024-05-08T08:54:35.672 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.672 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.672 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:35.672 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361110+0000", 2024-05-08T08:54:35.672 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.497e-06 2024-05-08T08:54:35.672 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.672 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.672 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:35.672 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439916+0000", 2024-05-08T08:54:35.672 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078805372999999998 2024-05-08T08:54:35.673 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.673 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.673 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:35.673 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440000+0000", 2024-05-08T08:54:35.673 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4730999999999999e-05 2024-05-08T08:54:35.673 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.673 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.673 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:35.673 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440070+0000", 2024-05-08T08:54:35.673 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9617999999999997e-05 2024-05-08T08:54:35.673 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.673 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:35.673 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.673 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.673 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.674 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28576 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889451328] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:35.674 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361113+0000", 2024-05-08T08:54:35.674 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 63.245256695999998, 2024-05-08T08:54:35.674 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079088862999999995, 2024-05-08T08:54:35.674 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:35.674 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:35.674 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:35.674 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:35.674 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:35.674 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:35.674 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28576 2024-05-08T08:54:35.674 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.674 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:35.674 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.674 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:35.675 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361113+0000", 2024-05-08T08:54:35.675 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:35.675 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.675 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.675 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:35.675 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361113+0000", 2024-05-08T08:54:35.675 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:35.675 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.675 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.675 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:35.675 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361114+0000", 2024-05-08T08:54:35.675 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1299999999999999e-07 2024-05-08T08:54:35.675 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.675 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.676 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:35.676 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361115+0000", 2024-05-08T08:54:35.676 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.4300000000000001e-06 2024-05-08T08:54:35.676 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.676 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.676 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:35.676 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361118+0000", 2024-05-08T08:54:35.676 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9000000000000002e-06 2024-05-08T08:54:35.676 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.676 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.676 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:35.676 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361120+0000", 2024-05-08T08:54:35.676 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4739999999999999e-06 2024-05-08T08:54:35.676 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.676 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.677 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:35.677 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440096+0000", 2024-05-08T08:54:35.677 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078975629000000006 2024-05-08T08:54:35.677 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.677 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.677 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:35.677 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440135+0000", 2024-05-08T08:54:35.677 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9202000000000002e-05 2024-05-08T08:54:35.677 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.677 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.677 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:35.677 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440202+0000", 2024-05-08T08:54:35.677 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7014999999999994e-05 2024-05-08T08:54:35.677 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.677 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:35.677 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.678 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.678 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.678 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28577 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889462640] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:35.678 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:54:35.678 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 63.245246514000002, 2024-05-08T08:54:35.678 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079194622000000006, 2024-05-08T08:54:35.678 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:35.678 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:35.678 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:35.678 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:35.678 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:35.678 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:35.678 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28577 2024-05-08T08:54:35.678 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.678 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:35.679 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.679 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:35.679 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:54:35.679 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:35.679 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.679 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.679 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:35.679 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:54:35.679 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:35.679 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.679 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.679 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:35.679 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:54:35.679 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1400000000000001e-07 2024-05-08T08:54:35.679 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.680 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.680 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:35.680 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361125+0000", 2024-05-08T08:54:35.680 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.372e-06 2024-05-08T08:54:35.680 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.680 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.680 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:35.680 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361128+0000", 2024-05-08T08:54:35.680 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0309999999999999e-06 2024-05-08T08:54:35.680 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.680 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.680 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:35.680 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361131+0000", 2024-05-08T08:54:35.680 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4040000000000002e-06 2024-05-08T08:54:35.680 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.680 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.681 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:35.681 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440225+0000", 2024-05-08T08:54:35.681 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079094600000000001 2024-05-08T08:54:35.681 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.681 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.681 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:35.681 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440260+0000", 2024-05-08T08:54:35.681 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4703000000000003e-05 2024-05-08T08:54:35.681 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.681 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.681 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:35.681 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440318+0000", 2024-05-08T08:54:35.681 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8298000000000003e-05 2024-05-08T08:54:35.681 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.681 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:35.681 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.682 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.682 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.682 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28578 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889474192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:35.682 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361134+0000", 2024-05-08T08:54:35.682 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 63.245236089000002, 2024-05-08T08:54:35.682 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079310295000000003, 2024-05-08T08:54:35.682 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:35.682 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:35.682 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:35.682 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:35.682 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:35.682 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:35.682 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28578 2024-05-08T08:54:35.682 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.682 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:35.683 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.683 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:35.683 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361134+0000", 2024-05-08T08:54:35.683 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:35.683 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.683 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.683 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:35.683 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361134+0000", 2024-05-08T08:54:35.683 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:35.683 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.683 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.683 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:35.683 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.362958+0000", 2024-05-08T08:54:35.683 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0018237380000000001 2024-05-08T08:54:35.683 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.684 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.684 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:35.684 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.362961+0000", 2024-05-08T08:54:35.684 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.123e-06 2024-05-08T08:54:35.684 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.684 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.684 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:35.684 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.397404+0000", 2024-05-08T08:54:35.684 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.034443226 2024-05-08T08:54:35.684 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.684 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.684 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:35.684 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.397439+0000", 2024-05-08T08:54:35.684 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4690000000000002e-05 2024-05-08T08:54:35.684 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.685 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.685 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:35.685 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440340+0000", 2024-05-08T08:54:35.685 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.042901419000000003 2024-05-08T08:54:35.685 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.685 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.685 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:35.685 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440376+0000", 2024-05-08T08:54:35.685 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5444999999999997e-05 2024-05-08T08:54:35.685 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:35.685 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:35.685 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:35.685 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440444+0000", 2024-05-08T08:54:35.685 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8653999999999997e-05 2024-05-08T08:54:35.685 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.686 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:35.686 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.686 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:35.686 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:35.686 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:35.687 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:54:35.688 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:35.689+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:35.788 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:35.791+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:35.797 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:35.797 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:54:35.797 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:54:35.797 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:35.806 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:54:35.820 INFO:teuthology.orchestra.run.smithi059.stderr:reweighted osd.1 to 1 (10000) 2024-05-08T08:54:35.823 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-05-08T08:54:35.839 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-05-08T08:54:35.889 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:35.892+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:35.938 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:35.938 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:54:35.938 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:54:35.938 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:54:35.938 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:35.949 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:54:35.990 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:35.992+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:36.071 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:36.071 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:54:36.071 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:54:36.071 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:54:36.071 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.071 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:437 3.17 3:ea93a512:::rbd_data.1140100c9703.0000000000000077:head [set-alloc-hint object_size 2097152 write_size 2097152,write 1170432~1024] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e63)", 2024-05-08T08:54:36.071 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:08.390804+0000", 2024-05-08T08:54:36.071 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 87.681412859000005, 2024-05-08T08:54:36.072 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0406068690000001, 2024-05-08T08:54:36.072 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:36.072 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:36.072 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:54:36.072 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:36.072 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:36.072 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:36.072 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 437 2024-05-08T08:54:36.072 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.072 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:36.072 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.072 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:36.072 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390804+0000", 2024-05-08T08:54:36.072 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:36.072 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.072 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.073 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:36.073 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390804+0000", 2024-05-08T08:54:36.073 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:36.073 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.073 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.073 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:36.073 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390806+0000", 2024-05-08T08:54:36.073 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.666e-06 2024-05-08T08:54:36.073 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.073 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.073 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:36.073 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390818+0000", 2024-05-08T08:54:36.073 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2187999999999999e-05 2024-05-08T08:54:36.073 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.073 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.073 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:36.074 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390834+0000", 2024-05-08T08:54:36.074 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6191000000000001e-05 2024-05-08T08:54:36.074 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.074 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.074 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:36.074 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390845+0000", 2024-05-08T08:54:36.074 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0831e-05 2024-05-08T08:54:36.074 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.074 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.074 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:36.074 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390922+0000", 2024-05-08T08:54:36.074 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7253e-05 2024-05-08T08:54:36.074 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.074 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.074 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op must wait for map", 2024-05-08T08:54:36.074 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390930+0000", 2024-05-08T08:54:36.075 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8259999999999993e-06 2024-05-08T08:54:36.075 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.075 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.075 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:36.075 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392155+0000", 2024-05-08T08:54:36.075 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0012256649999999999 2024-05-08T08:54:36.075 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.075 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.075 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:54:36.075 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392165+0000", 2024-05-08T08:54:36.075 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5799999999999998e-06 2024-05-08T08:54:36.075 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.076 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.076 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:36.076 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.429596+0000", 2024-05-08T08:54:36.076 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0374313639999999 2024-05-08T08:54:36.076 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.076 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.077 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:36.077 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.429728+0000", 2024-05-08T08:54:36.077 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000131552 2024-05-08T08:54:36.077 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.077 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.077 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 0,1", 2024-05-08T08:54:36.077 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.429860+0000", 2024-05-08T08:54:36.077 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000132137 2024-05-08T08:54:36.077 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.077 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.077 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:54:36.077 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.430811+0000", 2024-05-08T08:54:36.077 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00095040200000000004 2024-05-08T08:54:36.077 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.077 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.077 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:54:36.077 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431126+0000", 2024-05-08T08:54:36.078 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000315006 2024-05-08T08:54:36.078 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.078 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.078 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:54:36.078 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431272+0000", 2024-05-08T08:54:36.078 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000146487 2024-05-08T08:54:36.078 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.078 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.078 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:54:36.078 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431315+0000", 2024-05-08T08:54:36.078 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2697000000000001e-05 2024-05-08T08:54:36.078 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.078 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.078 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:36.078 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431411+0000", 2024-05-08T08:54:36.079 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6024000000000001e-05 2024-05-08T08:54:36.079 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:36.079 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:36.079 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:36.079 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.079 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.079 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:439 3.17 3:ea93a512:::rbd_data.1140100c9703.0000000000000077:head [read 1163264~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-05-08T08:54:36.079 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:08.390871+0000", 2024-05-08T08:54:36.079 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 87.681345347999994, 2024-05-08T08:54:36.079 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0408790590000001, 2024-05-08T08:54:36.079 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:36.079 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:36.079 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:36.079 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:36.079 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:36.080 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:36.080 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 439 2024-05-08T08:54:36.080 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.080 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:36.080 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.080 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:36.080 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390871+0000", 2024-05-08T08:54:36.080 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:36.080 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.080 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.080 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:36.080 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390871+0000", 2024-05-08T08:54:36.080 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:36.080 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.080 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.080 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:36.081 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390872+0000", 2024-05-08T08:54:36.081 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6300000000000005e-07 2024-05-08T08:54:36.081 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.081 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.081 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:36.081 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390875+0000", 2024-05-08T08:54:36.081 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3459999999999998e-06 2024-05-08T08:54:36.081 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.081 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.081 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:36.081 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390891+0000", 2024-05-08T08:54:36.081 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5255e-05 2024-05-08T08:54:36.081 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.081 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.081 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:36.081 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390897+0000", 2024-05-08T08:54:36.082 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2360000000000001e-06 2024-05-08T08:54:36.082 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.082 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.082 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:36.082 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390995+0000", 2024-05-08T08:54:36.082 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8522000000000003e-05 2024-05-08T08:54:36.082 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.082 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.082 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting_for_map not empty", 2024-05-08T08:54:36.082 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.391005+0000", 2024-05-08T08:54:36.082 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1460000000000002e-06 2024-05-08T08:54:36.082 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.082 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.082 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:36.082 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392191+0000", 2024-05-08T08:54:36.083 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0011860270000000001 2024-05-08T08:54:36.083 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.083 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.083 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:54:36.083 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392197+0000", 2024-05-08T08:54:36.083 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4289999999999998e-06 2024-05-08T08:54:36.083 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.083 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.083 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:36.083 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.430596+0000", 2024-05-08T08:54:36.083 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0383989090000001 2024-05-08T08:54:36.083 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.083 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.083 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for rw locks", 2024-05-08T08:54:36.083 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.430676+0000", 2024-05-08T08:54:36.083 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9781000000000001e-05 2024-05-08T08:54:36.083 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.084 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.084 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:36.084 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431460+0000", 2024-05-08T08:54:36.084 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00078427300000000004 2024-05-08T08:54:36.084 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.084 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.084 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:36.084 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431540+0000", 2024-05-08T08:54:36.084 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0224999999999998e-05 2024-05-08T08:54:36.084 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.084 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.084 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:36.084 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431750+0000", 2024-05-08T08:54:36.084 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00021024699999999999 2024-05-08T08:54:36.084 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:36.084 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:36.084 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:36.085 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.085 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.085 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:945 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 4096000~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:36.085 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:54:36.085 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 86.621195780999997, 2024-05-08T08:54:36.085 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086774741000000002, 2024-05-08T08:54:36.085 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:36.085 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:36.085 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:36.085 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:36.085 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:36.085 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:36.085 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 945 2024-05-08T08:54:36.085 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.085 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:36.085 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.086 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:36.086 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:54:36.086 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:36.086 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.086 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.086 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:36.086 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:54:36.086 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:36.086 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.086 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.086 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:36.086 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:54:36.086 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3499999999999996e-07 2024-05-08T08:54:36.086 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.086 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.087 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:36.087 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451024+0000", 2024-05-08T08:54:36.087 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.627e-06 2024-05-08T08:54:36.087 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.087 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.087 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:36.087 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451037+0000", 2024-05-08T08:54:36.087 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2431e-05 2024-05-08T08:54:36.087 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.087 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.087 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:36.087 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451042+0000", 2024-05-08T08:54:36.087 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8409999999999996e-06 2024-05-08T08:54:36.087 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.087 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.087 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:36.088 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537608+0000", 2024-05-08T08:54:36.088 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086565243 2024-05-08T08:54:36.088 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.088 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.088 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:36.088 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537687+0000", 2024-05-08T08:54:36.088 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9519999999999998e-05 2024-05-08T08:54:36.088 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.088 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.088 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:36.088 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537796+0000", 2024-05-08T08:54:36.088 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000108544 2024-05-08T08:54:36.088 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:36.088 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:36.088 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:36.088 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.088 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.089 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:964 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 458752~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:36.089 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451051+0000", 2024-05-08T08:54:36.089 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 86.621165607999998, 2024-05-08T08:54:36.089 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086990490000000004, 2024-05-08T08:54:36.089 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:36.089 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:36.089 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:36.089 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:36.089 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:36.089 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:36.089 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 964 2024-05-08T08:54:36.089 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.089 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:36.089 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.089 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:36.090 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451051+0000", 2024-05-08T08:54:36.090 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:36.090 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.090 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.090 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:36.090 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451051+0000", 2024-05-08T08:54:36.090 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:36.090 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.090 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.090 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:36.090 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451052+0000", 2024-05-08T08:54:36.090 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7299999999999996e-07 2024-05-08T08:54:36.090 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.090 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.090 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:36.091 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451055+0000", 2024-05-08T08:54:36.091 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.2770000000000001e-06 2024-05-08T08:54:36.091 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.091 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.091 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:36.091 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451061+0000", 2024-05-08T08:54:36.091 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4060000000000002e-06 2024-05-08T08:54:36.091 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.091 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.091 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:36.091 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451066+0000", 2024-05-08T08:54:36.091 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0839999999999999e-06 2024-05-08T08:54:36.091 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.091 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.091 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:36.091 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537839+0000", 2024-05-08T08:54:36.092 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086772962999999995 2024-05-08T08:54:36.092 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.092 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.092 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:36.092 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537934+0000", 2024-05-08T08:54:36.092 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4742999999999997e-05 2024-05-08T08:54:36.092 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.092 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.092 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:36.092 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538042+0000", 2024-05-08T08:54:36.092 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000107444 2024-05-08T08:54:36.092 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:36.092 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:36.092 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:36.092 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.092 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.092 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:965 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 950272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:36.093 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451075+0000", 2024-05-08T08:54:36.093 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 86.621141546000004, 2024-05-08T08:54:36.093 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087230196999999995, 2024-05-08T08:54:36.093 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:36.093 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:36.093 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:36.093 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:36.093 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:36.093 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:36.093 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 965 2024-05-08T08:54:36.093 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.093 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:36.093 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.093 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:36.093 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451075+0000", 2024-05-08T08:54:36.093 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:36.093 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.094 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.094 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:36.094 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451075+0000", 2024-05-08T08:54:36.094 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:36.094 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.094 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.094 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:36.094 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451076+0000", 2024-05-08T08:54:36.094 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5000000000000002e-07 2024-05-08T08:54:36.094 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.094 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.094 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:36.094 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451078+0000", 2024-05-08T08:54:36.094 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.5639999999999999e-06 2024-05-08T08:54:36.094 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.094 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.094 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:36.095 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451085+0000", 2024-05-08T08:54:36.095 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5640000000000002e-06 2024-05-08T08:54:36.095 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.095 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.095 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:36.095 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451090+0000", 2024-05-08T08:54:36.095 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1699999999999996e-06 2024-05-08T08:54:36.095 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.095 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.095 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:36.095 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538106+0000", 2024-05-08T08:54:36.095 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087015888999999999 2024-05-08T08:54:36.095 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.095 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.095 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:36.095 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538190+0000", 2024-05-08T08:54:36.095 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3961999999999993e-05 2024-05-08T08:54:36.096 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.096 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.096 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:36.096 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538305+0000", 2024-05-08T08:54:36.096 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011529800000000001 2024-05-08T08:54:36.096 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:36.096 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:36.096 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:36.096 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.096 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.096 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:966 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 1441792~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:36.096 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:54:36.096 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 86.621118172999999, 2024-05-08T08:54:36.096 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087446323000000006, 2024-05-08T08:54:36.096 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:36.096 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:36.097 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:36.097 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:36.097 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:36.097 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:36.097 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 966 2024-05-08T08:54:36.097 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.097 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:36.097 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.097 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:36.097 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:54:36.097 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:36.097 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.097 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.097 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:36.097 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:54:36.097 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:36.097 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.098 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.098 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:36.098 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:54:36.098 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3700000000000001e-07 2024-05-08T08:54:36.098 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.098 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.098 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:36.098 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451101+0000", 2024-05-08T08:54:36.098 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2560000000000001e-06 2024-05-08T08:54:36.098 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.098 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.098 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:36.098 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451107+0000", 2024-05-08T08:54:36.098 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1070000000000001e-06 2024-05-08T08:54:36.098 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.098 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.098 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:36.099 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451112+0000", 2024-05-08T08:54:36.099 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1420000000000001e-06 2024-05-08T08:54:36.099 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.099 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.099 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:36.099 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538341+0000", 2024-05-08T08:54:36.099 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087228302999999993 2024-05-08T08:54:36.099 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.099 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.099 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:36.099 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538424+0000", 2024-05-08T08:54:36.099 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3497000000000004e-05 2024-05-08T08:54:36.099 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.099 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.099 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:36.100 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538545+0000", 2024-05-08T08:54:36.100 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000120581 2024-05-08T08:54:36.100 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:36.100 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:36.100 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:36.100 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.100 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.100 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:967 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:36.100 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451120+0000", 2024-05-08T08:54:36.100 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 86.621096698000002, 2024-05-08T08:54:36.100 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087632725999999994, 2024-05-08T08:54:36.100 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:36.100 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:36.100 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:36.100 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:36.100 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:36.101 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:36.101 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 967 2024-05-08T08:54:36.101 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.101 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:36.101 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.101 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:36.101 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451120+0000", 2024-05-08T08:54:36.101 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:36.101 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.101 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.101 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:36.101 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451120+0000", 2024-05-08T08:54:36.101 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:36.101 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.101 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.102 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:36.102 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451121+0000", 2024-05-08T08:54:36.102 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8800000000000002e-07 2024-05-08T08:54:36.102 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.102 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.102 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:36.102 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451130+0000", 2024-05-08T08:54:36.102 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2299999999999997e-06 2024-05-08T08:54:36.102 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.102 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.102 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:36.102 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451137+0000", 2024-05-08T08:54:36.102 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4669999999999998e-06 2024-05-08T08:54:36.102 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.102 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.102 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:36.102 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451143+0000", 2024-05-08T08:54:36.103 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2839999999999997e-06 2024-05-08T08:54:36.103 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.103 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.103 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:36.103 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538575+0000", 2024-05-08T08:54:36.103 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087432789999999996 2024-05-08T08:54:36.103 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.103 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.103 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:36.103 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538648+0000", 2024-05-08T08:54:36.103 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2877e-05 2024-05-08T08:54:36.103 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.103 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.103 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:36.103 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538753+0000", 2024-05-08T08:54:36.103 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010449 2024-05-08T08:54:36.103 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:36.104 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:36.104 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:36.104 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.104 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.104 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:968 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 2424832~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:36.104 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451152+0000", 2024-05-08T08:54:36.104 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 86.621064613000001, 2024-05-08T08:54:36.104 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087813325999999997, 2024-05-08T08:54:36.104 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:36.104 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:36.104 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:36.104 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:36.104 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:36.104 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:36.104 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 968 2024-05-08T08:54:36.104 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.105 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:36.105 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.105 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:36.105 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451152+0000", 2024-05-08T08:54:36.105 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:36.105 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.105 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.105 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:36.105 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451152+0000", 2024-05-08T08:54:36.105 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:36.105 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.105 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.105 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:36.105 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451153+0000", 2024-05-08T08:54:36.105 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3099999999999998e-07 2024-05-08T08:54:36.105 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.106 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.106 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:36.106 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451155+0000", 2024-05-08T08:54:36.106 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4760000000000001e-06 2024-05-08T08:54:36.106 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.106 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.106 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:36.106 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451162+0000", 2024-05-08T08:54:36.106 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5969999999999997e-06 2024-05-08T08:54:36.106 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.106 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.106 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:36.106 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451167+0000", 2024-05-08T08:54:36.106 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.977e-06 2024-05-08T08:54:36.106 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.106 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.106 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:36.107 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538781+0000", 2024-05-08T08:54:36.107 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087614846999999996 2024-05-08T08:54:36.107 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.107 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.107 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:36.107 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538857+0000", 2024-05-08T08:54:36.107 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5358000000000006e-05 2024-05-08T08:54:36.107 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.107 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.107 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:36.107 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538965+0000", 2024-05-08T08:54:36.107 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010864 2024-05-08T08:54:36.107 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:36.107 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:36.107 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:36.107 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.108 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.108 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:969 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 2916352~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:36.108 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:54:36.108 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 86.621042908000007, 2024-05-08T08:54:36.108 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087998980000000004, 2024-05-08T08:54:36.108 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:36.108 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:36.108 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:36.108 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:36.108 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:36.108 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:36.108 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 969 2024-05-08T08:54:36.108 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.108 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:36.108 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.108 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:36.108 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:54:36.109 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:36.109 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.109 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.109 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:36.109 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:54:36.109 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:36.109 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.109 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.109 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:36.109 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:54:36.109 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3599999999999999e-07 2024-05-08T08:54:36.109 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.109 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.109 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:36.109 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451177+0000", 2024-05-08T08:54:36.109 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4590000000000001e-06 2024-05-08T08:54:36.109 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.110 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.110 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:36.110 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451184+0000", 2024-05-08T08:54:36.110 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4109999999999999e-06 2024-05-08T08:54:36.110 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.110 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.110 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:36.110 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451191+0000", 2024-05-08T08:54:36.110 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7279999999999998e-06 2024-05-08T08:54:36.110 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.110 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.110 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:36.110 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538997+0000", 2024-05-08T08:54:36.110 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087806170000000003 2024-05-08T08:54:36.110 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.111 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.111 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:36.111 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539069+0000", 2024-05-08T08:54:36.111 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2373999999999995e-05 2024-05-08T08:54:36.111 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.111 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.111 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:36.111 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539173+0000", 2024-05-08T08:54:36.111 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000103402 2024-05-08T08:54:36.111 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:36.111 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:36.111 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:36.111 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.111 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.112 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:970 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 3407872~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:36.112 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451198+0000", 2024-05-08T08:54:36.112 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 86.621018311, 2024-05-08T08:54:36.112 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088183368999999998, 2024-05-08T08:54:36.112 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:36.112 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:36.112 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:36.112 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:36.112 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:36.112 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:36.112 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 970 2024-05-08T08:54:36.112 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.112 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:36.112 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.112 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:36.112 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451198+0000", 2024-05-08T08:54:36.112 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:36.113 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.113 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.113 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:36.113 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451198+0000", 2024-05-08T08:54:36.113 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:36.113 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.113 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.113 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:36.113 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451199+0000", 2024-05-08T08:54:36.113 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7400000000000003e-07 2024-05-08T08:54:36.113 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.113 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.113 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:36.113 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451211+0000", 2024-05-08T08:54:36.113 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2109999999999999e-05 2024-05-08T08:54:36.113 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.114 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.114 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:36.114 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451219+0000", 2024-05-08T08:54:36.114 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4170000000000003e-06 2024-05-08T08:54:36.114 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.114 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.114 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:36.114 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451224+0000", 2024-05-08T08:54:36.114 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3990000000000003e-06 2024-05-08T08:54:36.114 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.114 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.114 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:36.114 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539204+0000", 2024-05-08T08:54:36.114 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087980099000000006 2024-05-08T08:54:36.114 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.115 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.115 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:36.115 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539276+0000", 2024-05-08T08:54:36.115 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1513000000000005e-05 2024-05-08T08:54:36.115 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.115 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.115 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:36.115 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539382+0000", 2024-05-08T08:54:36.115 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010615699999999999 2024-05-08T08:54:36.115 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:36.115 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:36.115 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:36.115 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.115 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.115 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:971 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 3899392~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:36.115 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451233+0000", 2024-05-08T08:54:36.115 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 86.620983604000003, 2024-05-08T08:54:36.115 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088431525999999996, 2024-05-08T08:54:36.116 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:36.116 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:36.116 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:36.116 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:36.116 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:36.116 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:36.116 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 971 2024-05-08T08:54:36.116 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.116 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:36.116 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.116 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:36.116 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451233+0000", 2024-05-08T08:54:36.116 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:36.116 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.116 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.116 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:36.116 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451233+0000", 2024-05-08T08:54:36.117 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:36.117 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.117 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.117 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:36.117 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451234+0000", 2024-05-08T08:54:36.117 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8500000000000001e-07 2024-05-08T08:54:36.117 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.117 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.117 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:36.117 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451237+0000", 2024-05-08T08:54:36.117 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8320000000000002e-06 2024-05-08T08:54:36.117 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.117 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.117 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:36.117 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451243+0000", 2024-05-08T08:54:36.117 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6749999999999996e-06 2024-05-08T08:54:36.117 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.118 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.118 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:36.118 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451249+0000", 2024-05-08T08:54:36.118 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1229999999999997e-06 2024-05-08T08:54:36.118 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.118 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.118 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:36.118 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539412+0000", 2024-05-08T08:54:36.118 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088162668999999999 2024-05-08T08:54:36.118 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.118 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.118 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:36.118 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539505+0000", 2024-05-08T08:54:36.118 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3240999999999997e-05 2024-05-08T08:54:36.118 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.118 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.118 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:36.119 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539665+0000", 2024-05-08T08:54:36.119 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00015930100000000001 2024-05-08T08:54:36.119 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:36.119 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:36.119 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:36.119 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.119 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.119 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:972 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 196608~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:36.119 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451257+0000", 2024-05-08T08:54:36.119 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 86.620959658999993, 2024-05-08T08:54:36.119 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088719928000000003, 2024-05-08T08:54:36.119 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:36.119 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:36.119 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:36.119 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:36.119 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:36.119 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:36.120 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 972 2024-05-08T08:54:36.120 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.120 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:36.120 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.120 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:36.120 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451257+0000", 2024-05-08T08:54:36.120 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:36.120 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.120 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.120 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:36.120 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451257+0000", 2024-05-08T08:54:36.120 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:36.120 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.120 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.120 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:36.120 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451258+0000", 2024-05-08T08:54:36.121 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7999999999999996e-07 2024-05-08T08:54:36.121 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.121 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.121 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:36.121 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451260+0000", 2024-05-08T08:54:36.121 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4609999999999998e-06 2024-05-08T08:54:36.121 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.121 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.121 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:36.121 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451267+0000", 2024-05-08T08:54:36.121 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.81e-06 2024-05-08T08:54:36.121 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.121 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.121 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:36.121 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451273+0000", 2024-05-08T08:54:36.121 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0859999999999998e-06 2024-05-08T08:54:36.122 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.122 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.122 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:36.122 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539718+0000", 2024-05-08T08:54:36.122 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088445059000000006 2024-05-08T08:54:36.122 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.122 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.122 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:36.122 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539834+0000", 2024-05-08T08:54:36.122 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000116426 2024-05-08T08:54:36.122 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.122 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.122 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:36.122 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539977+0000", 2024-05-08T08:54:36.122 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014260600000000001 2024-05-08T08:54:36.122 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:36.123 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:36.123 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:36.123 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.123 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.123 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:973 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 688128~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:36.123 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:54:36.123 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 86.620937075000001, 2024-05-08T08:54:36.123 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088918496, 2024-05-08T08:54:36.123 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:36.123 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:36.123 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:36.123 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:36.123 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:36.123 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:36.123 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 973 2024-05-08T08:54:36.123 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.123 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:36.124 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.124 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:36.124 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:54:36.124 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:36.124 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.124 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.124 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:36.124 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:54:36.124 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:36.124 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.124 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.124 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:36.124 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:54:36.124 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9099999999999999e-07 2024-05-08T08:54:36.124 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.124 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.124 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:36.124 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451282+0000", 2024-05-08T08:54:36.125 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1550000000000001e-06 2024-05-08T08:54:36.125 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.125 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.125 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:36.125 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451288+0000", 2024-05-08T08:54:36.125 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8889999999999999e-06 2024-05-08T08:54:36.125 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.125 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.125 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:36.125 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451294+0000", 2024-05-08T08:54:36.125 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0850000000000002e-06 2024-05-08T08:54:36.125 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.125 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.125 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:36.125 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540011+0000", 2024-05-08T08:54:36.126 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088717314000000005 2024-05-08T08:54:36.126 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.126 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.126 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:36.126 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540089+0000", 2024-05-08T08:54:36.126 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7438e-05 2024-05-08T08:54:36.126 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.126 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.126 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:36.126 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540198+0000", 2024-05-08T08:54:36.126 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000109224 2024-05-08T08:54:36.126 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:36.126 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:36.126 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:36.126 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.126 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.126 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:974 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 1179648~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:36.127 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451301+0000", 2024-05-08T08:54:36.127 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 86.620915441999998, 2024-05-08T08:54:36.127 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089105078000000004, 2024-05-08T08:54:36.127 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:36.127 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:36.127 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:36.127 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:36.127 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:36.127 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:36.127 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 974 2024-05-08T08:54:36.127 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.127 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:36.127 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.127 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:36.128 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451301+0000", 2024-05-08T08:54:36.128 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:36.128 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.128 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.128 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:36.128 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451301+0000", 2024-05-08T08:54:36.128 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:36.128 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.128 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.128 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:36.128 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451302+0000", 2024-05-08T08:54:36.128 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7800000000000002e-07 2024-05-08T08:54:36.128 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.128 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.128 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:36.129 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451304+0000", 2024-05-08T08:54:36.129 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.65e-06 2024-05-08T08:54:36.129 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.129 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.129 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:36.129 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451311+0000", 2024-05-08T08:54:36.129 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.472e-06 2024-05-08T08:54:36.129 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.129 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.129 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:36.129 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451316+0000", 2024-05-08T08:54:36.129 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9189999999999998e-06 2024-05-08T08:54:36.129 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.129 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.129 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:36.129 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540234+0000", 2024-05-08T08:54:36.130 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088917752000000003 2024-05-08T08:54:36.130 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.130 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.130 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:36.130 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540308+0000", 2024-05-08T08:54:36.130 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4758000000000005e-05 2024-05-08T08:54:36.130 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.130 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.130 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:36.130 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540406+0000", 2024-05-08T08:54:36.130 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8048999999999996e-05 2024-05-08T08:54:36.130 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:36.130 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:36.130 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:36.131 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.131 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.131 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:975 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 1671168~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:36.131 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:54:36.131 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 86.620873111999998, 2024-05-08T08:54:36.131 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089294139999999994, 2024-05-08T08:54:36.131 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:36.131 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:36.131 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:36.131 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:36.131 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:36.131 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:36.131 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 975 2024-05-08T08:54:36.131 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.131 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:36.131 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.131 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:36.132 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:54:36.132 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:36.132 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.132 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.132 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:36.132 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:54:36.132 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:36.132 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.132 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.132 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:36.132 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:54:36.132 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1299999999999999e-07 2024-05-08T08:54:36.133 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.133 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.133 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:36.133 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451348+0000", 2024-05-08T08:54:36.133 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.484e-06 2024-05-08T08:54:36.133 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.133 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.133 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:36.133 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451356+0000", 2024-05-08T08:54:36.133 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8499999999999994e-06 2024-05-08T08:54:36.133 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.133 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.133 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:36.133 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451366+0000", 2024-05-08T08:54:36.133 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.9850000000000001e-06 2024-05-08T08:54:36.133 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.134 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.134 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:36.134 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540448+0000", 2024-05-08T08:54:36.134 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089082254 2024-05-08T08:54:36.134 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.134 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.134 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:36.134 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540528+0000", 2024-05-08T08:54:36.134 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0234999999999993e-05 2024-05-08T08:54:36.134 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.134 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.134 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:36.134 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540638+0000", 2024-05-08T08:54:36.135 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010951900000000001 2024-05-08T08:54:36.135 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:36.135 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:36.135 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:36.135 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.135 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.135 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:976 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 2162688~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:36.135 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451375+0000", 2024-05-08T08:54:36.135 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 86.620841365000004, 2024-05-08T08:54:36.135 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089472745000000006, 2024-05-08T08:54:36.135 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:36.135 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:36.135 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:36.135 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:36.135 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:36.135 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:36.135 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 976 2024-05-08T08:54:36.136 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.136 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:36.136 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.136 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:36.136 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451375+0000", 2024-05-08T08:54:36.136 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:36.136 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.136 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.136 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:36.136 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451375+0000", 2024-05-08T08:54:36.136 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:36.136 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.136 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.136 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:36.137 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451376+0000", 2024-05-08T08:54:36.137 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2399999999999998e-07 2024-05-08T08:54:36.137 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.137 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.137 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:36.137 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451379+0000", 2024-05-08T08:54:36.137 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6359999999999998e-06 2024-05-08T08:54:36.137 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.137 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.137 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:36.137 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451387+0000", 2024-05-08T08:54:36.137 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9929999999999995e-06 2024-05-08T08:54:36.137 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.137 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.137 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:36.137 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451393+0000", 2024-05-08T08:54:36.138 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4699999999999999e-06 2024-05-08T08:54:36.138 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.138 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.138 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:36.138 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540677+0000", 2024-05-08T08:54:36.138 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089283504999999999 2024-05-08T08:54:36.138 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.138 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.138 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:36.139 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540743+0000", 2024-05-08T08:54:36.139 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6303999999999994e-05 2024-05-08T08:54:36.139 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.139 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.139 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:36.139 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540848+0000", 2024-05-08T08:54:36.139 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000105213 2024-05-08T08:54:36.139 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:36.139 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:36.139 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:36.139 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.139 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.139 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:977 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 2654208~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:36.139 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:54:36.139 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 86.620815933000003, 2024-05-08T08:54:36.139 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089698473000000001, 2024-05-08T08:54:36.140 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:36.140 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:36.140 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:36.140 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:36.140 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:36.140 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:36.140 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 977 2024-05-08T08:54:36.140 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.140 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:36.140 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.140 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:36.140 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:54:36.140 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:36.140 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.140 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.141 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:36.141 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:54:36.141 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:36.141 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.141 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.141 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:36.141 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:54:36.141 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2899999999999999e-07 2024-05-08T08:54:36.141 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.141 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.141 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:36.141 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451404+0000", 2024-05-08T08:54:36.141 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6649999999999999e-06 2024-05-08T08:54:36.142 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.142 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.142 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:36.142 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451410+0000", 2024-05-08T08:54:36.142 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3269999999999997e-06 2024-05-08T08:54:36.142 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.142 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.142 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:36.142 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451415+0000", 2024-05-08T08:54:36.142 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9529999999999998e-06 2024-05-08T08:54:36.142 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.142 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.142 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:36.143 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540912+0000", 2024-05-08T08:54:36.143 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089496950000000006 2024-05-08T08:54:36.143 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.143 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.143 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:36.143 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541000+0000", 2024-05-08T08:54:36.143 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7429000000000006e-05 2024-05-08T08:54:36.143 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.143 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.143 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:36.143 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541099+0000", 2024-05-08T08:54:36.143 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.9720000000000001e-05 2024-05-08T08:54:36.143 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:36.143 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:36.143 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:36.143 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.143 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.144 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:978 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 3145728~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:36.144 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:54:36.144 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 86.620794724999996, 2024-05-08T08:54:36.144 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089900635000000007, 2024-05-08T08:54:36.144 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:36.144 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:36.144 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:36.144 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:36.144 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:36.144 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:36.144 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 978 2024-05-08T08:54:36.144 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.144 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:36.144 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.144 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:36.144 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:54:36.144 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:36.145 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.145 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.145 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:36.145 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:54:36.145 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:36.145 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.145 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.145 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:36.145 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:54:36.145 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9400000000000001e-07 2024-05-08T08:54:36.145 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.145 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.145 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:36.145 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451425+0000", 2024-05-08T08:54:36.145 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2460000000000002e-06 2024-05-08T08:54:36.145 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.146 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.146 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:36.146 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451431+0000", 2024-05-08T08:54:36.146 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2419999999999997e-06 2024-05-08T08:54:36.146 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.146 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.146 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:36.146 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451436+0000", 2024-05-08T08:54:36.146 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.887e-06 2024-05-08T08:54:36.146 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.146 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.146 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:36.146 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541152+0000", 2024-05-08T08:54:36.146 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089716194999999999 2024-05-08T08:54:36.146 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.147 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.147 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:36.147 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541224+0000", 2024-05-08T08:54:36.147 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2416000000000006e-05 2024-05-08T08:54:36.147 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.147 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.147 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:36.147 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541323+0000", 2024-05-08T08:54:36.147 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8255e-05 2024-05-08T08:54:36.147 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:36.147 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:36.147 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:36.147 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.147 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.147 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:979 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 3637248~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:36.148 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451462+0000", 2024-05-08T08:54:36.148 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 86.620754276, 2024-05-08T08:54:36.148 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.090036441999999994, 2024-05-08T08:54:36.148 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:36.148 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:36.148 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:36.148 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:36.148 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:36.148 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:36.148 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 979 2024-05-08T08:54:36.148 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.148 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:36.148 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.148 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:36.148 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451462+0000", 2024-05-08T08:54:36.148 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:36.149 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.149 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.149 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:36.149 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451462+0000", 2024-05-08T08:54:36.149 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:36.149 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.149 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.149 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:36.149 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451463+0000", 2024-05-08T08:54:36.149 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4300000000000002e-07 2024-05-08T08:54:36.149 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.149 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.149 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:36.149 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451471+0000", 2024-05-08T08:54:36.149 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0800000000000006e-06 2024-05-08T08:54:36.149 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.149 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.149 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:36.150 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451481+0000", 2024-05-08T08:54:36.150 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.38e-06 2024-05-08T08:54:36.150 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.150 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.150 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:36.150 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451487+0000", 2024-05-08T08:54:36.150 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4590000000000003e-06 2024-05-08T08:54:36.150 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.150 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.150 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:36.151 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541350+0000", 2024-05-08T08:54:36.151 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089862599000000001 2024-05-08T08:54:36.151 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.151 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.151 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:36.151 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541407+0000", 2024-05-08T08:54:36.151 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7667999999999997e-05 2024-05-08T08:54:36.151 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.151 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.151 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:36.151 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541499+0000", 2024-05-08T08:54:36.151 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1513000000000003e-05 2024-05-08T08:54:36.151 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:36.151 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:36.151 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:36.152 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.152 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.152 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:980 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 4128768~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:36.152 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451496+0000", 2024-05-08T08:54:36.152 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 86.620720520000006, 2024-05-08T08:54:36.152 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.090258985, 2024-05-08T08:54:36.152 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:36.152 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:36.152 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:36.152 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:36.152 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:36.152 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:36.152 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 980 2024-05-08T08:54:36.152 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.152 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:36.152 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.153 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:36.153 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451496+0000", 2024-05-08T08:54:36.153 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:36.153 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.153 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.153 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:36.153 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451496+0000", 2024-05-08T08:54:36.153 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:36.153 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.153 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.153 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:36.153 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451497+0000", 2024-05-08T08:54:36.153 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7599999999999997e-07 2024-05-08T08:54:36.153 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.153 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.153 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:36.153 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451500+0000", 2024-05-08T08:54:36.153 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7930000000000002e-06 2024-05-08T08:54:36.154 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.154 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.154 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:36.154 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451506+0000", 2024-05-08T08:54:36.154 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6980000000000001e-06 2024-05-08T08:54:36.154 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.154 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.154 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:36.154 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451512+0000", 2024-05-08T08:54:36.154 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2490000000000001e-06 2024-05-08T08:54:36.154 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.154 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.154 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:36.154 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541550+0000", 2024-05-08T08:54:36.154 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.090037961999999999 2024-05-08T08:54:36.155 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.155 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.155 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:36.155 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541647+0000", 2024-05-08T08:54:36.155 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6924999999999998e-05 2024-05-08T08:54:36.155 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:36.155 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:36.155 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:36.155 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541755+0000", 2024-05-08T08:54:36.155 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000108682 2024-05-08T08:54:36.155 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:36.155 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:36.155 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:36.155 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:36.155 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:36.156 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:36.157 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:54:36.157 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:36.159+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:36.258 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:36.261+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:36.262 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:36.262 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:54:36.262 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:54:36.262 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:36.271 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:54:36.359 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:36.362+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:36.375 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:36.375 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:54:36.375 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:54:36.375 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:54:36.375 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:36.384 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:54:36.460 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:36.463+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:36.490 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:36.491 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:54:36.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:54:36.491 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:54:36.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.491 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4621.0:9089 3.1f e33/18 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:head v 33'117, mlcod=31'115)", 2024-05-08T08:54:36.491 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:54:36.491 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 123.97279583300001, 2024-05-08T08:54:36.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.111244524, 2024-05-08T08:54:36.491 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:36.491 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:36.491 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:54:36.491 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:36.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:36.492 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:54:36.492 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:36.492 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.492 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.492 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:36.492 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:54:36.492 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:36.492 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.492 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.492 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:36.492 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:54:36.492 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5900000000000001e-07 2024-05-08T08:54:36.492 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.492 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.492 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:36.492 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539600+0000", 2024-05-08T08:54:36.493 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.019981364000000001 2024-05-08T08:54:36.493 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.493 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.493 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:36.493 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539624+0000", 2024-05-08T08:54:36.493 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4043e-05 2024-05-08T08:54:36.493 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.493 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.493 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:36.493 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539635+0000", 2024-05-08T08:54:36.493 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1511e-05 2024-05-08T08:54:36.493 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.493 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.493 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:36.494 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539728+0000", 2024-05-08T08:54:36.494 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2498999999999996e-05 2024-05-08T08:54:36.494 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.494 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.494 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:36.494 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539746+0000", 2024-05-08T08:54:36.494 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8289000000000001e-05 2024-05-08T08:54:36.494 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.494 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.494 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:54:36.494 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.630822+0000", 2024-05-08T08:54:36.494 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.091076283999999993 2024-05-08T08:54:36.494 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.494 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.494 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:36.494 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.630863+0000", 2024-05-08T08:54:36.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0374999999999999e-05 2024-05-08T08:54:36.495 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:36.495 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:36.495 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:36.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.495 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4195.0:4 3.11 3:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-05-08T08:54:36.495 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:49.764654+0000", 2024-05-08T08:54:36.495 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 106.727760292, 2024-05-08T08:54:36.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.30714275499999999, 2024-05-08T08:54:36.495 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:36.495 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:36.495 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-05-08T08:54:36.495 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:36.495 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4195", 2024-05-08T08:54:36.495 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/59179521", 2024-05-08T08:54:36.496 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4 2024-05-08T08:54:36.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.496 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:36.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.496 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:36.496 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764654+0000", 2024-05-08T08:54:36.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:36.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.496 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:36.496 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764654+0000", 2024-05-08T08:54:36.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:36.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.496 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:36.496 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764655+0000", 2024-05-08T08:54:36.497 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.452e-06 2024-05-08T08:54:36.497 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.497 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.497 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:36.497 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764661+0000", 2024-05-08T08:54:36.497 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5369999999999997e-06 2024-05-08T08:54:36.497 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.497 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.497 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:36.497 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764671+0000", 2024-05-08T08:54:36.497 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.028e-05 2024-05-08T08:54:36.497 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.497 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.497 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:36.497 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764680+0000", 2024-05-08T08:54:36.497 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0629999999999995e-06 2024-05-08T08:54:36.498 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.498 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.498 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:36.498 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764753+0000", 2024-05-08T08:54:36.498 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2679e-05 2024-05-08T08:54:36.498 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.498 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.498 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-05-08T08:54:36.498 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764765+0000", 2024-05-08T08:54:36.498 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2415000000000001e-05 2024-05-08T08:54:36.498 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.498 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.498 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:36.498 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:50.071692+0000", 2024-05-08T08:54:36.499 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.306927324 2024-05-08T08:54:36.499 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.499 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.499 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:36.499 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:50.071796+0000", 2024-05-08T08:54:36.499 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104005 2024-05-08T08:54:36.499 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:36.499 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:36.499 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:36.499 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.499 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.499 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4416.0:20531 3.16 3:6bb2ed16:::rbd_object_map.1140bc4a55c7:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-05-08T08:54:36.499 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:58.673786+0000", 2024-05-08T08:54:36.499 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.818627888000002, 2024-05-08T08:54:36.499 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.94684119899999997, 2024-05-08T08:54:36.500 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:36.500 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:36.500 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:54:36.500 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:36.500 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4416", 2024-05-08T08:54:36.500 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:54:36.500 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20531 2024-05-08T08:54:36.500 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.500 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:36.500 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.500 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:36.500 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673786+0000", 2024-05-08T08:54:36.500 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:36.500 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.500 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.500 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:36.500 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673786+0000", 2024-05-08T08:54:36.501 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:36.501 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.501 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.501 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:36.501 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673787+0000", 2024-05-08T08:54:36.501 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9800000000000003e-07 2024-05-08T08:54:36.501 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.501 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.501 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:36.501 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673791+0000", 2024-05-08T08:54:36.501 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.104e-06 2024-05-08T08:54:36.501 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.501 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.501 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:36.501 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673801+0000", 2024-05-08T08:54:36.502 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9960000000000005e-06 2024-05-08T08:54:36.502 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.502 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.502 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:36.502 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673806+0000", 2024-05-08T08:54:36.502 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2320000000000001e-06 2024-05-08T08:54:36.502 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.502 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.502 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:36.502 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673860+0000", 2024-05-08T08:54:36.502 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3468999999999999e-05 2024-05-08T08:54:36.502 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.502 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.502 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-05-08T08:54:36.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673866+0000", 2024-05-08T08:54:36.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3500000000000002e-06 2024-05-08T08:54:36.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.503 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:36.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.616700+0000", 2024-05-08T08:54:36.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.94283390700000003 2024-05-08T08:54:36.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.503 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:36.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.616941+0000", 2024-05-08T08:54:36.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00024080400000000001 2024-05-08T08:54:36.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.503 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-05-08T08:54:36.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.617072+0000", 2024-05-08T08:54:36.504 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000131688 2024-05-08T08:54:36.504 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.504 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.504 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:54:36.504 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.619571+0000", 2024-05-08T08:54:36.504 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0024990149999999998 2024-05-08T08:54:36.504 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.504 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.504 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:54:36.504 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.620568+0000", 2024-05-08T08:54:36.504 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00099711000000000005 2024-05-08T08:54:36.504 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.504 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.504 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:54:36.504 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.620602+0000", 2024-05-08T08:54:36.504 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.324e-05 2024-05-08T08:54:36.505 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.505 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.505 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:36.505 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.620627+0000", 2024-05-08T08:54:36.505 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5585999999999999e-05 2024-05-08T08:54:36.505 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:36.505 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:36.505 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:36.505 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.505 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.505 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5490 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4227072~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:36.505 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.881809+0000", 2024-05-08T08:54:36.505 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 10.610605376000001, 2024-05-08T08:54:36.505 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047460447000000003, 2024-05-08T08:54:36.505 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:36.505 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:36.506 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:36.506 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:36.506 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:36.506 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:36.506 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5490 2024-05-08T08:54:36.506 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.506 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:36.506 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.506 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:36.506 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881809+0000", 2024-05-08T08:54:36.506 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:36.506 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.506 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.506 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:36.506 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881809+0000", 2024-05-08T08:54:36.507 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:36.507 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.507 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.507 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:36.507 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881809+0000", 2024-05-08T08:54:36.507 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7899999999999999e-07 2024-05-08T08:54:36.507 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.507 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.507 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:36.507 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881811+0000", 2024-05-08T08:54:36.507 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0339999999999999e-06 2024-05-08T08:54:36.507 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.507 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.507 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:36.507 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881818+0000", 2024-05-08T08:54:36.508 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5760000000000002e-06 2024-05-08T08:54:36.508 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.508 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.508 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:36.508 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881823+0000", 2024-05-08T08:54:36.508 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4160000000000003e-06 2024-05-08T08:54:36.508 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.508 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.508 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:36.508 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929157+0000", 2024-05-08T08:54:36.508 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047334029999999999 2024-05-08T08:54:36.508 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.508 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.508 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:36.508 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929211+0000", 2024-05-08T08:54:36.509 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3560999999999998e-05 2024-05-08T08:54:36.509 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.509 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.509 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:36.509 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929269+0000", 2024-05-08T08:54:36.509 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8350999999999997e-05 2024-05-08T08:54:36.509 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:36.509 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:36.509 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:36.509 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.509 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.509 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5492 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4272128~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:36.509 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.881830+0000", 2024-05-08T08:54:36.509 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 10.610584082000001, 2024-05-08T08:54:36.509 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047614006, 2024-05-08T08:54:36.509 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:36.510 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:36.510 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:36.510 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:36.510 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:36.510 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:36.510 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5492 2024-05-08T08:54:36.510 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.510 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:36.510 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.510 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:36.510 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881830+0000", 2024-05-08T08:54:36.510 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:36.510 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.510 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.510 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:36.511 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881830+0000", 2024-05-08T08:54:36.511 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:36.511 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.511 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.511 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:36.511 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881830+0000", 2024-05-08T08:54:36.511 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4099999999999999e-07 2024-05-08T08:54:36.511 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.511 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.511 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:36.511 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881832+0000", 2024-05-08T08:54:36.511 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0370000000000001e-06 2024-05-08T08:54:36.511 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.511 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.511 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:36.511 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881839+0000", 2024-05-08T08:54:36.512 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3289999999999999e-06 2024-05-08T08:54:36.512 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.512 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.512 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:36.512 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881844+0000", 2024-05-08T08:54:36.512 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1170000000000003e-06 2024-05-08T08:54:36.512 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.512 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.512 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:36.512 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929296+0000", 2024-05-08T08:54:36.512 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047452418000000003 2024-05-08T08:54:36.512 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.512 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.512 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:36.512 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929360+0000", 2024-05-08T08:54:36.513 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4197999999999998e-05 2024-05-08T08:54:36.513 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.513 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.513 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:36.513 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929444+0000", 2024-05-08T08:54:36.513 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3466000000000004e-05 2024-05-08T08:54:36.513 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:36.513 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:36.513 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:36.513 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.513 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.513 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5494 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4317184~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:36.513 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.881850+0000", 2024-05-08T08:54:36.513 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 10.610564332999999, 2024-05-08T08:54:36.513 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047723887, 2024-05-08T08:54:36.514 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:36.514 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:36.514 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:36.514 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:36.514 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:36.514 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:36.514 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5494 2024-05-08T08:54:36.514 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.514 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:36.514 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.514 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:36.514 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881850+0000", 2024-05-08T08:54:36.514 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:36.514 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.514 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.515 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:36.515 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881850+0000", 2024-05-08T08:54:36.515 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:36.515 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.515 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.515 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:36.515 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881850+0000", 2024-05-08T08:54:36.515 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8500000000000002e-07 2024-05-08T08:54:36.515 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.515 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.515 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:36.515 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881856+0000", 2024-05-08T08:54:36.515 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9510000000000003e-06 2024-05-08T08:54:36.515 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.515 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.515 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:36.516 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881862+0000", 2024-05-08T08:54:36.516 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3049999999999998e-06 2024-05-08T08:54:36.516 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.516 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.516 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:36.516 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881868+0000", 2024-05-08T08:54:36.516 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8340000000000004e-06 2024-05-08T08:54:36.516 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.516 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.516 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:36.516 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929475+0000", 2024-05-08T08:54:36.516 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047606796 2024-05-08T08:54:36.516 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.516 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.517 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:36.517 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929518+0000", 2024-05-08T08:54:36.517 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3387000000000003e-05 2024-05-08T08:54:36.517 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.517 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.517 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:36.517 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929574+0000", 2024-05-08T08:54:36.517 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5129e-05 2024-05-08T08:54:36.517 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:36.517 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:36.517 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:36.517 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.517 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.517 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5495 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4362240~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:36.517 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.881875+0000", 2024-05-08T08:54:36.518 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 10.610539466000001, 2024-05-08T08:54:36.518 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047847967999999998, 2024-05-08T08:54:36.518 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:36.518 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:36.518 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:36.518 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:36.518 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:36.518 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:36.518 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5495 2024-05-08T08:54:36.518 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.518 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:36.518 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.518 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:36.518 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881875+0000", 2024-05-08T08:54:36.519 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:36.519 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.519 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.519 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:36.519 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881875+0000", 2024-05-08T08:54:36.519 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:36.519 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.519 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.519 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:36.519 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881875+0000", 2024-05-08T08:54:36.519 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-07 2024-05-08T08:54:36.519 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.519 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.519 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:36.519 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881877+0000", 2024-05-08T08:54:36.519 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9659999999999999e-06 2024-05-08T08:54:36.520 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.520 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.520 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:36.520 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881883+0000", 2024-05-08T08:54:36.520 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7649999999999999e-06 2024-05-08T08:54:36.520 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.520 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.520 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:36.520 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881887+0000", 2024-05-08T08:54:36.520 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6689999999999997e-06 2024-05-08T08:54:36.520 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.520 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.520 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:36.520 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929600+0000", 2024-05-08T08:54:36.520 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047712746 2024-05-08T08:54:36.520 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.520 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.521 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:36.521 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929659+0000", 2024-05-08T08:54:36.521 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8936e-05 2024-05-08T08:54:36.521 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.521 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.521 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:36.521 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929723+0000", 2024-05-08T08:54:36.521 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3524999999999999e-05 2024-05-08T08:54:36.521 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:36.521 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:36.521 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:36.521 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.521 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.521 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5496 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4407296~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:36.521 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.881894+0000", 2024-05-08T08:54:36.521 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 10.610519602, 2024-05-08T08:54:36.522 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047998015999999998, 2024-05-08T08:54:36.522 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:36.522 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:36.522 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:36.522 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:36.522 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:36.522 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:36.522 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5496 2024-05-08T08:54:36.522 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.522 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:36.522 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.522 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:36.522 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881894+0000", 2024-05-08T08:54:36.522 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:36.522 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.522 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.523 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:36.523 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881894+0000", 2024-05-08T08:54:36.523 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:36.523 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.523 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.523 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:36.523 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881895+0000", 2024-05-08T08:54:36.523 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6699999999999999e-07 2024-05-08T08:54:36.523 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.523 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.523 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:36.523 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881897+0000", 2024-05-08T08:54:36.523 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0229999999999999e-06 2024-05-08T08:54:36.523 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.523 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.523 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:36.524 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881903+0000", 2024-05-08T08:54:36.524 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9850000000000003e-06 2024-05-08T08:54:36.524 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.524 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.524 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:36.524 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881908+0000", 2024-05-08T08:54:36.524 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9019999999999998e-06 2024-05-08T08:54:36.524 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.524 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.524 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:36.524 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929751+0000", 2024-05-08T08:54:36.524 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047843550999999998 2024-05-08T08:54:36.524 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.524 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.524 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:36.525 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929823+0000", 2024-05-08T08:54:36.525 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1172000000000002e-05 2024-05-08T08:54:36.525 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.525 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.525 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:36.525 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929892+0000", 2024-05-08T08:54:36.525 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9915999999999999e-05 2024-05-08T08:54:36.525 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:36.525 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:36.525 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:36.525 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.525 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.525 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5498 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4452352~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:36.525 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.881914+0000", 2024-05-08T08:54:36.525 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 10.610499844, 2024-05-08T08:54:36.525 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048203553000000003, 2024-05-08T08:54:36.525 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:36.526 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:36.526 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:36.526 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:36.526 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:36.526 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:36.526 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5498 2024-05-08T08:54:36.526 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.526 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:36.526 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.526 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:36.526 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881914+0000", 2024-05-08T08:54:36.526 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:36.526 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.526 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.526 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:36.526 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881914+0000", 2024-05-08T08:54:36.526 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:36.527 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.527 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.527 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:36.527 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881915+0000", 2024-05-08T08:54:36.527 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8500000000000002e-07 2024-05-08T08:54:36.527 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.527 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.527 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:36.527 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881917+0000", 2024-05-08T08:54:36.527 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.181e-06 2024-05-08T08:54:36.527 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.527 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.527 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:36.527 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881924+0000", 2024-05-08T08:54:36.527 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9619999999999996e-06 2024-05-08T08:54:36.527 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.527 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.527 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:36.528 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881929+0000", 2024-05-08T08:54:36.528 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1399999999999999e-06 2024-05-08T08:54:36.528 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.528 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.528 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:36.528 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929952+0000", 2024-05-08T08:54:36.528 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048023134000000002 2024-05-08T08:54:36.528 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.528 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.528 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:36.528 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.930029+0000", 2024-05-08T08:54:36.528 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7490000000000005e-05 2024-05-08T08:54:36.528 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.528 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.528 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:36.529 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.930118+0000", 2024-05-08T08:54:36.529 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8261000000000001e-05 2024-05-08T08:54:36.529 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:36.529 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:36.529 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:36.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.529 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.529 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5563 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4321280~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:36.529 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.884201+0000", 2024-05-08T08:54:36.529 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 10.608212593999999, 2024-05-08T08:54:36.529 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047379969000000001, 2024-05-08T08:54:36.529 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:36.529 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:36.529 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:36.529 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:36.530 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:36.530 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:36.530 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5563 2024-05-08T08:54:36.530 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.530 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:36.530 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.530 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:36.530 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884201+0000", 2024-05-08T08:54:36.530 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:36.530 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.530 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.530 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:36.530 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884201+0000", 2024-05-08T08:54:36.530 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:36.530 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.531 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.531 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:36.531 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884202+0000", 2024-05-08T08:54:36.531 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8000000000000002e-07 2024-05-08T08:54:36.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.531 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.531 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:36.531 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884208+0000", 2024-05-08T08:54:36.531 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.674e-06 2024-05-08T08:54:36.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.531 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.531 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:36.531 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884215+0000", 2024-05-08T08:54:36.531 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0389999999999999e-06 2024-05-08T08:54:36.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.531 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.531 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:36.532 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884220+0000", 2024-05-08T08:54:36.532 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4950000000000002e-06 2024-05-08T08:54:36.532 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.532 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.532 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:36.532 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931417+0000", 2024-05-08T08:54:36.532 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047196674000000001 2024-05-08T08:54:36.532 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.532 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.532 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:36.532 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931493+0000", 2024-05-08T08:54:36.532 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5950000000000003e-05 2024-05-08T08:54:36.532 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.532 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.532 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:36.533 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931581+0000", 2024-05-08T08:54:36.533 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8857000000000007e-05 2024-05-08T08:54:36.533 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:36.533 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:36.533 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:36.533 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.533 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.533 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5564 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4366336~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:36.533 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.884225+0000", 2024-05-08T08:54:36.533 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 10.608189054, 2024-05-08T08:54:36.533 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047555989999999999, 2024-05-08T08:54:36.533 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:36.533 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:36.533 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:36.533 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:36.534 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:36.534 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:36.534 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5564 2024-05-08T08:54:36.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.534 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:36.534 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.534 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:36.534 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884225+0000", 2024-05-08T08:54:36.534 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:36.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.534 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.534 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:36.534 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884225+0000", 2024-05-08T08:54:36.534 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:36.535 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.535 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.535 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:36.535 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884225+0000", 2024-05-08T08:54:36.535 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2499999999999995e-07 2024-05-08T08:54:36.535 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.535 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.535 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:36.535 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884227+0000", 2024-05-08T08:54:36.535 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5829999999999999e-06 2024-05-08T08:54:36.535 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.535 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.535 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:36.535 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884232+0000", 2024-05-08T08:54:36.535 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1159999999999998e-06 2024-05-08T08:54:36.536 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.536 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.536 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:36.536 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884236+0000", 2024-05-08T08:54:36.536 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2490000000000003e-06 2024-05-08T08:54:36.536 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.536 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.536 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:36.536 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931617+0000", 2024-05-08T08:54:36.536 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047380310000000002 2024-05-08T08:54:36.536 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.536 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.536 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:36.536 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931689+0000", 2024-05-08T08:54:36.536 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2358000000000001e-05 2024-05-08T08:54:36.536 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.537 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.537 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:36.537 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931781+0000", 2024-05-08T08:54:36.537 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1848999999999994e-05 2024-05-08T08:54:36.537 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:36.537 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:36.537 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:36.537 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.537 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.537 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5565 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4411392~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:36.537 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.884251+0000", 2024-05-08T08:54:36.537 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 10.608162901, 2024-05-08T08:54:36.537 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047743671000000001, 2024-05-08T08:54:36.537 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:36.537 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:36.537 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:36.538 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:36.538 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:36.538 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:36.538 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5565 2024-05-08T08:54:36.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.538 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:36.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.538 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:36.538 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884251+0000", 2024-05-08T08:54:36.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:36.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.538 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:36.538 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884251+0000", 2024-05-08T08:54:36.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:36.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.539 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.539 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:36.539 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884252+0000", 2024-05-08T08:54:36.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3200000000000005e-07 2024-05-08T08:54:36.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.539 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.539 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:36.539 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884253+0000", 2024-05-08T08:54:36.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6980000000000001e-06 2024-05-08T08:54:36.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.539 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.539 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:36.539 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884259+0000", 2024-05-08T08:54:36.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3759999999999999e-06 2024-05-08T08:54:36.540 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.540 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.540 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:36.540 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884263+0000", 2024-05-08T08:54:36.540 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7280000000000003e-06 2024-05-08T08:54:36.540 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.540 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.540 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:36.540 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931818+0000", 2024-05-08T08:54:36.540 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047554368999999999 2024-05-08T08:54:36.540 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.540 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.540 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:36.540 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931897+0000", 2024-05-08T08:54:36.540 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8821999999999999e-05 2024-05-08T08:54:36.541 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.541 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.541 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:36.541 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931995+0000", 2024-05-08T08:54:36.541 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8146000000000005e-05 2024-05-08T08:54:36.541 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:36.541 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:36.541 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:36.541 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.541 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.541 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5600 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 3948544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:36.541 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.884619+0000", 2024-05-08T08:54:36.541 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 10.607794646, 2024-05-08T08:54:36.541 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047574067999999997, 2024-05-08T08:54:36.542 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:36.542 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:36.542 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:36.542 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:36.542 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:36.542 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:36.542 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5600 2024-05-08T08:54:36.542 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.542 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:36.542 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.542 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:36.542 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884619+0000", 2024-05-08T08:54:36.542 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:36.542 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.543 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:36.543 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884619+0000", 2024-05-08T08:54:36.543 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:36.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.543 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:36.543 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884620+0000", 2024-05-08T08:54:36.543 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.53e-07 2024-05-08T08:54:36.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.543 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:36.543 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884622+0000", 2024-05-08T08:54:36.543 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.796e-06 2024-05-08T08:54:36.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.544 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.544 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:36.544 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884627+0000", 2024-05-08T08:54:36.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2850000000000002e-06 2024-05-08T08:54:36.544 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.544 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.544 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:36.544 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884631+0000", 2024-05-08T08:54:36.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2899999999999996e-06 2024-05-08T08:54:36.544 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.544 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.544 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:36.544 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932024+0000", 2024-05-08T08:54:36.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047392680999999999 2024-05-08T08:54:36.544 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.545 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.545 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:36.545 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932102+0000", 2024-05-08T08:54:36.545 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7806999999999996e-05 2024-05-08T08:54:36.545 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.545 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.545 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:36.545 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932193+0000", 2024-05-08T08:54:36.545 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1855999999999996e-05 2024-05-08T08:54:36.545 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:36.545 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:36.545 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:36.545 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.545 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.546 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5603 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 3993600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:36.546 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.884680+0000", 2024-05-08T08:54:36.546 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 10.607734037, 2024-05-08T08:54:36.546 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047745034999999998, 2024-05-08T08:54:36.546 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:36.546 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:36.546 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:36.546 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:36.546 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:36.546 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:36.546 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5603 2024-05-08T08:54:36.546 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.546 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:36.546 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.546 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:36.546 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884680+0000", 2024-05-08T08:54:36.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:36.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.547 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:36.547 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884680+0000", 2024-05-08T08:54:36.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:36.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.547 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:36.547 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884680+0000", 2024-05-08T08:54:36.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4400000000000001e-07 2024-05-08T08:54:36.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.547 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:36.547 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884682+0000", 2024-05-08T08:54:36.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.514e-06 2024-05-08T08:54:36.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:36.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884687+0000", 2024-05-08T08:54:36.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0359999999999997e-06 2024-05-08T08:54:36.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:36.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884692+0000", 2024-05-08T08:54:36.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5990000000000004e-06 2024-05-08T08:54:36.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:36.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932235+0000", 2024-05-08T08:54:36.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047543635000000001 2024-05-08T08:54:36.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.549 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.549 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:36.549 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932323+0000", 2024-05-08T08:54:36.549 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7311000000000005e-05 2024-05-08T08:54:36.549 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.549 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.549 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:36.549 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932425+0000", 2024-05-08T08:54:36.549 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102496 2024-05-08T08:54:36.549 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:36.549 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:36.549 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:36.549 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.549 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.549 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5606 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4038656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:36.549 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.884729+0000", 2024-05-08T08:54:36.550 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 10.607684749000001, 2024-05-08T08:54:36.550 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047916488, 2024-05-08T08:54:36.550 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:36.550 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:36.550 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:36.550 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:36.550 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:36.550 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:36.550 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5606 2024-05-08T08:54:36.550 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.550 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:36.550 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.550 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:36.550 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884729+0000", 2024-05-08T08:54:36.550 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:36.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.551 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:36.551 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884729+0000", 2024-05-08T08:54:36.551 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:36.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.551 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:36.551 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.885940+0000", 2024-05-08T08:54:36.551 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001210509 2024-05-08T08:54:36.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.551 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:36.551 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.885945+0000", 2024-05-08T08:54:36.551 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7389999999999999e-06 2024-05-08T08:54:36.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.552 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:36.552 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.885955+0000", 2024-05-08T08:54:36.552 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0813e-05 2024-05-08T08:54:36.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.552 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.552 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:36.552 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.885962+0000", 2024-05-08T08:54:36.552 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1160000000000001e-06 2024-05-08T08:54:36.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.552 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.552 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:36.552 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932465+0000", 2024-05-08T08:54:36.552 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046502612999999998 2024-05-08T08:54:36.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.552 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.552 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:36.552 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932547+0000", 2024-05-08T08:54:36.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2025000000000002e-05 2024-05-08T08:54:36.553 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.553 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.553 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:36.553 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932646+0000", 2024-05-08T08:54:36.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8672999999999995e-05 2024-05-08T08:54:36.553 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:36.553 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:36.553 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:36.553 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.553 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.553 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5635 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4235264~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:36.553 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.886082+0000", 2024-05-08T08:54:36.553 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 10.60633163, 2024-05-08T08:54:36.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047438284999999997, 2024-05-08T08:54:36.553 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:36.554 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:36.554 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:36.554 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:36.554 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:36.554 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:36.554 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5635 2024-05-08T08:54:36.554 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.554 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:36.554 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.554 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:36.554 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886082+0000", 2024-05-08T08:54:36.554 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:36.554 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.554 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.554 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:36.555 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886082+0000", 2024-05-08T08:54:36.555 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:36.555 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.555 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.555 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:36.555 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886083+0000", 2024-05-08T08:54:36.555 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8999999999999996e-07 2024-05-08T08:54:36.555 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.555 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.555 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:36.555 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886086+0000", 2024-05-08T08:54:36.555 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7e-06 2024-05-08T08:54:36.555 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.555 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.555 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:36.555 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886093+0000", 2024-05-08T08:54:36.555 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5320000000000001e-06 2024-05-08T08:54:36.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.556 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.556 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:36.556 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886102+0000", 2024-05-08T08:54:36.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8070000000000006e-06 2024-05-08T08:54:36.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.556 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.556 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:36.556 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933357+0000", 2024-05-08T08:54:36.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047255093999999997 2024-05-08T08:54:36.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.556 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.556 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:36.556 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933433+0000", 2024-05-08T08:54:36.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5796000000000004e-05 2024-05-08T08:54:36.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.556 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.557 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:36.557 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933521+0000", 2024-05-08T08:54:36.557 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7766e-05 2024-05-08T08:54:36.557 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:36.557 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:36.557 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:36.557 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.557 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.557 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5636 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4280320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:36.557 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.886111+0000", 2024-05-08T08:54:36.557 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 10.606303012, 2024-05-08T08:54:36.557 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047663644999999998, 2024-05-08T08:54:36.557 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:36.557 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:36.557 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:36.557 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:36.557 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:36.558 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:36.558 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5636 2024-05-08T08:54:36.558 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.558 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:36.558 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.558 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:36.558 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886111+0000", 2024-05-08T08:54:36.558 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:36.558 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.558 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.558 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:36.558 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886111+0000", 2024-05-08T08:54:36.558 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:36.558 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.558 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.558 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:36.558 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886111+0000", 2024-05-08T08:54:36.559 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5200000000000002e-07 2024-05-08T08:54:36.559 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.559 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.559 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:36.559 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886114+0000", 2024-05-08T08:54:36.559 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1710000000000001e-06 2024-05-08T08:54:36.559 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.559 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.559 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:36.559 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886120+0000", 2024-05-08T08:54:36.559 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.404e-06 2024-05-08T08:54:36.559 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.559 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.559 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:36.559 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886125+0000", 2024-05-08T08:54:36.559 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2730000000000002e-06 2024-05-08T08:54:36.559 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.560 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.560 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:36.560 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933570+0000", 2024-05-08T08:54:36.560 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047444989999999999 2024-05-08T08:54:36.560 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.560 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.560 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:36.560 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933659+0000", 2024-05-08T08:54:36.560 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8912999999999994e-05 2024-05-08T08:54:36.560 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.560 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.560 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:36.560 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933775+0000", 2024-05-08T08:54:36.560 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000115442 2024-05-08T08:54:36.560 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:36.561 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:36.561 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:36.561 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.561 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.561 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5637 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4325376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:36.561 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.886132+0000", 2024-05-08T08:54:36.561 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 10.606281974, 2024-05-08T08:54:36.561 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047852219000000001, 2024-05-08T08:54:36.561 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:36.561 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:36.561 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:36.561 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:36.561 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:36.561 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:36.561 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5637 2024-05-08T08:54:36.561 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.561 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:36.562 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.562 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:36.562 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886132+0000", 2024-05-08T08:54:36.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:36.562 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.562 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.562 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:36.562 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886132+0000", 2024-05-08T08:54:36.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:36.562 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.562 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.562 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:36.562 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886132+0000", 2024-05-08T08:54:36.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7300000000000002e-07 2024-05-08T08:54:36.563 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.563 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.563 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:36.563 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886139+0000", 2024-05-08T08:54:36.563 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6109999999999999e-06 2024-05-08T08:54:36.563 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.563 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.563 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:36.563 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886146+0000", 2024-05-08T08:54:36.563 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6279999999999999e-06 2024-05-08T08:54:36.563 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.563 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.563 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:36.563 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886151+0000", 2024-05-08T08:54:36.563 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1769999999999997e-06 2024-05-08T08:54:36.563 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.563 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.564 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:36.564 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933812+0000", 2024-05-08T08:54:36.564 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047660912999999999 2024-05-08T08:54:36.564 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.564 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.564 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:36.564 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933888+0000", 2024-05-08T08:54:36.564 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5865000000000006e-05 2024-05-08T08:54:36.564 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.564 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.564 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:36.564 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933984+0000", 2024-05-08T08:54:36.564 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6651999999999996e-05 2024-05-08T08:54:36.564 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:36.564 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:36.564 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:36.565 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.565 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.565 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5638 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4370432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:36.565 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.886157+0000", 2024-05-08T08:54:36.565 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 10.60625671, 2024-05-08T08:54:36.565 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048077597, 2024-05-08T08:54:36.565 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:36.565 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:36.565 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:36.565 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:36.565 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:36.565 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:36.565 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5638 2024-05-08T08:54:36.565 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.566 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:36.566 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.566 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:36.566 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886157+0000", 2024-05-08T08:54:36.566 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:36.566 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.566 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.566 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:36.566 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886157+0000", 2024-05-08T08:54:36.566 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:36.566 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.566 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.566 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:36.566 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886158+0000", 2024-05-08T08:54:36.566 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1100000000000001e-07 2024-05-08T08:54:36.566 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.567 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.567 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:36.567 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886160+0000", 2024-05-08T08:54:36.567 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9800000000000001e-06 2024-05-08T08:54:36.567 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.567 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.567 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:36.567 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886166+0000", 2024-05-08T08:54:36.567 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.314e-06 2024-05-08T08:54:36.567 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.567 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.567 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:36.567 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886171+0000", 2024-05-08T08:54:36.567 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7700000000000001e-06 2024-05-08T08:54:36.567 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.567 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.567 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:36.568 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.934025+0000", 2024-05-08T08:54:36.568 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047854509000000003 2024-05-08T08:54:36.568 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.568 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.568 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:36.568 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.934121+0000", 2024-05-08T08:54:36.568 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5688999999999998e-05 2024-05-08T08:54:36.568 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.568 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.568 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:36.568 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.934235+0000", 2024-05-08T08:54:36.568 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000113924 2024-05-08T08:54:36.568 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:36.568 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:36.568 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:36.568 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.569 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.569 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5639 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4415488~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:36.569 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.886177+0000", 2024-05-08T08:54:36.569 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 10.60623696, 2024-05-08T08:54:36.569 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048249541, 2024-05-08T08:54:36.569 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:36.569 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:36.569 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:36.569 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:36.569 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:36.569 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:36.569 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5639 2024-05-08T08:54:36.569 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.569 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:36.569 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.569 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:36.569 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886177+0000", 2024-05-08T08:54:36.570 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:36.570 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.570 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.570 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:36.570 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886177+0000", 2024-05-08T08:54:36.570 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:36.570 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.570 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.570 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:36.570 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886177+0000", 2024-05-08T08:54:36.570 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.65e-07 2024-05-08T08:54:36.570 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.570 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.570 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:36.570 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886179+0000", 2024-05-08T08:54:36.571 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9530000000000002e-06 2024-05-08T08:54:36.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.571 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.571 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:36.571 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886186+0000", 2024-05-08T08:54:36.571 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3419999999999996e-06 2024-05-08T08:54:36.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.571 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.571 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:36.571 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886192+0000", 2024-05-08T08:54:36.571 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0560000000000001e-06 2024-05-08T08:54:36.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.571 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.571 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:36.571 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.934258+0000", 2024-05-08T08:54:36.571 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048066367999999998 2024-05-08T08:54:36.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.572 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.572 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:36.572 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.934338+0000", 2024-05-08T08:54:36.572 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9616000000000005e-05 2024-05-08T08:54:36.572 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.572 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.572 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:36.572 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.934427+0000", 2024-05-08T08:54:36.572 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8840999999999999e-05 2024-05-08T08:54:36.572 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:36.572 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:36.572 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:36.572 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:36.572 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:36.572 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:36.574 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:54:36.574 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:36.576+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:36.674 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:36.677+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:36.719 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:36.720 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:54:36.720 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:54:36.720 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:36.728 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:54:36.775 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:36.778+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:36.822 INFO:teuthology.orchestra.run.smithi059.stderr:noscrub is set 2024-05-08T08:54:36.822 INFO:teuthology.orchestra.run.smithi059.stderr:reweighted osd.2 to 1 (10000) 2024-05-08T08:54:36.837 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:36.837 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:54:36.837 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:54:36.837 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:54:36.837 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:36.843 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-05-08T08:54:36.847 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:54:36.879 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T08:54:36.891 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:36.894+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:36.965 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:36.965 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:54:36.965 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:54:36.965 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:54:36.965 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.965 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22073 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 782336~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:36.965 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:54:36.965 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 129.10914104299999, 2024-05-08T08:54:36.965 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047244234000000003, 2024-05-08T08:54:36.965 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:36.965 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:36.965 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:36.965 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:36.966 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:36.966 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:36.966 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22073 2024-05-08T08:54:36.966 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.966 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:36.966 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.966 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:36.966 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:54:36.966 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:36.966 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.966 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.966 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:36.966 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:54:36.966 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:36.967 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.967 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.967 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:36.967 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857668+0000", 2024-05-08T08:54:36.967 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7800000000000002e-07 2024-05-08T08:54:36.967 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.967 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.967 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:36.967 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857670+0000", 2024-05-08T08:54:36.967 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0820000000000001e-06 2024-05-08T08:54:36.967 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.967 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.967 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:36.967 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857678+0000", 2024-05-08T08:54:36.967 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7589999999999997e-06 2024-05-08T08:54:36.967 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.968 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.968 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:36.968 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857691+0000", 2024-05-08T08:54:36.968 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3672999999999999e-05 2024-05-08T08:54:36.968 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.968 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.968 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:36.968 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904780+0000", 2024-05-08T08:54:36.968 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047088616999999999 2024-05-08T08:54:36.968 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.968 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.968 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:36.968 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904840+0000", 2024-05-08T08:54:36.968 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0356999999999998e-05 2024-05-08T08:54:36.968 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.969 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.969 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:36.969 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904912+0000", 2024-05-08T08:54:36.969 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1367999999999998e-05 2024-05-08T08:54:36.969 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:36.969 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:36.969 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:36.969 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.969 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.969 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22075 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 798720~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:36.969 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:54:36.969 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 129.109110646, 2024-05-08T08:54:36.969 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047384643999999997, 2024-05-08T08:54:36.969 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:36.969 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:36.969 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:36.969 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:36.970 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:36.970 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:36.970 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22075 2024-05-08T08:54:36.970 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.970 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:36.970 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.970 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:36.970 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:54:36.970 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:36.970 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.970 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.970 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:36.970 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:54:36.970 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:36.970 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.970 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.971 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:36.971 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:54:36.971 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7399999999999999e-07 2024-05-08T08:54:36.971 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.971 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.971 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:36.971 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857705+0000", 2024-05-08T08:54:36.971 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1980000000000004e-06 2024-05-08T08:54:36.971 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.971 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.971 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:36.971 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857713+0000", 2024-05-08T08:54:36.971 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7039999999999994e-06 2024-05-08T08:54:36.971 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.971 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.971 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:36.972 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857718+0000", 2024-05-08T08:54:36.972 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.865e-06 2024-05-08T08:54:36.972 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.972 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.972 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:36.972 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904941+0000", 2024-05-08T08:54:36.972 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047223198000000001 2024-05-08T08:54:36.972 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.972 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.972 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:36.972 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904999+0000", 2024-05-08T08:54:36.972 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8254999999999997e-05 2024-05-08T08:54:36.972 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.972 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.972 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:36.973 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905082+0000", 2024-05-08T08:54:36.973 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3049999999999999e-05 2024-05-08T08:54:36.973 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:36.973 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:36.973 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:36.973 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.973 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.973 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22077 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 815104~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:36.973 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:54:36.973 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 129.10907263499999, 2024-05-08T08:54:36.973 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047519448999999998, 2024-05-08T08:54:36.973 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:36.973 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:36.973 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:36.973 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:36.973 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:36.973 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:36.974 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22077 2024-05-08T08:54:36.974 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.974 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:36.974 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.974 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:36.974 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:54:36.974 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:36.974 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.974 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.974 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:36.974 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:54:36.974 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:36.974 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.974 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.974 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:36.974 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:54:36.975 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.99e-07 2024-05-08T08:54:36.975 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.975 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.975 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:36.975 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857739+0000", 2024-05-08T08:54:36.975 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3599999999999999e-06 2024-05-08T08:54:36.975 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.975 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.975 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:36.975 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857745+0000", 2024-05-08T08:54:36.975 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5139999999999998e-06 2024-05-08T08:54:36.975 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.975 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.975 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:36.975 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857750+0000", 2024-05-08T08:54:36.975 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8550000000000001e-06 2024-05-08T08:54:36.975 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.976 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.976 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:36.976 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905110+0000", 2024-05-08T08:54:36.976 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047360126000000002 2024-05-08T08:54:36.976 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.976 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.976 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:36.976 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905175+0000", 2024-05-08T08:54:36.976 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4689000000000004e-05 2024-05-08T08:54:36.976 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.976 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.976 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:36.976 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905255+0000", 2024-05-08T08:54:36.976 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0306e-05 2024-05-08T08:54:36.976 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:36.976 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:36.976 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:36.977 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.977 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.977 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22079 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 831488~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:36.977 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:54:36.977 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 129.10905208700001, 2024-05-08T08:54:36.977 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.04767499, 2024-05-08T08:54:36.977 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:36.977 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:36.977 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:36.977 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:36.977 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:36.977 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:36.977 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22079 2024-05-08T08:54:36.977 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.977 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:36.978 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.978 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:36.978 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:54:36.978 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:36.978 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.978 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.978 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:36.978 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:54:36.978 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:36.978 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.978 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.978 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:36.978 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857757+0000", 2024-05-08T08:54:36.978 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8299999999999997e-07 2024-05-08T08:54:36.978 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.978 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.979 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:36.979 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857759+0000", 2024-05-08T08:54:36.979 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1390000000000002e-06 2024-05-08T08:54:36.979 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.979 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.979 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:36.979 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857765+0000", 2024-05-08T08:54:36.979 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4609999999999996e-06 2024-05-08T08:54:36.979 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.979 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.979 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:36.979 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857770+0000", 2024-05-08T08:54:36.979 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6689999999999997e-06 2024-05-08T08:54:36.979 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.979 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.979 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:36.980 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905281+0000", 2024-05-08T08:54:36.980 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047510668999999998 2024-05-08T08:54:36.980 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.980 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.980 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:36.980 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905344+0000", 2024-05-08T08:54:36.980 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2797000000000002e-05 2024-05-08T08:54:36.980 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.980 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.980 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:36.980 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905431+0000", 2024-05-08T08:54:36.980 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7771999999999999e-05 2024-05-08T08:54:36.980 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:36.980 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:36.980 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:36.980 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.980 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.981 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22082 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 847872~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:36.981 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:54:36.981 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 129.109032396, 2024-05-08T08:54:36.981 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047831934999999999, 2024-05-08T08:54:36.981 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:36.981 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:36.981 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:36.981 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:36.981 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:36.981 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:36.981 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22082 2024-05-08T08:54:36.981 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.981 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:36.981 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.981 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:36.981 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:54:36.981 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:36.982 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.982 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.982 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:36.982 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:54:36.982 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:36.982 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.982 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.982 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:36.982 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:54:36.982 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:54:36.982 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.982 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.982 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:36.982 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857778+0000", 2024-05-08T08:54:36.982 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0789999999999999e-06 2024-05-08T08:54:36.982 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.983 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.983 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:36.983 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857784+0000", 2024-05-08T08:54:36.983 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0480000000000004e-06 2024-05-08T08:54:36.983 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.983 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.983 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:36.983 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857790+0000", 2024-05-08T08:54:36.983 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2719999999999997e-06 2024-05-08T08:54:36.983 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.983 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.983 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:36.983 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905458+0000", 2024-05-08T08:54:36.983 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047668470999999997 2024-05-08T08:54:36.983 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.984 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.984 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:36.984 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905521+0000", 2024-05-08T08:54:36.984 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2952000000000003e-05 2024-05-08T08:54:36.984 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.984 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.984 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:36.984 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905608+0000", 2024-05-08T08:54:36.984 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6759000000000001e-05 2024-05-08T08:54:36.984 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:36.984 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:36.984 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:36.984 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.985 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.985 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22084 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 864256~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:36.985 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:54:36.985 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 129.10900844599999, 2024-05-08T08:54:36.985 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047977748000000001, 2024-05-08T08:54:36.985 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:36.985 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:36.985 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:36.985 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:36.985 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:36.985 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:36.985 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22084 2024-05-08T08:54:36.986 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.986 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:36.986 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.986 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:36.986 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:54:36.986 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:36.986 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.986 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.986 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:36.986 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:54:36.986 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:36.986 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.986 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.986 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:36.986 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:54:36.987 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:54:36.987 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.987 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.987 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:36.987 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857802+0000", 2024-05-08T08:54:36.987 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0779999999999998e-06 2024-05-08T08:54:36.987 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.987 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.987 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:36.987 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857812+0000", 2024-05-08T08:54:36.987 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7939999999999997e-06 2024-05-08T08:54:36.987 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.987 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.987 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:36.987 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857817+0000", 2024-05-08T08:54:36.987 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8899999999999998e-06 2024-05-08T08:54:36.988 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.988 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.988 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:36.988 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905636+0000", 2024-05-08T08:54:36.988 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047819252 2024-05-08T08:54:36.988 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.988 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.988 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:36.988 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905700+0000", 2024-05-08T08:54:36.988 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4016999999999994e-05 2024-05-08T08:54:36.988 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.988 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.988 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:36.988 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905778+0000", 2024-05-08T08:54:36.988 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7306999999999997e-05 2024-05-08T08:54:36.988 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:36.989 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:36.989 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:36.989 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.989 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.989 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22086 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 880640~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:36.989 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:54:36.989 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 129.108985096, 2024-05-08T08:54:36.989 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048119149, 2024-05-08T08:54:36.989 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:36.989 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:36.989 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:36.989 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:36.989 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:36.989 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:36.989 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22086 2024-05-08T08:54:36.989 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.989 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:36.990 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.990 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:36.990 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:54:36.990 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:36.990 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.990 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.990 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:36.990 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:54:36.990 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:36.990 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.990 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.990 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:36.990 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857824+0000", 2024-05-08T08:54:36.990 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4200000000000002e-07 2024-05-08T08:54:36.990 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.990 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.991 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:36.991 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857826+0000", 2024-05-08T08:54:36.991 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0590000000000001e-06 2024-05-08T08:54:36.991 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.991 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.991 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:36.991 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857832+0000", 2024-05-08T08:54:36.991 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8029999999999999e-06 2024-05-08T08:54:36.991 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.991 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.991 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:36.991 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857837+0000", 2024-05-08T08:54:36.991 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7929999999999997e-06 2024-05-08T08:54:36.991 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.991 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.991 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:36.991 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905803+0000", 2024-05-08T08:54:36.992 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047965761000000003 2024-05-08T08:54:36.992 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.992 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.992 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:36.992 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905862+0000", 2024-05-08T08:54:36.992 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9203000000000002e-05 2024-05-08T08:54:36.992 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.992 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.992 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:36.992 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905942+0000", 2024-05-08T08:54:36.992 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0187999999999999e-05 2024-05-08T08:54:36.992 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:36.992 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:36.992 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:36.992 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.992 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.992 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22088 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 897024~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:36.993 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:54:36.993 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 129.108965044, 2024-05-08T08:54:36.993 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048256049000000002, 2024-05-08T08:54:36.993 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:36.993 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:36.993 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:36.993 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:36.993 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:36.993 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:36.993 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22088 2024-05-08T08:54:36.993 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.993 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:36.993 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.993 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:36.993 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:54:36.993 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:36.993 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.993 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.994 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:36.994 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:54:36.994 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:36.994 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.994 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.994 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:36.994 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857844+0000", 2024-05-08T08:54:36.994 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.46e-07 2024-05-08T08:54:36.994 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.994 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.994 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:36.994 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857846+0000", 2024-05-08T08:54:36.994 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0899999999999999e-06 2024-05-08T08:54:36.994 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.994 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.994 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:36.995 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857855+0000", 2024-05-08T08:54:36.995 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8219999999999996e-06 2024-05-08T08:54:36.995 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.995 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.995 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:36.995 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857860+0000", 2024-05-08T08:54:36.995 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2989999999999996e-06 2024-05-08T08:54:36.995 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.995 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.995 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:36.995 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905966+0000", 2024-05-08T08:54:36.995 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048106542000000002 2024-05-08T08:54:36.995 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.995 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.995 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:36.995 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906024+0000", 2024-05-08T08:54:36.996 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7958000000000003e-05 2024-05-08T08:54:36.996 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.996 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.996 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:36.996 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906099+0000", 2024-05-08T08:54:36.996 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4992000000000003e-05 2024-05-08T08:54:36.996 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:36.996 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:36.996 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:36.996 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.996 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.996 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22091 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 913408~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:36.996 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:54:36.996 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 129.108941284, 2024-05-08T08:54:36.996 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048395290000000001, 2024-05-08T08:54:36.996 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:36.996 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:36.997 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:36.997 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:36.997 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:36.997 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:36.997 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22091 2024-05-08T08:54:36.997 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.997 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:36.997 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.997 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:36.997 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:54:36.997 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:36.997 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.997 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.997 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:36.997 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:54:36.997 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:36.997 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.998 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.998 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:36.998 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:54:36.998 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4900000000000001e-07 2024-05-08T08:54:36.998 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.998 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.998 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:36.998 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857870+0000", 2024-05-08T08:54:36.998 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2510000000000001e-06 2024-05-08T08:54:36.998 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.998 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.998 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:36.998 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857877+0000", 2024-05-08T08:54:36.998 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0809999999999996e-06 2024-05-08T08:54:36.998 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.999 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.999 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:36.999 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857881+0000", 2024-05-08T08:54:36.999 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.724e-06 2024-05-08T08:54:36.999 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.999 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.999 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:36.999 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906127+0000", 2024-05-08T08:54:36.999 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048245345000000002 2024-05-08T08:54:36.999 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.999 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.999 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:36.999 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906190+0000", 2024-05-08T08:54:36.999 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2762000000000007e-05 2024-05-08T08:54:36.999 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:36.999 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:36.999 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:36.999 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906262+0000", 2024-05-08T08:54:37.000 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2778e-05 2024-05-08T08:54:37.000 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:37.000 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:37.000 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:37.000 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.000 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.000 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22093 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 929792~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:37.000 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:54:37.000 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 129.10891892000001, 2024-05-08T08:54:37.000 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048532875000000003, 2024-05-08T08:54:37.000 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:37.000 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:37.000 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:37.000 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:37.000 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:37.001 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:37.001 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22093 2024-05-08T08:54:37.001 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.001 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:37.001 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.001 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:37.001 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:54:37.001 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:37.001 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.001 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.001 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:37.001 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:54:37.001 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:37.001 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.001 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.001 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:37.001 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857890+0000", 2024-05-08T08:54:37.002 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-07 2024-05-08T08:54:37.002 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.002 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.002 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:37.002 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857899+0000", 2024-05-08T08:54:37.002 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6889999999999999e-06 2024-05-08T08:54:37.002 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.002 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.002 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:37.002 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857906+0000", 2024-05-08T08:54:37.002 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6490000000000002e-06 2024-05-08T08:54:37.002 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.002 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.002 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:37.002 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857917+0000", 2024-05-08T08:54:37.002 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0764999999999999e-05 2024-05-08T08:54:37.003 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.003 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.003 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:37.003 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906287+0000", 2024-05-08T08:54:37.003 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048370442 2024-05-08T08:54:37.003 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.003 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.003 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:37.003 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906349+0000", 2024-05-08T08:54:37.003 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1537000000000004e-05 2024-05-08T08:54:37.003 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.003 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.003 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:37.003 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906422+0000", 2024-05-08T08:54:37.003 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3431999999999998e-05 2024-05-08T08:54:37.003 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:37.003 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:37.004 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:37.004 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.004 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.004 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22095 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 946176~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:37.004 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:54:37.004 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 129.10888421600001, 2024-05-08T08:54:37.004 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048657593999999998, 2024-05-08T08:54:37.004 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:37.004 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:37.004 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:37.004 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:37.004 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:37.004 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:37.004 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22095 2024-05-08T08:54:37.004 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.004 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:37.004 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.005 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:37.005 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:54:37.005 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:37.005 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.005 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.005 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:37.005 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:54:37.005 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:37.005 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.005 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.005 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:37.005 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857925+0000", 2024-05-08T08:54:37.005 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3099999999999998e-07 2024-05-08T08:54:37.005 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.005 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.005 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:37.006 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857927+0000", 2024-05-08T08:54:37.006 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.34e-06 2024-05-08T08:54:37.006 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.006 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.006 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:37.006 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857938+0000", 2024-05-08T08:54:37.006 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1426e-05 2024-05-08T08:54:37.006 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.006 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.006 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:37.006 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857943+0000", 2024-05-08T08:54:37.006 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5140000000000003e-06 2024-05-08T08:54:37.006 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.006 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.006 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:37.007 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906448+0000", 2024-05-08T08:54:37.007 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048504641000000001 2024-05-08T08:54:37.007 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.007 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.007 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:37.007 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906510+0000", 2024-05-08T08:54:37.007 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2663000000000006e-05 2024-05-08T08:54:37.007 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.007 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.007 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:37.007 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906582+0000", 2024-05-08T08:54:37.007 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1478999999999998e-05 2024-05-08T08:54:37.007 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:37.007 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:37.007 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:37.007 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.008 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.008 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22097 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 962560~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:37.008 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:54:37.008 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 129.108856527, 2024-05-08T08:54:37.008 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048791353000000003, 2024-05-08T08:54:37.008 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:37.008 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:37.008 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:37.008 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:37.008 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:37.008 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:37.008 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22097 2024-05-08T08:54:37.008 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.008 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:37.008 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.008 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:37.008 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:54:37.008 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:37.009 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.009 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.009 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:37.009 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:54:37.009 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:37.009 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.009 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.009 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:37.009 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:54:37.009 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9200000000000002e-07 2024-05-08T08:54:37.009 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.009 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.009 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:37.009 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857954+0000", 2024-05-08T08:54:37.009 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1900000000000002e-06 2024-05-08T08:54:37.009 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.009 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.010 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:37.010 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857961+0000", 2024-05-08T08:54:37.010 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4989999999999999e-06 2024-05-08T08:54:37.010 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.010 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.010 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:37.010 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857971+0000", 2024-05-08T08:54:37.010 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0485999999999999e-05 2024-05-08T08:54:37.010 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.010 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.010 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:37.010 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906607+0000", 2024-05-08T08:54:37.010 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048635109000000003 2024-05-08T08:54:37.010 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.010 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.010 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:37.011 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906667+0000", 2024-05-08T08:54:37.011 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0439000000000002e-05 2024-05-08T08:54:37.011 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.011 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.011 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:37.011 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906743+0000", 2024-05-08T08:54:37.011 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6237999999999998e-05 2024-05-08T08:54:37.011 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:37.011 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:37.011 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:37.011 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.011 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.011 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22099 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 978944~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:37.011 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:54:37.011 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 129.108830367, 2024-05-08T08:54:37.011 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048932055000000002, 2024-05-08T08:54:37.011 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:37.012 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:37.012 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:37.012 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:37.012 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:37.012 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:37.012 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22099 2024-05-08T08:54:37.012 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.012 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:37.012 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.012 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:37.012 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:54:37.012 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:37.012 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.012 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.012 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:37.012 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:54:37.012 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:37.013 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.013 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.013 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:37.013 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:54:37.013 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.27e-07 2024-05-08T08:54:37.013 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.013 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.013 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:37.013 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857987+0000", 2024-05-08T08:54:37.013 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5890000000000003e-06 2024-05-08T08:54:37.013 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.013 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.013 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:37.013 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857994+0000", 2024-05-08T08:54:37.013 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6619999999999999e-06 2024-05-08T08:54:37.014 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.014 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.014 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:37.014 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857998+0000", 2024-05-08T08:54:37.014 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7110000000000003e-06 2024-05-08T08:54:37.014 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.014 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.014 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:37.014 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906772+0000", 2024-05-08T08:54:37.014 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048773415000000001 2024-05-08T08:54:37.014 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.014 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.014 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:37.015 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906834+0000", 2024-05-08T08:54:37.015 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1752000000000001e-05 2024-05-08T08:54:37.015 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.015 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.015 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:37.015 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906910+0000", 2024-05-08T08:54:37.015 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6499000000000001e-05 2024-05-08T08:54:37.015 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:37.015 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:37.015 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:37.015 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.015 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.015 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22101 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 995328~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:37.015 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:54:37.015 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 129.108803236, 2024-05-08T08:54:37.016 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049087364000000001, 2024-05-08T08:54:37.016 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:37.016 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:37.016 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:37.016 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:37.016 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:37.016 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:37.016 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22101 2024-05-08T08:54:37.016 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.016 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:37.016 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.016 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:37.016 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:54:37.016 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:37.017 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.017 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.017 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:37.017 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:54:37.017 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:37.017 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.017 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.017 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:37.017 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858006+0000", 2024-05-08T08:54:37.017 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1399999999999997e-07 2024-05-08T08:54:37.017 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.017 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.017 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:37.017 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858008+0000", 2024-05-08T08:54:37.017 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2060000000000001e-06 2024-05-08T08:54:37.017 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.017 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.018 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:37.018 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858017+0000", 2024-05-08T08:54:37.018 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1579999999999994e-06 2024-05-08T08:54:37.018 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.018 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.018 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:37.018 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858022+0000", 2024-05-08T08:54:37.018 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4199999999999998e-06 2024-05-08T08:54:37.018 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.018 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.018 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:37.018 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906940+0000", 2024-05-08T08:54:37.018 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048917627999999998 2024-05-08T08:54:37.018 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.018 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.018 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:37.019 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907002+0000", 2024-05-08T08:54:37.019 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2399999999999999e-05 2024-05-08T08:54:37.019 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.019 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.019 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:37.019 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907092+0000", 2024-05-08T08:54:37.019 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0137999999999997e-05 2024-05-08T08:54:37.019 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:37.019 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:37.019 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:37.019 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.019 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.019 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22103 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1011712~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:37.019 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:54:37.019 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 129.10877845100001, 2024-05-08T08:54:37.019 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049228542, 2024-05-08T08:54:37.020 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:37.020 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:37.020 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:37.020 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:37.020 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:37.020 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:37.020 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22103 2024-05-08T08:54:37.020 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.020 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:37.020 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.020 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:37.020 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:54:37.020 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:37.020 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.020 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.020 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:37.021 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:54:37.021 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:37.021 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.021 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.021 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:37.021 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:54:37.021 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4299999999999999e-07 2024-05-08T08:54:37.021 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.021 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.021 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:37.021 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858032+0000", 2024-05-08T08:54:37.021 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0590000000000001e-06 2024-05-08T08:54:37.021 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.021 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.021 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:37.021 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858037+0000", 2024-05-08T08:54:37.021 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.977e-06 2024-05-08T08:54:37.022 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.022 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.022 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:37.022 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858042+0000", 2024-05-08T08:54:37.022 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7650000000000001e-06 2024-05-08T08:54:37.022 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.022 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.022 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:37.022 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907118+0000", 2024-05-08T08:54:37.022 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049076147000000001 2024-05-08T08:54:37.022 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.022 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.022 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:37.022 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907180+0000", 2024-05-08T08:54:37.022 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1971999999999996e-05 2024-05-08T08:54:37.022 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.022 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.023 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:37.023 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907258+0000", 2024-05-08T08:54:37.023 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8279e-05 2024-05-08T08:54:37.023 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:37.023 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:37.023 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:37.023 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.023 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.023 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22105 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1028096~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:37.023 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:54:37.023 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 129.10875920800001, 2024-05-08T08:54:37.023 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049394609999999999, 2024-05-08T08:54:37.023 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:37.023 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:37.023 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:37.023 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:37.023 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:37.024 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:37.024 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22105 2024-05-08T08:54:37.024 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.024 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:37.024 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.024 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:37.024 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:54:37.024 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:37.024 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.024 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.024 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:37.024 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:54:37.024 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:37.024 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.024 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.024 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:37.024 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858050+0000", 2024-05-08T08:54:37.025 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5999999999999999e-07 2024-05-08T08:54:37.025 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.025 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.025 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:37.025 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858051+0000", 2024-05-08T08:54:37.025 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8870000000000001e-06 2024-05-08T08:54:37.025 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.025 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.025 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:37.025 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858057+0000", 2024-05-08T08:54:37.025 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8019999999999997e-06 2024-05-08T08:54:37.025 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.025 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.025 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:37.025 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858062+0000", 2024-05-08T08:54:37.026 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.561e-06 2024-05-08T08:54:37.026 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.026 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.026 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:37.026 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907289+0000", 2024-05-08T08:54:37.026 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049226957000000002 2024-05-08T08:54:37.026 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.026 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.026 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:37.026 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907356+0000", 2024-05-08T08:54:37.026 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6915000000000006e-05 2024-05-08T08:54:37.026 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.026 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.026 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:37.027 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907444+0000", 2024-05-08T08:54:37.027 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8127999999999994e-05 2024-05-08T08:54:37.027 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:37.027 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:37.027 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:37.027 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.027 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.027 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22107 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1044480~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:37.027 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:54:37.027 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 129.10874086800001, 2024-05-08T08:54:37.027 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049799942999999999, 2024-05-08T08:54:37.027 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:37.027 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:37.027 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:37.027 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:37.027 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:37.028 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:37.028 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22107 2024-05-08T08:54:37.028 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.028 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:37.028 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.028 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:37.028 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:54:37.028 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:37.028 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.028 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.028 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:37.028 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:54:37.028 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:37.028 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.028 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.028 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:37.028 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:54:37.029 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4799999999999999e-07 2024-05-08T08:54:37.029 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.029 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.029 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:37.029 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858070+0000", 2024-05-08T08:54:37.029 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.012e-06 2024-05-08T08:54:37.029 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.029 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.029 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:37.029 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858077+0000", 2024-05-08T08:54:37.029 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6290000000000001e-06 2024-05-08T08:54:37.029 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.029 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.029 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:37.029 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858083+0000", 2024-05-08T08:54:37.030 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8880000000000002e-06 2024-05-08T08:54:37.030 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.030 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.030 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:37.030 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907671+0000", 2024-05-08T08:54:37.030 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049587191000000003 2024-05-08T08:54:37.030 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.030 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.030 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:37.030 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907763+0000", 2024-05-08T08:54:37.030 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2231000000000005e-05 2024-05-08T08:54:37.030 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.030 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.030 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:37.030 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907867+0000", 2024-05-08T08:54:37.030 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104644 2024-05-08T08:54:37.030 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:37.031 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:37.031 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:37.031 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.031 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.031 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22294 3.1c 3:3aad0389:::rbd_data.11406886b07b.0000000000000080:head [stat,delete] snapc 7=[7] RETRY=6 ondisk+retry+write+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:37.031 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:54:37.031 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 129.10814701999999, 2024-05-08T08:54:37.031 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051152191, 2024-05-08T08:54:37.031 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:37.031 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:37.031 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:37.031 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:37.031 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:37.031 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:37.031 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22294 2024-05-08T08:54:37.031 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.032 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:37.032 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.032 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:37.032 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:54:37.032 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:37.032 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.032 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.032 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:37.032 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:54:37.032 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:37.032 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.032 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.032 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:37.032 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858662+0000", 2024-05-08T08:54:37.032 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4799999999999998e-07 2024-05-08T08:54:37.032 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.032 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.033 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:37.033 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858664+0000", 2024-05-08T08:54:37.033 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1229999999999998e-06 2024-05-08T08:54:37.033 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.033 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.033 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:37.033 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858670+0000", 2024-05-08T08:54:37.033 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3099999999999997e-06 2024-05-08T08:54:37.033 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.033 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.033 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:37.033 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858676+0000", 2024-05-08T08:54:37.033 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9750000000000004e-06 2024-05-08T08:54:37.033 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.033 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.033 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:37.033 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907901+0000", 2024-05-08T08:54:37.034 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049224520000000001 2024-05-08T08:54:37.034 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.034 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.034 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:37.034 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.908085+0000", 2024-05-08T08:54:37.034 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00018377500000000001 2024-05-08T08:54:37.034 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.034 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.034 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:37.034 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.909814+0000", 2024-05-08T08:54:37.034 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00172894 2024-05-08T08:54:37.034 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:37.034 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:37.034 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:37.034 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.034 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.035 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4416.0:13461 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-05-08T08:54:37.035 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:22.487456+0000", 2024-05-08T08:54:37.035 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 74.479351979, 2024-05-08T08:54:37.035 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.99102620799999996, 2024-05-08T08:54:37.035 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:37.035 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:37.035 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:37.035 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:37.035 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4416", 2024-05-08T08:54:37.035 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:54:37.035 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 13461 2024-05-08T08:54:37.035 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.035 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:37.035 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.035 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:37.035 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487456+0000", 2024-05-08T08:54:37.036 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:37.036 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.036 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.036 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:37.036 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487456+0000", 2024-05-08T08:54:37.036 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:37.036 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.036 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.036 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:37.036 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487458+0000", 2024-05-08T08:54:37.036 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5659999999999999e-06 2024-05-08T08:54:37.036 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.036 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.036 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:37.036 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487463+0000", 2024-05-08T08:54:37.036 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6890000000000003e-06 2024-05-08T08:54:37.036 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.037 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.037 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:37.037 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487472+0000", 2024-05-08T08:54:37.037 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7710000000000001e-06 2024-05-08T08:54:37.037 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.037 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.037 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:37.037 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487478+0000", 2024-05-08T08:54:37.037 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1390000000000003e-06 2024-05-08T08:54:37.037 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.037 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.037 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:37.037 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487520+0000", 2024-05-08T08:54:37.037 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2259999999999999e-05 2024-05-08T08:54:37.037 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.037 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.038 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-05-08T08:54:37.038 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487528+0000", 2024-05-08T08:54:37.038 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7479999999999993e-06 2024-05-08T08:54:37.038 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.038 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.038 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:37.038 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:23.478139+0000", 2024-05-08T08:54:37.038 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.99061165200000001 2024-05-08T08:54:37.038 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.038 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.038 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:37.038 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:23.478301+0000", 2024-05-08T08:54:37.038 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000161602 2024-05-08T08:54:37.038 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.038 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.038 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:37.039 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:23.478483+0000", 2024-05-08T08:54:37.039 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000181781 2024-05-08T08:54:37.039 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:37.039 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:37.039 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:37.039 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.039 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.039 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5752.0:404 3.1a 3:5f5e8104:::rbd_data.1140bc4a55c7.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 5175296~45056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-05-08T08:54:37.039 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:00.390863+0000", 2024-05-08T08:54:37.039 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 36.575945783999998, 2024-05-08T08:54:37.039 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.055189174000000001, 2024-05-08T08:54:37.039 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:37.039 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:37.039 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:54:37.039 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:37.039 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5752", 2024-05-08T08:54:37.040 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/3534419815", 2024-05-08T08:54:37.040 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 404 2024-05-08T08:54:37.040 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.040 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:37.040 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.040 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:37.040 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.390863+0000", 2024-05-08T08:54:37.040 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:37.040 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.040 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.040 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:37.040 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.390863+0000", 2024-05-08T08:54:37.040 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:37.040 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.040 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.040 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:37.040 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.390864+0000", 2024-05-08T08:54:37.041 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.471e-06 2024-05-08T08:54:37.041 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.041 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.041 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:37.041 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397475+0000", 2024-05-08T08:54:37.041 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0066109109999999997 2024-05-08T08:54:37.041 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.041 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.041 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:37.041 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397498+0000", 2024-05-08T08:54:37.041 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2674e-05 2024-05-08T08:54:37.041 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.041 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.041 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:37.041 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397508+0000", 2024-05-08T08:54:37.041 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9019999999999994e-06 2024-05-08T08:54:37.042 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.042 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.042 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:37.042 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397609+0000", 2024-05-08T08:54:37.042 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010121899999999999 2024-05-08T08:54:37.042 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.042 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.042 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:37.042 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397711+0000", 2024-05-08T08:54:37.042 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102489 2024-05-08T08:54:37.042 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.042 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.042 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-05-08T08:54:37.042 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397999+0000", 2024-05-08T08:54:37.042 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00028733799999999999 2024-05-08T08:54:37.043 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.043 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.043 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:54:37.043 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.432093+0000", 2024-05-08T08:54:37.043 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.034094865000000002 2024-05-08T08:54:37.043 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.043 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.043 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:54:37.043 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.445975+0000", 2024-05-08T08:54:37.043 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013881187 2024-05-08T08:54:37.043 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.043 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.043 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:54:37.043 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.446013+0000", 2024-05-08T08:54:37.043 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8155000000000003e-05 2024-05-08T08:54:37.043 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:37.043 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:37.044 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:37.044 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.446052+0000", 2024-05-08T08:54:37.044 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8962999999999999e-05 2024-05-08T08:54:37.044 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:37.044 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:37.044 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:37.044 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:37.044 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:37.044 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:37.045 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:54:37.045 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:54:37.046 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:54:37.047 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:37.048+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:37.055 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:54:37.055 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:54:37.072 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:54:37.072 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:54:37.089 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:54:37.090 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:54:37.107 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:54:37.107 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:54:37.124 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:54:37.125 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:54:37.146 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:37.149+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:37.192 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:37.192 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:37.201 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:54:37.246 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:37.249+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:37.347 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:37.347 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:37.347 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:37.350+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:37.356 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:54:37.448 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:37.450+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:37.461 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:37.461 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:37.470 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:54:37.549 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:37.552+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:37.582 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:37.582 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:37.591 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:54:37.650 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:37.653+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:37.741 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:37.741 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:37.751 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:54:37.751 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:37.753+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:37.830 INFO:teuthology.orchestra.run.smithi059.stderr:reweighted osd.3 to 1 (10000) 2024-05-08T08:54:37.847 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-05-08T08:54:37.867 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:37.867 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:37.867 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:37.870+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:37.877 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:54:37.952 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:37.954+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:38.042 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:38.042 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:38.051 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:54:38.052 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:38.055+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:38.152 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:38.155+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:38.162 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:38.162 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:38.171 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:54:38.255 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:38.258+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:38.321 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:38.321 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:38.330 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:54:38.372 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:38.374+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:38.456 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:38.458+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:38.480 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:38.480 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:38.489 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:54:38.557 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:38.559+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:38.598 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:38.598 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:38.607 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:54:38.692 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:38.695+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:38.758 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:38.759 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:38.759 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:38.761+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:38.767 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:54:38.837 INFO:teuthology.orchestra.run.smithi059.stderr:reweighted osd.4 to 1 (10000) 2024-05-08T08:54:38.843 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-05-08T08:54:38.853 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-05-08T08:54:38.853 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T08:54:38.881 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:38.881 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:38.890 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:54:38.895 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:38.898+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:38.960 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:38.963+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:39.001 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:39.001 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:39.010 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:54:39.061 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:39.064+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:39.116 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:39.116 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:39.125 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:54:39.136 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T08:54:39.169 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:39.172+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:39.221 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:54:39.221 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-05-08T08:54:39.230 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:39.230 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:39.233 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":202,"stamp":"2024-05-08T08:54:38.838683+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1153096012,"num_objects":2911,"num_object_clones":438,"num_object_copies":5822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2911,"num_whiteouts":1643,"num_read":501824,"num_read_kb":1618870,"num_write":24065,"num_write_kb":453750,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3104,"num_bytes_recovered":190761442,"num_keys_recovered":470,"num_objects_omap":43,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21705,"ondisk_log_size":21705,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":471859200,"kb_used":377032,"kb_used_data":82280,"kb_used_omap":7,"kb_used_meta":294648,"kb_avail":471482168,"statfs":{"total":483183820800,"available":482797740032,"internally_reserved":0,"allocated":84254720,"data_stored":643684963,"data_compressed":32949010,"data_compressed_allocated":62701568,"data_compressed_original":631976448,"omap_allocated":7973,"internal_metadata":301719771},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-205841523,"num_objects":-121,"num_object_clones":-17,"num_object_copies":-242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-121,"num_whiteouts":-6,"num_read":1198,"num_read_kb":1026,"num_write":276,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.032230"},"pg_stats":[{"pgid":"3.1f","version":"124'651","reported_seq":19951,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.835655+0000","last_change":"2024-05-08T08:54:36.835854+0000","last_active":"2024-05-08T08:54:36.835655+0000","last_peered":"2024-05-08T08:54:36.835655+0000","last_clean":"2024-05-08T08:54:36.835655+0000","last_became_active":"2024-05-08T08:53:49.580563+0000","last_became_peered":"2024-05-08T08:53:49.580563+0000","last_unstale":"2024-05-08T08:54:36.835655+0000","last_undegraded":"2024-05-08T08:54:36.835655+0000","last_fullsized":"2024-05-08T08:54:36.835655+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":651,"log_dups_size":0,"ondisk_log_size":651,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:03:44.387830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000879771,"stat_sum":{"num_bytes":64664935,"num_objects":95,"num_object_clones":17,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":45,"num_read":19059,"num_read_kb":85554,"num_write":900,"num_write_kb":33084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2b","length":"9"}]},{"pgid":"3.1e","version":"125'805","reported_seq":11277,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.959741+0000","last_change":"2024-05-08T08:54:36.827994+0000","last_active":"2024-05-08T08:54:36.959741+0000","last_peered":"2024-05-08T08:54:36.959741+0000","last_clean":"2024-05-08T08:54:36.959741+0000","last_became_active":"2024-05-08T08:52:11.930844+0000","last_became_peered":"2024-05-08T08:52:11.930844+0000","last_unstale":"2024-05-08T08:54:36.959741+0000","last_undegraded":"2024-05-08T08:54:36.959741+0000","last_fullsized":"2024-05-08T08:54:36.959741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":805,"log_dups_size":0,"ondisk_log_size":805,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:37:20.288832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001925115,"stat_sum":{"num_bytes":20532352,"num_objects":94,"num_object_clones":12,"num_object_copies":188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":60,"num_read":13187,"num_read_kb":36671,"num_write":617,"num_write_kb":12702,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2b","length":"9"}]},{"pgid":"3.1d","version":"125'666","reported_seq":4010,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.885839+0000","last_change":"2024-05-08T08:54:36.825851+0000","last_active":"2024-05-08T08:54:36.885839+0000","last_peered":"2024-05-08T08:54:36.885839+0000","last_clean":"2024-05-08T08:54:36.885839+0000","last_became_active":"2024-05-08T08:53:59.618085+0000","last_became_peered":"2024-05-08T08:53:59.618085+0000","last_unstale":"2024-05-08T08:54:36.885839+0000","last_undegraded":"2024-05-08T08:54:36.885839+0000","last_fullsized":"2024-05-08T08:54:36.885839+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":666,"log_dups_size":0,"ondisk_log_size":666,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:23:12.297537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072215199999999999,"stat_sum":{"num_bytes":14415630,"num_objects":83,"num_object_clones":11,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":47,"num_read":8201,"num_read_kb":30455,"num_write":835,"num_write_kb":7273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":141,"num_bytes_recovered":5594929,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2b","length":"9"}]},{"pgid":"3.1c","version":"125'596","reported_seq":18144,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.887393+0000","last_change":"2024-05-08T08:54:36.826322+0000","last_active":"2024-05-08T08:54:36.887393+0000","last_peered":"2024-05-08T08:54:36.887393+0000","last_clean":"2024-05-08T08:54:36.887393+0000","last_became_active":"2024-05-08T08:53:58.621483+0000","last_became_peered":"2024-05-08T08:53:58.621483+0000","last_unstale":"2024-05-08T08:54:36.887393+0000","last_undegraded":"2024-05-08T08:54:36.887393+0000","last_fullsized":"2024-05-08T08:54:36.887393+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":596,"log_dups_size":0,"ondisk_log_size":596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:00:24.021853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094490299999999998,"stat_sum":{"num_bytes":55879221,"num_objects":95,"num_object_clones":11,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":54,"num_read":17976,"num_read_kb":100941,"num_write":657,"num_write_kb":27844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":287,"num_bytes_recovered":49590415,"num_keys_recovered":166,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2b","length":"9"}]},{"pgid":"3.1b","version":"124'553","reported_seq":25224,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.825428+0000","last_change":"2024-05-08T08:54:36.825629+0000","last_active":"2024-05-08T08:54:36.825428+0000","last_peered":"2024-05-08T08:54:36.825428+0000","last_clean":"2024-05-08T08:54:36.825428+0000","last_became_active":"2024-05-08T08:53:49.583097+0000","last_became_peered":"2024-05-08T08:53:49.583097+0000","last_unstale":"2024-05-08T08:54:36.825428+0000","last_undegraded":"2024-05-08T08:54:36.825428+0000","last_fullsized":"2024-05-08T08:54:36.825428+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":553,"log_dups_size":0,"ondisk_log_size":553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:24:45.532150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055716000000000001,"stat_sum":{"num_bytes":47790336,"num_objects":100,"num_object_clones":13,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":62,"num_read":24904,"num_read_kb":45652,"num_write":634,"num_write_kb":23301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":129,"num_bytes_recovered":3090520,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2b","length":"9"}]},{"pgid":"3.1a","version":"125'760","reported_seq":21427,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.943653+0000","last_change":"2024-05-08T08:54:36.826046+0000","last_active":"2024-05-08T08:54:36.943653+0000","last_peered":"2024-05-08T08:54:36.943653+0000","last_clean":"2024-05-08T08:54:36.943653+0000","last_became_active":"2024-05-08T08:53:24.484124+0000","last_became_peered":"2024-05-08T08:53:24.484124+0000","last_unstale":"2024-05-08T08:54:36.943653+0000","last_undegraded":"2024-05-08T08:54:36.943653+0000","last_fullsized":"2024-05-08T08:54:36.943653+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":760,"log_dups_size":0,"ondisk_log_size":760,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:50:59.189663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080935299999999996,"stat_sum":{"num_bytes":27794812,"num_objects":95,"num_object_clones":16,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":47,"num_read":23962,"num_read_kb":59334,"num_write":840,"num_write_kb":19082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":130,"num_bytes_recovered":8549042,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2b","length":"9"}]},{"pgid":"3.19","version":"124'380","reported_seq":10352,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.825750+0000","last_change":"2024-05-08T08:54:36.825750+0000","last_active":"2024-05-08T08:54:36.825750+0000","last_peered":"2024-05-08T08:54:36.825750+0000","last_clean":"2024-05-08T08:54:36.825750+0000","last_became_active":"2024-05-08T08:53:24.483403+0000","last_became_peered":"2024-05-08T08:53:24.483403+0000","last_unstale":"2024-05-08T08:54:36.825750+0000","last_undegraded":"2024-05-08T08:54:36.825750+0000","last_fullsized":"2024-05-08T08:54:36.825750+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":380,"log_dups_size":0,"ondisk_log_size":380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:22:50.312657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054762300000000001,"stat_sum":{"num_bytes":20318328,"num_objects":90,"num_object_clones":12,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":56,"num_read":9401,"num_read_kb":48041,"num_write":346,"num_write_kb":16514,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":8814624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2b","length":"9"}]},{"pgid":"3.18","version":"124'638","reported_seq":19227,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.826645+0000","last_change":"2024-05-08T08:54:36.826645+0000","last_active":"2024-05-08T08:54:36.826645+0000","last_peered":"2024-05-08T08:54:36.826645+0000","last_clean":"2024-05-08T08:54:36.826645+0000","last_became_active":"2024-05-08T08:52:11.927366+0000","last_became_peered":"2024-05-08T08:52:11.927366+0000","last_unstale":"2024-05-08T08:54:36.826645+0000","last_undegraded":"2024-05-08T08:54:36.826645+0000","last_fullsized":"2024-05-08T08:54:36.826645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":638,"log_dups_size":0,"ondisk_log_size":638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:06:16.899829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012129930000000001,"stat_sum":{"num_bytes":80472280,"num_objects":100,"num_object_clones":19,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":57,"num_read":20615,"num_read_kb":52536,"num_write":636,"num_write_kb":23374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2b","length":"9"}]},{"pgid":"3.17","version":"124'753","reported_seq":18455,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.826705+0000","last_change":"2024-05-08T08:54:36.827112+0000","last_active":"2024-05-08T08:54:36.826705+0000","last_peered":"2024-05-08T08:54:36.826705+0000","last_clean":"2024-05-08T08:54:36.826705+0000","last_became_active":"2024-05-08T08:53:11.358391+0000","last_became_peered":"2024-05-08T08:53:11.358391+0000","last_unstale":"2024-05-08T08:54:36.826705+0000","last_undegraded":"2024-05-08T08:54:36.826705+0000","last_fullsized":"2024-05-08T08:54:36.826705+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":753,"log_dups_size":0,"ondisk_log_size":753,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:15:31.272261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001337396,"stat_sum":{"num_bytes":58717330,"num_objects":82,"num_object_clones":13,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":39,"num_read":20201,"num_read_kb":76886,"num_write":897,"num_write_kb":16919,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":97,"num_bytes_recovered":2508994,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2b","length":"9"}]},{"pgid":"3.16","version":"125'1357","reported_seq":21214,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.957927+0000","last_change":"2024-05-08T08:54:36.835627+0000","last_active":"2024-05-08T08:54:36.957927+0000","last_peered":"2024-05-08T08:54:36.957927+0000","last_clean":"2024-05-08T08:54:36.957927+0000","last_became_active":"2024-05-08T08:53:59.615917+0000","last_became_peered":"2024-05-08T08:53:59.615917+0000","last_unstale":"2024-05-08T08:54:36.957927+0000","last_undegraded":"2024-05-08T08:54:36.957927+0000","last_fullsized":"2024-05-08T08:54:36.957927+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":1357,"log_dups_size":0,"ondisk_log_size":1357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:13:12.400754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012988489999999999,"stat_sum":{"num_bytes":40021352,"num_objects":83,"num_object_clones":12,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":42,"num_read":25230,"num_read_kb":87053,"num_write":2178,"num_write_kb":27176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":118,"num_bytes_recovered":21349055,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2b","length":"9"}]},{"pgid":"3.15","version":"125'603","reported_seq":16017,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.916428+0000","last_change":"2024-05-08T08:54:36.835546+0000","last_active":"2024-05-08T08:54:36.916428+0000","last_peered":"2024-05-08T08:54:36.916428+0000","last_clean":"2024-05-08T08:54:36.916428+0000","last_became_active":"2024-05-08T08:52:11.931695+0000","last_became_peered":"2024-05-08T08:52:11.931695+0000","last_unstale":"2024-05-08T08:54:36.916428+0000","last_undegraded":"2024-05-08T08:54:36.916428+0000","last_fullsized":"2024-05-08T08:54:36.916428+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:22:50.380548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089158499999999997,"stat_sum":{"num_bytes":25074782,"num_objects":83,"num_object_clones":11,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":50,"num_read":17145,"num_read_kb":52692,"num_write":715,"num_write_kb":8746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2b","length":"9"}]},{"pgid":"3.14","version":"124'625","reported_seq":12155,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.834704+0000","last_change":"2024-05-08T08:54:36.834881+0000","last_active":"2024-05-08T08:54:36.834704+0000","last_peered":"2024-05-08T08:54:36.834704+0000","last_clean":"2024-05-08T08:54:36.834704+0000","last_became_active":"2024-05-08T08:53:23.481362+0000","last_became_peered":"2024-05-08T08:53:23.481362+0000","last_unstale":"2024-05-08T08:54:36.834704+0000","last_undegraded":"2024-05-08T08:54:36.834704+0000","last_fullsized":"2024-05-08T08:54:36.834704+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":625,"log_dups_size":0,"ondisk_log_size":625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:06:33.760998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066539699999999997,"stat_sum":{"num_bytes":35524723,"num_objects":82,"num_object_clones":16,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":43,"num_read":10444,"num_read_kb":32909,"num_write":595,"num_write_kb":11437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":3612768,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2b","length":"9"}]},{"pgid":"2.7","version":"0'0","reported_seq":238,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.835108+0000","last_change":"2024-05-08T08:53:20.362387+0000","last_active":"2024-05-08T08:54:36.835108+0000","last_peered":"2024-05-08T08:54:36.835108+0000","last_clean":"2024-05-08T08:54:36.835108+0000","last_became_active":"2024-05-08T08:53:20.362113+0000","last_became_peered":"2024-05-08T08:53:20.362113+0000","last_unstale":"2024-05-08T08:54:36.835108+0000","last_undegraded":"2024-05-08T08:54:36.835108+0000","last_fullsized":"2024-05-08T08:54:36.835108+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:28:25.204125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"125'528","reported_seq":19653,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.910281+0000","last_change":"2024-05-08T08:54:36.827679+0000","last_active":"2024-05-08T08:54:36.910281+0000","last_peered":"2024-05-08T08:54:36.910281+0000","last_clean":"2024-05-08T08:54:36.910281+0000","last_became_active":"2024-05-08T08:53:49.579888+0000","last_became_peered":"2024-05-08T08:53:49.579888+0000","last_unstale":"2024-05-08T08:54:36.910281+0000","last_undegraded":"2024-05-08T08:54:36.910281+0000","last_fullsized":"2024-05-08T08:54:36.910281+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":528,"log_dups_size":0,"ondisk_log_size":528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:02:37.637195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001567492,"stat_sum":{"num_bytes":40662233,"num_objects":81,"num_object_clones":9,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":48,"num_read":21227,"num_read_kb":88195,"num_write":524,"num_write_kb":17531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2b","length":"9"}]},{"pgid":"2.6","version":"0'0","reported_seq":229,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.826254+0000","last_change":"2024-05-08T08:52:03.896159+0000","last_active":"2024-05-08T08:54:36.826254+0000","last_peered":"2024-05-08T08:54:36.826254+0000","last_clean":"2024-05-08T08:54:36.826254+0000","last_became_active":"2024-05-08T08:52:03.895797+0000","last_became_peered":"2024-05-08T08:52:03.895797+0000","last_unstale":"2024-05-08T08:54:36.826254+0000","last_undegraded":"2024-05-08T08:54:36.826254+0000","last_fullsized":"2024-05-08T08:54:36.826254+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:34:33.513525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"125'433","reported_seq":6399,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.826275+0000","last_change":"2024-05-08T08:54:36.826275+0000","last_active":"2024-05-08T08:54:36.826275+0000","last_peered":"2024-05-08T08:54:36.826275+0000","last_clean":"2024-05-08T08:54:36.826275+0000","last_became_active":"2024-05-08T08:54:32.801917+0000","last_became_peered":"2024-05-08T08:54:32.801917+0000","last_unstale":"2024-05-08T08:54:36.826275+0000","last_undegraded":"2024-05-08T08:54:36.826275+0000","last_fullsized":"2024-05-08T08:54:36.826275+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:08:55.683221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096415599999999998,"stat_sum":{"num_bytes":15654049,"num_objects":95,"num_object_clones":17,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":50,"num_read":6596,"num_read_kb":34394,"num_write":375,"num_write_kb":12018,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":162,"num_bytes_recovered":8401073,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2b","length":"9"}]},{"pgid":"2.5","version":"0'0","reported_seq":228,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.834873+0000","last_change":"2024-05-08T08:52:04.144764+0000","last_active":"2024-05-08T08:54:36.834873+0000","last_peered":"2024-05-08T08:54:36.834873+0000","last_clean":"2024-05-08T08:54:36.834873+0000","last_became_active":"2024-05-08T08:52:04.144444+0000","last_became_peered":"2024-05-08T08:52:04.144444+0000","last_unstale":"2024-05-08T08:54:36.834873+0000","last_undegraded":"2024-05-08T08:54:36.834873+0000","last_fullsized":"2024-05-08T08:54:36.834873+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:58:31.498922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"124'728","reported_seq":25659,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.938664+0000","last_change":"2024-05-08T08:54:36.827387+0000","last_active":"2024-05-08T08:54:36.938664+0000","last_peered":"2024-05-08T08:54:36.938664+0000","last_clean":"2024-05-08T08:54:36.938664+0000","last_became_active":"2024-05-08T08:53:58.618388+0000","last_became_peered":"2024-05-08T08:53:58.618388+0000","last_unstale":"2024-05-08T08:54:36.938664+0000","last_undegraded":"2024-05-08T08:54:36.938664+0000","last_fullsized":"2024-05-08T08:54:36.938664+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":728,"log_dups_size":0,"ondisk_log_size":728,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:53:00.560684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001856866,"stat_sum":{"num_bytes":49267216,"num_objects":115,"num_object_clones":19,"num_object_copies":230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":115,"num_whiteouts":63,"num_read":21930,"num_read_kb":51626,"num_write":572,"num_write_kb":12044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":215,"num_bytes_recovered":10925568,"num_keys_recovered":44,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2b","length":"9"}]},{"pgid":"2.4","version":"0'0","reported_seq":229,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.825458+0000","last_change":"2024-05-08T08:52:04.144543+0000","last_active":"2024-05-08T08:54:36.825458+0000","last_peered":"2024-05-08T08:54:36.825458+0000","last_clean":"2024-05-08T08:54:36.825458+0000","last_became_active":"2024-05-08T08:52:04.144242+0000","last_became_peered":"2024-05-08T08:52:04.144242+0000","last_unstale":"2024-05-08T08:54:36.825458+0000","last_undegraded":"2024-05-08T08:54:36.825458+0000","last_fullsized":"2024-05-08T08:54:36.825458+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:57:36.844633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"125'863","reported_seq":13601,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.828032+0000","last_change":"2024-05-08T08:54:36.828032+0000","last_active":"2024-05-08T08:54:36.828032+0000","last_peered":"2024-05-08T08:54:36.828032+0000","last_clean":"2024-05-08T08:54:36.828032+0000","last_became_active":"2024-05-08T08:53:58.615614+0000","last_became_peered":"2024-05-08T08:53:58.615614+0000","last_unstale":"2024-05-08T08:54:36.828032+0000","last_undegraded":"2024-05-08T08:54:36.828032+0000","last_fullsized":"2024-05-08T08:54:36.828032+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":863,"log_dups_size":0,"ondisk_log_size":863,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:57:46.227211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019662109999999998,"stat_sum":{"num_bytes":16686331,"num_objects":103,"num_object_clones":15,"num_object_copies":206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103,"num_whiteouts":67,"num_read":15031,"num_read_kb":32630,"num_write":1091,"num_write_kb":10601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":142,"num_bytes_recovered":4609275,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2b","length":"9"}]},{"pgid":"3.3","version":"125'567","reported_seq":10885,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.935607+0000","last_change":"2024-05-08T08:54:36.826519+0000","last_active":"2024-05-08T08:54:36.935607+0000","last_peered":"2024-05-08T08:54:36.935607+0000","last_clean":"2024-05-08T08:54:36.935607+0000","last_became_active":"2024-05-08T08:53:24.480423+0000","last_became_peered":"2024-05-08T08:53:24.480423+0000","last_unstale":"2024-05-08T08:54:36.935607+0000","last_undegraded":"2024-05-08T08:54:36.935607+0000","last_fullsized":"2024-05-08T08:54:36.935607+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":567,"log_dups_size":0,"ondisk_log_size":567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:40:16.111591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001006863,"stat_sum":{"num_bytes":30593870,"num_objects":80,"num_object_clones":13,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":45,"num_read":11335,"num_read_kb":33106,"num_write":644,"num_write_kb":6500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":101,"num_bytes_recovered":5689443,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2b","length":"9"}]},{"pgid":"2.2","version":"0'0","reported_seq":209,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.826218+0000","last_change":"2024-05-08T08:53:55.365435+0000","last_active":"2024-05-08T08:54:36.826218+0000","last_peered":"2024-05-08T08:54:36.826218+0000","last_clean":"2024-05-08T08:54:36.826218+0000","last_became_active":"2024-05-08T08:53:55.365059+0000","last_became_peered":"2024-05-08T08:53:55.365059+0000","last_unstale":"2024-05-08T08:54:36.826218+0000","last_undegraded":"2024-05-08T08:54:36.826218+0000","last_fullsized":"2024-05-08T08:54:36.826218+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:15:39.278668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"124'639","reported_seq":13714,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.827079+0000","last_change":"2024-05-08T08:54:36.827304+0000","last_active":"2024-05-08T08:54:36.827079+0000","last_peered":"2024-05-08T08:54:36.827079+0000","last_clean":"2024-05-08T08:54:36.827079+0000","last_became_active":"2024-05-08T08:52:11.928301+0000","last_became_peered":"2024-05-08T08:52:11.928301+0000","last_unstale":"2024-05-08T08:54:36.827079+0000","last_undegraded":"2024-05-08T08:54:36.827079+0000","last_fullsized":"2024-05-08T08:54:36.827079+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":639,"log_dups_size":0,"ondisk_log_size":639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:45:01.844140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012830719999999999,"stat_sum":{"num_bytes":45686375,"num_objects":93,"num_object_clones":16,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":51,"num_read":10493,"num_read_kb":43924,"num_write":600,"num_write_kb":16296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2b","length":"9"}]},{"pgid":"2.1","version":"0'0","reported_seq":229,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.825153+0000","last_change":"2024-05-08T08:52:03.891768+0000","last_active":"2024-05-08T08:54:36.825153+0000","last_peered":"2024-05-08T08:54:36.825153+0000","last_clean":"2024-05-08T08:54:36.825153+0000","last_became_active":"2024-05-08T08:52:03.891374+0000","last_became_peered":"2024-05-08T08:52:03.891374+0000","last_unstale":"2024-05-08T08:54:36.825153+0000","last_undegraded":"2024-05-08T08:54:36.825153+0000","last_fullsized":"2024-05-08T08:54:36.825153+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:18:10.240492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"125'981","reported_seq":26827,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.825569+0000","last_change":"2024-05-08T08:54:36.825569+0000","last_active":"2024-05-08T08:54:36.825569+0000","last_peered":"2024-05-08T08:54:36.825569+0000","last_clean":"2024-05-08T08:54:36.825569+0000","last_became_active":"2024-05-08T08:53:49.581797+0000","last_became_peered":"2024-05-08T08:53:49.581797+0000","last_unstale":"2024-05-08T08:54:36.825569+0000","last_undegraded":"2024-05-08T08:54:36.825569+0000","last_fullsized":"2024-05-08T08:54:36.825569+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":981,"log_dups_size":0,"ondisk_log_size":981,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:18:13.421855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060326200000000003,"stat_sum":{"num_bytes":45467468,"num_objects":104,"num_object_clones":16,"num_object_copies":208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":54,"num_read":29789,"num_read_kb":55110,"num_write":748,"num_write_kb":9610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":135,"num_bytes_recovered":3549534,"num_keys_recovered":46,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2b","length":"9"}]},{"pgid":"2.0","version":"0'0","reported_seq":228,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.833923+0000","last_change":"2024-05-08T08:52:03.891551+0000","last_active":"2024-05-08T08:54:36.833923+0000","last_peered":"2024-05-08T08:54:36.833923+0000","last_clean":"2024-05-08T08:54:36.833923+0000","last_became_active":"2024-05-08T08:52:03.891296+0000","last_became_peered":"2024-05-08T08:52:03.891296+0000","last_unstale":"2024-05-08T08:54:36.833923+0000","last_undegraded":"2024-05-08T08:54:36.833923+0000","last_fullsized":"2024-05-08T08:54:36.833923+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:54.806343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"125'896","reported_seq":14006,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.923309+0000","last_change":"2024-05-08T08:54:36.827665+0000","last_active":"2024-05-08T08:54:36.923309+0000","last_peered":"2024-05-08T08:54:36.923309+0000","last_clean":"2024-05-08T08:54:36.923309+0000","last_became_active":"2024-05-08T08:53:58.616032+0000","last_became_peered":"2024-05-08T08:53:58.616032+0000","last_unstale":"2024-05-08T08:54:36.923309+0000","last_undegraded":"2024-05-08T08:54:36.923309+0000","last_fullsized":"2024-05-08T08:54:36.923309+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":896,"log_dups_size":0,"ondisk_log_size":896,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:13:59.148304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015983410000000001,"stat_sum":{"num_bytes":40747457,"num_objects":103,"num_object_clones":18,"num_object_copies":206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103,"num_whiteouts":57,"num_read":15365,"num_read_kb":63333,"num_write":1196,"num_write_kb":13964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":125,"num_bytes_recovered":4950942,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2b","length":"9"}]},{"pgid":"2.3","version":"0'0","reported_seq":209,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.826619+0000","last_change":"2024-05-08T08:53:55.362409+0000","last_active":"2024-05-08T08:54:36.826619+0000","last_peered":"2024-05-08T08:54:36.826619+0000","last_clean":"2024-05-08T08:54:36.826619+0000","last_became_active":"2024-05-08T08:53:55.362055+0000","last_became_peered":"2024-05-08T08:53:55.362055+0000","last_unstale":"2024-05-08T08:54:36.826619+0000","last_undegraded":"2024-05-08T08:54:36.826619+0000","last_fullsized":"2024-05-08T08:54:36.826619+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:34:23.719119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":384,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.834009+0000","last_change":"2024-05-08T08:51:58.861899+0000","last_active":"2024-05-08T08:54:36.834009+0000","last_peered":"2024-05-08T08:54:36.834009+0000","last_clean":"2024-05-08T08:54:36.834009+0000","last_became_active":"2024-05-08T08:51:58.861596+0000","last_became_peered":"2024-05-08T08:51:58.861596+0000","last_unstale":"2024-05-08T08:54:36.834009+0000","last_undegraded":"2024-05-08T08:54:36.834009+0000","last_fullsized":"2024-05-08T08:54:36.834009+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:51:57.849666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:51:57.849666+0000","last_clean_scrub_stamp":"2024-05-08T08:51:57.849666+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:53:21.739950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"125'343","reported_seq":9435,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.828424+0000","last_change":"2024-05-08T08:54:36.828424+0000","last_active":"2024-05-08T08:54:36.828424+0000","last_peered":"2024-05-08T08:54:36.828424+0000","last_clean":"2024-05-08T08:54:36.828424+0000","last_became_active":"2024-05-08T08:52:11.926975+0000","last_became_peered":"2024-05-08T08:52:11.926975+0000","last_unstale":"2024-05-08T08:54:36.828424+0000","last_undegraded":"2024-05-08T08:54:36.828424+0000","last_fullsized":"2024-05-08T08:54:36.828424+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:35:58.570037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002105834,"stat_sum":{"num_bytes":31656032,"num_objects":88,"num_object_clones":15,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":48,"num_read":8890,"num_read_kb":36552,"num_write":315,"num_write_kb":8985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2b","length":"9"}]},{"pgid":"3.9","version":"125'430","reported_seq":14065,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.825806+0000","last_change":"2024-05-08T08:54:36.825806+0000","last_active":"2024-05-08T08:54:36.825806+0000","last_peered":"2024-05-08T08:54:36.825806+0000","last_clean":"2024-05-08T08:54:36.825806+0000","last_became_active":"2024-05-08T08:53:23.478564+0000","last_became_peered":"2024-05-08T08:53:23.478564+0000","last_unstale":"2024-05-08T08:54:36.825806+0000","last_undegraded":"2024-05-08T08:54:36.825806+0000","last_fullsized":"2024-05-08T08:54:36.825806+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":430,"log_dups_size":0,"ondisk_log_size":430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:07:33.542111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071444800000000001,"stat_sum":{"num_bytes":36214831,"num_objects":92,"num_object_clones":11,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":52,"num_read":14021,"num_read_kb":62064,"num_write":368,"num_write_kb":16560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":131,"num_bytes_recovered":12377088,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2b","length":"9"}]},{"pgid":"3.a","version":"125'1122","reported_seq":11276,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.915030+0000","last_change":"2024-05-08T08:54:36.834866+0000","last_active":"2024-05-08T08:54:36.915030+0000","last_peered":"2024-05-08T08:54:36.915030+0000","last_clean":"2024-05-08T08:54:36.915030+0000","last_became_active":"2024-05-08T08:53:24.480359+0000","last_became_peered":"2024-05-08T08:53:24.480359+0000","last_unstale":"2024-05-08T08:54:36.915030+0000","last_undegraded":"2024-05-08T08:54:36.915030+0000","last_fullsized":"2024-05-08T08:54:36.915030+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":1122,"log_dups_size":0,"ondisk_log_size":1122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:13:17.330414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070201699999999998,"stat_sum":{"num_bytes":21900143,"num_objects":84,"num_object_clones":12,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":50,"num_read":15403,"num_read_kb":29111,"num_write":1358,"num_write_kb":6314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":2175343,"num_keys_recovered":61,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2b","length":"9"}]},{"pgid":"3.b","version":"125'527","reported_seq":11224,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.826465+0000","last_change":"2024-05-08T08:54:36.826465+0000","last_active":"2024-05-08T08:54:36.826465+0000","last_peered":"2024-05-08T08:54:36.826465+0000","last_clean":"2024-05-08T08:54:36.826465+0000","last_became_active":"2024-05-08T08:52:11.927196+0000","last_became_peered":"2024-05-08T08:52:11.927196+0000","last_unstale":"2024-05-08T08:54:36.826465+0000","last_undegraded":"2024-05-08T08:54:36.826465+0000","last_fullsized":"2024-05-08T08:54:36.826465+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":527,"log_dups_size":0,"ondisk_log_size":527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:13:45.973320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087152499999999995,"stat_sum":{"num_bytes":39118437,"num_objects":96,"num_object_clones":12,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":60,"num_read":11551,"num_read_kb":39176,"num_write":449,"num_write_kb":7713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2b","length":"9"}]},{"pgid":"3.c","version":"124'335","reported_seq":13605,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.835019+0000","last_change":"2024-05-08T08:54:36.835208+0000","last_active":"2024-05-08T08:54:36.835019+0000","last_peered":"2024-05-08T08:54:36.835019+0000","last_clean":"2024-05-08T08:54:36.835019+0000","last_became_active":"2024-05-08T08:53:58.615095+0000","last_became_peered":"2024-05-08T08:53:58.615095+0000","last_unstale":"2024-05-08T08:54:36.835019+0000","last_undegraded":"2024-05-08T08:54:36.835019+0000","last_fullsized":"2024-05-08T08:54:36.835019+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:19:42.959484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093134299999999995,"stat_sum":{"num_bytes":15228603,"num_objects":82,"num_object_clones":10,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":49,"num_read":13093,"num_read_kb":41263,"num_write":300,"num_write_kb":7340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":107,"num_bytes_recovered":2139323,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2b","length":"9"}]},{"pgid":"3.d","version":"125'415","reported_seq":8430,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.914429+0000","last_change":"2024-05-08T08:54:36.835776+0000","last_active":"2024-05-08T08:54:36.914429+0000","last_peered":"2024-05-08T08:54:36.914429+0000","last_clean":"2024-05-08T08:54:36.914429+0000","last_became_active":"2024-05-08T08:52:11.932594+0000","last_became_peered":"2024-05-08T08:52:11.932594+0000","last_unstale":"2024-05-08T08:54:36.914429+0000","last_undegraded":"2024-05-08T08:54:36.914429+0000","last_fullsized":"2024-05-08T08:54:36.914429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":415,"log_dups_size":0,"ondisk_log_size":415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:34:34.694953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089293900000000004,"stat_sum":{"num_bytes":18975285,"num_objects":83,"num_object_clones":10,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":51,"num_read":9211,"num_read_kb":34021,"num_write":364,"num_write_kb":9996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2b","length":"9"}]},{"pgid":"3.e","version":"125'550","reported_seq":8816,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.914782+0000","last_change":"2024-05-08T08:54:36.826081+0000","last_active":"2024-05-08T08:54:36.914782+0000","last_peered":"2024-05-08T08:54:36.914782+0000","last_clean":"2024-05-08T08:54:36.914782+0000","last_became_active":"2024-05-08T08:53:58.621584+0000","last_became_peered":"2024-05-08T08:53:58.621584+0000","last_unstale":"2024-05-08T08:54:36.914782+0000","last_undegraded":"2024-05-08T08:54:36.914782+0000","last_fullsized":"2024-05-08T08:54:36.914782+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":550,"log_dups_size":0,"ondisk_log_size":550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:56:25.857586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000891752,"stat_sum":{"num_bytes":27483200,"num_objects":70,"num_object_clones":14,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":34,"num_read":15911,"num_read_kb":46955,"num_write":583,"num_write_kb":12214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":188,"num_bytes_recovered":11952192,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2b","length":"9"}]},{"pgid":"3.f","version":"125'506","reported_seq":7632,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.836091+0000","last_change":"2024-05-08T08:54:36.836091+0000","last_active":"2024-05-08T08:54:36.836091+0000","last_peered":"2024-05-08T08:54:36.836091+0000","last_clean":"2024-05-08T08:54:36.836091+0000","last_became_active":"2024-05-08T08:53:58.615064+0000","last_became_peered":"2024-05-08T08:53:58.615064+0000","last_unstale":"2024-05-08T08:54:36.836091+0000","last_undegraded":"2024-05-08T08:54:36.836091+0000","last_fullsized":"2024-05-08T08:54:36.836091+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:43:36.481505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010320520000000001,"stat_sum":{"num_bytes":24172770,"num_objects":75,"num_object_clones":8,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":44,"num_read":8460,"num_read_kb":38018,"num_write":513,"num_write_kb":9621,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":107,"num_bytes_recovered":4068050,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2b","length":"9"}]},{"pgid":"3.10","version":"123'693","reported_seq":7967,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.825480+0000","last_change":"2024-05-08T08:54:36.826525+0000","last_active":"2024-05-08T08:54:36.825480+0000","last_peered":"2024-05-08T08:54:36.825480+0000","last_clean":"2024-05-08T08:54:36.825480+0000","last_became_active":"2024-05-08T08:53:58.618787+0000","last_became_peered":"2024-05-08T08:53:58.618787+0000","last_unstale":"2024-05-08T08:54:36.825480+0000","last_undegraded":"2024-05-08T08:54:36.825480+0000","last_fullsized":"2024-05-08T08:54:36.825480+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":693,"log_dups_size":0,"ondisk_log_size":693,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:05:11.277267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001293523,"stat_sum":{"num_bytes":32900689,"num_objects":83,"num_object_clones":13,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":51,"num_read":9053,"num_read_kb":41233,"num_write":952,"num_write_kb":14552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":112,"num_bytes_recovered":7552081,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2b","length":"9"}]},{"pgid":"3.11","version":"125'664","reported_seq":11366,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.834629+0000","last_change":"2024-05-08T08:54:36.834629+0000","last_active":"2024-05-08T08:54:36.834629+0000","last_peered":"2024-05-08T08:54:36.834629+0000","last_clean":"2024-05-08T08:54:36.834629+0000","last_became_active":"2024-05-08T08:53:24.479891+0000","last_became_peered":"2024-05-08T08:53:24.479891+0000","last_unstale":"2024-05-08T08:54:36.834629+0000","last_undegraded":"2024-05-08T08:54:36.834629+0000","last_fullsized":"2024-05-08T08:54:36.834629+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":664,"log_dups_size":0,"ondisk_log_size":664,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:42:24.013868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050629799999999997,"stat_sum":{"num_bytes":31926821,"num_objects":94,"num_object_clones":16,"num_object_copies":188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":57,"num_read":13002,"num_read_kb":36339,"num_write":537,"num_write_kb":10910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":131,"num_bytes_recovered":2604032,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2b","length":"9"}]},{"pgid":"3.12","version":"125'1429","reported_seq":24704,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.960646+0000","last_change":"2024-05-08T08:54:36.834580+0000","last_active":"2024-05-08T08:54:36.960646+0000","last_peered":"2024-05-08T08:54:36.960646+0000","last_clean":"2024-05-08T08:54:36.960646+0000","last_became_active":"2024-05-08T08:53:49.580557+0000","last_became_peered":"2024-05-08T08:53:49.580557+0000","last_unstale":"2024-05-08T08:54:36.960646+0000","last_undegraded":"2024-05-08T08:54:36.960646+0000","last_fullsized":"2024-05-08T08:54:36.960646+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":1429,"log_dups_size":0,"ondisk_log_size":1429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:57:47.121534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047675199999999999,"stat_sum":{"num_bytes":44968122,"num_objects":106,"num_object_clones":11,"num_object_copies":212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":106,"num_whiteouts":57,"num_read":30248,"num_read_kb":62116,"num_write":2020,"num_write_kb":12048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2b","length":"9"}]},{"pgid":"3.13","version":"123'552","reported_seq":10216,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.834953+0000","last_change":"2024-05-08T08:54:36.835139+0000","last_active":"2024-05-08T08:54:36.834953+0000","last_peered":"2024-05-08T08:54:36.834953+0000","last_clean":"2024-05-08T08:54:36.834953+0000","last_became_active":"2024-05-08T08:53:58.615947+0000","last_became_peered":"2024-05-08T08:53:58.615947+0000","last_unstale":"2024-05-08T08:54:36.834953+0000","last_undegraded":"2024-05-08T08:54:36.834953+0000","last_fullsized":"2024-05-08T08:54:36.834953+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":552,"log_dups_size":0,"ondisk_log_size":552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:16:16.764806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086118900000000005,"stat_sum":{"num_bytes":51989631,"num_objects":100,"num_object_clones":20,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":53,"num_read":10774,"num_read_kb":40880,"num_write":547,"num_write_kb":9065,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":130,"num_bytes_recovered":6657151,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2b","length":"9"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1152505644,"num_objects":2909,"num_object_clones":438,"num_object_copies":5818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2909,"num_whiteouts":1643,"num_read":501708,"num_read_kb":1618770,"num_write":23906,"num_write_kb":451334,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3104,"num_bytes_recovered":190761442,"num_keys_recovered":470,"num_objects_omap":43,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":88178688,"data_stored":737510251,"data_compressed":37851007,"data_compressed_allocated":72433664,"data_compressed_original":727909888,"omap_allocated":0,"internal_metadata":0},"log_size":21588,"ondisk_log_size":21588,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":11046,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":0,"seq":47244640290,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640293,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73068,"kb_used_data":15524,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94298772,"statfs":{"total":96636764160,"available":96561942528,"internally_reserved":0,"allocated":15896576,"data_stored":126394164,"data_compressed":6761617,"data_compressed_allocated":12402688,"data_compressed_original":124585984,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640292,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":91084,"kb_used_data":19628,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94280756,"statfs":{"total":96636764160,"available":96543494144,"internally_reserved":0,"allocated":20099072,"data_stored":149553879,"data_compressed":7726016,"data_compressed_allocated":14344192,"data_compressed_original":146203136,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640291,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46392,"kb_used_data":11052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94325448,"statfs":{"total":96636764160,"available":96589258752,"internally_reserved":0,"allocated":11317248,"data_stored":77138265,"data_compressed":4700015,"data_compressed_allocated":8282112,"data_compressed_original":75612672,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640290,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82408,"kb_used_data":19284,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94289432,"statfs":{"total":96636764160,"available":96552378368,"internally_reserved":0,"allocated":19746816,"data_stored":168967396,"data_compressed":7765083,"data_compressed_allocated":15867904,"data_compressed_original":166971904,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640293,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":84080,"kb_used_data":16792,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94287760,"statfs":{"total":96636764160,"available":96550666240,"internally_reserved":0,"allocated":17195008,"data_stored":121631259,"data_compressed":5996279,"data_compressed_allocated":11804672,"data_compressed_original":118602752,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5523,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5523,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":15650816,"data_stored":120421551,"data_compressed":5990756,"data_compressed_allocated":11767808,"data_compressed_original":118029312,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":18259968,"data_stored":168348056,"data_compressed":7765083,"data_compressed_allocated":15867904,"data_compressed_original":166971904,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":9830400,"data_stored":76518925,"data_compressed":4700015,"data_compressed_allocated":8282112,"data_compressed_original":75612672,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":18321408,"data_stored":147148316,"data_compressed":7613617,"data_compressed_allocated":14131200,"data_compressed_original":144469504,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":14409728,"data_stored":125774824,"data_compressed":6761617,"data_compressed_allocated":12402688,"data_compressed_original":124585984,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":11706368,"data_stored":99298579,"data_compressed":5019919,"data_compressed_allocated":9981952,"data_compressed_original":98240512,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T08:54:39.235 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T08:54:39.239 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:54:39.264 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:54:39.264 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:54:39.264 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:39.266+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:39.300 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:54:39.300 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:54:39.319 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:54:39.319 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:54:39.340 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:54:39.341 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:54:39.365 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:39.368+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:39.366 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:54:39.366 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:54:39.383 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:54:39.384 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:54:39.387 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:39.387 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:54:39.387 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:54:39.387 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:54:39.387 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:39.396 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:54:39.481 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:39.484+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:39.544 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:39.544 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:54:39.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:54:39.544 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:54:39.544 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.544 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22073 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 782336~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:39.544 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:54:39.544 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 131.68821645, 2024-05-08T08:54:39.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047244234000000003, 2024-05-08T08:54:39.544 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:39.544 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:39.544 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:39.545 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:39.545 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:39.545 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:39.545 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22073 2024-05-08T08:54:39.545 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.545 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:39.545 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.545 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:39.545 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:54:39.545 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:39.545 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.545 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.545 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:39.545 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:54:39.545 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:39.545 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.546 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.546 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:39.546 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857668+0000", 2024-05-08T08:54:39.546 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7800000000000002e-07 2024-05-08T08:54:39.546 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.546 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.546 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:39.546 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857670+0000", 2024-05-08T08:54:39.546 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0820000000000001e-06 2024-05-08T08:54:39.546 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.546 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.546 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:39.546 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857678+0000", 2024-05-08T08:54:39.546 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7589999999999997e-06 2024-05-08T08:54:39.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.547 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:39.547 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857691+0000", 2024-05-08T08:54:39.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3672999999999999e-05 2024-05-08T08:54:39.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.547 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:39.547 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904780+0000", 2024-05-08T08:54:39.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047088616999999999 2024-05-08T08:54:39.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.547 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:39.547 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904840+0000", 2024-05-08T08:54:39.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0356999999999998e-05 2024-05-08T08:54:39.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:39.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904912+0000", 2024-05-08T08:54:39.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1367999999999998e-05 2024-05-08T08:54:39.548 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:39.548 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:39.548 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:39.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.548 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22075 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 798720~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:39.548 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:54:39.548 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 131.68818605300001, 2024-05-08T08:54:39.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047384643999999997, 2024-05-08T08:54:39.548 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:39.549 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:39.549 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:39.549 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:39.549 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:39.549 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:39.549 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22075 2024-05-08T08:54:39.549 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.549 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:39.549 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.549 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:39.549 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:54:39.549 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:39.549 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.549 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.549 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:39.550 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:54:39.550 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:39.550 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.550 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.550 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:39.550 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:54:39.550 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7399999999999999e-07 2024-05-08T08:54:39.550 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.550 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.550 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:39.551 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857705+0000", 2024-05-08T08:54:39.551 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1980000000000004e-06 2024-05-08T08:54:39.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.551 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:39.551 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857713+0000", 2024-05-08T08:54:39.551 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7039999999999994e-06 2024-05-08T08:54:39.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.551 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:39.552 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857718+0000", 2024-05-08T08:54:39.552 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.865e-06 2024-05-08T08:54:39.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.552 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.552 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:39.552 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904941+0000", 2024-05-08T08:54:39.552 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047223198000000001 2024-05-08T08:54:39.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.552 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.552 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:39.552 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904999+0000", 2024-05-08T08:54:39.552 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8254999999999997e-05 2024-05-08T08:54:39.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.552 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.552 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:39.553 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905082+0000", 2024-05-08T08:54:39.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3049999999999999e-05 2024-05-08T08:54:39.553 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:39.553 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:39.553 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:39.553 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.553 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.553 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22077 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 815104~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:39.553 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:54:39.553 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 131.68814804199999, 2024-05-08T08:54:39.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047519448999999998, 2024-05-08T08:54:39.553 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:39.553 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:39.553 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:39.553 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:39.553 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:39.554 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:39.554 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22077 2024-05-08T08:54:39.554 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.554 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:39.554 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.554 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:39.554 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:54:39.554 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:39.554 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.554 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.554 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:39.554 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:54:39.554 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:39.554 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.554 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.555 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:39.555 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:54:39.555 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.99e-07 2024-05-08T08:54:39.555 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.555 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.555 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:39.555 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857739+0000", 2024-05-08T08:54:39.555 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3599999999999999e-06 2024-05-08T08:54:39.555 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.555 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.555 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:39.555 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857745+0000", 2024-05-08T08:54:39.555 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5139999999999998e-06 2024-05-08T08:54:39.555 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.556 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.556 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:39.556 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857750+0000", 2024-05-08T08:54:39.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8550000000000001e-06 2024-05-08T08:54:39.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.556 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.556 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:39.556 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905110+0000", 2024-05-08T08:54:39.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047360126000000002 2024-05-08T08:54:39.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.556 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.556 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:39.556 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905175+0000", 2024-05-08T08:54:39.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4689000000000004e-05 2024-05-08T08:54:39.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.556 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.556 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:39.557 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905255+0000", 2024-05-08T08:54:39.557 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0306e-05 2024-05-08T08:54:39.557 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:39.557 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:39.557 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:39.557 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.557 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.557 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22079 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 831488~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:39.557 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:54:39.557 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 131.68812749400001, 2024-05-08T08:54:39.557 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.04767499, 2024-05-08T08:54:39.557 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:39.557 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:39.557 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:39.557 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:39.558 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:39.558 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:39.558 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22079 2024-05-08T08:54:39.558 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.558 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:39.558 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.558 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:39.558 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:54:39.558 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:39.558 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.558 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.558 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:39.558 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:54:39.558 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:39.558 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.559 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.559 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:39.559 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857757+0000", 2024-05-08T08:54:39.559 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8299999999999997e-07 2024-05-08T08:54:39.559 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.559 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.559 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:39.559 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857759+0000", 2024-05-08T08:54:39.559 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1390000000000002e-06 2024-05-08T08:54:39.559 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.559 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.559 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:39.559 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857765+0000", 2024-05-08T08:54:39.559 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4609999999999996e-06 2024-05-08T08:54:39.559 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.559 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.560 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:39.560 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857770+0000", 2024-05-08T08:54:39.560 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6689999999999997e-06 2024-05-08T08:54:39.560 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.560 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.560 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:39.560 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905281+0000", 2024-05-08T08:54:39.560 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047510668999999998 2024-05-08T08:54:39.560 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.560 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.560 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:39.560 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905344+0000", 2024-05-08T08:54:39.560 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2797000000000002e-05 2024-05-08T08:54:39.560 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.560 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.560 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:39.561 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905431+0000", 2024-05-08T08:54:39.561 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7771999999999999e-05 2024-05-08T08:54:39.561 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:39.561 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:39.561 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:39.561 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.561 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.561 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22082 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 847872~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:39.561 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:54:39.561 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 131.68810780300001, 2024-05-08T08:54:39.561 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047831934999999999, 2024-05-08T08:54:39.561 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:39.561 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:39.561 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:39.562 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:39.562 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:39.562 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:39.562 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22082 2024-05-08T08:54:39.562 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.562 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:39.562 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.562 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:39.562 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:54:39.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:39.563 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.563 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.563 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:39.563 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:54:39.563 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:39.563 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.563 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.563 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:39.563 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:54:39.563 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:54:39.563 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.563 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.563 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:39.564 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857778+0000", 2024-05-08T08:54:39.564 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0789999999999999e-06 2024-05-08T08:54:39.564 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.564 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.564 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:39.564 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857784+0000", 2024-05-08T08:54:39.564 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0480000000000004e-06 2024-05-08T08:54:39.564 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.564 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.564 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:39.564 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857790+0000", 2024-05-08T08:54:39.564 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2719999999999997e-06 2024-05-08T08:54:39.564 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.564 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.564 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:39.565 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905458+0000", 2024-05-08T08:54:39.565 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047668470999999997 2024-05-08T08:54:39.565 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.565 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.565 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:39.565 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905521+0000", 2024-05-08T08:54:39.565 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2952000000000003e-05 2024-05-08T08:54:39.565 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.565 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.565 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:39.565 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905608+0000", 2024-05-08T08:54:39.565 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6759000000000001e-05 2024-05-08T08:54:39.565 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:39.565 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:39.565 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:39.565 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.566 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.566 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22084 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 864256~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:39.566 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:54:39.566 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 131.68808385299999, 2024-05-08T08:54:39.566 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047977748000000001, 2024-05-08T08:54:39.566 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:39.566 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:39.566 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:39.566 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:39.566 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:39.566 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:39.566 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22084 2024-05-08T08:54:39.566 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.566 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:39.567 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.567 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:39.567 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:54:39.567 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:39.567 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.567 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.567 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:39.567 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:54:39.567 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:39.567 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.567 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.567 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:39.567 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:54:39.567 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:54:39.567 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.567 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.567 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:39.568 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857802+0000", 2024-05-08T08:54:39.568 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0779999999999998e-06 2024-05-08T08:54:39.568 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.568 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.568 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:39.568 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857812+0000", 2024-05-08T08:54:39.568 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7939999999999997e-06 2024-05-08T08:54:39.568 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.568 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.568 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:39.568 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857817+0000", 2024-05-08T08:54:39.568 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8899999999999998e-06 2024-05-08T08:54:39.568 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.568 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.569 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:39.569 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905636+0000", 2024-05-08T08:54:39.569 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047819252 2024-05-08T08:54:39.569 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.569 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.569 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:39.569 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905700+0000", 2024-05-08T08:54:39.569 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4016999999999994e-05 2024-05-08T08:54:39.569 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.569 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.569 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:39.569 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905778+0000", 2024-05-08T08:54:39.569 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7306999999999997e-05 2024-05-08T08:54:39.570 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:39.570 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:39.570 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:39.570 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.570 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.570 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22086 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 880640~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:39.570 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:54:39.570 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 131.688060503, 2024-05-08T08:54:39.570 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048119149, 2024-05-08T08:54:39.570 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:39.570 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:39.571 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:39.571 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:39.571 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:39.571 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:39.571 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22086 2024-05-08T08:54:39.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.571 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:39.571 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.571 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:39.571 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:54:39.571 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:39.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.571 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.571 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:39.571 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:54:39.572 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:39.572 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.572 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.572 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:39.572 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857824+0000", 2024-05-08T08:54:39.572 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4200000000000002e-07 2024-05-08T08:54:39.572 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.572 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.572 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:39.572 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857826+0000", 2024-05-08T08:54:39.572 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0590000000000001e-06 2024-05-08T08:54:39.573 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.573 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.573 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:39.573 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857832+0000", 2024-05-08T08:54:39.573 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8029999999999999e-06 2024-05-08T08:54:39.573 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.573 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.573 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:39.573 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857837+0000", 2024-05-08T08:54:39.573 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7929999999999997e-06 2024-05-08T08:54:39.573 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.573 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.573 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:39.573 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905803+0000", 2024-05-08T08:54:39.573 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047965761000000003 2024-05-08T08:54:39.574 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.574 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.574 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:39.574 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905862+0000", 2024-05-08T08:54:39.574 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9203000000000002e-05 2024-05-08T08:54:39.574 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.574 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.574 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:39.574 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905942+0000", 2024-05-08T08:54:39.574 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0187999999999999e-05 2024-05-08T08:54:39.574 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:39.574 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:39.574 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:39.574 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.574 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.575 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22088 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 897024~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:39.575 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:54:39.575 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 131.68804045100001, 2024-05-08T08:54:39.575 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048256049000000002, 2024-05-08T08:54:39.575 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:39.575 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:39.575 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:39.575 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:39.575 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:39.575 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:39.575 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22088 2024-05-08T08:54:39.575 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.575 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:39.575 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.575 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:39.575 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:54:39.576 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:39.576 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.576 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.576 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:39.576 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:54:39.576 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:39.576 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.576 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.576 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:39.576 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857844+0000", 2024-05-08T08:54:39.576 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.46e-07 2024-05-08T08:54:39.576 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.576 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.576 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:39.576 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857846+0000", 2024-05-08T08:54:39.576 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0899999999999999e-06 2024-05-08T08:54:39.576 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.577 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.577 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:39.577 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857855+0000", 2024-05-08T08:54:39.577 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8219999999999996e-06 2024-05-08T08:54:39.577 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.577 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.577 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:39.577 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857860+0000", 2024-05-08T08:54:39.577 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2989999999999996e-06 2024-05-08T08:54:39.577 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.577 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.577 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:39.577 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905966+0000", 2024-05-08T08:54:39.577 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048106542000000002 2024-05-08T08:54:39.578 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.578 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.578 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:39.578 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906024+0000", 2024-05-08T08:54:39.578 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7958000000000003e-05 2024-05-08T08:54:39.578 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.578 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.578 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:39.578 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906099+0000", 2024-05-08T08:54:39.578 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4992000000000003e-05 2024-05-08T08:54:39.578 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:39.578 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:39.578 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:39.578 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.578 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.579 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22091 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 913408~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:39.579 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:54:39.579 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 131.688016691, 2024-05-08T08:54:39.579 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048395290000000001, 2024-05-08T08:54:39.579 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:39.579 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:39.579 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:39.579 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:39.579 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:39.579 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:39.579 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22091 2024-05-08T08:54:39.579 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.579 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:39.579 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.579 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:39.579 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:54:39.579 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:39.580 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.580 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.580 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:39.580 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:54:39.580 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:39.580 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.580 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.580 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:39.580 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:54:39.580 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4900000000000001e-07 2024-05-08T08:54:39.580 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.580 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.580 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:39.580 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857870+0000", 2024-05-08T08:54:39.580 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2510000000000001e-06 2024-05-08T08:54:39.580 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.581 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.581 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:39.581 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857877+0000", 2024-05-08T08:54:39.581 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0809999999999996e-06 2024-05-08T08:54:39.581 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.581 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.581 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:39.581 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857881+0000", 2024-05-08T08:54:39.581 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.724e-06 2024-05-08T08:54:39.581 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.581 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.581 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:39.581 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906127+0000", 2024-05-08T08:54:39.581 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048245345000000002 2024-05-08T08:54:39.581 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.581 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.581 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:39.582 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906190+0000", 2024-05-08T08:54:39.582 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2762000000000007e-05 2024-05-08T08:54:39.582 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.582 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.582 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:39.582 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906262+0000", 2024-05-08T08:54:39.582 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2778e-05 2024-05-08T08:54:39.582 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:39.582 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:39.582 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:39.582 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.582 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.582 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22093 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 929792~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:39.582 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:54:39.582 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 131.68799432700001, 2024-05-08T08:54:39.583 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048532875000000003, 2024-05-08T08:54:39.583 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:39.583 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:39.583 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:39.583 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:39.583 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:39.583 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:39.583 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22093 2024-05-08T08:54:39.583 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.583 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:39.583 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.584 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:39.584 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:54:39.584 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:39.584 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.584 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.584 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:39.584 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:54:39.584 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:39.584 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.584 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.584 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:39.584 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857890+0000", 2024-05-08T08:54:39.584 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-07 2024-05-08T08:54:39.584 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.584 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.585 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:39.585 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857899+0000", 2024-05-08T08:54:39.585 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6889999999999999e-06 2024-05-08T08:54:39.585 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.585 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.585 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:39.585 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857906+0000", 2024-05-08T08:54:39.585 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6490000000000002e-06 2024-05-08T08:54:39.585 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.585 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.585 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:39.585 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857917+0000", 2024-05-08T08:54:39.585 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0764999999999999e-05 2024-05-08T08:54:39.585 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.585 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.585 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:39.586 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906287+0000", 2024-05-08T08:54:39.586 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048370442 2024-05-08T08:54:39.586 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.586 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.586 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:39.586 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906349+0000", 2024-05-08T08:54:39.586 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1537000000000004e-05 2024-05-08T08:54:39.586 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.586 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.586 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:39.586 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906422+0000", 2024-05-08T08:54:39.586 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3431999999999998e-05 2024-05-08T08:54:39.586 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:39.586 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:39.586 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:39.586 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.587 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.587 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22095 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 946176~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:39.587 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:54:39.587 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 131.68795962300001, 2024-05-08T08:54:39.587 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048657593999999998, 2024-05-08T08:54:39.587 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:39.587 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:39.587 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:39.587 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:39.587 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:39.587 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:39.587 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22095 2024-05-08T08:54:39.587 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.587 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:39.587 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.587 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:39.587 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:54:39.588 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:39.588 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.588 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.588 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:39.588 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:54:39.588 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:39.588 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.588 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.588 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:39.588 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857925+0000", 2024-05-08T08:54:39.588 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3099999999999998e-07 2024-05-08T08:54:39.588 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.588 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.588 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:39.588 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857927+0000", 2024-05-08T08:54:39.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.34e-06 2024-05-08T08:54:39.589 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.589 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.589 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:39.589 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857938+0000", 2024-05-08T08:54:39.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1426e-05 2024-05-08T08:54:39.589 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.589 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.589 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:39.589 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857943+0000", 2024-05-08T08:54:39.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5140000000000003e-06 2024-05-08T08:54:39.589 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.589 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.589 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:39.589 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906448+0000", 2024-05-08T08:54:39.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048504641000000001 2024-05-08T08:54:39.590 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.590 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.590 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:39.590 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906510+0000", 2024-05-08T08:54:39.590 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2663000000000006e-05 2024-05-08T08:54:39.590 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.590 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.590 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:39.590 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906582+0000", 2024-05-08T08:54:39.590 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1478999999999998e-05 2024-05-08T08:54:39.590 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:39.590 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:39.590 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:39.590 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.590 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.590 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22097 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 962560~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:39.591 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:54:39.591 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 131.68793193400001, 2024-05-08T08:54:39.591 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048791353000000003, 2024-05-08T08:54:39.591 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:39.591 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:39.591 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:39.591 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:39.591 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:39.591 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:39.591 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22097 2024-05-08T08:54:39.591 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.591 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:39.592 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.592 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:39.592 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:54:39.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:39.592 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.592 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.592 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:39.592 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:54:39.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:39.592 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.592 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.592 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:39.592 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:54:39.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9200000000000002e-07 2024-05-08T08:54:39.592 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.592 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.592 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:39.593 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857954+0000", 2024-05-08T08:54:39.593 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1900000000000002e-06 2024-05-08T08:54:39.593 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.593 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.593 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:39.593 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857961+0000", 2024-05-08T08:54:39.593 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4989999999999999e-06 2024-05-08T08:54:39.593 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.593 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.593 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:39.593 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857971+0000", 2024-05-08T08:54:39.594 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0485999999999999e-05 2024-05-08T08:54:39.594 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.594 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.594 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:39.594 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906607+0000", 2024-05-08T08:54:39.594 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048635109000000003 2024-05-08T08:54:39.594 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.594 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.594 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:39.594 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906667+0000", 2024-05-08T08:54:39.594 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0439000000000002e-05 2024-05-08T08:54:39.594 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.594 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.595 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:39.595 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906743+0000", 2024-05-08T08:54:39.595 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6237999999999998e-05 2024-05-08T08:54:39.595 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:39.595 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:39.595 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:39.595 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.595 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.595 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22099 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 978944~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:39.595 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:54:39.595 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 131.687905774, 2024-05-08T08:54:39.595 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048932055000000002, 2024-05-08T08:54:39.595 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:39.595 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:39.595 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:39.595 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:39.596 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:39.596 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:39.596 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22099 2024-05-08T08:54:39.596 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.596 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:39.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.596 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:39.596 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:54:39.596 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:39.596 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.596 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:39.596 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:54:39.596 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:39.596 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.597 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:39.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:54:39.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.27e-07 2024-05-08T08:54:39.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.597 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:39.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857987+0000", 2024-05-08T08:54:39.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5890000000000003e-06 2024-05-08T08:54:39.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.597 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:39.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857994+0000", 2024-05-08T08:54:39.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6619999999999999e-06 2024-05-08T08:54:39.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:39.598 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857998+0000", 2024-05-08T08:54:39.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7110000000000003e-06 2024-05-08T08:54:39.598 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:39.598 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906772+0000", 2024-05-08T08:54:39.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048773415000000001 2024-05-08T08:54:39.598 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:39.598 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906834+0000", 2024-05-08T08:54:39.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1752000000000001e-05 2024-05-08T08:54:39.598 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.599 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:39.599 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906910+0000", 2024-05-08T08:54:39.599 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6499000000000001e-05 2024-05-08T08:54:39.599 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:39.599 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:39.599 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:39.599 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.599 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.599 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22101 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 995328~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:39.599 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:54:39.599 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 131.687878643, 2024-05-08T08:54:39.599 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049087364000000001, 2024-05-08T08:54:39.599 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:39.599 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:39.599 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:39.600 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:39.600 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:39.600 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:39.600 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22101 2024-05-08T08:54:39.600 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.600 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:39.600 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.600 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:39.600 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:54:39.600 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:39.600 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.600 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.600 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:39.600 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:54:39.600 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:39.600 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.601 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:39.601 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858006+0000", 2024-05-08T08:54:39.601 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1399999999999997e-07 2024-05-08T08:54:39.601 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.601 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:39.601 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858008+0000", 2024-05-08T08:54:39.601 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2060000000000001e-06 2024-05-08T08:54:39.601 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.601 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:39.601 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858017+0000", 2024-05-08T08:54:39.601 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1579999999999994e-06 2024-05-08T08:54:39.601 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.601 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:39.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858022+0000", 2024-05-08T08:54:39.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4199999999999998e-06 2024-05-08T08:54:39.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:39.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906940+0000", 2024-05-08T08:54:39.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048917627999999998 2024-05-08T08:54:39.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:39.603 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907002+0000", 2024-05-08T08:54:39.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2399999999999999e-05 2024-05-08T08:54:39.603 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.603 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:39.603 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907092+0000", 2024-05-08T08:54:39.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0137999999999997e-05 2024-05-08T08:54:39.603 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:39.603 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:39.603 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:39.603 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.603 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22103 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1011712~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:39.603 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:54:39.604 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 131.68785385800001, 2024-05-08T08:54:39.604 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049228542, 2024-05-08T08:54:39.604 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:39.604 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:39.604 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:39.604 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:39.604 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:39.604 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:39.604 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22103 2024-05-08T08:54:39.604 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.605 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:39.605 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.605 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:39.605 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:54:39.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:39.605 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.605 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.605 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:39.605 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:54:39.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:39.605 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.605 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.605 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:39.605 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:54:39.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4299999999999999e-07 2024-05-08T08:54:39.605 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.605 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.606 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:39.606 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858032+0000", 2024-05-08T08:54:39.606 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0590000000000001e-06 2024-05-08T08:54:39.606 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.606 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:39.606 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858037+0000", 2024-05-08T08:54:39.606 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.977e-06 2024-05-08T08:54:39.606 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.606 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:39.606 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858042+0000", 2024-05-08T08:54:39.606 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7650000000000001e-06 2024-05-08T08:54:39.606 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.607 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:39.607 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907118+0000", 2024-05-08T08:54:39.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049076147000000001 2024-05-08T08:54:39.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.607 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:39.607 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907180+0000", 2024-05-08T08:54:39.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1971999999999996e-05 2024-05-08T08:54:39.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.607 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:39.607 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907258+0000", 2024-05-08T08:54:39.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8279e-05 2024-05-08T08:54:39.607 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:39.607 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:39.607 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:39.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.608 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22105 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1028096~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:39.608 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:54:39.608 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 131.68783461500001, 2024-05-08T08:54:39.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049394609999999999, 2024-05-08T08:54:39.608 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:39.608 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:39.608 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:39.608 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:39.608 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:39.608 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:39.608 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22105 2024-05-08T08:54:39.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.608 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:39.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.608 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:39.608 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:54:39.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:39.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.609 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:39.609 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:54:39.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:39.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.609 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:39.609 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858050+0000", 2024-05-08T08:54:39.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5999999999999999e-07 2024-05-08T08:54:39.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.609 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:39.609 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858051+0000", 2024-05-08T08:54:39.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8870000000000001e-06 2024-05-08T08:54:39.610 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.610 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:39.610 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858057+0000", 2024-05-08T08:54:39.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8019999999999997e-06 2024-05-08T08:54:39.610 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.610 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:39.610 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858062+0000", 2024-05-08T08:54:39.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.561e-06 2024-05-08T08:54:39.610 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.610 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:39.610 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907289+0000", 2024-05-08T08:54:39.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049226957000000002 2024-05-08T08:54:39.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.611 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:39.611 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907356+0000", 2024-05-08T08:54:39.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6915000000000006e-05 2024-05-08T08:54:39.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.611 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:39.611 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907444+0000", 2024-05-08T08:54:39.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8127999999999994e-05 2024-05-08T08:54:39.611 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:39.611 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:39.611 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:39.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.611 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22107 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1044480~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:39.612 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:54:39.612 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 131.68781627499999, 2024-05-08T08:54:39.612 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049799942999999999, 2024-05-08T08:54:39.612 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:39.612 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:39.612 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:39.612 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:39.612 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:39.612 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:39.612 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22107 2024-05-08T08:54:39.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.612 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:39.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.612 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:39.612 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:54:39.612 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:39.613 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.613 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.613 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:39.613 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:54:39.613 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:39.613 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.613 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.613 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:39.613 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:54:39.613 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4799999999999999e-07 2024-05-08T08:54:39.613 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.613 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.613 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:39.613 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858070+0000", 2024-05-08T08:54:39.613 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.012e-06 2024-05-08T08:54:39.613 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.614 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.614 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:39.614 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858077+0000", 2024-05-08T08:54:39.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6290000000000001e-06 2024-05-08T08:54:39.614 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.614 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.614 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:39.614 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858083+0000", 2024-05-08T08:54:39.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8880000000000002e-06 2024-05-08T08:54:39.614 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.614 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.614 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:39.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907671+0000", 2024-05-08T08:54:39.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049587191000000003 2024-05-08T08:54:39.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:39.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907763+0000", 2024-05-08T08:54:39.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2231000000000005e-05 2024-05-08T08:54:39.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:39.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907867+0000", 2024-05-08T08:54:39.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104644 2024-05-08T08:54:39.615 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:39.615 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:39.615 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:39.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.616 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22294 3.1c 3:3aad0389:::rbd_data.11406886b07b.0000000000000080:head [stat,delete] snapc 7=[7] RETRY=6 ondisk+retry+write+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:39.616 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:54:39.616 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 131.68722242699999, 2024-05-08T08:54:39.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051152191, 2024-05-08T08:54:39.616 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:39.616 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:39.616 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:39.616 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:39.616 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:39.616 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:39.616 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22294 2024-05-08T08:54:39.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.616 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:39.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:39.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:54:39.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:39.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.617 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.617 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:39.617 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:54:39.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:39.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.617 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.617 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:39.617 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858662+0000", 2024-05-08T08:54:39.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4799999999999998e-07 2024-05-08T08:54:39.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.617 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.617 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:39.617 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858664+0000", 2024-05-08T08:54:39.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1229999999999998e-06 2024-05-08T08:54:39.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.618 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.618 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:39.618 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858670+0000", 2024-05-08T08:54:39.618 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3099999999999997e-06 2024-05-08T08:54:39.618 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.618 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.618 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:39.618 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858676+0000", 2024-05-08T08:54:39.618 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9750000000000004e-06 2024-05-08T08:54:39.618 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.618 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.618 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:39.618 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907901+0000", 2024-05-08T08:54:39.618 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049224520000000001 2024-05-08T08:54:39.618 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.618 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.619 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:39.619 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.908085+0000", 2024-05-08T08:54:39.619 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00018377500000000001 2024-05-08T08:54:39.619 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.619 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.619 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:39.619 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.909814+0000", 2024-05-08T08:54:39.619 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00172894 2024-05-08T08:54:39.619 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:39.619 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:39.619 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:39.619 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.619 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.619 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4416.0:13461 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-05-08T08:54:39.619 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:22.487456+0000", 2024-05-08T08:54:39.619 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 77.058427386000005, 2024-05-08T08:54:39.620 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.99102620799999996, 2024-05-08T08:54:39.620 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:39.620 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:39.620 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:39.620 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:39.620 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4416", 2024-05-08T08:54:39.620 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:54:39.620 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 13461 2024-05-08T08:54:39.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.620 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:39.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.620 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:39.620 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487456+0000", 2024-05-08T08:54:39.620 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:39.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:39.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487456+0000", 2024-05-08T08:54:39.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:39.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:39.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487458+0000", 2024-05-08T08:54:39.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5659999999999999e-06 2024-05-08T08:54:39.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:39.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487463+0000", 2024-05-08T08:54:39.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6890000000000003e-06 2024-05-08T08:54:39.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:39.622 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487472+0000", 2024-05-08T08:54:39.622 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7710000000000001e-06 2024-05-08T08:54:39.622 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.622 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.622 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:39.622 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487478+0000", 2024-05-08T08:54:39.622 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1390000000000003e-06 2024-05-08T08:54:39.622 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.622 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.622 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:39.622 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487520+0000", 2024-05-08T08:54:39.622 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2259999999999999e-05 2024-05-08T08:54:39.622 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.623 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.623 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-05-08T08:54:39.623 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487528+0000", 2024-05-08T08:54:39.623 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7479999999999993e-06 2024-05-08T08:54:39.623 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.623 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.623 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:39.623 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:23.478139+0000", 2024-05-08T08:54:39.623 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.99061165200000001 2024-05-08T08:54:39.623 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.623 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.623 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:39.623 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:23.478301+0000", 2024-05-08T08:54:39.623 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000161602 2024-05-08T08:54:39.623 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.623 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.624 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:39.624 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:23.478483+0000", 2024-05-08T08:54:39.624 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000181781 2024-05-08T08:54:39.624 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:39.624 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:39.624 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:39.624 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.624 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5752.0:404 3.1a 3:5f5e8104:::rbd_data.1140bc4a55c7.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 5175296~45056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-05-08T08:54:39.624 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:00.390863+0000", 2024-05-08T08:54:39.624 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 39.155021191000003, 2024-05-08T08:54:39.624 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.055189174000000001, 2024-05-08T08:54:39.624 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:39.624 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:39.624 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:54:39.625 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:39.625 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5752", 2024-05-08T08:54:39.625 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/3534419815", 2024-05-08T08:54:39.625 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 404 2024-05-08T08:54:39.625 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.625 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:39.625 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.625 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:39.625 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.390863+0000", 2024-05-08T08:54:39.625 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:39.625 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.625 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.625 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:39.625 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.390863+0000", 2024-05-08T08:54:39.626 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:39.626 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.626 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.626 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:39.626 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.390864+0000", 2024-05-08T08:54:39.626 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.471e-06 2024-05-08T08:54:39.626 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.626 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.626 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:39.626 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397475+0000", 2024-05-08T08:54:39.626 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0066109109999999997 2024-05-08T08:54:39.626 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.626 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.626 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:39.626 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397498+0000", 2024-05-08T08:54:39.626 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2674e-05 2024-05-08T08:54:39.626 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.627 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.627 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:39.627 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397508+0000", 2024-05-08T08:54:39.627 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9019999999999994e-06 2024-05-08T08:54:39.627 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.627 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.627 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:39.627 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397609+0000", 2024-05-08T08:54:39.627 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010121899999999999 2024-05-08T08:54:39.627 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.627 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.627 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:39.627 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397711+0000", 2024-05-08T08:54:39.627 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102489 2024-05-08T08:54:39.627 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.627 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.628 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-05-08T08:54:39.628 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397999+0000", 2024-05-08T08:54:39.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00028733799999999999 2024-05-08T08:54:39.628 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.628 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:54:39.628 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.432093+0000", 2024-05-08T08:54:39.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.034094865000000002 2024-05-08T08:54:39.628 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.628 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:54:39.628 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.445975+0000", 2024-05-08T08:54:39.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013881187 2024-05-08T08:54:39.628 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.628 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:54:39.628 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.446013+0000", 2024-05-08T08:54:39.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8155000000000003e-05 2024-05-08T08:54:39.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:39.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:39.629 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:39.629 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.446052+0000", 2024-05-08T08:54:39.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8962999999999999e-05 2024-05-08T08:54:39.629 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:39.629 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:39.629 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:39.629 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:39.629 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:39.629 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:39.631 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:54:39.632 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-05-08T08:54:39.632 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:54:39.633 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":202,"stamp":"2024-05-08T08:54:38.838683+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1153096012,"num_objects":2911,"num_object_clones":438,"num_object_copies":5822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2911,"num_whiteouts":1643,"num_read":501824,"num_read_kb":1618870,"num_write":24065,"num_write_kb":453750,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3104,"num_bytes_recovered":190761442,"num_keys_recovered":470,"num_objects_omap":43,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21705,"ondisk_log_size":21705,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":471859200,"kb_used":377032,"kb_used_data":82280,"kb_used_omap":7,"kb_used_meta":294648,"kb_avail":471482168,"statfs":{"total":483183820800,"available":482797740032,"internally_reserved":0,"allocated":84254720,"data_stored":643684963,"data_compressed":32949010,"data_compressed_allocated":62701568,"data_compressed_original":631976448,"omap_allocated":7973,"internal_metadata":301719771},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-205841523,"num_objects":-121,"num_object_clones":-17,"num_object_copies":-242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-121,"num_whiteouts":-6,"num_read":1198,"num_read_kb":1026,"num_write":276,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.032230"},"pg_stats":[{"pgid":"3.1f","version":"124'651","reported_seq":19951,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.835655+0000","last_change":"2024-05-08T08:54:36.835854+0000","last_active":"2024-05-08T08:54:36.835655+0000","last_peered":"2024-05-08T08:54:36.835655+0000","last_clean":"2024-05-08T08:54:36.835655+0000","last_became_active":"2024-05-08T08:53:49.580563+0000","last_became_peered":"2024-05-08T08:53:49.580563+0000","last_unstale":"2024-05-08T08:54:36.835655+0000","last_undegraded":"2024-05-08T08:54:36.835655+0000","last_fullsized":"2024-05-08T08:54:36.835655+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":651,"log_dups_size":0,"ondisk_log_size":651,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:03:44.387830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000879771,"stat_sum":{"num_bytes":64664935,"num_objects":95,"num_object_clones":17,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":45,"num_read":19059,"num_read_kb":85554,"num_write":900,"num_write_kb":33084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2b","length":"9"}]},{"pgid":"3.1e","version":"125'805","reported_seq":11277,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.959741+0000","last_change":"2024-05-08T08:54:36.827994+0000","last_active":"2024-05-08T08:54:36.959741+0000","last_peered":"2024-05-08T08:54:36.959741+0000","last_clean":"2024-05-08T08:54:36.959741+0000","last_became_active":"2024-05-08T08:52:11.930844+0000","last_became_peered":"2024-05-08T08:52:11.930844+0000","last_unstale":"2024-05-08T08:54:36.959741+0000","last_undegraded":"2024-05-08T08:54:36.959741+0000","last_fullsized":"2024-05-08T08:54:36.959741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":805,"log_dups_size":0,"ondisk_log_size":805,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:37:20.288832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001925115,"stat_sum":{"num_bytes":20532352,"num_objects":94,"num_object_clones":12,"num_object_copies":188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":60,"num_read":13187,"num_read_kb":36671,"num_write":617,"num_write_kb":12702,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2b","length":"9"}]},{"pgid":"3.1d","version":"125'666","reported_seq":4010,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.885839+0000","last_change":"2024-05-08T08:54:36.825851+0000","last_active":"2024-05-08T08:54:36.885839+0000","last_peered":"2024-05-08T08:54:36.885839+0000","last_clean":"2024-05-08T08:54:36.885839+0000","last_became_active":"2024-05-08T08:53:59.618085+0000","last_became_peered":"2024-05-08T08:53:59.618085+0000","last_unstale":"2024-05-08T08:54:36.885839+0000","last_undegraded":"2024-05-08T08:54:36.885839+0000","last_fullsized":"2024-05-08T08:54:36.885839+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":666,"log_dups_size":0,"ondisk_log_size":666,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:23:12.297537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072215199999999999,"stat_sum":{"num_bytes":14415630,"num_objects":83,"num_object_clones":11,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":47,"num_read":8201,"num_read_kb":30455,"num_write":835,"num_write_kb":7273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":141,"num_bytes_recovered":5594929,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2b","length":"9"}]},{"pgid":"3.1c","version":"125'596","reported_seq":18144,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.887393+0000","last_change":"2024-05-08T08:54:36.826322+0000","last_active":"2024-05-08T08:54:36.887393+0000","last_peered":"2024-05-08T08:54:36.887393+0000","last_clean":"2024-05-08T08:54:36.887393+0000","last_became_active":"2024-05-08T08:53:58.621483+0000","last_became_peered":"2024-05-08T08:53:58.621483+0000","last_unstale":"2024-05-08T08:54:36.887393+0000","last_undegraded":"2024-05-08T08:54:36.887393+0000","last_fullsized":"2024-05-08T08:54:36.887393+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":596,"log_dups_size":0,"ondisk_log_size":596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:00:24.021853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094490299999999998,"stat_sum":{"num_bytes":55879221,"num_objects":95,"num_object_clones":11,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":54,"num_read":17976,"num_read_kb":100941,"num_write":657,"num_write_kb":27844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":287,"num_bytes_recovered":49590415,"num_keys_recovered":166,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2b","length":"9"}]},{"pgid":"3.1b","version":"124'553","reported_seq":25224,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.825428+0000","last_change":"2024-05-08T08:54:36.825629+0000","last_active":"2024-05-08T08:54:36.825428+0000","last_peered":"2024-05-08T08:54:36.825428+0000","last_clean":"2024-05-08T08:54:36.825428+0000","last_became_active":"2024-05-08T08:53:49.583097+0000","last_became_peered":"2024-05-08T08:53:49.583097+0000","last_unstale":"2024-05-08T08:54:36.825428+0000","last_undegraded":"2024-05-08T08:54:36.825428+0000","last_fullsized":"2024-05-08T08:54:36.825428+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":553,"log_dups_size":0,"ondisk_log_size":553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:24:45.532150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055716000000000001,"stat_sum":{"num_bytes":47790336,"num_objects":100,"num_object_clones":13,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":62,"num_read":24904,"num_read_kb":45652,"num_write":634,"num_write_kb":23301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":129,"num_bytes_recovered":3090520,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2b","length":"9"}]},{"pgid":"3.1a","version":"125'760","reported_seq":21427,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.943653+0000","last_change":"2024-05-08T08:54:36.826046+0000","last_active":"2024-05-08T08:54:36.943653+0000","last_peered":"2024-05-08T08:54:36.943653+0000","last_clean":"2024-05-08T08:54:36.943653+0000","last_became_active":"2024-05-08T08:53:24.484124+0000","last_became_peered":"2024-05-08T08:53:24.484124+0000","last_unstale":"2024-05-08T08:54:36.943653+0000","last_undegraded":"2024-05-08T08:54:36.943653+0000","last_fullsized":"2024-05-08T08:54:36.943653+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":760,"log_dups_size":0,"ondisk_log_size":760,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:50:59.189663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080935299999999996,"stat_sum":{"num_bytes":27794812,"num_objects":95,"num_object_clones":16,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":47,"num_read":23962,"num_read_kb":59334,"num_write":840,"num_write_kb":19082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":130,"num_bytes_recovered":8549042,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2b","length":"9"}]},{"pgid":"3.19","version":"124'380","reported_seq":10352,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.825750+0000","last_change":"2024-05-08T08:54:36.825750+0000","last_active":"2024-05-08T08:54:36.825750+0000","last_peered":"2024-05-08T08:54:36.825750+0000","last_clean":"2024-05-08T08:54:36.825750+0000","last_became_active":"2024-05-08T08:53:24.483403+0000","last_became_peered":"2024-05-08T08:53:24.483403+0000","last_unstale":"2024-05-08T08:54:36.825750+0000","last_undegraded":"2024-05-08T08:54:36.825750+0000","last_fullsized":"2024-05-08T08:54:36.825750+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":380,"log_dups_size":0,"ondisk_log_size":380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:22:50.312657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054762300000000001,"stat_sum":{"num_bytes":20318328,"num_objects":90,"num_object_clones":12,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":56,"num_read":9401,"num_read_kb":48041,"num_write":346,"num_write_kb":16514,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":8814624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2b","length":"9"}]},{"pgid":"3.18","version":"124'638","reported_seq":19227,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.826645+0000","last_change":"2024-05-08T08:54:36.826645+0000","last_active":"2024-05-08T08:54:36.826645+0000","last_peered":"2024-05-08T08:54:36.826645+0000","last_clean":"2024-05-08T08:54:36.826645+0000","last_became_active":"2024-05-08T08:52:11.927366+0000","last_became_peered":"2024-05-08T08:52:11.927366+0000","last_unstale":"2024-05-08T08:54:36.826645+0000","last_undegraded":"2024-05-08T08:54:36.826645+0000","last_fullsized":"2024-05-08T08:54:36.826645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":638,"log_dups_size":0,"ondisk_log_size":638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:06:16.899829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012129930000000001,"stat_sum":{"num_bytes":80472280,"num_objects":100,"num_object_clones":19,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":57,"num_read":20615,"num_read_kb":52536,"num_write":636,"num_write_kb":23374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2b","length":"9"}]},{"pgid":"3.17","version":"124'753","reported_seq":18455,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.826705+0000","last_change":"2024-05-08T08:54:36.827112+0000","last_active":"2024-05-08T08:54:36.826705+0000","last_peered":"2024-05-08T08:54:36.826705+0000","last_clean":"2024-05-08T08:54:36.826705+0000","last_became_active":"2024-05-08T08:53:11.358391+0000","last_became_peered":"2024-05-08T08:53:11.358391+0000","last_unstale":"2024-05-08T08:54:36.826705+0000","last_undegraded":"2024-05-08T08:54:36.826705+0000","last_fullsized":"2024-05-08T08:54:36.826705+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":753,"log_dups_size":0,"ondisk_log_size":753,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:15:31.272261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001337396,"stat_sum":{"num_bytes":58717330,"num_objects":82,"num_object_clones":13,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":39,"num_read":20201,"num_read_kb":76886,"num_write":897,"num_write_kb":16919,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":97,"num_bytes_recovered":2508994,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2b","length":"9"}]},{"pgid":"3.16","version":"125'1357","reported_seq":21214,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.957927+0000","last_change":"2024-05-08T08:54:36.835627+0000","last_active":"2024-05-08T08:54:36.957927+0000","last_peered":"2024-05-08T08:54:36.957927+0000","last_clean":"2024-05-08T08:54:36.957927+0000","last_became_active":"2024-05-08T08:53:59.615917+0000","last_became_peered":"2024-05-08T08:53:59.615917+0000","last_unstale":"2024-05-08T08:54:36.957927+0000","last_undegraded":"2024-05-08T08:54:36.957927+0000","last_fullsized":"2024-05-08T08:54:36.957927+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":1357,"log_dups_size":0,"ondisk_log_size":1357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:13:12.400754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012988489999999999,"stat_sum":{"num_bytes":40021352,"num_objects":83,"num_object_clones":12,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":42,"num_read":25230,"num_read_kb":87053,"num_write":2178,"num_write_kb":27176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":118,"num_bytes_recovered":21349055,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2b","length":"9"}]},{"pgid":"3.15","version":"125'603","reported_seq":16017,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.916428+0000","last_change":"2024-05-08T08:54:36.835546+0000","last_active":"2024-05-08T08:54:36.916428+0000","last_peered":"2024-05-08T08:54:36.916428+0000","last_clean":"2024-05-08T08:54:36.916428+0000","last_became_active":"2024-05-08T08:52:11.931695+0000","last_became_peered":"2024-05-08T08:52:11.931695+0000","last_unstale":"2024-05-08T08:54:36.916428+0000","last_undegraded":"2024-05-08T08:54:36.916428+0000","last_fullsized":"2024-05-08T08:54:36.916428+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:22:50.380548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089158499999999997,"stat_sum":{"num_bytes":25074782,"num_objects":83,"num_object_clones":11,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":50,"num_read":17145,"num_read_kb":52692,"num_write":715,"num_write_kb":8746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2b","length":"9"}]},{"pgid":"3.14","version":"124'625","reported_seq":12155,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.834704+0000","last_change":"2024-05-08T08:54:36.834881+0000","last_active":"2024-05-08T08:54:36.834704+0000","last_peered":"2024-05-08T08:54:36.834704+0000","last_clean":"2024-05-08T08:54:36.834704+0000","last_became_active":"2024-05-08T08:53:23.481362+0000","last_became_peered":"2024-05-08T08:53:23.481362+0000","last_unstale":"2024-05-08T08:54:36.834704+0000","last_undegraded":"2024-05-08T08:54:36.834704+0000","last_fullsized":"2024-05-08T08:54:36.834704+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":625,"log_dups_size":0,"ondisk_log_size":625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:06:33.760998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066539699999999997,"stat_sum":{"num_bytes":35524723,"num_objects":82,"num_object_clones":16,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":43,"num_read":10444,"num_read_kb":32909,"num_write":595,"num_write_kb":11437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":3612768,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2b","length":"9"}]},{"pgid":"2.7","version":"0'0","reported_seq":238,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.835108+0000","last_change":"2024-05-08T08:53:20.362387+0000","last_active":"2024-05-08T08:54:36.835108+0000","last_peered":"2024-05-08T08:54:36.835108+0000","last_clean":"2024-05-08T08:54:36.835108+0000","last_became_active":"2024-05-08T08:53:20.362113+0000","last_became_peered":"2024-05-08T08:53:20.362113+0000","last_unstale":"2024-05-08T08:54:36.835108+0000","last_undegraded":"2024-05-08T08:54:36.835108+0000","last_fullsized":"2024-05-08T08:54:36.835108+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:28:25.204125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"125'528","reported_seq":19653,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.910281+0000","last_change":"2024-05-08T08:54:36.827679+0000","last_active":"2024-05-08T08:54:36.910281+0000","last_peered":"2024-05-08T08:54:36.910281+0000","last_clean":"2024-05-08T08:54:36.910281+0000","last_became_active":"2024-05-08T08:53:49.579888+0000","last_became_peered":"2024-05-08T08:53:49.579888+0000","last_unstale":"2024-05-08T08:54:36.910281+0000","last_undegraded":"2024-05-08T08:54:36.910281+0000","last_fullsized":"2024-05-08T08:54:36.910281+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":528,"log_dups_size":0,"ondisk_log_size":528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:02:37.637195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001567492,"stat_sum":{"num_bytes":40662233,"num_objects":81,"num_object_clones":9,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":48,"num_read":21227,"num_read_kb":88195,"num_write":524,"num_write_kb":17531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2b","length":"9"}]},{"pgid":"2.6","version":"0'0","reported_seq":229,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.826254+0000","last_change":"2024-05-08T08:52:03.896159+0000","last_active":"2024-05-08T08:54:36.826254+0000","last_peered":"2024-05-08T08:54:36.826254+0000","last_clean":"2024-05-08T08:54:36.826254+0000","last_became_active":"2024-05-08T08:52:03.895797+0000","last_became_peered":"2024-05-08T08:52:03.895797+0000","last_unstale":"2024-05-08T08:54:36.826254+0000","last_undegraded":"2024-05-08T08:54:36.826254+0000","last_fullsized":"2024-05-08T08:54:36.826254+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:34:33.513525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"125'433","reported_seq":6399,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.826275+0000","last_change":"2024-05-08T08:54:36.826275+0000","last_active":"2024-05-08T08:54:36.826275+0000","last_peered":"2024-05-08T08:54:36.826275+0000","last_clean":"2024-05-08T08:54:36.826275+0000","last_became_active":"2024-05-08T08:54:32.801917+0000","last_became_peered":"2024-05-08T08:54:32.801917+0000","last_unstale":"2024-05-08T08:54:36.826275+0000","last_undegraded":"2024-05-08T08:54:36.826275+0000","last_fullsized":"2024-05-08T08:54:36.826275+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:08:55.683221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096415599999999998,"stat_sum":{"num_bytes":15654049,"num_objects":95,"num_object_clones":17,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":50,"num_read":6596,"num_read_kb":34394,"num_write":375,"num_write_kb":12018,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":162,"num_bytes_recovered":8401073,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2b","length":"9"}]},{"pgid":"2.5","version":"0'0","reported_seq":228,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.834873+0000","last_change":"2024-05-08T08:52:04.144764+0000","last_active":"2024-05-08T08:54:36.834873+0000","last_peered":"2024-05-08T08:54:36.834873+0000","last_clean":"2024-05-08T08:54:36.834873+0000","last_became_active":"2024-05-08T08:52:04.144444+0000","last_became_peered":"2024-05-08T08:52:04.144444+0000","last_unstale":"2024-05-08T08:54:36.834873+0000","last_undegraded":"2024-05-08T08:54:36.834873+0000","last_fullsized":"2024-05-08T08:54:36.834873+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:58:31.498922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"124'728","reported_seq":25659,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.938664+0000","last_change":"2024-05-08T08:54:36.827387+0000","last_active":"2024-05-08T08:54:36.938664+0000","last_peered":"2024-05-08T08:54:36.938664+0000","last_clean":"2024-05-08T08:54:36.938664+0000","last_became_active":"2024-05-08T08:53:58.618388+0000","last_became_peered":"2024-05-08T08:53:58.618388+0000","last_unstale":"2024-05-08T08:54:36.938664+0000","last_undegraded":"2024-05-08T08:54:36.938664+0000","last_fullsized":"2024-05-08T08:54:36.938664+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":728,"log_dups_size":0,"ondisk_log_size":728,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:53:00.560684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001856866,"stat_sum":{"num_bytes":49267216,"num_objects":115,"num_object_clones":19,"num_object_copies":230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":115,"num_whiteouts":63,"num_read":21930,"num_read_kb":51626,"num_write":572,"num_write_kb":12044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":215,"num_bytes_recovered":10925568,"num_keys_recovered":44,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2b","length":"9"}]},{"pgid":"2.4","version":"0'0","reported_seq":229,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.825458+0000","last_change":"2024-05-08T08:52:04.144543+0000","last_active":"2024-05-08T08:54:36.825458+0000","last_peered":"2024-05-08T08:54:36.825458+0000","last_clean":"2024-05-08T08:54:36.825458+0000","last_became_active":"2024-05-08T08:52:04.144242+0000","last_became_peered":"2024-05-08T08:52:04.144242+0000","last_unstale":"2024-05-08T08:54:36.825458+0000","last_undegraded":"2024-05-08T08:54:36.825458+0000","last_fullsized":"2024-05-08T08:54:36.825458+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:57:36.844633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"125'863","reported_seq":13601,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.828032+0000","last_change":"2024-05-08T08:54:36.828032+0000","last_active":"2024-05-08T08:54:36.828032+0000","last_peered":"2024-05-08T08:54:36.828032+0000","last_clean":"2024-05-08T08:54:36.828032+0000","last_became_active":"2024-05-08T08:53:58.615614+0000","last_became_peered":"2024-05-08T08:53:58.615614+0000","last_unstale":"2024-05-08T08:54:36.828032+0000","last_undegraded":"2024-05-08T08:54:36.828032+0000","last_fullsized":"2024-05-08T08:54:36.828032+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":863,"log_dups_size":0,"ondisk_log_size":863,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:57:46.227211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019662109999999998,"stat_sum":{"num_bytes":16686331,"num_objects":103,"num_object_clones":15,"num_object_copies":206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103,"num_whiteouts":67,"num_read":15031,"num_read_kb":32630,"num_write":1091,"num_write_kb":10601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":142,"num_bytes_recovered":4609275,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2b","length":"9"}]},{"pgid":"3.3","version":"125'567","reported_seq":10885,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.935607+0000","last_change":"2024-05-08T08:54:36.826519+0000","last_active":"2024-05-08T08:54:36.935607+0000","last_peered":"2024-05-08T08:54:36.935607+0000","last_clean":"2024-05-08T08:54:36.935607+0000","last_became_active":"2024-05-08T08:53:24.480423+0000","last_became_peered":"2024-05-08T08:53:24.480423+0000","last_unstale":"2024-05-08T08:54:36.935607+0000","last_undegraded":"2024-05-08T08:54:36.935607+0000","last_fullsized":"2024-05-08T08:54:36.935607+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":567,"log_dups_size":0,"ondisk_log_size":567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:40:16.111591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001006863,"stat_sum":{"num_bytes":30593870,"num_objects":80,"num_object_clones":13,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":45,"num_read":11335,"num_read_kb":33106,"num_write":644,"num_write_kb":6500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":101,"num_bytes_recovered":5689443,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2b","length":"9"}]},{"pgid":"2.2","version":"0'0","reported_seq":209,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.826218+0000","last_change":"2024-05-08T08:53:55.365435+0000","last_active":"2024-05-08T08:54:36.826218+0000","last_peered":"2024-05-08T08:54:36.826218+0000","last_clean":"2024-05-08T08:54:36.826218+0000","last_became_active":"2024-05-08T08:53:55.365059+0000","last_became_peered":"2024-05-08T08:53:55.365059+0000","last_unstale":"2024-05-08T08:54:36.826218+0000","last_undegraded":"2024-05-08T08:54:36.826218+0000","last_fullsized":"2024-05-08T08:54:36.826218+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:15:39.278668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"124'639","reported_seq":13714,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.827079+0000","last_change":"2024-05-08T08:54:36.827304+0000","last_active":"2024-05-08T08:54:36.827079+0000","last_peered":"2024-05-08T08:54:36.827079+0000","last_clean":"2024-05-08T08:54:36.827079+0000","last_became_active":"2024-05-08T08:52:11.928301+0000","last_became_peered":"2024-05-08T08:52:11.928301+0000","last_unstale":"2024-05-08T08:54:36.827079+0000","last_undegraded":"2024-05-08T08:54:36.827079+0000","last_fullsized":"2024-05-08T08:54:36.827079+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":639,"log_dups_size":0,"ondisk_log_size":639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:45:01.844140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012830719999999999,"stat_sum":{"num_bytes":45686375,"num_objects":93,"num_object_clones":16,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":51,"num_read":10493,"num_read_kb":43924,"num_write":600,"num_write_kb":16296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2b","length":"9"}]},{"pgid":"2.1","version":"0'0","reported_seq":229,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.825153+0000","last_change":"2024-05-08T08:52:03.891768+0000","last_active":"2024-05-08T08:54:36.825153+0000","last_peered":"2024-05-08T08:54:36.825153+0000","last_clean":"2024-05-08T08:54:36.825153+0000","last_became_active":"2024-05-08T08:52:03.891374+0000","last_became_peered":"2024-05-08T08:52:03.891374+0000","last_unstale":"2024-05-08T08:54:36.825153+0000","last_undegraded":"2024-05-08T08:54:36.825153+0000","last_fullsized":"2024-05-08T08:54:36.825153+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:18:10.240492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"125'981","reported_seq":26827,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.825569+0000","last_change":"2024-05-08T08:54:36.825569+0000","last_active":"2024-05-08T08:54:36.825569+0000","last_peered":"2024-05-08T08:54:36.825569+0000","last_clean":"2024-05-08T08:54:36.825569+0000","last_became_active":"2024-05-08T08:53:49.581797+0000","last_became_peered":"2024-05-08T08:53:49.581797+0000","last_unstale":"2024-05-08T08:54:36.825569+0000","last_undegraded":"2024-05-08T08:54:36.825569+0000","last_fullsized":"2024-05-08T08:54:36.825569+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":981,"log_dups_size":0,"ondisk_log_size":981,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:18:13.421855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060326200000000003,"stat_sum":{"num_bytes":45467468,"num_objects":104,"num_object_clones":16,"num_object_copies":208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":54,"num_read":29789,"num_read_kb":55110,"num_write":748,"num_write_kb":9610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":135,"num_bytes_recovered":3549534,"num_keys_recovered":46,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2b","length":"9"}]},{"pgid":"2.0","version":"0'0","reported_seq":228,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.833923+0000","last_change":"2024-05-08T08:52:03.891551+0000","last_active":"2024-05-08T08:54:36.833923+0000","last_peered":"2024-05-08T08:54:36.833923+0000","last_clean":"2024-05-08T08:54:36.833923+0000","last_became_active":"2024-05-08T08:52:03.891296+0000","last_became_peered":"2024-05-08T08:52:03.891296+0000","last_unstale":"2024-05-08T08:54:36.833923+0000","last_undegraded":"2024-05-08T08:54:36.833923+0000","last_fullsized":"2024-05-08T08:54:36.833923+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:54.806343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"125'896","reported_seq":14006,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.923309+0000","last_change":"2024-05-08T08:54:36.827665+0000","last_active":"2024-05-08T08:54:36.923309+0000","last_peered":"2024-05-08T08:54:36.923309+0000","last_clean":"2024-05-08T08:54:36.923309+0000","last_became_active":"2024-05-08T08:53:58.616032+0000","last_became_peered":"2024-05-08T08:53:58.616032+0000","last_unstale":"2024-05-08T08:54:36.923309+0000","last_undegraded":"2024-05-08T08:54:36.923309+0000","last_fullsized":"2024-05-08T08:54:36.923309+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":896,"log_dups_size":0,"ondisk_log_size":896,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:13:59.148304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015983410000000001,"stat_sum":{"num_bytes":40747457,"num_objects":103,"num_object_clones":18,"num_object_copies":206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103,"num_whiteouts":57,"num_read":15365,"num_read_kb":63333,"num_write":1196,"num_write_kb":13964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":125,"num_bytes_recovered":4950942,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2b","length":"9"}]},{"pgid":"2.3","version":"0'0","reported_seq":209,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.826619+0000","last_change":"2024-05-08T08:53:55.362409+0000","last_active":"2024-05-08T08:54:36.826619+0000","last_peered":"2024-05-08T08:54:36.826619+0000","last_clean":"2024-05-08T08:54:36.826619+0000","last_became_active":"2024-05-08T08:53:55.362055+0000","last_became_peered":"2024-05-08T08:53:55.362055+0000","last_unstale":"2024-05-08T08:54:36.826619+0000","last_undegraded":"2024-05-08T08:54:36.826619+0000","last_fullsized":"2024-05-08T08:54:36.826619+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:34:23.719119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":384,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.834009+0000","last_change":"2024-05-08T08:51:58.861899+0000","last_active":"2024-05-08T08:54:36.834009+0000","last_peered":"2024-05-08T08:54:36.834009+0000","last_clean":"2024-05-08T08:54:36.834009+0000","last_became_active":"2024-05-08T08:51:58.861596+0000","last_became_peered":"2024-05-08T08:51:58.861596+0000","last_unstale":"2024-05-08T08:54:36.834009+0000","last_undegraded":"2024-05-08T08:54:36.834009+0000","last_fullsized":"2024-05-08T08:54:36.834009+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:51:57.849666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:51:57.849666+0000","last_clean_scrub_stamp":"2024-05-08T08:51:57.849666+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:53:21.739950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"125'343","reported_seq":9435,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.828424+0000","last_change":"2024-05-08T08:54:36.828424+0000","last_active":"2024-05-08T08:54:36.828424+0000","last_peered":"2024-05-08T08:54:36.828424+0000","last_clean":"2024-05-08T08:54:36.828424+0000","last_became_active":"2024-05-08T08:52:11.926975+0000","last_became_peered":"2024-05-08T08:52:11.926975+0000","last_unstale":"2024-05-08T08:54:36.828424+0000","last_undegraded":"2024-05-08T08:54:36.828424+0000","last_fullsized":"2024-05-08T08:54:36.828424+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:35:58.570037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002105834,"stat_sum":{"num_bytes":31656032,"num_objects":88,"num_object_clones":15,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":48,"num_read":8890,"num_read_kb":36552,"num_write":315,"num_write_kb":8985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2b","length":"9"}]},{"pgid":"3.9","version":"125'430","reported_seq":14065,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.825806+0000","last_change":"2024-05-08T08:54:36.825806+0000","last_active":"2024-05-08T08:54:36.825806+0000","last_peered":"2024-05-08T08:54:36.825806+0000","last_clean":"2024-05-08T08:54:36.825806+0000","last_became_active":"2024-05-08T08:53:23.478564+0000","last_became_peered":"2024-05-08T08:53:23.478564+0000","last_unstale":"2024-05-08T08:54:36.825806+0000","last_undegraded":"2024-05-08T08:54:36.825806+0000","last_fullsized":"2024-05-08T08:54:36.825806+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":430,"log_dups_size":0,"ondisk_log_size":430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:07:33.542111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071444800000000001,"stat_sum":{"num_bytes":36214831,"num_objects":92,"num_object_clones":11,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":52,"num_read":14021,"num_read_kb":62064,"num_write":368,"num_write_kb":16560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":131,"num_bytes_recovered":12377088,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2b","length":"9"}]},{"pgid":"3.a","version":"125'1122","reported_seq":11276,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.915030+0000","last_change":"2024-05-08T08:54:36.834866+0000","last_active":"2024-05-08T08:54:36.915030+0000","last_peered":"2024-05-08T08:54:36.915030+0000","last_clean":"2024-05-08T08:54:36.915030+0000","last_became_active":"2024-05-08T08:53:24.480359+0000","last_became_peered":"2024-05-08T08:53:24.480359+0000","last_unstale":"2024-05-08T08:54:36.915030+0000","last_undegraded":"2024-05-08T08:54:36.915030+0000","last_fullsized":"2024-05-08T08:54:36.915030+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":1122,"log_dups_size":0,"ondisk_log_size":1122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:13:17.330414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070201699999999998,"stat_sum":{"num_bytes":21900143,"num_objects":84,"num_object_clones":12,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":50,"num_read":15403,"num_read_kb":29111,"num_write":1358,"num_write_kb":6314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":2175343,"num_keys_recovered":61,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2b","length":"9"}]},{"pgid":"3.b","version":"125'527","reported_seq":11224,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.826465+0000","last_change":"2024-05-08T08:54:36.826465+0000","last_active":"2024-05-08T08:54:36.826465+0000","last_peered":"2024-05-08T08:54:36.826465+0000","last_clean":"2024-05-08T08:54:36.826465+0000","last_became_active":"2024-05-08T08:52:11.927196+0000","last_became_peered":"2024-05-08T08:52:11.927196+0000","last_unstale":"2024-05-08T08:54:36.826465+0000","last_undegraded":"2024-05-08T08:54:36.826465+0000","last_fullsized":"2024-05-08T08:54:36.826465+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":527,"log_dups_size":0,"ondisk_log_size":527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:13:45.973320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087152499999999995,"stat_sum":{"num_bytes":39118437,"num_objects":96,"num_object_clones":12,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":60,"num_read":11551,"num_read_kb":39176,"num_write":449,"num_write_kb":7713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2b","length":"9"}]},{"pgid":"3.c","version":"124'335","reported_seq":13605,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.835019+0000","last_change":"2024-05-08T08:54:36.835208+0000","last_active":"2024-05-08T08:54:36.835019+0000","last_peered":"2024-05-08T08:54:36.835019+0000","last_clean":"2024-05-08T08:54:36.835019+0000","last_became_active":"2024-05-08T08:53:58.615095+0000","last_became_peered":"2024-05-08T08:53:58.615095+0000","last_unstale":"2024-05-08T08:54:36.835019+0000","last_undegraded":"2024-05-08T08:54:36.835019+0000","last_fullsized":"2024-05-08T08:54:36.835019+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:19:42.959484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093134299999999995,"stat_sum":{"num_bytes":15228603,"num_objects":82,"num_object_clones":10,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":49,"num_read":13093,"num_read_kb":41263,"num_write":300,"num_write_kb":7340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":107,"num_bytes_recovered":2139323,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2b","length":"9"}]},{"pgid":"3.d","version":"125'415","reported_seq":8430,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.914429+0000","last_change":"2024-05-08T08:54:36.835776+0000","last_active":"2024-05-08T08:54:36.914429+0000","last_peered":"2024-05-08T08:54:36.914429+0000","last_clean":"2024-05-08T08:54:36.914429+0000","last_became_active":"2024-05-08T08:52:11.932594+0000","last_became_peered":"2024-05-08T08:52:11.932594+0000","last_unstale":"2024-05-08T08:54:36.914429+0000","last_undegraded":"2024-05-08T08:54:36.914429+0000","last_fullsized":"2024-05-08T08:54:36.914429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":415,"log_dups_size":0,"ondisk_log_size":415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:34:34.694953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089293900000000004,"stat_sum":{"num_bytes":18975285,"num_objects":83,"num_object_clones":10,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":51,"num_read":9211,"num_read_kb":34021,"num_write":364,"num_write_kb":9996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2b","length":"9"}]},{"pgid":"3.e","version":"125'550","reported_seq":8816,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.914782+0000","last_change":"2024-05-08T08:54:36.826081+0000","last_active":"2024-05-08T08:54:36.914782+0000","last_peered":"2024-05-08T08:54:36.914782+0000","last_clean":"2024-05-08T08:54:36.914782+0000","last_became_active":"2024-05-08T08:53:58.621584+0000","last_became_peered":"2024-05-08T08:53:58.621584+0000","last_unstale":"2024-05-08T08:54:36.914782+0000","last_undegraded":"2024-05-08T08:54:36.914782+0000","last_fullsized":"2024-05-08T08:54:36.914782+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":550,"log_dups_size":0,"ondisk_log_size":550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:56:25.857586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000891752,"stat_sum":{"num_bytes":27483200,"num_objects":70,"num_object_clones":14,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":34,"num_read":15911,"num_read_kb":46955,"num_write":583,"num_write_kb":12214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":188,"num_bytes_recovered":11952192,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2b","length":"9"}]},{"pgid":"3.f","version":"125'506","reported_seq":7632,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.836091+0000","last_change":"2024-05-08T08:54:36.836091+0000","last_active":"2024-05-08T08:54:36.836091+0000","last_peered":"2024-05-08T08:54:36.836091+0000","last_clean":"2024-05-08T08:54:36.836091+0000","last_became_active":"2024-05-08T08:53:58.615064+0000","last_became_peered":"2024-05-08T08:53:58.615064+0000","last_unstale":"2024-05-08T08:54:36.836091+0000","last_undegraded":"2024-05-08T08:54:36.836091+0000","last_fullsized":"2024-05-08T08:54:36.836091+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:43:36.481505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010320520000000001,"stat_sum":{"num_bytes":24172770,"num_objects":75,"num_object_clones":8,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":44,"num_read":8460,"num_read_kb":38018,"num_write":513,"num_write_kb":9621,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":107,"num_bytes_recovered":4068050,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2b","length":"9"}]},{"pgid":"3.10","version":"123'693","reported_seq":7967,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.825480+0000","last_change":"2024-05-08T08:54:36.826525+0000","last_active":"2024-05-08T08:54:36.825480+0000","last_peered":"2024-05-08T08:54:36.825480+0000","last_clean":"2024-05-08T08:54:36.825480+0000","last_became_active":"2024-05-08T08:53:58.618787+0000","last_became_peered":"2024-05-08T08:53:58.618787+0000","last_unstale":"2024-05-08T08:54:36.825480+0000","last_undegraded":"2024-05-08T08:54:36.825480+0000","last_fullsized":"2024-05-08T08:54:36.825480+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":693,"log_dups_size":0,"ondisk_log_size":693,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:05:11.277267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001293523,"stat_sum":{"num_bytes":32900689,"num_objects":83,"num_object_clones":13,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":51,"num_read":9053,"num_read_kb":41233,"num_write":952,"num_write_kb":14552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":112,"num_bytes_recovered":7552081,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2b","length":"9"}]},{"pgid":"3.11","version":"125'664","reported_seq":11366,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.834629+0000","last_change":"2024-05-08T08:54:36.834629+0000","last_active":"2024-05-08T08:54:36.834629+0000","last_peered":"2024-05-08T08:54:36.834629+0000","last_clean":"2024-05-08T08:54:36.834629+0000","last_became_active":"2024-05-08T08:53:24.479891+0000","last_became_peered":"2024-05-08T08:53:24.479891+0000","last_unstale":"2024-05-08T08:54:36.834629+0000","last_undegraded":"2024-05-08T08:54:36.834629+0000","last_fullsized":"2024-05-08T08:54:36.834629+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":664,"log_dups_size":0,"ondisk_log_size":664,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:42:24.013868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050629799999999997,"stat_sum":{"num_bytes":31926821,"num_objects":94,"num_object_clones":16,"num_object_copies":188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":57,"num_read":13002,"num_read_kb":36339,"num_write":537,"num_write_kb":10910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":131,"num_bytes_recovered":2604032,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2b","length":"9"}]},{"pgid":"3.12","version":"125'1429","reported_seq":24704,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.960646+0000","last_change":"2024-05-08T08:54:36.834580+0000","last_active":"2024-05-08T08:54:36.960646+0000","last_peered":"2024-05-08T08:54:36.960646+0000","last_clean":"2024-05-08T08:54:36.960646+0000","last_became_active":"2024-05-08T08:53:49.580557+0000","last_became_peered":"2024-05-08T08:53:49.580557+0000","last_unstale":"2024-05-08T08:54:36.960646+0000","last_undegraded":"2024-05-08T08:54:36.960646+0000","last_fullsized":"2024-05-08T08:54:36.960646+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":1429,"log_dups_size":0,"ondisk_log_size":1429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:57:47.121534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047675199999999999,"stat_sum":{"num_bytes":44968122,"num_objects":106,"num_object_clones":11,"num_object_copies":212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":106,"num_whiteouts":57,"num_read":30248,"num_read_kb":62116,"num_write":2020,"num_write_kb":12048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2b","length":"9"}]},{"pgid":"3.13","version":"123'552","reported_seq":10216,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T08:54:36.834953+0000","last_change":"2024-05-08T08:54:36.835139+0000","last_active":"2024-05-08T08:54:36.834953+0000","last_peered":"2024-05-08T08:54:36.834953+0000","last_clean":"2024-05-08T08:54:36.834953+0000","last_became_active":"2024-05-08T08:53:58.615947+0000","last_became_peered":"2024-05-08T08:53:58.615947+0000","last_unstale":"2024-05-08T08:54:36.834953+0000","last_undegraded":"2024-05-08T08:54:36.834953+0000","last_fullsized":"2024-05-08T08:54:36.834953+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":552,"log_dups_size":0,"ondisk_log_size":552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:16:16.764806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086118900000000005,"stat_sum":{"num_bytes":51989631,"num_objects":100,"num_object_clones":20,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":53,"num_read":10774,"num_read_kb":40880,"num_write":547,"num_write_kb":9065,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":130,"num_bytes_recovered":6657151,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2b","length":"9"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1152505644,"num_objects":2909,"num_object_clones":438,"num_object_copies":5818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2909,"num_whiteouts":1643,"num_read":501708,"num_read_kb":1618770,"num_write":23906,"num_write_kb":451334,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3104,"num_bytes_recovered":190761442,"num_keys_recovered":470,"num_objects_omap":43,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":88178688,"data_stored":737510251,"data_compressed":37851007,"data_compressed_allocated":72433664,"data_compressed_original":727909888,"omap_allocated":0,"internal_metadata":0},"log_size":21588,"ondisk_log_size":21588,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":11046,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":0,"seq":47244640290,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640293,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73068,"kb_used_data":15524,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94298772,"statfs":{"total":96636764160,"available":96561942528,"internally_reserved":0,"allocated":15896576,"data_stored":126394164,"data_compressed":6761617,"data_compressed_allocated":12402688,"data_compressed_original":124585984,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640292,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":91084,"kb_used_data":19628,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94280756,"statfs":{"total":96636764160,"available":96543494144,"internally_reserved":0,"allocated":20099072,"data_stored":149553879,"data_compressed":7726016,"data_compressed_allocated":14344192,"data_compressed_original":146203136,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640291,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46392,"kb_used_data":11052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94325448,"statfs":{"total":96636764160,"available":96589258752,"internally_reserved":0,"allocated":11317248,"data_stored":77138265,"data_compressed":4700015,"data_compressed_allocated":8282112,"data_compressed_original":75612672,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640290,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82408,"kb_used_data":19284,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94289432,"statfs":{"total":96636764160,"available":96552378368,"internally_reserved":0,"allocated":19746816,"data_stored":168967396,"data_compressed":7765083,"data_compressed_allocated":15867904,"data_compressed_original":166971904,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640293,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":84080,"kb_used_data":16792,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94287760,"statfs":{"total":96636764160,"available":96550666240,"internally_reserved":0,"allocated":17195008,"data_stored":121631259,"data_compressed":5996279,"data_compressed_allocated":11804672,"data_compressed_original":118602752,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5523,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5523,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":15650816,"data_stored":120421551,"data_compressed":5990756,"data_compressed_allocated":11767808,"data_compressed_original":118029312,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":18259968,"data_stored":168348056,"data_compressed":7765083,"data_compressed_allocated":15867904,"data_compressed_original":166971904,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":9830400,"data_stored":76518925,"data_compressed":4700015,"data_compressed_allocated":8282112,"data_compressed_original":75612672,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":18321408,"data_stored":147148316,"data_compressed":7613617,"data_compressed_allocated":14131200,"data_compressed_original":144469504,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":14409728,"data_stored":125774824,"data_compressed":6761617,"data_compressed_allocated":12402688,"data_compressed_original":124585984,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":11706368,"data_stored":99298579,"data_compressed":5019919,"data_compressed_allocated":9981952,"data_compressed_original":98240512,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T08:54:39.635 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-05-08T08:54:39.635 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:39.633+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:39.636 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T08:54:39.636 INFO:tasks.thrashosds.thrasher:Adding osd 5 2024-05-08T08:54:39.636 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 5 2024-05-08T08:54:39.731 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:39.733+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:39.778 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:39.778 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:54:39.778 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:54:39.778 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:39.787 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:54:39.832 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:39.834+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:39.837 INFO:teuthology.orchestra.run.smithi059.stderr:nodeep-scrub is set 2024-05-08T08:54:39.899 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:39.899 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:54:39.899 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:54:39.899 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:54:39.899 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:39.909 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:54:39.952 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:39.955+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:39.994 INFO:tasks.ceph.mon.a.smithi059.stderr:2024-05-08T08:54:39.997+0000 7fc9cc811640 -1 mon.a@0(leader).osd e128 definitely_dead 0 2024-05-08T08:54:40.030 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:40.030 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:54:40.030 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:54:40.030 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:54:40.031 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.031 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6690 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1093632~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:40.031 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:54:40.031 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 128.31693836299999, 2024-05-08T08:54:40.031 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052462754, 2024-05-08T08:54:40.031 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:40.031 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:40.031 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:40.031 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:40.031 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:40.031 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:40.031 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6690 2024-05-08T08:54:40.031 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.031 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:40.031 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.031 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:40.032 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:54:40.032 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:40.032 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.032 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.032 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:40.032 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:54:40.032 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:40.032 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.032 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.032 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:40.032 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715346+0000", 2024-05-08T08:54:40.032 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.84e-07 2024-05-08T08:54:40.032 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.032 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.032 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:40.033 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715348+0000", 2024-05-08T08:54:40.033 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.114e-06 2024-05-08T08:54:40.033 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.033 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.033 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:40.033 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715353+0000", 2024-05-08T08:54:40.033 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5720000000000002e-06 2024-05-08T08:54:40.033 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.033 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.033 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:40.033 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715358+0000", 2024-05-08T08:54:40.033 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7140000000000001e-06 2024-05-08T08:54:40.033 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.033 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.033 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:40.033 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767638+0000", 2024-05-08T08:54:40.033 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052279562000000002 2024-05-08T08:54:40.034 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.034 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.034 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:40.034 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767713+0000", 2024-05-08T08:54:40.034 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5652e-05 2024-05-08T08:54:40.034 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.034 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.034 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:40.034 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767808+0000", 2024-05-08T08:54:40.034 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4755999999999999e-05 2024-05-08T08:54:40.034 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:40.034 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:40.034 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:40.034 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.034 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.034 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6692 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1110016~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:40.035 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:54:40.035 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 128.316919273, 2024-05-08T08:54:40.035 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052666495000000001, 2024-05-08T08:54:40.035 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:40.035 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:40.035 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:40.035 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:40.035 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:40.035 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:40.035 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6692 2024-05-08T08:54:40.035 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.035 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:40.035 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.035 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:40.035 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:54:40.036 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:40.036 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.036 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.036 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:40.036 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:54:40.036 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:40.036 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.036 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.036 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:40.036 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:54:40.036 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8599999999999999e-07 2024-05-08T08:54:40.036 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.036 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.036 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:40.036 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715372+0000", 2024-05-08T08:54:40.036 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0990000000000001e-06 2024-05-08T08:54:40.037 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.037 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.037 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:40.037 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715378+0000", 2024-05-08T08:54:40.037 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0680000000000002e-06 2024-05-08T08:54:40.037 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.037 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.037 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:40.037 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715383+0000", 2024-05-08T08:54:40.037 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8740000000000003e-06 2024-05-08T08:54:40.037 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.037 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.037 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:40.037 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767848+0000", 2024-05-08T08:54:40.038 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052464668999999998 2024-05-08T08:54:40.038 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.038 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.038 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:40.038 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767927+0000", 2024-05-08T08:54:40.038 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9228999999999996e-05 2024-05-08T08:54:40.038 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.038 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.038 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:40.038 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768031+0000", 2024-05-08T08:54:40.038 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010417 2024-05-08T08:54:40.038 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:40.038 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:40.038 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:40.039 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.039 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.039 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6694 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1126400~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:40.039 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:54:40.039 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 128.31689437599999, 2024-05-08T08:54:40.039 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052848829, 2024-05-08T08:54:40.039 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:40.039 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:40.039 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:40.039 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:40.039 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:40.039 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:40.039 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6694 2024-05-08T08:54:40.039 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.039 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:40.040 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.040 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:40.040 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:54:40.040 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:40.040 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.040 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.040 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:40.040 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:54:40.040 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:40.040 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.040 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.040 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:40.040 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715390+0000", 2024-05-08T08:54:40.040 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9900000000000001e-07 2024-05-08T08:54:40.040 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.040 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.041 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:40.041 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715392+0000", 2024-05-08T08:54:40.041 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8279999999999999e-06 2024-05-08T08:54:40.041 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.041 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.041 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:40.041 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715398+0000", 2024-05-08T08:54:40.041 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7619999999999998e-06 2024-05-08T08:54:40.041 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.041 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.041 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:40.041 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715403+0000", 2024-05-08T08:54:40.041 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7790000000000003e-06 2024-05-08T08:54:40.041 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.042 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.042 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:40.042 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768067+0000", 2024-05-08T08:54:40.042 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052663785999999997 2024-05-08T08:54:40.042 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.042 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.042 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:40.042 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768142+0000", 2024-05-08T08:54:40.042 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5018999999999994e-05 2024-05-08T08:54:40.042 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.042 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.042 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:40.042 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768238+0000", 2024-05-08T08:54:40.043 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6255999999999995e-05 2024-05-08T08:54:40.043 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:40.043 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:40.043 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:40.043 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.043 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.043 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6696 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1142784~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:40.043 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:54:40.043 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 128.31687486800001, 2024-05-08T08:54:40.043 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.053051057999999998, 2024-05-08T08:54:40.043 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:40.043 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:40.043 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:40.043 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:40.044 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:40.044 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:40.044 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6696 2024-05-08T08:54:40.044 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.044 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:40.044 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.044 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:40.044 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:54:40.044 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:40.044 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.044 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.044 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:40.044 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:54:40.044 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:40.044 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.044 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.045 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:40.045 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:54:40.045 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-05-08T08:54:40.045 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.045 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.045 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:40.045 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715411+0000", 2024-05-08T08:54:40.045 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9630000000000001e-06 2024-05-08T08:54:40.045 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.045 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.045 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:40.045 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715417+0000", 2024-05-08T08:54:40.045 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7250000000000002e-06 2024-05-08T08:54:40.045 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.045 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.046 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:40.046 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715421+0000", 2024-05-08T08:54:40.046 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4089999999999996e-06 2024-05-08T08:54:40.046 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.046 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.046 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:40.046 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768277+0000", 2024-05-08T08:54:40.046 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052855293999999997 2024-05-08T08:54:40.046 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.046 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.046 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:40.046 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768358+0000", 2024-05-08T08:54:40.046 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1720999999999999e-05 2024-05-08T08:54:40.047 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.047 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.047 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:40.047 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768460+0000", 2024-05-08T08:54:40.047 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010154600000000001 2024-05-08T08:54:40.047 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:40.047 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:40.047 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:40.047 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.047 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.047 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6698 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1159168~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:40.047 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:54:40.047 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 128.31685600099999, 2024-05-08T08:54:40.047 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.053245726, 2024-05-08T08:54:40.048 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:40.048 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:40.048 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:40.048 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:40.048 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:40.048 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:40.048 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6698 2024-05-08T08:54:40.048 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.048 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:40.048 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.048 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:40.048 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:54:40.048 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:40.048 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.049 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.049 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:40.049 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:54:40.049 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:40.049 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.049 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.049 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:40.049 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716936+0000", 2024-05-08T08:54:40.049 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001508665 2024-05-08T08:54:40.049 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.049 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.049 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:40.049 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716941+0000", 2024-05-08T08:54:40.049 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1910000000000002e-06 2024-05-08T08:54:40.050 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.050 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.050 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:40.050 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716950+0000", 2024-05-08T08:54:40.050 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1900000000000001e-06 2024-05-08T08:54:40.050 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.050 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.050 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:40.050 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716958+0000", 2024-05-08T08:54:40.050 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8019999999999992e-06 2024-05-08T08:54:40.050 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.050 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.050 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:40.051 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768499+0000", 2024-05-08T08:54:40.051 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051540926000000001 2024-05-08T08:54:40.051 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.051 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.051 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:40.051 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768574+0000", 2024-05-08T08:54:40.051 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4988999999999996e-05 2024-05-08T08:54:40.051 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.051 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.051 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:40.051 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768673+0000", 2024-05-08T08:54:40.051 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9963000000000005e-05 2024-05-08T08:54:40.051 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:40.052 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:40.052 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:40.052 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.052 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.052 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6706 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1224704~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:40.052 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:54:40.052 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 128.31525069400001, 2024-05-08T08:54:40.052 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052435598, 2024-05-08T08:54:40.052 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:40.052 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:40.052 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:40.052 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:40.052 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:40.052 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:40.053 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6706 2024-05-08T08:54:40.053 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.053 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:40.053 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.053 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:40.053 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:54:40.053 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:40.053 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.053 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.053 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:40.053 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:54:40.053 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:40.053 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.053 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.054 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:40.054 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:54:40.054 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:54:40.054 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.054 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.054 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:40.054 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717035+0000", 2024-05-08T08:54:40.054 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9290000000000001e-06 2024-05-08T08:54:40.054 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.054 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.054 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:40.054 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717041+0000", 2024-05-08T08:54:40.054 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6720000000000001e-06 2024-05-08T08:54:40.054 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.054 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.055 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:40.055 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717046+0000", 2024-05-08T08:54:40.055 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5650000000000003e-06 2024-05-08T08:54:40.055 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.055 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.055 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:40.055 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769311+0000", 2024-05-08T08:54:40.055 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052265645999999999 2024-05-08T08:54:40.055 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.055 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.055 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:40.055 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769379+0000", 2024-05-08T08:54:40.055 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7391000000000006e-05 2024-05-08T08:54:40.055 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.055 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.056 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:40.056 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769469+0000", 2024-05-08T08:54:40.056 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0025999999999995e-05 2024-05-08T08:54:40.056 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:40.056 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:40.056 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:40.056 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.056 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.056 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6708 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1241088~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:40.056 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:54:40.056 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 128.31523258999999, 2024-05-08T08:54:40.056 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052640552, 2024-05-08T08:54:40.056 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:40.056 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:40.056 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:40.056 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:40.057 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:40.057 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:40.057 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6708 2024-05-08T08:54:40.057 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.057 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:40.057 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.057 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:40.057 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:54:40.057 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:40.057 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.057 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.057 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:40.057 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:54:40.057 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:40.057 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.057 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.058 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:40.058 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717052+0000", 2024-05-08T08:54:40.058 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:54:40.058 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.058 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.058 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:40.058 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717053+0000", 2024-05-08T08:54:40.058 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8330000000000001e-06 2024-05-08T08:54:40.058 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.058 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.058 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:40.058 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717058+0000", 2024-05-08T08:54:40.058 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.933e-06 2024-05-08T08:54:40.058 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.058 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.058 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:40.059 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717063+0000", 2024-05-08T08:54:40.059 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2180000000000001e-06 2024-05-08T08:54:40.059 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.059 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.059 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:40.059 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769507+0000", 2024-05-08T08:54:40.059 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052444866999999999 2024-05-08T08:54:40.059 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.059 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.059 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:40.059 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769589+0000", 2024-05-08T08:54:40.059 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2039000000000005e-05 2024-05-08T08:54:40.059 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.059 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.059 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:40.060 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769692+0000", 2024-05-08T08:54:40.060 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102252 2024-05-08T08:54:40.060 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:40.060 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:40.060 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:40.060 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.060 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.060 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6710 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1257472~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:40.060 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:54:40.060 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 128.31521456600001, 2024-05-08T08:54:40.060 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.05283057, 2024-05-08T08:54:40.060 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:40.060 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:40.060 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:40.060 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:40.060 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:40.061 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:40.061 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6710 2024-05-08T08:54:40.061 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.061 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:40.061 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.061 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:40.061 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:54:40.061 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:40.061 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.061 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.061 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:40.061 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:54:40.061 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:40.061 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.062 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.062 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:40.062 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718951+0000", 2024-05-08T08:54:40.062 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0018817809999999999 2024-05-08T08:54:40.062 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.062 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.062 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:40.062 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718956+0000", 2024-05-08T08:54:40.062 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5399999999999997e-06 2024-05-08T08:54:40.062 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.062 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.062 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:40.062 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718964+0000", 2024-05-08T08:54:40.062 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6629999999999999e-06 2024-05-08T08:54:40.063 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.063 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.063 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:40.063 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718972+0000", 2024-05-08T08:54:40.063 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5229999999999998e-06 2024-05-08T08:54:40.063 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.063 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.063 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:40.063 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769729+0000", 2024-05-08T08:54:40.063 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050756959999999997 2024-05-08T08:54:40.063 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.063 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.063 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:40.063 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769803+0000", 2024-05-08T08:54:40.064 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4538999999999999e-05 2024-05-08T08:54:40.064 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.064 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.064 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:40.064 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769900+0000", 2024-05-08T08:54:40.064 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6564000000000006e-05 2024-05-08T08:54:40.064 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:40.064 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:40.064 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:40.064 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.064 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.064 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6726 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1388544~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:40.065 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:54:40.065 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 128.31313979399999, 2024-05-08T08:54:40.065 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052451506000000002, 2024-05-08T08:54:40.065 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:40.065 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:40.065 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:40.065 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:40.065 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:40.065 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:40.065 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6726 2024-05-08T08:54:40.065 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.065 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:40.065 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.065 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:40.065 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:54:40.066 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:40.066 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.066 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.066 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:40.066 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:54:40.066 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:40.066 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.066 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.066 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:40.066 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719200+0000", 2024-05-08T08:54:40.066 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5865999999999997e-05 2024-05-08T08:54:40.066 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.066 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.066 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:40.067 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719204+0000", 2024-05-08T08:54:40.067 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8630000000000001e-06 2024-05-08T08:54:40.067 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.067 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.067 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:40.067 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719215+0000", 2024-05-08T08:54:40.067 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1590000000000001e-05 2024-05-08T08:54:40.067 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.067 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.067 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:40.067 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719220+0000", 2024-05-08T08:54:40.067 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1619999999999999e-06 2024-05-08T08:54:40.067 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.067 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.068 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:40.068 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771425+0000", 2024-05-08T08:54:40.068 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052204249000000001 2024-05-08T08:54:40.068 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.068 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.068 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:40.068 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771498+0000", 2024-05-08T08:54:40.068 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3782999999999995e-05 2024-05-08T08:54:40.068 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.068 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.068 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:40.068 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771595+0000", 2024-05-08T08:54:40.068 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6992999999999998e-05 2024-05-08T08:54:40.069 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:40.069 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:40.069 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:40.069 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.069 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.069 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6728 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1404928~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:40.069 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:54:40.069 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 128.31305238900001, 2024-05-08T08:54:40.069 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052590747, 2024-05-08T08:54:40.069 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:40.069 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:40.069 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:40.069 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:40.069 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:40.069 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:40.070 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6728 2024-05-08T08:54:40.070 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.070 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:40.070 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.070 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:40.070 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:54:40.070 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:40.070 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.070 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.070 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:40.070 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:54:40.070 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:40.070 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.071 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.071 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:40.071 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719232+0000", 2024-05-08T08:54:40.071 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4000000000000001e-07 2024-05-08T08:54:40.071 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.071 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.071 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:40.071 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719234+0000", 2024-05-08T08:54:40.071 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4499999999999998e-06 2024-05-08T08:54:40.071 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.071 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.071 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:40.071 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719240+0000", 2024-05-08T08:54:40.071 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5720000000000002e-06 2024-05-08T08:54:40.072 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.072 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.072 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:40.072 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719245+0000", 2024-05-08T08:54:40.072 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8690000000000003e-06 2024-05-08T08:54:40.072 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.072 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.072 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:40.072 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771636+0000", 2024-05-08T08:54:40.072 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052390979999999997 2024-05-08T08:54:40.072 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.072 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.072 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:40.072 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771717+0000", 2024-05-08T08:54:40.072 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1291999999999993e-05 2024-05-08T08:54:40.073 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.073 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.073 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:40.073 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771822+0000", 2024-05-08T08:54:40.073 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104944 2024-05-08T08:54:40.073 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:40.073 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:40.073 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:40.073 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.073 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.073 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6730 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1421312~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:40.073 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:54:40.073 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 128.313032147, 2024-05-08T08:54:40.073 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052770477000000003, 2024-05-08T08:54:40.073 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:40.074 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:40.074 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:40.074 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:40.074 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:40.074 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:40.074 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6730 2024-05-08T08:54:40.074 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.074 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:40.074 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.074 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:40.074 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:54:40.074 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:40.074 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.075 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.075 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:40.075 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:54:40.075 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:40.075 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.075 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.075 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:40.075 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:54:40.075 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9000000000000002e-07 2024-05-08T08:54:40.075 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.075 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.075 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:40.075 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719254+0000", 2024-05-08T08:54:40.075 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4099999999999998e-06 2024-05-08T08:54:40.076 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.076 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.076 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:40.076 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719263+0000", 2024-05-08T08:54:40.076 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4109999999999996e-06 2024-05-08T08:54:40.076 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.076 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.076 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:40.076 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719268+0000", 2024-05-08T08:54:40.076 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2669999999999997e-06 2024-05-08T08:54:40.076 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.076 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.076 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:40.076 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771846+0000", 2024-05-08T08:54:40.076 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052577677000000003 2024-05-08T08:54:40.076 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.077 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.077 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:40.077 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771920+0000", 2024-05-08T08:54:40.077 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4460000000000002e-05 2024-05-08T08:54:40.077 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.077 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.077 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:40.077 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772022+0000", 2024-05-08T08:54:40.077 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010186199999999999 2024-05-08T08:54:40.077 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:40.077 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:40.077 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:40.077 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.077 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.077 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6732 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1437696~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:40.077 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:54:40.077 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 128.31300734000001, 2024-05-08T08:54:40.078 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052905575000000003, 2024-05-08T08:54:40.078 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:40.078 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:40.078 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:40.078 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:40.078 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:40.078 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:40.078 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6732 2024-05-08T08:54:40.078 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.078 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:40.078 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.078 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:40.078 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:54:40.078 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:40.078 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.079 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.079 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:40.079 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:54:40.079 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:40.079 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.079 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.079 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:40.079 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721392+0000", 2024-05-08T08:54:40.079 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002115287 2024-05-08T08:54:40.079 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.079 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.079 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:40.079 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721396+0000", 2024-05-08T08:54:40.079 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1679999999999997e-06 2024-05-08T08:54:40.079 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.080 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.080 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:40.080 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721404+0000", 2024-05-08T08:54:40.080 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4409999999999993e-06 2024-05-08T08:54:40.080 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.080 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.080 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:40.080 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721410+0000", 2024-05-08T08:54:40.080 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9150000000000001e-06 2024-05-08T08:54:40.080 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.080 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.080 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:40.080 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772061+0000", 2024-05-08T08:54:40.080 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050651068 2024-05-08T08:54:40.080 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.080 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.081 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:40.081 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772127+0000", 2024-05-08T08:54:40.081 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5424000000000002e-05 2024-05-08T08:54:40.081 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.081 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.081 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:40.081 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772182+0000", 2024-05-08T08:54:40.081 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5272000000000002e-05 2024-05-08T08:54:40.081 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:40.081 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:40.081 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:40.081 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.081 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.081 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6749 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1585152~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:40.082 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:54:40.082 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 128.31070657000001, 2024-05-08T08:54:40.082 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052350264000000001, 2024-05-08T08:54:40.082 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:40.082 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:40.082 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:40.082 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:40.082 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:40.082 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:40.082 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6749 2024-05-08T08:54:40.082 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.082 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:40.082 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.082 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:40.083 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:54:40.083 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:40.083 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.083 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.083 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:40.083 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:54:40.083 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:40.083 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.083 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.083 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:40.083 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721578+0000", 2024-05-08T08:54:40.083 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0399999999999996e-07 2024-05-08T08:54:40.083 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.083 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.083 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:40.084 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721580+0000", 2024-05-08T08:54:40.084 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5450000000000002e-06 2024-05-08T08:54:40.084 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.084 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.084 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:40.084 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721587+0000", 2024-05-08T08:54:40.084 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0190000000000001e-06 2024-05-08T08:54:40.084 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.084 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.084 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:40.084 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721593+0000", 2024-05-08T08:54:40.084 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5659999999999997e-06 2024-05-08T08:54:40.084 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.084 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.085 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:40.085 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773759+0000", 2024-05-08T08:54:40.085 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052166553999999997 2024-05-08T08:54:40.085 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.085 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.085 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:40.085 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773833+0000", 2024-05-08T08:54:40.085 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3913999999999997e-05 2024-05-08T08:54:40.085 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.085 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.085 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:40.085 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773927+0000", 2024-05-08T08:54:40.085 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4061999999999995e-05 2024-05-08T08:54:40.085 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:40.085 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:40.086 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:40.086 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.086 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.086 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6751 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1601536~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:40.086 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:54:40.086 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 128.310683291, 2024-05-08T08:54:40.086 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052461302000000001, 2024-05-08T08:54:40.086 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:40.086 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:40.086 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:40.086 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:40.086 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:40.086 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:40.086 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6751 2024-05-08T08:54:40.086 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.086 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:40.087 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.087 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:40.087 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:54:40.087 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:40.087 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.087 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.087 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:40.087 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:54:40.087 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:40.087 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.087 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.087 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:40.087 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722939+0000", 2024-05-08T08:54:40.087 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001338601 2024-05-08T08:54:40.087 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.087 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.088 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:40.088 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722944+0000", 2024-05-08T08:54:40.088 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4469999999999999e-06 2024-05-08T08:54:40.088 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.088 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.088 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:40.088 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722955+0000", 2024-05-08T08:54:40.088 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1892999999999999e-05 2024-05-08T08:54:40.088 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.088 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.088 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:40.088 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722963+0000", 2024-05-08T08:54:40.088 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1080000000000004e-06 2024-05-08T08:54:40.088 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.088 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.088 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:40.089 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773964+0000", 2024-05-08T08:54:40.089 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051001022 2024-05-08T08:54:40.089 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.089 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.089 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:40.089 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.774006+0000", 2024-05-08T08:54:40.089 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2729999999999999e-05 2024-05-08T08:54:40.089 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.089 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.089 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:40.089 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.774062+0000", 2024-05-08T08:54:40.089 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5501000000000003e-05 2024-05-08T08:54:40.089 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:40.089 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:40.089 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:40.089 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.089 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.090 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6760 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1683456~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:40.090 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:54:40.090 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 128.309173915, 2024-05-08T08:54:40.090 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052571312000000002, 2024-05-08T08:54:40.090 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:40.090 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:40.090 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:40.090 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:40.090 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:40.090 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:40.090 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6760 2024-05-08T08:54:40.090 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.090 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:40.090 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.090 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:40.091 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:54:40.091 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:40.091 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.091 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.091 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:40.091 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:54:40.091 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:40.091 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.091 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.091 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:40.091 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:54:40.091 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2100000000000002e-07 2024-05-08T08:54:40.091 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.091 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.092 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:40.092 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723113+0000", 2024-05-08T08:54:40.092 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3149999999999999e-06 2024-05-08T08:54:40.092 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.092 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.092 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:40.092 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723118+0000", 2024-05-08T08:54:40.092 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8860000000000001e-06 2024-05-08T08:54:40.092 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.092 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.092 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:40.092 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723123+0000", 2024-05-08T08:54:40.092 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6770000000000003e-06 2024-05-08T08:54:40.092 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.092 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.092 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:40.092 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775414+0000", 2024-05-08T08:54:40.093 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052290692999999999 2024-05-08T08:54:40.093 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.093 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.093 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:40.093 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775459+0000", 2024-05-08T08:54:40.093 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5639999999999997e-05 2024-05-08T08:54:40.093 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.093 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.093 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:40.093 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775681+0000", 2024-05-08T08:54:40.093 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00022168000000000001 2024-05-08T08:54:40.094 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:40.094 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:40.094 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:40.094 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.094 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.094 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6761 3.16 3:6bd6d6da:::rbd_data.1140782590a5.0000000000000015:6 [read 1095680~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:40.094 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:54:40.094 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 128.30915469499999, 2024-05-08T08:54:40.094 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052751275, 2024-05-08T08:54:40.094 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:40.094 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:40.094 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:40.094 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:40.094 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:40.094 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:40.095 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6761 2024-05-08T08:54:40.095 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.095 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:40.095 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.095 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:40.095 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:54:40.095 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:40.095 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.095 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.095 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:40.095 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:54:40.095 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:40.095 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.095 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.095 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:40.096 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724939+0000", 2024-05-08T08:54:40.096 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001809817 2024-05-08T08:54:40.096 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.096 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.096 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:40.096 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724943+0000", 2024-05-08T08:54:40.096 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4920000000000004e-06 2024-05-08T08:54:40.096 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.096 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.096 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:40.096 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724956+0000", 2024-05-08T08:54:40.096 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2427e-05 2024-05-08T08:54:40.096 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.096 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.096 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:40.096 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724963+0000", 2024-05-08T08:54:40.097 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2559999999999996e-06 2024-05-08T08:54:40.097 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.097 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.097 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:40.097 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775711+0000", 2024-05-08T08:54:40.097 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050747621 2024-05-08T08:54:40.097 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.097 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.097 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:40.097 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775785+0000", 2024-05-08T08:54:40.097 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4634000000000004e-05 2024-05-08T08:54:40.097 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.097 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.097 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:40.097 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775880+0000", 2024-05-08T08:54:40.097 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5027999999999999e-05 2024-05-08T08:54:40.097 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:40.098 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:40.098 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:40.098 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.098 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.098 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6770 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1765376~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:40.098 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:54:40.098 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 128.30712360000001, 2024-05-08T08:54:40.098 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052339296, 2024-05-08T08:54:40.098 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:40.098 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:40.098 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:40.098 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:40.098 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:40.098 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:40.098 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6770 2024-05-08T08:54:40.098 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.099 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:40.099 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.099 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:40.099 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:54:40.099 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:40.099 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.099 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.099 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:40.099 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:54:40.099 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:40.099 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.099 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.099 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:40.099 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725161+0000", 2024-05-08T08:54:40.099 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-05-08T08:54:40.100 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.100 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.100 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:40.100 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725163+0000", 2024-05-08T08:54:40.100 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4370000000000001e-06 2024-05-08T08:54:40.100 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.100 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.100 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:40.100 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725169+0000", 2024-05-08T08:54:40.100 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8409999999999996e-06 2024-05-08T08:54:40.100 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.100 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.100 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:40.100 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725174+0000", 2024-05-08T08:54:40.100 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7249999999999997e-06 2024-05-08T08:54:40.100 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.100 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.101 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:40.101 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777343+0000", 2024-05-08T08:54:40.101 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052169633999999999 2024-05-08T08:54:40.101 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.101 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.101 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:40.101 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777409+0000", 2024-05-08T08:54:40.101 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6033999999999998e-05 2024-05-08T08:54:40.101 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.101 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.101 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:40.101 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777499+0000", 2024-05-08T08:54:40.101 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0224999999999998e-05 2024-05-08T08:54:40.101 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:40.101 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:40.102 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:40.102 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.102 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.102 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6771 3.16 3:6bd6d6da:::rbd_data.1140782590a5.0000000000000015:6 [read 1177600~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:40.102 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:54:40.102 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 128.307103783, 2024-05-08T08:54:40.102 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052438604999999999, 2024-05-08T08:54:40.102 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:40.102 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:40.102 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:40.102 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:40.102 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:40.102 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:40.102 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6771 2024-05-08T08:54:40.102 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.102 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:40.102 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.103 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:40.103 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:54:40.103 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:40.103 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.103 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.103 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:40.103 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:54:40.103 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:40.103 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.103 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.103 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:40.103 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:54:40.103 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:54:40.103 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.103 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.103 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:40.103 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725188+0000", 2024-05-08T08:54:40.104 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1579999999999999e-06 2024-05-08T08:54:40.104 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.104 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.104 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:40.104 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725195+0000", 2024-05-08T08:54:40.104 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8909999999999995e-06 2024-05-08T08:54:40.104 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.104 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.104 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:40.104 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725200+0000", 2024-05-08T08:54:40.104 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9810000000000003e-06 2024-05-08T08:54:40.104 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.104 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.104 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:40.104 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777527+0000", 2024-05-08T08:54:40.105 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052326582000000003 2024-05-08T08:54:40.105 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.105 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.105 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:40.105 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777566+0000", 2024-05-08T08:54:40.105 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9447999999999999e-05 2024-05-08T08:54:40.105 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.105 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.105 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:40.105 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777619+0000", 2024-05-08T08:54:40.105 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2135000000000001e-05 2024-05-08T08:54:40.105 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:40.105 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:40.105 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:40.106 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.106 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.106 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6772 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1781760~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:40.106 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:54:40.106 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 128.30707572599999, 2024-05-08T08:54:40.106 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052601799999999997, 2024-05-08T08:54:40.106 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:40.106 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:40.106 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:40.106 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:40.106 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:40.106 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:40.106 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6772 2024-05-08T08:54:40.106 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.106 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:40.106 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.106 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:40.107 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:54:40.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:40.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.107 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:40.107 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:54:40.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:40.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.107 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:40.107 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726939+0000", 2024-05-08T08:54:40.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001730493 2024-05-08T08:54:40.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.107 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:40.107 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726943+0000", 2024-05-08T08:54:40.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7330000000000003e-06 2024-05-08T08:54:40.108 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.108 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.108 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:40.108 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726954+0000", 2024-05-08T08:54:40.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1107000000000001e-05 2024-05-08T08:54:40.108 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.108 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.108 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:40.108 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726962+0000", 2024-05-08T08:54:40.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.503e-06 2024-05-08T08:54:40.108 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.108 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.108 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:40.108 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777653+0000", 2024-05-08T08:54:40.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050691542999999999 2024-05-08T08:54:40.108 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.109 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.109 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:40.109 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777719+0000", 2024-05-08T08:54:40.109 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5228000000000006e-05 2024-05-08T08:54:40.109 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.109 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.109 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:40.109 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777810+0000", 2024-05-08T08:54:40.109 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1193000000000006e-05 2024-05-08T08:54:40.109 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:40.109 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:40.109 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:40.109 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.109 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.109 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6782 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1880064~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:40.109 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:54:40.109 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 128.30510721900001, 2024-05-08T08:54:40.110 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052326492000000002, 2024-05-08T08:54:40.110 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:40.110 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:40.110 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:40.110 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:40.110 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:40.110 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:40.110 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6782 2024-05-08T08:54:40.110 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.110 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:40.110 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.110 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:40.110 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:54:40.110 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:40.110 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.111 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.111 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:40.111 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:54:40.111 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:40.111 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.111 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.111 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:40.111 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728939+0000", 2024-05-08T08:54:40.111 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001762719 2024-05-08T08:54:40.111 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.111 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.111 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:40.111 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728944+0000", 2024-05-08T08:54:40.111 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.746e-06 2024-05-08T08:54:40.112 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.112 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.112 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:40.112 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728955+0000", 2024-05-08T08:54:40.112 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1043000000000001e-05 2024-05-08T08:54:40.112 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.112 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.112 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:40.112 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728963+0000", 2024-05-08T08:54:40.112 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5809999999999999e-06 2024-05-08T08:54:40.112 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.112 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.112 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:40.112 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779320+0000", 2024-05-08T08:54:40.112 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050357290999999998 2024-05-08T08:54:40.113 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.113 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.113 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:40.113 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779401+0000", 2024-05-08T08:54:40.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0574000000000005e-05 2024-05-08T08:54:40.113 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:40.113 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:40.113 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:40.113 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779503+0000", 2024-05-08T08:54:40.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102538 2024-05-08T08:54:40.113 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:40.113 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:40.113 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:40.113 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:40.113 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:40.114 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:40.115 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:40.117+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:40.118 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:54:40.216 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:40.218+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:40.229 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:40.229 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:54:40.229 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:54:40.229 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:40.238 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:54:40.316 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:40.319+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:40.347 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:40.347 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:54:40.347 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:54:40.347 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:54:40.347 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:40.356 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:54:40.357 INFO:teuthology.orchestra.run.smithi059.stderr:marked in osd.5. 2024-05-08T08:54:40.379 INFO:tasks.thrashosds.thrasher:Added osd 5 2024-05-08T08:54:40.417 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:40.420+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:40.471 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:40.471 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:54:40.472 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:54:40.472 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:54:40.472 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.472 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4621.0:9089 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 23339520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:54:40.472 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:54:40.472 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 127.980644915, 2024-05-08T08:54:40.472 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.13814907200000001, 2024-05-08T08:54:40.472 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:40.472 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:40.472 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:54:40.472 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:40.472 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4621", 2024-05-08T08:54:40.472 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:40.472 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9089 2024-05-08T08:54:40.472 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.472 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:40.473 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.473 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:40.473 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:54:40.473 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:40.473 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.473 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.473 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:40.473 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:54:40.473 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:40.473 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.473 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.473 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:40.473 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493005+0000", 2024-05-08T08:54:40.473 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1700000000000001e-06 2024-05-08T08:54:40.473 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.473 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.474 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:40.474 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.513963+0000", 2024-05-08T08:54:40.474 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.020957581999999999 2024-05-08T08:54:40.474 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.474 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.474 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:40.474 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.513995+0000", 2024-05-08T08:54:40.474 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1993999999999999e-05 2024-05-08T08:54:40.474 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.474 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.474 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:40.474 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514007+0000", 2024-05-08T08:54:40.474 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1581e-05 2024-05-08T08:54:40.475 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.475 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.475 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:40.475 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514122+0000", 2024-05-08T08:54:40.475 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000115282 2024-05-08T08:54:40.475 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.475 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.475 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op must wait for map", 2024-05-08T08:54:40.475 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514128+0000", 2024-05-08T08:54:40.475 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9950000000000002e-06 2024-05-08T08:54:40.475 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.475 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.475 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:40.475 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.515800+0000", 2024-05-08T08:54:40.476 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001671886 2024-05-08T08:54:40.476 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.476 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.476 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:40.476 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.515893+0000", 2024-05-08T08:54:40.476 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3490000000000001e-05 2024-05-08T08:54:40.476 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.476 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.476 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 3", 2024-05-08T08:54:40.476 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.517647+0000", 2024-05-08T08:54:40.476 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0017535630000000001 2024-05-08T08:54:40.476 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.476 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.476 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:54:40.477 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.627578+0000", 2024-05-08T08:54:40.477 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.109931557 2024-05-08T08:54:40.477 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.477 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.477 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:54:40.477 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631055+0000", 2024-05-08T08:54:40.477 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0034761420000000002 2024-05-08T08:54:40.477 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.477 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.477 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:54:40.477 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631096+0000", 2024-05-08T08:54:40.477 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1059000000000001e-05 2024-05-08T08:54:40.477 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.477 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.478 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:40.478 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631151+0000", 2024-05-08T08:54:40.478 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5770999999999998e-05 2024-05-08T08:54:40.478 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:40.478 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:40.478 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:40.478 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.478 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.478 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1071 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11026432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:40.478 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:54:40.478 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 123.988195424, 2024-05-08T08:54:40.478 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070580037999999998, 2024-05-08T08:54:40.478 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:40.478 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:40.478 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:40.478 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:40.478 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:40.479 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:40.479 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1071 2024-05-08T08:54:40.479 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.479 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:40.479 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.479 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:40.479 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:54:40.479 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:40.479 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.479 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.479 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:40.479 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:54:40.479 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:40.479 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.479 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.479 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:40.480 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:54:40.480 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2800000000000002e-07 2024-05-08T08:54:40.480 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.480 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.480 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:40.480 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485454+0000", 2024-05-08T08:54:40.480 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2290000000000002e-06 2024-05-08T08:54:40.480 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.480 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.480 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:40.480 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485461+0000", 2024-05-08T08:54:40.480 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9920000000000001e-06 2024-05-08T08:54:40.480 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.480 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.480 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:40.481 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485467+0000", 2024-05-08T08:54:40.481 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3979999999999998e-06 2024-05-08T08:54:40.481 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.481 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.481 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:40.481 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555880+0000", 2024-05-08T08:54:40.481 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070413165 2024-05-08T08:54:40.481 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.481 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.481 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:40.481 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555926+0000", 2024-05-08T08:54:40.481 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6217000000000002e-05 2024-05-08T08:54:40.481 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.481 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.481 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:40.481 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556032+0000", 2024-05-08T08:54:40.482 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000105609 2024-05-08T08:54:40.482 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:40.482 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:40.482 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:40.482 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.482 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.482 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1073 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11067392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:40.482 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:54:40.482 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 123.988174203, 2024-05-08T08:54:40.482 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070821404000000004, 2024-05-08T08:54:40.482 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:40.482 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:40.482 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:40.482 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:40.482 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:40.483 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:40.483 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1073 2024-05-08T08:54:40.483 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.483 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:40.483 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.483 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:40.483 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:54:40.483 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:40.483 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.483 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.483 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:40.483 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:54:40.483 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:40.483 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.483 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.483 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:40.483 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:54:40.484 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.08e-07 2024-05-08T08:54:40.484 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.484 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.484 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:40.484 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485476+0000", 2024-05-08T08:54:40.484 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3929999999999998e-06 2024-05-08T08:54:40.484 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.484 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.484 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:40.484 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485485+0000", 2024-05-08T08:54:40.484 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1880000000000008e-06 2024-05-08T08:54:40.484 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.484 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.484 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:40.484 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485494+0000", 2024-05-08T08:54:40.485 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3109999999999995e-06 2024-05-08T08:54:40.485 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.485 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.485 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:40.485 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556082+0000", 2024-05-08T08:54:40.485 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070587426999999994 2024-05-08T08:54:40.485 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.485 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.485 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:40.485 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556177+0000", 2024-05-08T08:54:40.485 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5051999999999997e-05 2024-05-08T08:54:40.485 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.485 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.486 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:40.486 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556294+0000", 2024-05-08T08:54:40.486 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000117625 2024-05-08T08:54:40.486 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:40.486 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:40.486 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:40.486 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.486 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.486 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1075 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11108352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:40.486 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:54:40.486 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 123.988143626, 2024-05-08T08:54:40.486 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071045026999999997, 2024-05-08T08:54:40.486 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:40.486 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:40.487 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:40.487 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:40.487 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:40.487 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:40.487 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1075 2024-05-08T08:54:40.487 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.487 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:40.487 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.487 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:40.487 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:54:40.487 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:40.487 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.487 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.487 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:40.487 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:54:40.487 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:40.488 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.488 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.488 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:40.488 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:54:40.488 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.32e-07 2024-05-08T08:54:40.488 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.488 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.488 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:40.488 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485518+0000", 2024-05-08T08:54:40.488 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3769e-05 2024-05-08T08:54:40.488 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.488 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.488 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:40.488 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485527+0000", 2024-05-08T08:54:40.488 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6779999999999995e-06 2024-05-08T08:54:40.488 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.489 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.489 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:40.489 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485532+0000", 2024-05-08T08:54:40.489 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9830000000000004e-06 2024-05-08T08:54:40.489 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.489 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.489 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:40.489 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556340+0000", 2024-05-08T08:54:40.489 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070807743000000006 2024-05-08T08:54:40.489 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.489 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.489 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:40.489 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556435+0000", 2024-05-08T08:54:40.489 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4732e-05 2024-05-08T08:54:40.489 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.489 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.489 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:40.490 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556549+0000", 2024-05-08T08:54:40.490 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011369 2024-05-08T08:54:40.490 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:40.490 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:40.490 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:40.490 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.490 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.490 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1078 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11149312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:40.490 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:54:40.490 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 123.988103427, 2024-05-08T08:54:40.490 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071167276000000002, 2024-05-08T08:54:40.490 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:40.490 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:40.490 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:40.490 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:40.490 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:40.491 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:40.491 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1078 2024-05-08T08:54:40.491 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.491 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:40.491 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.491 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:40.491 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:54:40.491 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:40.491 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.491 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.491 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:40.491 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:54:40.491 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:40.491 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.491 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.491 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:40.491 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:54:40.492 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5600000000000001e-07 2024-05-08T08:54:40.492 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.492 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.492 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:40.492 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485547+0000", 2024-05-08T08:54:40.492 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3590000000000002e-06 2024-05-08T08:54:40.492 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.492 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.492 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:40.492 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485553+0000", 2024-05-08T08:54:40.492 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2770000000000002e-06 2024-05-08T08:54:40.492 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.492 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.493 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:40.493 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485558+0000", 2024-05-08T08:54:40.493 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.95e-06 2024-05-08T08:54:40.493 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.493 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.493 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:40.493 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556582+0000", 2024-05-08T08:54:40.493 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071024444000000006 2024-05-08T08:54:40.493 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.493 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.493 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:40.493 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556637+0000", 2024-05-08T08:54:40.493 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4672000000000001e-05 2024-05-08T08:54:40.493 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.493 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.493 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:40.493 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556711+0000", 2024-05-08T08:54:40.494 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4117999999999997e-05 2024-05-08T08:54:40.494 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:40.494 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:40.494 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:40.494 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.494 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.494 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1080 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11190272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:40.494 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:54:40.494 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 123.988082694, 2024-05-08T08:54:40.494 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071350721000000006, 2024-05-08T08:54:40.494 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:40.494 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:40.494 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:40.494 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:40.494 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:40.495 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:40.495 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1080 2024-05-08T08:54:40.495 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.495 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:40.495 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.495 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:40.495 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:54:40.495 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:40.495 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.495 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.495 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:40.495 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:54:40.495 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:40.495 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.495 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.495 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:40.495 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:54:40.496 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.03e-07 2024-05-08T08:54:40.496 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.496 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.496 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:40.496 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485567+0000", 2024-05-08T08:54:40.496 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1090000000000001e-06 2024-05-08T08:54:40.496 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.496 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.496 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:40.496 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485573+0000", 2024-05-08T08:54:40.496 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3049999999999998e-06 2024-05-08T08:54:40.496 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.496 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.496 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:40.496 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485578+0000", 2024-05-08T08:54:40.496 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0130000000000001e-06 2024-05-08T08:54:40.497 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.497 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.497 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:40.497 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556743+0000", 2024-05-08T08:54:40.497 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071164449000000005 2024-05-08T08:54:40.497 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.497 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.497 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:40.497 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556797+0000", 2024-05-08T08:54:40.497 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4589999999999997e-05 2024-05-08T08:54:40.497 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.497 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.497 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:40.497 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556915+0000", 2024-05-08T08:54:40.497 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000117852 2024-05-08T08:54:40.498 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:40.498 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:40.498 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:40.498 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.498 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.498 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1082 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11231232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:40.498 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:54:40.498 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 123.988062451, 2024-05-08T08:54:40.498 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071591396000000002, 2024-05-08T08:54:40.498 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:40.498 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:40.498 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:40.498 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:40.498 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:40.498 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:40.499 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1082 2024-05-08T08:54:40.499 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.499 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:40.499 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.499 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:40.499 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:54:40.499 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:40.499 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.499 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.499 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:40.499 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:54:40.499 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:40.499 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.499 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.499 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:40.499 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:54:40.500 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1800000000000001e-07 2024-05-08T08:54:40.500 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.500 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.500 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:40.500 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485587+0000", 2024-05-08T08:54:40.500 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2589999999999999e-06 2024-05-08T08:54:40.500 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.500 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.500 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:40.500 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485594+0000", 2024-05-08T08:54:40.500 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0789999999999997e-06 2024-05-08T08:54:40.500 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.500 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.500 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:40.500 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485598+0000", 2024-05-08T08:54:40.501 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7480000000000001e-06 2024-05-08T08:54:40.501 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.501 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.501 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:40.501 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556971+0000", 2024-05-08T08:54:40.501 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071372774999999999 2024-05-08T08:54:40.501 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.501 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.501 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:40.501 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557069+0000", 2024-05-08T08:54:40.501 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.7732999999999994e-05 2024-05-08T08:54:40.501 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.501 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.501 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:40.501 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557176+0000", 2024-05-08T08:54:40.502 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000107384 2024-05-08T08:54:40.502 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:40.502 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:40.502 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:40.502 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.502 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.502 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1084 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11272192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:40.502 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:54:40.502 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 123.98802381199999, 2024-05-08T08:54:40.502 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071808977999999996, 2024-05-08T08:54:40.502 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:40.502 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:40.502 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:40.502 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:40.503 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:40.503 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:40.503 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1084 2024-05-08T08:54:40.503 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.503 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:40.503 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.503 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:40.503 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:54:40.503 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:40.503 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.503 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.503 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:40.503 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:54:40.503 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:40.503 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.503 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.503 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:40.504 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485624+0000", 2024-05-08T08:54:40.504 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:54:40.504 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.504 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.504 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:40.504 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485626+0000", 2024-05-08T08:54:40.504 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9460000000000001e-06 2024-05-08T08:54:40.504 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.504 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.504 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:40.504 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485633+0000", 2024-05-08T08:54:40.504 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1149999999999996e-06 2024-05-08T08:54:40.504 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.504 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.504 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:40.504 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485638+0000", 2024-05-08T08:54:40.504 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5670000000000002e-06 2024-05-08T08:54:40.505 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.505 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.505 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:40.505 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557226+0000", 2024-05-08T08:54:40.505 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071587267999999996 2024-05-08T08:54:40.505 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.505 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.505 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:40.505 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557321+0000", 2024-05-08T08:54:40.505 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5785000000000005e-05 2024-05-08T08:54:40.505 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.505 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.505 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:40.505 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557432+0000", 2024-05-08T08:54:40.505 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011092800000000001 2024-05-08T08:54:40.505 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:40.506 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:40.506 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:40.506 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.506 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.506 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1086 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11313152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:40.506 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:54:40.506 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 123.988000485, 2024-05-08T08:54:40.506 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071933132999999996, 2024-05-08T08:54:40.506 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:40.506 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:40.506 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:40.506 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:40.506 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:40.506 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:40.506 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1086 2024-05-08T08:54:40.507 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.507 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:40.507 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.507 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:40.507 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:54:40.507 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:40.507 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.507 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.507 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:40.507 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:54:40.507 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:40.507 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.508 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.508 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:40.508 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486328+0000", 2024-05-08T08:54:40.508 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00068107900000000002 2024-05-08T08:54:40.508 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.508 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.508 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:40.508 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486331+0000", 2024-05-08T08:54:40.508 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.585e-06 2024-05-08T08:54:40.508 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.508 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.508 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:40.508 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486343+0000", 2024-05-08T08:54:40.508 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1856e-05 2024-05-08T08:54:40.508 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.509 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.509 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:40.509 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486350+0000", 2024-05-08T08:54:40.509 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8600000000000004e-06 2024-05-08T08:54:40.509 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.509 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.509 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:40.509 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557470+0000", 2024-05-08T08:54:40.509 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071120326999999997 2024-05-08T08:54:40.509 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.509 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.509 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:40.509 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557514+0000", 2024-05-08T08:54:40.509 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3909000000000003e-05 2024-05-08T08:54:40.509 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.509 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.510 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:40.510 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557580+0000", 2024-05-08T08:54:40.510 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5517000000000003e-05 2024-05-08T08:54:40.510 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:40.510 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:40.510 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:40.510 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.510 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.510 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1089 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11354112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:40.510 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:54:40.510 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 123.98728761300001, 2024-05-08T08:54:40.510 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071457381, 2024-05-08T08:54:40.510 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:40.511 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:40.511 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:40.511 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:40.511 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:40.511 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:40.511 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1089 2024-05-08T08:54:40.511 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.511 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:40.511 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.511 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:40.511 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:54:40.511 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:40.511 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.511 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.511 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:40.511 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:54:40.512 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:40.512 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.512 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.512 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:40.512 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486851+0000", 2024-05-08T08:54:40.512 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00049102 2024-05-08T08:54:40.512 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.512 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.512 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:40.512 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486854+0000", 2024-05-08T08:54:40.512 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6859999999999999e-06 2024-05-08T08:54:40.512 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.512 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.512 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:40.512 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486864+0000", 2024-05-08T08:54:40.512 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6080000000000002e-06 2024-05-08T08:54:40.512 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.513 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.513 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:40.513 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486870+0000", 2024-05-08T08:54:40.513 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.477e-06 2024-05-08T08:54:40.513 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.513 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.513 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:40.513 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557612+0000", 2024-05-08T08:54:40.513 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070741557999999996 2024-05-08T08:54:40.513 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.513 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.513 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:40.513 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557695+0000", 2024-05-08T08:54:40.513 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3300000000000005e-05 2024-05-08T08:54:40.513 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.514 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.514 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:40.514 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557817+0000", 2024-05-08T08:54:40.514 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012173200000000001 2024-05-08T08:54:40.514 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:40.514 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:40.514 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:40.514 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.514 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.514 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1091 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11395072~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:40.514 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:54:40.514 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 123.98676681800001, 2024-05-08T08:54:40.514 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071184737999999997, 2024-05-08T08:54:40.514 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:40.515 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:40.515 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:40.515 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:40.515 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:40.515 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:40.515 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1091 2024-05-08T08:54:40.515 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.515 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:40.515 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.515 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:40.515 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:54:40.515 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:40.515 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.515 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.516 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:40.516 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:54:40.516 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:40.516 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.516 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.516 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:40.516 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486881+0000", 2024-05-08T08:54:40.516 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0699999999999997e-07 2024-05-08T08:54:40.516 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.516 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.516 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:40.516 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486883+0000", 2024-05-08T08:54:40.516 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.402e-06 2024-05-08T08:54:40.517 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.517 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.517 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:40.517 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486890+0000", 2024-05-08T08:54:40.517 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0480000000000001e-06 2024-05-08T08:54:40.517 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.517 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.517 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:40.517 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486896+0000", 2024-05-08T08:54:40.517 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7640000000000002e-06 2024-05-08T08:54:40.518 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.518 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.518 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:40.518 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557860+0000", 2024-05-08T08:54:40.518 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070963791999999998 2024-05-08T08:54:40.518 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.518 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.518 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:40.518 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557952+0000", 2024-05-08T08:54:40.518 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2082000000000004e-05 2024-05-08T08:54:40.518 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.518 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.518 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:40.518 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558065+0000", 2024-05-08T08:54:40.519 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000113143 2024-05-08T08:54:40.519 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:40.519 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:40.519 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:40.519 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.519 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.519 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1094 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11436032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:40.519 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:54:40.519 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 123.98674364999999, 2024-05-08T08:54:40.519 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071400507000000002, 2024-05-08T08:54:40.519 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:40.519 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:40.519 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:40.519 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:40.519 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:40.520 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:40.520 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1094 2024-05-08T08:54:40.520 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.520 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:40.520 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.520 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:40.520 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:54:40.520 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:40.520 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.520 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.520 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:40.520 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:54:40.521 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:40.521 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.521 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.521 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:40.521 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487940+0000", 2024-05-08T08:54:40.521 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001036868 2024-05-08T08:54:40.521 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.521 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.521 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:40.521 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487944+0000", 2024-05-08T08:54:40.521 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7069999999999998e-06 2024-05-08T08:54:40.521 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.521 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.522 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:40.522 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487951+0000", 2024-05-08T08:54:40.522 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2649999999999999e-06 2024-05-08T08:54:40.522 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.522 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.522 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:40.522 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487961+0000", 2024-05-08T08:54:40.522 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4110000000000002e-06 2024-05-08T08:54:40.522 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.522 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.522 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:40.522 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558117+0000", 2024-05-08T08:54:40.522 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070156662999999994 2024-05-08T08:54:40.522 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.523 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.523 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:40.523 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558214+0000", 2024-05-08T08:54:40.523 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6444e-05 2024-05-08T08:54:40.523 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.523 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.523 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:40.523 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558304+0000", 2024-05-08T08:54:40.523 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0148999999999994e-05 2024-05-08T08:54:40.523 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:40.523 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:40.523 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:40.523 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.523 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.523 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1096 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11476992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:40.523 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:54:40.524 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 123.985677119, 2024-05-08T08:54:40.524 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070480074000000004, 2024-05-08T08:54:40.524 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:40.524 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:40.524 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:40.524 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:40.524 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:40.524 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:40.524 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1096 2024-05-08T08:54:40.524 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.524 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:40.524 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.524 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:40.524 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:54:40.524 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:40.524 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.524 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.525 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:40.525 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:54:40.525 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:40.525 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.525 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.525 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:40.525 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:54:40.525 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3700000000000001e-07 2024-05-08T08:54:40.525 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.525 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.525 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:40.525 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487973+0000", 2024-05-08T08:54:40.525 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0870000000000001e-06 2024-05-08T08:54:40.525 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.526 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.526 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:40.526 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487977+0000", 2024-05-08T08:54:40.526 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0600000000000001e-06 2024-05-08T08:54:40.526 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.526 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.526 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:40.526 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487980+0000", 2024-05-08T08:54:40.526 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1089999999999998e-06 2024-05-08T08:54:40.526 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.526 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.526 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:40.526 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558338+0000", 2024-05-08T08:54:40.526 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070358233000000006 2024-05-08T08:54:40.526 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.527 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.527 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:40.527 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558392+0000", 2024-05-08T08:54:40.527 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4537999999999999e-05 2024-05-08T08:54:40.527 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.527 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.527 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:40.527 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558450+0000", 2024-05-08T08:54:40.527 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7710000000000001e-05 2024-05-08T08:54:40.527 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:40.527 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:40.527 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:40.527 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.527 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.528 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1099 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11517952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:40.528 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:54:40.528 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 123.985663243, 2024-05-08T08:54:40.528 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070720506000000002, 2024-05-08T08:54:40.528 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:40.528 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:40.528 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:40.528 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:40.528 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:40.528 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:40.528 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1099 2024-05-08T08:54:40.528 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.528 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:40.528 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.529 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:40.529 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:54:40.529 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:40.529 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.529 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.529 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:40.529 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:54:40.529 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:40.529 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.529 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.529 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:40.529 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:54:40.529 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3999999999999998e-07 2024-05-08T08:54:40.529 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.529 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.530 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:40.530 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487986+0000", 2024-05-08T08:54:40.530 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3519999999999999e-06 2024-05-08T08:54:40.530 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.530 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.530 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:40.530 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487989+0000", 2024-05-08T08:54:40.530 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.23e-06 2024-05-08T08:54:40.530 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.530 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.530 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:40.530 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487991+0000", 2024-05-08T08:54:40.530 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6350000000000002e-06 2024-05-08T08:54:40.530 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.530 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.531 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:40.531 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558489+0000", 2024-05-08T08:54:40.531 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070497924000000003 2024-05-08T08:54:40.531 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.531 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.531 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:40.531 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558583+0000", 2024-05-08T08:54:40.531 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3405000000000004e-05 2024-05-08T08:54:40.531 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.531 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.531 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:40.531 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558704+0000", 2024-05-08T08:54:40.531 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012171999999999999 2024-05-08T08:54:40.531 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:40.531 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:40.531 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:40.531 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.532 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.532 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1100 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11558912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:40.532 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:54:40.532 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 123.985650244, 2024-05-08T08:54:40.532 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070962266999999996, 2024-05-08T08:54:40.532 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:40.532 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:40.532 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:40.532 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:40.532 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:40.532 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:40.533 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1100 2024-05-08T08:54:40.533 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.533 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:40.533 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.533 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:40.533 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:54:40.533 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:40.533 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.533 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.533 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:40.534 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:54:40.534 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:40.534 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.534 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.534 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:40.534 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488430+0000", 2024-05-08T08:54:40.534 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000432676 2024-05-08T08:54:40.534 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.535 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.535 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:40.535 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488433+0000", 2024-05-08T08:54:40.535 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.45e-06 2024-05-08T08:54:40.535 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.535 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.535 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:40.535 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488442+0000", 2024-05-08T08:54:40.535 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.687e-06 2024-05-08T08:54:40.536 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.536 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.536 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:40.536 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488446+0000", 2024-05-08T08:54:40.536 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4939999999999997e-06 2024-05-08T08:54:40.536 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.536 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.536 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:40.536 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558756+0000", 2024-05-08T08:54:40.536 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070310051999999998 2024-05-08T08:54:40.537 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.537 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.537 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:40.537 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558844+0000", 2024-05-08T08:54:40.537 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7690999999999998e-05 2024-05-08T08:54:40.537 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.537 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.537 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:40.537 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558959+0000", 2024-05-08T08:54:40.538 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000115217 2024-05-08T08:54:40.538 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:40.538 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:40.538 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:40.538 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.538 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.539 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1103 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11599872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:40.539 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:54:40.539 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 123.98519256900001, 2024-05-08T08:54:40.539 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070777100999999995, 2024-05-08T08:54:40.539 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:40.539 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:40.539 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:40.539 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:40.539 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:40.540 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:40.540 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1103 2024-05-08T08:54:40.540 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.540 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:40.540 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.540 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:40.540 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:54:40.540 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:40.540 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.541 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.541 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:40.541 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:54:40.541 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:40.541 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.541 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.541 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:40.541 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488959+0000", 2024-05-08T08:54:40.541 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00050457200000000001 2024-05-08T08:54:40.542 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.542 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.542 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:40.542 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488969+0000", 2024-05-08T08:54:40.542 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.7159999999999998e-06 2024-05-08T08:54:40.542 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.542 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.542 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:40.542 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488981+0000", 2024-05-08T08:54:40.543 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2150000000000001e-05 2024-05-08T08:54:40.543 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.543 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.543 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:40.543 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488988+0000", 2024-05-08T08:54:40.543 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8759999999999999e-06 2024-05-08T08:54:40.543 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.543 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.544 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:40.544 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558992+0000", 2024-05-08T08:54:40.544 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070004358000000003 2024-05-08T08:54:40.544 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.544 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.544 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:40.544 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559093+0000", 2024-05-08T08:54:40.544 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000100299 2024-05-08T08:54:40.544 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.545 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.545 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:40.545 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559232+0000", 2024-05-08T08:54:40.545 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013913 2024-05-08T08:54:40.545 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:40.545 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:40.545 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:40.545 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.546 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.546 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1104 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11640832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:40.546 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:54:40.546 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 123.98464775399999, 2024-05-08T08:54:40.546 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070521947000000001, 2024-05-08T08:54:40.546 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:40.546 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:40.546 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:40.546 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:40.546 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:40.546 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:40.546 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1104 2024-05-08T08:54:40.547 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.547 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:40.547 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.547 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:40.547 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:54:40.547 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:40.547 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.547 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.547 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:40.547 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:54:40.547 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:40.547 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.547 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.547 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:40.547 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489700+0000", 2024-05-08T08:54:40.547 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00070067600000000001 2024-05-08T08:54:40.547 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.548 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.548 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:40.548 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489704+0000", 2024-05-08T08:54:40.548 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1640000000000003e-06 2024-05-08T08:54:40.548 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.548 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.548 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:40.548 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489716+0000", 2024-05-08T08:54:40.548 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1379000000000001e-05 2024-05-08T08:54:40.548 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.548 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.548 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:40.549 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489724+0000", 2024-05-08T08:54:40.549 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8949999999999998e-06 2024-05-08T08:54:40.549 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.549 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.549 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:40.549 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559278+0000", 2024-05-08T08:54:40.549 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.069554151999999994 2024-05-08T08:54:40.549 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.549 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.549 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:40.549 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559387+0000", 2024-05-08T08:54:40.549 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000109258 2024-05-08T08:54:40.549 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.549 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.549 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:40.550 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559521+0000", 2024-05-08T08:54:40.550 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000134423 2024-05-08T08:54:40.550 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:40.550 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:40.550 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:40.550 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.550 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.550 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:8636 3.17 3:e8a0a3e4:::rbd_header.11402b26385:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-05-08T08:54:40.550 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:54.150971+0000", 2024-05-08T08:54:40.550 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 106.32267576, 2024-05-08T08:54:40.550 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.217273233, 2024-05-08T08:54:40.550 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:40.550 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:40.550 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:40.550 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:40.550 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:54:40.550 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:54:40.551 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 8636 2024-05-08T08:54:40.551 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.551 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:40.551 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.551 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:40.551 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150971+0000", 2024-05-08T08:54:40.551 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:40.551 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.551 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.551 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:40.551 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150971+0000", 2024-05-08T08:54:40.551 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:40.551 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.551 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.551 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:40.551 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150972+0000", 2024-05-08T08:54:40.551 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.71e-07 2024-05-08T08:54:40.552 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.552 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.552 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:40.552 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150977+0000", 2024-05-08T08:54:40.552 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.152e-06 2024-05-08T08:54:40.552 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.552 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.552 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:40.552 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150990+0000", 2024-05-08T08:54:40.552 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2364999999999999e-05 2024-05-08T08:54:40.552 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.552 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.552 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:40.552 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150998+0000", 2024-05-08T08:54:40.552 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0739999999999993e-06 2024-05-08T08:54:40.552 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.552 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.553 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:40.553 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.151054+0000", 2024-05-08T08:54:40.553 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5949000000000003e-05 2024-05-08T08:54:40.553 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.553 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.553 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:54:40.553 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.151060+0000", 2024-05-08T08:54:40.553 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6120000000000004e-06 2024-05-08T08:54:40.553 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.553 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.553 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:40.553 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.367640+0000", 2024-05-08T08:54:40.553 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2165792739999999 2024-05-08T08:54:40.553 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.553 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.553 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:40.553 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.367839+0000", 2024-05-08T08:54:40.554 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000199185 2024-05-08T08:54:40.554 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.554 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.554 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:40.554 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.368245+0000", 2024-05-08T08:54:40.554 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00040575099999999998 2024-05-08T08:54:40.554 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:40.554 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:40.554 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:40.554 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.554 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.554 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5302.0:1516 3.c 3:319bf789:::rbd_data.1140d3fc54c.0000000000000025:head [stat,write 1892352~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:54:40.554 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:27.822220+0000", 2024-05-08T08:54:40.554 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 72.651426822999994, 2024-05-08T08:54:40.554 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.092643744, 2024-05-08T08:54:40.555 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:40.555 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:40.555 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:54:40.555 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:40.555 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5302", 2024-05-08T08:54:40.555 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/359277859", 2024-05-08T08:54:40.555 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1516 2024-05-08T08:54:40.555 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.555 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:40.555 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.555 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:40.555 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822220+0000", 2024-05-08T08:54:40.555 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:40.556 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.556 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.556 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:40.556 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822220+0000", 2024-05-08T08:54:40.556 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:40.556 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.556 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.556 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:40.556 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822221+0000", 2024-05-08T08:54:40.556 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3500000000000002e-07 2024-05-08T08:54:40.556 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.556 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.556 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:40.556 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822228+0000", 2024-05-08T08:54:40.556 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9959999999999996e-06 2024-05-08T08:54:40.556 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.557 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.557 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:40.557 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822234+0000", 2024-05-08T08:54:40.557 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5429999999999999e-06 2024-05-08T08:54:40.557 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.557 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.557 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:40.557 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822238+0000", 2024-05-08T08:54:40.557 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.732e-06 2024-05-08T08:54:40.557 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.557 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.557 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:40.557 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822285+0000", 2024-05-08T08:54:40.557 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6810000000000001e-05 2024-05-08T08:54:40.557 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.558 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.558 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:40.558 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822358+0000", 2024-05-08T08:54:40.558 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3473000000000006e-05 2024-05-08T08:54:40.558 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.558 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.558 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 5", 2024-05-08T08:54:40.558 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822489+0000", 2024-05-08T08:54:40.558 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013011199999999999 2024-05-08T08:54:40.558 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.558 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.558 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:54:40.558 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.825332+0000", 2024-05-08T08:54:40.558 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0028434799999999998 2024-05-08T08:54:40.558 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.559 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.559 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:54:40.559 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.914753+0000", 2024-05-08T08:54:40.559 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089421424999999999 2024-05-08T08:54:40.559 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.559 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.559 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:54:40.559 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.914807+0000", 2024-05-08T08:54:40.559 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3829000000000002e-05 2024-05-08T08:54:40.559 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.559 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.560 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:40.560 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.914864+0000", 2024-05-08T08:54:40.560 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6909e-05 2024-05-08T08:54:40.560 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:40.560 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:40.560 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:40.560 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.560 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.560 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:24567 3.6 3:66af1c40:::rbd_data.1140d3fc54c.000000000000005a:18 [read 2080768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:40.560 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.216598+0000", 2024-05-08T08:54:40.560 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 68.257048742999999, 2024-05-08T08:54:40.560 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071602647000000005, 2024-05-08T08:54:40.560 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:40.560 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:40.560 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:40.560 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:40.560 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:40.561 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:40.561 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 24567 2024-05-08T08:54:40.561 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.561 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:40.561 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.561 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:40.561 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216598+0000", 2024-05-08T08:54:40.561 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:40.561 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.561 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.561 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:40.561 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216598+0000", 2024-05-08T08:54:40.561 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:40.561 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.561 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.561 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:40.562 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216599+0000", 2024-05-08T08:54:40.562 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8999999999999997e-07 2024-05-08T08:54:40.562 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.562 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.562 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:40.562 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216601+0000", 2024-05-08T08:54:40.562 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9889999999999999e-06 2024-05-08T08:54:40.562 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.562 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.562 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:40.562 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216607+0000", 2024-05-08T08:54:40.562 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3600000000000001e-06 2024-05-08T08:54:40.562 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.562 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.562 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:40.563 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216612+0000", 2024-05-08T08:54:40.563 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.69e-06 2024-05-08T08:54:40.563 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.563 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.563 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:40.563 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.287076+0000", 2024-05-08T08:54:40.563 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070464355000000006 2024-05-08T08:54:40.563 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.563 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.563 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:40.563 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.287151+0000", 2024-05-08T08:54:40.563 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4924000000000003e-05 2024-05-08T08:54:40.563 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.563 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.563 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:40.563 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.288201+0000", 2024-05-08T08:54:40.563 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0010498389999999999 2024-05-08T08:54:40.564 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:40.564 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:40.564 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:40.564 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:40.564 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:40.564 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:40.565 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:54:40.566 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:40.568+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:40.666 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:40.669+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:40.675 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:40.675 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:54:40.675 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:54:40.675 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:40.684 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:54:40.767 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:40.770+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:40.794 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:40.794 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:54:40.794 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:54:40.794 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:54:40.794 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:40.803 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:54:40.868 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:40.871+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:40.955 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:40.955 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:54:40.955 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:54:40.955 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:54:40.955 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.955 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:12443 3.0 3:02d0158e:::rbd_data.114071bcf828.0000000000000037:head [stat,delete] snapc 19=[19] RETRY=3 ondisk+retry+write+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:40.955 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.610027+0000", 2024-05-08T08:54:40.955 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 69.346938448000003, 2024-05-08T08:54:40.955 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.083586333999999998, 2024-05-08T08:54:40.955 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:40.956 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:40.956 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:40.956 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:40.956 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:40.956 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:40.956 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 12443 2024-05-08T08:54:40.956 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.956 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:40.956 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.956 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:40.956 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610027+0000", 2024-05-08T08:54:40.956 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:40.956 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.956 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.956 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:40.956 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610027+0000", 2024-05-08T08:54:40.957 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:40.957 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.957 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.957 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:40.957 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610028+0000", 2024-05-08T08:54:40.957 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.13e-07 2024-05-08T08:54:40.957 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.957 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.957 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:40.957 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610030+0000", 2024-05-08T08:54:40.957 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7669999999999999e-06 2024-05-08T08:54:40.957 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.957 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.957 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:40.957 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610039+0000", 2024-05-08T08:54:40.957 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4959999999999996e-06 2024-05-08T08:54:40.958 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.958 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.958 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:40.958 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610045+0000", 2024-05-08T08:54:40.958 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6779999999999997e-06 2024-05-08T08:54:40.958 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.958 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.958 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:40.958 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.660873+0000", 2024-05-08T08:54:40.959 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.050828748 2024-05-08T08:54:40.959 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.959 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.959 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:40.959 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.661017+0000", 2024-05-08T08:54:40.959 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014391999999999999 2024-05-08T08:54:40.959 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.959 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.959 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:40.959 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.693613+0000", 2024-05-08T08:54:40.959 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.032596111999999997 2024-05-08T08:54:40.959 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:40.959 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:40.959 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:40.959 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.960 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.960 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:14301 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [watch reconnect cookie 139837155901536 gen 12] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:40.960 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.645078+0000", 2024-05-08T08:54:40.960 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 69.311887810000002, 2024-05-08T08:54:40.960 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088281852999999993, 2024-05-08T08:54:40.960 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:40.960 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:40.960 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:40.960 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:40.960 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:40.960 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:40.960 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14301 2024-05-08T08:54:40.960 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.960 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:40.960 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.960 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:40.961 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645078+0000", 2024-05-08T08:54:40.961 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:40.961 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.961 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.961 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:40.961 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645078+0000", 2024-05-08T08:54:40.961 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:40.961 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.961 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.961 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:40.961 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645084+0000", 2024-05-08T08:54:40.961 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4620000000000001e-06 2024-05-08T08:54:40.961 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.961 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.961 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:40.961 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645088+0000", 2024-05-08T08:54:40.961 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8039999999999999e-06 2024-05-08T08:54:40.962 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.962 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.962 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:40.962 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645106+0000", 2024-05-08T08:54:40.962 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.8318999999999999e-05 2024-05-08T08:54:40.962 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.962 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.962 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:40.962 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645113+0000", 2024-05-08T08:54:40.962 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0310000000000001e-06 2024-05-08T08:54:40.962 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.962 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.962 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:40.962 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.670493+0000", 2024-05-08T08:54:40.962 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.025379993 2024-05-08T08:54:40.962 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.963 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.963 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:40.963 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.670571+0000", 2024-05-08T08:54:40.963 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7613000000000004e-05 2024-05-08T08:54:40.963 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.963 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.963 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:40.963 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733360+0000", 2024-05-08T08:54:40.963 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.062788630999999998 2024-05-08T08:54:40.963 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:40.963 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:40.963 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:40.963 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.963 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.963 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:14420 3.0 3:06a8948e:::rbd_data.114071bcf828.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:40.963 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.649639+0000", 2024-05-08T08:54:40.964 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 69.307326661999994, 2024-05-08T08:54:40.964 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.083851897999999994, 2024-05-08T08:54:40.964 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:40.964 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:40.964 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:54:40.964 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:40.964 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:40.964 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:40.964 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14420 2024-05-08T08:54:40.964 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.964 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:40.964 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.964 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:40.964 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649639+0000", 2024-05-08T08:54:40.964 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:40.965 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.965 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.965 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:40.965 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649639+0000", 2024-05-08T08:54:40.965 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:40.965 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.965 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.965 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:40.965 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649640+0000", 2024-05-08T08:54:40.965 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6899999999999997e-07 2024-05-08T08:54:40.965 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.965 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.965 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:40.965 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649895+0000", 2024-05-08T08:54:40.965 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00025531000000000002 2024-05-08T08:54:40.966 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.966 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.966 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:40.966 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649905+0000", 2024-05-08T08:54:40.966 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0198e-05 2024-05-08T08:54:40.966 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.966 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.966 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:40.966 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649911+0000", 2024-05-08T08:54:40.966 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2310000000000001e-06 2024-05-08T08:54:40.966 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.966 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.966 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:40.966 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.684099+0000", 2024-05-08T08:54:40.966 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.034188017000000001 2024-05-08T08:54:40.966 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.967 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.967 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:40.967 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.684172+0000", 2024-05-08T08:54:40.967 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3201999999999995e-05 2024-05-08T08:54:40.967 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.967 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.967 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 2", 2024-05-08T08:54:40.967 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.684418+0000", 2024-05-08T08:54:40.967 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00024559099999999997 2024-05-08T08:54:40.967 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.967 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.967 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:54:40.967 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.705538+0000", 2024-05-08T08:54:40.967 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.021119707000000001 2024-05-08T08:54:40.967 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.967 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.967 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:54:40.968 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733399+0000", 2024-05-08T08:54:40.968 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.027861543999999999 2024-05-08T08:54:40.968 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.968 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.968 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:54:40.968 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733445+0000", 2024-05-08T08:54:40.968 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5587000000000002e-05 2024-05-08T08:54:40.968 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.968 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.968 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:40.968 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733491+0000", 2024-05-08T08:54:40.968 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5841999999999999e-05 2024-05-08T08:54:40.968 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:40.968 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:40.969 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:40.969 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.969 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.969 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28547 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889179216] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:40.969 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.355654+0000", 2024-05-08T08:54:40.969 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 68.601311433999996, 2024-05-08T08:54:40.969 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078692261999999999, 2024-05-08T08:54:40.969 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:40.969 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:40.969 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:40.969 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:40.969 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:40.969 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:40.969 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28547 2024-05-08T08:54:40.969 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.969 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:40.969 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.970 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:40.970 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355654+0000", 2024-05-08T08:54:40.970 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:40.970 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.970 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.970 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:40.970 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355654+0000", 2024-05-08T08:54:40.970 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:40.970 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.970 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.970 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:40.970 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355654+0000", 2024-05-08T08:54:40.970 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3999999999999998e-07 2024-05-08T08:54:40.970 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.970 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.970 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:40.971 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355656+0000", 2024-05-08T08:54:40.971 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.353e-06 2024-05-08T08:54:40.971 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.971 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.971 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:40.971 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355663+0000", 2024-05-08T08:54:40.971 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2259999999999999e-06 2024-05-08T08:54:40.971 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.971 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.971 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:40.971 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355666+0000", 2024-05-08T08:54:40.971 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7930000000000002e-06 2024-05-08T08:54:40.971 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.971 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.972 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:40.972 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434117+0000", 2024-05-08T08:54:40.972 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078451231999999996 2024-05-08T08:54:40.972 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.972 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.972 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:40.972 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434198+0000", 2024-05-08T08:54:40.972 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0879999999999998e-05 2024-05-08T08:54:40.972 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.972 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.972 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:40.972 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434346+0000", 2024-05-08T08:54:40.972 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014853800000000001 2024-05-08T08:54:40.972 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:40.972 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:40.973 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:40.973 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.973 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.973 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28548 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889189696] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:40.973 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.355669+0000", 2024-05-08T08:54:40.973 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 68.601296083999998, 2024-05-08T08:54:40.973 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078956606999999998, 2024-05-08T08:54:40.973 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:40.973 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:40.973 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:40.973 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:40.973 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:40.973 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:40.973 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28548 2024-05-08T08:54:40.973 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.973 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:40.973 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.974 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:40.974 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355669+0000", 2024-05-08T08:54:40.974 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:40.974 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.974 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.974 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:40.974 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355669+0000", 2024-05-08T08:54:40.974 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:40.974 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.974 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.974 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:40.974 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355762+0000", 2024-05-08T08:54:40.974 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3015000000000003e-05 2024-05-08T08:54:40.974 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.974 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.974 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:40.975 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355765+0000", 2024-05-08T08:54:40.975 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.368e-06 2024-05-08T08:54:40.975 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.975 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.975 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:40.975 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355771+0000", 2024-05-08T08:54:40.975 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5989999999999998e-06 2024-05-08T08:54:40.975 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.975 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.975 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:40.975 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355775+0000", 2024-05-08T08:54:40.975 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7280000000000001e-06 2024-05-08T08:54:40.975 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.975 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.975 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:40.975 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434379+0000", 2024-05-08T08:54:40.975 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078603662000000005 2024-05-08T08:54:40.976 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.976 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.976 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:40.976 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434473+0000", 2024-05-08T08:54:40.976 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4104000000000006e-05 2024-05-08T08:54:40.976 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.976 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.976 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:40.976 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434626+0000", 2024-05-08T08:54:40.976 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00015313100000000001 2024-05-08T08:54:40.976 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:40.976 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:40.976 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:40.976 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.976 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.976 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28549 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889202400] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:40.977 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.355781+0000", 2024-05-08T08:54:40.977 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 68.601184509999996, 2024-05-08T08:54:40.977 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079117946999999994, 2024-05-08T08:54:40.977 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:40.977 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:40.977 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:40.977 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:40.977 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:40.977 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:40.977 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28549 2024-05-08T08:54:40.978 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.978 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:40.978 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.978 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:40.978 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355781+0000", 2024-05-08T08:54:40.978 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:40.978 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.978 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.978 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:40.978 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355781+0000", 2024-05-08T08:54:40.978 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:40.978 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.978 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.978 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:40.978 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356162+0000", 2024-05-08T08:54:40.978 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00038099100000000001 2024-05-08T08:54:40.979 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.979 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.979 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:40.979 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356165+0000", 2024-05-08T08:54:40.979 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1240000000000001e-06 2024-05-08T08:54:40.979 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.979 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.979 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:40.979 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356173+0000", 2024-05-08T08:54:40.979 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.6199999999999999e-06 2024-05-08T08:54:40.979 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.979 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.979 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:40.979 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356182+0000", 2024-05-08T08:54:40.979 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7760000000000003e-06 2024-05-08T08:54:40.979 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.979 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.980 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:40.980 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434675+0000", 2024-05-08T08:54:40.980 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078493119 2024-05-08T08:54:40.980 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.980 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.980 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:40.980 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434765+0000", 2024-05-08T08:54:40.980 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0402000000000006e-05 2024-05-08T08:54:40.980 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.980 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.980 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:40.980 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434899+0000", 2024-05-08T08:54:40.980 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013391499999999999 2024-05-08T08:54:40.980 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:40.980 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:40.980 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:40.980 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.981 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.981 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28550 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888597280] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:40.981 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:54:40.981 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 68.600773464, 2024-05-08T08:54:40.981 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078933990999999995, 2024-05-08T08:54:40.981 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:40.981 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:40.981 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:40.981 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:40.981 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:40.981 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:40.981 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28550 2024-05-08T08:54:40.981 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.981 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:40.981 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.981 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:40.981 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:54:40.982 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:40.982 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.982 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.982 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:40.982 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:54:40.982 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:40.982 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.982 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.982 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:40.982 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:54:40.982 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.34e-07 2024-05-08T08:54:40.982 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.982 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.982 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:40.983 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356195+0000", 2024-05-08T08:54:40.983 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8710000000000001e-06 2024-05-08T08:54:40.983 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.983 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.983 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:40.983 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356200+0000", 2024-05-08T08:54:40.983 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3000000000000003e-06 2024-05-08T08:54:40.983 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.983 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.983 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:40.983 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356203+0000", 2024-05-08T08:54:40.983 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4039999999999999e-06 2024-05-08T08:54:40.983 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.984 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.984 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:40.984 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434928+0000", 2024-05-08T08:54:40.984 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078725331999999995 2024-05-08T08:54:40.984 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.984 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.984 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:40.984 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435007+0000", 2024-05-08T08:54:40.984 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8415000000000001e-05 2024-05-08T08:54:40.984 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.984 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.984 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:40.984 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435126+0000", 2024-05-08T08:54:40.984 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011923500000000001 2024-05-08T08:54:40.984 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:40.985 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:40.985 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:40.985 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.985 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.985 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28551 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888527200] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:40.985 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.356210+0000", 2024-05-08T08:54:40.985 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 68.600755116000002, 2024-05-08T08:54:40.985 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079195072000000005, 2024-05-08T08:54:40.985 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:40.985 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:40.985 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:40.985 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:40.985 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:40.985 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:40.985 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28551 2024-05-08T08:54:40.985 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.985 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:40.986 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.986 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:40.986 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356210+0000", 2024-05-08T08:54:40.986 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:40.986 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.986 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.986 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:40.986 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356210+0000", 2024-05-08T08:54:40.986 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:40.986 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.986 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.986 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:40.986 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356211+0000", 2024-05-08T08:54:40.986 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4499999999999998e-07 2024-05-08T08:54:40.986 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.986 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.986 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:40.987 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356212+0000", 2024-05-08T08:54:40.987 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6029999999999999e-06 2024-05-08T08:54:40.987 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.987 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.987 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:40.987 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356221+0000", 2024-05-08T08:54:40.987 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4200000000000007e-06 2024-05-08T08:54:40.987 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.987 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.987 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:40.987 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356224+0000", 2024-05-08T08:54:40.987 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.038e-06 2024-05-08T08:54:40.987 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.987 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.987 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:40.987 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435178+0000", 2024-05-08T08:54:40.988 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078954625000000001 2024-05-08T08:54:40.988 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.988 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.988 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:40.988 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435256+0000", 2024-05-08T08:54:40.988 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7337999999999997e-05 2024-05-08T08:54:40.988 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.988 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.988 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:40.988 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435405+0000", 2024-05-08T08:54:40.988 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000149703 2024-05-08T08:54:40.988 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:40.988 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:40.988 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:40.988 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.988 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.989 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28552 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889231456] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:40.989 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.356228+0000", 2024-05-08T08:54:40.989 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 68.600737437999996, 2024-05-08T08:54:40.989 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079346920000000001, 2024-05-08T08:54:40.989 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:40.989 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:40.989 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:40.989 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:40.989 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:40.989 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:40.989 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28552 2024-05-08T08:54:40.989 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.989 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:40.989 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.990 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:40.990 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356228+0000", 2024-05-08T08:54:40.990 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:40.990 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.990 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.990 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:40.990 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356228+0000", 2024-05-08T08:54:40.990 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:40.990 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.990 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.990 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:40.990 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356959+0000", 2024-05-08T08:54:40.990 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00073097099999999996 2024-05-08T08:54:40.990 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.990 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.991 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:40.991 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356967+0000", 2024-05-08T08:54:40.991 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0409999999999998e-06 2024-05-08T08:54:40.991 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.991 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.991 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:40.991 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356979+0000", 2024-05-08T08:54:40.991 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1489e-05 2024-05-08T08:54:40.991 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.991 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.991 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:40.991 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356986+0000", 2024-05-08T08:54:40.991 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3699999999999997e-06 2024-05-08T08:54:40.991 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.991 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.991 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:40.991 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435433+0000", 2024-05-08T08:54:40.992 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078447520000000007 2024-05-08T08:54:40.992 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.992 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.992 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:40.992 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435484+0000", 2024-05-08T08:54:40.992 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0229000000000003e-05 2024-05-08T08:54:40.992 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.992 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.992 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:40.992 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435575+0000", 2024-05-08T08:54:40.992 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1299999999999997e-05 2024-05-08T08:54:40.992 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:40.992 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:40.992 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:40.992 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.992 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.992 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28553 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888713648] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:40.993 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:54:40.993 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 68.599962779999998, 2024-05-08T08:54:40.993 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078782034000000001, 2024-05-08T08:54:40.993 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:40.993 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:40.993 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:40.993 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:40.993 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:40.993 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:40.993 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28553 2024-05-08T08:54:40.993 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.994 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:40.994 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.994 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:40.994 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:54:40.994 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:40.994 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.994 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.994 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:40.994 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:54:40.994 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:40.994 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.994 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.994 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:40.994 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:54:40.994 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6600000000000002e-07 2024-05-08T08:54:40.994 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.995 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.995 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:40.995 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357006+0000", 2024-05-08T08:54:40.995 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7190000000000001e-06 2024-05-08T08:54:40.995 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.995 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.995 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:40.995 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357013+0000", 2024-05-08T08:54:40.995 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.435e-06 2024-05-08T08:54:40.995 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.995 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.995 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:40.995 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357018+0000", 2024-05-08T08:54:40.995 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7260000000000002e-06 2024-05-08T08:54:40.996 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.996 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.996 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:40.996 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435623+0000", 2024-05-08T08:54:40.996 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078604972999999995 2024-05-08T08:54:40.996 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.996 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.996 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:40.996 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435695+0000", 2024-05-08T08:54:40.996 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1959999999999995e-05 2024-05-08T08:54:40.996 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.996 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.996 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:40.996 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435785+0000", 2024-05-08T08:54:40.996 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.9754999999999997e-05 2024-05-08T08:54:40.996 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:40.996 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:40.997 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:40.997 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.997 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.997 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28554 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888980272] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:40.997 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357024+0000", 2024-05-08T08:54:40.997 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 68.599941143999999, 2024-05-08T08:54:40.997 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078999870999999999, 2024-05-08T08:54:40.997 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:40.997 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:40.997 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:40.997 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:40.997 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:40.997 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:40.997 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28554 2024-05-08T08:54:40.997 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.997 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:40.998 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.998 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:40.998 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357024+0000", 2024-05-08T08:54:40.998 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:40.998 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.998 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.998 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:40.998 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357024+0000", 2024-05-08T08:54:40.998 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:40.998 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.998 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.998 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:40.998 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357025+0000", 2024-05-08T08:54:40.998 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:54:40.998 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.998 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.999 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:40.999 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357027+0000", 2024-05-08T08:54:40.999 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2050000000000001e-06 2024-05-08T08:54:40.999 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.999 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.999 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:40.999 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357033+0000", 2024-05-08T08:54:40.999 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5679999999999996e-06 2024-05-08T08:54:40.999 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.999 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:40.999 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:40.999 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357038+0000", 2024-05-08T08:54:40.999 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2889999999999999e-06 2024-05-08T08:54:40.999 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:40.999 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.000 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:41.000 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435823+0000", 2024-05-08T08:54:41.000 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078785677999999998 2024-05-08T08:54:41.000 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.000 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.000 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:41.000 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435892+0000", 2024-05-08T08:54:41.000 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8379000000000004e-05 2024-05-08T08:54:41.000 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.000 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.000 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:41.000 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436024+0000", 2024-05-08T08:54:41.000 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013239799999999999 2024-05-08T08:54:41.000 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:41.000 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:41.000 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:41.000 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.001 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.001 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28555 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889001104] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:41.001 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:54:41.001 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 68.599922379000006, 2024-05-08T08:54:41.001 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079115742000000003, 2024-05-08T08:54:41.001 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:41.001 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:41.001 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:41.001 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:41.001 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:41.001 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:41.001 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28555 2024-05-08T08:54:41.001 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.001 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:41.001 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.001 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:41.001 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:54:41.002 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:41.002 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.002 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.002 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:41.002 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:54:41.002 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:41.002 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.002 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.002 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:41.002 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:54:41.002 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9200000000000002e-07 2024-05-08T08:54:41.002 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.002 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.002 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:41.002 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357046+0000", 2024-05-08T08:54:41.002 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1670000000000002e-06 2024-05-08T08:54:41.003 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.003 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.003 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:41.003 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357052+0000", 2024-05-08T08:54:41.003 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.139e-06 2024-05-08T08:54:41.003 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.003 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.003 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:41.003 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357056+0000", 2024-05-08T08:54:41.003 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2440000000000004e-06 2024-05-08T08:54:41.003 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.003 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.003 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:41.004 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436059+0000", 2024-05-08T08:54:41.004 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079003070999999994 2024-05-08T08:54:41.004 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.004 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.004 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:41.004 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436097+0000", 2024-05-08T08:54:41.004 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7784000000000003e-05 2024-05-08T08:54:41.004 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.004 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.004 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:41.004 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436159+0000", 2024-05-08T08:54:41.004 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2045000000000007e-05 2024-05-08T08:54:41.004 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:41.004 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:41.004 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:41.004 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.005 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.005 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28556 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889022064] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:41.005 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357065+0000", 2024-05-08T08:54:41.005 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 68.599900345999998, 2024-05-08T08:54:41.005 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079327976999999994, 2024-05-08T08:54:41.005 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:41.005 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:41.005 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:41.005 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:41.005 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:41.005 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:41.005 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28556 2024-05-08T08:54:41.005 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.005 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:41.005 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.005 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:41.006 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357065+0000", 2024-05-08T08:54:41.006 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:41.006 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.006 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.006 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:41.006 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357065+0000", 2024-05-08T08:54:41.006 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:41.006 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.006 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.006 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:41.006 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358958+0000", 2024-05-08T08:54:41.006 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001892915 2024-05-08T08:54:41.006 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.006 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.006 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:41.006 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358962+0000", 2024-05-08T08:54:41.007 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4599999999999999e-06 2024-05-08T08:54:41.007 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.007 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.007 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:41.007 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358970+0000", 2024-05-08T08:54:41.007 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.6090000000000001e-06 2024-05-08T08:54:41.007 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.007 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.007 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:41.007 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358975+0000", 2024-05-08T08:54:41.007 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7639999999999996e-06 2024-05-08T08:54:41.007 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.007 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.007 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:41.007 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436196+0000", 2024-05-08T08:54:41.008 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.077221564000000006 2024-05-08T08:54:41.008 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.008 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.008 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:41.008 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436269+0000", 2024-05-08T08:54:41.008 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2720999999999997e-05 2024-05-08T08:54:41.008 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.008 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.008 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:41.008 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436393+0000", 2024-05-08T08:54:41.008 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012394399999999999 2024-05-08T08:54:41.008 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:41.008 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:41.008 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:41.008 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.008 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.008 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28563 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888855520] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:41.009 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:54:41.009 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 68.597890946999996, 2024-05-08T08:54:41.009 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078795566999999997, 2024-05-08T08:54:41.009 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:41.009 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:41.009 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:41.009 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:41.009 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:41.009 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:41.009 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28563 2024-05-08T08:54:41.009 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.009 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:41.009 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.009 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:41.009 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:54:41.009 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:41.009 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.010 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.010 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:41.010 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:54:41.010 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:41.010 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.010 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.010 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:41.010 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:54:41.010 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1e-07 2024-05-08T08:54:41.010 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.010 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.010 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:41.010 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359080+0000", 2024-05-08T08:54:41.010 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8940000000000001e-06 2024-05-08T08:54:41.010 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.011 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.011 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:41.011 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359083+0000", 2024-05-08T08:54:41.011 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1949999999999999e-06 2024-05-08T08:54:41.011 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.011 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.011 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:41.011 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359085+0000", 2024-05-08T08:54:41.011 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4710000000000001e-06 2024-05-08T08:54:41.011 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.011 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.011 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:41.011 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437604+0000", 2024-05-08T08:54:41.011 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078518278999999996 2024-05-08T08:54:41.011 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.011 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.012 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:41.012 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437704+0000", 2024-05-08T08:54:41.012 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010010599999999999 2024-05-08T08:54:41.012 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.012 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.012 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:41.012 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437870+0000", 2024-05-08T08:54:41.012 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00016641200000000001 2024-05-08T08:54:41.012 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:41.012 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:41.012 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:41.012 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.012 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.012 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28564 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888538944] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:41.012 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.359089+0000", 2024-05-08T08:54:41.012 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 68.597876732000003, 2024-05-08T08:54:41.012 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078952675, 2024-05-08T08:54:41.013 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:41.013 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:41.013 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:41.013 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:41.013 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:41.013 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:41.013 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28564 2024-05-08T08:54:41.013 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.013 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:41.013 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.013 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:41.013 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359089+0000", 2024-05-08T08:54:41.013 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:41.013 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.013 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.013 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:41.013 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359089+0000", 2024-05-08T08:54:41.014 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:41.014 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.014 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.014 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:41.014 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360959+0000", 2024-05-08T08:54:41.014 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0018707089999999999 2024-05-08T08:54:41.014 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.014 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.014 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:41.014 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360963+0000", 2024-05-08T08:54:41.014 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0680000000000001e-06 2024-05-08T08:54:41.015 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.015 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.015 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:41.015 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360969+0000", 2024-05-08T08:54:41.015 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1920000000000002e-06 2024-05-08T08:54:41.015 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.015 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.015 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:41.015 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360973+0000", 2024-05-08T08:54:41.015 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.031e-06 2024-05-08T08:54:41.015 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.015 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.015 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:41.015 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437916+0000", 2024-05-08T08:54:41.015 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.076942939000000002 2024-05-08T08:54:41.015 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.016 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.016 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:41.016 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437973+0000", 2024-05-08T08:54:41.016 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7358999999999997e-05 2024-05-08T08:54:41.016 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.016 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.016 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:41.016 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.438041+0000", 2024-05-08T08:54:41.016 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8376999999999999e-05 2024-05-08T08:54:41.016 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:41.016 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:41.016 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:41.016 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.016 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.016 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28574 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889427600] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:41.016 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:54:41.016 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 68.595872224999994, 2024-05-08T08:54:41.017 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078782476000000004, 2024-05-08T08:54:41.017 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:41.017 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:41.017 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:41.017 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:41.017 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:41.017 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:41.017 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28574 2024-05-08T08:54:41.017 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.017 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:41.017 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.017 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:41.017 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:54:41.017 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:41.017 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.017 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.018 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:41.018 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:54:41.018 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:41.018 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.018 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.018 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:41.018 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:54:41.018 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1799999999999999e-07 2024-05-08T08:54:41.018 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.018 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.018 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:41.018 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361095+0000", 2024-05-08T08:54:41.018 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3850000000000001e-06 2024-05-08T08:54:41.018 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.018 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.018 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:41.019 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361098+0000", 2024-05-08T08:54:41.019 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7989999999999998e-06 2024-05-08T08:54:41.019 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.019 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.019 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:41.019 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361100+0000", 2024-05-08T08:54:41.019 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4669999999999998e-06 2024-05-08T08:54:41.019 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.019 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.019 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:41.019 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439768+0000", 2024-05-08T08:54:41.019 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078668057999999999 2024-05-08T08:54:41.019 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.019 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.019 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:41.019 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439811+0000", 2024-05-08T08:54:41.019 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2459000000000001e-05 2024-05-08T08:54:41.020 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.020 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.020 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:41.020 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439876+0000", 2024-05-08T08:54:41.020 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5090000000000002e-05 2024-05-08T08:54:41.020 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:41.020 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:41.020 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:41.020 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.020 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.020 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28575 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889439216] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:41.020 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361103+0000", 2024-05-08T08:54:41.020 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 68.595862210000007, 2024-05-08T08:54:41.020 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078966828000000003, 2024-05-08T08:54:41.020 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:41.021 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:41.021 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:41.021 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:41.021 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:41.021 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:41.021 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28575 2024-05-08T08:54:41.021 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.021 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:41.021 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.021 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:41.021 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361103+0000", 2024-05-08T08:54:41.021 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:41.021 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.021 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.021 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:41.022 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361103+0000", 2024-05-08T08:54:41.022 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:41.022 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.022 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.022 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:41.022 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361104+0000", 2024-05-08T08:54:41.022 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1400000000000001e-07 2024-05-08T08:54:41.022 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.022 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.022 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:41.022 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361105+0000", 2024-05-08T08:54:41.022 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5540000000000001e-06 2024-05-08T08:54:41.022 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.022 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.022 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:41.023 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361108+0000", 2024-05-08T08:54:41.023 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.841e-06 2024-05-08T08:54:41.023 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.023 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.023 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:41.023 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361110+0000", 2024-05-08T08:54:41.023 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.497e-06 2024-05-08T08:54:41.023 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.023 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.023 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:41.023 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439916+0000", 2024-05-08T08:54:41.023 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078805372999999998 2024-05-08T08:54:41.023 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.023 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.023 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:41.023 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440000+0000", 2024-05-08T08:54:41.023 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4730999999999999e-05 2024-05-08T08:54:41.024 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.024 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.024 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:41.024 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440070+0000", 2024-05-08T08:54:41.024 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9617999999999997e-05 2024-05-08T08:54:41.024 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:41.024 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:41.024 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:41.024 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.024 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.024 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28576 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889451328] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:41.025 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361113+0000", 2024-05-08T08:54:41.025 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 68.595852023999996, 2024-05-08T08:54:41.025 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079088862999999995, 2024-05-08T08:54:41.025 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:41.025 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:41.025 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:41.025 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:41.025 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:41.025 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:41.025 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28576 2024-05-08T08:54:41.025 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.025 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:41.025 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.025 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:41.025 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361113+0000", 2024-05-08T08:54:41.025 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:41.025 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.026 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.026 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:41.026 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361113+0000", 2024-05-08T08:54:41.026 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:41.026 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.026 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.026 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:41.026 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361114+0000", 2024-05-08T08:54:41.026 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1299999999999999e-07 2024-05-08T08:54:41.026 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.026 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.026 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:41.026 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361115+0000", 2024-05-08T08:54:41.026 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.4300000000000001e-06 2024-05-08T08:54:41.026 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.026 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.027 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:41.027 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361118+0000", 2024-05-08T08:54:41.027 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9000000000000002e-06 2024-05-08T08:54:41.027 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.027 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.027 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:41.027 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361120+0000", 2024-05-08T08:54:41.027 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4739999999999999e-06 2024-05-08T08:54:41.027 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.027 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.027 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:41.027 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440096+0000", 2024-05-08T08:54:41.027 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078975629000000006 2024-05-08T08:54:41.027 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.027 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.027 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:41.028 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440135+0000", 2024-05-08T08:54:41.028 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9202000000000002e-05 2024-05-08T08:54:41.028 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.028 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.028 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:41.028 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440202+0000", 2024-05-08T08:54:41.028 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7014999999999994e-05 2024-05-08T08:54:41.028 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:41.028 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:41.028 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:41.028 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.028 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.028 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28577 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889462640] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:41.028 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:54:41.028 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 68.595841841999999, 2024-05-08T08:54:41.028 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079194622000000006, 2024-05-08T08:54:41.028 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:41.029 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:41.029 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:41.029 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:41.029 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:41.029 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:41.029 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28577 2024-05-08T08:54:41.029 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.029 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:41.029 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.029 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:41.029 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:54:41.029 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:41.029 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.029 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.029 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:41.029 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:54:41.029 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:41.030 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.030 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.030 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:41.030 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:54:41.030 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1400000000000001e-07 2024-05-08T08:54:41.030 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.030 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.030 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:41.030 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361125+0000", 2024-05-08T08:54:41.030 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.372e-06 2024-05-08T08:54:41.030 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.030 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.030 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:41.030 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361128+0000", 2024-05-08T08:54:41.030 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0309999999999999e-06 2024-05-08T08:54:41.030 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.031 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.031 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:41.031 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361131+0000", 2024-05-08T08:54:41.031 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4040000000000002e-06 2024-05-08T08:54:41.031 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.031 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.031 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:41.031 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440225+0000", 2024-05-08T08:54:41.031 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079094600000000001 2024-05-08T08:54:41.031 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.031 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.031 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:41.031 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440260+0000", 2024-05-08T08:54:41.031 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4703000000000003e-05 2024-05-08T08:54:41.031 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.031 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.031 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:41.032 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440318+0000", 2024-05-08T08:54:41.032 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8298000000000003e-05 2024-05-08T08:54:41.032 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:41.032 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:41.032 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:41.032 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.032 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.032 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28578 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889474192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:41.032 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361134+0000", 2024-05-08T08:54:41.032 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 68.595831416999999, 2024-05-08T08:54:41.032 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079310295000000003, 2024-05-08T08:54:41.032 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:41.032 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:41.032 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:41.032 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:41.032 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:41.033 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:41.033 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28578 2024-05-08T08:54:41.033 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.033 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:41.033 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.033 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:41.033 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361134+0000", 2024-05-08T08:54:41.033 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:41.033 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.033 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.033 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:41.033 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361134+0000", 2024-05-08T08:54:41.033 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:41.033 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.033 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.033 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:41.033 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.362958+0000", 2024-05-08T08:54:41.033 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0018237380000000001 2024-05-08T08:54:41.034 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.034 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.034 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:41.034 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.362961+0000", 2024-05-08T08:54:41.034 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.123e-06 2024-05-08T08:54:41.034 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.034 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.034 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:41.034 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.397404+0000", 2024-05-08T08:54:41.034 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.034443226 2024-05-08T08:54:41.034 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.034 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.034 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:41.034 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.397439+0000", 2024-05-08T08:54:41.034 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4690000000000002e-05 2024-05-08T08:54:41.035 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.035 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.035 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:41.035 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440340+0000", 2024-05-08T08:54:41.035 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.042901419000000003 2024-05-08T08:54:41.035 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.035 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.035 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:41.035 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440376+0000", 2024-05-08T08:54:41.035 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5444999999999997e-05 2024-05-08T08:54:41.035 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.035 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.035 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:41.036 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440444+0000", 2024-05-08T08:54:41.036 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8653999999999997e-05 2024-05-08T08:54:41.036 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:41.036 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:41.036 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:41.036 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:41.036 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:41.036 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:41.037 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:54:41.038 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:41.040+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:41.138 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:41.141+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:41.187 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:41.187 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:54:41.187 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:54:41.187 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:41.196 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:54:41.239 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:41.242+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:41.305 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:41.306 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:54:41.306 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:54:41.306 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:54:41.306 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:41.314 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:54:41.358 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:41.361+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:41.396 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T08:54:41.431 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:41.431 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:54:41.431 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:54:41.431 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:54:41.431 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.431 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:437 3.17 3:ea93a512:::rbd_data.1140100c9703.0000000000000077:head [set-alloc-hint object_size 2097152 write_size 2097152,write 1170432~1024] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e63)", 2024-05-08T08:54:41.431 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:08.390804+0000", 2024-05-08T08:54:41.431 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 93.041906003999998, 2024-05-08T08:54:41.431 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0406068690000001, 2024-05-08T08:54:41.431 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:41.432 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:41.432 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:54:41.432 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:41.432 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:41.432 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:41.432 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 437 2024-05-08T08:54:41.432 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.432 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:41.432 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.432 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:41.432 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390804+0000", 2024-05-08T08:54:41.432 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:41.432 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.432 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.432 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:41.432 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390804+0000", 2024-05-08T08:54:41.432 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:41.433 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.433 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.433 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:41.433 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390806+0000", 2024-05-08T08:54:41.433 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.666e-06 2024-05-08T08:54:41.433 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.433 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.433 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:41.433 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390818+0000", 2024-05-08T08:54:41.433 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2187999999999999e-05 2024-05-08T08:54:41.433 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.433 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.433 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:41.433 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390834+0000", 2024-05-08T08:54:41.433 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6191000000000001e-05 2024-05-08T08:54:41.433 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.433 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.434 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:41.434 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390845+0000", 2024-05-08T08:54:41.434 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0831e-05 2024-05-08T08:54:41.434 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.434 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.434 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:41.434 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390922+0000", 2024-05-08T08:54:41.434 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7253e-05 2024-05-08T08:54:41.434 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.434 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.434 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op must wait for map", 2024-05-08T08:54:41.434 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390930+0000", 2024-05-08T08:54:41.434 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8259999999999993e-06 2024-05-08T08:54:41.434 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.434 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.435 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:41.435 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392155+0000", 2024-05-08T08:54:41.435 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0012256649999999999 2024-05-08T08:54:41.435 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.435 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.435 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:54:41.435 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392165+0000", 2024-05-08T08:54:41.435 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5799999999999998e-06 2024-05-08T08:54:41.435 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.435 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.435 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:41.435 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.429596+0000", 2024-05-08T08:54:41.435 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0374313639999999 2024-05-08T08:54:41.435 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.435 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.435 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:41.436 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.429728+0000", 2024-05-08T08:54:41.436 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000131552 2024-05-08T08:54:41.436 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.436 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.436 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 0,1", 2024-05-08T08:54:41.436 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.429860+0000", 2024-05-08T08:54:41.436 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000132137 2024-05-08T08:54:41.436 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.436 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.436 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:54:41.436 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.430811+0000", 2024-05-08T08:54:41.436 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00095040200000000004 2024-05-08T08:54:41.436 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.436 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.437 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:54:41.437 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431126+0000", 2024-05-08T08:54:41.437 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000315006 2024-05-08T08:54:41.437 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.437 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.437 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:54:41.437 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431272+0000", 2024-05-08T08:54:41.437 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000146487 2024-05-08T08:54:41.437 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.437 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.437 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:54:41.437 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431315+0000", 2024-05-08T08:54:41.437 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2697000000000001e-05 2024-05-08T08:54:41.437 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.438 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.438 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:41.438 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431411+0000", 2024-05-08T08:54:41.438 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6024000000000001e-05 2024-05-08T08:54:41.438 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:41.438 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:41.438 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:41.438 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.438 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.438 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:439 3.17 3:ea93a512:::rbd_data.1140100c9703.0000000000000077:head [read 1163264~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-05-08T08:54:41.438 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:08.390871+0000", 2024-05-08T08:54:41.438 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 93.041838493, 2024-05-08T08:54:41.439 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0408790590000001, 2024-05-08T08:54:41.439 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:41.439 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:41.439 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:41.439 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:41.439 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:41.439 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:41.439 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 439 2024-05-08T08:54:41.439 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.439 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:41.439 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.439 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:41.439 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390871+0000", 2024-05-08T08:54:41.439 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:41.439 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.440 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.440 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:41.440 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390871+0000", 2024-05-08T08:54:41.440 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:41.440 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.440 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.440 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:41.440 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390872+0000", 2024-05-08T08:54:41.440 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6300000000000005e-07 2024-05-08T08:54:41.440 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.440 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.440 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:41.440 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390875+0000", 2024-05-08T08:54:41.440 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3459999999999998e-06 2024-05-08T08:54:41.440 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.441 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.441 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:41.441 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390891+0000", 2024-05-08T08:54:41.441 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5255e-05 2024-05-08T08:54:41.441 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.441 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.441 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:41.441 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390897+0000", 2024-05-08T08:54:41.441 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2360000000000001e-06 2024-05-08T08:54:41.441 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.441 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.441 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:41.441 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390995+0000", 2024-05-08T08:54:41.441 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8522000000000003e-05 2024-05-08T08:54:41.441 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.442 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.442 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting_for_map not empty", 2024-05-08T08:54:41.442 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.391005+0000", 2024-05-08T08:54:41.442 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1460000000000002e-06 2024-05-08T08:54:41.442 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.442 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.442 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:41.442 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392191+0000", 2024-05-08T08:54:41.442 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0011860270000000001 2024-05-08T08:54:41.442 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.442 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.442 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:54:41.443 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392197+0000", 2024-05-08T08:54:41.443 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4289999999999998e-06 2024-05-08T08:54:41.443 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.443 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.443 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:41.443 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.430596+0000", 2024-05-08T08:54:41.443 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0383989090000001 2024-05-08T08:54:41.443 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.443 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.443 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for rw locks", 2024-05-08T08:54:41.443 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.430676+0000", 2024-05-08T08:54:41.443 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9781000000000001e-05 2024-05-08T08:54:41.443 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.443 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.443 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:41.444 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431460+0000", 2024-05-08T08:54:41.444 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00078427300000000004 2024-05-08T08:54:41.444 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.444 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.444 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:41.444 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431540+0000", 2024-05-08T08:54:41.444 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0224999999999998e-05 2024-05-08T08:54:41.444 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.444 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.444 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:41.444 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431750+0000", 2024-05-08T08:54:41.444 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00021024699999999999 2024-05-08T08:54:41.444 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:41.444 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:41.444 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:41.444 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.444 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.445 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:945 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 4096000~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:41.445 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:54:41.445 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 91.981688926000004, 2024-05-08T08:54:41.445 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086774741000000002, 2024-05-08T08:54:41.445 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:41.445 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:41.445 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:41.445 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:41.445 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:41.445 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:41.445 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 945 2024-05-08T08:54:41.445 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.445 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:41.445 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.445 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:41.446 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:54:41.446 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:41.446 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.446 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.446 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:41.446 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:54:41.446 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:41.446 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.446 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.446 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:41.446 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:54:41.446 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3499999999999996e-07 2024-05-08T08:54:41.446 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.446 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.447 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:41.447 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451024+0000", 2024-05-08T08:54:41.447 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.627e-06 2024-05-08T08:54:41.447 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.447 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.447 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:41.447 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451037+0000", 2024-05-08T08:54:41.447 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2431e-05 2024-05-08T08:54:41.447 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.447 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.447 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:41.447 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451042+0000", 2024-05-08T08:54:41.447 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8409999999999996e-06 2024-05-08T08:54:41.447 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.448 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.448 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:41.448 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537608+0000", 2024-05-08T08:54:41.448 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086565243 2024-05-08T08:54:41.448 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.448 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.448 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:41.448 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537687+0000", 2024-05-08T08:54:41.448 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9519999999999998e-05 2024-05-08T08:54:41.448 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.448 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.448 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:41.449 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537796+0000", 2024-05-08T08:54:41.449 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000108544 2024-05-08T08:54:41.449 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:41.449 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:41.449 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:41.449 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.449 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.449 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:964 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 458752~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:41.449 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451051+0000", 2024-05-08T08:54:41.449 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 91.981658753000005, 2024-05-08T08:54:41.449 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086990490000000004, 2024-05-08T08:54:41.449 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:41.449 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:41.449 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:41.449 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:41.449 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:41.450 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:41.450 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 964 2024-05-08T08:54:41.450 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.450 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:41.450 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.450 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:41.450 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451051+0000", 2024-05-08T08:54:41.450 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:41.450 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.450 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.450 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:41.450 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451051+0000", 2024-05-08T08:54:41.450 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:41.450 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.450 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.451 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:41.451 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451052+0000", 2024-05-08T08:54:41.451 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7299999999999996e-07 2024-05-08T08:54:41.451 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.451 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.451 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:41.451 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451055+0000", 2024-05-08T08:54:41.451 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.2770000000000001e-06 2024-05-08T08:54:41.451 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.451 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.451 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:41.451 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451061+0000", 2024-05-08T08:54:41.451 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4060000000000002e-06 2024-05-08T08:54:41.451 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.451 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.452 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:41.452 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451066+0000", 2024-05-08T08:54:41.452 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0839999999999999e-06 2024-05-08T08:54:41.452 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.452 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.452 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:41.452 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537839+0000", 2024-05-08T08:54:41.452 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086772962999999995 2024-05-08T08:54:41.452 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.452 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.452 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:41.452 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537934+0000", 2024-05-08T08:54:41.452 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4742999999999997e-05 2024-05-08T08:54:41.452 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.452 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.452 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:41.453 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538042+0000", 2024-05-08T08:54:41.453 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000107444 2024-05-08T08:54:41.453 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:41.453 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:41.453 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:41.453 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.453 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.453 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:965 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 950272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:41.453 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451075+0000", 2024-05-08T08:54:41.453 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 91.981634690999996, 2024-05-08T08:54:41.453 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087230196999999995, 2024-05-08T08:54:41.453 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:41.453 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:41.453 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:41.453 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:41.453 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:41.454 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:41.454 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 965 2024-05-08T08:54:41.454 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.454 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:41.454 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.454 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:41.454 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451075+0000", 2024-05-08T08:54:41.454 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:41.454 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.454 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.454 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:41.454 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451075+0000", 2024-05-08T08:54:41.454 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:41.454 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.454 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.455 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:41.455 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451076+0000", 2024-05-08T08:54:41.455 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5000000000000002e-07 2024-05-08T08:54:41.455 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.455 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.455 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:41.455 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451078+0000", 2024-05-08T08:54:41.455 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.5639999999999999e-06 2024-05-08T08:54:41.455 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.455 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.455 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:41.455 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451085+0000", 2024-05-08T08:54:41.455 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5640000000000002e-06 2024-05-08T08:54:41.455 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.455 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.455 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:41.456 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451090+0000", 2024-05-08T08:54:41.456 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1699999999999996e-06 2024-05-08T08:54:41.456 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.456 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.456 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:41.456 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538106+0000", 2024-05-08T08:54:41.456 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087015888999999999 2024-05-08T08:54:41.456 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.456 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.456 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:41.456 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538190+0000", 2024-05-08T08:54:41.456 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3961999999999993e-05 2024-05-08T08:54:41.456 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.456 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.456 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:41.456 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538305+0000", 2024-05-08T08:54:41.456 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011529800000000001 2024-05-08T08:54:41.457 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:41.457 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:41.457 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:41.457 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.457 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.457 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:966 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 1441792~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:41.457 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:54:41.457 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 91.981611318000006, 2024-05-08T08:54:41.457 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087446323000000006, 2024-05-08T08:54:41.457 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:41.457 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:41.457 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:41.457 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:41.457 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:41.457 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:41.457 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 966 2024-05-08T08:54:41.457 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.458 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:41.458 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.458 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:41.458 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:54:41.458 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:41.458 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.458 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.458 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:41.458 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:54:41.458 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:41.458 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.459 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.459 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:41.459 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:54:41.459 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3700000000000001e-07 2024-05-08T08:54:41.459 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.459 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.459 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:41.459 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451101+0000", 2024-05-08T08:54:41.459 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2560000000000001e-06 2024-05-08T08:54:41.459 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.459 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.459 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:41.460 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451107+0000", 2024-05-08T08:54:41.460 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1070000000000001e-06 2024-05-08T08:54:41.460 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.460 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.460 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:41.460 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451112+0000", 2024-05-08T08:54:41.460 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1420000000000001e-06 2024-05-08T08:54:41.460 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.460 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.460 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:41.460 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538341+0000", 2024-05-08T08:54:41.460 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087228302999999993 2024-05-08T08:54:41.460 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.460 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.460 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:41.461 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538424+0000", 2024-05-08T08:54:41.461 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3497000000000004e-05 2024-05-08T08:54:41.461 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.461 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.461 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:41.461 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538545+0000", 2024-05-08T08:54:41.461 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000120581 2024-05-08T08:54:41.461 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:41.461 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:41.461 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:41.461 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.461 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.461 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:967 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:41.462 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451120+0000", 2024-05-08T08:54:41.462 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 91.981589842999995, 2024-05-08T08:54:41.462 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087632725999999994, 2024-05-08T08:54:41.462 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:41.462 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:41.462 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:41.462 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:41.462 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:41.462 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:41.462 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 967 2024-05-08T08:54:41.462 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.462 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:41.462 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.462 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:41.462 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451120+0000", 2024-05-08T08:54:41.463 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:41.463 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.463 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.463 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:41.463 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451120+0000", 2024-05-08T08:54:41.463 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:41.463 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.463 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.463 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:41.463 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451121+0000", 2024-05-08T08:54:41.463 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8800000000000002e-07 2024-05-08T08:54:41.463 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.463 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.463 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:41.463 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451130+0000", 2024-05-08T08:54:41.463 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2299999999999997e-06 2024-05-08T08:54:41.463 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.464 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.464 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:41.464 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451137+0000", 2024-05-08T08:54:41.464 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4669999999999998e-06 2024-05-08T08:54:41.464 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.464 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.464 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:41.464 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451143+0000", 2024-05-08T08:54:41.464 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2839999999999997e-06 2024-05-08T08:54:41.464 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.464 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.464 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:41.464 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538575+0000", 2024-05-08T08:54:41.464 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087432789999999996 2024-05-08T08:54:41.464 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.464 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.464 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:41.465 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538648+0000", 2024-05-08T08:54:41.465 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2877e-05 2024-05-08T08:54:41.465 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.465 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.465 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:41.465 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538753+0000", 2024-05-08T08:54:41.465 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010449 2024-05-08T08:54:41.465 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:41.465 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:41.465 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:41.465 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.465 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.465 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:968 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 2424832~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:41.465 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451152+0000", 2024-05-08T08:54:41.465 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 91.981557757999994, 2024-05-08T08:54:41.465 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087813325999999997, 2024-05-08T08:54:41.465 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:41.465 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:41.466 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:41.466 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:41.466 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:41.466 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:41.466 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 968 2024-05-08T08:54:41.466 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.466 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:41.466 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.466 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:41.466 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451152+0000", 2024-05-08T08:54:41.466 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:41.466 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.467 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.467 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:41.467 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451152+0000", 2024-05-08T08:54:41.467 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:41.467 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.467 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.467 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:41.467 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451153+0000", 2024-05-08T08:54:41.467 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3099999999999998e-07 2024-05-08T08:54:41.467 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.467 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.467 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:41.467 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451155+0000", 2024-05-08T08:54:41.467 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4760000000000001e-06 2024-05-08T08:54:41.467 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.467 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.468 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:41.468 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451162+0000", 2024-05-08T08:54:41.468 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5969999999999997e-06 2024-05-08T08:54:41.468 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.468 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.468 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:41.468 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451167+0000", 2024-05-08T08:54:41.468 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.977e-06 2024-05-08T08:54:41.468 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.468 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.468 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:41.468 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538781+0000", 2024-05-08T08:54:41.468 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087614846999999996 2024-05-08T08:54:41.468 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.468 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.468 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:41.469 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538857+0000", 2024-05-08T08:54:41.469 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5358000000000006e-05 2024-05-08T08:54:41.469 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.469 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.469 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:41.469 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538965+0000", 2024-05-08T08:54:41.469 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010864 2024-05-08T08:54:41.469 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:41.469 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:41.469 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:41.469 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.469 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.469 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:969 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 2916352~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:41.470 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:54:41.470 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 91.981536052999999, 2024-05-08T08:54:41.470 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087998980000000004, 2024-05-08T08:54:41.470 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:41.470 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:41.470 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:41.470 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:41.470 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:41.470 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:41.470 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 969 2024-05-08T08:54:41.470 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.470 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:41.470 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.470 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:41.471 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:54:41.471 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:41.471 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.471 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.471 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:41.471 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:54:41.471 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:41.471 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.471 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.471 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:41.471 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:54:41.471 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3599999999999999e-07 2024-05-08T08:54:41.471 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.471 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.471 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:41.471 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451177+0000", 2024-05-08T08:54:41.471 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4590000000000001e-06 2024-05-08T08:54:41.472 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.472 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.472 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:41.472 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451184+0000", 2024-05-08T08:54:41.472 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4109999999999999e-06 2024-05-08T08:54:41.472 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.472 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.472 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:41.472 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451191+0000", 2024-05-08T08:54:41.472 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7279999999999998e-06 2024-05-08T08:54:41.472 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.472 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.472 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:41.472 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538997+0000", 2024-05-08T08:54:41.473 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087806170000000003 2024-05-08T08:54:41.473 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.473 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.473 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:41.473 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539069+0000", 2024-05-08T08:54:41.473 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2373999999999995e-05 2024-05-08T08:54:41.473 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.473 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.473 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:41.473 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539173+0000", 2024-05-08T08:54:41.473 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000103402 2024-05-08T08:54:41.473 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:41.473 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:41.473 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:41.473 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.473 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.474 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:970 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 3407872~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:41.474 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451198+0000", 2024-05-08T08:54:41.474 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 91.981511456000007, 2024-05-08T08:54:41.474 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088183368999999998, 2024-05-08T08:54:41.474 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:41.474 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:41.474 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:41.474 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:41.474 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:41.474 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:41.474 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 970 2024-05-08T08:54:41.474 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.474 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:41.475 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.475 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:41.475 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451198+0000", 2024-05-08T08:54:41.475 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:41.475 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.475 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.475 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:41.475 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451198+0000", 2024-05-08T08:54:41.475 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:41.475 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.475 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.475 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:41.475 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451199+0000", 2024-05-08T08:54:41.475 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7400000000000003e-07 2024-05-08T08:54:41.475 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.476 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.476 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:41.476 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451211+0000", 2024-05-08T08:54:41.476 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2109999999999999e-05 2024-05-08T08:54:41.476 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.476 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.476 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:41.476 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451219+0000", 2024-05-08T08:54:41.476 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4170000000000003e-06 2024-05-08T08:54:41.476 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.476 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.476 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:41.476 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451224+0000", 2024-05-08T08:54:41.476 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3990000000000003e-06 2024-05-08T08:54:41.476 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.476 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.477 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:41.477 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539204+0000", 2024-05-08T08:54:41.477 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087980099000000006 2024-05-08T08:54:41.477 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.477 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.477 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:41.477 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539276+0000", 2024-05-08T08:54:41.477 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1513000000000005e-05 2024-05-08T08:54:41.477 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.477 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.477 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:41.477 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539382+0000", 2024-05-08T08:54:41.477 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010615699999999999 2024-05-08T08:54:41.477 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:41.477 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:41.477 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:41.477 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.478 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.478 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:971 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 3899392~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:41.478 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451233+0000", 2024-05-08T08:54:41.478 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 91.981476748999995, 2024-05-08T08:54:41.478 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088431525999999996, 2024-05-08T08:54:41.478 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:41.478 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:41.478 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:41.478 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:41.478 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:41.478 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:41.478 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 971 2024-05-08T08:54:41.479 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.479 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:41.479 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.479 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:41.479 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451233+0000", 2024-05-08T08:54:41.479 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:41.479 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.479 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.479 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:41.479 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451233+0000", 2024-05-08T08:54:41.479 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:41.479 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.480 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.480 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:41.480 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451234+0000", 2024-05-08T08:54:41.480 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8500000000000001e-07 2024-05-08T08:54:41.480 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.480 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.480 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:41.480 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451237+0000", 2024-05-08T08:54:41.480 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8320000000000002e-06 2024-05-08T08:54:41.480 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.480 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.480 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:41.480 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451243+0000", 2024-05-08T08:54:41.480 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6749999999999996e-06 2024-05-08T08:54:41.480 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.481 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.481 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:41.481 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451249+0000", 2024-05-08T08:54:41.481 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1229999999999997e-06 2024-05-08T08:54:41.481 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.481 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.481 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:41.481 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539412+0000", 2024-05-08T08:54:41.481 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088162668999999999 2024-05-08T08:54:41.481 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.481 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.481 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:41.481 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539505+0000", 2024-05-08T08:54:41.481 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3240999999999997e-05 2024-05-08T08:54:41.481 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.481 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.481 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:41.482 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539665+0000", 2024-05-08T08:54:41.482 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00015930100000000001 2024-05-08T08:54:41.482 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:41.482 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:41.482 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:41.482 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.482 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.482 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:972 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 196608~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:41.482 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451257+0000", 2024-05-08T08:54:41.482 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 91.981452804, 2024-05-08T08:54:41.482 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088719928000000003, 2024-05-08T08:54:41.482 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:41.482 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:41.482 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:41.482 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:41.482 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:41.483 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:41.483 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 972 2024-05-08T08:54:41.483 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.483 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:41.483 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.483 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:41.483 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451257+0000", 2024-05-08T08:54:41.483 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:41.483 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.483 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.483 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:41.483 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451257+0000", 2024-05-08T08:54:41.483 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:41.483 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.483 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.483 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:41.484 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451258+0000", 2024-05-08T08:54:41.484 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7999999999999996e-07 2024-05-08T08:54:41.484 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.484 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.484 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:41.484 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451260+0000", 2024-05-08T08:54:41.484 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4609999999999998e-06 2024-05-08T08:54:41.484 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.484 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.484 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:41.484 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451267+0000", 2024-05-08T08:54:41.484 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.81e-06 2024-05-08T08:54:41.484 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.484 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.484 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:41.484 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451273+0000", 2024-05-08T08:54:41.485 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0859999999999998e-06 2024-05-08T08:54:41.485 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.485 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.485 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:41.485 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539718+0000", 2024-05-08T08:54:41.485 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088445059000000006 2024-05-08T08:54:41.485 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.485 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.485 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:41.485 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539834+0000", 2024-05-08T08:54:41.485 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000116426 2024-05-08T08:54:41.485 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.486 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.486 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:41.486 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539977+0000", 2024-05-08T08:54:41.486 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014260600000000001 2024-05-08T08:54:41.486 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:41.486 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:41.486 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:41.486 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.486 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.486 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:973 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 688128~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:41.486 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:54:41.486 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 91.981430219999993, 2024-05-08T08:54:41.486 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088918496, 2024-05-08T08:54:41.486 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:41.486 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:41.486 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:41.487 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:41.487 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:41.487 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:41.487 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 973 2024-05-08T08:54:41.487 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.487 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:41.487 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.487 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:41.487 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:54:41.487 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:41.487 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.487 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.487 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:41.487 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:54:41.487 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:41.487 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.487 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.488 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:41.488 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:54:41.488 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9099999999999999e-07 2024-05-08T08:54:41.488 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.488 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.488 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:41.488 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451282+0000", 2024-05-08T08:54:41.488 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1550000000000001e-06 2024-05-08T08:54:41.488 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.488 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.488 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:41.488 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451288+0000", 2024-05-08T08:54:41.488 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8889999999999999e-06 2024-05-08T08:54:41.488 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.488 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.488 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:41.489 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451294+0000", 2024-05-08T08:54:41.489 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0850000000000002e-06 2024-05-08T08:54:41.489 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.489 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.489 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:41.489 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540011+0000", 2024-05-08T08:54:41.489 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088717314000000005 2024-05-08T08:54:41.489 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.489 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.489 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:41.489 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540089+0000", 2024-05-08T08:54:41.489 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7438e-05 2024-05-08T08:54:41.489 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.489 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.489 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:41.489 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540198+0000", 2024-05-08T08:54:41.490 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000109224 2024-05-08T08:54:41.490 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:41.490 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:41.490 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:41.490 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.490 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.490 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:974 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 1179648~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:41.490 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451301+0000", 2024-05-08T08:54:41.490 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 91.981408587000004, 2024-05-08T08:54:41.490 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089105078000000004, 2024-05-08T08:54:41.490 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:41.490 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:41.490 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:41.490 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:41.490 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:41.490 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:41.491 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 974 2024-05-08T08:54:41.491 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.491 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:41.491 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.491 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:41.491 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451301+0000", 2024-05-08T08:54:41.491 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:41.491 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.491 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.491 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:41.491 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451301+0000", 2024-05-08T08:54:41.491 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:41.491 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.491 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.491 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:41.491 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451302+0000", 2024-05-08T08:54:41.491 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7800000000000002e-07 2024-05-08T08:54:41.492 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.492 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.492 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:41.492 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451304+0000", 2024-05-08T08:54:41.492 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.65e-06 2024-05-08T08:54:41.492 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.492 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.492 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:41.492 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451311+0000", 2024-05-08T08:54:41.492 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.472e-06 2024-05-08T08:54:41.492 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.492 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.492 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:41.493 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451316+0000", 2024-05-08T08:54:41.493 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9189999999999998e-06 2024-05-08T08:54:41.493 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.493 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.493 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:41.493 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540234+0000", 2024-05-08T08:54:41.493 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088917752000000003 2024-05-08T08:54:41.493 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.493 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.493 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:41.493 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540308+0000", 2024-05-08T08:54:41.493 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4758000000000005e-05 2024-05-08T08:54:41.493 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.493 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.493 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:41.493 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540406+0000", 2024-05-08T08:54:41.494 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8048999999999996e-05 2024-05-08T08:54:41.494 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:41.494 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:41.494 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:41.494 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.494 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.494 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:975 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 1671168~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:41.494 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:54:41.494 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 91.981366257000005, 2024-05-08T08:54:41.494 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089294139999999994, 2024-05-08T08:54:41.494 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:41.494 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:41.494 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:41.494 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:41.494 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:41.495 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:41.495 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 975 2024-05-08T08:54:41.495 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.495 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:41.495 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.495 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:41.495 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:54:41.495 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:41.495 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.495 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.495 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:41.495 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:54:41.495 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:41.495 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.495 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.495 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:41.496 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:54:41.496 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1299999999999999e-07 2024-05-08T08:54:41.496 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.496 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.496 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:41.496 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451348+0000", 2024-05-08T08:54:41.496 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.484e-06 2024-05-08T08:54:41.496 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.496 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.496 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:41.496 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451356+0000", 2024-05-08T08:54:41.496 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8499999999999994e-06 2024-05-08T08:54:41.496 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.496 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.496 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:41.496 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451366+0000", 2024-05-08T08:54:41.496 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.9850000000000001e-06 2024-05-08T08:54:41.497 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.497 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.497 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:41.497 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540448+0000", 2024-05-08T08:54:41.497 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089082254 2024-05-08T08:54:41.497 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.497 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.497 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:41.497 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540528+0000", 2024-05-08T08:54:41.497 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0234999999999993e-05 2024-05-08T08:54:41.497 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.497 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.497 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:41.497 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540638+0000", 2024-05-08T08:54:41.497 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010951900000000001 2024-05-08T08:54:41.497 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:41.498 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:41.498 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:41.498 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.498 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.498 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:976 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 2162688~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:41.498 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451375+0000", 2024-05-08T08:54:41.498 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 91.981334509999996, 2024-05-08T08:54:41.498 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089472745000000006, 2024-05-08T08:54:41.498 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:41.498 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:41.498 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:41.498 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:41.498 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:41.498 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:41.499 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 976 2024-05-08T08:54:41.499 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.499 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:41.499 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.499 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:41.499 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451375+0000", 2024-05-08T08:54:41.499 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:41.499 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.499 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.499 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:41.499 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451375+0000", 2024-05-08T08:54:41.499 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:41.499 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.499 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.499 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:41.499 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451376+0000", 2024-05-08T08:54:41.500 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2399999999999998e-07 2024-05-08T08:54:41.500 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.500 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.500 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:41.500 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451379+0000", 2024-05-08T08:54:41.500 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6359999999999998e-06 2024-05-08T08:54:41.500 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.500 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.500 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:41.500 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451387+0000", 2024-05-08T08:54:41.500 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9929999999999995e-06 2024-05-08T08:54:41.500 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.500 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.501 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:41.501 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451393+0000", 2024-05-08T08:54:41.501 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4699999999999999e-06 2024-05-08T08:54:41.501 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.501 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.501 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:41.501 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540677+0000", 2024-05-08T08:54:41.501 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089283504999999999 2024-05-08T08:54:41.501 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.501 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.501 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:41.501 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540743+0000", 2024-05-08T08:54:41.501 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6303999999999994e-05 2024-05-08T08:54:41.501 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.501 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.502 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:41.502 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540848+0000", 2024-05-08T08:54:41.502 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000105213 2024-05-08T08:54:41.502 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:41.502 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:41.502 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:41.502 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.502 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.502 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:977 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 2654208~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:41.502 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:54:41.502 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 91.981309077999995, 2024-05-08T08:54:41.502 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089698473000000001, 2024-05-08T08:54:41.502 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:41.502 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:41.503 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:41.503 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:41.503 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:41.503 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:41.503 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 977 2024-05-08T08:54:41.503 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.503 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:41.503 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.503 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:41.503 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:54:41.503 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:41.503 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.503 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.503 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:41.503 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:54:41.503 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:41.503 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.504 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.504 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:41.504 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:54:41.504 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2899999999999999e-07 2024-05-08T08:54:41.504 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.504 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.504 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:41.504 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451404+0000", 2024-05-08T08:54:41.504 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6649999999999999e-06 2024-05-08T08:54:41.504 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.504 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.504 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:41.504 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451410+0000", 2024-05-08T08:54:41.504 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3269999999999997e-06 2024-05-08T08:54:41.504 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.504 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.504 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:41.505 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451415+0000", 2024-05-08T08:54:41.505 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9529999999999998e-06 2024-05-08T08:54:41.505 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.505 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.505 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:41.505 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540912+0000", 2024-05-08T08:54:41.505 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089496950000000006 2024-05-08T08:54:41.505 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.505 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.505 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:41.505 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541000+0000", 2024-05-08T08:54:41.505 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7429000000000006e-05 2024-05-08T08:54:41.505 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.505 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.505 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:41.505 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541099+0000", 2024-05-08T08:54:41.506 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.9720000000000001e-05 2024-05-08T08:54:41.506 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:41.506 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:41.506 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:41.506 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.506 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.506 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:978 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 3145728~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:41.506 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:54:41.506 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 91.981287870000003, 2024-05-08T08:54:41.506 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089900635000000007, 2024-05-08T08:54:41.506 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:41.506 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:41.506 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:41.506 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:41.506 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:41.506 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:41.507 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 978 2024-05-08T08:54:41.507 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.507 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:41.507 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.507 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:41.507 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:54:41.507 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:41.507 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.507 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.507 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:41.507 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:54:41.507 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:41.507 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.507 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.507 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:41.507 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:54:41.507 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9400000000000001e-07 2024-05-08T08:54:41.508 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.508 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.508 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:41.508 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451425+0000", 2024-05-08T08:54:41.508 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2460000000000002e-06 2024-05-08T08:54:41.508 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.508 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.508 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:41.508 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451431+0000", 2024-05-08T08:54:41.508 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2419999999999997e-06 2024-05-08T08:54:41.508 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.508 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.508 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:41.508 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451436+0000", 2024-05-08T08:54:41.509 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.887e-06 2024-05-08T08:54:41.509 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.509 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.509 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:41.509 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541152+0000", 2024-05-08T08:54:41.509 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089716194999999999 2024-05-08T08:54:41.509 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.509 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.509 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:41.509 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541224+0000", 2024-05-08T08:54:41.509 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2416000000000006e-05 2024-05-08T08:54:41.509 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.509 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.510 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:41.510 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541323+0000", 2024-05-08T08:54:41.510 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8255e-05 2024-05-08T08:54:41.510 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:41.510 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:41.510 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:41.510 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.510 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.510 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:979 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 3637248~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:41.510 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451462+0000", 2024-05-08T08:54:41.510 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 91.981247421000006, 2024-05-08T08:54:41.510 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.090036441999999994, 2024-05-08T08:54:41.510 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:41.510 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:41.510 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:41.511 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:41.511 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:41.511 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:41.511 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 979 2024-05-08T08:54:41.511 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.511 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:41.511 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.511 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:41.511 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451462+0000", 2024-05-08T08:54:41.511 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:41.511 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.511 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.511 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:41.511 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451462+0000", 2024-05-08T08:54:41.511 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:41.511 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.511 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.511 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:41.512 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451463+0000", 2024-05-08T08:54:41.512 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4300000000000002e-07 2024-05-08T08:54:41.512 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.512 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.512 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:41.512 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451471+0000", 2024-05-08T08:54:41.512 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0800000000000006e-06 2024-05-08T08:54:41.512 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.512 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.512 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:41.512 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451481+0000", 2024-05-08T08:54:41.512 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.38e-06 2024-05-08T08:54:41.512 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.512 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.512 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:41.512 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451487+0000", 2024-05-08T08:54:41.512 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4590000000000003e-06 2024-05-08T08:54:41.513 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.513 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.513 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:41.513 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541350+0000", 2024-05-08T08:54:41.513 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089862599000000001 2024-05-08T08:54:41.513 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.513 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.513 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:41.513 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541407+0000", 2024-05-08T08:54:41.513 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7667999999999997e-05 2024-05-08T08:54:41.513 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.513 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.513 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:41.513 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541499+0000", 2024-05-08T08:54:41.513 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1513000000000003e-05 2024-05-08T08:54:41.513 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:41.514 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:41.514 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:41.514 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.514 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.514 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:980 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 4128768~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:41.514 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451496+0000", 2024-05-08T08:54:41.514 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 91.981213664999999, 2024-05-08T08:54:41.514 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.090258985, 2024-05-08T08:54:41.514 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:41.514 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:41.514 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:41.514 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:41.514 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:41.514 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:41.514 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 980 2024-05-08T08:54:41.515 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.515 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:41.515 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.515 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:41.515 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451496+0000", 2024-05-08T08:54:41.515 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:41.515 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.515 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.515 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:41.515 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451496+0000", 2024-05-08T08:54:41.515 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:41.515 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.515 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.515 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:41.515 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451497+0000", 2024-05-08T08:54:41.515 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7599999999999997e-07 2024-05-08T08:54:41.515 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.516 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.516 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:41.516 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451500+0000", 2024-05-08T08:54:41.516 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7930000000000002e-06 2024-05-08T08:54:41.516 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.516 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.516 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:41.516 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451506+0000", 2024-05-08T08:54:41.516 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6980000000000001e-06 2024-05-08T08:54:41.516 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.516 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.516 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:41.516 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451512+0000", 2024-05-08T08:54:41.516 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2490000000000001e-06 2024-05-08T08:54:41.516 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.517 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.517 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:41.517 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541550+0000", 2024-05-08T08:54:41.517 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.090037961999999999 2024-05-08T08:54:41.517 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.517 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.517 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:41.517 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541647+0000", 2024-05-08T08:54:41.517 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6924999999999998e-05 2024-05-08T08:54:41.517 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:41.517 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:41.517 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:41.517 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541755+0000", 2024-05-08T08:54:41.517 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000108682 2024-05-08T08:54:41.517 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:41.517 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:41.517 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:41.518 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:41.518 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:41.518 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:41.519 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:54:41.519 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:41.522+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:41.521 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:54:41.521 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:54:41.539 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:54:41.539 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:54:41.555 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:54:41.555 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:54:41.573 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:54:41.573 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:54:41.590 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:54:41.591 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:54:41.608 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:54:41.608 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:54:41.620 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:41.622+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:41.634 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:41.634 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:41.642 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:54:41.720 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:41.723+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:41.747 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:41.747 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:41.756 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:54:41.821 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:41.824+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:41.858 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-05-08T08:54:41.861 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:41.861 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:41.870 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:54:41.954 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:41.957+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:42.016 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:42.016 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:42.022 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:42.025+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:42.025 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:54:42.123 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:42.126+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:42.172 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:42.172 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:42.181 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:54:42.224 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:42.226+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:42.325 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:42.327+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:42.327 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:42.327 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:42.336 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:54:42.361 INFO:teuthology.orchestra.run.smithi059.stderr:noscrub is unset 2024-05-08T08:54:42.425 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:42.428+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:42.482 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:42.482 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:42.490 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:54:42.531 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:42.534+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:42.628 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:42.630+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:42.635 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:42.635 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:42.645 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:54:42.729 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:42.732+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:42.821 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:42.821 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:42.830 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:42.833+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:42.831 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:54:42.931 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:42.934+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:42.944 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:42.944 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:42.953 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:54:43.037 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:43.040+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:43.109 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:43.109 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:43.119 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:54:43.133 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:43.135+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:43.234 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:43.236+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:43.269 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:43.269 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:43.279 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:54:43.334 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:43.337+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:43.435 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:43.437+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:43.440 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:43.440 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:43.450 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:54:43.535 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:43.538+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:43.600 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:43.601 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:43.609 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:54:43.620 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T08:54:43.636 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:43.639+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:43.720 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:43.720 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:43.729 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:54:43.737 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:43.740+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:43.779 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:54:43.779 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:54:43.798 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:54:43.798 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:54:43.815 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:54:43.815 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:54:43.836 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:54:43.837 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:54:43.839 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:43.841+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:43.842 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:43.842 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:54:43.842 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:54:43.842 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:43.851 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:54:43.855 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:54:43.856 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:54:43.874 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:54:43.875 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:54:43.940 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:43.942+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:43.961 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:43.962 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:54:43.962 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:54:43.962 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:54:43.962 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:43.970 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:54:44.041 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:44.044+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:44.082 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:44.082 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:54:44.082 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:54:44.082 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:54:44.082 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.082 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:437 3.17 3:ea93a512:::rbd_data.1140100c9703.0000000000000077:head [set-alloc-hint object_size 2097152 write_size 2097152,write 1170432~1024] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e63)", 2024-05-08T08:54:44.082 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:08.390804+0000", 2024-05-08T08:54:44.082 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 95.692925629000001, 2024-05-08T08:54:44.082 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0406068690000001, 2024-05-08T08:54:44.082 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:44.082 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:44.082 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:54:44.083 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:44.083 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:44.083 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:44.083 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 437 2024-05-08T08:54:44.083 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.083 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:44.083 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.083 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:44.083 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390804+0000", 2024-05-08T08:54:44.083 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:44.083 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.083 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.083 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:44.083 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390804+0000", 2024-05-08T08:54:44.083 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:44.084 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.084 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.084 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:44.084 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390806+0000", 2024-05-08T08:54:44.084 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.666e-06 2024-05-08T08:54:44.084 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.084 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.084 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:44.084 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390818+0000", 2024-05-08T08:54:44.084 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2187999999999999e-05 2024-05-08T08:54:44.084 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.084 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.084 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:44.084 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390834+0000", 2024-05-08T08:54:44.084 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6191000000000001e-05 2024-05-08T08:54:44.085 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.085 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.085 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:44.085 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390845+0000", 2024-05-08T08:54:44.085 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0831e-05 2024-05-08T08:54:44.085 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.085 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.085 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:44.085 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390922+0000", 2024-05-08T08:54:44.085 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7253e-05 2024-05-08T08:54:44.085 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.085 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.085 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op must wait for map", 2024-05-08T08:54:44.085 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390930+0000", 2024-05-08T08:54:44.085 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8259999999999993e-06 2024-05-08T08:54:44.086 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.086 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.086 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:44.086 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392155+0000", 2024-05-08T08:54:44.086 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0012256649999999999 2024-05-08T08:54:44.086 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.086 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.086 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:54:44.086 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392165+0000", 2024-05-08T08:54:44.086 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5799999999999998e-06 2024-05-08T08:54:44.086 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.086 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.086 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:44.086 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.429596+0000", 2024-05-08T08:54:44.086 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0374313639999999 2024-05-08T08:54:44.087 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.087 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.087 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:44.087 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.429728+0000", 2024-05-08T08:54:44.087 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000131552 2024-05-08T08:54:44.087 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.087 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.087 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 0,1", 2024-05-08T08:54:44.087 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.429860+0000", 2024-05-08T08:54:44.087 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000132137 2024-05-08T08:54:44.087 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.087 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.087 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:54:44.087 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.430811+0000", 2024-05-08T08:54:44.088 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00095040200000000004 2024-05-08T08:54:44.088 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.088 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.088 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:54:44.088 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431126+0000", 2024-05-08T08:54:44.088 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000315006 2024-05-08T08:54:44.088 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.088 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.088 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:54:44.088 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431272+0000", 2024-05-08T08:54:44.088 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000146487 2024-05-08T08:54:44.088 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.088 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.088 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:54:44.088 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431315+0000", 2024-05-08T08:54:44.088 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2697000000000001e-05 2024-05-08T08:54:44.089 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.089 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.089 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:44.089 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431411+0000", 2024-05-08T08:54:44.089 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6024000000000001e-05 2024-05-08T08:54:44.089 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:44.089 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:44.089 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:44.089 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.089 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.089 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:439 3.17 3:ea93a512:::rbd_data.1140100c9703.0000000000000077:head [read 1163264~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-05-08T08:54:44.089 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:08.390871+0000", 2024-05-08T08:54:44.089 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 95.692858118000004, 2024-05-08T08:54:44.089 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0408790590000001, 2024-05-08T08:54:44.089 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:44.090 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:44.090 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:44.090 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:44.090 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:44.090 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:44.090 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 439 2024-05-08T08:54:44.090 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.090 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:44.090 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.090 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:44.090 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390871+0000", 2024-05-08T08:54:44.090 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:44.090 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.090 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.090 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:44.091 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390871+0000", 2024-05-08T08:54:44.091 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:44.091 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.091 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.091 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:44.091 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390872+0000", 2024-05-08T08:54:44.091 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6300000000000005e-07 2024-05-08T08:54:44.091 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.091 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.091 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:44.091 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390875+0000", 2024-05-08T08:54:44.091 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3459999999999998e-06 2024-05-08T08:54:44.091 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.091 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.091 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:44.091 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390891+0000", 2024-05-08T08:54:44.092 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5255e-05 2024-05-08T08:54:44.092 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.092 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.092 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:44.092 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390897+0000", 2024-05-08T08:54:44.092 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2360000000000001e-06 2024-05-08T08:54:44.092 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.092 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.092 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:44.092 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390995+0000", 2024-05-08T08:54:44.092 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8522000000000003e-05 2024-05-08T08:54:44.092 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.092 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.092 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting_for_map not empty", 2024-05-08T08:54:44.092 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.391005+0000", 2024-05-08T08:54:44.092 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1460000000000002e-06 2024-05-08T08:54:44.093 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.093 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.093 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:44.093 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392191+0000", 2024-05-08T08:54:44.093 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0011860270000000001 2024-05-08T08:54:44.093 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.093 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.093 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:54:44.093 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392197+0000", 2024-05-08T08:54:44.093 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4289999999999998e-06 2024-05-08T08:54:44.093 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.093 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.093 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:44.093 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.430596+0000", 2024-05-08T08:54:44.093 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0383989090000001 2024-05-08T08:54:44.093 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.094 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.094 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for rw locks", 2024-05-08T08:54:44.094 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.430676+0000", 2024-05-08T08:54:44.094 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9781000000000001e-05 2024-05-08T08:54:44.094 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.094 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.094 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:44.094 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431460+0000", 2024-05-08T08:54:44.094 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00078427300000000004 2024-05-08T08:54:44.094 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.094 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.094 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:44.094 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431540+0000", 2024-05-08T08:54:44.094 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0224999999999998e-05 2024-05-08T08:54:44.095 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.095 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.095 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:44.095 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431750+0000", 2024-05-08T08:54:44.095 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00021024699999999999 2024-05-08T08:54:44.095 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:44.095 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:44.095 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:44.095 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.095 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.095 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:945 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 4096000~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:44.095 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:54:44.095 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 94.632708550999993, 2024-05-08T08:54:44.095 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086774741000000002, 2024-05-08T08:54:44.095 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:44.096 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:44.096 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:44.096 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:44.096 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:44.096 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:44.096 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 945 2024-05-08T08:54:44.096 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.096 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:44.096 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.096 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:44.096 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:54:44.096 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:44.096 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.096 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.096 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:44.096 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:54:44.096 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:44.097 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.097 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.097 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:44.097 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:54:44.097 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3499999999999996e-07 2024-05-08T08:54:44.097 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.097 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.097 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:44.097 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451024+0000", 2024-05-08T08:54:44.097 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.627e-06 2024-05-08T08:54:44.097 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.097 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.097 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:44.097 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451037+0000", 2024-05-08T08:54:44.097 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2431e-05 2024-05-08T08:54:44.097 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.098 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.098 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:44.098 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451042+0000", 2024-05-08T08:54:44.098 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8409999999999996e-06 2024-05-08T08:54:44.098 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.098 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.098 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:44.098 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537608+0000", 2024-05-08T08:54:44.098 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086565243 2024-05-08T08:54:44.098 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.098 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.098 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:44.098 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537687+0000", 2024-05-08T08:54:44.098 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9519999999999998e-05 2024-05-08T08:54:44.099 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.099 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.099 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:44.099 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537796+0000", 2024-05-08T08:54:44.099 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000108544 2024-05-08T08:54:44.099 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:44.099 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:44.099 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:44.099 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.099 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.099 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:964 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 458752~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:44.099 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451051+0000", 2024-05-08T08:54:44.099 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 94.632678377999994, 2024-05-08T08:54:44.099 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086990490000000004, 2024-05-08T08:54:44.099 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:44.100 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:44.100 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:44.100 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:44.100 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:44.100 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:44.100 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 964 2024-05-08T08:54:44.100 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.100 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:44.100 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.100 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:44.100 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451051+0000", 2024-05-08T08:54:44.100 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:44.100 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.100 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.100 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:44.100 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451051+0000", 2024-05-08T08:54:44.101 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:44.101 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.101 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.101 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:44.101 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451052+0000", 2024-05-08T08:54:44.101 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7299999999999996e-07 2024-05-08T08:54:44.101 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.101 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.101 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:44.101 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451055+0000", 2024-05-08T08:54:44.101 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.2770000000000001e-06 2024-05-08T08:54:44.101 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.101 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.101 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:44.101 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451061+0000", 2024-05-08T08:54:44.101 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4060000000000002e-06 2024-05-08T08:54:44.102 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.102 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.102 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:44.102 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451066+0000", 2024-05-08T08:54:44.102 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0839999999999999e-06 2024-05-08T08:54:44.102 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.102 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.102 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:44.102 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537839+0000", 2024-05-08T08:54:44.102 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086772962999999995 2024-05-08T08:54:44.102 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.102 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.102 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:44.102 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537934+0000", 2024-05-08T08:54:44.103 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4742999999999997e-05 2024-05-08T08:54:44.103 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.103 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.103 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:44.103 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538042+0000", 2024-05-08T08:54:44.103 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000107444 2024-05-08T08:54:44.103 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:44.103 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:44.103 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:44.103 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.103 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.103 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:965 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 950272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:44.103 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451075+0000", 2024-05-08T08:54:44.103 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 94.632654316, 2024-05-08T08:54:44.104 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087230196999999995, 2024-05-08T08:54:44.104 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:44.104 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:44.104 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:44.104 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:44.104 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:44.104 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:44.104 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 965 2024-05-08T08:54:44.104 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.104 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:44.104 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.104 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:44.104 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451075+0000", 2024-05-08T08:54:44.104 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:44.104 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.105 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.105 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:44.105 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451075+0000", 2024-05-08T08:54:44.105 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:44.105 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.105 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.105 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:44.105 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451076+0000", 2024-05-08T08:54:44.105 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5000000000000002e-07 2024-05-08T08:54:44.105 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.105 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.105 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:44.105 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451078+0000", 2024-05-08T08:54:44.105 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.5639999999999999e-06 2024-05-08T08:54:44.106 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.106 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.106 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:44.106 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451085+0000", 2024-05-08T08:54:44.106 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5640000000000002e-06 2024-05-08T08:54:44.106 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.106 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.106 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:44.106 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451090+0000", 2024-05-08T08:54:44.106 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1699999999999996e-06 2024-05-08T08:54:44.106 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.106 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.106 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:44.106 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538106+0000", 2024-05-08T08:54:44.106 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087015888999999999 2024-05-08T08:54:44.107 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.107 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.107 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:44.107 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538190+0000", 2024-05-08T08:54:44.107 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3961999999999993e-05 2024-05-08T08:54:44.107 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.107 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.107 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:44.107 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538305+0000", 2024-05-08T08:54:44.107 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011529800000000001 2024-05-08T08:54:44.107 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:44.107 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:44.107 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:44.107 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.107 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.108 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:966 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 1441792~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:44.108 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:54:44.108 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 94.632630942999995, 2024-05-08T08:54:44.108 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087446323000000006, 2024-05-08T08:54:44.108 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:44.108 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:44.108 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:44.108 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:44.108 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:44.108 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:44.108 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 966 2024-05-08T08:54:44.108 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.108 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:44.108 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.108 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:44.108 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:54:44.109 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:44.109 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.109 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.109 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:44.109 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:54:44.109 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:44.109 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.109 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.109 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:44.109 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:54:44.109 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3700000000000001e-07 2024-05-08T08:54:44.109 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.109 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.109 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:44.109 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451101+0000", 2024-05-08T08:54:44.110 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2560000000000001e-06 2024-05-08T08:54:44.110 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.110 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.110 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:44.110 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451107+0000", 2024-05-08T08:54:44.110 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1070000000000001e-06 2024-05-08T08:54:44.110 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.110 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.110 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:44.110 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451112+0000", 2024-05-08T08:54:44.110 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1420000000000001e-06 2024-05-08T08:54:44.110 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.110 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.110 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:44.111 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538341+0000", 2024-05-08T08:54:44.111 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087228302999999993 2024-05-08T08:54:44.111 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.111 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.111 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:44.111 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538424+0000", 2024-05-08T08:54:44.111 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3497000000000004e-05 2024-05-08T08:54:44.111 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.111 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.111 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:44.111 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538545+0000", 2024-05-08T08:54:44.111 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000120581 2024-05-08T08:54:44.111 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:44.111 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:44.111 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:44.111 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.112 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.112 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:967 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:44.112 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451120+0000", 2024-05-08T08:54:44.112 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 94.632609467999998, 2024-05-08T08:54:44.112 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087632725999999994, 2024-05-08T08:54:44.112 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:44.112 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:44.112 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:44.112 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:44.112 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:44.112 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:44.112 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 967 2024-05-08T08:54:44.112 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.112 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:44.112 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.112 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:44.113 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451120+0000", 2024-05-08T08:54:44.113 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:44.113 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.113 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.113 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:44.113 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451120+0000", 2024-05-08T08:54:44.113 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:44.113 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.113 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.113 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:44.113 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451121+0000", 2024-05-08T08:54:44.113 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8800000000000002e-07 2024-05-08T08:54:44.113 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.113 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.113 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:44.113 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451130+0000", 2024-05-08T08:54:44.113 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2299999999999997e-06 2024-05-08T08:54:44.114 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.114 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.114 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:44.114 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451137+0000", 2024-05-08T08:54:44.114 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4669999999999998e-06 2024-05-08T08:54:44.114 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.114 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.114 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:44.114 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451143+0000", 2024-05-08T08:54:44.114 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2839999999999997e-06 2024-05-08T08:54:44.114 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.114 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.114 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:44.114 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538575+0000", 2024-05-08T08:54:44.114 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087432789999999996 2024-05-08T08:54:44.115 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.115 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.115 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:44.115 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538648+0000", 2024-05-08T08:54:44.115 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2877e-05 2024-05-08T08:54:44.115 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.115 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.115 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:44.115 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538753+0000", 2024-05-08T08:54:44.115 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010449 2024-05-08T08:54:44.115 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:44.115 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:44.115 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:44.115 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.115 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.116 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:968 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 2424832~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:44.116 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451152+0000", 2024-05-08T08:54:44.116 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 94.632577382999997, 2024-05-08T08:54:44.116 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087813325999999997, 2024-05-08T08:54:44.116 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:44.116 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:44.116 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:44.116 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:44.116 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:44.116 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:44.116 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 968 2024-05-08T08:54:44.116 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.116 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:44.116 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.116 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:44.117 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451152+0000", 2024-05-08T08:54:44.117 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:44.117 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.117 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.117 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:44.117 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451152+0000", 2024-05-08T08:54:44.117 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:44.117 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.117 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.117 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:44.117 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451153+0000", 2024-05-08T08:54:44.117 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3099999999999998e-07 2024-05-08T08:54:44.117 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.117 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.117 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:44.118 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451155+0000", 2024-05-08T08:54:44.118 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4760000000000001e-06 2024-05-08T08:54:44.118 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.118 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.118 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:44.118 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451162+0000", 2024-05-08T08:54:44.118 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5969999999999997e-06 2024-05-08T08:54:44.118 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.118 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.118 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:44.118 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451167+0000", 2024-05-08T08:54:44.118 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.977e-06 2024-05-08T08:54:44.118 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.118 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.118 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:44.118 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538781+0000", 2024-05-08T08:54:44.119 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087614846999999996 2024-05-08T08:54:44.119 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.119 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.119 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:44.119 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538857+0000", 2024-05-08T08:54:44.119 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5358000000000006e-05 2024-05-08T08:54:44.119 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.119 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.119 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:44.119 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538965+0000", 2024-05-08T08:54:44.119 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010864 2024-05-08T08:54:44.119 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:44.119 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:44.119 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:44.119 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.120 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.120 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:969 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 2916352~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:44.120 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:54:44.120 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 94.632555678000003, 2024-05-08T08:54:44.120 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087998980000000004, 2024-05-08T08:54:44.120 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:44.120 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:44.120 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:44.120 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:44.120 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:44.120 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:44.120 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 969 2024-05-08T08:54:44.120 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.120 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:44.120 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.120 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:44.121 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:54:44.121 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:44.121 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.121 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.121 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:44.121 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:54:44.121 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:44.121 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.121 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.121 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:44.121 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:54:44.121 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3599999999999999e-07 2024-05-08T08:54:44.121 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.121 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.121 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:44.121 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451177+0000", 2024-05-08T08:54:44.122 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4590000000000001e-06 2024-05-08T08:54:44.122 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.122 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.122 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:44.122 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451184+0000", 2024-05-08T08:54:44.122 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4109999999999999e-06 2024-05-08T08:54:44.122 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.122 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.122 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:44.122 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451191+0000", 2024-05-08T08:54:44.122 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7279999999999998e-06 2024-05-08T08:54:44.122 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.122 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.122 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:44.122 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538997+0000", 2024-05-08T08:54:44.122 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087806170000000003 2024-05-08T08:54:44.122 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.123 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.123 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:44.123 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539069+0000", 2024-05-08T08:54:44.123 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2373999999999995e-05 2024-05-08T08:54:44.123 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.123 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.123 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:44.123 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539173+0000", 2024-05-08T08:54:44.123 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000103402 2024-05-08T08:54:44.123 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:44.123 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:44.123 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:44.123 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.123 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.123 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:970 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 3407872~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:44.123 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451198+0000", 2024-05-08T08:54:44.124 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 94.632531080999996, 2024-05-08T08:54:44.124 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088183368999999998, 2024-05-08T08:54:44.124 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:44.124 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:44.124 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:44.124 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:44.124 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:44.124 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:44.124 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 970 2024-05-08T08:54:44.124 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.124 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:44.124 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.124 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:44.124 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451198+0000", 2024-05-08T08:54:44.124 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:44.125 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.125 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.125 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:44.125 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451198+0000", 2024-05-08T08:54:44.125 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:44.125 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.125 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.125 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:44.125 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451199+0000", 2024-05-08T08:54:44.125 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7400000000000003e-07 2024-05-08T08:54:44.125 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.125 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.125 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:44.125 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451211+0000", 2024-05-08T08:54:44.125 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2109999999999999e-05 2024-05-08T08:54:44.126 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.126 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.126 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:44.126 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451219+0000", 2024-05-08T08:54:44.126 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4170000000000003e-06 2024-05-08T08:54:44.126 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.126 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.126 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:44.126 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451224+0000", 2024-05-08T08:54:44.126 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3990000000000003e-06 2024-05-08T08:54:44.126 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.126 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.126 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:44.126 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539204+0000", 2024-05-08T08:54:44.126 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087980099000000006 2024-05-08T08:54:44.126 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.127 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.127 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:44.127 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539276+0000", 2024-05-08T08:54:44.127 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1513000000000005e-05 2024-05-08T08:54:44.127 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.127 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.127 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:44.127 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539382+0000", 2024-05-08T08:54:44.127 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010615699999999999 2024-05-08T08:54:44.127 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:44.127 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:44.127 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:44.127 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.127 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.128 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:971 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 3899392~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:44.128 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451233+0000", 2024-05-08T08:54:44.128 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 94.632496373999999, 2024-05-08T08:54:44.128 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088431525999999996, 2024-05-08T08:54:44.128 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:44.128 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:44.128 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:44.128 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:44.128 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:44.128 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:44.128 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 971 2024-05-08T08:54:44.128 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.128 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:44.128 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.128 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:44.128 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451233+0000", 2024-05-08T08:54:44.128 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:44.129 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.129 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.129 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:44.129 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451233+0000", 2024-05-08T08:54:44.129 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:44.129 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.129 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.129 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:44.129 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451234+0000", 2024-05-08T08:54:44.129 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8500000000000001e-07 2024-05-08T08:54:44.129 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.129 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.129 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:44.129 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451237+0000", 2024-05-08T08:54:44.129 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8320000000000002e-06 2024-05-08T08:54:44.129 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.130 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.130 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:44.130 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451243+0000", 2024-05-08T08:54:44.130 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6749999999999996e-06 2024-05-08T08:54:44.130 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.130 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.130 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:44.130 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451249+0000", 2024-05-08T08:54:44.130 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1229999999999997e-06 2024-05-08T08:54:44.130 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.130 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.130 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:44.130 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539412+0000", 2024-05-08T08:54:44.130 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088162668999999999 2024-05-08T08:54:44.131 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.131 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.131 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:44.131 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539505+0000", 2024-05-08T08:54:44.131 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3240999999999997e-05 2024-05-08T08:54:44.131 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.131 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.131 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:44.131 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539665+0000", 2024-05-08T08:54:44.131 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00015930100000000001 2024-05-08T08:54:44.131 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:44.131 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:44.131 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:44.131 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.131 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.131 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:972 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 196608~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:44.132 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451257+0000", 2024-05-08T08:54:44.132 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 94.632472429000003, 2024-05-08T08:54:44.132 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088719928000000003, 2024-05-08T08:54:44.132 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:44.132 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:44.132 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:44.132 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:44.132 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:44.132 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:44.132 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 972 2024-05-08T08:54:44.132 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.132 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:44.132 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.132 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:44.132 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451257+0000", 2024-05-08T08:54:44.133 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:44.133 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.133 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.133 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:44.133 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451257+0000", 2024-05-08T08:54:44.133 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:44.133 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.133 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.133 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:44.133 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451258+0000", 2024-05-08T08:54:44.133 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7999999999999996e-07 2024-05-08T08:54:44.133 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.133 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.133 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:44.133 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451260+0000", 2024-05-08T08:54:44.133 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4609999999999998e-06 2024-05-08T08:54:44.134 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.134 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.134 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:44.134 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451267+0000", 2024-05-08T08:54:44.134 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.81e-06 2024-05-08T08:54:44.134 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.134 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.134 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:44.134 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451273+0000", 2024-05-08T08:54:44.134 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0859999999999998e-06 2024-05-08T08:54:44.134 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.134 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.134 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:44.134 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539718+0000", 2024-05-08T08:54:44.134 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088445059000000006 2024-05-08T08:54:44.135 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.135 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.135 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:44.135 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539834+0000", 2024-05-08T08:54:44.135 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000116426 2024-05-08T08:54:44.135 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.135 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.135 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:44.135 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539977+0000", 2024-05-08T08:54:44.135 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014260600000000001 2024-05-08T08:54:44.135 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:44.135 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:44.135 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:44.135 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.135 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.136 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:973 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 688128~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:44.136 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:54:44.136 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 94.632449844999996, 2024-05-08T08:54:44.136 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088918496, 2024-05-08T08:54:44.136 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:44.136 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:44.136 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:44.136 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:44.136 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:44.136 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:44.136 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 973 2024-05-08T08:54:44.136 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.136 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:44.136 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.136 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:44.136 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:54:44.137 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:44.137 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.137 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.137 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:44.137 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:54:44.137 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:44.137 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.137 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.137 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:44.137 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:54:44.137 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9099999999999999e-07 2024-05-08T08:54:44.137 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.137 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.137 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:44.137 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451282+0000", 2024-05-08T08:54:44.137 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1550000000000001e-06 2024-05-08T08:54:44.137 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.138 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.138 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:44.138 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451288+0000", 2024-05-08T08:54:44.138 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8889999999999999e-06 2024-05-08T08:54:44.138 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.138 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.138 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:44.138 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451294+0000", 2024-05-08T08:54:44.138 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0850000000000002e-06 2024-05-08T08:54:44.138 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.138 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.138 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:44.138 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540011+0000", 2024-05-08T08:54:44.138 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088717314000000005 2024-05-08T08:54:44.138 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.138 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.139 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:44.139 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540089+0000", 2024-05-08T08:54:44.139 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7438e-05 2024-05-08T08:54:44.139 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.139 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.139 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:44.139 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540198+0000", 2024-05-08T08:54:44.139 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000109224 2024-05-08T08:54:44.139 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:44.139 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:44.139 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:44.139 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.139 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.139 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:974 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 1179648~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:44.139 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451301+0000", 2024-05-08T08:54:44.139 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 94.632428211999994, 2024-05-08T08:54:44.139 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089105078000000004, 2024-05-08T08:54:44.140 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:44.140 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:44.140 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:44.140 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:44.140 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:44.140 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:44.140 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 974 2024-05-08T08:54:44.140 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.140 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:44.140 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.140 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:44.140 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451301+0000", 2024-05-08T08:54:44.140 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:44.140 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.140 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.141 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:44.141 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451301+0000", 2024-05-08T08:54:44.141 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:44.141 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.141 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.141 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:44.141 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451302+0000", 2024-05-08T08:54:44.141 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7800000000000002e-07 2024-05-08T08:54:44.141 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.141 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.141 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:44.141 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451304+0000", 2024-05-08T08:54:44.141 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.65e-06 2024-05-08T08:54:44.141 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.141 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.141 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:44.142 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451311+0000", 2024-05-08T08:54:44.142 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.472e-06 2024-05-08T08:54:44.142 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.142 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.142 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:44.142 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451316+0000", 2024-05-08T08:54:44.142 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9189999999999998e-06 2024-05-08T08:54:44.142 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.142 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.142 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:44.142 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540234+0000", 2024-05-08T08:54:44.142 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088917752000000003 2024-05-08T08:54:44.142 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.142 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.142 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:44.142 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540308+0000", 2024-05-08T08:54:44.143 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4758000000000005e-05 2024-05-08T08:54:44.143 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.143 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.143 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:44.143 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540406+0000", 2024-05-08T08:54:44.143 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8048999999999996e-05 2024-05-08T08:54:44.143 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:44.143 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:44.143 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:44.143 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.143 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.143 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:975 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 1671168~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:44.143 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:54:44.143 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 94.632385881999994, 2024-05-08T08:54:44.143 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089294139999999994, 2024-05-08T08:54:44.143 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:44.143 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:44.144 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:44.144 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:44.144 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:44.144 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:44.144 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 975 2024-05-08T08:54:44.144 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.144 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:44.144 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.144 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:44.144 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:54:44.144 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:44.144 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.144 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.144 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:44.144 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:54:44.145 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:44.145 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.145 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.145 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:44.145 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:54:44.145 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1299999999999999e-07 2024-05-08T08:54:44.145 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.145 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.145 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:44.145 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451348+0000", 2024-05-08T08:54:44.145 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.484e-06 2024-05-08T08:54:44.145 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.145 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.145 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:44.145 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451356+0000", 2024-05-08T08:54:44.146 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8499999999999994e-06 2024-05-08T08:54:44.146 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.146 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.146 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:44.146 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451366+0000", 2024-05-08T08:54:44.146 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.9850000000000001e-06 2024-05-08T08:54:44.146 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.146 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.146 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:44.146 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540448+0000", 2024-05-08T08:54:44.146 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089082254 2024-05-08T08:54:44.146 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.146 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.146 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:44.147 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540528+0000", 2024-05-08T08:54:44.147 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0234999999999993e-05 2024-05-08T08:54:44.147 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.147 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.147 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:44.147 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540638+0000", 2024-05-08T08:54:44.147 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010951900000000001 2024-05-08T08:54:44.147 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:44.147 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:44.147 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:44.147 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.147 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.147 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:976 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 2162688~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:44.147 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451375+0000", 2024-05-08T08:54:44.147 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 94.632354135, 2024-05-08T08:54:44.148 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089472745000000006, 2024-05-08T08:54:44.148 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:44.148 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:44.148 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:44.148 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:44.148 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:44.148 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:44.148 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 976 2024-05-08T08:54:44.148 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.148 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:44.148 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.148 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:44.148 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451375+0000", 2024-05-08T08:54:44.148 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:44.148 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.149 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.149 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:44.149 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451375+0000", 2024-05-08T08:54:44.149 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:44.149 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.149 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.149 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:44.149 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451376+0000", 2024-05-08T08:54:44.149 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2399999999999998e-07 2024-05-08T08:54:44.149 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.149 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.149 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:44.149 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451379+0000", 2024-05-08T08:54:44.149 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6359999999999998e-06 2024-05-08T08:54:44.149 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.149 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.149 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:44.150 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451387+0000", 2024-05-08T08:54:44.150 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9929999999999995e-06 2024-05-08T08:54:44.150 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.150 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.150 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:44.150 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451393+0000", 2024-05-08T08:54:44.150 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4699999999999999e-06 2024-05-08T08:54:44.150 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.150 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.150 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:44.150 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540677+0000", 2024-05-08T08:54:44.150 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089283504999999999 2024-05-08T08:54:44.150 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.150 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.150 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:44.151 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540743+0000", 2024-05-08T08:54:44.151 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6303999999999994e-05 2024-05-08T08:54:44.151 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.151 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.151 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:44.151 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540848+0000", 2024-05-08T08:54:44.151 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000105213 2024-05-08T08:54:44.151 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:44.151 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:44.151 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:44.151 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.151 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.152 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:977 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 2654208~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:44.152 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:54:44.152 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 94.632328702999999, 2024-05-08T08:54:44.152 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089698473000000001, 2024-05-08T08:54:44.152 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:44.152 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:44.152 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:44.152 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:44.152 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:44.152 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:44.152 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 977 2024-05-08T08:54:44.152 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.153 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:44.153 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.153 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:44.153 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:54:44.153 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:44.153 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.153 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.153 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:44.153 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:54:44.153 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:44.153 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.153 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.153 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:44.153 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:54:44.153 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2899999999999999e-07 2024-05-08T08:54:44.154 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.154 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.154 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:44.154 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451404+0000", 2024-05-08T08:54:44.154 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6649999999999999e-06 2024-05-08T08:54:44.154 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.154 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.154 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:44.154 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451410+0000", 2024-05-08T08:54:44.154 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3269999999999997e-06 2024-05-08T08:54:44.154 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.154 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.154 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:44.154 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451415+0000", 2024-05-08T08:54:44.154 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9529999999999998e-06 2024-05-08T08:54:44.155 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.155 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.155 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:44.155 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540912+0000", 2024-05-08T08:54:44.155 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089496950000000006 2024-05-08T08:54:44.155 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.155 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.155 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:44.155 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541000+0000", 2024-05-08T08:54:44.155 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7429000000000006e-05 2024-05-08T08:54:44.155 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.155 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.155 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:44.155 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541099+0000", 2024-05-08T08:54:44.155 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.9720000000000001e-05 2024-05-08T08:54:44.156 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:44.156 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:44.156 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:44.156 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.156 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.156 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:978 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 3145728~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:44.156 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:54:44.156 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 94.632307495000006, 2024-05-08T08:54:44.156 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089900635000000007, 2024-05-08T08:54:44.156 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:44.156 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:44.156 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:44.156 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:44.156 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:44.156 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:44.157 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 978 2024-05-08T08:54:44.157 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.157 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:44.157 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.157 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:44.157 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:54:44.157 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:44.157 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.157 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.157 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:44.157 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:54:44.157 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:44.157 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.157 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.157 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:44.158 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:54:44.158 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9400000000000001e-07 2024-05-08T08:54:44.158 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.158 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.158 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:44.158 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451425+0000", 2024-05-08T08:54:44.158 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2460000000000002e-06 2024-05-08T08:54:44.158 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.158 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.158 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:44.158 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451431+0000", 2024-05-08T08:54:44.158 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2419999999999997e-06 2024-05-08T08:54:44.158 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.158 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.158 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:44.159 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451436+0000", 2024-05-08T08:54:44.159 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.887e-06 2024-05-08T08:54:44.159 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.159 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.159 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:44.159 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541152+0000", 2024-05-08T08:54:44.159 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089716194999999999 2024-05-08T08:54:44.159 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.159 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.159 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:44.159 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541224+0000", 2024-05-08T08:54:44.159 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2416000000000006e-05 2024-05-08T08:54:44.159 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.159 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.159 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:44.160 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541323+0000", 2024-05-08T08:54:44.160 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8255e-05 2024-05-08T08:54:44.160 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:44.160 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:44.160 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:44.160 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.160 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.160 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:979 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 3637248~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:44.160 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451462+0000", 2024-05-08T08:54:44.160 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 94.632267045999996, 2024-05-08T08:54:44.160 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.090036441999999994, 2024-05-08T08:54:44.160 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:44.160 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:44.160 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:44.160 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:44.161 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:44.161 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:44.161 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 979 2024-05-08T08:54:44.161 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.161 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:44.161 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.161 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:44.161 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451462+0000", 2024-05-08T08:54:44.161 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:44.161 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.161 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.161 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:44.161 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451462+0000", 2024-05-08T08:54:44.161 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:44.161 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.161 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.162 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:44.162 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451463+0000", 2024-05-08T08:54:44.162 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4300000000000002e-07 2024-05-08T08:54:44.162 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.162 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.162 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:44.162 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451471+0000", 2024-05-08T08:54:44.162 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0800000000000006e-06 2024-05-08T08:54:44.162 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.162 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.162 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:44.162 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451481+0000", 2024-05-08T08:54:44.162 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.38e-06 2024-05-08T08:54:44.162 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.162 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.163 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:44.163 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451487+0000", 2024-05-08T08:54:44.163 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4590000000000003e-06 2024-05-08T08:54:44.163 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.163 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.163 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:44.163 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541350+0000", 2024-05-08T08:54:44.163 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089862599000000001 2024-05-08T08:54:44.163 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.163 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.163 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:44.163 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541407+0000", 2024-05-08T08:54:44.163 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7667999999999997e-05 2024-05-08T08:54:44.163 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.163 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.164 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:44.164 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541499+0000", 2024-05-08T08:54:44.164 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1513000000000003e-05 2024-05-08T08:54:44.164 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:44.164 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:44.164 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:44.164 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.164 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.164 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:980 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 4128768~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:44.164 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451496+0000", 2024-05-08T08:54:44.164 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 94.632233290000002, 2024-05-08T08:54:44.164 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.090258985, 2024-05-08T08:54:44.164 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:44.164 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:44.164 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:44.165 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:44.165 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:44.165 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:44.165 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 980 2024-05-08T08:54:44.165 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.165 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:44.165 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.165 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:44.165 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451496+0000", 2024-05-08T08:54:44.165 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:44.165 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.165 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.165 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:44.165 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451496+0000", 2024-05-08T08:54:44.165 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:44.165 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.166 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.166 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:44.166 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451497+0000", 2024-05-08T08:54:44.166 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7599999999999997e-07 2024-05-08T08:54:44.166 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.166 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.166 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:44.166 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451500+0000", 2024-05-08T08:54:44.166 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7930000000000002e-06 2024-05-08T08:54:44.166 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.166 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.166 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:44.166 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451506+0000", 2024-05-08T08:54:44.166 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6980000000000001e-06 2024-05-08T08:54:44.166 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.167 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.167 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:44.167 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451512+0000", 2024-05-08T08:54:44.167 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2490000000000001e-06 2024-05-08T08:54:44.167 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.167 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.167 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:44.167 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541550+0000", 2024-05-08T08:54:44.167 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.090037961999999999 2024-05-08T08:54:44.167 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.167 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.167 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:44.167 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541647+0000", 2024-05-08T08:54:44.167 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6924999999999998e-05 2024-05-08T08:54:44.167 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:44.167 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:44.168 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:44.168 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541755+0000", 2024-05-08T08:54:44.168 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000108682 2024-05-08T08:54:44.168 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:44.168 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:44.168 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:44.168 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:44.168 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:44.168 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:44.169 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:54:44.170 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:44.172+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:44.271 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:44.273+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:44.278 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:44.279 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:54:44.279 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:54:44.279 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:44.288 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:54:44.372 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:44.375+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:44.388 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-05-08T08:54:44.449 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:44.449 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:54:44.449 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:54:44.449 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:54:44.449 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:44.459 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:54:44.501 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:44.504+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:44.574 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:44.576+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:44.612 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:44.612 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:54:44.612 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:54:44.613 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:54:44.613 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.613 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4621.0:9089 3.1f e33/18 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:head v 33'117, mlcod=31'115)", 2024-05-08T08:54:44.613 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:54:44.613 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 132.09471057299999, 2024-05-08T08:54:44.613 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.111244524, 2024-05-08T08:54:44.613 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:44.613 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:44.613 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:54:44.613 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:44.613 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.613 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:44.613 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:54:44.613 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:44.613 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.614 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.614 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:44.614 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:54:44.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:44.614 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.614 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.614 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:44.614 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:54:44.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5900000000000001e-07 2024-05-08T08:54:44.614 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.614 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.614 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:44.614 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539600+0000", 2024-05-08T08:54:44.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.019981364000000001 2024-05-08T08:54:44.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:44.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539624+0000", 2024-05-08T08:54:44.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4043e-05 2024-05-08T08:54:44.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:44.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539635+0000", 2024-05-08T08:54:44.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1511e-05 2024-05-08T08:54:44.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:44.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539728+0000", 2024-05-08T08:54:44.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2498999999999996e-05 2024-05-08T08:54:44.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:44.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539746+0000", 2024-05-08T08:54:44.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8289000000000001e-05 2024-05-08T08:54:44.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.617 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.617 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:54:44.617 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.630822+0000", 2024-05-08T08:54:44.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.091076283999999993 2024-05-08T08:54:44.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.617 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.617 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:44.617 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.630863+0000", 2024-05-08T08:54:44.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0374999999999999e-05 2024-05-08T08:54:44.617 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:44.617 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:44.617 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:44.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.617 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.618 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4195.0:4 3.11 3:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-05-08T08:54:44.618 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:49.764654+0000", 2024-05-08T08:54:44.618 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 114.84967503199999, 2024-05-08T08:54:44.618 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.30714275499999999, 2024-05-08T08:54:44.618 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:44.618 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:44.618 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-05-08T08:54:44.618 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:44.618 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4195", 2024-05-08T08:54:44.618 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/59179521", 2024-05-08T08:54:44.619 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4 2024-05-08T08:54:44.619 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.619 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:44.619 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.622 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:44.622 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764654+0000", 2024-05-08T08:54:44.622 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:44.623 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.623 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.623 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:44.623 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764654+0000", 2024-05-08T08:54:44.623 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:44.623 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.623 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.623 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:44.623 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764655+0000", 2024-05-08T08:54:44.623 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.452e-06 2024-05-08T08:54:44.623 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.623 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.623 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:44.623 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764661+0000", 2024-05-08T08:54:44.623 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5369999999999997e-06 2024-05-08T08:54:44.624 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.624 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:44.624 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764671+0000", 2024-05-08T08:54:44.624 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.028e-05 2024-05-08T08:54:44.624 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.624 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:44.624 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764680+0000", 2024-05-08T08:54:44.624 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0629999999999995e-06 2024-05-08T08:54:44.624 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.624 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:44.624 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764753+0000", 2024-05-08T08:54:44.624 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2679e-05 2024-05-08T08:54:44.625 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.625 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.625 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-05-08T08:54:44.625 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764765+0000", 2024-05-08T08:54:44.625 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2415000000000001e-05 2024-05-08T08:54:44.625 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.625 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.625 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:44.625 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:50.071692+0000", 2024-05-08T08:54:44.625 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.306927324 2024-05-08T08:54:44.625 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.625 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.625 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:44.625 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:50.071796+0000", 2024-05-08T08:54:44.625 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104005 2024-05-08T08:54:44.625 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:44.626 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:44.626 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:44.626 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.626 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.626 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4416.0:20531 3.16 3:6bb2ed16:::rbd_object_map.1140bc4a55c7:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-05-08T08:54:44.626 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:58.673786+0000", 2024-05-08T08:54:44.626 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 45.940542628000003, 2024-05-08T08:54:44.626 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.94684119899999997, 2024-05-08T08:54:44.626 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:44.626 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:44.626 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:54:44.626 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:44.626 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4416", 2024-05-08T08:54:44.626 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:54:44.626 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20531 2024-05-08T08:54:44.627 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.627 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:44.627 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.627 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:44.627 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673786+0000", 2024-05-08T08:54:44.627 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:44.627 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.627 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.627 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:44.627 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673786+0000", 2024-05-08T08:54:44.627 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:44.627 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.627 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.627 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:44.627 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673787+0000", 2024-05-08T08:54:44.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9800000000000003e-07 2024-05-08T08:54:44.628 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.628 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:44.628 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673791+0000", 2024-05-08T08:54:44.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.104e-06 2024-05-08T08:54:44.628 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.628 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:44.628 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673801+0000", 2024-05-08T08:54:44.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9960000000000005e-06 2024-05-08T08:54:44.628 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.628 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:44.628 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673806+0000", 2024-05-08T08:54:44.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2320000000000001e-06 2024-05-08T08:54:44.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.629 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:44.629 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673860+0000", 2024-05-08T08:54:44.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3468999999999999e-05 2024-05-08T08:54:44.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.629 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-05-08T08:54:44.629 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673866+0000", 2024-05-08T08:54:44.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3500000000000002e-06 2024-05-08T08:54:44.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.629 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:44.629 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.616700+0000", 2024-05-08T08:54:44.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.94283390700000003 2024-05-08T08:54:44.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.630 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.630 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:44.630 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.616941+0000", 2024-05-08T08:54:44.630 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00024080400000000001 2024-05-08T08:54:44.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.630 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.630 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-05-08T08:54:44.630 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.617072+0000", 2024-05-08T08:54:44.630 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000131688 2024-05-08T08:54:44.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.630 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.630 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:54:44.631 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.619571+0000", 2024-05-08T08:54:44.631 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0024990149999999998 2024-05-08T08:54:44.631 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.631 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.631 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:54:44.631 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.620568+0000", 2024-05-08T08:54:44.631 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00099711000000000005 2024-05-08T08:54:44.631 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.631 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.631 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:54:44.631 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.620602+0000", 2024-05-08T08:54:44.631 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.324e-05 2024-05-08T08:54:44.631 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.631 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.631 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:44.632 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.620627+0000", 2024-05-08T08:54:44.632 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5585999999999999e-05 2024-05-08T08:54:44.632 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:44.632 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:44.632 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:44.632 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.632 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.632 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5490 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4227072~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:44.632 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.881809+0000", 2024-05-08T08:54:44.632 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.732520116, 2024-05-08T08:54:44.632 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047460447000000003, 2024-05-08T08:54:44.632 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:44.632 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:44.632 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:44.632 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:44.633 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:44.633 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:44.633 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5490 2024-05-08T08:54:44.633 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.633 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:44.633 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.633 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:44.633 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881809+0000", 2024-05-08T08:54:44.633 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:44.633 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.633 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.633 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:44.633 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881809+0000", 2024-05-08T08:54:44.633 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:44.633 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.634 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:44.634 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881809+0000", 2024-05-08T08:54:44.634 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7899999999999999e-07 2024-05-08T08:54:44.634 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.634 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:44.634 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881811+0000", 2024-05-08T08:54:44.634 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0339999999999999e-06 2024-05-08T08:54:44.634 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.634 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:44.634 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881818+0000", 2024-05-08T08:54:44.634 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5760000000000002e-06 2024-05-08T08:54:44.634 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.635 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.635 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:44.635 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881823+0000", 2024-05-08T08:54:44.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4160000000000003e-06 2024-05-08T08:54:44.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.635 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.635 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:44.635 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929157+0000", 2024-05-08T08:54:44.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047334029999999999 2024-05-08T08:54:44.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.635 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.635 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:44.635 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929211+0000", 2024-05-08T08:54:44.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3560999999999998e-05 2024-05-08T08:54:44.636 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.636 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:44.636 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929269+0000", 2024-05-08T08:54:44.636 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8350999999999997e-05 2024-05-08T08:54:44.636 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:44.636 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:44.636 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:44.636 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.636 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5492 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4272128~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:44.636 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.881830+0000", 2024-05-08T08:54:44.636 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.732498822, 2024-05-08T08:54:44.636 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047614006, 2024-05-08T08:54:44.636 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:44.636 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:44.637 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:44.637 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:44.637 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:44.637 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:44.637 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5492 2024-05-08T08:54:44.637 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.637 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:44.637 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.637 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:44.637 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881830+0000", 2024-05-08T08:54:44.637 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:44.637 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.637 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.637 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:44.637 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881830+0000", 2024-05-08T08:54:44.638 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:44.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.638 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.638 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:44.638 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881830+0000", 2024-05-08T08:54:44.638 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4099999999999999e-07 2024-05-08T08:54:44.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.638 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.638 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:44.638 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881832+0000", 2024-05-08T08:54:44.638 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0370000000000001e-06 2024-05-08T08:54:44.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.638 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.638 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:44.638 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881839+0000", 2024-05-08T08:54:44.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3289999999999999e-06 2024-05-08T08:54:44.639 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.639 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.639 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:44.639 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881844+0000", 2024-05-08T08:54:44.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1170000000000003e-06 2024-05-08T08:54:44.639 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.639 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.639 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:44.639 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929296+0000", 2024-05-08T08:54:44.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047452418000000003 2024-05-08T08:54:44.639 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.639 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.639 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:44.639 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929360+0000", 2024-05-08T08:54:44.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4197999999999998e-05 2024-05-08T08:54:44.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.640 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.640 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:44.640 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929444+0000", 2024-05-08T08:54:44.640 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3466000000000004e-05 2024-05-08T08:54:44.640 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:44.640 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:44.640 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:44.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.640 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.640 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5494 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4317184~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:44.640 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.881850+0000", 2024-05-08T08:54:44.640 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.732479073, 2024-05-08T08:54:44.640 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047723887, 2024-05-08T08:54:44.640 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:44.641 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:44.641 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:44.641 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:44.641 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:44.641 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:44.641 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5494 2024-05-08T08:54:44.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.641 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:44.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.641 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:44.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881850+0000", 2024-05-08T08:54:44.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:44.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.641 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:44.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881850+0000", 2024-05-08T08:54:44.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:44.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:44.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881850+0000", 2024-05-08T08:54:44.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8500000000000002e-07 2024-05-08T08:54:44.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:44.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881856+0000", 2024-05-08T08:54:44.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9510000000000003e-06 2024-05-08T08:54:44.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:44.643 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881862+0000", 2024-05-08T08:54:44.643 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3049999999999998e-06 2024-05-08T08:54:44.643 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.643 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.643 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:44.643 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881868+0000", 2024-05-08T08:54:44.643 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8340000000000004e-06 2024-05-08T08:54:44.643 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.643 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.643 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:44.643 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929475+0000", 2024-05-08T08:54:44.643 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047606796 2024-05-08T08:54:44.643 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.643 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.644 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:44.644 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929518+0000", 2024-05-08T08:54:44.644 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3387000000000003e-05 2024-05-08T08:54:44.644 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.644 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.644 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:44.644 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929574+0000", 2024-05-08T08:54:44.644 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5129e-05 2024-05-08T08:54:44.644 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:44.644 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:44.644 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:44.644 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.644 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.644 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5495 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4362240~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:44.644 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.881875+0000", 2024-05-08T08:54:44.645 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.732454206, 2024-05-08T08:54:44.645 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047847967999999998, 2024-05-08T08:54:44.645 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:44.645 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:44.645 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:44.645 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:44.645 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:44.645 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:44.645 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5495 2024-05-08T08:54:44.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.645 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:44.645 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.645 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:44.645 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881875+0000", 2024-05-08T08:54:44.645 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:44.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.646 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:44.646 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881875+0000", 2024-05-08T08:54:44.646 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:44.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.646 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:44.646 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881875+0000", 2024-05-08T08:54:44.646 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-07 2024-05-08T08:54:44.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.646 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:44.646 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881877+0000", 2024-05-08T08:54:44.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9659999999999999e-06 2024-05-08T08:54:44.647 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.647 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.647 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:44.647 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881883+0000", 2024-05-08T08:54:44.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7649999999999999e-06 2024-05-08T08:54:44.647 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.647 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.647 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:44.647 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881887+0000", 2024-05-08T08:54:44.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6689999999999997e-06 2024-05-08T08:54:44.647 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.647 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.647 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:44.647 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929600+0000", 2024-05-08T08:54:44.648 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047712746 2024-05-08T08:54:44.648 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.648 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.648 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:44.648 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929659+0000", 2024-05-08T08:54:44.648 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8936e-05 2024-05-08T08:54:44.648 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.648 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.648 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:44.648 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929723+0000", 2024-05-08T08:54:44.648 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3524999999999999e-05 2024-05-08T08:54:44.648 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:44.648 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:44.648 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:44.649 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.649 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.649 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5496 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4407296~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:44.649 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.881894+0000", 2024-05-08T08:54:44.649 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.732434342000001, 2024-05-08T08:54:44.649 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047998015999999998, 2024-05-08T08:54:44.649 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:44.649 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:44.649 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:44.649 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:44.649 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:44.649 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:44.649 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5496 2024-05-08T08:54:44.649 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.650 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:44.650 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.650 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:44.650 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881894+0000", 2024-05-08T08:54:44.650 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:44.650 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.650 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.650 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:44.650 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881894+0000", 2024-05-08T08:54:44.650 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:44.650 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.650 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.650 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:44.650 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881895+0000", 2024-05-08T08:54:44.651 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6699999999999999e-07 2024-05-08T08:54:44.651 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.651 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.651 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:44.651 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881897+0000", 2024-05-08T08:54:44.651 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0229999999999999e-06 2024-05-08T08:54:44.651 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.651 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.651 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:44.651 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881903+0000", 2024-05-08T08:54:44.651 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9850000000000003e-06 2024-05-08T08:54:44.651 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.651 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.651 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:44.651 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881908+0000", 2024-05-08T08:54:44.652 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9019999999999998e-06 2024-05-08T08:54:44.652 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.652 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.652 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:44.652 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929751+0000", 2024-05-08T08:54:44.652 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047843550999999998 2024-05-08T08:54:44.652 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.652 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.652 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:44.652 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929823+0000", 2024-05-08T08:54:44.652 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1172000000000002e-05 2024-05-08T08:54:44.652 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.652 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.652 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:44.652 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929892+0000", 2024-05-08T08:54:44.652 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9915999999999999e-05 2024-05-08T08:54:44.653 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:44.653 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:44.653 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:44.653 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.653 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.653 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5498 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4452352~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:44.653 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.881914+0000", 2024-05-08T08:54:44.653 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.732414584000001, 2024-05-08T08:54:44.653 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048203553000000003, 2024-05-08T08:54:44.653 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:44.653 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:44.653 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:44.653 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:44.653 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:44.653 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:44.654 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5498 2024-05-08T08:54:44.654 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.654 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:44.654 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.654 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:44.654 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881914+0000", 2024-05-08T08:54:44.654 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:44.654 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.654 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.654 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:44.654 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881914+0000", 2024-05-08T08:54:44.654 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:44.655 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.655 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.655 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:44.655 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881915+0000", 2024-05-08T08:54:44.655 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8500000000000002e-07 2024-05-08T08:54:44.655 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.655 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.655 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:44.655 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881917+0000", 2024-05-08T08:54:44.655 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.181e-06 2024-05-08T08:54:44.655 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.655 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.655 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:44.655 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881924+0000", 2024-05-08T08:54:44.655 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9619999999999996e-06 2024-05-08T08:54:44.655 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.656 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.656 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:44.656 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881929+0000", 2024-05-08T08:54:44.656 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1399999999999999e-06 2024-05-08T08:54:44.656 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.656 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.656 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:44.656 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929952+0000", 2024-05-08T08:54:44.656 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048023134000000002 2024-05-08T08:54:44.656 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.656 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.656 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:44.656 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.930029+0000", 2024-05-08T08:54:44.656 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7490000000000005e-05 2024-05-08T08:54:44.657 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.657 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.657 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:44.657 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.930118+0000", 2024-05-08T08:54:44.657 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8261000000000001e-05 2024-05-08T08:54:44.657 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:44.657 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:44.657 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:44.657 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.657 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.657 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5563 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4321280~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:44.657 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.884201+0000", 2024-05-08T08:54:44.657 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.730127333999999, 2024-05-08T08:54:44.657 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047379969000000001, 2024-05-08T08:54:44.658 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:44.658 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:44.658 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:44.658 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:44.658 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:44.658 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:44.658 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5563 2024-05-08T08:54:44.658 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.658 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:44.658 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.658 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:44.658 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884201+0000", 2024-05-08T08:54:44.658 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:44.658 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.658 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.659 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:44.659 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884201+0000", 2024-05-08T08:54:44.659 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:44.659 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.659 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.659 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:44.659 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884202+0000", 2024-05-08T08:54:44.659 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8000000000000002e-07 2024-05-08T08:54:44.659 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.659 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.659 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:44.659 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884208+0000", 2024-05-08T08:54:44.659 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.674e-06 2024-05-08T08:54:44.659 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.660 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.660 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:44.660 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884215+0000", 2024-05-08T08:54:44.660 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0389999999999999e-06 2024-05-08T08:54:44.660 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.660 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.660 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:44.660 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884220+0000", 2024-05-08T08:54:44.660 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4950000000000002e-06 2024-05-08T08:54:44.660 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.660 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.660 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:44.660 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931417+0000", 2024-05-08T08:54:44.660 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047196674000000001 2024-05-08T08:54:44.660 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.661 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.661 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:44.661 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931493+0000", 2024-05-08T08:54:44.661 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5950000000000003e-05 2024-05-08T08:54:44.661 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.661 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.661 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:44.661 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931581+0000", 2024-05-08T08:54:44.661 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8857000000000007e-05 2024-05-08T08:54:44.661 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:44.661 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:44.661 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:44.661 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.661 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.661 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5564 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4366336~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:44.662 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.884225+0000", 2024-05-08T08:54:44.662 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.730103794000001, 2024-05-08T08:54:44.662 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047555989999999999, 2024-05-08T08:54:44.662 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:44.662 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:44.662 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:44.662 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:44.662 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:44.662 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:44.662 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5564 2024-05-08T08:54:44.662 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.662 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:44.662 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.662 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:44.663 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884225+0000", 2024-05-08T08:54:44.663 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:44.663 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.663 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.663 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:44.663 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884225+0000", 2024-05-08T08:54:44.663 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:44.663 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.663 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.663 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:44.663 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884225+0000", 2024-05-08T08:54:44.663 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2499999999999995e-07 2024-05-08T08:54:44.663 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.663 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.663 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:44.663 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884227+0000", 2024-05-08T08:54:44.664 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5829999999999999e-06 2024-05-08T08:54:44.664 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.664 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.664 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:44.664 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884232+0000", 2024-05-08T08:54:44.664 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1159999999999998e-06 2024-05-08T08:54:44.664 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.664 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.664 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:44.664 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884236+0000", 2024-05-08T08:54:44.664 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2490000000000003e-06 2024-05-08T08:54:44.664 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.664 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.664 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:44.664 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931617+0000", 2024-05-08T08:54:44.664 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047380310000000002 2024-05-08T08:54:44.665 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.665 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.665 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:44.665 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931689+0000", 2024-05-08T08:54:44.665 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2358000000000001e-05 2024-05-08T08:54:44.665 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.665 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.665 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:44.665 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931781+0000", 2024-05-08T08:54:44.665 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1848999999999994e-05 2024-05-08T08:54:44.665 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:44.665 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:44.665 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:44.665 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.665 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.666 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5565 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4411392~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:44.666 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.884251+0000", 2024-05-08T08:54:44.666 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.730077641000001, 2024-05-08T08:54:44.666 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047743671000000001, 2024-05-08T08:54:44.666 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:44.666 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:44.666 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:44.666 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:44.666 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:44.666 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:44.666 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5565 2024-05-08T08:54:44.666 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.666 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:44.666 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.667 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:44.667 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884251+0000", 2024-05-08T08:54:44.667 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:44.667 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.667 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.667 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:44.667 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884251+0000", 2024-05-08T08:54:44.667 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:44.667 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.667 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.667 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:44.667 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884252+0000", 2024-05-08T08:54:44.667 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3200000000000005e-07 2024-05-08T08:54:44.667 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.667 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.668 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:44.668 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884253+0000", 2024-05-08T08:54:44.668 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6980000000000001e-06 2024-05-08T08:54:44.668 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.668 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.668 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:44.668 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884259+0000", 2024-05-08T08:54:44.668 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3759999999999999e-06 2024-05-08T08:54:44.668 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.668 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.668 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:44.668 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884263+0000", 2024-05-08T08:54:44.668 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7280000000000003e-06 2024-05-08T08:54:44.668 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.668 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.669 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:44.669 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931818+0000", 2024-05-08T08:54:44.669 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047554368999999999 2024-05-08T08:54:44.669 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.669 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.669 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:44.669 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931897+0000", 2024-05-08T08:54:44.669 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8821999999999999e-05 2024-05-08T08:54:44.669 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.669 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.669 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:44.669 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931995+0000", 2024-05-08T08:54:44.669 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8146000000000005e-05 2024-05-08T08:54:44.669 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:44.669 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:44.670 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:44.670 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.670 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.670 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5600 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 3948544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:44.670 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.884619+0000", 2024-05-08T08:54:44.670 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.729709386, 2024-05-08T08:54:44.670 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047574067999999997, 2024-05-08T08:54:44.670 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:44.670 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:44.670 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:44.670 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:44.670 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:44.670 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:44.670 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5600 2024-05-08T08:54:44.671 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.671 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:44.671 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.671 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:44.671 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884619+0000", 2024-05-08T08:54:44.671 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:44.671 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.671 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.671 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:44.671 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884619+0000", 2024-05-08T08:54:44.671 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:44.671 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.671 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.671 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:44.671 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884620+0000", 2024-05-08T08:54:44.672 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.53e-07 2024-05-08T08:54:44.672 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.672 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.672 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:44.672 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884622+0000", 2024-05-08T08:54:44.672 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.796e-06 2024-05-08T08:54:44.672 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.672 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.672 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:44.672 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884627+0000", 2024-05-08T08:54:44.672 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2850000000000002e-06 2024-05-08T08:54:44.672 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.672 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.672 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:44.672 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884631+0000", 2024-05-08T08:54:44.672 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2899999999999996e-06 2024-05-08T08:54:44.673 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.673 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.673 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:44.673 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932024+0000", 2024-05-08T08:54:44.673 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047392680999999999 2024-05-08T08:54:44.673 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.673 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.673 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:44.673 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932102+0000", 2024-05-08T08:54:44.673 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7806999999999996e-05 2024-05-08T08:54:44.673 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.673 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.673 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:44.673 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932193+0000", 2024-05-08T08:54:44.673 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1855999999999996e-05 2024-05-08T08:54:44.674 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:44.674 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:44.674 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:44.674 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.674 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.674 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5603 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 3993600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:44.674 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.884680+0000", 2024-05-08T08:54:44.674 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.729648777000001, 2024-05-08T08:54:44.674 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047745034999999998, 2024-05-08T08:54:44.674 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:44.674 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:44.674 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:44.674 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:44.674 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:44.674 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:44.675 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5603 2024-05-08T08:54:44.675 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.675 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:44.675 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.675 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:44.675 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884680+0000", 2024-05-08T08:54:44.675 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:44.675 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.675 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.675 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:44.675 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884680+0000", 2024-05-08T08:54:44.675 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:44.675 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.675 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.676 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:44.676 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884680+0000", 2024-05-08T08:54:44.676 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4400000000000001e-07 2024-05-08T08:54:44.676 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.676 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.676 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:44.676 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884682+0000", 2024-05-08T08:54:44.676 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.514e-06 2024-05-08T08:54:44.676 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.676 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.676 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:44.676 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884687+0000", 2024-05-08T08:54:44.676 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0359999999999997e-06 2024-05-08T08:54:44.676 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.676 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.676 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:44.677 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884692+0000", 2024-05-08T08:54:44.677 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5990000000000004e-06 2024-05-08T08:54:44.677 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.677 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.677 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:44.677 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932235+0000", 2024-05-08T08:54:44.677 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047543635000000001 2024-05-08T08:54:44.677 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.677 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.677 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:44.677 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932323+0000", 2024-05-08T08:54:44.677 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7311000000000005e-05 2024-05-08T08:54:44.677 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.677 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.677 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:44.678 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932425+0000", 2024-05-08T08:54:44.678 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102496 2024-05-08T08:54:44.678 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:44.678 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:44.678 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:44.678 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.678 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.678 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5606 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4038656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:44.678 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.884729+0000", 2024-05-08T08:54:44.678 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.729599489000002, 2024-05-08T08:54:44.678 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047916488, 2024-05-08T08:54:44.678 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:44.678 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:44.678 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:44.678 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:44.679 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:44.679 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:44.679 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5606 2024-05-08T08:54:44.679 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.679 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:44.679 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.679 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:44.679 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884729+0000", 2024-05-08T08:54:44.679 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:44.679 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.679 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.679 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:44.679 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884729+0000", 2024-05-08T08:54:44.679 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:44.679 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.680 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.680 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:44.680 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.885940+0000", 2024-05-08T08:54:44.680 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001210509 2024-05-08T08:54:44.680 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.680 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.680 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:44.680 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.885945+0000", 2024-05-08T08:54:44.680 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7389999999999999e-06 2024-05-08T08:54:44.680 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.680 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.680 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:44.680 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.885955+0000", 2024-05-08T08:54:44.680 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0813e-05 2024-05-08T08:54:44.680 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.680 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.681 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:44.681 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.885962+0000", 2024-05-08T08:54:44.681 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1160000000000001e-06 2024-05-08T08:54:44.681 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.681 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.681 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:44.681 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932465+0000", 2024-05-08T08:54:44.681 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046502612999999998 2024-05-08T08:54:44.681 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.681 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.681 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:44.681 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932547+0000", 2024-05-08T08:54:44.681 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2025000000000002e-05 2024-05-08T08:54:44.681 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.681 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.682 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:44.682 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932646+0000", 2024-05-08T08:54:44.682 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8672999999999995e-05 2024-05-08T08:54:44.682 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:44.682 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:44.682 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:44.682 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.682 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.682 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5635 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4235264~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:44.682 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.886082+0000", 2024-05-08T08:54:44.682 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.728246370000001, 2024-05-08T08:54:44.682 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047438284999999997, 2024-05-08T08:54:44.682 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:44.683 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:44.683 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:44.683 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:44.683 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:44.683 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:44.683 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5635 2024-05-08T08:54:44.683 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.683 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:44.683 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.683 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:44.683 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886082+0000", 2024-05-08T08:54:44.683 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:44.683 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.683 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.683 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:44.684 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886082+0000", 2024-05-08T08:54:44.684 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:44.684 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.684 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.684 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:44.684 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886083+0000", 2024-05-08T08:54:44.684 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8999999999999996e-07 2024-05-08T08:54:44.684 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.684 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.684 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:44.685 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886086+0000", 2024-05-08T08:54:44.685 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7e-06 2024-05-08T08:54:44.685 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.685 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.685 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:44.685 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886093+0000", 2024-05-08T08:54:44.685 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5320000000000001e-06 2024-05-08T08:54:44.685 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.685 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.685 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:44.685 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886102+0000", 2024-05-08T08:54:44.685 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8070000000000006e-06 2024-05-08T08:54:44.685 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.685 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.686 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:44.686 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933357+0000", 2024-05-08T08:54:44.686 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047255093999999997 2024-05-08T08:54:44.686 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.686 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.686 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:44.686 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933433+0000", 2024-05-08T08:54:44.686 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5796000000000004e-05 2024-05-08T08:54:44.686 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.686 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.686 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:44.686 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933521+0000", 2024-05-08T08:54:44.686 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7766e-05 2024-05-08T08:54:44.686 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:44.687 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:44.687 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:44.687 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.687 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.687 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5636 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4280320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:44.687 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.886111+0000", 2024-05-08T08:54:44.687 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.728217751999999, 2024-05-08T08:54:44.687 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047663644999999998, 2024-05-08T08:54:44.687 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:44.687 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:44.687 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:44.687 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:44.687 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:44.687 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:44.688 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5636 2024-05-08T08:54:44.688 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.688 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:44.688 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.688 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:44.688 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886111+0000", 2024-05-08T08:54:44.688 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:44.688 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.688 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.688 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:44.688 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886111+0000", 2024-05-08T08:54:44.688 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:44.688 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.688 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.688 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:44.689 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886111+0000", 2024-05-08T08:54:44.689 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5200000000000002e-07 2024-05-08T08:54:44.689 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.689 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.689 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:44.689 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886114+0000", 2024-05-08T08:54:44.689 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1710000000000001e-06 2024-05-08T08:54:44.689 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.689 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.689 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:44.689 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886120+0000", 2024-05-08T08:54:44.689 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.404e-06 2024-05-08T08:54:44.689 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.689 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.689 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:44.690 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886125+0000", 2024-05-08T08:54:44.690 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2730000000000002e-06 2024-05-08T08:54:44.690 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.690 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.690 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:44.690 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933570+0000", 2024-05-08T08:54:44.690 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047444989999999999 2024-05-08T08:54:44.690 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.690 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.690 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:44.690 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933659+0000", 2024-05-08T08:54:44.690 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8912999999999994e-05 2024-05-08T08:54:44.690 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.690 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.691 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:44.691 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933775+0000", 2024-05-08T08:54:44.691 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000115442 2024-05-08T08:54:44.691 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:44.691 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:44.691 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:44.691 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.691 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.691 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5637 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4325376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:44.692 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.886132+0000", 2024-05-08T08:54:44.692 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.728196713999999, 2024-05-08T08:54:44.692 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047852219000000001, 2024-05-08T08:54:44.692 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:44.692 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:44.692 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:44.692 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:44.692 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:44.692 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:44.692 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5637 2024-05-08T08:54:44.692 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.692 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:44.692 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.692 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:44.693 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886132+0000", 2024-05-08T08:54:44.693 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:44.693 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.693 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.693 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:44.693 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886132+0000", 2024-05-08T08:54:44.693 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:44.693 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.693 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.693 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:44.693 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886132+0000", 2024-05-08T08:54:44.693 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7300000000000002e-07 2024-05-08T08:54:44.693 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.693 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.693 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:44.693 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886139+0000", 2024-05-08T08:54:44.694 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6109999999999999e-06 2024-05-08T08:54:44.694 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.694 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.694 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:44.694 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886146+0000", 2024-05-08T08:54:44.694 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6279999999999999e-06 2024-05-08T08:54:44.694 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.694 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.694 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:44.694 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886151+0000", 2024-05-08T08:54:44.694 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1769999999999997e-06 2024-05-08T08:54:44.694 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.694 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.694 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:44.695 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933812+0000", 2024-05-08T08:54:44.695 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047660912999999999 2024-05-08T08:54:44.695 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.695 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.695 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:44.695 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933888+0000", 2024-05-08T08:54:44.695 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5865000000000006e-05 2024-05-08T08:54:44.695 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.695 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.695 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:44.695 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933984+0000", 2024-05-08T08:54:44.695 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6651999999999996e-05 2024-05-08T08:54:44.695 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:44.695 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:44.695 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:44.696 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.696 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.696 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5638 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4370432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:44.696 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.886157+0000", 2024-05-08T08:54:44.696 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.728171450000001, 2024-05-08T08:54:44.696 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048077597, 2024-05-08T08:54:44.696 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:44.696 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:44.696 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:44.696 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:44.696 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:44.696 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:44.696 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5638 2024-05-08T08:54:44.696 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.696 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:44.697 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.697 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:44.697 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886157+0000", 2024-05-08T08:54:44.697 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:44.697 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.697 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.697 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:44.697 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886157+0000", 2024-05-08T08:54:44.697 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:44.697 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.697 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.697 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:44.697 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886158+0000", 2024-05-08T08:54:44.697 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1100000000000001e-07 2024-05-08T08:54:44.698 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.698 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.698 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:44.698 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886160+0000", 2024-05-08T08:54:44.698 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9800000000000001e-06 2024-05-08T08:54:44.698 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.698 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.698 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:44.698 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886166+0000", 2024-05-08T08:54:44.698 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.314e-06 2024-05-08T08:54:44.698 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.698 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.698 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:44.698 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886171+0000", 2024-05-08T08:54:44.699 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7700000000000001e-06 2024-05-08T08:54:44.699 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.699 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.699 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:44.699 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.934025+0000", 2024-05-08T08:54:44.699 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047854509000000003 2024-05-08T08:54:44.699 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.699 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.699 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:44.699 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.934121+0000", 2024-05-08T08:54:44.699 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5688999999999998e-05 2024-05-08T08:54:44.699 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.699 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.700 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:44.700 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.934235+0000", 2024-05-08T08:54:44.700 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000113924 2024-05-08T08:54:44.700 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:44.700 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:44.700 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:44.700 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.700 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.700 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5639 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4415488~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:44.700 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.886177+0000", 2024-05-08T08:54:44.700 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.728151700000002, 2024-05-08T08:54:44.700 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048249541, 2024-05-08T08:54:44.700 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:44.700 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:44.700 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:44.701 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:44.701 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:44.701 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:44.701 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5639 2024-05-08T08:54:44.701 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.701 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:44.701 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.701 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:44.701 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886177+0000", 2024-05-08T08:54:44.701 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:44.701 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.701 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.701 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:44.701 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886177+0000", 2024-05-08T08:54:44.701 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:44.702 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.702 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.702 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:44.702 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886177+0000", 2024-05-08T08:54:44.702 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.65e-07 2024-05-08T08:54:44.702 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.702 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.702 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:44.702 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886179+0000", 2024-05-08T08:54:44.702 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9530000000000002e-06 2024-05-08T08:54:44.702 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.702 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.702 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:44.703 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886186+0000", 2024-05-08T08:54:44.703 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3419999999999996e-06 2024-05-08T08:54:44.703 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.703 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.703 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:44.703 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886192+0000", 2024-05-08T08:54:44.703 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0560000000000001e-06 2024-05-08T08:54:44.703 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.703 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.703 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:44.703 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.934258+0000", 2024-05-08T08:54:44.703 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048066367999999998 2024-05-08T08:54:44.703 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.703 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.703 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:44.703 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.934338+0000", 2024-05-08T08:54:44.704 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9616000000000005e-05 2024-05-08T08:54:44.704 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:44.704 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:44.704 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:44.704 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.934427+0000", 2024-05-08T08:54:44.704 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8840999999999999e-05 2024-05-08T08:54:44.704 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:44.704 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:44.704 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:44.704 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:44.704 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:44.704 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:44.706 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:54:44.706 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:44.708+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:44.806 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:44.809+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:44.862 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:44.862 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:54:44.862 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:54:44.862 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:44.872 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:54:44.913 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:44.916+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:45.008 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:45.010+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:45.023 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:45.024 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:54:45.024 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:54:45.024 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:54:45.024 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:45.033 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:54:45.109 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:45.112+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:45.141 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:45.142 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:54:45.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:54:45.142 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:54:45.142 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.142 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22073 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 782336~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:45.142 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:54:45.142 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 137.28574098199999, 2024-05-08T08:54:45.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047244234000000003, 2024-05-08T08:54:45.142 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:45.142 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:45.142 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:45.142 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:45.143 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:45.143 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:45.143 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22073 2024-05-08T08:54:45.143 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.143 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:45.143 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.143 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:45.143 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:54:45.143 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:45.143 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.143 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.143 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:45.143 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:54:45.143 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:45.143 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.144 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.144 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:45.144 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857668+0000", 2024-05-08T08:54:45.144 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7800000000000002e-07 2024-05-08T08:54:45.144 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.144 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.144 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:45.144 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857670+0000", 2024-05-08T08:54:45.144 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0820000000000001e-06 2024-05-08T08:54:45.144 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.144 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.144 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:45.144 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857678+0000", 2024-05-08T08:54:45.144 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7589999999999997e-06 2024-05-08T08:54:45.145 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.145 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.145 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:45.145 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857691+0000", 2024-05-08T08:54:45.145 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3672999999999999e-05 2024-05-08T08:54:45.145 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.145 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.145 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:45.145 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904780+0000", 2024-05-08T08:54:45.145 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047088616999999999 2024-05-08T08:54:45.145 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.145 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.145 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:45.145 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904840+0000", 2024-05-08T08:54:45.145 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0356999999999998e-05 2024-05-08T08:54:45.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.146 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.146 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:45.146 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904912+0000", 2024-05-08T08:54:45.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1367999999999998e-05 2024-05-08T08:54:45.146 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.146 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:45.146 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.146 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.146 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22075 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 798720~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:45.146 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:54:45.146 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 137.285710585, 2024-05-08T08:54:45.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047384643999999997, 2024-05-08T08:54:45.147 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:45.147 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:45.147 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:45.147 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:45.147 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:45.147 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:45.147 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22075 2024-05-08T08:54:45.147 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.147 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:45.147 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.147 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:45.147 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:54:45.147 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:45.147 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.147 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.147 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:45.148 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:54:45.148 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:45.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.148 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.148 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:45.148 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:54:45.148 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7399999999999999e-07 2024-05-08T08:54:45.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.148 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.148 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:45.148 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857705+0000", 2024-05-08T08:54:45.148 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1980000000000004e-06 2024-05-08T08:54:45.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.148 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.148 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:45.149 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857713+0000", 2024-05-08T08:54:45.149 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7039999999999994e-06 2024-05-08T08:54:45.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.149 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:45.149 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857718+0000", 2024-05-08T08:54:45.149 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.865e-06 2024-05-08T08:54:45.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.149 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:45.149 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904941+0000", 2024-05-08T08:54:45.149 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047223198000000001 2024-05-08T08:54:45.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.149 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:45.149 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904999+0000", 2024-05-08T08:54:45.150 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8254999999999997e-05 2024-05-08T08:54:45.150 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.150 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.150 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:45.150 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905082+0000", 2024-05-08T08:54:45.150 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3049999999999999e-05 2024-05-08T08:54:45.150 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.150 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:45.150 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.150 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.150 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.150 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22077 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 815104~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:45.150 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:54:45.150 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 137.28567257399999, 2024-05-08T08:54:45.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047519448999999998, 2024-05-08T08:54:45.151 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:45.151 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:45.151 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:45.151 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:45.151 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:45.151 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:45.151 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22077 2024-05-08T08:54:45.151 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.151 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:45.151 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.151 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:45.151 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:54:45.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:45.152 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.152 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.152 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:45.152 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:54:45.152 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:45.152 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.152 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.152 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:45.152 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:54:45.152 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.99e-07 2024-05-08T08:54:45.152 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.152 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.152 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:45.152 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857739+0000", 2024-05-08T08:54:45.152 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3599999999999999e-06 2024-05-08T08:54:45.153 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.153 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.153 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:45.153 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857745+0000", 2024-05-08T08:54:45.153 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5139999999999998e-06 2024-05-08T08:54:45.153 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.153 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.153 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:45.153 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857750+0000", 2024-05-08T08:54:45.153 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8550000000000001e-06 2024-05-08T08:54:45.153 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.153 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.153 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:45.153 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905110+0000", 2024-05-08T08:54:45.153 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047360126000000002 2024-05-08T08:54:45.154 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.154 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.154 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:45.154 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905175+0000", 2024-05-08T08:54:45.154 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4689000000000004e-05 2024-05-08T08:54:45.154 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.154 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.154 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:45.154 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905255+0000", 2024-05-08T08:54:45.154 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0306e-05 2024-05-08T08:54:45.154 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.154 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:45.154 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.154 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.154 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.155 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22079 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 831488~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:45.155 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:54:45.155 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 137.28565202600001, 2024-05-08T08:54:45.155 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.04767499, 2024-05-08T08:54:45.155 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:45.155 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:45.155 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:45.155 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:45.155 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:45.155 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:45.155 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22079 2024-05-08T08:54:45.155 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.155 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:45.156 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.156 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:45.156 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:54:45.156 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:45.156 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.156 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.156 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:45.156 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:54:45.156 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:45.156 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.156 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.156 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:45.156 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857757+0000", 2024-05-08T08:54:45.156 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8299999999999997e-07 2024-05-08T08:54:45.156 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.157 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.157 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:45.157 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857759+0000", 2024-05-08T08:54:45.157 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1390000000000002e-06 2024-05-08T08:54:45.157 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.157 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.157 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:45.157 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857765+0000", 2024-05-08T08:54:45.157 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4609999999999996e-06 2024-05-08T08:54:45.157 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.157 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.157 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:45.157 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857770+0000", 2024-05-08T08:54:45.157 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6689999999999997e-06 2024-05-08T08:54:45.157 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.158 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.158 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:45.158 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905281+0000", 2024-05-08T08:54:45.158 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047510668999999998 2024-05-08T08:54:45.158 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.158 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.158 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:45.158 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905344+0000", 2024-05-08T08:54:45.158 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2797000000000002e-05 2024-05-08T08:54:45.158 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.158 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.158 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:45.158 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905431+0000", 2024-05-08T08:54:45.158 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7771999999999999e-05 2024-05-08T08:54:45.159 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.159 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:45.159 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.159 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.159 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.159 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22082 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 847872~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:45.159 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:54:45.159 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 137.285632335, 2024-05-08T08:54:45.159 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047831934999999999, 2024-05-08T08:54:45.159 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:45.159 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:45.159 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:45.159 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:45.159 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:45.159 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:45.159 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22082 2024-05-08T08:54:45.160 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.160 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:45.160 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.160 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:45.160 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:54:45.160 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:45.160 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.160 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.160 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:45.160 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:54:45.160 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:45.160 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.160 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.160 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:45.160 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:54:45.160 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:54:45.161 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.161 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.161 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:45.161 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857778+0000", 2024-05-08T08:54:45.161 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0789999999999999e-06 2024-05-08T08:54:45.161 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.161 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.161 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:45.161 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857784+0000", 2024-05-08T08:54:45.161 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0480000000000004e-06 2024-05-08T08:54:45.161 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.161 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.161 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:45.161 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857790+0000", 2024-05-08T08:54:45.161 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2719999999999997e-06 2024-05-08T08:54:45.162 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.162 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.162 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:45.162 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905458+0000", 2024-05-08T08:54:45.162 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047668470999999997 2024-05-08T08:54:45.162 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.162 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.162 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:45.162 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905521+0000", 2024-05-08T08:54:45.162 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2952000000000003e-05 2024-05-08T08:54:45.162 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.162 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.162 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:45.162 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905608+0000", 2024-05-08T08:54:45.163 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6759000000000001e-05 2024-05-08T08:54:45.163 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.163 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:45.163 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.163 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.163 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.163 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22084 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 864256~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:45.163 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:54:45.163 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 137.28560838499999, 2024-05-08T08:54:45.163 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047977748000000001, 2024-05-08T08:54:45.163 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:45.163 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:45.163 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:45.163 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:45.163 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:45.163 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:45.164 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22084 2024-05-08T08:54:45.164 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.164 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:45.164 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.164 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:45.164 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:54:45.164 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:45.164 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.164 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.164 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:45.164 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:54:45.164 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:45.164 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.164 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.165 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:45.165 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:54:45.165 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:54:45.165 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.165 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.165 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:45.165 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857802+0000", 2024-05-08T08:54:45.165 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0779999999999998e-06 2024-05-08T08:54:45.165 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.165 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.165 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:45.165 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857812+0000", 2024-05-08T08:54:45.165 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7939999999999997e-06 2024-05-08T08:54:45.165 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.165 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.165 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:45.165 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857817+0000", 2024-05-08T08:54:45.166 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8899999999999998e-06 2024-05-08T08:54:45.166 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.166 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.166 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:45.166 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905636+0000", 2024-05-08T08:54:45.166 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047819252 2024-05-08T08:54:45.166 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.166 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.166 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:45.166 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905700+0000", 2024-05-08T08:54:45.166 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4016999999999994e-05 2024-05-08T08:54:45.166 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.166 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.166 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:45.167 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905778+0000", 2024-05-08T08:54:45.167 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7306999999999997e-05 2024-05-08T08:54:45.167 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.167 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:45.167 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.167 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.167 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.167 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22086 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 880640~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:45.167 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:54:45.167 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 137.285585035, 2024-05-08T08:54:45.167 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048119149, 2024-05-08T08:54:45.167 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:45.167 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:45.167 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:45.167 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:45.168 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:45.168 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:45.168 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22086 2024-05-08T08:54:45.168 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.168 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:45.168 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.168 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:45.168 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:54:45.168 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:45.168 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.168 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.168 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:45.168 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:54:45.168 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:45.169 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.169 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.169 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:45.169 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857824+0000", 2024-05-08T08:54:45.169 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4200000000000002e-07 2024-05-08T08:54:45.169 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.169 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.169 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:45.169 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857826+0000", 2024-05-08T08:54:45.169 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0590000000000001e-06 2024-05-08T08:54:45.169 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.169 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.169 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:45.169 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857832+0000", 2024-05-08T08:54:45.170 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8029999999999999e-06 2024-05-08T08:54:45.170 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.170 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.170 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:45.170 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857837+0000", 2024-05-08T08:54:45.170 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7929999999999997e-06 2024-05-08T08:54:45.170 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.170 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.170 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:45.170 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905803+0000", 2024-05-08T08:54:45.170 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047965761000000003 2024-05-08T08:54:45.170 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.170 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.170 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:45.171 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905862+0000", 2024-05-08T08:54:45.171 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9203000000000002e-05 2024-05-08T08:54:45.171 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.171 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.171 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:45.171 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905942+0000", 2024-05-08T08:54:45.171 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0187999999999999e-05 2024-05-08T08:54:45.171 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.171 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:45.171 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.171 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.171 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.171 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22088 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 897024~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:45.171 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:54:45.172 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 137.285564983, 2024-05-08T08:54:45.172 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048256049000000002, 2024-05-08T08:54:45.172 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:45.172 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:45.172 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:45.172 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:45.172 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:45.172 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:45.172 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22088 2024-05-08T08:54:45.172 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.172 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:45.172 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.172 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:45.172 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:54:45.172 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:45.173 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.173 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.173 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:45.173 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:54:45.173 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:45.173 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.173 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.173 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:45.173 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857844+0000", 2024-05-08T08:54:45.173 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.46e-07 2024-05-08T08:54:45.173 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.173 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.173 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:45.173 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857846+0000", 2024-05-08T08:54:45.174 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0899999999999999e-06 2024-05-08T08:54:45.174 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.174 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.174 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:45.174 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857855+0000", 2024-05-08T08:54:45.174 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8219999999999996e-06 2024-05-08T08:54:45.174 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.174 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.174 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:45.174 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857860+0000", 2024-05-08T08:54:45.174 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2989999999999996e-06 2024-05-08T08:54:45.174 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.174 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.174 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:45.174 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905966+0000", 2024-05-08T08:54:45.175 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048106542000000002 2024-05-08T08:54:45.175 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.175 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.175 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:45.175 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906024+0000", 2024-05-08T08:54:45.175 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7958000000000003e-05 2024-05-08T08:54:45.175 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.175 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.175 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:45.175 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906099+0000", 2024-05-08T08:54:45.175 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4992000000000003e-05 2024-05-08T08:54:45.175 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.175 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:45.175 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.176 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.176 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.176 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22091 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 913408~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:45.176 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:54:45.176 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 137.285541223, 2024-05-08T08:54:45.176 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048395290000000001, 2024-05-08T08:54:45.176 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:45.176 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:45.176 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:45.176 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:45.176 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:45.176 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:45.176 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22091 2024-05-08T08:54:45.176 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.176 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:45.176 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.177 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:45.177 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:54:45.177 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:45.177 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.177 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.177 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:45.177 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:54:45.177 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:45.177 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.177 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.177 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:45.177 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:54:45.177 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4900000000000001e-07 2024-05-08T08:54:45.177 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.177 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.178 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:45.178 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857870+0000", 2024-05-08T08:54:45.178 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2510000000000001e-06 2024-05-08T08:54:45.178 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.178 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.178 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:45.178 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857877+0000", 2024-05-08T08:54:45.178 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0809999999999996e-06 2024-05-08T08:54:45.178 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.178 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.178 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:45.178 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857881+0000", 2024-05-08T08:54:45.178 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.724e-06 2024-05-08T08:54:45.178 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.178 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.179 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:45.179 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906127+0000", 2024-05-08T08:54:45.179 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048245345000000002 2024-05-08T08:54:45.179 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.179 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.179 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:45.179 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906190+0000", 2024-05-08T08:54:45.179 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2762000000000007e-05 2024-05-08T08:54:45.179 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.179 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.179 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:45.179 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906262+0000", 2024-05-08T08:54:45.179 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2778e-05 2024-05-08T08:54:45.179 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.180 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:45.180 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.180 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.180 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.180 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22093 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 929792~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:45.180 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:54:45.180 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 137.28551885900001, 2024-05-08T08:54:45.180 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048532875000000003, 2024-05-08T08:54:45.180 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:45.180 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:45.180 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:45.180 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:45.180 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:45.180 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:45.180 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22093 2024-05-08T08:54:45.180 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.180 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:45.181 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.181 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:45.181 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:54:45.181 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:45.181 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.181 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.181 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:45.181 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:54:45.181 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:45.181 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.181 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.181 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:45.181 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857890+0000", 2024-05-08T08:54:45.181 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-07 2024-05-08T08:54:45.181 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.182 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.182 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:45.182 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857899+0000", 2024-05-08T08:54:45.182 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6889999999999999e-06 2024-05-08T08:54:45.182 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.182 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.182 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:45.182 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857906+0000", 2024-05-08T08:54:45.182 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6490000000000002e-06 2024-05-08T08:54:45.182 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.182 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.182 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:45.182 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857917+0000", 2024-05-08T08:54:45.182 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0764999999999999e-05 2024-05-08T08:54:45.183 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.183 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.183 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:45.183 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906287+0000", 2024-05-08T08:54:45.183 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048370442 2024-05-08T08:54:45.183 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.183 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.183 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:45.183 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906349+0000", 2024-05-08T08:54:45.183 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1537000000000004e-05 2024-05-08T08:54:45.183 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.183 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.183 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:45.183 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906422+0000", 2024-05-08T08:54:45.183 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3431999999999998e-05 2024-05-08T08:54:45.183 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.184 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:45.184 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.184 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.184 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.184 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22095 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 946176~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:45.184 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:54:45.184 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 137.28548415500001, 2024-05-08T08:54:45.184 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048657593999999998, 2024-05-08T08:54:45.184 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:45.184 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:45.184 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:45.184 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:45.184 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:45.184 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:45.185 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22095 2024-05-08T08:54:45.185 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.185 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:45.185 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.185 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:45.185 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:54:45.185 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:45.185 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.185 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.185 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:45.185 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:54:45.185 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:45.185 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.185 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.186 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:45.186 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857925+0000", 2024-05-08T08:54:45.186 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3099999999999998e-07 2024-05-08T08:54:45.186 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.186 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.186 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:45.186 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857927+0000", 2024-05-08T08:54:45.186 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.34e-06 2024-05-08T08:54:45.186 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.186 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.186 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:45.187 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857938+0000", 2024-05-08T08:54:45.187 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1426e-05 2024-05-08T08:54:45.187 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.187 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.187 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:45.187 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857943+0000", 2024-05-08T08:54:45.187 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5140000000000003e-06 2024-05-08T08:54:45.187 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.187 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.187 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:45.187 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906448+0000", 2024-05-08T08:54:45.188 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048504641000000001 2024-05-08T08:54:45.188 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.188 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.188 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:45.188 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906510+0000", 2024-05-08T08:54:45.188 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2663000000000006e-05 2024-05-08T08:54:45.188 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.188 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.188 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:45.188 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906582+0000", 2024-05-08T08:54:45.188 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1478999999999998e-05 2024-05-08T08:54:45.188 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.188 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:45.188 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.188 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.189 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.189 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22097 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 962560~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:45.189 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:54:45.189 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 137.285456466, 2024-05-08T08:54:45.189 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048791353000000003, 2024-05-08T08:54:45.189 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:45.189 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:45.189 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:45.189 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:45.189 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:45.189 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:45.189 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22097 2024-05-08T08:54:45.189 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.189 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:45.189 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.190 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:45.190 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:54:45.190 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:45.190 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.190 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.190 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:45.190 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:54:45.190 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:45.190 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.190 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.190 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:45.190 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:54:45.190 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9200000000000002e-07 2024-05-08T08:54:45.191 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.191 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.191 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:45.191 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857954+0000", 2024-05-08T08:54:45.191 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1900000000000002e-06 2024-05-08T08:54:45.191 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.191 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.191 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:45.191 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857961+0000", 2024-05-08T08:54:45.191 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4989999999999999e-06 2024-05-08T08:54:45.191 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.191 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.191 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:45.191 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857971+0000", 2024-05-08T08:54:45.191 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0485999999999999e-05 2024-05-08T08:54:45.192 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.192 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.192 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:45.192 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906607+0000", 2024-05-08T08:54:45.192 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048635109000000003 2024-05-08T08:54:45.192 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.192 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.192 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:45.192 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906667+0000", 2024-05-08T08:54:45.192 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0439000000000002e-05 2024-05-08T08:54:45.192 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.192 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.192 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:45.193 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906743+0000", 2024-05-08T08:54:45.193 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6237999999999998e-05 2024-05-08T08:54:45.193 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.193 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:45.193 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.193 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.193 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.193 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22099 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 978944~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:45.193 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:54:45.193 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 137.28543030599999, 2024-05-08T08:54:45.193 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048932055000000002, 2024-05-08T08:54:45.193 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:45.193 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:45.193 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:45.194 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:45.194 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:45.194 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:45.194 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22099 2024-05-08T08:54:45.194 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.194 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:45.194 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.194 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:45.194 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:54:45.194 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:45.194 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.194 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.194 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:45.194 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:54:45.195 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:45.195 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.195 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.195 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:45.195 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:54:45.195 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.27e-07 2024-05-08T08:54:45.195 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.195 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.195 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:45.195 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857987+0000", 2024-05-08T08:54:45.195 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5890000000000003e-06 2024-05-08T08:54:45.195 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.195 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.195 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:45.196 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857994+0000", 2024-05-08T08:54:45.196 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6619999999999999e-06 2024-05-08T08:54:45.196 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.196 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.196 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:45.196 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857998+0000", 2024-05-08T08:54:45.196 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7110000000000003e-06 2024-05-08T08:54:45.196 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.196 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.196 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:45.196 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906772+0000", 2024-05-08T08:54:45.196 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048773415000000001 2024-05-08T08:54:45.196 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.196 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.196 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:45.197 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906834+0000", 2024-05-08T08:54:45.197 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1752000000000001e-05 2024-05-08T08:54:45.197 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.197 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.197 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:45.197 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906910+0000", 2024-05-08T08:54:45.197 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6499000000000001e-05 2024-05-08T08:54:45.197 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.197 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:45.197 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.197 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.197 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.197 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22101 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 995328~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:45.197 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:54:45.198 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 137.285403175, 2024-05-08T08:54:45.198 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049087364000000001, 2024-05-08T08:54:45.198 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:45.198 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:45.198 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:45.198 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:45.198 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:45.198 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:45.198 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22101 2024-05-08T08:54:45.198 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.198 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:45.198 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.198 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:45.198 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:54:45.199 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:45.199 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.199 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.199 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:45.199 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:54:45.199 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:45.199 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.199 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.199 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:45.199 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858006+0000", 2024-05-08T08:54:45.199 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1399999999999997e-07 2024-05-08T08:54:45.199 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.199 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.199 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:45.200 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858008+0000", 2024-05-08T08:54:45.200 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2060000000000001e-06 2024-05-08T08:54:45.200 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.200 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.200 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:45.200 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858017+0000", 2024-05-08T08:54:45.200 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1579999999999994e-06 2024-05-08T08:54:45.200 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.200 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.200 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:45.200 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858022+0000", 2024-05-08T08:54:45.200 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4199999999999998e-06 2024-05-08T08:54:45.200 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.201 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.201 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:45.201 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906940+0000", 2024-05-08T08:54:45.201 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048917627999999998 2024-05-08T08:54:45.201 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.201 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.201 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:45.201 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907002+0000", 2024-05-08T08:54:45.201 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2399999999999999e-05 2024-05-08T08:54:45.201 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.201 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.201 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:45.201 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907092+0000", 2024-05-08T08:54:45.201 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0137999999999997e-05 2024-05-08T08:54:45.201 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.202 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:45.202 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.202 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.202 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.202 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22103 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1011712~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:45.202 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:54:45.202 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 137.28537839000001, 2024-05-08T08:54:45.202 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049228542, 2024-05-08T08:54:45.202 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:45.202 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:45.202 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:45.202 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:45.202 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:45.202 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:45.203 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22103 2024-05-08T08:54:45.203 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.203 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:45.203 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.203 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:45.203 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:54:45.203 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:45.203 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.203 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.203 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:45.203 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:54:45.203 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:45.203 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.203 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.203 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:45.203 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:54:45.204 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4299999999999999e-07 2024-05-08T08:54:45.204 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.204 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.204 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:45.204 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858032+0000", 2024-05-08T08:54:45.204 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0590000000000001e-06 2024-05-08T08:54:45.204 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.204 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.204 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:45.204 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858037+0000", 2024-05-08T08:54:45.204 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.977e-06 2024-05-08T08:54:45.204 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.204 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.204 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:45.204 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858042+0000", 2024-05-08T08:54:45.204 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7650000000000001e-06 2024-05-08T08:54:45.205 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.205 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.205 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:45.205 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907118+0000", 2024-05-08T08:54:45.205 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049076147000000001 2024-05-08T08:54:45.205 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.205 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.205 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:45.205 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907180+0000", 2024-05-08T08:54:45.205 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1971999999999996e-05 2024-05-08T08:54:45.205 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.205 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.205 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:45.205 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907258+0000", 2024-05-08T08:54:45.206 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8279e-05 2024-05-08T08:54:45.206 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.206 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:45.206 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.206 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.206 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.206 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22105 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1028096~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:45.206 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:54:45.206 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 137.28535914700001, 2024-05-08T08:54:45.206 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049394609999999999, 2024-05-08T08:54:45.206 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:45.206 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:45.206 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:45.206 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:45.206 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:45.207 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:45.207 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22105 2024-05-08T08:54:45.207 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.207 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:45.207 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.207 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:45.207 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:54:45.207 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:45.207 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.207 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.207 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:45.207 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:54:45.207 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:45.207 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.207 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.208 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:45.208 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858050+0000", 2024-05-08T08:54:45.208 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5999999999999999e-07 2024-05-08T08:54:45.208 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.208 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.208 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:45.208 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858051+0000", 2024-05-08T08:54:45.208 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8870000000000001e-06 2024-05-08T08:54:45.208 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.208 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.208 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:45.208 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858057+0000", 2024-05-08T08:54:45.208 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8019999999999997e-06 2024-05-08T08:54:45.208 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.209 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.209 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:45.209 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858062+0000", 2024-05-08T08:54:45.209 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.561e-06 2024-05-08T08:54:45.209 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.209 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.209 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:45.209 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907289+0000", 2024-05-08T08:54:45.209 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049226957000000002 2024-05-08T08:54:45.209 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.209 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.209 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:45.209 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907356+0000", 2024-05-08T08:54:45.209 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6915000000000006e-05 2024-05-08T08:54:45.210 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.210 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.210 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:45.210 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907444+0000", 2024-05-08T08:54:45.210 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8127999999999994e-05 2024-05-08T08:54:45.210 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.210 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:45.210 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.210 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.210 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.210 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22107 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1044480~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:45.210 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:54:45.210 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 137.28534080700001, 2024-05-08T08:54:45.210 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049799942999999999, 2024-05-08T08:54:45.210 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:45.211 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:45.211 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:45.211 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:45.211 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:45.211 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:45.211 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22107 2024-05-08T08:54:45.211 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.211 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:45.211 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.211 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:45.211 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:54:45.211 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:45.211 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.211 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.212 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:45.212 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:54:45.212 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:45.212 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.212 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.212 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:45.212 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:54:45.212 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4799999999999999e-07 2024-05-08T08:54:45.212 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.212 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.212 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:45.212 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858070+0000", 2024-05-08T08:54:45.212 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.012e-06 2024-05-08T08:54:45.212 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.212 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.213 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:45.213 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858077+0000", 2024-05-08T08:54:45.213 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6290000000000001e-06 2024-05-08T08:54:45.213 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.213 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.213 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:45.213 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858083+0000", 2024-05-08T08:54:45.213 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8880000000000002e-06 2024-05-08T08:54:45.213 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.213 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.213 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:45.213 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907671+0000", 2024-05-08T08:54:45.213 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049587191000000003 2024-05-08T08:54:45.213 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.214 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.214 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:45.214 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907763+0000", 2024-05-08T08:54:45.214 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2231000000000005e-05 2024-05-08T08:54:45.214 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.214 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.214 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:45.214 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907867+0000", 2024-05-08T08:54:45.214 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104644 2024-05-08T08:54:45.214 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.214 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:45.214 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.214 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.214 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.214 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22294 3.1c 3:3aad0389:::rbd_data.11406886b07b.0000000000000080:head [stat,delete] snapc 7=[7] RETRY=6 ondisk+retry+write+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:45.215 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:54:45.215 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 137.28474695899999, 2024-05-08T08:54:45.215 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051152191, 2024-05-08T08:54:45.215 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:45.215 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:45.215 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:45.215 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:45.215 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:45.215 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:45.215 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22294 2024-05-08T08:54:45.215 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.215 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:45.215 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.215 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:45.215 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:54:45.216 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:45.216 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.216 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.216 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:45.216 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:54:45.216 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:45.216 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.216 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.216 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:45.216 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858662+0000", 2024-05-08T08:54:45.216 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4799999999999998e-07 2024-05-08T08:54:45.216 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.216 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.216 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:45.216 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858664+0000", 2024-05-08T08:54:45.217 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1229999999999998e-06 2024-05-08T08:54:45.217 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.217 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.217 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:45.217 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858670+0000", 2024-05-08T08:54:45.217 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3099999999999997e-06 2024-05-08T08:54:45.217 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.217 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.217 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:45.217 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858676+0000", 2024-05-08T08:54:45.217 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9750000000000004e-06 2024-05-08T08:54:45.217 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.217 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.217 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:45.217 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907901+0000", 2024-05-08T08:54:45.217 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049224520000000001 2024-05-08T08:54:45.218 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.218 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.218 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:45.218 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.908085+0000", 2024-05-08T08:54:45.218 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00018377500000000001 2024-05-08T08:54:45.218 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.218 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.218 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:45.218 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.909814+0000", 2024-05-08T08:54:45.218 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00172894 2024-05-08T08:54:45.218 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.218 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:45.218 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.218 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.218 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.219 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4416.0:13461 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-05-08T08:54:45.219 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:22.487456+0000", 2024-05-08T08:54:45.219 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 82.655951918, 2024-05-08T08:54:45.219 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.99102620799999996, 2024-05-08T08:54:45.219 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:45.219 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:45.219 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:45.219 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:45.219 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4416", 2024-05-08T08:54:45.219 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:54:45.219 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 13461 2024-05-08T08:54:45.219 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.219 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:45.219 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.219 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:45.219 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487456+0000", 2024-05-08T08:54:45.220 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:45.220 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.220 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.220 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:45.220 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487456+0000", 2024-05-08T08:54:45.220 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:45.220 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.220 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.220 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:45.220 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487458+0000", 2024-05-08T08:54:45.220 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5659999999999999e-06 2024-05-08T08:54:45.220 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.220 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.220 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:45.220 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487463+0000", 2024-05-08T08:54:45.221 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6890000000000003e-06 2024-05-08T08:54:45.221 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.221 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.221 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:45.221 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487472+0000", 2024-05-08T08:54:45.221 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7710000000000001e-06 2024-05-08T08:54:45.221 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.221 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.221 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:45.221 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487478+0000", 2024-05-08T08:54:45.221 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1390000000000003e-06 2024-05-08T08:54:45.221 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.221 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.221 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:45.221 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487520+0000", 2024-05-08T08:54:45.222 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2259999999999999e-05 2024-05-08T08:54:45.222 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.222 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.222 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-05-08T08:54:45.222 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487528+0000", 2024-05-08T08:54:45.222 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7479999999999993e-06 2024-05-08T08:54:45.222 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.222 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.222 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:45.222 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:23.478139+0000", 2024-05-08T08:54:45.222 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.99061165200000001 2024-05-08T08:54:45.222 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.222 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.222 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:45.222 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:23.478301+0000", 2024-05-08T08:54:45.223 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000161602 2024-05-08T08:54:45.223 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.223 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.223 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:45.223 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:23.478483+0000", 2024-05-08T08:54:45.223 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000181781 2024-05-08T08:54:45.223 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.223 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:45.223 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.223 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.223 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.223 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5752.0:404 3.1a 3:5f5e8104:::rbd_data.1140bc4a55c7.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 5175296~45056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-05-08T08:54:45.223 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:00.390863+0000", 2024-05-08T08:54:45.223 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 44.752545722999997, 2024-05-08T08:54:45.223 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.055189174000000001, 2024-05-08T08:54:45.224 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:45.224 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:45.224 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:54:45.224 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:45.224 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5752", 2024-05-08T08:54:45.224 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/3534419815", 2024-05-08T08:54:45.224 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 404 2024-05-08T08:54:45.224 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.224 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:45.224 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.224 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:45.224 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.390863+0000", 2024-05-08T08:54:45.224 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:45.224 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.224 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.225 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:45.225 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.390863+0000", 2024-05-08T08:54:45.225 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:45.225 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.225 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.225 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:45.225 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.390864+0000", 2024-05-08T08:54:45.225 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.471e-06 2024-05-08T08:54:45.225 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.225 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.225 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:45.225 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397475+0000", 2024-05-08T08:54:45.225 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0066109109999999997 2024-05-08T08:54:45.225 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.226 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.226 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:45.226 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397498+0000", 2024-05-08T08:54:45.226 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2674e-05 2024-05-08T08:54:45.226 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.226 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.226 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:45.226 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397508+0000", 2024-05-08T08:54:45.226 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9019999999999994e-06 2024-05-08T08:54:45.226 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.226 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.226 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:45.226 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397609+0000", 2024-05-08T08:54:45.226 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010121899999999999 2024-05-08T08:54:45.226 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.227 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.227 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:45.227 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397711+0000", 2024-05-08T08:54:45.227 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102489 2024-05-08T08:54:45.227 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.227 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.227 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-05-08T08:54:45.227 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397999+0000", 2024-05-08T08:54:45.227 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00028733799999999999 2024-05-08T08:54:45.227 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.227 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.227 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:54:45.227 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.432093+0000", 2024-05-08T08:54:45.227 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.034094865000000002 2024-05-08T08:54:45.227 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.228 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.228 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:54:45.228 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.445975+0000", 2024-05-08T08:54:45.228 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013881187 2024-05-08T08:54:45.228 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.228 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.228 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:54:45.228 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.446013+0000", 2024-05-08T08:54:45.228 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8155000000000003e-05 2024-05-08T08:54:45.228 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.228 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.228 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:45.228 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.446052+0000", 2024-05-08T08:54:45.228 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8962999999999999e-05 2024-05-08T08:54:45.228 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.229 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:45.229 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.229 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.229 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:45.229 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:45.230 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:54:45.231 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:45.233+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:45.331 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:45.334+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:45.345 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:45.346 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:54:45.346 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:54:45.346 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:45.355 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:54:45.357 INFO:teuthology.orchestra.run.smithi059.stderr:nodeep-scrub is unset 2024-05-08T08:54:45.380 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-05-08T08:54:45.380 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T08:54:45.380 INFO:tasks.thrashosds.thrasher:inject_pause on osd.1 2024-05-08T08:54:45.380 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-05-08T08:54:45.380 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-05-08T08:54:45.381 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok config set filestore_inject_stall 3 2024-05-08T08:54:45.460 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:45.463+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:45.474 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:45.474 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:54:45.474 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:54:45.474 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:54:45.474 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:45.483 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:54:45.532 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:45.535+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:45.535 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:45.535 INFO:teuthology.orchestra.run.smithi059.stdout: "success": "" 2024-05-08T08:54:45.535 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:45.604 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:45.605 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:54:45.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:54:45.605 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:54:45.605 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.605 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6690 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1093632~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:45.605 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:54:45.605 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 133.89130875800001, 2024-05-08T08:54:45.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052462754, 2024-05-08T08:54:45.605 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:45.605 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:45.605 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:45.605 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:45.605 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:45.605 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:45.605 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6690 2024-05-08T08:54:45.606 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.606 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:45.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.606 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:45.606 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:54:45.606 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:45.606 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.606 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:45.606 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:54:45.606 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:45.606 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.606 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:45.607 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715346+0000", 2024-05-08T08:54:45.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.84e-07 2024-05-08T08:54:45.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.607 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:45.607 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715348+0000", 2024-05-08T08:54:45.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.114e-06 2024-05-08T08:54:45.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.607 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:45.607 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715353+0000", 2024-05-08T08:54:45.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5720000000000002e-06 2024-05-08T08:54:45.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.607 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:45.607 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715358+0000", 2024-05-08T08:54:45.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7140000000000001e-06 2024-05-08T08:54:45.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.608 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:45.608 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767638+0000", 2024-05-08T08:54:45.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052279562000000002 2024-05-08T08:54:45.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.608 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:45.608 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767713+0000", 2024-05-08T08:54:45.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5652e-05 2024-05-08T08:54:45.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.608 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:45.608 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767808+0000", 2024-05-08T08:54:45.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4755999999999999e-05 2024-05-08T08:54:45.609 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.609 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:45.609 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.609 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6692 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1110016~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:45.609 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:54:45.609 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 133.89128966800001, 2024-05-08T08:54:45.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052666495000000001, 2024-05-08T08:54:45.609 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:45.609 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:45.609 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:45.609 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:45.609 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:45.609 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:45.609 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6692 2024-05-08T08:54:45.610 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.610 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:45.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.610 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:45.610 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:54:45.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:45.610 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.610 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:45.610 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:54:45.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:45.610 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.610 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:45.610 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:54:45.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8599999999999999e-07 2024-05-08T08:54:45.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.611 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:45.611 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715372+0000", 2024-05-08T08:54:45.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0990000000000001e-06 2024-05-08T08:54:45.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.611 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:45.611 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715378+0000", 2024-05-08T08:54:45.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0680000000000002e-06 2024-05-08T08:54:45.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.611 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:45.611 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715383+0000", 2024-05-08T08:54:45.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8740000000000003e-06 2024-05-08T08:54:45.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.612 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:45.612 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767848+0000", 2024-05-08T08:54:45.612 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052464668999999998 2024-05-08T08:54:45.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.612 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:45.612 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767927+0000", 2024-05-08T08:54:45.612 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9228999999999996e-05 2024-05-08T08:54:45.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.612 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:45.612 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768031+0000", 2024-05-08T08:54:45.612 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010417 2024-05-08T08:54:45.612 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.612 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:45.612 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.613 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.613 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6694 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1126400~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:45.613 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:54:45.613 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 133.89126477100001, 2024-05-08T08:54:45.613 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052848829, 2024-05-08T08:54:45.613 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:45.613 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:45.613 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:45.613 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:45.613 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:45.613 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:45.613 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6694 2024-05-08T08:54:45.614 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.614 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:45.614 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.614 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:45.614 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:54:45.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:45.614 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.614 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.614 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:45.614 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:54:45.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:45.614 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.614 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.614 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:45.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715390+0000", 2024-05-08T08:54:45.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9900000000000001e-07 2024-05-08T08:54:45.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:45.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715392+0000", 2024-05-08T08:54:45.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8279999999999999e-06 2024-05-08T08:54:45.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:45.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715398+0000", 2024-05-08T08:54:45.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7619999999999998e-06 2024-05-08T08:54:45.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:45.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715403+0000", 2024-05-08T08:54:45.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7790000000000003e-06 2024-05-08T08:54:45.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:45.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768067+0000", 2024-05-08T08:54:45.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052663785999999997 2024-05-08T08:54:45.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:45.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768142+0000", 2024-05-08T08:54:45.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5018999999999994e-05 2024-05-08T08:54:45.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:45.617 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768238+0000", 2024-05-08T08:54:45.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6255999999999995e-05 2024-05-08T08:54:45.617 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.617 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:45.617 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.617 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.617 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6696 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1142784~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:45.617 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:54:45.617 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 133.891245263, 2024-05-08T08:54:45.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.053051057999999998, 2024-05-08T08:54:45.617 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:45.617 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:45.618 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:45.618 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:45.618 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:45.618 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:45.618 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6696 2024-05-08T08:54:45.618 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.618 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:45.618 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.618 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:45.618 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:54:45.618 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:45.618 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.618 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.618 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:45.618 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:54:45.618 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:45.618 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.619 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.619 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:45.619 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:54:45.619 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-05-08T08:54:45.619 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.619 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.619 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:45.619 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715411+0000", 2024-05-08T08:54:45.619 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9630000000000001e-06 2024-05-08T08:54:45.619 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.619 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.619 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:45.619 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715417+0000", 2024-05-08T08:54:45.619 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7250000000000002e-06 2024-05-08T08:54:45.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.620 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:45.620 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715421+0000", 2024-05-08T08:54:45.620 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4089999999999996e-06 2024-05-08T08:54:45.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.620 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:45.620 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768277+0000", 2024-05-08T08:54:45.620 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052855293999999997 2024-05-08T08:54:45.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.620 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:45.620 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768358+0000", 2024-05-08T08:54:45.620 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1720999999999999e-05 2024-05-08T08:54:45.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:45.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768460+0000", 2024-05-08T08:54:45.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010154600000000001 2024-05-08T08:54:45.621 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.621 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:45.621 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.621 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6698 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1159168~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:45.621 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:54:45.621 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 133.89122639600001, 2024-05-08T08:54:45.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.053245726, 2024-05-08T08:54:45.621 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:45.622 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:45.622 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:45.622 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:45.622 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:45.622 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:45.622 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6698 2024-05-08T08:54:45.622 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.622 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:45.622 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.622 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:45.622 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:54:45.622 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:45.622 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.622 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.622 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:45.622 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:54:45.623 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:45.623 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.623 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.623 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:45.623 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716936+0000", 2024-05-08T08:54:45.623 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001508665 2024-05-08T08:54:45.623 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.623 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.623 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:45.623 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716941+0000", 2024-05-08T08:54:45.623 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1910000000000002e-06 2024-05-08T08:54:45.623 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.623 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.623 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:45.623 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716950+0000", 2024-05-08T08:54:45.623 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1900000000000001e-06 2024-05-08T08:54:45.623 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.624 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:45.624 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716958+0000", 2024-05-08T08:54:45.624 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8019999999999992e-06 2024-05-08T08:54:45.624 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.624 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:45.624 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768499+0000", 2024-05-08T08:54:45.624 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051540926000000001 2024-05-08T08:54:45.624 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.624 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:45.624 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768574+0000", 2024-05-08T08:54:45.624 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4988999999999996e-05 2024-05-08T08:54:45.624 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.624 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:45.625 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768673+0000", 2024-05-08T08:54:45.625 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9963000000000005e-05 2024-05-08T08:54:45.625 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.625 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:45.625 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.625 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.625 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.625 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6706 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1224704~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:45.625 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:54:45.625 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 133.889621089, 2024-05-08T08:54:45.625 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052435598, 2024-05-08T08:54:45.625 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:45.625 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:45.625 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:45.625 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:45.626 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:45.626 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:45.626 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6706 2024-05-08T08:54:45.626 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.626 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:45.626 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.626 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:45.626 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:54:45.626 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:45.626 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.626 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.626 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:45.626 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:54:45.626 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:45.627 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.627 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.627 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:45.627 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:54:45.627 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:54:45.627 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.627 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.627 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:45.627 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717035+0000", 2024-05-08T08:54:45.627 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9290000000000001e-06 2024-05-08T08:54:45.627 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.627 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.627 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:45.627 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717041+0000", 2024-05-08T08:54:45.627 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6720000000000001e-06 2024-05-08T08:54:45.627 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.627 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.628 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:45.628 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717046+0000", 2024-05-08T08:54:45.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5650000000000003e-06 2024-05-08T08:54:45.628 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.628 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:45.628 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769311+0000", 2024-05-08T08:54:45.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052265645999999999 2024-05-08T08:54:45.628 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.628 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:45.628 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769379+0000", 2024-05-08T08:54:45.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7391000000000006e-05 2024-05-08T08:54:45.628 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.628 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:45.628 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769469+0000", 2024-05-08T08:54:45.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0025999999999995e-05 2024-05-08T08:54:45.629 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.629 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:45.629 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.629 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6708 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1241088~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:45.629 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:54:45.629 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 133.88960298500001, 2024-05-08T08:54:45.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052640552, 2024-05-08T08:54:45.629 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:45.629 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:45.629 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:45.629 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:45.629 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:45.629 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:45.629 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6708 2024-05-08T08:54:45.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.630 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:45.630 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.630 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:45.630 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:54:45.630 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:45.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.630 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.630 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:45.630 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:54:45.630 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:45.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.630 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.630 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:45.630 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717052+0000", 2024-05-08T08:54:45.630 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:54:45.631 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.631 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.631 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:45.631 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717053+0000", 2024-05-08T08:54:45.631 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8330000000000001e-06 2024-05-08T08:54:45.631 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.631 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.631 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:45.631 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717058+0000", 2024-05-08T08:54:45.631 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.933e-06 2024-05-08T08:54:45.631 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.631 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.631 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:45.631 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717063+0000", 2024-05-08T08:54:45.631 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2180000000000001e-06 2024-05-08T08:54:45.631 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.631 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.632 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:45.632 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769507+0000", 2024-05-08T08:54:45.632 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052444866999999999 2024-05-08T08:54:45.632 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.632 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.632 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:45.632 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769589+0000", 2024-05-08T08:54:45.632 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2039000000000005e-05 2024-05-08T08:54:45.632 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.632 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.632 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:45.632 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769692+0000", 2024-05-08T08:54:45.632 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102252 2024-05-08T08:54:45.632 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.632 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:45.632 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.632 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.632 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.633 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6710 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1257472~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:45.633 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:54:45.633 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 133.889584961, 2024-05-08T08:54:45.633 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.05283057, 2024-05-08T08:54:45.633 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:45.633 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:45.633 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:45.633 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:45.633 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:45.633 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:45.633 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6710 2024-05-08T08:54:45.633 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.633 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:45.633 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.634 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:45.634 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:54:45.634 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:45.634 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.634 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:45.634 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:54:45.634 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:45.634 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.634 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:45.634 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718951+0000", 2024-05-08T08:54:45.634 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0018817809999999999 2024-05-08T08:54:45.634 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.635 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:45.635 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718956+0000", 2024-05-08T08:54:45.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5399999999999997e-06 2024-05-08T08:54:45.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.635 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.635 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:45.635 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718964+0000", 2024-05-08T08:54:45.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6629999999999999e-06 2024-05-08T08:54:45.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.635 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.635 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:45.635 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718972+0000", 2024-05-08T08:54:45.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5229999999999998e-06 2024-05-08T08:54:45.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.636 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:45.636 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769729+0000", 2024-05-08T08:54:45.636 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050756959999999997 2024-05-08T08:54:45.636 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.636 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:45.636 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769803+0000", 2024-05-08T08:54:45.636 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4538999999999999e-05 2024-05-08T08:54:45.636 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.636 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:45.636 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769900+0000", 2024-05-08T08:54:45.636 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6564000000000006e-05 2024-05-08T08:54:45.636 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.636 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:45.636 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.637 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.637 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.637 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6726 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1388544~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:45.637 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:54:45.637 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 133.88751018900001, 2024-05-08T08:54:45.637 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052451506000000002, 2024-05-08T08:54:45.637 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:45.637 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:45.637 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:45.637 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:45.637 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:45.637 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:45.637 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6726 2024-05-08T08:54:45.637 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.637 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:45.637 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.637 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:45.638 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:54:45.638 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:45.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.638 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.638 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:45.638 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:54:45.638 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:45.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.638 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.638 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:45.638 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719200+0000", 2024-05-08T08:54:45.638 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5865999999999997e-05 2024-05-08T08:54:45.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.638 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.638 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:45.638 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719204+0000", 2024-05-08T08:54:45.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8630000000000001e-06 2024-05-08T08:54:45.639 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.639 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.639 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:45.639 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719215+0000", 2024-05-08T08:54:45.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1590000000000001e-05 2024-05-08T08:54:45.639 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.639 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.639 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:45.639 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719220+0000", 2024-05-08T08:54:45.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1619999999999999e-06 2024-05-08T08:54:45.639 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.639 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.639 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:45.639 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771425+0000", 2024-05-08T08:54:45.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052204249000000001 2024-05-08T08:54:45.639 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.639 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.640 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:45.640 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771498+0000", 2024-05-08T08:54:45.640 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3782999999999995e-05 2024-05-08T08:54:45.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.640 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.640 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:45.640 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771595+0000", 2024-05-08T08:54:45.640 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6992999999999998e-05 2024-05-08T08:54:45.640 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.640 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:45.640 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.640 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.640 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6728 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1404928~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:45.641 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:54:45.641 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 133.88742278399999, 2024-05-08T08:54:45.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052590747, 2024-05-08T08:54:45.641 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:45.641 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:45.641 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:45.641 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:45.641 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:45.641 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:45.641 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6728 2024-05-08T08:54:45.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.641 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:45.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.641 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:45.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:54:45.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:45.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:45.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:54:45.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:45.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:45.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719232+0000", 2024-05-08T08:54:45.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4000000000000001e-07 2024-05-08T08:54:45.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:45.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719234+0000", 2024-05-08T08:54:45.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4499999999999998e-06 2024-05-08T08:54:45.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.643 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:45.643 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719240+0000", 2024-05-08T08:54:45.643 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5720000000000002e-06 2024-05-08T08:54:45.643 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.643 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.643 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:45.643 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719245+0000", 2024-05-08T08:54:45.643 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8690000000000003e-06 2024-05-08T08:54:45.643 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.643 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.643 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:45.643 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771636+0000", 2024-05-08T08:54:45.643 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052390979999999997 2024-05-08T08:54:45.643 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.643 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.643 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:45.644 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771717+0000", 2024-05-08T08:54:45.644 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1291999999999993e-05 2024-05-08T08:54:45.644 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.644 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.644 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:45.644 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771822+0000", 2024-05-08T08:54:45.644 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104944 2024-05-08T08:54:45.644 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.644 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:45.644 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.644 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.644 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.644 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6730 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1421312~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:45.644 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:54:45.644 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 133.88740254199999, 2024-05-08T08:54:45.644 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052770477000000003, 2024-05-08T08:54:45.644 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:45.644 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:45.645 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:45.645 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:45.645 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:45.645 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:45.645 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6730 2024-05-08T08:54:45.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.645 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:45.645 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.645 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:45.645 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:54:45.645 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:45.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.645 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.645 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:45.645 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:54:45.646 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:45.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.646 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:45.646 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:54:45.646 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9000000000000002e-07 2024-05-08T08:54:45.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.646 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:45.646 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719254+0000", 2024-05-08T08:54:45.646 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4099999999999998e-06 2024-05-08T08:54:45.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.646 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:45.647 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719263+0000", 2024-05-08T08:54:45.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4109999999999996e-06 2024-05-08T08:54:45.647 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.647 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.647 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:45.647 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719268+0000", 2024-05-08T08:54:45.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2669999999999997e-06 2024-05-08T08:54:45.647 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.647 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.647 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:45.647 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771846+0000", 2024-05-08T08:54:45.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052577677000000003 2024-05-08T08:54:45.647 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.647 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.647 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:45.647 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771920+0000", 2024-05-08T08:54:45.648 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4460000000000002e-05 2024-05-08T08:54:45.648 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.648 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.648 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:45.648 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772022+0000", 2024-05-08T08:54:45.648 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010186199999999999 2024-05-08T08:54:45.648 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.648 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:45.648 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.648 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.648 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.648 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6732 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1437696~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:45.648 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:54:45.648 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 133.887377735, 2024-05-08T08:54:45.648 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052905575000000003, 2024-05-08T08:54:45.648 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:45.648 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:45.649 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:45.649 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:45.649 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:45.649 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:45.649 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6732 2024-05-08T08:54:45.649 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.649 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:45.649 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.649 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:45.649 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:54:45.649 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:45.649 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.649 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.649 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:45.649 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:54:45.649 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:45.649 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.650 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.650 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:45.650 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721392+0000", 2024-05-08T08:54:45.650 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002115287 2024-05-08T08:54:45.650 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.650 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.650 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:45.650 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721396+0000", 2024-05-08T08:54:45.650 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1679999999999997e-06 2024-05-08T08:54:45.650 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.650 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.650 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:45.650 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721404+0000", 2024-05-08T08:54:45.650 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4409999999999993e-06 2024-05-08T08:54:45.651 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.651 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.651 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:45.651 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721410+0000", 2024-05-08T08:54:45.651 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9150000000000001e-06 2024-05-08T08:54:45.651 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.651 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.651 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:45.651 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772061+0000", 2024-05-08T08:54:45.651 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050651068 2024-05-08T08:54:45.651 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.651 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.651 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:45.651 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772127+0000", 2024-05-08T08:54:45.651 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5424000000000002e-05 2024-05-08T08:54:45.652 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.652 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.652 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:45.652 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772182+0000", 2024-05-08T08:54:45.652 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5272000000000002e-05 2024-05-08T08:54:45.652 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.652 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:45.652 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.652 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.652 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.652 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6749 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1585152~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:45.652 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:54:45.652 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 133.885076965, 2024-05-08T08:54:45.652 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052350264000000001, 2024-05-08T08:54:45.652 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:45.652 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:45.653 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:45.653 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:45.653 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:45.653 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:45.653 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6749 2024-05-08T08:54:45.653 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.653 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:45.653 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.653 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:45.653 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:54:45.653 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:45.653 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.653 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.653 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:45.653 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:54:45.653 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:45.654 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.654 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.654 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:45.654 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721578+0000", 2024-05-08T08:54:45.654 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0399999999999996e-07 2024-05-08T08:54:45.654 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.654 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.654 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:45.654 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721580+0000", 2024-05-08T08:54:45.654 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5450000000000002e-06 2024-05-08T08:54:45.654 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.654 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.654 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:45.654 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721587+0000", 2024-05-08T08:54:45.655 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0190000000000001e-06 2024-05-08T08:54:45.655 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.655 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.655 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:45.655 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721593+0000", 2024-05-08T08:54:45.655 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5659999999999997e-06 2024-05-08T08:54:45.655 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.655 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.655 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:45.655 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773759+0000", 2024-05-08T08:54:45.655 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052166553999999997 2024-05-08T08:54:45.655 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.655 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.655 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:45.655 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773833+0000", 2024-05-08T08:54:45.656 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3913999999999997e-05 2024-05-08T08:54:45.656 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.656 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.656 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:45.656 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773927+0000", 2024-05-08T08:54:45.656 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4061999999999995e-05 2024-05-08T08:54:45.656 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.656 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:45.656 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.656 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.656 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.656 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6751 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1601536~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:45.656 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:54:45.656 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 133.88505368599999, 2024-05-08T08:54:45.656 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052461302000000001, 2024-05-08T08:54:45.657 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:45.657 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:45.657 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:45.657 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:45.657 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:45.657 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:45.657 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6751 2024-05-08T08:54:45.657 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.657 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:45.657 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.657 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:45.657 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:54:45.657 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:45.657 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.657 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.657 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:45.658 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:54:45.658 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:45.658 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.658 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.658 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:45.658 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722939+0000", 2024-05-08T08:54:45.658 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001338601 2024-05-08T08:54:45.658 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.658 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.658 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:45.658 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722944+0000", 2024-05-08T08:54:45.658 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4469999999999999e-06 2024-05-08T08:54:45.658 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.658 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.659 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:45.659 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722955+0000", 2024-05-08T08:54:45.659 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1892999999999999e-05 2024-05-08T08:54:45.659 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.659 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.659 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:45.659 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722963+0000", 2024-05-08T08:54:45.659 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1080000000000004e-06 2024-05-08T08:54:45.659 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.659 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.659 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:45.659 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773964+0000", 2024-05-08T08:54:45.659 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051001022 2024-05-08T08:54:45.659 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.659 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.659 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:45.660 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.774006+0000", 2024-05-08T08:54:45.660 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2729999999999999e-05 2024-05-08T08:54:45.660 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.660 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.660 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:45.660 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.774062+0000", 2024-05-08T08:54:45.660 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5501000000000003e-05 2024-05-08T08:54:45.660 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.660 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:45.660 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.660 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.660 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.660 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6760 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1683456~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:45.660 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:54:45.660 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 133.88354430999999, 2024-05-08T08:54:45.660 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052571312000000002, 2024-05-08T08:54:45.661 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:45.661 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:45.661 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:45.661 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:45.661 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:45.661 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:45.661 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6760 2024-05-08T08:54:45.661 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.661 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:45.661 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.661 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:45.661 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:54:45.661 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:45.661 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.661 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.661 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:45.661 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:54:45.662 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:45.662 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.662 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.662 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:45.662 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:54:45.662 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2100000000000002e-07 2024-05-08T08:54:45.662 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.662 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.662 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:45.662 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723113+0000", 2024-05-08T08:54:45.662 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3149999999999999e-06 2024-05-08T08:54:45.662 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.662 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.662 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:45.662 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723118+0000", 2024-05-08T08:54:45.663 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8860000000000001e-06 2024-05-08T08:54:45.663 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.663 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.663 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:45.663 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723123+0000", 2024-05-08T08:54:45.663 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6770000000000003e-06 2024-05-08T08:54:45.663 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.663 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.663 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:45.663 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775414+0000", 2024-05-08T08:54:45.663 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052290692999999999 2024-05-08T08:54:45.663 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.663 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.663 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:45.663 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775459+0000", 2024-05-08T08:54:45.663 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5639999999999997e-05 2024-05-08T08:54:45.663 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.664 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.664 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:45.664 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775681+0000", 2024-05-08T08:54:45.664 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00022168000000000001 2024-05-08T08:54:45.664 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.664 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:45.664 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.664 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.664 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.664 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6761 3.16 3:6bd6d6da:::rbd_data.1140782590a5.0000000000000015:6 [read 1095680~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:45.664 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:54:45.664 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 133.88352509000001, 2024-05-08T08:54:45.664 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052751275, 2024-05-08T08:54:45.664 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:45.664 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:45.664 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:45.664 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:45.665 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:45.665 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:45.665 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6761 2024-05-08T08:54:45.665 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.665 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:45.665 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.665 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:45.665 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:54:45.665 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:45.665 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.665 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.665 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:45.665 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:54:45.665 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:45.665 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.665 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.665 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:45.666 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724939+0000", 2024-05-08T08:54:45.666 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001809817 2024-05-08T08:54:45.666 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.666 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.666 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:45.666 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724943+0000", 2024-05-08T08:54:45.666 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4920000000000004e-06 2024-05-08T08:54:45.666 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.666 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.666 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:45.666 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724956+0000", 2024-05-08T08:54:45.666 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2427e-05 2024-05-08T08:54:45.666 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.666 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.667 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:45.667 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724963+0000", 2024-05-08T08:54:45.667 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2559999999999996e-06 2024-05-08T08:54:45.667 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.667 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.667 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:45.667 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775711+0000", 2024-05-08T08:54:45.667 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050747621 2024-05-08T08:54:45.667 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.667 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.667 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:45.667 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775785+0000", 2024-05-08T08:54:45.667 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4634000000000004e-05 2024-05-08T08:54:45.667 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.667 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.667 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:45.667 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775880+0000", 2024-05-08T08:54:45.668 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5027999999999999e-05 2024-05-08T08:54:45.668 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.668 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:45.668 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.668 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.668 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.668 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6770 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1765376~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:45.668 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:54:45.668 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 133.881493995, 2024-05-08T08:54:45.668 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052339296, 2024-05-08T08:54:45.668 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:45.668 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:45.668 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:45.668 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:45.668 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:45.668 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:45.668 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6770 2024-05-08T08:54:45.668 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.669 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:45.669 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.669 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:45.669 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:54:45.669 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:45.669 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.669 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.669 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:45.669 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:54:45.669 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:45.669 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.669 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.669 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:45.669 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725161+0000", 2024-05-08T08:54:45.670 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-05-08T08:54:45.670 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.670 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.670 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:45.670 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725163+0000", 2024-05-08T08:54:45.670 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4370000000000001e-06 2024-05-08T08:54:45.670 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.670 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.670 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:45.670 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725169+0000", 2024-05-08T08:54:45.670 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8409999999999996e-06 2024-05-08T08:54:45.670 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.670 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.671 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:45.671 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725174+0000", 2024-05-08T08:54:45.671 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7249999999999997e-06 2024-05-08T08:54:45.671 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.671 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.671 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:45.671 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777343+0000", 2024-05-08T08:54:45.671 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052169633999999999 2024-05-08T08:54:45.671 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.671 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.671 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:45.671 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777409+0000", 2024-05-08T08:54:45.671 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6033999999999998e-05 2024-05-08T08:54:45.671 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.671 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.672 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:45.672 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777499+0000", 2024-05-08T08:54:45.672 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0224999999999998e-05 2024-05-08T08:54:45.672 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.672 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:45.672 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.672 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.672 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.672 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6771 3.16 3:6bd6d6da:::rbd_data.1140782590a5.0000000000000015:6 [read 1177600~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:45.672 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:54:45.672 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 133.88147417799999, 2024-05-08T08:54:45.672 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052438604999999999, 2024-05-08T08:54:45.672 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:45.672 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:45.672 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:45.672 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:45.672 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:45.673 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:45.673 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6771 2024-05-08T08:54:45.673 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.673 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:45.673 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.673 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:45.673 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:54:45.673 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:45.673 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.673 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.673 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:45.673 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:54:45.673 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:45.673 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.674 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.674 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:45.674 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:54:45.674 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:54:45.674 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.674 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.674 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:45.674 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725188+0000", 2024-05-08T08:54:45.674 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1579999999999999e-06 2024-05-08T08:54:45.674 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.674 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.674 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:45.674 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725195+0000", 2024-05-08T08:54:45.674 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8909999999999995e-06 2024-05-08T08:54:45.674 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.675 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.675 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:45.675 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725200+0000", 2024-05-08T08:54:45.675 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9810000000000003e-06 2024-05-08T08:54:45.675 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.675 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.675 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:45.675 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777527+0000", 2024-05-08T08:54:45.675 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052326582000000003 2024-05-08T08:54:45.675 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.675 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.675 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:45.675 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777566+0000", 2024-05-08T08:54:45.675 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9447999999999999e-05 2024-05-08T08:54:45.675 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.675 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.676 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:45.676 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777619+0000", 2024-05-08T08:54:45.676 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2135000000000001e-05 2024-05-08T08:54:45.676 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.676 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:45.676 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.676 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.676 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.676 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6772 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1781760~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:45.676 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:54:45.676 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 133.88144612100001, 2024-05-08T08:54:45.676 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052601799999999997, 2024-05-08T08:54:45.676 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:45.676 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:45.676 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:45.676 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:45.677 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:45.677 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:45.677 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6772 2024-05-08T08:54:45.677 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.677 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:45.677 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.677 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:45.677 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:54:45.677 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:45.677 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.677 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.677 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:45.677 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:54:45.677 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:45.677 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.677 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.678 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:45.678 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726939+0000", 2024-05-08T08:54:45.678 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001730493 2024-05-08T08:54:45.678 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.678 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.678 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:45.678 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726943+0000", 2024-05-08T08:54:45.678 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7330000000000003e-06 2024-05-08T08:54:45.678 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.678 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.678 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:45.678 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726954+0000", 2024-05-08T08:54:45.678 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1107000000000001e-05 2024-05-08T08:54:45.678 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.679 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.679 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:45.679 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726962+0000", 2024-05-08T08:54:45.679 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.503e-06 2024-05-08T08:54:45.679 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.679 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.679 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:45.679 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777653+0000", 2024-05-08T08:54:45.679 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050691542999999999 2024-05-08T08:54:45.679 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.679 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.679 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:45.679 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777719+0000", 2024-05-08T08:54:45.679 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5228000000000006e-05 2024-05-08T08:54:45.680 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.680 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.680 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:45.680 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777810+0000", 2024-05-08T08:54:45.680 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1193000000000006e-05 2024-05-08T08:54:45.680 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.680 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:45.680 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.680 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.680 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.680 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6782 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1880064~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:45.680 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:54:45.680 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 133.879477614, 2024-05-08T08:54:45.680 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052326492000000002, 2024-05-08T08:54:45.680 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:45.681 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:45.681 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:45.681 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:45.681 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:45.681 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:45.681 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6782 2024-05-08T08:54:45.681 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.681 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:45.681 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.681 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:45.681 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:54:45.681 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:45.681 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.681 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.682 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:45.682 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:54:45.682 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:45.682 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.682 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.682 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:45.682 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728939+0000", 2024-05-08T08:54:45.682 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001762719 2024-05-08T08:54:45.682 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.682 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.682 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:45.682 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728944+0000", 2024-05-08T08:54:45.682 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.746e-06 2024-05-08T08:54:45.682 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.682 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.683 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:45.683 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728955+0000", 2024-05-08T08:54:45.683 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1043000000000001e-05 2024-05-08T08:54:45.683 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.683 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.683 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:45.683 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728963+0000", 2024-05-08T08:54:45.683 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5809999999999999e-06 2024-05-08T08:54:45.683 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.683 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.683 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:45.683 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779320+0000", 2024-05-08T08:54:45.683 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050357290999999998 2024-05-08T08:54:45.683 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.683 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.684 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:45.684 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779401+0000", 2024-05-08T08:54:45.684 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0574000000000005e-05 2024-05-08T08:54:45.684 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:45.684 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:45.684 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:45.684 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779503+0000", 2024-05-08T08:54:45.684 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102538 2024-05-08T08:54:45.684 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.684 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:45.684 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.684 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:45.684 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:45.684 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:45.686 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:54:45.687 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:45.689+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:45.787 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:45.789+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:45.797 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:45.797 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:54:45.797 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:54:45.797 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:45.806 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:54:45.886 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T08:54:45.887 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:45.890+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:45.916 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:45.916 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:54:45.916 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:54:45.916 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:54:45.916 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:45.925 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:54:46.010 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:46.013+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:46.015 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:54:46.015 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:54:46.033 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:54:46.033 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:54:46.051 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:54:46.051 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:54:46.070 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:54:46.070 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:54:46.084 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:46.084 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:46.090 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:46.092+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:46.090 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:54:46.090 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:54:46.094 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:54:46.110 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:54:46.110 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:54:46.190 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:46.193+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:46.228 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:46.228 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:46.238 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:54:46.291 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:46.293+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:46.391 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:46.394+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:46.400 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:46.401 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:46.410 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:54:46.492 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:46.495+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:46.561 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:46.561 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:46.570 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:54:46.613 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:46.616+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:46.683 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:46.683 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:46.692 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:54:46.695 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:46.697+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:46.796 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:46.799+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:46.852 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:46.852 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:46.863 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:54:46.897 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:46.900+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:46.998 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:47.001+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:47.021 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:47.022 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:47.031 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:54:47.099 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:47.101+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:47.145 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:47.145 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:47.154 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:54:47.238 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:47.241+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:47.299 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:47.299 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:47.301 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:47.304+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:47.308 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:54:47.376 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-05-08T08:54:47.421 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:47.424+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:47.471 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:47.471 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:47.481 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:54:47.525 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:47.528+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:47.587 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:47.587 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:47.596 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:54:47.636 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:47.639+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:47.705 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:47.708+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:47.740 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:47.740 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:47.749 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:54:47.806 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:47.809+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:47.894 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:47.894 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:47.903 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:54:47.906 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:47.910+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:48.007 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:48.010+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:48.008 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:48.008 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:48.017 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:54:48.108 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:48.110+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:48.121 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:48.121 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:48.123 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T08:54:48.130 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:54:48.208 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:48.211+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:48.236 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:48.237 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:48.246 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:54:48.253 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:54:48.253 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:54:48.271 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:54:48.271 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:54:48.289 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:54:48.289 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:54:48.306 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:54:48.307 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:54:48.309 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:48.311+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:48.325 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:54:48.325 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:54:48.343 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:54:48.343 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:54:48.362 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:48.362 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:54:48.363 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:54:48.363 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:48.365 INFO:teuthology.orchestra.run.smithi059.stderr:noscrub is set 2024-05-08T08:54:48.377 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:54:48.424 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:48.427+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:48.510 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:48.513+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:48.531 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:48.531 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:54:48.531 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:54:48.531 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:54:48.532 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:48.540 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:54:48.611 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:48.614+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:48.693 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:48.693 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:54:48.693 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:54:48.693 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:54:48.693 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.693 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4621.0:9089 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 23339520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:54:48.693 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:54:48.693 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 136.20140677200001, 2024-05-08T08:54:48.693 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.13814907200000001, 2024-05-08T08:54:48.693 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:48.693 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:48.693 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:54:48.694 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:48.694 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4621", 2024-05-08T08:54:48.694 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:48.694 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9089 2024-05-08T08:54:48.694 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.694 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:48.694 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.694 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:48.694 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:54:48.694 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:48.694 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.694 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.694 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:48.694 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:54:48.695 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:48.695 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.695 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.695 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:48.695 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493005+0000", 2024-05-08T08:54:48.695 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1700000000000001e-06 2024-05-08T08:54:48.695 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.695 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.695 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:48.695 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.513963+0000", 2024-05-08T08:54:48.695 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.020957581999999999 2024-05-08T08:54:48.695 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.695 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.695 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:48.696 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.513995+0000", 2024-05-08T08:54:48.696 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1993999999999999e-05 2024-05-08T08:54:48.696 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.696 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.696 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:48.696 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514007+0000", 2024-05-08T08:54:48.696 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1581e-05 2024-05-08T08:54:48.696 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.696 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.696 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:48.696 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514122+0000", 2024-05-08T08:54:48.696 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000115282 2024-05-08T08:54:48.696 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.696 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.696 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op must wait for map", 2024-05-08T08:54:48.697 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514128+0000", 2024-05-08T08:54:48.697 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9950000000000002e-06 2024-05-08T08:54:48.697 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.697 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.697 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:48.697 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.515800+0000", 2024-05-08T08:54:48.697 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001671886 2024-05-08T08:54:48.697 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.697 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.697 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:48.697 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.515893+0000", 2024-05-08T08:54:48.697 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3490000000000001e-05 2024-05-08T08:54:48.697 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.697 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.697 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 3", 2024-05-08T08:54:48.698 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.517647+0000", 2024-05-08T08:54:48.698 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0017535630000000001 2024-05-08T08:54:48.698 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.698 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.698 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:54:48.698 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.627578+0000", 2024-05-08T08:54:48.698 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.109931557 2024-05-08T08:54:48.698 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.698 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.698 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:54:48.698 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631055+0000", 2024-05-08T08:54:48.698 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0034761420000000002 2024-05-08T08:54:48.698 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.698 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.699 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:54:48.699 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631096+0000", 2024-05-08T08:54:48.699 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1059000000000001e-05 2024-05-08T08:54:48.699 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.699 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.699 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:48.699 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631151+0000", 2024-05-08T08:54:48.699 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5770999999999998e-05 2024-05-08T08:54:48.699 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:48.699 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:48.699 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:48.699 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.699 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.699 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1071 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11026432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:48.699 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:54:48.700 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 132.20895728100001, 2024-05-08T08:54:48.700 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070580037999999998, 2024-05-08T08:54:48.700 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:48.700 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:48.700 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:48.700 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:48.700 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:48.700 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:48.700 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1071 2024-05-08T08:54:48.700 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.700 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:48.700 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.700 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:48.700 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:54:48.700 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:48.701 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.701 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.701 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:48.701 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:54:48.701 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:48.701 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.701 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.701 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:48.701 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:54:48.701 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2800000000000002e-07 2024-05-08T08:54:48.701 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.701 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.701 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:48.701 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485454+0000", 2024-05-08T08:54:48.701 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2290000000000002e-06 2024-05-08T08:54:48.702 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.702 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.702 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:48.702 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485461+0000", 2024-05-08T08:54:48.702 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9920000000000001e-06 2024-05-08T08:54:48.702 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.702 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.702 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:48.702 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485467+0000", 2024-05-08T08:54:48.702 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3979999999999998e-06 2024-05-08T08:54:48.702 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.702 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.702 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:48.703 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555880+0000", 2024-05-08T08:54:48.703 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070413165 2024-05-08T08:54:48.703 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.703 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.703 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:48.703 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555926+0000", 2024-05-08T08:54:48.703 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6217000000000002e-05 2024-05-08T08:54:48.703 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.703 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.703 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:48.703 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556032+0000", 2024-05-08T08:54:48.703 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000105609 2024-05-08T08:54:48.703 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:48.703 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:48.703 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:48.703 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.704 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.705 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1073 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11067392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:48.706 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:54:48.706 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 132.20893606000001, 2024-05-08T08:54:48.706 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070821404000000004, 2024-05-08T08:54:48.706 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:48.706 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:48.706 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:48.706 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:48.706 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:48.706 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:48.706 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1073 2024-05-08T08:54:48.706 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.706 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:48.706 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.706 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:48.707 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:54:48.707 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:48.707 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.707 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.707 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:48.707 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:54:48.707 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:48.707 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.707 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.707 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:48.707 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:54:48.707 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.08e-07 2024-05-08T08:54:48.707 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.707 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.707 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:48.708 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485476+0000", 2024-05-08T08:54:48.708 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3929999999999998e-06 2024-05-08T08:54:48.708 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.708 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.708 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:48.708 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485485+0000", 2024-05-08T08:54:48.708 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1880000000000008e-06 2024-05-08T08:54:48.708 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.708 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.708 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:48.708 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485494+0000", 2024-05-08T08:54:48.708 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3109999999999995e-06 2024-05-08T08:54:48.709 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.709 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.709 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:48.709 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556082+0000", 2024-05-08T08:54:48.709 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070587426999999994 2024-05-08T08:54:48.709 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.709 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.709 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:48.709 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556177+0000", 2024-05-08T08:54:48.709 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5051999999999997e-05 2024-05-08T08:54:48.709 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.709 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.709 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:48.709 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556294+0000", 2024-05-08T08:54:48.709 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000117625 2024-05-08T08:54:48.709 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:48.709 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:48.710 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:48.710 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.710 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.710 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1075 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11108352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:48.710 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:54:48.710 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 132.208905483, 2024-05-08T08:54:48.710 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071045026999999997, 2024-05-08T08:54:48.710 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:48.710 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:48.710 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:48.710 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:48.710 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:48.710 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:48.710 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1075 2024-05-08T08:54:48.710 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.710 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:48.711 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.711 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:48.711 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:54:48.711 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:48.711 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.711 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.711 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:48.711 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:54:48.711 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:48.711 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.711 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.711 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:48.711 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:54:48.711 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.32e-07 2024-05-08T08:54:48.711 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.711 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.712 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:48.712 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485518+0000", 2024-05-08T08:54:48.712 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3769e-05 2024-05-08T08:54:48.712 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.712 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.712 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:48.712 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485527+0000", 2024-05-08T08:54:48.712 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6779999999999995e-06 2024-05-08T08:54:48.712 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.712 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.712 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:48.712 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485532+0000", 2024-05-08T08:54:48.712 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9830000000000004e-06 2024-05-08T08:54:48.712 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.712 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.712 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:48.712 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556340+0000", 2024-05-08T08:54:48.713 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070807743000000006 2024-05-08T08:54:48.713 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.713 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.713 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:48.713 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556435+0000", 2024-05-08T08:54:48.713 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4732e-05 2024-05-08T08:54:48.713 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.713 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.713 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:48.713 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556549+0000", 2024-05-08T08:54:48.713 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011369 2024-05-08T08:54:48.713 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:48.713 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:48.713 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:48.713 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.713 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.714 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1078 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11149312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:48.714 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:54:48.714 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 132.20886528400001, 2024-05-08T08:54:48.714 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071167276000000002, 2024-05-08T08:54:48.714 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:48.714 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:48.714 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:48.714 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:48.714 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:48.714 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:48.714 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1078 2024-05-08T08:54:48.714 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.714 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:48.714 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.714 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:48.714 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:54:48.714 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:48.715 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.715 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.715 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:48.715 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:54:48.715 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:48.715 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.715 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.715 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:48.715 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:54:48.715 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5600000000000001e-07 2024-05-08T08:54:48.715 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.715 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.715 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:48.715 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485547+0000", 2024-05-08T08:54:48.715 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3590000000000002e-06 2024-05-08T08:54:48.715 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.716 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.716 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:48.716 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485553+0000", 2024-05-08T08:54:48.716 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2770000000000002e-06 2024-05-08T08:54:48.716 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.716 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.716 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:48.716 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485558+0000", 2024-05-08T08:54:48.716 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.95e-06 2024-05-08T08:54:48.716 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.716 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.716 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:48.716 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556582+0000", 2024-05-08T08:54:48.716 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071024444000000006 2024-05-08T08:54:48.716 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.717 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.717 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:48.717 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556637+0000", 2024-05-08T08:54:48.717 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4672000000000001e-05 2024-05-08T08:54:48.717 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.717 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.717 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:48.717 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556711+0000", 2024-05-08T08:54:48.717 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4117999999999997e-05 2024-05-08T08:54:48.717 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:48.717 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:48.717 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:48.717 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.717 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.717 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1080 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11190272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:48.717 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:54:48.717 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 132.208844551, 2024-05-08T08:54:48.718 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071350721000000006, 2024-05-08T08:54:48.718 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:48.718 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:48.718 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:48.718 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:48.718 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:48.718 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:48.718 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1080 2024-05-08T08:54:48.718 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.718 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:48.718 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.718 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:48.718 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:54:48.719 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:48.719 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.719 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.719 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:48.719 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:54:48.719 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:48.719 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.719 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.719 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:48.719 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:54:48.719 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.03e-07 2024-05-08T08:54:48.719 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.719 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.719 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:48.719 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485567+0000", 2024-05-08T08:54:48.720 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1090000000000001e-06 2024-05-08T08:54:48.720 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.720 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.720 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:48.720 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485573+0000", 2024-05-08T08:54:48.720 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3049999999999998e-06 2024-05-08T08:54:48.720 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.720 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.720 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:48.720 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485578+0000", 2024-05-08T08:54:48.720 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0130000000000001e-06 2024-05-08T08:54:48.720 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.720 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.721 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:48.721 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556743+0000", 2024-05-08T08:54:48.721 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071164449000000005 2024-05-08T08:54:48.721 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.721 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.721 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:48.721 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556797+0000", 2024-05-08T08:54:48.721 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4589999999999997e-05 2024-05-08T08:54:48.721 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.721 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.721 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:48.721 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556915+0000", 2024-05-08T08:54:48.721 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000117852 2024-05-08T08:54:48.721 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:48.722 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:48.722 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:48.722 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.722 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.722 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1082 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11231232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:48.722 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:54:48.722 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 132.208824308, 2024-05-08T08:54:48.722 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071591396000000002, 2024-05-08T08:54:48.722 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:48.723 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:48.723 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:48.723 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:48.723 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:48.723 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:48.723 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1082 2024-05-08T08:54:48.723 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.723 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:48.723 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.723 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:48.723 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:54:48.723 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:48.723 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.724 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.724 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:48.724 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:54:48.724 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:48.724 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.724 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.724 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:48.724 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:54:48.724 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1800000000000001e-07 2024-05-08T08:54:48.724 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.724 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.724 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:48.724 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485587+0000", 2024-05-08T08:54:48.725 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2589999999999999e-06 2024-05-08T08:54:48.725 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.725 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.725 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:48.725 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485594+0000", 2024-05-08T08:54:48.725 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0789999999999997e-06 2024-05-08T08:54:48.725 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.725 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.725 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:48.725 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485598+0000", 2024-05-08T08:54:48.725 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7480000000000001e-06 2024-05-08T08:54:48.725 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.725 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.725 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:48.726 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556971+0000", 2024-05-08T08:54:48.726 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071372774999999999 2024-05-08T08:54:48.726 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.726 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.726 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:48.726 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557069+0000", 2024-05-08T08:54:48.726 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.7732999999999994e-05 2024-05-08T08:54:48.726 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.726 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.726 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:48.726 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557176+0000", 2024-05-08T08:54:48.726 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000107384 2024-05-08T08:54:48.726 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:48.726 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:48.726 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:48.727 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.727 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.727 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1084 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11272192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:48.727 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:54:48.727 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 132.20878566900001, 2024-05-08T08:54:48.727 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071808977999999996, 2024-05-08T08:54:48.727 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:48.727 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:48.727 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:48.727 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:48.727 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:48.727 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:48.727 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1084 2024-05-08T08:54:48.727 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.727 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:48.727 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.728 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:48.728 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:54:48.728 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:48.728 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.728 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.728 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:48.728 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:54:48.728 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:48.728 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.728 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.728 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:48.728 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485624+0000", 2024-05-08T08:54:48.729 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:54:48.729 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.729 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.729 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:48.729 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485626+0000", 2024-05-08T08:54:48.729 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9460000000000001e-06 2024-05-08T08:54:48.729 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.729 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.729 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:48.729 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485633+0000", 2024-05-08T08:54:48.729 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1149999999999996e-06 2024-05-08T08:54:48.729 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.729 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.729 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:48.729 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485638+0000", 2024-05-08T08:54:48.730 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5670000000000002e-06 2024-05-08T08:54:48.730 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.730 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.730 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:48.730 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557226+0000", 2024-05-08T08:54:48.730 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071587267999999996 2024-05-08T08:54:48.730 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.730 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.730 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:48.730 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557321+0000", 2024-05-08T08:54:48.730 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5785000000000005e-05 2024-05-08T08:54:48.730 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.730 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.730 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:48.731 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557432+0000", 2024-05-08T08:54:48.731 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011092800000000001 2024-05-08T08:54:48.731 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:48.731 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:48.731 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:48.731 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.731 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.731 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1086 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11313152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:48.731 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:54:48.731 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 132.208762342, 2024-05-08T08:54:48.731 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071933132999999996, 2024-05-08T08:54:48.731 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:48.731 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:48.731 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:48.731 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:48.731 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:48.732 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:48.732 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1086 2024-05-08T08:54:48.732 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.732 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:48.732 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.732 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:48.732 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:54:48.732 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:48.732 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.732 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.732 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:48.732 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:54:48.732 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:48.732 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.732 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.732 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:48.732 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486328+0000", 2024-05-08T08:54:48.733 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00068107900000000002 2024-05-08T08:54:48.733 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.733 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.733 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:48.733 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486331+0000", 2024-05-08T08:54:48.733 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.585e-06 2024-05-08T08:54:48.733 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.733 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.733 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:48.733 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486343+0000", 2024-05-08T08:54:48.733 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1856e-05 2024-05-08T08:54:48.733 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.733 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.733 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:48.733 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486350+0000", 2024-05-08T08:54:48.733 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8600000000000004e-06 2024-05-08T08:54:48.734 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.734 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.734 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:48.734 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557470+0000", 2024-05-08T08:54:48.734 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071120326999999997 2024-05-08T08:54:48.734 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.734 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.734 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:48.734 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557514+0000", 2024-05-08T08:54:48.734 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3909000000000003e-05 2024-05-08T08:54:48.734 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.734 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.734 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:48.734 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557580+0000", 2024-05-08T08:54:48.734 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5517000000000003e-05 2024-05-08T08:54:48.735 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:48.735 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:48.735 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:48.735 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.735 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.735 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1089 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11354112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:48.735 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:54:48.735 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 132.20804946999999, 2024-05-08T08:54:48.735 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071457381, 2024-05-08T08:54:48.735 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:48.735 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:48.735 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:48.735 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:48.735 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:48.735 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:48.736 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1089 2024-05-08T08:54:48.736 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.736 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:48.736 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.736 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:48.736 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:54:48.736 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:48.736 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.736 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.736 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:48.736 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:54:48.736 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:48.736 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.736 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.736 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:48.736 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486851+0000", 2024-05-08T08:54:48.736 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00049102 2024-05-08T08:54:48.737 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.737 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.737 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:48.737 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486854+0000", 2024-05-08T08:54:48.737 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6859999999999999e-06 2024-05-08T08:54:48.737 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.737 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.737 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:48.737 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486864+0000", 2024-05-08T08:54:48.737 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6080000000000002e-06 2024-05-08T08:54:48.737 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.737 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.737 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:48.737 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486870+0000", 2024-05-08T08:54:48.737 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.477e-06 2024-05-08T08:54:48.738 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.738 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.738 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:48.738 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557612+0000", 2024-05-08T08:54:48.738 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070741557999999996 2024-05-08T08:54:48.738 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.738 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.738 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:48.738 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557695+0000", 2024-05-08T08:54:48.738 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3300000000000005e-05 2024-05-08T08:54:48.738 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.738 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.738 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:48.738 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557817+0000", 2024-05-08T08:54:48.738 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012173200000000001 2024-05-08T08:54:48.738 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:48.738 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:48.739 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:48.739 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.739 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.739 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1091 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11395072~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:48.739 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:54:48.739 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 132.20752867499999, 2024-05-08T08:54:48.739 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071184737999999997, 2024-05-08T08:54:48.739 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:48.739 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:48.739 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:48.739 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:48.739 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:48.739 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:48.739 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1091 2024-05-08T08:54:48.739 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.739 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:48.740 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.740 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:48.740 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:54:48.740 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:48.740 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.740 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.740 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:48.740 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:54:48.740 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:48.740 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.740 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.740 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:48.740 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486881+0000", 2024-05-08T08:54:48.741 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0699999999999997e-07 2024-05-08T08:54:48.741 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.741 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.741 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:48.741 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486883+0000", 2024-05-08T08:54:48.741 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.402e-06 2024-05-08T08:54:48.741 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.741 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.741 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:48.741 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486890+0000", 2024-05-08T08:54:48.741 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0480000000000001e-06 2024-05-08T08:54:48.741 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.741 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.741 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:48.741 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486896+0000", 2024-05-08T08:54:48.742 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7640000000000002e-06 2024-05-08T08:54:48.742 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.742 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.742 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:48.742 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557860+0000", 2024-05-08T08:54:48.742 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070963791999999998 2024-05-08T08:54:48.742 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.742 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.742 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:48.742 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557952+0000", 2024-05-08T08:54:48.742 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2082000000000004e-05 2024-05-08T08:54:48.742 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.742 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.742 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:48.742 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558065+0000", 2024-05-08T08:54:48.743 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000113143 2024-05-08T08:54:48.743 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:48.743 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:48.743 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:48.743 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.743 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.743 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1094 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11436032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:48.743 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:54:48.743 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 132.20750550700001, 2024-05-08T08:54:48.743 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071400507000000002, 2024-05-08T08:54:48.743 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:48.743 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:48.743 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:48.743 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:48.744 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:48.744 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:48.744 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1094 2024-05-08T08:54:48.744 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.744 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:48.744 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.744 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:48.744 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:54:48.744 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:48.744 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.744 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.744 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:48.744 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:54:48.744 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:48.745 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.745 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.745 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:48.745 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487940+0000", 2024-05-08T08:54:48.745 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001036868 2024-05-08T08:54:48.745 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.745 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.745 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:48.745 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487944+0000", 2024-05-08T08:54:48.745 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7069999999999998e-06 2024-05-08T08:54:48.745 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.745 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.745 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:48.745 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487951+0000", 2024-05-08T08:54:48.746 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2649999999999999e-06 2024-05-08T08:54:48.746 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.746 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.746 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:48.746 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487961+0000", 2024-05-08T08:54:48.746 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4110000000000002e-06 2024-05-08T08:54:48.746 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.746 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.746 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:48.746 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558117+0000", 2024-05-08T08:54:48.746 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070156662999999994 2024-05-08T08:54:48.746 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.746 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.746 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:48.746 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558214+0000", 2024-05-08T08:54:48.747 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6444e-05 2024-05-08T08:54:48.747 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.747 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.747 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:48.747 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558304+0000", 2024-05-08T08:54:48.747 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0148999999999994e-05 2024-05-08T08:54:48.747 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:48.747 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:48.747 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:48.747 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.747 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.747 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1096 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11476992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:48.747 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:54:48.748 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 132.20643897599999, 2024-05-08T08:54:48.748 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070480074000000004, 2024-05-08T08:54:48.748 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:48.748 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:48.748 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:48.748 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:48.748 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:48.748 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:48.748 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1096 2024-05-08T08:54:48.748 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.748 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:48.748 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.749 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:48.749 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:54:48.749 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:48.749 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.749 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.749 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:48.749 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:54:48.749 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:48.749 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.749 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.749 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:48.749 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:54:48.750 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3700000000000001e-07 2024-05-08T08:54:48.750 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.750 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.750 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:48.750 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487973+0000", 2024-05-08T08:54:48.750 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0870000000000001e-06 2024-05-08T08:54:48.750 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.750 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.750 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:48.750 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487977+0000", 2024-05-08T08:54:48.750 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0600000000000001e-06 2024-05-08T08:54:48.750 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.750 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.750 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:48.750 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487980+0000", 2024-05-08T08:54:48.751 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1089999999999998e-06 2024-05-08T08:54:48.751 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.751 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.751 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:48.751 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558338+0000", 2024-05-08T08:54:48.751 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070358233000000006 2024-05-08T08:54:48.751 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.751 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.751 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:48.751 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558392+0000", 2024-05-08T08:54:48.751 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4537999999999999e-05 2024-05-08T08:54:48.751 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.751 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.751 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:48.751 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558450+0000", 2024-05-08T08:54:48.751 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7710000000000001e-05 2024-05-08T08:54:48.752 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:48.752 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:48.752 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:48.752 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.752 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.752 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1099 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11517952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:48.752 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:54:48.752 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 132.20642509999999, 2024-05-08T08:54:48.752 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070720506000000002, 2024-05-08T08:54:48.752 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:48.752 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:48.752 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:48.752 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:48.752 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:48.752 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:48.753 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1099 2024-05-08T08:54:48.753 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.753 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:48.753 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.753 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:48.753 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:54:48.753 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:48.753 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.753 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.753 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:48.753 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:54:48.753 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:48.753 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.753 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.753 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:48.753 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:54:48.754 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3999999999999998e-07 2024-05-08T08:54:48.754 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.754 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.754 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:48.754 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487986+0000", 2024-05-08T08:54:48.754 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3519999999999999e-06 2024-05-08T08:54:48.754 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.754 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.754 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:48.754 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487989+0000", 2024-05-08T08:54:48.754 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.23e-06 2024-05-08T08:54:48.754 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.754 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.754 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:48.754 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487991+0000", 2024-05-08T08:54:48.754 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6350000000000002e-06 2024-05-08T08:54:48.755 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.755 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.755 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:48.755 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558489+0000", 2024-05-08T08:54:48.755 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070497924000000003 2024-05-08T08:54:48.755 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.755 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.755 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:48.755 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558583+0000", 2024-05-08T08:54:48.755 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3405000000000004e-05 2024-05-08T08:54:48.755 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.755 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.755 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:48.755 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558704+0000", 2024-05-08T08:54:48.755 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012171999999999999 2024-05-08T08:54:48.755 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:48.756 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:48.756 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:48.756 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.756 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.756 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1100 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11558912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:48.756 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:54:48.756 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 132.20641210100001, 2024-05-08T08:54:48.756 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070962266999999996, 2024-05-08T08:54:48.756 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:48.756 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:48.756 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:48.756 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:48.756 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:48.756 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:48.756 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1100 2024-05-08T08:54:48.756 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.757 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:48.757 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.757 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:48.757 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:54:48.757 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:48.757 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.757 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.757 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:48.757 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:54:48.757 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:48.757 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.757 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.757 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:48.757 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488430+0000", 2024-05-08T08:54:48.758 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000432676 2024-05-08T08:54:48.758 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.758 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.758 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:48.758 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488433+0000", 2024-05-08T08:54:48.758 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.45e-06 2024-05-08T08:54:48.758 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.758 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.758 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:48.758 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488442+0000", 2024-05-08T08:54:48.758 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.687e-06 2024-05-08T08:54:48.758 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.758 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.758 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:48.758 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488446+0000", 2024-05-08T08:54:48.759 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4939999999999997e-06 2024-05-08T08:54:48.759 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.759 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.759 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:48.759 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558756+0000", 2024-05-08T08:54:48.759 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070310051999999998 2024-05-08T08:54:48.759 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.759 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.759 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:48.759 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558844+0000", 2024-05-08T08:54:48.759 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7690999999999998e-05 2024-05-08T08:54:48.759 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.759 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.759 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:48.759 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558959+0000", 2024-05-08T08:54:48.760 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000115217 2024-05-08T08:54:48.760 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:48.760 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:48.760 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:48.760 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.760 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.760 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1103 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11599872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:48.760 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:54:48.760 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 132.20595442600001, 2024-05-08T08:54:48.760 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070777100999999995, 2024-05-08T08:54:48.760 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:48.760 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:48.760 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:48.760 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:48.761 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:48.761 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:48.761 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1103 2024-05-08T08:54:48.761 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.761 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:48.761 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.761 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:48.761 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:54:48.761 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:48.761 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.761 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.761 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:48.761 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:54:48.761 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:48.761 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.761 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.762 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:48.762 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488959+0000", 2024-05-08T08:54:48.762 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00050457200000000001 2024-05-08T08:54:48.762 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.762 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.762 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:48.762 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488969+0000", 2024-05-08T08:54:48.762 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.7159999999999998e-06 2024-05-08T08:54:48.762 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.762 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.762 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:48.762 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488981+0000", 2024-05-08T08:54:48.762 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2150000000000001e-05 2024-05-08T08:54:48.762 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.763 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.763 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:48.763 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488988+0000", 2024-05-08T08:54:48.763 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8759999999999999e-06 2024-05-08T08:54:48.763 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.763 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.763 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:48.763 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558992+0000", 2024-05-08T08:54:48.763 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070004358000000003 2024-05-08T08:54:48.763 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.763 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.763 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:48.763 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559093+0000", 2024-05-08T08:54:48.763 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000100299 2024-05-08T08:54:48.764 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.764 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.764 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:48.764 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559232+0000", 2024-05-08T08:54:48.764 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013913 2024-05-08T08:54:48.764 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:48.764 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:48.764 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:48.764 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.764 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.764 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1104 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11640832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:48.764 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:54:48.764 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 132.20540961099999, 2024-05-08T08:54:48.764 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070521947000000001, 2024-05-08T08:54:48.764 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:48.765 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:48.765 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:48.765 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:48.765 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:48.765 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:48.765 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1104 2024-05-08T08:54:48.765 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.765 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:48.765 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.765 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:48.765 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:54:48.765 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:48.765 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.765 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.765 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:48.765 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:54:48.766 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:48.766 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.766 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.766 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:48.766 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489700+0000", 2024-05-08T08:54:48.766 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00070067600000000001 2024-05-08T08:54:48.766 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.766 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.766 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:48.766 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489704+0000", 2024-05-08T08:54:48.766 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1640000000000003e-06 2024-05-08T08:54:48.766 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.766 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.766 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:48.767 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489716+0000", 2024-05-08T08:54:48.767 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1379000000000001e-05 2024-05-08T08:54:48.767 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.767 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.767 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:48.767 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489724+0000", 2024-05-08T08:54:48.767 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8949999999999998e-06 2024-05-08T08:54:48.767 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.767 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.767 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:48.767 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559278+0000", 2024-05-08T08:54:48.767 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.069554151999999994 2024-05-08T08:54:48.767 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.767 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.767 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:48.767 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559387+0000", 2024-05-08T08:54:48.768 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000109258 2024-05-08T08:54:48.768 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.768 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.768 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:48.768 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559521+0000", 2024-05-08T08:54:48.768 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000134423 2024-05-08T08:54:48.768 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:48.768 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:48.768 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:48.768 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.768 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.768 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:8636 3.17 3:e8a0a3e4:::rbd_header.11402b26385:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-05-08T08:54:48.768 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:54.150971+0000", 2024-05-08T08:54:48.768 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 114.543437617, 2024-05-08T08:54:48.768 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.217273233, 2024-05-08T08:54:48.768 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:48.769 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:48.769 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:48.769 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:48.769 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:54:48.769 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:54:48.769 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 8636 2024-05-08T08:54:48.769 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.769 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:48.769 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.769 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:48.769 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150971+0000", 2024-05-08T08:54:48.769 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:48.769 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.769 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.769 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:48.770 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150971+0000", 2024-05-08T08:54:48.770 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:48.770 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.770 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.770 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:48.770 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150972+0000", 2024-05-08T08:54:48.770 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.71e-07 2024-05-08T08:54:48.770 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.770 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.770 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:48.770 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150977+0000", 2024-05-08T08:54:48.770 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.152e-06 2024-05-08T08:54:48.770 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.770 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.770 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:48.771 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150990+0000", 2024-05-08T08:54:48.771 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2364999999999999e-05 2024-05-08T08:54:48.771 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.771 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.771 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:48.771 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150998+0000", 2024-05-08T08:54:48.771 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0739999999999993e-06 2024-05-08T08:54:48.771 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.771 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.771 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:48.771 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.151054+0000", 2024-05-08T08:54:48.771 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5949000000000003e-05 2024-05-08T08:54:48.771 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.771 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.771 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:54:48.772 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.151060+0000", 2024-05-08T08:54:48.772 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6120000000000004e-06 2024-05-08T08:54:48.772 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.772 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.772 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:48.772 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.367640+0000", 2024-05-08T08:54:48.772 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2165792739999999 2024-05-08T08:54:48.772 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.772 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.772 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:48.772 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.367839+0000", 2024-05-08T08:54:48.772 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000199185 2024-05-08T08:54:48.772 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.772 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.772 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:48.772 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.368245+0000", 2024-05-08T08:54:48.773 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00040575099999999998 2024-05-08T08:54:48.773 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:48.773 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:48.773 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:48.773 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.773 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.773 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5302.0:1516 3.c 3:319bf789:::rbd_data.1140d3fc54c.0000000000000025:head [stat,write 1892352~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:54:48.773 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:27.822220+0000", 2024-05-08T08:54:48.773 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 80.872188679999994, 2024-05-08T08:54:48.773 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.092643744, 2024-05-08T08:54:48.773 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:48.773 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:48.773 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:54:48.773 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:48.773 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5302", 2024-05-08T08:54:48.773 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/359277859", 2024-05-08T08:54:48.773 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1516 2024-05-08T08:54:48.774 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.774 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:48.774 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.774 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:48.774 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822220+0000", 2024-05-08T08:54:48.774 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:48.774 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.774 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.774 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:48.774 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822220+0000", 2024-05-08T08:54:48.774 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:48.774 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.774 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.774 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:48.775 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822221+0000", 2024-05-08T08:54:48.775 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3500000000000002e-07 2024-05-08T08:54:48.775 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.775 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.775 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:48.775 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822228+0000", 2024-05-08T08:54:48.775 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9959999999999996e-06 2024-05-08T08:54:48.775 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.775 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.775 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:48.775 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822234+0000", 2024-05-08T08:54:48.775 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5429999999999999e-06 2024-05-08T08:54:48.775 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.775 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.775 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:48.776 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822238+0000", 2024-05-08T08:54:48.776 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.732e-06 2024-05-08T08:54:48.776 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.776 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.776 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:48.776 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822285+0000", 2024-05-08T08:54:48.776 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6810000000000001e-05 2024-05-08T08:54:48.776 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.776 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.776 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:48.776 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822358+0000", 2024-05-08T08:54:48.776 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3473000000000006e-05 2024-05-08T08:54:48.776 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.776 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.776 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 5", 2024-05-08T08:54:48.776 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822489+0000", 2024-05-08T08:54:48.777 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013011199999999999 2024-05-08T08:54:48.777 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.777 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.777 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:54:48.777 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.825332+0000", 2024-05-08T08:54:48.777 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0028434799999999998 2024-05-08T08:54:48.777 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.777 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.777 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:54:48.777 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.914753+0000", 2024-05-08T08:54:48.777 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089421424999999999 2024-05-08T08:54:48.777 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.777 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.777 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:54:48.777 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.914807+0000", 2024-05-08T08:54:48.777 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3829000000000002e-05 2024-05-08T08:54:48.778 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.778 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.778 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:48.778 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.914864+0000", 2024-05-08T08:54:48.778 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6909e-05 2024-05-08T08:54:48.778 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:48.778 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:48.778 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:48.778 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.778 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.778 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:24567 3.6 3:66af1c40:::rbd_data.1140d3fc54c.000000000000005a:18 [read 2080768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:48.778 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.216598+0000", 2024-05-08T08:54:48.778 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 76.477810599999998, 2024-05-08T08:54:48.778 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071602647000000005, 2024-05-08T08:54:48.778 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:48.779 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:48.779 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:48.779 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:48.779 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:48.779 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:48.779 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 24567 2024-05-08T08:54:48.779 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.779 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:48.779 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.779 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:48.779 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216598+0000", 2024-05-08T08:54:48.779 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:48.779 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.779 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.779 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:48.779 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216598+0000", 2024-05-08T08:54:48.779 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:48.780 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.780 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.780 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:48.780 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216599+0000", 2024-05-08T08:54:48.780 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8999999999999997e-07 2024-05-08T08:54:48.780 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.780 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.780 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:48.780 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216601+0000", 2024-05-08T08:54:48.780 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9889999999999999e-06 2024-05-08T08:54:48.780 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.780 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.780 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:48.780 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216607+0000", 2024-05-08T08:54:48.780 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3600000000000001e-06 2024-05-08T08:54:48.780 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.781 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.781 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:48.781 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216612+0000", 2024-05-08T08:54:48.781 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.69e-06 2024-05-08T08:54:48.781 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.781 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.781 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:48.781 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.287076+0000", 2024-05-08T08:54:48.781 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070464355000000006 2024-05-08T08:54:48.781 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.781 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.781 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:48.781 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.287151+0000", 2024-05-08T08:54:48.781 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4924000000000003e-05 2024-05-08T08:54:48.781 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:48.781 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:48.781 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:48.782 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.288201+0000", 2024-05-08T08:54:48.782 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0010498389999999999 2024-05-08T08:54:48.782 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:48.782 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:48.782 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:48.782 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:48.782 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:48.782 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:48.784 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:54:48.784 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:48.786+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:48.884 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:48.886+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:48.894 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:48.894 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:54:48.894 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:54:48.894 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:48.903 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:54:48.987 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:48.990+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:49.053 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:49.053 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:54:49.053 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:54:49.053 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:54:49.053 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:49.063 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:54:49.085 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:49.088+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:49.186 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:49.189+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:49.213 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:49.213 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:54:49.214 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:54:49.214 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:54:49.214 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.214 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:12443 3.0 3:02d0158e:::rbd_data.114071bcf828.0000000000000037:head [stat,delete] snapc 19=[19] RETRY=3 ondisk+retry+write+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:49.214 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.610027+0000", 2024-05-08T08:54:49.214 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 77.60535453, 2024-05-08T08:54:49.214 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.083586333999999998, 2024-05-08T08:54:49.214 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:49.214 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:49.214 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:49.214 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:49.214 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:49.214 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:49.215 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 12443 2024-05-08T08:54:49.215 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.215 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:49.215 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.215 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:49.215 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610027+0000", 2024-05-08T08:54:49.215 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:49.215 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.215 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.215 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:49.215 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610027+0000", 2024-05-08T08:54:49.215 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:49.215 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.215 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.216 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:49.216 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610028+0000", 2024-05-08T08:54:49.216 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.13e-07 2024-05-08T08:54:49.216 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.216 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.216 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:49.216 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610030+0000", 2024-05-08T08:54:49.216 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7669999999999999e-06 2024-05-08T08:54:49.216 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.216 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.216 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:49.216 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610039+0000", 2024-05-08T08:54:49.216 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4959999999999996e-06 2024-05-08T08:54:49.216 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.216 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.217 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:49.217 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610045+0000", 2024-05-08T08:54:49.217 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6779999999999997e-06 2024-05-08T08:54:49.217 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.217 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.217 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:49.217 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.660873+0000", 2024-05-08T08:54:49.217 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.050828748 2024-05-08T08:54:49.217 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.217 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.217 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:49.217 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.661017+0000", 2024-05-08T08:54:49.217 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014391999999999999 2024-05-08T08:54:49.217 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.217 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.218 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:49.218 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.693613+0000", 2024-05-08T08:54:49.218 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.032596111999999997 2024-05-08T08:54:49.218 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.218 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:49.218 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.218 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.218 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.218 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:14301 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [watch reconnect cookie 139837155901536 gen 12] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:49.218 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.645078+0000", 2024-05-08T08:54:49.218 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 77.570303891999998, 2024-05-08T08:54:49.218 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088281852999999993, 2024-05-08T08:54:49.218 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:49.218 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:49.219 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:49.219 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:49.219 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:49.219 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:49.219 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14301 2024-05-08T08:54:49.219 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.219 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:49.219 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.219 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:49.219 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645078+0000", 2024-05-08T08:54:49.219 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:49.219 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.219 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.219 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:49.219 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645078+0000", 2024-05-08T08:54:49.220 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:49.220 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.220 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.220 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:49.220 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645084+0000", 2024-05-08T08:54:49.220 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4620000000000001e-06 2024-05-08T08:54:49.220 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.220 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.220 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:49.220 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645088+0000", 2024-05-08T08:54:49.220 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8039999999999999e-06 2024-05-08T08:54:49.220 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.220 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.220 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:49.220 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645106+0000", 2024-05-08T08:54:49.221 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.8318999999999999e-05 2024-05-08T08:54:49.221 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.221 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.221 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:49.221 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645113+0000", 2024-05-08T08:54:49.221 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0310000000000001e-06 2024-05-08T08:54:49.221 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.221 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.221 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:49.221 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.670493+0000", 2024-05-08T08:54:49.221 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.025379993 2024-05-08T08:54:49.221 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.221 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.221 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:49.221 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.670571+0000", 2024-05-08T08:54:49.222 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7613000000000004e-05 2024-05-08T08:54:49.222 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.222 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.222 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:49.222 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733360+0000", 2024-05-08T08:54:49.222 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.062788630999999998 2024-05-08T08:54:49.222 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.222 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:49.222 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.222 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.222 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.222 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:14420 3.0 3:06a8948e:::rbd_data.114071bcf828.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:49.222 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.649639+0000", 2024-05-08T08:54:49.222 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 77.565742744000005, 2024-05-08T08:54:49.223 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.083851897999999994, 2024-05-08T08:54:49.223 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:49.223 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:49.223 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:54:49.223 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:49.223 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:49.223 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:49.223 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14420 2024-05-08T08:54:49.223 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.223 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:49.223 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.223 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:49.223 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649639+0000", 2024-05-08T08:54:49.223 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:49.223 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.224 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.224 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:49.224 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649639+0000", 2024-05-08T08:54:49.224 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:49.224 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.224 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.224 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:49.224 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649640+0000", 2024-05-08T08:54:49.224 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6899999999999997e-07 2024-05-08T08:54:49.224 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.224 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.224 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:49.224 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649895+0000", 2024-05-08T08:54:49.224 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00025531000000000002 2024-05-08T08:54:49.224 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.224 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.225 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:49.225 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649905+0000", 2024-05-08T08:54:49.225 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0198e-05 2024-05-08T08:54:49.225 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.225 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.225 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:49.225 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649911+0000", 2024-05-08T08:54:49.225 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2310000000000001e-06 2024-05-08T08:54:49.225 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.225 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.225 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:49.225 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.684099+0000", 2024-05-08T08:54:49.225 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.034188017000000001 2024-05-08T08:54:49.225 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.225 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.226 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:49.226 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.684172+0000", 2024-05-08T08:54:49.226 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3201999999999995e-05 2024-05-08T08:54:49.226 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.226 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.226 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 2", 2024-05-08T08:54:49.226 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.684418+0000", 2024-05-08T08:54:49.226 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00024559099999999997 2024-05-08T08:54:49.226 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.226 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.226 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:54:49.226 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.705538+0000", 2024-05-08T08:54:49.226 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.021119707000000001 2024-05-08T08:54:49.226 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.227 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.227 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:54:49.227 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733399+0000", 2024-05-08T08:54:49.227 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.027861543999999999 2024-05-08T08:54:49.227 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.227 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.227 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:54:49.227 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733445+0000", 2024-05-08T08:54:49.227 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5587000000000002e-05 2024-05-08T08:54:49.227 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.227 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.227 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:49.227 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733491+0000", 2024-05-08T08:54:49.227 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5841999999999999e-05 2024-05-08T08:54:49.227 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.228 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:49.228 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.228 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.228 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.228 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28547 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889179216] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:49.228 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.355654+0000", 2024-05-08T08:54:49.228 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 76.859727516000007, 2024-05-08T08:54:49.228 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078692261999999999, 2024-05-08T08:54:49.228 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:49.228 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:49.228 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:49.228 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:49.228 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:49.228 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:49.228 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28547 2024-05-08T08:54:49.229 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.229 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:49.229 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.229 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:49.229 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355654+0000", 2024-05-08T08:54:49.229 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:49.229 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.229 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.229 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:49.229 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355654+0000", 2024-05-08T08:54:49.229 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:49.229 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.229 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.229 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:49.229 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355654+0000", 2024-05-08T08:54:49.230 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3999999999999998e-07 2024-05-08T08:54:49.230 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.230 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.230 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:49.230 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355656+0000", 2024-05-08T08:54:49.230 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.353e-06 2024-05-08T08:54:49.230 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.230 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.230 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:49.230 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355663+0000", 2024-05-08T08:54:49.230 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2259999999999999e-06 2024-05-08T08:54:49.230 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.230 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.230 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:49.231 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355666+0000", 2024-05-08T08:54:49.231 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7930000000000002e-06 2024-05-08T08:54:49.231 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.231 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.231 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:49.231 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434117+0000", 2024-05-08T08:54:49.231 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078451231999999996 2024-05-08T08:54:49.231 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.231 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.231 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:49.231 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434198+0000", 2024-05-08T08:54:49.231 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0879999999999998e-05 2024-05-08T08:54:49.231 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.231 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.231 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:49.232 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434346+0000", 2024-05-08T08:54:49.232 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014853800000000001 2024-05-08T08:54:49.232 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.232 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:49.232 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.232 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.232 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.232 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28548 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889189696] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:49.232 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.355669+0000", 2024-05-08T08:54:49.232 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 76.859712165999994, 2024-05-08T08:54:49.232 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078956606999999998, 2024-05-08T08:54:49.232 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:49.232 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:49.232 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:49.233 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:49.233 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:49.233 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:49.233 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28548 2024-05-08T08:54:49.233 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.233 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:49.233 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.233 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:49.233 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355669+0000", 2024-05-08T08:54:49.233 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:49.233 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.233 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.233 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:49.233 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355669+0000", 2024-05-08T08:54:49.233 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:49.234 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.234 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.234 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:49.234 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355762+0000", 2024-05-08T08:54:49.234 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3015000000000003e-05 2024-05-08T08:54:49.234 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.234 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.234 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:49.234 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355765+0000", 2024-05-08T08:54:49.234 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.368e-06 2024-05-08T08:54:49.234 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.234 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.234 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:49.234 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355771+0000", 2024-05-08T08:54:49.235 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5989999999999998e-06 2024-05-08T08:54:49.235 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.235 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.235 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:49.235 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355775+0000", 2024-05-08T08:54:49.235 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7280000000000001e-06 2024-05-08T08:54:49.235 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.235 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.235 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:49.235 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434379+0000", 2024-05-08T08:54:49.235 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078603662000000005 2024-05-08T08:54:49.235 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.235 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.235 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:49.235 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434473+0000", 2024-05-08T08:54:49.236 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4104000000000006e-05 2024-05-08T08:54:49.236 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.236 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.236 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:49.236 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434626+0000", 2024-05-08T08:54:49.236 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00015313100000000001 2024-05-08T08:54:49.236 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.236 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:49.236 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.236 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.236 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.236 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28549 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889202400] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:49.236 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.355781+0000", 2024-05-08T08:54:49.236 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 76.859600592000007, 2024-05-08T08:54:49.236 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079117946999999994, 2024-05-08T08:54:49.236 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:49.236 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:49.237 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:49.237 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:49.237 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:49.237 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:49.237 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28549 2024-05-08T08:54:49.237 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.237 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:49.237 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.237 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:49.237 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355781+0000", 2024-05-08T08:54:49.237 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:49.237 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.237 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.237 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:49.237 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355781+0000", 2024-05-08T08:54:49.238 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:49.238 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.238 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.238 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:49.238 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356162+0000", 2024-05-08T08:54:49.238 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00038099100000000001 2024-05-08T08:54:49.238 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.238 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.238 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:49.238 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356165+0000", 2024-05-08T08:54:49.238 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1240000000000001e-06 2024-05-08T08:54:49.238 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.238 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.238 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:49.238 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356173+0000", 2024-05-08T08:54:49.239 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.6199999999999999e-06 2024-05-08T08:54:49.239 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.239 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.239 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:49.239 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356182+0000", 2024-05-08T08:54:49.239 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7760000000000003e-06 2024-05-08T08:54:49.239 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.239 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.239 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:49.239 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434675+0000", 2024-05-08T08:54:49.239 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078493119 2024-05-08T08:54:49.239 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.239 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.239 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:49.239 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434765+0000", 2024-05-08T08:54:49.240 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0402000000000006e-05 2024-05-08T08:54:49.240 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.240 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.240 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:49.240 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434899+0000", 2024-05-08T08:54:49.240 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013391499999999999 2024-05-08T08:54:49.240 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.240 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:49.240 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.240 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.240 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.240 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28550 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888597280] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:49.240 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:54:49.240 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 76.859189545999996, 2024-05-08T08:54:49.240 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078933990999999995, 2024-05-08T08:54:49.240 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:49.241 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:49.241 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:49.241 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:49.241 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:49.241 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:49.241 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28550 2024-05-08T08:54:49.241 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.241 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:49.241 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.241 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:49.241 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:54:49.241 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:49.241 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.241 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.241 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:49.242 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:54:49.242 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:49.242 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.242 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.242 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:49.242 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:54:49.242 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.34e-07 2024-05-08T08:54:49.242 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.242 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.242 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:49.242 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356195+0000", 2024-05-08T08:54:49.242 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8710000000000001e-06 2024-05-08T08:54:49.242 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.242 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.243 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:49.243 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356200+0000", 2024-05-08T08:54:49.243 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3000000000000003e-06 2024-05-08T08:54:49.243 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.243 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.243 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:49.243 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356203+0000", 2024-05-08T08:54:49.243 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4039999999999999e-06 2024-05-08T08:54:49.243 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.243 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.243 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:49.243 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434928+0000", 2024-05-08T08:54:49.243 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078725331999999995 2024-05-08T08:54:49.243 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.243 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.244 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:49.244 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435007+0000", 2024-05-08T08:54:49.244 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8415000000000001e-05 2024-05-08T08:54:49.244 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.244 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.244 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:49.244 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435126+0000", 2024-05-08T08:54:49.244 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011923500000000001 2024-05-08T08:54:49.244 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.244 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:49.244 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.244 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.245 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.245 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28551 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888527200] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:49.245 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.356210+0000", 2024-05-08T08:54:49.245 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 76.859171197999999, 2024-05-08T08:54:49.245 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079195072000000005, 2024-05-08T08:54:49.245 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:49.245 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:49.245 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:49.245 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:49.245 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:49.245 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:49.245 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28551 2024-05-08T08:54:49.245 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.245 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:49.245 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.246 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:49.246 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356210+0000", 2024-05-08T08:54:49.246 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:49.246 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.246 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.246 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:49.246 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356210+0000", 2024-05-08T08:54:49.246 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:49.246 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.246 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.246 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:49.246 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356211+0000", 2024-05-08T08:54:49.246 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4499999999999998e-07 2024-05-08T08:54:49.246 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.247 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.247 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:49.247 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356212+0000", 2024-05-08T08:54:49.247 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6029999999999999e-06 2024-05-08T08:54:49.247 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.247 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.247 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:49.247 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356221+0000", 2024-05-08T08:54:49.247 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4200000000000007e-06 2024-05-08T08:54:49.247 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.247 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.247 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:49.247 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356224+0000", 2024-05-08T08:54:49.247 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.038e-06 2024-05-08T08:54:49.247 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.248 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.248 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:49.248 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435178+0000", 2024-05-08T08:54:49.248 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078954625000000001 2024-05-08T08:54:49.248 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.248 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.248 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:49.248 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435256+0000", 2024-05-08T08:54:49.248 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7337999999999997e-05 2024-05-08T08:54:49.248 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.248 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.248 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:49.248 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435405+0000", 2024-05-08T08:54:49.248 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000149703 2024-05-08T08:54:49.248 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.249 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:49.249 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.249 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.249 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.249 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28552 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889231456] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:49.249 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.356228+0000", 2024-05-08T08:54:49.249 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 76.859153520000007, 2024-05-08T08:54:49.249 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079346920000000001, 2024-05-08T08:54:49.249 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:49.249 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:49.249 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:49.249 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:49.249 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:49.249 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:49.250 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28552 2024-05-08T08:54:49.250 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.250 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:49.250 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.250 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:49.250 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356228+0000", 2024-05-08T08:54:49.250 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:49.250 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.250 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.250 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:49.250 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356228+0000", 2024-05-08T08:54:49.250 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:49.250 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.250 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.251 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:49.251 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356959+0000", 2024-05-08T08:54:49.251 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00073097099999999996 2024-05-08T08:54:49.251 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.251 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.251 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:49.251 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356967+0000", 2024-05-08T08:54:49.251 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0409999999999998e-06 2024-05-08T08:54:49.251 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.251 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.251 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:49.251 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356979+0000", 2024-05-08T08:54:49.251 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1489e-05 2024-05-08T08:54:49.251 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.252 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.252 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:49.252 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356986+0000", 2024-05-08T08:54:49.252 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3699999999999997e-06 2024-05-08T08:54:49.252 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.252 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.252 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:49.252 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435433+0000", 2024-05-08T08:54:49.252 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078447520000000007 2024-05-08T08:54:49.252 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.252 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.252 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:49.252 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435484+0000", 2024-05-08T08:54:49.252 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0229000000000003e-05 2024-05-08T08:54:49.252 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.253 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.253 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:49.253 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435575+0000", 2024-05-08T08:54:49.253 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1299999999999997e-05 2024-05-08T08:54:49.253 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.253 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:49.253 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.253 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.253 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.253 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28553 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888713648] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:49.253 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:54:49.253 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 76.858378861999995, 2024-05-08T08:54:49.253 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078782034000000001, 2024-05-08T08:54:49.253 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:49.253 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:49.254 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:49.254 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:49.254 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:49.254 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:49.254 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28553 2024-05-08T08:54:49.254 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.254 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:49.254 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.254 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:49.254 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:54:49.254 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:49.254 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.254 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.254 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:49.254 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:54:49.255 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:49.255 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.255 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.255 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:49.255 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:54:49.255 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6600000000000002e-07 2024-05-08T08:54:49.255 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.255 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.255 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:49.255 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357006+0000", 2024-05-08T08:54:49.255 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7190000000000001e-06 2024-05-08T08:54:49.255 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.255 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.255 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:49.256 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357013+0000", 2024-05-08T08:54:49.256 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.435e-06 2024-05-08T08:54:49.256 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.256 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.256 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:49.256 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357018+0000", 2024-05-08T08:54:49.256 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7260000000000002e-06 2024-05-08T08:54:49.256 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.256 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.256 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:49.256 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435623+0000", 2024-05-08T08:54:49.256 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078604972999999995 2024-05-08T08:54:49.256 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.256 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.256 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:49.256 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435695+0000", 2024-05-08T08:54:49.256 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1959999999999995e-05 2024-05-08T08:54:49.257 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.257 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.257 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:49.257 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435785+0000", 2024-05-08T08:54:49.257 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.9754999999999997e-05 2024-05-08T08:54:49.257 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.257 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:49.257 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.257 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.257 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.257 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28554 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888980272] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:49.257 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357024+0000", 2024-05-08T08:54:49.257 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 76.858357225999995, 2024-05-08T08:54:49.257 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078999870999999999, 2024-05-08T08:54:49.257 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:49.258 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:49.258 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:49.258 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:49.258 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:49.258 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:49.258 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28554 2024-05-08T08:54:49.258 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.258 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:49.258 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.258 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:49.258 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357024+0000", 2024-05-08T08:54:49.258 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:49.258 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.258 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.258 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:49.259 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357024+0000", 2024-05-08T08:54:49.259 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:49.259 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.259 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.259 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:49.259 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357025+0000", 2024-05-08T08:54:49.259 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:54:49.259 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.259 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.259 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:49.259 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357027+0000", 2024-05-08T08:54:49.259 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2050000000000001e-06 2024-05-08T08:54:49.259 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.259 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.260 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:49.260 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357033+0000", 2024-05-08T08:54:49.260 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5679999999999996e-06 2024-05-08T08:54:49.260 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.260 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.260 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:49.260 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357038+0000", 2024-05-08T08:54:49.260 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2889999999999999e-06 2024-05-08T08:54:49.260 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.260 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.260 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:49.260 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435823+0000", 2024-05-08T08:54:49.260 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078785677999999998 2024-05-08T08:54:49.260 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.260 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.261 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:49.261 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435892+0000", 2024-05-08T08:54:49.261 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8379000000000004e-05 2024-05-08T08:54:49.261 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.261 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.261 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:49.261 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436024+0000", 2024-05-08T08:54:49.261 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013239799999999999 2024-05-08T08:54:49.261 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.261 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:49.261 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.261 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.261 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.261 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28555 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889001104] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:49.261 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:54:49.261 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 76.858338461000002, 2024-05-08T08:54:49.262 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079115742000000003, 2024-05-08T08:54:49.262 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:49.262 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:49.262 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:49.262 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:49.262 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:49.262 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:49.262 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28555 2024-05-08T08:54:49.262 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.262 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:49.262 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.262 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:49.262 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:54:49.262 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:49.263 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.263 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.263 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:49.263 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:54:49.263 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:49.263 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.263 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.263 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:49.263 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:54:49.263 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9200000000000002e-07 2024-05-08T08:54:49.263 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.263 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.263 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:49.263 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357046+0000", 2024-05-08T08:54:49.264 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1670000000000002e-06 2024-05-08T08:54:49.264 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.264 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.264 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:49.264 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357052+0000", 2024-05-08T08:54:49.264 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.139e-06 2024-05-08T08:54:49.264 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.264 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.264 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:49.264 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357056+0000", 2024-05-08T08:54:49.264 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2440000000000004e-06 2024-05-08T08:54:49.264 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.264 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.264 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:49.265 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436059+0000", 2024-05-08T08:54:49.265 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079003070999999994 2024-05-08T08:54:49.265 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.265 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.265 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:49.265 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436097+0000", 2024-05-08T08:54:49.265 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7784000000000003e-05 2024-05-08T08:54:49.265 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.265 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.265 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:49.265 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436159+0000", 2024-05-08T08:54:49.265 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2045000000000007e-05 2024-05-08T08:54:49.265 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.265 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:49.265 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.266 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.266 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.266 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28556 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889022064] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:49.266 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357065+0000", 2024-05-08T08:54:49.266 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 76.858316427999995, 2024-05-08T08:54:49.266 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079327976999999994, 2024-05-08T08:54:49.266 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:49.266 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:49.266 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:49.266 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:49.266 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:49.266 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:49.266 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28556 2024-05-08T08:54:49.267 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.267 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:49.267 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.267 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:49.267 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357065+0000", 2024-05-08T08:54:49.267 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:49.267 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.267 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.267 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:49.267 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357065+0000", 2024-05-08T08:54:49.267 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:49.267 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.267 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.267 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:49.267 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358958+0000", 2024-05-08T08:54:49.267 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001892915 2024-05-08T08:54:49.268 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.268 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.268 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:49.268 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358962+0000", 2024-05-08T08:54:49.268 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4599999999999999e-06 2024-05-08T08:54:49.268 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.268 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.268 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:49.268 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358970+0000", 2024-05-08T08:54:49.268 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.6090000000000001e-06 2024-05-08T08:54:49.268 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.268 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.268 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:49.268 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358975+0000", 2024-05-08T08:54:49.268 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7639999999999996e-06 2024-05-08T08:54:49.269 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.269 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.269 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:49.269 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436196+0000", 2024-05-08T08:54:49.269 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.077221564000000006 2024-05-08T08:54:49.269 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.269 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.269 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:49.269 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436269+0000", 2024-05-08T08:54:49.269 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2720999999999997e-05 2024-05-08T08:54:49.269 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.269 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.269 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:49.269 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436393+0000", 2024-05-08T08:54:49.269 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012394399999999999 2024-05-08T08:54:49.270 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.270 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:49.270 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.270 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.270 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.270 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28563 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888855520] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:49.270 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:54:49.270 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 76.856307029000007, 2024-05-08T08:54:49.270 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078795566999999997, 2024-05-08T08:54:49.270 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:49.270 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:49.270 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:49.270 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:49.270 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:49.271 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:49.271 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28563 2024-05-08T08:54:49.271 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.271 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:49.271 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.271 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:49.271 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:54:49.271 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:49.271 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.271 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.271 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:49.271 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:54:49.271 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:49.271 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.271 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.272 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:49.272 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:54:49.272 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1e-07 2024-05-08T08:54:49.272 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.272 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.272 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:49.272 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359080+0000", 2024-05-08T08:54:49.272 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8940000000000001e-06 2024-05-08T08:54:49.272 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.272 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.272 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:49.272 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359083+0000", 2024-05-08T08:54:49.272 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1949999999999999e-06 2024-05-08T08:54:49.272 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.272 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.273 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:49.273 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359085+0000", 2024-05-08T08:54:49.273 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4710000000000001e-06 2024-05-08T08:54:49.273 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.273 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.273 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:49.273 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437604+0000", 2024-05-08T08:54:49.273 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078518278999999996 2024-05-08T08:54:49.273 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.273 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.273 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:49.273 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437704+0000", 2024-05-08T08:54:49.273 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010010599999999999 2024-05-08T08:54:49.273 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.273 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.274 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:49.274 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437870+0000", 2024-05-08T08:54:49.274 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00016641200000000001 2024-05-08T08:54:49.274 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.274 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:49.274 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.274 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.274 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.274 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28564 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888538944] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:49.274 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.359089+0000", 2024-05-08T08:54:49.274 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 76.856292814, 2024-05-08T08:54:49.274 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078952675, 2024-05-08T08:54:49.274 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:49.274 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:49.274 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:49.275 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:49.275 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:49.275 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:49.275 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28564 2024-05-08T08:54:49.275 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.275 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:49.275 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.275 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:49.275 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359089+0000", 2024-05-08T08:54:49.275 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:49.275 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.275 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.275 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:49.275 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359089+0000", 2024-05-08T08:54:49.276 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:49.276 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.276 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.276 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:49.276 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360959+0000", 2024-05-08T08:54:49.276 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0018707089999999999 2024-05-08T08:54:49.276 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.276 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.276 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:49.276 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360963+0000", 2024-05-08T08:54:49.276 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0680000000000001e-06 2024-05-08T08:54:49.276 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.276 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.276 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:49.276 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360969+0000", 2024-05-08T08:54:49.276 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1920000000000002e-06 2024-05-08T08:54:49.277 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.277 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.277 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:49.277 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360973+0000", 2024-05-08T08:54:49.277 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.031e-06 2024-05-08T08:54:49.277 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.277 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.277 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:49.277 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437916+0000", 2024-05-08T08:54:49.277 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.076942939000000002 2024-05-08T08:54:49.277 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.277 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.277 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:49.277 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437973+0000", 2024-05-08T08:54:49.277 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7358999999999997e-05 2024-05-08T08:54:49.277 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.277 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.278 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:49.278 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.438041+0000", 2024-05-08T08:54:49.278 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8376999999999999e-05 2024-05-08T08:54:49.278 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.278 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:49.278 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.278 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.278 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.278 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28574 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889427600] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:49.278 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:54:49.278 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 76.854288307000004, 2024-05-08T08:54:49.278 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078782476000000004, 2024-05-08T08:54:49.278 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:49.279 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:49.279 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:49.279 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:49.279 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:49.279 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:49.279 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28574 2024-05-08T08:54:49.279 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.279 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:49.279 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.279 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:49.279 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:54:49.279 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:49.279 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.279 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.279 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:49.279 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:54:49.280 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:49.280 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.280 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.280 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:49.280 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:54:49.280 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1799999999999999e-07 2024-05-08T08:54:49.280 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.280 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.280 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:49.280 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361095+0000", 2024-05-08T08:54:49.280 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3850000000000001e-06 2024-05-08T08:54:49.280 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.280 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.280 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:49.280 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361098+0000", 2024-05-08T08:54:49.280 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7989999999999998e-06 2024-05-08T08:54:49.281 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.281 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.281 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:49.281 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361100+0000", 2024-05-08T08:54:49.281 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4669999999999998e-06 2024-05-08T08:54:49.281 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.281 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.281 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:49.281 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439768+0000", 2024-05-08T08:54:49.281 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078668057999999999 2024-05-08T08:54:49.281 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.281 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.281 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:49.281 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439811+0000", 2024-05-08T08:54:49.281 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2459000000000001e-05 2024-05-08T08:54:49.281 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.282 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.282 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:49.282 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439876+0000", 2024-05-08T08:54:49.282 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5090000000000002e-05 2024-05-08T08:54:49.282 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.282 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:49.282 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.282 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.282 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.282 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28575 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889439216] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:49.282 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361103+0000", 2024-05-08T08:54:49.282 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 76.854278292000004, 2024-05-08T08:54:49.282 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078966828000000003, 2024-05-08T08:54:49.282 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:49.282 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:49.283 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:49.283 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:49.283 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:49.283 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:49.283 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28575 2024-05-08T08:54:49.283 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.283 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:49.283 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.283 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:49.283 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361103+0000", 2024-05-08T08:54:49.283 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:49.283 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.283 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.283 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:49.283 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361103+0000", 2024-05-08T08:54:49.284 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:49.284 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.284 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.284 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:49.284 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361104+0000", 2024-05-08T08:54:49.284 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1400000000000001e-07 2024-05-08T08:54:49.284 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.284 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.284 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:49.284 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361105+0000", 2024-05-08T08:54:49.284 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5540000000000001e-06 2024-05-08T08:54:49.284 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.284 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.284 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:49.284 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361108+0000", 2024-05-08T08:54:49.285 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.841e-06 2024-05-08T08:54:49.285 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.285 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.285 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:49.285 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361110+0000", 2024-05-08T08:54:49.285 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.497e-06 2024-05-08T08:54:49.285 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.285 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.285 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:49.285 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439916+0000", 2024-05-08T08:54:49.285 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078805372999999998 2024-05-08T08:54:49.285 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.285 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.285 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:49.285 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440000+0000", 2024-05-08T08:54:49.285 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4730999999999999e-05 2024-05-08T08:54:49.286 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.286 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.286 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:49.286 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440070+0000", 2024-05-08T08:54:49.286 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9617999999999997e-05 2024-05-08T08:54:49.286 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.286 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:49.286 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.286 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.286 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.286 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28576 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889451328] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:49.286 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361113+0000", 2024-05-08T08:54:49.286 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 76.854268106000006, 2024-05-08T08:54:49.286 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079088862999999995, 2024-05-08T08:54:49.286 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:49.287 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:49.287 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:49.287 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:49.287 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:49.287 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:49.287 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28576 2024-05-08T08:54:49.287 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.287 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:49.287 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.287 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:49.287 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361113+0000", 2024-05-08T08:54:49.287 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:49.287 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.287 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.287 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:49.287 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361113+0000", 2024-05-08T08:54:49.288 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:49.288 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.288 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.288 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:49.288 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361114+0000", 2024-05-08T08:54:49.288 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1299999999999999e-07 2024-05-08T08:54:49.288 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.288 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.288 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:49.288 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361115+0000", 2024-05-08T08:54:49.288 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.4300000000000001e-06 2024-05-08T08:54:49.288 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.288 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.288 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:49.288 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361118+0000", 2024-05-08T08:54:49.289 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9000000000000002e-06 2024-05-08T08:54:49.289 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.289 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.289 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:49.289 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361120+0000", 2024-05-08T08:54:49.289 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4739999999999999e-06 2024-05-08T08:54:49.289 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.289 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.289 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:49.289 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440096+0000", 2024-05-08T08:54:49.289 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078975629000000006 2024-05-08T08:54:49.289 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.289 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.289 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:49.289 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440135+0000", 2024-05-08T08:54:49.289 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9202000000000002e-05 2024-05-08T08:54:49.289 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.290 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.290 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:49.290 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440202+0000", 2024-05-08T08:54:49.290 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7014999999999994e-05 2024-05-08T08:54:49.290 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.290 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:49.290 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.290 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.290 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.290 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28577 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889462640] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:49.290 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:54:49.290 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 76.854257923999995, 2024-05-08T08:54:49.290 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079194622000000006, 2024-05-08T08:54:49.290 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:49.290 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:49.291 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:49.291 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:49.291 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:49.291 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:49.291 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28577 2024-05-08T08:54:49.291 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.291 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:49.291 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.291 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:49.291 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:54:49.291 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:49.291 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.291 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.291 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:49.291 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:54:49.292 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:49.292 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.292 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.292 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:49.292 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:54:49.292 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1400000000000001e-07 2024-05-08T08:54:49.292 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.292 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.292 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:49.292 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361125+0000", 2024-05-08T08:54:49.292 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.372e-06 2024-05-08T08:54:49.292 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.292 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.292 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:49.292 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361128+0000", 2024-05-08T08:54:49.293 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0309999999999999e-06 2024-05-08T08:54:49.293 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.293 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.293 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:49.293 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361131+0000", 2024-05-08T08:54:49.293 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4040000000000002e-06 2024-05-08T08:54:49.293 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.293 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.293 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:49.293 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440225+0000", 2024-05-08T08:54:49.293 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079094600000000001 2024-05-08T08:54:49.293 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.293 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.293 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:49.293 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440260+0000", 2024-05-08T08:54:49.293 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4703000000000003e-05 2024-05-08T08:54:49.294 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.294 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.294 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:49.294 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440318+0000", 2024-05-08T08:54:49.294 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8298000000000003e-05 2024-05-08T08:54:49.294 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.294 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:49.294 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.294 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.294 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.294 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28578 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889474192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:49.294 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361134+0000", 2024-05-08T08:54:49.294 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 76.854247498999996, 2024-05-08T08:54:49.294 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079310295000000003, 2024-05-08T08:54:49.294 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:49.295 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:49.295 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:49.295 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:49.295 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:49.295 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:49.295 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28578 2024-05-08T08:54:49.295 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.295 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:49.295 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.295 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:49.295 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361134+0000", 2024-05-08T08:54:49.295 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:49.295 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.295 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.295 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:49.295 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361134+0000", 2024-05-08T08:54:49.296 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:49.296 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.296 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.296 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:49.296 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.362958+0000", 2024-05-08T08:54:49.296 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0018237380000000001 2024-05-08T08:54:49.296 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.296 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.296 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:49.296 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.362961+0000", 2024-05-08T08:54:49.296 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.123e-06 2024-05-08T08:54:49.296 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.296 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.296 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:49.296 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.397404+0000", 2024-05-08T08:54:49.296 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.034443226 2024-05-08T08:54:49.297 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.297 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.297 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:49.297 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.397439+0000", 2024-05-08T08:54:49.297 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4690000000000002e-05 2024-05-08T08:54:49.297 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.297 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.297 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:49.297 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440340+0000", 2024-05-08T08:54:49.297 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.042901419000000003 2024-05-08T08:54:49.297 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.297 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.297 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:49.297 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440376+0000", 2024-05-08T08:54:49.297 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5444999999999997e-05 2024-05-08T08:54:49.297 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.298 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.298 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:49.298 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440444+0000", 2024-05-08T08:54:49.298 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8653999999999997e-05 2024-05-08T08:54:49.298 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.298 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:49.298 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.298 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.298 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:49.298 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:49.300 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:54:49.300 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:49.302+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:49.401 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:49.403+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:49.418 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:49.418 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:54:49.418 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:54:49.418 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:49.428 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:54:49.512 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:49.515+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:49.578 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:49.578 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:54:49.578 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:54:49.578 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:54:49.578 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:49.587 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:54:49.629 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:49.631+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:49.703 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:49.706+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:49.739 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:49.739 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:54:49.740 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:54:49.740 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:54:49.740 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.740 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:437 3.17 3:ea93a512:::rbd_data.1140100c9703.0000000000000077:head [set-alloc-hint object_size 2097152 write_size 2097152,write 1170432~1024] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e63)", 2024-05-08T08:54:49.740 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:08.390804+0000", 2024-05-08T08:54:49.740 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 101.35051622, 2024-05-08T08:54:49.740 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0406068690000001, 2024-05-08T08:54:49.740 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:49.740 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:49.740 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:54:49.740 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:49.740 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:49.740 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:49.740 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 437 2024-05-08T08:54:49.741 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.741 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:49.741 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.741 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:49.741 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390804+0000", 2024-05-08T08:54:49.741 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:49.741 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.741 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.741 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:49.741 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390804+0000", 2024-05-08T08:54:49.741 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:49.741 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.741 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.741 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:49.742 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390806+0000", 2024-05-08T08:54:49.742 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.666e-06 2024-05-08T08:54:49.742 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.742 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.742 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:49.742 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390818+0000", 2024-05-08T08:54:49.742 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2187999999999999e-05 2024-05-08T08:54:49.742 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.742 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.742 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:49.742 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390834+0000", 2024-05-08T08:54:49.742 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6191000000000001e-05 2024-05-08T08:54:49.742 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.742 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.742 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:49.743 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390845+0000", 2024-05-08T08:54:49.743 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0831e-05 2024-05-08T08:54:49.743 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.743 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.743 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:49.743 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390922+0000", 2024-05-08T08:54:49.743 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7253e-05 2024-05-08T08:54:49.743 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.743 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.743 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op must wait for map", 2024-05-08T08:54:49.743 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390930+0000", 2024-05-08T08:54:49.743 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8259999999999993e-06 2024-05-08T08:54:49.743 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.743 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.743 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:49.744 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392155+0000", 2024-05-08T08:54:49.744 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0012256649999999999 2024-05-08T08:54:49.744 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.744 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.744 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:54:49.744 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392165+0000", 2024-05-08T08:54:49.744 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5799999999999998e-06 2024-05-08T08:54:49.744 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.744 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.744 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:49.744 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.429596+0000", 2024-05-08T08:54:49.744 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0374313639999999 2024-05-08T08:54:49.744 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.744 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.744 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:49.745 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.429728+0000", 2024-05-08T08:54:49.745 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000131552 2024-05-08T08:54:49.745 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.745 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.745 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 0,1", 2024-05-08T08:54:49.745 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.429860+0000", 2024-05-08T08:54:49.745 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000132137 2024-05-08T08:54:49.745 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.745 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.745 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:54:49.745 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.430811+0000", 2024-05-08T08:54:49.745 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00095040200000000004 2024-05-08T08:54:49.745 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.745 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.745 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:54:49.746 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431126+0000", 2024-05-08T08:54:49.746 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000315006 2024-05-08T08:54:49.746 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.746 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.746 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:54:49.746 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431272+0000", 2024-05-08T08:54:49.746 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000146487 2024-05-08T08:54:49.746 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.746 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.746 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:54:49.746 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431315+0000", 2024-05-08T08:54:49.746 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2697000000000001e-05 2024-05-08T08:54:49.746 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.746 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.747 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:49.747 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431411+0000", 2024-05-08T08:54:49.747 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6024000000000001e-05 2024-05-08T08:54:49.747 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.747 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:49.747 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.747 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.747 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.747 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:439 3.17 3:ea93a512:::rbd_data.1140100c9703.0000000000000077:head [read 1163264~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-05-08T08:54:49.747 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:08.390871+0000", 2024-05-08T08:54:49.747 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 101.35044870900001, 2024-05-08T08:54:49.747 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0408790590000001, 2024-05-08T08:54:49.747 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:49.747 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:49.747 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:49.748 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:49.748 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:49.748 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:49.748 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 439 2024-05-08T08:54:49.748 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.748 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:49.748 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.748 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:49.748 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390871+0000", 2024-05-08T08:54:49.748 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:49.748 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.748 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.748 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:49.748 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390871+0000", 2024-05-08T08:54:49.748 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:49.748 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.749 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.749 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:49.749 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390872+0000", 2024-05-08T08:54:49.749 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6300000000000005e-07 2024-05-08T08:54:49.749 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.749 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.749 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:49.749 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390875+0000", 2024-05-08T08:54:49.749 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3459999999999998e-06 2024-05-08T08:54:49.749 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.749 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.749 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:49.749 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390891+0000", 2024-05-08T08:54:49.749 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5255e-05 2024-05-08T08:54:49.749 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.750 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.750 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:49.750 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390897+0000", 2024-05-08T08:54:49.750 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2360000000000001e-06 2024-05-08T08:54:49.750 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.750 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.750 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:49.750 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390995+0000", 2024-05-08T08:54:49.750 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8522000000000003e-05 2024-05-08T08:54:49.750 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.750 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.750 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting_for_map not empty", 2024-05-08T08:54:49.750 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.391005+0000", 2024-05-08T08:54:49.751 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1460000000000002e-06 2024-05-08T08:54:49.751 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.751 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.751 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:49.751 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392191+0000", 2024-05-08T08:54:49.751 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0011860270000000001 2024-05-08T08:54:49.751 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.751 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.751 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:54:49.751 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392197+0000", 2024-05-08T08:54:49.751 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4289999999999998e-06 2024-05-08T08:54:49.751 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.751 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.751 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:49.751 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.430596+0000", 2024-05-08T08:54:49.752 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0383989090000001 2024-05-08T08:54:49.752 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.752 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.752 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for rw locks", 2024-05-08T08:54:49.752 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.430676+0000", 2024-05-08T08:54:49.752 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9781000000000001e-05 2024-05-08T08:54:49.752 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.752 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.752 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:49.752 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431460+0000", 2024-05-08T08:54:49.752 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00078427300000000004 2024-05-08T08:54:49.752 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.752 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.752 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:49.753 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431540+0000", 2024-05-08T08:54:49.753 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0224999999999998e-05 2024-05-08T08:54:49.753 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.753 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.753 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:49.753 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431750+0000", 2024-05-08T08:54:49.753 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00021024699999999999 2024-05-08T08:54:49.753 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.753 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:49.753 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.753 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.753 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.753 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:945 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 4096000~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:49.753 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:54:49.753 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 100.29029914199999, 2024-05-08T08:54:49.753 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086774741000000002, 2024-05-08T08:54:49.754 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:49.754 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:49.754 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:49.754 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:49.754 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:49.754 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:49.754 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 945 2024-05-08T08:54:49.754 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.754 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:49.754 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.754 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:49.754 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:54:49.754 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:49.754 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.755 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.755 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:49.755 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:54:49.755 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:49.755 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.755 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.755 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:49.755 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:54:49.755 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3499999999999996e-07 2024-05-08T08:54:49.755 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.755 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.755 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:49.755 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451024+0000", 2024-05-08T08:54:49.755 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.627e-06 2024-05-08T08:54:49.755 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.756 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.756 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:49.756 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451037+0000", 2024-05-08T08:54:49.756 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2431e-05 2024-05-08T08:54:49.756 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.756 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.756 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:49.756 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451042+0000", 2024-05-08T08:54:49.756 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8409999999999996e-06 2024-05-08T08:54:49.756 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.756 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.756 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:49.756 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537608+0000", 2024-05-08T08:54:49.756 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086565243 2024-05-08T08:54:49.757 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.757 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.757 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:49.757 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537687+0000", 2024-05-08T08:54:49.757 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9519999999999998e-05 2024-05-08T08:54:49.757 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.757 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.757 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:49.757 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537796+0000", 2024-05-08T08:54:49.757 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000108544 2024-05-08T08:54:49.757 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.757 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:49.757 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.757 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.757 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.758 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:964 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 458752~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:49.758 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451051+0000", 2024-05-08T08:54:49.758 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 100.290268969, 2024-05-08T08:54:49.758 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086990490000000004, 2024-05-08T08:54:49.758 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:49.758 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:49.758 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:49.758 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:49.758 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:49.758 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:49.758 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 964 2024-05-08T08:54:49.758 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.758 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:49.759 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.759 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:49.759 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451051+0000", 2024-05-08T08:54:49.759 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:49.759 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.759 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.759 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:49.759 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451051+0000", 2024-05-08T08:54:49.759 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:49.759 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.759 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.759 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:49.759 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451052+0000", 2024-05-08T08:54:49.759 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7299999999999996e-07 2024-05-08T08:54:49.759 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.760 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.760 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:49.760 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451055+0000", 2024-05-08T08:54:49.760 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.2770000000000001e-06 2024-05-08T08:54:49.760 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.760 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.760 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:49.760 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451061+0000", 2024-05-08T08:54:49.760 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4060000000000002e-06 2024-05-08T08:54:49.761 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.761 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.761 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:49.761 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451066+0000", 2024-05-08T08:54:49.761 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0839999999999999e-06 2024-05-08T08:54:49.761 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.761 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.761 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:49.761 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537839+0000", 2024-05-08T08:54:49.761 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086772962999999995 2024-05-08T08:54:49.761 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.761 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.762 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:49.762 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537934+0000", 2024-05-08T08:54:49.762 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4742999999999997e-05 2024-05-08T08:54:49.762 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.762 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.762 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:49.762 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538042+0000", 2024-05-08T08:54:49.762 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000107444 2024-05-08T08:54:49.762 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.762 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:49.762 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.762 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.762 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.762 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:965 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 950272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:49.763 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451075+0000", 2024-05-08T08:54:49.763 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 100.290244907, 2024-05-08T08:54:49.763 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087230196999999995, 2024-05-08T08:54:49.763 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:49.763 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:49.763 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:49.763 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:49.763 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:49.763 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:49.763 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 965 2024-05-08T08:54:49.763 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.763 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:49.764 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.764 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:49.764 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451075+0000", 2024-05-08T08:54:49.764 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:49.764 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.764 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.764 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:49.764 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451075+0000", 2024-05-08T08:54:49.764 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:49.764 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.764 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.764 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:49.765 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451076+0000", 2024-05-08T08:54:49.765 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5000000000000002e-07 2024-05-08T08:54:49.765 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.765 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.765 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:49.765 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451078+0000", 2024-05-08T08:54:49.765 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.5639999999999999e-06 2024-05-08T08:54:49.765 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.766 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.766 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:49.766 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451085+0000", 2024-05-08T08:54:49.766 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5640000000000002e-06 2024-05-08T08:54:49.766 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.766 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.767 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:49.767 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451090+0000", 2024-05-08T08:54:49.767 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1699999999999996e-06 2024-05-08T08:54:49.767 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.767 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.767 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:49.767 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538106+0000", 2024-05-08T08:54:49.768 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087015888999999999 2024-05-08T08:54:49.768 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.768 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.768 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:49.768 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538190+0000", 2024-05-08T08:54:49.768 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3961999999999993e-05 2024-05-08T08:54:49.769 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.769 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.769 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:49.769 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538305+0000", 2024-05-08T08:54:49.769 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011529800000000001 2024-05-08T08:54:49.769 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.769 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:49.770 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.770 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.770 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.770 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:966 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 1441792~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:49.770 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:54:49.770 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 100.290221534, 2024-05-08T08:54:49.770 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087446323000000006, 2024-05-08T08:54:49.771 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:49.771 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:49.771 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:49.771 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:49.771 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:49.771 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:49.771 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 966 2024-05-08T08:54:49.771 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.771 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:49.772 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.772 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:49.772 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:54:49.772 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:49.772 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.772 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.773 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:49.773 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:54:49.773 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:49.773 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.773 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.773 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:49.773 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:54:49.774 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3700000000000001e-07 2024-05-08T08:54:49.774 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.774 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.774 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:49.774 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451101+0000", 2024-05-08T08:54:49.774 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2560000000000001e-06 2024-05-08T08:54:49.775 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.775 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.775 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:49.775 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451107+0000", 2024-05-08T08:54:49.775 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1070000000000001e-06 2024-05-08T08:54:49.775 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.775 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.775 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:49.775 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451112+0000", 2024-05-08T08:54:49.775 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1420000000000001e-06 2024-05-08T08:54:49.776 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.776 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.776 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:49.776 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538341+0000", 2024-05-08T08:54:49.776 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087228302999999993 2024-05-08T08:54:49.776 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.776 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.776 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:49.776 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538424+0000", 2024-05-08T08:54:49.776 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3497000000000004e-05 2024-05-08T08:54:49.776 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.776 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.776 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:49.777 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538545+0000", 2024-05-08T08:54:49.777 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000120581 2024-05-08T08:54:49.777 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.777 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:49.777 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.777 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.777 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.777 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:967 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:49.777 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451120+0000", 2024-05-08T08:54:49.777 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 100.290200059, 2024-05-08T08:54:49.777 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087632725999999994, 2024-05-08T08:54:49.777 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:49.777 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:49.777 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:49.777 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:49.778 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:49.778 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:49.778 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 967 2024-05-08T08:54:49.778 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.778 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:49.778 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.778 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:49.778 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451120+0000", 2024-05-08T08:54:49.778 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:49.778 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.778 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.779 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:49.779 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451120+0000", 2024-05-08T08:54:49.779 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:49.779 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.779 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.779 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:49.779 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451121+0000", 2024-05-08T08:54:49.779 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8800000000000002e-07 2024-05-08T08:54:49.779 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.779 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.779 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:49.779 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451130+0000", 2024-05-08T08:54:49.779 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2299999999999997e-06 2024-05-08T08:54:49.779 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.780 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.780 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:49.780 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451137+0000", 2024-05-08T08:54:49.780 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4669999999999998e-06 2024-05-08T08:54:49.780 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.780 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.780 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:49.780 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451143+0000", 2024-05-08T08:54:49.780 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2839999999999997e-06 2024-05-08T08:54:49.780 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.780 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.780 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:49.780 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538575+0000", 2024-05-08T08:54:49.780 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087432789999999996 2024-05-08T08:54:49.780 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.781 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.781 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:49.781 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538648+0000", 2024-05-08T08:54:49.781 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2877e-05 2024-05-08T08:54:49.781 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.781 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.781 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:49.781 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538753+0000", 2024-05-08T08:54:49.781 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010449 2024-05-08T08:54:49.781 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.781 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:49.781 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.781 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.781 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.781 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:968 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 2424832~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:49.782 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451152+0000", 2024-05-08T08:54:49.782 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 100.290167974, 2024-05-08T08:54:49.782 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087813325999999997, 2024-05-08T08:54:49.782 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:49.782 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:49.782 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:49.782 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:49.782 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:49.782 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:49.782 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 968 2024-05-08T08:54:49.782 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.782 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:49.782 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.782 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:49.783 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451152+0000", 2024-05-08T08:54:49.783 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:49.783 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.783 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.783 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:49.783 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451152+0000", 2024-05-08T08:54:49.783 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:49.783 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.783 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.783 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:49.783 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451153+0000", 2024-05-08T08:54:49.783 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3099999999999998e-07 2024-05-08T08:54:49.783 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.783 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.783 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:49.784 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451155+0000", 2024-05-08T08:54:49.784 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4760000000000001e-06 2024-05-08T08:54:49.784 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.784 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.784 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:49.784 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451162+0000", 2024-05-08T08:54:49.784 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5969999999999997e-06 2024-05-08T08:54:49.784 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.784 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.784 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:49.784 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451167+0000", 2024-05-08T08:54:49.784 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.977e-06 2024-05-08T08:54:49.784 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.784 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.784 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:49.785 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538781+0000", 2024-05-08T08:54:49.785 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087614846999999996 2024-05-08T08:54:49.785 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.785 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.785 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:49.785 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538857+0000", 2024-05-08T08:54:49.785 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5358000000000006e-05 2024-05-08T08:54:49.785 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.785 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.785 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:49.785 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538965+0000", 2024-05-08T08:54:49.785 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010864 2024-05-08T08:54:49.786 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.786 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:49.786 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.786 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.786 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.786 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:969 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 2916352~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:49.786 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:54:49.786 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 100.290146269, 2024-05-08T08:54:49.786 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087998980000000004, 2024-05-08T08:54:49.786 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:49.787 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:49.787 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:49.787 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:49.787 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:49.787 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:49.787 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 969 2024-05-08T08:54:49.787 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.787 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:49.787 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.787 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:49.787 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:54:49.787 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:49.787 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.787 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.788 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:49.788 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:54:49.788 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:49.788 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.788 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.788 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:49.788 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:54:49.788 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3599999999999999e-07 2024-05-08T08:54:49.788 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.788 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.788 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:49.788 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451177+0000", 2024-05-08T08:54:49.788 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4590000000000001e-06 2024-05-08T08:54:49.788 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.788 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.789 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:49.789 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451184+0000", 2024-05-08T08:54:49.789 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4109999999999999e-06 2024-05-08T08:54:49.789 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.789 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.789 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:49.789 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451191+0000", 2024-05-08T08:54:49.789 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7279999999999998e-06 2024-05-08T08:54:49.789 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.789 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.789 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:49.789 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538997+0000", 2024-05-08T08:54:49.789 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087806170000000003 2024-05-08T08:54:49.789 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.790 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.790 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:49.790 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539069+0000", 2024-05-08T08:54:49.790 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2373999999999995e-05 2024-05-08T08:54:49.790 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.790 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.790 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:49.790 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539173+0000", 2024-05-08T08:54:49.790 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000103402 2024-05-08T08:54:49.790 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.790 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:49.791 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.791 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.791 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.791 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:970 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 3407872~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:49.791 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451198+0000", 2024-05-08T08:54:49.791 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 100.290121672, 2024-05-08T08:54:49.791 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088183368999999998, 2024-05-08T08:54:49.791 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:49.791 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:49.791 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:49.791 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:49.791 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:49.791 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:49.791 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 970 2024-05-08T08:54:49.791 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.791 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:49.792 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.792 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:49.792 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451198+0000", 2024-05-08T08:54:49.792 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:49.792 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.792 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.792 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:49.792 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451198+0000", 2024-05-08T08:54:49.792 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:49.792 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.792 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.792 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:49.792 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451199+0000", 2024-05-08T08:54:49.792 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7400000000000003e-07 2024-05-08T08:54:49.792 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.793 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.793 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:49.793 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451211+0000", 2024-05-08T08:54:49.793 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2109999999999999e-05 2024-05-08T08:54:49.793 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.793 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.793 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:49.793 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451219+0000", 2024-05-08T08:54:49.793 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4170000000000003e-06 2024-05-08T08:54:49.793 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.793 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.793 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:49.793 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451224+0000", 2024-05-08T08:54:49.793 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3990000000000003e-06 2024-05-08T08:54:49.793 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.794 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.794 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:49.794 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539204+0000", 2024-05-08T08:54:49.794 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087980099000000006 2024-05-08T08:54:49.794 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.794 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.794 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:49.794 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539276+0000", 2024-05-08T08:54:49.794 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1513000000000005e-05 2024-05-08T08:54:49.794 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.794 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.794 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:49.794 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539382+0000", 2024-05-08T08:54:49.795 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010615699999999999 2024-05-08T08:54:49.795 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.795 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:49.795 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.795 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.795 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.795 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:971 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 3899392~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:49.795 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451233+0000", 2024-05-08T08:54:49.795 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 100.290086965, 2024-05-08T08:54:49.795 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088431525999999996, 2024-05-08T08:54:49.795 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:49.795 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:49.795 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:49.795 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:49.795 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:49.796 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:49.796 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 971 2024-05-08T08:54:49.796 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.796 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:49.796 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.796 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:49.796 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451233+0000", 2024-05-08T08:54:49.796 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:49.796 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.796 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.796 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:49.796 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451233+0000", 2024-05-08T08:54:49.796 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:49.796 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.797 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.797 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:49.797 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451234+0000", 2024-05-08T08:54:49.797 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8500000000000001e-07 2024-05-08T08:54:49.797 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.797 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.797 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:49.797 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451237+0000", 2024-05-08T08:54:49.797 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8320000000000002e-06 2024-05-08T08:54:49.797 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.797 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.797 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:49.797 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451243+0000", 2024-05-08T08:54:49.797 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6749999999999996e-06 2024-05-08T08:54:49.797 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.797 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.798 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:49.798 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451249+0000", 2024-05-08T08:54:49.798 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1229999999999997e-06 2024-05-08T08:54:49.798 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.798 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.798 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:49.798 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539412+0000", 2024-05-08T08:54:49.798 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088162668999999999 2024-05-08T08:54:49.798 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.798 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.798 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:49.798 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539505+0000", 2024-05-08T08:54:49.798 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3240999999999997e-05 2024-05-08T08:54:49.798 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.799 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.799 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:49.799 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539665+0000", 2024-05-08T08:54:49.799 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00015930100000000001 2024-05-08T08:54:49.799 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.799 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:49.799 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.799 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.799 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.799 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:972 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 196608~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:49.799 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451257+0000", 2024-05-08T08:54:49.799 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 100.29006302000001, 2024-05-08T08:54:49.799 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088719928000000003, 2024-05-08T08:54:49.799 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:49.800 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:49.800 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:49.800 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:49.800 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:49.800 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:49.800 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 972 2024-05-08T08:54:49.800 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.800 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:49.800 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.800 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:49.800 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451257+0000", 2024-05-08T08:54:49.800 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:49.800 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.800 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.800 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:49.801 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451257+0000", 2024-05-08T08:54:49.801 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:49.801 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.801 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.801 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:49.801 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451258+0000", 2024-05-08T08:54:49.801 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7999999999999996e-07 2024-05-08T08:54:49.801 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.801 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.801 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:49.801 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451260+0000", 2024-05-08T08:54:49.801 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4609999999999998e-06 2024-05-08T08:54:49.801 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.801 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.801 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:49.802 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451267+0000", 2024-05-08T08:54:49.802 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.81e-06 2024-05-08T08:54:49.802 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.802 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.802 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:49.802 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451273+0000", 2024-05-08T08:54:49.802 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0859999999999998e-06 2024-05-08T08:54:49.802 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.802 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.802 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:49.802 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539718+0000", 2024-05-08T08:54:49.802 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088445059000000006 2024-05-08T08:54:49.803 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.803 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.803 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:49.803 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539834+0000", 2024-05-08T08:54:49.803 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000116426 2024-05-08T08:54:49.803 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.803 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.803 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:49.803 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539977+0000", 2024-05-08T08:54:49.803 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014260600000000001 2024-05-08T08:54:49.803 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.803 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:49.803 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.803 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.803 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.804 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:973 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 688128~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:49.804 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:54:49.804 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 100.290040436, 2024-05-08T08:54:49.804 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088918496, 2024-05-08T08:54:49.804 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:49.804 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:49.804 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:49.804 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:49.804 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:49.804 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:49.804 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 973 2024-05-08T08:54:49.804 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.804 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:49.804 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.804 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:49.805 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:54:49.805 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:49.805 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.805 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.805 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:49.805 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:54:49.805 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:49.805 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.805 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.805 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:49.805 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:54:49.805 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9099999999999999e-07 2024-05-08T08:54:49.805 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.805 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.805 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:49.806 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451282+0000", 2024-05-08T08:54:49.806 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1550000000000001e-06 2024-05-08T08:54:49.806 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.806 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.806 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:49.806 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451288+0000", 2024-05-08T08:54:49.806 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8889999999999999e-06 2024-05-08T08:54:49.806 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.806 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.806 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:49.806 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451294+0000", 2024-05-08T08:54:49.806 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0850000000000002e-06 2024-05-08T08:54:49.806 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.806 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.807 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:49.807 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540011+0000", 2024-05-08T08:54:49.807 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088717314000000005 2024-05-08T08:54:49.807 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.807 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.807 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:49.807 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540089+0000", 2024-05-08T08:54:49.807 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7438e-05 2024-05-08T08:54:49.807 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.807 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.807 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:49.807 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540198+0000", 2024-05-08T08:54:49.807 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000109224 2024-05-08T08:54:49.807 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.807 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:49.807 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.808 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.808 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.808 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:974 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 1179648~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:49.808 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451301+0000", 2024-05-08T08:54:49.808 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 100.290018803, 2024-05-08T08:54:49.808 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089105078000000004, 2024-05-08T08:54:49.808 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:49.808 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:49.808 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:49.808 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:49.808 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:49.808 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:49.808 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 974 2024-05-08T08:54:49.808 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.809 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:49.809 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.809 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:49.809 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451301+0000", 2024-05-08T08:54:49.809 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:49.809 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.809 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.809 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:49.809 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451301+0000", 2024-05-08T08:54:49.809 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:49.809 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.809 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.809 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:49.809 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451302+0000", 2024-05-08T08:54:49.809 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7800000000000002e-07 2024-05-08T08:54:49.809 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.810 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.810 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:49.810 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451304+0000", 2024-05-08T08:54:49.810 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.65e-06 2024-05-08T08:54:49.810 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.810 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.810 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:49.810 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451311+0000", 2024-05-08T08:54:49.810 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.472e-06 2024-05-08T08:54:49.810 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.810 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.810 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:49.810 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451316+0000", 2024-05-08T08:54:49.810 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9189999999999998e-06 2024-05-08T08:54:49.810 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.811 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.811 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:49.811 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540234+0000", 2024-05-08T08:54:49.811 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088917752000000003 2024-05-08T08:54:49.811 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.811 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.811 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:49.811 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540308+0000", 2024-05-08T08:54:49.811 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4758000000000005e-05 2024-05-08T08:54:49.811 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.811 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.811 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:49.811 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540406+0000", 2024-05-08T08:54:49.811 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8048999999999996e-05 2024-05-08T08:54:49.811 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.812 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:49.812 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.812 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.812 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.812 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:975 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 1671168~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:49.812 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:54:49.812 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 100.289976473, 2024-05-08T08:54:49.812 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089294139999999994, 2024-05-08T08:54:49.812 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:49.812 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:49.812 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:49.812 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:49.812 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:49.812 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:49.813 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 975 2024-05-08T08:54:49.813 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.813 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:49.813 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.813 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:49.813 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:54:49.813 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:49.813 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.813 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.813 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:49.813 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:54:49.813 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:49.813 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.813 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.814 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:49.814 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:54:49.814 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1299999999999999e-07 2024-05-08T08:54:49.814 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.814 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.814 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:49.814 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451348+0000", 2024-05-08T08:54:49.814 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.484e-06 2024-05-08T08:54:49.814 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.814 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.814 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:49.814 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451356+0000", 2024-05-08T08:54:49.814 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8499999999999994e-06 2024-05-08T08:54:49.814 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.815 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.815 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:49.815 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451366+0000", 2024-05-08T08:54:49.815 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.9850000000000001e-06 2024-05-08T08:54:49.815 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.815 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.815 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:49.815 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540448+0000", 2024-05-08T08:54:49.815 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089082254 2024-05-08T08:54:49.815 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.815 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.815 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:49.815 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540528+0000", 2024-05-08T08:54:49.815 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0234999999999993e-05 2024-05-08T08:54:49.815 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.816 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.816 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:49.816 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540638+0000", 2024-05-08T08:54:49.816 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010951900000000001 2024-05-08T08:54:49.816 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.816 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:49.816 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.816 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.816 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.816 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:976 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 2162688~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:49.816 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451375+0000", 2024-05-08T08:54:49.816 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 100.289944726, 2024-05-08T08:54:49.816 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089472745000000006, 2024-05-08T08:54:49.816 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:49.817 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:49.817 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:49.817 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:49.817 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:49.817 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:49.817 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 976 2024-05-08T08:54:49.817 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.817 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:49.817 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.817 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:49.817 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451375+0000", 2024-05-08T08:54:49.817 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:49.817 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.818 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.818 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:49.818 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451375+0000", 2024-05-08T08:54:49.818 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:49.818 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.818 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.818 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:49.818 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451376+0000", 2024-05-08T08:54:49.818 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2399999999999998e-07 2024-05-08T08:54:49.818 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.818 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.818 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:49.818 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451379+0000", 2024-05-08T08:54:49.818 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6359999999999998e-06 2024-05-08T08:54:49.818 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.819 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.819 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:49.819 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451387+0000", 2024-05-08T08:54:49.819 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9929999999999995e-06 2024-05-08T08:54:49.819 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.819 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.819 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:49.819 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451393+0000", 2024-05-08T08:54:49.819 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4699999999999999e-06 2024-05-08T08:54:49.819 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.819 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.819 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:49.819 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540677+0000", 2024-05-08T08:54:49.819 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089283504999999999 2024-05-08T08:54:49.819 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.819 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.820 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:49.820 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540743+0000", 2024-05-08T08:54:49.820 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6303999999999994e-05 2024-05-08T08:54:49.820 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.820 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.820 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:49.820 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540848+0000", 2024-05-08T08:54:49.820 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000105213 2024-05-08T08:54:49.820 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.820 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:49.820 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.820 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.820 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.820 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:977 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 2654208~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:49.820 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:54:49.820 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 100.289919294, 2024-05-08T08:54:49.821 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089698473000000001, 2024-05-08T08:54:49.821 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:49.821 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:49.821 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:49.821 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:49.821 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:49.821 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:49.821 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 977 2024-05-08T08:54:49.821 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.821 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:49.821 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.821 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:49.821 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:54:49.821 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:49.821 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.821 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.822 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:49.822 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:54:49.822 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:49.822 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.822 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.822 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:49.822 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:54:49.822 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2899999999999999e-07 2024-05-08T08:54:49.822 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.822 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.822 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:49.822 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451404+0000", 2024-05-08T08:54:49.822 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6649999999999999e-06 2024-05-08T08:54:49.822 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.823 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.823 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:49.823 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451410+0000", 2024-05-08T08:54:49.823 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3269999999999997e-06 2024-05-08T08:54:49.823 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.823 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.823 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:49.823 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451415+0000", 2024-05-08T08:54:49.823 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9529999999999998e-06 2024-05-08T08:54:49.823 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.823 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.823 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:49.823 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540912+0000", 2024-05-08T08:54:49.823 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089496950000000006 2024-05-08T08:54:49.823 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.823 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.824 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:49.824 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541000+0000", 2024-05-08T08:54:49.824 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7429000000000006e-05 2024-05-08T08:54:49.824 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.824 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.824 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:49.824 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541099+0000", 2024-05-08T08:54:49.824 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.9720000000000001e-05 2024-05-08T08:54:49.824 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.824 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:49.825 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.825 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.825 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.825 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:978 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 3145728~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:49.825 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:54:49.825 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 100.28989808599999, 2024-05-08T08:54:49.825 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089900635000000007, 2024-05-08T08:54:49.825 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:49.825 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:49.825 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:49.825 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:49.825 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:49.825 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:49.825 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 978 2024-05-08T08:54:49.825 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.825 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:49.826 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.826 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:49.826 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:54:49.826 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:49.826 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.826 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.826 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:49.826 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:54:49.826 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:49.826 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.826 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.826 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:49.826 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:54:49.826 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9400000000000001e-07 2024-05-08T08:54:49.826 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.827 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.827 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:49.827 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451425+0000", 2024-05-08T08:54:49.827 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2460000000000002e-06 2024-05-08T08:54:49.827 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.827 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.827 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:49.827 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451431+0000", 2024-05-08T08:54:49.827 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2419999999999997e-06 2024-05-08T08:54:49.827 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.827 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.827 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:49.827 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451436+0000", 2024-05-08T08:54:49.827 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.887e-06 2024-05-08T08:54:49.827 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.828 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.828 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:49.828 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541152+0000", 2024-05-08T08:54:49.828 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089716194999999999 2024-05-08T08:54:49.828 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.828 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.828 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:49.828 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541224+0000", 2024-05-08T08:54:49.828 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2416000000000006e-05 2024-05-08T08:54:49.828 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.828 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.828 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:49.828 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541323+0000", 2024-05-08T08:54:49.828 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8255e-05 2024-05-08T08:54:49.828 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.829 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:49.829 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.829 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.829 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.829 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:979 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 3637248~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:49.829 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451462+0000", 2024-05-08T08:54:49.829 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 100.289857637, 2024-05-08T08:54:49.829 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.090036441999999994, 2024-05-08T08:54:49.829 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:49.829 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:49.829 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:49.829 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:49.829 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:49.829 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:49.829 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 979 2024-05-08T08:54:49.830 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.830 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:49.830 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.830 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:49.830 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451462+0000", 2024-05-08T08:54:49.830 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:49.830 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.830 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.830 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:49.830 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451462+0000", 2024-05-08T08:54:49.830 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:49.830 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.830 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.830 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:49.830 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451463+0000", 2024-05-08T08:54:49.831 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4300000000000002e-07 2024-05-08T08:54:49.831 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.831 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.831 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:49.831 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451471+0000", 2024-05-08T08:54:49.831 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0800000000000006e-06 2024-05-08T08:54:49.831 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.831 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.831 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:49.831 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451481+0000", 2024-05-08T08:54:49.831 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.38e-06 2024-05-08T08:54:49.831 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.831 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.832 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:49.832 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451487+0000", 2024-05-08T08:54:49.832 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4590000000000003e-06 2024-05-08T08:54:49.832 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.832 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.832 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:49.832 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541350+0000", 2024-05-08T08:54:49.832 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089862599000000001 2024-05-08T08:54:49.832 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.832 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.832 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:49.832 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541407+0000", 2024-05-08T08:54:49.832 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7667999999999997e-05 2024-05-08T08:54:49.832 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.833 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.833 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:49.833 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541499+0000", 2024-05-08T08:54:49.833 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1513000000000003e-05 2024-05-08T08:54:49.833 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.833 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:49.833 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.833 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.833 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.833 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:980 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 4128768~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:49.833 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451496+0000", 2024-05-08T08:54:49.833 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 100.289823881, 2024-05-08T08:54:49.833 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.090258985, 2024-05-08T08:54:49.833 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:49.833 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:49.833 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:49.834 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:49.834 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:49.834 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:49.834 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 980 2024-05-08T08:54:49.834 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.834 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:49.834 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.834 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:49.834 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451496+0000", 2024-05-08T08:54:49.834 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:49.834 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.834 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.834 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:49.834 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451496+0000", 2024-05-08T08:54:49.835 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:49.835 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.835 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.835 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:49.835 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451497+0000", 2024-05-08T08:54:49.835 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7599999999999997e-07 2024-05-08T08:54:49.835 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.835 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.835 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:49.835 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451500+0000", 2024-05-08T08:54:49.835 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7930000000000002e-06 2024-05-08T08:54:49.835 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.835 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.836 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:49.836 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451506+0000", 2024-05-08T08:54:49.836 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6980000000000001e-06 2024-05-08T08:54:49.836 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.836 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.836 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:49.836 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451512+0000", 2024-05-08T08:54:49.836 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2490000000000001e-06 2024-05-08T08:54:49.836 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.836 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.836 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:49.836 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541550+0000", 2024-05-08T08:54:49.836 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.090037961999999999 2024-05-08T08:54:49.836 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.836 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.837 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:49.837 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541647+0000", 2024-05-08T08:54:49.837 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6924999999999998e-05 2024-05-08T08:54:49.837 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:49.837 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:49.837 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:49.837 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541755+0000", 2024-05-08T08:54:49.837 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000108682 2024-05-08T08:54:49.837 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.838 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:49.838 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.838 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:49.838 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:49.838 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:49.839 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:54:49.840 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:49.842+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:49.940 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:49.943+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:49.946 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:49.946 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:54:49.946 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:54:49.946 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:49.955 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:54:50.042 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:50.045+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:50.102 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:50.102 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:54:50.102 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:54:50.102 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:54:50.102 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:50.110 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:54:50.143 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:50.146+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:50.244 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:50.247+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:50.257 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:50.257 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:54:50.257 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:54:50.257 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:54:50.258 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.258 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4621.0:9089 3.1f e33/18 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:head v 33'117, mlcod=31'115)", 2024-05-08T08:54:50.258 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:54:50.258 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 137.73975722599999, 2024-05-08T08:54:50.258 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.111244524, 2024-05-08T08:54:50.258 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:50.258 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:50.258 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:54:50.258 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:50.258 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.258 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:50.258 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:54:50.258 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:50.258 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.258 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.259 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:50.259 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:54:50.259 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:50.259 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.259 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.259 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:50.259 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:54:50.259 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5900000000000001e-07 2024-05-08T08:54:50.259 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.259 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.259 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:50.259 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539600+0000", 2024-05-08T08:54:50.259 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.019981364000000001 2024-05-08T08:54:50.259 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.260 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.260 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:50.260 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539624+0000", 2024-05-08T08:54:50.260 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4043e-05 2024-05-08T08:54:50.260 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.260 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.260 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:50.260 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539635+0000", 2024-05-08T08:54:50.260 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1511e-05 2024-05-08T08:54:50.260 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.260 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.260 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:50.260 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539728+0000", 2024-05-08T08:54:50.260 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2498999999999996e-05 2024-05-08T08:54:50.260 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.260 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.261 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:50.261 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539746+0000", 2024-05-08T08:54:50.261 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8289000000000001e-05 2024-05-08T08:54:50.261 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.261 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.261 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:54:50.261 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.630822+0000", 2024-05-08T08:54:50.261 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.091076283999999993 2024-05-08T08:54:50.261 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.261 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.261 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:50.261 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.630863+0000", 2024-05-08T08:54:50.261 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0374999999999999e-05 2024-05-08T08:54:50.261 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:50.261 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:50.262 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:50.262 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.262 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.262 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4195.0:4 3.11 3:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-05-08T08:54:50.262 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:49.764654+0000", 2024-05-08T08:54:50.262 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 120.494721685, 2024-05-08T08:54:50.262 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.30714275499999999, 2024-05-08T08:54:50.262 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:50.262 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:50.262 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-05-08T08:54:50.262 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:50.262 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4195", 2024-05-08T08:54:50.262 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/59179521", 2024-05-08T08:54:50.262 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4 2024-05-08T08:54:50.262 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.263 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:50.263 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.263 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:50.263 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764654+0000", 2024-05-08T08:54:50.263 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:50.263 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.263 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.263 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:50.263 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764654+0000", 2024-05-08T08:54:50.263 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:50.263 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.264 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.264 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:50.264 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764655+0000", 2024-05-08T08:54:50.264 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.452e-06 2024-05-08T08:54:50.264 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.264 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.264 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:50.264 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764661+0000", 2024-05-08T08:54:50.264 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5369999999999997e-06 2024-05-08T08:54:50.264 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.264 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.264 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:50.264 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764671+0000", 2024-05-08T08:54:50.264 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.028e-05 2024-05-08T08:54:50.264 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.265 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.265 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:50.265 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764680+0000", 2024-05-08T08:54:50.265 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0629999999999995e-06 2024-05-08T08:54:50.265 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.265 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.265 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:50.265 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764753+0000", 2024-05-08T08:54:50.265 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2679e-05 2024-05-08T08:54:50.265 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.265 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.265 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-05-08T08:54:50.265 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764765+0000", 2024-05-08T08:54:50.265 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2415000000000001e-05 2024-05-08T08:54:50.265 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.266 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.266 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:50.266 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:50.071692+0000", 2024-05-08T08:54:50.266 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.306927324 2024-05-08T08:54:50.266 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.266 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.266 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:50.266 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:50.071796+0000", 2024-05-08T08:54:50.266 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104005 2024-05-08T08:54:50.266 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:50.266 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:50.266 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:50.266 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.266 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.267 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4416.0:20531 3.16 3:6bb2ed16:::rbd_object_map.1140bc4a55c7:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-05-08T08:54:50.267 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:58.673786+0000", 2024-05-08T08:54:50.267 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 51.585589280999997, 2024-05-08T08:54:50.267 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.94684119899999997, 2024-05-08T08:54:50.267 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:50.267 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:50.267 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:54:50.267 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:50.267 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4416", 2024-05-08T08:54:50.267 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:54:50.267 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20531 2024-05-08T08:54:50.267 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.267 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:50.268 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.268 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:50.268 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673786+0000", 2024-05-08T08:54:50.268 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:50.268 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.268 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.268 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:50.268 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673786+0000", 2024-05-08T08:54:50.268 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:50.268 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.268 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.268 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:50.268 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673787+0000", 2024-05-08T08:54:50.268 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9800000000000003e-07 2024-05-08T08:54:50.269 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.269 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.269 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:50.269 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673791+0000", 2024-05-08T08:54:50.269 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.104e-06 2024-05-08T08:54:50.269 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.269 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.269 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:50.269 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673801+0000", 2024-05-08T08:54:50.269 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9960000000000005e-06 2024-05-08T08:54:50.269 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.269 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.269 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:50.269 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673806+0000", 2024-05-08T08:54:50.269 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2320000000000001e-06 2024-05-08T08:54:50.270 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.270 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.270 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:50.270 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673860+0000", 2024-05-08T08:54:50.270 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3468999999999999e-05 2024-05-08T08:54:50.270 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.270 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.270 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-05-08T08:54:50.270 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673866+0000", 2024-05-08T08:54:50.270 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3500000000000002e-06 2024-05-08T08:54:50.270 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.270 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.270 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:50.270 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.616700+0000", 2024-05-08T08:54:50.270 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.94283390700000003 2024-05-08T08:54:50.271 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.271 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.271 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:50.271 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.616941+0000", 2024-05-08T08:54:50.271 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00024080400000000001 2024-05-08T08:54:50.271 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.271 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.271 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-05-08T08:54:50.271 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.617072+0000", 2024-05-08T08:54:50.271 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000131688 2024-05-08T08:54:50.271 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.271 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.271 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:54:50.271 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.619571+0000", 2024-05-08T08:54:50.272 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0024990149999999998 2024-05-08T08:54:50.272 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.272 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.272 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:54:50.272 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.620568+0000", 2024-05-08T08:54:50.272 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00099711000000000005 2024-05-08T08:54:50.272 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.272 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.272 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:54:50.272 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.620602+0000", 2024-05-08T08:54:50.272 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.324e-05 2024-05-08T08:54:50.272 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.272 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.272 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:50.273 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.620627+0000", 2024-05-08T08:54:50.273 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5585999999999999e-05 2024-05-08T08:54:50.273 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:50.273 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:50.273 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:50.273 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.273 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.273 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5490 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4227072~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:50.273 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.881809+0000", 2024-05-08T08:54:50.273 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 24.377566769000001, 2024-05-08T08:54:50.273 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047460447000000003, 2024-05-08T08:54:50.273 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:50.273 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:50.273 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:50.273 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:50.274 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:50.274 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:50.274 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5490 2024-05-08T08:54:50.274 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.274 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:50.274 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.274 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:50.274 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881809+0000", 2024-05-08T08:54:50.274 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:50.274 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.274 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.274 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:50.274 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881809+0000", 2024-05-08T08:54:50.274 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:50.275 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.275 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.275 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:50.275 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881809+0000", 2024-05-08T08:54:50.275 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7899999999999999e-07 2024-05-08T08:54:50.275 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.275 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.275 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:50.275 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881811+0000", 2024-05-08T08:54:50.275 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0339999999999999e-06 2024-05-08T08:54:50.275 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.275 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.275 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:50.275 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881818+0000", 2024-05-08T08:54:50.275 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5760000000000002e-06 2024-05-08T08:54:50.276 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.276 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.276 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:50.276 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881823+0000", 2024-05-08T08:54:50.276 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4160000000000003e-06 2024-05-08T08:54:50.276 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.276 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.276 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:50.276 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929157+0000", 2024-05-08T08:54:50.276 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047334029999999999 2024-05-08T08:54:50.276 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.276 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.276 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:50.276 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929211+0000", 2024-05-08T08:54:50.276 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3560999999999998e-05 2024-05-08T08:54:50.277 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.277 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.277 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:50.277 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929269+0000", 2024-05-08T08:54:50.277 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8350999999999997e-05 2024-05-08T08:54:50.277 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:50.277 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:50.277 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:50.277 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.277 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.277 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5492 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4272128~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:50.277 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.881830+0000", 2024-05-08T08:54:50.277 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 24.377545475000002, 2024-05-08T08:54:50.277 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047614006, 2024-05-08T08:54:50.277 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:50.278 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:50.278 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:50.278 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:50.278 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:50.278 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:50.278 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5492 2024-05-08T08:54:50.278 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.278 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:50.278 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.278 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:50.278 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881830+0000", 2024-05-08T08:54:50.278 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:50.278 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.278 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.279 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:50.279 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881830+0000", 2024-05-08T08:54:50.279 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:50.279 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.279 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.279 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:50.279 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881830+0000", 2024-05-08T08:54:50.279 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4099999999999999e-07 2024-05-08T08:54:50.279 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.279 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.279 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:50.279 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881832+0000", 2024-05-08T08:54:50.279 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0370000000000001e-06 2024-05-08T08:54:50.279 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.279 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.280 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:50.280 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881839+0000", 2024-05-08T08:54:50.280 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3289999999999999e-06 2024-05-08T08:54:50.280 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.280 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.280 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:50.280 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881844+0000", 2024-05-08T08:54:50.280 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1170000000000003e-06 2024-05-08T08:54:50.280 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.280 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.280 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:50.280 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929296+0000", 2024-05-08T08:54:50.281 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047452418000000003 2024-05-08T08:54:50.281 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.281 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.281 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:50.281 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929360+0000", 2024-05-08T08:54:50.281 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4197999999999998e-05 2024-05-08T08:54:50.281 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.281 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.281 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:50.281 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929444+0000", 2024-05-08T08:54:50.281 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3466000000000004e-05 2024-05-08T08:54:50.281 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:50.281 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:50.281 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:50.281 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.282 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.282 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5494 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4317184~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:50.282 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.881850+0000", 2024-05-08T08:54:50.282 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 24.377525726000002, 2024-05-08T08:54:50.282 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047723887, 2024-05-08T08:54:50.282 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:50.282 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:50.282 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:50.282 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:50.282 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:50.282 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:50.282 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5494 2024-05-08T08:54:50.282 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.282 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:50.282 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.283 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:50.283 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881850+0000", 2024-05-08T08:54:50.283 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:50.283 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.283 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.283 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:50.283 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881850+0000", 2024-05-08T08:54:50.283 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:50.283 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.283 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.283 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:50.283 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881850+0000", 2024-05-08T08:54:50.283 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8500000000000002e-07 2024-05-08T08:54:50.283 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.283 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.284 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:50.284 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881856+0000", 2024-05-08T08:54:50.284 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9510000000000003e-06 2024-05-08T08:54:50.284 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.284 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.284 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:50.284 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881862+0000", 2024-05-08T08:54:50.284 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3049999999999998e-06 2024-05-08T08:54:50.284 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.284 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.284 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:50.284 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881868+0000", 2024-05-08T08:54:50.284 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8340000000000004e-06 2024-05-08T08:54:50.284 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.284 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.285 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:50.285 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929475+0000", 2024-05-08T08:54:50.285 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047606796 2024-05-08T08:54:50.285 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.285 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.285 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:50.285 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929518+0000", 2024-05-08T08:54:50.285 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3387000000000003e-05 2024-05-08T08:54:50.285 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.285 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.285 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:50.285 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929574+0000", 2024-05-08T08:54:50.285 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5129e-05 2024-05-08T08:54:50.285 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:50.286 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:50.286 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:50.286 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.286 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.286 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5495 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4362240~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:50.286 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.881875+0000", 2024-05-08T08:54:50.286 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 24.377500859000001, 2024-05-08T08:54:50.286 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047847967999999998, 2024-05-08T08:54:50.286 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:50.286 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:50.286 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:50.287 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:50.287 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:50.287 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:50.287 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5495 2024-05-08T08:54:50.287 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.287 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:50.287 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.287 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:50.287 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881875+0000", 2024-05-08T08:54:50.287 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:50.287 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.287 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.287 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:50.287 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881875+0000", 2024-05-08T08:54:50.287 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:50.288 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.288 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.288 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:50.288 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881875+0000", 2024-05-08T08:54:50.288 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-07 2024-05-08T08:54:50.288 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.288 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.288 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:50.288 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881877+0000", 2024-05-08T08:54:50.288 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9659999999999999e-06 2024-05-08T08:54:50.288 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.288 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.288 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:50.288 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881883+0000", 2024-05-08T08:54:50.288 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7649999999999999e-06 2024-05-08T08:54:50.289 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.289 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.289 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:50.289 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881887+0000", 2024-05-08T08:54:50.289 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6689999999999997e-06 2024-05-08T08:54:50.289 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.289 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.289 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:50.289 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929600+0000", 2024-05-08T08:54:50.289 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047712746 2024-05-08T08:54:50.289 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.289 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.289 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:50.289 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929659+0000", 2024-05-08T08:54:50.289 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8936e-05 2024-05-08T08:54:50.290 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.290 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.290 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:50.290 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929723+0000", 2024-05-08T08:54:50.290 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3524999999999999e-05 2024-05-08T08:54:50.290 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:50.290 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:50.290 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:50.290 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.290 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.290 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5496 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4407296~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:50.290 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.881894+0000", 2024-05-08T08:54:50.290 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 24.377480994999999, 2024-05-08T08:54:50.291 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047998015999999998, 2024-05-08T08:54:50.291 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:50.291 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:50.291 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:50.291 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:50.291 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:50.291 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:50.291 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5496 2024-05-08T08:54:50.291 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.291 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:50.291 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.291 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:50.291 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881894+0000", 2024-05-08T08:54:50.291 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:50.291 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.292 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.292 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:50.292 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881894+0000", 2024-05-08T08:54:50.292 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:50.292 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.292 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.292 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:50.292 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881895+0000", 2024-05-08T08:54:50.292 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6699999999999999e-07 2024-05-08T08:54:50.292 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.292 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.292 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:50.292 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881897+0000", 2024-05-08T08:54:50.292 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0229999999999999e-06 2024-05-08T08:54:50.293 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.293 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.293 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:50.293 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881903+0000", 2024-05-08T08:54:50.293 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9850000000000003e-06 2024-05-08T08:54:50.293 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.293 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.293 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:50.293 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881908+0000", 2024-05-08T08:54:50.293 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9019999999999998e-06 2024-05-08T08:54:50.293 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.293 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.293 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:50.293 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929751+0000", 2024-05-08T08:54:50.293 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047843550999999998 2024-05-08T08:54:50.294 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.294 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.294 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:50.294 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929823+0000", 2024-05-08T08:54:50.294 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1172000000000002e-05 2024-05-08T08:54:50.294 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.294 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.294 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:50.294 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929892+0000", 2024-05-08T08:54:50.294 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9915999999999999e-05 2024-05-08T08:54:50.294 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:50.295 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:50.295 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:50.295 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.295 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.295 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5498 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4452352~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:50.295 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.881914+0000", 2024-05-08T08:54:50.295 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 24.377461236999999, 2024-05-08T08:54:50.295 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048203553000000003, 2024-05-08T08:54:50.295 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:50.295 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:50.295 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:50.295 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:50.295 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:50.295 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:50.295 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5498 2024-05-08T08:54:50.296 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.296 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:50.296 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.296 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:50.296 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881914+0000", 2024-05-08T08:54:50.296 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:50.296 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.296 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.296 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:50.296 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881914+0000", 2024-05-08T08:54:50.296 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:50.296 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.296 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.296 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:50.296 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881915+0000", 2024-05-08T08:54:50.297 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8500000000000002e-07 2024-05-08T08:54:50.297 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.297 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.297 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:50.297 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881917+0000", 2024-05-08T08:54:50.297 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.181e-06 2024-05-08T08:54:50.297 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.297 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.297 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:50.297 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881924+0000", 2024-05-08T08:54:50.297 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9619999999999996e-06 2024-05-08T08:54:50.297 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.297 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.297 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:50.297 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881929+0000", 2024-05-08T08:54:50.297 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1399999999999999e-06 2024-05-08T08:54:50.298 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.298 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.298 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:50.298 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929952+0000", 2024-05-08T08:54:50.298 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048023134000000002 2024-05-08T08:54:50.298 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.298 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.298 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:50.298 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.930029+0000", 2024-05-08T08:54:50.298 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7490000000000005e-05 2024-05-08T08:54:50.298 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.298 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.298 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:50.298 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.930118+0000", 2024-05-08T08:54:50.299 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8261000000000001e-05 2024-05-08T08:54:50.299 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:50.299 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:50.299 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:50.299 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.299 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.299 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5563 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4321280~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:50.299 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.884201+0000", 2024-05-08T08:54:50.299 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 24.375173987, 2024-05-08T08:54:50.299 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047379969000000001, 2024-05-08T08:54:50.299 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:50.299 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:50.299 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:50.299 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:50.300 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:50.300 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:50.300 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5563 2024-05-08T08:54:50.300 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.300 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:50.300 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.300 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:50.300 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884201+0000", 2024-05-08T08:54:50.300 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:50.300 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.300 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.300 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:50.300 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884201+0000", 2024-05-08T08:54:50.300 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:50.300 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.301 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.301 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:50.301 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884202+0000", 2024-05-08T08:54:50.301 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8000000000000002e-07 2024-05-08T08:54:50.301 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.301 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.301 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:50.301 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884208+0000", 2024-05-08T08:54:50.301 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.674e-06 2024-05-08T08:54:50.301 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.301 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.301 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:50.301 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884215+0000", 2024-05-08T08:54:50.301 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0389999999999999e-06 2024-05-08T08:54:50.301 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.302 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.302 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:50.302 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884220+0000", 2024-05-08T08:54:50.302 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4950000000000002e-06 2024-05-08T08:54:50.302 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.302 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.302 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:50.302 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931417+0000", 2024-05-08T08:54:50.302 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047196674000000001 2024-05-08T08:54:50.302 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.302 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.302 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:50.302 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931493+0000", 2024-05-08T08:54:50.302 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5950000000000003e-05 2024-05-08T08:54:50.302 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.303 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.303 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:50.303 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931581+0000", 2024-05-08T08:54:50.303 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8857000000000007e-05 2024-05-08T08:54:50.303 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:50.303 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:50.303 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:50.303 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.303 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.303 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5564 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4366336~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:50.303 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.884225+0000", 2024-05-08T08:54:50.303 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 24.375150446999999, 2024-05-08T08:54:50.303 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047555989999999999, 2024-05-08T08:54:50.303 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:50.304 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:50.304 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:50.304 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:50.304 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:50.304 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:50.304 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5564 2024-05-08T08:54:50.304 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.304 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:50.304 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.304 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:50.304 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884225+0000", 2024-05-08T08:54:50.304 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:50.304 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.304 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.304 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:50.305 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884225+0000", 2024-05-08T08:54:50.305 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:50.305 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.305 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.305 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:50.305 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884225+0000", 2024-05-08T08:54:50.305 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2499999999999995e-07 2024-05-08T08:54:50.305 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.305 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.305 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:50.305 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884227+0000", 2024-05-08T08:54:50.305 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5829999999999999e-06 2024-05-08T08:54:50.305 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.305 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.305 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:50.305 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884232+0000", 2024-05-08T08:54:50.306 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1159999999999998e-06 2024-05-08T08:54:50.306 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.306 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.306 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:50.306 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884236+0000", 2024-05-08T08:54:50.306 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2490000000000003e-06 2024-05-08T08:54:50.306 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.306 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.306 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:50.306 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931617+0000", 2024-05-08T08:54:50.306 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047380310000000002 2024-05-08T08:54:50.306 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.306 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.306 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:50.306 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931689+0000", 2024-05-08T08:54:50.307 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2358000000000001e-05 2024-05-08T08:54:50.307 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.307 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.307 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:50.307 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931781+0000", 2024-05-08T08:54:50.307 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1848999999999994e-05 2024-05-08T08:54:50.307 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:50.307 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:50.307 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:50.307 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.307 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.307 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5565 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4411392~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:50.307 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.884251+0000", 2024-05-08T08:54:50.307 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 24.375124293999999, 2024-05-08T08:54:50.307 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047743671000000001, 2024-05-08T08:54:50.307 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:50.308 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:50.308 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:50.308 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:50.308 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:50.308 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:50.308 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5565 2024-05-08T08:54:50.308 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.308 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:50.308 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.308 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:50.308 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884251+0000", 2024-05-08T08:54:50.308 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:50.308 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.308 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.308 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:50.309 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884251+0000", 2024-05-08T08:54:50.309 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:50.309 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.309 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.309 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:50.309 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884252+0000", 2024-05-08T08:54:50.309 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3200000000000005e-07 2024-05-08T08:54:50.309 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.309 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.309 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:50.309 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884253+0000", 2024-05-08T08:54:50.309 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6980000000000001e-06 2024-05-08T08:54:50.310 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.310 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.310 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:50.310 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884259+0000", 2024-05-08T08:54:50.310 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3759999999999999e-06 2024-05-08T08:54:50.310 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.310 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.310 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:50.310 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884263+0000", 2024-05-08T08:54:50.310 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7280000000000003e-06 2024-05-08T08:54:50.310 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.311 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.311 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:50.311 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931818+0000", 2024-05-08T08:54:50.311 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047554368999999999 2024-05-08T08:54:50.311 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.311 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.311 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:50.311 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931897+0000", 2024-05-08T08:54:50.311 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8821999999999999e-05 2024-05-08T08:54:50.311 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.311 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.311 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:50.311 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931995+0000", 2024-05-08T08:54:50.311 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8146000000000005e-05 2024-05-08T08:54:50.311 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:50.312 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:50.312 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:50.312 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.312 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.312 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5600 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 3948544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:50.312 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.884619+0000", 2024-05-08T08:54:50.312 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 24.374756039000001, 2024-05-08T08:54:50.312 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047574067999999997, 2024-05-08T08:54:50.312 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:50.312 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:50.312 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:50.312 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:50.312 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:50.312 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:50.312 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5600 2024-05-08T08:54:50.312 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.313 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:50.313 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.313 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:50.313 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884619+0000", 2024-05-08T08:54:50.313 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:50.313 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.313 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.313 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:50.313 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884619+0000", 2024-05-08T08:54:50.313 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:50.313 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.313 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.313 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:50.313 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884620+0000", 2024-05-08T08:54:50.313 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.53e-07 2024-05-08T08:54:50.313 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.314 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.314 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:50.314 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884622+0000", 2024-05-08T08:54:50.314 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.796e-06 2024-05-08T08:54:50.314 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.314 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.314 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:50.314 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884627+0000", 2024-05-08T08:54:50.314 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2850000000000002e-06 2024-05-08T08:54:50.314 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.314 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.314 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:50.314 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884631+0000", 2024-05-08T08:54:50.314 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2899999999999996e-06 2024-05-08T08:54:50.314 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.314 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.315 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:50.315 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932024+0000", 2024-05-08T08:54:50.315 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047392680999999999 2024-05-08T08:54:50.315 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.315 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.315 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:50.315 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932102+0000", 2024-05-08T08:54:50.315 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7806999999999996e-05 2024-05-08T08:54:50.315 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.315 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.315 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:50.315 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932193+0000", 2024-05-08T08:54:50.315 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1855999999999996e-05 2024-05-08T08:54:50.315 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:50.315 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:50.315 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:50.315 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.316 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.316 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5603 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 3993600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:50.316 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.884680+0000", 2024-05-08T08:54:50.316 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 24.374695429999999, 2024-05-08T08:54:50.316 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047745034999999998, 2024-05-08T08:54:50.316 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:50.316 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:50.316 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:50.316 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:50.316 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:50.316 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:50.316 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5603 2024-05-08T08:54:50.316 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.316 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:50.316 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.317 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:50.317 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884680+0000", 2024-05-08T08:54:50.317 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:50.317 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.317 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.317 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:50.317 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884680+0000", 2024-05-08T08:54:50.317 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:50.317 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.317 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.317 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:50.317 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884680+0000", 2024-05-08T08:54:50.317 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4400000000000001e-07 2024-05-08T08:54:50.317 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.317 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.317 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:50.317 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884682+0000", 2024-05-08T08:54:50.318 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.514e-06 2024-05-08T08:54:50.318 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.318 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.318 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:50.318 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884687+0000", 2024-05-08T08:54:50.318 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0359999999999997e-06 2024-05-08T08:54:50.318 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.318 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.318 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:50.318 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884692+0000", 2024-05-08T08:54:50.318 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5990000000000004e-06 2024-05-08T08:54:50.318 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.318 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.318 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:50.318 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932235+0000", 2024-05-08T08:54:50.318 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047543635000000001 2024-05-08T08:54:50.319 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.319 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.319 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:50.319 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932323+0000", 2024-05-08T08:54:50.319 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7311000000000005e-05 2024-05-08T08:54:50.319 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.319 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.319 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:50.319 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932425+0000", 2024-05-08T08:54:50.319 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102496 2024-05-08T08:54:50.319 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:50.319 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:50.319 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:50.319 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.319 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.319 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5606 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4038656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:50.319 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.884729+0000", 2024-05-08T08:54:50.320 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 24.374646142, 2024-05-08T08:54:50.320 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047916488, 2024-05-08T08:54:50.320 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:50.320 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:50.320 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:50.320 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:50.320 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:50.320 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:50.320 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5606 2024-05-08T08:54:50.320 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.320 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:50.320 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.320 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:50.320 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884729+0000", 2024-05-08T08:54:50.320 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:50.320 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.321 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.321 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:50.321 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884729+0000", 2024-05-08T08:54:50.321 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:50.321 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.321 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.321 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:50.321 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.885940+0000", 2024-05-08T08:54:50.321 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001210509 2024-05-08T08:54:50.321 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.321 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.321 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:50.321 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.885945+0000", 2024-05-08T08:54:50.321 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7389999999999999e-06 2024-05-08T08:54:50.321 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.321 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.321 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:50.322 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.885955+0000", 2024-05-08T08:54:50.322 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0813e-05 2024-05-08T08:54:50.322 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.322 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.322 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:50.322 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.885962+0000", 2024-05-08T08:54:50.322 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1160000000000001e-06 2024-05-08T08:54:50.322 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.322 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.322 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:50.322 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932465+0000", 2024-05-08T08:54:50.322 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046502612999999998 2024-05-08T08:54:50.322 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.322 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.322 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:50.322 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932547+0000", 2024-05-08T08:54:50.322 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2025000000000002e-05 2024-05-08T08:54:50.323 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.323 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.323 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:50.323 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932646+0000", 2024-05-08T08:54:50.323 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8672999999999995e-05 2024-05-08T08:54:50.323 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:50.323 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:50.323 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:50.323 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.323 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.323 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5635 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4235264~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:50.323 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.886082+0000", 2024-05-08T08:54:50.323 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 24.373293022999999, 2024-05-08T08:54:50.323 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047438284999999997, 2024-05-08T08:54:50.323 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:50.323 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:50.324 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:50.324 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:50.324 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:50.324 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:50.324 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5635 2024-05-08T08:54:50.324 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.324 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:50.324 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.324 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:50.324 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886082+0000", 2024-05-08T08:54:50.324 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:50.324 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.324 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.324 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:50.324 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886082+0000", 2024-05-08T08:54:50.324 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:50.324 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.325 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.325 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:50.325 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886083+0000", 2024-05-08T08:54:50.325 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8999999999999996e-07 2024-05-08T08:54:50.325 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.325 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.325 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:50.325 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886086+0000", 2024-05-08T08:54:50.325 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7e-06 2024-05-08T08:54:50.325 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.325 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.325 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:50.325 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886093+0000", 2024-05-08T08:54:50.325 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5320000000000001e-06 2024-05-08T08:54:50.325 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.325 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.326 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:50.326 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886102+0000", 2024-05-08T08:54:50.326 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8070000000000006e-06 2024-05-08T08:54:50.326 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.326 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.326 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:50.326 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933357+0000", 2024-05-08T08:54:50.326 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047255093999999997 2024-05-08T08:54:50.326 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.326 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.326 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:50.326 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933433+0000", 2024-05-08T08:54:50.326 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5796000000000004e-05 2024-05-08T08:54:50.326 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.326 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.326 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:50.327 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933521+0000", 2024-05-08T08:54:50.327 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7766e-05 2024-05-08T08:54:50.327 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:50.327 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:50.327 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:50.327 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.327 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.327 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5636 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4280320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:50.327 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.886111+0000", 2024-05-08T08:54:50.327 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 24.373264405, 2024-05-08T08:54:50.327 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047663644999999998, 2024-05-08T08:54:50.327 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:50.327 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:50.327 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:50.327 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:50.327 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:50.328 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:50.328 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5636 2024-05-08T08:54:50.328 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.328 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:50.328 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.328 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:50.328 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886111+0000", 2024-05-08T08:54:50.328 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:50.328 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.328 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.328 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:50.328 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886111+0000", 2024-05-08T08:54:50.328 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:50.328 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.328 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.328 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:50.328 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886111+0000", 2024-05-08T08:54:50.329 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5200000000000002e-07 2024-05-08T08:54:50.329 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.329 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.329 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:50.329 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886114+0000", 2024-05-08T08:54:50.329 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1710000000000001e-06 2024-05-08T08:54:50.329 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.329 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.329 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:50.329 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886120+0000", 2024-05-08T08:54:50.329 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.404e-06 2024-05-08T08:54:50.329 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.329 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.329 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:50.329 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886125+0000", 2024-05-08T08:54:50.329 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2730000000000002e-06 2024-05-08T08:54:50.329 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.330 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.330 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:50.330 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933570+0000", 2024-05-08T08:54:50.330 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047444989999999999 2024-05-08T08:54:50.330 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.330 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.330 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:50.330 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933659+0000", 2024-05-08T08:54:50.330 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8912999999999994e-05 2024-05-08T08:54:50.330 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.330 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.330 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:50.330 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933775+0000", 2024-05-08T08:54:50.330 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000115442 2024-05-08T08:54:50.330 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:50.330 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:50.330 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:50.331 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.331 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.331 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5637 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4325376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:50.331 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.886132+0000", 2024-05-08T08:54:50.331 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 24.373243367000001, 2024-05-08T08:54:50.331 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047852219000000001, 2024-05-08T08:54:50.331 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:50.331 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:50.331 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:50.331 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:50.331 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:50.331 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:50.331 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5637 2024-05-08T08:54:50.331 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.331 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:50.331 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.332 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:50.332 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886132+0000", 2024-05-08T08:54:50.332 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:50.332 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.332 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.332 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:50.332 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886132+0000", 2024-05-08T08:54:50.332 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:50.332 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.332 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.332 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:50.332 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886132+0000", 2024-05-08T08:54:50.332 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7300000000000002e-07 2024-05-08T08:54:50.332 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.332 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.332 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:50.333 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886139+0000", 2024-05-08T08:54:50.333 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6109999999999999e-06 2024-05-08T08:54:50.333 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.333 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.333 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:50.333 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886146+0000", 2024-05-08T08:54:50.333 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6279999999999999e-06 2024-05-08T08:54:50.333 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.333 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.333 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:50.333 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886151+0000", 2024-05-08T08:54:50.333 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1769999999999997e-06 2024-05-08T08:54:50.333 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.333 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.333 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:50.333 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933812+0000", 2024-05-08T08:54:50.333 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047660912999999999 2024-05-08T08:54:50.334 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.334 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.334 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:50.334 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933888+0000", 2024-05-08T08:54:50.334 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5865000000000006e-05 2024-05-08T08:54:50.334 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.334 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.334 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:50.334 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933984+0000", 2024-05-08T08:54:50.334 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6651999999999996e-05 2024-05-08T08:54:50.334 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:50.334 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:50.334 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:50.334 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.334 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.334 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5638 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4370432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:50.335 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.886157+0000", 2024-05-08T08:54:50.335 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 24.373218102999999, 2024-05-08T08:54:50.335 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048077597, 2024-05-08T08:54:50.335 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:50.335 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:50.335 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:50.335 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:50.335 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:50.335 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:50.335 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5638 2024-05-08T08:54:50.335 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.335 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:50.335 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.335 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:50.335 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886157+0000", 2024-05-08T08:54:50.335 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:50.336 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.336 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.336 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:50.336 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886157+0000", 2024-05-08T08:54:50.336 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:50.336 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.336 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.336 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:50.336 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886158+0000", 2024-05-08T08:54:50.336 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1100000000000001e-07 2024-05-08T08:54:50.336 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.336 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.336 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:50.336 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886160+0000", 2024-05-08T08:54:50.336 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9800000000000001e-06 2024-05-08T08:54:50.336 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.336 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.337 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:50.337 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886166+0000", 2024-05-08T08:54:50.337 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.314e-06 2024-05-08T08:54:50.337 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.337 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.337 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:50.337 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886171+0000", 2024-05-08T08:54:50.337 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7700000000000001e-06 2024-05-08T08:54:50.337 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.337 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.337 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:50.337 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.934025+0000", 2024-05-08T08:54:50.337 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047854509000000003 2024-05-08T08:54:50.337 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.337 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.337 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:50.338 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.934121+0000", 2024-05-08T08:54:50.338 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5688999999999998e-05 2024-05-08T08:54:50.338 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.338 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.338 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:50.338 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.934235+0000", 2024-05-08T08:54:50.338 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000113924 2024-05-08T08:54:50.338 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:50.338 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:50.338 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:50.338 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.338 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.338 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5639 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4415488~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:50.338 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.886177+0000", 2024-05-08T08:54:50.338 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 24.373198352999999, 2024-05-08T08:54:50.338 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048249541, 2024-05-08T08:54:50.338 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:50.339 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:50.339 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:50.339 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:50.339 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:50.339 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:50.339 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5639 2024-05-08T08:54:50.339 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.339 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:50.339 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.339 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:50.339 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886177+0000", 2024-05-08T08:54:50.339 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:50.339 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.339 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.339 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:50.339 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886177+0000", 2024-05-08T08:54:50.339 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:50.340 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.340 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.340 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:50.340 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886177+0000", 2024-05-08T08:54:50.340 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.65e-07 2024-05-08T08:54:50.340 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.340 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.340 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:50.340 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886179+0000", 2024-05-08T08:54:50.340 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9530000000000002e-06 2024-05-08T08:54:50.340 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.340 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.340 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:50.340 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886186+0000", 2024-05-08T08:54:50.340 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3419999999999996e-06 2024-05-08T08:54:50.340 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.340 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.341 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:50.341 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886192+0000", 2024-05-08T08:54:50.341 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0560000000000001e-06 2024-05-08T08:54:50.341 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.341 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.341 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:50.341 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.934258+0000", 2024-05-08T08:54:50.341 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048066367999999998 2024-05-08T08:54:50.341 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.341 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.341 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:50.341 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.934338+0000", 2024-05-08T08:54:50.341 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9616000000000005e-05 2024-05-08T08:54:50.341 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.341 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.341 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:50.341 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.934427+0000", 2024-05-08T08:54:50.342 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8840999999999999e-05 2024-05-08T08:54:50.342 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:50.342 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:50.342 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:50.342 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:50.342 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:50.342 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:50.343 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:54:50.358 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T08:54:50.383 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-05-08T08:54:50.384 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:50.387+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:50.447 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:50.449+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:50.500 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:50.500 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:54:50.500 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.501 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4416.0:31776 3.9 3:924204dd:::rbd_data.114072661cb0.0000000000000003:head [delete] snapc 21=[] ondisk+write+known_if_redirected+supports_pool_eio e139)", 2024-05-08T08:54:50.501 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:50.499057+0000", 2024-05-08T08:54:50.501 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.0049132869999999997, 2024-05-08T08:54:50.501 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0049331469999999997, 2024-05-08T08:54:50.501 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:50.501 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:50.501 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "waiting for sub ops", 2024-05-08T08:54:50.501 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:50.501 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4416", 2024-05-08T08:54:50.501 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:54:50.501 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 31776 2024-05-08T08:54:50.501 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.501 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:50.502 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.502 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:50.502 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:50.499057+0000", 2024-05-08T08:54:50.502 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:50.502 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.502 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.502 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:50.502 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:50.499057+0000", 2024-05-08T08:54:50.502 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:50.502 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.502 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.502 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:50.502 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:50.499058+0000", 2024-05-08T08:54:50.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.299e-06 2024-05-08T08:54:50.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.503 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:50.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:50.499064+0000", 2024-05-08T08:54:50.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.879e-06 2024-05-08T08:54:50.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.503 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:50.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:50.499081+0000", 2024-05-08T08:54:50.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7207999999999999e-05 2024-05-08T08:54:50.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.503 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:50.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:50.499090+0000", 2024-05-08T08:54:50.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0319999999999992e-06 2024-05-08T08:54:50.504 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.504 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.504 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:50.504 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:50.499168+0000", 2024-05-08T08:54:50.504 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7869999999999998e-05 2024-05-08T08:54:50.504 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.504 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.504 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:50.504 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:50.499253+0000", 2024-05-08T08:54:50.504 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4344000000000004e-05 2024-05-08T08:54:50.504 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.504 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.504 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 2", 2024-05-08T08:54:50.504 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:50.499361+0000", 2024-05-08T08:54:50.504 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010846899999999999 2024-05-08T08:54:50.505 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:50.505 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:50.505 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:54:50.505 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:50.501067+0000", 2024-05-08T08:54:50.505 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001706174 2024-05-08T08:54:50.505 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:50.505 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:50.505 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:50.505 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:50.505 INFO:teuthology.orchestra.run.smithi192.stdout: ], 2024-05-08T08:54:50.505 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 1 2024-05-08T08:54:50.505 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:50.509 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:54:50.510 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:54:50.511 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:54:50.530 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:54:50.530 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:54:50.545 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-05-08T08:54:50.545 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T08:54:50.546 INFO:tasks.thrashosds.thrasher:Removing osd 5, in_osds are: [0, 1, 2, 3, 4, 5] 2024-05-08T08:54:50.546 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 5 2024-05-08T08:54:50.547 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:50.550+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:50.549 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:54:50.549 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:54:50.567 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:54:50.567 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:54:50.585 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:54:50.585 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:54:50.603 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:54:50.603 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:54:50.620 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:50.620 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:50.629 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:54:50.655 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:50.658+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:50.734 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:50.734 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:50.743 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:54:50.786 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:50.789+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:50.847 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:50.847 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:50.849 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:50.852+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:50.856 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:54:50.859 INFO:tasks.ceph.mon.a.smithi059.stderr:2024-05-08T08:54:50.861+0000 7fc9cc811640 -1 mon.a@0(leader).osd e139 definitely_dead 0 2024-05-08T08:54:50.950 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:50.953+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:51.001 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:51.001 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:51.010 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:54:51.050 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:51.054+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:51.114 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:51.114 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:51.123 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:54:51.152 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:51.154+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:51.234 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:51.235 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:51.243 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:54:51.253 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:51.256+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:51.353 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:51.353 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:51.353 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:51.356+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:51.361 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:54:51.395 INFO:teuthology.orchestra.run.smithi059.stderr:nodeep-scrub is set 2024-05-08T08:54:51.396 INFO:teuthology.orchestra.run.smithi059.stderr:marked out osd.5. 2024-05-08T08:54:51.455 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:51.457+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:51.511 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:51.511 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:51.520 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:54:51.564 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:51.567+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:51.630 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:51.630 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:51.639 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:54:51.680 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:51.683+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:51.758 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:51.760+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:51.789 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:51.789 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:51.798 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:54:51.858 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:51.861+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:51.908 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:51.908 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:51.917 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:54:51.961 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:51.964+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:52.027 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:52.027 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:52.036 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:54:52.078 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:52.080+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:52.161 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:52.164+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:52.185 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:52.186 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:52.194 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:54:52.262 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:52.265+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:52.304 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:52.304 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:52.312 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:54:52.363 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:52.366+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:52.431 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:52.431 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:52.440 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:54:52.484 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:52.487+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:52.546 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:52.546 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:52.554 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:54:52.565 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:52.568+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:52.615 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T08:54:52.665 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:52.668+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:52.699 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:52.699 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:52.708 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:54:52.740 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:54:52.740 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:54:52.758 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:54:52.758 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:54:52.766 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:52.769+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:52.775 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:54:52.776 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:54:52.793 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:54:52.794 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:54:52.811 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:54:52.811 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:54:52.816 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:52.816 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:54:52.816 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:54:52.816 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:52.825 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:54:52.828 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:54:52.828 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:54:52.880 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:52.883+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:52.929 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:52.929 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:54:52.929 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:54:52.930 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:54:52.930 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:52.938 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:54:52.981 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:52.984+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:53.045 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:53.045 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:54:53.045 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:54:53.045 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:54:53.045 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.045 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22073 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 782336~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:53.045 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:54:53.045 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 145.189202384, 2024-05-08T08:54:53.045 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047244234000000003, 2024-05-08T08:54:53.045 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:53.045 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:53.045 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:53.045 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:53.046 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:53.046 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:53.046 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22073 2024-05-08T08:54:53.046 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.046 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:53.046 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.046 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:53.046 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:54:53.046 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:53.046 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.046 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.046 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:53.046 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:54:53.047 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:53.047 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.047 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.047 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:53.047 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857668+0000", 2024-05-08T08:54:53.047 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7800000000000002e-07 2024-05-08T08:54:53.047 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.047 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.047 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:53.047 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857670+0000", 2024-05-08T08:54:53.047 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0820000000000001e-06 2024-05-08T08:54:53.047 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.047 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.047 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:53.047 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857678+0000", 2024-05-08T08:54:53.048 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7589999999999997e-06 2024-05-08T08:54:53.048 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.048 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.048 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:53.048 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857691+0000", 2024-05-08T08:54:53.048 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3672999999999999e-05 2024-05-08T08:54:53.048 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.048 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.048 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:53.048 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904780+0000", 2024-05-08T08:54:53.048 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047088616999999999 2024-05-08T08:54:53.048 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.048 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.048 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:53.048 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904840+0000", 2024-05-08T08:54:53.049 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0356999999999998e-05 2024-05-08T08:54:53.049 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.049 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.049 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:53.049 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904912+0000", 2024-05-08T08:54:53.049 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1367999999999998e-05 2024-05-08T08:54:53.049 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.049 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:53.049 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.049 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.049 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.049 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22075 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 798720~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:53.049 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:54:53.049 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 145.18917198700001, 2024-05-08T08:54:53.049 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047384643999999997, 2024-05-08T08:54:53.049 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:53.050 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:53.050 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:53.050 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:53.050 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:53.050 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:53.050 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22075 2024-05-08T08:54:53.050 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.050 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:53.050 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.050 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:53.050 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:54:53.050 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:53.050 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.050 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.050 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:53.051 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:54:53.051 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:53.051 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.051 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.051 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:53.051 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:54:53.051 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7399999999999999e-07 2024-05-08T08:54:53.051 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.051 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.051 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:53.051 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857705+0000", 2024-05-08T08:54:53.051 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1980000000000004e-06 2024-05-08T08:54:53.051 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.051 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.051 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:53.051 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857713+0000", 2024-05-08T08:54:53.052 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7039999999999994e-06 2024-05-08T08:54:53.052 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.052 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.052 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:53.052 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857718+0000", 2024-05-08T08:54:53.052 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.865e-06 2024-05-08T08:54:53.052 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.052 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.052 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:53.052 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904941+0000", 2024-05-08T08:54:53.052 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047223198000000001 2024-05-08T08:54:53.052 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.052 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.052 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:53.052 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904999+0000", 2024-05-08T08:54:53.053 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8254999999999997e-05 2024-05-08T08:54:53.053 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.053 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.053 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:53.053 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905082+0000", 2024-05-08T08:54:53.053 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3049999999999999e-05 2024-05-08T08:54:53.053 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.053 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:53.053 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.053 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.053 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.053 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22077 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 815104~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:53.053 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:54:53.053 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 145.18913397599999, 2024-05-08T08:54:53.053 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047519448999999998, 2024-05-08T08:54:53.053 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:53.053 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:53.054 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:53.054 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:53.054 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:53.054 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:53.054 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22077 2024-05-08T08:54:53.054 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.054 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:53.054 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.054 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:53.054 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:54:53.054 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:53.054 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.054 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.054 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:53.054 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:54:53.055 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:53.055 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.055 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.055 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:53.055 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:54:53.055 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.99e-07 2024-05-08T08:54:53.055 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.055 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.055 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:53.055 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857739+0000", 2024-05-08T08:54:53.055 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3599999999999999e-06 2024-05-08T08:54:53.055 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.055 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.055 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:53.056 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857745+0000", 2024-05-08T08:54:53.056 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5139999999999998e-06 2024-05-08T08:54:53.056 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.056 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.056 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:53.056 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857750+0000", 2024-05-08T08:54:53.056 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8550000000000001e-06 2024-05-08T08:54:53.056 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.056 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.056 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:53.056 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905110+0000", 2024-05-08T08:54:53.056 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047360126000000002 2024-05-08T08:54:53.056 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.056 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.056 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:53.056 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905175+0000", 2024-05-08T08:54:53.057 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4689000000000004e-05 2024-05-08T08:54:53.057 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.057 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.057 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:53.057 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905255+0000", 2024-05-08T08:54:53.057 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0306e-05 2024-05-08T08:54:53.057 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.057 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:53.057 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.057 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.057 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.057 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22079 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 831488~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:53.057 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:54:53.057 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 145.18911342800001, 2024-05-08T08:54:53.058 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.04767499, 2024-05-08T08:54:53.058 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:53.058 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:53.058 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:53.058 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:53.058 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:53.058 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:53.058 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22079 2024-05-08T08:54:53.058 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.058 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:53.058 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.058 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:53.058 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:54:53.058 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:53.059 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.059 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.059 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:53.059 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:54:53.059 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:53.059 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.059 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.059 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:53.059 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857757+0000", 2024-05-08T08:54:53.059 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8299999999999997e-07 2024-05-08T08:54:53.059 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.059 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.059 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:53.059 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857759+0000", 2024-05-08T08:54:53.059 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1390000000000002e-06 2024-05-08T08:54:53.060 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.060 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.060 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:53.060 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857765+0000", 2024-05-08T08:54:53.060 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4609999999999996e-06 2024-05-08T08:54:53.060 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.060 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.060 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:53.060 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857770+0000", 2024-05-08T08:54:53.060 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6689999999999997e-06 2024-05-08T08:54:53.060 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.060 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.060 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:53.060 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905281+0000", 2024-05-08T08:54:53.061 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047510668999999998 2024-05-08T08:54:53.061 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.061 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.061 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:53.061 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905344+0000", 2024-05-08T08:54:53.061 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2797000000000002e-05 2024-05-08T08:54:53.061 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.061 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.061 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:53.061 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905431+0000", 2024-05-08T08:54:53.061 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7771999999999999e-05 2024-05-08T08:54:53.061 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.061 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:53.061 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.061 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.062 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.062 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22082 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 847872~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:53.062 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:54:53.062 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 145.18909373700001, 2024-05-08T08:54:53.062 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047831934999999999, 2024-05-08T08:54:53.062 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:53.062 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:53.062 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:53.062 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:53.062 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:53.062 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:53.062 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22082 2024-05-08T08:54:53.062 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.062 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:53.063 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.063 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:53.063 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:54:53.063 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:53.063 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.063 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.063 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:53.063 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:54:53.063 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:53.063 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.063 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.063 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:53.063 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:54:53.063 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:54:53.063 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.063 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.064 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:53.064 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857778+0000", 2024-05-08T08:54:53.064 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0789999999999999e-06 2024-05-08T08:54:53.064 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.064 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.064 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:53.064 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857784+0000", 2024-05-08T08:54:53.064 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0480000000000004e-06 2024-05-08T08:54:53.064 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.064 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.064 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:53.064 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857790+0000", 2024-05-08T08:54:53.064 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2719999999999997e-06 2024-05-08T08:54:53.064 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.064 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.065 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:53.065 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905458+0000", 2024-05-08T08:54:53.065 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047668470999999997 2024-05-08T08:54:53.065 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.065 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.065 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:53.065 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905521+0000", 2024-05-08T08:54:53.065 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2952000000000003e-05 2024-05-08T08:54:53.065 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.065 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.065 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:53.065 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905608+0000", 2024-05-08T08:54:53.065 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6759000000000001e-05 2024-05-08T08:54:53.065 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.066 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:53.066 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.066 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.066 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.066 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22084 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 864256~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:53.066 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:54:53.066 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 145.18906978699999, 2024-05-08T08:54:53.066 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047977748000000001, 2024-05-08T08:54:53.066 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:53.066 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:53.066 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:53.066 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:53.066 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:53.066 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:53.066 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22084 2024-05-08T08:54:53.067 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.067 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:53.067 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.067 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:53.067 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:54:53.067 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:53.067 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.067 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.067 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:53.067 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:54:53.067 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:53.067 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.067 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.067 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:53.067 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:54:53.067 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:54:53.067 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.068 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.068 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:53.068 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857802+0000", 2024-05-08T08:54:53.068 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0779999999999998e-06 2024-05-08T08:54:53.068 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.068 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.068 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:53.068 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857812+0000", 2024-05-08T08:54:53.068 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7939999999999997e-06 2024-05-08T08:54:53.068 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.068 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.068 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:53.068 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857817+0000", 2024-05-08T08:54:53.068 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8899999999999998e-06 2024-05-08T08:54:53.068 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.068 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.068 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:53.069 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905636+0000", 2024-05-08T08:54:53.069 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047819252 2024-05-08T08:54:53.069 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.069 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.069 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:53.069 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905700+0000", 2024-05-08T08:54:53.069 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4016999999999994e-05 2024-05-08T08:54:53.069 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.069 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.069 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:53.069 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905778+0000", 2024-05-08T08:54:53.069 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7306999999999997e-05 2024-05-08T08:54:53.069 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.069 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:53.069 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.069 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.069 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.070 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22086 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 880640~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:53.070 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:54:53.070 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 145.189046437, 2024-05-08T08:54:53.070 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048119149, 2024-05-08T08:54:53.070 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:53.070 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:53.070 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:53.070 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:53.070 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:53.070 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:53.070 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22086 2024-05-08T08:54:53.070 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.070 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:53.070 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.070 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:53.071 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:54:53.071 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:53.071 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.071 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.071 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:53.071 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:54:53.071 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:53.071 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.071 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.071 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:53.071 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857824+0000", 2024-05-08T08:54:53.071 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4200000000000002e-07 2024-05-08T08:54:53.071 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.071 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.071 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:53.071 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857826+0000", 2024-05-08T08:54:53.071 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0590000000000001e-06 2024-05-08T08:54:53.072 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.072 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.072 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:53.072 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857832+0000", 2024-05-08T08:54:53.072 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8029999999999999e-06 2024-05-08T08:54:53.072 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.072 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.072 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:53.072 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857837+0000", 2024-05-08T08:54:53.072 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7929999999999997e-06 2024-05-08T08:54:53.072 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.072 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.072 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:53.072 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905803+0000", 2024-05-08T08:54:53.072 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047965761000000003 2024-05-08T08:54:53.072 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.073 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.073 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:53.073 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905862+0000", 2024-05-08T08:54:53.073 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9203000000000002e-05 2024-05-08T08:54:53.073 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.073 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.073 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:53.073 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905942+0000", 2024-05-08T08:54:53.073 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0187999999999999e-05 2024-05-08T08:54:53.073 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.073 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:53.073 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.073 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.073 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.073 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22088 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 897024~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:53.074 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:54:53.074 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 145.18902638500001, 2024-05-08T08:54:53.074 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048256049000000002, 2024-05-08T08:54:53.074 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:53.074 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:53.074 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:53.074 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:53.074 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:53.074 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:53.074 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22088 2024-05-08T08:54:53.074 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.074 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:53.074 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.074 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:53.074 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:54:53.074 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:53.075 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.075 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.075 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:53.075 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:54:53.075 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:53.075 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.075 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.075 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:53.075 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857844+0000", 2024-05-08T08:54:53.075 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.46e-07 2024-05-08T08:54:53.075 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.075 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.075 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:53.075 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857846+0000", 2024-05-08T08:54:53.075 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0899999999999999e-06 2024-05-08T08:54:53.076 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.076 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.076 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:53.076 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857855+0000", 2024-05-08T08:54:53.076 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8219999999999996e-06 2024-05-08T08:54:53.076 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.076 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.076 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:53.076 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857860+0000", 2024-05-08T08:54:53.076 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2989999999999996e-06 2024-05-08T08:54:53.076 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.076 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.076 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:53.076 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905966+0000", 2024-05-08T08:54:53.076 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048106542000000002 2024-05-08T08:54:53.076 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.077 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.077 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:53.077 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906024+0000", 2024-05-08T08:54:53.077 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7958000000000003e-05 2024-05-08T08:54:53.077 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.077 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.077 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:53.077 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906099+0000", 2024-05-08T08:54:53.077 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4992000000000003e-05 2024-05-08T08:54:53.077 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.077 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:53.077 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.077 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.077 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.077 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22091 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 913408~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:53.077 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:54:53.077 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 145.189002625, 2024-05-08T08:54:53.078 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048395290000000001, 2024-05-08T08:54:53.078 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:53.078 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:53.078 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:53.078 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:53.078 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:53.078 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:53.078 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22091 2024-05-08T08:54:53.078 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.078 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:53.078 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.078 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:53.078 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:54:53.078 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:53.078 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.079 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.079 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:53.079 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:54:53.079 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:53.079 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.079 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.079 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:53.079 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:54:53.079 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4900000000000001e-07 2024-05-08T08:54:53.079 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.079 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.079 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:53.079 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857870+0000", 2024-05-08T08:54:53.079 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2510000000000001e-06 2024-05-08T08:54:53.079 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.079 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.079 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:53.080 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857877+0000", 2024-05-08T08:54:53.080 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0809999999999996e-06 2024-05-08T08:54:53.080 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.080 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.080 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:53.080 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857881+0000", 2024-05-08T08:54:53.080 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.724e-06 2024-05-08T08:54:53.080 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.080 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.080 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:53.080 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906127+0000", 2024-05-08T08:54:53.080 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048245345000000002 2024-05-08T08:54:53.080 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.080 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.080 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:53.081 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906190+0000", 2024-05-08T08:54:53.081 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2762000000000007e-05 2024-05-08T08:54:53.081 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.081 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.081 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:53.081 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906262+0000", 2024-05-08T08:54:53.081 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2778e-05 2024-05-08T08:54:53.081 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.081 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:53.081 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.081 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.081 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.081 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22093 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 929792~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:53.081 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:54:53.082 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 145.18898026100001, 2024-05-08T08:54:53.082 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048532875000000003, 2024-05-08T08:54:53.082 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:53.082 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:53.082 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:53.082 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:53.082 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:53.082 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:53.082 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22093 2024-05-08T08:54:53.082 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.082 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:53.082 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.082 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:53.082 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:54:53.083 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:53.083 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.083 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.083 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:53.083 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:54:53.083 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:53.083 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.083 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.083 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:53.083 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857890+0000", 2024-05-08T08:54:53.083 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-07 2024-05-08T08:54:53.083 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.083 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.083 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:53.084 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857899+0000", 2024-05-08T08:54:53.084 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6889999999999999e-06 2024-05-08T08:54:53.084 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.084 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.084 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:53.084 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857906+0000", 2024-05-08T08:54:53.084 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6490000000000002e-06 2024-05-08T08:54:53.084 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.084 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.084 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:53.084 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857917+0000", 2024-05-08T08:54:53.084 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0764999999999999e-05 2024-05-08T08:54:53.084 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.084 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.084 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:53.084 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906287+0000", 2024-05-08T08:54:53.085 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048370442 2024-05-08T08:54:53.085 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.085 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.085 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:53.085 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906349+0000", 2024-05-08T08:54:53.085 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1537000000000004e-05 2024-05-08T08:54:53.085 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.085 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.085 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:53.085 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906422+0000", 2024-05-08T08:54:53.085 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3431999999999998e-05 2024-05-08T08:54:53.085 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.085 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:53.085 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.085 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.086 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.086 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22095 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 946176~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:53.086 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:54:53.086 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 145.18894555700001, 2024-05-08T08:54:53.086 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048657593999999998, 2024-05-08T08:54:53.086 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:53.086 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:53.086 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:53.086 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:53.086 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:53.086 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:53.086 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22095 2024-05-08T08:54:53.086 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.086 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:53.087 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.087 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:53.087 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:54:53.087 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:53.087 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.087 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.087 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:53.087 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:54:53.087 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:53.087 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.087 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.087 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:53.087 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857925+0000", 2024-05-08T08:54:53.087 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3099999999999998e-07 2024-05-08T08:54:53.087 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.087 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.088 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:53.088 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857927+0000", 2024-05-08T08:54:53.088 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.34e-06 2024-05-08T08:54:53.088 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.088 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.088 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:53.088 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857938+0000", 2024-05-08T08:54:53.088 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1426e-05 2024-05-08T08:54:53.088 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.088 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.088 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:53.088 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857943+0000", 2024-05-08T08:54:53.088 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5140000000000003e-06 2024-05-08T08:54:53.088 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.088 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.088 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:53.089 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906448+0000", 2024-05-08T08:54:53.089 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048504641000000001 2024-05-08T08:54:53.089 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.089 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.089 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:53.089 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906510+0000", 2024-05-08T08:54:53.089 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2663000000000006e-05 2024-05-08T08:54:53.089 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.089 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.089 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:53.089 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906582+0000", 2024-05-08T08:54:53.089 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1478999999999998e-05 2024-05-08T08:54:53.089 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.089 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:53.089 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.089 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.089 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.090 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22097 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 962560~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:53.090 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:54:53.090 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 145.188917868, 2024-05-08T08:54:53.090 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048791353000000003, 2024-05-08T08:54:53.090 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:53.090 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:53.090 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:53.090 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:53.090 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:53.090 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:53.090 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22097 2024-05-08T08:54:53.090 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.090 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:53.090 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.090 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:53.090 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:54:53.091 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:53.091 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.091 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.091 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:53.091 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:54:53.091 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:53.091 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.091 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.091 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:53.091 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:54:53.091 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9200000000000002e-07 2024-05-08T08:54:53.091 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.091 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.091 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:53.091 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857954+0000", 2024-05-08T08:54:53.091 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1900000000000002e-06 2024-05-08T08:54:53.091 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.092 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.092 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:53.092 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857961+0000", 2024-05-08T08:54:53.092 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4989999999999999e-06 2024-05-08T08:54:53.092 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.092 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.092 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:53.092 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857971+0000", 2024-05-08T08:54:53.092 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0485999999999999e-05 2024-05-08T08:54:53.092 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.092 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.092 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:53.092 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906607+0000", 2024-05-08T08:54:53.092 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048635109000000003 2024-05-08T08:54:53.092 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.092 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.093 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:53.093 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906667+0000", 2024-05-08T08:54:53.093 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0439000000000002e-05 2024-05-08T08:54:53.093 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.093 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.093 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:53.093 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906743+0000", 2024-05-08T08:54:53.093 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6237999999999998e-05 2024-05-08T08:54:53.093 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.093 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:53.093 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.093 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.093 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.093 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22099 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 978944~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:53.093 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:54:53.093 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 145.188891708, 2024-05-08T08:54:53.093 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048932055000000002, 2024-05-08T08:54:53.093 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:53.094 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:53.094 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:53.094 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:53.094 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:53.094 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:53.094 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22099 2024-05-08T08:54:53.094 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.094 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:53.094 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.094 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:53.094 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:54:53.094 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:53.094 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.094 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.094 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:53.095 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:54:53.095 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:53.095 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.095 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.095 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:53.095 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:54:53.095 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.27e-07 2024-05-08T08:54:53.095 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.095 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.095 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:53.095 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857987+0000", 2024-05-08T08:54:53.095 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5890000000000003e-06 2024-05-08T08:54:53.095 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.095 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.095 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:53.095 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857994+0000", 2024-05-08T08:54:53.096 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6619999999999999e-06 2024-05-08T08:54:53.096 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.096 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.096 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:53.096 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857998+0000", 2024-05-08T08:54:53.096 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7110000000000003e-06 2024-05-08T08:54:53.096 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.096 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.096 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:53.096 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906772+0000", 2024-05-08T08:54:53.096 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048773415000000001 2024-05-08T08:54:53.096 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.096 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.096 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:53.096 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906834+0000", 2024-05-08T08:54:53.096 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1752000000000001e-05 2024-05-08T08:54:53.097 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.097 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.097 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:53.097 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906910+0000", 2024-05-08T08:54:53.097 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6499000000000001e-05 2024-05-08T08:54:53.097 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.097 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:53.097 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.097 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.097 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.097 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22101 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 995328~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:53.097 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:54:53.097 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 145.188864577, 2024-05-08T08:54:53.097 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049087364000000001, 2024-05-08T08:54:53.097 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:53.097 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:53.097 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:53.098 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:53.098 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:53.098 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:53.098 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22101 2024-05-08T08:54:53.098 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.098 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:53.098 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.098 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:53.098 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:54:53.098 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:53.098 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.098 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.098 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:53.098 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:54:53.098 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:53.099 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.099 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.099 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:53.099 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858006+0000", 2024-05-08T08:54:53.099 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1399999999999997e-07 2024-05-08T08:54:53.099 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.099 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.099 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:53.099 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858008+0000", 2024-05-08T08:54:53.099 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2060000000000001e-06 2024-05-08T08:54:53.099 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.099 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.099 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:53.099 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858017+0000", 2024-05-08T08:54:53.099 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1579999999999994e-06 2024-05-08T08:54:53.099 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.100 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.100 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:53.100 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858022+0000", 2024-05-08T08:54:53.100 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4199999999999998e-06 2024-05-08T08:54:53.100 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.100 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.100 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:53.100 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906940+0000", 2024-05-08T08:54:53.100 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048917627999999998 2024-05-08T08:54:53.100 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.100 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.100 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:53.100 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907002+0000", 2024-05-08T08:54:53.100 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2399999999999999e-05 2024-05-08T08:54:53.100 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.100 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.100 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:53.101 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907092+0000", 2024-05-08T08:54:53.101 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0137999999999997e-05 2024-05-08T08:54:53.101 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.101 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:53.101 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.101 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.101 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.101 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22103 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1011712~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:53.101 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:54:53.101 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 145.18883979200001, 2024-05-08T08:54:53.101 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049228542, 2024-05-08T08:54:53.101 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:53.101 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:53.101 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:53.101 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:53.101 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:53.101 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:53.102 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22103 2024-05-08T08:54:53.102 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.102 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:53.102 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.102 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:53.102 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:54:53.102 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:53.102 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.102 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.102 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:53.102 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:54:53.102 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:53.102 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.102 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.102 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:53.102 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:54:53.102 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4299999999999999e-07 2024-05-08T08:54:53.103 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.103 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.103 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:53.103 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858032+0000", 2024-05-08T08:54:53.103 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0590000000000001e-06 2024-05-08T08:54:53.103 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.103 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.103 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:53.103 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858037+0000", 2024-05-08T08:54:53.103 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.977e-06 2024-05-08T08:54:53.103 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.103 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.103 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:53.103 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858042+0000", 2024-05-08T08:54:53.103 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7650000000000001e-06 2024-05-08T08:54:53.103 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.104 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.104 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:53.104 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907118+0000", 2024-05-08T08:54:53.104 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049076147000000001 2024-05-08T08:54:53.104 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.104 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.104 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:53.104 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907180+0000", 2024-05-08T08:54:53.104 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1971999999999996e-05 2024-05-08T08:54:53.104 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.104 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.104 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:53.104 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907258+0000", 2024-05-08T08:54:53.104 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8279e-05 2024-05-08T08:54:53.104 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.104 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:53.104 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.105 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.105 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.105 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22105 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1028096~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:53.105 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:54:53.105 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 145.18882054900001, 2024-05-08T08:54:53.105 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049394609999999999, 2024-05-08T08:54:53.105 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:53.105 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:53.105 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:53.105 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:53.105 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:53.105 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:53.105 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22105 2024-05-08T08:54:53.105 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.105 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:53.105 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.106 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:53.106 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:54:53.106 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:53.106 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.106 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.106 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:53.106 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:54:53.106 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:53.106 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.106 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.106 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:53.106 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858050+0000", 2024-05-08T08:54:53.106 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5999999999999999e-07 2024-05-08T08:54:53.106 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.106 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.106 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:53.107 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858051+0000", 2024-05-08T08:54:53.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8870000000000001e-06 2024-05-08T08:54:53.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.107 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:53.107 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858057+0000", 2024-05-08T08:54:53.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8019999999999997e-06 2024-05-08T08:54:53.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.107 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:53.107 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858062+0000", 2024-05-08T08:54:53.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.561e-06 2024-05-08T08:54:53.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.107 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:53.107 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907289+0000", 2024-05-08T08:54:53.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049226957000000002 2024-05-08T08:54:53.108 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.108 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.108 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:53.108 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907356+0000", 2024-05-08T08:54:53.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6915000000000006e-05 2024-05-08T08:54:53.108 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.108 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.108 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:53.108 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907444+0000", 2024-05-08T08:54:53.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8127999999999994e-05 2024-05-08T08:54:53.108 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.108 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:53.108 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.108 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.108 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.109 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22107 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1044480~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:53.109 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:54:53.109 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 145.18880220899999, 2024-05-08T08:54:53.109 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049799942999999999, 2024-05-08T08:54:53.109 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:53.109 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:53.109 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:53.109 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:53.109 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:53.109 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:53.109 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22107 2024-05-08T08:54:53.109 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.109 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:53.109 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.109 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:53.109 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:54:53.110 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:53.110 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.110 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.110 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:53.110 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:54:53.110 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:53.110 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.110 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.110 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:53.110 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:54:53.110 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4799999999999999e-07 2024-05-08T08:54:53.110 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.110 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.110 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:53.110 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858070+0000", 2024-05-08T08:54:53.110 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.012e-06 2024-05-08T08:54:53.110 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.111 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.111 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:53.111 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858077+0000", 2024-05-08T08:54:53.111 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6290000000000001e-06 2024-05-08T08:54:53.111 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.111 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.111 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:53.111 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858083+0000", 2024-05-08T08:54:53.111 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8880000000000002e-06 2024-05-08T08:54:53.111 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.111 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.111 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:53.111 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907671+0000", 2024-05-08T08:54:53.111 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049587191000000003 2024-05-08T08:54:53.111 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.111 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.111 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:53.112 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907763+0000", 2024-05-08T08:54:53.112 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2231000000000005e-05 2024-05-08T08:54:53.112 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.112 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.112 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:53.112 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907867+0000", 2024-05-08T08:54:53.112 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104644 2024-05-08T08:54:53.112 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.112 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:53.112 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.112 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.112 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.112 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22294 3.1c 3:3aad0389:::rbd_data.11406886b07b.0000000000000080:head [stat,delete] snapc 7=[7] RETRY=6 ondisk+retry+write+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:53.112 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:54:53.112 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 145.18820836099999, 2024-05-08T08:54:53.112 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051152191, 2024-05-08T08:54:53.113 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:53.113 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:53.113 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:53.113 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:53.113 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:53.113 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:53.113 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22294 2024-05-08T08:54:53.113 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.113 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:53.113 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.113 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:53.113 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:54:53.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:53.113 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.113 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.113 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:53.113 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:54:53.114 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:53.114 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.114 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.114 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:53.114 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858662+0000", 2024-05-08T08:54:53.114 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4799999999999998e-07 2024-05-08T08:54:53.114 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.114 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.114 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:53.114 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858664+0000", 2024-05-08T08:54:53.114 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1229999999999998e-06 2024-05-08T08:54:53.114 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.114 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.114 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:53.114 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858670+0000", 2024-05-08T08:54:53.114 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3099999999999997e-06 2024-05-08T08:54:53.115 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.115 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.115 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:53.115 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858676+0000", 2024-05-08T08:54:53.115 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9750000000000004e-06 2024-05-08T08:54:53.115 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.115 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.115 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:53.115 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907901+0000", 2024-05-08T08:54:53.115 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049224520000000001 2024-05-08T08:54:53.115 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.115 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.115 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:53.115 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.908085+0000", 2024-05-08T08:54:53.115 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00018377500000000001 2024-05-08T08:54:53.116 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.116 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.116 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:53.116 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.909814+0000", 2024-05-08T08:54:53.116 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00172894 2024-05-08T08:54:53.116 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.116 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:53.116 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.116 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.116 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.116 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4416.0:13461 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-05-08T08:54:53.116 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:22.487456+0000", 2024-05-08T08:54:53.116 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 90.559413320000004, 2024-05-08T08:54:53.116 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.99102620799999996, 2024-05-08T08:54:53.116 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:53.116 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:53.116 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:53.117 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:53.117 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4416", 2024-05-08T08:54:53.117 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:54:53.117 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 13461 2024-05-08T08:54:53.117 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.117 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:53.117 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.117 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:53.117 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487456+0000", 2024-05-08T08:54:53.117 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:53.117 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.117 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.117 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:53.117 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487456+0000", 2024-05-08T08:54:53.117 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:53.117 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.117 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.117 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:53.118 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487458+0000", 2024-05-08T08:54:53.118 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5659999999999999e-06 2024-05-08T08:54:53.118 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.118 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.118 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:53.118 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487463+0000", 2024-05-08T08:54:53.118 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6890000000000003e-06 2024-05-08T08:54:53.118 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.118 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.118 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:53.118 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487472+0000", 2024-05-08T08:54:53.118 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7710000000000001e-06 2024-05-08T08:54:53.118 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.118 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.118 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:53.118 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487478+0000", 2024-05-08T08:54:53.119 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1390000000000003e-06 2024-05-08T08:54:53.119 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.119 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.119 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:53.119 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487520+0000", 2024-05-08T08:54:53.119 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2259999999999999e-05 2024-05-08T08:54:53.119 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.119 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.119 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-05-08T08:54:53.119 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487528+0000", 2024-05-08T08:54:53.119 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7479999999999993e-06 2024-05-08T08:54:53.119 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.119 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.119 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:53.119 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:23.478139+0000", 2024-05-08T08:54:53.119 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.99061165200000001 2024-05-08T08:54:53.119 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.120 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.120 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:53.120 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:23.478301+0000", 2024-05-08T08:54:53.120 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000161602 2024-05-08T08:54:53.120 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.120 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.120 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:53.120 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:23.478483+0000", 2024-05-08T08:54:53.120 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000181781 2024-05-08T08:54:53.120 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.120 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:53.120 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.120 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.120 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.120 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5752.0:404 3.1a 3:5f5e8104:::rbd_data.1140bc4a55c7.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 5175296~45056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-05-08T08:54:53.120 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:00.390863+0000", 2024-05-08T08:54:53.120 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 52.656007125000002, 2024-05-08T08:54:53.121 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.055189174000000001, 2024-05-08T08:54:53.121 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:53.121 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:53.121 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:54:53.121 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:53.121 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5752", 2024-05-08T08:54:53.121 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/3534419815", 2024-05-08T08:54:53.121 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 404 2024-05-08T08:54:53.121 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.121 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:53.121 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.121 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:53.121 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.390863+0000", 2024-05-08T08:54:53.121 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:53.121 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.121 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.121 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:53.122 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.390863+0000", 2024-05-08T08:54:53.122 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:53.122 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.122 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.122 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:53.122 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.390864+0000", 2024-05-08T08:54:53.122 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.471e-06 2024-05-08T08:54:53.122 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.122 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.122 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:53.122 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397475+0000", 2024-05-08T08:54:53.122 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0066109109999999997 2024-05-08T08:54:53.122 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.122 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.122 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:53.122 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397498+0000", 2024-05-08T08:54:53.122 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2674e-05 2024-05-08T08:54:53.123 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.123 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.123 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:53.123 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397508+0000", 2024-05-08T08:54:53.123 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9019999999999994e-06 2024-05-08T08:54:53.123 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.123 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.123 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:53.123 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397609+0000", 2024-05-08T08:54:53.123 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010121899999999999 2024-05-08T08:54:53.123 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.123 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.123 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:53.123 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397711+0000", 2024-05-08T08:54:53.123 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102489 2024-05-08T08:54:53.123 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.123 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.124 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-05-08T08:54:53.124 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397999+0000", 2024-05-08T08:54:53.124 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00028733799999999999 2024-05-08T08:54:53.124 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.124 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.124 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:54:53.124 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.432093+0000", 2024-05-08T08:54:53.124 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.034094865000000002 2024-05-08T08:54:53.124 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.124 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.124 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:54:53.124 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.445975+0000", 2024-05-08T08:54:53.124 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013881187 2024-05-08T08:54:53.124 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.124 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.124 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:54:53.124 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.446013+0000", 2024-05-08T08:54:53.125 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8155000000000003e-05 2024-05-08T08:54:53.125 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.125 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.125 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:53.125 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.446052+0000", 2024-05-08T08:54:53.125 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8962999999999999e-05 2024-05-08T08:54:53.125 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.125 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:53.125 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.125 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.125 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:53.125 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:53.127 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:54:53.127 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:53.129+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:53.227 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:53.230+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:53.271 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:53.271 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:54:53.272 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:54:53.272 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:53.280 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:54:53.328 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:53.331+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:53.385 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:53.385 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:54:53.385 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:54:53.385 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:54:53.385 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:53.394 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:54:53.419 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-05-08T08:54:53.465 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:53.468+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:53.530 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:53.533+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:53.568 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:53.568 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:54:53.568 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:54:53.568 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:54:53.568 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.568 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6690 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1093632~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:53.568 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:54:53.568 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 141.85408004199999, 2024-05-08T08:54:53.568 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052462754, 2024-05-08T08:54:53.568 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:53.568 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:53.568 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:53.568 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:53.569 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:53.569 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:53.569 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6690 2024-05-08T08:54:53.569 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.569 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:53.569 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.569 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:53.569 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:54:53.569 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:53.569 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.569 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.569 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:53.569 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:54:53.569 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:53.570 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.570 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.570 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:53.570 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715346+0000", 2024-05-08T08:54:53.570 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.84e-07 2024-05-08T08:54:53.570 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.570 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.570 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:53.570 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715348+0000", 2024-05-08T08:54:53.570 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.114e-06 2024-05-08T08:54:53.570 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.570 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.570 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:53.570 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715353+0000", 2024-05-08T08:54:53.571 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5720000000000002e-06 2024-05-08T08:54:53.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.571 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.571 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:53.571 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715358+0000", 2024-05-08T08:54:53.571 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7140000000000001e-06 2024-05-08T08:54:53.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.571 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.571 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:53.571 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767638+0000", 2024-05-08T08:54:53.571 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052279562000000002 2024-05-08T08:54:53.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.571 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.571 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:53.571 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767713+0000", 2024-05-08T08:54:53.571 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5652e-05 2024-05-08T08:54:53.572 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.572 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.572 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:53.572 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767808+0000", 2024-05-08T08:54:53.572 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4755999999999999e-05 2024-05-08T08:54:53.572 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.572 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:53.572 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.572 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.572 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.572 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6692 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1110016~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:53.572 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:54:53.572 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 141.854060952, 2024-05-08T08:54:53.572 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052666495000000001, 2024-05-08T08:54:53.572 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:53.573 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:53.573 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:53.573 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:53.573 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:53.573 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:53.573 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6692 2024-05-08T08:54:53.573 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.573 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:53.573 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.573 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:53.573 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:54:53.573 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:53.573 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.573 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.573 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:53.574 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:54:53.574 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:53.574 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.574 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.574 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:53.574 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:54:53.574 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8599999999999999e-07 2024-05-08T08:54:53.574 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.574 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.574 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:53.574 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715372+0000", 2024-05-08T08:54:53.574 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0990000000000001e-06 2024-05-08T08:54:53.574 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.574 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.575 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:53.575 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715378+0000", 2024-05-08T08:54:53.575 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0680000000000002e-06 2024-05-08T08:54:53.575 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.575 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.575 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:53.575 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715383+0000", 2024-05-08T08:54:53.575 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8740000000000003e-06 2024-05-08T08:54:53.575 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.575 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.575 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:53.575 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767848+0000", 2024-05-08T08:54:53.575 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052464668999999998 2024-05-08T08:54:53.575 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.576 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.576 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:53.576 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767927+0000", 2024-05-08T08:54:53.576 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9228999999999996e-05 2024-05-08T08:54:53.576 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.576 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.576 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:53.576 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768031+0000", 2024-05-08T08:54:53.576 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010417 2024-05-08T08:54:53.576 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.576 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:53.576 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.576 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.576 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.576 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6694 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1126400~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:53.576 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:54:53.577 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 141.85403605499999, 2024-05-08T08:54:53.577 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052848829, 2024-05-08T08:54:53.577 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:53.577 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:53.577 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:53.577 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:53.577 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:53.577 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:53.577 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6694 2024-05-08T08:54:53.577 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.577 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:53.577 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.577 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:53.577 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:54:53.578 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:53.578 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.578 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.578 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:53.578 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:54:53.578 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:53.578 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.578 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.578 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:53.578 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715390+0000", 2024-05-08T08:54:53.578 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9900000000000001e-07 2024-05-08T08:54:53.578 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.578 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.578 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:53.579 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715392+0000", 2024-05-08T08:54:53.579 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8279999999999999e-06 2024-05-08T08:54:53.579 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.579 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.579 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:53.579 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715398+0000", 2024-05-08T08:54:53.579 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7619999999999998e-06 2024-05-08T08:54:53.579 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.579 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.579 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:53.579 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715403+0000", 2024-05-08T08:54:53.579 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7790000000000003e-06 2024-05-08T08:54:53.579 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.579 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.579 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:53.579 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768067+0000", 2024-05-08T08:54:53.580 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052663785999999997 2024-05-08T08:54:53.580 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.580 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.580 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:53.580 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768142+0000", 2024-05-08T08:54:53.580 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5018999999999994e-05 2024-05-08T08:54:53.580 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.580 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.580 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:53.580 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768238+0000", 2024-05-08T08:54:53.580 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6255999999999995e-05 2024-05-08T08:54:53.580 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.580 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:53.580 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.580 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.581 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.581 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6696 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1142784~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:53.581 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:54:53.581 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 141.85401654699999, 2024-05-08T08:54:53.581 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.053051057999999998, 2024-05-08T08:54:53.581 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:53.581 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:53.581 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:53.581 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:53.581 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:53.581 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:53.581 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6696 2024-05-08T08:54:53.581 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.581 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:53.582 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.582 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:53.582 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:54:53.582 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:53.582 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.582 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.582 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:53.582 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:54:53.582 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:53.582 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.582 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.582 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:53.582 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:54:53.582 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-05-08T08:54:53.583 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.583 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.583 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:53.583 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715411+0000", 2024-05-08T08:54:53.583 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9630000000000001e-06 2024-05-08T08:54:53.583 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.583 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.583 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:53.583 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715417+0000", 2024-05-08T08:54:53.583 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7250000000000002e-06 2024-05-08T08:54:53.583 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.583 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.583 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:53.583 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715421+0000", 2024-05-08T08:54:53.583 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4089999999999996e-06 2024-05-08T08:54:53.583 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.584 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.584 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:53.584 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768277+0000", 2024-05-08T08:54:53.584 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052855293999999997 2024-05-08T08:54:53.584 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.584 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.584 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:53.584 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768358+0000", 2024-05-08T08:54:53.584 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1720999999999999e-05 2024-05-08T08:54:53.584 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.584 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.584 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:53.584 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768460+0000", 2024-05-08T08:54:53.584 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010154600000000001 2024-05-08T08:54:53.585 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.585 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:53.585 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.585 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.585 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.585 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6698 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1159168~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:53.585 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:54:53.585 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 141.85399767999999, 2024-05-08T08:54:53.585 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.053245726, 2024-05-08T08:54:53.585 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:53.585 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:53.585 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:53.585 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:53.586 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:53.586 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:53.586 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6698 2024-05-08T08:54:53.586 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.586 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:53.586 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.586 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:53.586 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:54:53.586 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:53.586 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.586 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.586 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:53.586 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:54:53.586 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:53.587 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.587 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.587 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:53.587 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716936+0000", 2024-05-08T08:54:53.587 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001508665 2024-05-08T08:54:53.587 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.587 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.587 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:53.587 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716941+0000", 2024-05-08T08:54:53.587 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1910000000000002e-06 2024-05-08T08:54:53.587 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.587 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.587 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:53.587 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716950+0000", 2024-05-08T08:54:53.587 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1900000000000001e-06 2024-05-08T08:54:53.587 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.588 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.588 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:53.588 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716958+0000", 2024-05-08T08:54:53.588 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8019999999999992e-06 2024-05-08T08:54:53.588 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.588 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.588 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:53.588 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768499+0000", 2024-05-08T08:54:53.588 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051540926000000001 2024-05-08T08:54:53.588 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.588 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.588 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:53.588 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768574+0000", 2024-05-08T08:54:53.588 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4988999999999996e-05 2024-05-08T08:54:53.588 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.588 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.589 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:53.589 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768673+0000", 2024-05-08T08:54:53.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9963000000000005e-05 2024-05-08T08:54:53.589 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.589 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:53.589 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.589 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.589 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.589 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6706 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1224704~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:53.589 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:54:53.589 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 141.85239237299999, 2024-05-08T08:54:53.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052435598, 2024-05-08T08:54:53.589 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:53.589 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:53.589 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:53.589 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:53.589 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:53.590 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:53.590 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6706 2024-05-08T08:54:53.590 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.590 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:53.590 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.590 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:53.590 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:54:53.590 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:53.590 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.590 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.590 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:53.590 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:54:53.590 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:53.590 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.590 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.590 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:53.591 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:54:53.591 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:54:53.591 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.591 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.591 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:53.591 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717035+0000", 2024-05-08T08:54:53.591 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9290000000000001e-06 2024-05-08T08:54:53.591 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.591 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.591 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:53.591 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717041+0000", 2024-05-08T08:54:53.591 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6720000000000001e-06 2024-05-08T08:54:53.591 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.591 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.591 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:53.592 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717046+0000", 2024-05-08T08:54:53.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5650000000000003e-06 2024-05-08T08:54:53.592 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.592 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.592 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:53.592 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769311+0000", 2024-05-08T08:54:53.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052265645999999999 2024-05-08T08:54:53.592 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.592 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.592 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:53.592 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769379+0000", 2024-05-08T08:54:53.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7391000000000006e-05 2024-05-08T08:54:53.592 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.592 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.592 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:53.592 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769469+0000", 2024-05-08T08:54:53.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0025999999999995e-05 2024-05-08T08:54:53.593 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.593 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:53.593 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.593 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.593 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.593 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6708 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1241088~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:53.593 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:54:53.593 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 141.85237426899999, 2024-05-08T08:54:53.593 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052640552, 2024-05-08T08:54:53.593 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:53.593 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:53.593 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:53.593 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:53.593 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:53.594 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:53.594 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6708 2024-05-08T08:54:53.594 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.594 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:53.594 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.594 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:53.594 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:54:53.594 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:53.594 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.594 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.594 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:53.594 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:54:53.594 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:53.594 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.595 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.595 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:53.595 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717052+0000", 2024-05-08T08:54:53.595 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:54:53.595 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.595 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.595 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:53.595 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717053+0000", 2024-05-08T08:54:53.595 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8330000000000001e-06 2024-05-08T08:54:53.595 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.595 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.595 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:53.595 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717058+0000", 2024-05-08T08:54:53.595 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.933e-06 2024-05-08T08:54:53.596 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.596 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:53.596 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717063+0000", 2024-05-08T08:54:53.596 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2180000000000001e-06 2024-05-08T08:54:53.596 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.596 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:53.596 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769507+0000", 2024-05-08T08:54:53.596 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052444866999999999 2024-05-08T08:54:53.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.597 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:53.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769589+0000", 2024-05-08T08:54:53.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2039000000000005e-05 2024-05-08T08:54:53.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.597 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:53.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769692+0000", 2024-05-08T08:54:53.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102252 2024-05-08T08:54:53.597 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.597 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:53.597 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.598 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6710 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1257472~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:53.598 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:54:53.598 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 141.85235624500001, 2024-05-08T08:54:53.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.05283057, 2024-05-08T08:54:53.598 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:53.598 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:53.598 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:53.598 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:53.598 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:53.598 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:53.598 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6710 2024-05-08T08:54:53.598 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.598 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:53.599 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.599 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:53.599 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:54:53.599 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:53.599 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.599 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.599 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:53.599 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:54:53.599 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:53.599 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.599 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.599 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:53.599 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718951+0000", 2024-05-08T08:54:53.599 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0018817809999999999 2024-05-08T08:54:53.599 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.600 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.600 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:53.600 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718956+0000", 2024-05-08T08:54:53.600 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5399999999999997e-06 2024-05-08T08:54:53.600 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.600 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.600 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:53.600 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718964+0000", 2024-05-08T08:54:53.600 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6629999999999999e-06 2024-05-08T08:54:53.600 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.600 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.600 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:53.600 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718972+0000", 2024-05-08T08:54:53.600 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5229999999999998e-06 2024-05-08T08:54:53.601 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.601 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:53.601 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769729+0000", 2024-05-08T08:54:53.601 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050756959999999997 2024-05-08T08:54:53.601 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.601 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:53.601 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769803+0000", 2024-05-08T08:54:53.601 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4538999999999999e-05 2024-05-08T08:54:53.601 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.601 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:53.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769900+0000", 2024-05-08T08:54:53.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6564000000000006e-05 2024-05-08T08:54:53.602 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.602 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:53.602 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.602 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6726 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1388544~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:53.602 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:54:53.602 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 141.850281473, 2024-05-08T08:54:53.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052451506000000002, 2024-05-08T08:54:53.602 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:53.602 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:53.602 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:53.603 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:53.603 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:53.603 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:53.603 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6726 2024-05-08T08:54:53.603 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.603 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:53.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.603 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:53.603 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:54:53.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:53.603 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.603 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:53.603 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:54:53.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:53.603 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.604 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.604 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:53.604 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719200+0000", 2024-05-08T08:54:53.604 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5865999999999997e-05 2024-05-08T08:54:53.604 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.604 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.604 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:53.604 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719204+0000", 2024-05-08T08:54:53.604 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8630000000000001e-06 2024-05-08T08:54:53.604 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.604 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.604 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:53.604 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719215+0000", 2024-05-08T08:54:53.604 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1590000000000001e-05 2024-05-08T08:54:53.604 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.604 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.605 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:53.605 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719220+0000", 2024-05-08T08:54:53.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1619999999999999e-06 2024-05-08T08:54:53.605 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.605 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.605 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:53.605 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771425+0000", 2024-05-08T08:54:53.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052204249000000001 2024-05-08T08:54:53.605 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.605 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.605 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:53.605 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771498+0000", 2024-05-08T08:54:53.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3782999999999995e-05 2024-05-08T08:54:53.605 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.605 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.606 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:53.606 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771595+0000", 2024-05-08T08:54:53.606 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6992999999999998e-05 2024-05-08T08:54:53.606 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.606 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:53.606 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.606 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.606 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6728 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1404928~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:53.606 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:54:53.606 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 141.85019406800001, 2024-05-08T08:54:53.606 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052590747, 2024-05-08T08:54:53.606 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:53.606 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:53.606 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:53.607 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:53.607 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:53.607 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:53.607 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6728 2024-05-08T08:54:53.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.607 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:53.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.607 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:53.607 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:54:53.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:53.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.607 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:53.607 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:54:53.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:53.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.608 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:53.608 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719232+0000", 2024-05-08T08:54:53.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4000000000000001e-07 2024-05-08T08:54:53.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.608 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:53.608 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719234+0000", 2024-05-08T08:54:53.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4499999999999998e-06 2024-05-08T08:54:53.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.608 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:53.608 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719240+0000", 2024-05-08T08:54:53.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5720000000000002e-06 2024-05-08T08:54:53.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.609 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:53.609 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719245+0000", 2024-05-08T08:54:53.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8690000000000003e-06 2024-05-08T08:54:53.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.609 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:53.609 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771636+0000", 2024-05-08T08:54:53.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052390979999999997 2024-05-08T08:54:53.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.609 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:53.609 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771717+0000", 2024-05-08T08:54:53.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1291999999999993e-05 2024-05-08T08:54:53.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.610 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:53.610 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771822+0000", 2024-05-08T08:54:53.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104944 2024-05-08T08:54:53.610 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.610 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:53.610 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.610 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.610 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6730 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1421312~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:53.610 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:54:53.610 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 141.850173826, 2024-05-08T08:54:53.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052770477000000003, 2024-05-08T08:54:53.610 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:53.611 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:53.611 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:53.611 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:53.611 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:53.611 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:53.611 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6730 2024-05-08T08:54:53.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.611 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:53.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.611 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:53.611 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:54:53.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:53.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.611 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:53.611 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:54:53.612 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:53.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.612 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:53.612 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:54:53.612 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9000000000000002e-07 2024-05-08T08:54:53.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.612 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:53.612 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719254+0000", 2024-05-08T08:54:53.612 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4099999999999998e-06 2024-05-08T08:54:53.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.612 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:53.613 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719263+0000", 2024-05-08T08:54:53.613 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4109999999999996e-06 2024-05-08T08:54:53.613 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.613 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.613 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:53.613 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719268+0000", 2024-05-08T08:54:53.613 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2669999999999997e-06 2024-05-08T08:54:53.613 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.613 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.613 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:53.613 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771846+0000", 2024-05-08T08:54:53.613 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052577677000000003 2024-05-08T08:54:53.613 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.613 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.613 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:53.613 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771920+0000", 2024-05-08T08:54:53.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4460000000000002e-05 2024-05-08T08:54:53.614 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.614 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.614 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:53.614 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772022+0000", 2024-05-08T08:54:53.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010186199999999999 2024-05-08T08:54:53.614 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.614 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:53.614 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.614 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.614 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.614 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6732 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1437696~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:53.614 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:54:53.614 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 141.85014901900001, 2024-05-08T08:54:53.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052905575000000003, 2024-05-08T08:54:53.615 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:53.615 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:53.615 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:53.615 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:53.615 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:53.615 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:53.615 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6732 2024-05-08T08:54:53.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.615 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:53.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:53.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:54:53.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:53.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:53.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:54:53.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:53.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:53.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721392+0000", 2024-05-08T08:54:53.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002115287 2024-05-08T08:54:53.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:53.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721396+0000", 2024-05-08T08:54:53.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1679999999999997e-06 2024-05-08T08:54:53.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:53.617 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721404+0000", 2024-05-08T08:54:53.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4409999999999993e-06 2024-05-08T08:54:53.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.617 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.617 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:53.617 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721410+0000", 2024-05-08T08:54:53.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9150000000000001e-06 2024-05-08T08:54:53.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.617 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.617 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:53.617 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772061+0000", 2024-05-08T08:54:53.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050651068 2024-05-08T08:54:53.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.617 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.617 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:53.618 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772127+0000", 2024-05-08T08:54:53.618 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5424000000000002e-05 2024-05-08T08:54:53.618 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.618 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.618 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:53.618 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772182+0000", 2024-05-08T08:54:53.618 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5272000000000002e-05 2024-05-08T08:54:53.618 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.618 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:53.618 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.618 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.618 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.618 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6749 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1585152~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:53.618 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:54:53.618 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 141.84784824900001, 2024-05-08T08:54:53.618 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052350264000000001, 2024-05-08T08:54:53.619 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:53.619 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:53.619 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:53.619 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:53.619 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:53.619 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:53.619 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6749 2024-05-08T08:54:53.619 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.619 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:53.619 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.619 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:53.619 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:54:53.619 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:53.619 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.619 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.620 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:53.620 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:54:53.620 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:53.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.620 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:53.620 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721578+0000", 2024-05-08T08:54:53.620 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0399999999999996e-07 2024-05-08T08:54:53.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.620 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:53.620 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721580+0000", 2024-05-08T08:54:53.620 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5450000000000002e-06 2024-05-08T08:54:53.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.620 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:53.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721587+0000", 2024-05-08T08:54:53.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0190000000000001e-06 2024-05-08T08:54:53.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:53.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721593+0000", 2024-05-08T08:54:53.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5659999999999997e-06 2024-05-08T08:54:53.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:53.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773759+0000", 2024-05-08T08:54:53.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052166553999999997 2024-05-08T08:54:53.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:53.622 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773833+0000", 2024-05-08T08:54:53.622 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3913999999999997e-05 2024-05-08T08:54:53.622 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.622 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.622 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:53.622 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773927+0000", 2024-05-08T08:54:53.622 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4061999999999995e-05 2024-05-08T08:54:53.622 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.622 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:53.622 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.622 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.622 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.622 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6751 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1601536~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:53.622 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:54:53.623 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 141.84782497, 2024-05-08T08:54:53.623 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052461302000000001, 2024-05-08T08:54:53.623 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:53.623 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:53.623 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:53.623 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:53.623 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:53.623 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:53.623 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6751 2024-05-08T08:54:53.623 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.623 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:53.623 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.623 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:53.623 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:54:53.623 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:53.623 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.624 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:53.624 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:54:53.624 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:53.624 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.624 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:53.624 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722939+0000", 2024-05-08T08:54:53.624 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001338601 2024-05-08T08:54:53.624 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.624 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:53.624 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722944+0000", 2024-05-08T08:54:53.624 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4469999999999999e-06 2024-05-08T08:54:53.624 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.625 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:53.625 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722955+0000", 2024-05-08T08:54:53.625 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1892999999999999e-05 2024-05-08T08:54:53.625 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.625 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.625 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:53.625 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722963+0000", 2024-05-08T08:54:53.625 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1080000000000004e-06 2024-05-08T08:54:53.625 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.625 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.625 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:53.625 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773964+0000", 2024-05-08T08:54:53.625 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051001022 2024-05-08T08:54:53.625 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.626 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.626 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:53.626 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.774006+0000", 2024-05-08T08:54:53.626 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2729999999999999e-05 2024-05-08T08:54:53.626 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.626 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.626 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:53.626 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.774062+0000", 2024-05-08T08:54:53.626 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5501000000000003e-05 2024-05-08T08:54:53.626 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.626 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:53.626 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.626 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.626 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.627 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6760 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1683456~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:53.627 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:54:53.627 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 141.846315594, 2024-05-08T08:54:53.627 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052571312000000002, 2024-05-08T08:54:53.627 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:53.627 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:53.627 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:53.627 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:53.627 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:53.627 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:53.627 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6760 2024-05-08T08:54:53.627 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.627 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:53.627 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.627 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:53.627 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:54:53.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:53.628 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.628 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:53.628 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:54:53.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:53.628 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.628 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:53.628 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:54:53.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2100000000000002e-07 2024-05-08T08:54:53.628 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.628 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:53.628 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723113+0000", 2024-05-08T08:54:53.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3149999999999999e-06 2024-05-08T08:54:53.628 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.629 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:53.629 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723118+0000", 2024-05-08T08:54:53.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8860000000000001e-06 2024-05-08T08:54:53.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.629 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:53.629 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723123+0000", 2024-05-08T08:54:53.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6770000000000003e-06 2024-05-08T08:54:53.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.629 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:53.629 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775414+0000", 2024-05-08T08:54:53.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052290692999999999 2024-05-08T08:54:53.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.630 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:53.630 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775459+0000", 2024-05-08T08:54:53.630 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5639999999999997e-05 2024-05-08T08:54:53.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.630 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.630 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:53.630 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775681+0000", 2024-05-08T08:54:53.630 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00022168000000000001 2024-05-08T08:54:53.630 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.630 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:53.630 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.630 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.630 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6761 3.16 3:6bd6d6da:::rbd_data.1140782590a5.0000000000000015:6 [read 1095680~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:53.630 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:54:53.630 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 141.84629637399999, 2024-05-08T08:54:53.631 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052751275, 2024-05-08T08:54:53.631 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:53.631 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:53.631 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:53.631 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:53.631 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:53.631 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:53.631 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6761 2024-05-08T08:54:53.631 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.631 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:53.631 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.631 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:53.631 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:54:53.631 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:53.631 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.631 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.631 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:53.632 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:54:53.632 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:53.632 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.632 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.632 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:53.632 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724939+0000", 2024-05-08T08:54:53.632 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001809817 2024-05-08T08:54:53.632 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.632 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.632 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:53.632 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724943+0000", 2024-05-08T08:54:53.632 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4920000000000004e-06 2024-05-08T08:54:53.632 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.632 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.633 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:53.633 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724956+0000", 2024-05-08T08:54:53.633 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2427e-05 2024-05-08T08:54:53.633 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.633 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.633 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:53.633 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724963+0000", 2024-05-08T08:54:53.633 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2559999999999996e-06 2024-05-08T08:54:53.633 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.633 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.633 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:53.633 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775711+0000", 2024-05-08T08:54:53.633 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050747621 2024-05-08T08:54:53.633 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.633 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.634 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:53.634 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775785+0000", 2024-05-08T08:54:53.634 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4634000000000004e-05 2024-05-08T08:54:53.634 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.634 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:53.634 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775880+0000", 2024-05-08T08:54:53.634 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5027999999999999e-05 2024-05-08T08:54:53.634 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.634 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:53.634 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.634 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.634 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6770 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1765376~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:53.634 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:54:53.635 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 141.84426527900001, 2024-05-08T08:54:53.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052339296, 2024-05-08T08:54:53.635 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:53.635 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:53.635 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:53.635 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:53.635 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:53.635 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:53.635 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6770 2024-05-08T08:54:53.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.635 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:53.635 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.635 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:53.635 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:54:53.636 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:53.636 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.636 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:53.636 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:54:53.636 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:53.636 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.636 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:53.636 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725161+0000", 2024-05-08T08:54:53.636 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-05-08T08:54:53.636 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.636 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:53.636 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725163+0000", 2024-05-08T08:54:53.636 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4370000000000001e-06 2024-05-08T08:54:53.637 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.637 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.637 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:53.637 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725169+0000", 2024-05-08T08:54:53.637 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8409999999999996e-06 2024-05-08T08:54:53.637 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.637 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.637 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:53.637 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725174+0000", 2024-05-08T08:54:53.637 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7249999999999997e-06 2024-05-08T08:54:53.637 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.637 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.637 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:53.637 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777343+0000", 2024-05-08T08:54:53.637 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052169633999999999 2024-05-08T08:54:53.637 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.637 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.638 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:53.638 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777409+0000", 2024-05-08T08:54:53.638 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6033999999999998e-05 2024-05-08T08:54:53.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.638 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.638 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:53.638 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777499+0000", 2024-05-08T08:54:53.638 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0224999999999998e-05 2024-05-08T08:54:53.638 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.638 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:53.638 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.638 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.638 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6771 3.16 3:6bd6d6da:::rbd_data.1140782590a5.0000000000000015:6 [read 1177600~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:53.638 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:54:53.638 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 141.844245462, 2024-05-08T08:54:53.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052438604999999999, 2024-05-08T08:54:53.639 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:53.639 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:53.639 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:53.639 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:53.639 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:53.639 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:53.639 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6771 2024-05-08T08:54:53.639 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.639 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:53.639 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.639 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:53.639 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:54:53.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:53.639 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.639 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.639 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:53.640 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:54:53.640 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:53.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.640 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.640 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:53.640 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:54:53.640 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:54:53.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.640 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.640 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:53.640 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725188+0000", 2024-05-08T08:54:53.640 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1579999999999999e-06 2024-05-08T08:54:53.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.640 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.640 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:53.640 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725195+0000", 2024-05-08T08:54:53.640 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8909999999999995e-06 2024-05-08T08:54:53.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.641 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:53.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725200+0000", 2024-05-08T08:54:53.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9810000000000003e-06 2024-05-08T08:54:53.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.641 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:53.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777527+0000", 2024-05-08T08:54:53.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052326582000000003 2024-05-08T08:54:53.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.641 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:53.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777566+0000", 2024-05-08T08:54:53.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9447999999999999e-05 2024-05-08T08:54:53.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:53.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777619+0000", 2024-05-08T08:54:53.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2135000000000001e-05 2024-05-08T08:54:53.642 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.642 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:53.642 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.642 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6772 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1781760~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:53.642 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:54:53.642 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 141.84421740499999, 2024-05-08T08:54:53.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052601799999999997, 2024-05-08T08:54:53.642 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:53.642 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:53.642 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:53.642 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:53.643 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:53.643 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:53.643 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6772 2024-05-08T08:54:53.643 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.643 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:53.643 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.643 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:53.643 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:54:53.643 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:53.643 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.643 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.643 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:53.643 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:54:53.643 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:53.643 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.643 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.643 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:53.644 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726939+0000", 2024-05-08T08:54:53.644 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001730493 2024-05-08T08:54:53.644 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.644 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.644 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:53.644 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726943+0000", 2024-05-08T08:54:53.644 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7330000000000003e-06 2024-05-08T08:54:53.644 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.644 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.644 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:53.644 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726954+0000", 2024-05-08T08:54:53.644 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1107000000000001e-05 2024-05-08T08:54:53.644 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.644 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.644 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:53.644 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726962+0000", 2024-05-08T08:54:53.644 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.503e-06 2024-05-08T08:54:53.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.645 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.645 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:53.645 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777653+0000", 2024-05-08T08:54:53.645 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050691542999999999 2024-05-08T08:54:53.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.645 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.645 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:53.645 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777719+0000", 2024-05-08T08:54:53.645 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5228000000000006e-05 2024-05-08T08:54:53.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.645 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.645 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:53.645 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777810+0000", 2024-05-08T08:54:53.645 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1193000000000006e-05 2024-05-08T08:54:53.645 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.645 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:53.646 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.646 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6782 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1880064~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:53.646 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:54:53.646 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 141.84224889800001, 2024-05-08T08:54:53.646 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052326492000000002, 2024-05-08T08:54:53.646 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:53.646 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:53.646 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:53.646 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:53.646 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:53.646 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:53.646 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6782 2024-05-08T08:54:53.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.647 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:53.647 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.647 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:53.647 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:54:53.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:53.647 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.647 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.647 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:53.647 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:54:53.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:53.647 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.647 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.647 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:53.647 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728939+0000", 2024-05-08T08:54:53.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001762719 2024-05-08T08:54:53.647 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.647 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.648 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:53.648 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728944+0000", 2024-05-08T08:54:53.648 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.746e-06 2024-05-08T08:54:53.648 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.648 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.648 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:53.648 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728955+0000", 2024-05-08T08:54:53.648 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1043000000000001e-05 2024-05-08T08:54:53.648 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.648 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.648 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:53.648 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728963+0000", 2024-05-08T08:54:53.648 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5809999999999999e-06 2024-05-08T08:54:53.648 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.648 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.648 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:53.648 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779320+0000", 2024-05-08T08:54:53.649 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050357290999999998 2024-05-08T08:54:53.649 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.649 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.649 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:53.649 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779401+0000", 2024-05-08T08:54:53.649 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0574000000000005e-05 2024-05-08T08:54:53.649 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:53.649 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:53.649 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:53.649 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779503+0000", 2024-05-08T08:54:53.649 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102538 2024-05-08T08:54:53.649 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.649 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:53.649 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.649 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:53.649 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:53.649 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:53.651 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:53.653+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:53.651 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:54:53.751 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:53.754+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:53.788 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:53.788 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:54:53.788 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:54:53.788 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:53.799 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:54:53.852 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:53.855+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:53.934 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:53.934 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:54:53.934 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:54:53.934 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:54:53.934 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:53.945 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:54:53.986 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:53.989+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:54.054 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:54.057+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:54.121 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:54.121 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:54:54.121 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:54:54.121 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:54:54.122 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.122 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4621.0:9089 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 23339520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:54:54.122 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:54:54.122 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 141.62978250399999, 2024-05-08T08:54:54.122 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.13814907200000001, 2024-05-08T08:54:54.122 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:54.122 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:54.122 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:54:54.122 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:54.122 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4621", 2024-05-08T08:54:54.122 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:54.122 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9089 2024-05-08T08:54:54.122 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.122 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:54.123 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.123 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:54.123 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:54:54.123 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:54.123 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.123 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.123 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:54.123 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:54:54.123 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:54.123 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.123 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.123 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:54.123 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493005+0000", 2024-05-08T08:54:54.123 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1700000000000001e-06 2024-05-08T08:54:54.124 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.124 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.124 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:54.124 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.513963+0000", 2024-05-08T08:54:54.124 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.020957581999999999 2024-05-08T08:54:54.124 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.124 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.124 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:54.124 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.513995+0000", 2024-05-08T08:54:54.124 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1993999999999999e-05 2024-05-08T08:54:54.124 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.124 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.124 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:54.124 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514007+0000", 2024-05-08T08:54:54.125 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1581e-05 2024-05-08T08:54:54.125 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.125 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.125 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:54.125 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514122+0000", 2024-05-08T08:54:54.125 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000115282 2024-05-08T08:54:54.125 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.125 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.125 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op must wait for map", 2024-05-08T08:54:54.125 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514128+0000", 2024-05-08T08:54:54.125 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9950000000000002e-06 2024-05-08T08:54:54.125 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.125 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.125 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:54.125 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.515800+0000", 2024-05-08T08:54:54.125 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001671886 2024-05-08T08:54:54.126 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.126 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.126 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:54.126 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.515893+0000", 2024-05-08T08:54:54.126 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3490000000000001e-05 2024-05-08T08:54:54.126 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.126 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.126 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 3", 2024-05-08T08:54:54.126 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.517647+0000", 2024-05-08T08:54:54.126 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0017535630000000001 2024-05-08T08:54:54.126 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.126 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.126 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:54:54.126 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.627578+0000", 2024-05-08T08:54:54.126 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.109931557 2024-05-08T08:54:54.126 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.127 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.127 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:54:54.127 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631055+0000", 2024-05-08T08:54:54.127 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0034761420000000002 2024-05-08T08:54:54.127 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.127 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.127 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:54:54.127 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631096+0000", 2024-05-08T08:54:54.127 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1059000000000001e-05 2024-05-08T08:54:54.127 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.127 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.127 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:54.127 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631151+0000", 2024-05-08T08:54:54.127 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5770999999999998e-05 2024-05-08T08:54:54.127 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.127 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:54.127 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.128 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.128 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.128 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1071 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11026432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:54.128 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:54:54.128 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 137.63733301299999, 2024-05-08T08:54:54.128 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070580037999999998, 2024-05-08T08:54:54.128 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:54.128 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:54.128 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:54.128 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:54.128 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:54.128 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:54.128 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1071 2024-05-08T08:54:54.129 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.129 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:54.129 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.129 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:54.129 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:54:54.129 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:54.129 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.129 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.129 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:54.129 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:54:54.129 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:54.129 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.129 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.129 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:54.130 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:54:54.130 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2800000000000002e-07 2024-05-08T08:54:54.130 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.130 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.130 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:54.130 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485454+0000", 2024-05-08T08:54:54.130 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2290000000000002e-06 2024-05-08T08:54:54.130 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.130 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.130 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:54.130 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485461+0000", 2024-05-08T08:54:54.130 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9920000000000001e-06 2024-05-08T08:54:54.130 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.130 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.131 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:54.131 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485467+0000", 2024-05-08T08:54:54.131 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3979999999999998e-06 2024-05-08T08:54:54.131 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.131 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.131 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:54.131 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555880+0000", 2024-05-08T08:54:54.131 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070413165 2024-05-08T08:54:54.131 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.131 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.131 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:54.131 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555926+0000", 2024-05-08T08:54:54.131 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6217000000000002e-05 2024-05-08T08:54:54.131 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.131 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.132 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:54.132 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556032+0000", 2024-05-08T08:54:54.132 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000105609 2024-05-08T08:54:54.132 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.132 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:54.132 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.132 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.132 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.132 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1073 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11067392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:54.132 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:54:54.132 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 137.63731179199999, 2024-05-08T08:54:54.132 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070821404000000004, 2024-05-08T08:54:54.132 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:54.132 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:54.133 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:54.133 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:54.133 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:54.133 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:54.133 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1073 2024-05-08T08:54:54.133 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.133 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:54.133 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.133 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:54.133 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:54:54.133 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:54.133 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.133 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.133 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:54.133 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:54:54.134 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:54.134 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.134 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.134 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:54.134 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:54:54.134 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.08e-07 2024-05-08T08:54:54.134 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.134 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.134 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:54.134 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485476+0000", 2024-05-08T08:54:54.134 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3929999999999998e-06 2024-05-08T08:54:54.134 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.134 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.134 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:54.135 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485485+0000", 2024-05-08T08:54:54.135 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1880000000000008e-06 2024-05-08T08:54:54.135 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.135 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.135 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:54.135 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485494+0000", 2024-05-08T08:54:54.135 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3109999999999995e-06 2024-05-08T08:54:54.135 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.135 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.135 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:54.135 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556082+0000", 2024-05-08T08:54:54.135 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070587426999999994 2024-05-08T08:54:54.135 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.135 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.136 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:54.136 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556177+0000", 2024-05-08T08:54:54.136 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5051999999999997e-05 2024-05-08T08:54:54.136 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.136 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.136 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:54.136 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556294+0000", 2024-05-08T08:54:54.136 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000117625 2024-05-08T08:54:54.136 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.136 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:54.136 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.136 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.136 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.136 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1075 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11108352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:54.137 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:54:54.137 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 137.637281215, 2024-05-08T08:54:54.137 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071045026999999997, 2024-05-08T08:54:54.137 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:54.137 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:54.137 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:54.137 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:54.137 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:54.137 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:54.137 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1075 2024-05-08T08:54:54.137 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.137 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:54.137 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.137 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:54.137 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:54:54.138 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:54.138 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.138 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.138 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:54.138 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:54:54.138 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:54.138 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.138 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.138 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:54.138 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:54:54.138 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.32e-07 2024-05-08T08:54:54.138 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.138 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.138 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:54.139 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485518+0000", 2024-05-08T08:54:54.139 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3769e-05 2024-05-08T08:54:54.139 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.139 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.139 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:54.139 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485527+0000", 2024-05-08T08:54:54.139 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6779999999999995e-06 2024-05-08T08:54:54.139 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.139 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.139 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:54.139 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485532+0000", 2024-05-08T08:54:54.139 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9830000000000004e-06 2024-05-08T08:54:54.139 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.139 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.139 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:54.140 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556340+0000", 2024-05-08T08:54:54.140 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070807743000000006 2024-05-08T08:54:54.140 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.140 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.140 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:54.140 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556435+0000", 2024-05-08T08:54:54.140 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4732e-05 2024-05-08T08:54:54.140 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.140 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.140 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:54.140 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556549+0000", 2024-05-08T08:54:54.140 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011369 2024-05-08T08:54:54.140 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.140 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:54.140 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.140 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.140 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.141 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1078 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11149312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:54.141 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:54:54.141 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 137.63724101599999, 2024-05-08T08:54:54.141 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071167276000000002, 2024-05-08T08:54:54.141 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:54.141 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:54.141 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:54.141 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:54.141 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:54.141 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:54.141 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1078 2024-05-08T08:54:54.141 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.141 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:54.141 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.141 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:54.141 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:54:54.141 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:54.142 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.142 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.142 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:54.142 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:54:54.142 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:54.142 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.142 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.142 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:54.142 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:54:54.142 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5600000000000001e-07 2024-05-08T08:54:54.142 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.142 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.142 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:54.142 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485547+0000", 2024-05-08T08:54:54.142 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3590000000000002e-06 2024-05-08T08:54:54.142 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.143 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.143 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:54.143 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485553+0000", 2024-05-08T08:54:54.143 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2770000000000002e-06 2024-05-08T08:54:54.143 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.143 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.143 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:54.143 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485558+0000", 2024-05-08T08:54:54.143 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.95e-06 2024-05-08T08:54:54.143 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.143 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.143 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:54.143 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556582+0000", 2024-05-08T08:54:54.143 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071024444000000006 2024-05-08T08:54:54.144 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.144 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.144 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:54.144 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556637+0000", 2024-05-08T08:54:54.144 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4672000000000001e-05 2024-05-08T08:54:54.144 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.144 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.144 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:54.144 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556711+0000", 2024-05-08T08:54:54.144 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4117999999999997e-05 2024-05-08T08:54:54.144 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.144 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:54.144 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.144 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.144 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.144 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1080 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11190272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:54.145 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:54:54.145 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 137.637220283, 2024-05-08T08:54:54.145 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071350721000000006, 2024-05-08T08:54:54.145 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:54.145 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:54.145 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:54.145 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:54.145 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:54.145 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:54.145 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1080 2024-05-08T08:54:54.145 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.145 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:54.145 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.145 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:54.145 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:54:54.145 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:54.146 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.146 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.146 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:54.146 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:54:54.146 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:54.146 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.146 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.146 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:54.146 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:54:54.146 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.03e-07 2024-05-08T08:54:54.146 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.146 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.146 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:54.146 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485567+0000", 2024-05-08T08:54:54.147 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1090000000000001e-06 2024-05-08T08:54:54.147 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.147 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.147 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:54.147 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485573+0000", 2024-05-08T08:54:54.147 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3049999999999998e-06 2024-05-08T08:54:54.147 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.147 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.147 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:54.147 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485578+0000", 2024-05-08T08:54:54.147 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0130000000000001e-06 2024-05-08T08:54:54.147 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.147 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.147 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:54.147 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556743+0000", 2024-05-08T08:54:54.147 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071164449000000005 2024-05-08T08:54:54.148 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.148 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.148 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:54.148 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556797+0000", 2024-05-08T08:54:54.148 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4589999999999997e-05 2024-05-08T08:54:54.148 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.148 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.148 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:54.148 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556915+0000", 2024-05-08T08:54:54.148 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000117852 2024-05-08T08:54:54.148 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.148 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:54.148 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.148 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.148 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.149 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1082 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11231232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:54.149 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:54:54.149 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 137.63720004000001, 2024-05-08T08:54:54.149 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071591396000000002, 2024-05-08T08:54:54.149 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:54.149 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:54.149 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:54.149 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:54.149 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:54.149 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:54.149 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1082 2024-05-08T08:54:54.149 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.149 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:54.149 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.149 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:54.149 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:54:54.149 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:54.150 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.150 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.150 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:54.150 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:54:54.150 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:54.150 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.150 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.150 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:54.150 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:54:54.150 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1800000000000001e-07 2024-05-08T08:54:54.150 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.150 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.150 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:54.150 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485587+0000", 2024-05-08T08:54:54.150 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2589999999999999e-06 2024-05-08T08:54:54.150 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.151 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.151 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:54.151 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485594+0000", 2024-05-08T08:54:54.151 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0789999999999997e-06 2024-05-08T08:54:54.151 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.151 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.151 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:54.151 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485598+0000", 2024-05-08T08:54:54.151 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7480000000000001e-06 2024-05-08T08:54:54.151 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.151 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.151 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:54.151 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556971+0000", 2024-05-08T08:54:54.151 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071372774999999999 2024-05-08T08:54:54.151 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.151 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.151 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:54.152 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557069+0000", 2024-05-08T08:54:54.152 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.7732999999999994e-05 2024-05-08T08:54:54.152 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.152 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.152 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:54.152 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557176+0000", 2024-05-08T08:54:54.152 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000107384 2024-05-08T08:54:54.152 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.152 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:54.152 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.152 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.152 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.152 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1084 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11272192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:54.152 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:54:54.152 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 137.63716140099999, 2024-05-08T08:54:54.152 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071808977999999996, 2024-05-08T08:54:54.153 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:54.153 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:54.153 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:54.153 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:54.153 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:54.153 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:54.153 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1084 2024-05-08T08:54:54.153 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.153 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:54.153 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.153 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:54.153 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:54:54.153 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:54.153 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.153 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.153 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:54.153 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:54:54.154 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:54.154 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.154 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.154 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:54.154 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485624+0000", 2024-05-08T08:54:54.154 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:54:54.154 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.154 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.154 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:54.154 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485626+0000", 2024-05-08T08:54:54.154 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9460000000000001e-06 2024-05-08T08:54:54.154 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.154 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.154 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:54.154 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485633+0000", 2024-05-08T08:54:54.154 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1149999999999996e-06 2024-05-08T08:54:54.155 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.155 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.155 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:54.155 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485638+0000", 2024-05-08T08:54:54.155 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5670000000000002e-06 2024-05-08T08:54:54.155 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.155 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.155 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:54.155 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557226+0000", 2024-05-08T08:54:54.155 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071587267999999996 2024-05-08T08:54:54.155 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.155 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.155 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:54.155 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557321+0000", 2024-05-08T08:54:54.155 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5785000000000005e-05 2024-05-08T08:54:54.156 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.156 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.156 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:54.156 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557432+0000", 2024-05-08T08:54:54.156 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011092800000000001 2024-05-08T08:54:54.156 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.156 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:54.156 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.156 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.156 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.156 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1086 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11313152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:54.156 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:54:54.156 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 137.63713807400001, 2024-05-08T08:54:54.156 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071933132999999996, 2024-05-08T08:54:54.156 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:54.156 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:54.156 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:54.157 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:54.157 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:54.157 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:54.157 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1086 2024-05-08T08:54:54.157 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.157 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:54.157 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.157 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:54.157 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:54:54.157 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:54.157 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.157 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.157 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:54.157 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:54:54.157 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:54.157 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.157 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.158 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:54.158 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486328+0000", 2024-05-08T08:54:54.158 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00068107900000000002 2024-05-08T08:54:54.158 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.158 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.158 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:54.158 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486331+0000", 2024-05-08T08:54:54.158 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.585e-06 2024-05-08T08:54:54.158 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.158 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.158 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:54.158 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486343+0000", 2024-05-08T08:54:54.158 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1856e-05 2024-05-08T08:54:54.158 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.158 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.158 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:54.158 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486350+0000", 2024-05-08T08:54:54.159 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8600000000000004e-06 2024-05-08T08:54:54.159 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.159 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.159 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:54.159 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557470+0000", 2024-05-08T08:54:54.159 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071120326999999997 2024-05-08T08:54:54.159 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.159 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.159 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:54.159 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557514+0000", 2024-05-08T08:54:54.159 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3909000000000003e-05 2024-05-08T08:54:54.159 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.159 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.159 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:54.159 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557580+0000", 2024-05-08T08:54:54.159 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5517000000000003e-05 2024-05-08T08:54:54.160 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.160 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:54.160 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.160 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.160 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.160 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1089 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11354112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:54.160 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:54:54.160 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 137.636425202, 2024-05-08T08:54:54.160 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071457381, 2024-05-08T08:54:54.160 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:54.160 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:54.160 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:54.160 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:54.160 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:54.160 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:54.160 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1089 2024-05-08T08:54:54.160 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.161 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:54.161 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.161 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:54.161 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:54:54.161 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:54.161 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.161 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.161 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:54.161 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:54:54.161 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:54.161 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.161 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.161 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:54.161 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486851+0000", 2024-05-08T08:54:54.161 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00049102 2024-05-08T08:54:54.162 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.162 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.162 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:54.162 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486854+0000", 2024-05-08T08:54:54.162 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6859999999999999e-06 2024-05-08T08:54:54.162 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.162 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.162 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:54.162 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486864+0000", 2024-05-08T08:54:54.162 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6080000000000002e-06 2024-05-08T08:54:54.162 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.162 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.162 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:54.162 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486870+0000", 2024-05-08T08:54:54.162 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.477e-06 2024-05-08T08:54:54.162 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.162 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.163 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:54.163 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557612+0000", 2024-05-08T08:54:54.163 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070741557999999996 2024-05-08T08:54:54.163 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.163 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.163 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:54.163 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557695+0000", 2024-05-08T08:54:54.163 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3300000000000005e-05 2024-05-08T08:54:54.163 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.163 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.163 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:54.163 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557817+0000", 2024-05-08T08:54:54.163 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012173200000000001 2024-05-08T08:54:54.163 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.163 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:54.163 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.164 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.164 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.164 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1091 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11395072~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:54.164 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:54:54.164 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 137.635904407, 2024-05-08T08:54:54.164 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071184737999999997, 2024-05-08T08:54:54.164 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:54.164 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:54.164 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:54.164 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:54.164 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:54.164 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:54.164 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1091 2024-05-08T08:54:54.164 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.164 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:54.164 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.164 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:54.165 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:54:54.165 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:54.165 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.165 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.165 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:54.165 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:54:54.165 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:54.165 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.165 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.165 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:54.165 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486881+0000", 2024-05-08T08:54:54.165 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0699999999999997e-07 2024-05-08T08:54:54.165 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.165 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.165 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:54.165 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486883+0000", 2024-05-08T08:54:54.166 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.402e-06 2024-05-08T08:54:54.166 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.166 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.166 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:54.166 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486890+0000", 2024-05-08T08:54:54.166 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0480000000000001e-06 2024-05-08T08:54:54.166 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.166 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.166 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:54.166 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486896+0000", 2024-05-08T08:54:54.166 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7640000000000002e-06 2024-05-08T08:54:54.166 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.166 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.166 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:54.166 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557860+0000", 2024-05-08T08:54:54.166 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070963791999999998 2024-05-08T08:54:54.167 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.167 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.167 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:54.167 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557952+0000", 2024-05-08T08:54:54.167 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2082000000000004e-05 2024-05-08T08:54:54.167 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.167 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.167 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:54.167 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558065+0000", 2024-05-08T08:54:54.167 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000113143 2024-05-08T08:54:54.167 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.167 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:54.167 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.167 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.167 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.167 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1094 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11436032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:54.167 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:54:54.168 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 137.63588123900001, 2024-05-08T08:54:54.168 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071400507000000002, 2024-05-08T08:54:54.168 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:54.168 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:54.168 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:54.168 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:54.168 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:54.168 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:54.168 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1094 2024-05-08T08:54:54.168 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.168 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:54.168 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.168 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:54.168 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:54:54.168 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:54.168 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.168 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.169 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:54.169 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:54:54.169 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:54.169 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.169 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.169 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:54.169 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487940+0000", 2024-05-08T08:54:54.169 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001036868 2024-05-08T08:54:54.169 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.169 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.169 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:54.169 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487944+0000", 2024-05-08T08:54:54.169 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7069999999999998e-06 2024-05-08T08:54:54.169 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.169 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.169 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:54.170 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487951+0000", 2024-05-08T08:54:54.170 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2649999999999999e-06 2024-05-08T08:54:54.170 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.170 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.170 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:54.170 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487961+0000", 2024-05-08T08:54:54.170 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4110000000000002e-06 2024-05-08T08:54:54.170 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.170 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.170 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:54.170 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558117+0000", 2024-05-08T08:54:54.170 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070156662999999994 2024-05-08T08:54:54.170 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.170 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.170 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:54.170 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558214+0000", 2024-05-08T08:54:54.171 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6444e-05 2024-05-08T08:54:54.171 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.171 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.171 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:54.171 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558304+0000", 2024-05-08T08:54:54.171 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0148999999999994e-05 2024-05-08T08:54:54.171 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.171 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:54.171 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.171 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.171 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.171 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1096 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11476992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:54.171 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:54:54.171 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 137.63481470799999, 2024-05-08T08:54:54.171 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070480074000000004, 2024-05-08T08:54:54.171 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:54.172 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:54.172 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:54.172 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:54.172 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:54.172 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:54.172 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1096 2024-05-08T08:54:54.172 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.172 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:54.172 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.172 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:54.172 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:54:54.172 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:54.172 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.172 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.172 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:54.172 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:54:54.173 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:54.173 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.173 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.173 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:54.173 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:54:54.173 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3700000000000001e-07 2024-05-08T08:54:54.173 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.173 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.173 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:54.173 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487973+0000", 2024-05-08T08:54:54.173 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0870000000000001e-06 2024-05-08T08:54:54.173 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.173 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.173 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:54.173 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487977+0000", 2024-05-08T08:54:54.174 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0600000000000001e-06 2024-05-08T08:54:54.174 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.174 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.174 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:54.174 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487980+0000", 2024-05-08T08:54:54.174 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1089999999999998e-06 2024-05-08T08:54:54.174 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.174 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.174 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:54.174 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558338+0000", 2024-05-08T08:54:54.174 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070358233000000006 2024-05-08T08:54:54.174 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.174 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.174 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:54.174 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558392+0000", 2024-05-08T08:54:54.175 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4537999999999999e-05 2024-05-08T08:54:54.175 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.175 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.175 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:54.175 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558450+0000", 2024-05-08T08:54:54.175 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7710000000000001e-05 2024-05-08T08:54:54.175 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.175 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:54.175 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.175 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.175 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.175 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1099 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11517952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:54.175 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:54:54.175 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 137.634800832, 2024-05-08T08:54:54.175 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070720506000000002, 2024-05-08T08:54:54.176 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:54.176 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:54.176 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:54.176 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:54.176 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:54.176 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:54.176 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1099 2024-05-08T08:54:54.176 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.176 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:54.176 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.176 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:54.176 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:54:54.176 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:54.176 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.177 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.177 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:54.177 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:54:54.177 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:54.177 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.177 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.177 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:54.177 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:54:54.177 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3999999999999998e-07 2024-05-08T08:54:54.177 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.177 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.177 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:54.177 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487986+0000", 2024-05-08T08:54:54.177 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3519999999999999e-06 2024-05-08T08:54:54.177 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.178 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.178 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:54.178 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487989+0000", 2024-05-08T08:54:54.178 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.23e-06 2024-05-08T08:54:54.178 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.178 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.178 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:54.178 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487991+0000", 2024-05-08T08:54:54.178 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6350000000000002e-06 2024-05-08T08:54:54.178 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.178 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.178 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:54.178 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558489+0000", 2024-05-08T08:54:54.178 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070497924000000003 2024-05-08T08:54:54.178 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.178 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.179 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:54.179 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558583+0000", 2024-05-08T08:54:54.179 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3405000000000004e-05 2024-05-08T08:54:54.179 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.179 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.179 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:54.179 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558704+0000", 2024-05-08T08:54:54.179 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012171999999999999 2024-05-08T08:54:54.179 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.179 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:54.179 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.179 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.179 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.179 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1100 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11558912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:54.180 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:54:54.180 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 137.63478783299999, 2024-05-08T08:54:54.180 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070962266999999996, 2024-05-08T08:54:54.180 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:54.180 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:54.180 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:54.180 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:54.180 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:54.180 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:54.180 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1100 2024-05-08T08:54:54.180 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.180 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:54.180 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.180 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:54.181 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:54:54.181 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:54.181 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.181 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.181 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:54.181 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:54:54.181 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:54.181 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.181 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.181 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:54.181 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488430+0000", 2024-05-08T08:54:54.181 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000432676 2024-05-08T08:54:54.181 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.182 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.182 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:54.182 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488433+0000", 2024-05-08T08:54:54.182 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.45e-06 2024-05-08T08:54:54.182 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.182 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.182 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:54.182 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488442+0000", 2024-05-08T08:54:54.182 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.687e-06 2024-05-08T08:54:54.182 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.182 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.182 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:54.182 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488446+0000", 2024-05-08T08:54:54.182 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4939999999999997e-06 2024-05-08T08:54:54.182 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.182 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.183 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:54.183 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558756+0000", 2024-05-08T08:54:54.183 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070310051999999998 2024-05-08T08:54:54.183 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.183 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.183 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:54.183 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558844+0000", 2024-05-08T08:54:54.183 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7690999999999998e-05 2024-05-08T08:54:54.183 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.183 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.183 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:54.183 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558959+0000", 2024-05-08T08:54:54.183 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000115217 2024-05-08T08:54:54.183 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.183 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:54.183 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.184 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.184 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.184 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1103 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11599872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:54.184 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:54:54.184 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 137.63433015800001, 2024-05-08T08:54:54.184 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070777100999999995, 2024-05-08T08:54:54.184 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:54.184 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:54.184 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:54.184 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:54.184 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:54.184 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:54.184 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1103 2024-05-08T08:54:54.184 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.184 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:54.184 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.185 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:54.185 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:54:54.185 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:54.185 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.185 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.185 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:54.185 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:54:54.185 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:54.185 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.185 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.185 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:54.185 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488959+0000", 2024-05-08T08:54:54.185 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00050457200000000001 2024-05-08T08:54:54.185 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.185 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.185 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:54.186 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488969+0000", 2024-05-08T08:54:54.186 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.7159999999999998e-06 2024-05-08T08:54:54.186 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.186 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.186 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:54.186 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488981+0000", 2024-05-08T08:54:54.186 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2150000000000001e-05 2024-05-08T08:54:54.186 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.186 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.186 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:54.186 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488988+0000", 2024-05-08T08:54:54.186 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8759999999999999e-06 2024-05-08T08:54:54.186 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.187 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.187 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:54.187 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558992+0000", 2024-05-08T08:54:54.187 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070004358000000003 2024-05-08T08:54:54.187 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.187 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.187 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:54.187 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559093+0000", 2024-05-08T08:54:54.187 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000100299 2024-05-08T08:54:54.187 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.187 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.187 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:54.187 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559232+0000", 2024-05-08T08:54:54.187 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013913 2024-05-08T08:54:54.187 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.188 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:54.188 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.188 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.188 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.188 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1104 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11640832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:54:54.188 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:54:54.188 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 137.633785343, 2024-05-08T08:54:54.188 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070521947000000001, 2024-05-08T08:54:54.188 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:54.188 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:54.188 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:54.188 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:54.188 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:54:54.188 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:54:54.188 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1104 2024-05-08T08:54:54.188 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.189 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:54.189 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.189 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:54.189 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:54:54.189 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:54.189 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.189 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.189 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:54.189 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:54:54.189 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:54.189 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.189 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.189 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:54.189 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489700+0000", 2024-05-08T08:54:54.190 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00070067600000000001 2024-05-08T08:54:54.190 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.190 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.190 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:54.190 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489704+0000", 2024-05-08T08:54:54.190 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1640000000000003e-06 2024-05-08T08:54:54.190 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.190 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.190 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:54.190 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489716+0000", 2024-05-08T08:54:54.190 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1379000000000001e-05 2024-05-08T08:54:54.190 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.190 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.191 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:54.191 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489724+0000", 2024-05-08T08:54:54.191 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8949999999999998e-06 2024-05-08T08:54:54.191 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.191 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.191 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:54.191 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559278+0000", 2024-05-08T08:54:54.191 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.069554151999999994 2024-05-08T08:54:54.191 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.191 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.191 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:54.191 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559387+0000", 2024-05-08T08:54:54.191 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000109258 2024-05-08T08:54:54.191 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.191 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.191 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:54.192 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559521+0000", 2024-05-08T08:54:54.192 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000134423 2024-05-08T08:54:54.192 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.192 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:54.192 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.192 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.192 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.192 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:8636 3.17 3:e8a0a3e4:::rbd_header.11402b26385:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-05-08T08:54:54.192 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:54.150971+0000", 2024-05-08T08:54:54.192 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 119.971813349, 2024-05-08T08:54:54.192 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.217273233, 2024-05-08T08:54:54.192 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:54.192 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:54.192 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:54.192 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:54.192 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:54:54.193 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:54:54.193 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 8636 2024-05-08T08:54:54.193 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.193 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:54.193 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.193 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:54.193 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150971+0000", 2024-05-08T08:54:54.193 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:54.193 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.193 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.193 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:54.193 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150971+0000", 2024-05-08T08:54:54.193 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:54.193 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.193 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.194 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:54.194 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150972+0000", 2024-05-08T08:54:54.194 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.71e-07 2024-05-08T08:54:54.194 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.194 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.194 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:54.194 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150977+0000", 2024-05-08T08:54:54.194 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.152e-06 2024-05-08T08:54:54.194 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.194 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.194 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:54.194 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150990+0000", 2024-05-08T08:54:54.194 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2364999999999999e-05 2024-05-08T08:54:54.194 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.194 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.195 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:54.195 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150998+0000", 2024-05-08T08:54:54.195 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0739999999999993e-06 2024-05-08T08:54:54.195 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.195 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.195 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:54.195 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.151054+0000", 2024-05-08T08:54:54.195 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5949000000000003e-05 2024-05-08T08:54:54.195 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.195 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.195 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:54:54.195 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.151060+0000", 2024-05-08T08:54:54.195 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6120000000000004e-06 2024-05-08T08:54:54.195 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.195 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.196 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:54.196 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.367640+0000", 2024-05-08T08:54:54.196 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2165792739999999 2024-05-08T08:54:54.196 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.196 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.196 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:54.196 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.367839+0000", 2024-05-08T08:54:54.196 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000199185 2024-05-08T08:54:54.196 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.196 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.196 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:54.196 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.368245+0000", 2024-05-08T08:54:54.196 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00040575099999999998 2024-05-08T08:54:54.196 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.196 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:54.197 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.197 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.197 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.197 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5302.0:1516 3.c 3:319bf789:::rbd_data.1140d3fc54c.0000000000000025:head [stat,write 1892352~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:54:54.197 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:27.822220+0000", 2024-05-08T08:54:54.197 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 86.300564412, 2024-05-08T08:54:54.197 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.092643744, 2024-05-08T08:54:54.197 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:54.197 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:54.197 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:54:54.197 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:54.197 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5302", 2024-05-08T08:54:54.197 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/359277859", 2024-05-08T08:54:54.197 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1516 2024-05-08T08:54:54.197 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.198 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:54.198 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.198 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:54.198 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822220+0000", 2024-05-08T08:54:54.198 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:54.198 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.198 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.198 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:54.198 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822220+0000", 2024-05-08T08:54:54.198 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:54.198 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.198 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.198 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:54.199 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822221+0000", 2024-05-08T08:54:54.199 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3500000000000002e-07 2024-05-08T08:54:54.199 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.199 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.199 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:54.199 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822228+0000", 2024-05-08T08:54:54.199 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9959999999999996e-06 2024-05-08T08:54:54.199 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.199 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.199 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:54.199 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822234+0000", 2024-05-08T08:54:54.199 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5429999999999999e-06 2024-05-08T08:54:54.199 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.199 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.199 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:54.200 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822238+0000", 2024-05-08T08:54:54.200 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.732e-06 2024-05-08T08:54:54.200 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.200 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.200 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:54.200 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822285+0000", 2024-05-08T08:54:54.200 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6810000000000001e-05 2024-05-08T08:54:54.200 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.200 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.200 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:54.200 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822358+0000", 2024-05-08T08:54:54.200 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3473000000000006e-05 2024-05-08T08:54:54.200 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.200 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.200 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 5", 2024-05-08T08:54:54.200 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822489+0000", 2024-05-08T08:54:54.200 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013011199999999999 2024-05-08T08:54:54.201 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.201 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.201 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:54:54.201 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.825332+0000", 2024-05-08T08:54:54.201 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0028434799999999998 2024-05-08T08:54:54.201 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.201 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.201 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:54:54.201 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.914753+0000", 2024-05-08T08:54:54.201 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089421424999999999 2024-05-08T08:54:54.201 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.201 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.201 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:54:54.201 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.914807+0000", 2024-05-08T08:54:54.202 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3829000000000002e-05 2024-05-08T08:54:54.202 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.202 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.202 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:54.202 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.914864+0000", 2024-05-08T08:54:54.202 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6909e-05 2024-05-08T08:54:54.202 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.202 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:54.202 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.202 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.202 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.202 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:24567 3.6 3:66af1c40:::rbd_data.1140d3fc54c.000000000000005a:18 [read 2080768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:54.202 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.216598+0000", 2024-05-08T08:54:54.202 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 81.906186332000004, 2024-05-08T08:54:54.203 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071602647000000005, 2024-05-08T08:54:54.203 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:54.203 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:54.203 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:54.203 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:54.203 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:54.203 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:54.203 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 24567 2024-05-08T08:54:54.203 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.203 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:54.203 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.203 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:54.203 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216598+0000", 2024-05-08T08:54:54.203 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:54.203 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.203 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.204 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:54.204 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216598+0000", 2024-05-08T08:54:54.204 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:54.204 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.204 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.204 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:54.204 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216599+0000", 2024-05-08T08:54:54.204 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8999999999999997e-07 2024-05-08T08:54:54.204 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.204 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.204 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:54.204 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216601+0000", 2024-05-08T08:54:54.204 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9889999999999999e-06 2024-05-08T08:54:54.204 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.204 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.204 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:54.205 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216607+0000", 2024-05-08T08:54:54.205 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3600000000000001e-06 2024-05-08T08:54:54.205 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.205 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.205 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:54.205 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216612+0000", 2024-05-08T08:54:54.205 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.69e-06 2024-05-08T08:54:54.205 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.205 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.205 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:54.205 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.287076+0000", 2024-05-08T08:54:54.205 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070464355000000006 2024-05-08T08:54:54.205 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.205 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.205 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:54.205 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.287151+0000", 2024-05-08T08:54:54.206 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4924000000000003e-05 2024-05-08T08:54:54.206 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.206 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.206 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:54.206 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.288201+0000", 2024-05-08T08:54:54.206 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0010498389999999999 2024-05-08T08:54:54.206 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.206 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:54.206 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.206 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.206 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:54.206 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:54.207 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:54.210+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:54.208 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:54:54.308 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:54.311+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:54.390 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:54.390 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:54:54.390 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:54:54.390 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:54.401 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:54:54.409 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:54.412+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:54.427 INFO:teuthology.orchestra.run.smithi059.stderr:noscrub is unset 2024-05-08T08:54:54.510 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:54.513+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:54.533 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:54.533 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:54:54.533 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:54:54.533 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:54:54.533 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:54.543 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:54:54.611 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:54.614+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:54.668 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:54.668 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:54:54.668 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:54:54.669 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:54:54.669 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.669 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:12443 3.0 3:02d0158e:::rbd_data.114071bcf828.0000000000000037:head [stat,delete] snapc 19=[19] RETRY=3 ondisk+retry+write+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:54.669 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.610027+0000", 2024-05-08T08:54:54.669 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 83.059718825000004, 2024-05-08T08:54:54.669 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.083586333999999998, 2024-05-08T08:54:54.669 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:54.669 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:54.669 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:54.669 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:54.669 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:54.669 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:54.669 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 12443 2024-05-08T08:54:54.669 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.669 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:54.670 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.670 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:54.670 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610027+0000", 2024-05-08T08:54:54.670 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:54.670 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.670 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.670 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:54.670 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610027+0000", 2024-05-08T08:54:54.670 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:54.670 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.670 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.670 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:54.670 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610028+0000", 2024-05-08T08:54:54.670 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.13e-07 2024-05-08T08:54:54.671 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.671 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.671 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:54.671 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610030+0000", 2024-05-08T08:54:54.671 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7669999999999999e-06 2024-05-08T08:54:54.671 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.671 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.671 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:54.671 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610039+0000", 2024-05-08T08:54:54.671 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4959999999999996e-06 2024-05-08T08:54:54.671 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.671 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.671 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:54.671 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610045+0000", 2024-05-08T08:54:54.672 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6779999999999997e-06 2024-05-08T08:54:54.672 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.672 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.672 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:54.672 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.660873+0000", 2024-05-08T08:54:54.672 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.050828748 2024-05-08T08:54:54.672 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.672 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.672 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:54.672 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.661017+0000", 2024-05-08T08:54:54.672 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014391999999999999 2024-05-08T08:54:54.672 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.672 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.672 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:54.672 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.693613+0000", 2024-05-08T08:54:54.673 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.032596111999999997 2024-05-08T08:54:54.673 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.673 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:54.673 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.673 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.673 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.673 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:14301 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [watch reconnect cookie 139837155901536 gen 12] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:54.673 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.645078+0000", 2024-05-08T08:54:54.673 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 83.024668187000003, 2024-05-08T08:54:54.673 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088281852999999993, 2024-05-08T08:54:54.673 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:54.673 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:54.673 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:54.673 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:54.673 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:54.674 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:54.674 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14301 2024-05-08T08:54:54.674 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.674 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:54.674 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.674 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:54.674 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645078+0000", 2024-05-08T08:54:54.674 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:54.674 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.674 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.674 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:54.674 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645078+0000", 2024-05-08T08:54:54.674 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:54.674 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.675 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.675 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:54.675 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645084+0000", 2024-05-08T08:54:54.675 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4620000000000001e-06 2024-05-08T08:54:54.675 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.675 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.675 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:54.675 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645088+0000", 2024-05-08T08:54:54.675 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8039999999999999e-06 2024-05-08T08:54:54.675 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.675 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.675 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:54.675 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645106+0000", 2024-05-08T08:54:54.676 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.8318999999999999e-05 2024-05-08T08:54:54.676 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.676 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.676 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:54.676 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645113+0000", 2024-05-08T08:54:54.676 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0310000000000001e-06 2024-05-08T08:54:54.676 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.676 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.676 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:54.676 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.670493+0000", 2024-05-08T08:54:54.676 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.025379993 2024-05-08T08:54:54.676 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.676 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.676 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:54.677 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.670571+0000", 2024-05-08T08:54:54.677 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7613000000000004e-05 2024-05-08T08:54:54.677 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.677 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.677 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:54.677 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733360+0000", 2024-05-08T08:54:54.677 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.062788630999999998 2024-05-08T08:54:54.677 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.677 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:54.677 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.677 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.677 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.677 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:14420 3.0 3:06a8948e:::rbd_data.114071bcf828.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:54.677 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.649639+0000", 2024-05-08T08:54:54.678 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 83.020107038999996, 2024-05-08T08:54:54.678 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.083851897999999994, 2024-05-08T08:54:54.678 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:54.678 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:54.678 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:54:54.678 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:54.678 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:54.678 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:54.678 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14420 2024-05-08T08:54:54.678 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.678 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:54.678 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.678 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:54.678 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649639+0000", 2024-05-08T08:54:54.679 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:54.679 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.679 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.679 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:54.679 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649639+0000", 2024-05-08T08:54:54.679 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:54.679 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.679 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.679 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:54.679 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649640+0000", 2024-05-08T08:54:54.679 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6899999999999997e-07 2024-05-08T08:54:54.679 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.679 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.679 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:54.679 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649895+0000", 2024-05-08T08:54:54.679 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00025531000000000002 2024-05-08T08:54:54.680 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.680 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.680 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:54.680 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649905+0000", 2024-05-08T08:54:54.680 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0198e-05 2024-05-08T08:54:54.680 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.680 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.680 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:54.680 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649911+0000", 2024-05-08T08:54:54.680 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2310000000000001e-06 2024-05-08T08:54:54.680 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.681 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.681 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:54.681 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.684099+0000", 2024-05-08T08:54:54.681 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.034188017000000001 2024-05-08T08:54:54.681 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.681 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.681 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:54.681 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.684172+0000", 2024-05-08T08:54:54.681 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3201999999999995e-05 2024-05-08T08:54:54.681 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.681 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.681 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 2", 2024-05-08T08:54:54.681 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.684418+0000", 2024-05-08T08:54:54.681 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00024559099999999997 2024-05-08T08:54:54.681 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.681 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.682 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:54:54.682 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.705538+0000", 2024-05-08T08:54:54.682 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.021119707000000001 2024-05-08T08:54:54.682 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.682 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.682 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:54:54.682 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733399+0000", 2024-05-08T08:54:54.682 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.027861543999999999 2024-05-08T08:54:54.682 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.682 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.682 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:54:54.682 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733445+0000", 2024-05-08T08:54:54.682 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5587000000000002e-05 2024-05-08T08:54:54.682 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.683 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.683 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:54.683 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733491+0000", 2024-05-08T08:54:54.683 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5841999999999999e-05 2024-05-08T08:54:54.683 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.683 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:54.683 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.683 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.683 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.683 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28547 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889179216] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:54.683 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.355654+0000", 2024-05-08T08:54:54.683 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 82.314091810999997, 2024-05-08T08:54:54.683 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078692261999999999, 2024-05-08T08:54:54.683 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:54.683 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:54.683 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:54.684 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:54.684 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:54.684 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:54.684 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28547 2024-05-08T08:54:54.684 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.684 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:54.684 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.684 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:54.684 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355654+0000", 2024-05-08T08:54:54.684 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:54.684 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.684 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.684 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:54.684 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355654+0000", 2024-05-08T08:54:54.684 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:54.684 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.685 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.685 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:54.685 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355654+0000", 2024-05-08T08:54:54.685 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3999999999999998e-07 2024-05-08T08:54:54.685 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.685 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.685 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:54.685 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355656+0000", 2024-05-08T08:54:54.685 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.353e-06 2024-05-08T08:54:54.685 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.685 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.685 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:54.685 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355663+0000", 2024-05-08T08:54:54.685 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2259999999999999e-06 2024-05-08T08:54:54.685 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.686 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.686 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:54.686 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355666+0000", 2024-05-08T08:54:54.686 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7930000000000002e-06 2024-05-08T08:54:54.686 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.686 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.686 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:54.686 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434117+0000", 2024-05-08T08:54:54.686 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078451231999999996 2024-05-08T08:54:54.686 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.686 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.686 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:54.686 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434198+0000", 2024-05-08T08:54:54.686 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0879999999999998e-05 2024-05-08T08:54:54.686 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.686 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.687 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:54.687 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434346+0000", 2024-05-08T08:54:54.687 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014853800000000001 2024-05-08T08:54:54.687 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.687 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:54.687 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.687 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.687 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.687 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28548 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889189696] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:54.687 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.355669+0000", 2024-05-08T08:54:54.687 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 82.314076460999999, 2024-05-08T08:54:54.687 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078956606999999998, 2024-05-08T08:54:54.687 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:54.687 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:54.687 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:54.688 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:54.688 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:54.688 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:54.688 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28548 2024-05-08T08:54:54.688 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.688 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:54.688 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.688 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:54.688 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355669+0000", 2024-05-08T08:54:54.688 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:54.688 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.688 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.688 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:54.688 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355669+0000", 2024-05-08T08:54:54.688 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:54.688 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.689 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.689 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:54.689 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355762+0000", 2024-05-08T08:54:54.689 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3015000000000003e-05 2024-05-08T08:54:54.689 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.689 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.689 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:54.689 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355765+0000", 2024-05-08T08:54:54.689 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.368e-06 2024-05-08T08:54:54.689 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.689 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.689 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:54.689 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355771+0000", 2024-05-08T08:54:54.689 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5989999999999998e-06 2024-05-08T08:54:54.689 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.689 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.690 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:54.690 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355775+0000", 2024-05-08T08:54:54.690 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7280000000000001e-06 2024-05-08T08:54:54.690 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.690 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.690 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:54.690 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434379+0000", 2024-05-08T08:54:54.690 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078603662000000005 2024-05-08T08:54:54.690 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.690 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.690 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:54.690 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434473+0000", 2024-05-08T08:54:54.690 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4104000000000006e-05 2024-05-08T08:54:54.690 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.690 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.691 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:54.691 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434626+0000", 2024-05-08T08:54:54.691 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00015313100000000001 2024-05-08T08:54:54.691 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.691 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:54.691 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.691 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.691 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.691 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28549 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889202400] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:54.691 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.355781+0000", 2024-05-08T08:54:54.691 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 82.313964886999997, 2024-05-08T08:54:54.691 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079117946999999994, 2024-05-08T08:54:54.691 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:54.691 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:54.691 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:54.691 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:54.691 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:54.692 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:54.692 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28549 2024-05-08T08:54:54.692 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.692 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:54.692 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.692 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:54.692 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355781+0000", 2024-05-08T08:54:54.692 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:54.692 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.692 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.692 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:54.692 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355781+0000", 2024-05-08T08:54:54.692 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:54.692 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.692 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.692 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:54.692 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356162+0000", 2024-05-08T08:54:54.693 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00038099100000000001 2024-05-08T08:54:54.693 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.693 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.693 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:54.693 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356165+0000", 2024-05-08T08:54:54.693 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1240000000000001e-06 2024-05-08T08:54:54.693 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.693 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.693 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:54.693 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356173+0000", 2024-05-08T08:54:54.693 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.6199999999999999e-06 2024-05-08T08:54:54.693 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.693 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.693 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:54.693 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356182+0000", 2024-05-08T08:54:54.693 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7760000000000003e-06 2024-05-08T08:54:54.693 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.694 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.694 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:54.694 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434675+0000", 2024-05-08T08:54:54.694 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078493119 2024-05-08T08:54:54.694 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.694 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.694 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:54.694 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434765+0000", 2024-05-08T08:54:54.694 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0402000000000006e-05 2024-05-08T08:54:54.694 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.694 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.694 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:54.694 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434899+0000", 2024-05-08T08:54:54.694 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013391499999999999 2024-05-08T08:54:54.694 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.695 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:54.695 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.695 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.695 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.695 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28550 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888597280] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:54.695 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:54:54.695 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 82.313553841000001, 2024-05-08T08:54:54.695 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078933990999999995, 2024-05-08T08:54:54.695 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:54.695 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:54.695 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:54.695 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:54.695 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:54.695 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:54.695 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28550 2024-05-08T08:54:54.695 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.696 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:54.696 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.696 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:54.696 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:54:54.696 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:54.696 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.696 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.696 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:54.696 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:54:54.696 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:54.696 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.696 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.696 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:54.696 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:54:54.696 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.34e-07 2024-05-08T08:54:54.696 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.697 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.697 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:54.697 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356195+0000", 2024-05-08T08:54:54.697 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8710000000000001e-06 2024-05-08T08:54:54.697 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.697 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.697 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:54.697 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356200+0000", 2024-05-08T08:54:54.697 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3000000000000003e-06 2024-05-08T08:54:54.697 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.697 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.697 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:54.697 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356203+0000", 2024-05-08T08:54:54.697 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4039999999999999e-06 2024-05-08T08:54:54.697 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.697 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.698 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:54.698 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434928+0000", 2024-05-08T08:54:54.698 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078725331999999995 2024-05-08T08:54:54.698 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.698 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.698 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:54.698 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435007+0000", 2024-05-08T08:54:54.698 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8415000000000001e-05 2024-05-08T08:54:54.698 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.698 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.698 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:54.698 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435126+0000", 2024-05-08T08:54:54.699 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011923500000000001 2024-05-08T08:54:54.699 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.699 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:54.699 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.699 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.699 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.699 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28551 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888527200] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:54.699 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.356210+0000", 2024-05-08T08:54:54.699 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 82.313535493000003, 2024-05-08T08:54:54.699 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079195072000000005, 2024-05-08T08:54:54.699 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:54.699 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:54.699 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:54.699 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:54.700 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:54.700 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:54.700 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28551 2024-05-08T08:54:54.700 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.700 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:54.700 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.700 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:54.700 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356210+0000", 2024-05-08T08:54:54.700 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:54.700 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.701 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.701 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:54.701 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356210+0000", 2024-05-08T08:54:54.701 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:54.701 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.701 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.701 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:54.701 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356211+0000", 2024-05-08T08:54:54.701 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4499999999999998e-07 2024-05-08T08:54:54.702 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.702 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.702 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:54.702 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356212+0000", 2024-05-08T08:54:54.702 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6029999999999999e-06 2024-05-08T08:54:54.702 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.702 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.702 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:54.702 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356221+0000", 2024-05-08T08:54:54.702 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4200000000000007e-06 2024-05-08T08:54:54.702 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.702 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.702 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:54.702 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356224+0000", 2024-05-08T08:54:54.702 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.038e-06 2024-05-08T08:54:54.703 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.703 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.703 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:54.703 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435178+0000", 2024-05-08T08:54:54.703 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078954625000000001 2024-05-08T08:54:54.703 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.703 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.703 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:54.703 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435256+0000", 2024-05-08T08:54:54.703 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7337999999999997e-05 2024-05-08T08:54:54.703 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.703 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.703 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:54.703 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435405+0000", 2024-05-08T08:54:54.703 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000149703 2024-05-08T08:54:54.703 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.703 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:54.704 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.704 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.704 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.704 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28552 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889231456] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:54.704 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.356228+0000", 2024-05-08T08:54:54.704 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 82.313517814999997, 2024-05-08T08:54:54.704 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079346920000000001, 2024-05-08T08:54:54.704 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:54.704 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:54.704 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:54.704 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:54.704 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:54.704 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:54.704 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28552 2024-05-08T08:54:54.704 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.704 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:54.705 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.705 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:54.705 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356228+0000", 2024-05-08T08:54:54.705 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:54.705 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.705 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.705 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:54.705 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356228+0000", 2024-05-08T08:54:54.705 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:54.705 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.705 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.705 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:54.705 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356959+0000", 2024-05-08T08:54:54.705 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00073097099999999996 2024-05-08T08:54:54.705 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.705 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.705 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:54.705 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356967+0000", 2024-05-08T08:54:54.706 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0409999999999998e-06 2024-05-08T08:54:54.706 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.706 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.706 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:54.706 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356979+0000", 2024-05-08T08:54:54.706 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1489e-05 2024-05-08T08:54:54.706 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.706 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.706 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:54.706 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356986+0000", 2024-05-08T08:54:54.706 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3699999999999997e-06 2024-05-08T08:54:54.706 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.706 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.706 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:54.707 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435433+0000", 2024-05-08T08:54:54.707 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078447520000000007 2024-05-08T08:54:54.707 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.707 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.707 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:54.707 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435484+0000", 2024-05-08T08:54:54.707 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0229000000000003e-05 2024-05-08T08:54:54.707 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.707 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.707 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:54.707 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435575+0000", 2024-05-08T08:54:54.707 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1299999999999997e-05 2024-05-08T08:54:54.707 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.707 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:54.707 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.707 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.707 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.708 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28553 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888713648] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:54.708 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:54:54.708 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 82.312743157, 2024-05-08T08:54:54.708 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078782034000000001, 2024-05-08T08:54:54.708 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:54.708 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:54.708 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:54.708 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:54.708 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:54.708 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:54.708 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28553 2024-05-08T08:54:54.708 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.708 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:54.708 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.708 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:54.708 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:54:54.709 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:54.709 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.709 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.709 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:54.709 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:54:54.709 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:54.709 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.709 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.709 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:54.709 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:54:54.709 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6600000000000002e-07 2024-05-08T08:54:54.709 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.709 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.709 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:54.709 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357006+0000", 2024-05-08T08:54:54.710 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7190000000000001e-06 2024-05-08T08:54:54.710 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.710 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.710 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:54.710 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357013+0000", 2024-05-08T08:54:54.710 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.435e-06 2024-05-08T08:54:54.710 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.710 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.710 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:54.710 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357018+0000", 2024-05-08T08:54:54.710 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7260000000000002e-06 2024-05-08T08:54:54.710 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.710 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.711 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:54.711 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435623+0000", 2024-05-08T08:54:54.711 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078604972999999995 2024-05-08T08:54:54.711 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.711 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.711 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:54.711 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435695+0000", 2024-05-08T08:54:54.711 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1959999999999995e-05 2024-05-08T08:54:54.711 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.711 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.711 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:54.711 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435785+0000", 2024-05-08T08:54:54.711 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.9754999999999997e-05 2024-05-08T08:54:54.711 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.711 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:54.711 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.712 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.712 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.712 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28554 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888980272] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:54.712 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357024+0000", 2024-05-08T08:54:54.712 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 82.312721521, 2024-05-08T08:54:54.712 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078999870999999999, 2024-05-08T08:54:54.712 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:54.712 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:54.712 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:54.712 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:54.712 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:54.712 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:54.712 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28554 2024-05-08T08:54:54.712 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.712 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:54.712 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.712 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:54.713 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357024+0000", 2024-05-08T08:54:54.713 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:54.713 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.713 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.713 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:54.713 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357024+0000", 2024-05-08T08:54:54.713 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:54.713 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.713 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.713 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:54.713 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357025+0000", 2024-05-08T08:54:54.713 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:54:54.713 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.713 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.713 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:54.713 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357027+0000", 2024-05-08T08:54:54.713 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2050000000000001e-06 2024-05-08T08:54:54.714 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.714 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.714 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:54.714 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357033+0000", 2024-05-08T08:54:54.714 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5679999999999996e-06 2024-05-08T08:54:54.714 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.714 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.714 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:54.714 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357038+0000", 2024-05-08T08:54:54.714 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2889999999999999e-06 2024-05-08T08:54:54.714 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.714 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.714 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:54.714 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435823+0000", 2024-05-08T08:54:54.714 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078785677999999998 2024-05-08T08:54:54.715 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.715 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.715 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:54.715 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435892+0000", 2024-05-08T08:54:54.715 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8379000000000004e-05 2024-05-08T08:54:54.715 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.715 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.715 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:54.715 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436024+0000", 2024-05-08T08:54:54.715 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013239799999999999 2024-05-08T08:54:54.715 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.715 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:54.715 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.715 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.715 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.715 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28555 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889001104] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:54.715 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:54:54.716 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 82.312702755999993, 2024-05-08T08:54:54.716 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079115742000000003, 2024-05-08T08:54:54.716 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:54.716 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:54.716 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:54.716 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:54.716 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:54.716 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:54.716 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28555 2024-05-08T08:54:54.716 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.716 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:54.716 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.716 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:54.716 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:54:54.716 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:54.716 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.716 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.717 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:54.717 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:54:54.717 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:54.717 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.717 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.717 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:54.717 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:54:54.717 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9200000000000002e-07 2024-05-08T08:54:54.717 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.717 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.717 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:54.717 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357046+0000", 2024-05-08T08:54:54.717 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1670000000000002e-06 2024-05-08T08:54:54.717 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.717 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.717 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:54.717 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357052+0000", 2024-05-08T08:54:54.718 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.139e-06 2024-05-08T08:54:54.718 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.718 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.718 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:54.718 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357056+0000", 2024-05-08T08:54:54.718 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2440000000000004e-06 2024-05-08T08:54:54.718 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.718 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.718 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:54.718 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436059+0000", 2024-05-08T08:54:54.718 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079003070999999994 2024-05-08T08:54:54.718 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.718 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.718 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:54.718 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436097+0000", 2024-05-08T08:54:54.719 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7784000000000003e-05 2024-05-08T08:54:54.719 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.719 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.719 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:54.719 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436159+0000", 2024-05-08T08:54:54.719 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2045000000000007e-05 2024-05-08T08:54:54.719 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.719 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:54.719 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.719 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.719 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.719 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28556 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889022064] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:54.719 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357065+0000", 2024-05-08T08:54:54.719 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 82.312680723, 2024-05-08T08:54:54.719 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079327976999999994, 2024-05-08T08:54:54.720 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:54.720 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:54.720 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:54.720 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:54.720 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:54.720 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:54.720 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28556 2024-05-08T08:54:54.720 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.720 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:54.720 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.720 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:54.720 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357065+0000", 2024-05-08T08:54:54.720 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:54.720 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.720 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.720 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:54.720 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357065+0000", 2024-05-08T08:54:54.721 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:54.721 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.721 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.721 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:54.721 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358958+0000", 2024-05-08T08:54:54.721 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001892915 2024-05-08T08:54:54.721 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.721 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.721 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:54.721 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358962+0000", 2024-05-08T08:54:54.721 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4599999999999999e-06 2024-05-08T08:54:54.721 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.721 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.721 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:54.721 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358970+0000", 2024-05-08T08:54:54.721 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.6090000000000001e-06 2024-05-08T08:54:54.722 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.722 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.722 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:54.722 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358975+0000", 2024-05-08T08:54:54.722 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7639999999999996e-06 2024-05-08T08:54:54.722 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.722 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.722 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:54.722 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436196+0000", 2024-05-08T08:54:54.722 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.077221564000000006 2024-05-08T08:54:54.722 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.722 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.722 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:54.722 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436269+0000", 2024-05-08T08:54:54.722 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2720999999999997e-05 2024-05-08T08:54:54.722 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.723 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.723 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:54.723 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436393+0000", 2024-05-08T08:54:54.723 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012394399999999999 2024-05-08T08:54:54.723 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.723 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:54.723 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.723 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.723 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.723 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28563 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888855520] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:54.723 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:54:54.723 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 82.310671323999998, 2024-05-08T08:54:54.723 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078795566999999997, 2024-05-08T08:54:54.723 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:54.723 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:54.723 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:54.724 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:54.724 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:54.724 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:54.724 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28563 2024-05-08T08:54:54.724 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.724 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:54.724 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.724 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:54.724 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:54:54.724 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:54.724 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.724 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.724 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:54.724 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:54:54.724 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:54.724 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.724 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.725 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:54.725 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:54:54.725 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1e-07 2024-05-08T08:54:54.725 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.725 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.725 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:54.725 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359080+0000", 2024-05-08T08:54:54.725 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8940000000000001e-06 2024-05-08T08:54:54.725 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.725 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.725 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:54.725 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359083+0000", 2024-05-08T08:54:54.725 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1949999999999999e-06 2024-05-08T08:54:54.725 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.725 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.725 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:54.725 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359085+0000", 2024-05-08T08:54:54.726 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4710000000000001e-06 2024-05-08T08:54:54.726 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.726 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.726 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:54.726 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437604+0000", 2024-05-08T08:54:54.726 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078518278999999996 2024-05-08T08:54:54.726 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.726 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.726 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:54.726 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437704+0000", 2024-05-08T08:54:54.726 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010010599999999999 2024-05-08T08:54:54.726 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.726 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.726 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:54.726 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437870+0000", 2024-05-08T08:54:54.727 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00016641200000000001 2024-05-08T08:54:54.727 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.727 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:54.727 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.727 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.727 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.727 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28564 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888538944] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:54.727 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.359089+0000", 2024-05-08T08:54:54.727 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 82.310657109000005, 2024-05-08T08:54:54.727 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078952675, 2024-05-08T08:54:54.727 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:54.727 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:54.727 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:54.727 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:54.727 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:54.728 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:54.728 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28564 2024-05-08T08:54:54.728 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.728 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:54.728 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.728 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:54.728 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359089+0000", 2024-05-08T08:54:54.728 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:54.728 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.728 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.728 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:54.728 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359089+0000", 2024-05-08T08:54:54.728 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:54.728 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.728 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.728 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:54.728 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360959+0000", 2024-05-08T08:54:54.729 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0018707089999999999 2024-05-08T08:54:54.729 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.729 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.729 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:54.729 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360963+0000", 2024-05-08T08:54:54.729 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0680000000000001e-06 2024-05-08T08:54:54.729 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.729 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.729 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:54.729 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360969+0000", 2024-05-08T08:54:54.729 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1920000000000002e-06 2024-05-08T08:54:54.729 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.729 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.729 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:54.729 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360973+0000", 2024-05-08T08:54:54.729 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.031e-06 2024-05-08T08:54:54.729 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.730 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.730 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:54.730 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437916+0000", 2024-05-08T08:54:54.730 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.076942939000000002 2024-05-08T08:54:54.730 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.730 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.730 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:54.730 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437973+0000", 2024-05-08T08:54:54.730 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7358999999999997e-05 2024-05-08T08:54:54.730 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.730 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.730 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:54.730 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.438041+0000", 2024-05-08T08:54:54.730 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8376999999999999e-05 2024-05-08T08:54:54.730 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.731 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:54.731 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.731 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.731 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.731 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28574 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889427600] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:54.731 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:54:54.731 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 82.308652601999995, 2024-05-08T08:54:54.731 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078782476000000004, 2024-05-08T08:54:54.731 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:54.731 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:54.731 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:54.731 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:54.731 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:54.731 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:54.731 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28574 2024-05-08T08:54:54.731 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.731 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:54.732 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.732 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:54.732 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:54:54.732 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:54.732 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.732 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.732 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:54.732 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:54:54.732 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:54.732 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.732 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.732 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:54.732 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:54:54.732 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1799999999999999e-07 2024-05-08T08:54:54.732 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.732 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.732 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:54.733 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361095+0000", 2024-05-08T08:54:54.733 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3850000000000001e-06 2024-05-08T08:54:54.733 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.733 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.733 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:54.733 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361098+0000", 2024-05-08T08:54:54.733 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7989999999999998e-06 2024-05-08T08:54:54.733 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.733 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.733 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:54.733 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361100+0000", 2024-05-08T08:54:54.733 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4669999999999998e-06 2024-05-08T08:54:54.733 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.733 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.733 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:54.733 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439768+0000", 2024-05-08T08:54:54.734 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078668057999999999 2024-05-08T08:54:54.734 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.734 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.734 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:54.734 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439811+0000", 2024-05-08T08:54:54.734 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2459000000000001e-05 2024-05-08T08:54:54.734 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.734 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.734 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:54.734 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439876+0000", 2024-05-08T08:54:54.734 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5090000000000002e-05 2024-05-08T08:54:54.734 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.734 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:54.734 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.734 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.734 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.735 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28575 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889439216] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:54.735 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361103+0000", 2024-05-08T08:54:54.735 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 82.308642586999994, 2024-05-08T08:54:54.735 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078966828000000003, 2024-05-08T08:54:54.735 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:54.735 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:54.735 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:54.735 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:54.735 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:54.735 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:54.735 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28575 2024-05-08T08:54:54.735 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.735 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:54.735 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.735 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:54.735 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361103+0000", 2024-05-08T08:54:54.736 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:54.736 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.736 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.736 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:54.736 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361103+0000", 2024-05-08T08:54:54.736 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:54.736 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.736 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.736 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:54.736 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361104+0000", 2024-05-08T08:54:54.736 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1400000000000001e-07 2024-05-08T08:54:54.736 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.736 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.736 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:54.736 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361105+0000", 2024-05-08T08:54:54.736 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5540000000000001e-06 2024-05-08T08:54:54.736 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.737 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.737 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:54.737 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361108+0000", 2024-05-08T08:54:54.737 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.841e-06 2024-05-08T08:54:54.737 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.737 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.737 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:54.737 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361110+0000", 2024-05-08T08:54:54.737 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.497e-06 2024-05-08T08:54:54.737 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.737 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.737 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:54.737 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439916+0000", 2024-05-08T08:54:54.737 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078805372999999998 2024-05-08T08:54:54.737 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.737 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.738 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:54.738 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440000+0000", 2024-05-08T08:54:54.738 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4730999999999999e-05 2024-05-08T08:54:54.738 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.738 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.738 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:54.738 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440070+0000", 2024-05-08T08:54:54.738 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9617999999999997e-05 2024-05-08T08:54:54.738 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.738 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:54.738 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.738 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.738 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.738 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28576 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889451328] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:54.738 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361113+0000", 2024-05-08T08:54:54.739 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 82.308632400999997, 2024-05-08T08:54:54.739 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079088862999999995, 2024-05-08T08:54:54.739 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:54.739 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:54.739 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:54.739 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:54.739 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:54.739 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:54.739 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28576 2024-05-08T08:54:54.739 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.739 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:54.739 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.739 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:54.739 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361113+0000", 2024-05-08T08:54:54.739 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:54.739 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.740 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.740 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:54.740 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361113+0000", 2024-05-08T08:54:54.740 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:54.740 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.740 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.740 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:54.740 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361114+0000", 2024-05-08T08:54:54.740 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1299999999999999e-07 2024-05-08T08:54:54.740 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.740 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.740 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:54.740 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361115+0000", 2024-05-08T08:54:54.740 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.4300000000000001e-06 2024-05-08T08:54:54.740 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.741 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.741 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:54.741 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361118+0000", 2024-05-08T08:54:54.741 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9000000000000002e-06 2024-05-08T08:54:54.741 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.741 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.741 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:54.741 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361120+0000", 2024-05-08T08:54:54.741 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4739999999999999e-06 2024-05-08T08:54:54.741 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.741 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.741 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:54.741 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440096+0000", 2024-05-08T08:54:54.741 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078975629000000006 2024-05-08T08:54:54.741 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.741 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.742 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:54.742 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440135+0000", 2024-05-08T08:54:54.742 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9202000000000002e-05 2024-05-08T08:54:54.742 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.742 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.742 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:54.742 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440202+0000", 2024-05-08T08:54:54.742 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7014999999999994e-05 2024-05-08T08:54:54.742 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.742 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:54.742 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.742 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.742 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.742 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28577 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889462640] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:54.742 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:54:54.743 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 82.308622219, 2024-05-08T08:54:54.743 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079194622000000006, 2024-05-08T08:54:54.743 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:54.743 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:54.743 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:54.743 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:54.743 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:54.743 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:54.743 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28577 2024-05-08T08:54:54.743 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.743 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:54.743 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.743 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:54.743 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:54:54.743 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:54.744 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.744 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.744 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:54.744 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:54:54.744 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:54.744 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.744 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.744 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:54.744 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:54:54.744 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1400000000000001e-07 2024-05-08T08:54:54.744 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.744 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.744 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:54.744 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361125+0000", 2024-05-08T08:54:54.744 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.372e-06 2024-05-08T08:54:54.745 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.745 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.745 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:54.745 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361128+0000", 2024-05-08T08:54:54.745 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0309999999999999e-06 2024-05-08T08:54:54.745 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.745 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.745 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:54.745 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361131+0000", 2024-05-08T08:54:54.745 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4040000000000002e-06 2024-05-08T08:54:54.745 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.745 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.745 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:54.745 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440225+0000", 2024-05-08T08:54:54.745 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079094600000000001 2024-05-08T08:54:54.746 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.746 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.746 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:54.746 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440260+0000", 2024-05-08T08:54:54.746 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4703000000000003e-05 2024-05-08T08:54:54.746 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.746 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.746 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:54.746 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440318+0000", 2024-05-08T08:54:54.746 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8298000000000003e-05 2024-05-08T08:54:54.746 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.746 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:54.746 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.746 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.746 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.747 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28578 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889474192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:54.747 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361134+0000", 2024-05-08T08:54:54.747 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 82.308611794000001, 2024-05-08T08:54:54.747 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079310295000000003, 2024-05-08T08:54:54.747 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:54.747 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:54.747 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:54.747 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:54.747 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:54.747 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:54.747 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28578 2024-05-08T08:54:54.747 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.747 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:54.747 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.747 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:54.748 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361134+0000", 2024-05-08T08:54:54.748 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:54.748 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.748 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.748 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:54.748 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361134+0000", 2024-05-08T08:54:54.748 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:54.748 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.748 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.748 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:54.748 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.362958+0000", 2024-05-08T08:54:54.748 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0018237380000000001 2024-05-08T08:54:54.748 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.748 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.748 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:54.749 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.362961+0000", 2024-05-08T08:54:54.749 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.123e-06 2024-05-08T08:54:54.749 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.749 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.749 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:54.749 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.397404+0000", 2024-05-08T08:54:54.749 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.034443226 2024-05-08T08:54:54.749 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.749 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.749 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:54.749 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.397439+0000", 2024-05-08T08:54:54.749 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4690000000000002e-05 2024-05-08T08:54:54.749 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.749 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.749 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:54.750 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440340+0000", 2024-05-08T08:54:54.750 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.042901419000000003 2024-05-08T08:54:54.750 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.750 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.750 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:54.750 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440376+0000", 2024-05-08T08:54:54.750 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5444999999999997e-05 2024-05-08T08:54:54.750 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:54.750 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:54.750 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:54.750 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440444+0000", 2024-05-08T08:54:54.750 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8653999999999997e-05 2024-05-08T08:54:54.750 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.750 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:54.751 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.751 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:54.751 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:54.751 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:54.752 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:54.755+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:54.753 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:54:54.841 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T08:54:54.853 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:54.855+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:54.909 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:54.910 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:54:54.910 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:54:54.910 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:54.919 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:54:54.953 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:54.956+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:54.980 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:54:54.980 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:54:55.004 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:54:55.004 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:54:55.030 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:54:55.031 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:54:55.037 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:55.037 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:55.048 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:54:55.049 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:54:55.049 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:54:55.054 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:55.057+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:55.073 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:54:55.074 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:54:55.093 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:54:55.093 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:54:55.155 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:55.158+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:55.166 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:55.166 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:55.176 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:54:55.256 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:55.259+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:55.284 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:55.285 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:55.294 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:54:55.379 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:55.382+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:55.458 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:55.460+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:55.462 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:55.462 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:55.472 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:54:55.559 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:55.561+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:55.618 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:55.618 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:55.627 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:54:55.660 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:55.663+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:55.732 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:55.732 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:55.741 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:54:55.784 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:55.787+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:55.846 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:55.847 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:55.855 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:54:55.861 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:55.863+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:55.962 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:55.965+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:56.001 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:56.001 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:56.010 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:54:56.095 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:56.098+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:56.157 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:56.157 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:56.164 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:56.167+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:56.166 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:54:56.265 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:56.268+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:56.310 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:56.311 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:56.319 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:54:56.403 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:56.406+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:56.418 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4] out_osds: [5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-05-08T08:54:56.418 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T08:54:56.418 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-05-08T08:54:56.418 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-05-08T08:54:56.448 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-05-08T08:54:56.472 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:56.472 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:56.481 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:54:56.506 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:56.509+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:56.568 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:56.571+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:56.636 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:56.636 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:56.645 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:54:56.670 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:56.672+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:56.753 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:54:56.754 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:56.754 INFO:teuthology.orchestra.run.smithi059.stdout: "epoch": 145, 2024-05-08T08:54:56.754 INFO:teuthology.orchestra.run.smithi059.stdout: "fsid": "38dfae8c-5c5e-4b0f-a264-91f22c6c5554", 2024-05-08T08:54:56.754 INFO:teuthology.orchestra.run.smithi059.stdout: "created": "2024-05-08T08:51:45.571369+0000", 2024-05-08T08:54:56.754 INFO:teuthology.orchestra.run.smithi059.stdout: "modified": "2024-05-08T08:54:56.362070+0000", 2024-05-08T08:54:56.754 INFO:teuthology.orchestra.run.smithi059.stdout: "last_up_change": "2024-05-08T08:51:56.847706+0000", 2024-05-08T08:54:56.754 INFO:teuthology.orchestra.run.smithi059.stdout: "last_in_change": "2024-05-08T08:54:51.397440+0000", 2024-05-08T08:54:56.754 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-05-08T08:54:56.754 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_num": 5804032, 2024-05-08T08:54:56.754 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_set": [ 2024-05-08T08:54:56.754 INFO:teuthology.orchestra.run.smithi059.stdout: "nodeep-scrub", 2024-05-08T08:54:56.754 INFO:teuthology.orchestra.run.smithi059.stdout: "pglog_hardlimit", 2024-05-08T08:54:56.755 INFO:teuthology.orchestra.run.smithi059.stdout: "purged_snapdirs", 2024-05-08T08:54:56.755 INFO:teuthology.orchestra.run.smithi059.stdout: "recovery_deletes", 2024-05-08T08:54:56.755 INFO:teuthology.orchestra.run.smithi059.stdout: "sortbitwise" 2024-05-08T08:54:56.755 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T08:54:56.755 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_version": 4, 2024-05-08T08:54:56.755 INFO:teuthology.orchestra.run.smithi059.stdout: "full_ratio": 0.94999998807907104, 2024-05-08T08:54:56.755 INFO:teuthology.orchestra.run.smithi059.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-05-08T08:54:56.755 INFO:teuthology.orchestra.run.smithi059.stdout: "nearfull_ratio": 0.85000002384185791, 2024-05-08T08:54:56.755 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_snapshot": "", 2024-05-08T08:54:56.755 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_max": 3, 2024-05-08T08:54:56.755 INFO:teuthology.orchestra.run.smithi059.stdout: "max_osd": 6, 2024-05-08T08:54:56.755 INFO:teuthology.orchestra.run.smithi059.stdout: "require_min_compat_client": "luminous", 2024-05-08T08:54:56.755 INFO:teuthology.orchestra.run.smithi059.stdout: "min_compat_client": "luminous", 2024-05-08T08:54:56.755 INFO:teuthology.orchestra.run.smithi059.stdout: "require_osd_release": "squid", 2024-05-08T08:54:56.755 INFO:teuthology.orchestra.run.smithi059.stdout: "allow_crimson": false, 2024-05-08T08:54:56.756 INFO:teuthology.orchestra.run.smithi059.stdout: "pools": [ 2024-05-08T08:54:56.756 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:56.756 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 1, 2024-05-08T08:54:56.756 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_name": ".mgr", 2024-05-08T08:54:56.756 INFO:teuthology.orchestra.run.smithi059.stdout: "create_time": "2024-05-08T08:51:57.787790+0000", 2024-05-08T08:54:56.756 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T08:54:56.756 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_names": "hashpspool", 2024-05-08T08:54:56.756 INFO:teuthology.orchestra.run.smithi059.stdout: "type": 1, 2024-05-08T08:54:56.756 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 2, 2024-05-08T08:54:56.756 INFO:teuthology.orchestra.run.smithi059.stdout: "min_size": 1, 2024-05-08T08:54:56.756 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_rule": 0, 2024-05-08T08:54:56.756 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_count": 0, 2024-05-08T08:54:56.756 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_target": 0, 2024-05-08T08:54:56.756 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_barrier": 0, 2024-05-08T08:54:56.757 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-08T08:54:56.757 INFO:teuthology.orchestra.run.smithi059.stdout: "object_hash": 2, 2024-05-08T08:54:56.757 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscale_mode": "off", 2024-05-08T08:54:56.757 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num": 1, 2024-05-08T08:54:56.757 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num": 1, 2024-05-08T08:54:56.757 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num_target": 1, 2024-05-08T08:54:56.757 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_target": 1, 2024-05-08T08:54:56.757 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_pending": 1, 2024-05-08T08:54:56.757 INFO:teuthology.orchestra.run.smithi059.stdout: "last_pg_merge_meta": { 2024-05-08T08:54:56.757 INFO:teuthology.orchestra.run.smithi059.stdout: "source_pgid": "0.0", 2024-05-08T08:54:56.757 INFO:teuthology.orchestra.run.smithi059.stdout: "ready_epoch": 0, 2024-05-08T08:54:56.757 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_started": 0, 2024-05-08T08:54:56.757 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_clean": 0, 2024-05-08T08:54:56.757 INFO:teuthology.orchestra.run.smithi059.stdout: "source_version": "0'0", 2024-05-08T08:54:56.757 INFO:teuthology.orchestra.run.smithi059.stdout: "target_version": "0'0" 2024-05-08T08:54:56.758 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:56.758 INFO:teuthology.orchestra.run.smithi059.stdout: "last_change": "14", 2024-05-08T08:54:56.758 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend": "0", 2024-05-08T08:54:56.758 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-08T08:54:56.758 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_preluminous": "0", 2024-05-08T08:54:56.758 INFO:teuthology.orchestra.run.smithi059.stdout: "auid": 0, 2024-05-08T08:54:56.758 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_mode": "selfmanaged", 2024-05-08T08:54:56.758 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_seq": 0, 2024-05-08T08:54:56.758 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_epoch": 0, 2024-05-08T08:54:56.758 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_snaps": [], 2024-05-08T08:54:56.758 INFO:teuthology.orchestra.run.smithi059.stdout: "removed_snaps": "[]", 2024-05-08T08:54:56.758 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_bytes": 0, 2024-05-08T08:54:56.758 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_objects": 0, 2024-05-08T08:54:56.758 INFO:teuthology.orchestra.run.smithi059.stdout: "tiers": [], 2024-05-08T08:54:56.759 INFO:teuthology.orchestra.run.smithi059.stdout: "tier_of": -1, 2024-05-08T08:54:56.759 INFO:teuthology.orchestra.run.smithi059.stdout: "read_tier": -1, 2024-05-08T08:54:56.759 INFO:teuthology.orchestra.run.smithi059.stdout: "write_tier": -1, 2024-05-08T08:54:56.759 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_mode": "none", 2024-05-08T08:54:56.759 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_bytes": 0, 2024-05-08T08:54:56.759 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_objects": 0, 2024-05-08T08:54:56.759 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-08T08:54:56.759 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-08T08:54:56.759 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-08T08:54:56.759 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_flush_age": 0, 2024-05-08T08:54:56.759 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_evict_age": 0, 2024-05-08T08:54:56.759 INFO:teuthology.orchestra.run.smithi059.stdout: "erasure_code_profile": "", 2024-05-08T08:54:56.759 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_params": { 2024-05-08T08:54:56.759 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "none" 2024-05-08T08:54:56.759 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:56.759 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_period": 0, 2024-05-08T08:54:56.760 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_count": 0, 2024-05-08T08:54:56.760 INFO:teuthology.orchestra.run.smithi059.stdout: "use_gmt_hitset": true, 2024-05-08T08:54:56.760 INFO:teuthology.orchestra.run.smithi059.stdout: "min_read_recency_for_promote": 0, 2024-05-08T08:54:56.760 INFO:teuthology.orchestra.run.smithi059.stdout: "min_write_recency_for_promote": 0, 2024-05-08T08:54:56.760 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_grade_decay_rate": 0, 2024-05-08T08:54:56.760 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_search_last_n": 0, 2024-05-08T08:54:56.760 INFO:teuthology.orchestra.run.smithi059.stdout: "grade_table": [], 2024-05-08T08:54:56.760 INFO:teuthology.orchestra.run.smithi059.stdout: "stripe_width": 0, 2024-05-08T08:54:56.760 INFO:teuthology.orchestra.run.smithi059.stdout: "expected_num_objects": 0, 2024-05-08T08:54:56.760 INFO:teuthology.orchestra.run.smithi059.stdout: "fast_read": false, 2024-05-08T08:54:56.760 INFO:teuthology.orchestra.run.smithi059.stdout: "options": { 2024-05-08T08:54:56.760 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_max": 32, 2024-05-08T08:54:56.760 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_min": 1 2024-05-08T08:54:56.760 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:56.761 INFO:teuthology.orchestra.run.smithi059.stdout: "application_metadata": { 2024-05-08T08:54:56.761 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": {} 2024-05-08T08:54:56.761 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:56.761 INFO:teuthology.orchestra.run.smithi059.stdout: "read_balance": { 2024-05-08T08:54:56.761 INFO:teuthology.orchestra.run.smithi059.stdout: "score_type": "Fair distribution", 2024-05-08T08:54:56.761 INFO:teuthology.orchestra.run.smithi059.stdout: "score_acting": 5, 2024-05-08T08:54:56.761 INFO:teuthology.orchestra.run.smithi059.stdout: "score_stable": 5, 2024-05-08T08:54:56.761 INFO:teuthology.orchestra.run.smithi059.stdout: "optimal_score": 0.40000000596046448, 2024-05-08T08:54:56.761 INFO:teuthology.orchestra.run.smithi059.stdout: "raw_score_acting": 2, 2024-05-08T08:54:56.761 INFO:teuthology.orchestra.run.smithi059.stdout: "raw_score_stable": 2, 2024-05-08T08:54:56.761 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity_weighted": 0.71152502298355103, 2024-05-08T08:54:56.761 INFO:teuthology.orchestra.run.smithi059.stdout: "average_primary_affinity": 0.85000002384185791, 2024-05-08T08:54:56.761 INFO:teuthology.orchestra.run.smithi059.stdout: "average_primary_affinity_weighted": 0.70999997854232788 2024-05-08T08:54:56.761 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:56.761 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:56.761 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:56.762 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 2, 2024-05-08T08:54:56.762 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_name": "rbd", 2024-05-08T08:54:56.762 INFO:teuthology.orchestra.run.smithi059.stdout: "create_time": "2024-05-08T08:52:02.563677+0000", 2024-05-08T08:54:56.762 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T08:54:56.762 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_names": "hashpspool", 2024-05-08T08:54:56.762 INFO:teuthology.orchestra.run.smithi059.stdout: "type": 1, 2024-05-08T08:54:56.762 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 2, 2024-05-08T08:54:56.762 INFO:teuthology.orchestra.run.smithi059.stdout: "min_size": 1, 2024-05-08T08:54:56.762 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_rule": 0, 2024-05-08T08:54:56.762 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_count": 0, 2024-05-08T08:54:56.762 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_target": 0, 2024-05-08T08:54:56.762 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_barrier": 0, 2024-05-08T08:54:56.762 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-08T08:54:56.763 INFO:teuthology.orchestra.run.smithi059.stdout: "object_hash": 2, 2024-05-08T08:54:56.763 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscale_mode": "off", 2024-05-08T08:54:56.763 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num": 8, 2024-05-08T08:54:56.763 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num": 8, 2024-05-08T08:54:56.763 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num_target": 8, 2024-05-08T08:54:56.763 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_target": 8, 2024-05-08T08:54:56.763 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_pending": 8, 2024-05-08T08:54:56.763 INFO:teuthology.orchestra.run.smithi059.stdout: "last_pg_merge_meta": { 2024-05-08T08:54:56.763 INFO:teuthology.orchestra.run.smithi059.stdout: "source_pgid": "0.0", 2024-05-08T08:54:56.763 INFO:teuthology.orchestra.run.smithi059.stdout: "ready_epoch": 0, 2024-05-08T08:54:56.763 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_started": 0, 2024-05-08T08:54:56.763 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_clean": 0, 2024-05-08T08:54:56.763 INFO:teuthology.orchestra.run.smithi059.stdout: "source_version": "0'0", 2024-05-08T08:54:56.763 INFO:teuthology.orchestra.run.smithi059.stdout: "target_version": "0'0" 2024-05-08T08:54:56.763 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:56.763 INFO:teuthology.orchestra.run.smithi059.stdout: "last_change": "17", 2024-05-08T08:54:56.764 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend": "0", 2024-05-08T08:54:56.764 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-08T08:54:56.764 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_preluminous": "0", 2024-05-08T08:54:56.764 INFO:teuthology.orchestra.run.smithi059.stdout: "auid": 0, 2024-05-08T08:54:56.764 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_mode": "selfmanaged", 2024-05-08T08:54:56.764 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_seq": 0, 2024-05-08T08:54:56.764 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_epoch": 0, 2024-05-08T08:54:56.764 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_snaps": [], 2024-05-08T08:54:56.764 INFO:teuthology.orchestra.run.smithi059.stdout: "removed_snaps": "[]", 2024-05-08T08:54:56.764 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_bytes": 0, 2024-05-08T08:54:56.764 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_objects": 0, 2024-05-08T08:54:56.764 INFO:teuthology.orchestra.run.smithi059.stdout: "tiers": [], 2024-05-08T08:54:56.764 INFO:teuthology.orchestra.run.smithi059.stdout: "tier_of": -1, 2024-05-08T08:54:56.764 INFO:teuthology.orchestra.run.smithi059.stdout: "read_tier": -1, 2024-05-08T08:54:56.764 INFO:teuthology.orchestra.run.smithi059.stdout: "write_tier": -1, 2024-05-08T08:54:56.765 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_mode": "none", 2024-05-08T08:54:56.765 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_bytes": 0, 2024-05-08T08:54:56.765 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_objects": 0, 2024-05-08T08:54:56.765 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-08T08:54:56.765 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-08T08:54:56.765 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-08T08:54:56.765 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_flush_age": 0, 2024-05-08T08:54:56.765 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_evict_age": 0, 2024-05-08T08:54:56.765 INFO:teuthology.orchestra.run.smithi059.stdout: "erasure_code_profile": "", 2024-05-08T08:54:56.765 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_params": { 2024-05-08T08:54:56.765 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "none" 2024-05-08T08:54:56.765 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:56.765 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_period": 0, 2024-05-08T08:54:56.765 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_count": 0, 2024-05-08T08:54:56.765 INFO:teuthology.orchestra.run.smithi059.stdout: "use_gmt_hitset": true, 2024-05-08T08:54:56.766 INFO:teuthology.orchestra.run.smithi059.stdout: "min_read_recency_for_promote": 0, 2024-05-08T08:54:56.766 INFO:teuthology.orchestra.run.smithi059.stdout: "min_write_recency_for_promote": 0, 2024-05-08T08:54:56.766 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_grade_decay_rate": 0, 2024-05-08T08:54:56.766 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_search_last_n": 0, 2024-05-08T08:54:56.766 INFO:teuthology.orchestra.run.smithi059.stdout: "grade_table": [], 2024-05-08T08:54:56.766 INFO:teuthology.orchestra.run.smithi059.stdout: "stripe_width": 0, 2024-05-08T08:54:56.766 INFO:teuthology.orchestra.run.smithi059.stdout: "expected_num_objects": 0, 2024-05-08T08:54:56.766 INFO:teuthology.orchestra.run.smithi059.stdout: "fast_read": false, 2024-05-08T08:54:56.766 INFO:teuthology.orchestra.run.smithi059.stdout: "options": {}, 2024-05-08T08:54:56.766 INFO:teuthology.orchestra.run.smithi059.stdout: "application_metadata": { 2024-05-08T08:54:56.766 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd": {} 2024-05-08T08:54:56.766 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:56.766 INFO:teuthology.orchestra.run.smithi059.stdout: "read_balance": { 2024-05-08T08:54:56.766 INFO:teuthology.orchestra.run.smithi059.stdout: "score_type": "Fair distribution", 2024-05-08T08:54:56.767 INFO:teuthology.orchestra.run.smithi059.stdout: "score_acting": 1.8799999952316284, 2024-05-08T08:54:56.767 INFO:teuthology.orchestra.run.smithi059.stdout: "score_stable": 1.8799999952316284, 2024-05-08T08:54:56.767 INFO:teuthology.orchestra.run.smithi059.stdout: "optimal_score": 1, 2024-05-08T08:54:56.767 INFO:teuthology.orchestra.run.smithi059.stdout: "raw_score_acting": 1.8799999952316284, 2024-05-08T08:54:56.767 INFO:teuthology.orchestra.run.smithi059.stdout: "raw_score_stable": 1.8799999952316284, 2024-05-08T08:54:56.767 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity_weighted": 0.71152502298355103, 2024-05-08T08:54:56.767 INFO:teuthology.orchestra.run.smithi059.stdout: "average_primary_affinity": 0.85000002384185791, 2024-05-08T08:54:56.767 INFO:teuthology.orchestra.run.smithi059.stdout: "average_primary_affinity_weighted": 0.70999997854232788 2024-05-08T08:54:56.767 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:56.767 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:56.767 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:56.767 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 3, 2024-05-08T08:54:56.767 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_name": "pool_client.0", 2024-05-08T08:54:56.767 INFO:teuthology.orchestra.run.smithi059.stdout: "create_time": "2024-05-08T08:52:09.955467+0000", 2024-05-08T08:54:56.767 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 8193, 2024-05-08T08:54:56.768 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-05-08T08:54:56.768 INFO:teuthology.orchestra.run.smithi059.stdout: "type": 1, 2024-05-08T08:54:56.768 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 2, 2024-05-08T08:54:56.768 INFO:teuthology.orchestra.run.smithi059.stdout: "min_size": 1, 2024-05-08T08:54:56.768 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_rule": 0, 2024-05-08T08:54:56.768 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_count": 0, 2024-05-08T08:54:56.768 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_target": 0, 2024-05-08T08:54:56.768 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_barrier": 0, 2024-05-08T08:54:56.768 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-08T08:54:56.768 INFO:teuthology.orchestra.run.smithi059.stdout: "object_hash": 2, 2024-05-08T08:54:56.768 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscale_mode": "off", 2024-05-08T08:54:56.768 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num": 32, 2024-05-08T08:54:56.768 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num": 32, 2024-05-08T08:54:56.768 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num_target": 32, 2024-05-08T08:54:56.768 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_target": 32, 2024-05-08T08:54:56.769 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_pending": 32, 2024-05-08T08:54:56.769 INFO:teuthology.orchestra.run.smithi059.stdout: "last_pg_merge_meta": { 2024-05-08T08:54:56.769 INFO:teuthology.orchestra.run.smithi059.stdout: "source_pgid": "0.0", 2024-05-08T08:54:56.769 INFO:teuthology.orchestra.run.smithi059.stdout: "ready_epoch": 0, 2024-05-08T08:54:56.769 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_started": 0, 2024-05-08T08:54:56.769 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_clean": 0, 2024-05-08T08:54:56.769 INFO:teuthology.orchestra.run.smithi059.stdout: "source_version": "0'0", 2024-05-08T08:54:56.769 INFO:teuthology.orchestra.run.smithi059.stdout: "target_version": "0'0" 2024-05-08T08:54:56.769 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:56.769 INFO:teuthology.orchestra.run.smithi059.stdout: "last_change": "143", 2024-05-08T08:54:56.769 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend": "0", 2024-05-08T08:54:56.769 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-08T08:54:56.769 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_preluminous": "0", 2024-05-08T08:54:56.769 INFO:teuthology.orchestra.run.smithi059.stdout: "auid": 0, 2024-05-08T08:54:56.770 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_mode": "selfmanaged", 2024-05-08T08:54:56.770 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_seq": 51, 2024-05-08T08:54:56.770 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_epoch": 143, 2024-05-08T08:54:56.770 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_snaps": [], 2024-05-08T08:54:56.770 INFO:teuthology.orchestra.run.smithi059.stdout: "removed_snaps": "[]", 2024-05-08T08:54:56.770 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_bytes": 0, 2024-05-08T08:54:56.770 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_objects": 0, 2024-05-08T08:54:56.770 INFO:teuthology.orchestra.run.smithi059.stdout: "tiers": [], 2024-05-08T08:54:56.770 INFO:teuthology.orchestra.run.smithi059.stdout: "tier_of": -1, 2024-05-08T08:54:56.770 INFO:teuthology.orchestra.run.smithi059.stdout: "read_tier": -1, 2024-05-08T08:54:56.770 INFO:teuthology.orchestra.run.smithi059.stdout: "write_tier": -1, 2024-05-08T08:54:56.770 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_mode": "none", 2024-05-08T08:54:56.770 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_bytes": 0, 2024-05-08T08:54:56.770 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_objects": 0, 2024-05-08T08:54:56.770 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-08T08:54:56.771 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-08T08:54:56.771 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-08T08:54:56.771 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_flush_age": 0, 2024-05-08T08:54:56.771 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_evict_age": 0, 2024-05-08T08:54:56.771 INFO:teuthology.orchestra.run.smithi059.stdout: "erasure_code_profile": "", 2024-05-08T08:54:56.771 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_params": { 2024-05-08T08:54:56.771 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "none" 2024-05-08T08:54:56.771 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:56.771 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_period": 0, 2024-05-08T08:54:56.771 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_count": 0, 2024-05-08T08:54:56.771 INFO:teuthology.orchestra.run.smithi059.stdout: "use_gmt_hitset": true, 2024-05-08T08:54:56.771 INFO:teuthology.orchestra.run.smithi059.stdout: "min_read_recency_for_promote": 0, 2024-05-08T08:54:56.771 INFO:teuthology.orchestra.run.smithi059.stdout: "min_write_recency_for_promote": 0, 2024-05-08T08:54:56.771 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_grade_decay_rate": 0, 2024-05-08T08:54:56.771 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_search_last_n": 0, 2024-05-08T08:54:56.772 INFO:teuthology.orchestra.run.smithi059.stdout: "grade_table": [], 2024-05-08T08:54:56.772 INFO:teuthology.orchestra.run.smithi059.stdout: "stripe_width": 0, 2024-05-08T08:54:56.772 INFO:teuthology.orchestra.run.smithi059.stdout: "expected_num_objects": 0, 2024-05-08T08:54:56.772 INFO:teuthology.orchestra.run.smithi059.stdout: "fast_read": false, 2024-05-08T08:54:56.772 INFO:teuthology.orchestra.run.smithi059.stdout: "options": {}, 2024-05-08T08:54:56.772 INFO:teuthology.orchestra.run.smithi059.stdout: "application_metadata": { 2024-05-08T08:54:56.772 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd": {} 2024-05-08T08:54:56.772 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:56.772 INFO:teuthology.orchestra.run.smithi059.stdout: "read_balance": { 2024-05-08T08:54:56.772 INFO:teuthology.orchestra.run.smithi059.stdout: "score_type": "Fair distribution", 2024-05-08T08:54:56.772 INFO:teuthology.orchestra.run.smithi059.stdout: "score_acting": 1.7200000286102295, 2024-05-08T08:54:56.772 INFO:teuthology.orchestra.run.smithi059.stdout: "score_stable": 1.7200000286102295, 2024-05-08T08:54:56.772 INFO:teuthology.orchestra.run.smithi059.stdout: "optimal_score": 1, 2024-05-08T08:54:56.772 INFO:teuthology.orchestra.run.smithi059.stdout: "raw_score_acting": 1.7200000286102295, 2024-05-08T08:54:56.773 INFO:teuthology.orchestra.run.smithi059.stdout: "raw_score_stable": 1.7200000286102295, 2024-05-08T08:54:56.773 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity_weighted": 0.71152502298355103, 2024-05-08T08:54:56.773 INFO:teuthology.orchestra.run.smithi059.stdout: "average_primary_affinity": 0.85000002384185791, 2024-05-08T08:54:56.773 INFO:teuthology.orchestra.run.smithi059.stdout: "average_primary_affinity_weighted": 0.70999997854232788 2024-05-08T08:54:56.773 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:56.773 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:56.773 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T08:54:56.773 INFO:teuthology.orchestra.run.smithi059.stdout: "osds": [ 2024-05-08T08:54:56.773 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:56.773 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 0, 2024-05-08T08:54:56.773 INFO:teuthology.orchestra.run.smithi059.stdout: "uuid": "300801a5-1919-4b23-8c6d-bc7dad61cc4d", 2024-05-08T08:54:56.773 INFO:teuthology.orchestra.run.smithi059.stdout: "up": 1, 2024-05-08T08:54:56.773 INFO:teuthology.orchestra.run.smithi059.stdout: "in": 1, 2024-05-08T08:54:56.773 INFO:teuthology.orchestra.run.smithi059.stdout: "weight": 1, 2024-05-08T08:54:56.773 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity": 0, 2024-05-08T08:54:56.774 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_begin": 0, 2024-05-08T08:54:56.774 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_end": 0, 2024-05-08T08:54:56.774 INFO:teuthology.orchestra.run.smithi059.stdout: "up_from": 11, 2024-05-08T08:54:56.774 INFO:teuthology.orchestra.run.smithi059.stdout: "up_thru": 144, 2024-05-08T08:54:56.774 INFO:teuthology.orchestra.run.smithi059.stdout: "down_at": 0, 2024-05-08T08:54:56.774 INFO:teuthology.orchestra.run.smithi059.stdout: "lost_at": 0, 2024-05-08T08:54:56.774 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrs": { 2024-05-08T08:54:56.774 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:54:56.774 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:56.774 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:54:56.774 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6800", 2024-05-08T08:54:56.774 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 802160151 2024-05-08T08:54:56.774 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:56.774 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:56.774 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:54:56.775 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6801", 2024-05-08T08:54:56.775 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 802160151 2024-05-08T08:54:56.775 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:56.775 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:56.775 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:56.775 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addrs": { 2024-05-08T08:54:56.775 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:54:56.775 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:56.775 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:54:56.775 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6802", 2024-05-08T08:54:56.775 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 802160151 2024-05-08T08:54:56.775 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:56.775 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:56.775 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:54:56.775 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6803", 2024-05-08T08:54:56.775 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 802160151 2024-05-08T08:54:56.776 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:56.776 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:56.776 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:56.776 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addrs": { 2024-05-08T08:54:56.776 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:54:56.776 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:56.776 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:54:56.776 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6806", 2024-05-08T08:54:56.776 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 802160151 2024-05-08T08:54:56.776 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:56.776 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:56.776 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:54:56.776 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6807", 2024-05-08T08:54:56.776 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 802160151 2024-05-08T08:54:56.776 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:56.777 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:56.777 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:56.777 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addrs": { 2024-05-08T08:54:56.777 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:54:56.777 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:56.777 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:54:56.777 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6804", 2024-05-08T08:54:56.777 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 802160151 2024-05-08T08:54:56.777 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:56.777 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:56.777 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:54:56.777 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6805", 2024-05-08T08:54:56.777 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 802160151 2024-05-08T08:54:56.777 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:56.777 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:56.777 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:56.778 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "172.21.15.59:6801/802160151", 2024-05-08T08:54:56.778 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "172.21.15.59:6803/802160151", 2024-05-08T08:54:56.778 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addr": "172.21.15.59:6807/802160151", 2024-05-08T08:54:56.778 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addr": "172.21.15.59:6805/802160151", 2024-05-08T08:54:56.778 INFO:teuthology.orchestra.run.smithi059.stdout: "state": [ 2024-05-08T08:54:56.778 INFO:teuthology.orchestra.run.smithi059.stdout: "exists", 2024-05-08T08:54:56.778 INFO:teuthology.orchestra.run.smithi059.stdout: "up" 2024-05-08T08:54:56.778 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:56.778 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:56.778 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:56.778 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 1, 2024-05-08T08:54:56.778 INFO:teuthology.orchestra.run.smithi059.stdout: "uuid": "0fead0de-6a65-4eba-b400-eea5692702e0", 2024-05-08T08:54:56.778 INFO:teuthology.orchestra.run.smithi059.stdout: "up": 1, 2024-05-08T08:54:56.778 INFO:teuthology.orchestra.run.smithi059.stdout: "in": 1, 2024-05-08T08:54:56.778 INFO:teuthology.orchestra.run.smithi059.stdout: "weight": 1, 2024-05-08T08:54:56.779 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity": 1, 2024-05-08T08:54:56.779 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_begin": 0, 2024-05-08T08:54:56.779 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_end": 0, 2024-05-08T08:54:56.779 INFO:teuthology.orchestra.run.smithi059.stdout: "up_from": 11, 2024-05-08T08:54:56.779 INFO:teuthology.orchestra.run.smithi059.stdout: "up_thru": 144, 2024-05-08T08:54:56.779 INFO:teuthology.orchestra.run.smithi059.stdout: "down_at": 0, 2024-05-08T08:54:56.779 INFO:teuthology.orchestra.run.smithi059.stdout: "lost_at": 0, 2024-05-08T08:54:56.779 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrs": { 2024-05-08T08:54:56.779 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:54:56.779 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:56.779 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:54:56.779 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6816", 2024-05-08T08:54:56.779 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4116801011 2024-05-08T08:54:56.779 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:56.779 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:56.780 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:54:56.780 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6817", 2024-05-08T08:54:56.780 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4116801011 2024-05-08T08:54:56.780 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:56.780 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:56.780 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:56.780 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addrs": { 2024-05-08T08:54:56.780 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:54:56.780 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:56.780 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:54:56.780 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6818", 2024-05-08T08:54:56.780 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4116801011 2024-05-08T08:54:56.780 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:56.780 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:56.780 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:54:56.780 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6819", 2024-05-08T08:54:56.781 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4116801011 2024-05-08T08:54:56.781 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:56.781 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:56.781 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:56.781 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addrs": { 2024-05-08T08:54:56.781 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:54:56.781 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:56.781 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:54:56.781 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6822", 2024-05-08T08:54:56.781 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4116801011 2024-05-08T08:54:56.781 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:56.781 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:56.781 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:54:56.781 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6823", 2024-05-08T08:54:56.781 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4116801011 2024-05-08T08:54:56.782 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:56.782 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:56.782 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:56.782 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addrs": { 2024-05-08T08:54:56.782 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:54:56.782 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:56.782 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:54:56.782 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6820", 2024-05-08T08:54:56.782 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4116801011 2024-05-08T08:54:56.782 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:56.782 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:56.782 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:54:56.782 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6821", 2024-05-08T08:54:56.782 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4116801011 2024-05-08T08:54:56.782 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:56.783 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:56.783 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:56.783 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "172.21.15.59:6817/4116801011", 2024-05-08T08:54:56.783 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "172.21.15.59:6819/4116801011", 2024-05-08T08:54:56.783 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addr": "172.21.15.59:6823/4116801011", 2024-05-08T08:54:56.783 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addr": "172.21.15.59:6821/4116801011", 2024-05-08T08:54:56.783 INFO:teuthology.orchestra.run.smithi059.stdout: "state": [ 2024-05-08T08:54:56.783 INFO:teuthology.orchestra.run.smithi059.stdout: "exists", 2024-05-08T08:54:56.783 INFO:teuthology.orchestra.run.smithi059.stdout: "up" 2024-05-08T08:54:56.783 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:56.783 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:56.783 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:56.783 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 2, 2024-05-08T08:54:56.783 INFO:teuthology.orchestra.run.smithi059.stdout: "uuid": "1a233de6-1b82-4a51-8e96-4cd4abdf16c8", 2024-05-08T08:54:56.783 INFO:teuthology.orchestra.run.smithi059.stdout: "up": 1, 2024-05-08T08:54:56.784 INFO:teuthology.orchestra.run.smithi059.stdout: "in": 1, 2024-05-08T08:54:56.784 INFO:teuthology.orchestra.run.smithi059.stdout: "weight": 1, 2024-05-08T08:54:56.784 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity": 1, 2024-05-08T08:54:56.784 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_begin": 0, 2024-05-08T08:54:56.784 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_end": 0, 2024-05-08T08:54:56.784 INFO:teuthology.orchestra.run.smithi059.stdout: "up_from": 11, 2024-05-08T08:54:56.784 INFO:teuthology.orchestra.run.smithi059.stdout: "up_thru": 144, 2024-05-08T08:54:56.784 INFO:teuthology.orchestra.run.smithi059.stdout: "down_at": 0, 2024-05-08T08:54:56.784 INFO:teuthology.orchestra.run.smithi059.stdout: "lost_at": 0, 2024-05-08T08:54:56.784 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrs": { 2024-05-08T08:54:56.784 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:54:56.784 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:56.784 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:54:56.784 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6808", 2024-05-08T08:54:56.784 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1547661179 2024-05-08T08:54:56.784 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:56.785 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:56.785 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:54:56.785 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6809", 2024-05-08T08:54:56.785 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1547661179 2024-05-08T08:54:56.785 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:56.785 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:56.785 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:56.785 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addrs": { 2024-05-08T08:54:56.785 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:54:56.785 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:56.785 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:54:56.785 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6810", 2024-05-08T08:54:56.785 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1547661179 2024-05-08T08:54:56.785 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:56.785 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:56.786 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:54:56.786 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6811", 2024-05-08T08:54:56.786 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1547661179 2024-05-08T08:54:56.786 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:56.786 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:56.786 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:56.786 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addrs": { 2024-05-08T08:54:56.786 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:54:56.786 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:56.786 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:54:56.786 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6814", 2024-05-08T08:54:56.786 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1547661179 2024-05-08T08:54:56.786 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:56.786 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:56.787 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:54:56.787 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6815", 2024-05-08T08:54:56.787 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1547661179 2024-05-08T08:54:56.787 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:56.787 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:56.787 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:56.787 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addrs": { 2024-05-08T08:54:56.787 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:54:56.787 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:56.787 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:54:56.787 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6812", 2024-05-08T08:54:56.787 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1547661179 2024-05-08T08:54:56.787 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:56.787 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:56.787 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:54:56.787 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6813", 2024-05-08T08:54:56.788 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1547661179 2024-05-08T08:54:56.788 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:56.788 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:56.788 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:56.788 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "172.21.15.59:6809/1547661179", 2024-05-08T08:54:56.788 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "172.21.15.59:6811/1547661179", 2024-05-08T08:54:56.788 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addr": "172.21.15.59:6815/1547661179", 2024-05-08T08:54:56.788 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addr": "172.21.15.59:6813/1547661179", 2024-05-08T08:54:56.788 INFO:teuthology.orchestra.run.smithi059.stdout: "state": [ 2024-05-08T08:54:56.788 INFO:teuthology.orchestra.run.smithi059.stdout: "exists", 2024-05-08T08:54:56.788 INFO:teuthology.orchestra.run.smithi059.stdout: "up" 2024-05-08T08:54:56.788 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:56.788 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:56.788 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:56.788 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 3, 2024-05-08T08:54:56.789 INFO:teuthology.orchestra.run.smithi059.stdout: "uuid": "cc4870b8-dcd9-4e43-978d-8de6c6396abf", 2024-05-08T08:54:56.789 INFO:teuthology.orchestra.run.smithi059.stdout: "up": 1, 2024-05-08T08:54:56.789 INFO:teuthology.orchestra.run.smithi059.stdout: "in": 1, 2024-05-08T08:54:56.789 INFO:teuthology.orchestra.run.smithi059.stdout: "weight": 1, 2024-05-08T08:54:56.789 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity": 1, 2024-05-08T08:54:56.789 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_begin": 0, 2024-05-08T08:54:56.789 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_end": 0, 2024-05-08T08:54:56.789 INFO:teuthology.orchestra.run.smithi059.stdout: "up_from": 11, 2024-05-08T08:54:56.789 INFO:teuthology.orchestra.run.smithi059.stdout: "up_thru": 144, 2024-05-08T08:54:56.789 INFO:teuthology.orchestra.run.smithi059.stdout: "down_at": 0, 2024-05-08T08:54:56.789 INFO:teuthology.orchestra.run.smithi059.stdout: "lost_at": 0, 2024-05-08T08:54:56.789 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrs": { 2024-05-08T08:54:56.789 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:54:56.789 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:56.789 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:54:56.789 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6800", 2024-05-08T08:54:56.790 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4054400810 2024-05-08T08:54:56.790 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:56.790 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:56.790 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:54:56.790 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6801", 2024-05-08T08:54:56.790 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4054400810 2024-05-08T08:54:56.790 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:56.790 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:56.790 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:56.790 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addrs": { 2024-05-08T08:54:56.790 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:54:56.790 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:56.790 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:54:56.790 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6802", 2024-05-08T08:54:56.790 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4054400810 2024-05-08T08:54:56.791 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:56.791 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:56.791 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:54:56.791 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6803", 2024-05-08T08:54:56.791 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4054400810 2024-05-08T08:54:56.791 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:56.791 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:56.791 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:56.791 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addrs": { 2024-05-08T08:54:56.791 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:54:56.791 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:56.791 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:54:56.791 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6806", 2024-05-08T08:54:56.791 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4054400810 2024-05-08T08:54:56.791 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:56.791 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:56.792 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:54:56.792 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6807", 2024-05-08T08:54:56.792 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4054400810 2024-05-08T08:54:56.792 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:56.792 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:56.792 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:56.792 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addrs": { 2024-05-08T08:54:56.792 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:54:56.792 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:56.792 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:54:56.792 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6804", 2024-05-08T08:54:56.792 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4054400810 2024-05-08T08:54:56.792 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:56.792 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:56.792 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:54:56.792 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6805", 2024-05-08T08:54:56.793 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4054400810 2024-05-08T08:54:56.793 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:56.793 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:56.793 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:56.793 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "172.21.15.192:6801/4054400810", 2024-05-08T08:54:56.793 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "172.21.15.192:6803/4054400810", 2024-05-08T08:54:56.793 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addr": "172.21.15.192:6807/4054400810", 2024-05-08T08:54:56.793 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addr": "172.21.15.192:6805/4054400810", 2024-05-08T08:54:56.793 INFO:teuthology.orchestra.run.smithi059.stdout: "state": [ 2024-05-08T08:54:56.793 INFO:teuthology.orchestra.run.smithi059.stdout: "exists", 2024-05-08T08:54:56.793 INFO:teuthology.orchestra.run.smithi059.stdout: "up" 2024-05-08T08:54:56.793 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:56.793 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:56.793 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:56.794 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 4, 2024-05-08T08:54:56.794 INFO:teuthology.orchestra.run.smithi059.stdout: "uuid": "d1d05620-42fb-4107-a2f9-e83542d076ee", 2024-05-08T08:54:56.794 INFO:teuthology.orchestra.run.smithi059.stdout: "up": 1, 2024-05-08T08:54:56.794 INFO:teuthology.orchestra.run.smithi059.stdout: "in": 1, 2024-05-08T08:54:56.794 INFO:teuthology.orchestra.run.smithi059.stdout: "weight": 1, 2024-05-08T08:54:56.794 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity": 0.9892578125, 2024-05-08T08:54:56.794 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_begin": 0, 2024-05-08T08:54:56.794 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_end": 0, 2024-05-08T08:54:56.794 INFO:teuthology.orchestra.run.smithi059.stdout: "up_from": 11, 2024-05-08T08:54:56.794 INFO:teuthology.orchestra.run.smithi059.stdout: "up_thru": 144, 2024-05-08T08:54:56.794 INFO:teuthology.orchestra.run.smithi059.stdout: "down_at": 0, 2024-05-08T08:54:56.794 INFO:teuthology.orchestra.run.smithi059.stdout: "lost_at": 0, 2024-05-08T08:54:56.794 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrs": { 2024-05-08T08:54:56.794 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:54:56.795 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:56.795 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:54:56.795 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6808", 2024-05-08T08:54:56.795 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 58985565 2024-05-08T08:54:56.795 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:56.795 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:56.795 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:54:56.795 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6809", 2024-05-08T08:54:56.795 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 58985565 2024-05-08T08:54:56.795 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:56.795 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:56.795 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:56.795 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addrs": { 2024-05-08T08:54:56.795 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:54:56.795 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:56.796 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:54:56.796 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6810", 2024-05-08T08:54:56.796 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 58985565 2024-05-08T08:54:56.796 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:56.796 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:56.796 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:54:56.796 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6811", 2024-05-08T08:54:56.796 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 58985565 2024-05-08T08:54:56.796 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:56.796 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:56.796 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:56.796 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addrs": { 2024-05-08T08:54:56.796 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:54:56.796 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:56.796 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:54:56.796 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6814", 2024-05-08T08:54:56.797 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 58985565 2024-05-08T08:54:56.797 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:56.797 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:56.797 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:54:56.797 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6815", 2024-05-08T08:54:56.797 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 58985565 2024-05-08T08:54:56.797 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:56.797 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:56.797 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:56.797 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addrs": { 2024-05-08T08:54:56.797 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:54:56.797 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:56.797 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:54:56.797 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6812", 2024-05-08T08:54:56.797 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 58985565 2024-05-08T08:54:56.798 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:56.798 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:56.798 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:54:56.798 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6813", 2024-05-08T08:54:56.798 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 58985565 2024-05-08T08:54:56.798 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:56.798 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:56.798 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:56.798 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "172.21.15.192:6809/58985565", 2024-05-08T08:54:56.798 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "172.21.15.192:6811/58985565", 2024-05-08T08:54:56.798 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addr": "172.21.15.192:6815/58985565", 2024-05-08T08:54:56.798 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addr": "172.21.15.192:6813/58985565", 2024-05-08T08:54:56.798 INFO:teuthology.orchestra.run.smithi059.stdout: "state": [ 2024-05-08T08:54:56.798 INFO:teuthology.orchestra.run.smithi059.stdout: "exists", 2024-05-08T08:54:56.799 INFO:teuthology.orchestra.run.smithi059.stdout: "up" 2024-05-08T08:54:56.799 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:56.799 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:56.799 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:56.799 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 5, 2024-05-08T08:54:56.799 INFO:teuthology.orchestra.run.smithi059.stdout: "uuid": "ab26136c-31c3-4090-9fa2-988dba4f731f", 2024-05-08T08:54:56.799 INFO:teuthology.orchestra.run.smithi059.stdout: "up": 1, 2024-05-08T08:54:56.799 INFO:teuthology.orchestra.run.smithi059.stdout: "in": 0, 2024-05-08T08:54:56.799 INFO:teuthology.orchestra.run.smithi059.stdout: "weight": 0, 2024-05-08T08:54:56.799 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity": 0.2798919677734375, 2024-05-08T08:54:56.799 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_begin": 0, 2024-05-08T08:54:56.799 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_end": 0, 2024-05-08T08:54:56.799 INFO:teuthology.orchestra.run.smithi059.stdout: "up_from": 11, 2024-05-08T08:54:56.799 INFO:teuthology.orchestra.run.smithi059.stdout: "up_thru": 136, 2024-05-08T08:54:56.799 INFO:teuthology.orchestra.run.smithi059.stdout: "down_at": 0, 2024-05-08T08:54:56.800 INFO:teuthology.orchestra.run.smithi059.stdout: "lost_at": 0, 2024-05-08T08:54:56.800 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrs": { 2024-05-08T08:54:56.800 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:54:56.800 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:56.800 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:54:56.800 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6816", 2024-05-08T08:54:56.800 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1283571573 2024-05-08T08:54:56.800 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:56.800 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:56.800 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:54:56.800 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6817", 2024-05-08T08:54:56.800 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1283571573 2024-05-08T08:54:56.800 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:56.800 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:56.800 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:56.800 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addrs": { 2024-05-08T08:54:56.800 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:54:56.801 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:56.801 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:54:56.801 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6818", 2024-05-08T08:54:56.801 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1283571573 2024-05-08T08:54:56.801 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:56.801 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:56.801 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:54:56.801 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6819", 2024-05-08T08:54:56.801 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1283571573 2024-05-08T08:54:56.801 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:56.801 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:56.801 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:56.801 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addrs": { 2024-05-08T08:54:56.801 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:54:56.801 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:56.801 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:54:56.801 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6822", 2024-05-08T08:54:56.802 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1283571573 2024-05-08T08:54:56.802 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:56.802 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:56.802 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:54:56.802 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6823", 2024-05-08T08:54:56.802 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1283571573 2024-05-08T08:54:56.802 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:56.802 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:56.802 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:56.802 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addrs": { 2024-05-08T08:54:56.802 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:54:56.802 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:56.802 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:54:56.802 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6820", 2024-05-08T08:54:56.803 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1283571573 2024-05-08T08:54:56.803 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:56.803 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:56.803 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:54:56.803 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6821", 2024-05-08T08:54:56.803 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1283571573 2024-05-08T08:54:56.803 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:56.803 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:56.803 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:56.803 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "172.21.15.192:6817/1283571573", 2024-05-08T08:54:56.803 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "172.21.15.192:6819/1283571573", 2024-05-08T08:54:56.803 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addr": "172.21.15.192:6823/1283571573", 2024-05-08T08:54:56.803 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addr": "172.21.15.192:6821/1283571573", 2024-05-08T08:54:56.803 INFO:teuthology.orchestra.run.smithi059.stdout: "state": [ 2024-05-08T08:54:56.803 INFO:teuthology.orchestra.run.smithi059.stdout: "exists", 2024-05-08T08:54:56.804 INFO:teuthology.orchestra.run.smithi059.stdout: "up" 2024-05-08T08:54:56.804 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:56.804 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:56.804 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T08:54:56.804 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_xinfo": [ 2024-05-08T08:54:56.804 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:56.804 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 0, 2024-05-08T08:54:56.804 INFO:teuthology.orchestra.run.smithi059.stdout: "down_stamp": "0.000000", 2024-05-08T08:54:56.804 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_probability": 0, 2024-05-08T08:54:56.804 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_interval": 0, 2024-05-08T08:54:56.804 INFO:teuthology.orchestra.run.smithi059.stdout: "features": 4540701547738038271, 2024-05-08T08:54:56.804 INFO:teuthology.orchestra.run.smithi059.stdout: "old_weight": 0, 2024-05-08T08:54:56.804 INFO:teuthology.orchestra.run.smithi059.stdout: "last_purged_snaps_scrub": "2024-05-08T08:51:52.396946+0000", 2024-05-08T08:54:56.804 INFO:teuthology.orchestra.run.smithi059.stdout: "dead_epoch": 0 2024-05-08T08:54:56.804 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:56.805 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:56.805 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 1, 2024-05-08T08:54:56.805 INFO:teuthology.orchestra.run.smithi059.stdout: "down_stamp": "0.000000", 2024-05-08T08:54:56.805 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_probability": 0, 2024-05-08T08:54:56.805 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_interval": 0, 2024-05-08T08:54:56.805 INFO:teuthology.orchestra.run.smithi059.stdout: "features": 4540701547738038271, 2024-05-08T08:54:56.805 INFO:teuthology.orchestra.run.smithi059.stdout: "old_weight": 0, 2024-05-08T08:54:56.805 INFO:teuthology.orchestra.run.smithi059.stdout: "last_purged_snaps_scrub": "2024-05-08T08:51:52.453160+0000", 2024-05-08T08:54:56.805 INFO:teuthology.orchestra.run.smithi059.stdout: "dead_epoch": 0 2024-05-08T08:54:56.805 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:56.805 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:56.805 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 2, 2024-05-08T08:54:56.805 INFO:teuthology.orchestra.run.smithi059.stdout: "down_stamp": "0.000000", 2024-05-08T08:54:56.805 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_probability": 0, 2024-05-08T08:54:56.805 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_interval": 0, 2024-05-08T08:54:56.805 INFO:teuthology.orchestra.run.smithi059.stdout: "features": 4540701547738038271, 2024-05-08T08:54:56.805 INFO:teuthology.orchestra.run.smithi059.stdout: "old_weight": 0, 2024-05-08T08:54:56.806 INFO:teuthology.orchestra.run.smithi059.stdout: "last_purged_snaps_scrub": "2024-05-08T08:51:52.387606+0000", 2024-05-08T08:54:56.806 INFO:teuthology.orchestra.run.smithi059.stdout: "dead_epoch": 0 2024-05-08T08:54:56.806 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:56.806 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:56.806 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 3, 2024-05-08T08:54:56.806 INFO:teuthology.orchestra.run.smithi059.stdout: "down_stamp": "0.000000", 2024-05-08T08:54:56.806 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_probability": 0, 2024-05-08T08:54:56.806 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_interval": 0, 2024-05-08T08:54:56.806 INFO:teuthology.orchestra.run.smithi059.stdout: "features": 4540701547738038271, 2024-05-08T08:54:56.806 INFO:teuthology.orchestra.run.smithi059.stdout: "old_weight": 0, 2024-05-08T08:54:56.806 INFO:teuthology.orchestra.run.smithi059.stdout: "last_purged_snaps_scrub": "2024-05-08T08:51:52.360952+0000", 2024-05-08T08:54:56.806 INFO:teuthology.orchestra.run.smithi059.stdout: "dead_epoch": 0 2024-05-08T08:54:56.806 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:56.806 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:56.806 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 4, 2024-05-08T08:54:56.807 INFO:teuthology.orchestra.run.smithi059.stdout: "down_stamp": "0.000000", 2024-05-08T08:54:56.807 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_probability": 0, 2024-05-08T08:54:56.807 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_interval": 0, 2024-05-08T08:54:56.807 INFO:teuthology.orchestra.run.smithi059.stdout: "features": 4540701547738038271, 2024-05-08T08:54:56.807 INFO:teuthology.orchestra.run.smithi059.stdout: "old_weight": 0, 2024-05-08T08:54:56.807 INFO:teuthology.orchestra.run.smithi059.stdout: "last_purged_snaps_scrub": "2024-05-08T08:51:52.468681+0000", 2024-05-08T08:54:56.807 INFO:teuthology.orchestra.run.smithi059.stdout: "dead_epoch": 0 2024-05-08T08:54:56.807 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:56.807 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:56.807 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 5, 2024-05-08T08:54:56.807 INFO:teuthology.orchestra.run.smithi059.stdout: "down_stamp": "0.000000", 2024-05-08T08:54:56.807 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_probability": 0, 2024-05-08T08:54:56.807 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_interval": 0, 2024-05-08T08:54:56.807 INFO:teuthology.orchestra.run.smithi059.stdout: "features": 4540701547738038271, 2024-05-08T08:54:56.807 INFO:teuthology.orchestra.run.smithi059.stdout: "old_weight": 65536, 2024-05-08T08:54:56.807 INFO:teuthology.orchestra.run.smithi059.stdout: "last_purged_snaps_scrub": "2024-05-08T08:51:52.401080+0000", 2024-05-08T08:54:56.807 INFO:teuthology.orchestra.run.smithi059.stdout: "dead_epoch": 0 2024-05-08T08:54:56.808 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:56.808 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T08:54:56.808 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_upmap": [ 2024-05-08T08:54:56.808 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:56.808 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "3.4", 2024-05-08T08:54:56.808 INFO:teuthology.orchestra.run.smithi059.stdout: "osds": [ 2024-05-08T08:54:56.808 INFO:teuthology.orchestra.run.smithi059.stdout: 1, 2024-05-08T08:54:56.808 INFO:teuthology.orchestra.run.smithi059.stdout: 5 2024-05-08T08:54:56.808 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:56.808 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:56.808 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:56.808 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "3.7", 2024-05-08T08:54:56.808 INFO:teuthology.orchestra.run.smithi059.stdout: "osds": [ 2024-05-08T08:54:56.808 INFO:teuthology.orchestra.run.smithi059.stdout: 4, 2024-05-08T08:54:56.808 INFO:teuthology.orchestra.run.smithi059.stdout: 3 2024-05-08T08:54:56.808 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:56.809 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:56.809 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:56.809 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "3.17", 2024-05-08T08:54:56.809 INFO:teuthology.orchestra.run.smithi059.stdout: "osds": [ 2024-05-08T08:54:56.809 INFO:teuthology.orchestra.run.smithi059.stdout: 1, 2024-05-08T08:54:56.809 INFO:teuthology.orchestra.run.smithi059.stdout: 0 2024-05-08T08:54:56.809 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:56.809 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:56.809 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:56.809 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "3.1d", 2024-05-08T08:54:56.809 INFO:teuthology.orchestra.run.smithi059.stdout: "osds": [ 2024-05-08T08:54:56.809 INFO:teuthology.orchestra.run.smithi059.stdout: 0, 2024-05-08T08:54:56.809 INFO:teuthology.orchestra.run.smithi059.stdout: 5 2024-05-08T08:54:56.809 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:56.809 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:56.809 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T08:54:56.809 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_upmap_items": [], 2024-05-08T08:54:56.810 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_upmap_primaries": [], 2024-05-08T08:54:56.810 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_temp": [], 2024-05-08T08:54:56.810 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_temp": [], 2024-05-08T08:54:56.810 INFO:teuthology.orchestra.run.smithi059.stdout: "blocklist": {}, 2024-05-08T08:54:56.810 INFO:teuthology.orchestra.run.smithi059.stdout: "range_blocklist": {}, 2024-05-08T08:54:56.810 INFO:teuthology.orchestra.run.smithi059.stdout: "erasure_code_profiles": { 2024-05-08T08:54:56.810 INFO:teuthology.orchestra.run.smithi059.stdout: "default": { 2024-05-08T08:54:56.810 INFO:teuthology.orchestra.run.smithi059.stdout: "crush-failure-domain": "osd", 2024-05-08T08:54:56.810 INFO:teuthology.orchestra.run.smithi059.stdout: "k": "2", 2024-05-08T08:54:56.810 INFO:teuthology.orchestra.run.smithi059.stdout: "m": "1", 2024-05-08T08:54:56.810 INFO:teuthology.orchestra.run.smithi059.stdout: "plugin": "jerasure", 2024-05-08T08:54:56.810 INFO:teuthology.orchestra.run.smithi059.stdout: "technique": "reed_sol_van" 2024-05-08T08:54:56.810 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:56.810 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:56.810 INFO:teuthology.orchestra.run.smithi059.stdout: "removed_snaps_queue": [ 2024-05-08T08:54:56.810 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:56.811 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 3, 2024-05-08T08:54:56.811 INFO:teuthology.orchestra.run.smithi059.stdout: "snaps": [ 2024-05-08T08:54:56.811 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:56.811 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 31, 2024-05-08T08:54:56.811 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 2 2024-05-08T08:54:56.811 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:56.811 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:56.811 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:56.811 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T08:54:56.811 INFO:teuthology.orchestra.run.smithi059.stdout: "new_removed_snaps": [], 2024-05-08T08:54:56.811 INFO:teuthology.orchestra.run.smithi059.stdout: "new_purged_snaps": [], 2024-05-08T08:54:56.811 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_node_flags": {}, 2024-05-08T08:54:56.811 INFO:teuthology.orchestra.run.smithi059.stdout: "device_class_flags": {}, 2024-05-08T08:54:56.811 INFO:teuthology.orchestra.run.smithi059.stdout: "stretch_mode": { 2024-05-08T08:54:56.811 INFO:teuthology.orchestra.run.smithi059.stdout: "stretch_mode_enabled": false, 2024-05-08T08:54:56.812 INFO:teuthology.orchestra.run.smithi059.stdout: "stretch_bucket_count": 0, 2024-05-08T08:54:56.812 INFO:teuthology.orchestra.run.smithi059.stdout: "degraded_stretch_mode": 0, 2024-05-08T08:54:56.812 INFO:teuthology.orchestra.run.smithi059.stdout: "recovering_stretch_mode": 0, 2024-05-08T08:54:56.812 INFO:teuthology.orchestra.run.smithi059.stdout: "stretch_mode_bucket": 0 2024-05-08T08:54:56.812 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:56.812 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:56.814 INFO:tasks.thrashosds.thrasher:j is {'epoch': 145, 'fsid': '38dfae8c-5c5e-4b0f-a264-91f22c6c5554', 'created': '2024-05-08T08:51:45.571369+0000', 'modified': '2024-05-08T08:54:56.362070+0000', 'last_up_change': '2024-05-08T08:51:56.847706+0000', 'last_in_change': '2024-05-08T08:54:51.397440+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T08:51:57.787790+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '14', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 5, 'score_stable': 5, 'optimal_score': 0.4000000059604645, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.711525022983551, 'average_primary_affinity': 0.8500000238418579, 'average_primary_affinity_weighted': 0.7099999785423279}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-05-08T08:52:02.563677+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.711525022983551, 'average_primary_affinity': 0.8500000238418579, 'average_primary_affinity_weighted': 0.7099999785423279}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-05-08T08:52:09.955467+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '143', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 51, 'snap_epoch': 143, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.7200000286102295, 'score_stable': 1.7200000286102295, 'optimal_score': 1, 'raw_score_acting': 1.7200000286102295, 'raw_score_stable': 1.7200000286102295, 'primary_affinity_weighted': 0.711525022983551, 'average_primary_affinity': 0.8500000238418579, 'average_primary_affinity_weighted': 0.7099999785423279}}], 'osds': [{'osd': 0, 'uuid': '300801a5-1919-4b23-8c6d-bc7dad61cc4d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 144, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6800', 'nonce': 802160151}, {'type': 'v1', 'addr': '172.21.15.59:6801', 'nonce': 802160151}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6802', 'nonce': 802160151}, {'type': 'v1', 'addr': '172.21.15.59:6803', 'nonce': 802160151}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6806', 'nonce': 802160151}, {'type': 'v1', 'addr': '172.21.15.59:6807', 'nonce': 802160151}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6804', 'nonce': 802160151}, {'type': 'v1', 'addr': '172.21.15.59:6805', 'nonce': 802160151}]}, 'public_addr': '172.21.15.59:6801/802160151', 'cluster_addr': '172.21.15.59:6803/802160151', 'heartbeat_back_addr': '172.21.15.59:6807/802160151', 'heartbeat_front_addr': '172.21.15.59:6805/802160151', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '0fead0de-6a65-4eba-b400-eea5692702e0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 144, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6816', 'nonce': 4116801011}, {'type': 'v1', 'addr': '172.21.15.59:6817', 'nonce': 4116801011}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6818', 'nonce': 4116801011}, {'type': 'v1', 'addr': '172.21.15.59:6819', 'nonce': 4116801011}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6822', 'nonce': 4116801011}, {'type': 'v1', 'addr': '172.21.15.59:6823', 'nonce': 4116801011}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6820', 'nonce': 4116801011}, {'type': 'v1', 'addr': '172.21.15.59:6821', 'nonce': 4116801011}]}, 'public_addr': '172.21.15.59:6817/4116801011', 'cluster_addr': '172.21.15.59:6819/4116801011', 'heartbeat_back_addr': '172.21.15.59:6823/4116801011', 'heartbeat_front_addr': '172.21.15.59:6821/4116801011', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '1a233de6-1b82-4a51-8e96-4cd4abdf16c8', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 144, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6808', 'nonce': 1547661179}, {'type': 'v1', 'addr': '172.21.15.59:6809', 'nonce': 1547661179}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6810', 'nonce': 1547661179}, {'type': 'v1', 'addr': '172.21.15.59:6811', 'nonce': 1547661179}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6814', 'nonce': 1547661179}, {'type': 'v1', 'addr': '172.21.15.59:6815', 'nonce': 1547661179}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6812', 'nonce': 1547661179}, {'type': 'v1', 'addr': '172.21.15.59:6813', 'nonce': 1547661179}]}, 'public_addr': '172.21.15.59:6809/1547661179', 'cluster_addr': '172.21.15.59:6811/1547661179', 'heartbeat_back_addr': '172.21.15.59:6815/1547661179', 'heartbeat_front_addr': '172.21.15.59:6813/1547661179', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'cc4870b8-dcd9-4e43-978d-8de6c6396abf', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 144, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6800', 'nonce': 4054400810}, {'type': 'v1', 'addr': '172.21.15.192:6801', 'nonce': 4054400810}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6802', 'nonce': 4054400810}, {'type': 'v1', 'addr': '172.21.15.192:6803', 'nonce': 4054400810}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6806', 'nonce': 4054400810}, {'type': 'v1', 'addr': '172.21.15.192:6807', 'nonce': 4054400810}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6804', 'nonce': 4054400810}, {'type': 'v1', 'addr': '172.21.15.192:6805', 'nonce': 4054400810}]}, 'public_addr': '172.21.15.192:6801/4054400810', 'cluster_addr': '172.21.15.192:6803/4054400810', 'heartbeat_back_addr': '172.21.15.192:6807/4054400810', 'heartbeat_front_addr': '172.21.15.192:6805/4054400810', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'd1d05620-42fb-4107-a2f9-e83542d076ee', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.9892578125, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 144, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6808', 'nonce': 58985565}, {'type': 'v1', 'addr': '172.21.15.192:6809', 'nonce': 58985565}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6810', 'nonce': 58985565}, {'type': 'v1', 'addr': '172.21.15.192:6811', 'nonce': 58985565}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6814', 'nonce': 58985565}, {'type': 'v1', 'addr': '172.21.15.192:6815', 'nonce': 58985565}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6812', 'nonce': 58985565}, {'type': 'v1', 'addr': '172.21.15.192:6813', 'nonce': 58985565}]}, 'public_addr': '172.21.15.192:6809/58985565', 'cluster_addr': '172.21.15.192:6811/58985565', 'heartbeat_back_addr': '172.21.15.192:6815/58985565', 'heartbeat_front_addr': '172.21.15.192:6813/58985565', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'ab26136c-31c3-4090-9fa2-988dba4f731f', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 0.2798919677734375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 136, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6816', 'nonce': 1283571573}, {'type': 'v1', 'addr': '172.21.15.192:6817', 'nonce': 1283571573}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6818', 'nonce': 1283571573}, {'type': 'v1', 'addr': '172.21.15.192:6819', 'nonce': 1283571573}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6822', 'nonce': 1283571573}, {'type': 'v1', 'addr': '172.21.15.192:6823', 'nonce': 1283571573}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6820', 'nonce': 1283571573}, {'type': 'v1', 'addr': '172.21.15.192:6821', 'nonce': 1283571573}]}, 'public_addr': '172.21.15.192:6817/1283571573', 'cluster_addr': '172.21.15.192:6819/1283571573', 'heartbeat_back_addr': '172.21.15.192:6823/1283571573', 'heartbeat_front_addr': '172.21.15.192:6821/1283571573', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:51:52.396946+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:51:52.453160+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:51:52.387606+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:51:52.360952+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:51:52.468681+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-05-08T08:51:52.401080+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.4', 'osds': [1, 5]}, {'pgid': '3.7', 'osds': [4, 3]}, {'pgid': '3.17', 'osds': [1, 0]}, {'pgid': '3.1d', 'osds': [0, 5]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 31, 'length': 2}]}], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-05-08T08:54:56.814 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T08:54:56.815 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:56.815 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:56.815 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:54:56.816 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:56.816+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:56.914 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:56.916+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:56.966 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:56.966 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:56.976 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:54:57.017 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:57.020+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:57.106 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T08:54:57.128 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:57.128 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:57.128 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:57.131+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:57.137 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:54:57.161 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:54:57.162 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-05-08T08:54:57.174 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":229,"stamp":"2024-05-08T08:54:56.366644+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":970358030,"num_objects":2672,"num_object_clones":393,"num_object_copies":5344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2672,"num_whiteouts":1616,"num_read":505081,"num_read_kb":1621647,"num_write":24576,"num_write_kb":453874,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4647,"num_bytes_recovered":275353135,"num_keys_recovered":639,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22348,"ondisk_log_size":22348,"up":82,"acting":71,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":71,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":471859200,"kb_used":359532,"kb_used_data":60644,"kb_used_omap":7,"kb_used_meta":298808,"kb_avail":471499668,"statfs":{"total":483183820800,"available":482815660032,"internally_reserved":0,"allocated":62099456,"data_stored":441488900,"data_compressed":23190042,"data_compressed_allocated":43204608,"data_compressed_original":431415296,"omap_allocated":7973,"internal_metadata":305979611},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-47381236,"num_objects":-31,"num_object_clones":-6,"num_object_copies":-62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-31,"num_whiteouts":-6,"num_read":492,"num_read_kb":421,"num_write":93,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-151,"num_bytes_recovered":-21349055,"num_keys_recovered":-11,"num_objects_omap":-2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.963031"},"pg_stats":[{"pgid":"3.1f","version":"139'670","reported_seq":20061,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-05-08T08:54:52.409979+0000","last_change":"2024-05-08T08:54:51.410694+0000","last_active":"2024-05-08T08:54:52.409979+0000","last_peered":"2024-05-08T08:54:52.409979+0000","last_clean":"2024-05-08T08:54:52.409979+0000","last_became_active":"2024-05-08T08:53:49.580563+0000","last_became_peered":"2024-05-08T08:53:49.580563+0000","last_unstale":"2024-05-08T08:54:52.409979+0000","last_undegraded":"2024-05-08T08:54:52.409979+0000","last_fullsized":"2024-05-08T08:54:52.409979+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:55:59.780022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016004789999999999,"stat_sum":{"num_bytes":61477180,"num_objects":87,"num_object_clones":17,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":45,"num_read":19090,"num_read_kb":85578,"num_write":921,"num_write_kb":33089,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"20","length":"8"}]},{"pgid":"3.1e","version":"137'826","reported_seq":11441,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-05-08T08:54:52.403188+0000","last_change":"2024-05-08T08:54:51.403380+0000","last_active":"2024-05-08T08:54:52.403188+0000","last_peered":"2024-05-08T08:54:52.403188+0000","last_clean":"2024-05-08T08:54:52.403188+0000","last_became_active":"2024-05-08T08:52:11.930844+0000","last_became_peered":"2024-05-08T08:52:11.930844+0000","last_unstale":"2024-05-08T08:54:52.403188+0000","last_undegraded":"2024-05-08T08:54:52.403188+0000","last_fullsized":"2024-05-08T08:54:52.403188+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":826,"log_dups_size":0,"ondisk_log_size":826,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:41:53.924886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001027479,"stat_sum":{"num_bytes":19188815,"num_objects":89,"num_object_clones":12,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89,"num_whiteouts":60,"num_read":13302,"num_read_kb":36769,"num_write":632,"num_write_kb":12703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"20","length":"8"}]},{"pgid":"3.1d","version":"136'683","reported_seq":4535,"reported_epoch":139,"state":"remapped+peering","last_fresh":"2024-05-08T08:54:50.404178+0000","last_change":"2024-05-08T08:54:52.406106+0000","last_active":"2024-05-08T08:54:52.406106+0000","last_peered":"2024-05-08T08:54:50.404178+0000","last_clean":"2024-05-08T08:54:50.404178+0000","last_became_active":"2024-05-08T08:54:47.374724+0000","last_became_peered":"2024-05-08T08:54:47.374724+0000","last_unstale":"2024-05-08T08:54:50.404178+0000","last_undegraded":"2024-05-08T08:54:50.404178+0000","last_fullsized":"2024-05-08T08:54:50.404178+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":683,"log_dups_size":0,"ondisk_log_size":683,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049796400000000005,"stat_sum":{"num_bytes":12607676,"num_objects":77,"num_object_clones":10,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":46,"num_read":8232,"num_read_kb":30479,"num_write":852,"num_write_kb":7278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":284,"num_bytes_recovered":11016723,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,4,5],"up_primary":4,"acting_primary":0,"purged_snaps":[{"start":"21","length":"7"}]},{"pgid":"3.1c","version":"139'613","reported_seq":18629,"reported_epoch":140,"state":"remapped+peering","last_fresh":"2024-05-08T08:54:51.404752+0000","last_change":"2024-05-08T08:54:52.411380+0000","last_active":"2024-05-08T08:54:51.404752+0000","last_peered":"2024-05-08T08:54:50.535325+0000","last_clean":"2024-05-08T08:54:50.535325+0000","last_became_active":"2024-05-08T08:54:47.376532+0000","last_became_peered":"2024-05-08T08:54:47.376532+0000","last_unstale":"2024-05-08T08:54:51.404752+0000","last_undegraded":"2024-05-08T08:54:51.404752+0000","last_fullsized":"2024-05-08T08:54:51.404752+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":613,"log_dups_size":0,"ondisk_log_size":613,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064771700000000002,"stat_sum":{"num_bytes":53221941,"num_objects":89,"num_object_clones":10,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89,"num_whiteouts":54,"num_read":18049,"num_read_kb":101014,"num_write":682,"num_write_kb":27844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":448,"num_bytes_recovered":68382916,"num_keys_recovered":256,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,5],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"21","length":"7"}]},{"pgid":"3.1b","version":"139'580","reported_seq":25412,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-05-08T08:54:52.410871+0000","last_change":"2024-05-08T08:54:51.404487+0000","last_active":"2024-05-08T08:54:52.410871+0000","last_peered":"2024-05-08T08:54:52.410871+0000","last_clean":"2024-05-08T08:54:52.410871+0000","last_became_active":"2024-05-08T08:53:49.583097+0000","last_became_peered":"2024-05-08T08:53:49.583097+0000","last_unstale":"2024-05-08T08:54:52.410871+0000","last_undegraded":"2024-05-08T08:54:52.410871+0000","last_fullsized":"2024-05-08T08:54:52.410871+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":580,"log_dups_size":0,"ondisk_log_size":580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:33:07.859048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076240200000000002,"stat_sum":{"num_bytes":39856369,"num_objects":95,"num_object_clones":13,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":62,"num_read":25077,"num_read_kb":45800,"num_write":655,"num_write_kb":23307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":129,"num_bytes_recovered":3090520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"20","length":"8"}]},{"pgid":"3.1a","version":"133'770","reported_seq":21614,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-05-08T08:54:52.409697+0000","last_change":"2024-05-08T08:54:51.404222+0000","last_active":"2024-05-08T08:54:52.409697+0000","last_peered":"2024-05-08T08:54:52.409697+0000","last_clean":"2024-05-08T08:54:52.409697+0000","last_became_active":"2024-05-08T08:53:24.484124+0000","last_became_peered":"2024-05-08T08:53:24.484124+0000","last_unstale":"2024-05-08T08:54:52.409697+0000","last_undegraded":"2024-05-08T08:54:52.409697+0000","last_fullsized":"2024-05-08T08:54:52.409697+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":770,"log_dups_size":0,"ondisk_log_size":770,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:21:59.734037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049346199999999996,"stat_sum":{"num_bytes":22214012,"num_objects":87,"num_object_clones":13,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":46,"num_read":24052,"num_read_kb":59424,"num_write":844,"num_write_kb":19082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":130,"num_bytes_recovered":8549042,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"20","length":"8"}]},{"pgid":"3.19","version":"135'387","reported_seq":10451,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-05-08T08:54:52.402758+0000","last_change":"2024-05-08T08:54:51.402997+0000","last_active":"2024-05-08T08:54:52.402758+0000","last_peered":"2024-05-08T08:54:52.402758+0000","last_clean":"2024-05-08T08:54:52.402758+0000","last_became_active":"2024-05-08T08:53:24.483403+0000","last_became_peered":"2024-05-08T08:53:24.483403+0000","last_unstale":"2024-05-08T08:54:52.402758+0000","last_undegraded":"2024-05-08T08:54:52.402758+0000","last_fullsized":"2024-05-08T08:54:52.402758+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":387,"log_dups_size":0,"ondisk_log_size":387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:40:18.203426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001361309,"stat_sum":{"num_bytes":18166832,"num_objects":83,"num_object_clones":11,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":55,"num_read":9410,"num_read_kb":48045,"num_write":351,"num_write_kb":16514,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":8814624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"20","length":"8"}]},{"pgid":"3.18","version":"139'646","reported_seq":19327,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-05-08T08:54:52.402606+0000","last_change":"2024-05-08T08:54:51.403414+0000","last_active":"2024-05-08T08:54:52.402606+0000","last_peered":"2024-05-08T08:54:52.402606+0000","last_clean":"2024-05-08T08:54:52.402606+0000","last_became_active":"2024-05-08T08:52:11.927366+0000","last_became_peered":"2024-05-08T08:52:11.927366+0000","last_unstale":"2024-05-08T08:54:52.402606+0000","last_undegraded":"2024-05-08T08:54:52.402606+0000","last_fullsized":"2024-05-08T08:54:52.402606+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:39:34.592587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001364014,"stat_sum":{"num_bytes":76302507,"num_objects":93,"num_object_clones":18,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":57,"num_read":20625,"num_read_kb":52541,"num_write":642,"num_write_kb":23374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"20","length":"8"}]},{"pgid":"3.17","version":"139'769","reported_seq":18557,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-05-08T08:54:52.403077+0000","last_change":"2024-05-08T08:54:51.404027+0000","last_active":"2024-05-08T08:54:52.403077+0000","last_peered":"2024-05-08T08:54:52.403077+0000","last_clean":"2024-05-08T08:54:52.403077+0000","last_became_active":"2024-05-08T08:53:11.358391+0000","last_became_peered":"2024-05-08T08:53:11.358391+0000","last_unstale":"2024-05-08T08:54:52.403077+0000","last_undegraded":"2024-05-08T08:54:52.403077+0000","last_fullsized":"2024-05-08T08:54:52.403077+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":769,"log_dups_size":0,"ondisk_log_size":769,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:17:40.927831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001876628,"stat_sum":{"num_bytes":24745106,"num_objects":68,"num_object_clones":9,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":37,"num_read":20201,"num_read_kb":76886,"num_write":905,"num_write_kb":16919,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":97,"num_bytes_recovered":2508994,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"20","length":"8"}]},{"pgid":"3.16","version":"133'1382","reported_seq":21275,"reported_epoch":140,"state":"remapped+peering","last_fresh":"2024-05-08T08:54:51.408481+0000","last_change":"2024-05-08T08:54:52.409817+0000","last_active":"2024-05-08T08:54:51.408481+0000","last_peered":"2024-05-08T08:54:44.476842+0000","last_clean":"2024-05-08T08:54:38.848129+0000","last_became_active":"2024-05-08T08:54:42.728792+0000","last_became_peered":"2024-05-08T08:54:42.728792+0000","last_unstale":"2024-05-08T08:54:51.408481+0000","last_undegraded":"2024-05-08T08:54:51.408481+0000","last_fullsized":"2024-05-08T08:54:51.408481+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":1382,"log_dups_size":0,"ondisk_log_size":1382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0076892219999999999,"stat_sum":{"num_bytes":37865151,"num_objects":77,"num_object_clones":11,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":42,"num_read":25274,"num_read_kb":87087,"num_write":2202,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":118,"num_bytes_recovered":21349055,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,5],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"21","length":"7"}]},{"pgid":"3.15","version":"134'639","reported_seq":16256,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-05-08T08:54:52.410385+0000","last_change":"2024-05-08T08:54:51.410857+0000","last_active":"2024-05-08T08:54:52.410385+0000","last_peered":"2024-05-08T08:54:52.410385+0000","last_clean":"2024-05-08T08:54:52.410385+0000","last_became_active":"2024-05-08T08:52:11.931695+0000","last_became_peered":"2024-05-08T08:52:11.931695+0000","last_unstale":"2024-05-08T08:54:52.410385+0000","last_undegraded":"2024-05-08T08:54:52.410385+0000","last_fullsized":"2024-05-08T08:54:52.410385+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":639,"log_dups_size":0,"ondisk_log_size":639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:18:40.327618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001648006,"stat_sum":{"num_bytes":24120358,"num_objects":79,"num_object_clones":11,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":50,"num_read":17480,"num_read_kb":52978,"num_write":748,"num_write_kb":8757,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"20","length":"8"}]},{"pgid":"3.14","version":"140'649","reported_seq":12292,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-05-08T08:54:52.410134+0000","last_change":"2024-05-08T08:54:51.410615+0000","last_active":"2024-05-08T08:54:52.410134+0000","last_peered":"2024-05-08T08:54:52.410134+0000","last_clean":"2024-05-08T08:54:52.410134+0000","last_became_active":"2024-05-08T08:53:23.481362+0000","last_became_peered":"2024-05-08T08:53:23.481362+0000","last_unstale":"2024-05-08T08:54:52.410134+0000","last_undegraded":"2024-05-08T08:54:52.410134+0000","last_fullsized":"2024-05-08T08:54:52.410134+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":649,"log_dups_size":0,"ondisk_log_size":649,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:30:37.908701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013395919999999999,"stat_sum":{"num_bytes":32628816,"num_objects":75,"num_object_clones":14,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":41,"num_read":10548,"num_read_kb":32995,"num_write":620,"num_write_kb":11448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":3612768,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"20","length":"8"}]},{"pgid":"2.7","version":"0'0","reported_seq":270,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-05-08T08:54:52.410056+0000","last_change":"2024-05-08T08:53:20.362387+0000","last_active":"2024-05-08T08:54:52.410056+0000","last_peered":"2024-05-08T08:54:52.410056+0000","last_clean":"2024-05-08T08:54:52.410056+0000","last_became_active":"2024-05-08T08:53:20.362113+0000","last_became_peered":"2024-05-08T08:53:20.362113+0000","last_unstale":"2024-05-08T08:54:52.410056+0000","last_undegraded":"2024-05-08T08:54:52.410056+0000","last_fullsized":"2024-05-08T08:54:52.410056+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:28:25.204125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"139'562","reported_seq":19897,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-05-08T08:54:52.403178+0000","last_change":"2024-05-08T08:54:51.404687+0000","last_active":"2024-05-08T08:54:52.403178+0000","last_peered":"2024-05-08T08:54:52.403178+0000","last_clean":"2024-05-08T08:54:52.403178+0000","last_became_active":"2024-05-08T08:53:49.579888+0000","last_became_peered":"2024-05-08T08:53:49.579888+0000","last_unstale":"2024-05-08T08:54:52.403178+0000","last_undegraded":"2024-05-08T08:54:52.403178+0000","last_fullsized":"2024-05-08T08:54:52.403178+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":562,"log_dups_size":0,"ondisk_log_size":562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:42:03.154969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058769099999999999,"stat_sum":{"num_bytes":40280265,"num_objects":78,"num_object_clones":9,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":48,"num_read":21568,"num_read_kb":88487,"num_write":543,"num_write_kb":17537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"20","length":"8"}]},{"pgid":"2.6","version":"0'0","reported_seq":261,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-05-08T08:54:52.403141+0000","last_change":"2024-05-08T08:52:03.896159+0000","last_active":"2024-05-08T08:54:52.403141+0000","last_peered":"2024-05-08T08:54:52.403141+0000","last_clean":"2024-05-08T08:54:52.403141+0000","last_became_active":"2024-05-08T08:52:03.895797+0000","last_became_peered":"2024-05-08T08:52:03.895797+0000","last_unstale":"2024-05-08T08:54:52.403141+0000","last_undegraded":"2024-05-08T08:54:52.403141+0000","last_fullsized":"2024-05-08T08:54:52.403141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:34:33.513525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"138'441","reported_seq":6496,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-05-08T08:54:52.409816+0000","last_change":"2024-05-08T08:54:51.404541+0000","last_active":"2024-05-08T08:54:52.409816+0000","last_peered":"2024-05-08T08:54:52.409816+0000","last_clean":"2024-05-08T08:54:52.409816+0000","last_became_active":"2024-05-08T08:54:32.801917+0000","last_became_peered":"2024-05-08T08:54:32.801917+0000","last_unstale":"2024-05-08T08:54:52.409816+0000","last_undegraded":"2024-05-08T08:54:52.409816+0000","last_fullsized":"2024-05-08T08:54:52.409816+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":441,"log_dups_size":0,"ondisk_log_size":441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:06:15.074782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081773200000000005,"stat_sum":{"num_bytes":12354706,"num_objects":87,"num_object_clones":16,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":49,"num_read":6599,"num_read_kb":34395,"num_write":381,"num_write_kb":12018,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":162,"num_bytes_recovered":8401073,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"20","length":"8"}]},{"pgid":"2.5","version":"0'0","reported_seq":260,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-05-08T08:54:52.410582+0000","last_change":"2024-05-08T08:52:04.144764+0000","last_active":"2024-05-08T08:54:52.410582+0000","last_peered":"2024-05-08T08:54:52.410582+0000","last_clean":"2024-05-08T08:54:52.410582+0000","last_became_active":"2024-05-08T08:52:04.144444+0000","last_became_peered":"2024-05-08T08:52:04.144444+0000","last_unstale":"2024-05-08T08:54:52.410582+0000","last_undegraded":"2024-05-08T08:54:52.410582+0000","last_fullsized":"2024-05-08T08:54:52.410582+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:58:31.498922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"134'738","reported_seq":26138,"reported_epoch":140,"state":"remapped+peering","last_fresh":"2024-05-08T08:54:51.403633+0000","last_change":"2024-05-08T08:54:52.403236+0000","last_active":"2024-05-08T08:54:51.403633+0000","last_peered":"2024-05-08T08:54:50.411123+0000","last_clean":"2024-05-08T08:54:50.411123+0000","last_became_active":"2024-05-08T08:54:45.374633+0000","last_became_peered":"2024-05-08T08:54:45.374633+0000","last_unstale":"2024-05-08T08:54:51.403633+0000","last_undegraded":"2024-05-08T08:54:51.403633+0000","last_fullsized":"2024-05-08T08:54:51.403633+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":738,"log_dups_size":0,"ondisk_log_size":738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014618820000000001,"stat_sum":{"num_bytes":44272640,"num_objects":106,"num_object_clones":17,"num_object_copies":212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":106,"num_whiteouts":62,"num_read":21930,"num_read_kb":51626,"num_write":578,"num_write_kb":12044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":407,"num_bytes_recovered":20414976,"num_keys_recovered":77,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,5],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"21","length":"7"}]},{"pgid":"2.4","version":"0'0","reported_seq":261,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-05-08T08:54:52.403139+0000","last_change":"2024-05-08T08:52:04.144543+0000","last_active":"2024-05-08T08:54:52.403139+0000","last_peered":"2024-05-08T08:54:52.403139+0000","last_clean":"2024-05-08T08:54:52.403139+0000","last_became_active":"2024-05-08T08:52:04.144242+0000","last_became_peered":"2024-05-08T08:52:04.144242+0000","last_unstale":"2024-05-08T08:54:52.403139+0000","last_undegraded":"2024-05-08T08:54:52.403139+0000","last_fullsized":"2024-05-08T08:54:52.403139+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:57:36.844633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"128'869","reported_seq":14044,"reported_epoch":140,"state":"remapped+peering","last_fresh":"2024-05-08T08:54:51.402841+0000","last_change":"2024-05-08T08:54:52.404088+0000","last_active":"2024-05-08T08:54:51.402841+0000","last_peered":"2024-05-08T08:54:50.414605+0000","last_clean":"2024-05-08T08:54:50.414605+0000","last_became_active":"2024-05-08T08:54:46.373299+0000","last_became_peered":"2024-05-08T08:54:46.373299+0000","last_unstale":"2024-05-08T08:54:51.402841+0000","last_undegraded":"2024-05-08T08:54:51.402841+0000","last_fullsized":"2024-05-08T08:54:51.402841+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":869,"log_dups_size":0,"ondisk_log_size":869,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060484199999999997,"stat_sum":{"num_bytes":14327035,"num_objects":99,"num_object_clones":13,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":67,"num_read":15031,"num_read_kb":32630,"num_write":1093,"num_write_kb":10601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":318,"num_bytes_recovered":9218550,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,5],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"21","length":"7"}]},{"pgid":"3.3","version":"139'637","reported_seq":11170,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-05-08T08:54:52.410988+0000","last_change":"2024-05-08T08:54:51.404774+0000","last_active":"2024-05-08T08:54:52.410988+0000","last_peered":"2024-05-08T08:54:52.410988+0000","last_clean":"2024-05-08T08:54:52.410988+0000","last_became_active":"2024-05-08T08:53:24.480423+0000","last_became_peered":"2024-05-08T08:53:24.480423+0000","last_unstale":"2024-05-08T08:54:52.410988+0000","last_undegraded":"2024-05-08T08:54:52.410988+0000","last_fullsized":"2024-05-08T08:54:52.410988+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":637,"log_dups_size":0,"ondisk_log_size":637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:26:31.464322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097852500000000005,"stat_sum":{"num_bytes":27822342,"num_objects":72,"num_object_clones":12,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":45,"num_read":11608,"num_read_kb":33346,"num_write":707,"num_write_kb":6536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":101,"num_bytes_recovered":5689443,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"20","length":"8"}]},{"pgid":"2.2","version":"0'0","reported_seq":251,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-05-08T08:54:52.407241+0000","last_change":"2024-05-08T08:54:52.407241+0000","last_active":"2024-05-08T08:54:52.407241+0000","last_peered":"2024-05-08T08:54:52.407241+0000","last_clean":"2024-05-08T08:54:52.407241+0000","last_became_active":"2024-05-08T08:54:52.406927+0000","last_became_peered":"2024-05-08T08:54:52.406927+0000","last_unstale":"2024-05-08T08:54:52.407241+0000","last_undegraded":"2024-05-08T08:54:52.407241+0000","last_fullsized":"2024-05-08T08:54:52.407241+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:54:04.389705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"139'654","reported_seq":13816,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-05-08T08:54:52.402945+0000","last_change":"2024-05-08T08:54:51.404256+0000","last_active":"2024-05-08T08:54:52.402945+0000","last_peered":"2024-05-08T08:54:52.402945+0000","last_clean":"2024-05-08T08:54:52.402945+0000","last_became_active":"2024-05-08T08:52:11.928301+0000","last_became_peered":"2024-05-08T08:52:11.928301+0000","last_unstale":"2024-05-08T08:54:52.402945+0000","last_undegraded":"2024-05-08T08:54:52.402945+0000","last_fullsized":"2024-05-08T08:54:52.402945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":654,"log_dups_size":0,"ondisk_log_size":654,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:05:46.248287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021916219999999998,"stat_sum":{"num_bytes":36703847,"num_objects":79,"num_object_clones":12,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":48,"num_read":10493,"num_read_kb":43924,"num_write":607,"num_write_kb":16296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"20","length":"8"}]},{"pgid":"2.1","version":"0'0","reported_seq":261,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-05-08T08:54:52.402693+0000","last_change":"2024-05-08T08:52:03.891768+0000","last_active":"2024-05-08T08:54:52.402693+0000","last_peered":"2024-05-08T08:54:52.402693+0000","last_clean":"2024-05-08T08:54:52.402693+0000","last_became_active":"2024-05-08T08:52:03.891374+0000","last_became_peered":"2024-05-08T08:52:03.891374+0000","last_unstale":"2024-05-08T08:54:52.402693+0000","last_undegraded":"2024-05-08T08:54:52.402693+0000","last_fullsized":"2024-05-08T08:54:52.402693+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:18:10.240492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"136'1033","reported_seq":27178,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-05-08T08:54:52.410052+0000","last_change":"2024-05-08T08:54:51.404218+0000","last_active":"2024-05-08T08:54:52.410052+0000","last_peered":"2024-05-08T08:54:52.410052+0000","last_clean":"2024-05-08T08:54:52.410052+0000","last_became_active":"2024-05-08T08:53:49.581797+0000","last_became_peered":"2024-05-08T08:53:49.581797+0000","last_unstale":"2024-05-08T08:54:52.410052+0000","last_undegraded":"2024-05-08T08:54:52.410052+0000","last_fullsized":"2024-05-08T08:54:52.410052+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":1033,"log_dups_size":0,"ondisk_log_size":1033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:04:13.831084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056618300000000005,"stat_sum":{"num_bytes":41169228,"num_objects":95,"num_object_clones":15,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":54,"num_read":30363,"num_read_kb":55605,"num_write":788,"num_write_kb":9622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":135,"num_bytes_recovered":3549534,"num_keys_recovered":46,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"20","length":"8"}]},{"pgid":"2.0","version":"0'0","reported_seq":260,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-05-08T08:54:52.409322+0000","last_change":"2024-05-08T08:52:03.891551+0000","last_active":"2024-05-08T08:54:52.409322+0000","last_peered":"2024-05-08T08:54:52.409322+0000","last_clean":"2024-05-08T08:54:52.409322+0000","last_became_active":"2024-05-08T08:52:03.891296+0000","last_became_peered":"2024-05-08T08:52:03.891296+0000","last_unstale":"2024-05-08T08:54:52.409322+0000","last_undegraded":"2024-05-08T08:54:52.409322+0000","last_fullsized":"2024-05-08T08:54:52.409322+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:54.806343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"138'916","reported_seq":14468,"reported_epoch":140,"state":"remapped+peering","last_fresh":"2024-05-08T08:54:51.403716+0000","last_change":"2024-05-08T08:54:52.403456+0000","last_active":"2024-05-08T08:54:51.403716+0000","last_peered":"2024-05-08T08:54:50.411517+0000","last_clean":"2024-05-08T08:54:50.411517+0000","last_became_active":"2024-05-08T08:54:48.370622+0000","last_became_peered":"2024-05-08T08:54:48.370622+0000","last_unstale":"2024-05-08T08:54:51.403716+0000","last_undegraded":"2024-05-08T08:54:51.403716+0000","last_fullsized":"2024-05-08T08:54:51.403716+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":916,"log_dups_size":0,"ondisk_log_size":916,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00176542,"stat_sum":{"num_bytes":18746197,"num_objects":92,"num_object_clones":15,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":54,"num_read":15402,"num_read_kb":63356,"num_write":1209,"num_write_kb":13964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":295,"num_bytes_recovered":14447932,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,5],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"21","length":"7"}]},{"pgid":"2.3","version":"0'0","reported_seq":251,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-05-08T08:54:52.405718+0000","last_change":"2024-05-08T08:54:52.405718+0000","last_active":"2024-05-08T08:54:52.405718+0000","last_peered":"2024-05-08T08:54:52.405718+0000","last_clean":"2024-05-08T08:54:52.405718+0000","last_became_active":"2024-05-08T08:54:52.405382+0000","last_became_peered":"2024-05-08T08:54:52.405382+0000","last_unstale":"2024-05-08T08:54:52.405718+0000","last_undegraded":"2024-05-08T08:54:52.405718+0000","last_fullsized":"2024-05-08T08:54:52.405718+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:26:02.768306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":416,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-05-08T08:54:52.409428+0000","last_change":"2024-05-08T08:51:58.861899+0000","last_active":"2024-05-08T08:54:52.409428+0000","last_peered":"2024-05-08T08:54:52.409428+0000","last_clean":"2024-05-08T08:54:52.409428+0000","last_became_active":"2024-05-08T08:51:58.861596+0000","last_became_peered":"2024-05-08T08:51:58.861596+0000","last_unstale":"2024-05-08T08:54:52.409428+0000","last_undegraded":"2024-05-08T08:54:52.409428+0000","last_fullsized":"2024-05-08T08:54:52.409428+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:51:57.849666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:51:57.849666+0000","last_clean_scrub_stamp":"2024-05-08T08:51:57.849666+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:53:21.739950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"139'352","reported_seq":9532,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-05-08T08:54:52.402746+0000","last_change":"2024-05-08T08:54:51.403335+0000","last_active":"2024-05-08T08:54:52.402746+0000","last_peered":"2024-05-08T08:54:52.402746+0000","last_clean":"2024-05-08T08:54:52.402746+0000","last_became_active":"2024-05-08T08:52:11.926975+0000","last_became_peered":"2024-05-08T08:52:11.926975+0000","last_unstale":"2024-05-08T08:54:52.402746+0000","last_undegraded":"2024-05-08T08:54:52.402746+0000","last_fullsized":"2024-05-08T08:54:52.402746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:52:57.711747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013262910000000001,"stat_sum":{"num_bytes":26299440,"num_objects":79,"num_object_clones":15,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":48,"num_read":8892,"num_read_kb":36553,"num_write":324,"num_write_kb":8985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"20","length":"8"}]},{"pgid":"3.9","version":"139'438","reported_seq":14164,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-05-08T08:54:52.410230+0000","last_change":"2024-05-08T08:54:51.404868+0000","last_active":"2024-05-08T08:54:52.410230+0000","last_peered":"2024-05-08T08:54:52.410230+0000","last_clean":"2024-05-08T08:54:52.410230+0000","last_became_active":"2024-05-08T08:53:23.478564+0000","last_became_peered":"2024-05-08T08:53:23.478564+0000","last_unstale":"2024-05-08T08:54:52.410230+0000","last_undegraded":"2024-05-08T08:54:52.410230+0000","last_fullsized":"2024-05-08T08:54:52.410230+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":438,"log_dups_size":0,"ondisk_log_size":438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:37:14.671839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092059900000000003,"stat_sum":{"num_bytes":28709407,"num_objects":84,"num_object_clones":10,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":51,"num_read":14027,"num_read_kb":62066,"num_write":374,"num_write_kb":16560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":131,"num_bytes_recovered":12377088,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"20","length":"8"}]},{"pgid":"3.a","version":"138'1134","reported_seq":11375,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-05-08T08:54:52.409893+0000","last_change":"2024-05-08T08:54:51.410514+0000","last_active":"2024-05-08T08:54:52.409893+0000","last_peered":"2024-05-08T08:54:52.409893+0000","last_clean":"2024-05-08T08:54:52.409893+0000","last_became_active":"2024-05-08T08:53:24.480359+0000","last_became_peered":"2024-05-08T08:53:24.480359+0000","last_unstale":"2024-05-08T08:54:52.409893+0000","last_undegraded":"2024-05-08T08:54:52.409893+0000","last_fullsized":"2024-05-08T08:54:52.409893+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":1134,"log_dups_size":0,"ondisk_log_size":1134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:53:07.366999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014592419999999999,"stat_sum":{"num_bytes":20702063,"num_objects":80,"num_object_clones":12,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":50,"num_read":15411,"num_read_kb":29119,"num_write":1370,"num_write_kb":6314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":2175343,"num_keys_recovered":61,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"20","length":"8"}]},{"pgid":"3.b","version":"138'543","reported_seq":11333,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-05-08T08:54:52.402942+0000","last_change":"2024-05-08T08:54:51.402688+0000","last_active":"2024-05-08T08:54:52.402942+0000","last_peered":"2024-05-08T08:54:52.402942+0000","last_clean":"2024-05-08T08:54:52.402942+0000","last_became_active":"2024-05-08T08:52:11.927196+0000","last_became_peered":"2024-05-08T08:52:11.927196+0000","last_unstale":"2024-05-08T08:54:52.402942+0000","last_undegraded":"2024-05-08T08:54:52.402942+0000","last_fullsized":"2024-05-08T08:54:52.402942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":543,"log_dups_size":0,"ondisk_log_size":543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:49:41.163880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076944299999999997,"stat_sum":{"num_bytes":35086880,"num_objects":89,"num_object_clones":10,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89,"num_whiteouts":58,"num_read":11574,"num_read_kb":39192,"num_write":460,"num_write_kb":7713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"20","length":"8"}]},{"pgid":"3.c","version":"139'341","reported_seq":13673,"reported_epoch":139,"state":"remapped+peering","last_fresh":"2024-05-08T08:54:50.473059+0000","last_change":"2024-05-08T08:54:52.405756+0000","last_active":"2024-05-08T08:54:52.405756+0000","last_peered":"2024-05-08T08:54:50.473059+0000","last_clean":"2024-05-08T08:54:50.473059+0000","last_became_active":"2024-05-08T08:54:44.388044+0000","last_became_peered":"2024-05-08T08:54:44.388044+0000","last_unstale":"2024-05-08T08:54:50.473059+0000","last_undegraded":"2024-05-08T08:54:50.473059+0000","last_fullsized":"2024-05-08T08:54:50.473059+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078240099999999995,"stat_sum":{"num_bytes":10757307,"num_objects":76,"num_object_clones":9,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":48,"num_read":13093,"num_read_kb":41263,"num_write":304,"num_write_kb":7340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":241,"num_bytes_recovered":6900086,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,5],"up_primary":3,"acting_primary":0,"purged_snaps":[{"start":"21","length":"7"}]},{"pgid":"3.d","version":"139'448","reported_seq":8679,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-05-08T08:54:52.410488+0000","last_change":"2024-05-08T08:54:51.410867+0000","last_active":"2024-05-08T08:54:52.410488+0000","last_peered":"2024-05-08T08:54:52.410488+0000","last_clean":"2024-05-08T08:54:52.410488+0000","last_became_active":"2024-05-08T08:52:11.932594+0000","last_became_peered":"2024-05-08T08:52:11.932594+0000","last_unstale":"2024-05-08T08:54:52.410488+0000","last_undegraded":"2024-05-08T08:54:52.410488+0000","last_fullsized":"2024-05-08T08:54:52.410488+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":448,"log_dups_size":0,"ondisk_log_size":448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:40:06.613286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001436894,"stat_sum":{"num_bytes":13511168,"num_objects":72,"num_object_clones":8,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":50,"num_read":9572,"num_read_kb":34328,"num_write":388,"num_write_kb":10002,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"20","length":"8"}]},{"pgid":"3.e","version":"139'588","reported_seq":9309,"reported_epoch":140,"state":"remapped+peering","last_fresh":"2024-05-08T08:54:51.404527+0000","last_change":"2024-05-08T08:54:52.411651+0000","last_active":"2024-05-08T08:54:51.404527+0000","last_peered":"2024-05-08T08:54:50.560913+0000","last_clean":"2024-05-08T08:54:50.560913+0000","last_became_active":"2024-05-08T08:54:45.374844+0000","last_became_peered":"2024-05-08T08:54:45.374844+0000","last_unstale":"2024-05-08T08:54:51.404527+0000","last_undegraded":"2024-05-08T08:54:51.404527+0000","last_fullsized":"2024-05-08T08:54:51.404527+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":588,"log_dups_size":0,"ondisk_log_size":588,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039349600000000002,"stat_sum":{"num_bytes":16544832,"num_objects":57,"num_object_clones":11,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":33,"num_read":16187,"num_read_kb":47192,"num_write":610,"num_write_kb":12221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":317,"num_bytes_recovered":21501568,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,5],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"21","length":"7"}]},{"pgid":"3.f","version":"135'510","reported_seq":7675,"reported_epoch":140,"state":"remapped+peering","last_fresh":"2024-05-08T08:54:51.408641+0000","last_change":"2024-05-08T08:54:52.409706+0000","last_active":"2024-05-08T08:54:51.408641+0000","last_peered":"2024-05-08T08:54:46.378237+0000","last_clean":"2024-05-08T08:54:46.378237+0000","last_became_active":"2024-05-08T08:54:46.375855+0000","last_became_peered":"2024-05-08T08:54:46.375855+0000","last_unstale":"2024-05-08T08:54:51.408641+0000","last_undegraded":"2024-05-08T08:54:51.408641+0000","last_fullsized":"2024-05-08T08:54:51.408641+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":510,"log_dups_size":0,"ondisk_log_size":510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.00065011300000000001,"stat_sum":{"num_bytes":18809042,"num_objects":71,"num_object_clones":7,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":43,"num_read":8463,"num_read_kb":38019,"num_write":515,"num_write_kb":9621,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":249,"num_bytes_recovered":12330404,"num_keys_recovered":46,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,5],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"21","length":"7"}]},{"pgid":"3.10","version":"139'697","reported_seq":8033,"reported_epoch":141,"state":"remapped+peering","last_fresh":"2024-05-08T08:54:52.405008+0000","last_change":"2024-05-08T08:54:52.405008+0000","last_active":"2024-05-08T08:54:52.405008+0000","last_peered":"2024-05-08T08:54:50.472148+0000","last_clean":"2024-05-08T08:54:50.472148+0000","last_became_active":"2024-05-08T08:54:47.375835+0000","last_became_peered":"2024-05-08T08:54:47.375835+0000","last_unstale":"2024-05-08T08:54:52.405008+0000","last_undegraded":"2024-05-08T08:54:52.405008+0000","last_fullsized":"2024-05-08T08:54:52.405008+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":697,"log_dups_size":0,"ondisk_log_size":697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041294999999999998,"stat_sum":{"num_bytes":30190145,"num_objects":79,"num_object_clones":13,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":51,"num_read":9056,"num_read_kb":41234,"num_write":956,"num_write_kb":14552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":255,"num_bytes_recovered":15104162,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[{"start":"21","length":"7"}]},{"pgid":"3.11","version":"135'685","reported_seq":11487,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-05-08T08:54:52.409551+0000","last_change":"2024-05-08T08:54:51.409189+0000","last_active":"2024-05-08T08:54:52.409551+0000","last_peered":"2024-05-08T08:54:52.409551+0000","last_clean":"2024-05-08T08:54:52.409551+0000","last_became_active":"2024-05-08T08:53:24.479891+0000","last_became_peered":"2024-05-08T08:53:24.479891+0000","last_unstale":"2024-05-08T08:54:52.409551+0000","last_undegraded":"2024-05-08T08:54:52.409551+0000","last_fullsized":"2024-05-08T08:54:52.409551+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":685,"log_dups_size":0,"ondisk_log_size":685,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:40:58.178543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048480799999999998,"stat_sum":{"num_bytes":21197312,"num_objects":84,"num_object_clones":13,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":54,"num_read":13068,"num_read_kb":36393,"num_write":553,"num_write_kb":10914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":131,"num_bytes_recovered":2604032,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"20","length":"8"}]},{"pgid":"3.12","version":"132'1450","reported_seq":24816,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-05-08T08:54:52.409795+0000","last_change":"2024-05-08T08:54:51.409429+0000","last_active":"2024-05-08T08:54:52.409795+0000","last_peered":"2024-05-08T08:54:52.409795+0000","last_clean":"2024-05-08T08:54:52.409795+0000","last_became_active":"2024-05-08T08:53:49.580557+0000","last_became_peered":"2024-05-08T08:53:49.580557+0000","last_unstale":"2024-05-08T08:54:52.409795+0000","last_undegraded":"2024-05-08T08:54:52.409795+0000","last_fullsized":"2024-05-08T08:54:52.409795+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":1450,"log_dups_size":0,"ondisk_log_size":1450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:06:37.756250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047275000000000003,"stat_sum":{"num_bytes":41194644,"num_objects":97,"num_object_clones":9,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":56,"num_read":30279,"num_read_kb":62140,"num_write":2038,"num_write_kb":12052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"20","length":"8"}]},{"pgid":"3.13","version":"136'581","reported_seq":10728,"reported_epoch":140,"state":"remapped+peering","last_fresh":"2024-05-08T08:54:51.408509+0000","last_change":"2024-05-08T08:54:52.409945+0000","last_active":"2024-05-08T08:54:51.408509+0000","last_peered":"2024-05-08T08:54:50.400493+0000","last_clean":"2024-05-08T08:54:50.400493+0000","last_became_active":"2024-05-08T08:54:45.375017+0000","last_became_peered":"2024-05-08T08:54:45.375017+0000","last_unstale":"2024-05-08T08:54:51.408509+0000","last_undegraded":"2024-05-08T08:54:51.408509+0000","last_fullsized":"2024-05-08T08:54:51.408509+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":581,"log_dups_size":0,"ondisk_log_size":581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088758399999999996,"stat_sum":{"num_bytes":48694399,"num_objects":95,"num_object_clones":18,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":52,"num_read":11009,"num_read_kb":41083,"num_write":565,"num_write_kb":9071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":283,"num_bytes_recovered":13314302,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,5],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"21","length":"7"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":969767662,"num_objects":2670,"num_object_clones":393,"num_object_copies":5340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2670,"num_whiteouts":1616,"num_read":504965,"num_read_kb":1621547,"num_write":24417,"num_write_kb":451458,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4647,"num_bytes_recovered":275353135,"num_keys_recovered":639,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65069056,"data_stored":535611868,"data_compressed":28225501,"data_compressed_allocated":53035008,"data_compressed_original":528163840,"omap_allocated":0,"internal_metadata":0},"log_size":22231,"ondisk_log_size":22231,"up":64,"acting":53,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":11046,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":0,"seq":47244640293,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640296,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":75028,"kb_used_data":13320,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94296812,"statfs":{"total":96636764160,"available":96559935488,"internally_reserved":0,"allocated":13639680,"data_stored":106943586,"data_compressed":5545408,"data_compressed_allocated":10432512,"data_compressed_original":105320448,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640295,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87088,"kb_used_data":15644,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94284752,"statfs":{"total":96636764160,"available":96547586048,"internally_reserved":0,"allocated":16019456,"data_stored":117187385,"data_compressed":6118245,"data_compressed_allocated":10969088,"data_compressed_original":114229760,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640294,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42732,"kb_used_data":7392,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94329108,"statfs":{"total":96636764160,"available":96593006592,"internally_reserved":0,"allocated":7569408,"data_stored":46381369,"data_compressed":2583853,"data_compressed_allocated":5046272,"data_compressed_original":45205504,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640293,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72664,"kb_used_data":9544,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94299176,"statfs":{"total":96636764160,"available":96562356224,"internally_reserved":0,"allocated":9773056,"data_stored":70272902,"data_compressed":3557596,"data_compressed_allocated":6823936,"data_compressed_original":68849664,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640296,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82020,"kb_used_data":14744,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94289820,"statfs":{"total":96636764160,"available":96552775680,"internally_reserved":0,"allocated":15097856,"data_stored":100703658,"data_compressed":5384940,"data_compressed_allocated":9932800,"data_compressed_original":97809920,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5523,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5523,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":13369344,"data_stored":99410711,"data_compressed":5379417,"data_compressed_allocated":9895936,"data_compressed_original":97236480,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8101888,"data_stored":69570323,"data_compressed":3557596,"data_compressed_allocated":6823936,"data_compressed_original":68849664,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":5885952,"data_stored":45673131,"data_compressed":2583853,"data_compressed_allocated":5046272,"data_compressed_original":45205504,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":14290944,"data_stored":115894438,"data_compressed":6112722,"data_compressed_allocated":10932224,"data_compressed_original":113656320,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":11956224,"data_stored":106235348,"data_compressed":5545408,"data_compressed_allocated":10432512,"data_compressed_original":105320448,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":11464704,"data_stored":98827917,"data_compressed":5046505,"data_compressed_allocated":9904128,"data_compressed_original":97895424,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T08:54:57.176 INFO:tasks.thrashosds.thrasher:Setting 2.5 to [3, 2] 2024-05-08T08:54:57.176 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '2.5', '3', '2'] 2024-05-08T08:54:57.176 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 2.5 3 2 2024-05-08T08:54:57.217 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:57.220+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:57.236 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:54:57.236 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:54:57.255 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:54:57.255 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:54:57.273 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:54:57.273 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:54:57.275 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:57.276 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:54:57.276 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:54:57.276 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:54:57.276 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:57.285 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:54:57.291 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:54:57.291 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:54:57.310 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:54:57.310 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:54:57.318 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:57.321+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:57.329 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:54:57.329 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:54:57.361 INFO:teuthology.orchestra.run.smithi059.stderr:nodeep-scrub is unset 2024-05-08T08:54:57.412 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:57.412 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:54:57.412 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:54:57.412 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:54:57.412 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.412 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:12443 3.0 3:02d0158e:::rbd_data.114071bcf828.0000000000000037:head [stat,delete] snapc 19=[19] RETRY=3 ondisk+retry+write+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:57.412 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.610027+0000", 2024-05-08T08:54:57.412 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 85.803894623999994, 2024-05-08T08:54:57.412 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.083586333999999998, 2024-05-08T08:54:57.412 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:57.413 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:57.413 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:57.413 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:57.413 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:57.413 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:57.413 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 12443 2024-05-08T08:54:57.413 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.413 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:57.413 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.413 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:57.413 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610027+0000", 2024-05-08T08:54:57.413 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:57.413 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.413 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.413 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:57.413 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610027+0000", 2024-05-08T08:54:57.414 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:57.414 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.414 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.414 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:57.414 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610028+0000", 2024-05-08T08:54:57.414 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.13e-07 2024-05-08T08:54:57.414 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.414 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.414 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:57.414 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610030+0000", 2024-05-08T08:54:57.414 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7669999999999999e-06 2024-05-08T08:54:57.414 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.414 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.414 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:57.414 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610039+0000", 2024-05-08T08:54:57.415 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4959999999999996e-06 2024-05-08T08:54:57.415 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.415 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.415 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:57.415 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610045+0000", 2024-05-08T08:54:57.415 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6779999999999997e-06 2024-05-08T08:54:57.415 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.415 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.415 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:57.415 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.660873+0000", 2024-05-08T08:54:57.415 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.050828748 2024-05-08T08:54:57.415 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.415 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.415 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:57.416 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.661017+0000", 2024-05-08T08:54:57.416 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014391999999999999 2024-05-08T08:54:57.416 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.416 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.416 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:57.416 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.693613+0000", 2024-05-08T08:54:57.416 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.032596111999999997 2024-05-08T08:54:57.416 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:57.416 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:57.416 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:57.416 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.416 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.416 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:14301 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [watch reconnect cookie 139837155901536 gen 12] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:57.416 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.645078+0000", 2024-05-08T08:54:57.416 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 85.768843985999993, 2024-05-08T08:54:57.416 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088281852999999993, 2024-05-08T08:54:57.417 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:57.417 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:57.417 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:57.417 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:57.417 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:57.417 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:57.417 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14301 2024-05-08T08:54:57.417 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.417 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:57.417 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.417 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:57.417 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645078+0000", 2024-05-08T08:54:57.417 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:57.417 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.417 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.418 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:57.418 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645078+0000", 2024-05-08T08:54:57.418 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:57.418 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.418 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.418 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:57.418 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645084+0000", 2024-05-08T08:54:57.418 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4620000000000001e-06 2024-05-08T08:54:57.418 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.418 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.418 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:57.418 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645088+0000", 2024-05-08T08:54:57.418 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8039999999999999e-06 2024-05-08T08:54:57.418 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.419 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.419 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:57.419 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645106+0000", 2024-05-08T08:54:57.419 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.8318999999999999e-05 2024-05-08T08:54:57.419 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.419 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.419 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:57.419 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645113+0000", 2024-05-08T08:54:57.419 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0310000000000001e-06 2024-05-08T08:54:57.419 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.419 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.419 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:57.419 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.670493+0000", 2024-05-08T08:54:57.419 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.025379993 2024-05-08T08:54:57.419 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.419 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.419 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:57.420 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.670571+0000", 2024-05-08T08:54:57.420 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7613000000000004e-05 2024-05-08T08:54:57.420 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.420 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.420 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:57.420 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733360+0000", 2024-05-08T08:54:57.420 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.062788630999999998 2024-05-08T08:54:57.420 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:57.420 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:57.420 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:57.420 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.420 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.420 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:14420 3.0 3:06a8948e:::rbd_data.114071bcf828.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:57.420 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.649639+0000", 2024-05-08T08:54:57.420 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 85.764282838, 2024-05-08T08:54:57.420 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.083851897999999994, 2024-05-08T08:54:57.421 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:57.421 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:57.421 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:54:57.421 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:57.421 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:57.421 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:57.421 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14420 2024-05-08T08:54:57.421 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.421 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:57.421 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.421 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:57.421 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649639+0000", 2024-05-08T08:54:57.421 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:57.421 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.421 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.421 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:57.421 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649639+0000", 2024-05-08T08:54:57.422 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:57.422 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.422 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.422 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:57.422 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649640+0000", 2024-05-08T08:54:57.422 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6899999999999997e-07 2024-05-08T08:54:57.422 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.422 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.422 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:57.422 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649895+0000", 2024-05-08T08:54:57.422 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00025531000000000002 2024-05-08T08:54:57.422 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.422 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.422 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:57.422 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649905+0000", 2024-05-08T08:54:57.423 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0198e-05 2024-05-08T08:54:57.423 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.423 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.423 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:57.423 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649911+0000", 2024-05-08T08:54:57.423 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2310000000000001e-06 2024-05-08T08:54:57.423 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.423 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.423 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:57.423 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.684099+0000", 2024-05-08T08:54:57.423 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.034188017000000001 2024-05-08T08:54:57.423 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.423 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.423 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:57.423 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.684172+0000", 2024-05-08T08:54:57.423 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3201999999999995e-05 2024-05-08T08:54:57.423 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.423 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.424 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 2", 2024-05-08T08:54:57.424 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.684418+0000", 2024-05-08T08:54:57.424 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00024559099999999997 2024-05-08T08:54:57.424 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.424 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.424 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:54:57.424 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.705538+0000", 2024-05-08T08:54:57.424 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.021119707000000001 2024-05-08T08:54:57.424 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.424 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.424 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:54:57.424 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733399+0000", 2024-05-08T08:54:57.424 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.027861543999999999 2024-05-08T08:54:57.424 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.424 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.424 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:54:57.425 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733445+0000", 2024-05-08T08:54:57.425 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5587000000000002e-05 2024-05-08T08:54:57.425 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.425 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.425 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:57.425 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733491+0000", 2024-05-08T08:54:57.425 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5841999999999999e-05 2024-05-08T08:54:57.425 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:57.425 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:57.425 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:57.425 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.425 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.425 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28547 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889179216] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:57.425 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.355654+0000", 2024-05-08T08:54:57.425 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 85.058267610000001, 2024-05-08T08:54:57.425 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078692261999999999, 2024-05-08T08:54:57.425 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:57.426 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:57.426 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:57.426 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:57.426 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:57.426 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:57.426 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28547 2024-05-08T08:54:57.426 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.426 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:57.426 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.426 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:57.426 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355654+0000", 2024-05-08T08:54:57.426 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:57.426 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.426 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.427 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:57.427 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355654+0000", 2024-05-08T08:54:57.427 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:57.427 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.427 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.427 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:57.427 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355654+0000", 2024-05-08T08:54:57.427 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3999999999999998e-07 2024-05-08T08:54:57.427 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.427 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.427 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:57.427 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355656+0000", 2024-05-08T08:54:57.427 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.353e-06 2024-05-08T08:54:57.427 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.427 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.427 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:57.428 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355663+0000", 2024-05-08T08:54:57.428 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2259999999999999e-06 2024-05-08T08:54:57.428 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.428 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.428 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:57.428 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355666+0000", 2024-05-08T08:54:57.428 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7930000000000002e-06 2024-05-08T08:54:57.428 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.428 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.428 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:57.428 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434117+0000", 2024-05-08T08:54:57.428 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078451231999999996 2024-05-08T08:54:57.428 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.428 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.428 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:57.428 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434198+0000", 2024-05-08T08:54:57.428 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0879999999999998e-05 2024-05-08T08:54:57.428 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.429 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.429 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:57.429 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434346+0000", 2024-05-08T08:54:57.429 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014853800000000001 2024-05-08T08:54:57.429 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:57.429 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:57.429 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:57.429 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.429 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.429 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28548 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889189696] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:57.429 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.355669+0000", 2024-05-08T08:54:57.429 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 85.058252260000003, 2024-05-08T08:54:57.429 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078956606999999998, 2024-05-08T08:54:57.429 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:57.429 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:57.429 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:57.430 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:57.430 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:57.430 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:57.430 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28548 2024-05-08T08:54:57.430 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.430 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:57.430 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.430 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:57.430 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355669+0000", 2024-05-08T08:54:57.430 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:57.430 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.430 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.430 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:57.430 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355669+0000", 2024-05-08T08:54:57.430 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:57.430 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.431 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.431 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:57.431 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355762+0000", 2024-05-08T08:54:57.431 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3015000000000003e-05 2024-05-08T08:54:57.431 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.431 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.431 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:57.431 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355765+0000", 2024-05-08T08:54:57.431 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.368e-06 2024-05-08T08:54:57.431 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.431 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.431 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:57.431 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355771+0000", 2024-05-08T08:54:57.431 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5989999999999998e-06 2024-05-08T08:54:57.431 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.432 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.432 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:57.432 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355775+0000", 2024-05-08T08:54:57.432 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7280000000000001e-06 2024-05-08T08:54:57.432 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.432 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.432 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:57.432 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434379+0000", 2024-05-08T08:54:57.432 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078603662000000005 2024-05-08T08:54:57.432 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.432 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.432 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:57.432 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434473+0000", 2024-05-08T08:54:57.432 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4104000000000006e-05 2024-05-08T08:54:57.432 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.432 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.432 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:57.432 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434626+0000", 2024-05-08T08:54:57.433 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00015313100000000001 2024-05-08T08:54:57.433 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:57.433 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:57.433 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:57.433 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.433 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.433 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28549 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889202400] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:57.433 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.355781+0000", 2024-05-08T08:54:57.433 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 85.058140686000002, 2024-05-08T08:54:57.433 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079117946999999994, 2024-05-08T08:54:57.433 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:57.433 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:57.433 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:57.433 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:57.433 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:57.433 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:57.433 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28549 2024-05-08T08:54:57.434 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.434 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:57.434 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.434 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:57.434 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355781+0000", 2024-05-08T08:54:57.434 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:57.434 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.434 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.434 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:57.434 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355781+0000", 2024-05-08T08:54:57.434 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:57.434 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.434 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.434 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:57.434 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356162+0000", 2024-05-08T08:54:57.435 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00038099100000000001 2024-05-08T08:54:57.435 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.435 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.435 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:57.435 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356165+0000", 2024-05-08T08:54:57.435 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1240000000000001e-06 2024-05-08T08:54:57.435 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.435 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.435 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:57.435 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356173+0000", 2024-05-08T08:54:57.435 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.6199999999999999e-06 2024-05-08T08:54:57.435 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.435 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.435 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:57.435 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356182+0000", 2024-05-08T08:54:57.435 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7760000000000003e-06 2024-05-08T08:54:57.435 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.436 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.436 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:57.436 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434675+0000", 2024-05-08T08:54:57.436 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078493119 2024-05-08T08:54:57.436 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.436 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.436 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:57.436 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434765+0000", 2024-05-08T08:54:57.436 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0402000000000006e-05 2024-05-08T08:54:57.436 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.436 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.436 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:57.436 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434899+0000", 2024-05-08T08:54:57.436 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013391499999999999 2024-05-08T08:54:57.436 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:57.436 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:57.436 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:57.437 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.437 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.437 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28550 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888597280] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:57.437 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:54:57.437 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 85.057729640000005, 2024-05-08T08:54:57.437 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078933990999999995, 2024-05-08T08:54:57.437 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:57.437 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:57.437 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:57.437 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:57.437 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:57.437 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:57.437 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28550 2024-05-08T08:54:57.437 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.437 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:57.437 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.438 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:57.438 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:54:57.438 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:57.438 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.438 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.438 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:57.438 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:54:57.438 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:57.438 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.438 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.438 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:57.438 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:54:57.438 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.34e-07 2024-05-08T08:54:57.438 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.439 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.439 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:57.439 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356195+0000", 2024-05-08T08:54:57.439 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8710000000000001e-06 2024-05-08T08:54:57.439 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.439 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.439 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:57.439 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356200+0000", 2024-05-08T08:54:57.439 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3000000000000003e-06 2024-05-08T08:54:57.439 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.439 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.439 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:57.439 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356203+0000", 2024-05-08T08:54:57.439 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4039999999999999e-06 2024-05-08T08:54:57.439 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.439 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.440 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:57.440 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434928+0000", 2024-05-08T08:54:57.440 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078725331999999995 2024-05-08T08:54:57.440 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.440 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.440 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:57.440 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435007+0000", 2024-05-08T08:54:57.440 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8415000000000001e-05 2024-05-08T08:54:57.440 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.440 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.440 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:57.440 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435126+0000", 2024-05-08T08:54:57.440 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011923500000000001 2024-05-08T08:54:57.440 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:57.440 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:57.440 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:57.441 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.441 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.441 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28551 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888527200] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:57.441 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.356210+0000", 2024-05-08T08:54:57.441 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 85.057711291999993, 2024-05-08T08:54:57.441 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079195072000000005, 2024-05-08T08:54:57.441 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:57.441 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:57.441 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:57.441 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:57.441 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:57.441 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:57.441 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28551 2024-05-08T08:54:57.441 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.441 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:57.441 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.442 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:57.442 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356210+0000", 2024-05-08T08:54:57.442 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:57.442 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.442 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.442 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:57.442 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356210+0000", 2024-05-08T08:54:57.442 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:57.442 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.442 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.442 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:57.442 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356211+0000", 2024-05-08T08:54:57.442 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4499999999999998e-07 2024-05-08T08:54:57.442 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.442 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.443 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:57.443 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356212+0000", 2024-05-08T08:54:57.443 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6029999999999999e-06 2024-05-08T08:54:57.443 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.443 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.443 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:57.443 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356221+0000", 2024-05-08T08:54:57.443 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4200000000000007e-06 2024-05-08T08:54:57.443 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.443 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.443 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:57.443 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356224+0000", 2024-05-08T08:54:57.443 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.038e-06 2024-05-08T08:54:57.443 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.443 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.443 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:57.443 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435178+0000", 2024-05-08T08:54:57.444 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078954625000000001 2024-05-08T08:54:57.444 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.444 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.444 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:57.444 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435256+0000", 2024-05-08T08:54:57.444 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7337999999999997e-05 2024-05-08T08:54:57.444 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.444 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.444 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:57.444 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435405+0000", 2024-05-08T08:54:57.444 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000149703 2024-05-08T08:54:57.444 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:57.444 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:57.444 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:57.444 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.445 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.445 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28552 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889231456] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:57.445 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.356228+0000", 2024-05-08T08:54:57.445 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 85.057693614000002, 2024-05-08T08:54:57.445 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079346920000000001, 2024-05-08T08:54:57.445 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:57.445 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:57.445 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:57.445 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:57.445 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:57.445 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:57.445 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28552 2024-05-08T08:54:57.445 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.445 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:57.445 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.445 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:57.446 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356228+0000", 2024-05-08T08:54:57.446 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:57.446 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.446 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.446 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:57.446 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356228+0000", 2024-05-08T08:54:57.446 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:57.446 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.446 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.446 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:57.446 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356959+0000", 2024-05-08T08:54:57.446 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00073097099999999996 2024-05-08T08:54:57.446 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.446 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.446 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:57.447 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356967+0000", 2024-05-08T08:54:57.447 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0409999999999998e-06 2024-05-08T08:54:57.447 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.447 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.447 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:57.447 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356979+0000", 2024-05-08T08:54:57.447 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1489e-05 2024-05-08T08:54:57.447 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.447 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.447 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:57.447 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356986+0000", 2024-05-08T08:54:57.447 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3699999999999997e-06 2024-05-08T08:54:57.447 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.447 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.447 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:57.448 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435433+0000", 2024-05-08T08:54:57.448 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078447520000000007 2024-05-08T08:54:57.448 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.448 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.448 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:57.448 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435484+0000", 2024-05-08T08:54:57.448 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0229000000000003e-05 2024-05-08T08:54:57.448 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.448 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.448 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:57.448 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435575+0000", 2024-05-08T08:54:57.448 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1299999999999997e-05 2024-05-08T08:54:57.448 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:57.448 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:57.448 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:57.448 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.449 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.449 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28553 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888713648] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:57.449 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:54:57.449 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 85.056918956000004, 2024-05-08T08:54:57.449 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078782034000000001, 2024-05-08T08:54:57.449 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:57.449 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:57.449 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:57.449 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:57.449 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:57.449 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:57.449 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28553 2024-05-08T08:54:57.449 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.449 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:57.449 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.449 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:57.449 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:54:57.450 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:57.450 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.450 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.450 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:57.450 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:54:57.450 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:57.450 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.450 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.450 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:57.450 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:54:57.450 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6600000000000002e-07 2024-05-08T08:54:57.450 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.450 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.450 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:57.450 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357006+0000", 2024-05-08T08:54:57.451 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7190000000000001e-06 2024-05-08T08:54:57.451 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.451 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.451 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:57.451 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357013+0000", 2024-05-08T08:54:57.451 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.435e-06 2024-05-08T08:54:57.451 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.451 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.451 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:57.451 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357018+0000", 2024-05-08T08:54:57.451 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7260000000000002e-06 2024-05-08T08:54:57.451 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.451 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.451 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:57.451 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435623+0000", 2024-05-08T08:54:57.452 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078604972999999995 2024-05-08T08:54:57.452 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.452 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.452 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:57.452 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435695+0000", 2024-05-08T08:54:57.452 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1959999999999995e-05 2024-05-08T08:54:57.452 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.452 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.452 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:57.452 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435785+0000", 2024-05-08T08:54:57.452 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.9754999999999997e-05 2024-05-08T08:54:57.452 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:57.452 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:57.452 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:57.452 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.452 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.453 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28554 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888980272] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:57.453 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357024+0000", 2024-05-08T08:54:57.453 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 85.056897320000004, 2024-05-08T08:54:57.453 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078999870999999999, 2024-05-08T08:54:57.453 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:57.453 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:57.453 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:57.453 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:57.453 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:57.453 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:57.453 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28554 2024-05-08T08:54:57.453 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.453 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:57.453 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.453 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:57.453 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357024+0000", 2024-05-08T08:54:57.453 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:57.453 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.454 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.454 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:57.454 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357024+0000", 2024-05-08T08:54:57.454 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:57.454 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.454 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.454 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:57.454 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357025+0000", 2024-05-08T08:54:57.454 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:54:57.454 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.454 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.454 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:57.454 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357027+0000", 2024-05-08T08:54:57.454 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2050000000000001e-06 2024-05-08T08:54:57.454 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.455 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.455 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:57.455 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357033+0000", 2024-05-08T08:54:57.455 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5679999999999996e-06 2024-05-08T08:54:57.455 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.455 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.455 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:57.455 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357038+0000", 2024-05-08T08:54:57.455 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2889999999999999e-06 2024-05-08T08:54:57.455 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.455 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.455 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:57.455 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435823+0000", 2024-05-08T08:54:57.455 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078785677999999998 2024-05-08T08:54:57.455 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.455 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.456 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:57.456 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435892+0000", 2024-05-08T08:54:57.456 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8379000000000004e-05 2024-05-08T08:54:57.456 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.456 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.456 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:57.456 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436024+0000", 2024-05-08T08:54:57.456 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013239799999999999 2024-05-08T08:54:57.456 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:57.456 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:57.456 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:57.456 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.456 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.456 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28555 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889001104] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:57.456 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:54:57.456 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 85.056878554999997, 2024-05-08T08:54:57.457 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079115742000000003, 2024-05-08T08:54:57.457 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:57.457 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:57.457 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:57.457 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:57.457 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:57.457 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:57.457 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28555 2024-05-08T08:54:57.457 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.457 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:57.457 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.457 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:57.457 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:54:57.457 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:57.457 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.457 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.457 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:57.458 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:54:57.458 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:57.458 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.458 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.458 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:57.458 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:54:57.458 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9200000000000002e-07 2024-05-08T08:54:57.458 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.458 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.458 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:57.458 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357046+0000", 2024-05-08T08:54:57.458 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1670000000000002e-06 2024-05-08T08:54:57.458 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.458 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.458 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:57.458 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357052+0000", 2024-05-08T08:54:57.459 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.139e-06 2024-05-08T08:54:57.459 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.459 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.459 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:57.459 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357056+0000", 2024-05-08T08:54:57.459 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2440000000000004e-06 2024-05-08T08:54:57.459 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.459 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.459 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:57.459 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436059+0000", 2024-05-08T08:54:57.459 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079003070999999994 2024-05-08T08:54:57.459 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.459 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.459 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:57.459 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436097+0000", 2024-05-08T08:54:57.459 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7784000000000003e-05 2024-05-08T08:54:57.459 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.459 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.460 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:57.460 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436159+0000", 2024-05-08T08:54:57.460 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2045000000000007e-05 2024-05-08T08:54:57.460 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:57.460 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:57.460 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:57.460 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.460 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.460 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28556 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889022064] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:57.460 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357065+0000", 2024-05-08T08:54:57.460 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 85.056856522000004, 2024-05-08T08:54:57.460 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079327976999999994, 2024-05-08T08:54:57.460 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:57.460 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:57.461 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:57.461 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:57.461 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:57.461 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:57.461 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28556 2024-05-08T08:54:57.461 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.461 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:57.461 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.461 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:57.461 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357065+0000", 2024-05-08T08:54:57.461 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:57.461 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.462 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.462 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:57.462 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357065+0000", 2024-05-08T08:54:57.462 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:57.462 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.462 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.462 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:57.462 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358958+0000", 2024-05-08T08:54:57.462 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001892915 2024-05-08T08:54:57.462 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.462 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.462 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:57.462 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358962+0000", 2024-05-08T08:54:57.462 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4599999999999999e-06 2024-05-08T08:54:57.462 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.463 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.463 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:57.463 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358970+0000", 2024-05-08T08:54:57.463 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.6090000000000001e-06 2024-05-08T08:54:57.463 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.463 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.463 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:57.463 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358975+0000", 2024-05-08T08:54:57.463 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7639999999999996e-06 2024-05-08T08:54:57.463 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.463 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.463 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:57.463 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436196+0000", 2024-05-08T08:54:57.463 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.077221564000000006 2024-05-08T08:54:57.463 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.463 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.464 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:57.464 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436269+0000", 2024-05-08T08:54:57.464 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2720999999999997e-05 2024-05-08T08:54:57.464 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.464 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.464 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:57.464 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436393+0000", 2024-05-08T08:54:57.464 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012394399999999999 2024-05-08T08:54:57.465 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:57.465 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:57.465 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:57.465 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.465 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.465 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28563 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888855520] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:57.465 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:54:57.465 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 85.054847123000002, 2024-05-08T08:54:57.465 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078795566999999997, 2024-05-08T08:54:57.465 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:57.465 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:57.465 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:57.465 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:57.465 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:57.465 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:57.465 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28563 2024-05-08T08:54:57.465 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.466 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:57.466 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.466 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:57.466 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:54:57.466 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:57.466 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.466 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.466 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:57.466 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:54:57.466 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:57.466 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.466 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.466 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:57.466 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:54:57.467 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1e-07 2024-05-08T08:54:57.467 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.467 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.467 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:57.467 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359080+0000", 2024-05-08T08:54:57.467 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8940000000000001e-06 2024-05-08T08:54:57.467 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.467 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.467 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:57.467 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359083+0000", 2024-05-08T08:54:57.467 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1949999999999999e-06 2024-05-08T08:54:57.467 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.467 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.467 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:57.467 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359085+0000", 2024-05-08T08:54:57.467 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4710000000000001e-06 2024-05-08T08:54:57.467 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.468 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.468 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:57.468 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437604+0000", 2024-05-08T08:54:57.468 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078518278999999996 2024-05-08T08:54:57.468 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.468 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.468 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:57.468 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437704+0000", 2024-05-08T08:54:57.468 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010010599999999999 2024-05-08T08:54:57.468 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.468 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.468 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:57.468 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437870+0000", 2024-05-08T08:54:57.468 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00016641200000000001 2024-05-08T08:54:57.468 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:57.468 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:57.468 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:57.469 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.469 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.469 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28564 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888538944] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:57.469 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.359089+0000", 2024-05-08T08:54:57.469 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 85.054832907999995, 2024-05-08T08:54:57.469 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078952675, 2024-05-08T08:54:57.469 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:57.469 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:57.469 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:57.469 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:57.469 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:57.469 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:57.469 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28564 2024-05-08T08:54:57.469 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.469 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:57.469 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.470 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:57.470 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359089+0000", 2024-05-08T08:54:57.470 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:57.470 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.470 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.470 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:57.470 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359089+0000", 2024-05-08T08:54:57.470 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:57.470 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.470 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.470 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:57.470 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360959+0000", 2024-05-08T08:54:57.470 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0018707089999999999 2024-05-08T08:54:57.470 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.470 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.470 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:57.471 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360963+0000", 2024-05-08T08:54:57.471 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0680000000000001e-06 2024-05-08T08:54:57.471 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.471 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.471 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:57.471 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360969+0000", 2024-05-08T08:54:57.471 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1920000000000002e-06 2024-05-08T08:54:57.471 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.471 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.471 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:57.471 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360973+0000", 2024-05-08T08:54:57.471 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.031e-06 2024-05-08T08:54:57.471 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.471 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.471 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:57.471 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437916+0000", 2024-05-08T08:54:57.472 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.076942939000000002 2024-05-08T08:54:57.472 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.472 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.472 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:57.472 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437973+0000", 2024-05-08T08:54:57.472 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7358999999999997e-05 2024-05-08T08:54:57.472 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.472 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.472 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:57.472 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.438041+0000", 2024-05-08T08:54:57.472 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8376999999999999e-05 2024-05-08T08:54:57.472 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:57.472 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:57.472 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:57.472 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.472 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.473 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28574 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889427600] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:57.473 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:54:57.473 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 85.052828400999999, 2024-05-08T08:54:57.473 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078782476000000004, 2024-05-08T08:54:57.473 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:57.473 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:57.473 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:57.473 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:57.473 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:57.473 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:57.473 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28574 2024-05-08T08:54:57.473 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.473 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:57.473 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.473 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:57.473 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:54:57.473 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:57.474 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.474 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.474 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:57.474 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:54:57.474 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:57.474 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.474 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.474 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:57.474 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:54:57.474 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1799999999999999e-07 2024-05-08T08:54:57.474 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.474 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.474 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:57.474 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361095+0000", 2024-05-08T08:54:57.474 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3850000000000001e-06 2024-05-08T08:54:57.475 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.475 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.475 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:57.475 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361098+0000", 2024-05-08T08:54:57.475 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7989999999999998e-06 2024-05-08T08:54:57.475 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.475 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.475 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:57.475 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361100+0000", 2024-05-08T08:54:57.475 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4669999999999998e-06 2024-05-08T08:54:57.475 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.475 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.475 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:57.475 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439768+0000", 2024-05-08T08:54:57.475 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078668057999999999 2024-05-08T08:54:57.476 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.476 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.476 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:57.476 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439811+0000", 2024-05-08T08:54:57.476 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2459000000000001e-05 2024-05-08T08:54:57.476 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.476 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.476 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:57.476 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439876+0000", 2024-05-08T08:54:57.476 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5090000000000002e-05 2024-05-08T08:54:57.476 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:57.476 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:57.476 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:57.476 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.476 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.476 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28575 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889439216] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:57.476 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361103+0000", 2024-05-08T08:54:57.476 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 85.052818385999998, 2024-05-08T08:54:57.477 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078966828000000003, 2024-05-08T08:54:57.477 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:57.477 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:57.477 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:57.477 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:57.477 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:57.477 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:57.477 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28575 2024-05-08T08:54:57.477 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.477 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:57.477 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.477 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:57.477 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361103+0000", 2024-05-08T08:54:57.477 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:57.477 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.477 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.477 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:57.478 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361103+0000", 2024-05-08T08:54:57.478 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:57.478 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.478 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.478 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:57.478 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361104+0000", 2024-05-08T08:54:57.478 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1400000000000001e-07 2024-05-08T08:54:57.478 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.478 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.478 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:57.478 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361105+0000", 2024-05-08T08:54:57.478 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5540000000000001e-06 2024-05-08T08:54:57.478 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.478 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.478 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:57.479 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361108+0000", 2024-05-08T08:54:57.479 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.841e-06 2024-05-08T08:54:57.479 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.479 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.479 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:57.479 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361110+0000", 2024-05-08T08:54:57.479 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.497e-06 2024-05-08T08:54:57.479 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.479 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.479 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:57.479 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439916+0000", 2024-05-08T08:54:57.479 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078805372999999998 2024-05-08T08:54:57.479 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.479 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.480 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:57.480 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440000+0000", 2024-05-08T08:54:57.480 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4730999999999999e-05 2024-05-08T08:54:57.480 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.480 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.480 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:57.480 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440070+0000", 2024-05-08T08:54:57.480 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9617999999999997e-05 2024-05-08T08:54:57.480 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:57.480 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:57.480 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:57.480 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.480 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.480 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28576 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889451328] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:57.480 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361113+0000", 2024-05-08T08:54:57.480 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 85.052808200000001, 2024-05-08T08:54:57.480 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079088862999999995, 2024-05-08T08:54:57.481 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:57.481 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:57.481 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:57.481 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:57.481 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:57.481 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:57.481 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28576 2024-05-08T08:54:57.481 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.481 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:57.481 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.481 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:57.481 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361113+0000", 2024-05-08T08:54:57.481 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:57.481 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.481 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.481 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:57.481 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361113+0000", 2024-05-08T08:54:57.481 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:57.482 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.482 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.482 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:57.482 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361114+0000", 2024-05-08T08:54:57.482 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1299999999999999e-07 2024-05-08T08:54:57.482 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.482 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.482 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:57.482 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361115+0000", 2024-05-08T08:54:57.482 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.4300000000000001e-06 2024-05-08T08:54:57.482 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.482 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.482 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:57.482 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361118+0000", 2024-05-08T08:54:57.483 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9000000000000002e-06 2024-05-08T08:54:57.483 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.483 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.483 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:57.483 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361120+0000", 2024-05-08T08:54:57.483 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4739999999999999e-06 2024-05-08T08:54:57.483 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.483 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.483 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:57.483 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440096+0000", 2024-05-08T08:54:57.483 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078975629000000006 2024-05-08T08:54:57.483 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.483 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.483 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:57.483 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440135+0000", 2024-05-08T08:54:57.483 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9202000000000002e-05 2024-05-08T08:54:57.483 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.484 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.484 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:57.484 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440202+0000", 2024-05-08T08:54:57.484 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7014999999999994e-05 2024-05-08T08:54:57.484 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:57.484 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:57.484 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:57.484 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.484 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.484 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28577 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889462640] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:57.484 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:54:57.484 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 85.052798018000004, 2024-05-08T08:54:57.484 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079194622000000006, 2024-05-08T08:54:57.484 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:57.484 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:57.485 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:57.485 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:57.485 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:57.485 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:57.485 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28577 2024-05-08T08:54:57.485 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.485 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:57.485 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.485 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:57.485 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:54:57.485 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:57.485 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.485 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.485 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:57.485 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:54:57.485 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:57.486 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.486 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.486 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:57.486 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:54:57.486 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1400000000000001e-07 2024-05-08T08:54:57.486 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.486 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.486 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:57.486 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361125+0000", 2024-05-08T08:54:57.486 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.372e-06 2024-05-08T08:54:57.486 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.486 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.486 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:57.487 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361128+0000", 2024-05-08T08:54:57.487 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0309999999999999e-06 2024-05-08T08:54:57.487 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.487 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.487 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:57.487 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361131+0000", 2024-05-08T08:54:57.487 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4040000000000002e-06 2024-05-08T08:54:57.487 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.487 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.487 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:57.487 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440225+0000", 2024-05-08T08:54:57.487 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079094600000000001 2024-05-08T08:54:57.487 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.487 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.488 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:57.488 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440260+0000", 2024-05-08T08:54:57.488 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4703000000000003e-05 2024-05-08T08:54:57.488 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.488 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.488 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:57.488 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440318+0000", 2024-05-08T08:54:57.488 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8298000000000003e-05 2024-05-08T08:54:57.488 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:57.488 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:57.488 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:57.488 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.488 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.488 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28578 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889474192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:54:57.488 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361134+0000", 2024-05-08T08:54:57.488 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 85.052787593000005, 2024-05-08T08:54:57.489 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079310295000000003, 2024-05-08T08:54:57.489 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:57.489 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:57.489 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:57.489 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:57.489 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:54:57.489 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:54:57.489 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28578 2024-05-08T08:54:57.489 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.489 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:57.489 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.489 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:57.489 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361134+0000", 2024-05-08T08:54:57.489 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:57.490 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.490 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.490 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:57.490 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361134+0000", 2024-05-08T08:54:57.490 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:57.490 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.490 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.490 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:57.490 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.362958+0000", 2024-05-08T08:54:57.490 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0018237380000000001 2024-05-08T08:54:57.490 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.490 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.490 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:57.490 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.362961+0000", 2024-05-08T08:54:57.491 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.123e-06 2024-05-08T08:54:57.491 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.491 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.491 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:57.491 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.397404+0000", 2024-05-08T08:54:57.491 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.034443226 2024-05-08T08:54:57.491 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.491 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.491 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:57.491 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.397439+0000", 2024-05-08T08:54:57.491 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4690000000000002e-05 2024-05-08T08:54:57.491 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.491 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.491 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:57.491 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440340+0000", 2024-05-08T08:54:57.491 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.042901419000000003 2024-05-08T08:54:57.491 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.492 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.492 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:57.492 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440376+0000", 2024-05-08T08:54:57.492 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5444999999999997e-05 2024-05-08T08:54:57.492 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.492 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.492 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:57.492 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440444+0000", 2024-05-08T08:54:57.492 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8653999999999997e-05 2024-05-08T08:54:57.492 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:57.492 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:57.492 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:57.492 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:57.492 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:57.492 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:57.494 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:54:57.494 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:57.496+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:57.594 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:57.597+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:57.645 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:57.645 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:54:57.645 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:54:57.645 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:57.654 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:54:57.695 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:57.697+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:57.795 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:57.798+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:57.803 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:57.803 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:54:57.803 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:54:57.804 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:54:57.804 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:57.812 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:54:57.897 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:57.899+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:57.923 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:54:57.924 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:54:57.924 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:54:57.924 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:54:57.924 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.924 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:437 3.17 3:ea93a512:::rbd_data.1140100c9703.0000000000000077:head [set-alloc-hint object_size 2097152 write_size 2097152,write 1170432~1024] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e63)", 2024-05-08T08:54:57.924 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:08.390804+0000", 2024-05-08T08:54:57.924 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 109.53474979400001, 2024-05-08T08:54:57.924 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0406068690000001, 2024-05-08T08:54:57.924 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:57.924 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:57.924 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:54:57.924 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:57.924 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:57.924 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:57.925 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 437 2024-05-08T08:54:57.925 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.925 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:57.925 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.925 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:57.925 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390804+0000", 2024-05-08T08:54:57.925 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:57.925 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.925 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.925 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:57.925 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390804+0000", 2024-05-08T08:54:57.925 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:57.925 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.925 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.925 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:57.925 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390806+0000", 2024-05-08T08:54:57.926 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.666e-06 2024-05-08T08:54:57.926 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.926 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.926 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:57.926 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390818+0000", 2024-05-08T08:54:57.926 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2187999999999999e-05 2024-05-08T08:54:57.926 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.926 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.926 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:57.926 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390834+0000", 2024-05-08T08:54:57.926 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6191000000000001e-05 2024-05-08T08:54:57.926 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.926 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.926 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:57.927 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390845+0000", 2024-05-08T08:54:57.927 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0831e-05 2024-05-08T08:54:57.927 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.927 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.927 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:57.927 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390922+0000", 2024-05-08T08:54:57.927 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7253e-05 2024-05-08T08:54:57.927 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.927 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.927 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op must wait for map", 2024-05-08T08:54:57.927 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390930+0000", 2024-05-08T08:54:57.927 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8259999999999993e-06 2024-05-08T08:54:57.927 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.927 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.927 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:57.928 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392155+0000", 2024-05-08T08:54:57.928 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0012256649999999999 2024-05-08T08:54:57.928 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.928 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.928 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:54:57.928 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392165+0000", 2024-05-08T08:54:57.928 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5799999999999998e-06 2024-05-08T08:54:57.928 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.928 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.928 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:57.928 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.429596+0000", 2024-05-08T08:54:57.928 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0374313639999999 2024-05-08T08:54:57.928 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.928 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.928 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:57.929 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.429728+0000", 2024-05-08T08:54:57.929 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000131552 2024-05-08T08:54:57.929 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.929 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.929 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 0,1", 2024-05-08T08:54:57.929 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.429860+0000", 2024-05-08T08:54:57.929 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000132137 2024-05-08T08:54:57.929 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.929 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.929 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:54:57.929 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.430811+0000", 2024-05-08T08:54:57.929 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00095040200000000004 2024-05-08T08:54:57.929 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.929 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.929 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:54:57.930 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431126+0000", 2024-05-08T08:54:57.930 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000315006 2024-05-08T08:54:57.930 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.930 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.930 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:54:57.930 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431272+0000", 2024-05-08T08:54:57.930 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000146487 2024-05-08T08:54:57.930 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.930 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.930 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:54:57.930 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431315+0000", 2024-05-08T08:54:57.930 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2697000000000001e-05 2024-05-08T08:54:57.930 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.930 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.930 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:57.931 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431411+0000", 2024-05-08T08:54:57.931 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6024000000000001e-05 2024-05-08T08:54:57.931 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:57.931 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:57.931 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:57.931 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.931 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.931 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:439 3.17 3:ea93a512:::rbd_data.1140100c9703.0000000000000077:head [read 1163264~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-05-08T08:54:57.931 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:08.390871+0000", 2024-05-08T08:54:57.931 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 109.534682283, 2024-05-08T08:54:57.931 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0408790590000001, 2024-05-08T08:54:57.931 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:57.931 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:57.931 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:57.931 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:57.932 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:57.932 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:57.932 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 439 2024-05-08T08:54:57.932 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.932 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:57.932 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.932 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:57.932 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390871+0000", 2024-05-08T08:54:57.932 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:57.932 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.932 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.932 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:57.932 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390871+0000", 2024-05-08T08:54:57.932 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:57.932 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.933 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.933 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:57.933 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390872+0000", 2024-05-08T08:54:57.933 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6300000000000005e-07 2024-05-08T08:54:57.933 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.933 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.933 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:57.933 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390875+0000", 2024-05-08T08:54:57.933 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3459999999999998e-06 2024-05-08T08:54:57.933 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.933 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.933 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:57.933 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390891+0000", 2024-05-08T08:54:57.933 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5255e-05 2024-05-08T08:54:57.933 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.934 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.934 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:57.934 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390897+0000", 2024-05-08T08:54:57.934 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2360000000000001e-06 2024-05-08T08:54:57.934 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.934 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.934 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:57.934 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390995+0000", 2024-05-08T08:54:57.934 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8522000000000003e-05 2024-05-08T08:54:57.934 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.934 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.934 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting_for_map not empty", 2024-05-08T08:54:57.934 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.391005+0000", 2024-05-08T08:54:57.934 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1460000000000002e-06 2024-05-08T08:54:57.934 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.935 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.935 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:57.935 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392191+0000", 2024-05-08T08:54:57.935 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0011860270000000001 2024-05-08T08:54:57.935 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.935 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.935 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:54:57.935 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392197+0000", 2024-05-08T08:54:57.935 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4289999999999998e-06 2024-05-08T08:54:57.935 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.935 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.935 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:57.935 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.430596+0000", 2024-05-08T08:54:57.935 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0383989090000001 2024-05-08T08:54:57.935 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.935 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.935 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for rw locks", 2024-05-08T08:54:57.936 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.430676+0000", 2024-05-08T08:54:57.936 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9781000000000001e-05 2024-05-08T08:54:57.936 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.936 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.936 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:57.936 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431460+0000", 2024-05-08T08:54:57.936 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00078427300000000004 2024-05-08T08:54:57.936 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.936 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.936 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:57.936 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431540+0000", 2024-05-08T08:54:57.936 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0224999999999998e-05 2024-05-08T08:54:57.936 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.936 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.936 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:57.936 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431750+0000", 2024-05-08T08:54:57.937 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00021024699999999999 2024-05-08T08:54:57.937 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:57.937 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:57.937 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:57.937 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.937 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.937 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:945 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 4096000~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:57.937 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:54:57.937 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 108.474532716, 2024-05-08T08:54:57.937 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086774741000000002, 2024-05-08T08:54:57.937 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:57.937 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:57.937 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:57.937 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:57.937 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:57.938 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:57.938 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 945 2024-05-08T08:54:57.938 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.938 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:57.938 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.938 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:57.938 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:54:57.938 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:57.938 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.938 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.938 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:57.938 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:54:57.938 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:57.938 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.939 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.939 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:57.939 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:54:57.939 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3499999999999996e-07 2024-05-08T08:54:57.939 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.939 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.939 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:57.939 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451024+0000", 2024-05-08T08:54:57.939 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.627e-06 2024-05-08T08:54:57.939 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.939 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.939 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:57.939 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451037+0000", 2024-05-08T08:54:57.939 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2431e-05 2024-05-08T08:54:57.939 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.939 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.940 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:57.940 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451042+0000", 2024-05-08T08:54:57.940 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8409999999999996e-06 2024-05-08T08:54:57.940 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.940 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.940 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:57.940 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537608+0000", 2024-05-08T08:54:57.940 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086565243 2024-05-08T08:54:57.940 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.940 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.940 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:57.940 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537687+0000", 2024-05-08T08:54:57.940 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9519999999999998e-05 2024-05-08T08:54:57.940 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.940 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.940 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:57.941 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537796+0000", 2024-05-08T08:54:57.941 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000108544 2024-05-08T08:54:57.941 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:57.941 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:57.941 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:57.941 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.941 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.941 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:964 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 458752~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:57.941 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451051+0000", 2024-05-08T08:54:57.941 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 108.474502543, 2024-05-08T08:54:57.941 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086990490000000004, 2024-05-08T08:54:57.941 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:57.941 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:57.941 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:57.941 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:57.942 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:57.942 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:57.942 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 964 2024-05-08T08:54:57.942 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.942 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:57.942 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.942 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:57.942 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451051+0000", 2024-05-08T08:54:57.942 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:57.942 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.942 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.942 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:57.942 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451051+0000", 2024-05-08T08:54:57.942 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:57.942 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.942 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.943 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:57.943 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451052+0000", 2024-05-08T08:54:57.943 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7299999999999996e-07 2024-05-08T08:54:57.943 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.943 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.943 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:57.943 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451055+0000", 2024-05-08T08:54:57.943 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.2770000000000001e-06 2024-05-08T08:54:57.943 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.943 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.943 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:57.943 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451061+0000", 2024-05-08T08:54:57.943 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4060000000000002e-06 2024-05-08T08:54:57.943 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.943 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.943 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:57.944 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451066+0000", 2024-05-08T08:54:57.944 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0839999999999999e-06 2024-05-08T08:54:57.944 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.944 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.944 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:57.944 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537839+0000", 2024-05-08T08:54:57.944 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086772962999999995 2024-05-08T08:54:57.944 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.944 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.944 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:57.944 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537934+0000", 2024-05-08T08:54:57.944 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4742999999999997e-05 2024-05-08T08:54:57.944 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.944 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.944 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:57.945 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538042+0000", 2024-05-08T08:54:57.945 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000107444 2024-05-08T08:54:57.945 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:57.945 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:57.945 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:57.945 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.945 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.945 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:965 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 950272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:57.945 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451075+0000", 2024-05-08T08:54:57.945 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 108.47447848100001, 2024-05-08T08:54:57.945 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087230196999999995, 2024-05-08T08:54:57.945 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:57.945 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:57.945 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:57.945 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:57.945 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:57.946 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:57.946 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 965 2024-05-08T08:54:57.946 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.946 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:57.946 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.946 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:57.946 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451075+0000", 2024-05-08T08:54:57.946 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:57.946 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.946 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.946 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:57.946 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451075+0000", 2024-05-08T08:54:57.946 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:57.946 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.946 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.946 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:57.947 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451076+0000", 2024-05-08T08:54:57.947 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5000000000000002e-07 2024-05-08T08:54:57.947 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.947 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.947 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:57.947 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451078+0000", 2024-05-08T08:54:57.947 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.5639999999999999e-06 2024-05-08T08:54:57.947 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.947 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.947 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:57.947 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451085+0000", 2024-05-08T08:54:57.947 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5640000000000002e-06 2024-05-08T08:54:57.947 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.947 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.947 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:57.947 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451090+0000", 2024-05-08T08:54:57.948 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1699999999999996e-06 2024-05-08T08:54:57.948 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.948 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.948 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:57.948 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538106+0000", 2024-05-08T08:54:57.948 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087015888999999999 2024-05-08T08:54:57.948 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.948 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.948 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:57.948 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538190+0000", 2024-05-08T08:54:57.948 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3961999999999993e-05 2024-05-08T08:54:57.948 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.948 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.948 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:57.948 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538305+0000", 2024-05-08T08:54:57.948 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011529800000000001 2024-05-08T08:54:57.949 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:57.949 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:57.949 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:57.949 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.949 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.949 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:966 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 1441792~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:57.949 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:54:57.949 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 108.474455108, 2024-05-08T08:54:57.949 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087446323000000006, 2024-05-08T08:54:57.949 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:57.949 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:57.949 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:57.949 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:57.949 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:57.949 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:57.950 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 966 2024-05-08T08:54:57.950 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.950 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:57.950 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.950 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:57.950 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:54:57.950 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:57.950 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.950 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.950 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:57.950 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:54:57.950 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:57.950 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.950 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.951 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:57.951 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:54:57.951 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3700000000000001e-07 2024-05-08T08:54:57.951 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.951 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.951 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:57.951 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451101+0000", 2024-05-08T08:54:57.951 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2560000000000001e-06 2024-05-08T08:54:57.951 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.951 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.951 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:57.951 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451107+0000", 2024-05-08T08:54:57.951 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1070000000000001e-06 2024-05-08T08:54:57.951 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.951 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.952 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:57.952 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451112+0000", 2024-05-08T08:54:57.952 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1420000000000001e-06 2024-05-08T08:54:57.952 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.952 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.952 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:57.952 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538341+0000", 2024-05-08T08:54:57.952 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087228302999999993 2024-05-08T08:54:57.952 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.952 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.952 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:57.952 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538424+0000", 2024-05-08T08:54:57.952 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3497000000000004e-05 2024-05-08T08:54:57.952 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.952 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.952 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:57.953 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538545+0000", 2024-05-08T08:54:57.953 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000120581 2024-05-08T08:54:57.953 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:57.953 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:57.953 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:57.953 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.953 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.953 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:967 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:57.953 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451120+0000", 2024-05-08T08:54:57.953 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 108.474433633, 2024-05-08T08:54:57.953 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087632725999999994, 2024-05-08T08:54:57.953 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:57.953 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:57.953 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:57.953 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:57.954 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:57.954 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:57.954 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 967 2024-05-08T08:54:57.954 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.954 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:57.954 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.954 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:57.954 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451120+0000", 2024-05-08T08:54:57.954 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:57.954 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.954 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.954 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:57.954 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451120+0000", 2024-05-08T08:54:57.954 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:57.954 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.955 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.955 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:57.955 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451121+0000", 2024-05-08T08:54:57.955 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8800000000000002e-07 2024-05-08T08:54:57.955 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.955 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.955 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:57.955 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451130+0000", 2024-05-08T08:54:57.955 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2299999999999997e-06 2024-05-08T08:54:57.955 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.955 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.955 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:57.955 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451137+0000", 2024-05-08T08:54:57.955 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4669999999999998e-06 2024-05-08T08:54:57.955 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.956 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.956 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:57.956 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451143+0000", 2024-05-08T08:54:57.956 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2839999999999997e-06 2024-05-08T08:54:57.956 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.956 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.956 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:57.956 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538575+0000", 2024-05-08T08:54:57.956 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087432789999999996 2024-05-08T08:54:57.956 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.956 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.956 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:57.956 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538648+0000", 2024-05-08T08:54:57.956 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2877e-05 2024-05-08T08:54:57.956 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.957 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.957 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:57.957 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538753+0000", 2024-05-08T08:54:57.957 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010449 2024-05-08T08:54:57.957 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:57.957 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:57.957 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:57.957 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.957 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.957 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:968 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 2424832~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:57.957 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451152+0000", 2024-05-08T08:54:57.957 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 108.474401548, 2024-05-08T08:54:57.957 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087813325999999997, 2024-05-08T08:54:57.957 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:57.957 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:57.957 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:57.958 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:57.958 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:57.958 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:57.958 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 968 2024-05-08T08:54:57.958 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.958 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:57.958 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.958 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:57.958 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451152+0000", 2024-05-08T08:54:57.958 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:57.958 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.958 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.958 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:57.958 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451152+0000", 2024-05-08T08:54:57.958 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:57.959 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.959 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.959 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:57.959 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451153+0000", 2024-05-08T08:54:57.959 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3099999999999998e-07 2024-05-08T08:54:57.959 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.959 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.959 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:57.959 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451155+0000", 2024-05-08T08:54:57.959 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4760000000000001e-06 2024-05-08T08:54:57.959 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.959 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.959 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:57.959 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451162+0000", 2024-05-08T08:54:57.959 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5969999999999997e-06 2024-05-08T08:54:57.959 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.960 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.960 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:57.960 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451167+0000", 2024-05-08T08:54:57.960 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.977e-06 2024-05-08T08:54:57.960 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.960 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.960 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:57.960 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538781+0000", 2024-05-08T08:54:57.960 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087614846999999996 2024-05-08T08:54:57.960 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.960 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.960 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:57.960 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538857+0000", 2024-05-08T08:54:57.960 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5358000000000006e-05 2024-05-08T08:54:57.960 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.960 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.961 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:57.961 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538965+0000", 2024-05-08T08:54:57.961 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010864 2024-05-08T08:54:57.961 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:57.961 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:57.961 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:57.961 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.961 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.961 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:969 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 2916352~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:57.961 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:54:57.961 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 108.47437984299999, 2024-05-08T08:54:57.961 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087998980000000004, 2024-05-08T08:54:57.961 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:57.961 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:57.962 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:57.962 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:57.962 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:57.962 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:57.962 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 969 2024-05-08T08:54:57.962 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.962 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:57.962 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.962 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:57.962 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:54:57.962 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:57.962 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.962 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.962 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:57.962 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:54:57.963 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:57.963 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.963 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.963 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:57.963 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:54:57.963 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3599999999999999e-07 2024-05-08T08:54:57.963 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.963 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.963 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:57.963 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451177+0000", 2024-05-08T08:54:57.963 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4590000000000001e-06 2024-05-08T08:54:57.963 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.963 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.963 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:57.963 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451184+0000", 2024-05-08T08:54:57.963 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4109999999999999e-06 2024-05-08T08:54:57.964 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.964 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.964 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:57.964 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451191+0000", 2024-05-08T08:54:57.964 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7279999999999998e-06 2024-05-08T08:54:57.964 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.964 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.964 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:57.964 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538997+0000", 2024-05-08T08:54:57.964 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087806170000000003 2024-05-08T08:54:57.964 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.964 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.964 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:57.964 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539069+0000", 2024-05-08T08:54:57.964 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2373999999999995e-05 2024-05-08T08:54:57.964 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.965 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.965 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:57.965 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539173+0000", 2024-05-08T08:54:57.965 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000103402 2024-05-08T08:54:57.965 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:57.965 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:57.965 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:57.965 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.965 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.965 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:970 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 3407872~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:57.965 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451198+0000", 2024-05-08T08:54:57.965 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 108.474355246, 2024-05-08T08:54:57.965 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088183368999999998, 2024-05-08T08:54:57.965 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:57.965 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:57.966 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:57.966 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:57.966 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:57.966 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:57.966 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 970 2024-05-08T08:54:57.966 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.966 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:57.966 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.966 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:57.966 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451198+0000", 2024-05-08T08:54:57.966 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:57.966 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.966 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.966 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:57.967 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451198+0000", 2024-05-08T08:54:57.967 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:57.967 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.967 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.967 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:57.967 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451199+0000", 2024-05-08T08:54:57.967 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7400000000000003e-07 2024-05-08T08:54:57.967 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.967 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.967 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:57.967 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451211+0000", 2024-05-08T08:54:57.967 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2109999999999999e-05 2024-05-08T08:54:57.967 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.967 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.967 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:57.968 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451219+0000", 2024-05-08T08:54:57.968 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4170000000000003e-06 2024-05-08T08:54:57.968 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.968 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.968 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:57.968 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451224+0000", 2024-05-08T08:54:57.968 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3990000000000003e-06 2024-05-08T08:54:57.968 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.968 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.968 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:57.968 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539204+0000", 2024-05-08T08:54:57.968 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087980099000000006 2024-05-08T08:54:57.968 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.968 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.968 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:57.968 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539276+0000", 2024-05-08T08:54:57.969 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1513000000000005e-05 2024-05-08T08:54:57.969 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.969 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.969 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:57.969 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539382+0000", 2024-05-08T08:54:57.969 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010615699999999999 2024-05-08T08:54:57.969 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:57.969 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:57.969 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:57.969 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.969 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.969 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:971 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 3899392~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:57.969 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451233+0000", 2024-05-08T08:54:57.969 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 108.474320539, 2024-05-08T08:54:57.969 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088431525999999996, 2024-05-08T08:54:57.970 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:57.970 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:57.970 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:57.970 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:57.970 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:57.970 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:57.970 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 971 2024-05-08T08:54:57.970 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.970 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:57.970 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.970 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:57.970 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451233+0000", 2024-05-08T08:54:57.970 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:57.970 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.970 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.971 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:57.971 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451233+0000", 2024-05-08T08:54:57.971 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:57.971 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.971 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.971 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:57.971 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451234+0000", 2024-05-08T08:54:57.971 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8500000000000001e-07 2024-05-08T08:54:57.971 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.971 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.971 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:57.971 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451237+0000", 2024-05-08T08:54:57.971 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8320000000000002e-06 2024-05-08T08:54:57.971 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.971 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.971 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:57.972 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451243+0000", 2024-05-08T08:54:57.972 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6749999999999996e-06 2024-05-08T08:54:57.972 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.972 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.972 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:57.972 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451249+0000", 2024-05-08T08:54:57.972 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1229999999999997e-06 2024-05-08T08:54:57.972 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.972 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.972 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:57.972 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539412+0000", 2024-05-08T08:54:57.972 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088162668999999999 2024-05-08T08:54:57.972 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.972 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.972 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:57.972 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539505+0000", 2024-05-08T08:54:57.973 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3240999999999997e-05 2024-05-08T08:54:57.973 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.973 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.973 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:57.973 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539665+0000", 2024-05-08T08:54:57.973 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00015930100000000001 2024-05-08T08:54:57.973 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:57.973 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:57.973 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:57.973 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.973 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.973 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:972 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 196608~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:57.973 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451257+0000", 2024-05-08T08:54:57.973 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 108.47429659399999, 2024-05-08T08:54:57.973 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088719928000000003, 2024-05-08T08:54:57.973 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:57.973 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:57.974 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:57.974 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:57.974 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:57.974 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:57.974 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 972 2024-05-08T08:54:57.974 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.974 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:57.974 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.974 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:57.974 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451257+0000", 2024-05-08T08:54:57.974 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:57.974 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.974 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.974 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:57.974 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451257+0000", 2024-05-08T08:54:57.975 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:57.975 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.975 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.975 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:57.975 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451258+0000", 2024-05-08T08:54:57.975 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7999999999999996e-07 2024-05-08T08:54:57.975 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.975 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.975 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:57.975 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451260+0000", 2024-05-08T08:54:57.975 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4609999999999998e-06 2024-05-08T08:54:57.975 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.975 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.975 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:57.975 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451267+0000", 2024-05-08T08:54:57.976 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.81e-06 2024-05-08T08:54:57.976 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.976 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.976 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:57.976 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451273+0000", 2024-05-08T08:54:57.976 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0859999999999998e-06 2024-05-08T08:54:57.976 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.976 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.976 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:57.976 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539718+0000", 2024-05-08T08:54:57.976 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088445059000000006 2024-05-08T08:54:57.976 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.976 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.976 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:57.976 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539834+0000", 2024-05-08T08:54:57.976 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000116426 2024-05-08T08:54:57.976 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.976 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.977 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:57.977 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539977+0000", 2024-05-08T08:54:57.977 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014260600000000001 2024-05-08T08:54:57.977 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:57.977 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:57.977 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:57.977 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.977 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.977 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:973 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 688128~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:57.977 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:54:57.977 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 108.47427401, 2024-05-08T08:54:57.977 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088918496, 2024-05-08T08:54:57.977 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:57.977 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:57.977 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:57.977 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:57.978 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:57.978 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:57.978 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 973 2024-05-08T08:54:57.978 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.978 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:57.978 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.978 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:57.978 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:54:57.978 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:57.978 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.978 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.978 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:57.978 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:54:57.979 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:57.979 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.979 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.979 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:57.979 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:54:57.979 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9099999999999999e-07 2024-05-08T08:54:57.979 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.979 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.979 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:57.979 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451282+0000", 2024-05-08T08:54:57.979 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1550000000000001e-06 2024-05-08T08:54:57.979 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.979 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.979 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:57.979 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451288+0000", 2024-05-08T08:54:57.980 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8889999999999999e-06 2024-05-08T08:54:57.980 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.980 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.980 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:57.980 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451294+0000", 2024-05-08T08:54:57.980 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0850000000000002e-06 2024-05-08T08:54:57.980 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.980 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.980 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:57.980 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540011+0000", 2024-05-08T08:54:57.980 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088717314000000005 2024-05-08T08:54:57.980 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.980 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.980 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:57.980 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540089+0000", 2024-05-08T08:54:57.981 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7438e-05 2024-05-08T08:54:57.981 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.981 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.981 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:57.981 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540198+0000", 2024-05-08T08:54:57.981 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000109224 2024-05-08T08:54:57.981 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:57.981 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:57.981 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:57.981 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.981 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.981 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:974 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 1179648~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:57.981 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451301+0000", 2024-05-08T08:54:57.981 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 108.474252377, 2024-05-08T08:54:57.981 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089105078000000004, 2024-05-08T08:54:57.981 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:57.982 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:57.982 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:57.982 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:57.982 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:57.982 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:57.982 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 974 2024-05-08T08:54:57.982 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.982 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:57.982 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.982 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:57.982 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451301+0000", 2024-05-08T08:54:57.982 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:57.982 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.982 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.982 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:57.983 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451301+0000", 2024-05-08T08:54:57.983 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:57.983 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.983 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.983 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:57.983 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451302+0000", 2024-05-08T08:54:57.983 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7800000000000002e-07 2024-05-08T08:54:57.983 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.983 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.983 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:57.983 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451304+0000", 2024-05-08T08:54:57.983 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.65e-06 2024-05-08T08:54:57.983 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.983 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.983 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:57.983 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451311+0000", 2024-05-08T08:54:57.985 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.472e-06 2024-05-08T08:54:57.985 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.985 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.985 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:57.985 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451316+0000", 2024-05-08T08:54:57.985 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9189999999999998e-06 2024-05-08T08:54:57.985 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.985 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.986 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:57.986 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540234+0000", 2024-05-08T08:54:57.986 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088917752000000003 2024-05-08T08:54:57.986 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.986 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.986 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:57.986 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540308+0000", 2024-05-08T08:54:57.986 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4758000000000005e-05 2024-05-08T08:54:57.986 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.986 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.986 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:57.986 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540406+0000", 2024-05-08T08:54:57.986 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8048999999999996e-05 2024-05-08T08:54:57.986 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:57.986 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:57.987 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:57.987 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.987 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.987 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:975 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 1671168~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:57.987 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:54:57.987 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 108.474210047, 2024-05-08T08:54:57.987 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089294139999999994, 2024-05-08T08:54:57.987 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:57.987 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:57.987 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:57.987 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:57.987 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:57.987 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:57.987 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 975 2024-05-08T08:54:57.987 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.987 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:57.988 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.988 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:57.988 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:54:57.988 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:57.988 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.988 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.988 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:57.988 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:54:57.988 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:57.988 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.988 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.988 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:57.988 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:54:57.988 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1299999999999999e-07 2024-05-08T08:54:57.988 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.989 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.989 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:57.989 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451348+0000", 2024-05-08T08:54:57.989 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.484e-06 2024-05-08T08:54:57.989 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.989 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.989 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:57.989 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451356+0000", 2024-05-08T08:54:57.989 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8499999999999994e-06 2024-05-08T08:54:57.989 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.989 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.989 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:57.989 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451366+0000", 2024-05-08T08:54:57.989 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.9850000000000001e-06 2024-05-08T08:54:57.989 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.990 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.990 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:57.990 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540448+0000", 2024-05-08T08:54:57.990 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089082254 2024-05-08T08:54:57.990 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.990 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.990 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:57.990 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540528+0000", 2024-05-08T08:54:57.990 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0234999999999993e-05 2024-05-08T08:54:57.990 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.990 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.990 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:57.990 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540638+0000", 2024-05-08T08:54:57.990 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010951900000000001 2024-05-08T08:54:57.990 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:57.991 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:57.991 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:57.991 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.991 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.991 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:976 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 2162688~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:57.991 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451375+0000", 2024-05-08T08:54:57.991 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 108.47417830000001, 2024-05-08T08:54:57.991 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089472745000000006, 2024-05-08T08:54:57.991 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:57.991 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:57.991 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:57.991 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:57.991 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:57.991 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:57.991 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 976 2024-05-08T08:54:57.992 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.992 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:57.992 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.992 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:57.992 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451375+0000", 2024-05-08T08:54:57.992 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:57.992 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.992 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.992 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:57.992 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451375+0000", 2024-05-08T08:54:57.992 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:57.992 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.992 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.992 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:57.992 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451376+0000", 2024-05-08T08:54:57.992 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2399999999999998e-07 2024-05-08T08:54:57.993 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.993 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.993 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:57.993 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451379+0000", 2024-05-08T08:54:57.993 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6359999999999998e-06 2024-05-08T08:54:57.993 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.993 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.993 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:57.993 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451387+0000", 2024-05-08T08:54:57.993 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9929999999999995e-06 2024-05-08T08:54:57.993 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.993 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.993 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:57.993 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451393+0000", 2024-05-08T08:54:57.993 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4699999999999999e-06 2024-05-08T08:54:57.994 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.994 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.994 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:57.994 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540677+0000", 2024-05-08T08:54:57.994 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089283504999999999 2024-05-08T08:54:57.994 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.994 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.994 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:57.994 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540743+0000", 2024-05-08T08:54:57.994 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6303999999999994e-05 2024-05-08T08:54:57.994 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.994 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.994 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:57.994 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540848+0000", 2024-05-08T08:54:57.995 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000105213 2024-05-08T08:54:57.995 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:57.995 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:57.995 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:57.995 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.995 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.995 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:977 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 2654208~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:57.995 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:54:57.995 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 108.474152868, 2024-05-08T08:54:57.995 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089698473000000001, 2024-05-08T08:54:57.995 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:57.995 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:57.995 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:57.995 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:57.995 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:57.996 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:57.996 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 977 2024-05-08T08:54:57.996 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.996 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:57.996 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.996 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:57.996 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:54:57.996 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:57.996 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.996 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.996 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:57.996 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:54:57.996 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:57.996 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.996 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.997 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:57.997 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:54:57.997 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2899999999999999e-07 2024-05-08T08:54:57.997 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.997 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.997 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:57.997 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451404+0000", 2024-05-08T08:54:57.997 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6649999999999999e-06 2024-05-08T08:54:57.997 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.997 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.997 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:57.997 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451410+0000", 2024-05-08T08:54:57.997 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3269999999999997e-06 2024-05-08T08:54:57.997 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.997 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.997 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:57.998 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451415+0000", 2024-05-08T08:54:57.998 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9529999999999998e-06 2024-05-08T08:54:57.998 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.998 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.998 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:57.998 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540912+0000", 2024-05-08T08:54:57.998 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089496950000000006 2024-05-08T08:54:57.998 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.998 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.998 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:57.998 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541000+0000", 2024-05-08T08:54:57.998 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7429000000000006e-05 2024-05-08T08:54:57.998 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.998 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.999 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:57.999 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541099+0000", 2024-05-08T08:54:57.999 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.9720000000000001e-05 2024-05-08T08:54:57.999 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:57.999 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:57.999 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:57.999 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:57.999 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:57.999 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:978 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 3145728~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:57.999 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:54:57.999 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 108.47413166, 2024-05-08T08:54:57.999 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089900635000000007, 2024-05-08T08:54:57.999 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:57.999 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:57.999 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:58.000 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:58.000 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:58.000 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:58.000 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 978 2024-05-08T08:54:58.000 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:58.000 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:58.000 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:58.000 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:58.000 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:54:58.000 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:58.000 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:58.000 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:58.000 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:58.000 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:54:58.000 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:58.000 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:58.001 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:58.001 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:58.001 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:54:58.001 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9400000000000001e-07 2024-05-08T08:54:58.001 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:58.001 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:58.001 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:58.001 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451425+0000", 2024-05-08T08:54:58.001 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2460000000000002e-06 2024-05-08T08:54:58.001 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:58.001 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:58.001 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:58.001 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451431+0000", 2024-05-08T08:54:58.001 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2419999999999997e-06 2024-05-08T08:54:58.001 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:58.002 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:58.002 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:58.002 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451436+0000", 2024-05-08T08:54:58.002 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.887e-06 2024-05-08T08:54:58.002 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:58.002 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:58.002 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:58.002 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541152+0000", 2024-05-08T08:54:58.002 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089716194999999999 2024-05-08T08:54:58.002 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:58.002 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:58.002 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:58.002 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541224+0000", 2024-05-08T08:54:58.002 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2416000000000006e-05 2024-05-08T08:54:58.002 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:58.003 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:58.003 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:58.003 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541323+0000", 2024-05-08T08:54:58.003 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8255e-05 2024-05-08T08:54:58.003 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:58.003 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:58.003 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:58.003 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:58.003 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:58.003 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:979 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 3637248~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:58.003 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451462+0000", 2024-05-08T08:54:58.003 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 108.474091211, 2024-05-08T08:54:58.003 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.090036441999999994, 2024-05-08T08:54:58.003 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:58.003 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:58.004 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:58.004 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:58.004 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:58.004 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:58.004 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 979 2024-05-08T08:54:58.004 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:58.004 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:58.004 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:58.004 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:58.004 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451462+0000", 2024-05-08T08:54:58.004 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:58.004 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:58.004 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:58.004 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:58.004 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451462+0000", 2024-05-08T08:54:58.005 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:58.005 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:58.005 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:58.005 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:58.005 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451463+0000", 2024-05-08T08:54:58.005 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4300000000000002e-07 2024-05-08T08:54:58.005 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:58.005 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:58.005 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:58.005 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451471+0000", 2024-05-08T08:54:58.005 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0800000000000006e-06 2024-05-08T08:54:58.005 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:58.005 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:58.005 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:58.005 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451481+0000", 2024-05-08T08:54:58.005 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.38e-06 2024-05-08T08:54:58.006 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:58.006 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:58.006 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:58.006 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451487+0000", 2024-05-08T08:54:58.006 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4590000000000003e-06 2024-05-08T08:54:58.006 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:58.006 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:58.006 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:58.006 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541350+0000", 2024-05-08T08:54:58.006 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089862599000000001 2024-05-08T08:54:58.006 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:58.006 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:58.006 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:58.006 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541407+0000", 2024-05-08T08:54:58.007 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7667999999999997e-05 2024-05-08T08:54:58.007 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:58.007 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:58.007 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:58.007 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541499+0000", 2024-05-08T08:54:58.007 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1513000000000003e-05 2024-05-08T08:54:58.007 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:58.007 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:58.007 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:58.007 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:58.007 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:58.008 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:980 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 4128768~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:54:58.008 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451496+0000", 2024-05-08T08:54:58.008 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 108.47405745499999, 2024-05-08T08:54:58.008 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.090258985, 2024-05-08T08:54:58.008 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:54:58.008 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:54:58.008 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:54:58.008 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:54:58.008 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:54:58.008 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:54:58.008 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 980 2024-05-08T08:54:58.008 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:58.008 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:54:58.008 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:58.008 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:54:58.008 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451496+0000", 2024-05-08T08:54:58.008 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:58.009 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:58.009 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:58.009 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:54:58.009 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451496+0000", 2024-05-08T08:54:58.009 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:54:58.009 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:58.009 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:58.009 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:54:58.009 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451497+0000", 2024-05-08T08:54:58.009 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7599999999999997e-07 2024-05-08T08:54:58.009 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:58.009 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:58.009 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:54:58.009 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451500+0000", 2024-05-08T08:54:58.009 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7930000000000002e-06 2024-05-08T08:54:58.010 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:58.010 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:58.010 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:54:58.010 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451506+0000", 2024-05-08T08:54:58.010 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6980000000000001e-06 2024-05-08T08:54:58.010 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:58.010 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:58.010 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:54:58.010 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451512+0000", 2024-05-08T08:54:58.010 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2490000000000001e-06 2024-05-08T08:54:58.010 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:58.010 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:58.010 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:54:58.010 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541550+0000", 2024-05-08T08:54:58.010 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.090037961999999999 2024-05-08T08:54:58.010 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:58.011 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:58.011 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:54:58.011 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541647+0000", 2024-05-08T08:54:58.011 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6924999999999998e-05 2024-05-08T08:54:58.011 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:54:58.011 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:54:58.011 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:54:58.011 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541755+0000", 2024-05-08T08:54:58.011 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000108682 2024-05-08T08:54:58.011 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:58.011 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:58.011 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:58.011 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:54:58.011 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:54:58.011 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:54:58.013 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:54:58.013 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:58.015+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:58.113 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:58.116+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:58.128 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:58.128 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:54:58.128 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:54:58.128 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:58.137 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:54:58.214 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:58.216+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:58.242 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:58.242 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:54:58.242 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:54:58.243 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:54:58.243 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:58.252 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:54:58.315 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:58.318+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:58.367 INFO:teuthology.orchestra.run.smithi059.stderr:set 2.5 pg_upmap mapping to [3,2] 2024-05-08T08:54:58.402 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:58.402 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:54:58.402 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:54:58.402 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:54:58.402 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.402 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4621.0:9089 3.1f e33/18 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:head v 33'117, mlcod=31'115)", 2024-05-08T08:54:58.402 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:54:58.402 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 145.884357986, 2024-05-08T08:54:58.402 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.111244524, 2024-05-08T08:54:58.402 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:58.403 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:58.403 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:54:58.403 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:58.403 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.403 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:58.403 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:54:58.403 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:58.403 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.403 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.403 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:58.403 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:54:58.403 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:58.403 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.403 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.403 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:58.403 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:54:58.403 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5900000000000001e-07 2024-05-08T08:54:58.404 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.404 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.404 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:58.404 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539600+0000", 2024-05-08T08:54:58.404 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.019981364000000001 2024-05-08T08:54:58.404 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.404 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.404 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:58.404 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539624+0000", 2024-05-08T08:54:58.404 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4043e-05 2024-05-08T08:54:58.404 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.404 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.404 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:58.404 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539635+0000", 2024-05-08T08:54:58.404 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1511e-05 2024-05-08T08:54:58.404 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.405 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.405 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:58.405 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539728+0000", 2024-05-08T08:54:58.405 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2498999999999996e-05 2024-05-08T08:54:58.405 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.405 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.405 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:58.405 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539746+0000", 2024-05-08T08:54:58.405 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8289000000000001e-05 2024-05-08T08:54:58.405 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.405 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.405 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:54:58.405 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.630822+0000", 2024-05-08T08:54:58.405 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.091076283999999993 2024-05-08T08:54:58.405 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.406 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.406 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:58.406 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.630863+0000", 2024-05-08T08:54:58.406 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0374999999999999e-05 2024-05-08T08:54:58.406 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.406 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:58.406 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.406 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.406 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.406 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4195.0:4 3.11 3:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-05-08T08:54:58.406 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:49.764654+0000", 2024-05-08T08:54:58.406 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 128.639322445, 2024-05-08T08:54:58.406 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.30714275499999999, 2024-05-08T08:54:58.406 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:58.406 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:58.407 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-05-08T08:54:58.407 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:58.407 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4195", 2024-05-08T08:54:58.407 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/59179521", 2024-05-08T08:54:58.407 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4 2024-05-08T08:54:58.407 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.407 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:58.407 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.407 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:58.407 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764654+0000", 2024-05-08T08:54:58.407 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:58.407 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.407 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.407 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:58.407 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764654+0000", 2024-05-08T08:54:58.407 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:58.408 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.408 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.408 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:58.408 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764655+0000", 2024-05-08T08:54:58.408 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.452e-06 2024-05-08T08:54:58.408 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.408 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.408 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:58.408 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764661+0000", 2024-05-08T08:54:58.408 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5369999999999997e-06 2024-05-08T08:54:58.408 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.408 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.408 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:58.408 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764671+0000", 2024-05-08T08:54:58.408 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.028e-05 2024-05-08T08:54:58.409 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.409 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.409 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:58.409 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764680+0000", 2024-05-08T08:54:58.409 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0629999999999995e-06 2024-05-08T08:54:58.409 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.409 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.409 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:58.409 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764753+0000", 2024-05-08T08:54:58.409 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2679e-05 2024-05-08T08:54:58.409 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.409 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.409 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-05-08T08:54:58.409 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764765+0000", 2024-05-08T08:54:58.409 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2415000000000001e-05 2024-05-08T08:54:58.410 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.410 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.410 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:58.410 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:50.071692+0000", 2024-05-08T08:54:58.410 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.306927324 2024-05-08T08:54:58.410 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.410 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.410 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:58.410 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:50.071796+0000", 2024-05-08T08:54:58.410 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104005 2024-05-08T08:54:58.410 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.410 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:58.410 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.410 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.410 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.411 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4416.0:20531 3.16 3:6bb2ed16:::rbd_object_map.1140bc4a55c7:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-05-08T08:54:58.411 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:58.673786+0000", 2024-05-08T08:54:58.411 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 59.730190041, 2024-05-08T08:54:58.411 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.94684119899999997, 2024-05-08T08:54:58.411 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:58.411 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:58.411 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:54:58.411 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:58.411 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4416", 2024-05-08T08:54:58.411 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:54:58.411 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20531 2024-05-08T08:54:58.411 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.411 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:58.411 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.411 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:58.412 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673786+0000", 2024-05-08T08:54:58.412 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:58.412 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.412 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.412 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:58.412 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673786+0000", 2024-05-08T08:54:58.412 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:58.412 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.412 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.412 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:58.412 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673787+0000", 2024-05-08T08:54:58.412 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9800000000000003e-07 2024-05-08T08:54:58.412 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.412 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.412 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:58.412 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673791+0000", 2024-05-08T08:54:58.412 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.104e-06 2024-05-08T08:54:58.412 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.413 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.413 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:58.413 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673801+0000", 2024-05-08T08:54:58.413 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9960000000000005e-06 2024-05-08T08:54:58.413 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.413 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.413 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:58.413 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673806+0000", 2024-05-08T08:54:58.413 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2320000000000001e-06 2024-05-08T08:54:58.413 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.413 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.413 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:58.413 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673860+0000", 2024-05-08T08:54:58.413 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3468999999999999e-05 2024-05-08T08:54:58.413 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.413 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.413 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-05-08T08:54:58.414 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673866+0000", 2024-05-08T08:54:58.414 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3500000000000002e-06 2024-05-08T08:54:58.414 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.414 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.414 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:58.414 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.616700+0000", 2024-05-08T08:54:58.414 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.94283390700000003 2024-05-08T08:54:58.414 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.414 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.414 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:58.414 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.616941+0000", 2024-05-08T08:54:58.414 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00024080400000000001 2024-05-08T08:54:58.414 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.414 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.415 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-05-08T08:54:58.415 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.617072+0000", 2024-05-08T08:54:58.415 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000131688 2024-05-08T08:54:58.415 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.415 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.415 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:54:58.415 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.619571+0000", 2024-05-08T08:54:58.415 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0024990149999999998 2024-05-08T08:54:58.415 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.415 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.415 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:54:58.415 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.620568+0000", 2024-05-08T08:54:58.415 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00099711000000000005 2024-05-08T08:54:58.415 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.415 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.415 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:54:58.416 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.620602+0000", 2024-05-08T08:54:58.416 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.324e-05 2024-05-08T08:54:58.416 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.416 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.416 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:58.416 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.620627+0000", 2024-05-08T08:54:58.416 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5585999999999999e-05 2024-05-08T08:54:58.416 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.416 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:58.416 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.416 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.416 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.416 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5490 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4227072~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:58.416 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.881809+0000", 2024-05-08T08:54:58.416 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 32.522167529000001, 2024-05-08T08:54:58.416 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047460447000000003, 2024-05-08T08:54:58.416 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:58.417 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:58.417 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:58.417 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:58.417 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:58.417 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:58.417 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5490 2024-05-08T08:54:58.417 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.417 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:58.417 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.417 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:58.417 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881809+0000", 2024-05-08T08:54:58.417 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:58.417 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.417 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.417 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:58.417 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881809+0000", 2024-05-08T08:54:58.418 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:58.418 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.418 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.418 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:58.418 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881809+0000", 2024-05-08T08:54:58.418 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7899999999999999e-07 2024-05-08T08:54:58.418 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.418 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.418 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:58.418 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881811+0000", 2024-05-08T08:54:58.418 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0339999999999999e-06 2024-05-08T08:54:58.418 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.418 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.418 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:58.419 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881818+0000", 2024-05-08T08:54:58.419 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5760000000000002e-06 2024-05-08T08:54:58.419 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.419 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.419 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:58.419 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881823+0000", 2024-05-08T08:54:58.419 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4160000000000003e-06 2024-05-08T08:54:58.419 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.419 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.419 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:58.419 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929157+0000", 2024-05-08T08:54:58.419 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047334029999999999 2024-05-08T08:54:58.419 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.419 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.419 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:58.419 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929211+0000", 2024-05-08T08:54:58.420 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3560999999999998e-05 2024-05-08T08:54:58.420 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.420 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.420 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:58.420 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929269+0000", 2024-05-08T08:54:58.420 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8350999999999997e-05 2024-05-08T08:54:58.420 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.420 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:58.420 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.420 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.420 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.420 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5492 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4272128~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:58.420 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.881830+0000", 2024-05-08T08:54:58.420 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 32.522146235000001, 2024-05-08T08:54:58.420 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047614006, 2024-05-08T08:54:58.421 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:58.421 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:58.421 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:58.421 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:58.421 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:58.421 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:58.421 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5492 2024-05-08T08:54:58.421 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.421 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:58.421 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.421 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:58.421 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881830+0000", 2024-05-08T08:54:58.421 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:58.421 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.421 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.421 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:58.422 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881830+0000", 2024-05-08T08:54:58.422 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:58.422 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.422 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.422 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:58.422 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881830+0000", 2024-05-08T08:54:58.422 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4099999999999999e-07 2024-05-08T08:54:58.422 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.422 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.422 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:58.422 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881832+0000", 2024-05-08T08:54:58.422 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0370000000000001e-06 2024-05-08T08:54:58.422 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.422 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.422 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:58.422 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881839+0000", 2024-05-08T08:54:58.423 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3289999999999999e-06 2024-05-08T08:54:58.423 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.423 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.423 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:58.423 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881844+0000", 2024-05-08T08:54:58.423 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1170000000000003e-06 2024-05-08T08:54:58.423 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.423 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.423 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:58.423 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929296+0000", 2024-05-08T08:54:58.423 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047452418000000003 2024-05-08T08:54:58.423 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.423 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.423 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:58.423 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929360+0000", 2024-05-08T08:54:58.423 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4197999999999998e-05 2024-05-08T08:54:58.424 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.424 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.424 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:58.424 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929444+0000", 2024-05-08T08:54:58.424 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3466000000000004e-05 2024-05-08T08:54:58.424 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.424 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:58.424 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.424 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.424 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.424 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5494 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4317184~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:58.424 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.881850+0000", 2024-05-08T08:54:58.424 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 32.522126485999998, 2024-05-08T08:54:58.424 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047723887, 2024-05-08T08:54:58.424 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:58.424 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:58.424 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:58.424 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:58.425 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:58.425 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:58.425 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5494 2024-05-08T08:54:58.425 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.425 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:58.425 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.425 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:58.425 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881850+0000", 2024-05-08T08:54:58.425 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:58.425 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.425 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.425 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:58.425 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881850+0000", 2024-05-08T08:54:58.425 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:58.425 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.425 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.426 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:58.426 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881850+0000", 2024-05-08T08:54:58.426 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8500000000000002e-07 2024-05-08T08:54:58.426 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.426 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.426 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:58.426 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881856+0000", 2024-05-08T08:54:58.426 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9510000000000003e-06 2024-05-08T08:54:58.426 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.426 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.426 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:58.426 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881862+0000", 2024-05-08T08:54:58.426 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3049999999999998e-06 2024-05-08T08:54:58.426 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.427 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.427 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:58.427 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881868+0000", 2024-05-08T08:54:58.427 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8340000000000004e-06 2024-05-08T08:54:58.427 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.427 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.427 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:58.427 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929475+0000", 2024-05-08T08:54:58.427 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047606796 2024-05-08T08:54:58.427 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.427 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.427 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:58.427 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929518+0000", 2024-05-08T08:54:58.427 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3387000000000003e-05 2024-05-08T08:54:58.427 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.428 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.428 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:58.428 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929574+0000", 2024-05-08T08:54:58.428 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5129e-05 2024-05-08T08:54:58.428 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.428 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:58.428 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.428 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.428 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.428 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5495 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4362240~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:58.428 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.881875+0000", 2024-05-08T08:54:58.428 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 32.522101618999997, 2024-05-08T08:54:58.428 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047847967999999998, 2024-05-08T08:54:58.428 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:58.428 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:58.429 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:58.429 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:58.429 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:58.429 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:58.429 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5495 2024-05-08T08:54:58.429 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.429 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:58.429 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.429 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:58.429 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881875+0000", 2024-05-08T08:54:58.429 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:58.429 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.429 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.429 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:58.429 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881875+0000", 2024-05-08T08:54:58.430 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:58.430 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.430 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.430 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:58.430 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881875+0000", 2024-05-08T08:54:58.430 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-07 2024-05-08T08:54:58.430 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.430 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.430 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:58.430 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881877+0000", 2024-05-08T08:54:58.430 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9659999999999999e-06 2024-05-08T08:54:58.430 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.430 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.430 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:58.430 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881883+0000", 2024-05-08T08:54:58.431 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7649999999999999e-06 2024-05-08T08:54:58.431 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.431 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.431 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:58.431 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881887+0000", 2024-05-08T08:54:58.431 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6689999999999997e-06 2024-05-08T08:54:58.431 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.431 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.431 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:58.431 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929600+0000", 2024-05-08T08:54:58.431 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047712746 2024-05-08T08:54:58.431 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.431 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.431 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:58.431 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929659+0000", 2024-05-08T08:54:58.431 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8936e-05 2024-05-08T08:54:58.432 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.432 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.432 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:58.432 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929723+0000", 2024-05-08T08:54:58.432 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3524999999999999e-05 2024-05-08T08:54:58.432 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.432 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:58.432 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.432 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.432 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.432 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5496 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4407296~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:58.432 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.881894+0000", 2024-05-08T08:54:58.432 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 32.522081755000002, 2024-05-08T08:54:58.432 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047998015999999998, 2024-05-08T08:54:58.432 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:58.432 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:58.433 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:58.433 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:58.433 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:58.433 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:58.433 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5496 2024-05-08T08:54:58.433 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.433 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:58.433 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.433 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:58.433 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881894+0000", 2024-05-08T08:54:58.433 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:58.433 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.433 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.433 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:58.433 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881894+0000", 2024-05-08T08:54:58.433 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:58.433 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.434 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.434 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:58.434 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881895+0000", 2024-05-08T08:54:58.434 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6699999999999999e-07 2024-05-08T08:54:58.434 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.434 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.434 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:58.434 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881897+0000", 2024-05-08T08:54:58.434 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0229999999999999e-06 2024-05-08T08:54:58.434 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.434 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.434 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:58.434 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881903+0000", 2024-05-08T08:54:58.434 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9850000000000003e-06 2024-05-08T08:54:58.434 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.434 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.434 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:58.434 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881908+0000", 2024-05-08T08:54:58.435 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9019999999999998e-06 2024-05-08T08:54:58.435 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.435 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.435 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:58.435 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929751+0000", 2024-05-08T08:54:58.435 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047843550999999998 2024-05-08T08:54:58.435 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.435 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.435 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:58.435 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929823+0000", 2024-05-08T08:54:58.435 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1172000000000002e-05 2024-05-08T08:54:58.435 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.435 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.435 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:58.435 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929892+0000", 2024-05-08T08:54:58.436 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9915999999999999e-05 2024-05-08T08:54:58.436 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.436 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:58.436 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.436 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.436 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.436 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5498 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4452352~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:58.436 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.881914+0000", 2024-05-08T08:54:58.436 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 32.522061997000002, 2024-05-08T08:54:58.436 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048203553000000003, 2024-05-08T08:54:58.436 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:58.436 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:58.436 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:58.436 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:58.436 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:58.436 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:58.436 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5498 2024-05-08T08:54:58.437 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.437 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:58.437 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.437 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:58.437 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881914+0000", 2024-05-08T08:54:58.437 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:58.437 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.437 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.437 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:58.437 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881914+0000", 2024-05-08T08:54:58.437 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:58.437 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.437 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.437 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:58.437 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881915+0000", 2024-05-08T08:54:58.437 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8500000000000002e-07 2024-05-08T08:54:58.437 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.437 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.438 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:58.438 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881917+0000", 2024-05-08T08:54:58.438 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.181e-06 2024-05-08T08:54:58.438 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.438 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.438 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:58.438 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881924+0000", 2024-05-08T08:54:58.438 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9619999999999996e-06 2024-05-08T08:54:58.438 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.438 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.438 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:58.438 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881929+0000", 2024-05-08T08:54:58.438 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1399999999999999e-06 2024-05-08T08:54:58.438 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.438 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.438 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:58.439 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929952+0000", 2024-05-08T08:54:58.439 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048023134000000002 2024-05-08T08:54:58.439 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.439 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.439 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:58.439 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.930029+0000", 2024-05-08T08:54:58.439 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7490000000000005e-05 2024-05-08T08:54:58.439 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.439 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.439 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:58.439 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.930118+0000", 2024-05-08T08:54:58.439 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8261000000000001e-05 2024-05-08T08:54:58.439 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.439 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:58.439 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.439 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.440 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.440 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5563 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4321280~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:58.440 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.884201+0000", 2024-05-08T08:54:58.440 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 32.519774747, 2024-05-08T08:54:58.440 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047379969000000001, 2024-05-08T08:54:58.440 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:58.440 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:58.440 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:58.440 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:58.440 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:58.440 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:58.440 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5563 2024-05-08T08:54:58.440 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.440 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:58.440 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.441 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:58.441 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884201+0000", 2024-05-08T08:54:58.441 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:58.441 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.441 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.441 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:58.441 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884201+0000", 2024-05-08T08:54:58.441 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:58.441 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.441 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.441 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:58.441 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884202+0000", 2024-05-08T08:54:58.441 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8000000000000002e-07 2024-05-08T08:54:58.441 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.441 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.442 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:58.442 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884208+0000", 2024-05-08T08:54:58.442 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.674e-06 2024-05-08T08:54:58.442 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.442 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.442 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:58.442 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884215+0000", 2024-05-08T08:54:58.442 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0389999999999999e-06 2024-05-08T08:54:58.442 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.442 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.442 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:58.442 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884220+0000", 2024-05-08T08:54:58.442 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4950000000000002e-06 2024-05-08T08:54:58.442 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.442 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.443 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:58.443 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931417+0000", 2024-05-08T08:54:58.443 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047196674000000001 2024-05-08T08:54:58.443 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.443 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.443 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:58.443 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931493+0000", 2024-05-08T08:54:58.443 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5950000000000003e-05 2024-05-08T08:54:58.443 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.443 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.443 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:58.443 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931581+0000", 2024-05-08T08:54:58.443 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8857000000000007e-05 2024-05-08T08:54:58.443 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.443 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:58.443 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.444 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.444 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.444 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5564 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4366336~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:58.444 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.884225+0000", 2024-05-08T08:54:58.444 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 32.519751206999999, 2024-05-08T08:54:58.444 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047555989999999999, 2024-05-08T08:54:58.444 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:58.444 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:58.444 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:58.444 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:58.444 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:58.444 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:58.444 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5564 2024-05-08T08:54:58.444 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.444 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:58.444 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.444 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:58.445 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884225+0000", 2024-05-08T08:54:58.445 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:58.445 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.445 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.445 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:58.445 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884225+0000", 2024-05-08T08:54:58.445 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:58.445 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.445 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.445 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:58.445 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884225+0000", 2024-05-08T08:54:58.445 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2499999999999995e-07 2024-05-08T08:54:58.445 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.445 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.445 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:58.446 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884227+0000", 2024-05-08T08:54:58.446 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5829999999999999e-06 2024-05-08T08:54:58.446 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.446 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.446 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:58.446 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884232+0000", 2024-05-08T08:54:58.446 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1159999999999998e-06 2024-05-08T08:54:58.446 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.446 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.446 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:58.446 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884236+0000", 2024-05-08T08:54:58.446 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2490000000000003e-06 2024-05-08T08:54:58.446 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.446 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.446 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:58.446 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931617+0000", 2024-05-08T08:54:58.447 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047380310000000002 2024-05-08T08:54:58.447 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.447 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.447 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:58.447 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931689+0000", 2024-05-08T08:54:58.447 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2358000000000001e-05 2024-05-08T08:54:58.447 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.447 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.447 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:58.447 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931781+0000", 2024-05-08T08:54:58.447 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1848999999999994e-05 2024-05-08T08:54:58.447 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.447 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:58.447 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.447 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.447 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.448 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5565 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4411392~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:58.448 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.884251+0000", 2024-05-08T08:54:58.448 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 32.519725053999998, 2024-05-08T08:54:58.448 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047743671000000001, 2024-05-08T08:54:58.448 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:58.448 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:58.448 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:58.448 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:58.448 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:58.448 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:58.448 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5565 2024-05-08T08:54:58.448 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.448 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:58.448 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.448 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:58.448 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884251+0000", 2024-05-08T08:54:58.449 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:58.449 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.449 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.449 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:58.449 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884251+0000", 2024-05-08T08:54:58.449 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:58.449 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.449 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.449 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:58.449 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884252+0000", 2024-05-08T08:54:58.449 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3200000000000005e-07 2024-05-08T08:54:58.449 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.449 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.449 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:58.449 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884253+0000", 2024-05-08T08:54:58.450 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6980000000000001e-06 2024-05-08T08:54:58.450 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.450 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.450 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:58.450 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884259+0000", 2024-05-08T08:54:58.450 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3759999999999999e-06 2024-05-08T08:54:58.450 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.450 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.450 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:58.450 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884263+0000", 2024-05-08T08:54:58.450 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7280000000000003e-06 2024-05-08T08:54:58.450 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.450 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.450 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:58.451 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931818+0000", 2024-05-08T08:54:58.451 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047554368999999999 2024-05-08T08:54:58.451 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.451 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.451 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:58.451 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931897+0000", 2024-05-08T08:54:58.451 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8821999999999999e-05 2024-05-08T08:54:58.451 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.451 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.451 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:58.451 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931995+0000", 2024-05-08T08:54:58.451 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8146000000000005e-05 2024-05-08T08:54:58.451 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.451 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:58.451 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.451 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.452 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.452 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5600 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 3948544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:58.452 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.884619+0000", 2024-05-08T08:54:58.452 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 32.519356799000001, 2024-05-08T08:54:58.452 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047574067999999997, 2024-05-08T08:54:58.452 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:58.452 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:58.452 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:58.452 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:58.452 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:58.452 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:58.452 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5600 2024-05-08T08:54:58.452 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.452 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:58.452 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.453 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:58.453 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884619+0000", 2024-05-08T08:54:58.453 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:58.453 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.453 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.453 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:58.453 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884619+0000", 2024-05-08T08:54:58.453 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:58.453 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.453 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.453 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:58.453 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884620+0000", 2024-05-08T08:54:58.453 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.53e-07 2024-05-08T08:54:58.453 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.453 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.453 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:58.453 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884622+0000", 2024-05-08T08:54:58.454 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.796e-06 2024-05-08T08:54:58.454 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.454 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.454 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:58.454 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884627+0000", 2024-05-08T08:54:58.454 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2850000000000002e-06 2024-05-08T08:54:58.454 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.454 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.454 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:58.454 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884631+0000", 2024-05-08T08:54:58.454 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2899999999999996e-06 2024-05-08T08:54:58.454 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.454 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.454 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:58.454 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932024+0000", 2024-05-08T08:54:58.455 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047392680999999999 2024-05-08T08:54:58.455 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.455 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.455 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:58.455 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932102+0000", 2024-05-08T08:54:58.455 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7806999999999996e-05 2024-05-08T08:54:58.455 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.455 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.455 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:58.455 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932193+0000", 2024-05-08T08:54:58.455 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1855999999999996e-05 2024-05-08T08:54:58.455 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.455 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:58.455 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.455 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.455 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.456 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5603 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 3993600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:58.456 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.884680+0000", 2024-05-08T08:54:58.456 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 32.519296189999999, 2024-05-08T08:54:58.456 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047745034999999998, 2024-05-08T08:54:58.456 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:58.456 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:58.456 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:58.456 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:58.456 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:58.456 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:58.456 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5603 2024-05-08T08:54:58.456 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.456 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:58.456 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.456 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:58.456 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884680+0000", 2024-05-08T08:54:58.457 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:58.457 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.457 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.457 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:58.457 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884680+0000", 2024-05-08T08:54:58.457 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:58.457 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.457 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.457 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:58.457 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884680+0000", 2024-05-08T08:54:58.457 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4400000000000001e-07 2024-05-08T08:54:58.457 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.457 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.457 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:58.457 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884682+0000", 2024-05-08T08:54:58.457 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.514e-06 2024-05-08T08:54:58.458 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.458 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.458 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:58.458 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884687+0000", 2024-05-08T08:54:58.458 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0359999999999997e-06 2024-05-08T08:54:58.458 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.458 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.458 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:58.458 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884692+0000", 2024-05-08T08:54:58.458 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5990000000000004e-06 2024-05-08T08:54:58.458 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.458 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.458 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:58.458 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932235+0000", 2024-05-08T08:54:58.458 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047543635000000001 2024-05-08T08:54:58.459 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.459 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.459 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:58.459 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932323+0000", 2024-05-08T08:54:58.459 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7311000000000005e-05 2024-05-08T08:54:58.459 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.459 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.459 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:58.459 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932425+0000", 2024-05-08T08:54:58.459 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102496 2024-05-08T08:54:58.459 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.459 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:58.459 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.459 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.459 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.459 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5606 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4038656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:58.459 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.884729+0000", 2024-05-08T08:54:58.460 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 32.519246901999999, 2024-05-08T08:54:58.460 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047916488, 2024-05-08T08:54:58.460 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:58.460 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:58.460 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:58.460 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:58.460 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:58.460 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:58.460 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5606 2024-05-08T08:54:58.460 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.460 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:58.460 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.460 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:58.460 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884729+0000", 2024-05-08T08:54:58.460 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:58.460 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.460 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.461 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:58.461 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884729+0000", 2024-05-08T08:54:58.461 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:58.461 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.461 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.461 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:58.461 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.885940+0000", 2024-05-08T08:54:58.461 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001210509 2024-05-08T08:54:58.461 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.461 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.461 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:58.461 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.885945+0000", 2024-05-08T08:54:58.461 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7389999999999999e-06 2024-05-08T08:54:58.461 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.461 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.461 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:58.462 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.885955+0000", 2024-05-08T08:54:58.462 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0813e-05 2024-05-08T08:54:58.462 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.462 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.462 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:58.462 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.885962+0000", 2024-05-08T08:54:58.462 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1160000000000001e-06 2024-05-08T08:54:58.462 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.462 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.462 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:58.462 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932465+0000", 2024-05-08T08:54:58.462 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046502612999999998 2024-05-08T08:54:58.462 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.462 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.463 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:58.463 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932547+0000", 2024-05-08T08:54:58.463 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2025000000000002e-05 2024-05-08T08:54:58.463 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.463 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.463 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:58.463 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932646+0000", 2024-05-08T08:54:58.463 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8672999999999995e-05 2024-05-08T08:54:58.463 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.463 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:58.463 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.463 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.463 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.463 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5635 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4235264~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:58.463 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.886082+0000", 2024-05-08T08:54:58.463 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 32.517893782999998, 2024-05-08T08:54:58.463 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047438284999999997, 2024-05-08T08:54:58.464 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:58.464 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:58.464 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:58.464 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:58.464 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:58.464 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:58.464 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5635 2024-05-08T08:54:58.464 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.464 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:58.464 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.464 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:58.464 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886082+0000", 2024-05-08T08:54:58.464 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:58.464 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.464 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.464 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:58.465 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886082+0000", 2024-05-08T08:54:58.465 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:58.465 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.465 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.465 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:58.465 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886083+0000", 2024-05-08T08:54:58.465 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8999999999999996e-07 2024-05-08T08:54:58.465 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.465 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.465 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:58.465 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886086+0000", 2024-05-08T08:54:58.465 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7e-06 2024-05-08T08:54:58.465 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.465 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.465 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:58.465 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886093+0000", 2024-05-08T08:54:58.465 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5320000000000001e-06 2024-05-08T08:54:58.466 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.466 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.466 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:58.466 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886102+0000", 2024-05-08T08:54:58.466 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8070000000000006e-06 2024-05-08T08:54:58.466 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.466 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.466 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:58.466 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933357+0000", 2024-05-08T08:54:58.466 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047255093999999997 2024-05-08T08:54:58.466 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.466 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.466 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:58.466 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933433+0000", 2024-05-08T08:54:58.467 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5796000000000004e-05 2024-05-08T08:54:58.467 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.467 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.467 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:58.467 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933521+0000", 2024-05-08T08:54:58.467 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7766e-05 2024-05-08T08:54:58.467 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.467 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:58.467 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.467 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.467 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.467 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5636 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4280320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:58.467 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.886111+0000", 2024-05-08T08:54:58.467 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 32.517865165000003, 2024-05-08T08:54:58.467 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047663644999999998, 2024-05-08T08:54:58.467 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:58.467 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:58.468 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:58.468 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:58.468 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:58.468 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:58.468 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5636 2024-05-08T08:54:58.468 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.468 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:58.468 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.468 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:58.468 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886111+0000", 2024-05-08T08:54:58.468 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:58.468 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.468 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.468 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:58.468 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886111+0000", 2024-05-08T08:54:58.468 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:58.468 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.469 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.469 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:58.469 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886111+0000", 2024-05-08T08:54:58.469 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5200000000000002e-07 2024-05-08T08:54:58.469 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.469 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.469 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:58.469 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886114+0000", 2024-05-08T08:54:58.469 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1710000000000001e-06 2024-05-08T08:54:58.469 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.469 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.469 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:58.469 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886120+0000", 2024-05-08T08:54:58.469 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.404e-06 2024-05-08T08:54:58.469 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.469 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.470 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:58.470 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886125+0000", 2024-05-08T08:54:58.470 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2730000000000002e-06 2024-05-08T08:54:58.470 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.470 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.470 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:58.470 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933570+0000", 2024-05-08T08:54:58.470 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047444989999999999 2024-05-08T08:54:58.470 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.470 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.470 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:58.470 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933659+0000", 2024-05-08T08:54:58.470 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8912999999999994e-05 2024-05-08T08:54:58.470 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.470 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.471 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:58.471 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933775+0000", 2024-05-08T08:54:58.471 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000115442 2024-05-08T08:54:58.471 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.471 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:58.471 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.471 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.471 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.471 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5637 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4325376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:58.471 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.886132+0000", 2024-05-08T08:54:58.471 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 32.517844127000004, 2024-05-08T08:54:58.471 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047852219000000001, 2024-05-08T08:54:58.471 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:58.471 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:58.471 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:58.471 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:58.472 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:58.472 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:58.472 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5637 2024-05-08T08:54:58.472 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.472 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:58.472 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.472 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:58.472 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886132+0000", 2024-05-08T08:54:58.472 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:58.472 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.472 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.472 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:58.472 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886132+0000", 2024-05-08T08:54:58.472 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:58.472 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.472 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.473 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:58.473 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886132+0000", 2024-05-08T08:54:58.473 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7300000000000002e-07 2024-05-08T08:54:58.473 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.473 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.473 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:58.473 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886139+0000", 2024-05-08T08:54:58.473 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6109999999999999e-06 2024-05-08T08:54:58.473 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.473 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.473 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:58.473 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886146+0000", 2024-05-08T08:54:58.473 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6279999999999999e-06 2024-05-08T08:54:58.473 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.473 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.473 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:58.474 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886151+0000", 2024-05-08T08:54:58.474 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1769999999999997e-06 2024-05-08T08:54:58.474 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.474 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.474 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:58.474 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933812+0000", 2024-05-08T08:54:58.474 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047660912999999999 2024-05-08T08:54:58.474 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.474 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.474 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:58.474 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933888+0000", 2024-05-08T08:54:58.474 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5865000000000006e-05 2024-05-08T08:54:58.474 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.474 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.474 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:58.475 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933984+0000", 2024-05-08T08:54:58.475 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6651999999999996e-05 2024-05-08T08:54:58.475 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.475 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:58.475 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.475 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.475 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.475 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5638 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4370432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:58.475 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.886157+0000", 2024-05-08T08:54:58.475 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 32.517818863000002, 2024-05-08T08:54:58.475 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048077597, 2024-05-08T08:54:58.475 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:58.475 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:58.475 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:58.475 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:58.475 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:58.476 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:58.476 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5638 2024-05-08T08:54:58.476 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.476 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:58.476 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.476 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:58.476 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886157+0000", 2024-05-08T08:54:58.476 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:58.476 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.476 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.476 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:58.476 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886157+0000", 2024-05-08T08:54:58.476 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:58.476 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.476 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.476 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:58.476 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886158+0000", 2024-05-08T08:54:58.477 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1100000000000001e-07 2024-05-08T08:54:58.477 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.477 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.477 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:58.477 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886160+0000", 2024-05-08T08:54:58.477 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9800000000000001e-06 2024-05-08T08:54:58.477 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.477 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.477 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:58.477 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886166+0000", 2024-05-08T08:54:58.477 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.314e-06 2024-05-08T08:54:58.477 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.477 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.477 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:58.477 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886171+0000", 2024-05-08T08:54:58.477 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7700000000000001e-06 2024-05-08T08:54:58.478 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.478 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.478 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:58.478 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.934025+0000", 2024-05-08T08:54:58.478 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047854509000000003 2024-05-08T08:54:58.478 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.478 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.478 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:58.478 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.934121+0000", 2024-05-08T08:54:58.478 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5688999999999998e-05 2024-05-08T08:54:58.478 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.478 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.478 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:58.478 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.934235+0000", 2024-05-08T08:54:58.478 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000113924 2024-05-08T08:54:58.479 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.479 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:58.479 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.479 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.479 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.479 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5639 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4415488~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:54:58.479 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.886177+0000", 2024-05-08T08:54:58.479 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 32.517799113000002, 2024-05-08T08:54:58.479 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048249541, 2024-05-08T08:54:58.479 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:58.479 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:58.479 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:58.479 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:58.479 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:54:58.479 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:54:58.479 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5639 2024-05-08T08:54:58.480 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.480 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:58.480 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.480 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:58.480 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886177+0000", 2024-05-08T08:54:58.480 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:58.480 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.480 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.480 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:58.480 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886177+0000", 2024-05-08T08:54:58.480 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:58.480 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.480 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.480 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:58.480 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886177+0000", 2024-05-08T08:54:58.480 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.65e-07 2024-05-08T08:54:58.481 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.481 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.481 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:58.481 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886179+0000", 2024-05-08T08:54:58.481 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9530000000000002e-06 2024-05-08T08:54:58.481 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.481 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.481 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:58.481 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886186+0000", 2024-05-08T08:54:58.481 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3419999999999996e-06 2024-05-08T08:54:58.481 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.481 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.481 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:58.481 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886192+0000", 2024-05-08T08:54:58.481 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0560000000000001e-06 2024-05-08T08:54:58.482 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.482 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.482 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:58.482 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.934258+0000", 2024-05-08T08:54:58.482 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048066367999999998 2024-05-08T08:54:58.482 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.482 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.482 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:58.482 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.934338+0000", 2024-05-08T08:54:58.482 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9616000000000005e-05 2024-05-08T08:54:58.482 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.482 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.482 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:58.482 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.934427+0000", 2024-05-08T08:54:58.482 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8840999999999999e-05 2024-05-08T08:54:58.483 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.483 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:58.483 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.483 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.483 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:58.483 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:58.484 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:54:58.484 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:58.486+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:58.585 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:54:58.587+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:54:58.630 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:58.630 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:54:58.630 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:54:58.630 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:58.639 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:54:58.686 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:58.689+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:58.783 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:58.783 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:54:58.783 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:54:58.783 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:54:58.783 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:58.786 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:58.789+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:58.792 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:54:58.887 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:58.890+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:58.898 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:58.898 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:54:58.898 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:54:58.898 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:54:58.898 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.898 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22073 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 782336~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:58.898 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:54:58.898 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 151.04218195799999, 2024-05-08T08:54:58.898 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047244234000000003, 2024-05-08T08:54:58.898 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:58.898 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:58.898 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:58.898 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:58.899 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:58.899 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:58.899 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22073 2024-05-08T08:54:58.899 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.899 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:58.899 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.899 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:58.899 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:54:58.899 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:58.899 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.899 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.899 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:58.899 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:54:58.899 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:58.900 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.900 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.900 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:58.900 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857668+0000", 2024-05-08T08:54:58.900 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7800000000000002e-07 2024-05-08T08:54:58.900 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.900 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.900 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:58.900 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857670+0000", 2024-05-08T08:54:58.900 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0820000000000001e-06 2024-05-08T08:54:58.900 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.900 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.900 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:58.900 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857678+0000", 2024-05-08T08:54:58.900 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7589999999999997e-06 2024-05-08T08:54:58.901 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.901 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.901 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:58.901 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857691+0000", 2024-05-08T08:54:58.901 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3672999999999999e-05 2024-05-08T08:54:58.901 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.901 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.901 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:58.901 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904780+0000", 2024-05-08T08:54:58.901 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047088616999999999 2024-05-08T08:54:58.901 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.901 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.901 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:58.901 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904840+0000", 2024-05-08T08:54:58.901 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0356999999999998e-05 2024-05-08T08:54:58.901 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.901 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.902 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:58.902 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904912+0000", 2024-05-08T08:54:58.902 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1367999999999998e-05 2024-05-08T08:54:58.902 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.902 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:58.902 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.902 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.902 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.902 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22075 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 798720~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:58.902 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:54:58.902 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 151.042151561, 2024-05-08T08:54:58.902 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047384643999999997, 2024-05-08T08:54:58.902 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:58.902 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:58.902 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:58.903 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:58.903 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:58.903 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:58.903 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22075 2024-05-08T08:54:58.903 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.903 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:58.903 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.903 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:58.903 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:54:58.903 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:58.903 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.903 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.903 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:58.903 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:54:58.903 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:58.903 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.903 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.904 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:58.904 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:54:58.904 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7399999999999999e-07 2024-05-08T08:54:58.904 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.904 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.904 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:58.904 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857705+0000", 2024-05-08T08:54:58.904 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1980000000000004e-06 2024-05-08T08:54:58.904 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.904 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.904 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:58.904 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857713+0000", 2024-05-08T08:54:58.904 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7039999999999994e-06 2024-05-08T08:54:58.904 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.904 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.904 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:58.904 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857718+0000", 2024-05-08T08:54:58.905 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.865e-06 2024-05-08T08:54:58.905 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.905 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.905 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:58.905 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904941+0000", 2024-05-08T08:54:58.905 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047223198000000001 2024-05-08T08:54:58.905 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.905 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.905 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:58.905 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904999+0000", 2024-05-08T08:54:58.905 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8254999999999997e-05 2024-05-08T08:54:58.905 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.905 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.905 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:58.905 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905082+0000", 2024-05-08T08:54:58.905 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3049999999999999e-05 2024-05-08T08:54:58.906 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.906 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:58.906 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.906 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.906 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.906 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22077 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 815104~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:58.906 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:54:58.906 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 151.04211355000001, 2024-05-08T08:54:58.906 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047519448999999998, 2024-05-08T08:54:58.906 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:58.906 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:58.906 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:58.906 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:58.906 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:58.907 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:58.907 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22077 2024-05-08T08:54:58.907 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.907 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:58.907 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.907 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:58.907 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:54:58.907 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:58.907 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.907 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.907 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:58.907 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:54:58.907 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:58.907 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.907 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.907 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:58.908 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:54:58.908 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.99e-07 2024-05-08T08:54:58.908 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.908 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.908 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:58.908 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857739+0000", 2024-05-08T08:54:58.908 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3599999999999999e-06 2024-05-08T08:54:58.908 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.908 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.908 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:58.908 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857745+0000", 2024-05-08T08:54:58.908 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5139999999999998e-06 2024-05-08T08:54:58.908 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.908 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.908 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:58.908 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857750+0000", 2024-05-08T08:54:58.909 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8550000000000001e-06 2024-05-08T08:54:58.909 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.909 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.909 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:58.909 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905110+0000", 2024-05-08T08:54:58.909 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047360126000000002 2024-05-08T08:54:58.909 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.909 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.909 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:58.909 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905175+0000", 2024-05-08T08:54:58.909 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4689000000000004e-05 2024-05-08T08:54:58.909 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.909 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.909 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:58.909 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905255+0000", 2024-05-08T08:54:58.910 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0306e-05 2024-05-08T08:54:58.910 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.910 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:58.910 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.910 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.910 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.910 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22079 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 831488~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:58.910 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:54:58.910 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 151.042093002, 2024-05-08T08:54:58.910 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.04767499, 2024-05-08T08:54:58.910 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:58.910 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:58.910 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:58.910 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:58.911 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:58.911 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:58.911 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22079 2024-05-08T08:54:58.911 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.911 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:58.911 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.911 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:58.911 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:54:58.911 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:58.911 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.911 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.911 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:58.911 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:54:58.911 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:58.911 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.912 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.912 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:58.912 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857757+0000", 2024-05-08T08:54:58.912 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8299999999999997e-07 2024-05-08T08:54:58.912 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.912 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.912 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:58.912 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857759+0000", 2024-05-08T08:54:58.912 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1390000000000002e-06 2024-05-08T08:54:58.912 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.912 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.912 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:58.912 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857765+0000", 2024-05-08T08:54:58.912 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4609999999999996e-06 2024-05-08T08:54:58.912 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.912 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.913 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:58.913 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857770+0000", 2024-05-08T08:54:58.913 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6689999999999997e-06 2024-05-08T08:54:58.913 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.913 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.913 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:58.913 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905281+0000", 2024-05-08T08:54:58.913 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047510668999999998 2024-05-08T08:54:58.913 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.913 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.913 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:58.913 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905344+0000", 2024-05-08T08:54:58.913 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2797000000000002e-05 2024-05-08T08:54:58.913 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.913 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.913 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:58.913 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905431+0000", 2024-05-08T08:54:58.914 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7771999999999999e-05 2024-05-08T08:54:58.914 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.914 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:58.914 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.914 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.914 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.914 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22082 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 847872~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:58.914 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:54:58.914 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 151.042073311, 2024-05-08T08:54:58.914 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047831934999999999, 2024-05-08T08:54:58.914 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:58.914 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:58.914 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:58.914 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:58.914 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:58.914 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:58.915 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22082 2024-05-08T08:54:58.915 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.915 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:58.915 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.915 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:58.915 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:54:58.915 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:58.915 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.915 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.915 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:58.915 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:54:58.915 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:58.915 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.915 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.915 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:58.915 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:54:58.916 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:54:58.916 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.916 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.916 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:58.916 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857778+0000", 2024-05-08T08:54:58.916 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0789999999999999e-06 2024-05-08T08:54:58.916 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.916 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.916 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:58.916 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857784+0000", 2024-05-08T08:54:58.916 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0480000000000004e-06 2024-05-08T08:54:58.916 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.916 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.916 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:58.916 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857790+0000", 2024-05-08T08:54:58.916 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2719999999999997e-06 2024-05-08T08:54:58.916 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.917 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.917 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:58.917 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905458+0000", 2024-05-08T08:54:58.917 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047668470999999997 2024-05-08T08:54:58.917 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.917 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.917 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:58.917 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905521+0000", 2024-05-08T08:54:58.917 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2952000000000003e-05 2024-05-08T08:54:58.917 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.917 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.917 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:58.917 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905608+0000", 2024-05-08T08:54:58.917 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6759000000000001e-05 2024-05-08T08:54:58.917 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.917 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:58.918 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.918 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.918 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.918 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22084 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 864256~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:58.918 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:54:58.918 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 151.04204936100001, 2024-05-08T08:54:58.918 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047977748000000001, 2024-05-08T08:54:58.918 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:58.918 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:58.918 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:58.918 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:58.918 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:58.918 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:58.918 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22084 2024-05-08T08:54:58.919 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.919 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:58.919 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.919 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:58.919 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:54:58.919 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:58.919 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.919 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.919 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:58.919 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:54:58.919 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:58.919 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.919 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.919 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:58.919 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:54:58.919 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:54:58.919 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.920 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.920 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:58.920 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857802+0000", 2024-05-08T08:54:58.920 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0779999999999998e-06 2024-05-08T08:54:58.920 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.920 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.920 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:58.920 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857812+0000", 2024-05-08T08:54:58.920 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7939999999999997e-06 2024-05-08T08:54:58.920 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.920 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.920 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:58.920 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857817+0000", 2024-05-08T08:54:58.920 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8899999999999998e-06 2024-05-08T08:54:58.920 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.920 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.921 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:58.921 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905636+0000", 2024-05-08T08:54:58.921 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047819252 2024-05-08T08:54:58.921 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.921 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.921 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:58.921 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905700+0000", 2024-05-08T08:54:58.921 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4016999999999994e-05 2024-05-08T08:54:58.921 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.921 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.921 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:58.921 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905778+0000", 2024-05-08T08:54:58.921 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7306999999999997e-05 2024-05-08T08:54:58.921 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.921 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:58.921 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.922 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.922 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.922 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22086 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 880640~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:58.922 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:54:58.922 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 151.04202601099999, 2024-05-08T08:54:58.922 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048119149, 2024-05-08T08:54:58.922 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:58.922 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:58.922 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:58.922 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:58.922 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:58.922 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:58.922 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22086 2024-05-08T08:54:58.922 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.922 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:58.923 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.923 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:58.923 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:54:58.923 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:58.923 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.923 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.923 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:58.923 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:54:58.923 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:58.923 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.923 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.923 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:58.923 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857824+0000", 2024-05-08T08:54:58.923 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4200000000000002e-07 2024-05-08T08:54:58.923 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.924 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.924 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:58.924 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857826+0000", 2024-05-08T08:54:58.924 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0590000000000001e-06 2024-05-08T08:54:58.924 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.924 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.924 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:58.924 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857832+0000", 2024-05-08T08:54:58.924 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8029999999999999e-06 2024-05-08T08:54:58.924 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.924 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.924 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:58.924 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857837+0000", 2024-05-08T08:54:58.924 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7929999999999997e-06 2024-05-08T08:54:58.924 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.925 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.925 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:58.925 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905803+0000", 2024-05-08T08:54:58.925 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047965761000000003 2024-05-08T08:54:58.925 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.925 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.925 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:58.925 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905862+0000", 2024-05-08T08:54:58.925 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9203000000000002e-05 2024-05-08T08:54:58.925 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.925 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.925 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:58.925 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905942+0000", 2024-05-08T08:54:58.925 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0187999999999999e-05 2024-05-08T08:54:58.925 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.925 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:58.926 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.926 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.926 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.926 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22088 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 897024~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:58.926 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:54:58.926 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 151.04200595899999, 2024-05-08T08:54:58.926 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048256049000000002, 2024-05-08T08:54:58.926 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:58.926 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:58.926 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:58.926 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:58.926 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:58.926 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:58.926 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22088 2024-05-08T08:54:58.926 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.927 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:58.927 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.927 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:58.927 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:54:58.927 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:58.927 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.927 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.927 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:58.927 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:54:58.927 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:58.927 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.927 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.927 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:58.927 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857844+0000", 2024-05-08T08:54:58.927 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.46e-07 2024-05-08T08:54:58.928 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.928 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.928 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:58.928 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857846+0000", 2024-05-08T08:54:58.928 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0899999999999999e-06 2024-05-08T08:54:58.928 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.928 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.928 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:58.928 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857855+0000", 2024-05-08T08:54:58.928 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8219999999999996e-06 2024-05-08T08:54:58.928 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.928 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.928 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:58.928 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857860+0000", 2024-05-08T08:54:58.928 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2989999999999996e-06 2024-05-08T08:54:58.928 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.928 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.929 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:58.929 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905966+0000", 2024-05-08T08:54:58.929 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048106542000000002 2024-05-08T08:54:58.929 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.929 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.929 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:58.929 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906024+0000", 2024-05-08T08:54:58.929 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7958000000000003e-05 2024-05-08T08:54:58.929 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.929 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.929 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:58.929 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906099+0000", 2024-05-08T08:54:58.929 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4992000000000003e-05 2024-05-08T08:54:58.929 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.929 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:58.930 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.930 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.930 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.930 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22091 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 913408~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:58.930 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:54:58.930 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 151.04198219899999, 2024-05-08T08:54:58.930 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048395290000000001, 2024-05-08T08:54:58.930 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:58.930 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:58.930 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:58.930 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:58.930 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:58.930 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:58.930 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22091 2024-05-08T08:54:58.930 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.931 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:58.931 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.931 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:58.931 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:54:58.931 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:58.931 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.931 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.931 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:58.931 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:54:58.931 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:58.931 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.931 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.931 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:58.931 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:54:58.931 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4900000000000001e-07 2024-05-08T08:54:58.932 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.932 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.932 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:58.932 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857870+0000", 2024-05-08T08:54:58.933 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2510000000000001e-06 2024-05-08T08:54:58.933 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.934 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.934 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:58.934 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857877+0000", 2024-05-08T08:54:58.934 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0809999999999996e-06 2024-05-08T08:54:58.934 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.934 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.934 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:58.934 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857881+0000", 2024-05-08T08:54:58.934 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.724e-06 2024-05-08T08:54:58.934 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.934 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.934 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:58.934 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906127+0000", 2024-05-08T08:54:58.934 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048245345000000002 2024-05-08T08:54:58.935 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.935 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.935 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:58.935 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906190+0000", 2024-05-08T08:54:58.935 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2762000000000007e-05 2024-05-08T08:54:58.935 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.935 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.935 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:58.935 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906262+0000", 2024-05-08T08:54:58.935 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2778e-05 2024-05-08T08:54:58.935 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.935 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:58.935 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.935 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.935 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.935 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22093 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 929792~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:58.935 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:54:58.936 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 151.041959835, 2024-05-08T08:54:58.936 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048532875000000003, 2024-05-08T08:54:58.936 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:58.936 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:58.936 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:58.936 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:58.936 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:58.936 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:58.936 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22093 2024-05-08T08:54:58.936 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.936 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:58.936 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.936 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:58.936 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:54:58.936 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:58.936 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.937 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.937 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:58.937 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:54:58.937 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:58.937 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.937 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.937 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:58.937 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857890+0000", 2024-05-08T08:54:58.937 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-07 2024-05-08T08:54:58.937 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.937 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.937 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:58.937 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857899+0000", 2024-05-08T08:54:58.937 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6889999999999999e-06 2024-05-08T08:54:58.937 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.938 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.938 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:58.938 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857906+0000", 2024-05-08T08:54:58.938 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6490000000000002e-06 2024-05-08T08:54:58.938 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.938 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.938 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:58.938 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857917+0000", 2024-05-08T08:54:58.938 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0764999999999999e-05 2024-05-08T08:54:58.938 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.938 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.938 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:58.938 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906287+0000", 2024-05-08T08:54:58.938 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048370442 2024-05-08T08:54:58.939 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.939 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.939 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:58.939 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906349+0000", 2024-05-08T08:54:58.939 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1537000000000004e-05 2024-05-08T08:54:58.939 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.939 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.939 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:58.939 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906422+0000", 2024-05-08T08:54:58.939 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3431999999999998e-05 2024-05-08T08:54:58.939 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.939 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:58.939 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.939 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.939 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.939 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22095 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 946176~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:58.940 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:54:58.940 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 151.041925131, 2024-05-08T08:54:58.940 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048657593999999998, 2024-05-08T08:54:58.940 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:58.940 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:58.940 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:58.940 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:58.940 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:58.940 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:58.940 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22095 2024-05-08T08:54:58.940 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.940 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:58.940 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.940 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:58.940 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:54:58.941 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:58.941 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.941 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.941 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:58.941 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:54:58.941 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:58.941 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.941 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.941 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:58.941 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857925+0000", 2024-05-08T08:54:58.941 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3099999999999998e-07 2024-05-08T08:54:58.941 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.941 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.941 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:58.942 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857927+0000", 2024-05-08T08:54:58.942 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.34e-06 2024-05-08T08:54:58.942 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.942 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.942 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:58.942 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857938+0000", 2024-05-08T08:54:58.942 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1426e-05 2024-05-08T08:54:58.942 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.942 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.942 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:58.942 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857943+0000", 2024-05-08T08:54:58.942 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5140000000000003e-06 2024-05-08T08:54:58.942 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.942 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.942 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:58.943 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906448+0000", 2024-05-08T08:54:58.943 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048504641000000001 2024-05-08T08:54:58.943 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.943 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.943 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:58.943 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906510+0000", 2024-05-08T08:54:58.943 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2663000000000006e-05 2024-05-08T08:54:58.943 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.943 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.943 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:58.943 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906582+0000", 2024-05-08T08:54:58.943 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1478999999999998e-05 2024-05-08T08:54:58.943 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.943 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:58.943 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.944 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.944 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.944 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22097 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 962560~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:58.944 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:54:58.944 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 151.04189744199999, 2024-05-08T08:54:58.944 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048791353000000003, 2024-05-08T08:54:58.944 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:58.944 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:58.944 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:58.944 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:58.944 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:58.944 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:58.944 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22097 2024-05-08T08:54:58.944 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.945 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:58.945 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.945 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:58.945 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:54:58.945 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:58.945 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.945 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.945 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:58.945 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:54:58.945 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:58.945 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.945 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.945 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:58.945 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:54:58.945 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9200000000000002e-07 2024-05-08T08:54:58.946 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.946 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.946 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:58.946 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857954+0000", 2024-05-08T08:54:58.946 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1900000000000002e-06 2024-05-08T08:54:58.946 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.946 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.946 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:58.946 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857961+0000", 2024-05-08T08:54:58.946 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4989999999999999e-06 2024-05-08T08:54:58.946 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.946 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.946 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:58.946 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857971+0000", 2024-05-08T08:54:58.947 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0485999999999999e-05 2024-05-08T08:54:58.947 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.947 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.947 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:58.947 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906607+0000", 2024-05-08T08:54:58.947 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048635109000000003 2024-05-08T08:54:58.947 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.947 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.947 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:58.947 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906667+0000", 2024-05-08T08:54:58.947 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0439000000000002e-05 2024-05-08T08:54:58.947 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.947 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.947 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:58.947 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906743+0000", 2024-05-08T08:54:58.947 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6237999999999998e-05 2024-05-08T08:54:58.948 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.948 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:58.948 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.948 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.948 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.948 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22099 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 978944~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:58.948 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:54:58.948 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 151.04187128199999, 2024-05-08T08:54:58.948 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048932055000000002, 2024-05-08T08:54:58.948 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:58.948 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:58.948 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:58.948 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:58.948 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:58.948 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:58.949 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22099 2024-05-08T08:54:58.949 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.949 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:58.949 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.949 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:58.949 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:54:58.949 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:58.949 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.949 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.949 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:58.949 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:54:58.949 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:58.949 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.949 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.949 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:58.950 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:54:58.950 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.27e-07 2024-05-08T08:54:58.950 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.950 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.950 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:58.950 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857987+0000", 2024-05-08T08:54:58.950 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5890000000000003e-06 2024-05-08T08:54:58.950 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.950 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.950 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:58.950 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857994+0000", 2024-05-08T08:54:58.950 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6619999999999999e-06 2024-05-08T08:54:58.950 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.950 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.951 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:58.951 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857998+0000", 2024-05-08T08:54:58.951 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7110000000000003e-06 2024-05-08T08:54:58.951 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.951 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.951 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:58.951 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906772+0000", 2024-05-08T08:54:58.951 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048773415000000001 2024-05-08T08:54:58.951 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.951 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.951 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:58.951 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906834+0000", 2024-05-08T08:54:58.951 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1752000000000001e-05 2024-05-08T08:54:58.951 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.951 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.952 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:58.952 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906910+0000", 2024-05-08T08:54:58.952 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6499000000000001e-05 2024-05-08T08:54:58.952 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.952 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:58.952 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.952 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.952 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.952 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22101 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 995328~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:58.952 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:54:58.952 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 151.04184415099999, 2024-05-08T08:54:58.952 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049087364000000001, 2024-05-08T08:54:58.952 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:58.952 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:58.952 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:58.952 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:58.953 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:58.953 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:58.953 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22101 2024-05-08T08:54:58.953 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.953 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:58.953 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.953 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:58.953 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:54:58.953 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:58.953 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.953 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.953 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:58.953 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:54:58.953 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:58.953 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.954 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.954 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:58.954 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858006+0000", 2024-05-08T08:54:58.954 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1399999999999997e-07 2024-05-08T08:54:58.954 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.954 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.954 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:58.954 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858008+0000", 2024-05-08T08:54:58.954 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2060000000000001e-06 2024-05-08T08:54:58.954 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.954 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.954 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:58.954 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858017+0000", 2024-05-08T08:54:58.954 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1579999999999994e-06 2024-05-08T08:54:58.954 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.955 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.955 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:58.955 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858022+0000", 2024-05-08T08:54:58.955 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4199999999999998e-06 2024-05-08T08:54:58.955 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.955 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.955 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:58.955 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906940+0000", 2024-05-08T08:54:58.955 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048917627999999998 2024-05-08T08:54:58.955 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.955 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.955 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:58.955 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907002+0000", 2024-05-08T08:54:58.955 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2399999999999999e-05 2024-05-08T08:54:58.955 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.955 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.956 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:58.956 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907092+0000", 2024-05-08T08:54:58.956 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0137999999999997e-05 2024-05-08T08:54:58.956 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.956 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:58.956 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.956 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.956 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.956 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22103 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1011712~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:58.956 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:54:58.956 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 151.041819366, 2024-05-08T08:54:58.956 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049228542, 2024-05-08T08:54:58.956 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:58.956 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:58.956 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:58.956 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:58.957 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:58.957 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:58.957 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22103 2024-05-08T08:54:58.957 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.957 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:58.957 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.957 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:58.957 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:54:58.957 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:58.957 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.957 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.957 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:58.957 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:54:58.957 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:58.957 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.958 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.958 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:58.958 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:54:58.958 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4299999999999999e-07 2024-05-08T08:54:58.958 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.958 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.958 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:58.958 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858032+0000", 2024-05-08T08:54:58.958 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0590000000000001e-06 2024-05-08T08:54:58.958 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.958 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.958 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:58.958 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858037+0000", 2024-05-08T08:54:58.958 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.977e-06 2024-05-08T08:54:58.958 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.959 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.959 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:58.959 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858042+0000", 2024-05-08T08:54:58.959 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7650000000000001e-06 2024-05-08T08:54:58.959 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.959 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.959 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:58.959 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907118+0000", 2024-05-08T08:54:58.959 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049076147000000001 2024-05-08T08:54:58.959 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.959 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.959 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:58.959 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907180+0000", 2024-05-08T08:54:58.959 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1971999999999996e-05 2024-05-08T08:54:58.959 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.959 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.960 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:58.960 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907258+0000", 2024-05-08T08:54:58.960 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8279e-05 2024-05-08T08:54:58.960 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.960 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:58.960 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.960 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.960 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.960 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22105 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1028096~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:58.960 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:54:58.960 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 151.041800123, 2024-05-08T08:54:58.960 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049394609999999999, 2024-05-08T08:54:58.960 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:58.960 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:58.960 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:58.960 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:58.961 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:58.961 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:58.961 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22105 2024-05-08T08:54:58.961 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.961 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:58.961 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.961 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:58.961 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:54:58.961 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:58.961 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.961 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.961 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:58.961 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:54:58.961 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:58.961 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.961 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.962 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:58.962 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858050+0000", 2024-05-08T08:54:58.962 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5999999999999999e-07 2024-05-08T08:54:58.962 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.962 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.962 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:58.962 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858051+0000", 2024-05-08T08:54:58.962 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8870000000000001e-06 2024-05-08T08:54:58.962 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.962 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.962 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:58.962 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858057+0000", 2024-05-08T08:54:58.962 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8019999999999997e-06 2024-05-08T08:54:58.962 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.962 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.963 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:58.963 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858062+0000", 2024-05-08T08:54:58.963 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.561e-06 2024-05-08T08:54:58.963 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.963 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.963 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:58.963 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907289+0000", 2024-05-08T08:54:58.963 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049226957000000002 2024-05-08T08:54:58.963 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.963 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.963 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:58.963 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907356+0000", 2024-05-08T08:54:58.963 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6915000000000006e-05 2024-05-08T08:54:58.963 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.963 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.964 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:58.964 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907444+0000", 2024-05-08T08:54:58.964 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8127999999999994e-05 2024-05-08T08:54:58.964 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.964 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:58.964 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.964 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.964 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.964 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22107 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1044480~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:58.964 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:54:58.964 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 151.041781783, 2024-05-08T08:54:58.964 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049799942999999999, 2024-05-08T08:54:58.964 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:58.964 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:58.964 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:58.965 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:58.965 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:58.965 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:58.965 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22107 2024-05-08T08:54:58.965 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.965 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:58.965 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.965 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:58.965 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:54:58.965 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:58.965 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.965 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.965 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:58.965 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:54:58.965 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:58.965 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.966 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.966 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:58.966 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:54:58.966 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4799999999999999e-07 2024-05-08T08:54:58.966 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.966 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.966 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:58.966 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858070+0000", 2024-05-08T08:54:58.966 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.012e-06 2024-05-08T08:54:58.966 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.966 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.966 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:58.966 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858077+0000", 2024-05-08T08:54:58.966 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6290000000000001e-06 2024-05-08T08:54:58.966 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.967 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.967 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:58.967 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858083+0000", 2024-05-08T08:54:58.967 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8880000000000002e-06 2024-05-08T08:54:58.967 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.967 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.967 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:58.967 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907671+0000", 2024-05-08T08:54:58.967 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049587191000000003 2024-05-08T08:54:58.967 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.967 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.967 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:58.967 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907763+0000", 2024-05-08T08:54:58.967 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2231000000000005e-05 2024-05-08T08:54:58.967 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.968 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.968 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:58.968 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907867+0000", 2024-05-08T08:54:58.968 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104644 2024-05-08T08:54:58.968 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.968 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:58.968 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.968 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.968 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.968 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22294 3.1c 3:3aad0389:::rbd_data.11406886b07b.0000000000000080:head [stat,delete] snapc 7=[7] RETRY=6 ondisk+retry+write+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:54:58.968 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:54:58.968 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 151.04118793500001, 2024-05-08T08:54:58.968 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051152191, 2024-05-08T08:54:58.968 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:58.968 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:58.968 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:58.969 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:58.969 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:54:58.969 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:54:58.969 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22294 2024-05-08T08:54:58.969 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.969 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:58.969 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.969 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:58.969 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:54:58.969 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:58.969 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.969 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.969 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:58.969 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:54:58.969 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:58.970 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.970 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.970 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:58.970 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858662+0000", 2024-05-08T08:54:58.970 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4799999999999998e-07 2024-05-08T08:54:58.970 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.970 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.970 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:58.970 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858664+0000", 2024-05-08T08:54:58.970 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1229999999999998e-06 2024-05-08T08:54:58.970 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.970 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.970 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:58.970 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858670+0000", 2024-05-08T08:54:58.970 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3099999999999997e-06 2024-05-08T08:54:58.971 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.971 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.971 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:58.971 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858676+0000", 2024-05-08T08:54:58.971 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9750000000000004e-06 2024-05-08T08:54:58.971 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.971 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.971 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:58.971 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907901+0000", 2024-05-08T08:54:58.971 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049224520000000001 2024-05-08T08:54:58.971 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.971 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.971 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:58.971 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.908085+0000", 2024-05-08T08:54:58.971 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00018377500000000001 2024-05-08T08:54:58.972 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.972 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.972 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:58.972 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.909814+0000", 2024-05-08T08:54:58.972 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00172894 2024-05-08T08:54:58.972 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.972 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:58.972 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.972 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.972 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.972 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4416.0:13461 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-05-08T08:54:58.972 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:22.487456+0000", 2024-05-08T08:54:58.972 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 96.412392894000007, 2024-05-08T08:54:58.972 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.99102620799999996, 2024-05-08T08:54:58.972 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:58.973 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:58.973 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:58.973 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:58.973 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4416", 2024-05-08T08:54:58.973 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:54:58.973 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 13461 2024-05-08T08:54:58.973 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.973 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:58.973 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.973 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:58.973 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487456+0000", 2024-05-08T08:54:58.973 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:58.973 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.973 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.973 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:58.973 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487456+0000", 2024-05-08T08:54:58.973 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:58.974 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.974 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.974 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:58.974 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487458+0000", 2024-05-08T08:54:58.974 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5659999999999999e-06 2024-05-08T08:54:58.974 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.974 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.974 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:58.974 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487463+0000", 2024-05-08T08:54:58.974 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6890000000000003e-06 2024-05-08T08:54:58.974 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.974 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.974 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:58.974 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487472+0000", 2024-05-08T08:54:58.975 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7710000000000001e-06 2024-05-08T08:54:58.975 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.975 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.975 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:58.975 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487478+0000", 2024-05-08T08:54:58.975 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1390000000000003e-06 2024-05-08T08:54:58.975 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.975 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.975 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:58.975 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487520+0000", 2024-05-08T08:54:58.975 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2259999999999999e-05 2024-05-08T08:54:58.975 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.975 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.975 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-05-08T08:54:58.975 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487528+0000", 2024-05-08T08:54:58.975 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7479999999999993e-06 2024-05-08T08:54:58.976 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.976 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.976 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:58.976 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:23.478139+0000", 2024-05-08T08:54:58.976 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.99061165200000001 2024-05-08T08:54:58.976 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.976 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.976 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:58.976 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:23.478301+0000", 2024-05-08T08:54:58.976 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000161602 2024-05-08T08:54:58.976 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.976 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.976 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:58.976 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:23.478483+0000", 2024-05-08T08:54:58.976 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000181781 2024-05-08T08:54:58.976 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.976 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:58.977 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.977 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.977 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.977 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5752.0:404 3.1a 3:5f5e8104:::rbd_data.1140bc4a55c7.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 5175296~45056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-05-08T08:54:58.977 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:00.390863+0000", 2024-05-08T08:54:58.977 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 58.508986698999998, 2024-05-08T08:54:58.977 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.055189174000000001, 2024-05-08T08:54:58.977 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:58.977 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:58.977 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:54:58.977 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:58.977 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5752", 2024-05-08T08:54:58.977 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/3534419815", 2024-05-08T08:54:58.977 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 404 2024-05-08T08:54:58.977 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.977 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:58.977 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.978 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:58.978 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.390863+0000", 2024-05-08T08:54:58.978 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:58.978 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.978 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.978 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:58.978 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.390863+0000", 2024-05-08T08:54:58.978 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:58.978 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.978 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.978 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:58.978 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.390864+0000", 2024-05-08T08:54:58.978 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.471e-06 2024-05-08T08:54:58.978 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.978 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.979 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:58.979 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397475+0000", 2024-05-08T08:54:58.979 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0066109109999999997 2024-05-08T08:54:58.979 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.979 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.979 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:58.979 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397498+0000", 2024-05-08T08:54:58.979 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2674e-05 2024-05-08T08:54:58.979 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.979 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.979 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:58.979 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397508+0000", 2024-05-08T08:54:58.979 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9019999999999994e-06 2024-05-08T08:54:58.979 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.979 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.979 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:58.980 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397609+0000", 2024-05-08T08:54:58.980 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010121899999999999 2024-05-08T08:54:58.980 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.980 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.980 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:58.980 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397711+0000", 2024-05-08T08:54:58.980 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102489 2024-05-08T08:54:58.980 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.980 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.980 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-05-08T08:54:58.980 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397999+0000", 2024-05-08T08:54:58.980 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00028733799999999999 2024-05-08T08:54:58.980 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.980 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.980 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:54:58.980 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.432093+0000", 2024-05-08T08:54:58.981 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.034094865000000002 2024-05-08T08:54:58.981 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.981 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.981 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:54:58.981 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.445975+0000", 2024-05-08T08:54:58.981 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013881187 2024-05-08T08:54:58.981 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.981 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.981 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:54:58.981 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.446013+0000", 2024-05-08T08:54:58.981 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8155000000000003e-05 2024-05-08T08:54:58.981 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:58.981 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:58.981 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:58.981 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.446052+0000", 2024-05-08T08:54:58.981 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8962999999999999e-05 2024-05-08T08:54:58.982 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.982 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:58.982 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.982 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:58.982 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:58.982 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:58.983 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:54:58.987 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:58.991+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:59.088 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:59.088 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:54:59.088 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:54:59.088 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:59.088 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:59.091+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:59.097 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:54:59.189 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:59.192+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:59.241 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:59.241 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:54:59.241 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:54:59.241 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:54:59.241 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:59.249 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:54:59.293 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:54:59.296+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:54:59.343 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T08:54:59.355 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:54:59.355 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:54:59.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:54:59.355 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:54:59.356 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.356 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6690 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1093632~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:59.356 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:54:59.356 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 147.64219349699999, 2024-05-08T08:54:59.356 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052462754, 2024-05-08T08:54:59.356 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:59.356 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:59.356 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:59.356 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:59.356 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:59.356 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:59.356 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6690 2024-05-08T08:54:59.356 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.356 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:59.357 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.357 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:59.357 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:54:59.357 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:59.357 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.357 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.357 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:59.357 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:54:59.357 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:59.357 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.357 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.357 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:59.357 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715346+0000", 2024-05-08T08:54:59.357 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.84e-07 2024-05-08T08:54:59.357 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.358 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.358 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:59.358 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715348+0000", 2024-05-08T08:54:59.358 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.114e-06 2024-05-08T08:54:59.358 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.358 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.358 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:59.358 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715353+0000", 2024-05-08T08:54:59.358 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5720000000000002e-06 2024-05-08T08:54:59.358 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.358 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.358 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:59.358 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715358+0000", 2024-05-08T08:54:59.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7140000000000001e-06 2024-05-08T08:54:59.359 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.359 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.359 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:59.359 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767638+0000", 2024-05-08T08:54:59.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052279562000000002 2024-05-08T08:54:59.359 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.359 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.359 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:59.359 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767713+0000", 2024-05-08T08:54:59.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5652e-05 2024-05-08T08:54:59.359 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.359 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.359 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:59.359 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767808+0000", 2024-05-08T08:54:59.360 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4755999999999999e-05 2024-05-08T08:54:59.360 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:59.360 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:59.360 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:59.360 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.360 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.360 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6692 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1110016~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:59.360 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:54:59.360 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 147.642174407, 2024-05-08T08:54:59.360 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052666495000000001, 2024-05-08T08:54:59.360 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:59.360 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:59.360 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:59.360 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:59.361 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:59.361 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:59.361 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6692 2024-05-08T08:54:59.361 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.361 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:59.361 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.361 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:59.361 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:54:59.361 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:59.361 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.361 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.361 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:59.361 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:54:59.361 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:59.362 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.362 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.362 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:59.362 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:54:59.362 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8599999999999999e-07 2024-05-08T08:54:59.362 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.362 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.362 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:59.362 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715372+0000", 2024-05-08T08:54:59.362 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0990000000000001e-06 2024-05-08T08:54:59.362 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.362 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.362 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:59.362 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715378+0000", 2024-05-08T08:54:59.363 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0680000000000002e-06 2024-05-08T08:54:59.363 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.363 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.363 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:59.363 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715383+0000", 2024-05-08T08:54:59.363 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8740000000000003e-06 2024-05-08T08:54:59.363 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.363 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.363 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:59.363 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767848+0000", 2024-05-08T08:54:59.363 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052464668999999998 2024-05-08T08:54:59.363 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.363 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.363 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:59.363 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767927+0000", 2024-05-08T08:54:59.363 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9228999999999996e-05 2024-05-08T08:54:59.364 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.364 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.364 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:59.364 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768031+0000", 2024-05-08T08:54:59.364 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010417 2024-05-08T08:54:59.364 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:59.364 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:59.364 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:59.364 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.364 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.364 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6694 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1126400~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:59.364 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:54:59.364 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 147.64214951, 2024-05-08T08:54:59.364 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052848829, 2024-05-08T08:54:59.365 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:59.365 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:59.365 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:59.365 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:59.365 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:59.365 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:59.365 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6694 2024-05-08T08:54:59.365 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.365 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:59.365 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.365 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:59.365 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:54:59.365 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:59.365 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.366 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.366 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:59.366 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:54:59.366 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:59.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.366 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.366 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:59.366 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715390+0000", 2024-05-08T08:54:59.366 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9900000000000001e-07 2024-05-08T08:54:59.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.366 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.366 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:59.366 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715392+0000", 2024-05-08T08:54:59.366 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8279999999999999e-06 2024-05-08T08:54:59.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.367 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.367 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:59.367 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715398+0000", 2024-05-08T08:54:59.367 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7619999999999998e-06 2024-05-08T08:54:59.367 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.367 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.367 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:59.367 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715403+0000", 2024-05-08T08:54:59.367 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7790000000000003e-06 2024-05-08T08:54:59.367 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.367 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.367 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:59.367 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768067+0000", 2024-05-08T08:54:59.367 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052663785999999997 2024-05-08T08:54:59.367 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.368 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.368 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:59.368 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768142+0000", 2024-05-08T08:54:59.368 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5018999999999994e-05 2024-05-08T08:54:59.368 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.368 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.368 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:59.368 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768238+0000", 2024-05-08T08:54:59.368 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6255999999999995e-05 2024-05-08T08:54:59.368 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:59.368 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:59.368 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:59.368 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.368 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.368 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6696 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1142784~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:59.369 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:54:59.369 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 147.64213000199999, 2024-05-08T08:54:59.369 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.053051057999999998, 2024-05-08T08:54:59.369 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:59.369 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:59.369 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:59.369 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:59.369 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:59.369 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:59.369 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6696 2024-05-08T08:54:59.369 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.369 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:59.369 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.369 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:59.369 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:54:59.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:59.370 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.370 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.370 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:59.370 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:54:59.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:59.370 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.370 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.370 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:59.370 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:54:59.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-05-08T08:54:59.370 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.370 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.371 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:59.371 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715411+0000", 2024-05-08T08:54:59.371 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9630000000000001e-06 2024-05-08T08:54:59.371 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.371 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.371 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:59.371 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715417+0000", 2024-05-08T08:54:59.371 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7250000000000002e-06 2024-05-08T08:54:59.371 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.371 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.371 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:59.371 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715421+0000", 2024-05-08T08:54:59.371 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4089999999999996e-06 2024-05-08T08:54:59.371 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.371 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.372 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:59.372 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768277+0000", 2024-05-08T08:54:59.372 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052855293999999997 2024-05-08T08:54:59.372 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.372 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.372 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:59.372 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768358+0000", 2024-05-08T08:54:59.372 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1720999999999999e-05 2024-05-08T08:54:59.372 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.372 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.372 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:59.372 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768460+0000", 2024-05-08T08:54:59.372 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010154600000000001 2024-05-08T08:54:59.373 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:59.373 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:59.373 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:59.373 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.373 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.373 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6698 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1159168~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:59.373 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:54:59.373 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 147.64211113499999, 2024-05-08T08:54:59.373 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.053245726, 2024-05-08T08:54:59.373 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:59.373 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:59.373 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:59.373 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:59.373 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:59.373 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:59.374 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6698 2024-05-08T08:54:59.374 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.374 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:59.374 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.374 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:59.374 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:54:59.374 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:59.374 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.374 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.374 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:59.374 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:54:59.374 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:59.374 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.374 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.375 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:59.375 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716936+0000", 2024-05-08T08:54:59.375 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001508665 2024-05-08T08:54:59.375 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.375 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.375 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:59.375 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716941+0000", 2024-05-08T08:54:59.375 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1910000000000002e-06 2024-05-08T08:54:59.375 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.375 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.375 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:59.375 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716950+0000", 2024-05-08T08:54:59.375 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1900000000000001e-06 2024-05-08T08:54:59.375 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.375 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.376 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:59.376 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716958+0000", 2024-05-08T08:54:59.376 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8019999999999992e-06 2024-05-08T08:54:59.376 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.376 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.376 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:59.376 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768499+0000", 2024-05-08T08:54:59.376 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051540926000000001 2024-05-08T08:54:59.376 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.376 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.376 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:59.376 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768574+0000", 2024-05-08T08:54:59.376 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4988999999999996e-05 2024-05-08T08:54:59.376 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.377 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.377 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:59.377 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768673+0000", 2024-05-08T08:54:59.377 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9963000000000005e-05 2024-05-08T08:54:59.377 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:59.377 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:59.377 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:59.377 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.377 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.377 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6706 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1224704~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:59.377 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:54:59.377 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 147.64050582799999, 2024-05-08T08:54:59.377 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052435598, 2024-05-08T08:54:59.377 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:59.377 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:59.378 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:59.378 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:59.378 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:59.378 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:59.378 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6706 2024-05-08T08:54:59.378 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.378 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:59.378 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.378 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:59.378 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:54:59.378 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:59.378 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.378 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.378 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:59.379 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:54:59.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:59.379 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.379 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.379 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:59.379 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:54:59.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:54:59.379 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.379 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.379 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:59.379 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717035+0000", 2024-05-08T08:54:59.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9290000000000001e-06 2024-05-08T08:54:59.379 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.379 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.379 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:59.380 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717041+0000", 2024-05-08T08:54:59.380 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6720000000000001e-06 2024-05-08T08:54:59.380 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.380 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.380 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:59.380 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717046+0000", 2024-05-08T08:54:59.380 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5650000000000003e-06 2024-05-08T08:54:59.380 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.380 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.380 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:59.380 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769311+0000", 2024-05-08T08:54:59.380 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052265645999999999 2024-05-08T08:54:59.380 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.380 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.381 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:59.381 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769379+0000", 2024-05-08T08:54:59.381 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7391000000000006e-05 2024-05-08T08:54:59.381 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.381 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.381 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:59.381 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769469+0000", 2024-05-08T08:54:59.381 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0025999999999995e-05 2024-05-08T08:54:59.381 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:59.381 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:59.381 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:59.381 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.381 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.381 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6708 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1241088~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:59.381 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:54:59.381 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 147.640487724, 2024-05-08T08:54:59.382 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052640552, 2024-05-08T08:54:59.382 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:59.382 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:59.382 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:59.382 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:59.382 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:59.382 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:59.382 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6708 2024-05-08T08:54:59.382 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.382 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:59.382 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.382 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:59.382 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:54:59.383 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:59.383 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.383 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.383 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:59.383 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:54:59.383 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:59.383 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.383 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.383 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:59.383 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717052+0000", 2024-05-08T08:54:59.383 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:54:59.383 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.383 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.383 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:59.383 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717053+0000", 2024-05-08T08:54:59.384 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8330000000000001e-06 2024-05-08T08:54:59.384 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.384 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:59.384 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717058+0000", 2024-05-08T08:54:59.384 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.933e-06 2024-05-08T08:54:59.384 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.384 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:59.384 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717063+0000", 2024-05-08T08:54:59.384 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2180000000000001e-06 2024-05-08T08:54:59.384 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.385 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:59.385 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769507+0000", 2024-05-08T08:54:59.385 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052444866999999999 2024-05-08T08:54:59.385 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.385 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.385 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:59.385 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769589+0000", 2024-05-08T08:54:59.385 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2039000000000005e-05 2024-05-08T08:54:59.385 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.385 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.385 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:59.385 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769692+0000", 2024-05-08T08:54:59.385 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102252 2024-05-08T08:54:59.385 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:59.386 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:59.386 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:59.386 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.386 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.386 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6710 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1257472~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:59.386 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:54:59.386 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 147.64046970000001, 2024-05-08T08:54:59.386 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.05283057, 2024-05-08T08:54:59.386 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:59.386 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:59.386 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:59.386 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:59.386 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:59.386 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:59.386 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6710 2024-05-08T08:54:59.387 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.387 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:59.387 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.387 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:59.387 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:54:59.387 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:59.387 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.387 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.387 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:59.387 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:54:59.387 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:59.387 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.387 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.387 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:59.388 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718951+0000", 2024-05-08T08:54:59.388 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0018817809999999999 2024-05-08T08:54:59.388 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.388 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.388 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:59.388 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718956+0000", 2024-05-08T08:54:59.388 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5399999999999997e-06 2024-05-08T08:54:59.388 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.388 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.388 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:59.388 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718964+0000", 2024-05-08T08:54:59.388 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6629999999999999e-06 2024-05-08T08:54:59.388 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.388 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.388 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:59.389 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718972+0000", 2024-05-08T08:54:59.389 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5229999999999998e-06 2024-05-08T08:54:59.389 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.389 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.389 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:59.389 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769729+0000", 2024-05-08T08:54:59.389 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050756959999999997 2024-05-08T08:54:59.389 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.389 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.389 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:59.389 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769803+0000", 2024-05-08T08:54:59.389 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4538999999999999e-05 2024-05-08T08:54:59.389 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.389 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.390 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:59.390 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769900+0000", 2024-05-08T08:54:59.390 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6564000000000006e-05 2024-05-08T08:54:59.390 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:59.390 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:59.390 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:59.390 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.390 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.390 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6726 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1388544~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:59.390 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:54:59.390 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 147.638394928, 2024-05-08T08:54:59.390 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052451506000000002, 2024-05-08T08:54:59.390 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:59.390 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:59.391 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:59.391 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:59.391 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:59.391 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:59.391 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6726 2024-05-08T08:54:59.391 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.391 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:59.391 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.391 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:59.391 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:54:59.391 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:59.391 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.391 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.391 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:59.392 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:54:59.392 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:59.392 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.392 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.392 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:59.392 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719200+0000", 2024-05-08T08:54:59.392 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5865999999999997e-05 2024-05-08T08:54:59.392 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.392 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.392 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:59.392 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719204+0000", 2024-05-08T08:54:59.392 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8630000000000001e-06 2024-05-08T08:54:59.392 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.392 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.392 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:59.393 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719215+0000", 2024-05-08T08:54:59.393 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1590000000000001e-05 2024-05-08T08:54:59.393 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.393 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.393 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:59.393 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719220+0000", 2024-05-08T08:54:59.393 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1619999999999999e-06 2024-05-08T08:54:59.393 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.393 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.393 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:59.393 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771425+0000", 2024-05-08T08:54:59.393 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052204249000000001 2024-05-08T08:54:59.393 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.393 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.394 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:59.394 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771498+0000", 2024-05-08T08:54:59.394 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3782999999999995e-05 2024-05-08T08:54:59.394 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.394 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.394 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:59.394 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771595+0000", 2024-05-08T08:54:59.394 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6992999999999998e-05 2024-05-08T08:54:59.394 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:59.394 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:59.394 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:59.394 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.394 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.394 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6728 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1404928~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:59.395 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:54:59.395 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 147.63830752300001, 2024-05-08T08:54:59.395 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052590747, 2024-05-08T08:54:59.395 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:59.395 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:59.395 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:59.395 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:59.395 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:59.395 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:59.395 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6728 2024-05-08T08:54:59.395 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.395 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:59.395 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.395 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:59.395 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:54:59.396 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:59.396 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.396 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.396 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:59.396 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:54:59.396 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:59.396 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.396 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.396 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:59.396 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719232+0000", 2024-05-08T08:54:59.396 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4000000000000001e-07 2024-05-08T08:54:59.396 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.396 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.396 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:59.397 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719234+0000", 2024-05-08T08:54:59.397 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4499999999999998e-06 2024-05-08T08:54:59.397 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.397 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.397 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:59.397 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719240+0000", 2024-05-08T08:54:59.397 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5720000000000002e-06 2024-05-08T08:54:59.397 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.397 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.397 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:59.397 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719245+0000", 2024-05-08T08:54:59.397 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8690000000000003e-06 2024-05-08T08:54:59.397 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.397 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.397 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:59.398 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771636+0000", 2024-05-08T08:54:59.398 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052390979999999997 2024-05-08T08:54:59.398 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.398 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.398 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:59.398 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771717+0000", 2024-05-08T08:54:59.398 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1291999999999993e-05 2024-05-08T08:54:59.398 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.398 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.398 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:59.398 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771822+0000", 2024-05-08T08:54:59.398 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104944 2024-05-08T08:54:59.398 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:59.398 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:59.399 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:59.399 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.399 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.399 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6730 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1421312~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:59.399 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:54:59.399 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 147.638287281, 2024-05-08T08:54:59.399 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052770477000000003, 2024-05-08T08:54:59.399 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:59.399 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:59.399 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:59.399 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:59.399 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:59.399 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:59.399 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6730 2024-05-08T08:54:59.400 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.400 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:59.400 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.400 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:59.400 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:54:59.400 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:59.400 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.400 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.400 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:59.400 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:54:59.400 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:59.400 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.400 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.400 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:59.400 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:54:59.401 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9000000000000002e-07 2024-05-08T08:54:59.401 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.401 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.401 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:59.401 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719254+0000", 2024-05-08T08:54:59.401 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4099999999999998e-06 2024-05-08T08:54:59.401 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.401 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.401 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:59.401 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719263+0000", 2024-05-08T08:54:59.401 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4109999999999996e-06 2024-05-08T08:54:59.401 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.401 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.401 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:59.401 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719268+0000", 2024-05-08T08:54:59.402 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2669999999999997e-06 2024-05-08T08:54:59.402 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.402 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.402 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:59.402 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771846+0000", 2024-05-08T08:54:59.402 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052577677000000003 2024-05-08T08:54:59.402 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.402 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.402 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:59.402 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771920+0000", 2024-05-08T08:54:59.402 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4460000000000002e-05 2024-05-08T08:54:59.402 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.402 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.402 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:59.403 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772022+0000", 2024-05-08T08:54:59.403 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010186199999999999 2024-05-08T08:54:59.403 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:59.403 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:59.403 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:59.403 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.403 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.403 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6732 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1437696~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:59.403 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:54:59.403 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 147.63826247399999, 2024-05-08T08:54:59.403 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052905575000000003, 2024-05-08T08:54:59.403 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:59.403 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:59.403 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:59.403 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:59.404 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:59.404 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:59.404 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6732 2024-05-08T08:54:59.404 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.404 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:59.404 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.404 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:59.404 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:54:59.404 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:59.404 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.404 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.404 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:59.404 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:54:59.404 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:59.404 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.405 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.405 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:59.405 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721392+0000", 2024-05-08T08:54:59.405 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002115287 2024-05-08T08:54:59.405 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.405 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.405 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:59.405 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721396+0000", 2024-05-08T08:54:59.405 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1679999999999997e-06 2024-05-08T08:54:59.405 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.405 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.405 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:59.405 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721404+0000", 2024-05-08T08:54:59.405 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4409999999999993e-06 2024-05-08T08:54:59.406 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.406 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.406 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:59.406 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721410+0000", 2024-05-08T08:54:59.406 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9150000000000001e-06 2024-05-08T08:54:59.406 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.406 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.406 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:59.406 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772061+0000", 2024-05-08T08:54:59.406 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050651068 2024-05-08T08:54:59.406 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.406 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.406 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:59.406 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772127+0000", 2024-05-08T08:54:59.407 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5424000000000002e-05 2024-05-08T08:54:59.407 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.407 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.407 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:59.407 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772182+0000", 2024-05-08T08:54:59.407 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5272000000000002e-05 2024-05-08T08:54:59.407 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:59.407 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:59.407 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:59.407 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.407 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.407 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6749 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1585152~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:59.407 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:54:59.407 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 147.63596170400001, 2024-05-08T08:54:59.407 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052350264000000001, 2024-05-08T08:54:59.408 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:59.408 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:59.408 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:59.408 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:59.408 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:59.408 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:59.408 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6749 2024-05-08T08:54:59.408 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.408 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:59.408 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.408 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:59.408 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:54:59.408 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:59.408 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.409 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.409 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:59.409 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:54:59.409 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:59.409 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.409 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.409 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:59.409 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721578+0000", 2024-05-08T08:54:59.409 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0399999999999996e-07 2024-05-08T08:54:59.409 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.409 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.409 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:59.409 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721580+0000", 2024-05-08T08:54:59.409 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5450000000000002e-06 2024-05-08T08:54:59.410 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.410 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.410 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:59.410 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721587+0000", 2024-05-08T08:54:59.410 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0190000000000001e-06 2024-05-08T08:54:59.410 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.410 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.410 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:59.410 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721593+0000", 2024-05-08T08:54:59.410 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5659999999999997e-06 2024-05-08T08:54:59.410 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.410 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.410 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:59.410 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773759+0000", 2024-05-08T08:54:59.410 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052166553999999997 2024-05-08T08:54:59.411 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.411 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.411 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:59.411 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773833+0000", 2024-05-08T08:54:59.411 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3913999999999997e-05 2024-05-08T08:54:59.411 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.411 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.411 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:59.411 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773927+0000", 2024-05-08T08:54:59.411 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4061999999999995e-05 2024-05-08T08:54:59.411 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:59.411 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:59.411 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:59.411 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.412 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.412 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6751 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1601536~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:59.412 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:54:59.412 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 147.63593842500001, 2024-05-08T08:54:59.412 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052461302000000001, 2024-05-08T08:54:59.412 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:59.412 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:59.412 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:59.412 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:59.412 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:59.412 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:59.412 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6751 2024-05-08T08:54:59.412 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.412 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:59.413 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.413 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:59.413 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:54:59.413 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:59.413 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.413 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.413 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:59.413 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:54:59.413 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:59.413 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.413 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.413 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:59.413 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722939+0000", 2024-05-08T08:54:59.413 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001338601 2024-05-08T08:54:59.413 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.414 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.414 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:59.414 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722944+0000", 2024-05-08T08:54:59.414 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4469999999999999e-06 2024-05-08T08:54:59.414 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.414 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.414 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:59.414 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722955+0000", 2024-05-08T08:54:59.414 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1892999999999999e-05 2024-05-08T08:54:59.414 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.414 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.414 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:59.414 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722963+0000", 2024-05-08T08:54:59.414 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1080000000000004e-06 2024-05-08T08:54:59.414 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.415 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.415 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:59.415 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773964+0000", 2024-05-08T08:54:59.415 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051001022 2024-05-08T08:54:59.415 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.415 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.415 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:59.415 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.774006+0000", 2024-05-08T08:54:59.415 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2729999999999999e-05 2024-05-08T08:54:59.415 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.415 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.415 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:59.415 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.774062+0000", 2024-05-08T08:54:59.415 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5501000000000003e-05 2024-05-08T08:54:59.416 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:59.416 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:59.416 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:59.416 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.416 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.416 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6760 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1683456~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:59.416 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:54:59.416 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 147.634429049, 2024-05-08T08:54:59.416 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052571312000000002, 2024-05-08T08:54:59.416 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:59.416 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:59.416 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:59.416 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:59.416 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:59.416 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:59.417 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6760 2024-05-08T08:54:59.417 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.417 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:59.417 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.417 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:59.417 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:54:59.417 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:59.417 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.417 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.417 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:59.417 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:54:59.417 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:59.417 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.417 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.417 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:59.418 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:54:59.418 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2100000000000002e-07 2024-05-08T08:54:59.418 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.418 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.418 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:59.418 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723113+0000", 2024-05-08T08:54:59.418 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3149999999999999e-06 2024-05-08T08:54:59.418 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.418 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.418 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:59.418 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723118+0000", 2024-05-08T08:54:59.418 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8860000000000001e-06 2024-05-08T08:54:59.418 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.418 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.419 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:59.419 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723123+0000", 2024-05-08T08:54:59.419 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6770000000000003e-06 2024-05-08T08:54:59.419 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.419 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.419 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:59.419 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775414+0000", 2024-05-08T08:54:59.419 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052290692999999999 2024-05-08T08:54:59.419 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.419 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.419 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:59.419 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775459+0000", 2024-05-08T08:54:59.419 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5639999999999997e-05 2024-05-08T08:54:59.419 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.419 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.420 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:59.420 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775681+0000", 2024-05-08T08:54:59.420 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00022168000000000001 2024-05-08T08:54:59.420 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:59.420 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:59.420 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:59.420 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.420 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.420 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6761 3.16 3:6bd6d6da:::rbd_data.1140782590a5.0000000000000015:6 [read 1095680~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:59.420 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:54:59.420 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 147.63440982899999, 2024-05-08T08:54:59.420 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052751275, 2024-05-08T08:54:59.420 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:59.420 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:59.420 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:59.420 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:59.421 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:59.421 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:59.421 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6761 2024-05-08T08:54:59.421 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.421 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:59.421 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.421 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:59.421 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:54:59.421 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:59.421 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.421 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.421 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:59.421 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:54:59.421 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:59.421 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.422 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.422 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:59.422 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724939+0000", 2024-05-08T08:54:59.422 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001809817 2024-05-08T08:54:59.422 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.422 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.422 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:59.422 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724943+0000", 2024-05-08T08:54:59.422 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4920000000000004e-06 2024-05-08T08:54:59.422 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.422 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.422 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:59.422 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724956+0000", 2024-05-08T08:54:59.422 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2427e-05 2024-05-08T08:54:59.423 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.423 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.423 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:59.423 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724963+0000", 2024-05-08T08:54:59.423 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2559999999999996e-06 2024-05-08T08:54:59.423 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.423 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.423 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:59.423 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775711+0000", 2024-05-08T08:54:59.423 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050747621 2024-05-08T08:54:59.423 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.423 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.423 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:59.423 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775785+0000", 2024-05-08T08:54:59.423 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4634000000000004e-05 2024-05-08T08:54:59.424 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.424 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.424 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:59.424 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775880+0000", 2024-05-08T08:54:59.424 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5027999999999999e-05 2024-05-08T08:54:59.424 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:59.424 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:59.424 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:59.424 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.424 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.424 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6770 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1765376~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:59.424 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:54:59.424 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 147.63237873400001, 2024-05-08T08:54:59.424 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052339296, 2024-05-08T08:54:59.424 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:59.425 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:59.425 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:59.425 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:59.425 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:59.425 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:59.425 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6770 2024-05-08T08:54:59.425 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.425 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:59.425 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.425 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:59.425 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:54:59.425 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:59.425 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.425 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.425 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:59.426 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:54:59.426 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:59.426 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.426 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.426 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:59.426 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725161+0000", 2024-05-08T08:54:59.426 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-05-08T08:54:59.426 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.426 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.426 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:59.426 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725163+0000", 2024-05-08T08:54:59.426 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4370000000000001e-06 2024-05-08T08:54:59.426 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.427 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.427 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:59.427 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725169+0000", 2024-05-08T08:54:59.427 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8409999999999996e-06 2024-05-08T08:54:59.427 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.427 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.427 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:59.427 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725174+0000", 2024-05-08T08:54:59.427 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7249999999999997e-06 2024-05-08T08:54:59.427 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.427 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.427 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:59.427 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777343+0000", 2024-05-08T08:54:59.427 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052169633999999999 2024-05-08T08:54:59.427 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.428 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.428 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:59.428 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777409+0000", 2024-05-08T08:54:59.428 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6033999999999998e-05 2024-05-08T08:54:59.428 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.428 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.428 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:59.428 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777499+0000", 2024-05-08T08:54:59.428 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0224999999999998e-05 2024-05-08T08:54:59.428 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:59.428 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:59.428 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:59.428 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.428 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.429 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6771 3.16 3:6bd6d6da:::rbd_data.1140782590a5.0000000000000015:6 [read 1177600~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:59.429 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:54:59.429 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 147.632358917, 2024-05-08T08:54:59.429 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052438604999999999, 2024-05-08T08:54:59.429 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:59.429 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:59.429 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:59.429 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:59.429 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:59.429 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:59.429 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6771 2024-05-08T08:54:59.429 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.429 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:59.429 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.429 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:59.430 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:54:59.430 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:59.430 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.430 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.430 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:59.430 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:54:59.430 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:59.430 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.430 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.430 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:59.430 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:54:59.430 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:54:59.430 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.430 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.430 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:59.431 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725188+0000", 2024-05-08T08:54:59.431 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1579999999999999e-06 2024-05-08T08:54:59.431 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.431 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.431 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:59.431 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725195+0000", 2024-05-08T08:54:59.431 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8909999999999995e-06 2024-05-08T08:54:59.431 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.431 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.431 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:59.431 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725200+0000", 2024-05-08T08:54:59.431 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9810000000000003e-06 2024-05-08T08:54:59.431 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.431 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.432 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:59.432 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777527+0000", 2024-05-08T08:54:59.432 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052326582000000003 2024-05-08T08:54:59.432 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.432 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.432 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:59.432 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777566+0000", 2024-05-08T08:54:59.432 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9447999999999999e-05 2024-05-08T08:54:59.432 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.432 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.432 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:59.432 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777619+0000", 2024-05-08T08:54:59.432 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2135000000000001e-05 2024-05-08T08:54:59.432 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:59.433 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:59.433 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:59.433 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.433 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.433 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6772 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1781760~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:59.433 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:54:59.433 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 147.63233086, 2024-05-08T08:54:59.433 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052601799999999997, 2024-05-08T08:54:59.433 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:59.433 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:59.433 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:59.433 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:59.433 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:59.433 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:59.433 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6772 2024-05-08T08:54:59.434 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.434 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:59.434 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.434 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:59.434 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:54:59.434 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:59.434 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.434 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.434 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:59.434 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:54:59.434 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:59.434 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.434 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.434 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:59.435 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726939+0000", 2024-05-08T08:54:59.435 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001730493 2024-05-08T08:54:59.435 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.435 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.435 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:59.435 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726943+0000", 2024-05-08T08:54:59.435 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7330000000000003e-06 2024-05-08T08:54:59.435 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.435 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.435 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:59.435 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726954+0000", 2024-05-08T08:54:59.435 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1107000000000001e-05 2024-05-08T08:54:59.435 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.435 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.435 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:59.436 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726962+0000", 2024-05-08T08:54:59.436 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.503e-06 2024-05-08T08:54:59.436 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.436 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.436 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:59.436 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777653+0000", 2024-05-08T08:54:59.436 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050691542999999999 2024-05-08T08:54:59.436 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.436 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.436 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:59.436 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777719+0000", 2024-05-08T08:54:59.436 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5228000000000006e-05 2024-05-08T08:54:59.436 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.436 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.436 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:59.437 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777810+0000", 2024-05-08T08:54:59.437 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1193000000000006e-05 2024-05-08T08:54:59.437 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:59.437 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:59.437 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:59.437 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.437 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.437 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6782 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1880064~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:54:59.437 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:54:59.437 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 147.63036235300001, 2024-05-08T08:54:59.437 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052326492000000002, 2024-05-08T08:54:59.437 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:54:59.437 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:54:59.437 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:54:59.437 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:54:59.438 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:54:59.438 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:54:59.438 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6782 2024-05-08T08:54:59.438 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.438 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:54:59.438 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.438 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:54:59.438 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:54:59.438 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:59.438 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.438 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.438 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:54:59.438 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:54:59.438 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:54:59.438 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.439 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.439 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:54:59.439 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728939+0000", 2024-05-08T08:54:59.439 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001762719 2024-05-08T08:54:59.439 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.439 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.439 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:54:59.439 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728944+0000", 2024-05-08T08:54:59.439 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.746e-06 2024-05-08T08:54:59.439 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.439 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.439 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:54:59.439 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728955+0000", 2024-05-08T08:54:59.439 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1043000000000001e-05 2024-05-08T08:54:59.439 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.440 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.440 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:54:59.440 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728963+0000", 2024-05-08T08:54:59.440 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5809999999999999e-06 2024-05-08T08:54:59.440 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.440 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.440 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:54:59.440 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779320+0000", 2024-05-08T08:54:59.440 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050357290999999998 2024-05-08T08:54:59.440 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.440 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.440 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:54:59.440 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779401+0000", 2024-05-08T08:54:59.440 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0574000000000005e-05 2024-05-08T08:54:59.440 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:54:59.441 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:54:59.441 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:54:59.441 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779503+0000", 2024-05-08T08:54:59.441 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102538 2024-05-08T08:54:59.441 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:59.441 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:59.441 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:59.441 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:54:59.441 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:54:59.441 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:54:59.441 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-05-08T08:54:59.443 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:54:59.444 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:54:59.445+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:54:59.500 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:54:59.500 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:54:59.520 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:54:59.520 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:54:59.539 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:54:59.539 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:54:59.543 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:59.546+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:59.561 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:54:59.561 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:54:59.569 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:59.569 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:59.580 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:54:59.580 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:54:59.581 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:54:59.600 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:54:59.600 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:54:59.644 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:54:59.647+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:54:59.691 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:59.692 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:59.701 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:54:59.745 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:59.748+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:59.811 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:59.811 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:59.820 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:54:59.861 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:54:59.864+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:54:59.947 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:54:59.950+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:54:59.970 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:54:59.971 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:54:59.979 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:55:00.063 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:55:00.066+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:55:00.129 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:00.129 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:00.138 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:55:00.180 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:55:00.182+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:55:00.249 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:55:00.252+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:55:00.288 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:00.288 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:00.297 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:55:00.350 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:55:00.353+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:55:00.386 INFO:teuthology.orchestra.run.smithi059.stderr:noscrub is set 2024-05-08T08:55:00.419 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:00.419 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:00.430 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:55:00.471 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:55:00.474+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:55:00.552 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:55:00.555+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:55:00.584 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:00.585 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:00.594 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:55:00.653 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:55:00.656+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:55:00.703 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:00.703 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:00.711 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:55:00.755 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:55:00.758+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:55:00.818 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:00.818 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:00.826 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:55:00.867 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:55:00.870+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:55:00.956 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:55:00.958+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:55:00.970 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:00.971 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:00.979 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:55:01.057 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:55:01.060+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:55:01.124 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:01.124 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:01.132 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:55:01.158 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:55:01.161+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:55:01.238 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:01.239 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:01.248 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:55:01.258 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:55:01.261+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:55:01.352 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:01.352 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:01.359 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:55:01.362+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:55:01.360 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:55:01.460 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:55:01.462+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:55:01.472 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:01.472 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:01.481 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:55:01.561 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:55:01.563+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:55:01.592 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:01.592 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:01.601 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:55:01.612 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T08:55:01.662 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:55:01.664+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:55:01.708 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:01.709 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:01.717 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:55:01.738 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:55:01.738 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:55:01.756 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:55:01.756 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:55:01.763 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:55:01.766+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:55:01.773 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:55:01.773 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:55:01.792 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:55:01.792 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:55:01.809 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:55:01.809 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:55:01.825 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:01.825 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:01.827 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:55:01.827 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:55:01.834 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:55:01.879 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:55:01.882+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:55:01.944 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:55:01.944 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:55:01.944 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:55:01.944 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:55:01.953 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:55:01.994 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:55:01.997+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:55:02.065 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:55:02.067+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:55:02.102 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:55:02.102 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:55:02.102 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:55:02.103 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:55:02.103 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:55:02.112 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:55:02.166 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:55:02.169+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:55:02.263 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:55:02.264 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:55:02.264 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:55:02.264 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:55:02.264 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.264 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4621.0:9089 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 23339520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:55:02.264 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:55:02.264 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 149.77254638100001, 2024-05-08T08:55:02.264 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.13814907200000001, 2024-05-08T08:55:02.264 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:02.264 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:02.264 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:55:02.264 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:02.264 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4621", 2024-05-08T08:55:02.264 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:02.264 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9089 2024-05-08T08:55:02.265 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.265 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:02.265 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.265 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:02.265 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:55:02.265 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:02.265 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.265 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.265 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:02.265 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:55:02.265 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:02.265 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.265 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.265 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:02.265 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493005+0000", 2024-05-08T08:55:02.265 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1700000000000001e-06 2024-05-08T08:55:02.266 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.266 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.266 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:02.266 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.513963+0000", 2024-05-08T08:55:02.266 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.020957581999999999 2024-05-08T08:55:02.266 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.266 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.266 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:02.266 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.513995+0000", 2024-05-08T08:55:02.266 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1993999999999999e-05 2024-05-08T08:55:02.266 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.266 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.266 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:02.266 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514007+0000", 2024-05-08T08:55:02.267 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1581e-05 2024-05-08T08:55:02.267 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.267 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.267 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:02.267 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514122+0000", 2024-05-08T08:55:02.267 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000115282 2024-05-08T08:55:02.267 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.267 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.267 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op must wait for map", 2024-05-08T08:55:02.267 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514128+0000", 2024-05-08T08:55:02.267 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9950000000000002e-06 2024-05-08T08:55:02.267 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.267 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.267 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:02.267 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.515800+0000", 2024-05-08T08:55:02.268 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001671886 2024-05-08T08:55:02.268 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.268 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.268 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:02.268 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.515893+0000", 2024-05-08T08:55:02.268 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3490000000000001e-05 2024-05-08T08:55:02.268 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.268 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.268 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 3", 2024-05-08T08:55:02.268 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.517647+0000", 2024-05-08T08:55:02.268 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0017535630000000001 2024-05-08T08:55:02.268 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.268 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.268 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:55:02.268 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.627578+0000", 2024-05-08T08:55:02.268 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.109931557 2024-05-08T08:55:02.269 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.269 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.269 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:55:02.269 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631055+0000", 2024-05-08T08:55:02.269 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0034761420000000002 2024-05-08T08:55:02.269 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.269 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.269 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:55:02.269 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631096+0000", 2024-05-08T08:55:02.269 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1059000000000001e-05 2024-05-08T08:55:02.269 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.269 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.269 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:02.269 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631151+0000", 2024-05-08T08:55:02.269 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5770999999999998e-05 2024-05-08T08:55:02.270 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.270 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:02.270 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.270 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.270 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.270 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1071 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11026432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:55:02.270 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:55:02.270 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 145.78009689000001, 2024-05-08T08:55:02.270 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070580037999999998, 2024-05-08T08:55:02.270 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:02.270 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:02.270 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:02.270 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:02.270 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:55:02.270 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:55:02.271 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1071 2024-05-08T08:55:02.271 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.271 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:02.271 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.271 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:02.271 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:55:02.271 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:02.271 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.271 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.271 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:02.271 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:55:02.271 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:02.271 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.271 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.271 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:02.271 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:55:02.272 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2800000000000002e-07 2024-05-08T08:55:02.272 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.272 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.272 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:02.272 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485454+0000", 2024-05-08T08:55:02.272 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2290000000000002e-06 2024-05-08T08:55:02.272 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.272 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.272 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:02.272 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485461+0000", 2024-05-08T08:55:02.272 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9920000000000001e-06 2024-05-08T08:55:02.272 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.272 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.272 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:02.273 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485467+0000", 2024-05-08T08:55:02.273 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3979999999999998e-06 2024-05-08T08:55:02.273 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.273 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.273 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:02.273 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555880+0000", 2024-05-08T08:55:02.273 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070413165 2024-05-08T08:55:02.273 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.273 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.273 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:02.273 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555926+0000", 2024-05-08T08:55:02.273 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6217000000000002e-05 2024-05-08T08:55:02.273 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.273 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.273 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:02.273 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556032+0000", 2024-05-08T08:55:02.274 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000105609 2024-05-08T08:55:02.274 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.274 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:02.274 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.274 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.274 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.274 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1073 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11067392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:55:02.274 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:55:02.274 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 145.78007566900001, 2024-05-08T08:55:02.274 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070821404000000004, 2024-05-08T08:55:02.274 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:02.274 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:02.274 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:02.274 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:02.275 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:55:02.275 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:55:02.275 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1073 2024-05-08T08:55:02.275 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.275 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:02.275 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.275 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:02.275 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:55:02.275 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:02.275 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.275 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.275 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:02.275 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:55:02.275 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:02.275 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.275 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.276 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:02.276 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:55:02.276 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.08e-07 2024-05-08T08:55:02.276 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.276 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.276 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:02.276 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485476+0000", 2024-05-08T08:55:02.276 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3929999999999998e-06 2024-05-08T08:55:02.276 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.276 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.276 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:02.276 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485485+0000", 2024-05-08T08:55:02.276 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1880000000000008e-06 2024-05-08T08:55:02.276 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.276 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.276 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:02.277 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485494+0000", 2024-05-08T08:55:02.277 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3109999999999995e-06 2024-05-08T08:55:02.277 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.277 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.277 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:02.277 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556082+0000", 2024-05-08T08:55:02.277 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070587426999999994 2024-05-08T08:55:02.277 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.277 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.277 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:02.277 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556177+0000", 2024-05-08T08:55:02.277 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5051999999999997e-05 2024-05-08T08:55:02.277 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.277 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.277 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:02.278 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556294+0000", 2024-05-08T08:55:02.278 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000117625 2024-05-08T08:55:02.278 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.278 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:02.278 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.278 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.278 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.278 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1075 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11108352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:55:02.278 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:55:02.278 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 145.78004509199999, 2024-05-08T08:55:02.278 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071045026999999997, 2024-05-08T08:55:02.278 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:02.278 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:02.278 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:02.278 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:02.279 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:55:02.279 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:55:02.279 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1075 2024-05-08T08:55:02.279 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.279 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:02.279 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.279 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:02.279 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:55:02.279 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:02.279 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.279 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.279 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:02.279 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:55:02.279 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:02.279 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.279 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.280 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:02.280 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:55:02.280 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.32e-07 2024-05-08T08:55:02.280 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.280 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.280 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:02.280 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485518+0000", 2024-05-08T08:55:02.280 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3769e-05 2024-05-08T08:55:02.280 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.280 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.280 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:02.280 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485527+0000", 2024-05-08T08:55:02.280 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6779999999999995e-06 2024-05-08T08:55:02.280 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.281 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.281 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:02.281 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485532+0000", 2024-05-08T08:55:02.281 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9830000000000004e-06 2024-05-08T08:55:02.281 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.281 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.281 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:02.281 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556340+0000", 2024-05-08T08:55:02.281 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070807743000000006 2024-05-08T08:55:02.281 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.281 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.281 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:02.281 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556435+0000", 2024-05-08T08:55:02.281 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4732e-05 2024-05-08T08:55:02.281 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.282 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.282 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:02.282 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556549+0000", 2024-05-08T08:55:02.282 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011369 2024-05-08T08:55:02.282 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.282 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:02.282 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.282 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.282 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.282 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1078 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11149312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:55:02.282 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:55:02.282 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 145.78000489300001, 2024-05-08T08:55:02.282 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071167276000000002, 2024-05-08T08:55:02.282 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:02.282 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:02.283 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:02.283 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:02.283 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:55:02.283 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:55:02.283 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1078 2024-05-08T08:55:02.283 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.283 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:02.283 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.283 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:02.283 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:55:02.283 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:02.283 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.283 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.283 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:02.283 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:55:02.283 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:02.283 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.284 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.284 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:02.284 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:55:02.284 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5600000000000001e-07 2024-05-08T08:55:02.284 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.284 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.284 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:02.284 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485547+0000", 2024-05-08T08:55:02.284 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3590000000000002e-06 2024-05-08T08:55:02.284 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.284 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.284 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:02.284 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485553+0000", 2024-05-08T08:55:02.284 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2770000000000002e-06 2024-05-08T08:55:02.284 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.284 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.284 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:02.285 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485558+0000", 2024-05-08T08:55:02.285 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.95e-06 2024-05-08T08:55:02.285 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.285 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.285 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:02.285 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556582+0000", 2024-05-08T08:55:02.285 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071024444000000006 2024-05-08T08:55:02.285 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.285 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.285 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:02.285 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556637+0000", 2024-05-08T08:55:02.285 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4672000000000001e-05 2024-05-08T08:55:02.285 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.285 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.285 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:02.286 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556711+0000", 2024-05-08T08:55:02.286 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4117999999999997e-05 2024-05-08T08:55:02.286 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.286 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:02.286 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.286 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.286 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.286 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1080 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11190272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:55:02.286 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:55:02.286 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 145.77998416, 2024-05-08T08:55:02.286 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071350721000000006, 2024-05-08T08:55:02.286 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:02.286 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:02.286 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:02.287 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:02.287 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:55:02.287 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:55:02.287 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1080 2024-05-08T08:55:02.287 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.287 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:02.287 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.287 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:02.287 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:55:02.287 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:02.287 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.287 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.287 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:02.287 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:55:02.287 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:02.287 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.288 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.288 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:02.288 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:55:02.288 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.03e-07 2024-05-08T08:55:02.288 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.288 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.288 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:02.288 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485567+0000", 2024-05-08T08:55:02.288 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1090000000000001e-06 2024-05-08T08:55:02.288 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.288 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.288 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:02.288 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485573+0000", 2024-05-08T08:55:02.288 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3049999999999998e-06 2024-05-08T08:55:02.289 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.289 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.289 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:02.289 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485578+0000", 2024-05-08T08:55:02.289 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0130000000000001e-06 2024-05-08T08:55:02.289 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.289 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.289 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:02.289 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556743+0000", 2024-05-08T08:55:02.289 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071164449000000005 2024-05-08T08:55:02.289 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.289 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.289 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:02.289 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556797+0000", 2024-05-08T08:55:02.290 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4589999999999997e-05 2024-05-08T08:55:02.290 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.290 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.290 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:02.290 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556915+0000", 2024-05-08T08:55:02.290 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000117852 2024-05-08T08:55:02.290 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.290 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:02.290 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.290 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.290 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.290 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1082 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11231232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:55:02.290 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:55:02.290 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 145.779963917, 2024-05-08T08:55:02.291 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071591396000000002, 2024-05-08T08:55:02.291 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:02.291 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:02.291 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:02.291 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:02.291 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:55:02.291 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:55:02.291 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1082 2024-05-08T08:55:02.291 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.291 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:02.291 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.291 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:02.291 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:55:02.291 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:02.291 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.292 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.292 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:02.292 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:55:02.292 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:02.292 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.292 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.292 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:02.292 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:55:02.292 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1800000000000001e-07 2024-05-08T08:55:02.292 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.292 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.292 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:02.292 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485587+0000", 2024-05-08T08:55:02.292 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2589999999999999e-06 2024-05-08T08:55:02.292 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.292 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.293 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:02.293 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485594+0000", 2024-05-08T08:55:02.293 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0789999999999997e-06 2024-05-08T08:55:02.293 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.293 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.293 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:02.293 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485598+0000", 2024-05-08T08:55:02.293 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7480000000000001e-06 2024-05-08T08:55:02.293 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.293 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.293 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:02.293 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556971+0000", 2024-05-08T08:55:02.293 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071372774999999999 2024-05-08T08:55:02.293 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.293 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.293 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:02.294 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557069+0000", 2024-05-08T08:55:02.294 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.7732999999999994e-05 2024-05-08T08:55:02.294 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.294 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.294 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:02.294 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557176+0000", 2024-05-08T08:55:02.294 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000107384 2024-05-08T08:55:02.294 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.294 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:02.294 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.294 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.294 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.294 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1084 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11272192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:55:02.294 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:55:02.295 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 145.77992527800001, 2024-05-08T08:55:02.295 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071808977999999996, 2024-05-08T08:55:02.295 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:02.295 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:02.295 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:02.295 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:02.295 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:55:02.295 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:55:02.295 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1084 2024-05-08T08:55:02.295 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.295 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:02.295 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.295 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:02.295 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:55:02.295 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:02.296 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.296 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.296 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:02.296 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:55:02.296 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:02.296 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.296 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.296 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:02.296 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485624+0000", 2024-05-08T08:55:02.296 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:55:02.296 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.296 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.296 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:02.296 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485626+0000", 2024-05-08T08:55:02.296 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9460000000000001e-06 2024-05-08T08:55:02.296 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.297 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.297 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:02.297 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485633+0000", 2024-05-08T08:55:02.297 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1149999999999996e-06 2024-05-08T08:55:02.297 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.297 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.297 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:02.297 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485638+0000", 2024-05-08T08:55:02.297 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5670000000000002e-06 2024-05-08T08:55:02.297 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.297 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.297 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:02.297 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557226+0000", 2024-05-08T08:55:02.297 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071587267999999996 2024-05-08T08:55:02.297 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.297 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.298 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:02.298 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557321+0000", 2024-05-08T08:55:02.298 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5785000000000005e-05 2024-05-08T08:55:02.298 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.298 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.298 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:02.298 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557432+0000", 2024-05-08T08:55:02.298 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011092800000000001 2024-05-08T08:55:02.298 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.298 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:02.298 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.298 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.298 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.299 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1086 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11313152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:55:02.299 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:55:02.299 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 145.779901951, 2024-05-08T08:55:02.299 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071933132999999996, 2024-05-08T08:55:02.299 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:02.299 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:02.299 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:02.299 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:02.299 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:55:02.299 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:55:02.299 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1086 2024-05-08T08:55:02.299 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.299 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:02.299 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.300 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:02.300 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:55:02.300 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:02.300 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.300 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.300 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:02.300 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:55:02.300 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:02.300 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.300 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.300 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:02.300 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486328+0000", 2024-05-08T08:55:02.300 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00068107900000000002 2024-05-08T08:55:02.300 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.300 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.301 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:02.301 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486331+0000", 2024-05-08T08:55:02.301 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.585e-06 2024-05-08T08:55:02.301 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.301 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.301 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:02.301 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486343+0000", 2024-05-08T08:55:02.301 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1856e-05 2024-05-08T08:55:02.301 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.301 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.301 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:02.301 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486350+0000", 2024-05-08T08:55:02.301 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8600000000000004e-06 2024-05-08T08:55:02.301 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.301 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.301 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:02.302 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557470+0000", 2024-05-08T08:55:02.302 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071120326999999997 2024-05-08T08:55:02.302 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.302 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.302 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:02.302 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557514+0000", 2024-05-08T08:55:02.302 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3909000000000003e-05 2024-05-08T08:55:02.302 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.302 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.302 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:02.302 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557580+0000", 2024-05-08T08:55:02.302 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5517000000000003e-05 2024-05-08T08:55:02.302 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.302 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:02.303 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.303 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.303 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.303 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1089 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11354112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:55:02.303 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:55:02.303 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 145.77918907899999, 2024-05-08T08:55:02.303 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071457381, 2024-05-08T08:55:02.303 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:02.303 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:02.303 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:02.303 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:02.303 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:55:02.303 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:55:02.303 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1089 2024-05-08T08:55:02.303 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.304 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:02.304 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.304 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:02.304 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:55:02.304 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:02.304 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.304 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.304 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:02.304 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:55:02.304 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:02.304 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.304 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.304 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:02.304 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486851+0000", 2024-05-08T08:55:02.304 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00049102 2024-05-08T08:55:02.305 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.305 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.305 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:02.305 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486854+0000", 2024-05-08T08:55:02.305 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6859999999999999e-06 2024-05-08T08:55:02.305 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.305 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.305 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:02.305 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486864+0000", 2024-05-08T08:55:02.305 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6080000000000002e-06 2024-05-08T08:55:02.305 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.305 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.305 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:02.305 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486870+0000", 2024-05-08T08:55:02.305 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.477e-06 2024-05-08T08:55:02.305 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.306 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.306 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:02.306 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557612+0000", 2024-05-08T08:55:02.306 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070741557999999996 2024-05-08T08:55:02.306 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.306 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.306 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:02.306 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557695+0000", 2024-05-08T08:55:02.306 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3300000000000005e-05 2024-05-08T08:55:02.306 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.306 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.306 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:02.306 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557817+0000", 2024-05-08T08:55:02.306 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012173200000000001 2024-05-08T08:55:02.306 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.307 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:02.307 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.307 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.307 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.307 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1091 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11395072~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:55:02.307 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:55:02.307 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 145.77866828399999, 2024-05-08T08:55:02.307 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071184737999999997, 2024-05-08T08:55:02.307 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:02.307 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:02.307 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:02.307 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:02.307 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:55:02.307 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:55:02.307 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1091 2024-05-08T08:55:02.307 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.308 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:02.308 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.308 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:02.308 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:55:02.308 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:02.308 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.308 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.308 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:02.308 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:55:02.308 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:02.308 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.308 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.308 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:02.308 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486881+0000", 2024-05-08T08:55:02.308 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0699999999999997e-07 2024-05-08T08:55:02.309 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.309 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.309 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:02.309 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486883+0000", 2024-05-08T08:55:02.309 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.402e-06 2024-05-08T08:55:02.309 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.309 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.309 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:02.309 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486890+0000", 2024-05-08T08:55:02.309 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0480000000000001e-06 2024-05-08T08:55:02.309 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.309 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.309 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:02.309 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486896+0000", 2024-05-08T08:55:02.309 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7640000000000002e-06 2024-05-08T08:55:02.310 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.310 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.310 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:02.310 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557860+0000", 2024-05-08T08:55:02.310 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070963791999999998 2024-05-08T08:55:02.310 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.310 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.310 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:02.310 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557952+0000", 2024-05-08T08:55:02.310 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2082000000000004e-05 2024-05-08T08:55:02.310 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.310 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.310 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:02.310 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558065+0000", 2024-05-08T08:55:02.310 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000113143 2024-05-08T08:55:02.311 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.311 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:02.311 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.311 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.311 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.311 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1094 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11436032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:55:02.311 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:55:02.311 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 145.77864511600001, 2024-05-08T08:55:02.311 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071400507000000002, 2024-05-08T08:55:02.311 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:02.311 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:02.311 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:02.311 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:02.311 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:55:02.311 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:55:02.311 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1094 2024-05-08T08:55:02.312 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.312 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:02.312 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.312 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:02.312 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:55:02.312 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:02.312 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.312 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.312 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:02.312 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:55:02.312 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:02.312 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.312 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.312 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:02.312 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487940+0000", 2024-05-08T08:55:02.313 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001036868 2024-05-08T08:55:02.313 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.313 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.313 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:02.313 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487944+0000", 2024-05-08T08:55:02.313 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7069999999999998e-06 2024-05-08T08:55:02.313 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.313 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.313 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:02.313 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487951+0000", 2024-05-08T08:55:02.313 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2649999999999999e-06 2024-05-08T08:55:02.313 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.313 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.313 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:02.313 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487961+0000", 2024-05-08T08:55:02.314 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4110000000000002e-06 2024-05-08T08:55:02.314 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.314 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.314 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:02.314 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558117+0000", 2024-05-08T08:55:02.314 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070156662999999994 2024-05-08T08:55:02.314 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.314 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.314 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:02.314 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558214+0000", 2024-05-08T08:55:02.314 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6444e-05 2024-05-08T08:55:02.314 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.314 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.314 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:02.315 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558304+0000", 2024-05-08T08:55:02.315 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0148999999999994e-05 2024-05-08T08:55:02.315 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.315 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:02.315 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.315 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.315 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.315 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1096 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11476992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:55:02.315 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:55:02.315 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 145.77757858499999, 2024-05-08T08:55:02.315 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070480074000000004, 2024-05-08T08:55:02.315 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:02.315 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:02.315 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:02.315 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:02.316 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:55:02.316 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:55:02.316 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1096 2024-05-08T08:55:02.316 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.316 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:02.316 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.316 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:02.316 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:55:02.316 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:02.316 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.316 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.316 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:02.316 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:55:02.316 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:02.316 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.316 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.317 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:02.317 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:55:02.317 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3700000000000001e-07 2024-05-08T08:55:02.317 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.317 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.317 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:02.317 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487973+0000", 2024-05-08T08:55:02.317 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0870000000000001e-06 2024-05-08T08:55:02.317 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.317 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.317 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:02.317 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487977+0000", 2024-05-08T08:55:02.317 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0600000000000001e-06 2024-05-08T08:55:02.317 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.317 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.317 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:02.318 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487980+0000", 2024-05-08T08:55:02.318 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1089999999999998e-06 2024-05-08T08:55:02.318 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.318 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.318 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:02.318 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558338+0000", 2024-05-08T08:55:02.318 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070358233000000006 2024-05-08T08:55:02.318 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.318 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.318 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:02.318 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558392+0000", 2024-05-08T08:55:02.318 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4537999999999999e-05 2024-05-08T08:55:02.318 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.318 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.319 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:02.319 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558450+0000", 2024-05-08T08:55:02.319 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7710000000000001e-05 2024-05-08T08:55:02.319 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.319 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:02.319 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.319 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.319 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.319 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1099 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11517952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:55:02.319 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:55:02.319 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 145.77756470899999, 2024-05-08T08:55:02.319 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070720506000000002, 2024-05-08T08:55:02.319 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:02.319 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:02.319 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:02.319 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:02.320 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:55:02.320 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:55:02.320 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1099 2024-05-08T08:55:02.320 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.320 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:02.320 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.320 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:02.320 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:55:02.320 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:02.320 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.320 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.320 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:02.320 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:55:02.320 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:02.320 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.320 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.321 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:02.321 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:55:02.321 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3999999999999998e-07 2024-05-08T08:55:02.321 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.321 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.321 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:02.321 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487986+0000", 2024-05-08T08:55:02.321 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3519999999999999e-06 2024-05-08T08:55:02.321 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.321 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.321 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:02.321 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487989+0000", 2024-05-08T08:55:02.321 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.23e-06 2024-05-08T08:55:02.321 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.322 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.322 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:02.322 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487991+0000", 2024-05-08T08:55:02.322 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6350000000000002e-06 2024-05-08T08:55:02.322 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.322 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.322 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:02.322 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558489+0000", 2024-05-08T08:55:02.322 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070497924000000003 2024-05-08T08:55:02.322 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.322 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.322 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:02.322 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558583+0000", 2024-05-08T08:55:02.322 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3405000000000004e-05 2024-05-08T08:55:02.323 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.323 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.323 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:02.323 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558704+0000", 2024-05-08T08:55:02.323 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012171999999999999 2024-05-08T08:55:02.323 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.323 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:02.323 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.323 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.323 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.323 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1100 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11558912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:55:02.323 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:55:02.323 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 145.77755171000001, 2024-05-08T08:55:02.323 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070962266999999996, 2024-05-08T08:55:02.323 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:02.324 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:02.324 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:02.324 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:02.324 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:55:02.324 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:55:02.324 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1100 2024-05-08T08:55:02.324 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.324 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:02.324 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.324 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:02.324 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:55:02.324 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:02.324 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.324 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.324 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:02.324 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:55:02.325 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:02.325 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.325 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.325 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:02.325 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488430+0000", 2024-05-08T08:55:02.325 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000432676 2024-05-08T08:55:02.325 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.325 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.325 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:02.325 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488433+0000", 2024-05-08T08:55:02.325 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.45e-06 2024-05-08T08:55:02.325 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.325 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.325 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:02.325 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488442+0000", 2024-05-08T08:55:02.325 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.687e-06 2024-05-08T08:55:02.326 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.326 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.326 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:02.326 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488446+0000", 2024-05-08T08:55:02.326 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4939999999999997e-06 2024-05-08T08:55:02.326 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.326 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.326 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:02.326 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558756+0000", 2024-05-08T08:55:02.326 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070310051999999998 2024-05-08T08:55:02.326 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.326 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.326 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:02.326 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558844+0000", 2024-05-08T08:55:02.327 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7690999999999998e-05 2024-05-08T08:55:02.327 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.327 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.327 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:02.327 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558959+0000", 2024-05-08T08:55:02.327 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000115217 2024-05-08T08:55:02.327 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.327 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:02.327 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.327 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.327 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.327 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1103 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11599872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:55:02.327 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:55:02.327 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 145.777094035, 2024-05-08T08:55:02.327 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070777100999999995, 2024-05-08T08:55:02.327 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:02.327 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:02.328 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:02.328 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:02.328 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:55:02.328 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:55:02.328 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1103 2024-05-08T08:55:02.328 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.328 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:02.328 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.328 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:02.328 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:55:02.328 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:02.328 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.328 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.328 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:02.328 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:55:02.328 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:02.329 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.329 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.329 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:02.329 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488959+0000", 2024-05-08T08:55:02.329 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00050457200000000001 2024-05-08T08:55:02.329 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.329 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.329 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:02.329 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488969+0000", 2024-05-08T08:55:02.329 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.7159999999999998e-06 2024-05-08T08:55:02.329 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.329 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.329 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:02.329 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488981+0000", 2024-05-08T08:55:02.329 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2150000000000001e-05 2024-05-08T08:55:02.330 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.330 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.330 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:02.330 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488988+0000", 2024-05-08T08:55:02.330 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8759999999999999e-06 2024-05-08T08:55:02.330 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.330 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.330 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:02.330 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558992+0000", 2024-05-08T08:55:02.330 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070004358000000003 2024-05-08T08:55:02.330 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.330 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.330 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:02.330 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559093+0000", 2024-05-08T08:55:02.330 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000100299 2024-05-08T08:55:02.331 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.331 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.331 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:02.331 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559232+0000", 2024-05-08T08:55:02.331 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013913 2024-05-08T08:55:02.331 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.331 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:02.331 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.331 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.331 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.331 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1104 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11640832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:55:02.331 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:55:02.331 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 145.77654921999999, 2024-05-08T08:55:02.331 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070521947000000001, 2024-05-08T08:55:02.331 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:02.332 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:02.332 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:02.332 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:02.332 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:55:02.332 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:55:02.332 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1104 2024-05-08T08:55:02.332 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.332 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:02.332 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.332 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:02.332 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:55:02.332 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:02.332 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.332 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.332 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:02.333 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:55:02.333 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:02.333 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.333 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.333 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:02.333 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489700+0000", 2024-05-08T08:55:02.333 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00070067600000000001 2024-05-08T08:55:02.333 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.333 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.333 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:02.333 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489704+0000", 2024-05-08T08:55:02.333 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1640000000000003e-06 2024-05-08T08:55:02.333 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.333 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.333 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:02.334 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489716+0000", 2024-05-08T08:55:02.334 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1379000000000001e-05 2024-05-08T08:55:02.334 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.334 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.334 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:02.334 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489724+0000", 2024-05-08T08:55:02.334 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8949999999999998e-06 2024-05-08T08:55:02.334 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.334 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.334 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:02.334 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559278+0000", 2024-05-08T08:55:02.334 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.069554151999999994 2024-05-08T08:55:02.334 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.334 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.334 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:02.335 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559387+0000", 2024-05-08T08:55:02.335 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000109258 2024-05-08T08:55:02.335 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.335 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.335 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:02.335 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559521+0000", 2024-05-08T08:55:02.335 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000134423 2024-05-08T08:55:02.335 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.335 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:02.335 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.335 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.335 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.335 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:8636 3.17 3:e8a0a3e4:::rbd_header.11402b26385:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-05-08T08:55:02.335 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:54.150971+0000", 2024-05-08T08:55:02.335 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 128.11457722599999, 2024-05-08T08:55:02.336 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.217273233, 2024-05-08T08:55:02.336 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:02.336 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:02.336 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:02.336 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:02.336 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:55:02.336 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:55:02.336 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 8636 2024-05-08T08:55:02.336 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.336 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:02.336 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.336 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:02.336 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150971+0000", 2024-05-08T08:55:02.336 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:02.336 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.336 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.337 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:02.337 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150971+0000", 2024-05-08T08:55:02.337 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:02.337 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.337 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.337 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:02.337 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150972+0000", 2024-05-08T08:55:02.337 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.71e-07 2024-05-08T08:55:02.337 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.337 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.337 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:02.337 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150977+0000", 2024-05-08T08:55:02.337 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.152e-06 2024-05-08T08:55:02.337 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.337 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.338 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:02.338 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150990+0000", 2024-05-08T08:55:02.338 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2364999999999999e-05 2024-05-08T08:55:02.338 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.338 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.338 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:02.338 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150998+0000", 2024-05-08T08:55:02.338 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0739999999999993e-06 2024-05-08T08:55:02.338 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.338 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.338 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:02.338 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.151054+0000", 2024-05-08T08:55:02.338 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5949000000000003e-05 2024-05-08T08:55:02.338 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.338 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.339 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:55:02.339 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.151060+0000", 2024-05-08T08:55:02.339 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6120000000000004e-06 2024-05-08T08:55:02.339 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.339 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.339 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:02.339 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.367640+0000", 2024-05-08T08:55:02.339 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2165792739999999 2024-05-08T08:55:02.339 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.339 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.339 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:02.339 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.367839+0000", 2024-05-08T08:55:02.339 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000199185 2024-05-08T08:55:02.339 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.339 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.340 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:02.340 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.368245+0000", 2024-05-08T08:55:02.340 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00040575099999999998 2024-05-08T08:55:02.340 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.340 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:02.340 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.340 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.340 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.340 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5302.0:1516 3.c 3:319bf789:::rbd_data.1140d3fc54c.0000000000000025:head [stat,write 1892352~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:55:02.340 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:27.822220+0000", 2024-05-08T08:55:02.340 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 94.443328288999993, 2024-05-08T08:55:02.340 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.092643744, 2024-05-08T08:55:02.340 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:02.340 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:02.340 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:55:02.341 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:02.341 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5302", 2024-05-08T08:55:02.341 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/359277859", 2024-05-08T08:55:02.341 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1516 2024-05-08T08:55:02.341 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.341 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:02.341 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.341 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:02.341 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822220+0000", 2024-05-08T08:55:02.341 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:02.341 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.341 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.341 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:02.341 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822220+0000", 2024-05-08T08:55:02.342 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:02.342 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.342 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.342 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:02.342 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822221+0000", 2024-05-08T08:55:02.342 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3500000000000002e-07 2024-05-08T08:55:02.342 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.342 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.342 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:02.342 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822228+0000", 2024-05-08T08:55:02.342 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9959999999999996e-06 2024-05-08T08:55:02.342 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.342 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.342 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:02.342 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822234+0000", 2024-05-08T08:55:02.343 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5429999999999999e-06 2024-05-08T08:55:02.343 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.343 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.343 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:02.343 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822238+0000", 2024-05-08T08:55:02.343 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.732e-06 2024-05-08T08:55:02.343 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.343 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.343 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:02.343 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822285+0000", 2024-05-08T08:55:02.343 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6810000000000001e-05 2024-05-08T08:55:02.343 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.343 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.343 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:02.344 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822358+0000", 2024-05-08T08:55:02.344 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3473000000000006e-05 2024-05-08T08:55:02.344 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.344 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.344 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 5", 2024-05-08T08:55:02.344 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822489+0000", 2024-05-08T08:55:02.344 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013011199999999999 2024-05-08T08:55:02.344 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.344 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.344 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:55:02.344 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.825332+0000", 2024-05-08T08:55:02.344 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0028434799999999998 2024-05-08T08:55:02.344 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.344 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.344 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:55:02.345 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.914753+0000", 2024-05-08T08:55:02.345 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089421424999999999 2024-05-08T08:55:02.345 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.345 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.345 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:55:02.345 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.914807+0000", 2024-05-08T08:55:02.345 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3829000000000002e-05 2024-05-08T08:55:02.345 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.345 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.345 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:02.345 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.914864+0000", 2024-05-08T08:55:02.345 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6909e-05 2024-05-08T08:55:02.345 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.345 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:02.346 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.346 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.346 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.346 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:24567 3.6 3:66af1c40:::rbd_data.1140d3fc54c.000000000000005a:18 [read 2080768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:02.346 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.216598+0000", 2024-05-08T08:55:02.346 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 90.048950208999997, 2024-05-08T08:55:02.346 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071602647000000005, 2024-05-08T08:55:02.346 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:02.346 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:02.346 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:02.346 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:02.346 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:02.346 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:02.346 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 24567 2024-05-08T08:55:02.346 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.347 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:02.347 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.347 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:02.347 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216598+0000", 2024-05-08T08:55:02.347 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:02.347 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.347 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.347 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:02.347 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216598+0000", 2024-05-08T08:55:02.347 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:02.347 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.347 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.347 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:02.347 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216599+0000", 2024-05-08T08:55:02.347 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8999999999999997e-07 2024-05-08T08:55:02.348 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.348 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.348 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:02.348 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216601+0000", 2024-05-08T08:55:02.348 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9889999999999999e-06 2024-05-08T08:55:02.348 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.348 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.348 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:02.348 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216607+0000", 2024-05-08T08:55:02.348 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3600000000000001e-06 2024-05-08T08:55:02.348 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.348 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.348 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:02.348 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216612+0000", 2024-05-08T08:55:02.348 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.69e-06 2024-05-08T08:55:02.348 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.349 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.349 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:02.349 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.287076+0000", 2024-05-08T08:55:02.349 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070464355000000006 2024-05-08T08:55:02.349 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.349 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.349 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:02.349 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.287151+0000", 2024-05-08T08:55:02.349 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4924000000000003e-05 2024-05-08T08:55:02.349 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.349 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.349 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:02.349 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.288201+0000", 2024-05-08T08:55:02.349 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0010498389999999999 2024-05-08T08:55:02.349 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.350 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:02.350 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.350 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.350 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:02.350 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:55:02.351 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:55:02.352 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:55:02.354+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:55:02.407 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-05-08T08:55:02.452 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:55:02.455+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:55:02.470 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:55:02.470 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:55:02.470 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:55:02.470 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:55:02.480 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:55:02.553 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:55:02.556+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:55:02.602 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:55:02.602 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:55:02.602 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:55:02.602 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:55:02.602 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:55:02.611 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:55:02.654 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:55:02.657+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:55:02.723 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:55:02.723 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:55:02.723 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:55:02.723 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:55:02.724 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.724 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:12443 3.0 3:02d0158e:::rbd_data.114071bcf828.0000000000000037:head [stat,delete] snapc 19=[19] RETRY=3 ondisk+retry+write+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:02.724 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.610027+0000", 2024-05-08T08:55:02.724 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 91.114606163000005, 2024-05-08T08:55:02.724 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.083586333999999998, 2024-05-08T08:55:02.724 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:02.724 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:02.724 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:02.724 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:02.724 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:02.724 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:02.724 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 12443 2024-05-08T08:55:02.724 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.724 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:02.724 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.725 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:02.725 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610027+0000", 2024-05-08T08:55:02.725 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:02.725 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.725 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.725 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:02.725 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610027+0000", 2024-05-08T08:55:02.725 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:02.725 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.725 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.725 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:02.725 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610028+0000", 2024-05-08T08:55:02.725 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.13e-07 2024-05-08T08:55:02.725 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.725 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.725 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:02.726 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610030+0000", 2024-05-08T08:55:02.726 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7669999999999999e-06 2024-05-08T08:55:02.726 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.726 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.726 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:02.726 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610039+0000", 2024-05-08T08:55:02.726 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4959999999999996e-06 2024-05-08T08:55:02.726 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.726 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.726 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:02.726 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610045+0000", 2024-05-08T08:55:02.726 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6779999999999997e-06 2024-05-08T08:55:02.726 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.726 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.726 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:02.727 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.660873+0000", 2024-05-08T08:55:02.727 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.050828748 2024-05-08T08:55:02.727 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.727 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.727 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:02.727 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.661017+0000", 2024-05-08T08:55:02.727 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014391999999999999 2024-05-08T08:55:02.727 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.727 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.727 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:02.727 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.693613+0000", 2024-05-08T08:55:02.727 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.032596111999999997 2024-05-08T08:55:02.727 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.727 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:02.727 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.728 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.728 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.728 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:14301 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [watch reconnect cookie 139837155901536 gen 12] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:02.728 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.645078+0000", 2024-05-08T08:55:02.728 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 91.079555525000004, 2024-05-08T08:55:02.728 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088281852999999993, 2024-05-08T08:55:02.728 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:02.728 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:02.728 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:02.728 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:02.728 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:02.728 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:02.728 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14301 2024-05-08T08:55:02.728 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.728 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:02.728 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.729 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:02.729 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645078+0000", 2024-05-08T08:55:02.729 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:02.729 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.729 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.729 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:02.729 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645078+0000", 2024-05-08T08:55:02.729 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:02.729 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.729 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.729 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:02.729 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645084+0000", 2024-05-08T08:55:02.729 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4620000000000001e-06 2024-05-08T08:55:02.729 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.729 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.729 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:02.730 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645088+0000", 2024-05-08T08:55:02.730 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8039999999999999e-06 2024-05-08T08:55:02.730 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.730 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.730 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:02.730 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645106+0000", 2024-05-08T08:55:02.730 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.8318999999999999e-05 2024-05-08T08:55:02.730 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.730 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.730 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:02.730 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645113+0000", 2024-05-08T08:55:02.730 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0310000000000001e-06 2024-05-08T08:55:02.730 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.730 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.730 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:02.731 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.670493+0000", 2024-05-08T08:55:02.731 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.025379993 2024-05-08T08:55:02.731 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.731 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.731 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:02.731 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.670571+0000", 2024-05-08T08:55:02.731 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7613000000000004e-05 2024-05-08T08:55:02.731 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.731 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.731 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:02.731 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733360+0000", 2024-05-08T08:55:02.731 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.062788630999999998 2024-05-08T08:55:02.731 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.731 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:02.731 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.731 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.732 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.732 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:14420 3.0 3:06a8948e:::rbd_data.114071bcf828.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:02.732 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.649639+0000", 2024-05-08T08:55:02.732 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 91.074994376999996, 2024-05-08T08:55:02.732 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.083851897999999994, 2024-05-08T08:55:02.732 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:02.732 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:02.732 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:55:02.732 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:02.732 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:02.732 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:02.732 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14420 2024-05-08T08:55:02.732 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.732 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:02.732 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.732 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:02.733 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649639+0000", 2024-05-08T08:55:02.733 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:02.733 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.733 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.733 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:02.733 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649639+0000", 2024-05-08T08:55:02.733 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:02.733 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.733 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.733 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:02.733 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649640+0000", 2024-05-08T08:55:02.733 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6899999999999997e-07 2024-05-08T08:55:02.733 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.733 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.733 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:02.733 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649895+0000", 2024-05-08T08:55:02.734 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00025531000000000002 2024-05-08T08:55:02.734 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.734 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.734 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:02.734 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649905+0000", 2024-05-08T08:55:02.734 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0198e-05 2024-05-08T08:55:02.734 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.734 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.734 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:02.734 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649911+0000", 2024-05-08T08:55:02.734 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2310000000000001e-06 2024-05-08T08:55:02.734 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.734 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.734 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:02.734 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.684099+0000", 2024-05-08T08:55:02.735 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.034188017000000001 2024-05-08T08:55:02.735 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.735 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.735 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:02.735 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.684172+0000", 2024-05-08T08:55:02.735 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3201999999999995e-05 2024-05-08T08:55:02.735 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.735 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.735 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 2", 2024-05-08T08:55:02.735 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.684418+0000", 2024-05-08T08:55:02.735 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00024559099999999997 2024-05-08T08:55:02.735 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.735 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.735 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:55:02.735 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.705538+0000", 2024-05-08T08:55:02.735 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.021119707000000001 2024-05-08T08:55:02.736 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.736 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.736 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:55:02.736 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733399+0000", 2024-05-08T08:55:02.736 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.027861543999999999 2024-05-08T08:55:02.736 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.736 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.736 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:55:02.736 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733445+0000", 2024-05-08T08:55:02.736 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5587000000000002e-05 2024-05-08T08:55:02.736 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.736 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.736 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:02.736 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733491+0000", 2024-05-08T08:55:02.736 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5841999999999999e-05 2024-05-08T08:55:02.736 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.737 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:02.737 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.737 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.737 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.737 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28547 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889179216] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:02.737 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.355654+0000", 2024-05-08T08:55:02.737 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 90.368979148999998, 2024-05-08T08:55:02.737 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078692261999999999, 2024-05-08T08:55:02.737 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:02.737 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:02.737 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:02.737 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:02.737 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:02.737 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:02.737 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28547 2024-05-08T08:55:02.737 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.737 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:02.738 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.738 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:02.738 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355654+0000", 2024-05-08T08:55:02.738 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:02.738 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.738 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.738 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:02.738 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355654+0000", 2024-05-08T08:55:02.738 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:02.738 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.738 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.738 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:02.738 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355654+0000", 2024-05-08T08:55:02.738 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3999999999999998e-07 2024-05-08T08:55:02.739 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.739 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.739 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:02.739 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355656+0000", 2024-05-08T08:55:02.739 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.353e-06 2024-05-08T08:55:02.739 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.739 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.739 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:02.739 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355663+0000", 2024-05-08T08:55:02.739 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2259999999999999e-06 2024-05-08T08:55:02.739 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.739 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.739 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:02.739 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355666+0000", 2024-05-08T08:55:02.739 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7930000000000002e-06 2024-05-08T08:55:02.739 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.740 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.740 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:02.740 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434117+0000", 2024-05-08T08:55:02.740 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078451231999999996 2024-05-08T08:55:02.740 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.740 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.740 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:02.740 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434198+0000", 2024-05-08T08:55:02.740 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0879999999999998e-05 2024-05-08T08:55:02.740 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.740 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.740 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:02.740 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434346+0000", 2024-05-08T08:55:02.740 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014853800000000001 2024-05-08T08:55:02.740 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.740 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:02.741 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.741 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.741 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.741 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28548 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889189696] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:02.741 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.355669+0000", 2024-05-08T08:55:02.741 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 90.368963798999999, 2024-05-08T08:55:02.741 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078956606999999998, 2024-05-08T08:55:02.741 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:02.741 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:02.741 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:02.741 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:02.741 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:02.741 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:02.741 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28548 2024-05-08T08:55:02.741 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.741 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:02.741 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.742 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:02.742 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355669+0000", 2024-05-08T08:55:02.742 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:02.742 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.742 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.742 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:02.742 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355669+0000", 2024-05-08T08:55:02.742 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:02.742 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.742 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.742 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:02.742 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355762+0000", 2024-05-08T08:55:02.742 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3015000000000003e-05 2024-05-08T08:55:02.742 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.742 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.743 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:02.743 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355765+0000", 2024-05-08T08:55:02.743 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.368e-06 2024-05-08T08:55:02.743 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.743 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.743 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:02.743 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355771+0000", 2024-05-08T08:55:02.743 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5989999999999998e-06 2024-05-08T08:55:02.743 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.743 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.743 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:02.743 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355775+0000", 2024-05-08T08:55:02.743 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7280000000000001e-06 2024-05-08T08:55:02.743 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.743 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.743 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:02.744 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434379+0000", 2024-05-08T08:55:02.744 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078603662000000005 2024-05-08T08:55:02.744 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.744 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.744 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:02.744 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434473+0000", 2024-05-08T08:55:02.744 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4104000000000006e-05 2024-05-08T08:55:02.744 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.744 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.744 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:02.744 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434626+0000", 2024-05-08T08:55:02.744 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00015313100000000001 2024-05-08T08:55:02.744 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.744 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:02.744 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.745 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.745 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.745 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28549 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889202400] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:02.745 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.355781+0000", 2024-05-08T08:55:02.745 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 90.368852224999998, 2024-05-08T08:55:02.745 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079117946999999994, 2024-05-08T08:55:02.745 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:02.745 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:02.745 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:02.745 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:02.745 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:02.745 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:02.745 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28549 2024-05-08T08:55:02.745 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.745 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:02.746 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.746 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:02.746 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355781+0000", 2024-05-08T08:55:02.746 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:02.746 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.746 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.746 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:02.746 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355781+0000", 2024-05-08T08:55:02.746 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:02.746 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.746 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.746 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:02.746 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356162+0000", 2024-05-08T08:55:02.746 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00038099100000000001 2024-05-08T08:55:02.746 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.747 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.747 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:02.747 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356165+0000", 2024-05-08T08:55:02.747 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1240000000000001e-06 2024-05-08T08:55:02.747 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.747 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.747 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:02.747 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356173+0000", 2024-05-08T08:55:02.747 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.6199999999999999e-06 2024-05-08T08:55:02.747 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.747 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.747 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:02.747 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356182+0000", 2024-05-08T08:55:02.747 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7760000000000003e-06 2024-05-08T08:55:02.747 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.748 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.748 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:02.748 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434675+0000", 2024-05-08T08:55:02.748 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078493119 2024-05-08T08:55:02.748 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.748 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.748 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:02.748 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434765+0000", 2024-05-08T08:55:02.748 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0402000000000006e-05 2024-05-08T08:55:02.748 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.748 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.748 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:02.748 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434899+0000", 2024-05-08T08:55:02.748 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013391499999999999 2024-05-08T08:55:02.748 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.748 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:02.749 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.749 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.749 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.749 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28550 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888597280] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:02.749 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:55:02.749 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 90.368441179000001, 2024-05-08T08:55:02.749 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078933990999999995, 2024-05-08T08:55:02.749 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:02.749 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:02.749 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:02.749 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:02.749 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:02.749 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:02.749 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28550 2024-05-08T08:55:02.749 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.749 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:02.750 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.750 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:02.750 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:55:02.750 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:02.750 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.750 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.750 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:02.750 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:55:02.750 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:02.750 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.750 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.750 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:02.750 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:55:02.750 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.34e-07 2024-05-08T08:55:02.750 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.751 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.751 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:02.751 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356195+0000", 2024-05-08T08:55:02.751 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8710000000000001e-06 2024-05-08T08:55:02.751 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.751 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.751 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:02.751 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356200+0000", 2024-05-08T08:55:02.751 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3000000000000003e-06 2024-05-08T08:55:02.751 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.751 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.751 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:02.751 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356203+0000", 2024-05-08T08:55:02.751 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4039999999999999e-06 2024-05-08T08:55:02.751 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.751 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.751 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:02.752 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434928+0000", 2024-05-08T08:55:02.752 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078725331999999995 2024-05-08T08:55:02.752 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.752 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.752 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:02.752 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435007+0000", 2024-05-08T08:55:02.752 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8415000000000001e-05 2024-05-08T08:55:02.752 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.752 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.752 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:02.752 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435126+0000", 2024-05-08T08:55:02.752 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011923500000000001 2024-05-08T08:55:02.752 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.752 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:02.752 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.753 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.753 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.753 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28551 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888527200] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:02.753 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.356210+0000", 2024-05-08T08:55:02.753 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 90.368422831000004, 2024-05-08T08:55:02.753 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079195072000000005, 2024-05-08T08:55:02.753 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:02.753 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:02.753 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:02.753 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:02.753 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:02.753 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:02.753 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28551 2024-05-08T08:55:02.753 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.753 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:02.753 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.754 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:02.754 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356210+0000", 2024-05-08T08:55:02.754 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:02.754 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.754 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.754 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:02.754 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356210+0000", 2024-05-08T08:55:02.754 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:02.754 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.754 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.754 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:02.754 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356211+0000", 2024-05-08T08:55:02.754 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4499999999999998e-07 2024-05-08T08:55:02.754 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.755 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.755 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:02.755 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356212+0000", 2024-05-08T08:55:02.755 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6029999999999999e-06 2024-05-08T08:55:02.755 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.755 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.755 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:02.755 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356221+0000", 2024-05-08T08:55:02.755 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4200000000000007e-06 2024-05-08T08:55:02.755 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.755 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.755 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:02.755 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356224+0000", 2024-05-08T08:55:02.755 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.038e-06 2024-05-08T08:55:02.755 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.756 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.756 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:02.756 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435178+0000", 2024-05-08T08:55:02.756 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078954625000000001 2024-05-08T08:55:02.756 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.756 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.756 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:02.756 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435256+0000", 2024-05-08T08:55:02.756 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7337999999999997e-05 2024-05-08T08:55:02.756 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.756 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.756 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:02.756 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435405+0000", 2024-05-08T08:55:02.756 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000149703 2024-05-08T08:55:02.756 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.757 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:02.757 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.757 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.757 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.757 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28552 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889231456] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:02.757 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.356228+0000", 2024-05-08T08:55:02.757 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 90.368405152999998, 2024-05-08T08:55:02.757 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079346920000000001, 2024-05-08T08:55:02.757 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:02.757 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:02.757 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:02.757 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:02.757 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:02.757 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:02.757 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28552 2024-05-08T08:55:02.757 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.758 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:02.758 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.758 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:02.758 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356228+0000", 2024-05-08T08:55:02.758 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:02.758 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.758 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.758 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:02.758 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356228+0000", 2024-05-08T08:55:02.758 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:02.758 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.758 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.758 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:02.758 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356959+0000", 2024-05-08T08:55:02.759 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00073097099999999996 2024-05-08T08:55:02.759 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.759 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.759 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:02.759 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356967+0000", 2024-05-08T08:55:02.759 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0409999999999998e-06 2024-05-08T08:55:02.759 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.759 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.759 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:02.759 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356979+0000", 2024-05-08T08:55:02.759 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1489e-05 2024-05-08T08:55:02.759 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.759 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.759 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:02.759 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356986+0000", 2024-05-08T08:55:02.760 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3699999999999997e-06 2024-05-08T08:55:02.760 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.760 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.760 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:02.760 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435433+0000", 2024-05-08T08:55:02.760 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078447520000000007 2024-05-08T08:55:02.760 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.760 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.760 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:02.760 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435484+0000", 2024-05-08T08:55:02.760 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0229000000000003e-05 2024-05-08T08:55:02.760 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.760 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.760 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:02.760 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435575+0000", 2024-05-08T08:55:02.760 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1299999999999997e-05 2024-05-08T08:55:02.761 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.761 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:02.761 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.761 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.761 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.761 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28553 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888713648] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:02.761 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:55:02.761 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 90.367630495, 2024-05-08T08:55:02.761 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078782034000000001, 2024-05-08T08:55:02.761 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:02.761 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:02.761 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:02.761 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:02.761 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:02.762 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:02.762 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28553 2024-05-08T08:55:02.762 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.762 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:02.762 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.762 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:02.762 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:55:02.762 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:02.762 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.762 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.762 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:02.762 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:55:02.762 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:02.762 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.762 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.763 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:02.763 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:55:02.763 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6600000000000002e-07 2024-05-08T08:55:02.763 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.763 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.763 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:02.763 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357006+0000", 2024-05-08T08:55:02.763 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7190000000000001e-06 2024-05-08T08:55:02.763 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.763 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.763 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:02.763 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357013+0000", 2024-05-08T08:55:02.763 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.435e-06 2024-05-08T08:55:02.763 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.763 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.764 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:02.764 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357018+0000", 2024-05-08T08:55:02.764 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7260000000000002e-06 2024-05-08T08:55:02.764 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.764 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.764 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:02.764 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435623+0000", 2024-05-08T08:55:02.764 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078604972999999995 2024-05-08T08:55:02.764 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.764 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.764 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:02.764 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435695+0000", 2024-05-08T08:55:02.764 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1959999999999995e-05 2024-05-08T08:55:02.764 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.764 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.765 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:02.765 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435785+0000", 2024-05-08T08:55:02.765 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.9754999999999997e-05 2024-05-08T08:55:02.765 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.765 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:02.765 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.765 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.765 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.765 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28554 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888980272] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:02.765 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357024+0000", 2024-05-08T08:55:02.765 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 90.367608859000001, 2024-05-08T08:55:02.765 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078999870999999999, 2024-05-08T08:55:02.765 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:02.765 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:02.765 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:02.765 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:02.766 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:02.766 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:02.766 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28554 2024-05-08T08:55:02.766 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.766 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:02.766 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.766 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:02.766 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357024+0000", 2024-05-08T08:55:02.766 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:02.766 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.766 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.766 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:02.766 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357024+0000", 2024-05-08T08:55:02.766 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:02.767 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.767 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.767 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:02.767 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357025+0000", 2024-05-08T08:55:02.767 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:55:02.767 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.767 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.767 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:02.767 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357027+0000", 2024-05-08T08:55:02.767 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2050000000000001e-06 2024-05-08T08:55:02.767 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.767 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.767 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:02.767 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357033+0000", 2024-05-08T08:55:02.767 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5679999999999996e-06 2024-05-08T08:55:02.767 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.768 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.768 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:02.768 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357038+0000", 2024-05-08T08:55:02.768 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2889999999999999e-06 2024-05-08T08:55:02.768 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.768 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.768 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:02.768 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435823+0000", 2024-05-08T08:55:02.768 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078785677999999998 2024-05-08T08:55:02.768 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.768 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.768 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:02.768 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435892+0000", 2024-05-08T08:55:02.768 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8379000000000004e-05 2024-05-08T08:55:02.769 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.769 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.769 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:02.769 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436024+0000", 2024-05-08T08:55:02.769 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013239799999999999 2024-05-08T08:55:02.769 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.769 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:02.769 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.769 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.769 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.769 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28555 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889001104] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:02.769 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:55:02.769 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 90.367590093999993, 2024-05-08T08:55:02.769 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079115742000000003, 2024-05-08T08:55:02.770 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:02.770 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:02.770 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:02.770 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:02.770 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:02.770 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:02.770 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28555 2024-05-08T08:55:02.770 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.770 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:02.770 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.770 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:02.770 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:55:02.770 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:02.770 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.770 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.771 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:02.771 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:55:02.771 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:02.771 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.771 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.771 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:02.771 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:55:02.771 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9200000000000002e-07 2024-05-08T08:55:02.771 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.771 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.771 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:02.771 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357046+0000", 2024-05-08T08:55:02.771 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1670000000000002e-06 2024-05-08T08:55:02.771 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.771 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.772 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:02.772 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357052+0000", 2024-05-08T08:55:02.772 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.139e-06 2024-05-08T08:55:02.772 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.772 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.772 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:02.772 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357056+0000", 2024-05-08T08:55:02.772 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2440000000000004e-06 2024-05-08T08:55:02.772 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.772 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.772 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:02.772 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436059+0000", 2024-05-08T08:55:02.772 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079003070999999994 2024-05-08T08:55:02.772 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.772 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.773 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:02.773 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436097+0000", 2024-05-08T08:55:02.773 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7784000000000003e-05 2024-05-08T08:55:02.773 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.773 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.773 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:02.773 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436159+0000", 2024-05-08T08:55:02.773 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2045000000000007e-05 2024-05-08T08:55:02.773 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.773 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:02.773 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.773 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.773 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.773 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28556 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889022064] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:02.773 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357065+0000", 2024-05-08T08:55:02.774 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 90.367568061, 2024-05-08T08:55:02.774 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079327976999999994, 2024-05-08T08:55:02.774 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:02.774 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:02.774 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:02.774 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:02.774 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:02.774 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:02.774 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28556 2024-05-08T08:55:02.774 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.774 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:02.774 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.775 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:02.775 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357065+0000", 2024-05-08T08:55:02.775 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:02.775 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.775 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.775 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:02.775 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357065+0000", 2024-05-08T08:55:02.775 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:02.775 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.775 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.775 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:02.775 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358958+0000", 2024-05-08T08:55:02.775 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001892915 2024-05-08T08:55:02.775 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.776 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.776 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:02.776 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358962+0000", 2024-05-08T08:55:02.776 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4599999999999999e-06 2024-05-08T08:55:02.776 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.776 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.776 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:02.776 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358970+0000", 2024-05-08T08:55:02.776 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.6090000000000001e-06 2024-05-08T08:55:02.776 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.776 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.776 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:02.776 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358975+0000", 2024-05-08T08:55:02.776 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7639999999999996e-06 2024-05-08T08:55:02.777 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.777 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.777 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:02.777 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436196+0000", 2024-05-08T08:55:02.777 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.077221564000000006 2024-05-08T08:55:02.777 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.777 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.777 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:02.777 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436269+0000", 2024-05-08T08:55:02.777 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2720999999999997e-05 2024-05-08T08:55:02.777 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.777 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.777 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:02.777 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436393+0000", 2024-05-08T08:55:02.778 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012394399999999999 2024-05-08T08:55:02.778 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.778 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:02.778 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.778 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.778 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.778 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28563 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888855520] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:02.778 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:55:02.778 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 90.365558661999998, 2024-05-08T08:55:02.778 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078795566999999997, 2024-05-08T08:55:02.778 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:02.778 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:02.778 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:02.778 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:02.779 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:02.779 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:02.779 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28563 2024-05-08T08:55:02.779 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.779 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:02.779 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.779 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:02.779 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:55:02.779 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:02.779 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.779 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.779 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:02.779 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:55:02.779 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:02.779 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.780 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.780 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:02.780 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:55:02.780 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1e-07 2024-05-08T08:55:02.780 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.780 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.780 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:02.780 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359080+0000", 2024-05-08T08:55:02.780 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8940000000000001e-06 2024-05-08T08:55:02.780 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.780 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.780 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:02.780 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359083+0000", 2024-05-08T08:55:02.780 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1949999999999999e-06 2024-05-08T08:55:02.780 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.781 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.781 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:02.781 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359085+0000", 2024-05-08T08:55:02.781 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4710000000000001e-06 2024-05-08T08:55:02.781 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.781 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.781 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:02.781 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437604+0000", 2024-05-08T08:55:02.781 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078518278999999996 2024-05-08T08:55:02.781 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.781 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.781 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:02.781 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437704+0000", 2024-05-08T08:55:02.781 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010010599999999999 2024-05-08T08:55:02.781 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.781 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.782 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:02.782 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437870+0000", 2024-05-08T08:55:02.782 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00016641200000000001 2024-05-08T08:55:02.782 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.782 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:02.782 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.782 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.782 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.782 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28564 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888538944] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:02.782 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.359089+0000", 2024-05-08T08:55:02.782 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 90.365544447000005, 2024-05-08T08:55:02.782 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078952675, 2024-05-08T08:55:02.782 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:02.782 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:02.783 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:02.783 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:02.783 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:02.783 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:02.783 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28564 2024-05-08T08:55:02.783 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.783 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:02.783 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.783 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:02.783 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359089+0000", 2024-05-08T08:55:02.783 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:02.783 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.783 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.783 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:02.783 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359089+0000", 2024-05-08T08:55:02.784 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:02.784 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.784 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.784 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:02.784 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360959+0000", 2024-05-08T08:55:02.784 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0018707089999999999 2024-05-08T08:55:02.784 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.784 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.784 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:02.784 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360963+0000", 2024-05-08T08:55:02.784 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0680000000000001e-06 2024-05-08T08:55:02.784 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.784 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.784 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:02.785 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360969+0000", 2024-05-08T08:55:02.785 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1920000000000002e-06 2024-05-08T08:55:02.785 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.785 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.785 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:02.785 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360973+0000", 2024-05-08T08:55:02.785 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.031e-06 2024-05-08T08:55:02.785 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.785 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.785 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:02.785 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437916+0000", 2024-05-08T08:55:02.785 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.076942939000000002 2024-05-08T08:55:02.785 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.785 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.785 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:02.786 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437973+0000", 2024-05-08T08:55:02.786 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7358999999999997e-05 2024-05-08T08:55:02.786 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.786 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.786 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:02.786 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.438041+0000", 2024-05-08T08:55:02.786 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8376999999999999e-05 2024-05-08T08:55:02.786 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.786 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:02.786 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.786 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.786 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.786 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28574 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889427600] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:02.786 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:55:02.787 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 90.363539939999995, 2024-05-08T08:55:02.787 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078782476000000004, 2024-05-08T08:55:02.787 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:02.787 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:02.787 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:02.787 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:02.787 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:02.787 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:02.787 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28574 2024-05-08T08:55:02.787 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.787 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:02.787 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.787 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:02.787 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:55:02.787 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:02.788 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.788 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.788 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:02.788 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:55:02.788 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:02.788 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.788 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.788 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:02.788 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:55:02.788 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1799999999999999e-07 2024-05-08T08:55:02.788 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.788 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.788 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:02.788 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361095+0000", 2024-05-08T08:55:02.788 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3850000000000001e-06 2024-05-08T08:55:02.789 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.789 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.789 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:02.789 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361098+0000", 2024-05-08T08:55:02.789 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7989999999999998e-06 2024-05-08T08:55:02.789 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.789 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.789 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:02.789 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361100+0000", 2024-05-08T08:55:02.789 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4669999999999998e-06 2024-05-08T08:55:02.789 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.789 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.789 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:02.789 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439768+0000", 2024-05-08T08:55:02.789 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078668057999999999 2024-05-08T08:55:02.790 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.790 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.790 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:02.790 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439811+0000", 2024-05-08T08:55:02.790 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2459000000000001e-05 2024-05-08T08:55:02.790 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.790 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.790 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:02.790 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439876+0000", 2024-05-08T08:55:02.790 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5090000000000002e-05 2024-05-08T08:55:02.790 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.790 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:02.790 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.790 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.791 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.791 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28575 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889439216] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:02.791 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361103+0000", 2024-05-08T08:55:02.791 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 90.363529924999995, 2024-05-08T08:55:02.791 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078966828000000003, 2024-05-08T08:55:02.791 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:02.791 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:02.791 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:02.791 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:02.791 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:02.791 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:02.791 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28575 2024-05-08T08:55:02.791 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.791 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:02.791 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.792 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:02.792 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361103+0000", 2024-05-08T08:55:02.792 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:02.792 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.792 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.792 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:02.792 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361103+0000", 2024-05-08T08:55:02.792 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:02.792 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.792 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.792 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:02.792 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361104+0000", 2024-05-08T08:55:02.792 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1400000000000001e-07 2024-05-08T08:55:02.792 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.792 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.793 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:02.793 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361105+0000", 2024-05-08T08:55:02.793 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5540000000000001e-06 2024-05-08T08:55:02.793 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.793 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.793 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:02.793 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361108+0000", 2024-05-08T08:55:02.793 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.841e-06 2024-05-08T08:55:02.793 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.793 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.793 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:02.793 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361110+0000", 2024-05-08T08:55:02.793 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.497e-06 2024-05-08T08:55:02.793 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.794 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.794 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:02.794 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439916+0000", 2024-05-08T08:55:02.794 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078805372999999998 2024-05-08T08:55:02.794 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.794 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.794 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:02.794 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440000+0000", 2024-05-08T08:55:02.794 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4730999999999999e-05 2024-05-08T08:55:02.794 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.794 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.794 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:02.794 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440070+0000", 2024-05-08T08:55:02.795 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9617999999999997e-05 2024-05-08T08:55:02.795 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.795 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:02.795 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.795 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.795 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.795 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28576 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889451328] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:02.795 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361113+0000", 2024-05-08T08:55:02.795 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 90.363519738999997, 2024-05-08T08:55:02.795 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079088862999999995, 2024-05-08T08:55:02.795 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:02.795 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:02.795 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:02.795 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:02.795 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:02.795 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:02.796 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28576 2024-05-08T08:55:02.796 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.796 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:02.796 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.796 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:02.796 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361113+0000", 2024-05-08T08:55:02.796 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:02.796 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.796 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.796 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:02.796 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361113+0000", 2024-05-08T08:55:02.796 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:02.796 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.797 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.797 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:02.797 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361114+0000", 2024-05-08T08:55:02.797 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1299999999999999e-07 2024-05-08T08:55:02.797 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.797 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.797 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:02.797 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361115+0000", 2024-05-08T08:55:02.797 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.4300000000000001e-06 2024-05-08T08:55:02.797 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.797 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.797 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:02.797 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361118+0000", 2024-05-08T08:55:02.797 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9000000000000002e-06 2024-05-08T08:55:02.797 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.797 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.798 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:02.798 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361120+0000", 2024-05-08T08:55:02.798 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4739999999999999e-06 2024-05-08T08:55:02.798 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.798 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.798 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:02.798 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440096+0000", 2024-05-08T08:55:02.798 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078975629000000006 2024-05-08T08:55:02.798 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.798 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.798 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:02.798 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440135+0000", 2024-05-08T08:55:02.798 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9202000000000002e-05 2024-05-08T08:55:02.798 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.799 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.799 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:02.799 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440202+0000", 2024-05-08T08:55:02.799 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7014999999999994e-05 2024-05-08T08:55:02.799 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.799 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:02.799 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.799 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.799 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.799 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28577 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889462640] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:02.799 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:55:02.799 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 90.363509557, 2024-05-08T08:55:02.799 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079194622000000006, 2024-05-08T08:55:02.799 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:02.799 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:02.800 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:02.800 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:02.800 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:02.800 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:02.800 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28577 2024-05-08T08:55:02.800 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.800 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:02.800 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.800 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:02.800 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:55:02.800 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:02.800 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.800 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.800 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:02.801 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:55:02.801 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:02.801 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.801 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.801 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:02.801 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:55:02.801 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1400000000000001e-07 2024-05-08T08:55:02.801 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.801 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.801 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:02.801 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361125+0000", 2024-05-08T08:55:02.801 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.372e-06 2024-05-08T08:55:02.801 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.801 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.802 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:02.802 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361128+0000", 2024-05-08T08:55:02.802 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0309999999999999e-06 2024-05-08T08:55:02.802 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.802 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.802 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:02.802 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361131+0000", 2024-05-08T08:55:02.802 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4040000000000002e-06 2024-05-08T08:55:02.802 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.802 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.802 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:02.802 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440225+0000", 2024-05-08T08:55:02.802 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079094600000000001 2024-05-08T08:55:02.802 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.803 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.803 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:02.803 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440260+0000", 2024-05-08T08:55:02.803 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4703000000000003e-05 2024-05-08T08:55:02.803 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.803 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.803 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:02.803 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440318+0000", 2024-05-08T08:55:02.803 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8298000000000003e-05 2024-05-08T08:55:02.803 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.803 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:02.803 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.803 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.803 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.803 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28578 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889474192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:02.804 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361134+0000", 2024-05-08T08:55:02.804 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 90.363499132000001, 2024-05-08T08:55:02.804 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079310295000000003, 2024-05-08T08:55:02.804 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:02.804 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:02.804 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:02.804 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:02.804 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:02.804 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:02.804 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28578 2024-05-08T08:55:02.804 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.804 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:02.804 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.804 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:02.805 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361134+0000", 2024-05-08T08:55:02.805 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:02.805 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.805 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.805 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:02.805 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361134+0000", 2024-05-08T08:55:02.805 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:02.805 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.805 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.805 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:02.805 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.362958+0000", 2024-05-08T08:55:02.805 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0018237380000000001 2024-05-08T08:55:02.805 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.805 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.805 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:02.806 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.362961+0000", 2024-05-08T08:55:02.806 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.123e-06 2024-05-08T08:55:02.806 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.806 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.806 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:02.806 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.397404+0000", 2024-05-08T08:55:02.806 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.034443226 2024-05-08T08:55:02.806 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.806 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.806 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:02.806 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.397439+0000", 2024-05-08T08:55:02.806 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4690000000000002e-05 2024-05-08T08:55:02.806 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.806 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.806 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:02.807 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440340+0000", 2024-05-08T08:55:02.807 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.042901419000000003 2024-05-08T08:55:02.807 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.807 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.807 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:02.807 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440376+0000", 2024-05-08T08:55:02.807 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5444999999999997e-05 2024-05-08T08:55:02.807 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:02.807 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:02.807 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:02.807 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440444+0000", 2024-05-08T08:55:02.807 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8653999999999997e-05 2024-05-08T08:55:02.807 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.807 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:02.807 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.807 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:02.808 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:02.808 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:55:02.809 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:55:02.809 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:55:02.812+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:55:02.910 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:55:02.913+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:55:02.960 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:55:02.960 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:55:02.960 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:55:02.961 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:55:02.969 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:55:03.011 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:55:03.013+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:55:03.111 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:55:03.114+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:55:03.119 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:55:03.119 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:55:03.119 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:55:03.119 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:55:03.119 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:55:03.128 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:55:03.212 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:55:03.215+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:55:03.278 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:55:03.278 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:55:03.278 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:55:03.278 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:55:03.278 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.278 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:437 3.17 3:ea93a512:::rbd_data.1140100c9703.0000000000000077:head [set-alloc-hint object_size 2097152 write_size 2097152,write 1170432~1024] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e63)", 2024-05-08T08:55:03.278 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:08.390804+0000", 2024-05-08T08:55:03.278 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 114.88920754, 2024-05-08T08:55:03.279 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0406068690000001, 2024-05-08T08:55:03.279 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:03.279 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:03.279 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:55:03.279 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:03.279 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:03.279 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:03.279 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 437 2024-05-08T08:55:03.279 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.279 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:03.279 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.279 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:03.279 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390804+0000", 2024-05-08T08:55:03.279 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:03.279 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.280 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.280 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:03.280 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390804+0000", 2024-05-08T08:55:03.280 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:03.280 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.280 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.280 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:03.280 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390806+0000", 2024-05-08T08:55:03.280 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.666e-06 2024-05-08T08:55:03.280 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.280 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.280 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:03.280 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390818+0000", 2024-05-08T08:55:03.280 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2187999999999999e-05 2024-05-08T08:55:03.280 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.281 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.281 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:03.281 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390834+0000", 2024-05-08T08:55:03.281 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6191000000000001e-05 2024-05-08T08:55:03.281 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.281 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.281 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:03.281 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390845+0000", 2024-05-08T08:55:03.281 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0831e-05 2024-05-08T08:55:03.281 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.281 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.281 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:03.281 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390922+0000", 2024-05-08T08:55:03.281 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7253e-05 2024-05-08T08:55:03.281 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.282 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.282 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op must wait for map", 2024-05-08T08:55:03.282 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390930+0000", 2024-05-08T08:55:03.282 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8259999999999993e-06 2024-05-08T08:55:03.282 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.282 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.282 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:03.282 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392155+0000", 2024-05-08T08:55:03.282 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0012256649999999999 2024-05-08T08:55:03.282 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.282 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.282 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:55:03.282 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392165+0000", 2024-05-08T08:55:03.282 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5799999999999998e-06 2024-05-08T08:55:03.283 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.283 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.283 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:03.283 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.429596+0000", 2024-05-08T08:55:03.283 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0374313639999999 2024-05-08T08:55:03.283 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.283 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.283 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:03.283 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.429728+0000", 2024-05-08T08:55:03.283 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000131552 2024-05-08T08:55:03.283 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.283 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.283 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 0,1", 2024-05-08T08:55:03.283 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.429860+0000", 2024-05-08T08:55:03.283 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000132137 2024-05-08T08:55:03.283 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.284 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.284 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:55:03.284 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.430811+0000", 2024-05-08T08:55:03.284 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00095040200000000004 2024-05-08T08:55:03.284 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.284 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.284 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:55:03.284 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431126+0000", 2024-05-08T08:55:03.284 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000315006 2024-05-08T08:55:03.284 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.284 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.284 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:55:03.284 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431272+0000", 2024-05-08T08:55:03.284 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000146487 2024-05-08T08:55:03.284 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.285 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.285 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:55:03.285 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431315+0000", 2024-05-08T08:55:03.285 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2697000000000001e-05 2024-05-08T08:55:03.285 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.285 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.285 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:03.285 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431411+0000", 2024-05-08T08:55:03.285 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6024000000000001e-05 2024-05-08T08:55:03.285 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:03.285 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:03.285 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:03.285 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.285 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.285 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:439 3.17 3:ea93a512:::rbd_data.1140100c9703.0000000000000077:head [read 1163264~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-05-08T08:55:03.286 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:08.390871+0000", 2024-05-08T08:55:03.286 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 114.889140029, 2024-05-08T08:55:03.286 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0408790590000001, 2024-05-08T08:55:03.286 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:03.286 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:03.286 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:03.286 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:03.286 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:03.286 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:03.286 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 439 2024-05-08T08:55:03.286 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.286 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:03.286 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.286 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:03.287 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390871+0000", 2024-05-08T08:55:03.287 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:03.287 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.287 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.287 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:03.287 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390871+0000", 2024-05-08T08:55:03.287 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:03.287 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.287 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.287 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:03.287 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390872+0000", 2024-05-08T08:55:03.287 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6300000000000005e-07 2024-05-08T08:55:03.287 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.287 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.287 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:03.287 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390875+0000", 2024-05-08T08:55:03.287 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3459999999999998e-06 2024-05-08T08:55:03.288 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.288 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.288 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:03.288 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390891+0000", 2024-05-08T08:55:03.288 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5255e-05 2024-05-08T08:55:03.288 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.288 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.288 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:03.288 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390897+0000", 2024-05-08T08:55:03.288 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2360000000000001e-06 2024-05-08T08:55:03.288 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.288 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.288 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:03.288 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390995+0000", 2024-05-08T08:55:03.288 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8522000000000003e-05 2024-05-08T08:55:03.288 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.289 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.289 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting_for_map not empty", 2024-05-08T08:55:03.289 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.391005+0000", 2024-05-08T08:55:03.289 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1460000000000002e-06 2024-05-08T08:55:03.289 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.289 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.289 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:03.289 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392191+0000", 2024-05-08T08:55:03.289 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0011860270000000001 2024-05-08T08:55:03.289 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.289 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.289 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:55:03.289 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392197+0000", 2024-05-08T08:55:03.289 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4289999999999998e-06 2024-05-08T08:55:03.289 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.289 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.290 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:03.290 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.430596+0000", 2024-05-08T08:55:03.290 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0383989090000001 2024-05-08T08:55:03.290 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.290 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.290 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for rw locks", 2024-05-08T08:55:03.290 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.430676+0000", 2024-05-08T08:55:03.290 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9781000000000001e-05 2024-05-08T08:55:03.290 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.290 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.290 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:03.290 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431460+0000", 2024-05-08T08:55:03.290 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00078427300000000004 2024-05-08T08:55:03.290 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.291 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.291 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:03.291 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431540+0000", 2024-05-08T08:55:03.291 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0224999999999998e-05 2024-05-08T08:55:03.291 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.291 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.291 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:03.291 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431750+0000", 2024-05-08T08:55:03.291 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00021024699999999999 2024-05-08T08:55:03.291 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:03.291 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:03.291 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:03.291 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.291 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.291 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:945 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 4096000~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:03.291 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:55:03.292 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 113.82899046199999, 2024-05-08T08:55:03.292 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086774741000000002, 2024-05-08T08:55:03.292 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:03.292 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:03.292 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:03.292 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:03.292 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:03.292 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:03.292 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 945 2024-05-08T08:55:03.292 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.292 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:03.292 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.292 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:03.292 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:55:03.292 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:03.293 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.293 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.293 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:03.293 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:55:03.293 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:03.293 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.293 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.293 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:03.293 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451021+0000", 2024-05-08T08:55:03.293 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3499999999999996e-07 2024-05-08T08:55:03.293 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.293 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.293 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:03.293 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451024+0000", 2024-05-08T08:55:03.293 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.627e-06 2024-05-08T08:55:03.293 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.294 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.294 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:03.294 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451037+0000", 2024-05-08T08:55:03.294 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2431e-05 2024-05-08T08:55:03.294 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.294 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.294 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:03.294 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451042+0000", 2024-05-08T08:55:03.294 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8409999999999996e-06 2024-05-08T08:55:03.294 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.294 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.294 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:03.294 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537608+0000", 2024-05-08T08:55:03.294 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086565243 2024-05-08T08:55:03.294 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.295 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.295 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:03.295 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537687+0000", 2024-05-08T08:55:03.295 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9519999999999998e-05 2024-05-08T08:55:03.295 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.295 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.295 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:03.295 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537796+0000", 2024-05-08T08:55:03.295 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000108544 2024-05-08T08:55:03.295 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:03.295 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:03.295 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:03.295 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.295 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.295 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:964 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 458752~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:03.295 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451051+0000", 2024-05-08T08:55:03.296 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 113.82896028899999, 2024-05-08T08:55:03.296 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086990490000000004, 2024-05-08T08:55:03.296 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:03.296 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:03.296 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:03.296 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:03.296 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:03.296 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:03.296 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 964 2024-05-08T08:55:03.296 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.296 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:03.296 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.296 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:03.296 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451051+0000", 2024-05-08T08:55:03.296 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:03.296 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.297 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.297 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:03.297 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451051+0000", 2024-05-08T08:55:03.297 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:03.297 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.297 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.297 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:03.297 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451052+0000", 2024-05-08T08:55:03.297 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7299999999999996e-07 2024-05-08T08:55:03.297 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.297 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.297 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:03.297 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451055+0000", 2024-05-08T08:55:03.297 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.2770000000000001e-06 2024-05-08T08:55:03.297 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.298 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.298 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:03.298 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451061+0000", 2024-05-08T08:55:03.298 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4060000000000002e-06 2024-05-08T08:55:03.298 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.298 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.298 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:03.298 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451066+0000", 2024-05-08T08:55:03.298 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0839999999999999e-06 2024-05-08T08:55:03.298 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.298 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.298 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:03.298 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537839+0000", 2024-05-08T08:55:03.298 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.086772962999999995 2024-05-08T08:55:03.299 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.299 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.299 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:03.299 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.537934+0000", 2024-05-08T08:55:03.299 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4742999999999997e-05 2024-05-08T08:55:03.299 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.299 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.299 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:03.299 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538042+0000", 2024-05-08T08:55:03.299 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000107444 2024-05-08T08:55:03.299 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:03.299 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:03.299 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:03.299 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.300 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.300 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:965 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 950272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:03.300 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451075+0000", 2024-05-08T08:55:03.300 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 113.828936227, 2024-05-08T08:55:03.300 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087230196999999995, 2024-05-08T08:55:03.300 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:03.300 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:03.300 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:03.300 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:03.300 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:03.300 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:03.300 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 965 2024-05-08T08:55:03.300 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.300 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:03.300 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.301 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:03.301 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451075+0000", 2024-05-08T08:55:03.301 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:03.301 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.301 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.301 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:03.301 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451075+0000", 2024-05-08T08:55:03.301 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:03.301 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.301 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.301 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:03.301 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451076+0000", 2024-05-08T08:55:03.301 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5000000000000002e-07 2024-05-08T08:55:03.301 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.301 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.301 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:03.302 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451078+0000", 2024-05-08T08:55:03.302 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.5639999999999999e-06 2024-05-08T08:55:03.302 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.302 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.302 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:03.302 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451085+0000", 2024-05-08T08:55:03.302 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5640000000000002e-06 2024-05-08T08:55:03.302 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.302 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.302 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:03.302 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451090+0000", 2024-05-08T08:55:03.302 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1699999999999996e-06 2024-05-08T08:55:03.302 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.302 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.303 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:03.303 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538106+0000", 2024-05-08T08:55:03.303 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087015888999999999 2024-05-08T08:55:03.303 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.303 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.303 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:03.303 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538190+0000", 2024-05-08T08:55:03.303 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3961999999999993e-05 2024-05-08T08:55:03.303 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.303 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.303 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:03.303 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538305+0000", 2024-05-08T08:55:03.303 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011529800000000001 2024-05-08T08:55:03.303 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:03.303 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:03.304 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:03.304 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.304 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.304 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:966 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 1441792~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:03.304 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:55:03.304 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 113.828912854, 2024-05-08T08:55:03.304 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087446323000000006, 2024-05-08T08:55:03.304 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:03.304 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:03.304 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:03.304 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:03.304 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:03.304 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:03.304 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 966 2024-05-08T08:55:03.304 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.305 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:03.305 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.305 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:03.305 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:55:03.305 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:03.305 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.305 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.305 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:03.305 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:55:03.305 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:03.305 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.305 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.305 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:03.305 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:55:03.305 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3700000000000001e-07 2024-05-08T08:55:03.306 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.306 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.306 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:03.306 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451101+0000", 2024-05-08T08:55:03.306 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2560000000000001e-06 2024-05-08T08:55:03.306 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.306 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.306 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:03.306 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451107+0000", 2024-05-08T08:55:03.306 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1070000000000001e-06 2024-05-08T08:55:03.306 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.306 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.306 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:03.306 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451112+0000", 2024-05-08T08:55:03.307 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1420000000000001e-06 2024-05-08T08:55:03.307 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.307 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.307 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:03.307 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538341+0000", 2024-05-08T08:55:03.307 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087228302999999993 2024-05-08T08:55:03.307 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.307 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.307 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:03.307 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538424+0000", 2024-05-08T08:55:03.307 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3497000000000004e-05 2024-05-08T08:55:03.307 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.307 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.307 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:03.307 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538545+0000", 2024-05-08T08:55:03.308 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000120581 2024-05-08T08:55:03.308 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:03.308 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:03.308 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:03.308 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.308 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.308 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:967 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:03.308 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451120+0000", 2024-05-08T08:55:03.308 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 113.828891379, 2024-05-08T08:55:03.308 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087632725999999994, 2024-05-08T08:55:03.308 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:03.308 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:03.308 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:03.308 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:03.308 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:03.309 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:03.309 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 967 2024-05-08T08:55:03.309 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.309 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:03.309 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.309 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:03.309 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451120+0000", 2024-05-08T08:55:03.309 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:03.309 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.309 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.309 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:03.309 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451120+0000", 2024-05-08T08:55:03.309 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:03.309 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.309 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.309 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:03.309 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451121+0000", 2024-05-08T08:55:03.310 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8800000000000002e-07 2024-05-08T08:55:03.310 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.310 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.310 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:03.310 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451130+0000", 2024-05-08T08:55:03.310 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2299999999999997e-06 2024-05-08T08:55:03.310 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.310 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.310 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:03.310 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451137+0000", 2024-05-08T08:55:03.310 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4669999999999998e-06 2024-05-08T08:55:03.310 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.310 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.310 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:03.310 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451143+0000", 2024-05-08T08:55:03.310 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2839999999999997e-06 2024-05-08T08:55:03.311 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.311 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.311 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:03.311 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538575+0000", 2024-05-08T08:55:03.311 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087432789999999996 2024-05-08T08:55:03.311 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.311 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.311 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:03.311 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538648+0000", 2024-05-08T08:55:03.311 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2877e-05 2024-05-08T08:55:03.311 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.311 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.311 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:03.311 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538753+0000", 2024-05-08T08:55:03.311 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010449 2024-05-08T08:55:03.312 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:03.312 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:03.312 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:03.312 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.312 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.312 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:968 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 2424832~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:03.312 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451152+0000", 2024-05-08T08:55:03.312 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 113.828859294, 2024-05-08T08:55:03.312 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087813325999999997, 2024-05-08T08:55:03.312 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:03.312 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:03.312 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:03.312 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:03.313 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:03.313 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:03.313 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 968 2024-05-08T08:55:03.313 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.313 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:03.313 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.313 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:03.313 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451152+0000", 2024-05-08T08:55:03.313 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:03.313 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.313 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.313 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:03.313 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451152+0000", 2024-05-08T08:55:03.313 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:03.313 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.313 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.314 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:03.314 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451153+0000", 2024-05-08T08:55:03.314 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3099999999999998e-07 2024-05-08T08:55:03.314 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.314 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.314 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:03.314 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451155+0000", 2024-05-08T08:55:03.314 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4760000000000001e-06 2024-05-08T08:55:03.314 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.314 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.314 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:03.314 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451162+0000", 2024-05-08T08:55:03.314 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5969999999999997e-06 2024-05-08T08:55:03.314 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.314 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.315 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:03.315 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451167+0000", 2024-05-08T08:55:03.315 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.977e-06 2024-05-08T08:55:03.315 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.315 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.315 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:03.315 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538781+0000", 2024-05-08T08:55:03.315 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087614846999999996 2024-05-08T08:55:03.315 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.315 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.315 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:03.315 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538857+0000", 2024-05-08T08:55:03.315 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5358000000000006e-05 2024-05-08T08:55:03.315 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.315 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.316 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:03.316 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538965+0000", 2024-05-08T08:55:03.316 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010864 2024-05-08T08:55:03.316 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:03.316 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:03.316 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:03.316 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.316 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.316 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:969 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 2916352~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:03.316 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:55:03.316 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 113.828837589, 2024-05-08T08:55:03.316 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087998980000000004, 2024-05-08T08:55:03.316 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:03.316 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:03.317 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:03.317 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:03.317 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:03.317 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:03.317 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 969 2024-05-08T08:55:03.317 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.317 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:03.317 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.317 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:03.317 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:55:03.317 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:03.317 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.317 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.317 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:03.317 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:55:03.317 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:03.318 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.318 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.318 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:03.318 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:55:03.318 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3599999999999999e-07 2024-05-08T08:55:03.318 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.318 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.318 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:03.318 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451177+0000", 2024-05-08T08:55:03.318 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4590000000000001e-06 2024-05-08T08:55:03.318 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.318 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.318 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:03.318 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451184+0000", 2024-05-08T08:55:03.318 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4109999999999999e-06 2024-05-08T08:55:03.318 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.319 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.319 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:03.319 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451191+0000", 2024-05-08T08:55:03.319 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7279999999999998e-06 2024-05-08T08:55:03.319 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.319 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.319 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:03.319 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538997+0000", 2024-05-08T08:55:03.319 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087806170000000003 2024-05-08T08:55:03.319 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.319 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.319 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:03.319 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539069+0000", 2024-05-08T08:55:03.319 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2373999999999995e-05 2024-05-08T08:55:03.319 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.319 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.320 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:03.320 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539173+0000", 2024-05-08T08:55:03.320 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000103402 2024-05-08T08:55:03.320 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:03.320 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:03.320 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:03.320 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.320 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.320 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:970 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 3407872~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:03.320 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451198+0000", 2024-05-08T08:55:03.320 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 113.828812992, 2024-05-08T08:55:03.320 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088183368999999998, 2024-05-08T08:55:03.320 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:03.320 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:03.321 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:03.321 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:03.321 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:03.321 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:03.321 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 970 2024-05-08T08:55:03.321 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.321 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:03.321 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.321 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:03.321 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451198+0000", 2024-05-08T08:55:03.321 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:03.321 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.321 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.321 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:03.321 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451198+0000", 2024-05-08T08:55:03.322 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:03.322 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.322 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.322 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:03.322 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451199+0000", 2024-05-08T08:55:03.322 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7400000000000003e-07 2024-05-08T08:55:03.322 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.322 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.322 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:03.322 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451211+0000", 2024-05-08T08:55:03.322 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2109999999999999e-05 2024-05-08T08:55:03.322 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.322 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.322 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:03.323 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451219+0000", 2024-05-08T08:55:03.323 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4170000000000003e-06 2024-05-08T08:55:03.323 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.323 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.323 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:03.323 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451224+0000", 2024-05-08T08:55:03.323 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3990000000000003e-06 2024-05-08T08:55:03.323 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.323 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.323 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:03.323 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539204+0000", 2024-05-08T08:55:03.323 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087980099000000006 2024-05-08T08:55:03.323 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.323 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.323 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:03.323 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539276+0000", 2024-05-08T08:55:03.324 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1513000000000005e-05 2024-05-08T08:55:03.324 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.324 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.324 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:03.324 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539382+0000", 2024-05-08T08:55:03.324 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010615699999999999 2024-05-08T08:55:03.324 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:03.324 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:03.324 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:03.324 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.324 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.324 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:971 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 3899392~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:03.324 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451233+0000", 2024-05-08T08:55:03.324 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 113.828778285, 2024-05-08T08:55:03.324 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088431525999999996, 2024-05-08T08:55:03.324 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:03.325 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:03.325 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:03.325 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:03.325 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:03.325 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:03.325 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 971 2024-05-08T08:55:03.325 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.325 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:03.325 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.325 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:03.325 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451233+0000", 2024-05-08T08:55:03.325 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:03.325 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.325 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.325 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:03.325 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451233+0000", 2024-05-08T08:55:03.326 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:03.326 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.326 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.326 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:03.326 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451234+0000", 2024-05-08T08:55:03.326 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8500000000000001e-07 2024-05-08T08:55:03.326 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.326 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.326 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:03.326 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451237+0000", 2024-05-08T08:55:03.326 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8320000000000002e-06 2024-05-08T08:55:03.326 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.326 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.326 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:03.326 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451243+0000", 2024-05-08T08:55:03.327 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6749999999999996e-06 2024-05-08T08:55:03.327 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.327 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.327 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:03.327 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451249+0000", 2024-05-08T08:55:03.327 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1229999999999997e-06 2024-05-08T08:55:03.327 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.327 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.327 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:03.327 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539412+0000", 2024-05-08T08:55:03.327 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088162668999999999 2024-05-08T08:55:03.327 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.327 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.327 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:03.327 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539505+0000", 2024-05-08T08:55:03.327 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3240999999999997e-05 2024-05-08T08:55:03.328 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.328 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.328 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:03.328 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539665+0000", 2024-05-08T08:55:03.328 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00015930100000000001 2024-05-08T08:55:03.328 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:03.328 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:03.328 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:03.328 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.328 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.328 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:972 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 196608~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:03.328 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451257+0000", 2024-05-08T08:55:03.328 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 113.82875434, 2024-05-08T08:55:03.328 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088719928000000003, 2024-05-08T08:55:03.328 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:03.328 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:03.328 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:03.329 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:03.329 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:03.329 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:03.329 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 972 2024-05-08T08:55:03.329 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.329 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:03.329 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.329 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:03.329 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451257+0000", 2024-05-08T08:55:03.329 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:03.329 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.329 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.329 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:03.329 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451257+0000", 2024-05-08T08:55:03.329 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:03.330 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.330 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.330 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:03.330 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451258+0000", 2024-05-08T08:55:03.330 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7999999999999996e-07 2024-05-08T08:55:03.330 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.330 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.330 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:03.330 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451260+0000", 2024-05-08T08:55:03.330 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4609999999999998e-06 2024-05-08T08:55:03.330 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.330 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.330 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:03.330 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451267+0000", 2024-05-08T08:55:03.331 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.81e-06 2024-05-08T08:55:03.331 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.331 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.331 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:03.331 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451273+0000", 2024-05-08T08:55:03.331 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0859999999999998e-06 2024-05-08T08:55:03.331 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.331 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.331 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:03.331 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539718+0000", 2024-05-08T08:55:03.331 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088445059000000006 2024-05-08T08:55:03.331 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.331 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.331 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:03.331 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539834+0000", 2024-05-08T08:55:03.331 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000116426 2024-05-08T08:55:03.331 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.332 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.332 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:03.332 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539977+0000", 2024-05-08T08:55:03.332 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014260600000000001 2024-05-08T08:55:03.332 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:03.332 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:03.332 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:03.332 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.332 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.332 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:973 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 688128~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:03.332 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:55:03.332 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 113.828731756, 2024-05-08T08:55:03.332 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088918496, 2024-05-08T08:55:03.332 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:03.332 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:03.332 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:03.333 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:03.333 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:03.333 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:03.333 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 973 2024-05-08T08:55:03.333 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.333 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:03.333 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.333 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:03.333 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:55:03.333 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:03.333 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.333 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.333 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:03.333 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:55:03.333 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:03.333 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.334 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.334 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:03.334 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:55:03.334 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9099999999999999e-07 2024-05-08T08:55:03.334 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.334 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.334 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:03.334 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451282+0000", 2024-05-08T08:55:03.334 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1550000000000001e-06 2024-05-08T08:55:03.334 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.334 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.334 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:03.334 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451288+0000", 2024-05-08T08:55:03.334 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8889999999999999e-06 2024-05-08T08:55:03.335 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.335 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.335 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:03.335 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451294+0000", 2024-05-08T08:55:03.335 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0850000000000002e-06 2024-05-08T08:55:03.335 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.335 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.335 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:03.335 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540011+0000", 2024-05-08T08:55:03.335 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088717314000000005 2024-05-08T08:55:03.335 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.335 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.335 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:03.335 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540089+0000", 2024-05-08T08:55:03.335 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7438e-05 2024-05-08T08:55:03.335 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.336 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.336 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:03.336 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540198+0000", 2024-05-08T08:55:03.336 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000109224 2024-05-08T08:55:03.336 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:03.336 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:03.336 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:03.336 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.336 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.336 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:974 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 1179648~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:03.336 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451301+0000", 2024-05-08T08:55:03.336 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 113.82871012299999, 2024-05-08T08:55:03.336 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089105078000000004, 2024-05-08T08:55:03.336 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:03.336 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:03.336 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:03.336 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:03.337 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:03.337 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:03.337 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 974 2024-05-08T08:55:03.337 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.337 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:03.337 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.337 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:03.337 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451301+0000", 2024-05-08T08:55:03.337 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:03.337 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.337 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.337 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:03.337 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451301+0000", 2024-05-08T08:55:03.337 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:03.337 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.337 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.338 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:03.338 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451302+0000", 2024-05-08T08:55:03.338 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7800000000000002e-07 2024-05-08T08:55:03.338 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.338 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.338 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:03.338 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451304+0000", 2024-05-08T08:55:03.338 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.65e-06 2024-05-08T08:55:03.338 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.338 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.338 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:03.338 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451311+0000", 2024-05-08T08:55:03.338 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.472e-06 2024-05-08T08:55:03.338 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.338 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.339 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:03.339 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451316+0000", 2024-05-08T08:55:03.339 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9189999999999998e-06 2024-05-08T08:55:03.339 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.339 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.339 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:03.339 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540234+0000", 2024-05-08T08:55:03.339 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088917752000000003 2024-05-08T08:55:03.339 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.339 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.339 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:03.339 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540308+0000", 2024-05-08T08:55:03.339 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4758000000000005e-05 2024-05-08T08:55:03.339 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.339 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.339 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:03.340 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540406+0000", 2024-05-08T08:55:03.340 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8048999999999996e-05 2024-05-08T08:55:03.340 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:03.340 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:03.340 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:03.340 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.340 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.340 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:975 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 1671168~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:03.340 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:55:03.340 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 113.82866779299999, 2024-05-08T08:55:03.340 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089294139999999994, 2024-05-08T08:55:03.340 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:03.340 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:03.340 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:03.340 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:03.340 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:03.341 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:03.341 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 975 2024-05-08T08:55:03.341 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.341 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:03.341 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.341 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:03.341 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:55:03.341 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:03.341 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.341 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.341 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:03.341 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:55:03.341 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:03.341 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.341 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.341 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:03.342 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:55:03.342 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1299999999999999e-07 2024-05-08T08:55:03.342 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.342 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.342 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:03.342 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451348+0000", 2024-05-08T08:55:03.342 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.484e-06 2024-05-08T08:55:03.342 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.342 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.342 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:03.342 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451356+0000", 2024-05-08T08:55:03.342 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8499999999999994e-06 2024-05-08T08:55:03.342 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.342 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.342 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:03.343 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451366+0000", 2024-05-08T08:55:03.343 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.9850000000000001e-06 2024-05-08T08:55:03.343 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.343 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.343 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:03.343 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540448+0000", 2024-05-08T08:55:03.343 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089082254 2024-05-08T08:55:03.343 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.343 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.343 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:03.343 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540528+0000", 2024-05-08T08:55:03.343 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0234999999999993e-05 2024-05-08T08:55:03.343 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.343 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.343 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:03.344 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540638+0000", 2024-05-08T08:55:03.344 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010951900000000001 2024-05-08T08:55:03.344 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:03.344 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:03.344 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:03.344 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.344 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.344 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:976 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 2162688~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:03.344 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451375+0000", 2024-05-08T08:55:03.344 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 113.828636046, 2024-05-08T08:55:03.344 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089472745000000006, 2024-05-08T08:55:03.344 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:03.344 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:03.344 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:03.344 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:03.344 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:03.345 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:03.345 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 976 2024-05-08T08:55:03.345 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.345 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:03.345 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.345 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:03.345 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451375+0000", 2024-05-08T08:55:03.345 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:03.345 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.345 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.345 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:03.345 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451375+0000", 2024-05-08T08:55:03.345 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:03.345 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.345 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.345 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:03.346 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451376+0000", 2024-05-08T08:55:03.346 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2399999999999998e-07 2024-05-08T08:55:03.346 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.346 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.346 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:03.346 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451379+0000", 2024-05-08T08:55:03.346 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6359999999999998e-06 2024-05-08T08:55:03.346 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.346 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.346 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:03.346 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451387+0000", 2024-05-08T08:55:03.346 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9929999999999995e-06 2024-05-08T08:55:03.346 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.346 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.347 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:03.347 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451393+0000", 2024-05-08T08:55:03.347 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4699999999999999e-06 2024-05-08T08:55:03.347 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.347 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.347 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:03.347 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540677+0000", 2024-05-08T08:55:03.347 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089283504999999999 2024-05-08T08:55:03.347 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.347 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.347 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:03.347 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540743+0000", 2024-05-08T08:55:03.347 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6303999999999994e-05 2024-05-08T08:55:03.347 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.347 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.347 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:03.348 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540848+0000", 2024-05-08T08:55:03.348 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000105213 2024-05-08T08:55:03.348 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:03.348 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:03.348 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:03.348 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.348 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.348 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:977 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 2654208~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:03.348 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:55:03.348 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 113.828610614, 2024-05-08T08:55:03.348 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089698473000000001, 2024-05-08T08:55:03.348 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:03.348 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:03.348 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:03.348 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:03.348 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:03.349 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:03.349 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 977 2024-05-08T08:55:03.349 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.349 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:03.349 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.349 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:03.349 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:55:03.349 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:03.349 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.349 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.349 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:03.349 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:55:03.349 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:03.349 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.349 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.350 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:03.350 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:55:03.350 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2899999999999999e-07 2024-05-08T08:55:03.350 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.350 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.350 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:03.350 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451404+0000", 2024-05-08T08:55:03.350 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6649999999999999e-06 2024-05-08T08:55:03.350 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.350 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.350 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:03.350 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451410+0000", 2024-05-08T08:55:03.350 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3269999999999997e-06 2024-05-08T08:55:03.350 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.351 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.351 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:03.351 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451415+0000", 2024-05-08T08:55:03.351 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9529999999999998e-06 2024-05-08T08:55:03.351 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.351 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.351 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:03.351 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540912+0000", 2024-05-08T08:55:03.351 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089496950000000006 2024-05-08T08:55:03.351 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.351 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.351 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:03.351 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541000+0000", 2024-05-08T08:55:03.351 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7429000000000006e-05 2024-05-08T08:55:03.351 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.352 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.352 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:03.352 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541099+0000", 2024-05-08T08:55:03.352 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.9720000000000001e-05 2024-05-08T08:55:03.352 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:03.352 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:03.352 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:03.352 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.352 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.352 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:978 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 3145728~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:03.352 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:55:03.352 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 113.82858940600001, 2024-05-08T08:55:03.352 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089900635000000007, 2024-05-08T08:55:03.352 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:03.352 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:03.352 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:03.353 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:03.353 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:03.353 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:03.353 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 978 2024-05-08T08:55:03.353 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.353 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:03.353 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.353 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:03.353 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:55:03.353 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:03.353 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.353 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.353 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:03.353 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:55:03.353 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:03.354 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.354 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.354 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:03.354 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:55:03.354 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9400000000000001e-07 2024-05-08T08:55:03.354 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.354 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.354 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:03.354 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451425+0000", 2024-05-08T08:55:03.354 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2460000000000002e-06 2024-05-08T08:55:03.354 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.354 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.354 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:03.354 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451431+0000", 2024-05-08T08:55:03.355 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2419999999999997e-06 2024-05-08T08:55:03.355 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.355 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.355 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:03.355 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451436+0000", 2024-05-08T08:55:03.355 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.887e-06 2024-05-08T08:55:03.355 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.355 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.355 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:03.355 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541152+0000", 2024-05-08T08:55:03.355 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089716194999999999 2024-05-08T08:55:03.355 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.355 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.355 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:03.355 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541224+0000", 2024-05-08T08:55:03.355 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2416000000000006e-05 2024-05-08T08:55:03.356 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.356 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.356 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:03.356 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541323+0000", 2024-05-08T08:55:03.356 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8255e-05 2024-05-08T08:55:03.356 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:03.356 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:03.356 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:03.356 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.356 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.356 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:979 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 3637248~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:03.356 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451462+0000", 2024-05-08T08:55:03.356 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 113.828548957, 2024-05-08T08:55:03.356 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.090036441999999994, 2024-05-08T08:55:03.356 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:03.356 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:03.357 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:03.357 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:03.357 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:03.357 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:03.357 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 979 2024-05-08T08:55:03.357 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.357 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:03.357 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.357 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:03.357 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451462+0000", 2024-05-08T08:55:03.357 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:03.357 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.357 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.357 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:03.357 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451462+0000", 2024-05-08T08:55:03.357 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:03.358 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.358 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.358 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:03.358 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451463+0000", 2024-05-08T08:55:03.358 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4300000000000002e-07 2024-05-08T08:55:03.358 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.358 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.358 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:03.358 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451471+0000", 2024-05-08T08:55:03.358 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0800000000000006e-06 2024-05-08T08:55:03.358 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.358 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.358 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:03.358 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451481+0000", 2024-05-08T08:55:03.358 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.38e-06 2024-05-08T08:55:03.359 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.359 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.359 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:03.359 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451487+0000", 2024-05-08T08:55:03.359 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4590000000000003e-06 2024-05-08T08:55:03.359 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.359 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.359 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:03.359 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541350+0000", 2024-05-08T08:55:03.359 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089862599000000001 2024-05-08T08:55:03.359 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.359 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.359 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:03.359 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541407+0000", 2024-05-08T08:55:03.360 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7667999999999997e-05 2024-05-08T08:55:03.360 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.360 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.360 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:03.360 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541499+0000", 2024-05-08T08:55:03.360 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1513000000000003e-05 2024-05-08T08:55:03.360 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:03.360 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:03.360 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:03.360 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.360 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.360 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:980 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 4128768~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:03.360 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451496+0000", 2024-05-08T08:55:03.360 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 113.828515201, 2024-05-08T08:55:03.360 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.090258985, 2024-05-08T08:55:03.360 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:03.361 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:03.361 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:03.361 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:03.361 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:03.361 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:03.361 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 980 2024-05-08T08:55:03.361 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.361 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:03.361 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.361 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:03.361 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451496+0000", 2024-05-08T08:55:03.361 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:03.361 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.361 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.361 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:03.361 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451496+0000", 2024-05-08T08:55:03.361 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:03.362 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.362 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.362 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:03.362 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451497+0000", 2024-05-08T08:55:03.362 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7599999999999997e-07 2024-05-08T08:55:03.362 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.362 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.362 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:03.362 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451500+0000", 2024-05-08T08:55:03.362 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7930000000000002e-06 2024-05-08T08:55:03.362 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.362 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.362 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:03.362 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451506+0000", 2024-05-08T08:55:03.363 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6980000000000001e-06 2024-05-08T08:55:03.363 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.363 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.363 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:03.363 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451512+0000", 2024-05-08T08:55:03.363 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2490000000000001e-06 2024-05-08T08:55:03.363 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.363 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.363 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:03.363 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541550+0000", 2024-05-08T08:55:03.363 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.090037961999999999 2024-05-08T08:55:03.363 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.363 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.363 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:03.363 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541647+0000", 2024-05-08T08:55:03.363 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6924999999999998e-05 2024-05-08T08:55:03.364 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.364 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.364 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:03.364 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541755+0000", 2024-05-08T08:55:03.364 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000108682 2024-05-08T08:55:03.364 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:03.364 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:03.364 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:03.364 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:03.364 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:03.364 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:55:03.365 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:55:03.366 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:55:03.368+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:55:03.388 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4] out_osds: [5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-05-08T08:55:03.388 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T08:55:03.388 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-05-08T08:55:03.388 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-05-08T08:55:03.407 INFO:teuthology.orchestra.run.smithi059.stderr:nodeep-scrub is set 2024-05-08T08:55:03.466 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:55:03.468+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:55:03.490 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:55:03.490 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:55:03.490 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:55:03.490 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:55:03.500 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:55:03.567 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:55:03.569+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:55:03.646 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:55:03.646 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:55:03.646 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:55:03.646 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:55:03.646 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:55:03.654 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:55:03.698 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:55:03.701+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:55:03.709 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:55:03.709 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:55:03.709 INFO:teuthology.orchestra.run.smithi059.stdout: "epoch": 152, 2024-05-08T08:55:03.709 INFO:teuthology.orchestra.run.smithi059.stdout: "fsid": "38dfae8c-5c5e-4b0f-a264-91f22c6c5554", 2024-05-08T08:55:03.710 INFO:teuthology.orchestra.run.smithi059.stdout: "created": "2024-05-08T08:51:45.571369+0000", 2024-05-08T08:55:03.710 INFO:teuthology.orchestra.run.smithi059.stdout: "modified": "2024-05-08T08:55:03.405154+0000", 2024-05-08T08:55:03.710 INFO:teuthology.orchestra.run.smithi059.stdout: "last_up_change": "2024-05-08T08:51:56.847706+0000", 2024-05-08T08:55:03.710 INFO:teuthology.orchestra.run.smithi059.stdout: "last_in_change": "2024-05-08T08:54:51.397440+0000", 2024-05-08T08:55:03.710 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-05-08T08:55:03.710 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_num": 5806080, 2024-05-08T08:55:03.710 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_set": [ 2024-05-08T08:55:03.710 INFO:teuthology.orchestra.run.smithi059.stdout: "nodeep-scrub", 2024-05-08T08:55:03.710 INFO:teuthology.orchestra.run.smithi059.stdout: "noscrub", 2024-05-08T08:55:03.710 INFO:teuthology.orchestra.run.smithi059.stdout: "pglog_hardlimit", 2024-05-08T08:55:03.710 INFO:teuthology.orchestra.run.smithi059.stdout: "purged_snapdirs", 2024-05-08T08:55:03.710 INFO:teuthology.orchestra.run.smithi059.stdout: "recovery_deletes", 2024-05-08T08:55:03.710 INFO:teuthology.orchestra.run.smithi059.stdout: "sortbitwise" 2024-05-08T08:55:03.711 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T08:55:03.711 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_version": 4, 2024-05-08T08:55:03.711 INFO:teuthology.orchestra.run.smithi059.stdout: "full_ratio": 0.94999998807907104, 2024-05-08T08:55:03.711 INFO:teuthology.orchestra.run.smithi059.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-05-08T08:55:03.711 INFO:teuthology.orchestra.run.smithi059.stdout: "nearfull_ratio": 0.85000002384185791, 2024-05-08T08:55:03.711 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_snapshot": "", 2024-05-08T08:55:03.711 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_max": 3, 2024-05-08T08:55:03.711 INFO:teuthology.orchestra.run.smithi059.stdout: "max_osd": 6, 2024-05-08T08:55:03.711 INFO:teuthology.orchestra.run.smithi059.stdout: "require_min_compat_client": "luminous", 2024-05-08T08:55:03.711 INFO:teuthology.orchestra.run.smithi059.stdout: "min_compat_client": "luminous", 2024-05-08T08:55:03.711 INFO:teuthology.orchestra.run.smithi059.stdout: "require_osd_release": "squid", 2024-05-08T08:55:03.711 INFO:teuthology.orchestra.run.smithi059.stdout: "allow_crimson": false, 2024-05-08T08:55:03.711 INFO:teuthology.orchestra.run.smithi059.stdout: "pools": [ 2024-05-08T08:55:03.711 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.711 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 1, 2024-05-08T08:55:03.711 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_name": ".mgr", 2024-05-08T08:55:03.712 INFO:teuthology.orchestra.run.smithi059.stdout: "create_time": "2024-05-08T08:51:57.787790+0000", 2024-05-08T08:55:03.712 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T08:55:03.712 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_names": "hashpspool", 2024-05-08T08:55:03.712 INFO:teuthology.orchestra.run.smithi059.stdout: "type": 1, 2024-05-08T08:55:03.712 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 2, 2024-05-08T08:55:03.712 INFO:teuthology.orchestra.run.smithi059.stdout: "min_size": 1, 2024-05-08T08:55:03.712 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_rule": 0, 2024-05-08T08:55:03.712 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_count": 0, 2024-05-08T08:55:03.712 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_target": 0, 2024-05-08T08:55:03.712 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_barrier": 0, 2024-05-08T08:55:03.712 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-08T08:55:03.712 INFO:teuthology.orchestra.run.smithi059.stdout: "object_hash": 2, 2024-05-08T08:55:03.712 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscale_mode": "off", 2024-05-08T08:55:03.712 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num": 1, 2024-05-08T08:55:03.712 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num": 1, 2024-05-08T08:55:03.712 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num_target": 1, 2024-05-08T08:55:03.713 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_target": 1, 2024-05-08T08:55:03.713 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_pending": 1, 2024-05-08T08:55:03.713 INFO:teuthology.orchestra.run.smithi059.stdout: "last_pg_merge_meta": { 2024-05-08T08:55:03.713 INFO:teuthology.orchestra.run.smithi059.stdout: "source_pgid": "0.0", 2024-05-08T08:55:03.713 INFO:teuthology.orchestra.run.smithi059.stdout: "ready_epoch": 0, 2024-05-08T08:55:03.713 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_started": 0, 2024-05-08T08:55:03.713 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_clean": 0, 2024-05-08T08:55:03.713 INFO:teuthology.orchestra.run.smithi059.stdout: "source_version": "0'0", 2024-05-08T08:55:03.713 INFO:teuthology.orchestra.run.smithi059.stdout: "target_version": "0'0" 2024-05-08T08:55:03.713 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.713 INFO:teuthology.orchestra.run.smithi059.stdout: "last_change": "14", 2024-05-08T08:55:03.713 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend": "0", 2024-05-08T08:55:03.713 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-08T08:55:03.713 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_preluminous": "0", 2024-05-08T08:55:03.714 INFO:teuthology.orchestra.run.smithi059.stdout: "auid": 0, 2024-05-08T08:55:03.714 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_mode": "selfmanaged", 2024-05-08T08:55:03.714 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_seq": 0, 2024-05-08T08:55:03.714 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_epoch": 0, 2024-05-08T08:55:03.714 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_snaps": [], 2024-05-08T08:55:03.714 INFO:teuthology.orchestra.run.smithi059.stdout: "removed_snaps": "[]", 2024-05-08T08:55:03.714 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_bytes": 0, 2024-05-08T08:55:03.714 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_objects": 0, 2024-05-08T08:55:03.714 INFO:teuthology.orchestra.run.smithi059.stdout: "tiers": [], 2024-05-08T08:55:03.714 INFO:teuthology.orchestra.run.smithi059.stdout: "tier_of": -1, 2024-05-08T08:55:03.714 INFO:teuthology.orchestra.run.smithi059.stdout: "read_tier": -1, 2024-05-08T08:55:03.714 INFO:teuthology.orchestra.run.smithi059.stdout: "write_tier": -1, 2024-05-08T08:55:03.714 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_mode": "none", 2024-05-08T08:55:03.714 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_bytes": 0, 2024-05-08T08:55:03.715 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_objects": 0, 2024-05-08T08:55:03.715 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-08T08:55:03.715 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-08T08:55:03.715 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-08T08:55:03.715 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_flush_age": 0, 2024-05-08T08:55:03.715 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_evict_age": 0, 2024-05-08T08:55:03.715 INFO:teuthology.orchestra.run.smithi059.stdout: "erasure_code_profile": "", 2024-05-08T08:55:03.715 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_params": { 2024-05-08T08:55:03.715 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "none" 2024-05-08T08:55:03.715 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.715 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_period": 0, 2024-05-08T08:55:03.715 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_count": 0, 2024-05-08T08:55:03.715 INFO:teuthology.orchestra.run.smithi059.stdout: "use_gmt_hitset": true, 2024-05-08T08:55:03.715 INFO:teuthology.orchestra.run.smithi059.stdout: "min_read_recency_for_promote": 0, 2024-05-08T08:55:03.715 INFO:teuthology.orchestra.run.smithi059.stdout: "min_write_recency_for_promote": 0, 2024-05-08T08:55:03.716 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_grade_decay_rate": 0, 2024-05-08T08:55:03.716 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_search_last_n": 0, 2024-05-08T08:55:03.716 INFO:teuthology.orchestra.run.smithi059.stdout: "grade_table": [], 2024-05-08T08:55:03.716 INFO:teuthology.orchestra.run.smithi059.stdout: "stripe_width": 0, 2024-05-08T08:55:03.716 INFO:teuthology.orchestra.run.smithi059.stdout: "expected_num_objects": 0, 2024-05-08T08:55:03.716 INFO:teuthology.orchestra.run.smithi059.stdout: "fast_read": false, 2024-05-08T08:55:03.716 INFO:teuthology.orchestra.run.smithi059.stdout: "options": { 2024-05-08T08:55:03.716 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_max": 32, 2024-05-08T08:55:03.716 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_min": 1 2024-05-08T08:55:03.716 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.716 INFO:teuthology.orchestra.run.smithi059.stdout: "application_metadata": { 2024-05-08T08:55:03.716 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": {} 2024-05-08T08:55:03.716 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.716 INFO:teuthology.orchestra.run.smithi059.stdout: "read_balance": { 2024-05-08T08:55:03.716 INFO:teuthology.orchestra.run.smithi059.stdout: "score_type": "Fair distribution", 2024-05-08T08:55:03.716 INFO:teuthology.orchestra.run.smithi059.stdout: "score_acting": 5, 2024-05-08T08:55:03.717 INFO:teuthology.orchestra.run.smithi059.stdout: "score_stable": 5, 2024-05-08T08:55:03.717 INFO:teuthology.orchestra.run.smithi059.stdout: "optimal_score": 0.40000000596046448, 2024-05-08T08:55:03.717 INFO:teuthology.orchestra.run.smithi059.stdout: "raw_score_acting": 2, 2024-05-08T08:55:03.717 INFO:teuthology.orchestra.run.smithi059.stdout: "raw_score_stable": 2, 2024-05-08T08:55:03.717 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity_weighted": 0.71152502298355103, 2024-05-08T08:55:03.717 INFO:teuthology.orchestra.run.smithi059.stdout: "average_primary_affinity": 0.85000002384185791, 2024-05-08T08:55:03.717 INFO:teuthology.orchestra.run.smithi059.stdout: "average_primary_affinity_weighted": 0.70999997854232788 2024-05-08T08:55:03.717 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:03.717 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.717 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.717 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 2, 2024-05-08T08:55:03.717 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_name": "rbd", 2024-05-08T08:55:03.717 INFO:teuthology.orchestra.run.smithi059.stdout: "create_time": "2024-05-08T08:52:02.563677+0000", 2024-05-08T08:55:03.717 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T08:55:03.717 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_names": "hashpspool", 2024-05-08T08:55:03.717 INFO:teuthology.orchestra.run.smithi059.stdout: "type": 1, 2024-05-08T08:55:03.718 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 2, 2024-05-08T08:55:03.718 INFO:teuthology.orchestra.run.smithi059.stdout: "min_size": 1, 2024-05-08T08:55:03.718 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_rule": 0, 2024-05-08T08:55:03.718 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_count": 0, 2024-05-08T08:55:03.718 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_target": 0, 2024-05-08T08:55:03.718 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_barrier": 0, 2024-05-08T08:55:03.718 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-08T08:55:03.718 INFO:teuthology.orchestra.run.smithi059.stdout: "object_hash": 2, 2024-05-08T08:55:03.718 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscale_mode": "off", 2024-05-08T08:55:03.718 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num": 8, 2024-05-08T08:55:03.718 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num": 8, 2024-05-08T08:55:03.718 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num_target": 8, 2024-05-08T08:55:03.718 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_target": 8, 2024-05-08T08:55:03.718 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_pending": 8, 2024-05-08T08:55:03.718 INFO:teuthology.orchestra.run.smithi059.stdout: "last_pg_merge_meta": { 2024-05-08T08:55:03.719 INFO:teuthology.orchestra.run.smithi059.stdout: "source_pgid": "0.0", 2024-05-08T08:55:03.719 INFO:teuthology.orchestra.run.smithi059.stdout: "ready_epoch": 0, 2024-05-08T08:55:03.719 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_started": 0, 2024-05-08T08:55:03.719 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_clean": 0, 2024-05-08T08:55:03.719 INFO:teuthology.orchestra.run.smithi059.stdout: "source_version": "0'0", 2024-05-08T08:55:03.719 INFO:teuthology.orchestra.run.smithi059.stdout: "target_version": "0'0" 2024-05-08T08:55:03.719 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.719 INFO:teuthology.orchestra.run.smithi059.stdout: "last_change": "17", 2024-05-08T08:55:03.719 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend": "0", 2024-05-08T08:55:03.719 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-08T08:55:03.719 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_preluminous": "0", 2024-05-08T08:55:03.719 INFO:teuthology.orchestra.run.smithi059.stdout: "auid": 0, 2024-05-08T08:55:03.719 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_mode": "selfmanaged", 2024-05-08T08:55:03.719 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_seq": 0, 2024-05-08T08:55:03.719 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_epoch": 0, 2024-05-08T08:55:03.719 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_snaps": [], 2024-05-08T08:55:03.720 INFO:teuthology.orchestra.run.smithi059.stdout: "removed_snaps": "[]", 2024-05-08T08:55:03.720 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_bytes": 0, 2024-05-08T08:55:03.720 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_objects": 0, 2024-05-08T08:55:03.720 INFO:teuthology.orchestra.run.smithi059.stdout: "tiers": [], 2024-05-08T08:55:03.720 INFO:teuthology.orchestra.run.smithi059.stdout: "tier_of": -1, 2024-05-08T08:55:03.720 INFO:teuthology.orchestra.run.smithi059.stdout: "read_tier": -1, 2024-05-08T08:55:03.720 INFO:teuthology.orchestra.run.smithi059.stdout: "write_tier": -1, 2024-05-08T08:55:03.720 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_mode": "none", 2024-05-08T08:55:03.720 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_bytes": 0, 2024-05-08T08:55:03.720 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_objects": 0, 2024-05-08T08:55:03.720 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-08T08:55:03.720 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-08T08:55:03.720 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-08T08:55:03.720 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_flush_age": 0, 2024-05-08T08:55:03.720 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_evict_age": 0, 2024-05-08T08:55:03.720 INFO:teuthology.orchestra.run.smithi059.stdout: "erasure_code_profile": "", 2024-05-08T08:55:03.720 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_params": { 2024-05-08T08:55:03.721 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "none" 2024-05-08T08:55:03.721 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.721 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_period": 0, 2024-05-08T08:55:03.721 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_count": 0, 2024-05-08T08:55:03.721 INFO:teuthology.orchestra.run.smithi059.stdout: "use_gmt_hitset": true, 2024-05-08T08:55:03.721 INFO:teuthology.orchestra.run.smithi059.stdout: "min_read_recency_for_promote": 0, 2024-05-08T08:55:03.721 INFO:teuthology.orchestra.run.smithi059.stdout: "min_write_recency_for_promote": 0, 2024-05-08T08:55:03.721 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_grade_decay_rate": 0, 2024-05-08T08:55:03.721 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_search_last_n": 0, 2024-05-08T08:55:03.721 INFO:teuthology.orchestra.run.smithi059.stdout: "grade_table": [], 2024-05-08T08:55:03.721 INFO:teuthology.orchestra.run.smithi059.stdout: "stripe_width": 0, 2024-05-08T08:55:03.721 INFO:teuthology.orchestra.run.smithi059.stdout: "expected_num_objects": 0, 2024-05-08T08:55:03.721 INFO:teuthology.orchestra.run.smithi059.stdout: "fast_read": false, 2024-05-08T08:55:03.721 INFO:teuthology.orchestra.run.smithi059.stdout: "options": {}, 2024-05-08T08:55:03.721 INFO:teuthology.orchestra.run.smithi059.stdout: "application_metadata": { 2024-05-08T08:55:03.721 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd": {} 2024-05-08T08:55:03.722 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.722 INFO:teuthology.orchestra.run.smithi059.stdout: "read_balance": { 2024-05-08T08:55:03.722 INFO:teuthology.orchestra.run.smithi059.stdout: "score_type": "Fair distribution", 2024-05-08T08:55:03.722 INFO:teuthology.orchestra.run.smithi059.stdout: "score_acting": 1.8799999952316284, 2024-05-08T08:55:03.722 INFO:teuthology.orchestra.run.smithi059.stdout: "score_stable": 1.8799999952316284, 2024-05-08T08:55:03.722 INFO:teuthology.orchestra.run.smithi059.stdout: "optimal_score": 1, 2024-05-08T08:55:03.722 INFO:teuthology.orchestra.run.smithi059.stdout: "raw_score_acting": 1.8799999952316284, 2024-05-08T08:55:03.722 INFO:teuthology.orchestra.run.smithi059.stdout: "raw_score_stable": 1.8799999952316284, 2024-05-08T08:55:03.722 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity_weighted": 0.71152502298355103, 2024-05-08T08:55:03.722 INFO:teuthology.orchestra.run.smithi059.stdout: "average_primary_affinity": 0.85000002384185791, 2024-05-08T08:55:03.722 INFO:teuthology.orchestra.run.smithi059.stdout: "average_primary_affinity_weighted": 0.70999997854232788 2024-05-08T08:55:03.722 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:03.722 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.722 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.722 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 3, 2024-05-08T08:55:03.723 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_name": "pool_client.0", 2024-05-08T08:55:03.723 INFO:teuthology.orchestra.run.smithi059.stdout: "create_time": "2024-05-08T08:52:09.955467+0000", 2024-05-08T08:55:03.723 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 8193, 2024-05-08T08:55:03.723 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-05-08T08:55:03.723 INFO:teuthology.orchestra.run.smithi059.stdout: "type": 1, 2024-05-08T08:55:03.723 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 2, 2024-05-08T08:55:03.723 INFO:teuthology.orchestra.run.smithi059.stdout: "min_size": 1, 2024-05-08T08:55:03.723 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_rule": 0, 2024-05-08T08:55:03.723 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_count": 0, 2024-05-08T08:55:03.723 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_target": 0, 2024-05-08T08:55:03.723 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_barrier": 0, 2024-05-08T08:55:03.723 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-08T08:55:03.723 INFO:teuthology.orchestra.run.smithi059.stdout: "object_hash": 2, 2024-05-08T08:55:03.723 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscale_mode": "off", 2024-05-08T08:55:03.723 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num": 32, 2024-05-08T08:55:03.723 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num": 32, 2024-05-08T08:55:03.724 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num_target": 32, 2024-05-08T08:55:03.724 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_target": 32, 2024-05-08T08:55:03.724 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_pending": 32, 2024-05-08T08:55:03.724 INFO:teuthology.orchestra.run.smithi059.stdout: "last_pg_merge_meta": { 2024-05-08T08:55:03.724 INFO:teuthology.orchestra.run.smithi059.stdout: "source_pgid": "0.0", 2024-05-08T08:55:03.724 INFO:teuthology.orchestra.run.smithi059.stdout: "ready_epoch": 0, 2024-05-08T08:55:03.724 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_started": 0, 2024-05-08T08:55:03.724 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_clean": 0, 2024-05-08T08:55:03.724 INFO:teuthology.orchestra.run.smithi059.stdout: "source_version": "0'0", 2024-05-08T08:55:03.724 INFO:teuthology.orchestra.run.smithi059.stdout: "target_version": "0'0" 2024-05-08T08:55:03.724 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.724 INFO:teuthology.orchestra.run.smithi059.stdout: "last_change": "152", 2024-05-08T08:55:03.724 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend": "0", 2024-05-08T08:55:03.724 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-08T08:55:03.725 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_preluminous": "0", 2024-05-08T08:55:03.725 INFO:teuthology.orchestra.run.smithi059.stdout: "auid": 0, 2024-05-08T08:55:03.725 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_mode": "selfmanaged", 2024-05-08T08:55:03.725 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_seq": 51, 2024-05-08T08:55:03.725 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_epoch": 152, 2024-05-08T08:55:03.725 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_snaps": [], 2024-05-08T08:55:03.725 INFO:teuthology.orchestra.run.smithi059.stdout: "removed_snaps": "[]", 2024-05-08T08:55:03.725 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_bytes": 0, 2024-05-08T08:55:03.725 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_objects": 0, 2024-05-08T08:55:03.725 INFO:teuthology.orchestra.run.smithi059.stdout: "tiers": [], 2024-05-08T08:55:03.725 INFO:teuthology.orchestra.run.smithi059.stdout: "tier_of": -1, 2024-05-08T08:55:03.725 INFO:teuthology.orchestra.run.smithi059.stdout: "read_tier": -1, 2024-05-08T08:55:03.725 INFO:teuthology.orchestra.run.smithi059.stdout: "write_tier": -1, 2024-05-08T08:55:03.725 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_mode": "none", 2024-05-08T08:55:03.725 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_bytes": 0, 2024-05-08T08:55:03.725 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_objects": 0, 2024-05-08T08:55:03.726 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-08T08:55:03.726 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-08T08:55:03.726 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-08T08:55:03.726 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_flush_age": 0, 2024-05-08T08:55:03.726 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_evict_age": 0, 2024-05-08T08:55:03.726 INFO:teuthology.orchestra.run.smithi059.stdout: "erasure_code_profile": "", 2024-05-08T08:55:03.726 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_params": { 2024-05-08T08:55:03.726 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "none" 2024-05-08T08:55:03.726 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.726 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_period": 0, 2024-05-08T08:55:03.726 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_count": 0, 2024-05-08T08:55:03.726 INFO:teuthology.orchestra.run.smithi059.stdout: "use_gmt_hitset": true, 2024-05-08T08:55:03.726 INFO:teuthology.orchestra.run.smithi059.stdout: "min_read_recency_for_promote": 0, 2024-05-08T08:55:03.726 INFO:teuthology.orchestra.run.smithi059.stdout: "min_write_recency_for_promote": 0, 2024-05-08T08:55:03.726 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_grade_decay_rate": 0, 2024-05-08T08:55:03.727 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_search_last_n": 0, 2024-05-08T08:55:03.727 INFO:teuthology.orchestra.run.smithi059.stdout: "grade_table": [], 2024-05-08T08:55:03.727 INFO:teuthology.orchestra.run.smithi059.stdout: "stripe_width": 0, 2024-05-08T08:55:03.727 INFO:teuthology.orchestra.run.smithi059.stdout: "expected_num_objects": 0, 2024-05-08T08:55:03.727 INFO:teuthology.orchestra.run.smithi059.stdout: "fast_read": false, 2024-05-08T08:55:03.727 INFO:teuthology.orchestra.run.smithi059.stdout: "options": {}, 2024-05-08T08:55:03.727 INFO:teuthology.orchestra.run.smithi059.stdout: "application_metadata": { 2024-05-08T08:55:03.727 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd": {} 2024-05-08T08:55:03.727 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.727 INFO:teuthology.orchestra.run.smithi059.stdout: "read_balance": { 2024-05-08T08:55:03.727 INFO:teuthology.orchestra.run.smithi059.stdout: "score_type": "Fair distribution", 2024-05-08T08:55:03.727 INFO:teuthology.orchestra.run.smithi059.stdout: "score_acting": 1.7200000286102295, 2024-05-08T08:55:03.727 INFO:teuthology.orchestra.run.smithi059.stdout: "score_stable": 1.7200000286102295, 2024-05-08T08:55:03.727 INFO:teuthology.orchestra.run.smithi059.stdout: "optimal_score": 1, 2024-05-08T08:55:03.727 INFO:teuthology.orchestra.run.smithi059.stdout: "raw_score_acting": 1.7200000286102295, 2024-05-08T08:55:03.727 INFO:teuthology.orchestra.run.smithi059.stdout: "raw_score_stable": 1.7200000286102295, 2024-05-08T08:55:03.728 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity_weighted": 0.71152502298355103, 2024-05-08T08:55:03.728 INFO:teuthology.orchestra.run.smithi059.stdout: "average_primary_affinity": 0.85000002384185791, 2024-05-08T08:55:03.728 INFO:teuthology.orchestra.run.smithi059.stdout: "average_primary_affinity_weighted": 0.70999997854232788 2024-05-08T08:55:03.728 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:03.728 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:03.728 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T08:55:03.728 INFO:teuthology.orchestra.run.smithi059.stdout: "osds": [ 2024-05-08T08:55:03.728 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.728 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 0, 2024-05-08T08:55:03.728 INFO:teuthology.orchestra.run.smithi059.stdout: "uuid": "300801a5-1919-4b23-8c6d-bc7dad61cc4d", 2024-05-08T08:55:03.728 INFO:teuthology.orchestra.run.smithi059.stdout: "up": 1, 2024-05-08T08:55:03.728 INFO:teuthology.orchestra.run.smithi059.stdout: "in": 1, 2024-05-08T08:55:03.728 INFO:teuthology.orchestra.run.smithi059.stdout: "weight": 1, 2024-05-08T08:55:03.728 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity": 0, 2024-05-08T08:55:03.728 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_begin": 0, 2024-05-08T08:55:03.728 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_end": 0, 2024-05-08T08:55:03.729 INFO:teuthology.orchestra.run.smithi059.stdout: "up_from": 11, 2024-05-08T08:55:03.729 INFO:teuthology.orchestra.run.smithi059.stdout: "up_thru": 144, 2024-05-08T08:55:03.729 INFO:teuthology.orchestra.run.smithi059.stdout: "down_at": 0, 2024-05-08T08:55:03.729 INFO:teuthology.orchestra.run.smithi059.stdout: "lost_at": 0, 2024-05-08T08:55:03.729 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrs": { 2024-05-08T08:55:03.729 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:55:03.729 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.729 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:55:03.729 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6800", 2024-05-08T08:55:03.729 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 802160151 2024-05-08T08:55:03.729 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.729 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.729 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:55:03.729 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6801", 2024-05-08T08:55:03.729 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 802160151 2024-05-08T08:55:03.729 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:03.730 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:03.730 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.730 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addrs": { 2024-05-08T08:55:03.730 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:55:03.730 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.730 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:55:03.730 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6802", 2024-05-08T08:55:03.730 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 802160151 2024-05-08T08:55:03.730 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.730 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.730 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:55:03.730 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6803", 2024-05-08T08:55:03.730 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 802160151 2024-05-08T08:55:03.730 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:03.731 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:03.731 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.731 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addrs": { 2024-05-08T08:55:03.731 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:55:03.731 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.731 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:55:03.731 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6806", 2024-05-08T08:55:03.731 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 802160151 2024-05-08T08:55:03.731 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.731 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.731 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:55:03.731 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6807", 2024-05-08T08:55:03.731 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 802160151 2024-05-08T08:55:03.731 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:03.731 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:03.732 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.732 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addrs": { 2024-05-08T08:55:03.732 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:55:03.732 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.732 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:55:03.732 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6804", 2024-05-08T08:55:03.732 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 802160151 2024-05-08T08:55:03.732 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.732 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.732 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:55:03.732 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6805", 2024-05-08T08:55:03.732 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 802160151 2024-05-08T08:55:03.732 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:03.732 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:03.732 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.732 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "172.21.15.59:6801/802160151", 2024-05-08T08:55:03.732 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "172.21.15.59:6803/802160151", 2024-05-08T08:55:03.733 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addr": "172.21.15.59:6807/802160151", 2024-05-08T08:55:03.733 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addr": "172.21.15.59:6805/802160151", 2024-05-08T08:55:03.733 INFO:teuthology.orchestra.run.smithi059.stdout: "state": [ 2024-05-08T08:55:03.733 INFO:teuthology.orchestra.run.smithi059.stdout: "exists", 2024-05-08T08:55:03.733 INFO:teuthology.orchestra.run.smithi059.stdout: "up" 2024-05-08T08:55:03.733 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:03.733 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.733 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.733 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 1, 2024-05-08T08:55:03.733 INFO:teuthology.orchestra.run.smithi059.stdout: "uuid": "0fead0de-6a65-4eba-b400-eea5692702e0", 2024-05-08T08:55:03.733 INFO:teuthology.orchestra.run.smithi059.stdout: "up": 1, 2024-05-08T08:55:03.733 INFO:teuthology.orchestra.run.smithi059.stdout: "in": 1, 2024-05-08T08:55:03.733 INFO:teuthology.orchestra.run.smithi059.stdout: "weight": 1, 2024-05-08T08:55:03.733 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity": 1, 2024-05-08T08:55:03.733 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_begin": 0, 2024-05-08T08:55:03.733 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_end": 0, 2024-05-08T08:55:03.734 INFO:teuthology.orchestra.run.smithi059.stdout: "up_from": 11, 2024-05-08T08:55:03.734 INFO:teuthology.orchestra.run.smithi059.stdout: "up_thru": 144, 2024-05-08T08:55:03.734 INFO:teuthology.orchestra.run.smithi059.stdout: "down_at": 0, 2024-05-08T08:55:03.734 INFO:teuthology.orchestra.run.smithi059.stdout: "lost_at": 0, 2024-05-08T08:55:03.734 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrs": { 2024-05-08T08:55:03.734 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:55:03.734 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.734 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:55:03.734 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6816", 2024-05-08T08:55:03.734 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4116801011 2024-05-08T08:55:03.734 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.734 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.734 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:55:03.734 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6817", 2024-05-08T08:55:03.735 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4116801011 2024-05-08T08:55:03.735 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:03.735 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:03.735 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.735 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addrs": { 2024-05-08T08:55:03.735 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:55:03.735 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.735 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:55:03.735 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6818", 2024-05-08T08:55:03.735 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4116801011 2024-05-08T08:55:03.735 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.735 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.735 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:55:03.735 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6819", 2024-05-08T08:55:03.735 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4116801011 2024-05-08T08:55:03.735 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:03.736 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:03.736 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.736 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addrs": { 2024-05-08T08:55:03.736 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:55:03.736 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.736 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:55:03.736 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6822", 2024-05-08T08:55:03.736 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4116801011 2024-05-08T08:55:03.736 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.736 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.736 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:55:03.736 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6823", 2024-05-08T08:55:03.736 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4116801011 2024-05-08T08:55:03.736 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:03.736 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:03.736 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.737 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addrs": { 2024-05-08T08:55:03.737 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:55:03.737 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.737 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:55:03.737 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6820", 2024-05-08T08:55:03.737 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4116801011 2024-05-08T08:55:03.737 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.737 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.737 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:55:03.737 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6821", 2024-05-08T08:55:03.737 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4116801011 2024-05-08T08:55:03.737 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:03.737 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:03.737 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.737 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "172.21.15.59:6817/4116801011", 2024-05-08T08:55:03.737 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "172.21.15.59:6819/4116801011", 2024-05-08T08:55:03.738 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addr": "172.21.15.59:6823/4116801011", 2024-05-08T08:55:03.738 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addr": "172.21.15.59:6821/4116801011", 2024-05-08T08:55:03.738 INFO:teuthology.orchestra.run.smithi059.stdout: "state": [ 2024-05-08T08:55:03.738 INFO:teuthology.orchestra.run.smithi059.stdout: "exists", 2024-05-08T08:55:03.738 INFO:teuthology.orchestra.run.smithi059.stdout: "up" 2024-05-08T08:55:03.738 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:03.738 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.738 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.738 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 2, 2024-05-08T08:55:03.738 INFO:teuthology.orchestra.run.smithi059.stdout: "uuid": "1a233de6-1b82-4a51-8e96-4cd4abdf16c8", 2024-05-08T08:55:03.738 INFO:teuthology.orchestra.run.smithi059.stdout: "up": 1, 2024-05-08T08:55:03.738 INFO:teuthology.orchestra.run.smithi059.stdout: "in": 1, 2024-05-08T08:55:03.738 INFO:teuthology.orchestra.run.smithi059.stdout: "weight": 1, 2024-05-08T08:55:03.738 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity": 1, 2024-05-08T08:55:03.738 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_begin": 0, 2024-05-08T08:55:03.738 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_end": 0, 2024-05-08T08:55:03.739 INFO:teuthology.orchestra.run.smithi059.stdout: "up_from": 11, 2024-05-08T08:55:03.739 INFO:teuthology.orchestra.run.smithi059.stdout: "up_thru": 144, 2024-05-08T08:55:03.739 INFO:teuthology.orchestra.run.smithi059.stdout: "down_at": 0, 2024-05-08T08:55:03.739 INFO:teuthology.orchestra.run.smithi059.stdout: "lost_at": 0, 2024-05-08T08:55:03.739 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrs": { 2024-05-08T08:55:03.739 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:55:03.739 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.739 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:55:03.739 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6808", 2024-05-08T08:55:03.739 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1547661179 2024-05-08T08:55:03.739 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.739 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.739 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:55:03.739 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6809", 2024-05-08T08:55:03.739 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1547661179 2024-05-08T08:55:03.740 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:03.740 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:03.740 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.740 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addrs": { 2024-05-08T08:55:03.740 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:55:03.740 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.740 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:55:03.740 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6810", 2024-05-08T08:55:03.740 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1547661179 2024-05-08T08:55:03.740 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.740 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.740 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:55:03.740 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6811", 2024-05-08T08:55:03.740 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1547661179 2024-05-08T08:55:03.740 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:03.740 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:03.741 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.741 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addrs": { 2024-05-08T08:55:03.741 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:55:03.741 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.741 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:55:03.741 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6814", 2024-05-08T08:55:03.741 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1547661179 2024-05-08T08:55:03.741 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.741 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.741 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:55:03.741 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6815", 2024-05-08T08:55:03.741 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1547661179 2024-05-08T08:55:03.741 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:03.741 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:03.741 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.741 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addrs": { 2024-05-08T08:55:03.742 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:55:03.742 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.742 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:55:03.742 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6812", 2024-05-08T08:55:03.742 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1547661179 2024-05-08T08:55:03.742 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.742 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.742 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:55:03.742 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6813", 2024-05-08T08:55:03.742 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1547661179 2024-05-08T08:55:03.742 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:03.742 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:03.742 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.742 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "172.21.15.59:6809/1547661179", 2024-05-08T08:55:03.742 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "172.21.15.59:6811/1547661179", 2024-05-08T08:55:03.743 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addr": "172.21.15.59:6815/1547661179", 2024-05-08T08:55:03.743 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addr": "172.21.15.59:6813/1547661179", 2024-05-08T08:55:03.743 INFO:teuthology.orchestra.run.smithi059.stdout: "state": [ 2024-05-08T08:55:03.743 INFO:teuthology.orchestra.run.smithi059.stdout: "exists", 2024-05-08T08:55:03.743 INFO:teuthology.orchestra.run.smithi059.stdout: "up" 2024-05-08T08:55:03.743 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:03.743 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.743 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.743 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 3, 2024-05-08T08:55:03.743 INFO:teuthology.orchestra.run.smithi059.stdout: "uuid": "cc4870b8-dcd9-4e43-978d-8de6c6396abf", 2024-05-08T08:55:03.743 INFO:teuthology.orchestra.run.smithi059.stdout: "up": 1, 2024-05-08T08:55:03.743 INFO:teuthology.orchestra.run.smithi059.stdout: "in": 1, 2024-05-08T08:55:03.743 INFO:teuthology.orchestra.run.smithi059.stdout: "weight": 1, 2024-05-08T08:55:03.743 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity": 1, 2024-05-08T08:55:03.743 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_begin": 0, 2024-05-08T08:55:03.743 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_end": 0, 2024-05-08T08:55:03.744 INFO:teuthology.orchestra.run.smithi059.stdout: "up_from": 11, 2024-05-08T08:55:03.744 INFO:teuthology.orchestra.run.smithi059.stdout: "up_thru": 147, 2024-05-08T08:55:03.744 INFO:teuthology.orchestra.run.smithi059.stdout: "down_at": 0, 2024-05-08T08:55:03.744 INFO:teuthology.orchestra.run.smithi059.stdout: "lost_at": 0, 2024-05-08T08:55:03.744 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrs": { 2024-05-08T08:55:03.744 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:55:03.744 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.744 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:55:03.744 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6800", 2024-05-08T08:55:03.744 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4054400810 2024-05-08T08:55:03.744 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.744 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.744 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:55:03.744 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6801", 2024-05-08T08:55:03.744 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4054400810 2024-05-08T08:55:03.744 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:03.745 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:03.745 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.745 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addrs": { 2024-05-08T08:55:03.745 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:55:03.745 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.745 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:55:03.745 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6802", 2024-05-08T08:55:03.745 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4054400810 2024-05-08T08:55:03.745 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.745 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.745 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:55:03.745 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6803", 2024-05-08T08:55:03.745 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4054400810 2024-05-08T08:55:03.745 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:03.745 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:03.745 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.746 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addrs": { 2024-05-08T08:55:03.746 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:55:03.746 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.746 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:55:03.746 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6806", 2024-05-08T08:55:03.746 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4054400810 2024-05-08T08:55:03.746 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.746 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.746 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:55:03.746 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6807", 2024-05-08T08:55:03.746 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4054400810 2024-05-08T08:55:03.746 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:03.746 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:03.746 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.746 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addrs": { 2024-05-08T08:55:03.747 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:55:03.747 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.747 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:55:03.747 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6804", 2024-05-08T08:55:03.747 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4054400810 2024-05-08T08:55:03.747 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.747 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.747 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:55:03.747 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6805", 2024-05-08T08:55:03.747 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4054400810 2024-05-08T08:55:03.747 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:03.747 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:03.747 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.747 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "172.21.15.192:6801/4054400810", 2024-05-08T08:55:03.747 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "172.21.15.192:6803/4054400810", 2024-05-08T08:55:03.747 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addr": "172.21.15.192:6807/4054400810", 2024-05-08T08:55:03.748 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addr": "172.21.15.192:6805/4054400810", 2024-05-08T08:55:03.748 INFO:teuthology.orchestra.run.smithi059.stdout: "state": [ 2024-05-08T08:55:03.748 INFO:teuthology.orchestra.run.smithi059.stdout: "exists", 2024-05-08T08:55:03.748 INFO:teuthology.orchestra.run.smithi059.stdout: "up" 2024-05-08T08:55:03.748 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:03.748 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.748 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.748 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 4, 2024-05-08T08:55:03.748 INFO:teuthology.orchestra.run.smithi059.stdout: "uuid": "d1d05620-42fb-4107-a2f9-e83542d076ee", 2024-05-08T08:55:03.748 INFO:teuthology.orchestra.run.smithi059.stdout: "up": 1, 2024-05-08T08:55:03.748 INFO:teuthology.orchestra.run.smithi059.stdout: "in": 1, 2024-05-08T08:55:03.748 INFO:teuthology.orchestra.run.smithi059.stdout: "weight": 1, 2024-05-08T08:55:03.748 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity": 0.9892578125, 2024-05-08T08:55:03.748 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_begin": 0, 2024-05-08T08:55:03.748 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_end": 0, 2024-05-08T08:55:03.748 INFO:teuthology.orchestra.run.smithi059.stdout: "up_from": 11, 2024-05-08T08:55:03.749 INFO:teuthology.orchestra.run.smithi059.stdout: "up_thru": 145, 2024-05-08T08:55:03.749 INFO:teuthology.orchestra.run.smithi059.stdout: "down_at": 0, 2024-05-08T08:55:03.749 INFO:teuthology.orchestra.run.smithi059.stdout: "lost_at": 0, 2024-05-08T08:55:03.749 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrs": { 2024-05-08T08:55:03.749 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:55:03.749 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.749 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:55:03.749 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6808", 2024-05-08T08:55:03.749 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 58985565 2024-05-08T08:55:03.749 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.749 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.749 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:55:03.749 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6809", 2024-05-08T08:55:03.749 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 58985565 2024-05-08T08:55:03.749 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:03.750 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:03.750 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.750 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addrs": { 2024-05-08T08:55:03.750 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:55:03.750 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.750 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:55:03.750 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6810", 2024-05-08T08:55:03.750 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 58985565 2024-05-08T08:55:03.750 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.750 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.750 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:55:03.750 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6811", 2024-05-08T08:55:03.750 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 58985565 2024-05-08T08:55:03.750 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:03.750 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:03.751 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.751 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addrs": { 2024-05-08T08:55:03.751 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:55:03.751 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.751 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:55:03.751 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6814", 2024-05-08T08:55:03.751 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 58985565 2024-05-08T08:55:03.751 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.751 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.751 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:55:03.751 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6815", 2024-05-08T08:55:03.751 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 58985565 2024-05-08T08:55:03.751 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:03.751 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:03.751 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.751 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addrs": { 2024-05-08T08:55:03.752 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:55:03.752 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.752 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:55:03.752 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6812", 2024-05-08T08:55:03.752 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 58985565 2024-05-08T08:55:03.752 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.752 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.752 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:55:03.752 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6813", 2024-05-08T08:55:03.752 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 58985565 2024-05-08T08:55:03.752 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:03.752 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:03.752 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.752 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "172.21.15.192:6809/58985565", 2024-05-08T08:55:03.753 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "172.21.15.192:6811/58985565", 2024-05-08T08:55:03.753 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addr": "172.21.15.192:6815/58985565", 2024-05-08T08:55:03.753 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addr": "172.21.15.192:6813/58985565", 2024-05-08T08:55:03.753 INFO:teuthology.orchestra.run.smithi059.stdout: "state": [ 2024-05-08T08:55:03.753 INFO:teuthology.orchestra.run.smithi059.stdout: "exists", 2024-05-08T08:55:03.753 INFO:teuthology.orchestra.run.smithi059.stdout: "up" 2024-05-08T08:55:03.753 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:03.753 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.753 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.753 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 5, 2024-05-08T08:55:03.753 INFO:teuthology.orchestra.run.smithi059.stdout: "uuid": "ab26136c-31c3-4090-9fa2-988dba4f731f", 2024-05-08T08:55:03.753 INFO:teuthology.orchestra.run.smithi059.stdout: "up": 1, 2024-05-08T08:55:03.753 INFO:teuthology.orchestra.run.smithi059.stdout: "in": 0, 2024-05-08T08:55:03.753 INFO:teuthology.orchestra.run.smithi059.stdout: "weight": 0, 2024-05-08T08:55:03.753 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity": 0.2798919677734375, 2024-05-08T08:55:03.753 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_begin": 0, 2024-05-08T08:55:03.754 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_end": 0, 2024-05-08T08:55:03.754 INFO:teuthology.orchestra.run.smithi059.stdout: "up_from": 11, 2024-05-08T08:55:03.754 INFO:teuthology.orchestra.run.smithi059.stdout: "up_thru": 136, 2024-05-08T08:55:03.754 INFO:teuthology.orchestra.run.smithi059.stdout: "down_at": 0, 2024-05-08T08:55:03.754 INFO:teuthology.orchestra.run.smithi059.stdout: "lost_at": 0, 2024-05-08T08:55:03.754 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrs": { 2024-05-08T08:55:03.754 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:55:03.754 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.754 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:55:03.754 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6816", 2024-05-08T08:55:03.754 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1283571573 2024-05-08T08:55:03.754 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.754 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.754 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:55:03.754 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6817", 2024-05-08T08:55:03.754 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1283571573 2024-05-08T08:55:03.755 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:03.755 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:03.755 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.755 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addrs": { 2024-05-08T08:55:03.755 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:55:03.755 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.755 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:55:03.755 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6818", 2024-05-08T08:55:03.755 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1283571573 2024-05-08T08:55:03.755 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.755 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.755 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:55:03.755 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6819", 2024-05-08T08:55:03.755 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1283571573 2024-05-08T08:55:03.755 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:03.755 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:03.756 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.756 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addrs": { 2024-05-08T08:55:03.756 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:55:03.756 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.756 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:55:03.756 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6822", 2024-05-08T08:55:03.756 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1283571573 2024-05-08T08:55:03.756 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.756 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.756 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:55:03.756 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6823", 2024-05-08T08:55:03.756 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1283571573 2024-05-08T08:55:03.756 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:03.756 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:03.757 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.757 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addrs": { 2024-05-08T08:55:03.757 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T08:55:03.757 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.757 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T08:55:03.757 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6820", 2024-05-08T08:55:03.757 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1283571573 2024-05-08T08:55:03.757 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.757 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.757 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T08:55:03.757 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.192:6821", 2024-05-08T08:55:03.757 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1283571573 2024-05-08T08:55:03.757 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:03.757 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:03.757 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.757 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "172.21.15.192:6817/1283571573", 2024-05-08T08:55:03.758 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "172.21.15.192:6819/1283571573", 2024-05-08T08:55:03.758 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addr": "172.21.15.192:6823/1283571573", 2024-05-08T08:55:03.758 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addr": "172.21.15.192:6821/1283571573", 2024-05-08T08:55:03.758 INFO:teuthology.orchestra.run.smithi059.stdout: "state": [ 2024-05-08T08:55:03.758 INFO:teuthology.orchestra.run.smithi059.stdout: "exists", 2024-05-08T08:55:03.758 INFO:teuthology.orchestra.run.smithi059.stdout: "up" 2024-05-08T08:55:03.758 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:03.758 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:03.758 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T08:55:03.758 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_xinfo": [ 2024-05-08T08:55:03.758 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.758 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 0, 2024-05-08T08:55:03.758 INFO:teuthology.orchestra.run.smithi059.stdout: "down_stamp": "0.000000", 2024-05-08T08:55:03.758 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_probability": 0, 2024-05-08T08:55:03.758 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_interval": 0, 2024-05-08T08:55:03.758 INFO:teuthology.orchestra.run.smithi059.stdout: "features": 4540701547738038271, 2024-05-08T08:55:03.759 INFO:teuthology.orchestra.run.smithi059.stdout: "old_weight": 0, 2024-05-08T08:55:03.759 INFO:teuthology.orchestra.run.smithi059.stdout: "last_purged_snaps_scrub": "2024-05-08T08:51:52.396946+0000", 2024-05-08T08:55:03.759 INFO:teuthology.orchestra.run.smithi059.stdout: "dead_epoch": 0 2024-05-08T08:55:03.759 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.759 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.759 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 1, 2024-05-08T08:55:03.759 INFO:teuthology.orchestra.run.smithi059.stdout: "down_stamp": "0.000000", 2024-05-08T08:55:03.759 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_probability": 0, 2024-05-08T08:55:03.759 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_interval": 0, 2024-05-08T08:55:03.759 INFO:teuthology.orchestra.run.smithi059.stdout: "features": 4540701547738038271, 2024-05-08T08:55:03.759 INFO:teuthology.orchestra.run.smithi059.stdout: "old_weight": 0, 2024-05-08T08:55:03.759 INFO:teuthology.orchestra.run.smithi059.stdout: "last_purged_snaps_scrub": "2024-05-08T08:51:52.453160+0000", 2024-05-08T08:55:03.759 INFO:teuthology.orchestra.run.smithi059.stdout: "dead_epoch": 0 2024-05-08T08:55:03.759 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.760 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.760 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 2, 2024-05-08T08:55:03.760 INFO:teuthology.orchestra.run.smithi059.stdout: "down_stamp": "0.000000", 2024-05-08T08:55:03.760 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_probability": 0, 2024-05-08T08:55:03.760 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_interval": 0, 2024-05-08T08:55:03.760 INFO:teuthology.orchestra.run.smithi059.stdout: "features": 4540701547738038271, 2024-05-08T08:55:03.760 INFO:teuthology.orchestra.run.smithi059.stdout: "old_weight": 0, 2024-05-08T08:55:03.760 INFO:teuthology.orchestra.run.smithi059.stdout: "last_purged_snaps_scrub": "2024-05-08T08:51:52.387606+0000", 2024-05-08T08:55:03.760 INFO:teuthology.orchestra.run.smithi059.stdout: "dead_epoch": 0 2024-05-08T08:55:03.760 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.760 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.760 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 3, 2024-05-08T08:55:03.760 INFO:teuthology.orchestra.run.smithi059.stdout: "down_stamp": "0.000000", 2024-05-08T08:55:03.760 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_probability": 0, 2024-05-08T08:55:03.760 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_interval": 0, 2024-05-08T08:55:03.761 INFO:teuthology.orchestra.run.smithi059.stdout: "features": 4540701547738038271, 2024-05-08T08:55:03.761 INFO:teuthology.orchestra.run.smithi059.stdout: "old_weight": 0, 2024-05-08T08:55:03.761 INFO:teuthology.orchestra.run.smithi059.stdout: "last_purged_snaps_scrub": "2024-05-08T08:51:52.360952+0000", 2024-05-08T08:55:03.761 INFO:teuthology.orchestra.run.smithi059.stdout: "dead_epoch": 0 2024-05-08T08:55:03.761 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.761 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.761 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 4, 2024-05-08T08:55:03.761 INFO:teuthology.orchestra.run.smithi059.stdout: "down_stamp": "0.000000", 2024-05-08T08:55:03.761 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_probability": 0, 2024-05-08T08:55:03.761 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_interval": 0, 2024-05-08T08:55:03.761 INFO:teuthology.orchestra.run.smithi059.stdout: "features": 4540701547738038271, 2024-05-08T08:55:03.761 INFO:teuthology.orchestra.run.smithi059.stdout: "old_weight": 0, 2024-05-08T08:55:03.761 INFO:teuthology.orchestra.run.smithi059.stdout: "last_purged_snaps_scrub": "2024-05-08T08:51:52.468681+0000", 2024-05-08T08:55:03.761 INFO:teuthology.orchestra.run.smithi059.stdout: "dead_epoch": 0 2024-05-08T08:55:03.761 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.761 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.762 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 5, 2024-05-08T08:55:03.762 INFO:teuthology.orchestra.run.smithi059.stdout: "down_stamp": "0.000000", 2024-05-08T08:55:03.762 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_probability": 0, 2024-05-08T08:55:03.762 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_interval": 0, 2024-05-08T08:55:03.762 INFO:teuthology.orchestra.run.smithi059.stdout: "features": 4540701547738038271, 2024-05-08T08:55:03.762 INFO:teuthology.orchestra.run.smithi059.stdout: "old_weight": 65536, 2024-05-08T08:55:03.762 INFO:teuthology.orchestra.run.smithi059.stdout: "last_purged_snaps_scrub": "2024-05-08T08:51:52.401080+0000", 2024-05-08T08:55:03.762 INFO:teuthology.orchestra.run.smithi059.stdout: "dead_epoch": 0 2024-05-08T08:55:03.762 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:03.762 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T08:55:03.762 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_upmap": [ 2024-05-08T08:55:03.762 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.762 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "2.5", 2024-05-08T08:55:03.762 INFO:teuthology.orchestra.run.smithi059.stdout: "osds": [ 2024-05-08T08:55:03.762 INFO:teuthology.orchestra.run.smithi059.stdout: 3, 2024-05-08T08:55:03.763 INFO:teuthology.orchestra.run.smithi059.stdout: 2 2024-05-08T08:55:03.763 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:03.763 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.763 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.763 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "3.4", 2024-05-08T08:55:03.763 INFO:teuthology.orchestra.run.smithi059.stdout: "osds": [ 2024-05-08T08:55:03.763 INFO:teuthology.orchestra.run.smithi059.stdout: 1, 2024-05-08T08:55:03.763 INFO:teuthology.orchestra.run.smithi059.stdout: 5 2024-05-08T08:55:03.763 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:03.763 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.763 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.763 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "3.7", 2024-05-08T08:55:03.763 INFO:teuthology.orchestra.run.smithi059.stdout: "osds": [ 2024-05-08T08:55:03.763 INFO:teuthology.orchestra.run.smithi059.stdout: 4, 2024-05-08T08:55:03.763 INFO:teuthology.orchestra.run.smithi059.stdout: 3 2024-05-08T08:55:03.763 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:03.763 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.764 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.764 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "3.17", 2024-05-08T08:55:03.764 INFO:teuthology.orchestra.run.smithi059.stdout: "osds": [ 2024-05-08T08:55:03.764 INFO:teuthology.orchestra.run.smithi059.stdout: 1, 2024-05-08T08:55:03.764 INFO:teuthology.orchestra.run.smithi059.stdout: 0 2024-05-08T08:55:03.764 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:03.764 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.764 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.764 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "3.1d", 2024-05-08T08:55:03.764 INFO:teuthology.orchestra.run.smithi059.stdout: "osds": [ 2024-05-08T08:55:03.764 INFO:teuthology.orchestra.run.smithi059.stdout: 0, 2024-05-08T08:55:03.764 INFO:teuthology.orchestra.run.smithi059.stdout: 5 2024-05-08T08:55:03.764 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:03.764 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:03.764 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T08:55:03.765 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_upmap_items": [], 2024-05-08T08:55:03.765 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_upmap_primaries": [], 2024-05-08T08:55:03.765 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_temp": [], 2024-05-08T08:55:03.765 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_temp": [], 2024-05-08T08:55:03.765 INFO:teuthology.orchestra.run.smithi059.stdout: "blocklist": {}, 2024-05-08T08:55:03.765 INFO:teuthology.orchestra.run.smithi059.stdout: "range_blocklist": {}, 2024-05-08T08:55:03.765 INFO:teuthology.orchestra.run.smithi059.stdout: "erasure_code_profiles": { 2024-05-08T08:55:03.765 INFO:teuthology.orchestra.run.smithi059.stdout: "default": { 2024-05-08T08:55:03.765 INFO:teuthology.orchestra.run.smithi059.stdout: "crush-failure-domain": "osd", 2024-05-08T08:55:03.765 INFO:teuthology.orchestra.run.smithi059.stdout: "k": "2", 2024-05-08T08:55:03.765 INFO:teuthology.orchestra.run.smithi059.stdout: "m": "1", 2024-05-08T08:55:03.765 INFO:teuthology.orchestra.run.smithi059.stdout: "plugin": "jerasure", 2024-05-08T08:55:03.765 INFO:teuthology.orchestra.run.smithi059.stdout: "technique": "reed_sol_van" 2024-05-08T08:55:03.765 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:03.766 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:03.766 INFO:teuthology.orchestra.run.smithi059.stdout: "removed_snaps_queue": [ 2024-05-08T08:55:03.766 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.766 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 3, 2024-05-08T08:55:03.766 INFO:teuthology.orchestra.run.smithi059.stdout: "snaps": [ 2024-05-08T08:55:03.766 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.766 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 25, 2024-05-08T08:55:03.766 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 8 2024-05-08T08:55:03.766 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:03.766 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:03.766 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:03.766 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T08:55:03.766 INFO:teuthology.orchestra.run.smithi059.stdout: "new_removed_snaps": [ 2024-05-08T08:55:03.766 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.766 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 3, 2024-05-08T08:55:03.766 INFO:teuthology.orchestra.run.smithi059.stdout: "snaps": [ 2024-05-08T08:55:03.767 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:03.767 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 25, 2024-05-08T08:55:03.767 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 1 2024-05-08T08:55:03.767 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:03.767 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:03.767 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:03.767 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T08:55:03.767 INFO:teuthology.orchestra.run.smithi059.stdout: "new_purged_snaps": [], 2024-05-08T08:55:03.767 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_node_flags": {}, 2024-05-08T08:55:03.767 INFO:teuthology.orchestra.run.smithi059.stdout: "device_class_flags": {}, 2024-05-08T08:55:03.767 INFO:teuthology.orchestra.run.smithi059.stdout: "stretch_mode": { 2024-05-08T08:55:03.767 INFO:teuthology.orchestra.run.smithi059.stdout: "stretch_mode_enabled": false, 2024-05-08T08:55:03.767 INFO:teuthology.orchestra.run.smithi059.stdout: "stretch_bucket_count": 0, 2024-05-08T08:55:03.767 INFO:teuthology.orchestra.run.smithi059.stdout: "degraded_stretch_mode": 0, 2024-05-08T08:55:03.767 INFO:teuthology.orchestra.run.smithi059.stdout: "recovering_stretch_mode": 0, 2024-05-08T08:55:03.767 INFO:teuthology.orchestra.run.smithi059.stdout: "stretch_mode_bucket": 0 2024-05-08T08:55:03.768 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:03.768 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:55:03.769 INFO:tasks.thrashosds.thrasher:j is {'epoch': 152, 'fsid': '38dfae8c-5c5e-4b0f-a264-91f22c6c5554', 'created': '2024-05-08T08:51:45.571369+0000', 'modified': '2024-05-08T08:55:03.405154+0000', 'last_up_change': '2024-05-08T08:51:56.847706+0000', 'last_in_change': '2024-05-08T08:54:51.397440+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T08:51:57.787790+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '14', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 5, 'score_stable': 5, 'optimal_score': 0.4000000059604645, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.711525022983551, 'average_primary_affinity': 0.8500000238418579, 'average_primary_affinity_weighted': 0.7099999785423279}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-05-08T08:52:02.563677+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.711525022983551, 'average_primary_affinity': 0.8500000238418579, 'average_primary_affinity_weighted': 0.7099999785423279}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-05-08T08:52:09.955467+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '152', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 51, 'snap_epoch': 152, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.7200000286102295, 'score_stable': 1.7200000286102295, 'optimal_score': 1, 'raw_score_acting': 1.7200000286102295, 'raw_score_stable': 1.7200000286102295, 'primary_affinity_weighted': 0.711525022983551, 'average_primary_affinity': 0.8500000238418579, 'average_primary_affinity_weighted': 0.7099999785423279}}], 'osds': [{'osd': 0, 'uuid': '300801a5-1919-4b23-8c6d-bc7dad61cc4d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 144, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6800', 'nonce': 802160151}, {'type': 'v1', 'addr': '172.21.15.59:6801', 'nonce': 802160151}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6802', 'nonce': 802160151}, {'type': 'v1', 'addr': '172.21.15.59:6803', 'nonce': 802160151}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6806', 'nonce': 802160151}, {'type': 'v1', 'addr': '172.21.15.59:6807', 'nonce': 802160151}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6804', 'nonce': 802160151}, {'type': 'v1', 'addr': '172.21.15.59:6805', 'nonce': 802160151}]}, 'public_addr': '172.21.15.59:6801/802160151', 'cluster_addr': '172.21.15.59:6803/802160151', 'heartbeat_back_addr': '172.21.15.59:6807/802160151', 'heartbeat_front_addr': '172.21.15.59:6805/802160151', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '0fead0de-6a65-4eba-b400-eea5692702e0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 144, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6816', 'nonce': 4116801011}, {'type': 'v1', 'addr': '172.21.15.59:6817', 'nonce': 4116801011}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6818', 'nonce': 4116801011}, {'type': 'v1', 'addr': '172.21.15.59:6819', 'nonce': 4116801011}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6822', 'nonce': 4116801011}, {'type': 'v1', 'addr': '172.21.15.59:6823', 'nonce': 4116801011}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6820', 'nonce': 4116801011}, {'type': 'v1', 'addr': '172.21.15.59:6821', 'nonce': 4116801011}]}, 'public_addr': '172.21.15.59:6817/4116801011', 'cluster_addr': '172.21.15.59:6819/4116801011', 'heartbeat_back_addr': '172.21.15.59:6823/4116801011', 'heartbeat_front_addr': '172.21.15.59:6821/4116801011', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '1a233de6-1b82-4a51-8e96-4cd4abdf16c8', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 144, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6808', 'nonce': 1547661179}, {'type': 'v1', 'addr': '172.21.15.59:6809', 'nonce': 1547661179}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6810', 'nonce': 1547661179}, {'type': 'v1', 'addr': '172.21.15.59:6811', 'nonce': 1547661179}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6814', 'nonce': 1547661179}, {'type': 'v1', 'addr': '172.21.15.59:6815', 'nonce': 1547661179}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6812', 'nonce': 1547661179}, {'type': 'v1', 'addr': '172.21.15.59:6813', 'nonce': 1547661179}]}, 'public_addr': '172.21.15.59:6809/1547661179', 'cluster_addr': '172.21.15.59:6811/1547661179', 'heartbeat_back_addr': '172.21.15.59:6815/1547661179', 'heartbeat_front_addr': '172.21.15.59:6813/1547661179', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'cc4870b8-dcd9-4e43-978d-8de6c6396abf', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 147, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6800', 'nonce': 4054400810}, {'type': 'v1', 'addr': '172.21.15.192:6801', 'nonce': 4054400810}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6802', 'nonce': 4054400810}, {'type': 'v1', 'addr': '172.21.15.192:6803', 'nonce': 4054400810}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6806', 'nonce': 4054400810}, {'type': 'v1', 'addr': '172.21.15.192:6807', 'nonce': 4054400810}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6804', 'nonce': 4054400810}, {'type': 'v1', 'addr': '172.21.15.192:6805', 'nonce': 4054400810}]}, 'public_addr': '172.21.15.192:6801/4054400810', 'cluster_addr': '172.21.15.192:6803/4054400810', 'heartbeat_back_addr': '172.21.15.192:6807/4054400810', 'heartbeat_front_addr': '172.21.15.192:6805/4054400810', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'd1d05620-42fb-4107-a2f9-e83542d076ee', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.9892578125, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 145, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6808', 'nonce': 58985565}, {'type': 'v1', 'addr': '172.21.15.192:6809', 'nonce': 58985565}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6810', 'nonce': 58985565}, {'type': 'v1', 'addr': '172.21.15.192:6811', 'nonce': 58985565}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6814', 'nonce': 58985565}, {'type': 'v1', 'addr': '172.21.15.192:6815', 'nonce': 58985565}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6812', 'nonce': 58985565}, {'type': 'v1', 'addr': '172.21.15.192:6813', 'nonce': 58985565}]}, 'public_addr': '172.21.15.192:6809/58985565', 'cluster_addr': '172.21.15.192:6811/58985565', 'heartbeat_back_addr': '172.21.15.192:6815/58985565', 'heartbeat_front_addr': '172.21.15.192:6813/58985565', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'ab26136c-31c3-4090-9fa2-988dba4f731f', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 0.2798919677734375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 136, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6816', 'nonce': 1283571573}, {'type': 'v1', 'addr': '172.21.15.192:6817', 'nonce': 1283571573}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6818', 'nonce': 1283571573}, {'type': 'v1', 'addr': '172.21.15.192:6819', 'nonce': 1283571573}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6822', 'nonce': 1283571573}, {'type': 'v1', 'addr': '172.21.15.192:6823', 'nonce': 1283571573}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6820', 'nonce': 1283571573}, {'type': 'v1', 'addr': '172.21.15.192:6821', 'nonce': 1283571573}]}, 'public_addr': '172.21.15.192:6817/1283571573', 'cluster_addr': '172.21.15.192:6819/1283571573', 'heartbeat_back_addr': '172.21.15.192:6823/1283571573', 'heartbeat_front_addr': '172.21.15.192:6821/1283571573', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:51:52.396946+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:51:52.453160+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:51:52.387606+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:51:52.360952+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:51:52.468681+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-05-08T08:51:52.401080+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '2.5', 'osds': [3, 2]}, {'pgid': '3.4', 'osds': [1, 5]}, {'pgid': '3.7', 'osds': [4, 3]}, {'pgid': '3.17', 'osds': [1, 0]}, {'pgid': '3.1d', 'osds': [0, 5]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 25, 'length': 8}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 25, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-05-08T08:55:03.769 INFO:tasks.thrashosds.thrasher:Clearing pg_upmap on 3.7 2024-05-08T08:55:03.770 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd rm-pg-upmap 3.7 2024-05-08T08:55:03.770 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:55:03.771+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:55:03.771 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:55:03.771 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:55:03.771 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:55:03.771 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:55:03.772 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.772 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4621.0:9089 3.1f e33/18 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:head v 33'117, mlcod=31'115)", 2024-05-08T08:55:03.772 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:55:03.772 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 151.24433259599999, 2024-05-08T08:55:03.772 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.111244524, 2024-05-08T08:55:03.772 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:03.772 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:03.772 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:55:03.772 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:03.772 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.772 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:03.772 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:55:03.772 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:03.772 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.772 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.773 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:03.773 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:55:03.773 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:03.773 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.773 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.773 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:03.773 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:55:03.773 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5900000000000001e-07 2024-05-08T08:55:03.773 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.773 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.773 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:03.773 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539600+0000", 2024-05-08T08:55:03.773 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.019981364000000001 2024-05-08T08:55:03.773 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.774 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.774 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:03.774 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539624+0000", 2024-05-08T08:55:03.774 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4043e-05 2024-05-08T08:55:03.774 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.774 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.774 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:03.774 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539635+0000", 2024-05-08T08:55:03.774 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1511e-05 2024-05-08T08:55:03.774 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.774 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.774 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:03.774 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539728+0000", 2024-05-08T08:55:03.774 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2498999999999996e-05 2024-05-08T08:55:03.774 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.775 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.775 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:03.775 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539746+0000", 2024-05-08T08:55:03.775 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8289000000000001e-05 2024-05-08T08:55:03.775 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.775 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.775 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:55:03.775 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.630822+0000", 2024-05-08T08:55:03.775 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.091076283999999993 2024-05-08T08:55:03.775 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.775 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.775 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:03.775 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.630863+0000", 2024-05-08T08:55:03.775 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0374999999999999e-05 2024-05-08T08:55:03.776 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:03.776 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:03.776 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:03.776 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.776 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.776 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4195.0:4 3.11 3:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-05-08T08:55:03.776 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:49.764654+0000", 2024-05-08T08:55:03.776 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 133.999297055, 2024-05-08T08:55:03.776 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.30714275499999999, 2024-05-08T08:55:03.776 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:03.776 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:03.776 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-05-08T08:55:03.776 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:03.776 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4195", 2024-05-08T08:55:03.777 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/59179521", 2024-05-08T08:55:03.777 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4 2024-05-08T08:55:03.777 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.777 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:03.777 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.777 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:03.777 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764654+0000", 2024-05-08T08:55:03.777 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:03.777 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.777 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.777 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:03.777 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764654+0000", 2024-05-08T08:55:03.777 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:03.777 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.777 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.778 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:03.778 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764655+0000", 2024-05-08T08:55:03.778 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.452e-06 2024-05-08T08:55:03.778 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.778 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.778 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:03.778 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764661+0000", 2024-05-08T08:55:03.778 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5369999999999997e-06 2024-05-08T08:55:03.778 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.778 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.778 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:03.778 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764671+0000", 2024-05-08T08:55:03.778 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.028e-05 2024-05-08T08:55:03.778 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.779 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.779 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:03.779 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764680+0000", 2024-05-08T08:55:03.779 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0629999999999995e-06 2024-05-08T08:55:03.779 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.779 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.779 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:03.779 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764753+0000", 2024-05-08T08:55:03.779 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2679e-05 2024-05-08T08:55:03.779 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.779 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.779 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-05-08T08:55:03.779 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764765+0000", 2024-05-08T08:55:03.779 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2415000000000001e-05 2024-05-08T08:55:03.779 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.780 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.780 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:03.780 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:50.071692+0000", 2024-05-08T08:55:03.780 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.306927324 2024-05-08T08:55:03.780 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.780 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.780 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:03.780 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:50.071796+0000", 2024-05-08T08:55:03.780 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104005 2024-05-08T08:55:03.780 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:03.780 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:03.780 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:03.780 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.780 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.781 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4416.0:20531 3.16 3:6bb2ed16:::rbd_object_map.1140bc4a55c7:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-05-08T08:55:03.781 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:58.673786+0000", 2024-05-08T08:55:03.781 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.090164650999995, 2024-05-08T08:55:03.781 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.94684119899999997, 2024-05-08T08:55:03.781 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:03.781 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:03.781 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:55:03.781 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:03.781 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4416", 2024-05-08T08:55:03.781 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:55:03.781 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20531 2024-05-08T08:55:03.781 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.781 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:03.781 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.781 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:03.782 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673786+0000", 2024-05-08T08:55:03.782 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:03.782 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.782 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.782 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:03.782 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673786+0000", 2024-05-08T08:55:03.782 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:03.782 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.782 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.782 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:03.782 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673787+0000", 2024-05-08T08:55:03.782 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9800000000000003e-07 2024-05-08T08:55:03.782 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.782 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.782 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:03.783 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673791+0000", 2024-05-08T08:55:03.783 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.104e-06 2024-05-08T08:55:03.783 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.783 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.783 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:03.783 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673801+0000", 2024-05-08T08:55:03.783 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9960000000000005e-06 2024-05-08T08:55:03.783 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.783 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.783 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:03.783 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673806+0000", 2024-05-08T08:55:03.783 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2320000000000001e-06 2024-05-08T08:55:03.783 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.783 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.784 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:03.784 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673860+0000", 2024-05-08T08:55:03.784 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3468999999999999e-05 2024-05-08T08:55:03.784 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.784 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.784 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-05-08T08:55:03.784 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673866+0000", 2024-05-08T08:55:03.784 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3500000000000002e-06 2024-05-08T08:55:03.784 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.784 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.784 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:03.784 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.616700+0000", 2024-05-08T08:55:03.784 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.94283390700000003 2024-05-08T08:55:03.784 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.784 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.785 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:03.785 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.616941+0000", 2024-05-08T08:55:03.785 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00024080400000000001 2024-05-08T08:55:03.785 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.785 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.785 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-05-08T08:55:03.785 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.617072+0000", 2024-05-08T08:55:03.785 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000131688 2024-05-08T08:55:03.785 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.785 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.785 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:55:03.785 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.619571+0000", 2024-05-08T08:55:03.785 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0024990149999999998 2024-05-08T08:55:03.786 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.786 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.786 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:55:03.786 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.620568+0000", 2024-05-08T08:55:03.786 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00099711000000000005 2024-05-08T08:55:03.786 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.786 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.786 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:55:03.786 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.620602+0000", 2024-05-08T08:55:03.786 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.324e-05 2024-05-08T08:55:03.786 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.786 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.786 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:03.786 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.620627+0000", 2024-05-08T08:55:03.786 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5585999999999999e-05 2024-05-08T08:55:03.787 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:03.787 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:03.787 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:03.787 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.787 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.787 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5490 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4227072~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:03.787 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.881809+0000", 2024-05-08T08:55:03.787 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.882142139000003, 2024-05-08T08:55:03.787 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047460447000000003, 2024-05-08T08:55:03.787 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:03.787 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:03.787 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:03.787 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:03.787 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:03.787 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:03.788 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5490 2024-05-08T08:55:03.788 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.788 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:03.788 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.788 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:03.788 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881809+0000", 2024-05-08T08:55:03.788 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:03.788 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.788 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.788 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:03.788 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881809+0000", 2024-05-08T08:55:03.788 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:03.788 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.788 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.789 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:03.789 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881809+0000", 2024-05-08T08:55:03.789 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7899999999999999e-07 2024-05-08T08:55:03.789 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.789 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.789 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:03.789 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881811+0000", 2024-05-08T08:55:03.789 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0339999999999999e-06 2024-05-08T08:55:03.789 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.789 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.789 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:03.789 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881818+0000", 2024-05-08T08:55:03.789 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5760000000000002e-06 2024-05-08T08:55:03.789 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.789 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.790 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:03.790 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881823+0000", 2024-05-08T08:55:03.790 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4160000000000003e-06 2024-05-08T08:55:03.790 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.790 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.790 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:03.790 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929157+0000", 2024-05-08T08:55:03.790 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047334029999999999 2024-05-08T08:55:03.790 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.790 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.790 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:03.790 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929211+0000", 2024-05-08T08:55:03.790 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3560999999999998e-05 2024-05-08T08:55:03.790 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.791 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.791 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:03.791 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929269+0000", 2024-05-08T08:55:03.791 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8350999999999997e-05 2024-05-08T08:55:03.791 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:03.791 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:03.791 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:03.791 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.791 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.791 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5492 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4272128~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:03.791 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.881830+0000", 2024-05-08T08:55:03.791 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.882120845000003, 2024-05-08T08:55:03.791 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047614006, 2024-05-08T08:55:03.791 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:03.791 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:03.792 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:03.792 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:03.792 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:03.792 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:03.792 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5492 2024-05-08T08:55:03.792 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.792 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:03.792 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.792 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:03.792 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881830+0000", 2024-05-08T08:55:03.792 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:03.792 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.792 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.792 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:03.792 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881830+0000", 2024-05-08T08:55:03.792 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:03.793 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.793 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.793 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:03.793 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881830+0000", 2024-05-08T08:55:03.793 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4099999999999999e-07 2024-05-08T08:55:03.793 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.793 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.793 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:03.793 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881832+0000", 2024-05-08T08:55:03.793 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0370000000000001e-06 2024-05-08T08:55:03.793 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.793 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.793 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:03.793 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881839+0000", 2024-05-08T08:55:03.793 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3289999999999999e-06 2024-05-08T08:55:03.793 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.794 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.794 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:03.794 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881844+0000", 2024-05-08T08:55:03.794 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1170000000000003e-06 2024-05-08T08:55:03.794 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.794 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.794 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:03.794 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929296+0000", 2024-05-08T08:55:03.794 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047452418000000003 2024-05-08T08:55:03.794 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.794 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.794 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:03.794 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929360+0000", 2024-05-08T08:55:03.794 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4197999999999998e-05 2024-05-08T08:55:03.795 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.795 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.795 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:03.795 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929444+0000", 2024-05-08T08:55:03.795 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3466000000000004e-05 2024-05-08T08:55:03.795 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:03.795 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:03.795 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:03.795 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.795 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.795 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5494 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4317184~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:03.795 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.881850+0000", 2024-05-08T08:55:03.795 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.882101096, 2024-05-08T08:55:03.795 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047723887, 2024-05-08T08:55:03.795 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:03.795 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:03.795 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:03.796 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:03.796 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:03.796 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:03.796 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5494 2024-05-08T08:55:03.796 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.796 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:03.796 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.796 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:03.796 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881850+0000", 2024-05-08T08:55:03.796 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:03.796 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.796 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.796 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:03.796 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881850+0000", 2024-05-08T08:55:03.796 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:03.796 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.797 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.797 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:03.797 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881850+0000", 2024-05-08T08:55:03.797 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8500000000000002e-07 2024-05-08T08:55:03.797 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.797 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.797 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:03.797 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881856+0000", 2024-05-08T08:55:03.797 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9510000000000003e-06 2024-05-08T08:55:03.797 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.797 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.797 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:03.797 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881862+0000", 2024-05-08T08:55:03.797 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3049999999999998e-06 2024-05-08T08:55:03.797 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.797 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.798 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:03.798 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881868+0000", 2024-05-08T08:55:03.798 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8340000000000004e-06 2024-05-08T08:55:03.798 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.798 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.798 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:03.798 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929475+0000", 2024-05-08T08:55:03.798 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047606796 2024-05-08T08:55:03.798 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.798 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.798 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:03.798 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929518+0000", 2024-05-08T08:55:03.798 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3387000000000003e-05 2024-05-08T08:55:03.798 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.799 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.799 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:03.799 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929574+0000", 2024-05-08T08:55:03.799 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5129e-05 2024-05-08T08:55:03.799 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:03.799 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:03.799 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:03.799 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.799 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.799 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5495 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4362240~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:03.799 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.881875+0000", 2024-05-08T08:55:03.799 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.882076228999999, 2024-05-08T08:55:03.799 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047847967999999998, 2024-05-08T08:55:03.799 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:03.799 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:03.800 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:03.800 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:03.800 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:03.800 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:03.800 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5495 2024-05-08T08:55:03.800 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.800 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:03.800 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.800 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:03.800 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881875+0000", 2024-05-08T08:55:03.800 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:03.800 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.800 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.800 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:03.800 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881875+0000", 2024-05-08T08:55:03.801 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:03.801 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.801 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.801 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:03.801 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881875+0000", 2024-05-08T08:55:03.801 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-07 2024-05-08T08:55:03.801 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.801 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.801 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:03.801 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881877+0000", 2024-05-08T08:55:03.801 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9659999999999999e-06 2024-05-08T08:55:03.801 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.801 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.801 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:03.802 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881883+0000", 2024-05-08T08:55:03.802 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7649999999999999e-06 2024-05-08T08:55:03.802 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.802 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.802 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:03.802 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881887+0000", 2024-05-08T08:55:03.802 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6689999999999997e-06 2024-05-08T08:55:03.802 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.802 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.802 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:03.802 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929600+0000", 2024-05-08T08:55:03.802 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047712746 2024-05-08T08:55:03.802 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.802 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.803 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:03.803 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929659+0000", 2024-05-08T08:55:03.803 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8936e-05 2024-05-08T08:55:03.803 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.803 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.803 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:03.803 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929723+0000", 2024-05-08T08:55:03.803 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3524999999999999e-05 2024-05-08T08:55:03.803 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:03.803 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:03.803 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:03.803 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.803 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.803 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5496 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4407296~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:03.803 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.881894+0000", 2024-05-08T08:55:03.803 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.882056364999997, 2024-05-08T08:55:03.803 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047998015999999998, 2024-05-08T08:55:03.804 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:03.804 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:03.804 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:03.804 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:03.804 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:03.804 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:03.804 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5496 2024-05-08T08:55:03.804 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.804 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:03.804 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.804 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:03.804 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881894+0000", 2024-05-08T08:55:03.804 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:03.804 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.805 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.805 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:03.805 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881894+0000", 2024-05-08T08:55:03.805 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:03.805 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.805 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.805 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:03.805 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881895+0000", 2024-05-08T08:55:03.805 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6699999999999999e-07 2024-05-08T08:55:03.805 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.805 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.805 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:03.805 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881897+0000", 2024-05-08T08:55:03.805 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0229999999999999e-06 2024-05-08T08:55:03.806 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.806 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.806 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:03.806 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881903+0000", 2024-05-08T08:55:03.806 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9850000000000003e-06 2024-05-08T08:55:03.806 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.806 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.806 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:03.806 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881908+0000", 2024-05-08T08:55:03.806 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9019999999999998e-06 2024-05-08T08:55:03.806 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.806 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.806 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:03.806 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929751+0000", 2024-05-08T08:55:03.807 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047843550999999998 2024-05-08T08:55:03.807 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.807 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.807 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:03.807 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929823+0000", 2024-05-08T08:55:03.807 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1172000000000002e-05 2024-05-08T08:55:03.807 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.807 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.807 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:03.807 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929892+0000", 2024-05-08T08:55:03.807 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9915999999999999e-05 2024-05-08T08:55:03.807 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:03.807 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:03.807 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:03.807 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.807 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.808 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5498 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4452352~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:03.808 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.881914+0000", 2024-05-08T08:55:03.808 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.882036607000003, 2024-05-08T08:55:03.808 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048203553000000003, 2024-05-08T08:55:03.808 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:03.808 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:03.808 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:03.808 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:03.808 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:03.808 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:03.808 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5498 2024-05-08T08:55:03.808 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.808 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:03.808 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.809 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:03.809 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881914+0000", 2024-05-08T08:55:03.809 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:03.809 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.809 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.809 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:03.809 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881914+0000", 2024-05-08T08:55:03.809 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:03.809 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.809 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.809 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:03.809 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881915+0000", 2024-05-08T08:55:03.809 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8500000000000002e-07 2024-05-08T08:55:03.809 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.809 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.809 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:03.810 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881917+0000", 2024-05-08T08:55:03.810 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.181e-06 2024-05-08T08:55:03.810 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.810 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.810 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:03.810 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881924+0000", 2024-05-08T08:55:03.810 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9619999999999996e-06 2024-05-08T08:55:03.810 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.810 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.810 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:03.810 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881929+0000", 2024-05-08T08:55:03.810 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1399999999999999e-06 2024-05-08T08:55:03.810 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.810 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.811 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:03.811 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929952+0000", 2024-05-08T08:55:03.811 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048023134000000002 2024-05-08T08:55:03.811 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.811 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.811 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:03.811 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.930029+0000", 2024-05-08T08:55:03.811 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7490000000000005e-05 2024-05-08T08:55:03.811 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.811 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.811 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:03.811 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.930118+0000", 2024-05-08T08:55:03.811 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8261000000000001e-05 2024-05-08T08:55:03.811 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:03.811 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:03.811 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:03.812 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.812 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.812 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5563 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4321280~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:03.812 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.884201+0000", 2024-05-08T08:55:03.812 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.879749357000001, 2024-05-08T08:55:03.812 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047379969000000001, 2024-05-08T08:55:03.812 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:03.812 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:03.812 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:03.812 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:03.812 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:03.812 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:03.812 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5563 2024-05-08T08:55:03.812 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.812 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:03.812 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.812 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:03.813 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884201+0000", 2024-05-08T08:55:03.813 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:03.813 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.813 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.813 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:03.813 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884201+0000", 2024-05-08T08:55:03.813 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:03.813 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.813 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.813 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:03.813 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884202+0000", 2024-05-08T08:55:03.813 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8000000000000002e-07 2024-05-08T08:55:03.813 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.813 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.813 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:03.814 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884208+0000", 2024-05-08T08:55:03.814 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.674e-06 2024-05-08T08:55:03.814 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.814 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.814 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:03.814 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884215+0000", 2024-05-08T08:55:03.814 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0389999999999999e-06 2024-05-08T08:55:03.814 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.814 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.814 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:03.814 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884220+0000", 2024-05-08T08:55:03.814 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4950000000000002e-06 2024-05-08T08:55:03.814 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.814 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.814 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:03.815 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931417+0000", 2024-05-08T08:55:03.815 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047196674000000001 2024-05-08T08:55:03.815 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.815 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.815 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:03.815 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931493+0000", 2024-05-08T08:55:03.815 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5950000000000003e-05 2024-05-08T08:55:03.815 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.815 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.815 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:03.815 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931581+0000", 2024-05-08T08:55:03.815 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8857000000000007e-05 2024-05-08T08:55:03.815 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:03.815 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:03.815 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:03.816 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.816 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.816 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5564 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4366336~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:03.816 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.884225+0000", 2024-05-08T08:55:03.816 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.879725817000001, 2024-05-08T08:55:03.816 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047555989999999999, 2024-05-08T08:55:03.816 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:03.816 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:03.816 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:03.816 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:03.816 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:03.816 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:03.816 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5564 2024-05-08T08:55:03.816 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.816 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:03.816 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.816 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:03.817 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884225+0000", 2024-05-08T08:55:03.817 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:03.817 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.817 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.817 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:03.817 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884225+0000", 2024-05-08T08:55:03.817 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:03.817 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.817 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.817 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:03.817 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884225+0000", 2024-05-08T08:55:03.817 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2499999999999995e-07 2024-05-08T08:55:03.817 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.817 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.817 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:03.817 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884227+0000", 2024-05-08T08:55:03.818 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5829999999999999e-06 2024-05-08T08:55:03.818 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.818 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.818 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:03.818 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884232+0000", 2024-05-08T08:55:03.818 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1159999999999998e-06 2024-05-08T08:55:03.818 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.818 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.818 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:03.818 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884236+0000", 2024-05-08T08:55:03.818 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2490000000000003e-06 2024-05-08T08:55:03.818 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.818 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.818 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:03.818 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931617+0000", 2024-05-08T08:55:03.819 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047380310000000002 2024-05-08T08:55:03.819 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.819 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.819 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:03.819 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931689+0000", 2024-05-08T08:55:03.819 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2358000000000001e-05 2024-05-08T08:55:03.819 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.819 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.819 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:03.819 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931781+0000", 2024-05-08T08:55:03.819 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1848999999999994e-05 2024-05-08T08:55:03.819 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:03.819 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:03.819 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:03.819 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.819 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.820 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5565 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4411392~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:03.820 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.884251+0000", 2024-05-08T08:55:03.820 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.879699664, 2024-05-08T08:55:03.820 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047743671000000001, 2024-05-08T08:55:03.820 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:03.820 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:03.820 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:03.820 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:03.820 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:03.820 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:03.820 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5565 2024-05-08T08:55:03.820 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.820 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:03.820 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.820 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:03.820 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884251+0000", 2024-05-08T08:55:03.820 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:03.821 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.821 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.821 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:03.821 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884251+0000", 2024-05-08T08:55:03.821 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:03.821 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.821 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.821 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:03.821 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884252+0000", 2024-05-08T08:55:03.821 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3200000000000005e-07 2024-05-08T08:55:03.821 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.821 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.821 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:03.821 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884253+0000", 2024-05-08T08:55:03.821 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6980000000000001e-06 2024-05-08T08:55:03.821 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.822 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.822 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:03.822 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884259+0000", 2024-05-08T08:55:03.822 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3759999999999999e-06 2024-05-08T08:55:03.822 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.822 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.822 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:03.822 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884263+0000", 2024-05-08T08:55:03.822 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7280000000000003e-06 2024-05-08T08:55:03.822 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.822 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.822 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:03.822 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931818+0000", 2024-05-08T08:55:03.822 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047554368999999999 2024-05-08T08:55:03.822 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.823 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.823 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:03.823 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931897+0000", 2024-05-08T08:55:03.823 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8821999999999999e-05 2024-05-08T08:55:03.823 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.823 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.823 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:03.823 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931995+0000", 2024-05-08T08:55:03.823 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8146000000000005e-05 2024-05-08T08:55:03.823 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:03.823 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:03.823 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:03.823 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.823 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.823 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5600 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 3948544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:03.824 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.884619+0000", 2024-05-08T08:55:03.824 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.879331409000002, 2024-05-08T08:55:03.824 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047574067999999997, 2024-05-08T08:55:03.824 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:03.824 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:03.824 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:03.824 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:03.824 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:03.824 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:03.824 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5600 2024-05-08T08:55:03.824 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.824 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:03.824 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.824 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:03.824 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884619+0000", 2024-05-08T08:55:03.824 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:03.824 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.825 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.825 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:03.825 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884619+0000", 2024-05-08T08:55:03.825 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:03.825 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.825 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.825 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:03.825 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884620+0000", 2024-05-08T08:55:03.825 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.53e-07 2024-05-08T08:55:03.825 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.825 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.825 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:03.825 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884622+0000", 2024-05-08T08:55:03.825 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.796e-06 2024-05-08T08:55:03.825 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.825 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.826 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:03.826 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884627+0000", 2024-05-08T08:55:03.826 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2850000000000002e-06 2024-05-08T08:55:03.826 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.826 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.826 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:03.826 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884631+0000", 2024-05-08T08:55:03.826 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2899999999999996e-06 2024-05-08T08:55:03.826 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.826 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.826 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:03.826 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932024+0000", 2024-05-08T08:55:03.826 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047392680999999999 2024-05-08T08:55:03.826 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.827 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.827 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:03.827 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932102+0000", 2024-05-08T08:55:03.827 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7806999999999996e-05 2024-05-08T08:55:03.827 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.827 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.827 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:03.827 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932193+0000", 2024-05-08T08:55:03.827 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1855999999999996e-05 2024-05-08T08:55:03.827 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:03.827 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:03.827 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:03.827 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.827 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.827 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5603 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 3993600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:03.827 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.884680+0000", 2024-05-08T08:55:03.828 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.8792708, 2024-05-08T08:55:03.828 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047745034999999998, 2024-05-08T08:55:03.828 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:03.828 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:03.828 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:03.828 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:03.828 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:03.828 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:03.828 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5603 2024-05-08T08:55:03.828 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.828 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:03.828 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.828 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:03.828 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884680+0000", 2024-05-08T08:55:03.828 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:03.829 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.829 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.829 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:03.829 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884680+0000", 2024-05-08T08:55:03.829 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:03.829 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.829 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.829 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:03.829 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884680+0000", 2024-05-08T08:55:03.829 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4400000000000001e-07 2024-05-08T08:55:03.829 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.829 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.829 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:03.829 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884682+0000", 2024-05-08T08:55:03.829 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.514e-06 2024-05-08T08:55:03.830 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.830 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.830 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:03.830 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884687+0000", 2024-05-08T08:55:03.830 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0359999999999997e-06 2024-05-08T08:55:03.830 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.830 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.830 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:03.830 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884692+0000", 2024-05-08T08:55:03.830 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5990000000000004e-06 2024-05-08T08:55:03.830 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.830 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.830 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:03.830 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932235+0000", 2024-05-08T08:55:03.830 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047543635000000001 2024-05-08T08:55:03.831 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.831 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.831 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:03.831 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932323+0000", 2024-05-08T08:55:03.831 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7311000000000005e-05 2024-05-08T08:55:03.831 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.831 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.831 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:03.831 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932425+0000", 2024-05-08T08:55:03.831 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102496 2024-05-08T08:55:03.831 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:03.831 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:03.831 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:03.831 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.831 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.831 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5606 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4038656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:03.832 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.884729+0000", 2024-05-08T08:55:03.832 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.879221512000001, 2024-05-08T08:55:03.832 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047916488, 2024-05-08T08:55:03.832 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:03.832 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:03.832 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:03.832 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:03.832 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:03.832 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:03.832 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5606 2024-05-08T08:55:03.832 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.832 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:03.832 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.832 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:03.832 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884729+0000", 2024-05-08T08:55:03.832 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:03.833 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.833 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.833 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:03.833 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884729+0000", 2024-05-08T08:55:03.833 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:03.833 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.833 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.833 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:03.833 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.885940+0000", 2024-05-08T08:55:03.833 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001210509 2024-05-08T08:55:03.833 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.833 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.833 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:03.833 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.885945+0000", 2024-05-08T08:55:03.833 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7389999999999999e-06 2024-05-08T08:55:03.833 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.833 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.834 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:03.834 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.885955+0000", 2024-05-08T08:55:03.834 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0813e-05 2024-05-08T08:55:03.834 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.834 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.834 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:03.834 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.885962+0000", 2024-05-08T08:55:03.834 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1160000000000001e-06 2024-05-08T08:55:03.834 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.834 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.834 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:03.834 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932465+0000", 2024-05-08T08:55:03.834 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046502612999999998 2024-05-08T08:55:03.834 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.834 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.835 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:03.835 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932547+0000", 2024-05-08T08:55:03.835 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2025000000000002e-05 2024-05-08T08:55:03.835 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.835 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.835 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:03.835 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932646+0000", 2024-05-08T08:55:03.835 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8672999999999995e-05 2024-05-08T08:55:03.835 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:03.835 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:03.835 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:03.835 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.835 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.835 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5635 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4235264~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:03.835 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.886082+0000", 2024-05-08T08:55:03.835 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.877868393, 2024-05-08T08:55:03.836 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047438284999999997, 2024-05-08T08:55:03.836 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:03.836 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:03.836 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:03.836 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:03.836 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:03.836 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:03.836 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5635 2024-05-08T08:55:03.836 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.836 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:03.836 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.836 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:03.836 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886082+0000", 2024-05-08T08:55:03.836 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:03.836 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.836 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.836 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:03.837 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886082+0000", 2024-05-08T08:55:03.837 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:03.837 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.837 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.837 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:03.837 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886083+0000", 2024-05-08T08:55:03.837 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8999999999999996e-07 2024-05-08T08:55:03.837 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.837 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.837 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:03.837 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886086+0000", 2024-05-08T08:55:03.837 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7e-06 2024-05-08T08:55:03.837 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.837 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.837 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:03.838 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886093+0000", 2024-05-08T08:55:03.838 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5320000000000001e-06 2024-05-08T08:55:03.838 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.838 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.838 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:03.838 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886102+0000", 2024-05-08T08:55:03.838 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8070000000000006e-06 2024-05-08T08:55:03.838 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.838 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.838 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:03.838 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933357+0000", 2024-05-08T08:55:03.838 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047255093999999997 2024-05-08T08:55:03.838 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.838 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.838 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:03.839 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933433+0000", 2024-05-08T08:55:03.839 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5796000000000004e-05 2024-05-08T08:55:03.839 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.839 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.839 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:03.839 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933521+0000", 2024-05-08T08:55:03.839 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7766e-05 2024-05-08T08:55:03.839 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:03.839 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:03.839 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:03.839 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.839 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.839 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5636 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4280320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:03.839 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.886111+0000", 2024-05-08T08:55:03.840 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.877839774999998, 2024-05-08T08:55:03.840 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047663644999999998, 2024-05-08T08:55:03.840 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:03.840 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:03.840 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:03.840 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:03.840 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:03.840 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:03.840 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5636 2024-05-08T08:55:03.840 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.840 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:03.840 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.840 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:03.840 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886111+0000", 2024-05-08T08:55:03.840 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:03.841 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.841 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.841 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:03.841 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886111+0000", 2024-05-08T08:55:03.841 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:03.841 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.841 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.841 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:03.841 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886111+0000", 2024-05-08T08:55:03.841 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5200000000000002e-07 2024-05-08T08:55:03.841 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.841 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.841 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:03.841 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886114+0000", 2024-05-08T08:55:03.841 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1710000000000001e-06 2024-05-08T08:55:03.841 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.841 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.842 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:03.842 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886120+0000", 2024-05-08T08:55:03.842 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.404e-06 2024-05-08T08:55:03.842 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.842 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.842 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:03.842 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886125+0000", 2024-05-08T08:55:03.842 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2730000000000002e-06 2024-05-08T08:55:03.842 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.842 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.842 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:03.842 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933570+0000", 2024-05-08T08:55:03.842 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047444989999999999 2024-05-08T08:55:03.842 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.842 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.843 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:03.843 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933659+0000", 2024-05-08T08:55:03.843 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8912999999999994e-05 2024-05-08T08:55:03.843 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.843 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.843 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:03.843 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933775+0000", 2024-05-08T08:55:03.843 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000115442 2024-05-08T08:55:03.843 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:03.843 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:03.843 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:03.843 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.843 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.843 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5637 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4325376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:03.843 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.886132+0000", 2024-05-08T08:55:03.843 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.877818736999998, 2024-05-08T08:55:03.844 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047852219000000001, 2024-05-08T08:55:03.844 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:03.844 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:03.844 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:03.844 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:03.844 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:03.844 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:03.844 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5637 2024-05-08T08:55:03.844 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.844 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:03.844 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.844 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:03.844 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886132+0000", 2024-05-08T08:55:03.844 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:03.844 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.844 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.845 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:03.845 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886132+0000", 2024-05-08T08:55:03.845 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:03.845 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.845 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.845 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:03.845 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886132+0000", 2024-05-08T08:55:03.845 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7300000000000002e-07 2024-05-08T08:55:03.845 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.845 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.845 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:03.845 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886139+0000", 2024-05-08T08:55:03.845 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6109999999999999e-06 2024-05-08T08:55:03.845 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.845 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.845 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:03.846 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886146+0000", 2024-05-08T08:55:03.846 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6279999999999999e-06 2024-05-08T08:55:03.846 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.846 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.846 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:03.846 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886151+0000", 2024-05-08T08:55:03.846 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1769999999999997e-06 2024-05-08T08:55:03.846 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.846 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.846 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:03.846 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933812+0000", 2024-05-08T08:55:03.846 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047660912999999999 2024-05-08T08:55:03.846 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.846 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.846 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:03.847 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933888+0000", 2024-05-08T08:55:03.847 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5865000000000006e-05 2024-05-08T08:55:03.847 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.847 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.847 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:03.847 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933984+0000", 2024-05-08T08:55:03.847 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6651999999999996e-05 2024-05-08T08:55:03.847 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:03.847 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:03.847 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:03.847 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.847 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.847 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5638 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4370432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:03.847 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.886157+0000", 2024-05-08T08:55:03.847 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.877793472999997, 2024-05-08T08:55:03.847 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048077597, 2024-05-08T08:55:03.848 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:03.848 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:03.848 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:03.848 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:03.848 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:03.848 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:03.848 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5638 2024-05-08T08:55:03.848 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.848 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:03.848 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.848 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:03.848 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886157+0000", 2024-05-08T08:55:03.848 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:03.848 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.848 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.848 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:03.849 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886157+0000", 2024-05-08T08:55:03.849 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:03.849 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.849 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.849 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:03.849 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886158+0000", 2024-05-08T08:55:03.849 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1100000000000001e-07 2024-05-08T08:55:03.849 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.849 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.849 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:03.849 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886160+0000", 2024-05-08T08:55:03.849 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9800000000000001e-06 2024-05-08T08:55:03.849 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.849 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.849 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:03.850 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886166+0000", 2024-05-08T08:55:03.850 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.314e-06 2024-05-08T08:55:03.850 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.850 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.850 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:03.850 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886171+0000", 2024-05-08T08:55:03.850 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7700000000000001e-06 2024-05-08T08:55:03.850 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.850 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.850 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:03.850 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.934025+0000", 2024-05-08T08:55:03.850 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047854509000000003 2024-05-08T08:55:03.850 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.851 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.851 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:03.851 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.934121+0000", 2024-05-08T08:55:03.851 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5688999999999998e-05 2024-05-08T08:55:03.851 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.851 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.851 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:03.851 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.934235+0000", 2024-05-08T08:55:03.851 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000113924 2024-05-08T08:55:03.851 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:03.851 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:03.851 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:03.851 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.851 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.851 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5639 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4415488~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:03.851 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.886177+0000", 2024-05-08T08:55:03.852 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.877773722999997, 2024-05-08T08:55:03.852 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048249541, 2024-05-08T08:55:03.852 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:03.852 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:03.852 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:03.852 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:03.852 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:03.852 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:03.852 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5639 2024-05-08T08:55:03.852 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.852 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:03.852 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.852 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:03.852 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886177+0000", 2024-05-08T08:55:03.852 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:03.852 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.852 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.853 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:03.853 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886177+0000", 2024-05-08T08:55:03.853 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:03.853 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.853 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.853 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:03.853 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886177+0000", 2024-05-08T08:55:03.853 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.65e-07 2024-05-08T08:55:03.853 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.853 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.853 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:03.853 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886179+0000", 2024-05-08T08:55:03.853 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9530000000000002e-06 2024-05-08T08:55:03.853 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.853 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.853 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:03.854 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886186+0000", 2024-05-08T08:55:03.854 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3419999999999996e-06 2024-05-08T08:55:03.854 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.854 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.854 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:03.854 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886192+0000", 2024-05-08T08:55:03.854 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0560000000000001e-06 2024-05-08T08:55:03.854 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.854 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.854 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:03.854 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.934258+0000", 2024-05-08T08:55:03.854 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048066367999999998 2024-05-08T08:55:03.854 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.854 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.854 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:03.854 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.934338+0000", 2024-05-08T08:55:03.855 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9616000000000005e-05 2024-05-08T08:55:03.855 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:03.855 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:03.855 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:03.855 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.934427+0000", 2024-05-08T08:55:03.855 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8840999999999999e-05 2024-05-08T08:55:03.855 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:03.855 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:03.855 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:03.855 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:03.855 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:03.855 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:55:03.855 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:55:03.856 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T08:55:03.900 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:55:03.903+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:55:03.961 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:55:03.961 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:55:03.962 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:55:03.962 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:55:03.971 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:55:03.971 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:55:03.973+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:55:03.984 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:55:03.984 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:55:04.003 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:55:04.003 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:55:04.020 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:55:04.021 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:55:04.038 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:55:04.038 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:55:04.055 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:55:04.056 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:55:04.071 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:55:04.074+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:55:04.073 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:55:04.074 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:55:04.115 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:04.115 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:04.124 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:55:04.172 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:55:04.174+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:55:04.229 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:04.230 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:04.239 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:55:04.280 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:55:04.283+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:55:04.373 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:55:04.376+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:55:04.385 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:04.385 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:04.394 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:55:04.422 INFO:teuthology.orchestra.run.smithi059.stderr:clear 3.7 pg_upmap mapping 2024-05-08T08:55:04.474 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:55:04.476+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:55:04.513 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:04.514 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:04.526 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:55:04.610 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:55:04.613+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:55:04.673 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:04.673 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:04.676 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:55:04.679+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:55:04.681 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:55:04.777 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:55:04.780+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:55:04.792 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:04.792 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:04.800 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:55:04.878 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:55:04.881+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:55:04.910 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:04.910 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:04.919 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:55:04.979 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:55:04.982+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:55:05.028 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:05.028 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:05.037 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:55:05.080 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:55:05.083+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:55:05.180 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:55:05.183+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:55:05.186 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:05.186 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:05.195 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:55:05.281 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:55:05.284+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:55:05.345 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:05.346 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:05.354 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:55:05.396 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:55:05.398+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:55:05.430 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-05-08T08:55:05.483 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:55:05.486+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:55:05.506 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:05.506 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:05.516 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:55:05.599 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:55:05.602+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:55:05.667 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:05.667 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:05.676 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:55:05.718 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:55:05.720+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:55:05.787 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:55:05.789+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:55:05.826 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:05.826 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:05.835 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:55:05.888 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:55:05.890+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:55:05.985 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:05.986 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:05.989 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:55:05.992+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:55:05.994 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:55:06.086 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T08:55:06.089 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:55:06.092+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:55:06.111 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:06.111 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:06.120 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:55:06.190 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:55:06.193+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:55:06.211 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:55:06.211 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:55:06.225 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:06.225 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:06.229 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:55:06.229 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:55:06.233 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:55:06.246 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:55:06.246 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:55:06.264 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:55:06.264 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:55:06.282 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:55:06.283 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:55:06.291 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:55:06.293+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:55:06.300 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:55:06.301 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:55:06.343 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:55:06.343 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:55:06.343 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:55:06.343 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:55:06.343 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.343 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4621.0:9089 3.1f e33/18 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:head v 33'117, mlcod=31'115)", 2024-05-08T08:55:06.344 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:55:06.344 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 153.82547894300001, 2024-05-08T08:55:06.344 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.111244524, 2024-05-08T08:55:06.344 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:06.344 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:06.344 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:55:06.344 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:06.344 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.344 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:06.344 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:55:06.344 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:06.344 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.344 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.344 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:06.344 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:55:06.344 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:06.344 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.345 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:06.345 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:55:06.345 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5900000000000001e-07 2024-05-08T08:55:06.345 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.345 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:06.345 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539600+0000", 2024-05-08T08:55:06.345 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.019981364000000001 2024-05-08T08:55:06.345 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.345 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:06.345 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539624+0000", 2024-05-08T08:55:06.345 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4043e-05 2024-05-08T08:55:06.345 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.346 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:06.346 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539635+0000", 2024-05-08T08:55:06.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1511e-05 2024-05-08T08:55:06.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.346 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:06.346 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539728+0000", 2024-05-08T08:55:06.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2498999999999996e-05 2024-05-08T08:55:06.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.346 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:06.346 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539746+0000", 2024-05-08T08:55:06.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8289000000000001e-05 2024-05-08T08:55:06.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.347 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:55:06.347 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.630822+0000", 2024-05-08T08:55:06.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.091076283999999993 2024-05-08T08:55:06.347 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.347 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.347 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:06.347 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.630863+0000", 2024-05-08T08:55:06.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0374999999999999e-05 2024-05-08T08:55:06.347 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.347 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:06.347 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.347 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.347 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.347 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4195.0:4 3.11 3:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-05-08T08:55:06.347 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:49.764654+0000", 2024-05-08T08:55:06.348 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 136.58044340199999, 2024-05-08T08:55:06.348 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.30714275499999999, 2024-05-08T08:55:06.348 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:06.348 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:06.348 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-05-08T08:55:06.348 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:06.348 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4195", 2024-05-08T08:55:06.348 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/59179521", 2024-05-08T08:55:06.348 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4 2024-05-08T08:55:06.348 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.348 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:06.348 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.348 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:06.348 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764654+0000", 2024-05-08T08:55:06.348 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:06.348 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.348 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.349 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:06.349 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764654+0000", 2024-05-08T08:55:06.349 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:06.349 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.349 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.349 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:06.349 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764655+0000", 2024-05-08T08:55:06.349 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.452e-06 2024-05-08T08:55:06.349 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.349 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.349 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:06.349 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764661+0000", 2024-05-08T08:55:06.349 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5369999999999997e-06 2024-05-08T08:55:06.349 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.349 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.349 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:06.349 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764671+0000", 2024-05-08T08:55:06.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.028e-05 2024-05-08T08:55:06.350 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.350 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.350 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:06.350 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764680+0000", 2024-05-08T08:55:06.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0629999999999995e-06 2024-05-08T08:55:06.350 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.350 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.350 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:06.350 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764753+0000", 2024-05-08T08:55:06.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2679e-05 2024-05-08T08:55:06.350 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.350 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.350 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-05-08T08:55:06.350 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764765+0000", 2024-05-08T08:55:06.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2415000000000001e-05 2024-05-08T08:55:06.351 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.351 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.351 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:06.351 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:50.071692+0000", 2024-05-08T08:55:06.351 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.306927324 2024-05-08T08:55:06.351 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.351 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.351 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:06.351 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:50.071796+0000", 2024-05-08T08:55:06.351 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104005 2024-05-08T08:55:06.351 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.351 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:06.351 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.351 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.352 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.352 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4416.0:20531 3.16 3:6bb2ed16:::rbd_object_map.1140bc4a55c7:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-05-08T08:55:06.352 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:58.673786+0000", 2024-05-08T08:55:06.352 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 67.671310997999996, 2024-05-08T08:55:06.352 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.94684119899999997, 2024-05-08T08:55:06.352 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:06.352 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:06.352 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:55:06.352 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:06.352 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4416", 2024-05-08T08:55:06.352 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:55:06.352 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20531 2024-05-08T08:55:06.352 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.352 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:06.352 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.352 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:06.353 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673786+0000", 2024-05-08T08:55:06.353 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:06.353 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.353 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.353 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:06.353 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673786+0000", 2024-05-08T08:55:06.353 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:06.353 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.353 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.353 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:06.353 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673787+0000", 2024-05-08T08:55:06.353 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9800000000000003e-07 2024-05-08T08:55:06.353 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.353 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.353 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:06.354 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673791+0000", 2024-05-08T08:55:06.354 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.104e-06 2024-05-08T08:55:06.354 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.354 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.354 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:06.354 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673801+0000", 2024-05-08T08:55:06.354 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9960000000000005e-06 2024-05-08T08:55:06.354 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.354 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.354 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:06.354 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673806+0000", 2024-05-08T08:55:06.354 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2320000000000001e-06 2024-05-08T08:55:06.354 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.354 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.354 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:06.354 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673860+0000", 2024-05-08T08:55:06.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3468999999999999e-05 2024-05-08T08:55:06.355 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.355 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.355 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-05-08T08:55:06.355 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673866+0000", 2024-05-08T08:55:06.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3500000000000002e-06 2024-05-08T08:55:06.355 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.355 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.355 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:06.355 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.616700+0000", 2024-05-08T08:55:06.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.94283390700000003 2024-05-08T08:55:06.355 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.355 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.355 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:06.355 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.616941+0000", 2024-05-08T08:55:06.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00024080400000000001 2024-05-08T08:55:06.356 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.356 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.356 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-05-08T08:55:06.356 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.617072+0000", 2024-05-08T08:55:06.356 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000131688 2024-05-08T08:55:06.356 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.356 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.356 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:55:06.356 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.619571+0000", 2024-05-08T08:55:06.356 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0024990149999999998 2024-05-08T08:55:06.356 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.356 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.356 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:55:06.356 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.620568+0000", 2024-05-08T08:55:06.356 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00099711000000000005 2024-05-08T08:55:06.356 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.357 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.357 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:55:06.357 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.620602+0000", 2024-05-08T08:55:06.357 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.324e-05 2024-05-08T08:55:06.357 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.357 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.357 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:06.357 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.620627+0000", 2024-05-08T08:55:06.357 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5585999999999999e-05 2024-05-08T08:55:06.357 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.357 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:06.357 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.357 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.357 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.357 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5490 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4227072~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:06.358 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.881809+0000", 2024-05-08T08:55:06.358 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 40.463288486000003, 2024-05-08T08:55:06.358 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047460447000000003, 2024-05-08T08:55:06.358 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:06.358 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:06.358 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:06.358 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:06.358 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:06.358 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:06.358 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5490 2024-05-08T08:55:06.358 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.358 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:06.358 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.358 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:06.358 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881809+0000", 2024-05-08T08:55:06.358 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:06.359 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.359 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.359 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:06.359 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881809+0000", 2024-05-08T08:55:06.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:06.359 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.359 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.359 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:06.359 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881809+0000", 2024-05-08T08:55:06.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7899999999999999e-07 2024-05-08T08:55:06.359 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.359 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.359 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:06.359 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881811+0000", 2024-05-08T08:55:06.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0339999999999999e-06 2024-05-08T08:55:06.359 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.359 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.360 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:06.360 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881818+0000", 2024-05-08T08:55:06.360 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5760000000000002e-06 2024-05-08T08:55:06.360 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.360 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.360 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:06.360 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881823+0000", 2024-05-08T08:55:06.360 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4160000000000003e-06 2024-05-08T08:55:06.360 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.360 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.360 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:06.360 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929157+0000", 2024-05-08T08:55:06.360 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047334029999999999 2024-05-08T08:55:06.360 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.360 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.360 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:06.361 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929211+0000", 2024-05-08T08:55:06.361 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3560999999999998e-05 2024-05-08T08:55:06.361 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.361 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.361 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:06.361 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929269+0000", 2024-05-08T08:55:06.361 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8350999999999997e-05 2024-05-08T08:55:06.361 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.361 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:06.361 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.361 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.361 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.361 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5492 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4272128~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:06.361 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.881830+0000", 2024-05-08T08:55:06.361 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 40.463267192000004, 2024-05-08T08:55:06.362 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047614006, 2024-05-08T08:55:06.362 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:06.362 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:06.362 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:06.362 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:06.362 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:06.362 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:06.362 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5492 2024-05-08T08:55:06.362 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.362 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:06.362 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.362 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:06.362 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881830+0000", 2024-05-08T08:55:06.362 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:06.363 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.363 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.363 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:06.363 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881830+0000", 2024-05-08T08:55:06.363 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:06.363 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.363 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.363 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:06.363 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881830+0000", 2024-05-08T08:55:06.363 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4099999999999999e-07 2024-05-08T08:55:06.363 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.363 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.363 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:06.363 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881832+0000", 2024-05-08T08:55:06.363 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0370000000000001e-06 2024-05-08T08:55:06.364 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.364 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.364 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:06.364 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881839+0000", 2024-05-08T08:55:06.364 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3289999999999999e-06 2024-05-08T08:55:06.364 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.364 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.364 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:06.364 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881844+0000", 2024-05-08T08:55:06.364 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1170000000000003e-06 2024-05-08T08:55:06.364 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.364 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.364 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:06.364 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929296+0000", 2024-05-08T08:55:06.364 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047452418000000003 2024-05-08T08:55:06.365 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.365 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.365 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:06.365 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929360+0000", 2024-05-08T08:55:06.365 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4197999999999998e-05 2024-05-08T08:55:06.365 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.365 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.365 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:06.365 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929444+0000", 2024-05-08T08:55:06.365 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3466000000000004e-05 2024-05-08T08:55:06.365 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.365 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:06.365 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.365 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.365 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.365 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5494 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4317184~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:06.366 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.881850+0000", 2024-05-08T08:55:06.366 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 40.463247443, 2024-05-08T08:55:06.366 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047723887, 2024-05-08T08:55:06.366 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:06.366 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:06.366 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:06.366 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:06.366 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:06.366 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:06.366 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5494 2024-05-08T08:55:06.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.366 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:06.366 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.366 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:06.366 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881850+0000", 2024-05-08T08:55:06.366 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:06.367 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.367 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.367 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:06.367 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881850+0000", 2024-05-08T08:55:06.367 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:06.367 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.367 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.367 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:06.367 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881850+0000", 2024-05-08T08:55:06.367 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8500000000000002e-07 2024-05-08T08:55:06.367 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.367 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.367 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:06.367 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881856+0000", 2024-05-08T08:55:06.367 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9510000000000003e-06 2024-05-08T08:55:06.368 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.368 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.368 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:06.368 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881862+0000", 2024-05-08T08:55:06.368 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3049999999999998e-06 2024-05-08T08:55:06.368 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.368 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.368 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:06.368 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881868+0000", 2024-05-08T08:55:06.368 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8340000000000004e-06 2024-05-08T08:55:06.368 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.368 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.368 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:06.368 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929475+0000", 2024-05-08T08:55:06.369 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047606796 2024-05-08T08:55:06.369 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.369 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.369 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:06.369 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929518+0000", 2024-05-08T08:55:06.369 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3387000000000003e-05 2024-05-08T08:55:06.369 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.369 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.369 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:06.369 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929574+0000", 2024-05-08T08:55:06.369 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5129e-05 2024-05-08T08:55:06.369 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.369 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:06.369 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.369 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.370 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.370 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5495 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4362240~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:06.370 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.881875+0000", 2024-05-08T08:55:06.370 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 40.463222576, 2024-05-08T08:55:06.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047847967999999998, 2024-05-08T08:55:06.370 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:06.370 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:06.370 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:06.370 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:06.370 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:06.370 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:06.370 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5495 2024-05-08T08:55:06.370 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.370 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:06.371 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.371 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:06.371 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881875+0000", 2024-05-08T08:55:06.371 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:06.371 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.371 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.371 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:06.371 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881875+0000", 2024-05-08T08:55:06.371 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:06.371 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.371 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.371 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:06.371 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881875+0000", 2024-05-08T08:55:06.371 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-07 2024-05-08T08:55:06.371 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.372 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.372 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:06.372 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881877+0000", 2024-05-08T08:55:06.372 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9659999999999999e-06 2024-05-08T08:55:06.372 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.372 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.372 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:06.372 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881883+0000", 2024-05-08T08:55:06.372 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7649999999999999e-06 2024-05-08T08:55:06.372 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.372 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.372 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:06.372 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881887+0000", 2024-05-08T08:55:06.372 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6689999999999997e-06 2024-05-08T08:55:06.372 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.373 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.373 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:06.373 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929600+0000", 2024-05-08T08:55:06.373 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047712746 2024-05-08T08:55:06.373 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.373 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.373 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:06.373 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929659+0000", 2024-05-08T08:55:06.373 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8936e-05 2024-05-08T08:55:06.373 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.373 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.373 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:06.373 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929723+0000", 2024-05-08T08:55:06.373 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3524999999999999e-05 2024-05-08T08:55:06.373 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.374 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:06.374 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.374 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.374 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.374 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5496 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4407296~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:06.374 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.881894+0000", 2024-05-08T08:55:06.374 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 40.463202711999998, 2024-05-08T08:55:06.374 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047998015999999998, 2024-05-08T08:55:06.374 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:06.374 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:06.374 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:06.374 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:06.374 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:06.374 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:06.374 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5496 2024-05-08T08:55:06.375 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.375 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:06.375 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.375 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:06.375 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881894+0000", 2024-05-08T08:55:06.375 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:06.375 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.375 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.375 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:06.375 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881894+0000", 2024-05-08T08:55:06.375 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:06.375 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.375 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.375 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:06.375 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881895+0000", 2024-05-08T08:55:06.375 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6699999999999999e-07 2024-05-08T08:55:06.376 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.376 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.376 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:06.376 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881897+0000", 2024-05-08T08:55:06.376 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0229999999999999e-06 2024-05-08T08:55:06.376 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.376 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.376 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:06.376 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881903+0000", 2024-05-08T08:55:06.376 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9850000000000003e-06 2024-05-08T08:55:06.376 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.376 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.376 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:06.376 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881908+0000", 2024-05-08T08:55:06.376 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9019999999999998e-06 2024-05-08T08:55:06.376 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.377 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.377 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:06.377 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929751+0000", 2024-05-08T08:55:06.377 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047843550999999998 2024-05-08T08:55:06.377 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.377 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.377 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:06.377 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929823+0000", 2024-05-08T08:55:06.377 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1172000000000002e-05 2024-05-08T08:55:06.377 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.377 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.377 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:06.377 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929892+0000", 2024-05-08T08:55:06.378 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9915999999999999e-05 2024-05-08T08:55:06.378 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.378 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:06.378 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.378 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.378 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.378 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5498 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4452352~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:06.378 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.881914+0000", 2024-05-08T08:55:06.378 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 40.463182953999997, 2024-05-08T08:55:06.378 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048203553000000003, 2024-05-08T08:55:06.378 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:06.378 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:06.378 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:06.378 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:06.378 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:06.379 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:06.379 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5498 2024-05-08T08:55:06.379 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.379 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:06.379 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.379 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:06.379 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881914+0000", 2024-05-08T08:55:06.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:06.379 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.379 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.379 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:06.379 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881914+0000", 2024-05-08T08:55:06.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:06.379 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.379 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.379 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:06.380 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881915+0000", 2024-05-08T08:55:06.380 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8500000000000002e-07 2024-05-08T08:55:06.380 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.380 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.380 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:06.380 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881917+0000", 2024-05-08T08:55:06.380 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.181e-06 2024-05-08T08:55:06.380 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.380 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.380 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:06.380 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881924+0000", 2024-05-08T08:55:06.380 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9619999999999996e-06 2024-05-08T08:55:06.380 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.381 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.381 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:06.381 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881929+0000", 2024-05-08T08:55:06.381 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1399999999999999e-06 2024-05-08T08:55:06.381 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.381 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.381 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:06.381 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929952+0000", 2024-05-08T08:55:06.381 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048023134000000002 2024-05-08T08:55:06.381 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.381 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.381 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:06.381 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.930029+0000", 2024-05-08T08:55:06.381 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7490000000000005e-05 2024-05-08T08:55:06.381 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.382 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.382 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:06.382 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.930118+0000", 2024-05-08T08:55:06.382 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8261000000000001e-05 2024-05-08T08:55:06.382 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.382 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:06.382 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.382 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.382 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.382 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5563 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4321280~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:06.382 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.884201+0000", 2024-05-08T08:55:06.382 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 40.460895704000002, 2024-05-08T08:55:06.382 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047379969000000001, 2024-05-08T08:55:06.383 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:06.383 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:06.383 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:06.383 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:06.383 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:06.383 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:06.383 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5563 2024-05-08T08:55:06.383 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.383 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:06.383 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.383 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:06.383 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884201+0000", 2024-05-08T08:55:06.383 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:06.383 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.383 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.384 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:06.384 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884201+0000", 2024-05-08T08:55:06.384 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:06.384 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.384 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:06.384 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884202+0000", 2024-05-08T08:55:06.384 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8000000000000002e-07 2024-05-08T08:55:06.384 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.384 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:06.384 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884208+0000", 2024-05-08T08:55:06.384 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.674e-06 2024-05-08T08:55:06.384 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.385 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.385 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:06.385 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884215+0000", 2024-05-08T08:55:06.385 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0389999999999999e-06 2024-05-08T08:55:06.385 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.385 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.385 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:06.385 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884220+0000", 2024-05-08T08:55:06.385 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4950000000000002e-06 2024-05-08T08:55:06.385 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.385 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.385 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:06.385 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931417+0000", 2024-05-08T08:55:06.385 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047196674000000001 2024-05-08T08:55:06.386 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.386 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.386 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:06.386 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931493+0000", 2024-05-08T08:55:06.386 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5950000000000003e-05 2024-05-08T08:55:06.386 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.386 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.386 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:06.386 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931581+0000", 2024-05-08T08:55:06.386 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8857000000000007e-05 2024-05-08T08:55:06.386 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.386 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:06.386 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.386 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.386 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.387 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5564 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4366336~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:06.387 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.884225+0000", 2024-05-08T08:55:06.387 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 40.460872164000001, 2024-05-08T08:55:06.387 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047555989999999999, 2024-05-08T08:55:06.387 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:06.387 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:06.387 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:06.387 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:06.387 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:06.387 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:06.387 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5564 2024-05-08T08:55:06.387 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.387 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:06.387 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.388 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:06.388 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884225+0000", 2024-05-08T08:55:06.388 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:06.388 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.388 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.388 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:06.388 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884225+0000", 2024-05-08T08:55:06.388 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:06.388 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.388 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.388 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:06.388 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884225+0000", 2024-05-08T08:55:06.388 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2499999999999995e-07 2024-05-08T08:55:06.388 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.388 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.389 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:06.389 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884227+0000", 2024-05-08T08:55:06.389 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5829999999999999e-06 2024-05-08T08:55:06.389 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.389 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.389 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:06.389 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884232+0000", 2024-05-08T08:55:06.389 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1159999999999998e-06 2024-05-08T08:55:06.389 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.389 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.389 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:06.389 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884236+0000", 2024-05-08T08:55:06.389 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2490000000000003e-06 2024-05-08T08:55:06.389 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.389 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.389 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:06.390 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931617+0000", 2024-05-08T08:55:06.390 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047380310000000002 2024-05-08T08:55:06.390 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.390 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.390 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:06.390 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931689+0000", 2024-05-08T08:55:06.390 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2358000000000001e-05 2024-05-08T08:55:06.390 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.390 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.390 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:06.390 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931781+0000", 2024-05-08T08:55:06.390 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1848999999999994e-05 2024-05-08T08:55:06.390 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.390 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:06.390 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.391 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.391 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.391 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5565 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4411392~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:06.391 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.884251+0000", 2024-05-08T08:55:06.391 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 40.460846011000001, 2024-05-08T08:55:06.391 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047743671000000001, 2024-05-08T08:55:06.391 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:06.391 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:06.391 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:06.391 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:06.391 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:06.391 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:06.391 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5565 2024-05-08T08:55:06.391 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.392 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:06.392 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.392 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:06.392 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884251+0000", 2024-05-08T08:55:06.392 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:06.392 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.392 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.392 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:06.392 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884251+0000", 2024-05-08T08:55:06.392 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:06.392 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.392 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.392 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:06.392 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884252+0000", 2024-05-08T08:55:06.392 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3200000000000005e-07 2024-05-08T08:55:06.392 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.393 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.393 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:06.393 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884253+0000", 2024-05-08T08:55:06.393 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6980000000000001e-06 2024-05-08T08:55:06.393 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.393 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.393 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:06.393 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884259+0000", 2024-05-08T08:55:06.393 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3759999999999999e-06 2024-05-08T08:55:06.393 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.393 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.393 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:06.393 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884263+0000", 2024-05-08T08:55:06.393 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7280000000000003e-06 2024-05-08T08:55:06.394 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.394 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.394 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:06.394 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931818+0000", 2024-05-08T08:55:06.394 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047554368999999999 2024-05-08T08:55:06.394 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.394 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.394 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:06.394 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931897+0000", 2024-05-08T08:55:06.394 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8821999999999999e-05 2024-05-08T08:55:06.394 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.394 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.394 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:06.394 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931995+0000", 2024-05-08T08:55:06.395 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8146000000000005e-05 2024-05-08T08:55:06.395 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.395 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:06.395 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.395 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.395 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.395 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5600 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 3948544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:06.395 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.884619+0000", 2024-05-08T08:55:06.395 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 40.460477756000003, 2024-05-08T08:55:06.395 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047574067999999997, 2024-05-08T08:55:06.395 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:06.395 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:06.395 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:06.396 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:06.396 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:06.396 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:06.396 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5600 2024-05-08T08:55:06.396 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.396 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:06.396 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.396 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:06.396 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884619+0000", 2024-05-08T08:55:06.396 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:06.396 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.396 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.396 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:06.396 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884619+0000", 2024-05-08T08:55:06.396 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:06.397 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.397 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.397 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:06.397 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884620+0000", 2024-05-08T08:55:06.397 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.53e-07 2024-05-08T08:55:06.397 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.397 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.397 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:06.397 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884622+0000", 2024-05-08T08:55:06.397 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.796e-06 2024-05-08T08:55:06.397 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.397 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.397 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:06.397 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884627+0000", 2024-05-08T08:55:06.397 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2850000000000002e-06 2024-05-08T08:55:06.397 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.398 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.398 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:06.398 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884631+0000", 2024-05-08T08:55:06.398 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2899999999999996e-06 2024-05-08T08:55:06.398 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.398 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.398 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:06.398 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932024+0000", 2024-05-08T08:55:06.398 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047392680999999999 2024-05-08T08:55:06.398 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.398 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.398 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:06.398 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932102+0000", 2024-05-08T08:55:06.398 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7806999999999996e-05 2024-05-08T08:55:06.399 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.399 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.399 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:06.399 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932193+0000", 2024-05-08T08:55:06.399 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1855999999999996e-05 2024-05-08T08:55:06.399 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.399 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:06.399 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.399 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.399 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.399 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5603 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 3993600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:06.399 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.884680+0000", 2024-05-08T08:55:06.399 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 40.460417147000001, 2024-05-08T08:55:06.399 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047745034999999998, 2024-05-08T08:55:06.399 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:06.400 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:06.400 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:06.400 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:06.400 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:06.400 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:06.400 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5603 2024-05-08T08:55:06.400 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.400 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:06.400 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.400 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:06.400 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884680+0000", 2024-05-08T08:55:06.400 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:06.400 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.400 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.401 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:06.401 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884680+0000", 2024-05-08T08:55:06.401 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:06.401 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.401 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.401 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:06.401 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884680+0000", 2024-05-08T08:55:06.401 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4400000000000001e-07 2024-05-08T08:55:06.401 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.401 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.401 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:06.401 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884682+0000", 2024-05-08T08:55:06.401 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.514e-06 2024-05-08T08:55:06.401 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.402 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.402 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:06.402 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884687+0000", 2024-05-08T08:55:06.402 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0359999999999997e-06 2024-05-08T08:55:06.402 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.402 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.402 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:06.402 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884692+0000", 2024-05-08T08:55:06.402 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5990000000000004e-06 2024-05-08T08:55:06.402 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.402 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.402 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:06.402 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932235+0000", 2024-05-08T08:55:06.402 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047543635000000001 2024-05-08T08:55:06.403 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.403 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.403 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:06.403 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932323+0000", 2024-05-08T08:55:06.403 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7311000000000005e-05 2024-05-08T08:55:06.403 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.403 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.403 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:06.403 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932425+0000", 2024-05-08T08:55:06.403 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102496 2024-05-08T08:55:06.404 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.404 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:06.404 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.404 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.404 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.404 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5606 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4038656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:06.404 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.884729+0000", 2024-05-08T08:55:06.404 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 40.460367859000002, 2024-05-08T08:55:06.404 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047916488, 2024-05-08T08:55:06.404 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:06.404 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:06.404 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:06.404 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:06.404 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:06.405 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:06.405 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5606 2024-05-08T08:55:06.405 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.405 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:06.405 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.405 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:06.405 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884729+0000", 2024-05-08T08:55:06.405 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:06.405 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.405 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.405 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:06.405 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884729+0000", 2024-05-08T08:55:06.405 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:06.405 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.405 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.406 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:06.406 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.885940+0000", 2024-05-08T08:55:06.406 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001210509 2024-05-08T08:55:06.406 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.406 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.406 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:06.406 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.885945+0000", 2024-05-08T08:55:06.406 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7389999999999999e-06 2024-05-08T08:55:06.406 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.406 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.406 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:06.406 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.885955+0000", 2024-05-08T08:55:06.406 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0813e-05 2024-05-08T08:55:06.406 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.406 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.407 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:06.407 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.885962+0000", 2024-05-08T08:55:06.407 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1160000000000001e-06 2024-05-08T08:55:06.407 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.407 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.407 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:06.407 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932465+0000", 2024-05-08T08:55:06.407 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046502612999999998 2024-05-08T08:55:06.407 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.407 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.407 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:06.407 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932547+0000", 2024-05-08T08:55:06.407 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2025000000000002e-05 2024-05-08T08:55:06.407 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.407 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.408 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:06.408 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932646+0000", 2024-05-08T08:55:06.408 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8672999999999995e-05 2024-05-08T08:55:06.408 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.408 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:06.408 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.408 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.408 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.408 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5635 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4235264~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:06.408 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.886082+0000", 2024-05-08T08:55:06.408 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 40.459014740000001, 2024-05-08T08:55:06.408 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047438284999999997, 2024-05-08T08:55:06.408 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:06.408 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:06.409 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:06.409 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:06.409 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:06.409 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:06.409 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5635 2024-05-08T08:55:06.409 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.409 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:06.409 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.409 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:06.409 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886082+0000", 2024-05-08T08:55:06.409 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:06.409 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.409 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.409 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:06.409 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886082+0000", 2024-05-08T08:55:06.409 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:06.409 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.410 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.410 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:06.410 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886083+0000", 2024-05-08T08:55:06.410 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8999999999999996e-07 2024-05-08T08:55:06.410 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.410 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.410 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:06.410 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886086+0000", 2024-05-08T08:55:06.410 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7e-06 2024-05-08T08:55:06.410 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.410 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.410 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:06.410 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886093+0000", 2024-05-08T08:55:06.410 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5320000000000001e-06 2024-05-08T08:55:06.411 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.411 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.411 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:06.411 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886102+0000", 2024-05-08T08:55:06.411 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8070000000000006e-06 2024-05-08T08:55:06.411 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.411 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.411 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:06.411 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933357+0000", 2024-05-08T08:55:06.411 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047255093999999997 2024-05-08T08:55:06.411 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.411 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.411 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:06.411 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933433+0000", 2024-05-08T08:55:06.412 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5796000000000004e-05 2024-05-08T08:55:06.412 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.412 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.412 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:06.412 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933521+0000", 2024-05-08T08:55:06.412 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7766e-05 2024-05-08T08:55:06.412 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.412 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:06.412 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.412 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.412 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.412 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5636 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4280320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:06.412 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.886111+0000", 2024-05-08T08:55:06.412 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 40.458986121999999, 2024-05-08T08:55:06.412 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047663644999999998, 2024-05-08T08:55:06.412 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:06.412 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:06.413 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:06.413 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:06.413 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:06.413 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:06.413 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5636 2024-05-08T08:55:06.413 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.413 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:06.413 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.413 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:06.413 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886111+0000", 2024-05-08T08:55:06.413 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:06.413 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.413 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.413 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:06.413 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886111+0000", 2024-05-08T08:55:06.414 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:06.414 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.414 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.414 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:06.414 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886111+0000", 2024-05-08T08:55:06.414 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5200000000000002e-07 2024-05-08T08:55:06.414 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.414 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.414 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:06.414 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886114+0000", 2024-05-08T08:55:06.414 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1710000000000001e-06 2024-05-08T08:55:06.414 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.414 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.414 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:06.414 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886120+0000", 2024-05-08T08:55:06.415 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.404e-06 2024-05-08T08:55:06.415 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.415 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.415 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:06.415 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886125+0000", 2024-05-08T08:55:06.415 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2730000000000002e-06 2024-05-08T08:55:06.415 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.415 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.415 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:06.415 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933570+0000", 2024-05-08T08:55:06.415 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047444989999999999 2024-05-08T08:55:06.415 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.415 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.415 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:06.415 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933659+0000", 2024-05-08T08:55:06.416 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8912999999999994e-05 2024-05-08T08:55:06.416 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.416 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.416 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:06.416 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933775+0000", 2024-05-08T08:55:06.416 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000115442 2024-05-08T08:55:06.416 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.416 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:06.416 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.416 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.416 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.416 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5637 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4325376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:06.416 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.886132+0000", 2024-05-08T08:55:06.416 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 40.458965083999999, 2024-05-08T08:55:06.416 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047852219000000001, 2024-05-08T08:55:06.416 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:06.417 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:06.417 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:06.417 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:06.417 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:06.417 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:06.417 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5637 2024-05-08T08:55:06.417 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.417 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:06.417 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.417 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:06.417 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886132+0000", 2024-05-08T08:55:06.417 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:06.417 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.417 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.417 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:06.417 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886132+0000", 2024-05-08T08:55:06.417 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:06.418 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.418 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.418 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:06.418 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886132+0000", 2024-05-08T08:55:06.418 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7300000000000002e-07 2024-05-08T08:55:06.418 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.418 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.418 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:06.418 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886139+0000", 2024-05-08T08:55:06.418 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6109999999999999e-06 2024-05-08T08:55:06.418 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.418 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.418 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:06.418 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886146+0000", 2024-05-08T08:55:06.418 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6279999999999999e-06 2024-05-08T08:55:06.419 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.419 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.419 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:06.419 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886151+0000", 2024-05-08T08:55:06.419 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1769999999999997e-06 2024-05-08T08:55:06.419 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.419 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.419 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:06.419 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933812+0000", 2024-05-08T08:55:06.419 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047660912999999999 2024-05-08T08:55:06.419 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.419 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.419 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:06.419 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933888+0000", 2024-05-08T08:55:06.419 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5865000000000006e-05 2024-05-08T08:55:06.419 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.420 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.420 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:06.420 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933984+0000", 2024-05-08T08:55:06.420 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6651999999999996e-05 2024-05-08T08:55:06.420 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.420 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:06.420 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.420 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.420 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.420 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5638 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4370432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:06.420 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.886157+0000", 2024-05-08T08:55:06.420 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 40.458939819999998, 2024-05-08T08:55:06.420 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048077597, 2024-05-08T08:55:06.420 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:06.420 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:06.421 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:06.421 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:06.421 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:06.421 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:06.421 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5638 2024-05-08T08:55:06.421 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.421 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:06.421 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.421 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:06.421 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886157+0000", 2024-05-08T08:55:06.421 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:06.421 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.421 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.421 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:06.421 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886157+0000", 2024-05-08T08:55:06.421 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:06.422 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.422 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.422 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:06.422 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886158+0000", 2024-05-08T08:55:06.422 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1100000000000001e-07 2024-05-08T08:55:06.422 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.422 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.422 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:06.422 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886160+0000", 2024-05-08T08:55:06.422 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9800000000000001e-06 2024-05-08T08:55:06.422 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.422 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.422 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:06.422 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886166+0000", 2024-05-08T08:55:06.422 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.314e-06 2024-05-08T08:55:06.423 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.423 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.423 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:06.423 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886171+0000", 2024-05-08T08:55:06.423 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7700000000000001e-06 2024-05-08T08:55:06.423 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.423 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.423 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:06.423 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.934025+0000", 2024-05-08T08:55:06.423 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047854509000000003 2024-05-08T08:55:06.423 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.423 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.423 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:06.423 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.934121+0000", 2024-05-08T08:55:06.423 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5688999999999998e-05 2024-05-08T08:55:06.424 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.424 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.424 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:06.424 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.934235+0000", 2024-05-08T08:55:06.424 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000113924 2024-05-08T08:55:06.424 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.424 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:06.424 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.424 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.424 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.424 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5639 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4415488~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:06.424 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.886177+0000", 2024-05-08T08:55:06.424 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 40.458920069999998, 2024-05-08T08:55:06.424 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048249541, 2024-05-08T08:55:06.424 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:06.425 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:06.425 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:06.425 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:06.425 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:06.425 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:06.425 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5639 2024-05-08T08:55:06.425 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.425 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:06.425 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.425 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:06.425 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886177+0000", 2024-05-08T08:55:06.425 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:06.425 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.425 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.426 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:06.426 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886177+0000", 2024-05-08T08:55:06.426 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:06.426 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.426 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.426 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:06.426 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886177+0000", 2024-05-08T08:55:06.426 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.65e-07 2024-05-08T08:55:06.426 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.426 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.426 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:06.426 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886179+0000", 2024-05-08T08:55:06.426 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9530000000000002e-06 2024-05-08T08:55:06.426 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.427 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.427 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:06.427 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886186+0000", 2024-05-08T08:55:06.427 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3419999999999996e-06 2024-05-08T08:55:06.427 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.427 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.427 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:06.427 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886192+0000", 2024-05-08T08:55:06.427 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0560000000000001e-06 2024-05-08T08:55:06.427 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.427 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.427 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:06.427 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.934258+0000", 2024-05-08T08:55:06.427 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048066367999999998 2024-05-08T08:55:06.427 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.427 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.428 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:06.428 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.934338+0000", 2024-05-08T08:55:06.428 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9616000000000005e-05 2024-05-08T08:55:06.428 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.428 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.428 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:06.428 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.934427+0000", 2024-05-08T08:55:06.428 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8840999999999999e-05 2024-05-08T08:55:06.428 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.428 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:06.428 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.428 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.428 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:06.428 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:55:06.430 INFO:teuthology.orchestra.run.smithi059.stderr:noscrub is unset 2024-05-08T08:55:06.431 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:55:06.431 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:55:06.432+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:55:06.530 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:55:06.533+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:55:06.540 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:55:06.540 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:55:06.540 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:55:06.540 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:55:06.549 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:55:06.631 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:55:06.634+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:55:06.660 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:55:06.660 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:55:06.661 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:55:06.661 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:55:06.661 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:55:06.670 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:55:06.733 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:55:06.735+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:55:06.784 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:55:06.894 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:55:06.894 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:55:06.894 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:55:06.894 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.894 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22073 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 782336~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:06.894 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:55:06.894 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 158.9288933, 2024-05-08T08:55:06.894 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047244234000000003, 2024-05-08T08:55:06.894 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:06.895 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:06.898 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:06.898 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:06.898 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:06.898 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:06.898 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22073 2024-05-08T08:55:06.898 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.899 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:06.899 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.899 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:06.899 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:55:06.899 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:06.899 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.899 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.899 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:06.899 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:55:06.899 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:06.899 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.899 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.899 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:06.899 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857668+0000", 2024-05-08T08:55:06.900 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7800000000000002e-07 2024-05-08T08:55:06.900 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.900 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.900 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:06.900 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857670+0000", 2024-05-08T08:55:06.900 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0820000000000001e-06 2024-05-08T08:55:06.900 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.900 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.900 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:06.900 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857678+0000", 2024-05-08T08:55:06.900 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7589999999999997e-06 2024-05-08T08:55:06.900 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.900 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.900 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:06.900 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857691+0000", 2024-05-08T08:55:06.901 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3672999999999999e-05 2024-05-08T08:55:06.901 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.901 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.901 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:06.901 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904780+0000", 2024-05-08T08:55:06.901 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047088616999999999 2024-05-08T08:55:06.901 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.901 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.901 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:06.901 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904840+0000", 2024-05-08T08:55:06.901 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0356999999999998e-05 2024-05-08T08:55:06.901 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.902 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.902 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:06.902 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904912+0000", 2024-05-08T08:55:06.902 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1367999999999998e-05 2024-05-08T08:55:06.902 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.902 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:06.902 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.902 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.902 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.902 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22075 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 798720~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:06.903 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:55:06.903 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 158.92886290300001, 2024-05-08T08:55:06.903 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047384643999999997, 2024-05-08T08:55:06.903 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:06.903 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:06.903 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:06.903 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:06.903 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:06.903 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:06.903 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22075 2024-05-08T08:55:06.903 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.903 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:06.904 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.904 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:06.904 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:55:06.904 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:06.904 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.904 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.904 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:06.904 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:55:06.904 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:06.904 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.904 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.904 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:06.904 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:55:06.904 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7399999999999999e-07 2024-05-08T08:55:06.904 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.905 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.905 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:06.905 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857705+0000", 2024-05-08T08:55:06.905 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1980000000000004e-06 2024-05-08T08:55:06.905 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.905 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.905 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:06.905 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857713+0000", 2024-05-08T08:55:06.905 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7039999999999994e-06 2024-05-08T08:55:06.905 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.905 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.905 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:06.906 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857718+0000", 2024-05-08T08:55:06.906 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.865e-06 2024-05-08T08:55:06.906 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.906 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.906 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:06.906 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904941+0000", 2024-05-08T08:55:06.906 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047223198000000001 2024-05-08T08:55:06.906 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.906 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.906 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:06.906 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904999+0000", 2024-05-08T08:55:06.906 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8254999999999997e-05 2024-05-08T08:55:06.906 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.906 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.907 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:06.907 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905082+0000", 2024-05-08T08:55:06.907 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3049999999999999e-05 2024-05-08T08:55:06.907 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.907 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:06.907 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.907 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.907 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.907 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22077 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 815104~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:06.907 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:55:06.907 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 158.92882489199999, 2024-05-08T08:55:06.907 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047519448999999998, 2024-05-08T08:55:06.907 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:06.907 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:06.907 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:06.907 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:06.908 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:06.908 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:06.908 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22077 2024-05-08T08:55:06.908 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.908 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:06.908 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.908 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:06.908 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:55:06.908 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:06.908 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.908 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.908 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:06.908 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:55:06.908 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:06.908 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.909 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.909 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:06.909 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:55:06.909 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.99e-07 2024-05-08T08:55:06.909 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.909 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.909 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:06.909 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857739+0000", 2024-05-08T08:55:06.909 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3599999999999999e-06 2024-05-08T08:55:06.909 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.909 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.909 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:06.910 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857745+0000", 2024-05-08T08:55:06.910 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5139999999999998e-06 2024-05-08T08:55:06.910 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.910 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.910 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:06.910 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857750+0000", 2024-05-08T08:55:06.910 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8550000000000001e-06 2024-05-08T08:55:06.910 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.910 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.910 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:06.910 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905110+0000", 2024-05-08T08:55:06.910 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047360126000000002 2024-05-08T08:55:06.910 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.910 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.911 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:06.911 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905175+0000", 2024-05-08T08:55:06.911 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4689000000000004e-05 2024-05-08T08:55:06.911 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.911 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.911 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:06.911 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905255+0000", 2024-05-08T08:55:06.911 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0306e-05 2024-05-08T08:55:06.911 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.911 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:06.911 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.911 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.912 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.912 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22079 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 831488~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:06.912 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:55:06.912 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 158.92880434400001, 2024-05-08T08:55:06.912 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.04767499, 2024-05-08T08:55:06.912 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:06.912 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:06.912 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:06.912 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:06.912 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:06.912 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:06.912 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22079 2024-05-08T08:55:06.913 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.913 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:06.913 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.913 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:06.913 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:55:06.913 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:06.913 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.913 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.913 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:06.914 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:55:06.914 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:06.914 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.914 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.914 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:06.914 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857757+0000", 2024-05-08T08:55:06.914 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8299999999999997e-07 2024-05-08T08:55:06.914 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.914 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.914 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:06.914 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857759+0000", 2024-05-08T08:55:06.914 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1390000000000002e-06 2024-05-08T08:55:06.914 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.914 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.914 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:06.915 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857765+0000", 2024-05-08T08:55:06.915 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4609999999999996e-06 2024-05-08T08:55:06.915 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.915 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.915 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:06.915 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857770+0000", 2024-05-08T08:55:06.915 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6689999999999997e-06 2024-05-08T08:55:06.915 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.915 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.915 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:06.915 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905281+0000", 2024-05-08T08:55:06.915 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047510668999999998 2024-05-08T08:55:06.915 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.915 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.916 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:06.916 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905344+0000", 2024-05-08T08:55:06.916 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2797000000000002e-05 2024-05-08T08:55:06.916 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.916 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.916 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:06.916 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905431+0000", 2024-05-08T08:55:06.916 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7771999999999999e-05 2024-05-08T08:55:06.916 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.916 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:06.916 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.916 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.916 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.916 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22082 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 847872~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:06.916 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:55:06.917 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 158.92878465300001, 2024-05-08T08:55:06.917 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047831934999999999, 2024-05-08T08:55:06.917 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:06.917 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:06.917 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:06.917 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:06.917 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:06.917 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:06.917 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22082 2024-05-08T08:55:06.917 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.917 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:06.917 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.917 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:06.917 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:55:06.917 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:06.917 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.918 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.918 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:06.918 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:55:06.918 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:06.918 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.918 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.918 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:06.918 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:55:06.919 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:55:06.919 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.919 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.919 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:06.919 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857778+0000", 2024-05-08T08:55:06.919 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0789999999999999e-06 2024-05-08T08:55:06.919 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.919 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.919 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:06.919 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857784+0000", 2024-05-08T08:55:06.919 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0480000000000004e-06 2024-05-08T08:55:06.919 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.919 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.920 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:06.920 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857790+0000", 2024-05-08T08:55:06.920 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2719999999999997e-06 2024-05-08T08:55:06.920 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.920 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.920 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:06.920 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905458+0000", 2024-05-08T08:55:06.920 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047668470999999997 2024-05-08T08:55:06.920 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.920 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.920 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:06.920 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905521+0000", 2024-05-08T08:55:06.921 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2952000000000003e-05 2024-05-08T08:55:06.921 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.921 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.921 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:06.921 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905608+0000", 2024-05-08T08:55:06.921 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6759000000000001e-05 2024-05-08T08:55:06.921 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.921 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:06.921 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.921 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.921 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.921 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22084 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 864256~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:06.921 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:55:06.921 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 158.92876070299999, 2024-05-08T08:55:06.921 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047977748000000001, 2024-05-08T08:55:06.922 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:06.922 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:06.922 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:06.922 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:06.922 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:06.922 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:06.922 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22084 2024-05-08T08:55:06.922 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.922 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:06.922 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.922 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:06.922 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:55:06.923 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:06.923 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.923 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.923 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:06.923 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:55:06.923 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:06.923 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.923 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.924 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:06.924 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:55:06.924 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:55:06.924 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.924 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.924 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:06.924 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857802+0000", 2024-05-08T08:55:06.924 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0779999999999998e-06 2024-05-08T08:55:06.924 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.924 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.924 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:06.925 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857812+0000", 2024-05-08T08:55:06.925 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7939999999999997e-06 2024-05-08T08:55:06.925 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.925 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.925 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:06.925 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857817+0000", 2024-05-08T08:55:06.925 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8899999999999998e-06 2024-05-08T08:55:06.925 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.925 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.925 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:06.925 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905636+0000", 2024-05-08T08:55:06.925 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047819252 2024-05-08T08:55:06.925 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.925 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.925 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:06.926 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905700+0000", 2024-05-08T08:55:06.926 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4016999999999994e-05 2024-05-08T08:55:06.926 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.926 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.926 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:06.926 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905778+0000", 2024-05-08T08:55:06.926 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7306999999999997e-05 2024-05-08T08:55:06.926 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.926 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:06.926 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.926 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.926 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.926 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22086 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 880640~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:06.926 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:55:06.926 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 158.928737353, 2024-05-08T08:55:06.927 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048119149, 2024-05-08T08:55:06.927 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:06.927 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:06.927 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:06.927 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:06.927 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:06.927 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:06.927 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22086 2024-05-08T08:55:06.927 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.927 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:06.927 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.927 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:06.927 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:55:06.927 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:06.928 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.928 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.928 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:06.928 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:55:06.928 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:06.928 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.928 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.928 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:06.928 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857824+0000", 2024-05-08T08:55:06.928 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4200000000000002e-07 2024-05-08T08:55:06.928 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.928 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.928 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:06.928 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857826+0000", 2024-05-08T08:55:06.928 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0590000000000001e-06 2024-05-08T08:55:06.929 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.929 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.929 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:06.929 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857832+0000", 2024-05-08T08:55:06.929 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8029999999999999e-06 2024-05-08T08:55:06.929 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.929 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.929 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:06.929 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857837+0000", 2024-05-08T08:55:06.929 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7929999999999997e-06 2024-05-08T08:55:06.929 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.929 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.929 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:06.929 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905803+0000", 2024-05-08T08:55:06.929 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047965761000000003 2024-05-08T08:55:06.930 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.930 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.930 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:06.930 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905862+0000", 2024-05-08T08:55:06.930 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9203000000000002e-05 2024-05-08T08:55:06.930 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.930 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.930 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:06.930 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905942+0000", 2024-05-08T08:55:06.930 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0187999999999999e-05 2024-05-08T08:55:06.930 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.930 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:06.930 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.930 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.930 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.931 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22088 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 897024~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:06.931 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:55:06.931 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 158.92871730100001, 2024-05-08T08:55:06.931 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048256049000000002, 2024-05-08T08:55:06.931 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:06.931 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:06.931 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:06.931 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:06.931 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:06.931 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:06.931 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22088 2024-05-08T08:55:06.931 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.931 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:06.931 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.932 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:06.932 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:55:06.932 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:06.932 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.932 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.932 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:06.932 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:55:06.932 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:06.932 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.932 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.932 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:06.932 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857844+0000", 2024-05-08T08:55:06.932 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.46e-07 2024-05-08T08:55:06.933 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.933 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.933 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:06.933 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857846+0000", 2024-05-08T08:55:06.933 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0899999999999999e-06 2024-05-08T08:55:06.933 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.933 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.933 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:06.933 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857855+0000", 2024-05-08T08:55:06.934 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8219999999999996e-06 2024-05-08T08:55:06.934 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.934 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.934 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:06.934 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857860+0000", 2024-05-08T08:55:06.934 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2989999999999996e-06 2024-05-08T08:55:06.934 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.934 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.934 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:06.935 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905966+0000", 2024-05-08T08:55:06.935 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048106542000000002 2024-05-08T08:55:06.935 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.935 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.935 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:06.935 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906024+0000", 2024-05-08T08:55:06.935 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7958000000000003e-05 2024-05-08T08:55:06.935 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.935 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.935 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:06.935 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906099+0000", 2024-05-08T08:55:06.935 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4992000000000003e-05 2024-05-08T08:55:06.935 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.935 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:06.936 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.936 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.936 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.936 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22091 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 913408~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:06.936 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:55:06.936 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 158.928693541, 2024-05-08T08:55:06.936 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048395290000000001, 2024-05-08T08:55:06.936 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:06.936 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:06.936 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:06.936 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:06.936 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:06.936 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:06.936 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22091 2024-05-08T08:55:06.936 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.937 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:06.937 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.937 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:06.937 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:55:06.937 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:06.937 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.937 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.937 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:06.937 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:55:06.937 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:06.937 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.937 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.937 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:06.937 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:55:06.937 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4900000000000001e-07 2024-05-08T08:55:06.937 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.938 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.938 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:06.938 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857870+0000", 2024-05-08T08:55:06.938 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2510000000000001e-06 2024-05-08T08:55:06.938 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.938 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.938 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:06.938 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857877+0000", 2024-05-08T08:55:06.938 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0809999999999996e-06 2024-05-08T08:55:06.938 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.938 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.938 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:06.938 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857881+0000", 2024-05-08T08:55:06.938 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.724e-06 2024-05-08T08:55:06.938 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.939 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.939 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:06.939 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906127+0000", 2024-05-08T08:55:06.939 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048245345000000002 2024-05-08T08:55:06.939 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.939 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.939 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:06.939 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906190+0000", 2024-05-08T08:55:06.939 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2762000000000007e-05 2024-05-08T08:55:06.939 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.939 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.939 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:06.939 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906262+0000", 2024-05-08T08:55:06.939 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2778e-05 2024-05-08T08:55:06.939 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.939 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:06.940 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.940 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.940 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.940 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22093 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 929792~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:06.940 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:55:06.940 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 158.92867117700001, 2024-05-08T08:55:06.940 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048532875000000003, 2024-05-08T08:55:06.940 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:06.940 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:06.940 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:06.940 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:06.940 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:06.940 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:06.940 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22093 2024-05-08T08:55:06.940 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.940 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:06.941 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.941 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:06.941 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:55:06.941 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:06.941 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.941 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.941 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:06.941 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:55:06.941 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:06.941 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.941 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.941 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:06.941 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857890+0000", 2024-05-08T08:55:06.941 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-07 2024-05-08T08:55:06.941 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.942 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.942 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:06.942 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857899+0000", 2024-05-08T08:55:06.942 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6889999999999999e-06 2024-05-08T08:55:06.942 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.942 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.942 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:06.942 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857906+0000", 2024-05-08T08:55:06.942 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6490000000000002e-06 2024-05-08T08:55:06.942 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.942 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.942 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:06.942 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857917+0000", 2024-05-08T08:55:06.942 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0764999999999999e-05 2024-05-08T08:55:06.943 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.943 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.943 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:06.943 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906287+0000", 2024-05-08T08:55:06.943 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048370442 2024-05-08T08:55:06.943 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.943 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.943 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:06.943 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906349+0000", 2024-05-08T08:55:06.943 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1537000000000004e-05 2024-05-08T08:55:06.943 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.943 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.943 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:06.943 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906422+0000", 2024-05-08T08:55:06.944 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3431999999999998e-05 2024-05-08T08:55:06.944 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.944 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:06.944 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.944 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.944 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.944 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22095 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 946176~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:06.944 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:55:06.944 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 158.92863647300001, 2024-05-08T08:55:06.944 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048657593999999998, 2024-05-08T08:55:06.944 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:06.944 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:06.945 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:06.945 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:06.945 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:06.945 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:06.945 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22095 2024-05-08T08:55:06.945 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.945 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:06.946 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.946 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:06.946 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:55:06.946 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:06.946 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.946 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.946 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:06.946 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:55:06.946 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:06.946 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.946 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.946 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:06.946 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857925+0000", 2024-05-08T08:55:06.946 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3099999999999998e-07 2024-05-08T08:55:06.946 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.947 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.947 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:06.947 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857927+0000", 2024-05-08T08:55:06.947 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.34e-06 2024-05-08T08:55:06.947 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.947 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.947 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:06.947 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857938+0000", 2024-05-08T08:55:06.947 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1426e-05 2024-05-08T08:55:06.947 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.947 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.947 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:06.947 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857943+0000", 2024-05-08T08:55:06.947 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5140000000000003e-06 2024-05-08T08:55:06.947 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.948 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.948 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:06.948 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906448+0000", 2024-05-08T08:55:06.948 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048504641000000001 2024-05-08T08:55:06.948 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.948 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.948 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:06.948 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906510+0000", 2024-05-08T08:55:06.948 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2663000000000006e-05 2024-05-08T08:55:06.948 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.948 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.948 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:06.948 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906582+0000", 2024-05-08T08:55:06.948 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1478999999999998e-05 2024-05-08T08:55:06.948 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.949 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:06.949 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.949 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.949 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.949 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22097 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 962560~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:06.949 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:55:06.949 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 158.92860878400001, 2024-05-08T08:55:06.949 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048791353000000003, 2024-05-08T08:55:06.949 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:06.949 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:06.949 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:06.949 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:06.949 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:06.949 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:06.949 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22097 2024-05-08T08:55:06.950 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.950 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:06.950 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.950 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:06.950 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:55:06.950 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:06.950 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.950 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.950 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:06.950 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:55:06.950 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:06.950 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.950 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.951 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:06.951 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:55:06.951 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9200000000000002e-07 2024-05-08T08:55:06.951 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.951 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.951 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:06.951 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857954+0000", 2024-05-08T08:55:06.951 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1900000000000002e-06 2024-05-08T08:55:06.951 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.951 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.951 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:06.951 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857961+0000", 2024-05-08T08:55:06.951 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4989999999999999e-06 2024-05-08T08:55:06.951 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.951 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.952 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:06.952 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857971+0000", 2024-05-08T08:55:06.952 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0485999999999999e-05 2024-05-08T08:55:06.952 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.952 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.952 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:06.952 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906607+0000", 2024-05-08T08:55:06.952 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048635109000000003 2024-05-08T08:55:06.952 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.952 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.952 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:06.952 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906667+0000", 2024-05-08T08:55:06.952 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0439000000000002e-05 2024-05-08T08:55:06.952 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.953 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.953 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:06.953 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906743+0000", 2024-05-08T08:55:06.953 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6237999999999998e-05 2024-05-08T08:55:06.953 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.953 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:06.953 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.953 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.953 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.953 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22099 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 978944~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:06.953 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:55:06.953 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 158.928582624, 2024-05-08T08:55:06.953 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048932055000000002, 2024-05-08T08:55:06.953 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:06.953 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:06.954 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:06.954 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:06.954 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:06.954 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:06.954 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22099 2024-05-08T08:55:06.954 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.954 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:06.954 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.954 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:06.954 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:55:06.954 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:06.954 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.954 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.954 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:06.954 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:55:06.955 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:06.955 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.955 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.955 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:06.955 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:55:06.955 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.27e-07 2024-05-08T08:55:06.955 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.955 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.955 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:06.955 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857987+0000", 2024-05-08T08:55:06.955 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5890000000000003e-06 2024-05-08T08:55:06.955 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.955 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.956 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:06.956 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857994+0000", 2024-05-08T08:55:06.956 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6619999999999999e-06 2024-05-08T08:55:06.956 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.956 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.956 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:06.956 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857998+0000", 2024-05-08T08:55:06.956 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7110000000000003e-06 2024-05-08T08:55:06.956 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.956 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.956 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:06.956 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906772+0000", 2024-05-08T08:55:06.957 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048773415000000001 2024-05-08T08:55:06.957 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.957 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.957 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:06.957 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906834+0000", 2024-05-08T08:55:06.957 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1752000000000001e-05 2024-05-08T08:55:06.957 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.957 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.957 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:06.957 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906910+0000", 2024-05-08T08:55:06.957 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6499000000000001e-05 2024-05-08T08:55:06.957 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.957 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:06.957 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.957 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.958 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.958 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22101 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 995328~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:06.958 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:55:06.958 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 158.928555493, 2024-05-08T08:55:06.958 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049087364000000001, 2024-05-08T08:55:06.958 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:06.958 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:06.958 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:06.958 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:06.958 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:06.958 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:06.958 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22101 2024-05-08T08:55:06.958 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.958 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:06.958 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.959 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:06.959 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:55:06.959 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:06.959 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.959 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.959 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:06.959 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:55:06.959 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:06.959 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.959 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.959 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:06.960 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858006+0000", 2024-05-08T08:55:06.960 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1399999999999997e-07 2024-05-08T08:55:06.960 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.960 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.960 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:06.960 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858008+0000", 2024-05-08T08:55:06.960 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2060000000000001e-06 2024-05-08T08:55:06.960 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.960 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.960 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:06.960 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858017+0000", 2024-05-08T08:55:06.960 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1579999999999994e-06 2024-05-08T08:55:06.961 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.961 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.961 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:06.961 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858022+0000", 2024-05-08T08:55:06.961 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4199999999999998e-06 2024-05-08T08:55:06.961 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.962 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.962 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:06.962 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906940+0000", 2024-05-08T08:55:06.962 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048917627999999998 2024-05-08T08:55:06.962 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.962 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.962 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:06.962 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907002+0000", 2024-05-08T08:55:06.962 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2399999999999999e-05 2024-05-08T08:55:06.962 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.962 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.962 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:06.962 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907092+0000", 2024-05-08T08:55:06.963 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0137999999999997e-05 2024-05-08T08:55:06.963 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.963 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:06.963 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.963 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.963 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.963 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22103 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1011712~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:06.963 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:55:06.963 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 158.92853070800001, 2024-05-08T08:55:06.963 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049228542, 2024-05-08T08:55:06.963 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:06.963 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:06.963 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:06.963 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:06.964 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:06.964 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:06.964 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22103 2024-05-08T08:55:06.964 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.964 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:06.964 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.964 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:06.964 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:55:06.964 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:06.964 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.964 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.964 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:06.964 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:55:06.964 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:06.965 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.965 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.965 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:06.965 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:55:06.965 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4299999999999999e-07 2024-05-08T08:55:06.965 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.965 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.965 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:06.965 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858032+0000", 2024-05-08T08:55:06.965 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0590000000000001e-06 2024-05-08T08:55:06.965 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.965 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.965 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:06.965 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858037+0000", 2024-05-08T08:55:06.965 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.977e-06 2024-05-08T08:55:06.966 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.966 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.966 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:06.966 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858042+0000", 2024-05-08T08:55:06.966 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7650000000000001e-06 2024-05-08T08:55:06.966 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.966 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.966 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:06.966 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907118+0000", 2024-05-08T08:55:06.967 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049076147000000001 2024-05-08T08:55:06.967 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.967 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.967 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:06.967 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907180+0000", 2024-05-08T08:55:06.967 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1971999999999996e-05 2024-05-08T08:55:06.967 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.967 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.967 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:06.967 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907258+0000", 2024-05-08T08:55:06.967 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8279e-05 2024-05-08T08:55:06.967 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.967 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:06.967 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.967 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.967 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.968 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22105 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1028096~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:06.968 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:55:06.968 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 158.92851146500001, 2024-05-08T08:55:06.968 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049394609999999999, 2024-05-08T08:55:06.968 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:06.968 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:06.968 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:06.968 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:06.968 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:06.968 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:06.968 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22105 2024-05-08T08:55:06.968 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.968 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:06.968 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.968 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:06.968 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:55:06.969 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:06.969 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.969 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.969 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:06.969 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:55:06.969 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:06.969 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.969 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.969 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:06.969 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858050+0000", 2024-05-08T08:55:06.969 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5999999999999999e-07 2024-05-08T08:55:06.969 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.969 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.969 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:06.970 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858051+0000", 2024-05-08T08:55:06.970 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8870000000000001e-06 2024-05-08T08:55:06.970 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.970 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.970 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:06.970 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858057+0000", 2024-05-08T08:55:06.970 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8019999999999997e-06 2024-05-08T08:55:06.970 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.970 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.970 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:06.970 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858062+0000", 2024-05-08T08:55:06.970 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.561e-06 2024-05-08T08:55:06.971 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.971 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.971 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:06.971 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907289+0000", 2024-05-08T08:55:06.971 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049226957000000002 2024-05-08T08:55:06.971 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.971 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.971 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:06.971 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907356+0000", 2024-05-08T08:55:06.971 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6915000000000006e-05 2024-05-08T08:55:06.971 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.971 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.971 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:06.972 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907444+0000", 2024-05-08T08:55:06.972 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8127999999999994e-05 2024-05-08T08:55:06.972 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.972 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:06.972 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.972 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.972 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.972 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22107 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1044480~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:06.972 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:55:06.972 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 158.92849312499999, 2024-05-08T08:55:06.972 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049799942999999999, 2024-05-08T08:55:06.972 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:06.972 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:06.972 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:06.972 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:06.973 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:06.973 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:06.973 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22107 2024-05-08T08:55:06.973 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.973 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:06.973 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.973 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:06.973 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:55:06.973 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:06.973 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.973 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.973 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:06.973 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:55:06.973 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:06.973 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.974 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.974 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:06.974 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:55:06.974 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4799999999999999e-07 2024-05-08T08:55:06.974 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.974 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.974 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:06.974 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858070+0000", 2024-05-08T08:55:06.974 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.012e-06 2024-05-08T08:55:06.974 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.974 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.974 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:06.974 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858077+0000", 2024-05-08T08:55:06.974 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6290000000000001e-06 2024-05-08T08:55:06.975 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.975 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.975 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:06.975 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858083+0000", 2024-05-08T08:55:06.975 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8880000000000002e-06 2024-05-08T08:55:06.975 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.975 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.975 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:06.975 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907671+0000", 2024-05-08T08:55:06.975 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049587191000000003 2024-05-08T08:55:06.975 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.975 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.975 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:06.975 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907763+0000", 2024-05-08T08:55:06.976 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2231000000000005e-05 2024-05-08T08:55:06.976 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.976 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.976 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:06.976 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907867+0000", 2024-05-08T08:55:06.976 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104644 2024-05-08T08:55:06.976 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.976 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:06.976 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.976 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.976 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.976 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22294 3.1c 3:3aad0389:::rbd_data.11406886b07b.0000000000000080:head [stat,delete] snapc 7=[7] RETRY=6 ondisk+retry+write+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:06.976 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:55:06.976 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 158.92789927699999, 2024-05-08T08:55:06.976 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051152191, 2024-05-08T08:55:06.977 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:06.977 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:06.977 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:06.977 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:06.977 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:06.977 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:06.977 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22294 2024-05-08T08:55:06.977 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.977 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:06.977 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.977 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:06.978 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:55:06.978 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:06.978 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.978 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.978 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:06.978 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:55:06.978 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:06.978 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.978 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.978 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:06.978 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858662+0000", 2024-05-08T08:55:06.978 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4799999999999998e-07 2024-05-08T08:55:06.978 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.978 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.978 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:06.979 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858664+0000", 2024-05-08T08:55:06.979 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1229999999999998e-06 2024-05-08T08:55:06.979 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.979 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.979 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:06.979 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858670+0000", 2024-05-08T08:55:06.979 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3099999999999997e-06 2024-05-08T08:55:06.979 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.979 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.979 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:06.979 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858676+0000", 2024-05-08T08:55:06.979 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9750000000000004e-06 2024-05-08T08:55:06.979 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.979 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.979 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:06.979 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907901+0000", 2024-05-08T08:55:06.980 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049224520000000001 2024-05-08T08:55:06.980 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.980 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.980 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:06.980 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.908085+0000", 2024-05-08T08:55:06.980 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00018377500000000001 2024-05-08T08:55:06.980 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.980 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.980 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:06.980 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.909814+0000", 2024-05-08T08:55:06.980 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00172894 2024-05-08T08:55:06.980 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.980 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:06.980 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.980 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.981 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.981 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4416.0:13461 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-05-08T08:55:06.981 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:22.487456+0000", 2024-05-08T08:55:06.981 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 104.29910423600001, 2024-05-08T08:55:06.981 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.99102620799999996, 2024-05-08T08:55:06.981 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:06.981 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:06.981 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:06.981 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:06.981 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4416", 2024-05-08T08:55:06.981 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:55:06.981 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 13461 2024-05-08T08:55:06.981 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.981 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:06.981 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.981 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:06.982 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487456+0000", 2024-05-08T08:55:06.982 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:06.982 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.982 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.982 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:06.982 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487456+0000", 2024-05-08T08:55:06.982 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:06.982 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.982 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.982 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:06.982 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487458+0000", 2024-05-08T08:55:06.982 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5659999999999999e-06 2024-05-08T08:55:06.982 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.982 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.983 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:06.983 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487463+0000", 2024-05-08T08:55:06.983 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6890000000000003e-06 2024-05-08T08:55:06.983 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.983 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.983 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:06.983 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487472+0000", 2024-05-08T08:55:06.983 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7710000000000001e-06 2024-05-08T08:55:06.983 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.983 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.983 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:06.983 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487478+0000", 2024-05-08T08:55:06.983 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1390000000000003e-06 2024-05-08T08:55:06.983 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.983 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.984 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:06.984 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487520+0000", 2024-05-08T08:55:06.984 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2259999999999999e-05 2024-05-08T08:55:06.984 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.984 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.984 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-05-08T08:55:06.984 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487528+0000", 2024-05-08T08:55:06.984 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7479999999999993e-06 2024-05-08T08:55:06.984 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.984 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.984 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:06.984 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:23.478139+0000", 2024-05-08T08:55:06.984 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.99061165200000001 2024-05-08T08:55:06.984 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.984 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.985 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:06.985 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:23.478301+0000", 2024-05-08T08:55:06.985 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000161602 2024-05-08T08:55:06.985 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.985 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.985 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:06.985 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:23.478483+0000", 2024-05-08T08:55:06.985 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000181781 2024-05-08T08:55:06.985 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.985 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:06.985 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.985 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.985 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.985 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5752.0:404 3.1a 3:5f5e8104:::rbd_data.1140bc4a55c7.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 5175296~45056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-05-08T08:55:06.985 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:00.390863+0000", 2024-05-08T08:55:06.986 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 66.395698041000003, 2024-05-08T08:55:06.986 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.055189174000000001, 2024-05-08T08:55:06.986 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:06.986 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:06.986 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:55:06.986 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:06.986 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5752", 2024-05-08T08:55:06.986 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/3534419815", 2024-05-08T08:55:06.986 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 404 2024-05-08T08:55:06.986 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.986 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:06.986 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.986 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:06.986 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.390863+0000", 2024-05-08T08:55:06.987 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:06.987 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.987 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.987 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:06.987 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.390863+0000", 2024-05-08T08:55:06.987 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:06.987 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.987 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.987 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:06.987 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.390864+0000", 2024-05-08T08:55:06.987 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.471e-06 2024-05-08T08:55:06.988 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.988 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.988 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:06.988 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397475+0000", 2024-05-08T08:55:06.988 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0066109109999999997 2024-05-08T08:55:06.988 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.988 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.988 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:06.988 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397498+0000", 2024-05-08T08:55:06.988 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2674e-05 2024-05-08T08:55:06.988 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.988 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.988 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:06.989 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397508+0000", 2024-05-08T08:55:06.989 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9019999999999994e-06 2024-05-08T08:55:06.989 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.989 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.989 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:06.989 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397609+0000", 2024-05-08T08:55:06.989 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010121899999999999 2024-05-08T08:55:06.989 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.989 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.989 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:06.989 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397711+0000", 2024-05-08T08:55:06.989 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102489 2024-05-08T08:55:06.989 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.989 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.989 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-05-08T08:55:06.989 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397999+0000", 2024-05-08T08:55:06.990 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00028733799999999999 2024-05-08T08:55:06.990 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.990 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.990 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:55:06.990 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.432093+0000", 2024-05-08T08:55:06.990 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.034094865000000002 2024-05-08T08:55:06.990 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.990 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.990 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:55:06.990 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.445975+0000", 2024-05-08T08:55:06.990 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013881187 2024-05-08T08:55:06.990 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.990 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.990 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:55:06.990 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.446013+0000", 2024-05-08T08:55:06.991 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8155000000000003e-05 2024-05-08T08:55:06.991 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:06.991 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:06.991 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:06.991 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.446052+0000", 2024-05-08T08:55:06.991 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8962999999999999e-05 2024-05-08T08:55:06.991 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.991 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:06.991 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.991 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:06.991 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:06.991 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:55:06.993 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:55:06.993 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:55:06.996+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:55:07.094 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:55:07.096+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:55:07.139 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:55:07.139 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:55:07.140 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:55:07.140 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:55:07.148 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:55:07.194 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:55:07.198+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:55:07.292 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:55:07.293 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:55:07.293 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:55:07.293 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:55:07.293 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:55:07.295 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:55:07.297+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:55:07.301 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:55:07.396 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:55:07.399+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:55:07.452 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:55:07.452 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:55:07.452 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:55:07.452 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:55:07.452 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.452 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6690 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1093632~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:07.452 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:55:07.452 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 155.738338524, 2024-05-08T08:55:07.452 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052462754, 2024-05-08T08:55:07.452 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:07.452 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:07.453 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:07.453 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:07.453 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:07.453 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:07.453 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6690 2024-05-08T08:55:07.453 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.453 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:07.453 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.453 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:07.453 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:55:07.453 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:07.453 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.453 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.454 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:07.454 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:55:07.454 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:07.454 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.454 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.454 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:07.454 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715346+0000", 2024-05-08T08:55:07.454 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.84e-07 2024-05-08T08:55:07.454 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.454 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.454 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:07.454 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715348+0000", 2024-05-08T08:55:07.454 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.114e-06 2024-05-08T08:55:07.455 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.455 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.455 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:07.455 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715353+0000", 2024-05-08T08:55:07.455 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5720000000000002e-06 2024-05-08T08:55:07.455 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.455 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.455 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:07.455 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715358+0000", 2024-05-08T08:55:07.455 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7140000000000001e-06 2024-05-08T08:55:07.455 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.455 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.455 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:07.455 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767638+0000", 2024-05-08T08:55:07.455 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052279562000000002 2024-05-08T08:55:07.456 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.456 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.456 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:07.456 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767713+0000", 2024-05-08T08:55:07.456 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5652e-05 2024-05-08T08:55:07.456 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.456 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.456 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:07.456 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767808+0000", 2024-05-08T08:55:07.456 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4755999999999999e-05 2024-05-08T08:55:07.456 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:07.456 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:07.456 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:07.457 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.457 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.457 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6692 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1110016~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:07.457 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:55:07.457 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 155.738319434, 2024-05-08T08:55:07.457 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052666495000000001, 2024-05-08T08:55:07.457 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:07.457 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:07.457 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:07.457 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:07.457 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:07.457 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:07.457 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6692 2024-05-08T08:55:07.457 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.457 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:07.458 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.458 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:07.458 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:55:07.458 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:07.458 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.458 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.458 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:07.458 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:55:07.458 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:07.458 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.458 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.458 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:07.458 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:55:07.458 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8599999999999999e-07 2024-05-08T08:55:07.459 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.459 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.459 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:07.459 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715372+0000", 2024-05-08T08:55:07.459 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0990000000000001e-06 2024-05-08T08:55:07.459 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.459 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.459 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:07.459 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715378+0000", 2024-05-08T08:55:07.459 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0680000000000002e-06 2024-05-08T08:55:07.459 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.459 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.459 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:07.459 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715383+0000", 2024-05-08T08:55:07.459 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8740000000000003e-06 2024-05-08T08:55:07.460 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.460 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.460 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:07.460 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767848+0000", 2024-05-08T08:55:07.460 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052464668999999998 2024-05-08T08:55:07.460 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.460 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.460 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:07.460 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767927+0000", 2024-05-08T08:55:07.460 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9228999999999996e-05 2024-05-08T08:55:07.460 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.460 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.460 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:07.461 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768031+0000", 2024-05-08T08:55:07.461 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010417 2024-05-08T08:55:07.461 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:07.461 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:07.461 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:07.461 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.461 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.461 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6694 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1126400~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:07.461 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:55:07.461 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 155.738294537, 2024-05-08T08:55:07.461 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052848829, 2024-05-08T08:55:07.461 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:07.461 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:07.461 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:07.461 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:07.462 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:07.462 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:07.462 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6694 2024-05-08T08:55:07.462 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.462 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:07.462 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.462 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:07.462 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:55:07.462 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:07.462 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.462 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.462 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:07.462 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:55:07.463 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:07.463 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.463 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.463 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:07.463 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715390+0000", 2024-05-08T08:55:07.463 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9900000000000001e-07 2024-05-08T08:55:07.463 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.463 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.463 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:07.463 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715392+0000", 2024-05-08T08:55:07.463 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8279999999999999e-06 2024-05-08T08:55:07.463 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.463 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.463 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:07.463 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715398+0000", 2024-05-08T08:55:07.464 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7619999999999998e-06 2024-05-08T08:55:07.464 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.464 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.464 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:07.464 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715403+0000", 2024-05-08T08:55:07.464 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7790000000000003e-06 2024-05-08T08:55:07.464 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.464 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.464 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:07.464 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768067+0000", 2024-05-08T08:55:07.464 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052663785999999997 2024-05-08T08:55:07.464 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.464 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.464 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:07.465 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768142+0000", 2024-05-08T08:55:07.465 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5018999999999994e-05 2024-05-08T08:55:07.465 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.465 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.465 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:07.465 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768238+0000", 2024-05-08T08:55:07.465 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6255999999999995e-05 2024-05-08T08:55:07.465 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:07.465 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:07.465 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:07.465 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.465 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.465 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6696 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1142784~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:07.465 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:55:07.466 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 155.73827502899999, 2024-05-08T08:55:07.466 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.053051057999999998, 2024-05-08T08:55:07.466 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:07.466 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:07.466 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:07.466 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:07.466 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:07.466 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:07.466 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6696 2024-05-08T08:55:07.466 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.466 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:07.466 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.466 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:07.467 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:55:07.467 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:07.467 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.467 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.467 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:07.467 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:55:07.467 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:07.467 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.467 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.467 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:07.467 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:55:07.467 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-05-08T08:55:07.467 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.467 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.468 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:07.468 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715411+0000", 2024-05-08T08:55:07.468 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9630000000000001e-06 2024-05-08T08:55:07.468 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.468 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.468 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:07.468 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715417+0000", 2024-05-08T08:55:07.468 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7250000000000002e-06 2024-05-08T08:55:07.468 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.468 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.468 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:07.468 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715421+0000", 2024-05-08T08:55:07.468 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4089999999999996e-06 2024-05-08T08:55:07.469 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.469 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.469 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:07.469 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768277+0000", 2024-05-08T08:55:07.469 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052855293999999997 2024-05-08T08:55:07.469 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.469 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.469 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:07.469 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768358+0000", 2024-05-08T08:55:07.469 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1720999999999999e-05 2024-05-08T08:55:07.469 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.469 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.469 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:07.470 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768460+0000", 2024-05-08T08:55:07.470 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010154600000000001 2024-05-08T08:55:07.470 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:07.470 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:07.470 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:07.470 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.470 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.470 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6698 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1159168~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:07.470 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:55:07.470 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 155.738256162, 2024-05-08T08:55:07.470 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.053245726, 2024-05-08T08:55:07.470 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:07.470 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:07.470 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:07.470 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:07.471 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:07.471 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:07.471 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6698 2024-05-08T08:55:07.471 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.471 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:07.471 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.471 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:07.471 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:55:07.471 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:07.471 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.471 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.471 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:07.471 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:55:07.471 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:07.471 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.472 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.472 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:07.472 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716936+0000", 2024-05-08T08:55:07.472 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001508665 2024-05-08T08:55:07.472 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.472 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.472 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:07.472 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716941+0000", 2024-05-08T08:55:07.472 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1910000000000002e-06 2024-05-08T08:55:07.472 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.472 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.472 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:07.472 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716950+0000", 2024-05-08T08:55:07.472 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1900000000000001e-06 2024-05-08T08:55:07.473 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.473 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.473 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:07.473 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716958+0000", 2024-05-08T08:55:07.473 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8019999999999992e-06 2024-05-08T08:55:07.473 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.473 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.473 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:07.473 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768499+0000", 2024-05-08T08:55:07.473 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051540926000000001 2024-05-08T08:55:07.473 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.473 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.473 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:07.473 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768574+0000", 2024-05-08T08:55:07.474 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4988999999999996e-05 2024-05-08T08:55:07.474 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.474 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.474 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:07.474 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768673+0000", 2024-05-08T08:55:07.474 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9963000000000005e-05 2024-05-08T08:55:07.474 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:07.474 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:07.474 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:07.474 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.474 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.474 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6706 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1224704~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:07.474 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:55:07.474 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 155.73665085499999, 2024-05-08T08:55:07.475 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052435598, 2024-05-08T08:55:07.475 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:07.475 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:07.475 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:07.475 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:07.475 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:07.475 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:07.475 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6706 2024-05-08T08:55:07.475 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.475 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:07.475 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.475 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:07.475 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:55:07.475 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:07.475 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.475 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.476 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:07.476 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:55:07.476 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:07.476 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.476 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.476 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:07.476 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:55:07.476 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:55:07.476 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.476 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.476 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:07.476 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717035+0000", 2024-05-08T08:55:07.476 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9290000000000001e-06 2024-05-08T08:55:07.476 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.476 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.477 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:07.477 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717041+0000", 2024-05-08T08:55:07.477 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6720000000000001e-06 2024-05-08T08:55:07.477 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.477 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.477 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:07.477 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717046+0000", 2024-05-08T08:55:07.477 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5650000000000003e-06 2024-05-08T08:55:07.477 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.477 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.477 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:07.477 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769311+0000", 2024-05-08T08:55:07.477 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052265645999999999 2024-05-08T08:55:07.477 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.477 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.478 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:07.478 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769379+0000", 2024-05-08T08:55:07.478 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7391000000000006e-05 2024-05-08T08:55:07.478 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.478 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.478 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:07.478 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769469+0000", 2024-05-08T08:55:07.478 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0025999999999995e-05 2024-05-08T08:55:07.478 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:07.478 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:07.478 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:07.478 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.478 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.478 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6708 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1241088~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:07.479 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:55:07.479 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 155.736632751, 2024-05-08T08:55:07.479 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052640552, 2024-05-08T08:55:07.479 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:07.479 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:07.479 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:07.479 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:07.479 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:07.479 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:07.479 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6708 2024-05-08T08:55:07.479 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.479 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:07.479 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.479 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:07.479 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:55:07.480 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:07.480 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.480 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.480 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:07.480 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:55:07.480 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:07.480 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.480 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.480 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:07.480 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717052+0000", 2024-05-08T08:55:07.480 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:55:07.480 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.480 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.481 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:07.481 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717053+0000", 2024-05-08T08:55:07.481 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8330000000000001e-06 2024-05-08T08:55:07.481 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.481 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.481 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:07.481 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717058+0000", 2024-05-08T08:55:07.481 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.933e-06 2024-05-08T08:55:07.481 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.481 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.481 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:07.481 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717063+0000", 2024-05-08T08:55:07.481 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2180000000000001e-06 2024-05-08T08:55:07.481 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.482 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.482 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:07.482 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769507+0000", 2024-05-08T08:55:07.482 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052444866999999999 2024-05-08T08:55:07.482 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.482 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.482 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:07.482 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769589+0000", 2024-05-08T08:55:07.482 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2039000000000005e-05 2024-05-08T08:55:07.482 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.482 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.482 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:07.482 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769692+0000", 2024-05-08T08:55:07.483 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102252 2024-05-08T08:55:07.483 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:07.483 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:07.483 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:07.483 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.483 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.483 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6710 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1257472~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:07.483 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:55:07.483 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 155.73661472699999, 2024-05-08T08:55:07.483 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.05283057, 2024-05-08T08:55:07.483 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:07.483 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:07.484 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:07.484 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:07.484 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:07.484 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:07.484 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6710 2024-05-08T08:55:07.484 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.484 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:07.484 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.484 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:07.484 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:55:07.484 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:07.484 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.485 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.485 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:07.485 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:55:07.485 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:07.485 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.485 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.485 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:07.485 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718951+0000", 2024-05-08T08:55:07.485 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0018817809999999999 2024-05-08T08:55:07.485 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.485 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.485 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:07.485 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718956+0000", 2024-05-08T08:55:07.486 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5399999999999997e-06 2024-05-08T08:55:07.486 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.486 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.486 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:07.486 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718964+0000", 2024-05-08T08:55:07.486 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6629999999999999e-06 2024-05-08T08:55:07.486 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.486 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.486 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:07.486 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718972+0000", 2024-05-08T08:55:07.486 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5229999999999998e-06 2024-05-08T08:55:07.486 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.486 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.486 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:07.487 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769729+0000", 2024-05-08T08:55:07.487 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050756959999999997 2024-05-08T08:55:07.487 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.487 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.487 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:07.487 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769803+0000", 2024-05-08T08:55:07.487 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4538999999999999e-05 2024-05-08T08:55:07.487 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.487 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.487 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:07.487 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769900+0000", 2024-05-08T08:55:07.487 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6564000000000006e-05 2024-05-08T08:55:07.487 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:07.488 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:07.488 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:07.488 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.488 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.488 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6726 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1388544~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:07.488 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:55:07.488 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 155.734539955, 2024-05-08T08:55:07.488 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052451506000000002, 2024-05-08T08:55:07.488 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:07.488 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:07.488 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:07.488 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:07.489 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:07.489 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:07.489 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6726 2024-05-08T08:55:07.489 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.489 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:07.489 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.489 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:07.489 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:55:07.489 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:07.489 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.489 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.489 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:07.489 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:55:07.489 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:07.490 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.490 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.490 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:07.490 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719200+0000", 2024-05-08T08:55:07.490 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5865999999999997e-05 2024-05-08T08:55:07.490 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.490 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.490 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:07.490 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719204+0000", 2024-05-08T08:55:07.490 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8630000000000001e-06 2024-05-08T08:55:07.490 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.490 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.490 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:07.491 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719215+0000", 2024-05-08T08:55:07.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1590000000000001e-05 2024-05-08T08:55:07.491 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:07.491 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719220+0000", 2024-05-08T08:55:07.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1619999999999999e-06 2024-05-08T08:55:07.491 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:07.491 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771425+0000", 2024-05-08T08:55:07.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052204249000000001 2024-05-08T08:55:07.491 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:07.491 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771498+0000", 2024-05-08T08:55:07.492 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3782999999999995e-05 2024-05-08T08:55:07.492 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.492 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.492 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:07.492 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771595+0000", 2024-05-08T08:55:07.492 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6992999999999998e-05 2024-05-08T08:55:07.492 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:07.492 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:07.492 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:07.492 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.492 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.492 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6728 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1404928~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:07.492 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:55:07.492 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 155.73445254999999, 2024-05-08T08:55:07.492 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052590747, 2024-05-08T08:55:07.492 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:07.493 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:07.493 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:07.493 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:07.493 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:07.493 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:07.493 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6728 2024-05-08T08:55:07.493 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.493 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:07.493 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.493 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:07.493 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:55:07.493 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:07.493 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.493 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.493 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:07.493 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:55:07.494 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:07.494 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.494 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.494 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:07.494 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719232+0000", 2024-05-08T08:55:07.494 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4000000000000001e-07 2024-05-08T08:55:07.494 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.494 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.494 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:07.494 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719234+0000", 2024-05-08T08:55:07.494 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4499999999999998e-06 2024-05-08T08:55:07.494 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.494 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.494 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:07.494 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719240+0000", 2024-05-08T08:55:07.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5720000000000002e-06 2024-05-08T08:55:07.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.495 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:07.495 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719245+0000", 2024-05-08T08:55:07.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8690000000000003e-06 2024-05-08T08:55:07.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.495 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:07.495 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771636+0000", 2024-05-08T08:55:07.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052390979999999997 2024-05-08T08:55:07.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.495 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:07.495 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771717+0000", 2024-05-08T08:55:07.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1291999999999993e-05 2024-05-08T08:55:07.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.496 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:07.496 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771822+0000", 2024-05-08T08:55:07.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104944 2024-05-08T08:55:07.496 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:07.496 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:07.496 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:07.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.496 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6730 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1421312~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:07.496 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:55:07.496 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 155.73443230800001, 2024-05-08T08:55:07.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052770477000000003, 2024-05-08T08:55:07.496 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:07.496 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:07.497 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:07.497 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:07.497 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:07.497 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:07.497 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6730 2024-05-08T08:55:07.497 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.497 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:07.497 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.497 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:07.497 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:55:07.497 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:07.497 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.497 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.497 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:07.497 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:55:07.497 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:07.498 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.498 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.498 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:07.498 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:55:07.498 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9000000000000002e-07 2024-05-08T08:55:07.498 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.498 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.498 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:07.498 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719254+0000", 2024-05-08T08:55:07.498 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4099999999999998e-06 2024-05-08T08:55:07.498 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.498 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.498 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:07.498 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719263+0000", 2024-05-08T08:55:07.498 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4109999999999996e-06 2024-05-08T08:55:07.499 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.499 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.499 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:07.499 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719268+0000", 2024-05-08T08:55:07.499 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2669999999999997e-06 2024-05-08T08:55:07.499 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.499 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.499 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:07.499 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771846+0000", 2024-05-08T08:55:07.499 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052577677000000003 2024-05-08T08:55:07.499 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.499 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.499 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:07.499 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771920+0000", 2024-05-08T08:55:07.500 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4460000000000002e-05 2024-05-08T08:55:07.500 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.500 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.500 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:07.500 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772022+0000", 2024-05-08T08:55:07.500 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010186199999999999 2024-05-08T08:55:07.500 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:07.500 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:07.500 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:07.500 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.500 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.500 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6732 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1437696~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:07.501 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:55:07.501 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 155.73440750099999, 2024-05-08T08:55:07.501 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052905575000000003, 2024-05-08T08:55:07.501 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:07.501 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:07.501 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:07.501 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:07.501 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:07.501 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:07.501 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6732 2024-05-08T08:55:07.501 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.501 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:07.501 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.501 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:07.501 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:55:07.501 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:07.501 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.502 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.502 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:07.502 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:55:07.502 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:07.502 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.502 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.502 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:07.502 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721392+0000", 2024-05-08T08:55:07.502 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002115287 2024-05-08T08:55:07.502 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.502 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.502 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:07.502 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721396+0000", 2024-05-08T08:55:07.502 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1679999999999997e-06 2024-05-08T08:55:07.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.503 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:07.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721404+0000", 2024-05-08T08:55:07.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4409999999999993e-06 2024-05-08T08:55:07.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.503 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:07.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721410+0000", 2024-05-08T08:55:07.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9150000000000001e-06 2024-05-08T08:55:07.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.503 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:07.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772061+0000", 2024-05-08T08:55:07.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050651068 2024-05-08T08:55:07.504 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.504 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.504 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:07.504 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772127+0000", 2024-05-08T08:55:07.504 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5424000000000002e-05 2024-05-08T08:55:07.504 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.504 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.504 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:07.504 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772182+0000", 2024-05-08T08:55:07.504 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5272000000000002e-05 2024-05-08T08:55:07.504 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:07.504 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:07.504 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:07.504 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.505 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.505 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6749 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1585152~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:07.505 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:55:07.505 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 155.73210673099999, 2024-05-08T08:55:07.505 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052350264000000001, 2024-05-08T08:55:07.505 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:07.505 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:07.505 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:07.505 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:07.505 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:07.505 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:07.505 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6749 2024-05-08T08:55:07.505 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.505 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:07.505 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.506 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:07.506 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:55:07.506 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:07.506 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.506 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.506 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:07.506 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:55:07.506 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:07.506 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.506 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.506 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:07.506 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721578+0000", 2024-05-08T08:55:07.506 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0399999999999996e-07 2024-05-08T08:55:07.506 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.506 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.507 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:07.507 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721580+0000", 2024-05-08T08:55:07.507 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5450000000000002e-06 2024-05-08T08:55:07.507 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.507 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.507 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:07.507 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721587+0000", 2024-05-08T08:55:07.507 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0190000000000001e-06 2024-05-08T08:55:07.507 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.507 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.507 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:07.507 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721593+0000", 2024-05-08T08:55:07.507 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5659999999999997e-06 2024-05-08T08:55:07.508 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.508 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.508 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:07.508 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773759+0000", 2024-05-08T08:55:07.509 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052166553999999997 2024-05-08T08:55:07.509 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.509 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.509 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:07.509 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773833+0000", 2024-05-08T08:55:07.509 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3913999999999997e-05 2024-05-08T08:55:07.509 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.510 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.510 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:07.510 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773927+0000", 2024-05-08T08:55:07.510 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4061999999999995e-05 2024-05-08T08:55:07.510 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:07.510 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:07.510 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:07.510 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.510 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.510 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6751 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1601536~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:07.510 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:55:07.510 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 155.73208345200001, 2024-05-08T08:55:07.510 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052461302000000001, 2024-05-08T08:55:07.511 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:07.511 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:07.511 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:07.511 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:07.511 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:07.511 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:07.511 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6751 2024-05-08T08:55:07.511 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.511 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:07.511 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.511 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:07.511 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:55:07.511 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:07.511 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.512 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.512 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:07.512 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:55:07.512 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:07.512 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.512 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.512 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:07.512 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722939+0000", 2024-05-08T08:55:07.512 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001338601 2024-05-08T08:55:07.512 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.512 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.512 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:07.512 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722944+0000", 2024-05-08T08:55:07.512 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4469999999999999e-06 2024-05-08T08:55:07.512 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.513 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.513 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:07.513 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722955+0000", 2024-05-08T08:55:07.513 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1892999999999999e-05 2024-05-08T08:55:07.513 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.513 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.513 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:07.513 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722963+0000", 2024-05-08T08:55:07.513 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1080000000000004e-06 2024-05-08T08:55:07.513 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.513 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.513 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:07.513 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773964+0000", 2024-05-08T08:55:07.513 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051001022 2024-05-08T08:55:07.513 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.514 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.514 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:07.514 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.774006+0000", 2024-05-08T08:55:07.514 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2729999999999999e-05 2024-05-08T08:55:07.514 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.514 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.514 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:07.514 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.774062+0000", 2024-05-08T08:55:07.514 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5501000000000003e-05 2024-05-08T08:55:07.514 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:07.514 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:07.514 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:07.515 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.515 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.515 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6760 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1683456~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:07.515 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:55:07.515 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 155.73057407600001, 2024-05-08T08:55:07.515 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052571312000000002, 2024-05-08T08:55:07.515 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:07.515 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:07.515 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:07.515 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:07.515 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:07.515 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:07.515 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6760 2024-05-08T08:55:07.515 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.515 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:07.515 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.516 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:07.516 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:55:07.516 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:07.516 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.516 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.516 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:07.516 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:55:07.516 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:07.516 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.516 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.516 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:07.516 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:55:07.516 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2100000000000002e-07 2024-05-08T08:55:07.516 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.516 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.517 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:07.517 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723113+0000", 2024-05-08T08:55:07.517 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3149999999999999e-06 2024-05-08T08:55:07.517 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.517 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.517 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:07.517 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723118+0000", 2024-05-08T08:55:07.517 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8860000000000001e-06 2024-05-08T08:55:07.517 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.517 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.517 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:07.517 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723123+0000", 2024-05-08T08:55:07.517 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6770000000000003e-06 2024-05-08T08:55:07.517 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.517 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.517 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:07.518 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775414+0000", 2024-05-08T08:55:07.518 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052290692999999999 2024-05-08T08:55:07.518 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.518 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.518 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:07.518 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775459+0000", 2024-05-08T08:55:07.518 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5639999999999997e-05 2024-05-08T08:55:07.518 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.518 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.518 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:07.518 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775681+0000", 2024-05-08T08:55:07.518 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00022168000000000001 2024-05-08T08:55:07.518 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:07.519 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:07.519 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:07.519 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.519 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.519 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6761 3.16 3:6bd6d6da:::rbd_data.1140782590a5.0000000000000015:6 [read 1095680~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:07.519 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:55:07.519 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 155.730554856, 2024-05-08T08:55:07.519 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052751275, 2024-05-08T08:55:07.519 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:07.519 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:07.519 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:07.519 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:07.519 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:07.519 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:07.519 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6761 2024-05-08T08:55:07.520 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.520 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:07.520 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.520 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:07.520 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:55:07.520 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:07.520 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.520 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.520 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:07.520 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:55:07.520 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:07.520 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.520 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.520 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:07.521 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724939+0000", 2024-05-08T08:55:07.521 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001809817 2024-05-08T08:55:07.521 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.521 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.521 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:07.521 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724943+0000", 2024-05-08T08:55:07.521 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4920000000000004e-06 2024-05-08T08:55:07.521 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.521 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.521 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:07.521 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724956+0000", 2024-05-08T08:55:07.521 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2427e-05 2024-05-08T08:55:07.521 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.521 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.522 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:07.522 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724963+0000", 2024-05-08T08:55:07.522 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2559999999999996e-06 2024-05-08T08:55:07.522 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.522 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.522 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:07.522 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775711+0000", 2024-05-08T08:55:07.522 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050747621 2024-05-08T08:55:07.522 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.522 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.522 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:07.522 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775785+0000", 2024-05-08T08:55:07.522 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4634000000000004e-05 2024-05-08T08:55:07.522 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.523 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.523 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:07.523 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775880+0000", 2024-05-08T08:55:07.523 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5027999999999999e-05 2024-05-08T08:55:07.523 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:07.523 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:07.523 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:07.523 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.523 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.523 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6770 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1765376~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:07.523 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:55:07.523 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 155.72852376099999, 2024-05-08T08:55:07.523 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052339296, 2024-05-08T08:55:07.523 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:07.523 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:07.524 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:07.524 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:07.524 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:07.524 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:07.524 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6770 2024-05-08T08:55:07.524 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.524 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:07.524 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.524 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:07.524 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:55:07.524 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:07.524 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.524 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.524 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:07.524 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:55:07.524 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:07.525 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.525 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.525 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:07.525 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725161+0000", 2024-05-08T08:55:07.525 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-05-08T08:55:07.525 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.525 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.525 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:07.525 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725163+0000", 2024-05-08T08:55:07.525 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4370000000000001e-06 2024-05-08T08:55:07.525 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.525 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.525 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:07.525 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725169+0000", 2024-05-08T08:55:07.525 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8409999999999996e-06 2024-05-08T08:55:07.525 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.526 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.526 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:07.526 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725174+0000", 2024-05-08T08:55:07.526 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7249999999999997e-06 2024-05-08T08:55:07.526 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.526 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.526 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:07.526 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777343+0000", 2024-05-08T08:55:07.526 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052169633999999999 2024-05-08T08:55:07.526 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.526 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.526 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:07.526 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777409+0000", 2024-05-08T08:55:07.526 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6033999999999998e-05 2024-05-08T08:55:07.527 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.527 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.527 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:07.527 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777499+0000", 2024-05-08T08:55:07.527 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0224999999999998e-05 2024-05-08T08:55:07.527 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:07.527 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:07.527 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:07.527 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.527 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.527 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6771 3.16 3:6bd6d6da:::rbd_data.1140782590a5.0000000000000015:6 [read 1177600~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:07.527 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:55:07.527 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 155.72850394400001, 2024-05-08T08:55:07.527 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052438604999999999, 2024-05-08T08:55:07.527 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:07.527 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:07.528 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:07.528 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:07.528 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:07.528 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:07.528 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6771 2024-05-08T08:55:07.528 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.528 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:07.528 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.528 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:07.528 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:55:07.528 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:07.528 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.528 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.528 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:07.528 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:55:07.528 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:07.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.529 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.529 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:07.529 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:55:07.529 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:55:07.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.529 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.529 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:07.529 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725188+0000", 2024-05-08T08:55:07.529 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1579999999999999e-06 2024-05-08T08:55:07.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.529 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.529 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:07.529 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725195+0000", 2024-05-08T08:55:07.530 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8909999999999995e-06 2024-05-08T08:55:07.530 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.530 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.530 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:07.530 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725200+0000", 2024-05-08T08:55:07.530 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9810000000000003e-06 2024-05-08T08:55:07.530 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.530 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.530 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:07.530 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777527+0000", 2024-05-08T08:55:07.530 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052326582000000003 2024-05-08T08:55:07.530 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.530 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.530 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:07.530 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777566+0000", 2024-05-08T08:55:07.531 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9447999999999999e-05 2024-05-08T08:55:07.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.531 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.531 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:07.531 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777619+0000", 2024-05-08T08:55:07.531 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2135000000000001e-05 2024-05-08T08:55:07.531 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:07.531 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:07.531 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:07.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.531 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.531 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6772 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1781760~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:07.531 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:55:07.531 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 155.728475887, 2024-05-08T08:55:07.531 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052601799999999997, 2024-05-08T08:55:07.532 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:07.532 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:07.532 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:07.532 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:07.532 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:07.532 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:07.532 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6772 2024-05-08T08:55:07.532 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.532 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:07.532 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.532 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:07.532 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:55:07.532 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:07.532 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.532 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.532 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:07.532 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:55:07.533 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:07.533 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.533 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.533 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:07.533 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726939+0000", 2024-05-08T08:55:07.533 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001730493 2024-05-08T08:55:07.533 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.533 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.533 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:07.533 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726943+0000", 2024-05-08T08:55:07.533 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7330000000000003e-06 2024-05-08T08:55:07.533 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.533 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.533 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:07.533 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726954+0000", 2024-05-08T08:55:07.533 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1107000000000001e-05 2024-05-08T08:55:07.533 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.534 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.534 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:07.534 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726962+0000", 2024-05-08T08:55:07.534 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.503e-06 2024-05-08T08:55:07.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.534 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.534 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:07.534 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777653+0000", 2024-05-08T08:55:07.534 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050691542999999999 2024-05-08T08:55:07.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.534 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.534 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:07.534 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777719+0000", 2024-05-08T08:55:07.534 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5228000000000006e-05 2024-05-08T08:55:07.535 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.535 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.535 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:07.535 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777810+0000", 2024-05-08T08:55:07.535 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1193000000000006e-05 2024-05-08T08:55:07.535 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:07.535 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:07.535 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:07.535 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.535 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.535 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6782 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1880064~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:07.535 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:55:07.535 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 155.72650737999999, 2024-05-08T08:55:07.535 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052326492000000002, 2024-05-08T08:55:07.535 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:07.535 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:07.536 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:07.536 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:07.536 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:07.536 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:07.536 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6782 2024-05-08T08:55:07.536 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.536 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:07.536 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.536 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:07.536 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:55:07.536 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:07.536 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.536 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.536 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:07.536 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:55:07.536 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:07.537 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.537 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.537 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:07.537 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728939+0000", 2024-05-08T08:55:07.537 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001762719 2024-05-08T08:55:07.537 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.537 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.537 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:07.537 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728944+0000", 2024-05-08T08:55:07.537 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.746e-06 2024-05-08T08:55:07.537 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.537 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.537 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:07.537 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728955+0000", 2024-05-08T08:55:07.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1043000000000001e-05 2024-05-08T08:55:07.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.538 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:07.538 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728963+0000", 2024-05-08T08:55:07.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5809999999999999e-06 2024-05-08T08:55:07.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.538 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:07.538 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779320+0000", 2024-05-08T08:55:07.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050357290999999998 2024-05-08T08:55:07.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.538 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:07.538 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779401+0000", 2024-05-08T08:55:07.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0574000000000005e-05 2024-05-08T08:55:07.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:07.539 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:07.539 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:07.539 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779503+0000", 2024-05-08T08:55:07.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102538 2024-05-08T08:55:07.539 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:07.539 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:07.539 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:07.539 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:07.539 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:07.539 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:55:07.541 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:55:07.542 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:55:07.544+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:55:07.642 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:55:07.644+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:55:07.663 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:55:07.664 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:55:07.664 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:55:07.664 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:55:07.673 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:55:07.742 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:55:07.745+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:55:07.783 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:55:07.783 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:55:07.783 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:55:07.783 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:55:07.783 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:55:07.792 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:55:07.843 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:55:07.845+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:55:07.904 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:55:07.904 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:55:07.904 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:55:07.904 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:55:07.904 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.904 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4621.0:9089 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 23339520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:55:07.904 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:55:07.904 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 155.41252388999999, 2024-05-08T08:55:07.904 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.13814907200000001, 2024-05-08T08:55:07.905 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:07.905 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:07.905 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:55:07.905 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:07.905 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4621", 2024-05-08T08:55:07.905 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:07.905 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9089 2024-05-08T08:55:07.905 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.905 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:07.905 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.905 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:07.905 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:55:07.905 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:07.905 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.905 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.906 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:07.906 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:55:07.906 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:07.906 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.906 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.906 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:07.906 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493005+0000", 2024-05-08T08:55:07.906 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1700000000000001e-06 2024-05-08T08:55:07.906 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.906 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.906 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:07.906 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.513963+0000", 2024-05-08T08:55:07.906 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.020957581999999999 2024-05-08T08:55:07.906 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.907 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.907 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:07.907 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.513995+0000", 2024-05-08T08:55:07.907 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1993999999999999e-05 2024-05-08T08:55:07.907 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.907 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.907 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:07.907 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514007+0000", 2024-05-08T08:55:07.907 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1581e-05 2024-05-08T08:55:07.907 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.907 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.907 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:07.907 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514122+0000", 2024-05-08T08:55:07.907 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000115282 2024-05-08T08:55:07.907 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.907 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.908 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op must wait for map", 2024-05-08T08:55:07.908 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514128+0000", 2024-05-08T08:55:07.908 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9950000000000002e-06 2024-05-08T08:55:07.908 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.908 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.908 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:07.908 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.515800+0000", 2024-05-08T08:55:07.908 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001671886 2024-05-08T08:55:07.908 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.908 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.908 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:07.908 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.515893+0000", 2024-05-08T08:55:07.908 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3490000000000001e-05 2024-05-08T08:55:07.908 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.908 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.908 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 3", 2024-05-08T08:55:07.909 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.517647+0000", 2024-05-08T08:55:07.909 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0017535630000000001 2024-05-08T08:55:07.909 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.909 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.909 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:55:07.909 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.627578+0000", 2024-05-08T08:55:07.909 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.109931557 2024-05-08T08:55:07.909 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.909 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.909 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:55:07.909 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631055+0000", 2024-05-08T08:55:07.909 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0034761420000000002 2024-05-08T08:55:07.909 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.909 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.909 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:55:07.909 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631096+0000", 2024-05-08T08:55:07.909 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1059000000000001e-05 2024-05-08T08:55:07.910 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.910 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.910 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:07.910 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631151+0000", 2024-05-08T08:55:07.910 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5770999999999998e-05 2024-05-08T08:55:07.910 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:07.910 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:07.910 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:07.910 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.910 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.910 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1071 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11026432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:55:07.910 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:55:07.910 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 151.42007439899999, 2024-05-08T08:55:07.910 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070580037999999998, 2024-05-08T08:55:07.910 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:07.911 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:07.911 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:07.911 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:07.911 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:55:07.911 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:55:07.911 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1071 2024-05-08T08:55:07.911 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.911 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:07.911 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.911 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:07.911 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:55:07.911 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:07.911 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.911 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.912 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:07.912 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:55:07.912 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:07.912 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.912 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.912 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:07.912 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:55:07.912 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2800000000000002e-07 2024-05-08T08:55:07.912 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.912 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.912 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:07.912 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485454+0000", 2024-05-08T08:55:07.912 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2290000000000002e-06 2024-05-08T08:55:07.912 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.913 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.913 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:07.913 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485461+0000", 2024-05-08T08:55:07.913 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9920000000000001e-06 2024-05-08T08:55:07.913 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.913 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.913 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:07.913 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485467+0000", 2024-05-08T08:55:07.913 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3979999999999998e-06 2024-05-08T08:55:07.913 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.913 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.913 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:07.913 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555880+0000", 2024-05-08T08:55:07.913 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070413165 2024-05-08T08:55:07.913 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.913 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.913 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:07.914 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555926+0000", 2024-05-08T08:55:07.914 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6217000000000002e-05 2024-05-08T08:55:07.914 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.914 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.914 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:07.914 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556032+0000", 2024-05-08T08:55:07.914 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000105609 2024-05-08T08:55:07.914 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:07.914 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:07.914 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:07.914 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.914 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.914 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1073 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11067392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:55:07.914 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:55:07.915 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 151.42005317799999, 2024-05-08T08:55:07.915 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070821404000000004, 2024-05-08T08:55:07.915 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:07.915 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:07.915 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:07.915 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:07.915 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:55:07.915 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:55:07.915 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1073 2024-05-08T08:55:07.915 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.915 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:07.915 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.915 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:07.915 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:55:07.915 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:07.916 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.916 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.916 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:07.916 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:55:07.916 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:07.916 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.916 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.916 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:07.916 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:55:07.916 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.08e-07 2024-05-08T08:55:07.916 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.916 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.916 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:07.916 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485476+0000", 2024-05-08T08:55:07.916 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3929999999999998e-06 2024-05-08T08:55:07.916 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.917 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.917 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:07.917 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485485+0000", 2024-05-08T08:55:07.917 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1880000000000008e-06 2024-05-08T08:55:07.917 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.917 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.917 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:07.917 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485494+0000", 2024-05-08T08:55:07.917 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3109999999999995e-06 2024-05-08T08:55:07.917 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.917 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.917 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:07.917 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556082+0000", 2024-05-08T08:55:07.917 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070587426999999994 2024-05-08T08:55:07.917 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.918 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.918 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:07.918 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556177+0000", 2024-05-08T08:55:07.918 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5051999999999997e-05 2024-05-08T08:55:07.918 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.918 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.918 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:07.918 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556294+0000", 2024-05-08T08:55:07.918 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000117625 2024-05-08T08:55:07.918 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:07.918 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:07.918 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:07.918 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.918 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.918 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1075 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11108352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:55:07.919 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:55:07.919 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 151.420022601, 2024-05-08T08:55:07.919 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071045026999999997, 2024-05-08T08:55:07.919 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:07.919 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:07.919 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:07.919 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:07.919 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:55:07.919 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:55:07.919 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1075 2024-05-08T08:55:07.919 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.919 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:07.919 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.919 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:07.919 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:55:07.919 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:07.919 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.920 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.920 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:07.920 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:55:07.920 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:07.920 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.920 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.920 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:07.920 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:55:07.920 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.32e-07 2024-05-08T08:55:07.920 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.920 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.920 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:07.920 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485518+0000", 2024-05-08T08:55:07.920 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3769e-05 2024-05-08T08:55:07.920 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.920 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.920 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:07.921 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485527+0000", 2024-05-08T08:55:07.921 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6779999999999995e-06 2024-05-08T08:55:07.921 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.921 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.921 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:07.921 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485532+0000", 2024-05-08T08:55:07.921 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9830000000000004e-06 2024-05-08T08:55:07.921 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.921 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.921 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:07.921 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556340+0000", 2024-05-08T08:55:07.921 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070807743000000006 2024-05-08T08:55:07.921 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.921 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.921 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:07.921 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556435+0000", 2024-05-08T08:55:07.921 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4732e-05 2024-05-08T08:55:07.922 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.922 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.922 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:07.922 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556549+0000", 2024-05-08T08:55:07.922 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011369 2024-05-08T08:55:07.922 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:07.922 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:07.922 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:07.922 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.922 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.922 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1078 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11149312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:55:07.922 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:55:07.922 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 151.41998240199999, 2024-05-08T08:55:07.923 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071167276000000002, 2024-05-08T08:55:07.923 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:07.923 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:07.923 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:07.923 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:07.923 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:55:07.923 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:55:07.923 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1078 2024-05-08T08:55:07.923 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.923 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:07.923 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.923 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:07.923 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:55:07.923 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:07.923 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.923 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.924 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:07.924 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:55:07.924 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:07.924 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.924 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.924 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:07.924 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:55:07.924 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5600000000000001e-07 2024-05-08T08:55:07.924 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.924 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.924 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:07.924 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485547+0000", 2024-05-08T08:55:07.924 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3590000000000002e-06 2024-05-08T08:55:07.924 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.924 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.924 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:07.924 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485553+0000", 2024-05-08T08:55:07.925 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2770000000000002e-06 2024-05-08T08:55:07.925 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.925 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.925 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:07.925 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485558+0000", 2024-05-08T08:55:07.925 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.95e-06 2024-05-08T08:55:07.925 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.925 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.925 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:07.925 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556582+0000", 2024-05-08T08:55:07.925 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071024444000000006 2024-05-08T08:55:07.925 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.925 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.925 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:07.925 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556637+0000", 2024-05-08T08:55:07.926 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4672000000000001e-05 2024-05-08T08:55:07.926 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.926 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.926 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:07.926 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556711+0000", 2024-05-08T08:55:07.926 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4117999999999997e-05 2024-05-08T08:55:07.926 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:07.926 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:07.926 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:07.926 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.926 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.926 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1080 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11190272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:55:07.926 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:55:07.926 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 151.419961669, 2024-05-08T08:55:07.927 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071350721000000006, 2024-05-08T08:55:07.927 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:07.927 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:07.927 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:07.927 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:07.927 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:55:07.927 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:55:07.927 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1080 2024-05-08T08:55:07.927 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.927 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:07.927 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.927 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:07.927 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:55:07.927 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:07.927 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.927 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.928 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:07.928 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:55:07.928 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:07.928 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.928 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.928 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:07.928 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:55:07.928 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.03e-07 2024-05-08T08:55:07.928 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.928 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.928 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:07.928 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485567+0000", 2024-05-08T08:55:07.928 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1090000000000001e-06 2024-05-08T08:55:07.928 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.928 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.928 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:07.929 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485573+0000", 2024-05-08T08:55:07.929 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3049999999999998e-06 2024-05-08T08:55:07.929 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.929 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.929 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:07.929 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485578+0000", 2024-05-08T08:55:07.929 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0130000000000001e-06 2024-05-08T08:55:07.929 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.929 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.929 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:07.929 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556743+0000", 2024-05-08T08:55:07.929 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071164449000000005 2024-05-08T08:55:07.929 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.929 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.929 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:07.929 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556797+0000", 2024-05-08T08:55:07.930 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4589999999999997e-05 2024-05-08T08:55:07.930 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.930 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.930 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:07.930 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556915+0000", 2024-05-08T08:55:07.930 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000117852 2024-05-08T08:55:07.930 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:07.930 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:07.930 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:07.930 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.930 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.930 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1082 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11231232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:55:07.930 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:55:07.930 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 151.41994142600001, 2024-05-08T08:55:07.931 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071591396000000002, 2024-05-08T08:55:07.931 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:07.931 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:07.931 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:07.931 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:07.931 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:55:07.931 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:55:07.931 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1082 2024-05-08T08:55:07.931 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.931 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:07.931 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.931 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:07.931 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:55:07.931 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:07.931 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.932 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.932 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:07.932 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:55:07.932 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:07.932 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.932 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.932 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:07.932 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:55:07.932 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1800000000000001e-07 2024-05-08T08:55:07.932 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.932 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.932 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:07.932 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485587+0000", 2024-05-08T08:55:07.932 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2589999999999999e-06 2024-05-08T08:55:07.933 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.933 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.933 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:07.933 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485594+0000", 2024-05-08T08:55:07.933 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0789999999999997e-06 2024-05-08T08:55:07.933 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.933 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.933 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:07.933 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485598+0000", 2024-05-08T08:55:07.933 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7480000000000001e-06 2024-05-08T08:55:07.933 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.933 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.933 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:07.933 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556971+0000", 2024-05-08T08:55:07.933 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071372774999999999 2024-05-08T08:55:07.934 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.934 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.934 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:07.934 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557069+0000", 2024-05-08T08:55:07.934 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.7732999999999994e-05 2024-05-08T08:55:07.934 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.934 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.934 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:07.934 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557176+0000", 2024-05-08T08:55:07.934 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000107384 2024-05-08T08:55:07.934 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:07.934 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:07.934 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:07.934 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.934 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.935 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1084 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11272192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:55:07.935 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:55:07.935 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 151.41990278700001, 2024-05-08T08:55:07.935 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071808977999999996, 2024-05-08T08:55:07.935 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:07.935 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:07.935 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:07.935 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:07.935 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:55:07.935 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:55:07.935 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1084 2024-05-08T08:55:07.935 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.935 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:07.935 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.935 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:07.935 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:55:07.936 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:07.936 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.936 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.936 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:07.936 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:55:07.936 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:07.936 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.936 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.936 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:07.936 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485624+0000", 2024-05-08T08:55:07.936 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:55:07.936 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.936 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.936 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:07.936 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485626+0000", 2024-05-08T08:55:07.936 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9460000000000001e-06 2024-05-08T08:55:07.937 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.937 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.937 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:07.937 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485633+0000", 2024-05-08T08:55:07.937 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1149999999999996e-06 2024-05-08T08:55:07.937 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.937 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.937 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:07.937 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485638+0000", 2024-05-08T08:55:07.937 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5670000000000002e-06 2024-05-08T08:55:07.937 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.937 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.937 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:07.937 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557226+0000", 2024-05-08T08:55:07.938 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071587267999999996 2024-05-08T08:55:07.938 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.938 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.938 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:07.938 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557321+0000", 2024-05-08T08:55:07.938 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5785000000000005e-05 2024-05-08T08:55:07.938 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.938 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.938 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:07.938 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557432+0000", 2024-05-08T08:55:07.938 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011092800000000001 2024-05-08T08:55:07.938 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:07.938 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:07.938 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:07.939 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.939 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.939 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1086 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11313152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:55:07.939 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:55:07.939 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 151.41987946, 2024-05-08T08:55:07.939 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071933132999999996, 2024-05-08T08:55:07.939 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:07.939 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:07.939 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:07.939 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:07.939 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:55:07.939 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:55:07.939 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1086 2024-05-08T08:55:07.939 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.939 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:07.940 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.940 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:07.940 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:55:07.940 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:07.940 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.940 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.940 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:07.940 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:55:07.940 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:07.940 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.940 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.940 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:07.940 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486328+0000", 2024-05-08T08:55:07.940 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00068107900000000002 2024-05-08T08:55:07.940 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.940 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.941 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:07.941 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486331+0000", 2024-05-08T08:55:07.941 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.585e-06 2024-05-08T08:55:07.941 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.941 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.941 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:07.941 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486343+0000", 2024-05-08T08:55:07.941 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1856e-05 2024-05-08T08:55:07.941 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.941 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.941 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:07.941 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486350+0000", 2024-05-08T08:55:07.941 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8600000000000004e-06 2024-05-08T08:55:07.941 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.941 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.941 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:07.942 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557470+0000", 2024-05-08T08:55:07.942 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071120326999999997 2024-05-08T08:55:07.942 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.942 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.942 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:07.942 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557514+0000", 2024-05-08T08:55:07.942 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3909000000000003e-05 2024-05-08T08:55:07.942 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.942 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.942 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:07.942 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557580+0000", 2024-05-08T08:55:07.942 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5517000000000003e-05 2024-05-08T08:55:07.942 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:07.942 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:07.943 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:07.943 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.943 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.943 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1089 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11354112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:55:07.943 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:55:07.943 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 151.419166588, 2024-05-08T08:55:07.943 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071457381, 2024-05-08T08:55:07.943 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:07.943 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:07.943 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:07.943 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:07.943 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:55:07.943 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:55:07.943 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1089 2024-05-08T08:55:07.944 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.944 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:07.944 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.944 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:07.944 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:55:07.944 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:07.944 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.944 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.944 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:07.944 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:55:07.944 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:07.944 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.944 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.944 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:07.944 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486851+0000", 2024-05-08T08:55:07.945 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00049102 2024-05-08T08:55:07.945 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.945 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.945 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:07.945 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486854+0000", 2024-05-08T08:55:07.945 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6859999999999999e-06 2024-05-08T08:55:07.945 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.945 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.945 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:07.945 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486864+0000", 2024-05-08T08:55:07.945 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6080000000000002e-06 2024-05-08T08:55:07.945 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.945 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.945 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:07.945 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486870+0000", 2024-05-08T08:55:07.946 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.477e-06 2024-05-08T08:55:07.946 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.946 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.946 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:07.946 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557612+0000", 2024-05-08T08:55:07.946 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070741557999999996 2024-05-08T08:55:07.946 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.946 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.946 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:07.946 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557695+0000", 2024-05-08T08:55:07.946 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3300000000000005e-05 2024-05-08T08:55:07.946 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.946 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.946 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:07.947 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557817+0000", 2024-05-08T08:55:07.947 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012173200000000001 2024-05-08T08:55:07.947 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:07.947 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:07.947 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:07.947 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.947 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.947 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1091 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11395072~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:55:07.947 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:55:07.947 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 151.418645793, 2024-05-08T08:55:07.947 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071184737999999997, 2024-05-08T08:55:07.947 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:07.947 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:07.947 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:07.947 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:07.948 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:55:07.948 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:55:07.948 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1091 2024-05-08T08:55:07.948 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.948 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:07.948 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.948 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:07.948 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:55:07.948 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:07.948 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.948 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.948 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:07.948 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:55:07.948 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:07.949 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.949 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.949 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:07.949 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486881+0000", 2024-05-08T08:55:07.949 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0699999999999997e-07 2024-05-08T08:55:07.949 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.949 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.949 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:07.949 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486883+0000", 2024-05-08T08:55:07.949 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.402e-06 2024-05-08T08:55:07.949 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.949 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.949 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:07.949 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486890+0000", 2024-05-08T08:55:07.949 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0480000000000001e-06 2024-05-08T08:55:07.950 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.950 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.950 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:07.950 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486896+0000", 2024-05-08T08:55:07.950 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7640000000000002e-06 2024-05-08T08:55:07.950 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.950 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.950 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:07.950 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557860+0000", 2024-05-08T08:55:07.950 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070963791999999998 2024-05-08T08:55:07.950 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.950 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.950 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:07.950 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557952+0000", 2024-05-08T08:55:07.951 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2082000000000004e-05 2024-05-08T08:55:07.951 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.951 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.951 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:07.951 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558065+0000", 2024-05-08T08:55:07.951 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000113143 2024-05-08T08:55:07.951 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:07.951 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:07.951 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:07.951 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.951 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.951 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1094 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11436032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:55:07.951 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:55:07.951 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 151.41862262500001, 2024-05-08T08:55:07.951 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071400507000000002, 2024-05-08T08:55:07.951 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:07.952 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:07.952 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:07.952 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:07.952 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:55:07.952 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:55:07.952 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1094 2024-05-08T08:55:07.952 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.952 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:07.952 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.952 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:07.952 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:55:07.952 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:07.952 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.952 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.952 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:07.952 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:55:07.953 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:07.953 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.953 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.953 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:07.953 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487940+0000", 2024-05-08T08:55:07.953 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001036868 2024-05-08T08:55:07.953 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.953 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.953 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:07.953 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487944+0000", 2024-05-08T08:55:07.953 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7069999999999998e-06 2024-05-08T08:55:07.953 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.953 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.953 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:07.954 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487951+0000", 2024-05-08T08:55:07.954 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2649999999999999e-06 2024-05-08T08:55:07.954 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.954 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.954 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:07.954 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487961+0000", 2024-05-08T08:55:07.954 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4110000000000002e-06 2024-05-08T08:55:07.954 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.954 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.954 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:07.954 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558117+0000", 2024-05-08T08:55:07.954 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070156662999999994 2024-05-08T08:55:07.954 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.954 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.955 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:07.955 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558214+0000", 2024-05-08T08:55:07.955 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6444e-05 2024-05-08T08:55:07.955 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.955 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.955 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:07.955 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558304+0000", 2024-05-08T08:55:07.955 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0148999999999994e-05 2024-05-08T08:55:07.955 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:07.955 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:07.955 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:07.955 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.955 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.955 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1096 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11476992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:55:07.955 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:55:07.955 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 151.41755609399999, 2024-05-08T08:55:07.956 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070480074000000004, 2024-05-08T08:55:07.956 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:07.956 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:07.956 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:07.956 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:07.956 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:55:07.956 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:55:07.956 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1096 2024-05-08T08:55:07.956 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.956 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:07.956 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.956 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:07.956 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:55:07.956 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:07.956 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.956 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.957 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:07.957 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:55:07.957 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:07.957 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.957 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.957 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:07.957 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:55:07.957 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3700000000000001e-07 2024-05-08T08:55:07.957 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.957 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.957 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:07.957 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487973+0000", 2024-05-08T08:55:07.957 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0870000000000001e-06 2024-05-08T08:55:07.957 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.957 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.957 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:07.958 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487977+0000", 2024-05-08T08:55:07.958 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0600000000000001e-06 2024-05-08T08:55:07.958 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.958 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.958 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:07.958 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487980+0000", 2024-05-08T08:55:07.958 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1089999999999998e-06 2024-05-08T08:55:07.958 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.958 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.958 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:07.958 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558338+0000", 2024-05-08T08:55:07.958 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070358233000000006 2024-05-08T08:55:07.958 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.958 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.958 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:07.959 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558392+0000", 2024-05-08T08:55:07.959 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4537999999999999e-05 2024-05-08T08:55:07.959 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.959 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.959 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:07.959 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558450+0000", 2024-05-08T08:55:07.959 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7710000000000001e-05 2024-05-08T08:55:07.959 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:07.959 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:07.959 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:07.959 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.959 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.959 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1099 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11517952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:55:07.959 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:55:07.960 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 151.41754221799999, 2024-05-08T08:55:07.960 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070720506000000002, 2024-05-08T08:55:07.960 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:07.960 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:07.960 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:07.960 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:07.960 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:55:07.960 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:55:07.960 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1099 2024-05-08T08:55:07.960 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.960 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:07.960 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.960 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:07.960 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:55:07.960 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:07.960 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.961 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.961 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:07.961 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:55:07.961 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:07.961 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.961 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.961 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:07.961 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:55:07.961 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3999999999999998e-07 2024-05-08T08:55:07.961 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.961 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.961 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:07.961 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487986+0000", 2024-05-08T08:55:07.961 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3519999999999999e-06 2024-05-08T08:55:07.962 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.962 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.962 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:07.962 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487989+0000", 2024-05-08T08:55:07.962 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.23e-06 2024-05-08T08:55:07.962 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.962 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.962 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:07.962 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487991+0000", 2024-05-08T08:55:07.962 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6350000000000002e-06 2024-05-08T08:55:07.962 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.962 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.962 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:07.962 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558489+0000", 2024-05-08T08:55:07.962 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070497924000000003 2024-05-08T08:55:07.962 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.962 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.963 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:07.963 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558583+0000", 2024-05-08T08:55:07.963 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3405000000000004e-05 2024-05-08T08:55:07.963 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.963 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.963 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:07.963 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558704+0000", 2024-05-08T08:55:07.963 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012171999999999999 2024-05-08T08:55:07.963 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:07.963 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:07.963 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:07.963 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.963 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.963 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1100 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11558912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:55:07.963 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:55:07.964 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 151.41752921899999, 2024-05-08T08:55:07.964 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070962266999999996, 2024-05-08T08:55:07.964 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:07.964 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:07.964 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:07.964 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:07.964 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:55:07.964 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:55:07.964 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1100 2024-05-08T08:55:07.964 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.964 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:07.964 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.964 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:07.964 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:55:07.964 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:07.965 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.965 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.965 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:07.965 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:55:07.965 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:07.965 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.965 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.965 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:07.965 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488430+0000", 2024-05-08T08:55:07.965 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000432676 2024-05-08T08:55:07.965 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.965 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.965 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:07.965 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488433+0000", 2024-05-08T08:55:07.965 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.45e-06 2024-05-08T08:55:07.965 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.966 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.966 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:07.966 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488442+0000", 2024-05-08T08:55:07.966 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.687e-06 2024-05-08T08:55:07.966 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.966 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.966 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:07.966 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488446+0000", 2024-05-08T08:55:07.966 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4939999999999997e-06 2024-05-08T08:55:07.966 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.966 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.966 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:07.966 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558756+0000", 2024-05-08T08:55:07.966 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070310051999999998 2024-05-08T08:55:07.966 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.966 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.967 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:07.967 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558844+0000", 2024-05-08T08:55:07.967 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7690999999999998e-05 2024-05-08T08:55:07.967 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.967 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.967 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:07.967 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558959+0000", 2024-05-08T08:55:07.967 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000115217 2024-05-08T08:55:07.967 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:07.967 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:07.967 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:07.967 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.967 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.967 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1103 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11599872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:55:07.967 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:55:07.967 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 151.41707154400001, 2024-05-08T08:55:07.968 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070777100999999995, 2024-05-08T08:55:07.968 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:07.968 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:07.968 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:07.968 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:07.968 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:55:07.968 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:55:07.968 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1103 2024-05-08T08:55:07.968 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.968 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:07.968 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.968 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:07.968 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:55:07.968 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:07.968 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.968 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.969 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:07.969 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:55:07.969 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:07.969 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.969 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.969 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:07.969 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488959+0000", 2024-05-08T08:55:07.969 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00050457200000000001 2024-05-08T08:55:07.969 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.969 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.969 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:07.969 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488969+0000", 2024-05-08T08:55:07.969 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.7159999999999998e-06 2024-05-08T08:55:07.969 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.969 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.969 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:07.969 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488981+0000", 2024-05-08T08:55:07.970 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2150000000000001e-05 2024-05-08T08:55:07.970 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.970 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.970 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:07.970 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488988+0000", 2024-05-08T08:55:07.970 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8759999999999999e-06 2024-05-08T08:55:07.970 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.970 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.970 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:07.970 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558992+0000", 2024-05-08T08:55:07.970 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070004358000000003 2024-05-08T08:55:07.970 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.970 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.970 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:07.970 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559093+0000", 2024-05-08T08:55:07.971 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000100299 2024-05-08T08:55:07.971 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.971 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.971 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:07.971 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559232+0000", 2024-05-08T08:55:07.971 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013913 2024-05-08T08:55:07.971 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:07.971 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:07.971 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:07.971 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.971 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.971 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1104 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11640832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:55:07.971 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:55:07.971 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 151.416526729, 2024-05-08T08:55:07.971 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070521947000000001, 2024-05-08T08:55:07.972 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:07.972 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:07.972 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:07.972 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:07.972 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:55:07.972 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:55:07.972 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1104 2024-05-08T08:55:07.972 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.972 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:07.972 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.972 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:07.972 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:55:07.972 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:07.972 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.972 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.973 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:07.973 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:55:07.973 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:07.973 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.973 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.973 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:07.973 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489700+0000", 2024-05-08T08:55:07.973 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00070067600000000001 2024-05-08T08:55:07.973 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.973 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.973 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:07.973 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489704+0000", 2024-05-08T08:55:07.973 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1640000000000003e-06 2024-05-08T08:55:07.973 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.974 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.974 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:07.974 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489716+0000", 2024-05-08T08:55:07.974 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1379000000000001e-05 2024-05-08T08:55:07.974 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.974 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.974 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:07.974 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489724+0000", 2024-05-08T08:55:07.974 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8949999999999998e-06 2024-05-08T08:55:07.974 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.974 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.974 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:07.974 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559278+0000", 2024-05-08T08:55:07.974 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.069554151999999994 2024-05-08T08:55:07.975 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.975 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.975 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:07.975 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559387+0000", 2024-05-08T08:55:07.975 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000109258 2024-05-08T08:55:07.975 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.975 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.975 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:07.975 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559521+0000", 2024-05-08T08:55:07.975 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000134423 2024-05-08T08:55:07.975 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:07.975 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:07.975 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:07.975 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.975 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.975 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:8636 3.17 3:e8a0a3e4:::rbd_header.11402b26385:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-05-08T08:55:07.975 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:54.150971+0000", 2024-05-08T08:55:07.976 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 133.754554735, 2024-05-08T08:55:07.976 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.217273233, 2024-05-08T08:55:07.976 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:07.976 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:07.976 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:07.976 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:07.976 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:55:07.976 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:55:07.976 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 8636 2024-05-08T08:55:07.976 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.976 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:07.976 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.976 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:07.976 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150971+0000", 2024-05-08T08:55:07.976 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:07.976 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.976 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.977 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:07.977 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150971+0000", 2024-05-08T08:55:07.977 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:07.977 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.977 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.977 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:07.977 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150972+0000", 2024-05-08T08:55:07.977 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.71e-07 2024-05-08T08:55:07.977 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.977 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.977 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:07.977 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150977+0000", 2024-05-08T08:55:07.977 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.152e-06 2024-05-08T08:55:07.977 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.977 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.977 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:07.978 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150990+0000", 2024-05-08T08:55:07.978 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2364999999999999e-05 2024-05-08T08:55:07.978 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.978 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.978 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:07.978 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150998+0000", 2024-05-08T08:55:07.978 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0739999999999993e-06 2024-05-08T08:55:07.978 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.978 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.978 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:07.978 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.151054+0000", 2024-05-08T08:55:07.978 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5949000000000003e-05 2024-05-08T08:55:07.978 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.978 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.978 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:55:07.979 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.151060+0000", 2024-05-08T08:55:07.979 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6120000000000004e-06 2024-05-08T08:55:07.979 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.979 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.979 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:07.979 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.367640+0000", 2024-05-08T08:55:07.979 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2165792739999999 2024-05-08T08:55:07.979 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.979 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.979 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:07.979 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.367839+0000", 2024-05-08T08:55:07.979 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000199185 2024-05-08T08:55:07.979 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.979 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.979 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:07.979 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.368245+0000", 2024-05-08T08:55:07.980 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00040575099999999998 2024-05-08T08:55:07.980 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:07.980 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:07.980 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:07.980 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.980 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.980 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5302.0:1516 3.c 3:319bf789:::rbd_data.1140d3fc54c.0000000000000025:head [stat,write 1892352~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:55:07.980 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:27.822220+0000", 2024-05-08T08:55:07.980 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 100.083305798, 2024-05-08T08:55:07.980 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.092643744, 2024-05-08T08:55:07.980 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:07.980 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:07.980 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:55:07.980 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:07.980 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5302", 2024-05-08T08:55:07.981 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/359277859", 2024-05-08T08:55:07.981 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1516 2024-05-08T08:55:07.981 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.981 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:07.981 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.981 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:07.981 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822220+0000", 2024-05-08T08:55:07.981 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:07.981 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.981 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.981 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:07.981 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822220+0000", 2024-05-08T08:55:07.981 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:07.981 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.981 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.981 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:07.981 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822221+0000", 2024-05-08T08:55:07.982 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3500000000000002e-07 2024-05-08T08:55:07.982 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.982 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.982 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:07.982 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822228+0000", 2024-05-08T08:55:07.982 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9959999999999996e-06 2024-05-08T08:55:07.982 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.982 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.982 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:07.982 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822234+0000", 2024-05-08T08:55:07.982 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5429999999999999e-06 2024-05-08T08:55:07.982 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.982 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.982 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:07.983 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822238+0000", 2024-05-08T08:55:07.983 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.732e-06 2024-05-08T08:55:07.983 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.983 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.983 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:07.983 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822285+0000", 2024-05-08T08:55:07.983 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6810000000000001e-05 2024-05-08T08:55:07.983 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.983 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.983 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:07.983 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822358+0000", 2024-05-08T08:55:07.983 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3473000000000006e-05 2024-05-08T08:55:07.983 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.983 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.983 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 5", 2024-05-08T08:55:07.984 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822489+0000", 2024-05-08T08:55:07.984 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013011199999999999 2024-05-08T08:55:07.984 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.984 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.984 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:55:07.984 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.825332+0000", 2024-05-08T08:55:07.984 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0028434799999999998 2024-05-08T08:55:07.984 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.984 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.984 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:55:07.984 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.914753+0000", 2024-05-08T08:55:07.984 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089421424999999999 2024-05-08T08:55:07.984 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.984 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.985 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:55:07.985 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.914807+0000", 2024-05-08T08:55:07.985 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3829000000000002e-05 2024-05-08T08:55:07.985 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.985 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.985 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:07.985 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.914864+0000", 2024-05-08T08:55:07.985 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6909e-05 2024-05-08T08:55:07.985 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:07.985 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:07.985 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:07.985 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.985 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.985 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:24567 3.6 3:66af1c40:::rbd_data.1140d3fc54c.000000000000005a:18 [read 2080768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:07.985 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.216598+0000", 2024-05-08T08:55:07.985 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 95.688927718000002, 2024-05-08T08:55:07.986 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071602647000000005, 2024-05-08T08:55:07.986 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:07.986 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:07.986 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:07.986 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:07.986 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:07.986 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:07.986 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 24567 2024-05-08T08:55:07.986 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.986 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:07.986 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.986 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:07.986 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216598+0000", 2024-05-08T08:55:07.986 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:07.986 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.987 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.987 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:07.987 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216598+0000", 2024-05-08T08:55:07.987 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:07.987 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.987 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.987 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:07.987 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216599+0000", 2024-05-08T08:55:07.987 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8999999999999997e-07 2024-05-08T08:55:07.987 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.987 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.987 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:07.987 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216601+0000", 2024-05-08T08:55:07.987 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9889999999999999e-06 2024-05-08T08:55:07.987 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.987 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.988 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:07.988 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216607+0000", 2024-05-08T08:55:07.988 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3600000000000001e-06 2024-05-08T08:55:07.988 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.988 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.988 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:07.988 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216612+0000", 2024-05-08T08:55:07.988 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.69e-06 2024-05-08T08:55:07.988 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.988 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.988 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:07.988 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.287076+0000", 2024-05-08T08:55:07.988 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070464355000000006 2024-05-08T08:55:07.988 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.988 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.989 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:07.989 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.287151+0000", 2024-05-08T08:55:07.989 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4924000000000003e-05 2024-05-08T08:55:07.989 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:07.989 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:07.989 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:07.989 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.288201+0000", 2024-05-08T08:55:07.989 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0010498389999999999 2024-05-08T08:55:07.989 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:07.989 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:07.989 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:07.989 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:07.989 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:07.989 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:55:07.991 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:55:07.991 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:55:07.994+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:55:08.092 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:55:08.094+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:55:08.143 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:55:08.143 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:55:08.143 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:55:08.143 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:55:08.152 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:55:08.193 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:55:08.196+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:55:08.293 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:55:08.296+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:55:08.301 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:55:08.302 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:55:08.302 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:55:08.302 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:55:08.302 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:55:08.311 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:55:08.312 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T08:55:08.394 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:55:08.397+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:55:08.432 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-05-08T08:55:08.474 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:55:08.475 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:55:08.475 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:55:08.475 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:55:08.475 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.475 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:12443 3.0 3:02d0158e:::rbd_data.114071bcf828.0000000000000037:head [stat,delete] snapc 19=[19] RETRY=3 ondisk+retry+write+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:08.475 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.610027+0000", 2024-05-08T08:55:08.475 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 96.866121923999998, 2024-05-08T08:55:08.475 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.083586333999999998, 2024-05-08T08:55:08.475 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:08.475 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:08.475 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:08.475 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:08.475 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:08.475 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:08.476 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 12443 2024-05-08T08:55:08.476 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.476 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:08.476 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.476 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:08.476 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610027+0000", 2024-05-08T08:55:08.476 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:08.476 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.476 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.476 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:08.476 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610027+0000", 2024-05-08T08:55:08.476 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:08.476 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.476 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.476 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:08.477 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610028+0000", 2024-05-08T08:55:08.477 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.13e-07 2024-05-08T08:55:08.477 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.477 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.477 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:08.477 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610030+0000", 2024-05-08T08:55:08.477 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7669999999999999e-06 2024-05-08T08:55:08.477 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.477 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.477 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:08.477 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610039+0000", 2024-05-08T08:55:08.477 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4959999999999996e-06 2024-05-08T08:55:08.477 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.477 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.478 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:08.478 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610045+0000", 2024-05-08T08:55:08.478 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6779999999999997e-06 2024-05-08T08:55:08.478 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.478 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.478 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:08.478 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.660873+0000", 2024-05-08T08:55:08.478 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.050828748 2024-05-08T08:55:08.478 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.478 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.478 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:08.478 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.661017+0000", 2024-05-08T08:55:08.478 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014391999999999999 2024-05-08T08:55:08.478 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.478 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.479 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:08.479 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.693613+0000", 2024-05-08T08:55:08.479 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.032596111999999997 2024-05-08T08:55:08.479 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:08.479 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:08.479 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:08.479 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.479 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.479 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:14301 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [watch reconnect cookie 139837155901536 gen 12] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:08.479 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.645078+0000", 2024-05-08T08:55:08.479 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 96.831071285999997, 2024-05-08T08:55:08.479 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088281852999999993, 2024-05-08T08:55:08.479 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:08.479 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:08.479 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:08.479 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:08.480 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:08.480 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:08.480 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14301 2024-05-08T08:55:08.480 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.480 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:08.480 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.480 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:08.480 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645078+0000", 2024-05-08T08:55:08.480 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:08.480 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.480 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.480 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:08.480 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645078+0000", 2024-05-08T08:55:08.480 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:08.480 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.480 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.480 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:08.481 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645084+0000", 2024-05-08T08:55:08.481 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4620000000000001e-06 2024-05-08T08:55:08.481 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.481 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.481 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:08.481 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645088+0000", 2024-05-08T08:55:08.481 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8039999999999999e-06 2024-05-08T08:55:08.481 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.481 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.481 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:08.481 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645106+0000", 2024-05-08T08:55:08.481 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.8318999999999999e-05 2024-05-08T08:55:08.481 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.481 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.481 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:08.481 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645113+0000", 2024-05-08T08:55:08.482 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0310000000000001e-06 2024-05-08T08:55:08.482 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.482 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.482 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:08.482 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.670493+0000", 2024-05-08T08:55:08.482 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.025379993 2024-05-08T08:55:08.482 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.482 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.482 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:08.482 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.670571+0000", 2024-05-08T08:55:08.482 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7613000000000004e-05 2024-05-08T08:55:08.482 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.482 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.482 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:08.482 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733360+0000", 2024-05-08T08:55:08.483 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.062788630999999998 2024-05-08T08:55:08.483 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:08.483 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:08.483 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:08.483 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.483 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.483 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:14420 3.0 3:06a8948e:::rbd_data.114071bcf828.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:08.483 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.649639+0000", 2024-05-08T08:55:08.483 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 96.826510138000003, 2024-05-08T08:55:08.483 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.083851897999999994, 2024-05-08T08:55:08.483 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:08.483 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:08.483 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:55:08.483 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:08.483 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:08.483 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:08.484 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14420 2024-05-08T08:55:08.484 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.484 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:08.484 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.484 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:08.484 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649639+0000", 2024-05-08T08:55:08.484 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:08.484 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.484 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.484 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:08.484 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649639+0000", 2024-05-08T08:55:08.484 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:08.484 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.484 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.484 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:08.484 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649640+0000", 2024-05-08T08:55:08.485 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6899999999999997e-07 2024-05-08T08:55:08.485 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.485 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.485 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:08.485 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649895+0000", 2024-05-08T08:55:08.485 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00025531000000000002 2024-05-08T08:55:08.485 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.485 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.485 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:08.485 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649905+0000", 2024-05-08T08:55:08.485 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0198e-05 2024-05-08T08:55:08.485 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.485 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.485 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:08.485 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649911+0000", 2024-05-08T08:55:08.486 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2310000000000001e-06 2024-05-08T08:55:08.486 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.486 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.486 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:08.486 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.684099+0000", 2024-05-08T08:55:08.486 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.034188017000000001 2024-05-08T08:55:08.486 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.486 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.486 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:08.486 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.684172+0000", 2024-05-08T08:55:08.486 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3201999999999995e-05 2024-05-08T08:55:08.486 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.487 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.487 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 2", 2024-05-08T08:55:08.487 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.684418+0000", 2024-05-08T08:55:08.487 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00024559099999999997 2024-05-08T08:55:08.487 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.487 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.487 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:55:08.487 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.705538+0000", 2024-05-08T08:55:08.487 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.021119707000000001 2024-05-08T08:55:08.487 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.487 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.487 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:55:08.487 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733399+0000", 2024-05-08T08:55:08.487 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.027861543999999999 2024-05-08T08:55:08.488 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.488 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.488 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:55:08.488 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733445+0000", 2024-05-08T08:55:08.488 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5587000000000002e-05 2024-05-08T08:55:08.488 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.488 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.488 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:08.488 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733491+0000", 2024-05-08T08:55:08.488 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5841999999999999e-05 2024-05-08T08:55:08.488 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:08.488 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:08.488 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:08.488 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.488 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.489 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28547 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889179216] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:08.489 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.355654+0000", 2024-05-08T08:55:08.489 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 96.120494910000005, 2024-05-08T08:55:08.489 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078692261999999999, 2024-05-08T08:55:08.489 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:08.489 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:08.489 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:08.489 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:08.489 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:08.489 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:08.489 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28547 2024-05-08T08:55:08.489 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.489 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:08.489 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.489 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:08.490 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355654+0000", 2024-05-08T08:55:08.490 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:08.490 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.490 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.490 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:08.490 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355654+0000", 2024-05-08T08:55:08.490 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:08.490 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.490 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.490 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:08.490 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355654+0000", 2024-05-08T08:55:08.490 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3999999999999998e-07 2024-05-08T08:55:08.490 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.490 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.490 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:08.491 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355656+0000", 2024-05-08T08:55:08.491 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.353e-06 2024-05-08T08:55:08.491 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.491 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.491 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:08.491 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355663+0000", 2024-05-08T08:55:08.491 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2259999999999999e-06 2024-05-08T08:55:08.491 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.491 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.491 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:08.491 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355666+0000", 2024-05-08T08:55:08.491 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7930000000000002e-06 2024-05-08T08:55:08.491 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.491 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.491 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:08.492 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434117+0000", 2024-05-08T08:55:08.492 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078451231999999996 2024-05-08T08:55:08.492 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.492 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.492 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:08.492 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434198+0000", 2024-05-08T08:55:08.492 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0879999999999998e-05 2024-05-08T08:55:08.492 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.492 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.492 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:08.492 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434346+0000", 2024-05-08T08:55:08.492 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014853800000000001 2024-05-08T08:55:08.492 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:08.492 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:08.492 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:08.492 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.493 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.493 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28548 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889189696] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:08.493 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.355669+0000", 2024-05-08T08:55:08.493 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 96.120479560000007, 2024-05-08T08:55:08.493 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078956606999999998, 2024-05-08T08:55:08.493 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:08.493 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:08.493 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:08.493 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:08.493 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:08.493 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:08.493 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28548 2024-05-08T08:55:08.493 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.493 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:08.493 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.493 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:08.494 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355669+0000", 2024-05-08T08:55:08.494 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:08.494 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.494 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.494 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:08.494 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355669+0000", 2024-05-08T08:55:08.494 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:08.494 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.494 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.494 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:08.494 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355762+0000", 2024-05-08T08:55:08.494 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3015000000000003e-05 2024-05-08T08:55:08.494 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.494 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.494 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:08.495 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355765+0000", 2024-05-08T08:55:08.495 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.368e-06 2024-05-08T08:55:08.495 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.495 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.495 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:08.495 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355771+0000", 2024-05-08T08:55:08.495 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5989999999999998e-06 2024-05-08T08:55:08.495 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.495 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.495 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:08.495 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355775+0000", 2024-05-08T08:55:08.495 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7280000000000001e-06 2024-05-08T08:55:08.495 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.495 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.495 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:08.496 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434379+0000", 2024-05-08T08:55:08.496 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078603662000000005 2024-05-08T08:55:08.496 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.496 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.496 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:08.496 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434473+0000", 2024-05-08T08:55:08.496 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4104000000000006e-05 2024-05-08T08:55:08.496 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.496 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.496 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:08.496 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434626+0000", 2024-05-08T08:55:08.496 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00015313100000000001 2024-05-08T08:55:08.496 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:08.496 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:08.496 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:08.497 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.497 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.497 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28549 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889202400] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:08.497 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.355781+0000", 2024-05-08T08:55:08.497 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 96.120367986000005, 2024-05-08T08:55:08.497 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079117946999999994, 2024-05-08T08:55:08.497 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:08.497 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:08.497 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:08.497 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:08.497 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:08.497 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:08.497 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28549 2024-05-08T08:55:08.497 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.497 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:08.497 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.498 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:08.498 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355781+0000", 2024-05-08T08:55:08.498 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:08.498 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.498 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.498 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:08.498 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355781+0000", 2024-05-08T08:55:08.498 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:08.498 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.498 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.498 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:08.498 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356162+0000", 2024-05-08T08:55:08.498 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00038099100000000001 2024-05-08T08:55:08.498 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.499 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.499 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:08.499 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356165+0000", 2024-05-08T08:55:08.499 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1240000000000001e-06 2024-05-08T08:55:08.499 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.499 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.499 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:08.499 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356173+0000", 2024-05-08T08:55:08.499 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.6199999999999999e-06 2024-05-08T08:55:08.499 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.499 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.499 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:08.499 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356182+0000", 2024-05-08T08:55:08.499 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7760000000000003e-06 2024-05-08T08:55:08.499 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.500 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.500 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:08.500 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434675+0000", 2024-05-08T08:55:08.500 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078493119 2024-05-08T08:55:08.500 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.500 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.500 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:08.500 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434765+0000", 2024-05-08T08:55:08.500 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0402000000000006e-05 2024-05-08T08:55:08.500 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.500 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.500 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:08.500 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434899+0000", 2024-05-08T08:55:08.500 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013391499999999999 2024-05-08T08:55:08.500 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:08.500 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:08.501 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:08.501 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.501 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.501 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28550 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888597280] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:08.501 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:55:08.501 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 96.119956939999994, 2024-05-08T08:55:08.501 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078933990999999995, 2024-05-08T08:55:08.501 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:08.501 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:08.501 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:08.501 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:08.501 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:08.501 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:08.501 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28550 2024-05-08T08:55:08.501 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.502 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:08.502 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.502 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:08.502 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:55:08.502 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:08.502 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.502 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.502 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:08.502 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:55:08.502 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:08.502 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.502 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.502 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:08.502 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:55:08.503 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.34e-07 2024-05-08T08:55:08.503 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.503 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.503 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:08.503 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356195+0000", 2024-05-08T08:55:08.503 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8710000000000001e-06 2024-05-08T08:55:08.503 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.503 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.503 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:08.503 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356200+0000", 2024-05-08T08:55:08.503 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3000000000000003e-06 2024-05-08T08:55:08.503 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.503 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.503 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:08.503 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356203+0000", 2024-05-08T08:55:08.504 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4039999999999999e-06 2024-05-08T08:55:08.504 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.504 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.504 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:08.504 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434928+0000", 2024-05-08T08:55:08.504 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078725331999999995 2024-05-08T08:55:08.504 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.504 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.504 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:08.504 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435007+0000", 2024-05-08T08:55:08.504 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8415000000000001e-05 2024-05-08T08:55:08.504 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.504 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.504 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:08.504 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435126+0000", 2024-05-08T08:55:08.504 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011923500000000001 2024-05-08T08:55:08.505 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:08.505 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:08.505 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:08.505 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.505 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.505 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28551 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888527200] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:08.505 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.356210+0000", 2024-05-08T08:55:08.505 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 96.119938591999997, 2024-05-08T08:55:08.505 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079195072000000005, 2024-05-08T08:55:08.505 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:08.505 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:08.505 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:08.505 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:08.505 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:08.506 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:08.506 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28551 2024-05-08T08:55:08.506 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.506 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:08.506 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.506 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:08.506 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356210+0000", 2024-05-08T08:55:08.506 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:08.506 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.506 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.506 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:08.506 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356210+0000", 2024-05-08T08:55:08.506 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:08.506 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.506 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.507 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:08.507 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356211+0000", 2024-05-08T08:55:08.507 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4499999999999998e-07 2024-05-08T08:55:08.507 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.507 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.507 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:08.507 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356212+0000", 2024-05-08T08:55:08.507 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6029999999999999e-06 2024-05-08T08:55:08.507 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.507 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.507 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:08.507 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356221+0000", 2024-05-08T08:55:08.507 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4200000000000007e-06 2024-05-08T08:55:08.507 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.507 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.508 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:08.508 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356224+0000", 2024-05-08T08:55:08.508 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.038e-06 2024-05-08T08:55:08.508 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.508 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.508 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:08.508 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435178+0000", 2024-05-08T08:55:08.508 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078954625000000001 2024-05-08T08:55:08.508 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.508 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.508 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:08.508 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435256+0000", 2024-05-08T08:55:08.508 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7337999999999997e-05 2024-05-08T08:55:08.508 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.508 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.508 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:08.509 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435405+0000", 2024-05-08T08:55:08.509 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000149703 2024-05-08T08:55:08.509 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:08.509 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:08.509 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:08.509 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.509 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.509 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28552 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889231456] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:08.509 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.356228+0000", 2024-05-08T08:55:08.509 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 96.119920914000005, 2024-05-08T08:55:08.509 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079346920000000001, 2024-05-08T08:55:08.509 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:08.509 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:08.509 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:08.509 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:08.510 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:08.510 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:08.510 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28552 2024-05-08T08:55:08.510 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.510 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:08.510 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.510 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:08.510 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356228+0000", 2024-05-08T08:55:08.510 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:08.510 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.510 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.510 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:08.511 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356228+0000", 2024-05-08T08:55:08.511 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:08.511 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.511 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.511 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:08.511 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356959+0000", 2024-05-08T08:55:08.511 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00073097099999999996 2024-05-08T08:55:08.511 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.511 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.511 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:08.511 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356967+0000", 2024-05-08T08:55:08.511 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0409999999999998e-06 2024-05-08T08:55:08.511 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.511 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.511 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:08.512 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356979+0000", 2024-05-08T08:55:08.512 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1489e-05 2024-05-08T08:55:08.512 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.512 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.512 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:08.512 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356986+0000", 2024-05-08T08:55:08.512 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3699999999999997e-06 2024-05-08T08:55:08.512 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.512 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.512 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:08.512 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435433+0000", 2024-05-08T08:55:08.512 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078447520000000007 2024-05-08T08:55:08.512 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.512 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.512 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:08.512 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435484+0000", 2024-05-08T08:55:08.513 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0229000000000003e-05 2024-05-08T08:55:08.513 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.513 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.513 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:08.513 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435575+0000", 2024-05-08T08:55:08.513 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1299999999999997e-05 2024-05-08T08:55:08.513 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:08.513 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:08.513 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:08.513 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.513 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.513 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28553 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888713648] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:08.513 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:55:08.514 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 96.119146255999993, 2024-05-08T08:55:08.514 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078782034000000001, 2024-05-08T08:55:08.514 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:08.514 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:08.514 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:08.514 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:08.514 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:08.514 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:08.514 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28553 2024-05-08T08:55:08.514 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.514 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:08.514 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.514 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:08.515 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:55:08.515 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:08.515 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.515 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.515 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:08.515 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:55:08.515 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:08.515 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.515 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.515 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:08.515 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:55:08.515 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6600000000000002e-07 2024-05-08T08:55:08.515 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.515 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.515 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:08.516 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357006+0000", 2024-05-08T08:55:08.516 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7190000000000001e-06 2024-05-08T08:55:08.516 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.516 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.516 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:08.516 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357013+0000", 2024-05-08T08:55:08.516 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.435e-06 2024-05-08T08:55:08.516 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.516 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.516 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:08.516 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357018+0000", 2024-05-08T08:55:08.516 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7260000000000002e-06 2024-05-08T08:55:08.516 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.516 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.517 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:08.517 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435623+0000", 2024-05-08T08:55:08.517 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078604972999999995 2024-05-08T08:55:08.517 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.517 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.517 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:08.517 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435695+0000", 2024-05-08T08:55:08.517 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1959999999999995e-05 2024-05-08T08:55:08.517 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.517 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.517 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:08.517 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435785+0000", 2024-05-08T08:55:08.517 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.9754999999999997e-05 2024-05-08T08:55:08.517 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:08.517 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:08.518 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:08.518 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.518 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.518 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28554 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888980272] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:08.518 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357024+0000", 2024-05-08T08:55:08.518 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 96.119124619999994, 2024-05-08T08:55:08.518 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078999870999999999, 2024-05-08T08:55:08.518 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:08.518 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:08.518 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:08.518 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:08.518 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:08.518 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:08.518 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28554 2024-05-08T08:55:08.518 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.519 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:08.519 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.519 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:08.519 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357024+0000", 2024-05-08T08:55:08.519 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:08.519 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.519 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.519 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:08.519 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357024+0000", 2024-05-08T08:55:08.519 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:08.519 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.519 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.519 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:08.519 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357025+0000", 2024-05-08T08:55:08.519 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:55:08.519 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.520 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.520 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:08.520 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357027+0000", 2024-05-08T08:55:08.520 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2050000000000001e-06 2024-05-08T08:55:08.520 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.520 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.520 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:08.520 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357033+0000", 2024-05-08T08:55:08.520 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5679999999999996e-06 2024-05-08T08:55:08.520 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.520 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.520 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:08.520 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357038+0000", 2024-05-08T08:55:08.520 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2889999999999999e-06 2024-05-08T08:55:08.520 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.520 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.521 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:08.521 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435823+0000", 2024-05-08T08:55:08.521 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078785677999999998 2024-05-08T08:55:08.521 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.521 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.521 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:08.521 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435892+0000", 2024-05-08T08:55:08.521 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8379000000000004e-05 2024-05-08T08:55:08.521 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.521 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.521 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:08.521 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436024+0000", 2024-05-08T08:55:08.521 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013239799999999999 2024-05-08T08:55:08.521 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:08.521 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:08.522 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:08.522 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.522 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.522 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28555 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889001104] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:08.522 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:55:08.522 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 96.119105855000001, 2024-05-08T08:55:08.522 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079115742000000003, 2024-05-08T08:55:08.522 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:08.522 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:08.522 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:08.522 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:08.522 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:08.522 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:08.522 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28555 2024-05-08T08:55:08.522 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.523 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:08.523 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.523 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:08.523 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:55:08.523 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:08.523 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.523 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.523 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:08.523 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:55:08.523 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:08.523 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.523 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.523 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:08.523 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:55:08.523 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9200000000000002e-07 2024-05-08T08:55:08.523 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.524 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.524 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:08.524 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357046+0000", 2024-05-08T08:55:08.524 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1670000000000002e-06 2024-05-08T08:55:08.524 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.524 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.524 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:08.524 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357052+0000", 2024-05-08T08:55:08.524 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.139e-06 2024-05-08T08:55:08.524 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.524 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.524 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:08.524 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357056+0000", 2024-05-08T08:55:08.524 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2440000000000004e-06 2024-05-08T08:55:08.524 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.524 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.525 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:08.525 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436059+0000", 2024-05-08T08:55:08.525 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079003070999999994 2024-05-08T08:55:08.525 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.525 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.525 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:08.525 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436097+0000", 2024-05-08T08:55:08.525 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7784000000000003e-05 2024-05-08T08:55:08.525 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.525 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.525 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:08.525 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436159+0000", 2024-05-08T08:55:08.525 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2045000000000007e-05 2024-05-08T08:55:08.525 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:08.525 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:08.525 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:08.526 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.526 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.526 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28556 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889022064] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:08.526 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357065+0000", 2024-05-08T08:55:08.526 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 96.119083821999993, 2024-05-08T08:55:08.526 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079327976999999994, 2024-05-08T08:55:08.526 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:08.526 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:08.526 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:08.526 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:08.526 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:08.526 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:08.526 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28556 2024-05-08T08:55:08.526 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.527 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:08.527 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.527 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:08.527 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357065+0000", 2024-05-08T08:55:08.527 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:08.527 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.527 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.527 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:08.527 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357065+0000", 2024-05-08T08:55:08.527 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:08.527 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.527 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.527 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:08.527 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358958+0000", 2024-05-08T08:55:08.527 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001892915 2024-05-08T08:55:08.528 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.528 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.528 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:08.528 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358962+0000", 2024-05-08T08:55:08.528 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4599999999999999e-06 2024-05-08T08:55:08.528 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.528 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.528 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:08.528 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358970+0000", 2024-05-08T08:55:08.528 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.6090000000000001e-06 2024-05-08T08:55:08.528 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.528 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.528 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:08.528 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358975+0000", 2024-05-08T08:55:08.529 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7639999999999996e-06 2024-05-08T08:55:08.529 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.529 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.529 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:08.529 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436196+0000", 2024-05-08T08:55:08.529 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.077221564000000006 2024-05-08T08:55:08.529 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.529 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.529 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:08.529 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436269+0000", 2024-05-08T08:55:08.529 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2720999999999997e-05 2024-05-08T08:55:08.529 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.529 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.529 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:08.530 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436393+0000", 2024-05-08T08:55:08.530 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012394399999999999 2024-05-08T08:55:08.530 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:08.530 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:08.530 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:08.530 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.530 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.530 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28563 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888855520] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:08.530 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:55:08.530 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 96.117074423000005, 2024-05-08T08:55:08.530 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078795566999999997, 2024-05-08T08:55:08.530 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:08.530 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:08.530 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:08.530 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:08.531 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:08.531 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:08.531 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28563 2024-05-08T08:55:08.531 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.531 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:08.531 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.531 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:08.531 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:55:08.531 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:08.531 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.531 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.531 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:08.531 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:55:08.531 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:08.531 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.531 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.532 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:08.532 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:55:08.532 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1e-07 2024-05-08T08:55:08.532 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.532 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.532 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:08.532 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359080+0000", 2024-05-08T08:55:08.532 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8940000000000001e-06 2024-05-08T08:55:08.532 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.532 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.532 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:08.532 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359083+0000", 2024-05-08T08:55:08.532 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1949999999999999e-06 2024-05-08T08:55:08.532 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.533 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.533 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:08.533 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359085+0000", 2024-05-08T08:55:08.533 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4710000000000001e-06 2024-05-08T08:55:08.533 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.533 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.533 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:08.533 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437604+0000", 2024-05-08T08:55:08.533 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078518278999999996 2024-05-08T08:55:08.533 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.533 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.533 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:08.533 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437704+0000", 2024-05-08T08:55:08.533 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010010599999999999 2024-05-08T08:55:08.533 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.534 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.534 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:08.534 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437870+0000", 2024-05-08T08:55:08.534 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00016641200000000001 2024-05-08T08:55:08.534 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:08.534 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:08.534 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:08.534 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.534 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.534 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28564 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888538944] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:08.534 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.359089+0000", 2024-05-08T08:55:08.534 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 96.117060207999998, 2024-05-08T08:55:08.534 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078952675, 2024-05-08T08:55:08.534 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:08.535 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:08.535 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:08.535 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:08.535 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:08.535 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:08.535 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28564 2024-05-08T08:55:08.535 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.535 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:08.535 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.535 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:08.535 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359089+0000", 2024-05-08T08:55:08.535 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:08.535 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.535 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.535 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:08.535 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359089+0000", 2024-05-08T08:55:08.536 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:08.536 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.536 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.536 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:08.536 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360959+0000", 2024-05-08T08:55:08.536 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0018707089999999999 2024-05-08T08:55:08.536 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.536 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.536 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:08.536 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360963+0000", 2024-05-08T08:55:08.536 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0680000000000001e-06 2024-05-08T08:55:08.536 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.536 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.536 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:08.536 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360969+0000", 2024-05-08T08:55:08.537 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1920000000000002e-06 2024-05-08T08:55:08.537 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.537 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.537 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:08.537 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360973+0000", 2024-05-08T08:55:08.537 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.031e-06 2024-05-08T08:55:08.537 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.537 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.537 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:08.537 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437916+0000", 2024-05-08T08:55:08.537 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.076942939000000002 2024-05-08T08:55:08.537 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.537 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.537 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:08.537 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437973+0000", 2024-05-08T08:55:08.538 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7358999999999997e-05 2024-05-08T08:55:08.538 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.538 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.538 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:08.538 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.438041+0000", 2024-05-08T08:55:08.538 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8376999999999999e-05 2024-05-08T08:55:08.538 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:08.538 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:08.538 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:08.538 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.538 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.538 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28574 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889427600] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:08.538 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:55:08.538 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 96.115055701000003, 2024-05-08T08:55:08.539 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078782476000000004, 2024-05-08T08:55:08.539 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:08.539 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:08.539 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:08.539 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:08.539 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:08.539 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:08.539 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28574 2024-05-08T08:55:08.539 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.539 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:08.539 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.539 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:08.539 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:55:08.539 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:08.539 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.540 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.540 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:08.540 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:55:08.540 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:08.540 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.540 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.540 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:08.540 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:55:08.540 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1799999999999999e-07 2024-05-08T08:55:08.540 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.540 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.540 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:08.540 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361095+0000", 2024-05-08T08:55:08.540 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3850000000000001e-06 2024-05-08T08:55:08.540 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.541 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.541 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:08.541 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361098+0000", 2024-05-08T08:55:08.541 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7989999999999998e-06 2024-05-08T08:55:08.541 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.541 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.541 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:08.541 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361100+0000", 2024-05-08T08:55:08.541 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4669999999999998e-06 2024-05-08T08:55:08.541 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.541 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.541 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:08.541 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439768+0000", 2024-05-08T08:55:08.541 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078668057999999999 2024-05-08T08:55:08.541 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.542 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.542 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:08.542 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439811+0000", 2024-05-08T08:55:08.542 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2459000000000001e-05 2024-05-08T08:55:08.542 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.542 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.542 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:08.542 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439876+0000", 2024-05-08T08:55:08.542 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5090000000000002e-05 2024-05-08T08:55:08.542 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:08.542 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:08.542 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:08.542 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.542 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.542 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28575 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889439216] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:08.543 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361103+0000", 2024-05-08T08:55:08.543 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 96.115045686000002, 2024-05-08T08:55:08.543 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078966828000000003, 2024-05-08T08:55:08.543 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:08.543 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:08.543 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:08.543 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:08.543 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:08.543 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:08.543 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28575 2024-05-08T08:55:08.543 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.543 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:08.543 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.543 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:08.543 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361103+0000", 2024-05-08T08:55:08.544 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:08.544 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.544 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.544 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:08.544 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361103+0000", 2024-05-08T08:55:08.544 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:08.544 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.544 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.544 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:08.544 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361104+0000", 2024-05-08T08:55:08.544 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1400000000000001e-07 2024-05-08T08:55:08.544 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.544 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.544 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:08.544 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361105+0000", 2024-05-08T08:55:08.545 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5540000000000001e-06 2024-05-08T08:55:08.545 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.545 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.545 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:08.545 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361108+0000", 2024-05-08T08:55:08.545 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.841e-06 2024-05-08T08:55:08.545 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.545 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.545 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:08.545 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361110+0000", 2024-05-08T08:55:08.545 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.497e-06 2024-05-08T08:55:08.545 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.545 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.545 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:08.545 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439916+0000", 2024-05-08T08:55:08.545 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078805372999999998 2024-05-08T08:55:08.546 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.546 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.546 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:08.546 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440000+0000", 2024-05-08T08:55:08.546 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4730999999999999e-05 2024-05-08T08:55:08.546 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.546 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.546 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:08.546 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440070+0000", 2024-05-08T08:55:08.546 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9617999999999997e-05 2024-05-08T08:55:08.546 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:08.546 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:08.546 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:08.546 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.547 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.547 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28576 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889451328] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:08.547 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361113+0000", 2024-05-08T08:55:08.547 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 96.115035500000005, 2024-05-08T08:55:08.547 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079088862999999995, 2024-05-08T08:55:08.547 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:08.547 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:08.547 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:08.547 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:08.547 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:08.547 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:08.547 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28576 2024-05-08T08:55:08.547 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.547 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:08.547 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.547 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:08.548 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361113+0000", 2024-05-08T08:55:08.548 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:08.548 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.548 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.548 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:08.548 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361113+0000", 2024-05-08T08:55:08.548 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:08.548 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.548 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.548 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:08.548 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361114+0000", 2024-05-08T08:55:08.548 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1299999999999999e-07 2024-05-08T08:55:08.548 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.548 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.548 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:08.548 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361115+0000", 2024-05-08T08:55:08.549 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.4300000000000001e-06 2024-05-08T08:55:08.549 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.549 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.549 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:08.549 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361118+0000", 2024-05-08T08:55:08.549 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9000000000000002e-06 2024-05-08T08:55:08.549 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.549 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.549 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:08.549 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361120+0000", 2024-05-08T08:55:08.549 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4739999999999999e-06 2024-05-08T08:55:08.549 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.549 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.549 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:08.550 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440096+0000", 2024-05-08T08:55:08.550 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078975629000000006 2024-05-08T08:55:08.550 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.550 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.550 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:08.550 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440135+0000", 2024-05-08T08:55:08.550 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9202000000000002e-05 2024-05-08T08:55:08.550 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.550 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.550 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:08.550 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440202+0000", 2024-05-08T08:55:08.550 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7014999999999994e-05 2024-05-08T08:55:08.550 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:08.550 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:08.551 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:08.551 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.551 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.551 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28577 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889462640] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:08.551 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:55:08.551 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 96.115025317999994, 2024-05-08T08:55:08.551 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079194622000000006, 2024-05-08T08:55:08.551 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:08.551 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:08.551 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:08.551 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:08.551 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:08.551 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:08.551 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28577 2024-05-08T08:55:08.551 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.551 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:08.552 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.552 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:08.552 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:55:08.552 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:08.552 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.552 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.552 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:08.552 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:55:08.552 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:08.552 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.552 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.552 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:08.552 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:55:08.552 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1400000000000001e-07 2024-05-08T08:55:08.552 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.553 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.553 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:08.553 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361125+0000", 2024-05-08T08:55:08.553 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.372e-06 2024-05-08T08:55:08.553 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.553 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.553 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:08.553 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361128+0000", 2024-05-08T08:55:08.553 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0309999999999999e-06 2024-05-08T08:55:08.553 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.553 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.553 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:08.553 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361131+0000", 2024-05-08T08:55:08.553 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4040000000000002e-06 2024-05-08T08:55:08.553 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.554 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.554 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:08.554 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440225+0000", 2024-05-08T08:55:08.554 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079094600000000001 2024-05-08T08:55:08.554 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.554 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.554 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:08.554 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440260+0000", 2024-05-08T08:55:08.554 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4703000000000003e-05 2024-05-08T08:55:08.554 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.554 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.554 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:08.554 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440318+0000", 2024-05-08T08:55:08.554 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8298000000000003e-05 2024-05-08T08:55:08.554 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:08.555 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:08.555 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:08.555 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.555 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.555 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28578 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889474192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:08.555 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361134+0000", 2024-05-08T08:55:08.555 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 96.115014892999994, 2024-05-08T08:55:08.555 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079310295000000003, 2024-05-08T08:55:08.555 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:08.555 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:08.555 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:08.555 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:08.555 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:08.555 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:08.555 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28578 2024-05-08T08:55:08.555 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.556 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:08.556 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.556 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:08.556 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361134+0000", 2024-05-08T08:55:08.556 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:08.556 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.556 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.556 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:08.556 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361134+0000", 2024-05-08T08:55:08.556 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:08.556 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.556 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.556 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:08.556 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.362958+0000", 2024-05-08T08:55:08.556 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0018237380000000001 2024-05-08T08:55:08.556 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.557 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.557 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:08.557 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.362961+0000", 2024-05-08T08:55:08.557 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.123e-06 2024-05-08T08:55:08.557 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.557 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.557 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:08.557 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.397404+0000", 2024-05-08T08:55:08.557 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.034443226 2024-05-08T08:55:08.557 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.557 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.557 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:08.557 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.397439+0000", 2024-05-08T08:55:08.557 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4690000000000002e-05 2024-05-08T08:55:08.557 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.558 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.558 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:08.558 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440340+0000", 2024-05-08T08:55:08.558 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.042901419000000003 2024-05-08T08:55:08.558 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.558 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.558 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:08.558 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440376+0000", 2024-05-08T08:55:08.558 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5444999999999997e-05 2024-05-08T08:55:08.558 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:08.558 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:08.558 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:08.558 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440444+0000", 2024-05-08T08:55:08.558 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8653999999999997e-05 2024-05-08T08:55:08.558 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:08.558 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:08.559 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:08.559 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:08.559 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:08.559 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:55:08.560 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:55:08.561 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:55:08.561 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:55:08.561 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:55:08.562 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:55:08.562 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:55:08.562 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:55:08.562 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:55:08.562+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:55:08.570 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:55:08.570 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:55:08.591 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:55:08.591 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:55:08.611 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:55:08.611 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:55:08.660 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:55:08.663+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:55:08.691 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:08.691 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:08.700 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:55:08.761 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:55:08.764+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:55:08.809 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:08.809 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:08.818 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:55:08.862 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:55:08.865+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:55:08.928 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:08.928 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:08.937 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:55:08.981 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:55:08.984+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:55:09.045 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:09.045 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:09.054 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:55:09.094 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:55:09.097+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:55:09.165 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:55:09.168+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:55:09.198 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:09.198 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:09.207 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:55:09.265 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:55:09.268+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:55:09.311 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:09.312 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:09.320 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:55:09.405 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:55:09.408+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:55:09.439 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4] out_osds: [5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-05-08T08:55:09.439 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T08:55:09.439 INFO:tasks.thrashosds.thrasher:Removing osd 0, in_osds are: [0, 1, 2, 3, 4] 2024-05-08T08:55:09.440 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 0 2024-05-08T08:55:09.456 INFO:teuthology.orchestra.run.smithi059.stderr:nodeep-scrub is unset 2024-05-08T08:55:09.467 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:55:09.469+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:55:09.472 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:09.472 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:09.482 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:55:09.568 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:55:09.571+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:55:09.632 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:09.632 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:09.641 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:55:09.683 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:55:09.685+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:55:09.771 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:55:09.772+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:55:09.772 INFO:tasks.ceph.mon.a.smithi059.stderr:2024-05-08T08:55:09.774+0000 7fc9cc811640 -1 mon.a@0(leader).osd e158 definitely_dead 0 2024-05-08T08:55:09.788 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:09.788 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:09.797 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:55:09.873 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:55:09.875+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:55:09.903 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:09.903 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:09.912 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:55:09.996 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:55:09.999+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:55:10.057 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:10.057 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:10.066 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:55:10.108 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:55:10.110+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:55:10.175 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:55:10.178+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:55:10.212 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:10.212 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:10.222 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:55:10.277 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:55:10.278+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:55:10.336 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:10.336 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:10.345 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:55:10.386 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:55:10.389+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:55:10.471 INFO:teuthology.orchestra.run.smithi059.stderr:marked out osd.0. 2024-05-08T08:55:10.478 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:55:10.481+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:55:10.504 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:10.504 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:10.513 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:55:10.579 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:55:10.582+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:55:10.628 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T08:55:10.664 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:10.664 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:10.673 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:55:10.680 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:55:10.683+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:55:10.753 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:55:10.754 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:55:10.780 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:55:10.782+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:55:10.786 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:55:10.786 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:55:10.804 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:55:10.804 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:55:10.822 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:55:10.823 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:55:10.836 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:55:10.836 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:55:10.836 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:55:10.837 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:55:10.840 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:55:10.840 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:55:10.845 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:55:10.858 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:55:10.859 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:55:10.912 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:55:10.913+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:55:10.967 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:55:10.967 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:55:10.968 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:55:10.968 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:55:10.968 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:55:10.976 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:55:10.982 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:55:10.984+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:55:11.083 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:55:11.085+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:55:11.128 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:55:11.128 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:55:11.128 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:55:11.128 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:55:11.128 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.128 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:12443 3.0 3:02d0158e:::rbd_data.114071bcf828.0000000000000037:head [stat,delete] snapc 19=[19] RETRY=3 ondisk+retry+write+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:11.128 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.610027+0000", 2024-05-08T08:55:11.128 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 99.519651789999998, 2024-05-08T08:55:11.129 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.083586333999999998, 2024-05-08T08:55:11.129 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:11.129 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:11.129 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:11.129 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:11.129 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:11.129 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:11.129 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 12443 2024-05-08T08:55:11.129 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.129 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:11.129 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.129 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:11.129 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610027+0000", 2024-05-08T08:55:11.129 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:11.129 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.129 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.130 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:11.130 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610027+0000", 2024-05-08T08:55:11.130 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:11.130 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.130 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.130 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:11.130 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610028+0000", 2024-05-08T08:55:11.130 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.13e-07 2024-05-08T08:55:11.130 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.131 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.131 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:11.131 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610030+0000", 2024-05-08T08:55:11.131 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7669999999999999e-06 2024-05-08T08:55:11.131 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.131 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.131 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:11.131 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610039+0000", 2024-05-08T08:55:11.131 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4959999999999996e-06 2024-05-08T08:55:11.131 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.131 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.131 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:11.131 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610045+0000", 2024-05-08T08:55:11.131 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6779999999999997e-06 2024-05-08T08:55:11.131 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.131 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.132 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:11.132 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.660873+0000", 2024-05-08T08:55:11.132 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.050828748 2024-05-08T08:55:11.132 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.132 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.132 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:11.132 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.661017+0000", 2024-05-08T08:55:11.132 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014391999999999999 2024-05-08T08:55:11.132 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.132 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.132 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:11.132 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.693613+0000", 2024-05-08T08:55:11.132 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.032596111999999997 2024-05-08T08:55:11.132 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.132 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:11.133 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.133 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.133 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.133 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:14301 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [watch reconnect cookie 139837155901536 gen 12] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:11.133 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.645078+0000", 2024-05-08T08:55:11.133 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 99.484601151999996, 2024-05-08T08:55:11.133 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088281852999999993, 2024-05-08T08:55:11.133 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:11.133 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:11.133 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:11.133 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:11.133 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:11.134 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:11.134 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14301 2024-05-08T08:55:11.134 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.134 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:11.134 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.134 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:11.134 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645078+0000", 2024-05-08T08:55:11.134 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:11.134 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.134 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.134 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:11.134 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645078+0000", 2024-05-08T08:55:11.135 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:11.135 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.135 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.135 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:11.135 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645084+0000", 2024-05-08T08:55:11.135 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4620000000000001e-06 2024-05-08T08:55:11.135 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.135 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.135 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:11.135 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645088+0000", 2024-05-08T08:55:11.135 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8039999999999999e-06 2024-05-08T08:55:11.135 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.135 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.135 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:11.136 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645106+0000", 2024-05-08T08:55:11.136 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.8318999999999999e-05 2024-05-08T08:55:11.136 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.136 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.136 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:11.136 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645113+0000", 2024-05-08T08:55:11.136 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0310000000000001e-06 2024-05-08T08:55:11.136 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.136 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.136 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:11.136 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.670493+0000", 2024-05-08T08:55:11.136 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.025379993 2024-05-08T08:55:11.136 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.136 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.136 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:11.137 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.670571+0000", 2024-05-08T08:55:11.137 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7613000000000004e-05 2024-05-08T08:55:11.137 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.137 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.137 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:11.137 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733360+0000", 2024-05-08T08:55:11.137 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.062788630999999998 2024-05-08T08:55:11.137 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.137 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:11.137 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.137 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.137 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.137 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:14420 3.0 3:06a8948e:::rbd_data.114071bcf828.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:11.137 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.649639+0000", 2024-05-08T08:55:11.137 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 99.480040004000003, 2024-05-08T08:55:11.137 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.083851897999999994, 2024-05-08T08:55:11.138 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:11.138 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:11.138 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:55:11.138 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:11.138 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:11.138 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:11.138 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14420 2024-05-08T08:55:11.138 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.138 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:11.138 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.138 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:11.138 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649639+0000", 2024-05-08T08:55:11.138 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:11.138 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.139 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.139 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:11.139 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649639+0000", 2024-05-08T08:55:11.139 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:11.139 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.139 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.139 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:11.139 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649640+0000", 2024-05-08T08:55:11.139 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6899999999999997e-07 2024-05-08T08:55:11.139 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.139 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.139 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:11.139 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649895+0000", 2024-05-08T08:55:11.139 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00025531000000000002 2024-05-08T08:55:11.139 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.139 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.140 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:11.140 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649905+0000", 2024-05-08T08:55:11.140 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0198e-05 2024-05-08T08:55:11.140 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.140 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.140 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:11.140 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649911+0000", 2024-05-08T08:55:11.140 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2310000000000001e-06 2024-05-08T08:55:11.140 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.140 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.140 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:11.140 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.684099+0000", 2024-05-08T08:55:11.140 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.034188017000000001 2024-05-08T08:55:11.140 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.140 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.141 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:11.141 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.684172+0000", 2024-05-08T08:55:11.141 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3201999999999995e-05 2024-05-08T08:55:11.141 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.141 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.141 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 2", 2024-05-08T08:55:11.141 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.684418+0000", 2024-05-08T08:55:11.141 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00024559099999999997 2024-05-08T08:55:11.141 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.141 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.141 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:55:11.141 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.705538+0000", 2024-05-08T08:55:11.141 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.021119707000000001 2024-05-08T08:55:11.141 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.142 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.142 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:55:11.142 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733399+0000", 2024-05-08T08:55:11.142 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.027861543999999999 2024-05-08T08:55:11.142 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.142 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.142 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:55:11.142 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733445+0000", 2024-05-08T08:55:11.142 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5587000000000002e-05 2024-05-08T08:55:11.142 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.142 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.142 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:11.142 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733491+0000", 2024-05-08T08:55:11.142 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5841999999999999e-05 2024-05-08T08:55:11.143 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.143 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:11.143 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.143 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.143 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.143 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28547 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889179216] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:11.143 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.355654+0000", 2024-05-08T08:55:11.143 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 98.774024776000005, 2024-05-08T08:55:11.143 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078692261999999999, 2024-05-08T08:55:11.143 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:11.143 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:11.143 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:11.143 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:11.143 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:11.143 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:11.144 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28547 2024-05-08T08:55:11.144 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.144 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:11.144 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.144 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:11.144 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355654+0000", 2024-05-08T08:55:11.144 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:11.144 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.144 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.144 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:11.144 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355654+0000", 2024-05-08T08:55:11.144 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:11.144 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.145 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.145 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:11.145 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355654+0000", 2024-05-08T08:55:11.145 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3999999999999998e-07 2024-05-08T08:55:11.145 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.145 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.145 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:11.145 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355656+0000", 2024-05-08T08:55:11.145 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.353e-06 2024-05-08T08:55:11.145 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.145 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.145 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:11.145 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355663+0000", 2024-05-08T08:55:11.145 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2259999999999999e-06 2024-05-08T08:55:11.146 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.146 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.146 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:11.146 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355666+0000", 2024-05-08T08:55:11.146 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7930000000000002e-06 2024-05-08T08:55:11.146 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.146 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.146 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:11.146 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434117+0000", 2024-05-08T08:55:11.146 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078451231999999996 2024-05-08T08:55:11.146 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.146 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.146 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:11.147 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434198+0000", 2024-05-08T08:55:11.147 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0879999999999998e-05 2024-05-08T08:55:11.147 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.147 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.147 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:11.147 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434346+0000", 2024-05-08T08:55:11.147 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014853800000000001 2024-05-08T08:55:11.147 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.147 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:11.147 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.147 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.147 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.147 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28548 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889189696] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:11.147 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.355669+0000", 2024-05-08T08:55:11.147 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 98.774009426000006, 2024-05-08T08:55:11.148 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078956606999999998, 2024-05-08T08:55:11.148 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:11.148 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:11.148 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:11.148 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:11.148 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:11.148 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:11.148 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28548 2024-05-08T08:55:11.148 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.148 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:11.148 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.148 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:11.148 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355669+0000", 2024-05-08T08:55:11.148 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:11.149 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.149 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.149 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:11.149 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355669+0000", 2024-05-08T08:55:11.149 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:11.149 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.149 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.149 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:11.149 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355762+0000", 2024-05-08T08:55:11.149 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3015000000000003e-05 2024-05-08T08:55:11.149 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.149 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.149 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:11.149 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355765+0000", 2024-05-08T08:55:11.149 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.368e-06 2024-05-08T08:55:11.150 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.150 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.150 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:11.150 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355771+0000", 2024-05-08T08:55:11.150 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5989999999999998e-06 2024-05-08T08:55:11.150 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.150 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.150 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:11.150 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355775+0000", 2024-05-08T08:55:11.150 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7280000000000001e-06 2024-05-08T08:55:11.150 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.150 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.150 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:11.151 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434379+0000", 2024-05-08T08:55:11.151 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078603662000000005 2024-05-08T08:55:11.151 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.151 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.151 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:11.151 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434473+0000", 2024-05-08T08:55:11.151 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4104000000000006e-05 2024-05-08T08:55:11.151 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.151 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.151 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:11.151 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434626+0000", 2024-05-08T08:55:11.151 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00015313100000000001 2024-05-08T08:55:11.152 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.152 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:11.152 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.152 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.152 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.152 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28549 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889202400] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:11.152 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.355781+0000", 2024-05-08T08:55:11.152 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 98.773897852000005, 2024-05-08T08:55:11.152 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079117946999999994, 2024-05-08T08:55:11.152 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:11.152 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:11.152 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:11.152 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:11.152 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:11.153 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:11.153 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28549 2024-05-08T08:55:11.153 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.153 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:11.153 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.153 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:11.153 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355781+0000", 2024-05-08T08:55:11.153 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:11.153 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.153 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.153 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:11.153 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355781+0000", 2024-05-08T08:55:11.153 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:11.153 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.153 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.153 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:11.154 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356162+0000", 2024-05-08T08:55:11.154 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00038099100000000001 2024-05-08T08:55:11.154 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.154 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.154 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:11.154 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356165+0000", 2024-05-08T08:55:11.154 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1240000000000001e-06 2024-05-08T08:55:11.154 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.154 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.154 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:11.154 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356173+0000", 2024-05-08T08:55:11.154 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.6199999999999999e-06 2024-05-08T08:55:11.154 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.154 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.155 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:11.155 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356182+0000", 2024-05-08T08:55:11.155 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7760000000000003e-06 2024-05-08T08:55:11.155 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.155 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.155 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:11.155 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434675+0000", 2024-05-08T08:55:11.155 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078493119 2024-05-08T08:55:11.155 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.155 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.155 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:11.155 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434765+0000", 2024-05-08T08:55:11.155 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0402000000000006e-05 2024-05-08T08:55:11.155 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.156 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.156 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:11.156 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434899+0000", 2024-05-08T08:55:11.156 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013391499999999999 2024-05-08T08:55:11.156 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.156 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:11.156 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.156 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.156 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.156 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28550 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888597280] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:11.156 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:55:11.156 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 98.773486805999994, 2024-05-08T08:55:11.156 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078933990999999995, 2024-05-08T08:55:11.156 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:11.157 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:11.157 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:11.157 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:11.157 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:11.157 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:11.157 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28550 2024-05-08T08:55:11.157 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.157 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:11.157 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.157 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:11.157 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:55:11.157 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:11.157 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.157 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.157 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:11.157 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:55:11.158 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:11.158 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.158 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.158 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:11.158 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:55:11.158 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.34e-07 2024-05-08T08:55:11.158 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.158 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.158 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:11.158 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356195+0000", 2024-05-08T08:55:11.158 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8710000000000001e-06 2024-05-08T08:55:11.158 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.158 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.158 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:11.158 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356200+0000", 2024-05-08T08:55:11.159 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3000000000000003e-06 2024-05-08T08:55:11.159 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.159 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.159 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:11.159 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356203+0000", 2024-05-08T08:55:11.159 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4039999999999999e-06 2024-05-08T08:55:11.159 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.159 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.159 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:11.159 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434928+0000", 2024-05-08T08:55:11.159 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078725331999999995 2024-05-08T08:55:11.159 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.159 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.159 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:11.159 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435007+0000", 2024-05-08T08:55:11.159 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8415000000000001e-05 2024-05-08T08:55:11.160 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.160 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.160 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:11.160 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435126+0000", 2024-05-08T08:55:11.160 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011923500000000001 2024-05-08T08:55:11.160 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.160 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:11.160 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.160 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.160 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.160 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28551 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888527200] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:11.160 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.356210+0000", 2024-05-08T08:55:11.160 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 98.773468457999996, 2024-05-08T08:55:11.160 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079195072000000005, 2024-05-08T08:55:11.160 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:11.161 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:11.161 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:11.161 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:11.161 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:11.161 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:11.161 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28551 2024-05-08T08:55:11.161 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.161 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:11.161 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.161 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:11.161 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356210+0000", 2024-05-08T08:55:11.161 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:11.161 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.161 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.161 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:11.161 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356210+0000", 2024-05-08T08:55:11.162 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:11.162 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.162 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.162 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:11.162 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356211+0000", 2024-05-08T08:55:11.162 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4499999999999998e-07 2024-05-08T08:55:11.162 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.162 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.162 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:11.162 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356212+0000", 2024-05-08T08:55:11.162 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6029999999999999e-06 2024-05-08T08:55:11.162 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.162 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.162 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:11.162 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356221+0000", 2024-05-08T08:55:11.163 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4200000000000007e-06 2024-05-08T08:55:11.163 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.163 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.163 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:11.163 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356224+0000", 2024-05-08T08:55:11.163 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.038e-06 2024-05-08T08:55:11.163 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.163 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.163 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:11.163 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435178+0000", 2024-05-08T08:55:11.163 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078954625000000001 2024-05-08T08:55:11.163 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.163 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.163 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:11.164 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435256+0000", 2024-05-08T08:55:11.164 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7337999999999997e-05 2024-05-08T08:55:11.164 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.164 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.164 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:11.164 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435405+0000", 2024-05-08T08:55:11.164 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000149703 2024-05-08T08:55:11.164 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.164 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:11.164 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.164 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.164 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.164 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28552 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889231456] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:11.164 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.356228+0000", 2024-05-08T08:55:11.164 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 98.773450780000005, 2024-05-08T08:55:11.164 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079346920000000001, 2024-05-08T08:55:11.165 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:11.165 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:11.165 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:11.165 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:11.165 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:11.165 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:11.165 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28552 2024-05-08T08:55:11.165 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.165 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:11.165 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.165 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:11.165 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356228+0000", 2024-05-08T08:55:11.165 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:11.165 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.166 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.166 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:11.166 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356228+0000", 2024-05-08T08:55:11.166 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:11.166 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.166 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.166 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:11.166 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356959+0000", 2024-05-08T08:55:11.166 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00073097099999999996 2024-05-08T08:55:11.166 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.166 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.166 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:11.166 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356967+0000", 2024-05-08T08:55:11.166 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0409999999999998e-06 2024-05-08T08:55:11.166 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.167 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.167 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:11.167 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356979+0000", 2024-05-08T08:55:11.167 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1489e-05 2024-05-08T08:55:11.167 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.167 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.167 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:11.167 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356986+0000", 2024-05-08T08:55:11.167 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3699999999999997e-06 2024-05-08T08:55:11.167 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.167 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.167 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:11.167 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435433+0000", 2024-05-08T08:55:11.167 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078447520000000007 2024-05-08T08:55:11.167 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.167 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.167 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:11.168 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435484+0000", 2024-05-08T08:55:11.168 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0229000000000003e-05 2024-05-08T08:55:11.168 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.168 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.168 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:11.168 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435575+0000", 2024-05-08T08:55:11.168 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1299999999999997e-05 2024-05-08T08:55:11.168 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.168 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:11.168 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.168 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.168 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.168 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28553 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888713648] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:11.168 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:55:11.168 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 98.772676122000007, 2024-05-08T08:55:11.169 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078782034000000001, 2024-05-08T08:55:11.169 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:11.169 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:11.169 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:11.169 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:11.169 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:11.169 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:11.169 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28553 2024-05-08T08:55:11.169 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.169 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:11.169 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.169 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:11.169 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:55:11.169 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:11.170 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.170 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.170 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:11.170 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:55:11.170 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:11.170 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.170 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.170 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:11.170 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:55:11.170 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6600000000000002e-07 2024-05-08T08:55:11.170 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.170 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.170 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:11.170 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357006+0000", 2024-05-08T08:55:11.171 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7190000000000001e-06 2024-05-08T08:55:11.171 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.171 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.171 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:11.171 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357013+0000", 2024-05-08T08:55:11.171 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.435e-06 2024-05-08T08:55:11.171 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.171 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.171 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:11.171 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357018+0000", 2024-05-08T08:55:11.171 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7260000000000002e-06 2024-05-08T08:55:11.171 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.171 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.171 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:11.172 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435623+0000", 2024-05-08T08:55:11.172 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078604972999999995 2024-05-08T08:55:11.172 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.172 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.172 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:11.172 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435695+0000", 2024-05-08T08:55:11.172 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1959999999999995e-05 2024-05-08T08:55:11.172 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.172 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.172 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:11.172 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435785+0000", 2024-05-08T08:55:11.172 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.9754999999999997e-05 2024-05-08T08:55:11.172 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.172 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:11.172 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.173 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.173 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.173 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28554 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888980272] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:11.173 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357024+0000", 2024-05-08T08:55:11.173 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 98.772654485999993, 2024-05-08T08:55:11.173 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078999870999999999, 2024-05-08T08:55:11.173 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:11.173 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:11.173 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:11.173 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:11.173 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:11.173 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:11.173 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28554 2024-05-08T08:55:11.174 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.174 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:11.174 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.174 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:11.174 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357024+0000", 2024-05-08T08:55:11.174 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:11.174 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.174 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.174 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:11.174 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357024+0000", 2024-05-08T08:55:11.174 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:11.174 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.174 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.174 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:11.175 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357025+0000", 2024-05-08T08:55:11.175 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:55:11.175 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.175 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.175 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:11.175 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357027+0000", 2024-05-08T08:55:11.175 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2050000000000001e-06 2024-05-08T08:55:11.175 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.175 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.175 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:11.175 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357033+0000", 2024-05-08T08:55:11.175 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5679999999999996e-06 2024-05-08T08:55:11.175 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.175 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.175 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:11.175 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357038+0000", 2024-05-08T08:55:11.176 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2889999999999999e-06 2024-05-08T08:55:11.176 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.176 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.176 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:11.176 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435823+0000", 2024-05-08T08:55:11.176 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078785677999999998 2024-05-08T08:55:11.176 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.176 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.176 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:11.176 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435892+0000", 2024-05-08T08:55:11.176 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8379000000000004e-05 2024-05-08T08:55:11.176 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.176 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.177 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:11.177 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436024+0000", 2024-05-08T08:55:11.177 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013239799999999999 2024-05-08T08:55:11.177 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.177 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:11.177 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.177 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.177 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.177 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28555 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889001104] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:11.177 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:55:11.177 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 98.772635721, 2024-05-08T08:55:11.177 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079115742000000003, 2024-05-08T08:55:11.177 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:11.177 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:11.178 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:11.178 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:11.178 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:11.178 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:11.178 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28555 2024-05-08T08:55:11.178 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.178 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:11.178 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.178 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:11.178 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:55:11.178 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:11.178 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.178 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.178 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:11.179 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:55:11.179 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:11.179 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.179 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.179 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:11.179 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:55:11.179 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9200000000000002e-07 2024-05-08T08:55:11.179 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.179 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.179 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:11.179 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357046+0000", 2024-05-08T08:55:11.179 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1670000000000002e-06 2024-05-08T08:55:11.179 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.179 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.179 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:11.179 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357052+0000", 2024-05-08T08:55:11.179 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.139e-06 2024-05-08T08:55:11.180 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.180 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.180 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:11.180 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357056+0000", 2024-05-08T08:55:11.180 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2440000000000004e-06 2024-05-08T08:55:11.180 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.180 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.180 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:11.180 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436059+0000", 2024-05-08T08:55:11.180 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079003070999999994 2024-05-08T08:55:11.180 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.180 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.180 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:11.180 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436097+0000", 2024-05-08T08:55:11.180 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7784000000000003e-05 2024-05-08T08:55:11.180 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.181 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.181 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:11.181 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436159+0000", 2024-05-08T08:55:11.181 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2045000000000007e-05 2024-05-08T08:55:11.181 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.181 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:11.181 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.181 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.181 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.181 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28556 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889022064] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:11.181 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357065+0000", 2024-05-08T08:55:11.181 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 98.772613688000007, 2024-05-08T08:55:11.181 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079327976999999994, 2024-05-08T08:55:11.182 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:11.182 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:11.182 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:11.182 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:11.182 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:11.182 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:11.182 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28556 2024-05-08T08:55:11.182 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.182 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:11.182 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.182 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:11.182 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357065+0000", 2024-05-08T08:55:11.182 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:11.182 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.183 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.183 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:11.183 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357065+0000", 2024-05-08T08:55:11.183 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:11.183 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.183 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.183 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:11.183 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358958+0000", 2024-05-08T08:55:11.183 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001892915 2024-05-08T08:55:11.183 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.183 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.183 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:11.183 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358962+0000", 2024-05-08T08:55:11.183 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4599999999999999e-06 2024-05-08T08:55:11.183 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.184 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.184 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:11.184 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358970+0000", 2024-05-08T08:55:11.184 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.6090000000000001e-06 2024-05-08T08:55:11.184 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.184 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.184 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:11.184 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358975+0000", 2024-05-08T08:55:11.184 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7639999999999996e-06 2024-05-08T08:55:11.184 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.184 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.184 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:11.185 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436196+0000", 2024-05-08T08:55:11.185 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.077221564000000006 2024-05-08T08:55:11.185 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.185 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.185 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:11.185 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436269+0000", 2024-05-08T08:55:11.185 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2720999999999997e-05 2024-05-08T08:55:11.185 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.185 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.185 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:11.185 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436393+0000", 2024-05-08T08:55:11.185 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012394399999999999 2024-05-08T08:55:11.185 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.185 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:11.186 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.186 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.186 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.186 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28563 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888855520] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:11.186 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:55:11.186 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 98.770604289000005, 2024-05-08T08:55:11.186 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078795566999999997, 2024-05-08T08:55:11.186 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:11.186 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:11.186 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:11.186 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:11.186 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:11.186 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:11.187 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28563 2024-05-08T08:55:11.187 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.187 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:11.187 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.187 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:11.187 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:55:11.187 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:11.187 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.187 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.187 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:11.187 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:55:11.187 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:11.187 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.187 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.187 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:11.187 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:55:11.187 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1e-07 2024-05-08T08:55:11.188 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.188 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.188 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:11.188 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359080+0000", 2024-05-08T08:55:11.188 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8940000000000001e-06 2024-05-08T08:55:11.188 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.188 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.188 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:11.188 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359083+0000", 2024-05-08T08:55:11.188 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1949999999999999e-06 2024-05-08T08:55:11.188 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.188 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.188 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:11.188 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359085+0000", 2024-05-08T08:55:11.188 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4710000000000001e-06 2024-05-08T08:55:11.189 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.189 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.189 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:11.189 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437604+0000", 2024-05-08T08:55:11.189 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078518278999999996 2024-05-08T08:55:11.189 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.189 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.189 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:11.189 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437704+0000", 2024-05-08T08:55:11.189 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010010599999999999 2024-05-08T08:55:11.189 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.189 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.189 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:11.189 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437870+0000", 2024-05-08T08:55:11.189 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00016641200000000001 2024-05-08T08:55:11.189 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.189 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:11.190 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.190 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.190 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.190 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28564 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888538944] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:11.190 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.359089+0000", 2024-05-08T08:55:11.190 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 98.770590073999998, 2024-05-08T08:55:11.190 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078952675, 2024-05-08T08:55:11.190 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:11.190 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:11.190 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:11.190 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:11.190 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:11.190 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:11.190 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28564 2024-05-08T08:55:11.191 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.193 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:11.193 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.193 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:11.193 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359089+0000", 2024-05-08T08:55:11.193 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:11.194 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.194 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.194 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:11.194 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359089+0000", 2024-05-08T08:55:11.194 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:11.194 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.194 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.194 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:11.194 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360959+0000", 2024-05-08T08:55:11.194 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0018707089999999999 2024-05-08T08:55:11.194 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.194 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.194 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:11.194 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360963+0000", 2024-05-08T08:55:11.195 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0680000000000001e-06 2024-05-08T08:55:11.195 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.195 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.195 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:11.195 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360969+0000", 2024-05-08T08:55:11.195 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1920000000000002e-06 2024-05-08T08:55:11.195 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.195 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.195 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:11.195 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360973+0000", 2024-05-08T08:55:11.195 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.031e-06 2024-05-08T08:55:11.195 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.195 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.195 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:11.195 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437916+0000", 2024-05-08T08:55:11.196 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.076942939000000002 2024-05-08T08:55:11.196 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.196 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.196 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:11.196 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437973+0000", 2024-05-08T08:55:11.196 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7358999999999997e-05 2024-05-08T08:55:11.196 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.196 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.196 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:11.196 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.438041+0000", 2024-05-08T08:55:11.196 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8376999999999999e-05 2024-05-08T08:55:11.196 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.196 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:11.196 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.196 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.196 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.197 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28574 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889427600] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:11.197 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:55:11.197 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 98.768585567000002, 2024-05-08T08:55:11.197 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078782476000000004, 2024-05-08T08:55:11.197 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:11.197 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:11.197 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:11.197 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:11.197 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:11.198 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:11.198 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28574 2024-05-08T08:55:11.198 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.198 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:11.198 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.198 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:11.198 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:55:11.198 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:11.198 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.198 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.198 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:11.198 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:55:11.199 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:11.199 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.199 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.199 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:11.199 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:55:11.199 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1799999999999999e-07 2024-05-08T08:55:11.199 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.199 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.199 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:11.199 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361095+0000", 2024-05-08T08:55:11.199 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3850000000000001e-06 2024-05-08T08:55:11.199 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.199 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.199 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:11.199 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361098+0000", 2024-05-08T08:55:11.200 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7989999999999998e-06 2024-05-08T08:55:11.200 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.200 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.200 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:11.200 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361100+0000", 2024-05-08T08:55:11.200 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4669999999999998e-06 2024-05-08T08:55:11.200 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.200 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.200 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:11.200 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439768+0000", 2024-05-08T08:55:11.200 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078668057999999999 2024-05-08T08:55:11.200 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.200 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.200 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:11.201 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439811+0000", 2024-05-08T08:55:11.201 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2459000000000001e-05 2024-05-08T08:55:11.201 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.201 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.201 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:11.201 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439876+0000", 2024-05-08T08:55:11.201 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5090000000000002e-05 2024-05-08T08:55:11.201 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.201 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:11.201 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.201 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.201 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.201 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28575 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889439216] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:11.201 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361103+0000", 2024-05-08T08:55:11.202 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 98.768575552000001, 2024-05-08T08:55:11.202 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078966828000000003, 2024-05-08T08:55:11.202 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:11.202 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:11.202 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:11.202 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:11.202 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:11.202 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:11.202 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28575 2024-05-08T08:55:11.203 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.203 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:11.203 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.203 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:11.203 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361103+0000", 2024-05-08T08:55:11.203 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:11.203 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.203 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.203 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:11.203 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361103+0000", 2024-05-08T08:55:11.203 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:11.203 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.203 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.203 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:11.204 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361104+0000", 2024-05-08T08:55:11.204 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1400000000000001e-07 2024-05-08T08:55:11.204 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.204 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.204 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:11.204 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361105+0000", 2024-05-08T08:55:11.204 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5540000000000001e-06 2024-05-08T08:55:11.204 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.204 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.204 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:11.204 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361108+0000", 2024-05-08T08:55:11.204 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.841e-06 2024-05-08T08:55:11.204 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.205 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.205 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:11.205 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361110+0000", 2024-05-08T08:55:11.205 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.497e-06 2024-05-08T08:55:11.205 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.205 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.205 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:11.205 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439916+0000", 2024-05-08T08:55:11.205 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078805372999999998 2024-05-08T08:55:11.205 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.205 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.205 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:11.205 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440000+0000", 2024-05-08T08:55:11.205 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4730999999999999e-05 2024-05-08T08:55:11.205 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.206 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.206 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:11.206 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440070+0000", 2024-05-08T08:55:11.206 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9617999999999997e-05 2024-05-08T08:55:11.206 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.206 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:11.206 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.206 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.206 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.206 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28576 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889451328] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:11.206 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361113+0000", 2024-05-08T08:55:11.206 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 98.768565366000004, 2024-05-08T08:55:11.206 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079088862999999995, 2024-05-08T08:55:11.207 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:11.207 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:11.207 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:11.207 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:11.207 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:11.207 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:11.207 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28576 2024-05-08T08:55:11.207 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.207 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:11.207 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.207 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:11.207 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361113+0000", 2024-05-08T08:55:11.207 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:11.207 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.207 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.207 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:11.208 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361113+0000", 2024-05-08T08:55:11.208 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:11.208 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.208 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.208 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:11.208 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361114+0000", 2024-05-08T08:55:11.208 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1299999999999999e-07 2024-05-08T08:55:11.208 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.208 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.208 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:11.208 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361115+0000", 2024-05-08T08:55:11.208 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.4300000000000001e-06 2024-05-08T08:55:11.208 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.208 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.208 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:11.208 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361118+0000", 2024-05-08T08:55:11.208 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9000000000000002e-06 2024-05-08T08:55:11.209 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.209 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.209 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:11.209 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361120+0000", 2024-05-08T08:55:11.209 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4739999999999999e-06 2024-05-08T08:55:11.209 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.209 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.209 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:11.209 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440096+0000", 2024-05-08T08:55:11.209 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078975629000000006 2024-05-08T08:55:11.209 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.209 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.209 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:11.209 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440135+0000", 2024-05-08T08:55:11.209 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9202000000000002e-05 2024-05-08T08:55:11.210 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.210 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.210 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:11.210 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440202+0000", 2024-05-08T08:55:11.210 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7014999999999994e-05 2024-05-08T08:55:11.210 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.210 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:11.210 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.210 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.210 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.210 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28577 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889462640] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:11.210 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:55:11.210 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 98.768555183999993, 2024-05-08T08:55:11.210 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079194622000000006, 2024-05-08T08:55:11.210 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:11.211 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:11.211 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:11.211 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:11.211 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:11.211 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:11.211 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28577 2024-05-08T08:55:11.211 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.211 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:11.211 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.211 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:11.211 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:55:11.211 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:11.211 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.211 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.211 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:11.211 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:55:11.212 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:11.212 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.212 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.212 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:11.212 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:55:11.212 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1400000000000001e-07 2024-05-08T08:55:11.212 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.212 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.212 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:11.212 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361125+0000", 2024-05-08T08:55:11.212 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.372e-06 2024-05-08T08:55:11.212 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.212 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.212 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:11.212 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361128+0000", 2024-05-08T08:55:11.212 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0309999999999999e-06 2024-05-08T08:55:11.213 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.213 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.213 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:11.213 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361131+0000", 2024-05-08T08:55:11.213 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4040000000000002e-06 2024-05-08T08:55:11.213 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.213 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.213 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:11.213 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440225+0000", 2024-05-08T08:55:11.213 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079094600000000001 2024-05-08T08:55:11.213 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.213 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.213 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:11.213 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440260+0000", 2024-05-08T08:55:11.213 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4703000000000003e-05 2024-05-08T08:55:11.214 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.214 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.214 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:11.214 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440318+0000", 2024-05-08T08:55:11.214 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8298000000000003e-05 2024-05-08T08:55:11.214 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.214 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:11.214 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.214 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.214 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.214 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28578 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889474192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:11.214 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361134+0000", 2024-05-08T08:55:11.214 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 98.768544758999994, 2024-05-08T08:55:11.214 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079310295000000003, 2024-05-08T08:55:11.214 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:11.215 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:11.215 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:11.215 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:11.215 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:11.215 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:11.215 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28578 2024-05-08T08:55:11.215 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.215 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:11.215 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.215 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:11.215 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361134+0000", 2024-05-08T08:55:11.215 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:11.215 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.215 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.215 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:11.215 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361134+0000", 2024-05-08T08:55:11.216 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:11.216 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.216 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.216 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:11.216 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.362958+0000", 2024-05-08T08:55:11.216 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0018237380000000001 2024-05-08T08:55:11.216 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.216 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.216 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:11.216 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.362961+0000", 2024-05-08T08:55:11.216 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.123e-06 2024-05-08T08:55:11.216 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.216 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.216 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:11.216 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.397404+0000", 2024-05-08T08:55:11.216 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.034443226 2024-05-08T08:55:11.217 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.217 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.217 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:11.217 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.397439+0000", 2024-05-08T08:55:11.217 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4690000000000002e-05 2024-05-08T08:55:11.217 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.217 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.217 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:11.217 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440340+0000", 2024-05-08T08:55:11.217 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.042901419000000003 2024-05-08T08:55:11.217 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.217 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.217 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:11.217 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440376+0000", 2024-05-08T08:55:11.217 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5444999999999997e-05 2024-05-08T08:55:11.217 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.217 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.218 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:11.218 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440444+0000", 2024-05-08T08:55:11.218 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8653999999999997e-05 2024-05-08T08:55:11.218 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.218 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:11.218 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.218 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.218 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:11.218 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:55:11.220 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:55:11.220 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:55:11.222+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:55:11.320 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:55:11.323+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:55:11.370 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:55:11.370 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:55:11.370 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:55:11.370 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:55:11.379 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:55:11.423 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:55:11.425+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:55:11.477 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-05-08T08:55:11.494 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:55:11.494 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:55:11.494 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:55:11.494 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:55:11.494 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:55:11.504 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:55:11.522 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:55:11.525+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:55:11.623 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:55:11.625+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:55:11.658 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:55:11.658 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:55:11.658 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:55:11.658 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:55:11.658 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.658 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:437 3.17 3:ea93a512:::rbd_data.1140100c9703.0000000000000077:head [set-alloc-hint object_size 2097152 write_size 2097152,write 1170432~1024] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e63)", 2024-05-08T08:55:11.658 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:08.390804+0000", 2024-05-08T08:55:11.658 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 123.268712893, 2024-05-08T08:55:11.658 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0406068690000001, 2024-05-08T08:55:11.659 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:11.659 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:11.659 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:55:11.659 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:11.659 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:11.659 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:11.659 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 437 2024-05-08T08:55:11.659 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.659 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:11.659 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.659 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:11.659 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390804+0000", 2024-05-08T08:55:11.659 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:11.659 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.659 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.660 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:11.660 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390804+0000", 2024-05-08T08:55:11.660 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:11.660 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.660 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.660 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:11.660 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390806+0000", 2024-05-08T08:55:11.660 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.666e-06 2024-05-08T08:55:11.660 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.660 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.660 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:11.660 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390818+0000", 2024-05-08T08:55:11.660 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2187999999999999e-05 2024-05-08T08:55:11.660 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.661 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.661 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:11.661 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390834+0000", 2024-05-08T08:55:11.661 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6191000000000001e-05 2024-05-08T08:55:11.661 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.661 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.661 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:11.661 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390845+0000", 2024-05-08T08:55:11.661 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0831e-05 2024-05-08T08:55:11.661 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.661 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.661 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:11.661 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390922+0000", 2024-05-08T08:55:11.661 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7253e-05 2024-05-08T08:55:11.661 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.662 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.662 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op must wait for map", 2024-05-08T08:55:11.662 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390930+0000", 2024-05-08T08:55:11.662 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8259999999999993e-06 2024-05-08T08:55:11.662 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.662 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.662 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:11.662 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392155+0000", 2024-05-08T08:55:11.662 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0012256649999999999 2024-05-08T08:55:11.662 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.662 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.662 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:55:11.662 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392165+0000", 2024-05-08T08:55:11.662 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5799999999999998e-06 2024-05-08T08:55:11.663 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.663 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.663 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:11.663 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.429596+0000", 2024-05-08T08:55:11.663 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0374313639999999 2024-05-08T08:55:11.663 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.663 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.663 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:11.663 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.429728+0000", 2024-05-08T08:55:11.663 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000131552 2024-05-08T08:55:11.663 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.663 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.663 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 0,1", 2024-05-08T08:55:11.663 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.429860+0000", 2024-05-08T08:55:11.663 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000132137 2024-05-08T08:55:11.664 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.664 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.664 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:55:11.664 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.430811+0000", 2024-05-08T08:55:11.664 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00095040200000000004 2024-05-08T08:55:11.664 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.664 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.664 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:55:11.664 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431126+0000", 2024-05-08T08:55:11.664 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000315006 2024-05-08T08:55:11.664 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.664 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.664 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:55:11.664 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431272+0000", 2024-05-08T08:55:11.664 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000146487 2024-05-08T08:55:11.664 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.665 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.665 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:55:11.665 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431315+0000", 2024-05-08T08:55:11.665 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2697000000000001e-05 2024-05-08T08:55:11.665 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.665 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.665 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:11.665 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431411+0000", 2024-05-08T08:55:11.665 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6024000000000001e-05 2024-05-08T08:55:11.665 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.665 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:11.665 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.665 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.665 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.665 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:439 3.17 3:ea93a512:::rbd_data.1140100c9703.0000000000000077:head [read 1163264~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-05-08T08:55:11.666 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:08.390871+0000", 2024-05-08T08:55:11.666 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 123.268645382, 2024-05-08T08:55:11.666 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0408790590000001, 2024-05-08T08:55:11.666 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:11.666 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:11.666 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:11.666 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:11.666 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:11.666 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:11.666 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 439 2024-05-08T08:55:11.666 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.666 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:11.666 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.666 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:11.667 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390871+0000", 2024-05-08T08:55:11.667 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:11.667 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.667 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.667 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:11.667 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390871+0000", 2024-05-08T08:55:11.667 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:11.667 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.667 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.667 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:11.667 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390872+0000", 2024-05-08T08:55:11.667 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6300000000000005e-07 2024-05-08T08:55:11.667 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.667 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.667 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:11.668 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390875+0000", 2024-05-08T08:55:11.668 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3459999999999998e-06 2024-05-08T08:55:11.668 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.668 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.668 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:11.668 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390891+0000", 2024-05-08T08:55:11.668 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5255e-05 2024-05-08T08:55:11.668 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.668 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.668 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:11.668 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390897+0000", 2024-05-08T08:55:11.668 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2360000000000001e-06 2024-05-08T08:55:11.668 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.668 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.668 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:11.668 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390995+0000", 2024-05-08T08:55:11.669 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8522000000000003e-05 2024-05-08T08:55:11.669 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.669 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.669 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting_for_map not empty", 2024-05-08T08:55:11.669 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.391005+0000", 2024-05-08T08:55:11.669 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1460000000000002e-06 2024-05-08T08:55:11.669 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.669 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.669 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:11.669 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392191+0000", 2024-05-08T08:55:11.669 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0011860270000000001 2024-05-08T08:55:11.669 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.669 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.669 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:55:11.669 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392197+0000", 2024-05-08T08:55:11.670 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4289999999999998e-06 2024-05-08T08:55:11.670 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.670 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.670 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:11.670 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.430596+0000", 2024-05-08T08:55:11.670 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0383989090000001 2024-05-08T08:55:11.670 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.670 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.670 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for rw locks", 2024-05-08T08:55:11.670 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.430676+0000", 2024-05-08T08:55:11.670 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9781000000000001e-05 2024-05-08T08:55:11.670 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.670 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.670 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:11.670 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431460+0000", 2024-05-08T08:55:11.671 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00078427300000000004 2024-05-08T08:55:11.671 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.671 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.671 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:11.671 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431540+0000", 2024-05-08T08:55:11.671 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0224999999999998e-05 2024-05-08T08:55:11.671 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.671 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.671 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:11.671 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431750+0000", 2024-05-08T08:55:11.671 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00021024699999999999 2024-05-08T08:55:11.671 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.671 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:11.671 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.672 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.672 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.672 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:965 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 950272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:11.672 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451075+0000", 2024-05-08T08:55:11.672 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 122.20844158, 2024-05-08T08:55:11.672 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087230196999999995, 2024-05-08T08:55:11.672 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:11.672 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:11.672 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:11.672 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:11.672 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:11.672 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:11.672 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 965 2024-05-08T08:55:11.673 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.673 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:11.673 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.673 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:11.673 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451075+0000", 2024-05-08T08:55:11.673 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:11.673 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.673 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.673 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:11.673 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451075+0000", 2024-05-08T08:55:11.673 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:11.673 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.673 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.673 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:11.673 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451076+0000", 2024-05-08T08:55:11.674 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5000000000000002e-07 2024-05-08T08:55:11.674 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.674 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.674 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:11.674 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451078+0000", 2024-05-08T08:55:11.674 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.5639999999999999e-06 2024-05-08T08:55:11.674 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.674 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.674 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:11.674 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451085+0000", 2024-05-08T08:55:11.674 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5640000000000002e-06 2024-05-08T08:55:11.674 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.674 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.674 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:11.675 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451090+0000", 2024-05-08T08:55:11.675 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1699999999999996e-06 2024-05-08T08:55:11.675 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.675 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.675 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:11.675 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538106+0000", 2024-05-08T08:55:11.675 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087015888999999999 2024-05-08T08:55:11.675 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.675 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.675 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:11.675 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538190+0000", 2024-05-08T08:55:11.675 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3961999999999993e-05 2024-05-08T08:55:11.675 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.675 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.675 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:11.676 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538305+0000", 2024-05-08T08:55:11.676 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011529800000000001 2024-05-08T08:55:11.676 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.676 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:11.676 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.676 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.676 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.676 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:966 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 1441792~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:11.676 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:55:11.676 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 122.20841820699999, 2024-05-08T08:55:11.676 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087446323000000006, 2024-05-08T08:55:11.676 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:11.676 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:11.676 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:11.677 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:11.677 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:11.677 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:11.677 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 966 2024-05-08T08:55:11.677 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.677 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:11.677 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.677 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:11.677 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:55:11.677 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:11.677 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.677 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.677 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:11.677 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:55:11.677 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:11.677 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.678 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.678 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:11.678 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:55:11.678 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3700000000000001e-07 2024-05-08T08:55:11.678 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.678 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.678 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:11.678 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451101+0000", 2024-05-08T08:55:11.678 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2560000000000001e-06 2024-05-08T08:55:11.678 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.678 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.678 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:11.678 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451107+0000", 2024-05-08T08:55:11.678 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1070000000000001e-06 2024-05-08T08:55:11.679 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.679 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.679 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:11.679 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451112+0000", 2024-05-08T08:55:11.679 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1420000000000001e-06 2024-05-08T08:55:11.679 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.679 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.679 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:11.679 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538341+0000", 2024-05-08T08:55:11.679 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087228302999999993 2024-05-08T08:55:11.679 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.679 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.679 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:11.679 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538424+0000", 2024-05-08T08:55:11.680 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3497000000000004e-05 2024-05-08T08:55:11.680 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.680 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.680 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:11.680 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538545+0000", 2024-05-08T08:55:11.680 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000120581 2024-05-08T08:55:11.680 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.680 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:11.680 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.680 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.680 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.680 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:967 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:11.680 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451120+0000", 2024-05-08T08:55:11.680 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 122.208396732, 2024-05-08T08:55:11.681 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087632725999999994, 2024-05-08T08:55:11.681 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:11.681 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:11.681 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:11.681 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:11.681 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:11.681 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:11.681 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 967 2024-05-08T08:55:11.681 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.681 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:11.681 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.681 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:11.681 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451120+0000", 2024-05-08T08:55:11.681 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:11.681 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.682 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.682 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:11.682 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451120+0000", 2024-05-08T08:55:11.682 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:11.682 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.682 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.682 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:11.682 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451121+0000", 2024-05-08T08:55:11.682 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8800000000000002e-07 2024-05-08T08:55:11.682 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.682 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.682 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:11.682 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451130+0000", 2024-05-08T08:55:11.682 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2299999999999997e-06 2024-05-08T08:55:11.683 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.683 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.683 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:11.683 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451137+0000", 2024-05-08T08:55:11.683 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4669999999999998e-06 2024-05-08T08:55:11.683 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.683 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.683 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:11.683 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451143+0000", 2024-05-08T08:55:11.683 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2839999999999997e-06 2024-05-08T08:55:11.683 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.683 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.683 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:11.683 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538575+0000", 2024-05-08T08:55:11.683 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087432789999999996 2024-05-08T08:55:11.684 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.684 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.684 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:11.684 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538648+0000", 2024-05-08T08:55:11.684 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2877e-05 2024-05-08T08:55:11.684 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.684 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.684 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:11.684 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538753+0000", 2024-05-08T08:55:11.684 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010449 2024-05-08T08:55:11.684 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.684 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:11.684 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.684 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.684 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.685 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:968 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 2424832~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:11.685 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451152+0000", 2024-05-08T08:55:11.685 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 122.208364647, 2024-05-08T08:55:11.685 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087813325999999997, 2024-05-08T08:55:11.685 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:11.685 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:11.685 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:11.685 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:11.685 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:11.685 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:11.685 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 968 2024-05-08T08:55:11.685 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.685 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:11.685 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.685 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:11.686 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451152+0000", 2024-05-08T08:55:11.686 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:11.686 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.686 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.686 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:11.686 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451152+0000", 2024-05-08T08:55:11.686 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:11.686 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.686 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.686 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:11.686 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451153+0000", 2024-05-08T08:55:11.686 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3099999999999998e-07 2024-05-08T08:55:11.686 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.686 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.687 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:11.687 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451155+0000", 2024-05-08T08:55:11.687 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4760000000000001e-06 2024-05-08T08:55:11.687 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.687 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.687 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:11.687 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451162+0000", 2024-05-08T08:55:11.687 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5969999999999997e-06 2024-05-08T08:55:11.687 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.687 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.687 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:11.687 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451167+0000", 2024-05-08T08:55:11.687 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.977e-06 2024-05-08T08:55:11.687 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.687 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.688 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:11.688 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538781+0000", 2024-05-08T08:55:11.688 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087614846999999996 2024-05-08T08:55:11.688 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.688 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.688 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:11.688 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538857+0000", 2024-05-08T08:55:11.688 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5358000000000006e-05 2024-05-08T08:55:11.688 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.688 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.688 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:11.688 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538965+0000", 2024-05-08T08:55:11.688 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010864 2024-05-08T08:55:11.688 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.688 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:11.689 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.689 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.689 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.689 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:969 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 2916352~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:11.689 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:55:11.689 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 122.208342942, 2024-05-08T08:55:11.689 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087998980000000004, 2024-05-08T08:55:11.689 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:11.689 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:11.689 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:11.689 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:11.689 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:11.689 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:11.689 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 969 2024-05-08T08:55:11.689 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.690 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:11.690 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.690 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:11.690 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:55:11.690 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:11.690 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.690 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.690 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:11.690 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:55:11.690 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:11.690 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.690 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.690 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:11.690 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:55:11.690 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3599999999999999e-07 2024-05-08T08:55:11.691 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.691 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.691 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:11.691 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451177+0000", 2024-05-08T08:55:11.691 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4590000000000001e-06 2024-05-08T08:55:11.691 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.691 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.691 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:11.691 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451184+0000", 2024-05-08T08:55:11.691 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4109999999999999e-06 2024-05-08T08:55:11.691 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.691 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.691 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:11.691 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451191+0000", 2024-05-08T08:55:11.691 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7279999999999998e-06 2024-05-08T08:55:11.692 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.692 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.692 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:11.692 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538997+0000", 2024-05-08T08:55:11.692 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087806170000000003 2024-05-08T08:55:11.692 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.692 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.692 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:11.692 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539069+0000", 2024-05-08T08:55:11.692 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2373999999999995e-05 2024-05-08T08:55:11.692 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.692 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.692 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:11.692 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539173+0000", 2024-05-08T08:55:11.693 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000103402 2024-05-08T08:55:11.693 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.693 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:11.693 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.693 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.693 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.693 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:970 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 3407872~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:11.693 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451198+0000", 2024-05-08T08:55:11.693 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 122.208318345, 2024-05-08T08:55:11.693 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088183368999999998, 2024-05-08T08:55:11.693 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:11.693 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:11.693 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:11.693 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:11.693 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:11.693 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:11.694 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 970 2024-05-08T08:55:11.694 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.694 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:11.694 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.694 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:11.694 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451198+0000", 2024-05-08T08:55:11.694 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:11.694 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.694 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.694 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:11.694 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451198+0000", 2024-05-08T08:55:11.694 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:11.694 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.694 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.694 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:11.695 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451199+0000", 2024-05-08T08:55:11.695 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7400000000000003e-07 2024-05-08T08:55:11.695 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.695 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.695 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:11.695 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451211+0000", 2024-05-08T08:55:11.695 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2109999999999999e-05 2024-05-08T08:55:11.695 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.695 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.695 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:11.695 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451219+0000", 2024-05-08T08:55:11.695 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4170000000000003e-06 2024-05-08T08:55:11.695 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.695 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.696 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:11.696 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451224+0000", 2024-05-08T08:55:11.696 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3990000000000003e-06 2024-05-08T08:55:11.696 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.696 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.696 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:11.696 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539204+0000", 2024-05-08T08:55:11.696 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087980099000000006 2024-05-08T08:55:11.696 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.696 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.696 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:11.696 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539276+0000", 2024-05-08T08:55:11.696 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1513000000000005e-05 2024-05-08T08:55:11.696 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.696 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.697 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:11.697 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539382+0000", 2024-05-08T08:55:11.697 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010615699999999999 2024-05-08T08:55:11.697 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.697 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:11.697 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.697 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.697 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.697 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:971 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 3899392~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:11.697 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451233+0000", 2024-05-08T08:55:11.697 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 122.208283638, 2024-05-08T08:55:11.697 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088431525999999996, 2024-05-08T08:55:11.697 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:11.698 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:11.698 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:11.698 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:11.698 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:11.698 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:11.698 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 971 2024-05-08T08:55:11.698 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.698 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:11.698 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.698 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:11.698 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451233+0000", 2024-05-08T08:55:11.698 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:11.698 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.698 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.699 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:11.699 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451233+0000", 2024-05-08T08:55:11.699 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:11.699 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.699 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.699 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:11.699 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451234+0000", 2024-05-08T08:55:11.699 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8500000000000001e-07 2024-05-08T08:55:11.699 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.699 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.699 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:11.699 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451237+0000", 2024-05-08T08:55:11.699 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8320000000000002e-06 2024-05-08T08:55:11.699 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.700 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.700 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:11.700 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451243+0000", 2024-05-08T08:55:11.700 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6749999999999996e-06 2024-05-08T08:55:11.700 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.700 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.700 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:11.700 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451249+0000", 2024-05-08T08:55:11.700 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1229999999999997e-06 2024-05-08T08:55:11.700 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.700 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.700 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:11.700 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539412+0000", 2024-05-08T08:55:11.700 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088162668999999999 2024-05-08T08:55:11.700 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.701 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.701 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:11.701 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539505+0000", 2024-05-08T08:55:11.701 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3240999999999997e-05 2024-05-08T08:55:11.701 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.701 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.701 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:11.701 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539665+0000", 2024-05-08T08:55:11.701 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00015930100000000001 2024-05-08T08:55:11.701 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.701 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:11.701 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.701 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.701 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.701 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:972 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 196608~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:11.701 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451257+0000", 2024-05-08T08:55:11.702 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 122.208259693, 2024-05-08T08:55:11.702 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088719928000000003, 2024-05-08T08:55:11.702 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:11.702 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:11.702 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:11.702 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:11.702 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:11.702 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:11.702 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 972 2024-05-08T08:55:11.702 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.702 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:11.702 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.702 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:11.702 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451257+0000", 2024-05-08T08:55:11.703 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:11.703 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.703 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.703 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:11.703 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451257+0000", 2024-05-08T08:55:11.703 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:11.703 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.703 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.703 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:11.703 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451258+0000", 2024-05-08T08:55:11.703 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7999999999999996e-07 2024-05-08T08:55:11.703 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.703 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.703 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:11.703 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451260+0000", 2024-05-08T08:55:11.704 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4609999999999998e-06 2024-05-08T08:55:11.704 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.704 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.704 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:11.704 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451267+0000", 2024-05-08T08:55:11.704 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.81e-06 2024-05-08T08:55:11.704 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.704 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.704 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:11.704 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451273+0000", 2024-05-08T08:55:11.704 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0859999999999998e-06 2024-05-08T08:55:11.704 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.704 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.704 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:11.705 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539718+0000", 2024-05-08T08:55:11.705 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088445059000000006 2024-05-08T08:55:11.705 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.705 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.705 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:11.705 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539834+0000", 2024-05-08T08:55:11.705 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000116426 2024-05-08T08:55:11.705 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.705 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.705 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:11.705 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539977+0000", 2024-05-08T08:55:11.705 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014260600000000001 2024-05-08T08:55:11.705 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.705 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:11.705 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.706 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.706 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.706 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:973 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 688128~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:11.706 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:55:11.706 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 122.208237109, 2024-05-08T08:55:11.706 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088918496, 2024-05-08T08:55:11.706 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:11.706 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:11.706 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:11.706 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:11.706 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:11.706 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:11.706 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 973 2024-05-08T08:55:11.707 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.707 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:11.707 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.707 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:11.707 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:55:11.707 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:11.707 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.707 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.707 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:11.707 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:55:11.707 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:11.707 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.707 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.707 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:11.707 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:55:11.708 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9099999999999999e-07 2024-05-08T08:55:11.708 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.708 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.708 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:11.708 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451282+0000", 2024-05-08T08:55:11.708 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1550000000000001e-06 2024-05-08T08:55:11.708 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.708 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.708 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:11.708 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451288+0000", 2024-05-08T08:55:11.708 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8889999999999999e-06 2024-05-08T08:55:11.708 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.708 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.708 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:11.708 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451294+0000", 2024-05-08T08:55:11.708 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0850000000000002e-06 2024-05-08T08:55:11.709 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.709 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.709 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:11.709 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540011+0000", 2024-05-08T08:55:11.709 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088717314000000005 2024-05-08T08:55:11.709 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.709 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.709 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:11.709 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540089+0000", 2024-05-08T08:55:11.709 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7438e-05 2024-05-08T08:55:11.709 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.709 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.709 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:11.709 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540198+0000", 2024-05-08T08:55:11.709 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000109224 2024-05-08T08:55:11.710 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.710 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:11.710 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.710 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.710 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.710 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:974 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 1179648~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:11.710 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451301+0000", 2024-05-08T08:55:11.710 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 122.20821547600001, 2024-05-08T08:55:11.710 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089105078000000004, 2024-05-08T08:55:11.710 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:11.710 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:11.710 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:11.710 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:11.710 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:11.711 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:11.711 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 974 2024-05-08T08:55:11.711 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.711 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:11.711 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.711 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:11.711 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451301+0000", 2024-05-08T08:55:11.711 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:11.711 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.711 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.711 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:11.711 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451301+0000", 2024-05-08T08:55:11.711 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:11.711 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.712 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.712 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:11.712 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451302+0000", 2024-05-08T08:55:11.712 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7800000000000002e-07 2024-05-08T08:55:11.712 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.712 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.712 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:11.712 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451304+0000", 2024-05-08T08:55:11.712 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.65e-06 2024-05-08T08:55:11.712 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.712 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.712 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:11.712 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451311+0000", 2024-05-08T08:55:11.712 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.472e-06 2024-05-08T08:55:11.712 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.713 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.713 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:11.713 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451316+0000", 2024-05-08T08:55:11.713 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9189999999999998e-06 2024-05-08T08:55:11.713 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.713 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.713 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:11.713 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540234+0000", 2024-05-08T08:55:11.713 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088917752000000003 2024-05-08T08:55:11.713 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.713 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.713 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:11.714 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540308+0000", 2024-05-08T08:55:11.714 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4758000000000005e-05 2024-05-08T08:55:11.714 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.714 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.714 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:11.714 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540406+0000", 2024-05-08T08:55:11.714 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8048999999999996e-05 2024-05-08T08:55:11.714 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.714 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:11.714 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.714 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.714 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.714 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:975 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 1671168~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:11.714 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:55:11.715 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 122.20817314599999, 2024-05-08T08:55:11.715 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089294139999999994, 2024-05-08T08:55:11.715 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:11.715 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:11.715 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:11.715 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:11.715 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:11.715 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:11.715 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 975 2024-05-08T08:55:11.715 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.715 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:11.715 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.715 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:11.715 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:55:11.715 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:11.716 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.716 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.716 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:11.716 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:55:11.716 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:11.716 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.716 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.716 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:11.716 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:55:11.716 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1299999999999999e-07 2024-05-08T08:55:11.716 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.716 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.716 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:11.716 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451348+0000", 2024-05-08T08:55:11.716 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.484e-06 2024-05-08T08:55:11.717 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.717 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.717 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:11.717 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451356+0000", 2024-05-08T08:55:11.717 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8499999999999994e-06 2024-05-08T08:55:11.717 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.717 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.717 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:11.717 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451366+0000", 2024-05-08T08:55:11.717 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.9850000000000001e-06 2024-05-08T08:55:11.717 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.717 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.717 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:11.717 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540448+0000", 2024-05-08T08:55:11.717 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089082254 2024-05-08T08:55:11.718 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.718 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.718 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:11.718 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540528+0000", 2024-05-08T08:55:11.718 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0234999999999993e-05 2024-05-08T08:55:11.718 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.718 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.718 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:11.718 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540638+0000", 2024-05-08T08:55:11.718 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010951900000000001 2024-05-08T08:55:11.718 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.718 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:11.718 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.718 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.718 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.719 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:976 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 2162688~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:11.719 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451375+0000", 2024-05-08T08:55:11.719 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 122.208141399, 2024-05-08T08:55:11.719 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089472745000000006, 2024-05-08T08:55:11.719 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:11.719 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:11.719 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:11.719 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:11.719 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:11.719 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:11.719 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 976 2024-05-08T08:55:11.719 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.719 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:11.719 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.719 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:11.720 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451375+0000", 2024-05-08T08:55:11.720 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:11.720 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.720 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.720 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:11.720 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451375+0000", 2024-05-08T08:55:11.720 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:11.720 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.720 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.720 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:11.720 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451376+0000", 2024-05-08T08:55:11.720 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2399999999999998e-07 2024-05-08T08:55:11.720 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.720 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.720 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:11.721 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451379+0000", 2024-05-08T08:55:11.721 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6359999999999998e-06 2024-05-08T08:55:11.721 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.721 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.721 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:11.721 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451387+0000", 2024-05-08T08:55:11.721 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9929999999999995e-06 2024-05-08T08:55:11.721 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.721 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.721 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:11.721 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451393+0000", 2024-05-08T08:55:11.721 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4699999999999999e-06 2024-05-08T08:55:11.721 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.721 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.721 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:11.722 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540677+0000", 2024-05-08T08:55:11.722 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089283504999999999 2024-05-08T08:55:11.722 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.722 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.722 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:11.722 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540743+0000", 2024-05-08T08:55:11.722 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6303999999999994e-05 2024-05-08T08:55:11.722 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.722 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.722 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:11.722 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540848+0000", 2024-05-08T08:55:11.722 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000105213 2024-05-08T08:55:11.722 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.722 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:11.722 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.723 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.723 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.723 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:977 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 2654208~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:11.723 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:55:11.723 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 122.208115967, 2024-05-08T08:55:11.723 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089698473000000001, 2024-05-08T08:55:11.723 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:11.723 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:11.723 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:11.723 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:11.723 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:11.723 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:11.723 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 977 2024-05-08T08:55:11.723 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.723 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:11.724 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.724 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:11.724 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:55:11.724 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:11.724 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.724 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.724 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:11.724 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:55:11.724 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:11.724 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.724 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.724 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:11.724 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:55:11.724 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2899999999999999e-07 2024-05-08T08:55:11.725 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.725 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.725 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:11.725 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451404+0000", 2024-05-08T08:55:11.725 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6649999999999999e-06 2024-05-08T08:55:11.725 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.725 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.725 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:11.725 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451410+0000", 2024-05-08T08:55:11.725 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3269999999999997e-06 2024-05-08T08:55:11.725 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.725 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.725 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:11.725 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451415+0000", 2024-05-08T08:55:11.726 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9529999999999998e-06 2024-05-08T08:55:11.726 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.726 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.726 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:11.726 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540912+0000", 2024-05-08T08:55:11.726 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089496950000000006 2024-05-08T08:55:11.726 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.726 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.726 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:11.726 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541000+0000", 2024-05-08T08:55:11.726 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7429000000000006e-05 2024-05-08T08:55:11.726 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.726 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.726 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:11.727 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541099+0000", 2024-05-08T08:55:11.727 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.9720000000000001e-05 2024-05-08T08:55:11.727 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.727 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:11.727 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.727 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.727 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.727 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:978 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 3145728~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:11.727 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:55:11.727 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 122.20809475900001, 2024-05-08T08:55:11.727 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089900635000000007, 2024-05-08T08:55:11.727 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:11.727 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:11.727 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:11.727 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:11.728 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:11.728 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:11.728 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 978 2024-05-08T08:55:11.728 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.728 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:11.728 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.728 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:11.728 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:55:11.728 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:11.728 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.728 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.728 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:11.728 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:55:11.728 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:11.728 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.729 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.729 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:11.729 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:55:11.729 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9400000000000001e-07 2024-05-08T08:55:11.729 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.729 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.729 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:11.729 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451425+0000", 2024-05-08T08:55:11.729 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2460000000000002e-06 2024-05-08T08:55:11.729 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.729 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.729 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:11.729 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451431+0000", 2024-05-08T08:55:11.729 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2419999999999997e-06 2024-05-08T08:55:11.729 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.730 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.730 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:11.730 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451436+0000", 2024-05-08T08:55:11.730 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.887e-06 2024-05-08T08:55:11.730 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.730 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.730 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:11.730 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541152+0000", 2024-05-08T08:55:11.730 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089716194999999999 2024-05-08T08:55:11.730 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.730 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.730 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:11.730 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541224+0000", 2024-05-08T08:55:11.730 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2416000000000006e-05 2024-05-08T08:55:11.731 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.731 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.731 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:11.731 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541323+0000", 2024-05-08T08:55:11.731 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8255e-05 2024-05-08T08:55:11.731 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.731 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:11.731 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.731 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.731 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.731 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:979 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 3637248~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:11.731 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451462+0000", 2024-05-08T08:55:11.731 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 122.20805430999999, 2024-05-08T08:55:11.731 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.090036441999999994, 2024-05-08T08:55:11.731 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:11.732 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:11.732 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:11.732 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:11.732 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:11.732 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:11.732 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 979 2024-05-08T08:55:11.732 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.732 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:11.732 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.732 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:11.732 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451462+0000", 2024-05-08T08:55:11.732 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:11.732 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.732 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.732 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:11.733 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451462+0000", 2024-05-08T08:55:11.733 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:11.733 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.733 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.733 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:11.733 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451463+0000", 2024-05-08T08:55:11.733 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4300000000000002e-07 2024-05-08T08:55:11.733 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.733 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.733 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:11.733 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451471+0000", 2024-05-08T08:55:11.733 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0800000000000006e-06 2024-05-08T08:55:11.733 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.733 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.733 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:11.734 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451481+0000", 2024-05-08T08:55:11.734 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.38e-06 2024-05-08T08:55:11.734 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.734 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.734 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:11.734 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451487+0000", 2024-05-08T08:55:11.734 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4590000000000003e-06 2024-05-08T08:55:11.734 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.734 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.734 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:11.734 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541350+0000", 2024-05-08T08:55:11.734 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089862599000000001 2024-05-08T08:55:11.734 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.734 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.735 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:11.735 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541407+0000", 2024-05-08T08:55:11.735 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7667999999999997e-05 2024-05-08T08:55:11.735 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.735 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.735 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:11.735 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541499+0000", 2024-05-08T08:55:11.735 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1513000000000003e-05 2024-05-08T08:55:11.735 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.735 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:11.735 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.735 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.735 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.735 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:980 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 4128768~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:11.735 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451496+0000", 2024-05-08T08:55:11.736 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 122.208020554, 2024-05-08T08:55:11.736 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.090258985, 2024-05-08T08:55:11.736 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:11.736 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:11.736 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:11.736 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:11.736 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:11.736 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:11.736 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 980 2024-05-08T08:55:11.736 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.736 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:11.736 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.736 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:11.736 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451496+0000", 2024-05-08T08:55:11.736 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:11.737 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.737 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.737 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:11.737 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451496+0000", 2024-05-08T08:55:11.737 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:11.737 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.737 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.737 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:11.737 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451497+0000", 2024-05-08T08:55:11.737 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7599999999999997e-07 2024-05-08T08:55:11.737 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.737 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.737 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:11.737 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451500+0000", 2024-05-08T08:55:11.738 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7930000000000002e-06 2024-05-08T08:55:11.738 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.738 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.738 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:11.738 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451506+0000", 2024-05-08T08:55:11.738 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6980000000000001e-06 2024-05-08T08:55:11.738 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.738 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.738 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:11.738 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451512+0000", 2024-05-08T08:55:11.738 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2490000000000001e-06 2024-05-08T08:55:11.738 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.738 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.738 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:11.739 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541550+0000", 2024-05-08T08:55:11.739 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.090037961999999999 2024-05-08T08:55:11.739 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.739 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.739 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:11.739 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541647+0000", 2024-05-08T08:55:11.739 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6924999999999998e-05 2024-05-08T08:55:11.739 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.739 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.739 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:11.739 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541755+0000", 2024-05-08T08:55:11.739 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000108682 2024-05-08T08:55:11.739 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.739 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:11.739 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.740 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.740 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.740 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:33397 3.7 3:e5a023c9:::rbd_header.11402419351a:head [watch reconnect cookie 140139347723488 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e156)", 2024-05-08T08:55:11.740 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:55:07.435461+0000", 2024-05-08T08:55:11.740 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 4.2240552549999997, 2024-05-08T08:55:11.740 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.009753092, 2024-05-08T08:55:11.740 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:11.740 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:11.740 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:11.740 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:11.740 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:55:11.740 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:55:11.740 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 33397 2024-05-08T08:55:11.740 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.741 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:11.741 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.741 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:11.741 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:07.435461+0000", 2024-05-08T08:55:11.741 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:11.741 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.741 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.741 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:11.741 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:07.435461+0000", 2024-05-08T08:55:11.741 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:11.741 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.741 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.741 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:11.741 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:07.435463+0000", 2024-05-08T08:55:11.741 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.7749999999999999e-06 2024-05-08T08:55:11.742 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.742 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.742 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:11.742 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:07.435471+0000", 2024-05-08T08:55:11.742 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.2199999999999992e-06 2024-05-08T08:55:11.742 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.742 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.742 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:11.742 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:07.435487+0000", 2024-05-08T08:55:11.742 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5806000000000001e-05 2024-05-08T08:55:11.742 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.742 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.742 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:11.742 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:07.435867+0000", 2024-05-08T08:55:11.743 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00037973999999999999 2024-05-08T08:55:11.743 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.743 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.743 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:11.743 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:07.436510+0000", 2024-05-08T08:55:11.743 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00064263299999999998 2024-05-08T08:55:11.743 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.743 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.743 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:55:11.743 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:07.436524+0000", 2024-05-08T08:55:11.743 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.4316e-05 2024-05-08T08:55:11.743 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.743 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.743 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:11.743 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:08.439082+0000", 2024-05-08T08:55:11.744 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.002558235 2024-05-08T08:55:11.744 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.744 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.744 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:11.744 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:08.439286+0000", 2024-05-08T08:55:11.744 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00020426600000000001 2024-05-08T08:55:11.744 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.744 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.744 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:11.744 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:08.445215+0000", 2024-05-08T08:55:11.744 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0059281009999999999 2024-05-08T08:55:11.744 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.744 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:11.744 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.745 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.745 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.745 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:33398 3.7 3.93c405a7 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e156)", 2024-05-08T08:55:11.745 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:55:07.435912+0000", 2024-05-08T08:55:11.745 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 4.2236050570000003, 2024-05-08T08:55:11.745 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0044546430000001, 2024-05-08T08:55:11.745 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:11.745 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:11.745 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "reached pg", 2024-05-08T08:55:11.745 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:11.745 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:55:11.745 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:55:11.745 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 33398 2024-05-08T08:55:11.745 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.746 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:11.746 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.746 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:11.746 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:07.435912+0000", 2024-05-08T08:55:11.746 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:11.746 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.746 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.746 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:11.746 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:07.435912+0000", 2024-05-08T08:55:11.746 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:11.746 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.746 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.746 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:11.746 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:07.435913+0000", 2024-05-08T08:55:11.747 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2200000000000002e-07 2024-05-08T08:55:11.747 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.747 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.747 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:11.747 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:07.435919+0000", 2024-05-08T08:55:11.747 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3820000000000001e-06 2024-05-08T08:55:11.747 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.747 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.747 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:11.747 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:07.435932+0000", 2024-05-08T08:55:11.747 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2788e-05 2024-05-08T08:55:11.747 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.747 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.747 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:11.747 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:07.435940+0000", 2024-05-08T08:55:11.748 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.6710000000000005e-06 2024-05-08T08:55:11.748 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.748 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.748 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:11.748 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:07.436558+0000", 2024-05-08T08:55:11.748 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00061782299999999998 2024-05-08T08:55:11.748 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.748 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.748 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:55:11.748 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:07.436572+0000", 2024-05-08T08:55:11.748 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.34e-05 2024-05-08T08:55:11.748 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.748 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.748 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:11.748 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:08.440285+0000", 2024-05-08T08:55:11.749 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.003713388 2024-05-08T08:55:11.749 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:11.749 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:11.749 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:11.749 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:08.440366+0000", 2024-05-08T08:55:11.749 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1268999999999997e-05 2024-05-08T08:55:11.749 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.749 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:11.749 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.749 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:11.749 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:11.749 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:55:11.751 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:55:11.751 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:55:11.753+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:55:11.851 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:55:11.854+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:55:11.869 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:55:11.869 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:55:11.869 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:55:11.869 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:55:11.879 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:55:11.962 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:55:11.965+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:55:12.025 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:55:12.025 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:55:12.025 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:55:12.025 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:55:12.025 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:55:12.033 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:55:12.074 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:55:12.077+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:55:12.154 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:55:12.156+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:55:12.180 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:55:12.180 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:55:12.180 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:55:12.180 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:55:12.180 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.180 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4621.0:9089 3.1f e33/18 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:head v 33'117, mlcod=31'115)", 2024-05-08T08:55:12.181 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:55:12.181 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 159.662344964, 2024-05-08T08:55:12.181 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.111244524, 2024-05-08T08:55:12.181 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:12.181 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:12.181 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:55:12.181 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:12.181 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.181 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:12.181 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:55:12.181 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:12.181 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.181 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.181 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:12.182 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:55:12.182 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:12.182 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.182 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.182 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:12.182 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:55:12.182 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5900000000000001e-07 2024-05-08T08:55:12.182 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.182 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.182 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:12.182 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539600+0000", 2024-05-08T08:55:12.182 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.019981364000000001 2024-05-08T08:55:12.182 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.182 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.183 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:12.183 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539624+0000", 2024-05-08T08:55:12.183 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4043e-05 2024-05-08T08:55:12.183 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.183 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.183 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:12.183 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539635+0000", 2024-05-08T08:55:12.183 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1511e-05 2024-05-08T08:55:12.183 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.183 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.183 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:12.183 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539728+0000", 2024-05-08T08:55:12.183 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2498999999999996e-05 2024-05-08T08:55:12.183 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.183 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.184 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:12.184 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539746+0000", 2024-05-08T08:55:12.184 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8289000000000001e-05 2024-05-08T08:55:12.184 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.184 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.184 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:55:12.184 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.630822+0000", 2024-05-08T08:55:12.184 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.091076283999999993 2024-05-08T08:55:12.184 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.184 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.184 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:12.184 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.630863+0000", 2024-05-08T08:55:12.184 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0374999999999999e-05 2024-05-08T08:55:12.184 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.185 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:12.185 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.185 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.185 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.185 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4195.0:4 3.11 3:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-05-08T08:55:12.185 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:49.764654+0000", 2024-05-08T08:55:12.185 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 142.41730942300001, 2024-05-08T08:55:12.185 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.30714275499999999, 2024-05-08T08:55:12.185 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:12.185 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:12.185 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-05-08T08:55:12.185 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:12.185 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4195", 2024-05-08T08:55:12.185 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/59179521", 2024-05-08T08:55:12.186 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4 2024-05-08T08:55:12.186 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.186 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:12.186 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.186 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:12.186 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764654+0000", 2024-05-08T08:55:12.186 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:12.186 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.186 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.186 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:12.186 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764654+0000", 2024-05-08T08:55:12.186 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:12.186 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.186 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.187 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:12.187 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764655+0000", 2024-05-08T08:55:12.187 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.452e-06 2024-05-08T08:55:12.187 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.187 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.187 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:12.187 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764661+0000", 2024-05-08T08:55:12.187 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5369999999999997e-06 2024-05-08T08:55:12.187 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.187 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.187 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:12.187 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764671+0000", 2024-05-08T08:55:12.187 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.028e-05 2024-05-08T08:55:12.187 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.188 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.188 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:12.188 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764680+0000", 2024-05-08T08:55:12.188 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0629999999999995e-06 2024-05-08T08:55:12.188 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.188 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.188 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:12.188 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764753+0000", 2024-05-08T08:55:12.188 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2679e-05 2024-05-08T08:55:12.188 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.188 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.188 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-05-08T08:55:12.188 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764765+0000", 2024-05-08T08:55:12.188 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2415000000000001e-05 2024-05-08T08:55:12.189 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.189 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.189 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:12.189 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:50.071692+0000", 2024-05-08T08:55:12.189 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.306927324 2024-05-08T08:55:12.189 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.189 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.189 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:12.189 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:50.071796+0000", 2024-05-08T08:55:12.189 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104005 2024-05-08T08:55:12.189 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.189 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:12.189 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.189 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.189 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.189 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4416.0:20531 3.16 3:6bb2ed16:::rbd_object_map.1140bc4a55c7:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-05-08T08:55:12.190 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:58.673786+0000", 2024-05-08T08:55:12.190 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 73.508177019000001, 2024-05-08T08:55:12.190 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.94684119899999997, 2024-05-08T08:55:12.190 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:12.190 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:12.190 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:55:12.190 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:12.190 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4416", 2024-05-08T08:55:12.190 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:55:12.190 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20531 2024-05-08T08:55:12.190 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.190 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:12.190 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.190 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:12.191 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673786+0000", 2024-05-08T08:55:12.191 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:12.191 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.191 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.191 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:12.191 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673786+0000", 2024-05-08T08:55:12.191 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:12.191 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.191 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.191 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:12.191 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673787+0000", 2024-05-08T08:55:12.191 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9800000000000003e-07 2024-05-08T08:55:12.191 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.191 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.191 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:12.191 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673791+0000", 2024-05-08T08:55:12.192 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.104e-06 2024-05-08T08:55:12.192 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.192 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.192 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:12.192 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673801+0000", 2024-05-08T08:55:12.192 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9960000000000005e-06 2024-05-08T08:55:12.192 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.192 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.192 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:12.192 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673806+0000", 2024-05-08T08:55:12.192 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2320000000000001e-06 2024-05-08T08:55:12.192 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.192 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.192 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:12.192 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673860+0000", 2024-05-08T08:55:12.193 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3468999999999999e-05 2024-05-08T08:55:12.193 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.193 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.193 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-05-08T08:55:12.193 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673866+0000", 2024-05-08T08:55:12.193 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3500000000000002e-06 2024-05-08T08:55:12.193 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.193 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.193 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:12.193 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.616700+0000", 2024-05-08T08:55:12.193 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.94283390700000003 2024-05-08T08:55:12.193 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.193 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.193 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:12.193 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.616941+0000", 2024-05-08T08:55:12.193 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00024080400000000001 2024-05-08T08:55:12.194 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.194 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.194 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-05-08T08:55:12.194 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.617072+0000", 2024-05-08T08:55:12.194 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000131688 2024-05-08T08:55:12.194 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.194 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.194 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:55:12.194 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.619571+0000", 2024-05-08T08:55:12.194 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0024990149999999998 2024-05-08T08:55:12.194 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.194 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.194 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:55:12.194 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.620568+0000", 2024-05-08T08:55:12.195 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00099711000000000005 2024-05-08T08:55:12.195 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.195 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.195 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:55:12.195 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.620602+0000", 2024-05-08T08:55:12.195 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.324e-05 2024-05-08T08:55:12.195 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.195 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.195 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:12.195 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.620627+0000", 2024-05-08T08:55:12.195 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5585999999999999e-05 2024-05-08T08:55:12.195 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.195 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:12.195 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.195 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.196 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.196 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5490 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4227072~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:12.196 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.881809+0000", 2024-05-08T08:55:12.196 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 46.300154507000002, 2024-05-08T08:55:12.196 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047460447000000003, 2024-05-08T08:55:12.196 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:12.196 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:12.196 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:12.196 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:12.196 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:12.196 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:12.196 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5490 2024-05-08T08:55:12.196 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.196 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:12.196 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.197 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:12.197 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881809+0000", 2024-05-08T08:55:12.197 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:12.197 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.197 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.197 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:12.197 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881809+0000", 2024-05-08T08:55:12.197 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:12.197 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.197 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.197 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:12.197 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881809+0000", 2024-05-08T08:55:12.197 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7899999999999999e-07 2024-05-08T08:55:12.197 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.198 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.198 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:12.198 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881811+0000", 2024-05-08T08:55:12.198 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0339999999999999e-06 2024-05-08T08:55:12.198 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.198 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.198 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:12.198 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881818+0000", 2024-05-08T08:55:12.198 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5760000000000002e-06 2024-05-08T08:55:12.198 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.198 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.199 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:12.199 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881823+0000", 2024-05-08T08:55:12.199 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4160000000000003e-06 2024-05-08T08:55:12.199 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.199 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.199 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:12.199 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929157+0000", 2024-05-08T08:55:12.199 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047334029999999999 2024-05-08T08:55:12.199 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.199 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.199 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:12.199 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929211+0000", 2024-05-08T08:55:12.199 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3560999999999998e-05 2024-05-08T08:55:12.199 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.200 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.200 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:12.200 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929269+0000", 2024-05-08T08:55:12.200 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8350999999999997e-05 2024-05-08T08:55:12.200 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.200 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:12.200 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.200 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.200 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.200 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5492 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4272128~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:12.200 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.881830+0000", 2024-05-08T08:55:12.200 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 46.300133213000002, 2024-05-08T08:55:12.200 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047614006, 2024-05-08T08:55:12.200 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:12.201 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:12.201 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:12.201 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:12.201 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:12.201 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:12.201 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5492 2024-05-08T08:55:12.201 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.201 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:12.201 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.201 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:12.201 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881830+0000", 2024-05-08T08:55:12.201 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:12.201 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.201 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.201 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:12.202 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881830+0000", 2024-05-08T08:55:12.202 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:12.202 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.202 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.202 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:12.202 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881830+0000", 2024-05-08T08:55:12.202 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4099999999999999e-07 2024-05-08T08:55:12.202 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.202 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.202 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:12.202 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881832+0000", 2024-05-08T08:55:12.202 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0370000000000001e-06 2024-05-08T08:55:12.202 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.202 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.203 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:12.203 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881839+0000", 2024-05-08T08:55:12.203 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3289999999999999e-06 2024-05-08T08:55:12.203 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.203 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.203 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:12.203 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881844+0000", 2024-05-08T08:55:12.203 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1170000000000003e-06 2024-05-08T08:55:12.203 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.203 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.203 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:12.203 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929296+0000", 2024-05-08T08:55:12.203 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047452418000000003 2024-05-08T08:55:12.203 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.203 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.203 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:12.203 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929360+0000", 2024-05-08T08:55:12.204 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4197999999999998e-05 2024-05-08T08:55:12.204 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.204 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.204 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:12.204 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929444+0000", 2024-05-08T08:55:12.204 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3466000000000004e-05 2024-05-08T08:55:12.204 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.204 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:12.204 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.204 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.204 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.204 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5494 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4317184~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:12.204 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.881850+0000", 2024-05-08T08:55:12.204 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 46.300113463999999, 2024-05-08T08:55:12.205 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047723887, 2024-05-08T08:55:12.205 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:12.205 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:12.205 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:12.205 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:12.205 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:12.205 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:12.205 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5494 2024-05-08T08:55:12.205 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.205 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:12.205 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.205 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:12.205 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881850+0000", 2024-05-08T08:55:12.205 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:12.205 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.205 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.206 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:12.206 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881850+0000", 2024-05-08T08:55:12.206 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:12.206 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.206 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.206 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:12.206 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881850+0000", 2024-05-08T08:55:12.206 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8500000000000002e-07 2024-05-08T08:55:12.206 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.206 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.206 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:12.206 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881856+0000", 2024-05-08T08:55:12.206 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9510000000000003e-06 2024-05-08T08:55:12.206 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.207 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.207 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:12.207 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881862+0000", 2024-05-08T08:55:12.207 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3049999999999998e-06 2024-05-08T08:55:12.207 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.207 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.207 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:12.207 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881868+0000", 2024-05-08T08:55:12.207 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8340000000000004e-06 2024-05-08T08:55:12.207 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.207 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.207 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:12.207 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929475+0000", 2024-05-08T08:55:12.207 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047606796 2024-05-08T08:55:12.207 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.208 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.208 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:12.208 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929518+0000", 2024-05-08T08:55:12.208 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3387000000000003e-05 2024-05-08T08:55:12.208 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.208 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.208 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:12.208 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929574+0000", 2024-05-08T08:55:12.208 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5129e-05 2024-05-08T08:55:12.208 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.208 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:12.208 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.208 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.208 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.208 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5495 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4362240~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:12.209 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.881875+0000", 2024-05-08T08:55:12.209 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 46.300088596999998, 2024-05-08T08:55:12.209 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047847967999999998, 2024-05-08T08:55:12.209 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:12.209 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:12.209 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:12.209 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:12.209 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:12.209 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:12.209 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5495 2024-05-08T08:55:12.209 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.209 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:12.209 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.209 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:12.210 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881875+0000", 2024-05-08T08:55:12.210 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:12.210 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.210 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.210 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:12.210 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881875+0000", 2024-05-08T08:55:12.210 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:12.210 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.210 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.210 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:12.210 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881875+0000", 2024-05-08T08:55:12.210 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-07 2024-05-08T08:55:12.210 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.210 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.211 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:12.211 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881877+0000", 2024-05-08T08:55:12.211 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9659999999999999e-06 2024-05-08T08:55:12.211 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.211 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.211 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:12.211 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881883+0000", 2024-05-08T08:55:12.211 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7649999999999999e-06 2024-05-08T08:55:12.211 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.211 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.211 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:12.211 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881887+0000", 2024-05-08T08:55:12.211 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6689999999999997e-06 2024-05-08T08:55:12.211 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.211 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.211 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:12.212 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929600+0000", 2024-05-08T08:55:12.212 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047712746 2024-05-08T08:55:12.212 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.212 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.212 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:12.212 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929659+0000", 2024-05-08T08:55:12.212 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8936e-05 2024-05-08T08:55:12.212 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.212 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.212 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:12.212 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929723+0000", 2024-05-08T08:55:12.212 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3524999999999999e-05 2024-05-08T08:55:12.212 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.212 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:12.212 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.213 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.213 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.213 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5496 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4407296~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:12.213 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.881894+0000", 2024-05-08T08:55:12.213 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 46.300068733000003, 2024-05-08T08:55:12.213 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047998015999999998, 2024-05-08T08:55:12.213 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:12.213 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:12.213 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:12.213 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:12.213 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:12.213 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:12.213 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5496 2024-05-08T08:55:12.213 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.213 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:12.214 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.214 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:12.214 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881894+0000", 2024-05-08T08:55:12.214 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:12.214 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.214 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.214 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:12.214 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881894+0000", 2024-05-08T08:55:12.214 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:12.214 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.214 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.214 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:12.215 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881895+0000", 2024-05-08T08:55:12.215 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6699999999999999e-07 2024-05-08T08:55:12.215 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.215 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.215 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:12.215 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881897+0000", 2024-05-08T08:55:12.215 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0229999999999999e-06 2024-05-08T08:55:12.215 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.215 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.215 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:12.215 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881903+0000", 2024-05-08T08:55:12.215 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9850000000000003e-06 2024-05-08T08:55:12.215 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.215 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.216 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:12.216 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881908+0000", 2024-05-08T08:55:12.216 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9019999999999998e-06 2024-05-08T08:55:12.216 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.216 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.216 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:12.216 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929751+0000", 2024-05-08T08:55:12.216 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047843550999999998 2024-05-08T08:55:12.216 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.216 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.216 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:12.216 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929823+0000", 2024-05-08T08:55:12.216 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1172000000000002e-05 2024-05-08T08:55:12.216 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.216 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.217 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:12.217 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929892+0000", 2024-05-08T08:55:12.217 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9915999999999999e-05 2024-05-08T08:55:12.217 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.217 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:12.217 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.217 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.217 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.217 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5498 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4452352~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:12.218 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.881914+0000", 2024-05-08T08:55:12.218 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 46.300048975000003, 2024-05-08T08:55:12.218 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048203553000000003, 2024-05-08T08:55:12.218 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:12.218 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:12.218 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:12.218 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:12.218 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:12.218 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:12.218 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5498 2024-05-08T08:55:12.218 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.218 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:12.218 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.218 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:12.222 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881914+0000", 2024-05-08T08:55:12.222 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:12.222 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.222 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.222 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:12.222 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881914+0000", 2024-05-08T08:55:12.222 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:12.222 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.222 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.222 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:12.222 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881915+0000", 2024-05-08T08:55:12.222 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8500000000000002e-07 2024-05-08T08:55:12.222 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.223 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.223 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:12.223 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881917+0000", 2024-05-08T08:55:12.223 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.181e-06 2024-05-08T08:55:12.223 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.223 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.223 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:12.223 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881924+0000", 2024-05-08T08:55:12.223 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9619999999999996e-06 2024-05-08T08:55:12.223 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.223 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.223 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:12.223 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881929+0000", 2024-05-08T08:55:12.223 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1399999999999999e-06 2024-05-08T08:55:12.223 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.224 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.224 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:12.224 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929952+0000", 2024-05-08T08:55:12.224 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048023134000000002 2024-05-08T08:55:12.224 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.224 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.224 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:12.224 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.930029+0000", 2024-05-08T08:55:12.224 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7490000000000005e-05 2024-05-08T08:55:12.224 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.224 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.224 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:12.224 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.930118+0000", 2024-05-08T08:55:12.224 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8261000000000001e-05 2024-05-08T08:55:12.225 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.225 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:12.225 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.225 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.225 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.225 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5563 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4321280~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:12.225 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.884201+0000", 2024-05-08T08:55:12.225 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 46.297761725000001, 2024-05-08T08:55:12.225 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047379969000000001, 2024-05-08T08:55:12.225 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:12.225 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:12.225 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:12.225 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:12.225 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:12.225 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:12.226 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5563 2024-05-08T08:55:12.226 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.226 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:12.226 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.226 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:12.226 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884201+0000", 2024-05-08T08:55:12.226 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:12.226 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.226 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.226 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:12.226 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884201+0000", 2024-05-08T08:55:12.226 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:12.227 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.227 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.227 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:12.227 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884202+0000", 2024-05-08T08:55:12.227 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8000000000000002e-07 2024-05-08T08:55:12.227 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.227 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.227 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:12.227 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884208+0000", 2024-05-08T08:55:12.227 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.674e-06 2024-05-08T08:55:12.227 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.227 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.227 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:12.227 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884215+0000", 2024-05-08T08:55:12.227 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0389999999999999e-06 2024-05-08T08:55:12.228 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.228 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.228 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:12.228 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884220+0000", 2024-05-08T08:55:12.228 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4950000000000002e-06 2024-05-08T08:55:12.228 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.228 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.228 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:12.228 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931417+0000", 2024-05-08T08:55:12.228 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047196674000000001 2024-05-08T08:55:12.228 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.228 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.228 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:12.228 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931493+0000", 2024-05-08T08:55:12.229 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5950000000000003e-05 2024-05-08T08:55:12.229 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.229 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.229 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:12.229 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931581+0000", 2024-05-08T08:55:12.229 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8857000000000007e-05 2024-05-08T08:55:12.229 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.229 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:12.229 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.229 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.229 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.229 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5564 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4366336~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:12.229 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.884225+0000", 2024-05-08T08:55:12.229 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 46.297738185, 2024-05-08T08:55:12.230 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047555989999999999, 2024-05-08T08:55:12.230 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:12.230 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:12.230 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:12.230 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:12.230 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:12.230 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:12.230 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5564 2024-05-08T08:55:12.230 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.230 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:12.230 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.230 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:12.230 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884225+0000", 2024-05-08T08:55:12.230 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:12.231 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.231 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.231 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:12.231 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884225+0000", 2024-05-08T08:55:12.231 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:12.231 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.231 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.231 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:12.231 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884225+0000", 2024-05-08T08:55:12.231 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2499999999999995e-07 2024-05-08T08:55:12.231 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.231 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.231 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:12.231 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884227+0000", 2024-05-08T08:55:12.231 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5829999999999999e-06 2024-05-08T08:55:12.231 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.232 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.232 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:12.232 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884232+0000", 2024-05-08T08:55:12.232 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1159999999999998e-06 2024-05-08T08:55:12.232 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.232 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.232 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:12.232 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884236+0000", 2024-05-08T08:55:12.232 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2490000000000003e-06 2024-05-08T08:55:12.232 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.232 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.232 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:12.232 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931617+0000", 2024-05-08T08:55:12.232 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047380310000000002 2024-05-08T08:55:12.232 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.232 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.233 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:12.233 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931689+0000", 2024-05-08T08:55:12.233 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2358000000000001e-05 2024-05-08T08:55:12.233 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.233 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.233 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:12.233 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931781+0000", 2024-05-08T08:55:12.233 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1848999999999994e-05 2024-05-08T08:55:12.233 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.233 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:12.233 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.233 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.233 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.233 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5565 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4411392~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:12.233 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.884251+0000", 2024-05-08T08:55:12.234 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 46.297712032, 2024-05-08T08:55:12.234 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047743671000000001, 2024-05-08T08:55:12.234 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:12.234 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:12.234 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:12.234 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:12.234 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:12.234 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:12.234 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5565 2024-05-08T08:55:12.234 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.234 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:12.234 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.234 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:12.235 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884251+0000", 2024-05-08T08:55:12.235 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:12.235 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.235 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.235 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:12.235 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884251+0000", 2024-05-08T08:55:12.235 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:12.235 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.235 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.235 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:12.235 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884252+0000", 2024-05-08T08:55:12.235 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3200000000000005e-07 2024-05-08T08:55:12.235 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.235 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.235 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:12.236 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884253+0000", 2024-05-08T08:55:12.236 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6980000000000001e-06 2024-05-08T08:55:12.236 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.236 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.236 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:12.236 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884259+0000", 2024-05-08T08:55:12.236 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3759999999999999e-06 2024-05-08T08:55:12.236 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.236 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.236 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:12.236 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884263+0000", 2024-05-08T08:55:12.236 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7280000000000003e-06 2024-05-08T08:55:12.236 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.236 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.236 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:12.236 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931818+0000", 2024-05-08T08:55:12.237 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047554368999999999 2024-05-08T08:55:12.237 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.237 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.237 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:12.237 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931897+0000", 2024-05-08T08:55:12.237 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8821999999999999e-05 2024-05-08T08:55:12.237 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.237 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.237 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:12.237 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931995+0000", 2024-05-08T08:55:12.237 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8146000000000005e-05 2024-05-08T08:55:12.237 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.237 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:12.237 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.237 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.238 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.238 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5600 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 3948544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:12.238 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.884619+0000", 2024-05-08T08:55:12.238 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 46.297343777000002, 2024-05-08T08:55:12.238 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047574067999999997, 2024-05-08T08:55:12.238 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:12.238 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:12.238 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:12.238 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:12.238 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:12.238 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:12.238 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5600 2024-05-08T08:55:12.238 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.238 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:12.238 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.239 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:12.239 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884619+0000", 2024-05-08T08:55:12.239 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:12.239 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.239 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.239 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:12.239 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884619+0000", 2024-05-08T08:55:12.239 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:12.239 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.239 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.239 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:12.239 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884620+0000", 2024-05-08T08:55:12.239 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.53e-07 2024-05-08T08:55:12.239 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.239 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.240 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:12.240 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884622+0000", 2024-05-08T08:55:12.240 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.796e-06 2024-05-08T08:55:12.240 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.240 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.240 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:12.240 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884627+0000", 2024-05-08T08:55:12.240 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2850000000000002e-06 2024-05-08T08:55:12.240 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.240 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.240 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:12.240 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884631+0000", 2024-05-08T08:55:12.240 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2899999999999996e-06 2024-05-08T08:55:12.240 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.240 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.240 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:12.241 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932024+0000", 2024-05-08T08:55:12.241 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047392680999999999 2024-05-08T08:55:12.241 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.241 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.241 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:12.241 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932102+0000", 2024-05-08T08:55:12.241 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7806999999999996e-05 2024-05-08T08:55:12.241 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.241 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.241 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:12.241 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932193+0000", 2024-05-08T08:55:12.241 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1855999999999996e-05 2024-05-08T08:55:12.241 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.241 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:12.242 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.242 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.242 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.242 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5603 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 3993600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:12.242 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.884680+0000", 2024-05-08T08:55:12.242 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 46.297283168, 2024-05-08T08:55:12.242 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047745034999999998, 2024-05-08T08:55:12.242 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:12.242 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:12.242 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:12.242 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:12.242 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:12.242 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:12.242 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5603 2024-05-08T08:55:12.242 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.243 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:12.243 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.243 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:12.243 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884680+0000", 2024-05-08T08:55:12.243 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:12.243 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.243 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.243 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:12.243 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884680+0000", 2024-05-08T08:55:12.243 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:12.243 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.243 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.243 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:12.244 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884680+0000", 2024-05-08T08:55:12.244 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4400000000000001e-07 2024-05-08T08:55:12.244 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.244 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.244 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:12.244 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884682+0000", 2024-05-08T08:55:12.244 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.514e-06 2024-05-08T08:55:12.244 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.244 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.244 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:12.244 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884687+0000", 2024-05-08T08:55:12.244 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0359999999999997e-06 2024-05-08T08:55:12.244 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.245 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.245 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:12.245 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884692+0000", 2024-05-08T08:55:12.245 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5990000000000004e-06 2024-05-08T08:55:12.245 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.245 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.245 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:12.245 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932235+0000", 2024-05-08T08:55:12.245 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047543635000000001 2024-05-08T08:55:12.245 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.245 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.245 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:12.245 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932323+0000", 2024-05-08T08:55:12.246 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7311000000000005e-05 2024-05-08T08:55:12.246 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.246 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.246 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:12.246 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932425+0000", 2024-05-08T08:55:12.246 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102496 2024-05-08T08:55:12.246 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.246 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:12.246 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.246 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.246 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.246 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5606 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4038656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:12.246 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.884729+0000", 2024-05-08T08:55:12.247 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 46.29723388, 2024-05-08T08:55:12.247 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047916488, 2024-05-08T08:55:12.247 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:12.247 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:12.247 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:12.247 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:12.247 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:12.247 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:12.247 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5606 2024-05-08T08:55:12.247 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.247 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:12.247 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.247 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:12.247 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884729+0000", 2024-05-08T08:55:12.247 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:12.247 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.248 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.248 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:12.248 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884729+0000", 2024-05-08T08:55:12.248 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:12.248 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.248 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.248 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:12.248 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.885940+0000", 2024-05-08T08:55:12.248 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001210509 2024-05-08T08:55:12.248 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.248 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.248 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:12.248 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.885945+0000", 2024-05-08T08:55:12.248 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7389999999999999e-06 2024-05-08T08:55:12.248 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.249 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.249 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:12.249 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.885955+0000", 2024-05-08T08:55:12.249 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0813e-05 2024-05-08T08:55:12.249 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.249 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.249 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:12.249 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.885962+0000", 2024-05-08T08:55:12.249 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1160000000000001e-06 2024-05-08T08:55:12.249 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.249 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.249 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:12.249 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932465+0000", 2024-05-08T08:55:12.249 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046502612999999998 2024-05-08T08:55:12.250 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.250 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.250 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:12.250 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932547+0000", 2024-05-08T08:55:12.250 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2025000000000002e-05 2024-05-08T08:55:12.250 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.250 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.250 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:12.250 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932646+0000", 2024-05-08T08:55:12.250 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8672999999999995e-05 2024-05-08T08:55:12.250 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.250 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:12.250 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.250 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.251 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.251 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5635 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4235264~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:12.251 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.886082+0000", 2024-05-08T08:55:12.251 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 46.295880760999999, 2024-05-08T08:55:12.251 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047438284999999997, 2024-05-08T08:55:12.251 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:12.251 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:12.251 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:12.251 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:12.251 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:12.251 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:12.251 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5635 2024-05-08T08:55:12.251 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.251 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:12.251 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.252 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:12.252 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886082+0000", 2024-05-08T08:55:12.252 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:12.252 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.252 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.252 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:12.252 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886082+0000", 2024-05-08T08:55:12.252 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:12.252 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.252 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.252 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:12.252 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886083+0000", 2024-05-08T08:55:12.252 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8999999999999996e-07 2024-05-08T08:55:12.252 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.253 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.253 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:12.253 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886086+0000", 2024-05-08T08:55:12.253 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7e-06 2024-05-08T08:55:12.253 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.253 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.253 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:12.253 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886093+0000", 2024-05-08T08:55:12.253 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5320000000000001e-06 2024-05-08T08:55:12.253 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.253 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.253 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:12.253 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886102+0000", 2024-05-08T08:55:12.253 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8070000000000006e-06 2024-05-08T08:55:12.253 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.254 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.254 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:12.254 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933357+0000", 2024-05-08T08:55:12.254 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047255093999999997 2024-05-08T08:55:12.254 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.254 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.254 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:12.254 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933433+0000", 2024-05-08T08:55:12.254 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5796000000000004e-05 2024-05-08T08:55:12.254 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.254 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.254 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:12.254 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933521+0000", 2024-05-08T08:55:12.254 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7766e-05 2024-05-08T08:55:12.255 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.255 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:12.255 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.255 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.255 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.255 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5636 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4280320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:12.255 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.886111+0000", 2024-05-08T08:55:12.255 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 46.295852142999998, 2024-05-08T08:55:12.255 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047663644999999998, 2024-05-08T08:55:12.255 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:12.255 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:12.255 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:12.255 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:12.255 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:12.255 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:12.255 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5636 2024-05-08T08:55:12.256 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.256 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:12.256 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.256 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:12.256 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886111+0000", 2024-05-08T08:55:12.256 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:12.256 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.256 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.256 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:12.256 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886111+0000", 2024-05-08T08:55:12.256 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:12.256 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.256 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.256 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:12.256 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886111+0000", 2024-05-08T08:55:12.257 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5200000000000002e-07 2024-05-08T08:55:12.257 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.257 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.257 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:12.257 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886114+0000", 2024-05-08T08:55:12.257 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1710000000000001e-06 2024-05-08T08:55:12.257 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.257 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.257 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:12.257 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886120+0000", 2024-05-08T08:55:12.257 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.404e-06 2024-05-08T08:55:12.257 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.257 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.257 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:12.257 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886125+0000", 2024-05-08T08:55:12.258 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2730000000000002e-06 2024-05-08T08:55:12.258 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.258 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.258 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:12.258 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933570+0000", 2024-05-08T08:55:12.258 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047444989999999999 2024-05-08T08:55:12.258 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.258 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.258 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:12.258 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933659+0000", 2024-05-08T08:55:12.258 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8912999999999994e-05 2024-05-08T08:55:12.258 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.258 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.258 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:12.258 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933775+0000", 2024-05-08T08:55:12.259 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000115442 2024-05-08T08:55:12.259 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.259 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:12.259 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.259 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.259 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.259 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5637 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4325376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:12.259 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.886132+0000", 2024-05-08T08:55:12.259 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 46.295831104999998, 2024-05-08T08:55:12.259 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047852219000000001, 2024-05-08T08:55:12.259 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:12.259 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:12.259 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:12.259 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:12.259 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:12.260 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:12.260 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5637 2024-05-08T08:55:12.260 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.260 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:12.260 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.260 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:12.260 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886132+0000", 2024-05-08T08:55:12.260 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:12.260 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.260 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.260 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:12.260 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886132+0000", 2024-05-08T08:55:12.260 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:12.260 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.260 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.261 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:12.261 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886132+0000", 2024-05-08T08:55:12.261 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7300000000000002e-07 2024-05-08T08:55:12.261 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.261 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.261 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:12.261 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886139+0000", 2024-05-08T08:55:12.261 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6109999999999999e-06 2024-05-08T08:55:12.261 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.261 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.261 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:12.261 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886146+0000", 2024-05-08T08:55:12.261 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6279999999999999e-06 2024-05-08T08:55:12.261 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.261 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.261 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:12.262 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886151+0000", 2024-05-08T08:55:12.262 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1769999999999997e-06 2024-05-08T08:55:12.262 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.262 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.262 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:12.262 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933812+0000", 2024-05-08T08:55:12.262 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047660912999999999 2024-05-08T08:55:12.262 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.262 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.262 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:12.262 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933888+0000", 2024-05-08T08:55:12.262 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5865000000000006e-05 2024-05-08T08:55:12.262 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.262 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.262 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:12.263 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933984+0000", 2024-05-08T08:55:12.263 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6651999999999996e-05 2024-05-08T08:55:12.263 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.263 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:12.263 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.263 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.263 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.263 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5638 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4370432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:12.263 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.886157+0000", 2024-05-08T08:55:12.263 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 46.295805841000004, 2024-05-08T08:55:12.263 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048077597, 2024-05-08T08:55:12.263 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:12.263 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:12.263 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:12.263 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:12.264 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:12.264 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:12.264 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5638 2024-05-08T08:55:12.264 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.264 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:12.264 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.264 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:12.264 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886157+0000", 2024-05-08T08:55:12.264 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:12.264 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.264 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.264 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:12.264 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886157+0000", 2024-05-08T08:55:12.264 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:12.264 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.264 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.265 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:12.265 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886158+0000", 2024-05-08T08:55:12.265 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1100000000000001e-07 2024-05-08T08:55:12.265 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.265 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.265 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:12.265 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886160+0000", 2024-05-08T08:55:12.265 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9800000000000001e-06 2024-05-08T08:55:12.265 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.265 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.265 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:12.265 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886166+0000", 2024-05-08T08:55:12.265 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.314e-06 2024-05-08T08:55:12.265 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.265 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.266 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:12.266 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886171+0000", 2024-05-08T08:55:12.266 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7700000000000001e-06 2024-05-08T08:55:12.266 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.266 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.266 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:12.266 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.934025+0000", 2024-05-08T08:55:12.266 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047854509000000003 2024-05-08T08:55:12.266 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.266 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.266 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:12.266 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.934121+0000", 2024-05-08T08:55:12.266 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5688999999999998e-05 2024-05-08T08:55:12.266 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.267 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.267 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:12.267 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.934235+0000", 2024-05-08T08:55:12.267 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000113924 2024-05-08T08:55:12.267 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.267 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:12.267 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.267 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.267 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.267 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5639 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4415488~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:12.267 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.886177+0000", 2024-05-08T08:55:12.267 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 46.295786090999997, 2024-05-08T08:55:12.267 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048249541, 2024-05-08T08:55:12.267 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:12.267 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:12.268 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:12.268 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:12.268 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:12.268 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:12.268 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5639 2024-05-08T08:55:12.268 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.268 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:12.268 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.268 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:12.268 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886177+0000", 2024-05-08T08:55:12.268 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:12.268 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.268 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.269 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:12.269 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886177+0000", 2024-05-08T08:55:12.269 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:12.269 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.269 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.269 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:12.269 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886177+0000", 2024-05-08T08:55:12.269 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.65e-07 2024-05-08T08:55:12.269 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.269 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.269 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:12.269 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886179+0000", 2024-05-08T08:55:12.269 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9530000000000002e-06 2024-05-08T08:55:12.269 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.269 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.270 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:12.270 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886186+0000", 2024-05-08T08:55:12.270 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3419999999999996e-06 2024-05-08T08:55:12.270 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.270 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.270 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:12.270 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886192+0000", 2024-05-08T08:55:12.270 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0560000000000001e-06 2024-05-08T08:55:12.270 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.270 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.270 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:12.270 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.934258+0000", 2024-05-08T08:55:12.270 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048066367999999998 2024-05-08T08:55:12.270 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.271 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.271 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:12.271 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.934338+0000", 2024-05-08T08:55:12.271 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9616000000000005e-05 2024-05-08T08:55:12.271 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.271 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.271 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:12.271 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.934427+0000", 2024-05-08T08:55:12.271 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8840999999999999e-05 2024-05-08T08:55:12.271 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.271 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:12.271 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.271 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.271 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:12.271 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:55:12.273 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:55:12.273 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:55:12.276+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:55:12.373 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:55:12.376+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:55:12.420 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:55:12.420 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:55:12.420 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:55:12.420 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:55:12.428 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:55:12.474 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:55:12.476+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:55:12.474 INFO:teuthology.orchestra.run.smithi059.stderr:noscrub is set 2024-05-08T08:55:12.575 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:55:12.577+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:55:12.613 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:55:12.613 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:55:12.613 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:55:12.614 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:55:12.614 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:55:12.627 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:55:12.711 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:55:12.714+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:55:12.777 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:55:12.780+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:55:12.798 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:55:12.798 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:55:12.798 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:55:12.799 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:55:12.799 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.799 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22073 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 782336~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:12.799 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:55:12.799 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 164.94211867999999, 2024-05-08T08:55:12.799 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047244234000000003, 2024-05-08T08:55:12.799 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:12.799 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:12.799 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:12.799 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:12.799 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:12.799 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:12.799 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22073 2024-05-08T08:55:12.799 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.800 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:12.800 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.800 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:12.800 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:55:12.800 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:12.800 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.800 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.800 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:12.800 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:55:12.800 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:12.800 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.800 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.800 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:12.800 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857668+0000", 2024-05-08T08:55:12.801 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7800000000000002e-07 2024-05-08T08:55:12.801 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.801 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.801 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:12.801 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857670+0000", 2024-05-08T08:55:12.801 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0820000000000001e-06 2024-05-08T08:55:12.801 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.801 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.801 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:12.801 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857678+0000", 2024-05-08T08:55:12.801 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7589999999999997e-06 2024-05-08T08:55:12.801 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.801 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.802 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:12.802 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857691+0000", 2024-05-08T08:55:12.802 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3672999999999999e-05 2024-05-08T08:55:12.802 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.802 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.802 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:12.802 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904780+0000", 2024-05-08T08:55:12.802 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047088616999999999 2024-05-08T08:55:12.802 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.802 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.802 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:12.802 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904840+0000", 2024-05-08T08:55:12.802 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0356999999999998e-05 2024-05-08T08:55:12.802 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.803 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.803 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:12.803 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904912+0000", 2024-05-08T08:55:12.803 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1367999999999998e-05 2024-05-08T08:55:12.803 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.803 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:12.803 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.803 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.803 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.803 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22075 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 798720~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:12.803 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:55:12.803 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 164.942088283, 2024-05-08T08:55:12.803 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047384643999999997, 2024-05-08T08:55:12.804 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:12.804 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:12.804 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:12.804 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:12.804 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:12.804 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:12.804 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22075 2024-05-08T08:55:12.804 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.804 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:12.804 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.804 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:12.804 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:55:12.804 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:12.804 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.805 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.805 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:12.805 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:55:12.805 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:12.805 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.805 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.805 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:12.805 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:55:12.805 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7399999999999999e-07 2024-05-08T08:55:12.805 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.805 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.805 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:12.805 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857705+0000", 2024-05-08T08:55:12.805 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1980000000000004e-06 2024-05-08T08:55:12.805 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.806 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.806 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:12.806 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857713+0000", 2024-05-08T08:55:12.806 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7039999999999994e-06 2024-05-08T08:55:12.806 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.806 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.806 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:12.806 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857718+0000", 2024-05-08T08:55:12.806 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.865e-06 2024-05-08T08:55:12.806 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.806 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.806 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:12.806 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904941+0000", 2024-05-08T08:55:12.806 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047223198000000001 2024-05-08T08:55:12.807 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.807 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.807 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:12.807 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904999+0000", 2024-05-08T08:55:12.807 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8254999999999997e-05 2024-05-08T08:55:12.807 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.807 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.807 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:12.807 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905082+0000", 2024-05-08T08:55:12.807 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3049999999999999e-05 2024-05-08T08:55:12.807 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.807 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:12.807 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.807 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.808 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.808 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22077 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 815104~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:12.808 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:55:12.808 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 164.94205027199999, 2024-05-08T08:55:12.808 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047519448999999998, 2024-05-08T08:55:12.808 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:12.808 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:12.808 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:12.808 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:12.808 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:12.808 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:12.808 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22077 2024-05-08T08:55:12.808 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.808 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:12.808 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.809 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:12.809 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:55:12.809 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:12.809 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.809 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.809 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:12.809 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:55:12.809 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:12.809 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.809 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.809 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:12.809 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:55:12.809 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.99e-07 2024-05-08T08:55:12.809 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.809 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.809 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:12.810 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857739+0000", 2024-05-08T08:55:12.810 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3599999999999999e-06 2024-05-08T08:55:12.810 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.810 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.810 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:12.810 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857745+0000", 2024-05-08T08:55:12.810 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5139999999999998e-06 2024-05-08T08:55:12.810 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.810 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.810 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:12.810 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857750+0000", 2024-05-08T08:55:12.810 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8550000000000001e-06 2024-05-08T08:55:12.810 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.810 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.811 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:12.811 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905110+0000", 2024-05-08T08:55:12.811 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047360126000000002 2024-05-08T08:55:12.811 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.811 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.811 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:12.811 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905175+0000", 2024-05-08T08:55:12.811 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4689000000000004e-05 2024-05-08T08:55:12.811 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.811 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.811 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:12.811 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905255+0000", 2024-05-08T08:55:12.811 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0306e-05 2024-05-08T08:55:12.812 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.812 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:12.812 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.812 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.812 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.812 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22079 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 831488~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:12.812 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:55:12.812 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 164.94202972400001, 2024-05-08T08:55:12.812 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.04767499, 2024-05-08T08:55:12.812 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:12.813 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:12.813 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:12.813 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:12.813 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:12.813 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:12.813 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22079 2024-05-08T08:55:12.813 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.813 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:12.813 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.813 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:12.813 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:55:12.813 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:12.813 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.813 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.814 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:12.814 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:55:12.814 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:12.814 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.814 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.814 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:12.814 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857757+0000", 2024-05-08T08:55:12.814 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8299999999999997e-07 2024-05-08T08:55:12.814 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.814 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.814 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:12.814 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857759+0000", 2024-05-08T08:55:12.814 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1390000000000002e-06 2024-05-08T08:55:12.814 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.815 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.815 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:12.815 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857765+0000", 2024-05-08T08:55:12.815 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4609999999999996e-06 2024-05-08T08:55:12.815 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.815 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.815 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:12.815 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857770+0000", 2024-05-08T08:55:12.815 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6689999999999997e-06 2024-05-08T08:55:12.815 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.815 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.816 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:12.816 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905281+0000", 2024-05-08T08:55:12.816 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047510668999999998 2024-05-08T08:55:12.816 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.816 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.816 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:12.816 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905344+0000", 2024-05-08T08:55:12.816 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2797000000000002e-05 2024-05-08T08:55:12.816 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.816 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.816 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:12.816 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905431+0000", 2024-05-08T08:55:12.817 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7771999999999999e-05 2024-05-08T08:55:12.817 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.817 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:12.817 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.817 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.817 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.817 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22082 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 847872~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:12.817 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:55:12.817 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 164.942010033, 2024-05-08T08:55:12.817 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047831934999999999, 2024-05-08T08:55:12.817 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:12.817 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:12.817 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:12.817 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:12.818 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:12.818 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:12.818 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22082 2024-05-08T08:55:12.818 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.818 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:12.818 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.818 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:12.818 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:55:12.818 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:12.818 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.818 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.818 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:12.818 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:55:12.818 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:12.819 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.819 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.819 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:12.819 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:55:12.819 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:55:12.819 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.819 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.819 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:12.819 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857778+0000", 2024-05-08T08:55:12.819 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0789999999999999e-06 2024-05-08T08:55:12.819 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.819 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.819 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:12.819 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857784+0000", 2024-05-08T08:55:12.820 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0480000000000004e-06 2024-05-08T08:55:12.820 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.820 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.820 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:12.820 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857790+0000", 2024-05-08T08:55:12.820 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2719999999999997e-06 2024-05-08T08:55:12.820 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.820 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.820 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:12.820 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905458+0000", 2024-05-08T08:55:12.820 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047668470999999997 2024-05-08T08:55:12.820 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.820 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.820 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:12.821 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905521+0000", 2024-05-08T08:55:12.821 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2952000000000003e-05 2024-05-08T08:55:12.821 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.821 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.821 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:12.821 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905608+0000", 2024-05-08T08:55:12.821 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6759000000000001e-05 2024-05-08T08:55:12.821 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.821 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:12.821 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.821 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.821 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.821 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22084 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 864256~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:12.821 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:55:12.821 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 164.94198608299999, 2024-05-08T08:55:12.822 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047977748000000001, 2024-05-08T08:55:12.822 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:12.822 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:12.822 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:12.822 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:12.822 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:12.822 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:12.822 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22084 2024-05-08T08:55:12.822 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.822 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:12.822 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.822 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:12.822 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:55:12.822 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:12.823 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.823 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.823 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:12.823 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:55:12.823 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:12.823 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.823 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.823 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:12.823 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:55:12.823 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:55:12.823 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.823 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.823 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:12.823 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857802+0000", 2024-05-08T08:55:12.823 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0779999999999998e-06 2024-05-08T08:55:12.824 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.824 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.824 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:12.824 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857812+0000", 2024-05-08T08:55:12.824 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7939999999999997e-06 2024-05-08T08:55:12.824 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.824 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.824 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:12.824 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857817+0000", 2024-05-08T08:55:12.824 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8899999999999998e-06 2024-05-08T08:55:12.824 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.824 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.824 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:12.824 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905636+0000", 2024-05-08T08:55:12.825 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047819252 2024-05-08T08:55:12.825 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.825 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.825 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:12.825 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905700+0000", 2024-05-08T08:55:12.825 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4016999999999994e-05 2024-05-08T08:55:12.825 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.825 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.825 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:12.825 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905778+0000", 2024-05-08T08:55:12.825 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7306999999999997e-05 2024-05-08T08:55:12.825 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.825 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:12.826 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.826 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.826 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.826 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22086 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 880640~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:12.826 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:55:12.826 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 164.941962733, 2024-05-08T08:55:12.826 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048119149, 2024-05-08T08:55:12.826 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:12.826 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:12.826 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:12.826 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:12.826 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:12.826 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:12.826 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22086 2024-05-08T08:55:12.827 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.827 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:12.827 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.827 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:12.827 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:55:12.827 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:12.827 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.827 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.827 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:12.827 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:55:12.827 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:12.827 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.827 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.827 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:12.827 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857824+0000", 2024-05-08T08:55:12.828 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4200000000000002e-07 2024-05-08T08:55:12.828 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.828 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.828 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:12.828 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857826+0000", 2024-05-08T08:55:12.828 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0590000000000001e-06 2024-05-08T08:55:12.828 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.828 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.828 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:12.828 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857832+0000", 2024-05-08T08:55:12.828 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8029999999999999e-06 2024-05-08T08:55:12.828 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.828 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.828 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:12.828 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857837+0000", 2024-05-08T08:55:12.829 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7929999999999997e-06 2024-05-08T08:55:12.829 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.829 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.829 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:12.829 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905803+0000", 2024-05-08T08:55:12.829 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047965761000000003 2024-05-08T08:55:12.829 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.829 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.829 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:12.829 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905862+0000", 2024-05-08T08:55:12.829 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9203000000000002e-05 2024-05-08T08:55:12.829 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.829 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.829 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:12.829 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905942+0000", 2024-05-08T08:55:12.830 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0187999999999999e-05 2024-05-08T08:55:12.830 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.830 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:12.830 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.830 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.830 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.830 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22088 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 897024~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:12.830 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:55:12.830 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 164.941942681, 2024-05-08T08:55:12.830 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048256049000000002, 2024-05-08T08:55:12.830 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:12.830 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:12.830 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:12.830 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:12.831 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:12.831 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:12.831 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22088 2024-05-08T08:55:12.831 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.831 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:12.831 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.831 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:12.831 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:55:12.831 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:12.831 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.831 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.831 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:12.831 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:55:12.831 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:12.832 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.832 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.832 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:12.832 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857844+0000", 2024-05-08T08:55:12.832 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.46e-07 2024-05-08T08:55:12.832 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.832 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.832 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:12.832 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857846+0000", 2024-05-08T08:55:12.832 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0899999999999999e-06 2024-05-08T08:55:12.832 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.832 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.832 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:12.832 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857855+0000", 2024-05-08T08:55:12.833 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8219999999999996e-06 2024-05-08T08:55:12.833 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.833 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.833 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:12.833 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857860+0000", 2024-05-08T08:55:12.833 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2989999999999996e-06 2024-05-08T08:55:12.833 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.833 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.833 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:12.833 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905966+0000", 2024-05-08T08:55:12.833 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048106542000000002 2024-05-08T08:55:12.833 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.833 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.833 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:12.833 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906024+0000", 2024-05-08T08:55:12.834 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7958000000000003e-05 2024-05-08T08:55:12.834 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.834 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.834 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:12.834 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906099+0000", 2024-05-08T08:55:12.834 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4992000000000003e-05 2024-05-08T08:55:12.834 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.834 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:12.834 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.834 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.834 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.834 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22091 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 913408~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:12.834 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:55:12.835 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 164.941918921, 2024-05-08T08:55:12.835 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048395290000000001, 2024-05-08T08:55:12.835 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:12.835 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:12.835 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:12.835 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:12.835 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:12.835 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:12.835 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22091 2024-05-08T08:55:12.835 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.835 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:12.835 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.835 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:12.835 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:55:12.835 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:12.836 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.836 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.836 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:12.836 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:55:12.836 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:12.836 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.836 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.836 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:12.836 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:55:12.836 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4900000000000001e-07 2024-05-08T08:55:12.837 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.837 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.837 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:12.837 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857870+0000", 2024-05-08T08:55:12.837 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2510000000000001e-06 2024-05-08T08:55:12.837 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.837 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.837 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:12.837 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857877+0000", 2024-05-08T08:55:12.837 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0809999999999996e-06 2024-05-08T08:55:12.837 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.837 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.837 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:12.837 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857881+0000", 2024-05-08T08:55:12.837 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.724e-06 2024-05-08T08:55:12.838 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.838 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.838 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:12.838 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906127+0000", 2024-05-08T08:55:12.838 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048245345000000002 2024-05-08T08:55:12.838 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.838 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.841 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:12.841 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906190+0000", 2024-05-08T08:55:12.842 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2762000000000007e-05 2024-05-08T08:55:12.842 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.842 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.842 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:12.842 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906262+0000", 2024-05-08T08:55:12.842 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2778e-05 2024-05-08T08:55:12.842 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.842 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:12.842 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.842 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.842 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.842 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22093 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 929792~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:12.842 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:55:12.842 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 164.94189655700001, 2024-05-08T08:55:12.843 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048532875000000003, 2024-05-08T08:55:12.843 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:12.843 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:12.843 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:12.843 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:12.843 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:12.843 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:12.843 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22093 2024-05-08T08:55:12.843 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.843 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:12.843 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.843 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:12.844 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:55:12.844 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:12.844 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.844 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.844 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:12.844 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:55:12.844 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:12.844 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.844 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.844 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:12.844 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857890+0000", 2024-05-08T08:55:12.844 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-07 2024-05-08T08:55:12.844 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.844 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.845 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:12.845 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857899+0000", 2024-05-08T08:55:12.845 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6889999999999999e-06 2024-05-08T08:55:12.845 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.845 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.845 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:12.845 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857906+0000", 2024-05-08T08:55:12.845 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6490000000000002e-06 2024-05-08T08:55:12.845 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.845 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.845 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:12.845 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857917+0000", 2024-05-08T08:55:12.845 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0764999999999999e-05 2024-05-08T08:55:12.846 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.846 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.846 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:12.846 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906287+0000", 2024-05-08T08:55:12.846 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048370442 2024-05-08T08:55:12.846 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.846 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.846 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:12.846 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906349+0000", 2024-05-08T08:55:12.846 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1537000000000004e-05 2024-05-08T08:55:12.846 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.846 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.846 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:12.847 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906422+0000", 2024-05-08T08:55:12.847 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3431999999999998e-05 2024-05-08T08:55:12.847 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.847 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:12.847 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.847 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.847 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.847 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22095 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 946176~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:12.847 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:55:12.847 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 164.94186185300001, 2024-05-08T08:55:12.847 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048657593999999998, 2024-05-08T08:55:12.847 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:12.847 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:12.847 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:12.848 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:12.848 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:12.848 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:12.848 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22095 2024-05-08T08:55:12.848 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.848 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:12.848 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.848 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:12.848 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:55:12.848 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:12.848 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.848 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.848 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:12.848 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:55:12.848 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:12.849 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.849 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.849 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:12.849 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857925+0000", 2024-05-08T08:55:12.849 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3099999999999998e-07 2024-05-08T08:55:12.849 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.849 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.849 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:12.849 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857927+0000", 2024-05-08T08:55:12.849 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.34e-06 2024-05-08T08:55:12.849 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.849 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.849 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:12.849 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857938+0000", 2024-05-08T08:55:12.850 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1426e-05 2024-05-08T08:55:12.850 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.850 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.850 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:12.850 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857943+0000", 2024-05-08T08:55:12.850 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5140000000000003e-06 2024-05-08T08:55:12.850 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.850 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.850 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:12.850 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906448+0000", 2024-05-08T08:55:12.850 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048504641000000001 2024-05-08T08:55:12.850 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.850 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.851 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:12.851 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906510+0000", 2024-05-08T08:55:12.851 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2663000000000006e-05 2024-05-08T08:55:12.851 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.851 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.851 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:12.851 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906582+0000", 2024-05-08T08:55:12.851 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1478999999999998e-05 2024-05-08T08:55:12.851 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.851 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:12.851 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.851 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.851 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.851 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22097 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 962560~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:12.851 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:55:12.852 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 164.941834164, 2024-05-08T08:55:12.852 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048791353000000003, 2024-05-08T08:55:12.852 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:12.852 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:12.852 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:12.852 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:12.852 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:12.852 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:12.852 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22097 2024-05-08T08:55:12.852 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.852 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:12.852 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.852 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:12.852 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:55:12.853 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:12.853 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.853 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.853 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:12.853 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:55:12.853 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:12.853 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.853 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.853 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:12.853 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:55:12.853 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9200000000000002e-07 2024-05-08T08:55:12.853 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.853 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.853 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:12.854 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857954+0000", 2024-05-08T08:55:12.854 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1900000000000002e-06 2024-05-08T08:55:12.854 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.854 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.854 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:12.854 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857961+0000", 2024-05-08T08:55:12.854 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4989999999999999e-06 2024-05-08T08:55:12.854 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.854 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.854 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:12.854 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857971+0000", 2024-05-08T08:55:12.854 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0485999999999999e-05 2024-05-08T08:55:12.854 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.854 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.855 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:12.855 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906607+0000", 2024-05-08T08:55:12.855 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048635109000000003 2024-05-08T08:55:12.855 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.855 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.855 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:12.855 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906667+0000", 2024-05-08T08:55:12.855 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0439000000000002e-05 2024-05-08T08:55:12.855 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.855 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.855 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:12.855 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906743+0000", 2024-05-08T08:55:12.855 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6237999999999998e-05 2024-05-08T08:55:12.855 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.856 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:12.856 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.856 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.856 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.856 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22099 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 978944~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:12.856 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:55:12.856 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 164.94180800399999, 2024-05-08T08:55:12.856 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048932055000000002, 2024-05-08T08:55:12.856 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:12.856 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:12.856 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:12.856 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:12.856 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:12.856 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:12.857 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22099 2024-05-08T08:55:12.857 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.857 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:12.857 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.857 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:12.857 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:55:12.857 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:12.857 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.857 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.857 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:12.857 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:55:12.857 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:12.857 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.858 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.858 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:12.858 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:55:12.858 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.27e-07 2024-05-08T08:55:12.858 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.858 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.858 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:12.858 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857987+0000", 2024-05-08T08:55:12.858 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5890000000000003e-06 2024-05-08T08:55:12.858 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.858 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.858 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:12.859 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857994+0000", 2024-05-08T08:55:12.859 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6619999999999999e-06 2024-05-08T08:55:12.859 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.859 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.859 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:12.859 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857998+0000", 2024-05-08T08:55:12.859 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7110000000000003e-06 2024-05-08T08:55:12.859 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.859 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.859 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:12.859 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906772+0000", 2024-05-08T08:55:12.859 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048773415000000001 2024-05-08T08:55:12.859 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.859 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.860 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:12.860 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906834+0000", 2024-05-08T08:55:12.860 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1752000000000001e-05 2024-05-08T08:55:12.860 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.860 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.860 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:12.860 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906910+0000", 2024-05-08T08:55:12.860 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6499000000000001e-05 2024-05-08T08:55:12.860 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.860 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:12.860 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.860 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.860 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.860 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22101 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 995328~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:12.861 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:55:12.861 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 164.941780873, 2024-05-08T08:55:12.861 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049087364000000001, 2024-05-08T08:55:12.861 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:12.861 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:12.861 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:12.861 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:12.861 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:12.861 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:12.861 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22101 2024-05-08T08:55:12.861 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.861 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:12.861 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.861 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:12.861 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:55:12.862 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:12.862 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.862 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.862 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:12.862 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:55:12.862 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:12.862 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.862 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.862 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:12.862 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858006+0000", 2024-05-08T08:55:12.862 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1399999999999997e-07 2024-05-08T08:55:12.862 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.862 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.863 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:12.863 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858008+0000", 2024-05-08T08:55:12.863 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2060000000000001e-06 2024-05-08T08:55:12.863 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.863 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.863 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:12.863 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858017+0000", 2024-05-08T08:55:12.863 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1579999999999994e-06 2024-05-08T08:55:12.863 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.863 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.863 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:12.863 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858022+0000", 2024-05-08T08:55:12.863 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4199999999999998e-06 2024-05-08T08:55:12.863 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.863 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.864 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:12.864 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906940+0000", 2024-05-08T08:55:12.864 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048917627999999998 2024-05-08T08:55:12.864 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.864 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.864 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:12.864 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907002+0000", 2024-05-08T08:55:12.864 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2399999999999999e-05 2024-05-08T08:55:12.864 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.864 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.864 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:12.864 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907092+0000", 2024-05-08T08:55:12.864 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0137999999999997e-05 2024-05-08T08:55:12.864 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.864 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:12.865 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.865 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.865 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.865 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22103 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1011712~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:12.865 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:55:12.865 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 164.94175608800001, 2024-05-08T08:55:12.865 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049228542, 2024-05-08T08:55:12.865 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:12.865 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:12.865 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:12.865 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:12.865 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:12.865 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:12.866 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22103 2024-05-08T08:55:12.866 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.866 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:12.866 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.866 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:12.866 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:55:12.866 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:12.866 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.866 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.866 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:12.866 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:55:12.866 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:12.866 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.867 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.867 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:12.867 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:55:12.867 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4299999999999999e-07 2024-05-08T08:55:12.867 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.867 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.867 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:12.867 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858032+0000", 2024-05-08T08:55:12.867 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0590000000000001e-06 2024-05-08T08:55:12.867 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.867 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.867 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:12.867 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858037+0000", 2024-05-08T08:55:12.867 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.977e-06 2024-05-08T08:55:12.867 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.868 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.868 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:12.868 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858042+0000", 2024-05-08T08:55:12.868 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7650000000000001e-06 2024-05-08T08:55:12.868 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.868 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.868 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:12.868 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907118+0000", 2024-05-08T08:55:12.868 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049076147000000001 2024-05-08T08:55:12.868 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.868 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.868 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:12.868 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907180+0000", 2024-05-08T08:55:12.869 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1971999999999996e-05 2024-05-08T08:55:12.869 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.869 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.869 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:12.869 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907258+0000", 2024-05-08T08:55:12.869 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8279e-05 2024-05-08T08:55:12.869 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.869 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:12.869 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.869 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.869 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.869 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22105 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1028096~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:12.869 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:55:12.869 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 164.94173684500001, 2024-05-08T08:55:12.869 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049394609999999999, 2024-05-08T08:55:12.869 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:12.870 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:12.870 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:12.870 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:12.870 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:12.870 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:12.870 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22105 2024-05-08T08:55:12.870 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.870 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:12.870 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.870 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:12.870 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:55:12.870 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:12.870 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.870 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.870 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:12.871 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:55:12.871 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:12.871 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.871 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.871 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:12.871 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858050+0000", 2024-05-08T08:55:12.871 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5999999999999999e-07 2024-05-08T08:55:12.871 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.871 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.871 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:12.871 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858051+0000", 2024-05-08T08:55:12.871 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8870000000000001e-06 2024-05-08T08:55:12.871 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.872 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.872 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:12.872 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858057+0000", 2024-05-08T08:55:12.872 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8019999999999997e-06 2024-05-08T08:55:12.872 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.872 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.872 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:12.872 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858062+0000", 2024-05-08T08:55:12.872 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.561e-06 2024-05-08T08:55:12.872 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.872 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.872 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:12.872 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907289+0000", 2024-05-08T08:55:12.872 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049226957000000002 2024-05-08T08:55:12.872 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.872 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.873 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:12.873 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907356+0000", 2024-05-08T08:55:12.873 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6915000000000006e-05 2024-05-08T08:55:12.873 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.873 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.873 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:12.873 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907444+0000", 2024-05-08T08:55:12.873 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8127999999999994e-05 2024-05-08T08:55:12.873 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.873 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:12.873 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.873 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.873 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.873 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22107 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1044480~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:12.874 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:55:12.874 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 164.94171850500001, 2024-05-08T08:55:12.874 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049799942999999999, 2024-05-08T08:55:12.874 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:12.874 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:12.874 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:12.874 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:12.874 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:12.874 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:12.874 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22107 2024-05-08T08:55:12.874 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.874 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:12.874 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.874 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:12.875 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:55:12.875 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:12.875 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.875 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.875 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:12.875 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:55:12.875 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:12.875 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.875 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.875 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:12.875 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:55:12.875 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4799999999999999e-07 2024-05-08T08:55:12.875 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.875 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.875 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:12.876 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858070+0000", 2024-05-08T08:55:12.876 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.012e-06 2024-05-08T08:55:12.876 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.876 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.876 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:12.876 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858077+0000", 2024-05-08T08:55:12.876 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6290000000000001e-06 2024-05-08T08:55:12.876 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.876 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.876 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:12.876 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858083+0000", 2024-05-08T08:55:12.876 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8880000000000002e-06 2024-05-08T08:55:12.876 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.876 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.876 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:12.877 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907671+0000", 2024-05-08T08:55:12.877 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049587191000000003 2024-05-08T08:55:12.877 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.877 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.877 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:12.877 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907763+0000", 2024-05-08T08:55:12.877 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2231000000000005e-05 2024-05-08T08:55:12.877 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.877 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.877 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:12.877 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907867+0000", 2024-05-08T08:55:12.877 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104644 2024-05-08T08:55:12.877 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.877 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:12.877 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.878 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.878 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.878 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22294 3.1c 3:3aad0389:::rbd_data.11406886b07b.0000000000000080:head [stat,delete] snapc 7=[7] RETRY=6 ondisk+retry+write+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:12.878 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:55:12.878 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 164.94112465699999, 2024-05-08T08:55:12.878 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051152191, 2024-05-08T08:55:12.878 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:12.878 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:12.878 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:12.878 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:12.878 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:12.878 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:12.878 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22294 2024-05-08T08:55:12.878 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.879 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:12.879 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.879 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:12.879 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:55:12.879 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:12.879 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.879 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.879 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:12.879 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:55:12.879 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:12.879 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.879 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.879 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:12.879 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858662+0000", 2024-05-08T08:55:12.879 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4799999999999998e-07 2024-05-08T08:55:12.880 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.880 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.880 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:12.880 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858664+0000", 2024-05-08T08:55:12.880 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1229999999999998e-06 2024-05-08T08:55:12.880 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.880 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.880 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:12.880 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858670+0000", 2024-05-08T08:55:12.880 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3099999999999997e-06 2024-05-08T08:55:12.880 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.880 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.880 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:12.880 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858676+0000", 2024-05-08T08:55:12.881 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9750000000000004e-06 2024-05-08T08:55:12.881 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.881 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.881 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:12.881 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907901+0000", 2024-05-08T08:55:12.881 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049224520000000001 2024-05-08T08:55:12.881 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.881 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.881 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:12.881 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.908085+0000", 2024-05-08T08:55:12.881 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00018377500000000001 2024-05-08T08:55:12.881 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.881 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.881 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:12.881 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.909814+0000", 2024-05-08T08:55:12.882 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00172894 2024-05-08T08:55:12.882 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.882 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:12.882 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.882 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.882 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.882 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4416.0:13461 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-05-08T08:55:12.882 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:22.487456+0000", 2024-05-08T08:55:12.882 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 110.312329616, 2024-05-08T08:55:12.882 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.99102620799999996, 2024-05-08T08:55:12.882 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:12.882 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:12.882 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:12.883 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:12.883 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4416", 2024-05-08T08:55:12.883 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:55:12.883 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 13461 2024-05-08T08:55:12.883 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.883 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:12.883 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.883 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:12.883 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487456+0000", 2024-05-08T08:55:12.883 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:12.883 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.883 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.883 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:12.883 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487456+0000", 2024-05-08T08:55:12.883 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:12.884 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.884 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.884 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:12.884 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487458+0000", 2024-05-08T08:55:12.884 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5659999999999999e-06 2024-05-08T08:55:12.884 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.884 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.884 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:12.884 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487463+0000", 2024-05-08T08:55:12.884 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6890000000000003e-06 2024-05-08T08:55:12.884 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.884 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.884 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:12.884 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487472+0000", 2024-05-08T08:55:12.885 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7710000000000001e-06 2024-05-08T08:55:12.885 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.885 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.885 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:12.885 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487478+0000", 2024-05-08T08:55:12.885 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1390000000000003e-06 2024-05-08T08:55:12.885 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.885 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.885 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:12.885 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487520+0000", 2024-05-08T08:55:12.885 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2259999999999999e-05 2024-05-08T08:55:12.885 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.885 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.885 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-05-08T08:55:12.886 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487528+0000", 2024-05-08T08:55:12.886 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7479999999999993e-06 2024-05-08T08:55:12.886 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.886 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.886 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:12.886 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:23.478139+0000", 2024-05-08T08:55:12.886 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.99061165200000001 2024-05-08T08:55:12.886 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.886 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.886 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:12.886 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:23.478301+0000", 2024-05-08T08:55:12.886 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000161602 2024-05-08T08:55:12.886 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.886 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.886 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:12.887 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:23.478483+0000", 2024-05-08T08:55:12.887 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000181781 2024-05-08T08:55:12.887 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.887 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:12.887 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.887 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.887 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.887 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5752.0:404 3.1a 3:5f5e8104:::rbd_data.1140bc4a55c7.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 5175296~45056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-05-08T08:55:12.887 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:00.390863+0000", 2024-05-08T08:55:12.887 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 72.408923420999997, 2024-05-08T08:55:12.887 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.055189174000000001, 2024-05-08T08:55:12.887 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:12.887 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:12.887 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:55:12.887 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:12.887 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5752", 2024-05-08T08:55:12.888 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/3534419815", 2024-05-08T08:55:12.888 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 404 2024-05-08T08:55:12.888 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.888 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:12.888 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.888 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:12.888 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.390863+0000", 2024-05-08T08:55:12.888 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:12.888 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.888 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.888 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:12.888 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.390863+0000", 2024-05-08T08:55:12.888 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:12.888 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.888 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.889 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:12.889 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.390864+0000", 2024-05-08T08:55:12.889 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.471e-06 2024-05-08T08:55:12.889 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.889 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.889 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:12.889 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397475+0000", 2024-05-08T08:55:12.889 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0066109109999999997 2024-05-08T08:55:12.889 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.889 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.889 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:12.889 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397498+0000", 2024-05-08T08:55:12.889 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2674e-05 2024-05-08T08:55:12.889 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.890 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.890 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:12.890 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397508+0000", 2024-05-08T08:55:12.890 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9019999999999994e-06 2024-05-08T08:55:12.890 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.890 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.890 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:12.890 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397609+0000", 2024-05-08T08:55:12.890 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010121899999999999 2024-05-08T08:55:12.890 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.890 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.890 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:12.890 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397711+0000", 2024-05-08T08:55:12.890 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102489 2024-05-08T08:55:12.890 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.891 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.891 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-05-08T08:55:12.891 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397999+0000", 2024-05-08T08:55:12.891 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00028733799999999999 2024-05-08T08:55:12.891 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.891 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.891 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:55:12.891 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.432093+0000", 2024-05-08T08:55:12.891 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.034094865000000002 2024-05-08T08:55:12.891 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.891 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.891 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:55:12.891 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.445975+0000", 2024-05-08T08:55:12.891 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013881187 2024-05-08T08:55:12.891 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.892 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.892 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:55:12.892 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.446013+0000", 2024-05-08T08:55:12.892 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8155000000000003e-05 2024-05-08T08:55:12.892 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:12.892 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:12.892 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:12.892 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.446052+0000", 2024-05-08T08:55:12.892 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8962999999999999e-05 2024-05-08T08:55:12.892 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.892 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:12.892 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.892 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:12.892 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:12.892 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:55:12.893 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T08:55:12.895 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:55:12.896+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:55:12.895 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:55:12.995 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:55:12.997+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:55:13.048 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:55:13.048 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:55:13.060 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:55:13.060 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:55:13.060 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:55:13.060 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:55:13.070 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:55:13.074 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:55:13.074 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:55:13.094 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:55:13.094 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:55:13.095 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:55:13.097+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:55:13.115 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:55:13.115 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:55:13.136 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:55:13.137 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:55:13.158 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:55:13.158 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:55:13.211 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:55:13.213+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:55:13.226 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:13.226 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:13.236 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:55:13.298 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:55:13.301+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:55:13.353 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:13.354 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:13.364 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:55:13.399 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:55:13.402+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:55:13.494 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:13.494 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:13.500 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:55:13.502+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:55:13.505 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:55:13.600 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:55:13.603+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:55:13.666 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:13.667 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:13.676 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:55:13.701 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:55:13.704+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:55:13.802 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:55:13.805+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:55:13.839 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:13.839 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:13.849 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:55:13.903 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:55:13.906+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:55:13.965 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:13.966 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:13.975 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:55:14.004 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:55:14.006+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:55:14.086 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:14.086 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:14.095 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:55:14.104 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:55:14.107+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:55:14.205 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:55:14.207+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:55:14.211 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:14.211 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:14.220 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:55:14.306 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:55:14.309+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:55:14.382 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:14.382 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:14.393 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:55:14.407 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:55:14.410+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:55:14.508 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:55:14.510+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:55:14.517 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-05-08T08:55:14.526 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:14.527 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:14.539 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:55:14.608 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:55:14.611+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:55:14.709 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:55:14.712+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:55:14.711 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:14.711 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:14.720 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:55:14.810 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:55:14.813+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:55:14.830 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:14.830 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:14.838 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:55:14.912 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:55:14.914+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:55:14.943 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:14.943 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:14.952 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:55:15.037 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:55:15.040+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:55:15.097 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:15.097 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:15.106 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:55:15.147 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:55:15.149+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:55:15.170 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T08:55:15.214 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:55:15.216+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:55:15.252 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:15.252 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:15.261 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:55:15.298 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:55:15.298 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:55:15.315 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:55:15.315 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:55:15.332 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:55:15.332 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:55:15.345 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:55:15.348+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:55:15.350 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:55:15.350 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:55:15.365 INFO:teuthology.orchestra.run.smithi059.stderr:nodeep-scrub is set 2024-05-08T08:55:15.374 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:55:15.374 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:55:15.395 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:55:15.395 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:55:15.417 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:55:15.418 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:55:15.418 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:55:15.418 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:55:15.418 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:55:15.427 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:55:15.448 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:55:15.451+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:55:15.492 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4] out_osds: [5, 0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-05-08T08:55:15.492 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T08:55:15.493 INFO:tasks.thrashosds.thrasher:inject_pause on osd.0 2024-05-08T08:55:15.493 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-05-08T08:55:15.493 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-05-08T08:55:15.493 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok config set filestore_inject_stall 3 2024-05-08T08:55:15.516 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:55:15.518+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:55:15.574 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:55:15.574 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:55:15.574 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:55:15.574 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:55:15.574 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.574 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22073 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 782336~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:15.575 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:55:15.575 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 167.71847745299999, 2024-05-08T08:55:15.575 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047244234000000003, 2024-05-08T08:55:15.575 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:15.575 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:15.575 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:15.575 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:15.575 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:15.575 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:15.575 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22073 2024-05-08T08:55:15.575 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.575 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:15.575 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.576 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:15.576 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:55:15.576 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:15.576 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.576 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.576 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:15.576 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:55:15.576 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:15.576 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.576 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.576 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:15.576 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857668+0000", 2024-05-08T08:55:15.576 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7800000000000002e-07 2024-05-08T08:55:15.576 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.577 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.577 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:15.577 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857670+0000", 2024-05-08T08:55:15.577 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0820000000000001e-06 2024-05-08T08:55:15.577 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.577 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.577 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:15.577 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857678+0000", 2024-05-08T08:55:15.577 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7589999999999997e-06 2024-05-08T08:55:15.577 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.577 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.577 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:15.577 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857691+0000", 2024-05-08T08:55:15.577 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3672999999999999e-05 2024-05-08T08:55:15.577 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.578 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.578 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:15.578 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904780+0000", 2024-05-08T08:55:15.578 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047088616999999999 2024-05-08T08:55:15.578 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.578 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.578 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:15.578 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904840+0000", 2024-05-08T08:55:15.578 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0356999999999998e-05 2024-05-08T08:55:15.578 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.578 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.578 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:15.578 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904912+0000", 2024-05-08T08:55:15.578 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1367999999999998e-05 2024-05-08T08:55:15.578 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:15.579 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:15.579 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:15.579 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.579 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.579 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22075 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 798720~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:15.579 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:55:15.579 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 167.718447056, 2024-05-08T08:55:15.579 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047384643999999997, 2024-05-08T08:55:15.579 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:15.579 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:15.579 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:15.579 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:15.579 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:15.579 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:15.579 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22075 2024-05-08T08:55:15.579 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.580 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:15.580 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.580 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:15.580 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:55:15.580 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:15.580 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.580 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.580 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:15.580 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:55:15.580 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:15.580 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.580 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.580 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:15.580 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:55:15.580 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7399999999999999e-07 2024-05-08T08:55:15.580 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.581 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.581 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:15.581 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857705+0000", 2024-05-08T08:55:15.581 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1980000000000004e-06 2024-05-08T08:55:15.581 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.581 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.581 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:15.581 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857713+0000", 2024-05-08T08:55:15.581 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7039999999999994e-06 2024-05-08T08:55:15.581 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.581 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.581 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:15.581 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857718+0000", 2024-05-08T08:55:15.581 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.865e-06 2024-05-08T08:55:15.581 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.581 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.581 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:15.582 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904941+0000", 2024-05-08T08:55:15.582 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047223198000000001 2024-05-08T08:55:15.582 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.582 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.582 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:15.582 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904999+0000", 2024-05-08T08:55:15.582 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8254999999999997e-05 2024-05-08T08:55:15.582 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.582 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.582 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:15.582 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905082+0000", 2024-05-08T08:55:15.582 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3049999999999999e-05 2024-05-08T08:55:15.582 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:15.582 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:15.582 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:15.583 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.583 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.583 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22077 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 815104~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:15.583 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:55:15.583 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 167.71840904499999, 2024-05-08T08:55:15.583 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047519448999999998, 2024-05-08T08:55:15.583 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:15.583 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:15.583 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:15.583 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:15.583 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:15.583 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:15.583 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22077 2024-05-08T08:55:15.583 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.583 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:15.583 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.584 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:15.584 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:55:15.584 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:15.584 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.584 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.584 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:15.584 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:55:15.584 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:15.584 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.584 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.584 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:15.584 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:55:15.584 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.99e-07 2024-05-08T08:55:15.584 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.584 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.585 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:15.585 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857739+0000", 2024-05-08T08:55:15.585 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3599999999999999e-06 2024-05-08T08:55:15.585 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.585 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.585 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:15.585 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857745+0000", 2024-05-08T08:55:15.585 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5139999999999998e-06 2024-05-08T08:55:15.585 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.585 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.585 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:15.585 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857750+0000", 2024-05-08T08:55:15.585 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8550000000000001e-06 2024-05-08T08:55:15.585 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.586 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.586 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:15.586 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905110+0000", 2024-05-08T08:55:15.586 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047360126000000002 2024-05-08T08:55:15.586 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.586 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.586 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:15.586 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905175+0000", 2024-05-08T08:55:15.586 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4689000000000004e-05 2024-05-08T08:55:15.586 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.586 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.587 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:15.587 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905255+0000", 2024-05-08T08:55:15.587 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0306e-05 2024-05-08T08:55:15.587 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:15.587 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:15.587 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:15.587 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.587 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.587 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22079 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 831488~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:15.587 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:55:15.587 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 167.71838849700001, 2024-05-08T08:55:15.587 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.04767499, 2024-05-08T08:55:15.587 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:15.587 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:15.587 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:15.588 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:15.588 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:15.588 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:15.588 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22079 2024-05-08T08:55:15.588 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.588 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:15.588 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.588 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:15.588 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:55:15.588 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:15.588 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.588 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.588 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:15.589 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:55:15.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:15.589 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.589 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.589 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:15.589 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857757+0000", 2024-05-08T08:55:15.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8299999999999997e-07 2024-05-08T08:55:15.589 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.589 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.589 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:15.589 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857759+0000", 2024-05-08T08:55:15.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1390000000000002e-06 2024-05-08T08:55:15.589 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.589 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.590 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:15.590 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857765+0000", 2024-05-08T08:55:15.590 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4609999999999996e-06 2024-05-08T08:55:15.590 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.590 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.590 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:15.590 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857770+0000", 2024-05-08T08:55:15.590 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6689999999999997e-06 2024-05-08T08:55:15.590 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.590 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.590 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:15.590 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905281+0000", 2024-05-08T08:55:15.590 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047510668999999998 2024-05-08T08:55:15.590 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.591 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.591 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:15.591 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905344+0000", 2024-05-08T08:55:15.591 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2797000000000002e-05 2024-05-08T08:55:15.591 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.591 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.591 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:15.591 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905431+0000", 2024-05-08T08:55:15.591 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7771999999999999e-05 2024-05-08T08:55:15.591 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:15.591 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:15.591 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:15.591 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.591 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.591 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22082 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 847872~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:15.592 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:55:15.592 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 167.718368806, 2024-05-08T08:55:15.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047831934999999999, 2024-05-08T08:55:15.592 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:15.592 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:15.592 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:15.592 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:15.592 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:15.592 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:15.592 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22082 2024-05-08T08:55:15.592 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.592 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:15.592 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.592 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:15.593 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:55:15.593 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:15.593 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.593 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.593 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:15.593 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:55:15.593 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:15.593 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.593 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.593 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:15.593 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:55:15.593 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:55:15.593 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.593 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.593 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:15.594 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857778+0000", 2024-05-08T08:55:15.594 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0789999999999999e-06 2024-05-08T08:55:15.594 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.594 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.594 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:15.594 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857784+0000", 2024-05-08T08:55:15.594 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0480000000000004e-06 2024-05-08T08:55:15.594 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.594 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.594 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:15.594 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857790+0000", 2024-05-08T08:55:15.594 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2719999999999997e-06 2024-05-08T08:55:15.594 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.594 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.594 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:15.594 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905458+0000", 2024-05-08T08:55:15.595 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047668470999999997 2024-05-08T08:55:15.595 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.595 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.595 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:15.595 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905521+0000", 2024-05-08T08:55:15.595 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2952000000000003e-05 2024-05-08T08:55:15.595 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.595 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.595 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:15.595 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905608+0000", 2024-05-08T08:55:15.595 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6759000000000001e-05 2024-05-08T08:55:15.595 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:15.595 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:15.595 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:15.595 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.596 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22084 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 864256~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:15.596 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:55:15.596 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 167.71834485599999, 2024-05-08T08:55:15.596 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047977748000000001, 2024-05-08T08:55:15.596 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:15.596 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:15.596 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:15.596 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:15.596 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:15.596 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:15.596 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22084 2024-05-08T08:55:15.596 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.596 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:15.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.597 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:15.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:55:15.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:15.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.597 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:15.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:55:15.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:15.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.597 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:15.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:55:15.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:55:15.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.597 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:15.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857802+0000", 2024-05-08T08:55:15.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0779999999999998e-06 2024-05-08T08:55:15.598 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:15.598 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857812+0000", 2024-05-08T08:55:15.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7939999999999997e-06 2024-05-08T08:55:15.598 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:15.598 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857817+0000", 2024-05-08T08:55:15.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8899999999999998e-06 2024-05-08T08:55:15.598 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:15.598 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905636+0000", 2024-05-08T08:55:15.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047819252 2024-05-08T08:55:15.599 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.599 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.599 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:15.599 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905700+0000", 2024-05-08T08:55:15.599 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4016999999999994e-05 2024-05-08T08:55:15.599 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.599 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.599 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:15.599 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905778+0000", 2024-05-08T08:55:15.599 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7306999999999997e-05 2024-05-08T08:55:15.599 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:15.599 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:15.599 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:15.599 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.599 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.599 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22086 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 880640~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:15.600 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:55:15.600 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 167.718321506, 2024-05-08T08:55:15.600 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048119149, 2024-05-08T08:55:15.600 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:15.600 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:15.600 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:15.600 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:15.600 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:15.600 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:15.600 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22086 2024-05-08T08:55:15.600 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.600 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:15.600 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.600 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:15.600 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:55:15.600 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:15.600 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.601 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:15.601 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:55:15.601 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:15.601 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.601 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:15.601 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857824+0000", 2024-05-08T08:55:15.601 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4200000000000002e-07 2024-05-08T08:55:15.601 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.601 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:15.601 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857826+0000", 2024-05-08T08:55:15.601 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0590000000000001e-06 2024-05-08T08:55:15.601 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:15.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857832+0000", 2024-05-08T08:55:15.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8029999999999999e-06 2024-05-08T08:55:15.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:15.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857837+0000", 2024-05-08T08:55:15.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7929999999999997e-06 2024-05-08T08:55:15.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:15.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905803+0000", 2024-05-08T08:55:15.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047965761000000003 2024-05-08T08:55:15.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.603 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:15.603 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905862+0000", 2024-05-08T08:55:15.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9203000000000002e-05 2024-05-08T08:55:15.603 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.603 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:15.603 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905942+0000", 2024-05-08T08:55:15.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0187999999999999e-05 2024-05-08T08:55:15.603 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:15.603 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:15.603 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:15.603 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.603 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22088 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 897024~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:15.603 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:55:15.604 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 167.718301454, 2024-05-08T08:55:15.604 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048256049000000002, 2024-05-08T08:55:15.604 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:15.604 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:15.604 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:15.604 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:15.604 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:15.604 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:15.604 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22088 2024-05-08T08:55:15.604 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.604 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:15.604 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.604 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:15.604 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:55:15.604 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:15.604 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.604 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.605 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:15.605 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:55:15.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:15.605 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.605 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.605 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:15.605 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857844+0000", 2024-05-08T08:55:15.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.46e-07 2024-05-08T08:55:15.605 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.605 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.605 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:15.605 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857846+0000", 2024-05-08T08:55:15.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0899999999999999e-06 2024-05-08T08:55:15.605 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.606 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:15.606 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857855+0000", 2024-05-08T08:55:15.606 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8219999999999996e-06 2024-05-08T08:55:15.606 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.606 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:15.606 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857860+0000", 2024-05-08T08:55:15.606 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2989999999999996e-06 2024-05-08T08:55:15.606 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.606 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:15.606 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905966+0000", 2024-05-08T08:55:15.606 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048106542000000002 2024-05-08T08:55:15.606 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.607 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:15.607 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906024+0000", 2024-05-08T08:55:15.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7958000000000003e-05 2024-05-08T08:55:15.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.607 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:15.607 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906099+0000", 2024-05-08T08:55:15.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4992000000000003e-05 2024-05-08T08:55:15.607 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:15.607 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:15.607 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:15.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.607 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22091 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 913408~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:15.607 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:55:15.608 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 167.71827769399999, 2024-05-08T08:55:15.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048395290000000001, 2024-05-08T08:55:15.608 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:15.608 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:15.608 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:15.608 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:15.608 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:15.608 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:15.608 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22091 2024-05-08T08:55:15.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.608 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:15.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.608 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:15.608 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:55:15.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:15.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.609 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:15.609 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:55:15.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:15.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.609 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:15.609 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:55:15.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4900000000000001e-07 2024-05-08T08:55:15.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.609 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:15.609 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857870+0000", 2024-05-08T08:55:15.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2510000000000001e-06 2024-05-08T08:55:15.610 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.610 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:15.610 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857877+0000", 2024-05-08T08:55:15.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0809999999999996e-06 2024-05-08T08:55:15.610 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.611 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:15.611 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857881+0000", 2024-05-08T08:55:15.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.724e-06 2024-05-08T08:55:15.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.611 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:15.611 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906127+0000", 2024-05-08T08:55:15.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048245345000000002 2024-05-08T08:55:15.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.611 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:15.611 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906190+0000", 2024-05-08T08:55:15.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2762000000000007e-05 2024-05-08T08:55:15.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.612 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:15.612 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906262+0000", 2024-05-08T08:55:15.612 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2778e-05 2024-05-08T08:55:15.612 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:15.612 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:15.612 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:15.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.612 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22093 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 929792~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:15.612 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:55:15.612 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 167.71825533000001, 2024-05-08T08:55:15.612 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048532875000000003, 2024-05-08T08:55:15.612 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:15.613 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:15.613 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:15.613 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:15.613 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:15.613 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:15.613 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22093 2024-05-08T08:55:15.613 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.613 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:15.613 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.613 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:15.613 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:55:15.613 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:15.613 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.613 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.613 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:15.614 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:55:15.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:15.614 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.614 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.614 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:15.614 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857890+0000", 2024-05-08T08:55:15.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-07 2024-05-08T08:55:15.614 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.614 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.614 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:15.614 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857899+0000", 2024-05-08T08:55:15.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6889999999999999e-06 2024-05-08T08:55:15.614 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.614 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:15.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857906+0000", 2024-05-08T08:55:15.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6490000000000002e-06 2024-05-08T08:55:15.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:15.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857917+0000", 2024-05-08T08:55:15.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0764999999999999e-05 2024-05-08T08:55:15.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:15.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906287+0000", 2024-05-08T08:55:15.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048370442 2024-05-08T08:55:15.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:15.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906349+0000", 2024-05-08T08:55:15.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1537000000000004e-05 2024-05-08T08:55:15.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:15.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906422+0000", 2024-05-08T08:55:15.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3431999999999998e-05 2024-05-08T08:55:15.616 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:15.616 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:15.616 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:15.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.616 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22095 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 946176~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:15.617 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:55:15.617 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 167.718220626, 2024-05-08T08:55:15.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048657593999999998, 2024-05-08T08:55:15.617 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:15.617 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:15.617 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:15.617 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:15.617 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:15.617 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:15.617 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22095 2024-05-08T08:55:15.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.617 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:15.617 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.617 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:15.617 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:55:15.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:15.618 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.618 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.618 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:15.618 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:55:15.618 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:15.618 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.618 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.618 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:15.618 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857925+0000", 2024-05-08T08:55:15.618 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3099999999999998e-07 2024-05-08T08:55:15.618 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.618 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.618 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:15.618 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857927+0000", 2024-05-08T08:55:15.618 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.34e-06 2024-05-08T08:55:15.619 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.619 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.619 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:15.619 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857938+0000", 2024-05-08T08:55:15.619 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1426e-05 2024-05-08T08:55:15.619 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.619 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.619 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:15.619 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857943+0000", 2024-05-08T08:55:15.619 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5140000000000003e-06 2024-05-08T08:55:15.619 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.619 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.619 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:15.619 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906448+0000", 2024-05-08T08:55:15.620 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048504641000000001 2024-05-08T08:55:15.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.620 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:15.620 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906510+0000", 2024-05-08T08:55:15.620 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2663000000000006e-05 2024-05-08T08:55:15.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.620 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:15.620 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906582+0000", 2024-05-08T08:55:15.620 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1478999999999998e-05 2024-05-08T08:55:15.620 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:15.620 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:15.620 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:15.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.621 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22097 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 962560~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:15.621 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:55:15.621 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 167.718192937, 2024-05-08T08:55:15.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048791353000000003, 2024-05-08T08:55:15.621 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:15.621 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:15.621 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:15.621 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:15.621 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:15.621 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:15.621 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22097 2024-05-08T08:55:15.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.621 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:15.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:15.622 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:55:15.622 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:15.622 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.622 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.622 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:15.622 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:55:15.622 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:15.622 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.622 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.622 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:15.622 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:55:15.622 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9200000000000002e-07 2024-05-08T08:55:15.622 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.622 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.622 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:15.622 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857954+0000", 2024-05-08T08:55:15.623 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1900000000000002e-06 2024-05-08T08:55:15.623 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.623 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.623 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:15.623 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857961+0000", 2024-05-08T08:55:15.623 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4989999999999999e-06 2024-05-08T08:55:15.623 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.623 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.623 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:15.623 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857971+0000", 2024-05-08T08:55:15.623 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0485999999999999e-05 2024-05-08T08:55:15.623 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.623 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.623 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:15.623 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906607+0000", 2024-05-08T08:55:15.623 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048635109000000003 2024-05-08T08:55:15.624 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.624 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:15.624 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906667+0000", 2024-05-08T08:55:15.624 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0439000000000002e-05 2024-05-08T08:55:15.624 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.624 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:15.624 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906743+0000", 2024-05-08T08:55:15.624 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6237999999999998e-05 2024-05-08T08:55:15.624 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:15.624 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:15.624 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:15.624 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.624 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22099 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 978944~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:15.625 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:55:15.625 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 167.71816677699999, 2024-05-08T08:55:15.625 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048932055000000002, 2024-05-08T08:55:15.625 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:15.625 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:15.625 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:15.625 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:15.625 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:15.625 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:15.625 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22099 2024-05-08T08:55:15.625 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.625 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:15.625 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.625 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:15.625 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:55:15.625 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:15.626 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.626 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.626 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:15.626 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:55:15.626 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:15.626 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.626 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.626 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:15.626 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:55:15.626 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.27e-07 2024-05-08T08:55:15.626 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.626 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.626 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:15.627 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857987+0000", 2024-05-08T08:55:15.627 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5890000000000003e-06 2024-05-08T08:55:15.627 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.627 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.627 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:15.627 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857994+0000", 2024-05-08T08:55:15.627 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6619999999999999e-06 2024-05-08T08:55:15.627 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.627 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.627 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:15.627 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857998+0000", 2024-05-08T08:55:15.627 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7110000000000003e-06 2024-05-08T08:55:15.627 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.627 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.627 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:15.628 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906772+0000", 2024-05-08T08:55:15.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048773415000000001 2024-05-08T08:55:15.628 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.628 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:15.628 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906834+0000", 2024-05-08T08:55:15.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1752000000000001e-05 2024-05-08T08:55:15.628 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.628 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:15.628 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906910+0000", 2024-05-08T08:55:15.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6499000000000001e-05 2024-05-08T08:55:15.628 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:15.628 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:15.628 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:15.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.629 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22101 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 995328~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:15.629 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:55:15.629 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 167.718139646, 2024-05-08T08:55:15.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049087364000000001, 2024-05-08T08:55:15.629 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:15.629 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:15.629 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:15.629 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:15.629 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:15.629 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:15.629 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22101 2024-05-08T08:55:15.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.630 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:15.630 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.630 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:15.630 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:55:15.630 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:15.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.630 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.630 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:15.630 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:55:15.630 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:15.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.630 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.630 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:15.630 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858006+0000", 2024-05-08T08:55:15.630 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1399999999999997e-07 2024-05-08T08:55:15.631 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.631 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.631 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:15.631 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858008+0000", 2024-05-08T08:55:15.631 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2060000000000001e-06 2024-05-08T08:55:15.631 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.631 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.631 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:15.631 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858017+0000", 2024-05-08T08:55:15.631 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1579999999999994e-06 2024-05-08T08:55:15.631 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.631 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.631 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:15.631 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858022+0000", 2024-05-08T08:55:15.631 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4199999999999998e-06 2024-05-08T08:55:15.631 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.631 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.632 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:15.632 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906940+0000", 2024-05-08T08:55:15.632 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048917627999999998 2024-05-08T08:55:15.632 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.632 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.632 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:15.632 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907002+0000", 2024-05-08T08:55:15.632 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2399999999999999e-05 2024-05-08T08:55:15.632 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.632 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.632 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:15.632 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907092+0000", 2024-05-08T08:55:15.632 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0137999999999997e-05 2024-05-08T08:55:15.632 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:15.632 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:15.633 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:15.633 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.633 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.633 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22103 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1011712~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:15.633 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:55:15.633 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 167.718114861, 2024-05-08T08:55:15.633 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049228542, 2024-05-08T08:55:15.633 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:15.633 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:15.633 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:15.633 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:15.633 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:15.633 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:15.633 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22103 2024-05-08T08:55:15.633 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.634 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:15.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.634 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:15.634 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:55:15.634 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:15.634 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.634 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:15.634 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:55:15.634 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:15.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.635 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.635 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:15.635 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:55:15.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4299999999999999e-07 2024-05-08T08:55:15.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.635 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.635 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:15.635 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858032+0000", 2024-05-08T08:55:15.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0590000000000001e-06 2024-05-08T08:55:15.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.635 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.635 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:15.636 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858037+0000", 2024-05-08T08:55:15.636 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.977e-06 2024-05-08T08:55:15.636 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.636 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:15.636 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858042+0000", 2024-05-08T08:55:15.636 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7650000000000001e-06 2024-05-08T08:55:15.636 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.636 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:15.636 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907118+0000", 2024-05-08T08:55:15.636 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049076147000000001 2024-05-08T08:55:15.636 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.637 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.637 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:15.637 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907180+0000", 2024-05-08T08:55:15.637 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1971999999999996e-05 2024-05-08T08:55:15.637 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.637 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.637 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:15.637 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907258+0000", 2024-05-08T08:55:15.637 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8279e-05 2024-05-08T08:55:15.637 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:15.637 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:15.637 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:15.637 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.637 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.637 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22105 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1028096~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:15.638 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:55:15.638 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 167.71809561800001, 2024-05-08T08:55:15.638 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049394609999999999, 2024-05-08T08:55:15.638 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:15.638 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:15.638 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:15.638 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:15.638 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:15.638 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:15.638 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22105 2024-05-08T08:55:15.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.638 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:15.638 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.638 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:15.639 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:55:15.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:15.639 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.639 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.639 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:15.639 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:55:15.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:15.639 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.639 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.639 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:15.639 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858050+0000", 2024-05-08T08:55:15.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5999999999999999e-07 2024-05-08T08:55:15.639 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.639 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.639 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:15.640 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858051+0000", 2024-05-08T08:55:15.640 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8870000000000001e-06 2024-05-08T08:55:15.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.640 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.640 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:15.640 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858057+0000", 2024-05-08T08:55:15.640 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8019999999999997e-06 2024-05-08T08:55:15.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.640 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.640 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:15.640 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858062+0000", 2024-05-08T08:55:15.640 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.561e-06 2024-05-08T08:55:15.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.640 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.641 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:15.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907289+0000", 2024-05-08T08:55:15.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049226957000000002 2024-05-08T08:55:15.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.641 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:15.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907356+0000", 2024-05-08T08:55:15.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6915000000000006e-05 2024-05-08T08:55:15.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.641 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:15.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907444+0000", 2024-05-08T08:55:15.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8127999999999994e-05 2024-05-08T08:55:15.641 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:15.642 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:15.642 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:15.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.642 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22107 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1044480~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:15.642 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:55:15.642 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 167.71807727800001, 2024-05-08T08:55:15.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049799942999999999, 2024-05-08T08:55:15.642 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:15.642 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:15.642 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:15.642 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:15.642 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:15.642 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:15.642 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22107 2024-05-08T08:55:15.643 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.643 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:15.643 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.643 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:15.643 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:55:15.643 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:15.643 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.643 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.643 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:15.643 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:55:15.643 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:15.643 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.643 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.643 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:15.643 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:55:15.644 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4799999999999999e-07 2024-05-08T08:55:15.644 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.644 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.644 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:15.644 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858070+0000", 2024-05-08T08:55:15.644 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.012e-06 2024-05-08T08:55:15.644 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.644 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.644 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:15.644 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858077+0000", 2024-05-08T08:55:15.644 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6290000000000001e-06 2024-05-08T08:55:15.644 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.644 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.644 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:15.644 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858083+0000", 2024-05-08T08:55:15.644 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8880000000000002e-06 2024-05-08T08:55:15.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.645 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.645 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:15.645 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907671+0000", 2024-05-08T08:55:15.645 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049587191000000003 2024-05-08T08:55:15.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.645 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.645 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:15.645 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907763+0000", 2024-05-08T08:55:15.645 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2231000000000005e-05 2024-05-08T08:55:15.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.645 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.645 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:15.645 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907867+0000", 2024-05-08T08:55:15.645 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104644 2024-05-08T08:55:15.646 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:15.646 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:15.646 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:15.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.646 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22294 3.1c 3:3aad0389:::rbd_data.11406886b07b.0000000000000080:head [stat,delete] snapc 7=[7] RETRY=6 ondisk+retry+write+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:15.646 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:55:15.646 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 167.71748342999999, 2024-05-08T08:55:15.646 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051152191, 2024-05-08T08:55:15.646 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:15.646 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:15.646 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:15.646 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:15.647 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:15.647 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:15.647 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22294 2024-05-08T08:55:15.647 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.647 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:15.647 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.647 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:15.647 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:55:15.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:15.647 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.647 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.647 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:15.647 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:55:15.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:15.648 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.648 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.648 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:15.648 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858662+0000", 2024-05-08T08:55:15.648 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4799999999999998e-07 2024-05-08T08:55:15.648 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.648 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.648 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:15.648 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858664+0000", 2024-05-08T08:55:15.648 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1229999999999998e-06 2024-05-08T08:55:15.648 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.648 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.648 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:15.648 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858670+0000", 2024-05-08T08:55:15.648 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3099999999999997e-06 2024-05-08T08:55:15.649 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.649 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.649 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:15.649 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858676+0000", 2024-05-08T08:55:15.649 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9750000000000004e-06 2024-05-08T08:55:15.649 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.649 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.649 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:15.649 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907901+0000", 2024-05-08T08:55:15.649 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049224520000000001 2024-05-08T08:55:15.649 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.649 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.649 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:15.649 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.908085+0000", 2024-05-08T08:55:15.649 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00018377500000000001 2024-05-08T08:55:15.650 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.650 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.650 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:15.650 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.909814+0000", 2024-05-08T08:55:15.650 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00172894 2024-05-08T08:55:15.650 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:15.650 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:15.650 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:15.650 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.650 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.650 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4416.0:13461 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-05-08T08:55:15.650 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:22.487456+0000", 2024-05-08T08:55:15.650 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 113.088688389, 2024-05-08T08:55:15.650 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.99102620799999996, 2024-05-08T08:55:15.651 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:15.651 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:15.651 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:15.651 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:15.651 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4416", 2024-05-08T08:55:15.651 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:55:15.651 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 13461 2024-05-08T08:55:15.651 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.651 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:15.651 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.651 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:15.651 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487456+0000", 2024-05-08T08:55:15.651 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:15.651 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.651 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.652 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:15.652 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487456+0000", 2024-05-08T08:55:15.652 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:15.652 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.652 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.652 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:15.652 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487458+0000", 2024-05-08T08:55:15.652 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5659999999999999e-06 2024-05-08T08:55:15.652 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.652 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.652 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:15.652 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487463+0000", 2024-05-08T08:55:15.652 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6890000000000003e-06 2024-05-08T08:55:15.652 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.652 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.653 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:15.653 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487472+0000", 2024-05-08T08:55:15.653 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7710000000000001e-06 2024-05-08T08:55:15.653 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.653 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.653 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:15.653 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487478+0000", 2024-05-08T08:55:15.653 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1390000000000003e-06 2024-05-08T08:55:15.653 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.653 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.653 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:15.653 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487520+0000", 2024-05-08T08:55:15.653 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2259999999999999e-05 2024-05-08T08:55:15.653 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.653 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.654 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-05-08T08:55:15.654 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487528+0000", 2024-05-08T08:55:15.654 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7479999999999993e-06 2024-05-08T08:55:15.654 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.654 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.654 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:15.654 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:23.478139+0000", 2024-05-08T08:55:15.654 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.99061165200000001 2024-05-08T08:55:15.654 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.654 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.654 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:15.654 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:23.478301+0000", 2024-05-08T08:55:15.654 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000161602 2024-05-08T08:55:15.654 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.655 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.655 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:15.655 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:23.478483+0000", 2024-05-08T08:55:15.655 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000181781 2024-05-08T08:55:15.655 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:15.655 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:15.655 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:15.655 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.655 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.655 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5752.0:404 3.1a 3:5f5e8104:::rbd_data.1140bc4a55c7.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 5175296~45056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-05-08T08:55:15.655 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:00.390863+0000", 2024-05-08T08:55:15.655 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 75.185282193999996, 2024-05-08T08:55:15.655 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.055189174000000001, 2024-05-08T08:55:15.655 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:15.655 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:15.656 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:55:15.656 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:15.656 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5752", 2024-05-08T08:55:15.656 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/3534419815", 2024-05-08T08:55:15.656 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 404 2024-05-08T08:55:15.656 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.656 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:15.656 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.656 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:15.656 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.390863+0000", 2024-05-08T08:55:15.656 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:15.656 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.656 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.657 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:15.657 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.390863+0000", 2024-05-08T08:55:15.657 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:15.657 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.657 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.657 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:15.657 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.390864+0000", 2024-05-08T08:55:15.657 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.471e-06 2024-05-08T08:55:15.657 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.657 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.657 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:15.657 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397475+0000", 2024-05-08T08:55:15.657 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0066109109999999997 2024-05-08T08:55:15.657 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.657 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.657 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:15.658 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397498+0000", 2024-05-08T08:55:15.658 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2674e-05 2024-05-08T08:55:15.658 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.658 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.658 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:15.658 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397508+0000", 2024-05-08T08:55:15.658 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9019999999999994e-06 2024-05-08T08:55:15.658 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.658 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.658 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:15.658 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397609+0000", 2024-05-08T08:55:15.658 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010121899999999999 2024-05-08T08:55:15.658 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.658 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.658 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:15.659 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397711+0000", 2024-05-08T08:55:15.659 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102489 2024-05-08T08:55:15.659 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.659 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.659 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-05-08T08:55:15.659 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397999+0000", 2024-05-08T08:55:15.659 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00028733799999999999 2024-05-08T08:55:15.659 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.659 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.659 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:55:15.659 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.432093+0000", 2024-05-08T08:55:15.659 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.034094865000000002 2024-05-08T08:55:15.659 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.659 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.660 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:55:15.660 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.445975+0000", 2024-05-08T08:55:15.660 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013881187 2024-05-08T08:55:15.660 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.660 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.660 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:55:15.660 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.446013+0000", 2024-05-08T08:55:15.660 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8155000000000003e-05 2024-05-08T08:55:15.660 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:15.660 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:15.660 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:15.660 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.446052+0000", 2024-05-08T08:55:15.660 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8962999999999999e-05 2024-05-08T08:55:15.660 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:15.660 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:15.660 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:15.661 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:15.661 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:15.661 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:55:15.663 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:55:15.663 INFO:teuthology.orchestra.run.smithi059.stdout: "success": "" 2024-05-08T08:55:15.663 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:55:15.663 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:55:15.663 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:55:15.664+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:55:15.763 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:55:15.765+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:55:15.809 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:55:15.809 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:55:15.809 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:55:15.809 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:55:15.817 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:55:15.864 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:55:15.867+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:55:15.962 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:55:15.962 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:55:15.962 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:55:15.962 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:55:15.962 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:55:15.965 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:55:15.968+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:55:15.970 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:55:16.066 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:55:16.069+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:55:16.117 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:55:16.118 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:55:16.118 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:55:16.118 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:55:16.118 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.118 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6690 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1093632~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:16.118 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:55:16.118 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 164.40407898800001, 2024-05-08T08:55:16.118 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052462754, 2024-05-08T08:55:16.118 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:16.118 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:16.118 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:16.118 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:16.118 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:16.118 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:16.119 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6690 2024-05-08T08:55:16.119 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.119 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:16.119 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.119 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:16.119 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:55:16.119 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:16.119 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.119 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.119 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:16.119 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:55:16.119 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:16.119 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.119 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.119 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:16.120 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715346+0000", 2024-05-08T08:55:16.120 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.84e-07 2024-05-08T08:55:16.120 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.120 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.120 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:16.120 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715348+0000", 2024-05-08T08:55:16.120 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.114e-06 2024-05-08T08:55:16.120 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.120 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.120 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:16.120 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715353+0000", 2024-05-08T08:55:16.120 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5720000000000002e-06 2024-05-08T08:55:16.120 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.120 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.120 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:16.120 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715358+0000", 2024-05-08T08:55:16.120 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7140000000000001e-06 2024-05-08T08:55:16.121 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.121 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.121 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:16.121 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767638+0000", 2024-05-08T08:55:16.121 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052279562000000002 2024-05-08T08:55:16.121 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.121 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.121 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:16.121 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767713+0000", 2024-05-08T08:55:16.121 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5652e-05 2024-05-08T08:55:16.121 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.121 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.121 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:16.121 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767808+0000", 2024-05-08T08:55:16.122 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4755999999999999e-05 2024-05-08T08:55:16.122 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:16.122 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:16.122 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:16.122 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.122 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.122 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6692 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1110016~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:16.122 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:55:16.122 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 164.40405989800001, 2024-05-08T08:55:16.123 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052666495000000001, 2024-05-08T08:55:16.123 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:16.123 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:16.123 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:16.123 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:16.123 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:16.123 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:16.123 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6692 2024-05-08T08:55:16.123 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.123 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:16.123 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.123 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:16.123 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:55:16.123 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:16.123 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.123 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.123 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:16.124 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:55:16.124 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:16.124 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.124 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.124 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:16.124 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:55:16.124 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8599999999999999e-07 2024-05-08T08:55:16.124 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.124 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.124 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:16.124 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715372+0000", 2024-05-08T08:55:16.124 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0990000000000001e-06 2024-05-08T08:55:16.124 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.124 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.124 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:16.124 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715378+0000", 2024-05-08T08:55:16.125 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0680000000000002e-06 2024-05-08T08:55:16.125 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.125 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.125 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:16.125 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715383+0000", 2024-05-08T08:55:16.125 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8740000000000003e-06 2024-05-08T08:55:16.125 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.125 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.125 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:16.125 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767848+0000", 2024-05-08T08:55:16.125 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052464668999999998 2024-05-08T08:55:16.125 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.125 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.125 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:16.126 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767927+0000", 2024-05-08T08:55:16.126 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9228999999999996e-05 2024-05-08T08:55:16.126 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.126 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.126 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:16.126 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768031+0000", 2024-05-08T08:55:16.126 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010417 2024-05-08T08:55:16.126 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:16.126 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:16.127 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:16.127 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.127 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.127 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6694 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1126400~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:16.127 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:55:16.127 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 164.40403500100001, 2024-05-08T08:55:16.127 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052848829, 2024-05-08T08:55:16.127 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:16.127 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:16.127 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:16.127 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:16.127 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:16.127 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:16.127 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6694 2024-05-08T08:55:16.127 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.128 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:16.128 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.128 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:16.128 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:55:16.128 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:16.128 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.128 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.128 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:16.128 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:55:16.128 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:16.128 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.128 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.128 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:16.128 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715390+0000", 2024-05-08T08:55:16.128 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9900000000000001e-07 2024-05-08T08:55:16.129 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.129 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.129 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:16.129 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715392+0000", 2024-05-08T08:55:16.129 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8279999999999999e-06 2024-05-08T08:55:16.129 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.129 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.129 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:16.129 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715398+0000", 2024-05-08T08:55:16.129 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7619999999999998e-06 2024-05-08T08:55:16.129 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.129 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.129 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:16.129 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715403+0000", 2024-05-08T08:55:16.129 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7790000000000003e-06 2024-05-08T08:55:16.130 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.130 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.130 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:16.130 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768067+0000", 2024-05-08T08:55:16.130 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052663785999999997 2024-05-08T08:55:16.130 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.130 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.130 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:16.130 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768142+0000", 2024-05-08T08:55:16.130 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5018999999999994e-05 2024-05-08T08:55:16.130 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.130 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.130 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:16.130 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768238+0000", 2024-05-08T08:55:16.131 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6255999999999995e-05 2024-05-08T08:55:16.131 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:16.131 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:16.131 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:16.131 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.131 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.131 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6696 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1142784~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:16.131 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:55:16.131 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 164.404015493, 2024-05-08T08:55:16.131 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.053051057999999998, 2024-05-08T08:55:16.131 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:16.131 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:16.131 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:16.131 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:16.131 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:16.132 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:16.132 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6696 2024-05-08T08:55:16.132 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.132 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:16.132 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.132 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:16.132 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:55:16.132 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:16.132 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.132 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.132 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:16.132 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:55:16.132 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:16.132 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.133 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.133 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:16.133 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:55:16.133 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-05-08T08:55:16.133 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.133 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.133 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:16.133 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715411+0000", 2024-05-08T08:55:16.133 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9630000000000001e-06 2024-05-08T08:55:16.133 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.133 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.133 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:16.133 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715417+0000", 2024-05-08T08:55:16.133 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7250000000000002e-06 2024-05-08T08:55:16.133 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.133 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.134 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:16.134 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715421+0000", 2024-05-08T08:55:16.134 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4089999999999996e-06 2024-05-08T08:55:16.134 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.134 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.134 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:16.134 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768277+0000", 2024-05-08T08:55:16.134 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052855293999999997 2024-05-08T08:55:16.134 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.134 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.134 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:16.134 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768358+0000", 2024-05-08T08:55:16.134 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1720999999999999e-05 2024-05-08T08:55:16.134 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.134 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.135 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:16.135 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768460+0000", 2024-05-08T08:55:16.135 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010154600000000001 2024-05-08T08:55:16.135 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:16.135 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:16.135 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:16.135 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.135 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.135 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6698 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1159168~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:16.135 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:55:16.135 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 164.40399662600001, 2024-05-08T08:55:16.135 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.053245726, 2024-05-08T08:55:16.135 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:16.135 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:16.135 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:16.136 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:16.136 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:16.136 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:16.136 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6698 2024-05-08T08:55:16.136 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.136 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:16.136 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.136 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:16.136 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:55:16.136 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:16.136 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.137 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.137 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:16.137 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:55:16.137 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:16.137 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.137 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.137 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:16.137 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716936+0000", 2024-05-08T08:55:16.137 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001508665 2024-05-08T08:55:16.137 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.137 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.137 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:16.137 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716941+0000", 2024-05-08T08:55:16.137 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1910000000000002e-06 2024-05-08T08:55:16.137 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.137 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.137 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:16.138 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716950+0000", 2024-05-08T08:55:16.138 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1900000000000001e-06 2024-05-08T08:55:16.138 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.138 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.138 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:16.138 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716958+0000", 2024-05-08T08:55:16.138 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8019999999999992e-06 2024-05-08T08:55:16.138 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.138 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.138 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:16.138 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768499+0000", 2024-05-08T08:55:16.138 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051540926000000001 2024-05-08T08:55:16.138 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.138 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.139 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:16.139 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768574+0000", 2024-05-08T08:55:16.139 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4988999999999996e-05 2024-05-08T08:55:16.139 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.139 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.139 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:16.139 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768673+0000", 2024-05-08T08:55:16.139 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9963000000000005e-05 2024-05-08T08:55:16.139 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:16.139 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:16.139 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:16.139 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.139 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.139 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6706 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1224704~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:16.139 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:55:16.139 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 164.402391319, 2024-05-08T08:55:16.139 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052435598, 2024-05-08T08:55:16.140 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:16.140 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:16.140 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:16.140 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:16.140 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:16.140 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:16.140 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6706 2024-05-08T08:55:16.140 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.140 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:16.140 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.140 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:16.140 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:55:16.140 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:16.141 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.141 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.141 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:16.141 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:55:16.141 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:16.141 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.141 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.141 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:16.141 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:55:16.141 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:55:16.141 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.141 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.141 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:16.141 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717035+0000", 2024-05-08T08:55:16.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9290000000000001e-06 2024-05-08T08:55:16.142 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.142 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.142 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:16.142 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717041+0000", 2024-05-08T08:55:16.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6720000000000001e-06 2024-05-08T08:55:16.142 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.142 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.142 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:16.142 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717046+0000", 2024-05-08T08:55:16.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5650000000000003e-06 2024-05-08T08:55:16.142 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.142 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.142 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:16.143 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769311+0000", 2024-05-08T08:55:16.143 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052265645999999999 2024-05-08T08:55:16.143 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.143 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.143 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:16.143 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769379+0000", 2024-05-08T08:55:16.143 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7391000000000006e-05 2024-05-08T08:55:16.143 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.143 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.143 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:16.143 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769469+0000", 2024-05-08T08:55:16.143 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0025999999999995e-05 2024-05-08T08:55:16.143 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:16.143 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:16.144 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:16.144 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.144 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.144 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6708 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1241088~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:16.144 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:55:16.144 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 164.40237321500001, 2024-05-08T08:55:16.144 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052640552, 2024-05-08T08:55:16.144 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:16.144 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:16.144 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:16.144 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:16.144 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:16.144 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:16.144 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6708 2024-05-08T08:55:16.145 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.145 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:16.145 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.145 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:16.145 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:55:16.145 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:16.145 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.145 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.145 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:16.145 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:55:16.145 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:16.145 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.145 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.146 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:16.146 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717052+0000", 2024-05-08T08:55:16.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:55:16.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.146 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.146 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:16.146 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717053+0000", 2024-05-08T08:55:16.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8330000000000001e-06 2024-05-08T08:55:16.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.146 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.146 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:16.146 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717058+0000", 2024-05-08T08:55:16.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.933e-06 2024-05-08T08:55:16.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.146 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.147 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:16.147 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717063+0000", 2024-05-08T08:55:16.147 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2180000000000001e-06 2024-05-08T08:55:16.147 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.147 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.147 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:16.147 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769507+0000", 2024-05-08T08:55:16.147 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052444866999999999 2024-05-08T08:55:16.147 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.147 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.147 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:16.147 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769589+0000", 2024-05-08T08:55:16.147 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2039000000000005e-05 2024-05-08T08:55:16.147 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.147 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.148 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:16.148 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769692+0000", 2024-05-08T08:55:16.148 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102252 2024-05-08T08:55:16.148 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:16.148 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:16.148 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:16.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.148 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.148 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6710 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1257472~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:16.148 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:55:16.148 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 164.402355191, 2024-05-08T08:55:16.148 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.05283057, 2024-05-08T08:55:16.148 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:16.148 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:16.148 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:16.148 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:16.148 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:16.149 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:16.149 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6710 2024-05-08T08:55:16.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.149 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:16.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.149 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:16.149 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:55:16.149 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:16.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.149 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:16.149 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:55:16.149 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:16.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.149 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:16.150 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718951+0000", 2024-05-08T08:55:16.150 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0018817809999999999 2024-05-08T08:55:16.150 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.150 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.150 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:16.150 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718956+0000", 2024-05-08T08:55:16.150 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5399999999999997e-06 2024-05-08T08:55:16.150 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.150 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.150 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:16.150 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718964+0000", 2024-05-08T08:55:16.150 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6629999999999999e-06 2024-05-08T08:55:16.150 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.150 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.151 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:16.151 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718972+0000", 2024-05-08T08:55:16.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5229999999999998e-06 2024-05-08T08:55:16.151 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.151 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.151 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:16.151 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769729+0000", 2024-05-08T08:55:16.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050756959999999997 2024-05-08T08:55:16.151 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.151 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.151 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:16.151 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769803+0000", 2024-05-08T08:55:16.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4538999999999999e-05 2024-05-08T08:55:16.151 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.151 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.152 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:16.152 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769900+0000", 2024-05-08T08:55:16.152 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6564000000000006e-05 2024-05-08T08:55:16.152 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:16.152 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:16.152 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:16.152 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.152 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.152 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6726 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1388544~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:16.152 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:55:16.152 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 164.40028041900001, 2024-05-08T08:55:16.152 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052451506000000002, 2024-05-08T08:55:16.152 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:16.152 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:16.152 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:16.153 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:16.153 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:16.153 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:16.153 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6726 2024-05-08T08:55:16.153 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.153 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:16.153 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.153 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:16.153 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:55:16.153 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:16.153 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.153 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.153 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:16.153 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:55:16.153 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:16.153 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.154 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.154 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:16.154 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719200+0000", 2024-05-08T08:55:16.154 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5865999999999997e-05 2024-05-08T08:55:16.154 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.154 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.154 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:16.154 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719204+0000", 2024-05-08T08:55:16.154 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8630000000000001e-06 2024-05-08T08:55:16.154 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.154 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.154 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:16.154 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719215+0000", 2024-05-08T08:55:16.154 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1590000000000001e-05 2024-05-08T08:55:16.154 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.154 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.155 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:16.155 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719220+0000", 2024-05-08T08:55:16.155 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1619999999999999e-06 2024-05-08T08:55:16.155 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.155 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.155 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:16.155 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771425+0000", 2024-05-08T08:55:16.155 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052204249000000001 2024-05-08T08:55:16.155 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.155 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.155 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:16.155 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771498+0000", 2024-05-08T08:55:16.155 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3782999999999995e-05 2024-05-08T08:55:16.155 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.155 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.156 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:16.156 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771595+0000", 2024-05-08T08:55:16.156 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6992999999999998e-05 2024-05-08T08:55:16.156 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:16.156 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:16.156 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:16.156 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.156 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.156 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6728 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1404928~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:16.156 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:55:16.156 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 164.400193014, 2024-05-08T08:55:16.156 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052590747, 2024-05-08T08:55:16.156 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:16.156 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:16.156 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:16.156 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:16.157 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:16.157 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:16.157 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6728 2024-05-08T08:55:16.157 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.157 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:16.157 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.157 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:16.157 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:55:16.157 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:16.157 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.157 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.157 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:16.157 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:55:16.157 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:16.157 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.157 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.157 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:16.158 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719232+0000", 2024-05-08T08:55:16.158 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4000000000000001e-07 2024-05-08T08:55:16.158 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.158 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.158 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:16.158 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719234+0000", 2024-05-08T08:55:16.158 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4499999999999998e-06 2024-05-08T08:55:16.158 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.158 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.158 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:16.158 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719240+0000", 2024-05-08T08:55:16.158 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5720000000000002e-06 2024-05-08T08:55:16.158 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.158 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.158 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:16.158 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719245+0000", 2024-05-08T08:55:16.159 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8690000000000003e-06 2024-05-08T08:55:16.159 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.159 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.159 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:16.159 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771636+0000", 2024-05-08T08:55:16.159 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052390979999999997 2024-05-08T08:55:16.159 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.159 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.159 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:16.159 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771717+0000", 2024-05-08T08:55:16.159 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1291999999999993e-05 2024-05-08T08:55:16.159 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.159 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.159 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:16.159 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771822+0000", 2024-05-08T08:55:16.160 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104944 2024-05-08T08:55:16.160 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:16.160 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:16.160 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:16.160 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.160 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.160 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6730 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1421312~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:16.160 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:55:16.160 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 164.40017277199999, 2024-05-08T08:55:16.160 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052770477000000003, 2024-05-08T08:55:16.160 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:16.160 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:16.160 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:16.160 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:16.160 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:16.161 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:16.161 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6730 2024-05-08T08:55:16.161 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.161 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:16.161 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.161 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:16.161 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:55:16.161 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:16.161 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.161 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.161 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:16.161 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:55:16.161 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:16.161 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.161 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.161 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:16.161 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:55:16.162 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9000000000000002e-07 2024-05-08T08:55:16.162 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.162 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.162 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:16.162 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719254+0000", 2024-05-08T08:55:16.162 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4099999999999998e-06 2024-05-08T08:55:16.162 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.162 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.162 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:16.162 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719263+0000", 2024-05-08T08:55:16.162 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4109999999999996e-06 2024-05-08T08:55:16.162 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.162 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.162 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:16.163 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719268+0000", 2024-05-08T08:55:16.163 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2669999999999997e-06 2024-05-08T08:55:16.163 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.163 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.163 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:16.163 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771846+0000", 2024-05-08T08:55:16.163 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052577677000000003 2024-05-08T08:55:16.163 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.163 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.163 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:16.163 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771920+0000", 2024-05-08T08:55:16.163 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4460000000000002e-05 2024-05-08T08:55:16.163 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.163 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.163 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:16.164 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772022+0000", 2024-05-08T08:55:16.164 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010186199999999999 2024-05-08T08:55:16.164 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:16.164 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:16.164 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:16.164 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.164 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.164 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6732 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1437696~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:16.164 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:55:16.164 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 164.400147965, 2024-05-08T08:55:16.164 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052905575000000003, 2024-05-08T08:55:16.164 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:16.164 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:16.165 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:16.165 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:16.165 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:16.165 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:16.165 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6732 2024-05-08T08:55:16.165 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.165 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:16.165 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.165 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:16.165 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:55:16.165 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:16.165 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.165 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.165 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:16.166 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:55:16.166 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:16.166 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.166 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.166 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:16.166 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721392+0000", 2024-05-08T08:55:16.166 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002115287 2024-05-08T08:55:16.166 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.166 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.166 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:16.166 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721396+0000", 2024-05-08T08:55:16.167 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1679999999999997e-06 2024-05-08T08:55:16.167 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.167 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.167 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:16.167 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721404+0000", 2024-05-08T08:55:16.167 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4409999999999993e-06 2024-05-08T08:55:16.167 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.167 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.167 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:16.167 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721410+0000", 2024-05-08T08:55:16.167 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9150000000000001e-06 2024-05-08T08:55:16.167 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.167 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.167 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:16.168 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772061+0000", 2024-05-08T08:55:16.168 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050651068 2024-05-08T08:55:16.168 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.168 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.168 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:16.168 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772127+0000", 2024-05-08T08:55:16.168 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5424000000000002e-05 2024-05-08T08:55:16.168 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.168 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.168 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:16.168 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772182+0000", 2024-05-08T08:55:16.168 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5272000000000002e-05 2024-05-08T08:55:16.168 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:16.168 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:16.169 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:16.169 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.169 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.169 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6749 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1585152~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:16.169 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:55:16.169 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 164.397847195, 2024-05-08T08:55:16.169 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052350264000000001, 2024-05-08T08:55:16.169 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:16.169 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:16.169 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:16.169 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:16.169 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:16.169 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:16.169 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6749 2024-05-08T08:55:16.169 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.170 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:16.170 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.170 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:16.170 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:55:16.170 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:16.170 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.170 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.170 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:16.170 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:55:16.170 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:16.170 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.170 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.170 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:16.170 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721578+0000", 2024-05-08T08:55:16.170 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0399999999999996e-07 2024-05-08T08:55:16.171 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.171 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.171 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:16.171 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721580+0000", 2024-05-08T08:55:16.171 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5450000000000002e-06 2024-05-08T08:55:16.171 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.171 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.171 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:16.171 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721587+0000", 2024-05-08T08:55:16.171 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0190000000000001e-06 2024-05-08T08:55:16.171 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.171 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.171 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:16.172 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721593+0000", 2024-05-08T08:55:16.172 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5659999999999997e-06 2024-05-08T08:55:16.172 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.172 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.173 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:16.173 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773759+0000", 2024-05-08T08:55:16.173 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052166553999999997 2024-05-08T08:55:16.173 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.173 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.173 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:16.173 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773833+0000", 2024-05-08T08:55:16.173 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3913999999999997e-05 2024-05-08T08:55:16.173 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.173 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.173 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:16.173 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773927+0000", 2024-05-08T08:55:16.173 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4061999999999995e-05 2024-05-08T08:55:16.173 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:16.173 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:16.173 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:16.174 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.174 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.174 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6751 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1601536~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:16.174 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:55:16.174 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 164.39782391599999, 2024-05-08T08:55:16.174 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052461302000000001, 2024-05-08T08:55:16.174 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:16.174 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:16.174 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:16.174 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:16.174 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:16.174 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:16.174 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6751 2024-05-08T08:55:16.174 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.174 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:16.175 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.175 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:16.175 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:55:16.175 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:16.175 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.175 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.175 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:16.175 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:55:16.175 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:16.175 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.175 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.175 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:16.175 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722939+0000", 2024-05-08T08:55:16.175 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001338601 2024-05-08T08:55:16.175 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.176 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.176 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:16.176 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722944+0000", 2024-05-08T08:55:16.176 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4469999999999999e-06 2024-05-08T08:55:16.176 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.176 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.176 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:16.176 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722955+0000", 2024-05-08T08:55:16.176 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1892999999999999e-05 2024-05-08T08:55:16.176 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.176 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.176 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:16.176 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722963+0000", 2024-05-08T08:55:16.176 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1080000000000004e-06 2024-05-08T08:55:16.176 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.176 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.177 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:16.177 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773964+0000", 2024-05-08T08:55:16.177 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051001022 2024-05-08T08:55:16.177 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.177 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.177 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:16.177 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.774006+0000", 2024-05-08T08:55:16.177 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2729999999999999e-05 2024-05-08T08:55:16.177 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.177 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.177 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:16.177 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.774062+0000", 2024-05-08T08:55:16.177 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5501000000000003e-05 2024-05-08T08:55:16.177 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:16.177 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:16.177 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:16.178 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.178 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.178 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6760 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1683456~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:16.178 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:55:16.178 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 164.39631453999999, 2024-05-08T08:55:16.178 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052571312000000002, 2024-05-08T08:55:16.178 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:16.178 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:16.178 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:16.178 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:16.178 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:16.178 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:16.178 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6760 2024-05-08T08:55:16.178 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.178 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:16.178 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.179 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:16.179 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:55:16.179 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:16.179 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.179 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.179 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:16.179 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:55:16.179 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:16.179 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.179 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.179 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:16.179 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:55:16.179 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2100000000000002e-07 2024-05-08T08:55:16.179 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.180 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.180 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:16.180 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723113+0000", 2024-05-08T08:55:16.180 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3149999999999999e-06 2024-05-08T08:55:16.180 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.180 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.180 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:16.180 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723118+0000", 2024-05-08T08:55:16.180 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8860000000000001e-06 2024-05-08T08:55:16.180 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.180 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.180 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:16.180 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723123+0000", 2024-05-08T08:55:16.180 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6770000000000003e-06 2024-05-08T08:55:16.180 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.180 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.181 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:16.181 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775414+0000", 2024-05-08T08:55:16.181 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052290692999999999 2024-05-08T08:55:16.181 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.181 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.181 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:16.181 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775459+0000", 2024-05-08T08:55:16.181 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5639999999999997e-05 2024-05-08T08:55:16.181 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.181 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.181 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:16.181 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775681+0000", 2024-05-08T08:55:16.181 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00022168000000000001 2024-05-08T08:55:16.181 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:16.181 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:16.181 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:16.181 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.182 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.182 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6761 3.16 3:6bd6d6da:::rbd_data.1140782590a5.0000000000000015:6 [read 1095680~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:16.182 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:55:16.182 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 164.39629532000001, 2024-05-08T08:55:16.182 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052751275, 2024-05-08T08:55:16.182 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:16.182 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:16.182 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:16.182 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:16.182 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:16.182 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:16.182 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6761 2024-05-08T08:55:16.182 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.182 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:16.182 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.182 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:16.183 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:55:16.183 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:16.183 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.183 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.183 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:16.183 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:55:16.183 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:16.183 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.183 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.183 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:16.183 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724939+0000", 2024-05-08T08:55:16.183 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001809817 2024-05-08T08:55:16.183 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.183 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.183 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:16.183 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724943+0000", 2024-05-08T08:55:16.184 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4920000000000004e-06 2024-05-08T08:55:16.184 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.184 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.184 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:16.184 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724956+0000", 2024-05-08T08:55:16.184 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2427e-05 2024-05-08T08:55:16.184 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.184 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.184 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:16.184 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724963+0000", 2024-05-08T08:55:16.184 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2559999999999996e-06 2024-05-08T08:55:16.184 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.184 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.184 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:16.184 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775711+0000", 2024-05-08T08:55:16.184 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050747621 2024-05-08T08:55:16.185 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.185 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.185 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:16.185 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775785+0000", 2024-05-08T08:55:16.185 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4634000000000004e-05 2024-05-08T08:55:16.185 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.185 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.185 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:16.185 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775880+0000", 2024-05-08T08:55:16.185 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5027999999999999e-05 2024-05-08T08:55:16.185 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:16.185 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:16.185 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:16.185 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.186 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.186 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6770 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1765376~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:16.186 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:55:16.186 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 164.394264225, 2024-05-08T08:55:16.186 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052339296, 2024-05-08T08:55:16.186 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:16.186 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:16.186 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:16.186 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:16.186 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:16.186 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:16.186 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6770 2024-05-08T08:55:16.186 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.186 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:16.187 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.187 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:16.187 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:55:16.187 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:16.187 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.187 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.187 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:16.187 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:55:16.187 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:16.187 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.187 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.187 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:16.187 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725161+0000", 2024-05-08T08:55:16.187 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-05-08T08:55:16.187 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.187 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.188 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:16.188 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725163+0000", 2024-05-08T08:55:16.188 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4370000000000001e-06 2024-05-08T08:55:16.188 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.188 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.188 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:16.188 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725169+0000", 2024-05-08T08:55:16.188 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8409999999999996e-06 2024-05-08T08:55:16.188 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.188 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.188 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:16.188 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725174+0000", 2024-05-08T08:55:16.188 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7249999999999997e-06 2024-05-08T08:55:16.188 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.188 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.188 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:16.189 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777343+0000", 2024-05-08T08:55:16.189 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052169633999999999 2024-05-08T08:55:16.189 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.189 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.189 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:16.189 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777409+0000", 2024-05-08T08:55:16.189 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6033999999999998e-05 2024-05-08T08:55:16.189 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.189 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.189 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:16.189 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777499+0000", 2024-05-08T08:55:16.189 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0224999999999998e-05 2024-05-08T08:55:16.189 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:16.189 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:16.190 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:16.190 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.190 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.190 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6771 3.16 3:6bd6d6da:::rbd_data.1140782590a5.0000000000000015:6 [read 1177600~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:16.190 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:55:16.190 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 164.39424440799999, 2024-05-08T08:55:16.190 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052438604999999999, 2024-05-08T08:55:16.190 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:16.190 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:16.190 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:16.190 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:16.190 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:16.190 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:16.190 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6771 2024-05-08T08:55:16.191 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.191 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:16.191 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.191 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:16.191 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:55:16.191 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:16.191 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.191 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.191 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:16.191 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:55:16.191 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:16.191 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.191 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.191 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:16.191 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:55:16.191 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:55:16.191 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.192 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.192 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:16.192 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725188+0000", 2024-05-08T08:55:16.192 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1579999999999999e-06 2024-05-08T08:55:16.192 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.192 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.192 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:16.192 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725195+0000", 2024-05-08T08:55:16.192 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8909999999999995e-06 2024-05-08T08:55:16.192 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.192 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.192 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:16.192 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725200+0000", 2024-05-08T08:55:16.192 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9810000000000003e-06 2024-05-08T08:55:16.192 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.193 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.193 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:16.193 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777527+0000", 2024-05-08T08:55:16.193 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052326582000000003 2024-05-08T08:55:16.193 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.193 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.193 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:16.193 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777566+0000", 2024-05-08T08:55:16.193 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9447999999999999e-05 2024-05-08T08:55:16.193 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.193 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.193 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:16.193 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777619+0000", 2024-05-08T08:55:16.193 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2135000000000001e-05 2024-05-08T08:55:16.193 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:16.193 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:16.193 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:16.194 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.194 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.194 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6772 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1781760~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:16.194 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:55:16.194 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 164.39421635100001, 2024-05-08T08:55:16.194 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052601799999999997, 2024-05-08T08:55:16.194 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:16.194 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:16.194 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:16.194 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:16.194 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:16.194 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:16.194 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6772 2024-05-08T08:55:16.194 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.195 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:16.195 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.195 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:16.195 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:55:16.195 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:16.195 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.195 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.195 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:16.195 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:55:16.195 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:16.195 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.195 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.195 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:16.195 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726939+0000", 2024-05-08T08:55:16.195 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001730493 2024-05-08T08:55:16.196 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.196 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.196 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:16.196 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726943+0000", 2024-05-08T08:55:16.196 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7330000000000003e-06 2024-05-08T08:55:16.196 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.196 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.196 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:16.196 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726954+0000", 2024-05-08T08:55:16.196 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1107000000000001e-05 2024-05-08T08:55:16.196 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.196 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.196 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:16.196 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726962+0000", 2024-05-08T08:55:16.197 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.503e-06 2024-05-08T08:55:16.197 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.197 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.197 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:16.197 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777653+0000", 2024-05-08T08:55:16.197 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050691542999999999 2024-05-08T08:55:16.197 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.197 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.197 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:16.197 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777719+0000", 2024-05-08T08:55:16.197 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5228000000000006e-05 2024-05-08T08:55:16.197 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.197 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.197 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:16.197 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777810+0000", 2024-05-08T08:55:16.198 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1193000000000006e-05 2024-05-08T08:55:16.198 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:16.198 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:16.198 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:16.198 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.198 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.198 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6782 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1880064~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:16.198 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:55:16.198 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 164.392247844, 2024-05-08T08:55:16.198 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052326492000000002, 2024-05-08T08:55:16.198 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:16.198 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:16.198 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:16.198 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:16.198 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:16.199 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:16.199 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6782 2024-05-08T08:55:16.199 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.199 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:16.199 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.199 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:16.199 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:55:16.199 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:16.199 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.199 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.199 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:16.199 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:55:16.199 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:16.199 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.199 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.200 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:16.200 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728939+0000", 2024-05-08T08:55:16.200 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001762719 2024-05-08T08:55:16.200 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.200 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.200 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:16.200 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728944+0000", 2024-05-08T08:55:16.200 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.746e-06 2024-05-08T08:55:16.200 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.200 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.200 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:16.200 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728955+0000", 2024-05-08T08:55:16.200 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1043000000000001e-05 2024-05-08T08:55:16.200 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.200 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.201 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:16.201 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728963+0000", 2024-05-08T08:55:16.201 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5809999999999999e-06 2024-05-08T08:55:16.201 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.201 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.201 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:16.201 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779320+0000", 2024-05-08T08:55:16.201 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050357290999999998 2024-05-08T08:55:16.201 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.201 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.201 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:16.201 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779401+0000", 2024-05-08T08:55:16.201 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0574000000000005e-05 2024-05-08T08:55:16.201 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:16.201 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:16.202 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:16.202 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779503+0000", 2024-05-08T08:55:16.202 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102538 2024-05-08T08:55:16.202 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:16.202 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:16.202 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:16.202 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:16.202 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:16.202 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:55:16.204 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:55:16.204 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:55:16.206+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:55:16.304 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:55:16.306+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:55:16.323 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:55:16.324 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:55:16.324 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:55:16.324 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:55:16.332 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:55:16.416 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:55:16.419+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:55:16.488 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:55:16.488 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:55:16.488 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:55:16.488 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:55:16.488 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:55:16.497 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:55:16.539 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:55:16.541+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:55:16.606 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:55:16.609+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:55:16.649 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:55:16.650 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:55:16.650 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:55:16.650 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:55:16.650 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.650 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4621.0:9089 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 23339520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:55:16.650 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:55:16.650 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 164.15842040000001, 2024-05-08T08:55:16.650 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.13814907200000001, 2024-05-08T08:55:16.650 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:16.650 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:16.650 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:55:16.651 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:16.651 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4621", 2024-05-08T08:55:16.651 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:16.651 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9089 2024-05-08T08:55:16.651 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.651 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:16.651 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.651 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:16.651 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:55:16.651 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:16.651 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.651 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.651 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:16.652 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:55:16.652 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:16.652 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.652 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.652 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:16.652 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493005+0000", 2024-05-08T08:55:16.652 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1700000000000001e-06 2024-05-08T08:55:16.652 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.652 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.652 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:16.652 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.513963+0000", 2024-05-08T08:55:16.652 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.020957581999999999 2024-05-08T08:55:16.652 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.652 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.653 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:16.653 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.513995+0000", 2024-05-08T08:55:16.653 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1993999999999999e-05 2024-05-08T08:55:16.653 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.653 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.653 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:16.653 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514007+0000", 2024-05-08T08:55:16.653 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1581e-05 2024-05-08T08:55:16.653 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.653 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.653 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:16.653 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514122+0000", 2024-05-08T08:55:16.653 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000115282 2024-05-08T08:55:16.653 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.653 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.654 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op must wait for map", 2024-05-08T08:55:16.654 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514128+0000", 2024-05-08T08:55:16.654 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9950000000000002e-06 2024-05-08T08:55:16.654 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.654 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.654 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:16.654 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.515800+0000", 2024-05-08T08:55:16.654 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001671886 2024-05-08T08:55:16.654 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.654 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.654 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:16.654 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.515893+0000", 2024-05-08T08:55:16.654 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3490000000000001e-05 2024-05-08T08:55:16.654 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.655 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.655 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 3", 2024-05-08T08:55:16.655 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.517647+0000", 2024-05-08T08:55:16.655 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0017535630000000001 2024-05-08T08:55:16.655 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.655 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.655 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:55:16.655 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.627578+0000", 2024-05-08T08:55:16.655 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.109931557 2024-05-08T08:55:16.655 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.655 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.655 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:55:16.655 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631055+0000", 2024-05-08T08:55:16.655 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0034761420000000002 2024-05-08T08:55:16.656 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.656 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.656 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:55:16.656 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631096+0000", 2024-05-08T08:55:16.656 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1059000000000001e-05 2024-05-08T08:55:16.656 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.656 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.656 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:16.656 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631151+0000", 2024-05-08T08:55:16.656 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5770999999999998e-05 2024-05-08T08:55:16.656 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:16.656 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:16.656 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:16.657 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.657 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.657 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1071 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11026432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:55:16.657 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:55:16.657 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 160.16597090900001, 2024-05-08T08:55:16.657 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070580037999999998, 2024-05-08T08:55:16.657 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:16.657 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:16.657 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:16.657 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:16.657 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:55:16.657 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:55:16.657 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1071 2024-05-08T08:55:16.657 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.658 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:16.658 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.658 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:16.658 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:55:16.658 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:16.658 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.658 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.658 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:16.658 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:55:16.658 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:16.658 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.658 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.658 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:16.659 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:55:16.659 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2800000000000002e-07 2024-05-08T08:55:16.659 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.659 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.659 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:16.659 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485454+0000", 2024-05-08T08:55:16.659 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2290000000000002e-06 2024-05-08T08:55:16.659 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.659 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.659 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:16.659 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485461+0000", 2024-05-08T08:55:16.659 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9920000000000001e-06 2024-05-08T08:55:16.659 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.659 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.659 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:16.660 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485467+0000", 2024-05-08T08:55:16.660 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3979999999999998e-06 2024-05-08T08:55:16.660 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.660 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.660 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:16.660 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555880+0000", 2024-05-08T08:55:16.660 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070413165 2024-05-08T08:55:16.660 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.660 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.660 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:16.660 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555926+0000", 2024-05-08T08:55:16.660 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6217000000000002e-05 2024-05-08T08:55:16.660 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.660 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.660 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:16.661 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556032+0000", 2024-05-08T08:55:16.661 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000105609 2024-05-08T08:55:16.661 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:16.661 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:16.661 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:16.661 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.661 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.661 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1073 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11067392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:55:16.661 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:55:16.661 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 160.16594968800001, 2024-05-08T08:55:16.661 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070821404000000004, 2024-05-08T08:55:16.661 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:16.661 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:16.661 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:16.661 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:16.662 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:55:16.662 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:55:16.662 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1073 2024-05-08T08:55:16.662 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.662 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:16.662 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.662 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:16.662 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:55:16.662 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:16.662 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.662 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.662 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:16.662 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:55:16.662 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:16.663 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.663 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.663 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:16.663 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:55:16.663 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.08e-07 2024-05-08T08:55:16.663 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.663 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.663 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:16.663 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485476+0000", 2024-05-08T08:55:16.663 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3929999999999998e-06 2024-05-08T08:55:16.663 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.663 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.663 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:16.663 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485485+0000", 2024-05-08T08:55:16.664 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1880000000000008e-06 2024-05-08T08:55:16.664 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.664 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.664 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:16.664 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485494+0000", 2024-05-08T08:55:16.664 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3109999999999995e-06 2024-05-08T08:55:16.664 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.664 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.664 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:16.664 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556082+0000", 2024-05-08T08:55:16.664 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070587426999999994 2024-05-08T08:55:16.664 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.664 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.664 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:16.665 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556177+0000", 2024-05-08T08:55:16.665 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5051999999999997e-05 2024-05-08T08:55:16.665 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.665 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.665 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:16.665 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556294+0000", 2024-05-08T08:55:16.665 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000117625 2024-05-08T08:55:16.665 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:16.665 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:16.665 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:16.665 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.665 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.665 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1075 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11108352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:55:16.665 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:55:16.665 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 160.16591911099999, 2024-05-08T08:55:16.666 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071045026999999997, 2024-05-08T08:55:16.666 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:16.666 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:16.666 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:16.666 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:16.666 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:55:16.666 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:55:16.666 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1075 2024-05-08T08:55:16.666 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.666 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:16.666 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.666 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:16.666 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:55:16.666 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:16.667 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.667 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.667 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:16.667 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:55:16.667 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:16.667 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.667 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.667 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:16.667 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:55:16.667 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.32e-07 2024-05-08T08:55:16.667 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.667 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.667 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:16.668 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485518+0000", 2024-05-08T08:55:16.668 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3769e-05 2024-05-08T08:55:16.668 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.668 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.668 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:16.668 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485527+0000", 2024-05-08T08:55:16.668 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6779999999999995e-06 2024-05-08T08:55:16.668 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.668 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.668 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:16.668 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485532+0000", 2024-05-08T08:55:16.668 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9830000000000004e-06 2024-05-08T08:55:16.668 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.668 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.669 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:16.669 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556340+0000", 2024-05-08T08:55:16.669 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070807743000000006 2024-05-08T08:55:16.669 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.669 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.669 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:16.669 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556435+0000", 2024-05-08T08:55:16.669 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4732e-05 2024-05-08T08:55:16.669 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.669 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.669 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:16.669 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556549+0000", 2024-05-08T08:55:16.669 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011369 2024-05-08T08:55:16.669 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:16.670 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:16.670 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:16.670 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.670 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.670 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1078 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11149312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:55:16.670 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:55:16.670 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 160.16587891200001, 2024-05-08T08:55:16.670 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071167276000000002, 2024-05-08T08:55:16.670 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:16.670 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:16.670 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:16.670 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:16.670 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:55:16.670 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:55:16.670 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1078 2024-05-08T08:55:16.671 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.671 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:16.671 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.671 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:16.671 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:55:16.671 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:16.671 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.671 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.671 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:16.671 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:55:16.671 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:16.671 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.671 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.671 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:16.671 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:55:16.671 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5600000000000001e-07 2024-05-08T08:55:16.672 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.672 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.672 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:16.672 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485547+0000", 2024-05-08T08:55:16.672 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3590000000000002e-06 2024-05-08T08:55:16.672 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.672 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.672 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:16.672 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485553+0000", 2024-05-08T08:55:16.672 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2770000000000002e-06 2024-05-08T08:55:16.672 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.672 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.672 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:16.672 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485558+0000", 2024-05-08T08:55:16.672 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.95e-06 2024-05-08T08:55:16.672 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.673 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.673 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:16.673 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556582+0000", 2024-05-08T08:55:16.673 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071024444000000006 2024-05-08T08:55:16.673 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.673 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.673 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:16.673 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556637+0000", 2024-05-08T08:55:16.673 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4672000000000001e-05 2024-05-08T08:55:16.673 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.673 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.673 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:16.673 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556711+0000", 2024-05-08T08:55:16.673 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4117999999999997e-05 2024-05-08T08:55:16.673 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:16.673 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:16.674 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:16.674 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.674 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.674 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1080 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11190272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:55:16.674 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:55:16.674 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 160.165858179, 2024-05-08T08:55:16.674 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071350721000000006, 2024-05-08T08:55:16.674 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:16.674 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:16.674 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:16.674 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:16.674 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:55:16.674 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:55:16.674 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1080 2024-05-08T08:55:16.674 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.675 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:16.675 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.675 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:16.675 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:55:16.675 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:16.675 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.675 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.675 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:16.675 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:55:16.675 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:16.675 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.675 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.675 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:16.675 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:55:16.675 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.03e-07 2024-05-08T08:55:16.676 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.676 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.676 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:16.676 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485567+0000", 2024-05-08T08:55:16.676 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1090000000000001e-06 2024-05-08T08:55:16.676 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.676 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.676 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:16.676 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485573+0000", 2024-05-08T08:55:16.676 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3049999999999998e-06 2024-05-08T08:55:16.676 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.676 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.676 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:16.676 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485578+0000", 2024-05-08T08:55:16.676 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0130000000000001e-06 2024-05-08T08:55:16.677 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.677 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.677 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:16.677 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556743+0000", 2024-05-08T08:55:16.677 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071164449000000005 2024-05-08T08:55:16.677 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.677 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.677 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:16.677 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556797+0000", 2024-05-08T08:55:16.677 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4589999999999997e-05 2024-05-08T08:55:16.677 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.677 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.677 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:16.677 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556915+0000", 2024-05-08T08:55:16.678 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000117852 2024-05-08T08:55:16.678 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:16.678 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:16.678 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:16.678 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.678 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.678 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1082 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11231232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:55:16.678 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:55:16.678 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 160.165837936, 2024-05-08T08:55:16.678 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071591396000000002, 2024-05-08T08:55:16.678 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:16.678 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:16.678 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:16.678 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:16.679 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:55:16.679 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:55:16.679 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1082 2024-05-08T08:55:16.679 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.679 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:16.679 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.679 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:16.679 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:55:16.679 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:16.679 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.679 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.679 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:16.679 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:55:16.679 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:16.679 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.680 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.680 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:16.680 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:55:16.680 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1800000000000001e-07 2024-05-08T08:55:16.680 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.680 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.680 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:16.680 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485587+0000", 2024-05-08T08:55:16.680 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2589999999999999e-06 2024-05-08T08:55:16.680 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.680 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.680 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:16.680 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485594+0000", 2024-05-08T08:55:16.680 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0789999999999997e-06 2024-05-08T08:55:16.680 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.681 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.681 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:16.681 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485598+0000", 2024-05-08T08:55:16.681 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7480000000000001e-06 2024-05-08T08:55:16.681 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.681 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.681 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:16.681 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556971+0000", 2024-05-08T08:55:16.681 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071372774999999999 2024-05-08T08:55:16.681 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.681 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.681 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:16.681 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557069+0000", 2024-05-08T08:55:16.681 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.7732999999999994e-05 2024-05-08T08:55:16.681 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.682 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.682 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:16.682 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557176+0000", 2024-05-08T08:55:16.682 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000107384 2024-05-08T08:55:16.682 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:16.682 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:16.682 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:16.682 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.682 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.682 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1084 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11272192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:55:16.682 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:55:16.682 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 160.16579929700001, 2024-05-08T08:55:16.682 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071808977999999996, 2024-05-08T08:55:16.683 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:16.683 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:16.683 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:16.683 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:16.683 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:55:16.683 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:55:16.683 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1084 2024-05-08T08:55:16.683 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.683 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:16.683 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.683 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:16.683 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:55:16.683 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:16.683 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.683 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.684 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:16.684 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:55:16.684 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:16.684 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.684 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.684 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:16.684 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485624+0000", 2024-05-08T08:55:16.684 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:55:16.684 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.684 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.684 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:16.684 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485626+0000", 2024-05-08T08:55:16.684 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9460000000000001e-06 2024-05-08T08:55:16.684 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.685 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.685 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:16.685 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485633+0000", 2024-05-08T08:55:16.685 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1149999999999996e-06 2024-05-08T08:55:16.685 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.685 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.685 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:16.685 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485638+0000", 2024-05-08T08:55:16.685 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5670000000000002e-06 2024-05-08T08:55:16.685 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.685 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.685 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:16.685 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557226+0000", 2024-05-08T08:55:16.685 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071587267999999996 2024-05-08T08:55:16.685 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.685 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.686 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:16.686 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557321+0000", 2024-05-08T08:55:16.686 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5785000000000005e-05 2024-05-08T08:55:16.686 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.686 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.686 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:16.686 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557432+0000", 2024-05-08T08:55:16.686 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011092800000000001 2024-05-08T08:55:16.686 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:16.686 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:16.686 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:16.686 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.686 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.686 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1086 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11313152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:55:16.687 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:55:16.687 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 160.16577597, 2024-05-08T08:55:16.687 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071933132999999996, 2024-05-08T08:55:16.687 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:16.687 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:16.687 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:16.687 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:16.687 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:55:16.687 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:55:16.687 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1086 2024-05-08T08:55:16.687 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.687 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:16.687 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.687 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:16.687 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:55:16.687 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:16.687 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.688 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.688 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:16.688 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:55:16.688 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:16.688 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.688 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.688 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:16.688 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486328+0000", 2024-05-08T08:55:16.688 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00068107900000000002 2024-05-08T08:55:16.688 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.688 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.688 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:16.688 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486331+0000", 2024-05-08T08:55:16.688 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.585e-06 2024-05-08T08:55:16.688 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.688 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.688 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:16.689 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486343+0000", 2024-05-08T08:55:16.689 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1856e-05 2024-05-08T08:55:16.689 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.689 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.689 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:16.689 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486350+0000", 2024-05-08T08:55:16.689 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8600000000000004e-06 2024-05-08T08:55:16.689 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.689 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.689 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:16.689 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557470+0000", 2024-05-08T08:55:16.689 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071120326999999997 2024-05-08T08:55:16.689 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.689 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.689 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:16.689 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557514+0000", 2024-05-08T08:55:16.690 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3909000000000003e-05 2024-05-08T08:55:16.690 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.690 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.690 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:16.690 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557580+0000", 2024-05-08T08:55:16.690 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5517000000000003e-05 2024-05-08T08:55:16.690 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:16.690 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:16.690 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:16.690 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.690 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.690 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1089 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11354112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:55:16.690 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:55:16.690 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 160.16506309799999, 2024-05-08T08:55:16.690 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071457381, 2024-05-08T08:55:16.691 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:16.691 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:16.691 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:16.691 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:16.691 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:55:16.691 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:55:16.691 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1089 2024-05-08T08:55:16.691 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.691 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:16.691 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.691 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:16.691 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:55:16.691 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:16.691 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.691 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.691 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:16.691 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:55:16.692 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:16.692 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.692 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.692 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:16.692 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486851+0000", 2024-05-08T08:55:16.692 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00049102 2024-05-08T08:55:16.692 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.692 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.692 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:16.692 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486854+0000", 2024-05-08T08:55:16.692 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6859999999999999e-06 2024-05-08T08:55:16.692 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.692 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.692 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:16.692 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486864+0000", 2024-05-08T08:55:16.692 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6080000000000002e-06 2024-05-08T08:55:16.692 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.693 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.693 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:16.693 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486870+0000", 2024-05-08T08:55:16.693 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.477e-06 2024-05-08T08:55:16.693 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.693 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.693 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:16.693 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557612+0000", 2024-05-08T08:55:16.693 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070741557999999996 2024-05-08T08:55:16.693 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.693 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.693 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:16.693 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557695+0000", 2024-05-08T08:55:16.693 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3300000000000005e-05 2024-05-08T08:55:16.693 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.693 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.694 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:16.694 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557817+0000", 2024-05-08T08:55:16.694 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012173200000000001 2024-05-08T08:55:16.694 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:16.694 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:16.694 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:16.694 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.694 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.694 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1091 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11395072~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:55:16.694 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:55:16.694 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 160.16454230299999, 2024-05-08T08:55:16.694 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071184737999999997, 2024-05-08T08:55:16.694 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:16.694 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:16.695 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:16.695 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:16.695 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:55:16.695 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:55:16.695 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1091 2024-05-08T08:55:16.695 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.695 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:16.695 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.695 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:16.695 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:55:16.695 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:16.695 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.695 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.695 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:16.695 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:55:16.695 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:16.696 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.696 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.696 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:16.696 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486881+0000", 2024-05-08T08:55:16.696 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0699999999999997e-07 2024-05-08T08:55:16.696 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.696 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.696 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:16.696 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486883+0000", 2024-05-08T08:55:16.696 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.402e-06 2024-05-08T08:55:16.696 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.696 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.696 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:16.696 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486890+0000", 2024-05-08T08:55:16.696 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0480000000000001e-06 2024-05-08T08:55:16.696 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.696 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.697 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:16.697 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486896+0000", 2024-05-08T08:55:16.697 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7640000000000002e-06 2024-05-08T08:55:16.697 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.697 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.697 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:16.697 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557860+0000", 2024-05-08T08:55:16.697 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070963791999999998 2024-05-08T08:55:16.697 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.697 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.697 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:16.697 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557952+0000", 2024-05-08T08:55:16.697 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2082000000000004e-05 2024-05-08T08:55:16.697 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.697 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.697 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:16.698 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558065+0000", 2024-05-08T08:55:16.698 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000113143 2024-05-08T08:55:16.698 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:16.698 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:16.698 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:16.698 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.698 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.698 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1094 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11436032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:55:16.698 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:55:16.698 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 160.16451913500001, 2024-05-08T08:55:16.698 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071400507000000002, 2024-05-08T08:55:16.698 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:16.698 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:16.699 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:16.699 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:16.699 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:55:16.699 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:55:16.699 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1094 2024-05-08T08:55:16.699 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.699 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:16.699 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.699 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:16.699 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:55:16.699 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:16.699 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.699 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.699 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:16.699 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:55:16.699 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:16.699 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.700 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.700 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:16.700 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487940+0000", 2024-05-08T08:55:16.700 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001036868 2024-05-08T08:55:16.700 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.700 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.700 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:16.700 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487944+0000", 2024-05-08T08:55:16.700 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7069999999999998e-06 2024-05-08T08:55:16.700 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.700 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.700 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:16.700 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487951+0000", 2024-05-08T08:55:16.700 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2649999999999999e-06 2024-05-08T08:55:16.700 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.700 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.700 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:16.701 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487961+0000", 2024-05-08T08:55:16.701 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4110000000000002e-06 2024-05-08T08:55:16.701 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.701 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.701 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:16.701 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558117+0000", 2024-05-08T08:55:16.701 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070156662999999994 2024-05-08T08:55:16.701 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.701 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.701 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:16.701 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558214+0000", 2024-05-08T08:55:16.701 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6444e-05 2024-05-08T08:55:16.701 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.701 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.701 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:16.701 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558304+0000", 2024-05-08T08:55:16.701 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0148999999999994e-05 2024-05-08T08:55:16.702 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:16.702 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:16.702 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:16.702 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.702 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.702 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1096 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11476992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:55:16.702 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:55:16.702 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 160.16345260400001, 2024-05-08T08:55:16.702 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070480074000000004, 2024-05-08T08:55:16.702 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:16.702 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:16.702 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:16.702 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:16.702 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:55:16.703 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:55:16.703 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1096 2024-05-08T08:55:16.703 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.703 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:16.703 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.703 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:16.703 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:55:16.703 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:16.703 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.703 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.703 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:16.703 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:55:16.703 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:16.703 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.703 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.703 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:16.704 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:55:16.704 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3700000000000001e-07 2024-05-08T08:55:16.704 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.704 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.704 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:16.704 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487973+0000", 2024-05-08T08:55:16.704 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0870000000000001e-06 2024-05-08T08:55:16.704 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.704 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.704 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:16.704 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487977+0000", 2024-05-08T08:55:16.704 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0600000000000001e-06 2024-05-08T08:55:16.704 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.704 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.704 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:16.704 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487980+0000", 2024-05-08T08:55:16.704 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1089999999999998e-06 2024-05-08T08:55:16.705 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.705 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.705 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:16.705 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558338+0000", 2024-05-08T08:55:16.705 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070358233000000006 2024-05-08T08:55:16.705 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.705 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.705 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:16.705 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558392+0000", 2024-05-08T08:55:16.705 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4537999999999999e-05 2024-05-08T08:55:16.705 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.705 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.705 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:16.705 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558450+0000", 2024-05-08T08:55:16.705 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7710000000000001e-05 2024-05-08T08:55:16.705 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:16.706 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:16.706 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:16.706 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.706 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.706 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1099 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11517952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:55:16.706 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:55:16.706 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 160.16343872799999, 2024-05-08T08:55:16.706 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070720506000000002, 2024-05-08T08:55:16.706 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:16.706 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:16.706 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:16.706 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:16.706 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:55:16.707 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:55:16.707 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1099 2024-05-08T08:55:16.707 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.707 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:16.707 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.707 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:16.707 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:55:16.707 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:16.707 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.707 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.707 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:16.707 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:55:16.707 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:16.707 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.707 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.708 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:16.708 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:55:16.708 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3999999999999998e-07 2024-05-08T08:55:16.708 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.708 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.708 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:16.708 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487986+0000", 2024-05-08T08:55:16.708 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3519999999999999e-06 2024-05-08T08:55:16.708 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.708 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.708 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:16.708 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487989+0000", 2024-05-08T08:55:16.708 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.23e-06 2024-05-08T08:55:16.708 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.708 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.709 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:16.709 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487991+0000", 2024-05-08T08:55:16.709 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6350000000000002e-06 2024-05-08T08:55:16.709 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.709 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.709 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:16.709 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558489+0000", 2024-05-08T08:55:16.709 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070497924000000003 2024-05-08T08:55:16.709 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.709 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.709 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:16.709 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558583+0000", 2024-05-08T08:55:16.709 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3405000000000004e-05 2024-05-08T08:55:16.709 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.709 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.709 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:16.709 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558704+0000", 2024-05-08T08:55:16.710 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012171999999999999 2024-05-08T08:55:16.710 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:16.710 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:16.710 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:16.710 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.710 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.710 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1100 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11558912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:55:16.710 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:55:16.710 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 160.16342572900001, 2024-05-08T08:55:16.710 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070962266999999996, 2024-05-08T08:55:16.710 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:16.710 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:16.710 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:16.710 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:16.710 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:55:16.710 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:55:16.711 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1100 2024-05-08T08:55:16.711 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.711 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:16.711 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.711 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:16.711 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:55:16.711 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:16.711 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.711 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.711 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:16.711 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:55:16.711 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:16.711 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.711 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.711 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:16.712 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488430+0000", 2024-05-08T08:55:16.712 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000432676 2024-05-08T08:55:16.712 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.712 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.712 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:16.712 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488433+0000", 2024-05-08T08:55:16.712 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.45e-06 2024-05-08T08:55:16.712 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.712 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.712 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:16.712 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488442+0000", 2024-05-08T08:55:16.712 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.687e-06 2024-05-08T08:55:16.712 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.712 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.712 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:16.712 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488446+0000", 2024-05-08T08:55:16.712 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4939999999999997e-06 2024-05-08T08:55:16.713 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.713 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.713 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:16.713 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558756+0000", 2024-05-08T08:55:16.713 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070310051999999998 2024-05-08T08:55:16.713 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.713 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.713 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:16.713 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558844+0000", 2024-05-08T08:55:16.713 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7690999999999998e-05 2024-05-08T08:55:16.713 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.713 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.713 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:16.713 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558959+0000", 2024-05-08T08:55:16.713 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000115217 2024-05-08T08:55:16.713 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:16.714 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:16.714 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:16.714 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.714 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.714 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1103 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11599872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:55:16.714 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:55:16.714 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 160.162968054, 2024-05-08T08:55:16.714 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070777100999999995, 2024-05-08T08:55:16.714 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:16.714 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:16.714 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:16.714 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:16.715 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:55:16.715 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:55:16.715 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1103 2024-05-08T08:55:16.715 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.715 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:16.715 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.715 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:16.715 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:55:16.715 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:16.715 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.715 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.715 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:16.715 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:55:16.715 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:16.716 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.716 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.716 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:16.716 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488959+0000", 2024-05-08T08:55:16.716 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00050457200000000001 2024-05-08T08:55:16.716 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.716 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.716 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:16.716 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488969+0000", 2024-05-08T08:55:16.716 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.7159999999999998e-06 2024-05-08T08:55:16.716 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.716 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.716 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:16.716 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488981+0000", 2024-05-08T08:55:16.716 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2150000000000001e-05 2024-05-08T08:55:16.717 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.717 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.717 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:16.717 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488988+0000", 2024-05-08T08:55:16.717 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8759999999999999e-06 2024-05-08T08:55:16.717 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.717 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.717 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:16.717 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558992+0000", 2024-05-08T08:55:16.717 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070004358000000003 2024-05-08T08:55:16.717 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.717 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.717 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:16.717 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559093+0000", 2024-05-08T08:55:16.717 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000100299 2024-05-08T08:55:16.717 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.718 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.718 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:16.718 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559232+0000", 2024-05-08T08:55:16.718 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013913 2024-05-08T08:55:16.718 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:16.718 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:16.718 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:16.718 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.718 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.718 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1104 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11640832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:55:16.718 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:55:16.718 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 160.16242323899999, 2024-05-08T08:55:16.718 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070521947000000001, 2024-05-08T08:55:16.718 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:16.719 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:16.719 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:16.719 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:16.719 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:55:16.719 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:55:16.719 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1104 2024-05-08T08:55:16.719 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.719 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:16.719 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.719 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:16.719 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:55:16.719 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:16.719 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.719 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.719 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:16.719 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:55:16.720 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:16.720 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.720 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.722 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:16.722 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489700+0000", 2024-05-08T08:55:16.722 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00070067600000000001 2024-05-08T08:55:16.722 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.722 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.722 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:16.722 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489704+0000", 2024-05-08T08:55:16.722 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1640000000000003e-06 2024-05-08T08:55:16.722 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.722 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.722 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:16.722 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489716+0000", 2024-05-08T08:55:16.723 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1379000000000001e-05 2024-05-08T08:55:16.723 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.723 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.723 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:16.723 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489724+0000", 2024-05-08T08:55:16.723 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8949999999999998e-06 2024-05-08T08:55:16.723 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.723 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.723 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:16.723 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559278+0000", 2024-05-08T08:55:16.723 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.069554151999999994 2024-05-08T08:55:16.723 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.723 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.723 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:16.723 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559387+0000", 2024-05-08T08:55:16.723 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000109258 2024-05-08T08:55:16.724 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.724 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.724 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:16.724 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559521+0000", 2024-05-08T08:55:16.724 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000134423 2024-05-08T08:55:16.724 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:16.724 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:16.724 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:16.724 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.724 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.724 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:8636 3.17 3:e8a0a3e4:::rbd_header.11402b26385:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-05-08T08:55:16.724 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:54.150971+0000", 2024-05-08T08:55:16.724 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 142.50045124499999, 2024-05-08T08:55:16.725 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.217273233, 2024-05-08T08:55:16.725 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:16.725 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:16.725 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:16.725 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:16.725 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:55:16.725 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:55:16.725 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 8636 2024-05-08T08:55:16.725 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.725 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:16.725 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.725 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:16.726 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150971+0000", 2024-05-08T08:55:16.726 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:16.726 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.726 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.726 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:16.726 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150971+0000", 2024-05-08T08:55:16.726 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:16.726 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.726 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.726 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:16.726 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150972+0000", 2024-05-08T08:55:16.726 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.71e-07 2024-05-08T08:55:16.726 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.727 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.727 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:16.727 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150977+0000", 2024-05-08T08:55:16.727 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.152e-06 2024-05-08T08:55:16.727 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.727 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.727 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:16.727 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150990+0000", 2024-05-08T08:55:16.727 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2364999999999999e-05 2024-05-08T08:55:16.727 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.727 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.727 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:16.727 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150998+0000", 2024-05-08T08:55:16.727 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0739999999999993e-06 2024-05-08T08:55:16.728 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.728 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.728 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:16.728 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.151054+0000", 2024-05-08T08:55:16.728 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5949000000000003e-05 2024-05-08T08:55:16.728 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.728 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.728 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:55:16.728 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.151060+0000", 2024-05-08T08:55:16.728 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6120000000000004e-06 2024-05-08T08:55:16.728 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.728 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.728 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:16.729 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.367640+0000", 2024-05-08T08:55:16.729 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2165792739999999 2024-05-08T08:55:16.729 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.729 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.729 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:16.729 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.367839+0000", 2024-05-08T08:55:16.729 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000199185 2024-05-08T08:55:16.729 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.729 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.729 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:16.729 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.368245+0000", 2024-05-08T08:55:16.729 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00040575099999999998 2024-05-08T08:55:16.729 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:16.729 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:16.730 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:16.730 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.730 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.730 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5302.0:1516 3.c 3:319bf789:::rbd_data.1140d3fc54c.0000000000000025:head [stat,write 1892352~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:55:16.730 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:27.822220+0000", 2024-05-08T08:55:16.730 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 108.82920230800001, 2024-05-08T08:55:16.730 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.092643744, 2024-05-08T08:55:16.730 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:16.730 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:16.730 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:55:16.730 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:16.730 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5302", 2024-05-08T08:55:16.730 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/359277859", 2024-05-08T08:55:16.731 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1516 2024-05-08T08:55:16.731 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.731 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:16.731 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.731 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:16.731 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822220+0000", 2024-05-08T08:55:16.731 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:16.731 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.731 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.731 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:16.731 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822220+0000", 2024-05-08T08:55:16.731 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:16.732 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.732 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.732 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:16.732 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822221+0000", 2024-05-08T08:55:16.732 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3500000000000002e-07 2024-05-08T08:55:16.732 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.732 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.732 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:16.732 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822228+0000", 2024-05-08T08:55:16.732 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9959999999999996e-06 2024-05-08T08:55:16.732 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.732 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.732 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:16.732 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822234+0000", 2024-05-08T08:55:16.733 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5429999999999999e-06 2024-05-08T08:55:16.733 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.733 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.733 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:16.733 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822238+0000", 2024-05-08T08:55:16.733 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.732e-06 2024-05-08T08:55:16.733 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.733 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.733 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:16.733 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822285+0000", 2024-05-08T08:55:16.733 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6810000000000001e-05 2024-05-08T08:55:16.733 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.733 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.733 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:16.734 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822358+0000", 2024-05-08T08:55:16.734 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3473000000000006e-05 2024-05-08T08:55:16.734 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.734 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.734 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 5", 2024-05-08T08:55:16.734 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822489+0000", 2024-05-08T08:55:16.734 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013011199999999999 2024-05-08T08:55:16.734 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.734 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.734 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:55:16.734 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.825332+0000", 2024-05-08T08:55:16.734 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0028434799999999998 2024-05-08T08:55:16.734 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.734 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.734 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:55:16.735 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.914753+0000", 2024-05-08T08:55:16.735 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089421424999999999 2024-05-08T08:55:16.735 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.735 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.735 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:55:16.735 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.914807+0000", 2024-05-08T08:55:16.735 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3829000000000002e-05 2024-05-08T08:55:16.735 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.735 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.735 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:16.735 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.914864+0000", 2024-05-08T08:55:16.735 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6909e-05 2024-05-08T08:55:16.735 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:16.735 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:16.735 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:16.736 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.736 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.736 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:24567 3.6 3:66af1c40:::rbd_data.1140d3fc54c.000000000000005a:18 [read 2080768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:16.736 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.216598+0000", 2024-05-08T08:55:16.736 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 104.434824228, 2024-05-08T08:55:16.736 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071602647000000005, 2024-05-08T08:55:16.736 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:16.736 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:16.736 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:16.736 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:16.736 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:16.736 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:16.736 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 24567 2024-05-08T08:55:16.736 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.736 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:16.736 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.737 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:16.737 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216598+0000", 2024-05-08T08:55:16.737 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:16.737 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.737 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.737 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:16.737 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216598+0000", 2024-05-08T08:55:16.737 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:16.737 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.737 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.737 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:16.737 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216599+0000", 2024-05-08T08:55:16.737 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8999999999999997e-07 2024-05-08T08:55:16.737 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.738 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.738 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:16.738 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216601+0000", 2024-05-08T08:55:16.738 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9889999999999999e-06 2024-05-08T08:55:16.738 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.738 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.738 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:16.738 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216607+0000", 2024-05-08T08:55:16.738 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3600000000000001e-06 2024-05-08T08:55:16.738 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.738 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.738 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:16.738 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216612+0000", 2024-05-08T08:55:16.738 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.69e-06 2024-05-08T08:55:16.738 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.738 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.739 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:16.739 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.287076+0000", 2024-05-08T08:55:16.739 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070464355000000006 2024-05-08T08:55:16.739 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.739 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.739 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:16.739 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.287151+0000", 2024-05-08T08:55:16.739 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4924000000000003e-05 2024-05-08T08:55:16.739 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:16.739 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:16.739 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:16.739 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.288201+0000", 2024-05-08T08:55:16.739 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0010498389999999999 2024-05-08T08:55:16.739 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:16.739 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:16.739 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:16.739 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:16.740 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:16.740 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:55:16.741 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:55:16.742 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:55:16.744+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:55:16.841 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:55:16.844+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:55:16.892 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:55:16.892 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:55:16.892 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:55:16.892 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:55:16.901 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:55:16.944 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:55:16.947+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:55:17.011 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:55:17.011 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:55:17.011 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:55:17.011 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:55:17.011 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:55:17.019 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:55:17.043 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:55:17.046+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:55:17.144 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:55:17.147+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:55:17.171 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:55:17.171 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:55:17.171 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:55:17.171 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:55:17.171 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.171 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:12443 3.0 3:02d0158e:::rbd_data.114071bcf828.0000000000000037:head [stat,delete] snapc 19=[19] RETRY=3 ondisk+retry+write+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:17.171 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.610027+0000", 2024-05-08T08:55:17.171 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 105.562640864, 2024-05-08T08:55:17.171 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.083586333999999998, 2024-05-08T08:55:17.171 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:17.171 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:17.171 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:17.172 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:17.172 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:17.172 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:17.172 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 12443 2024-05-08T08:55:17.172 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.172 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:17.172 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.172 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:17.172 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610027+0000", 2024-05-08T08:55:17.172 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:17.172 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.172 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.172 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:17.172 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610027+0000", 2024-05-08T08:55:17.173 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:17.173 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.173 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.173 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:17.173 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610028+0000", 2024-05-08T08:55:17.173 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.13e-07 2024-05-08T08:55:17.173 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.173 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.173 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:17.173 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610030+0000", 2024-05-08T08:55:17.173 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7669999999999999e-06 2024-05-08T08:55:17.173 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.173 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.173 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:17.173 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610039+0000", 2024-05-08T08:55:17.173 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4959999999999996e-06 2024-05-08T08:55:17.174 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.174 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.174 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:17.174 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610045+0000", 2024-05-08T08:55:17.174 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6779999999999997e-06 2024-05-08T08:55:17.174 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.174 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.174 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:17.174 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.660873+0000", 2024-05-08T08:55:17.174 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.050828748 2024-05-08T08:55:17.174 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.174 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.174 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:17.174 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.661017+0000", 2024-05-08T08:55:17.175 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014391999999999999 2024-05-08T08:55:17.175 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.175 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.175 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:17.175 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.693613+0000", 2024-05-08T08:55:17.175 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.032596111999999997 2024-05-08T08:55:17.175 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:17.175 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:17.175 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:17.175 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.175 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.176 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:14301 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [watch reconnect cookie 139837155901536 gen 12] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:17.176 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.645078+0000", 2024-05-08T08:55:17.176 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 105.527590226, 2024-05-08T08:55:17.176 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088281852999999993, 2024-05-08T08:55:17.176 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:17.176 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:17.176 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:17.176 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:17.176 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:17.176 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:17.176 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14301 2024-05-08T08:55:17.176 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.176 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:17.176 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.176 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:17.177 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645078+0000", 2024-05-08T08:55:17.177 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:17.177 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.177 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.177 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:17.177 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645078+0000", 2024-05-08T08:55:17.177 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:17.177 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.177 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.177 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:17.177 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645084+0000", 2024-05-08T08:55:17.177 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4620000000000001e-06 2024-05-08T08:55:17.177 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.177 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.178 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:17.178 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645088+0000", 2024-05-08T08:55:17.178 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8039999999999999e-06 2024-05-08T08:55:17.178 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.178 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.178 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:17.178 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645106+0000", 2024-05-08T08:55:17.178 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.8318999999999999e-05 2024-05-08T08:55:17.178 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.178 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.178 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:17.178 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645113+0000", 2024-05-08T08:55:17.178 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0310000000000001e-06 2024-05-08T08:55:17.178 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.178 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.179 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:17.179 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.670493+0000", 2024-05-08T08:55:17.179 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.025379993 2024-05-08T08:55:17.179 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.179 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.179 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:17.179 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.670571+0000", 2024-05-08T08:55:17.179 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7613000000000004e-05 2024-05-08T08:55:17.179 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.179 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.179 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:17.179 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733360+0000", 2024-05-08T08:55:17.179 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.062788630999999998 2024-05-08T08:55:17.179 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:17.179 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:17.180 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:17.180 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.180 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.180 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:14420 3.0 3:06a8948e:::rbd_data.114071bcf828.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:17.180 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.649639+0000", 2024-05-08T08:55:17.180 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 105.52302907799999, 2024-05-08T08:55:17.180 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.083851897999999994, 2024-05-08T08:55:17.180 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:17.180 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:17.180 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:55:17.180 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:17.180 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:17.180 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:17.180 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14420 2024-05-08T08:55:17.180 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.180 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:17.181 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.181 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:17.181 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649639+0000", 2024-05-08T08:55:17.181 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:17.181 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.181 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.181 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:17.181 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649639+0000", 2024-05-08T08:55:17.181 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:17.181 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.181 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.181 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:17.181 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649640+0000", 2024-05-08T08:55:17.181 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6899999999999997e-07 2024-05-08T08:55:17.181 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.182 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.182 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:17.182 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649895+0000", 2024-05-08T08:55:17.182 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00025531000000000002 2024-05-08T08:55:17.182 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.182 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.182 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:17.182 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649905+0000", 2024-05-08T08:55:17.182 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0198e-05 2024-05-08T08:55:17.182 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.182 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.182 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:17.182 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649911+0000", 2024-05-08T08:55:17.182 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2310000000000001e-06 2024-05-08T08:55:17.183 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.183 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.183 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:17.183 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.684099+0000", 2024-05-08T08:55:17.183 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.034188017000000001 2024-05-08T08:55:17.183 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.183 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.183 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:17.183 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.684172+0000", 2024-05-08T08:55:17.183 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3201999999999995e-05 2024-05-08T08:55:17.183 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.183 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.183 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 2", 2024-05-08T08:55:17.183 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.684418+0000", 2024-05-08T08:55:17.183 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00024559099999999997 2024-05-08T08:55:17.183 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.184 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.184 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:55:17.184 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.705538+0000", 2024-05-08T08:55:17.184 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.021119707000000001 2024-05-08T08:55:17.184 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.184 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.184 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:55:17.184 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733399+0000", 2024-05-08T08:55:17.184 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.027861543999999999 2024-05-08T08:55:17.184 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.184 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.184 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:55:17.184 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733445+0000", 2024-05-08T08:55:17.184 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5587000000000002e-05 2024-05-08T08:55:17.184 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.185 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.185 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:17.185 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733491+0000", 2024-05-08T08:55:17.185 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5841999999999999e-05 2024-05-08T08:55:17.185 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:17.185 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:17.185 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:17.185 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.185 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.185 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28548 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889189696] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:17.185 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.355669+0000", 2024-05-08T08:55:17.185 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 104.8169985, 2024-05-08T08:55:17.185 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078956606999999998, 2024-05-08T08:55:17.185 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:17.186 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:17.186 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:17.186 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:17.186 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:17.186 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:17.186 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28548 2024-05-08T08:55:17.186 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.186 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:17.186 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.186 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:17.186 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355669+0000", 2024-05-08T08:55:17.186 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:17.186 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.186 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.186 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:17.187 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355669+0000", 2024-05-08T08:55:17.187 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:17.187 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.187 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.187 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:17.187 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355762+0000", 2024-05-08T08:55:17.187 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3015000000000003e-05 2024-05-08T08:55:17.187 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.187 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.187 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:17.187 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355765+0000", 2024-05-08T08:55:17.187 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.368e-06 2024-05-08T08:55:17.187 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.187 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.188 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:17.188 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355771+0000", 2024-05-08T08:55:17.188 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5989999999999998e-06 2024-05-08T08:55:17.188 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.188 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.188 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:17.188 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355775+0000", 2024-05-08T08:55:17.188 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7280000000000001e-06 2024-05-08T08:55:17.188 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.188 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.188 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:17.188 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434379+0000", 2024-05-08T08:55:17.188 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078603662000000005 2024-05-08T08:55:17.188 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.188 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.189 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:17.189 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434473+0000", 2024-05-08T08:55:17.189 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4104000000000006e-05 2024-05-08T08:55:17.189 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.189 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.189 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:17.189 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434626+0000", 2024-05-08T08:55:17.189 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00015313100000000001 2024-05-08T08:55:17.189 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:17.189 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:17.189 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:17.189 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.189 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.189 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28549 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889202400] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:17.189 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.355781+0000", 2024-05-08T08:55:17.190 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 104.816886926, 2024-05-08T08:55:17.190 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079117946999999994, 2024-05-08T08:55:17.190 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:17.190 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:17.190 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:17.190 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:17.190 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:17.190 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:17.190 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28549 2024-05-08T08:55:17.190 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.190 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:17.190 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.190 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:17.191 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355781+0000", 2024-05-08T08:55:17.191 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:17.191 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.191 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.191 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:17.191 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355781+0000", 2024-05-08T08:55:17.191 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:17.191 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.191 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.191 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:17.191 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356162+0000", 2024-05-08T08:55:17.191 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00038099100000000001 2024-05-08T08:55:17.191 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.191 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.191 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:17.191 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356165+0000", 2024-05-08T08:55:17.192 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1240000000000001e-06 2024-05-08T08:55:17.192 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.192 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.192 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:17.192 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356173+0000", 2024-05-08T08:55:17.192 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.6199999999999999e-06 2024-05-08T08:55:17.192 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.192 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.192 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:17.192 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356182+0000", 2024-05-08T08:55:17.192 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7760000000000003e-06 2024-05-08T08:55:17.192 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.192 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.192 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:17.193 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434675+0000", 2024-05-08T08:55:17.193 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078493119 2024-05-08T08:55:17.193 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.193 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.193 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:17.193 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434765+0000", 2024-05-08T08:55:17.193 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0402000000000006e-05 2024-05-08T08:55:17.193 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.193 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.193 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:17.193 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434899+0000", 2024-05-08T08:55:17.193 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013391499999999999 2024-05-08T08:55:17.193 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:17.193 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:17.193 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:17.194 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.194 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.194 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28550 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888597280] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:17.194 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:55:17.194 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 104.81647588, 2024-05-08T08:55:17.194 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078933990999999995, 2024-05-08T08:55:17.194 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:17.194 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:17.194 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:17.194 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:17.194 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:17.194 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:17.194 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28550 2024-05-08T08:55:17.194 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.195 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:17.195 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.195 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:17.195 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:55:17.195 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:17.195 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.195 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.195 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:17.195 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:55:17.195 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:17.195 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.195 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.195 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:17.195 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:55:17.196 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.34e-07 2024-05-08T08:55:17.196 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.196 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.196 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:17.196 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356195+0000", 2024-05-08T08:55:17.196 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8710000000000001e-06 2024-05-08T08:55:17.196 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.196 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.196 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:17.196 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356200+0000", 2024-05-08T08:55:17.196 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3000000000000003e-06 2024-05-08T08:55:17.196 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.196 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.196 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:17.196 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356203+0000", 2024-05-08T08:55:17.197 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4039999999999999e-06 2024-05-08T08:55:17.197 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.197 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.197 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:17.197 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434928+0000", 2024-05-08T08:55:17.197 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078725331999999995 2024-05-08T08:55:17.197 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.197 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.197 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:17.197 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435007+0000", 2024-05-08T08:55:17.197 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8415000000000001e-05 2024-05-08T08:55:17.197 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.197 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.197 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:17.197 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435126+0000", 2024-05-08T08:55:17.198 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011923500000000001 2024-05-08T08:55:17.198 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:17.198 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:17.198 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:17.198 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.198 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.198 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28551 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888527200] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:17.198 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.356210+0000", 2024-05-08T08:55:17.198 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 104.816457532, 2024-05-08T08:55:17.198 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079195072000000005, 2024-05-08T08:55:17.198 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:17.198 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:17.198 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:17.198 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:17.199 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:17.199 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:17.199 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28551 2024-05-08T08:55:17.199 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.199 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:17.199 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.199 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:17.199 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356210+0000", 2024-05-08T08:55:17.199 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:17.199 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.199 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.199 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:17.199 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356210+0000", 2024-05-08T08:55:17.200 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:17.200 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.200 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.200 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:17.200 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356211+0000", 2024-05-08T08:55:17.200 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4499999999999998e-07 2024-05-08T08:55:17.200 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.200 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.200 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:17.200 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356212+0000", 2024-05-08T08:55:17.200 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6029999999999999e-06 2024-05-08T08:55:17.200 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.200 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.201 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:17.201 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356221+0000", 2024-05-08T08:55:17.201 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4200000000000007e-06 2024-05-08T08:55:17.201 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.201 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.201 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:17.201 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356224+0000", 2024-05-08T08:55:17.201 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.038e-06 2024-05-08T08:55:17.201 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.201 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.201 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:17.201 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435178+0000", 2024-05-08T08:55:17.201 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078954625000000001 2024-05-08T08:55:17.201 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.202 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.202 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:17.202 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435256+0000", 2024-05-08T08:55:17.202 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7337999999999997e-05 2024-05-08T08:55:17.202 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.202 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.202 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:17.202 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435405+0000", 2024-05-08T08:55:17.202 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000149703 2024-05-08T08:55:17.202 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:17.202 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:17.202 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:17.202 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.202 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.202 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28552 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889231456] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:17.203 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.356228+0000", 2024-05-08T08:55:17.203 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 104.816439854, 2024-05-08T08:55:17.203 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079346920000000001, 2024-05-08T08:55:17.203 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:17.203 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:17.203 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:17.203 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:17.203 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:17.203 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:17.203 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28552 2024-05-08T08:55:17.203 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.203 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:17.203 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.203 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:17.204 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356228+0000", 2024-05-08T08:55:17.204 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:17.204 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.204 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.204 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:17.204 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356228+0000", 2024-05-08T08:55:17.204 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:17.204 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.204 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.204 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:17.204 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356959+0000", 2024-05-08T08:55:17.204 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00073097099999999996 2024-05-08T08:55:17.204 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.205 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.205 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:17.205 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356967+0000", 2024-05-08T08:55:17.205 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0409999999999998e-06 2024-05-08T08:55:17.205 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.205 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.205 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:17.205 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356979+0000", 2024-05-08T08:55:17.205 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1489e-05 2024-05-08T08:55:17.205 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.205 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.205 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:17.205 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356986+0000", 2024-05-08T08:55:17.205 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3699999999999997e-06 2024-05-08T08:55:17.206 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.206 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.206 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:17.206 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435433+0000", 2024-05-08T08:55:17.206 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078447520000000007 2024-05-08T08:55:17.206 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.206 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.206 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:17.206 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435484+0000", 2024-05-08T08:55:17.206 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0229000000000003e-05 2024-05-08T08:55:17.206 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.206 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.207 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:17.207 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435575+0000", 2024-05-08T08:55:17.207 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1299999999999997e-05 2024-05-08T08:55:17.207 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:17.207 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:17.207 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:17.207 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.207 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.207 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28553 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888713648] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:17.207 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:55:17.208 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 104.815665196, 2024-05-08T08:55:17.208 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078782034000000001, 2024-05-08T08:55:17.208 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:17.208 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:17.208 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:17.208 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:17.208 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:17.208 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:17.208 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28553 2024-05-08T08:55:17.208 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.209 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:17.209 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.209 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:17.209 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:55:17.209 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:17.209 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.209 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.209 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:17.209 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:55:17.209 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:17.209 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.210 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.210 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:17.210 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:55:17.210 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6600000000000002e-07 2024-05-08T08:55:17.210 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.210 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.210 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:17.210 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357006+0000", 2024-05-08T08:55:17.210 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7190000000000001e-06 2024-05-08T08:55:17.211 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.211 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.211 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:17.211 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357013+0000", 2024-05-08T08:55:17.211 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.435e-06 2024-05-08T08:55:17.211 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.211 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.211 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:17.211 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357018+0000", 2024-05-08T08:55:17.211 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7260000000000002e-06 2024-05-08T08:55:17.211 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.211 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.211 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:17.212 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435623+0000", 2024-05-08T08:55:17.212 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078604972999999995 2024-05-08T08:55:17.212 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.212 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.212 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:17.212 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435695+0000", 2024-05-08T08:55:17.212 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1959999999999995e-05 2024-05-08T08:55:17.212 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.212 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.212 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:17.212 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435785+0000", 2024-05-08T08:55:17.212 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.9754999999999997e-05 2024-05-08T08:55:17.212 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:17.212 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:17.213 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:17.213 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.213 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.213 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28554 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888980272] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:17.213 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357024+0000", 2024-05-08T08:55:17.213 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 104.81564356, 2024-05-08T08:55:17.213 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078999870999999999, 2024-05-08T08:55:17.213 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:17.213 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:17.213 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:17.213 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:17.213 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:17.213 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:17.213 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28554 2024-05-08T08:55:17.213 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.214 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:17.214 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.214 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:17.214 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357024+0000", 2024-05-08T08:55:17.214 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:17.214 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.214 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.214 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:17.214 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357024+0000", 2024-05-08T08:55:17.214 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:17.214 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.214 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.214 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:17.215 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357025+0000", 2024-05-08T08:55:17.215 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:55:17.215 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.215 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.215 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:17.215 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357027+0000", 2024-05-08T08:55:17.215 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2050000000000001e-06 2024-05-08T08:55:17.215 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.215 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.215 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:17.215 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357033+0000", 2024-05-08T08:55:17.215 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5679999999999996e-06 2024-05-08T08:55:17.215 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.215 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.216 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:17.216 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357038+0000", 2024-05-08T08:55:17.216 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2889999999999999e-06 2024-05-08T08:55:17.216 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.216 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.216 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:17.216 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435823+0000", 2024-05-08T08:55:17.216 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078785677999999998 2024-05-08T08:55:17.216 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.216 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.216 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:17.216 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435892+0000", 2024-05-08T08:55:17.216 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8379000000000004e-05 2024-05-08T08:55:17.217 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.217 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.217 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:17.217 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436024+0000", 2024-05-08T08:55:17.217 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013239799999999999 2024-05-08T08:55:17.217 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:17.217 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:17.217 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:17.217 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.217 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.217 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28555 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889001104] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:17.217 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:55:17.217 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 104.81562479500001, 2024-05-08T08:55:17.218 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079115742000000003, 2024-05-08T08:55:17.218 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:17.218 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:17.218 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:17.218 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:17.218 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:17.218 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:17.218 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28555 2024-05-08T08:55:17.218 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.218 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:17.218 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.218 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:17.219 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:55:17.219 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:17.219 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.219 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.219 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:17.219 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:55:17.219 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:17.219 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.219 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.219 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:17.220 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:55:17.220 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9200000000000002e-07 2024-05-08T08:55:17.220 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.220 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.220 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:17.220 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357046+0000", 2024-05-08T08:55:17.220 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1670000000000002e-06 2024-05-08T08:55:17.220 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.220 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.220 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:17.221 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357052+0000", 2024-05-08T08:55:17.221 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.139e-06 2024-05-08T08:55:17.221 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.221 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.221 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:17.221 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357056+0000", 2024-05-08T08:55:17.221 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2440000000000004e-06 2024-05-08T08:55:17.221 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.221 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.221 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:17.221 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436059+0000", 2024-05-08T08:55:17.222 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079003070999999994 2024-05-08T08:55:17.222 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.222 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.222 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:17.222 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436097+0000", 2024-05-08T08:55:17.222 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7784000000000003e-05 2024-05-08T08:55:17.222 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.222 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.222 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:17.222 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436159+0000", 2024-05-08T08:55:17.222 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2045000000000007e-05 2024-05-08T08:55:17.223 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:17.223 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:17.223 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:17.223 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.223 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.223 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28556 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889022064] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:17.223 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357065+0000", 2024-05-08T08:55:17.223 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 104.815602762, 2024-05-08T08:55:17.223 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079327976999999994, 2024-05-08T08:55:17.223 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:17.224 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:17.224 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:17.224 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:17.224 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:17.224 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:17.224 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28556 2024-05-08T08:55:17.224 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.224 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:17.224 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.224 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:17.224 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357065+0000", 2024-05-08T08:55:17.225 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:17.225 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.225 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.225 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:17.225 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357065+0000", 2024-05-08T08:55:17.225 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:17.225 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.225 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.225 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:17.225 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358958+0000", 2024-05-08T08:55:17.225 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001892915 2024-05-08T08:55:17.225 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.225 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.225 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:17.226 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358962+0000", 2024-05-08T08:55:17.226 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4599999999999999e-06 2024-05-08T08:55:17.226 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.226 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.226 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:17.226 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358970+0000", 2024-05-08T08:55:17.226 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.6090000000000001e-06 2024-05-08T08:55:17.226 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.226 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.226 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:17.226 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358975+0000", 2024-05-08T08:55:17.226 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7639999999999996e-06 2024-05-08T08:55:17.226 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.226 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.227 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:17.227 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436196+0000", 2024-05-08T08:55:17.227 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.077221564000000006 2024-05-08T08:55:17.227 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.227 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.227 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:17.227 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436269+0000", 2024-05-08T08:55:17.227 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2720999999999997e-05 2024-05-08T08:55:17.227 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.227 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.227 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:17.227 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436393+0000", 2024-05-08T08:55:17.227 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012394399999999999 2024-05-08T08:55:17.227 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:17.228 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:17.228 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:17.228 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.228 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.228 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28563 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888855520] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:17.228 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:55:17.228 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 104.813593363, 2024-05-08T08:55:17.228 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078795566999999997, 2024-05-08T08:55:17.228 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:17.228 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:17.228 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:17.228 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:17.229 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:17.229 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:17.229 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28563 2024-05-08T08:55:17.229 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.229 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:17.229 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.229 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:17.229 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:55:17.229 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:17.229 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.230 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.230 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:17.230 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:55:17.230 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:17.230 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.230 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.230 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:17.230 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:55:17.230 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1e-07 2024-05-08T08:55:17.230 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.230 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.230 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:17.231 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359080+0000", 2024-05-08T08:55:17.231 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8940000000000001e-06 2024-05-08T08:55:17.231 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.231 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.231 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:17.231 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359083+0000", 2024-05-08T08:55:17.231 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1949999999999999e-06 2024-05-08T08:55:17.231 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.231 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.231 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:17.231 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359085+0000", 2024-05-08T08:55:17.231 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4710000000000001e-06 2024-05-08T08:55:17.231 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.232 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.232 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:17.232 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437604+0000", 2024-05-08T08:55:17.232 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078518278999999996 2024-05-08T08:55:17.232 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.232 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.232 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:17.232 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437704+0000", 2024-05-08T08:55:17.232 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010010599999999999 2024-05-08T08:55:17.232 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.232 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.232 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:17.232 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437870+0000", 2024-05-08T08:55:17.233 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00016641200000000001 2024-05-08T08:55:17.233 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:17.233 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:17.233 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:17.233 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.233 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.233 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28564 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888538944] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:17.233 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.359089+0000", 2024-05-08T08:55:17.233 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 104.813579148, 2024-05-08T08:55:17.233 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078952675, 2024-05-08T08:55:17.233 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:17.234 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:17.234 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:17.234 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:17.234 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:17.234 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:17.234 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28564 2024-05-08T08:55:17.234 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.234 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:17.234 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.235 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:17.235 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359089+0000", 2024-05-08T08:55:17.235 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:17.235 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.235 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.235 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:17.235 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359089+0000", 2024-05-08T08:55:17.235 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:17.235 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.235 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.235 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:17.235 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360959+0000", 2024-05-08T08:55:17.236 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0018707089999999999 2024-05-08T08:55:17.236 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.236 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.236 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:17.236 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360963+0000", 2024-05-08T08:55:17.236 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0680000000000001e-06 2024-05-08T08:55:17.236 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.236 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.236 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:17.236 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360969+0000", 2024-05-08T08:55:17.236 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1920000000000002e-06 2024-05-08T08:55:17.237 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.237 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.237 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:17.237 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360973+0000", 2024-05-08T08:55:17.237 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.031e-06 2024-05-08T08:55:17.237 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.237 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.237 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:17.237 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437916+0000", 2024-05-08T08:55:17.237 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.076942939000000002 2024-05-08T08:55:17.237 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.238 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.238 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:17.238 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437973+0000", 2024-05-08T08:55:17.238 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7358999999999997e-05 2024-05-08T08:55:17.238 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.238 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.238 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:17.238 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.438041+0000", 2024-05-08T08:55:17.238 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8376999999999999e-05 2024-05-08T08:55:17.238 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:17.238 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:17.238 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:17.238 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.239 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.239 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28574 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889427600] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:17.239 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:55:17.239 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 104.81157464099999, 2024-05-08T08:55:17.239 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078782476000000004, 2024-05-08T08:55:17.239 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:17.239 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:17.239 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:17.239 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:17.239 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:17.239 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:17.239 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28574 2024-05-08T08:55:17.239 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.239 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:17.240 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.240 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:17.240 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:55:17.240 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:17.240 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.240 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.240 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:17.240 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:55:17.240 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:17.240 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.240 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.240 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:17.240 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:55:17.240 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1799999999999999e-07 2024-05-08T08:55:17.241 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.241 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.241 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:17.241 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361095+0000", 2024-05-08T08:55:17.241 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3850000000000001e-06 2024-05-08T08:55:17.241 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.241 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.241 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:17.241 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361098+0000", 2024-05-08T08:55:17.241 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7989999999999998e-06 2024-05-08T08:55:17.241 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.241 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.241 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:17.242 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361100+0000", 2024-05-08T08:55:17.242 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4669999999999998e-06 2024-05-08T08:55:17.242 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.242 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.242 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:17.242 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439768+0000", 2024-05-08T08:55:17.242 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078668057999999999 2024-05-08T08:55:17.242 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.242 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.242 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:17.242 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439811+0000", 2024-05-08T08:55:17.243 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2459000000000001e-05 2024-05-08T08:55:17.243 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.243 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.243 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:17.243 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439876+0000", 2024-05-08T08:55:17.243 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5090000000000002e-05 2024-05-08T08:55:17.243 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:17.243 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:17.243 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:17.243 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.243 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.244 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28575 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889439216] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:17.244 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361103+0000", 2024-05-08T08:55:17.244 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 104.81156462600001, 2024-05-08T08:55:17.244 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078966828000000003, 2024-05-08T08:55:17.244 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:17.244 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:17.244 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:17.244 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:17.244 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:17.244 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:17.244 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28575 2024-05-08T08:55:17.245 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.245 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:17.245 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.245 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:17.245 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361103+0000", 2024-05-08T08:55:17.245 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:17.245 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.245 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.245 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:17.246 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361103+0000", 2024-05-08T08:55:17.246 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:17.246 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.246 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.246 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:17.246 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361104+0000", 2024-05-08T08:55:17.246 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1400000000000001e-07 2024-05-08T08:55:17.246 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.246 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.246 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:17.246 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361105+0000", 2024-05-08T08:55:17.247 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5540000000000001e-06 2024-05-08T08:55:17.247 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.247 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.247 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:17.247 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361108+0000", 2024-05-08T08:55:17.247 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.841e-06 2024-05-08T08:55:17.247 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.247 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.247 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:17.248 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361110+0000", 2024-05-08T08:55:17.248 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.497e-06 2024-05-08T08:55:17.248 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.248 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.248 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:17.248 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439916+0000", 2024-05-08T08:55:17.248 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078805372999999998 2024-05-08T08:55:17.248 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.248 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.248 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:17.248 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440000+0000", 2024-05-08T08:55:17.248 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4730999999999999e-05 2024-05-08T08:55:17.248 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.249 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.249 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:17.249 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440070+0000", 2024-05-08T08:55:17.249 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9617999999999997e-05 2024-05-08T08:55:17.249 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:17.249 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:17.249 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:17.249 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.249 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.249 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28576 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889451328] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:17.249 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361113+0000", 2024-05-08T08:55:17.249 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 104.81155443999999, 2024-05-08T08:55:17.249 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079088862999999995, 2024-05-08T08:55:17.250 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:17.250 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:17.250 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:17.250 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:17.250 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:17.250 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:17.250 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28576 2024-05-08T08:55:17.250 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.250 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:17.250 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.250 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:17.250 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361113+0000", 2024-05-08T08:55:17.251 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:17.251 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.251 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.251 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:17.251 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361113+0000", 2024-05-08T08:55:17.251 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:17.251 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.251 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.251 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:17.251 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361114+0000", 2024-05-08T08:55:17.251 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1299999999999999e-07 2024-05-08T08:55:17.251 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.251 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.252 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:17.252 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361115+0000", 2024-05-08T08:55:17.252 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.4300000000000001e-06 2024-05-08T08:55:17.252 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.252 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.252 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:17.252 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361118+0000", 2024-05-08T08:55:17.252 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9000000000000002e-06 2024-05-08T08:55:17.252 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.252 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.252 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:17.252 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361120+0000", 2024-05-08T08:55:17.252 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4739999999999999e-06 2024-05-08T08:55:17.252 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.253 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.253 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:17.253 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440096+0000", 2024-05-08T08:55:17.253 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078975629000000006 2024-05-08T08:55:17.253 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.253 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.253 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:17.253 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440135+0000", 2024-05-08T08:55:17.253 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9202000000000002e-05 2024-05-08T08:55:17.253 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.253 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.253 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:17.253 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440202+0000", 2024-05-08T08:55:17.253 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7014999999999994e-05 2024-05-08T08:55:17.253 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:17.253 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:17.254 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:17.254 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.254 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.254 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28577 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889462640] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:17.254 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:55:17.254 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 104.811544258, 2024-05-08T08:55:17.254 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079194622000000006, 2024-05-08T08:55:17.254 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:17.254 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:17.254 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:17.254 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:17.254 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:17.254 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:17.254 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28577 2024-05-08T08:55:17.255 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.255 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:17.255 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.255 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:17.255 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:55:17.255 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:17.255 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.255 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.255 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:17.255 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:55:17.255 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:17.255 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.255 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.255 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:17.255 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:55:17.256 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1400000000000001e-07 2024-05-08T08:55:17.256 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.256 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.256 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:17.256 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361125+0000", 2024-05-08T08:55:17.256 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.372e-06 2024-05-08T08:55:17.256 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.256 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.256 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:17.256 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361128+0000", 2024-05-08T08:55:17.256 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0309999999999999e-06 2024-05-08T08:55:17.256 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.256 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.256 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:17.256 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361131+0000", 2024-05-08T08:55:17.257 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4040000000000002e-06 2024-05-08T08:55:17.257 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.257 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.257 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:17.257 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440225+0000", 2024-05-08T08:55:17.257 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079094600000000001 2024-05-08T08:55:17.257 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.257 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.257 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:17.257 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440260+0000", 2024-05-08T08:55:17.257 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4703000000000003e-05 2024-05-08T08:55:17.257 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.257 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.257 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:17.257 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440318+0000", 2024-05-08T08:55:17.258 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8298000000000003e-05 2024-05-08T08:55:17.258 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:17.258 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:17.258 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:17.258 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.258 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.258 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28578 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889474192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:17.258 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361134+0000", 2024-05-08T08:55:17.258 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 104.811533833, 2024-05-08T08:55:17.258 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079310295000000003, 2024-05-08T08:55:17.258 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:17.258 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:17.258 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:17.258 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:17.258 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:17.259 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:17.259 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28578 2024-05-08T08:55:17.259 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.259 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:17.259 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.259 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:17.259 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361134+0000", 2024-05-08T08:55:17.259 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:17.259 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.259 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.259 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:17.259 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361134+0000", 2024-05-08T08:55:17.259 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:17.259 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.260 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.260 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:17.260 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.362958+0000", 2024-05-08T08:55:17.260 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0018237380000000001 2024-05-08T08:55:17.260 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.260 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.260 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:17.260 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.362961+0000", 2024-05-08T08:55:17.260 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.123e-06 2024-05-08T08:55:17.260 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.260 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.260 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:17.260 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.397404+0000", 2024-05-08T08:55:17.260 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.034443226 2024-05-08T08:55:17.260 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.261 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.261 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:17.261 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.397439+0000", 2024-05-08T08:55:17.261 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4690000000000002e-05 2024-05-08T08:55:17.261 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.261 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.261 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:17.261 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440340+0000", 2024-05-08T08:55:17.261 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.042901419000000003 2024-05-08T08:55:17.261 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.261 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.261 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:17.261 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440376+0000", 2024-05-08T08:55:17.261 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5444999999999997e-05 2024-05-08T08:55:17.261 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.262 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.262 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:17.262 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440444+0000", 2024-05-08T08:55:17.262 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8653999999999997e-05 2024-05-08T08:55:17.262 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:17.262 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:17.262 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:17.262 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.262 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.262 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:34133 3.10 3:0cc147c9:::rbd_data.1140100c9703.0000000000000035:head [delete] snapc 14=[] ondisk+write+known_if_redirected+supports_pool_eio e164)", 2024-05-08T08:55:17.262 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:55:15.461161+0000", 2024-05-08T08:55:17.262 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 1.7115064960000002, 2024-05-08T08:55:17.262 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.92225079300000001, 2024-05-08T08:55:17.262 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:17.263 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:17.263 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:55:17.263 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:17.263 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:55:17.263 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:55:17.263 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 34133 2024-05-08T08:55:17.263 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.263 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:17.263 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.263 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:17.263 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:15.461161+0000", 2024-05-08T08:55:17.263 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:17.263 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.263 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.263 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:17.264 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:15.461161+0000", 2024-05-08T08:55:17.264 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:17.264 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.264 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.264 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:17.264 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:15.461162+0000", 2024-05-08T08:55:17.264 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0200000000000001e-07 2024-05-08T08:55:17.264 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.264 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.264 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:17.264 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:15.461166+0000", 2024-05-08T08:55:17.264 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2289999999999997e-06 2024-05-08T08:55:17.264 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.264 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.265 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:17.265 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:15.461175+0000", 2024-05-08T08:55:17.265 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0070000000000003e-06 2024-05-08T08:55:17.265 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.265 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.265 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:17.265 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:15.461182+0000", 2024-05-08T08:55:17.265 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5209999999999999e-06 2024-05-08T08:55:17.265 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.265 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.265 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:17.265 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:15.461248+0000", 2024-05-08T08:55:17.265 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5730999999999998e-05 2024-05-08T08:55:17.265 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.265 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.266 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:55:17.266 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:15.461259+0000", 2024-05-08T08:55:17.266 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0812000000000001e-05 2024-05-08T08:55:17.266 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.266 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.266 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:17.266 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:16.376737+0000", 2024-05-08T08:55:17.266 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.91547897199999995 2024-05-08T08:55:17.266 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.266 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.266 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:17.266 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:16.376832+0000", 2024-05-08T08:55:17.266 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4575999999999997e-05 2024-05-08T08:55:17.266 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.267 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.267 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 2", 2024-05-08T08:55:17.267 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:16.376905+0000", 2024-05-08T08:55:17.267 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3208999999999997e-05 2024-05-08T08:55:17.267 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.267 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.267 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:55:17.267 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:16.378405+0000", 2024-05-08T08:55:17.267 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0014998660000000001 2024-05-08T08:55:17.267 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.267 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.267 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:55:17.267 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:16.383294+0000", 2024-05-08T08:55:17.267 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0048889600000000004 2024-05-08T08:55:17.267 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.268 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.268 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:55:17.268 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:16.383366+0000", 2024-05-08T08:55:17.268 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1428999999999997e-05 2024-05-08T08:55:17.268 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:17.268 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:17.268 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:17.268 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:16.383412+0000", 2024-05-08T08:55:17.268 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6678999999999998e-05 2024-05-08T08:55:17.268 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:17.268 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:17.268 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:17.268 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:17.268 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:17.268 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:55:17.270 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:55:17.271 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:55:17.272+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:55:17.370 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:55:17.373+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:55:17.387 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-05-08T08:55:17.408 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T08:55:17.428 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:55:17.428 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:55:17.428 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:55:17.429 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:55:17.439 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:55:17.481 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:55:17.483+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:55:17.569 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:55:17.569 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:55:17.572 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:55:17.575+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:55:17.588 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:55:17.588 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:55:17.600 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:55:17.600 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:55:17.600 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:55:17.600 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:55:17.600 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:55:17.606 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:55:17.607 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:55:17.609 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:55:17.625 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:55:17.625 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:55:17.643 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:55:17.643 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:55:17.661 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:55:17.661 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:55:17.673 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:55:17.676+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:55:17.735 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:17.735 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:17.744 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:55:17.774 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:55:17.776+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:55:17.850 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:17.850 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:17.859 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:55:17.902 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:55:17.905+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:55:17.963 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:17.963 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:17.972 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:55:18.013 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:55:18.015+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:55:18.076 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:55:18.078+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:55:18.117 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:18.117 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:18.125 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:55:18.210 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:55:18.213+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:55:18.270 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:18.271 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:18.277 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:55:18.279+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:55:18.279 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:55:18.378 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:55:18.380+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:55:18.382 INFO:teuthology.orchestra.run.smithi059.stderr:noscrub is unset 2024-05-08T08:55:18.428 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:18.429 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:18.438 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:55:18.481 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:55:18.484+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:55:18.548 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:18.548 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:18.557 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:55:18.578 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:55:18.581+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:55:18.680 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:55:18.682+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:55:18.703 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:18.703 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:18.713 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:55:18.781 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:55:18.783+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:55:18.816 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:18.817 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:18.826 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:55:18.881 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:55:18.883+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:55:18.932 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:18.932 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:18.940 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:55:18.985 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:55:18.988+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:55:19.062 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:19.062 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:19.071 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:55:19.112 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:55:19.115+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:55:19.185 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:55:19.188+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:55:19.222 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:19.222 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:19.231 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:55:19.286 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:55:19.288+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:55:19.341 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:19.341 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:19.350 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:55:19.386 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:55:19.389+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:55:19.487 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:55:19.490+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:55:19.503 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:19.503 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:19.513 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:55:19.596 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:55:19.599+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:55:19.663 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:19.663 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:19.672 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:55:19.673 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T08:55:19.692 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:55:19.695+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:55:19.794 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:55:19.797+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:55:19.825 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:19.825 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:19.834 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:55:19.839 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:55:19.839 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:55:19.857 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:55:19.857 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:55:19.874 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:55:19.874 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:55:19.892 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:55:19.893 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:55:19.896 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:55:19.898+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:55:19.910 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:55:19.910 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:55:19.928 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:55:19.928 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:55:19.950 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:55:19.950 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:55:19.950 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:55:19.950 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:55:19.959 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:55:19.999 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:55:20.001+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:55:20.099 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:55:20.102+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:55:20.108 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:55:20.108 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:55:20.108 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:55:20.108 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:55:20.108 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:55:20.117 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:55:20.200 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:55:20.204+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:55:20.228 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:55:20.228 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:55:20.228 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:55:20.228 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:55:20.228 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.228 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:437 3.17 3:ea93a512:::rbd_data.1140100c9703.0000000000000077:head [set-alloc-hint object_size 2097152 write_size 2097152,write 1170432~1024] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e63)", 2024-05-08T08:55:20.228 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:08.390804+0000", 2024-05-08T08:55:20.228 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 131.83870921100001, 2024-05-08T08:55:20.228 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0406068690000001, 2024-05-08T08:55:20.228 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:20.228 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:20.229 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:55:20.229 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:20.229 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:20.229 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:20.229 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 437 2024-05-08T08:55:20.229 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.229 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:20.229 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.229 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:20.229 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390804+0000", 2024-05-08T08:55:20.229 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:20.229 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.229 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.229 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:20.229 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390804+0000", 2024-05-08T08:55:20.230 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:20.230 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.230 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.230 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:20.230 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390806+0000", 2024-05-08T08:55:20.230 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.666e-06 2024-05-08T08:55:20.230 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.230 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.230 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:20.230 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390818+0000", 2024-05-08T08:55:20.230 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2187999999999999e-05 2024-05-08T08:55:20.230 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.230 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.230 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:20.231 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390834+0000", 2024-05-08T08:55:20.231 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6191000000000001e-05 2024-05-08T08:55:20.231 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.231 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.231 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:20.231 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390845+0000", 2024-05-08T08:55:20.231 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0831e-05 2024-05-08T08:55:20.231 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.231 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.231 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:20.231 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390922+0000", 2024-05-08T08:55:20.231 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7253e-05 2024-05-08T08:55:20.231 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.231 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.231 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op must wait for map", 2024-05-08T08:55:20.232 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390930+0000", 2024-05-08T08:55:20.232 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8259999999999993e-06 2024-05-08T08:55:20.232 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.232 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.232 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:20.232 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392155+0000", 2024-05-08T08:55:20.232 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0012256649999999999 2024-05-08T08:55:20.232 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.232 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.232 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:55:20.232 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392165+0000", 2024-05-08T08:55:20.232 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5799999999999998e-06 2024-05-08T08:55:20.232 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.232 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.232 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:20.232 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.429596+0000", 2024-05-08T08:55:20.232 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0374313639999999 2024-05-08T08:55:20.233 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.233 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.233 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:20.233 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.429728+0000", 2024-05-08T08:55:20.233 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000131552 2024-05-08T08:55:20.233 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.233 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.233 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 0,1", 2024-05-08T08:55:20.233 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.429860+0000", 2024-05-08T08:55:20.233 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000132137 2024-05-08T08:55:20.233 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.233 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.233 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:55:20.233 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.430811+0000", 2024-05-08T08:55:20.233 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00095040200000000004 2024-05-08T08:55:20.234 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.234 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.234 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:55:20.234 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431126+0000", 2024-05-08T08:55:20.234 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000315006 2024-05-08T08:55:20.234 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.234 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.234 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:55:20.234 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431272+0000", 2024-05-08T08:55:20.234 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000146487 2024-05-08T08:55:20.234 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.234 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.234 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:55:20.234 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431315+0000", 2024-05-08T08:55:20.235 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2697000000000001e-05 2024-05-08T08:55:20.235 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.235 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.235 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:20.235 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431411+0000", 2024-05-08T08:55:20.235 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6024000000000001e-05 2024-05-08T08:55:20.235 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:20.235 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:20.235 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:20.235 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.235 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.235 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:439 3.17 3:ea93a512:::rbd_data.1140100c9703.0000000000000077:head [read 1163264~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-05-08T08:55:20.235 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:08.390871+0000", 2024-05-08T08:55:20.235 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 131.83864170000001, 2024-05-08T08:55:20.235 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0408790590000001, 2024-05-08T08:55:20.236 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:20.236 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:20.236 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:20.236 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:20.236 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:20.236 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:20.236 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 439 2024-05-08T08:55:20.236 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.236 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:20.236 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.236 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:20.236 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390871+0000", 2024-05-08T08:55:20.236 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:20.236 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.236 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.236 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:20.236 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390871+0000", 2024-05-08T08:55:20.237 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:20.237 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.237 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.237 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:20.237 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390872+0000", 2024-05-08T08:55:20.237 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6300000000000005e-07 2024-05-08T08:55:20.237 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.237 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.237 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:20.237 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390875+0000", 2024-05-08T08:55:20.237 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3459999999999998e-06 2024-05-08T08:55:20.237 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.237 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.237 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:20.238 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390891+0000", 2024-05-08T08:55:20.238 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5255e-05 2024-05-08T08:55:20.238 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.238 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.238 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:20.238 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390897+0000", 2024-05-08T08:55:20.238 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2360000000000001e-06 2024-05-08T08:55:20.238 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.238 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.238 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:20.238 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390995+0000", 2024-05-08T08:55:20.238 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8522000000000003e-05 2024-05-08T08:55:20.238 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.238 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.239 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting_for_map not empty", 2024-05-08T08:55:20.239 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.391005+0000", 2024-05-08T08:55:20.239 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1460000000000002e-06 2024-05-08T08:55:20.239 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.239 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.239 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:20.239 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392191+0000", 2024-05-08T08:55:20.239 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0011860270000000001 2024-05-08T08:55:20.239 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.239 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.239 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:55:20.239 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392197+0000", 2024-05-08T08:55:20.239 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4289999999999998e-06 2024-05-08T08:55:20.239 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.239 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.240 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:20.240 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.430596+0000", 2024-05-08T08:55:20.240 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0383989090000001 2024-05-08T08:55:20.240 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.240 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.240 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for rw locks", 2024-05-08T08:55:20.240 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.430676+0000", 2024-05-08T08:55:20.240 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9781000000000001e-05 2024-05-08T08:55:20.240 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.240 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.240 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:20.240 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431460+0000", 2024-05-08T08:55:20.240 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00078427300000000004 2024-05-08T08:55:20.240 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.240 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.240 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:20.241 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431540+0000", 2024-05-08T08:55:20.241 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0224999999999998e-05 2024-05-08T08:55:20.241 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.241 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.241 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:20.241 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431750+0000", 2024-05-08T08:55:20.241 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00021024699999999999 2024-05-08T08:55:20.241 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:20.241 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:20.241 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:20.241 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.241 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.241 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:965 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 950272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:20.241 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451075+0000", 2024-05-08T08:55:20.241 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 130.77843789799999, 2024-05-08T08:55:20.241 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087230196999999995, 2024-05-08T08:55:20.242 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:20.242 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:20.242 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:20.242 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:20.242 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:20.242 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:20.242 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 965 2024-05-08T08:55:20.242 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.242 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:20.242 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.242 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:20.242 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451075+0000", 2024-05-08T08:55:20.242 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:20.242 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.242 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.243 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:20.243 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451075+0000", 2024-05-08T08:55:20.243 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:20.243 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.243 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.243 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:20.243 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451076+0000", 2024-05-08T08:55:20.243 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5000000000000002e-07 2024-05-08T08:55:20.243 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.243 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.243 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:20.243 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451078+0000", 2024-05-08T08:55:20.243 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.5639999999999999e-06 2024-05-08T08:55:20.243 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.243 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.243 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:20.244 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451085+0000", 2024-05-08T08:55:20.244 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5640000000000002e-06 2024-05-08T08:55:20.244 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.244 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.244 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:20.244 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451090+0000", 2024-05-08T08:55:20.244 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1699999999999996e-06 2024-05-08T08:55:20.244 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.244 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.244 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:20.244 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538106+0000", 2024-05-08T08:55:20.244 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087015888999999999 2024-05-08T08:55:20.244 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.244 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.244 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:20.244 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538190+0000", 2024-05-08T08:55:20.244 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3961999999999993e-05 2024-05-08T08:55:20.245 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.245 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.245 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:20.245 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538305+0000", 2024-05-08T08:55:20.245 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011529800000000001 2024-05-08T08:55:20.245 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:20.245 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:20.245 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:20.245 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.245 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.245 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:966 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 1441792~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:20.245 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:55:20.245 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 130.77841452499999, 2024-05-08T08:55:20.245 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087446323000000006, 2024-05-08T08:55:20.245 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:20.246 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:20.246 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:20.246 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:20.246 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:20.246 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:20.246 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 966 2024-05-08T08:55:20.246 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.246 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:20.246 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.246 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:20.246 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:55:20.246 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:20.246 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.246 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.246 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:20.247 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:55:20.247 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:20.247 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.247 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.247 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:20.247 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:55:20.247 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3700000000000001e-07 2024-05-08T08:55:20.247 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.247 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.247 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:20.247 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451101+0000", 2024-05-08T08:55:20.247 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2560000000000001e-06 2024-05-08T08:55:20.247 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.247 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.247 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:20.248 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451107+0000", 2024-05-08T08:55:20.248 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1070000000000001e-06 2024-05-08T08:55:20.248 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.248 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.248 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:20.248 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451112+0000", 2024-05-08T08:55:20.248 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1420000000000001e-06 2024-05-08T08:55:20.248 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.248 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.248 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:20.248 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538341+0000", 2024-05-08T08:55:20.248 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087228302999999993 2024-05-08T08:55:20.248 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.248 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.248 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:20.248 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538424+0000", 2024-05-08T08:55:20.248 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3497000000000004e-05 2024-05-08T08:55:20.249 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.249 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.249 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:20.249 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538545+0000", 2024-05-08T08:55:20.249 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000120581 2024-05-08T08:55:20.249 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:20.249 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:20.249 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:20.249 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.249 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.249 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:967 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:20.249 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451120+0000", 2024-05-08T08:55:20.249 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 130.77839305000001, 2024-05-08T08:55:20.249 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087632725999999994, 2024-05-08T08:55:20.249 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:20.249 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:20.249 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:20.250 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:20.250 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:20.250 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:20.250 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 967 2024-05-08T08:55:20.250 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.250 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:20.250 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.250 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:20.250 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451120+0000", 2024-05-08T08:55:20.250 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:20.250 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.250 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.250 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:20.250 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451120+0000", 2024-05-08T08:55:20.251 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:20.251 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.251 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.251 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:20.251 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451121+0000", 2024-05-08T08:55:20.251 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8800000000000002e-07 2024-05-08T08:55:20.251 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.251 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.251 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:20.251 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451130+0000", 2024-05-08T08:55:20.251 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2299999999999997e-06 2024-05-08T08:55:20.251 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.251 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.251 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:20.251 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451137+0000", 2024-05-08T08:55:20.252 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4669999999999998e-06 2024-05-08T08:55:20.252 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.252 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.252 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:20.252 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451143+0000", 2024-05-08T08:55:20.252 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2839999999999997e-06 2024-05-08T08:55:20.252 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.252 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.252 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:20.252 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538575+0000", 2024-05-08T08:55:20.252 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087432789999999996 2024-05-08T08:55:20.252 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.252 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.252 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:20.253 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538648+0000", 2024-05-08T08:55:20.253 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2877e-05 2024-05-08T08:55:20.253 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.253 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.253 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:20.253 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538753+0000", 2024-05-08T08:55:20.253 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010449 2024-05-08T08:55:20.253 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:20.253 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:20.253 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:20.253 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.253 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.253 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:968 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 2424832~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:20.253 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451152+0000", 2024-05-08T08:55:20.253 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 130.77836096499999, 2024-05-08T08:55:20.253 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087813325999999997, 2024-05-08T08:55:20.254 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:20.254 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:20.254 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:20.254 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:20.254 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:20.254 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:20.254 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 968 2024-05-08T08:55:20.254 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.254 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:20.254 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.254 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:20.254 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451152+0000", 2024-05-08T08:55:20.254 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:20.254 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.254 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.255 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:20.255 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451152+0000", 2024-05-08T08:55:20.255 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:20.255 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.255 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.255 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:20.255 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451153+0000", 2024-05-08T08:55:20.255 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3099999999999998e-07 2024-05-08T08:55:20.255 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.255 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.255 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:20.255 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451155+0000", 2024-05-08T08:55:20.255 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4760000000000001e-06 2024-05-08T08:55:20.255 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.256 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.256 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:20.256 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451162+0000", 2024-05-08T08:55:20.256 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5969999999999997e-06 2024-05-08T08:55:20.256 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.256 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.256 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:20.256 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451167+0000", 2024-05-08T08:55:20.256 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.977e-06 2024-05-08T08:55:20.256 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.256 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.256 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:20.256 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538781+0000", 2024-05-08T08:55:20.256 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087614846999999996 2024-05-08T08:55:20.256 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.256 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.257 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:20.257 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538857+0000", 2024-05-08T08:55:20.257 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5358000000000006e-05 2024-05-08T08:55:20.257 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.257 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.257 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:20.257 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538965+0000", 2024-05-08T08:55:20.257 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010864 2024-05-08T08:55:20.257 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:20.257 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:20.257 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:20.257 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.257 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.257 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:969 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 2916352~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:20.258 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:55:20.258 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 130.77833926, 2024-05-08T08:55:20.258 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087998980000000004, 2024-05-08T08:55:20.258 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:20.258 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:20.258 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:20.258 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:20.258 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:20.258 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:20.258 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 969 2024-05-08T08:55:20.258 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.258 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:20.258 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.258 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:20.259 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:55:20.259 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:20.259 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.259 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.259 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:20.259 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:55:20.259 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:20.259 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.259 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.259 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:20.259 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:55:20.259 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3599999999999999e-07 2024-05-08T08:55:20.259 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.259 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.259 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:20.259 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451177+0000", 2024-05-08T08:55:20.260 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4590000000000001e-06 2024-05-08T08:55:20.260 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.260 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.260 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:20.260 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451184+0000", 2024-05-08T08:55:20.260 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4109999999999999e-06 2024-05-08T08:55:20.260 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.260 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.260 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:20.260 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451191+0000", 2024-05-08T08:55:20.260 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7279999999999998e-06 2024-05-08T08:55:20.260 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.260 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.260 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:20.260 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538997+0000", 2024-05-08T08:55:20.261 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087806170000000003 2024-05-08T08:55:20.261 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.261 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.261 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:20.261 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539069+0000", 2024-05-08T08:55:20.261 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2373999999999995e-05 2024-05-08T08:55:20.261 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.261 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.261 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:20.261 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539173+0000", 2024-05-08T08:55:20.261 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000103402 2024-05-08T08:55:20.261 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:20.261 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:20.261 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:20.261 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.262 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.262 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:970 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 3407872~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:20.262 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451198+0000", 2024-05-08T08:55:20.262 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 130.778314663, 2024-05-08T08:55:20.262 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088183368999999998, 2024-05-08T08:55:20.262 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:20.262 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:20.262 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:20.262 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:20.262 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:20.262 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:20.262 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 970 2024-05-08T08:55:20.262 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.262 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:20.262 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.263 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:20.263 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451198+0000", 2024-05-08T08:55:20.263 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:20.263 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.263 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.263 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:20.263 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451198+0000", 2024-05-08T08:55:20.263 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:20.263 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.263 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.263 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:20.263 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451199+0000", 2024-05-08T08:55:20.263 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7400000000000003e-07 2024-05-08T08:55:20.263 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.263 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.264 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:20.264 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451211+0000", 2024-05-08T08:55:20.264 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2109999999999999e-05 2024-05-08T08:55:20.264 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.264 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.264 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:20.264 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451219+0000", 2024-05-08T08:55:20.264 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4170000000000003e-06 2024-05-08T08:55:20.264 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.264 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.264 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:20.264 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451224+0000", 2024-05-08T08:55:20.264 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3990000000000003e-06 2024-05-08T08:55:20.264 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.264 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.264 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:20.265 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539204+0000", 2024-05-08T08:55:20.265 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087980099000000006 2024-05-08T08:55:20.265 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.265 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.265 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:20.265 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539276+0000", 2024-05-08T08:55:20.265 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1513000000000005e-05 2024-05-08T08:55:20.265 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.265 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.265 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:20.265 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539382+0000", 2024-05-08T08:55:20.265 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010615699999999999 2024-05-08T08:55:20.265 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:20.265 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:20.265 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:20.265 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.266 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.266 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:971 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 3899392~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:20.266 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451233+0000", 2024-05-08T08:55:20.266 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 130.77827995600001, 2024-05-08T08:55:20.266 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088431525999999996, 2024-05-08T08:55:20.266 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:20.266 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:20.266 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:20.266 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:20.266 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:20.266 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:20.266 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 971 2024-05-08T08:55:20.266 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.266 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:20.267 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.267 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:20.267 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451233+0000", 2024-05-08T08:55:20.267 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:20.267 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.267 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.267 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:20.267 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451233+0000", 2024-05-08T08:55:20.267 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:20.267 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.267 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.267 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:20.267 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451234+0000", 2024-05-08T08:55:20.267 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8500000000000001e-07 2024-05-08T08:55:20.268 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.268 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.268 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:20.268 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451237+0000", 2024-05-08T08:55:20.268 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8320000000000002e-06 2024-05-08T08:55:20.268 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.268 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.268 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:20.268 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451243+0000", 2024-05-08T08:55:20.268 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6749999999999996e-06 2024-05-08T08:55:20.268 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.268 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.268 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:20.268 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451249+0000", 2024-05-08T08:55:20.269 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1229999999999997e-06 2024-05-08T08:55:20.269 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.269 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.269 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:20.269 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539412+0000", 2024-05-08T08:55:20.269 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088162668999999999 2024-05-08T08:55:20.269 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.269 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.269 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:20.269 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539505+0000", 2024-05-08T08:55:20.269 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3240999999999997e-05 2024-05-08T08:55:20.269 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.269 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.269 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:20.270 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539665+0000", 2024-05-08T08:55:20.270 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00015930100000000001 2024-05-08T08:55:20.270 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:20.270 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:20.270 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:20.270 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.270 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.270 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:972 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 196608~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:20.270 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451257+0000", 2024-05-08T08:55:20.270 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 130.778256011, 2024-05-08T08:55:20.270 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088719928000000003, 2024-05-08T08:55:20.270 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:20.270 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:20.270 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:20.271 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:20.271 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:20.271 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:20.271 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 972 2024-05-08T08:55:20.271 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.271 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:20.271 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.271 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:20.271 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451257+0000", 2024-05-08T08:55:20.271 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:20.271 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.271 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.271 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:20.271 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451257+0000", 2024-05-08T08:55:20.272 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:20.272 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.272 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.272 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:20.272 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451258+0000", 2024-05-08T08:55:20.272 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7999999999999996e-07 2024-05-08T08:55:20.272 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.272 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.272 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:20.272 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451260+0000", 2024-05-08T08:55:20.272 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4609999999999998e-06 2024-05-08T08:55:20.272 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.272 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.272 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:20.272 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451267+0000", 2024-05-08T08:55:20.273 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.81e-06 2024-05-08T08:55:20.273 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.273 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.273 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:20.273 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451273+0000", 2024-05-08T08:55:20.273 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0859999999999998e-06 2024-05-08T08:55:20.273 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.273 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.273 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:20.273 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539718+0000", 2024-05-08T08:55:20.273 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088445059000000006 2024-05-08T08:55:20.273 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.273 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.273 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:20.273 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539834+0000", 2024-05-08T08:55:20.273 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000116426 2024-05-08T08:55:20.274 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.274 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.274 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:20.274 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539977+0000", 2024-05-08T08:55:20.274 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014260600000000001 2024-05-08T08:55:20.274 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:20.274 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:20.274 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:20.274 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.274 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.274 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:973 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 688128~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:20.274 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:55:20.274 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 130.778233427, 2024-05-08T08:55:20.275 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088918496, 2024-05-08T08:55:20.275 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:20.275 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:20.275 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:20.275 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:20.275 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:20.275 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:20.275 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 973 2024-05-08T08:55:20.275 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.275 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:20.275 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.275 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:20.275 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:55:20.275 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:20.275 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.276 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.276 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:20.276 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:55:20.276 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:20.276 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.276 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.276 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:20.276 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:55:20.276 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9099999999999999e-07 2024-05-08T08:55:20.276 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.276 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.276 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:20.276 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451282+0000", 2024-05-08T08:55:20.276 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1550000000000001e-06 2024-05-08T08:55:20.276 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.276 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.277 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:20.277 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451288+0000", 2024-05-08T08:55:20.277 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8889999999999999e-06 2024-05-08T08:55:20.277 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.277 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.277 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:20.277 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451294+0000", 2024-05-08T08:55:20.277 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0850000000000002e-06 2024-05-08T08:55:20.277 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.277 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.277 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:20.277 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540011+0000", 2024-05-08T08:55:20.277 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088717314000000005 2024-05-08T08:55:20.277 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.277 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.277 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:20.278 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540089+0000", 2024-05-08T08:55:20.278 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7438e-05 2024-05-08T08:55:20.278 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.278 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.278 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:20.278 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540198+0000", 2024-05-08T08:55:20.278 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000109224 2024-05-08T08:55:20.278 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:20.278 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:20.278 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:20.278 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.278 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.278 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:974 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 1179648~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:20.278 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451301+0000", 2024-05-08T08:55:20.279 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 130.77821179399999, 2024-05-08T08:55:20.279 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089105078000000004, 2024-05-08T08:55:20.279 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:20.279 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:20.279 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:20.279 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:20.279 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:20.279 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:20.279 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 974 2024-05-08T08:55:20.279 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.279 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:20.279 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.279 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:20.279 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451301+0000", 2024-05-08T08:55:20.279 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:20.280 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.280 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.280 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:20.280 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451301+0000", 2024-05-08T08:55:20.280 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:20.280 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.280 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.280 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:20.280 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451302+0000", 2024-05-08T08:55:20.280 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7800000000000002e-07 2024-05-08T08:55:20.280 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.280 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.280 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:20.280 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451304+0000", 2024-05-08T08:55:20.280 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.65e-06 2024-05-08T08:55:20.280 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.281 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.281 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:20.281 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451311+0000", 2024-05-08T08:55:20.281 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.472e-06 2024-05-08T08:55:20.281 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.281 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.281 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:20.281 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451316+0000", 2024-05-08T08:55:20.281 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9189999999999998e-06 2024-05-08T08:55:20.281 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.281 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.281 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:20.281 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540234+0000", 2024-05-08T08:55:20.281 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088917752000000003 2024-05-08T08:55:20.281 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.281 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.282 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:20.282 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540308+0000", 2024-05-08T08:55:20.282 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4758000000000005e-05 2024-05-08T08:55:20.282 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.282 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.282 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:20.282 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540406+0000", 2024-05-08T08:55:20.282 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8048999999999996e-05 2024-05-08T08:55:20.282 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:20.282 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:20.282 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:20.282 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.282 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.282 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:975 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 1671168~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:20.283 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:55:20.283 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 130.778169464, 2024-05-08T08:55:20.283 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089294139999999994, 2024-05-08T08:55:20.283 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:20.283 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:20.283 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:20.283 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:20.283 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:20.283 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:20.283 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 975 2024-05-08T08:55:20.283 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.283 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:20.283 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.283 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:20.284 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:55:20.284 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:20.284 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.284 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.284 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:20.284 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:55:20.284 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:20.284 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.284 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.284 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:20.284 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:55:20.284 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1299999999999999e-07 2024-05-08T08:55:20.284 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.284 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.284 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:20.284 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451348+0000", 2024-05-08T08:55:20.285 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.484e-06 2024-05-08T08:55:20.285 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.285 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.285 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:20.285 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451356+0000", 2024-05-08T08:55:20.285 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8499999999999994e-06 2024-05-08T08:55:20.285 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.285 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.285 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:20.285 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451366+0000", 2024-05-08T08:55:20.285 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.9850000000000001e-06 2024-05-08T08:55:20.285 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.285 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.285 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:20.286 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540448+0000", 2024-05-08T08:55:20.286 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089082254 2024-05-08T08:55:20.286 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.286 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.286 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:20.286 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540528+0000", 2024-05-08T08:55:20.286 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0234999999999993e-05 2024-05-08T08:55:20.286 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.286 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.286 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:20.286 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540638+0000", 2024-05-08T08:55:20.286 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010951900000000001 2024-05-08T08:55:20.286 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:20.286 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:20.286 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:20.287 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.287 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.287 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:976 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 2162688~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:20.287 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451375+0000", 2024-05-08T08:55:20.287 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 130.77813771699999, 2024-05-08T08:55:20.287 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089472745000000006, 2024-05-08T08:55:20.287 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:20.287 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:20.287 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:20.287 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:20.287 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:20.287 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:20.287 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 976 2024-05-08T08:55:20.287 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.287 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:20.288 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.288 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:20.288 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451375+0000", 2024-05-08T08:55:20.288 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:20.288 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.288 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.288 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:20.288 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451375+0000", 2024-05-08T08:55:20.288 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:20.288 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.288 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.288 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:20.288 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451376+0000", 2024-05-08T08:55:20.288 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2399999999999998e-07 2024-05-08T08:55:20.288 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.289 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.289 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:20.289 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451379+0000", 2024-05-08T08:55:20.289 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6359999999999998e-06 2024-05-08T08:55:20.289 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.289 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.289 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:20.289 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451387+0000", 2024-05-08T08:55:20.289 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9929999999999995e-06 2024-05-08T08:55:20.289 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.289 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.289 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:20.289 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451393+0000", 2024-05-08T08:55:20.289 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4699999999999999e-06 2024-05-08T08:55:20.289 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.290 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.290 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:20.290 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540677+0000", 2024-05-08T08:55:20.290 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089283504999999999 2024-05-08T08:55:20.290 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.290 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.290 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:20.290 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540743+0000", 2024-05-08T08:55:20.290 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6303999999999994e-05 2024-05-08T08:55:20.290 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.290 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.290 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:20.290 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540848+0000", 2024-05-08T08:55:20.290 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000105213 2024-05-08T08:55:20.291 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:20.291 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:20.291 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:20.291 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.291 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.291 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:977 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 2654208~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:20.291 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:55:20.291 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 130.77811228499999, 2024-05-08T08:55:20.291 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089698473000000001, 2024-05-08T08:55:20.291 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:20.291 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:20.291 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:20.291 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:20.291 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:20.291 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:20.291 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 977 2024-05-08T08:55:20.292 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.292 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:20.292 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.292 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:20.292 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:55:20.292 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:20.292 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.292 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.292 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:20.292 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:55:20.292 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:20.292 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.292 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.292 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:20.292 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:55:20.292 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2899999999999999e-07 2024-05-08T08:55:20.293 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.293 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.293 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:20.293 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451404+0000", 2024-05-08T08:55:20.293 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6649999999999999e-06 2024-05-08T08:55:20.293 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.293 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.293 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:20.293 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451410+0000", 2024-05-08T08:55:20.293 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3269999999999997e-06 2024-05-08T08:55:20.293 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.293 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.293 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:20.293 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451415+0000", 2024-05-08T08:55:20.293 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9529999999999998e-06 2024-05-08T08:55:20.294 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.294 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.294 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:20.294 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540912+0000", 2024-05-08T08:55:20.294 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089496950000000006 2024-05-08T08:55:20.294 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.294 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.294 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:20.294 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541000+0000", 2024-05-08T08:55:20.294 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7429000000000006e-05 2024-05-08T08:55:20.294 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.294 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.294 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:20.294 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541099+0000", 2024-05-08T08:55:20.294 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.9720000000000001e-05 2024-05-08T08:55:20.295 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:20.295 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:20.295 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:20.295 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.295 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.295 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:978 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 3145728~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:20.295 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:55:20.295 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 130.778091077, 2024-05-08T08:55:20.295 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089900635000000007, 2024-05-08T08:55:20.295 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:20.295 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:20.295 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:20.295 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:20.295 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:20.295 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:20.295 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 978 2024-05-08T08:55:20.295 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.296 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:20.296 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.296 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:20.296 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:55:20.296 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:20.296 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.296 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.296 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:20.296 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:55:20.296 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:20.296 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.296 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.296 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:20.296 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:55:20.296 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9400000000000001e-07 2024-05-08T08:55:20.297 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.297 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.297 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:20.297 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451425+0000", 2024-05-08T08:55:20.297 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2460000000000002e-06 2024-05-08T08:55:20.297 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.297 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.297 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:20.297 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451431+0000", 2024-05-08T08:55:20.297 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2419999999999997e-06 2024-05-08T08:55:20.297 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.297 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.297 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:20.297 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451436+0000", 2024-05-08T08:55:20.297 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.887e-06 2024-05-08T08:55:20.297 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.297 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.298 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:20.298 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541152+0000", 2024-05-08T08:55:20.298 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089716194999999999 2024-05-08T08:55:20.298 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.298 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.298 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:20.298 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541224+0000", 2024-05-08T08:55:20.298 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2416000000000006e-05 2024-05-08T08:55:20.298 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.298 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.298 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:20.298 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541323+0000", 2024-05-08T08:55:20.298 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8255e-05 2024-05-08T08:55:20.298 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:20.298 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:20.299 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:20.299 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.299 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.299 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:979 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 3637248~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:20.299 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451462+0000", 2024-05-08T08:55:20.299 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 130.77805062799999, 2024-05-08T08:55:20.299 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.090036441999999994, 2024-05-08T08:55:20.299 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:20.299 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:20.299 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:20.299 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:20.299 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:20.299 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:20.299 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 979 2024-05-08T08:55:20.299 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.300 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:20.300 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.300 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:20.300 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451462+0000", 2024-05-08T08:55:20.300 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:20.300 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.300 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.300 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:20.300 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451462+0000", 2024-05-08T08:55:20.300 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:20.300 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.300 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.300 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:20.301 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451463+0000", 2024-05-08T08:55:20.301 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4300000000000002e-07 2024-05-08T08:55:20.301 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.301 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.301 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:20.301 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451471+0000", 2024-05-08T08:55:20.301 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0800000000000006e-06 2024-05-08T08:55:20.301 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.301 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.301 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:20.301 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451481+0000", 2024-05-08T08:55:20.301 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.38e-06 2024-05-08T08:55:20.301 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.301 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.301 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:20.301 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451487+0000", 2024-05-08T08:55:20.302 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4590000000000003e-06 2024-05-08T08:55:20.302 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.302 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.302 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:20.302 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541350+0000", 2024-05-08T08:55:20.302 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089862599000000001 2024-05-08T08:55:20.302 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.302 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.302 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:20.302 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541407+0000", 2024-05-08T08:55:20.302 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7667999999999997e-05 2024-05-08T08:55:20.302 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.302 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.302 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:20.302 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541499+0000", 2024-05-08T08:55:20.303 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1513000000000003e-05 2024-05-08T08:55:20.303 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:20.303 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:20.303 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:20.303 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.303 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.303 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:980 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 4128768~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:20.303 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451496+0000", 2024-05-08T08:55:20.303 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 130.77801687199999, 2024-05-08T08:55:20.303 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.090258985, 2024-05-08T08:55:20.303 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:20.303 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:20.303 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:20.303 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:20.303 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:20.303 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:20.303 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 980 2024-05-08T08:55:20.304 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.304 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:20.304 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.304 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:20.304 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451496+0000", 2024-05-08T08:55:20.304 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:20.304 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.304 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.304 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:20.304 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451496+0000", 2024-05-08T08:55:20.304 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:20.304 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.304 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.304 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:20.304 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451497+0000", 2024-05-08T08:55:20.304 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7599999999999997e-07 2024-05-08T08:55:20.305 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.305 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.305 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:20.305 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451500+0000", 2024-05-08T08:55:20.305 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7930000000000002e-06 2024-05-08T08:55:20.305 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.305 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.305 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:20.305 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451506+0000", 2024-05-08T08:55:20.305 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6980000000000001e-06 2024-05-08T08:55:20.305 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.305 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.305 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:20.305 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451512+0000", 2024-05-08T08:55:20.305 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2490000000000001e-06 2024-05-08T08:55:20.306 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.306 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.306 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:20.306 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541550+0000", 2024-05-08T08:55:20.306 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.090037961999999999 2024-05-08T08:55:20.306 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.306 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.306 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:20.306 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541647+0000", 2024-05-08T08:55:20.306 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6924999999999998e-05 2024-05-08T08:55:20.306 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.306 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.306 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:20.306 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541755+0000", 2024-05-08T08:55:20.306 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000108682 2024-05-08T08:55:20.307 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:20.307 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:20.307 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:20.307 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.307 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.307 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:33397 3.7 3:e5a023c9:::rbd_header.11402419351a:head [watch reconnect cookie 140139347723488 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e156)", 2024-05-08T08:55:20.307 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:55:07.435461+0000", 2024-05-08T08:55:20.307 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 12.794051573000001, 2024-05-08T08:55:20.307 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.009753092, 2024-05-08T08:55:20.307 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:20.307 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:20.307 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:20.307 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:20.307 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:55:20.307 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:55:20.308 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 33397 2024-05-08T08:55:20.308 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.308 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:20.308 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.308 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:20.308 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:07.435461+0000", 2024-05-08T08:55:20.308 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:20.308 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.308 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.308 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:20.308 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:07.435461+0000", 2024-05-08T08:55:20.308 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:20.308 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.308 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.308 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:20.308 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:07.435463+0000", 2024-05-08T08:55:20.309 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.7749999999999999e-06 2024-05-08T08:55:20.309 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.309 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.309 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:20.309 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:07.435471+0000", 2024-05-08T08:55:20.309 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.2199999999999992e-06 2024-05-08T08:55:20.309 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.309 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.309 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:20.309 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:07.435487+0000", 2024-05-08T08:55:20.309 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5806000000000001e-05 2024-05-08T08:55:20.309 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.309 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.309 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:20.309 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:07.435867+0000", 2024-05-08T08:55:20.309 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00037973999999999999 2024-05-08T08:55:20.310 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.310 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.310 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:20.310 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:07.436510+0000", 2024-05-08T08:55:20.310 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00064263299999999998 2024-05-08T08:55:20.310 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.310 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.310 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:55:20.310 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:07.436524+0000", 2024-05-08T08:55:20.310 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.4316e-05 2024-05-08T08:55:20.310 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.310 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.310 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:20.310 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:08.439082+0000", 2024-05-08T08:55:20.310 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.002558235 2024-05-08T08:55:20.311 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.311 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.311 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:20.311 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:08.439286+0000", 2024-05-08T08:55:20.311 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00020426600000000001 2024-05-08T08:55:20.311 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.311 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.311 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:20.311 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:08.445215+0000", 2024-05-08T08:55:20.311 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0059281009999999999 2024-05-08T08:55:20.311 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:20.311 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:20.311 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:20.311 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.311 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.311 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:33398 3.7 3.93c405a7 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e156)", 2024-05-08T08:55:20.312 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:55:07.435912+0000", 2024-05-08T08:55:20.312 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 12.793601375, 2024-05-08T08:55:20.312 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0044546430000001, 2024-05-08T08:55:20.312 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:20.312 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:20.312 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "reached pg", 2024-05-08T08:55:20.312 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:20.312 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:55:20.312 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:55:20.312 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 33398 2024-05-08T08:55:20.312 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.312 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:20.312 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.312 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:20.313 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:07.435912+0000", 2024-05-08T08:55:20.313 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:20.313 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.313 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.313 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:20.313 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:07.435912+0000", 2024-05-08T08:55:20.313 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:20.313 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.313 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.313 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:20.314 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:07.435913+0000", 2024-05-08T08:55:20.314 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2200000000000002e-07 2024-05-08T08:55:20.314 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.314 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.314 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:20.314 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:07.435919+0000", 2024-05-08T08:55:20.314 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3820000000000001e-06 2024-05-08T08:55:20.314 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.314 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.314 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:20.314 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:07.435932+0000", 2024-05-08T08:55:20.314 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2788e-05 2024-05-08T08:55:20.314 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.314 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.315 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:20.315 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:07.435940+0000", 2024-05-08T08:55:20.315 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.6710000000000005e-06 2024-05-08T08:55:20.315 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.315 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.315 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:20.315 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:07.436558+0000", 2024-05-08T08:55:20.315 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00061782299999999998 2024-05-08T08:55:20.315 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.315 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.315 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:55:20.315 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:07.436572+0000", 2024-05-08T08:55:20.315 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.34e-05 2024-05-08T08:55:20.315 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.316 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.316 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:20.316 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:08.440285+0000", 2024-05-08T08:55:20.316 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.003713388 2024-05-08T08:55:20.316 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:20.316 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:20.316 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:20.316 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:08.440366+0000", 2024-05-08T08:55:20.316 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1268999999999997e-05 2024-05-08T08:55:20.316 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:20.316 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:20.316 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:20.316 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:20.316 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:20.316 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:55:20.318 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:55:20.318 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:55:20.320+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:55:20.401 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-05-08T08:55:20.418 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:55:20.421+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:55:20.444 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:55:20.444 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:55:20.444 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:55:20.444 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:55:20.453 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:55:20.537 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:55:20.540+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:55:20.611 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:55:20.612 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:55:20.612 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:55:20.612 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:55:20.612 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:55:20.620 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:55:20.622+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:55:20.621 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:55:20.664 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4] out_osds: [5, 0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-05-08T08:55:20.664 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T08:55:20.665 INFO:tasks.thrashosds.thrasher:inject_pause on osd.0 2024-05-08T08:55:20.665 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-05-08T08:55:20.665 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-05-08T08:55:20.665 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok config set filestore_inject_stall 3 2024-05-08T08:55:20.721 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:55:20.724+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:55:20.770 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:55:20.770 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:55:20.770 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:55:20.770 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:55:20.770 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.770 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4621.0:9089 3.1f e33/18 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:head v 33'117, mlcod=31'115)", 2024-05-08T08:55:20.770 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:55:20.771 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 168.25212082799999, 2024-05-08T08:55:20.771 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.111244524, 2024-05-08T08:55:20.771 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:20.771 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:20.771 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:55:20.771 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:20.771 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.771 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:20.771 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:55:20.771 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:20.771 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.771 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.771 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:20.771 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:55:20.772 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:20.772 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.772 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.772 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:20.772 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:55:20.772 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5900000000000001e-07 2024-05-08T08:55:20.772 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.772 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.772 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:20.772 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539600+0000", 2024-05-08T08:55:20.772 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.019981364000000001 2024-05-08T08:55:20.772 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.772 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.773 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:20.773 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539624+0000", 2024-05-08T08:55:20.773 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4043e-05 2024-05-08T08:55:20.773 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.773 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.773 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:20.773 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539635+0000", 2024-05-08T08:55:20.773 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1511e-05 2024-05-08T08:55:20.773 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.773 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.773 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:20.773 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539728+0000", 2024-05-08T08:55:20.773 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2498999999999996e-05 2024-05-08T08:55:20.773 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.774 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.774 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:20.774 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539746+0000", 2024-05-08T08:55:20.774 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8289000000000001e-05 2024-05-08T08:55:20.774 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.774 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.774 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:55:20.774 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.630822+0000", 2024-05-08T08:55:20.774 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.091076283999999993 2024-05-08T08:55:20.774 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.774 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.774 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:20.774 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.630863+0000", 2024-05-08T08:55:20.775 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0374999999999999e-05 2024-05-08T08:55:20.775 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:20.775 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:20.775 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:20.775 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.775 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.775 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4195.0:4 3.11 3:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-05-08T08:55:20.775 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:49.764654+0000", 2024-05-08T08:55:20.775 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 151.007085287, 2024-05-08T08:55:20.775 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.30714275499999999, 2024-05-08T08:55:20.775 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:20.775 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:20.775 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-05-08T08:55:20.775 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:20.775 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4195", 2024-05-08T08:55:20.776 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/59179521", 2024-05-08T08:55:20.776 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4 2024-05-08T08:55:20.776 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.776 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:20.776 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.776 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:20.776 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764654+0000", 2024-05-08T08:55:20.776 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:20.776 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.776 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.776 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:20.776 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764654+0000", 2024-05-08T08:55:20.776 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:20.776 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.776 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.776 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:20.777 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764655+0000", 2024-05-08T08:55:20.777 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.452e-06 2024-05-08T08:55:20.777 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.777 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.777 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:20.777 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764661+0000", 2024-05-08T08:55:20.777 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5369999999999997e-06 2024-05-08T08:55:20.777 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.777 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.777 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:20.777 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764671+0000", 2024-05-08T08:55:20.777 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.028e-05 2024-05-08T08:55:20.777 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.777 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.777 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:20.778 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764680+0000", 2024-05-08T08:55:20.778 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0629999999999995e-06 2024-05-08T08:55:20.778 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.778 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.778 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:20.778 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764753+0000", 2024-05-08T08:55:20.778 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2679e-05 2024-05-08T08:55:20.778 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.778 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.778 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-05-08T08:55:20.778 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764765+0000", 2024-05-08T08:55:20.778 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2415000000000001e-05 2024-05-08T08:55:20.778 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.778 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.779 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:20.779 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:50.071692+0000", 2024-05-08T08:55:20.779 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.306927324 2024-05-08T08:55:20.779 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.779 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.779 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:20.779 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:50.071796+0000", 2024-05-08T08:55:20.779 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104005 2024-05-08T08:55:20.779 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:20.779 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:20.779 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:20.779 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.779 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.779 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4416.0:20531 3.16 3:6bb2ed16:::rbd_object_map.1140bc4a55c7:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-05-08T08:55:20.779 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:58.673786+0000", 2024-05-08T08:55:20.780 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 82.097952883000005, 2024-05-08T08:55:20.780 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.94684119899999997, 2024-05-08T08:55:20.780 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:20.780 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:20.780 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:55:20.780 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:20.780 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4416", 2024-05-08T08:55:20.780 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:55:20.780 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20531 2024-05-08T08:55:20.780 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.780 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:20.780 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.780 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:20.780 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673786+0000", 2024-05-08T08:55:20.780 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:20.780 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.780 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.781 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:20.781 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673786+0000", 2024-05-08T08:55:20.781 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:20.781 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.781 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.781 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:20.781 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673787+0000", 2024-05-08T08:55:20.781 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9800000000000003e-07 2024-05-08T08:55:20.781 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.781 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.781 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:20.781 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673791+0000", 2024-05-08T08:55:20.781 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.104e-06 2024-05-08T08:55:20.781 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.781 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.782 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:20.782 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673801+0000", 2024-05-08T08:55:20.782 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9960000000000005e-06 2024-05-08T08:55:20.782 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.782 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.782 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:20.782 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673806+0000", 2024-05-08T08:55:20.782 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2320000000000001e-06 2024-05-08T08:55:20.782 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.782 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.782 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:20.782 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673860+0000", 2024-05-08T08:55:20.783 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3468999999999999e-05 2024-05-08T08:55:20.783 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.783 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.783 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-05-08T08:55:20.783 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673866+0000", 2024-05-08T08:55:20.783 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3500000000000002e-06 2024-05-08T08:55:20.783 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.783 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.783 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:20.783 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.616700+0000", 2024-05-08T08:55:20.783 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.94283390700000003 2024-05-08T08:55:20.783 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.783 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.783 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:20.784 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.616941+0000", 2024-05-08T08:55:20.784 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00024080400000000001 2024-05-08T08:55:20.784 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.784 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.784 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-05-08T08:55:20.784 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.617072+0000", 2024-05-08T08:55:20.784 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000131688 2024-05-08T08:55:20.784 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.784 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.784 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:55:20.784 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.619571+0000", 2024-05-08T08:55:20.784 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0024990149999999998 2024-05-08T08:55:20.785 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.785 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.785 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:55:20.785 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.620568+0000", 2024-05-08T08:55:20.785 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00099711000000000005 2024-05-08T08:55:20.785 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.785 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.785 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:55:20.785 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.620602+0000", 2024-05-08T08:55:20.785 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.324e-05 2024-05-08T08:55:20.785 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.785 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.785 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:20.785 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.620627+0000", 2024-05-08T08:55:20.786 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5585999999999999e-05 2024-05-08T08:55:20.786 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:20.786 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:20.786 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:20.786 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.786 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.786 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5492 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4272128~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:20.786 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.881830+0000", 2024-05-08T08:55:20.786 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 54.889909076999999, 2024-05-08T08:55:20.786 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047614006, 2024-05-08T08:55:20.786 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:20.786 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:20.786 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:20.786 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:20.786 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:20.787 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:20.787 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5492 2024-05-08T08:55:20.787 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.787 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:20.787 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.787 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:20.787 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881830+0000", 2024-05-08T08:55:20.787 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:20.787 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.787 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.787 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:20.787 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881830+0000", 2024-05-08T08:55:20.787 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:20.787 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.788 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.788 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:20.788 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881830+0000", 2024-05-08T08:55:20.788 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4099999999999999e-07 2024-05-08T08:55:20.788 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.788 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.788 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:20.788 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881832+0000", 2024-05-08T08:55:20.788 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0370000000000001e-06 2024-05-08T08:55:20.788 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.788 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.788 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:20.788 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881839+0000", 2024-05-08T08:55:20.788 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3289999999999999e-06 2024-05-08T08:55:20.788 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.789 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.789 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:20.789 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881844+0000", 2024-05-08T08:55:20.789 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1170000000000003e-06 2024-05-08T08:55:20.789 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.789 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.789 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:20.789 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929296+0000", 2024-05-08T08:55:20.789 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047452418000000003 2024-05-08T08:55:20.789 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.789 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.789 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:20.789 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929360+0000", 2024-05-08T08:55:20.789 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4197999999999998e-05 2024-05-08T08:55:20.790 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.790 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.790 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:20.790 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929444+0000", 2024-05-08T08:55:20.790 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3466000000000004e-05 2024-05-08T08:55:20.790 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:20.790 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:20.790 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:20.790 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.790 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.790 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5494 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4317184~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:20.790 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.881850+0000", 2024-05-08T08:55:20.790 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 54.889889328000002, 2024-05-08T08:55:20.790 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047723887, 2024-05-08T08:55:20.791 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:20.791 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:20.791 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:20.791 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:20.791 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:20.791 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:20.791 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5494 2024-05-08T08:55:20.791 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.791 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:20.791 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.792 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:20.792 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881850+0000", 2024-05-08T08:55:20.792 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:20.792 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.792 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.792 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:20.792 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881850+0000", 2024-05-08T08:55:20.792 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:20.792 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.792 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.792 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:20.792 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881850+0000", 2024-05-08T08:55:20.792 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8500000000000002e-07 2024-05-08T08:55:20.792 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.793 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.793 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:20.793 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881856+0000", 2024-05-08T08:55:20.793 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9510000000000003e-06 2024-05-08T08:55:20.793 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.793 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.793 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:20.793 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881862+0000", 2024-05-08T08:55:20.793 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3049999999999998e-06 2024-05-08T08:55:20.793 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.793 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.793 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:20.793 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881868+0000", 2024-05-08T08:55:20.793 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8340000000000004e-06 2024-05-08T08:55:20.793 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.793 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.794 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:20.794 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929475+0000", 2024-05-08T08:55:20.794 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047606796 2024-05-08T08:55:20.794 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.794 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.794 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:20.794 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929518+0000", 2024-05-08T08:55:20.794 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3387000000000003e-05 2024-05-08T08:55:20.794 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.794 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.794 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:20.794 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929574+0000", 2024-05-08T08:55:20.794 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5129e-05 2024-05-08T08:55:20.794 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:20.794 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:20.795 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:20.795 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.795 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.795 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5495 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4362240~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:20.795 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.881875+0000", 2024-05-08T08:55:20.795 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 54.889864461000002, 2024-05-08T08:55:20.795 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047847967999999998, 2024-05-08T08:55:20.795 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:20.795 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:20.795 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:20.795 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:20.795 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:20.795 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:20.795 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5495 2024-05-08T08:55:20.795 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.795 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:20.795 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.796 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:20.796 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881875+0000", 2024-05-08T08:55:20.796 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:20.796 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.796 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.796 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:20.796 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881875+0000", 2024-05-08T08:55:20.796 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:20.796 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.796 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.796 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:20.796 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881875+0000", 2024-05-08T08:55:20.796 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-07 2024-05-08T08:55:20.796 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.796 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.796 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:20.796 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881877+0000", 2024-05-08T08:55:20.797 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9659999999999999e-06 2024-05-08T08:55:20.797 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.797 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.797 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:20.797 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881883+0000", 2024-05-08T08:55:20.797 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7649999999999999e-06 2024-05-08T08:55:20.797 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.797 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.797 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:20.797 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881887+0000", 2024-05-08T08:55:20.797 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6689999999999997e-06 2024-05-08T08:55:20.797 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.797 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.798 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:20.798 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929600+0000", 2024-05-08T08:55:20.798 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047712746 2024-05-08T08:55:20.798 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.798 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.798 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:20.798 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929659+0000", 2024-05-08T08:55:20.798 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8936e-05 2024-05-08T08:55:20.798 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.798 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.798 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:20.798 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929723+0000", 2024-05-08T08:55:20.798 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3524999999999999e-05 2024-05-08T08:55:20.798 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:20.798 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:20.799 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:20.799 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.799 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.799 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5496 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4407296~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:20.799 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.881894+0000", 2024-05-08T08:55:20.799 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 54.889844597, 2024-05-08T08:55:20.799 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047998015999999998, 2024-05-08T08:55:20.799 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:20.799 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:20.799 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:20.799 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:20.799 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:20.799 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:20.799 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5496 2024-05-08T08:55:20.799 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.800 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:20.800 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.800 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:20.800 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881894+0000", 2024-05-08T08:55:20.800 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:20.800 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.800 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.800 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:20.800 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881894+0000", 2024-05-08T08:55:20.800 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:20.800 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.800 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.800 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:20.800 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881895+0000", 2024-05-08T08:55:20.800 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6699999999999999e-07 2024-05-08T08:55:20.800 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.800 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.801 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:20.801 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881897+0000", 2024-05-08T08:55:20.801 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0229999999999999e-06 2024-05-08T08:55:20.801 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.801 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.801 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:20.801 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881903+0000", 2024-05-08T08:55:20.801 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9850000000000003e-06 2024-05-08T08:55:20.801 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.801 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.801 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:20.801 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881908+0000", 2024-05-08T08:55:20.801 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9019999999999998e-06 2024-05-08T08:55:20.801 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.801 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.802 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:20.802 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929751+0000", 2024-05-08T08:55:20.802 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047843550999999998 2024-05-08T08:55:20.802 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.802 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.802 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:20.802 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929823+0000", 2024-05-08T08:55:20.802 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1172000000000002e-05 2024-05-08T08:55:20.802 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.802 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.802 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:20.802 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929892+0000", 2024-05-08T08:55:20.803 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9915999999999999e-05 2024-05-08T08:55:20.803 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:20.803 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:20.803 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:20.803 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.803 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.803 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5498 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4452352~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:20.803 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.881914+0000", 2024-05-08T08:55:20.804 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 54.889824838999999, 2024-05-08T08:55:20.804 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048203553000000003, 2024-05-08T08:55:20.804 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:20.804 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:20.804 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:20.804 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:20.804 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:20.804 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:20.804 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5498 2024-05-08T08:55:20.804 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.804 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:20.804 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.804 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:20.804 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881914+0000", 2024-05-08T08:55:20.805 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:20.805 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.805 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.805 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:20.805 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881914+0000", 2024-05-08T08:55:20.805 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:20.805 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.805 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.805 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:20.805 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881915+0000", 2024-05-08T08:55:20.805 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8500000000000002e-07 2024-05-08T08:55:20.805 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.805 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.805 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:20.806 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881917+0000", 2024-05-08T08:55:20.806 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.181e-06 2024-05-08T08:55:20.806 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.806 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.806 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:20.806 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881924+0000", 2024-05-08T08:55:20.806 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9619999999999996e-06 2024-05-08T08:55:20.806 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.806 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.806 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:20.806 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881929+0000", 2024-05-08T08:55:20.806 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1399999999999999e-06 2024-05-08T08:55:20.806 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.806 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.806 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:20.807 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929952+0000", 2024-05-08T08:55:20.807 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048023134000000002 2024-05-08T08:55:20.807 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.807 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.807 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:20.807 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.930029+0000", 2024-05-08T08:55:20.807 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7490000000000005e-05 2024-05-08T08:55:20.807 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.807 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.807 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:20.807 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.930118+0000", 2024-05-08T08:55:20.807 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8261000000000001e-05 2024-05-08T08:55:20.807 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:20.807 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:20.807 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:20.808 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.808 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.808 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5564 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4366336~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:20.808 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.884225+0000", 2024-05-08T08:55:20.808 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 54.887514049000004, 2024-05-08T08:55:20.808 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047555989999999999, 2024-05-08T08:55:20.808 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:20.808 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:20.808 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:20.808 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:20.808 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:20.808 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:20.808 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5564 2024-05-08T08:55:20.808 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.808 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:20.809 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.809 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:20.809 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884225+0000", 2024-05-08T08:55:20.809 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:20.809 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.809 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.809 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:20.809 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884225+0000", 2024-05-08T08:55:20.809 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:20.809 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.809 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.809 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:20.809 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884225+0000", 2024-05-08T08:55:20.809 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2499999999999995e-07 2024-05-08T08:55:20.810 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.810 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.810 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:20.810 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884227+0000", 2024-05-08T08:55:20.810 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5829999999999999e-06 2024-05-08T08:55:20.810 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.810 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.810 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:20.810 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884232+0000", 2024-05-08T08:55:20.810 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1159999999999998e-06 2024-05-08T08:55:20.810 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.810 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.810 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:20.810 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884236+0000", 2024-05-08T08:55:20.810 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2490000000000003e-06 2024-05-08T08:55:20.811 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.811 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.811 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:20.811 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931617+0000", 2024-05-08T08:55:20.811 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047380310000000002 2024-05-08T08:55:20.811 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.811 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.811 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:20.811 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931689+0000", 2024-05-08T08:55:20.811 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2358000000000001e-05 2024-05-08T08:55:20.811 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.811 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.811 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:20.811 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931781+0000", 2024-05-08T08:55:20.811 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1848999999999994e-05 2024-05-08T08:55:20.811 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:20.814 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:20.814 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:20.814 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.814 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.814 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5565 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4411392~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:20.814 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.884251+0000", 2024-05-08T08:55:20.814 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 54.887487896000003, 2024-05-08T08:55:20.815 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047743671000000001, 2024-05-08T08:55:20.815 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:20.815 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:20.815 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:20.815 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:20.815 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:20.815 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:20.815 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5565 2024-05-08T08:55:20.815 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.815 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:20.815 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.815 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:20.815 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884251+0000", 2024-05-08T08:55:20.815 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:20.815 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.815 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.816 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:20.816 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884251+0000", 2024-05-08T08:55:20.816 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:20.816 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.816 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.816 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:20.816 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884252+0000", 2024-05-08T08:55:20.816 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3200000000000005e-07 2024-05-08T08:55:20.816 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.816 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.816 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:20.816 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884253+0000", 2024-05-08T08:55:20.816 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6980000000000001e-06 2024-05-08T08:55:20.818 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.818 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.818 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:20.818 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884259+0000", 2024-05-08T08:55:20.818 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3759999999999999e-06 2024-05-08T08:55:20.818 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.818 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.819 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:20.819 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884263+0000", 2024-05-08T08:55:20.819 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7280000000000003e-06 2024-05-08T08:55:20.819 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.819 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.819 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:20.819 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931818+0000", 2024-05-08T08:55:20.819 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047554368999999999 2024-05-08T08:55:20.819 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.819 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.819 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:20.819 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931897+0000", 2024-05-08T08:55:20.820 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8821999999999999e-05 2024-05-08T08:55:20.820 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.820 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.820 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:20.820 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931995+0000", 2024-05-08T08:55:20.820 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8146000000000005e-05 2024-05-08T08:55:20.820 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:20.820 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:20.820 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:20.820 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.820 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.820 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5600 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 3948544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:20.820 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.884619+0000", 2024-05-08T08:55:20.820 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 54.887119640999998, 2024-05-08T08:55:20.821 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047574067999999997, 2024-05-08T08:55:20.821 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:20.821 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:20.821 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:20.821 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:20.821 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:20.821 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:20.821 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5600 2024-05-08T08:55:20.821 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.821 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:20.821 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.821 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:20.821 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884619+0000", 2024-05-08T08:55:20.821 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:20.821 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.822 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.822 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:20.822 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884619+0000", 2024-05-08T08:55:20.822 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:20.822 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.822 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.822 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:20.822 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884620+0000", 2024-05-08T08:55:20.822 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.53e-07 2024-05-08T08:55:20.822 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.822 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.822 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:20.822 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884622+0000", 2024-05-08T08:55:20.823 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.796e-06 2024-05-08T08:55:20.823 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.823 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.823 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:20.823 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884627+0000", 2024-05-08T08:55:20.823 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2850000000000002e-06 2024-05-08T08:55:20.823 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.823 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.823 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:20.823 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884631+0000", 2024-05-08T08:55:20.823 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2899999999999996e-06 2024-05-08T08:55:20.823 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.823 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.823 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:20.824 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932024+0000", 2024-05-08T08:55:20.824 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047392680999999999 2024-05-08T08:55:20.824 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.824 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.824 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:20.824 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932102+0000", 2024-05-08T08:55:20.824 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7806999999999996e-05 2024-05-08T08:55:20.824 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.824 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.824 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:20.824 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932193+0000", 2024-05-08T08:55:20.824 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1855999999999996e-05 2024-05-08T08:55:20.824 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:20.824 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:20.824 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:20.824 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.825 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.825 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5603 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 3993600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:20.825 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.884680+0000", 2024-05-08T08:55:20.825 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 54.887059032000003, 2024-05-08T08:55:20.825 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047745034999999998, 2024-05-08T08:55:20.825 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:20.825 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:20.825 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:20.825 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:20.825 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:20.825 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:20.825 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5603 2024-05-08T08:55:20.825 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.825 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:20.826 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.826 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:20.826 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884680+0000", 2024-05-08T08:55:20.826 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:20.826 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.826 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.826 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:20.826 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884680+0000", 2024-05-08T08:55:20.826 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:20.826 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.826 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.826 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:20.826 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884680+0000", 2024-05-08T08:55:20.826 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4400000000000001e-07 2024-05-08T08:55:20.827 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.827 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.827 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:20.827 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884682+0000", 2024-05-08T08:55:20.827 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.514e-06 2024-05-08T08:55:20.827 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.827 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.827 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:20.827 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884687+0000", 2024-05-08T08:55:20.827 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0359999999999997e-06 2024-05-08T08:55:20.827 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.827 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.827 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:20.827 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884692+0000", 2024-05-08T08:55:20.827 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5990000000000004e-06 2024-05-08T08:55:20.828 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.828 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.828 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:20.828 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932235+0000", 2024-05-08T08:55:20.828 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047543635000000001 2024-05-08T08:55:20.828 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.828 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.828 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:20.828 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932323+0000", 2024-05-08T08:55:20.828 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7311000000000005e-05 2024-05-08T08:55:20.828 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.828 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.828 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:20.829 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932425+0000", 2024-05-08T08:55:20.829 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102496 2024-05-08T08:55:20.829 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:20.829 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:20.829 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:20.829 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.829 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.829 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5606 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4038656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:20.829 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.884729+0000", 2024-05-08T08:55:20.829 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 54.887009743999997, 2024-05-08T08:55:20.829 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047916488, 2024-05-08T08:55:20.829 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:20.829 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:20.829 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:20.830 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:20.830 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:20.830 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:20.830 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5606 2024-05-08T08:55:20.830 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.830 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:20.830 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.830 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:20.830 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884729+0000", 2024-05-08T08:55:20.830 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:20.830 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.830 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.830 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:20.830 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884729+0000", 2024-05-08T08:55:20.830 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:20.831 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.831 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.831 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:20.831 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.885940+0000", 2024-05-08T08:55:20.831 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001210509 2024-05-08T08:55:20.831 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.831 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.831 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:20.831 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.885945+0000", 2024-05-08T08:55:20.831 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7389999999999999e-06 2024-05-08T08:55:20.831 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.831 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.831 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:20.831 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.885955+0000", 2024-05-08T08:55:20.831 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0813e-05 2024-05-08T08:55:20.831 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.832 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.832 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:20.832 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.885962+0000", 2024-05-08T08:55:20.832 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1160000000000001e-06 2024-05-08T08:55:20.832 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.832 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.832 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:20.832 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932465+0000", 2024-05-08T08:55:20.832 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046502612999999998 2024-05-08T08:55:20.832 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.832 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.832 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:20.832 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932547+0000", 2024-05-08T08:55:20.832 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2025000000000002e-05 2024-05-08T08:55:20.832 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.833 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.833 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:20.833 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932646+0000", 2024-05-08T08:55:20.833 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8672999999999995e-05 2024-05-08T08:55:20.833 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:20.833 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:20.833 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:20.833 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.833 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.833 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5636 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4280320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:20.833 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.886111+0000", 2024-05-08T08:55:20.833 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 54.885628007000001, 2024-05-08T08:55:20.833 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047663644999999998, 2024-05-08T08:55:20.833 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:20.833 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:20.833 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:20.833 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:20.834 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:20.834 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:20.834 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5636 2024-05-08T08:55:20.834 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.834 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:20.834 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.834 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:20.834 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886111+0000", 2024-05-08T08:55:20.834 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:20.834 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.834 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.834 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:20.834 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886111+0000", 2024-05-08T08:55:20.834 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:20.835 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.835 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.835 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:20.835 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886111+0000", 2024-05-08T08:55:20.835 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5200000000000002e-07 2024-05-08T08:55:20.835 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.835 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.835 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:20.835 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886114+0000", 2024-05-08T08:55:20.835 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1710000000000001e-06 2024-05-08T08:55:20.835 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.835 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.835 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:20.835 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886120+0000", 2024-05-08T08:55:20.835 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.404e-06 2024-05-08T08:55:20.836 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.836 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.836 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:20.836 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886125+0000", 2024-05-08T08:55:20.836 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2730000000000002e-06 2024-05-08T08:55:20.836 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.836 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.836 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:20.836 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933570+0000", 2024-05-08T08:55:20.836 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047444989999999999 2024-05-08T08:55:20.836 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.836 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.836 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:20.837 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933659+0000", 2024-05-08T08:55:20.837 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8912999999999994e-05 2024-05-08T08:55:20.837 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.837 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.837 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:20.837 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933775+0000", 2024-05-08T08:55:20.837 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000115442 2024-05-08T08:55:20.837 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:20.837 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:20.837 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:20.837 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.837 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.837 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5637 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4325376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:20.837 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.886132+0000", 2024-05-08T08:55:20.837 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 54.885606969000001, 2024-05-08T08:55:20.837 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047852219000000001, 2024-05-08T08:55:20.838 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:20.838 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:20.838 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:20.838 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:20.838 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:20.838 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:20.838 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5637 2024-05-08T08:55:20.838 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.838 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:20.838 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.838 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:20.838 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886132+0000", 2024-05-08T08:55:20.838 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:20.838 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.839 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.839 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:20.839 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886132+0000", 2024-05-08T08:55:20.839 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:20.839 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.839 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.839 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:20.839 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886132+0000", 2024-05-08T08:55:20.839 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7300000000000002e-07 2024-05-08T08:55:20.839 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.839 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.839 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:20.839 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886139+0000", 2024-05-08T08:55:20.839 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6109999999999999e-06 2024-05-08T08:55:20.840 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.840 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.840 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:20.840 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886146+0000", 2024-05-08T08:55:20.840 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6279999999999999e-06 2024-05-08T08:55:20.840 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.840 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.840 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:20.840 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886151+0000", 2024-05-08T08:55:20.840 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1769999999999997e-06 2024-05-08T08:55:20.840 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.840 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.840 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:20.841 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933812+0000", 2024-05-08T08:55:20.841 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047660912999999999 2024-05-08T08:55:20.841 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.841 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.841 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:20.841 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933888+0000", 2024-05-08T08:55:20.841 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5865000000000006e-05 2024-05-08T08:55:20.841 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.841 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.841 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:20.841 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933984+0000", 2024-05-08T08:55:20.841 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6651999999999996e-05 2024-05-08T08:55:20.841 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:20.841 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:20.841 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:20.841 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.842 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.842 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5638 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4370432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:20.842 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.886157+0000", 2024-05-08T08:55:20.842 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 54.885581705, 2024-05-08T08:55:20.842 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048077597, 2024-05-08T08:55:20.842 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:20.842 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:20.842 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:20.842 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:20.842 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:20.842 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:20.842 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5638 2024-05-08T08:55:20.842 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.843 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:20.843 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.843 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:20.843 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886157+0000", 2024-05-08T08:55:20.843 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:20.843 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.843 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.843 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:20.843 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886157+0000", 2024-05-08T08:55:20.843 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:20.843 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.843 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.843 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:20.844 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886158+0000", 2024-05-08T08:55:20.844 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1100000000000001e-07 2024-05-08T08:55:20.844 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.844 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.844 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:20.844 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886160+0000", 2024-05-08T08:55:20.844 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9800000000000001e-06 2024-05-08T08:55:20.844 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.844 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.844 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:20.844 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886166+0000", 2024-05-08T08:55:20.844 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.314e-06 2024-05-08T08:55:20.844 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.844 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.845 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:20.845 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886171+0000", 2024-05-08T08:55:20.845 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7700000000000001e-06 2024-05-08T08:55:20.845 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.845 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.845 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:20.845 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.934025+0000", 2024-05-08T08:55:20.845 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047854509000000003 2024-05-08T08:55:20.845 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.845 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.845 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:20.845 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.934121+0000", 2024-05-08T08:55:20.845 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5688999999999998e-05 2024-05-08T08:55:20.845 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.845 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.846 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:20.846 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.934235+0000", 2024-05-08T08:55:20.846 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000113924 2024-05-08T08:55:20.846 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:20.846 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:20.846 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:20.846 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.846 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.846 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5639 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4415488~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:20.846 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.886177+0000", 2024-05-08T08:55:20.846 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 54.885561955, 2024-05-08T08:55:20.846 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048249541, 2024-05-08T08:55:20.846 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:20.847 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:20.847 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:20.847 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:20.847 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:20.847 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:20.847 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5639 2024-05-08T08:55:20.847 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.847 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:20.847 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.847 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:20.847 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886177+0000", 2024-05-08T08:55:20.847 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:20.847 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.847 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.847 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:20.848 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886177+0000", 2024-05-08T08:55:20.848 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:20.848 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.848 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.848 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:20.848 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886177+0000", 2024-05-08T08:55:20.848 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.65e-07 2024-05-08T08:55:20.848 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.848 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.848 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:20.848 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886179+0000", 2024-05-08T08:55:20.848 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9530000000000002e-06 2024-05-08T08:55:20.848 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.848 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.849 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:20.849 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886186+0000", 2024-05-08T08:55:20.849 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3419999999999996e-06 2024-05-08T08:55:20.849 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.849 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.849 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:20.849 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886192+0000", 2024-05-08T08:55:20.849 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0560000000000001e-06 2024-05-08T08:55:20.849 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.849 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.849 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:20.849 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.934258+0000", 2024-05-08T08:55:20.849 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048066367999999998 2024-05-08T08:55:20.849 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.849 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.850 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:20.850 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.934338+0000", 2024-05-08T08:55:20.850 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9616000000000005e-05 2024-05-08T08:55:20.850 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.850 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.850 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:20.850 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.934427+0000", 2024-05-08T08:55:20.850 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8840999999999999e-05 2024-05-08T08:55:20.850 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:20.850 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:20.850 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:20.850 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.850 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.850 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4416.0:34114 3.12 3:4e31a7e4:::rbd_data.1140100c9703.0000000000000008:head [delete] snapc 14=[] ondisk+write+known_if_redirected+supports_pool_eio e164)", 2024-05-08T08:55:20.851 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:55:15.450741+0000", 2024-05-08T08:55:20.851 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.32099821, 2024-05-08T08:55:20.851 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.92638831499999996, 2024-05-08T08:55:20.851 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:20.851 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:20.851 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:55:20.851 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:20.851 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4416", 2024-05-08T08:55:20.851 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:55:20.851 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 34114 2024-05-08T08:55:20.851 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.851 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:20.851 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.851 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:20.851 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:15.450741+0000", 2024-05-08T08:55:20.852 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:20.852 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.852 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.852 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:20.852 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:15.450741+0000", 2024-05-08T08:55:20.852 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:20.852 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.852 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.852 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:20.852 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:15.450741+0000", 2024-05-08T08:55:20.852 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9900000000000001e-07 2024-05-08T08:55:20.852 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.852 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.852 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:20.852 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:15.450744+0000", 2024-05-08T08:55:20.853 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3259999999999998e-06 2024-05-08T08:55:20.853 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.853 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.853 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:20.853 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:15.450751+0000", 2024-05-08T08:55:20.853 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6129999999999998e-06 2024-05-08T08:55:20.853 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.853 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.853 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:20.853 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:15.450757+0000", 2024-05-08T08:55:20.853 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3129999999999998e-06 2024-05-08T08:55:20.853 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.853 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.853 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:20.853 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:15.450825+0000", 2024-05-08T08:55:20.854 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8307999999999997e-05 2024-05-08T08:55:20.854 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.854 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.854 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-05-08T08:55:20.854 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:15.450840+0000", 2024-05-08T08:55:20.854 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4968e-05 2024-05-08T08:55:20.854 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.854 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.854 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:20.854 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:16.374193+0000", 2024-05-08T08:55:20.854 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.92335363000000004 2024-05-08T08:55:20.854 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.854 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.854 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:20.855 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:16.374298+0000", 2024-05-08T08:55:20.855 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104759 2024-05-08T08:55:20.855 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.855 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.855 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-05-08T08:55:20.855 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:16.374372+0000", 2024-05-08T08:55:20.855 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4232000000000003e-05 2024-05-08T08:55:20.855 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.855 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.855 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:55:20.855 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:16.376496+0000", 2024-05-08T08:55:20.855 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0021231259999999999 2024-05-08T08:55:20.855 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.855 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.855 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:55:20.856 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:16.377065+0000", 2024-05-08T08:55:20.856 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00056950700000000002 2024-05-08T08:55:20.856 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.856 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.856 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:55:20.856 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:16.377100+0000", 2024-05-08T08:55:20.856 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4579999999999998e-05 2024-05-08T08:55:20.856 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.856 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.856 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:20.856 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:16.377129+0000", 2024-05-08T08:55:20.856 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9454e-05 2024-05-08T08:55:20.856 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:20.857 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:20.857 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:20.857 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.857 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.857 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4416.0:34124 3.12 3:4f2dce52:::rbd_data.1140100c9703.0000000000000015:head [delete] snapc 14=[] ondisk+write+known_if_redirected+supports_pool_eio e164)", 2024-05-08T08:55:20.857 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:55:15.458266+0000", 2024-05-08T08:55:20.857 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.3134728579999999, 2024-05-08T08:55:20.857 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.925342255, 2024-05-08T08:55:20.857 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:20.857 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:20.857 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:55:20.857 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:20.857 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4416", 2024-05-08T08:55:20.857 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:55:20.857 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 34124 2024-05-08T08:55:20.858 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.858 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:20.858 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.858 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:20.858 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:15.458266+0000", 2024-05-08T08:55:20.858 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:20.858 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.858 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.858 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:20.858 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:15.458266+0000", 2024-05-08T08:55:20.858 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:20.858 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.858 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.858 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:20.858 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:15.458267+0000", 2024-05-08T08:55:20.859 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9900000000000001e-07 2024-05-08T08:55:20.859 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.859 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.859 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:20.859 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:15.458269+0000", 2024-05-08T08:55:20.859 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6520000000000002e-06 2024-05-08T08:55:20.859 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.859 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.859 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:20.859 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:15.458276+0000", 2024-05-08T08:55:20.859 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1779999999999997e-06 2024-05-08T08:55:20.859 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.859 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.859 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:20.860 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:15.458283+0000", 2024-05-08T08:55:20.860 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7680000000000003e-06 2024-05-08T08:55:20.860 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.860 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.860 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:20.860 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:15.458330+0000", 2024-05-08T08:55:20.860 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7111999999999999e-05 2024-05-08T08:55:20.860 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.860 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.860 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-05-08T08:55:20.860 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:15.458340+0000", 2024-05-08T08:55:20.860 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5680000000000005e-06 2024-05-08T08:55:20.860 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.860 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.861 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:20.861 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:16.375925+0000", 2024-05-08T08:55:20.861 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.91758470599999997 2024-05-08T08:55:20.861 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.861 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.861 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:20.861 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:16.376000+0000", 2024-05-08T08:55:20.861 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5325000000000001e-05 2024-05-08T08:55:20.861 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.861 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.861 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-05-08T08:55:20.861 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:16.376062+0000", 2024-05-08T08:55:20.861 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1630000000000005e-05 2024-05-08T08:55:20.861 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.861 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.862 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:55:20.862 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:16.380141+0000", 2024-05-08T08:55:20.862 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0040797530000000002 2024-05-08T08:55:20.862 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.862 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.862 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:55:20.862 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:16.383551+0000", 2024-05-08T08:55:20.862 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0034098319999999998 2024-05-08T08:55:20.862 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.862 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.862 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:55:20.862 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:16.383583+0000", 2024-05-08T08:55:20.862 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2140000000000001e-05 2024-05-08T08:55:20.862 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.862 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.863 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:20.863 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:16.383608+0000", 2024-05-08T08:55:20.863 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5092e-05 2024-05-08T08:55:20.863 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:20.863 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:20.863 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:20.863 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.863 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.863 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4416.0:34130 3.12 3:4fed2d06:::rbd_data.1140100c9703.0000000000000031:head [delete] snapc 14=[] ondisk+write+known_if_redirected+supports_pool_eio e164)", 2024-05-08T08:55:20.863 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:55:15.460999+0000", 2024-05-08T08:55:20.863 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.310739796, 2024-05-08T08:55:20.863 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.92270473099999994, 2024-05-08T08:55:20.863 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:20.863 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:20.863 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:55:20.864 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:20.864 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4416", 2024-05-08T08:55:20.864 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:55:20.864 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 34130 2024-05-08T08:55:20.864 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.864 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:20.864 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.864 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:20.864 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:15.460999+0000", 2024-05-08T08:55:20.864 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:20.864 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.864 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.864 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:20.864 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:15.460999+0000", 2024-05-08T08:55:20.864 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:20.865 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.865 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.865 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:20.865 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:15.461000+0000", 2024-05-08T08:55:20.865 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1999999999999999e-07 2024-05-08T08:55:20.865 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.865 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.865 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:20.865 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:15.461004+0000", 2024-05-08T08:55:20.865 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1620000000000001e-06 2024-05-08T08:55:20.865 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.865 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.865 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:20.865 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:15.461016+0000", 2024-05-08T08:55:20.866 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2269999999999999e-05 2024-05-08T08:55:20.866 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.866 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.866 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:20.866 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:15.461024+0000", 2024-05-08T08:55:20.866 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6450000000000005e-06 2024-05-08T08:55:20.866 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.866 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.866 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:20.866 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:15.461077+0000", 2024-05-08T08:55:20.866 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3242000000000003e-05 2024-05-08T08:55:20.866 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.866 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.866 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-05-08T08:55:20.867 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:15.461088+0000", 2024-05-08T08:55:20.867 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0808999999999999e-05 2024-05-08T08:55:20.867 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.867 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.867 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:20.867 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:16.376527+0000", 2024-05-08T08:55:20.867 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.91543937200000003 2024-05-08T08:55:20.867 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.867 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.867 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:20.867 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:16.376599+0000", 2024-05-08T08:55:20.867 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1298999999999996e-05 2024-05-08T08:55:20.867 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.867 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.867 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-05-08T08:55:20.868 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:16.376657+0000", 2024-05-08T08:55:20.868 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8418000000000002e-05 2024-05-08T08:55:20.868 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.868 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.868 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:55:20.868 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:16.380144+0000", 2024-05-08T08:55:20.868 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003487066 2024-05-08T08:55:20.868 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.868 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.868 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:55:20.868 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:16.383639+0000", 2024-05-08T08:55:20.868 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0034950570000000002 2024-05-08T08:55:20.868 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.868 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.868 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:55:20.869 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:16.383670+0000", 2024-05-08T08:55:20.869 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1065999999999997e-05 2024-05-08T08:55:20.869 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:20.869 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:20.869 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:20.869 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:16.383704+0000", 2024-05-08T08:55:20.869 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3704999999999997e-05 2024-05-08T08:55:20.869 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:20.869 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:20.869 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:20.869 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:20.869 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:20.869 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:55:20.871 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:55:20.872 INFO:teuthology.orchestra.run.smithi059.stdout: "success": "" 2024-05-08T08:55:20.872 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:55:20.872 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:55:20.872 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:55:20.874+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:55:20.971 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:55:20.974+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:55:21.018 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:55:21.019 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:55:21.019 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:55:21.019 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:55:21.027 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:55:21.072 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:55:21.075+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:55:21.132 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:55:21.132 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:55:21.132 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:55:21.132 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:55:21.133 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:55:21.141 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:55:21.173 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:55:21.175+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:55:21.247 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:55:21.247 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:55:21.247 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:55:21.247 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:55:21.248 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.248 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22073 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 782336~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:21.248 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:55:21.248 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 173.391545769, 2024-05-08T08:55:21.248 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047244234000000003, 2024-05-08T08:55:21.248 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:21.248 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:21.248 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:21.248 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:21.248 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:21.248 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:21.248 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22073 2024-05-08T08:55:21.248 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.249 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:21.249 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.249 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:21.249 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:55:21.249 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:21.249 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.249 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.249 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:21.249 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:55:21.249 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:21.249 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.249 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.249 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:21.249 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857668+0000", 2024-05-08T08:55:21.249 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7800000000000002e-07 2024-05-08T08:55:21.249 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.250 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.250 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:21.250 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857670+0000", 2024-05-08T08:55:21.250 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0820000000000001e-06 2024-05-08T08:55:21.250 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.250 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.250 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:21.250 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857678+0000", 2024-05-08T08:55:21.250 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7589999999999997e-06 2024-05-08T08:55:21.250 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.250 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.250 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:21.250 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857691+0000", 2024-05-08T08:55:21.251 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3672999999999999e-05 2024-05-08T08:55:21.251 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.251 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.251 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:21.251 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904780+0000", 2024-05-08T08:55:21.251 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047088616999999999 2024-05-08T08:55:21.251 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.251 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.251 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:21.251 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904840+0000", 2024-05-08T08:55:21.251 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0356999999999998e-05 2024-05-08T08:55:21.251 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.251 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.251 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:21.251 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904912+0000", 2024-05-08T08:55:21.252 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1367999999999998e-05 2024-05-08T08:55:21.252 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.252 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:21.252 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.252 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.252 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.252 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22075 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 798720~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:21.252 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:55:21.252 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 173.39151537199999, 2024-05-08T08:55:21.252 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047384643999999997, 2024-05-08T08:55:21.252 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:21.252 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:21.252 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:21.252 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:21.253 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:21.253 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:21.253 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22075 2024-05-08T08:55:21.253 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.253 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:21.253 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.253 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:21.253 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:55:21.253 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:21.253 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.253 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.253 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:21.253 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:55:21.253 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:21.253 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.254 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.254 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:21.254 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:55:21.254 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7399999999999999e-07 2024-05-08T08:55:21.254 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.254 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.254 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:21.254 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857705+0000", 2024-05-08T08:55:21.254 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1980000000000004e-06 2024-05-08T08:55:21.254 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.254 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.254 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:21.254 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857713+0000", 2024-05-08T08:55:21.254 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7039999999999994e-06 2024-05-08T08:55:21.254 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.255 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.255 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:21.255 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857718+0000", 2024-05-08T08:55:21.255 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.865e-06 2024-05-08T08:55:21.255 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.255 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.255 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:21.255 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904941+0000", 2024-05-08T08:55:21.255 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047223198000000001 2024-05-08T08:55:21.255 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.255 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.255 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:21.255 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904999+0000", 2024-05-08T08:55:21.255 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8254999999999997e-05 2024-05-08T08:55:21.255 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.255 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.256 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:21.256 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905082+0000", 2024-05-08T08:55:21.256 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3049999999999999e-05 2024-05-08T08:55:21.256 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.256 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:21.256 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.256 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.256 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.256 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22077 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 815104~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:21.256 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:55:21.256 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 173.391477361, 2024-05-08T08:55:21.256 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047519448999999998, 2024-05-08T08:55:21.256 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:21.256 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:21.256 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:21.257 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:21.257 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:21.257 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:21.257 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22077 2024-05-08T08:55:21.257 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.257 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:21.257 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.257 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:21.257 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:55:21.257 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:21.257 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.257 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.257 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:21.257 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:55:21.257 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:21.258 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.258 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.258 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:21.258 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:55:21.258 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.99e-07 2024-05-08T08:55:21.258 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.258 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.258 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:21.258 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857739+0000", 2024-05-08T08:55:21.258 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3599999999999999e-06 2024-05-08T08:55:21.258 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.258 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.258 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:21.258 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857745+0000", 2024-05-08T08:55:21.258 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5139999999999998e-06 2024-05-08T08:55:21.259 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.259 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.259 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:21.259 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857750+0000", 2024-05-08T08:55:21.259 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8550000000000001e-06 2024-05-08T08:55:21.259 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.259 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.259 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:21.259 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905110+0000", 2024-05-08T08:55:21.259 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047360126000000002 2024-05-08T08:55:21.259 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.259 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.259 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:21.259 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905175+0000", 2024-05-08T08:55:21.260 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4689000000000004e-05 2024-05-08T08:55:21.260 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.260 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.260 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:21.260 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905255+0000", 2024-05-08T08:55:21.260 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0306e-05 2024-05-08T08:55:21.260 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.260 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:21.260 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.260 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.260 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.260 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22079 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 831488~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:21.260 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:55:21.260 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 173.39145681299999, 2024-05-08T08:55:21.260 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.04767499, 2024-05-08T08:55:21.261 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:21.261 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:21.261 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:21.261 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:21.261 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:21.261 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:21.261 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22079 2024-05-08T08:55:21.261 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.261 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:21.261 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.261 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:21.261 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:55:21.261 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:21.261 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.261 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.262 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:21.262 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:55:21.262 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:21.262 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.262 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.262 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:21.262 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857757+0000", 2024-05-08T08:55:21.262 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8299999999999997e-07 2024-05-08T08:55:21.262 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.262 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.262 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:21.262 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857759+0000", 2024-05-08T08:55:21.262 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1390000000000002e-06 2024-05-08T08:55:21.263 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.263 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.263 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:21.263 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857765+0000", 2024-05-08T08:55:21.263 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4609999999999996e-06 2024-05-08T08:55:21.263 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.263 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.263 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:21.263 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857770+0000", 2024-05-08T08:55:21.263 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6689999999999997e-06 2024-05-08T08:55:21.263 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.263 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.263 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:21.263 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905281+0000", 2024-05-08T08:55:21.263 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047510668999999998 2024-05-08T08:55:21.263 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.264 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.264 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:21.264 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905344+0000", 2024-05-08T08:55:21.264 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2797000000000002e-05 2024-05-08T08:55:21.264 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.264 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.264 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:21.264 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905431+0000", 2024-05-08T08:55:21.264 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7771999999999999e-05 2024-05-08T08:55:21.264 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.264 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:21.264 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.264 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.264 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.265 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22082 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 847872~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:21.265 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:55:21.265 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 173.39143712200001, 2024-05-08T08:55:21.265 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047831934999999999, 2024-05-08T08:55:21.265 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:21.265 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:21.265 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:21.265 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:21.265 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:21.265 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:21.265 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22082 2024-05-08T08:55:21.265 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.265 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:21.265 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.266 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:21.266 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:55:21.266 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:21.266 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.266 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.266 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:21.266 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:55:21.266 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:21.266 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.266 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.266 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:21.266 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:55:21.266 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:55:21.266 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.266 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.267 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:21.267 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857778+0000", 2024-05-08T08:55:21.267 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0789999999999999e-06 2024-05-08T08:55:21.267 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.267 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.267 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:21.267 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857784+0000", 2024-05-08T08:55:21.267 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0480000000000004e-06 2024-05-08T08:55:21.267 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.267 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.267 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:21.267 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857790+0000", 2024-05-08T08:55:21.267 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2719999999999997e-06 2024-05-08T08:55:21.267 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.267 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.268 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:21.268 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905458+0000", 2024-05-08T08:55:21.268 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047668470999999997 2024-05-08T08:55:21.268 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.268 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.268 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:21.268 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905521+0000", 2024-05-08T08:55:21.268 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2952000000000003e-05 2024-05-08T08:55:21.268 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.268 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.268 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:21.268 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905608+0000", 2024-05-08T08:55:21.268 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6759000000000001e-05 2024-05-08T08:55:21.268 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.268 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:21.269 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.269 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.269 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.269 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22084 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 864256~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:21.269 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:55:21.269 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 173.391413172, 2024-05-08T08:55:21.269 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047977748000000001, 2024-05-08T08:55:21.269 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:21.269 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:21.269 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:21.269 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:21.269 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:21.269 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:21.269 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22084 2024-05-08T08:55:21.269 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.270 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:21.270 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.270 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:21.270 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:55:21.270 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:21.270 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.270 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.270 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:21.270 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:55:21.270 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:21.270 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.270 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.270 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:21.270 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:55:21.270 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:55:21.271 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.271 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.271 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:21.271 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857802+0000", 2024-05-08T08:55:21.271 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0779999999999998e-06 2024-05-08T08:55:21.271 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.271 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.271 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:21.271 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857812+0000", 2024-05-08T08:55:21.271 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7939999999999997e-06 2024-05-08T08:55:21.271 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.271 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.271 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:21.271 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857817+0000", 2024-05-08T08:55:21.271 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8899999999999998e-06 2024-05-08T08:55:21.272 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.272 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.272 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:21.272 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905636+0000", 2024-05-08T08:55:21.272 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047819252 2024-05-08T08:55:21.272 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.272 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.272 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:21.272 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905700+0000", 2024-05-08T08:55:21.273 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4016999999999994e-05 2024-05-08T08:55:21.273 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.273 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.273 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:21.273 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905778+0000", 2024-05-08T08:55:21.273 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7306999999999997e-05 2024-05-08T08:55:21.273 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.273 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:21.273 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.273 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.273 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.273 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22086 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 880640~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:21.273 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:55:21.273 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 173.39138982200001, 2024-05-08T08:55:21.274 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048119149, 2024-05-08T08:55:21.274 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:21.274 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:21.274 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:21.274 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:21.274 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:21.274 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:21.274 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22086 2024-05-08T08:55:21.274 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.274 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:21.274 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.274 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:21.274 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:55:21.274 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:21.275 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.275 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.275 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:21.275 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:55:21.275 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:21.275 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.275 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.275 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:21.275 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857824+0000", 2024-05-08T08:55:21.275 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4200000000000002e-07 2024-05-08T08:55:21.275 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.275 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.275 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:21.275 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857826+0000", 2024-05-08T08:55:21.275 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0590000000000001e-06 2024-05-08T08:55:21.276 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.276 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.276 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:21.276 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857832+0000", 2024-05-08T08:55:21.276 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8029999999999999e-06 2024-05-08T08:55:21.276 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.276 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.276 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:21.276 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857837+0000", 2024-05-08T08:55:21.276 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7929999999999997e-06 2024-05-08T08:55:21.276 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.276 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.276 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:21.276 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905803+0000", 2024-05-08T08:55:21.276 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047965761000000003 2024-05-08T08:55:21.277 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.277 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.277 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:21.277 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905862+0000", 2024-05-08T08:55:21.277 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9203000000000002e-05 2024-05-08T08:55:21.277 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.277 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.277 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:21.277 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905942+0000", 2024-05-08T08:55:21.277 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0187999999999999e-05 2024-05-08T08:55:21.277 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.277 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:21.277 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.277 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.277 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.278 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22088 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 897024~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:21.278 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:55:21.278 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 173.39136977000001, 2024-05-08T08:55:21.278 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048256049000000002, 2024-05-08T08:55:21.278 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:21.278 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:21.278 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:21.278 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:21.278 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:21.278 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:21.278 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22088 2024-05-08T08:55:21.278 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.278 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:21.278 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.279 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:21.279 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:55:21.279 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:21.279 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.279 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.279 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:21.279 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:55:21.279 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:21.279 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.279 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.279 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:21.279 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857844+0000", 2024-05-08T08:55:21.279 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.46e-07 2024-05-08T08:55:21.279 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.279 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.280 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:21.280 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857846+0000", 2024-05-08T08:55:21.280 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0899999999999999e-06 2024-05-08T08:55:21.280 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.280 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.280 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:21.280 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857855+0000", 2024-05-08T08:55:21.280 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8219999999999996e-06 2024-05-08T08:55:21.280 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.280 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.280 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:21.280 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857860+0000", 2024-05-08T08:55:21.280 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2989999999999996e-06 2024-05-08T08:55:21.280 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.281 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.281 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:21.281 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905966+0000", 2024-05-08T08:55:21.281 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048106542000000002 2024-05-08T08:55:21.281 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.281 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.281 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:21.281 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906024+0000", 2024-05-08T08:55:21.281 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7958000000000003e-05 2024-05-08T08:55:21.281 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.281 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.281 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:21.281 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906099+0000", 2024-05-08T08:55:21.281 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4992000000000003e-05 2024-05-08T08:55:21.281 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.282 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:21.282 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.282 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.282 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.282 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22091 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 913408~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:21.282 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:55:21.282 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 173.39134601000001, 2024-05-08T08:55:21.282 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048395290000000001, 2024-05-08T08:55:21.282 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:21.282 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:21.282 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:21.282 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:21.282 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:21.282 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:21.282 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22091 2024-05-08T08:55:21.283 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.283 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:21.283 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.283 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:21.283 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:55:21.283 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:21.283 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.283 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.283 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:21.283 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:55:21.283 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:21.283 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.283 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.283 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:21.283 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:55:21.284 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4900000000000001e-07 2024-05-08T08:55:21.284 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.284 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.284 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:21.284 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857870+0000", 2024-05-08T08:55:21.284 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2510000000000001e-06 2024-05-08T08:55:21.284 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.284 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.284 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:21.284 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857877+0000", 2024-05-08T08:55:21.284 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0809999999999996e-06 2024-05-08T08:55:21.284 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.284 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.284 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:21.284 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857881+0000", 2024-05-08T08:55:21.285 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.724e-06 2024-05-08T08:55:21.285 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.285 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.285 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:21.285 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906127+0000", 2024-05-08T08:55:21.285 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048245345000000002 2024-05-08T08:55:21.285 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.285 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.285 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:21.285 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906190+0000", 2024-05-08T08:55:21.285 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2762000000000007e-05 2024-05-08T08:55:21.285 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.285 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.285 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:21.285 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906262+0000", 2024-05-08T08:55:21.286 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2778e-05 2024-05-08T08:55:21.286 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.286 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:21.286 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.286 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.286 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.286 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22093 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 929792~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:21.286 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:55:21.286 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 173.39132364599999, 2024-05-08T08:55:21.286 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048532875000000003, 2024-05-08T08:55:21.286 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:21.286 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:21.286 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:21.286 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:21.287 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:21.287 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:21.287 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22093 2024-05-08T08:55:21.287 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.287 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:21.287 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.287 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:21.287 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:55:21.287 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:21.287 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.287 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.287 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:21.287 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:55:21.287 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:21.288 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.288 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.288 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:21.288 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857890+0000", 2024-05-08T08:55:21.288 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-07 2024-05-08T08:55:21.288 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.288 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.288 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:21.288 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857899+0000", 2024-05-08T08:55:21.288 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6889999999999999e-06 2024-05-08T08:55:21.288 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.288 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.288 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:21.288 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857906+0000", 2024-05-08T08:55:21.288 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6490000000000002e-06 2024-05-08T08:55:21.289 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.289 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.289 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:21.289 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857917+0000", 2024-05-08T08:55:21.289 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0764999999999999e-05 2024-05-08T08:55:21.289 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.289 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.289 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:21.289 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906287+0000", 2024-05-08T08:55:21.289 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048370442 2024-05-08T08:55:21.289 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.289 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.289 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:21.289 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906349+0000", 2024-05-08T08:55:21.289 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1537000000000004e-05 2024-05-08T08:55:21.290 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.290 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.290 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:21.290 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906422+0000", 2024-05-08T08:55:21.290 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3431999999999998e-05 2024-05-08T08:55:21.290 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.290 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:21.290 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.290 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.290 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.290 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22095 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 946176~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:21.290 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:55:21.290 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 173.39128894199999, 2024-05-08T08:55:21.291 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048657593999999998, 2024-05-08T08:55:21.291 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:21.291 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:21.291 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:21.291 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:21.291 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:21.291 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:21.291 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22095 2024-05-08T08:55:21.291 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.291 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:21.291 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.291 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:21.291 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:55:21.291 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:21.291 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.292 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.292 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:21.292 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:55:21.292 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:21.292 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.292 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.292 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:21.292 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857925+0000", 2024-05-08T08:55:21.292 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3099999999999998e-07 2024-05-08T08:55:21.292 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.292 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.292 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:21.292 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857927+0000", 2024-05-08T08:55:21.292 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.34e-06 2024-05-08T08:55:21.292 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.293 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.293 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:21.293 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857938+0000", 2024-05-08T08:55:21.293 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1426e-05 2024-05-08T08:55:21.293 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.293 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.293 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:21.293 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857943+0000", 2024-05-08T08:55:21.293 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5140000000000003e-06 2024-05-08T08:55:21.293 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.293 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.293 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:21.293 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906448+0000", 2024-05-08T08:55:21.293 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048504641000000001 2024-05-08T08:55:21.293 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.294 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.294 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:21.294 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906510+0000", 2024-05-08T08:55:21.294 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2663000000000006e-05 2024-05-08T08:55:21.294 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.294 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.294 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:21.294 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906582+0000", 2024-05-08T08:55:21.294 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1478999999999998e-05 2024-05-08T08:55:21.294 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.294 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:21.294 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.294 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.294 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.294 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22097 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 962560~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:21.295 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:55:21.295 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 173.39126125300001, 2024-05-08T08:55:21.295 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048791353000000003, 2024-05-08T08:55:21.295 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:21.295 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:21.295 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:21.295 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:21.295 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:21.295 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:21.295 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22097 2024-05-08T08:55:21.295 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.295 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:21.295 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.295 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:21.295 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:55:21.296 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:21.296 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.296 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.296 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:21.296 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:55:21.296 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:21.296 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.296 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.296 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:21.296 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:55:21.296 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9200000000000002e-07 2024-05-08T08:55:21.296 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.296 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.296 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:21.296 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857954+0000", 2024-05-08T08:55:21.296 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1900000000000002e-06 2024-05-08T08:55:21.297 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.297 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.297 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:21.297 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857961+0000", 2024-05-08T08:55:21.297 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4989999999999999e-06 2024-05-08T08:55:21.297 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.297 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.297 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:21.297 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857971+0000", 2024-05-08T08:55:21.297 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0485999999999999e-05 2024-05-08T08:55:21.297 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.297 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.297 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:21.297 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906607+0000", 2024-05-08T08:55:21.297 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048635109000000003 2024-05-08T08:55:21.298 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.298 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.298 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:21.298 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906667+0000", 2024-05-08T08:55:21.298 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0439000000000002e-05 2024-05-08T08:55:21.298 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.298 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.298 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:21.298 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906743+0000", 2024-05-08T08:55:21.298 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6237999999999998e-05 2024-05-08T08:55:21.298 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.298 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:21.298 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.298 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.299 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.299 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22099 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 978944~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:21.299 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:55:21.299 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 173.39123509300001, 2024-05-08T08:55:21.299 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048932055000000002, 2024-05-08T08:55:21.299 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:21.299 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:21.299 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:21.299 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:21.299 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:21.299 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:21.299 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22099 2024-05-08T08:55:21.299 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.299 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:21.299 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.300 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:21.300 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:55:21.300 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:21.300 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.300 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.300 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:21.300 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:55:21.300 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:21.300 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.300 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.300 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:21.300 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:55:21.300 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.27e-07 2024-05-08T08:55:21.300 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.301 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.301 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:21.301 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857987+0000", 2024-05-08T08:55:21.301 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5890000000000003e-06 2024-05-08T08:55:21.301 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.301 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.301 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:21.301 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857994+0000", 2024-05-08T08:55:21.301 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6619999999999999e-06 2024-05-08T08:55:21.301 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.301 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.301 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:21.301 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857998+0000", 2024-05-08T08:55:21.301 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7110000000000003e-06 2024-05-08T08:55:21.302 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.302 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.302 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:21.302 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906772+0000", 2024-05-08T08:55:21.302 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048773415000000001 2024-05-08T08:55:21.302 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.302 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.302 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:21.302 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906834+0000", 2024-05-08T08:55:21.302 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1752000000000001e-05 2024-05-08T08:55:21.302 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.302 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.302 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:21.302 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906910+0000", 2024-05-08T08:55:21.303 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6499000000000001e-05 2024-05-08T08:55:21.303 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.303 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:21.303 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.303 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.303 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.303 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22101 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 995328~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:21.303 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:55:21.303 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 173.39120796200001, 2024-05-08T08:55:21.303 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049087364000000001, 2024-05-08T08:55:21.303 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:21.303 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:21.303 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:21.303 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:21.303 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:21.304 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:21.304 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22101 2024-05-08T08:55:21.304 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.304 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:21.304 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.304 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:21.304 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:55:21.304 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:21.304 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.304 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.304 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:21.304 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:55:21.304 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:21.304 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.305 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.305 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:21.305 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858006+0000", 2024-05-08T08:55:21.305 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1399999999999997e-07 2024-05-08T08:55:21.305 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.305 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.305 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:21.305 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858008+0000", 2024-05-08T08:55:21.305 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2060000000000001e-06 2024-05-08T08:55:21.305 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.305 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.305 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:21.305 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858017+0000", 2024-05-08T08:55:21.305 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1579999999999994e-06 2024-05-08T08:55:21.305 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.306 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.306 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:21.306 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858022+0000", 2024-05-08T08:55:21.306 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4199999999999998e-06 2024-05-08T08:55:21.306 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.306 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.306 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:21.306 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906940+0000", 2024-05-08T08:55:21.306 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048917627999999998 2024-05-08T08:55:21.306 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.306 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.306 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:21.306 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907002+0000", 2024-05-08T08:55:21.306 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2399999999999999e-05 2024-05-08T08:55:21.307 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.307 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.307 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:21.307 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907092+0000", 2024-05-08T08:55:21.307 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0137999999999997e-05 2024-05-08T08:55:21.307 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.307 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:21.307 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.307 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.307 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.307 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22103 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1011712~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:21.307 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:55:21.307 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 173.39118317699999, 2024-05-08T08:55:21.308 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049228542, 2024-05-08T08:55:21.308 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:21.308 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:21.308 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:21.308 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:21.308 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:21.308 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:21.308 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22103 2024-05-08T08:55:21.308 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.308 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:21.308 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.308 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:21.308 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:55:21.308 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:21.308 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.309 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.309 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:21.309 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:55:21.309 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:21.309 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.309 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.309 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:21.309 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:55:21.309 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4299999999999999e-07 2024-05-08T08:55:21.309 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.309 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.309 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:21.309 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858032+0000", 2024-05-08T08:55:21.309 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0590000000000001e-06 2024-05-08T08:55:21.309 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.310 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.310 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:21.310 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858037+0000", 2024-05-08T08:55:21.310 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.977e-06 2024-05-08T08:55:21.310 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.310 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.310 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:21.310 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858042+0000", 2024-05-08T08:55:21.310 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7650000000000001e-06 2024-05-08T08:55:21.310 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.310 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.310 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:21.310 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907118+0000", 2024-05-08T08:55:21.310 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049076147000000001 2024-05-08T08:55:21.310 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.311 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.311 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:21.311 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907180+0000", 2024-05-08T08:55:21.311 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1971999999999996e-05 2024-05-08T08:55:21.311 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.311 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.311 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:21.311 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907258+0000", 2024-05-08T08:55:21.311 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8279e-05 2024-05-08T08:55:21.311 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.311 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:21.311 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.311 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.311 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.312 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22105 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1028096~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:21.312 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:55:21.312 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 173.39116393399999, 2024-05-08T08:55:21.312 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049394609999999999, 2024-05-08T08:55:21.312 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:21.312 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:21.312 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:21.312 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:21.312 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:21.312 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:21.312 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22105 2024-05-08T08:55:21.312 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.312 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:21.312 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.312 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:21.313 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:55:21.313 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:21.313 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.313 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.313 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:21.313 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:55:21.313 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:21.313 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.313 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.313 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:21.313 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858050+0000", 2024-05-08T08:55:21.313 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5999999999999999e-07 2024-05-08T08:55:21.313 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.313 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.313 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:21.314 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858051+0000", 2024-05-08T08:55:21.314 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8870000000000001e-06 2024-05-08T08:55:21.314 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.314 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.314 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:21.314 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858057+0000", 2024-05-08T08:55:21.314 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8019999999999997e-06 2024-05-08T08:55:21.314 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.314 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.314 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:21.314 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858062+0000", 2024-05-08T08:55:21.314 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.561e-06 2024-05-08T08:55:21.314 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.315 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.315 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:21.315 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907289+0000", 2024-05-08T08:55:21.315 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049226957000000002 2024-05-08T08:55:21.315 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.315 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.315 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:21.315 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907356+0000", 2024-05-08T08:55:21.315 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6915000000000006e-05 2024-05-08T08:55:21.315 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.315 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.315 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:21.315 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907444+0000", 2024-05-08T08:55:21.315 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8127999999999994e-05 2024-05-08T08:55:21.315 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.316 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:21.316 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.316 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.316 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.316 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22107 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1044480~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:21.316 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:55:21.316 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 173.39114559399999, 2024-05-08T08:55:21.316 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049799942999999999, 2024-05-08T08:55:21.316 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:21.316 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:21.316 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:21.316 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:21.316 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:21.316 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:21.316 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22107 2024-05-08T08:55:21.317 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.317 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:21.317 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.317 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:21.317 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:55:21.317 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:21.317 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.317 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.317 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:21.317 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:55:21.317 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:21.317 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.317 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.317 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:21.317 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:55:21.318 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4799999999999999e-07 2024-05-08T08:55:21.318 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.318 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.318 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:21.318 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858070+0000", 2024-05-08T08:55:21.318 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.012e-06 2024-05-08T08:55:21.318 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.318 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.318 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:21.318 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858077+0000", 2024-05-08T08:55:21.318 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6290000000000001e-06 2024-05-08T08:55:21.318 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.318 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.318 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:21.319 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858083+0000", 2024-05-08T08:55:21.319 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8880000000000002e-06 2024-05-08T08:55:21.319 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.319 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.319 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:21.319 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907671+0000", 2024-05-08T08:55:21.319 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049587191000000003 2024-05-08T08:55:21.319 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.319 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.319 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:21.319 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907763+0000", 2024-05-08T08:55:21.319 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2231000000000005e-05 2024-05-08T08:55:21.319 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.319 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.319 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:21.320 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907867+0000", 2024-05-08T08:55:21.320 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104644 2024-05-08T08:55:21.320 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.320 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:21.320 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.320 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.320 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.320 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22294 3.1c 3:3aad0389:::rbd_data.11406886b07b.0000000000000080:head [stat,delete] snapc 7=[7] RETRY=6 ondisk+retry+write+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:21.320 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:55:21.320 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 173.390551746, 2024-05-08T08:55:21.320 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051152191, 2024-05-08T08:55:21.320 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:21.320 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:21.320 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:21.320 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:21.321 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:21.321 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:21.321 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22294 2024-05-08T08:55:21.321 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.321 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:21.321 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.321 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:21.321 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:55:21.321 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:21.321 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.321 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.321 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:21.321 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:55:21.321 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:21.321 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.322 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.322 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:21.322 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858662+0000", 2024-05-08T08:55:21.322 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4799999999999998e-07 2024-05-08T08:55:21.322 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.322 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.322 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:21.322 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858664+0000", 2024-05-08T08:55:21.322 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1229999999999998e-06 2024-05-08T08:55:21.322 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.322 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.322 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:21.322 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858670+0000", 2024-05-08T08:55:21.322 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3099999999999997e-06 2024-05-08T08:55:21.323 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.323 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.323 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:21.323 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858676+0000", 2024-05-08T08:55:21.323 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9750000000000004e-06 2024-05-08T08:55:21.323 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.323 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.323 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:21.323 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907901+0000", 2024-05-08T08:55:21.323 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049224520000000001 2024-05-08T08:55:21.323 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.323 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.323 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:21.323 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.908085+0000", 2024-05-08T08:55:21.323 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00018377500000000001 2024-05-08T08:55:21.324 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.324 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.324 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:21.324 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.909814+0000", 2024-05-08T08:55:21.324 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00172894 2024-05-08T08:55:21.324 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.324 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:21.324 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.324 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.324 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.324 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4416.0:13461 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-05-08T08:55:21.324 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:22.487456+0000", 2024-05-08T08:55:21.324 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 118.761756705, 2024-05-08T08:55:21.324 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.99102620799999996, 2024-05-08T08:55:21.324 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:21.325 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:21.325 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:21.325 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:21.325 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4416", 2024-05-08T08:55:21.325 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:55:21.325 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 13461 2024-05-08T08:55:21.325 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.325 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:21.325 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.325 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:21.325 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487456+0000", 2024-05-08T08:55:21.325 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:21.325 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.325 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.325 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:21.325 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487456+0000", 2024-05-08T08:55:21.326 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:21.326 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.326 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.326 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:21.326 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487458+0000", 2024-05-08T08:55:21.326 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5659999999999999e-06 2024-05-08T08:55:21.326 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.326 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.326 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:21.326 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487463+0000", 2024-05-08T08:55:21.326 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6890000000000003e-06 2024-05-08T08:55:21.326 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.326 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.326 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:21.327 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487472+0000", 2024-05-08T08:55:21.327 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7710000000000001e-06 2024-05-08T08:55:21.327 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.327 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.327 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:21.327 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487478+0000", 2024-05-08T08:55:21.327 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1390000000000003e-06 2024-05-08T08:55:21.327 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.327 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.327 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:21.327 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487520+0000", 2024-05-08T08:55:21.327 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2259999999999999e-05 2024-05-08T08:55:21.327 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.327 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.327 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-05-08T08:55:21.328 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487528+0000", 2024-05-08T08:55:21.328 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7479999999999993e-06 2024-05-08T08:55:21.328 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.328 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.328 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:21.328 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:23.478139+0000", 2024-05-08T08:55:21.328 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.99061165200000001 2024-05-08T08:55:21.328 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.328 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.328 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:21.328 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:23.478301+0000", 2024-05-08T08:55:21.328 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000161602 2024-05-08T08:55:21.328 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.328 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.328 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:21.328 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:23.478483+0000", 2024-05-08T08:55:21.329 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000181781 2024-05-08T08:55:21.329 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.329 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:21.329 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.329 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.329 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.329 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5752.0:404 3.1a 3:5f5e8104:::rbd_data.1140bc4a55c7.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 5175296~45056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-05-08T08:55:21.329 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:00.390863+0000", 2024-05-08T08:55:21.329 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 80.858350509999994, 2024-05-08T08:55:21.329 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.055189174000000001, 2024-05-08T08:55:21.329 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:21.329 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:21.329 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:55:21.329 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:21.330 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5752", 2024-05-08T08:55:21.330 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/3534419815", 2024-05-08T08:55:21.330 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 404 2024-05-08T08:55:21.330 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.330 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:21.330 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.330 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:21.330 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.390863+0000", 2024-05-08T08:55:21.330 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:21.330 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.330 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.330 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:21.330 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.390863+0000", 2024-05-08T08:55:21.330 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:21.330 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.331 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.331 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:21.331 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.390864+0000", 2024-05-08T08:55:21.331 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.471e-06 2024-05-08T08:55:21.331 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.331 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.331 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:21.331 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397475+0000", 2024-05-08T08:55:21.331 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0066109109999999997 2024-05-08T08:55:21.331 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.331 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.331 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:21.331 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397498+0000", 2024-05-08T08:55:21.331 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2674e-05 2024-05-08T08:55:21.331 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.332 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.332 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:21.332 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397508+0000", 2024-05-08T08:55:21.332 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9019999999999994e-06 2024-05-08T08:55:21.332 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.332 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.332 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:21.332 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397609+0000", 2024-05-08T08:55:21.332 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010121899999999999 2024-05-08T08:55:21.332 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.332 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.332 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:21.332 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397711+0000", 2024-05-08T08:55:21.332 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102489 2024-05-08T08:55:21.332 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.332 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.333 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-05-08T08:55:21.333 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397999+0000", 2024-05-08T08:55:21.333 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00028733799999999999 2024-05-08T08:55:21.333 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.333 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.333 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:55:21.333 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.432093+0000", 2024-05-08T08:55:21.333 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.034094865000000002 2024-05-08T08:55:21.333 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.333 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.333 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:55:21.333 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.445975+0000", 2024-05-08T08:55:21.333 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013881187 2024-05-08T08:55:21.333 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.333 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.334 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:55:21.334 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.446013+0000", 2024-05-08T08:55:21.334 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8155000000000003e-05 2024-05-08T08:55:21.334 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.334 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.334 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:21.334 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.446052+0000", 2024-05-08T08:55:21.334 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8962999999999999e-05 2024-05-08T08:55:21.334 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.334 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:21.334 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.334 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.334 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:21.334 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:55:21.336 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:55:21.336 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:55:21.338+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:55:21.396 INFO:teuthology.orchestra.run.smithi059.stderr:nodeep-scrub is unset 2024-05-08T08:55:21.455 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:55:21.455 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:55:21.455 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:55:21.455 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:55:21.458 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:55:21.461+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:55:21.465 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:55:21.537 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:55:21.540+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:55:21.576 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:55:21.577 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:55:21.577 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:55:21.577 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:55:21.577 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:55:21.586 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:55:21.638 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:55:21.640+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:55:21.692 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:55:21.692 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:55:21.692 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:55:21.692 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:55:21.692 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.692 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6690 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1093632~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:21.692 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:55:21.692 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 169.97829055700001, 2024-05-08T08:55:21.692 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052462754, 2024-05-08T08:55:21.692 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:21.692 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:21.693 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:21.693 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:21.693 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:21.693 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:21.693 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6690 2024-05-08T08:55:21.693 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.693 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:21.693 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.693 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:21.693 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:55:21.693 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:21.693 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.693 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.693 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:21.693 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:55:21.694 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:21.694 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.694 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.694 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:21.694 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715346+0000", 2024-05-08T08:55:21.694 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.84e-07 2024-05-08T08:55:21.694 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.694 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.694 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:21.694 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715348+0000", 2024-05-08T08:55:21.694 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.114e-06 2024-05-08T08:55:21.694 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.694 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.694 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:21.695 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715353+0000", 2024-05-08T08:55:21.695 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5720000000000002e-06 2024-05-08T08:55:21.695 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.695 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.695 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:21.695 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715358+0000", 2024-05-08T08:55:21.695 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7140000000000001e-06 2024-05-08T08:55:21.695 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.695 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.695 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:21.695 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767638+0000", 2024-05-08T08:55:21.695 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052279562000000002 2024-05-08T08:55:21.695 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.695 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.695 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:21.696 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767713+0000", 2024-05-08T08:55:21.696 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5652e-05 2024-05-08T08:55:21.696 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.696 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.696 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:21.696 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767808+0000", 2024-05-08T08:55:21.696 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4755999999999999e-05 2024-05-08T08:55:21.696 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.696 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:21.696 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.696 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.696 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.696 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6692 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1110016~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:21.696 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:55:21.696 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 169.97827146700001, 2024-05-08T08:55:21.696 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052666495000000001, 2024-05-08T08:55:21.697 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:21.697 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:21.697 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:21.697 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:21.697 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:21.697 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:21.697 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6692 2024-05-08T08:55:21.697 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.697 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:21.697 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.697 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:21.697 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:55:21.697 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:21.697 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.697 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.697 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:21.698 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:55:21.698 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:21.698 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.698 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.698 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:21.698 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:55:21.698 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8599999999999999e-07 2024-05-08T08:55:21.698 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.698 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.698 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:21.698 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715372+0000", 2024-05-08T08:55:21.698 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0990000000000001e-06 2024-05-08T08:55:21.698 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.698 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.699 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:21.699 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715378+0000", 2024-05-08T08:55:21.699 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0680000000000002e-06 2024-05-08T08:55:21.699 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.699 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.699 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:21.699 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715383+0000", 2024-05-08T08:55:21.699 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8740000000000003e-06 2024-05-08T08:55:21.699 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.699 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.699 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:21.699 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767848+0000", 2024-05-08T08:55:21.699 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052464668999999998 2024-05-08T08:55:21.699 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.700 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.700 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:21.700 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767927+0000", 2024-05-08T08:55:21.700 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9228999999999996e-05 2024-05-08T08:55:21.700 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.700 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.700 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:21.700 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768031+0000", 2024-05-08T08:55:21.701 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010417 2024-05-08T08:55:21.701 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.701 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:21.701 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.701 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.701 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.701 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6694 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1126400~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:21.701 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:55:21.701 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 169.97824657000001, 2024-05-08T08:55:21.701 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052848829, 2024-05-08T08:55:21.701 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:21.701 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:21.701 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:21.701 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:21.701 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:21.701 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:21.702 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6694 2024-05-08T08:55:21.702 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.702 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:21.702 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.702 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:21.702 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:55:21.702 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:21.702 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.702 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.702 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:21.703 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:55:21.703 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:21.703 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.703 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.703 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:21.703 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715390+0000", 2024-05-08T08:55:21.703 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9900000000000001e-07 2024-05-08T08:55:21.703 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.703 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.703 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:21.703 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715392+0000", 2024-05-08T08:55:21.703 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8279999999999999e-06 2024-05-08T08:55:21.703 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.703 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.704 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:21.704 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715398+0000", 2024-05-08T08:55:21.704 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7619999999999998e-06 2024-05-08T08:55:21.704 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.704 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.704 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:21.704 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715403+0000", 2024-05-08T08:55:21.704 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7790000000000003e-06 2024-05-08T08:55:21.704 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.704 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.704 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:21.704 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768067+0000", 2024-05-08T08:55:21.704 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052663785999999997 2024-05-08T08:55:21.704 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.704 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.705 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:21.705 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768142+0000", 2024-05-08T08:55:21.705 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5018999999999994e-05 2024-05-08T08:55:21.705 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.705 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.705 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:21.705 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768238+0000", 2024-05-08T08:55:21.705 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6255999999999995e-05 2024-05-08T08:55:21.705 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.705 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:21.705 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.705 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.705 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.705 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6696 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1142784~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:21.705 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:55:21.706 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 169.978227062, 2024-05-08T08:55:21.706 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.053051057999999998, 2024-05-08T08:55:21.706 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:21.706 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:21.706 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:21.706 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:21.706 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:21.706 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:21.706 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6696 2024-05-08T08:55:21.706 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.706 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:21.706 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.706 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:21.707 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:55:21.707 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:21.707 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.707 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.707 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:21.707 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:55:21.707 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:21.707 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.707 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.707 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:21.707 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:55:21.707 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-05-08T08:55:21.707 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.707 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.707 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:21.708 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715411+0000", 2024-05-08T08:55:21.708 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9630000000000001e-06 2024-05-08T08:55:21.708 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.708 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.708 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:21.708 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715417+0000", 2024-05-08T08:55:21.708 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7250000000000002e-06 2024-05-08T08:55:21.708 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.708 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.708 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:21.708 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715421+0000", 2024-05-08T08:55:21.708 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4089999999999996e-06 2024-05-08T08:55:21.708 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.708 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.709 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:21.709 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768277+0000", 2024-05-08T08:55:21.709 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052855293999999997 2024-05-08T08:55:21.709 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.709 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.709 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:21.709 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768358+0000", 2024-05-08T08:55:21.709 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1720999999999999e-05 2024-05-08T08:55:21.709 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.709 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.710 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:21.710 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768460+0000", 2024-05-08T08:55:21.710 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010154600000000001 2024-05-08T08:55:21.710 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.710 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:21.710 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.710 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.710 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.710 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6698 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1159168~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:21.710 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:55:21.710 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 169.97820819500001, 2024-05-08T08:55:21.710 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.053245726, 2024-05-08T08:55:21.710 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:21.711 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:21.711 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:21.711 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:21.711 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:21.711 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:21.711 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6698 2024-05-08T08:55:21.711 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.711 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:21.711 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.711 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:21.711 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:55:21.711 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:21.711 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.711 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.711 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:21.712 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:55:21.712 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:21.712 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.712 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.712 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:21.712 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716936+0000", 2024-05-08T08:55:21.712 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001508665 2024-05-08T08:55:21.712 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.712 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.712 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:21.712 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716941+0000", 2024-05-08T08:55:21.712 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1910000000000002e-06 2024-05-08T08:55:21.712 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.712 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.712 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:21.713 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716950+0000", 2024-05-08T08:55:21.713 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1900000000000001e-06 2024-05-08T08:55:21.713 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.713 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.713 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:21.713 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716958+0000", 2024-05-08T08:55:21.713 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8019999999999992e-06 2024-05-08T08:55:21.713 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.713 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.713 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:21.713 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768499+0000", 2024-05-08T08:55:21.713 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051540926000000001 2024-05-08T08:55:21.713 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.713 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.713 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:21.714 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768574+0000", 2024-05-08T08:55:21.714 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4988999999999996e-05 2024-05-08T08:55:21.714 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.714 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.714 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:21.714 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768673+0000", 2024-05-08T08:55:21.714 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9963000000000005e-05 2024-05-08T08:55:21.714 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.714 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:21.715 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.715 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.715 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.715 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6706 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1224704~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:21.715 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:55:21.715 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 169.976602888, 2024-05-08T08:55:21.715 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052435598, 2024-05-08T08:55:21.715 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:21.715 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:21.715 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:21.715 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:21.715 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:21.715 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:21.715 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6706 2024-05-08T08:55:21.716 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.716 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:21.716 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.716 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:21.716 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:55:21.716 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:21.716 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.716 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.716 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:21.716 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:55:21.716 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:21.716 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.716 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.716 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:21.716 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:55:21.717 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:55:21.717 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.717 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.717 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:21.717 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717035+0000", 2024-05-08T08:55:21.717 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9290000000000001e-06 2024-05-08T08:55:21.717 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.717 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.717 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:21.717 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717041+0000", 2024-05-08T08:55:21.717 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6720000000000001e-06 2024-05-08T08:55:21.717 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.717 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.717 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:21.717 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717046+0000", 2024-05-08T08:55:21.718 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5650000000000003e-06 2024-05-08T08:55:21.718 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.718 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.718 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:21.718 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769311+0000", 2024-05-08T08:55:21.718 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052265645999999999 2024-05-08T08:55:21.718 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.718 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.718 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:21.718 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769379+0000", 2024-05-08T08:55:21.718 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7391000000000006e-05 2024-05-08T08:55:21.718 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.718 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.718 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:21.718 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769469+0000", 2024-05-08T08:55:21.719 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0025999999999995e-05 2024-05-08T08:55:21.719 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.719 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:21.719 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.719 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.719 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.719 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6708 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1241088~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:21.719 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:55:21.719 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 169.97658478400001, 2024-05-08T08:55:21.719 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052640552, 2024-05-08T08:55:21.719 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:21.719 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:21.719 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:21.720 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:21.720 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:21.720 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:21.720 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6708 2024-05-08T08:55:21.720 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.720 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:21.720 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.720 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:21.720 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:55:21.720 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:21.720 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.721 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.721 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:21.721 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:55:21.721 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:21.721 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.721 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.721 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:21.721 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717052+0000", 2024-05-08T08:55:21.721 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:55:21.721 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.721 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.721 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:21.721 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717053+0000", 2024-05-08T08:55:21.721 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8330000000000001e-06 2024-05-08T08:55:21.722 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.722 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.722 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:21.722 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717058+0000", 2024-05-08T08:55:21.722 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.933e-06 2024-05-08T08:55:21.722 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.722 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.722 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:21.722 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717063+0000", 2024-05-08T08:55:21.722 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2180000000000001e-06 2024-05-08T08:55:21.722 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.722 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.722 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:21.722 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769507+0000", 2024-05-08T08:55:21.723 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052444866999999999 2024-05-08T08:55:21.723 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.723 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.723 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:21.723 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769589+0000", 2024-05-08T08:55:21.723 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2039000000000005e-05 2024-05-08T08:55:21.723 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.723 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.723 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:21.723 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769692+0000", 2024-05-08T08:55:21.723 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102252 2024-05-08T08:55:21.723 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.723 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:21.723 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.724 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.724 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.724 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6710 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1257472~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:21.724 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:55:21.724 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 169.97656676, 2024-05-08T08:55:21.724 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.05283057, 2024-05-08T08:55:21.724 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:21.724 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:21.724 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:21.724 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:21.724 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:21.724 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:21.724 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6710 2024-05-08T08:55:21.724 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.725 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:21.725 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.725 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:21.725 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:55:21.725 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:21.725 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.725 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.725 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:21.725 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:55:21.725 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:21.725 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.725 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.725 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:21.725 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718951+0000", 2024-05-08T08:55:21.726 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0018817809999999999 2024-05-08T08:55:21.726 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.726 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.726 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:21.726 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718956+0000", 2024-05-08T08:55:21.726 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5399999999999997e-06 2024-05-08T08:55:21.726 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.726 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.726 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:21.726 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718964+0000", 2024-05-08T08:55:21.726 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6629999999999999e-06 2024-05-08T08:55:21.726 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.726 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.726 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:21.726 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718972+0000", 2024-05-08T08:55:21.727 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5229999999999998e-06 2024-05-08T08:55:21.727 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.727 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.727 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:21.727 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769729+0000", 2024-05-08T08:55:21.727 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050756959999999997 2024-05-08T08:55:21.727 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.727 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.727 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:21.727 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769803+0000", 2024-05-08T08:55:21.727 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4538999999999999e-05 2024-05-08T08:55:21.727 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.727 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.727 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:21.728 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769900+0000", 2024-05-08T08:55:21.728 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6564000000000006e-05 2024-05-08T08:55:21.728 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.728 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:21.728 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.728 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.728 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.728 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6726 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1388544~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:21.728 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:55:21.728 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 169.97449198800001, 2024-05-08T08:55:21.728 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052451506000000002, 2024-05-08T08:55:21.728 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:21.728 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:21.728 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:21.728 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:21.728 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:21.729 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:21.729 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6726 2024-05-08T08:55:21.729 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.729 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:21.729 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.729 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:21.729 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:55:21.729 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:21.729 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.729 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.729 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:21.729 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:55:21.729 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:21.729 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.729 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.730 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:21.730 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719200+0000", 2024-05-08T08:55:21.730 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5865999999999997e-05 2024-05-08T08:55:21.730 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.730 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.730 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:21.730 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719204+0000", 2024-05-08T08:55:21.730 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8630000000000001e-06 2024-05-08T08:55:21.730 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.730 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.730 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:21.730 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719215+0000", 2024-05-08T08:55:21.730 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1590000000000001e-05 2024-05-08T08:55:21.731 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.731 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.731 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:21.731 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719220+0000", 2024-05-08T08:55:21.731 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1619999999999999e-06 2024-05-08T08:55:21.731 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.731 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.731 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:21.731 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771425+0000", 2024-05-08T08:55:21.731 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052204249000000001 2024-05-08T08:55:21.731 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.731 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.731 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:21.731 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771498+0000", 2024-05-08T08:55:21.731 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3782999999999995e-05 2024-05-08T08:55:21.732 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.732 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.732 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:21.732 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771595+0000", 2024-05-08T08:55:21.732 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6992999999999998e-05 2024-05-08T08:55:21.732 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.732 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:21.732 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.732 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.732 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.732 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6728 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1404928~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:21.732 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:55:21.732 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 169.97440458299999, 2024-05-08T08:55:21.732 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052590747, 2024-05-08T08:55:21.732 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:21.733 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:21.733 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:21.733 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:21.733 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:21.733 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:21.733 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6728 2024-05-08T08:55:21.733 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.733 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:21.733 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.733 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:21.733 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:55:21.733 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:21.733 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.733 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.734 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:21.734 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:55:21.734 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:21.734 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.734 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.734 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:21.734 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719232+0000", 2024-05-08T08:55:21.734 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4000000000000001e-07 2024-05-08T08:55:21.734 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.734 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.734 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:21.734 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719234+0000", 2024-05-08T08:55:21.734 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4499999999999998e-06 2024-05-08T08:55:21.734 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.734 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.735 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:21.735 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719240+0000", 2024-05-08T08:55:21.735 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5720000000000002e-06 2024-05-08T08:55:21.735 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.735 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.735 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:21.735 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719245+0000", 2024-05-08T08:55:21.735 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8690000000000003e-06 2024-05-08T08:55:21.735 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.735 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.735 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:21.735 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771636+0000", 2024-05-08T08:55:21.735 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052390979999999997 2024-05-08T08:55:21.735 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.736 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.736 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:21.736 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771717+0000", 2024-05-08T08:55:21.736 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1291999999999993e-05 2024-05-08T08:55:21.736 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.736 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.736 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:21.736 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771822+0000", 2024-05-08T08:55:21.736 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104944 2024-05-08T08:55:21.736 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.736 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:21.736 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.736 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.736 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.736 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6730 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1421312~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:21.737 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:55:21.737 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 169.97438434099999, 2024-05-08T08:55:21.737 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052770477000000003, 2024-05-08T08:55:21.737 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:21.737 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:21.737 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:21.737 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:21.737 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:21.737 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:21.737 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6730 2024-05-08T08:55:21.737 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.737 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:21.737 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.737 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:21.737 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:55:21.738 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:21.738 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.738 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.738 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:21.738 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:55:21.738 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:21.738 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.738 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.738 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:21.738 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:55:21.738 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9000000000000002e-07 2024-05-08T08:55:21.738 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.738 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.738 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:21.738 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719254+0000", 2024-05-08T08:55:21.739 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4099999999999998e-06 2024-05-08T08:55:21.739 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.739 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.739 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:21.739 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719263+0000", 2024-05-08T08:55:21.739 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4109999999999996e-06 2024-05-08T08:55:21.739 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.739 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.739 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:21.739 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719268+0000", 2024-05-08T08:55:21.739 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2669999999999997e-06 2024-05-08T08:55:21.739 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.739 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.739 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:21.740 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771846+0000", 2024-05-08T08:55:21.740 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052577677000000003 2024-05-08T08:55:21.740 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.740 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.740 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:21.740 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771920+0000", 2024-05-08T08:55:21.740 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4460000000000002e-05 2024-05-08T08:55:21.740 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.740 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.740 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:21.740 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772022+0000", 2024-05-08T08:55:21.740 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010186199999999999 2024-05-08T08:55:21.740 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.740 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:21.740 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.741 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.741 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.741 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6732 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1437696~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:21.741 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:55:21.741 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 169.974359534, 2024-05-08T08:55:21.741 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052905575000000003, 2024-05-08T08:55:21.741 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:21.741 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:21.741 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:21.741 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:21.741 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:21.741 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:21.741 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6732 2024-05-08T08:55:21.741 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.741 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:21.741 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.742 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:21.742 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:55:21.742 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:21.742 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.742 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.742 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:21.742 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:55:21.742 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:21.742 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.742 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.742 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:21.742 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721392+0000", 2024-05-08T08:55:21.742 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002115287 2024-05-08T08:55:21.742 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.742 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.743 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:21.743 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721396+0000", 2024-05-08T08:55:21.743 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1679999999999997e-06 2024-05-08T08:55:21.743 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.743 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.743 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:21.743 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721404+0000", 2024-05-08T08:55:21.743 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4409999999999993e-06 2024-05-08T08:55:21.743 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.743 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.743 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:21.743 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721410+0000", 2024-05-08T08:55:21.743 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9150000000000001e-06 2024-05-08T08:55:21.743 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.743 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.744 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:21.744 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772061+0000", 2024-05-08T08:55:21.744 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050651068 2024-05-08T08:55:21.744 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.744 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.744 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:21.744 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772127+0000", 2024-05-08T08:55:21.744 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5424000000000002e-05 2024-05-08T08:55:21.744 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.744 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.744 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:21.744 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772182+0000", 2024-05-08T08:55:21.744 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5272000000000002e-05 2024-05-08T08:55:21.744 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.744 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:21.745 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.745 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.745 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.745 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6749 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1585152~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:21.745 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:55:21.745 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 169.972058764, 2024-05-08T08:55:21.745 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052350264000000001, 2024-05-08T08:55:21.745 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:21.745 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:21.745 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:21.745 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:21.745 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:21.745 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:21.745 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6749 2024-05-08T08:55:21.745 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.746 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:21.746 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.746 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:21.746 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:55:21.746 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:21.746 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.746 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.746 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:21.746 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:55:21.746 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:21.746 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.746 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.746 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:21.746 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721578+0000", 2024-05-08T08:55:21.747 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0399999999999996e-07 2024-05-08T08:55:21.747 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.747 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.747 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:21.747 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721580+0000", 2024-05-08T08:55:21.747 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5450000000000002e-06 2024-05-08T08:55:21.747 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.747 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.747 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:21.747 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721587+0000", 2024-05-08T08:55:21.747 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0190000000000001e-06 2024-05-08T08:55:21.747 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.747 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.747 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:21.747 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721593+0000", 2024-05-08T08:55:21.748 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5659999999999997e-06 2024-05-08T08:55:21.748 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.748 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.748 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:21.748 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773759+0000", 2024-05-08T08:55:21.748 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052166553999999997 2024-05-08T08:55:21.748 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.748 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.748 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:21.748 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773833+0000", 2024-05-08T08:55:21.748 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3913999999999997e-05 2024-05-08T08:55:21.748 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.748 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.748 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:21.749 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773927+0000", 2024-05-08T08:55:21.749 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4061999999999995e-05 2024-05-08T08:55:21.749 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.749 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:21.749 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.749 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.749 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.749 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6751 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1601536~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:21.749 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:55:21.749 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 169.97203548499999, 2024-05-08T08:55:21.749 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052461302000000001, 2024-05-08T08:55:21.749 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:21.749 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:21.749 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:21.749 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:21.750 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:21.750 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:21.750 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6751 2024-05-08T08:55:21.750 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.750 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:21.750 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.750 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:21.750 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:55:21.750 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:21.750 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.750 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.750 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:21.750 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:55:21.751 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:21.751 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.751 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.751 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:21.751 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722939+0000", 2024-05-08T08:55:21.751 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001338601 2024-05-08T08:55:21.751 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.751 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.751 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:21.751 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722944+0000", 2024-05-08T08:55:21.751 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4469999999999999e-06 2024-05-08T08:55:21.751 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.751 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.751 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:21.751 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722955+0000", 2024-05-08T08:55:21.752 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1892999999999999e-05 2024-05-08T08:55:21.752 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.752 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.752 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:21.752 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722963+0000", 2024-05-08T08:55:21.752 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1080000000000004e-06 2024-05-08T08:55:21.752 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.752 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.752 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:21.752 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773964+0000", 2024-05-08T08:55:21.752 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051001022 2024-05-08T08:55:21.752 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.752 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.753 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:21.753 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.774006+0000", 2024-05-08T08:55:21.753 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2729999999999999e-05 2024-05-08T08:55:21.753 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.753 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.753 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:21.753 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.774062+0000", 2024-05-08T08:55:21.753 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5501000000000003e-05 2024-05-08T08:55:21.753 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.753 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:21.753 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.753 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.753 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.753 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6760 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1683456~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:21.754 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:55:21.754 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 169.97052610899999, 2024-05-08T08:55:21.754 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052571312000000002, 2024-05-08T08:55:21.754 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:21.754 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:21.754 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:21.754 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:21.754 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:21.754 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:21.754 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6760 2024-05-08T08:55:21.754 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.754 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:21.754 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.754 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:21.755 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:55:21.755 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:21.755 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.755 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.755 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:21.755 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:55:21.755 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:21.755 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.755 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.755 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:21.755 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:55:21.755 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2100000000000002e-07 2024-05-08T08:55:21.755 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.756 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.756 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:21.756 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723113+0000", 2024-05-08T08:55:21.756 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3149999999999999e-06 2024-05-08T08:55:21.756 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.756 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.756 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:21.756 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723118+0000", 2024-05-08T08:55:21.756 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8860000000000001e-06 2024-05-08T08:55:21.756 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.756 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.756 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:21.756 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723123+0000", 2024-05-08T08:55:21.756 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6770000000000003e-06 2024-05-08T08:55:21.756 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.757 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.757 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:21.757 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775414+0000", 2024-05-08T08:55:21.757 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052290692999999999 2024-05-08T08:55:21.757 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.757 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.757 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:21.757 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775459+0000", 2024-05-08T08:55:21.757 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5639999999999997e-05 2024-05-08T08:55:21.757 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.757 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.757 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:21.757 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775681+0000", 2024-05-08T08:55:21.757 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00022168000000000001 2024-05-08T08:55:21.757 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.758 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:21.758 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.758 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.758 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.758 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6761 3.16 3:6bd6d6da:::rbd_data.1140782590a5.0000000000000015:6 [read 1095680~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:21.758 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:55:21.758 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 169.97050688900001, 2024-05-08T08:55:21.758 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052751275, 2024-05-08T08:55:21.758 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:21.758 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:21.758 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:21.758 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:21.758 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:21.759 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:21.759 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6761 2024-05-08T08:55:21.759 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.759 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:21.759 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.759 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:21.759 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:55:21.759 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:21.759 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.759 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.759 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:21.759 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:55:21.759 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:21.759 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.759 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.760 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:21.760 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724939+0000", 2024-05-08T08:55:21.760 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001809817 2024-05-08T08:55:21.760 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.760 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.760 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:21.760 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724943+0000", 2024-05-08T08:55:21.760 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4920000000000004e-06 2024-05-08T08:55:21.760 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.760 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.760 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:21.760 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724956+0000", 2024-05-08T08:55:21.760 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2427e-05 2024-05-08T08:55:21.760 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.760 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.761 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:21.761 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724963+0000", 2024-05-08T08:55:21.761 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2559999999999996e-06 2024-05-08T08:55:21.761 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.761 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.761 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:21.761 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775711+0000", 2024-05-08T08:55:21.761 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050747621 2024-05-08T08:55:21.761 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.761 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.761 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:21.761 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775785+0000", 2024-05-08T08:55:21.761 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4634000000000004e-05 2024-05-08T08:55:21.761 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.762 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.762 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:21.762 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775880+0000", 2024-05-08T08:55:21.762 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5027999999999999e-05 2024-05-08T08:55:21.762 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.762 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:21.762 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.762 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.762 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.762 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6770 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1765376~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:21.762 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:55:21.762 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 169.968475794, 2024-05-08T08:55:21.762 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052339296, 2024-05-08T08:55:21.762 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:21.763 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:21.763 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:21.763 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:21.763 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:21.763 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:21.763 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6770 2024-05-08T08:55:21.763 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.763 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:21.763 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.763 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:21.763 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:55:21.763 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:21.763 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.763 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.764 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:21.764 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:55:21.764 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:21.764 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.764 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.764 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:21.764 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725161+0000", 2024-05-08T08:55:21.764 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-05-08T08:55:21.764 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.764 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.764 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:21.764 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725163+0000", 2024-05-08T08:55:21.764 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4370000000000001e-06 2024-05-08T08:55:21.765 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.765 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.765 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:21.765 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725169+0000", 2024-05-08T08:55:21.765 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8409999999999996e-06 2024-05-08T08:55:21.765 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.765 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.765 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:21.765 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725174+0000", 2024-05-08T08:55:21.765 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7249999999999997e-06 2024-05-08T08:55:21.765 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.765 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.766 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:21.766 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777343+0000", 2024-05-08T08:55:21.766 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052169633999999999 2024-05-08T08:55:21.766 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.766 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.766 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:21.766 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777409+0000", 2024-05-08T08:55:21.766 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6033999999999998e-05 2024-05-08T08:55:21.766 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.766 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.766 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:21.766 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777499+0000", 2024-05-08T08:55:21.766 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0224999999999998e-05 2024-05-08T08:55:21.766 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.767 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:21.767 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.767 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.767 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.767 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6771 3.16 3:6bd6d6da:::rbd_data.1140782590a5.0000000000000015:6 [read 1177600~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:21.767 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:55:21.767 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 169.96845597699999, 2024-05-08T08:55:21.767 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052438604999999999, 2024-05-08T08:55:21.767 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:21.767 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:21.767 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:21.767 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:21.767 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:21.767 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:21.767 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6771 2024-05-08T08:55:21.768 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.768 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:21.768 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.768 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:21.768 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:55:21.768 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:21.768 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.768 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.768 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:21.768 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:55:21.768 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:21.768 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.768 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.768 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:21.769 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:55:21.769 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:55:21.769 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.769 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.769 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:21.769 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725188+0000", 2024-05-08T08:55:21.769 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1579999999999999e-06 2024-05-08T08:55:21.769 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.769 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.769 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:21.769 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725195+0000", 2024-05-08T08:55:21.769 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8909999999999995e-06 2024-05-08T08:55:21.769 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.769 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.769 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:21.770 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725200+0000", 2024-05-08T08:55:21.770 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9810000000000003e-06 2024-05-08T08:55:21.770 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.770 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.770 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:21.770 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777527+0000", 2024-05-08T08:55:21.770 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052326582000000003 2024-05-08T08:55:21.770 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.770 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.770 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:21.770 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777566+0000", 2024-05-08T08:55:21.770 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9447999999999999e-05 2024-05-08T08:55:21.770 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.770 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.771 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:21.771 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777619+0000", 2024-05-08T08:55:21.771 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2135000000000001e-05 2024-05-08T08:55:21.771 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.771 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:21.771 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.771 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.771 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.771 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6772 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1781760~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:21.771 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:55:21.771 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 169.96842792000001, 2024-05-08T08:55:21.772 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052601799999999997, 2024-05-08T08:55:21.772 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:21.772 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:21.772 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:21.772 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:21.772 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:21.772 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:21.772 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6772 2024-05-08T08:55:21.772 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.772 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:21.772 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.773 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:21.773 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:55:21.773 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:21.773 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.773 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.773 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:21.773 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:55:21.773 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:21.773 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.773 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.773 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:21.773 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726939+0000", 2024-05-08T08:55:21.773 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001730493 2024-05-08T08:55:21.774 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.774 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.774 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:21.774 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726943+0000", 2024-05-08T08:55:21.774 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7330000000000003e-06 2024-05-08T08:55:21.774 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.774 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.774 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:21.774 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726954+0000", 2024-05-08T08:55:21.774 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1107000000000001e-05 2024-05-08T08:55:21.774 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.774 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.774 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:21.775 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726962+0000", 2024-05-08T08:55:21.775 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.503e-06 2024-05-08T08:55:21.775 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.775 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.775 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:21.775 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777653+0000", 2024-05-08T08:55:21.775 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050691542999999999 2024-05-08T08:55:21.775 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.775 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.775 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:21.775 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777719+0000", 2024-05-08T08:55:21.775 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5228000000000006e-05 2024-05-08T08:55:21.775 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.775 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.776 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:21.776 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777810+0000", 2024-05-08T08:55:21.776 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1193000000000006e-05 2024-05-08T08:55:21.776 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.776 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:21.776 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.776 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.776 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.776 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6782 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1880064~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:21.776 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:55:21.776 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 169.966459413, 2024-05-08T08:55:21.776 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052326492000000002, 2024-05-08T08:55:21.776 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:21.776 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:21.776 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:21.777 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:21.777 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:21.777 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:21.777 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6782 2024-05-08T08:55:21.777 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.777 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:21.777 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.777 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:21.777 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:55:21.777 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:21.777 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.777 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.777 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:21.777 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:55:21.778 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:21.778 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.778 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.778 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:21.778 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728939+0000", 2024-05-08T08:55:21.778 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001762719 2024-05-08T08:55:21.778 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.778 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.778 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:21.778 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728944+0000", 2024-05-08T08:55:21.779 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.746e-06 2024-05-08T08:55:21.779 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.779 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.779 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:21.779 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728955+0000", 2024-05-08T08:55:21.779 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1043000000000001e-05 2024-05-08T08:55:21.779 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.779 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.779 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:21.779 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728963+0000", 2024-05-08T08:55:21.779 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5809999999999999e-06 2024-05-08T08:55:21.779 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.779 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.779 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:21.780 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779320+0000", 2024-05-08T08:55:21.780 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050357290999999998 2024-05-08T08:55:21.780 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.780 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.780 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:21.780 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779401+0000", 2024-05-08T08:55:21.780 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0574000000000005e-05 2024-05-08T08:55:21.780 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:21.780 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:21.780 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:21.780 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779503+0000", 2024-05-08T08:55:21.780 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102538 2024-05-08T08:55:21.780 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.781 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:21.781 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.781 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:21.781 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:21.781 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:55:21.783 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:55:21.785+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:55:21.783 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:55:21.883 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:55:21.885+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:55:21.898 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:55:21.899 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:55:21.899 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:55:21.899 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:55:21.908 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:55:21.939 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T08:55:21.984 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:55:21.987+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:55:22.065 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:55:22.065 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:55:22.065 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:55:22.065 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:55:22.065 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:55:22.078 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:55:22.085 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:55:22.086 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:55:22.086 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:55:22.088+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:55:22.105 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:55:22.105 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:55:22.122 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:55:22.123 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:55:22.140 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:55:22.140 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:55:22.158 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:55:22.207 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:55:22.207 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:55:22.208 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:55:22.209 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:22.209 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:22.209 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:55:22.250 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:55:22.252+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:55:22.312 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:55:22.314+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:55:22.359 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:22.359 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:22.368 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:55:22.412 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:55:22.415+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:55:22.514 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:55:22.516+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:55:22.528 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:22.529 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:22.539 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:55:22.616 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:55:22.619+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:55:22.653 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:22.653 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:22.662 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:55:22.717 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:55:22.719+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:55:22.773 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:22.773 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:22.783 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:55:22.818 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:55:22.820+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:55:22.892 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:22.893 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:22.902 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:55:22.918 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:55:22.920+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:55:23.012 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:23.012 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:23.019 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:55:23.022+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:55:23.021 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:55:23.120 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:55:23.122+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:55:23.129 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:23.129 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:23.138 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:55:23.224 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:55:23.226+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:55:23.283 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:23.283 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:23.291 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:55:23.333 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:55:23.334+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:55:23.418 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-05-08T08:55:23.423 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:55:23.424+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:55:23.444 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:23.444 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:23.454 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:55:23.523 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:55:23.526+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:55:23.600 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:23.601 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:23.609 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:55:23.624 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:55:23.627+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:55:23.714 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:23.714 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:23.723 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:55:23.726 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:55:23.727+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:55:23.825 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:55:23.827+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:55:23.828 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:23.828 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:23.836 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:55:23.926 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:55:23.928+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:55:23.983 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:23.983 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:23.992 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:55:24.032 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:55:24.034+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:55:24.129 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:55:24.131+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:55:24.137 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:24.137 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:24.146 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:55:24.209 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T08:55:24.230 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:55:24.231+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:55:24.251 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:24.251 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:24.259 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:55:24.331 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:55:24.334+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:55:24.355 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:55:24.356 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:55:24.374 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:55:24.374 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:55:24.384 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:55:24.384 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:55:24.384 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:55:24.384 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:55:24.391 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:55:24.391 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:55:24.393 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:55:24.409 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:55:24.409 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:55:24.422 INFO:teuthology.orchestra.run.smithi059.stderr:noscrub is set 2024-05-08T08:55:24.432 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:55:24.435+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:55:24.433 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:55:24.433 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:55:24.454 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:55:24.454 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:55:24.513 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:55:24.513 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:55:24.513 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:55:24.513 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:55:24.513 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:55:24.523 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:55:24.533 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:55:24.535+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:55:24.634 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:55:24.636+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:55:24.636 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:55:24.636 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:55:24.636 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:55:24.636 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:55:24.636 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.636 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4621.0:9089 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 23339520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:55:24.636 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:55:24.636 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 172.14479702099999, 2024-05-08T08:55:24.637 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.13814907200000001, 2024-05-08T08:55:24.637 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:24.637 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:24.637 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:55:24.637 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:24.637 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4621", 2024-05-08T08:55:24.637 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:24.637 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9089 2024-05-08T08:55:24.637 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.637 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:24.637 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.637 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:24.637 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:55:24.637 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:24.638 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.638 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.638 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:24.638 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:55:24.638 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:24.638 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.638 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.638 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:24.638 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493005+0000", 2024-05-08T08:55:24.638 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1700000000000001e-06 2024-05-08T08:55:24.638 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.638 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.638 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:24.638 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.513963+0000", 2024-05-08T08:55:24.639 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.020957581999999999 2024-05-08T08:55:24.639 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.639 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.639 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:24.639 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.513995+0000", 2024-05-08T08:55:24.639 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1993999999999999e-05 2024-05-08T08:55:24.639 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.639 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.639 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:24.639 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514007+0000", 2024-05-08T08:55:24.639 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1581e-05 2024-05-08T08:55:24.639 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.639 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.640 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:24.640 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514122+0000", 2024-05-08T08:55:24.640 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000115282 2024-05-08T08:55:24.640 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.640 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.640 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op must wait for map", 2024-05-08T08:55:24.640 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514128+0000", 2024-05-08T08:55:24.640 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9950000000000002e-06 2024-05-08T08:55:24.640 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.640 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.640 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:24.640 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.515800+0000", 2024-05-08T08:55:24.640 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001671886 2024-05-08T08:55:24.640 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.640 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.641 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:24.641 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.515893+0000", 2024-05-08T08:55:24.641 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3490000000000001e-05 2024-05-08T08:55:24.641 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.641 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.641 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 3", 2024-05-08T08:55:24.641 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.517647+0000", 2024-05-08T08:55:24.641 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0017535630000000001 2024-05-08T08:55:24.641 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.641 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.641 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:55:24.641 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.627578+0000", 2024-05-08T08:55:24.641 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.109931557 2024-05-08T08:55:24.641 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.642 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.642 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:55:24.642 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631055+0000", 2024-05-08T08:55:24.642 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0034761420000000002 2024-05-08T08:55:24.642 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.642 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.642 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:55:24.642 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631096+0000", 2024-05-08T08:55:24.642 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1059000000000001e-05 2024-05-08T08:55:24.642 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.642 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.642 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:24.642 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631151+0000", 2024-05-08T08:55:24.642 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5770999999999998e-05 2024-05-08T08:55:24.643 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:24.643 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:24.643 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:24.643 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.643 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.643 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1071 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11026432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:55:24.643 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:55:24.643 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 168.15234752999999, 2024-05-08T08:55:24.643 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070580037999999998, 2024-05-08T08:55:24.643 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:24.643 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:24.643 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:24.643 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:24.643 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:55:24.644 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:55:24.644 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1071 2024-05-08T08:55:24.644 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.644 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:24.644 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.644 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:24.644 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:55:24.644 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:24.644 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.644 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.644 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:24.644 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:55:24.644 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:24.644 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.644 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.645 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:24.645 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:55:24.645 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2800000000000002e-07 2024-05-08T08:55:24.645 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.645 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.645 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:24.645 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485454+0000", 2024-05-08T08:55:24.645 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2290000000000002e-06 2024-05-08T08:55:24.645 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.645 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.645 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:24.645 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485461+0000", 2024-05-08T08:55:24.645 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9920000000000001e-06 2024-05-08T08:55:24.645 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.646 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.646 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:24.646 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485467+0000", 2024-05-08T08:55:24.646 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3979999999999998e-06 2024-05-08T08:55:24.646 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.646 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.646 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:24.646 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555880+0000", 2024-05-08T08:55:24.646 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070413165 2024-05-08T08:55:24.646 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.646 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.646 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:24.646 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555926+0000", 2024-05-08T08:55:24.646 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6217000000000002e-05 2024-05-08T08:55:24.646 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.647 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.647 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:24.647 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556032+0000", 2024-05-08T08:55:24.647 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000105609 2024-05-08T08:55:24.647 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:24.647 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:24.647 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:24.647 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.647 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.647 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1073 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11067392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:55:24.647 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:55:24.647 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 168.15232630899999, 2024-05-08T08:55:24.647 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070821404000000004, 2024-05-08T08:55:24.647 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:24.647 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:24.648 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:24.648 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:24.648 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:55:24.648 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:55:24.648 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1073 2024-05-08T08:55:24.648 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.648 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:24.648 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.648 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:24.648 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:55:24.648 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:24.648 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.648 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.648 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:24.649 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:55:24.649 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:24.649 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.649 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.649 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:24.649 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:55:24.649 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.08e-07 2024-05-08T08:55:24.649 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.649 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.649 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:24.649 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485476+0000", 2024-05-08T08:55:24.649 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3929999999999998e-06 2024-05-08T08:55:24.649 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.649 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.649 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:24.650 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485485+0000", 2024-05-08T08:55:24.650 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1880000000000008e-06 2024-05-08T08:55:24.650 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.650 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.650 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:24.650 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485494+0000", 2024-05-08T08:55:24.650 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3109999999999995e-06 2024-05-08T08:55:24.650 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.650 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.650 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:24.650 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556082+0000", 2024-05-08T08:55:24.650 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070587426999999994 2024-05-08T08:55:24.650 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.650 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.651 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:24.651 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556177+0000", 2024-05-08T08:55:24.651 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5051999999999997e-05 2024-05-08T08:55:24.651 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.651 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.651 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:24.651 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556294+0000", 2024-05-08T08:55:24.651 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000117625 2024-05-08T08:55:24.651 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:24.651 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:24.651 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:24.651 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.651 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.651 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1075 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11108352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:55:24.652 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:55:24.652 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 168.152295732, 2024-05-08T08:55:24.652 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071045026999999997, 2024-05-08T08:55:24.652 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:24.652 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:24.652 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:24.652 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:24.652 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:55:24.652 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:55:24.652 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1075 2024-05-08T08:55:24.652 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.652 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:24.652 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.652 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:24.653 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:55:24.653 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:24.653 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.653 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.653 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:24.653 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:55:24.653 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:24.653 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.653 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.653 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:24.653 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:55:24.653 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.32e-07 2024-05-08T08:55:24.653 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.653 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.653 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:24.654 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485518+0000", 2024-05-08T08:55:24.654 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3769e-05 2024-05-08T08:55:24.654 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.654 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.654 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:24.654 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485527+0000", 2024-05-08T08:55:24.654 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6779999999999995e-06 2024-05-08T08:55:24.654 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.654 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.654 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:24.654 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485532+0000", 2024-05-08T08:55:24.654 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9830000000000004e-06 2024-05-08T08:55:24.654 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.654 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.655 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:24.655 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556340+0000", 2024-05-08T08:55:24.655 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070807743000000006 2024-05-08T08:55:24.655 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.655 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.655 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:24.655 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556435+0000", 2024-05-08T08:55:24.655 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4732e-05 2024-05-08T08:55:24.655 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.655 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.655 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:24.655 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556549+0000", 2024-05-08T08:55:24.655 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011369 2024-05-08T08:55:24.655 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:24.656 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:24.656 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:24.656 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.656 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.656 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1078 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11149312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:55:24.656 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:55:24.656 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 168.15225553299999, 2024-05-08T08:55:24.656 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071167276000000002, 2024-05-08T08:55:24.656 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:24.656 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:24.656 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:24.656 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:24.656 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:55:24.656 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:55:24.657 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1078 2024-05-08T08:55:24.657 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.657 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:24.657 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.657 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:24.657 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:55:24.657 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:24.657 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.657 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.657 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:24.657 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:55:24.657 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:24.657 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.657 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.657 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:24.658 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:55:24.658 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5600000000000001e-07 2024-05-08T08:55:24.658 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.658 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.658 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:24.658 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485547+0000", 2024-05-08T08:55:24.658 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3590000000000002e-06 2024-05-08T08:55:24.658 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.658 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.658 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:24.658 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485553+0000", 2024-05-08T08:55:24.658 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2770000000000002e-06 2024-05-08T08:55:24.658 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.658 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.658 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:24.659 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485558+0000", 2024-05-08T08:55:24.659 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.95e-06 2024-05-08T08:55:24.659 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.659 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.659 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:24.659 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556582+0000", 2024-05-08T08:55:24.659 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071024444000000006 2024-05-08T08:55:24.659 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.659 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.659 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:24.659 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556637+0000", 2024-05-08T08:55:24.659 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4672000000000001e-05 2024-05-08T08:55:24.659 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.659 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.659 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:24.660 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556711+0000", 2024-05-08T08:55:24.660 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4117999999999997e-05 2024-05-08T08:55:24.660 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:24.660 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:24.660 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:24.660 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.660 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.660 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1080 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11190272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:55:24.660 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:55:24.660 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 168.1522348, 2024-05-08T08:55:24.660 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071350721000000006, 2024-05-08T08:55:24.660 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:24.660 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:24.660 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:24.660 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:24.661 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:55:24.661 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:55:24.661 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1080 2024-05-08T08:55:24.661 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.661 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:24.661 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.661 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:24.661 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:55:24.661 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:24.661 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.661 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.661 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:24.661 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:55:24.661 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:24.661 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.661 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.662 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:24.662 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:55:24.662 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.03e-07 2024-05-08T08:55:24.662 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.662 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.662 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:24.662 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485567+0000", 2024-05-08T08:55:24.662 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1090000000000001e-06 2024-05-08T08:55:24.662 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.662 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.662 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:24.662 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485573+0000", 2024-05-08T08:55:24.662 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3049999999999998e-06 2024-05-08T08:55:24.662 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.663 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.663 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:24.663 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485578+0000", 2024-05-08T08:55:24.663 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0130000000000001e-06 2024-05-08T08:55:24.663 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.663 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.663 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:24.663 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556743+0000", 2024-05-08T08:55:24.663 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071164449000000005 2024-05-08T08:55:24.663 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.663 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.663 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:24.663 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556797+0000", 2024-05-08T08:55:24.663 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4589999999999997e-05 2024-05-08T08:55:24.663 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.664 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.664 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:24.664 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556915+0000", 2024-05-08T08:55:24.664 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000117852 2024-05-08T08:55:24.664 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:24.664 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:24.664 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:24.664 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.664 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.664 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1082 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11231232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:55:24.664 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:55:24.664 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 168.15221455700001, 2024-05-08T08:55:24.664 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071591396000000002, 2024-05-08T08:55:24.664 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:24.664 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:24.665 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:24.665 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:24.665 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:55:24.665 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:55:24.665 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1082 2024-05-08T08:55:24.665 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.665 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:24.665 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.665 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:24.665 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:55:24.665 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:24.665 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.665 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.665 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:24.665 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:55:24.666 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:24.666 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.666 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.666 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:24.666 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:55:24.666 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1800000000000001e-07 2024-05-08T08:55:24.666 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.666 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.666 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:24.666 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485587+0000", 2024-05-08T08:55:24.666 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2589999999999999e-06 2024-05-08T08:55:24.666 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.666 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.666 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:24.667 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485594+0000", 2024-05-08T08:55:24.667 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0789999999999997e-06 2024-05-08T08:55:24.667 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.667 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.667 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:24.667 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485598+0000", 2024-05-08T08:55:24.667 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7480000000000001e-06 2024-05-08T08:55:24.667 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.667 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.667 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:24.667 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556971+0000", 2024-05-08T08:55:24.667 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071372774999999999 2024-05-08T08:55:24.667 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.667 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.667 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:24.668 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557069+0000", 2024-05-08T08:55:24.668 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.7732999999999994e-05 2024-05-08T08:55:24.668 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.668 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.668 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:24.668 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557176+0000", 2024-05-08T08:55:24.668 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000107384 2024-05-08T08:55:24.668 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:24.668 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:24.668 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:24.668 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.668 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.668 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1084 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11272192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:55:24.668 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:55:24.668 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 168.15217591800001, 2024-05-08T08:55:24.669 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071808977999999996, 2024-05-08T08:55:24.669 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:24.669 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:24.669 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:24.669 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:24.669 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:55:24.669 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:55:24.669 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1084 2024-05-08T08:55:24.669 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.669 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:24.669 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.669 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:24.669 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:55:24.669 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:24.670 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.670 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.670 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:24.670 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:55:24.670 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:24.670 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.670 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.670 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:24.670 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485624+0000", 2024-05-08T08:55:24.670 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:55:24.670 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.670 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.670 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:24.670 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485626+0000", 2024-05-08T08:55:24.670 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9460000000000001e-06 2024-05-08T08:55:24.671 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.671 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.671 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:24.671 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485633+0000", 2024-05-08T08:55:24.671 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1149999999999996e-06 2024-05-08T08:55:24.671 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.671 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.671 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:24.671 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485638+0000", 2024-05-08T08:55:24.671 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5670000000000002e-06 2024-05-08T08:55:24.671 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.671 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.671 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:24.671 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557226+0000", 2024-05-08T08:55:24.671 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071587267999999996 2024-05-08T08:55:24.672 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.672 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.672 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:24.672 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557321+0000", 2024-05-08T08:55:24.672 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5785000000000005e-05 2024-05-08T08:55:24.672 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.672 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.672 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:24.672 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557432+0000", 2024-05-08T08:55:24.672 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011092800000000001 2024-05-08T08:55:24.672 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:24.672 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:24.672 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:24.672 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.672 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.673 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1086 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11313152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:55:24.673 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:55:24.673 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 168.152152591, 2024-05-08T08:55:24.673 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071933132999999996, 2024-05-08T08:55:24.673 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:24.673 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:24.673 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:24.673 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:24.673 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:55:24.673 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:55:24.673 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1086 2024-05-08T08:55:24.673 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.673 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:24.673 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.673 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:24.673 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:55:24.674 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:24.674 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.674 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.674 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:24.674 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:55:24.674 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:24.674 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.674 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.674 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:24.674 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486328+0000", 2024-05-08T08:55:24.674 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00068107900000000002 2024-05-08T08:55:24.674 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.674 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.674 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:24.675 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486331+0000", 2024-05-08T08:55:24.675 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.585e-06 2024-05-08T08:55:24.675 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.675 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.675 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:24.675 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486343+0000", 2024-05-08T08:55:24.675 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1856e-05 2024-05-08T08:55:24.675 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.675 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.675 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:24.675 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486350+0000", 2024-05-08T08:55:24.675 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8600000000000004e-06 2024-05-08T08:55:24.675 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.675 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.675 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:24.676 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557470+0000", 2024-05-08T08:55:24.676 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071120326999999997 2024-05-08T08:55:24.676 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.676 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.676 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:24.676 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557514+0000", 2024-05-08T08:55:24.676 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3909000000000003e-05 2024-05-08T08:55:24.676 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.676 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.676 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:24.676 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557580+0000", 2024-05-08T08:55:24.676 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5517000000000003e-05 2024-05-08T08:55:24.676 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:24.676 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:24.677 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:24.677 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.677 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.677 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1089 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11354112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:55:24.677 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:55:24.677 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 168.151439719, 2024-05-08T08:55:24.677 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071457381, 2024-05-08T08:55:24.677 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:24.677 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:24.677 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:24.677 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:24.677 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:55:24.677 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:55:24.677 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1089 2024-05-08T08:55:24.677 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.678 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:24.678 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.678 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:24.678 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:55:24.678 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:24.678 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.678 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.678 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:24.678 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:55:24.678 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:24.678 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.678 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.678 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:24.678 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486851+0000", 2024-05-08T08:55:24.679 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00049102 2024-05-08T08:55:24.679 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.679 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.679 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:24.679 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486854+0000", 2024-05-08T08:55:24.679 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6859999999999999e-06 2024-05-08T08:55:24.679 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.679 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.679 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:24.679 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486864+0000", 2024-05-08T08:55:24.679 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6080000000000002e-06 2024-05-08T08:55:24.679 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.679 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.679 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:24.680 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486870+0000", 2024-05-08T08:55:24.680 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.477e-06 2024-05-08T08:55:24.680 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.680 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.680 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:24.680 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557612+0000", 2024-05-08T08:55:24.680 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070741557999999996 2024-05-08T08:55:24.680 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.680 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.680 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:24.680 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557695+0000", 2024-05-08T08:55:24.680 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3300000000000005e-05 2024-05-08T08:55:24.680 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.680 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.680 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:24.681 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557817+0000", 2024-05-08T08:55:24.681 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012173200000000001 2024-05-08T08:55:24.681 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:24.681 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:24.681 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:24.681 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.681 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.681 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1091 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11395072~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:55:24.681 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:55:24.681 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 168.150918924, 2024-05-08T08:55:24.681 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071184737999999997, 2024-05-08T08:55:24.681 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:24.681 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:24.681 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:24.682 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:24.682 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:55:24.682 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:55:24.682 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1091 2024-05-08T08:55:24.682 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.682 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:24.682 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.682 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:24.682 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:55:24.682 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:24.682 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.682 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.682 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:24.682 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:55:24.682 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:24.683 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.683 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.683 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:24.683 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486881+0000", 2024-05-08T08:55:24.683 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0699999999999997e-07 2024-05-08T08:55:24.683 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.683 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.683 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:24.683 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486883+0000", 2024-05-08T08:55:24.683 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.402e-06 2024-05-08T08:55:24.683 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.683 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.683 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:24.683 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486890+0000", 2024-05-08T08:55:24.683 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0480000000000001e-06 2024-05-08T08:55:24.684 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.684 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.684 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:24.684 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486896+0000", 2024-05-08T08:55:24.684 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7640000000000002e-06 2024-05-08T08:55:24.684 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.684 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.684 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:24.684 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557860+0000", 2024-05-08T08:55:24.684 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070963791999999998 2024-05-08T08:55:24.684 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.684 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.684 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:24.684 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557952+0000", 2024-05-08T08:55:24.684 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2082000000000004e-05 2024-05-08T08:55:24.685 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.685 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.685 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:24.685 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558065+0000", 2024-05-08T08:55:24.685 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000113143 2024-05-08T08:55:24.685 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:24.685 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:24.685 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:24.685 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.685 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.685 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1094 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11436032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:55:24.685 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:55:24.685 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 168.15089575600001, 2024-05-08T08:55:24.685 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071400507000000002, 2024-05-08T08:55:24.685 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:24.686 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:24.686 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:24.686 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:24.686 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:55:24.686 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:55:24.686 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1094 2024-05-08T08:55:24.686 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.686 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:24.686 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.686 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:24.686 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:55:24.686 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:24.686 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.686 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.686 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:24.687 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:55:24.687 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:24.687 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.687 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.687 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:24.687 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487940+0000", 2024-05-08T08:55:24.687 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001036868 2024-05-08T08:55:24.687 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.687 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.687 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:24.687 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487944+0000", 2024-05-08T08:55:24.687 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7069999999999998e-06 2024-05-08T08:55:24.687 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.687 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.687 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:24.688 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487951+0000", 2024-05-08T08:55:24.688 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2649999999999999e-06 2024-05-08T08:55:24.688 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.688 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.688 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:24.688 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487961+0000", 2024-05-08T08:55:24.688 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4110000000000002e-06 2024-05-08T08:55:24.688 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.688 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.688 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:24.688 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558117+0000", 2024-05-08T08:55:24.688 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070156662999999994 2024-05-08T08:55:24.688 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.688 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.689 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:24.689 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558214+0000", 2024-05-08T08:55:24.689 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6444e-05 2024-05-08T08:55:24.689 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.689 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.689 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:24.689 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558304+0000", 2024-05-08T08:55:24.689 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0148999999999994e-05 2024-05-08T08:55:24.689 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:24.689 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:24.689 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:24.689 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.689 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.689 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1096 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11476992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:55:24.690 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:55:24.690 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 168.14982922499999, 2024-05-08T08:55:24.690 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070480074000000004, 2024-05-08T08:55:24.690 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:24.690 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:24.690 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:24.690 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:24.690 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:55:24.690 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:55:24.690 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1096 2024-05-08T08:55:24.690 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.690 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:24.690 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.690 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:24.690 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:55:24.691 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:24.691 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.691 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.691 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:24.691 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:55:24.691 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:24.691 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.691 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.691 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:24.691 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:55:24.691 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3700000000000001e-07 2024-05-08T08:55:24.691 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.691 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.691 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:24.691 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487973+0000", 2024-05-08T08:55:24.692 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0870000000000001e-06 2024-05-08T08:55:24.692 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.692 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.692 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:24.692 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487977+0000", 2024-05-08T08:55:24.692 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0600000000000001e-06 2024-05-08T08:55:24.692 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.692 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.692 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:24.692 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487980+0000", 2024-05-08T08:55:24.692 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1089999999999998e-06 2024-05-08T08:55:24.692 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.692 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.692 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:24.692 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558338+0000", 2024-05-08T08:55:24.693 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070358233000000006 2024-05-08T08:55:24.693 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.693 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.693 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:24.693 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558392+0000", 2024-05-08T08:55:24.693 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4537999999999999e-05 2024-05-08T08:55:24.693 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.693 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.693 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:24.693 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558450+0000", 2024-05-08T08:55:24.693 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7710000000000001e-05 2024-05-08T08:55:24.693 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:24.693 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:24.693 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:24.694 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.694 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.694 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1099 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11517952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:55:24.694 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:55:24.694 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 168.14981534899999, 2024-05-08T08:55:24.694 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070720506000000002, 2024-05-08T08:55:24.694 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:24.694 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:24.694 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:24.694 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:24.694 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:55:24.694 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:55:24.694 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1099 2024-05-08T08:55:24.694 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.694 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:24.695 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.695 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:24.695 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:55:24.695 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:24.695 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.695 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.695 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:24.695 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:55:24.695 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:24.695 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.695 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.695 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:24.695 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:55:24.695 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3999999999999998e-07 2024-05-08T08:55:24.696 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.696 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.696 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:24.696 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487986+0000", 2024-05-08T08:55:24.696 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3519999999999999e-06 2024-05-08T08:55:24.696 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.696 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.696 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:24.696 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487989+0000", 2024-05-08T08:55:24.696 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.23e-06 2024-05-08T08:55:24.696 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.696 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.696 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:24.696 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487991+0000", 2024-05-08T08:55:24.696 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6350000000000002e-06 2024-05-08T08:55:24.697 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.697 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.697 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:24.697 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558489+0000", 2024-05-08T08:55:24.697 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070497924000000003 2024-05-08T08:55:24.697 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.697 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.697 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:24.697 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558583+0000", 2024-05-08T08:55:24.697 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3405000000000004e-05 2024-05-08T08:55:24.697 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.697 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.697 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:24.697 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558704+0000", 2024-05-08T08:55:24.697 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012171999999999999 2024-05-08T08:55:24.697 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:24.698 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:24.698 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:24.698 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.698 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.698 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1100 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11558912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:55:24.698 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:55:24.698 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 168.14980234999999, 2024-05-08T08:55:24.698 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070962266999999996, 2024-05-08T08:55:24.698 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:24.698 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:24.698 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:24.698 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:24.698 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:55:24.698 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:55:24.699 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1100 2024-05-08T08:55:24.699 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.699 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:24.699 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.699 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:24.699 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:55:24.699 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:24.699 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.699 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.699 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:24.699 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:55:24.699 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:24.699 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.699 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.700 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:24.700 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488430+0000", 2024-05-08T08:55:24.700 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000432676 2024-05-08T08:55:24.700 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.700 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.700 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:24.700 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488433+0000", 2024-05-08T08:55:24.700 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.45e-06 2024-05-08T08:55:24.700 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.700 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.700 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:24.700 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488442+0000", 2024-05-08T08:55:24.700 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.687e-06 2024-05-08T08:55:24.700 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.700 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.701 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:24.701 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488446+0000", 2024-05-08T08:55:24.701 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4939999999999997e-06 2024-05-08T08:55:24.701 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.701 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.701 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:24.701 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558756+0000", 2024-05-08T08:55:24.701 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070310051999999998 2024-05-08T08:55:24.701 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.701 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.701 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:24.701 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558844+0000", 2024-05-08T08:55:24.701 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7690999999999998e-05 2024-05-08T08:55:24.702 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.702 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.702 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:24.702 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558959+0000", 2024-05-08T08:55:24.702 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000115217 2024-05-08T08:55:24.702 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:24.702 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:24.702 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:24.702 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.702 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.702 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1103 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11599872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:55:24.702 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:55:24.702 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 168.14934467500001, 2024-05-08T08:55:24.703 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070777100999999995, 2024-05-08T08:55:24.703 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:24.703 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:24.703 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:24.703 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:24.703 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:55:24.703 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:55:24.703 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1103 2024-05-08T08:55:24.703 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.703 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:24.703 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.703 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:24.704 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:55:24.704 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:24.704 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.704 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.704 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:24.704 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:55:24.704 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:24.704 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.704 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.704 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:24.704 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488959+0000", 2024-05-08T08:55:24.704 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00050457200000000001 2024-05-08T08:55:24.704 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.704 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.705 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:24.705 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488969+0000", 2024-05-08T08:55:24.705 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.7159999999999998e-06 2024-05-08T08:55:24.705 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.705 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.705 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:24.705 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488981+0000", 2024-05-08T08:55:24.705 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2150000000000001e-05 2024-05-08T08:55:24.705 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.705 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.705 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:24.705 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488988+0000", 2024-05-08T08:55:24.705 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8759999999999999e-06 2024-05-08T08:55:24.706 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.706 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.706 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:24.706 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558992+0000", 2024-05-08T08:55:24.706 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070004358000000003 2024-05-08T08:55:24.706 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.706 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.706 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:24.706 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559093+0000", 2024-05-08T08:55:24.706 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000100299 2024-05-08T08:55:24.706 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.706 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.706 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:24.707 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559232+0000", 2024-05-08T08:55:24.707 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013913 2024-05-08T08:55:24.707 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:24.707 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:24.707 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:24.707 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.707 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.707 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1104 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11640832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:55:24.707 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:55:24.707 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 168.14879986, 2024-05-08T08:55:24.707 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070521947000000001, 2024-05-08T08:55:24.707 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:24.707 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:24.708 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:24.708 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:24.708 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:55:24.708 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:55:24.708 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1104 2024-05-08T08:55:24.708 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.708 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:24.708 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.708 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:24.708 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:55:24.708 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:24.708 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.709 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.709 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:24.709 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:55:24.709 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:24.709 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.709 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.709 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:24.709 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489700+0000", 2024-05-08T08:55:24.709 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00070067600000000001 2024-05-08T08:55:24.709 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.709 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.709 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:24.709 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489704+0000", 2024-05-08T08:55:24.709 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1640000000000003e-06 2024-05-08T08:55:24.710 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.710 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.710 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:24.710 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489716+0000", 2024-05-08T08:55:24.710 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1379000000000001e-05 2024-05-08T08:55:24.710 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.710 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.710 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:24.710 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489724+0000", 2024-05-08T08:55:24.710 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8949999999999998e-06 2024-05-08T08:55:24.710 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.710 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.710 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:24.711 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559278+0000", 2024-05-08T08:55:24.711 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.069554151999999994 2024-05-08T08:55:24.711 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.711 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.711 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:24.711 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559387+0000", 2024-05-08T08:55:24.711 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000109258 2024-05-08T08:55:24.711 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.711 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.711 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:24.711 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559521+0000", 2024-05-08T08:55:24.711 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000134423 2024-05-08T08:55:24.711 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:24.711 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:24.711 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:24.712 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.712 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.712 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:8636 3.17 3:e8a0a3e4:::rbd_header.11402b26385:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-05-08T08:55:24.712 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:54.150971+0000", 2024-05-08T08:55:24.712 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 150.486827866, 2024-05-08T08:55:24.712 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.217273233, 2024-05-08T08:55:24.712 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:24.712 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:24.712 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:24.712 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:24.712 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:55:24.712 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:55:24.712 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 8636 2024-05-08T08:55:24.712 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.713 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:24.713 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.713 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:24.713 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150971+0000", 2024-05-08T08:55:24.713 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:24.713 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.713 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.713 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:24.713 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150971+0000", 2024-05-08T08:55:24.713 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:24.713 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.713 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.713 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:24.713 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150972+0000", 2024-05-08T08:55:24.714 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.71e-07 2024-05-08T08:55:24.714 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.714 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.714 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:24.714 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150977+0000", 2024-05-08T08:55:24.714 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.152e-06 2024-05-08T08:55:24.714 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.714 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.714 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:24.714 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150990+0000", 2024-05-08T08:55:24.714 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2364999999999999e-05 2024-05-08T08:55:24.714 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.714 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.714 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:24.715 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150998+0000", 2024-05-08T08:55:24.715 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0739999999999993e-06 2024-05-08T08:55:24.715 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.715 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.715 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:24.715 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.151054+0000", 2024-05-08T08:55:24.715 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5949000000000003e-05 2024-05-08T08:55:24.715 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.715 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.715 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:55:24.715 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.151060+0000", 2024-05-08T08:55:24.715 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6120000000000004e-06 2024-05-08T08:55:24.715 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.715 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.715 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:24.716 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.367640+0000", 2024-05-08T08:55:24.716 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2165792739999999 2024-05-08T08:55:24.716 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.716 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.716 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:24.716 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.367839+0000", 2024-05-08T08:55:24.716 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000199185 2024-05-08T08:55:24.716 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.716 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.716 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:24.716 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.368245+0000", 2024-05-08T08:55:24.716 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00040575099999999998 2024-05-08T08:55:24.716 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:24.716 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:24.717 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:24.717 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.717 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.717 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5302.0:1516 3.c 3:319bf789:::rbd_data.1140d3fc54c.0000000000000025:head [stat,write 1892352~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:55:24.717 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:27.822220+0000", 2024-05-08T08:55:24.717 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 116.815578929, 2024-05-08T08:55:24.717 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.092643744, 2024-05-08T08:55:24.717 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:24.717 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:24.717 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:55:24.717 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:24.717 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5302", 2024-05-08T08:55:24.717 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/359277859", 2024-05-08T08:55:24.717 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1516 2024-05-08T08:55:24.718 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.718 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:24.718 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.718 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:24.718 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822220+0000", 2024-05-08T08:55:24.718 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:24.718 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.718 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.718 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:24.718 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822220+0000", 2024-05-08T08:55:24.718 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:24.718 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.718 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.718 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:24.719 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822221+0000", 2024-05-08T08:55:24.719 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3500000000000002e-07 2024-05-08T08:55:24.719 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.719 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.719 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:24.719 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822228+0000", 2024-05-08T08:55:24.719 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9959999999999996e-06 2024-05-08T08:55:24.719 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.719 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.719 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:24.719 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822234+0000", 2024-05-08T08:55:24.719 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5429999999999999e-06 2024-05-08T08:55:24.719 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.719 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.719 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:24.720 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822238+0000", 2024-05-08T08:55:24.720 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.732e-06 2024-05-08T08:55:24.720 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.720 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.720 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:24.720 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822285+0000", 2024-05-08T08:55:24.720 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6810000000000001e-05 2024-05-08T08:55:24.720 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.720 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.720 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:24.720 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822358+0000", 2024-05-08T08:55:24.720 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3473000000000006e-05 2024-05-08T08:55:24.720 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.720 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.721 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 5", 2024-05-08T08:55:24.721 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822489+0000", 2024-05-08T08:55:24.721 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013011199999999999 2024-05-08T08:55:24.721 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.721 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.721 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:55:24.721 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.825332+0000", 2024-05-08T08:55:24.721 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0028434799999999998 2024-05-08T08:55:24.721 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.721 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.721 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:55:24.721 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.914753+0000", 2024-05-08T08:55:24.721 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089421424999999999 2024-05-08T08:55:24.721 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.721 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.721 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:55:24.722 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.914807+0000", 2024-05-08T08:55:24.722 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3829000000000002e-05 2024-05-08T08:55:24.722 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.722 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.722 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:24.722 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.914864+0000", 2024-05-08T08:55:24.722 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6909e-05 2024-05-08T08:55:24.722 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:24.722 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:24.722 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:24.722 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.722 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.722 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:24567 3.6 3:66af1c40:::rbd_data.1140d3fc54c.000000000000005a:18 [read 2080768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:24.722 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.216598+0000", 2024-05-08T08:55:24.723 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 112.421200849, 2024-05-08T08:55:24.723 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071602647000000005, 2024-05-08T08:55:24.723 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:24.723 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:24.723 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:24.723 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:24.723 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:24.723 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:24.723 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 24567 2024-05-08T08:55:24.723 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.723 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:24.723 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.723 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:24.723 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216598+0000", 2024-05-08T08:55:24.723 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:24.724 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.724 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.724 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:24.724 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216598+0000", 2024-05-08T08:55:24.724 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:24.724 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.724 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.724 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:24.724 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216599+0000", 2024-05-08T08:55:24.724 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8999999999999997e-07 2024-05-08T08:55:24.724 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.724 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.724 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:24.724 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216601+0000", 2024-05-08T08:55:24.725 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9889999999999999e-06 2024-05-08T08:55:24.725 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.725 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.725 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:24.725 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216607+0000", 2024-05-08T08:55:24.725 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3600000000000001e-06 2024-05-08T08:55:24.725 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.725 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.725 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:24.725 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216612+0000", 2024-05-08T08:55:24.725 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.69e-06 2024-05-08T08:55:24.725 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.725 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.725 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:24.725 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.287076+0000", 2024-05-08T08:55:24.726 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070464355000000006 2024-05-08T08:55:24.726 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.726 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.726 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:24.726 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.287151+0000", 2024-05-08T08:55:24.726 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4924000000000003e-05 2024-05-08T08:55:24.726 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:24.726 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:24.726 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:24.726 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.288201+0000", 2024-05-08T08:55:24.726 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0010498389999999999 2024-05-08T08:55:24.726 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:24.726 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:24.726 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:24.726 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:24.727 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:24.727 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:55:24.728 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:55:24.735 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:55:24.737+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:55:24.836 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:55:24.838+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:55:24.837 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:55:24.837 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:55:24.837 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:55:24.837 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:55:24.846 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:55:24.936 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:55:24.939+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:55:24.995 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:55:24.995 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:55:24.995 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:55:24.995 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:55:24.995 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:55:25.004 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:55:25.049 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:55:25.052+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:55:25.116 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:55:25.116 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:55:25.116 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:55:25.117 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:55:25.117 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.117 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:12443 3.0 3:02d0158e:::rbd_data.114071bcf828.0000000000000037:head [stat,delete] snapc 19=[19] RETRY=3 ondisk+retry+write+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:25.117 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.610027+0000", 2024-05-08T08:55:25.117 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 113.508056792, 2024-05-08T08:55:25.117 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.083586333999999998, 2024-05-08T08:55:25.117 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:25.117 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:25.117 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:25.117 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:25.117 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:25.117 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:25.117 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 12443 2024-05-08T08:55:25.117 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.117 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:25.118 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.118 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:25.118 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610027+0000", 2024-05-08T08:55:25.118 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:25.118 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.118 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.118 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:25.118 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610027+0000", 2024-05-08T08:55:25.118 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:25.118 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.118 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.118 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:25.118 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610028+0000", 2024-05-08T08:55:25.118 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.13e-07 2024-05-08T08:55:25.119 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.119 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.119 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:25.119 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610030+0000", 2024-05-08T08:55:25.119 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7669999999999999e-06 2024-05-08T08:55:25.119 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.119 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.119 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:25.119 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610039+0000", 2024-05-08T08:55:25.119 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4959999999999996e-06 2024-05-08T08:55:25.119 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.119 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.119 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:25.119 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610045+0000", 2024-05-08T08:55:25.119 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6779999999999997e-06 2024-05-08T08:55:25.120 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.120 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.120 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:25.120 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.660873+0000", 2024-05-08T08:55:25.120 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.050828748 2024-05-08T08:55:25.120 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.120 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.120 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:25.120 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.661017+0000", 2024-05-08T08:55:25.120 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014391999999999999 2024-05-08T08:55:25.120 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.120 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.120 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:25.120 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.693613+0000", 2024-05-08T08:55:25.120 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.032596111999999997 2024-05-08T08:55:25.121 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.121 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:25.121 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.121 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.121 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.121 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:14301 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [watch reconnect cookie 139837155901536 gen 12] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:25.121 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.645078+0000", 2024-05-08T08:55:25.121 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 113.473006154, 2024-05-08T08:55:25.121 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088281852999999993, 2024-05-08T08:55:25.121 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:25.121 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:25.121 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:25.121 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:25.121 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:25.121 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:25.122 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14301 2024-05-08T08:55:25.122 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.122 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:25.122 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.122 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:25.122 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645078+0000", 2024-05-08T08:55:25.122 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:25.122 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.122 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.122 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:25.122 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645078+0000", 2024-05-08T08:55:25.122 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:25.122 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.122 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.123 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:25.123 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645084+0000", 2024-05-08T08:55:25.123 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4620000000000001e-06 2024-05-08T08:55:25.123 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.123 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.123 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:25.123 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645088+0000", 2024-05-08T08:55:25.123 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8039999999999999e-06 2024-05-08T08:55:25.123 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.123 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.123 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:25.123 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645106+0000", 2024-05-08T08:55:25.123 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.8318999999999999e-05 2024-05-08T08:55:25.123 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.123 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.124 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:25.124 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645113+0000", 2024-05-08T08:55:25.124 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0310000000000001e-06 2024-05-08T08:55:25.124 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.124 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.124 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:25.124 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.670493+0000", 2024-05-08T08:55:25.124 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.025379993 2024-05-08T08:55:25.124 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.124 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.124 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:25.124 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.670571+0000", 2024-05-08T08:55:25.124 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7613000000000004e-05 2024-05-08T08:55:25.124 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.125 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.125 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:25.125 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733360+0000", 2024-05-08T08:55:25.125 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.062788630999999998 2024-05-08T08:55:25.125 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.125 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:25.125 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.125 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.125 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.125 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:14420 3.0 3:06a8948e:::rbd_data.114071bcf828.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:25.125 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.649639+0000", 2024-05-08T08:55:25.125 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 113.468445006, 2024-05-08T08:55:25.125 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.083851897999999994, 2024-05-08T08:55:25.125 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:25.125 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:25.126 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:55:25.126 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:25.126 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:25.126 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:25.126 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14420 2024-05-08T08:55:25.126 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.126 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:25.126 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.126 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:25.126 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649639+0000", 2024-05-08T08:55:25.126 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:25.126 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.126 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.126 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:25.127 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649639+0000", 2024-05-08T08:55:25.127 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:25.127 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.127 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.127 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:25.127 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649640+0000", 2024-05-08T08:55:25.127 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6899999999999997e-07 2024-05-08T08:55:25.127 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.127 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.127 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:25.127 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649895+0000", 2024-05-08T08:55:25.127 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00025531000000000002 2024-05-08T08:55:25.127 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.127 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.127 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:25.128 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649905+0000", 2024-05-08T08:55:25.128 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0198e-05 2024-05-08T08:55:25.128 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.128 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.128 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:25.128 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649911+0000", 2024-05-08T08:55:25.128 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2310000000000001e-06 2024-05-08T08:55:25.128 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.128 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.128 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:25.128 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.684099+0000", 2024-05-08T08:55:25.128 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.034188017000000001 2024-05-08T08:55:25.128 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.128 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.128 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:25.128 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.684172+0000", 2024-05-08T08:55:25.129 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3201999999999995e-05 2024-05-08T08:55:25.129 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.129 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.129 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 2", 2024-05-08T08:55:25.129 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.684418+0000", 2024-05-08T08:55:25.129 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00024559099999999997 2024-05-08T08:55:25.129 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.129 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.129 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:55:25.129 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.705538+0000", 2024-05-08T08:55:25.129 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.021119707000000001 2024-05-08T08:55:25.129 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.129 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.129 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:55:25.129 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733399+0000", 2024-05-08T08:55:25.130 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.027861543999999999 2024-05-08T08:55:25.130 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.130 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.130 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:55:25.130 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733445+0000", 2024-05-08T08:55:25.130 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5587000000000002e-05 2024-05-08T08:55:25.130 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.130 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.130 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:25.130 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733491+0000", 2024-05-08T08:55:25.130 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5841999999999999e-05 2024-05-08T08:55:25.130 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.130 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:25.131 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.131 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.131 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.131 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28548 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889189696] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:25.131 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.355669+0000", 2024-05-08T08:55:25.131 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 112.762414428, 2024-05-08T08:55:25.131 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078956606999999998, 2024-05-08T08:55:25.131 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:25.131 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:25.131 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:25.131 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:25.131 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:25.131 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:25.131 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28548 2024-05-08T08:55:25.131 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.132 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:25.132 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.132 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:25.132 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355669+0000", 2024-05-08T08:55:25.132 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:25.132 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.132 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.132 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:25.132 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355669+0000", 2024-05-08T08:55:25.132 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:25.132 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.132 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.132 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:25.132 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355762+0000", 2024-05-08T08:55:25.133 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3015000000000003e-05 2024-05-08T08:55:25.133 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.133 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.133 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:25.133 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355765+0000", 2024-05-08T08:55:25.133 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.368e-06 2024-05-08T08:55:25.133 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.133 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.133 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:25.133 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355771+0000", 2024-05-08T08:55:25.133 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5989999999999998e-06 2024-05-08T08:55:25.133 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.133 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.133 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:25.133 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355775+0000", 2024-05-08T08:55:25.134 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7280000000000001e-06 2024-05-08T08:55:25.134 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.134 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.134 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:25.134 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434379+0000", 2024-05-08T08:55:25.134 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078603662000000005 2024-05-08T08:55:25.134 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.134 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.134 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:25.134 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434473+0000", 2024-05-08T08:55:25.134 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4104000000000006e-05 2024-05-08T08:55:25.134 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.134 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.135 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:25.135 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434626+0000", 2024-05-08T08:55:25.135 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00015313100000000001 2024-05-08T08:55:25.135 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.135 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:25.135 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.135 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.135 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.135 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28549 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889202400] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:25.135 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.355781+0000", 2024-05-08T08:55:25.135 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 112.762302854, 2024-05-08T08:55:25.135 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079117946999999994, 2024-05-08T08:55:25.135 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:25.135 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:25.135 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:25.136 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:25.136 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:25.136 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:25.136 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28549 2024-05-08T08:55:25.136 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.136 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:25.136 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.136 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:25.136 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355781+0000", 2024-05-08T08:55:25.136 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:25.136 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.136 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.136 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:25.136 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355781+0000", 2024-05-08T08:55:25.136 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:25.137 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.137 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.137 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:25.137 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356162+0000", 2024-05-08T08:55:25.137 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00038099100000000001 2024-05-08T08:55:25.137 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.137 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.137 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:25.137 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356165+0000", 2024-05-08T08:55:25.137 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1240000000000001e-06 2024-05-08T08:55:25.137 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.137 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.137 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:25.137 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356173+0000", 2024-05-08T08:55:25.137 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.6199999999999999e-06 2024-05-08T08:55:25.138 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.138 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.138 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:25.138 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356182+0000", 2024-05-08T08:55:25.138 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7760000000000003e-06 2024-05-08T08:55:25.138 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.138 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.138 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:25.138 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434675+0000", 2024-05-08T08:55:25.138 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078493119 2024-05-08T08:55:25.138 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.138 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.138 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:25.138 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434765+0000", 2024-05-08T08:55:25.139 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0402000000000006e-05 2024-05-08T08:55:25.139 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.139 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.139 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:25.139 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434899+0000", 2024-05-08T08:55:25.139 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013391499999999999 2024-05-08T08:55:25.139 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.139 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:25.139 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.139 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.139 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.139 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28550 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888597280] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:25.139 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:55:25.139 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 112.761891808, 2024-05-08T08:55:25.140 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078933990999999995, 2024-05-08T08:55:25.140 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:25.140 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:25.140 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:25.140 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:25.140 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:25.140 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:25.140 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28550 2024-05-08T08:55:25.140 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.140 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:25.140 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.140 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:25.140 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:55:25.140 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:25.140 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.140 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.141 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:25.141 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:55:25.141 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:25.141 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.141 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.141 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:25.141 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:55:25.141 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.34e-07 2024-05-08T08:55:25.141 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.141 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.141 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:25.141 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356195+0000", 2024-05-08T08:55:25.141 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8710000000000001e-06 2024-05-08T08:55:25.141 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.142 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.142 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:25.142 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356200+0000", 2024-05-08T08:55:25.142 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3000000000000003e-06 2024-05-08T08:55:25.142 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.142 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.142 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:25.142 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356203+0000", 2024-05-08T08:55:25.142 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4039999999999999e-06 2024-05-08T08:55:25.142 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.142 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.142 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:25.142 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434928+0000", 2024-05-08T08:55:25.142 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078725331999999995 2024-05-08T08:55:25.142 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.143 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.143 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:25.143 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435007+0000", 2024-05-08T08:55:25.143 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8415000000000001e-05 2024-05-08T08:55:25.143 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.143 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.143 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:25.143 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435126+0000", 2024-05-08T08:55:25.143 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011923500000000001 2024-05-08T08:55:25.143 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.143 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:25.143 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.143 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.143 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.144 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28551 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888527200] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:25.144 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.356210+0000", 2024-05-08T08:55:25.144 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 112.76187346, 2024-05-08T08:55:25.144 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079195072000000005, 2024-05-08T08:55:25.144 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:25.144 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:25.144 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:25.144 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:25.144 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:25.144 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:25.144 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28551 2024-05-08T08:55:25.144 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.144 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:25.144 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.145 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:25.145 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356210+0000", 2024-05-08T08:55:25.145 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:25.145 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.145 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.145 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:25.145 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356210+0000", 2024-05-08T08:55:25.145 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:25.145 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.145 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.145 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:25.145 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356211+0000", 2024-05-08T08:55:25.145 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4499999999999998e-07 2024-05-08T08:55:25.145 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.145 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.146 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:25.146 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356212+0000", 2024-05-08T08:55:25.146 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6029999999999999e-06 2024-05-08T08:55:25.146 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.146 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.146 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:25.146 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356221+0000", 2024-05-08T08:55:25.146 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4200000000000007e-06 2024-05-08T08:55:25.146 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.146 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.146 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:25.146 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356224+0000", 2024-05-08T08:55:25.147 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.038e-06 2024-05-08T08:55:25.147 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.147 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.147 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:25.147 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435178+0000", 2024-05-08T08:55:25.147 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078954625000000001 2024-05-08T08:55:25.147 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.147 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.147 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:25.147 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435256+0000", 2024-05-08T08:55:25.147 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7337999999999997e-05 2024-05-08T08:55:25.147 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.147 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.147 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:25.147 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435405+0000", 2024-05-08T08:55:25.148 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000149703 2024-05-08T08:55:25.148 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.148 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:25.148 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.148 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.148 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.148 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28552 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889231456] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:25.148 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.356228+0000", 2024-05-08T08:55:25.148 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 112.761855782, 2024-05-08T08:55:25.148 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079346920000000001, 2024-05-08T08:55:25.148 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:25.148 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:25.148 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:25.148 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:25.148 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:25.149 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:25.149 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28552 2024-05-08T08:55:25.149 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.149 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:25.149 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.149 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:25.149 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356228+0000", 2024-05-08T08:55:25.149 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:25.149 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.149 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.149 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:25.149 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356228+0000", 2024-05-08T08:55:25.149 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:25.149 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.150 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.150 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:25.150 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356959+0000", 2024-05-08T08:55:25.150 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00073097099999999996 2024-05-08T08:55:25.150 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.150 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.150 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:25.150 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356967+0000", 2024-05-08T08:55:25.150 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0409999999999998e-06 2024-05-08T08:55:25.150 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.150 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.150 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:25.150 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356979+0000", 2024-05-08T08:55:25.150 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1489e-05 2024-05-08T08:55:25.151 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.151 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.151 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:25.151 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356986+0000", 2024-05-08T08:55:25.151 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3699999999999997e-06 2024-05-08T08:55:25.151 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.151 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.151 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:25.151 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435433+0000", 2024-05-08T08:55:25.151 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078447520000000007 2024-05-08T08:55:25.151 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.151 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.151 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:25.151 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435484+0000", 2024-05-08T08:55:25.152 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0229000000000003e-05 2024-05-08T08:55:25.152 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.152 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.152 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:25.152 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435575+0000", 2024-05-08T08:55:25.152 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1299999999999997e-05 2024-05-08T08:55:25.152 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.152 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:25.152 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.152 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.152 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.152 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28553 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888713648] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:25.152 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:55:25.152 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 112.761081124, 2024-05-08T08:55:25.152 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078782034000000001, 2024-05-08T08:55:25.153 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:25.153 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:25.153 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:25.153 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:25.153 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:25.153 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:25.153 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28553 2024-05-08T08:55:25.153 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.153 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:25.153 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.153 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:25.153 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:55:25.153 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:25.153 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.153 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.154 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:25.154 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:55:25.154 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:25.154 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.154 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.154 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:25.154 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:55:25.154 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6600000000000002e-07 2024-05-08T08:55:25.154 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.154 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.154 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:25.154 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357006+0000", 2024-05-08T08:55:25.154 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7190000000000001e-06 2024-05-08T08:55:25.154 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.155 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.155 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:25.155 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357013+0000", 2024-05-08T08:55:25.155 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.435e-06 2024-05-08T08:55:25.155 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.155 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.155 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:25.155 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357018+0000", 2024-05-08T08:55:25.155 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7260000000000002e-06 2024-05-08T08:55:25.155 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.155 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.155 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:25.155 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435623+0000", 2024-05-08T08:55:25.155 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078604972999999995 2024-05-08T08:55:25.156 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.156 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.156 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:25.156 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435695+0000", 2024-05-08T08:55:25.156 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1959999999999995e-05 2024-05-08T08:55:25.156 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.156 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.156 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:25.156 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435785+0000", 2024-05-08T08:55:25.156 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.9754999999999997e-05 2024-05-08T08:55:25.156 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.156 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:25.156 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.156 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.157 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.157 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28554 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888980272] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:25.157 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357024+0000", 2024-05-08T08:55:25.157 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 112.761059488, 2024-05-08T08:55:25.157 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078999870999999999, 2024-05-08T08:55:25.157 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:25.157 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:25.157 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:25.157 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:25.157 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:25.157 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:25.157 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28554 2024-05-08T08:55:25.157 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.157 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:25.158 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.158 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:25.158 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357024+0000", 2024-05-08T08:55:25.158 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:25.158 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.158 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.158 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:25.158 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357024+0000", 2024-05-08T08:55:25.158 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:25.158 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.158 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.158 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:25.158 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357025+0000", 2024-05-08T08:55:25.158 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:55:25.159 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.159 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.159 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:25.159 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357027+0000", 2024-05-08T08:55:25.159 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2050000000000001e-06 2024-05-08T08:55:25.159 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.159 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.159 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:25.159 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357033+0000", 2024-05-08T08:55:25.159 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5679999999999996e-06 2024-05-08T08:55:25.159 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.159 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.159 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:25.159 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357038+0000", 2024-05-08T08:55:25.159 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2889999999999999e-06 2024-05-08T08:55:25.160 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.160 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.160 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:25.160 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435823+0000", 2024-05-08T08:55:25.160 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078785677999999998 2024-05-08T08:55:25.160 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.160 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.160 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:25.160 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435892+0000", 2024-05-08T08:55:25.160 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8379000000000004e-05 2024-05-08T08:55:25.160 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.160 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.160 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:25.160 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436024+0000", 2024-05-08T08:55:25.160 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013239799999999999 2024-05-08T08:55:25.161 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.161 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:25.161 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.161 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.161 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.161 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28555 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889001104] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:25.161 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:55:25.161 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 112.76104072299999, 2024-05-08T08:55:25.161 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079115742000000003, 2024-05-08T08:55:25.161 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:25.161 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:25.161 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:25.161 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:25.161 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:25.162 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:25.162 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28555 2024-05-08T08:55:25.162 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.162 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:25.162 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.162 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:25.162 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:55:25.162 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:25.162 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.162 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.162 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:25.162 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:55:25.162 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:25.162 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.162 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.163 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:25.163 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:55:25.163 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9200000000000002e-07 2024-05-08T08:55:25.163 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.163 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.163 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:25.163 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357046+0000", 2024-05-08T08:55:25.163 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1670000000000002e-06 2024-05-08T08:55:25.163 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.163 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.163 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:25.163 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357052+0000", 2024-05-08T08:55:25.163 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.139e-06 2024-05-08T08:55:25.163 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.163 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.164 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:25.164 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357056+0000", 2024-05-08T08:55:25.164 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2440000000000004e-06 2024-05-08T08:55:25.164 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.164 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.164 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:25.164 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436059+0000", 2024-05-08T08:55:25.164 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079003070999999994 2024-05-08T08:55:25.164 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.164 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.164 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:25.164 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436097+0000", 2024-05-08T08:55:25.164 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7784000000000003e-05 2024-05-08T08:55:25.164 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.164 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.165 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:25.165 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436159+0000", 2024-05-08T08:55:25.165 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2045000000000007e-05 2024-05-08T08:55:25.165 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.165 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:25.165 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.165 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.165 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.165 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28556 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889022064] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:25.165 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357065+0000", 2024-05-08T08:55:25.165 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 112.76101869, 2024-05-08T08:55:25.165 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079327976999999994, 2024-05-08T08:55:25.165 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:25.165 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:25.165 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:25.166 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:25.166 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:25.166 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:25.166 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28556 2024-05-08T08:55:25.166 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.166 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:25.166 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.166 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:25.166 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357065+0000", 2024-05-08T08:55:25.166 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:25.166 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.166 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.166 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:25.166 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357065+0000", 2024-05-08T08:55:25.167 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:25.167 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.167 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.167 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:25.167 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358958+0000", 2024-05-08T08:55:25.167 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001892915 2024-05-08T08:55:25.167 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.167 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.167 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:25.167 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358962+0000", 2024-05-08T08:55:25.167 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4599999999999999e-06 2024-05-08T08:55:25.167 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.167 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.167 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:25.167 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358970+0000", 2024-05-08T08:55:25.168 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.6090000000000001e-06 2024-05-08T08:55:25.168 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.168 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.168 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:25.168 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358975+0000", 2024-05-08T08:55:25.168 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7639999999999996e-06 2024-05-08T08:55:25.168 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.168 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.168 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:25.168 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436196+0000", 2024-05-08T08:55:25.168 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.077221564000000006 2024-05-08T08:55:25.168 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.168 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.168 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:25.168 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436269+0000", 2024-05-08T08:55:25.169 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2720999999999997e-05 2024-05-08T08:55:25.169 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.169 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.169 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:25.169 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436393+0000", 2024-05-08T08:55:25.169 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012394399999999999 2024-05-08T08:55:25.169 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.169 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:25.169 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.169 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.169 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.169 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28563 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888855520] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:25.169 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:55:25.169 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 112.759009291, 2024-05-08T08:55:25.170 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078795566999999997, 2024-05-08T08:55:25.170 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:25.170 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:25.170 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:25.170 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:25.170 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:25.170 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:25.170 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28563 2024-05-08T08:55:25.170 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.170 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:25.170 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.170 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:25.170 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:55:25.171 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:25.171 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.171 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.171 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:25.171 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:55:25.171 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:25.171 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.171 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.171 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:25.171 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:55:25.171 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1e-07 2024-05-08T08:55:25.171 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.171 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.171 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:25.172 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359080+0000", 2024-05-08T08:55:25.172 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8940000000000001e-06 2024-05-08T08:55:25.172 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.172 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.172 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:25.172 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359083+0000", 2024-05-08T08:55:25.172 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1949999999999999e-06 2024-05-08T08:55:25.172 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.172 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.172 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:25.172 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359085+0000", 2024-05-08T08:55:25.172 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4710000000000001e-06 2024-05-08T08:55:25.172 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.172 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.173 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:25.173 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437604+0000", 2024-05-08T08:55:25.173 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078518278999999996 2024-05-08T08:55:25.173 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.173 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.173 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:25.173 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437704+0000", 2024-05-08T08:55:25.173 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010010599999999999 2024-05-08T08:55:25.173 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.173 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.173 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:25.173 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437870+0000", 2024-05-08T08:55:25.173 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00016641200000000001 2024-05-08T08:55:25.173 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.173 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:25.174 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.174 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.174 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.174 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28564 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888538944] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:25.174 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.359089+0000", 2024-05-08T08:55:25.174 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 112.75899507600001, 2024-05-08T08:55:25.174 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078952675, 2024-05-08T08:55:25.174 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:25.174 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:25.174 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:25.174 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:25.174 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:25.174 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:25.174 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28564 2024-05-08T08:55:25.175 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.175 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:25.175 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.175 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:25.175 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359089+0000", 2024-05-08T08:55:25.175 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:25.175 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.175 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.175 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:25.175 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359089+0000", 2024-05-08T08:55:25.175 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:25.175 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.175 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.175 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:25.175 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360959+0000", 2024-05-08T08:55:25.176 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0018707089999999999 2024-05-08T08:55:25.176 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.176 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.176 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:25.176 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360963+0000", 2024-05-08T08:55:25.176 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0680000000000001e-06 2024-05-08T08:55:25.176 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.176 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.176 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:25.176 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360969+0000", 2024-05-08T08:55:25.176 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1920000000000002e-06 2024-05-08T08:55:25.176 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.176 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.176 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:25.176 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360973+0000", 2024-05-08T08:55:25.177 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.031e-06 2024-05-08T08:55:25.177 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.177 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.177 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:25.177 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437916+0000", 2024-05-08T08:55:25.177 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.076942939000000002 2024-05-08T08:55:25.177 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.177 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.177 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:25.177 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437973+0000", 2024-05-08T08:55:25.177 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7358999999999997e-05 2024-05-08T08:55:25.177 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.177 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.177 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:25.178 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.438041+0000", 2024-05-08T08:55:25.178 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8376999999999999e-05 2024-05-08T08:55:25.178 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.178 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:25.178 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.178 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.178 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.178 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28574 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889427600] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:25.178 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:55:25.178 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 112.756990569, 2024-05-08T08:55:25.178 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078782476000000004, 2024-05-08T08:55:25.178 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:25.178 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:25.178 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:25.179 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:25.179 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:25.179 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:25.179 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28574 2024-05-08T08:55:25.179 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.179 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:25.179 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.179 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:25.179 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:55:25.179 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:25.179 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.179 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.179 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:25.179 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:55:25.179 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:25.179 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.180 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.180 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:25.180 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:55:25.180 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1799999999999999e-07 2024-05-08T08:55:25.180 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.180 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.180 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:25.180 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361095+0000", 2024-05-08T08:55:25.180 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3850000000000001e-06 2024-05-08T08:55:25.180 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.180 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.180 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:25.180 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361098+0000", 2024-05-08T08:55:25.180 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7989999999999998e-06 2024-05-08T08:55:25.180 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.181 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.181 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:25.181 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361100+0000", 2024-05-08T08:55:25.181 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4669999999999998e-06 2024-05-08T08:55:25.181 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.181 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.181 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:25.181 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439768+0000", 2024-05-08T08:55:25.181 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078668057999999999 2024-05-08T08:55:25.181 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.181 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.181 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:25.181 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439811+0000", 2024-05-08T08:55:25.181 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2459000000000001e-05 2024-05-08T08:55:25.182 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.182 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.182 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:25.182 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439876+0000", 2024-05-08T08:55:25.182 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5090000000000002e-05 2024-05-08T08:55:25.182 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.182 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:25.182 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.182 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.182 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.182 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28575 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889439216] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:25.182 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361103+0000", 2024-05-08T08:55:25.182 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 112.75698055399999, 2024-05-08T08:55:25.182 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078966828000000003, 2024-05-08T08:55:25.182 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:25.183 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:25.183 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:25.183 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:25.183 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:25.183 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:25.183 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28575 2024-05-08T08:55:25.183 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.183 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:25.183 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.183 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:25.183 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361103+0000", 2024-05-08T08:55:25.183 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:25.183 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.183 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.183 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:25.184 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361103+0000", 2024-05-08T08:55:25.184 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:25.184 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.184 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.184 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:25.184 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361104+0000", 2024-05-08T08:55:25.184 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1400000000000001e-07 2024-05-08T08:55:25.184 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.184 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.184 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:25.184 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361105+0000", 2024-05-08T08:55:25.184 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5540000000000001e-06 2024-05-08T08:55:25.184 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.184 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.184 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:25.185 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361108+0000", 2024-05-08T08:55:25.185 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.841e-06 2024-05-08T08:55:25.185 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.185 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.185 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:25.185 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361110+0000", 2024-05-08T08:55:25.185 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.497e-06 2024-05-08T08:55:25.185 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.185 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.185 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:25.185 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439916+0000", 2024-05-08T08:55:25.185 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078805372999999998 2024-05-08T08:55:25.185 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.185 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.186 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:25.186 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440000+0000", 2024-05-08T08:55:25.186 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4730999999999999e-05 2024-05-08T08:55:25.186 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.186 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.186 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:25.186 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440070+0000", 2024-05-08T08:55:25.186 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9617999999999997e-05 2024-05-08T08:55:25.186 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.186 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:25.186 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.186 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.186 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.186 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28576 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889451328] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:25.186 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361113+0000", 2024-05-08T08:55:25.187 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 112.756970368, 2024-05-08T08:55:25.187 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079088862999999995, 2024-05-08T08:55:25.187 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:25.187 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:25.187 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:25.187 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:25.187 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:25.187 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:25.187 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28576 2024-05-08T08:55:25.187 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.187 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:25.187 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.187 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:25.187 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361113+0000", 2024-05-08T08:55:25.187 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:25.188 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.188 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.188 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:25.188 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361113+0000", 2024-05-08T08:55:25.188 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:25.188 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.188 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.188 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:25.188 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361114+0000", 2024-05-08T08:55:25.188 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1299999999999999e-07 2024-05-08T08:55:25.188 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.188 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.188 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:25.188 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361115+0000", 2024-05-08T08:55:25.188 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.4300000000000001e-06 2024-05-08T08:55:25.189 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.189 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.189 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:25.189 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361118+0000", 2024-05-08T08:55:25.189 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9000000000000002e-06 2024-05-08T08:55:25.189 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.189 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.189 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:25.189 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361120+0000", 2024-05-08T08:55:25.189 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4739999999999999e-06 2024-05-08T08:55:25.189 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.189 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.189 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:25.189 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440096+0000", 2024-05-08T08:55:25.190 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078975629000000006 2024-05-08T08:55:25.190 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.190 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.190 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:25.190 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440135+0000", 2024-05-08T08:55:25.190 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9202000000000002e-05 2024-05-08T08:55:25.190 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.190 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.190 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:25.190 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440202+0000", 2024-05-08T08:55:25.190 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7014999999999994e-05 2024-05-08T08:55:25.190 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.190 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:25.190 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.190 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.191 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.191 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28577 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889462640] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:25.191 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:55:25.191 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 112.756960186, 2024-05-08T08:55:25.191 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079194622000000006, 2024-05-08T08:55:25.191 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:25.191 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:25.191 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:25.191 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:25.191 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:25.191 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:25.191 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28577 2024-05-08T08:55:25.191 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.191 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:25.191 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.192 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:25.192 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:55:25.192 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:25.192 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.192 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.192 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:25.192 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:55:25.192 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:25.192 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.192 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.192 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:25.192 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:55:25.192 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1400000000000001e-07 2024-05-08T08:55:25.192 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.192 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.193 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:25.193 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361125+0000", 2024-05-08T08:55:25.193 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.372e-06 2024-05-08T08:55:25.193 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.193 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.193 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:25.193 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361128+0000", 2024-05-08T08:55:25.193 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0309999999999999e-06 2024-05-08T08:55:25.193 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.193 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.193 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:25.193 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361131+0000", 2024-05-08T08:55:25.193 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4040000000000002e-06 2024-05-08T08:55:25.193 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.194 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.194 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:25.194 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440225+0000", 2024-05-08T08:55:25.194 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079094600000000001 2024-05-08T08:55:25.194 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.194 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.194 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:25.194 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440260+0000", 2024-05-08T08:55:25.194 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4703000000000003e-05 2024-05-08T08:55:25.194 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.194 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.194 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:25.194 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440318+0000", 2024-05-08T08:55:25.194 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8298000000000003e-05 2024-05-08T08:55:25.194 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.195 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:25.195 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.195 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.195 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.195 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28578 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889474192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:25.195 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361134+0000", 2024-05-08T08:55:25.195 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 112.756949761, 2024-05-08T08:55:25.195 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079310295000000003, 2024-05-08T08:55:25.195 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:25.195 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:25.195 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:25.195 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:25.195 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:25.195 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:25.195 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28578 2024-05-08T08:55:25.195 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.196 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:25.196 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.196 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:25.196 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361134+0000", 2024-05-08T08:55:25.196 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:25.196 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.196 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.196 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:25.196 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361134+0000", 2024-05-08T08:55:25.196 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:25.196 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.196 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.196 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:25.196 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.362958+0000", 2024-05-08T08:55:25.196 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0018237380000000001 2024-05-08T08:55:25.196 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.197 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.197 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:25.197 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.362961+0000", 2024-05-08T08:55:25.197 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.123e-06 2024-05-08T08:55:25.197 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.197 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.197 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:25.197 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.397404+0000", 2024-05-08T08:55:25.197 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.034443226 2024-05-08T08:55:25.197 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.197 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.197 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:25.197 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.397439+0000", 2024-05-08T08:55:25.197 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4690000000000002e-05 2024-05-08T08:55:25.197 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.198 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.198 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:25.198 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440340+0000", 2024-05-08T08:55:25.198 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.042901419000000003 2024-05-08T08:55:25.198 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.198 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.198 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:25.198 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440376+0000", 2024-05-08T08:55:25.198 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5444999999999997e-05 2024-05-08T08:55:25.198 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.198 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.198 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:25.198 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440444+0000", 2024-05-08T08:55:25.198 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8653999999999997e-05 2024-05-08T08:55:25.199 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.199 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:25.199 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.199 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.199 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.199 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:34133 3.10 3:0cc147c9:::rbd_data.1140100c9703.0000000000000035:head [delete] snapc 14=[] ondisk+write+known_if_redirected+supports_pool_eio e164)", 2024-05-08T08:55:25.199 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:55:15.461161+0000", 2024-05-08T08:55:25.199 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 9.6569224239999993, 2024-05-08T08:55:25.199 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.92225079300000001, 2024-05-08T08:55:25.199 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:25.199 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:25.199 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:55:25.199 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:25.199 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:55:25.199 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:55:25.200 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 34133 2024-05-08T08:55:25.200 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.200 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:25.200 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.200 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:25.200 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:15.461161+0000", 2024-05-08T08:55:25.200 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:25.200 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.200 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.200 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:25.200 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:15.461161+0000", 2024-05-08T08:55:25.200 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:25.200 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.200 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.200 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:25.201 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:15.461162+0000", 2024-05-08T08:55:25.201 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0200000000000001e-07 2024-05-08T08:55:25.201 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.201 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.201 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:25.201 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:15.461166+0000", 2024-05-08T08:55:25.201 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2289999999999997e-06 2024-05-08T08:55:25.201 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.201 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.201 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:25.201 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:15.461175+0000", 2024-05-08T08:55:25.201 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0070000000000003e-06 2024-05-08T08:55:25.201 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.201 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.201 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:25.202 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:15.461182+0000", 2024-05-08T08:55:25.202 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5209999999999999e-06 2024-05-08T08:55:25.202 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.202 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.202 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:25.202 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:15.461248+0000", 2024-05-08T08:55:25.202 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5730999999999998e-05 2024-05-08T08:55:25.202 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.202 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.202 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:55:25.202 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:15.461259+0000", 2024-05-08T08:55:25.202 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0812000000000001e-05 2024-05-08T08:55:25.202 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.202 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.203 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:25.203 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:16.376737+0000", 2024-05-08T08:55:25.203 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.91547897199999995 2024-05-08T08:55:25.203 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.203 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.203 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:25.203 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:16.376832+0000", 2024-05-08T08:55:25.203 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4575999999999997e-05 2024-05-08T08:55:25.203 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.203 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.203 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 2", 2024-05-08T08:55:25.203 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:16.376905+0000", 2024-05-08T08:55:25.203 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3208999999999997e-05 2024-05-08T08:55:25.203 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.203 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.203 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:55:25.203 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:16.378405+0000", 2024-05-08T08:55:25.204 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0014998660000000001 2024-05-08T08:55:25.204 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.204 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.204 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:55:25.204 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:16.383294+0000", 2024-05-08T08:55:25.204 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0048889600000000004 2024-05-08T08:55:25.204 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.204 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.204 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:55:25.204 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:16.383366+0000", 2024-05-08T08:55:25.204 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1428999999999997e-05 2024-05-08T08:55:25.204 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.204 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.204 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:25.204 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:16.383412+0000", 2024-05-08T08:55:25.205 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6678999999999998e-05 2024-05-08T08:55:25.205 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.205 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:25.205 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.205 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.205 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:25.205 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:55:25.206 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:55:25.207 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:55:25.208+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:55:25.307 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:55:25.310+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:55:25.357 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:55:25.357 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:55:25.357 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:55:25.357 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:55:25.367 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:55:25.408 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:55:25.409+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:55:25.508 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:55:25.511+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:55:25.522 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:55:25.522 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:55:25.523 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:55:25.523 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:55:25.523 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:55:25.532 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:55:25.617 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:55:25.620+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:55:25.684 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:55:25.685 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:55:25.685 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:55:25.685 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:55:25.685 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.685 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:437 3.17 3:ea93a512:::rbd_data.1140100c9703.0000000000000077:head [set-alloc-hint object_size 2097152 write_size 2097152,write 1170432~1024] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e63)", 2024-05-08T08:55:25.685 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:08.390804+0000", 2024-05-08T08:55:25.685 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 137.29549637599999, 2024-05-08T08:55:25.685 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0406068690000001, 2024-05-08T08:55:25.685 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:25.685 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:25.685 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:55:25.685 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:25.685 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:25.686 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:25.686 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 437 2024-05-08T08:55:25.686 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.686 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:25.686 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.686 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:25.686 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390804+0000", 2024-05-08T08:55:25.686 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:25.686 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.686 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.686 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:25.686 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390804+0000", 2024-05-08T08:55:25.686 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:25.686 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.687 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.687 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:25.687 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390806+0000", 2024-05-08T08:55:25.687 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.666e-06 2024-05-08T08:55:25.687 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.687 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.687 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:25.687 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390818+0000", 2024-05-08T08:55:25.687 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2187999999999999e-05 2024-05-08T08:55:25.687 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.687 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.687 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:25.687 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390834+0000", 2024-05-08T08:55:25.687 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6191000000000001e-05 2024-05-08T08:55:25.687 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.688 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.688 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:25.688 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390845+0000", 2024-05-08T08:55:25.688 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0831e-05 2024-05-08T08:55:25.688 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.688 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.688 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:25.688 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390922+0000", 2024-05-08T08:55:25.688 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7253e-05 2024-05-08T08:55:25.688 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.688 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.688 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op must wait for map", 2024-05-08T08:55:25.688 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390930+0000", 2024-05-08T08:55:25.688 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8259999999999993e-06 2024-05-08T08:55:25.688 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.689 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.689 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:25.689 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392155+0000", 2024-05-08T08:55:25.689 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0012256649999999999 2024-05-08T08:55:25.689 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.689 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.689 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:55:25.689 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392165+0000", 2024-05-08T08:55:25.689 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5799999999999998e-06 2024-05-08T08:55:25.689 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.689 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.689 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:25.689 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.429596+0000", 2024-05-08T08:55:25.689 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0374313639999999 2024-05-08T08:55:25.689 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.690 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.690 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:25.690 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.429728+0000", 2024-05-08T08:55:25.690 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000131552 2024-05-08T08:55:25.690 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.690 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.690 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 0,1", 2024-05-08T08:55:25.690 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.429860+0000", 2024-05-08T08:55:25.690 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000132137 2024-05-08T08:55:25.690 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.690 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.690 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:55:25.690 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.430811+0000", 2024-05-08T08:55:25.690 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00095040200000000004 2024-05-08T08:55:25.691 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.691 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.691 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:55:25.691 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431126+0000", 2024-05-08T08:55:25.691 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000315006 2024-05-08T08:55:25.691 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.691 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.691 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:55:25.691 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431272+0000", 2024-05-08T08:55:25.691 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000146487 2024-05-08T08:55:25.691 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.691 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.691 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:55:25.691 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431315+0000", 2024-05-08T08:55:25.691 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2697000000000001e-05 2024-05-08T08:55:25.691 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.692 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.692 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:25.692 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431411+0000", 2024-05-08T08:55:25.692 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6024000000000001e-05 2024-05-08T08:55:25.692 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.692 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:25.692 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.692 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.692 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.692 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:439 3.17 3:ea93a512:::rbd_data.1140100c9703.0000000000000077:head [read 1163264~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-05-08T08:55:25.692 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:08.390871+0000", 2024-05-08T08:55:25.692 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 137.29542886499999, 2024-05-08T08:55:25.692 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0408790590000001, 2024-05-08T08:55:25.692 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:25.692 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:25.693 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:25.693 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:25.693 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:25.693 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:25.693 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 439 2024-05-08T08:55:25.693 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.693 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:25.693 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.693 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:25.693 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390871+0000", 2024-05-08T08:55:25.693 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:25.693 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.693 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.693 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:25.693 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390871+0000", 2024-05-08T08:55:25.693 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:25.694 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.694 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.694 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:25.694 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390872+0000", 2024-05-08T08:55:25.694 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6300000000000005e-07 2024-05-08T08:55:25.694 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.694 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.694 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:25.694 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390875+0000", 2024-05-08T08:55:25.694 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3459999999999998e-06 2024-05-08T08:55:25.694 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.694 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.694 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:25.694 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390891+0000", 2024-05-08T08:55:25.695 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5255e-05 2024-05-08T08:55:25.695 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.695 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.695 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:25.695 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390897+0000", 2024-05-08T08:55:25.695 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2360000000000001e-06 2024-05-08T08:55:25.695 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.695 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.695 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:25.695 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390995+0000", 2024-05-08T08:55:25.695 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8522000000000003e-05 2024-05-08T08:55:25.695 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.695 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.695 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting_for_map not empty", 2024-05-08T08:55:25.695 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.391005+0000", 2024-05-08T08:55:25.696 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1460000000000002e-06 2024-05-08T08:55:25.696 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.696 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.696 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:25.696 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392191+0000", 2024-05-08T08:55:25.696 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0011860270000000001 2024-05-08T08:55:25.696 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.696 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.696 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:55:25.696 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392197+0000", 2024-05-08T08:55:25.696 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4289999999999998e-06 2024-05-08T08:55:25.696 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.696 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.696 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:25.696 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.430596+0000", 2024-05-08T08:55:25.696 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0383989090000001 2024-05-08T08:55:25.697 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.697 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.697 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for rw locks", 2024-05-08T08:55:25.697 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.430676+0000", 2024-05-08T08:55:25.697 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9781000000000001e-05 2024-05-08T08:55:25.697 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.697 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.697 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:25.697 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431460+0000", 2024-05-08T08:55:25.697 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00078427300000000004 2024-05-08T08:55:25.697 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.697 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.697 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:25.698 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431540+0000", 2024-05-08T08:55:25.698 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0224999999999998e-05 2024-05-08T08:55:25.698 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.698 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.698 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:25.698 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431750+0000", 2024-05-08T08:55:25.698 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00021024699999999999 2024-05-08T08:55:25.698 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.698 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:25.698 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.698 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.698 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.698 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:965 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 950272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:25.698 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451075+0000", 2024-05-08T08:55:25.699 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 136.235225063, 2024-05-08T08:55:25.699 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087230196999999995, 2024-05-08T08:55:25.699 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:25.699 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:25.699 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:25.699 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:25.699 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:25.699 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:25.699 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 965 2024-05-08T08:55:25.699 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.699 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:25.699 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.699 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:25.699 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451075+0000", 2024-05-08T08:55:25.699 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:25.700 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.700 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.700 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:25.700 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451075+0000", 2024-05-08T08:55:25.700 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:25.700 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.700 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.700 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:25.700 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451076+0000", 2024-05-08T08:55:25.700 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5000000000000002e-07 2024-05-08T08:55:25.700 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.700 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.700 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:25.700 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451078+0000", 2024-05-08T08:55:25.700 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.5639999999999999e-06 2024-05-08T08:55:25.701 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.701 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.701 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:25.701 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451085+0000", 2024-05-08T08:55:25.701 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5640000000000002e-06 2024-05-08T08:55:25.701 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.701 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.701 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:25.701 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451090+0000", 2024-05-08T08:55:25.701 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1699999999999996e-06 2024-05-08T08:55:25.701 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.701 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.701 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:25.701 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538106+0000", 2024-05-08T08:55:25.701 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087015888999999999 2024-05-08T08:55:25.702 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.702 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.702 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:25.702 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538190+0000", 2024-05-08T08:55:25.702 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3961999999999993e-05 2024-05-08T08:55:25.702 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.702 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.702 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:25.702 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538305+0000", 2024-05-08T08:55:25.702 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011529800000000001 2024-05-08T08:55:25.702 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.702 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:25.702 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.702 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.702 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.703 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:966 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 1441792~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:25.703 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:55:25.703 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 136.23520169, 2024-05-08T08:55:25.703 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087446323000000006, 2024-05-08T08:55:25.703 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:25.703 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:25.703 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:25.703 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:25.703 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:25.703 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:25.703 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 966 2024-05-08T08:55:25.703 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.703 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:25.703 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.703 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:25.704 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:55:25.704 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:25.704 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.704 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.704 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:25.704 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:55:25.704 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:25.704 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.704 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.704 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:25.704 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:55:25.704 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3700000000000001e-07 2024-05-08T08:55:25.704 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.704 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.704 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:25.705 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451101+0000", 2024-05-08T08:55:25.705 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2560000000000001e-06 2024-05-08T08:55:25.705 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.705 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.705 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:25.705 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451107+0000", 2024-05-08T08:55:25.705 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1070000000000001e-06 2024-05-08T08:55:25.705 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.705 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.705 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:25.705 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451112+0000", 2024-05-08T08:55:25.705 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1420000000000001e-06 2024-05-08T08:55:25.705 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.705 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.705 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:25.705 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538341+0000", 2024-05-08T08:55:25.706 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087228302999999993 2024-05-08T08:55:25.706 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.706 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.706 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:25.706 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538424+0000", 2024-05-08T08:55:25.706 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3497000000000004e-05 2024-05-08T08:55:25.706 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.706 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.706 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:25.706 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538545+0000", 2024-05-08T08:55:25.706 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000120581 2024-05-08T08:55:25.706 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.706 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:25.706 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.707 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.707 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.707 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:967 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:25.707 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451120+0000", 2024-05-08T08:55:25.707 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 136.23518021500001, 2024-05-08T08:55:25.707 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087632725999999994, 2024-05-08T08:55:25.707 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:25.707 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:25.707 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:25.707 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:25.707 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:25.707 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:25.707 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 967 2024-05-08T08:55:25.707 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.707 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:25.707 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.708 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:25.708 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451120+0000", 2024-05-08T08:55:25.708 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:25.708 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.708 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.708 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:25.708 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451120+0000", 2024-05-08T08:55:25.708 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:25.708 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.708 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.708 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:25.708 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451121+0000", 2024-05-08T08:55:25.708 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8800000000000002e-07 2024-05-08T08:55:25.708 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.708 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.709 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:25.709 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451130+0000", 2024-05-08T08:55:25.709 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2299999999999997e-06 2024-05-08T08:55:25.709 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.709 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.709 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:25.709 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451137+0000", 2024-05-08T08:55:25.709 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4669999999999998e-06 2024-05-08T08:55:25.709 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.709 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.709 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:25.709 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451143+0000", 2024-05-08T08:55:25.709 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2839999999999997e-06 2024-05-08T08:55:25.709 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.709 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.710 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:25.710 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538575+0000", 2024-05-08T08:55:25.710 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087432789999999996 2024-05-08T08:55:25.710 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.710 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.710 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:25.710 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538648+0000", 2024-05-08T08:55:25.710 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2877e-05 2024-05-08T08:55:25.710 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.710 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.710 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:25.710 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538753+0000", 2024-05-08T08:55:25.710 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010449 2024-05-08T08:55:25.710 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.710 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:25.711 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.711 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.711 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.711 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:968 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 2424832~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:25.711 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451152+0000", 2024-05-08T08:55:25.711 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 136.23514813, 2024-05-08T08:55:25.711 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087813325999999997, 2024-05-08T08:55:25.711 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:25.711 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:25.711 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:25.711 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:25.711 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:25.711 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:25.711 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 968 2024-05-08T08:55:25.711 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.712 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:25.712 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.712 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:25.712 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451152+0000", 2024-05-08T08:55:25.712 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:25.712 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.712 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.712 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:25.712 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451152+0000", 2024-05-08T08:55:25.712 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:25.712 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.712 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.712 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:25.712 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451153+0000", 2024-05-08T08:55:25.712 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3099999999999998e-07 2024-05-08T08:55:25.712 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.713 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.713 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:25.713 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451155+0000", 2024-05-08T08:55:25.713 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4760000000000001e-06 2024-05-08T08:55:25.713 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.713 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.713 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:25.713 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451162+0000", 2024-05-08T08:55:25.713 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5969999999999997e-06 2024-05-08T08:55:25.713 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.713 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.713 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:25.713 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451167+0000", 2024-05-08T08:55:25.713 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.977e-06 2024-05-08T08:55:25.714 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.714 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.714 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:25.714 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538781+0000", 2024-05-08T08:55:25.714 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087614846999999996 2024-05-08T08:55:25.714 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.714 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.714 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:25.714 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538857+0000", 2024-05-08T08:55:25.714 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5358000000000006e-05 2024-05-08T08:55:25.714 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.714 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.714 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:25.714 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538965+0000", 2024-05-08T08:55:25.714 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010864 2024-05-08T08:55:25.715 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.715 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:25.715 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.715 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.715 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.715 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:969 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 2916352~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:25.715 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:55:25.715 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 136.235126425, 2024-05-08T08:55:25.715 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087998980000000004, 2024-05-08T08:55:25.715 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:25.715 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:25.715 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:25.715 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:25.715 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:25.715 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:25.715 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 969 2024-05-08T08:55:25.716 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.716 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:25.716 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.716 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:25.716 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:55:25.716 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:25.716 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.716 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.716 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:25.716 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:55:25.716 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:25.716 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.716 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.716 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:25.716 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:55:25.717 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3599999999999999e-07 2024-05-08T08:55:25.717 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.717 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.717 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:25.717 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451177+0000", 2024-05-08T08:55:25.717 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4590000000000001e-06 2024-05-08T08:55:25.717 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.717 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.717 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:25.717 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451184+0000", 2024-05-08T08:55:25.717 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4109999999999999e-06 2024-05-08T08:55:25.717 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.717 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.717 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:25.717 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451191+0000", 2024-05-08T08:55:25.717 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7279999999999998e-06 2024-05-08T08:55:25.718 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.718 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.718 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:25.718 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538997+0000", 2024-05-08T08:55:25.718 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087806170000000003 2024-05-08T08:55:25.718 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.718 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.718 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:25.718 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539069+0000", 2024-05-08T08:55:25.718 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2373999999999995e-05 2024-05-08T08:55:25.718 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.718 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.718 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:25.718 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539173+0000", 2024-05-08T08:55:25.718 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000103402 2024-05-08T08:55:25.719 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.719 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:25.719 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.719 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.719 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.719 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:970 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 3407872~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:25.719 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451198+0000", 2024-05-08T08:55:25.719 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 136.23510182800001, 2024-05-08T08:55:25.719 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088183368999999998, 2024-05-08T08:55:25.719 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:25.719 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:25.719 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:25.719 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:25.719 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:25.719 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:25.720 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 970 2024-05-08T08:55:25.720 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.720 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:25.720 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.720 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:25.720 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451198+0000", 2024-05-08T08:55:25.720 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:25.720 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.720 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.720 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:25.720 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451198+0000", 2024-05-08T08:55:25.720 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:25.720 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.720 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.720 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:25.720 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451199+0000", 2024-05-08T08:55:25.721 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7400000000000003e-07 2024-05-08T08:55:25.721 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.721 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.721 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:25.721 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451211+0000", 2024-05-08T08:55:25.721 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2109999999999999e-05 2024-05-08T08:55:25.721 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.721 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.721 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:25.721 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451219+0000", 2024-05-08T08:55:25.721 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4170000000000003e-06 2024-05-08T08:55:25.721 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.721 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.721 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:25.721 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451224+0000", 2024-05-08T08:55:25.722 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3990000000000003e-06 2024-05-08T08:55:25.722 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.722 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.722 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:25.722 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539204+0000", 2024-05-08T08:55:25.722 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087980099000000006 2024-05-08T08:55:25.722 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.722 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.722 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:25.722 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539276+0000", 2024-05-08T08:55:25.722 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1513000000000005e-05 2024-05-08T08:55:25.722 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.722 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.722 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:25.722 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539382+0000", 2024-05-08T08:55:25.723 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010615699999999999 2024-05-08T08:55:25.723 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.723 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:25.723 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.723 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.723 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.723 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:971 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 3899392~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:25.723 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451233+0000", 2024-05-08T08:55:25.723 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 136.23506712099999, 2024-05-08T08:55:25.723 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088431525999999996, 2024-05-08T08:55:25.723 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:25.723 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:25.723 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:25.723 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:25.723 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:25.724 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:25.724 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 971 2024-05-08T08:55:25.724 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.724 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:25.724 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.724 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:25.724 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451233+0000", 2024-05-08T08:55:25.724 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:25.724 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.724 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.724 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:25.724 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451233+0000", 2024-05-08T08:55:25.724 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:25.724 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.724 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.725 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:25.725 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451234+0000", 2024-05-08T08:55:25.725 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8500000000000001e-07 2024-05-08T08:55:25.725 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.725 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.725 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:25.725 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451237+0000", 2024-05-08T08:55:25.725 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8320000000000002e-06 2024-05-08T08:55:25.725 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.725 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.725 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:25.725 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451243+0000", 2024-05-08T08:55:25.725 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6749999999999996e-06 2024-05-08T08:55:25.725 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.725 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.726 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:25.726 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451249+0000", 2024-05-08T08:55:25.726 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1229999999999997e-06 2024-05-08T08:55:25.726 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.726 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.726 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:25.726 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539412+0000", 2024-05-08T08:55:25.726 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088162668999999999 2024-05-08T08:55:25.726 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.726 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.726 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:25.726 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539505+0000", 2024-05-08T08:55:25.726 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3240999999999997e-05 2024-05-08T08:55:25.726 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.726 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.727 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:25.727 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539665+0000", 2024-05-08T08:55:25.727 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00015930100000000001 2024-05-08T08:55:25.727 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.727 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:25.727 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.727 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.727 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.727 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:972 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 196608~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:25.727 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451257+0000", 2024-05-08T08:55:25.727 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 136.235043176, 2024-05-08T08:55:25.727 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088719928000000003, 2024-05-08T08:55:25.727 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:25.727 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:25.727 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:25.728 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:25.728 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:25.728 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:25.728 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 972 2024-05-08T08:55:25.728 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.728 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:25.728 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.728 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:25.728 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451257+0000", 2024-05-08T08:55:25.728 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:25.728 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.728 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.728 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:25.728 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451257+0000", 2024-05-08T08:55:25.729 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:25.729 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.729 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.729 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:25.729 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451258+0000", 2024-05-08T08:55:25.729 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7999999999999996e-07 2024-05-08T08:55:25.729 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.729 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.729 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:25.729 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451260+0000", 2024-05-08T08:55:25.729 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4609999999999998e-06 2024-05-08T08:55:25.729 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.729 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.729 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:25.729 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451267+0000", 2024-05-08T08:55:25.730 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.81e-06 2024-05-08T08:55:25.730 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.730 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.730 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:25.730 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451273+0000", 2024-05-08T08:55:25.730 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0859999999999998e-06 2024-05-08T08:55:25.730 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.730 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.730 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:25.730 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539718+0000", 2024-05-08T08:55:25.730 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088445059000000006 2024-05-08T08:55:25.730 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.730 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.730 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:25.734 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539834+0000", 2024-05-08T08:55:25.734 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000116426 2024-05-08T08:55:25.734 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.734 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.734 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:25.734 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539977+0000", 2024-05-08T08:55:25.734 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014260600000000001 2024-05-08T08:55:25.734 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.734 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:25.734 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.734 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.734 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.735 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:973 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 688128~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:25.735 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:55:25.735 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 136.23502059200001, 2024-05-08T08:55:25.735 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088918496, 2024-05-08T08:55:25.735 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:25.735 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:25.735 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:25.735 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:25.735 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:25.735 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:25.735 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 973 2024-05-08T08:55:25.735 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.735 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:25.735 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.735 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:25.735 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:55:25.736 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:25.736 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.736 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.736 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:25.736 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:55:25.736 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:25.736 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.736 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.736 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:25.736 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:55:25.736 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9099999999999999e-07 2024-05-08T08:55:25.736 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.736 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.736 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:25.736 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451282+0000", 2024-05-08T08:55:25.737 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1550000000000001e-06 2024-05-08T08:55:25.737 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.737 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.737 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:25.737 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451288+0000", 2024-05-08T08:55:25.737 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8889999999999999e-06 2024-05-08T08:55:25.737 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.737 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.737 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:25.737 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451294+0000", 2024-05-08T08:55:25.737 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0850000000000002e-06 2024-05-08T08:55:25.737 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.737 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.737 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:25.737 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540011+0000", 2024-05-08T08:55:25.738 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088717314000000005 2024-05-08T08:55:25.738 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.738 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.738 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:25.738 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540089+0000", 2024-05-08T08:55:25.738 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7438e-05 2024-05-08T08:55:25.738 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.738 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.738 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:25.738 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540198+0000", 2024-05-08T08:55:25.738 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000109224 2024-05-08T08:55:25.738 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.738 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:25.738 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.738 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.739 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.739 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:974 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 1179648~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:25.739 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451301+0000", 2024-05-08T08:55:25.739 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 136.234998959, 2024-05-08T08:55:25.739 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089105078000000004, 2024-05-08T08:55:25.739 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:25.739 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:25.739 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:25.739 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:25.739 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:25.739 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:25.739 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 974 2024-05-08T08:55:25.739 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.739 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:25.739 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.740 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:25.740 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451301+0000", 2024-05-08T08:55:25.740 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:25.740 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.740 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.740 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:25.740 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451301+0000", 2024-05-08T08:55:25.740 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:25.740 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.740 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.740 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:25.740 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451302+0000", 2024-05-08T08:55:25.740 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7800000000000002e-07 2024-05-08T08:55:25.740 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.740 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.741 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:25.741 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451304+0000", 2024-05-08T08:55:25.741 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.65e-06 2024-05-08T08:55:25.741 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.741 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.741 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:25.741 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451311+0000", 2024-05-08T08:55:25.741 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.472e-06 2024-05-08T08:55:25.741 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.741 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.741 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:25.741 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451316+0000", 2024-05-08T08:55:25.741 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9189999999999998e-06 2024-05-08T08:55:25.741 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.741 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.742 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:25.742 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540234+0000", 2024-05-08T08:55:25.742 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088917752000000003 2024-05-08T08:55:25.742 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.742 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.742 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:25.742 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540308+0000", 2024-05-08T08:55:25.742 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4758000000000005e-05 2024-05-08T08:55:25.742 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.742 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.742 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:25.742 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540406+0000", 2024-05-08T08:55:25.742 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8048999999999996e-05 2024-05-08T08:55:25.742 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.742 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:25.742 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.743 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.743 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.743 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:975 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 1671168~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:25.743 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:55:25.743 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 136.23495662900001, 2024-05-08T08:55:25.743 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089294139999999994, 2024-05-08T08:55:25.743 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:25.743 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:25.743 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:25.743 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:25.743 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:25.743 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:25.743 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 975 2024-05-08T08:55:25.743 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.743 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:25.743 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.744 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:25.744 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:55:25.744 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:25.744 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.744 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.744 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:25.744 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:55:25.744 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:25.744 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.744 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.744 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:25.744 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:55:25.744 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1299999999999999e-07 2024-05-08T08:55:25.744 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.744 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.744 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:25.745 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451348+0000", 2024-05-08T08:55:25.745 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.484e-06 2024-05-08T08:55:25.745 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.745 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.745 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:25.745 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451356+0000", 2024-05-08T08:55:25.745 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8499999999999994e-06 2024-05-08T08:55:25.745 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.745 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.745 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:25.745 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451366+0000", 2024-05-08T08:55:25.745 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.9850000000000001e-06 2024-05-08T08:55:25.745 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.745 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.745 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:25.746 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540448+0000", 2024-05-08T08:55:25.746 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089082254 2024-05-08T08:55:25.746 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.746 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.746 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:25.746 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540528+0000", 2024-05-08T08:55:25.746 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0234999999999993e-05 2024-05-08T08:55:25.746 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.746 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.746 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:25.746 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540638+0000", 2024-05-08T08:55:25.746 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010951900000000001 2024-05-08T08:55:25.746 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.746 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:25.746 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.747 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.747 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.747 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:976 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 2162688~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:25.747 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451375+0000", 2024-05-08T08:55:25.747 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 136.234924882, 2024-05-08T08:55:25.747 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089472745000000006, 2024-05-08T08:55:25.747 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:25.747 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:25.747 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:25.747 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:25.747 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:25.747 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:25.747 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 976 2024-05-08T08:55:25.747 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.747 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:25.748 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.748 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:25.748 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451375+0000", 2024-05-08T08:55:25.748 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:25.748 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.748 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.748 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:25.748 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451375+0000", 2024-05-08T08:55:25.748 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:25.748 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.748 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.748 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:25.748 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451376+0000", 2024-05-08T08:55:25.748 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2399999999999998e-07 2024-05-08T08:55:25.748 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.748 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.748 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:25.749 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451379+0000", 2024-05-08T08:55:25.749 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6359999999999998e-06 2024-05-08T08:55:25.749 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.749 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.749 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:25.749 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451387+0000", 2024-05-08T08:55:25.749 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9929999999999995e-06 2024-05-08T08:55:25.749 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.749 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.749 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:25.749 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451393+0000", 2024-05-08T08:55:25.749 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4699999999999999e-06 2024-05-08T08:55:25.749 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.749 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.750 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:25.750 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540677+0000", 2024-05-08T08:55:25.750 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089283504999999999 2024-05-08T08:55:25.750 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.750 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.750 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:25.750 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540743+0000", 2024-05-08T08:55:25.750 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6303999999999994e-05 2024-05-08T08:55:25.750 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.750 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.750 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:25.750 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540848+0000", 2024-05-08T08:55:25.750 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000105213 2024-05-08T08:55:25.750 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.750 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:25.750 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.751 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.751 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.751 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:977 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 2654208~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:25.751 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:55:25.751 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 136.23489945, 2024-05-08T08:55:25.751 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089698473000000001, 2024-05-08T08:55:25.751 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:25.751 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:25.751 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:25.751 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:25.751 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:25.751 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:25.751 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 977 2024-05-08T08:55:25.751 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.751 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:25.751 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.752 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:25.752 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:55:25.752 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:25.752 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.752 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.752 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:25.752 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:55:25.752 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:25.752 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.752 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.752 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:25.752 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:55:25.752 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2899999999999999e-07 2024-05-08T08:55:25.752 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.752 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.753 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:25.753 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451404+0000", 2024-05-08T08:55:25.753 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6649999999999999e-06 2024-05-08T08:55:25.753 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.753 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.753 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:25.753 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451410+0000", 2024-05-08T08:55:25.753 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3269999999999997e-06 2024-05-08T08:55:25.753 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.753 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.753 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:25.753 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451415+0000", 2024-05-08T08:55:25.753 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9529999999999998e-06 2024-05-08T08:55:25.753 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.753 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.753 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:25.754 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540912+0000", 2024-05-08T08:55:25.754 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089496950000000006 2024-05-08T08:55:25.754 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.754 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.754 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:25.754 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541000+0000", 2024-05-08T08:55:25.754 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7429000000000006e-05 2024-05-08T08:55:25.754 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.754 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.754 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:25.754 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541099+0000", 2024-05-08T08:55:25.754 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.9720000000000001e-05 2024-05-08T08:55:25.754 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.754 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:25.755 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.755 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.755 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.755 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:978 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 3145728~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:25.755 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:55:25.755 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 136.23487824200001, 2024-05-08T08:55:25.755 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089900635000000007, 2024-05-08T08:55:25.755 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:25.755 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:25.755 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:25.755 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:25.755 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:25.755 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:25.755 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 978 2024-05-08T08:55:25.755 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.756 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:25.756 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.756 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:25.756 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:55:25.756 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:25.756 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.756 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.756 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:25.756 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:55:25.756 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:25.756 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.756 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.756 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:25.756 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:55:25.756 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9400000000000001e-07 2024-05-08T08:55:25.756 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.757 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.757 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:25.757 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451425+0000", 2024-05-08T08:55:25.757 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2460000000000002e-06 2024-05-08T08:55:25.757 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.757 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.757 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:25.757 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451431+0000", 2024-05-08T08:55:25.757 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2419999999999997e-06 2024-05-08T08:55:25.757 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.757 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.757 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:25.757 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451436+0000", 2024-05-08T08:55:25.757 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.887e-06 2024-05-08T08:55:25.758 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.758 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.758 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:25.758 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541152+0000", 2024-05-08T08:55:25.758 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089716194999999999 2024-05-08T08:55:25.758 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.758 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.758 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:25.758 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541224+0000", 2024-05-08T08:55:25.758 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2416000000000006e-05 2024-05-08T08:55:25.758 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.758 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.758 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:25.758 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541323+0000", 2024-05-08T08:55:25.759 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8255e-05 2024-05-08T08:55:25.759 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.759 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:25.759 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.759 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.759 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.759 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:979 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 3637248~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:25.759 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451462+0000", 2024-05-08T08:55:25.759 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 136.234837793, 2024-05-08T08:55:25.759 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.090036441999999994, 2024-05-08T08:55:25.759 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:25.759 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:25.759 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:25.759 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:25.759 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:25.760 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:25.760 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 979 2024-05-08T08:55:25.760 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.760 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:25.760 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.760 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:25.760 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451462+0000", 2024-05-08T08:55:25.760 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:25.760 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.760 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.760 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:25.760 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451462+0000", 2024-05-08T08:55:25.760 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:25.760 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.761 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.761 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:25.761 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451463+0000", 2024-05-08T08:55:25.761 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4300000000000002e-07 2024-05-08T08:55:25.761 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.761 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.761 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:25.761 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451471+0000", 2024-05-08T08:55:25.761 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0800000000000006e-06 2024-05-08T08:55:25.761 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.761 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.761 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:25.761 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451481+0000", 2024-05-08T08:55:25.761 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.38e-06 2024-05-08T08:55:25.761 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.761 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.762 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:25.762 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451487+0000", 2024-05-08T08:55:25.762 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4590000000000003e-06 2024-05-08T08:55:25.762 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.762 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.762 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:25.762 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541350+0000", 2024-05-08T08:55:25.762 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089862599000000001 2024-05-08T08:55:25.762 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.762 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.762 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:25.762 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541407+0000", 2024-05-08T08:55:25.762 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7667999999999997e-05 2024-05-08T08:55:25.762 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.762 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.763 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:25.763 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541499+0000", 2024-05-08T08:55:25.763 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1513000000000003e-05 2024-05-08T08:55:25.763 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.763 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:25.763 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.763 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.763 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.763 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:980 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 4128768~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:25.763 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451496+0000", 2024-05-08T08:55:25.763 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 136.234804037, 2024-05-08T08:55:25.763 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.090258985, 2024-05-08T08:55:25.763 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:25.763 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:25.763 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:25.763 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:25.764 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:25.764 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:25.764 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 980 2024-05-08T08:55:25.764 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.764 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:25.764 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.764 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:25.764 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451496+0000", 2024-05-08T08:55:25.764 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:25.764 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.764 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.764 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:25.764 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451496+0000", 2024-05-08T08:55:25.764 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:25.765 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.765 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.765 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:25.765 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451497+0000", 2024-05-08T08:55:25.765 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7599999999999997e-07 2024-05-08T08:55:25.765 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.765 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.765 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:25.765 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451500+0000", 2024-05-08T08:55:25.765 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7930000000000002e-06 2024-05-08T08:55:25.765 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.765 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.765 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:25.765 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451506+0000", 2024-05-08T08:55:25.765 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6980000000000001e-06 2024-05-08T08:55:25.766 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.766 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.766 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:25.766 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451512+0000", 2024-05-08T08:55:25.766 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2490000000000001e-06 2024-05-08T08:55:25.766 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.766 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.766 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:25.766 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541550+0000", 2024-05-08T08:55:25.766 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.090037961999999999 2024-05-08T08:55:25.766 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.766 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.766 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:25.766 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541647+0000", 2024-05-08T08:55:25.767 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6924999999999998e-05 2024-05-08T08:55:25.767 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.767 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.767 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:25.767 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541755+0000", 2024-05-08T08:55:25.767 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000108682 2024-05-08T08:55:25.767 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.767 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:25.767 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.767 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.767 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.767 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:33397 3.7 3:e5a023c9:::rbd_header.11402419351a:head [watch reconnect cookie 140139347723488 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e156)", 2024-05-08T08:55:25.767 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:55:07.435461+0000", 2024-05-08T08:55:25.767 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 18.250838737999999, 2024-05-08T08:55:25.767 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.009753092, 2024-05-08T08:55:25.768 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:25.768 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:25.768 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:25.768 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:25.768 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:55:25.768 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:55:25.768 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 33397 2024-05-08T08:55:25.768 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.768 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:25.768 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.768 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:25.768 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:07.435461+0000", 2024-05-08T08:55:25.768 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:25.768 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.768 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.768 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:25.768 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:07.435461+0000", 2024-05-08T08:55:25.769 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:25.769 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.769 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.769 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:25.769 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:07.435463+0000", 2024-05-08T08:55:25.769 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.7749999999999999e-06 2024-05-08T08:55:25.769 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.769 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.769 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:25.769 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:07.435471+0000", 2024-05-08T08:55:25.769 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.2199999999999992e-06 2024-05-08T08:55:25.769 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.769 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.769 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:25.769 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:07.435487+0000", 2024-05-08T08:55:25.769 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5806000000000001e-05 2024-05-08T08:55:25.769 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.770 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.770 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:25.770 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:07.435867+0000", 2024-05-08T08:55:25.770 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00037973999999999999 2024-05-08T08:55:25.770 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.770 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.770 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:25.770 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:07.436510+0000", 2024-05-08T08:55:25.770 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00064263299999999998 2024-05-08T08:55:25.770 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.770 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.770 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:55:25.770 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:07.436524+0000", 2024-05-08T08:55:25.770 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.4316e-05 2024-05-08T08:55:25.771 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.771 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.771 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:25.771 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:08.439082+0000", 2024-05-08T08:55:25.771 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.002558235 2024-05-08T08:55:25.771 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.771 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.771 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:25.771 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:08.439286+0000", 2024-05-08T08:55:25.771 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00020426600000000001 2024-05-08T08:55:25.771 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.771 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.771 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:25.771 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:08.445215+0000", 2024-05-08T08:55:25.771 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0059281009999999999 2024-05-08T08:55:25.771 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.771 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:25.772 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.772 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.772 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.772 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:33398 3.7 3.93c405a7 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e156)", 2024-05-08T08:55:25.772 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:55:07.435912+0000", 2024-05-08T08:55:25.772 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 18.250388539999999, 2024-05-08T08:55:25.772 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0044546430000001, 2024-05-08T08:55:25.772 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:25.772 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:25.772 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "reached pg", 2024-05-08T08:55:25.772 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:25.772 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:55:25.772 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:55:25.772 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 33398 2024-05-08T08:55:25.772 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.773 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:25.773 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.773 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:25.773 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:07.435912+0000", 2024-05-08T08:55:25.773 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:25.773 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.773 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.773 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:25.773 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:07.435912+0000", 2024-05-08T08:55:25.773 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:25.773 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.773 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.773 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:25.773 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:07.435913+0000", 2024-05-08T08:55:25.773 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2200000000000002e-07 2024-05-08T08:55:25.773 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.774 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.774 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:25.774 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:07.435919+0000", 2024-05-08T08:55:25.774 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3820000000000001e-06 2024-05-08T08:55:25.774 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.774 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.774 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:25.774 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:07.435932+0000", 2024-05-08T08:55:25.774 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2788e-05 2024-05-08T08:55:25.774 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.774 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.774 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:25.774 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:07.435940+0000", 2024-05-08T08:55:25.774 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.6710000000000005e-06 2024-05-08T08:55:25.775 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.775 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.775 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:25.775 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:07.436558+0000", 2024-05-08T08:55:25.775 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00061782299999999998 2024-05-08T08:55:25.775 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.775 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.775 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:55:25.775 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:07.436572+0000", 2024-05-08T08:55:25.775 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.34e-05 2024-05-08T08:55:25.775 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.775 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.775 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:25.775 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:08.440285+0000", 2024-05-08T08:55:25.775 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.003713388 2024-05-08T08:55:25.775 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:25.776 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:25.776 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:25.776 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:08.440366+0000", 2024-05-08T08:55:25.776 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1268999999999997e-05 2024-05-08T08:55:25.776 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.776 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:25.776 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.776 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:25.776 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:25.776 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:55:25.778 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:55:25.778 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:55:25.780+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:55:25.874 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4] out_osds: [5, 0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-05-08T08:55:25.874 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T08:55:25.874 INFO:tasks.thrashosds.thrasher:inject_pause on osd.5 2024-05-08T08:55:25.874 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-05-08T08:55:25.874 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-05-08T08:55:25.874 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok config set filestore_inject_stall 3 2024-05-08T08:55:25.878 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:55:25.880+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:55:25.885 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:55:25.885 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:55:25.886 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:55:25.886 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:55:25.895 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:55:25.979 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:55:25.980+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:55:25.982 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:55:25.982 INFO:teuthology.orchestra.run.smithi192.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-05-08T08:55:25.982 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:55:26.040 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:55:26.041 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:55:26.041 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:55:26.041 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:55:26.041 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:55:26.049 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:55:26.079 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:55:26.081+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:55:26.156 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:55:26.157 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:55:26.157 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:55:26.157 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:55:26.157 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.157 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4621.0:9089 3.1f e33/18 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:head v 33'117, mlcod=31'115)", 2024-05-08T08:55:26.157 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:55:26.157 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 173.638308421, 2024-05-08T08:55:26.157 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.111244524, 2024-05-08T08:55:26.157 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:26.157 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:26.157 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:55:26.157 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:26.157 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.157 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:26.158 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:55:26.158 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:26.158 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.158 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.158 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:26.158 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:55:26.158 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:26.158 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.158 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.158 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:26.158 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:55:26.158 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5900000000000001e-07 2024-05-08T08:55:26.158 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.159 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.159 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:26.159 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539600+0000", 2024-05-08T08:55:26.159 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.019981364000000001 2024-05-08T08:55:26.159 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.159 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.159 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:26.159 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539624+0000", 2024-05-08T08:55:26.159 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4043e-05 2024-05-08T08:55:26.159 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.159 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.159 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:26.159 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539635+0000", 2024-05-08T08:55:26.159 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1511e-05 2024-05-08T08:55:26.159 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.160 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.160 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:26.160 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539728+0000", 2024-05-08T08:55:26.160 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2498999999999996e-05 2024-05-08T08:55:26.160 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.160 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.160 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:26.160 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539746+0000", 2024-05-08T08:55:26.160 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8289000000000001e-05 2024-05-08T08:55:26.160 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.160 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.160 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:55:26.160 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.630822+0000", 2024-05-08T08:55:26.160 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.091076283999999993 2024-05-08T08:55:26.161 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.161 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.161 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:26.161 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.630863+0000", 2024-05-08T08:55:26.161 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0374999999999999e-05 2024-05-08T08:55:26.161 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.161 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:26.161 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.161 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.161 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.161 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4195.0:4 3.11 3:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-05-08T08:55:26.161 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:49.764654+0000", 2024-05-08T08:55:26.161 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 156.39327288000001, 2024-05-08T08:55:26.161 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.30714275499999999, 2024-05-08T08:55:26.161 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:26.162 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:26.162 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-05-08T08:55:26.162 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:26.162 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4195", 2024-05-08T08:55:26.162 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/59179521", 2024-05-08T08:55:26.162 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4 2024-05-08T08:55:26.162 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.162 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:26.162 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.162 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:26.162 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764654+0000", 2024-05-08T08:55:26.162 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:26.162 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.162 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.162 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:26.163 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764654+0000", 2024-05-08T08:55:26.163 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:26.163 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.163 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.163 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:26.163 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764655+0000", 2024-05-08T08:55:26.163 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.452e-06 2024-05-08T08:55:26.163 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.163 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.163 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:26.163 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764661+0000", 2024-05-08T08:55:26.163 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5369999999999997e-06 2024-05-08T08:55:26.163 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.163 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.164 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:26.164 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764671+0000", 2024-05-08T08:55:26.164 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.028e-05 2024-05-08T08:55:26.164 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.164 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.164 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:26.164 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764680+0000", 2024-05-08T08:55:26.164 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0629999999999995e-06 2024-05-08T08:55:26.164 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.164 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.164 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:26.164 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764753+0000", 2024-05-08T08:55:26.164 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2679e-05 2024-05-08T08:55:26.164 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.164 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.165 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-05-08T08:55:26.165 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764765+0000", 2024-05-08T08:55:26.165 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2415000000000001e-05 2024-05-08T08:55:26.165 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.165 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.165 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:26.165 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:50.071692+0000", 2024-05-08T08:55:26.165 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.306927324 2024-05-08T08:55:26.165 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.165 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.165 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:26.165 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:50.071796+0000", 2024-05-08T08:55:26.165 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104005 2024-05-08T08:55:26.165 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.165 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:26.165 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.165 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.166 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.166 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4416.0:20531 3.16 3:6bb2ed16:::rbd_object_map.1140bc4a55c7:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-05-08T08:55:26.166 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:58.673786+0000", 2024-05-08T08:55:26.166 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 87.484140475999993, 2024-05-08T08:55:26.166 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.94684119899999997, 2024-05-08T08:55:26.166 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:26.166 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:26.166 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:55:26.166 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:26.166 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4416", 2024-05-08T08:55:26.166 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:55:26.166 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20531 2024-05-08T08:55:26.166 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.166 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:26.167 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.167 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:26.167 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673786+0000", 2024-05-08T08:55:26.167 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:26.167 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.167 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.167 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:26.167 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673786+0000", 2024-05-08T08:55:26.167 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:26.167 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.167 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.167 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:26.167 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673787+0000", 2024-05-08T08:55:26.167 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9800000000000003e-07 2024-05-08T08:55:26.167 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.168 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.168 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:26.168 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673791+0000", 2024-05-08T08:55:26.168 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.104e-06 2024-05-08T08:55:26.168 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.168 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.168 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:26.168 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673801+0000", 2024-05-08T08:55:26.168 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9960000000000005e-06 2024-05-08T08:55:26.168 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.168 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.168 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:26.168 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673806+0000", 2024-05-08T08:55:26.168 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2320000000000001e-06 2024-05-08T08:55:26.168 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.168 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.169 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:26.169 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673860+0000", 2024-05-08T08:55:26.169 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3468999999999999e-05 2024-05-08T08:55:26.169 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.169 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.169 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-05-08T08:55:26.169 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673866+0000", 2024-05-08T08:55:26.169 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3500000000000002e-06 2024-05-08T08:55:26.169 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.169 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.169 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:26.169 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.616700+0000", 2024-05-08T08:55:26.169 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.94283390700000003 2024-05-08T08:55:26.169 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.169 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.169 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:26.170 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.616941+0000", 2024-05-08T08:55:26.170 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00024080400000000001 2024-05-08T08:55:26.170 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.170 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.170 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-05-08T08:55:26.170 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.617072+0000", 2024-05-08T08:55:26.170 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000131688 2024-05-08T08:55:26.170 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.170 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.170 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:55:26.170 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.619571+0000", 2024-05-08T08:55:26.170 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0024990149999999998 2024-05-08T08:55:26.170 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.170 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.171 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:55:26.171 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.620568+0000", 2024-05-08T08:55:26.171 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00099711000000000005 2024-05-08T08:55:26.171 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.171 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.171 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:55:26.171 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.620602+0000", 2024-05-08T08:55:26.171 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.324e-05 2024-05-08T08:55:26.171 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.171 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.171 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:26.171 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.620627+0000", 2024-05-08T08:55:26.171 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5585999999999999e-05 2024-05-08T08:55:26.171 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.171 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:26.171 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.171 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.172 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.172 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5492 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4272128~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:26.172 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.881830+0000", 2024-05-08T08:55:26.172 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 60.276096670000001, 2024-05-08T08:55:26.172 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047614006, 2024-05-08T08:55:26.172 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:26.172 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:26.172 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:26.172 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:26.172 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:26.172 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:26.172 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5492 2024-05-08T08:55:26.172 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.173 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:26.173 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.173 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:26.173 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881830+0000", 2024-05-08T08:55:26.173 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:26.173 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.173 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.173 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:26.173 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881830+0000", 2024-05-08T08:55:26.173 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:26.173 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.173 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.173 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:26.173 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881830+0000", 2024-05-08T08:55:26.174 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4099999999999999e-07 2024-05-08T08:55:26.174 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.174 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.174 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:26.174 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881832+0000", 2024-05-08T08:55:26.174 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0370000000000001e-06 2024-05-08T08:55:26.174 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.174 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.174 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:26.174 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881839+0000", 2024-05-08T08:55:26.174 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3289999999999999e-06 2024-05-08T08:55:26.174 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.174 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.174 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:26.175 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881844+0000", 2024-05-08T08:55:26.175 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1170000000000003e-06 2024-05-08T08:55:26.175 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.175 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.175 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:26.175 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929296+0000", 2024-05-08T08:55:26.175 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047452418000000003 2024-05-08T08:55:26.175 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.175 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.175 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:26.175 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929360+0000", 2024-05-08T08:55:26.175 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4197999999999998e-05 2024-05-08T08:55:26.175 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.175 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.175 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:26.175 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929444+0000", 2024-05-08T08:55:26.176 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3466000000000004e-05 2024-05-08T08:55:26.176 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.176 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:26.176 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.176 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.176 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.176 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5494 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4317184~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:26.176 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.881850+0000", 2024-05-08T08:55:26.176 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 60.276076920999998, 2024-05-08T08:55:26.176 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047723887, 2024-05-08T08:55:26.176 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:26.176 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:26.176 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:26.176 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:26.176 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:26.176 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:26.177 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5494 2024-05-08T08:55:26.177 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.177 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:26.177 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.177 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:26.177 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881850+0000", 2024-05-08T08:55:26.177 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:26.177 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.177 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.177 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:26.177 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881850+0000", 2024-05-08T08:55:26.177 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:26.177 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.177 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.178 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:26.178 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881850+0000", 2024-05-08T08:55:26.178 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8500000000000002e-07 2024-05-08T08:55:26.178 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.178 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.178 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:26.178 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881856+0000", 2024-05-08T08:55:26.178 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9510000000000003e-06 2024-05-08T08:55:26.178 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.178 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.178 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:26.178 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881862+0000", 2024-05-08T08:55:26.178 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3049999999999998e-06 2024-05-08T08:55:26.178 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.178 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.178 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:26.179 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881868+0000", 2024-05-08T08:55:26.179 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8340000000000004e-06 2024-05-08T08:55:26.179 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.179 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.179 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:26.179 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929475+0000", 2024-05-08T08:55:26.179 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047606796 2024-05-08T08:55:26.179 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.179 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.179 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:26.179 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929518+0000", 2024-05-08T08:55:26.179 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3387000000000003e-05 2024-05-08T08:55:26.179 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.179 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.179 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:26.180 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929574+0000", 2024-05-08T08:55:26.180 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5129e-05 2024-05-08T08:55:26.180 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.180 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:26.180 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.180 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.180 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.180 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5495 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4362240~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:26.180 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.881875+0000", 2024-05-08T08:55:26.180 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 60.276052053999997, 2024-05-08T08:55:26.180 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047847967999999998, 2024-05-08T08:55:26.180 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:26.180 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:26.180 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:26.181 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:26.181 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:26.181 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:26.181 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5495 2024-05-08T08:55:26.181 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.181 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:26.181 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.181 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:26.181 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881875+0000", 2024-05-08T08:55:26.181 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:26.181 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.181 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.181 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:26.181 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881875+0000", 2024-05-08T08:55:26.181 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:26.181 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.182 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.182 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:26.182 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881875+0000", 2024-05-08T08:55:26.182 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-07 2024-05-08T08:55:26.182 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.182 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.182 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:26.182 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881877+0000", 2024-05-08T08:55:26.182 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9659999999999999e-06 2024-05-08T08:55:26.182 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.182 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.182 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:26.182 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881883+0000", 2024-05-08T08:55:26.182 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7649999999999999e-06 2024-05-08T08:55:26.183 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.183 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.183 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:26.183 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881887+0000", 2024-05-08T08:55:26.183 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6689999999999997e-06 2024-05-08T08:55:26.183 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.183 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.183 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:26.183 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929600+0000", 2024-05-08T08:55:26.183 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047712746 2024-05-08T08:55:26.183 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.183 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.183 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:26.183 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929659+0000", 2024-05-08T08:55:26.183 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8936e-05 2024-05-08T08:55:26.184 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.184 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.184 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:26.184 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929723+0000", 2024-05-08T08:55:26.184 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3524999999999999e-05 2024-05-08T08:55:26.184 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.184 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:26.184 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.184 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.184 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.184 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5496 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4407296~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:26.184 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.881894+0000", 2024-05-08T08:55:26.184 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 60.276032190000002, 2024-05-08T08:55:26.184 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047998015999999998, 2024-05-08T08:55:26.184 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:26.185 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:26.185 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:26.185 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:26.185 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:26.185 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:26.185 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5496 2024-05-08T08:55:26.185 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.185 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:26.185 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.185 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:26.185 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881894+0000", 2024-05-08T08:55:26.185 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:26.185 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.185 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.185 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:26.185 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881894+0000", 2024-05-08T08:55:26.186 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:26.186 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.186 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.186 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:26.186 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881895+0000", 2024-05-08T08:55:26.186 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6699999999999999e-07 2024-05-08T08:55:26.186 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.186 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.186 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:26.186 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881897+0000", 2024-05-08T08:55:26.186 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0229999999999999e-06 2024-05-08T08:55:26.186 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.186 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.186 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:26.187 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881903+0000", 2024-05-08T08:55:26.187 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9850000000000003e-06 2024-05-08T08:55:26.187 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.187 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.187 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:26.187 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881908+0000", 2024-05-08T08:55:26.187 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9019999999999998e-06 2024-05-08T08:55:26.187 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.187 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.187 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:26.187 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929751+0000", 2024-05-08T08:55:26.187 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047843550999999998 2024-05-08T08:55:26.187 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.187 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.187 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:26.188 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929823+0000", 2024-05-08T08:55:26.188 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1172000000000002e-05 2024-05-08T08:55:26.188 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.188 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.188 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:26.188 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929892+0000", 2024-05-08T08:55:26.188 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9915999999999999e-05 2024-05-08T08:55:26.188 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.188 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:26.188 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.188 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.188 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.188 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5498 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4452352~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:26.188 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.881914+0000", 2024-05-08T08:55:26.188 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 60.276012432000002, 2024-05-08T08:55:26.188 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048203553000000003, 2024-05-08T08:55:26.189 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:26.189 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:26.189 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:26.189 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:26.189 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:26.189 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:26.189 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5498 2024-05-08T08:55:26.189 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.189 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:26.189 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.189 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:26.189 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881914+0000", 2024-05-08T08:55:26.189 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:26.189 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.190 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.190 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:26.190 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881914+0000", 2024-05-08T08:55:26.190 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:26.190 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.190 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.190 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:26.190 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881915+0000", 2024-05-08T08:55:26.190 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8500000000000002e-07 2024-05-08T08:55:26.190 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.190 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.190 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:26.190 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881917+0000", 2024-05-08T08:55:26.190 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.181e-06 2024-05-08T08:55:26.190 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.191 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.191 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:26.191 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881924+0000", 2024-05-08T08:55:26.191 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9619999999999996e-06 2024-05-08T08:55:26.191 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.191 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.191 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:26.191 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881929+0000", 2024-05-08T08:55:26.191 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1399999999999999e-06 2024-05-08T08:55:26.191 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.191 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.191 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:26.191 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929952+0000", 2024-05-08T08:55:26.191 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048023134000000002 2024-05-08T08:55:26.192 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.192 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.192 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:26.192 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.930029+0000", 2024-05-08T08:55:26.192 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7490000000000005e-05 2024-05-08T08:55:26.192 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.192 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.192 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:26.192 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.930118+0000", 2024-05-08T08:55:26.192 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8261000000000001e-05 2024-05-08T08:55:26.192 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.192 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:26.192 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.192 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.192 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.193 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5564 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4366336~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:26.193 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.884225+0000", 2024-05-08T08:55:26.193 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 60.273701641999999, 2024-05-08T08:55:26.193 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047555989999999999, 2024-05-08T08:55:26.193 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:26.193 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:26.193 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:26.193 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:26.193 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:26.193 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:26.193 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5564 2024-05-08T08:55:26.193 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.193 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:26.193 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.193 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:26.193 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884225+0000", 2024-05-08T08:55:26.194 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:26.194 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.194 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.194 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:26.194 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884225+0000", 2024-05-08T08:55:26.194 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:26.194 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.194 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.194 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:26.194 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884225+0000", 2024-05-08T08:55:26.194 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2499999999999995e-07 2024-05-08T08:55:26.194 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.194 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.194 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:26.195 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884227+0000", 2024-05-08T08:55:26.195 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5829999999999999e-06 2024-05-08T08:55:26.195 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.195 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.195 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:26.195 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884232+0000", 2024-05-08T08:55:26.195 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1159999999999998e-06 2024-05-08T08:55:26.195 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.195 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.195 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:26.195 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884236+0000", 2024-05-08T08:55:26.195 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2490000000000003e-06 2024-05-08T08:55:26.195 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.195 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.195 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:26.196 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931617+0000", 2024-05-08T08:55:26.196 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047380310000000002 2024-05-08T08:55:26.196 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.196 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.196 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:26.196 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931689+0000", 2024-05-08T08:55:26.196 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2358000000000001e-05 2024-05-08T08:55:26.196 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.196 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.196 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:26.196 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931781+0000", 2024-05-08T08:55:26.196 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1848999999999994e-05 2024-05-08T08:55:26.196 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.196 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:26.196 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.197 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.197 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.197 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5565 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4411392~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:26.197 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.884251+0000", 2024-05-08T08:55:26.197 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 60.273675488999999, 2024-05-08T08:55:26.197 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047743671000000001, 2024-05-08T08:55:26.197 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:26.197 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:26.197 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:26.197 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:26.197 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:26.197 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:26.197 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5565 2024-05-08T08:55:26.197 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.197 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:26.198 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.198 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:26.198 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884251+0000", 2024-05-08T08:55:26.198 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:26.198 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.198 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.198 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:26.198 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884251+0000", 2024-05-08T08:55:26.198 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:26.198 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.198 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.198 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:26.198 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884252+0000", 2024-05-08T08:55:26.198 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3200000000000005e-07 2024-05-08T08:55:26.199 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.199 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.199 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:26.199 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884253+0000", 2024-05-08T08:55:26.199 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6980000000000001e-06 2024-05-08T08:55:26.199 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.199 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.199 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:26.199 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884259+0000", 2024-05-08T08:55:26.199 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3759999999999999e-06 2024-05-08T08:55:26.199 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.199 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.199 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:26.199 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884263+0000", 2024-05-08T08:55:26.199 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7280000000000003e-06 2024-05-08T08:55:26.200 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.200 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.200 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:26.200 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931818+0000", 2024-05-08T08:55:26.200 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047554368999999999 2024-05-08T08:55:26.200 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.200 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.200 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:26.200 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931897+0000", 2024-05-08T08:55:26.200 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8821999999999999e-05 2024-05-08T08:55:26.200 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.200 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.200 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:26.200 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931995+0000", 2024-05-08T08:55:26.201 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8146000000000005e-05 2024-05-08T08:55:26.201 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.201 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:26.201 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.201 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.201 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.201 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5600 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 3948544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:26.201 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.884619+0000", 2024-05-08T08:55:26.201 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 60.273307234000001, 2024-05-08T08:55:26.201 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047574067999999997, 2024-05-08T08:55:26.201 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:26.201 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:26.201 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:26.201 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:26.201 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:26.202 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:26.202 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5600 2024-05-08T08:55:26.202 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.202 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:26.202 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.202 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:26.202 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884619+0000", 2024-05-08T08:55:26.202 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:26.202 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.202 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.202 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:26.202 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884619+0000", 2024-05-08T08:55:26.202 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:26.202 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.203 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.203 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:26.203 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884620+0000", 2024-05-08T08:55:26.203 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.53e-07 2024-05-08T08:55:26.203 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.203 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.203 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:26.203 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884622+0000", 2024-05-08T08:55:26.203 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.796e-06 2024-05-08T08:55:26.203 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.203 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.203 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:26.203 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884627+0000", 2024-05-08T08:55:26.203 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2850000000000002e-06 2024-05-08T08:55:26.203 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.204 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.204 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:26.204 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884631+0000", 2024-05-08T08:55:26.204 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2899999999999996e-06 2024-05-08T08:55:26.204 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.204 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.204 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:26.204 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932024+0000", 2024-05-08T08:55:26.204 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047392680999999999 2024-05-08T08:55:26.204 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.204 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.204 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:26.204 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932102+0000", 2024-05-08T08:55:26.204 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7806999999999996e-05 2024-05-08T08:55:26.204 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.204 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.205 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:26.205 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932193+0000", 2024-05-08T08:55:26.205 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1855999999999996e-05 2024-05-08T08:55:26.205 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.205 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:26.205 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.205 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.205 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.205 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5603 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 3993600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:26.205 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.884680+0000", 2024-05-08T08:55:26.205 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 60.273246624999999, 2024-05-08T08:55:26.205 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047745034999999998, 2024-05-08T08:55:26.205 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:26.205 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:26.205 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:26.206 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:26.206 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:26.206 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:26.206 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5603 2024-05-08T08:55:26.206 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.206 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:26.206 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.206 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:26.206 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884680+0000", 2024-05-08T08:55:26.206 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:26.206 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.206 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.206 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:26.206 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884680+0000", 2024-05-08T08:55:26.207 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:26.207 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.207 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.207 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:26.207 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884680+0000", 2024-05-08T08:55:26.207 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4400000000000001e-07 2024-05-08T08:55:26.207 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.207 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.207 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:26.207 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884682+0000", 2024-05-08T08:55:26.207 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.514e-06 2024-05-08T08:55:26.207 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.207 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.207 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:26.207 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884687+0000", 2024-05-08T08:55:26.208 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0359999999999997e-06 2024-05-08T08:55:26.208 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.208 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.208 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:26.208 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884692+0000", 2024-05-08T08:55:26.208 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5990000000000004e-06 2024-05-08T08:55:26.208 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.208 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.208 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:26.208 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932235+0000", 2024-05-08T08:55:26.208 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047543635000000001 2024-05-08T08:55:26.208 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.208 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.208 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:26.208 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932323+0000", 2024-05-08T08:55:26.209 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7311000000000005e-05 2024-05-08T08:55:26.209 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.209 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.209 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:26.209 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932425+0000", 2024-05-08T08:55:26.209 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102496 2024-05-08T08:55:26.209 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.209 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:26.209 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.209 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.209 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.209 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5606 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4038656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:26.209 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.884729+0000", 2024-05-08T08:55:26.209 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 60.273197336999999, 2024-05-08T08:55:26.209 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047916488, 2024-05-08T08:55:26.210 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:26.210 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:26.210 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:26.210 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:26.210 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:26.210 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:26.210 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5606 2024-05-08T08:55:26.210 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.210 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:26.210 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.210 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:26.210 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884729+0000", 2024-05-08T08:55:26.210 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:26.210 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.210 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.211 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:26.211 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884729+0000", 2024-05-08T08:55:26.211 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:26.211 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.211 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.211 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:26.211 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.885940+0000", 2024-05-08T08:55:26.211 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001210509 2024-05-08T08:55:26.211 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.211 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.211 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:26.211 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.885945+0000", 2024-05-08T08:55:26.211 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7389999999999999e-06 2024-05-08T08:55:26.211 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.212 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.212 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:26.212 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.885955+0000", 2024-05-08T08:55:26.212 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0813e-05 2024-05-08T08:55:26.212 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.212 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.212 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:26.212 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.885962+0000", 2024-05-08T08:55:26.212 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1160000000000001e-06 2024-05-08T08:55:26.212 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.212 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.212 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:26.212 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932465+0000", 2024-05-08T08:55:26.212 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046502612999999998 2024-05-08T08:55:26.212 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.212 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.213 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:26.213 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932547+0000", 2024-05-08T08:55:26.213 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2025000000000002e-05 2024-05-08T08:55:26.213 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.213 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.213 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:26.213 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932646+0000", 2024-05-08T08:55:26.213 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8672999999999995e-05 2024-05-08T08:55:26.213 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.213 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:26.213 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.213 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.213 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.213 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5636 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4280320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:26.214 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.886111+0000", 2024-05-08T08:55:26.214 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 60.271815599999996, 2024-05-08T08:55:26.214 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047663644999999998, 2024-05-08T08:55:26.214 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:26.214 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:26.214 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:26.214 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:26.214 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:26.214 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:26.214 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5636 2024-05-08T08:55:26.214 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.214 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:26.214 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.214 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:26.215 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886111+0000", 2024-05-08T08:55:26.215 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:26.215 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.215 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.215 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:26.215 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886111+0000", 2024-05-08T08:55:26.215 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:26.215 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.215 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.215 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:26.215 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886111+0000", 2024-05-08T08:55:26.215 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5200000000000002e-07 2024-05-08T08:55:26.215 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.215 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.215 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:26.216 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886114+0000", 2024-05-08T08:55:26.216 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1710000000000001e-06 2024-05-08T08:55:26.216 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.216 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.216 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:26.216 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886120+0000", 2024-05-08T08:55:26.216 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.404e-06 2024-05-08T08:55:26.216 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.216 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.216 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:26.216 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886125+0000", 2024-05-08T08:55:26.216 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2730000000000002e-06 2024-05-08T08:55:26.216 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.216 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.216 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:26.216 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933570+0000", 2024-05-08T08:55:26.217 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047444989999999999 2024-05-08T08:55:26.217 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.217 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.217 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:26.217 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933659+0000", 2024-05-08T08:55:26.217 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8912999999999994e-05 2024-05-08T08:55:26.217 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.217 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.217 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:26.217 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933775+0000", 2024-05-08T08:55:26.217 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000115442 2024-05-08T08:55:26.217 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.217 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:26.217 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.217 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.217 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.217 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5637 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4325376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:26.218 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.886132+0000", 2024-05-08T08:55:26.218 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 60.271794561999997, 2024-05-08T08:55:26.218 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047852219000000001, 2024-05-08T08:55:26.218 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:26.218 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:26.218 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:26.218 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:26.218 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:26.218 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:26.218 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5637 2024-05-08T08:55:26.218 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.218 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:26.218 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.218 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:26.218 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886132+0000", 2024-05-08T08:55:26.219 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:26.219 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.219 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.219 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:26.219 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886132+0000", 2024-05-08T08:55:26.219 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:26.219 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.219 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.219 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:26.219 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886132+0000", 2024-05-08T08:55:26.219 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7300000000000002e-07 2024-05-08T08:55:26.219 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.219 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.219 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:26.219 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886139+0000", 2024-05-08T08:55:26.220 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6109999999999999e-06 2024-05-08T08:55:26.220 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.220 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.220 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:26.220 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886146+0000", 2024-05-08T08:55:26.220 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6279999999999999e-06 2024-05-08T08:55:26.220 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.220 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.220 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:26.220 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886151+0000", 2024-05-08T08:55:26.220 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1769999999999997e-06 2024-05-08T08:55:26.220 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.220 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.220 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:26.220 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933812+0000", 2024-05-08T08:55:26.220 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047660912999999999 2024-05-08T08:55:26.221 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.221 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.221 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:26.221 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933888+0000", 2024-05-08T08:55:26.221 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5865000000000006e-05 2024-05-08T08:55:26.221 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.221 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.221 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:26.221 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933984+0000", 2024-05-08T08:55:26.221 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6651999999999996e-05 2024-05-08T08:55:26.221 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.221 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:26.221 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.221 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.221 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.221 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5638 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4370432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:26.222 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.886157+0000", 2024-05-08T08:55:26.222 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 60.271769298000002, 2024-05-08T08:55:26.222 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048077597, 2024-05-08T08:55:26.222 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:26.222 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:26.222 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:26.222 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:26.222 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:26.222 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:26.222 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5638 2024-05-08T08:55:26.222 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.222 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:26.222 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.222 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:26.222 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886157+0000", 2024-05-08T08:55:26.223 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:26.223 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.223 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.223 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:26.223 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886157+0000", 2024-05-08T08:55:26.223 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:26.223 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.223 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.223 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:26.223 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886158+0000", 2024-05-08T08:55:26.223 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1100000000000001e-07 2024-05-08T08:55:26.223 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.223 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.223 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:26.223 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886160+0000", 2024-05-08T08:55:26.224 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9800000000000001e-06 2024-05-08T08:55:26.224 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.224 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.224 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:26.224 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886166+0000", 2024-05-08T08:55:26.224 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.314e-06 2024-05-08T08:55:26.224 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.224 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.224 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:26.224 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886171+0000", 2024-05-08T08:55:26.224 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7700000000000001e-06 2024-05-08T08:55:26.224 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.224 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.224 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:26.224 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.934025+0000", 2024-05-08T08:55:26.224 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047854509000000003 2024-05-08T08:55:26.225 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.225 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.225 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:26.225 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.934121+0000", 2024-05-08T08:55:26.225 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5688999999999998e-05 2024-05-08T08:55:26.225 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.225 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.225 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:26.225 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.934235+0000", 2024-05-08T08:55:26.225 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000113924 2024-05-08T08:55:26.225 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.225 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:26.225 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.225 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.225 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.225 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5639 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4415488~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:26.226 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.886177+0000", 2024-05-08T08:55:26.226 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 60.271749548000003, 2024-05-08T08:55:26.226 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048249541, 2024-05-08T08:55:26.226 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:26.226 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:26.226 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:26.226 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:26.226 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:26.226 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:26.226 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5639 2024-05-08T08:55:26.226 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.226 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:26.226 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.226 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:26.226 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886177+0000", 2024-05-08T08:55:26.227 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:26.227 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.227 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.227 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:26.227 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886177+0000", 2024-05-08T08:55:26.227 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:26.227 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.227 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.227 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:26.227 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886177+0000", 2024-05-08T08:55:26.227 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.65e-07 2024-05-08T08:55:26.227 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.227 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.227 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:26.227 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886179+0000", 2024-05-08T08:55:26.227 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9530000000000002e-06 2024-05-08T08:55:26.228 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.228 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.228 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:26.228 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886186+0000", 2024-05-08T08:55:26.228 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3419999999999996e-06 2024-05-08T08:55:26.228 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.228 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.228 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:26.228 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886192+0000", 2024-05-08T08:55:26.228 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0560000000000001e-06 2024-05-08T08:55:26.228 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.228 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.228 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:26.228 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.934258+0000", 2024-05-08T08:55:26.228 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048066367999999998 2024-05-08T08:55:26.229 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.229 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.229 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:26.229 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.934338+0000", 2024-05-08T08:55:26.229 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9616000000000005e-05 2024-05-08T08:55:26.229 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.229 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.229 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:26.229 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.934427+0000", 2024-05-08T08:55:26.229 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8840999999999999e-05 2024-05-08T08:55:26.229 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.229 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:26.229 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.229 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.229 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.229 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4416.0:34114 3.12 3:4e31a7e4:::rbd_data.1140100c9703.0000000000000008:head [delete] snapc 14=[] ondisk+write+known_if_redirected+supports_pool_eio e164)", 2024-05-08T08:55:26.230 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:55:15.450741+0000", 2024-05-08T08:55:26.230 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 10.707185803, 2024-05-08T08:55:26.230 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.92638831499999996, 2024-05-08T08:55:26.230 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:26.230 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:26.230 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:55:26.230 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:26.230 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4416", 2024-05-08T08:55:26.230 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:55:26.230 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 34114 2024-05-08T08:55:26.230 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.230 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:26.230 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.231 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:26.231 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:15.450741+0000", 2024-05-08T08:55:26.231 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:26.231 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.231 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.231 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:26.231 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:15.450741+0000", 2024-05-08T08:55:26.231 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:26.231 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.231 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.231 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:26.231 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:15.450741+0000", 2024-05-08T08:55:26.231 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9900000000000001e-07 2024-05-08T08:55:26.231 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.231 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.231 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:26.231 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:15.450744+0000", 2024-05-08T08:55:26.232 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3259999999999998e-06 2024-05-08T08:55:26.232 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.232 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.232 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:26.232 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:15.450751+0000", 2024-05-08T08:55:26.232 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6129999999999998e-06 2024-05-08T08:55:26.232 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.232 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.232 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:26.232 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:15.450757+0000", 2024-05-08T08:55:26.232 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3129999999999998e-06 2024-05-08T08:55:26.232 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.232 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.232 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:26.232 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:15.450825+0000", 2024-05-08T08:55:26.233 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8307999999999997e-05 2024-05-08T08:55:26.233 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.233 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.233 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-05-08T08:55:26.233 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:15.450840+0000", 2024-05-08T08:55:26.233 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4968e-05 2024-05-08T08:55:26.233 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.233 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.233 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:26.233 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:16.374193+0000", 2024-05-08T08:55:26.233 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.92335363000000004 2024-05-08T08:55:26.233 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.233 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.233 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:26.233 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:16.374298+0000", 2024-05-08T08:55:26.233 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104759 2024-05-08T08:55:26.234 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.234 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.234 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-05-08T08:55:26.234 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:16.374372+0000", 2024-05-08T08:55:26.234 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4232000000000003e-05 2024-05-08T08:55:26.234 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.234 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.234 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:55:26.234 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:16.376496+0000", 2024-05-08T08:55:26.234 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0021231259999999999 2024-05-08T08:55:26.234 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.234 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.234 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:55:26.234 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:16.377065+0000", 2024-05-08T08:55:26.235 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00056950700000000002 2024-05-08T08:55:26.235 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.235 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.235 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:55:26.235 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:16.377100+0000", 2024-05-08T08:55:26.235 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4579999999999998e-05 2024-05-08T08:55:26.235 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.235 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.235 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:26.235 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:16.377129+0000", 2024-05-08T08:55:26.235 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9454e-05 2024-05-08T08:55:26.235 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.235 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:26.235 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.236 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.236 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.236 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4416.0:34124 3.12 3:4f2dce52:::rbd_data.1140100c9703.0000000000000015:head [delete] snapc 14=[] ondisk+write+known_if_redirected+supports_pool_eio e164)", 2024-05-08T08:55:26.236 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:55:15.458266+0000", 2024-05-08T08:55:26.236 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 10.699660451, 2024-05-08T08:55:26.236 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.925342255, 2024-05-08T08:55:26.236 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:26.236 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:26.236 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:55:26.236 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:26.236 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4416", 2024-05-08T08:55:26.236 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:55:26.236 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 34124 2024-05-08T08:55:26.236 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.236 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:26.236 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.237 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:26.237 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:15.458266+0000", 2024-05-08T08:55:26.237 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:26.237 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.237 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.237 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:26.237 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:15.458266+0000", 2024-05-08T08:55:26.237 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:26.237 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.237 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.237 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:26.237 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:15.458267+0000", 2024-05-08T08:55:26.237 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9900000000000001e-07 2024-05-08T08:55:26.237 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.238 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.238 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:26.238 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:15.458269+0000", 2024-05-08T08:55:26.238 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6520000000000002e-06 2024-05-08T08:55:26.238 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.238 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.238 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:26.238 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:15.458276+0000", 2024-05-08T08:55:26.238 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1779999999999997e-06 2024-05-08T08:55:26.238 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.238 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.238 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:26.238 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:15.458283+0000", 2024-05-08T08:55:26.238 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7680000000000003e-06 2024-05-08T08:55:26.239 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.239 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.239 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:26.239 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:15.458330+0000", 2024-05-08T08:55:26.239 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7111999999999999e-05 2024-05-08T08:55:26.239 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.239 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.239 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-05-08T08:55:26.239 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:15.458340+0000", 2024-05-08T08:55:26.239 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5680000000000005e-06 2024-05-08T08:55:26.239 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.239 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.239 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:26.239 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:16.375925+0000", 2024-05-08T08:55:26.240 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.91758470599999997 2024-05-08T08:55:26.240 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.240 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.240 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:26.240 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:16.376000+0000", 2024-05-08T08:55:26.240 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5325000000000001e-05 2024-05-08T08:55:26.240 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.240 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.240 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-05-08T08:55:26.240 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:16.376062+0000", 2024-05-08T08:55:26.240 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1630000000000005e-05 2024-05-08T08:55:26.240 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.240 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.240 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:55:26.240 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:16.380141+0000", 2024-05-08T08:55:26.241 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0040797530000000002 2024-05-08T08:55:26.241 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.241 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.241 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:55:26.241 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:16.383551+0000", 2024-05-08T08:55:26.241 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0034098319999999998 2024-05-08T08:55:26.241 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.241 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.241 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:55:26.241 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:16.383583+0000", 2024-05-08T08:55:26.241 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2140000000000001e-05 2024-05-08T08:55:26.241 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.241 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.241 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:26.241 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:16.383608+0000", 2024-05-08T08:55:26.241 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5092e-05 2024-05-08T08:55:26.242 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.242 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:26.242 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.242 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.242 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.242 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4416.0:34130 3.12 3:4fed2d06:::rbd_data.1140100c9703.0000000000000031:head [delete] snapc 14=[] ondisk+write+known_if_redirected+supports_pool_eio e164)", 2024-05-08T08:55:26.242 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:55:15.460999+0000", 2024-05-08T08:55:26.242 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 10.696927389000001, 2024-05-08T08:55:26.242 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.92270473099999994, 2024-05-08T08:55:26.242 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:26.242 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:26.242 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:55:26.242 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:26.242 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4416", 2024-05-08T08:55:26.242 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:55:26.243 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 34130 2024-05-08T08:55:26.243 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.243 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:26.243 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.243 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:26.243 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:15.460999+0000", 2024-05-08T08:55:26.243 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:26.243 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.243 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.243 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:26.243 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:15.460999+0000", 2024-05-08T08:55:26.243 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:26.243 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.243 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.243 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:26.243 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:15.461000+0000", 2024-05-08T08:55:26.244 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1999999999999999e-07 2024-05-08T08:55:26.244 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.244 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.244 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:26.244 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:15.461004+0000", 2024-05-08T08:55:26.244 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1620000000000001e-06 2024-05-08T08:55:26.244 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.244 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.244 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:26.244 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:15.461016+0000", 2024-05-08T08:55:26.244 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2269999999999999e-05 2024-05-08T08:55:26.244 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.244 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.244 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:26.244 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:15.461024+0000", 2024-05-08T08:55:26.245 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6450000000000005e-06 2024-05-08T08:55:26.245 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.245 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.245 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:26.245 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:15.461077+0000", 2024-05-08T08:55:26.245 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3242000000000003e-05 2024-05-08T08:55:26.245 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.245 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.245 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-05-08T08:55:26.245 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:15.461088+0000", 2024-05-08T08:55:26.245 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0808999999999999e-05 2024-05-08T08:55:26.245 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.245 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.245 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:26.245 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:16.376527+0000", 2024-05-08T08:55:26.245 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.91543937200000003 2024-05-08T08:55:26.245 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.246 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.246 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:26.246 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:16.376599+0000", 2024-05-08T08:55:26.246 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1298999999999996e-05 2024-05-08T08:55:26.246 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.246 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.246 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-05-08T08:55:26.246 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:16.376657+0000", 2024-05-08T08:55:26.246 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8418000000000002e-05 2024-05-08T08:55:26.246 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.246 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.246 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:55:26.246 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:16.380144+0000", 2024-05-08T08:55:26.246 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003487066 2024-05-08T08:55:26.246 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.247 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.247 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:55:26.247 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:16.383639+0000", 2024-05-08T08:55:26.247 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0034950570000000002 2024-05-08T08:55:26.247 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.247 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.247 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:55:26.247 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:16.383670+0000", 2024-05-08T08:55:26.247 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1065999999999997e-05 2024-05-08T08:55:26.247 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.247 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.247 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:26.247 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:16.383704+0000", 2024-05-08T08:55:26.247 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3704999999999997e-05 2024-05-08T08:55:26.247 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.248 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:26.248 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.248 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.248 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:26.248 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:55:26.249 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:55:26.250 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:55:26.251+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:55:26.350 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:55:26.353+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:55:26.354 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:55:26.354 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:55:26.354 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:55:26.355 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:55:26.363 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:55:26.441 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-05-08T08:55:26.451 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:55:26.454+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:55:26.466 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T08:55:26.476 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:55:26.477 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:55:26.477 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:55:26.477 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:55:26.477 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:55:26.486 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:55:26.552 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:55:26.554+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:55:26.629 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:55:26.629 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:55:26.638 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:55:26.638 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:55:26.638 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:55:26.639 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:55:26.639 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.639 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22073 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 782336~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:26.639 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:55:26.639 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 178.78259981900001, 2024-05-08T08:55:26.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047244234000000003, 2024-05-08T08:55:26.639 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:26.639 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:26.639 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:26.639 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:26.639 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:26.639 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:26.639 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22073 2024-05-08T08:55:26.639 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.639 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:26.640 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.640 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:26.640 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:55:26.640 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:26.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.640 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.640 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:26.640 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:55:26.640 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:26.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.640 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.640 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:26.640 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857668+0000", 2024-05-08T08:55:26.640 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7800000000000002e-07 2024-05-08T08:55:26.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.641 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:26.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857670+0000", 2024-05-08T08:55:26.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0820000000000001e-06 2024-05-08T08:55:26.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.641 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:26.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857678+0000", 2024-05-08T08:55:26.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7589999999999997e-06 2024-05-08T08:55:26.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.641 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:26.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857691+0000", 2024-05-08T08:55:26.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3672999999999999e-05 2024-05-08T08:55:26.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:26.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904780+0000", 2024-05-08T08:55:26.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047088616999999999 2024-05-08T08:55:26.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:26.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904840+0000", 2024-05-08T08:55:26.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0356999999999998e-05 2024-05-08T08:55:26.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:26.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904912+0000", 2024-05-08T08:55:26.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1367999999999998e-05 2024-05-08T08:55:26.643 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.643 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:26.643 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.643 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.643 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.643 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22075 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 798720~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:26.643 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:55:26.643 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 178.78256942199999, 2024-05-08T08:55:26.643 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047384643999999997, 2024-05-08T08:55:26.643 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:26.643 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:26.643 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:26.643 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:26.643 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:26.643 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:26.644 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22075 2024-05-08T08:55:26.644 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.644 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:26.644 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.644 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:26.644 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:55:26.644 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:26.644 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.644 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.644 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:26.644 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:55:26.644 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:26.644 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.644 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.644 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:26.644 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:55:26.645 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7399999999999999e-07 2024-05-08T08:55:26.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.645 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.645 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:26.645 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857705+0000", 2024-05-08T08:55:26.645 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1980000000000004e-06 2024-05-08T08:55:26.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.645 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.645 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:26.645 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857713+0000", 2024-05-08T08:55:26.645 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7039999999999994e-06 2024-05-08T08:55:26.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.645 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.645 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:26.645 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857718+0000", 2024-05-08T08:55:26.646 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.865e-06 2024-05-08T08:55:26.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.646 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:26.646 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904941+0000", 2024-05-08T08:55:26.646 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047223198000000001 2024-05-08T08:55:26.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.646 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:26.646 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904999+0000", 2024-05-08T08:55:26.646 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8254999999999997e-05 2024-05-08T08:55:26.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.646 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:26.646 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905082+0000", 2024-05-08T08:55:26.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3049999999999999e-05 2024-05-08T08:55:26.647 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.647 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:26.647 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.647 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.647 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.647 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22077 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 815104~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:26.647 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:55:26.647 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 178.78253141100001, 2024-05-08T08:55:26.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047519448999999998, 2024-05-08T08:55:26.647 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:26.647 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:26.647 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:26.647 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:26.647 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:26.647 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:26.648 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22077 2024-05-08T08:55:26.648 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.648 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:26.648 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.648 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:26.648 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:55:26.648 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:26.648 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.648 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.648 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:26.648 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:55:26.649 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:26.649 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.649 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.649 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:26.649 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:55:26.649 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.99e-07 2024-05-08T08:55:26.649 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.649 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.649 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:26.649 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857739+0000", 2024-05-08T08:55:26.649 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3599999999999999e-06 2024-05-08T08:55:26.649 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.649 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.649 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:26.650 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857745+0000", 2024-05-08T08:55:26.650 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5139999999999998e-06 2024-05-08T08:55:26.650 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.650 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.650 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:26.650 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857750+0000", 2024-05-08T08:55:26.650 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8550000000000001e-06 2024-05-08T08:55:26.650 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.650 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.650 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:26.650 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905110+0000", 2024-05-08T08:55:26.650 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047360126000000002 2024-05-08T08:55:26.650 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.650 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.650 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:26.651 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905175+0000", 2024-05-08T08:55:26.651 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4689000000000004e-05 2024-05-08T08:55:26.651 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.651 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.651 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:26.651 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905255+0000", 2024-05-08T08:55:26.651 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0306e-05 2024-05-08T08:55:26.651 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.651 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:26.651 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.651 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.651 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.651 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22079 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 831488~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:26.651 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:55:26.651 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 178.782510863, 2024-05-08T08:55:26.651 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.04767499, 2024-05-08T08:55:26.651 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:26.652 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:26.652 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:26.652 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:26.652 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:26.652 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:26.652 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22079 2024-05-08T08:55:26.652 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.652 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:26.652 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.652 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:26.652 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:55:26.652 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:26.652 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.652 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.653 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:26.653 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:55:26.653 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:26.653 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.653 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.653 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:26.653 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857757+0000", 2024-05-08T08:55:26.653 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8299999999999997e-07 2024-05-08T08:55:26.653 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.653 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.653 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:26.653 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857759+0000", 2024-05-08T08:55:26.653 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1390000000000002e-06 2024-05-08T08:55:26.653 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.653 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.653 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:26.654 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857765+0000", 2024-05-08T08:55:26.654 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4609999999999996e-06 2024-05-08T08:55:26.654 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.654 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.654 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:26.654 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857770+0000", 2024-05-08T08:55:26.654 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6689999999999997e-06 2024-05-08T08:55:26.654 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.654 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.654 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:26.654 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905281+0000", 2024-05-08T08:55:26.654 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047510668999999998 2024-05-08T08:55:26.654 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.654 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.655 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:26.655 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905344+0000", 2024-05-08T08:55:26.655 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2797000000000002e-05 2024-05-08T08:55:26.655 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.655 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.655 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:26.655 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905431+0000", 2024-05-08T08:55:26.655 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7771999999999999e-05 2024-05-08T08:55:26.655 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.655 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:26.655 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.655 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.655 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.655 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22082 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 847872~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:26.655 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:55:26.656 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 178.78249117199999, 2024-05-08T08:55:26.656 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047831934999999999, 2024-05-08T08:55:26.656 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:26.656 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:26.656 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:26.656 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:26.656 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:26.656 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:26.656 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22082 2024-05-08T08:55:26.656 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.656 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:26.656 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.656 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:26.656 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:55:26.656 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:26.657 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.657 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.657 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:26.657 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:55:26.657 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:26.657 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.657 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.657 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:26.657 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:55:26.657 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:55:26.657 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.657 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.657 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:26.657 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857778+0000", 2024-05-08T08:55:26.657 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0789999999999999e-06 2024-05-08T08:55:26.657 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.658 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.658 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:26.658 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857784+0000", 2024-05-08T08:55:26.658 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0480000000000004e-06 2024-05-08T08:55:26.658 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.658 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.658 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:26.658 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857790+0000", 2024-05-08T08:55:26.658 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2719999999999997e-06 2024-05-08T08:55:26.658 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.658 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.658 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:26.658 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905458+0000", 2024-05-08T08:55:26.659 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047668470999999997 2024-05-08T08:55:26.659 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.659 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.659 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:26.659 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905521+0000", 2024-05-08T08:55:26.659 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2952000000000003e-05 2024-05-08T08:55:26.659 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.659 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.659 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:26.659 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905608+0000", 2024-05-08T08:55:26.659 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6759000000000001e-05 2024-05-08T08:55:26.659 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.660 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:26.660 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.660 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.660 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.660 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22084 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 864256~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:26.660 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:55:26.660 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 178.78246722200001, 2024-05-08T08:55:26.660 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047977748000000001, 2024-05-08T08:55:26.660 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:26.660 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:26.660 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:26.660 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:26.660 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:26.660 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:26.660 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22084 2024-05-08T08:55:26.661 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.661 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:26.661 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.661 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:26.661 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:55:26.661 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:26.661 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.661 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.661 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:26.661 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:55:26.661 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:26.661 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.661 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.661 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:26.661 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:55:26.662 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:55:26.662 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.662 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.662 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:26.662 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857802+0000", 2024-05-08T08:55:26.662 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0779999999999998e-06 2024-05-08T08:55:26.662 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.662 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.662 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:26.662 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857812+0000", 2024-05-08T08:55:26.662 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7939999999999997e-06 2024-05-08T08:55:26.662 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.663 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.663 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:26.663 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857817+0000", 2024-05-08T08:55:26.663 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8899999999999998e-06 2024-05-08T08:55:26.663 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.663 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.663 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:26.663 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905636+0000", 2024-05-08T08:55:26.663 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047819252 2024-05-08T08:55:26.663 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.663 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.663 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:26.663 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905700+0000", 2024-05-08T08:55:26.663 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4016999999999994e-05 2024-05-08T08:55:26.663 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.663 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.664 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:26.664 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905778+0000", 2024-05-08T08:55:26.664 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7306999999999997e-05 2024-05-08T08:55:26.664 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.664 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:26.664 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.664 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.664 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.664 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22086 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 880640~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:26.664 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:55:26.664 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 178.78244387199999, 2024-05-08T08:55:26.664 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048119149, 2024-05-08T08:55:26.664 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:26.664 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:26.664 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:26.664 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:26.665 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:26.665 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:26.665 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22086 2024-05-08T08:55:26.665 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.665 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:26.665 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.665 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:26.665 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:55:26.665 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:26.665 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.665 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.665 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:26.665 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:55:26.665 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:26.666 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.666 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.666 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:26.666 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857824+0000", 2024-05-08T08:55:26.666 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4200000000000002e-07 2024-05-08T08:55:26.666 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.666 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.666 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:26.666 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857826+0000", 2024-05-08T08:55:26.666 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0590000000000001e-06 2024-05-08T08:55:26.666 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.666 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.666 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:26.666 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857832+0000", 2024-05-08T08:55:26.667 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8029999999999999e-06 2024-05-08T08:55:26.667 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.667 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.667 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:26.667 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857837+0000", 2024-05-08T08:55:26.667 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7929999999999997e-06 2024-05-08T08:55:26.667 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.667 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.667 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:26.667 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905803+0000", 2024-05-08T08:55:26.667 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047965761000000003 2024-05-08T08:55:26.667 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.667 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.667 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:26.668 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905862+0000", 2024-05-08T08:55:26.668 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9203000000000002e-05 2024-05-08T08:55:26.668 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.668 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.668 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:26.668 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905942+0000", 2024-05-08T08:55:26.668 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0187999999999999e-05 2024-05-08T08:55:26.668 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.668 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:26.668 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.668 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.668 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.668 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22088 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 897024~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:26.668 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:55:26.669 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 178.78242381999999, 2024-05-08T08:55:26.669 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048256049000000002, 2024-05-08T08:55:26.669 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:26.669 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:26.669 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:26.669 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:26.669 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:26.670 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:26.670 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22088 2024-05-08T08:55:26.670 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.670 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:26.670 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.670 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:26.670 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:55:26.670 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:26.671 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.671 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.671 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:26.671 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:55:26.671 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:26.671 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.671 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.671 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:26.671 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857844+0000", 2024-05-08T08:55:26.671 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.46e-07 2024-05-08T08:55:26.671 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.671 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.671 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:26.671 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857846+0000", 2024-05-08T08:55:26.672 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0899999999999999e-06 2024-05-08T08:55:26.672 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.672 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.672 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:26.672 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857855+0000", 2024-05-08T08:55:26.672 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8219999999999996e-06 2024-05-08T08:55:26.672 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.672 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.672 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:26.672 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857860+0000", 2024-05-08T08:55:26.672 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2989999999999996e-06 2024-05-08T08:55:26.672 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.672 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.672 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:26.672 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905966+0000", 2024-05-08T08:55:26.673 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048106542000000002 2024-05-08T08:55:26.673 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.673 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.673 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:26.673 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906024+0000", 2024-05-08T08:55:26.673 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7958000000000003e-05 2024-05-08T08:55:26.673 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.673 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.673 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:26.673 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906099+0000", 2024-05-08T08:55:26.673 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4992000000000003e-05 2024-05-08T08:55:26.673 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.674 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:26.674 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.674 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.674 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.674 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22091 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 913408~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:26.674 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:55:26.674 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 178.78240005999999, 2024-05-08T08:55:26.674 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048395290000000001, 2024-05-08T08:55:26.674 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:26.674 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:26.674 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:26.674 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:26.674 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:26.674 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:26.674 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22091 2024-05-08T08:55:26.675 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.675 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:26.675 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.675 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:26.675 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:55:26.675 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:26.675 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.675 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.675 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:26.675 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:55:26.675 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:26.675 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.675 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.675 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:26.676 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:55:26.676 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4900000000000001e-07 2024-05-08T08:55:26.676 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.676 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.676 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:26.676 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857870+0000", 2024-05-08T08:55:26.676 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2510000000000001e-06 2024-05-08T08:55:26.676 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.676 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.676 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:26.676 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857877+0000", 2024-05-08T08:55:26.676 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0809999999999996e-06 2024-05-08T08:55:26.676 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.676 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.677 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:26.677 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857881+0000", 2024-05-08T08:55:26.677 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.724e-06 2024-05-08T08:55:26.677 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.677 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.677 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:26.677 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906127+0000", 2024-05-08T08:55:26.677 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048245345000000002 2024-05-08T08:55:26.677 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.677 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.677 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:26.677 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906190+0000", 2024-05-08T08:55:26.677 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2762000000000007e-05 2024-05-08T08:55:26.677 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.677 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.677 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:26.678 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906262+0000", 2024-05-08T08:55:26.678 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2778e-05 2024-05-08T08:55:26.678 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.678 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:26.678 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.678 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.678 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.678 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22093 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 929792~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:26.678 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:55:26.678 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 178.782377696, 2024-05-08T08:55:26.678 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048532875000000003, 2024-05-08T08:55:26.678 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:26.678 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:26.678 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:26.679 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:26.679 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:26.679 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:26.679 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22093 2024-05-08T08:55:26.679 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.679 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:26.679 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.679 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:26.679 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:55:26.679 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:26.679 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.679 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.679 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:26.679 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:55:26.679 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:26.680 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.680 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.680 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:26.680 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857890+0000", 2024-05-08T08:55:26.680 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-07 2024-05-08T08:55:26.680 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.680 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.680 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:26.680 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857899+0000", 2024-05-08T08:55:26.680 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6889999999999999e-06 2024-05-08T08:55:26.680 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.680 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.680 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:26.680 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857906+0000", 2024-05-08T08:55:26.680 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6490000000000002e-06 2024-05-08T08:55:26.681 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.681 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.681 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:26.681 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857917+0000", 2024-05-08T08:55:26.681 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0764999999999999e-05 2024-05-08T08:55:26.681 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.681 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.681 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:26.681 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906287+0000", 2024-05-08T08:55:26.681 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048370442 2024-05-08T08:55:26.681 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.681 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.681 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:26.681 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906349+0000", 2024-05-08T08:55:26.681 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1537000000000004e-05 2024-05-08T08:55:26.681 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.682 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.682 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:26.682 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906422+0000", 2024-05-08T08:55:26.682 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3431999999999998e-05 2024-05-08T08:55:26.682 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.682 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:26.682 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.682 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.682 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.682 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22095 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 946176~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:26.682 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:55:26.682 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 178.782342992, 2024-05-08T08:55:26.682 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048657593999999998, 2024-05-08T08:55:26.682 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:26.682 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:26.683 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:26.683 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:26.683 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:26.683 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:26.683 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22095 2024-05-08T08:55:26.683 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.683 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:26.683 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.683 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:26.683 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:55:26.684 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:26.684 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.684 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.684 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:26.684 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:55:26.684 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:26.684 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.684 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.684 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:26.684 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857925+0000", 2024-05-08T08:55:26.684 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3099999999999998e-07 2024-05-08T08:55:26.684 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.684 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.684 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:26.684 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857927+0000", 2024-05-08T08:55:26.684 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.34e-06 2024-05-08T08:55:26.685 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.685 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.685 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:26.685 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857938+0000", 2024-05-08T08:55:26.685 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1426e-05 2024-05-08T08:55:26.685 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.685 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.685 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:26.685 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857943+0000", 2024-05-08T08:55:26.685 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5140000000000003e-06 2024-05-08T08:55:26.685 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.685 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.685 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:26.685 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906448+0000", 2024-05-08T08:55:26.685 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048504641000000001 2024-05-08T08:55:26.685 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.686 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.686 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:26.686 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906510+0000", 2024-05-08T08:55:26.686 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2663000000000006e-05 2024-05-08T08:55:26.686 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.686 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.686 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:26.686 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906582+0000", 2024-05-08T08:55:26.686 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1478999999999998e-05 2024-05-08T08:55:26.686 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.686 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:26.686 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.686 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.686 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.686 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22097 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 962560~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:26.687 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:55:26.687 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 178.78231530299999, 2024-05-08T08:55:26.687 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048791353000000003, 2024-05-08T08:55:26.687 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:26.687 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:26.687 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:26.687 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:26.687 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:26.687 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:26.687 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22097 2024-05-08T08:55:26.687 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.687 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:26.687 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.687 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:26.687 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:55:26.688 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:26.688 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.688 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.688 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:26.688 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:55:26.688 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:26.688 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.688 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.688 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:26.688 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:55:26.688 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9200000000000002e-07 2024-05-08T08:55:26.688 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.688 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.688 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:26.688 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857954+0000", 2024-05-08T08:55:26.689 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1900000000000002e-06 2024-05-08T08:55:26.689 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.689 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.689 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:26.689 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857961+0000", 2024-05-08T08:55:26.689 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4989999999999999e-06 2024-05-08T08:55:26.689 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.689 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.689 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:26.689 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857971+0000", 2024-05-08T08:55:26.689 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0485999999999999e-05 2024-05-08T08:55:26.689 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.689 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.689 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:26.689 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906607+0000", 2024-05-08T08:55:26.690 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048635109000000003 2024-05-08T08:55:26.690 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.690 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.690 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:26.690 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906667+0000", 2024-05-08T08:55:26.690 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0439000000000002e-05 2024-05-08T08:55:26.690 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.690 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.690 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:26.690 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906743+0000", 2024-05-08T08:55:26.690 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6237999999999998e-05 2024-05-08T08:55:26.690 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.690 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:26.690 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.690 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.690 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.691 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22099 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 978944~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:26.691 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:55:26.691 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 178.78228914300001, 2024-05-08T08:55:26.691 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048932055000000002, 2024-05-08T08:55:26.691 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:26.691 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:26.691 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:26.691 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:26.691 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:26.691 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:26.691 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22099 2024-05-08T08:55:26.691 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.691 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:26.691 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.691 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:26.691 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:55:26.692 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:26.692 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.692 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.692 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:26.692 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:55:26.692 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:26.692 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.692 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.692 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:26.692 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:55:26.692 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.27e-07 2024-05-08T08:55:26.692 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.692 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.692 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:26.692 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857987+0000", 2024-05-08T08:55:26.693 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5890000000000003e-06 2024-05-08T08:55:26.693 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.693 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.693 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:26.693 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857994+0000", 2024-05-08T08:55:26.693 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6619999999999999e-06 2024-05-08T08:55:26.693 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.693 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.693 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:26.693 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857998+0000", 2024-05-08T08:55:26.693 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7110000000000003e-06 2024-05-08T08:55:26.693 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.693 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.693 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:26.693 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906772+0000", 2024-05-08T08:55:26.694 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048773415000000001 2024-05-08T08:55:26.694 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.694 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.694 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:26.694 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906834+0000", 2024-05-08T08:55:26.694 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1752000000000001e-05 2024-05-08T08:55:26.694 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.694 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.694 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:26.694 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906910+0000", 2024-05-08T08:55:26.694 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6499000000000001e-05 2024-05-08T08:55:26.694 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.694 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:26.694 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.695 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.695 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.695 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22101 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 995328~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:26.695 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:55:26.695 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 178.78226201199999, 2024-05-08T08:55:26.695 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049087364000000001, 2024-05-08T08:55:26.695 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:26.695 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:26.695 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:26.695 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:26.695 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:26.695 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:26.695 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22101 2024-05-08T08:55:26.695 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.695 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:26.696 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.696 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:26.696 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:55:26.696 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:26.696 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.696 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.696 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:26.696 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:55:26.696 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:26.696 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.696 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.696 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:26.696 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858006+0000", 2024-05-08T08:55:26.696 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1399999999999997e-07 2024-05-08T08:55:26.696 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.696 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.697 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:26.697 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858008+0000", 2024-05-08T08:55:26.697 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2060000000000001e-06 2024-05-08T08:55:26.697 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.697 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.697 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:26.697 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858017+0000", 2024-05-08T08:55:26.697 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1579999999999994e-06 2024-05-08T08:55:26.697 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.697 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.697 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:26.697 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858022+0000", 2024-05-08T08:55:26.697 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4199999999999998e-06 2024-05-08T08:55:26.697 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.697 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.698 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:26.698 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906940+0000", 2024-05-08T08:55:26.698 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048917627999999998 2024-05-08T08:55:26.698 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.698 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.698 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:26.698 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907002+0000", 2024-05-08T08:55:26.698 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2399999999999999e-05 2024-05-08T08:55:26.698 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.698 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.698 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:26.698 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907092+0000", 2024-05-08T08:55:26.698 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0137999999999997e-05 2024-05-08T08:55:26.699 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.699 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:26.699 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.699 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.699 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.699 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22103 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1011712~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:26.699 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:55:26.699 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 178.782237227, 2024-05-08T08:55:26.699 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049228542, 2024-05-08T08:55:26.699 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:26.699 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:26.699 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:26.699 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:26.700 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:26.700 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:26.700 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22103 2024-05-08T08:55:26.700 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.700 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:26.700 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.700 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:26.700 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:55:26.700 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:26.700 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.700 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.700 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:26.700 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:55:26.700 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:26.700 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.700 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.700 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:26.701 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:55:26.701 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4299999999999999e-07 2024-05-08T08:55:26.701 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.701 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.701 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:26.701 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858032+0000", 2024-05-08T08:55:26.701 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0590000000000001e-06 2024-05-08T08:55:26.701 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.701 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.701 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:26.701 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858037+0000", 2024-05-08T08:55:26.701 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.977e-06 2024-05-08T08:55:26.701 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.701 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.701 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:26.701 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858042+0000", 2024-05-08T08:55:26.701 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7650000000000001e-06 2024-05-08T08:55:26.702 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.702 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.702 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:26.702 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907118+0000", 2024-05-08T08:55:26.702 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049076147000000001 2024-05-08T08:55:26.702 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.702 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.702 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:26.702 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907180+0000", 2024-05-08T08:55:26.702 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1971999999999996e-05 2024-05-08T08:55:26.702 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.702 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.702 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:26.702 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907258+0000", 2024-05-08T08:55:26.702 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8279e-05 2024-05-08T08:55:26.703 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.703 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:26.703 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.703 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.703 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.703 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22105 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1028096~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:26.703 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:55:26.703 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 178.782217984, 2024-05-08T08:55:26.703 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049394609999999999, 2024-05-08T08:55:26.703 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:26.703 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:26.703 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:26.703 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:26.703 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:26.703 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:26.703 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22105 2024-05-08T08:55:26.704 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.704 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:26.704 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.704 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:26.704 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:55:26.704 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:26.704 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.704 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.704 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:26.704 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:55:26.704 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:26.704 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.704 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.704 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:26.704 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858050+0000", 2024-05-08T08:55:26.705 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5999999999999999e-07 2024-05-08T08:55:26.705 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.705 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.705 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:26.705 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858051+0000", 2024-05-08T08:55:26.705 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8870000000000001e-06 2024-05-08T08:55:26.705 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.705 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.705 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:26.705 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858057+0000", 2024-05-08T08:55:26.705 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8019999999999997e-06 2024-05-08T08:55:26.705 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.705 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.705 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:26.706 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858062+0000", 2024-05-08T08:55:26.706 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.561e-06 2024-05-08T08:55:26.706 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.706 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.706 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:26.706 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907289+0000", 2024-05-08T08:55:26.706 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049226957000000002 2024-05-08T08:55:26.706 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.706 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.706 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:26.706 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907356+0000", 2024-05-08T08:55:26.706 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6915000000000006e-05 2024-05-08T08:55:26.706 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.706 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.707 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:26.707 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907444+0000", 2024-05-08T08:55:26.707 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8127999999999994e-05 2024-05-08T08:55:26.707 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.707 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:26.707 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.707 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.707 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.707 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22107 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1044480~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:26.707 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:55:26.707 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 178.782199644, 2024-05-08T08:55:26.707 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049799942999999999, 2024-05-08T08:55:26.707 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:26.707 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:26.707 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:26.708 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:26.708 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:26.708 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:26.708 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22107 2024-05-08T08:55:26.708 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.708 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:26.708 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.708 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:26.708 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:55:26.708 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:26.708 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.708 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.708 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:26.708 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:55:26.708 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:26.708 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.709 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.709 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:26.709 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:55:26.709 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4799999999999999e-07 2024-05-08T08:55:26.709 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.709 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.709 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:26.709 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858070+0000", 2024-05-08T08:55:26.709 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.012e-06 2024-05-08T08:55:26.709 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.709 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.709 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:26.709 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858077+0000", 2024-05-08T08:55:26.709 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6290000000000001e-06 2024-05-08T08:55:26.709 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.709 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.710 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:26.710 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858083+0000", 2024-05-08T08:55:26.710 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8880000000000002e-06 2024-05-08T08:55:26.710 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.710 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.710 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:26.710 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907671+0000", 2024-05-08T08:55:26.710 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049587191000000003 2024-05-08T08:55:26.710 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.710 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.710 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:26.710 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907763+0000", 2024-05-08T08:55:26.710 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2231000000000005e-05 2024-05-08T08:55:26.710 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.711 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.711 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:26.711 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907867+0000", 2024-05-08T08:55:26.711 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104644 2024-05-08T08:55:26.711 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.711 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:26.711 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.711 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.711 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.711 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22294 3.1c 3:3aad0389:::rbd_data.11406886b07b.0000000000000080:head [stat,delete] snapc 7=[7] RETRY=6 ondisk+retry+write+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:26.711 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:55:26.711 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 178.78160579600001, 2024-05-08T08:55:26.711 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051152191, 2024-05-08T08:55:26.711 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:26.711 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:26.711 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:26.711 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:26.712 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:26.712 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:26.712 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22294 2024-05-08T08:55:26.712 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.712 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:26.712 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.712 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:26.712 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:55:26.712 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:26.712 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.712 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.712 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:26.712 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:55:26.712 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:26.712 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.712 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.713 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:26.713 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858662+0000", 2024-05-08T08:55:26.713 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4799999999999998e-07 2024-05-08T08:55:26.713 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.713 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.713 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:26.713 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858664+0000", 2024-05-08T08:55:26.713 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1229999999999998e-06 2024-05-08T08:55:26.713 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.713 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.713 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:26.713 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858670+0000", 2024-05-08T08:55:26.713 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3099999999999997e-06 2024-05-08T08:55:26.713 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.713 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.713 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:26.714 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858676+0000", 2024-05-08T08:55:26.714 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9750000000000004e-06 2024-05-08T08:55:26.714 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.714 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.714 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:26.714 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907901+0000", 2024-05-08T08:55:26.714 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049224520000000001 2024-05-08T08:55:26.714 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.714 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.714 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:26.714 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.908085+0000", 2024-05-08T08:55:26.714 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00018377500000000001 2024-05-08T08:55:26.714 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.714 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.715 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:26.715 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.909814+0000", 2024-05-08T08:55:26.715 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00172894 2024-05-08T08:55:26.715 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.715 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:26.715 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.715 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.715 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.715 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4416.0:13461 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-05-08T08:55:26.715 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:22.487456+0000", 2024-05-08T08:55:26.715 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 124.152810755, 2024-05-08T08:55:26.715 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.99102620799999996, 2024-05-08T08:55:26.715 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:26.715 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:26.715 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:26.715 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:26.716 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4416", 2024-05-08T08:55:26.716 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:55:26.716 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 13461 2024-05-08T08:55:26.716 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.716 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:26.716 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.716 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:26.716 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487456+0000", 2024-05-08T08:55:26.716 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:26.716 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.716 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.716 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:26.716 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487456+0000", 2024-05-08T08:55:26.716 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:26.717 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.717 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.717 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:26.717 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487458+0000", 2024-05-08T08:55:26.717 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5659999999999999e-06 2024-05-08T08:55:26.717 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.717 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.717 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:26.717 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487463+0000", 2024-05-08T08:55:26.717 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6890000000000003e-06 2024-05-08T08:55:26.717 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.717 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.717 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:26.717 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487472+0000", 2024-05-08T08:55:26.717 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7710000000000001e-06 2024-05-08T08:55:26.717 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.718 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.718 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:26.718 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487478+0000", 2024-05-08T08:55:26.718 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1390000000000003e-06 2024-05-08T08:55:26.718 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.718 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.718 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:26.718 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487520+0000", 2024-05-08T08:55:26.718 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2259999999999999e-05 2024-05-08T08:55:26.718 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.718 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.718 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-05-08T08:55:26.718 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487528+0000", 2024-05-08T08:55:26.719 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7479999999999993e-06 2024-05-08T08:55:26.719 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.719 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.719 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:26.719 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:23.478139+0000", 2024-05-08T08:55:26.719 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.99061165200000001 2024-05-08T08:55:26.719 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.719 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.719 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:26.719 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:23.478301+0000", 2024-05-08T08:55:26.719 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000161602 2024-05-08T08:55:26.719 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.719 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.719 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:26.719 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:23.478483+0000", 2024-05-08T08:55:26.719 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000181781 2024-05-08T08:55:26.720 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.720 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:26.720 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.720 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.720 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.720 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5752.0:404 3.1a 3:5f5e8104:::rbd_data.1140bc4a55c7.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 5175296~45056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-05-08T08:55:26.720 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:00.390863+0000", 2024-05-08T08:55:26.720 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 86.249404560000002, 2024-05-08T08:55:26.720 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.055189174000000001, 2024-05-08T08:55:26.720 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:26.720 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:26.720 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:55:26.720 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:26.720 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5752", 2024-05-08T08:55:26.720 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/3534419815", 2024-05-08T08:55:26.720 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 404 2024-05-08T08:55:26.721 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.721 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:26.721 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.721 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:26.721 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.390863+0000", 2024-05-08T08:55:26.721 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:26.721 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.721 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.721 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:26.721 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.390863+0000", 2024-05-08T08:55:26.721 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:26.721 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.721 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.721 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:26.721 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.390864+0000", 2024-05-08T08:55:26.721 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.471e-06 2024-05-08T08:55:26.721 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.722 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.722 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:26.722 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397475+0000", 2024-05-08T08:55:26.722 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0066109109999999997 2024-05-08T08:55:26.722 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.722 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.722 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:26.722 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397498+0000", 2024-05-08T08:55:26.722 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2674e-05 2024-05-08T08:55:26.722 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.722 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.722 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:26.722 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397508+0000", 2024-05-08T08:55:26.722 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9019999999999994e-06 2024-05-08T08:55:26.722 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.723 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.723 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:26.723 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397609+0000", 2024-05-08T08:55:26.723 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010121899999999999 2024-05-08T08:55:26.723 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.723 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.723 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:26.723 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397711+0000", 2024-05-08T08:55:26.723 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102489 2024-05-08T08:55:26.723 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.723 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.723 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-05-08T08:55:26.723 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397999+0000", 2024-05-08T08:55:26.723 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00028733799999999999 2024-05-08T08:55:26.723 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.723 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.724 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:55:26.724 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.432093+0000", 2024-05-08T08:55:26.724 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.034094865000000002 2024-05-08T08:55:26.724 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.724 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.724 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:55:26.724 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.445975+0000", 2024-05-08T08:55:26.724 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013881187 2024-05-08T08:55:26.724 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.724 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.724 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:55:26.724 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.446013+0000", 2024-05-08T08:55:26.724 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8155000000000003e-05 2024-05-08T08:55:26.724 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:26.724 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:26.724 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:26.725 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.446052+0000", 2024-05-08T08:55:26.725 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8962999999999999e-05 2024-05-08T08:55:26.725 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.725 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:26.725 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.725 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:26.725 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:26.725 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:55:26.726 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:55:26.726 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:55:26.727 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:55:26.727 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:55:26.728 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:55:26.728 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:55:26.729+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:55:26.728 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:55:26.729 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:55:26.729 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:55:26.729 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:55:26.729 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:55:26.730 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:55:26.827 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:55:26.829+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:55:26.873 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:26.873 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:26.882 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:55:26.966 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:55:26.969+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:55:27.027 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:27.027 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:27.028 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:55:27.030+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:55:27.036 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:55:27.128 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:55:27.131+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:55:27.181 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:27.181 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:27.190 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:55:27.231 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:55:27.234+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:55:27.305 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:27.305 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:27.315 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:55:27.330 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:55:27.333+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:55:27.424 INFO:teuthology.orchestra.run.smithi059.stderr:nodeep-scrub is set 2024-05-08T08:55:27.425 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:27.426 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:27.432 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:55:27.434+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:55:27.441 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:55:27.533 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:55:27.535+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:55:27.560 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:27.560 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:27.569 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:55:27.633 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:55:27.636+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:55:27.719 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:27.720 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:27.729 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:55:27.735 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:55:27.737+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:55:27.835 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:55:27.838+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:55:27.838 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:27.838 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:27.847 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:55:27.937 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:55:27.939+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:55:27.956 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:27.956 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:27.965 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:55:28.038 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:55:28.041+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:55:28.075 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:28.075 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:28.084 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:55:28.139 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:55:28.141+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:55:28.193 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:28.193 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:28.203 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:55:28.246 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:55:28.249+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:55:28.312 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:28.313 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:28.321 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:55:28.363 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:55:28.365+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:55:28.430 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:28.430 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:28.439 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:55:28.441 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:55:28.443+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:55:28.541 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:55:28.544+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:55:28.561 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:28.561 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:28.572 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:55:28.642 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:55:28.645+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:55:28.678 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:28.678 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:28.686 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:55:28.730 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T08:55:28.774 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:55:28.777+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:55:28.832 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:28.832 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:28.841 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:55:28.844 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:55:28.846+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:55:28.858 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:55:28.858 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:55:28.876 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:55:28.876 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:55:28.893 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:55:28.893 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:55:28.911 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:55:28.911 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:55:28.928 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:55:28.928 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:55:28.944 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:55:28.947+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:55:28.947 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:55:28.947 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:55:28.948 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:55:28.948 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:55:28.948 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:55:28.949 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:55:28.949 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:55:28.958 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:55:29.046 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:55:29.049+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:55:29.065 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:55:29.065 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:55:29.065 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:55:29.065 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:55:29.065 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.065 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22073 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 782336~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:29.065 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:55:29.065 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 181.20900333099999, 2024-05-08T08:55:29.065 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047244234000000003, 2024-05-08T08:55:29.065 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:29.065 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:29.065 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:29.066 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:29.066 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:29.066 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:29.066 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22073 2024-05-08T08:55:29.066 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.066 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:29.066 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.066 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:29.066 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:55:29.066 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:29.066 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.066 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.067 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:29.067 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:55:29.067 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:29.067 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.067 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.067 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:29.067 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857668+0000", 2024-05-08T08:55:29.067 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7800000000000002e-07 2024-05-08T08:55:29.067 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.067 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.067 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:29.067 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857670+0000", 2024-05-08T08:55:29.067 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0820000000000001e-06 2024-05-08T08:55:29.068 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.068 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.068 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:29.068 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857678+0000", 2024-05-08T08:55:29.068 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7589999999999997e-06 2024-05-08T08:55:29.068 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.068 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.068 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:29.068 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857691+0000", 2024-05-08T08:55:29.068 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3672999999999999e-05 2024-05-08T08:55:29.068 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.068 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.068 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:29.068 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904780+0000", 2024-05-08T08:55:29.069 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047088616999999999 2024-05-08T08:55:29.069 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.069 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.069 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:29.069 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904840+0000", 2024-05-08T08:55:29.069 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0356999999999998e-05 2024-05-08T08:55:29.069 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.069 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.069 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:29.069 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904912+0000", 2024-05-08T08:55:29.069 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1367999999999998e-05 2024-05-08T08:55:29.069 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.069 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:29.069 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.070 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.070 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.070 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22075 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 798720~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:29.070 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:55:29.070 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 181.208972934, 2024-05-08T08:55:29.070 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047384643999999997, 2024-05-08T08:55:29.070 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:29.070 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:29.070 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:29.070 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:29.070 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:29.070 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:29.070 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22075 2024-05-08T08:55:29.070 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.071 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:29.071 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.071 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:29.071 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:55:29.071 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:29.071 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.071 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.071 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:29.071 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:55:29.071 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:29.071 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.071 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.071 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:29.071 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:55:29.071 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7399999999999999e-07 2024-05-08T08:55:29.071 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.071 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.072 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:29.072 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857705+0000", 2024-05-08T08:55:29.072 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1980000000000004e-06 2024-05-08T08:55:29.072 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.072 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.072 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:29.072 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857713+0000", 2024-05-08T08:55:29.072 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7039999999999994e-06 2024-05-08T08:55:29.072 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.072 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.072 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:29.072 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857718+0000", 2024-05-08T08:55:29.072 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.865e-06 2024-05-08T08:55:29.072 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.073 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.073 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:29.073 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904941+0000", 2024-05-08T08:55:29.073 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047223198000000001 2024-05-08T08:55:29.073 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.073 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.073 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:29.073 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904999+0000", 2024-05-08T08:55:29.073 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8254999999999997e-05 2024-05-08T08:55:29.073 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.073 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.073 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:29.073 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905082+0000", 2024-05-08T08:55:29.073 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3049999999999999e-05 2024-05-08T08:55:29.074 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.074 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:29.074 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.074 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.074 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.074 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22077 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 815104~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:29.074 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:55:29.074 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 181.20893492299999, 2024-05-08T08:55:29.074 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047519448999999998, 2024-05-08T08:55:29.074 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:29.074 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:29.074 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:29.074 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:29.074 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:29.075 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:29.075 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22077 2024-05-08T08:55:29.075 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.075 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:29.075 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.075 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:29.075 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:55:29.075 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:29.075 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.075 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.075 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:29.075 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:55:29.075 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:29.076 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.076 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.076 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:29.076 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:55:29.076 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.99e-07 2024-05-08T08:55:29.076 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.076 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.076 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:29.076 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857739+0000", 2024-05-08T08:55:29.076 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3599999999999999e-06 2024-05-08T08:55:29.076 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.076 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.076 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:29.077 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857745+0000", 2024-05-08T08:55:29.077 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5139999999999998e-06 2024-05-08T08:55:29.077 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.077 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.077 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:29.077 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857750+0000", 2024-05-08T08:55:29.077 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8550000000000001e-06 2024-05-08T08:55:29.077 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.077 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.077 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:29.077 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905110+0000", 2024-05-08T08:55:29.077 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047360126000000002 2024-05-08T08:55:29.077 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.078 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.078 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:29.078 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905175+0000", 2024-05-08T08:55:29.078 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4689000000000004e-05 2024-05-08T08:55:29.078 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.078 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.078 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:29.078 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905255+0000", 2024-05-08T08:55:29.078 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0306e-05 2024-05-08T08:55:29.078 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.078 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:29.078 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.078 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.079 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.079 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22079 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 831488~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:29.079 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:55:29.079 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 181.20891437500001, 2024-05-08T08:55:29.079 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.04767499, 2024-05-08T08:55:29.079 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:29.079 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:29.079 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:29.079 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:29.079 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:29.079 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:29.079 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22079 2024-05-08T08:55:29.080 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.080 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:29.080 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.080 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:29.080 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:55:29.080 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:29.080 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.080 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.080 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:29.080 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:55:29.080 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:29.080 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.080 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.081 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:29.081 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857757+0000", 2024-05-08T08:55:29.081 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8299999999999997e-07 2024-05-08T08:55:29.081 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.081 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.081 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:29.081 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857759+0000", 2024-05-08T08:55:29.081 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1390000000000002e-06 2024-05-08T08:55:29.081 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.081 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.081 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:29.081 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857765+0000", 2024-05-08T08:55:29.081 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4609999999999996e-06 2024-05-08T08:55:29.081 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.081 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.081 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:29.081 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857770+0000", 2024-05-08T08:55:29.081 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6689999999999997e-06 2024-05-08T08:55:29.082 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.082 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.082 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:29.082 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905281+0000", 2024-05-08T08:55:29.082 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047510668999999998 2024-05-08T08:55:29.082 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.082 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.082 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:29.082 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905344+0000", 2024-05-08T08:55:29.082 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2797000000000002e-05 2024-05-08T08:55:29.082 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.082 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.082 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:29.082 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905431+0000", 2024-05-08T08:55:29.082 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7771999999999999e-05 2024-05-08T08:55:29.083 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.083 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:29.083 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.083 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.083 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.083 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22082 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 847872~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:29.083 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:55:29.083 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 181.208894684, 2024-05-08T08:55:29.083 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047831934999999999, 2024-05-08T08:55:29.083 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:29.083 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:29.083 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:29.083 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:29.083 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:29.083 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:29.084 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22082 2024-05-08T08:55:29.084 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.084 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:29.084 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.084 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:29.084 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:55:29.084 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:29.084 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.084 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.084 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:29.084 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:55:29.084 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:29.084 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.084 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.084 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:29.084 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:55:29.085 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:55:29.085 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.085 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.085 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:29.085 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857778+0000", 2024-05-08T08:55:29.085 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0789999999999999e-06 2024-05-08T08:55:29.085 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.085 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.085 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:29.085 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857784+0000", 2024-05-08T08:55:29.085 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0480000000000004e-06 2024-05-08T08:55:29.085 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.085 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.085 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:29.086 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857790+0000", 2024-05-08T08:55:29.086 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2719999999999997e-06 2024-05-08T08:55:29.086 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.086 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.086 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:29.086 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905458+0000", 2024-05-08T08:55:29.086 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047668470999999997 2024-05-08T08:55:29.086 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.086 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.086 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:29.086 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905521+0000", 2024-05-08T08:55:29.086 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2952000000000003e-05 2024-05-08T08:55:29.086 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.086 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.086 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:29.087 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905608+0000", 2024-05-08T08:55:29.087 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6759000000000001e-05 2024-05-08T08:55:29.087 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.087 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:29.087 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.087 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.087 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.087 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22084 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 864256~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:29.087 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:55:29.087 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 181.20887073399999, 2024-05-08T08:55:29.087 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047977748000000001, 2024-05-08T08:55:29.087 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:29.087 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:29.087 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:29.088 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:29.088 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:29.088 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:29.088 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22084 2024-05-08T08:55:29.088 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.088 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:29.088 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.088 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:29.088 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:55:29.088 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:29.088 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.088 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.088 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:29.088 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:55:29.088 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:29.088 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.088 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.089 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:29.089 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:55:29.089 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:55:29.089 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.089 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.089 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:29.089 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857802+0000", 2024-05-08T08:55:29.089 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0779999999999998e-06 2024-05-08T08:55:29.089 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.089 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.089 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:29.089 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857812+0000", 2024-05-08T08:55:29.089 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7939999999999997e-06 2024-05-08T08:55:29.089 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.089 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.089 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:29.090 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857817+0000", 2024-05-08T08:55:29.090 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8899999999999998e-06 2024-05-08T08:55:29.090 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.090 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.090 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:29.090 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905636+0000", 2024-05-08T08:55:29.090 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047819252 2024-05-08T08:55:29.090 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.090 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.090 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:29.090 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905700+0000", 2024-05-08T08:55:29.090 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4016999999999994e-05 2024-05-08T08:55:29.090 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.090 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.090 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:29.091 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905778+0000", 2024-05-08T08:55:29.091 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7306999999999997e-05 2024-05-08T08:55:29.091 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.091 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:29.091 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.091 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.091 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.091 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22086 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 880640~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:29.091 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:55:29.091 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 181.20884738399999, 2024-05-08T08:55:29.091 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048119149, 2024-05-08T08:55:29.091 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:29.091 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:29.091 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:29.091 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:29.092 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:29.092 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:29.092 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22086 2024-05-08T08:55:29.092 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.092 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:29.092 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.092 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:29.092 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:55:29.092 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:29.092 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.092 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.092 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:29.092 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:55:29.092 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:29.092 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.093 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.093 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:29.093 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857824+0000", 2024-05-08T08:55:29.093 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4200000000000002e-07 2024-05-08T08:55:29.093 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.093 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.093 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:29.093 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857826+0000", 2024-05-08T08:55:29.093 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0590000000000001e-06 2024-05-08T08:55:29.093 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.093 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.093 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:29.093 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857832+0000", 2024-05-08T08:55:29.093 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8029999999999999e-06 2024-05-08T08:55:29.093 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.094 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.094 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:29.094 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857837+0000", 2024-05-08T08:55:29.094 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7929999999999997e-06 2024-05-08T08:55:29.094 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.094 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.094 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:29.094 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905803+0000", 2024-05-08T08:55:29.094 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047965761000000003 2024-05-08T08:55:29.094 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.094 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.094 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:29.094 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905862+0000", 2024-05-08T08:55:29.094 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9203000000000002e-05 2024-05-08T08:55:29.094 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.095 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.095 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:29.095 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905942+0000", 2024-05-08T08:55:29.095 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0187999999999999e-05 2024-05-08T08:55:29.095 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.095 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:29.095 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.095 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.095 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.095 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22088 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 897024~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:29.095 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:55:29.095 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 181.208827332, 2024-05-08T08:55:29.095 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048256049000000002, 2024-05-08T08:55:29.095 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:29.095 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:29.095 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:29.095 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:29.096 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:29.096 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:29.096 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22088 2024-05-08T08:55:29.096 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.096 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:29.096 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.096 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:29.096 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:55:29.096 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:29.096 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.096 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.096 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:29.096 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:55:29.096 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:29.096 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.096 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.096 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:29.097 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857844+0000", 2024-05-08T08:55:29.097 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.46e-07 2024-05-08T08:55:29.097 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.097 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.097 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:29.097 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857846+0000", 2024-05-08T08:55:29.097 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0899999999999999e-06 2024-05-08T08:55:29.097 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.097 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.097 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:29.097 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857855+0000", 2024-05-08T08:55:29.097 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8219999999999996e-06 2024-05-08T08:55:29.097 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.097 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.097 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:29.097 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857860+0000", 2024-05-08T08:55:29.098 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2989999999999996e-06 2024-05-08T08:55:29.098 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.098 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.098 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:29.098 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905966+0000", 2024-05-08T08:55:29.098 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048106542000000002 2024-05-08T08:55:29.098 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.098 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.098 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:29.098 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906024+0000", 2024-05-08T08:55:29.099 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7958000000000003e-05 2024-05-08T08:55:29.099 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.099 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.099 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:29.099 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906099+0000", 2024-05-08T08:55:29.099 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4992000000000003e-05 2024-05-08T08:55:29.099 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.099 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:29.099 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.099 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.099 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.099 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22091 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 913408~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:29.099 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:55:29.099 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 181.20880357199999, 2024-05-08T08:55:29.099 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048395290000000001, 2024-05-08T08:55:29.100 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:29.100 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:29.100 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:29.100 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:29.100 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:29.100 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:29.100 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22091 2024-05-08T08:55:29.100 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.100 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:29.100 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.100 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:29.100 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:55:29.100 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:29.100 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.100 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.100 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:29.100 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:55:29.101 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:29.101 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.101 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.101 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:29.101 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:55:29.101 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4900000000000001e-07 2024-05-08T08:55:29.101 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.101 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.101 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:29.101 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857870+0000", 2024-05-08T08:55:29.101 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2510000000000001e-06 2024-05-08T08:55:29.101 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.101 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.101 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:29.101 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857877+0000", 2024-05-08T08:55:29.101 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0809999999999996e-06 2024-05-08T08:55:29.102 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.102 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.102 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:29.102 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857881+0000", 2024-05-08T08:55:29.102 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.724e-06 2024-05-08T08:55:29.102 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.102 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.102 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:29.102 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906127+0000", 2024-05-08T08:55:29.102 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048245345000000002 2024-05-08T08:55:29.102 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.102 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.102 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:29.102 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906190+0000", 2024-05-08T08:55:29.102 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2762000000000007e-05 2024-05-08T08:55:29.103 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.103 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.103 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:29.103 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906262+0000", 2024-05-08T08:55:29.103 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2778e-05 2024-05-08T08:55:29.103 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.103 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:29.103 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.103 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.103 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.103 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22093 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 929792~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:29.103 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:55:29.103 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 181.208781208, 2024-05-08T08:55:29.103 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048532875000000003, 2024-05-08T08:55:29.103 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:29.103 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:29.103 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:29.104 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:29.104 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:29.104 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:29.104 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22093 2024-05-08T08:55:29.104 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.104 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:29.104 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.104 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:29.104 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:55:29.104 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:29.104 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.104 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.104 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:29.104 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:55:29.104 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:29.104 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.105 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.105 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:29.105 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857890+0000", 2024-05-08T08:55:29.105 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-07 2024-05-08T08:55:29.105 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.105 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.105 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:29.105 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857899+0000", 2024-05-08T08:55:29.105 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6889999999999999e-06 2024-05-08T08:55:29.105 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.105 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.105 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:29.105 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857906+0000", 2024-05-08T08:55:29.105 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6490000000000002e-06 2024-05-08T08:55:29.105 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.105 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.105 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:29.106 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857917+0000", 2024-05-08T08:55:29.106 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0764999999999999e-05 2024-05-08T08:55:29.106 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.106 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.106 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:29.106 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906287+0000", 2024-05-08T08:55:29.106 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048370442 2024-05-08T08:55:29.106 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.106 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.106 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:29.106 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906349+0000", 2024-05-08T08:55:29.106 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1537000000000004e-05 2024-05-08T08:55:29.106 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.106 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.107 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:29.107 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906422+0000", 2024-05-08T08:55:29.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3431999999999998e-05 2024-05-08T08:55:29.107 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.107 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:29.107 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.107 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22095 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 946176~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:29.107 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:55:29.107 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 181.208746504, 2024-05-08T08:55:29.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048657593999999998, 2024-05-08T08:55:29.107 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:29.108 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:29.108 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:29.108 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:29.108 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:29.108 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:29.108 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22095 2024-05-08T08:55:29.108 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.108 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:29.108 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.108 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:29.108 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:55:29.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:29.108 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.108 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.109 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:29.109 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:55:29.109 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:29.109 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.109 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.109 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:29.109 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857925+0000", 2024-05-08T08:55:29.109 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3099999999999998e-07 2024-05-08T08:55:29.109 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.109 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.109 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:29.109 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857927+0000", 2024-05-08T08:55:29.109 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.34e-06 2024-05-08T08:55:29.109 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.110 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.110 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:29.110 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857938+0000", 2024-05-08T08:55:29.110 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1426e-05 2024-05-08T08:55:29.110 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.110 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.110 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:29.110 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857943+0000", 2024-05-08T08:55:29.110 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5140000000000003e-06 2024-05-08T08:55:29.110 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.110 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.110 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:29.110 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906448+0000", 2024-05-08T08:55:29.111 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048504641000000001 2024-05-08T08:55:29.111 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.111 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.111 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:29.111 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906510+0000", 2024-05-08T08:55:29.111 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2663000000000006e-05 2024-05-08T08:55:29.111 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.111 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.111 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:29.111 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906582+0000", 2024-05-08T08:55:29.111 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1478999999999998e-05 2024-05-08T08:55:29.111 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.111 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:29.112 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.112 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.112 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.112 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22097 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 962560~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:29.112 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:55:29.112 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 181.208718815, 2024-05-08T08:55:29.112 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048791353000000003, 2024-05-08T08:55:29.112 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:29.112 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:29.112 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:29.112 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:29.112 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:29.112 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:29.112 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22097 2024-05-08T08:55:29.113 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.113 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:29.113 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.113 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:29.113 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:55:29.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:29.113 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.113 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.113 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:29.113 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:55:29.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:29.113 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.113 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.113 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:29.114 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:55:29.114 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9200000000000002e-07 2024-05-08T08:55:29.114 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.114 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.114 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:29.114 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857954+0000", 2024-05-08T08:55:29.114 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1900000000000002e-06 2024-05-08T08:55:29.114 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.114 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.114 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:29.114 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857961+0000", 2024-05-08T08:55:29.114 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4989999999999999e-06 2024-05-08T08:55:29.114 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.114 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.115 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:29.115 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857971+0000", 2024-05-08T08:55:29.115 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0485999999999999e-05 2024-05-08T08:55:29.115 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.115 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.115 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:29.115 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906607+0000", 2024-05-08T08:55:29.116 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048635109000000003 2024-05-08T08:55:29.116 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.116 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.116 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:29.116 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906667+0000", 2024-05-08T08:55:29.116 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0439000000000002e-05 2024-05-08T08:55:29.116 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.116 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.116 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:29.116 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906743+0000", 2024-05-08T08:55:29.116 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6237999999999998e-05 2024-05-08T08:55:29.116 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.116 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:29.116 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.116 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.117 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.117 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22099 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 978944~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:29.117 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:55:29.117 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 181.20869265499999, 2024-05-08T08:55:29.117 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048932055000000002, 2024-05-08T08:55:29.117 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:29.117 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:29.117 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:29.117 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:29.117 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:29.117 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:29.117 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22099 2024-05-08T08:55:29.117 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.118 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:29.118 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.118 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:29.118 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:55:29.118 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:29.118 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.118 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.118 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:29.118 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:55:29.118 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:29.118 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.118 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.118 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:29.119 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:55:29.119 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.27e-07 2024-05-08T08:55:29.119 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.119 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.119 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:29.119 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857987+0000", 2024-05-08T08:55:29.119 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5890000000000003e-06 2024-05-08T08:55:29.119 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.119 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.119 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:29.119 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857994+0000", 2024-05-08T08:55:29.119 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6619999999999999e-06 2024-05-08T08:55:29.119 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.119 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.119 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:29.119 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857998+0000", 2024-05-08T08:55:29.120 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7110000000000003e-06 2024-05-08T08:55:29.120 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.120 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.120 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:29.120 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906772+0000", 2024-05-08T08:55:29.120 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048773415000000001 2024-05-08T08:55:29.120 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.120 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.120 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:29.120 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906834+0000", 2024-05-08T08:55:29.120 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1752000000000001e-05 2024-05-08T08:55:29.120 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.120 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.120 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:29.121 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906910+0000", 2024-05-08T08:55:29.121 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6499000000000001e-05 2024-05-08T08:55:29.121 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.121 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:29.121 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.121 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.121 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.121 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22101 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 995328~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:29.121 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:55:29.121 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 181.208665524, 2024-05-08T08:55:29.121 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049087364000000001, 2024-05-08T08:55:29.121 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:29.121 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:29.121 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:29.121 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:29.122 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:29.122 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:29.122 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22101 2024-05-08T08:55:29.122 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.122 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:29.122 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.122 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:29.122 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:55:29.122 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:29.122 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.122 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.122 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:29.122 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:55:29.122 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:29.123 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.123 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.123 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:29.123 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858006+0000", 2024-05-08T08:55:29.123 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1399999999999997e-07 2024-05-08T08:55:29.123 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.123 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.123 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:29.123 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858008+0000", 2024-05-08T08:55:29.123 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2060000000000001e-06 2024-05-08T08:55:29.123 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.123 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.123 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:29.123 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858017+0000", 2024-05-08T08:55:29.123 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1579999999999994e-06 2024-05-08T08:55:29.124 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.124 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.124 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:29.124 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858022+0000", 2024-05-08T08:55:29.124 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4199999999999998e-06 2024-05-08T08:55:29.124 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.124 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.124 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:29.124 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906940+0000", 2024-05-08T08:55:29.124 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048917627999999998 2024-05-08T08:55:29.124 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.124 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.124 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:29.124 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907002+0000", 2024-05-08T08:55:29.124 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2399999999999999e-05 2024-05-08T08:55:29.125 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.125 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.125 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:29.125 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907092+0000", 2024-05-08T08:55:29.125 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0137999999999997e-05 2024-05-08T08:55:29.125 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.125 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:29.125 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.125 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.125 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.125 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22103 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1011712~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:29.125 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:55:29.125 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 181.208640739, 2024-05-08T08:55:29.125 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049228542, 2024-05-08T08:55:29.125 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:29.125 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:29.125 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:29.126 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:29.126 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:29.126 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:29.126 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22103 2024-05-08T08:55:29.126 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.126 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:29.126 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.126 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:29.126 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:55:29.126 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:29.126 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.126 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.126 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:29.126 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:55:29.126 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:29.127 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.127 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.127 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:29.127 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:55:29.127 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4299999999999999e-07 2024-05-08T08:55:29.127 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.127 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.127 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:29.127 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858032+0000", 2024-05-08T08:55:29.127 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0590000000000001e-06 2024-05-08T08:55:29.127 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.127 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.127 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:29.127 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858037+0000", 2024-05-08T08:55:29.127 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.977e-06 2024-05-08T08:55:29.127 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.127 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.128 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:29.128 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858042+0000", 2024-05-08T08:55:29.128 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7650000000000001e-06 2024-05-08T08:55:29.128 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.128 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.128 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:29.128 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907118+0000", 2024-05-08T08:55:29.128 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049076147000000001 2024-05-08T08:55:29.128 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.128 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.128 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:29.128 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907180+0000", 2024-05-08T08:55:29.128 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1971999999999996e-05 2024-05-08T08:55:29.128 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.128 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.128 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:29.128 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907258+0000", 2024-05-08T08:55:29.129 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8279e-05 2024-05-08T08:55:29.129 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.129 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:29.129 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.129 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.129 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.129 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22105 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1028096~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:29.129 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:55:29.129 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 181.20862149600001, 2024-05-08T08:55:29.129 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049394609999999999, 2024-05-08T08:55:29.129 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:29.129 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:29.129 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:29.129 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:29.129 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:29.129 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:29.129 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22105 2024-05-08T08:55:29.130 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.130 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:29.130 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.130 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:29.130 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:55:29.130 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:29.130 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.130 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.130 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:29.130 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:55:29.130 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:29.130 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.131 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.131 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:29.131 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858050+0000", 2024-05-08T08:55:29.131 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5999999999999999e-07 2024-05-08T08:55:29.131 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.131 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.131 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:29.131 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858051+0000", 2024-05-08T08:55:29.131 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8870000000000001e-06 2024-05-08T08:55:29.131 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.131 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.131 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:29.131 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858057+0000", 2024-05-08T08:55:29.131 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8019999999999997e-06 2024-05-08T08:55:29.131 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.131 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.132 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:29.132 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858062+0000", 2024-05-08T08:55:29.132 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.561e-06 2024-05-08T08:55:29.132 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.132 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.132 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:29.132 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907289+0000", 2024-05-08T08:55:29.132 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049226957000000002 2024-05-08T08:55:29.132 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.132 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.132 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:29.132 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907356+0000", 2024-05-08T08:55:29.132 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6915000000000006e-05 2024-05-08T08:55:29.132 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.132 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.132 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:29.132 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907444+0000", 2024-05-08T08:55:29.133 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8127999999999994e-05 2024-05-08T08:55:29.133 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.133 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:29.133 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.133 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.133 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.133 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22107 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1044480~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:29.133 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:55:29.133 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 181.20860315600001, 2024-05-08T08:55:29.133 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049799942999999999, 2024-05-08T08:55:29.133 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:29.133 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:29.133 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:29.133 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:29.133 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:29.134 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:29.134 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22107 2024-05-08T08:55:29.134 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.134 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:29.134 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.134 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:29.134 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:55:29.134 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:29.134 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.135 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.135 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:29.135 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:55:29.135 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:29.135 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.135 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.135 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:29.135 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:55:29.135 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4799999999999999e-07 2024-05-08T08:55:29.135 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.135 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.135 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:29.135 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858070+0000", 2024-05-08T08:55:29.136 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.012e-06 2024-05-08T08:55:29.136 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.136 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.136 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:29.136 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858077+0000", 2024-05-08T08:55:29.136 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6290000000000001e-06 2024-05-08T08:55:29.136 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.136 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.136 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:29.136 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858083+0000", 2024-05-08T08:55:29.136 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8880000000000002e-06 2024-05-08T08:55:29.136 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.136 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.137 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:29.137 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907671+0000", 2024-05-08T08:55:29.137 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049587191000000003 2024-05-08T08:55:29.137 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.137 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.137 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:29.137 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907763+0000", 2024-05-08T08:55:29.137 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2231000000000005e-05 2024-05-08T08:55:29.137 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.137 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.137 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:29.137 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907867+0000", 2024-05-08T08:55:29.137 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104644 2024-05-08T08:55:29.138 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.138 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:29.138 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.138 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.138 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.138 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22294 3.1c 3:3aad0389:::rbd_data.11406886b07b.0000000000000080:head [stat,delete] snapc 7=[7] RETRY=6 ondisk+retry+write+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:29.138 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:55:29.138 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 181.20800930799999, 2024-05-08T08:55:29.138 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051152191, 2024-05-08T08:55:29.138 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:29.138 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:29.138 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:29.139 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:29.139 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:29.139 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:29.139 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22294 2024-05-08T08:55:29.139 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.139 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:29.139 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.139 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:29.139 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:55:29.139 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:29.139 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.139 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.140 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:29.140 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:55:29.140 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:29.140 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.140 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.140 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:29.140 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858662+0000", 2024-05-08T08:55:29.140 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4799999999999998e-07 2024-05-08T08:55:29.140 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.140 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.140 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:29.140 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858664+0000", 2024-05-08T08:55:29.140 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1229999999999998e-06 2024-05-08T08:55:29.141 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.141 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.141 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:29.141 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858670+0000", 2024-05-08T08:55:29.141 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3099999999999997e-06 2024-05-08T08:55:29.141 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.141 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.141 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:29.141 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858676+0000", 2024-05-08T08:55:29.141 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9750000000000004e-06 2024-05-08T08:55:29.141 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.142 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.142 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:29.142 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907901+0000", 2024-05-08T08:55:29.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049224520000000001 2024-05-08T08:55:29.142 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.142 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.142 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:29.142 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.908085+0000", 2024-05-08T08:55:29.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00018377500000000001 2024-05-08T08:55:29.142 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.142 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.142 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:29.143 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.909814+0000", 2024-05-08T08:55:29.143 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00172894 2024-05-08T08:55:29.143 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.143 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:29.143 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.143 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.143 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.143 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4416.0:13461 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-05-08T08:55:29.143 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:22.487456+0000", 2024-05-08T08:55:29.143 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 126.579214267, 2024-05-08T08:55:29.143 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.99102620799999996, 2024-05-08T08:55:29.143 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:29.143 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:29.143 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:29.143 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:29.144 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4416", 2024-05-08T08:55:29.144 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:55:29.144 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 13461 2024-05-08T08:55:29.144 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.144 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:29.144 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.144 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:29.144 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487456+0000", 2024-05-08T08:55:29.144 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:29.144 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.144 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.144 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:29.144 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487456+0000", 2024-05-08T08:55:29.144 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:29.144 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.144 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.144 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:29.145 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487458+0000", 2024-05-08T08:55:29.145 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5659999999999999e-06 2024-05-08T08:55:29.145 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.145 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.145 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:29.145 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487463+0000", 2024-05-08T08:55:29.145 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6890000000000003e-06 2024-05-08T08:55:29.145 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.145 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.145 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:29.145 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487472+0000", 2024-05-08T08:55:29.145 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7710000000000001e-06 2024-05-08T08:55:29.145 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.145 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.145 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:29.145 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487478+0000", 2024-05-08T08:55:29.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1390000000000003e-06 2024-05-08T08:55:29.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.146 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.146 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:29.146 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487520+0000", 2024-05-08T08:55:29.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2259999999999999e-05 2024-05-08T08:55:29.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.146 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.146 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-05-08T08:55:29.146 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487528+0000", 2024-05-08T08:55:29.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7479999999999993e-06 2024-05-08T08:55:29.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.146 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.146 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:29.146 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:23.478139+0000", 2024-05-08T08:55:29.147 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.99061165200000001 2024-05-08T08:55:29.147 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.147 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.147 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:29.147 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:23.478301+0000", 2024-05-08T08:55:29.147 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000161602 2024-05-08T08:55:29.147 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.147 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.147 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:29.147 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:23.478483+0000", 2024-05-08T08:55:29.147 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000181781 2024-05-08T08:55:29.147 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.147 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:29.147 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.147 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.147 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.147 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5752.0:404 3.1a 3:5f5e8104:::rbd_data.1140bc4a55c7.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 5175296~45056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-05-08T08:55:29.148 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:00.390863+0000", 2024-05-08T08:55:29.148 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 88.675808071999995, 2024-05-08T08:55:29.148 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.055189174000000001, 2024-05-08T08:55:29.148 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:29.148 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:29.148 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:55:29.148 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:29.148 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5752", 2024-05-08T08:55:29.148 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/3534419815", 2024-05-08T08:55:29.148 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 404 2024-05-08T08:55:29.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.148 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:29.148 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.148 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:29.148 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.390863+0000", 2024-05-08T08:55:29.148 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:29.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.149 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:29.149 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.390863+0000", 2024-05-08T08:55:29.149 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:29.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.149 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:29.149 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.390864+0000", 2024-05-08T08:55:29.149 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.471e-06 2024-05-08T08:55:29.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.150 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.150 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:29.150 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397475+0000", 2024-05-08T08:55:29.150 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0066109109999999997 2024-05-08T08:55:29.150 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.150 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.150 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:29.150 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397498+0000", 2024-05-08T08:55:29.150 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2674e-05 2024-05-08T08:55:29.150 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.150 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.150 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:29.150 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397508+0000", 2024-05-08T08:55:29.150 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9019999999999994e-06 2024-05-08T08:55:29.150 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.151 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.151 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:29.151 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397609+0000", 2024-05-08T08:55:29.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010121899999999999 2024-05-08T08:55:29.151 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.151 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.151 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:29.151 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397711+0000", 2024-05-08T08:55:29.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102489 2024-05-08T08:55:29.151 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.151 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.151 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-05-08T08:55:29.151 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397999+0000", 2024-05-08T08:55:29.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00028733799999999999 2024-05-08T08:55:29.151 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.151 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.151 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:55:29.152 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.432093+0000", 2024-05-08T08:55:29.152 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.034094865000000002 2024-05-08T08:55:29.152 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.152 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.152 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:55:29.152 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.445975+0000", 2024-05-08T08:55:29.152 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013881187 2024-05-08T08:55:29.152 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.152 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.152 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:55:29.152 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.446013+0000", 2024-05-08T08:55:29.152 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8155000000000003e-05 2024-05-08T08:55:29.152 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.152 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.152 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:29.153 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.446052+0000", 2024-05-08T08:55:29.153 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8962999999999999e-05 2024-05-08T08:55:29.153 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.153 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:29.153 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.153 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.153 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:29.153 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:55:29.154 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:55:29.155 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:55:29.156+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:55:29.255 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:55:29.257+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:55:29.300 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:55:29.300 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:55:29.300 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:55:29.300 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:55:29.308 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:55:29.392 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:55:29.395+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:55:29.448 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-05-08T08:55:29.463 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:55:29.463 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:55:29.463 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:55:29.463 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:55:29.463 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:55:29.475 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:55:29.492 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:55:29.495+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:55:29.559 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:55:29.562+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:55:29.627 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:55:29.628 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:55:29.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:55:29.628 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:55:29.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.628 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6690 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1093632~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:29.628 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:55:29.628 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 177.914248939, 2024-05-08T08:55:29.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052462754, 2024-05-08T08:55:29.628 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:29.628 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:29.628 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:29.628 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:29.629 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:29.629 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:29.629 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6690 2024-05-08T08:55:29.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.629 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:29.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.629 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:29.629 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:55:29.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:29.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.629 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:29.629 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:55:29.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:29.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.630 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.630 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:29.630 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715346+0000", 2024-05-08T08:55:29.630 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.84e-07 2024-05-08T08:55:29.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.630 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.630 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:29.630 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715348+0000", 2024-05-08T08:55:29.630 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.114e-06 2024-05-08T08:55:29.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.630 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.630 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:29.630 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715353+0000", 2024-05-08T08:55:29.630 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5720000000000002e-06 2024-05-08T08:55:29.631 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.631 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.631 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:29.631 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715358+0000", 2024-05-08T08:55:29.631 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7140000000000001e-06 2024-05-08T08:55:29.631 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.631 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.631 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:29.631 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767638+0000", 2024-05-08T08:55:29.631 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052279562000000002 2024-05-08T08:55:29.631 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.631 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.631 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:29.631 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767713+0000", 2024-05-08T08:55:29.631 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5652e-05 2024-05-08T08:55:29.631 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.631 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.632 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:29.632 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767808+0000", 2024-05-08T08:55:29.632 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4755999999999999e-05 2024-05-08T08:55:29.632 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.632 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:29.632 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.632 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.632 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.632 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6692 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1110016~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:29.632 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:55:29.632 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 177.91422984900001, 2024-05-08T08:55:29.632 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052666495000000001, 2024-05-08T08:55:29.632 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:29.632 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:29.632 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:29.632 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:29.632 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:29.633 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:29.633 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6692 2024-05-08T08:55:29.633 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.633 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:29.633 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.633 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:29.633 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:55:29.633 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:29.633 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.633 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.633 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:29.633 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:55:29.633 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:29.633 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.633 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.633 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:29.634 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:55:29.634 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8599999999999999e-07 2024-05-08T08:55:29.634 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.634 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:29.634 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715372+0000", 2024-05-08T08:55:29.634 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0990000000000001e-06 2024-05-08T08:55:29.634 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.634 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:29.634 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715378+0000", 2024-05-08T08:55:29.634 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0680000000000002e-06 2024-05-08T08:55:29.634 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.635 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.635 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:29.635 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715383+0000", 2024-05-08T08:55:29.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8740000000000003e-06 2024-05-08T08:55:29.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.635 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.635 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:29.635 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767848+0000", 2024-05-08T08:55:29.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052464668999999998 2024-05-08T08:55:29.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.635 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.635 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:29.635 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767927+0000", 2024-05-08T08:55:29.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9228999999999996e-05 2024-05-08T08:55:29.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.635 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.636 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:29.636 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768031+0000", 2024-05-08T08:55:29.636 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010417 2024-05-08T08:55:29.636 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.636 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:29.636 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.636 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.636 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6694 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1126400~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:29.636 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:55:29.636 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 177.91420495200001, 2024-05-08T08:55:29.636 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052848829, 2024-05-08T08:55:29.636 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:29.636 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:29.636 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:29.636 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:29.636 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:29.636 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:29.637 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6694 2024-05-08T08:55:29.637 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.637 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:29.637 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.637 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:29.637 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:55:29.637 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:29.637 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.637 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.637 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:29.637 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:55:29.637 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:29.637 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.637 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.637 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:29.637 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715390+0000", 2024-05-08T08:55:29.637 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9900000000000001e-07 2024-05-08T08:55:29.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.638 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.638 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:29.638 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715392+0000", 2024-05-08T08:55:29.638 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8279999999999999e-06 2024-05-08T08:55:29.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.638 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.638 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:29.638 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715398+0000", 2024-05-08T08:55:29.638 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7619999999999998e-06 2024-05-08T08:55:29.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.638 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.638 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:29.638 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715403+0000", 2024-05-08T08:55:29.638 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7790000000000003e-06 2024-05-08T08:55:29.639 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.639 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.639 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:29.639 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768067+0000", 2024-05-08T08:55:29.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052663785999999997 2024-05-08T08:55:29.639 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.639 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.639 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:29.639 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768142+0000", 2024-05-08T08:55:29.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5018999999999994e-05 2024-05-08T08:55:29.639 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.639 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.639 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:29.639 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768238+0000", 2024-05-08T08:55:29.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6255999999999995e-05 2024-05-08T08:55:29.639 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.639 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:29.640 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.640 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.640 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6696 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1142784~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:29.640 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:55:29.640 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 177.914185444, 2024-05-08T08:55:29.640 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.053051057999999998, 2024-05-08T08:55:29.640 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:29.640 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:29.640 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:29.640 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:29.640 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:29.640 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:29.640 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6696 2024-05-08T08:55:29.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.640 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:29.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.641 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:29.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:55:29.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:29.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.641 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:29.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:55:29.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:29.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.641 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:29.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:55:29.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-05-08T08:55:29.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:29.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715411+0000", 2024-05-08T08:55:29.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9630000000000001e-06 2024-05-08T08:55:29.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:29.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715417+0000", 2024-05-08T08:55:29.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7250000000000002e-06 2024-05-08T08:55:29.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:29.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715421+0000", 2024-05-08T08:55:29.643 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4089999999999996e-06 2024-05-08T08:55:29.643 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.643 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.643 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:29.643 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768277+0000", 2024-05-08T08:55:29.643 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052855293999999997 2024-05-08T08:55:29.643 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.643 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.643 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:29.643 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768358+0000", 2024-05-08T08:55:29.643 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1720999999999999e-05 2024-05-08T08:55:29.643 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.643 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.643 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:29.643 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768460+0000", 2024-05-08T08:55:29.643 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010154600000000001 2024-05-08T08:55:29.643 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.644 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:29.644 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.644 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.644 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.644 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6698 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1159168~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:29.644 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:55:29.644 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 177.914166577, 2024-05-08T08:55:29.644 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.053245726, 2024-05-08T08:55:29.644 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:29.644 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:29.644 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:29.644 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:29.644 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:29.644 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:29.644 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6698 2024-05-08T08:55:29.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.645 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:29.645 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.645 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:29.645 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:55:29.645 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:29.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.645 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.645 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:29.645 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:55:29.645 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:29.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.645 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.645 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:29.645 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716936+0000", 2024-05-08T08:55:29.645 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001508665 2024-05-08T08:55:29.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.646 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:29.646 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716941+0000", 2024-05-08T08:55:29.646 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1910000000000002e-06 2024-05-08T08:55:29.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.646 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:29.646 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716950+0000", 2024-05-08T08:55:29.646 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1900000000000001e-06 2024-05-08T08:55:29.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.646 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:29.646 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716958+0000", 2024-05-08T08:55:29.646 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8019999999999992e-06 2024-05-08T08:55:29.647 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.647 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.647 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:29.647 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768499+0000", 2024-05-08T08:55:29.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051540926000000001 2024-05-08T08:55:29.647 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.647 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.647 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:29.647 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768574+0000", 2024-05-08T08:55:29.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4988999999999996e-05 2024-05-08T08:55:29.647 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.647 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.647 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:29.647 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768673+0000", 2024-05-08T08:55:29.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9963000000000005e-05 2024-05-08T08:55:29.647 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.648 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:29.648 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.648 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.648 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.648 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6706 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1224704~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:29.648 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:55:29.648 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 177.91256127, 2024-05-08T08:55:29.648 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052435598, 2024-05-08T08:55:29.648 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:29.648 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:29.648 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:29.648 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:29.648 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:29.648 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:29.648 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6706 2024-05-08T08:55:29.648 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.649 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:29.649 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.649 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:29.649 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:55:29.649 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:29.649 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.649 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.649 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:29.649 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:55:29.649 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:29.649 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.649 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.649 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:29.649 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:55:29.649 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:55:29.649 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.650 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.650 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:29.650 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717035+0000", 2024-05-08T08:55:29.650 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9290000000000001e-06 2024-05-08T08:55:29.650 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.650 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.650 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:29.650 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717041+0000", 2024-05-08T08:55:29.650 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6720000000000001e-06 2024-05-08T08:55:29.650 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.650 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.650 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:29.650 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717046+0000", 2024-05-08T08:55:29.650 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5650000000000003e-06 2024-05-08T08:55:29.650 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.651 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.651 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:29.651 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769311+0000", 2024-05-08T08:55:29.651 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052265645999999999 2024-05-08T08:55:29.651 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.651 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.651 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:29.651 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769379+0000", 2024-05-08T08:55:29.651 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7391000000000006e-05 2024-05-08T08:55:29.651 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.651 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.651 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:29.651 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769469+0000", 2024-05-08T08:55:29.651 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0025999999999995e-05 2024-05-08T08:55:29.652 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.652 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:29.652 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.652 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.652 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.652 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6708 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1241088~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:29.652 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:55:29.652 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 177.91254316600001, 2024-05-08T08:55:29.652 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052640552, 2024-05-08T08:55:29.652 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:29.652 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:29.652 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:29.652 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:29.652 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:29.652 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:29.652 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6708 2024-05-08T08:55:29.652 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.653 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:29.653 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.653 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:29.653 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:55:29.653 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:29.653 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.653 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.653 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:29.653 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:55:29.653 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:29.653 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.653 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.653 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:29.653 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717052+0000", 2024-05-08T08:55:29.653 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:55:29.654 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.654 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.654 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:29.654 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717053+0000", 2024-05-08T08:55:29.654 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8330000000000001e-06 2024-05-08T08:55:29.654 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.654 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.654 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:29.654 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717058+0000", 2024-05-08T08:55:29.654 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.933e-06 2024-05-08T08:55:29.654 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.655 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.655 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:29.655 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717063+0000", 2024-05-08T08:55:29.655 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2180000000000001e-06 2024-05-08T08:55:29.655 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.655 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.655 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:29.655 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769507+0000", 2024-05-08T08:55:29.655 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052444866999999999 2024-05-08T08:55:29.655 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.655 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.655 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:29.655 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769589+0000", 2024-05-08T08:55:29.656 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2039000000000005e-05 2024-05-08T08:55:29.656 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.656 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.656 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:29.656 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769692+0000", 2024-05-08T08:55:29.656 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102252 2024-05-08T08:55:29.656 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.656 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:29.656 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.656 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.656 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.656 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6710 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1257472~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:29.656 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:55:29.656 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 177.91252514199999, 2024-05-08T08:55:29.657 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.05283057, 2024-05-08T08:55:29.657 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:29.657 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:29.657 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:29.657 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:29.657 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:29.657 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:29.657 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6710 2024-05-08T08:55:29.657 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.657 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:29.657 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.657 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:29.657 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:55:29.657 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:29.658 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.658 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.658 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:29.658 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:55:29.658 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:29.658 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.658 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.658 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:29.658 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718951+0000", 2024-05-08T08:55:29.658 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0018817809999999999 2024-05-08T08:55:29.658 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.658 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.659 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:29.659 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718956+0000", 2024-05-08T08:55:29.659 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5399999999999997e-06 2024-05-08T08:55:29.659 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.659 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.659 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:29.659 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718964+0000", 2024-05-08T08:55:29.659 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6629999999999999e-06 2024-05-08T08:55:29.659 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.659 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.659 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:29.659 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718972+0000", 2024-05-08T08:55:29.659 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5229999999999998e-06 2024-05-08T08:55:29.659 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.659 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.660 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:29.660 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769729+0000", 2024-05-08T08:55:29.660 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050756959999999997 2024-05-08T08:55:29.660 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.660 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.660 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:29.660 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769803+0000", 2024-05-08T08:55:29.660 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4538999999999999e-05 2024-05-08T08:55:29.660 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.660 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.660 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:29.660 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769900+0000", 2024-05-08T08:55:29.660 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6564000000000006e-05 2024-05-08T08:55:29.660 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.660 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:29.660 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.661 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.661 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.661 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6726 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1388544~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:29.661 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:55:29.661 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 177.91045037000001, 2024-05-08T08:55:29.661 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052451506000000002, 2024-05-08T08:55:29.661 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:29.661 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:29.661 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:29.661 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:29.661 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:29.661 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:29.661 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6726 2024-05-08T08:55:29.661 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.661 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:29.661 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.662 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:29.662 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:55:29.662 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:29.662 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.662 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.662 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:29.662 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:55:29.662 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:29.662 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.662 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.662 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:29.662 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719200+0000", 2024-05-08T08:55:29.663 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5865999999999997e-05 2024-05-08T08:55:29.663 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.663 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.663 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:29.663 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719204+0000", 2024-05-08T08:55:29.663 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8630000000000001e-06 2024-05-08T08:55:29.663 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.663 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.663 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:29.663 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719215+0000", 2024-05-08T08:55:29.663 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1590000000000001e-05 2024-05-08T08:55:29.663 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.663 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.663 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:29.663 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719220+0000", 2024-05-08T08:55:29.663 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1619999999999999e-06 2024-05-08T08:55:29.664 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.664 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.664 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:29.664 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771425+0000", 2024-05-08T08:55:29.664 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052204249000000001 2024-05-08T08:55:29.664 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.664 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.664 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:29.664 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771498+0000", 2024-05-08T08:55:29.664 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3782999999999995e-05 2024-05-08T08:55:29.664 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.664 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.664 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:29.664 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771595+0000", 2024-05-08T08:55:29.665 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6992999999999998e-05 2024-05-08T08:55:29.665 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.665 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:29.665 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.665 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.665 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.665 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6728 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1404928~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:29.665 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:55:29.665 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 177.91036296499999, 2024-05-08T08:55:29.665 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052590747, 2024-05-08T08:55:29.665 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:29.665 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:29.665 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:29.665 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:29.665 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:29.665 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:29.666 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6728 2024-05-08T08:55:29.666 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.666 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:29.666 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.666 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:29.666 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:55:29.666 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:29.666 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.666 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.666 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:29.666 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:55:29.666 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:29.666 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.667 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.667 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:29.667 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719232+0000", 2024-05-08T08:55:29.667 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4000000000000001e-07 2024-05-08T08:55:29.667 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.667 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.667 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:29.667 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719234+0000", 2024-05-08T08:55:29.667 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4499999999999998e-06 2024-05-08T08:55:29.667 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.667 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.667 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:29.667 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719240+0000", 2024-05-08T08:55:29.667 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5720000000000002e-06 2024-05-08T08:55:29.667 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.667 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.668 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:29.668 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719245+0000", 2024-05-08T08:55:29.668 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8690000000000003e-06 2024-05-08T08:55:29.668 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.668 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.668 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:29.668 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771636+0000", 2024-05-08T08:55:29.668 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052390979999999997 2024-05-08T08:55:29.668 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.668 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.668 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:29.668 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771717+0000", 2024-05-08T08:55:29.668 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1291999999999993e-05 2024-05-08T08:55:29.668 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.668 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.668 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:29.669 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771822+0000", 2024-05-08T08:55:29.669 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104944 2024-05-08T08:55:29.669 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.669 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:29.669 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.669 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.669 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.669 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6730 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1421312~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:29.669 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:55:29.669 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 177.91034272300001, 2024-05-08T08:55:29.669 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052770477000000003, 2024-05-08T08:55:29.669 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:29.669 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:29.669 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:29.669 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:29.669 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:29.670 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:29.670 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6730 2024-05-08T08:55:29.670 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.670 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:29.670 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.670 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:29.670 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:55:29.670 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:29.670 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.670 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.670 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:29.670 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:55:29.670 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:29.670 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.671 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.671 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:29.671 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:55:29.671 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9000000000000002e-07 2024-05-08T08:55:29.671 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.671 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.671 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:29.671 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719254+0000", 2024-05-08T08:55:29.671 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4099999999999998e-06 2024-05-08T08:55:29.671 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.671 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.671 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:29.671 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719263+0000", 2024-05-08T08:55:29.671 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4109999999999996e-06 2024-05-08T08:55:29.671 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.671 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.671 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:29.672 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719268+0000", 2024-05-08T08:55:29.672 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2669999999999997e-06 2024-05-08T08:55:29.672 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.672 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.672 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:29.672 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771846+0000", 2024-05-08T08:55:29.672 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052577677000000003 2024-05-08T08:55:29.672 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.672 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.672 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:29.672 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771920+0000", 2024-05-08T08:55:29.672 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4460000000000002e-05 2024-05-08T08:55:29.672 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.672 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.673 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:29.673 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772022+0000", 2024-05-08T08:55:29.673 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010186199999999999 2024-05-08T08:55:29.673 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.673 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:29.673 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.673 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.673 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.673 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6732 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1437696~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:29.673 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:55:29.673 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 177.910317916, 2024-05-08T08:55:29.673 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052905575000000003, 2024-05-08T08:55:29.673 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:29.673 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:29.673 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:29.673 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:29.673 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:29.674 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:29.674 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6732 2024-05-08T08:55:29.674 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.674 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:29.674 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.674 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:29.674 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:55:29.674 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:29.674 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.674 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.674 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:29.674 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:55:29.674 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:29.674 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.674 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.674 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:29.675 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721392+0000", 2024-05-08T08:55:29.675 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002115287 2024-05-08T08:55:29.675 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.675 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.675 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:29.675 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721396+0000", 2024-05-08T08:55:29.675 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1679999999999997e-06 2024-05-08T08:55:29.675 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.675 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.675 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:29.675 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721404+0000", 2024-05-08T08:55:29.675 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4409999999999993e-06 2024-05-08T08:55:29.675 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.675 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.676 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:29.676 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721410+0000", 2024-05-08T08:55:29.676 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9150000000000001e-06 2024-05-08T08:55:29.676 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.676 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.676 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:29.676 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772061+0000", 2024-05-08T08:55:29.676 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050651068 2024-05-08T08:55:29.676 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.676 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.676 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:29.676 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772127+0000", 2024-05-08T08:55:29.676 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5424000000000002e-05 2024-05-08T08:55:29.676 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.676 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.676 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:29.677 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772182+0000", 2024-05-08T08:55:29.677 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5272000000000002e-05 2024-05-08T08:55:29.677 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.677 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:29.677 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.677 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.677 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.677 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6749 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1585152~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:29.677 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:55:29.677 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 177.90801714599999, 2024-05-08T08:55:29.677 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052350264000000001, 2024-05-08T08:55:29.677 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:29.677 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:29.677 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:29.677 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:29.677 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:29.677 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:29.678 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6749 2024-05-08T08:55:29.678 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.678 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:29.678 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.678 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:29.678 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:55:29.678 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:29.678 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.678 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.678 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:29.678 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:55:29.678 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:29.678 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.678 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.679 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:29.679 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721578+0000", 2024-05-08T08:55:29.679 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0399999999999996e-07 2024-05-08T08:55:29.679 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.679 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.679 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:29.679 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721580+0000", 2024-05-08T08:55:29.679 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5450000000000002e-06 2024-05-08T08:55:29.679 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.679 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.679 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:29.679 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721587+0000", 2024-05-08T08:55:29.679 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0190000000000001e-06 2024-05-08T08:55:29.680 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.680 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.680 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:29.680 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721593+0000", 2024-05-08T08:55:29.680 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5659999999999997e-06 2024-05-08T08:55:29.680 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.680 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.680 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:29.680 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773759+0000", 2024-05-08T08:55:29.680 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052166553999999997 2024-05-08T08:55:29.680 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.680 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.680 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:29.680 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773833+0000", 2024-05-08T08:55:29.680 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3913999999999997e-05 2024-05-08T08:55:29.681 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.681 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.681 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:29.681 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773927+0000", 2024-05-08T08:55:29.681 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4061999999999995e-05 2024-05-08T08:55:29.681 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.681 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:29.681 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.681 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.681 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.681 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6751 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1601536~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:29.681 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:55:29.681 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 177.90799386699999, 2024-05-08T08:55:29.681 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052461302000000001, 2024-05-08T08:55:29.681 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:29.682 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:29.682 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:29.682 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:29.682 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:29.682 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:29.682 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6751 2024-05-08T08:55:29.682 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.682 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:29.682 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.682 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:29.682 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:55:29.682 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:29.682 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.682 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.683 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:29.683 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:55:29.683 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:29.683 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.683 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.683 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:29.683 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722939+0000", 2024-05-08T08:55:29.683 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001338601 2024-05-08T08:55:29.683 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.683 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.683 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:29.683 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722944+0000", 2024-05-08T08:55:29.683 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4469999999999999e-06 2024-05-08T08:55:29.683 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.683 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.683 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:29.684 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722955+0000", 2024-05-08T08:55:29.684 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1892999999999999e-05 2024-05-08T08:55:29.684 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.684 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.684 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:29.684 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722963+0000", 2024-05-08T08:55:29.684 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1080000000000004e-06 2024-05-08T08:55:29.684 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.684 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.684 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:29.684 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773964+0000", 2024-05-08T08:55:29.684 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051001022 2024-05-08T08:55:29.684 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.684 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.684 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:29.684 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.774006+0000", 2024-05-08T08:55:29.684 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2729999999999999e-05 2024-05-08T08:55:29.685 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.685 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.685 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:29.685 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.774062+0000", 2024-05-08T08:55:29.685 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5501000000000003e-05 2024-05-08T08:55:29.685 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.685 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:29.685 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.685 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.685 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.685 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6760 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1683456~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:29.685 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:55:29.685 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 177.90648449099999, 2024-05-08T08:55:29.685 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052571312000000002, 2024-05-08T08:55:29.685 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:29.685 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:29.685 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:29.686 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:29.686 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:29.686 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:29.686 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6760 2024-05-08T08:55:29.686 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.686 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:29.686 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.686 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:29.686 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:55:29.686 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:29.686 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.686 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.686 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:29.686 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:55:29.687 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:29.687 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.687 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.687 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:29.687 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:55:29.687 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2100000000000002e-07 2024-05-08T08:55:29.687 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.687 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.687 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:29.687 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723113+0000", 2024-05-08T08:55:29.687 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3149999999999999e-06 2024-05-08T08:55:29.687 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.687 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.687 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:29.687 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723118+0000", 2024-05-08T08:55:29.688 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8860000000000001e-06 2024-05-08T08:55:29.688 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.688 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.688 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:29.688 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723123+0000", 2024-05-08T08:55:29.688 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6770000000000003e-06 2024-05-08T08:55:29.688 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.688 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.688 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:29.688 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775414+0000", 2024-05-08T08:55:29.688 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052290692999999999 2024-05-08T08:55:29.688 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.688 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.688 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:29.688 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775459+0000", 2024-05-08T08:55:29.689 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5639999999999997e-05 2024-05-08T08:55:29.689 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.689 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.689 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:29.689 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775681+0000", 2024-05-08T08:55:29.689 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00022168000000000001 2024-05-08T08:55:29.689 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.689 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:29.689 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.689 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.689 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.689 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6761 3.16 3:6bd6d6da:::rbd_data.1140782590a5.0000000000000015:6 [read 1095680~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:29.689 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:55:29.689 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 177.906465271, 2024-05-08T08:55:29.690 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052751275, 2024-05-08T08:55:29.690 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:29.690 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:29.690 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:29.690 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:29.690 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:29.690 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:29.690 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6761 2024-05-08T08:55:29.690 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.690 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:29.690 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.690 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:29.690 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:55:29.690 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:29.690 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.691 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.691 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:29.691 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:55:29.691 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:29.691 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.691 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.691 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:29.691 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724939+0000", 2024-05-08T08:55:29.691 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001809817 2024-05-08T08:55:29.691 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.691 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.691 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:29.691 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724943+0000", 2024-05-08T08:55:29.691 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4920000000000004e-06 2024-05-08T08:55:29.691 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.691 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.691 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:29.692 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724956+0000", 2024-05-08T08:55:29.692 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2427e-05 2024-05-08T08:55:29.692 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.692 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.692 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:29.692 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724963+0000", 2024-05-08T08:55:29.692 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2559999999999996e-06 2024-05-08T08:55:29.692 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.692 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.692 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:29.692 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775711+0000", 2024-05-08T08:55:29.692 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050747621 2024-05-08T08:55:29.692 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.692 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.692 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:29.692 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775785+0000", 2024-05-08T08:55:29.693 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4634000000000004e-05 2024-05-08T08:55:29.693 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.693 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.693 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:29.693 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775880+0000", 2024-05-08T08:55:29.693 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5027999999999999e-05 2024-05-08T08:55:29.693 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.693 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:29.693 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.693 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.693 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.693 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6770 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1765376~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:29.693 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:55:29.693 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 177.904434176, 2024-05-08T08:55:29.693 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052339296, 2024-05-08T08:55:29.694 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:29.694 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:29.694 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:29.694 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:29.694 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:29.694 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:29.694 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6770 2024-05-08T08:55:29.694 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.694 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:29.694 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.694 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:29.694 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:55:29.694 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:29.694 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.694 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.695 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:29.695 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:55:29.695 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:29.695 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.695 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.695 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:29.695 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725161+0000", 2024-05-08T08:55:29.695 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-05-08T08:55:29.695 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.695 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.695 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:29.695 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725163+0000", 2024-05-08T08:55:29.695 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4370000000000001e-06 2024-05-08T08:55:29.695 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.695 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.695 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:29.696 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725169+0000", 2024-05-08T08:55:29.696 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8409999999999996e-06 2024-05-08T08:55:29.696 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.696 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.696 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:29.696 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725174+0000", 2024-05-08T08:55:29.696 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7249999999999997e-06 2024-05-08T08:55:29.696 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.696 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.696 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:29.696 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777343+0000", 2024-05-08T08:55:29.696 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052169633999999999 2024-05-08T08:55:29.696 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.696 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.696 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:29.696 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777409+0000", 2024-05-08T08:55:29.697 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6033999999999998e-05 2024-05-08T08:55:29.697 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.697 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.697 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:29.697 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777499+0000", 2024-05-08T08:55:29.697 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0224999999999998e-05 2024-05-08T08:55:29.697 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.697 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:29.697 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.697 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.697 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.697 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6771 3.16 3:6bd6d6da:::rbd_data.1140782590a5.0000000000000015:6 [read 1177600~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:29.697 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:55:29.697 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 177.90441435899999, 2024-05-08T08:55:29.697 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052438604999999999, 2024-05-08T08:55:29.697 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:29.698 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:29.698 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:29.698 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:29.698 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:29.698 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:29.698 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6771 2024-05-08T08:55:29.698 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.698 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:29.698 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.698 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:29.698 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:55:29.698 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:29.698 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.698 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.698 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:29.698 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:55:29.698 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:29.699 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.699 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.699 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:29.699 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:55:29.699 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:55:29.699 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.699 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.699 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:29.699 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725188+0000", 2024-05-08T08:55:29.699 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1579999999999999e-06 2024-05-08T08:55:29.699 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.699 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.699 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:29.699 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725195+0000", 2024-05-08T08:55:29.699 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8909999999999995e-06 2024-05-08T08:55:29.699 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.699 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.700 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:29.700 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725200+0000", 2024-05-08T08:55:29.700 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9810000000000003e-06 2024-05-08T08:55:29.700 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.700 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.700 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:29.700 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777527+0000", 2024-05-08T08:55:29.700 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052326582000000003 2024-05-08T08:55:29.700 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.700 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.700 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:29.700 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777566+0000", 2024-05-08T08:55:29.701 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9447999999999999e-05 2024-05-08T08:55:29.701 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.701 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.701 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:29.701 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777619+0000", 2024-05-08T08:55:29.701 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2135000000000001e-05 2024-05-08T08:55:29.701 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.701 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:29.701 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.701 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.701 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.701 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6772 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1781760~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:29.701 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:55:29.701 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 177.90438630200001, 2024-05-08T08:55:29.701 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052601799999999997, 2024-05-08T08:55:29.701 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:29.701 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:29.701 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:29.702 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:29.702 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:29.702 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:29.702 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6772 2024-05-08T08:55:29.702 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.702 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:29.702 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.702 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:29.702 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:55:29.702 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:29.702 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.702 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.702 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:29.702 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:55:29.702 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:29.703 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.703 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.703 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:29.703 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726939+0000", 2024-05-08T08:55:29.703 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001730493 2024-05-08T08:55:29.703 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.703 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.703 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:29.703 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726943+0000", 2024-05-08T08:55:29.703 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7330000000000003e-06 2024-05-08T08:55:29.703 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.703 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.703 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:29.703 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726954+0000", 2024-05-08T08:55:29.703 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1107000000000001e-05 2024-05-08T08:55:29.704 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.704 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.704 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:29.704 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726962+0000", 2024-05-08T08:55:29.704 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.503e-06 2024-05-08T08:55:29.704 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.704 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.704 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:29.704 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777653+0000", 2024-05-08T08:55:29.704 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050691542999999999 2024-05-08T08:55:29.704 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.704 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.704 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:29.704 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777719+0000", 2024-05-08T08:55:29.704 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5228000000000006e-05 2024-05-08T08:55:29.704 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.704 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.705 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:29.705 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777810+0000", 2024-05-08T08:55:29.705 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1193000000000006e-05 2024-05-08T08:55:29.705 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.705 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:29.705 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.705 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.705 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.705 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6782 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1880064~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:29.705 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:55:29.705 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 177.90241779499999, 2024-05-08T08:55:29.705 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052326492000000002, 2024-05-08T08:55:29.705 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:29.705 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:29.705 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:29.705 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:29.705 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:29.706 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:29.706 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6782 2024-05-08T08:55:29.706 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.706 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:29.706 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.706 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:29.706 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:55:29.706 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:29.706 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.706 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.706 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:29.706 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:55:29.706 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:29.706 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.706 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.706 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:29.707 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728939+0000", 2024-05-08T08:55:29.707 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001762719 2024-05-08T08:55:29.707 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.707 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.707 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:29.707 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728944+0000", 2024-05-08T08:55:29.707 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.746e-06 2024-05-08T08:55:29.707 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.707 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.707 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:29.707 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728955+0000", 2024-05-08T08:55:29.707 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1043000000000001e-05 2024-05-08T08:55:29.707 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.707 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.707 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:29.707 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728963+0000", 2024-05-08T08:55:29.708 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5809999999999999e-06 2024-05-08T08:55:29.708 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.708 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.708 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:29.708 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779320+0000", 2024-05-08T08:55:29.708 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050357290999999998 2024-05-08T08:55:29.708 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.708 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.708 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:29.708 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779401+0000", 2024-05-08T08:55:29.708 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0574000000000005e-05 2024-05-08T08:55:29.708 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:29.708 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:29.708 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:29.708 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779503+0000", 2024-05-08T08:55:29.709 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102538 2024-05-08T08:55:29.709 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.709 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:29.709 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.709 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:29.709 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:29.709 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:55:29.710 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:55:29.711 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:55:29.712+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:55:29.811 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:55:29.814+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:55:29.820 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:55:29.820 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:55:29.820 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:55:29.820 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:55:29.829 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:55:29.912 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:55:29.915+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:55:29.978 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:55:29.979 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:55:29.979 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:55:29.979 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:55:29.979 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:55:29.987 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:55:30.014 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:55:30.016+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:55:30.114 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:55:30.117+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:55:30.138 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:55:30.139 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:55:30.139 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:55:30.139 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:55:30.139 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.139 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4621.0:9089 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 23339520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-05-08T08:55:30.139 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:55:30.139 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 177.647346715, 2024-05-08T08:55:30.139 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.13814907200000001, 2024-05-08T08:55:30.139 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:30.139 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:30.139 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:55:30.139 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:30.139 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4621", 2024-05-08T08:55:30.139 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:30.139 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 9089 2024-05-08T08:55:30.140 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.140 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:30.140 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.140 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:30.140 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:55:30.140 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:30.140 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.140 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.140 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:30.140 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493002+0000", 2024-05-08T08:55:30.140 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:30.140 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.140 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.140 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:30.140 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.493005+0000", 2024-05-08T08:55:30.140 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1700000000000001e-06 2024-05-08T08:55:30.140 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.141 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.141 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:30.141 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.513963+0000", 2024-05-08T08:55:30.141 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.020957581999999999 2024-05-08T08:55:30.141 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.141 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.141 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:30.141 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.513995+0000", 2024-05-08T08:55:30.141 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1993999999999999e-05 2024-05-08T08:55:30.141 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.141 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.141 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:30.141 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514007+0000", 2024-05-08T08:55:30.141 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1581e-05 2024-05-08T08:55:30.141 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.141 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.142 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:30.142 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514122+0000", 2024-05-08T08:55:30.142 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000115282 2024-05-08T08:55:30.142 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.142 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.142 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op must wait for map", 2024-05-08T08:55:30.142 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.514128+0000", 2024-05-08T08:55:30.142 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9950000000000002e-06 2024-05-08T08:55:30.142 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.142 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.142 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:30.142 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.515800+0000", 2024-05-08T08:55:30.142 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001671886 2024-05-08T08:55:30.142 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.142 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.142 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:30.143 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.515893+0000", 2024-05-08T08:55:30.143 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3490000000000001e-05 2024-05-08T08:55:30.143 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.143 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.143 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 3", 2024-05-08T08:55:30.143 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.517647+0000", 2024-05-08T08:55:30.143 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0017535630000000001 2024-05-08T08:55:30.143 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.143 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.143 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:55:30.143 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.627578+0000", 2024-05-08T08:55:30.143 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.109931557 2024-05-08T08:55:30.143 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.143 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.143 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:55:30.143 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631055+0000", 2024-05-08T08:55:30.143 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0034761420000000002 2024-05-08T08:55:30.144 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.144 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.144 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:55:30.144 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631096+0000", 2024-05-08T08:55:30.144 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1059000000000001e-05 2024-05-08T08:55:30.144 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.144 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.144 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:30.144 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:32.631151+0000", 2024-05-08T08:55:30.144 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5770999999999998e-05 2024-05-08T08:55:30.144 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.144 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:30.144 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.144 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.144 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.145 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1071 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11026432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:55:30.145 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:55:30.145 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 173.654897224, 2024-05-08T08:55:30.145 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070580037999999998, 2024-05-08T08:55:30.145 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:30.145 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:30.145 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:30.145 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:30.145 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:55:30.145 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:55:30.145 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1071 2024-05-08T08:55:30.145 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.145 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:30.145 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.145 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:30.146 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:55:30.146 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:30.146 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.146 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.146 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:30.146 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:55:30.146 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:30.146 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.146 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.146 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:30.146 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485452+0000", 2024-05-08T08:55:30.146 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2800000000000002e-07 2024-05-08T08:55:30.146 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.146 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.146 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:30.147 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485454+0000", 2024-05-08T08:55:30.147 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2290000000000002e-06 2024-05-08T08:55:30.147 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.147 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.147 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:30.147 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485461+0000", 2024-05-08T08:55:30.147 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9920000000000001e-06 2024-05-08T08:55:30.147 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.147 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.147 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:30.147 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485467+0000", 2024-05-08T08:55:30.147 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3979999999999998e-06 2024-05-08T08:55:30.147 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.147 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.147 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:30.147 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555880+0000", 2024-05-08T08:55:30.148 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070413165 2024-05-08T08:55:30.148 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.148 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.148 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:30.148 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.555926+0000", 2024-05-08T08:55:30.148 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6217000000000002e-05 2024-05-08T08:55:30.148 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.148 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.148 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:30.148 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556032+0000", 2024-05-08T08:55:30.148 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000105609 2024-05-08T08:55:30.148 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.148 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:30.148 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.148 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.148 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.148 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1073 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11067392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:55:30.149 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:55:30.149 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 173.654876003, 2024-05-08T08:55:30.149 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070821404000000004, 2024-05-08T08:55:30.149 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:30.149 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:30.149 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:30.149 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:30.149 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:55:30.149 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:55:30.149 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1073 2024-05-08T08:55:30.149 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.149 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:30.149 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.149 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:30.149 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:55:30.149 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:30.150 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.150 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.150 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:30.150 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:55:30.150 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:30.150 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.150 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.150 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:30.150 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485473+0000", 2024-05-08T08:55:30.150 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.08e-07 2024-05-08T08:55:30.150 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.150 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.150 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:30.150 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485476+0000", 2024-05-08T08:55:30.150 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3929999999999998e-06 2024-05-08T08:55:30.151 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.151 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.151 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:30.151 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485485+0000", 2024-05-08T08:55:30.151 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1880000000000008e-06 2024-05-08T08:55:30.151 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.151 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.151 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:30.151 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485494+0000", 2024-05-08T08:55:30.151 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3109999999999995e-06 2024-05-08T08:55:30.151 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.151 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.151 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:30.151 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556082+0000", 2024-05-08T08:55:30.151 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070587426999999994 2024-05-08T08:55:30.151 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.152 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.152 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:30.152 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556177+0000", 2024-05-08T08:55:30.152 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5051999999999997e-05 2024-05-08T08:55:30.152 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.152 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.152 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:30.152 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556294+0000", 2024-05-08T08:55:30.152 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000117625 2024-05-08T08:55:30.152 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.152 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:30.152 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.152 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.152 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.152 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1075 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11108352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:55:30.152 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:55:30.153 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 173.65484542600001, 2024-05-08T08:55:30.153 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071045026999999997, 2024-05-08T08:55:30.153 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:30.153 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:30.153 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:30.153 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:30.153 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:55:30.153 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:55:30.153 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1075 2024-05-08T08:55:30.153 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.153 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:30.153 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.153 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:30.153 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:55:30.153 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:30.153 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.153 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.154 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:30.154 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:55:30.154 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:30.154 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.154 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.154 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:30.154 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485504+0000", 2024-05-08T08:55:30.154 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.32e-07 2024-05-08T08:55:30.154 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.154 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.154 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:30.154 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485518+0000", 2024-05-08T08:55:30.154 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3769e-05 2024-05-08T08:55:30.154 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.154 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.155 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:30.155 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485527+0000", 2024-05-08T08:55:30.155 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6779999999999995e-06 2024-05-08T08:55:30.155 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.155 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.155 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:30.155 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485532+0000", 2024-05-08T08:55:30.155 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9830000000000004e-06 2024-05-08T08:55:30.155 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.155 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.155 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:30.155 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556340+0000", 2024-05-08T08:55:30.155 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070807743000000006 2024-05-08T08:55:30.155 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.155 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.155 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:30.156 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556435+0000", 2024-05-08T08:55:30.156 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4732e-05 2024-05-08T08:55:30.156 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.156 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.156 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:30.156 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556549+0000", 2024-05-08T08:55:30.156 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011369 2024-05-08T08:55:30.156 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.156 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:30.156 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.156 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.156 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.156 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1078 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11149312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:55:30.156 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:55:30.156 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 173.654805227, 2024-05-08T08:55:30.156 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071167276000000002, 2024-05-08T08:55:30.156 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:30.157 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:30.157 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:30.157 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:30.157 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:55:30.157 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:55:30.157 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1078 2024-05-08T08:55:30.157 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.157 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:30.157 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.157 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:30.157 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:55:30.157 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:30.157 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.157 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.157 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:30.157 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:55:30.157 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:30.157 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.158 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.158 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:30.158 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485544+0000", 2024-05-08T08:55:30.158 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5600000000000001e-07 2024-05-08T08:55:30.158 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.158 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.158 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:30.158 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485547+0000", 2024-05-08T08:55:30.158 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3590000000000002e-06 2024-05-08T08:55:30.158 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.158 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.158 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:30.158 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485553+0000", 2024-05-08T08:55:30.158 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2770000000000002e-06 2024-05-08T08:55:30.158 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.159 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.159 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:30.159 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485558+0000", 2024-05-08T08:55:30.159 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.95e-06 2024-05-08T08:55:30.159 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.159 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.159 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:30.159 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556582+0000", 2024-05-08T08:55:30.159 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071024444000000006 2024-05-08T08:55:30.159 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.159 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.159 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:30.159 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556637+0000", 2024-05-08T08:55:30.159 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4672000000000001e-05 2024-05-08T08:55:30.159 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.159 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.160 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:30.160 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556711+0000", 2024-05-08T08:55:30.160 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4117999999999997e-05 2024-05-08T08:55:30.160 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.160 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:30.160 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.160 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.160 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.160 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1080 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11190272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:55:30.160 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:55:30.160 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 173.65478449400001, 2024-05-08T08:55:30.160 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071350721000000006, 2024-05-08T08:55:30.160 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:30.160 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:30.160 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:30.160 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:30.161 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:55:30.161 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:55:30.161 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1080 2024-05-08T08:55:30.161 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.161 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:30.161 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.161 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:30.161 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:55:30.161 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:30.161 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.161 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.161 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:30.161 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:55:30.161 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:30.161 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.162 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.162 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:30.162 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485565+0000", 2024-05-08T08:55:30.162 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.03e-07 2024-05-08T08:55:30.162 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.162 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.162 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:30.162 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485567+0000", 2024-05-08T08:55:30.162 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1090000000000001e-06 2024-05-08T08:55:30.162 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.162 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.162 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:30.163 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485573+0000", 2024-05-08T08:55:30.163 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3049999999999998e-06 2024-05-08T08:55:30.163 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.163 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.163 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:30.163 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485578+0000", 2024-05-08T08:55:30.163 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0130000000000001e-06 2024-05-08T08:55:30.163 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.163 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.163 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:30.163 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556743+0000", 2024-05-08T08:55:30.163 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071164449000000005 2024-05-08T08:55:30.163 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.163 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.163 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:30.164 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556797+0000", 2024-05-08T08:55:30.164 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4589999999999997e-05 2024-05-08T08:55:30.164 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.164 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.164 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:30.164 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556915+0000", 2024-05-08T08:55:30.164 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000117852 2024-05-08T08:55:30.164 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.164 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:30.164 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.164 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.164 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.164 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1082 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11231232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:55:30.164 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:55:30.164 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 173.65476425099999, 2024-05-08T08:55:30.164 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071591396000000002, 2024-05-08T08:55:30.164 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:30.165 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:30.165 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:30.165 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:30.165 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:55:30.165 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:55:30.165 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1082 2024-05-08T08:55:30.165 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.165 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:30.165 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.165 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:30.165 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:55:30.165 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:30.165 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.165 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.165 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:30.166 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:55:30.166 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:30.166 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.166 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.166 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:30.166 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485585+0000", 2024-05-08T08:55:30.166 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1800000000000001e-07 2024-05-08T08:55:30.166 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.166 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.166 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:30.166 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485587+0000", 2024-05-08T08:55:30.166 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2589999999999999e-06 2024-05-08T08:55:30.166 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.166 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.166 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:30.167 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485594+0000", 2024-05-08T08:55:30.167 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0789999999999997e-06 2024-05-08T08:55:30.167 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.167 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.167 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:30.167 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485598+0000", 2024-05-08T08:55:30.167 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7480000000000001e-06 2024-05-08T08:55:30.167 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.167 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.167 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:30.167 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.556971+0000", 2024-05-08T08:55:30.167 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071372774999999999 2024-05-08T08:55:30.167 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.167 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.167 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:30.167 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557069+0000", 2024-05-08T08:55:30.168 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.7732999999999994e-05 2024-05-08T08:55:30.168 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.168 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.168 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:30.168 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557176+0000", 2024-05-08T08:55:30.168 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000107384 2024-05-08T08:55:30.168 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.168 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:30.168 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.168 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.168 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.168 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1084 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11272192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:55:30.168 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:55:30.168 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 173.65472561199999, 2024-05-08T08:55:30.168 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071808977999999996, 2024-05-08T08:55:30.168 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:30.169 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:30.169 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:30.169 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:30.169 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:55:30.169 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:55:30.169 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1084 2024-05-08T08:55:30.169 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.169 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:30.169 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.169 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:30.169 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:55:30.169 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:30.169 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.169 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.169 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:30.169 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485623+0000", 2024-05-08T08:55:30.170 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:30.170 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.170 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.170 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:30.170 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485624+0000", 2024-05-08T08:55:30.170 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:55:30.170 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.170 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.170 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:30.170 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485626+0000", 2024-05-08T08:55:30.170 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9460000000000001e-06 2024-05-08T08:55:30.170 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.170 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.170 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:30.171 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485633+0000", 2024-05-08T08:55:30.171 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1149999999999996e-06 2024-05-08T08:55:30.171 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.171 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.171 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:30.171 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485638+0000", 2024-05-08T08:55:30.171 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5670000000000002e-06 2024-05-08T08:55:30.171 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.171 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.171 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:30.171 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557226+0000", 2024-05-08T08:55:30.171 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071587267999999996 2024-05-08T08:55:30.171 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.171 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.171 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:30.171 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557321+0000", 2024-05-08T08:55:30.171 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5785000000000005e-05 2024-05-08T08:55:30.171 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.172 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.172 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:30.172 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557432+0000", 2024-05-08T08:55:30.172 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011092800000000001 2024-05-08T08:55:30.172 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.172 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:30.172 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.172 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.172 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.172 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1086 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11313152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:55:30.172 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:55:30.172 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 173.65470228500001, 2024-05-08T08:55:30.172 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071933132999999996, 2024-05-08T08:55:30.172 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:30.172 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:30.172 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:30.173 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:30.173 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:55:30.173 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:55:30.173 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1086 2024-05-08T08:55:30.173 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.173 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:30.173 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.173 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:30.173 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:55:30.173 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:30.173 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.173 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.173 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:30.173 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.485647+0000", 2024-05-08T08:55:30.173 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:30.173 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.174 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.174 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:30.174 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486328+0000", 2024-05-08T08:55:30.174 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00068107900000000002 2024-05-08T08:55:30.174 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.174 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.174 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:30.174 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486331+0000", 2024-05-08T08:55:30.174 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.585e-06 2024-05-08T08:55:30.174 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.174 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.174 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:30.174 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486343+0000", 2024-05-08T08:55:30.174 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1856e-05 2024-05-08T08:55:30.174 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.174 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.175 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:30.175 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486350+0000", 2024-05-08T08:55:30.175 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8600000000000004e-06 2024-05-08T08:55:30.175 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.175 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.175 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:30.175 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557470+0000", 2024-05-08T08:55:30.175 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071120326999999997 2024-05-08T08:55:30.175 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.175 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.175 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:30.175 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557514+0000", 2024-05-08T08:55:30.175 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3909000000000003e-05 2024-05-08T08:55:30.175 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.175 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.175 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:30.175 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557580+0000", 2024-05-08T08:55:30.176 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5517000000000003e-05 2024-05-08T08:55:30.176 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.176 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:30.176 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.176 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.176 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.176 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1089 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11354112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:55:30.176 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:55:30.176 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 173.65398941300001, 2024-05-08T08:55:30.176 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071457381, 2024-05-08T08:55:30.176 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:30.176 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:30.176 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:30.176 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:30.176 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:55:30.176 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:55:30.177 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1089 2024-05-08T08:55:30.177 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.177 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:30.177 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.177 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:30.177 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:55:30.177 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:30.177 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.177 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.177 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:30.177 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486360+0000", 2024-05-08T08:55:30.177 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:30.177 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.177 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.177 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:30.177 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486851+0000", 2024-05-08T08:55:30.177 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00049102 2024-05-08T08:55:30.178 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.178 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.178 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:30.178 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486854+0000", 2024-05-08T08:55:30.178 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6859999999999999e-06 2024-05-08T08:55:30.178 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.178 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.178 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:30.178 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486864+0000", 2024-05-08T08:55:30.178 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6080000000000002e-06 2024-05-08T08:55:30.178 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.178 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.178 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:30.178 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486870+0000", 2024-05-08T08:55:30.178 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.477e-06 2024-05-08T08:55:30.178 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.178 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.179 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:30.179 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557612+0000", 2024-05-08T08:55:30.179 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070741557999999996 2024-05-08T08:55:30.179 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.179 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.179 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:30.179 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557695+0000", 2024-05-08T08:55:30.179 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3300000000000005e-05 2024-05-08T08:55:30.179 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.179 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.179 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:30.179 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557817+0000", 2024-05-08T08:55:30.179 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012173200000000001 2024-05-08T08:55:30.179 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.179 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:30.180 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.180 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.180 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.180 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1091 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11395072~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:55:30.180 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:55:30.180 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 173.65346861800001, 2024-05-08T08:55:30.180 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071184737999999997, 2024-05-08T08:55:30.180 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:30.180 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:30.180 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:30.180 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:30.180 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:55:30.180 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:55:30.180 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1091 2024-05-08T08:55:30.180 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.180 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:30.180 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.181 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:30.181 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:55:30.181 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:30.181 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.181 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.181 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:30.181 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486880+0000", 2024-05-08T08:55:30.181 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:30.181 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.181 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.181 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:30.181 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486881+0000", 2024-05-08T08:55:30.181 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0699999999999997e-07 2024-05-08T08:55:30.181 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.182 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.182 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:30.182 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486883+0000", 2024-05-08T08:55:30.182 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.402e-06 2024-05-08T08:55:30.182 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.182 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.182 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:30.182 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486890+0000", 2024-05-08T08:55:30.182 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0480000000000001e-06 2024-05-08T08:55:30.182 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.182 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.182 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:30.182 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486896+0000", 2024-05-08T08:55:30.182 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7640000000000002e-06 2024-05-08T08:55:30.182 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.182 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.183 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:30.183 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557860+0000", 2024-05-08T08:55:30.183 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070963791999999998 2024-05-08T08:55:30.183 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.183 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.183 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:30.183 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.557952+0000", 2024-05-08T08:55:30.183 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2082000000000004e-05 2024-05-08T08:55:30.183 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.183 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.183 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:30.183 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558065+0000", 2024-05-08T08:55:30.183 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000113143 2024-05-08T08:55:30.183 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.183 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:30.184 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.184 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.184 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.184 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1094 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11436032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:55:30.184 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:55:30.184 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 173.65344544999999, 2024-05-08T08:55:30.184 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071400507000000002, 2024-05-08T08:55:30.184 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:30.184 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:30.184 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:30.184 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:30.184 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:55:30.184 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:55:30.184 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1094 2024-05-08T08:55:30.184 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.185 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:30.185 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.185 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:30.185 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:55:30.185 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:30.185 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.185 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.185 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:30.185 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.486904+0000", 2024-05-08T08:55:30.185 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:30.185 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.185 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.185 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:30.185 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487940+0000", 2024-05-08T08:55:30.185 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001036868 2024-05-08T08:55:30.185 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.185 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.186 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:30.186 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487944+0000", 2024-05-08T08:55:30.186 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7069999999999998e-06 2024-05-08T08:55:30.186 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.186 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.186 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:30.186 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487951+0000", 2024-05-08T08:55:30.186 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2649999999999999e-06 2024-05-08T08:55:30.186 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.186 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.186 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:30.186 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487961+0000", 2024-05-08T08:55:30.186 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4110000000000002e-06 2024-05-08T08:55:30.186 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.186 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.186 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:30.187 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558117+0000", 2024-05-08T08:55:30.187 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070156662999999994 2024-05-08T08:55:30.187 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.187 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.187 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:30.187 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558214+0000", 2024-05-08T08:55:30.187 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6444e-05 2024-05-08T08:55:30.187 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.187 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.187 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:30.187 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558304+0000", 2024-05-08T08:55:30.187 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0148999999999994e-05 2024-05-08T08:55:30.187 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.187 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:30.187 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.187 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.188 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.188 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1096 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11476992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:55:30.188 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:55:30.188 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 173.652378919, 2024-05-08T08:55:30.188 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070480074000000004, 2024-05-08T08:55:30.188 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:30.188 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:30.188 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:30.188 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:30.188 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:55:30.188 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:55:30.188 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1096 2024-05-08T08:55:30.188 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.188 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:30.188 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.189 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:30.189 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:55:30.189 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:30.189 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.189 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.189 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:30.189 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:55:30.189 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:30.189 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.189 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.189 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:30.189 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487970+0000", 2024-05-08T08:55:30.189 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3700000000000001e-07 2024-05-08T08:55:30.189 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.189 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.190 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:30.190 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487973+0000", 2024-05-08T08:55:30.190 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0870000000000001e-06 2024-05-08T08:55:30.190 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.190 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.190 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:30.190 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487977+0000", 2024-05-08T08:55:30.190 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0600000000000001e-06 2024-05-08T08:55:30.190 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.190 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.190 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:30.190 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487980+0000", 2024-05-08T08:55:30.190 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1089999999999998e-06 2024-05-08T08:55:30.190 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.191 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.191 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:30.191 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558338+0000", 2024-05-08T08:55:30.191 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070358233000000006 2024-05-08T08:55:30.191 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.191 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.191 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:30.191 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558392+0000", 2024-05-08T08:55:30.191 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4537999999999999e-05 2024-05-08T08:55:30.191 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.191 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.191 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:30.191 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558450+0000", 2024-05-08T08:55:30.191 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7710000000000001e-05 2024-05-08T08:55:30.191 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.192 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:30.192 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.192 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.192 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.192 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1099 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11517952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:55:30.192 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:55:30.192 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 173.652365043, 2024-05-08T08:55:30.192 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070720506000000002, 2024-05-08T08:55:30.192 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:30.192 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:30.192 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:30.192 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:30.192 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:55:30.192 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:55:30.192 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1099 2024-05-08T08:55:30.192 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.193 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:30.193 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.193 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:30.193 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:55:30.193 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:30.193 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.193 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.193 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:30.193 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:55:30.193 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:30.193 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.193 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.193 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:30.193 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487984+0000", 2024-05-08T08:55:30.193 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3999999999999998e-07 2024-05-08T08:55:30.193 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.194 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.194 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:30.194 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487986+0000", 2024-05-08T08:55:30.194 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3519999999999999e-06 2024-05-08T08:55:30.194 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.194 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.194 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:30.194 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487989+0000", 2024-05-08T08:55:30.194 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.23e-06 2024-05-08T08:55:30.194 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.194 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.194 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:30.194 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487991+0000", 2024-05-08T08:55:30.194 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6350000000000002e-06 2024-05-08T08:55:30.194 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.194 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.195 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:30.195 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558489+0000", 2024-05-08T08:55:30.195 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070497924000000003 2024-05-08T08:55:30.195 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.195 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.195 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:30.195 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558583+0000", 2024-05-08T08:55:30.195 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3405000000000004e-05 2024-05-08T08:55:30.195 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.195 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.195 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:30.195 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558704+0000", 2024-05-08T08:55:30.195 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012171999999999999 2024-05-08T08:55:30.195 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.195 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:30.195 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.195 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.196 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.196 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1100 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11558912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:55:30.196 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:55:30.196 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 173.652352044, 2024-05-08T08:55:30.196 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070962266999999996, 2024-05-08T08:55:30.196 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:30.196 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:30.196 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:30.196 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:30.196 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:55:30.196 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:55:30.196 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1100 2024-05-08T08:55:30.196 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.196 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:30.196 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.196 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:30.196 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:55:30.197 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:30.197 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.197 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.197 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:30.197 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.487997+0000", 2024-05-08T08:55:30.197 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:30.197 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.197 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.197 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:30.197 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488430+0000", 2024-05-08T08:55:30.197 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000432676 2024-05-08T08:55:30.197 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.197 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.197 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:30.197 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488433+0000", 2024-05-08T08:55:30.197 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.45e-06 2024-05-08T08:55:30.198 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.198 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.198 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:30.198 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488442+0000", 2024-05-08T08:55:30.198 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.687e-06 2024-05-08T08:55:30.198 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.198 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.198 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:30.198 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488446+0000", 2024-05-08T08:55:30.198 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4939999999999997e-06 2024-05-08T08:55:30.198 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.198 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.198 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:30.198 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558756+0000", 2024-05-08T08:55:30.198 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070310051999999998 2024-05-08T08:55:30.198 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.199 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.199 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:30.199 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558844+0000", 2024-05-08T08:55:30.199 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7690999999999998e-05 2024-05-08T08:55:30.199 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.199 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.199 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:30.199 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558959+0000", 2024-05-08T08:55:30.199 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000115217 2024-05-08T08:55:30.199 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.199 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:30.199 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.199 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.199 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.199 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1103 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11599872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:55:30.199 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:55:30.199 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 173.65189436899999, 2024-05-08T08:55:30.200 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070777100999999995, 2024-05-08T08:55:30.200 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:30.200 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:30.200 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:30.200 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:30.200 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:55:30.200 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:55:30.200 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1103 2024-05-08T08:55:30.200 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.200 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:30.200 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.200 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:30.200 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:55:30.200 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:30.200 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.200 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.200 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:30.201 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488455+0000", 2024-05-08T08:55:30.201 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:30.201 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.201 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.201 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:30.201 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488959+0000", 2024-05-08T08:55:30.201 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00050457200000000001 2024-05-08T08:55:30.201 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.201 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.201 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:30.201 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488969+0000", 2024-05-08T08:55:30.201 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.7159999999999998e-06 2024-05-08T08:55:30.201 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.201 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.201 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:30.201 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488981+0000", 2024-05-08T08:55:30.201 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2150000000000001e-05 2024-05-08T08:55:30.202 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.202 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.202 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:30.202 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488988+0000", 2024-05-08T08:55:30.202 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8759999999999999e-06 2024-05-08T08:55:30.202 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.202 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.202 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:30.202 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.558992+0000", 2024-05-08T08:55:30.202 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070004358000000003 2024-05-08T08:55:30.202 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.202 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.202 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:30.202 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559093+0000", 2024-05-08T08:55:30.202 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000100299 2024-05-08T08:55:30.202 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.202 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.203 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:30.203 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559232+0000", 2024-05-08T08:55:30.203 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013913 2024-05-08T08:55:30.203 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.203 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:30.203 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.203 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.203 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.203 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4684.0:1104 3.1f 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:8 [read 11640832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:55:30.203 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:55:30.203 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 173.65134955400001, 2024-05-08T08:55:30.203 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070521947000000001, 2024-05-08T08:55:30.203 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:30.203 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:30.203 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:30.203 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:30.203 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4684", 2024-05-08T08:55:30.204 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3798813132", 2024-05-08T08:55:30.204 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1104 2024-05-08T08:55:30.204 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.204 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:30.204 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.204 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:30.204 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:55:30.204 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:30.204 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.204 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.204 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:30.204 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.488999+0000", 2024-05-08T08:55:30.204 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:30.204 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.205 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.205 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:30.205 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489700+0000", 2024-05-08T08:55:30.205 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00070067600000000001 2024-05-08T08:55:30.205 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.205 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.205 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:30.205 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489704+0000", 2024-05-08T08:55:30.205 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1640000000000003e-06 2024-05-08T08:55:30.205 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.205 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.205 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:30.205 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489716+0000", 2024-05-08T08:55:30.205 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1379000000000001e-05 2024-05-08T08:55:30.205 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.205 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.205 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:30.206 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.489724+0000", 2024-05-08T08:55:30.206 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8949999999999998e-06 2024-05-08T08:55:30.206 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.206 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.206 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:30.206 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559278+0000", 2024-05-08T08:55:30.206 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.069554151999999994 2024-05-08T08:55:30.206 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.206 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.206 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:30.206 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559387+0000", 2024-05-08T08:55:30.206 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000109258 2024-05-08T08:55:30.206 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.206 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.206 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:30.206 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:36.559521+0000", 2024-05-08T08:55:30.207 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000134423 2024-05-08T08:55:30.207 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.207 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:30.207 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.207 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.207 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.207 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:8636 3.17 3:e8a0a3e4:::rbd_header.11402b26385:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-05-08T08:55:30.207 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:52:54.150971+0000", 2024-05-08T08:55:30.207 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 155.98937756000001, 2024-05-08T08:55:30.207 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.217273233, 2024-05-08T08:55:30.207 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:30.207 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:30.207 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:30.207 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:30.207 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:55:30.207 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:55:30.208 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 8636 2024-05-08T08:55:30.208 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.208 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:30.208 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.208 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:30.208 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150971+0000", 2024-05-08T08:55:30.208 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:30.208 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.208 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.208 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:30.208 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150971+0000", 2024-05-08T08:55:30.208 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:30.208 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.208 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.208 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:30.208 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150972+0000", 2024-05-08T08:55:30.208 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.71e-07 2024-05-08T08:55:30.209 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.209 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.209 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:30.209 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150977+0000", 2024-05-08T08:55:30.209 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.152e-06 2024-05-08T08:55:30.209 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.209 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.209 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:30.209 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150990+0000", 2024-05-08T08:55:30.209 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2364999999999999e-05 2024-05-08T08:55:30.209 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.209 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.209 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:30.209 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.150998+0000", 2024-05-08T08:55:30.209 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0739999999999993e-06 2024-05-08T08:55:30.209 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.209 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.210 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:30.210 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.151054+0000", 2024-05-08T08:55:30.210 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5949000000000003e-05 2024-05-08T08:55:30.210 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.210 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.210 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:55:30.210 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:54.151060+0000", 2024-05-08T08:55:30.210 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6120000000000004e-06 2024-05-08T08:55:30.210 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.210 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.210 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:30.210 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.367640+0000", 2024-05-08T08:55:30.210 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2165792739999999 2024-05-08T08:55:30.210 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.210 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.210 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:30.211 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.367839+0000", 2024-05-08T08:55:30.211 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000199185 2024-05-08T08:55:30.211 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.211 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.211 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:30.211 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:52:55.368245+0000", 2024-05-08T08:55:30.211 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00040575099999999998 2024-05-08T08:55:30.211 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.211 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:30.211 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.211 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.211 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.211 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5302.0:1516 3.c 3:319bf789:::rbd_data.1140d3fc54c.0000000000000025:head [stat,write 1892352~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:55:30.211 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:27.822220+0000", 2024-05-08T08:55:30.211 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 122.31812862300001, 2024-05-08T08:55:30.211 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.092643744, 2024-05-08T08:55:30.212 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:30.212 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:30.212 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:55:30.212 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:30.212 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5302", 2024-05-08T08:55:30.212 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/359277859", 2024-05-08T08:55:30.212 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1516 2024-05-08T08:55:30.212 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.212 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:30.212 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.212 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:30.212 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822220+0000", 2024-05-08T08:55:30.212 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:30.212 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.212 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.212 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:30.213 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822220+0000", 2024-05-08T08:55:30.213 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:30.213 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.213 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.213 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:30.213 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822221+0000", 2024-05-08T08:55:30.213 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3500000000000002e-07 2024-05-08T08:55:30.213 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.213 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.213 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:30.213 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822228+0000", 2024-05-08T08:55:30.213 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9959999999999996e-06 2024-05-08T08:55:30.213 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.213 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.213 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:30.213 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822234+0000", 2024-05-08T08:55:30.214 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5429999999999999e-06 2024-05-08T08:55:30.214 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.214 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.214 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:30.214 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822238+0000", 2024-05-08T08:55:30.214 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.732e-06 2024-05-08T08:55:30.214 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.214 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.214 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:30.214 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822285+0000", 2024-05-08T08:55:30.214 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6810000000000001e-05 2024-05-08T08:55:30.214 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.214 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.214 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:30.214 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822358+0000", 2024-05-08T08:55:30.215 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3473000000000006e-05 2024-05-08T08:55:30.215 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.215 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.215 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 5", 2024-05-08T08:55:30.215 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.822489+0000", 2024-05-08T08:55:30.215 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013011199999999999 2024-05-08T08:55:30.215 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.215 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.215 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:55:30.215 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.825332+0000", 2024-05-08T08:55:30.215 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0028434799999999998 2024-05-08T08:55:30.215 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.215 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.215 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:55:30.215 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.914753+0000", 2024-05-08T08:55:30.215 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089421424999999999 2024-05-08T08:55:30.215 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.216 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.216 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:55:30.216 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.914807+0000", 2024-05-08T08:55:30.216 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3829000000000002e-05 2024-05-08T08:55:30.216 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.216 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.216 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:30.216 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:27.914864+0000", 2024-05-08T08:55:30.216 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6909e-05 2024-05-08T08:55:30.216 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.216 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:30.216 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.216 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.216 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.216 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:24567 3.6 3:66af1c40:::rbd_data.1140d3fc54c.000000000000005a:18 [read 2080768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:30.216 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.216598+0000", 2024-05-08T08:55:30.217 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 117.923750543, 2024-05-08T08:55:30.217 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071602647000000005, 2024-05-08T08:55:30.217 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:30.217 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:30.217 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:30.217 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:30.217 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:30.217 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:30.217 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 24567 2024-05-08T08:55:30.217 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.217 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:30.217 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.217 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:30.217 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216598+0000", 2024-05-08T08:55:30.217 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:30.218 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.218 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.218 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:30.218 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216598+0000", 2024-05-08T08:55:30.218 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:30.218 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.218 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.218 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:30.218 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216599+0000", 2024-05-08T08:55:30.218 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8999999999999997e-07 2024-05-08T08:55:30.218 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.218 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.218 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:30.218 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216601+0000", 2024-05-08T08:55:30.218 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9889999999999999e-06 2024-05-08T08:55:30.218 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.219 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.219 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:30.219 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216607+0000", 2024-05-08T08:55:30.219 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3600000000000001e-06 2024-05-08T08:55:30.219 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.219 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.219 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:30.219 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.216612+0000", 2024-05-08T08:55:30.219 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.69e-06 2024-05-08T08:55:30.219 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.219 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.219 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:30.219 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.287076+0000", 2024-05-08T08:55:30.219 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070464355000000006 2024-05-08T08:55:30.219 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.220 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.220 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:30.220 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.287151+0000", 2024-05-08T08:55:30.220 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4924000000000003e-05 2024-05-08T08:55:30.220 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.220 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.220 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:30.220 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.288201+0000", 2024-05-08T08:55:30.220 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0010498389999999999 2024-05-08T08:55:30.220 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.220 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:30.220 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.220 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.220 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:30.220 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:55:30.222 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:55:30.222 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:55:30.224+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:55:30.323 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:55:30.325+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:55:30.373 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:55:30.373 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:55:30.373 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:55:30.373 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:55:30.382 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:55:30.423 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:55:30.426+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:55:30.451 INFO:teuthology.orchestra.run.smithi059.stderr:noscrub is unset 2024-05-08T08:55:30.497 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:55:30.497 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:55:30.497 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:55:30.497 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:55:30.497 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:55:30.507 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:55:30.524 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:55:30.526+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:55:30.625 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:55:30.627+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:55:30.661 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:55:30.661 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:55:30.661 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:55:30.661 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:55:30.661 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.661 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:12443 3.0 3:02d0158e:::rbd_data.114071bcf828.0000000000000037:head [stat,delete] snapc 19=[19] RETRY=3 ondisk+retry+write+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:30.661 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.610027+0000", 2024-05-08T08:55:30.661 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 119.052849619, 2024-05-08T08:55:30.661 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.083586333999999998, 2024-05-08T08:55:30.661 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:30.662 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:30.662 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:30.662 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:30.662 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:30.662 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:30.662 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 12443 2024-05-08T08:55:30.662 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.662 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:30.662 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.662 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:30.662 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610027+0000", 2024-05-08T08:55:30.662 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:30.662 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.662 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.662 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:30.662 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610027+0000", 2024-05-08T08:55:30.663 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:30.663 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.663 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.663 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:30.663 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610028+0000", 2024-05-08T08:55:30.663 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.13e-07 2024-05-08T08:55:30.663 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.663 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.663 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:30.663 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610030+0000", 2024-05-08T08:55:30.663 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7669999999999999e-06 2024-05-08T08:55:30.663 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.663 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.663 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:30.663 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610039+0000", 2024-05-08T08:55:30.663 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4959999999999996e-06 2024-05-08T08:55:30.664 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.664 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.664 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:30.664 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.610045+0000", 2024-05-08T08:55:30.664 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6779999999999997e-06 2024-05-08T08:55:30.664 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.664 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.664 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:30.664 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.660873+0000", 2024-05-08T08:55:30.664 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.050828748 2024-05-08T08:55:30.664 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.664 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.664 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:30.664 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.661017+0000", 2024-05-08T08:55:30.664 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014391999999999999 2024-05-08T08:55:30.664 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.665 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.665 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:30.665 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.693613+0000", 2024-05-08T08:55:30.665 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.032596111999999997 2024-05-08T08:55:30.665 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.665 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:30.665 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.665 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.665 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.665 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:14301 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [watch reconnect cookie 139837155901536 gen 12] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:30.665 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.645078+0000", 2024-05-08T08:55:30.665 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 119.017798981, 2024-05-08T08:55:30.665 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088281852999999993, 2024-05-08T08:55:30.665 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:30.665 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:30.666 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:30.666 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:30.666 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:30.666 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:30.666 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14301 2024-05-08T08:55:30.666 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.666 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:30.666 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.666 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:30.666 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645078+0000", 2024-05-08T08:55:30.666 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:30.666 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.666 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.667 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:30.667 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645078+0000", 2024-05-08T08:55:30.667 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:30.667 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.667 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.667 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:30.667 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645084+0000", 2024-05-08T08:55:30.667 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4620000000000001e-06 2024-05-08T08:55:30.667 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.667 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.667 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:30.667 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645088+0000", 2024-05-08T08:55:30.667 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8039999999999999e-06 2024-05-08T08:55:30.667 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.667 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.667 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:30.667 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645106+0000", 2024-05-08T08:55:30.668 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.8318999999999999e-05 2024-05-08T08:55:30.668 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.668 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.668 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:30.668 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.645113+0000", 2024-05-08T08:55:30.668 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0310000000000001e-06 2024-05-08T08:55:30.668 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.668 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.668 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:30.668 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.670493+0000", 2024-05-08T08:55:30.668 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.025379993 2024-05-08T08:55:30.668 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.668 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.668 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:30.669 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.670571+0000", 2024-05-08T08:55:30.669 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7613000000000004e-05 2024-05-08T08:55:30.669 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.669 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.669 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:30.669 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733360+0000", 2024-05-08T08:55:30.669 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.062788630999999998 2024-05-08T08:55:30.669 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.669 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:30.669 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.669 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.669 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.669 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:14420 3.0 3:06a8948e:::rbd_data.114071bcf828.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:30.669 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:31.649639+0000", 2024-05-08T08:55:30.669 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 119.01323783300001, 2024-05-08T08:55:30.669 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.083851897999999994, 2024-05-08T08:55:30.670 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:30.670 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:30.670 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:55:30.670 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:30.670 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:30.670 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:30.670 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 14420 2024-05-08T08:55:30.670 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.670 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:30.670 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.670 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:30.670 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649639+0000", 2024-05-08T08:55:30.670 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:30.670 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.670 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.671 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:30.671 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649639+0000", 2024-05-08T08:55:30.671 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:30.671 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.671 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.671 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:30.671 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649640+0000", 2024-05-08T08:55:30.671 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6899999999999997e-07 2024-05-08T08:55:30.671 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.671 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.671 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:30.671 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649895+0000", 2024-05-08T08:55:30.671 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00025531000000000002 2024-05-08T08:55:30.671 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.671 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.672 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:30.672 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649905+0000", 2024-05-08T08:55:30.672 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0198e-05 2024-05-08T08:55:30.672 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.672 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.672 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:30.672 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.649911+0000", 2024-05-08T08:55:30.672 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2310000000000001e-06 2024-05-08T08:55:30.672 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.672 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.672 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:30.672 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.684099+0000", 2024-05-08T08:55:30.672 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.034188017000000001 2024-05-08T08:55:30.672 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.672 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.672 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:30.673 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.684172+0000", 2024-05-08T08:55:30.673 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3201999999999995e-05 2024-05-08T08:55:30.673 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.673 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.673 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 2", 2024-05-08T08:55:30.673 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.684418+0000", 2024-05-08T08:55:30.673 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00024559099999999997 2024-05-08T08:55:30.673 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.673 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.673 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:55:30.673 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.705538+0000", 2024-05-08T08:55:30.673 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.021119707000000001 2024-05-08T08:55:30.673 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.673 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.673 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:55:30.673 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733399+0000", 2024-05-08T08:55:30.674 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.027861543999999999 2024-05-08T08:55:30.674 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.674 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.674 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:55:30.674 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733445+0000", 2024-05-08T08:55:30.674 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5587000000000002e-05 2024-05-08T08:55:30.674 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.674 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.674 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:30.674 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:31.733491+0000", 2024-05-08T08:55:30.674 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5841999999999999e-05 2024-05-08T08:55:30.674 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.674 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:30.674 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.674 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.674 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.674 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28548 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889189696] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:30.675 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.355669+0000", 2024-05-08T08:55:30.675 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 118.30720725499999, 2024-05-08T08:55:30.675 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078956606999999998, 2024-05-08T08:55:30.675 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:30.675 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:30.675 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:30.675 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:30.675 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:30.675 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:30.675 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28548 2024-05-08T08:55:30.675 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.675 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:30.675 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.675 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:30.675 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355669+0000", 2024-05-08T08:55:30.675 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:30.676 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.676 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.676 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:30.676 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355669+0000", 2024-05-08T08:55:30.676 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:30.676 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.676 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.676 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:30.676 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355762+0000", 2024-05-08T08:55:30.676 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3015000000000003e-05 2024-05-08T08:55:30.676 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.676 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.676 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:30.676 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355765+0000", 2024-05-08T08:55:30.676 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.368e-06 2024-05-08T08:55:30.676 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.677 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.677 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:30.677 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355771+0000", 2024-05-08T08:55:30.677 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5989999999999998e-06 2024-05-08T08:55:30.677 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.677 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.677 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:30.677 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355775+0000", 2024-05-08T08:55:30.677 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7280000000000001e-06 2024-05-08T08:55:30.677 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.677 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.677 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:30.677 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434379+0000", 2024-05-08T08:55:30.677 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078603662000000005 2024-05-08T08:55:30.677 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.677 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.678 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:30.678 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434473+0000", 2024-05-08T08:55:30.678 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4104000000000006e-05 2024-05-08T08:55:30.678 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.678 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.678 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:30.678 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434626+0000", 2024-05-08T08:55:30.678 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00015313100000000001 2024-05-08T08:55:30.678 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.678 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:30.678 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.678 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.678 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.678 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28549 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889202400] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:30.678 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.355781+0000", 2024-05-08T08:55:30.678 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 118.30709568100001, 2024-05-08T08:55:30.678 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079117946999999994, 2024-05-08T08:55:30.679 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:30.679 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:30.679 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:30.679 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:30.679 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:30.679 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:30.679 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28549 2024-05-08T08:55:30.679 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.679 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:30.679 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.679 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:30.679 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355781+0000", 2024-05-08T08:55:30.679 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:30.679 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.679 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.679 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:30.679 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.355781+0000", 2024-05-08T08:55:30.680 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:30.680 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.680 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.680 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:30.680 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356162+0000", 2024-05-08T08:55:30.680 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00038099100000000001 2024-05-08T08:55:30.680 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.680 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.680 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:30.680 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356165+0000", 2024-05-08T08:55:30.680 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1240000000000001e-06 2024-05-08T08:55:30.680 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.680 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.680 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:30.680 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356173+0000", 2024-05-08T08:55:30.680 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.6199999999999999e-06 2024-05-08T08:55:30.680 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.680 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.681 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:30.681 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356182+0000", 2024-05-08T08:55:30.681 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7760000000000003e-06 2024-05-08T08:55:30.681 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.681 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.681 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:30.681 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434675+0000", 2024-05-08T08:55:30.681 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078493119 2024-05-08T08:55:30.681 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.681 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.681 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:30.681 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434765+0000", 2024-05-08T08:55:30.681 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0402000000000006e-05 2024-05-08T08:55:30.681 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.681 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.681 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:30.682 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434899+0000", 2024-05-08T08:55:30.682 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013391499999999999 2024-05-08T08:55:30.682 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.682 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:30.682 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.682 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.682 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.682 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28550 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888597280] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:30.682 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:55:30.682 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 118.306684635, 2024-05-08T08:55:30.682 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078933990999999995, 2024-05-08T08:55:30.682 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:30.682 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:30.682 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:30.682 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:30.682 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:30.683 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:30.683 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28550 2024-05-08T08:55:30.683 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.683 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:30.683 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.683 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:30.683 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:55:30.683 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:30.683 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.683 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.683 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:30.683 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:55:30.683 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:30.683 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.683 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.683 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:30.683 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356192+0000", 2024-05-08T08:55:30.684 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.34e-07 2024-05-08T08:55:30.684 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.684 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.684 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:30.684 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356195+0000", 2024-05-08T08:55:30.684 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8710000000000001e-06 2024-05-08T08:55:30.684 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.684 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.684 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:30.684 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356200+0000", 2024-05-08T08:55:30.684 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3000000000000003e-06 2024-05-08T08:55:30.684 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.684 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.684 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:30.684 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356203+0000", 2024-05-08T08:55:30.684 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4039999999999999e-06 2024-05-08T08:55:30.684 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.685 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.685 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:30.685 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.434928+0000", 2024-05-08T08:55:30.685 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078725331999999995 2024-05-08T08:55:30.685 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.685 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.685 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:30.685 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435007+0000", 2024-05-08T08:55:30.685 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8415000000000001e-05 2024-05-08T08:55:30.685 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.685 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.685 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:30.685 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435126+0000", 2024-05-08T08:55:30.685 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011923500000000001 2024-05-08T08:55:30.685 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.685 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:30.685 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.686 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.686 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.686 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28551 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888527200] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:30.686 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.356210+0000", 2024-05-08T08:55:30.686 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 118.306666287, 2024-05-08T08:55:30.686 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079195072000000005, 2024-05-08T08:55:30.686 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:30.686 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:30.686 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:30.686 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:30.686 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:30.686 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:30.686 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28551 2024-05-08T08:55:30.686 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.687 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:30.687 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.687 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:30.687 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356210+0000", 2024-05-08T08:55:30.687 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:30.687 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.687 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.687 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:30.687 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356210+0000", 2024-05-08T08:55:30.687 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:30.687 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.687 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.687 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:30.687 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356211+0000", 2024-05-08T08:55:30.687 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4499999999999998e-07 2024-05-08T08:55:30.687 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.687 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.687 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:30.688 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356212+0000", 2024-05-08T08:55:30.688 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6029999999999999e-06 2024-05-08T08:55:30.688 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.688 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.688 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:30.688 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356221+0000", 2024-05-08T08:55:30.688 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4200000000000007e-06 2024-05-08T08:55:30.688 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.688 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.688 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:30.688 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356224+0000", 2024-05-08T08:55:30.688 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.038e-06 2024-05-08T08:55:30.688 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.688 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.688 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:30.688 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435178+0000", 2024-05-08T08:55:30.688 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078954625000000001 2024-05-08T08:55:30.689 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.689 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.689 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:30.689 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435256+0000", 2024-05-08T08:55:30.689 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7337999999999997e-05 2024-05-08T08:55:30.689 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.689 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.689 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:30.689 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435405+0000", 2024-05-08T08:55:30.689 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000149703 2024-05-08T08:55:30.689 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.689 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:30.689 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.689 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.689 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.689 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28552 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889231456] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:30.689 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.356228+0000", 2024-05-08T08:55:30.690 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 118.30664860900001, 2024-05-08T08:55:30.690 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079346920000000001, 2024-05-08T08:55:30.690 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:30.690 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:30.690 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:30.690 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:30.690 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:30.690 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:30.690 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28552 2024-05-08T08:55:30.690 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.690 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:30.690 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.690 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:30.690 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356228+0000", 2024-05-08T08:55:30.690 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:30.690 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.690 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.691 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:30.691 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356228+0000", 2024-05-08T08:55:30.691 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:30.691 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.691 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.691 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:30.691 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356959+0000", 2024-05-08T08:55:30.691 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00073097099999999996 2024-05-08T08:55:30.691 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.691 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.691 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:30.691 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356967+0000", 2024-05-08T08:55:30.691 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0409999999999998e-06 2024-05-08T08:55:30.691 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.691 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.691 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:30.691 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356979+0000", 2024-05-08T08:55:30.692 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1489e-05 2024-05-08T08:55:30.692 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.692 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.692 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:30.692 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.356986+0000", 2024-05-08T08:55:30.692 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3699999999999997e-06 2024-05-08T08:55:30.692 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.692 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.692 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:30.692 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435433+0000", 2024-05-08T08:55:30.692 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078447520000000007 2024-05-08T08:55:30.692 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.692 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.692 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:30.692 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435484+0000", 2024-05-08T08:55:30.693 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0229000000000003e-05 2024-05-08T08:55:30.693 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.693 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.693 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:30.693 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435575+0000", 2024-05-08T08:55:30.693 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1299999999999997e-05 2024-05-08T08:55:30.693 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.693 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:30.693 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.693 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.693 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.693 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28553 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888713648] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:30.693 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:55:30.693 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 118.305873951, 2024-05-08T08:55:30.693 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078782034000000001, 2024-05-08T08:55:30.693 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:30.693 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:30.693 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:30.694 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:30.694 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:30.694 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:30.694 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28553 2024-05-08T08:55:30.694 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.694 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:30.694 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.694 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:30.694 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:55:30.694 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:30.694 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.694 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.694 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:30.694 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:55:30.694 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:30.695 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.695 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.695 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:30.695 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357003+0000", 2024-05-08T08:55:30.695 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6600000000000002e-07 2024-05-08T08:55:30.695 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.695 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.695 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:30.695 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357006+0000", 2024-05-08T08:55:30.695 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7190000000000001e-06 2024-05-08T08:55:30.695 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.695 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.695 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:30.695 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357013+0000", 2024-05-08T08:55:30.695 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.435e-06 2024-05-08T08:55:30.695 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.695 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.695 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:30.696 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357018+0000", 2024-05-08T08:55:30.696 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7260000000000002e-06 2024-05-08T08:55:30.696 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.696 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.696 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:30.696 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435623+0000", 2024-05-08T08:55:30.696 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078604972999999995 2024-05-08T08:55:30.696 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.696 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.696 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:30.696 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435695+0000", 2024-05-08T08:55:30.696 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1959999999999995e-05 2024-05-08T08:55:30.696 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.696 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.696 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:30.696 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435785+0000", 2024-05-08T08:55:30.697 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.9754999999999997e-05 2024-05-08T08:55:30.697 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.697 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:30.697 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.697 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.697 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.697 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28554 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888980272] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:30.697 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357024+0000", 2024-05-08T08:55:30.697 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 118.305852315, 2024-05-08T08:55:30.697 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078999870999999999, 2024-05-08T08:55:30.697 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:30.697 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:30.697 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:30.697 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:30.697 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:30.698 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:30.698 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28554 2024-05-08T08:55:30.698 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.698 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:30.698 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.698 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:30.698 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357024+0000", 2024-05-08T08:55:30.698 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:30.698 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.698 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.698 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:30.698 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357024+0000", 2024-05-08T08:55:30.698 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:30.698 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.698 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.699 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:30.699 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357025+0000", 2024-05-08T08:55:30.699 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:55:30.699 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.699 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.699 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:30.699 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357027+0000", 2024-05-08T08:55:30.699 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2050000000000001e-06 2024-05-08T08:55:30.699 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.699 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.699 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:30.699 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357033+0000", 2024-05-08T08:55:30.699 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5679999999999996e-06 2024-05-08T08:55:30.699 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.699 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.699 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:30.700 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357038+0000", 2024-05-08T08:55:30.700 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2889999999999999e-06 2024-05-08T08:55:30.700 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.700 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.700 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:30.700 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435823+0000", 2024-05-08T08:55:30.700 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078785677999999998 2024-05-08T08:55:30.700 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.700 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.700 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:30.700 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.435892+0000", 2024-05-08T08:55:30.700 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8379000000000004e-05 2024-05-08T08:55:30.700 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.700 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.700 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:30.700 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436024+0000", 2024-05-08T08:55:30.700 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013239799999999999 2024-05-08T08:55:30.701 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.701 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:30.701 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.701 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.701 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.701 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28555 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889001104] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:30.701 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:55:30.701 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 118.30583355, 2024-05-08T08:55:30.701 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079115742000000003, 2024-05-08T08:55:30.701 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:30.701 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:30.701 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:30.701 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:30.701 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:30.701 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:30.701 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28555 2024-05-08T08:55:30.701 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.702 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:30.702 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.702 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:30.702 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:55:30.702 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:30.702 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.702 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.702 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:30.702 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:55:30.702 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:30.702 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.702 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.702 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:30.702 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357043+0000", 2024-05-08T08:55:30.702 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9200000000000002e-07 2024-05-08T08:55:30.703 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.703 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.703 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:30.703 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357046+0000", 2024-05-08T08:55:30.703 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1670000000000002e-06 2024-05-08T08:55:30.703 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.703 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.703 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:30.703 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357052+0000", 2024-05-08T08:55:30.703 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.139e-06 2024-05-08T08:55:30.703 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.703 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.703 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:30.703 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357056+0000", 2024-05-08T08:55:30.703 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2440000000000004e-06 2024-05-08T08:55:30.703 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.703 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.704 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:30.704 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436059+0000", 2024-05-08T08:55:30.704 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079003070999999994 2024-05-08T08:55:30.704 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.704 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.704 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:30.704 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436097+0000", 2024-05-08T08:55:30.704 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7784000000000003e-05 2024-05-08T08:55:30.704 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.704 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.704 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:30.704 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436159+0000", 2024-05-08T08:55:30.704 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2045000000000007e-05 2024-05-08T08:55:30.704 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.704 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:30.704 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.704 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.704 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.705 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28556 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889022064] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:30.705 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.357065+0000", 2024-05-08T08:55:30.705 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 118.305811517, 2024-05-08T08:55:30.705 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079327976999999994, 2024-05-08T08:55:30.705 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:30.705 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:30.705 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:30.705 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:30.705 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:30.705 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:30.705 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28556 2024-05-08T08:55:30.705 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.705 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:30.705 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.705 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:30.705 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357065+0000", 2024-05-08T08:55:30.705 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:30.706 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.706 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.706 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:30.706 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.357065+0000", 2024-05-08T08:55:30.706 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:30.706 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.706 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.706 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:30.706 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358958+0000", 2024-05-08T08:55:30.706 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001892915 2024-05-08T08:55:30.706 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.706 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.706 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:30.706 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358962+0000", 2024-05-08T08:55:30.706 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4599999999999999e-06 2024-05-08T08:55:30.706 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.707 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.707 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:30.707 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358970+0000", 2024-05-08T08:55:30.707 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.6090000000000001e-06 2024-05-08T08:55:30.707 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.707 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.707 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:30.707 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.358975+0000", 2024-05-08T08:55:30.707 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7639999999999996e-06 2024-05-08T08:55:30.707 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.707 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.707 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:30.707 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436196+0000", 2024-05-08T08:55:30.707 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.077221564000000006 2024-05-08T08:55:30.707 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.707 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.707 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:30.708 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436269+0000", 2024-05-08T08:55:30.708 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2720999999999997e-05 2024-05-08T08:55:30.708 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.708 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.708 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:30.708 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.436393+0000", 2024-05-08T08:55:30.708 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012394399999999999 2024-05-08T08:55:30.708 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.708 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:30.708 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.708 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.708 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.708 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28563 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888855520] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:30.708 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:55:30.708 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 118.30380211799999, 2024-05-08T08:55:30.708 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078795566999999997, 2024-05-08T08:55:30.708 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:30.709 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:30.709 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:30.709 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:30.709 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:30.709 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:30.709 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28563 2024-05-08T08:55:30.709 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.709 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:30.709 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.709 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:30.709 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:55:30.709 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:30.709 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.709 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.709 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:30.709 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:55:30.709 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:30.710 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.710 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.710 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:30.710 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359075+0000", 2024-05-08T08:55:30.710 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1e-07 2024-05-08T08:55:30.710 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.710 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.710 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:30.710 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359080+0000", 2024-05-08T08:55:30.710 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8940000000000001e-06 2024-05-08T08:55:30.710 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.710 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.710 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:30.710 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359083+0000", 2024-05-08T08:55:30.710 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1949999999999999e-06 2024-05-08T08:55:30.710 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.711 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.711 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:30.711 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359085+0000", 2024-05-08T08:55:30.711 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4710000000000001e-06 2024-05-08T08:55:30.711 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.711 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.711 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:30.711 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437604+0000", 2024-05-08T08:55:30.711 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078518278999999996 2024-05-08T08:55:30.711 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.711 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.711 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:30.711 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437704+0000", 2024-05-08T08:55:30.711 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010010599999999999 2024-05-08T08:55:30.711 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.711 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.711 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:30.712 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437870+0000", 2024-05-08T08:55:30.712 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00016641200000000001 2024-05-08T08:55:30.712 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.712 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:30.712 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.712 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.712 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.712 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28564 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836888538944] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:30.712 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.359089+0000", 2024-05-08T08:55:30.712 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 118.303787903, 2024-05-08T08:55:30.712 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078952675, 2024-05-08T08:55:30.712 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:30.712 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:30.712 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:30.712 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:30.712 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:30.712 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:30.713 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28564 2024-05-08T08:55:30.713 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.713 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:30.713 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.713 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:30.713 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359089+0000", 2024-05-08T08:55:30.713 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:30.713 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.713 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.713 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:30.713 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.359089+0000", 2024-05-08T08:55:30.713 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:30.713 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.713 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.713 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:30.713 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360959+0000", 2024-05-08T08:55:30.714 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0018707089999999999 2024-05-08T08:55:30.714 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.714 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.714 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:30.714 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360963+0000", 2024-05-08T08:55:30.714 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0680000000000001e-06 2024-05-08T08:55:30.714 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.714 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.714 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:30.714 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360969+0000", 2024-05-08T08:55:30.714 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1920000000000002e-06 2024-05-08T08:55:30.714 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.714 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.714 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:30.714 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.360973+0000", 2024-05-08T08:55:30.714 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.031e-06 2024-05-08T08:55:30.715 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.715 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.715 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:30.715 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437916+0000", 2024-05-08T08:55:30.715 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.076942939000000002 2024-05-08T08:55:30.715 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.715 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.715 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:30.715 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.437973+0000", 2024-05-08T08:55:30.715 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7358999999999997e-05 2024-05-08T08:55:30.715 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.715 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.715 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:30.715 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.438041+0000", 2024-05-08T08:55:30.715 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8376999999999999e-05 2024-05-08T08:55:30.715 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.715 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:30.715 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.716 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.716 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.716 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28574 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889427600] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:30.716 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:55:30.716 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 118.301783396, 2024-05-08T08:55:30.716 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078782476000000004, 2024-05-08T08:55:30.716 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:30.716 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:30.716 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:30.716 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:30.716 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:30.716 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:30.716 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28574 2024-05-08T08:55:30.716 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.716 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:30.717 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.717 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:30.717 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:55:30.717 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:30.717 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.717 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.717 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:30.717 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:55:30.717 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:30.717 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.717 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.717 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:30.717 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361093+0000", 2024-05-08T08:55:30.717 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1799999999999999e-07 2024-05-08T08:55:30.717 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.717 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.717 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:30.718 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361095+0000", 2024-05-08T08:55:30.718 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3850000000000001e-06 2024-05-08T08:55:30.718 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.718 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.718 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:30.718 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361098+0000", 2024-05-08T08:55:30.718 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7989999999999998e-06 2024-05-08T08:55:30.718 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.718 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.718 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:30.718 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361100+0000", 2024-05-08T08:55:30.718 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4669999999999998e-06 2024-05-08T08:55:30.718 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.718 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.718 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:30.718 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439768+0000", 2024-05-08T08:55:30.719 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078668057999999999 2024-05-08T08:55:30.719 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.719 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.719 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:30.719 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439811+0000", 2024-05-08T08:55:30.719 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2459000000000001e-05 2024-05-08T08:55:30.719 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.719 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.719 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:30.719 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439876+0000", 2024-05-08T08:55:30.719 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5090000000000002e-05 2024-05-08T08:55:30.719 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.719 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:30.719 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.719 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.720 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.720 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28575 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889439216] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:30.720 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361103+0000", 2024-05-08T08:55:30.720 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 118.301773381, 2024-05-08T08:55:30.720 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078966828000000003, 2024-05-08T08:55:30.720 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:30.720 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:30.720 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:30.720 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:30.720 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:30.720 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:30.720 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28575 2024-05-08T08:55:30.720 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.720 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:30.720 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.720 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:30.720 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361103+0000", 2024-05-08T08:55:30.721 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:30.721 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.721 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.721 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:30.721 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361103+0000", 2024-05-08T08:55:30.721 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:30.721 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.721 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.721 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:30.721 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361104+0000", 2024-05-08T08:55:30.721 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1400000000000001e-07 2024-05-08T08:55:30.721 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.721 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.721 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:30.721 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361105+0000", 2024-05-08T08:55:30.722 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5540000000000001e-06 2024-05-08T08:55:30.722 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.722 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.722 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:30.722 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361108+0000", 2024-05-08T08:55:30.722 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.841e-06 2024-05-08T08:55:30.722 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.722 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.722 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:30.722 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361110+0000", 2024-05-08T08:55:30.722 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.497e-06 2024-05-08T08:55:30.722 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.722 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.722 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:30.722 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.439916+0000", 2024-05-08T08:55:30.722 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078805372999999998 2024-05-08T08:55:30.723 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.723 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.723 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:30.723 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440000+0000", 2024-05-08T08:55:30.723 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4730999999999999e-05 2024-05-08T08:55:30.723 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.723 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.723 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:30.723 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440070+0000", 2024-05-08T08:55:30.723 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9617999999999997e-05 2024-05-08T08:55:30.723 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.723 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:30.723 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.723 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.723 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.723 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28576 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889451328] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:30.724 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361113+0000", 2024-05-08T08:55:30.724 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 118.30176319500001, 2024-05-08T08:55:30.724 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079088862999999995, 2024-05-08T08:55:30.724 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:30.724 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:30.724 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:30.724 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:30.724 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:30.724 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:30.724 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28576 2024-05-08T08:55:30.724 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.724 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:30.724 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.724 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:30.724 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361113+0000", 2024-05-08T08:55:30.724 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:30.725 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.725 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.725 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:30.725 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361113+0000", 2024-05-08T08:55:30.725 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:30.725 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.725 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.725 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:30.725 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361114+0000", 2024-05-08T08:55:30.725 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1299999999999999e-07 2024-05-08T08:55:30.725 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.725 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.725 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:30.725 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361115+0000", 2024-05-08T08:55:30.725 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.4300000000000001e-06 2024-05-08T08:55:30.725 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.725 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.726 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:30.726 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361118+0000", 2024-05-08T08:55:30.726 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9000000000000002e-06 2024-05-08T08:55:30.726 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.726 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.726 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:30.726 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361120+0000", 2024-05-08T08:55:30.726 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4739999999999999e-06 2024-05-08T08:55:30.726 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.726 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.726 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:30.726 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440096+0000", 2024-05-08T08:55:30.726 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.078975629000000006 2024-05-08T08:55:30.726 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.726 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.726 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:30.727 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440135+0000", 2024-05-08T08:55:30.727 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9202000000000002e-05 2024-05-08T08:55:30.727 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.727 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.727 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:30.727 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440202+0000", 2024-05-08T08:55:30.727 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7014999999999994e-05 2024-05-08T08:55:30.727 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.727 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:30.727 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.727 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.727 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.727 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28577 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889462640] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:30.727 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:55:30.727 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 118.301753013, 2024-05-08T08:55:30.728 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079194622000000006, 2024-05-08T08:55:30.728 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:30.728 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:30.728 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:30.728 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:30.728 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:30.728 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:30.728 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28577 2024-05-08T08:55:30.728 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.728 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:30.728 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.728 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:30.728 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:55:30.728 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:30.728 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.728 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.729 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:30.729 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:55:30.729 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:30.729 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.729 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.729 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:30.729 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361124+0000", 2024-05-08T08:55:30.729 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1400000000000001e-07 2024-05-08T08:55:30.729 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.729 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.729 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:30.729 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361125+0000", 2024-05-08T08:55:30.729 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.372e-06 2024-05-08T08:55:30.729 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.729 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.729 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:30.730 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361128+0000", 2024-05-08T08:55:30.730 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0309999999999999e-06 2024-05-08T08:55:30.730 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.730 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.730 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:30.730 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361131+0000", 2024-05-08T08:55:30.730 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4040000000000002e-06 2024-05-08T08:55:30.730 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.730 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.730 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:30.730 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440225+0000", 2024-05-08T08:55:30.730 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079094600000000001 2024-05-08T08:55:30.730 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.730 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.730 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:30.731 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440260+0000", 2024-05-08T08:55:30.731 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4703000000000003e-05 2024-05-08T08:55:30.731 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.731 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.731 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:30.731 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440318+0000", 2024-05-08T08:55:30.731 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8298000000000003e-05 2024-05-08T08:55:30.731 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.731 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:30.731 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.731 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.731 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.731 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5093.0:28578 3.0 3:00dd1c36:::rbd_header.114071bcf828:head [notify cookie 139836889474192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-05-08T08:55:30.731 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:32.361134+0000", 2024-05-08T08:55:30.731 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 118.301742588, 2024-05-08T08:55:30.731 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.079310295000000003, 2024-05-08T08:55:30.732 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:30.732 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:30.732 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:30.732 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:30.732 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5093", 2024-05-08T08:55:30.732 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/4245213914", 2024-05-08T08:55:30.732 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 28578 2024-05-08T08:55:30.732 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.732 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:30.732 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.732 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:30.732 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361134+0000", 2024-05-08T08:55:30.732 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:30.732 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.732 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.732 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:30.732 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.361134+0000", 2024-05-08T08:55:30.733 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:30.733 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.733 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.733 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:30.733 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.362958+0000", 2024-05-08T08:55:30.733 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0018237380000000001 2024-05-08T08:55:30.733 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.733 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.733 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:30.733 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.362961+0000", 2024-05-08T08:55:30.733 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.123e-06 2024-05-08T08:55:30.733 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.733 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.733 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:30.733 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.397404+0000", 2024-05-08T08:55:30.733 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.034443226 2024-05-08T08:55:30.733 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.734 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.734 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:30.734 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.397439+0000", 2024-05-08T08:55:30.734 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4690000000000002e-05 2024-05-08T08:55:30.734 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.734 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.734 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:30.734 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440340+0000", 2024-05-08T08:55:30.734 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.042901419000000003 2024-05-08T08:55:30.734 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.734 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.734 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:30.734 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440376+0000", 2024-05-08T08:55:30.734 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5444999999999997e-05 2024-05-08T08:55:30.734 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.734 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.734 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:30.735 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:32.440444+0000", 2024-05-08T08:55:30.735 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8653999999999997e-05 2024-05-08T08:55:30.735 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.735 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:30.735 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.735 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.735 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.735 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:34133 3.10 3:0cc147c9:::rbd_data.1140100c9703.0000000000000035:head [delete] snapc 14=[] ondisk+write+known_if_redirected+supports_pool_eio e164)", 2024-05-08T08:55:30.735 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:55:15.461161+0000", 2024-05-08T08:55:30.735 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 15.201715251, 2024-05-08T08:55:30.735 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.92225079300000001, 2024-05-08T08:55:30.735 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:30.735 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:30.735 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:55:30.735 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:30.735 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:55:30.735 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:55:30.736 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 34133 2024-05-08T08:55:30.736 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.736 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:30.736 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.736 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:30.736 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:15.461161+0000", 2024-05-08T08:55:30.736 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:30.736 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.736 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.736 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:30.736 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:15.461161+0000", 2024-05-08T08:55:30.736 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:30.736 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.736 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.736 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:30.736 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:15.461162+0000", 2024-05-08T08:55:30.736 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0200000000000001e-07 2024-05-08T08:55:30.737 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.737 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.737 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:30.737 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:15.461166+0000", 2024-05-08T08:55:30.737 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2289999999999997e-06 2024-05-08T08:55:30.737 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.737 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.737 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:30.737 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:15.461175+0000", 2024-05-08T08:55:30.737 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0070000000000003e-06 2024-05-08T08:55:30.737 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.737 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.737 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:30.737 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:15.461182+0000", 2024-05-08T08:55:30.737 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5209999999999999e-06 2024-05-08T08:55:30.737 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.737 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.738 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:30.738 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:15.461248+0000", 2024-05-08T08:55:30.738 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5730999999999998e-05 2024-05-08T08:55:30.738 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.738 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.738 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:55:30.738 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:15.461259+0000", 2024-05-08T08:55:30.738 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0812000000000001e-05 2024-05-08T08:55:30.738 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.738 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.738 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:30.738 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:16.376737+0000", 2024-05-08T08:55:30.738 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.91547897199999995 2024-05-08T08:55:30.738 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.738 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.738 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:30.739 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:16.376832+0000", 2024-05-08T08:55:30.739 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4575999999999997e-05 2024-05-08T08:55:30.739 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.739 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.739 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 2", 2024-05-08T08:55:30.739 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:16.376905+0000", 2024-05-08T08:55:30.739 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3208999999999997e-05 2024-05-08T08:55:30.739 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.739 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.739 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:55:30.739 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:16.378405+0000", 2024-05-08T08:55:30.739 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0014998660000000001 2024-05-08T08:55:30.739 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.739 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.739 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:55:30.739 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:16.383294+0000", 2024-05-08T08:55:30.739 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0048889600000000004 2024-05-08T08:55:30.740 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.740 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.740 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:55:30.740 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:16.383366+0000", 2024-05-08T08:55:30.740 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1428999999999997e-05 2024-05-08T08:55:30.740 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:30.740 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:30.740 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:30.740 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:16.383412+0000", 2024-05-08T08:55:30.740 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6678999999999998e-05 2024-05-08T08:55:30.740 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.740 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:30.740 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.740 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:30.740 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:30.740 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:55:30.742 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:55:30.742 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:55:30.744+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:55:30.842 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:55:30.845+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:55:30.892 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:55:30.892 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:55:30.892 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-05-08T08:55:30.892 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:55:30.900 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:55:30.943 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:55:30.946+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:55:30.959 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T08:55:30.991 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4] out_osds: [5, 0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-05-08T08:55:30.991 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T08:55:30.992 INFO:tasks.thrashosds.thrasher:inject_pause on osd.3 2024-05-08T08:55:30.992 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-05-08T08:55:30.992 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-05-08T08:55:30.992 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok config set filestore_inject_stall 3 2024-05-08T08:55:31.011 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:55:31.012 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:55:31.012 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:55:31.012 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:55:31.012 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:55:31.022 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:55:31.044 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:55:31.046+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:55:31.087 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:55:31.088 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:55:31.106 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:55:31.106 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:55:31.109 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:55:31.109 INFO:teuthology.orchestra.run.smithi192.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-05-08T08:55:31.109 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:55:31.123 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:55:31.123 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:55:31.133 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:31.134 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:31.141 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:55:31.141 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:55:31.142 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:55:31.145 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:55:31.147+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:55:31.159 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:55:31.159 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:55:31.176 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:55:31.177 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:55:31.245 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:55:31.248+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:55:31.265 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:31.265 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:31.273 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:55:31.346 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:55:31.349+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:55:31.378 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:31.379 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:31.387 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:55:31.447 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:55:31.449+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:55:31.498 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:31.498 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:31.508 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:55:31.551 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:55:31.554+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:55:31.623 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:31.623 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:31.632 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:55:31.649 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:55:31.651+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:55:31.750 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:55:31.752+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:55:31.777 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:31.777 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:31.786 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:55:31.851 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:55:31.853+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:55:31.892 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:31.893 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:31.901 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:55:31.951 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:55:31.954+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:55:32.008 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:32.008 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:32.016 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:55:32.057 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:55:32.060+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:55:32.154 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:55:32.156+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:55:32.160 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:32.160 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:32.169 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:55:32.255 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-05-08T08:55:32.257+0000 7fd767c2a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31038) UID: 0 2024-05-08T08:55:32.313 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:32.314 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:32.323 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:55:32.367 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:55:32.370+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:55:32.446 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:32.446 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:32.455 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:55:32.456 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:55:32.459+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:55:32.468 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-05-08T08:55:32.557 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T08:55:32.559+0000 7fed5d9a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31831) UID: 0 2024-05-08T08:55:32.611 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:32.611 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:32.621 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:55:32.664 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:55:32.667+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:55:32.731 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:32.731 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:32.739 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:55:32.758 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:55:32.760+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:55:32.859 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:55:32.862+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:55:32.889 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:32.889 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:32.898 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:55:32.960 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:55:32.962+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:55:33.007 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:33.007 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:33.016 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:55:33.061 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T08:55:33.063+0000 7fba42f3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31829) UID: 0 2024-05-08T08:55:33.125 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:33.125 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:33.134 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:55:33.177 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:55:33.180+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:55:33.188 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T08:55:33.243 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-05-08T08:55:33.244 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:55:33.253 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:55:33.262 INFO:tasks.ceph.osd.3.smithi192.stderr:2024-05-08T08:55:33.265+0000 7fe4e4ca0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31826) UID: 0 2024-05-08T08:55:33.314 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-05-08T08:55:33.315 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:55:33.333 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-05-08T08:55:33.333 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:55:33.350 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-05-08T08:55:33.350 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:55:33.362 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:55:33.365+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:55:33.364 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:55:33.365 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-05-08T08:55:33.365 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-05-08T08:55:33.365 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-05-08T08:55:33.365 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:55:33.369 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-05-08T08:55:33.369 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:55:33.374 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:55:33.388 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-05-08T08:55:33.388 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:55:33.406 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-05-08T08:55:33.406 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:55:33.460 INFO:teuthology.orchestra.run.smithi059.stderr:nodeep-scrub is unset 2024-05-08T08:55:33.463 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-05-08T08:55:33.465+0000 7f6369fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31043) UID: 0 2024-05-08T08:55:33.505 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:55:33.506 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-05-08T08:55:33.506 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-05-08T08:55:33.506 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-05-08T08:55:33.506 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.506 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:437 3.17 3:ea93a512:::rbd_data.1140100c9703.0000000000000077:head [set-alloc-hint object_size 2097152 write_size 2097152,write 1170432~1024] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e63)", 2024-05-08T08:55:33.506 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:08.390804+0000", 2024-05-08T08:55:33.506 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 145.11656260000001, 2024-05-08T08:55:33.506 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0406068690000001, 2024-05-08T08:55:33.506 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:33.506 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:33.506 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:55:33.507 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:33.507 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:33.507 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:33.507 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 437 2024-05-08T08:55:33.507 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.507 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:33.507 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.507 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:33.507 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390804+0000", 2024-05-08T08:55:33.507 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:33.507 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.507 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.507 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:33.507 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390804+0000", 2024-05-08T08:55:33.508 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:33.508 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.508 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.508 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:33.508 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390806+0000", 2024-05-08T08:55:33.508 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.666e-06 2024-05-08T08:55:33.508 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.508 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.508 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:33.508 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390818+0000", 2024-05-08T08:55:33.508 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2187999999999999e-05 2024-05-08T08:55:33.508 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.508 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.508 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:33.508 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390834+0000", 2024-05-08T08:55:33.509 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6191000000000001e-05 2024-05-08T08:55:33.509 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.509 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.509 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:33.509 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390845+0000", 2024-05-08T08:55:33.509 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0831e-05 2024-05-08T08:55:33.509 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.509 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.509 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:33.509 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390922+0000", 2024-05-08T08:55:33.509 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7253e-05 2024-05-08T08:55:33.509 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.509 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.510 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op must wait for map", 2024-05-08T08:55:33.510 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390930+0000", 2024-05-08T08:55:33.510 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8259999999999993e-06 2024-05-08T08:55:33.510 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.510 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.510 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:33.510 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392155+0000", 2024-05-08T08:55:33.510 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0012256649999999999 2024-05-08T08:55:33.510 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.510 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.510 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:55:33.510 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392165+0000", 2024-05-08T08:55:33.510 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5799999999999998e-06 2024-05-08T08:55:33.510 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.510 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.511 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:33.511 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.429596+0000", 2024-05-08T08:55:33.511 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0374313639999999 2024-05-08T08:55:33.511 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.511 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.511 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:33.511 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.429728+0000", 2024-05-08T08:55:33.511 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000131552 2024-05-08T08:55:33.511 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.511 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.511 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 0,1", 2024-05-08T08:55:33.511 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.429860+0000", 2024-05-08T08:55:33.511 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000132137 2024-05-08T08:55:33.511 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.511 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.512 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-05-08T08:55:33.512 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.430811+0000", 2024-05-08T08:55:33.512 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00095040200000000004 2024-05-08T08:55:33.512 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.512 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.512 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:55:33.512 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431126+0000", 2024-05-08T08:55:33.512 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000315006 2024-05-08T08:55:33.512 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.512 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.512 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:55:33.512 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431272+0000", 2024-05-08T08:55:33.512 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000146487 2024-05-08T08:55:33.512 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.513 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.513 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-05-08T08:55:33.513 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431315+0000", 2024-05-08T08:55:33.513 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2697000000000001e-05 2024-05-08T08:55:33.513 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.513 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.513 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:33.513 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431411+0000", 2024-05-08T08:55:33.513 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6024000000000001e-05 2024-05-08T08:55:33.513 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:33.513 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:33.513 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:33.513 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.513 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.513 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:439 3.17 3:ea93a512:::rbd_data.1140100c9703.0000000000000077:head [read 1163264~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-05-08T08:55:33.513 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:08.390871+0000", 2024-05-08T08:55:33.514 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 145.11649508900001, 2024-05-08T08:55:33.514 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0408790590000001, 2024-05-08T08:55:33.514 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:33.514 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:33.514 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:33.514 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:33.514 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:33.514 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:33.514 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 439 2024-05-08T08:55:33.514 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.514 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:33.514 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.514 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:33.514 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390871+0000", 2024-05-08T08:55:33.515 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:33.515 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.515 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.515 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:33.515 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390871+0000", 2024-05-08T08:55:33.515 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:33.515 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.515 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.515 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:33.515 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390872+0000", 2024-05-08T08:55:33.515 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6300000000000005e-07 2024-05-08T08:55:33.515 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.515 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.515 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:33.516 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390875+0000", 2024-05-08T08:55:33.516 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3459999999999998e-06 2024-05-08T08:55:33.516 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.516 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.516 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:33.516 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390891+0000", 2024-05-08T08:55:33.516 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5255e-05 2024-05-08T08:55:33.516 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.516 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.516 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:33.516 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390897+0000", 2024-05-08T08:55:33.516 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2360000000000001e-06 2024-05-08T08:55:33.516 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.516 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.516 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:33.516 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.390995+0000", 2024-05-08T08:55:33.517 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8522000000000003e-05 2024-05-08T08:55:33.517 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.517 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.517 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting_for_map not empty", 2024-05-08T08:55:33.517 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.391005+0000", 2024-05-08T08:55:33.517 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1460000000000002e-06 2024-05-08T08:55:33.517 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.517 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.517 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:33.517 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392191+0000", 2024-05-08T08:55:33.517 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0011860270000000001 2024-05-08T08:55:33.517 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.517 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.517 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:55:33.517 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:08.392197+0000", 2024-05-08T08:55:33.517 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4289999999999998e-06 2024-05-08T08:55:33.518 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.518 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.518 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:33.518 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.430596+0000", 2024-05-08T08:55:33.518 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0383989090000001 2024-05-08T08:55:33.518 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.518 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.518 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for rw locks", 2024-05-08T08:55:33.518 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.430676+0000", 2024-05-08T08:55:33.518 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9781000000000001e-05 2024-05-08T08:55:33.518 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.518 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.518 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:33.518 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431460+0000", 2024-05-08T08:55:33.518 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00078427300000000004 2024-05-08T08:55:33.519 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.519 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.519 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:33.519 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431540+0000", 2024-05-08T08:55:33.519 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0224999999999998e-05 2024-05-08T08:55:33.519 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.519 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.519 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:33.519 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.431750+0000", 2024-05-08T08:55:33.519 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00021024699999999999 2024-05-08T08:55:33.519 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:33.519 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:33.519 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:33.519 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.519 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.520 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:965 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 950272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:33.520 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451075+0000", 2024-05-08T08:55:33.520 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 144.05629128699999, 2024-05-08T08:55:33.520 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087230196999999995, 2024-05-08T08:55:33.520 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:33.520 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:33.520 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:33.520 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:33.520 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:33.520 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:33.520 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 965 2024-05-08T08:55:33.520 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.520 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:33.520 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.520 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:33.521 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451075+0000", 2024-05-08T08:55:33.521 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:33.521 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.521 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.521 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:33.521 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451075+0000", 2024-05-08T08:55:33.521 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:33.521 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.521 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.521 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:33.521 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451076+0000", 2024-05-08T08:55:33.521 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5000000000000002e-07 2024-05-08T08:55:33.521 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.521 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.521 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:33.522 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451078+0000", 2024-05-08T08:55:33.522 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.5639999999999999e-06 2024-05-08T08:55:33.522 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.522 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.522 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:33.522 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451085+0000", 2024-05-08T08:55:33.522 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5640000000000002e-06 2024-05-08T08:55:33.522 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.522 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.522 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:33.522 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451090+0000", 2024-05-08T08:55:33.522 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1699999999999996e-06 2024-05-08T08:55:33.522 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.522 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.523 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:33.523 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538106+0000", 2024-05-08T08:55:33.523 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087015888999999999 2024-05-08T08:55:33.523 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.523 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.523 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:33.523 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538190+0000", 2024-05-08T08:55:33.523 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3961999999999993e-05 2024-05-08T08:55:33.523 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.523 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.523 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:33.523 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538305+0000", 2024-05-08T08:55:33.523 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011529800000000001 2024-05-08T08:55:33.523 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:33.523 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:33.523 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:33.524 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.524 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.524 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:966 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 1441792~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:33.524 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:55:33.524 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 144.05626791399999, 2024-05-08T08:55:33.524 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087446323000000006, 2024-05-08T08:55:33.524 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:33.524 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:33.524 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:33.524 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:33.524 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:33.524 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:33.524 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 966 2024-05-08T08:55:33.524 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.524 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:33.524 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.525 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:33.525 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:55:33.525 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:33.525 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.525 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.525 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:33.525 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:55:33.525 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:33.525 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.525 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.525 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:33.525 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451099+0000", 2024-05-08T08:55:33.525 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3700000000000001e-07 2024-05-08T08:55:33.525 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.526 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.526 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:33.526 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451101+0000", 2024-05-08T08:55:33.526 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2560000000000001e-06 2024-05-08T08:55:33.526 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.526 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.526 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:33.526 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451107+0000", 2024-05-08T08:55:33.526 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1070000000000001e-06 2024-05-08T08:55:33.526 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.526 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.526 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:33.526 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451112+0000", 2024-05-08T08:55:33.526 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1420000000000001e-06 2024-05-08T08:55:33.526 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.527 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.527 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:33.527 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538341+0000", 2024-05-08T08:55:33.527 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087228302999999993 2024-05-08T08:55:33.527 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.527 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.527 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:33.527 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538424+0000", 2024-05-08T08:55:33.527 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3497000000000004e-05 2024-05-08T08:55:33.527 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.527 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.527 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:33.527 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538545+0000", 2024-05-08T08:55:33.527 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000120581 2024-05-08T08:55:33.527 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:33.527 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:33.528 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:33.528 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.528 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.528 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:967 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:33.528 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451120+0000", 2024-05-08T08:55:33.528 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 144.05624643900001, 2024-05-08T08:55:33.528 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087632725999999994, 2024-05-08T08:55:33.528 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:33.528 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:33.528 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:33.528 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:33.528 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:33.528 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:33.528 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 967 2024-05-08T08:55:33.528 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.528 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:33.529 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.529 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:33.529 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451120+0000", 2024-05-08T08:55:33.529 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:33.529 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.529 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.529 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:33.529 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451120+0000", 2024-05-08T08:55:33.529 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:33.529 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.529 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.529 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:33.529 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451121+0000", 2024-05-08T08:55:33.529 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8800000000000002e-07 2024-05-08T08:55:33.529 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.529 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.530 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:33.530 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451130+0000", 2024-05-08T08:55:33.530 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2299999999999997e-06 2024-05-08T08:55:33.530 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.530 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.530 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:33.530 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451137+0000", 2024-05-08T08:55:33.530 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4669999999999998e-06 2024-05-08T08:55:33.530 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.530 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.530 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:33.530 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451143+0000", 2024-05-08T08:55:33.530 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2839999999999997e-06 2024-05-08T08:55:33.530 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.531 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.531 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:33.531 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538575+0000", 2024-05-08T08:55:33.531 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087432789999999996 2024-05-08T08:55:33.531 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.531 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.531 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:33.531 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538648+0000", 2024-05-08T08:55:33.531 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2877e-05 2024-05-08T08:55:33.531 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.531 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.531 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:33.531 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538753+0000", 2024-05-08T08:55:33.531 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010449 2024-05-08T08:55:33.532 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:33.532 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:33.532 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:33.532 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.532 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.532 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:968 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 2424832~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:33.532 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451152+0000", 2024-05-08T08:55:33.532 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 144.05621435399999, 2024-05-08T08:55:33.532 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087813325999999997, 2024-05-08T08:55:33.532 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:33.532 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:33.532 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:33.532 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:33.532 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:33.532 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:33.533 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 968 2024-05-08T08:55:33.533 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.533 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:33.533 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.533 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:33.533 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451152+0000", 2024-05-08T08:55:33.533 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:33.533 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.533 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.533 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:33.533 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451152+0000", 2024-05-08T08:55:33.533 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:33.533 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.533 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.533 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:33.534 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451153+0000", 2024-05-08T08:55:33.534 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3099999999999998e-07 2024-05-08T08:55:33.534 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.534 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.534 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:33.534 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451155+0000", 2024-05-08T08:55:33.534 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4760000000000001e-06 2024-05-08T08:55:33.534 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.534 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.534 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:33.534 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451162+0000", 2024-05-08T08:55:33.534 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5969999999999997e-06 2024-05-08T08:55:33.534 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.534 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.534 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:33.535 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451167+0000", 2024-05-08T08:55:33.535 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.977e-06 2024-05-08T08:55:33.535 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.535 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.535 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:33.535 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538781+0000", 2024-05-08T08:55:33.535 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087614846999999996 2024-05-08T08:55:33.535 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.535 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.535 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:33.535 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538857+0000", 2024-05-08T08:55:33.535 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5358000000000006e-05 2024-05-08T08:55:33.535 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.535 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.536 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:33.536 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538965+0000", 2024-05-08T08:55:33.536 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010864 2024-05-08T08:55:33.536 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:33.536 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:33.536 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:33.536 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.536 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.536 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:969 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 2916352~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:33.536 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:55:33.536 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 144.056192649, 2024-05-08T08:55:33.536 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087998980000000004, 2024-05-08T08:55:33.536 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:33.536 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:33.536 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:33.536 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:33.536 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:33.537 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:33.537 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 969 2024-05-08T08:55:33.537 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.537 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:33.537 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.537 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:33.537 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:55:33.537 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:33.537 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.537 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.537 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:33.537 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:55:33.537 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:33.537 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.537 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.538 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:33.538 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451174+0000", 2024-05-08T08:55:33.538 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3599999999999999e-07 2024-05-08T08:55:33.538 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.538 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.538 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:33.538 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451177+0000", 2024-05-08T08:55:33.538 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4590000000000001e-06 2024-05-08T08:55:33.538 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.538 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.538 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:33.538 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451184+0000", 2024-05-08T08:55:33.538 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4109999999999999e-06 2024-05-08T08:55:33.538 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.538 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.539 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:33.539 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451191+0000", 2024-05-08T08:55:33.539 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7279999999999998e-06 2024-05-08T08:55:33.539 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.539 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.539 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:33.539 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.538997+0000", 2024-05-08T08:55:33.539 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087806170000000003 2024-05-08T08:55:33.539 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.539 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.539 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:33.539 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539069+0000", 2024-05-08T08:55:33.539 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2373999999999995e-05 2024-05-08T08:55:33.539 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.540 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.540 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:33.540 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539173+0000", 2024-05-08T08:55:33.540 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000103402 2024-05-08T08:55:33.540 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:33.540 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:33.540 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:33.540 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.540 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.540 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:970 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 3407872~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:33.540 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451198+0000", 2024-05-08T08:55:33.540 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 144.056168052, 2024-05-08T08:55:33.540 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088183368999999998, 2024-05-08T08:55:33.540 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:33.540 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:33.540 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:33.540 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:33.541 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:33.541 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:33.541 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 970 2024-05-08T08:55:33.541 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.541 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:33.541 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.541 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:33.541 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451198+0000", 2024-05-08T08:55:33.541 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:33.541 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.541 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.541 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:33.541 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451198+0000", 2024-05-08T08:55:33.541 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:33.541 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.541 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.542 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:33.542 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451199+0000", 2024-05-08T08:55:33.542 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7400000000000003e-07 2024-05-08T08:55:33.542 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.542 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.542 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:33.542 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451211+0000", 2024-05-08T08:55:33.542 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2109999999999999e-05 2024-05-08T08:55:33.542 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.542 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.542 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:33.542 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451219+0000", 2024-05-08T08:55:33.542 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4170000000000003e-06 2024-05-08T08:55:33.542 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.543 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.543 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:33.543 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451224+0000", 2024-05-08T08:55:33.543 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3990000000000003e-06 2024-05-08T08:55:33.543 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.543 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.543 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:33.543 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539204+0000", 2024-05-08T08:55:33.543 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.087980099000000006 2024-05-08T08:55:33.543 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.543 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.543 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:33.543 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539276+0000", 2024-05-08T08:55:33.543 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1513000000000005e-05 2024-05-08T08:55:33.543 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.544 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.544 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:33.544 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539382+0000", 2024-05-08T08:55:33.544 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010615699999999999 2024-05-08T08:55:33.544 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:33.544 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:33.544 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:33.544 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.544 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.544 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:971 3.9 3:932d05fd:::rbd_data.1140b92ee0d3.0000000000000000:13 [read 3899392~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:33.544 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451233+0000", 2024-05-08T08:55:33.544 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 144.05613334500001, 2024-05-08T08:55:33.544 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088431525999999996, 2024-05-08T08:55:33.544 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:33.544 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:33.544 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:33.545 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:33.545 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:33.545 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:33.545 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 971 2024-05-08T08:55:33.545 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.545 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:33.545 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.545 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:33.545 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451233+0000", 2024-05-08T08:55:33.545 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:33.545 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.545 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.545 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:33.545 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451233+0000", 2024-05-08T08:55:33.546 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:33.546 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.546 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.546 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:33.546 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451234+0000", 2024-05-08T08:55:33.546 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8500000000000001e-07 2024-05-08T08:55:33.546 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.546 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.546 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:33.546 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451237+0000", 2024-05-08T08:55:33.546 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8320000000000002e-06 2024-05-08T08:55:33.546 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.546 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.546 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:33.546 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451243+0000", 2024-05-08T08:55:33.547 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6749999999999996e-06 2024-05-08T08:55:33.547 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.547 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.547 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:33.547 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451249+0000", 2024-05-08T08:55:33.547 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1229999999999997e-06 2024-05-08T08:55:33.547 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.547 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.547 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:33.547 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539412+0000", 2024-05-08T08:55:33.548 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088162668999999999 2024-05-08T08:55:33.548 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.548 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.548 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:33.548 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539505+0000", 2024-05-08T08:55:33.548 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3240999999999997e-05 2024-05-08T08:55:33.548 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.548 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.548 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:33.548 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539665+0000", 2024-05-08T08:55:33.548 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00015930100000000001 2024-05-08T08:55:33.548 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:33.548 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:33.548 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:33.548 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.548 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.549 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:972 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 196608~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:33.549 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451257+0000", 2024-05-08T08:55:33.549 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 144.0561094, 2024-05-08T08:55:33.549 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088719928000000003, 2024-05-08T08:55:33.549 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:33.549 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:33.549 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:33.549 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:33.549 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:33.549 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:33.549 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 972 2024-05-08T08:55:33.549 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.549 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:33.549 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.550 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:33.550 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451257+0000", 2024-05-08T08:55:33.550 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:33.550 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.550 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.550 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:33.550 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451257+0000", 2024-05-08T08:55:33.550 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:33.550 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.550 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.550 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:33.550 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451258+0000", 2024-05-08T08:55:33.550 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7999999999999996e-07 2024-05-08T08:55:33.550 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.550 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.551 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:33.551 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451260+0000", 2024-05-08T08:55:33.551 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4609999999999998e-06 2024-05-08T08:55:33.551 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.551 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.551 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:33.551 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451267+0000", 2024-05-08T08:55:33.551 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.81e-06 2024-05-08T08:55:33.551 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.551 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.551 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:33.551 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451273+0000", 2024-05-08T08:55:33.551 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0859999999999998e-06 2024-05-08T08:55:33.551 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.551 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.551 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:33.552 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539718+0000", 2024-05-08T08:55:33.552 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088445059000000006 2024-05-08T08:55:33.552 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.552 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.552 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:33.552 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539834+0000", 2024-05-08T08:55:33.552 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000116426 2024-05-08T08:55:33.552 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.552 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.552 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:33.552 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.539977+0000", 2024-05-08T08:55:33.552 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014260600000000001 2024-05-08T08:55:33.552 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:33.552 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:33.553 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:33.553 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.553 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.553 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:973 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 688128~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:33.553 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:55:33.553 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 144.056086816, 2024-05-08T08:55:33.553 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088918496, 2024-05-08T08:55:33.553 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:33.553 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:33.553 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:33.553 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:33.553 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:33.553 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:33.553 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 973 2024-05-08T08:55:33.554 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.554 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:33.554 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.554 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:33.554 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:55:33.554 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:33.554 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.554 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.554 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:33.554 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:55:33.554 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:33.554 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.554 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.554 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:33.554 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451280+0000", 2024-05-08T08:55:33.555 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9099999999999999e-07 2024-05-08T08:55:33.555 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.555 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.555 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:33.555 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451282+0000", 2024-05-08T08:55:33.555 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1550000000000001e-06 2024-05-08T08:55:33.555 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.555 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.555 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:33.555 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451288+0000", 2024-05-08T08:55:33.555 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8889999999999999e-06 2024-05-08T08:55:33.555 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.555 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.555 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:33.556 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451294+0000", 2024-05-08T08:55:33.556 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0850000000000002e-06 2024-05-08T08:55:33.556 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.556 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.556 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:33.556 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540011+0000", 2024-05-08T08:55:33.556 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088717314000000005 2024-05-08T08:55:33.556 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.556 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.556 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:33.556 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540089+0000", 2024-05-08T08:55:33.556 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7438e-05 2024-05-08T08:55:33.556 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.556 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.556 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:33.556 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540198+0000", 2024-05-08T08:55:33.557 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000109224 2024-05-08T08:55:33.557 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:33.557 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:33.557 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:33.557 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.557 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.557 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:974 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 1179648~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:33.557 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451301+0000", 2024-05-08T08:55:33.557 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 144.05606518299999, 2024-05-08T08:55:33.557 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089105078000000004, 2024-05-08T08:55:33.557 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:33.557 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:33.557 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:33.557 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:33.557 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:33.558 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:33.558 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 974 2024-05-08T08:55:33.558 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.558 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:33.558 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.558 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:33.558 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451301+0000", 2024-05-08T08:55:33.558 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:33.558 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.558 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.558 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:33.558 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451301+0000", 2024-05-08T08:55:33.558 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:33.559 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.559 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.559 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:33.559 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451302+0000", 2024-05-08T08:55:33.559 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7800000000000002e-07 2024-05-08T08:55:33.559 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.559 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.559 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:33.559 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451304+0000", 2024-05-08T08:55:33.559 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.65e-06 2024-05-08T08:55:33.559 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.559 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.559 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:33.559 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451311+0000", 2024-05-08T08:55:33.559 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.472e-06 2024-05-08T08:55:33.559 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.560 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.560 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:33.560 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451316+0000", 2024-05-08T08:55:33.560 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9189999999999998e-06 2024-05-08T08:55:33.560 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.560 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.560 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:33.560 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540234+0000", 2024-05-08T08:55:33.560 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.088917752000000003 2024-05-08T08:55:33.560 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.560 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.560 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:33.560 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540308+0000", 2024-05-08T08:55:33.560 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4758000000000005e-05 2024-05-08T08:55:33.560 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.561 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.561 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:33.561 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540406+0000", 2024-05-08T08:55:33.561 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8048999999999996e-05 2024-05-08T08:55:33.561 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:33.561 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:33.561 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:33.561 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.561 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.561 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:975 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 1671168~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:33.561 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:55:33.561 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 144.056022853, 2024-05-08T08:55:33.561 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089294139999999994, 2024-05-08T08:55:33.561 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:33.561 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:33.561 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:33.562 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:33.562 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:33.562 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:33.562 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 975 2024-05-08T08:55:33.562 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.562 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:33.562 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.562 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:33.562 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:55:33.562 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:33.562 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.562 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.562 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:33.562 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:55:33.563 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:33.563 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.563 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.563 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:33.563 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451344+0000", 2024-05-08T08:55:33.563 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1299999999999999e-07 2024-05-08T08:55:33.563 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.563 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.563 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:33.563 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451348+0000", 2024-05-08T08:55:33.563 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.484e-06 2024-05-08T08:55:33.563 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.563 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.563 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:33.564 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451356+0000", 2024-05-08T08:55:33.564 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8499999999999994e-06 2024-05-08T08:55:33.564 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.564 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.564 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:33.564 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451366+0000", 2024-05-08T08:55:33.564 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.9850000000000001e-06 2024-05-08T08:55:33.564 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.564 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.564 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:33.564 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540448+0000", 2024-05-08T08:55:33.564 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089082254 2024-05-08T08:55:33.564 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.564 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.564 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:33.565 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540528+0000", 2024-05-08T08:55:33.565 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0234999999999993e-05 2024-05-08T08:55:33.565 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.565 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.565 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:33.565 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540638+0000", 2024-05-08T08:55:33.565 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010951900000000001 2024-05-08T08:55:33.565 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:33.565 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:33.565 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:33.565 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.565 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.565 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:976 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 2162688~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:33.565 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451375+0000", 2024-05-08T08:55:33.566 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 144.05599110599999, 2024-05-08T08:55:33.566 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089472745000000006, 2024-05-08T08:55:33.566 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:33.566 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:33.566 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:33.566 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:33.566 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:33.566 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:33.566 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 976 2024-05-08T08:55:33.566 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.566 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:33.566 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.566 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:33.566 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451375+0000", 2024-05-08T08:55:33.566 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:33.567 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.567 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.567 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:33.567 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451375+0000", 2024-05-08T08:55:33.567 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:33.567 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.567 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.567 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:33.567 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451376+0000", 2024-05-08T08:55:33.567 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2399999999999998e-07 2024-05-08T08:55:33.567 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.567 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.567 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:33.567 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451379+0000", 2024-05-08T08:55:33.568 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6359999999999998e-06 2024-05-08T08:55:33.568 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.568 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.568 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:33.568 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451387+0000", 2024-05-08T08:55:33.568 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9929999999999995e-06 2024-05-08T08:55:33.568 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.568 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.568 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:33.568 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451393+0000", 2024-05-08T08:55:33.568 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4699999999999999e-06 2024-05-08T08:55:33.568 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.568 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.568 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:33.568 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540677+0000", 2024-05-08T08:55:33.569 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089283504999999999 2024-05-08T08:55:33.569 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.569 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.569 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:33.569 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540743+0000", 2024-05-08T08:55:33.569 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6303999999999994e-05 2024-05-08T08:55:33.569 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.569 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.569 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:33.569 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540848+0000", 2024-05-08T08:55:33.569 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000105213 2024-05-08T08:55:33.569 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:33.569 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:33.569 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:33.569 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.570 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.570 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:977 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 2654208~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:33.570 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:55:33.570 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 144.05596567399999, 2024-05-08T08:55:33.570 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089698473000000001, 2024-05-08T08:55:33.570 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:33.570 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:33.570 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:33.570 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:33.570 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:33.570 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:33.570 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 977 2024-05-08T08:55:33.570 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.570 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:33.570 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.571 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:33.571 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:55:33.571 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:33.571 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.571 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.571 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:33.571 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:55:33.571 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:33.571 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.571 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.571 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:33.571 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451401+0000", 2024-05-08T08:55:33.571 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2899999999999999e-07 2024-05-08T08:55:33.571 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.572 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.572 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:33.572 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451404+0000", 2024-05-08T08:55:33.572 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6649999999999999e-06 2024-05-08T08:55:33.572 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.572 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.572 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:33.572 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451410+0000", 2024-05-08T08:55:33.572 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3269999999999997e-06 2024-05-08T08:55:33.572 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.572 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.572 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:33.572 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451415+0000", 2024-05-08T08:55:33.572 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9529999999999998e-06 2024-05-08T08:55:33.573 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.573 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.573 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:33.573 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.540912+0000", 2024-05-08T08:55:33.573 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089496950000000006 2024-05-08T08:55:33.573 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.573 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.573 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:33.573 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541000+0000", 2024-05-08T08:55:33.573 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7429000000000006e-05 2024-05-08T08:55:33.573 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.573 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.573 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:33.573 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541099+0000", 2024-05-08T08:55:33.573 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.9720000000000001e-05 2024-05-08T08:55:33.574 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:33.574 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:33.574 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:33.574 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.574 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.574 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:978 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 3145728~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:33.574 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:55:33.574 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 144.055944466, 2024-05-08T08:55:33.574 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089900635000000007, 2024-05-08T08:55:33.574 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:33.574 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:33.574 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:33.574 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:33.574 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:33.574 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:33.575 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 978 2024-05-08T08:55:33.575 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.575 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:33.575 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.575 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:33.575 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:55:33.575 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:33.575 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.575 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.575 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:33.575 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:55:33.575 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:33.575 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.576 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.576 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:33.576 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451422+0000", 2024-05-08T08:55:33.576 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9400000000000001e-07 2024-05-08T08:55:33.576 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.576 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.576 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:33.576 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451425+0000", 2024-05-08T08:55:33.576 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2460000000000002e-06 2024-05-08T08:55:33.576 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.576 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.576 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:33.576 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451431+0000", 2024-05-08T08:55:33.576 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2419999999999997e-06 2024-05-08T08:55:33.576 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.577 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.577 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:33.577 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451436+0000", 2024-05-08T08:55:33.577 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.887e-06 2024-05-08T08:55:33.577 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.577 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.577 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:33.577 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541152+0000", 2024-05-08T08:55:33.577 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089716194999999999 2024-05-08T08:55:33.577 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.577 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.577 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:33.577 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541224+0000", 2024-05-08T08:55:33.577 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2416000000000006e-05 2024-05-08T08:55:33.577 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.578 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.578 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:33.578 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541323+0000", 2024-05-08T08:55:33.578 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8255e-05 2024-05-08T08:55:33.578 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:33.578 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:33.578 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:33.578 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.578 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.578 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:979 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 3637248~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:33.578 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451462+0000", 2024-05-08T08:55:33.578 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 144.05590401699999, 2024-05-08T08:55:33.578 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.090036441999999994, 2024-05-08T08:55:33.578 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:33.578 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:33.579 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:33.579 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:33.579 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:33.579 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:33.579 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 979 2024-05-08T08:55:33.579 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.579 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:33.579 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.579 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:33.579 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451462+0000", 2024-05-08T08:55:33.579 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:33.579 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.579 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.579 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:33.580 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451462+0000", 2024-05-08T08:55:33.580 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:33.580 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.580 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.580 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:33.580 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451463+0000", 2024-05-08T08:55:33.580 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4300000000000002e-07 2024-05-08T08:55:33.580 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.580 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.580 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:33.580 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451471+0000", 2024-05-08T08:55:33.580 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0800000000000006e-06 2024-05-08T08:55:33.580 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.580 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.580 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:33.581 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451481+0000", 2024-05-08T08:55:33.581 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.38e-06 2024-05-08T08:55:33.581 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.581 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.581 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:33.581 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451487+0000", 2024-05-08T08:55:33.581 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4590000000000003e-06 2024-05-08T08:55:33.581 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.581 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.581 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:33.581 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541350+0000", 2024-05-08T08:55:33.581 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089862599000000001 2024-05-08T08:55:33.581 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.581 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.581 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:33.581 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541407+0000", 2024-05-08T08:55:33.582 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7667999999999997e-05 2024-05-08T08:55:33.582 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.582 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.582 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:33.582 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541499+0000", 2024-05-08T08:55:33.582 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1513000000000003e-05 2024-05-08T08:55:33.582 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:33.582 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:33.582 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:33.582 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.582 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.582 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.5083.0:980 3.9 3:9216dde7:::rbd_data.1140b92ee0d3.0000000000000001:13 [read 4128768~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-05-08T08:55:33.582 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:53:09.451496+0000", 2024-05-08T08:55:33.582 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 144.055870261, 2024-05-08T08:55:33.583 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.090258985, 2024-05-08T08:55:33.583 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:33.583 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:33.583 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:33.583 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:33.583 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.5083", 2024-05-08T08:55:33.583 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/3736942548", 2024-05-08T08:55:33.583 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 980 2024-05-08T08:55:33.583 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.583 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:33.583 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.583 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:33.583 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451496+0000", 2024-05-08T08:55:33.583 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:33.583 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.583 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.584 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:33.584 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451496+0000", 2024-05-08T08:55:33.584 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:33.584 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.584 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.584 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:33.584 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451497+0000", 2024-05-08T08:55:33.584 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7599999999999997e-07 2024-05-08T08:55:33.584 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.584 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.584 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:33.584 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451500+0000", 2024-05-08T08:55:33.584 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7930000000000002e-06 2024-05-08T08:55:33.584 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.585 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.585 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:33.585 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451506+0000", 2024-05-08T08:55:33.585 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6980000000000001e-06 2024-05-08T08:55:33.585 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.585 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.585 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:33.585 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.451512+0000", 2024-05-08T08:55:33.585 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2490000000000001e-06 2024-05-08T08:55:33.585 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.585 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.585 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:33.585 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541550+0000", 2024-05-08T08:55:33.585 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.090037961999999999 2024-05-08T08:55:33.585 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.585 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.586 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:33.586 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541647+0000", 2024-05-08T08:55:33.586 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6924999999999998e-05 2024-05-08T08:55:33.586 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.586 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.586 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:33.586 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:53:09.541755+0000", 2024-05-08T08:55:33.586 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000108682 2024-05-08T08:55:33.586 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:33.586 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:33.586 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:33.586 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.586 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.586 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:33397 3.7 3:e5a023c9:::rbd_header.11402419351a:head [watch reconnect cookie 140139347723488 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e156)", 2024-05-08T08:55:33.587 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:55:07.435461+0000", 2024-05-08T08:55:33.587 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 26.071904962000001, 2024-05-08T08:55:33.587 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.009753092, 2024-05-08T08:55:33.587 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:33.587 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:33.587 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-05-08T08:55:33.587 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:33.587 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:55:33.587 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:55:33.587 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 33397 2024-05-08T08:55:33.587 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.587 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:33.587 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.587 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:33.587 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:07.435461+0000", 2024-05-08T08:55:33.588 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:33.588 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.588 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.588 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:33.588 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:07.435461+0000", 2024-05-08T08:55:33.588 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:33.588 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.588 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.588 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:33.588 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:07.435463+0000", 2024-05-08T08:55:33.588 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.7749999999999999e-06 2024-05-08T08:55:33.588 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.588 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.588 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:33.588 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:07.435471+0000", 2024-05-08T08:55:33.589 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.2199999999999992e-06 2024-05-08T08:55:33.589 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.589 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.589 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:33.589 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:07.435487+0000", 2024-05-08T08:55:33.589 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5806000000000001e-05 2024-05-08T08:55:33.589 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.589 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.589 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:33.589 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:07.435867+0000", 2024-05-08T08:55:33.589 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00037973999999999999 2024-05-08T08:55:33.589 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.589 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.589 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:33.589 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:07.436510+0000", 2024-05-08T08:55:33.589 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00064263299999999998 2024-05-08T08:55:33.590 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.590 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.590 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:55:33.590 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:07.436524+0000", 2024-05-08T08:55:33.590 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.4316e-05 2024-05-08T08:55:33.590 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.590 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.590 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:33.590 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:08.439082+0000", 2024-05-08T08:55:33.590 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.002558235 2024-05-08T08:55:33.590 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.590 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.590 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-05-08T08:55:33.590 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:08.439286+0000", 2024-05-08T08:55:33.591 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00020426600000000001 2024-05-08T08:55:33.591 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.591 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.591 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:33.591 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:08.445215+0000", 2024-05-08T08:55:33.591 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0059281009999999999 2024-05-08T08:55:33.591 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:33.591 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:33.591 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:33.591 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.591 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.591 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.4416.0:33398 3.7 3.93c405a7 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e156)", 2024-05-08T08:55:33.591 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-05-08T08:55:07.435912+0000", 2024-05-08T08:55:33.591 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 26.071454763999999, 2024-05-08T08:55:33.591 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0044546430000001, 2024-05-08T08:55:33.591 INFO:teuthology.orchestra.run.smithi059.stdout: "continuous": false, 2024-05-08T08:55:33.592 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-05-08T08:55:33.592 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "reached pg", 2024-05-08T08:55:33.592 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-05-08T08:55:33.592 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.4416", 2024-05-08T08:55:33.592 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:55:33.592 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 33398 2024-05-08T08:55:33.592 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.592 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-05-08T08:55:33.592 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.592 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-05-08T08:55:33.592 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:07.435912+0000", 2024-05-08T08:55:33.592 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:33.592 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.592 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.592 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-05-08T08:55:33.593 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:07.435912+0000", 2024-05-08T08:55:33.593 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-05-08T08:55:33.593 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.593 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.593 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-05-08T08:55:33.593 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:07.435913+0000", 2024-05-08T08:55:33.593 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2200000000000002e-07 2024-05-08T08:55:33.593 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.593 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.593 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-05-08T08:55:33.593 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:07.435919+0000", 2024-05-08T08:55:33.593 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3820000000000001e-06 2024-05-08T08:55:33.593 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.593 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.593 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-05-08T08:55:33.594 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:07.435932+0000", 2024-05-08T08:55:33.594 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2788e-05 2024-05-08T08:55:33.594 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.594 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.594 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-05-08T08:55:33.594 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:07.435940+0000", 2024-05-08T08:55:33.594 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.6710000000000005e-06 2024-05-08T08:55:33.594 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.594 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.594 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:33.594 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:07.436558+0000", 2024-05-08T08:55:33.594 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00061782299999999998 2024-05-08T08:55:33.594 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.594 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.595 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-05-08T08:55:33.595 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:07.436572+0000", 2024-05-08T08:55:33.595 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.34e-05 2024-05-08T08:55:33.595 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.595 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.595 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-05-08T08:55:33.595 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:08.440285+0000", 2024-05-08T08:55:33.595 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.003713388 2024-05-08T08:55:33.595 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T08:55:33.595 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T08:55:33.595 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-05-08T08:55:33.595 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-05-08T08:55:08.440366+0000", 2024-05-08T08:55:33.595 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1268999999999997e-05 2024-05-08T08:55:33.595 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:33.595 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:33.596 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:33.596 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T08:55:33.596 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T08:55:33.596 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:55:33.598 INFO:teuthology.orchestra.run.smithi196.stdout:All operations completed A-OK! 2024-05-08T08:55:33.598 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:55:33.598 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-05-08T08:55:33.599+0000 7fb626715640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31040) UID: 0 2024-05-08T08:55:33.599 DEBUG:teuthology.parallel:result is None 2024-05-08T08:55:33.599 DEBUG:teuthology.run_tasks:Unwinding manager rbd_fsx 2024-05-08T08:55:33.609 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-05-08T08:55:33.617 INFO:tasks.thrashosds:joining thrashosds 2024-05-08T08:55:33.724 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:55:33.724 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:55:33.724 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:55:33.725 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:55:33.734 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:55:33.838 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:55:33.839 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:55:33.839 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:55:33.839 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:55:33.839 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:55:33.847 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:55:33.955 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:55:33.955 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:55:33.955 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:55:33.955 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:55:33.955 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.955 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4621.0:9089 3.1f e33/18 3:fbf5caf1:::rbd_data.11402f152156.0000000000000000:head v 33'117, mlcod=31'115)", 2024-05-08T08:55:33.955 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:55:33.955 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 181.43669951999999, 2024-05-08T08:55:33.955 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.111244524, 2024-05-08T08:55:33.955 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:33.955 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:33.955 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:55:33.956 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:33.956 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.956 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:33.956 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:55:33.956 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:33.956 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.956 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.956 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:33.956 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:55:33.956 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:33.956 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.956 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.956 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:33.956 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.519618+0000", 2024-05-08T08:55:33.956 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5900000000000001e-07 2024-05-08T08:55:33.957 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.957 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.957 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:33.957 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539600+0000", 2024-05-08T08:55:33.957 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.019981364000000001 2024-05-08T08:55:33.957 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.957 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.957 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:33.957 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539624+0000", 2024-05-08T08:55:33.957 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4043e-05 2024-05-08T08:55:33.957 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.957 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.957 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:33.957 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539635+0000", 2024-05-08T08:55:33.957 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1511e-05 2024-05-08T08:55:33.958 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.958 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.958 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:33.958 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539728+0000", 2024-05-08T08:55:33.958 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2498999999999996e-05 2024-05-08T08:55:33.958 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.958 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.958 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:33.958 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.539746+0000", 2024-05-08T08:55:33.958 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8289000000000001e-05 2024-05-08T08:55:33.958 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.958 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.958 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:55:33.958 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.630822+0000", 2024-05-08T08:55:33.958 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.091076283999999993 2024-05-08T08:55:33.959 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.959 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.959 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:33.959 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:32.630863+0000", 2024-05-08T08:55:33.959 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0374999999999999e-05 2024-05-08T08:55:33.959 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:33.959 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:33.959 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:33.959 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.959 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.959 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4195.0:4 3.11 3:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-05-08T08:55:33.959 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:49.764654+0000", 2024-05-08T08:55:33.959 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 164.191663979, 2024-05-08T08:55:33.959 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.30714275499999999, 2024-05-08T08:55:33.959 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:33.960 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:33.960 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-05-08T08:55:33.960 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:33.960 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4195", 2024-05-08T08:55:33.960 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/59179521", 2024-05-08T08:55:33.960 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4 2024-05-08T08:55:33.960 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.960 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:33.960 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.960 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:33.960 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764654+0000", 2024-05-08T08:55:33.960 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:33.960 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.960 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.960 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:33.961 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764654+0000", 2024-05-08T08:55:33.961 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:33.961 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.961 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.961 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:33.961 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764655+0000", 2024-05-08T08:55:33.961 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.452e-06 2024-05-08T08:55:33.961 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.961 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.961 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:33.961 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764661+0000", 2024-05-08T08:55:33.961 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5369999999999997e-06 2024-05-08T08:55:33.961 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.961 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.961 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:33.962 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764671+0000", 2024-05-08T08:55:33.962 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.028e-05 2024-05-08T08:55:33.962 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.962 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.962 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:33.962 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764680+0000", 2024-05-08T08:55:33.962 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0629999999999995e-06 2024-05-08T08:55:33.962 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.962 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.962 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:33.962 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764753+0000", 2024-05-08T08:55:33.962 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2679e-05 2024-05-08T08:55:33.962 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.962 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.962 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-05-08T08:55:33.963 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:49.764765+0000", 2024-05-08T08:55:33.963 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2415000000000001e-05 2024-05-08T08:55:33.963 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.963 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.963 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:33.963 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:50.071692+0000", 2024-05-08T08:55:33.963 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.306927324 2024-05-08T08:55:33.963 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.963 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.963 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:33.963 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:50.071796+0000", 2024-05-08T08:55:33.963 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104005 2024-05-08T08:55:33.963 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:33.963 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:33.963 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:33.963 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.964 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.964 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4416.0:20531 3.16 3:6bb2ed16:::rbd_object_map.1140bc4a55c7:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-05-08T08:55:33.964 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:58.673786+0000", 2024-05-08T08:55:33.964 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 95.282531574999993, 2024-05-08T08:55:33.964 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.94684119899999997, 2024-05-08T08:55:33.964 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:33.964 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:33.964 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:55:33.964 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:33.964 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4416", 2024-05-08T08:55:33.964 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:55:33.964 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20531 2024-05-08T08:55:33.964 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.964 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:33.964 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.964 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:33.965 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673786+0000", 2024-05-08T08:55:33.965 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:33.965 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.965 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.965 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:33.965 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673786+0000", 2024-05-08T08:55:33.965 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:33.965 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.965 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.965 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:33.965 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673787+0000", 2024-05-08T08:55:33.965 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9800000000000003e-07 2024-05-08T08:55:33.965 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.965 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.965 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:33.966 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673791+0000", 2024-05-08T08:55:33.966 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.104e-06 2024-05-08T08:55:33.966 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.966 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.966 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:33.966 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673801+0000", 2024-05-08T08:55:33.966 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9960000000000005e-06 2024-05-08T08:55:33.966 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.966 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.966 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:33.966 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673806+0000", 2024-05-08T08:55:33.966 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2320000000000001e-06 2024-05-08T08:55:33.966 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.966 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.966 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:33.967 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673860+0000", 2024-05-08T08:55:33.967 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3468999999999999e-05 2024-05-08T08:55:33.967 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.967 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.967 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-05-08T08:55:33.967 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:58.673866+0000", 2024-05-08T08:55:33.967 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3500000000000002e-06 2024-05-08T08:55:33.967 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.967 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.967 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:33.967 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.616700+0000", 2024-05-08T08:55:33.967 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.94283390700000003 2024-05-08T08:55:33.967 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.967 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.967 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:33.967 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.616941+0000", 2024-05-08T08:55:33.968 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00024080400000000001 2024-05-08T08:55:33.968 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.968 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.968 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-05-08T08:55:33.968 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.617072+0000", 2024-05-08T08:55:33.968 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000131688 2024-05-08T08:55:33.968 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.968 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.968 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:55:33.968 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.619571+0000", 2024-05-08T08:55:33.968 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0024990149999999998 2024-05-08T08:55:33.968 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.968 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.968 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:55:33.968 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.620568+0000", 2024-05-08T08:55:33.969 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00099711000000000005 2024-05-08T08:55:33.969 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.969 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.969 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:55:33.969 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.620602+0000", 2024-05-08T08:55:33.969 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.324e-05 2024-05-08T08:55:33.969 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.969 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.969 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:33.969 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:59.620627+0000", 2024-05-08T08:55:33.969 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5585999999999999e-05 2024-05-08T08:55:33.969 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:33.969 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:33.969 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:33.969 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.969 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.970 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5492 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4272128~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:33.970 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.881830+0000", 2024-05-08T08:55:33.970 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 68.074487769000001, 2024-05-08T08:55:33.970 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047614006, 2024-05-08T08:55:33.970 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:33.970 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:33.970 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:33.970 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:33.970 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:33.970 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:33.970 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5492 2024-05-08T08:55:33.970 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.970 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:33.970 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.970 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:33.970 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881830+0000", 2024-05-08T08:55:33.971 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:33.971 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.971 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.971 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:33.971 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881830+0000", 2024-05-08T08:55:33.971 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:33.971 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.971 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.971 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:33.971 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881830+0000", 2024-05-08T08:55:33.971 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4099999999999999e-07 2024-05-08T08:55:33.971 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.971 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.971 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:33.971 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881832+0000", 2024-05-08T08:55:33.971 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0370000000000001e-06 2024-05-08T08:55:33.972 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.972 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.972 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:33.972 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881839+0000", 2024-05-08T08:55:33.972 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3289999999999999e-06 2024-05-08T08:55:33.972 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.972 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.972 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:33.972 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881844+0000", 2024-05-08T08:55:33.972 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1170000000000003e-06 2024-05-08T08:55:33.972 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.972 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.972 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:33.972 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929296+0000", 2024-05-08T08:55:33.972 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047452418000000003 2024-05-08T08:55:33.973 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.973 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.973 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:33.973 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929360+0000", 2024-05-08T08:55:33.973 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4197999999999998e-05 2024-05-08T08:55:33.973 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.973 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.973 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:33.973 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929444+0000", 2024-05-08T08:55:33.973 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3466000000000004e-05 2024-05-08T08:55:33.973 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:33.973 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:33.973 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:33.973 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.973 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.973 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5494 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4317184~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:33.973 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.881850+0000", 2024-05-08T08:55:33.974 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 68.074468019999998, 2024-05-08T08:55:33.974 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047723887, 2024-05-08T08:55:33.974 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:33.974 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:33.974 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:33.974 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:33.974 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:33.974 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:33.974 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5494 2024-05-08T08:55:33.974 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.974 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:33.974 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.974 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:33.974 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881850+0000", 2024-05-08T08:55:33.974 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:33.974 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.975 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.975 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:33.975 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881850+0000", 2024-05-08T08:55:33.975 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:33.975 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.975 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.975 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:33.975 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881850+0000", 2024-05-08T08:55:33.975 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8500000000000002e-07 2024-05-08T08:55:33.975 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.975 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.975 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:33.975 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881856+0000", 2024-05-08T08:55:33.975 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9510000000000003e-06 2024-05-08T08:55:33.975 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.975 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.975 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:33.976 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881862+0000", 2024-05-08T08:55:33.976 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3049999999999998e-06 2024-05-08T08:55:33.976 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.976 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.976 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:33.976 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881868+0000", 2024-05-08T08:55:33.976 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8340000000000004e-06 2024-05-08T08:55:33.976 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.976 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.976 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:33.976 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929475+0000", 2024-05-08T08:55:33.976 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047606796 2024-05-08T08:55:33.976 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.976 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.976 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:33.977 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929518+0000", 2024-05-08T08:55:33.977 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3387000000000003e-05 2024-05-08T08:55:33.977 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.977 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.977 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:33.977 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929574+0000", 2024-05-08T08:55:33.977 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5129e-05 2024-05-08T08:55:33.977 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:33.977 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:33.977 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:33.977 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.977 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.977 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5495 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4362240~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:33.977 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.881875+0000", 2024-05-08T08:55:33.977 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 68.074443153000004, 2024-05-08T08:55:33.977 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047847967999999998, 2024-05-08T08:55:33.978 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:33.978 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:33.978 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:33.978 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:33.978 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:33.978 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:33.978 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5495 2024-05-08T08:55:33.978 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.978 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:33.978 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.978 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:33.978 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881875+0000", 2024-05-08T08:55:33.978 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:33.978 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.978 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.979 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:33.979 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881875+0000", 2024-05-08T08:55:33.979 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:33.979 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.979 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.979 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:33.979 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881875+0000", 2024-05-08T08:55:33.979 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-07 2024-05-08T08:55:33.979 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.979 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.979 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:33.979 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881877+0000", 2024-05-08T08:55:33.979 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9659999999999999e-06 2024-05-08T08:55:33.979 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.979 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.979 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:33.980 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881883+0000", 2024-05-08T08:55:33.980 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7649999999999999e-06 2024-05-08T08:55:33.980 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.980 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.980 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:33.980 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881887+0000", 2024-05-08T08:55:33.980 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6689999999999997e-06 2024-05-08T08:55:33.980 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.980 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.980 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:33.980 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929600+0000", 2024-05-08T08:55:33.980 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047712746 2024-05-08T08:55:33.980 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.980 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.980 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:33.980 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929659+0000", 2024-05-08T08:55:33.981 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8936e-05 2024-05-08T08:55:33.981 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.981 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.981 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:33.981 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929723+0000", 2024-05-08T08:55:33.981 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3524999999999999e-05 2024-05-08T08:55:33.981 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:33.981 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:33.981 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:33.981 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.981 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.981 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5496 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4407296~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:33.981 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.881894+0000", 2024-05-08T08:55:33.981 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 68.074423288999995, 2024-05-08T08:55:33.981 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047998015999999998, 2024-05-08T08:55:33.981 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:33.982 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:33.982 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:33.982 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:33.982 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:33.982 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:33.982 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5496 2024-05-08T08:55:33.982 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.982 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:33.982 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.982 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:33.982 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881894+0000", 2024-05-08T08:55:33.982 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:33.982 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.982 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.982 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:33.983 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881894+0000", 2024-05-08T08:55:33.983 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:33.983 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.983 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.983 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:33.983 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881895+0000", 2024-05-08T08:55:33.983 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6699999999999999e-07 2024-05-08T08:55:33.983 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.983 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.983 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:33.983 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881897+0000", 2024-05-08T08:55:33.983 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0229999999999999e-06 2024-05-08T08:55:33.983 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.983 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.983 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:33.984 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881903+0000", 2024-05-08T08:55:33.984 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9850000000000003e-06 2024-05-08T08:55:33.984 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.984 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.984 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:33.984 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881908+0000", 2024-05-08T08:55:33.984 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9019999999999998e-06 2024-05-08T08:55:33.984 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.984 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.984 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:33.984 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929751+0000", 2024-05-08T08:55:33.984 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047843550999999998 2024-05-08T08:55:33.984 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.984 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.984 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:33.985 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929823+0000", 2024-05-08T08:55:33.985 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1172000000000002e-05 2024-05-08T08:55:33.985 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.985 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.985 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:33.985 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929892+0000", 2024-05-08T08:55:33.985 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9915999999999999e-05 2024-05-08T08:55:33.985 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:33.985 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:33.985 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:33.985 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.985 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.985 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5498 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4452352~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:33.985 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.881914+0000", 2024-05-08T08:55:33.985 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 68.074403531000002, 2024-05-08T08:55:33.985 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048203553000000003, 2024-05-08T08:55:33.986 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:33.986 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:33.986 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:33.986 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:33.986 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:33.986 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:33.986 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5498 2024-05-08T08:55:33.986 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.986 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:33.986 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.986 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:33.986 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881914+0000", 2024-05-08T08:55:33.986 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:33.986 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.986 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.986 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:33.986 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881914+0000", 2024-05-08T08:55:33.987 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:33.987 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.987 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.987 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:33.987 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881915+0000", 2024-05-08T08:55:33.987 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8500000000000002e-07 2024-05-08T08:55:33.987 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.987 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.987 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:33.987 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881917+0000", 2024-05-08T08:55:33.987 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.181e-06 2024-05-08T08:55:33.987 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.987 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.987 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:33.987 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881924+0000", 2024-05-08T08:55:33.987 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9619999999999996e-06 2024-05-08T08:55:33.988 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.988 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.988 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:33.988 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.881929+0000", 2024-05-08T08:55:33.988 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1399999999999999e-06 2024-05-08T08:55:33.988 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.988 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.988 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:33.988 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.929952+0000", 2024-05-08T08:55:33.988 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048023134000000002 2024-05-08T08:55:33.988 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.988 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.988 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:33.988 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.930029+0000", 2024-05-08T08:55:33.988 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7490000000000005e-05 2024-05-08T08:55:33.988 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.989 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.989 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:33.989 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.930118+0000", 2024-05-08T08:55:33.989 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8261000000000001e-05 2024-05-08T08:55:33.989 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:33.989 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:33.989 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:33.989 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.989 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.989 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5564 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4366336~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:33.989 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.884225+0000", 2024-05-08T08:55:33.989 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 68.072092741000006, 2024-05-08T08:55:33.989 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047555989999999999, 2024-05-08T08:55:33.989 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:33.989 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:33.989 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:33.990 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:33.990 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:33.990 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:33.990 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5564 2024-05-08T08:55:33.990 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.990 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:33.990 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.990 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:33.990 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884225+0000", 2024-05-08T08:55:33.990 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:33.990 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.990 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.990 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:33.990 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884225+0000", 2024-05-08T08:55:33.990 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:33.990 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.991 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.991 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:33.991 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884225+0000", 2024-05-08T08:55:33.991 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2499999999999995e-07 2024-05-08T08:55:33.991 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.991 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.991 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:33.991 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884227+0000", 2024-05-08T08:55:33.991 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5829999999999999e-06 2024-05-08T08:55:33.991 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.991 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.991 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:33.991 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884232+0000", 2024-05-08T08:55:33.991 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1159999999999998e-06 2024-05-08T08:55:33.991 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.991 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.991 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:33.992 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884236+0000", 2024-05-08T08:55:33.992 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2490000000000003e-06 2024-05-08T08:55:33.992 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.992 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.992 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:33.992 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931617+0000", 2024-05-08T08:55:33.992 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047380310000000002 2024-05-08T08:55:33.992 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.992 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.992 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:33.992 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931689+0000", 2024-05-08T08:55:33.992 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2358000000000001e-05 2024-05-08T08:55:33.992 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.992 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.992 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:33.992 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931781+0000", 2024-05-08T08:55:33.993 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1848999999999994e-05 2024-05-08T08:55:33.993 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:33.993 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:33.993 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:33.993 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.993 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.993 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5565 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4411392~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:33.993 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.884251+0000", 2024-05-08T08:55:33.993 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 68.072066587999998, 2024-05-08T08:55:33.993 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047743671000000001, 2024-05-08T08:55:33.993 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:33.993 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:33.993 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:33.993 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:33.993 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:33.994 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:33.994 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5565 2024-05-08T08:55:33.994 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.994 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:33.994 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.994 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:33.994 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884251+0000", 2024-05-08T08:55:33.994 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:33.994 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.994 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.994 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:33.994 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884251+0000", 2024-05-08T08:55:33.994 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:33.994 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.994 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.994 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:33.995 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884252+0000", 2024-05-08T08:55:33.995 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3200000000000005e-07 2024-05-08T08:55:33.995 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.995 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.995 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:33.995 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884253+0000", 2024-05-08T08:55:33.995 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6980000000000001e-06 2024-05-08T08:55:33.995 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.995 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.995 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:33.995 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884259+0000", 2024-05-08T08:55:33.995 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3759999999999999e-06 2024-05-08T08:55:33.995 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.995 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.995 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:33.996 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884263+0000", 2024-05-08T08:55:33.996 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7280000000000003e-06 2024-05-08T08:55:33.996 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.996 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.996 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:33.996 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931818+0000", 2024-05-08T08:55:33.996 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047554368999999999 2024-05-08T08:55:33.996 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.996 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.996 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:33.996 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931897+0000", 2024-05-08T08:55:33.996 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8821999999999999e-05 2024-05-08T08:55:33.996 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.996 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.996 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:33.997 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.931995+0000", 2024-05-08T08:55:33.997 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8146000000000005e-05 2024-05-08T08:55:33.997 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:33.997 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:33.997 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:33.997 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.997 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.997 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5600 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 3948544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:33.997 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.884619+0000", 2024-05-08T08:55:33.997 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 68.071698333000001, 2024-05-08T08:55:33.997 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047574067999999997, 2024-05-08T08:55:33.997 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:33.997 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:33.997 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:33.997 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:33.997 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:33.998 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:33.998 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5600 2024-05-08T08:55:33.998 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.998 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:33.998 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.998 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:33.998 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884619+0000", 2024-05-08T08:55:33.998 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:33.998 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.998 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.998 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:33.998 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884619+0000", 2024-05-08T08:55:33.998 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:33.998 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.998 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.999 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:33.999 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884620+0000", 2024-05-08T08:55:33.999 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.53e-07 2024-05-08T08:55:33.999 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.999 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.999 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:33.999 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884622+0000", 2024-05-08T08:55:33.999 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.796e-06 2024-05-08T08:55:33.999 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.999 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:33.999 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:33.999 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884627+0000", 2024-05-08T08:55:33.999 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2850000000000002e-06 2024-05-08T08:55:33.999 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:33.999 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.000 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:34.000 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884631+0000", 2024-05-08T08:55:34.000 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2899999999999996e-06 2024-05-08T08:55:34.000 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.000 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.000 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:34.000 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932024+0000", 2024-05-08T08:55:34.000 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047392680999999999 2024-05-08T08:55:34.000 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.000 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.000 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:34.000 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932102+0000", 2024-05-08T08:55:34.000 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7806999999999996e-05 2024-05-08T08:55:34.000 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.000 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.000 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:34.000 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932193+0000", 2024-05-08T08:55:34.001 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1855999999999996e-05 2024-05-08T08:55:34.001 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.001 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:34.001 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.001 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.001 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.001 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5603 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 3993600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:34.001 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.884680+0000", 2024-05-08T08:55:34.001 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 68.071637723999999, 2024-05-08T08:55:34.001 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047745034999999998, 2024-05-08T08:55:34.001 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:34.001 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:34.001 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:34.001 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:34.001 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:34.001 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:34.001 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5603 2024-05-08T08:55:34.002 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.002 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:34.002 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.002 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:34.002 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884680+0000", 2024-05-08T08:55:34.002 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.002 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.002 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.002 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:34.002 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884680+0000", 2024-05-08T08:55:34.002 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.002 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.002 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.002 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:34.002 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884680+0000", 2024-05-08T08:55:34.003 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4400000000000001e-07 2024-05-08T08:55:34.003 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.003 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.003 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:34.003 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884682+0000", 2024-05-08T08:55:34.003 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.514e-06 2024-05-08T08:55:34.003 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.003 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.003 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:34.003 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884687+0000", 2024-05-08T08:55:34.003 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0359999999999997e-06 2024-05-08T08:55:34.003 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.003 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.003 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:34.003 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884692+0000", 2024-05-08T08:55:34.004 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5990000000000004e-06 2024-05-08T08:55:34.004 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.004 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.004 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:34.004 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932235+0000", 2024-05-08T08:55:34.004 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047543635000000001 2024-05-08T08:55:34.004 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.004 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.004 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:34.004 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932323+0000", 2024-05-08T08:55:34.004 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7311000000000005e-05 2024-05-08T08:55:34.004 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.004 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.004 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:34.004 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932425+0000", 2024-05-08T08:55:34.004 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102496 2024-05-08T08:55:34.005 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.005 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:34.005 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.005 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.005 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.005 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5606 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4038656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:34.005 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.884729+0000", 2024-05-08T08:55:34.005 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 68.071588435999999, 2024-05-08T08:55:34.005 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047916488, 2024-05-08T08:55:34.005 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:34.005 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:34.005 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:34.005 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:34.005 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:34.005 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:34.005 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5606 2024-05-08T08:55:34.006 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.006 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:34.006 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.006 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:34.006 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884729+0000", 2024-05-08T08:55:34.006 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.006 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.006 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.006 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:34.006 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.884729+0000", 2024-05-08T08:55:34.006 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.006 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.006 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.006 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:34.006 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.885940+0000", 2024-05-08T08:55:34.007 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001210509 2024-05-08T08:55:34.007 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.007 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.007 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:34.007 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.885945+0000", 2024-05-08T08:55:34.007 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7389999999999999e-06 2024-05-08T08:55:34.007 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.007 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.007 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:34.007 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.885955+0000", 2024-05-08T08:55:34.007 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0813e-05 2024-05-08T08:55:34.007 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.007 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.007 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:34.007 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.885962+0000", 2024-05-08T08:55:34.008 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1160000000000001e-06 2024-05-08T08:55:34.008 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.008 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.008 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:34.008 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932465+0000", 2024-05-08T08:55:34.008 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046502612999999998 2024-05-08T08:55:34.008 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.008 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.008 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:34.008 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932547+0000", 2024-05-08T08:55:34.008 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2025000000000002e-05 2024-05-08T08:55:34.008 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.008 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.008 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:34.008 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.932646+0000", 2024-05-08T08:55:34.009 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8672999999999995e-05 2024-05-08T08:55:34.009 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.009 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:34.009 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.009 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.009 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.009 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5636 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4280320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:34.009 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.886111+0000", 2024-05-08T08:55:34.009 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 68.070206698999996, 2024-05-08T08:55:34.009 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047663644999999998, 2024-05-08T08:55:34.009 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:34.009 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:34.009 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:34.009 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:34.009 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:34.009 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:34.010 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5636 2024-05-08T08:55:34.010 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.010 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:34.010 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.010 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:34.010 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886111+0000", 2024-05-08T08:55:34.010 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.010 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.010 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.010 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:34.010 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886111+0000", 2024-05-08T08:55:34.010 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.010 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.010 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.010 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:34.010 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886111+0000", 2024-05-08T08:55:34.011 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5200000000000002e-07 2024-05-08T08:55:34.011 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.011 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.011 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:34.011 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886114+0000", 2024-05-08T08:55:34.011 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1710000000000001e-06 2024-05-08T08:55:34.011 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.011 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.011 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:34.011 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886120+0000", 2024-05-08T08:55:34.011 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.404e-06 2024-05-08T08:55:34.011 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.011 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.011 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:34.011 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886125+0000", 2024-05-08T08:55:34.011 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2730000000000002e-06 2024-05-08T08:55:34.011 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.011 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.012 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:34.012 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933570+0000", 2024-05-08T08:55:34.012 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047444989999999999 2024-05-08T08:55:34.012 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.012 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.012 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:34.012 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933659+0000", 2024-05-08T08:55:34.012 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8912999999999994e-05 2024-05-08T08:55:34.012 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.012 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.012 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:34.012 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933775+0000", 2024-05-08T08:55:34.012 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000115442 2024-05-08T08:55:34.012 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.012 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:34.012 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.012 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.012 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.013 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5637 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4325376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:34.013 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.886132+0000", 2024-05-08T08:55:34.013 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 68.070185660999996, 2024-05-08T08:55:34.013 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047852219000000001, 2024-05-08T08:55:34.013 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:34.013 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:34.013 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:34.013 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:34.013 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:34.013 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:34.013 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5637 2024-05-08T08:55:34.013 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.013 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:34.013 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.013 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:34.013 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886132+0000", 2024-05-08T08:55:34.014 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.014 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.014 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.014 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:34.014 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886132+0000", 2024-05-08T08:55:34.014 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.014 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.014 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.014 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:34.014 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886132+0000", 2024-05-08T08:55:34.014 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7300000000000002e-07 2024-05-08T08:55:34.014 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.014 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.014 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:34.014 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886139+0000", 2024-05-08T08:55:34.015 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6109999999999999e-06 2024-05-08T08:55:34.015 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.015 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.015 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:34.015 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886146+0000", 2024-05-08T08:55:34.015 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6279999999999999e-06 2024-05-08T08:55:34.015 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.015 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.015 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:34.015 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886151+0000", 2024-05-08T08:55:34.015 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1769999999999997e-06 2024-05-08T08:55:34.015 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.015 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.015 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:34.015 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933812+0000", 2024-05-08T08:55:34.016 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047660912999999999 2024-05-08T08:55:34.016 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.016 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.016 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:34.016 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933888+0000", 2024-05-08T08:55:34.016 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5865000000000006e-05 2024-05-08T08:55:34.016 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.016 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.016 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:34.016 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.933984+0000", 2024-05-08T08:55:34.016 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6651999999999996e-05 2024-05-08T08:55:34.016 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.016 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:34.016 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.016 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.016 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.017 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5638 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4370432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:34.017 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.886157+0000", 2024-05-08T08:55:34.017 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 68.070160396999995, 2024-05-08T08:55:34.017 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048077597, 2024-05-08T08:55:34.017 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:34.017 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:34.017 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:34.017 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:34.017 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:34.017 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:34.017 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5638 2024-05-08T08:55:34.017 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.017 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:34.017 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.017 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:34.017 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886157+0000", 2024-05-08T08:55:34.017 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.018 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.018 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.018 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:34.018 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886157+0000", 2024-05-08T08:55:34.018 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.018 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.018 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.018 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:34.018 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886158+0000", 2024-05-08T08:55:34.018 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1100000000000001e-07 2024-05-08T08:55:34.018 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.018 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.018 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:34.018 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886160+0000", 2024-05-08T08:55:34.018 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9800000000000001e-06 2024-05-08T08:55:34.019 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.019 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.019 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:34.019 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886166+0000", 2024-05-08T08:55:34.019 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.314e-06 2024-05-08T08:55:34.019 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.019 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.019 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:34.019 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886171+0000", 2024-05-08T08:55:34.019 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7700000000000001e-06 2024-05-08T08:55:34.019 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.019 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.019 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:34.019 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.934025+0000", 2024-05-08T08:55:34.019 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047854509000000003 2024-05-08T08:55:34.019 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.020 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.020 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:34.020 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.934121+0000", 2024-05-08T08:55:34.020 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5688999999999998e-05 2024-05-08T08:55:34.020 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.020 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.020 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:34.020 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.934235+0000", 2024-05-08T08:55:34.020 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000113924 2024-05-08T08:55:34.020 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.020 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:34.020 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.020 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.020 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.020 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.6222.0:5639 3.16 3:69266fdf:::rbd_data.1140ab20a0d.0000000000000000:31 [read 4415488~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-05-08T08:55:34.020 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:25.886177+0000", 2024-05-08T08:55:34.020 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 68.070140647000002, 2024-05-08T08:55:34.021 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048249541, 2024-05-08T08:55:34.021 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:34.021 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:34.021 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:34.021 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:34.021 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.6222", 2024-05-08T08:55:34.021 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/567570609", 2024-05-08T08:55:34.021 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5639 2024-05-08T08:55:34.021 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.021 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:34.021 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.021 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:34.021 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886177+0000", 2024-05-08T08:55:34.021 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.021 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.021 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.021 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:34.022 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886177+0000", 2024-05-08T08:55:34.022 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.022 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.022 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.022 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:34.022 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886177+0000", 2024-05-08T08:55:34.022 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.65e-07 2024-05-08T08:55:34.022 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.022 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.022 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:34.022 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886179+0000", 2024-05-08T08:55:34.022 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9530000000000002e-06 2024-05-08T08:55:34.022 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.022 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.022 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:34.022 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886186+0000", 2024-05-08T08:55:34.023 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3419999999999996e-06 2024-05-08T08:55:34.023 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.023 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.023 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:34.023 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.886192+0000", 2024-05-08T08:55:34.023 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0560000000000001e-06 2024-05-08T08:55:34.023 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.023 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.023 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:34.023 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.934258+0000", 2024-05-08T08:55:34.023 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048066367999999998 2024-05-08T08:55:34.023 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.023 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.023 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:34.023 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.934338+0000", 2024-05-08T08:55:34.023 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9616000000000005e-05 2024-05-08T08:55:34.023 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.024 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.024 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:34.024 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:25.934427+0000", 2024-05-08T08:55:34.024 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8840999999999999e-05 2024-05-08T08:55:34.024 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.024 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:34.024 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.024 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.024 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.024 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4416.0:34114 3.12 3:4e31a7e4:::rbd_data.1140100c9703.0000000000000008:head [delete] snapc 14=[] ondisk+write+known_if_redirected+supports_pool_eio e164)", 2024-05-08T08:55:34.024 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:55:15.450741+0000", 2024-05-08T08:55:34.024 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.505576902000001, 2024-05-08T08:55:34.024 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.92638831499999996, 2024-05-08T08:55:34.024 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:34.024 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:34.025 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:55:34.025 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:34.025 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4416", 2024-05-08T08:55:34.025 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:55:34.025 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 34114 2024-05-08T08:55:34.025 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.025 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:34.025 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.025 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:34.025 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:15.450741+0000", 2024-05-08T08:55:34.025 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.025 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.025 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.025 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:34.025 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:15.450741+0000", 2024-05-08T08:55:34.025 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.025 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.026 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.026 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:34.026 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:15.450741+0000", 2024-05-08T08:55:34.026 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9900000000000001e-07 2024-05-08T08:55:34.026 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.026 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.026 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:34.026 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:15.450744+0000", 2024-05-08T08:55:34.026 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3259999999999998e-06 2024-05-08T08:55:34.026 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.026 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.026 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:34.026 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:15.450751+0000", 2024-05-08T08:55:34.026 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6129999999999998e-06 2024-05-08T08:55:34.026 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.026 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.027 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:34.027 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:15.450757+0000", 2024-05-08T08:55:34.027 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3129999999999998e-06 2024-05-08T08:55:34.027 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.027 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.027 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:34.027 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:15.450825+0000", 2024-05-08T08:55:34.027 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8307999999999997e-05 2024-05-08T08:55:34.027 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.027 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.027 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-05-08T08:55:34.027 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:15.450840+0000", 2024-05-08T08:55:34.027 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4968e-05 2024-05-08T08:55:34.027 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.027 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.028 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:34.028 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:16.374193+0000", 2024-05-08T08:55:34.028 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.92335363000000004 2024-05-08T08:55:34.028 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.028 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.028 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:34.028 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:16.374298+0000", 2024-05-08T08:55:34.028 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104759 2024-05-08T08:55:34.028 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.028 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.028 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-05-08T08:55:34.028 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:16.374372+0000", 2024-05-08T08:55:34.028 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4232000000000003e-05 2024-05-08T08:55:34.028 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.028 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.028 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:55:34.029 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:16.376496+0000", 2024-05-08T08:55:34.029 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0021231259999999999 2024-05-08T08:55:34.029 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.029 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.029 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:55:34.029 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:16.377065+0000", 2024-05-08T08:55:34.029 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00056950700000000002 2024-05-08T08:55:34.029 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.029 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.029 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:55:34.029 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:16.377100+0000", 2024-05-08T08:55:34.029 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4579999999999998e-05 2024-05-08T08:55:34.029 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.029 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.029 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:34.029 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:16.377129+0000", 2024-05-08T08:55:34.030 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9454e-05 2024-05-08T08:55:34.030 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.030 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:34.030 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.030 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.030 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.030 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4416.0:34124 3.12 3:4f2dce52:::rbd_data.1140100c9703.0000000000000015:head [delete] snapc 14=[] ondisk+write+known_if_redirected+supports_pool_eio e164)", 2024-05-08T08:55:34.030 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:55:15.458266+0000", 2024-05-08T08:55:34.030 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.49805155, 2024-05-08T08:55:34.030 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.925342255, 2024-05-08T08:55:34.030 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:34.030 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:34.030 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:55:34.030 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:34.030 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4416", 2024-05-08T08:55:34.031 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:55:34.031 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 34124 2024-05-08T08:55:34.031 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.031 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:34.031 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.031 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:34.031 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:15.458266+0000", 2024-05-08T08:55:34.031 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.031 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.031 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.031 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:34.031 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:15.458266+0000", 2024-05-08T08:55:34.031 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.031 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.031 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.031 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:34.032 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:15.458267+0000", 2024-05-08T08:55:34.032 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9900000000000001e-07 2024-05-08T08:55:34.032 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.032 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.032 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:34.032 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:15.458269+0000", 2024-05-08T08:55:34.032 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6520000000000002e-06 2024-05-08T08:55:34.032 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.032 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.032 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:34.032 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:15.458276+0000", 2024-05-08T08:55:34.032 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1779999999999997e-06 2024-05-08T08:55:34.032 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.032 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.032 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:34.033 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:15.458283+0000", 2024-05-08T08:55:34.033 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7680000000000003e-06 2024-05-08T08:55:34.033 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.033 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.033 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:34.033 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:15.458330+0000", 2024-05-08T08:55:34.033 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7111999999999999e-05 2024-05-08T08:55:34.033 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.033 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.033 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-05-08T08:55:34.033 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:15.458340+0000", 2024-05-08T08:55:34.033 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5680000000000005e-06 2024-05-08T08:55:34.033 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.033 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.033 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:34.033 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:16.375925+0000", 2024-05-08T08:55:34.033 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.91758470599999997 2024-05-08T08:55:34.034 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.034 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.034 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:34.034 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:16.376000+0000", 2024-05-08T08:55:34.034 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5325000000000001e-05 2024-05-08T08:55:34.034 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.034 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.034 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-05-08T08:55:34.034 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:16.376062+0000", 2024-05-08T08:55:34.034 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1630000000000005e-05 2024-05-08T08:55:34.034 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.034 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.034 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:55:34.034 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:16.380141+0000", 2024-05-08T08:55:34.034 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0040797530000000002 2024-05-08T08:55:34.034 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.035 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.035 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:55:34.035 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:16.383551+0000", 2024-05-08T08:55:34.035 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0034098319999999998 2024-05-08T08:55:34.035 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.035 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.035 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:55:34.035 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:16.383583+0000", 2024-05-08T08:55:34.035 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2140000000000001e-05 2024-05-08T08:55:34.035 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.035 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.035 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:34.035 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:16.383608+0000", 2024-05-08T08:55:34.035 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5092e-05 2024-05-08T08:55:34.035 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.035 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:34.035 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.036 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.036 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.036 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4416.0:34130 3.12 3:4fed2d06:::rbd_data.1140100c9703.0000000000000031:head [delete] snapc 14=[] ondisk+write+known_if_redirected+supports_pool_eio e164)", 2024-05-08T08:55:34.036 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:55:15.460999+0000", 2024-05-08T08:55:34.036 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.495318487999999, 2024-05-08T08:55:34.036 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.92270473099999994, 2024-05-08T08:55:34.036 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:34.036 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:34.036 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:55:34.036 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:34.036 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4416", 2024-05-08T08:55:34.036 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:55:34.036 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 34130 2024-05-08T08:55:34.036 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.036 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:34.036 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.037 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:34.037 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:15.460999+0000", 2024-05-08T08:55:34.037 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.037 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.037 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.037 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:34.037 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:15.460999+0000", 2024-05-08T08:55:34.037 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.037 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.037 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.037 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:34.037 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:15.461000+0000", 2024-05-08T08:55:34.037 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1999999999999999e-07 2024-05-08T08:55:34.037 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.037 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.037 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:34.037 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:15.461004+0000", 2024-05-08T08:55:34.038 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1620000000000001e-06 2024-05-08T08:55:34.038 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.038 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.038 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:34.038 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:15.461016+0000", 2024-05-08T08:55:34.038 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2269999999999999e-05 2024-05-08T08:55:34.038 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.038 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.038 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:34.038 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:15.461024+0000", 2024-05-08T08:55:34.038 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6450000000000005e-06 2024-05-08T08:55:34.038 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.038 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.038 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:34.038 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:15.461077+0000", 2024-05-08T08:55:34.038 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3242000000000003e-05 2024-05-08T08:55:34.038 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.039 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.039 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-05-08T08:55:34.039 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:15.461088+0000", 2024-05-08T08:55:34.039 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0808999999999999e-05 2024-05-08T08:55:34.039 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.039 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.039 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:34.039 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:16.376527+0000", 2024-05-08T08:55:34.039 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.91543937200000003 2024-05-08T08:55:34.039 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.039 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.039 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:34.039 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:16.376599+0000", 2024-05-08T08:55:34.039 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1298999999999996e-05 2024-05-08T08:55:34.039 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.039 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.040 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-05-08T08:55:34.040 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:16.376657+0000", 2024-05-08T08:55:34.040 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8418000000000002e-05 2024-05-08T08:55:34.040 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.040 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.040 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:55:34.040 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:16.380144+0000", 2024-05-08T08:55:34.040 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003487066 2024-05-08T08:55:34.040 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.040 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.040 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:55:34.040 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:16.383639+0000", 2024-05-08T08:55:34.040 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0034950570000000002 2024-05-08T08:55:34.040 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.040 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.041 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:55:34.041 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:16.383670+0000", 2024-05-08T08:55:34.041 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1065999999999997e-05 2024-05-08T08:55:34.041 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.041 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.041 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:34.041 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:55:16.383704+0000", 2024-05-08T08:55:34.041 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3704999999999997e-05 2024-05-08T08:55:34.041 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.041 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:34.041 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.041 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.041 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:34.041 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:55:34.043 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:55:34.148 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:55:34.148 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:55:34.148 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:55:34.148 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:55:34.157 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:55:34.261 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:55:34.262 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:55:34.262 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:55:34.262 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:55:34.262 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:55:34.270 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:55:34.377 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:55:34.377 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:55:34.377 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:55:34.377 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:55:34.377 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.377 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22073 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 782336~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:34.377 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:55:34.377 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 186.52097844599999, 2024-05-08T08:55:34.377 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047244234000000003, 2024-05-08T08:55:34.377 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:34.377 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:34.377 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:34.378 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:34.378 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:34.378 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:34.378 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22073 2024-05-08T08:55:34.378 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.378 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:34.378 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.378 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:34.378 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:55:34.378 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.378 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.378 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.378 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:34.378 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857667+0000", 2024-05-08T08:55:34.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.379 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.379 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.379 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:34.379 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857668+0000", 2024-05-08T08:55:34.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7800000000000002e-07 2024-05-08T08:55:34.379 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.379 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.379 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:34.379 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857670+0000", 2024-05-08T08:55:34.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0820000000000001e-06 2024-05-08T08:55:34.379 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.379 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.379 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:34.379 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857678+0000", 2024-05-08T08:55:34.380 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7589999999999997e-06 2024-05-08T08:55:34.380 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.380 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.380 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:34.380 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857691+0000", 2024-05-08T08:55:34.380 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3672999999999999e-05 2024-05-08T08:55:34.380 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.380 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.380 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:34.380 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904780+0000", 2024-05-08T08:55:34.380 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047088616999999999 2024-05-08T08:55:34.380 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.380 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.380 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:34.380 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904840+0000", 2024-05-08T08:55:34.380 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0356999999999998e-05 2024-05-08T08:55:34.381 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.381 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.381 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:34.381 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904912+0000", 2024-05-08T08:55:34.381 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1367999999999998e-05 2024-05-08T08:55:34.381 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.381 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:34.381 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.381 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.381 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.381 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22075 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 798720~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:34.381 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:55:34.381 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 186.520948049, 2024-05-08T08:55:34.381 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047384643999999997, 2024-05-08T08:55:34.382 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:34.382 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:34.382 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:34.382 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:34.382 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:34.382 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:34.382 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22075 2024-05-08T08:55:34.382 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.382 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:34.382 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.382 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:34.382 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:55:34.382 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.382 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.383 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.383 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:34.383 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:55:34.383 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.383 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.383 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.383 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:34.383 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857698+0000", 2024-05-08T08:55:34.383 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7399999999999999e-07 2024-05-08T08:55:34.383 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.383 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.383 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:34.383 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857705+0000", 2024-05-08T08:55:34.383 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1980000000000004e-06 2024-05-08T08:55:34.383 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.384 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:34.384 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857713+0000", 2024-05-08T08:55:34.384 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7039999999999994e-06 2024-05-08T08:55:34.384 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.384 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:34.384 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857718+0000", 2024-05-08T08:55:34.384 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.865e-06 2024-05-08T08:55:34.384 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.384 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:34.384 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904941+0000", 2024-05-08T08:55:34.384 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047223198000000001 2024-05-08T08:55:34.384 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.385 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.385 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:34.385 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.904999+0000", 2024-05-08T08:55:34.385 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8254999999999997e-05 2024-05-08T08:55:34.385 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.385 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.385 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:34.385 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905082+0000", 2024-05-08T08:55:34.385 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3049999999999999e-05 2024-05-08T08:55:34.385 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.385 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:34.385 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.385 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.385 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.385 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22077 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 815104~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:34.386 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:55:34.386 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 186.52091003800001, 2024-05-08T08:55:34.386 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047519448999999998, 2024-05-08T08:55:34.386 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:34.386 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:34.386 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:34.386 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:34.386 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:34.386 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:34.386 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22077 2024-05-08T08:55:34.386 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.386 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:34.386 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.386 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:34.386 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:55:34.387 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.387 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.387 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.387 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:34.387 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:55:34.387 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.387 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.387 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.387 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:34.387 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857736+0000", 2024-05-08T08:55:34.387 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.99e-07 2024-05-08T08:55:34.387 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.387 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.387 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:34.387 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857739+0000", 2024-05-08T08:55:34.388 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3599999999999999e-06 2024-05-08T08:55:34.388 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.388 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.388 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:34.388 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857745+0000", 2024-05-08T08:55:34.388 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5139999999999998e-06 2024-05-08T08:55:34.388 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.388 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.388 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:34.388 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857750+0000", 2024-05-08T08:55:34.388 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8550000000000001e-06 2024-05-08T08:55:34.388 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.388 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.388 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:34.388 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905110+0000", 2024-05-08T08:55:34.388 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047360126000000002 2024-05-08T08:55:34.388 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.389 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.389 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:34.389 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905175+0000", 2024-05-08T08:55:34.389 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4689000000000004e-05 2024-05-08T08:55:34.389 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.389 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.389 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:34.389 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905255+0000", 2024-05-08T08:55:34.389 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0306e-05 2024-05-08T08:55:34.389 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.389 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:34.389 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.389 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.389 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.389 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22079 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 831488~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:34.389 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:55:34.390 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 186.52088949, 2024-05-08T08:55:34.390 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.04767499, 2024-05-08T08:55:34.390 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:34.390 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:34.390 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:34.390 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:34.390 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:34.390 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:34.390 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22079 2024-05-08T08:55:34.390 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.390 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:34.390 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.390 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:34.390 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:55:34.391 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.391 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.391 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.391 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:34.391 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857756+0000", 2024-05-08T08:55:34.391 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.391 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.391 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.391 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:34.391 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857757+0000", 2024-05-08T08:55:34.391 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8299999999999997e-07 2024-05-08T08:55:34.391 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.391 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.391 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:34.391 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857759+0000", 2024-05-08T08:55:34.391 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1390000000000002e-06 2024-05-08T08:55:34.391 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.392 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.392 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:34.392 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857765+0000", 2024-05-08T08:55:34.392 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4609999999999996e-06 2024-05-08T08:55:34.392 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.392 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.392 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:34.392 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857770+0000", 2024-05-08T08:55:34.392 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6689999999999997e-06 2024-05-08T08:55:34.392 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.392 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.392 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:34.392 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905281+0000", 2024-05-08T08:55:34.392 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047510668999999998 2024-05-08T08:55:34.393 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.393 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.393 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:34.393 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905344+0000", 2024-05-08T08:55:34.393 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2797000000000002e-05 2024-05-08T08:55:34.393 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.393 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.393 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:34.393 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905431+0000", 2024-05-08T08:55:34.393 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7771999999999999e-05 2024-05-08T08:55:34.393 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.393 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:34.393 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.393 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.393 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.394 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22082 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 847872~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:34.394 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:55:34.394 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 186.520869799, 2024-05-08T08:55:34.394 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047831934999999999, 2024-05-08T08:55:34.394 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:34.394 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:34.394 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:34.394 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:34.394 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:34.394 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:34.394 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22082 2024-05-08T08:55:34.394 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.394 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:34.394 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.394 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:34.395 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:55:34.395 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.395 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.395 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.395 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:34.395 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:55:34.395 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.395 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.395 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.395 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:34.395 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857776+0000", 2024-05-08T08:55:34.395 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:55:34.395 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.395 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.395 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:34.396 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857778+0000", 2024-05-08T08:55:34.396 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0789999999999999e-06 2024-05-08T08:55:34.396 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.396 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.396 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:34.396 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857784+0000", 2024-05-08T08:55:34.396 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0480000000000004e-06 2024-05-08T08:55:34.396 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.396 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.396 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:34.396 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857790+0000", 2024-05-08T08:55:34.396 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2719999999999997e-06 2024-05-08T08:55:34.396 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.396 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.396 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:34.396 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905458+0000", 2024-05-08T08:55:34.397 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047668470999999997 2024-05-08T08:55:34.397 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.397 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.397 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:34.397 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905521+0000", 2024-05-08T08:55:34.397 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2952000000000003e-05 2024-05-08T08:55:34.397 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.397 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.397 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:34.397 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905608+0000", 2024-05-08T08:55:34.397 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6759000000000001e-05 2024-05-08T08:55:34.397 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.397 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:34.397 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.397 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.398 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.398 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22084 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 864256~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:34.398 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:55:34.398 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 186.52084584900001, 2024-05-08T08:55:34.398 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047977748000000001, 2024-05-08T08:55:34.398 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:34.398 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:34.398 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:34.398 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:34.398 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:34.398 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:34.398 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22084 2024-05-08T08:55:34.398 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.398 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:34.399 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.399 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:34.399 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:55:34.399 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.399 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.399 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.399 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:34.399 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:55:34.399 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.399 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.399 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.399 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:34.399 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857800+0000", 2024-05-08T08:55:34.399 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:55:34.399 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.400 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.400 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:34.400 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857802+0000", 2024-05-08T08:55:34.400 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0779999999999998e-06 2024-05-08T08:55:34.400 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.400 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.400 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:34.400 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857812+0000", 2024-05-08T08:55:34.400 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7939999999999997e-06 2024-05-08T08:55:34.400 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.400 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.400 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:34.400 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857817+0000", 2024-05-08T08:55:34.400 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8899999999999998e-06 2024-05-08T08:55:34.400 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.401 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.401 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:34.401 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905636+0000", 2024-05-08T08:55:34.401 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047819252 2024-05-08T08:55:34.401 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.401 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.401 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:34.401 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905700+0000", 2024-05-08T08:55:34.401 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4016999999999994e-05 2024-05-08T08:55:34.401 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.401 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.401 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:34.401 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905778+0000", 2024-05-08T08:55:34.401 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7306999999999997e-05 2024-05-08T08:55:34.401 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.402 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:34.402 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.402 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.402 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.402 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22086 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 880640~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:34.402 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:55:34.402 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 186.52082249899999, 2024-05-08T08:55:34.402 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048119149, 2024-05-08T08:55:34.402 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:34.402 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:34.402 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:34.402 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:34.402 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:34.402 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:34.403 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22086 2024-05-08T08:55:34.403 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.403 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:34.403 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.403 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:34.403 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:55:34.403 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.403 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.403 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.403 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:34.403 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857823+0000", 2024-05-08T08:55:34.403 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.403 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.403 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.403 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:34.404 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857824+0000", 2024-05-08T08:55:34.404 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4200000000000002e-07 2024-05-08T08:55:34.404 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.404 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.404 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:34.404 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857826+0000", 2024-05-08T08:55:34.404 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0590000000000001e-06 2024-05-08T08:55:34.404 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.404 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.404 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:34.404 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857832+0000", 2024-05-08T08:55:34.404 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8029999999999999e-06 2024-05-08T08:55:34.404 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.404 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.404 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:34.405 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857837+0000", 2024-05-08T08:55:34.405 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7929999999999997e-06 2024-05-08T08:55:34.405 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.405 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.405 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:34.405 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905803+0000", 2024-05-08T08:55:34.405 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047965761000000003 2024-05-08T08:55:34.405 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.405 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.405 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:34.405 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905862+0000", 2024-05-08T08:55:34.405 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9203000000000002e-05 2024-05-08T08:55:34.405 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.405 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.405 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:34.406 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905942+0000", 2024-05-08T08:55:34.406 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0187999999999999e-05 2024-05-08T08:55:34.406 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.406 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:34.406 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.406 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.406 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.406 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22088 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 897024~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:34.406 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:55:34.406 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 186.52080244699999, 2024-05-08T08:55:34.406 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048256049000000002, 2024-05-08T08:55:34.406 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:34.406 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:34.406 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:34.406 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:34.407 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:34.407 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:34.407 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22088 2024-05-08T08:55:34.407 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.407 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:34.407 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.407 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:34.407 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:55:34.407 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.407 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.407 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.407 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:34.407 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857843+0000", 2024-05-08T08:55:34.407 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.407 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.407 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.408 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:34.408 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857844+0000", 2024-05-08T08:55:34.408 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.46e-07 2024-05-08T08:55:34.408 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.408 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.408 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:34.408 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857846+0000", 2024-05-08T08:55:34.408 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0899999999999999e-06 2024-05-08T08:55:34.408 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.408 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.408 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:34.408 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857855+0000", 2024-05-08T08:55:34.408 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8219999999999996e-06 2024-05-08T08:55:34.408 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.408 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.409 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:34.409 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857860+0000", 2024-05-08T08:55:34.409 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2989999999999996e-06 2024-05-08T08:55:34.409 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.409 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.409 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:34.409 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.905966+0000", 2024-05-08T08:55:34.409 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048106542000000002 2024-05-08T08:55:34.409 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.409 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.409 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:34.409 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906024+0000", 2024-05-08T08:55:34.409 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7958000000000003e-05 2024-05-08T08:55:34.409 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.409 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.409 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:34.410 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906099+0000", 2024-05-08T08:55:34.410 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4992000000000003e-05 2024-05-08T08:55:34.410 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.410 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:34.410 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.410 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.410 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.410 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22091 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 913408~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:34.410 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:55:34.410 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 186.52077868699999, 2024-05-08T08:55:34.410 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048395290000000001, 2024-05-08T08:55:34.410 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:34.410 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:34.410 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:34.410 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:34.411 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:34.411 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:34.411 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22091 2024-05-08T08:55:34.411 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.411 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:34.411 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.411 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:34.411 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:55:34.411 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.411 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.411 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.411 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:34.411 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:55:34.411 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.411 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.411 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.412 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:34.412 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857867+0000", 2024-05-08T08:55:34.412 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4900000000000001e-07 2024-05-08T08:55:34.412 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.412 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.412 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:34.412 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857870+0000", 2024-05-08T08:55:34.412 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2510000000000001e-06 2024-05-08T08:55:34.412 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.412 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.412 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:34.412 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857877+0000", 2024-05-08T08:55:34.412 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0809999999999996e-06 2024-05-08T08:55:34.412 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.412 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.413 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:34.413 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857881+0000", 2024-05-08T08:55:34.413 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.724e-06 2024-05-08T08:55:34.413 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.413 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.413 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:34.413 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906127+0000", 2024-05-08T08:55:34.413 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048245345000000002 2024-05-08T08:55:34.413 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.413 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.413 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:34.413 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906190+0000", 2024-05-08T08:55:34.413 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2762000000000007e-05 2024-05-08T08:55:34.413 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.413 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.413 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:34.414 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906262+0000", 2024-05-08T08:55:34.414 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2778e-05 2024-05-08T08:55:34.414 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.414 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:34.414 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.414 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.414 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.414 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22093 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 929792~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:34.414 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:55:34.414 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 186.520756323, 2024-05-08T08:55:34.414 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048532875000000003, 2024-05-08T08:55:34.414 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:34.414 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:34.414 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:34.415 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:34.415 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:34.415 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:34.415 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22093 2024-05-08T08:55:34.415 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.415 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:34.415 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.415 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:34.415 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:55:34.415 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.415 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.415 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.415 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:34.415 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857889+0000", 2024-05-08T08:55:34.415 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.416 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.416 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.416 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:34.416 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857890+0000", 2024-05-08T08:55:34.416 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-07 2024-05-08T08:55:34.416 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.416 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.416 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:34.416 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857899+0000", 2024-05-08T08:55:34.416 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6889999999999999e-06 2024-05-08T08:55:34.416 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.416 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.416 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:34.416 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857906+0000", 2024-05-08T08:55:34.416 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6490000000000002e-06 2024-05-08T08:55:34.416 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.417 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.417 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:34.417 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857917+0000", 2024-05-08T08:55:34.417 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0764999999999999e-05 2024-05-08T08:55:34.417 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.417 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.417 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:34.417 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906287+0000", 2024-05-08T08:55:34.417 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048370442 2024-05-08T08:55:34.417 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.417 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.417 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:34.417 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906349+0000", 2024-05-08T08:55:34.417 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1537000000000004e-05 2024-05-08T08:55:34.418 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.418 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.418 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:34.418 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906422+0000", 2024-05-08T08:55:34.418 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3431999999999998e-05 2024-05-08T08:55:34.418 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.418 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:34.418 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.418 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.418 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.418 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22095 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 946176~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:34.418 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:55:34.418 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 186.520721619, 2024-05-08T08:55:34.418 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048657593999999998, 2024-05-08T08:55:34.418 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:34.419 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:34.419 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:34.419 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:34.419 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:34.419 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:34.419 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22095 2024-05-08T08:55:34.419 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.419 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:34.419 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.419 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:34.419 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:55:34.419 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.419 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.419 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.419 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:34.420 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857924+0000", 2024-05-08T08:55:34.420 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.420 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.420 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.420 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:34.420 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857925+0000", 2024-05-08T08:55:34.420 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3099999999999998e-07 2024-05-08T08:55:34.420 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.420 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.420 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:34.420 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857927+0000", 2024-05-08T08:55:34.420 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.34e-06 2024-05-08T08:55:34.420 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.420 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.420 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:34.420 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857938+0000", 2024-05-08T08:55:34.421 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1426e-05 2024-05-08T08:55:34.421 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.421 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.421 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:34.421 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857943+0000", 2024-05-08T08:55:34.421 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5140000000000003e-06 2024-05-08T08:55:34.421 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.421 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.421 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:34.421 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906448+0000", 2024-05-08T08:55:34.421 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048504641000000001 2024-05-08T08:55:34.421 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.421 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.421 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:34.421 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906510+0000", 2024-05-08T08:55:34.422 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2663000000000006e-05 2024-05-08T08:55:34.422 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.422 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.422 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:34.422 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906582+0000", 2024-05-08T08:55:34.422 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1478999999999998e-05 2024-05-08T08:55:34.422 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.422 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:34.422 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.422 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.422 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.422 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22097 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 962560~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:34.422 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:55:34.422 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 186.52069392999999, 2024-05-08T08:55:34.422 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048791353000000003, 2024-05-08T08:55:34.423 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:34.423 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:34.423 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:34.423 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:34.423 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:34.423 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:34.423 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22097 2024-05-08T08:55:34.423 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.423 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:34.423 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.423 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:34.423 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:55:34.423 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.423 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.423 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.424 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:34.424 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:55:34.424 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.424 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.424 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.424 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:34.424 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857952+0000", 2024-05-08T08:55:34.424 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9200000000000002e-07 2024-05-08T08:55:34.424 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.424 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.424 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:34.424 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857954+0000", 2024-05-08T08:55:34.424 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1900000000000002e-06 2024-05-08T08:55:34.424 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.425 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.425 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:34.425 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857961+0000", 2024-05-08T08:55:34.425 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4989999999999999e-06 2024-05-08T08:55:34.425 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.425 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.425 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:34.425 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857971+0000", 2024-05-08T08:55:34.425 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0485999999999999e-05 2024-05-08T08:55:34.425 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.425 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.425 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:34.425 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906607+0000", 2024-05-08T08:55:34.425 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048635109000000003 2024-05-08T08:55:34.425 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.425 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.426 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:34.426 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906667+0000", 2024-05-08T08:55:34.426 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0439000000000002e-05 2024-05-08T08:55:34.426 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.426 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.426 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:34.426 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906743+0000", 2024-05-08T08:55:34.426 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6237999999999998e-05 2024-05-08T08:55:34.426 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.426 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:34.426 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.426 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.426 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.426 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22099 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 978944~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:34.426 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:55:34.427 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 186.52066776999999, 2024-05-08T08:55:34.427 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048932055000000002, 2024-05-08T08:55:34.427 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:34.427 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:34.427 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:34.427 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:34.427 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:34.427 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:34.427 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22099 2024-05-08T08:55:34.427 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.427 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:34.427 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.427 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:34.427 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:55:34.427 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.428 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.428 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.428 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:34.428 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:55:34.428 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.428 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.428 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.428 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:34.428 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857978+0000", 2024-05-08T08:55:34.428 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.27e-07 2024-05-08T08:55:34.428 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.428 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.428 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:34.428 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857987+0000", 2024-05-08T08:55:34.428 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5890000000000003e-06 2024-05-08T08:55:34.429 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.429 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.429 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:34.429 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857994+0000", 2024-05-08T08:55:34.429 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6619999999999999e-06 2024-05-08T08:55:34.429 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.429 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.429 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:34.429 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.857998+0000", 2024-05-08T08:55:34.429 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7110000000000003e-06 2024-05-08T08:55:34.429 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.429 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.429 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:34.429 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906772+0000", 2024-05-08T08:55:34.429 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048773415000000001 2024-05-08T08:55:34.429 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.430 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.430 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:34.430 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906834+0000", 2024-05-08T08:55:34.430 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1752000000000001e-05 2024-05-08T08:55:34.430 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.430 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.430 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:34.430 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906910+0000", 2024-05-08T08:55:34.430 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6499000000000001e-05 2024-05-08T08:55:34.430 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.430 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:34.430 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.430 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.430 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.431 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22101 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 995328~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:34.431 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:55:34.431 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 186.52064063899999, 2024-05-08T08:55:34.431 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049087364000000001, 2024-05-08T08:55:34.431 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:34.431 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:34.431 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:34.431 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:34.431 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:34.431 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:34.431 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22101 2024-05-08T08:55:34.431 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.431 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:34.431 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.431 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:34.431 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:55:34.432 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.432 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.432 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.432 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:34.432 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858005+0000", 2024-05-08T08:55:34.432 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.432 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.432 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.432 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:34.432 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858006+0000", 2024-05-08T08:55:34.432 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1399999999999997e-07 2024-05-08T08:55:34.432 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.432 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.432 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:34.432 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858008+0000", 2024-05-08T08:55:34.433 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2060000000000001e-06 2024-05-08T08:55:34.433 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.433 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.433 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:34.433 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858017+0000", 2024-05-08T08:55:34.433 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1579999999999994e-06 2024-05-08T08:55:34.433 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.433 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.433 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:34.433 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858022+0000", 2024-05-08T08:55:34.433 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4199999999999998e-06 2024-05-08T08:55:34.433 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.433 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.433 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:34.433 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.906940+0000", 2024-05-08T08:55:34.433 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048917627999999998 2024-05-08T08:55:34.434 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.434 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.434 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:34.434 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907002+0000", 2024-05-08T08:55:34.434 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2399999999999999e-05 2024-05-08T08:55:34.434 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.434 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.434 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:34.434 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907092+0000", 2024-05-08T08:55:34.434 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0137999999999997e-05 2024-05-08T08:55:34.434 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.434 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:34.434 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.434 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.434 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.435 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22103 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1011712~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:34.435 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:55:34.435 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 186.520615854, 2024-05-08T08:55:34.435 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049228542, 2024-05-08T08:55:34.435 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:34.435 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:34.435 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:34.435 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:34.435 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:34.435 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:34.435 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22103 2024-05-08T08:55:34.435 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.435 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:34.435 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.435 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:34.436 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:55:34.436 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.436 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.436 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.436 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:34.436 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:55:34.436 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.436 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.436 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.436 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:34.436 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858030+0000", 2024-05-08T08:55:34.436 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4299999999999999e-07 2024-05-08T08:55:34.436 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.436 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.436 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:34.437 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858032+0000", 2024-05-08T08:55:34.437 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0590000000000001e-06 2024-05-08T08:55:34.437 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.437 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.437 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:34.437 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858037+0000", 2024-05-08T08:55:34.437 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.977e-06 2024-05-08T08:55:34.437 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.437 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.437 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:34.437 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858042+0000", 2024-05-08T08:55:34.437 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7650000000000001e-06 2024-05-08T08:55:34.437 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.437 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.437 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:34.437 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907118+0000", 2024-05-08T08:55:34.438 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049076147000000001 2024-05-08T08:55:34.438 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.438 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.438 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:34.438 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907180+0000", 2024-05-08T08:55:34.438 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1971999999999996e-05 2024-05-08T08:55:34.438 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.438 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.438 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:34.438 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907258+0000", 2024-05-08T08:55:34.438 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8279e-05 2024-05-08T08:55:34.438 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.438 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:34.438 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.438 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.439 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.439 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22105 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1028096~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:34.439 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:55:34.439 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 186.520596611, 2024-05-08T08:55:34.439 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049394609999999999, 2024-05-08T08:55:34.439 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:34.439 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:34.439 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:34.439 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:34.439 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:34.439 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:34.439 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22105 2024-05-08T08:55:34.439 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.439 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:34.439 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.439 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:34.440 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:55:34.440 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.440 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.440 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.440 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:34.440 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858049+0000", 2024-05-08T08:55:34.440 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.440 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.440 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.440 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:34.440 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858050+0000", 2024-05-08T08:55:34.440 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5999999999999999e-07 2024-05-08T08:55:34.440 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.440 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.440 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:34.440 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858051+0000", 2024-05-08T08:55:34.441 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8870000000000001e-06 2024-05-08T08:55:34.441 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.441 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.441 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:34.441 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858057+0000", 2024-05-08T08:55:34.441 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8019999999999997e-06 2024-05-08T08:55:34.441 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.441 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.441 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:34.441 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858062+0000", 2024-05-08T08:55:34.441 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.561e-06 2024-05-08T08:55:34.441 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.441 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.441 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:34.441 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907289+0000", 2024-05-08T08:55:34.442 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049226957000000002 2024-05-08T08:55:34.442 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.442 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.442 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:34.442 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907356+0000", 2024-05-08T08:55:34.442 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6915000000000006e-05 2024-05-08T08:55:34.442 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.442 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.442 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:34.442 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907444+0000", 2024-05-08T08:55:34.442 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8127999999999994e-05 2024-05-08T08:55:34.442 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.442 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:34.442 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.443 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.443 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.443 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22107 3.1c 3:3dd3c560:::rbd_data.1140782590a5.0000000000000013:6 [read 1044480~2048] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:34.443 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:55:34.443 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 186.52057827100001, 2024-05-08T08:55:34.443 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049799942999999999, 2024-05-08T08:55:34.443 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:34.443 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:34.443 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:34.443 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:34.443 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:34.443 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:34.443 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22107 2024-05-08T08:55:34.443 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.443 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:34.443 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.444 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:34.444 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:55:34.444 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.444 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.444 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.444 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:34.444 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:55:34.444 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.444 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.444 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.444 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:34.444 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858068+0000", 2024-05-08T08:55:34.444 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4799999999999999e-07 2024-05-08T08:55:34.444 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.444 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.445 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:34.445 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858070+0000", 2024-05-08T08:55:34.445 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.012e-06 2024-05-08T08:55:34.445 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.445 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.445 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:34.445 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858077+0000", 2024-05-08T08:55:34.445 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6290000000000001e-06 2024-05-08T08:55:34.445 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.445 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.445 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:34.445 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858083+0000", 2024-05-08T08:55:34.445 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8880000000000002e-06 2024-05-08T08:55:34.445 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.445 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.446 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:34.446 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907671+0000", 2024-05-08T08:55:34.446 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049587191000000003 2024-05-08T08:55:34.446 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.446 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.446 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:34.446 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907763+0000", 2024-05-08T08:55:34.446 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2231000000000005e-05 2024-05-08T08:55:34.446 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.446 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.446 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:34.446 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907867+0000", 2024-05-08T08:55:34.446 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104644 2024-05-08T08:55:34.446 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.446 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:34.447 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.447 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.447 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.447 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4576.0:22294 3.1c 3:3aad0389:::rbd_data.11406886b07b.0000000000000080:head [stat,delete] snapc 7=[7] RETRY=6 ondisk+retry+write+known_if_redirected+supports_pool_eio e31)", 2024-05-08T08:55:34.447 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:55:34.447 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 186.51998442300001, 2024-05-08T08:55:34.447 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051152191, 2024-05-08T08:55:34.447 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:34.447 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:34.447 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:34.447 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:34.447 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4576", 2024-05-08T08:55:34.447 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2902849796", 2024-05-08T08:55:34.447 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22294 2024-05-08T08:55:34.447 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.447 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:34.448 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.448 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:34.448 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:55:34.448 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.448 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.448 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.448 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:34.448 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858661+0000", 2024-05-08T08:55:34.448 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.448 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.448 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.448 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:34.448 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858662+0000", 2024-05-08T08:55:34.448 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4799999999999998e-07 2024-05-08T08:55:34.448 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.449 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.449 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:34.449 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858664+0000", 2024-05-08T08:55:34.449 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1229999999999998e-06 2024-05-08T08:55:34.449 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.449 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.449 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:34.449 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858670+0000", 2024-05-08T08:55:34.449 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3099999999999997e-06 2024-05-08T08:55:34.449 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.449 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.449 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:34.449 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.858676+0000", 2024-05-08T08:55:34.449 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9750000000000004e-06 2024-05-08T08:55:34.449 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.449 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.450 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:34.450 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.907901+0000", 2024-05-08T08:55:34.450 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049224520000000001 2024-05-08T08:55:34.450 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.450 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.450 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:34.450 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.908085+0000", 2024-05-08T08:55:34.450 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00018377500000000001 2024-05-08T08:55:34.450 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.450 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.450 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:34.450 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:27.909814+0000", 2024-05-08T08:55:34.450 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00172894 2024-05-08T08:55:34.450 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.450 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:34.451 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.451 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.451 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.451 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4416.0:13461 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-05-08T08:55:34.451 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:53:22.487456+0000", 2024-05-08T08:55:34.451 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 131.89118938199999, 2024-05-08T08:55:34.451 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.99102620799999996, 2024-05-08T08:55:34.451 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:34.451 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:34.451 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:34.451 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:34.451 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4416", 2024-05-08T08:55:34.451 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/2416533825", 2024-05-08T08:55:34.451 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 13461 2024-05-08T08:55:34.451 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.452 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:34.452 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.452 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:34.452 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487456+0000", 2024-05-08T08:55:34.452 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.452 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.452 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.452 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:34.452 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487456+0000", 2024-05-08T08:55:34.452 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.452 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.452 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.452 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:34.452 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487458+0000", 2024-05-08T08:55:34.452 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5659999999999999e-06 2024-05-08T08:55:34.453 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.453 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.453 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:34.453 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487463+0000", 2024-05-08T08:55:34.453 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6890000000000003e-06 2024-05-08T08:55:34.453 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.453 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.453 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:34.453 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487472+0000", 2024-05-08T08:55:34.453 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7710000000000001e-06 2024-05-08T08:55:34.453 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.453 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.453 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:34.453 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487478+0000", 2024-05-08T08:55:34.453 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1390000000000003e-06 2024-05-08T08:55:34.454 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.454 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.454 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:34.454 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487520+0000", 2024-05-08T08:55:34.454 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2259999999999999e-05 2024-05-08T08:55:34.454 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.454 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.454 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-05-08T08:55:34.454 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:22.487528+0000", 2024-05-08T08:55:34.454 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7479999999999993e-06 2024-05-08T08:55:34.454 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.454 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.454 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:34.454 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:23.478139+0000", 2024-05-08T08:55:34.455 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.99061165200000001 2024-05-08T08:55:34.455 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.455 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.455 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:34.455 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:23.478301+0000", 2024-05-08T08:55:34.455 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000161602 2024-05-08T08:55:34.455 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.455 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.455 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:34.455 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:53:23.478483+0000", 2024-05-08T08:55:34.455 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000181781 2024-05-08T08:55:34.455 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.455 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:34.455 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.455 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.456 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.456 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.5752.0:404 3.1a 3:5f5e8104:::rbd_data.1140bc4a55c7.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 5175296~45056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-05-08T08:55:34.456 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:54:00.390863+0000", 2024-05-08T08:55:34.456 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 93.987783187000005, 2024-05-08T08:55:34.456 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.055189174000000001, 2024-05-08T08:55:34.456 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:34.456 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:34.456 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:55:34.456 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:34.456 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.5752", 2024-05-08T08:55:34.456 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/3534419815", 2024-05-08T08:55:34.456 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 404 2024-05-08T08:55:34.456 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.456 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:34.456 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.457 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:34.457 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.390863+0000", 2024-05-08T08:55:34.457 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.457 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.457 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.457 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:34.457 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.390863+0000", 2024-05-08T08:55:34.457 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.457 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.457 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.457 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:34.457 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.390864+0000", 2024-05-08T08:55:34.457 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.471e-06 2024-05-08T08:55:34.457 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.457 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.458 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:34.458 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397475+0000", 2024-05-08T08:55:34.458 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0066109109999999997 2024-05-08T08:55:34.458 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.458 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.458 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:34.458 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397498+0000", 2024-05-08T08:55:34.458 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2674e-05 2024-05-08T08:55:34.458 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.458 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.458 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:34.458 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397508+0000", 2024-05-08T08:55:34.458 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9019999999999994e-06 2024-05-08T08:55:34.458 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.459 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.459 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:34.459 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397609+0000", 2024-05-08T08:55:34.459 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010121899999999999 2024-05-08T08:55:34.459 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.459 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.459 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:34.459 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397711+0000", 2024-05-08T08:55:34.459 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102489 2024-05-08T08:55:34.459 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.459 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.459 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-05-08T08:55:34.459 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.397999+0000", 2024-05-08T08:55:34.459 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00028733799999999999 2024-05-08T08:55:34.459 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.460 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.460 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-05-08T08:55:34.460 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.432093+0000", 2024-05-08T08:55:34.460 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.034094865000000002 2024-05-08T08:55:34.460 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.460 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.460 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:55:34.460 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.445975+0000", 2024-05-08T08:55:34.460 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013881187 2024-05-08T08:55:34.460 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.460 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.460 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-05-08T08:55:34.460 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.446013+0000", 2024-05-08T08:55:34.460 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8155000000000003e-05 2024-05-08T08:55:34.460 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.460 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.461 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:34.461 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:54:00.446052+0000", 2024-05-08T08:55:34.461 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8962999999999999e-05 2024-05-08T08:55:34.461 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.461 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:34.461 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.461 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.461 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:34.461 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:55:34.462 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:55:34.568 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:55:34.569 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:55:34.569 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-05-08T08:55:34.569 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:55:34.577 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:55:34.683 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:55:34.683 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-05-08T08:55:34.683 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-05-08T08:55:34.683 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-05-08T08:55:34.683 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:55:34.692 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:55:34.799 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T08:55:34.799 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-05-08T08:55:34.799 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-05-08T08:55:34.799 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-05-08T08:55:34.799 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.800 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6690 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1093632~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:34.800 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:55:34.800 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 183.08565553299999, 2024-05-08T08:55:34.800 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052462754, 2024-05-08T08:55:34.800 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:34.800 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:34.800 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:34.800 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:34.800 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:34.800 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:34.800 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6690 2024-05-08T08:55:34.800 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.800 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:34.800 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.801 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:34.801 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:55:34.801 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.801 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.801 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.801 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:34.801 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715345+0000", 2024-05-08T08:55:34.801 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.801 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.801 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.801 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:34.801 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715346+0000", 2024-05-08T08:55:34.801 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.84e-07 2024-05-08T08:55:34.801 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.801 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.802 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:34.802 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715348+0000", 2024-05-08T08:55:34.802 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.114e-06 2024-05-08T08:55:34.802 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.802 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.802 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:34.802 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715353+0000", 2024-05-08T08:55:34.802 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5720000000000002e-06 2024-05-08T08:55:34.802 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.802 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.802 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:34.802 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715358+0000", 2024-05-08T08:55:34.802 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7140000000000001e-06 2024-05-08T08:55:34.802 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.803 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.803 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:34.803 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767638+0000", 2024-05-08T08:55:34.803 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052279562000000002 2024-05-08T08:55:34.803 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.803 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.803 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:34.803 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767713+0000", 2024-05-08T08:55:34.803 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5652e-05 2024-05-08T08:55:34.803 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.803 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.803 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:34.803 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767808+0000", 2024-05-08T08:55:34.803 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4755999999999999e-05 2024-05-08T08:55:34.803 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.804 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:34.804 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.804 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.804 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.804 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6692 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1110016~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:34.804 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:55:34.804 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 183.085636443, 2024-05-08T08:55:34.804 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052666495000000001, 2024-05-08T08:55:34.804 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:34.804 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:34.804 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:34.804 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:34.804 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:34.804 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:34.804 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6692 2024-05-08T08:55:34.804 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.805 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:34.805 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.805 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:34.805 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:55:34.805 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.805 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.805 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.805 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:34.805 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:55:34.805 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.805 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.805 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.805 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:34.805 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715365+0000", 2024-05-08T08:55:34.805 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8599999999999999e-07 2024-05-08T08:55:34.805 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.806 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.806 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:34.806 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715372+0000", 2024-05-08T08:55:34.806 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0990000000000001e-06 2024-05-08T08:55:34.806 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.806 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.806 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:34.806 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715378+0000", 2024-05-08T08:55:34.806 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0680000000000002e-06 2024-05-08T08:55:34.806 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.806 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.806 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:34.806 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715383+0000", 2024-05-08T08:55:34.806 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8740000000000003e-06 2024-05-08T08:55:34.806 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.807 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.807 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:34.807 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767848+0000", 2024-05-08T08:55:34.807 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052464668999999998 2024-05-08T08:55:34.807 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.807 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.807 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:34.807 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.767927+0000", 2024-05-08T08:55:34.807 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9228999999999996e-05 2024-05-08T08:55:34.807 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.807 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.807 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:34.807 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768031+0000", 2024-05-08T08:55:34.807 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010417 2024-05-08T08:55:34.807 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.808 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:34.808 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.808 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.808 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.808 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6694 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1126400~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:34.808 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:55:34.808 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 183.085611546, 2024-05-08T08:55:34.808 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052848829, 2024-05-08T08:55:34.808 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:34.808 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:34.808 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:34.808 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:34.808 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:34.808 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:34.809 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6694 2024-05-08T08:55:34.809 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.809 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:34.809 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.809 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:34.809 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:55:34.809 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.809 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.809 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.809 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:34.809 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715389+0000", 2024-05-08T08:55:34.809 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.809 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.809 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.809 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:34.809 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715390+0000", 2024-05-08T08:55:34.810 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9900000000000001e-07 2024-05-08T08:55:34.810 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.810 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.810 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:34.810 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715392+0000", 2024-05-08T08:55:34.810 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8279999999999999e-06 2024-05-08T08:55:34.810 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.810 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.810 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:34.810 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715398+0000", 2024-05-08T08:55:34.810 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7619999999999998e-06 2024-05-08T08:55:34.810 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.810 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.810 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:34.811 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715403+0000", 2024-05-08T08:55:34.811 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7790000000000003e-06 2024-05-08T08:55:34.811 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.811 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.811 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:34.811 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768067+0000", 2024-05-08T08:55:34.811 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052663785999999997 2024-05-08T08:55:34.811 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.811 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.811 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:34.811 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768142+0000", 2024-05-08T08:55:34.811 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5018999999999994e-05 2024-05-08T08:55:34.811 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.811 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.811 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:34.811 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768238+0000", 2024-05-08T08:55:34.812 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6255999999999995e-05 2024-05-08T08:55:34.812 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.812 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:34.812 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.812 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.812 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.812 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6696 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1142784~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:34.812 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:55:34.812 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 183.08559203799999, 2024-05-08T08:55:34.812 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.053051057999999998, 2024-05-08T08:55:34.812 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:34.812 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:34.812 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:34.812 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:34.812 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:34.812 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:34.813 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6696 2024-05-08T08:55:34.813 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.813 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:34.813 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.813 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:34.813 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:55:34.813 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.813 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.813 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.813 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:34.813 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:55:34.813 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.813 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.813 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.813 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:34.813 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715409+0000", 2024-05-08T08:55:34.814 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-05-08T08:55:34.814 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.814 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.814 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:34.814 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715411+0000", 2024-05-08T08:55:34.814 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9630000000000001e-06 2024-05-08T08:55:34.814 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.814 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.814 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:34.814 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715417+0000", 2024-05-08T08:55:34.814 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7250000000000002e-06 2024-05-08T08:55:34.814 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.814 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.814 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:34.814 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715421+0000", 2024-05-08T08:55:34.815 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4089999999999996e-06 2024-05-08T08:55:34.815 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.815 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.815 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:34.815 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768277+0000", 2024-05-08T08:55:34.815 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052855293999999997 2024-05-08T08:55:34.815 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.815 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.815 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:34.815 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768358+0000", 2024-05-08T08:55:34.815 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1720999999999999e-05 2024-05-08T08:55:34.815 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.815 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.815 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:34.815 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768460+0000", 2024-05-08T08:55:34.815 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010154600000000001 2024-05-08T08:55:34.815 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.816 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:34.816 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.816 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.816 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.816 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6698 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1159168~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:34.816 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:55:34.816 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 183.08557317099999, 2024-05-08T08:55:34.816 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.053245726, 2024-05-08T08:55:34.816 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:34.816 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:34.816 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:34.816 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:34.816 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:34.816 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:34.816 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6698 2024-05-08T08:55:34.816 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.817 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:34.817 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.817 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:34.817 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:55:34.817 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.817 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.817 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.817 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:34.817 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.715428+0000", 2024-05-08T08:55:34.817 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.817 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.817 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.817 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:34.817 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716936+0000", 2024-05-08T08:55:34.817 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001508665 2024-05-08T08:55:34.818 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.818 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.818 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:34.818 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716941+0000", 2024-05-08T08:55:34.818 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1910000000000002e-06 2024-05-08T08:55:34.818 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.818 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.818 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:34.818 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716950+0000", 2024-05-08T08:55:34.818 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1900000000000001e-06 2024-05-08T08:55:34.818 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.818 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.818 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:34.818 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.716958+0000", 2024-05-08T08:55:34.818 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8019999999999992e-06 2024-05-08T08:55:34.819 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.819 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.819 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:34.819 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768499+0000", 2024-05-08T08:55:34.819 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051540926000000001 2024-05-08T08:55:34.819 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.819 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.819 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:34.819 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768574+0000", 2024-05-08T08:55:34.819 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4988999999999996e-05 2024-05-08T08:55:34.819 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.819 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.819 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:34.819 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.768673+0000", 2024-05-08T08:55:34.819 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9963000000000005e-05 2024-05-08T08:55:34.819 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.820 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:34.820 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.820 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.820 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.820 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6706 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1224704~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:34.820 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:55:34.820 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 183.08396786399999, 2024-05-08T08:55:34.820 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052435598, 2024-05-08T08:55:34.820 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:34.820 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:34.820 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:34.820 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:34.820 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:34.820 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:34.820 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6706 2024-05-08T08:55:34.820 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.820 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:34.821 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.821 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:34.821 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:55:34.821 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.821 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.821 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.821 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:34.821 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:55:34.821 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.821 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.821 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.821 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:34.821 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717033+0000", 2024-05-08T08:55:34.821 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:55:34.821 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.821 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.822 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:34.822 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717035+0000", 2024-05-08T08:55:34.822 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9290000000000001e-06 2024-05-08T08:55:34.822 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.822 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.822 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:34.822 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717041+0000", 2024-05-08T08:55:34.822 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6720000000000001e-06 2024-05-08T08:55:34.822 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.822 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.822 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:34.822 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717046+0000", 2024-05-08T08:55:34.822 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5650000000000003e-06 2024-05-08T08:55:34.822 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.822 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.823 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:34.823 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769311+0000", 2024-05-08T08:55:34.823 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052265645999999999 2024-05-08T08:55:34.823 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.823 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.823 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:34.823 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769379+0000", 2024-05-08T08:55:34.823 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7391000000000006e-05 2024-05-08T08:55:34.823 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.823 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.823 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:34.823 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769469+0000", 2024-05-08T08:55:34.823 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0025999999999995e-05 2024-05-08T08:55:34.823 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.823 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:34.823 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.824 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.824 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.824 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6708 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1241088~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:34.824 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:55:34.824 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 183.08394976, 2024-05-08T08:55:34.824 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052640552, 2024-05-08T08:55:34.824 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:34.824 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:34.824 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:34.824 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:34.824 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:34.824 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:34.824 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6708 2024-05-08T08:55:34.824 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.824 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:34.825 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.825 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:34.825 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:55:34.825 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.825 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.825 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.825 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:34.825 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717051+0000", 2024-05-08T08:55:34.825 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.825 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.825 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.825 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:34.825 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717052+0000", 2024-05-08T08:55:34.825 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:55:34.825 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.825 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.826 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:34.826 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717053+0000", 2024-05-08T08:55:34.826 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8330000000000001e-06 2024-05-08T08:55:34.826 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.826 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.826 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:34.826 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717058+0000", 2024-05-08T08:55:34.826 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.933e-06 2024-05-08T08:55:34.826 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.826 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.826 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:34.826 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717063+0000", 2024-05-08T08:55:34.826 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2180000000000001e-06 2024-05-08T08:55:34.826 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.827 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.827 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:34.827 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769507+0000", 2024-05-08T08:55:34.827 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052444866999999999 2024-05-08T08:55:34.827 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.827 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.827 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:34.827 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769589+0000", 2024-05-08T08:55:34.827 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2039000000000005e-05 2024-05-08T08:55:34.827 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.827 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.827 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:34.827 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769692+0000", 2024-05-08T08:55:34.827 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102252 2024-05-08T08:55:34.827 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.828 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:34.828 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.828 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.828 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.828 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6710 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1257472~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:34.828 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:55:34.828 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 183.08393173600001, 2024-05-08T08:55:34.828 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.05283057, 2024-05-08T08:55:34.828 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:34.828 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:34.828 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:34.828 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:34.828 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:34.828 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:34.828 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6710 2024-05-08T08:55:34.828 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.829 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:34.829 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.829 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:34.829 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:55:34.829 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.829 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.829 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.829 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:34.829 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.717069+0000", 2024-05-08T08:55:34.829 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.829 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.829 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.829 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:34.829 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718951+0000", 2024-05-08T08:55:34.829 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0018817809999999999 2024-05-08T08:55:34.829 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.830 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.830 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:34.830 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718956+0000", 2024-05-08T08:55:34.830 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5399999999999997e-06 2024-05-08T08:55:34.830 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.830 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.830 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:34.830 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718964+0000", 2024-05-08T08:55:34.830 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6629999999999999e-06 2024-05-08T08:55:34.830 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.830 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.830 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:34.830 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.718972+0000", 2024-05-08T08:55:34.830 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5229999999999998e-06 2024-05-08T08:55:34.830 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.830 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.831 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:34.831 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769729+0000", 2024-05-08T08:55:34.831 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050756959999999997 2024-05-08T08:55:34.831 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.831 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.831 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:34.831 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769803+0000", 2024-05-08T08:55:34.831 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4538999999999999e-05 2024-05-08T08:55:34.831 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.831 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.831 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:34.831 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.769900+0000", 2024-05-08T08:55:34.831 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6564000000000006e-05 2024-05-08T08:55:34.831 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.831 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:34.831 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.832 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.832 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.832 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6726 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1388544~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:34.832 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:55:34.832 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 183.081856964, 2024-05-08T08:55:34.832 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052451506000000002, 2024-05-08T08:55:34.832 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:34.832 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:34.832 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:34.832 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:34.832 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:34.832 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:34.832 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6726 2024-05-08T08:55:34.832 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.832 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:34.832 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.833 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:34.833 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:55:34.833 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.833 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.833 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.833 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:34.833 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719144+0000", 2024-05-08T08:55:34.833 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.833 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.833 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.833 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:34.833 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719200+0000", 2024-05-08T08:55:34.833 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5865999999999997e-05 2024-05-08T08:55:34.833 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.833 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.833 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:34.834 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719204+0000", 2024-05-08T08:55:34.834 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8630000000000001e-06 2024-05-08T08:55:34.834 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.834 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.834 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:34.834 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719215+0000", 2024-05-08T08:55:34.834 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1590000000000001e-05 2024-05-08T08:55:34.834 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.834 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.834 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:34.834 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719220+0000", 2024-05-08T08:55:34.834 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1619999999999999e-06 2024-05-08T08:55:34.834 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.834 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.834 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:34.835 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771425+0000", 2024-05-08T08:55:34.835 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052204249000000001 2024-05-08T08:55:34.835 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.835 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.835 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:34.835 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771498+0000", 2024-05-08T08:55:34.835 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3782999999999995e-05 2024-05-08T08:55:34.835 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.835 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.835 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:34.835 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771595+0000", 2024-05-08T08:55:34.835 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6992999999999998e-05 2024-05-08T08:55:34.835 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.835 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:34.835 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.835 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.836 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.836 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6728 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1404928~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:34.836 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:55:34.836 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 183.08176955900001, 2024-05-08T08:55:34.836 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052590747, 2024-05-08T08:55:34.836 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:34.836 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:34.836 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:34.836 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:34.836 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:34.836 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:34.836 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6728 2024-05-08T08:55:34.836 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.836 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:34.836 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.836 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:34.837 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:55:34.837 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.837 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.837 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.837 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:34.837 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719231+0000", 2024-05-08T08:55:34.837 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.837 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.837 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.837 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:34.837 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719232+0000", 2024-05-08T08:55:34.837 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4000000000000001e-07 2024-05-08T08:55:34.837 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.837 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.837 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:34.838 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719234+0000", 2024-05-08T08:55:34.838 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4499999999999998e-06 2024-05-08T08:55:34.838 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.838 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.838 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:34.838 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719240+0000", 2024-05-08T08:55:34.838 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5720000000000002e-06 2024-05-08T08:55:34.838 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.838 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.838 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:34.838 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719245+0000", 2024-05-08T08:55:34.838 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8690000000000003e-06 2024-05-08T08:55:34.838 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.838 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.839 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:34.839 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771636+0000", 2024-05-08T08:55:34.839 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052390979999999997 2024-05-08T08:55:34.839 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.839 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.839 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:34.839 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771717+0000", 2024-05-08T08:55:34.839 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1291999999999993e-05 2024-05-08T08:55:34.839 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.839 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.839 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:34.839 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771822+0000", 2024-05-08T08:55:34.839 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000104944 2024-05-08T08:55:34.839 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.839 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:34.840 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.840 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.840 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.840 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6730 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1421312~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:34.840 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:55:34.840 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 183.081749317, 2024-05-08T08:55:34.840 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052770477000000003, 2024-05-08T08:55:34.840 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:34.840 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:34.840 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:34.840 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:34.840 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:34.840 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:34.840 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6730 2024-05-08T08:55:34.840 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.840 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:34.841 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.841 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:34.841 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:55:34.841 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.841 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.841 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.841 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:34.841 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:55:34.841 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.841 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.841 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.841 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:34.841 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719252+0000", 2024-05-08T08:55:34.841 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9000000000000002e-07 2024-05-08T08:55:34.841 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.841 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.842 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:34.842 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719254+0000", 2024-05-08T08:55:34.842 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4099999999999998e-06 2024-05-08T08:55:34.842 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.842 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.842 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:34.842 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719263+0000", 2024-05-08T08:55:34.842 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4109999999999996e-06 2024-05-08T08:55:34.842 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.842 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.842 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:34.842 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719268+0000", 2024-05-08T08:55:34.842 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2669999999999997e-06 2024-05-08T08:55:34.842 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.842 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.843 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:34.843 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771846+0000", 2024-05-08T08:55:34.843 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052577677000000003 2024-05-08T08:55:34.843 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.843 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.843 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:34.843 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.771920+0000", 2024-05-08T08:55:34.843 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4460000000000002e-05 2024-05-08T08:55:34.843 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.843 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.843 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:34.843 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772022+0000", 2024-05-08T08:55:34.843 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010186199999999999 2024-05-08T08:55:34.843 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.843 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:34.844 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.844 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.844 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.844 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6732 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1437696~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:34.844 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:55:34.844 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 183.08172450999999, 2024-05-08T08:55:34.844 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052905575000000003, 2024-05-08T08:55:34.844 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:34.844 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:34.844 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:34.844 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:34.844 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:34.844 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:34.844 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6732 2024-05-08T08:55:34.844 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.844 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:34.844 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.845 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:34.845 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:55:34.845 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.845 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.845 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.845 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:34.845 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.719276+0000", 2024-05-08T08:55:34.845 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.845 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.845 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.845 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:34.845 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721392+0000", 2024-05-08T08:55:34.845 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002115287 2024-05-08T08:55:34.845 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.845 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.845 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:34.846 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721396+0000", 2024-05-08T08:55:34.846 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1679999999999997e-06 2024-05-08T08:55:34.846 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.846 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.846 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:34.846 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721404+0000", 2024-05-08T08:55:34.846 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4409999999999993e-06 2024-05-08T08:55:34.846 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.846 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.846 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:34.846 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721410+0000", 2024-05-08T08:55:34.846 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9150000000000001e-06 2024-05-08T08:55:34.846 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.846 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.846 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:34.847 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772061+0000", 2024-05-08T08:55:34.847 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050651068 2024-05-08T08:55:34.847 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.847 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.847 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:34.847 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772127+0000", 2024-05-08T08:55:34.847 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5424000000000002e-05 2024-05-08T08:55:34.847 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.847 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.847 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:34.847 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.772182+0000", 2024-05-08T08:55:34.847 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5272000000000002e-05 2024-05-08T08:55:34.847 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.847 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:34.847 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.847 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.847 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.848 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6749 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1585152~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:34.848 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:55:34.848 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 183.07942374000001, 2024-05-08T08:55:34.848 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052350264000000001, 2024-05-08T08:55:34.848 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:34.848 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:34.848 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:34.848 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:34.848 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:34.848 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:34.848 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6749 2024-05-08T08:55:34.848 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.848 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:34.848 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.848 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:34.849 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:55:34.849 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.849 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.849 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.849 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:34.849 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721577+0000", 2024-05-08T08:55:34.849 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.849 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.849 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.849 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:34.849 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721578+0000", 2024-05-08T08:55:34.849 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0399999999999996e-07 2024-05-08T08:55:34.849 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.849 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.849 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:34.850 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721580+0000", 2024-05-08T08:55:34.850 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5450000000000002e-06 2024-05-08T08:55:34.850 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.850 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.850 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:34.850 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721587+0000", 2024-05-08T08:55:34.850 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0190000000000001e-06 2024-05-08T08:55:34.850 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.850 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.850 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:34.850 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721593+0000", 2024-05-08T08:55:34.850 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5659999999999997e-06 2024-05-08T08:55:34.850 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.850 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.850 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:34.851 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773759+0000", 2024-05-08T08:55:34.851 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052166553999999997 2024-05-08T08:55:34.851 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.851 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.851 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:34.851 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773833+0000", 2024-05-08T08:55:34.851 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3913999999999997e-05 2024-05-08T08:55:34.851 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.851 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.851 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:34.851 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773927+0000", 2024-05-08T08:55:34.851 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4061999999999995e-05 2024-05-08T08:55:34.851 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.851 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:34.851 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.852 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.852 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.852 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6751 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1601536~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:34.852 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:55:34.852 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 183.07940046100001, 2024-05-08T08:55:34.852 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052461302000000001, 2024-05-08T08:55:34.852 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:34.852 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:34.852 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:34.852 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:34.852 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:34.852 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:34.852 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6751 2024-05-08T08:55:34.852 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.852 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:34.852 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.853 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:34.853 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:55:34.853 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.853 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.853 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.853 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:34.853 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.721600+0000", 2024-05-08T08:55:34.853 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.853 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.853 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.853 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:34.853 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722939+0000", 2024-05-08T08:55:34.853 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001338601 2024-05-08T08:55:34.853 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.853 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.854 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:34.854 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722944+0000", 2024-05-08T08:55:34.854 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4469999999999999e-06 2024-05-08T08:55:34.854 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.854 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.854 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:34.854 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722955+0000", 2024-05-08T08:55:34.854 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1892999999999999e-05 2024-05-08T08:55:34.854 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.854 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.854 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:34.854 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.722963+0000", 2024-05-08T08:55:34.854 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1080000000000004e-06 2024-05-08T08:55:34.854 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.854 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.855 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:34.855 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.773964+0000", 2024-05-08T08:55:34.855 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.051001022 2024-05-08T08:55:34.855 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.855 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.855 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:34.855 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.774006+0000", 2024-05-08T08:55:34.855 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2729999999999999e-05 2024-05-08T08:55:34.855 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.855 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.855 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:34.855 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.774062+0000", 2024-05-08T08:55:34.855 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5501000000000003e-05 2024-05-08T08:55:34.855 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.855 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:34.855 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.856 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.856 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.856 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6760 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1683456~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:34.856 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:55:34.856 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 183.077891085, 2024-05-08T08:55:34.856 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052571312000000002, 2024-05-08T08:55:34.856 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:34.856 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:34.856 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:34.856 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:34.856 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:34.856 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:34.856 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6760 2024-05-08T08:55:34.856 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.856 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:34.856 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.857 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:34.857 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:55:34.857 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.857 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.857 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.857 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:34.857 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:55:34.857 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.857 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.857 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.857 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:34.857 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723110+0000", 2024-05-08T08:55:34.857 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2100000000000002e-07 2024-05-08T08:55:34.857 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.857 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.858 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:34.858 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723113+0000", 2024-05-08T08:55:34.858 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3149999999999999e-06 2024-05-08T08:55:34.858 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.858 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.858 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:34.858 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723118+0000", 2024-05-08T08:55:34.858 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8860000000000001e-06 2024-05-08T08:55:34.858 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.858 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.858 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:34.858 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723123+0000", 2024-05-08T08:55:34.858 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6770000000000003e-06 2024-05-08T08:55:34.858 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.858 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.859 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:34.859 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775414+0000", 2024-05-08T08:55:34.859 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052290692999999999 2024-05-08T08:55:34.859 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.859 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.859 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:34.859 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775459+0000", 2024-05-08T08:55:34.859 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5639999999999997e-05 2024-05-08T08:55:34.859 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.859 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.859 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:34.859 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775681+0000", 2024-05-08T08:55:34.859 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00022168000000000001 2024-05-08T08:55:34.859 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.859 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:34.859 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.860 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.860 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.860 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6761 3.16 3:6bd6d6da:::rbd_data.1140782590a5.0000000000000015:6 [read 1095680~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:34.860 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:55:34.860 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 183.07787186499999, 2024-05-08T08:55:34.860 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052751275, 2024-05-08T08:55:34.860 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:34.860 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:34.860 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:34.860 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:34.860 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:34.860 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:34.860 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6761 2024-05-08T08:55:34.860 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.860 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:34.860 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.861 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:34.861 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:55:34.861 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.861 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.861 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.861 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:34.861 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.723129+0000", 2024-05-08T08:55:34.861 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.861 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.861 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.861 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:34.861 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724939+0000", 2024-05-08T08:55:34.861 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001809817 2024-05-08T08:55:34.861 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.861 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.861 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:34.862 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724943+0000", 2024-05-08T08:55:34.862 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4920000000000004e-06 2024-05-08T08:55:34.862 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.862 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.862 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:34.862 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724956+0000", 2024-05-08T08:55:34.862 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2427e-05 2024-05-08T08:55:34.862 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.862 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.862 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:34.862 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.724963+0000", 2024-05-08T08:55:34.862 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2559999999999996e-06 2024-05-08T08:55:34.862 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.862 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.863 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:34.863 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775711+0000", 2024-05-08T08:55:34.863 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050747621 2024-05-08T08:55:34.863 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.863 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.863 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:34.863 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775785+0000", 2024-05-08T08:55:34.863 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4634000000000004e-05 2024-05-08T08:55:34.863 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.863 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.863 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:34.863 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.775880+0000", 2024-05-08T08:55:34.863 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5027999999999999e-05 2024-05-08T08:55:34.863 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.863 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:34.863 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.864 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.864 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.864 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6770 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1765376~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:34.864 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:55:34.864 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 183.07584077000001, 2024-05-08T08:55:34.864 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052339296, 2024-05-08T08:55:34.864 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:34.864 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:34.864 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:34.864 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:34.864 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:34.864 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:34.864 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6770 2024-05-08T08:55:34.864 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.864 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:34.864 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.865 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:34.865 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:55:34.865 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.865 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.865 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.865 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:34.865 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725160+0000", 2024-05-08T08:55:34.865 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.865 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.865 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.865 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:34.865 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725161+0000", 2024-05-08T08:55:34.865 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-05-08T08:55:34.865 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.865 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.865 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:34.866 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725163+0000", 2024-05-08T08:55:34.866 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4370000000000001e-06 2024-05-08T08:55:34.866 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.866 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.866 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:34.866 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725169+0000", 2024-05-08T08:55:34.866 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8409999999999996e-06 2024-05-08T08:55:34.866 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.866 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.866 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:34.866 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725174+0000", 2024-05-08T08:55:34.866 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7249999999999997e-06 2024-05-08T08:55:34.866 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.866 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.867 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:34.867 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777343+0000", 2024-05-08T08:55:34.867 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052169633999999999 2024-05-08T08:55:34.867 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.867 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.867 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:34.867 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777409+0000", 2024-05-08T08:55:34.867 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6033999999999998e-05 2024-05-08T08:55:34.867 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.867 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.867 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:34.867 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777499+0000", 2024-05-08T08:55:34.867 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0224999999999998e-05 2024-05-08T08:55:34.867 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.867 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:34.868 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.868 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.868 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.868 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6771 3.16 3:6bd6d6da:::rbd_data.1140782590a5.0000000000000015:6 [read 1177600~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:34.868 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:55:34.868 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 183.075820953, 2024-05-08T08:55:34.868 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052438604999999999, 2024-05-08T08:55:34.868 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:34.868 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:34.868 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:34.868 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:34.868 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:34.868 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:34.868 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6771 2024-05-08T08:55:34.868 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.868 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:34.869 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.869 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:34.869 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:55:34.869 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.869 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.869 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.869 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:34.869 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:55:34.869 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.869 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.869 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.869 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:34.869 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725180+0000", 2024-05-08T08:55:34.869 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:55:34.869 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.869 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.870 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:34.870 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725188+0000", 2024-05-08T08:55:34.870 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1579999999999999e-06 2024-05-08T08:55:34.870 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.870 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.870 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:34.870 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725195+0000", 2024-05-08T08:55:34.870 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8909999999999995e-06 2024-05-08T08:55:34.870 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.870 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.870 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:34.870 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725200+0000", 2024-05-08T08:55:34.870 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9810000000000003e-06 2024-05-08T08:55:34.870 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.870 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.871 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:34.871 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777527+0000", 2024-05-08T08:55:34.871 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052326582000000003 2024-05-08T08:55:34.871 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.871 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.871 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:34.871 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777566+0000", 2024-05-08T08:55:34.871 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9447999999999999e-05 2024-05-08T08:55:34.871 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.871 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.871 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:34.871 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777619+0000", 2024-05-08T08:55:34.871 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2135000000000001e-05 2024-05-08T08:55:34.871 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.871 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:34.871 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.872 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.872 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.872 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6772 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1781760~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:34.872 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:55:34.872 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 183.075792896, 2024-05-08T08:55:34.872 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052601799999999997, 2024-05-08T08:55:34.872 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:34.872 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:34.872 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:34.872 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:34.872 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:34.872 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:34.872 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6772 2024-05-08T08:55:34.872 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.872 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:34.873 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.873 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:34.873 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:55:34.873 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.873 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.873 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.873 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:34.873 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.725208+0000", 2024-05-08T08:55:34.873 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.873 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.873 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.873 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:34.873 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726939+0000", 2024-05-08T08:55:34.873 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001730493 2024-05-08T08:55:34.873 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.874 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.874 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:34.874 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726943+0000", 2024-05-08T08:55:34.874 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7330000000000003e-06 2024-05-08T08:55:34.874 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.874 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.874 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:34.874 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726954+0000", 2024-05-08T08:55:34.874 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1107000000000001e-05 2024-05-08T08:55:34.874 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.874 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.874 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:34.874 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.726962+0000", 2024-05-08T08:55:34.874 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.503e-06 2024-05-08T08:55:34.874 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.875 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.875 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:34.875 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777653+0000", 2024-05-08T08:55:34.875 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050691542999999999 2024-05-08T08:55:34.875 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.875 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.875 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:34.875 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777719+0000", 2024-05-08T08:55:34.875 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5228000000000006e-05 2024-05-08T08:55:34.875 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.875 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.875 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:34.875 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.777810+0000", 2024-05-08T08:55:34.875 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1193000000000006e-05 2024-05-08T08:55:34.875 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.876 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:34.876 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.876 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.876 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.876 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4621.0:6782 3.16 3:68614ec1:::rbd_data.1140782590a5.0000000000000016:6 [read 1880064~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T08:55:34.876 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:55:34.876 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 183.07382438900001, 2024-05-08T08:55:34.876 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.052326492000000002, 2024-05-08T08:55:34.876 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-05-08T08:55:34.876 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-05-08T08:55:34.876 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-05-08T08:55:34.876 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-05-08T08:55:34.876 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4621", 2024-05-08T08:55:34.876 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.196:0/54171349", 2024-05-08T08:55:34.876 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6782 2024-05-08T08:55:34.876 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.876 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-05-08T08:55:34.877 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.877 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-05-08T08:55:34.877 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:55:34.877 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.877 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.877 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.877 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-05-08T08:55:34.877 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.727177+0000", 2024-05-08T08:55:34.877 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-05-08T08:55:34.877 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.877 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.877 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-05-08T08:55:34.877 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728939+0000", 2024-05-08T08:55:34.877 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001762719 2024-05-08T08:55:34.877 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.878 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.878 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-05-08T08:55:34.878 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728944+0000", 2024-05-08T08:55:34.878 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.746e-06 2024-05-08T08:55:34.878 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.878 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.878 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-05-08T08:55:34.878 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728955+0000", 2024-05-08T08:55:34.878 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1043000000000001e-05 2024-05-08T08:55:34.878 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.878 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.878 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-05-08T08:55:34.878 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.728963+0000", 2024-05-08T08:55:34.878 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5809999999999999e-06 2024-05-08T08:55:34.878 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.879 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.879 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-05-08T08:55:34.879 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779320+0000", 2024-05-08T08:55:34.879 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050357290999999998 2024-05-08T08:55:34.879 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.879 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.879 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-05-08T08:55:34.879 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779401+0000", 2024-05-08T08:55:34.879 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0574000000000005e-05 2024-05-08T08:55:34.879 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T08:55:34.879 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-05-08T08:55:34.879 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-05-08T08:55:34.879 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-05-08T08:52:31.779503+0000", 2024-05-08T08:55:34.879 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102538 2024-05-08T08:55:34.879 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.879 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:34.880 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.880 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T08:55:34.880 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T08:55:34.880 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T08:55:35.478 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-05-08T08:55:36.118 INFO:tasks.thrashosds.thrasher:inning osd 2024-05-08T08:55:36.119 INFO:tasks.thrashosds.thrasher:Adding osd 0 2024-05-08T08:55:36.119 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 0 2024-05-08T08:55:36.428 INFO:tasks.ceph.mon.a.smithi059.stderr:2024-05-08T08:55:36.430+0000 7fc9cc811640 -1 mon.a@0(leader).osd e183 definitely_dead 0 2024-05-08T08:55:36.481 INFO:teuthology.orchestra.run.smithi059.stderr:noscrub is unset 2024-05-08T08:55:36.486 INFO:teuthology.orchestra.run.smithi059.stderr:marked in osd.0. 2024-05-08T08:55:36.500 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-05-08T08:55:36.501 INFO:tasks.thrashosds.thrasher:Added osd 0 2024-05-08T08:55:36.501 INFO:tasks.thrashosds.thrasher:inning osd 2024-05-08T08:55:36.502 INFO:tasks.thrashosds.thrasher:Adding osd 5 2024-05-08T08:55:36.502 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 5 2024-05-08T08:55:36.856 INFO:tasks.ceph.mon.a.smithi059.stderr:2024-05-08T08:55:36.859+0000 7fc9cc811640 -1 mon.a@0(leader).osd e184 definitely_dead 0 2024-05-08T08:55:37.483 INFO:teuthology.orchestra.run.smithi059.stderr:nodeep-scrub is unset 2024-05-08T08:55:37.483 INFO:teuthology.orchestra.run.smithi059.stderr:marked in osd.5. 2024-05-08T08:55:37.510 INFO:tasks.thrashosds.thrasher:Added osd 5 2024-05-08T08:55:37.511 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-05-08T08:55:37.674 INFO:teuthology.orchestra.run.smithi059.stdout:mon.a: {} 2024-05-08T08:55:37.675 INFO:teuthology.orchestra.run.smithi059.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-05-08T08:55:37.684 INFO:teuthology.orchestra.run.smithi059.stdout:mon.b: {} 2024-05-08T08:55:37.685 INFO:teuthology.orchestra.run.smithi059.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-05-08T08:55:37.693 INFO:teuthology.orchestra.run.smithi059.stdout:mon.c: {} 2024-05-08T08:55:37.693 INFO:teuthology.orchestra.run.smithi059.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-05-08T08:55:37.704 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-05-08T08:55:38.488 INFO:teuthology.orchestra.run.smithi059.stderr:reweighted osd.0 to 1 (10000) 2024-05-08T08:55:38.532 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-05-08T08:55:39.486 INFO:teuthology.orchestra.run.smithi059.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-05-08T08:55:39.514 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-05-08T08:55:40.367 INFO:teuthology.orchestra.run.smithi059.stderr:reweighted osd.1 to 1 (10000) 2024-05-08T08:55:40.391 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-05-08T08:55:41.371 INFO:teuthology.orchestra.run.smithi059.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-05-08T08:55:41.388 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-05-08T08:55:42.369 INFO:teuthology.orchestra.run.smithi059.stderr:reweighted osd.2 to 1 (10000) 2024-05-08T08:55:42.387 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-05-08T08:55:43.370 INFO:teuthology.orchestra.run.smithi059.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-05-08T08:55:43.388 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-05-08T08:55:44.378 INFO:teuthology.orchestra.run.smithi059.stderr:reweighted osd.3 to 1 (10000) 2024-05-08T08:55:44.397 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-05-08T08:55:45.366 INFO:teuthology.orchestra.run.smithi059.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-05-08T08:55:45.386 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-05-08T08:55:46.527 INFO:teuthology.orchestra.run.smithi059.stderr:reweighted osd.4 to 1 (10000) 2024-05-08T08:55:46.544 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-05-08T08:55:47.533 INFO:teuthology.orchestra.run.smithi059.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-05-08T08:55:47.549 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-05-08T08:55:48.543 INFO:teuthology.orchestra.run.smithi059.stderr:reweighted osd.5 to 1 (10000) 2024-05-08T08:55:48.560 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-05-08T08:55:49.549 INFO:teuthology.orchestra.run.smithi059.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-05-08T08:55:49.566 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-05-08T08:55:49.566 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-05-08T08:55:49.566 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-05-08T08:55:49.567 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-05-08T08:55:49.567 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-05-08T08:55:49.567 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T08:55:49.871 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:55:49.871 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":197,"fsid":"38dfae8c-5c5e-4b0f-a264-91f22c6c5554","created":"2024-05-08T08:51:45.571369+0000","modified":"2024-05-08T08:55:49.544013+0000","last_up_change":"2024-05-08T08:51:56.847706+0000","last_in_change":"2024-05-08T08:55:37.484433+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T08:51:57.787790+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-05-08T08:52:02.563677+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-05-08T08:52:09.955467+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"182","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":51,"snap_epoch":182,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.309999942779541,"score_stable":1.309999942779541,"optimal_score":1,"raw_score_acting":1.309999942779541,"raw_score_stable":1.309999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"300801a5-1919-4b23-8c6d-bc7dad61cc4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":802160151},{"type":"v1","addr":"172.21.15.59:6801","nonce":802160151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":802160151},{"type":"v1","addr":"172.21.15.59:6803","nonce":802160151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":802160151},{"type":"v1","addr":"172.21.15.59:6807","nonce":802160151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":802160151},{"type":"v1","addr":"172.21.15.59:6805","nonce":802160151}]},"public_addr":"172.21.15.59:6801/802160151","cluster_addr":"172.21.15.59:6803/802160151","heartbeat_back_addr":"172.21.15.59:6807/802160151","heartbeat_front_addr":"172.21.15.59:6805/802160151","state":["exists","up"]},{"osd":1,"uuid":"0fead0de-6a65-4eba-b400-eea5692702e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":4116801011},{"type":"v1","addr":"172.21.15.59:6817","nonce":4116801011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":4116801011},{"type":"v1","addr":"172.21.15.59:6819","nonce":4116801011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":4116801011},{"type":"v1","addr":"172.21.15.59:6823","nonce":4116801011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":4116801011},{"type":"v1","addr":"172.21.15.59:6821","nonce":4116801011}]},"public_addr":"172.21.15.59:6817/4116801011","cluster_addr":"172.21.15.59:6819/4116801011","heartbeat_back_addr":"172.21.15.59:6823/4116801011","heartbeat_front_addr":"172.21.15.59:6821/4116801011","state":["exists","up"]},{"osd":2,"uuid":"1a233de6-1b82-4a51-8e96-4cd4abdf16c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1547661179},{"type":"v1","addr":"172.21.15.59:6809","nonce":1547661179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1547661179},{"type":"v1","addr":"172.21.15.59:6811","nonce":1547661179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1547661179},{"type":"v1","addr":"172.21.15.59:6815","nonce":1547661179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1547661179},{"type":"v1","addr":"172.21.15.59:6813","nonce":1547661179}]},"public_addr":"172.21.15.59:6809/1547661179","cluster_addr":"172.21.15.59:6811/1547661179","heartbeat_back_addr":"172.21.15.59:6815/1547661179","heartbeat_front_addr":"172.21.15.59:6813/1547661179","state":["exists","up"]},{"osd":3,"uuid":"cc4870b8-dcd9-4e43-978d-8de6c6396abf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":4054400810},{"type":"v1","addr":"172.21.15.192:6801","nonce":4054400810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":4054400810},{"type":"v1","addr":"172.21.15.192:6803","nonce":4054400810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":4054400810},{"type":"v1","addr":"172.21.15.192:6807","nonce":4054400810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":4054400810},{"type":"v1","addr":"172.21.15.192:6805","nonce":4054400810}]},"public_addr":"172.21.15.192:6801/4054400810","cluster_addr":"172.21.15.192:6803/4054400810","heartbeat_back_addr":"172.21.15.192:6807/4054400810","heartbeat_front_addr":"172.21.15.192:6805/4054400810","state":["exists","up"]},{"osd":4,"uuid":"d1d05620-42fb-4107-a2f9-e83542d076ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":58985565},{"type":"v1","addr":"172.21.15.192:6809","nonce":58985565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":58985565},{"type":"v1","addr":"172.21.15.192:6811","nonce":58985565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":58985565},{"type":"v1","addr":"172.21.15.192:6815","nonce":58985565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":58985565},{"type":"v1","addr":"172.21.15.192:6813","nonce":58985565}]},"public_addr":"172.21.15.192:6809/58985565","cluster_addr":"172.21.15.192:6811/58985565","heartbeat_back_addr":"172.21.15.192:6815/58985565","heartbeat_front_addr":"172.21.15.192:6813/58985565","state":["exists","up"]},{"osd":5,"uuid":"ab26136c-31c3-4090-9fa2-988dba4f731f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":1283571573},{"type":"v1","addr":"172.21.15.192:6817","nonce":1283571573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1283571573},{"type":"v1","addr":"172.21.15.192:6819","nonce":1283571573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":1283571573},{"type":"v1","addr":"172.21.15.192:6823","nonce":1283571573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":1283571573},{"type":"v1","addr":"172.21.15.192:6821","nonce":1283571573}]},"public_addr":"172.21.15.192:6817/1283571573","cluster_addr":"172.21.15.192:6819/1283571573","heartbeat_back_addr":"172.21.15.192:6823/1283571573","heartbeat_front_addr":"172.21.15.192:6821/1283571573","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T08:51:52.396946+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T08:51:52.453160+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T08:51:52.387606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T08:51:52.360952+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T08:51:52.468681+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T08:51:52.401080+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"2.5","osds":[3,2]},{"pgid":"3.4","osds":[1,5]},{"pgid":"3.17","osds":[1,0]},{"pgid":"3.1d","osds":[0,5]}],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T08:55:49.884 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-05-08T08:55:49.884 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T08:55:50.180 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:55:50.181 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":197,"fsid":"38dfae8c-5c5e-4b0f-a264-91f22c6c5554","created":"2024-05-08T08:51:45.571369+0000","modified":"2024-05-08T08:55:49.544013+0000","last_up_change":"2024-05-08T08:51:56.847706+0000","last_in_change":"2024-05-08T08:55:37.484433+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T08:51:57.787790+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-05-08T08:52:02.563677+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-05-08T08:52:09.955467+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"182","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":51,"snap_epoch":182,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.309999942779541,"score_stable":1.309999942779541,"optimal_score":1,"raw_score_acting":1.309999942779541,"raw_score_stable":1.309999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"300801a5-1919-4b23-8c6d-bc7dad61cc4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":802160151},{"type":"v1","addr":"172.21.15.59:6801","nonce":802160151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":802160151},{"type":"v1","addr":"172.21.15.59:6803","nonce":802160151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":802160151},{"type":"v1","addr":"172.21.15.59:6807","nonce":802160151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":802160151},{"type":"v1","addr":"172.21.15.59:6805","nonce":802160151}]},"public_addr":"172.21.15.59:6801/802160151","cluster_addr":"172.21.15.59:6803/802160151","heartbeat_back_addr":"172.21.15.59:6807/802160151","heartbeat_front_addr":"172.21.15.59:6805/802160151","state":["exists","up"]},{"osd":1,"uuid":"0fead0de-6a65-4eba-b400-eea5692702e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":4116801011},{"type":"v1","addr":"172.21.15.59:6817","nonce":4116801011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":4116801011},{"type":"v1","addr":"172.21.15.59:6819","nonce":4116801011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":4116801011},{"type":"v1","addr":"172.21.15.59:6823","nonce":4116801011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":4116801011},{"type":"v1","addr":"172.21.15.59:6821","nonce":4116801011}]},"public_addr":"172.21.15.59:6817/4116801011","cluster_addr":"172.21.15.59:6819/4116801011","heartbeat_back_addr":"172.21.15.59:6823/4116801011","heartbeat_front_addr":"172.21.15.59:6821/4116801011","state":["exists","up"]},{"osd":2,"uuid":"1a233de6-1b82-4a51-8e96-4cd4abdf16c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1547661179},{"type":"v1","addr":"172.21.15.59:6809","nonce":1547661179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1547661179},{"type":"v1","addr":"172.21.15.59:6811","nonce":1547661179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1547661179},{"type":"v1","addr":"172.21.15.59:6815","nonce":1547661179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1547661179},{"type":"v1","addr":"172.21.15.59:6813","nonce":1547661179}]},"public_addr":"172.21.15.59:6809/1547661179","cluster_addr":"172.21.15.59:6811/1547661179","heartbeat_back_addr":"172.21.15.59:6815/1547661179","heartbeat_front_addr":"172.21.15.59:6813/1547661179","state":["exists","up"]},{"osd":3,"uuid":"cc4870b8-dcd9-4e43-978d-8de6c6396abf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":4054400810},{"type":"v1","addr":"172.21.15.192:6801","nonce":4054400810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":4054400810},{"type":"v1","addr":"172.21.15.192:6803","nonce":4054400810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":4054400810},{"type":"v1","addr":"172.21.15.192:6807","nonce":4054400810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":4054400810},{"type":"v1","addr":"172.21.15.192:6805","nonce":4054400810}]},"public_addr":"172.21.15.192:6801/4054400810","cluster_addr":"172.21.15.192:6803/4054400810","heartbeat_back_addr":"172.21.15.192:6807/4054400810","heartbeat_front_addr":"172.21.15.192:6805/4054400810","state":["exists","up"]},{"osd":4,"uuid":"d1d05620-42fb-4107-a2f9-e83542d076ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":58985565},{"type":"v1","addr":"172.21.15.192:6809","nonce":58985565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":58985565},{"type":"v1","addr":"172.21.15.192:6811","nonce":58985565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":58985565},{"type":"v1","addr":"172.21.15.192:6815","nonce":58985565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":58985565},{"type":"v1","addr":"172.21.15.192:6813","nonce":58985565}]},"public_addr":"172.21.15.192:6809/58985565","cluster_addr":"172.21.15.192:6811/58985565","heartbeat_back_addr":"172.21.15.192:6815/58985565","heartbeat_front_addr":"172.21.15.192:6813/58985565","state":["exists","up"]},{"osd":5,"uuid":"ab26136c-31c3-4090-9fa2-988dba4f731f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":1283571573},{"type":"v1","addr":"172.21.15.192:6817","nonce":1283571573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1283571573},{"type":"v1","addr":"172.21.15.192:6819","nonce":1283571573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":1283571573},{"type":"v1","addr":"172.21.15.192:6823","nonce":1283571573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":1283571573},{"type":"v1","addr":"172.21.15.192:6821","nonce":1283571573}]},"public_addr":"172.21.15.192:6817/1283571573","cluster_addr":"172.21.15.192:6819/1283571573","heartbeat_back_addr":"172.21.15.192:6823/1283571573","heartbeat_front_addr":"172.21.15.192:6821/1283571573","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T08:51:52.396946+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T08:51:52.453160+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T08:51:52.387606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T08:51:52.360952+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T08:51:52.468681+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T08:51:52.401080+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"2.5","osds":[3,2]},{"pgid":"3.4","osds":[1,5]},{"pgid":"3.17","osds":[1,0]},{"pgid":"3.1d","osds":[0,5]}],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T08:55:50.194 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-05-08T08:55:50.194 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-05-08T08:55:50.195 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-05-08T08:55:50.195 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-05-08T08:55:50.196 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-05-08T08:55:50.196 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-05-08T08:55:50.353 INFO:teuthology.orchestra.run.smithi059.stdout:47244640308 2024-05-08T08:55:50.353 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-08T08:55:50.354 INFO:teuthology.orchestra.run.smithi059.stdout:47244640305 2024-05-08T08:55:50.354 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-08T08:55:50.412 INFO:teuthology.orchestra.run.smithi059.stdout:47244640306 2024-05-08T08:55:50.412 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-08T08:55:50.413 INFO:teuthology.orchestra.run.smithi059.stdout:47244640309 2024-05-08T08:55:50.413 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-08T08:55:50.424 INFO:teuthology.orchestra.run.smithi059.stdout:47244640305 2024-05-08T08:55:50.424 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-08T08:55:50.426 INFO:teuthology.orchestra.run.smithi059.stdout:47244640307 2024-05-08T08:55:50.426 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-08T08:55:50.778 INFO:teuthology.orchestra.run.smithi059.stdout:47244640307 2024-05-08T08:55:50.795 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640308 got 47244640307 for osd.0 2024-05-08T08:55:50.801 INFO:teuthology.orchestra.run.smithi059.stdout:47244640304 2024-05-08T08:55:50.819 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640305 got 47244640304 for osd.5 2024-05-08T08:55:50.880 INFO:teuthology.orchestra.run.smithi059.stdout:47244640306 2024-05-08T08:55:50.894 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640307 got 47244640306 for osd.3 2024-05-08T08:55:50.899 INFO:teuthology.orchestra.run.smithi059.stdout:47244640304 2024-05-08T08:55:50.911 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640305 got 47244640304 for osd.1 2024-05-08T08:55:50.912 INFO:teuthology.orchestra.run.smithi059.stdout:47244640308 2024-05-08T08:55:50.925 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640309 got 47244640308 for osd.4 2024-05-08T08:55:50.925 INFO:teuthology.orchestra.run.smithi059.stdout:47244640305 2024-05-08T08:55:50.938 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640306 got 47244640305 for osd.2 2024-05-08T08:55:51.795 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-08T08:55:51.820 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-08T08:55:51.896 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-08T08:55:51.913 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-08T08:55:51.926 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-08T08:55:51.938 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-08T08:55:52.163 INFO:teuthology.orchestra.run.smithi059.stdout:47244640307 2024-05-08T08:55:52.178 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640308 got 47244640307 for osd.0 2024-05-08T08:55:52.220 INFO:teuthology.orchestra.run.smithi059.stdout:47244640304 2024-05-08T08:55:52.233 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640305 got 47244640304 for osd.5 2024-05-08T08:55:52.339 INFO:teuthology.orchestra.run.smithi059.stdout:47244640304 2024-05-08T08:55:52.349 INFO:teuthology.orchestra.run.smithi059.stdout:47244640308 2024-05-08T08:55:52.358 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640305 got 47244640304 for osd.1 2024-05-08T08:55:52.358 INFO:teuthology.orchestra.run.smithi059.stdout:47244640306 2024-05-08T08:55:52.363 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640309 got 47244640308 for osd.4 2024-05-08T08:55:52.371 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640307 got 47244640306 for osd.3 2024-05-08T08:55:52.398 INFO:teuthology.orchestra.run.smithi059.stdout:47244640305 2024-05-08T08:55:52.411 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640306 got 47244640305 for osd.2 2024-05-08T08:55:53.180 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-08T08:55:53.234 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-08T08:55:53.359 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-08T08:55:53.364 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-08T08:55:53.372 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-08T08:55:53.412 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-08T08:55:53.574 INFO:teuthology.orchestra.run.smithi059.stdout:47244640308 2024-05-08T08:55:53.591 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640308 got 47244640308 for osd.0 2024-05-08T08:55:53.591 DEBUG:teuthology.parallel:result is None 2024-05-08T08:55:53.625 INFO:teuthology.orchestra.run.smithi059.stdout:47244640305 2024-05-08T08:55:53.640 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640305 got 47244640305 for osd.5 2024-05-08T08:55:53.640 DEBUG:teuthology.parallel:result is None 2024-05-08T08:55:53.745 INFO:teuthology.orchestra.run.smithi059.stdout:47244640307 2024-05-08T08:55:53.758 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640307 got 47244640307 for osd.3 2024-05-08T08:55:53.759 DEBUG:teuthology.parallel:result is None 2024-05-08T08:55:53.766 INFO:teuthology.orchestra.run.smithi059.stdout:47244640305 2024-05-08T08:55:53.783 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640305 got 47244640305 for osd.1 2024-05-08T08:55:53.783 DEBUG:teuthology.parallel:result is None 2024-05-08T08:55:53.785 INFO:teuthology.orchestra.run.smithi059.stdout:47244640309 2024-05-08T08:55:53.798 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640309 got 47244640309 for osd.4 2024-05-08T08:55:53.799 DEBUG:teuthology.parallel:result is None 2024-05-08T08:55:53.832 INFO:teuthology.orchestra.run.smithi059.stdout:47244640306 2024-05-08T08:55:53.845 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640306 got 47244640306 for osd.2 2024-05-08T08:55:53.845 DEBUG:teuthology.parallel:result is None 2024-05-08T08:55:53.845 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-05-08T08:55:53.846 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T08:55:54.185 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:55:54.186 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-05-08T08:55:54.197 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":311,"stamp":"2024-05-08T08:55:53.780431+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":1297,"num_object_clones":0,"num_object_copies":2594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1297,"num_whiteouts":1291,"num_read":513049,"num_read_kb":1628462,"num_write":26259,"num_write_kb":454255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11022,"num_bytes_recovered":423134873,"num_keys_recovered":904,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24946,"ondisk_log_size":24946,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":367776,"kb_used_data":14072,"kb_used_omap":9,"kb_used_meta":353590,"kb_avail":565863264,"statfs":{"total":579820584960,"available":579443982336,"internally_reserved":0,"allocated":14409728,"data_stored":7281508,"data_compressed":11905,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":9568,"internal_metadata":362076832},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-743,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001324"},"pg_stats":[{"pgid":"3.1f","version":"182'772","reported_seq":20838,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.562464+0000","last_change":"2024-05-08T08:55:43.388833+0000","last_active":"2024-05-08T08:55:50.562464+0000","last_peered":"2024-05-08T08:55:50.562464+0000","last_clean":"2024-05-08T08:55:50.562464+0000","last_became_active":"2024-05-08T08:55:43.386102+0000","last_became_peered":"2024-05-08T08:55:43.386102+0000","last_unstale":"2024-05-08T08:55:50.562464+0000","last_undegraded":"2024-05-08T08:55:50.562464+0000","last_fullsized":"2024-05-08T08:55:50.562464+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":772,"log_dups_size":0,"ondisk_log_size":772,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:25:19.005824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038328300000000002,"stat_sum":{"num_bytes":0,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":34,"num_read":19333,"num_read_kb":85778,"num_write":988,"num_write_kb":33105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":328,"num_bytes_recovered":19152951,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"181'922","reported_seq":11965,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.554583+0000","last_change":"2024-05-08T08:55:33.482489+0000","last_active":"2024-05-08T08:55:50.554583+0000","last_peered":"2024-05-08T08:55:50.554583+0000","last_clean":"2024-05-08T08:55:50.554583+0000","last_became_active":"2024-05-08T08:52:11.930844+0000","last_became_peered":"2024-05-08T08:52:11.930844+0000","last_unstale":"2024-05-08T08:55:50.554583+0000","last_undegraded":"2024-05-08T08:55:50.554583+0000","last_fullsized":"2024-05-08T08:55:50.554583+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":922,"log_dups_size":0,"ondisk_log_size":922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:34:27.496752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001408816,"stat_sum":{"num_bytes":0,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":49,"num_read":13744,"num_read_kb":37146,"num_write":694,"num_write_kb":12720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"181'753","reported_seq":4867,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.562479+0000","last_change":"2024-05-08T08:55:44.384669+0000","last_active":"2024-05-08T08:55:50.562479+0000","last_peered":"2024-05-08T08:55:50.562479+0000","last_clean":"2024-05-08T08:55:50.562479+0000","last_became_active":"2024-05-08T08:55:44.381069+0000","last_became_peered":"2024-05-08T08:55:44.381069+0000","last_unstale":"2024-05-08T08:55:50.562479+0000","last_undegraded":"2024-05-08T08:55:50.562479+0000","last_fullsized":"2024-05-08T08:55:50.562479+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":753,"log_dups_size":0,"ondisk_log_size":753,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:27:12.956766+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0093620049999999996,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":40,"num_read":8309,"num_read_kb":30535,"num_write":901,"num_write_kb":7282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":570,"num_bytes_recovered":16438479,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"182'740","reported_seq":19620,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.562811+0000","last_change":"2024-05-08T08:55:50.562811+0000","last_active":"2024-05-08T08:55:50.562811+0000","last_peered":"2024-05-08T08:55:50.562811+0000","last_clean":"2024-05-08T08:55:50.562811+0000","last_became_active":"2024-05-08T08:55:50.560370+0000","last_became_peered":"2024-05-08T08:55:50.560370+0000","last_unstale":"2024-05-08T08:55:50.562811+0000","last_undegraded":"2024-05-08T08:55:50.562811+0000","last_fullsized":"2024-05-08T08:55:50.562811+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":740,"log_dups_size":0,"ondisk_log_size":740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:20:09.589464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019815259999999999,"stat_sum":{"num_bytes":0,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":45,"num_read":18641,"num_read_kb":101548,"num_write":811,"num_write_kb":27861,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":609,"num_bytes_recovered":86192377,"num_keys_recovered":338,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"179'637","reported_seq":26061,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.562537+0000","last_change":"2024-05-08T08:55:43.388686+0000","last_active":"2024-05-08T08:55:50.562537+0000","last_peered":"2024-05-08T08:55:50.562537+0000","last_clean":"2024-05-08T08:55:50.562537+0000","last_became_active":"2024-05-08T08:55:43.386098+0000","last_became_peered":"2024-05-08T08:55:43.386098+0000","last_unstale":"2024-05-08T08:55:50.562537+0000","last_undegraded":"2024-05-08T08:55:50.562537+0000","last_fullsized":"2024-05-08T08:55:50.562537+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":637,"log_dups_size":0,"ondisk_log_size":637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:29:43.226913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012631949999999999,"stat_sum":{"num_bytes":0,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":50,"num_read":25112,"num_read_kb":45826,"num_write":688,"num_write_kb":23312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":455,"num_bytes_recovered":5234820,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"180'867","reported_seq":22286,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.559406+0000","last_change":"2024-05-08T08:55:33.475720+0000","last_active":"2024-05-08T08:55:50.559406+0000","last_peered":"2024-05-08T08:55:50.559406+0000","last_clean":"2024-05-08T08:55:50.559406+0000","last_became_active":"2024-05-08T08:53:24.484124+0000","last_became_peered":"2024-05-08T08:53:24.484124+0000","last_unstale":"2024-05-08T08:55:50.559406+0000","last_undegraded":"2024-05-08T08:55:50.559406+0000","last_fullsized":"2024-05-08T08:55:50.559406+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":867,"log_dups_size":0,"ondisk_log_size":867,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:57:41.478541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012989099999999999,"stat_sum":{"num_bytes":19,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":34,"num_read":24563,"num_read_kb":59887,"num_write":908,"num_write_kb":19093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":130,"num_bytes_recovered":8549042,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"178'426","reported_seq":10752,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.557157+0000","last_change":"2024-05-08T08:55:33.470036+0000","last_active":"2024-05-08T08:55:50.557157+0000","last_peered":"2024-05-08T08:55:50.557157+0000","last_clean":"2024-05-08T08:55:50.557157+0000","last_became_active":"2024-05-08T08:53:24.483403+0000","last_became_peered":"2024-05-08T08:53:24.483403+0000","last_unstale":"2024-05-08T08:55:50.557157+0000","last_undegraded":"2024-05-08T08:55:50.557157+0000","last_fullsized":"2024-05-08T08:55:50.557157+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":426,"log_dups_size":0,"ondisk_log_size":426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:23:41.607743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011263099999999999,"stat_sum":{"num_bytes":0,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":48,"num_read":9419,"num_read_kb":48048,"num_write":368,"num_write_kb":16514,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":8814624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"181'711","reported_seq":20343,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.555276+0000","last_change":"2024-05-08T08:55:41.379915+0000","last_active":"2024-05-08T08:55:50.555276+0000","last_peered":"2024-05-08T08:55:50.555276+0000","last_clean":"2024-05-08T08:55:50.555276+0000","last_became_active":"2024-05-08T08:55:41.378182+0000","last_became_peered":"2024-05-08T08:55:41.378182+0000","last_unstale":"2024-05-08T08:55:50.555276+0000","last_undegraded":"2024-05-08T08:55:50.555276+0000","last_fullsized":"2024-05-08T08:55:50.555276+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":711,"log_dups_size":0,"ondisk_log_size":711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:23:17.505431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098929300000000003,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":20665,"num_read_kb":52569,"num_write":672,"num_write_kb":23378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":4852260,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"182'852","reported_seq":19005,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.557278+0000","last_change":"2024-05-08T08:55:40.383627+0000","last_active":"2024-05-08T08:55:50.557278+0000","last_peered":"2024-05-08T08:55:50.557278+0000","last_clean":"2024-05-08T08:55:50.557278+0000","last_became_active":"2024-05-08T08:55:40.381448+0000","last_became_peered":"2024-05-08T08:55:40.381448+0000","last_unstale":"2024-05-08T08:55:50.557278+0000","last_undegraded":"2024-05-08T08:55:50.557278+0000","last_fullsized":"2024-05-08T08:55:50.557278+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:14:47.318689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018475399999999999,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":30,"num_read":20569,"num_read_kb":77197,"num_write":969,"num_write_kb":16935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":381,"num_bytes_recovered":5593379,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"182'1461","reported_seq":21996,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.557382+0000","last_change":"2024-05-08T08:55:44.381999+0000","last_active":"2024-05-08T08:55:50.557382+0000","last_peered":"2024-05-08T08:55:50.557382+0000","last_clean":"2024-05-08T08:55:50.557382+0000","last_became_active":"2024-05-08T08:55:44.379555+0000","last_became_peered":"2024-05-08T08:55:44.379555+0000","last_unstale":"2024-05-08T08:55:50.557382+0000","last_undegraded":"2024-05-08T08:55:50.557382+0000","last_fullsized":"2024-05-08T08:55:50.557382+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":1461,"log_dups_size":0,"ondisk_log_size":1461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:07:01.906180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0045105479999999996,"stat_sum":{"num_bytes":0,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":33,"num_read":25480,"num_read_kb":87263,"num_write":2254,"num_write_kb":27190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":420,"num_bytes_recovered":42698110,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"182'690","reported_seq":17152,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.555076+0000","last_change":"2024-05-08T08:55:42.384249+0000","last_active":"2024-05-08T08:55:50.555076+0000","last_peered":"2024-05-08T08:55:50.555076+0000","last_clean":"2024-05-08T08:55:50.555076+0000","last_became_active":"2024-05-08T08:55:42.382974+0000","last_became_peered":"2024-05-08T08:55:42.382974+0000","last_unstale":"2024-05-08T08:55:50.555076+0000","last_undegraded":"2024-05-08T08:55:50.555076+0000","last_fullsized":"2024-05-08T08:55:50.555076+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":690,"log_dups_size":0,"ondisk_log_size":690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:12:41.796610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029835819999999998,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":39,"num_read":17511,"num_read_kb":53002,"num_write":778,"num_write_kb":8761,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":284,"num_bytes_recovered":1296934,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"180'733","reported_seq":12777,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.555182+0000","last_change":"2024-05-08T08:55:33.469235+0000","last_active":"2024-05-08T08:55:50.555182+0000","last_peered":"2024-05-08T08:55:50.555182+0000","last_clean":"2024-05-08T08:55:50.555182+0000","last_became_active":"2024-05-08T08:53:23.481362+0000","last_became_peered":"2024-05-08T08:53:23.481362+0000","last_unstale":"2024-05-08T08:55:50.555182+0000","last_undegraded":"2024-05-08T08:55:50.555182+0000","last_fullsized":"2024-05-08T08:55:50.555182+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":733,"log_dups_size":0,"ondisk_log_size":733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:15:25.796542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032028149999999999,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":30,"num_read":10890,"num_read_kb":33288,"num_write":663,"num_write_kb":11454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":3612768,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":384,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.555233+0000","last_change":"2024-05-08T08:53:20.362387+0000","last_active":"2024-05-08T08:55:50.555233+0000","last_peered":"2024-05-08T08:55:50.555233+0000","last_clean":"2024-05-08T08:55:50.555233+0000","last_became_active":"2024-05-08T08:53:20.362113+0000","last_became_peered":"2024-05-08T08:53:20.362113+0000","last_unstale":"2024-05-08T08:55:50.555233+0000","last_undegraded":"2024-05-08T08:55:50.555233+0000","last_fullsized":"2024-05-08T08:55:50.555233+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:28:25.204125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"182'652","reported_seq":20646,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.562407+0000","last_change":"2024-05-08T08:55:40.380225+0000","last_active":"2024-05-08T08:55:50.562407+0000","last_peered":"2024-05-08T08:55:50.562407+0000","last_clean":"2024-05-08T08:55:50.562407+0000","last_became_active":"2024-05-08T08:55:40.378157+0000","last_became_peered":"2024-05-08T08:55:40.378157+0000","last_unstale":"2024-05-08T08:55:50.562407+0000","last_undegraded":"2024-05-08T08:55:50.562407+0000","last_fullsized":"2024-05-08T08:55:50.562407+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":652,"log_dups_size":0,"ondisk_log_size":652,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:03:49.159963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019021660000000001,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":40,"num_read":21842,"num_read_kb":88722,"num_write":580,"num_write_kb":17543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":288,"num_bytes_recovered":3368960,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":375,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.557262+0000","last_change":"2024-05-08T08:52:03.896159+0000","last_active":"2024-05-08T08:55:50.557262+0000","last_peered":"2024-05-08T08:55:50.557262+0000","last_clean":"2024-05-08T08:55:50.557262+0000","last_became_active":"2024-05-08T08:52:03.895797+0000","last_became_peered":"2024-05-08T08:52:03.895797+0000","last_unstale":"2024-05-08T08:55:50.557262+0000","last_undegraded":"2024-05-08T08:55:50.557262+0000","last_fullsized":"2024-05-08T08:55:50.557262+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:34:33.513525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"180'518","reported_seq":6927,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.554715+0000","last_change":"2024-05-08T08:55:33.483261+0000","last_active":"2024-05-08T08:55:50.554715+0000","last_peered":"2024-05-08T08:55:50.554715+0000","last_clean":"2024-05-08T08:55:50.554715+0000","last_became_active":"2024-05-08T08:55:08.437577+0000","last_became_peered":"2024-05-08T08:55:08.437577+0000","last_unstale":"2024-05-08T08:55:50.554715+0000","last_undegraded":"2024-05-08T08:55:50.554715+0000","last_fullsized":"2024-05-08T08:55:50.554715+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:55:04.162036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017459470000000001,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":39,"num_read":6769,"num_read_kb":34542,"num_write":419,"num_write_kb":12024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":324,"num_bytes_recovered":15121731,"num_keys_recovered":25,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":379,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.555279+0000","last_change":"2024-05-08T08:54:59.398530+0000","last_active":"2024-05-08T08:55:50.555279+0000","last_peered":"2024-05-08T08:55:50.555279+0000","last_clean":"2024-05-08T08:55:50.555279+0000","last_became_active":"2024-05-08T08:54:59.398181+0000","last_became_peered":"2024-05-08T08:54:59.398181+0000","last_unstale":"2024-05-08T08:55:50.555279+0000","last_undegraded":"2024-05-08T08:55:50.555279+0000","last_fullsized":"2024-05-08T08:55:50.555279+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:24:28.264614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"181'863","reported_seq":27577,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.557404+0000","last_change":"2024-05-08T08:55:43.386319+0000","last_active":"2024-05-08T08:55:50.557404+0000","last_peered":"2024-05-08T08:55:50.557404+0000","last_clean":"2024-05-08T08:55:50.557404+0000","last_became_active":"2024-05-08T08:55:43.384081+0000","last_became_peered":"2024-05-08T08:55:43.384081+0000","last_unstale":"2024-05-08T08:55:50.557404+0000","last_undegraded":"2024-05-08T08:55:50.557404+0000","last_fullsized":"2024-05-08T08:55:50.557404+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":863,"log_dups_size":0,"ondisk_log_size":863,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:02:20.851342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014858339999999999,"stat_sum":{"num_bytes":0,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":48,"num_read":22624,"num_read_kb":52229,"num_write":653,"num_write_kb":12062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":791,"num_bytes_recovered":28697088,"num_keys_recovered":110,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":385,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.557499+0000","last_change":"2024-05-08T08:55:37.500332+0000","last_active":"2024-05-08T08:55:50.557499+0000","last_peered":"2024-05-08T08:55:50.557499+0000","last_clean":"2024-05-08T08:55:50.557499+0000","last_became_active":"2024-05-08T08:55:37.499784+0000","last_became_peered":"2024-05-08T08:55:37.499784+0000","last_unstale":"2024-05-08T08:55:50.557499+0000","last_undegraded":"2024-05-08T08:55:50.557499+0000","last_fullsized":"2024-05-08T08:55:50.557499+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:53:12.058683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"181'946","reported_seq":14829,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.562555+0000","last_change":"2024-05-08T08:55:50.562555+0000","last_active":"2024-05-08T08:55:50.562555+0000","last_peered":"2024-05-08T08:55:50.562555+0000","last_clean":"2024-05-08T08:55:50.562555+0000","last_became_active":"2024-05-08T08:55:50.559883+0000","last_became_peered":"2024-05-08T08:55:50.559883+0000","last_unstale":"2024-05-08T08:55:50.562555+0000","last_undegraded":"2024-05-08T08:55:50.562555+0000","last_fullsized":"2024-05-08T08:55:50.562555+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":946,"log_dups_size":0,"ondisk_log_size":946,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:18:55.639194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013854900000000001,"stat_sum":{"num_bytes":0,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":55,"num_read":15316,"num_read_kb":32869,"num_write":1141,"num_write_kb":10612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":494,"num_bytes_recovered":13827825,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"182'817","reported_seq":12316,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.559335+0000","last_change":"2024-05-08T08:55:41.381790+0000","last_active":"2024-05-08T08:55:50.559335+0000","last_peered":"2024-05-08T08:55:50.559335+0000","last_clean":"2024-05-08T08:55:50.559335+0000","last_became_active":"2024-05-08T08:55:41.380391+0000","last_became_peered":"2024-05-08T08:55:41.380391+0000","last_unstale":"2024-05-08T08:55:50.559335+0000","last_undegraded":"2024-05-08T08:55:50.559335+0000","last_fullsized":"2024-05-08T08:55:50.559335+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":817,"log_dups_size":0,"ondisk_log_size":817,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:59:49.412777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002033212,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":34,"num_read":11854,"num_read_kb":33576,"num_write":867,"num_write_kb":6636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":383,"num_bytes_recovered":9012422,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":353,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.560520+0000","last_change":"2024-05-08T08:55:50.560520+0000","last_active":"2024-05-08T08:55:50.560520+0000","last_peered":"2024-05-08T08:55:50.560520+0000","last_clean":"2024-05-08T08:55:50.560520+0000","last_became_active":"2024-05-08T08:55:50.559612+0000","last_became_peered":"2024-05-08T08:55:50.559612+0000","last_unstale":"2024-05-08T08:55:50.560520+0000","last_undegraded":"2024-05-08T08:55:50.560520+0000","last_fullsized":"2024-05-08T08:55:50.560520+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:51:19.494663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"181'729","reported_seq":14229,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.558114+0000","last_change":"2024-05-08T08:55:33.470800+0000","last_active":"2024-05-08T08:55:50.558114+0000","last_peered":"2024-05-08T08:55:50.558114+0000","last_clean":"2024-05-08T08:55:50.558114+0000","last_became_active":"2024-05-08T08:52:11.928301+0000","last_became_peered":"2024-05-08T08:52:11.928301+0000","last_unstale":"2024-05-08T08:55:50.558114+0000","last_undegraded":"2024-05-08T08:55:50.558114+0000","last_fullsized":"2024-05-08T08:55:50.558114+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":729,"log_dups_size":0,"ondisk_log_size":729,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:03:13.796981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015902449999999999,"stat_sum":{"num_bytes":0,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":37,"num_read":10704,"num_read_kb":44099,"num_write":654,"num_write_kb":16306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":375,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.555092+0000","last_change":"2024-05-08T08:52:03.891768+0000","last_active":"2024-05-08T08:55:50.555092+0000","last_peered":"2024-05-08T08:55:50.555092+0000","last_clean":"2024-05-08T08:55:50.555092+0000","last_became_active":"2024-05-08T08:52:03.891374+0000","last_became_peered":"2024-05-08T08:52:03.891374+0000","last_unstale":"2024-05-08T08:55:50.555092+0000","last_undegraded":"2024-05-08T08:55:50.555092+0000","last_fullsized":"2024-05-08T08:55:50.555092+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:18:10.240492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"182'1161","reported_seq":28197,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.562344+0000","last_change":"2024-05-08T08:55:42.388348+0000","last_active":"2024-05-08T08:55:50.562344+0000","last_peered":"2024-05-08T08:55:50.562344+0000","last_clean":"2024-05-08T08:55:50.562344+0000","last_became_active":"2024-05-08T08:55:42.386239+0000","last_became_peered":"2024-05-08T08:55:42.386239+0000","last_unstale":"2024-05-08T08:55:50.562344+0000","last_undegraded":"2024-05-08T08:55:50.562344+0000","last_fullsized":"2024-05-08T08:55:50.562344+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":1161,"log_dups_size":0,"ondisk_log_size":1161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:06:05.510596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086553400000000005,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":31078,"num_read_kb":56218,"num_write":870,"num_write_kb":9640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":490,"num_bytes_recovered":5023153,"num_keys_recovered":69,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":374,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.554962+0000","last_change":"2024-05-08T08:52:03.891551+0000","last_active":"2024-05-08T08:55:50.554962+0000","last_peered":"2024-05-08T08:55:50.554962+0000","last_clean":"2024-05-08T08:55:50.554962+0000","last_became_active":"2024-05-08T08:52:03.891296+0000","last_became_peered":"2024-05-08T08:52:03.891296+0000","last_unstale":"2024-05-08T08:55:50.554962+0000","last_undegraded":"2024-05-08T08:55:50.554962+0000","last_fullsized":"2024-05-08T08:55:50.554962+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:54.806343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"181'1002","reported_seq":15538,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.557496+0000","last_change":"2024-05-08T08:55:43.382935+0000","last_active":"2024-05-08T08:55:50.557496+0000","last_peered":"2024-05-08T08:55:50.557496+0000","last_clean":"2024-05-08T08:55:50.557496+0000","last_became_active":"2024-05-08T08:55:43.381345+0000","last_became_peered":"2024-05-08T08:55:43.381345+0000","last_unstale":"2024-05-08T08:55:50.557496+0000","last_undegraded":"2024-05-08T08:55:50.557496+0000","last_fullsized":"2024-05-08T08:55:50.557496+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":1002,"log_dups_size":0,"ondisk_log_size":1002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:30:03.146595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012464850000000001,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":39,"num_read":15521,"num_read_kb":63440,"num_write":1271,"num_write_kb":13979,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":635,"num_bytes_recovered":18405521,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":349,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.560138+0000","last_change":"2024-05-08T08:55:50.560138+0000","last_active":"2024-05-08T08:55:50.560138+0000","last_peered":"2024-05-08T08:55:50.560138+0000","last_clean":"2024-05-08T08:55:50.560138+0000","last_became_active":"2024-05-08T08:55:50.559815+0000","last_became_peered":"2024-05-08T08:55:50.559815+0000","last_unstale":"2024-05-08T08:55:50.560138+0000","last_undegraded":"2024-05-08T08:55:50.560138+0000","last_fullsized":"2024-05-08T08:55:50.560138+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:32:21.109548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":556,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.555092+0000","last_change":"2024-05-08T08:55:42.381739+0000","last_active":"2024-05-08T08:55:50.555092+0000","last_peered":"2024-05-08T08:55:50.555092+0000","last_clean":"2024-05-08T08:55:50.555092+0000","last_became_active":"2024-05-08T08:55:42.381316+0000","last_became_peered":"2024-05-08T08:55:42.381316+0000","last_unstale":"2024-05-08T08:55:50.555092+0000","last_undegraded":"2024-05-08T08:55:50.555092+0000","last_fullsized":"2024-05-08T08:55:50.555092+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:51:57.849666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:51:57.849666+0000","last_clean_scrub_stamp":"2024-05-08T08:51:57.849666+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:59:25.417866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"182'398","reported_seq":9838,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.555377+0000","last_change":"2024-05-08T08:55:33.481972+0000","last_active":"2024-05-08T08:55:50.555377+0000","last_peered":"2024-05-08T08:55:50.555377+0000","last_clean":"2024-05-08T08:55:50.555377+0000","last_became_active":"2024-05-08T08:52:11.926975+0000","last_became_peered":"2024-05-08T08:52:11.926975+0000","last_unstale":"2024-05-08T08:55:50.555377+0000","last_undegraded":"2024-05-08T08:55:50.555377+0000","last_fullsized":"2024-05-08T08:55:50.555377+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":398,"log_dups_size":0,"ondisk_log_size":398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:13:12.117653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087585099999999997,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":35,"num_read":8901,"num_read_kb":36556,"num_write":340,"num_write_kb":8985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"180'503","reported_seq":14561,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.559353+0000","last_change":"2024-05-08T08:55:33.475165+0000","last_active":"2024-05-08T08:55:50.559353+0000","last_peered":"2024-05-08T08:55:50.559353+0000","last_clean":"2024-05-08T08:55:50.559353+0000","last_became_active":"2024-05-08T08:53:23.478564+0000","last_became_peered":"2024-05-08T08:53:23.478564+0000","last_unstale":"2024-05-08T08:55:50.559353+0000","last_undegraded":"2024-05-08T08:55:50.559353+0000","last_fullsized":"2024-05-08T08:55:50.559353+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":503,"log_dups_size":0,"ondisk_log_size":503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:32:37.241705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082315600000000002,"stat_sum":{"num_bytes":0,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":42,"num_read":14198,"num_read_kb":62213,"num_write":413,"num_write_kb":16566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":131,"num_bytes_recovered":12377088,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"182'1247","reported_seq":11910,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.554993+0000","last_change":"2024-05-08T08:55:33.468391+0000","last_active":"2024-05-08T08:55:50.554993+0000","last_peered":"2024-05-08T08:55:50.554993+0000","last_clean":"2024-05-08T08:55:50.554993+0000","last_became_active":"2024-05-08T08:53:24.480359+0000","last_became_peered":"2024-05-08T08:53:24.480359+0000","last_unstale":"2024-05-08T08:55:50.554993+0000","last_undegraded":"2024-05-08T08:55:50.554993+0000","last_fullsized":"2024-05-08T08:55:50.554993+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":1247,"log_dups_size":0,"ondisk_log_size":1247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:08:14.677080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025209339999999998,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":40,"num_read":15863,"num_read_kb":29509,"num_write":1451,"num_write_kb":6331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":2175343,"num_keys_recovered":61,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"180'628","reported_seq":12516,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.557293+0000","last_change":"2024-05-08T08:55:42.384129+0000","last_active":"2024-05-08T08:55:50.557293+0000","last_peered":"2024-05-08T08:55:50.557293+0000","last_clean":"2024-05-08T08:55:50.557293+0000","last_became_active":"2024-05-08T08:55:42.382485+0000","last_became_peered":"2024-05-08T08:55:42.382485+0000","last_unstale":"2024-05-08T08:55:50.557293+0000","last_undegraded":"2024-05-08T08:55:50.557293+0000","last_fullsized":"2024-05-08T08:55:50.557293+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":628,"log_dups_size":0,"ondisk_log_size":628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:10:53.437543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001692783,"stat_sum":{"num_bytes":0,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":50,"num_read":11950,"num_read_kb":39515,"num_write":513,"num_write_kb":7725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":332,"num_bytes_recovered":2080800,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"180'378","reported_seq":14238,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.565133+0000","last_change":"2024-05-08T08:55:50.565133+0000","last_active":"2024-05-08T08:55:50.565133+0000","last_peered":"2024-05-08T08:55:50.565133+0000","last_clean":"2024-05-08T08:55:50.565133+0000","last_became_active":"2024-05-08T08:55:50.563411+0000","last_became_peered":"2024-05-08T08:55:50.563411+0000","last_unstale":"2024-05-08T08:55:50.565133+0000","last_undegraded":"2024-05-08T08:55:50.565133+0000","last_fullsized":"2024-05-08T08:55:50.565133+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:22:13.206310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020039379999999998,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":40,"num_read":13099,"num_read_kb":41266,"num_write":323,"num_write_kb":7340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":509,"num_bytes_recovered":9113137,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"180'478","reported_seq":8966,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.555126+0000","last_change":"2024-05-08T08:55:33.469549+0000","last_active":"2024-05-08T08:55:50.555126+0000","last_peered":"2024-05-08T08:55:50.555126+0000","last_clean":"2024-05-08T08:55:50.555126+0000","last_became_active":"2024-05-08T08:52:11.932594+0000","last_became_peered":"2024-05-08T08:52:11.932594+0000","last_unstale":"2024-05-08T08:55:50.555126+0000","last_undegraded":"2024-05-08T08:55:50.555126+0000","last_fullsized":"2024-05-08T08:55:50.555126+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":478,"log_dups_size":0,"ondisk_log_size":478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:53:33.929731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033757140000000001,"stat_sum":{"num_bytes":0,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":43,"num_read":9572,"num_read_kb":34328,"num_write":402,"num_write_kb":10002,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"180'632","reported_seq":9873,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.562760+0000","last_change":"2024-05-08T08:55:50.562760+0000","last_active":"2024-05-08T08:55:50.562760+0000","last_peered":"2024-05-08T08:55:50.562760+0000","last_clean":"2024-05-08T08:55:50.562760+0000","last_became_active":"2024-05-08T08:55:50.560907+0000","last_became_peered":"2024-05-08T08:55:50.560907+0000","last_unstale":"2024-05-08T08:55:50.562760+0000","last_undegraded":"2024-05-08T08:55:50.562760+0000","last_fullsized":"2024-05-08T08:55:50.562760+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":632,"log_dups_size":0,"ondisk_log_size":632,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:04:24.220863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002456153,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":16215,"num_read_kb":47213,"num_write":634,"num_write_kb":12225,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":446,"num_bytes_recovered":28147344,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"182'588","reported_seq":8774,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.557417+0000","last_change":"2024-05-08T08:55:40.386164+0000","last_active":"2024-05-08T08:55:50.557417+0000","last_peered":"2024-05-08T08:55:50.557417+0000","last_clean":"2024-05-08T08:55:50.557417+0000","last_became_active":"2024-05-08T08:55:40.383962+0000","last_became_peered":"2024-05-08T08:55:40.383962+0000","last_unstale":"2024-05-08T08:55:50.557417+0000","last_undegraded":"2024-05-08T08:55:50.557417+0000","last_fullsized":"2024-05-08T08:55:50.557417+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":588,"log_dups_size":0,"ondisk_log_size":588,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:48:37.179607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037436610000000001,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":36,"num_read":8886,"num_read_kb":38380,"num_write":568,"num_write_kb":9633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":675,"num_bytes_recovered":18815289,"num_keys_recovered":79,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"180'749","reported_seq":8638,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.567799+0000","last_change":"2024-05-08T08:55:50.567799+0000","last_active":"2024-05-08T08:55:50.567799+0000","last_peered":"2024-05-08T08:55:50.567799+0000","last_clean":"2024-05-08T08:55:50.567799+0000","last_became_active":"2024-05-08T08:55:50.565716+0000","last_became_peered":"2024-05-08T08:55:50.565716+0000","last_unstale":"2024-05-08T08:55:50.567799+0000","last_undegraded":"2024-05-08T08:55:50.567799+0000","last_fullsized":"2024-05-08T08:55:50.567799+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":749,"log_dups_size":0,"ondisk_log_size":749,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:28:22.726479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095127799999999995,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":40,"num_read":9090,"num_read_kb":41259,"num_write":984,"num_write_kb":14557,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":541,"num_bytes_recovered":26549987,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"182'750","reported_seq":11879,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.554980+0000","last_change":"2024-05-08T08:55:33.467196+0000","last_active":"2024-05-08T08:55:50.554980+0000","last_peered":"2024-05-08T08:55:50.554980+0000","last_clean":"2024-05-08T08:55:50.554980+0000","last_became_active":"2024-05-08T08:53:24.479891+0000","last_became_peered":"2024-05-08T08:53:24.479891+0000","last_unstale":"2024-05-08T08:55:50.554980+0000","last_undegraded":"2024-05-08T08:55:50.554980+0000","last_fullsized":"2024-05-08T08:55:50.554980+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":750,"log_dups_size":0,"ondisk_log_size":750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:11:35.412500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015502269999999999,"stat_sum":{"num_bytes":0,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":43,"num_read":13241,"num_read_kb":36544,"num_write":586,"num_write_kb":10920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":131,"num_bytes_recovered":2604032,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"182'1575","reported_seq":25758,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.562352+0000","last_change":"2024-05-08T08:55:43.389357+0000","last_active":"2024-05-08T08:55:50.562352+0000","last_peered":"2024-05-08T08:55:50.562352+0000","last_clean":"2024-05-08T08:55:50.562352+0000","last_became_active":"2024-05-08T08:55:43.386543+0000","last_became_peered":"2024-05-08T08:55:43.386543+0000","last_unstale":"2024-05-08T08:55:50.562352+0000","last_undegraded":"2024-05-08T08:55:50.562352+0000","last_fullsized":"2024-05-08T08:55:50.562352+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":1575,"log_dups_size":0,"ondisk_log_size":1575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:38:01.859312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015403719999999999,"stat_sum":{"num_bytes":0,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":52,"num_read":30937,"num_read_kb":62696,"num_write":2139,"num_write_kb":12077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":330,"num_bytes_recovered":4235412,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"181'651","reported_seq":11695,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.555086+0000","last_change":"2024-05-08T08:55:43.380556+0000","last_active":"2024-05-08T08:55:50.555086+0000","last_peered":"2024-05-08T08:55:50.555086+0000","last_clean":"2024-05-08T08:55:50.555086+0000","last_became_active":"2024-05-08T08:55:43.379146+0000","last_became_peered":"2024-05-08T08:55:43.379146+0000","last_unstale":"2024-05-08T08:55:50.555086+0000","last_undegraded":"2024-05-08T08:55:50.555086+0000","last_fullsized":"2024-05-08T08:55:50.555086+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":651,"log_dups_size":0,"ondisk_log_size":651,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:09:48.532237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022701349999999999,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":40,"num_read":11037,"num_read_kb":41101,"num_write":598,"num_write_kb":9071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":589,"num_bytes_recovered":19963261,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":1295,"num_object_clones":0,"num_object_copies":2590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1295,"num_whiteouts":1291,"num_read":512933,"num_read_kb":1628362,"num_write":26100,"num_write_kb":451839,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11018,"num_bytes_recovered":421954137,"num_keys_recovered":904,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24829,"ondisk_log_size":24829,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":18287,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640306,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59884,"kb_used_data":2328,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311956,"statfs":{"total":96636764160,"available":96575442944,"internally_reserved":0,"allocated":2383872,"data_stored":1016789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640310,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55724,"kb_used_data":2332,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316116,"statfs":{"total":96636764160,"available":96579702784,"internally_reserved":0,"allocated":2387968,"data_stored":1016808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640308,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65504,"kb_used_data":2384,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306336,"statfs":{"total":96636764160,"available":96569688064,"internally_reserved":0,"allocated":2441216,"data_stored":1607157,"data_compressed":5523,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1593,"internal_metadata":64616903},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640307,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51560,"kb_used_data":2328,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320280,"statfs":{"total":96636764160,"available":96583966720,"internally_reserved":0,"allocated":2383872,"data_stored":1016789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640306,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65448,"kb_used_data":2332,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306392,"statfs":{"total":96636764160,"available":96569745408,"internally_reserved":0,"allocated":2387968,"data_stored":1016808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640309,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69656,"kb_used_data":2368,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302184,"statfs":{"total":96636764160,"available":96565436416,"internally_reserved":0,"allocated":2424832,"data_stored":1607157,"data_compressed":6382,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6382,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6382,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5523,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T08:55:54.199 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T08:55:54.497 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:55:54.498 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-05-08T08:55:54.509 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":311,"stamp":"2024-05-08T08:55:53.780431+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":1297,"num_object_clones":0,"num_object_copies":2594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1297,"num_whiteouts":1291,"num_read":513049,"num_read_kb":1628462,"num_write":26259,"num_write_kb":454255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11022,"num_bytes_recovered":423134873,"num_keys_recovered":904,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24946,"ondisk_log_size":24946,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":367776,"kb_used_data":14072,"kb_used_omap":9,"kb_used_meta":353590,"kb_avail":565863264,"statfs":{"total":579820584960,"available":579443982336,"internally_reserved":0,"allocated":14409728,"data_stored":7281508,"data_compressed":11905,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":9568,"internal_metadata":362076832},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-743,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001324"},"pg_stats":[{"pgid":"3.1f","version":"182'772","reported_seq":20838,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.562464+0000","last_change":"2024-05-08T08:55:43.388833+0000","last_active":"2024-05-08T08:55:50.562464+0000","last_peered":"2024-05-08T08:55:50.562464+0000","last_clean":"2024-05-08T08:55:50.562464+0000","last_became_active":"2024-05-08T08:55:43.386102+0000","last_became_peered":"2024-05-08T08:55:43.386102+0000","last_unstale":"2024-05-08T08:55:50.562464+0000","last_undegraded":"2024-05-08T08:55:50.562464+0000","last_fullsized":"2024-05-08T08:55:50.562464+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":772,"log_dups_size":0,"ondisk_log_size":772,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:25:19.005824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038328300000000002,"stat_sum":{"num_bytes":0,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":34,"num_read":19333,"num_read_kb":85778,"num_write":988,"num_write_kb":33105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":328,"num_bytes_recovered":19152951,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"181'922","reported_seq":11965,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.554583+0000","last_change":"2024-05-08T08:55:33.482489+0000","last_active":"2024-05-08T08:55:50.554583+0000","last_peered":"2024-05-08T08:55:50.554583+0000","last_clean":"2024-05-08T08:55:50.554583+0000","last_became_active":"2024-05-08T08:52:11.930844+0000","last_became_peered":"2024-05-08T08:52:11.930844+0000","last_unstale":"2024-05-08T08:55:50.554583+0000","last_undegraded":"2024-05-08T08:55:50.554583+0000","last_fullsized":"2024-05-08T08:55:50.554583+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":922,"log_dups_size":0,"ondisk_log_size":922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:34:27.496752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001408816,"stat_sum":{"num_bytes":0,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":49,"num_read":13744,"num_read_kb":37146,"num_write":694,"num_write_kb":12720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"181'753","reported_seq":4867,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.562479+0000","last_change":"2024-05-08T08:55:44.384669+0000","last_active":"2024-05-08T08:55:50.562479+0000","last_peered":"2024-05-08T08:55:50.562479+0000","last_clean":"2024-05-08T08:55:50.562479+0000","last_became_active":"2024-05-08T08:55:44.381069+0000","last_became_peered":"2024-05-08T08:55:44.381069+0000","last_unstale":"2024-05-08T08:55:50.562479+0000","last_undegraded":"2024-05-08T08:55:50.562479+0000","last_fullsized":"2024-05-08T08:55:50.562479+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":753,"log_dups_size":0,"ondisk_log_size":753,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:27:12.956766+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0093620049999999996,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":40,"num_read":8309,"num_read_kb":30535,"num_write":901,"num_write_kb":7282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":570,"num_bytes_recovered":16438479,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"182'740","reported_seq":19620,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.562811+0000","last_change":"2024-05-08T08:55:50.562811+0000","last_active":"2024-05-08T08:55:50.562811+0000","last_peered":"2024-05-08T08:55:50.562811+0000","last_clean":"2024-05-08T08:55:50.562811+0000","last_became_active":"2024-05-08T08:55:50.560370+0000","last_became_peered":"2024-05-08T08:55:50.560370+0000","last_unstale":"2024-05-08T08:55:50.562811+0000","last_undegraded":"2024-05-08T08:55:50.562811+0000","last_fullsized":"2024-05-08T08:55:50.562811+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":740,"log_dups_size":0,"ondisk_log_size":740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:20:09.589464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019815259999999999,"stat_sum":{"num_bytes":0,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":45,"num_read":18641,"num_read_kb":101548,"num_write":811,"num_write_kb":27861,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":609,"num_bytes_recovered":86192377,"num_keys_recovered":338,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"179'637","reported_seq":26061,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.562537+0000","last_change":"2024-05-08T08:55:43.388686+0000","last_active":"2024-05-08T08:55:50.562537+0000","last_peered":"2024-05-08T08:55:50.562537+0000","last_clean":"2024-05-08T08:55:50.562537+0000","last_became_active":"2024-05-08T08:55:43.386098+0000","last_became_peered":"2024-05-08T08:55:43.386098+0000","last_unstale":"2024-05-08T08:55:50.562537+0000","last_undegraded":"2024-05-08T08:55:50.562537+0000","last_fullsized":"2024-05-08T08:55:50.562537+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":637,"log_dups_size":0,"ondisk_log_size":637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:29:43.226913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012631949999999999,"stat_sum":{"num_bytes":0,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":50,"num_read":25112,"num_read_kb":45826,"num_write":688,"num_write_kb":23312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":455,"num_bytes_recovered":5234820,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"180'867","reported_seq":22286,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.559406+0000","last_change":"2024-05-08T08:55:33.475720+0000","last_active":"2024-05-08T08:55:50.559406+0000","last_peered":"2024-05-08T08:55:50.559406+0000","last_clean":"2024-05-08T08:55:50.559406+0000","last_became_active":"2024-05-08T08:53:24.484124+0000","last_became_peered":"2024-05-08T08:53:24.484124+0000","last_unstale":"2024-05-08T08:55:50.559406+0000","last_undegraded":"2024-05-08T08:55:50.559406+0000","last_fullsized":"2024-05-08T08:55:50.559406+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":867,"log_dups_size":0,"ondisk_log_size":867,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:57:41.478541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012989099999999999,"stat_sum":{"num_bytes":19,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":34,"num_read":24563,"num_read_kb":59887,"num_write":908,"num_write_kb":19093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":130,"num_bytes_recovered":8549042,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"178'426","reported_seq":10752,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.557157+0000","last_change":"2024-05-08T08:55:33.470036+0000","last_active":"2024-05-08T08:55:50.557157+0000","last_peered":"2024-05-08T08:55:50.557157+0000","last_clean":"2024-05-08T08:55:50.557157+0000","last_became_active":"2024-05-08T08:53:24.483403+0000","last_became_peered":"2024-05-08T08:53:24.483403+0000","last_unstale":"2024-05-08T08:55:50.557157+0000","last_undegraded":"2024-05-08T08:55:50.557157+0000","last_fullsized":"2024-05-08T08:55:50.557157+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":426,"log_dups_size":0,"ondisk_log_size":426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:23:41.607743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011263099999999999,"stat_sum":{"num_bytes":0,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":48,"num_read":9419,"num_read_kb":48048,"num_write":368,"num_write_kb":16514,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":8814624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"181'711","reported_seq":20343,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.555276+0000","last_change":"2024-05-08T08:55:41.379915+0000","last_active":"2024-05-08T08:55:50.555276+0000","last_peered":"2024-05-08T08:55:50.555276+0000","last_clean":"2024-05-08T08:55:50.555276+0000","last_became_active":"2024-05-08T08:55:41.378182+0000","last_became_peered":"2024-05-08T08:55:41.378182+0000","last_unstale":"2024-05-08T08:55:50.555276+0000","last_undegraded":"2024-05-08T08:55:50.555276+0000","last_fullsized":"2024-05-08T08:55:50.555276+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":711,"log_dups_size":0,"ondisk_log_size":711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:23:17.505431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098929300000000003,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":20665,"num_read_kb":52569,"num_write":672,"num_write_kb":23378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":4852260,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"182'852","reported_seq":19005,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.557278+0000","last_change":"2024-05-08T08:55:40.383627+0000","last_active":"2024-05-08T08:55:50.557278+0000","last_peered":"2024-05-08T08:55:50.557278+0000","last_clean":"2024-05-08T08:55:50.557278+0000","last_became_active":"2024-05-08T08:55:40.381448+0000","last_became_peered":"2024-05-08T08:55:40.381448+0000","last_unstale":"2024-05-08T08:55:50.557278+0000","last_undegraded":"2024-05-08T08:55:50.557278+0000","last_fullsized":"2024-05-08T08:55:50.557278+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:14:47.318689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018475399999999999,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":30,"num_read":20569,"num_read_kb":77197,"num_write":969,"num_write_kb":16935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":381,"num_bytes_recovered":5593379,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"182'1461","reported_seq":21996,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.557382+0000","last_change":"2024-05-08T08:55:44.381999+0000","last_active":"2024-05-08T08:55:50.557382+0000","last_peered":"2024-05-08T08:55:50.557382+0000","last_clean":"2024-05-08T08:55:50.557382+0000","last_became_active":"2024-05-08T08:55:44.379555+0000","last_became_peered":"2024-05-08T08:55:44.379555+0000","last_unstale":"2024-05-08T08:55:50.557382+0000","last_undegraded":"2024-05-08T08:55:50.557382+0000","last_fullsized":"2024-05-08T08:55:50.557382+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":1461,"log_dups_size":0,"ondisk_log_size":1461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:07:01.906180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0045105479999999996,"stat_sum":{"num_bytes":0,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":33,"num_read":25480,"num_read_kb":87263,"num_write":2254,"num_write_kb":27190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":420,"num_bytes_recovered":42698110,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"182'690","reported_seq":17152,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.555076+0000","last_change":"2024-05-08T08:55:42.384249+0000","last_active":"2024-05-08T08:55:50.555076+0000","last_peered":"2024-05-08T08:55:50.555076+0000","last_clean":"2024-05-08T08:55:50.555076+0000","last_became_active":"2024-05-08T08:55:42.382974+0000","last_became_peered":"2024-05-08T08:55:42.382974+0000","last_unstale":"2024-05-08T08:55:50.555076+0000","last_undegraded":"2024-05-08T08:55:50.555076+0000","last_fullsized":"2024-05-08T08:55:50.555076+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":690,"log_dups_size":0,"ondisk_log_size":690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:12:41.796610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029835819999999998,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":39,"num_read":17511,"num_read_kb":53002,"num_write":778,"num_write_kb":8761,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":284,"num_bytes_recovered":1296934,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"180'733","reported_seq":12777,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.555182+0000","last_change":"2024-05-08T08:55:33.469235+0000","last_active":"2024-05-08T08:55:50.555182+0000","last_peered":"2024-05-08T08:55:50.555182+0000","last_clean":"2024-05-08T08:55:50.555182+0000","last_became_active":"2024-05-08T08:53:23.481362+0000","last_became_peered":"2024-05-08T08:53:23.481362+0000","last_unstale":"2024-05-08T08:55:50.555182+0000","last_undegraded":"2024-05-08T08:55:50.555182+0000","last_fullsized":"2024-05-08T08:55:50.555182+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":733,"log_dups_size":0,"ondisk_log_size":733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:15:25.796542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032028149999999999,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":30,"num_read":10890,"num_read_kb":33288,"num_write":663,"num_write_kb":11454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":3612768,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":384,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.555233+0000","last_change":"2024-05-08T08:53:20.362387+0000","last_active":"2024-05-08T08:55:50.555233+0000","last_peered":"2024-05-08T08:55:50.555233+0000","last_clean":"2024-05-08T08:55:50.555233+0000","last_became_active":"2024-05-08T08:53:20.362113+0000","last_became_peered":"2024-05-08T08:53:20.362113+0000","last_unstale":"2024-05-08T08:55:50.555233+0000","last_undegraded":"2024-05-08T08:55:50.555233+0000","last_fullsized":"2024-05-08T08:55:50.555233+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:28:25.204125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"182'652","reported_seq":20646,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.562407+0000","last_change":"2024-05-08T08:55:40.380225+0000","last_active":"2024-05-08T08:55:50.562407+0000","last_peered":"2024-05-08T08:55:50.562407+0000","last_clean":"2024-05-08T08:55:50.562407+0000","last_became_active":"2024-05-08T08:55:40.378157+0000","last_became_peered":"2024-05-08T08:55:40.378157+0000","last_unstale":"2024-05-08T08:55:50.562407+0000","last_undegraded":"2024-05-08T08:55:50.562407+0000","last_fullsized":"2024-05-08T08:55:50.562407+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":652,"log_dups_size":0,"ondisk_log_size":652,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:03:49.159963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019021660000000001,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":40,"num_read":21842,"num_read_kb":88722,"num_write":580,"num_write_kb":17543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":288,"num_bytes_recovered":3368960,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":375,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.557262+0000","last_change":"2024-05-08T08:52:03.896159+0000","last_active":"2024-05-08T08:55:50.557262+0000","last_peered":"2024-05-08T08:55:50.557262+0000","last_clean":"2024-05-08T08:55:50.557262+0000","last_became_active":"2024-05-08T08:52:03.895797+0000","last_became_peered":"2024-05-08T08:52:03.895797+0000","last_unstale":"2024-05-08T08:55:50.557262+0000","last_undegraded":"2024-05-08T08:55:50.557262+0000","last_fullsized":"2024-05-08T08:55:50.557262+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:34:33.513525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"180'518","reported_seq":6927,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.554715+0000","last_change":"2024-05-08T08:55:33.483261+0000","last_active":"2024-05-08T08:55:50.554715+0000","last_peered":"2024-05-08T08:55:50.554715+0000","last_clean":"2024-05-08T08:55:50.554715+0000","last_became_active":"2024-05-08T08:55:08.437577+0000","last_became_peered":"2024-05-08T08:55:08.437577+0000","last_unstale":"2024-05-08T08:55:50.554715+0000","last_undegraded":"2024-05-08T08:55:50.554715+0000","last_fullsized":"2024-05-08T08:55:50.554715+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:55:04.162036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017459470000000001,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":39,"num_read":6769,"num_read_kb":34542,"num_write":419,"num_write_kb":12024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":324,"num_bytes_recovered":15121731,"num_keys_recovered":25,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":379,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.555279+0000","last_change":"2024-05-08T08:54:59.398530+0000","last_active":"2024-05-08T08:55:50.555279+0000","last_peered":"2024-05-08T08:55:50.555279+0000","last_clean":"2024-05-08T08:55:50.555279+0000","last_became_active":"2024-05-08T08:54:59.398181+0000","last_became_peered":"2024-05-08T08:54:59.398181+0000","last_unstale":"2024-05-08T08:55:50.555279+0000","last_undegraded":"2024-05-08T08:55:50.555279+0000","last_fullsized":"2024-05-08T08:55:50.555279+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:24:28.264614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"181'863","reported_seq":27577,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.557404+0000","last_change":"2024-05-08T08:55:43.386319+0000","last_active":"2024-05-08T08:55:50.557404+0000","last_peered":"2024-05-08T08:55:50.557404+0000","last_clean":"2024-05-08T08:55:50.557404+0000","last_became_active":"2024-05-08T08:55:43.384081+0000","last_became_peered":"2024-05-08T08:55:43.384081+0000","last_unstale":"2024-05-08T08:55:50.557404+0000","last_undegraded":"2024-05-08T08:55:50.557404+0000","last_fullsized":"2024-05-08T08:55:50.557404+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":863,"log_dups_size":0,"ondisk_log_size":863,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:02:20.851342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014858339999999999,"stat_sum":{"num_bytes":0,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":48,"num_read":22624,"num_read_kb":52229,"num_write":653,"num_write_kb":12062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":791,"num_bytes_recovered":28697088,"num_keys_recovered":110,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":385,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.557499+0000","last_change":"2024-05-08T08:55:37.500332+0000","last_active":"2024-05-08T08:55:50.557499+0000","last_peered":"2024-05-08T08:55:50.557499+0000","last_clean":"2024-05-08T08:55:50.557499+0000","last_became_active":"2024-05-08T08:55:37.499784+0000","last_became_peered":"2024-05-08T08:55:37.499784+0000","last_unstale":"2024-05-08T08:55:50.557499+0000","last_undegraded":"2024-05-08T08:55:50.557499+0000","last_fullsized":"2024-05-08T08:55:50.557499+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:53:12.058683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"181'946","reported_seq":14829,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.562555+0000","last_change":"2024-05-08T08:55:50.562555+0000","last_active":"2024-05-08T08:55:50.562555+0000","last_peered":"2024-05-08T08:55:50.562555+0000","last_clean":"2024-05-08T08:55:50.562555+0000","last_became_active":"2024-05-08T08:55:50.559883+0000","last_became_peered":"2024-05-08T08:55:50.559883+0000","last_unstale":"2024-05-08T08:55:50.562555+0000","last_undegraded":"2024-05-08T08:55:50.562555+0000","last_fullsized":"2024-05-08T08:55:50.562555+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":946,"log_dups_size":0,"ondisk_log_size":946,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:18:55.639194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013854900000000001,"stat_sum":{"num_bytes":0,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":55,"num_read":15316,"num_read_kb":32869,"num_write":1141,"num_write_kb":10612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":494,"num_bytes_recovered":13827825,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"182'817","reported_seq":12316,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.559335+0000","last_change":"2024-05-08T08:55:41.381790+0000","last_active":"2024-05-08T08:55:50.559335+0000","last_peered":"2024-05-08T08:55:50.559335+0000","last_clean":"2024-05-08T08:55:50.559335+0000","last_became_active":"2024-05-08T08:55:41.380391+0000","last_became_peered":"2024-05-08T08:55:41.380391+0000","last_unstale":"2024-05-08T08:55:50.559335+0000","last_undegraded":"2024-05-08T08:55:50.559335+0000","last_fullsized":"2024-05-08T08:55:50.559335+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":817,"log_dups_size":0,"ondisk_log_size":817,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:59:49.412777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002033212,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":34,"num_read":11854,"num_read_kb":33576,"num_write":867,"num_write_kb":6636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":383,"num_bytes_recovered":9012422,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":353,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.560520+0000","last_change":"2024-05-08T08:55:50.560520+0000","last_active":"2024-05-08T08:55:50.560520+0000","last_peered":"2024-05-08T08:55:50.560520+0000","last_clean":"2024-05-08T08:55:50.560520+0000","last_became_active":"2024-05-08T08:55:50.559612+0000","last_became_peered":"2024-05-08T08:55:50.559612+0000","last_unstale":"2024-05-08T08:55:50.560520+0000","last_undegraded":"2024-05-08T08:55:50.560520+0000","last_fullsized":"2024-05-08T08:55:50.560520+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:51:19.494663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"181'729","reported_seq":14229,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.558114+0000","last_change":"2024-05-08T08:55:33.470800+0000","last_active":"2024-05-08T08:55:50.558114+0000","last_peered":"2024-05-08T08:55:50.558114+0000","last_clean":"2024-05-08T08:55:50.558114+0000","last_became_active":"2024-05-08T08:52:11.928301+0000","last_became_peered":"2024-05-08T08:52:11.928301+0000","last_unstale":"2024-05-08T08:55:50.558114+0000","last_undegraded":"2024-05-08T08:55:50.558114+0000","last_fullsized":"2024-05-08T08:55:50.558114+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":729,"log_dups_size":0,"ondisk_log_size":729,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:03:13.796981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015902449999999999,"stat_sum":{"num_bytes":0,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":37,"num_read":10704,"num_read_kb":44099,"num_write":654,"num_write_kb":16306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":375,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.555092+0000","last_change":"2024-05-08T08:52:03.891768+0000","last_active":"2024-05-08T08:55:50.555092+0000","last_peered":"2024-05-08T08:55:50.555092+0000","last_clean":"2024-05-08T08:55:50.555092+0000","last_became_active":"2024-05-08T08:52:03.891374+0000","last_became_peered":"2024-05-08T08:52:03.891374+0000","last_unstale":"2024-05-08T08:55:50.555092+0000","last_undegraded":"2024-05-08T08:55:50.555092+0000","last_fullsized":"2024-05-08T08:55:50.555092+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:18:10.240492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"182'1161","reported_seq":28197,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.562344+0000","last_change":"2024-05-08T08:55:42.388348+0000","last_active":"2024-05-08T08:55:50.562344+0000","last_peered":"2024-05-08T08:55:50.562344+0000","last_clean":"2024-05-08T08:55:50.562344+0000","last_became_active":"2024-05-08T08:55:42.386239+0000","last_became_peered":"2024-05-08T08:55:42.386239+0000","last_unstale":"2024-05-08T08:55:50.562344+0000","last_undegraded":"2024-05-08T08:55:50.562344+0000","last_fullsized":"2024-05-08T08:55:50.562344+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":1161,"log_dups_size":0,"ondisk_log_size":1161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:06:05.510596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086553400000000005,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":31078,"num_read_kb":56218,"num_write":870,"num_write_kb":9640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":490,"num_bytes_recovered":5023153,"num_keys_recovered":69,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":374,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.554962+0000","last_change":"2024-05-08T08:52:03.891551+0000","last_active":"2024-05-08T08:55:50.554962+0000","last_peered":"2024-05-08T08:55:50.554962+0000","last_clean":"2024-05-08T08:55:50.554962+0000","last_became_active":"2024-05-08T08:52:03.891296+0000","last_became_peered":"2024-05-08T08:52:03.891296+0000","last_unstale":"2024-05-08T08:55:50.554962+0000","last_undegraded":"2024-05-08T08:55:50.554962+0000","last_fullsized":"2024-05-08T08:55:50.554962+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:54.806343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"181'1002","reported_seq":15538,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.557496+0000","last_change":"2024-05-08T08:55:43.382935+0000","last_active":"2024-05-08T08:55:50.557496+0000","last_peered":"2024-05-08T08:55:50.557496+0000","last_clean":"2024-05-08T08:55:50.557496+0000","last_became_active":"2024-05-08T08:55:43.381345+0000","last_became_peered":"2024-05-08T08:55:43.381345+0000","last_unstale":"2024-05-08T08:55:50.557496+0000","last_undegraded":"2024-05-08T08:55:50.557496+0000","last_fullsized":"2024-05-08T08:55:50.557496+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":1002,"log_dups_size":0,"ondisk_log_size":1002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:30:03.146595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012464850000000001,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":39,"num_read":15521,"num_read_kb":63440,"num_write":1271,"num_write_kb":13979,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":635,"num_bytes_recovered":18405521,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":349,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.560138+0000","last_change":"2024-05-08T08:55:50.560138+0000","last_active":"2024-05-08T08:55:50.560138+0000","last_peered":"2024-05-08T08:55:50.560138+0000","last_clean":"2024-05-08T08:55:50.560138+0000","last_became_active":"2024-05-08T08:55:50.559815+0000","last_became_peered":"2024-05-08T08:55:50.559815+0000","last_unstale":"2024-05-08T08:55:50.560138+0000","last_undegraded":"2024-05-08T08:55:50.560138+0000","last_fullsized":"2024-05-08T08:55:50.560138+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:32:21.109548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":556,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.555092+0000","last_change":"2024-05-08T08:55:42.381739+0000","last_active":"2024-05-08T08:55:50.555092+0000","last_peered":"2024-05-08T08:55:50.555092+0000","last_clean":"2024-05-08T08:55:50.555092+0000","last_became_active":"2024-05-08T08:55:42.381316+0000","last_became_peered":"2024-05-08T08:55:42.381316+0000","last_unstale":"2024-05-08T08:55:50.555092+0000","last_undegraded":"2024-05-08T08:55:50.555092+0000","last_fullsized":"2024-05-08T08:55:50.555092+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:51:57.849666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:51:57.849666+0000","last_clean_scrub_stamp":"2024-05-08T08:51:57.849666+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:59:25.417866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"182'398","reported_seq":9838,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.555377+0000","last_change":"2024-05-08T08:55:33.481972+0000","last_active":"2024-05-08T08:55:50.555377+0000","last_peered":"2024-05-08T08:55:50.555377+0000","last_clean":"2024-05-08T08:55:50.555377+0000","last_became_active":"2024-05-08T08:52:11.926975+0000","last_became_peered":"2024-05-08T08:52:11.926975+0000","last_unstale":"2024-05-08T08:55:50.555377+0000","last_undegraded":"2024-05-08T08:55:50.555377+0000","last_fullsized":"2024-05-08T08:55:50.555377+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":398,"log_dups_size":0,"ondisk_log_size":398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:13:12.117653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087585099999999997,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":35,"num_read":8901,"num_read_kb":36556,"num_write":340,"num_write_kb":8985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"180'503","reported_seq":14561,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.559353+0000","last_change":"2024-05-08T08:55:33.475165+0000","last_active":"2024-05-08T08:55:50.559353+0000","last_peered":"2024-05-08T08:55:50.559353+0000","last_clean":"2024-05-08T08:55:50.559353+0000","last_became_active":"2024-05-08T08:53:23.478564+0000","last_became_peered":"2024-05-08T08:53:23.478564+0000","last_unstale":"2024-05-08T08:55:50.559353+0000","last_undegraded":"2024-05-08T08:55:50.559353+0000","last_fullsized":"2024-05-08T08:55:50.559353+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":503,"log_dups_size":0,"ondisk_log_size":503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:32:37.241705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082315600000000002,"stat_sum":{"num_bytes":0,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":42,"num_read":14198,"num_read_kb":62213,"num_write":413,"num_write_kb":16566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":131,"num_bytes_recovered":12377088,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"182'1247","reported_seq":11910,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.554993+0000","last_change":"2024-05-08T08:55:33.468391+0000","last_active":"2024-05-08T08:55:50.554993+0000","last_peered":"2024-05-08T08:55:50.554993+0000","last_clean":"2024-05-08T08:55:50.554993+0000","last_became_active":"2024-05-08T08:53:24.480359+0000","last_became_peered":"2024-05-08T08:53:24.480359+0000","last_unstale":"2024-05-08T08:55:50.554993+0000","last_undegraded":"2024-05-08T08:55:50.554993+0000","last_fullsized":"2024-05-08T08:55:50.554993+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":1247,"log_dups_size":0,"ondisk_log_size":1247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:08:14.677080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025209339999999998,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":40,"num_read":15863,"num_read_kb":29509,"num_write":1451,"num_write_kb":6331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":2175343,"num_keys_recovered":61,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"180'628","reported_seq":12516,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.557293+0000","last_change":"2024-05-08T08:55:42.384129+0000","last_active":"2024-05-08T08:55:50.557293+0000","last_peered":"2024-05-08T08:55:50.557293+0000","last_clean":"2024-05-08T08:55:50.557293+0000","last_became_active":"2024-05-08T08:55:42.382485+0000","last_became_peered":"2024-05-08T08:55:42.382485+0000","last_unstale":"2024-05-08T08:55:50.557293+0000","last_undegraded":"2024-05-08T08:55:50.557293+0000","last_fullsized":"2024-05-08T08:55:50.557293+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":628,"log_dups_size":0,"ondisk_log_size":628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:10:53.437543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001692783,"stat_sum":{"num_bytes":0,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":50,"num_read":11950,"num_read_kb":39515,"num_write":513,"num_write_kb":7725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":332,"num_bytes_recovered":2080800,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"180'378","reported_seq":14238,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.565133+0000","last_change":"2024-05-08T08:55:50.565133+0000","last_active":"2024-05-08T08:55:50.565133+0000","last_peered":"2024-05-08T08:55:50.565133+0000","last_clean":"2024-05-08T08:55:50.565133+0000","last_became_active":"2024-05-08T08:55:50.563411+0000","last_became_peered":"2024-05-08T08:55:50.563411+0000","last_unstale":"2024-05-08T08:55:50.565133+0000","last_undegraded":"2024-05-08T08:55:50.565133+0000","last_fullsized":"2024-05-08T08:55:50.565133+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:22:13.206310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020039379999999998,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":40,"num_read":13099,"num_read_kb":41266,"num_write":323,"num_write_kb":7340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":509,"num_bytes_recovered":9113137,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"180'478","reported_seq":8966,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.555126+0000","last_change":"2024-05-08T08:55:33.469549+0000","last_active":"2024-05-08T08:55:50.555126+0000","last_peered":"2024-05-08T08:55:50.555126+0000","last_clean":"2024-05-08T08:55:50.555126+0000","last_became_active":"2024-05-08T08:52:11.932594+0000","last_became_peered":"2024-05-08T08:52:11.932594+0000","last_unstale":"2024-05-08T08:55:50.555126+0000","last_undegraded":"2024-05-08T08:55:50.555126+0000","last_fullsized":"2024-05-08T08:55:50.555126+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":478,"log_dups_size":0,"ondisk_log_size":478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:53:33.929731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033757140000000001,"stat_sum":{"num_bytes":0,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":43,"num_read":9572,"num_read_kb":34328,"num_write":402,"num_write_kb":10002,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"180'632","reported_seq":9873,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.562760+0000","last_change":"2024-05-08T08:55:50.562760+0000","last_active":"2024-05-08T08:55:50.562760+0000","last_peered":"2024-05-08T08:55:50.562760+0000","last_clean":"2024-05-08T08:55:50.562760+0000","last_became_active":"2024-05-08T08:55:50.560907+0000","last_became_peered":"2024-05-08T08:55:50.560907+0000","last_unstale":"2024-05-08T08:55:50.562760+0000","last_undegraded":"2024-05-08T08:55:50.562760+0000","last_fullsized":"2024-05-08T08:55:50.562760+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":632,"log_dups_size":0,"ondisk_log_size":632,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:04:24.220863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002456153,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":16215,"num_read_kb":47213,"num_write":634,"num_write_kb":12225,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":446,"num_bytes_recovered":28147344,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"182'588","reported_seq":8774,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.557417+0000","last_change":"2024-05-08T08:55:40.386164+0000","last_active":"2024-05-08T08:55:50.557417+0000","last_peered":"2024-05-08T08:55:50.557417+0000","last_clean":"2024-05-08T08:55:50.557417+0000","last_became_active":"2024-05-08T08:55:40.383962+0000","last_became_peered":"2024-05-08T08:55:40.383962+0000","last_unstale":"2024-05-08T08:55:50.557417+0000","last_undegraded":"2024-05-08T08:55:50.557417+0000","last_fullsized":"2024-05-08T08:55:50.557417+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":588,"log_dups_size":0,"ondisk_log_size":588,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:48:37.179607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037436610000000001,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":36,"num_read":8886,"num_read_kb":38380,"num_write":568,"num_write_kb":9633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":675,"num_bytes_recovered":18815289,"num_keys_recovered":79,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"180'749","reported_seq":8638,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.567799+0000","last_change":"2024-05-08T08:55:50.567799+0000","last_active":"2024-05-08T08:55:50.567799+0000","last_peered":"2024-05-08T08:55:50.567799+0000","last_clean":"2024-05-08T08:55:50.567799+0000","last_became_active":"2024-05-08T08:55:50.565716+0000","last_became_peered":"2024-05-08T08:55:50.565716+0000","last_unstale":"2024-05-08T08:55:50.567799+0000","last_undegraded":"2024-05-08T08:55:50.567799+0000","last_fullsized":"2024-05-08T08:55:50.567799+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":749,"log_dups_size":0,"ondisk_log_size":749,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:28:22.726479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095127799999999995,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":40,"num_read":9090,"num_read_kb":41259,"num_write":984,"num_write_kb":14557,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":541,"num_bytes_recovered":26549987,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"182'750","reported_seq":11879,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.554980+0000","last_change":"2024-05-08T08:55:33.467196+0000","last_active":"2024-05-08T08:55:50.554980+0000","last_peered":"2024-05-08T08:55:50.554980+0000","last_clean":"2024-05-08T08:55:50.554980+0000","last_became_active":"2024-05-08T08:53:24.479891+0000","last_became_peered":"2024-05-08T08:53:24.479891+0000","last_unstale":"2024-05-08T08:55:50.554980+0000","last_undegraded":"2024-05-08T08:55:50.554980+0000","last_fullsized":"2024-05-08T08:55:50.554980+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":750,"log_dups_size":0,"ondisk_log_size":750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:11:35.412500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015502269999999999,"stat_sum":{"num_bytes":0,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":43,"num_read":13241,"num_read_kb":36544,"num_write":586,"num_write_kb":10920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":131,"num_bytes_recovered":2604032,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"182'1575","reported_seq":25758,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.562352+0000","last_change":"2024-05-08T08:55:43.389357+0000","last_active":"2024-05-08T08:55:50.562352+0000","last_peered":"2024-05-08T08:55:50.562352+0000","last_clean":"2024-05-08T08:55:50.562352+0000","last_became_active":"2024-05-08T08:55:43.386543+0000","last_became_peered":"2024-05-08T08:55:43.386543+0000","last_unstale":"2024-05-08T08:55:50.562352+0000","last_undegraded":"2024-05-08T08:55:50.562352+0000","last_fullsized":"2024-05-08T08:55:50.562352+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":1575,"log_dups_size":0,"ondisk_log_size":1575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:38:01.859312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015403719999999999,"stat_sum":{"num_bytes":0,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":52,"num_read":30937,"num_read_kb":62696,"num_write":2139,"num_write_kb":12077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":330,"num_bytes_recovered":4235412,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"181'651","reported_seq":11695,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.555086+0000","last_change":"2024-05-08T08:55:43.380556+0000","last_active":"2024-05-08T08:55:50.555086+0000","last_peered":"2024-05-08T08:55:50.555086+0000","last_clean":"2024-05-08T08:55:50.555086+0000","last_became_active":"2024-05-08T08:55:43.379146+0000","last_became_peered":"2024-05-08T08:55:43.379146+0000","last_unstale":"2024-05-08T08:55:50.555086+0000","last_undegraded":"2024-05-08T08:55:50.555086+0000","last_fullsized":"2024-05-08T08:55:50.555086+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":651,"log_dups_size":0,"ondisk_log_size":651,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:09:48.532237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022701349999999999,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":40,"num_read":11037,"num_read_kb":41101,"num_write":598,"num_write_kb":9071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":589,"num_bytes_recovered":19963261,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":1295,"num_object_clones":0,"num_object_copies":2590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1295,"num_whiteouts":1291,"num_read":512933,"num_read_kb":1628362,"num_write":26100,"num_write_kb":451839,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11018,"num_bytes_recovered":421954137,"num_keys_recovered":904,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24829,"ondisk_log_size":24829,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":18287,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640306,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59884,"kb_used_data":2328,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311956,"statfs":{"total":96636764160,"available":96575442944,"internally_reserved":0,"allocated":2383872,"data_stored":1016789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640310,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55724,"kb_used_data":2332,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316116,"statfs":{"total":96636764160,"available":96579702784,"internally_reserved":0,"allocated":2387968,"data_stored":1016808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640308,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65504,"kb_used_data":2384,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306336,"statfs":{"total":96636764160,"available":96569688064,"internally_reserved":0,"allocated":2441216,"data_stored":1607157,"data_compressed":5523,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1593,"internal_metadata":64616903},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640307,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51560,"kb_used_data":2328,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320280,"statfs":{"total":96636764160,"available":96583966720,"internally_reserved":0,"allocated":2383872,"data_stored":1016789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640306,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65448,"kb_used_data":2332,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306392,"statfs":{"total":96636764160,"available":96569745408,"internally_reserved":0,"allocated":2387968,"data_stored":1016808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640309,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69656,"kb_used_data":2368,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302184,"statfs":{"total":96636764160,"available":96565436416,"internally_reserved":0,"allocated":2424832,"data_stored":1607157,"data_compressed":6382,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6382,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6382,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5523,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T08:55:54.511 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-05-08T08:55:54.512 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-05-08T08:55:54.831 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-05-08T08:55:54.842 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-08T08:55:54.842 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T08:55:55.142 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:55:55.142 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-05-08T08:55:55.154 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":311,"stamp":"2024-05-08T08:55:53.780431+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":1297,"num_object_clones":0,"num_object_copies":2594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1297,"num_whiteouts":1291,"num_read":513049,"num_read_kb":1628462,"num_write":26259,"num_write_kb":454255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11022,"num_bytes_recovered":423134873,"num_keys_recovered":904,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24946,"ondisk_log_size":24946,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":367776,"kb_used_data":14072,"kb_used_omap":9,"kb_used_meta":353590,"kb_avail":565863264,"statfs":{"total":579820584960,"available":579443982336,"internally_reserved":0,"allocated":14409728,"data_stored":7281508,"data_compressed":11905,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":9568,"internal_metadata":362076832},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-743,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001324"},"pg_stats":[{"pgid":"3.1f","version":"182'772","reported_seq":20838,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.562464+0000","last_change":"2024-05-08T08:55:43.388833+0000","last_active":"2024-05-08T08:55:50.562464+0000","last_peered":"2024-05-08T08:55:50.562464+0000","last_clean":"2024-05-08T08:55:50.562464+0000","last_became_active":"2024-05-08T08:55:43.386102+0000","last_became_peered":"2024-05-08T08:55:43.386102+0000","last_unstale":"2024-05-08T08:55:50.562464+0000","last_undegraded":"2024-05-08T08:55:50.562464+0000","last_fullsized":"2024-05-08T08:55:50.562464+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":772,"log_dups_size":0,"ondisk_log_size":772,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:25:19.005824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038328300000000002,"stat_sum":{"num_bytes":0,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":34,"num_read":19333,"num_read_kb":85778,"num_write":988,"num_write_kb":33105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":328,"num_bytes_recovered":19152951,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"181'922","reported_seq":11965,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.554583+0000","last_change":"2024-05-08T08:55:33.482489+0000","last_active":"2024-05-08T08:55:50.554583+0000","last_peered":"2024-05-08T08:55:50.554583+0000","last_clean":"2024-05-08T08:55:50.554583+0000","last_became_active":"2024-05-08T08:52:11.930844+0000","last_became_peered":"2024-05-08T08:52:11.930844+0000","last_unstale":"2024-05-08T08:55:50.554583+0000","last_undegraded":"2024-05-08T08:55:50.554583+0000","last_fullsized":"2024-05-08T08:55:50.554583+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":922,"log_dups_size":0,"ondisk_log_size":922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:34:27.496752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001408816,"stat_sum":{"num_bytes":0,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":49,"num_read":13744,"num_read_kb":37146,"num_write":694,"num_write_kb":12720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"181'753","reported_seq":4867,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.562479+0000","last_change":"2024-05-08T08:55:44.384669+0000","last_active":"2024-05-08T08:55:50.562479+0000","last_peered":"2024-05-08T08:55:50.562479+0000","last_clean":"2024-05-08T08:55:50.562479+0000","last_became_active":"2024-05-08T08:55:44.381069+0000","last_became_peered":"2024-05-08T08:55:44.381069+0000","last_unstale":"2024-05-08T08:55:50.562479+0000","last_undegraded":"2024-05-08T08:55:50.562479+0000","last_fullsized":"2024-05-08T08:55:50.562479+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":753,"log_dups_size":0,"ondisk_log_size":753,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:27:12.956766+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0093620049999999996,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":40,"num_read":8309,"num_read_kb":30535,"num_write":901,"num_write_kb":7282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":570,"num_bytes_recovered":16438479,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"182'740","reported_seq":19620,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.562811+0000","last_change":"2024-05-08T08:55:50.562811+0000","last_active":"2024-05-08T08:55:50.562811+0000","last_peered":"2024-05-08T08:55:50.562811+0000","last_clean":"2024-05-08T08:55:50.562811+0000","last_became_active":"2024-05-08T08:55:50.560370+0000","last_became_peered":"2024-05-08T08:55:50.560370+0000","last_unstale":"2024-05-08T08:55:50.562811+0000","last_undegraded":"2024-05-08T08:55:50.562811+0000","last_fullsized":"2024-05-08T08:55:50.562811+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":740,"log_dups_size":0,"ondisk_log_size":740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:20:09.589464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019815259999999999,"stat_sum":{"num_bytes":0,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":45,"num_read":18641,"num_read_kb":101548,"num_write":811,"num_write_kb":27861,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":609,"num_bytes_recovered":86192377,"num_keys_recovered":338,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"179'637","reported_seq":26061,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.562537+0000","last_change":"2024-05-08T08:55:43.388686+0000","last_active":"2024-05-08T08:55:50.562537+0000","last_peered":"2024-05-08T08:55:50.562537+0000","last_clean":"2024-05-08T08:55:50.562537+0000","last_became_active":"2024-05-08T08:55:43.386098+0000","last_became_peered":"2024-05-08T08:55:43.386098+0000","last_unstale":"2024-05-08T08:55:50.562537+0000","last_undegraded":"2024-05-08T08:55:50.562537+0000","last_fullsized":"2024-05-08T08:55:50.562537+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":637,"log_dups_size":0,"ondisk_log_size":637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:29:43.226913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012631949999999999,"stat_sum":{"num_bytes":0,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":50,"num_read":25112,"num_read_kb":45826,"num_write":688,"num_write_kb":23312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":455,"num_bytes_recovered":5234820,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"180'867","reported_seq":22286,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.559406+0000","last_change":"2024-05-08T08:55:33.475720+0000","last_active":"2024-05-08T08:55:50.559406+0000","last_peered":"2024-05-08T08:55:50.559406+0000","last_clean":"2024-05-08T08:55:50.559406+0000","last_became_active":"2024-05-08T08:53:24.484124+0000","last_became_peered":"2024-05-08T08:53:24.484124+0000","last_unstale":"2024-05-08T08:55:50.559406+0000","last_undegraded":"2024-05-08T08:55:50.559406+0000","last_fullsized":"2024-05-08T08:55:50.559406+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":867,"log_dups_size":0,"ondisk_log_size":867,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:57:41.478541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012989099999999999,"stat_sum":{"num_bytes":19,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":34,"num_read":24563,"num_read_kb":59887,"num_write":908,"num_write_kb":19093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":130,"num_bytes_recovered":8549042,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"178'426","reported_seq":10752,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.557157+0000","last_change":"2024-05-08T08:55:33.470036+0000","last_active":"2024-05-08T08:55:50.557157+0000","last_peered":"2024-05-08T08:55:50.557157+0000","last_clean":"2024-05-08T08:55:50.557157+0000","last_became_active":"2024-05-08T08:53:24.483403+0000","last_became_peered":"2024-05-08T08:53:24.483403+0000","last_unstale":"2024-05-08T08:55:50.557157+0000","last_undegraded":"2024-05-08T08:55:50.557157+0000","last_fullsized":"2024-05-08T08:55:50.557157+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":426,"log_dups_size":0,"ondisk_log_size":426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:23:41.607743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011263099999999999,"stat_sum":{"num_bytes":0,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":48,"num_read":9419,"num_read_kb":48048,"num_write":368,"num_write_kb":16514,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":8814624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"181'711","reported_seq":20343,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.555276+0000","last_change":"2024-05-08T08:55:41.379915+0000","last_active":"2024-05-08T08:55:50.555276+0000","last_peered":"2024-05-08T08:55:50.555276+0000","last_clean":"2024-05-08T08:55:50.555276+0000","last_became_active":"2024-05-08T08:55:41.378182+0000","last_became_peered":"2024-05-08T08:55:41.378182+0000","last_unstale":"2024-05-08T08:55:50.555276+0000","last_undegraded":"2024-05-08T08:55:50.555276+0000","last_fullsized":"2024-05-08T08:55:50.555276+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":711,"log_dups_size":0,"ondisk_log_size":711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:23:17.505431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098929300000000003,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":20665,"num_read_kb":52569,"num_write":672,"num_write_kb":23378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":4852260,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"182'852","reported_seq":19005,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.557278+0000","last_change":"2024-05-08T08:55:40.383627+0000","last_active":"2024-05-08T08:55:50.557278+0000","last_peered":"2024-05-08T08:55:50.557278+0000","last_clean":"2024-05-08T08:55:50.557278+0000","last_became_active":"2024-05-08T08:55:40.381448+0000","last_became_peered":"2024-05-08T08:55:40.381448+0000","last_unstale":"2024-05-08T08:55:50.557278+0000","last_undegraded":"2024-05-08T08:55:50.557278+0000","last_fullsized":"2024-05-08T08:55:50.557278+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:14:47.318689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018475399999999999,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":30,"num_read":20569,"num_read_kb":77197,"num_write":969,"num_write_kb":16935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":381,"num_bytes_recovered":5593379,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"182'1461","reported_seq":21996,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.557382+0000","last_change":"2024-05-08T08:55:44.381999+0000","last_active":"2024-05-08T08:55:50.557382+0000","last_peered":"2024-05-08T08:55:50.557382+0000","last_clean":"2024-05-08T08:55:50.557382+0000","last_became_active":"2024-05-08T08:55:44.379555+0000","last_became_peered":"2024-05-08T08:55:44.379555+0000","last_unstale":"2024-05-08T08:55:50.557382+0000","last_undegraded":"2024-05-08T08:55:50.557382+0000","last_fullsized":"2024-05-08T08:55:50.557382+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":1461,"log_dups_size":0,"ondisk_log_size":1461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:07:01.906180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0045105479999999996,"stat_sum":{"num_bytes":0,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":33,"num_read":25480,"num_read_kb":87263,"num_write":2254,"num_write_kb":27190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":420,"num_bytes_recovered":42698110,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"182'690","reported_seq":17152,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.555076+0000","last_change":"2024-05-08T08:55:42.384249+0000","last_active":"2024-05-08T08:55:50.555076+0000","last_peered":"2024-05-08T08:55:50.555076+0000","last_clean":"2024-05-08T08:55:50.555076+0000","last_became_active":"2024-05-08T08:55:42.382974+0000","last_became_peered":"2024-05-08T08:55:42.382974+0000","last_unstale":"2024-05-08T08:55:50.555076+0000","last_undegraded":"2024-05-08T08:55:50.555076+0000","last_fullsized":"2024-05-08T08:55:50.555076+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":690,"log_dups_size":0,"ondisk_log_size":690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:12:41.796610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029835819999999998,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":39,"num_read":17511,"num_read_kb":53002,"num_write":778,"num_write_kb":8761,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":284,"num_bytes_recovered":1296934,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"180'733","reported_seq":12777,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.555182+0000","last_change":"2024-05-08T08:55:33.469235+0000","last_active":"2024-05-08T08:55:50.555182+0000","last_peered":"2024-05-08T08:55:50.555182+0000","last_clean":"2024-05-08T08:55:50.555182+0000","last_became_active":"2024-05-08T08:53:23.481362+0000","last_became_peered":"2024-05-08T08:53:23.481362+0000","last_unstale":"2024-05-08T08:55:50.555182+0000","last_undegraded":"2024-05-08T08:55:50.555182+0000","last_fullsized":"2024-05-08T08:55:50.555182+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":733,"log_dups_size":0,"ondisk_log_size":733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:15:25.796542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032028149999999999,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":30,"num_read":10890,"num_read_kb":33288,"num_write":663,"num_write_kb":11454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":3612768,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":384,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.555233+0000","last_change":"2024-05-08T08:53:20.362387+0000","last_active":"2024-05-08T08:55:50.555233+0000","last_peered":"2024-05-08T08:55:50.555233+0000","last_clean":"2024-05-08T08:55:50.555233+0000","last_became_active":"2024-05-08T08:53:20.362113+0000","last_became_peered":"2024-05-08T08:53:20.362113+0000","last_unstale":"2024-05-08T08:55:50.555233+0000","last_undegraded":"2024-05-08T08:55:50.555233+0000","last_fullsized":"2024-05-08T08:55:50.555233+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:28:25.204125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"182'652","reported_seq":20646,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.562407+0000","last_change":"2024-05-08T08:55:40.380225+0000","last_active":"2024-05-08T08:55:50.562407+0000","last_peered":"2024-05-08T08:55:50.562407+0000","last_clean":"2024-05-08T08:55:50.562407+0000","last_became_active":"2024-05-08T08:55:40.378157+0000","last_became_peered":"2024-05-08T08:55:40.378157+0000","last_unstale":"2024-05-08T08:55:50.562407+0000","last_undegraded":"2024-05-08T08:55:50.562407+0000","last_fullsized":"2024-05-08T08:55:50.562407+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":652,"log_dups_size":0,"ondisk_log_size":652,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:03:49.159963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019021660000000001,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":40,"num_read":21842,"num_read_kb":88722,"num_write":580,"num_write_kb":17543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":288,"num_bytes_recovered":3368960,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":375,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.557262+0000","last_change":"2024-05-08T08:52:03.896159+0000","last_active":"2024-05-08T08:55:50.557262+0000","last_peered":"2024-05-08T08:55:50.557262+0000","last_clean":"2024-05-08T08:55:50.557262+0000","last_became_active":"2024-05-08T08:52:03.895797+0000","last_became_peered":"2024-05-08T08:52:03.895797+0000","last_unstale":"2024-05-08T08:55:50.557262+0000","last_undegraded":"2024-05-08T08:55:50.557262+0000","last_fullsized":"2024-05-08T08:55:50.557262+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:34:33.513525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"180'518","reported_seq":6927,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.554715+0000","last_change":"2024-05-08T08:55:33.483261+0000","last_active":"2024-05-08T08:55:50.554715+0000","last_peered":"2024-05-08T08:55:50.554715+0000","last_clean":"2024-05-08T08:55:50.554715+0000","last_became_active":"2024-05-08T08:55:08.437577+0000","last_became_peered":"2024-05-08T08:55:08.437577+0000","last_unstale":"2024-05-08T08:55:50.554715+0000","last_undegraded":"2024-05-08T08:55:50.554715+0000","last_fullsized":"2024-05-08T08:55:50.554715+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:55:04.162036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017459470000000001,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":39,"num_read":6769,"num_read_kb":34542,"num_write":419,"num_write_kb":12024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":324,"num_bytes_recovered":15121731,"num_keys_recovered":25,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":379,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.555279+0000","last_change":"2024-05-08T08:54:59.398530+0000","last_active":"2024-05-08T08:55:50.555279+0000","last_peered":"2024-05-08T08:55:50.555279+0000","last_clean":"2024-05-08T08:55:50.555279+0000","last_became_active":"2024-05-08T08:54:59.398181+0000","last_became_peered":"2024-05-08T08:54:59.398181+0000","last_unstale":"2024-05-08T08:55:50.555279+0000","last_undegraded":"2024-05-08T08:55:50.555279+0000","last_fullsized":"2024-05-08T08:55:50.555279+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:24:28.264614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"181'863","reported_seq":27577,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.557404+0000","last_change":"2024-05-08T08:55:43.386319+0000","last_active":"2024-05-08T08:55:50.557404+0000","last_peered":"2024-05-08T08:55:50.557404+0000","last_clean":"2024-05-08T08:55:50.557404+0000","last_became_active":"2024-05-08T08:55:43.384081+0000","last_became_peered":"2024-05-08T08:55:43.384081+0000","last_unstale":"2024-05-08T08:55:50.557404+0000","last_undegraded":"2024-05-08T08:55:50.557404+0000","last_fullsized":"2024-05-08T08:55:50.557404+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":863,"log_dups_size":0,"ondisk_log_size":863,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:02:20.851342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014858339999999999,"stat_sum":{"num_bytes":0,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":48,"num_read":22624,"num_read_kb":52229,"num_write":653,"num_write_kb":12062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":791,"num_bytes_recovered":28697088,"num_keys_recovered":110,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":385,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.557499+0000","last_change":"2024-05-08T08:55:37.500332+0000","last_active":"2024-05-08T08:55:50.557499+0000","last_peered":"2024-05-08T08:55:50.557499+0000","last_clean":"2024-05-08T08:55:50.557499+0000","last_became_active":"2024-05-08T08:55:37.499784+0000","last_became_peered":"2024-05-08T08:55:37.499784+0000","last_unstale":"2024-05-08T08:55:50.557499+0000","last_undegraded":"2024-05-08T08:55:50.557499+0000","last_fullsized":"2024-05-08T08:55:50.557499+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:53:12.058683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"181'946","reported_seq":14829,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.562555+0000","last_change":"2024-05-08T08:55:50.562555+0000","last_active":"2024-05-08T08:55:50.562555+0000","last_peered":"2024-05-08T08:55:50.562555+0000","last_clean":"2024-05-08T08:55:50.562555+0000","last_became_active":"2024-05-08T08:55:50.559883+0000","last_became_peered":"2024-05-08T08:55:50.559883+0000","last_unstale":"2024-05-08T08:55:50.562555+0000","last_undegraded":"2024-05-08T08:55:50.562555+0000","last_fullsized":"2024-05-08T08:55:50.562555+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":946,"log_dups_size":0,"ondisk_log_size":946,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:18:55.639194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013854900000000001,"stat_sum":{"num_bytes":0,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":55,"num_read":15316,"num_read_kb":32869,"num_write":1141,"num_write_kb":10612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":494,"num_bytes_recovered":13827825,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"182'817","reported_seq":12316,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.559335+0000","last_change":"2024-05-08T08:55:41.381790+0000","last_active":"2024-05-08T08:55:50.559335+0000","last_peered":"2024-05-08T08:55:50.559335+0000","last_clean":"2024-05-08T08:55:50.559335+0000","last_became_active":"2024-05-08T08:55:41.380391+0000","last_became_peered":"2024-05-08T08:55:41.380391+0000","last_unstale":"2024-05-08T08:55:50.559335+0000","last_undegraded":"2024-05-08T08:55:50.559335+0000","last_fullsized":"2024-05-08T08:55:50.559335+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":817,"log_dups_size":0,"ondisk_log_size":817,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:59:49.412777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002033212,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":34,"num_read":11854,"num_read_kb":33576,"num_write":867,"num_write_kb":6636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":383,"num_bytes_recovered":9012422,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":353,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.560520+0000","last_change":"2024-05-08T08:55:50.560520+0000","last_active":"2024-05-08T08:55:50.560520+0000","last_peered":"2024-05-08T08:55:50.560520+0000","last_clean":"2024-05-08T08:55:50.560520+0000","last_became_active":"2024-05-08T08:55:50.559612+0000","last_became_peered":"2024-05-08T08:55:50.559612+0000","last_unstale":"2024-05-08T08:55:50.560520+0000","last_undegraded":"2024-05-08T08:55:50.560520+0000","last_fullsized":"2024-05-08T08:55:50.560520+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:51:19.494663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"181'729","reported_seq":14229,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.558114+0000","last_change":"2024-05-08T08:55:33.470800+0000","last_active":"2024-05-08T08:55:50.558114+0000","last_peered":"2024-05-08T08:55:50.558114+0000","last_clean":"2024-05-08T08:55:50.558114+0000","last_became_active":"2024-05-08T08:52:11.928301+0000","last_became_peered":"2024-05-08T08:52:11.928301+0000","last_unstale":"2024-05-08T08:55:50.558114+0000","last_undegraded":"2024-05-08T08:55:50.558114+0000","last_fullsized":"2024-05-08T08:55:50.558114+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":729,"log_dups_size":0,"ondisk_log_size":729,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:03:13.796981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015902449999999999,"stat_sum":{"num_bytes":0,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":37,"num_read":10704,"num_read_kb":44099,"num_write":654,"num_write_kb":16306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":375,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.555092+0000","last_change":"2024-05-08T08:52:03.891768+0000","last_active":"2024-05-08T08:55:50.555092+0000","last_peered":"2024-05-08T08:55:50.555092+0000","last_clean":"2024-05-08T08:55:50.555092+0000","last_became_active":"2024-05-08T08:52:03.891374+0000","last_became_peered":"2024-05-08T08:52:03.891374+0000","last_unstale":"2024-05-08T08:55:50.555092+0000","last_undegraded":"2024-05-08T08:55:50.555092+0000","last_fullsized":"2024-05-08T08:55:50.555092+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:18:10.240492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"182'1161","reported_seq":28197,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.562344+0000","last_change":"2024-05-08T08:55:42.388348+0000","last_active":"2024-05-08T08:55:50.562344+0000","last_peered":"2024-05-08T08:55:50.562344+0000","last_clean":"2024-05-08T08:55:50.562344+0000","last_became_active":"2024-05-08T08:55:42.386239+0000","last_became_peered":"2024-05-08T08:55:42.386239+0000","last_unstale":"2024-05-08T08:55:50.562344+0000","last_undegraded":"2024-05-08T08:55:50.562344+0000","last_fullsized":"2024-05-08T08:55:50.562344+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":1161,"log_dups_size":0,"ondisk_log_size":1161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:06:05.510596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086553400000000005,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":31078,"num_read_kb":56218,"num_write":870,"num_write_kb":9640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":490,"num_bytes_recovered":5023153,"num_keys_recovered":69,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":374,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.554962+0000","last_change":"2024-05-08T08:52:03.891551+0000","last_active":"2024-05-08T08:55:50.554962+0000","last_peered":"2024-05-08T08:55:50.554962+0000","last_clean":"2024-05-08T08:55:50.554962+0000","last_became_active":"2024-05-08T08:52:03.891296+0000","last_became_peered":"2024-05-08T08:52:03.891296+0000","last_unstale":"2024-05-08T08:55:50.554962+0000","last_undegraded":"2024-05-08T08:55:50.554962+0000","last_fullsized":"2024-05-08T08:55:50.554962+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:54.806343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"181'1002","reported_seq":15538,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.557496+0000","last_change":"2024-05-08T08:55:43.382935+0000","last_active":"2024-05-08T08:55:50.557496+0000","last_peered":"2024-05-08T08:55:50.557496+0000","last_clean":"2024-05-08T08:55:50.557496+0000","last_became_active":"2024-05-08T08:55:43.381345+0000","last_became_peered":"2024-05-08T08:55:43.381345+0000","last_unstale":"2024-05-08T08:55:50.557496+0000","last_undegraded":"2024-05-08T08:55:50.557496+0000","last_fullsized":"2024-05-08T08:55:50.557496+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":1002,"log_dups_size":0,"ondisk_log_size":1002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:30:03.146595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012464850000000001,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":39,"num_read":15521,"num_read_kb":63440,"num_write":1271,"num_write_kb":13979,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":635,"num_bytes_recovered":18405521,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":349,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.560138+0000","last_change":"2024-05-08T08:55:50.560138+0000","last_active":"2024-05-08T08:55:50.560138+0000","last_peered":"2024-05-08T08:55:50.560138+0000","last_clean":"2024-05-08T08:55:50.560138+0000","last_became_active":"2024-05-08T08:55:50.559815+0000","last_became_peered":"2024-05-08T08:55:50.559815+0000","last_unstale":"2024-05-08T08:55:50.560138+0000","last_undegraded":"2024-05-08T08:55:50.560138+0000","last_fullsized":"2024-05-08T08:55:50.560138+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:32:21.109548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":556,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.555092+0000","last_change":"2024-05-08T08:55:42.381739+0000","last_active":"2024-05-08T08:55:50.555092+0000","last_peered":"2024-05-08T08:55:50.555092+0000","last_clean":"2024-05-08T08:55:50.555092+0000","last_became_active":"2024-05-08T08:55:42.381316+0000","last_became_peered":"2024-05-08T08:55:42.381316+0000","last_unstale":"2024-05-08T08:55:50.555092+0000","last_undegraded":"2024-05-08T08:55:50.555092+0000","last_fullsized":"2024-05-08T08:55:50.555092+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:51:57.849666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:51:57.849666+0000","last_clean_scrub_stamp":"2024-05-08T08:51:57.849666+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:59:25.417866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"182'398","reported_seq":9838,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.555377+0000","last_change":"2024-05-08T08:55:33.481972+0000","last_active":"2024-05-08T08:55:50.555377+0000","last_peered":"2024-05-08T08:55:50.555377+0000","last_clean":"2024-05-08T08:55:50.555377+0000","last_became_active":"2024-05-08T08:52:11.926975+0000","last_became_peered":"2024-05-08T08:52:11.926975+0000","last_unstale":"2024-05-08T08:55:50.555377+0000","last_undegraded":"2024-05-08T08:55:50.555377+0000","last_fullsized":"2024-05-08T08:55:50.555377+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":398,"log_dups_size":0,"ondisk_log_size":398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:13:12.117653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087585099999999997,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":35,"num_read":8901,"num_read_kb":36556,"num_write":340,"num_write_kb":8985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"180'503","reported_seq":14561,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.559353+0000","last_change":"2024-05-08T08:55:33.475165+0000","last_active":"2024-05-08T08:55:50.559353+0000","last_peered":"2024-05-08T08:55:50.559353+0000","last_clean":"2024-05-08T08:55:50.559353+0000","last_became_active":"2024-05-08T08:53:23.478564+0000","last_became_peered":"2024-05-08T08:53:23.478564+0000","last_unstale":"2024-05-08T08:55:50.559353+0000","last_undegraded":"2024-05-08T08:55:50.559353+0000","last_fullsized":"2024-05-08T08:55:50.559353+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":503,"log_dups_size":0,"ondisk_log_size":503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:32:37.241705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082315600000000002,"stat_sum":{"num_bytes":0,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":42,"num_read":14198,"num_read_kb":62213,"num_write":413,"num_write_kb":16566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":131,"num_bytes_recovered":12377088,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"182'1247","reported_seq":11910,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.554993+0000","last_change":"2024-05-08T08:55:33.468391+0000","last_active":"2024-05-08T08:55:50.554993+0000","last_peered":"2024-05-08T08:55:50.554993+0000","last_clean":"2024-05-08T08:55:50.554993+0000","last_became_active":"2024-05-08T08:53:24.480359+0000","last_became_peered":"2024-05-08T08:53:24.480359+0000","last_unstale":"2024-05-08T08:55:50.554993+0000","last_undegraded":"2024-05-08T08:55:50.554993+0000","last_fullsized":"2024-05-08T08:55:50.554993+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":1247,"log_dups_size":0,"ondisk_log_size":1247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:08:14.677080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025209339999999998,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":40,"num_read":15863,"num_read_kb":29509,"num_write":1451,"num_write_kb":6331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":2175343,"num_keys_recovered":61,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"180'628","reported_seq":12516,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.557293+0000","last_change":"2024-05-08T08:55:42.384129+0000","last_active":"2024-05-08T08:55:50.557293+0000","last_peered":"2024-05-08T08:55:50.557293+0000","last_clean":"2024-05-08T08:55:50.557293+0000","last_became_active":"2024-05-08T08:55:42.382485+0000","last_became_peered":"2024-05-08T08:55:42.382485+0000","last_unstale":"2024-05-08T08:55:50.557293+0000","last_undegraded":"2024-05-08T08:55:50.557293+0000","last_fullsized":"2024-05-08T08:55:50.557293+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":628,"log_dups_size":0,"ondisk_log_size":628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:10:53.437543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001692783,"stat_sum":{"num_bytes":0,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":50,"num_read":11950,"num_read_kb":39515,"num_write":513,"num_write_kb":7725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":332,"num_bytes_recovered":2080800,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"180'378","reported_seq":14238,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.565133+0000","last_change":"2024-05-08T08:55:50.565133+0000","last_active":"2024-05-08T08:55:50.565133+0000","last_peered":"2024-05-08T08:55:50.565133+0000","last_clean":"2024-05-08T08:55:50.565133+0000","last_became_active":"2024-05-08T08:55:50.563411+0000","last_became_peered":"2024-05-08T08:55:50.563411+0000","last_unstale":"2024-05-08T08:55:50.565133+0000","last_undegraded":"2024-05-08T08:55:50.565133+0000","last_fullsized":"2024-05-08T08:55:50.565133+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:22:13.206310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020039379999999998,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":40,"num_read":13099,"num_read_kb":41266,"num_write":323,"num_write_kb":7340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":509,"num_bytes_recovered":9113137,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"180'478","reported_seq":8966,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.555126+0000","last_change":"2024-05-08T08:55:33.469549+0000","last_active":"2024-05-08T08:55:50.555126+0000","last_peered":"2024-05-08T08:55:50.555126+0000","last_clean":"2024-05-08T08:55:50.555126+0000","last_became_active":"2024-05-08T08:52:11.932594+0000","last_became_peered":"2024-05-08T08:52:11.932594+0000","last_unstale":"2024-05-08T08:55:50.555126+0000","last_undegraded":"2024-05-08T08:55:50.555126+0000","last_fullsized":"2024-05-08T08:55:50.555126+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":478,"log_dups_size":0,"ondisk_log_size":478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:53:33.929731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033757140000000001,"stat_sum":{"num_bytes":0,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":43,"num_read":9572,"num_read_kb":34328,"num_write":402,"num_write_kb":10002,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"180'632","reported_seq":9873,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.562760+0000","last_change":"2024-05-08T08:55:50.562760+0000","last_active":"2024-05-08T08:55:50.562760+0000","last_peered":"2024-05-08T08:55:50.562760+0000","last_clean":"2024-05-08T08:55:50.562760+0000","last_became_active":"2024-05-08T08:55:50.560907+0000","last_became_peered":"2024-05-08T08:55:50.560907+0000","last_unstale":"2024-05-08T08:55:50.562760+0000","last_undegraded":"2024-05-08T08:55:50.562760+0000","last_fullsized":"2024-05-08T08:55:50.562760+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":632,"log_dups_size":0,"ondisk_log_size":632,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:04:24.220863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002456153,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":16215,"num_read_kb":47213,"num_write":634,"num_write_kb":12225,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":446,"num_bytes_recovered":28147344,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"182'588","reported_seq":8774,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.557417+0000","last_change":"2024-05-08T08:55:40.386164+0000","last_active":"2024-05-08T08:55:50.557417+0000","last_peered":"2024-05-08T08:55:50.557417+0000","last_clean":"2024-05-08T08:55:50.557417+0000","last_became_active":"2024-05-08T08:55:40.383962+0000","last_became_peered":"2024-05-08T08:55:40.383962+0000","last_unstale":"2024-05-08T08:55:50.557417+0000","last_undegraded":"2024-05-08T08:55:50.557417+0000","last_fullsized":"2024-05-08T08:55:50.557417+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":588,"log_dups_size":0,"ondisk_log_size":588,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:48:37.179607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037436610000000001,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":36,"num_read":8886,"num_read_kb":38380,"num_write":568,"num_write_kb":9633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":675,"num_bytes_recovered":18815289,"num_keys_recovered":79,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"180'749","reported_seq":8638,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.567799+0000","last_change":"2024-05-08T08:55:50.567799+0000","last_active":"2024-05-08T08:55:50.567799+0000","last_peered":"2024-05-08T08:55:50.567799+0000","last_clean":"2024-05-08T08:55:50.567799+0000","last_became_active":"2024-05-08T08:55:50.565716+0000","last_became_peered":"2024-05-08T08:55:50.565716+0000","last_unstale":"2024-05-08T08:55:50.567799+0000","last_undegraded":"2024-05-08T08:55:50.567799+0000","last_fullsized":"2024-05-08T08:55:50.567799+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":749,"log_dups_size":0,"ondisk_log_size":749,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:28:22.726479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095127799999999995,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":40,"num_read":9090,"num_read_kb":41259,"num_write":984,"num_write_kb":14557,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":541,"num_bytes_recovered":26549987,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"182'750","reported_seq":11879,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.554980+0000","last_change":"2024-05-08T08:55:33.467196+0000","last_active":"2024-05-08T08:55:50.554980+0000","last_peered":"2024-05-08T08:55:50.554980+0000","last_clean":"2024-05-08T08:55:50.554980+0000","last_became_active":"2024-05-08T08:53:24.479891+0000","last_became_peered":"2024-05-08T08:53:24.479891+0000","last_unstale":"2024-05-08T08:55:50.554980+0000","last_undegraded":"2024-05-08T08:55:50.554980+0000","last_fullsized":"2024-05-08T08:55:50.554980+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":750,"log_dups_size":0,"ondisk_log_size":750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:11:35.412500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015502269999999999,"stat_sum":{"num_bytes":0,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":43,"num_read":13241,"num_read_kb":36544,"num_write":586,"num_write_kb":10920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":131,"num_bytes_recovered":2604032,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"182'1575","reported_seq":25758,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.562352+0000","last_change":"2024-05-08T08:55:43.389357+0000","last_active":"2024-05-08T08:55:50.562352+0000","last_peered":"2024-05-08T08:55:50.562352+0000","last_clean":"2024-05-08T08:55:50.562352+0000","last_became_active":"2024-05-08T08:55:43.386543+0000","last_became_peered":"2024-05-08T08:55:43.386543+0000","last_unstale":"2024-05-08T08:55:50.562352+0000","last_undegraded":"2024-05-08T08:55:50.562352+0000","last_fullsized":"2024-05-08T08:55:50.562352+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":1575,"log_dups_size":0,"ondisk_log_size":1575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:38:01.859312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015403719999999999,"stat_sum":{"num_bytes":0,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":52,"num_read":30937,"num_read_kb":62696,"num_write":2139,"num_write_kb":12077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":330,"num_bytes_recovered":4235412,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"181'651","reported_seq":11695,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.555086+0000","last_change":"2024-05-08T08:55:43.380556+0000","last_active":"2024-05-08T08:55:50.555086+0000","last_peered":"2024-05-08T08:55:50.555086+0000","last_clean":"2024-05-08T08:55:50.555086+0000","last_became_active":"2024-05-08T08:55:43.379146+0000","last_became_peered":"2024-05-08T08:55:43.379146+0000","last_unstale":"2024-05-08T08:55:50.555086+0000","last_undegraded":"2024-05-08T08:55:50.555086+0000","last_fullsized":"2024-05-08T08:55:50.555086+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":651,"log_dups_size":0,"ondisk_log_size":651,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:09:48.532237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022701349999999999,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":40,"num_read":11037,"num_read_kb":41101,"num_write":598,"num_write_kb":9071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":589,"num_bytes_recovered":19963261,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":1295,"num_object_clones":0,"num_object_copies":2590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1295,"num_whiteouts":1291,"num_read":512933,"num_read_kb":1628362,"num_write":26100,"num_write_kb":451839,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11018,"num_bytes_recovered":421954137,"num_keys_recovered":904,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24829,"ondisk_log_size":24829,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":18287,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640306,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59884,"kb_used_data":2328,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311956,"statfs":{"total":96636764160,"available":96575442944,"internally_reserved":0,"allocated":2383872,"data_stored":1016789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640310,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55724,"kb_used_data":2332,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316116,"statfs":{"total":96636764160,"available":96579702784,"internally_reserved":0,"allocated":2387968,"data_stored":1016808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640308,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65504,"kb_used_data":2384,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306336,"statfs":{"total":96636764160,"available":96569688064,"internally_reserved":0,"allocated":2441216,"data_stored":1607157,"data_compressed":5523,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1593,"internal_metadata":64616903},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640307,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51560,"kb_used_data":2328,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320280,"statfs":{"total":96636764160,"available":96583966720,"internally_reserved":0,"allocated":2383872,"data_stored":1016789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640306,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65448,"kb_used_data":2332,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306392,"statfs":{"total":96636764160,"available":96569745408,"internally_reserved":0,"allocated":2387968,"data_stored":1016808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640309,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69656,"kb_used_data":2368,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302184,"statfs":{"total":96636764160,"available":96565436416,"internally_reserved":0,"allocated":2424832,"data_stored":1607157,"data_compressed":6382,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6382,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6382,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5523,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T08:55:55.156 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T08:55:55.453 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:55:55.454 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-05-08T08:55:55.465 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":311,"stamp":"2024-05-08T08:55:53.780431+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":1297,"num_object_clones":0,"num_object_copies":2594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1297,"num_whiteouts":1291,"num_read":513049,"num_read_kb":1628462,"num_write":26259,"num_write_kb":454255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11022,"num_bytes_recovered":423134873,"num_keys_recovered":904,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24946,"ondisk_log_size":24946,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":367776,"kb_used_data":14072,"kb_used_omap":9,"kb_used_meta":353590,"kb_avail":565863264,"statfs":{"total":579820584960,"available":579443982336,"internally_reserved":0,"allocated":14409728,"data_stored":7281508,"data_compressed":11905,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":9568,"internal_metadata":362076832},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-743,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001324"},"pg_stats":[{"pgid":"3.1f","version":"182'772","reported_seq":20838,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.562464+0000","last_change":"2024-05-08T08:55:43.388833+0000","last_active":"2024-05-08T08:55:50.562464+0000","last_peered":"2024-05-08T08:55:50.562464+0000","last_clean":"2024-05-08T08:55:50.562464+0000","last_became_active":"2024-05-08T08:55:43.386102+0000","last_became_peered":"2024-05-08T08:55:43.386102+0000","last_unstale":"2024-05-08T08:55:50.562464+0000","last_undegraded":"2024-05-08T08:55:50.562464+0000","last_fullsized":"2024-05-08T08:55:50.562464+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":772,"log_dups_size":0,"ondisk_log_size":772,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:25:19.005824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038328300000000002,"stat_sum":{"num_bytes":0,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":34,"num_read":19333,"num_read_kb":85778,"num_write":988,"num_write_kb":33105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":328,"num_bytes_recovered":19152951,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"181'922","reported_seq":11965,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.554583+0000","last_change":"2024-05-08T08:55:33.482489+0000","last_active":"2024-05-08T08:55:50.554583+0000","last_peered":"2024-05-08T08:55:50.554583+0000","last_clean":"2024-05-08T08:55:50.554583+0000","last_became_active":"2024-05-08T08:52:11.930844+0000","last_became_peered":"2024-05-08T08:52:11.930844+0000","last_unstale":"2024-05-08T08:55:50.554583+0000","last_undegraded":"2024-05-08T08:55:50.554583+0000","last_fullsized":"2024-05-08T08:55:50.554583+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":922,"log_dups_size":0,"ondisk_log_size":922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:34:27.496752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001408816,"stat_sum":{"num_bytes":0,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":49,"num_read":13744,"num_read_kb":37146,"num_write":694,"num_write_kb":12720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"181'753","reported_seq":4867,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.562479+0000","last_change":"2024-05-08T08:55:44.384669+0000","last_active":"2024-05-08T08:55:50.562479+0000","last_peered":"2024-05-08T08:55:50.562479+0000","last_clean":"2024-05-08T08:55:50.562479+0000","last_became_active":"2024-05-08T08:55:44.381069+0000","last_became_peered":"2024-05-08T08:55:44.381069+0000","last_unstale":"2024-05-08T08:55:50.562479+0000","last_undegraded":"2024-05-08T08:55:50.562479+0000","last_fullsized":"2024-05-08T08:55:50.562479+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":753,"log_dups_size":0,"ondisk_log_size":753,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:27:12.956766+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0093620049999999996,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":40,"num_read":8309,"num_read_kb":30535,"num_write":901,"num_write_kb":7282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":570,"num_bytes_recovered":16438479,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"182'740","reported_seq":19620,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.562811+0000","last_change":"2024-05-08T08:55:50.562811+0000","last_active":"2024-05-08T08:55:50.562811+0000","last_peered":"2024-05-08T08:55:50.562811+0000","last_clean":"2024-05-08T08:55:50.562811+0000","last_became_active":"2024-05-08T08:55:50.560370+0000","last_became_peered":"2024-05-08T08:55:50.560370+0000","last_unstale":"2024-05-08T08:55:50.562811+0000","last_undegraded":"2024-05-08T08:55:50.562811+0000","last_fullsized":"2024-05-08T08:55:50.562811+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":740,"log_dups_size":0,"ondisk_log_size":740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:20:09.589464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019815259999999999,"stat_sum":{"num_bytes":0,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":45,"num_read":18641,"num_read_kb":101548,"num_write":811,"num_write_kb":27861,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":609,"num_bytes_recovered":86192377,"num_keys_recovered":338,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"179'637","reported_seq":26061,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.562537+0000","last_change":"2024-05-08T08:55:43.388686+0000","last_active":"2024-05-08T08:55:50.562537+0000","last_peered":"2024-05-08T08:55:50.562537+0000","last_clean":"2024-05-08T08:55:50.562537+0000","last_became_active":"2024-05-08T08:55:43.386098+0000","last_became_peered":"2024-05-08T08:55:43.386098+0000","last_unstale":"2024-05-08T08:55:50.562537+0000","last_undegraded":"2024-05-08T08:55:50.562537+0000","last_fullsized":"2024-05-08T08:55:50.562537+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":637,"log_dups_size":0,"ondisk_log_size":637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:29:43.226913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012631949999999999,"stat_sum":{"num_bytes":0,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":50,"num_read":25112,"num_read_kb":45826,"num_write":688,"num_write_kb":23312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":455,"num_bytes_recovered":5234820,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"180'867","reported_seq":22286,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.559406+0000","last_change":"2024-05-08T08:55:33.475720+0000","last_active":"2024-05-08T08:55:50.559406+0000","last_peered":"2024-05-08T08:55:50.559406+0000","last_clean":"2024-05-08T08:55:50.559406+0000","last_became_active":"2024-05-08T08:53:24.484124+0000","last_became_peered":"2024-05-08T08:53:24.484124+0000","last_unstale":"2024-05-08T08:55:50.559406+0000","last_undegraded":"2024-05-08T08:55:50.559406+0000","last_fullsized":"2024-05-08T08:55:50.559406+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":867,"log_dups_size":0,"ondisk_log_size":867,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:57:41.478541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012989099999999999,"stat_sum":{"num_bytes":19,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":34,"num_read":24563,"num_read_kb":59887,"num_write":908,"num_write_kb":19093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":130,"num_bytes_recovered":8549042,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"178'426","reported_seq":10752,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.557157+0000","last_change":"2024-05-08T08:55:33.470036+0000","last_active":"2024-05-08T08:55:50.557157+0000","last_peered":"2024-05-08T08:55:50.557157+0000","last_clean":"2024-05-08T08:55:50.557157+0000","last_became_active":"2024-05-08T08:53:24.483403+0000","last_became_peered":"2024-05-08T08:53:24.483403+0000","last_unstale":"2024-05-08T08:55:50.557157+0000","last_undegraded":"2024-05-08T08:55:50.557157+0000","last_fullsized":"2024-05-08T08:55:50.557157+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":426,"log_dups_size":0,"ondisk_log_size":426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:23:41.607743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011263099999999999,"stat_sum":{"num_bytes":0,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":48,"num_read":9419,"num_read_kb":48048,"num_write":368,"num_write_kb":16514,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":8814624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"181'711","reported_seq":20343,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.555276+0000","last_change":"2024-05-08T08:55:41.379915+0000","last_active":"2024-05-08T08:55:50.555276+0000","last_peered":"2024-05-08T08:55:50.555276+0000","last_clean":"2024-05-08T08:55:50.555276+0000","last_became_active":"2024-05-08T08:55:41.378182+0000","last_became_peered":"2024-05-08T08:55:41.378182+0000","last_unstale":"2024-05-08T08:55:50.555276+0000","last_undegraded":"2024-05-08T08:55:50.555276+0000","last_fullsized":"2024-05-08T08:55:50.555276+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":711,"log_dups_size":0,"ondisk_log_size":711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:23:17.505431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098929300000000003,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":20665,"num_read_kb":52569,"num_write":672,"num_write_kb":23378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":4852260,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"182'852","reported_seq":19005,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.557278+0000","last_change":"2024-05-08T08:55:40.383627+0000","last_active":"2024-05-08T08:55:50.557278+0000","last_peered":"2024-05-08T08:55:50.557278+0000","last_clean":"2024-05-08T08:55:50.557278+0000","last_became_active":"2024-05-08T08:55:40.381448+0000","last_became_peered":"2024-05-08T08:55:40.381448+0000","last_unstale":"2024-05-08T08:55:50.557278+0000","last_undegraded":"2024-05-08T08:55:50.557278+0000","last_fullsized":"2024-05-08T08:55:50.557278+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:14:47.318689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018475399999999999,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":30,"num_read":20569,"num_read_kb":77197,"num_write":969,"num_write_kb":16935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":381,"num_bytes_recovered":5593379,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"182'1461","reported_seq":21996,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.557382+0000","last_change":"2024-05-08T08:55:44.381999+0000","last_active":"2024-05-08T08:55:50.557382+0000","last_peered":"2024-05-08T08:55:50.557382+0000","last_clean":"2024-05-08T08:55:50.557382+0000","last_became_active":"2024-05-08T08:55:44.379555+0000","last_became_peered":"2024-05-08T08:55:44.379555+0000","last_unstale":"2024-05-08T08:55:50.557382+0000","last_undegraded":"2024-05-08T08:55:50.557382+0000","last_fullsized":"2024-05-08T08:55:50.557382+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":1461,"log_dups_size":0,"ondisk_log_size":1461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:07:01.906180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0045105479999999996,"stat_sum":{"num_bytes":0,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":33,"num_read":25480,"num_read_kb":87263,"num_write":2254,"num_write_kb":27190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":420,"num_bytes_recovered":42698110,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"182'690","reported_seq":17152,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.555076+0000","last_change":"2024-05-08T08:55:42.384249+0000","last_active":"2024-05-08T08:55:50.555076+0000","last_peered":"2024-05-08T08:55:50.555076+0000","last_clean":"2024-05-08T08:55:50.555076+0000","last_became_active":"2024-05-08T08:55:42.382974+0000","last_became_peered":"2024-05-08T08:55:42.382974+0000","last_unstale":"2024-05-08T08:55:50.555076+0000","last_undegraded":"2024-05-08T08:55:50.555076+0000","last_fullsized":"2024-05-08T08:55:50.555076+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":690,"log_dups_size":0,"ondisk_log_size":690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:12:41.796610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029835819999999998,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":39,"num_read":17511,"num_read_kb":53002,"num_write":778,"num_write_kb":8761,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":284,"num_bytes_recovered":1296934,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"180'733","reported_seq":12777,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.555182+0000","last_change":"2024-05-08T08:55:33.469235+0000","last_active":"2024-05-08T08:55:50.555182+0000","last_peered":"2024-05-08T08:55:50.555182+0000","last_clean":"2024-05-08T08:55:50.555182+0000","last_became_active":"2024-05-08T08:53:23.481362+0000","last_became_peered":"2024-05-08T08:53:23.481362+0000","last_unstale":"2024-05-08T08:55:50.555182+0000","last_undegraded":"2024-05-08T08:55:50.555182+0000","last_fullsized":"2024-05-08T08:55:50.555182+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":733,"log_dups_size":0,"ondisk_log_size":733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:15:25.796542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032028149999999999,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":30,"num_read":10890,"num_read_kb":33288,"num_write":663,"num_write_kb":11454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":3612768,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":384,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.555233+0000","last_change":"2024-05-08T08:53:20.362387+0000","last_active":"2024-05-08T08:55:50.555233+0000","last_peered":"2024-05-08T08:55:50.555233+0000","last_clean":"2024-05-08T08:55:50.555233+0000","last_became_active":"2024-05-08T08:53:20.362113+0000","last_became_peered":"2024-05-08T08:53:20.362113+0000","last_unstale":"2024-05-08T08:55:50.555233+0000","last_undegraded":"2024-05-08T08:55:50.555233+0000","last_fullsized":"2024-05-08T08:55:50.555233+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:28:25.204125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"182'652","reported_seq":20646,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.562407+0000","last_change":"2024-05-08T08:55:40.380225+0000","last_active":"2024-05-08T08:55:50.562407+0000","last_peered":"2024-05-08T08:55:50.562407+0000","last_clean":"2024-05-08T08:55:50.562407+0000","last_became_active":"2024-05-08T08:55:40.378157+0000","last_became_peered":"2024-05-08T08:55:40.378157+0000","last_unstale":"2024-05-08T08:55:50.562407+0000","last_undegraded":"2024-05-08T08:55:50.562407+0000","last_fullsized":"2024-05-08T08:55:50.562407+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":652,"log_dups_size":0,"ondisk_log_size":652,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:03:49.159963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019021660000000001,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":40,"num_read":21842,"num_read_kb":88722,"num_write":580,"num_write_kb":17543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":288,"num_bytes_recovered":3368960,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":375,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.557262+0000","last_change":"2024-05-08T08:52:03.896159+0000","last_active":"2024-05-08T08:55:50.557262+0000","last_peered":"2024-05-08T08:55:50.557262+0000","last_clean":"2024-05-08T08:55:50.557262+0000","last_became_active":"2024-05-08T08:52:03.895797+0000","last_became_peered":"2024-05-08T08:52:03.895797+0000","last_unstale":"2024-05-08T08:55:50.557262+0000","last_undegraded":"2024-05-08T08:55:50.557262+0000","last_fullsized":"2024-05-08T08:55:50.557262+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:34:33.513525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"180'518","reported_seq":6927,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.554715+0000","last_change":"2024-05-08T08:55:33.483261+0000","last_active":"2024-05-08T08:55:50.554715+0000","last_peered":"2024-05-08T08:55:50.554715+0000","last_clean":"2024-05-08T08:55:50.554715+0000","last_became_active":"2024-05-08T08:55:08.437577+0000","last_became_peered":"2024-05-08T08:55:08.437577+0000","last_unstale":"2024-05-08T08:55:50.554715+0000","last_undegraded":"2024-05-08T08:55:50.554715+0000","last_fullsized":"2024-05-08T08:55:50.554715+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:55:04.162036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017459470000000001,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":39,"num_read":6769,"num_read_kb":34542,"num_write":419,"num_write_kb":12024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":324,"num_bytes_recovered":15121731,"num_keys_recovered":25,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":379,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.555279+0000","last_change":"2024-05-08T08:54:59.398530+0000","last_active":"2024-05-08T08:55:50.555279+0000","last_peered":"2024-05-08T08:55:50.555279+0000","last_clean":"2024-05-08T08:55:50.555279+0000","last_became_active":"2024-05-08T08:54:59.398181+0000","last_became_peered":"2024-05-08T08:54:59.398181+0000","last_unstale":"2024-05-08T08:55:50.555279+0000","last_undegraded":"2024-05-08T08:55:50.555279+0000","last_fullsized":"2024-05-08T08:55:50.555279+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:24:28.264614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"181'863","reported_seq":27577,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.557404+0000","last_change":"2024-05-08T08:55:43.386319+0000","last_active":"2024-05-08T08:55:50.557404+0000","last_peered":"2024-05-08T08:55:50.557404+0000","last_clean":"2024-05-08T08:55:50.557404+0000","last_became_active":"2024-05-08T08:55:43.384081+0000","last_became_peered":"2024-05-08T08:55:43.384081+0000","last_unstale":"2024-05-08T08:55:50.557404+0000","last_undegraded":"2024-05-08T08:55:50.557404+0000","last_fullsized":"2024-05-08T08:55:50.557404+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":863,"log_dups_size":0,"ondisk_log_size":863,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:02:20.851342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014858339999999999,"stat_sum":{"num_bytes":0,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":48,"num_read":22624,"num_read_kb":52229,"num_write":653,"num_write_kb":12062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":791,"num_bytes_recovered":28697088,"num_keys_recovered":110,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":385,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.557499+0000","last_change":"2024-05-08T08:55:37.500332+0000","last_active":"2024-05-08T08:55:50.557499+0000","last_peered":"2024-05-08T08:55:50.557499+0000","last_clean":"2024-05-08T08:55:50.557499+0000","last_became_active":"2024-05-08T08:55:37.499784+0000","last_became_peered":"2024-05-08T08:55:37.499784+0000","last_unstale":"2024-05-08T08:55:50.557499+0000","last_undegraded":"2024-05-08T08:55:50.557499+0000","last_fullsized":"2024-05-08T08:55:50.557499+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:53:12.058683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"181'946","reported_seq":14829,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.562555+0000","last_change":"2024-05-08T08:55:50.562555+0000","last_active":"2024-05-08T08:55:50.562555+0000","last_peered":"2024-05-08T08:55:50.562555+0000","last_clean":"2024-05-08T08:55:50.562555+0000","last_became_active":"2024-05-08T08:55:50.559883+0000","last_became_peered":"2024-05-08T08:55:50.559883+0000","last_unstale":"2024-05-08T08:55:50.562555+0000","last_undegraded":"2024-05-08T08:55:50.562555+0000","last_fullsized":"2024-05-08T08:55:50.562555+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":946,"log_dups_size":0,"ondisk_log_size":946,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:18:55.639194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013854900000000001,"stat_sum":{"num_bytes":0,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":55,"num_read":15316,"num_read_kb":32869,"num_write":1141,"num_write_kb":10612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":494,"num_bytes_recovered":13827825,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"182'817","reported_seq":12316,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.559335+0000","last_change":"2024-05-08T08:55:41.381790+0000","last_active":"2024-05-08T08:55:50.559335+0000","last_peered":"2024-05-08T08:55:50.559335+0000","last_clean":"2024-05-08T08:55:50.559335+0000","last_became_active":"2024-05-08T08:55:41.380391+0000","last_became_peered":"2024-05-08T08:55:41.380391+0000","last_unstale":"2024-05-08T08:55:50.559335+0000","last_undegraded":"2024-05-08T08:55:50.559335+0000","last_fullsized":"2024-05-08T08:55:50.559335+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":817,"log_dups_size":0,"ondisk_log_size":817,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:59:49.412777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002033212,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":34,"num_read":11854,"num_read_kb":33576,"num_write":867,"num_write_kb":6636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":383,"num_bytes_recovered":9012422,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":353,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.560520+0000","last_change":"2024-05-08T08:55:50.560520+0000","last_active":"2024-05-08T08:55:50.560520+0000","last_peered":"2024-05-08T08:55:50.560520+0000","last_clean":"2024-05-08T08:55:50.560520+0000","last_became_active":"2024-05-08T08:55:50.559612+0000","last_became_peered":"2024-05-08T08:55:50.559612+0000","last_unstale":"2024-05-08T08:55:50.560520+0000","last_undegraded":"2024-05-08T08:55:50.560520+0000","last_fullsized":"2024-05-08T08:55:50.560520+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:51:19.494663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"181'729","reported_seq":14229,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.558114+0000","last_change":"2024-05-08T08:55:33.470800+0000","last_active":"2024-05-08T08:55:50.558114+0000","last_peered":"2024-05-08T08:55:50.558114+0000","last_clean":"2024-05-08T08:55:50.558114+0000","last_became_active":"2024-05-08T08:52:11.928301+0000","last_became_peered":"2024-05-08T08:52:11.928301+0000","last_unstale":"2024-05-08T08:55:50.558114+0000","last_undegraded":"2024-05-08T08:55:50.558114+0000","last_fullsized":"2024-05-08T08:55:50.558114+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":729,"log_dups_size":0,"ondisk_log_size":729,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:03:13.796981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015902449999999999,"stat_sum":{"num_bytes":0,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":37,"num_read":10704,"num_read_kb":44099,"num_write":654,"num_write_kb":16306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":375,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.555092+0000","last_change":"2024-05-08T08:52:03.891768+0000","last_active":"2024-05-08T08:55:50.555092+0000","last_peered":"2024-05-08T08:55:50.555092+0000","last_clean":"2024-05-08T08:55:50.555092+0000","last_became_active":"2024-05-08T08:52:03.891374+0000","last_became_peered":"2024-05-08T08:52:03.891374+0000","last_unstale":"2024-05-08T08:55:50.555092+0000","last_undegraded":"2024-05-08T08:55:50.555092+0000","last_fullsized":"2024-05-08T08:55:50.555092+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:18:10.240492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"182'1161","reported_seq":28197,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.562344+0000","last_change":"2024-05-08T08:55:42.388348+0000","last_active":"2024-05-08T08:55:50.562344+0000","last_peered":"2024-05-08T08:55:50.562344+0000","last_clean":"2024-05-08T08:55:50.562344+0000","last_became_active":"2024-05-08T08:55:42.386239+0000","last_became_peered":"2024-05-08T08:55:42.386239+0000","last_unstale":"2024-05-08T08:55:50.562344+0000","last_undegraded":"2024-05-08T08:55:50.562344+0000","last_fullsized":"2024-05-08T08:55:50.562344+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":1161,"log_dups_size":0,"ondisk_log_size":1161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:06:05.510596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086553400000000005,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":31078,"num_read_kb":56218,"num_write":870,"num_write_kb":9640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":490,"num_bytes_recovered":5023153,"num_keys_recovered":69,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":374,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.554962+0000","last_change":"2024-05-08T08:52:03.891551+0000","last_active":"2024-05-08T08:55:50.554962+0000","last_peered":"2024-05-08T08:55:50.554962+0000","last_clean":"2024-05-08T08:55:50.554962+0000","last_became_active":"2024-05-08T08:52:03.891296+0000","last_became_peered":"2024-05-08T08:52:03.891296+0000","last_unstale":"2024-05-08T08:55:50.554962+0000","last_undegraded":"2024-05-08T08:55:50.554962+0000","last_fullsized":"2024-05-08T08:55:50.554962+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:54.806343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"181'1002","reported_seq":15538,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.557496+0000","last_change":"2024-05-08T08:55:43.382935+0000","last_active":"2024-05-08T08:55:50.557496+0000","last_peered":"2024-05-08T08:55:50.557496+0000","last_clean":"2024-05-08T08:55:50.557496+0000","last_became_active":"2024-05-08T08:55:43.381345+0000","last_became_peered":"2024-05-08T08:55:43.381345+0000","last_unstale":"2024-05-08T08:55:50.557496+0000","last_undegraded":"2024-05-08T08:55:50.557496+0000","last_fullsized":"2024-05-08T08:55:50.557496+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":1002,"log_dups_size":0,"ondisk_log_size":1002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:30:03.146595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012464850000000001,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":39,"num_read":15521,"num_read_kb":63440,"num_write":1271,"num_write_kb":13979,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":635,"num_bytes_recovered":18405521,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":349,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.560138+0000","last_change":"2024-05-08T08:55:50.560138+0000","last_active":"2024-05-08T08:55:50.560138+0000","last_peered":"2024-05-08T08:55:50.560138+0000","last_clean":"2024-05-08T08:55:50.560138+0000","last_became_active":"2024-05-08T08:55:50.559815+0000","last_became_peered":"2024-05-08T08:55:50.559815+0000","last_unstale":"2024-05-08T08:55:50.560138+0000","last_undegraded":"2024-05-08T08:55:50.560138+0000","last_fullsized":"2024-05-08T08:55:50.560138+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:32:21.109548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":556,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.555092+0000","last_change":"2024-05-08T08:55:42.381739+0000","last_active":"2024-05-08T08:55:50.555092+0000","last_peered":"2024-05-08T08:55:50.555092+0000","last_clean":"2024-05-08T08:55:50.555092+0000","last_became_active":"2024-05-08T08:55:42.381316+0000","last_became_peered":"2024-05-08T08:55:42.381316+0000","last_unstale":"2024-05-08T08:55:50.555092+0000","last_undegraded":"2024-05-08T08:55:50.555092+0000","last_fullsized":"2024-05-08T08:55:50.555092+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:51:57.849666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:51:57.849666+0000","last_clean_scrub_stamp":"2024-05-08T08:51:57.849666+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:59:25.417866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"182'398","reported_seq":9838,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.555377+0000","last_change":"2024-05-08T08:55:33.481972+0000","last_active":"2024-05-08T08:55:50.555377+0000","last_peered":"2024-05-08T08:55:50.555377+0000","last_clean":"2024-05-08T08:55:50.555377+0000","last_became_active":"2024-05-08T08:52:11.926975+0000","last_became_peered":"2024-05-08T08:52:11.926975+0000","last_unstale":"2024-05-08T08:55:50.555377+0000","last_undegraded":"2024-05-08T08:55:50.555377+0000","last_fullsized":"2024-05-08T08:55:50.555377+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":398,"log_dups_size":0,"ondisk_log_size":398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:13:12.117653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087585099999999997,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":35,"num_read":8901,"num_read_kb":36556,"num_write":340,"num_write_kb":8985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"180'503","reported_seq":14561,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.559353+0000","last_change":"2024-05-08T08:55:33.475165+0000","last_active":"2024-05-08T08:55:50.559353+0000","last_peered":"2024-05-08T08:55:50.559353+0000","last_clean":"2024-05-08T08:55:50.559353+0000","last_became_active":"2024-05-08T08:53:23.478564+0000","last_became_peered":"2024-05-08T08:53:23.478564+0000","last_unstale":"2024-05-08T08:55:50.559353+0000","last_undegraded":"2024-05-08T08:55:50.559353+0000","last_fullsized":"2024-05-08T08:55:50.559353+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":503,"log_dups_size":0,"ondisk_log_size":503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:32:37.241705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082315600000000002,"stat_sum":{"num_bytes":0,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":42,"num_read":14198,"num_read_kb":62213,"num_write":413,"num_write_kb":16566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":131,"num_bytes_recovered":12377088,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"182'1247","reported_seq":11910,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.554993+0000","last_change":"2024-05-08T08:55:33.468391+0000","last_active":"2024-05-08T08:55:50.554993+0000","last_peered":"2024-05-08T08:55:50.554993+0000","last_clean":"2024-05-08T08:55:50.554993+0000","last_became_active":"2024-05-08T08:53:24.480359+0000","last_became_peered":"2024-05-08T08:53:24.480359+0000","last_unstale":"2024-05-08T08:55:50.554993+0000","last_undegraded":"2024-05-08T08:55:50.554993+0000","last_fullsized":"2024-05-08T08:55:50.554993+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":1247,"log_dups_size":0,"ondisk_log_size":1247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:08:14.677080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025209339999999998,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":40,"num_read":15863,"num_read_kb":29509,"num_write":1451,"num_write_kb":6331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":2175343,"num_keys_recovered":61,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"180'628","reported_seq":12516,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.557293+0000","last_change":"2024-05-08T08:55:42.384129+0000","last_active":"2024-05-08T08:55:50.557293+0000","last_peered":"2024-05-08T08:55:50.557293+0000","last_clean":"2024-05-08T08:55:50.557293+0000","last_became_active":"2024-05-08T08:55:42.382485+0000","last_became_peered":"2024-05-08T08:55:42.382485+0000","last_unstale":"2024-05-08T08:55:50.557293+0000","last_undegraded":"2024-05-08T08:55:50.557293+0000","last_fullsized":"2024-05-08T08:55:50.557293+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":628,"log_dups_size":0,"ondisk_log_size":628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:10:53.437543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001692783,"stat_sum":{"num_bytes":0,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":50,"num_read":11950,"num_read_kb":39515,"num_write":513,"num_write_kb":7725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":332,"num_bytes_recovered":2080800,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"180'378","reported_seq":14238,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.565133+0000","last_change":"2024-05-08T08:55:50.565133+0000","last_active":"2024-05-08T08:55:50.565133+0000","last_peered":"2024-05-08T08:55:50.565133+0000","last_clean":"2024-05-08T08:55:50.565133+0000","last_became_active":"2024-05-08T08:55:50.563411+0000","last_became_peered":"2024-05-08T08:55:50.563411+0000","last_unstale":"2024-05-08T08:55:50.565133+0000","last_undegraded":"2024-05-08T08:55:50.565133+0000","last_fullsized":"2024-05-08T08:55:50.565133+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:22:13.206310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020039379999999998,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":40,"num_read":13099,"num_read_kb":41266,"num_write":323,"num_write_kb":7340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":509,"num_bytes_recovered":9113137,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"180'478","reported_seq":8966,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.555126+0000","last_change":"2024-05-08T08:55:33.469549+0000","last_active":"2024-05-08T08:55:50.555126+0000","last_peered":"2024-05-08T08:55:50.555126+0000","last_clean":"2024-05-08T08:55:50.555126+0000","last_became_active":"2024-05-08T08:52:11.932594+0000","last_became_peered":"2024-05-08T08:52:11.932594+0000","last_unstale":"2024-05-08T08:55:50.555126+0000","last_undegraded":"2024-05-08T08:55:50.555126+0000","last_fullsized":"2024-05-08T08:55:50.555126+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":478,"log_dups_size":0,"ondisk_log_size":478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:53:33.929731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033757140000000001,"stat_sum":{"num_bytes":0,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":43,"num_read":9572,"num_read_kb":34328,"num_write":402,"num_write_kb":10002,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"180'632","reported_seq":9873,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.562760+0000","last_change":"2024-05-08T08:55:50.562760+0000","last_active":"2024-05-08T08:55:50.562760+0000","last_peered":"2024-05-08T08:55:50.562760+0000","last_clean":"2024-05-08T08:55:50.562760+0000","last_became_active":"2024-05-08T08:55:50.560907+0000","last_became_peered":"2024-05-08T08:55:50.560907+0000","last_unstale":"2024-05-08T08:55:50.562760+0000","last_undegraded":"2024-05-08T08:55:50.562760+0000","last_fullsized":"2024-05-08T08:55:50.562760+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":632,"log_dups_size":0,"ondisk_log_size":632,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:04:24.220863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002456153,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":16215,"num_read_kb":47213,"num_write":634,"num_write_kb":12225,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":446,"num_bytes_recovered":28147344,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"182'588","reported_seq":8774,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.557417+0000","last_change":"2024-05-08T08:55:40.386164+0000","last_active":"2024-05-08T08:55:50.557417+0000","last_peered":"2024-05-08T08:55:50.557417+0000","last_clean":"2024-05-08T08:55:50.557417+0000","last_became_active":"2024-05-08T08:55:40.383962+0000","last_became_peered":"2024-05-08T08:55:40.383962+0000","last_unstale":"2024-05-08T08:55:50.557417+0000","last_undegraded":"2024-05-08T08:55:50.557417+0000","last_fullsized":"2024-05-08T08:55:50.557417+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":588,"log_dups_size":0,"ondisk_log_size":588,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:48:37.179607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037436610000000001,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":36,"num_read":8886,"num_read_kb":38380,"num_write":568,"num_write_kb":9633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":675,"num_bytes_recovered":18815289,"num_keys_recovered":79,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"180'749","reported_seq":8638,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.567799+0000","last_change":"2024-05-08T08:55:50.567799+0000","last_active":"2024-05-08T08:55:50.567799+0000","last_peered":"2024-05-08T08:55:50.567799+0000","last_clean":"2024-05-08T08:55:50.567799+0000","last_became_active":"2024-05-08T08:55:50.565716+0000","last_became_peered":"2024-05-08T08:55:50.565716+0000","last_unstale":"2024-05-08T08:55:50.567799+0000","last_undegraded":"2024-05-08T08:55:50.567799+0000","last_fullsized":"2024-05-08T08:55:50.567799+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":749,"log_dups_size":0,"ondisk_log_size":749,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:28:22.726479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095127799999999995,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":40,"num_read":9090,"num_read_kb":41259,"num_write":984,"num_write_kb":14557,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":541,"num_bytes_recovered":26549987,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"182'750","reported_seq":11879,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.554980+0000","last_change":"2024-05-08T08:55:33.467196+0000","last_active":"2024-05-08T08:55:50.554980+0000","last_peered":"2024-05-08T08:55:50.554980+0000","last_clean":"2024-05-08T08:55:50.554980+0000","last_became_active":"2024-05-08T08:53:24.479891+0000","last_became_peered":"2024-05-08T08:53:24.479891+0000","last_unstale":"2024-05-08T08:55:50.554980+0000","last_undegraded":"2024-05-08T08:55:50.554980+0000","last_fullsized":"2024-05-08T08:55:50.554980+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":750,"log_dups_size":0,"ondisk_log_size":750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:11:35.412500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015502269999999999,"stat_sum":{"num_bytes":0,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":43,"num_read":13241,"num_read_kb":36544,"num_write":586,"num_write_kb":10920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":131,"num_bytes_recovered":2604032,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"182'1575","reported_seq":25758,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.562352+0000","last_change":"2024-05-08T08:55:43.389357+0000","last_active":"2024-05-08T08:55:50.562352+0000","last_peered":"2024-05-08T08:55:50.562352+0000","last_clean":"2024-05-08T08:55:50.562352+0000","last_became_active":"2024-05-08T08:55:43.386543+0000","last_became_peered":"2024-05-08T08:55:43.386543+0000","last_unstale":"2024-05-08T08:55:50.562352+0000","last_undegraded":"2024-05-08T08:55:50.562352+0000","last_fullsized":"2024-05-08T08:55:50.562352+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":1575,"log_dups_size":0,"ondisk_log_size":1575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:38:01.859312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015403719999999999,"stat_sum":{"num_bytes":0,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":52,"num_read":30937,"num_read_kb":62696,"num_write":2139,"num_write_kb":12077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":330,"num_bytes_recovered":4235412,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"181'651","reported_seq":11695,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.555086+0000","last_change":"2024-05-08T08:55:43.380556+0000","last_active":"2024-05-08T08:55:50.555086+0000","last_peered":"2024-05-08T08:55:50.555086+0000","last_clean":"2024-05-08T08:55:50.555086+0000","last_became_active":"2024-05-08T08:55:43.379146+0000","last_became_peered":"2024-05-08T08:55:43.379146+0000","last_unstale":"2024-05-08T08:55:50.555086+0000","last_undegraded":"2024-05-08T08:55:50.555086+0000","last_fullsized":"2024-05-08T08:55:50.555086+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":651,"log_dups_size":0,"ondisk_log_size":651,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:09:48.532237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022701349999999999,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":40,"num_read":11037,"num_read_kb":41101,"num_write":598,"num_write_kb":9071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":589,"num_bytes_recovered":19963261,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":1295,"num_object_clones":0,"num_object_copies":2590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1295,"num_whiteouts":1291,"num_read":512933,"num_read_kb":1628362,"num_write":26100,"num_write_kb":451839,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11018,"num_bytes_recovered":421954137,"num_keys_recovered":904,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24829,"ondisk_log_size":24829,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":18287,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640306,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59884,"kb_used_data":2328,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311956,"statfs":{"total":96636764160,"available":96575442944,"internally_reserved":0,"allocated":2383872,"data_stored":1016789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640310,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55724,"kb_used_data":2332,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316116,"statfs":{"total":96636764160,"available":96579702784,"internally_reserved":0,"allocated":2387968,"data_stored":1016808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640308,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65504,"kb_used_data":2384,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306336,"statfs":{"total":96636764160,"available":96569688064,"internally_reserved":0,"allocated":2441216,"data_stored":1607157,"data_compressed":5523,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1593,"internal_metadata":64616903},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640307,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51560,"kb_used_data":2328,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320280,"statfs":{"total":96636764160,"available":96583966720,"internally_reserved":0,"allocated":2383872,"data_stored":1016789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640306,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65448,"kb_used_data":2332,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306392,"statfs":{"total":96636764160,"available":96569745408,"internally_reserved":0,"allocated":2387968,"data_stored":1016808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640309,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69656,"kb_used_data":2368,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302184,"statfs":{"total":96636764160,"available":96565436416,"internally_reserved":0,"allocated":2424832,"data_stored":1607157,"data_compressed":6382,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6382,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6382,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5523,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T08:55:55.467 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-08T08:55:55.467 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T08:55:55.768 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:55:55.768 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-05-08T08:55:55.780 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":311,"stamp":"2024-05-08T08:55:53.780431+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":1297,"num_object_clones":0,"num_object_copies":2594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1297,"num_whiteouts":1291,"num_read":513049,"num_read_kb":1628462,"num_write":26259,"num_write_kb":454255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11022,"num_bytes_recovered":423134873,"num_keys_recovered":904,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24946,"ondisk_log_size":24946,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":367776,"kb_used_data":14072,"kb_used_omap":9,"kb_used_meta":353590,"kb_avail":565863264,"statfs":{"total":579820584960,"available":579443982336,"internally_reserved":0,"allocated":14409728,"data_stored":7281508,"data_compressed":11905,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":9568,"internal_metadata":362076832},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-743,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001324"},"pg_stats":[{"pgid":"3.1f","version":"182'772","reported_seq":20838,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.562464+0000","last_change":"2024-05-08T08:55:43.388833+0000","last_active":"2024-05-08T08:55:50.562464+0000","last_peered":"2024-05-08T08:55:50.562464+0000","last_clean":"2024-05-08T08:55:50.562464+0000","last_became_active":"2024-05-08T08:55:43.386102+0000","last_became_peered":"2024-05-08T08:55:43.386102+0000","last_unstale":"2024-05-08T08:55:50.562464+0000","last_undegraded":"2024-05-08T08:55:50.562464+0000","last_fullsized":"2024-05-08T08:55:50.562464+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":772,"log_dups_size":0,"ondisk_log_size":772,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:25:19.005824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038328300000000002,"stat_sum":{"num_bytes":0,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":34,"num_read":19333,"num_read_kb":85778,"num_write":988,"num_write_kb":33105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":328,"num_bytes_recovered":19152951,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"181'922","reported_seq":11965,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.554583+0000","last_change":"2024-05-08T08:55:33.482489+0000","last_active":"2024-05-08T08:55:50.554583+0000","last_peered":"2024-05-08T08:55:50.554583+0000","last_clean":"2024-05-08T08:55:50.554583+0000","last_became_active":"2024-05-08T08:52:11.930844+0000","last_became_peered":"2024-05-08T08:52:11.930844+0000","last_unstale":"2024-05-08T08:55:50.554583+0000","last_undegraded":"2024-05-08T08:55:50.554583+0000","last_fullsized":"2024-05-08T08:55:50.554583+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":922,"log_dups_size":0,"ondisk_log_size":922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:34:27.496752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001408816,"stat_sum":{"num_bytes":0,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":49,"num_read":13744,"num_read_kb":37146,"num_write":694,"num_write_kb":12720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"181'753","reported_seq":4867,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.562479+0000","last_change":"2024-05-08T08:55:44.384669+0000","last_active":"2024-05-08T08:55:50.562479+0000","last_peered":"2024-05-08T08:55:50.562479+0000","last_clean":"2024-05-08T08:55:50.562479+0000","last_became_active":"2024-05-08T08:55:44.381069+0000","last_became_peered":"2024-05-08T08:55:44.381069+0000","last_unstale":"2024-05-08T08:55:50.562479+0000","last_undegraded":"2024-05-08T08:55:50.562479+0000","last_fullsized":"2024-05-08T08:55:50.562479+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":753,"log_dups_size":0,"ondisk_log_size":753,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:27:12.956766+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0093620049999999996,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":40,"num_read":8309,"num_read_kb":30535,"num_write":901,"num_write_kb":7282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":570,"num_bytes_recovered":16438479,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"182'740","reported_seq":19620,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.562811+0000","last_change":"2024-05-08T08:55:50.562811+0000","last_active":"2024-05-08T08:55:50.562811+0000","last_peered":"2024-05-08T08:55:50.562811+0000","last_clean":"2024-05-08T08:55:50.562811+0000","last_became_active":"2024-05-08T08:55:50.560370+0000","last_became_peered":"2024-05-08T08:55:50.560370+0000","last_unstale":"2024-05-08T08:55:50.562811+0000","last_undegraded":"2024-05-08T08:55:50.562811+0000","last_fullsized":"2024-05-08T08:55:50.562811+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":740,"log_dups_size":0,"ondisk_log_size":740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:20:09.589464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019815259999999999,"stat_sum":{"num_bytes":0,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":45,"num_read":18641,"num_read_kb":101548,"num_write":811,"num_write_kb":27861,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":609,"num_bytes_recovered":86192377,"num_keys_recovered":338,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"179'637","reported_seq":26061,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.562537+0000","last_change":"2024-05-08T08:55:43.388686+0000","last_active":"2024-05-08T08:55:50.562537+0000","last_peered":"2024-05-08T08:55:50.562537+0000","last_clean":"2024-05-08T08:55:50.562537+0000","last_became_active":"2024-05-08T08:55:43.386098+0000","last_became_peered":"2024-05-08T08:55:43.386098+0000","last_unstale":"2024-05-08T08:55:50.562537+0000","last_undegraded":"2024-05-08T08:55:50.562537+0000","last_fullsized":"2024-05-08T08:55:50.562537+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":637,"log_dups_size":0,"ondisk_log_size":637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:29:43.226913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012631949999999999,"stat_sum":{"num_bytes":0,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":50,"num_read":25112,"num_read_kb":45826,"num_write":688,"num_write_kb":23312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":455,"num_bytes_recovered":5234820,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"180'867","reported_seq":22286,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.559406+0000","last_change":"2024-05-08T08:55:33.475720+0000","last_active":"2024-05-08T08:55:50.559406+0000","last_peered":"2024-05-08T08:55:50.559406+0000","last_clean":"2024-05-08T08:55:50.559406+0000","last_became_active":"2024-05-08T08:53:24.484124+0000","last_became_peered":"2024-05-08T08:53:24.484124+0000","last_unstale":"2024-05-08T08:55:50.559406+0000","last_undegraded":"2024-05-08T08:55:50.559406+0000","last_fullsized":"2024-05-08T08:55:50.559406+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":867,"log_dups_size":0,"ondisk_log_size":867,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:57:41.478541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012989099999999999,"stat_sum":{"num_bytes":19,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":34,"num_read":24563,"num_read_kb":59887,"num_write":908,"num_write_kb":19093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":130,"num_bytes_recovered":8549042,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"178'426","reported_seq":10752,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.557157+0000","last_change":"2024-05-08T08:55:33.470036+0000","last_active":"2024-05-08T08:55:50.557157+0000","last_peered":"2024-05-08T08:55:50.557157+0000","last_clean":"2024-05-08T08:55:50.557157+0000","last_became_active":"2024-05-08T08:53:24.483403+0000","last_became_peered":"2024-05-08T08:53:24.483403+0000","last_unstale":"2024-05-08T08:55:50.557157+0000","last_undegraded":"2024-05-08T08:55:50.557157+0000","last_fullsized":"2024-05-08T08:55:50.557157+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":426,"log_dups_size":0,"ondisk_log_size":426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:23:41.607743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011263099999999999,"stat_sum":{"num_bytes":0,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":48,"num_read":9419,"num_read_kb":48048,"num_write":368,"num_write_kb":16514,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":8814624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"181'711","reported_seq":20343,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.555276+0000","last_change":"2024-05-08T08:55:41.379915+0000","last_active":"2024-05-08T08:55:50.555276+0000","last_peered":"2024-05-08T08:55:50.555276+0000","last_clean":"2024-05-08T08:55:50.555276+0000","last_became_active":"2024-05-08T08:55:41.378182+0000","last_became_peered":"2024-05-08T08:55:41.378182+0000","last_unstale":"2024-05-08T08:55:50.555276+0000","last_undegraded":"2024-05-08T08:55:50.555276+0000","last_fullsized":"2024-05-08T08:55:50.555276+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":711,"log_dups_size":0,"ondisk_log_size":711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:23:17.505431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098929300000000003,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":20665,"num_read_kb":52569,"num_write":672,"num_write_kb":23378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":4852260,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"182'852","reported_seq":19005,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.557278+0000","last_change":"2024-05-08T08:55:40.383627+0000","last_active":"2024-05-08T08:55:50.557278+0000","last_peered":"2024-05-08T08:55:50.557278+0000","last_clean":"2024-05-08T08:55:50.557278+0000","last_became_active":"2024-05-08T08:55:40.381448+0000","last_became_peered":"2024-05-08T08:55:40.381448+0000","last_unstale":"2024-05-08T08:55:50.557278+0000","last_undegraded":"2024-05-08T08:55:50.557278+0000","last_fullsized":"2024-05-08T08:55:50.557278+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:14:47.318689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018475399999999999,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":30,"num_read":20569,"num_read_kb":77197,"num_write":969,"num_write_kb":16935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":381,"num_bytes_recovered":5593379,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"182'1461","reported_seq":21996,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.557382+0000","last_change":"2024-05-08T08:55:44.381999+0000","last_active":"2024-05-08T08:55:50.557382+0000","last_peered":"2024-05-08T08:55:50.557382+0000","last_clean":"2024-05-08T08:55:50.557382+0000","last_became_active":"2024-05-08T08:55:44.379555+0000","last_became_peered":"2024-05-08T08:55:44.379555+0000","last_unstale":"2024-05-08T08:55:50.557382+0000","last_undegraded":"2024-05-08T08:55:50.557382+0000","last_fullsized":"2024-05-08T08:55:50.557382+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":1461,"log_dups_size":0,"ondisk_log_size":1461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:07:01.906180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0045105479999999996,"stat_sum":{"num_bytes":0,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":33,"num_read":25480,"num_read_kb":87263,"num_write":2254,"num_write_kb":27190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":420,"num_bytes_recovered":42698110,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"182'690","reported_seq":17152,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.555076+0000","last_change":"2024-05-08T08:55:42.384249+0000","last_active":"2024-05-08T08:55:50.555076+0000","last_peered":"2024-05-08T08:55:50.555076+0000","last_clean":"2024-05-08T08:55:50.555076+0000","last_became_active":"2024-05-08T08:55:42.382974+0000","last_became_peered":"2024-05-08T08:55:42.382974+0000","last_unstale":"2024-05-08T08:55:50.555076+0000","last_undegraded":"2024-05-08T08:55:50.555076+0000","last_fullsized":"2024-05-08T08:55:50.555076+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":690,"log_dups_size":0,"ondisk_log_size":690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:12:41.796610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029835819999999998,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":39,"num_read":17511,"num_read_kb":53002,"num_write":778,"num_write_kb":8761,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":284,"num_bytes_recovered":1296934,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"180'733","reported_seq":12777,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.555182+0000","last_change":"2024-05-08T08:55:33.469235+0000","last_active":"2024-05-08T08:55:50.555182+0000","last_peered":"2024-05-08T08:55:50.555182+0000","last_clean":"2024-05-08T08:55:50.555182+0000","last_became_active":"2024-05-08T08:53:23.481362+0000","last_became_peered":"2024-05-08T08:53:23.481362+0000","last_unstale":"2024-05-08T08:55:50.555182+0000","last_undegraded":"2024-05-08T08:55:50.555182+0000","last_fullsized":"2024-05-08T08:55:50.555182+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":733,"log_dups_size":0,"ondisk_log_size":733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:15:25.796542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032028149999999999,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":30,"num_read":10890,"num_read_kb":33288,"num_write":663,"num_write_kb":11454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":3612768,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":384,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.555233+0000","last_change":"2024-05-08T08:53:20.362387+0000","last_active":"2024-05-08T08:55:50.555233+0000","last_peered":"2024-05-08T08:55:50.555233+0000","last_clean":"2024-05-08T08:55:50.555233+0000","last_became_active":"2024-05-08T08:53:20.362113+0000","last_became_peered":"2024-05-08T08:53:20.362113+0000","last_unstale":"2024-05-08T08:55:50.555233+0000","last_undegraded":"2024-05-08T08:55:50.555233+0000","last_fullsized":"2024-05-08T08:55:50.555233+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:28:25.204125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"182'652","reported_seq":20646,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.562407+0000","last_change":"2024-05-08T08:55:40.380225+0000","last_active":"2024-05-08T08:55:50.562407+0000","last_peered":"2024-05-08T08:55:50.562407+0000","last_clean":"2024-05-08T08:55:50.562407+0000","last_became_active":"2024-05-08T08:55:40.378157+0000","last_became_peered":"2024-05-08T08:55:40.378157+0000","last_unstale":"2024-05-08T08:55:50.562407+0000","last_undegraded":"2024-05-08T08:55:50.562407+0000","last_fullsized":"2024-05-08T08:55:50.562407+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":652,"log_dups_size":0,"ondisk_log_size":652,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:03:49.159963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019021660000000001,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":40,"num_read":21842,"num_read_kb":88722,"num_write":580,"num_write_kb":17543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":288,"num_bytes_recovered":3368960,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":375,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.557262+0000","last_change":"2024-05-08T08:52:03.896159+0000","last_active":"2024-05-08T08:55:50.557262+0000","last_peered":"2024-05-08T08:55:50.557262+0000","last_clean":"2024-05-08T08:55:50.557262+0000","last_became_active":"2024-05-08T08:52:03.895797+0000","last_became_peered":"2024-05-08T08:52:03.895797+0000","last_unstale":"2024-05-08T08:55:50.557262+0000","last_undegraded":"2024-05-08T08:55:50.557262+0000","last_fullsized":"2024-05-08T08:55:50.557262+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:34:33.513525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"180'518","reported_seq":6927,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.554715+0000","last_change":"2024-05-08T08:55:33.483261+0000","last_active":"2024-05-08T08:55:50.554715+0000","last_peered":"2024-05-08T08:55:50.554715+0000","last_clean":"2024-05-08T08:55:50.554715+0000","last_became_active":"2024-05-08T08:55:08.437577+0000","last_became_peered":"2024-05-08T08:55:08.437577+0000","last_unstale":"2024-05-08T08:55:50.554715+0000","last_undegraded":"2024-05-08T08:55:50.554715+0000","last_fullsized":"2024-05-08T08:55:50.554715+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:55:04.162036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017459470000000001,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":39,"num_read":6769,"num_read_kb":34542,"num_write":419,"num_write_kb":12024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":324,"num_bytes_recovered":15121731,"num_keys_recovered":25,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":379,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.555279+0000","last_change":"2024-05-08T08:54:59.398530+0000","last_active":"2024-05-08T08:55:50.555279+0000","last_peered":"2024-05-08T08:55:50.555279+0000","last_clean":"2024-05-08T08:55:50.555279+0000","last_became_active":"2024-05-08T08:54:59.398181+0000","last_became_peered":"2024-05-08T08:54:59.398181+0000","last_unstale":"2024-05-08T08:55:50.555279+0000","last_undegraded":"2024-05-08T08:55:50.555279+0000","last_fullsized":"2024-05-08T08:55:50.555279+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:24:28.264614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"181'863","reported_seq":27577,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.557404+0000","last_change":"2024-05-08T08:55:43.386319+0000","last_active":"2024-05-08T08:55:50.557404+0000","last_peered":"2024-05-08T08:55:50.557404+0000","last_clean":"2024-05-08T08:55:50.557404+0000","last_became_active":"2024-05-08T08:55:43.384081+0000","last_became_peered":"2024-05-08T08:55:43.384081+0000","last_unstale":"2024-05-08T08:55:50.557404+0000","last_undegraded":"2024-05-08T08:55:50.557404+0000","last_fullsized":"2024-05-08T08:55:50.557404+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":863,"log_dups_size":0,"ondisk_log_size":863,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:02:20.851342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014858339999999999,"stat_sum":{"num_bytes":0,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":48,"num_read":22624,"num_read_kb":52229,"num_write":653,"num_write_kb":12062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":791,"num_bytes_recovered":28697088,"num_keys_recovered":110,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":385,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.557499+0000","last_change":"2024-05-08T08:55:37.500332+0000","last_active":"2024-05-08T08:55:50.557499+0000","last_peered":"2024-05-08T08:55:50.557499+0000","last_clean":"2024-05-08T08:55:50.557499+0000","last_became_active":"2024-05-08T08:55:37.499784+0000","last_became_peered":"2024-05-08T08:55:37.499784+0000","last_unstale":"2024-05-08T08:55:50.557499+0000","last_undegraded":"2024-05-08T08:55:50.557499+0000","last_fullsized":"2024-05-08T08:55:50.557499+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:53:12.058683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"181'946","reported_seq":14829,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.562555+0000","last_change":"2024-05-08T08:55:50.562555+0000","last_active":"2024-05-08T08:55:50.562555+0000","last_peered":"2024-05-08T08:55:50.562555+0000","last_clean":"2024-05-08T08:55:50.562555+0000","last_became_active":"2024-05-08T08:55:50.559883+0000","last_became_peered":"2024-05-08T08:55:50.559883+0000","last_unstale":"2024-05-08T08:55:50.562555+0000","last_undegraded":"2024-05-08T08:55:50.562555+0000","last_fullsized":"2024-05-08T08:55:50.562555+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":946,"log_dups_size":0,"ondisk_log_size":946,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:18:55.639194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013854900000000001,"stat_sum":{"num_bytes":0,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":55,"num_read":15316,"num_read_kb":32869,"num_write":1141,"num_write_kb":10612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":494,"num_bytes_recovered":13827825,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"182'817","reported_seq":12316,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.559335+0000","last_change":"2024-05-08T08:55:41.381790+0000","last_active":"2024-05-08T08:55:50.559335+0000","last_peered":"2024-05-08T08:55:50.559335+0000","last_clean":"2024-05-08T08:55:50.559335+0000","last_became_active":"2024-05-08T08:55:41.380391+0000","last_became_peered":"2024-05-08T08:55:41.380391+0000","last_unstale":"2024-05-08T08:55:50.559335+0000","last_undegraded":"2024-05-08T08:55:50.559335+0000","last_fullsized":"2024-05-08T08:55:50.559335+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":817,"log_dups_size":0,"ondisk_log_size":817,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:59:49.412777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002033212,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":34,"num_read":11854,"num_read_kb":33576,"num_write":867,"num_write_kb":6636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":383,"num_bytes_recovered":9012422,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":353,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.560520+0000","last_change":"2024-05-08T08:55:50.560520+0000","last_active":"2024-05-08T08:55:50.560520+0000","last_peered":"2024-05-08T08:55:50.560520+0000","last_clean":"2024-05-08T08:55:50.560520+0000","last_became_active":"2024-05-08T08:55:50.559612+0000","last_became_peered":"2024-05-08T08:55:50.559612+0000","last_unstale":"2024-05-08T08:55:50.560520+0000","last_undegraded":"2024-05-08T08:55:50.560520+0000","last_fullsized":"2024-05-08T08:55:50.560520+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:51:19.494663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"181'729","reported_seq":14229,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.558114+0000","last_change":"2024-05-08T08:55:33.470800+0000","last_active":"2024-05-08T08:55:50.558114+0000","last_peered":"2024-05-08T08:55:50.558114+0000","last_clean":"2024-05-08T08:55:50.558114+0000","last_became_active":"2024-05-08T08:52:11.928301+0000","last_became_peered":"2024-05-08T08:52:11.928301+0000","last_unstale":"2024-05-08T08:55:50.558114+0000","last_undegraded":"2024-05-08T08:55:50.558114+0000","last_fullsized":"2024-05-08T08:55:50.558114+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":729,"log_dups_size":0,"ondisk_log_size":729,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:03:13.796981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015902449999999999,"stat_sum":{"num_bytes":0,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":37,"num_read":10704,"num_read_kb":44099,"num_write":654,"num_write_kb":16306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":375,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.555092+0000","last_change":"2024-05-08T08:52:03.891768+0000","last_active":"2024-05-08T08:55:50.555092+0000","last_peered":"2024-05-08T08:55:50.555092+0000","last_clean":"2024-05-08T08:55:50.555092+0000","last_became_active":"2024-05-08T08:52:03.891374+0000","last_became_peered":"2024-05-08T08:52:03.891374+0000","last_unstale":"2024-05-08T08:55:50.555092+0000","last_undegraded":"2024-05-08T08:55:50.555092+0000","last_fullsized":"2024-05-08T08:55:50.555092+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:18:10.240492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"182'1161","reported_seq":28197,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.562344+0000","last_change":"2024-05-08T08:55:42.388348+0000","last_active":"2024-05-08T08:55:50.562344+0000","last_peered":"2024-05-08T08:55:50.562344+0000","last_clean":"2024-05-08T08:55:50.562344+0000","last_became_active":"2024-05-08T08:55:42.386239+0000","last_became_peered":"2024-05-08T08:55:42.386239+0000","last_unstale":"2024-05-08T08:55:50.562344+0000","last_undegraded":"2024-05-08T08:55:50.562344+0000","last_fullsized":"2024-05-08T08:55:50.562344+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":1161,"log_dups_size":0,"ondisk_log_size":1161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:06:05.510596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086553400000000005,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":31078,"num_read_kb":56218,"num_write":870,"num_write_kb":9640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":490,"num_bytes_recovered":5023153,"num_keys_recovered":69,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":374,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.554962+0000","last_change":"2024-05-08T08:52:03.891551+0000","last_active":"2024-05-08T08:55:50.554962+0000","last_peered":"2024-05-08T08:55:50.554962+0000","last_clean":"2024-05-08T08:55:50.554962+0000","last_became_active":"2024-05-08T08:52:03.891296+0000","last_became_peered":"2024-05-08T08:52:03.891296+0000","last_unstale":"2024-05-08T08:55:50.554962+0000","last_undegraded":"2024-05-08T08:55:50.554962+0000","last_fullsized":"2024-05-08T08:55:50.554962+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:54.806343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"181'1002","reported_seq":15538,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.557496+0000","last_change":"2024-05-08T08:55:43.382935+0000","last_active":"2024-05-08T08:55:50.557496+0000","last_peered":"2024-05-08T08:55:50.557496+0000","last_clean":"2024-05-08T08:55:50.557496+0000","last_became_active":"2024-05-08T08:55:43.381345+0000","last_became_peered":"2024-05-08T08:55:43.381345+0000","last_unstale":"2024-05-08T08:55:50.557496+0000","last_undegraded":"2024-05-08T08:55:50.557496+0000","last_fullsized":"2024-05-08T08:55:50.557496+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":1002,"log_dups_size":0,"ondisk_log_size":1002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:30:03.146595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012464850000000001,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":39,"num_read":15521,"num_read_kb":63440,"num_write":1271,"num_write_kb":13979,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":635,"num_bytes_recovered":18405521,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":349,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.560138+0000","last_change":"2024-05-08T08:55:50.560138+0000","last_active":"2024-05-08T08:55:50.560138+0000","last_peered":"2024-05-08T08:55:50.560138+0000","last_clean":"2024-05-08T08:55:50.560138+0000","last_became_active":"2024-05-08T08:55:50.559815+0000","last_became_peered":"2024-05-08T08:55:50.559815+0000","last_unstale":"2024-05-08T08:55:50.560138+0000","last_undegraded":"2024-05-08T08:55:50.560138+0000","last_fullsized":"2024-05-08T08:55:50.560138+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:32:21.109548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":556,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.555092+0000","last_change":"2024-05-08T08:55:42.381739+0000","last_active":"2024-05-08T08:55:50.555092+0000","last_peered":"2024-05-08T08:55:50.555092+0000","last_clean":"2024-05-08T08:55:50.555092+0000","last_became_active":"2024-05-08T08:55:42.381316+0000","last_became_peered":"2024-05-08T08:55:42.381316+0000","last_unstale":"2024-05-08T08:55:50.555092+0000","last_undegraded":"2024-05-08T08:55:50.555092+0000","last_fullsized":"2024-05-08T08:55:50.555092+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:51:57.849666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:51:57.849666+0000","last_clean_scrub_stamp":"2024-05-08T08:51:57.849666+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:59:25.417866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"182'398","reported_seq":9838,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.555377+0000","last_change":"2024-05-08T08:55:33.481972+0000","last_active":"2024-05-08T08:55:50.555377+0000","last_peered":"2024-05-08T08:55:50.555377+0000","last_clean":"2024-05-08T08:55:50.555377+0000","last_became_active":"2024-05-08T08:52:11.926975+0000","last_became_peered":"2024-05-08T08:52:11.926975+0000","last_unstale":"2024-05-08T08:55:50.555377+0000","last_undegraded":"2024-05-08T08:55:50.555377+0000","last_fullsized":"2024-05-08T08:55:50.555377+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":398,"log_dups_size":0,"ondisk_log_size":398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:13:12.117653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087585099999999997,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":35,"num_read":8901,"num_read_kb":36556,"num_write":340,"num_write_kb":8985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"180'503","reported_seq":14561,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.559353+0000","last_change":"2024-05-08T08:55:33.475165+0000","last_active":"2024-05-08T08:55:50.559353+0000","last_peered":"2024-05-08T08:55:50.559353+0000","last_clean":"2024-05-08T08:55:50.559353+0000","last_became_active":"2024-05-08T08:53:23.478564+0000","last_became_peered":"2024-05-08T08:53:23.478564+0000","last_unstale":"2024-05-08T08:55:50.559353+0000","last_undegraded":"2024-05-08T08:55:50.559353+0000","last_fullsized":"2024-05-08T08:55:50.559353+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":503,"log_dups_size":0,"ondisk_log_size":503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:32:37.241705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082315600000000002,"stat_sum":{"num_bytes":0,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":42,"num_read":14198,"num_read_kb":62213,"num_write":413,"num_write_kb":16566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":131,"num_bytes_recovered":12377088,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"182'1247","reported_seq":11910,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.554993+0000","last_change":"2024-05-08T08:55:33.468391+0000","last_active":"2024-05-08T08:55:50.554993+0000","last_peered":"2024-05-08T08:55:50.554993+0000","last_clean":"2024-05-08T08:55:50.554993+0000","last_became_active":"2024-05-08T08:53:24.480359+0000","last_became_peered":"2024-05-08T08:53:24.480359+0000","last_unstale":"2024-05-08T08:55:50.554993+0000","last_undegraded":"2024-05-08T08:55:50.554993+0000","last_fullsized":"2024-05-08T08:55:50.554993+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":1247,"log_dups_size":0,"ondisk_log_size":1247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:08:14.677080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025209339999999998,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":40,"num_read":15863,"num_read_kb":29509,"num_write":1451,"num_write_kb":6331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":2175343,"num_keys_recovered":61,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"180'628","reported_seq":12516,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.557293+0000","last_change":"2024-05-08T08:55:42.384129+0000","last_active":"2024-05-08T08:55:50.557293+0000","last_peered":"2024-05-08T08:55:50.557293+0000","last_clean":"2024-05-08T08:55:50.557293+0000","last_became_active":"2024-05-08T08:55:42.382485+0000","last_became_peered":"2024-05-08T08:55:42.382485+0000","last_unstale":"2024-05-08T08:55:50.557293+0000","last_undegraded":"2024-05-08T08:55:50.557293+0000","last_fullsized":"2024-05-08T08:55:50.557293+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":628,"log_dups_size":0,"ondisk_log_size":628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:10:53.437543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001692783,"stat_sum":{"num_bytes":0,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":50,"num_read":11950,"num_read_kb":39515,"num_write":513,"num_write_kb":7725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":332,"num_bytes_recovered":2080800,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"180'378","reported_seq":14238,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.565133+0000","last_change":"2024-05-08T08:55:50.565133+0000","last_active":"2024-05-08T08:55:50.565133+0000","last_peered":"2024-05-08T08:55:50.565133+0000","last_clean":"2024-05-08T08:55:50.565133+0000","last_became_active":"2024-05-08T08:55:50.563411+0000","last_became_peered":"2024-05-08T08:55:50.563411+0000","last_unstale":"2024-05-08T08:55:50.565133+0000","last_undegraded":"2024-05-08T08:55:50.565133+0000","last_fullsized":"2024-05-08T08:55:50.565133+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:22:13.206310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020039379999999998,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":40,"num_read":13099,"num_read_kb":41266,"num_write":323,"num_write_kb":7340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":509,"num_bytes_recovered":9113137,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"180'478","reported_seq":8966,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.555126+0000","last_change":"2024-05-08T08:55:33.469549+0000","last_active":"2024-05-08T08:55:50.555126+0000","last_peered":"2024-05-08T08:55:50.555126+0000","last_clean":"2024-05-08T08:55:50.555126+0000","last_became_active":"2024-05-08T08:52:11.932594+0000","last_became_peered":"2024-05-08T08:52:11.932594+0000","last_unstale":"2024-05-08T08:55:50.555126+0000","last_undegraded":"2024-05-08T08:55:50.555126+0000","last_fullsized":"2024-05-08T08:55:50.555126+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":478,"log_dups_size":0,"ondisk_log_size":478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:53:33.929731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033757140000000001,"stat_sum":{"num_bytes":0,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":43,"num_read":9572,"num_read_kb":34328,"num_write":402,"num_write_kb":10002,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"180'632","reported_seq":9873,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.562760+0000","last_change":"2024-05-08T08:55:50.562760+0000","last_active":"2024-05-08T08:55:50.562760+0000","last_peered":"2024-05-08T08:55:50.562760+0000","last_clean":"2024-05-08T08:55:50.562760+0000","last_became_active":"2024-05-08T08:55:50.560907+0000","last_became_peered":"2024-05-08T08:55:50.560907+0000","last_unstale":"2024-05-08T08:55:50.562760+0000","last_undegraded":"2024-05-08T08:55:50.562760+0000","last_fullsized":"2024-05-08T08:55:50.562760+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":632,"log_dups_size":0,"ondisk_log_size":632,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:04:24.220863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002456153,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":16215,"num_read_kb":47213,"num_write":634,"num_write_kb":12225,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":446,"num_bytes_recovered":28147344,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"182'588","reported_seq":8774,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.557417+0000","last_change":"2024-05-08T08:55:40.386164+0000","last_active":"2024-05-08T08:55:50.557417+0000","last_peered":"2024-05-08T08:55:50.557417+0000","last_clean":"2024-05-08T08:55:50.557417+0000","last_became_active":"2024-05-08T08:55:40.383962+0000","last_became_peered":"2024-05-08T08:55:40.383962+0000","last_unstale":"2024-05-08T08:55:50.557417+0000","last_undegraded":"2024-05-08T08:55:50.557417+0000","last_fullsized":"2024-05-08T08:55:50.557417+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":588,"log_dups_size":0,"ondisk_log_size":588,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:48:37.179607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037436610000000001,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":36,"num_read":8886,"num_read_kb":38380,"num_write":568,"num_write_kb":9633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":675,"num_bytes_recovered":18815289,"num_keys_recovered":79,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"180'749","reported_seq":8638,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.567799+0000","last_change":"2024-05-08T08:55:50.567799+0000","last_active":"2024-05-08T08:55:50.567799+0000","last_peered":"2024-05-08T08:55:50.567799+0000","last_clean":"2024-05-08T08:55:50.567799+0000","last_became_active":"2024-05-08T08:55:50.565716+0000","last_became_peered":"2024-05-08T08:55:50.565716+0000","last_unstale":"2024-05-08T08:55:50.567799+0000","last_undegraded":"2024-05-08T08:55:50.567799+0000","last_fullsized":"2024-05-08T08:55:50.567799+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":749,"log_dups_size":0,"ondisk_log_size":749,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:28:22.726479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095127799999999995,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":40,"num_read":9090,"num_read_kb":41259,"num_write":984,"num_write_kb":14557,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":541,"num_bytes_recovered":26549987,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"182'750","reported_seq":11879,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.554980+0000","last_change":"2024-05-08T08:55:33.467196+0000","last_active":"2024-05-08T08:55:50.554980+0000","last_peered":"2024-05-08T08:55:50.554980+0000","last_clean":"2024-05-08T08:55:50.554980+0000","last_became_active":"2024-05-08T08:53:24.479891+0000","last_became_peered":"2024-05-08T08:53:24.479891+0000","last_unstale":"2024-05-08T08:55:50.554980+0000","last_undegraded":"2024-05-08T08:55:50.554980+0000","last_fullsized":"2024-05-08T08:55:50.554980+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":750,"log_dups_size":0,"ondisk_log_size":750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:11:35.412500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015502269999999999,"stat_sum":{"num_bytes":0,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":43,"num_read":13241,"num_read_kb":36544,"num_write":586,"num_write_kb":10920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":131,"num_bytes_recovered":2604032,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"182'1575","reported_seq":25758,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.562352+0000","last_change":"2024-05-08T08:55:43.389357+0000","last_active":"2024-05-08T08:55:50.562352+0000","last_peered":"2024-05-08T08:55:50.562352+0000","last_clean":"2024-05-08T08:55:50.562352+0000","last_became_active":"2024-05-08T08:55:43.386543+0000","last_became_peered":"2024-05-08T08:55:43.386543+0000","last_unstale":"2024-05-08T08:55:50.562352+0000","last_undegraded":"2024-05-08T08:55:50.562352+0000","last_fullsized":"2024-05-08T08:55:50.562352+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":1575,"log_dups_size":0,"ondisk_log_size":1575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:38:01.859312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015403719999999999,"stat_sum":{"num_bytes":0,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":52,"num_read":30937,"num_read_kb":62696,"num_write":2139,"num_write_kb":12077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":330,"num_bytes_recovered":4235412,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"181'651","reported_seq":11695,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.555086+0000","last_change":"2024-05-08T08:55:43.380556+0000","last_active":"2024-05-08T08:55:50.555086+0000","last_peered":"2024-05-08T08:55:50.555086+0000","last_clean":"2024-05-08T08:55:50.555086+0000","last_became_active":"2024-05-08T08:55:43.379146+0000","last_became_peered":"2024-05-08T08:55:43.379146+0000","last_unstale":"2024-05-08T08:55:50.555086+0000","last_undegraded":"2024-05-08T08:55:50.555086+0000","last_fullsized":"2024-05-08T08:55:50.555086+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":651,"log_dups_size":0,"ondisk_log_size":651,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:09:48.532237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022701349999999999,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":40,"num_read":11037,"num_read_kb":41101,"num_write":598,"num_write_kb":9071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":589,"num_bytes_recovered":19963261,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":1295,"num_object_clones":0,"num_object_copies":2590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1295,"num_whiteouts":1291,"num_read":512933,"num_read_kb":1628362,"num_write":26100,"num_write_kb":451839,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11018,"num_bytes_recovered":421954137,"num_keys_recovered":904,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24829,"ondisk_log_size":24829,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":18287,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640306,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59884,"kb_used_data":2328,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311956,"statfs":{"total":96636764160,"available":96575442944,"internally_reserved":0,"allocated":2383872,"data_stored":1016789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640310,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55724,"kb_used_data":2332,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316116,"statfs":{"total":96636764160,"available":96579702784,"internally_reserved":0,"allocated":2387968,"data_stored":1016808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640308,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65504,"kb_used_data":2384,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306336,"statfs":{"total":96636764160,"available":96569688064,"internally_reserved":0,"allocated":2441216,"data_stored":1607157,"data_compressed":5523,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1593,"internal_metadata":64616903},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640307,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51560,"kb_used_data":2328,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320280,"statfs":{"total":96636764160,"available":96583966720,"internally_reserved":0,"allocated":2383872,"data_stored":1016789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640306,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65448,"kb_used_data":2332,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306392,"statfs":{"total":96636764160,"available":96569745408,"internally_reserved":0,"allocated":2387968,"data_stored":1016808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640309,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69656,"kb_used_data":2368,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302184,"statfs":{"total":96636764160,"available":96565436416,"internally_reserved":0,"allocated":2424832,"data_stored":1607157,"data_compressed":6382,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6382,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6382,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5523,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T08:55:55.782 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T08:55:56.082 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:55:56.082 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":198,"fsid":"38dfae8c-5c5e-4b0f-a264-91f22c6c5554","created":"2024-05-08T08:51:45.571369+0000","modified":"2024-05-08T08:55:50.545876+0000","last_up_change":"2024-05-08T08:51:56.847706+0000","last_in_change":"2024-05-08T08:55:37.484433+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T08:51:57.787790+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-05-08T08:52:02.563677+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-05-08T08:52:09.955467+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"182","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":51,"snap_epoch":182,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.309999942779541,"score_stable":1.309999942779541,"optimal_score":1,"raw_score_acting":1.309999942779541,"raw_score_stable":1.309999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"300801a5-1919-4b23-8c6d-bc7dad61cc4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":802160151},{"type":"v1","addr":"172.21.15.59:6801","nonce":802160151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":802160151},{"type":"v1","addr":"172.21.15.59:6803","nonce":802160151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":802160151},{"type":"v1","addr":"172.21.15.59:6807","nonce":802160151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":802160151},{"type":"v1","addr":"172.21.15.59:6805","nonce":802160151}]},"public_addr":"172.21.15.59:6801/802160151","cluster_addr":"172.21.15.59:6803/802160151","heartbeat_back_addr":"172.21.15.59:6807/802160151","heartbeat_front_addr":"172.21.15.59:6805/802160151","state":["exists","up"]},{"osd":1,"uuid":"0fead0de-6a65-4eba-b400-eea5692702e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":4116801011},{"type":"v1","addr":"172.21.15.59:6817","nonce":4116801011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":4116801011},{"type":"v1","addr":"172.21.15.59:6819","nonce":4116801011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":4116801011},{"type":"v1","addr":"172.21.15.59:6823","nonce":4116801011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":4116801011},{"type":"v1","addr":"172.21.15.59:6821","nonce":4116801011}]},"public_addr":"172.21.15.59:6817/4116801011","cluster_addr":"172.21.15.59:6819/4116801011","heartbeat_back_addr":"172.21.15.59:6823/4116801011","heartbeat_front_addr":"172.21.15.59:6821/4116801011","state":["exists","up"]},{"osd":2,"uuid":"1a233de6-1b82-4a51-8e96-4cd4abdf16c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1547661179},{"type":"v1","addr":"172.21.15.59:6809","nonce":1547661179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1547661179},{"type":"v1","addr":"172.21.15.59:6811","nonce":1547661179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1547661179},{"type":"v1","addr":"172.21.15.59:6815","nonce":1547661179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1547661179},{"type":"v1","addr":"172.21.15.59:6813","nonce":1547661179}]},"public_addr":"172.21.15.59:6809/1547661179","cluster_addr":"172.21.15.59:6811/1547661179","heartbeat_back_addr":"172.21.15.59:6815/1547661179","heartbeat_front_addr":"172.21.15.59:6813/1547661179","state":["exists","up"]},{"osd":3,"uuid":"cc4870b8-dcd9-4e43-978d-8de6c6396abf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":4054400810},{"type":"v1","addr":"172.21.15.192:6801","nonce":4054400810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":4054400810},{"type":"v1","addr":"172.21.15.192:6803","nonce":4054400810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":4054400810},{"type":"v1","addr":"172.21.15.192:6807","nonce":4054400810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":4054400810},{"type":"v1","addr":"172.21.15.192:6805","nonce":4054400810}]},"public_addr":"172.21.15.192:6801/4054400810","cluster_addr":"172.21.15.192:6803/4054400810","heartbeat_back_addr":"172.21.15.192:6807/4054400810","heartbeat_front_addr":"172.21.15.192:6805/4054400810","state":["exists","up"]},{"osd":4,"uuid":"d1d05620-42fb-4107-a2f9-e83542d076ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":58985565},{"type":"v1","addr":"172.21.15.192:6809","nonce":58985565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":58985565},{"type":"v1","addr":"172.21.15.192:6811","nonce":58985565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":58985565},{"type":"v1","addr":"172.21.15.192:6815","nonce":58985565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":58985565},{"type":"v1","addr":"172.21.15.192:6813","nonce":58985565}]},"public_addr":"172.21.15.192:6809/58985565","cluster_addr":"172.21.15.192:6811/58985565","heartbeat_back_addr":"172.21.15.192:6815/58985565","heartbeat_front_addr":"172.21.15.192:6813/58985565","state":["exists","up"]},{"osd":5,"uuid":"ab26136c-31c3-4090-9fa2-988dba4f731f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":1283571573},{"type":"v1","addr":"172.21.15.192:6817","nonce":1283571573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1283571573},{"type":"v1","addr":"172.21.15.192:6819","nonce":1283571573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":1283571573},{"type":"v1","addr":"172.21.15.192:6823","nonce":1283571573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":1283571573},{"type":"v1","addr":"172.21.15.192:6821","nonce":1283571573}]},"public_addr":"172.21.15.192:6817/1283571573","cluster_addr":"172.21.15.192:6819/1283571573","heartbeat_back_addr":"172.21.15.192:6823/1283571573","heartbeat_front_addr":"172.21.15.192:6821/1283571573","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T08:51:52.396946+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T08:51:52.453160+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T08:51:52.387606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T08:51:52.360952+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T08:51:52.468681+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T08:51:52.401080+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"2.5","osds":[3,2]},{"pgid":"3.4","osds":[1,5]},{"pgid":"3.17","osds":[1,0]},{"pgid":"3.1d","osds":[0,5]}],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T08:55:57.096 INFO:tasks.ceph:Scrubbing osd.0 2024-05-08T08:55:57.096 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-05-08T08:55:57.237 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:55:57.237 INFO:teuthology.orchestra.run.smithi059.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T08:55:57.237 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:55:57.248 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-05-08T08:55:57.547 INFO:teuthology.orchestra.run.smithi059.stderr:instructed osd(s) 0 to deep-scrub 2024-05-08T08:55:57.560 INFO:tasks.ceph:Scrubbing osd.1 2024-05-08T08:55:57.560 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-05-08T08:55:57.688 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:55:57.688 INFO:teuthology.orchestra.run.smithi059.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T08:55:57.688 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:55:57.699 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-05-08T08:55:58.002 INFO:teuthology.orchestra.run.smithi059.stderr:instructed osd(s) 1 to deep-scrub 2024-05-08T08:55:58.015 INFO:tasks.ceph:Scrubbing osd.2 2024-05-08T08:55:58.015 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-05-08T08:55:58.142 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:55:58.142 INFO:teuthology.orchestra.run.smithi059.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T08:55:58.142 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:55:58.153 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-05-08T08:55:58.450 INFO:teuthology.orchestra.run.smithi059.stderr:instructed osd(s) 2 to deep-scrub 2024-05-08T08:55:58.462 INFO:tasks.ceph:Scrubbing osd.3 2024-05-08T08:55:58.463 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-05-08T08:55:58.592 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:55:58.592 INFO:teuthology.orchestra.run.smithi059.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T08:55:58.592 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:55:58.603 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-05-08T08:55:58.902 INFO:teuthology.orchestra.run.smithi059.stderr:instructed osd(s) 3 to deep-scrub 2024-05-08T08:55:58.915 INFO:tasks.ceph:Scrubbing osd.4 2024-05-08T08:55:58.915 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-05-08T08:55:59.043 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:55:59.043 INFO:teuthology.orchestra.run.smithi059.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T08:55:59.043 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:55:59.054 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-05-08T08:55:59.350 INFO:teuthology.orchestra.run.smithi059.stderr:instructed osd(s) 4 to deep-scrub 2024-05-08T08:55:59.363 INFO:tasks.ceph:Scrubbing osd.5 2024-05-08T08:55:59.363 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-05-08T08:55:59.491 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T08:55:59.491 INFO:teuthology.orchestra.run.smithi059.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T08:55:59.491 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T08:55:59.502 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-05-08T08:55:59.819 INFO:teuthology.orchestra.run.smithi059.stderr:instructed osd(s) 5 to deep-scrub 2024-05-08T08:55:59.833 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T08:56:00.138 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:56:00.139 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-05-08T08:56:00.150 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":314,"stamp":"2024-05-08T08:55:59.781729+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":1297,"num_object_clones":0,"num_object_copies":2594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1297,"num_whiteouts":1291,"num_read":513049,"num_read_kb":1628462,"num_write":26259,"num_write_kb":454255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11022,"num_bytes_recovered":423134873,"num_keys_recovered":904,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24986,"ondisk_log_size":24986,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":367776,"kb_used_data":14072,"kb_used_omap":9,"kb_used_meta":353590,"kb_avail":565863264,"statfs":{"total":579820584960,"available":579443982336,"internally_reserved":0,"allocated":14409728,"data_stored":7281508,"data_compressed":11905,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":9568,"internal_metadata":362076832},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-743,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002195"},"pg_stats":[{"pgid":"3.1f","version":"182'772","reported_seq":20839,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.562512+0000","last_change":"2024-05-08T08:55:43.388833+0000","last_active":"2024-05-08T08:55:50.562512+0000","last_peered":"2024-05-08T08:55:50.562512+0000","last_clean":"2024-05-08T08:55:50.562512+0000","last_became_active":"2024-05-08T08:55:43.386102+0000","last_became_peered":"2024-05-08T08:55:43.386102+0000","last_unstale":"2024-05-08T08:55:50.562512+0000","last_undegraded":"2024-05-08T08:55:50.562512+0000","last_fullsized":"2024-05-08T08:55:50.562512+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":772,"log_dups_size":0,"ondisk_log_size":772,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038328300000000002,"stat_sum":{"num_bytes":0,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":34,"num_read":19333,"num_read_kb":85778,"num_write":988,"num_write_kb":33105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":328,"num_bytes_recovered":19152951,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"181'922","reported_seq":11965,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.554583+0000","last_change":"2024-05-08T08:55:33.482489+0000","last_active":"2024-05-08T08:55:50.554583+0000","last_peered":"2024-05-08T08:55:50.554583+0000","last_clean":"2024-05-08T08:55:50.554583+0000","last_became_active":"2024-05-08T08:52:11.930844+0000","last_became_peered":"2024-05-08T08:52:11.930844+0000","last_unstale":"2024-05-08T08:55:50.554583+0000","last_undegraded":"2024-05-08T08:55:50.554583+0000","last_fullsized":"2024-05-08T08:55:50.554583+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":922,"log_dups_size":0,"ondisk_log_size":922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:34:27.496752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001408816,"stat_sum":{"num_bytes":0,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":49,"num_read":13744,"num_read_kb":37146,"num_write":694,"num_write_kb":12720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"181'753","reported_seq":4868,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.562521+0000","last_change":"2024-05-08T08:55:44.384669+0000","last_active":"2024-05-08T08:55:50.562521+0000","last_peered":"2024-05-08T08:55:50.562521+0000","last_clean":"2024-05-08T08:55:50.562521+0000","last_became_active":"2024-05-08T08:55:44.381069+0000","last_became_peered":"2024-05-08T08:55:44.381069+0000","last_unstale":"2024-05-08T08:55:50.562521+0000","last_undegraded":"2024-05-08T08:55:50.562521+0000","last_fullsized":"2024-05-08T08:55:50.562521+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":753,"log_dups_size":0,"ondisk_log_size":753,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0093620049999999996,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":40,"num_read":8309,"num_read_kb":30535,"num_write":901,"num_write_kb":7282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":570,"num_bytes_recovered":16438479,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"182'740","reported_seq":19620,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.562811+0000","last_change":"2024-05-08T08:55:50.562811+0000","last_active":"2024-05-08T08:55:50.562811+0000","last_peered":"2024-05-08T08:55:50.562811+0000","last_clean":"2024-05-08T08:55:50.562811+0000","last_became_active":"2024-05-08T08:55:50.560370+0000","last_became_peered":"2024-05-08T08:55:50.560370+0000","last_unstale":"2024-05-08T08:55:50.562811+0000","last_undegraded":"2024-05-08T08:55:50.562811+0000","last_fullsized":"2024-05-08T08:55:50.562811+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":740,"log_dups_size":0,"ondisk_log_size":740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:20:09.589464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019815259999999999,"stat_sum":{"num_bytes":0,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":45,"num_read":18641,"num_read_kb":101548,"num_write":811,"num_write_kb":27861,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":609,"num_bytes_recovered":86192377,"num_keys_recovered":338,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"179'637","reported_seq":26062,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.562583+0000","last_change":"2024-05-08T08:55:43.388686+0000","last_active":"2024-05-08T08:55:50.562583+0000","last_peered":"2024-05-08T08:55:50.562583+0000","last_clean":"2024-05-08T08:55:50.562583+0000","last_became_active":"2024-05-08T08:55:43.386098+0000","last_became_peered":"2024-05-08T08:55:43.386098+0000","last_unstale":"2024-05-08T08:55:50.562583+0000","last_undegraded":"2024-05-08T08:55:50.562583+0000","last_fullsized":"2024-05-08T08:55:50.562583+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":637,"log_dups_size":0,"ondisk_log_size":637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012631949999999999,"stat_sum":{"num_bytes":0,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":50,"num_read":25112,"num_read_kb":45826,"num_write":688,"num_write_kb":23312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":455,"num_bytes_recovered":5234820,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"180'867","reported_seq":22287,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.559459+0000","last_change":"2024-05-08T08:55:33.475720+0000","last_active":"2024-05-08T08:55:50.559459+0000","last_peered":"2024-05-08T08:55:50.559459+0000","last_clean":"2024-05-08T08:55:50.559459+0000","last_became_active":"2024-05-08T08:53:24.484124+0000","last_became_peered":"2024-05-08T08:53:24.484124+0000","last_unstale":"2024-05-08T08:55:50.559459+0000","last_undegraded":"2024-05-08T08:55:50.559459+0000","last_fullsized":"2024-05-08T08:55:50.559459+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":867,"log_dups_size":0,"ondisk_log_size":867,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:57:41.478541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012989099999999999,"stat_sum":{"num_bytes":19,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":34,"num_read":24563,"num_read_kb":59887,"num_write":908,"num_write_kb":19093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":130,"num_bytes_recovered":8549042,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"178'426","reported_seq":10752,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.557157+0000","last_change":"2024-05-08T08:55:33.470036+0000","last_active":"2024-05-08T08:55:50.557157+0000","last_peered":"2024-05-08T08:55:50.557157+0000","last_clean":"2024-05-08T08:55:50.557157+0000","last_became_active":"2024-05-08T08:53:24.483403+0000","last_became_peered":"2024-05-08T08:53:24.483403+0000","last_unstale":"2024-05-08T08:55:50.557157+0000","last_undegraded":"2024-05-08T08:55:50.557157+0000","last_fullsized":"2024-05-08T08:55:50.557157+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":426,"log_dups_size":0,"ondisk_log_size":426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:23:41.607743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011263099999999999,"stat_sum":{"num_bytes":0,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":48,"num_read":9419,"num_read_kb":48048,"num_write":368,"num_write_kb":16514,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":8814624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"181'711","reported_seq":20343,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.555276+0000","last_change":"2024-05-08T08:55:41.379915+0000","last_active":"2024-05-08T08:55:50.555276+0000","last_peered":"2024-05-08T08:55:50.555276+0000","last_clean":"2024-05-08T08:55:50.555276+0000","last_became_active":"2024-05-08T08:55:41.378182+0000","last_became_peered":"2024-05-08T08:55:41.378182+0000","last_unstale":"2024-05-08T08:55:50.555276+0000","last_undegraded":"2024-05-08T08:55:50.555276+0000","last_fullsized":"2024-05-08T08:55:50.555276+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":711,"log_dups_size":0,"ondisk_log_size":711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:23:17.505431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098929300000000003,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":20665,"num_read_kb":52569,"num_write":672,"num_write_kb":23378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":4852260,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"182'852","reported_seq":19005,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.557278+0000","last_change":"2024-05-08T08:55:40.383627+0000","last_active":"2024-05-08T08:55:50.557278+0000","last_peered":"2024-05-08T08:55:50.557278+0000","last_clean":"2024-05-08T08:55:50.557278+0000","last_became_active":"2024-05-08T08:55:40.381448+0000","last_became_peered":"2024-05-08T08:55:40.381448+0000","last_unstale":"2024-05-08T08:55:50.557278+0000","last_undegraded":"2024-05-08T08:55:50.557278+0000","last_fullsized":"2024-05-08T08:55:50.557278+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:14:47.318689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018475399999999999,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":30,"num_read":20569,"num_read_kb":77197,"num_write":969,"num_write_kb":16935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":381,"num_bytes_recovered":5593379,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"182'1461","reported_seq":21996,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.557382+0000","last_change":"2024-05-08T08:55:44.381999+0000","last_active":"2024-05-08T08:55:50.557382+0000","last_peered":"2024-05-08T08:55:50.557382+0000","last_clean":"2024-05-08T08:55:50.557382+0000","last_became_active":"2024-05-08T08:55:44.379555+0000","last_became_peered":"2024-05-08T08:55:44.379555+0000","last_unstale":"2024-05-08T08:55:50.557382+0000","last_undegraded":"2024-05-08T08:55:50.557382+0000","last_fullsized":"2024-05-08T08:55:50.557382+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":1461,"log_dups_size":0,"ondisk_log_size":1461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:07:01.906180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0045105479999999996,"stat_sum":{"num_bytes":0,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":33,"num_read":25480,"num_read_kb":87263,"num_write":2254,"num_write_kb":27190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":420,"num_bytes_recovered":42698110,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"182'690","reported_seq":17152,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.555076+0000","last_change":"2024-05-08T08:55:42.384249+0000","last_active":"2024-05-08T08:55:50.555076+0000","last_peered":"2024-05-08T08:55:50.555076+0000","last_clean":"2024-05-08T08:55:50.555076+0000","last_became_active":"2024-05-08T08:55:42.382974+0000","last_became_peered":"2024-05-08T08:55:42.382974+0000","last_unstale":"2024-05-08T08:55:50.555076+0000","last_undegraded":"2024-05-08T08:55:50.555076+0000","last_fullsized":"2024-05-08T08:55:50.555076+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":690,"log_dups_size":0,"ondisk_log_size":690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:12:41.796610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029835819999999998,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":39,"num_read":17511,"num_read_kb":53002,"num_write":778,"num_write_kb":8761,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":284,"num_bytes_recovered":1296934,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"180'733","reported_seq":12777,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.555182+0000","last_change":"2024-05-08T08:55:33.469235+0000","last_active":"2024-05-08T08:55:50.555182+0000","last_peered":"2024-05-08T08:55:50.555182+0000","last_clean":"2024-05-08T08:55:50.555182+0000","last_became_active":"2024-05-08T08:53:23.481362+0000","last_became_peered":"2024-05-08T08:53:23.481362+0000","last_unstale":"2024-05-08T08:55:50.555182+0000","last_undegraded":"2024-05-08T08:55:50.555182+0000","last_fullsized":"2024-05-08T08:55:50.555182+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":733,"log_dups_size":0,"ondisk_log_size":733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:15:25.796542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032028149999999999,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":30,"num_read":10890,"num_read_kb":33288,"num_write":663,"num_write_kb":11454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":3612768,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":384,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.555233+0000","last_change":"2024-05-08T08:53:20.362387+0000","last_active":"2024-05-08T08:55:50.555233+0000","last_peered":"2024-05-08T08:55:50.555233+0000","last_clean":"2024-05-08T08:55:50.555233+0000","last_became_active":"2024-05-08T08:53:20.362113+0000","last_became_peered":"2024-05-08T08:53:20.362113+0000","last_unstale":"2024-05-08T08:55:50.555233+0000","last_undegraded":"2024-05-08T08:55:50.555233+0000","last_fullsized":"2024-05-08T08:55:50.555233+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:28:25.204125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"198'692","reported_seq":20696,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:57.744296+0000","last_change":"2024-05-08T08:55:57.744343+0000","last_active":"2024-05-08T08:55:57.744296+0000","last_peered":"2024-05-08T08:55:57.744296+0000","last_clean":"2024-05-08T08:55:57.744296+0000","last_became_active":"2024-05-08T08:55:40.378157+0000","last_became_peered":"2024-05-08T08:55:40.378157+0000","last_unstale":"2024-05-08T08:55:57.744296+0000","last_undegraded":"2024-05-08T08:55:57.744296+0000","last_fullsized":"2024-05-08T08:55:57.744296+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'692","last_scrub_stamp":"2024-05-08T08:55:57.744220+0000","last_deep_scrub":"198'692","last_deep_scrub_stamp":"2024-05-08T08:55:57.744220+0000","last_clean_scrub_stamp":"2024-05-08T08:55:57.744220+0000","objects_scrubbed":40,"log_size":692,"log_dups_size":0,"ondisk_log_size":692,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:31:14.921745+0000","scrub_duration":32,"objects_trimmed":0,"snaptrim_duration":0.0019021660000000001,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":40,"num_read":21842,"num_read_kb":88722,"num_write":580,"num_write_kb":17543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":288,"num_bytes_recovered":3368960,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":375,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.557262+0000","last_change":"2024-05-08T08:52:03.896159+0000","last_active":"2024-05-08T08:55:50.557262+0000","last_peered":"2024-05-08T08:55:50.557262+0000","last_clean":"2024-05-08T08:55:50.557262+0000","last_became_active":"2024-05-08T08:52:03.895797+0000","last_became_peered":"2024-05-08T08:52:03.895797+0000","last_unstale":"2024-05-08T08:55:50.557262+0000","last_undegraded":"2024-05-08T08:55:50.557262+0000","last_fullsized":"2024-05-08T08:55:50.557262+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:34:33.513525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"180'518","reported_seq":6927,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.554715+0000","last_change":"2024-05-08T08:55:33.483261+0000","last_active":"2024-05-08T08:55:50.554715+0000","last_peered":"2024-05-08T08:55:50.554715+0000","last_clean":"2024-05-08T08:55:50.554715+0000","last_became_active":"2024-05-08T08:55:08.437577+0000","last_became_peered":"2024-05-08T08:55:08.437577+0000","last_unstale":"2024-05-08T08:55:50.554715+0000","last_undegraded":"2024-05-08T08:55:50.554715+0000","last_fullsized":"2024-05-08T08:55:50.554715+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:55:04.162036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017459470000000001,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":39,"num_read":6769,"num_read_kb":34542,"num_write":419,"num_write_kb":12024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":324,"num_bytes_recovered":15121731,"num_keys_recovered":25,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":379,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.555279+0000","last_change":"2024-05-08T08:54:59.398530+0000","last_active":"2024-05-08T08:55:50.555279+0000","last_peered":"2024-05-08T08:55:50.555279+0000","last_clean":"2024-05-08T08:55:50.555279+0000","last_became_active":"2024-05-08T08:54:59.398181+0000","last_became_peered":"2024-05-08T08:54:59.398181+0000","last_unstale":"2024-05-08T08:55:50.555279+0000","last_undegraded":"2024-05-08T08:55:50.555279+0000","last_fullsized":"2024-05-08T08:55:50.555279+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:24:28.264614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"181'863","reported_seq":27577,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.557404+0000","last_change":"2024-05-08T08:55:43.386319+0000","last_active":"2024-05-08T08:55:50.557404+0000","last_peered":"2024-05-08T08:55:50.557404+0000","last_clean":"2024-05-08T08:55:50.557404+0000","last_became_active":"2024-05-08T08:55:43.384081+0000","last_became_peered":"2024-05-08T08:55:43.384081+0000","last_unstale":"2024-05-08T08:55:50.557404+0000","last_undegraded":"2024-05-08T08:55:50.557404+0000","last_fullsized":"2024-05-08T08:55:50.557404+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":863,"log_dups_size":0,"ondisk_log_size":863,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:02:20.851342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014858339999999999,"stat_sum":{"num_bytes":0,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":48,"num_read":22624,"num_read_kb":52229,"num_write":653,"num_write_kb":12062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":791,"num_bytes_recovered":28697088,"num_keys_recovered":110,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":385,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.557499+0000","last_change":"2024-05-08T08:55:37.500332+0000","last_active":"2024-05-08T08:55:50.557499+0000","last_peered":"2024-05-08T08:55:50.557499+0000","last_clean":"2024-05-08T08:55:50.557499+0000","last_became_active":"2024-05-08T08:55:37.499784+0000","last_became_peered":"2024-05-08T08:55:37.499784+0000","last_unstale":"2024-05-08T08:55:50.557499+0000","last_undegraded":"2024-05-08T08:55:50.557499+0000","last_fullsized":"2024-05-08T08:55:50.557499+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:53:12.058683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"181'946","reported_seq":14829,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.562555+0000","last_change":"2024-05-08T08:55:50.562555+0000","last_active":"2024-05-08T08:55:50.562555+0000","last_peered":"2024-05-08T08:55:50.562555+0000","last_clean":"2024-05-08T08:55:50.562555+0000","last_became_active":"2024-05-08T08:55:50.559883+0000","last_became_peered":"2024-05-08T08:55:50.559883+0000","last_unstale":"2024-05-08T08:55:50.562555+0000","last_undegraded":"2024-05-08T08:55:50.562555+0000","last_fullsized":"2024-05-08T08:55:50.562555+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":946,"log_dups_size":0,"ondisk_log_size":946,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:18:55.639194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013854900000000001,"stat_sum":{"num_bytes":0,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":55,"num_read":15316,"num_read_kb":32869,"num_write":1141,"num_write_kb":10612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":494,"num_bytes_recovered":13827825,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"182'817","reported_seq":12316,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.559335+0000","last_change":"2024-05-08T08:55:41.381790+0000","last_active":"2024-05-08T08:55:50.559335+0000","last_peered":"2024-05-08T08:55:50.559335+0000","last_clean":"2024-05-08T08:55:50.559335+0000","last_became_active":"2024-05-08T08:55:41.380391+0000","last_became_peered":"2024-05-08T08:55:41.380391+0000","last_unstale":"2024-05-08T08:55:50.559335+0000","last_undegraded":"2024-05-08T08:55:50.559335+0000","last_fullsized":"2024-05-08T08:55:50.559335+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":817,"log_dups_size":0,"ondisk_log_size":817,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:59:49.412777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002033212,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":34,"num_read":11854,"num_read_kb":33576,"num_write":867,"num_write_kb":6636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":383,"num_bytes_recovered":9012422,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":353,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.560520+0000","last_change":"2024-05-08T08:55:50.560520+0000","last_active":"2024-05-08T08:55:50.560520+0000","last_peered":"2024-05-08T08:55:50.560520+0000","last_clean":"2024-05-08T08:55:50.560520+0000","last_became_active":"2024-05-08T08:55:50.559612+0000","last_became_peered":"2024-05-08T08:55:50.559612+0000","last_unstale":"2024-05-08T08:55:50.560520+0000","last_undegraded":"2024-05-08T08:55:50.560520+0000","last_fullsized":"2024-05-08T08:55:50.560520+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:51:19.494663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"181'729","reported_seq":14229,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.558114+0000","last_change":"2024-05-08T08:55:33.470800+0000","last_active":"2024-05-08T08:55:50.558114+0000","last_peered":"2024-05-08T08:55:50.558114+0000","last_clean":"2024-05-08T08:55:50.558114+0000","last_became_active":"2024-05-08T08:52:11.928301+0000","last_became_peered":"2024-05-08T08:52:11.928301+0000","last_unstale":"2024-05-08T08:55:50.558114+0000","last_undegraded":"2024-05-08T08:55:50.558114+0000","last_fullsized":"2024-05-08T08:55:50.558114+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":729,"log_dups_size":0,"ondisk_log_size":729,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:03:13.796981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015902449999999999,"stat_sum":{"num_bytes":0,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":37,"num_read":10704,"num_read_kb":44099,"num_write":654,"num_write_kb":16306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":375,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.555092+0000","last_change":"2024-05-08T08:52:03.891768+0000","last_active":"2024-05-08T08:55:50.555092+0000","last_peered":"2024-05-08T08:55:50.555092+0000","last_clean":"2024-05-08T08:55:50.555092+0000","last_became_active":"2024-05-08T08:52:03.891374+0000","last_became_peered":"2024-05-08T08:52:03.891374+0000","last_unstale":"2024-05-08T08:55:50.555092+0000","last_undegraded":"2024-05-08T08:55:50.555092+0000","last_fullsized":"2024-05-08T08:55:50.555092+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:18:10.240492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"182'1161","reported_seq":28198,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.562406+0000","last_change":"2024-05-08T08:55:42.388348+0000","last_active":"2024-05-08T08:55:50.562406+0000","last_peered":"2024-05-08T08:55:50.562406+0000","last_clean":"2024-05-08T08:55:50.562406+0000","last_became_active":"2024-05-08T08:55:42.386239+0000","last_became_peered":"2024-05-08T08:55:42.386239+0000","last_unstale":"2024-05-08T08:55:50.562406+0000","last_undegraded":"2024-05-08T08:55:50.562406+0000","last_fullsized":"2024-05-08T08:55:50.562406+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":1161,"log_dups_size":0,"ondisk_log_size":1161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086553400000000005,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":31078,"num_read_kb":56218,"num_write":870,"num_write_kb":9640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":490,"num_bytes_recovered":5023153,"num_keys_recovered":69,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":374,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.554962+0000","last_change":"2024-05-08T08:52:03.891551+0000","last_active":"2024-05-08T08:55:50.554962+0000","last_peered":"2024-05-08T08:55:50.554962+0000","last_clean":"2024-05-08T08:55:50.554962+0000","last_became_active":"2024-05-08T08:52:03.891296+0000","last_became_peered":"2024-05-08T08:52:03.891296+0000","last_unstale":"2024-05-08T08:55:50.554962+0000","last_undegraded":"2024-05-08T08:55:50.554962+0000","last_fullsized":"2024-05-08T08:55:50.554962+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:54.806343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"181'1002","reported_seq":15538,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.557496+0000","last_change":"2024-05-08T08:55:43.382935+0000","last_active":"2024-05-08T08:55:50.557496+0000","last_peered":"2024-05-08T08:55:50.557496+0000","last_clean":"2024-05-08T08:55:50.557496+0000","last_became_active":"2024-05-08T08:55:43.381345+0000","last_became_peered":"2024-05-08T08:55:43.381345+0000","last_unstale":"2024-05-08T08:55:50.557496+0000","last_undegraded":"2024-05-08T08:55:50.557496+0000","last_fullsized":"2024-05-08T08:55:50.557496+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":1002,"log_dups_size":0,"ondisk_log_size":1002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:30:03.146595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012464850000000001,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":39,"num_read":15521,"num_read_kb":63440,"num_write":1271,"num_write_kb":13979,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":635,"num_bytes_recovered":18405521,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":349,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.560138+0000","last_change":"2024-05-08T08:55:50.560138+0000","last_active":"2024-05-08T08:55:50.560138+0000","last_peered":"2024-05-08T08:55:50.560138+0000","last_clean":"2024-05-08T08:55:50.560138+0000","last_became_active":"2024-05-08T08:55:50.559815+0000","last_became_peered":"2024-05-08T08:55:50.559815+0000","last_unstale":"2024-05-08T08:55:50.560138+0000","last_undegraded":"2024-05-08T08:55:50.560138+0000","last_fullsized":"2024-05-08T08:55:50.560138+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:02.873487+0000","last_clean_scrub_stamp":"2024-05-08T08:52:02.873487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:32:21.109548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":556,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.555092+0000","last_change":"2024-05-08T08:55:42.381739+0000","last_active":"2024-05-08T08:55:50.555092+0000","last_peered":"2024-05-08T08:55:50.555092+0000","last_clean":"2024-05-08T08:55:50.555092+0000","last_became_active":"2024-05-08T08:55:42.381316+0000","last_became_peered":"2024-05-08T08:55:42.381316+0000","last_unstale":"2024-05-08T08:55:50.555092+0000","last_undegraded":"2024-05-08T08:55:50.555092+0000","last_fullsized":"2024-05-08T08:55:50.555092+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:51:57.849666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:51:57.849666+0000","last_clean_scrub_stamp":"2024-05-08T08:51:57.849666+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:59:25.417866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"182'398","reported_seq":9838,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.555377+0000","last_change":"2024-05-08T08:55:33.481972+0000","last_active":"2024-05-08T08:55:50.555377+0000","last_peered":"2024-05-08T08:55:50.555377+0000","last_clean":"2024-05-08T08:55:50.555377+0000","last_became_active":"2024-05-08T08:52:11.926975+0000","last_became_peered":"2024-05-08T08:52:11.926975+0000","last_unstale":"2024-05-08T08:55:50.555377+0000","last_undegraded":"2024-05-08T08:55:50.555377+0000","last_fullsized":"2024-05-08T08:55:50.555377+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":398,"log_dups_size":0,"ondisk_log_size":398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:13:12.117653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087585099999999997,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":35,"num_read":8901,"num_read_kb":36556,"num_write":340,"num_write_kb":8985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"180'503","reported_seq":14561,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.559353+0000","last_change":"2024-05-08T08:55:33.475165+0000","last_active":"2024-05-08T08:55:50.559353+0000","last_peered":"2024-05-08T08:55:50.559353+0000","last_clean":"2024-05-08T08:55:50.559353+0000","last_became_active":"2024-05-08T08:53:23.478564+0000","last_became_peered":"2024-05-08T08:53:23.478564+0000","last_unstale":"2024-05-08T08:55:50.559353+0000","last_undegraded":"2024-05-08T08:55:50.559353+0000","last_fullsized":"2024-05-08T08:55:50.559353+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":503,"log_dups_size":0,"ondisk_log_size":503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:32:37.241705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082315600000000002,"stat_sum":{"num_bytes":0,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":42,"num_read":14198,"num_read_kb":62213,"num_write":413,"num_write_kb":16566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":131,"num_bytes_recovered":12377088,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"182'1247","reported_seq":11910,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.554993+0000","last_change":"2024-05-08T08:55:33.468391+0000","last_active":"2024-05-08T08:55:50.554993+0000","last_peered":"2024-05-08T08:55:50.554993+0000","last_clean":"2024-05-08T08:55:50.554993+0000","last_became_active":"2024-05-08T08:53:24.480359+0000","last_became_peered":"2024-05-08T08:53:24.480359+0000","last_unstale":"2024-05-08T08:55:50.554993+0000","last_undegraded":"2024-05-08T08:55:50.554993+0000","last_fullsized":"2024-05-08T08:55:50.554993+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":1247,"log_dups_size":0,"ondisk_log_size":1247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:08:14.677080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025209339999999998,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":40,"num_read":15863,"num_read_kb":29509,"num_write":1451,"num_write_kb":6331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":2175343,"num_keys_recovered":61,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"180'628","reported_seq":12516,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.557293+0000","last_change":"2024-05-08T08:55:42.384129+0000","last_active":"2024-05-08T08:55:50.557293+0000","last_peered":"2024-05-08T08:55:50.557293+0000","last_clean":"2024-05-08T08:55:50.557293+0000","last_became_active":"2024-05-08T08:55:42.382485+0000","last_became_peered":"2024-05-08T08:55:42.382485+0000","last_unstale":"2024-05-08T08:55:50.557293+0000","last_undegraded":"2024-05-08T08:55:50.557293+0000","last_fullsized":"2024-05-08T08:55:50.557293+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":628,"log_dups_size":0,"ondisk_log_size":628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:10:53.437543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001692783,"stat_sum":{"num_bytes":0,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":50,"num_read":11950,"num_read_kb":39515,"num_write":513,"num_write_kb":7725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":332,"num_bytes_recovered":2080800,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"180'378","reported_seq":14238,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.565133+0000","last_change":"2024-05-08T08:55:50.565133+0000","last_active":"2024-05-08T08:55:50.565133+0000","last_peered":"2024-05-08T08:55:50.565133+0000","last_clean":"2024-05-08T08:55:50.565133+0000","last_became_active":"2024-05-08T08:55:50.563411+0000","last_became_peered":"2024-05-08T08:55:50.563411+0000","last_unstale":"2024-05-08T08:55:50.565133+0000","last_undegraded":"2024-05-08T08:55:50.565133+0000","last_fullsized":"2024-05-08T08:55:50.565133+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:22:13.206310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020039379999999998,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":40,"num_read":13099,"num_read_kb":41266,"num_write":323,"num_write_kb":7340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":509,"num_bytes_recovered":9113137,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"180'478","reported_seq":8966,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.555126+0000","last_change":"2024-05-08T08:55:33.469549+0000","last_active":"2024-05-08T08:55:50.555126+0000","last_peered":"2024-05-08T08:55:50.555126+0000","last_clean":"2024-05-08T08:55:50.555126+0000","last_became_active":"2024-05-08T08:52:11.932594+0000","last_became_peered":"2024-05-08T08:52:11.932594+0000","last_unstale":"2024-05-08T08:55:50.555126+0000","last_undegraded":"2024-05-08T08:55:50.555126+0000","last_fullsized":"2024-05-08T08:55:50.555126+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":478,"log_dups_size":0,"ondisk_log_size":478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:53:33.929731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033757140000000001,"stat_sum":{"num_bytes":0,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":43,"num_read":9572,"num_read_kb":34328,"num_write":402,"num_write_kb":10002,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"180'632","reported_seq":9873,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.562760+0000","last_change":"2024-05-08T08:55:50.562760+0000","last_active":"2024-05-08T08:55:50.562760+0000","last_peered":"2024-05-08T08:55:50.562760+0000","last_clean":"2024-05-08T08:55:50.562760+0000","last_became_active":"2024-05-08T08:55:50.560907+0000","last_became_peered":"2024-05-08T08:55:50.560907+0000","last_unstale":"2024-05-08T08:55:50.562760+0000","last_undegraded":"2024-05-08T08:55:50.562760+0000","last_fullsized":"2024-05-08T08:55:50.562760+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":632,"log_dups_size":0,"ondisk_log_size":632,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:04:24.220863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002456153,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":16215,"num_read_kb":47213,"num_write":634,"num_write_kb":12225,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":446,"num_bytes_recovered":28147344,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"182'588","reported_seq":8774,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.557417+0000","last_change":"2024-05-08T08:55:40.386164+0000","last_active":"2024-05-08T08:55:50.557417+0000","last_peered":"2024-05-08T08:55:50.557417+0000","last_clean":"2024-05-08T08:55:50.557417+0000","last_became_active":"2024-05-08T08:55:40.383962+0000","last_became_peered":"2024-05-08T08:55:40.383962+0000","last_unstale":"2024-05-08T08:55:50.557417+0000","last_undegraded":"2024-05-08T08:55:50.557417+0000","last_fullsized":"2024-05-08T08:55:50.557417+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":588,"log_dups_size":0,"ondisk_log_size":588,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:48:37.179607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037436610000000001,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":36,"num_read":8886,"num_read_kb":38380,"num_write":568,"num_write_kb":9633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":675,"num_bytes_recovered":18815289,"num_keys_recovered":79,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"180'749","reported_seq":8638,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.567799+0000","last_change":"2024-05-08T08:55:50.567799+0000","last_active":"2024-05-08T08:55:50.567799+0000","last_peered":"2024-05-08T08:55:50.567799+0000","last_clean":"2024-05-08T08:55:50.567799+0000","last_became_active":"2024-05-08T08:55:50.565716+0000","last_became_peered":"2024-05-08T08:55:50.565716+0000","last_unstale":"2024-05-08T08:55:50.567799+0000","last_undegraded":"2024-05-08T08:55:50.567799+0000","last_fullsized":"2024-05-08T08:55:50.567799+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":749,"log_dups_size":0,"ondisk_log_size":749,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:28:22.726479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095127799999999995,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":40,"num_read":9090,"num_read_kb":41259,"num_write":984,"num_write_kb":14557,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":541,"num_bytes_recovered":26549987,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"182'750","reported_seq":11879,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.554980+0000","last_change":"2024-05-08T08:55:33.467196+0000","last_active":"2024-05-08T08:55:50.554980+0000","last_peered":"2024-05-08T08:55:50.554980+0000","last_clean":"2024-05-08T08:55:50.554980+0000","last_became_active":"2024-05-08T08:53:24.479891+0000","last_became_peered":"2024-05-08T08:53:24.479891+0000","last_unstale":"2024-05-08T08:55:50.554980+0000","last_undegraded":"2024-05-08T08:55:50.554980+0000","last_fullsized":"2024-05-08T08:55:50.554980+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":750,"log_dups_size":0,"ondisk_log_size":750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:11:35.412500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015502269999999999,"stat_sum":{"num_bytes":0,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":43,"num_read":13241,"num_read_kb":36544,"num_write":586,"num_write_kb":10920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":131,"num_bytes_recovered":2604032,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"182'1575","reported_seq":25759,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.562410+0000","last_change":"2024-05-08T08:55:43.389357+0000","last_active":"2024-05-08T08:55:50.562410+0000","last_peered":"2024-05-08T08:55:50.562410+0000","last_clean":"2024-05-08T08:55:50.562410+0000","last_became_active":"2024-05-08T08:55:43.386543+0000","last_became_peered":"2024-05-08T08:55:43.386543+0000","last_unstale":"2024-05-08T08:55:50.562410+0000","last_undegraded":"2024-05-08T08:55:50.562410+0000","last_fullsized":"2024-05-08T08:55:50.562410+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":1575,"log_dups_size":0,"ondisk_log_size":1575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015403719999999999,"stat_sum":{"num_bytes":0,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":52,"num_read":30937,"num_read_kb":62696,"num_write":2139,"num_write_kb":12077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":330,"num_bytes_recovered":4235412,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"181'651","reported_seq":11695,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:50.555086+0000","last_change":"2024-05-08T08:55:43.380556+0000","last_active":"2024-05-08T08:55:50.555086+0000","last_peered":"2024-05-08T08:55:50.555086+0000","last_clean":"2024-05-08T08:55:50.555086+0000","last_became_active":"2024-05-08T08:55:43.379146+0000","last_became_peered":"2024-05-08T08:55:43.379146+0000","last_unstale":"2024-05-08T08:55:50.555086+0000","last_undegraded":"2024-05-08T08:55:50.555086+0000","last_fullsized":"2024-05-08T08:55:50.555086+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:52:10.912114+0000","last_clean_scrub_stamp":"2024-05-08T08:52:10.912114+0000","objects_scrubbed":0,"log_size":651,"log_dups_size":0,"ondisk_log_size":651,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:09:48.532237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022701349999999999,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":40,"num_read":11037,"num_read_kb":41101,"num_write":598,"num_write_kb":9071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":589,"num_bytes_recovered":19963261,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":1295,"num_object_clones":0,"num_object_copies":2590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1295,"num_whiteouts":1291,"num_read":512933,"num_read_kb":1628362,"num_write":26100,"num_write_kb":451839,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11018,"num_bytes_recovered":421954137,"num_keys_recovered":904,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24869,"ondisk_log_size":24869,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":18287,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640307,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59884,"kb_used_data":2328,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311956,"statfs":{"total":96636764160,"available":96575442944,"internally_reserved":0,"allocated":2383872,"data_stored":1016789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640311,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55724,"kb_used_data":2332,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316116,"statfs":{"total":96636764160,"available":96579702784,"internally_reserved":0,"allocated":2387968,"data_stored":1016808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640309,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65504,"kb_used_data":2384,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306336,"statfs":{"total":96636764160,"available":96569688064,"internally_reserved":0,"allocated":2441216,"data_stored":1607157,"data_compressed":5523,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1593,"internal_metadata":64616903},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640308,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51560,"kb_used_data":2328,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320280,"statfs":{"total":96636764160,"available":96583966720,"internally_reserved":0,"allocated":2383872,"data_stored":1016789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640307,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65448,"kb_used_data":2332,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306392,"statfs":{"total":96636764160,"available":96569745408,"internally_reserved":0,"allocated":2387968,"data_stored":1016808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640310,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69656,"kb_used_data":2368,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302184,"statfs":{"total":96636764160,"available":96565436416,"internally_reserved":0,"allocated":2424832,"data_stored":1607157,"data_compressed":6382,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6382,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6382,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5523,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T08:56:00.155 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-05-08T08:52:10.912114+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=52, tm_sec=10, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=55, tm_sec=56, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:56:00.155 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-05-08T08:52:10.912114+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=52, tm_sec=10, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=55, tm_sec=56, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:56:00.155 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-05-08T08:52:10.912114+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=52, tm_sec=10, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=55, tm_sec=56, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:56:00.155 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-05-08T08:52:10.912114+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=52, tm_sec=10, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=55, tm_sec=56, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:56:00.155 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-05-08T08:52:10.912114+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=52, tm_sec=10, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=55, tm_sec=56, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:56:00.156 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-05-08T08:52:10.912114+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=52, tm_sec=10, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=55, tm_sec=56, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:56:00.156 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-05-08T08:52:10.912114+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=52, tm_sec=10, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=55, tm_sec=56, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:56:00.156 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-05-08T08:52:10.912114+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=52, tm_sec=10, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=55, tm_sec=56, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:56:00.156 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-05-08T08:52:10.912114+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=52, tm_sec=10, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=55, tm_sec=56, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:56:00.156 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-05-08T08:52:10.912114+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=52, tm_sec=10, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=55, tm_sec=56, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:56:00.156 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-05-08T08:52:10.912114+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=52, tm_sec=10, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=55, tm_sec=56, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:56:00.156 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-05-08T08:52:10.912114+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=52, tm_sec=10, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=55, tm_sec=56, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:56:00.156 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-05-08T08:52:02.873487+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=52, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=55, tm_sec=56, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:56:00.156 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-05-08T08:52:02.873487+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=52, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=55, tm_sec=56, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:56:00.157 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-05-08T08:52:10.912114+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=52, tm_sec=10, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=55, tm_sec=56, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:56:00.157 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-05-08T08:52:02.873487+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=52, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=55, tm_sec=56, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:56:00.157 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-05-08T08:52:10.912114+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=52, tm_sec=10, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=55, tm_sec=56, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:56:00.157 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-05-08T08:52:02.873487+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=52, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=55, tm_sec=56, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:56:00.157 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-05-08T08:52:10.912114+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=52, tm_sec=10, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=55, tm_sec=56, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:56:00.157 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-05-08T08:52:10.912114+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=52, tm_sec=10, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=55, tm_sec=56, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:56:00.157 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-05-08T08:52:02.873487+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=52, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=55, tm_sec=56, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:56:00.157 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-05-08T08:52:10.912114+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=52, tm_sec=10, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=55, tm_sec=56, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:56:00.157 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-05-08T08:52:02.873487+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=52, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=55, tm_sec=56, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:56:00.157 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-05-08T08:52:10.912114+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=52, tm_sec=10, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=55, tm_sec=56, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:56:00.158 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-05-08T08:52:02.873487+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=52, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=55, tm_sec=56, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:56:00.158 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-05-08T08:52:10.912114+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=52, tm_sec=10, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=55, tm_sec=56, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:56:00.158 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-05-08T08:52:02.873487+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=52, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=55, tm_sec=56, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:56:00.158 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-05-08T08:51:57.849666+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=51, tm_sec=57, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=55, tm_sec=56, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:56:00.158 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-05-08T08:52:10.912114+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=52, tm_sec=10, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=55, tm_sec=56, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:56:00.158 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-05-08T08:52:10.912114+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=52, tm_sec=10, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=55, tm_sec=56, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:56:00.158 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-05-08T08:52:10.912114+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=52, tm_sec=10, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=55, tm_sec=56, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:56:00.158 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-05-08T08:52:10.912114+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=52, tm_sec=10, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=55, tm_sec=56, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:56:00.158 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-05-08T08:52:10.912114+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=52, tm_sec=10, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=55, tm_sec=56, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:56:00.158 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-05-08T08:52:10.912114+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=52, tm_sec=10, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=55, tm_sec=56, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:56:00.159 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-05-08T08:52:10.912114+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=52, tm_sec=10, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=55, tm_sec=56, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:56:00.159 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-05-08T08:52:10.912114+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=52, tm_sec=10, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=55, tm_sec=56, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:56:00.159 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-05-08T08:52:10.912114+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=52, tm_sec=10, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=55, tm_sec=56, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:56:00.159 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-05-08T08:52:10.912114+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=52, tm_sec=10, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=55, tm_sec=56, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:56:00.159 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-05-08T08:52:10.912114+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=52, tm_sec=10, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=55, tm_sec=56, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:56:00.159 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-05-08T08:52:10.912114+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=52, tm_sec=10, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=55, tm_sec=56, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:56:00.159 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-05-08T08:56:20.160 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T08:56:20.473 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T08:56:20.474 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-05-08T08:56:20.485 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":324,"stamp":"2024-05-08T08:56:19.786166+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":1297,"num_object_clones":0,"num_object_copies":2594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1297,"num_whiteouts":1291,"num_read":513049,"num_read_kb":1628462,"num_write":26259,"num_write_kb":454255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11022,"num_bytes_recovered":423134873,"num_keys_recovered":904,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":26242,"ondisk_log_size":26242,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":394088,"kb_used_data":14072,"kb_used_omap":9,"kb_used_meta":379958,"kb_avail":565836952,"statfs":{"total":579820584960,"available":579417038848,"internally_reserved":0,"allocated":14409728,"data_stored":7281508,"data_compressed":11905,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":9568,"internal_metadata":389077664},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002829"},"pg_stats":[{"pgid":"3.1f","version":"198'806","reported_seq":20882,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:56:00.756554+0000","last_change":"2024-05-08T08:56:00.756625+0000","last_active":"2024-05-08T08:56:00.756554+0000","last_peered":"2024-05-08T08:56:00.756554+0000","last_clean":"2024-05-08T08:56:00.756554+0000","last_became_active":"2024-05-08T08:55:43.386102+0000","last_became_peered":"2024-05-08T08:55:43.386102+0000","last_unstale":"2024-05-08T08:56:00.756554+0000","last_undegraded":"2024-05-08T08:56:00.756554+0000","last_fullsized":"2024-05-08T08:56:00.756554+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'806","last_scrub_stamp":"2024-05-08T08:56:00.756473+0000","last_deep_scrub":"198'806","last_deep_scrub_stamp":"2024-05-08T08:56:00.756473+0000","last_clean_scrub_stamp":"2024-05-08T08:56:00.756473+0000","objects_scrubbed":34,"log_size":806,"log_dups_size":0,"ondisk_log_size":806,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:22:12.600590+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0.0038328300000000002,"stat_sum":{"num_bytes":0,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":34,"num_read":19333,"num_read_kb":85778,"num_write":988,"num_write_kb":33105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":328,"num_bytes_recovered":19152951,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"198'971","reported_seq":12025,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:56:00.556902+0000","last_change":"2024-05-08T08:56:00.556902+0000","last_active":"2024-05-08T08:56:00.556902+0000","last_peered":"2024-05-08T08:56:00.556902+0000","last_clean":"2024-05-08T08:56:00.556902+0000","last_became_active":"2024-05-08T08:52:11.930844+0000","last_became_peered":"2024-05-08T08:52:11.930844+0000","last_unstale":"2024-05-08T08:56:00.556902+0000","last_undegraded":"2024-05-08T08:56:00.556902+0000","last_fullsized":"2024-05-08T08:56:00.556902+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'971","last_scrub_stamp":"2024-05-08T08:56:00.556768+0000","last_deep_scrub":"198'971","last_deep_scrub_stamp":"2024-05-08T08:56:00.556768+0000","last_clean_scrub_stamp":"2024-05-08T08:56:00.556768+0000","objects_scrubbed":49,"log_size":971,"log_dups_size":0,"ondisk_log_size":971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:49:27.801757+0000","scrub_duration":36,"objects_trimmed":0,"snaptrim_duration":0.001408816,"stat_sum":{"num_bytes":0,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":49,"num_read":13744,"num_read_kb":37146,"num_write":694,"num_write_kb":12720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"198'793","reported_seq":4918,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:56:02.806219+0000","last_change":"2024-05-08T08:56:02.806259+0000","last_active":"2024-05-08T08:56:02.806219+0000","last_peered":"2024-05-08T08:56:02.806219+0000","last_clean":"2024-05-08T08:56:02.806219+0000","last_became_active":"2024-05-08T08:55:44.381069+0000","last_became_peered":"2024-05-08T08:55:44.381069+0000","last_unstale":"2024-05-08T08:56:02.806219+0000","last_undegraded":"2024-05-08T08:56:02.806219+0000","last_fullsized":"2024-05-08T08:56:02.806219+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'793","last_scrub_stamp":"2024-05-08T08:56:02.806128+0000","last_deep_scrub":"198'793","last_deep_scrub_stamp":"2024-05-08T08:56:02.806128+0000","last_clean_scrub_stamp":"2024-05-08T08:56:02.806128+0000","objects_scrubbed":40,"log_size":793,"log_dups_size":0,"ondisk_log_size":793,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:26:21.182258+0000","scrub_duration":29,"objects_trimmed":2,"snaptrim_duration":0.0093620049999999996,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":40,"num_read":8309,"num_read_kb":30535,"num_write":901,"num_write_kb":7282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":570,"num_bytes_recovered":16438479,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"198'786","reported_seq":19676,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:56:06.710718+0000","last_change":"2024-05-08T08:56:06.710784+0000","last_active":"2024-05-08T08:56:06.710718+0000","last_peered":"2024-05-08T08:56:06.710718+0000","last_clean":"2024-05-08T08:56:06.710718+0000","last_became_active":"2024-05-08T08:55:50.560370+0000","last_became_peered":"2024-05-08T08:55:50.560370+0000","last_unstale":"2024-05-08T08:56:06.710718+0000","last_undegraded":"2024-05-08T08:56:06.710718+0000","last_fullsized":"2024-05-08T08:56:06.710718+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'786","last_scrub_stamp":"2024-05-08T08:56:06.710632+0000","last_deep_scrub":"198'786","last_deep_scrub_stamp":"2024-05-08T08:56:06.710632+0000","last_clean_scrub_stamp":"2024-05-08T08:56:06.710632+0000","objects_scrubbed":46,"log_size":786,"log_dups_size":0,"ondisk_log_size":786,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:52:18.809107+0000","scrub_duration":37,"objects_trimmed":0,"snaptrim_duration":0.0019815259999999999,"stat_sum":{"num_bytes":0,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":45,"num_read":18641,"num_read_kb":101548,"num_write":811,"num_write_kb":27861,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":609,"num_bytes_recovered":86192377,"num_keys_recovered":338,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"198'687","reported_seq":26122,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:59.739130+0000","last_change":"2024-05-08T08:55:59.739156+0000","last_active":"2024-05-08T08:55:59.739130+0000","last_peered":"2024-05-08T08:55:59.739130+0000","last_clean":"2024-05-08T08:55:59.739130+0000","last_became_active":"2024-05-08T08:55:43.386098+0000","last_became_peered":"2024-05-08T08:55:43.386098+0000","last_unstale":"2024-05-08T08:55:59.739130+0000","last_undegraded":"2024-05-08T08:55:59.739130+0000","last_fullsized":"2024-05-08T08:55:59.739130+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'687","last_scrub_stamp":"2024-05-08T08:55:59.739059+0000","last_deep_scrub":"198'687","last_deep_scrub_stamp":"2024-05-08T08:55:59.739059+0000","last_clean_scrub_stamp":"2024-05-08T08:55:59.739059+0000","objects_scrubbed":50,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:34:11.034433+0000","scrub_duration":37,"objects_trimmed":0,"snaptrim_duration":0.0012631949999999999,"stat_sum":{"num_bytes":0,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":50,"num_read":25112,"num_read_kb":45826,"num_write":688,"num_write_kb":23312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":455,"num_bytes_recovered":5234820,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"198'902","reported_seq":22332,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:56:00.723362+0000","last_change":"2024-05-08T08:56:00.723404+0000","last_active":"2024-05-08T08:56:00.723362+0000","last_peered":"2024-05-08T08:56:00.723362+0000","last_clean":"2024-05-08T08:56:00.723362+0000","last_became_active":"2024-05-08T08:53:24.484124+0000","last_became_peered":"2024-05-08T08:53:24.484124+0000","last_unstale":"2024-05-08T08:56:00.723362+0000","last_undegraded":"2024-05-08T08:56:00.723362+0000","last_fullsized":"2024-05-08T08:56:00.723362+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'902","last_scrub_stamp":"2024-05-08T08:56:00.723282+0000","last_deep_scrub":"198'902","last_deep_scrub_stamp":"2024-05-08T08:56:00.723282+0000","last_clean_scrub_stamp":"2024-05-08T08:56:00.723282+0000","objects_scrubbed":35,"log_size":902,"log_dups_size":0,"ondisk_log_size":902,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:32:04.753572+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0.0012989099999999999,"stat_sum":{"num_bytes":19,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":34,"num_read":24563,"num_read_kb":59887,"num_write":908,"num_write_kb":19093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":130,"num_bytes_recovered":8549042,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"198'474","reported_seq":10810,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:56:00.951273+0000","last_change":"2024-05-08T08:56:00.951313+0000","last_active":"2024-05-08T08:56:00.951273+0000","last_peered":"2024-05-08T08:56:00.951273+0000","last_clean":"2024-05-08T08:56:00.951273+0000","last_became_active":"2024-05-08T08:53:24.483403+0000","last_became_peered":"2024-05-08T08:53:24.483403+0000","last_unstale":"2024-05-08T08:56:00.951273+0000","last_undegraded":"2024-05-08T08:56:00.951273+0000","last_fullsized":"2024-05-08T08:56:00.951273+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'474","last_scrub_stamp":"2024-05-08T08:56:00.951203+0000","last_deep_scrub":"198'474","last_deep_scrub_stamp":"2024-05-08T08:56:00.951203+0000","last_clean_scrub_stamp":"2024-05-08T08:56:00.951203+0000","objects_scrubbed":48,"log_size":474,"log_dups_size":0,"ondisk_log_size":474,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:25:30.444211+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0.0011263099999999999,"stat_sum":{"num_bytes":0,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":48,"num_read":9419,"num_read_kb":48048,"num_write":368,"num_write_kb":16514,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":8814624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"198'752","reported_seq":20394,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:56:02.579431+0000","last_change":"2024-05-08T08:56:02.579474+0000","last_active":"2024-05-08T08:56:02.579431+0000","last_peered":"2024-05-08T08:56:02.579431+0000","last_clean":"2024-05-08T08:56:02.579431+0000","last_became_active":"2024-05-08T08:55:41.378182+0000","last_became_peered":"2024-05-08T08:55:41.378182+0000","last_unstale":"2024-05-08T08:56:02.579431+0000","last_undegraded":"2024-05-08T08:56:02.579431+0000","last_fullsized":"2024-05-08T08:56:02.579431+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'752","last_scrub_stamp":"2024-05-08T08:56:02.579346+0000","last_deep_scrub":"198'752","last_deep_scrub_stamp":"2024-05-08T08:56:02.579346+0000","last_clean_scrub_stamp":"2024-05-08T08:56:02.579346+0000","objects_scrubbed":41,"log_size":752,"log_dups_size":0,"ondisk_log_size":752,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:25:06.429103+0000","scrub_duration":39,"objects_trimmed":0,"snaptrim_duration":0.00098929300000000003,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":20665,"num_read_kb":52569,"num_write":672,"num_write_kb":23378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":4852260,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"198'882","reported_seq":19045,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:56:02.869526+0000","last_change":"2024-05-08T08:56:02.869561+0000","last_active":"2024-05-08T08:56:02.869526+0000","last_peered":"2024-05-08T08:56:02.869526+0000","last_clean":"2024-05-08T08:56:02.869526+0000","last_became_active":"2024-05-08T08:55:40.381448+0000","last_became_peered":"2024-05-08T08:55:40.381448+0000","last_unstale":"2024-05-08T08:56:02.869526+0000","last_undegraded":"2024-05-08T08:56:02.869526+0000","last_fullsized":"2024-05-08T08:56:02.869526+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'882","last_scrub_stamp":"2024-05-08T08:56:02.869448+0000","last_deep_scrub":"198'882","last_deep_scrub_stamp":"2024-05-08T08:56:02.869448+0000","last_clean_scrub_stamp":"2024-05-08T08:56:02.869448+0000","objects_scrubbed":30,"log_size":882,"log_dups_size":0,"ondisk_log_size":882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:34:15.517600+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0.0018475399999999999,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":30,"num_read":20569,"num_read_kb":77197,"num_write":969,"num_write_kb":16935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":381,"num_bytes_recovered":5593379,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"198'1494","reported_seq":22039,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:56:01.769741+0000","last_change":"2024-05-08T08:56:01.769795+0000","last_active":"2024-05-08T08:56:01.769741+0000","last_peered":"2024-05-08T08:56:01.769741+0000","last_clean":"2024-05-08T08:56:01.769741+0000","last_became_active":"2024-05-08T08:55:44.379555+0000","last_became_peered":"2024-05-08T08:55:44.379555+0000","last_unstale":"2024-05-08T08:56:01.769741+0000","last_undegraded":"2024-05-08T08:56:01.769741+0000","last_fullsized":"2024-05-08T08:56:01.769741+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'1494","last_scrub_stamp":"2024-05-08T08:56:01.769651+0000","last_deep_scrub":"198'1494","last_deep_scrub_stamp":"2024-05-08T08:56:01.769651+0000","last_clean_scrub_stamp":"2024-05-08T08:56:01.769651+0000","objects_scrubbed":33,"log_size":1494,"log_dups_size":0,"ondisk_log_size":1494,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:02:59.082397+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0.0045105479999999996,"stat_sum":{"num_bytes":0,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":33,"num_read":25480,"num_read_kb":87263,"num_write":2254,"num_write_kb":27190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":420,"num_bytes_recovered":42698110,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"198'729","reported_seq":17201,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:56:07.261392+0000","last_change":"2024-05-08T08:56:07.261431+0000","last_active":"2024-05-08T08:56:07.261392+0000","last_peered":"2024-05-08T08:56:07.261392+0000","last_clean":"2024-05-08T08:56:07.261392+0000","last_became_active":"2024-05-08T08:55:42.382974+0000","last_became_peered":"2024-05-08T08:55:42.382974+0000","last_unstale":"2024-05-08T08:56:07.261392+0000","last_undegraded":"2024-05-08T08:56:07.261392+0000","last_fullsized":"2024-05-08T08:56:07.261392+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'729","last_scrub_stamp":"2024-05-08T08:56:07.261312+0000","last_deep_scrub":"198'729","last_deep_scrub_stamp":"2024-05-08T08:56:07.261312+0000","last_clean_scrub_stamp":"2024-05-08T08:56:07.261312+0000","objects_scrubbed":39,"log_size":729,"log_dups_size":0,"ondisk_log_size":729,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:26:07.413563+0000","scrub_duration":25,"objects_trimmed":0,"snaptrim_duration":0.0029835819999999998,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":39,"num_read":17511,"num_read_kb":53002,"num_write":778,"num_write_kb":8761,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":284,"num_bytes_recovered":1296934,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"198'763","reported_seq":12817,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:56:02.304403+0000","last_change":"2024-05-08T08:56:02.304439+0000","last_active":"2024-05-08T08:56:02.304403+0000","last_peered":"2024-05-08T08:56:02.304403+0000","last_clean":"2024-05-08T08:56:02.304403+0000","last_became_active":"2024-05-08T08:53:23.481362+0000","last_became_peered":"2024-05-08T08:53:23.481362+0000","last_unstale":"2024-05-08T08:56:02.304403+0000","last_undegraded":"2024-05-08T08:56:02.304403+0000","last_fullsized":"2024-05-08T08:56:02.304403+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'763","last_scrub_stamp":"2024-05-08T08:56:02.304321+0000","last_deep_scrub":"198'763","last_deep_scrub_stamp":"2024-05-08T08:56:02.304321+0000","last_clean_scrub_stamp":"2024-05-08T08:56:02.304321+0000","objects_scrubbed":30,"log_size":763,"log_dups_size":0,"ondisk_log_size":763,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:09:18.197629+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0.0032028149999999999,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":30,"num_read":10890,"num_read_kb":33288,"num_write":663,"num_write_kb":11454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":3612768,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":393,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:56:01.286174+0000","last_change":"2024-05-08T08:56:01.286197+0000","last_active":"2024-05-08T08:56:01.286174+0000","last_peered":"2024-05-08T08:56:01.286174+0000","last_clean":"2024-05-08T08:56:01.286174+0000","last_became_active":"2024-05-08T08:53:20.362113+0000","last_became_peered":"2024-05-08T08:53:20.362113+0000","last_unstale":"2024-05-08T08:56:01.286174+0000","last_undegraded":"2024-05-08T08:56:01.286174+0000","last_fullsized":"2024-05-08T08:56:01.286174+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:56:01.286087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:56:01.286087+0000","last_clean_scrub_stamp":"2024-05-08T08:56:01.286087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:40:20.646851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"198'692","reported_seq":20696,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:57.744296+0000","last_change":"2024-05-08T08:55:57.744343+0000","last_active":"2024-05-08T08:55:57.744296+0000","last_peered":"2024-05-08T08:55:57.744296+0000","last_clean":"2024-05-08T08:55:57.744296+0000","last_became_active":"2024-05-08T08:55:40.378157+0000","last_became_peered":"2024-05-08T08:55:40.378157+0000","last_unstale":"2024-05-08T08:55:57.744296+0000","last_undegraded":"2024-05-08T08:55:57.744296+0000","last_fullsized":"2024-05-08T08:55:57.744296+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'692","last_scrub_stamp":"2024-05-08T08:55:57.744220+0000","last_deep_scrub":"198'692","last_deep_scrub_stamp":"2024-05-08T08:55:57.744220+0000","last_clean_scrub_stamp":"2024-05-08T08:55:57.744220+0000","objects_scrubbed":40,"log_size":692,"log_dups_size":0,"ondisk_log_size":692,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:31:14.921745+0000","scrub_duration":32,"objects_trimmed":0,"snaptrim_duration":0.0019021660000000001,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":40,"num_read":21842,"num_read_kb":88722,"num_write":580,"num_write_kb":17543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":288,"num_bytes_recovered":3368960,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":384,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:58.871570+0000","last_change":"2024-05-08T08:55:58.871588+0000","last_active":"2024-05-08T08:55:58.871570+0000","last_peered":"2024-05-08T08:55:58.871570+0000","last_clean":"2024-05-08T08:55:58.871570+0000","last_became_active":"2024-05-08T08:52:03.895797+0000","last_became_peered":"2024-05-08T08:52:03.895797+0000","last_unstale":"2024-05-08T08:55:58.871570+0000","last_undegraded":"2024-05-08T08:55:58.871570+0000","last_fullsized":"2024-05-08T08:55:58.871570+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:55:58.871491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:55:58.871491+0000","last_clean_scrub_stamp":"2024-05-08T08:55:58.871491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:17:57.486372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"198'557","reported_seq":6976,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:56:01.543492+0000","last_change":"2024-05-08T08:56:01.543547+0000","last_active":"2024-05-08T08:56:01.543492+0000","last_peered":"2024-05-08T08:56:01.543492+0000","last_clean":"2024-05-08T08:56:01.543492+0000","last_became_active":"2024-05-08T08:55:08.437577+0000","last_became_peered":"2024-05-08T08:55:08.437577+0000","last_unstale":"2024-05-08T08:56:01.543492+0000","last_undegraded":"2024-05-08T08:56:01.543492+0000","last_fullsized":"2024-05-08T08:56:01.543492+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'557","last_scrub_stamp":"2024-05-08T08:56:01.543405+0000","last_deep_scrub":"198'557","last_deep_scrub_stamp":"2024-05-08T08:56:01.543405+0000","last_clean_scrub_stamp":"2024-05-08T08:56:01.543405+0000","objects_scrubbed":39,"log_size":557,"log_dups_size":0,"ondisk_log_size":557,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:19:03.049245+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0.0017459470000000001,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":39,"num_read":6769,"num_read_kb":34542,"num_write":419,"num_write_kb":12024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":324,"num_bytes_recovered":15121731,"num_keys_recovered":25,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":389,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:56:05.256153+0000","last_change":"2024-05-08T08:56:05.256153+0000","last_active":"2024-05-08T08:56:05.256153+0000","last_peered":"2024-05-08T08:56:05.256153+0000","last_clean":"2024-05-08T08:56:05.256153+0000","last_became_active":"2024-05-08T08:54:59.398181+0000","last_became_peered":"2024-05-08T08:54:59.398181+0000","last_unstale":"2024-05-08T08:56:05.256153+0000","last_undegraded":"2024-05-08T08:56:05.256153+0000","last_fullsized":"2024-05-08T08:56:05.256153+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:56:05.256035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:56:05.256035+0000","last_clean_scrub_stamp":"2024-05-08T08:56:05.256035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:45:56.500053+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"198'911","reported_seq":27635,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:56:05.867941+0000","last_change":"2024-05-08T08:56:05.867997+0000","last_active":"2024-05-08T08:56:05.867941+0000","last_peered":"2024-05-08T08:56:05.867941+0000","last_clean":"2024-05-08T08:56:05.867941+0000","last_became_active":"2024-05-08T08:55:43.384081+0000","last_became_peered":"2024-05-08T08:55:43.384081+0000","last_unstale":"2024-05-08T08:56:05.867941+0000","last_undegraded":"2024-05-08T08:56:05.867941+0000","last_fullsized":"2024-05-08T08:56:05.867941+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'911","last_scrub_stamp":"2024-05-08T08:56:05.867857+0000","last_deep_scrub":"198'911","last_deep_scrub_stamp":"2024-05-08T08:56:05.867857+0000","last_clean_scrub_stamp":"2024-05-08T08:56:05.867857+0000","objects_scrubbed":48,"log_size":911,"log_dups_size":0,"ondisk_log_size":911,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:53:21.062251+0000","scrub_duration":36,"objects_trimmed":0,"snaptrim_duration":0.0014858339999999999,"stat_sum":{"num_bytes":0,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":48,"num_read":22624,"num_read_kb":52229,"num_write":653,"num_write_kb":12062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":791,"num_bytes_recovered":28697088,"num_keys_recovered":110,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":395,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:56:01.888606+0000","last_change":"2024-05-08T08:56:01.888606+0000","last_active":"2024-05-08T08:56:01.888606+0000","last_peered":"2024-05-08T08:56:01.888606+0000","last_clean":"2024-05-08T08:56:01.888606+0000","last_became_active":"2024-05-08T08:55:37.499784+0000","last_became_peered":"2024-05-08T08:55:37.499784+0000","last_unstale":"2024-05-08T08:56:01.888606+0000","last_undegraded":"2024-05-08T08:56:01.888606+0000","last_fullsized":"2024-05-08T08:56:01.888606+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:56:01.888436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:56:01.888436+0000","last_clean_scrub_stamp":"2024-05-08T08:56:01.888436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:30:57.557116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"198'1001","reported_seq":14895,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:56:04.790697+0000","last_change":"2024-05-08T08:56:04.790727+0000","last_active":"2024-05-08T08:56:04.790697+0000","last_peered":"2024-05-08T08:56:04.790697+0000","last_clean":"2024-05-08T08:56:04.790697+0000","last_became_active":"2024-05-08T08:55:50.559883+0000","last_became_peered":"2024-05-08T08:55:50.559883+0000","last_unstale":"2024-05-08T08:56:04.790697+0000","last_undegraded":"2024-05-08T08:56:04.790697+0000","last_fullsized":"2024-05-08T08:56:04.790697+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'1001","last_scrub_stamp":"2024-05-08T08:56:04.790599+0000","last_deep_scrub":"198'1001","last_deep_scrub_stamp":"2024-05-08T08:56:04.790599+0000","last_clean_scrub_stamp":"2024-05-08T08:56:04.790599+0000","objects_scrubbed":55,"log_size":1001,"log_dups_size":0,"ondisk_log_size":1001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:42:29.979275+0000","scrub_duration":45,"objects_trimmed":0,"snaptrim_duration":0.0013854900000000001,"stat_sum":{"num_bytes":0,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":55,"num_read":15316,"num_read_kb":32869,"num_write":1141,"num_write_kb":10612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":494,"num_bytes_recovered":13827825,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"198'851","reported_seq":12360,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:56:01.682993+0000","last_change":"2024-05-08T08:56:01.683029+0000","last_active":"2024-05-08T08:56:01.682993+0000","last_peered":"2024-05-08T08:56:01.682993+0000","last_clean":"2024-05-08T08:56:01.682993+0000","last_became_active":"2024-05-08T08:55:41.380391+0000","last_became_peered":"2024-05-08T08:55:41.380391+0000","last_unstale":"2024-05-08T08:56:01.682993+0000","last_undegraded":"2024-05-08T08:56:01.682993+0000","last_fullsized":"2024-05-08T08:56:01.682993+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'851","last_scrub_stamp":"2024-05-08T08:56:01.682902+0000","last_deep_scrub":"198'851","last_deep_scrub_stamp":"2024-05-08T08:56:01.682902+0000","last_clean_scrub_stamp":"2024-05-08T08:56:01.682902+0000","objects_scrubbed":35,"log_size":851,"log_dups_size":0,"ondisk_log_size":851,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:58:35.148066+0000","scrub_duration":30,"objects_trimmed":0,"snaptrim_duration":0.002033212,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":34,"num_read":11854,"num_read_kb":33576,"num_write":867,"num_write_kb":6636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":383,"num_bytes_recovered":9012422,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":362,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:56:03.731346+0000","last_change":"2024-05-08T08:56:03.731391+0000","last_active":"2024-05-08T08:56:03.731346+0000","last_peered":"2024-05-08T08:56:03.731346+0000","last_clean":"2024-05-08T08:56:03.731346+0000","last_became_active":"2024-05-08T08:55:50.559612+0000","last_became_peered":"2024-05-08T08:55:50.559612+0000","last_unstale":"2024-05-08T08:56:03.731346+0000","last_undegraded":"2024-05-08T08:56:03.731346+0000","last_fullsized":"2024-05-08T08:56:03.731346+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:56:03.731246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:56:03.731246+0000","last_clean_scrub_stamp":"2024-05-08T08:56:03.731246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:13:48.824393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"198'766","reported_seq":14277,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:59.917095+0000","last_change":"2024-05-08T08:55:59.917095+0000","last_active":"2024-05-08T08:55:59.917095+0000","last_peered":"2024-05-08T08:55:59.917095+0000","last_clean":"2024-05-08T08:55:59.917095+0000","last_became_active":"2024-05-08T08:52:11.928301+0000","last_became_peered":"2024-05-08T08:52:11.928301+0000","last_unstale":"2024-05-08T08:55:59.917095+0000","last_undegraded":"2024-05-08T08:55:59.917095+0000","last_fullsized":"2024-05-08T08:55:59.917095+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'766","last_scrub_stamp":"2024-05-08T08:55:59.916908+0000","last_deep_scrub":"198'766","last_deep_scrub_stamp":"2024-05-08T08:55:59.916908+0000","last_clean_scrub_stamp":"2024-05-08T08:55:59.916908+0000","objects_scrubbed":37,"log_size":766,"log_dups_size":0,"ondisk_log_size":766,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:24:06.445802+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0.0015902449999999999,"stat_sum":{"num_bytes":0,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":37,"num_read":10704,"num_read_kb":44099,"num_write":654,"num_write_kb":16306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":385,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:58.523917+0000","last_change":"2024-05-08T08:55:58.523917+0000","last_active":"2024-05-08T08:55:58.523917+0000","last_peered":"2024-05-08T08:55:58.523917+0000","last_clean":"2024-05-08T08:55:58.523917+0000","last_became_active":"2024-05-08T08:52:03.891374+0000","last_became_peered":"2024-05-08T08:52:03.891374+0000","last_unstale":"2024-05-08T08:55:58.523917+0000","last_undegraded":"2024-05-08T08:55:58.523917+0000","last_fullsized":"2024-05-08T08:55:58.523917+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:55:58.523819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:55:58.523819+0000","last_clean_scrub_stamp":"2024-05-08T08:55:58.523819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:35:19.152074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"198'1202","reported_seq":28248,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:58.745038+0000","last_change":"2024-05-08T08:55:58.745075+0000","last_active":"2024-05-08T08:55:58.745038+0000","last_peered":"2024-05-08T08:55:58.745038+0000","last_clean":"2024-05-08T08:55:58.745038+0000","last_became_active":"2024-05-08T08:55:42.386239+0000","last_became_peered":"2024-05-08T08:55:42.386239+0000","last_unstale":"2024-05-08T08:55:58.745038+0000","last_undegraded":"2024-05-08T08:55:58.745038+0000","last_fullsized":"2024-05-08T08:55:58.745038+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'1202","last_scrub_stamp":"2024-05-08T08:55:58.744970+0000","last_deep_scrub":"198'1202","last_deep_scrub_stamp":"2024-05-08T08:55:58.744970+0000","last_clean_scrub_stamp":"2024-05-08T08:55:58.744970+0000","objects_scrubbed":41,"log_size":1202,"log_dups_size":0,"ondisk_log_size":1202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:32:59.115924+0000","scrub_duration":37,"objects_trimmed":0,"snaptrim_duration":0.00086553400000000005,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":31078,"num_read_kb":56218,"num_write":870,"num_write_kb":9640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":490,"num_bytes_recovered":5023153,"num_keys_recovered":69,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":384,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:59.372526+0000","last_change":"2024-05-08T08:55:59.372526+0000","last_active":"2024-05-08T08:55:59.372526+0000","last_peered":"2024-05-08T08:55:59.372526+0000","last_clean":"2024-05-08T08:55:59.372526+0000","last_became_active":"2024-05-08T08:52:03.891296+0000","last_became_peered":"2024-05-08T08:52:03.891296+0000","last_unstale":"2024-05-08T08:55:59.372526+0000","last_undegraded":"2024-05-08T08:55:59.372526+0000","last_fullsized":"2024-05-08T08:55:59.372526+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:55:59.372378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:55:59.372378+0000","last_clean_scrub_stamp":"2024-05-08T08:55:59.372378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:09:07.415268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"198'1041","reported_seq":15587,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:56:04.901987+0000","last_change":"2024-05-08T08:56:04.902034+0000","last_active":"2024-05-08T08:56:04.901987+0000","last_peered":"2024-05-08T08:56:04.901987+0000","last_clean":"2024-05-08T08:56:04.901987+0000","last_became_active":"2024-05-08T08:55:43.381345+0000","last_became_peered":"2024-05-08T08:55:43.381345+0000","last_unstale":"2024-05-08T08:56:04.901987+0000","last_undegraded":"2024-05-08T08:56:04.901987+0000","last_fullsized":"2024-05-08T08:56:04.901987+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'1041","last_scrub_stamp":"2024-05-08T08:56:04.901896+0000","last_deep_scrub":"198'1041","last_deep_scrub_stamp":"2024-05-08T08:56:04.901896+0000","last_clean_scrub_stamp":"2024-05-08T08:56:04.901896+0000","objects_scrubbed":39,"log_size":1041,"log_dups_size":0,"ondisk_log_size":1041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:56:12.915791+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0.0012464850000000001,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":39,"num_read":15521,"num_read_kb":63440,"num_write":1271,"num_write_kb":13979,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":635,"num_bytes_recovered":18405521,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":358,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:56:02.745143+0000","last_change":"2024-05-08T08:56:02.745181+0000","last_active":"2024-05-08T08:56:02.745143+0000","last_peered":"2024-05-08T08:56:02.745143+0000","last_clean":"2024-05-08T08:56:02.745143+0000","last_became_active":"2024-05-08T08:55:50.559815+0000","last_became_peered":"2024-05-08T08:55:50.559815+0000","last_unstale":"2024-05-08T08:56:02.745143+0000","last_undegraded":"2024-05-08T08:56:02.745143+0000","last_fullsized":"2024-05-08T08:56:02.745143+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:56:02.745017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:56:02.745017+0000","last_clean_scrub_stamp":"2024-05-08T08:56:02.745017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:21:55.003237+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"198'119","reported_seq":568,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:56:06.234831+0000","last_change":"2024-05-08T08:56:06.234866+0000","last_active":"2024-05-08T08:56:06.234831+0000","last_peered":"2024-05-08T08:56:06.234831+0000","last_clean":"2024-05-08T08:56:06.234831+0000","last_became_active":"2024-05-08T08:55:42.381316+0000","last_became_peered":"2024-05-08T08:55:42.381316+0000","last_unstale":"2024-05-08T08:56:06.234831+0000","last_undegraded":"2024-05-08T08:56:06.234831+0000","last_fullsized":"2024-05-08T08:56:06.234831+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'119","last_scrub_stamp":"2024-05-08T08:56:06.234748+0000","last_deep_scrub":"198'119","last_deep_scrub_stamp":"2024-05-08T08:56:06.234748+0000","last_clean_scrub_stamp":"2024-05-08T08:56:06.234748+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:50:55.648905+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"198'433","reported_seq":9883,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:59.545961+0000","last_change":"2024-05-08T08:55:59.546011+0000","last_active":"2024-05-08T08:55:59.545961+0000","last_peered":"2024-05-08T08:55:59.545961+0000","last_clean":"2024-05-08T08:55:59.545961+0000","last_became_active":"2024-05-08T08:52:11.926975+0000","last_became_peered":"2024-05-08T08:52:11.926975+0000","last_unstale":"2024-05-08T08:55:59.545961+0000","last_undegraded":"2024-05-08T08:55:59.545961+0000","last_fullsized":"2024-05-08T08:55:59.545961+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'433","last_scrub_stamp":"2024-05-08T08:55:59.545878+0000","last_deep_scrub":"198'433","last_deep_scrub_stamp":"2024-05-08T08:55:59.545878+0000","last_clean_scrub_stamp":"2024-05-08T08:55:59.545878+0000","objects_scrubbed":35,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:12:59.405151+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0.00087585099999999997,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":35,"num_read":8901,"num_read_kb":36556,"num_write":340,"num_write_kb":8985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"198'545","reported_seq":14613,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:55:59.718229+0000","last_change":"2024-05-08T08:55:59.718301+0000","last_active":"2024-05-08T08:55:59.718229+0000","last_peered":"2024-05-08T08:55:59.718229+0000","last_clean":"2024-05-08T08:55:59.718229+0000","last_became_active":"2024-05-08T08:53:23.478564+0000","last_became_peered":"2024-05-08T08:53:23.478564+0000","last_unstale":"2024-05-08T08:55:59.718229+0000","last_undegraded":"2024-05-08T08:55:59.718229+0000","last_fullsized":"2024-05-08T08:55:59.718229+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'545","last_scrub_stamp":"2024-05-08T08:55:59.718137+0000","last_deep_scrub":"198'545","last_deep_scrub_stamp":"2024-05-08T08:55:59.718137+0000","last_clean_scrub_stamp":"2024-05-08T08:55:59.718137+0000","objects_scrubbed":42,"log_size":545,"log_dups_size":0,"ondisk_log_size":545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:26:45.754499+0000","scrub_duration":34,"objects_trimmed":0,"snaptrim_duration":0.00082315600000000002,"stat_sum":{"num_bytes":0,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":42,"num_read":14198,"num_read_kb":62213,"num_write":413,"num_write_kb":16566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":131,"num_bytes_recovered":12377088,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"198'1288","reported_seq":11961,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:56:04.267816+0000","last_change":"2024-05-08T08:56:04.267867+0000","last_active":"2024-05-08T08:56:04.267816+0000","last_peered":"2024-05-08T08:56:04.267816+0000","last_clean":"2024-05-08T08:56:04.267816+0000","last_became_active":"2024-05-08T08:53:24.480359+0000","last_became_peered":"2024-05-08T08:53:24.480359+0000","last_unstale":"2024-05-08T08:56:04.267816+0000","last_undegraded":"2024-05-08T08:56:04.267816+0000","last_fullsized":"2024-05-08T08:56:04.267816+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'1288","last_scrub_stamp":"2024-05-08T08:56:04.267735+0000","last_deep_scrub":"198'1288","last_deep_scrub_stamp":"2024-05-08T08:56:04.267735+0000","last_clean_scrub_stamp":"2024-05-08T08:56:04.267735+0000","objects_scrubbed":41,"log_size":1288,"log_dups_size":0,"ondisk_log_size":1288,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:34:38.504249+0000","scrub_duration":38,"objects_trimmed":0,"snaptrim_duration":0.0025209339999999998,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":40,"num_read":15863,"num_read_kb":29509,"num_write":1451,"num_write_kb":6331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":2175343,"num_keys_recovered":61,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"198'678","reported_seq":12577,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:56:03.931514+0000","last_change":"2024-05-08T08:56:03.931544+0000","last_active":"2024-05-08T08:56:03.931514+0000","last_peered":"2024-05-08T08:56:03.931514+0000","last_clean":"2024-05-08T08:56:03.931514+0000","last_became_active":"2024-05-08T08:55:42.382485+0000","last_became_peered":"2024-05-08T08:55:42.382485+0000","last_unstale":"2024-05-08T08:56:03.931514+0000","last_undegraded":"2024-05-08T08:56:03.931514+0000","last_fullsized":"2024-05-08T08:56:03.931514+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'678","last_scrub_stamp":"2024-05-08T08:56:03.931422+0000","last_deep_scrub":"198'678","last_deep_scrub_stamp":"2024-05-08T08:56:03.931422+0000","last_clean_scrub_stamp":"2024-05-08T08:56:03.931422+0000","objects_scrubbed":50,"log_size":678,"log_dups_size":0,"ondisk_log_size":678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:13:40.380334+0000","scrub_duration":39,"objects_trimmed":0,"snaptrim_duration":0.001692783,"stat_sum":{"num_bytes":0,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":50,"num_read":11950,"num_read_kb":39515,"num_write":513,"num_write_kb":7725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":332,"num_bytes_recovered":2080800,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"198'418","reported_seq":14288,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:56:07.738048+0000","last_change":"2024-05-08T08:56:07.738086+0000","last_active":"2024-05-08T08:56:07.738048+0000","last_peered":"2024-05-08T08:56:07.738048+0000","last_clean":"2024-05-08T08:56:07.738048+0000","last_became_active":"2024-05-08T08:55:50.563411+0000","last_became_peered":"2024-05-08T08:55:50.563411+0000","last_unstale":"2024-05-08T08:56:07.738048+0000","last_undegraded":"2024-05-08T08:56:07.738048+0000","last_fullsized":"2024-05-08T08:56:07.738048+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'418","last_scrub_stamp":"2024-05-08T08:56:07.737967+0000","last_deep_scrub":"198'418","last_deep_scrub_stamp":"2024-05-08T08:56:07.737967+0000","last_clean_scrub_stamp":"2024-05-08T08:56:07.737967+0000","objects_scrubbed":40,"log_size":418,"log_dups_size":0,"ondisk_log_size":418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:58:45.309840+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0.0020039379999999998,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":40,"num_read":13099,"num_read_kb":41266,"num_write":323,"num_write_kb":7340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":509,"num_bytes_recovered":9113137,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"198'521","reported_seq":9019,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:56:00.353700+0000","last_change":"2024-05-08T08:56:00.353744+0000","last_active":"2024-05-08T08:56:00.353700+0000","last_peered":"2024-05-08T08:56:00.353700+0000","last_clean":"2024-05-08T08:56:00.353700+0000","last_became_active":"2024-05-08T08:52:11.932594+0000","last_became_peered":"2024-05-08T08:52:11.932594+0000","last_unstale":"2024-05-08T08:56:00.353700+0000","last_undegraded":"2024-05-08T08:56:00.353700+0000","last_fullsized":"2024-05-08T08:56:00.353700+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'521","last_scrub_stamp":"2024-05-08T08:56:00.353614+0000","last_deep_scrub":"198'521","last_deep_scrub_stamp":"2024-05-08T08:56:00.353614+0000","last_clean_scrub_stamp":"2024-05-08T08:56:00.353614+0000","objects_scrubbed":43,"log_size":521,"log_dups_size":0,"ondisk_log_size":521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:36:01.498593+0000","scrub_duration":29,"objects_trimmed":0,"snaptrim_duration":0.0033757140000000001,"stat_sum":{"num_bytes":0,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":43,"num_read":9572,"num_read_kb":34328,"num_write":402,"num_write_kb":10002,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"198'656","reported_seq":9906,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:56:05.739714+0000","last_change":"2024-05-08T08:56:05.739775+0000","last_active":"2024-05-08T08:56:05.739714+0000","last_peered":"2024-05-08T08:56:05.739714+0000","last_clean":"2024-05-08T08:56:05.739714+0000","last_became_active":"2024-05-08T08:55:50.560907+0000","last_became_peered":"2024-05-08T08:55:50.560907+0000","last_unstale":"2024-05-08T08:56:05.739714+0000","last_undegraded":"2024-05-08T08:56:05.739714+0000","last_fullsized":"2024-05-08T08:56:05.739714+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'656","last_scrub_stamp":"2024-05-08T08:56:05.739628+0000","last_deep_scrub":"198'656","last_deep_scrub_stamp":"2024-05-08T08:56:05.739628+0000","last_clean_scrub_stamp":"2024-05-08T08:56:05.739628+0000","objects_scrubbed":24,"log_size":656,"log_dups_size":0,"ondisk_log_size":656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:08:41.571501+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0.002456153,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":16215,"num_read_kb":47213,"num_write":634,"num_write_kb":12225,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":446,"num_bytes_recovered":28147344,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"198'624","reported_seq":8820,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:56:00.756109+0000","last_change":"2024-05-08T08:56:00.756156+0000","last_active":"2024-05-08T08:56:00.756109+0000","last_peered":"2024-05-08T08:56:00.756109+0000","last_clean":"2024-05-08T08:56:00.756109+0000","last_became_active":"2024-05-08T08:55:40.383962+0000","last_became_peered":"2024-05-08T08:55:40.383962+0000","last_unstale":"2024-05-08T08:56:00.756109+0000","last_undegraded":"2024-05-08T08:56:00.756109+0000","last_fullsized":"2024-05-08T08:56:00.756109+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'624","last_scrub_stamp":"2024-05-08T08:56:00.756025+0000","last_deep_scrub":"198'624","last_deep_scrub_stamp":"2024-05-08T08:56:00.756025+0000","last_clean_scrub_stamp":"2024-05-08T08:56:00.756025+0000","objects_scrubbed":36,"log_size":624,"log_dups_size":0,"ondisk_log_size":624,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:15:15.144162+0000","scrub_duration":30,"objects_trimmed":0,"snaptrim_duration":0.0037436610000000001,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":36,"num_read":8886,"num_read_kb":38380,"num_write":568,"num_write_kb":9633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":675,"num_bytes_recovered":18815289,"num_keys_recovered":79,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"198'789","reported_seq":8688,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:56:08.748682+0000","last_change":"2024-05-08T08:56:08.748726+0000","last_active":"2024-05-08T08:56:08.748682+0000","last_peered":"2024-05-08T08:56:08.748682+0000","last_clean":"2024-05-08T08:56:08.748682+0000","last_became_active":"2024-05-08T08:55:50.565716+0000","last_became_peered":"2024-05-08T08:55:50.565716+0000","last_unstale":"2024-05-08T08:56:08.748682+0000","last_undegraded":"2024-05-08T08:56:08.748682+0000","last_fullsized":"2024-05-08T08:56:08.748682+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'789","last_scrub_stamp":"2024-05-08T08:56:08.748603+0000","last_deep_scrub":"198'789","last_deep_scrub_stamp":"2024-05-08T08:56:08.748603+0000","last_clean_scrub_stamp":"2024-05-08T08:56:08.748603+0000","objects_scrubbed":40,"log_size":789,"log_dups_size":0,"ondisk_log_size":789,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:50:11.482902+0000","scrub_duration":25,"objects_trimmed":0,"snaptrim_duration":0.00095127799999999995,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":40,"num_read":9090,"num_read_kb":41259,"num_write":984,"num_write_kb":14557,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":541,"num_bytes_recovered":26549987,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"198'793","reported_seq":11932,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:56:03.311737+0000","last_change":"2024-05-08T08:56:03.311798+0000","last_active":"2024-05-08T08:56:03.311737+0000","last_peered":"2024-05-08T08:56:03.311737+0000","last_clean":"2024-05-08T08:56:03.311737+0000","last_became_active":"2024-05-08T08:53:24.479891+0000","last_became_peered":"2024-05-08T08:53:24.479891+0000","last_unstale":"2024-05-08T08:56:03.311737+0000","last_undegraded":"2024-05-08T08:56:03.311737+0000","last_fullsized":"2024-05-08T08:56:03.311737+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'793","last_scrub_stamp":"2024-05-08T08:56:03.311654+0000","last_deep_scrub":"198'793","last_deep_scrub_stamp":"2024-05-08T08:56:03.311654+0000","last_clean_scrub_stamp":"2024-05-08T08:56:03.311654+0000","objects_scrubbed":43,"log_size":793,"log_dups_size":0,"ondisk_log_size":793,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:18:48.273010+0000","scrub_duration":37,"objects_trimmed":0,"snaptrim_duration":0.0015502269999999999,"stat_sum":{"num_bytes":0,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":43,"num_read":13241,"num_read_kb":36544,"num_write":586,"num_write_kb":10920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":131,"num_bytes_recovered":2604032,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"198'1627","reported_seq":25821,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:56:01.803917+0000","last_change":"2024-05-08T08:56:01.803946+0000","last_active":"2024-05-08T08:56:01.803917+0000","last_peered":"2024-05-08T08:56:01.803917+0000","last_clean":"2024-05-08T08:56:01.803917+0000","last_became_active":"2024-05-08T08:55:43.386543+0000","last_became_peered":"2024-05-08T08:55:43.386543+0000","last_unstale":"2024-05-08T08:56:01.803917+0000","last_undegraded":"2024-05-08T08:56:01.803917+0000","last_fullsized":"2024-05-08T08:56:01.803917+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'1627","last_scrub_stamp":"2024-05-08T08:56:01.803821+0000","last_deep_scrub":"198'1627","last_deep_scrub_stamp":"2024-05-08T08:56:01.803821+0000","last_clean_scrub_stamp":"2024-05-08T08:56:01.803821+0000","objects_scrubbed":52,"log_size":1627,"log_dups_size":0,"ondisk_log_size":1627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:45:01.659395+0000","scrub_duration":42,"objects_trimmed":0,"snaptrim_duration":0.0015403719999999999,"stat_sum":{"num_bytes":0,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":52,"num_read":30937,"num_read_kb":62696,"num_write":2139,"num_write_kb":12077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":330,"num_bytes_recovered":4235412,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"198'691","reported_seq":11745,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-05-08T08:56:08.277285+0000","last_change":"2024-05-08T08:56:08.277355+0000","last_active":"2024-05-08T08:56:08.277285+0000","last_peered":"2024-05-08T08:56:08.277285+0000","last_clean":"2024-05-08T08:56:08.277285+0000","last_became_active":"2024-05-08T08:55:43.379146+0000","last_became_peered":"2024-05-08T08:55:43.379146+0000","last_unstale":"2024-05-08T08:56:08.277285+0000","last_undegraded":"2024-05-08T08:56:08.277285+0000","last_fullsized":"2024-05-08T08:56:08.277285+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'691","last_scrub_stamp":"2024-05-08T08:56:08.277186+0000","last_deep_scrub":"198'691","last_deep_scrub_stamp":"2024-05-08T08:56:08.277186+0000","last_clean_scrub_stamp":"2024-05-08T08:56:08.277186+0000","objects_scrubbed":40,"log_size":691,"log_dups_size":0,"ondisk_log_size":691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:51:48.939770+0000","scrub_duration":30,"objects_trimmed":0,"snaptrim_duration":0.0022701349999999999,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":40,"num_read":11037,"num_read_kb":41101,"num_write":598,"num_write_kb":9071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":589,"num_bytes_recovered":19963261,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":1295,"num_object_clones":0,"num_object_copies":2590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1295,"num_whiteouts":1291,"num_read":512933,"num_read_kb":1628362,"num_write":26100,"num_write_kb":451839,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11018,"num_bytes_recovered":421954137,"num_keys_recovered":904,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":26123,"ondisk_log_size":26123,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":18287,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640311,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59872,"kb_used_data":2328,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311968,"statfs":{"total":96636764160,"available":96575455232,"internally_reserved":0,"allocated":2383872,"data_stored":1016789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640315,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55716,"kb_used_data":2332,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316124,"statfs":{"total":96636764160,"available":96579710976,"internally_reserved":0,"allocated":2387968,"data_stored":1016808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640313,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87704,"kb_used_data":2384,"kb_used_omap":1,"kb_used_meta":85310,"kb_avail":94284136,"statfs":{"total":96636764160,"available":96546955264,"internally_reserved":0,"allocated":2441216,"data_stored":1607157,"data_compressed":5523,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1593,"internal_metadata":87357895},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640312,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51552,"kb_used_data":2328,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320288,"statfs":{"total":96636764160,"available":96583974912,"internally_reserved":0,"allocated":2383872,"data_stored":1016789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640311,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65444,"kb_used_data":2332,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306396,"statfs":{"total":96636764160,"available":96569749504,"internally_reserved":0,"allocated":2387968,"data_stored":1016808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640314,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73800,"kb_used_data":2368,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298040,"statfs":{"total":96636764160,"available":96561192960,"internally_reserved":0,"allocated":2424832,"data_stored":1607157,"data_compressed":6382,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6382,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6382,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5523,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T08:56:20.488 DEBUG:teuthology.orchestra.run.smithi059:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-05-08T08:56:20.802 INFO:teuthology.misc:Shutting down mds daemons... 2024-05-08T08:56:20.802 INFO:teuthology.misc:Shutting down osd daemons... 2024-05-08T08:56:20.803 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-05-08T08:56:20.803 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T08:56:20.837 INFO:tasks.ceph.osd.0:Stopped 2024-05-08T08:56:20.837 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-05-08T08:56:20.837 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T08:56:20.914 INFO:tasks.ceph.osd.1:Stopped 2024-05-08T08:56:20.915 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-05-08T08:56:20.915 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T08:56:20.988 INFO:tasks.ceph.osd.2:Stopped 2024-05-08T08:56:20.988 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-05-08T08:56:20.988 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T08:56:21.035 INFO:tasks.ceph.osd.3:Stopped 2024-05-08T08:56:21.035 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-05-08T08:56:21.035 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T08:56:21.075 INFO:tasks.ceph.osd.4:Stopped 2024-05-08T08:56:21.075 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-05-08T08:56:21.075 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T08:56:21.150 INFO:tasks.ceph.osd.5:Stopped 2024-05-08T08:56:21.150 INFO:teuthology.misc:Shutting down mgr daemons... 2024-05-08T08:56:21.150 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-05-08T08:56:21.150 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T08:56:21.219 INFO:tasks.ceph.mgr.x:Stopped 2024-05-08T08:56:21.219 INFO:teuthology.misc:Shutting down mon daemons... 2024-05-08T08:56:21.219 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-05-08T08:56:21.219 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T08:56:21.231 INFO:tasks.ceph.mon.a:Stopped 2024-05-08T08:56:21.231 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-05-08T08:56:21.231 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T08:56:21.284 INFO:tasks.ceph.mon.c:Stopped 2024-05-08T08:56:21.284 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-05-08T08:56:21.284 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T08:56:21.296 INFO:tasks.ceph.mon.b:Stopped 2024-05-08T08:56:21.296 INFO:tasks.ceph:Checking cluster log for badness... 2024-05-08T08:56:21.297 DEBUG:teuthology.orchestra.run.smithi059:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v MON_DOWN | egrep -v OSDMAP_FLAGS | egrep -v OSD_DOWN | egrep -v PG_DEGRADED | egrep -v PG_AVAILABILITY | egrep -v POOL_APP_NOT_ENABLED | head -n 1 2024-05-08T08:56:21.369 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi059.front.sepia.ceph.com 2024-05-08T08:56:21.369 DEBUG:teuthology.orchestra.run.smithi059:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-05-08T08:56:21.651 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi059.front.sepia.ceph.com 2024-05-08T08:56:21.651 DEBUG:teuthology.orchestra.run.smithi059:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-05-08T08:56:21.783 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi059.front.sepia.ceph.com 2024-05-08T08:56:21.784 DEBUG:teuthology.orchestra.run.smithi059:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-05-08T08:56:21.847 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi192.front.sepia.ceph.com 2024-05-08T08:56:21.847 DEBUG:teuthology.orchestra.run.smithi192:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-05-08T08:56:22.131 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi192.front.sepia.ceph.com 2024-05-08T08:56:22.132 DEBUG:teuthology.orchestra.run.smithi192:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-05-08T08:56:22.224 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi192.front.sepia.ceph.com 2024-05-08T08:56:22.224 DEBUG:teuthology.orchestra.run.smithi192:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-05-08T08:56:22.283 INFO:tasks.ceph:Archiving mon data... 2024-05-08T08:56:22.284 DEBUG:teuthology.misc:Transferring archived files from smithi059:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-05-07_21:16:03-rbd-squid-distro-default-smithi/7696006/data/mon.a.tgz 2024-05-08T08:56:22.284 DEBUG:teuthology.orchestra.run.smithi059:> mktemp 2024-05-08T08:56:22.303 INFO:teuthology.orchestra.run.smithi059.stdout:/tmp/tmp.OjUPW5uUuF 2024-05-08T08:56:22.304 DEBUG:teuthology.orchestra.run.smithi059:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.OjUPW5uUuF 2024-05-08T08:56:22.461 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 0666 /tmp/tmp.OjUPW5uUuF 2024-05-08T08:56:22.547 DEBUG:teuthology.orchestra.remote:smithi059:/tmp/tmp.OjUPW5uUuF is 367KB 2024-05-08T08:56:22.619 DEBUG:teuthology.orchestra.run.smithi059:> rm -fr /tmp/tmp.OjUPW5uUuF 2024-05-08T08:56:22.635 DEBUG:teuthology.misc:Transferring archived files from smithi059:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-05-07_21:16:03-rbd-squid-distro-default-smithi/7696006/data/mon.c.tgz 2024-05-08T08:56:22.635 DEBUG:teuthology.orchestra.run.smithi059:> mktemp 2024-05-08T08:56:22.690 INFO:teuthology.orchestra.run.smithi059.stdout:/tmp/tmp.xBPuAJC7ls 2024-05-08T08:56:22.690 DEBUG:teuthology.orchestra.run.smithi059:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.xBPuAJC7ls 2024-05-08T08:56:22.865 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 0666 /tmp/tmp.xBPuAJC7ls 2024-05-08T08:56:22.950 DEBUG:teuthology.orchestra.remote:smithi059:/tmp/tmp.xBPuAJC7ls is 492KB 2024-05-08T08:56:23.048 DEBUG:teuthology.orchestra.run.smithi059:> rm -fr /tmp/tmp.xBPuAJC7ls 2024-05-08T08:56:23.064 DEBUG:teuthology.misc:Transferring archived files from smithi192:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-05-07_21:16:03-rbd-squid-distro-default-smithi/7696006/data/mon.b.tgz 2024-05-08T08:56:23.064 DEBUG:teuthology.orchestra.run.smithi192:> mktemp 2024-05-08T08:56:23.082 INFO:teuthology.orchestra.run.smithi192.stdout:/tmp/tmp.qOuzlOXWMa 2024-05-08T08:56:23.083 DEBUG:teuthology.orchestra.run.smithi192:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.qOuzlOXWMa 2024-05-08T08:56:23.263 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 0666 /tmp/tmp.qOuzlOXWMa 2024-05-08T08:56:23.346 DEBUG:teuthology.orchestra.remote:smithi192:/tmp/tmp.qOuzlOXWMa is 491KB 2024-05-08T08:56:23.417 DEBUG:teuthology.orchestra.run.smithi192:> rm -fr /tmp/tmp.qOuzlOXWMa 2024-05-08T08:56:23.432 INFO:tasks.ceph:Cleaning ceph cluster... 2024-05-08T08:56:23.432 DEBUG:teuthology.orchestra.run.smithi059:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-08T08:56:23.434 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-08T08:56:23.474 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-08T08:56:23.523 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-05-08T08:56:23.552 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-05-08T08:56:23.570 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-05-08T08:56:23.570 INFO:tasks.ceph:Archiving crash dumps... 2024-05-08T08:56:23.571 DEBUG:teuthology.misc:Transferring archived files from smithi059:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-05-07_21:16:03-rbd-squid-distro-default-smithi/7696006/remote/smithi059/crash 2024-05-08T08:56:23.572 DEBUG:teuthology.orchestra.run.smithi059:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-08T08:56:23.599 DEBUG:teuthology.misc:Transferring archived files from smithi192:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-05-07_21:16:03-rbd-squid-distro-default-smithi/7696006/remote/smithi192/crash 2024-05-08T08:56:23.600 DEBUG:teuthology.orchestra.run.smithi192:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-08T08:56:23.624 DEBUG:teuthology.misc:Transferring archived files from smithi196:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-05-07_21:16:03-rbd-squid-distro-default-smithi/7696006/remote/smithi196/crash 2024-05-08T08:56:23.625 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-08T08:56:23.746 INFO:tasks.ceph:Compressing logs... 2024-05-08T08:56:23.747 DEBUG:teuthology.orchestra.run.smithi059:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-08T08:56:23.750 DEBUG:teuthology.orchestra.run.smithi192:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-08T08:56:23.752 DEBUG:teuthology.orchestra.run.smithi196:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-08T08:56:23.775 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-05-08T08:56:23.775 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-05-08T08:56:23.775 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31582.log 2024-05-08T08:56:23.775 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-05-08T08:56:23.775 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-client.admin.31582.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-05-08T08:56:23.775 INFO:teuthology.orchestra.run.smithi192.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31582.log.gz 2024-05-08T08:56:23.775 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-05-08T08:56:23.776 INFO:teuthology.orchestra.run.smithi192.stderr: 85.1% -- replaced with /var/log/ceph/ceph.log.gz 2024-05-08T08:56:23.776 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-05-08T08:56:23.780 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31622.log 2024-05-08T08:56:23.780 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-osd.4.log: 89.1% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-05-08T08:56:23.780 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31753.log 2024-05-08T08:56:23.780 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-client.admin.31622.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31622.log.gz 2024-05-08T08:56:23.781 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31713.log 2024-05-08T08:56:23.781 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31664.log 2024-05-08T08:56:23.781 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-client.admin.31753.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31753.log.gz 2024-05-08T08:56:23.781 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-client.admin.31713.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31713.log.gz 2024-05-08T08:56:23.781 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31542.log 2024-05-08T08:56:23.781 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-05-08T08:56:23.781 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-client.admin.31664.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31664.log.gz 2024-05-08T08:56:23.781 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-client.admin.31542.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31542.log.gz 2024-05-08T08:56:23.842 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-mon.b.log: 91.5% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-05-08T08:56:23.927 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26975.log 2024-05-08T08:56:23.927 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27773.log 2024-05-08T08:56:23.927 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27633.log 2024-05-08T08:56:23.927 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.26975.log: 57.1% -- replaced with /var/log/ceph/ceph-client.admin.26975.log.gz 2024-05-08T08:56:23.927 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.27773.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28972.log 2024-05-08T08:56:23.927 INFO:teuthology.orchestra.run.smithi196.stderr: 20.9% -- replaced with /var/log/ceph/ceph-client.admin.27773.log.gz 2024-05-08T08:56:23.927 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.27633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28437.log 2024-05-08T08:56:23.927 INFO:teuthology.orchestra.run.smithi196.stderr: 19.6% -- replaced with /var/log/ceph/ceph-client.admin.27633.log.gz 2024-05-08T08:56:23.927 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.28972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26932.log 2024-05-08T08:56:23.927 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.28437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27852.log 2024-05-08T08:56:23.927 INFO:teuthology.orchestra.run.smithi196.stderr: 95.8% -- replaced with /var/log/ceph/ceph-client.admin.28972.log.gz 2024-05-08T08:56:23.927 INFO:teuthology.orchestra.run.smithi196.stderr: 20.9% -- replaced with /var/log/ceph/ceph-client.admin.28437.log.gz 2024-05-08T08:56:23.928 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.26932.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27676.log 2024-05-08T08:56:23.928 INFO:teuthology.orchestra.run.smithi196.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.26932.log.gz 2024-05-08T08:56:23.928 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.27852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27956.log 2024-05-08T08:56:23.928 INFO:teuthology.orchestra.run.smithi196.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.27852.log.gz 2024-05-08T08:56:23.928 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.27676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27282.log 2024-05-08T08:56:23.928 INFO:teuthology.orchestra.run.smithi196.stderr: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.27676.log.gz 2024-05-08T08:56:23.928 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.27956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27422.log 2024-05-08T08:56:23.928 INFO:teuthology.orchestra.run.smithi196.stderr: 25.9% -- replaced with /var/log/ceph/ceph-client.admin.27956.log.gz 2024-05-08T08:56:23.928 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.27282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26370.log 2024-05-08T08:56:23.928 INFO:teuthology.orchestra.run.smithi196.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.27282.log.gz 2024-05-08T08:56:23.928 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.27422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27816.log 2024-05-08T08:56:23.928 INFO:teuthology.orchestra.run.smithi196.stderr: 19.6% -- replaced with /var/log/ceph/ceph-client.admin.27422.log.gz 2024-05-08T08:56:23.928 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.26370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27886.log 2024-05-08T08:56:23.928 INFO:teuthology.orchestra.run.smithi196.stderr: 19.6% -- replaced with /var/log/ceph/ceph-client.admin.26370.log.gz 2024-05-08T08:56:23.928 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.27816.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28516.log 2024-05-08T08:56:23.929 INFO:teuthology.orchestra.run.smithi196.stderr: 84.9% -- replaced with /var/log/ceph/ceph-client.admin.27816.log.gz 2024-05-08T08:56:23.929 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.27886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28193.log 2024-05-08T08:56:23.929 INFO:teuthology.orchestra.run.smithi196.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.27886.log.gz 2024-05-08T08:56:23.929 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.28516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26554.log 2024-05-08T08:56:23.929 INFO:teuthology.orchestra.run.smithi196.stderr: 77.2% -- replaced with /var/log/ceph/ceph-client.admin.28516.log.gz 2024-05-08T08:56:23.929 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.28193.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28376.log 2024-05-08T08:56:23.929 INFO:teuthology.orchestra.run.smithi196.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.28193.log.gz 2024-05-08T08:56:23.929 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.26554.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28963.log 2024-05-08T08:56:23.929 INFO:teuthology.orchestra.run.smithi196.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.26554.log.gz 2024-05-08T08:56:23.929 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.28376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27843.log 2024-05-08T08:56:23.929 INFO:teuthology.orchestra.run.smithi196.stderr: 79.5% -- replaced with /var/log/ceph/ceph-client.admin.28376.log.gz 2024-05-08T08:56:23.929 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.28963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28543.log 2024-05-08T08:56:23.929 INFO:teuthology.orchestra.run.smithi196.stderr: 20.9% -- replaced with /var/log/ceph/ceph-client.admin.28963.log.gz 2024-05-08T08:56:23.929 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.27843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26801.log 2024-05-08T08:56:23.929 INFO:teuthology.orchestra.run.smithi196.stderr: 19.6% -- replaced with /var/log/ceph/ceph-client.admin.27843.log.gz 2024-05-08T08:56:23.929 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.28543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28586.log 2024-05-08T08:56:23.930 INFO:teuthology.orchestra.run.smithi196.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.28543.log.gz 2024-05-08T08:56:23.930 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.26801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27947.log 2024-05-08T08:56:23.930 INFO:teuthology.orchestra.run.smithi196.stderr: 96.0%/var/log/ceph/ceph-client.admin.28586.log: -- replaced with /var/log/ceph/ceph-client.admin.26801.log.gz 2024-05-08T08:56:23.930 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27246.log 2024-05-08T08:56:23.930 INFO:teuthology.orchestra.run.smithi196.stderr: 24.7% -- replaced with /var/log/ceph/ceph-client.admin.28586.log.gz 2024-05-08T08:56:23.930 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.27947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26686.log 2024-05-08T08:56:23.930 INFO:teuthology.orchestra.run.smithi196.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.27947.log.gz 2024-05-08T08:56:23.930 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.27246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29427.log 2024-05-08T08:56:23.930 INFO:teuthology.orchestra.run.smithi196.stderr: 19.6% -- replaced with /var/log/ceph/ceph-client.admin.27246.log.gz 2024-05-08T08:56:23.930 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.26686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28062.log 2024-05-08T08:56:23.930 INFO:teuthology.orchestra.run.smithi196.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.26686.log.gz 2024-05-08T08:56:23.930 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.29427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29637.log 2024-05-08T08:56:23.930 INFO:teuthology.orchestra.run.smithi196.stderr: 56.9% -- replaced with /var/log/ceph/ceph-client.admin.29427.log.gz 2024-05-08T08:56:23.930 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.28062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26404.log 2024-05-08T08:56:23.931 INFO:teuthology.orchestra.run.smithi196.stderr: 95.0% -- replaced with /var/log/ceph/ceph-client.admin.28062.log.gz 2024-05-08T08:56:23.931 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.29637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28936.log 2024-05-08T08:56:23.931 INFO:teuthology.orchestra.run.smithi196.stderr: 58.1% -- replaced with /var/log/ceph/ceph-client.admin.29637.log.gz 2024-05-08T08:56:23.931 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.26404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27877.log 2024-05-08T08:56:23.931 INFO:teuthology.orchestra.run.smithi196.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.26404.log.gz 2024-05-08T08:56:23.931 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.28936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28227.log 2024-05-08T08:56:23.931 INFO:teuthology.orchestra.run.smithi196.stderr: 81.0% -- replaced with /var/log/ceph/ceph-client.admin.28936.log.gz 2024-05-08T08:56:23.931 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.27877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27746.log 2024-05-08T08:56:23.931 INFO:teuthology.orchestra.run.smithi196.stderr: 19.0% -- replaced with /var/log/ceph/ceph-client.admin.27877.log.gz 2024-05-08T08:56:23.931 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.28227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26520.log 2024-05-08T08:56:23.931 INFO:teuthology.orchestra.run.smithi196.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.28227.log.gz 2024-05-08T08:56:23.931 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.27746.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29533.log 2024-05-08T08:56:23.931 INFO:teuthology.orchestra.run.smithi196.stderr: 84.3% -- replaced with /var/log/ceph/ceph-client.admin.27746.log.gz 2024-05-08T08:56:23.931 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.26520.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27011.log 2024-05-08T08:56:23.931 INFO:teuthology.orchestra.run.smithi196.stderr: 93.5% -- replaced with /var/log/ceph/ceph-client.admin.26520.log.gz 2024-05-08T08:56:23.931 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.29533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28552.log 2024-05-08T08:56:23.932 INFO:teuthology.orchestra.run.smithi196.stderr: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.29533.log.gz 2024-05-08T08:56:23.932 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.27011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29287.log 2024-05-08T08:56:23.932 INFO:teuthology.orchestra.run.smithi196.stderr: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.27011.log.gz 2024-05-08T08:56:23.932 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.28552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29734.log 2024-05-08T08:56:23.932 INFO:teuthology.orchestra.run.smithi196.stderr: 56.8% -- replaced with /var/log/ceph/ceph-client.admin.28552.log.gz 2024-05-08T08:56:23.932 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.29287.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26896.log 2024-05-08T08:56:23.932 INFO:teuthology.orchestra.run.smithi196.stderr: 79.7% -- replaced with /var/log/ceph/ceph-client.admin.29287.log.gz 2024-05-08T08:56:23.932 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.29734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28446.log 2024-05-08T08:56:23.932 INFO:teuthology.orchestra.run.smithi196.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.29734.log.gz 2024-05-08T08:56:23.932 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.26896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29567.log 2024-05-08T08:56:23.932 INFO:teuthology.orchestra.run.smithi196.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.26896.log.gz 2024-05-08T08:56:23.932 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.28446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28622.log 2024-05-08T08:56:23.932 INFO:teuthology.orchestra.run.smithi196.stderr: 57.0% -- replaced with /var/log/ceph/ceph-client.admin.28446.log.gz 2024-05-08T08:56:23.932 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.29567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27045.log 2024-05-08T08:56:23.932 INFO:teuthology.orchestra.run.smithi196.stderr: 52.9% -- replaced with /var/log/ceph/ceph-client.admin.29567.log.gz 2024-05-08T08:56:23.933 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.28622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28683.log 2024-05-08T08:56:23.933 INFO:teuthology.orchestra.run.smithi196.stderr: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.28622.log.gz 2024-05-08T08:56:23.933 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.27045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28342.log 2024-05-08T08:56:23.933 INFO:teuthology.orchestra.run.smithi196.stderr: 52.8% -- replaced with /var/log/ceph/ceph-client.admin.27045.log.gz 2024-05-08T08:56:23.933 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.28683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26862.log 2024-05-08T08:56:23.933 INFO:teuthology.orchestra.run.smithi196.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.28683.log.gz 2024-05-08T08:56:23.933 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.28342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26484.log 2024-05-08T08:56:23.933 INFO:teuthology.orchestra.run.smithi196.stderr: 56.8% -- replaced with /var/log/ceph/ceph-client.admin.28342.log.gz 2024-05-08T08:56:23.933 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.26862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29384.log 2024-05-08T08:56:23.933 INFO:teuthology.orchestra.run.smithi196.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.26862.log.gz 2024-05-08T08:56:23.933 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.26484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27115.log 2024-05-08T08:56:23.933 INFO:teuthology.orchestra.run.smithi196.stderr: 25.3% -- replaced with /var/log/ceph/ceph-client.admin.26484.log.gz 2024-05-08T08:56:23.933 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.29384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27572.log 2024-05-08T08:56:23.933 INFO:teuthology.orchestra.run.smithi196.stderr: 20.9% -- replaced with /var/log/ceph/ceph-client.admin.29384.log.gz 2024-05-08T08:56:23.933 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.27115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27352.log 2024-05-08T08:56:23.934 INFO:teuthology.orchestra.run.smithi196.stderr: 57.4% -- replaced with /var/log/ceph/ceph-client.admin.27115.log.gz 2024-05-08T08:56:23.934 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.27572.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27395.log 2024-05-08T08:56:23.934 INFO:teuthology.orchestra.run.smithi196.stderr: 79.1% -- replaced with /var/log/ceph/ceph-client.admin.27572.log.gz 2024-05-08T08:56:23.934 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.27352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26590.log 2024-05-08T08:56:23.934 INFO:teuthology.orchestra.run.smithi196.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.27352.log.gz 2024-05-08T08:56:23.934 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.27395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26966.log 2024-05-08T08:56:23.934 INFO:teuthology.orchestra.run.smithi196.stderr: 53.0% -- replaced with /var/log/ceph/ceph-client.admin.27395.log.gz 2024-05-08T08:56:23.934 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.26590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28893.log 2024-05-08T08:56:23.934 INFO:teuthology.orchestra.run.smithi196.stderr: 94.3% -- replaced with /var/log/ceph/ceph-client.admin.26590.log.gz 2024-05-08T08:56:23.934 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.26966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28613.log 2024-05-08T08:56:23.934 INFO:teuthology.orchestra.run.smithi196.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.26966.log.gz 2024-05-08T08:56:23.934 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.28893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28017.log 2024-05-08T08:56:23.934 INFO:teuthology.orchestra.run.smithi196.stderr: 20.9% -- replaced with /var/log/ceph/ceph-client.admin.28893.log.gz 2024-05-08T08:56:23.935 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.28613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29112.log 2024-05-08T08:56:23.935 INFO:teuthology.orchestra.run.smithi196.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.28613.log.gz 2024-05-08T08:56:23.935 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.28017.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27922.log 2024-05-08T08:56:23.935 INFO:teuthology.orchestra.run.smithi196.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.28017.log.gz 2024-05-08T08:56:23.935 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.29112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27563.log 2024-05-08T08:56:23.935 INFO:teuthology.orchestra.run.smithi196.stderr: 84.0% -- replaced with /var/log/ceph/ceph-client.admin.29112.log.gz 2024-05-08T08:56:23.935 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.27922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28997.log 2024-05-08T08:56:23.935 INFO:teuthology.orchestra.run.smithi196.stderr: 93.8% -- replaced with /var/log/ceph/ceph-client.admin.27922.log.gz 2024-05-08T08:56:23.935 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.27563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28272.log 2024-05-08T08:56:23.935 INFO:teuthology.orchestra.run.smithi196.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.27563.log.gz 2024-05-08T08:56:23.935 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.28997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29664.log 2024-05-08T08:56:23.935 INFO:teuthology.orchestra.run.smithi196.stderr: 20.9% -- replaced with /var/log/ceph/ceph-client.admin.28997.log.gz 2024-05-08T08:56:23.935 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.28272.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27465.log 2024-05-08T08:56:23.935 INFO:teuthology.orchestra.run.smithi196.stderr: 93.7% -- replaced with /var/log/ceph/ceph-client.admin.28272.log.gz 2024-05-08T08:56:23.935 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.29664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28692.log 2024-05-08T08:56:23.935 INFO:teuthology.orchestra.run.smithi196.stderr: 20.9% -- replaced with /var/log/ceph/ceph-client.admin.29664.log.gz 2024-05-08T08:56:23.935 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.27465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27456.log 2024-05-08T08:56:23.936 INFO:teuthology.orchestra.run.smithi196.stderr: 57.1% -- replaced with /var/log/ceph/ceph-client.admin.27465.log.gz 2024-05-08T08:56:23.936 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.28692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28787.log 2024-05-08T08:56:23.936 INFO:teuthology.orchestra.run.smithi196.stderr: 93.9% -- replaced with /var/log/ceph/ceph-client.admin.28692.log.gz 2024-05-08T08:56:23.936 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.27456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28577.log 2024-05-08T08:56:23.936 INFO:teuthology.orchestra.run.smithi196.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.27456.log.gz 2024-05-08T08:56:23.936 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.28787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26731.log 2024-05-08T08:56:23.936 INFO:teuthology.orchestra.run.smithi196.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.28787.log.gz 2024-05-08T08:56:23.936 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.28577.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27255.log 2024-05-08T08:56:23.936 INFO:teuthology.orchestra.run.smithi196.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.28577.log.gz 2024-05-08T08:56:23.936 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.26731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29216.log 2024-05-08T08:56:23.936 INFO:teuthology.orchestra.run.smithi196.stderr: 93.0% -- replaced with /var/log/ceph/ceph-client.admin.26731.log.gz 2024-05-08T08:56:23.936 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.27255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26792.log 2024-05-08T08:56:23.936 INFO:teuthology.orchestra.run.smithi196.stderr: 57.7% -- replaced with /var/log/ceph/ceph-client.admin.27255.log.gz 2024-05-08T08:56:23.936 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.29216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29067.log 2024-05-08T08:56:23.936 INFO:teuthology.orchestra.run.smithi196.stderr: 53.7% -- replaced with /var/log/ceph/ceph-client.admin.29216.log.gz 2024-05-08T08:56:23.936 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.26792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27913.log 2024-05-08T08:56:23.937 INFO:teuthology.orchestra.run.smithi196.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.26792.log.gz 2024-05-08T08:56:23.937 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.29067.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29707.log 2024-05-08T08:56:23.937 INFO:teuthology.orchestra.run.smithi196.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.29067.log.gz 2024-05-08T08:56:23.937 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.27913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27081.log 2024-05-08T08:56:23.937 INFO:teuthology.orchestra.run.smithi196.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.27913.log.gz 2024-05-08T08:56:23.937 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.29707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29182.log 2024-05-08T08:56:23.937 INFO:teuthology.orchestra.run.smithi196.stderr: 54.5% -- replaced with /var/log/ceph/ceph-client.admin.29707.log.gz 2024-05-08T08:56:23.937 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.27081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27703.log 2024-05-08T08:56:23.937 INFO:teuthology.orchestra.run.smithi196.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.27081.log.gz 2024-05-08T08:56:23.937 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.29182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28927.log 2024-05-08T08:56:23.937 INFO:teuthology.orchestra.run.smithi196.stderr: 53.1% -- replaced with /var/log/ceph/ceph-client.admin.29182.log.gz 2024-05-08T08:56:23.937 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.27703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27501.log 2024-05-08T08:56:23.937 INFO:teuthology.orchestra.run.smithi196.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.27703.log.gz 2024-05-08T08:56:23.937 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.28927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28403.log 2024-05-08T08:56:23.937 INFO:teuthology.orchestra.run.smithi196.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.28927.log.gz 2024-05-08T08:56:23.937 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.27501.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26661.log 2024-05-08T08:56:23.938 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.28403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29348.log 2024-05-08T08:56:23.938 INFO:teuthology.orchestra.run.smithi196.stderr: 95.3% 20.9% -- replaced with /var/log/ceph/ceph-client.admin.27501.log.gz 2024-05-08T08:56:23.938 INFO:teuthology.orchestra.run.smithi196.stderr: -- replaced with /var/log/ceph/ceph-client.admin.28403.log.gz 2024-05-08T08:56:23.938 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.26661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29103.log 2024-05-08T08:56:23.938 INFO:teuthology.orchestra.run.smithi196.stderr: 84.9% -- replaced with /var/log/ceph/ceph-client.admin.26661.log.gz 2024-05-08T08:56:23.938 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.29348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28507.log 2024-05-08T08:56:23.938 INFO:teuthology.orchestra.run.smithi196.stderr: 19.6% -- replaced with /var/log/ceph/ceph-client.admin.29348.log.gz 2024-05-08T08:56:23.938 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.29103.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28306.log 2024-05-08T08:56:23.938 INFO:teuthology.orchestra.run.smithi196.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.29103.log.gz 2024-05-08T08:56:23.938 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.28507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29137.log 2024-05-08T08:56:23.938 INFO:teuthology.orchestra.run.smithi196.stderr: 20.9% -- replaced with /var/log/ceph/ceph-client.admin.28507.log.gz 2024-05-08T08:56:23.938 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.28306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26545.log 2024-05-08T08:56:23.938 INFO:teuthology.orchestra.run.smithi196.stderr: 55.5% -- replaced with /var/log/ceph/ceph-client.admin.28306.log.gz 2024-05-08T08:56:23.938 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.29137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29497.log 2024-05-08T08:56:23.939 INFO:teuthology.orchestra.run.smithi196.stderr: 20.9% -- replaced with /var/log/ceph/ceph-client.admin.29137.log.gz 2024-05-08T08:56:23.939 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.26545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27807.log 2024-05-08T08:56:23.939 INFO:teuthology.orchestra.run.smithi196.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.26545.log.gz 2024-05-08T08:56:23.939 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.29497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29207.log 2024-05-08T08:56:23.939 INFO:teuthology.orchestra.run.smithi196.stderr: 57.6% -- replaced with /var/log/ceph/ceph-client.admin.29497.log.gz 2024-05-08T08:56:23.939 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.27807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28647.log 2024-05-08T08:56:23.939 INFO:teuthology.orchestra.run.smithi196.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.27807.log.gz 2024-05-08T08:56:23.939 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.29207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28473.log 2024-05-08T08:56:23.939 INFO:teuthology.orchestra.run.smithi196.stderr: 20.9% -- replaced with /var/log/ceph/ceph-client.admin.29207.log.gz 2024-05-08T08:56:23.939 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.28647.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27782.log 2024-05-08T08:56:23.939 INFO:teuthology.orchestra.run.smithi196.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.28647.log.gz 2024-05-08T08:56:23.939 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.28473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28123.log 2024-05-08T08:56:23.939 INFO:teuthology.orchestra.run.smithi196.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.28473.log.gz 2024-05-08T08:56:23.939 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.27782.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28726.log 2024-05-08T08:56:23.939 INFO:teuthology.orchestra.run.smithi196.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.27782.log.gz 2024-05-08T08:56:23.939 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27002.log 2024-05-08T08:56:23.940 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.28123.log: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.28123.log.gz 2024-05-08T08:56:23.940 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.28726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27176.log 2024-05-08T08:56:23.940 INFO:teuthology.orchestra.run.smithi196.stderr: 25.3% -- replaced with /var/log/ceph/ceph-client.admin.28726.log.gz 2024-05-08T08:56:23.940 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.27002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28762.log 2024-05-08T08:56:23.940 INFO:teuthology.orchestra.run.smithi196.stderr: 20.9% -- replaced with /var/log/ceph/ceph-client.admin.27002.log.gz 2024-05-08T08:56:23.940 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.27176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27992.log 2024-05-08T08:56:23.940 INFO:teuthology.orchestra.run.smithi196.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.27176.log.gz 2024-05-08T08:56:23.940 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.28762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29594.log 2024-05-08T08:56:23.940 INFO:teuthology.orchestra.run.smithi196.stderr: 53.7% -- replaced with /var/log/ceph/ceph-client.admin.28762.log.gz 2024-05-08T08:56:23.940 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.27992.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28367.log 2024-05-08T08:56:23.940 INFO:teuthology.orchestra.run.smithi196.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.27992.log.gz 2024-05-08T08:56:23.940 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.29594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28832.log 2024-05-08T08:56:23.940 INFO:teuthology.orchestra.run.smithi196.stderr: 20.9% -- replaced with /var/log/ceph/ceph-client.admin.29594.log.gz 2024-05-08T08:56:23.940 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.28367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28166.log 2024-05-08T08:56:23.940 INFO:teuthology.orchestra.run.smithi196.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.28367.log.gz 2024-05-08T08:56:23.940 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.28832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27316.log 2024-05-08T08:56:23.941 INFO:teuthology.orchestra.run.smithi196.stderr: 53.9% -- replaced with /var/log/ceph/ceph-client.admin.28832.log.gz 2024-05-08T08:56:23.941 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.28166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29076.log 2024-05-08T08:56:23.941 INFO:teuthology.orchestra.run.smithi196.stderr: 24.7% -- replaced with /var/log/ceph/ceph-client.admin.28166.log.gz 2024-05-08T08:56:23.941 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.27316.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26765.log 2024-05-08T08:56:23.941 INFO:teuthology.orchestra.run.smithi196.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.27316.log.gz 2024-05-08T08:56:23.941 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.29076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28823.log 2024-05-08T08:56:23.941 INFO:teuthology.orchestra.run.smithi196.stderr: 53.4% -- replaced with /var/log/ceph/ceph-client.admin.29076.log.gz 2024-05-08T08:56:23.941 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.26765.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29173.log 2024-05-08T08:56:23.941 INFO:teuthology.orchestra.run.smithi196.stderr: 26.5% -- replaced with /var/log/ceph/ceph-client.admin.26765.log.gz 2024-05-08T08:56:23.941 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.28823.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28412.log 2024-05-08T08:56:23.941 INFO:teuthology.orchestra.run.smithi196.stderr: 20.9% -- replaced with /var/log/ceph/ceph-client.admin.28823.log.gz 2024-05-08T08:56:23.941 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.29173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26511.log 2024-05-08T08:56:23.941 INFO:teuthology.orchestra.run.smithi196.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.29173.log.gz 2024-05-08T08:56:23.941 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.28412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28053.log 2024-05-08T08:56:23.941 INFO:teuthology.orchestra.run.smithi196.stderr: 54.7% -- replaced with /var/log/ceph/ceph-client.admin.28412.log.gz 2024-05-08T08:56:23.941 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.26511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26441.log 2024-05-08T08:56:23.942 INFO:teuthology.orchestra.run.smithi196.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.26511.log.gz 2024-05-08T08:56:23.942 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.28053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26241.log 2024-05-08T08:56:23.942 INFO:teuthology.orchestra.run.smithi196.stderr: 19.0% -- replaced with /var/log/ceph/ceph-client.admin.28053.log.gz 2024-05-08T08:56:23.942 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.26441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26581.log 2024-05-08T08:56:23.942 INFO:teuthology.orchestra.run.smithi196.stderr: 20.9% -- replaced with /var/log/ceph/ceph-client.admin.26441.log.gz 2024-05-08T08:56:23.942 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.26241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29463.log 2024-05-08T08:56:23.942 INFO:teuthology.orchestra.run.smithi196.stderr: 20.9% -- replaced with /var/log/ceph/ceph-client.admin.26241.log.gz 2024-05-08T08:56:23.942 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.26581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29042.log 2024-05-08T08:56:23.942 INFO:teuthology.orchestra.run.smithi196.stderr: 20.9% -- replaced with /var/log/ceph/ceph-client.admin.26581.log.gz 2024-05-08T08:56:23.942 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.29463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29454.log 2024-05-08T08:56:23.942 INFO:teuthology.orchestra.run.smithi196.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.29463.log.gz 2024-05-08T08:56:23.942 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.29042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26335.log 2024-05-08T08:56:23.942 INFO:teuthology.orchestra.run.smithi196.stderr: 56.6% -- replaced with /var/log/ceph/ceph-client.admin.29042.log.gz 2024-05-08T08:56:23.942 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.29454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27361.log 2024-05-08T08:56:23.943 INFO:teuthology.orchestra.run.smithi196.stderr: 20.9% -- replaced with /var/log/ceph/ceph-client.admin.29454.log.gz 2024-05-08T08:56:23.943 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.26335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.26208.log 2024-05-08T08:56:23.943 INFO:teuthology.orchestra.run.smithi196.stderr: 20.9% -- replaced with /var/log/ceph/ceph-client.admin.26335.log.gz 2024-05-08T08:56:23.943 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.27361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26379.log 2024-05-08T08:56:23.943 INFO:teuthology.orchestra.run.smithi196.stderr: 84.8% -- replaced with /var/log/ceph/ceph-client.admin.27361.log.gz 2024-05-08T08:56:23.943 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.0.26208.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29488.log 2024-05-08T08:56:23.943 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.26379.log: 94.0% -- replaced with /var/log/ceph/ceph-client.0.26208.log.gz 2024-05-08T08:56:23.943 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27072.log 2024-05-08T08:56:23.943 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.29488.log: 95.8% -- replaced with /var/log/ceph/ceph-client.admin.26379.log.gz 2024-05-08T08:56:23.943 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29698.log 2024-05-08T08:56:23.943 INFO:teuthology.orchestra.run.smithi196.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.29488.log.gz 2024-05-08T08:56:23.943 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.27072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28796.log 2024-05-08T08:56:23.943 INFO:teuthology.orchestra.run.smithi196.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.27072.log.gz 2024-05-08T08:56:23.943 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.29698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27526.log 2024-05-08T08:56:23.943 INFO:teuthology.orchestra.run.smithi196.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.29698.log.gz 2024-05-08T08:56:23.943 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.28796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27737.log 2024-05-08T08:56:23.944 INFO:teuthology.orchestra.run.smithi196.stderr: 25.3% -- replaced with /var/log/ceph/ceph-client.admin.28796.log.gz 2024-05-08T08:56:23.944 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.27526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28717.log 2024-05-08T08:56:23.944 INFO:teuthology.orchestra.run.smithi196.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.27526.log.gz 2024-05-08T08:56:23.944 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.27737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29146.log 2024-05-08T08:56:23.944 INFO:teuthology.orchestra.run.smithi196.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.27737.log.gz 2024-05-08T08:56:23.944 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.28717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27036.log 2024-05-08T08:56:23.944 INFO:teuthology.orchestra.run.smithi196.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.28717.log.gz 2024-05-08T08:56:23.944 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.29146.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29558.log 2024-05-08T08:56:23.944 INFO:teuthology.orchestra.run.smithi196.stderr: 80.3% -- replaced with /var/log/ceph/ceph-client.admin.29146.log.gz 2024-05-08T08:56:23.944 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.27036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26756.log 2024-05-08T08:56:23.944 INFO:teuthology.orchestra.run.smithi196.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.27036.log.gz 2024-05-08T08:56:23.944 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.29558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26835.log 2024-05-08T08:56:23.944 INFO:teuthology.orchestra.run.smithi196.stderr: 21.5% -- replaced with /var/log/ceph/ceph-client.admin.29558.log.gz 2024-05-08T08:56:23.944 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.26756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29357.log 2024-05-08T08:56:23.944 INFO:teuthology.orchestra.run.smithi196.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.26756.log.gz 2024-05-08T08:56:23.944 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.26835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28297.log 2024-05-08T08:56:23.945 INFO:teuthology.orchestra.run.smithi196.stderr: 52.7% -- replaced with /var/log/ceph/ceph-client.admin.26835.log.gz 2024-05-08T08:56:23.945 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.29357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28902.log 2024-05-08T08:56:23.945 INFO:teuthology.orchestra.run.smithi196.stderr: 55.2% -- replaced with /var/log/ceph/ceph-client.admin.29357.log.gz 2024-05-08T08:56:23.945 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.28297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28157.log 2024-05-08T08:56:23.945 INFO:teuthology.orchestra.run.smithi196.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.28297.log.gz 2024-05-08T08:56:23.945 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.28902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27983.log 2024-05-08T08:56:23.945 INFO:teuthology.orchestra.run.smithi196.stderr: 56.8% -- replaced with /var/log/ceph/ceph-client.admin.28902.log.gz 2024-05-08T08:56:23.945 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.28157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27212.log 2024-05-08T08:56:23.945 INFO:teuthology.orchestra.run.smithi196.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.28157.log.gz 2024-05-08T08:56:23.945 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.27983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27221.log 2024-05-08T08:56:23.945 INFO:teuthology.orchestra.run.smithi196.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.27983.log.gz 2024-05-08T08:56:23.945 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.27212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29418.log 2024-05-08T08:56:23.945 INFO:teuthology.orchestra.run.smithi196.stderr: 20.9% -- replaced with /var/log/ceph/ceph-client.admin.27212.log.gz 2024-05-08T08:56:23.945 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.27221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28236.log 2024-05-08T08:56:23.945 INFO:teuthology.orchestra.run.smithi196.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.27221.log.gz 2024-05-08T08:56:23.945 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.29418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28753.log 2024-05-08T08:56:23.946 INFO:teuthology.orchestra.run.smithi196.stderr: 20.9% -- replaced with /var/log/ceph/ceph-client.admin.29418.log.gz 2024-05-08T08:56:23.946 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.28236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28096.log 2024-05-08T08:56:23.946 INFO:teuthology.orchestra.run.smithi196.stderr: 25.9% -- replaced with /var/log/ceph/ceph-client.admin.28236.log.gz 2024-05-08T08:56:23.946 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.28753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26941.log 2024-05-08T08:56:23.946 INFO:teuthology.orchestra.run.smithi196.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.28753.log.gz 2024-05-08T08:56:23.946 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.28096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29673.log 2024-05-08T08:56:23.946 INFO:teuthology.orchestra.run.smithi196.stderr: 76.4% -- replaced with /var/log/ceph/ceph-client.admin.28096.log.gz 2024-05-08T08:56:23.946 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.26941.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29743.log 2024-05-08T08:56:23.946 INFO:teuthology.orchestra.run.smithi196.stderr: 95.6% -- replaced with /var/log/ceph/ceph-client.admin.26941.log.gz 2024-05-08T08:56:23.946 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.29673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29524.log 2024-05-08T08:56:23.946 INFO:teuthology.orchestra.run.smithi196.stderr: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.29673.log.gz 2024-05-08T08:56:23.946 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.29743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26652.log 2024-05-08T08:56:23.946 INFO:teuthology.orchestra.run.smithi196.stderr: 80.9% -- replaced with /var/log/ceph/ceph-client.admin.29743.log.gz 2024-05-08T08:56:23.946 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.29524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27492.log 2024-05-08T08:56:23.947 INFO:teuthology.orchestra.run.smithi196.stderr: 20.9% -- replaced with /var/log/ceph/ceph-client.admin.29524.log.gz 2024-05-08T08:56:23.947 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.26652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26344.log 2024-05-08T08:56:23.947 INFO:teuthology.orchestra.run.smithi196.stderr: 19.0% -- replaced with /var/log/ceph/ceph-client.admin.26652.log.gz 2024-05-08T08:56:23.947 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.27492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26413.log 2024-05-08T08:56:23.947 INFO:teuthology.orchestra.run.smithi196.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.27492.log.gz 2024-05-08T08:56:23.947 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.26344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28026.log 2024-05-08T08:56:23.947 INFO:teuthology.orchestra.run.smithi196.stderr: 53.3% -- replaced with /var/log/ceph/ceph-client.admin.26344.log.gz 2024-05-08T08:56:23.947 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.26413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29278.log 2024-05-08T08:56:23.947 INFO:teuthology.orchestra.run.smithi196.stderr: 53.4% -- replaced with /var/log/ceph/ceph-client.admin.26413.log.gz 2024-05-08T08:56:23.947 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.28026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27667.log 2024-05-08T08:56:23.947 INFO:teuthology.orchestra.run.smithi196.stderr: 55.0% -- replaced with /var/log/ceph/ceph-client.admin.28026.log.gz 2024-05-08T08:56:23.947 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.29278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28263.log 2024-05-08T08:56:23.947 INFO:teuthology.orchestra.run.smithi196.stderr: 19.0% -- replaced with /var/log/ceph/ceph-client.admin.29278.log.gz 2024-05-08T08:56:23.947 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.27667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27106.log 2024-05-08T08:56:23.947 INFO:teuthology.orchestra.run.smithi196.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.27667.log.gz 2024-05-08T08:56:23.947 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.28263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29033.log 2024-05-08T08:56:23.948 INFO:teuthology.orchestra.run.smithi196.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.28263.log.gz 2024-05-08T08:56:23.948 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.27106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29006.log 2024-05-08T08:56:23.948 INFO:teuthology.orchestra.run.smithi196.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.27106.log.gz 2024-05-08T08:56:23.948 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.29033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29323.log 2024-05-08T08:56:23.948 INFO:teuthology.orchestra.run.smithi196.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.29033.log.gz 2024-05-08T08:56:23.948 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.29006.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27185.log 2024-05-08T08:56:23.948 INFO:teuthology.orchestra.run.smithi196.stderr: 53.4% -- replaced with /var/log/ceph/ceph-client.admin.29006.log.gz 2024-05-08T08:56:23.948 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.29323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26695.log 2024-05-08T08:56:23.948 INFO:teuthology.orchestra.run.smithi196.stderr: 57.2% -- replaced with /var/log/ceph/ceph-client.admin.29323.log.gz 2024-05-08T08:56:23.948 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.27185.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27431.log 2024-05-08T08:56:23.948 INFO:teuthology.orchestra.run.smithi196.stderr: 53.0% -- replaced with /var/log/ceph/ceph-client.admin.27185.log.gz 2024-05-08T08:56:23.948 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.26695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29393.log 2024-05-08T08:56:23.948 INFO:teuthology.orchestra.run.smithi196.stderr: 25.3% -- replaced with /var/log/ceph/ceph-client.admin.26695.log.gz 2024-05-08T08:56:23.948 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.27431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27291.log 2024-05-08T08:56:23.948 INFO:teuthology.orchestra.run.smithi196.stderr: 94.3% -- replaced with /var/log/ceph/ceph-client.admin.27431.log.gz 2024-05-08T08:56:23.948 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.29393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27535.log 2024-05-08T08:56:23.949 INFO:teuthology.orchestra.run.smithi196.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.29393.log.gz 2024-05-08T08:56:23.949 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.27291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28132.log 2024-05-08T08:56:23.949 INFO:teuthology.orchestra.run.smithi196.stderr: 95.6% -- replaced with /var/log/ceph/ceph-client.admin.27291.log.gz 2024-05-08T08:56:23.949 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.27535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27606.log 2024-05-08T08:56:23.949 INFO:teuthology.orchestra.run.smithi196.stderr: 54.2% -- replaced with /var/log/ceph/ceph-client.admin.27535.log.gz 2024-05-08T08:56:23.949 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.28132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27142.log 2024-05-08T08:56:23.949 INFO:teuthology.orchestra.run.smithi196.stderr: 51.4% -- replaced with /var/log/ceph/ceph-client.admin.28132.log.gz 2024-05-08T08:56:23.949 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.27606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28656.log 2024-05-08T08:56:23.949 INFO:teuthology.orchestra.run.smithi196.stderr: 83.5% -- replaced with /var/log/ceph/ceph-client.admin.27606.log.gz 2024-05-08T08:56:23.949 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.27142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29253.log 2024-05-08T08:56:23.949 INFO:teuthology.orchestra.run.smithi196.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.27142.log.gz 2024-05-08T08:56:23.949 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.28656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27642.log 2024-05-08T08:56:23.949 INFO:teuthology.orchestra.run.smithi196.stderr: 52.5% -- replaced with /var/log/ceph/ceph-client.admin.28656.log.gz 2024-05-08T08:56:23.949 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.29253.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26871.log 2024-05-08T08:56:23.949 INFO:teuthology.orchestra.run.smithi196.stderr: 84.2% -- replaced with /var/log/ceph/ceph-client.admin.29253.log.gz 2024-05-08T08:56:23.949 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.27642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29628.log 2024-05-08T08:56:23.950 INFO:teuthology.orchestra.run.smithi196.stderr: 92.9% -- replaced with /var/log/ceph/ceph-client.admin.27642.log.gz 2024-05-08T08:56:23.950 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.26871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26450.log 2024-05-08T08:56:23.950 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.29628.log: gzip 95.1% -5 -- replaced with /var/log/ceph/ceph-client.admin.26871.log.gz --verbose 2024-05-08T08:56:23.950 INFO:teuthology.orchestra.run.smithi196.stderr: -- /var/log/ceph/ceph-client.admin.27325.log 2024-05-08T08:56:23.950 INFO:teuthology.orchestra.run.smithi196.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.29628.log.gz 2024-05-08T08:56:23.950 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.26450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28333.log 2024-05-08T08:56:23.950 INFO:teuthology.orchestra.run.smithi196.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.26450.log.gz 2024-05-08T08:56:23.950 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.27325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27712.log 2024-05-08T08:56:23.950 INFO:teuthology.orchestra.run.smithi196.stderr: 54.5% -- replaced with /var/log/ceph/ceph-client.admin.27325.log.gz 2024-05-08T08:56:23.950 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.28333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28482.log 2024-05-08T08:56:23.950 INFO:teuthology.orchestra.run.smithi196.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.28333.log.gz 2024-05-08T08:56:23.950 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.27712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29603.log 2024-05-08T08:56:23.950 INFO:teuthology.orchestra.run.smithi196.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.27712.log.gz 2024-05-08T08:56:23.950 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.28482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28866.log 2024-05-08T08:56:23.950 INFO:teuthology.orchestra.run.smithi196.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.28482.log.gz 2024-05-08T08:56:23.951 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.29603.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26250.log 2024-05-08T08:56:23.951 INFO:teuthology.orchestra.run.smithi196.stderr: 94.6% -- replaced with /var/log/ceph/ceph-client.admin.29603.log.gz 2024-05-08T08:56:23.951 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.28866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28087.log 2024-05-08T08:56:23.951 INFO:teuthology.orchestra.run.smithi196.stderr: 52.6% -- replaced with /var/log/ceph/ceph-client.admin.28866.log.gz 2024-05-08T08:56:23.951 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.26250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28202.log 2024-05-08T08:56:23.951 INFO:teuthology.orchestra.run.smithi196.stderr: 94.8% -- replaced with /var/log/ceph/ceph-client.admin.26250.log.gz 2024-05-08T08:56:23.951 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.28087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26615.log 2024-05-08T08:56:23.951 INFO:teuthology.orchestra.run.smithi196.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.28087.log.gz 2024-05-08T08:56:23.951 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.28202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26905.log 2024-05-08T08:56:23.951 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.26615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26624.log 2024-05-08T08:56:23.951 INFO:teuthology.orchestra.run.smithi196.stderr: 20.9% -- replaced with /var/log/ceph/ceph-client.admin.26615.log.gz 2024-05-08T08:56:23.951 INFO:teuthology.orchestra.run.smithi196.stderr: 96.4% -- replaced with /var/log/ceph/ceph-client.admin.28202.log.gz 2024-05-08T08:56:23.951 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.26905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27386.log 2024-05-08T08:56:23.951 INFO:teuthology.orchestra.run.smithi196.stderr: 56.6% -- replaced with /var/log/ceph/ceph-client.admin.26905.log.gz 2024-05-08T08:56:23.952 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.26624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26722.log 2024-05-08T08:56:23.952 INFO:teuthology.orchestra.run.smithi196.stderr: 54.8% -- replaced with /var/log/ceph/ceph-client.admin.26624.log.gz 2024-05-08T08:56:23.952 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.27386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27151.log 2024-05-08T08:56:23.952 INFO:teuthology.orchestra.run.smithi196.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.27386.log.gz 2024-05-08T08:56:23.952 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.26722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29244.log 2024-05-08T08:56:23.952 INFO:teuthology.orchestra.run.smithi196.stderr: 20.9% -- replaced with /var/log/ceph/ceph-client.admin.26722.log.gz 2024-05-08T08:56:23.952 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.27151.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29314.log 2024-05-08T08:56:23.952 INFO:teuthology.orchestra.run.smithi196.stderr: 93.8% -- replaced with /var/log/ceph/ceph-client.admin.27151.log.gz 2024-05-08T08:56:23.952 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.29244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26826.log 2024-05-08T08:56:23.952 INFO:teuthology.orchestra.run.smithi196.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.29244.log.gz 2024-05-08T08:56:23.952 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.29314.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27597.log 2024-05-08T08:56:23.952 INFO:teuthology.orchestra.run.smithi196.stderr: 19.6% -- replaced with /var/log/ceph/ceph-client.admin.29314.log.gz 2024-05-08T08:56:23.952 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.26826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28857.log 2024-05-08T08:56:23.952 INFO:teuthology.orchestra.run.smithi196.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.26826.log.gz 2024-05-08T08:56:23.952 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.27597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26475.log 2024-05-08T08:56:23.953 INFO:teuthology.orchestra.run.smithi196.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.27597.log.gz 2024-05-08T08:56:23.953 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.28857.log: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.28857.log.gz 2024-05-08T08:56:23.953 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.26475.log: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.26475.log.gz 2024-05-08T08:56:23.953 INFO:teuthology.orchestra.run.smithi196.stderr: 2024-05-08T08:56:23.953 INFO:teuthology.orchestra.run.smithi196.stderr:real 0m0.067s 2024-05-08T08:56:23.953 INFO:teuthology.orchestra.run.smithi196.stderr:user 0m0.068s 2024-05-08T08:56:23.953 INFO:teuthology.orchestra.run.smithi196.stderr:sys 0m0.097s 2024-05-08T08:56:23.962 INFO:teuthology.orchestra.run.smithi192.stderr: 91.0% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-05-08T08:56:24.023 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47508.log 2024-05-08T08:56:24.023 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33632.log 2024-05-08T08:56:24.023 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.47508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45468.log 2024-05-08T08:56:24.023 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47508.log.gz 2024-05-08T08:56:24.023 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.33632.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33632.log.gz 2024-05-08T08:56:24.023 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58640.log 2024-05-08T08:56:24.023 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.45468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44940.log 2024-05-08T08:56:24.023 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45468.log.gz 2024-05-08T08:56:24.023 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.58640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34539.log 2024-05-08T08:56:24.023 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58640.log.gz 2024-05-08T08:56:24.023 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.44940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54440.log 2024-05-08T08:56:24.023 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44940.log.gz 2024-05-08T08:56:24.023 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.34539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45903.log 2024-05-08T08:56:24.024 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34539.log.gz 2024-05-08T08:56:24.024 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.54440.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39827.log 2024-05-08T08:56:24.024 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54440.log.gz 2024-05-08T08:56:24.024 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.45903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37985.log 2024-05-08T08:56:24.024 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45903.log.gz 2024-05-08T08:56:24.024 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.39827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58682.log 2024-05-08T08:56:24.024 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39827.log.gz 2024-05-08T08:56:24.024 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.37985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38304.log 2024-05-08T08:56:24.024 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37985.log.gz 2024-05-08T08:56:24.024 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.58682.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60236.log 2024-05-08T08:56:24.024 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58682.log.gz 2024-05-08T08:56:24.024 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.38304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54307.log 2024-05-08T08:56:24.024 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38304.log.gz 2024-05-08T08:56:24.024 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.60236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59452.log 2024-05-08T08:56:24.024 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60236.log.gz 2024-05-08T08:56:24.024 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.54307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47649.log 2024-05-08T08:56:24.025 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54307.log.gz 2024-05-08T08:56:24.025 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.59452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60572.log 2024-05-08T08:56:24.025 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59452.log.gz 2024-05-08T08:56:24.025 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.47649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35788.log 2024-05-08T08:56:24.025 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47649.log.gz 2024-05-08T08:56:24.025 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.60572.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.26464.log 2024-05-08T08:56:24.025 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60572.log.gz 2024-05-08T08:56:24.025 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.35788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59938.log 2024-05-08T08:56:24.025 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35788.log.gz 2024-05-08T08:56:24.025 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph.tmp-client.admin.26464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58934.log 2024-05-08T08:56:24.025 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.26464.log.gz 2024-05-08T08:56:24.025 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.59938.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45689.log 2024-05-08T08:56:24.025 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59938.log.gz 2024-05-08T08:56:24.025 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.58934.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54242.log 2024-05-08T08:56:24.025 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58934.log.gz 2024-05-08T08:56:24.025 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.45689.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58598.log 2024-05-08T08:56:24.025 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45689.log.gz 2024-05-08T08:56:24.026 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.54242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47963.log 2024-05-08T08:56:24.026 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54242.log.gz 2024-05-08T08:56:24.026 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.58598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40115.log 2024-05-08T08:56:24.026 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58598.log.gz 2024-05-08T08:56:24.026 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.47963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34131.log 2024-05-08T08:56:24.026 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47963.log.gz 2024-05-08T08:56:24.026 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.40115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43379.log 2024-05-08T08:56:24.026 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40115.log.gz 2024-05-08T08:56:24.026 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.34131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33867.log 2024-05-08T08:56:24.026 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34131.log.gz 2024-05-08T08:56:24.026 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.43379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51255.log 2024-05-08T08:56:24.026 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43379.log.gz 2024-05-08T08:56:24.026 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.33867.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-05-08T08:56:24.026 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33867.log.gz 2024-05-08T08:56:24.026 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.51255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45073.log 2024-05-08T08:56:24.027 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51255.log.gz 2024-05-08T08:56:24.027 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49475.log 2024-05-08T08:56:24.027 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.45073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56482.log 2024-05-08T08:56:24.027 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45073.log.gz 2024-05-08T08:56:24.027 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.49475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39775.log 2024-05-08T08:56:24.027 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49475.log.gz 2024-05-08T08:56:24.027 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.56482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34599.log 2024-05-08T08:56:24.027 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56482.log.gz 2024-05-08T08:56:24.027 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.39775.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39775.log.gz 2024-05-08T08:56:24.027 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48841.log 2024-05-08T08:56:24.027 INFO:teuthology.orchestra.run.smithi059.stderr: 91.8% -- replaced with /var/log/ceph/ceph.log.gz 2024-05-08T08:56:24.027 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46568.log 2024-05-08T08:56:24.027 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.34599.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34599.log.gz 2024-05-08T08:56:24.027 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.48841.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-05-08T08:56:24.027 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48841.log.gz 2024-05-08T08:56:24.027 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.46568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33909.log 2024-05-08T08:56:24.027 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46568.log.gz 2024-05-08T08:56:24.028 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34388.log 2024-05-08T08:56:24.028 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.33909.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33909.log.gzgzip 2024-05-08T08:56:24.028 INFO:teuthology.orchestra.run.smithi059.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.60018.log 2024-05-08T08:56:24.028 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.34388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60488.log 2024-05-08T08:56:24.028 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34388.log.gz 2024-05-08T08:56:24.028 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.60018.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59453.log 2024-05-08T08:56:24.028 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60018.log.gz 2024-05-08T08:56:24.028 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.60488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59200.log 2024-05-08T08:56:24.028 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60488.log.gz 2024-05-08T08:56:24.028 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.59453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31138.log 2024-05-08T08:56:24.028 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59453.log.gz 2024-05-08T08:56:24.028 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.59200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33951.log 2024-05-08T08:56:24.028 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59200.log.gz 2024-05-08T08:56:24.028 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.31138.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31138.log.gz 2024-05-08T08:56:24.028 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34659.log 2024-05-08T08:56:24.028 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.33951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37166.log 2024-05-08T08:56:24.029 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33951.log.gz 2024-05-08T08:56:24.029 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.34659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59454.log 2024-05-08T08:56:24.029 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34659.log.gz 2024-05-08T08:56:24.029 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.37166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43063.log 2024-05-08T08:56:24.029 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37166.log.gz 2024-05-08T08:56:24.029 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.59454.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.59454.log.gz 2024-05-08T08:56:24.029 INFO:teuthology.orchestra.run.smithi059.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.43703.log 2024-05-08T08:56:24.029 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.43063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36892.log 2024-05-08T08:56:24.029 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43063.log.gz 2024-05-08T08:56:24.029 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.43703.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-05-08T08:56:24.029 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43703.log.gz 2024-05-08T08:56:24.029 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.36892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41433.log 2024-05-08T08:56:24.029 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36892.log.gz 2024-05-08T08:56:24.029 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59606.log 2024-05-08T08:56:24.029 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.41433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60278.log 2024-05-08T08:56:24.029 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41433.log.gz 2024-05-08T08:56:24.029 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.59606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40503.log 2024-05-08T08:56:24.030 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59606.log.gz 2024-05-08T08:56:24.030 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.60278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35613.log 2024-05-08T08:56:24.030 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60278.log.gz 2024-05-08T08:56:24.030 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.40503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47689.log 2024-05-08T08:56:24.030 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40503.log.gz 2024-05-08T08:56:24.030 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.35613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52095.log 2024-05-08T08:56:24.030 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35613.log.gz 2024-05-08T08:56:24.030 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.47689.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55677.log 2024-05-08T08:56:24.030 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47689.log.gz 2024-05-08T08:56:24.030 INFO:teuthology.orchestra.run.smithi059.stderr: 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-05-08T08:56:24.030 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.52095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58467.log 2024-05-08T08:56:24.030 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52095.log.gz 2024-05-08T08:56:24.030 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.55677.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40957.log 2024-05-08T08:56:24.030 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55677.log.gz 2024-05-08T08:56:24.030 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.58467.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44529.log 2024-05-08T08:56:24.031 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58467.log.gz 2024-05-08T08:56:24.031 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.40957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41551.log 2024-05-08T08:56:24.031 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40957.log.gz 2024-05-08T08:56:24.031 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.44529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34135.log 2024-05-08T08:56:24.031 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44529.log.gz 2024-05-08T08:56:24.031 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.41551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45328.log 2024-05-08T08:56:24.031 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41551.log.gz 2024-05-08T08:56:24.031 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.34135.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54848.log 2024-05-08T08:56:24.031 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34135.log.gz 2024-05-08T08:56:24.031 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.45328.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.45328.log.gz /var/log/ceph/ceph-client.admin.38302.log 2024-05-08T08:56:24.031 INFO:teuthology.orchestra.run.smithi059.stderr: 2024-05-08T08:56:24.031 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.54848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43641.log 2024-05-08T08:56:24.031 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54848.log.gz 2024-05-08T08:56:24.031 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.38302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35366.log 2024-05-08T08:56:24.031 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38302.log.gz 2024-05-08T08:56:24.031 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.43641.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.43641.log.gz /var/log/ceph/ceph-client.admin.35228.log 2024-05-08T08:56:24.032 INFO:teuthology.orchestra.run.smithi059.stderr: 2024-05-08T08:56:24.032 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.35366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36490.log 2024-05-08T08:56:24.032 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35366.log.gz 2024-05-08T08:56:24.032 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.35228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34308.log 2024-05-08T08:56:24.032 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35228.log.gz 2024-05-08T08:56:24.032 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.36490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55052.log 2024-05-08T08:56:24.032 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36490.log.gz 2024-05-08T08:56:24.032 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.34308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55994.log 2024-05-08T08:56:24.032 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34308.log.gz 2024-05-08T08:56:24.032 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.55052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36768.log 2024-05-08T08:56:24.032 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55052.log.gz 2024-05-08T08:56:24.032 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.55994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49862.log 2024-05-08T08:56:24.032 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55994.log.gz 2024-05-08T08:56:24.032 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.36768.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36768.log.gz -5 2024-05-08T08:56:24.032 INFO:teuthology.orchestra.run.smithi059.stderr: --verbose -- /var/log/ceph/ceph-client.admin.45649.log 2024-05-08T08:56:24.032 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.49862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30835.log 2024-05-08T08:56:24.032 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49862.log.gz 2024-05-08T08:56:24.033 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.45649.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.56502.log 2024-05-08T08:56:24.033 INFO:teuthology.orchestra.run.smithi059.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45649.log.gz 2024-05-08T08:56:24.033 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.30835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50373.log 2024-05-08T08:56:24.033 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30835.log.gz 2024-05-08T08:56:24.033 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.56502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51798.log 2024-05-08T08:56:24.033 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56502.log.gz 2024-05-08T08:56:24.033 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.50373.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40188.log 2024-05-08T08:56:24.033 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50373.log.gz 2024-05-08T08:56:24.033 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.51798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46863.log 2024-05-08T08:56:24.033 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51798.log.gz 2024-05-08T08:56:24.033 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.40188.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34132.log 2024-05-08T08:56:24.033 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40188.log.gz 2024-05-08T08:56:24.033 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.46863.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.34129.log 2024-05-08T08:56:24.033 INFO:teuthology.orchestra.run.smithi059.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46863.log.gz 2024-05-08T08:56:24.033 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.34132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41773.log 2024-05-08T08:56:24.033 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34132.log.gz 2024-05-08T08:56:24.033 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.34129.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.34129.log.gz 2024-05-08T08:56:24.034 INFO:teuthology.orchestra.run.smithi059.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.60740.log 2024-05-08T08:56:24.034 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.41773.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57951.log 2024-05-08T08:56:24.034 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41773.log.gz 2024-05-08T08:56:24.034 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.60740.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.60740.log.gz 2024-05-08T08:56:24.034 INFO:teuthology.orchestra.run.smithi059.stderr: -- /var/log/ceph/ceph-client.admin.42870.log 2024-05-08T08:56:24.034 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.57951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47568.log 2024-05-08T08:56:24.034 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57951.log.gz 2024-05-08T08:56:24.034 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.42870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41908.log 2024-05-08T08:56:24.034 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42870.log.gz 2024-05-08T08:56:24.034 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.47568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59975.log 2024-05-08T08:56:24.034 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47568.log.gz 2024-05-08T08:56:24.034 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.41908.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41139.log 2024-05-08T08:56:24.034 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41908.log.gz 2024-05-08T08:56:24.034 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.59975.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59744.log 2024-05-08T08:56:24.034 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59975.log.gz 2024-05-08T08:56:24.034 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.41139.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60110.log 2024-05-08T08:56:24.035 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41139.log.gz 2024-05-08T08:56:24.035 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.59744.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56268.log 2024-05-08T08:56:24.035 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59744.log.gz 2024-05-08T08:56:24.035 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.60110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58402.log 2024-05-08T08:56:24.035 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60110.log.gz 2024-05-08T08:56:24.035 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.56268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34128.log 2024-05-08T08:56:24.035 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56268.log.gz 2024-05-08T08:56:24.035 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.58402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49710.log 2024-05-08T08:56:24.035 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58402.log.gz 2024-05-08T08:56:24.035 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.34128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39245.log 2024-05-08T08:56:24.035 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34128.log.gz 2024-05-08T08:56:24.035 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.49710.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60866.log 2024-05-08T08:56:24.035 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49710.log.gz 2024-05-08T08:56:24.035 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.39245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56850.log 2024-05-08T08:56:24.035 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39245.log.gz 2024-05-08T08:56:24.035 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.60866.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.60866.log.gz 2024-05-08T08:56:24.036 INFO:teuthology.orchestra.run.smithi059.stderr: -- /var/log/ceph/ceph-client.admin.59199.log 2024-05-08T08:56:24.036 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.56850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34959.log 2024-05-08T08:56:24.036 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56850.log.gz 2024-05-08T08:56:24.036 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.59199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39142.log 2024-05-08T08:56:24.036 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59199.log.gz 2024-05-08T08:56:24.036 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.34959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35043.log 2024-05-08T08:56:24.036 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34959.log.gz 2024-05-08T08:56:24.036 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.39142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59197.log 2024-05-08T08:56:24.036 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39142.log.gz 2024-05-08T08:56:24.036 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.35043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53711.log 2024-05-08T08:56:24.036 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35043.log.gz 2024-05-08T08:56:24.036 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.59197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42143.log 2024-05-08T08:56:24.036 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59197.log.gz 2024-05-08T08:56:24.036 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.53711.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.53711.log.gz /var/log/ceph/ceph-client.admin.44683.log 2024-05-08T08:56:24.036 INFO:teuthology.orchestra.run.smithi059.stderr: 2024-05-08T08:56:24.036 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.42143.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59878.log 2024-05-08T08:56:24.036 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42143.log.gz 2024-05-08T08:56:24.037 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.44683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60990.log 0.0% 2024-05-08T08:56:24.037 INFO:teuthology.orchestra.run.smithi059.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44683.log.gz 2024-05-08T08:56:24.037 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.59878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60194.log 2024-05-08T08:56:24.037 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59878.log.gz 2024-05-08T08:56:24.037 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.60990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55247.log 2024-05-08T08:56:24.037 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60990.log.gz 2024-05-08T08:56:24.037 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.60194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57696.log 2024-05-08T08:56:24.037 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60194.log.gz 2024-05-08T08:56:24.037 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.55247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52549.log 2024-05-08T08:56:24.037 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55247.log.gz 2024-05-08T08:56:24.037 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.57696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57187.log 2024-05-08T08:56:24.037 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57696.log.gz 2024-05-08T08:56:24.037 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.52549.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52549.log.gz -5 2024-05-08T08:56:24.037 INFO:teuthology.orchestra.run.smithi059.stderr: --verbose -- /var/log/ceph/ceph-client.admin.60698.log 2024-05-08T08:56:24.037 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.57187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44020.log 2024-05-08T08:56:24.037 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57187.log.gz 2024-05-08T08:56:24.037 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.60698.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-05-08T08:56:24.038 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60698.log.gz 2024-05-08T08:56:24.038 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.44020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34327.log 2024-05-08T08:56:24.038 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44020.log.gz 2024-05-08T08:56:24.038 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36123.log 2024-05-08T08:56:24.038 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.34327.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.34327.log.gz 2024-05-08T08:56:24.038 INFO:teuthology.orchestra.run.smithi059.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.33993.log 2024-05-08T08:56:24.038 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.36123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44447.log 2024-05-08T08:56:24.038 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36123.log.gz 2024-05-08T08:56:24.038 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.33993.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50907.log 2024-05-08T08:56:24.038 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33993.log.gz 2024-05-08T08:56:24.038 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.44447.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54059.log 2024-05-08T08:56:24.038 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44447.log.gz 2024-05-08T08:56:24.038 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.50907.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-05-08T08:56:24.038 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50907.log.gz 2024-05-08T08:56:24.038 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.54059.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54059.log.gz -5 2024-05-08T08:56:24.039 INFO:teuthology.orchestra.run.smithi059.stderr: --verbose -- /var/log/ceph/ceph-client.admin.42080.log 2024-05-08T08:56:24.039 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37065.log 2024-05-08T08:56:24.039 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.42080.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47983.log 2024-05-08T08:56:24.039 INFO:teuthology.orchestra.run.smithi059.stderr: 28.0% -- replaced with /var/log/ceph/ceph-client.admin.42080.log.gz 2024-05-08T08:56:24.039 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.37065.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.38047.log -- replaced with /var/log/ceph/ceph-client.admin.37065.log.gz 2024-05-08T08:56:24.039 INFO:teuthology.orchestra.run.smithi059.stderr: 2024-05-08T08:56:24.039 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52115.log 2024-05-08T08:56:24.039 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.47983.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47983.log.gz 2024-05-08T08:56:24.039 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.38047.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38047.log.gz 2024-05-08T08:56:24.039 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42337.log 2024-05-08T08:56:24.039 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.52115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59858.log 2024-05-08T08:56:24.039 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52115.log.gz 2024-05-08T08:56:24.039 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.42337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37360.log 2024-05-08T08:56:24.039 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42337.log.gz 2024-05-08T08:56:24.039 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.59858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34389.log 2024-05-08T08:56:24.039 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59858.log.gz 2024-05-08T08:56:24.039 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.37360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44763.log 2024-05-08T08:56:24.040 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37360.log.gz 2024-05-08T08:56:24.040 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58808.log 2024-05-08T08:56:24.040 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.34389.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34389.log.gz 2024-05-08T08:56:24.040 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.44763.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55779.log 2024-05-08T08:56:24.040 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44763.log.gz 2024-05-08T08:56:24.040 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.58808.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58808.log.gz 2024-05-08T08:56:24.040 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39184.log 2024-05-08T08:56:24.040 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34559.log 2024-05-08T08:56:24.040 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.55779.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55779.log.gz 2024-05-08T08:56:24.040 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.39184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37843.log 2024-05-08T08:56:24.040 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39184.log.gz 2024-05-08T08:56:24.040 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.34559.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.34559.log.gz --verbose 2024-05-08T08:56:24.040 INFO:teuthology.orchestra.run.smithi059.stderr: -- /var/log/ceph/ceph-client.admin.36204.log 2024-05-08T08:56:24.040 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39029.log 2024-05-08T08:56:24.040 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.37843.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37843.log.gz 2024-05-08T08:56:24.040 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.36204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45609.log 2024-05-08T08:56:24.040 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36204.log.gz 2024-05-08T08:56:24.041 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.39029.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.39029.log.gz 2024-05-08T08:56:24.041 INFO:teuthology.orchestra.run.smithi059.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.57421.log 2024-05-08T08:56:24.041 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35448.log 2024-05-08T08:56:24.041 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.45609.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45609.log.gz 2024-05-08T08:56:24.041 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.57421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35001.log 2024-05-08T08:56:24.041 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57421.log.gz 2024-05-08T08:56:24.041 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.35448.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43517.log 2024-05-08T08:56:24.041 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35448.log.gz 2024-05-08T08:56:24.041 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46609.log 2024-05-08T08:56:24.041 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.35001.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35001.log.gz 2024-05-08T08:56:24.041 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.43517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58299.log 2024-05-08T08:56:24.041 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43517.log.gz 2024-05-08T08:56:24.041 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.46609.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46609.log.gz 2024-05-08T08:56:24.041 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47355.log 2024-05-08T08:56:24.041 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52273.log 2024-05-08T08:56:24.041 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.58299.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58299.log.gz 2024-05-08T08:56:24.041 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.47355.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47355.log.gz 2024-05-08T08:56:24.042 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45923.log 2024-05-08T08:56:24.042 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36767.log 2024-05-08T08:56:24.042 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.52273.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52273.log.gz 2024-05-08T08:56:24.042 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.45923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43338.log 2024-05-08T08:56:24.042 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45923.log.gz 2024-05-08T08:56:24.042 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.36767.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.36767.log.gz 2024-05-08T08:56:24.042 INFO:teuthology.orchestra.run.smithi059.stderr: -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-05-08T08:56:24.042 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54715.log 2024-05-08T08:56:24.042 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.43338.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43338.log.gz 2024-05-08T08:56:24.042 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34917.log 2024-05-08T08:56:24.042 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.54715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36224.log 2024-05-08T08:56:24.042 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54715.log.gz 2024-05-08T08:56:24.042 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.34917.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40309.log 2024-05-08T08:56:24.042 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34917.log.gz 2024-05-08T08:56:24.042 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.36224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36615.log 2024-05-08T08:56:24.042 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36224.log.gz 2024-05-08T08:56:24.043 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.40309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37575.log 2024-05-08T08:56:24.043 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40309.log.gz 2024-05-08T08:56:24.043 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.36615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40864.log 2024-05-08T08:56:24.043 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36615.log.gz 2024-05-08T08:56:24.043 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.37575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42439.log 2024-05-08T08:56:24.043 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37575.log.gz 2024-05-08T08:56:24.043 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.40864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59449.log 2024-05-08T08:56:24.043 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40864.log.gz 2024-05-08T08:56:24.043 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.42439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60152.log 2024-05-08T08:56:24.043 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42439.log.gz 2024-05-08T08:56:24.043 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.59449.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56737.log 2024-05-08T08:56:24.043 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59449.log.gz 2024-05-08T08:56:24.043 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.60152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54490.log 2024-05-08T08:56:24.043 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60152.log.gz 2024-05-08T08:56:24.043 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.56737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55521.log 2024-05-08T08:56:24.043 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56737.log.gz 2024-05-08T08:56:24.043 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.54490.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-05-08T08:56:24.044 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54490.log.gz 2024-05-08T08:56:24.044 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.55521.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48923.log 2024-05-08T08:56:24.044 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55521.log.gz 2024-05-08T08:56:24.044 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46843.log 2024-05-08T08:56:24.044 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.48923.log: gzip -5 --verbose 0.0% -- /var/log/ceph/ceph-client.admin.59201.log 2024-05-08T08:56:24.044 INFO:teuthology.orchestra.run.smithi059.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48923.log.gz 2024-05-08T08:56:24.044 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.46843.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35225.log 2024-05-08T08:56:24.044 INFO:teuthology.orchestra.run.smithi059.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46843.log.gz 2024-05-08T08:56:24.044 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.59201.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48883.log 2024-05-08T08:56:24.044 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59201.log.gz 2024-05-08T08:56:24.044 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.35225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43105.log 2024-05-08T08:56:24.044 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35225.log.gz 2024-05-08T08:56:24.044 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48566.log 2024-05-08T08:56:24.044 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.48883.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48883.log.gz 2024-05-08T08:56:24.044 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59746.log 2024-05-08T08:56:24.044 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.43105.log: /var/log/ceph/ceph-client.admin.48566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42726.log 2024-05-08T08:56:24.044 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43105.log.gz 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48566.log.gz 2024-05-08T08:56:24.045 INFO:teuthology.orchestra.run.smithi059.stderr: 2024-05-08T08:56:24.045 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.59746.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59746.log.gz 2024-05-08T08:56:24.045 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51933.log 2024-05-08T08:56:24.045 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.42726.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42726.log.gz 2024-05-08T08:56:24.045 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53483.log 2024-05-08T08:56:24.045 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.51933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58064.log 2024-05-08T08:56:24.045 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51933.log.gz 2024-05-08T08:56:24.045 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.53483.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47729.log 2024-05-08T08:56:24.045 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53483.log.gz 2024-05-08T08:56:24.045 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57749.log 2024-05-08T08:56:24.045 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.58064.log: /var/log/ceph/ceph-client.admin.47729.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38191.log 2024-05-08T08:56:24.045 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58064.log.gz 2024-05-08T08:56:24.045 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47729.log.gz 2024-05-08T08:56:24.045 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.57749.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57749.log.gz 2024-05-08T08:56:24.045 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34381.log 2024-05-08T08:56:24.045 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.38191.log: /var/log/ceph/ceph-client.admin.59743.log 2024-05-08T08:56:24.045 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38191.log.gz 2024-05-08T08:56:24.046 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.34381.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34381.log.gz 2024-05-08T08:56:24.046 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42787.log 2024-05-08T08:56:24.046 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.59743.log: -- /var/log/ceph/ceph-client.admin.51377.log 2024-05-08T08:56:24.046 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59743.log.gz 2024-05-08T08:56:24.046 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.42787.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42787.log.gz 2024-05-08T08:56:24.046 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42653.log 2024-05-08T08:56:24.046 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.51377.log: -- /var/log/ceph/ceph-client.admin.58340.log 2024-05-08T08:56:24.046 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51377.log.gz/var/log/ceph/ceph-client.admin.42653.log: 2024-05-08T08:56:24.046 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.42653.log.gz -5 --verbose -- /var/log/ceph/ceph-client.admin.51142.log 2024-05-08T08:56:24.046 INFO:teuthology.orchestra.run.smithi059.stderr: 2024-05-08T08:56:24.046 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.58340.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58340.log.gz 2024-05-08T08:56:24.046 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47140.log 2024-05-08T08:56:24.046 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.51142.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51142.log.gz 2024-05-08T08:56:24.046 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60362.log 2024-05-08T08:56:24.047 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.47140.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47140.log.gz 2024-05-08T08:56:24.047 INFO:teuthology.orchestra.run.smithi059.stderr: --verbose -- /var/log/ceph/ceph-client.admin.40761.log 2024-05-08T08:56:24.047 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.60362.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60362.log.gz 2024-05-08T08:56:24.047 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35551.log 2024-05-08T08:56:24.047 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.40761.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40761.log.gz 2024-05-08T08:56:24.047 INFO:teuthology.orchestra.run.smithi059.stderr: --verbose -- /var/log/ceph/ceph-client.admin.36420.log 2024-05-08T08:56:24.047 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.35551.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35551.log.gz 2024-05-08T08:56:24.047 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59202.log 2024-05-08T08:56:24.047 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.36420.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36420.log.gz 2024-05-08T08:56:24.047 INFO:teuthology.orchestra.run.smithi059.stderr: --verbose -- /var/log/ceph/ceph-client.admin.34791.log 2024-05-08T08:56:24.047 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.59202.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59202.log.gz 2024-05-08T08:56:24.047 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46355.log 2024-05-08T08:56:24.047 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.34791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58766.log 2024-05-08T08:56:24.047 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34791.log.gz 2024-05-08T08:56:24.047 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.46355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59346.log 2024-05-08T08:56:24.047 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46355.log.gz 2024-05-08T08:56:24.047 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52824.log 2024-05-08T08:56:24.048 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.58766.log: /var/log/ceph/ceph-client.admin.59346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51418.log 2024-05-08T08:56:24.048 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58766.log.gz 2024-05-08T08:56:24.048 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59346.log.gz 2024-05-08T08:56:24.048 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.52824.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52824.log.gz 2024-05-08T08:56:24.048 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51694.log 2024-05-08T08:56:24.048 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.51418.log: -- /var/log/ceph/ceph-client.admin.40542.log 2024-05-08T08:56:24.048 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51418.log.gz 2024-05-08T08:56:24.048 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.51694.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51694.log.gz 2024-05-08T08:56:24.048 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58850.log 2024-05-08T08:56:24.048 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.40542.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40542.log.gz 2024-05-08T08:56:24.048 INFO:teuthology.orchestra.run.smithi059.stderr: --verbose -- /var/log/ceph/ceph-client.admin.53946.log 2024-05-08T08:56:24.048 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.58850.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58850.log.gz 2024-05-08T08:56:24.048 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53119.log 2024-05-08T08:56:24.048 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.53946.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53946.log.gz 2024-05-08T08:56:24.048 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60782.log 2024-05-08T08:56:24.048 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.53119.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53119.log.gz 2024-05-08T08:56:24.048 INFO:teuthology.orchestra.run.smithi059.stderr: --verbose -- /var/log/ceph/ceph-client.admin.58403.log 2024-05-08T08:56:24.049 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.60782.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60782.log.gz 2024-05-08T08:56:24.049 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37707.log 2024-05-08T08:56:24.049 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.58403.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58403.log.gz 2024-05-08T08:56:24.049 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52496.log 2024-05-08T08:56:24.049 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.37707.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37707.log.gz 2024-05-08T08:56:24.049 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58724.log 2024-05-08T08:56:24.049 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.52496.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52496.log.gz 2024-05-08T08:56:24.049 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38519.log 2024-05-08T08:56:24.049 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.58724.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58724.log.gz 2024-05-08T08:56:24.049 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34875.log 2024-05-08T08:56:24.049 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.38519.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38519.log.gz 2024-05-08T08:56:24.049 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60446.log 2024-05-08T08:56:24.049 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.34875.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34875.log.gz 2024-05-08T08:56:24.049 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34133.log 2024-05-08T08:56:24.049 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.60446.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60446.log.gz 2024-05-08T08:56:24.049 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58892.log 2024-05-08T08:56:24.050 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.34133.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34133.log.gz 2024-05-08T08:56:24.050 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51510.log 2024-05-08T08:56:24.050 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.58892.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58892.log.gz 2024-05-08T08:56:24.050 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46251.log 2024-05-08T08:56:24.050 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.51510.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51510.log.gz 2024-05-08T08:56:24.050 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57441.log 2024-05-08T08:56:24.050 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.46251.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46251.log.gz 2024-05-08T08:56:24.050 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46913.log 2024-05-08T08:56:24.050 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.57441.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57441.log.gz 2024-05-08T08:56:24.050 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46098.log 2024-05-08T08:56:24.050 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.46913.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46913.log.gz 2024-05-08T08:56:24.050 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57145.log 2024-05-08T08:56:24.050 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.46098.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46098.log.gz 2024-05-08T08:56:24.050 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54039.log 2024-05-08T08:56:24.050 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.57145.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57145.log.gz 2024-05-08T08:56:24.050 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35226.log 2024-05-08T08:56:24.050 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.54039.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54039.log.gz 2024-05-08T08:56:24.051 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35930.log 2024-05-08T08:56:24.051 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.35226.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35226.log.gz 2024-05-08T08:56:24.051 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37790.log 2024-05-08T08:56:24.051 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.35930.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35930.log.gz 2024-05-08T08:56:24.051 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34307.log 2024-05-08T08:56:24.051 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.37790.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37790.log.gz 2024-05-08T08:56:24.051 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50608.log 2024-05-08T08:56:24.051 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.34307.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34307.log.gz 2024-05-08T08:56:24.051 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52926.log 2024-05-08T08:56:24.051 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.50608.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50608.log.gz 2024-05-08T08:56:24.051 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53764.log 2024-05-08T08:56:24.051 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.52926.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52926.log.gz 2024-05-08T08:56:24.051 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52711.log 2024-05-08T08:56:24.051 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.53764.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48291.log 2024-05-08T08:56:24.051 INFO:teuthology.orchestra.run.smithi059.stderr: -- replaced with /var/log/ceph/ceph-client.admin.53764.log.gz 2024-05-08T08:56:24.051 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.52711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49943.log 2024-05-08T08:56:24.052 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52711.log.gz 2024-05-08T08:56:24.052 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58976.log 2024-05-08T08:56:24.052 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.48291.log: /var/log/ceph/ceph-client.admin.49943.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48291.log.gz 2024-05-08T08:56:24.052 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43579.log 2024-05-08T08:56:24.052 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49943.log.gz 2024-05-08T08:56:24.052 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53414.log 2024-05-08T08:56:24.052 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.58976.log: /var/log/ceph/ceph-client.admin.43579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59978.log 2024-05-08T08:56:24.052 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58976.log.gz 2024-05-08T08:56:24.052 INFO:teuthology.orchestra.run.smithi059.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43579.log.gz 2024-05-08T08:56:24.052 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.53414.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53414.log.gz 2024-05-08T08:56:24.052 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41432.log 2024-05-08T08:56:24.052 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.59978.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59978.log.gz 2024-05-08T08:56:24.052 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34658.log 2024-05-08T08:56:24.052 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.41432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37380.log 2024-05-08T08:56:24.052 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41432.log.gz 2024-05-08T08:56:24.053 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.34658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45388.log 2024-05-08T08:56:24.053 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34658.log.gz 2024-05-08T08:56:24.053 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39871.log 2024-05-08T08:56:24.053 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.37380.log: /var/log/ceph/ceph-client.admin.45388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40679.log 2024-05-08T08:56:24.053 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45388.log.gz 2024-05-08T08:56:24.053 INFO:teuthology.orchestra.run.smithi059.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37380.log.gz 2024-05-08T08:56:24.053 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.39871.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39871.log.gz 2024-05-08T08:56:24.053 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55051.log 2024-05-08T08:56:24.053 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.40679.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40679.log.gz 2024-05-08T08:56:24.053 INFO:teuthology.orchestra.run.smithi059.stderr: --verbose -- /var/log/ceph/ceph-client.admin.34833.log 2024-05-08T08:56:24.053 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.55051.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55051.log.gz 2024-05-08T08:56:24.053 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38584.log 2024-05-08T08:56:24.053 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.34833.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34833.log.gz 2024-05-08T08:56:24.053 INFO:teuthology.orchestra.run.smithi059.stderr: --verbose -- /var/log/ceph/ceph-client.admin.48238.log 2024-05-08T08:56:24.053 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.38584.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38584.log.gz 2024-05-08T08:56:24.053 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49548.log 2024-05-08T08:56:24.053 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.48238.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43845.log 2024-05-08T08:56:24.054 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48238.log.gz 2024-05-08T08:56:24.054 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.49548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59626.log 2024-05-08T08:56:24.054 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49548.log.gz 2024-05-08T08:56:24.054 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56207.log 2024-05-08T08:56:24.054 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.43845.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43845.log.gz 2024-05-08T08:56:24.054 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59666.log 2024-05-08T08:56:24.054 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.59626.log: /var/log/ceph/ceph-client.admin.56207.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59626.log.gz 2024-05-08T08:56:24.054 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56207.log.gz 2024-05-08T08:56:24.054 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39734.log 2024-05-08T08:56:24.054 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53160.log 2024-05-08T08:56:24.054 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.59666.log: /var/log/ceph/ceph-client.admin.39734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45175.log 2024-05-08T08:56:24.054 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59666.log.gz 2024-05-08T08:56:24.054 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39734.log.gz 2024-05-08T08:56:24.054 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.53160.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50773.log 2024-05-08T08:56:24.054 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53160.log.gz 2024-05-08T08:56:24.054 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37924.log 2024-05-08T08:56:24.054 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.45175.log: /var/log/ceph/ceph-client.admin.50773.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39459.log 2024-05-08T08:56:24.055 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50773.log.gz 2024-05-08T08:56:24.055 INFO:teuthology.orchestra.run.smithi059.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45175.log.gz 2024-05-08T08:56:24.055 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.37924.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37924.log.gz 2024-05-08T08:56:24.055 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49412.log 2024-05-08T08:56:24.055 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50098.log 2024-05-08T08:56:24.055 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.39459.log: /var/log/ceph/ceph-client.admin.49412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41713.log 2024-05-08T08:56:24.055 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39459.log.gz 2024-05-08T08:56:24.055 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49412.log.gz 2024-05-08T08:56:24.055 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.50098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54990.log 2024-05-08T08:56:24.055 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50098.log.gz 2024-05-08T08:56:24.055 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31067.log 2024-05-08T08:56:24.055 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.41713.log: /var/log/ceph/ceph-client.admin.54990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58514.log 2024-05-08T08:56:24.055 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41713.log.gz 2024-05-08T08:56:24.055 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54990.log.gz 2024-05-08T08:56:24.055 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.31067.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50691.log 2024-05-08T08:56:24.055 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31067.log.gz 2024-05-08T08:56:24.055 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.58514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41292.log 2024-05-08T08:56:24.056 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58514.log.gz 2024-05-08T08:56:24.056 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.50691.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50691.log.gz 2024-05-08T08:56:24.056 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59392.log 2024-05-08T08:56:24.056 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.41292.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41292.log.gz 2024-05-08T08:56:24.056 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44255.log 2024-05-08T08:56:24.056 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.59392.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59392.log.gz 2024-05-08T08:56:24.056 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47488.log 2024-05-08T08:56:24.056 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.44255.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44255.log.gz 2024-05-08T08:56:24.056 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46191.log 2024-05-08T08:56:24.056 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.47488.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47488.log.gz 2024-05-08T08:56:24.056 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39479.log 2024-05-08T08:56:24.056 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.46191.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46191.log.gz 2024-05-08T08:56:24.056 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48493.log 2024-05-08T08:56:24.056 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.39479.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39479.log.gz 2024-05-08T08:56:24.056 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52175.log 2024-05-08T08:56:24.056 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51797.log 2024-05-08T08:56:24.057 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.48493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55932.log 2024-05-08T08:56:24.057 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0%/var/log/ceph/ceph-client.admin.52175.log: -- replaced with /var/log/ceph/ceph-client.admin.48493.log.gz 2024-05-08T08:56:24.057 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52175.log.gz 2024-05-08T08:56:24.057 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.51797.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51797.log.gz 2024-05-08T08:56:24.057 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50191.log 2024-05-08T08:56:24.057 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.55932.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55932.log.gz 2024-05-08T08:56:24.057 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51008.log 2024-05-08T08:56:24.057 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.50191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54181.log 2024-05-08T08:56:24.057 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50191.log.gz 2024-05-08T08:56:24.057 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.51008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39911.log 2024-05-08T08:56:24.057 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51008.log.gz 2024-05-08T08:56:24.057 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43433.log 2024-05-08T08:56:24.057 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.54181.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54181.log.gz 2024-05-08T08:56:24.057 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50545.log 2024-05-08T08:56:24.057 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.39911.log: /var/log/ceph/ceph-client.admin.43433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33769.log 2024-05-08T08:56:24.058 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43433.log.gz 2024-05-08T08:56:24.058 INFO:teuthology.orchestra.run.smithi059.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39911.log.gz 2024-05-08T08:56:24.058 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.50545.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50545.log.gz 2024-05-08T08:56:24.058 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60530.log 2024-05-08T08:56:24.058 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.33769.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33769.log.gz 2024-05-08T08:56:24.058 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45548.log 2024-05-08T08:56:24.058 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.60530.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60530.log.gz 2024-05-08T08:56:24.058 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60824.log 2024-05-08T08:56:24.058 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.45548.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45548.log.gz 2024-05-08T08:56:24.058 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58556.log 2024-05-08T08:56:24.058 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.60824.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60824.log.gz 2024-05-08T08:56:24.058 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43475.log 2024-05-08T08:56:24.058 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.58556.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58556.log.gz 2024-05-08T08:56:24.058 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55348.log 2024-05-08T08:56:24.058 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.43475.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43475.log.gz 2024-05-08T08:56:24.058 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58205.log 2024-05-08T08:56:24.059 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.55348.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55348.log.gz 2024-05-08T08:56:24.059 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41509.log 2024-05-08T08:56:24.059 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.58205.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58205.log.gz 2024-05-08T08:56:24.059 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53394.log 2024-05-08T08:56:24.059 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.41509.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41509.log.gz 2024-05-08T08:56:24.059 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47609.log 2024-05-08T08:56:24.059 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.53394.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53394.log.gz 2024-05-08T08:56:24.059 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42235.log 2024-05-08T08:56:24.059 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.47609.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47609.log.gz 2024-05-08T08:56:24.059 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34699.log 2024-05-08T08:56:24.059 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.42235.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42235.log.gz 2024-05-08T08:56:24.059 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59018.log 2024-05-08T08:56:24.059 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.34699.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34699.log.gz 2024-05-08T08:56:24.059 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38794.log 2024-05-08T08:56:24.059 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.59018.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59018.log.gz 2024-05-08T08:56:24.059 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38867.log 2024-05-08T08:56:24.060 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.38794.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38794.log.gz 2024-05-08T08:56:24.060 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60320.log 2024-05-08T08:56:24.060 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.38867.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38867.log.gz 2024-05-08T08:56:24.060 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59060.log 2024-05-08T08:56:24.060 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.60320.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60320.log.gz 2024-05-08T08:56:24.060 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60614.log 2024-05-08T08:56:24.060 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.59060.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59060.log.gz 2024-05-08T08:56:24.060 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50486.log 2024-05-08T08:56:24.060 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.60614.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60614.log.gz 2024-05-08T08:56:24.060 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47213.log 2024-05-08T08:56:24.060 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.50486.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50486.log.gz 2024-05-08T08:56:24.060 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43918.log 2024-05-08T08:56:24.060 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.47213.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47213.log.gz 2024-05-08T08:56:24.060 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60656.log 2024-05-08T08:56:24.060 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.43918.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43918.log.gz 2024-05-08T08:56:24.060 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44193.log 2024-05-08T08:56:24.060 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44743.log/var/log/ceph/ceph-client.admin.60656.log: 2024-05-08T08:56:24.061 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60656.log.gz 2024-05-08T08:56:24.061 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.44193.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40543.log 2024-05-08T08:56:24.061 INFO:teuthology.orchestra.run.smithi059.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44193.log.gz 2024-05-08T08:56:24.061 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.44743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35655.log 2024-05-08T08:56:24.061 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44743.log.gz 2024-05-08T08:56:24.061 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51674.log 2024-05-08T08:56:24.061 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.40543.log: /var/log/ceph/ceph-client.admin.35655.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48688.log 2024-05-08T08:56:24.061 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40543.log.gz 2024-05-08T08:56:24.061 INFO:teuthology.orchestra.run.smithi059.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35655.log.gz 2024-05-08T08:56:24.061 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.51674.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51674.log.gz 2024-05-08T08:56:24.061 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60950.log 2024-05-08T08:56:24.061 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.48688.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48688.log.gz 2024-05-08T08:56:24.061 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45448.log 2024-05-08T08:56:24.061 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.60950.log: gzip -5 --verbose 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60950.log.gz 2024-05-08T08:56:24.061 INFO:teuthology.orchestra.run.smithi059.stderr: -- /var/log/ceph/ceph-client.admin.55556.log 2024-05-08T08:56:24.061 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.45448.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45448.log.gz 2024-05-08T08:56:24.061 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49158.log 2024-05-08T08:56:24.062 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.55556.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55556.log.gz 2024-05-08T08:56:24.062 INFO:teuthology.orchestra.run.smithi059.stderr: --verbose -- /var/log/ceph/ceph-client.admin.52221.log 2024-05-08T08:56:24.062 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.49158.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49158.log.gz 2024-05-08T08:56:24.062 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60908.log 2024-05-08T08:56:24.062 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.52221.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52221.log.gz 2024-05-08T08:56:24.062 INFO:teuthology.orchestra.run.smithi059.stderr: --verbose -- /var/log/ceph/ceph-client.admin.49211.log 2024-05-08T08:56:24.062 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.60908.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60908.log.gz 2024-05-08T08:56:24.062 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33825.log 2024-05-08T08:56:24.062 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.49211.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49211.log.gz 2024-05-08T08:56:24.062 INFO:teuthology.orchestra.run.smithi059.stderr: --verbose -- /var/log/ceph/ceph-client.admin.35105.log 2024-05-08T08:56:24.062 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.33825.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33825.log.gz 2024-05-08T08:56:24.062 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60404.log 2024-05-08T08:56:24.062 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.35105.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35105.log.gz 2024-05-08T08:56:24.062 INFO:teuthology.orchestra.run.smithi059.stderr: --verbose -- /var/log/ceph/ceph-client.admin.59198.log 2024-05-08T08:56:24.062 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.60404.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60404.log.gz 2024-05-08T08:56:24.063 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56952.log 2024-05-08T08:56:24.063 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.59198.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59198.log.gz 2024-05-08T08:56:24.063 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.56952.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56952.log.gz 2024-05-08T08:56:24.063 INFO:teuthology.orchestra.run.smithi059.stderr: 91.1% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-05-08T08:56:24.185 INFO:teuthology.orchestra.run.smithi059.stderr: 90.4% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-05-08T08:56:26.930 INFO:teuthology.orchestra.run.smithi192.stderr: 95.8% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-05-08T08:56:30.858 INFO:teuthology.orchestra.run.smithi059.stderr: 95.9% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-05-08T08:56:32.575 INFO:teuthology.orchestra.run.smithi192.stderr: 95.7% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-05-08T08:56:33.502 INFO:teuthology.orchestra.run.smithi192.stderr: 95.5% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-05-08T08:56:33.504 INFO:teuthology.orchestra.run.smithi192.stderr: 2024-05-08T08:56:33.504 INFO:teuthology.orchestra.run.smithi192.stderr:real 0m9.740s 2024-05-08T08:56:33.504 INFO:teuthology.orchestra.run.smithi192.stderr:user 0m21.049s 2024-05-08T08:56:33.504 INFO:teuthology.orchestra.run.smithi192.stderr:sys 0m0.868s 2024-05-08T08:56:35.556 INFO:teuthology.orchestra.run.smithi059.stderr: 95.6% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-05-08T08:56:37.114 INFO:teuthology.orchestra.run.smithi059.stderr: 95.8% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-05-08T08:56:37.115 INFO:teuthology.orchestra.run.smithi059.stderr: 2024-05-08T08:56:37.115 INFO:teuthology.orchestra.run.smithi059.stderr:real 0m13.352s 2024-05-08T08:56:37.116 INFO:teuthology.orchestra.run.smithi059.stderr:user 0m31.613s 2024-05-08T08:56:37.116 INFO:teuthology.orchestra.run.smithi059.stderr:sys 0m1.261s 2024-05-08T08:56:37.116 INFO:tasks.ceph:Archiving logs... 2024-05-08T08:56:37.116 DEBUG:teuthology.misc:Transferring archived files from smithi059:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-05-07_21:16:03-rbd-squid-distro-default-smithi/7696006/remote/smithi059/log 2024-05-08T08:56:37.117 DEBUG:teuthology.orchestra.run.smithi059:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-08T08:56:40.985 DEBUG:teuthology.misc:Transferring archived files from smithi192:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-05-07_21:16:03-rbd-squid-distro-default-smithi/7696006/remote/smithi192/log 2024-05-08T08:56:40.986 DEBUG:teuthology.orchestra.run.smithi192:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-08T08:56:43.583 DEBUG:teuthology.misc:Transferring archived files from smithi196:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-05-07_21:16:03-rbd-squid-distro-default-smithi/7696006/remote/smithi196/log 2024-05-08T08:56:43.584 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-08T08:56:43.744 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-08T08:56:43.760 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-08T08:56:43.760 DEBUG:teuthology.orchestra.run.smithi059:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-08T08:56:43.763 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-08T08:56:43.766 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-08T08:56:43.866 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-08T08:56:43.876 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-08T08:56:43.876 DEBUG:teuthology.orchestra.run.smithi059:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T08:56:43.879 DEBUG:teuthology.orchestra.run.smithi192:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T08:56:43.881 DEBUG:teuthology.orchestra.run.smithi196:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T08:56:43.894 INFO:teuthology.orchestra.run.smithi059.stderr:bash: line 1: ntpq: command not found 2024-05-08T08:56:43.896 INFO:teuthology.orchestra.run.smithi192.stderr:bash: line 1: ntpq: command not found 2024-05-08T08:56:43.898 INFO:teuthology.orchestra.run.smithi196.stderr:bash: line 1: ntpq: command not found 2024-05-08T08:56:43.899 INFO:teuthology.orchestra.run.smithi059.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T08:56:43.900 INFO:teuthology.orchestra.run.smithi059.stdout:=============================================================================== 2024-05-08T08:56:43.900 INFO:teuthology.orchestra.run.smithi059.stdout:^* hv01.front.sepia.ceph.com 3 6 377 22 -1777us[-1803us] +/- 63ms 2024-05-08T08:56:43.900 INFO:teuthology.orchestra.run.smithi059.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 20 +1178us[+1178us] +/- 76ms 2024-05-08T08:56:43.900 INFO:teuthology.orchestra.run.smithi059.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 20 +777us[ +777us] +/- 76ms 2024-05-08T08:56:43.900 INFO:teuthology.orchestra.run.smithi059.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T08:56:43.901 INFO:teuthology.orchestra.run.smithi192.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T08:56:43.901 INFO:teuthology.orchestra.run.smithi192.stdout:=============================================================================== 2024-05-08T08:56:43.902 INFO:teuthology.orchestra.run.smithi192.stdout:^* hv01.front.sepia.ceph.com 3 6 377 20 -1769us[-1777us] +/- 63ms 2024-05-08T08:56:43.902 INFO:teuthology.orchestra.run.smithi192.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 20 +1182us[+1174us] +/- 76ms 2024-05-08T08:56:43.902 INFO:teuthology.orchestra.run.smithi192.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 22 +757us[ +750us] +/- 76ms 2024-05-08T08:56:43.902 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T08:56:43.903 INFO:teuthology.orchestra.run.smithi196.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T08:56:43.903 INFO:teuthology.orchestra.run.smithi196.stdout:=============================================================================== 2024-05-08T08:56:43.903 INFO:teuthology.orchestra.run.smithi196.stdout:^* hv01.front.sepia.ceph.com 3 6 377 22 -1749us[-1762us] +/- 63ms 2024-05-08T08:56:43.903 INFO:teuthology.orchestra.run.smithi196.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 21 +1205us[+1205us] +/- 76ms 2024-05-08T08:56:43.903 INFO:teuthology.orchestra.run.smithi196.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 21 +776us[ +776us] +/- 76ms 2024-05-08T08:56:43.903 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T08:56:43.904 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-08T08:56:43.913 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-08T08:56:43.913 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-08T08:56:43.933 DEBUG:teuthology.orchestra.run.smithi059:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-08T08:56:44.031 DEBUG:teuthology.orchestra.run.smithi192:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-08T08:56:44.109 DEBUG:teuthology.orchestra.run.smithi196:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-08T08:56:44.175 DEBUG:teuthology.orchestra.run.smithi059:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-08T08:56:44.209 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T08:56:44.209 DEBUG:teuthology.orchestra.run.smithi192:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-08T08:56:44.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T08:56:44.240 DEBUG:teuthology.orchestra.run.smithi196:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-08T08:56:44.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T08:56:44.270 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-08T08:56:44.283 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-08T08:56:44.297 INFO:teuthology.task.internal:Duration was 725.136592 seconds 2024-05-08T08:56:44.298 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-08T08:56:44.314 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-08T08:56:44.315 DEBUG:teuthology.orchestra.run.smithi059:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-08T08:56:44.318 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-08T08:56:44.321 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-08T08:56:44.481 INFO:teuthology.orchestra.run.smithi059.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T08:56:44.481 INFO:teuthology.orchestra.run.smithi192.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T08:56:44.526 INFO:teuthology.orchestra.run.smithi196.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T08:56:44.889 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-08T08:56:44.889 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi059.front.sepia.ceph.com 2024-05-08T08:56:44.889 DEBUG:teuthology.orchestra.run.smithi059:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-08T08:56:44.915 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi192.front.sepia.ceph.com 2024-05-08T08:56:44.915 DEBUG:teuthology.orchestra.run.smithi192:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-08T08:56:44.941 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi196.front.sepia.ceph.com 2024-05-08T08:56:44.941 DEBUG:teuthology.orchestra.run.smithi196:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-08T08:56:44.967 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-08T08:56:44.968 DEBUG:teuthology.orchestra.run.smithi059:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T08:56:44.970 DEBUG:teuthology.orchestra.run.smithi192:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T08:56:44.983 DEBUG:teuthology.orchestra.run.smithi196:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T08:56:45.037 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-08T08:56:45.037 DEBUG:teuthology.orchestra.run.smithi059:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-08T08:56:45.043 DEBUG:teuthology.orchestra.run.smithi192:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-08T08:56:45.052 DEBUG:teuthology.orchestra.run.smithi196:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-08T08:56:45.182 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-08T08:56:45.193 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-08T08:56:45.193 DEBUG:teuthology.orchestra.run.smithi059:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-08T08:56:45.220 DEBUG:teuthology.orchestra.run.smithi192:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-08T08:56:45.244 DEBUG:teuthology.orchestra.run.smithi196:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-08T08:56:45.270 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-08T08:56:45.281 DEBUG:teuthology.orchestra.run.smithi059:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-08T08:56:45.283 DEBUG:teuthology.orchestra.run.smithi192:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-08T08:56:45.287 DEBUG:teuthology.orchestra.run.smithi196:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-08T08:56:45.305 INFO:teuthology.orchestra.run.smithi059.stdout:kernel.core_pattern = core 2024-05-08T08:56:45.308 INFO:teuthology.orchestra.run.smithi192.stdout:kernel.core_pattern = core 2024-05-08T08:56:45.333 INFO:teuthology.orchestra.run.smithi196.stdout:kernel.core_pattern = core 2024-05-08T08:56:45.354 DEBUG:teuthology.orchestra.run.smithi059:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-08T08:56:45.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T08:56:45.380 DEBUG:teuthology.orchestra.run.smithi192:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-08T08:56:45.395 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T08:56:45.395 DEBUG:teuthology.orchestra.run.smithi196:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-08T08:56:45.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T08:56:45.410 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-08T08:56:45.421 INFO:teuthology.task.internal:Transferring archived files... 2024-05-08T08:56:45.421 DEBUG:teuthology.misc:Transferring archived files from smithi059:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-05-07_21:16:03-rbd-squid-distro-default-smithi/7696006/remote/smithi059 2024-05-08T08:56:45.422 DEBUG:teuthology.orchestra.run.smithi059:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-08T08:56:45.463 DEBUG:teuthology.misc:Transferring archived files from smithi192:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-05-07_21:16:03-rbd-squid-distro-default-smithi/7696006/remote/smithi192 2024-05-08T08:56:45.463 DEBUG:teuthology.orchestra.run.smithi192:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-08T08:56:45.497 DEBUG:teuthology.misc:Transferring archived files from smithi196:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-05-07_21:16:03-rbd-squid-distro-default-smithi/7696006/remote/smithi196 2024-05-08T08:56:45.497 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-08T08:56:45.534 INFO:teuthology.task.internal:Removing archive directory... 2024-05-08T08:56:45.535 DEBUG:teuthology.orchestra.run.smithi059:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-08T08:56:45.537 DEBUG:teuthology.orchestra.run.smithi192:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-08T08:56:45.539 DEBUG:teuthology.orchestra.run.smithi196:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-08T08:56:45.582 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-08T08:56:45.594 INFO:teuthology.task.internal:Not uploading archives. 2024-05-08T08:56:45.594 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-08T08:56:45.604 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-08T08:56:45.604 DEBUG:teuthology.orchestra.run.smithi059:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-08T08:56:45.607 DEBUG:teuthology.orchestra.run.smithi192:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-08T08:56:45.609 DEBUG:teuthology.orchestra.run.smithi196:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-08T08:56:45.620 INFO:teuthology.orchestra.run.smithi059.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 8 08:56 /home/ubuntu/cephtest 2024-05-08T08:56:45.621 INFO:teuthology.orchestra.run.smithi192.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 8 08:56 /home/ubuntu/cephtest 2024-05-08T08:56:45.637 INFO:teuthology.orchestra.run.smithi196.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 8 08:56 /home/ubuntu/cephtest 2024-05-08T08:56:45.638 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-08T08:56:45.648 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-08T08:56:45.675 INFO:teuthology.run:Summary data: description: rbd/device/{base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-zlib supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_nbd} duration: 725.1365923881531 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-05-08T08:56:45.675 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-08T08:56:45.762 INFO:teuthology.run:pass